-- Logs begin at Sat 2022-10-01 20:01:15 UTC, end at Sat 2022-10-01 22:24:45 UTC. -- Oct 01 20:31:22.439373 ncdv-master-1 systemd[1]: Starting Devstack devstack@keystone.service... Oct 01 20:31:22.448191 ncdv-master-1 devstack@keystone.service[99128]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 01 20:31:22.450261 ncdv-master-1 devstack@keystone.service[99128]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Oct 1 20:31:22 2022] *** Oct 01 20:31:22.450261 ncdv-master-1 devstack@keystone.service[99128]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 01 20:31:22.450261 ncdv-master-1 devstack@keystone.service[99128]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 01 20:31:22.450261 ncdv-master-1 devstack@keystone.service[99128]: nodename: ncdv-master-1 Oct 01 20:31:22.450261 ncdv-master-1 devstack@keystone.service[99128]: machine: x86_64 Oct 01 20:31:22.450261 ncdv-master-1 devstack@keystone.service[99128]: clock source: unix Oct 01 20:31:22.450261 ncdv-master-1 devstack@keystone.service[99128]: pcre jit disabled Oct 01 20:31:22.450261 ncdv-master-1 devstack@keystone.service[99128]: detected number of CPU cores: 4 Oct 01 20:31:22.450261 ncdv-master-1 devstack@keystone.service[99128]: current working directory: / Oct 01 20:31:22.450261 ncdv-master-1 devstack@keystone.service[99128]: detected binary path: /usr/bin/uwsgi-core Oct 01 20:31:22.450811 ncdv-master-1 devstack@keystone.service[99128]: your processes number limit is 47680 Oct 01 20:31:22.450811 ncdv-master-1 devstack@keystone.service[99128]: your memory page size is 4096 bytes Oct 01 20:31:22.450811 ncdv-master-1 devstack@keystone.service[99128]: detected max file descriptor number: 2048 Oct 01 20:31:22.450811 ncdv-master-1 devstack@keystone.service[99128]: lock engine: pthread robust mutexes Oct 01 20:31:22.450811 ncdv-master-1 devstack@keystone.service[99128]: thunder lock: enabled Oct 01 20:31:22.450811 ncdv-master-1 devstack@keystone.service[99128]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 01 20:31:22.450811 ncdv-master-1 devstack@keystone.service[99128]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 01 20:31:22.494820 ncdv-master-1 devstack@keystone.service[99128]: Python main interpreter initialized at 0x5639c59e19c0 Oct 01 20:31:22.494820 ncdv-master-1 devstack@keystone.service[99128]: python threads support enabled Oct 01 20:31:22.494820 ncdv-master-1 devstack@keystone.service[99128]: your server socket listen backlog is limited to 100 connections Oct 01 20:31:22.494820 ncdv-master-1 devstack@keystone.service[99128]: your mercy for graceful operations on workers is 90 seconds Oct 01 20:31:22.495167 ncdv-master-1 devstack@keystone.service[99128]: mapped 403077 bytes (393 KB) for 2 cores Oct 01 20:31:22.495244 ncdv-master-1 devstack@keystone.service[99128]: *** Operational MODE: preforking *** Oct 01 20:31:22.495475 ncdv-master-1 devstack@keystone.service[99128]: *** uWSGI is running in multiple interpreter mode *** Oct 01 20:31:22.495475 ncdv-master-1 devstack@keystone.service[99128]: spawned uWSGI master process (pid: 99128) Oct 01 20:31:22.495675 ncdv-master-1 systemd[1]: Started Devstack devstack@keystone.service. Oct 01 20:31:22.496725 ncdv-master-1 devstack@keystone.service[99128]: spawned uWSGI worker 1 (pid: 99139, cores: 1) Oct 01 20:31:22.496725 ncdv-master-1 devstack@keystone.service[99128]: spawned uWSGI worker 2 (pid: 99140, cores: 1) Oct 01 20:31:22.496725 ncdv-master-1 devstack@keystone.service[99128]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 01 20:31:27.087810 ncdv-master-1 devstack@keystone.service[99140]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 01 20:31:27.087810 ncdv-master-1 devstack@keystone.service[99140]: from cryptography.utils import int_from_bytes Oct 01 20:31:27.088482 ncdv-master-1 devstack@keystone.service[99139]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 01 20:31:27.088482 ncdv-master-1 devstack@keystone.service[99139]: from cryptography.utils import int_from_bytes Oct 01 20:31:27.088919 ncdv-master-1 devstack@keystone.service[99140]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 01 20:31:27.088919 ncdv-master-1 devstack@keystone.service[99140]: from cryptography.utils import int_from_bytes Oct 01 20:31:27.089058 ncdv-master-1 devstack@keystone.service[99139]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 01 20:31:27.089058 ncdv-master-1 devstack@keystone.service[99139]: from cryptography.utils import int_from_bytes Oct 01 20:31:27.137830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 01 20:31:27.137996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 01 20:31:27.138082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 01 20:31:27.138176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 01 20:31:27.138256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] command line args: [] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 01 20:31:27.138334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] command line args: [] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 01 20:31:27.138446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 01 20:31:27.138634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 01 20:31:27.138724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ================================================================================ {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 01 20:31:27.138809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ================================================================================ {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 01 20:31:27.138882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] admin_token = **** {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.139060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] admin_token = **** {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.139148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] config_dir = [] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.139288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.139367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] config_dir = [] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.139473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] config_source = [] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.139554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.139658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] config_source = [] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.139744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] debug = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.139841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] debug = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.139929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.140160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.140288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.140387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.140495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] insecure_debug = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.140598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] insecure_debug = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.140696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.140797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.140903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.141006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.141111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] list_limit = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.141215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] list_limit = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.141325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] log_config_append = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.141428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] log_config_append = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.141525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.141620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.141727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] log_dir = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.141884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] log_dir = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.141984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] log_file = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.142102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] log_file = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.142177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.142299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.142387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.142465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.142572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.142652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.142770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.142855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.142937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.143062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.143144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.143216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.143288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.143369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.143442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.143520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.143644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.143795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.143865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.143950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.144096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.144192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.144297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.144423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.144542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.144638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.144788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] notification_format = cadf {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.144917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] notification_format = cadf {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.145062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.145185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.146190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.146286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.146381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] publish_errors = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.146471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] publish_errors = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.146647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.146737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.146881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.146958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.147082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.147164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.147298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.147397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.147534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.147619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.147768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] standard_threads = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.147846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] standard_threads = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.147999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] strict_password_check = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.148082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] strict_password_check = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.148243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.148375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.148456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] use_eventlog = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.148532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] use_eventlog = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.148609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] use_journal = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.148685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] use_journal = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.148757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] use_json = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.148887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] use_json = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.148969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] use_stderr = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.149042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] use_stderr = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.149112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] use_syslog = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.149182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] use_syslog = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.149300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] watch_log_file = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.149385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] watch_log_file = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:31:27.149532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.149612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.149772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.149851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.150036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.150122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.150276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.150354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.150548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.150640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.150791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.150878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.151039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.151117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.151277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.151355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.151525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.151611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.151773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.151924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.152043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.152118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.152242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.152331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.152487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.152570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.152722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.152805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.152969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.153069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.153214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.153298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.153451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.153532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.153700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.153848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.153955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.154039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.154196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.154273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.154433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.154538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.154709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.154787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.154972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.155049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.155225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.155309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.155475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.155553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.155703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.155780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.155934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.156012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.156161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.156272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.156397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.156503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.156644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.156751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.156891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.156996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.157121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.157201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.157351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.157433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.157593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.157674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.157840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.157920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.158076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.158157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.158319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.158396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.158562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.158641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.158796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.158880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.159033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.159110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.159263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.159341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.159482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.159556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.159712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.159797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.159946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.160023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.160182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.160261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.160427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.160506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.160657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.160734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.160891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.160969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.161694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.161778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.161957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.162043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.162169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.162248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.162417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.162525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.162671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.162746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.162907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.162987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.163141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.163219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.163378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.163480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.163609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.163689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.163844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.163921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.164070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.164156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.164319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.164397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.164552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.164635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.164789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.164875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.165019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.165109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.165274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.165360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.165558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.165650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.165725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.165796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.165957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.166039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.166191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.166269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.166453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.166596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.166714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.166794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.166963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.167043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.167203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.167289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.167441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.167517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.167673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.167751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.167909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.167998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.168155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.enabled = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.168233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.enabled = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.168389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.168473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.168624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.168709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.168861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.168938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.169097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.169176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.169344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.169430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.169578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.169656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.169809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.169887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.170042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.170127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.170272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.170350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.170521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.170604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.170782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.170872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.171022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.171100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.171267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.171346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.171508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.171659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.171735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.171805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.171968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.172091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.172205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.172292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.172434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.172524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.172670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.172748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.172896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.172974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.173126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.173210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.173353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.173431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.173587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.173665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.173814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.173901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.174054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.174133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.174288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.174366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.174551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.174643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.174790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.174870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.175043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.175130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.175300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.175379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.175537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.175644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.175770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] auth.external = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.175850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] auth.external = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.175997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] auth.mapped = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.176074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] auth.mapped = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.176292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.176380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.176523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.176602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.176776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] auth.password = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.176855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] auth.password = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.177007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] auth.token = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.177087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] auth.token = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.177242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.177320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.177477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] catalog.caching = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.177555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] catalog.caching = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.177720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.177800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.177950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.178028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.178192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.178278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.178436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.178536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.178690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.178779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.178926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] credential.caching = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.179005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] credential.caching = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.179169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] credential.driver = sql {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.179249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] credential.driver = sql {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.179414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.179499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.179660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.179739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.179913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.179993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.180156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.180235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.180386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.180472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.180632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.180712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.180879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.180957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.181114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.181201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.181368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.181447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.181643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.181728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.181889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.181975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.182148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.182234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.182389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.182467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.182663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.182743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.182899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] federation.caching = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.182980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] federation.caching = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.183132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.183211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.183378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] federation.driver = sql {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.183456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] federation.driver = sql {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.183620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.183700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.183850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.183929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.184095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.184174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.184328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.184413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.184580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.184658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.184827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.184971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.185101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.185180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.185286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.185367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.185526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.185678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.185762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] identity.caching = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.185835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] identity.caching = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.185995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.186075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.186240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.186328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.186471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.186576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.186727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.186810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.186971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] identity.driver = sql {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.187059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] identity.driver = sql {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.187199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.187279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.187435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.187517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.187678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.187765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.187928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.188007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.188155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.188234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.188384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.188470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.188610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.188689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.188853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.188933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.189100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.189189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.189346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.189434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.189600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.189684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.189847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.189939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.190114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.190193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.190352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.190431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.190610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.190705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.190843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.190922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.191075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.191153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.191305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.191394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.191540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.191621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.191780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.191859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.192016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.192103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.192286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.192369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.192517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.192597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.192764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.192853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.193010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.193090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.193245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.193325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.193492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.193583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.193738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.193816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.193965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.194044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.194202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.194298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.194428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.password = **** {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.194530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.password = **** {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.194688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.194772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.194921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.195012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.195157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.195242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.195392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.195472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.195623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.195710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.195871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.195951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.196117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.196197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.196610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.196704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.196840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.196919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.197089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.197169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.197335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.197426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.197571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.197650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.197807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.197886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.198040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.198120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.198271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.user = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.198356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.user = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.198563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.198650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.198788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.198871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.199019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.199099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.199265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.199345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.199511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.199591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.199754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.199832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.199986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.200065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.200212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.200292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.200445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.200524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.200674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.200754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.200909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.200989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.201135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.201213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.201381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.201463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.201623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.201703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.201866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.201945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.202109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.202188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.202353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.202437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.202601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.202682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.202848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.202927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.203089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.203328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.203407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.203566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.203646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.203798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.203885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.204038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.204118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.204282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.204363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.204518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.204605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.204752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.204831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] policy.driver = sql {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.204996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.205074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] policy.driver = sql {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.205223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.205310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.205468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.205546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.205699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.205778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] receipt.caching = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.205934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] receipt.caching = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.206022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.206171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.206251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.206410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.206509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.206671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.206762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.206903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.206983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.207139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.207219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] resource.caching = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.207379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] resource.caching = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.207467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.207627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] resource.driver = sql {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.207706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.208383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.208470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] resource.driver = sql {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.208618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.208704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.208869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.208950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.209104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] revoke.caching = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.209188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.209341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] revoke.caching = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.209491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.209568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.209639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.209809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] role.cache_time = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.209890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.210045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] role.cache_time = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.210131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] role.caching = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.210281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] role.caching = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.210361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] role.driver = sql {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.210530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] role.list_limit = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.210614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] role.driver = sql {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.210777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] role.list_limit = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.210868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.211024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.211104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.211272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.211350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.211518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.211598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.211766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.211845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.212008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.212087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.212254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.212373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.212496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.212575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.212727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.212805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.212961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.213041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.213207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.213287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.213453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.213532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.213696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.213776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.213943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.214089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.214331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.214409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.214588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.214672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.214844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.214924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.215095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.215176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.215344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.215425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.215582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.215663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.215815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.215894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.216068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.216150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.216311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.216444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.216526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.216613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.216776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.216925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.217004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.217076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.217214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.217304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.217463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.217544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.217708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.217787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.217948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.218036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.218180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.218259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.218409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] token.cache_time = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.218533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.218661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] token.cache_time = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.218753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] token.caching = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.218898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] token.caching = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.218979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.219135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.219215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] token.provider = fernet {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.219370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.219457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] token.provider = fernet {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.219604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.219682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.219859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.219940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.220100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.220187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.220339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.220418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.220581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.220660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.220822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.220914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] trust.driver = sql {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.221058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.221137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] trust.driver = sql {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.221292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.221371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.221527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.221613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.221759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.221838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.222006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.222086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.222261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.222351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.222483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.222609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.222750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.222833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.223015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.223096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] database.connection = **** {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.223265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.223344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] database.connection = **** {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.223546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.224125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.224298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.224378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.224534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.224617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.224774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.224856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.225010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.225089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.225242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.225320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.225476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.225558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.225710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.225790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.225943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.226022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.226175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.226256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.226412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.226510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.226668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.226748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.226917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.227000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.227153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.227233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.227383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.227462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.227612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.227692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.227849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.227928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.228001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=99139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 01 20:31:27.228071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:31:27.228239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=99140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 01 20:31:27.256950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.257137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.257227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.257374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.257537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.257618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.265747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.266095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.295468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.295651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.295744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.295890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.347261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.347721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.396020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.396251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.396339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.396487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.462146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.462297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.462387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.462468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.462621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.462708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.462847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.462946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.463059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.463143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.463273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.463356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.486671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.486856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.486943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.487087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.487172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.487312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.487411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.487536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.487617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.487752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.487833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.487967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.488051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.488181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.498937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.499294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.511974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.512219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.512451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.512566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.512676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.512800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.512886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.513024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.513108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.513247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.513329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=99139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.513472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.513687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.513902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=99140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 01 20:31:27.529875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.530087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 01 20:31:27.530249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.530431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.530534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 01 20:31:27.530660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 01 20:31:27.530813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.530946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.531081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 01 20:31:27.531214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.531301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.531480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.531562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.531741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.531823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.532013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.541843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.541940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.542125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.542223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.542401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.542484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.542720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.542804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.549651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=99140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.549745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=99139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.552556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=99140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.552643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=99139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.552844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.552940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.553127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.553210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.553409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.553492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.553690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.553781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.553952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.554034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.554220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.554301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.554521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.554608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.554788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.554869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.555059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.555151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.555326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.555407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.580042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.580170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.581495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=99140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.581630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=99139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.581782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.581920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.586998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=99140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.587159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=99139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.587287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.587447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.587572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.587731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.594047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=99140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.594151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=99139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.594341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.594425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.598408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=99140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.598573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=99139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.598762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.598857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.599060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.599143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.599347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.599431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.599635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.599719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.599927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.600010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.600204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.600295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.600493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.600575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.616003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.616247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.616332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.616529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.620137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=99139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.620369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=99140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.623366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.623598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.623693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.623897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.627861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=99139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.628083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=99140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.631561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=99139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.631744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=99140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.634850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.634984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.635156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.635285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.635448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.635575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.635733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.635872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.636022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.636155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.636311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.636442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.652519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=99139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.652613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=99140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.652824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.652910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.653100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.653188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.653369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.653450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.660186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.660309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.661737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.661819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.663304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.663390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.663592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.663676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.666194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=99140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.666286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=99139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.666504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.666601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.666798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.666880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.667080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.667171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.667464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.667601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.678541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=99140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.678630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=99139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.678829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.678913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.679111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.679195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.679393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.679485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.679775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.679858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.680060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.680141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.680338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.680428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.696052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=99140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.696152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=99139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.699223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=99140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.699327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=99139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.702117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.702244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.703575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.703698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.705031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=99140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.705191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=99139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.705321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.705492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.705714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.705898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.712823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.712929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.714294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=99140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.714378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=99139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.717146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.717231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.718373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.718464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.718688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.718772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.718970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.719055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.726729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=99140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.726823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=99139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.727020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.727104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.727301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.727397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.734137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=99140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.734228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=99139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 01 20:31:27.734422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.734524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.734730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.734814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.735011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.735094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.735312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.735393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.735584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.735667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.735869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.735959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.736144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.736228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.736421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.736501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.736688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.736776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.736954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.737035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.737223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.737303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.737487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.737569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.737752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=99140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.737831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=99139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 01 20:31:27.771908 ncdv-master-1 devstack@keystone.service[99140]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5639c59e19c0 pid: 99140 (default app) Oct 01 20:31:27.772165 ncdv-master-1 devstack@keystone.service[99139]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5639c59e19c0 pid: 99139 (default app) Oct 01 20:31:27.775162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a97de-8d06-4a41-9ad4-f520072203ae None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:27.775458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a97de-8d06-4a41-9ad4-f520072203ae None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:27.775730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a97de-8d06-4a41-9ad4-f520072203ae None None] PATH_INFO: `/v3` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:27.779828 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Sat Oct 1 20:31:27 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:31:34.934737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14c193-ae95-4458-9861-677bca894ca2 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:34.935303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14c193-ae95-4458-9861-677bca894ca2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:34.935303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14c193-ae95-4458-9861-677bca894ca2 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:34.939520 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:31:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:31:34.945238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecca60d-c233-4a1e-bbc6-9406e6927b94 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:34.945792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecca60d-c233-4a1e-bbc6-9406e6927b94 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:34.945792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecca60d-c233-4a1e-bbc6-9406e6927b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:35.022582 ncdv-master-1 devstack@keystone.service[99140]: INFO dbcounter [None req-cecca60d-c233-4a1e-bbc6-9406e6927b94 None None] Registered counter for database keystone Oct 01 20:31:35.041486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG oslo_db.sqlalchemy.engines [None req-cecca60d-c233-4a1e-bbc6-9406e6927b94 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=99140) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 01 20:31:35.047875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writer thread running {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 01 20:31:35.226034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG passlib.handlers.bcrypt [None req-cecca60d-c233-4a1e-bbc6-9406e6927b94 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=99140) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 01 20:31:35.226428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG passlib.handlers.bcrypt [None req-cecca60d-c233-4a1e-bbc6-9406e6927b94 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=99140) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 01 20:31:35.284326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-cecca60d-c233-4a1e-bbc6-9406e6927b94 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:31:35.286555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cecca60d-c233-4a1e-bbc6-9406e6927b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:31:35.413930 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:31:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 01 20:31:35.943082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14c193-ae95-4458-9861-677bca894ca2 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:36.019436 ncdv-master-1 devstack@keystone.service[99139]: INFO dbcounter [None req-3d14c193-ae95-4458-9861-677bca894ca2 None None] Registered counter for database keystone Oct 01 20:31:36.040720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG oslo_db.sqlalchemy.engines [None req-3d14c193-ae95-4458-9861-677bca894ca2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=99139) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 01 20:31:36.047104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writer thread running {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 01 20:31:36.163660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14c193-ae95-4458-9861-677bca894ca2 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:36.167984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff10357-be1d-4d8e-94ea-2c9082eeea29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:36.168608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff10357-be1d-4d8e-94ea-2c9082eeea29 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:36.168888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff10357-be1d-4d8e-94ea-2c9082eeea29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:36.169159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff10357-be1d-4d8e-94ea-2c9082eeea29 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:36.177802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff10357-be1d-4d8e-94ea-2c9082eeea29 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:36.229365 ncdv-master-1 devstack@keystone.service[99139]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fff10357-be1d-4d8e-94ea-2c9082eeea29 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 01 20:31:36.229707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff10357-be1d-4d8e-94ea-2c9082eeea29 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:36.241053 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Sat Oct 1 20:31:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 20:31:36.247230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecca60d-c233-4a1e-bbc6-9406e6927b94 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:36.264241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecca60d-c233-4a1e-bbc6-9406e6927b94 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:36.268657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf52e2-6e36-4e8d-93cc-fa3ac756fc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:36.269750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf52e2-6e36-4e8d-93cc-fa3ac756fc72 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:36.270027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf52e2-6e36-4e8d-93cc-fa3ac756fc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:36.270343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf52e2-6e36-4e8d-93cc-fa3ac756fc72 admin admin] PATH_INFO: `/v3/projects/549fa0b49ffd4207961a9dc3298560fa` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:36.271782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf52e2-6e36-4e8d-93cc-fa3ac756fc72 admin admin] RBAC: Authorizing `identity:get_project(project_id=549fa0b49ffd4207961a9dc3298560fa)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:36.322591 ncdv-master-1 devstack@keystone.service[99140]: WARNING keystone.common.rbac_enforcer.enforcer [None req-22cf52e2-6e36-4e8d-93cc-fa3ac756fc72 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 01 20:31:36.322930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf52e2-6e36-4e8d-93cc-fa3ac756fc72 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:36.327433 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Sat Oct 1 20:31:36 2022] GET /identity/v3/projects/549fa0b49ffd4207961a9dc3298560fa => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:31:38.668969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa2095-fc31-470a-b8d7-1660e112c107 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:38.669513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa2095-fc31-470a-b8d7-1660e112c107 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:38.669513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa2095-fc31-470a-b8d7-1660e112c107 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:38.673576 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:31:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:31:38.679793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9af9e9-87ae-451f-820b-89b2496e94a8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:38.680210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9af9e9-87ae-451f-820b-89b2496e94a8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:38.680343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9af9e9-87ae-451f-820b-89b2496e94a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:38.700441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-df9af9e9-87ae-451f-820b-89b2496e94a8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:31:38.701952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-df9af9e9-87ae-451f-820b-89b2496e94a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:31:38.709818 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:31:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 01 20:31:39.034083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa2095-fc31-470a-b8d7-1660e112c107 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:39.046404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa2095-fc31-470a-b8d7-1660e112c107 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:39.050612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de3b40-3131-4a1c-93f2-00e7c4bde5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:39.051205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de3b40-3131-4a1c-93f2-00e7c4bde5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:39.051977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de3b40-3131-4a1c-93f2-00e7c4bde5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:39.052249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de3b40-3131-4a1c-93f2-00e7c4bde5a7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:39.059827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de3b40-3131-4a1c-93f2-00e7c4bde5a7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:39.061577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de3b40-3131-4a1c-93f2-00e7c4bde5a7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:39.070864 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Sat Oct 1 20:31:39 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 20:31:39.076552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9af9e9-87ae-451f-820b-89b2496e94a8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:39.085633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9af9e9-87ae-451f-820b-89b2496e94a8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:39.090004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5218c-e5f1-4835-b66c-ec653707869e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:39.090669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5218c-e5f1-4835-b66c-ec653707869e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:39.090956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5218c-e5f1-4835-b66c-ec653707869e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:39.091234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5218c-e5f1-4835-b66c-ec653707869e admin admin] PATH_INFO: `/v3/users/a9033d8c4ef14e3eaead57cd2b91f40f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:39.092532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5218c-e5f1-4835-b66c-ec653707869e admin admin] RBAC: Authorizing `identity:get_user(user_id=a9033d8c4ef14e3eaead57cd2b91f40f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:39.094340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5218c-e5f1-4835-b66c-ec653707869e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:39.095899 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Sat Oct 1 20:31:39 2022] GET /identity/v3/users/a9033d8c4ef14e3eaead57cd2b91f40f => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:31:41.553185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde30b6-fef9-49fe-91cc-b584e60dabd1 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:41.553703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde30b6-fef9-49fe-91cc-b584e60dabd1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:41.553703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde30b6-fef9-49fe-91cc-b584e60dabd1 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:41.555065 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:31:41.560227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04638a-50e6-4310-9463-b9a9ce151027 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:41.560653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04638a-50e6-4310-9463-b9a9ce151027 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:41.560781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04638a-50e6-4310-9463-b9a9ce151027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:41.580633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-cb04638a-50e6-4310-9463-b9a9ce151027 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:31:41.581815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cb04638a-50e6-4310-9463-b9a9ce151027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:31:41.582636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ae1f8-3cbe-43d1-9583-a914731ea578 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:41.582924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ae1f8-3cbe-43d1-9583-a914731ea578 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:41.583195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ae1f8-3cbe-43d1-9583-a914731ea578 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:41.584651 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:31:41.589538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947aa50-425d-4b60-9538-f9d918029f7d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:41.589813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947aa50-425d-4b60-9538-f9d918029f7d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:41.590091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947aa50-425d-4b60-9538-f9d918029f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:41.634085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG passlib.handlers.bcrypt [None req-b947aa50-425d-4b60-9538-f9d918029f7d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=99139) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 01 20:31:41.634474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG passlib.handlers.bcrypt [None req-b947aa50-425d-4b60-9538-f9d918029f7d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=99139) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 01 20:31:41.684946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b947aa50-425d-4b60-9538-f9d918029f7d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:31:41.686573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b947aa50-425d-4b60-9538-f9d918029f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:31:41.773997 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:31:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:31:41.791326 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:31:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:31:42.102269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04638a-50e6-4310-9463-b9a9ce151027 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:42.111795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947aa50-425d-4b60-9538-f9d918029f7d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:42.126455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:42.211867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04638a-50e6-4310-9463-b9a9ce151027 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:42.217160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947aa50-425d-4b60-9538-f9d918029f7d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:42.219833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:42.247000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243eed4-2ccd-47ef-8584-4a0243dbc01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:42.247723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243eed4-2ccd-47ef-8584-4a0243dbc01b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:42.248005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243eed4-2ccd-47ef-8584-4a0243dbc01b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:42.248279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243eed4-2ccd-47ef-8584-4a0243dbc01b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:42.257336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d708db-d9dc-4c3d-ad95-2ebb5da838e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:42.258192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d708db-d9dc-4c3d-ad95-2ebb5da838e6 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:42.258477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d708db-d9dc-4c3d-ad95-2ebb5da838e6 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:42.258787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d708db-d9dc-4c3d-ad95-2ebb5da838e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:42.259701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243eed4-2ccd-47ef-8584-4a0243dbc01b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:31:42.268404 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-8243eed4-2ccd-47ef-8584-4a0243dbc01b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:31:42.268404 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:31:42.268404 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:31:42.268404 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:31:42.268404 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:31:42.268404 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:31:42.268404 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:31:42.268404 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:31:42.268404 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:31:42.268404 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:31:42.268404 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:31:42.268404 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:31:42.268404 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:31:42.268404 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:31:42.268404 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:31:42.268404 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:31:42.269461 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:31:42.269461 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:31:42.269461 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:31:42.269461 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:31:42.269461 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:31:42.269461 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:31:42.269461 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:31:42.269461 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:31:42.269461 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:31:42.269461 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:31:42.269461 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:31:42.269461 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:31:42.269461 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:31:42.269461 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:31:42.269461 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:31:42.269461 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:31:42.270710 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:31:42.270710 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:31:42.270710 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:31:42.270710 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:31:42.270710 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:31:42.270710 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:31:42.270710 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:31:42.270710 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:31:42.270710 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:31:42.270710 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:31:42.270710 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:31:42.270710 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:31:42.270710 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:31:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:31:42.275242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243eed4-2ccd-47ef-8584-4a0243dbc01b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:42.326743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d708db-d9dc-4c3d-ad95-2ebb5da838e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:42.328177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d708db-d9dc-4c3d-ad95-2ebb5da838e6 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:42.355402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:42.375041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243eed4-2ccd-47ef-8584-4a0243dbc01b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:42.407444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a6b19-02d9-45d3-a5cc-0df207265f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:42.408037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a6b19-02d9-45d3-a5cc-0df207265f6d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:42.408316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a6b19-02d9-45d3-a5cc-0df207265f6d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:42.408590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a6b19-02d9-45d3-a5cc-0df207265f6d None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:42.409248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a6b19-02d9-45d3-a5cc-0df207265f6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:31:42.409549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a6b19-02d9-45d3-a5cc-0df207265f6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:42.411447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a6b19-02d9-45d3-a5cc-0df207265f6d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:42.423088 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 8/15] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:31:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:31:42.428219 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 8/16] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:31:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 01 20:31:42.431565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a6b19-02d9-45d3-a5cc-0df207265f6d None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:42.433633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d708db-d9dc-4c3d-ad95-2ebb5da838e6 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:42.436432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:42.531199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a6b19-02d9-45d3-a5cc-0df207265f6d None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:42.532518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d708db-d9dc-4c3d-ad95-2ebb5da838e6 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:42.564371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a23d939-aaaf-49ab-8787-b2037339462e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:42.565001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23d939-aaaf-49ab-8787-b2037339462e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:42.565256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776e8f8-aaea-498e-be84-0d3804d88f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:42.565617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23d939-aaaf-49ab-8787-b2037339462e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:42.565617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23d939-aaaf-49ab-8787-b2037339462e None admin] PATH_INFO: `/v3/users/a9033d8c4ef14e3eaead57cd2b91f40f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:42.566099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776e8f8-aaea-498e-be84-0d3804d88f48 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:42.566406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776e8f8-aaea-498e-be84-0d3804d88f48 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:42.566699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a23d939-aaaf-49ab-8787-b2037339462e None admin] RBAC: Authorizing `identity:get_user(user_id=a9033d8c4ef14e3eaead57cd2b91f40f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:42.567309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776e8f8-aaea-498e-be84-0d3804d88f48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:42.568653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a23d939-aaaf-49ab-8787-b2037339462e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:42.570211 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 9/17] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:31:42 2022] GET /identity/v3/users/a9033d8c4ef14e3eaead57cd2b91f40f => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:31:42.576176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a23d939-aaaf-49ab-8787-b2037339462e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:42.592483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:42.682448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a23d939-aaaf-49ab-8787-b2037339462e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:42.695359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8776e8f8-aaea-498e-be84-0d3804d88f48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:42.697078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8776e8f8-aaea-498e-be84-0d3804d88f48 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:42.715437 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-8776e8f8-aaea-498e-be84-0d3804d88f48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:31:42.715437 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:31:42.715437 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:31:42.715437 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:31:42.715437 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:31:42.715437 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:31:42.715437 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:31:42.715437 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:31:42.715437 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:31:42.715437 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:31:42.715437 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:31:42.715437 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:31:42.715437 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:31:42.715437 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:31:42.715437 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:31:42.715437 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:31:42.716399 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:31:42.716399 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:31:42.716399 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:31:42.716399 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:31:42.716399 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:31:42.716399 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:31:42.716399 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:31:42.716399 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:31:42.716399 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:31:42.716399 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:31:42.716399 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:31:42.716399 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:31:42.716399 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:31:42.716399 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:31:42.716399 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:31:42.716399 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:31:42.716399 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:31:42.717223 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:31:42.717223 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:31:42.717223 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:31:42.717223 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 9/18] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:31:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:31:42.721950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c787dd-39df-4b4a-9c6d-1daded837c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:42.722204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776e8f8-aaea-498e-be84-0d3804d88f48 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:42.722611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c787dd-39df-4b4a-9c6d-1daded837c75 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:42.722897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c787dd-39df-4b4a-9c6d-1daded837c75 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:42.723210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c787dd-39df-4b4a-9c6d-1daded837c75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:42.724339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c787dd-39df-4b4a-9c6d-1daded837c75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:42.725799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c787dd-39df-4b4a-9c6d-1daded837c75 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:42.727381 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 10/19] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:31:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:31:42.732882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c787dd-39df-4b4a-9c6d-1daded837c75 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:42.778987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:42.819067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:42.823917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776e8f8-aaea-498e-be84-0d3804d88f48 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:42.839270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c787dd-39df-4b4a-9c6d-1daded837c75 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:42.858747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719fc31-0be3-417f-923b-d801e4f98b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:42.859296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719fc31-0be3-417f-923b-d801e4f98b7a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:42.859581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719fc31-0be3-417f-923b-d801e4f98b7a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:42.859858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719fc31-0be3-417f-923b-d801e4f98b7a None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:42.860499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719fc31-0be3-417f-923b-d801e4f98b7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:31:42.860805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719fc31-0be3-417f-923b-d801e4f98b7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:42.862188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719fc31-0be3-417f-923b-d801e4f98b7a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:42.872626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6414918c-7046-432c-ac12-5c5f3b04e06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:42.873225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414918c-7046-432c-ac12-5c5f3b04e06b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:42.873501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414918c-7046-432c-ac12-5c5f3b04e06b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:42.873771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414918c-7046-432c-ac12-5c5f3b04e06b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:42.874482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6414918c-7046-432c-ac12-5c5f3b04e06b None admin] RBAC: Adding query filter params (role.id=1c50896511054a5f994bee3d99fa0619, scope.domain.id=default, user.id=a9033d8c4ef14e3eaead57cd2b91f40f) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:31:42.874815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6414918c-7046-432c-ac12-5c5f3b04e06b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:42.876277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6414918c-7046-432c-ac12-5c5f3b04e06b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:42.879194 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 10/20] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:31:42.883056 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 11/21] 10.222.0.17 () {56 vars in 1322 bytes} [Sat Oct 1 20:31:42 2022] GET /identity/v3/role_assignments?user.id=a9033d8c4ef14e3eaead57cd2b91f40f&scope.domain.id=default&role.id=1c50896511054a5f994bee3d99fa0619 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:31:45.918135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d732101-4962-43cf-b84a-928189fdd125 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:45.919190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d732101-4962-43cf-b84a-928189fdd125 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:45.919697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d732101-4962-43cf-b84a-928189fdd125 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:45.921559 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 11/22] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:31:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:31:45.926951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c6f37-4fd2-4aaf-81b3-28f863913dfb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:45.927856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c6f37-4fd2-4aaf-81b3-28f863913dfb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:45.928332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c6f37-4fd2-4aaf-81b3-28f863913dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:45.950808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c41c6f37-4fd2-4aaf-81b3-28f863913dfb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:31:45.952341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c41c6f37-4fd2-4aaf-81b3-28f863913dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:31:46.040968 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 12/23] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:31:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:31:46.187834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ab41b-f0d6-486d-b81a-f2f6f589050e None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:46.187834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ab41b-f0d6-486d-b81a-f2f6f589050e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:46.187834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ab41b-f0d6-486d-b81a-f2f6f589050e None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:46.187834 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 12/24] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:31:46.195829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d3627-d795-4e9e-b912-922b7b15264d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:46.196314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d3627-d795-4e9e-b912-922b7b15264d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:46.196987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d3627-d795-4e9e-b912-922b7b15264d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:46.220050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d0046-5576-4c8d-8edc-cde98b9f5ba2 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:46.220335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d0046-5576-4c8d-8edc-cde98b9f5ba2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:46.220620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d0046-5576-4c8d-8edc-cde98b9f5ba2 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:46.221943 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 13/25] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:31:46.226615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b68d3627-d795-4e9e-b912-922b7b15264d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:31:46.227625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf96802-2358-4a02-8afe-bab2d0e5dfc6 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:46.227965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b68d3627-d795-4e9e-b912-922b7b15264d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:31:46.228252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf96802-2358-4a02-8afe-bab2d0e5dfc6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:46.228670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf96802-2358-4a02-8afe-bab2d0e5dfc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:46.263864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ebf96802-2358-4a02-8afe-bab2d0e5dfc6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:31:46.265578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ebf96802-2358-4a02-8afe-bab2d0e5dfc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:31:46.360787 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 14/26] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:31:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:31:46.400749 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 13/27] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:31:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:31:46.492478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf96802-2358-4a02-8afe-bab2d0e5dfc6 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:46.549902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:46.623017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf96802-2358-4a02-8afe-bab2d0e5dfc6 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:46.665142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3ab16-07c4-4d43-88d2-ea9abf750aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:46.666177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3ab16-07c4-4d43-88d2-ea9abf750aaf None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:46.666696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3ab16-07c4-4d43-88d2-ea9abf750aaf None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:46.667164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3ab16-07c4-4d43-88d2-ea9abf750aaf None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:46.668033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3ab16-07c4-4d43-88d2-ea9abf750aaf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:46.669622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3ab16-07c4-4d43-88d2-ea9abf750aaf None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:46.740596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff317e36-7d50-4b53-b366-774a08cb4b64 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:46.741241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff317e36-7d50-4b53-b366-774a08cb4b64 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:46.741714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff317e36-7d50-4b53-b366-774a08cb4b64 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:46.743328 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 14/28] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:31:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:31:46.748997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa8887-202f-4826-b86c-9fb5bf721e22 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:46.749617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa8887-202f-4826-b86c-9fb5bf721e22 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:46.750083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa8887-202f-4826-b86c-9fb5bf721e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:46.767351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:46.787492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b6aa8887-202f-4826-b86c-9fb5bf721e22 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:31:46.789009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b6aa8887-202f-4826-b86c-9fb5bf721e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:31:46.834996 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 15/29] 10.222.0.17 () {60 vars in 1131 bytes} [Sat Oct 1 20:31:46 2022] POST /identity/v3/roles => generated 219 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:31:46.852494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3ab16-07c4-4d43-88d2-ea9abf750aaf None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:46.930231 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 15/30] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:31:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:31:46.975887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa8887-202f-4826-b86c-9fb5bf721e22 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:47.113351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3ab16-07c4-4d43-88d2-ea9abf750aaf None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:47.130843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:47.182943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77218f3-7849-47b0-aed8-b840399de873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:47.183594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77218f3-7849-47b0-aed8-b840399de873 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:47.183879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77218f3-7849-47b0-aed8-b840399de873 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:47.184155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77218f3-7849-47b0-aed8-b840399de873 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:47.186397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa8887-202f-4826-b86c-9fb5bf721e22 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:47.200159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77218f3-7849-47b0-aed8-b840399de873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:31:47.207495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-a77218f3-7849-47b0-aed8-b840399de873 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:31:47.207495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:31:47.207495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:31:47.207495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:31:47.207495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:31:47.207495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:31:47.207495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:31:47.207495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:31:47.207495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:31:47.207495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:31:47.207495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:31:47.207495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:31:47.207495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:31:47.207495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:31:47.207495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:31:47.207495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:31:47.208458 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:31:47.208458 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:31:47.208458 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:31:47.208458 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:31:47.208458 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:31:47.208458 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:31:47.208458 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:31:47.208458 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:31:47.208458 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:31:47.208458 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:31:47.208458 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:31:47.208458 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:31:47.208458 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:31:47.208458 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:31:47.208458 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:31:47.208458 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:31:47.209300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:31:47.209300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:31:47.209300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:31:47.209300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:31:47.209300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:31:47.209300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:31:47.209300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:31:47.209300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:31:47.209300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:31:47.209300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:31:47.209300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:31:47.209300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:31:47.211831 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 16/31] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:31:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:31:47.224261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77218f3-7849-47b0-aed8-b840399de873 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:47.275201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85efc1-340f-491c-a795-06a129282587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:47.275868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85efc1-340f-491c-a795-06a129282587 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:47.276133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85efc1-340f-491c-a795-06a129282587 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:47.276573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85efc1-340f-491c-a795-06a129282587 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:47.285229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:47.297810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85efc1-340f-491c-a795-06a129282587 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:47.299530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85efc1-340f-491c-a795-06a129282587 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:47.317558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-7a85efc1-340f-491c-a795-06a129282587 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:31:47.317558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:31:47.317558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:31:47.317558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:31:47.317558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:31:47.317558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:31:47.317558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:31:47.317558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:31:47.317558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:31:47.317558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:31:47.317558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:31:47.317558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:31:47.317558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:31:47.317558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:31:47.317558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:31:47.317558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:31:47.318612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:31:47.318612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:31:47.318612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:31:47.318612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:31:47.318612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:31:47.318612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:31:47.318612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:31:47.318612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:31:47.318612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:31:47.318612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:31:47.318612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:31:47.318612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:31:47.318612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:31:47.318612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:31:47.318612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:31:47.318612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:31:47.318612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:31:47.319529 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:31:47.319529 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:31:47.319529 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:31:47.321161 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 16/32] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:31:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:31:47.322813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1abb5c-8b88-43f2-a5fc-9a77cc69ac2d None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:47.323230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1abb5c-8b88-43f2-a5fc-9a77cc69ac2d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:47.323868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1abb5c-8b88-43f2-a5fc-9a77cc69ac2d None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:47.325277 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 17/33] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:31:47.330190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1abb5c-8b88-43f2-a5fc-9a77cc69ac2d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:47.345468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77218f3-7849-47b0-aed8-b840399de873 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:47.383928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a324e6b-9050-4bde-ba9c-5fa5553a8e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:47.385077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a324e6b-9050-4bde-ba9c-5fa5553a8e56 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:47.385470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a324e6b-9050-4bde-ba9c-5fa5553a8e56 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:47.385847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a324e6b-9050-4bde-ba9c-5fa5553a8e56 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:47.386661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a324e6b-9050-4bde-ba9c-5fa5553a8e56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:31:47.387072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a324e6b-9050-4bde-ba9c-5fa5553a8e56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:47.388756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a324e6b-9050-4bde-ba9c-5fa5553a8e56 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:47.403005 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 17/34] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:31:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:31:47.404384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da69c47-ca61-416e-95c9-eac26d009191 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:47.404686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da69c47-ca61-416e-95c9-eac26d009191 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:47.404962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da69c47-ca61-416e-95c9-eac26d009191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:47.424826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1da69c47-ca61-416e-95c9-eac26d009191 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:31:47.427248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1da69c47-ca61-416e-95c9-eac26d009191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:31:47.454144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:47.469137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1abb5c-8b88-43f2-a5fc-9a77cc69ac2d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:47.500118 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 18/35] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:31:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:31:47.500954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da69c47-ca61-416e-95c9-eac26d009191 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:47.507370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a5d92-9582-415a-a1e8-7606a1430112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:47.507986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a5d92-9582-415a-a1e8-7606a1430112 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:47.508267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a5d92-9582-415a-a1e8-7606a1430112 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:47.508542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a5d92-9582-415a-a1e8-7606a1430112 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:47.509181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a5d92-9582-415a-a1e8-7606a1430112 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:31:47.509483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a5d92-9582-415a-a1e8-7606a1430112 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:47.510896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a5d92-9582-415a-a1e8-7606a1430112 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:47.530845 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 18/36] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:31:47.530845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a5d92-9582-415a-a1e8-7606a1430112 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:47.585254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:47.609436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da69c47-ca61-416e-95c9-eac26d009191 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:47.644397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a5d92-9582-415a-a1e8-7606a1430112 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:47.646310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d3d33-db09-4542-a61e-4ecfa44b7f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:47.647089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d3d33-db09-4542-a61e-4ecfa44b7f72 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:47.647369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d3d33-db09-4542-a61e-4ecfa44b7f72 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:47.647648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d3d33-db09-4542-a61e-4ecfa44b7f72 None admin] PATH_INFO: `/v3/users/a9033d8c4ef14e3eaead57cd2b91f40f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:47.649189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d3d33-db09-4542-a61e-4ecfa44b7f72 None admin] RBAC: Authorizing `identity:get_user(user_id=a9033d8c4ef14e3eaead57cd2b91f40f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:47.650670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d3d33-db09-4542-a61e-4ecfa44b7f72 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:47.652255 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 19/37] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:31:47 2022] GET /identity/v3/users/a9033d8c4ef14e3eaead57cd2b91f40f => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:31:47.653350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b14431-6899-42fb-965c-5b4b0d7c537c None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:47.653633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b14431-6899-42fb-965c-5b4b0d7c537c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:47.653905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b14431-6899-42fb-965c-5b4b0d7c537c None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:47.657465 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 20/38] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:31:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:31:47.657465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b14431-6899-42fb-965c-5b4b0d7c537c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:47.685892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9ec3a-0519-4e28-a6d3-6c5c9e29c167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:47.686523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9ec3a-0519-4e28-a6d3-6c5c9e29c167 None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:47.686818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9ec3a-0519-4e28-a6d3-6c5c9e29c167 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:47.687092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9ec3a-0519-4e28-a6d3-6c5c9e29c167 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:47.687751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9ec3a-0519-4e28-a6d3-6c5c9e29c167 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:47.688010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:47.689155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9ec3a-0519-4e28-a6d3-6c5c9e29c167 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:47.766687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b14431-6899-42fb-965c-5b4b0d7c537c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:47.802334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c763a8b-afee-4ddf-91db-ec348bbf0733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:47.802984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c763a8b-afee-4ddf-91db-ec348bbf0733 None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:47.803270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c763a8b-afee-4ddf-91db-ec348bbf0733 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:47.803548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c763a8b-afee-4ddf-91db-ec348bbf0733 None admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:47.804195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c763a8b-afee-4ddf-91db-ec348bbf0733 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:47.805671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c763a8b-afee-4ddf-91db-ec348bbf0733 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:47.817963 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 19/39] 10.222.0.17 () {60 vars in 1131 bytes} [Sat Oct 1 20:31:47 2022] POST /identity/v3/roles => generated 225 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:31:47.819232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9ec3a-0519-4e28-a6d3-6c5c9e29c167 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:47.842231 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 21/40] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Oct 1 20:31:47 2022] POST /identity/v3/projects => generated 301 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:31:47.843984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3c8fc-85b7-4725-b6d4-70046cea5d3a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:47.844472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3c8fc-85b7-4725-b6d4-70046cea5d3a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:47.844958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3c8fc-85b7-4725-b6d4-70046cea5d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:47.866681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fab3c8fc-85b7-4725-b6d4-70046cea5d3a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:31:47.868490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fab3c8fc-85b7-4725-b6d4-70046cea5d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:31:47.967237 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 22/41] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:31:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:31:47.968170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3c8fc-85b7-4725-b6d4-70046cea5d3a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:47.973939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9ec3a-0519-4e28-a6d3-6c5c9e29c167 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:47.980426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:47.983677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:48.021263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4221ec5a-6899-4d29-9d31-c3ad60804d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:48.022223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221ec5a-6899-4d29-9d31-c3ad60804d6d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:48.022868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221ec5a-6899-4d29-9d31-c3ad60804d6d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:48.023323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221ec5a-6899-4d29-9d31-c3ad60804d6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:48.024889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4221ec5a-6899-4d29-9d31-c3ad60804d6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:48.026819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4221ec5a-6899-4d29-9d31-c3ad60804d6d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:48.028808 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 20/42] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:31:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:31:48.037452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4221ec5a-6899-4d29-9d31-c3ad60804d6d None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:48.112211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3c8fc-85b7-4725-b6d4-70046cea5d3a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:48.151682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb21a25-0692-4068-9726-2cc3de2f14d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:48.152291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb21a25-0692-4068-9726-2cc3de2f14d9 None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:48.152573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb21a25-0692-4068-9726-2cc3de2f14d9 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:48.153949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb21a25-0692-4068-9726-2cc3de2f14d9 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:48.156648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb21a25-0692-4068-9726-2cc3de2f14d9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:48.160397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb21a25-0692-4068-9726-2cc3de2f14d9 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:48.178889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4221ec5a-6899-4d29-9d31-c3ad60804d6d None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:48.186577 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 23/43] 10.222.0.17 () {60 vars in 1131 bytes} [Sat Oct 1 20:31:47 2022] POST /identity/v3/roles => generated 223 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:31:48.228783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448df227-9477-4283-bc75-2f56363f77c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:48.229579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448df227-9477-4283-bc75-2f56363f77c9 None admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:48.229867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448df227-9477-4283-bc75-2f56363f77c9 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:48.230153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448df227-9477-4283-bc75-2f56363f77c9 None admin] PATH_INFO: `/v3/domains/default/users/a9033d8c4ef14e3eaead57cd2b91f40f/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:48.232409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448df227-9477-4283-bc75-2f56363f77c9 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a9033d8c4ef14e3eaead57cd2b91f40f, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:48.233921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448df227-9477-4283-bc75-2f56363f77c9 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:48.253336 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 21/44] 10.222.0.17 () {58 vars in 1361 bytes} [Sat Oct 1 20:31:48 2022] PUT /identity/v3/domains/default/users/a9033d8c4ef14e3eaead57cd2b91f40f/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:31:48.375936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb21a25-0692-4068-9726-2cc3de2f14d9 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:48.458396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:48.495815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb21a25-0692-4068-9726-2cc3de2f14d9 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:48.529649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c03885-b079-44bb-bf18-ed885013ffde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:48.530280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c03885-b079-44bb-bf18-ed885013ffde None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:48.530599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c03885-b079-44bb-bf18-ed885013ffde None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:48.530876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c03885-b079-44bb-bf18-ed885013ffde None admin] PATH_INFO: `/v3/domains/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:48.531989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c03885-b079-44bb-bf18-ed885013ffde None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:48.533418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c03885-b079-44bb-bf18-ed885013ffde None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:48.535026 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 24/45] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:31:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:31:48.541126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448df227-9477-4283-bc75-2f56363f77c9 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:48.575753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:48.642828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448df227-9477-4283-bc75-2f56363f77c9 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:48.677136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243a573-4354-46ea-b5a8-5b8e6d517ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:48.677749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243a573-4354-46ea-b5a8-5b8e6d517ddb None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:48.678032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243a573-4354-46ea-b5a8-5b8e6d517ddb None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:48.678304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243a573-4354-46ea-b5a8-5b8e6d517ddb None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:48.679051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4243a573-4354-46ea-b5a8-5b8e6d517ddb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:48.680515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4243a573-4354-46ea-b5a8-5b8e6d517ddb None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:48.710228 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 22/46] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Oct 1 20:31:48 2022] POST /identity/v3/projects => generated 312 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:31:50.680837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23e48c-052f-4a3b-98cd-bcee070c0a89 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:50.681378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23e48c-052f-4a3b-98cd-bcee070c0a89 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:50.681378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23e48c-052f-4a3b-98cd-bcee070c0a89 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:50.682838 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 25/47] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:31:50.689076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae09dad-de6a-4a05-bac4-0c67591844c4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:50.689591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae09dad-de6a-4a05-bac4-0c67591844c4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:50.689765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae09dad-de6a-4a05-bac4-0c67591844c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:50.712595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6ae09dad-de6a-4a05-bac4-0c67591844c4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:31:50.714117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6ae09dad-de6a-4a05-bac4-0c67591844c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:31:50.782153 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 23/48] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:31:51.106357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23e48c-052f-4a3b-98cd-bcee070c0a89 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:51.108245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcccded-bcf2-42e8-88ed-2ae1566ddbad None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:51.108536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcccded-bcf2-42e8-88ed-2ae1566ddbad None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:51.108816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcccded-bcf2-42e8-88ed-2ae1566ddbad None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:51.110163 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 24/49] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:31:51.115877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5a8e7-ba63-464b-860a-5b249b6e345f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:51.116165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5a8e7-ba63-464b-860a-5b249b6e345f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:51.116444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5a8e7-ba63-464b-860a-5b249b6e345f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:51.135600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f1a5a8e7-ba63-464b-860a-5b249b6e345f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:31:51.136792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f1a5a8e7-ba63-464b-860a-5b249b6e345f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:31:51.147722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:51.209018 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 25/50] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:31:51.211282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23e48c-052f-4a3b-98cd-bcee070c0a89 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:51.246256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9cc12-fb92-4ae2-9ed9-70e81260043b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:51.246921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9cc12-fb92-4ae2-9ed9-70e81260043b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:51.247202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9cc12-fb92-4ae2-9ed9-70e81260043b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:51.247476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9cc12-fb92-4ae2-9ed9-70e81260043b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:51.250066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:51.258473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9cc12-fb92-4ae2-9ed9-70e81260043b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:31:51.263344 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-d5c9cc12-fb92-4ae2-9ed9-70e81260043b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:31:51.263344 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:31:51.263344 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:31:51.263344 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:31:51.263344 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:31:51.263344 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:31:51.263344 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:31:51.263344 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:31:51.263344 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:31:51.263344 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:31:51.263344 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:31:51.263344 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:31:51.263344 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:31:51.263344 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:31:51.263344 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:31:51.263344 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:31:51.264195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:31:51.264195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:31:51.264195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:31:51.264195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:31:51.264195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:31:51.264195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:31:51.264195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:31:51.264195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:31:51.264195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:31:51.264195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:31:51.264195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:31:51.264195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:31:51.264195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:31:51.264195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:31:51.264195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:31:51.264195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:31:51.264949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:31:51.264949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:31:51.264949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:31:51.264949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:31:51.264949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:31:51.264949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:31:51.264949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:31:51.264949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:31:51.264949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:31:51.264949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:31:51.264949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:31:51.264949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:31:51.264949 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 26/51] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:31:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:31:51.273434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5a8e7-ba63-464b-860a-5b249b6e345f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:51.370832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5a8e7-ba63-464b-860a-5b249b6e345f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:51.404444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582af21-20e5-483c-9189-0b32f94dc34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:51.405057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582af21-20e5-483c-9189-0b32f94dc34e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:51.405344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582af21-20e5-483c-9189-0b32f94dc34e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:51.405625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582af21-20e5-483c-9189-0b32f94dc34e None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:51.406293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582af21-20e5-483c-9189-0b32f94dc34e None admin] RBAC: Adding query filter params (name=admin) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:31:51.406645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582af21-20e5-483c-9189-0b32f94dc34e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:51.408042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582af21-20e5-483c-9189-0b32f94dc34e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:51.416531 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 26/52] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:31:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:31:51.422157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9cc12-fb92-4ae2-9ed9-70e81260043b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:51.518009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9cc12-fb92-4ae2-9ed9-70e81260043b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:51.528933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582af21-20e5-483c-9189-0b32f94dc34e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:51.540374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:51.552972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee5794-e73f-4b1e-82e6-b82f51fffc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:51.553599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee5794-e73f-4b1e-82e6-b82f51fffc3d None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:51.553876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee5794-e73f-4b1e-82e6-b82f51fffc3d None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:51.554149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee5794-e73f-4b1e-82e6-b82f51fffc3d None admin] PATH_INFO: `/v3/users/a9033d8c4ef14e3eaead57cd2b91f40f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:51.555308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee5794-e73f-4b1e-82e6-b82f51fffc3d None admin] RBAC: Authorizing `identity:get_user(user_id=a9033d8c4ef14e3eaead57cd2b91f40f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:51.556749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee5794-e73f-4b1e-82e6-b82f51fffc3d None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:51.558317 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 27/53] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:31:51 2022] GET /identity/v3/users/a9033d8c4ef14e3eaead57cd2b91f40f => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:31:51.563579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee5794-e73f-4b1e-82e6-b82f51fffc3d None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:51.623242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:51.631534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582af21-20e5-483c-9189-0b32f94dc34e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:51.663067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee5794-e73f-4b1e-82e6-b82f51fffc3d None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:51.665282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52f35b-8dc1-4de5-a81e-ceb29319306b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:51.665933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52f35b-8dc1-4de5-a81e-ceb29319306b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:51.666224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52f35b-8dc1-4de5-a81e-ceb29319306b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:51.666513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52f35b-8dc1-4de5-a81e-ceb29319306b None admin] PATH_INFO: `/v3/domains/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:51.667641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52f35b-8dc1-4de5-a81e-ceb29319306b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:51.669228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52f35b-8dc1-4de5-a81e-ceb29319306b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:51.671091 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 27/54] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:31:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:31:51.678092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52f35b-8dc1-4de5-a81e-ceb29319306b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:51.702484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02689ba6-a91a-409e-b7c2-2895ab8f6fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:51.703217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02689ba6-a91a-409e-b7c2-2895ab8f6fb1 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:51.703499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02689ba6-a91a-409e-b7c2-2895ab8f6fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:51.703770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02689ba6-a91a-409e-b7c2-2895ab8f6fb1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:51.704919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02689ba6-a91a-409e-b7c2-2895ab8f6fb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:51.706375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02689ba6-a91a-409e-b7c2-2895ab8f6fb1 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:51.708604 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 28/55] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:31:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:31:51.714480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02689ba6-a91a-409e-b7c2-2895ab8f6fb1 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:51.760146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:51.784068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52f35b-8dc1-4de5-a81e-ceb29319306b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:51.814200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02689ba6-a91a-409e-b7c2-2895ab8f6fb1 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:51.817448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed275f-505a-4536-90dc-25f635baf67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:51.818049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed275f-505a-4536-90dc-25f635baf67b None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:51.818329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed275f-505a-4536-90dc-25f635baf67b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:51.818629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed275f-505a-4536-90dc-25f635baf67b None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:51.819263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed275f-505a-4536-90dc-25f635baf67b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:51.820709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed275f-505a-4536-90dc-25f635baf67b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:51.844787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:51.851053 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 28/56] 10.222.0.17 () {60 vars in 1140 bytes} [Sat Oct 1 20:31:51 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:31:51.852355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1333d0-e55d-4e53-a561-5605679da28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:51.853528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1333d0-e55d-4e53-a561-5605679da28d None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:51.854013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1333d0-e55d-4e53-a561-5605679da28d None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:51.854510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1333d0-e55d-4e53-a561-5605679da28d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:51.855714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1333d0-e55d-4e53-a561-5605679da28d None admin] RBAC: Adding query filter params (role.id=1c50896511054a5f994bee3d99fa0619, scope.domain.id=default, user.id=a9033d8c4ef14e3eaead57cd2b91f40f) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:31:51.856243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1333d0-e55d-4e53-a561-5605679da28d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:51.858724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1333d0-e55d-4e53-a561-5605679da28d None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:51.885049 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 29/57] 10.222.0.17 () {56 vars in 1322 bytes} [Sat Oct 1 20:31:51 2022] GET /identity/v3/role_assignments?user.id=a9033d8c4ef14e3eaead57cd2b91f40f&scope.domain.id=default&role.id=1c50896511054a5f994bee3d99fa0619 => generated 530 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:31:54.248114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c895e5-c72c-4aa9-859b-4d651a3718b7 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:54.248648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c895e5-c72c-4aa9-859b-4d651a3718b7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:54.248648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c895e5-c72c-4aa9-859b-4d651a3718b7 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:54.249972 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 29/58] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:31:54.255285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b2067-e1d3-48a8-93b5-29de35428484 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:54.255649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b2067-e1d3-48a8-93b5-29de35428484 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:54.255837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b2067-e1d3-48a8-93b5-29de35428484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:54.274122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-172b2067-e1d3-48a8-93b5-29de35428484 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:31:54.275356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-172b2067-e1d3-48a8-93b5-29de35428484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:31:54.342623 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 30/59] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:31:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:31:54.664905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c895e5-c72c-4aa9-859b-4d651a3718b7 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:54.764874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c895e5-c72c-4aa9-859b-4d651a3718b7 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:54.791086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:54.799309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518abc9-c8e6-4a85-b1f5-83785c6cd927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:54.799929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518abc9-c8e6-4a85-b1f5-83785c6cd927 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:54.800205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518abc9-c8e6-4a85-b1f5-83785c6cd927 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:54.800474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518abc9-c8e6-4a85-b1f5-83785c6cd927 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:54.802145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518abc9-c8e6-4a85-b1f5-83785c6cd927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:54.803625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518abc9-c8e6-4a85-b1f5-83785c6cd927 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:54.805163 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 30/60] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:31:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:31:54.810796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b2067-e1d3-48a8-93b5-29de35428484 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:54.906926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b2067-e1d3-48a8-93b5-29de35428484 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:54.931509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:54.940797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c36e6a-8fd3-4ce7-bda3-75218e67ecde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:54.941403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c36e6a-8fd3-4ce7-bda3-75218e67ecde None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:54.942268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c36e6a-8fd3-4ce7-bda3-75218e67ecde None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:54.942584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c36e6a-8fd3-4ce7-bda3-75218e67ecde None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:54.943234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c36e6a-8fd3-4ce7-bda3-75218e67ecde None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:54.944669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c36e6a-8fd3-4ce7-bda3-75218e67ecde None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:54.970908 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 31/61] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Oct 1 20:31:54 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:31:58.713339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2140cb-23a4-438a-950e-f6532ba39845 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:58.713339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2140cb-23a4-438a-950e-f6532ba39845 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:58.713339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2140cb-23a4-438a-950e-f6532ba39845 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:58.713339 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 31/62] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:31:58.720507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072f1b7-f6a7-4865-8710-845c08f8e17c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:58.721035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072f1b7-f6a7-4865-8710-845c08f8e17c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:58.721035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072f1b7-f6a7-4865-8710-845c08f8e17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:58.743856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c072f1b7-f6a7-4865-8710-845c08f8e17c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:31:58.745066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c072f1b7-f6a7-4865-8710-845c08f8e17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:31:58.798538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9831dd-c1de-4803-8f26-de91033207a7 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:58.798538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9831dd-c1de-4803-8f26-de91033207a7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:58.798538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9831dd-c1de-4803-8f26-de91033207a7 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:58.799848 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 32/63] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:31:58.813208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51c034-4eda-4955-81b1-6bd8a0cbd2d2 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:58.813208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51c034-4eda-4955-81b1-6bd8a0cbd2d2 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:58.813208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51c034-4eda-4955-81b1-6bd8a0cbd2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:58.829510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ca51c034-4eda-4955-81b1-6bd8a0cbd2d2 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:31:58.831021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ca51c034-4eda-4955-81b1-6bd8a0cbd2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:31:58.860821 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 32/64] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:31:58.903078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf3fac-60c7-4228-9fc3-f0330f6861b5 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:58.903704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf3fac-60c7-4228-9fc3-f0330f6861b5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:58.904182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf3fac-60c7-4228-9fc3-f0330f6861b5 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:58.905699 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 33/65] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:31:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:31:58.911745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587e16b-1b46-44fe-9cd5-274f85de367d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:58.912038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587e16b-1b46-44fe-9cd5-274f85de367d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:58.912567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587e16b-1b46-44fe-9cd5-274f85de367d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:58.928821 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 33/66] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:31:58.939789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b587e16b-1b46-44fe-9cd5-274f85de367d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:31:58.940988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b587e16b-1b46-44fe-9cd5-274f85de367d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:31:59.037405 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 34/67] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:31:59.202380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361af881-35cb-4138-ae71-34b6e89eee51 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:59.202728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361af881-35cb-4138-ae71-34b6e89eee51 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:59.202981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361af881-35cb-4138-ae71-34b6e89eee51 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:59.204457 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 34/68] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:31:59.218927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d16c7e-45de-4af0-940b-7d693c6e5241 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:59.219227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d16c7e-45de-4af0-940b-7d693c6e5241 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:59.219499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d16c7e-45de-4af0-940b-7d693c6e5241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:59.245869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a1d16c7e-45de-4af0-940b-7d693c6e5241 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:31:59.247123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a1d16c7e-45de-4af0-940b-7d693c6e5241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:31:59.264604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361af881-35cb-4138-ae71-34b6e89eee51 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:59.291684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:59.329259 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 35/69] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:31:59.330678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10635f24-1e18-4cb8-8914-c6706cbaab35 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:59.330973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10635f24-1e18-4cb8-8914-c6706cbaab35 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:59.331248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10635f24-1e18-4cb8-8914-c6706cbaab35 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:59.332601 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 36/70] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:31:59.337168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10635f24-1e18-4cb8-8914-c6706cbaab35 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:59.399224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361af881-35cb-4138-ae71-34b6e89eee51 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:59.436650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:59.446167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10635f24-1e18-4cb8-8914-c6706cbaab35 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:59.448626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fc200-66e9-42b2-93ed-ce3ea779d798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:59.450115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fc200-66e9-42b2-93ed-ce3ea779d798 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:59.450403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fc200-66e9-42b2-93ed-ce3ea779d798 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:59.450776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fc200-66e9-42b2-93ed-ce3ea779d798 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:59.464130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011fc200-66e9-42b2-93ed-ce3ea779d798 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:31:59.470340 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-011fc200-66e9-42b2-93ed-ce3ea779d798 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:31:59.470340 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:31:59.470340 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:31:59.470340 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:31:59.470340 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:31:59.470340 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:31:59.470340 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:31:59.470340 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:31:59.470340 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:31:59.470340 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:31:59.470340 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:31:59.470340 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:31:59.470340 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:31:59.470340 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:31:59.470340 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:31:59.470340 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:31:59.471205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:31:59.471205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:31:59.471205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:31:59.471205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:31:59.471205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:31:59.471205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:31:59.471205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:31:59.471205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:31:59.471205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:31:59.471205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:31:59.471205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:31:59.471205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:31:59.471205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:31:59.471205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:31:59.471205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:31:59.471205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:31:59.471963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:31:59.471963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:31:59.471963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:31:59.471963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:31:59.471963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:31:59.471963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:31:59.471963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:31:59.471963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:31:59.471963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:31:59.471963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:31:59.471963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:31:59.471963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:31:59.473313 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 35/71] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:31:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:31:59.475134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060a5bf-910f-4caf-b2be-68ba29431fb4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:59.475506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060a5bf-910f-4caf-b2be-68ba29431fb4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:59.475857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060a5bf-910f-4caf-b2be-68ba29431fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:59.488666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839e921-b061-4121-a0b1-de93582e683f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:59.489315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839e921-b061-4121-a0b1-de93582e683f None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:59.489602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839e921-b061-4121-a0b1-de93582e683f None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:59.489881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839e921-b061-4121-a0b1-de93582e683f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:59.494539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d060a5bf-910f-4caf-b2be-68ba29431fb4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:31:59.495721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d060a5bf-910f-4caf-b2be-68ba29431fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:31:59.500748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839e921-b061-4121-a0b1-de93582e683f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:31:59.506339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-1839e921-b061-4121-a0b1-de93582e683f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:31:59.506339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:31:59.506339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:31:59.506339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:31:59.506339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:31:59.506339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:31:59.506339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:31:59.506339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:31:59.506339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:31:59.506339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:31:59.506339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:31:59.506339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:31:59.506339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:31:59.506339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:31:59.506339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:31:59.506339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:31:59.507197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:31:59.507197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:31:59.507197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:31:59.507197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:31:59.507197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:31:59.507197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:31:59.507197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:31:59.507197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:31:59.507197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:31:59.507197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:31:59.507197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:31:59.507197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:31:59.507197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:31:59.507197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:31:59.507197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:31:59.507197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:31:59.508362 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:31:59.508362 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:31:59.508362 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:31:59.508362 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:31:59.508362 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:31:59.508362 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:31:59.508362 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:31:59.508362 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:31:59.508362 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:31:59.508362 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:31:59.508362 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:31:59.508362 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:31:59.508362 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 37/72] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:31:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:31:59.512860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839e921-b061-4121-a0b1-de93582e683f None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:59.546889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:59.564877 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 36/73] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:31:59.576047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060a5bf-910f-4caf-b2be-68ba29431fb4 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:59.622744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839e921-b061-4121-a0b1-de93582e683f None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:59.660807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fdc0e2-03ba-4132-a34a-8c75b47cafb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:59.661434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdc0e2-03ba-4132-a34a-8c75b47cafb3 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:59.661712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdc0e2-03ba-4132-a34a-8c75b47cafb3 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:59.661982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdc0e2-03ba-4132-a34a-8c75b47cafb3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:59.672684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fdc0e2-03ba-4132-a34a-8c75b47cafb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:31:59.677547 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-06fdc0e2-03ba-4132-a34a-8c75b47cafb3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:31:59.677547 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:31:59.677547 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:31:59.677547 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:31:59.677547 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:31:59.677547 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:31:59.677547 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:31:59.677547 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:31:59.677547 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:31:59.677547 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:31:59.677547 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:31:59.677547 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:31:59.677547 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:31:59.677547 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:31:59.677547 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:31:59.677547 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:31:59.678388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:31:59.678388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:31:59.678388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:31:59.678388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:31:59.678388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:31:59.678388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:31:59.678388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:31:59.678388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:31:59.678388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:31:59.678388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:31:59.678388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:31:59.678388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:31:59.678388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:31:59.678388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:31:59.678388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:31:59.678388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:31:59.679556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:31:59.679556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:31:59.679556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:31:59.679556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:31:59.679556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:31:59.679556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:31:59.679556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:31:59.679556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:31:59.679556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:31:59.679556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:31:59.679556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:31:59.679556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:31:59.679556 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 38/74] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:31:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:31:59.679556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fdc0e2-03ba-4132-a34a-8c75b47cafb3 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:59.708860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060a5bf-910f-4caf-b2be-68ba29431fb4 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:59.725069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:59.744149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d845f-db67-45a5-96a4-42b73a28b4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:59.744844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d845f-db67-45a5-96a4-42b73a28b4cf None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:59.745137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d845f-db67-45a5-96a4-42b73a28b4cf None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:59.745414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d845f-db67-45a5-96a4-42b73a28b4cf None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:59.746079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d845f-db67-45a5-96a4-42b73a28b4cf None admin] RBAC: Adding query filter params (name=member) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:31:59.746390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d845f-db67-45a5-96a4-42b73a28b4cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:59.747799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d845f-db67-45a5-96a4-42b73a28b4cf None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:59.757548 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 37/75] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:31:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:31:59.758433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d845f-db67-45a5-96a4-42b73a28b4cf None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:59.790799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:59.800461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fdc0e2-03ba-4132-a34a-8c75b47cafb3 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:59.836820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707f292-3abd-45f1-9b1f-1ee4dfa4f174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:59.837426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707f292-3abd-45f1-9b1f-1ee4dfa4f174 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:59.837705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707f292-3abd-45f1-9b1f-1ee4dfa4f174 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:59.837975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707f292-3abd-45f1-9b1f-1ee4dfa4f174 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:59.838673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707f292-3abd-45f1-9b1f-1ee4dfa4f174 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:31:59.838973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707f292-3abd-45f1-9b1f-1ee4dfa4f174 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:59.840343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707f292-3abd-45f1-9b1f-1ee4dfa4f174 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:59.849380 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 39/76] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:31:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:31:59.850159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707f292-3abd-45f1-9b1f-1ee4dfa4f174 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:59.869998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d845f-db67-45a5-96a4-42b73a28b4cf None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:59.903929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099ed25-731f-427d-b26e-2af7bf1a432b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:59.904564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099ed25-731f-427d-b26e-2af7bf1a432b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:59.904847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099ed25-731f-427d-b26e-2af7bf1a432b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:59.905121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099ed25-731f-427d-b26e-2af7bf1a432b None admin] PATH_INFO: `/v3/domains/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:59.906270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099ed25-731f-427d-b26e-2af7bf1a432b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:59.907782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099ed25-731f-427d-b26e-2af7bf1a432b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:31:59.909485 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 38/77] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:31:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:31:59.910710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099ed25-731f-427d-b26e-2af7bf1a432b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:31:59.955819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707f292-3abd-45f1-9b1f-1ee4dfa4f174 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:31:59.956060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:31:59.989751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f390e8-5d9d-4d5f-9fd3-00bdb8d22ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:31:59.990360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f390e8-5d9d-4d5f-9fd3-00bdb8d22ef9 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:31:59.990678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f390e8-5d9d-4d5f-9fd3-00bdb8d22ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:31:59.990956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f390e8-5d9d-4d5f-9fd3-00bdb8d22ef9 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:31:59.991631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f390e8-5d9d-4d5f-9fd3-00bdb8d22ef9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:31:59.991936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f390e8-5d9d-4d5f-9fd3-00bdb8d22ef9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:31:59.993329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f390e8-5d9d-4d5f-9fd3-00bdb8d22ef9 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:00.001199 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 40/78] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:31:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:00.001989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f390e8-5d9d-4d5f-9fd3-00bdb8d22ef9 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:00.009899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099ed25-731f-427d-b26e-2af7bf1a432b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:00.023466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:00.044544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4a31a-5b76-4466-84ed-b495bb873707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:00.045207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4a31a-5b76-4466-84ed-b495bb873707 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:00.045492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4a31a-5b76-4466-84ed-b495bb873707 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:00.045772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4a31a-5b76-4466-84ed-b495bb873707 None admin] PATH_INFO: `/v3/users/2e606e5ac9f84891835962715092f667` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:00.060519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4a31a-5b76-4466-84ed-b495bb873707 None admin] RBAC: Authorizing `identity:get_user(user_id=2e606e5ac9f84891835962715092f667)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:00.061920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4a31a-5b76-4466-84ed-b495bb873707 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:00.063586 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 39/79] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:31:59 2022] GET /identity/v3/users/2e606e5ac9f84891835962715092f667 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:00.064434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4a31a-5b76-4466-84ed-b495bb873707 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:00.109304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f390e8-5d9d-4d5f-9fd3-00bdb8d22ef9 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:00.143368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c8d1e-586d-4ee6-adb3-c28c5c69b6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:00.144000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c8d1e-586d-4ee6-adb3-c28c5c69b6d4 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:00.144284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c8d1e-586d-4ee6-adb3-c28c5c69b6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:00.145078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c8d1e-586d-4ee6-adb3-c28c5c69b6d4 None admin] PATH_INFO: `/v3/users/a9033d8c4ef14e3eaead57cd2b91f40f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:00.146218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c8d1e-586d-4ee6-adb3-c28c5c69b6d4 None admin] RBAC: Authorizing `identity:get_user(user_id=a9033d8c4ef14e3eaead57cd2b91f40f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:00.147702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c8d1e-586d-4ee6-adb3-c28c5c69b6d4 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:00.149265 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 41/80] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:00 2022] GET /identity/v3/users/a9033d8c4ef14e3eaead57cd2b91f40f => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:00.150078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c8d1e-586d-4ee6-adb3-c28c5c69b6d4 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:00.164310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4a31a-5b76-4466-84ed-b495bb873707 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:00.166940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:00.199442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f3c29-90a3-434b-9b85-979b99387415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:00.200068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f3c29-90a3-434b-9b85-979b99387415 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:00.200346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f3c29-90a3-434b-9b85-979b99387415 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:00.200624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f3c29-90a3-434b-9b85-979b99387415 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:00.210678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f3c29-90a3-434b-9b85-979b99387415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:00.215504 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-aa9f3c29-90a3-434b-9b85-979b99387415 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:00.215504 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:00.215504 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:00.215504 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:00.215504 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:00.215504 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:00.215504 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:00.215504 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:00.215504 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:00.215504 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:00.215504 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:00.215504 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:00.215504 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:00.215504 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:00.215504 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:00.215504 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:00.216445 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:00.216445 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:00.216445 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:00.216445 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:00.216445 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:00.216445 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:00.216445 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:00.216445 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:00.216445 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:00.216445 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:00.216445 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:00.216445 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:00.216445 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:00.216445 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:00.216445 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:00.216445 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:00.217519 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:00.217519 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:00.217519 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:00.217519 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:00.217519 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:00.217519 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:00.217519 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:00.217519 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:00.217519 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:00.217519 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:00.217519 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:00.217519 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:32:00.217519 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 40/81] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:00.217519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f3c29-90a3-434b-9b85-979b99387415 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:00.239447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:00.256850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c8d1e-586d-4ee6-adb3-c28c5c69b6d4 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:00.289598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86622cf2-340c-4862-9b86-8941e1007f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:00.290211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86622cf2-340c-4862-9b86-8941e1007f90 None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:00.290527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86622cf2-340c-4862-9b86-8941e1007f90 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:00.290802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86622cf2-340c-4862-9b86-8941e1007f90 None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:00.291449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86622cf2-340c-4862-9b86-8941e1007f90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:00.292950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86622cf2-340c-4862-9b86-8941e1007f90 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:00.314728 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 42/82] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Oct 1 20:32:00 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:32:00.315468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86622cf2-340c-4862-9b86-8941e1007f90 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:00.320603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f3c29-90a3-434b-9b85-979b99387415 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:00.357108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abddc1-9c48-4afc-8e0f-27f87d7af3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:00.357753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abddc1-9c48-4afc-8e0f-27f87d7af3b4 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:00.358038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abddc1-9c48-4afc-8e0f-27f87d7af3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:00.358312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abddc1-9c48-4afc-8e0f-27f87d7af3b4 None admin] PATH_INFO: `/v3/users/2e606e5ac9f84891835962715092f667` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:00.359555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26abddc1-9c48-4afc-8e0f-27f87d7af3b4 None admin] RBAC: Authorizing `identity:get_user(user_id=2e606e5ac9f84891835962715092f667)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:00.361056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26abddc1-9c48-4afc-8e0f-27f87d7af3b4 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:00.363021 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 41/83] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:00 2022] GET /identity/v3/users/2e606e5ac9f84891835962715092f667 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:00.364323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abddc1-9c48-4afc-8e0f-27f87d7af3b4 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:00.399607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:00.437157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86622cf2-340c-4862-9b86-8941e1007f90 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:00.475828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb9f82-47dc-4536-88f4-0cac42f0690d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:00.476458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb9f82-47dc-4536-88f4-0cac42f0690d None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:00.476743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb9f82-47dc-4536-88f4-0cac42f0690d None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:00.477024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb9f82-47dc-4536-88f4-0cac42f0690d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:00.480115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abddc1-9c48-4afc-8e0f-27f87d7af3b4 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:00.489247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:00.519830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a98394-ee28-4146-a2f3-c5de82c4b31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:00.520449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a98394-ee28-4146-a2f3-c5de82c4b31f None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:00.521437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a98394-ee28-4146-a2f3-c5de82c4b31f None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:00.521437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a98394-ee28-4146-a2f3-c5de82c4b31f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:00.565571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb9f82-47dc-4536-88f4-0cac42f0690d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:00.566986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb9f82-47dc-4536-88f4-0cac42f0690d None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:00.601501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a98394-ee28-4146-a2f3-c5de82c4b31f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:00.603031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a98394-ee28-4146-a2f3-c5de82c4b31f None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:00.643330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:00.695336 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 43/84] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:00.696039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb9f82-47dc-4536-88f4-0cac42f0690d None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:00.732907 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 42/85] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:00.733618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a98394-ee28-4146-a2f3-c5de82c4b31f None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:00.760170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:00.836842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb9f82-47dc-4536-88f4-0cac42f0690d None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:00.843593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a98394-ee28-4146-a2f3-c5de82c4b31f None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:00.876657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2ff19-b0fb-49da-b110-886cb958ebec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:00.877329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2ff19-b0fb-49da-b110-886cb958ebec None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:00.877621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2ff19-b0fb-49da-b110-886cb958ebec None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:00.877897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2ff19-b0fb-49da-b110-886cb958ebec None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:00.878613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2ff19-b0fb-49da-b110-886cb958ebec None admin] RBAC: Adding query filter params (name=member) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:00.879071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2ff19-b0fb-49da-b110-886cb958ebec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:00.880380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2ff19-b0fb-49da-b110-886cb958ebec None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:00.891110 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 44/86] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:00.891965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2ff19-b0fb-49da-b110-886cb958ebec None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:00.894227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993933d6-8b47-4a1b-bc38-f268847d5324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:00.895714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993933d6-8b47-4a1b-bc38-f268847d5324 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:00.896244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:00.896709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993933d6-8b47-4a1b-bc38-f268847d5324 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:00.897435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993933d6-8b47-4a1b-bc38-f268847d5324 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:00.971275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993933d6-8b47-4a1b-bc38-f268847d5324 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:00.972828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993933d6-8b47-4a1b-bc38-f268847d5324 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:01.022054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2ff19-b0fb-49da-b110-886cb958ebec None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:01.024730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:01.078640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067eef6-5149-4d18-ad58-c5f4130db5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:01.079906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067eef6-5149-4d18-ad58-c5f4130db5e0 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:01.080545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067eef6-5149-4d18-ad58-c5f4130db5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:01.081348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067eef6-5149-4d18-ad58-c5f4130db5e0 None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:01.083179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067eef6-5149-4d18-ad58-c5f4130db5e0 None admin] RBAC: Authorizing `identity:get_project(project_id=a503020af843467a992a8df7f46d3fec)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:01.085554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067eef6-5149-4d18-ad58-c5f4130db5e0 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:01.087548 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 43/88] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:01.087834 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 45/88] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:00 2022] GET /identity/v3/projects/a503020af843467a992a8df7f46d3fec => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:01.088930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067eef6-5149-4d18-ad58-c5f4130db5e0 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:01.093045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993933d6-8b47-4a1b-bc38-f268847d5324 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:01.124257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:01.196741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993933d6-8b47-4a1b-bc38-f268847d5324 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:01.223539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067eef6-5149-4d18-ad58-c5f4130db5e0 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:01.233757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99cccf-0e9d-4480-89aa-240b06b2673b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:01.234371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99cccf-0e9d-4480-89aa-240b06b2673b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:01.234723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99cccf-0e9d-4480-89aa-240b06b2673b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:01.234977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99cccf-0e9d-4480-89aa-240b06b2673b None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:01.236190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99cccf-0e9d-4480-89aa-240b06b2673b None admin] RBAC: Authorizing `identity:get_project(project_id=a503020af843467a992a8df7f46d3fec)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:01.237670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99cccf-0e9d-4480-89aa-240b06b2673b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:01.239911 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 44/89] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:01 2022] GET /identity/v3/projects/a503020af843467a992a8df7f46d3fec => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:01.240679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99cccf-0e9d-4480-89aa-240b06b2673b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:01.273023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68e189-3886-40e3-9f86-f38da8b4262e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:01.273664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68e189-3886-40e3-9f86-f38da8b4262e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:01.273946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68e189-3886-40e3-9f86-f38da8b4262e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:01.274222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68e189-3886-40e3-9f86-f38da8b4262e None admin] PATH_INFO: `/v3/users/2e606e5ac9f84891835962715092f667` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:01.275399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68e189-3886-40e3-9f86-f38da8b4262e None admin] RBAC: Authorizing `identity:get_user(user_id=2e606e5ac9f84891835962715092f667)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:01.277098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68e189-3886-40e3-9f86-f38da8b4262e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:01.279179 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 46/90] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:01 2022] GET /identity/v3/users/2e606e5ac9f84891835962715092f667 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:01.280153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68e189-3886-40e3-9f86-f38da8b4262e None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:01.324204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:01.364201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:01.369194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99cccf-0e9d-4480-89aa-240b06b2673b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:01.393890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68e189-3886-40e3-9f86-f38da8b4262e None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:01.417578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f53b8c-5d3c-4c80-b8ad-53e1e9e867c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:01.418667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f53b8c-5d3c-4c80-b8ad-53e1e9e867c9 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:01.419094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f53b8c-5d3c-4c80-b8ad-53e1e9e867c9 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:01.419605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f53b8c-5d3c-4c80-b8ad-53e1e9e867c9 None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:01.420886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f53b8c-5d3c-4c80-b8ad-53e1e9e867c9 None admin] RBAC: Authorizing `identity:get_project(project_id=a503020af843467a992a8df7f46d3fec)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:01.423119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f53b8c-5d3c-4c80-b8ad-53e1e9e867c9 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:01.429583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81f239-68e9-446f-a02d-1438b97d9f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:01.430308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81f239-68e9-446f-a02d-1438b97d9f83 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:01.430727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81f239-68e9-446f-a02d-1438b97d9f83 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:01.431093 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 45/91] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:01 2022] GET /identity/v3/projects/a503020af843467a992a8df7f46d3fec => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:01.431417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81f239-68e9-446f-a02d-1438b97d9f83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:01.431920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f53b8c-5d3c-4c80-b8ad-53e1e9e867c9 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:01.432273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81f239-68e9-446f-a02d-1438b97d9f83 None admin] RBAC: Adding query filter params (role.id=17f3aeacc5034f3891cc6556da651c1b, scope.project.id=a503020af843467a992a8df7f46d3fec, user.id=2e606e5ac9f84891835962715092f667) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:01.432644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81f239-68e9-446f-a02d-1438b97d9f83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:01.434141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81f239-68e9-446f-a02d-1438b97d9f83 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:01.440423 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 47/92] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:32:01 2022] GET /identity/v3/role_assignments?user.id=2e606e5ac9f84891835962715092f667&scope.project.id=a503020af843467a992a8df7f46d3fec&role.id=17f3aeacc5034f3891cc6556da651c1b => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:01.441329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81f239-68e9-446f-a02d-1438b97d9f83 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:01.553027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81f239-68e9-446f-a02d-1438b97d9f83 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:01.555059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:01.578727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f53b8c-5d3c-4c80-b8ad-53e1e9e867c9 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:01.606131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579cc6a-2a54-486b-ab14-e9aaed9fb379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:01.606791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579cc6a-2a54-486b-ab14-e9aaed9fb379 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:01.607076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579cc6a-2a54-486b-ab14-e9aaed9fb379 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:01.607348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579cc6a-2a54-486b-ab14-e9aaed9fb379 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:01.634536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819b141-b11a-4be3-903b-b7fdf7453eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:01.635085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819b141-b11a-4be3-903b-b7fdf7453eb9 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:01.635368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819b141-b11a-4be3-903b-b7fdf7453eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:01.635637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819b141-b11a-4be3-903b-b7fdf7453eb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:01.636386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819b141-b11a-4be3-903b-b7fdf7453eb9 None admin] RBAC: Adding query filter params (role.id=1c50896511054a5f994bee3d99fa0619, scope.project.id=a503020af843467a992a8df7f46d3fec, user.id=a9033d8c4ef14e3eaead57cd2b91f40f) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:01.636691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819b141-b11a-4be3-903b-b7fdf7453eb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:01.638215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819b141-b11a-4be3-903b-b7fdf7453eb9 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:01.652359 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 46/93] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:32:01 2022] GET /identity/v3/role_assignments?user.id=a9033d8c4ef14e3eaead57cd2b91f40f&scope.project.id=a503020af843467a992a8df7f46d3fec&role.id=1c50896511054a5f994bee3d99fa0619 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:01.658961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819b141-b11a-4be3-903b-b7fdf7453eb9 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:01.750933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:01.790206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2579cc6a-2a54-486b-ab14-e9aaed9fb379 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:01.791777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2579cc6a-2a54-486b-ab14-e9aaed9fb379 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:01.901613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819b141-b11a-4be3-903b-b7fdf7453eb9 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:01.943492 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 48/94] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:01.949405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579cc6a-2a54-486b-ab14-e9aaed9fb379 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:01.958875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:01.965365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11680a36-afad-49ce-a931-4f9b0e475f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:01.966003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11680a36-afad-49ce-a931-4f9b0e475f13 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:01.966287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11680a36-afad-49ce-a931-4f9b0e475f13 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:01.966606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11680a36-afad-49ce-a931-4f9b0e475f13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:01.967359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11680a36-afad-49ce-a931-4f9b0e475f13 None admin] RBAC: Adding query filter params (role.id=8b7b6e0028b04be8912abda33d3ef729, scope.project.id=a503020af843467a992a8df7f46d3fec, user.id=2e606e5ac9f84891835962715092f667) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:01.967669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11680a36-afad-49ce-a931-4f9b0e475f13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:01.969194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11680a36-afad-49ce-a931-4f9b0e475f13 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:01.985303 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 47/95] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:32:01 2022] GET /identity/v3/role_assignments?user.id=2e606e5ac9f84891835962715092f667&scope.project.id=a503020af843467a992a8df7f46d3fec&role.id=8b7b6e0028b04be8912abda33d3ef729 => generated 259 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:02.069510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579cc6a-2a54-486b-ab14-e9aaed9fb379 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:02.118342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9aa300-865e-40bd-9ee0-237bb8ec46c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:02.119028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9aa300-865e-40bd-9ee0-237bb8ec46c1 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:02.119315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9aa300-865e-40bd-9ee0-237bb8ec46c1 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:02.119595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9aa300-865e-40bd-9ee0-237bb8ec46c1 None admin] PATH_INFO: `/v3/projects/213ed96d79ee4169824ebf710c0f9d41` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:02.120763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9aa300-865e-40bd-9ee0-237bb8ec46c1 None admin] RBAC: Authorizing `identity:get_project(project_id=213ed96d79ee4169824ebf710c0f9d41)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:02.122234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9aa300-865e-40bd-9ee0-237bb8ec46c1 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:02.124091 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 49/96] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:01 2022] GET /identity/v3/projects/213ed96d79ee4169824ebf710c0f9d41 => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:02.131082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11680a36-afad-49ce-a931-4f9b0e475f13 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:02.273963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:02.282026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11680a36-afad-49ce-a931-4f9b0e475f13 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:02.327212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57774464-5847-4dc8-9af4-8dd273a96e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:02.328067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57774464-5847-4dc8-9af4-8dd273a96e23 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:02.328353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57774464-5847-4dc8-9af4-8dd273a96e23 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:02.328629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57774464-5847-4dc8-9af4-8dd273a96e23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:02.329368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57774464-5847-4dc8-9af4-8dd273a96e23 None admin] RBAC: Adding query filter params (role.id=17f3aeacc5034f3891cc6556da651c1b, scope.project.id=213ed96d79ee4169824ebf710c0f9d41, user.id=2e606e5ac9f84891835962715092f667) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:02.329678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57774464-5847-4dc8-9af4-8dd273a96e23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:02.331215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57774464-5847-4dc8-9af4-8dd273a96e23 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:02.337284 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 48/97] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:32:02 2022] GET /identity/v3/role_assignments?user.id=2e606e5ac9f84891835962715092f667&scope.project.id=213ed96d79ee4169824ebf710c0f9d41&role.id=17f3aeacc5034f3891cc6556da651c1b => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:03.443904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918c4e0-9019-42eb-867f-3b190be01803 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:03.444386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918c4e0-9019-42eb-867f-3b190be01803 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:03.444386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918c4e0-9019-42eb-867f-3b190be01803 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:03.445715 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 50/98] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:03.451206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dfc982-4bbc-4a7a-b11d-5df5a0918af9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:03.451613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dfc982-4bbc-4a7a-b11d-5df5a0918af9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:03.451779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dfc982-4bbc-4a7a-b11d-5df5a0918af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:03.479600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-63dfc982-4bbc-4a7a-b11d-5df5a0918af9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:03.480831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-63dfc982-4bbc-4a7a-b11d-5df5a0918af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:03.563774 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 49/99] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:03.586478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8424e5-83d8-4717-9d75-e87f3b2270ec None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:03.587120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8424e5-83d8-4717-9d75-e87f3b2270ec None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:03.587596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8424e5-83d8-4717-9d75-e87f3b2270ec None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:03.589171 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 51/100] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:03.594745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf81897-133d-4c7c-8cc6-3299adca9d04 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:03.595041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf81897-133d-4c7c-8cc6-3299adca9d04 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:03.595322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf81897-133d-4c7c-8cc6-3299adca9d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:03.617817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2bf81897-133d-4c7c-8cc6-3299adca9d04 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:03.619018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2bf81897-133d-4c7c-8cc6-3299adca9d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:03.719967 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 50/101] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:03.993953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8424e5-83d8-4717-9d75-e87f3b2270ec None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:04.089734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:04.124644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8424e5-83d8-4717-9d75-e87f3b2270ec None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:04.162696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e0f9b-a176-4bcc-b464-f92797fd6e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:04.163333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e0f9b-a176-4bcc-b464-f92797fd6e0b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:04.163615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e0f9b-a176-4bcc-b464-f92797fd6e0b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:04.163896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e0f9b-a176-4bcc-b464-f92797fd6e0b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:04.179586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e0f9b-a176-4bcc-b464-f92797fd6e0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:04.185025 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-152e0f9b-a176-4bcc-b464-f92797fd6e0b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 01 20:32:04.185025 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:04.185025 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:04.185025 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:04.185025 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:04.185025 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:04.185025 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:04.185025 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:04.185025 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:04.185025 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:04.185025 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:04.185025 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:04.185025 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:04.185025 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:04.185025 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:04.185025 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:04.185922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:04.185922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:04.185922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:04.185922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:04.185922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:04.185922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:04.185922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:04.185922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:04.185922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:04.185922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:04.185922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:04.185922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:04.185922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:04.185922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:04.185922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:04.185922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:04.186721 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:04.186721 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:04.186721 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:04.186721 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:04.186721 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:04.186721 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:04.186721 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:04.186721 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:04.186721 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:04.186721 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:04.186721 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 01 20:32:04.186721 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:04.186721 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 52/102] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:04.194392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf81897-133d-4c7c-8cc6-3299adca9d04 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:04.238873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:04.365651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf81897-133d-4c7c-8cc6-3299adca9d04 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:04.410940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d355a-b177-4ecf-92cb-891b9fa6163d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:04.411907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d355a-b177-4ecf-92cb-891b9fa6163d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:04.412378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d355a-b177-4ecf-92cb-891b9fa6163d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:04.412839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d355a-b177-4ecf-92cb-891b9fa6163d None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:04.413714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d355a-b177-4ecf-92cb-891b9fa6163d None admin] RBAC: Adding query filter params (name=reader) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:04.414205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d355a-b177-4ecf-92cb-891b9fa6163d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:04.415811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d355a-b177-4ecf-92cb-891b9fa6163d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:04.425052 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 51/103] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:04.430326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e0f9b-a176-4bcc-b464-f92797fd6e0b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:04.434539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d355a-b177-4ecf-92cb-891b9fa6163d None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:04.581818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:04.617508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d355a-b177-4ecf-92cb-891b9fa6163d None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:04.655157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e0f9b-a176-4bcc-b464-f92797fd6e0b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:04.708139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a52cb-cfe6-4cd2-bb82-8e8821c616d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:04.714844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a52cb-cfe6-4cd2-bb82-8e8821c616d8 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:04.715471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a52cb-cfe6-4cd2-bb82-8e8821c616d8 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:04.715894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a52cb-cfe6-4cd2-bb82-8e8821c616d8 None admin] PATH_INFO: `/v3/users/ac00bd52ce5943af943ce9ed56577c79` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:04.738850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:04.744988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a52cb-cfe6-4cd2-bb82-8e8821c616d8 None admin] RBAC: Authorizing `identity:get_user(user_id=ac00bd52ce5943af943ce9ed56577c79)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:04.746533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a52cb-cfe6-4cd2-bb82-8e8821c616d8 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:04.748270 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 52/104] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:04 2022] GET /identity/v3/users/ac00bd52ce5943af943ce9ed56577c79 => generated 289 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:04.754756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a52cb-cfe6-4cd2-bb82-8e8821c616d8 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:04.767181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ad16b-0187-4eb3-a64e-b045b2f8e08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:04.768761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ad16b-0187-4eb3-a64e-b045b2f8e08a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:04.769045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ad16b-0187-4eb3-a64e-b045b2f8e08a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:04.769355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ad16b-0187-4eb3-a64e-b045b2f8e08a None admin] PATH_INFO: `/v3/domains/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:04.770534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ad16b-0187-4eb3-a64e-b045b2f8e08a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:04.771974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ad16b-0187-4eb3-a64e-b045b2f8e08a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:04.774298 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 53/105] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:32:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:04.779271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ad16b-0187-4eb3-a64e-b045b2f8e08a None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:04.968736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a52cb-cfe6-4cd2-bb82-8e8821c616d8 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:04.970710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ad16b-0187-4eb3-a64e-b045b2f8e08a None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:05.051826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1992a0-81e9-4ca6-a0f0-76ee69600eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:05.052456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1992a0-81e9-4ca6-a0f0-76ee69600eef None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:05.052738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1992a0-81e9-4ca6-a0f0-76ee69600eef None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:05.053012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1992a0-81e9-4ca6-a0f0-76ee69600eef None admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:05.053678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1992a0-81e9-4ca6-a0f0-76ee69600eef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:05.055292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1992a0-81e9-4ca6-a0f0-76ee69600eef None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:05.065020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875a90d-3c27-4e12-b0fb-1f5b4d6f92cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:05.065645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875a90d-3c27-4e12-b0fb-1f5b4d6f92cd None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:05.065930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875a90d-3c27-4e12-b0fb-1f5b4d6f92cd None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:05.066886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875a90d-3c27-4e12-b0fb-1f5b4d6f92cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:05.084911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:05.111422 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 54/106] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Oct 1 20:32:04 2022] POST /identity/v3/projects => generated 302 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:32:05.198465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875a90d-3c27-4e12-b0fb-1f5b4d6f92cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:05.200006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875a90d-3c27-4e12-b0fb-1f5b4d6f92cd None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:05.307518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92813f76-da2f-4566-a81e-0c6dee011d42 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:05.307698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92813f76-da2f-4566-a81e-0c6dee011d42 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:05.307998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92813f76-da2f-4566-a81e-0c6dee011d42 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:05.312849 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 55/107] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:05.317542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04441027-9c9a-4f2a-8982-eba75f926735 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:05.317829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04441027-9c9a-4f2a-8982-eba75f926735 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:05.318112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04441027-9c9a-4f2a-8982-eba75f926735 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:05.319495 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 56/108] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:32:05.325585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bb115-9a24-4a39-92ef-6dfc2d307667 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:05.325888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bb115-9a24-4a39-92ef-6dfc2d307667 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:05.326150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bb115-9a24-4a39-92ef-6dfc2d307667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:05.353237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-de9bb115-9a24-4a39-92ef-6dfc2d307667 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:05.355377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-de9bb115-9a24-4a39-92ef-6dfc2d307667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:05.367575 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 53/109] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:05.368965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6b820-c481-4ada-ad99-9d8b8b0c2d3b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:05.369256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6b820-c481-4ada-ad99-9d8b8b0c2d3b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:05.369534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6b820-c481-4ada-ad99-9d8b8b0c2d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:05.414520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-98a6b820-c481-4ada-ad99-9d8b8b0c2d3b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:05.414520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-98a6b820-c481-4ada-ad99-9d8b8b0c2d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:05.440545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:05.476984 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 57/110] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:32:05.478308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9bb115-9a24-4a39-92ef-6dfc2d307667 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:05.481397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:05.554748 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 54/111] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:05.615651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9bb115-9a24-4a39-92ef-6dfc2d307667 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:05.654215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad30de-bf69-4266-a6a6-5f2dd7cff310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:05.654878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad30de-bf69-4266-a6a6-5f2dd7cff310 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:05.655197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad30de-bf69-4266-a6a6-5f2dd7cff310 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:05.655722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad30de-bf69-4266-a6a6-5f2dd7cff310 None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:05.656601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad30de-bf69-4266-a6a6-5f2dd7cff310 None admin] RBAC: Authorizing `identity:get_project(project_id=a503020af843467a992a8df7f46d3fec)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:05.658053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad30de-bf69-4266-a6a6-5f2dd7cff310 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:05.663016 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 58/112] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:05 2022] GET /identity/v3/projects/a503020af843467a992a8df7f46d3fec => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:05.666628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6b820-c481-4ada-ad99-9d8b8b0c2d3b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:05.784327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6b820-c481-4ada-ad99-9d8b8b0c2d3b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:05.828949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a5f5d-e96b-45b1-ae7d-bbc250ce3efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:05.829570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a5f5d-e96b-45b1-ae7d-bbc250ce3efd None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:05.829859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a5f5d-e96b-45b1-ae7d-bbc250ce3efd None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:05.830136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a5f5d-e96b-45b1-ae7d-bbc250ce3efd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:05.830891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a5f5d-e96b-45b1-ae7d-bbc250ce3efd None admin] RBAC: Adding query filter params (role.id=be70f76ecf3f465fb09d46916db16671, scope.project.id=a503020af843467a992a8df7f46d3fec, user.id=ac00bd52ce5943af943ce9ed56577c79) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:05.831201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a5f5d-e96b-45b1-ae7d-bbc250ce3efd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:05.832710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a5f5d-e96b-45b1-ae7d-bbc250ce3efd None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:05.843008 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 55/113] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:32:05 2022] GET /identity/v3/role_assignments?user.id=ac00bd52ce5943af943ce9ed56577c79&scope.project.id=a503020af843467a992a8df7f46d3fec&role.id=be70f76ecf3f465fb09d46916db16671 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:05.964157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad30de-bf69-4266-a6a6-5f2dd7cff310 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:05.975550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1109575-5e29-450d-bc98-4e078d8c7da5 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:05.976356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1109575-5e29-450d-bc98-4e078d8c7da5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:05.976855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1109575-5e29-450d-bc98-4e078d8c7da5 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:05.981103 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 56/114] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:05 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:05.986261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65fe99-5720-4ffd-afb5-ec9c542293b4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:05.986619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65fe99-5720-4ffd-afb5-ec9c542293b4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:05.986861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65fe99-5720-4ffd-afb5-ec9c542293b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:06.021844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0c65fe99-5720-4ffd-afb5-ec9c542293b4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:06.023079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0c65fe99-5720-4ffd-afb5-ec9c542293b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:06.059696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:06.084610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:06.148086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad30de-bf69-4266-a6a6-5f2dd7cff310 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:06.165879 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 57/115] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:06.167183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7293c-f4f3-428e-80f2-e2a48809369c None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:06.167533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7293c-f4f3-428e-80f2-e2a48809369c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:06.168692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7293c-f4f3-428e-80f2-e2a48809369c None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:06.170296 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 58/116] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:06.176734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba07860-6769-47d0-ac2c-b6720c244d3a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:06.177010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba07860-6769-47d0-ac2c-b6720c244d3a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:06.177292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba07860-6769-47d0-ac2c-b6720c244d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:06.203275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-dba07860-6769-47d0-ac2c-b6720c244d3a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:06.204463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-dba07860-6769-47d0-ac2c-b6720c244d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:06.220948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05030777-9d61-4069-a44c-55670aa45941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:06.221553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05030777-9d61-4069-a44c-55670aa45941 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:06.221839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05030777-9d61-4069-a44c-55670aa45941 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:06.222263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05030777-9d61-4069-a44c-55670aa45941 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:06.234665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05030777-9d61-4069-a44c-55670aa45941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:06.241202 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-05030777-9d61-4069-a44c-55670aa45941 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:06.241202 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:06.241202 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:06.241202 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:06.241202 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:06.241202 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:06.241202 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:06.241202 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:06.241202 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:06.241202 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:06.241202 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:06.241202 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:06.241202 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:06.241202 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:06.241202 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:06.241202 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:06.242074 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:06.242074 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:06.242074 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:06.242074 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:06.242074 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:06.242074 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:06.242074 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:06.242074 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:06.242074 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:06.242074 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:06.242074 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:06.242074 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:06.242074 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:06.242074 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:06.242074 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:06.242074 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:06.242898 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:06.242898 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:06.242898 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:06.242898 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:06.242898 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:06.242898 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:06.242898 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:06.242898 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:06.242898 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:06.242898 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:06.242898 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:06.242898 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:06.242898 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 59/117] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:06.243571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05030777-9d61-4069-a44c-55670aa45941 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:06.307800 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 59/118] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:06.308657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba07860-6769-47d0-ac2c-b6720c244d3a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:06.446596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05030777-9d61-4069-a44c-55670aa45941 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:06.464324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:06.544658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:06.547994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8224205-c275-4154-ae81-87a5cbc6f77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:06.548668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8224205-c275-4154-ae81-87a5cbc6f77c None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:06.548952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8224205-c275-4154-ae81-87a5cbc6f77c None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:06.549232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8224205-c275-4154-ae81-87a5cbc6f77c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:06.560379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba07860-6769-47d0-ac2c-b6720c244d3a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:06.579615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8224205-c275-4154-ae81-87a5cbc6f77c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:06.584930 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-f8224205-c275-4154-ae81-87a5cbc6f77c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:32:06.584930 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:06.584930 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:06.584930 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:06.584930 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:06.584930 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:06.584930 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:06.584930 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:06.584930 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:06.584930 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:06.584930 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:06.584930 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:06.584930 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:06.584930 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:06.584930 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:06.584930 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:06.585782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:06.585782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:06.585782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:06.585782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:06.585782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:06.585782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:06.585782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:06.585782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:06.585782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:06.585782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:06.585782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:06.585782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:06.585782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:06.585782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:06.585782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:06.585782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:06.586597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:06.586597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:06.586597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:06.586597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:06.586597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:06.586597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:06.586597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:06.586597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:06.586597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:06.586597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:06.586597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:32:06.586597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:06.586597 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 60/119] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:32:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:06.587252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8224205-c275-4154-ae81-87a5cbc6f77c None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:06.650371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4745d-f870-4b6e-9ab9-78b739a82005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:06.651015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4745d-f870-4b6e-9ab9-78b739a82005 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:06.651294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4745d-f870-4b6e-9ab9-78b739a82005 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:06.651566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4745d-f870-4b6e-9ab9-78b739a82005 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:06.652232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4745d-f870-4b6e-9ab9-78b739a82005 None admin] RBAC: Adding query filter params (name=member) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:06.652536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4745d-f870-4b6e-9ab9-78b739a82005 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:06.653912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4745d-f870-4b6e-9ab9-78b739a82005 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:06.684616 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 60/120] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:06.685584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4745d-f870-4b6e-9ab9-78b739a82005 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:06.717860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8224205-c275-4154-ae81-87a5cbc6f77c None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:06.753544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5daeeb-ff69-46ca-a76d-1aa8099502f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:06.754841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5daeeb-ff69-46ca-a76d-1aa8099502f0 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:06.755135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5daeeb-ff69-46ca-a76d-1aa8099502f0 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:06.755421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5daeeb-ff69-46ca-a76d-1aa8099502f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:06.766920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5daeeb-ff69-46ca-a76d-1aa8099502f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:06.772462 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-ca5daeeb-ff69-46ca-a76d-1aa8099502f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:06.772462 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:06.772462 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:06.772462 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:06.772462 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:06.772462 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:06.772462 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:06.772462 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:06.772462 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:06.772462 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:06.772462 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:06.772462 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:06.772462 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:06.772462 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:06.772462 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:06.772462 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:06.773297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:06.773297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:06.773297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:06.773297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:06.773297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:06.773297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:06.773297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:06.773297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:06.773297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:06.773297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:06.773297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:06.773297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:06.773297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:06.773297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:06.773297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:06.773297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:06.774053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:06.774053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:06.774053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:06.774053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:06.774053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:06.774053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:06.774053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:06.774053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:06.774053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:06.774053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:06.774053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:06.774053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:06.774053 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 61/121] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:06.775171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5daeeb-ff69-46ca-a76d-1aa8099502f0 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:06.828287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:06.909758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5daeeb-ff69-46ca-a76d-1aa8099502f0 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:06.969848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd7c0a-cd94-4974-8de7-15e87a8b76da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:06.970521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd7c0a-cd94-4974-8de7-15e87a8b76da None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:06.971379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd7c0a-cd94-4974-8de7-15e87a8b76da None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:06.971661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd7c0a-cd94-4974-8de7-15e87a8b76da None admin] PATH_INFO: `/v3/users/2e606e5ac9f84891835962715092f667` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:06.973866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd7c0a-cd94-4974-8de7-15e87a8b76da None admin] RBAC: Authorizing `identity:get_user(user_id=2e606e5ac9f84891835962715092f667)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:06.976428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd7c0a-cd94-4974-8de7-15e87a8b76da None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:06.982656 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 62/122] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:06 2022] GET /identity/v3/users/2e606e5ac9f84891835962715092f667 => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:06.983745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd7c0a-cd94-4974-8de7-15e87a8b76da None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:07.014743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4745d-f870-4b6e-9ab9-78b739a82005 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:07.106184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb41a7b-6260-406e-9ff6-d8127a665730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:07.108427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb41a7b-6260-406e-9ff6-d8127a665730 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:07.109189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb41a7b-6260-406e-9ff6-d8127a665730 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:07.111191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb41a7b-6260-406e-9ff6-d8127a665730 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:07.112129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb41a7b-6260-406e-9ff6-d8127a665730 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:07.112635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb41a7b-6260-406e-9ff6-d8127a665730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:07.114309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb41a7b-6260-406e-9ff6-d8127a665730 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:07.137219 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 61/123] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:07.138347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb41a7b-6260-406e-9ff6-d8127a665730 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:07.152948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:07.161217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd7c0a-cd94-4974-8de7-15e87a8b76da None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:07.182133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:07.213023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7249d0-2814-4c73-940c-cbe4d22431fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:07.213670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7249d0-2814-4c73-940c-cbe4d22431fc None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:07.213954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7249d0-2814-4c73-940c-cbe4d22431fc None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:07.214273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7249d0-2814-4c73-940c-cbe4d22431fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:07.228957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7249d0-2814-4c73-940c-cbe4d22431fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:07.234810 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-3e7249d0-2814-4c73-940c-cbe4d22431fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:07.234810 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:07.234810 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:07.234810 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:07.234810 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:07.234810 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:07.234810 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:07.234810 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:07.234810 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:07.234810 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:07.234810 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:07.234810 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:07.234810 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:07.234810 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:07.234810 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:07.234810 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:07.235668 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:07.235668 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:07.235668 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:07.235668 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:07.235668 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:07.235668 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:07.235668 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:07.235668 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:07.235668 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:07.235668 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:07.235668 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:07.235668 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:07.235668 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:07.235668 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:07.235668 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:07.235668 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:07.236435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:07.236435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:07.236435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:07.236435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:07.236435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:07.236435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:07.236435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:07.236435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:07.236435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:07.236435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:07.236435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:07.236435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:07.236435 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 63/124] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:32:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:07.237072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7249d0-2814-4c73-940c-cbe4d22431fc None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:07.307983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb41a7b-6260-406e-9ff6-d8127a665730 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:07.344108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74999cac-942b-4e21-976f-69d329de74c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:07.344882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74999cac-942b-4e21-976f-69d329de74c0 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:07.345164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74999cac-942b-4e21-976f-69d329de74c0 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:07.345438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74999cac-942b-4e21-976f-69d329de74c0 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:07.346153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74999cac-942b-4e21-976f-69d329de74c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:07.346629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74999cac-942b-4e21-976f-69d329de74c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:07.348690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74999cac-942b-4e21-976f-69d329de74c0 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:07.354510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7249d0-2814-4c73-940c-cbe4d22431fc None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:07.357680 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 62/125] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:07.358436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74999cac-942b-4e21-976f-69d329de74c0 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:07.392666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05878495-b4e4-4479-b9ea-e051c8d2c54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:07.393639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05878495-b4e4-4479-b9ea-e051c8d2c54a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:07.394078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05878495-b4e4-4479-b9ea-e051c8d2c54a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:07.394532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05878495-b4e4-4479-b9ea-e051c8d2c54a None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:07.395686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05878495-b4e4-4479-b9ea-e051c8d2c54a None admin] RBAC: Authorizing `identity:get_project(project_id=a503020af843467a992a8df7f46d3fec)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:07.397126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05878495-b4e4-4479-b9ea-e051c8d2c54a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:07.403614 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 64/126] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:07 2022] GET /identity/v3/projects/a503020af843467a992a8df7f46d3fec => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:07.404504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05878495-b4e4-4479-b9ea-e051c8d2c54a None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:07.433427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:07.450902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:07.474779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74999cac-942b-4e21-976f-69d329de74c0 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:07.517140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071fa56-a9e3-4d2b-9ae4-7118b11c623e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:07.518502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071fa56-a9e3-4d2b-9ae4-7118b11c623e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:07.518808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071fa56-a9e3-4d2b-9ae4-7118b11c623e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:07.519089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071fa56-a9e3-4d2b-9ae4-7118b11c623e None admin] PATH_INFO: `/v3/users/a9033d8c4ef14e3eaead57cd2b91f40f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:07.520853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071fa56-a9e3-4d2b-9ae4-7118b11c623e None admin] RBAC: Authorizing `identity:get_user(user_id=a9033d8c4ef14e3eaead57cd2b91f40f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:07.521064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05878495-b4e4-4479-b9ea-e051c8d2c54a None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:07.522369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071fa56-a9e3-4d2b-9ae4-7118b11c623e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:07.525498 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 63/127] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:07 2022] GET /identity/v3/users/a9033d8c4ef14e3eaead57cd2b91f40f => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:07.526293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071fa56-a9e3-4d2b-9ae4-7118b11c623e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:07.566755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03648c-8608-4a59-a016-44e9619ccf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:07.567776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03648c-8608-4a59-a016-44e9619ccf70 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:07.568158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03648c-8608-4a59-a016-44e9619ccf70 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:07.568500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03648c-8608-4a59-a016-44e9619ccf70 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:07.569252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03648c-8608-4a59-a016-44e9619ccf70 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:07.569651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03648c-8608-4a59-a016-44e9619ccf70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:07.571133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03648c-8608-4a59-a016-44e9619ccf70 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:07.582314 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 65/128] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:32:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:07.583736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03648c-8608-4a59-a016-44e9619ccf70 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:07.670828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071fa56-a9e3-4d2b-9ae4-7118b11c623e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:07.683379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:07.708924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03648c-8608-4a59-a016-44e9619ccf70 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:07.718366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203892ab-1dbf-4d38-82c6-01f742200a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:07.719645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203892ab-1dbf-4d38-82c6-01f742200a33 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:07.720192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203892ab-1dbf-4d38-82c6-01f742200a33 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:07.720646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203892ab-1dbf-4d38-82c6-01f742200a33 None admin] PATH_INFO: `/v3/users/2e606e5ac9f84891835962715092f667` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:07.721984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203892ab-1dbf-4d38-82c6-01f742200a33 None admin] RBAC: Authorizing `identity:get_user(user_id=2e606e5ac9f84891835962715092f667)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:07.723685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203892ab-1dbf-4d38-82c6-01f742200a33 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:07.725793 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 64/129] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:07 2022] GET /identity/v3/users/2e606e5ac9f84891835962715092f667 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:07.726943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203892ab-1dbf-4d38-82c6-01f742200a33 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:07.747107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:07.752819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccba15-9c05-42b7-a3a2-95a52b8ac81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:07.753525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccba15-9c05-42b7-a3a2-95a52b8ac81b None admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:07.753849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccba15-9c05-42b7-a3a2-95a52b8ac81b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:07.754134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccba15-9c05-42b7-a3a2-95a52b8ac81b None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec/users/2e606e5ac9f84891835962715092f667/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:07.756567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccba15-9c05-42b7-a3a2-95a52b8ac81b None admin] RBAC: Authorizing `identity:create_grant(project_id=a503020af843467a992a8df7f46d3fec, user_id=2e606e5ac9f84891835962715092f667, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:07.758069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccba15-9c05-42b7-a3a2-95a52b8ac81b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:07.781847 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 66/130] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:32:07 2022] PUT /identity/v3/projects/a503020af843467a992a8df7f46d3fec/users/2e606e5ac9f84891835962715092f667/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:32:07.781847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccba15-9c05-42b7-a3a2-95a52b8ac81b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:07.884078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203892ab-1dbf-4d38-82c6-01f742200a33 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:07.912173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccba15-9c05-42b7-a3a2-95a52b8ac81b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:07.963505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9450407-a91b-4875-9ae5-40d8314d793f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:07.964166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9450407-a91b-4875-9ae5-40d8314d793f None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:07.964451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9450407-a91b-4875-9ae5-40d8314d793f None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:07.964734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9450407-a91b-4875-9ae5-40d8314d793f None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:07.969666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9450407-a91b-4875-9ae5-40d8314d793f None admin] RBAC: Authorizing `identity:get_project(project_id=a503020af843467a992a8df7f46d3fec)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:07.971199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9450407-a91b-4875-9ae5-40d8314d793f None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:07.979585 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 65/131] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:07 2022] GET /identity/v3/projects/a503020af843467a992a8df7f46d3fec => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:07.980518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9450407-a91b-4875-9ae5-40d8314d793f None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:07.991410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420994f-9155-42cd-8245-dd42c3e7b0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:07.992068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420994f-9155-42cd-8245-dd42c3e7b0a3 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:07.992351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420994f-9155-42cd-8245-dd42c3e7b0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:07.992633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420994f-9155-42cd-8245-dd42c3e7b0a3 None admin] PATH_INFO: `/v3/users/2e606e5ac9f84891835962715092f667` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:07.993846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420994f-9155-42cd-8245-dd42c3e7b0a3 None admin] RBAC: Authorizing `identity:get_user(user_id=2e606e5ac9f84891835962715092f667)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:07.995753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420994f-9155-42cd-8245-dd42c3e7b0a3 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:07.997661 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 67/132] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:07 2022] GET /identity/v3/users/2e606e5ac9f84891835962715092f667 => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:07.998581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420994f-9155-42cd-8245-dd42c3e7b0a3 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:08.018608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:08.128136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420994f-9155-42cd-8245-dd42c3e7b0a3 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:08.146611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:08.169399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9450407-a91b-4875-9ae5-40d8314d793f None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:08.173255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25333e63-63e8-4e31-89b9-d2bfa7030307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:08.173934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25333e63-63e8-4e31-89b9-d2bfa7030307 None admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:08.174220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25333e63-63e8-4e31-89b9-d2bfa7030307 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:08.174528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25333e63-63e8-4e31-89b9-d2bfa7030307 None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec/users/a9033d8c4ef14e3eaead57cd2b91f40f/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:08.177795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25333e63-63e8-4e31-89b9-d2bfa7030307 None admin] RBAC: Authorizing `identity:create_grant(project_id=a503020af843467a992a8df7f46d3fec, user_id=a9033d8c4ef14e3eaead57cd2b91f40f, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:08.179881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25333e63-63e8-4e31-89b9-d2bfa7030307 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:08.201137 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 68/133] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:32:07 2022] PUT /identity/v3/projects/a503020af843467a992a8df7f46d3fec/users/a9033d8c4ef14e3eaead57cd2b91f40f/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:32:08.202243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25333e63-63e8-4e31-89b9-d2bfa7030307 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:08.220769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799c60b-ed5f-4d1c-a8d7-daaa01f8222d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:08.222258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799c60b-ed5f-4d1c-a8d7-daaa01f8222d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:08.223175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799c60b-ed5f-4d1c-a8d7-daaa01f8222d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:08.223986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799c60b-ed5f-4d1c-a8d7-daaa01f8222d None admin] PATH_INFO: `/v3/projects/213ed96d79ee4169824ebf710c0f9d41` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:08.225375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799c60b-ed5f-4d1c-a8d7-daaa01f8222d None admin] RBAC: Authorizing `identity:get_project(project_id=213ed96d79ee4169824ebf710c0f9d41)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:08.227799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799c60b-ed5f-4d1c-a8d7-daaa01f8222d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:08.230333 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 66/134] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:07 2022] GET /identity/v3/projects/213ed96d79ee4169824ebf710c0f9d41 => generated 312 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:08.236077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799c60b-ed5f-4d1c-a8d7-daaa01f8222d None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:08.370724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:08.402117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25333e63-63e8-4e31-89b9-d2bfa7030307 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:08.444743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb260f4-d66c-4d48-bd32-2d0fea188a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:08.445791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb260f4-d66c-4d48-bd32-2d0fea188a90 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:08.446290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb260f4-d66c-4d48-bd32-2d0fea188a90 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:08.447045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb260f4-d66c-4d48-bd32-2d0fea188a90 None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:08.449227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb260f4-d66c-4d48-bd32-2d0fea188a90 None admin] RBAC: Authorizing `identity:get_project(project_id=a503020af843467a992a8df7f46d3fec)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:08.451140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb260f4-d66c-4d48-bd32-2d0fea188a90 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:08.458165 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 69/135] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:08 2022] GET /identity/v3/projects/a503020af843467a992a8df7f46d3fec => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:08.459687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925672b9-204e-43bc-9129-3b82668597b7 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:08.460808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925672b9-204e-43bc-9129-3b82668597b7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:08.462115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925672b9-204e-43bc-9129-3b82668597b7 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:08.464277 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 70/136] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:08.467232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799c60b-ed5f-4d1c-a8d7-daaa01f8222d None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:08.467659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925672b9-204e-43bc-9129-3b82668597b7 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:08.513949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2216d-ed35-4518-9ffd-8b2e3f0bd605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:08.514587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2216d-ed35-4518-9ffd-8b2e3f0bd605 None admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:08.514871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2216d-ed35-4518-9ffd-8b2e3f0bd605 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:08.515142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2216d-ed35-4518-9ffd-8b2e3f0bd605 None admin] PATH_INFO: `/v3/projects/213ed96d79ee4169824ebf710c0f9d41/users/2e606e5ac9f84891835962715092f667/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:08.518061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2216d-ed35-4518-9ffd-8b2e3f0bd605 None admin] RBAC: Authorizing `identity:create_grant(project_id=213ed96d79ee4169824ebf710c0f9d41, user_id=2e606e5ac9f84891835962715092f667, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:08.519578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2216d-ed35-4518-9ffd-8b2e3f0bd605 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:08.558519 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 67/137] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:32:08 2022] PUT /identity/v3/projects/213ed96d79ee4169824ebf710c0f9d41/users/2e606e5ac9f84891835962715092f667/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:32:08.559636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d48f95-37f3-402c-aaf3-5d53a965e9c8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:08.559941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d48f95-37f3-402c-aaf3-5d53a965e9c8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:08.560198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d48f95-37f3-402c-aaf3-5d53a965e9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:08.602520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925672b9-204e-43bc-9129-3b82668597b7 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:08.619923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-96d48f95-37f3-402c-aaf3-5d53a965e9c8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:08.620868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-96d48f95-37f3-402c-aaf3-5d53a965e9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:08.635455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:08.652161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d2d2b-609b-43a4-9b7c-74ed35fe58c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:08.653223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d2d2b-609b-43a4-9b7c-74ed35fe58c2 None admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:08.653732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d2d2b-609b-43a4-9b7c-74ed35fe58c2 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:08.654207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d2d2b-609b-43a4-9b7c-74ed35fe58c2 None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec/users/2e606e5ac9f84891835962715092f667/roles/8b7b6e0028b04be8912abda33d3ef729` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:08.656853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d2d2b-609b-43a4-9b7c-74ed35fe58c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a503020af843467a992a8df7f46d3fec, user_id=2e606e5ac9f84891835962715092f667, role_id=8b7b6e0028b04be8912abda33d3ef729)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:08.660262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d2d2b-609b-43a4-9b7c-74ed35fe58c2 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:08.687125 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 71/138] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:32:08 2022] PUT /identity/v3/projects/a503020af843467a992a8df7f46d3fec/users/2e606e5ac9f84891835962715092f667/roles/8b7b6e0028b04be8912abda33d3ef729 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:32:08.770752 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 68/139] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:08.909363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22648d-fb64-45d8-9c52-4cebbade1f5d None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:08.911138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22648d-fb64-45d8-9c52-4cebbade1f5d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:08.911630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22648d-fb64-45d8-9c52-4cebbade1f5d None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:08.913322 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 72/140] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:08.919080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55998b-12d7-4742-8328-be0f170353f7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:08.919370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55998b-12d7-4742-8328-be0f170353f7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:08.919648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55998b-12d7-4742-8328-be0f170353f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:08.947838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5e55998b-12d7-4742-8328-be0f170353f7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:08.949059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5e55998b-12d7-4742-8328-be0f170353f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:09.035840 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 69/141] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:09.285058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22648d-fb64-45d8-9c52-4cebbade1f5d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:09.288558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:09.425205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22648d-fb64-45d8-9c52-4cebbade1f5d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:09.471782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86823c19-3edf-4368-99e2-8a6d4f709b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:09.472963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86823c19-3edf-4368-99e2-8a6d4f709b87 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:09.473249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86823c19-3edf-4368-99e2-8a6d4f709b87 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:09.473538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86823c19-3edf-4368-99e2-8a6d4f709b87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:09.474695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86823c19-3edf-4368-99e2-8a6d4f709b87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:09.476167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86823c19-3edf-4368-99e2-8a6d4f709b87 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:09.477931 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 73/142] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:32:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:09.483323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55998b-12d7-4742-8328-be0f170353f7 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:09.587897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:09.631647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55998b-12d7-4742-8328-be0f170353f7 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:09.663562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86823c19-3edf-4368-99e2-8a6d4f709b87 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:09.695380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a03130-6d11-4e74-bfff-8b1797cb9028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:09.696013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a03130-6d11-4e74-bfff-8b1797cb9028 None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:09.696853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a03130-6d11-4e74-bfff-8b1797cb9028 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:09.696853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a03130-6d11-4e74-bfff-8b1797cb9028 None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:09.697524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a03130-6d11-4e74-bfff-8b1797cb9028 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:09.699801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a03130-6d11-4e74-bfff-8b1797cb9028 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:09.764547 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 70/143] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Oct 1 20:32:09 2022] POST /identity/v3/users => generated 283 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:32:09.794965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:09.839451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86823c19-3edf-4368-99e2-8a6d4f709b87 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:09.897913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d3473-1046-460f-a5a4-7429cfba31eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:09.898596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d3473-1046-460f-a5a4-7429cfba31eb None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:09.898891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d3473-1046-460f-a5a4-7429cfba31eb None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:09.899147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d3473-1046-460f-a5a4-7429cfba31eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:09.919584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d3473-1046-460f-a5a4-7429cfba31eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:09.925300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-f17d3473-1046-460f-a5a4-7429cfba31eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 01 20:32:09.925300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:09.925300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:09.925300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:09.925300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:09.925300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:09.925300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:09.925300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:09.925300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:09.925300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:09.925300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:09.925300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:09.925300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:09.925300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:09.925300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:09.925300 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:09.926225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:09.926225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:09.926225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:09.926225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:09.926225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:09.926225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:09.926225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:09.926225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:09.926225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:09.926225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:09.926225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:09.926225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:09.926225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:09.926225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:09.926225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:09.926225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:09.927059 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:09.927059 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:09.927059 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:09.927059 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:09.927059 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:09.927059 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:09.927059 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:09.927059 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:09.927059 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:09.927059 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:09.927059 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 01 20:32:09.927059 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:09.927059 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 74/144] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:09.932711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a03130-6d11-4e74-bfff-8b1797cb9028 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:10.096125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a03130-6d11-4e74-bfff-8b1797cb9028 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:10.137082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:10.146303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f4f48-0ca1-48eb-915b-76b27ad15f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:10.147094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f4f48-0ca1-48eb-915b-76b27ad15f94 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:10.147416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f4f48-0ca1-48eb-915b-76b27ad15f94 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:10.147694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f4f48-0ca1-48eb-915b-76b27ad15f94 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:10.148390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f4f48-0ca1-48eb-915b-76b27ad15f94 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:10.148698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f4f48-0ca1-48eb-915b-76b27ad15f94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:10.150147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f4f48-0ca1-48eb-915b-76b27ad15f94 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:10.160677 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 71/145] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:10.171822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d3473-1046-460f-a5a4-7429cfba31eb None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:10.306296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d3473-1046-460f-a5a4-7429cfba31eb None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:10.347427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:10.353328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5aed7-43bf-4822-a993-7d4ae382fa08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:10.354158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5aed7-43bf-4822-a993-7d4ae382fa08 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:10.354646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5aed7-43bf-4822-a993-7d4ae382fa08 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:10.354951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5aed7-43bf-4822-a993-7d4ae382fa08 None admin] PATH_INFO: `/v3/users/ac00bd52ce5943af943ce9ed56577c79` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:10.356105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5aed7-43bf-4822-a993-7d4ae382fa08 None admin] RBAC: Authorizing `identity:get_user(user_id=ac00bd52ce5943af943ce9ed56577c79)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:10.357595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5aed7-43bf-4822-a993-7d4ae382fa08 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:10.359210 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 75/146] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:10 2022] GET /identity/v3/users/ac00bd52ce5943af943ce9ed56577c79 => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:10.364755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f4f48-0ca1-48eb-915b-76b27ad15f94 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:10.510214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f4f48-0ca1-48eb-915b-76b27ad15f94 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:10.558801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a2858-63a5-42b9-9f34-dc2cfc68791e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:10.559448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a2858-63a5-42b9-9f34-dc2cfc68791e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:10.559736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a2858-63a5-42b9-9f34-dc2cfc68791e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:10.560007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a2858-63a5-42b9-9f34-dc2cfc68791e None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:10.561707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a2858-63a5-42b9-9f34-dc2cfc68791e None admin] RBAC: Authorizing `identity:get_project(project_id=a503020af843467a992a8df7f46d3fec)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:10.563208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a2858-63a5-42b9-9f34-dc2cfc68791e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:10.565850 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 72/147] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:10 2022] GET /identity/v3/projects/a503020af843467a992a8df7f46d3fec => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:10.570726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5aed7-43bf-4822-a993-7d4ae382fa08 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:10.719518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5aed7-43bf-4822-a993-7d4ae382fa08 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:10.763414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4029e552-0cb2-4067-822f-96be6349be9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:10.764184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029e552-0cb2-4067-822f-96be6349be9a None admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:10.764463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029e552-0cb2-4067-822f-96be6349be9a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:10.764733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029e552-0cb2-4067-822f-96be6349be9a None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec/users/ac00bd52ce5943af943ce9ed56577c79/roles/be70f76ecf3f465fb09d46916db16671` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:10.767049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4029e552-0cb2-4067-822f-96be6349be9a None admin] RBAC: Authorizing `identity:create_grant(project_id=a503020af843467a992a8df7f46d3fec, user_id=ac00bd52ce5943af943ce9ed56577c79, role_id=be70f76ecf3f465fb09d46916db16671)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:10.768548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4029e552-0cb2-4067-822f-96be6349be9a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:10.793334 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 76/148] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:32:10 2022] PUT /identity/v3/projects/a503020af843467a992a8df7f46d3fec/users/ac00bd52ce5943af943ce9ed56577c79/roles/be70f76ecf3f465fb09d46916db16671 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:32:11.265384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65dacd-5640-4994-ab5f-c27c1caa74e5 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:11.265910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65dacd-5640-4994-ab5f-c27c1caa74e5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:11.265910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65dacd-5640-4994-ab5f-c27c1caa74e5 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:11.267731 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 73/149] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:11.274232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5ba02-c284-4352-b52d-053f258cd923 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:11.274768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5ba02-c284-4352-b52d-053f258cd923 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:11.274842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5ba02-c284-4352-b52d-053f258cd923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:11.297368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a5e5ba02-c284-4352-b52d-053f258cd923 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:11.298531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a5e5ba02-c284-4352-b52d-053f258cd923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:11.388369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:11.408766 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 77/150] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:11.855647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04af81a-cb34-4812-ad76-2aadb1dec5bc None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:11.856014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04af81a-cb34-4812-ad76-2aadb1dec5bc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:11.856198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04af81a-cb34-4812-ad76-2aadb1dec5bc None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:11.857868 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 74/151] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:11.864661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cef67-1633-4450-ad4f-789242182505 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:11.864951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cef67-1633-4450-ad4f-789242182505 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:11.865228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cef67-1633-4450-ad4f-789242182505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:11.887588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-707cef67-1633-4450-ad4f-789242182505 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:11.888849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-707cef67-1633-4450-ad4f-789242182505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:11.967755 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 78/152] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:12.035976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04af81a-cb34-4812-ad76-2aadb1dec5bc None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:12.112678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:12.160930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04af81a-cb34-4812-ad76-2aadb1dec5bc None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:12.198277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a623267-06e7-4b91-b4ae-eeb9bddce46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:12.198960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a623267-06e7-4b91-b4ae-eeb9bddce46a None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:12.199240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a623267-06e7-4b91-b4ae-eeb9bddce46a None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:12.199512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a623267-06e7-4b91-b4ae-eeb9bddce46a None admin] PATH_INFO: `/v3/roles/member` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:12.213764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a623267-06e7-4b91-b4ae-eeb9bddce46a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:12.219215 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-9a623267-06e7-4b91-b4ae-eeb9bddce46a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:12.219215 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:12.219215 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:12.219215 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:12.219215 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:12.219215 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:12.219215 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:12.219215 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:12.219215 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:12.219215 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:12.219215 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:12.219215 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:12.219215 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:12.219215 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:12.219215 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:12.219215 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:12.220049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:12.220049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:12.220049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:12.220049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:12.220049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:12.220049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:12.220049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:12.220049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:12.220049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:12.220049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:12.220049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:12.220049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:12.220049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:12.220049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:12.220049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:12.220049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:12.220822 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:12.220822 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:12.220822 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:12.220822 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:12.220822 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:12.220822 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:12.220822 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:12.220822 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:12.220822 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:12.220822 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:12.220822 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:12.220822 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:32:12.220822 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 75/153] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:12.226747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cef67-1633-4450-ad4f-789242182505 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:12.345422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cef67-1633-4450-ad4f-789242182505 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:12.363496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:12.385747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf61ee-bae9-4ed0-9e3a-b3da6c22ed39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:12.386367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf61ee-bae9-4ed0-9e3a-b3da6c22ed39 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:12.386679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf61ee-bae9-4ed0-9e3a-b3da6c22ed39 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:12.386954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf61ee-bae9-4ed0-9e3a-b3da6c22ed39 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:12.387691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf61ee-bae9-4ed0-9e3a-b3da6c22ed39 None admin] RBAC: Adding query filter params (name=member) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:12.388004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf61ee-bae9-4ed0-9e3a-b3da6c22ed39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:12.389380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf61ee-bae9-4ed0-9e3a-b3da6c22ed39 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:12.402981 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 79/154] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:12.408939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a623267-06e7-4b91-b4ae-eeb9bddce46a None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:12.529778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a623267-06e7-4b91-b4ae-eeb9bddce46a None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:12.549487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:12.567872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4201a41-ffc8-4d08-9d29-d257fd60f71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:12.568515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4201a41-ffc8-4d08-9d29-d257fd60f71a None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:12.568795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4201a41-ffc8-4d08-9d29-d257fd60f71a None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:12.569066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4201a41-ffc8-4d08-9d29-d257fd60f71a None admin] PATH_INFO: `/v3/users/2e606e5ac9f84891835962715092f667` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:12.573214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4201a41-ffc8-4d08-9d29-d257fd60f71a None admin] RBAC: Authorizing `identity:get_user(user_id=2e606e5ac9f84891835962715092f667)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:12.574778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4201a41-ffc8-4d08-9d29-d257fd60f71a None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:12.576381 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 76/155] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:12 2022] GET /identity/v3/users/2e606e5ac9f84891835962715092f667 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:12.582204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf61ee-bae9-4ed0-9e3a-b3da6c22ed39 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:12.636931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4201a41-ffc8-4d08-9d29-d257fd60f71a None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:12.766787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf61ee-bae9-4ed0-9e3a-b3da6c22ed39 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:12.828826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfae7b8-ed31-4ed6-aa65-0babf89f327d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:12.829451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfae7b8-ed31-4ed6-aa65-0babf89f327d None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:12.829731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfae7b8-ed31-4ed6-aa65-0babf89f327d None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:12.830009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfae7b8-ed31-4ed6-aa65-0babf89f327d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:12.837698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4201a41-ffc8-4d08-9d29-d257fd60f71a None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:12.902771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b0e44-9039-4219-a097-b3ba5bedf873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:12.903379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0e44-9039-4219-a097-b3ba5bedf873 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:12.903812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0e44-9039-4219-a097-b3ba5bedf873 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:12.904218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0e44-9039-4219-a097-b3ba5bedf873 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:12.924251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b0e44-9039-4219-a097-b3ba5bedf873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:12.930850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:12.931181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-b30b0e44-9039-4219-a097-b3ba5bedf873 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:32:12.931181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:12.931181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:12.931181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:12.931181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:12.931181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:12.931181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:12.931181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:12.931181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:12.931181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:12.931181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:12.931181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:12.931181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:12.931181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:12.931181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:12.931181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:12.932280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:12.932280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:12.932280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:12.932280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:12.932280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:12.932280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:12.932280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:12.932280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:12.932280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:12.932280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:12.932280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:12.932280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:12.932280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:12.932280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:12.932280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:12.932280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:12.933236 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:12.933236 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:12.933236 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:12.933236 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:12.933236 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:12.933236 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:12.933236 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:12.933236 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:12.933236 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:12.933236 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:12.933236 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:32:12.933236 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:32:12.933236 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 77/156] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:32:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:32:12.938060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b0e44-9039-4219-a097-b3ba5bedf873 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:12.962724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfae7b8-ed31-4ed6-aa65-0babf89f327d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:12.964323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfae7b8-ed31-4ed6-aa65-0babf89f327d None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:13.035412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:13.165312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b0e44-9039-4219-a097-b3ba5bedf873 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:13.208436 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 80/157] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:13.209631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc239ace-81f6-48ba-8617-1f77c3a42804 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:13.209984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc239ace-81f6-48ba-8617-1f77c3a42804 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:13.210183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc239ace-81f6-48ba-8617-1f77c3a42804 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:13.215500 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 81/158] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:13.216264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc239ace-81f6-48ba-8617-1f77c3a42804 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:13.248157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b4e56-466e-4282-b938-ab448c8505ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:13.248780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b4e56-466e-4282-b938-ab448c8505ab None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:13.249061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b4e56-466e-4282-b938-ab448c8505ab None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:13.249335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b4e56-466e-4282-b938-ab448c8505ab None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:13.250007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b4e56-466e-4282-b938-ab448c8505ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:13.250312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b4e56-466e-4282-b938-ab448c8505ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:13.251726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b4e56-466e-4282-b938-ab448c8505ab None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:13.269877 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 78/159] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:13.270946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4bd56-250a-4290-aa6b-3cd6fb1d4b3d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:13.271231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4bd56-250a-4290-aa6b-3cd6fb1d4b3d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:13.271510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4bd56-250a-4290-aa6b-3cd6fb1d4b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:13.311527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b9f4bd56-250a-4290-aa6b-3cd6fb1d4b3d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:13.312752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b9f4bd56-250a-4290-aa6b-3cd6fb1d4b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:13.402851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:13.425753 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 79/160] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:13.426543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4bd56-250a-4290-aa6b-3cd6fb1d4b3d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:13.449632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc239ace-81f6-48ba-8617-1f77c3a42804 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:13.510848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:13.519557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb01ae-358f-4ae8-809d-1568b264649f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:13.520322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb01ae-358f-4ae8-809d-1568b264649f None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:13.522532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb01ae-358f-4ae8-809d-1568b264649f None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:13.522532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb01ae-358f-4ae8-809d-1568b264649f None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:13.522532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb01ae-358f-4ae8-809d-1568b264649f None admin] RBAC: Authorizing `identity:get_project(project_id=a503020af843467a992a8df7f46d3fec)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:13.526689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb01ae-358f-4ae8-809d-1568b264649f None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:13.528604 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 82/161] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:13 2022] GET /identity/v3/projects/a503020af843467a992a8df7f46d3fec => generated 298 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:13.529874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c66be-d642-4d47-a077-b347972662cc None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:13.530255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c66be-d642-4d47-a077-b347972662cc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:13.530710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c66be-d642-4d47-a077-b347972662cc None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:13.532013 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 83/162] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:13.538240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c66be-d642-4d47-a077-b347972662cc None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:13.651993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4bd56-250a-4290-aa6b-3cd6fb1d4b3d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:13.718592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210c32a-8f52-4407-afb2-6e46f6c6d6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:13.719581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210c32a-8f52-4407-afb2-6e46f6c6d6d2 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:13.720111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210c32a-8f52-4407-afb2-6e46f6c6d6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:13.720577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210c32a-8f52-4407-afb2-6e46f6c6d6d2 None admin] PATH_INFO: `/v3/users/a9033d8c4ef14e3eaead57cd2b91f40f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:13.721962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210c32a-8f52-4407-afb2-6e46f6c6d6d2 None admin] RBAC: Authorizing `identity:get_user(user_id=a9033d8c4ef14e3eaead57cd2b91f40f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:13.723716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210c32a-8f52-4407-afb2-6e46f6c6d6d2 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:13.730515 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 80/163] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:13 2022] GET /identity/v3/users/a9033d8c4ef14e3eaead57cd2b91f40f => generated 247 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:13.731809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25de98d-ecd5-4552-88e7-dfca703b88a4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:13.732097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25de98d-ecd5-4552-88e7-dfca703b88a4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:13.732367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25de98d-ecd5-4552-88e7-dfca703b88a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:13.733510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c66be-d642-4d47-a077-b347972662cc None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:13.767582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d25de98d-ecd5-4552-88e7-dfca703b88a4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:13.768790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d25de98d-ecd5-4552-88e7-dfca703b88a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:13.803179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf657b4-5b66-4f6e-a24f-45d0d5a3dc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:13.803873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf657b4-5b66-4f6e-a24f-45d0d5a3dc3f None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:13.804171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf657b4-5b66-4f6e-a24f-45d0d5a3dc3f None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:13.804443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf657b4-5b66-4f6e-a24f-45d0d5a3dc3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:13.805183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf657b4-5b66-4f6e-a24f-45d0d5a3dc3f None admin] RBAC: Adding query filter params (role.id=17f3aeacc5034f3891cc6556da651c1b, scope.project.id=a503020af843467a992a8df7f46d3fec, user.id=2e606e5ac9f84891835962715092f667) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:13.805547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf657b4-5b66-4f6e-a24f-45d0d5a3dc3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:13.807075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf657b4-5b66-4f6e-a24f-45d0d5a3dc3f None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:13.816971 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 84/164] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:32:13 2022] GET /identity/v3/role_assignments?user.id=2e606e5ac9f84891835962715092f667&scope.project.id=a503020af843467a992a8df7f46d3fec&role.id=17f3aeacc5034f3891cc6556da651c1b => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:13.817921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf657b4-5b66-4f6e-a24f-45d0d5a3dc3f None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:13.846877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:13.961170 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 81/165] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:14.013445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf657b4-5b66-4f6e-a24f-45d0d5a3dc3f None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:14.079645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e0962-04ee-454c-9e94-8ea50d76ab00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:14.080399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e0962-04ee-454c-9e94-8ea50d76ab00 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:14.080689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e0962-04ee-454c-9e94-8ea50d76ab00 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:14.081026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e0962-04ee-454c-9e94-8ea50d76ab00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:14.163538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e0962-04ee-454c-9e94-8ea50d76ab00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:14.165031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e0962-04ee-454c-9e94-8ea50d76ab00 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:14.182294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:14.279428 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 85/166] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:14.285259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25de98d-ecd5-4552-88e7-dfca703b88a4 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:14.296169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e0962-04ee-454c-9e94-8ea50d76ab00 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:14.360249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:14.474284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e0962-04ee-454c-9e94-8ea50d76ab00 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:14.498650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:14.536825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25de98d-ecd5-4552-88e7-dfca703b88a4 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:14.544140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cfbcea-ce85-445e-8474-479aba689fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:14.544788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfbcea-ce85-445e-8474-479aba689fce None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:14.545072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfbcea-ce85-445e-8474-479aba689fce None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:14.545346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfbcea-ce85-445e-8474-479aba689fce None admin] PATH_INFO: `/v3/roles/member` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:14.565908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cfbcea-ce85-445e-8474-479aba689fce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:14.572030 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-05cfbcea-ce85-445e-8474-479aba689fce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:14.572030 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:14.572030 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:14.572030 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:14.572030 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:14.572030 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:14.572030 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:14.572030 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:14.572030 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:14.572030 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:14.572030 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:14.572030 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:14.572030 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:14.572030 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:14.572030 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:14.572030 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:14.573115 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:14.573115 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:14.573115 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:14.573115 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:14.573115 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:14.573115 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:14.573115 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:14.573115 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:14.573115 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:14.573115 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:14.573115 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:14.573115 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:14.573115 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:14.573115 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:14.573115 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:14.573115 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:14.573907 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:14.573907 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:14.573907 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:14.573907 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:14.573907 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:14.573907 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:14.573907 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:14.573907 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:14.573907 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:14.573907 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:14.573907 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:14.573907 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:14.582521 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 86/167] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:14.586504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cfbcea-ce85-445e-8474-479aba689fce None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:14.631242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ec2f1-2644-4414-89df-4a38e3c43be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:14.631892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ec2f1-2644-4414-89df-4a38e3c43be6 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:14.632167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ec2f1-2644-4414-89df-4a38e3c43be6 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:14.632436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ec2f1-2644-4414-89df-4a38e3c43be6 None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:14.633899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ec2f1-2644-4414-89df-4a38e3c43be6 None admin] RBAC: Authorizing `identity:get_project(project_id=a503020af843467a992a8df7f46d3fec)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:14.635368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ec2f1-2644-4414-89df-4a38e3c43be6 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:14.637164 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 82/168] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:14 2022] GET /identity/v3/projects/a503020af843467a992a8df7f46d3fec => generated 298 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:14.642694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ec2f1-2644-4414-89df-4a38e3c43be6 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:14.856392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ec2f1-2644-4414-89df-4a38e3c43be6 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:14.868086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:14.875346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cfbcea-ce85-445e-8474-479aba689fce None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:14.979245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f8520-9d37-4f80-9609-06df45b9a3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:14.979874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f8520-9d37-4f80-9609-06df45b9a3be None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:14.980353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f8520-9d37-4f80-9609-06df45b9a3be None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:14.980629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f8520-9d37-4f80-9609-06df45b9a3be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:14.981369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f8520-9d37-4f80-9609-06df45b9a3be None admin] RBAC: Adding query filter params (role.id=1c50896511054a5f994bee3d99fa0619, scope.project.id=a503020af843467a992a8df7f46d3fec, user.id=a9033d8c4ef14e3eaead57cd2b91f40f) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:14.981674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f8520-9d37-4f80-9609-06df45b9a3be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:14.983177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f8520-9d37-4f80-9609-06df45b9a3be None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:15.005007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eef0ad-f913-4e02-8a68-e60804d37388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:15.005702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eef0ad-f913-4e02-8a68-e60804d37388 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:15.012624 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 83/169] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:32:14 2022] GET /identity/v3/role_assignments?user.id=a9033d8c4ef14e3eaead57cd2b91f40f&scope.project.id=a503020af843467a992a8df7f46d3fec&role.id=1c50896511054a5f994bee3d99fa0619 => generated 608 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:15.013495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f8520-9d37-4f80-9609-06df45b9a3be None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:15.034702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eef0ad-f913-4e02-8a68-e60804d37388 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:15.037081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eef0ad-f913-4e02-8a68-e60804d37388 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:15.037975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eef0ad-f913-4e02-8a68-e60804d37388 None admin] RBAC: Adding query filter params (name=member) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:15.038309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eef0ad-f913-4e02-8a68-e60804d37388 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:15.049017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eef0ad-f913-4e02-8a68-e60804d37388 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:15.072250 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 87/170] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 486 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:15.077978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eef0ad-f913-4e02-8a68-e60804d37388 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:15.161611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:15.225466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f8520-9d37-4f80-9609-06df45b9a3be None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:15.293129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74115a-129d-4916-b061-301f677271ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:15.293773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74115a-129d-4916-b061-301f677271ca None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:15.294050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74115a-129d-4916-b061-301f677271ca None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:15.294319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74115a-129d-4916-b061-301f677271ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:15.317154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74115a-129d-4916-b061-301f677271ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:15.323431 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-ae74115a-129d-4916-b061-301f677271ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:15.323431 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:15.323431 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:15.323431 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:15.323431 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:15.323431 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:15.323431 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:15.323431 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:15.323431 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:15.323431 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:15.323431 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:15.323431 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:15.323431 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:15.323431 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:15.323431 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:15.323431 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:15.324412 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:15.324412 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:15.324412 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:15.324412 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:15.324412 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:15.324412 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:15.324412 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:15.324412 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:15.324412 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:15.324412 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:15.324412 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:15.324412 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:15.324412 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:15.324412 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:15.324412 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:15.324412 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:15.325191 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:15.325191 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:15.325191 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:15.325191 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:15.325191 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:15.325191 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:15.325191 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:15.325191 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:15.325191 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:15.325191 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:15.325191 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:15.325191 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:32:15.325191 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 84/171] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:32:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:15.333124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74115a-129d-4916-b061-301f677271ca None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:15.335788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eef0ad-f913-4e02-8a68-e60804d37388 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:15.385102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:15.409132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff88602-4c1b-4e61-95ad-500f75bf7e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:15.410392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff88602-4c1b-4e61-95ad-500f75bf7e35 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:15.410737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff88602-4c1b-4e61-95ad-500f75bf7e35 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:15.411012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff88602-4c1b-4e61-95ad-500f75bf7e35 None admin] PATH_INFO: `/v3/users/2e606e5ac9f84891835962715092f667` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:15.412248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff88602-4c1b-4e61-95ad-500f75bf7e35 None admin] RBAC: Authorizing `identity:get_user(user_id=2e606e5ac9f84891835962715092f667)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:15.413665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff88602-4c1b-4e61-95ad-500f75bf7e35 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:15.415701 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 88/172] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:15 2022] GET /identity/v3/users/2e606e5ac9f84891835962715092f667 => generated 275 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:15.416954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e824182-0ec6-4408-8edd-1fccde7d7f2a None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:15.417241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e824182-0ec6-4408-8edd-1fccde7d7f2a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:15.417515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e824182-0ec6-4408-8edd-1fccde7d7f2a None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:15.418888 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 89/173] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:15.423129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74f57b-ad84-4da8-a773-7b296c9900e8 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:15.423419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74f57b-ad84-4da8-a773-7b296c9900e8 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:15.423698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74f57b-ad84-4da8-a773-7b296c9900e8 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:15.425045 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 90/174] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:15.427532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74f57b-ad84-4da8-a773-7b296c9900e8 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:15.517358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74115a-129d-4916-b061-301f677271ca None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:15.561053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff9246-c3aa-4214-bc8d-bbd8c4091487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:15.561778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff9246-c3aa-4214-bc8d-bbd8c4091487 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:15.562014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff9246-c3aa-4214-bc8d-bbd8c4091487 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:15.562208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff9246-c3aa-4214-bc8d-bbd8c4091487 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:15.562964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff9246-c3aa-4214-bc8d-bbd8c4091487 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:15.563271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff9246-c3aa-4214-bc8d-bbd8c4091487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:15.564807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff9246-c3aa-4214-bc8d-bbd8c4091487 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:15.575683 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 85/175] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:32:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:15.583098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915de30-117d-4c9a-8872-8edc2b1c7557 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:15.583388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915de30-117d-4c9a-8872-8edc2b1c7557 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:15.583700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915de30-117d-4c9a-8872-8edc2b1c7557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:15.587182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74f57b-ad84-4da8-a773-7b296c9900e8 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:15.589435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:15.605096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e915de30-117d-4c9a-8872-8edc2b1c7557 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:15.606599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e915de30-117d-4c9a-8872-8edc2b1c7557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:15.629250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c19dbe-1457-48cd-805e-8ecb7a83b2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:15.630209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c19dbe-1457-48cd-805e-8ecb7a83b2ff None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:15.630656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c19dbe-1457-48cd-805e-8ecb7a83b2ff None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:15.631656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c19dbe-1457-48cd-805e-8ecb7a83b2ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:15.701620 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 86/176] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:15.701620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648fa4a-7b02-484c-97f7-af0e0b804d33 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:15.701620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648fa4a-7b02-484c-97f7-af0e0b804d33 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:15.701620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648fa4a-7b02-484c-97f7-af0e0b804d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:15.734214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b648fa4a-7b02-484c-97f7-af0e0b804d33 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:15.735492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b648fa4a-7b02-484c-97f7-af0e0b804d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:15.739030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c19dbe-1457-48cd-805e-8ecb7a83b2ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:15.740633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c19dbe-1457-48cd-805e-8ecb7a83b2ff None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:15.770863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:15.896162 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 87/177] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:15.897029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648fa4a-7b02-484c-97f7-af0e0b804d33 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:15.915618 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 91/178] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:15.921178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c19dbe-1457-48cd-805e-8ecb7a83b2ff None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:15.923608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:16.016339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648fa4a-7b02-484c-97f7-af0e0b804d33 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:16.062841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd4375-34a7-4827-8eb5-92945433e440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:16.063748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd4375-34a7-4827-8eb5-92945433e440 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:16.064389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd4375-34a7-4827-8eb5-92945433e440 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:16.064902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd4375-34a7-4827-8eb5-92945433e440 None admin] PATH_INFO: `/v3/users/2e606e5ac9f84891835962715092f667` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:16.066379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdd4375-34a7-4827-8eb5-92945433e440 None admin] RBAC: Authorizing `identity:get_user(user_id=2e606e5ac9f84891835962715092f667)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:16.068613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdd4375-34a7-4827-8eb5-92945433e440 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:16.070433 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 88/179] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:15 2022] GET /identity/v3/users/2e606e5ac9f84891835962715092f667 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:16.072275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efa790-22f1-4c11-9b6e-578b3d247c41 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:16.072764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efa790-22f1-4c11-9b6e-578b3d247c41 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:16.073227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efa790-22f1-4c11-9b6e-578b3d247c41 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:16.074787 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 89/180] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:16.083017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efa790-22f1-4c11-9b6e-578b3d247c41 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:16.114837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:16.162301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c19dbe-1457-48cd-805e-8ecb7a83b2ff None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:16.203169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3470cf-3cb2-4ab9-bb01-a28067e5a483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:16.203819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3470cf-3cb2-4ab9-bb01-a28067e5a483 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:16.204104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3470cf-3cb2-4ab9-bb01-a28067e5a483 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:16.204664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3470cf-3cb2-4ab9-bb01-a28067e5a483 None admin] PATH_INFO: `/v3/projects/213ed96d79ee4169824ebf710c0f9d41` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:16.205813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3470cf-3cb2-4ab9-bb01-a28067e5a483 None admin] RBAC: Authorizing `identity:get_project(project_id=213ed96d79ee4169824ebf710c0f9d41)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:16.207856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3470cf-3cb2-4ab9-bb01-a28067e5a483 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:16.209795 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 92/181] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:15 2022] GET /identity/v3/projects/213ed96d79ee4169824ebf710c0f9d41 => generated 312 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:16.214561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30b3ef-a9da-4bb8-9c06-e6e63c8d10d6 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:16.214561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30b3ef-a9da-4bb8-9c06-e6e63c8d10d6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:16.214561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30b3ef-a9da-4bb8-9c06-e6e63c8d10d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:16.220220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efa790-22f1-4c11-9b6e-578b3d247c41 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:16.235098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9f30b3ef-a9da-4bb8-9c06-e6e63c8d10d6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:16.236281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9f30b3ef-a9da-4bb8-9c06-e6e63c8d10d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:16.255171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5fb28-9e53-41e9-a2b0-876c59678c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:16.255779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5fb28-9e53-41e9-a2b0-876c59678c7f None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:16.256055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5fb28-9e53-41e9-a2b0-876c59678c7f None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:16.256321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5fb28-9e53-41e9-a2b0-876c59678c7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:16.297794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:16.314633 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 93/182] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:16.315515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30b3ef-a9da-4bb8-9c06-e6e63c8d10d6 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:16.334567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5fb28-9e53-41e9-a2b0-876c59678c7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:16.336024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5fb28-9e53-41e9-a2b0-876c59678c7f None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:16.355706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:16.418432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30b3ef-a9da-4bb8-9c06-e6e63c8d10d6 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:16.448764 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 90/183] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:16.448764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a8f05-d642-45d8-8b5c-1addeb536448 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:16.448764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a8f05-d642-45d8-8b5c-1addeb536448 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:16.448764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a8f05-d642-45d8-8b5c-1addeb536448 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:16.449143 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 91/184] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:16.449568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a8f05-d642-45d8-8b5c-1addeb536448 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:16.457722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a9610-4bcc-4878-bf25-028cc85460d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:16.458350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a9610-4bcc-4878-bf25-028cc85460d5 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:16.458664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a9610-4bcc-4878-bf25-028cc85460d5 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:16.458935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a9610-4bcc-4878-bf25-028cc85460d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:16.469121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a9610-4bcc-4878-bf25-028cc85460d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:16.474404 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-a22a9610-4bcc-4878-bf25-028cc85460d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 01 20:32:16.474404 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:16.474404 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:16.474404 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:16.474404 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:16.474404 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:16.474404 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:16.474404 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:16.474404 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:16.474404 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:16.474404 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:16.474404 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:16.474404 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:16.474404 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:16.474404 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:16.474404 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:16.475433 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:16.475433 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:16.475433 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:16.475433 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:16.475433 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:16.475433 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:16.475433 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:16.475433 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:16.475433 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:16.475433 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:16.475433 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:16.475433 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:16.475433 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:16.475433 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:16.475433 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:16.475433 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:16.476396 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:16.476396 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:16.476396 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:16.476396 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:16.476396 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:16.476396 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:16.476396 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:16.476396 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:16.476396 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:16.476396 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:16.476396 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 01 20:32:16.476396 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:16.476396 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 94/185] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:16.477191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a9610-4bcc-4878-bf25-028cc85460d5 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:16.539157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:16.559121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a8f05-d642-45d8-8b5c-1addeb536448 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:16.576839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:16.586882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a9610-4bcc-4878-bf25-028cc85460d5 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:16.595098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f6acb-2340-4da2-a95f-2443a70ef0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:16.595757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f6acb-2340-4da2-a95f-2443a70ef0b0 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:16.596074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f6acb-2340-4da2-a95f-2443a70ef0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:16.596428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f6acb-2340-4da2-a95f-2443a70ef0b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:16.597367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f6acb-2340-4da2-a95f-2443a70ef0b0 None admin] RBAC: Adding query filter params (role.id=17f3aeacc5034f3891cc6556da651c1b, scope.project.id=213ed96d79ee4169824ebf710c0f9d41, user.id=2e606e5ac9f84891835962715092f667) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:16.598041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f6acb-2340-4da2-a95f-2443a70ef0b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:16.599686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f6acb-2340-4da2-a95f-2443a70ef0b0 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:16.614562 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 92/186] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:32:16 2022] GET /identity/v3/role_assignments?user.id=2e606e5ac9f84891835962715092f667&scope.project.id=213ed96d79ee4169824ebf710c0f9d41&role.id=17f3aeacc5034f3891cc6556da651c1b => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:16.614562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f6acb-2340-4da2-a95f-2443a70ef0b0 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:16.631341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0630e2-5c56-4330-8150-3256a0d3b298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:16.632002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0630e2-5c56-4330-8150-3256a0d3b298 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:16.632313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0630e2-5c56-4330-8150-3256a0d3b298 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:16.632588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0630e2-5c56-4330-8150-3256a0d3b298 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:16.646611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0630e2-5c56-4330-8150-3256a0d3b298 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:16.652687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-af0630e2-5c56-4330-8150-3256a0d3b298 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:32:16.652687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:16.652687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:16.652687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:16.652687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:16.652687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:16.652687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:16.652687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:16.652687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:16.652687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:16.652687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:16.652687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:16.652687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:16.652687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:16.652687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:16.652687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:16.653654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:16.653654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:16.653654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:16.653654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:16.653654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:16.653654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:16.653654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:16.653654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:16.653654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:16.653654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:16.653654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:16.653654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:16.653654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:16.653654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:16.653654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:16.653654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:16.654435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:16.654435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:16.654435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:16.654435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:16.654435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:16.654435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:16.654435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:16.654435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:16.654435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:16.654435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:16.654435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:32:16.654435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:16.654435 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 95/187] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:32:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:16.655151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7ad76-e66f-4b4f-9836-86dd8a202c3f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:16.656671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7ad76-e66f-4b4f-9836-86dd8a202c3f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:16.656671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7ad76-e66f-4b4f-9836-86dd8a202c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:16.676309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-22a7ad76-e66f-4b4f-9836-86dd8a202c3f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:16.677574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-22a7ad76-e66f-4b4f-9836-86dd8a202c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:16.736776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f6acb-2340-4da2-a95f-2443a70ef0b0 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:16.757377 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 96/188] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:16.758045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7ad76-e66f-4b4f-9836-86dd8a202c3f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:16.778065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1de0df-4d6e-486b-bd5f-5677f12898cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:16.778755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1de0df-4d6e-486b-bd5f-5677f12898cd None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:16.779094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1de0df-4d6e-486b-bd5f-5677f12898cd None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:16.779609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1de0df-4d6e-486b-bd5f-5677f12898cd None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:16.781270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1de0df-4d6e-486b-bd5f-5677f12898cd None admin] RBAC: Authorizing `identity:get_project(project_id=a503020af843467a992a8df7f46d3fec)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:16.782767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1de0df-4d6e-486b-bd5f-5677f12898cd None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:16.784512 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 93/189] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:16 2022] GET /identity/v3/projects/a503020af843467a992a8df7f46d3fec => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:16.787505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1de0df-4d6e-486b-bd5f-5677f12898cd None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:16.832355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:16.862315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:16.897407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7ad76-e66f-4b4f-9836-86dd8a202c3f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:16.931990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1de0df-4d6e-486b-bd5f-5677f12898cd None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:16.942962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9b3ab-d7e3-4d08-8119-30b5ba165a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:16.943856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9b3ab-d7e3-4d08-8119-30b5ba165a4f None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:16.946599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9b3ab-d7e3-4d08-8119-30b5ba165a4f None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:16.946599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9b3ab-d7e3-4d08-8119-30b5ba165a4f None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:16.946599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9b3ab-d7e3-4d08-8119-30b5ba165a4f None admin] RBAC: Adding query filter params (name=reader) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:16.946599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9b3ab-d7e3-4d08-8119-30b5ba165a4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:16.948147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9b3ab-d7e3-4d08-8119-30b5ba165a4f None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:16.957771 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 97/190] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:16.958426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9b3ab-d7e3-4d08-8119-30b5ba165a4f None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:16.976982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4413e-f4c9-4676-aa77-c8ed73b7b273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:16.977779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4413e-f4c9-4676-aa77-c8ed73b7b273 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:16.978069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4413e-f4c9-4676-aa77-c8ed73b7b273 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:16.978349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4413e-f4c9-4676-aa77-c8ed73b7b273 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:16.988887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4413e-f4c9-4676-aa77-c8ed73b7b273 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:16.994080 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-68d4413e-f4c9-4676-aa77-c8ed73b7b273 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:16.994080 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:16.994080 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:16.994080 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:16.994080 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:16.994080 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:16.994080 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:16.994080 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:16.994080 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:16.994080 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:16.994080 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:16.994080 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:16.994080 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:16.994080 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:16.994080 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:16.994080 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:16.995224 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:16.995224 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:16.995224 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:16.995224 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:16.995224 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:16.995224 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:16.995224 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:16.995224 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:16.995224 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:16.995224 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:16.995224 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:16.995224 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:16.995224 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:16.995224 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:16.995224 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:16.995224 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:16.996015 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:16.996015 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:16.996015 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:16.996015 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:16.996015 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:16.996015 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:16.996015 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:16.996015 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:16.996015 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:16.996015 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:16.996015 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:16.996015 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:32:16.996015 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 94/191] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:32:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:16.996015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4413e-f4c9-4676-aa77-c8ed73b7b273 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:17.081370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9b3ab-d7e3-4d08-8119-30b5ba165a4f None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:17.098062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:17.113463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4413e-f4c9-4676-aa77-c8ed73b7b273 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:17.120416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572e0d2-7879-4a14-9dd2-2e0af954a90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:17.121020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572e0d2-7879-4a14-9dd2-2e0af954a90b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:17.121296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572e0d2-7879-4a14-9dd2-2e0af954a90b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:17.121566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572e0d2-7879-4a14-9dd2-2e0af954a90b None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:17.122224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0572e0d2-7879-4a14-9dd2-2e0af954a90b None admin] RBAC: Adding query filter params (name=admin) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:17.122553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0572e0d2-7879-4a14-9dd2-2e0af954a90b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:17.123912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0572e0d2-7879-4a14-9dd2-2e0af954a90b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:17.124842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:17.131998 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 98/192] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:17.132766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572e0d2-7879-4a14-9dd2-2e0af954a90b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:17.151742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32efc2-8064-47c4-bccc-960c983a8198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:17.152344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32efc2-8064-47c4-bccc-960c983a8198 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:17.152624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32efc2-8064-47c4-bccc-960c983a8198 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:17.152894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32efc2-8064-47c4-bccc-960c983a8198 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:17.153619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32efc2-8064-47c4-bccc-960c983a8198 None admin] RBAC: Adding query filter params (role.id=8b7b6e0028b04be8912abda33d3ef729, scope.project.id=a503020af843467a992a8df7f46d3fec, user.id=2e606e5ac9f84891835962715092f667) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:17.153930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32efc2-8064-47c4-bccc-960c983a8198 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:17.155417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32efc2-8064-47c4-bccc-960c983a8198 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:17.170767 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 95/193] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:32:16 2022] GET /identity/v3/role_assignments?user.id=2e606e5ac9f84891835962715092f667&scope.project.id=a503020af843467a992a8df7f46d3fec&role.id=8b7b6e0028b04be8912abda33d3ef729 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:17.170767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32efc2-8064-47c4-bccc-960c983a8198 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:17.244064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572e0d2-7879-4a14-9dd2-2e0af954a90b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:17.278174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32efc2-8064-47c4-bccc-960c983a8198 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:17.280507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb8086-ed53-4778-a557-b48236438e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:17.281145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb8086-ed53-4778-a557-b48236438e72 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:17.281428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb8086-ed53-4778-a557-b48236438e72 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:17.281700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb8086-ed53-4778-a557-b48236438e72 None admin] PATH_INFO: `/v3/users/ac00bd52ce5943af943ce9ed56577c79` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:17.282908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb8086-ed53-4778-a557-b48236438e72 None admin] RBAC: Authorizing `identity:get_user(user_id=ac00bd52ce5943af943ce9ed56577c79)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:17.284608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb8086-ed53-4778-a557-b48236438e72 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:17.286249 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 99/194] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:17 2022] GET /identity/v3/users/ac00bd52ce5943af943ce9ed56577c79 => generated 289 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:17.287072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb8086-ed53-4778-a557-b48236438e72 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:17.321137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d976fc-b1a2-4d19-9030-24ee197d3a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:17.322019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d976fc-b1a2-4d19-9030-24ee197d3a4b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:17.322301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d976fc-b1a2-4d19-9030-24ee197d3a4b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:17.322612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d976fc-b1a2-4d19-9030-24ee197d3a4b None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:17.323296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d976fc-b1a2-4d19-9030-24ee197d3a4b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:17.323603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d976fc-b1a2-4d19-9030-24ee197d3a4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:17.325011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d976fc-b1a2-4d19-9030-24ee197d3a4b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:17.339272 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 96/195] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:32:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:17.339272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d976fc-b1a2-4d19-9030-24ee197d3a4b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:17.340186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:17.380840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:17.408271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb8086-ed53-4778-a557-b48236438e72 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:17.449370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170d080-7b3b-43a1-9b35-cec0501f9a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:17.449998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170d080-7b3b-43a1-9b35-cec0501f9a20 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:17.450274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170d080-7b3b-43a1-9b35-cec0501f9a20 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:17.450568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170d080-7b3b-43a1-9b35-cec0501f9a20 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:17.451681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170d080-7b3b-43a1-9b35-cec0501f9a20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:17.453116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170d080-7b3b-43a1-9b35-cec0501f9a20 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:17.454743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d976fc-b1a2-4d19-9030-24ee197d3a4b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:17.454873 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 100/196] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:32:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:17.456041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170d080-7b3b-43a1-9b35-cec0501f9a20 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:17.490303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638e257-a1b9-44e2-b43d-0bdebde08739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:17.490965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638e257-a1b9-44e2-b43d-0bdebde08739 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:17.491246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638e257-a1b9-44e2-b43d-0bdebde08739 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:17.491519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638e257-a1b9-44e2-b43d-0bdebde08739 None admin] PATH_INFO: `/v3/users/3cfaf7eaf2004320b41cb14e9fbb3ae3` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:17.504584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638e257-a1b9-44e2-b43d-0bdebde08739 None admin] RBAC: Authorizing `identity:get_user(user_id=3cfaf7eaf2004320b41cb14e9fbb3ae3)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:17.506027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638e257-a1b9-44e2-b43d-0bdebde08739 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:17.507746 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 97/197] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:17 2022] GET /identity/v3/users/3cfaf7eaf2004320b41cb14e9fbb3ae3 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:17.511455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638e257-a1b9-44e2-b43d-0bdebde08739 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:17.555682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170d080-7b3b-43a1-9b35-cec0501f9a20 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:17.565706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:17.590594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a3268-e7eb-403e-977e-2ac49c9300ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:17.591193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a3268-e7eb-403e-977e-2ac49c9300ac None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:17.591470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a3268-e7eb-403e-977e-2ac49c9300ac None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:17.591738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a3268-e7eb-403e-977e-2ac49c9300ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:17.603345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:17.610398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638e257-a1b9-44e2-b43d-0bdebde08739 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:17.644594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50a3a3-8485-45b3-b747-85bbcabbd86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:17.645154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50a3a3-8485-45b3-b747-85bbcabbd86e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:17.645440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50a3a3-8485-45b3-b747-85bbcabbd86e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:17.645711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50a3a3-8485-45b3-b747-85bbcabbd86e None admin] PATH_INFO: `/v3/users/3cfaf7eaf2004320b41cb14e9fbb3ae3` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:17.646875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50a3a3-8485-45b3-b747-85bbcabbd86e None admin] RBAC: Authorizing `identity:get_user(user_id=3cfaf7eaf2004320b41cb14e9fbb3ae3)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:17.648317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50a3a3-8485-45b3-b747-85bbcabbd86e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:17.650077 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 98/198] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:17 2022] GET /identity/v3/users/3cfaf7eaf2004320b41cb14e9fbb3ae3 => generated 283 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:17.650677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50a3a3-8485-45b3-b747-85bbcabbd86e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:17.662307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a3268-e7eb-403e-977e-2ac49c9300ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:17.663790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a3268-e7eb-403e-977e-2ac49c9300ac None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:17.749353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50a3a3-8485-45b3-b747-85bbcabbd86e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:17.761485 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 101/199] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:17.763742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a3268-e7eb-403e-977e-2ac49c9300ac None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:17.775118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:17.782161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7241e-e575-48f1-9466-1a51f781877c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:17.782882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7241e-e575-48f1-9466-1a51f781877c None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:17.783191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7241e-e575-48f1-9466-1a51f781877c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:17.783463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7241e-e575-48f1-9466-1a51f781877c None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:17.785326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7241e-e575-48f1-9466-1a51f781877c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:17.786091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7241e-e575-48f1-9466-1a51f781877c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:17.808119 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 99/200] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Oct 1 20:32:17 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:32:17.809100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7241e-e575-48f1-9466-1a51f781877c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:17.843690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:17.878902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a3268-e7eb-403e-977e-2ac49c9300ac None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:17.917536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53581787-5542-4fe4-b16e-ad1fb8819211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:17.918300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53581787-5542-4fe4-b16e-ad1fb8819211 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:17.918613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53581787-5542-4fe4-b16e-ad1fb8819211 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:17.919389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53581787-5542-4fe4-b16e-ad1fb8819211 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:17.942060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7241e-e575-48f1-9466-1a51f781877c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:17.980209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d93e9-5367-463f-abbb-5a4f881859eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:17.980992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d93e9-5367-463f-abbb-5a4f881859eb None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:17.981369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d93e9-5367-463f-abbb-5a4f881859eb None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:17.981754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d93e9-5367-463f-abbb-5a4f881859eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:18.000788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53581787-5542-4fe4-b16e-ad1fb8819211 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:18.002771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53581787-5542-4fe4-b16e-ad1fb8819211 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:18.016376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:18.070838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d93e9-5367-463f-abbb-5a4f881859eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:18.072566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d93e9-5367-463f-abbb-5a4f881859eb None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:18.100214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:18.125696 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 102/201] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:18.130616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53581787-5542-4fe4-b16e-ad1fb8819211 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:18.218349 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 100/202] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:18.219340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d93e9-5367-463f-abbb-5a4f881859eb None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:18.252238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53581787-5542-4fe4-b16e-ad1fb8819211 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:18.264003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:18.300310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8372bd-ba83-4fd4-b63e-b1e55aad6af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:18.301479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8372bd-ba83-4fd4-b63e-b1e55aad6af2 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:18.302358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8372bd-ba83-4fd4-b63e-b1e55aad6af2 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:18.303115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8372bd-ba83-4fd4-b63e-b1e55aad6af2 None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:18.304719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8372bd-ba83-4fd4-b63e-b1e55aad6af2 None admin] RBAC: Authorizing `identity:get_project(project_id=a503020af843467a992a8df7f46d3fec)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:18.307087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8372bd-ba83-4fd4-b63e-b1e55aad6af2 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:18.309158 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 103/203] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:18 2022] GET /identity/v3/projects/a503020af843467a992a8df7f46d3fec => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:18.310724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8372bd-ba83-4fd4-b63e-b1e55aad6af2 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:18.346831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d93e9-5367-463f-abbb-5a4f881859eb None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:18.365689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:18.385631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f134e-b6eb-4805-bf16-425694459c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:18.386427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f134e-b6eb-4805-bf16-425694459c65 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:18.386826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f134e-b6eb-4805-bf16-425694459c65 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:18.387201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f134e-b6eb-4805-bf16-425694459c65 None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:18.388429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f134e-b6eb-4805-bf16-425694459c65 None admin] RBAC: Authorizing `identity:get_project(project_id=5acc2b4d60e244298e1c3401d66e7636)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:18.390439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f134e-b6eb-4805-bf16-425694459c65 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:18.393019 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 101/204] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:18 2022] GET /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:18.394916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f134e-b6eb-4805-bf16-425694459c65 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:18.451578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8372bd-ba83-4fd4-b63e-b1e55aad6af2 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:18.498070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f4d30-5b60-4aaf-81d1-eb3404390c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:18.499601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f4d30-5b60-4aaf-81d1-eb3404390c5f None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:18.500112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f4d30-5b60-4aaf-81d1-eb3404390c5f None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:18.500593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f4d30-5b60-4aaf-81d1-eb3404390c5f None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:18.502863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f4d30-5b60-4aaf-81d1-eb3404390c5f None admin] RBAC: Authorizing `identity:get_project(project_id=5acc2b4d60e244298e1c3401d66e7636)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:18.504875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f4d30-5b60-4aaf-81d1-eb3404390c5f None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:18.507920 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 104/205] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:18 2022] GET /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:18.509182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f4d30-5b60-4aaf-81d1-eb3404390c5f None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:18.526990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f134e-b6eb-4805-bf16-425694459c65 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:18.567225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:18.569034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68cc55-267c-4eaa-884d-1ffff483ab1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:18.570006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68cc55-267c-4eaa-884d-1ffff483ab1e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:18.570520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68cc55-267c-4eaa-884d-1ffff483ab1e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:18.570991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68cc55-267c-4eaa-884d-1ffff483ab1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:18.572237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68cc55-267c-4eaa-884d-1ffff483ab1e None admin] RBAC: Adding query filter params (role.id=be70f76ecf3f465fb09d46916db16671, scope.project.id=a503020af843467a992a8df7f46d3fec, user.id=ac00bd52ce5943af943ce9ed56577c79) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:18.572748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68cc55-267c-4eaa-884d-1ffff483ab1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:18.574410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68cc55-267c-4eaa-884d-1ffff483ab1e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:18.582516 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 102/206] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:32:18 2022] GET /identity/v3/role_assignments?user.id=ac00bd52ce5943af943ce9ed56577c79&scope.project.id=a503020af843467a992a8df7f46d3fec&role.id=be70f76ecf3f465fb09d46916db16671 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:18.583594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68cc55-267c-4eaa-884d-1ffff483ab1e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:18.627848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f4d30-5b60-4aaf-81d1-eb3404390c5f None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:18.643807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:18.664810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cd41a-203c-4a60-9860-184fb9f1c881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:18.665498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cd41a-203c-4a60-9860-184fb9f1c881 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:18.665690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cd41a-203c-4a60-9860-184fb9f1c881 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:18.665961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cd41a-203c-4a60-9860-184fb9f1c881 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:18.667059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54cd41a-203c-4a60-9860-184fb9f1c881 None admin] RBAC: Adding query filter params (role.id=1c50896511054a5f994bee3d99fa0619, scope.project.id=5acc2b4d60e244298e1c3401d66e7636, user.id=3cfaf7eaf2004320b41cb14e9fbb3ae3) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:18.667059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54cd41a-203c-4a60-9860-184fb9f1c881 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:18.668475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54cd41a-203c-4a60-9860-184fb9f1c881 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:18.675352 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 105/207] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:32:18 2022] GET /identity/v3/role_assignments?user.id=3cfaf7eaf2004320b41cb14e9fbb3ae3&scope.project.id=5acc2b4d60e244298e1c3401d66e7636&role.id=1c50896511054a5f994bee3d99fa0619 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:18.710471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68cc55-267c-4eaa-884d-1ffff483ab1e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:18.753206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fec3bc-150d-4b73-97a6-e69cdd1e8a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:18.754288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fec3bc-150d-4b73-97a6-e69cdd1e8a7a None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:18.754867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fec3bc-150d-4b73-97a6-e69cdd1e8a7a None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:18.755357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fec3bc-150d-4b73-97a6-e69cdd1e8a7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:18.756348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fec3bc-150d-4b73-97a6-e69cdd1e8a7a None admin] RBAC: Adding query filter params (role.id=8b7b6e0028b04be8912abda33d3ef729, scope.project.id=5acc2b4d60e244298e1c3401d66e7636, user.id=3cfaf7eaf2004320b41cb14e9fbb3ae3) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:18.756872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fec3bc-150d-4b73-97a6-e69cdd1e8a7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:18.758701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fec3bc-150d-4b73-97a6-e69cdd1e8a7a None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:18.765957 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 103/208] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:32:18 2022] GET /identity/v3/role_assignments?user.id=3cfaf7eaf2004320b41cb14e9fbb3ae3&scope.project.id=5acc2b4d60e244298e1c3401d66e7636&role.id=8b7b6e0028b04be8912abda33d3ef729 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:20.594903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839b2d-72fc-4efe-98dc-36169bbe2116 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:20.595920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839b2d-72fc-4efe-98dc-36169bbe2116 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:20.596429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839b2d-72fc-4efe-98dc-36169bbe2116 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:20.598604 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 106/209] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:20.604067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72395c0a-4b6d-4ab5-81a9-6fce8ccca370 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:20.604873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72395c0a-4b6d-4ab5-81a9-6fce8ccca370 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:20.605429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72395c0a-4b6d-4ab5-81a9-6fce8ccca370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:20.621380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6246d6-b873-4b0a-9d57-a7bcf77df0e3 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:20.621983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6246d6-b873-4b0a-9d57-a7bcf77df0e3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:20.622815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6246d6-b873-4b0a-9d57-a7bcf77df0e3 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:20.624628 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 107/210] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:32:20.627721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-72395c0a-4b6d-4ab5-81a9-6fce8ccca370 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:20.630040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-72395c0a-4b6d-4ab5-81a9-6fce8ccca370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:20.630265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a5f2e-f4e1-43b8-8968-238c3ec113b4 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:20.630554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a5f2e-f4e1-43b8-8968-238c3ec113b4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:20.630837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a5f2e-f4e1-43b8-8968-238c3ec113b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:20.649164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b72a5f2e-f4e1-43b8-8968-238c3ec113b4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:20.650788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b72a5f2e-f4e1-43b8-8968-238c3ec113b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:20.707659 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 104/211] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:20.722923 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 108/212] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:32:21.051837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72395c0a-4b6d-4ab5-81a9-6fce8ccca370 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:21.058514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a5f2e-f4e1-43b8-8968-238c3ec113b4 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:21.103734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:21.111864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:21.173166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72395c0a-4b6d-4ab5-81a9-6fce8ccca370 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:21.191381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a5f2e-f4e1-43b8-8968-238c3ec113b4 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:21.216530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d79884-50b1-4dc3-9470-d4a7bca3708d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:21.217170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d79884-50b1-4dc3-9470-d4a7bca3708d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:21.217450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d79884-50b1-4dc3-9470-d4a7bca3708d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:21.217722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d79884-50b1-4dc3-9470-d4a7bca3708d None admin] PATH_INFO: `/v3/domains/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:21.218875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d79884-50b1-4dc3-9470-d4a7bca3708d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:21.220355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d79884-50b1-4dc3-9470-d4a7bca3708d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:21.222508 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 105/213] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:32:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:21.223711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbce39-701f-4ec5-a00b-f3ae44a02659 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:21.223995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbce39-701f-4ec5-a00b-f3ae44a02659 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:21.224267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbce39-701f-4ec5-a00b-f3ae44a02659 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:21.225624 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 106/214] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:21.230332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbce39-701f-4ec5-a00b-f3ae44a02659 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:21.235854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaef8ad-29fc-4297-bf68-c53ae598dc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:21.236996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaef8ad-29fc-4297-bf68-c53ae598dc27 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:21.237730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaef8ad-29fc-4297-bf68-c53ae598dc27 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:21.238390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaef8ad-29fc-4297-bf68-c53ae598dc27 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:21.251296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaef8ad-29fc-4297-bf68-c53ae598dc27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:21.256429 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-bcaef8ad-29fc-4297-bf68-c53ae598dc27 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:21.256429 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:21.256429 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:21.256429 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:21.256429 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:21.256429 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:21.256429 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:21.256429 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:21.256429 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:21.256429 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:21.256429 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:21.256429 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:21.256429 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:21.256429 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:21.256429 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:21.256429 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:21.257290 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:21.257290 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:21.257290 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:21.257290 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:21.257290 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:21.257290 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:21.257290 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:21.257290 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:21.257290 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:21.257290 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:21.257290 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:21.257290 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:21.257290 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:21.257290 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:21.257290 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:21.257290 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:21.258100 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:21.258100 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:21.258100 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:21.258100 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:21.258100 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:21.258100 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:21.258100 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:21.258100 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:21.258100 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:21.258100 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:21.258100 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:21.258100 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:21.260174 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 109/215] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:21.263753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923371eb-7ef9-42c4-896c-2be606a60c01 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:21.264042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923371eb-7ef9-42c4-896c-2be606a60c01 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:21.264315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923371eb-7ef9-42c4-896c-2be606a60c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:21.282206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-923371eb-7ef9-42c4-896c-2be606a60c01 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:21.283428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-923371eb-7ef9-42c4-896c-2be606a60c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:21.335711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbce39-701f-4ec5-a00b-f3ae44a02659 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:21.349424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:21.349645 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 110/216] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:21.355789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71752b10-01f2-40a2-9155-d956f03b3ad5 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:21.356075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71752b10-01f2-40a2-9155-d956f03b3ad5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:21.356346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71752b10-01f2-40a2-9155-d956f03b3ad5 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:21.358178 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 111/217] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:21.358760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71752b10-01f2-40a2-9155-d956f03b3ad5 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:21.372302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac70f92-e459-4199-82ac-be886a3f0645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:21.372905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac70f92-e459-4199-82ac-be886a3f0645 None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:21.373348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac70f92-e459-4199-82ac-be886a3f0645 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:21.373452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac70f92-e459-4199-82ac-be886a3f0645 None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:21.374058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac70f92-e459-4199-82ac-be886a3f0645 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:21.375516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac70f92-e459-4199-82ac-be886a3f0645 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:21.395854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:21.410748 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 107/218] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Oct 1 20:32:21 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:32:21.410748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01217441-17a0-4396-aba9-4db87d4e241b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:21.410748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01217441-17a0-4396-aba9-4db87d4e241b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:21.410748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01217441-17a0-4396-aba9-4db87d4e241b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:21.423454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-01217441-17a0-4396-aba9-4db87d4e241b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:21.424779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-01217441-17a0-4396-aba9-4db87d4e241b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:21.504621 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 108/219] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:21.512837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71752b10-01f2-40a2-9155-d956f03b3ad5 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:21.552478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0926e59-e890-47ff-80f8-415d042774af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:21.553168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0926e59-e890-47ff-80f8-415d042774af None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:21.553456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0926e59-e890-47ff-80f8-415d042774af None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:21.553730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0926e59-e890-47ff-80f8-415d042774af None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:21.554415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0926e59-e890-47ff-80f8-415d042774af None admin] RBAC: Adding query filter params (name=member) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:21.554754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0926e59-e890-47ff-80f8-415d042774af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:21.556148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0926e59-e890-47ff-80f8-415d042774af None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:21.566358 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 112/220] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:21.572794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01217441-17a0-4396-aba9-4db87d4e241b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:21.679883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0926e59-e890-47ff-80f8-415d042774af None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:21.683622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:21.689851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01217441-17a0-4396-aba9-4db87d4e241b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:21.744431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df146aa-38da-4761-97db-baa1377f2c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:21.745464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df146aa-38da-4761-97db-baa1377f2c85 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:21.746355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df146aa-38da-4761-97db-baa1377f2c85 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:21.746789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df146aa-38da-4761-97db-baa1377f2c85 None admin] PATH_INFO: `/v3/users/f46b4b1ea99846afa5d0d196ef227a4d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:21.768196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df146aa-38da-4761-97db-baa1377f2c85 None admin] RBAC: Authorizing `identity:get_user(user_id=f46b4b1ea99846afa5d0d196ef227a4d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:21.770708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df146aa-38da-4761-97db-baa1377f2c85 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:21.773489 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 109/221] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:21 2022] GET /identity/v3/users/f46b4b1ea99846afa5d0d196ef227a4d => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:21.778694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df146aa-38da-4761-97db-baa1377f2c85 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:21.807858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:21.822640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0926e59-e890-47ff-80f8-415d042774af None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:21.870645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb28e5e-b38d-4461-b58b-de1ba86c84cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:21.871679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb28e5e-b38d-4461-b58b-de1ba86c84cb None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:21.872212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb28e5e-b38d-4461-b58b-de1ba86c84cb None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:21.872631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb28e5e-b38d-4461-b58b-de1ba86c84cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:21.886998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb28e5e-b38d-4461-b58b-de1ba86c84cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:21.892587 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-7bb28e5e-b38d-4461-b58b-de1ba86c84cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:32:21.892587 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:21.892587 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:21.892587 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:21.892587 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:21.892587 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:21.892587 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:21.892587 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:21.892587 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:21.892587 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:21.892587 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:21.892587 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:21.892587 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:21.892587 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:21.892587 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:21.892587 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:21.894228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:21.894228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:21.894228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:21.894228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:21.894228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:21.894228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:21.894228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:21.894228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:21.894228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:21.894228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:21.894228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:21.894228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:21.894228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:21.894228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:21.894228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:21.894228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:21.895096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:21.895096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:21.895096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:21.895096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:21.895096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:21.895096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:21.895096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:21.895096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:21.895096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:21.895096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:21.895096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:32:21.895096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:21.895096 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 113/222] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:32:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:32:21.895096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb28e5e-b38d-4461-b58b-de1ba86c84cb None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:21.896935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df146aa-38da-4761-97db-baa1377f2c85 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:21.942245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d87ad-2626-49a3-8318-e417505ca222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:21.942859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d87ad-2626-49a3-8318-e417505ca222 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:21.943139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d87ad-2626-49a3-8318-e417505ca222 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:21.943410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d87ad-2626-49a3-8318-e417505ca222 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:21.970121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:22.020419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d87ad-2626-49a3-8318-e417505ca222 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:22.021832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb28e5e-b38d-4461-b58b-de1ba86c84cb None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:22.022462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d87ad-2626-49a3-8318-e417505ca222 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:22.068393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072a5f6-b24c-4185-b90d-9d106cef59a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:22.069729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072a5f6-b24c-4185-b90d-9d106cef59a3 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:22.069729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072a5f6-b24c-4185-b90d-9d106cef59a3 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:22.069969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072a5f6-b24c-4185-b90d-9d106cef59a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:22.081956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072a5f6-b24c-4185-b90d-9d106cef59a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:22.089246 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-b072a5f6-b24c-4185-b90d-9d106cef59a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:22.089246 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:22.089246 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:22.089246 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:22.089246 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:22.089246 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:22.089246 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:22.089246 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:22.089246 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:22.089246 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:22.089246 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:22.089246 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:22.089246 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:22.089246 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:22.089246 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:22.089246 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:22.090334 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:22.090334 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:22.090334 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:22.090334 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:22.090334 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:22.090334 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:22.090334 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:22.090334 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:22.090334 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:22.090334 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:22.090334 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:22.090334 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:22.090334 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:22.090334 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:22.090334 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:22.090334 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:22.091378 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:22.091378 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:22.091378 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:22.091378 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:22.091378 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:22.091378 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:22.091378 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:22.091378 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:22.091378 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:22.091378 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:22.091378 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:22.091378 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:22.093178 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 114/223] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:32:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:22.093727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072a5f6-b24c-4185-b90d-9d106cef59a3 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:22.112245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:22.148216 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 110/224] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 01 20:32:22.149615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d87ad-2626-49a3-8318-e417505ca222 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:22.211586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:22.223648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072a5f6-b24c-4185-b90d-9d106cef59a3 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:22.262334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d87ad-2626-49a3-8318-e417505ca222 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:22.271661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f048d-a9df-4824-a696-c611be0695b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:22.272498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f048d-a9df-4824-a696-c611be0695b6 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:22.273389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f048d-a9df-4824-a696-c611be0695b6 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:22.273815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f048d-a9df-4824-a696-c611be0695b6 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:22.275079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f048d-a9df-4824-a696-c611be0695b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:22.275539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f048d-a9df-4824-a696-c611be0695b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:22.277604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f048d-a9df-4824-a696-c611be0695b6 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:22.290340 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 115/225] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:22.291785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f048d-a9df-4824-a696-c611be0695b6 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:22.301444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43833059-a56e-4142-baca-9af682dc4380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:22.302237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43833059-a56e-4142-baca-9af682dc4380 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:22.302656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43833059-a56e-4142-baca-9af682dc4380 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:22.303027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43833059-a56e-4142-baca-9af682dc4380 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:22.303908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43833059-a56e-4142-baca-9af682dc4380 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:22.304287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43833059-a56e-4142-baca-9af682dc4380 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:22.305755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43833059-a56e-4142-baca-9af682dc4380 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:22.314794 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 111/226] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:32:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:22.319695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43833059-a56e-4142-baca-9af682dc4380 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:22.393417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:22.428904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f048d-a9df-4824-a696-c611be0695b6 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:22.434285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43833059-a56e-4142-baca-9af682dc4380 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:22.454788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:22.478669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b246f-0aa9-4c5b-a7e3-28fd162af85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:22.479231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b246f-0aa9-4c5b-a7e3-28fd162af85b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:22.479518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b246f-0aa9-4c5b-a7e3-28fd162af85b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:22.479795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b246f-0aa9-4c5b-a7e3-28fd162af85b None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:22.480464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc001fbe-e4be-47c9-af32-7c6e6b4064fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:22.480942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b246f-0aa9-4c5b-a7e3-28fd162af85b None admin] RBAC: Authorizing `identity:get_project(project_id=5acc2b4d60e244298e1c3401d66e7636)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:22.481529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc001fbe-e4be-47c9-af32-7c6e6b4064fe None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:22.482001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc001fbe-e4be-47c9-af32-7c6e6b4064fe None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:22.482393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b246f-0aa9-4c5b-a7e3-28fd162af85b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:22.482693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc001fbe-e4be-47c9-af32-7c6e6b4064fe None admin] PATH_INFO: `/v3/users/3cfaf7eaf2004320b41cb14e9fbb3ae3` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:22.489343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc001fbe-e4be-47c9-af32-7c6e6b4064fe None admin] RBAC: Authorizing `identity:get_user(user_id=3cfaf7eaf2004320b41cb14e9fbb3ae3)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:22.489476 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 116/227] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:22 2022] GET /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:22.489476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b246f-0aa9-4c5b-a7e3-28fd162af85b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:22.492786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc001fbe-e4be-47c9-af32-7c6e6b4064fe None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:22.495006 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 112/228] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:22 2022] GET /identity/v3/users/3cfaf7eaf2004320b41cb14e9fbb3ae3 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:22.496084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc001fbe-e4be-47c9-af32-7c6e6b4064fe None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:22.596059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b246f-0aa9-4c5b-a7e3-28fd162af85b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:22.650088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2808c-de07-4845-a654-79c7e2966c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:22.650441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc001fbe-e4be-47c9-af32-7c6e6b4064fe None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:22.651297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2808c-de07-4845-a654-79c7e2966c15 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:22.652063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2808c-de07-4845-a654-79c7e2966c15 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:22.652558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2808c-de07-4845-a654-79c7e2966c15 None admin] PATH_INFO: `/v3/users/3cfaf7eaf2004320b41cb14e9fbb3ae3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:22.654807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e2808c-de07-4845-a654-79c7e2966c15 None admin] RBAC: Authorizing `identity:get_user(user_id=3cfaf7eaf2004320b41cb14e9fbb3ae3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:22.656438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e2808c-de07-4845-a654-79c7e2966c15 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:22.664257 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 117/229] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:22 2022] GET /identity/v3/users/3cfaf7eaf2004320b41cb14e9fbb3ae3 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:22.664257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2808c-de07-4845-a654-79c7e2966c15 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:22.680376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:22.705562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f0608e-b52f-4e4c-9fa4-dd9fd8011fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:22.708187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0608e-b52f-4e4c-9fa4-dd9fd8011fef None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:22.708619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0608e-b52f-4e4c-9fa4-dd9fd8011fef None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:22.709066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0608e-b52f-4e4c-9fa4-dd9fd8011fef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:22.709950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f0608e-b52f-4e4c-9fa4-dd9fd8011fef None admin] RBAC: Adding query filter params (role.id=17f3aeacc5034f3891cc6556da651c1b, scope.project.id=5acc2b4d60e244298e1c3401d66e7636, user.id=f46b4b1ea99846afa5d0d196ef227a4d) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:22.710404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f0608e-b52f-4e4c-9fa4-dd9fd8011fef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:22.712519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f0608e-b52f-4e4c-9fa4-dd9fd8011fef None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:22.720210 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 113/230] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:32:22 2022] GET /identity/v3/role_assignments?user.id=f46b4b1ea99846afa5d0d196ef227a4d&scope.project.id=5acc2b4d60e244298e1c3401d66e7636&role.id=17f3aeacc5034f3891cc6556da651c1b => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:22.726206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f0608e-b52f-4e4c-9fa4-dd9fd8011fef None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:22.819033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:22.830285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2808c-de07-4845-a654-79c7e2966c15 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:22.915498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40f5c8-1816-4c8b-baf1-ce991dd142b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:22.916158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40f5c8-1816-4c8b-baf1-ce991dd142b7 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:22.916447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40f5c8-1816-4c8b-baf1-ce991dd142b7 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:22.916717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40f5c8-1816-4c8b-baf1-ce991dd142b7 None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:22.919417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40f5c8-1816-4c8b-baf1-ce991dd142b7 None admin] RBAC: Authorizing `identity:get_project(project_id=5acc2b4d60e244298e1c3401d66e7636)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:22.921891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40f5c8-1816-4c8b-baf1-ce991dd142b7 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:22.924723 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 118/231] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:22 2022] GET /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:22.931909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40f5c8-1816-4c8b-baf1-ce991dd142b7 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:22.946757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f0608e-b52f-4e4c-9fa4-dd9fd8011fef None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:23.031846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e719a-3236-45a1-88cf-c0779fa8c0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:23.033384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e719a-3236-45a1-88cf-c0779fa8c0af None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:23.034129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e719a-3236-45a1-88cf-c0779fa8c0af None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:23.035382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e719a-3236-45a1-88cf-c0779fa8c0af None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:23.037389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e719a-3236-45a1-88cf-c0779fa8c0af None admin] RBAC: Authorizing `identity:get_project(project_id=5acc2b4d60e244298e1c3401d66e7636)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:23.044484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:23.046210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e719a-3236-45a1-88cf-c0779fa8c0af None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:23.048973 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 114/232] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:22 2022] GET /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636 => generated 302 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:23.054929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e719a-3236-45a1-88cf-c0779fa8c0af None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:23.072774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40f5c8-1816-4c8b-baf1-ce991dd142b7 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:23.114736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728f5e0-e5d2-4b6e-b917-af09aedbc160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:23.115405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728f5e0-e5d2-4b6e-b917-af09aedbc160 None admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:23.115689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728f5e0-e5d2-4b6e-b917-af09aedbc160 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:23.115961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728f5e0-e5d2-4b6e-b917-af09aedbc160 None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636/users/3cfaf7eaf2004320b41cb14e9fbb3ae3/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:23.117916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728f5e0-e5d2-4b6e-b917-af09aedbc160 None admin] RBAC: Authorizing `identity:create_grant(project_id=5acc2b4d60e244298e1c3401d66e7636, user_id=3cfaf7eaf2004320b41cb14e9fbb3ae3, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:23.119689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728f5e0-e5d2-4b6e-b917-af09aedbc160 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:23.140265 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 119/233] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:32:22 2022] PUT /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636/users/3cfaf7eaf2004320b41cb14e9fbb3ae3/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:32:23.204932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e719a-3236-45a1-88cf-c0779fa8c0af None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:23.224119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:23.242969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f7b39-d6ca-49d5-8910-bed7f563057f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:23.243643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f7b39-d6ca-49d5-8910-bed7f563057f None admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:23.243927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f7b39-d6ca-49d5-8910-bed7f563057f None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:23.244206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f7b39-d6ca-49d5-8910-bed7f563057f None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636/users/3cfaf7eaf2004320b41cb14e9fbb3ae3/roles/8b7b6e0028b04be8912abda33d3ef729` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:23.246461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f7b39-d6ca-49d5-8910-bed7f563057f None admin] RBAC: Authorizing `identity:create_grant(project_id=5acc2b4d60e244298e1c3401d66e7636, user_id=3cfaf7eaf2004320b41cb14e9fbb3ae3, role_id=8b7b6e0028b04be8912abda33d3ef729)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:23.248041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f7b39-d6ca-49d5-8910-bed7f563057f None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:23.265178 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 115/234] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:32:23 2022] PUT /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636/users/3cfaf7eaf2004320b41cb14e9fbb3ae3/roles/8b7b6e0028b04be8912abda33d3ef729 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:32:24.231001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0381c6e-b86d-4fa5-97fe-4d1dff74c8da None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:24.231548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0381c6e-b86d-4fa5-97fe-4d1dff74c8da None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:24.231548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0381c6e-b86d-4fa5-97fe-4d1dff74c8da None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:24.233157 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 120/235] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:24.238259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271f024-8d98-40c5-a0d3-69f88450bb92 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:24.238694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271f024-8d98-40c5-a0d3-69f88450bb92 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:24.238881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271f024-8d98-40c5-a0d3-69f88450bb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:24.263490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4271f024-8d98-40c5-a0d3-69f88450bb92 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:24.264697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4271f024-8d98-40c5-a0d3-69f88450bb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:24.353711 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 116/236] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:24.402133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b4d7d-9ba0-4e5e-8996-381256a3204d None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:24.403180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b4d7d-9ba0-4e5e-8996-381256a3204d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:24.403180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b4d7d-9ba0-4e5e-8996-381256a3204d None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:24.404152 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 121/237] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:32:24.412194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab28f6-5a77-4013-a11a-d01d4c63dfa7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:24.412488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab28f6-5a77-4013-a11a-d01d4c63dfa7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:24.412773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab28f6-5a77-4013-a11a-d01d4c63dfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:24.438247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7eab28f6-5a77-4013-a11a-d01d4c63dfa7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:24.439486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7eab28f6-5a77-4013-a11a-d01d4c63dfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:24.534968 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 117/238] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:24.802981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b4d7d-9ba0-4e5e-8996-381256a3204d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:24.934813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab28f6-5a77-4013-a11a-d01d4c63dfa7 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:24.939592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b4d7d-9ba0-4e5e-8996-381256a3204d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:24.980781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007c77d-e704-4954-8d25-f223b82aebdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:24.981421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007c77d-e704-4954-8d25-f223b82aebdf None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:24.981705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007c77d-e704-4954-8d25-f223b82aebdf None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:24.981980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007c77d-e704-4954-8d25-f223b82aebdf None admin] PATH_INFO: `/v3/domains/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:24.983472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007c77d-e704-4954-8d25-f223b82aebdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:24.985263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007c77d-e704-4954-8d25-f223b82aebdf None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:24.987983 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 122/239] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:32:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:24.993809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007c77d-e704-4954-8d25-f223b82aebdf None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:25.009439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:25.015338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:25.118621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab28f6-5a77-4013-a11a-d01d4c63dfa7 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:25.159796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c155f5-c771-4ecb-964e-3c8750796154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:25.160447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c155f5-c771-4ecb-964e-3c8750796154 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:25.160726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c155f5-c771-4ecb-964e-3c8750796154 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:25.160997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c155f5-c771-4ecb-964e-3c8750796154 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:25.174894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c155f5-c771-4ecb-964e-3c8750796154 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:25.180586 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-01c155f5-c771-4ecb-964e-3c8750796154 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 01 20:32:25.180586 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:25.180586 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:25.180586 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:25.180586 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:25.180586 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:25.180586 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:25.180586 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:25.180586 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:25.180586 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:25.180586 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:25.180586 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:25.180586 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:25.180586 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:25.180586 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:25.180586 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:25.181511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:25.181511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:25.181511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:25.181511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:25.181511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:25.181511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:25.181511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:25.181511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:25.181511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:25.181511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:25.181511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:25.181511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:25.181511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:25.181511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:25.181511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:25.181511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:25.182303 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:25.182303 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:25.182303 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:25.182303 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:25.182303 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:25.182303 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:25.182303 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:25.182303 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:25.182303 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:25.182303 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:25.182303 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 01 20:32:25.182303 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:32:25.182303 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 118/240] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:32:25.188831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c155f5-c771-4ecb-964e-3c8750796154 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:25.272713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007c77d-e704-4954-8d25-f223b82aebdf None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:25.349937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dacd1eb-9d50-4d88-a471-ec25617e632b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:25.350582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacd1eb-9d50-4d88-a471-ec25617e632b None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:25.350868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacd1eb-9d50-4d88-a471-ec25617e632b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:25.351141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacd1eb-9d50-4d88-a471-ec25617e632b None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:25.351953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dacd1eb-9d50-4d88-a471-ec25617e632b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:25.354677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dacd1eb-9d50-4d88-a471-ec25617e632b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:25.368239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:25.375208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c155f5-c771-4ecb-964e-3c8750796154 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:25.407499 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 123/241] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Oct 1 20:32:24 2022] POST /identity/v3/users => generated 293 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:32:25.430896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a989de25-c97b-48b0-9e2c-8ba55a8ca5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:25.431538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989de25-c97b-48b0-9e2c-8ba55a8ca5d8 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:25.431819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989de25-c97b-48b0-9e2c-8ba55a8ca5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:25.432090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989de25-c97b-48b0-9e2c-8ba55a8ca5d8 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:25.432830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a989de25-c97b-48b0-9e2c-8ba55a8ca5d8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:25.433139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a989de25-c97b-48b0-9e2c-8ba55a8ca5d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:25.434669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a989de25-c97b-48b0-9e2c-8ba55a8ca5d8 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:25.445556 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 119/242] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:25.451406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dacd1eb-9d50-4d88-a471-ec25617e632b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:25.561303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:25.591489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadd0b3-8f04-459d-aaf4-aaf900b7e0db None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:25.591797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadd0b3-8f04-459d-aaf4-aaf900b7e0db None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:25.592037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadd0b3-8f04-459d-aaf4-aaf900b7e0db None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:25.594655 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 120/243] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:25.598945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d76fc0-df62-439c-b669-e7620e03977b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:25.599225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d76fc0-df62-439c-b669-e7620e03977b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:25.599499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d76fc0-df62-439c-b669-e7620e03977b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:25.615432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dacd1eb-9d50-4d88-a471-ec25617e632b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:25.639272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-68d76fc0-df62-439c-b669-e7620e03977b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:25.642404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-68d76fc0-df62-439c-b669-e7620e03977b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:25.691534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7d488-a0cd-4e52-b57d-3c9db8becaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:25.692618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7d488-a0cd-4e52-b57d-3c9db8becaab None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:25.694086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7d488-a0cd-4e52-b57d-3c9db8becaab None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:25.694852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7d488-a0cd-4e52-b57d-3c9db8becaab None admin] PATH_INFO: `/v3/users/3d1456f8bb8541dd9fb30986addedab7` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:25.724386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7d488-a0cd-4e52-b57d-3c9db8becaab None admin] RBAC: Authorizing `identity:get_user(user_id=3d1456f8bb8541dd9fb30986addedab7)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:25.726032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7d488-a0cd-4e52-b57d-3c9db8becaab None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:25.732211 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 124/244] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:25 2022] GET /identity/v3/users/3d1456f8bb8541dd9fb30986addedab7 => generated 297 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:25.733688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7d488-a0cd-4e52-b57d-3c9db8becaab None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:25.756683 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 121/245] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:32:25.846745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7d488-a0cd-4e52-b57d-3c9db8becaab None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:25.867497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:25.884079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db241c7d-a028-4a68-9e35-7a33967ebe70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:25.884748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db241c7d-a028-4a68-9e35-7a33967ebe70 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:25.885120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db241c7d-a028-4a68-9e35-7a33967ebe70 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:25.885396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db241c7d-a028-4a68-9e35-7a33967ebe70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:25.964903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db241c7d-a028-4a68-9e35-7a33967ebe70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:25.966377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db241c7d-a028-4a68-9e35-7a33967ebe70 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:26.076367 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 125/246] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:26.083075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d76fc0-df62-439c-b669-e7620e03977b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:26.124602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db241c7d-a028-4a68-9e35-7a33967ebe70 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:26.152579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:26.215148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:26.217806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d76fc0-df62-439c-b669-e7620e03977b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:26.247911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db241c7d-a028-4a68-9e35-7a33967ebe70 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:26.260282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f21320-e693-439d-ade5-7aecb78bd3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:26.260510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f21320-e693-439d-ade5-7aecb78bd3cf None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:26.260713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f21320-e693-439d-ade5-7aecb78bd3cf None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:26.260913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f21320-e693-439d-ade5-7aecb78bd3cf None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:26.262921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f21320-e693-439d-ade5-7aecb78bd3cf None admin] RBAC: Authorizing `identity:get_project(project_id=5acc2b4d60e244298e1c3401d66e7636)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:26.264390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f21320-e693-439d-ade5-7aecb78bd3cf None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:26.268284 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 122/247] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:26 2022] GET /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:26.279511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f21320-e693-439d-ade5-7aecb78bd3cf None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:26.311245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98208f11-d402-4c57-ad3b-0ab1b3ee3559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:26.312522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98208f11-d402-4c57-ad3b-0ab1b3ee3559 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:26.313011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98208f11-d402-4c57-ad3b-0ab1b3ee3559 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:26.313460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98208f11-d402-4c57-ad3b-0ab1b3ee3559 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:26.329839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98208f11-d402-4c57-ad3b-0ab1b3ee3559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:26.335781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-98208f11-d402-4c57-ad3b-0ab1b3ee3559 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:26.335781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:26.335781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:26.335781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:26.335781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:26.335781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:26.335781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:26.335781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:26.335781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:26.335781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:26.335781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:26.335781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:26.335781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:26.335781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:26.335781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:26.335781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:26.336676 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:26.336676 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:26.336676 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:26.336676 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:26.336676 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:26.336676 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:26.336676 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:26.336676 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:26.336676 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:26.336676 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:26.336676 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:26.336676 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:26.336676 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:26.336676 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:26.336676 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:26.336676 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:26.337478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:26.337478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:26.337478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:26.337478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:26.337478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:26.337478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:26.337478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:26.337478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:26.337478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:26.337478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:26.337478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:26.337478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:26.339885 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 126/248] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:32:26.345919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98208f11-d402-4c57-ad3b-0ab1b3ee3559 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:26.401742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f21320-e693-439d-ade5-7aecb78bd3cf None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:26.439603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:26.458226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98284f16-7976-4e94-a1d2-1beee6bd953e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:26.459201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98284f16-7976-4e94-a1d2-1beee6bd953e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:26.459652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98284f16-7976-4e94-a1d2-1beee6bd953e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:26.460078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98284f16-7976-4e94-a1d2-1beee6bd953e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:26.460956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98284f16-7976-4e94-a1d2-1beee6bd953e None admin] RBAC: Adding query filter params (role.id=be70f76ecf3f465fb09d46916db16671, scope.project.id=5acc2b4d60e244298e1c3401d66e7636, user.id=3d1456f8bb8541dd9fb30986addedab7) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:26.461390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98208f11-d402-4c57-ad3b-0ab1b3ee3559 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:26.461554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98284f16-7976-4e94-a1d2-1beee6bd953e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:26.463265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98284f16-7976-4e94-a1d2-1beee6bd953e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:26.470724 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 123/249] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:32:26 2022] GET /identity/v3/role_assignments?user.id=3d1456f8bb8541dd9fb30986addedab7&scope.project.id=5acc2b4d60e244298e1c3401d66e7636&role.id=be70f76ecf3f465fb09d46916db16671 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:26.504354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd43e3d-5094-4035-8088-5f83d964d4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:26.505187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd43e3d-5094-4035-8088-5f83d964d4d5 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:26.505572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd43e3d-5094-4035-8088-5f83d964d4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:26.505992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd43e3d-5094-4035-8088-5f83d964d4d5 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:26.506835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd43e3d-5094-4035-8088-5f83d964d4d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:26.507265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd43e3d-5094-4035-8088-5f83d964d4d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:26.508759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd43e3d-5094-4035-8088-5f83d964d4d5 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:26.519614 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 127/250] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:26.525831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98284f16-7976-4e94-a1d2-1beee6bd953e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:26.562439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:26.663896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98284f16-7976-4e94-a1d2-1beee6bd953e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:26.677470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cbbb5-a6ab-44ab-a68a-688e26a8dc6b None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:26.678099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cbbb5-a6ab-44ab-a68a-688e26a8dc6b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:26.678639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cbbb5-a6ab-44ab-a68a-688e26a8dc6b None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:26.680628 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 128/251] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:26.686190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e15e3-5672-42e8-904b-ea759a8715a4 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:26.686722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e15e3-5672-42e8-904b-ea759a8715a4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:26.687137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e15e3-5672-42e8-904b-ea759a8715a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:26.711344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fb5ef-05eb-4d0e-98cd-166d49e91e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:26.712022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fb5ef-05eb-4d0e-98cd-166d49e91e8e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:26.712306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fb5ef-05eb-4d0e-98cd-166d49e91e8e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:26.712581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fb5ef-05eb-4d0e-98cd-166d49e91e8e None admin] PATH_INFO: `/v3/users/f46b4b1ea99846afa5d0d196ef227a4d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:26.713784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fb5ef-05eb-4d0e-98cd-166d49e91e8e None admin] RBAC: Authorizing `identity:get_user(user_id=f46b4b1ea99846afa5d0d196ef227a4d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:26.715311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fb5ef-05eb-4d0e-98cd-166d49e91e8e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:26.716996 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 124/252] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:26 2022] GET /identity/v3/users/f46b4b1ea99846afa5d0d196ef227a4d => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:26.723984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fb5ef-05eb-4d0e-98cd-166d49e91e8e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:26.739374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e51e15e3-5672-42e8-904b-ea759a8715a4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:26.740912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e51e15e3-5672-42e8-904b-ea759a8715a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:26.828456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:26.845398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fb5ef-05eb-4d0e-98cd-166d49e91e8e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:26.880821 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 129/253] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:26.919913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e2699-9aca-4277-957e-3085bab2505b None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:26.920328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77899cb-f7bb-4b76-ba4e-60c5fac3e280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:26.920477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e2699-9aca-4277-957e-3085bab2505b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:26.920705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e2699-9aca-4277-957e-3085bab2505b None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:26.921563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77899cb-f7bb-4b76-ba4e-60c5fac3e280 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:26.921976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77899cb-f7bb-4b76-ba4e-60c5fac3e280 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:26.922358 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 130/254] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:32:26.922617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77899cb-f7bb-4b76-ba4e-60c5fac3e280 None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:26.924412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77899cb-f7bb-4b76-ba4e-60c5fac3e280 None admin] RBAC: Authorizing `identity:get_project(project_id=5acc2b4d60e244298e1c3401d66e7636)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:26.926232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77899cb-f7bb-4b76-ba4e-60c5fac3e280 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:26.927180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bc3fa-7bff-4e31-a89d-b6e633131222 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:26.927467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bc3fa-7bff-4e31-a89d-b6e633131222 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:26.927746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bc3fa-7bff-4e31-a89d-b6e633131222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:26.928791 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 125/255] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:26 2022] GET /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:26.934006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77899cb-f7bb-4b76-ba4e-60c5fac3e280 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:26.953736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-789bc3fa-7bff-4e31-a89d-b6e633131222 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:26.955812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-789bc3fa-7bff-4e31-a89d-b6e633131222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:27.033817 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 131/256] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:32:27.128515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77899cb-f7bb-4b76-ba4e-60c5fac3e280 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:27.177086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf946c-0380-4d94-a762-b2e75d68d830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:27.177751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf946c-0380-4d94-a762-b2e75d68d830 None admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:27.178025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf946c-0380-4d94-a762-b2e75d68d830 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:27.178290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf946c-0380-4d94-a762-b2e75d68d830 None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636/users/f46b4b1ea99846afa5d0d196ef227a4d/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:27.180286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf946c-0380-4d94-a762-b2e75d68d830 None admin] RBAC: Authorizing `identity:create_grant(project_id=5acc2b4d60e244298e1c3401d66e7636, user_id=f46b4b1ea99846afa5d0d196ef227a4d, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:27.181776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf946c-0380-4d94-a762-b2e75d68d830 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:27.198193 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 126/257] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:32:26 2022] PUT /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636/users/f46b4b1ea99846afa5d0d196ef227a4d/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:32:27.417006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bc3fa-7bff-4e31-a89d-b6e633131222 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:27.419895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:27.532575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bc3fa-7bff-4e31-a89d-b6e633131222 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:27.570798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199a9a4-cba0-434e-8518-c37e345c2d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:27.571974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199a9a4-cba0-434e-8518-c37e345c2d68 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:27.572428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199a9a4-cba0-434e-8518-c37e345c2d68 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:27.572861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199a9a4-cba0-434e-8518-c37e345c2d68 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:27.585688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199a9a4-cba0-434e-8518-c37e345c2d68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:27.590835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-0199a9a4-cba0-434e-8518-c37e345c2d68 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:27.590835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:27.590835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:27.590835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:27.590835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:27.590835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:27.590835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:27.590835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:27.590835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:27.590835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:27.590835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:27.590835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:27.590835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:27.590835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:27.590835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:27.590835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:27.591888 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:27.591888 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:27.591888 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:27.591888 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:27.591888 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:27.591888 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:27.591888 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:27.591888 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:27.591888 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:27.591888 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:27.591888 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:27.591888 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:27.591888 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:27.591888 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:27.591888 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:27.591888 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:27.592887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:27.592887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:27.592887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:27.592887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:27.592887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:27.592887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:27.592887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:27.592887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:27.592887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:27.592887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:27.592887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:27.592887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:27.594857 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 132/258] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:32:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:27.601286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf946c-0380-4d94-a762-b2e75d68d830 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:27.632453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:27.660895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199a9a4-cba0-434e-8518-c37e345c2d68 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:27.738152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf946c-0380-4d94-a762-b2e75d68d830 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:27.771568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:27.776226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf1b2a-395f-4ccf-9d39-6b953b0f2be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:27.777176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf1b2a-395f-4ccf-9d39-6b953b0f2be6 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:27.777657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf1b2a-395f-4ccf-9d39-6b953b0f2be6 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:27.778122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf1b2a-395f-4ccf-9d39-6b953b0f2be6 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:27.779020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf1b2a-395f-4ccf-9d39-6b953b0f2be6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:27.779522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf1b2a-395f-4ccf-9d39-6b953b0f2be6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:27.781127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf1b2a-395f-4ccf-9d39-6b953b0f2be6 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:27.797344 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 127/259] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:32:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:27.803897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf1b2a-395f-4ccf-9d39-6b953b0f2be6 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:27.842068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199a9a4-cba0-434e-8518-c37e345c2d68 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:27.899709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c6a3c-0164-4f78-a633-74daff4731a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:27.900624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c6a3c-0164-4f78-a633-74daff4731a3 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:27.901044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c6a3c-0164-4f78-a633-74daff4731a3 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:27.901453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c6a3c-0164-4f78-a633-74daff4731a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:27.915637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c6a3c-0164-4f78-a633-74daff4731a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:27.921445 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-a44c6a3c-0164-4f78-a633-74daff4731a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:32:27.921445 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:27.921445 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:27.921445 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:27.921445 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:27.921445 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:27.921445 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:27.921445 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:27.921445 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:27.921445 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:27.921445 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:27.921445 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:27.921445 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:27.921445 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:27.921445 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:27.921445 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:27.922338 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:27.922338 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:27.922338 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:27.922338 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:27.922338 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:27.922338 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:27.922338 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:27.922338 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:27.922338 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:27.922338 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:27.922338 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:27.922338 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:27.922338 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:27.922338 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:27.922338 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:27.922338 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:27.923154 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:27.923154 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:27.923154 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:27.923154 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:27.923154 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:27.923154 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:27.923154 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:27.923154 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:27.923154 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:27.923154 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:27.923154 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:32:27.923154 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:27.932630 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 133/260] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:32:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:32:27.938575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c6a3c-0164-4f78-a633-74daff4731a3 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:27.944675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:27.988492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf1b2a-395f-4ccf-9d39-6b953b0f2be6 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:28.091755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f2f27-05f0-4e6a-8cc9-28ba0ea4a60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:28.092410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f2f27-05f0-4e6a-8cc9-28ba0ea4a60b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:28.092692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f2f27-05f0-4e6a-8cc9-28ba0ea4a60b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:28.092967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f2f27-05f0-4e6a-8cc9-28ba0ea4a60b None admin] PATH_INFO: `/v3/users/3cfaf7eaf2004320b41cb14e9fbb3ae3` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:28.094130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f2f27-05f0-4e6a-8cc9-28ba0ea4a60b None admin] RBAC: Authorizing `identity:get_user(user_id=3cfaf7eaf2004320b41cb14e9fbb3ae3)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:28.095286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c6a3c-0164-4f78-a633-74daff4731a3 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:28.095604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f2f27-05f0-4e6a-8cc9-28ba0ea4a60b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:28.097466 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 128/261] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:27 2022] GET /identity/v3/users/3cfaf7eaf2004320b41cb14e9fbb3ae3 => generated 283 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:28.107476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f2f27-05f0-4e6a-8cc9-28ba0ea4a60b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:28.136103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:28.148947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95ed9c-2f3b-4747-ba62-dbd99b63c760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:28.149562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95ed9c-2f3b-4747-ba62-dbd99b63c760 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:28.149838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95ed9c-2f3b-4747-ba62-dbd99b63c760 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:28.150106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95ed9c-2f3b-4747-ba62-dbd99b63c760 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:28.150821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95ed9c-2f3b-4747-ba62-dbd99b63c760 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:28.151680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95ed9c-2f3b-4747-ba62-dbd99b63c760 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:28.153059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95ed9c-2f3b-4747-ba62-dbd99b63c760 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:28.165053 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 134/262] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:28.171403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95ed9c-2f3b-4747-ba62-dbd99b63c760 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:28.287822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95ed9c-2f3b-4747-ba62-dbd99b63c760 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:28.322971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f2f27-05f0-4e6a-8cc9-28ba0ea4a60b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:28.330728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316644d6-e850-4e75-9a5c-fb2feab2972a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:28.331512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316644d6-e850-4e75-9a5c-fb2feab2972a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:28.331790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316644d6-e850-4e75-9a5c-fb2feab2972a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:28.332058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316644d6-e850-4e75-9a5c-fb2feab2972a None admin] PATH_INFO: `/v3/users/3cfaf7eaf2004320b41cb14e9fbb3ae3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:28.333326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316644d6-e850-4e75-9a5c-fb2feab2972a None admin] RBAC: Authorizing `identity:get_user(user_id=3cfaf7eaf2004320b41cb14e9fbb3ae3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:28.334901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316644d6-e850-4e75-9a5c-fb2feab2972a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:28.337883 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 135/263] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:28 2022] GET /identity/v3/users/3cfaf7eaf2004320b41cb14e9fbb3ae3 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:28.342522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316644d6-e850-4e75-9a5c-fb2feab2972a None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:28.365972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c0fb6-c96b-4bc6-8168-27671cd8589e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:28.367641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c0fb6-c96b-4bc6-8168-27671cd8589e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:28.367915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c0fb6-c96b-4bc6-8168-27671cd8589e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:28.368197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c0fb6-c96b-4bc6-8168-27671cd8589e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:28.415245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:28.428716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:28.502470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316644d6-e850-4e75-9a5c-fb2feab2972a None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:28.554202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c0fb6-c96b-4bc6-8168-27671cd8589e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:28.555885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c0fb6-c96b-4bc6-8168-27671cd8589e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:28.563761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1cd222-4bfb-493c-afed-da5e5853bd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:28.564566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1cd222-4bfb-493c-afed-da5e5853bd2b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:28.565005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1cd222-4bfb-493c-afed-da5e5853bd2b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:28.565514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1cd222-4bfb-493c-afed-da5e5853bd2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:28.659373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1cd222-4bfb-493c-afed-da5e5853bd2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:28.661084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1cd222-4bfb-493c-afed-da5e5853bd2b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:28.715942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:28.744730 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 129/264] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:28.750192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c0fb6-c96b-4bc6-8168-27671cd8589e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:28.770609 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 136/265] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 01 20:32:28.773983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1cd222-4bfb-493c-afed-da5e5853bd2b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:28.857342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:28.879227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1cd222-4bfb-493c-afed-da5e5853bd2b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:28.914576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9508b1f-6834-43a7-b196-7abc9be1b3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:28.915561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9508b1f-6834-43a7-b196-7abc9be1b3c3 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:28.916032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9508b1f-6834-43a7-b196-7abc9be1b3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:28.916493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9508b1f-6834-43a7-b196-7abc9be1b3c3 None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:28.917823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9508b1f-6834-43a7-b196-7abc9be1b3c3 None admin] RBAC: Authorizing `identity:get_project(project_id=5acc2b4d60e244298e1c3401d66e7636)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:28.919495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9508b1f-6834-43a7-b196-7abc9be1b3c3 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:28.926439 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 137/266] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:28 2022] GET /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:28.927764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9508b1f-6834-43a7-b196-7abc9be1b3c3 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:28.952446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:28.972042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c0fb6-c96b-4bc6-8168-27671cd8589e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:29.031688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9508b1f-6834-43a7-b196-7abc9be1b3c3 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:29.068282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11741d6-589a-43ad-b103-03f86acd1968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:29.069256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11741d6-589a-43ad-b103-03f86acd1968 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:29.069747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11741d6-589a-43ad-b103-03f86acd1968 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:29.070205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11741d6-589a-43ad-b103-03f86acd1968 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:29.070373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4cfce-9c7e-4741-9729-76b21ba40d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:29.071267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11741d6-589a-43ad-b103-03f86acd1968 None admin] RBAC: Adding query filter params (role.id=1c50896511054a5f994bee3d99fa0619, scope.project.id=5acc2b4d60e244298e1c3401d66e7636, user.id=3cfaf7eaf2004320b41cb14e9fbb3ae3) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:29.071697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4cfce-9c7e-4741-9729-76b21ba40d74 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:29.072013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11741d6-589a-43ad-b103-03f86acd1968 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:29.072340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4cfce-9c7e-4741-9729-76b21ba40d74 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:29.072752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4cfce-9c7e-4741-9729-76b21ba40d74 None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:29.073695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11741d6-589a-43ad-b103-03f86acd1968 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:29.074477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4cfce-9c7e-4741-9729-76b21ba40d74 None admin] RBAC: Authorizing `identity:get_project(project_id=5acc2b4d60e244298e1c3401d66e7636)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:29.076089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4cfce-9c7e-4741-9729-76b21ba40d74 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:29.078169 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 130/267] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:28 2022] GET /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636 => generated 302 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:29.083129 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 138/268] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:32:28 2022] GET /identity/v3/role_assignments?user.id=3cfaf7eaf2004320b41cb14e9fbb3ae3&scope.project.id=5acc2b4d60e244298e1c3401d66e7636&role.id=1c50896511054a5f994bee3d99fa0619 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:29.084235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4cfce-9c7e-4741-9729-76b21ba40d74 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:29.225939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4cfce-9c7e-4741-9729-76b21ba40d74 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:29.263155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:29.288304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93acad-fbb9-4f04-8feb-b62eb74f541e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:29.288935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93acad-fbb9-4f04-8feb-b62eb74f541e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:29.289212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93acad-fbb9-4f04-8feb-b62eb74f541e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:29.289486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93acad-fbb9-4f04-8feb-b62eb74f541e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:29.290288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd93acad-fbb9-4f04-8feb-b62eb74f541e None admin] RBAC: Adding query filter params (role.id=8b7b6e0028b04be8912abda33d3ef729, scope.project.id=5acc2b4d60e244298e1c3401d66e7636, user.id=3cfaf7eaf2004320b41cb14e9fbb3ae3) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:29.290690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd93acad-fbb9-4f04-8feb-b62eb74f541e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:29.292227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd93acad-fbb9-4f04-8feb-b62eb74f541e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:29.299059 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 131/269] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:32:29 2022] GET /identity/v3/role_assignments?user.id=3cfaf7eaf2004320b41cb14e9fbb3ae3&scope.project.id=5acc2b4d60e244298e1c3401d66e7636&role.id=8b7b6e0028b04be8912abda33d3ef729 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:29.463534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dccfd3-5d63-4d1e-a2ac-0aaceb3d819b None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:29.464043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dccfd3-5d63-4d1e-a2ac-0aaceb3d819b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:29.464115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dccfd3-5d63-4d1e-a2ac-0aaceb3d819b None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:29.465707 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 139/270] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:29.471841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ef41f-d506-4efd-8502-857cf2d2a1c7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:29.472176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ef41f-d506-4efd-8502-857cf2d2a1c7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:29.472402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ef41f-d506-4efd-8502-857cf2d2a1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:29.499949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-db5ef41f-d506-4efd-8502-857cf2d2a1c7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:29.501223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-db5ef41f-d506-4efd-8502-857cf2d2a1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:29.594538 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 132/271] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:29.757015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b041a3-afcb-4f00-bbde-2878653ecc19 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:29.757306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b041a3-afcb-4f00-bbde-2878653ecc19 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:29.757487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b041a3-afcb-4f00-bbde-2878653ecc19 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:29.759182 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 140/272] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:29.764673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999734b-4efc-4461-a985-05b4bfcc7767 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:29.765148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999734b-4efc-4461-a985-05b4bfcc7767 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:29.765220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999734b-4efc-4461-a985-05b4bfcc7767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:29.786022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c999734b-4efc-4461-a985-05b4bfcc7767 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:29.787261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c999734b-4efc-4461-a985-05b4bfcc7767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:29.868602 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 133/273] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:29.979789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b041a3-afcb-4f00-bbde-2878653ecc19 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:30.083289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:30.100264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b041a3-afcb-4f00-bbde-2878653ecc19 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:30.134893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c762e3-0cda-46ee-a95c-e7c44e09c1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:30.135522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c762e3-0cda-46ee-a95c-e7c44e09c1fe None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:30.135797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c762e3-0cda-46ee-a95c-e7c44e09c1fe None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:30.136121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c762e3-0cda-46ee-a95c-e7c44e09c1fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:30.147251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c762e3-0cda-46ee-a95c-e7c44e09c1fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:30.152082 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-e1c762e3-0cda-46ee-a95c-e7c44e09c1fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:30.152082 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:30.152082 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:30.152082 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:30.152082 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:30.152082 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:30.152082 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:30.152082 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:30.152082 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:30.152082 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:30.152082 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:30.152082 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:30.152082 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:30.152082 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:30.152082 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:30.152082 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:30.152938 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:30.152938 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:30.152938 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:30.152938 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:30.152938 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:30.152938 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:30.152938 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:30.152938 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:30.152938 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:30.152938 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:30.152938 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:30.152938 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:30.152938 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:30.152938 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:30.152938 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:30.152938 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:30.153761 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:30.153761 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:30.153761 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:30.153761 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:30.153761 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:30.153761 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:30.153761 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:30.153761 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:30.153761 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:30.153761 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:30.153761 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:30.153761 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:30.153761 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 141/274] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:30.160263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999734b-4efc-4461-a985-05b4bfcc7767 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:30.203366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:30.247411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c762e3-0cda-46ee-a95c-e7c44e09c1fe None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:30.266832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999734b-4efc-4461-a985-05b4bfcc7767 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:30.305272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc72d2-641b-4826-bc38-1f1039237365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:30.306238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc72d2-641b-4826-bc38-1f1039237365 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:30.306546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc72d2-641b-4826-bc38-1f1039237365 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:30.306882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc72d2-641b-4826-bc38-1f1039237365 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:30.307566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc72d2-641b-4826-bc38-1f1039237365 None admin] RBAC: Adding query filter params (name=member) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:30.307935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc72d2-641b-4826-bc38-1f1039237365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:30.309831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc72d2-641b-4826-bc38-1f1039237365 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:30.320906 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 134/275] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:30.327096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc72d2-641b-4826-bc38-1f1039237365 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:30.367442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c762e3-0cda-46ee-a95c-e7c44e09c1fe None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:30.410230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3f020-b815-417d-b768-3ceb35798fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:30.411167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3f020-b815-417d-b768-3ceb35798fb4 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:30.411276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3f020-b815-417d-b768-3ceb35798fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:30.411506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3f020-b815-417d-b768-3ceb35798fb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:30.412665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3f020-b815-417d-b768-3ceb35798fb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:30.414428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3f020-b815-417d-b768-3ceb35798fb4 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:30.421656 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 142/276] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:32:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:30.421656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786eb2bd-8e1a-43bd-83e8-2577737cdef7 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:30.421656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786eb2bd-8e1a-43bd-83e8-2577737cdef7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:30.421656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786eb2bd-8e1a-43bd-83e8-2577737cdef7 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:30.421656 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 143/277] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:30.423230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786eb2bd-8e1a-43bd-83e8-2577737cdef7 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:30.441290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:30.450795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc72d2-641b-4826-bc38-1f1039237365 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:30.452597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:30.486766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857506a1-d5c5-4885-b2bd-c032038d59dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:30.487432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857506a1-d5c5-4885-b2bd-c032038d59dc None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:30.487712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857506a1-d5c5-4885-b2bd-c032038d59dc None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:30.487983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857506a1-d5c5-4885-b2bd-c032038d59dc None admin] PATH_INFO: `/v3/users/5f91819bb3494bec83d19f211d69aaa3` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:30.502389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857506a1-d5c5-4885-b2bd-c032038d59dc None admin] RBAC: Authorizing `identity:get_user(user_id=5f91819bb3494bec83d19f211d69aaa3)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:30.504407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857506a1-d5c5-4885-b2bd-c032038d59dc None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:30.506043 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 135/278] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:30 2022] GET /identity/v3/users/5f91819bb3494bec83d19f211d69aaa3 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:30.507272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c7a75-def6-474c-85ad-3c95df8344e5 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:30.507559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c7a75-def6-474c-85ad-3c95df8344e5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:30.507826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c7a75-def6-474c-85ad-3c95df8344e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:30.526780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ce5c7a75-def6-474c-85ad-3c95df8344e5 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:30.527943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ce5c7a75-def6-474c-85ad-3c95df8344e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:30.540091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786eb2bd-8e1a-43bd-83e8-2577737cdef7 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:30.582654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa89cb-295a-4041-a308-9a5e49a5d926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:30.583278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa89cb-295a-4041-a308-9a5e49a5d926 None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:30.583557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa89cb-295a-4041-a308-9a5e49a5d926 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:30.584206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa89cb-295a-4041-a308-9a5e49a5d926 None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:30.584526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa89cb-295a-4041-a308-9a5e49a5d926 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:30.585953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa89cb-295a-4041-a308-9a5e49a5d926 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:30.600440 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 136/279] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:30.601174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c7a75-def6-474c-85ad-3c95df8344e5 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:30.626572 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 144/280] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Oct 1 20:32:30 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:32:30.712761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:30.725469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c7a75-def6-474c-85ad-3c95df8344e5 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:30.765886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19614623-6d0a-47e6-bddb-c5c06ca872a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:30.766564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19614623-6d0a-47e6-bddb-c5c06ca872a6 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:30.766826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19614623-6d0a-47e6-bddb-c5c06ca872a6 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:30.767102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19614623-6d0a-47e6-bddb-c5c06ca872a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:30.767839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19614623-6d0a-47e6-bddb-c5c06ca872a6 None admin] RBAC: Adding query filter params (role.id=17f3aeacc5034f3891cc6556da651c1b, user.id=5f91819bb3494bec83d19f211d69aaa3, scope.system=all) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:30.768150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19614623-6d0a-47e6-bddb-c5c06ca872a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:30.770190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19614623-6d0a-47e6-bddb-c5c06ca872a6 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:30.776378 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 137/281] 10.222.0.17 () {56 vars in 1308 bytes} [Sat Oct 1 20:32:30 2022] GET /identity/v3/role_assignments?user.id=5f91819bb3494bec83d19f211d69aaa3&scope.system=all&role.id=17f3aeacc5034f3891cc6556da651c1b => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:30.831446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8b0b6-d398-4427-a58f-228978b428fb None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:30.831758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8b0b6-d398-4427-a58f-228978b428fb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:30.832051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8b0b6-d398-4427-a58f-228978b428fb None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:30.833982 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 145/282] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:30.839564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ea7f5-ee7f-4b4b-8659-e2f57573a76b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:30.839879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ea7f5-ee7f-4b4b-8659-e2f57573a76b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:30.840124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ea7f5-ee7f-4b4b-8659-e2f57573a76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:30.860054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-911ea7f5-ee7f-4b4b-8659-e2f57573a76b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:30.861287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-911ea7f5-ee7f-4b4b-8659-e2f57573a76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:30.932396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8b0b6-d398-4427-a58f-228978b428fb None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:30.942130 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 138/283] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:31.019406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:31.060922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8b0b6-d398-4427-a58f-228978b428fb None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:31.099198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c6d155-a6cb-4c46-a7b3-404e60c2a58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:31.099756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6d155-a6cb-4c46-a7b3-404e60c2a58e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:31.100034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6d155-a6cb-4c46-a7b3-404e60c2a58e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:31.100305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6d155-a6cb-4c46-a7b3-404e60c2a58e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:31.110967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c6d155-a6cb-4c46-a7b3-404e60c2a58e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:31.115850 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-13c6d155-a6cb-4c46-a7b3-404e60c2a58e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 01 20:32:31.115850 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:31.115850 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:31.115850 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:31.115850 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:31.115850 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:31.115850 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:31.115850 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:31.115850 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:31.115850 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:31.115850 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:31.115850 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:31.115850 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:31.115850 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:31.115850 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:31.115850 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:31.116713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:31.116713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:31.116713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:31.116713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:31.116713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:31.116713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:31.116713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:31.116713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:31.116713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:31.116713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:31.116713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:31.116713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:31.116713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:31.116713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:31.116713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:31.116713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:31.117899 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:31.117899 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:31.117899 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:31.117899 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:31.117899 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:31.117899 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:31.117899 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:31.117899 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:31.117899 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:31.117899 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:31.117899 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 01 20:32:31.117899 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:31.117899 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 146/284] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:31.125665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ea7f5-ee7f-4b4b-8659-e2f57573a76b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:31.228594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:31.245780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ea7f5-ee7f-4b4b-8659-e2f57573a76b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:31.284616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9864af5-8342-4f18-bc64-fd0a0b7487a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:31.285225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9864af5-8342-4f18-bc64-fd0a0b7487a5 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:31.285504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9864af5-8342-4f18-bc64-fd0a0b7487a5 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:31.285775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9864af5-8342-4f18-bc64-fd0a0b7487a5 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:31.286437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9864af5-8342-4f18-bc64-fd0a0b7487a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:31.286769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9864af5-8342-4f18-bc64-fd0a0b7487a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:31.288208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9864af5-8342-4f18-bc64-fd0a0b7487a5 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:31.291264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c6d155-a6cb-4c46-a7b3-404e60c2a58e None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:31.297216 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 139/285] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:31.303009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9864af5-8342-4f18-bc64-fd0a0b7487a5 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:31.449630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c6d155-a6cb-4c46-a7b3-404e60c2a58e None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:31.463391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9864af5-8342-4f18-bc64-fd0a0b7487a5 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:31.502865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:31.516720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f99020-8dc0-47ad-9902-58f01440b1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:31.517446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f99020-8dc0-47ad-9902-58f01440b1d6 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:31.517732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f99020-8dc0-47ad-9902-58f01440b1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:31.518008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f99020-8dc0-47ad-9902-58f01440b1d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:31.528951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41822e-9135-491f-949c-84f3292ac7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:31.529586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41822e-9135-491f-949c-84f3292ac7c0 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:31.529864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41822e-9135-491f-949c-84f3292ac7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:31.530134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41822e-9135-491f-949c-84f3292ac7c0 None admin] PATH_INFO: `/v3/users/3d1456f8bb8541dd9fb30986addedab7` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:31.531483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed41822e-9135-491f-949c-84f3292ac7c0 None admin] RBAC: Authorizing `identity:get_user(user_id=3d1456f8bb8541dd9fb30986addedab7)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:31.532927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed41822e-9135-491f-949c-84f3292ac7c0 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:31.534652 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 140/286] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:31 2022] GET /identity/v3/users/3d1456f8bb8541dd9fb30986addedab7 => generated 297 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:31.545241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41822e-9135-491f-949c-84f3292ac7c0 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:31.548632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f99020-8dc0-47ad-9902-58f01440b1d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:31.560146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-f2f99020-8dc0-47ad-9902-58f01440b1d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:31.560146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:31.560146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:31.560146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:31.560146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:31.560146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:31.560146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:31.560146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:31.560146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:31.560146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:31.560146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:31.560146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:31.560146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:31.560146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:31.560146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:31.560146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:31.561039 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:31.561039 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:31.561039 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:31.561039 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:31.561039 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:31.561039 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:31.561039 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:31.561039 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:31.561039 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:31.561039 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:31.561039 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:31.561039 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:31.561039 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:31.561039 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:31.561039 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:31.561039 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:31.562244 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:31.562244 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:31.562244 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:31.562244 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:31.562244 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:31.562244 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:31.562244 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:31.562244 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:31.562244 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:31.562244 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:31.562244 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:31.562244 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:31.562244 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 147/287] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:32:31.567883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f99020-8dc0-47ad-9902-58f01440b1d6 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:31.579208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:31.675806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f99020-8dc0-47ad-9902-58f01440b1d6 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:31.711007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a40ae-c564-490c-8631-e454e6716f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:31.711186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41822e-9135-491f-949c-84f3292ac7c0 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:31.711627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a40ae-c564-490c-8631-e454e6716f8b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:31.711808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a40ae-c564-490c-8631-e454e6716f8b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:31.712083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a40ae-c564-490c-8631-e454e6716f8b None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:31.712758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a40ae-c564-490c-8631-e454e6716f8b None admin] RBAC: Adding query filter params (name=member) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:31.713065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a40ae-c564-490c-8631-e454e6716f8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:31.714650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a40ae-c564-490c-8631-e454e6716f8b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:31.723508 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 148/288] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:31.728724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a40ae-c564-490c-8631-e454e6716f8b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:31.775577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e30d09-700d-455c-906f-0477dafec197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:31.776986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e30d09-700d-455c-906f-0477dafec197 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:31.777464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e30d09-700d-455c-906f-0477dafec197 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:31.778000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e30d09-700d-455c-906f-0477dafec197 None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:31.779336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e30d09-700d-455c-906f-0477dafec197 None admin] RBAC: Authorizing `identity:get_project(project_id=5acc2b4d60e244298e1c3401d66e7636)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:31.786512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e30d09-700d-455c-906f-0477dafec197 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:31.789133 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 141/289] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:31 2022] GET /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:31.796137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:31.798012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e30d09-700d-455c-906f-0477dafec197 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:31.849495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a40ae-c564-490c-8631-e454e6716f8b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:31.884879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13085f-d2c9-4cec-b628-193a8d707131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:31.885579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13085f-d2c9-4cec-b628-193a8d707131 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:31.885859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13085f-d2c9-4cec-b628-193a8d707131 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:31.886135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13085f-d2c9-4cec-b628-193a8d707131 None admin] PATH_INFO: `/v3/users/f46b4b1ea99846afa5d0d196ef227a4d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:31.891626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13085f-d2c9-4cec-b628-193a8d707131 None admin] RBAC: Authorizing `identity:get_user(user_id=f46b4b1ea99846afa5d0d196ef227a4d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:31.893136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13085f-d2c9-4cec-b628-193a8d707131 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:31.898143 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 149/290] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:31 2022] GET /identity/v3/users/f46b4b1ea99846afa5d0d196ef227a4d => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:31.900356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13085f-d2c9-4cec-b628-193a8d707131 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:31.959302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:31.983050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e30d09-700d-455c-906f-0477dafec197 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:32.053331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1720da3-c775-49e5-ae54-d1aeafe18194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:32.053999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1720da3-c775-49e5-ae54-d1aeafe18194 None admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:32.054279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1720da3-c775-49e5-ae54-d1aeafe18194 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:32.054577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1720da3-c775-49e5-ae54-d1aeafe18194 None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636/users/3d1456f8bb8541dd9fb30986addedab7/roles/be70f76ecf3f465fb09d46916db16671` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:32.060190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1720da3-c775-49e5-ae54-d1aeafe18194 None admin] RBAC: Authorizing `identity:create_grant(project_id=5acc2b4d60e244298e1c3401d66e7636, user_id=3d1456f8bb8541dd9fb30986addedab7, role_id=be70f76ecf3f465fb09d46916db16671)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:32.061638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1720da3-c775-49e5-ae54-d1aeafe18194 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:32.088614 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 142/291] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:32:31 2022] PUT /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636/users/3d1456f8bb8541dd9fb30986addedab7/roles/be70f76ecf3f465fb09d46916db16671 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:32:32.095259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13085f-d2c9-4cec-b628-193a8d707131 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:32.115058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:32.136055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d15de-5c08-49aa-b71d-76dcdc4eb03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:32.136668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d15de-5c08-49aa-b71d-76dcdc4eb03f None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:32.136946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d15de-5c08-49aa-b71d-76dcdc4eb03f None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:32.137220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d15de-5c08-49aa-b71d-76dcdc4eb03f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:32.213033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d15de-5c08-49aa-b71d-76dcdc4eb03f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:32.214628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d15de-5c08-49aa-b71d-76dcdc4eb03f None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:32.325857 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 150/292] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 01 20:32:32.331782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1720da3-c775-49e5-ae54-d1aeafe18194 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:32.468240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1720da3-c775-49e5-ae54-d1aeafe18194 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:32.520482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48aa314-8d7c-4250-b0ab-bc57ac67e2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:32.521135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48aa314-8d7c-4250-b0ab-bc57ac67e2bd None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:32.521418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48aa314-8d7c-4250-b0ab-bc57ac67e2bd None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:32.521693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48aa314-8d7c-4250-b0ab-bc57ac67e2bd None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:32.522888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48aa314-8d7c-4250-b0ab-bc57ac67e2bd None admin] RBAC: Authorizing `identity:get_project(project_id=5acc2b4d60e244298e1c3401d66e7636)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:32.524338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48aa314-8d7c-4250-b0ab-bc57ac67e2bd None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:32.526206 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 143/293] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:32 2022] GET /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:32.532099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d15de-5c08-49aa-b71d-76dcdc4eb03f None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:32.574930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:32.655140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d15de-5c08-49aa-b71d-76dcdc4eb03f None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:32.701934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685ef12-810d-4f23-a6a9-832bd079add4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:32.702570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685ef12-810d-4f23-a6a9-832bd079add4 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:32.702852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685ef12-810d-4f23-a6a9-832bd079add4 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:32.703125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685ef12-810d-4f23-a6a9-832bd079add4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:32.703860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685ef12-810d-4f23-a6a9-832bd079add4 None admin] RBAC: Adding query filter params (role.id=17f3aeacc5034f3891cc6556da651c1b, scope.project.id=5acc2b4d60e244298e1c3401d66e7636, user.id=f46b4b1ea99846afa5d0d196ef227a4d) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:32.704168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685ef12-810d-4f23-a6a9-832bd079add4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:32.705614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685ef12-810d-4f23-a6a9-832bd079add4 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:32.712125 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 151/294] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:32:32 2022] GET /identity/v3/role_assignments?user.id=f46b4b1ea99846afa5d0d196ef227a4d&scope.project.id=5acc2b4d60e244298e1c3401d66e7636&role.id=17f3aeacc5034f3891cc6556da651c1b => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:33.518817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780827f-f1ca-460b-8241-62bf03c8fe89 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:33.519535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780827f-f1ca-460b-8241-62bf03c8fe89 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:33.519535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780827f-f1ca-460b-8241-62bf03c8fe89 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:33.520975 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 144/295] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:33.526021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21008cc5-d669-40d9-b63a-ad90bb8cb879 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:33.526303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21008cc5-d669-40d9-b63a-ad90bb8cb879 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:33.526601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21008cc5-d669-40d9-b63a-ad90bb8cb879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:33.547623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-21008cc5-d669-40d9-b63a-ad90bb8cb879 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:33.548817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-21008cc5-d669-40d9-b63a-ad90bb8cb879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:33.620901 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 152/296] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:32:33.851959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7256f-1de2-44e7-a2e7-6233e8cb2c56 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:33.852314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7256f-1de2-44e7-a2e7-6233e8cb2c56 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:33.852501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7256f-1de2-44e7-a2e7-6233e8cb2c56 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:33.854026 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 145/297] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:33.859277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8f224-83ff-47b4-8777-94f237c9b900 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:33.859570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8f224-83ff-47b4-8777-94f237c9b900 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:33.859850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8f224-83ff-47b4-8777-94f237c9b900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:33.878823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-98c8f224-83ff-47b4-8777-94f237c9b900 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:33.880012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-98c8f224-83ff-47b4-8777-94f237c9b900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:33.884344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717809b-22e6-48c7-b38a-976e6b2d3c5c None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:33.884630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717809b-22e6-48c7-b38a-976e6b2d3c5c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:33.884936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717809b-22e6-48c7-b38a-976e6b2d3c5c None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:33.886690 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 146/298] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:32:33.888154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:33.894329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70923e4-b664-45e4-99fa-a5cf943e81ae None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:33.894905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70923e4-b664-45e4-99fa-a5cf943e81ae None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:33.895381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70923e4-b664-45e4-99fa-a5cf943e81ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:33.914758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c70923e4-b664-45e4-99fa-a5cf943e81ae None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:33.916053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c70923e4-b664-45e4-99fa-a5cf943e81ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:33.919899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:33.970656 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 153/299] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:33.973258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c8f224-83ff-47b4-8777-94f237c9b900 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:33.995068 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 147/300] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:32:34.081935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c8f224-83ff-47b4-8777-94f237c9b900 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:34.116620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e3649-62db-4cd7-8a9c-8f88ab01d885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:34.117179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e3649-62db-4cd7-8a9c-8f88ab01d885 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:34.117457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e3649-62db-4cd7-8a9c-8f88ab01d885 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:34.117726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e3649-62db-4cd7-8a9c-8f88ab01d885 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:34.128560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e3649-62db-4cd7-8a9c-8f88ab01d885 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:34.133516 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-374e3649-62db-4cd7-8a9c-8f88ab01d885 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:34.133516 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:34.133516 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:34.133516 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:34.133516 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:34.133516 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:34.133516 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:34.133516 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:34.133516 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:34.133516 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:34.133516 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:34.133516 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:34.133516 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:34.133516 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:34.133516 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:34.133516 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:34.134388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:34.134388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:34.134388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:34.134388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:34.134388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:34.134388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:34.134388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:34.134388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:34.134388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:34.134388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:34.134388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:34.134388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:34.134388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:34.134388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:34.134388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:34.134388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:34.135221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:34.135221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:34.135221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:34.135221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:34.135221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:34.135221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:34.135221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:34.135221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:34.135221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:34.135221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:34.135221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:34.135221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:34.135221 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 154/301] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:34.140543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70923e4-b664-45e4-99fa-a5cf943e81ae None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:34.243629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70923e4-b664-45e4-99fa-a5cf943e81ae None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:34.283623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b891f7a-ea3c-4511-881b-c9b1aeb9139c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:34.284755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b891f7a-ea3c-4511-881b-c9b1aeb9139c None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:34.285034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b891f7a-ea3c-4511-881b-c9b1aeb9139c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:34.285305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b891f7a-ea3c-4511-881b-c9b1aeb9139c None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:34.285972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b891f7a-ea3c-4511-881b-c9b1aeb9139c None admin] RBAC: Adding query filter params (name=member) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:34.286275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b891f7a-ea3c-4511-881b-c9b1aeb9139c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:34.287705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b891f7a-ea3c-4511-881b-c9b1aeb9139c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:34.296324 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 148/302] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:34.301677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e3649-62db-4cd7-8a9c-8f88ab01d885 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:34.305146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:34.334218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b891f7a-ea3c-4511-881b-c9b1aeb9139c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:34.337513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:34.412262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e3649-62db-4cd7-8a9c-8f88ab01d885 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:34.449365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0be899-a89b-416a-ba60-26cdc38974a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:34.450280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0be899-a89b-416a-ba60-26cdc38974a4 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:34.450591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0be899-a89b-416a-ba60-26cdc38974a4 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:34.450869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0be899-a89b-416a-ba60-26cdc38974a4 None admin] PATH_INFO: `/v3/users/5f91819bb3494bec83d19f211d69aaa3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:34.451408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b891f7a-ea3c-4511-881b-c9b1aeb9139c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:34.452002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0be899-a89b-416a-ba60-26cdc38974a4 None admin] RBAC: Authorizing `identity:get_user(user_id=5f91819bb3494bec83d19f211d69aaa3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:34.453432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0be899-a89b-416a-ba60-26cdc38974a4 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:34.455223 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 155/303] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:34 2022] GET /identity/v3/users/5f91819bb3494bec83d19f211d69aaa3 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:34.459825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0be899-a89b-416a-ba60-26cdc38974a4 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:34.486455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027b2cb-a825-4c0b-9d61-c75561c86775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:34.487216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027b2cb-a825-4c0b-9d61-c75561c86775 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:34.487539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027b2cb-a825-4c0b-9d61-c75561c86775 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:34.487818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027b2cb-a825-4c0b-9d61-c75561c86775 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:34.488932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027b2cb-a825-4c0b-9d61-c75561c86775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:34.490373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027b2cb-a825-4c0b-9d61-c75561c86775 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:34.492143 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 149/304] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:32:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:34.495313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027b2cb-a825-4c0b-9d61-c75561c86775 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:34.538255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:34.574191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:34.575482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0be899-a89b-416a-ba60-26cdc38974a4 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:34.611160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027b2cb-a825-4c0b-9d61-c75561c86775 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:34.611494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406884d-bde0-48b1-9585-5bf38edde456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:34.612666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406884d-bde0-48b1-9585-5bf38edde456 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:34.612946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406884d-bde0-48b1-9585-5bf38edde456 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:34.613216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406884d-bde0-48b1-9585-5bf38edde456 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:34.624471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406884d-bde0-48b1-9585-5bf38edde456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:34.630525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-d406884d-bde0-48b1-9585-5bf38edde456 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 01 20:32:34.630525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:34.630525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:34.630525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:34.630525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:34.630525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:34.630525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:34.630525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:34.630525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:34.630525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:34.630525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:34.630525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:34.630525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:34.630525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:34.630525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:34.630525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:34.631423 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:34.631423 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:34.631423 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:34.631423 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:34.631423 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:34.631423 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:34.631423 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:34.631423 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:34.631423 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:34.631423 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:34.631423 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:34.631423 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:34.631423 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:34.631423 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:34.631423 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:34.631423 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:34.632234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:34.632234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:34.632234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:34.632234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:34.632234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:34.632234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:34.632234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:34.632234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:34.632234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:34.632234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:34.632234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 01 20:32:34.632234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:34.636512 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 156/305] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:34.637156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406884d-bde0-48b1-9585-5bf38edde456 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:34.650693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb774d-14a4-4693-a3fc-8d44c8a3a816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:34.651650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb774d-14a4-4693-a3fc-8d44c8a3a816 None admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:34.651935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb774d-14a4-4693-a3fc-8d44c8a3a816 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:34.652208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb774d-14a4-4693-a3fc-8d44c8a3a816 None admin] PATH_INFO: `/v3/system/users/5f91819bb3494bec83d19f211d69aaa3/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:34.654187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb774d-14a4-4693-a3fc-8d44c8a3a816 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5f91819bb3494bec83d19f211d69aaa3, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:34.655760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb774d-14a4-4693-a3fc-8d44c8a3a816 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:34.665171 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 150/306] 10.222.0.17 () {58 vars in 1334 bytes} [Sat Oct 1 20:32:34 2022] PUT /identity/v3/system/users/5f91819bb3494bec83d19f211d69aaa3/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:32:34.665920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb774d-14a4-4693-a3fc-8d44c8a3a816 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:34.763274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406884d-bde0-48b1-9585-5bf38edde456 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:34.813466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:34.822200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c22725-3695-4fd9-a1c9-a324a3b0c537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:34.822997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c22725-3695-4fd9-a1c9-a324a3b0c537 None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:34.823356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c22725-3695-4fd9-a1c9-a324a3b0c537 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:34.823744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c22725-3695-4fd9-a1c9-a324a3b0c537 None admin] PATH_INFO: `/v3/groups` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:34.824501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c22725-3695-4fd9-a1c9-a324a3b0c537 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:34.826909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c22725-3695-4fd9-a1c9-a324a3b0c537 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:34.841663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb774d-14a4-4693-a3fc-8d44c8a3a816 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:34.851406 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 157/307] 10.222.0.17 () {60 vars in 1134 bytes} [Sat Oct 1 20:32:34 2022] POST /identity/v3/groups => generated 229 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:32:34.891738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5ec47-a025-4684-b2b1-a3e815ff6302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:34.892377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5ec47-a025-4684-b2b1-a3e815ff6302 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:34.892659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5ec47-a025-4684-b2b1-a3e815ff6302 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:34.892936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5ec47-a025-4684-b2b1-a3e815ff6302 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:34.894352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5ec47-a025-4684-b2b1-a3e815ff6302 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:34.894689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5ec47-a025-4684-b2b1-a3e815ff6302 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:34.896121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5ec47-a025-4684-b2b1-a3e815ff6302 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:34.897856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:34.906559 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 151/308] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:34.912614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c22725-3695-4fd9-a1c9-a324a3b0c537 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:34.984264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73f695-df72-4cb6-b119-7d512f3ff461 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:34.984562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73f695-df72-4cb6-b119-7d512f3ff461 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:34.984847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73f695-df72-4cb6-b119-7d512f3ff461 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:34.986561 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 152/309] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:34.993840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa9bcd-1098-4786-b9ed-aac89a1f7079 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:34.994138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa9bcd-1098-4786-b9ed-aac89a1f7079 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:34.994389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa9bcd-1098-4786-b9ed-aac89a1f7079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:35.016706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-28aa9bcd-1098-4786-b9ed-aac89a1f7079 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:35.018231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-28aa9bcd-1098-4786-b9ed-aac89a1f7079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:35.054957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c22725-3695-4fd9-a1c9-a324a3b0c537 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:35.096684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1202d-3518-45c5-b69d-33c916763dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:35.098131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1202d-3518-45c5-b69d-33c916763dcb None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:35.098131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1202d-3518-45c5-b69d-33c916763dcb None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:35.098131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1202d-3518-45c5-b69d-33c916763dcb None admin] PATH_INFO: `/v3/users/2ad75097dc8c491aa307375a3bcce91f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:35.117616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e1202d-3518-45c5-b69d-33c916763dcb None admin] RBAC: Authorizing `identity:get_user(user_id=2ad75097dc8c491aa307375a3bcce91f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:35.118667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e1202d-3518-45c5-b69d-33c916763dcb None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:35.120350 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 158/310] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:34 2022] GET /identity/v3/users/2ad75097dc8c491aa307375a3bcce91f => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:35.121303 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 153/311] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:35.126041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1202d-3518-45c5-b69d-33c916763dcb None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:35.179991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:35.239283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1202d-3518-45c5-b69d-33c916763dcb None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:35.275370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12666e7-e937-4742-8e7b-a173d8229bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:35.275979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12666e7-e937-4742-8e7b-a173d8229bcd None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:35.276256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12666e7-e937-4742-8e7b-a173d8229bcd None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:35.276524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12666e7-e937-4742-8e7b-a173d8229bcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:35.277246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12666e7-e937-4742-8e7b-a173d8229bcd None admin] RBAC: Adding query filter params (role.id=be70f76ecf3f465fb09d46916db16671, user.id=2ad75097dc8c491aa307375a3bcce91f, scope.system=all) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:35.277551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12666e7-e937-4742-8e7b-a173d8229bcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:35.279001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12666e7-e937-4742-8e7b-a173d8229bcd None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:35.285295 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 159/312] 10.222.0.17 () {56 vars in 1308 bytes} [Sat Oct 1 20:32:35 2022] GET /identity/v3/role_assignments?user.id=2ad75097dc8c491aa307375a3bcce91f&scope.system=all&role.id=be70f76ecf3f465fb09d46916db16671 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:35.464167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa9bcd-1098-4786-b9ed-aac89a1f7079 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:35.572592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa9bcd-1098-4786-b9ed-aac89a1f7079 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:35.599970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:35.609610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd62866-04d0-434d-8b0a-29493bfb1414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:35.610242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd62866-04d0-434d-8b0a-29493bfb1414 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:35.610540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd62866-04d0-434d-8b0a-29493bfb1414 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:35.610818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd62866-04d0-434d-8b0a-29493bfb1414 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:35.621708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd62866-04d0-434d-8b0a-29493bfb1414 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:35.626660 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-5dd62866-04d0-434d-8b0a-29493bfb1414 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 01 20:32:35.626660 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:35.626660 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:35.626660 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:35.626660 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:35.626660 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:35.626660 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:35.626660 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:35.626660 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:35.626660 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:35.626660 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:35.626660 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:35.626660 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:35.626660 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:35.626660 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:35.626660 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:35.628091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:35.628091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:35.628091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:35.628091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:35.628091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:35.628091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:35.628091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:35.628091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:35.628091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:35.628091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:35.628091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:35.628091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:35.628091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:35.628091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:35.628091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:35.628091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:35.629287 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:35.629287 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:35.629287 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:35.629287 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:35.629287 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:35.629287 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:35.629287 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:35.629287 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:35.629287 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:35.629287 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:35.629287 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 01 20:32:35.629287 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:32:35.629287 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 154/313] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:35.637973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12666e7-e937-4742-8e7b-a173d8229bcd None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:35.747308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12666e7-e937-4742-8e7b-a173d8229bcd None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:35.761822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:35.783519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7655c-20d0-409c-9353-df644fba964b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:35.784650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7655c-20d0-409c-9353-df644fba964b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:35.784925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7655c-20d0-409c-9353-df644fba964b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:35.785192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7655c-20d0-409c-9353-df644fba964b None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:35.785850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7655c-20d0-409c-9353-df644fba964b None admin] RBAC: Adding query filter params (name=reader) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:35.786211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7655c-20d0-409c-9353-df644fba964b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:35.787598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7655c-20d0-409c-9353-df644fba964b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:35.796268 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 160/314] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:35.803784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd62866-04d0-434d-8b0a-29493bfb1414 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:35.916702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd62866-04d0-434d-8b0a-29493bfb1414 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:35.953523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c92256-b3ea-43fa-bb9c-250592dc8127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:35.954542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c92256-b3ea-43fa-bb9c-250592dc8127 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:35.955043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c92256-b3ea-43fa-bb9c-250592dc8127 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:35.955511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c92256-b3ea-43fa-bb9c-250592dc8127 None admin] PATH_INFO: `/v3/users/3d1456f8bb8541dd9fb30986addedab7` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:35.957222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c92256-b3ea-43fa-bb9c-250592dc8127 None admin] RBAC: Authorizing `identity:get_user(user_id=3d1456f8bb8541dd9fb30986addedab7)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:35.959029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c92256-b3ea-43fa-bb9c-250592dc8127 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:35.965343 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 155/315] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:35 2022] GET /identity/v3/users/3d1456f8bb8541dd9fb30986addedab7 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:35.966935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7655c-20d0-409c-9353-df644fba964b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:36.086463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7655c-20d0-409c-9353-df644fba964b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:36.122331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f4e73-546d-45bf-b879-88c036005927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:36.123366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f4e73-546d-45bf-b879-88c036005927 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:36.123843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f4e73-546d-45bf-b879-88c036005927 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:36.124306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f4e73-546d-45bf-b879-88c036005927 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:36.174073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:36.199768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f4e73-546d-45bf-b879-88c036005927 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:36.201240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f4e73-546d-45bf-b879-88c036005927 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:36.305889 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 161/316] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:36.307286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c92256-b3ea-43fa-bb9c-250592dc8127 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:36.366276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:36.416258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c92256-b3ea-43fa-bb9c-250592dc8127 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:36.453855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af0c2a-06f5-4171-a429-1c45123c702d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:36.454862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af0c2a-06f5-4171-a429-1c45123c702d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:36.455345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af0c2a-06f5-4171-a429-1c45123c702d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:36.455947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af0c2a-06f5-4171-a429-1c45123c702d None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:36.457254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af0c2a-06f5-4171-a429-1c45123c702d None admin] RBAC: Authorizing `identity:get_project(project_id=5acc2b4d60e244298e1c3401d66e7636)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:36.458924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af0c2a-06f5-4171-a429-1c45123c702d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:36.461182 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 156/317] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:36 2022] GET /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:36.466737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f4e73-546d-45bf-b879-88c036005927 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:36.544187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:36.569906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f4e73-546d-45bf-b879-88c036005927 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:36.604942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50205e-7bda-4024-ba0a-a94bfd51fc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:36.605542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50205e-7bda-4024-ba0a-a94bfd51fc38 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:36.605820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50205e-7bda-4024-ba0a-a94bfd51fc38 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:36.606087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50205e-7bda-4024-ba0a-a94bfd51fc38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:36.606835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50205e-7bda-4024-ba0a-a94bfd51fc38 None admin] RBAC: Adding query filter params (role.id=be70f76ecf3f465fb09d46916db16671, scope.project.id=5acc2b4d60e244298e1c3401d66e7636, user.id=3d1456f8bb8541dd9fb30986addedab7) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:36.607140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50205e-7bda-4024-ba0a-a94bfd51fc38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:36.608546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50205e-7bda-4024-ba0a-a94bfd51fc38 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:36.614788 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 162/318] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:32:36 2022] GET /identity/v3/role_assignments?user.id=3d1456f8bb8541dd9fb30986addedab7&scope.project.id=5acc2b4d60e244298e1c3401d66e7636&role.id=be70f76ecf3f465fb09d46916db16671 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:37.264189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b7c61-6e19-4098-84d0-990cd51c748c None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:37.264739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b7c61-6e19-4098-84d0-990cd51c748c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:37.264739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b7c61-6e19-4098-84d0-990cd51c748c None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:37.266940 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 157/319] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:37.272070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496572c-a71d-4408-ae93-edc3621c4c71 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:37.272450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496572c-a71d-4408-ae93-edc3621c4c71 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:37.272613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496572c-a71d-4408-ae93-edc3621c4c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:37.291064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0496572c-a71d-4408-ae93-edc3621c4c71 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:37.292254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0496572c-a71d-4408-ae93-edc3621c4c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:37.374350 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 163/320] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:37.436616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08328e61-10e5-43d6-99bb-39fa5bb312d2 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:37.436940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08328e61-10e5-43d6-99bb-39fa5bb312d2 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:37.437173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08328e61-10e5-43d6-99bb-39fa5bb312d2 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:37.438803 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 158/321] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:37.444529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00366f7c-404a-4af5-8855-79b46f60eed7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:37.444865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00366f7c-404a-4af5-8855-79b46f60eed7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:37.445077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00366f7c-404a-4af5-8855-79b46f60eed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:37.466106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-00366f7c-404a-4af5-8855-79b46f60eed7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:37.467339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-00366f7c-404a-4af5-8855-79b46f60eed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:37.529363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:37.536838 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 164/322] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:37.707564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08328e61-10e5-43d6-99bb-39fa5bb312d2 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:37.809221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08328e61-10e5-43d6-99bb-39fa5bb312d2 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:37.832243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:37.844787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefc2a2-6113-4d77-bd72-07f78d70b8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:37.845411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefc2a2-6113-4d77-bd72-07f78d70b8d4 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:37.845744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefc2a2-6113-4d77-bd72-07f78d70b8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:37.846074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefc2a2-6113-4d77-bd72-07f78d70b8d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:37.856738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefc2a2-6113-4d77-bd72-07f78d70b8d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:37.859668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00366f7c-404a-4af5-8855-79b46f60eed7 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:37.861629 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-0cefc2a2-6113-4d77-bd72-07f78d70b8d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:37.861629 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:37.861629 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:37.861629 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:37.861629 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:37.861629 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:37.861629 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:37.861629 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:37.861629 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:37.861629 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:37.861629 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:37.861629 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:37.861629 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:37.861629 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:37.861629 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:37.861629 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:37.862635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:37.862635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:37.862635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:37.862635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:37.862635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:37.862635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:37.862635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:37.862635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:37.862635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:37.862635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:37.862635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:37.862635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:37.862635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:37.862635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:37.862635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:37.862635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:37.863428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:37.863428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:37.863428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:37.863428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:37.863428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:37.863428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:37.863428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:37.863428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:37.863428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:37.863428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:37.863428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:37.863428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:32:37.864384 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 159/323] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:37.868422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefc2a2-6113-4d77-bd72-07f78d70b8d4 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:37.964155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00366f7c-404a-4af5-8855-79b46f60eed7 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:37.967165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefc2a2-6113-4d77-bd72-07f78d70b8d4 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:37.996717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f55fa3-3c86-4e20-8433-cb3c7177bf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:37.997333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f55fa3-3c86-4e20-8433-cb3c7177bf72 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:37.997609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f55fa3-3c86-4e20-8433-cb3c7177bf72 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:37.997876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f55fa3-3c86-4e20-8433-cb3c7177bf72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:37.999019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f55fa3-3c86-4e20-8433-cb3c7177bf72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:37.999440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1a355-6c27-4d1a-a5a1-abd96318ccf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:38.000038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1a355-6c27-4d1a-a5a1-abd96318ccf4 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:38.000315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1a355-6c27-4d1a-a5a1-abd96318ccf4 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:38.000421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f55fa3-3c86-4e20-8433-cb3c7177bf72 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:38.000584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1a355-6c27-4d1a-a5a1-abd96318ccf4 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:38.001236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1a355-6c27-4d1a-a5a1-abd96318ccf4 None admin] RBAC: Adding query filter params (name=member) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:38.001535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1a355-6c27-4d1a-a5a1-abd96318ccf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:38.001968 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 165/324] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:32:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:38.006831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1a355-6c27-4d1a-a5a1-abd96318ccf4 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:38.006917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27866ffe-2c8d-4fd8-b866-037575847253 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:38.006917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27866ffe-2c8d-4fd8-b866-037575847253 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:38.006917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27866ffe-2c8d-4fd8-b866-037575847253 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:38.006917 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 166/325] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:38.010591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17839d7d-dca7-49df-8241-bba83851745b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:38.010871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17839d7d-dca7-49df-8241-bba83851745b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:38.011150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17839d7d-dca7-49df-8241-bba83851745b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:38.018359 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 160/326] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:38.018359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1a355-6c27-4d1a-a5a1-abd96318ccf4 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:38.034356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-17839d7d-dca7-49df-8241-bba83851745b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:38.035560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-17839d7d-dca7-49df-8241-bba83851745b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:38.065963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:38.095040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:38.103252 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 167/327] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:32:38.104082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17839d7d-dca7-49df-8241-bba83851745b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:38.129635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1a355-6c27-4d1a-a5a1-abd96318ccf4 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:38.165653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc1e32-505b-4e0b-8bb1-7a7554ef1837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:38.166257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc1e32-505b-4e0b-8bb1-7a7554ef1837 None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:38.166571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc1e32-505b-4e0b-8bb1-7a7554ef1837 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:38.166831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc1e32-505b-4e0b-8bb1-7a7554ef1837 None admin] PATH_INFO: `/v3/groups` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:38.167621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc1e32-505b-4e0b-8bb1-7a7554ef1837 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:38.169069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc1e32-505b-4e0b-8bb1-7a7554ef1837 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:38.186038 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 161/328] 10.222.0.17 () {60 vars in 1134 bytes} [Sat Oct 1 20:32:38 2022] POST /identity/v3/groups => generated 226 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:32:38.213955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17839d7d-dca7-49df-8241-bba83851745b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:38.252992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191e283-fb62-43ba-8fb6-027957145acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:38.253561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191e283-fb62-43ba-8fb6-027957145acc None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:38.253836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191e283-fb62-43ba-8fb6-027957145acc None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:38.254102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191e283-fb62-43ba-8fb6-027957145acc None admin] PATH_INFO: `/v3/users/5f91819bb3494bec83d19f211d69aaa3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:38.255374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191e283-fb62-43ba-8fb6-027957145acc None admin] RBAC: Authorizing `identity:get_user(user_id=5f91819bb3494bec83d19f211d69aaa3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:38.256842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191e283-fb62-43ba-8fb6-027957145acc None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:38.258881 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 168/329] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:38 2022] GET /identity/v3/users/5f91819bb3494bec83d19f211d69aaa3 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:38.264687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc1e32-505b-4e0b-8bb1-7a7554ef1837 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:38.374270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc1e32-505b-4e0b-8bb1-7a7554ef1837 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:38.376950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:38.415391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fefd58-8ad2-481f-bc86-8c66d495ca33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:38.416094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fefd58-8ad2-481f-bc86-8c66d495ca33 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:38.416443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fefd58-8ad2-481f-bc86-8c66d495ca33 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:38.416729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fefd58-8ad2-481f-bc86-8c66d495ca33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:38.417443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fefd58-8ad2-481f-bc86-8c66d495ca33 None admin] RBAC: Adding query filter params (role.id=17f3aeacc5034f3891cc6556da651c1b, user.id=5f91819bb3494bec83d19f211d69aaa3, scope.system=all) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:38.417743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fefd58-8ad2-481f-bc86-8c66d495ca33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:38.419224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fefd58-8ad2-481f-bc86-8c66d495ca33 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:38.431065 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 162/330] 10.222.0.17 () {56 vars in 1308 bytes} [Sat Oct 1 20:32:38 2022] GET /identity/v3/role_assignments?user.id=5f91819bb3494bec83d19f211d69aaa3&scope.system=all&role.id=17f3aeacc5034f3891cc6556da651c1b => generated 510 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:38.432552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191e283-fb62-43ba-8fb6-027957145acc None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:38.506350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:38.550819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191e283-fb62-43ba-8fb6-027957145acc None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:38.591072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff77aa5-d79b-4c90-afba-ee79b9151016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:38.592059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff77aa5-d79b-4c90-afba-ee79b9151016 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:38.592491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff77aa5-d79b-4c90-afba-ee79b9151016 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:38.592908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff77aa5-d79b-4c90-afba-ee79b9151016 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:38.604848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff77aa5-d79b-4c90-afba-ee79b9151016 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:38.610886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-4ff77aa5-d79b-4c90-afba-ee79b9151016 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 01 20:32:38.610886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:38.610886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:38.610886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:38.610886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:38.610886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:38.610886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:38.610886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:38.610886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:38.610886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:38.610886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:38.610886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:38.610886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:38.610886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:38.610886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:38.610886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:38.611987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:38.611987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:38.611987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:38.611987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:38.611987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:38.611987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:38.611987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:38.611987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:38.611987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:38.611987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:38.611987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:38.611987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:38.611987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:38.611987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:38.611987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:38.611987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:38.612798 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:38.612798 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:38.612798 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:38.612798 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:38.612798 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:38.612798 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:38.612798 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:38.612798 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:38.612798 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:38.612798 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:38.612798 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 01 20:32:38.612798 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:38.615151 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 169/331] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:32:38.620984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fefd58-8ad2-481f-bc86-8c66d495ca33 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:38.738174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fefd58-8ad2-481f-bc86-8c66d495ca33 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:38.774661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d7f8a-1f87-46c1-b634-46301e4a6314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:38.775197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d7f8a-1f87-46c1-b634-46301e4a6314 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:38.775475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d7f8a-1f87-46c1-b634-46301e4a6314 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:38.775747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d7f8a-1f87-46c1-b634-46301e4a6314 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:38.776405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d7f8a-1f87-46c1-b634-46301e4a6314 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:38.776705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d7f8a-1f87-46c1-b634-46301e4a6314 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:38.778084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d7f8a-1f87-46c1-b634-46301e4a6314 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:38.787315 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 163/332] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:38.793078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff77aa5-d79b-4c90-afba-ee79b9151016 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:38.900662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff77aa5-d79b-4c90-afba-ee79b9151016 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:38.925307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:38.937403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87418971-253d-44a5-b832-6eabe04cf118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:38.938040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87418971-253d-44a5-b832-6eabe04cf118 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:38.938320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87418971-253d-44a5-b832-6eabe04cf118 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:38.938624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87418971-253d-44a5-b832-6eabe04cf118 None admin] PATH_INFO: `/v3/users/2ad75097dc8c491aa307375a3bcce91f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:38.939769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87418971-253d-44a5-b832-6eabe04cf118 None admin] RBAC: Authorizing `identity:get_user(user_id=2ad75097dc8c491aa307375a3bcce91f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:38.942011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87418971-253d-44a5-b832-6eabe04cf118 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:38.943929 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 170/333] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:38 2022] GET /identity/v3/users/2ad75097dc8c491aa307375a3bcce91f => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:38.949746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d7f8a-1f87-46c1-b634-46301e4a6314 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:38.983386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:39.077396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d7f8a-1f87-46c1-b634-46301e4a6314 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:39.122957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f9353-c851-4d29-a663-744a021d87e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:39.123622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f9353-c851-4d29-a663-744a021d87e1 None admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:39.123902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f9353-c851-4d29-a663-744a021d87e1 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:39.124173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f9353-c851-4d29-a663-744a021d87e1 None admin] PATH_INFO: `/v3/system/users/2ad75097dc8c491aa307375a3bcce91f/roles/be70f76ecf3f465fb09d46916db16671` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:39.126416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f9353-c851-4d29-a663-744a021d87e1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2ad75097dc8c491aa307375a3bcce91f, role_id=be70f76ecf3f465fb09d46916db16671)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:39.128433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f9353-c851-4d29-a663-744a021d87e1 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:39.137935 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 164/334] 10.222.0.17 () {58 vars in 1334 bytes} [Sat Oct 1 20:32:38 2022] PUT /identity/v3/system/users/2ad75097dc8c491aa307375a3bcce91f/roles/be70f76ecf3f465fb09d46916db16671 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:32:41.079064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f28a34-ac81-4aec-aa2b-f5bc3853c72f None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:41.079999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f28a34-ac81-4aec-aa2b-f5bc3853c72f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:41.080515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f28a34-ac81-4aec-aa2b-f5bc3853c72f None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:41.082914 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 171/335] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:41.089037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d39644-fc26-4020-8348-968df02cc332 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:41.089868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d39644-fc26-4020-8348-968df02cc332 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:41.090352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d39644-fc26-4020-8348-968df02cc332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:41.109013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d3d39644-fc26-4020-8348-968df02cc332 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:41.110529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d3d39644-fc26-4020-8348-968df02cc332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:41.209583 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 165/336] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:41.622853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6a62e-3d87-4123-99a5-3c29c3cabe78 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:41.623256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6a62e-3d87-4123-99a5-3c29c3cabe78 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:41.623431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6a62e-3d87-4123-99a5-3c29c3cabe78 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:41.625015 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 172/337] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:41.632133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77463c9b-df35-420f-910d-d0df25dd61e7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:41.632683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77463c9b-df35-420f-910d-d0df25dd61e7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:41.633258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77463c9b-df35-420f-910d-d0df25dd61e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:41.655685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-77463c9b-df35-420f-910d-d0df25dd61e7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:41.657177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-77463c9b-df35-420f-910d-d0df25dd61e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:41.674425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:41.746362 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 166/338] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:32:41.789531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6a62e-3d87-4123-99a5-3c29c3cabe78 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:41.923231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6a62e-3d87-4123-99a5-3c29c3cabe78 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:41.963795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd03c83-3c83-4c5d-aa3f-f80ab26e0711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:41.964441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd03c83-3c83-4c5d-aa3f-f80ab26e0711 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:41.964723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd03c83-3c83-4c5d-aa3f-f80ab26e0711 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:41.965000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd03c83-3c83-4c5d-aa3f-f80ab26e0711 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:41.980704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd03c83-3c83-4c5d-aa3f-f80ab26e0711 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:41.985494 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-0dd03c83-3c83-4c5d-aa3f-f80ab26e0711 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:41.985494 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:41.985494 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:41.985494 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:41.985494 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:41.985494 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:41.985494 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:41.985494 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:41.985494 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:41.985494 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:41.985494 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:41.985494 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:41.985494 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:41.985494 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:41.985494 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:41.985494 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:41.986384 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:41.986384 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:41.986384 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:41.986384 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:41.986384 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:41.986384 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:41.986384 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:41.986384 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:41.986384 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:41.986384 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:41.986384 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:41.986384 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:41.986384 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:41.986384 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:41.986384 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:41.986384 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:41.987291 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:41.987291 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:41.987291 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:41.987291 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:41.987291 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:41.987291 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:41.987291 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:41.987291 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:41.987291 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:41.987291 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:41.987291 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:41.987291 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:41.990823 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 173/339] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:41.999604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77463c9b-df35-420f-910d-d0df25dd61e7 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:42.158950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77463c9b-df35-420f-910d-d0df25dd61e7 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:42.182115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd03c83-3c83-4c5d-aa3f-f80ab26e0711 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:42.203004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47e53a-0868-4b7a-b9a8-39db3cc1b7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:42.204012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47e53a-0868-4b7a-b9a8-39db3cc1b7c4 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:42.204290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47e53a-0868-4b7a-b9a8-39db3cc1b7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:42.204563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47e53a-0868-4b7a-b9a8-39db3cc1b7c4 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:42.206067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47e53a-0868-4b7a-b9a8-39db3cc1b7c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:42.206325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47e53a-0868-4b7a-b9a8-39db3cc1b7c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:42.210522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47e53a-0868-4b7a-b9a8-39db3cc1b7c4 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:42.222258 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 167/340] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:42.227985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47e53a-0868-4b7a-b9a8-39db3cc1b7c4 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:42.256821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:42.277343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:42.329957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd03c83-3c83-4c5d-aa3f-f80ab26e0711 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:42.372475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febefaf7-5184-4aff-9fa6-2bbfe02e3c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:42.373124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febefaf7-5184-4aff-9fa6-2bbfe02e3c25 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:42.373403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febefaf7-5184-4aff-9fa6-2bbfe02e3c25 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:42.373675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febefaf7-5184-4aff-9fa6-2bbfe02e3c25 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:42.378916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47e53a-0868-4b7a-b9a8-39db3cc1b7c4 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:42.385251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febefaf7-5184-4aff-9fa6-2bbfe02e3c25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:42.390523 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-febefaf7-5184-4aff-9fa6-2bbfe02e3c25 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:42.390523 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:42.390523 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:42.390523 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:42.390523 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:42.390523 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:42.390523 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:42.390523 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:42.390523 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:42.390523 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:42.390523 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:42.390523 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:42.390523 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:42.390523 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:42.390523 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:42.390523 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:42.391401 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:42.391401 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:42.391401 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:42.391401 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:42.391401 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:42.391401 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:42.391401 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:42.391401 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:42.391401 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:42.391401 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:42.391401 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:42.391401 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:42.391401 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:42.391401 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:42.391401 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:42.391401 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:42.392195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:42.392195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:42.392195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:42.392195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:42.392195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:42.392195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:42.392195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:42.392195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:42.392195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:42.392195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:42.392195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:42.392195 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:42.392195 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 174/341] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:32:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:32:42.392861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77e6d0-9147-4313-965c-5f34e7939825 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:42.393517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77e6d0-9147-4313-965c-5f34e7939825 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:42.393517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77e6d0-9147-4313-965c-5f34e7939825 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:42.394861 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 175/342] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:42.406303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be03f42-7f10-4c7e-93c8-e217deb575f2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:42.406641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be03f42-7f10-4c7e-93c8-e217deb575f2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:42.407004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be03f42-7f10-4c7e-93c8-e217deb575f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:42.433593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7188ff-ee13-412b-8d8d-812b279b41b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:42.434636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7188ff-ee13-412b-8d8d-812b279b41b6 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:42.435144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7188ff-ee13-412b-8d8d-812b279b41b6 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:42.435608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7188ff-ee13-412b-8d8d-812b279b41b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:42.439420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6be03f42-7f10-4c7e-93c8-e217deb575f2 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:42.441310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6be03f42-7f10-4c7e-93c8-e217deb575f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:42.543637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7188ff-ee13-412b-8d8d-812b279b41b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:42.545103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7188ff-ee13-412b-8d8d-812b279b41b6 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:42.563731 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 176/343] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:42.564709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be03f42-7f10-4c7e-93c8-e217deb575f2 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:42.578501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:42.657650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:42.771242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be03f42-7f10-4c7e-93c8-e217deb575f2 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:42.773023 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 168/344] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 01 20:32:42.775537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7188ff-ee13-412b-8d8d-812b279b41b6 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:42.815517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913cd25-9b56-4a95-90c5-b59e82607a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:42.816133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913cd25-9b56-4a95-90c5-b59e82607a6b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:42.816411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913cd25-9b56-4a95-90c5-b59e82607a6b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:42.816680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913cd25-9b56-4a95-90c5-b59e82607a6b None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:42.817355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913cd25-9b56-4a95-90c5-b59e82607a6b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:42.817659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913cd25-9b56-4a95-90c5-b59e82607a6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:42.819073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913cd25-9b56-4a95-90c5-b59e82607a6b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:42.829163 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 177/345] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:32:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:42.835440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913cd25-9b56-4a95-90c5-b59e82607a6b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:42.946834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913cd25-9b56-4a95-90c5-b59e82607a6b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:42.949545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:42.985854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883ccf4-1041-4d71-8c37-fa802d14c10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:42.986975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883ccf4-1041-4d71-8c37-fa802d14c10b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:42.987258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883ccf4-1041-4d71-8c37-fa802d14c10b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:42.987533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883ccf4-1041-4d71-8c37-fa802d14c10b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:43.024788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7188ff-ee13-412b-8d8d-812b279b41b6 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:43.036849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:43.064817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d883ccf4-1041-4d71-8c37-fa802d14c10b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:43.066287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d883ccf4-1041-4d71-8c37-fa802d14c10b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:43.103774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1817854-9e9a-4d1a-a132-b103b6c964e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:43.104420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1817854-9e9a-4d1a-a132-b103b6c964e0 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:43.104703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1817854-9e9a-4d1a-a132-b103b6c964e0 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:43.104981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1817854-9e9a-4d1a-a132-b103b6c964e0 None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:43.111202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1817854-9e9a-4d1a-a132-b103b6c964e0 None admin] RBAC: Authorizing `identity:get_project(project_id=a503020af843467a992a8df7f46d3fec)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:43.112696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1817854-9e9a-4d1a-a132-b103b6c964e0 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:43.114681 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 169/346] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:42 2022] GET /identity/v3/projects/a503020af843467a992a8df7f46d3fec => generated 298 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:43.115899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6015f2-eb79-49f5-8e14-15be9cabe18b None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:43.116181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6015f2-eb79-49f5-8e14-15be9cabe18b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:43.116455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6015f2-eb79-49f5-8e14-15be9cabe18b None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:43.117893 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 170/347] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:43.120907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6015f2-eb79-49f5-8e14-15be9cabe18b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:43.178113 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 178/348] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:43.179524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9eb35-00d3-42ef-8125-4f9a38d7bbe6 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:43.179813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9eb35-00d3-42ef-8125-4f9a38d7bbe6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:43.180084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9eb35-00d3-42ef-8125-4f9a38d7bbe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:43.187639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:43.199648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2df9eb35-00d3-42ef-8125-4f9a38d7bbe6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:43.200870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2df9eb35-00d3-42ef-8125-4f9a38d7bbe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:43.277687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6015f2-eb79-49f5-8e14-15be9cabe18b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:43.286229 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 179/349] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:43.287192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9eb35-00d3-42ef-8125-4f9a38d7bbe6 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:43.323845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0b3c9-c23c-4062-a1cf-318139959409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:43.324804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0b3c9-c23c-4062-a1cf-318139959409 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:43.325359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0b3c9-c23c-4062-a1cf-318139959409 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:43.325644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0b3c9-c23c-4062-a1cf-318139959409 None admin] PATH_INFO: `/v3/groups/a8505ee3a8444d09b41523ea8381cf48` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:43.332996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0b3c9-c23c-4062-a1cf-318139959409 None admin] RBAC: Authorizing `identity:get_group(group_id=a8505ee3a8444d09b41523ea8381cf48)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:43.334452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0b3c9-c23c-4062-a1cf-318139959409 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:43.337558 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 171/350] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Oct 1 20:32:43 2022] GET /identity/v3/groups/a8505ee3a8444d09b41523ea8381cf48 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:43.338336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0b3c9-c23c-4062-a1cf-318139959409 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:43.395255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:43.424675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9eb35-00d3-42ef-8125-4f9a38d7bbe6 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:43.469528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0b3c9-c23c-4062-a1cf-318139959409 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:43.473447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d742fe-b565-4e7f-a261-2870a0a31913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:43.474311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d742fe-b565-4e7f-a261-2870a0a31913 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:43.475758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d742fe-b565-4e7f-a261-2870a0a31913 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:43.475758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d742fe-b565-4e7f-a261-2870a0a31913 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:43.477084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:43.492063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d742fe-b565-4e7f-a261-2870a0a31913 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:43.497422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-f8d742fe-b565-4e7f-a261-2870a0a31913 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:43.497422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:43.497422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:43.497422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:43.497422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:43.497422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:43.497422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:43.497422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:43.497422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:43.497422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:43.497422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:43.497422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:43.497422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:43.497422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:43.497422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:43.497422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:43.498345 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:43.498345 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:43.498345 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:43.498345 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:43.498345 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:43.498345 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:43.498345 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:43.498345 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:43.498345 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:43.498345 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:43.498345 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:43.498345 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:43.498345 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:43.498345 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:43.498345 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:43.498345 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:43.499504 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:43.499504 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:43.499504 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:43.499504 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:43.499504 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:43.499504 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:43.499504 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:43.499504 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:43.499504 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:43.499504 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:43.499504 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:43.499504 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:43.499504 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 180/351] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:43.504137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63cf11-6f86-45d3-9221-f5c7909f30ef None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:43.504426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63cf11-6f86-45d3-9221-f5c7909f30ef None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:43.504702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63cf11-6f86-45d3-9221-f5c7909f30ef None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:43.506693 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 181/352] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:43.507388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63cf11-6f86-45d3-9221-f5c7909f30ef None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:43.516125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975db64-b1ed-41c1-9b22-d537054164ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:43.517087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975db64-b1ed-41c1-9b22-d537054164ed None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:43.517563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975db64-b1ed-41c1-9b22-d537054164ed None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:43.518616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975db64-b1ed-41c1-9b22-d537054164ed None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:43.520072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975db64-b1ed-41c1-9b22-d537054164ed None admin] RBAC: Authorizing `identity:get_project(project_id=a503020af843467a992a8df7f46d3fec)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:43.521837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975db64-b1ed-41c1-9b22-d537054164ed None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:43.524221 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 172/353] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:43 2022] GET /identity/v3/projects/a503020af843467a992a8df7f46d3fec => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:43.525300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975db64-b1ed-41c1-9b22-d537054164ed None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:43.638599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63cf11-6f86-45d3-9221-f5c7909f30ef None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:43.657870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975db64-b1ed-41c1-9b22-d537054164ed None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:43.681815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5547ea-7d78-4847-86b8-edde9298876c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:43.682475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5547ea-7d78-4847-86b8-edde9298876c None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:43.682783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5547ea-7d78-4847-86b8-edde9298876c None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:43.683051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5547ea-7d78-4847-86b8-edde9298876c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:43.683779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5547ea-7d78-4847-86b8-edde9298876c None admin] RBAC: Adding query filter params (group.id=a8505ee3a8444d09b41523ea8381cf48, role.id=17f3aeacc5034f3891cc6556da651c1b, scope.project.id=a503020af843467a992a8df7f46d3fec) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:43.684113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5547ea-7d78-4847-86b8-edde9298876c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:43.685737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5547ea-7d78-4847-86b8-edde9298876c None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:43.691698 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 182/354] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Oct 1 20:32:43 2022] GET /identity/v3/role_assignments?group.id=a8505ee3a8444d09b41523ea8381cf48&scope.project.id=a503020af843467a992a8df7f46d3fec&role.id=17f3aeacc5034f3891cc6556da651c1b => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:43.693443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3e945-57ea-4e8b-b7e3-a4d58bad150c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:43.693726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3e945-57ea-4e8b-b7e3-a4d58bad150c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:43.693997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3e945-57ea-4e8b-b7e3-a4d58bad150c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:43.703061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305f020-1049-4e4b-b2b0-b336aa10c8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:43.703893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305f020-1049-4e4b-b2b0-b336aa10c8d1 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:43.704294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305f020-1049-4e4b-b2b0-b336aa10c8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:43.704634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305f020-1049-4e4b-b2b0-b336aa10c8d1 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:43.705316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305f020-1049-4e4b-b2b0-b336aa10c8d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:43.706256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305f020-1049-4e4b-b2b0-b336aa10c8d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:43.709755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305f020-1049-4e4b-b2b0-b336aa10c8d1 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:43.716786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d1d3e945-57ea-4e8b-b7e3-a4d58bad150c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:43.718047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d1d3e945-57ea-4e8b-b7e3-a4d58bad150c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:43.722552 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 173/355] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:43.722901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:43.727527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305f020-1049-4e4b-b2b0-b336aa10c8d1 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:43.793517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:43.812454 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 183/356] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:43.813415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3e945-57ea-4e8b-b7e3-a4d58bad150c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:43.934665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305f020-1049-4e4b-b2b0-b336aa10c8d1 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:43.938269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3e945-57ea-4e8b-b7e3-a4d58bad150c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:43.983259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f77545-45ee-4e13-a199-a27a88485289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:43.983909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f77545-45ee-4e13-a199-a27a88485289 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:43.984190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f77545-45ee-4e13-a199-a27a88485289 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:43.984464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f77545-45ee-4e13-a199-a27a88485289 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:43.995944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16463866-cbfc-4b6f-b00d-5b594e071c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:43.996135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f77545-45ee-4e13-a199-a27a88485289 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:43.997004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16463866-cbfc-4b6f-b00d-5b594e071c0a None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:43.997498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16463866-cbfc-4b6f-b00d-5b594e071c0a None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:43.997958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16463866-cbfc-4b6f-b00d-5b594e071c0a None admin] PATH_INFO: `/v3/groups/a8505ee3a8444d09b41523ea8381cf48` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:43.999307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16463866-cbfc-4b6f-b00d-5b594e071c0a None admin] RBAC: Authorizing `identity:get_group(group_id=a8505ee3a8444d09b41523ea8381cf48)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:44.001004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-d4f77545-45ee-4e13-a199-a27a88485289 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 01 20:32:44.001004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:44.001004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:44.001004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:44.001004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:44.001004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:44.001004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:44.001004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:44.001004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:44.001004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:44.001004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:44.001004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:44.001004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:44.001004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:44.001004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:44.001004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:44.001873 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:44.001873 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:44.001873 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:44.001873 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:44.001873 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:44.001873 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:44.001873 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:44.001873 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:44.001873 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:44.001873 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:44.001873 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:44.001873 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:44.001873 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:44.001873 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:44.001873 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:44.001873 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:44.003234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:44.003234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:44.003234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:44.003234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:44.003234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:44.003234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:44.003234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:44.003234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:44.003234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:44.003234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:44.003234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 01 20:32:44.003234 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:44.003234 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 184/357] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:44.003973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f77545-45ee-4e13-a199-a27a88485289 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:44.008792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16463866-cbfc-4b6f-b00d-5b594e071c0a None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:44.008792 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 174/358] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Oct 1 20:32:43 2022] GET /identity/v3/groups/a8505ee3a8444d09b41523ea8381cf48 => generated 226 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:44.009872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b16f6-13b6-4375-81a9-3b2a1bd8a987 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:44.009988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b16f6-13b6-4375-81a9-3b2a1bd8a987 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:44.011105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b16f6-13b6-4375-81a9-3b2a1bd8a987 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:44.012299 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 175/359] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:44.012766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b16f6-13b6-4375-81a9-3b2a1bd8a987 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:44.080056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:44.082607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:44.150842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f77545-45ee-4e13-a199-a27a88485289 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:44.175819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b16f6-13b6-4375-81a9-3b2a1bd8a987 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:44.189667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832f4e8-4477-459a-b9af-9ad82c65ddfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:44.190835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832f4e8-4477-459a-b9af-9ad82c65ddfc None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:44.191302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832f4e8-4477-459a-b9af-9ad82c65ddfc None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:44.191302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832f4e8-4477-459a-b9af-9ad82c65ddfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:44.222961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327f07d-c828-4827-b6aa-cdb904e74c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:44.223970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327f07d-c828-4827-b6aa-cdb904e74c53 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:44.227815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327f07d-c828-4827-b6aa-cdb904e74c53 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:44.227815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327f07d-c828-4827-b6aa-cdb904e74c53 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:44.227815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b327f07d-c828-4827-b6aa-cdb904e74c53 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:44.227815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b327f07d-c828-4827-b6aa-cdb904e74c53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:44.227815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b327f07d-c828-4827-b6aa-cdb904e74c53 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:44.239260 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 176/360] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:44.240787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb7f98-5985-45b3-87f8-2293e72fcfa9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:44.241304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb7f98-5985-45b3-87f8-2293e72fcfa9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:44.242771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb7f98-5985-45b3-87f8-2293e72fcfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:44.264410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-cbcb7f98-5985-45b3-87f8-2293e72fcfa9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:44.265601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cbcb7f98-5985-45b3-87f8-2293e72fcfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:44.281984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832f4e8-4477-459a-b9af-9ad82c65ddfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:44.284085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832f4e8-4477-459a-b9af-9ad82c65ddfc None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:44.344216 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 177/361] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:44.345165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb7f98-5985-45b3-87f8-2293e72fcfa9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:44.351327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:44.382193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:44.420068 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 185/362] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:44.422105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832f4e8-4477-459a-b9af-9ad82c65ddfc None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:44.476071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb7f98-5985-45b3-87f8-2293e72fcfa9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:44.519037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96516a71-85f1-4082-b906-d1ee4022ccca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:44.520163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96516a71-85f1-4082-b906-d1ee4022ccca None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:44.520664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96516a71-85f1-4082-b906-d1ee4022ccca None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:44.521147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96516a71-85f1-4082-b906-d1ee4022ccca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:44.522288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96516a71-85f1-4082-b906-d1ee4022ccca None admin] RBAC: Adding query filter params (group.id=a8505ee3a8444d09b41523ea8381cf48, role.id=8b7b6e0028b04be8912abda33d3ef729, scope.project.id=a503020af843467a992a8df7f46d3fec) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:44.522814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96516a71-85f1-4082-b906-d1ee4022ccca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:44.524531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96516a71-85f1-4082-b906-d1ee4022ccca None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:44.532712 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 178/363] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Oct 1 20:32:44 2022] GET /identity/v3/role_assignments?group.id=a8505ee3a8444d09b41523ea8381cf48&scope.project.id=a503020af843467a992a8df7f46d3fec&role.id=8b7b6e0028b04be8912abda33d3ef729 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:44.534415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96516a71-85f1-4082-b906-d1ee4022ccca None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:44.543427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832f4e8-4477-459a-b9af-9ad82c65ddfc None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:44.584155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9641c-3173-4b52-bd77-77fc1c83ef62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:44.584947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9641c-3173-4b52-bd77-77fc1c83ef62 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:44.585320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9641c-3173-4b52-bd77-77fc1c83ef62 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:44.585889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9641c-3173-4b52-bd77-77fc1c83ef62 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:44.598533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9641c-3173-4b52-bd77-77fc1c83ef62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:44.603716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-70e9641c-3173-4b52-bd77-77fc1c83ef62 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:44.603716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:44.603716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:44.603716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:44.603716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:44.603716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:44.603716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:44.603716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:44.603716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:44.603716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:44.603716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:44.603716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:44.603716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:44.603716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:44.603716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:44.603716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:44.604683 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:44.604683 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:44.604683 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:44.604683 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:44.604683 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:44.604683 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:44.604683 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:44.604683 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:44.604683 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:44.604683 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:44.604683 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:44.604683 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:44.604683 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:44.604683 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:44.604683 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:44.604683 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:44.605500 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:44.605500 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:44.605500 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:44.605500 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:44.605500 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:44.605500 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:44.605500 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:44.605500 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:44.605500 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:44.605500 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:44.605500 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:44.605500 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:44.605500 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 186/364] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:32:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:44.606166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9641c-3173-4b52-bd77-77fc1c83ef62 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:44.618584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:44.712888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:44.722988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96516a71-85f1-4082-b906-d1ee4022ccca None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:44.725879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9641c-3173-4b52-bd77-77fc1c83ef62 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:44.763976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592db0f-4076-4fc2-bba6-77dd93d278b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:44.764889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592db0f-4076-4fc2-bba6-77dd93d278b6 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:44.765456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592db0f-4076-4fc2-bba6-77dd93d278b6 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:44.765924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592db0f-4076-4fc2-bba6-77dd93d278b6 None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:44.767360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592db0f-4076-4fc2-bba6-77dd93d278b6 None admin] RBAC: Authorizing `identity:get_project(project_id=5acc2b4d60e244298e1c3401d66e7636)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:44.769409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592db0f-4076-4fc2-bba6-77dd93d278b6 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:44.771870 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 187/365] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:44 2022] GET /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:44.773035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592db0f-4076-4fc2-bba6-77dd93d278b6 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:44.774599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124cf3d-01f1-40ff-9299-83da329c876c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:44.775338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124cf3d-01f1-40ff-9299-83da329c876c None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:44.775691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124cf3d-01f1-40ff-9299-83da329c876c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:44.776291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124cf3d-01f1-40ff-9299-83da329c876c None admin] PATH_INFO: `/v3/users/2ad75097dc8c491aa307375a3bcce91f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:44.777616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124cf3d-01f1-40ff-9299-83da329c876c None admin] RBAC: Authorizing `identity:get_user(user_id=2ad75097dc8c491aa307375a3bcce91f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:44.779227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124cf3d-01f1-40ff-9299-83da329c876c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:44.781000 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 179/366] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:32:44 2022] GET /identity/v3/users/2ad75097dc8c491aa307375a3bcce91f => generated 293 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:44.784725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124cf3d-01f1-40ff-9299-83da329c876c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:44.886763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:44.924966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592db0f-4076-4fc2-bba6-77dd93d278b6 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:44.927576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124cf3d-01f1-40ff-9299-83da329c876c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:44.972662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b221137-a500-4003-b189-edbce340cbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:44.973696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7930c-cb68-4f4a-92bb-7c20d3520d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:44.973953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b221137-a500-4003-b189-edbce340cbf1 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:44.974346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7930c-cb68-4f4a-92bb-7c20d3520d5e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:44.974643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7930c-cb68-4f4a-92bb-7c20d3520d5e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:44.974918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7930c-cb68-4f4a-92bb-7c20d3520d5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:44.975104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b221137-a500-4003-b189-edbce340cbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:44.975732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b221137-a500-4003-b189-edbce340cbf1 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:44.977691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b221137-a500-4003-b189-edbce340cbf1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:44.978362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b221137-a500-4003-b189-edbce340cbf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:44.980052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b221137-a500-4003-b189-edbce340cbf1 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:44.986750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7930c-cb68-4f4a-92bb-7c20d3520d5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:44.992558 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 188/367] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:32:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:44.993677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b221137-a500-4003-b189-edbce340cbf1 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:44.994049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-f0b7930c-cb68-4f4a-92bb-7c20d3520d5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:32:44.994049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:44.994049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:44.994049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:44.994049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:44.994049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:44.994049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:44.994049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:44.994049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:44.994049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:44.994049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:44.994049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:44.994049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:44.994049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:44.994049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:44.994049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:44.994991 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:44.994991 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:44.994991 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:44.994991 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:44.994991 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:44.994991 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:44.994991 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:44.994991 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:44.994991 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:44.994991 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:44.994991 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:44.994991 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:44.994991 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:44.994991 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:44.994991 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:44.994991 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:44.995765 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:44.995765 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:44.995765 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:44.995765 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:44.995765 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:44.995765 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:44.995765 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:44.995765 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:44.995765 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:44.995765 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:44.995765 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:32:44.995765 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:32:44.995765 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 180/368] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:32:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:44.996426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7930c-cb68-4f4a-92bb-7c20d3520d5e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:45.031317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:45.132523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7930c-cb68-4f4a-92bb-7c20d3520d5e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:45.136280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b221137-a500-4003-b189-edbce340cbf1 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:45.177342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c68e3-449f-401f-a4d1-16b5beb7a784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:45.177980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c68e3-449f-401f-a4d1-16b5beb7a784 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:45.178329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c68e3-449f-401f-a4d1-16b5beb7a784 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:45.178640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c68e3-449f-401f-a4d1-16b5beb7a784 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:45.179477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c68e3-449f-401f-a4d1-16b5beb7a784 None admin] RBAC: Adding query filter params (role.id=be70f76ecf3f465fb09d46916db16671, user.id=2ad75097dc8c491aa307375a3bcce91f, scope.system=all) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:45.179793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c68e3-449f-401f-a4d1-16b5beb7a784 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:45.181370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c68e3-449f-401f-a4d1-16b5beb7a784 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:45.182278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d68ef-921a-446b-aba0-6d8a3541eb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:45.183178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d68ef-921a-446b-aba0-6d8a3541eb8e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:45.183776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d68ef-921a-446b-aba0-6d8a3541eb8e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:45.184262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d68ef-921a-446b-aba0-6d8a3541eb8e None admin] PATH_INFO: `/v3/groups/a8505ee3a8444d09b41523ea8381cf48` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:45.186120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d68ef-921a-446b-aba0-6d8a3541eb8e None admin] RBAC: Authorizing `identity:get_group(group_id=a8505ee3a8444d09b41523ea8381cf48)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:45.187925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d68ef-921a-446b-aba0-6d8a3541eb8e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:45.189696 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 189/369] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Oct 1 20:32:44 2022] GET /identity/v3/groups/a8505ee3a8444d09b41523ea8381cf48 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:45.191296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d68ef-921a-446b-aba0-6d8a3541eb8e None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:45.198609 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 181/370] 10.222.0.17 () {56 vars in 1308 bytes} [Sat Oct 1 20:32:44 2022] GET /identity/v3/role_assignments?user.id=2ad75097dc8c491aa307375a3bcce91f&scope.system=all&role.id=be70f76ecf3f465fb09d46916db16671 => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:45.199833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c68e3-449f-401f-a4d1-16b5beb7a784 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:45.209817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:45.371716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:45.396372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d68ef-921a-446b-aba0-6d8a3541eb8e None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:45.409132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c68e3-449f-401f-a4d1-16b5beb7a784 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:45.457866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec314a-624a-462d-939d-1fcf7e855ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:45.458506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec314a-624a-462d-939d-1fcf7e855ef2 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:45.458792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec314a-624a-462d-939d-1fcf7e855ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:45.459150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec314a-624a-462d-939d-1fcf7e855ef2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:45.464497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dc823-d936-49a0-bb8d-943831cceb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:45.465431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dc823-d936-49a0-bb8d-943831cceb39 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:45.465918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dc823-d936-49a0-bb8d-943831cceb39 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:45.466376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dc823-d936-49a0-bb8d-943831cceb39 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:45.467281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dc823-d936-49a0-bb8d-943831cceb39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:45.467792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dc823-d936-49a0-bb8d-943831cceb39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:45.469530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dc823-d936-49a0-bb8d-943831cceb39 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:45.481549 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 182/371] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:45.482937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dc823-d936-49a0-bb8d-943831cceb39 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:45.563149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec314a-624a-462d-939d-1fcf7e855ef2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:45.565881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec314a-624a-462d-939d-1fcf7e855ef2 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:45.577371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:45.617887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dc823-d936-49a0-bb8d-943831cceb39 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:45.662780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399938e-4ff3-4972-9f3b-86be32b701dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:45.663737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399938e-4ff3-4972-9f3b-86be32b701dd None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:45.664231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399938e-4ff3-4972-9f3b-86be32b701dd None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:45.666345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399938e-4ff3-4972-9f3b-86be32b701dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:45.667359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399938e-4ff3-4972-9f3b-86be32b701dd None admin] RBAC: Adding query filter params (group.id=a8505ee3a8444d09b41523ea8381cf48, role.id=17f3aeacc5034f3891cc6556da651c1b, scope.project.id=5acc2b4d60e244298e1c3401d66e7636) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:45.667861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399938e-4ff3-4972-9f3b-86be32b701dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:45.669505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399938e-4ff3-4972-9f3b-86be32b701dd None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:45.677433 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 183/372] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Oct 1 20:32:45 2022] GET /identity/v3/role_assignments?group.id=a8505ee3a8444d09b41523ea8381cf48&scope.project.id=5acc2b4d60e244298e1c3401d66e7636&role.id=17f3aeacc5034f3891cc6556da651c1b => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:45.678650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399938e-4ff3-4972-9f3b-86be32b701dd None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:45.693859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:45.700310 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 190/373] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:45.706667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec314a-624a-462d-939d-1fcf7e855ef2 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:45.815308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399938e-4ff3-4972-9f3b-86be32b701dd None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:45.841472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec314a-624a-462d-939d-1fcf7e855ef2 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:45.852171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:45.873247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b070333-c4a6-420a-8ae7-a73ee70ab399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:45.873880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b070333-c4a6-420a-8ae7-a73ee70ab399 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:45.874159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b070333-c4a6-420a-8ae7-a73ee70ab399 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:45.874434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b070333-c4a6-420a-8ae7-a73ee70ab399 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:45.893685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c3360-b9ea-4832-81fc-f1c25f99a260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:45.894707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c3360-b9ea-4832-81fc-f1c25f99a260 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:45.895185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c3360-b9ea-4832-81fc-f1c25f99a260 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:45.895652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c3360-b9ea-4832-81fc-f1c25f99a260 None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:45.897019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c3360-b9ea-4832-81fc-f1c25f99a260 None admin] RBAC: Authorizing `identity:get_project(project_id=5acc2b4d60e244298e1c3401d66e7636)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:45.899362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c3360-b9ea-4832-81fc-f1c25f99a260 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:45.901743 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 191/374] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:45 2022] GET /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:45.907456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c3360-b9ea-4832-81fc-f1c25f99a260 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:45.971490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b070333-c4a6-420a-8ae7-a73ee70ab399 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:45.971490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b070333-c4a6-420a-8ae7-a73ee70ab399 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:45.984749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:46.079691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c3360-b9ea-4832-81fc-f1c25f99a260 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:46.105528 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 184/375] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:46.111330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b070333-c4a6-420a-8ae7-a73ee70ab399 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:46.118866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45daa70-3a70-4853-a056-e282a12a6252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:46.119595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45daa70-3a70-4853-a056-e282a12a6252 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:46.119950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45daa70-3a70-4853-a056-e282a12a6252 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:46.120351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45daa70-3a70-4853-a056-e282a12a6252 None admin] PATH_INFO: `/v3/groups/a8505ee3a8444d09b41523ea8381cf48` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:46.121647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45daa70-3a70-4853-a056-e282a12a6252 None admin] RBAC: Authorizing `identity:get_group(group_id=a8505ee3a8444d09b41523ea8381cf48)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:46.123159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45daa70-3a70-4853-a056-e282a12a6252 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:46.125102 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 192/376] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Oct 1 20:32:45 2022] GET /identity/v3/groups/a8505ee3a8444d09b41523ea8381cf48 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:46.133286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45daa70-3a70-4853-a056-e282a12a6252 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:46.194440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:46.245432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b070333-c4a6-420a-8ae7-a73ee70ab399 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:46.259389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:46.286673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45daa70-3a70-4853-a056-e282a12a6252 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:46.310775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0798b-adc9-467f-9d53-cfb4c8a68b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:46.311656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0798b-adc9-467f-9d53-cfb4c8a68b1b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:46.312123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0798b-adc9-467f-9d53-cfb4c8a68b1b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:46.312609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0798b-adc9-467f-9d53-cfb4c8a68b1b None admin] PATH_INFO: `/v3/projects/549fa0b49ffd4207961a9dc3298560fa` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:46.315646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0798b-adc9-467f-9d53-cfb4c8a68b1b None admin] RBAC: Authorizing `identity:get_project(project_id=549fa0b49ffd4207961a9dc3298560fa)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:46.317260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0798b-adc9-467f-9d53-cfb4c8a68b1b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:46.321641 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 185/377] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:46 2022] GET /identity/v3/projects/549fa0b49ffd4207961a9dc3298560fa => generated 344 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:46.327385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0798b-adc9-467f-9d53-cfb4c8a68b1b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:46.359312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d7758-a24b-4440-b49f-11dea2422198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:46.361494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d7758-a24b-4440-b49f-11dea2422198 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:46.361992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d7758-a24b-4440-b49f-11dea2422198 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:46.362431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d7758-a24b-4440-b49f-11dea2422198 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:46.364084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d7758-a24b-4440-b49f-11dea2422198 None admin] RBAC: Adding query filter params (group.id=a8505ee3a8444d09b41523ea8381cf48, role.id=8b7b6e0028b04be8912abda33d3ef729, scope.project.id=5acc2b4d60e244298e1c3401d66e7636) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:46.365876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d7758-a24b-4440-b49f-11dea2422198 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:46.367556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d7758-a24b-4440-b49f-11dea2422198 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:46.381840 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 193/378] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Oct 1 20:32:46 2022] GET /identity/v3/role_assignments?group.id=a8505ee3a8444d09b41523ea8381cf48&scope.project.id=5acc2b4d60e244298e1c3401d66e7636&role.id=8b7b6e0028b04be8912abda33d3ef729 => generated 260 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:46.482109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0798b-adc9-467f-9d53-cfb4c8a68b1b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:46.520277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f86982-95e4-4c06-83bc-59c3239bf2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:46.521211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f86982-95e4-4c06-83bc-59c3239bf2ec None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:46.521644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f86982-95e4-4c06-83bc-59c3239bf2ec None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:46.522070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f86982-95e4-4c06-83bc-59c3239bf2ec None admin] PATH_INFO: `/v3/groups/3b4e4f0428644954ae373c4bfaf05a8f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:46.528043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f86982-95e4-4c06-83bc-59c3239bf2ec None admin] RBAC: Authorizing `identity:get_group(group_id=3b4e4f0428644954ae373c4bfaf05a8f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:46.529824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f86982-95e4-4c06-83bc-59c3239bf2ec None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:46.532190 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 186/379] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Oct 1 20:32:46 2022] GET /identity/v3/groups/3b4e4f0428644954ae373c4bfaf05a8f => generated 229 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:46.537938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d7758-a24b-4440-b49f-11dea2422198 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:46.647934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d7758-a24b-4440-b49f-11dea2422198 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:46.657344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:46.691288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359453f-bc01-46e5-a792-8308483e4137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:46.692022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359453f-bc01-46e5-a792-8308483e4137 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:46.692317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359453f-bc01-46e5-a792-8308483e4137 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:46.692590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359453f-bc01-46e5-a792-8308483e4137 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:46.693333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c359453f-bc01-46e5-a792-8308483e4137 None admin] RBAC: Adding query filter params (group.id=3b4e4f0428644954ae373c4bfaf05a8f, role.id=1c50896511054a5f994bee3d99fa0619, scope.project.id=549fa0b49ffd4207961a9dc3298560fa) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:46.693642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c359453f-bc01-46e5-a792-8308483e4137 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:46.695128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c359453f-bc01-46e5-a792-8308483e4137 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:46.703306 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 194/380] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Oct 1 20:32:46 2022] GET /identity/v3/role_assignments?group.id=3b4e4f0428644954ae373c4bfaf05a8f&scope.project.id=549fa0b49ffd4207961a9dc3298560fa&role.id=1c50896511054a5f994bee3d99fa0619 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:47.024516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977dd9a-e29a-4f4b-b6bf-240727ab194b None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:47.024892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977dd9a-e29a-4f4b-b6bf-240727ab194b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:47.025072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977dd9a-e29a-4f4b-b6bf-240727ab194b None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:47.026695 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 187/381] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:47.032587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a1492-4cd4-4ea3-83e6-a35630a5e423 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:47.032878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a1492-4cd4-4ea3-83e6-a35630a5e423 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:47.033156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a1492-4cd4-4ea3-83e6-a35630a5e423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:47.052936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b81a1492-4cd4-4ea3-83e6-a35630a5e423 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:47.054130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b81a1492-4cd4-4ea3-83e6-a35630a5e423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:47.130675 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 195/382] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:32:47.466980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977dd9a-e29a-4f4b-b6bf-240727ab194b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:47.517260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:47.588986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977dd9a-e29a-4f4b-b6bf-240727ab194b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:47.631871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214ae31-d3fc-4c67-bda3-9c598143dcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:47.632509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214ae31-d3fc-4c67-bda3-9c598143dcb1 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:47.632787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214ae31-d3fc-4c67-bda3-9c598143dcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:47.633060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214ae31-d3fc-4c67-bda3-9c598143dcb1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:47.644119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214ae31-d3fc-4c67-bda3-9c598143dcb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:47.649049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-1214ae31-d3fc-4c67-bda3-9c598143dcb1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:47.649049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:47.649049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:47.649049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:47.649049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:47.649049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:47.649049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:47.649049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:47.649049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:47.649049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:47.649049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:47.649049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:47.649049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:47.649049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:47.649049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:47.649049 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:47.649921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:47.649921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:47.649921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:47.649921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:47.649921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:47.649921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:47.649921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:47.649921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:47.649921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:47.649921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:47.649921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:47.649921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:47.649921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:47.649921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:47.649921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:47.649921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:47.650758 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:47.650758 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:47.650758 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:47.650758 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:47.650758 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:47.650758 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:47.650758 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:47.650758 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:47.650758 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:47.650758 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:47.650758 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:47.650758 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:32:47.650758 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 188/383] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:47.659844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a1492-4cd4-4ea3-83e6-a35630a5e423 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:47.725746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccbcd4-feff-4701-ae43-40beb9410f71 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:47.725937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccbcd4-feff-4701-ae43-40beb9410f71 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:47.726208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccbcd4-feff-4701-ae43-40beb9410f71 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:47.727563 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 189/384] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:47.732535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccf8c8-0ef8-4b51-b61a-cb8396858f1d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:47.732869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccf8c8-0ef8-4b51-b61a-cb8396858f1d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:47.733139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccf8c8-0ef8-4b51-b61a-cb8396858f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:47.760625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6cccf8c8-0ef8-4b51-b61a-cb8396858f1d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:47.762021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6cccf8c8-0ef8-4b51-b61a-cb8396858f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:47.778447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:47.782410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a1492-4cd4-4ea3-83e6-a35630a5e423 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:47.833858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e822b-5fa0-48b4-a7d8-3389fabd4304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:47.834919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e822b-5fa0-48b4-a7d8-3389fabd4304 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:47.835391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e822b-5fa0-48b4-a7d8-3389fabd4304 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:47.835866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e822b-5fa0-48b4-a7d8-3389fabd4304 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:47.836724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e822b-5fa0-48b4-a7d8-3389fabd4304 None admin] RBAC: Adding query filter params (name=member) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:47.837229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e822b-5fa0-48b4-a7d8-3389fabd4304 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:47.838922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e822b-5fa0-48b4-a7d8-3389fabd4304 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:47.857291 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 196/385] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:47.858852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e822b-5fa0-48b4-a7d8-3389fabd4304 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:47.877809 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 190/386] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:32:47.984341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e822b-5fa0-48b4-a7d8-3389fabd4304 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:48.026896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c15206-f3fa-4caa-a94b-2b1e3a6b9cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:48.027521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c15206-f3fa-4caa-a94b-2b1e3a6b9cb6 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:48.027798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c15206-f3fa-4caa-a94b-2b1e3a6b9cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:48.028112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c15206-f3fa-4caa-a94b-2b1e3a6b9cb6 None admin] PATH_INFO: `/v3/groups/a8505ee3a8444d09b41523ea8381cf48` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:48.029257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c15206-f3fa-4caa-a94b-2b1e3a6b9cb6 None admin] RBAC: Authorizing `identity:get_group(group_id=a8505ee3a8444d09b41523ea8381cf48)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:48.030719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c15206-f3fa-4caa-a94b-2b1e3a6b9cb6 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:48.035092 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 197/387] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Oct 1 20:32:47 2022] GET /identity/v3/groups/a8505ee3a8444d09b41523ea8381cf48 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:48.038237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cccf8c8-0ef8-4b51-b61a-cb8396858f1d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:48.080834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:48.173227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cccf8c8-0ef8-4b51-b61a-cb8396858f1d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:48.213702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4780a-4568-44ec-9be8-af23b145104a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:48.214724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4780a-4568-44ec-9be8-af23b145104a None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:48.215197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4780a-4568-44ec-9be8-af23b145104a None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:48.215659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4780a-4568-44ec-9be8-af23b145104a None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:48.217162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4780a-4568-44ec-9be8-af23b145104a None admin] RBAC: Authorizing `identity:get_project(project_id=a503020af843467a992a8df7f46d3fec)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:48.218973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4780a-4568-44ec-9be8-af23b145104a None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:48.221858 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 191/388] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:48 2022] GET /identity/v3/projects/a503020af843467a992a8df7f46d3fec => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:48.227812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c15206-f3fa-4caa-a94b-2b1e3a6b9cb6 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:48.271047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:48.356110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c15206-f3fa-4caa-a94b-2b1e3a6b9cb6 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:48.399104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a36a7-05db-4e92-9277-7202b7b5b833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:48.399762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a36a7-05db-4e92-9277-7202b7b5b833 None admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:48.400042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a36a7-05db-4e92-9277-7202b7b5b833 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:48.400318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a36a7-05db-4e92-9277-7202b7b5b833 None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec/groups/a8505ee3a8444d09b41523ea8381cf48/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:48.402273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a36a7-05db-4e92-9277-7202b7b5b833 None admin] RBAC: Authorizing `identity:create_grant(project_id=a503020af843467a992a8df7f46d3fec, group_id=a8505ee3a8444d09b41523ea8381cf48, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:48.403749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a36a7-05db-4e92-9277-7202b7b5b833 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:48.436013 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 198/389] 10.222.0.17 () {58 vars in 1442 bytes} [Sat Oct 1 20:32:48 2022] PUT /identity/v3/projects/a503020af843467a992a8df7f46d3fec/groups/a8505ee3a8444d09b41523ea8381cf48/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:32:48.517499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4780a-4568-44ec-9be8-af23b145104a None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:48.633411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:48.637376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4780a-4568-44ec-9be8-af23b145104a None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:48.681275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acdd4bd-871c-427b-8d25-c90d3b43b809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:48.682088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acdd4bd-871c-427b-8d25-c90d3b43b809 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:48.682371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acdd4bd-871c-427b-8d25-c90d3b43b809 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:48.682677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acdd4bd-871c-427b-8d25-c90d3b43b809 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:48.694737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acdd4bd-871c-427b-8d25-c90d3b43b809 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:48.700599 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-4acdd4bd-871c-427b-8d25-c90d3b43b809 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:48.700599 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:48.700599 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:48.700599 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:48.700599 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:48.700599 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:48.700599 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:48.700599 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:48.700599 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:48.700599 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:48.700599 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:48.700599 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:48.700599 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:48.700599 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:48.700599 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:48.700599 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:48.701486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:48.701486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:48.701486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:48.701486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:48.701486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:48.701486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:48.701486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:48.701486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:48.701486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:48.701486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:48.701486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:48.701486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:48.701486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:48.701486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:48.701486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:48.701486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:48.702271 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:48.702271 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:48.702271 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:48.702271 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:48.702271 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:48.702271 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:48.702271 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:48.702271 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:48.702271 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:48.702271 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:48.702271 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:48.702271 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:32:48.706722 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 192/390] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:32:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:48.708221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a36a7-05db-4e92-9277-7202b7b5b833 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:48.812915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:48.826200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a36a7-05db-4e92-9277-7202b7b5b833 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:48.870092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480f460-b5cf-425a-8067-03de67eb4d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:48.871085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480f460-b5cf-425a-8067-03de67eb4d1a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:48.871560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480f460-b5cf-425a-8067-03de67eb4d1a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:48.872021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480f460-b5cf-425a-8067-03de67eb4d1a None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:48.872912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4480f460-b5cf-425a-8067-03de67eb4d1a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:48.873426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4480f460-b5cf-425a-8067-03de67eb4d1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:48.875015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4480f460-b5cf-425a-8067-03de67eb4d1a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:48.884713 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 199/391] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:32:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:48.890540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acdd4bd-871c-427b-8d25-c90d3b43b809 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:48.998179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acdd4bd-871c-427b-8d25-c90d3b43b809 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:49.041845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365002e-9cc8-487a-ae18-d46fdc1dceca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:49.042511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365002e-9cc8-487a-ae18-d46fdc1dceca None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:49.050517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365002e-9cc8-487a-ae18-d46fdc1dceca None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:49.050517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365002e-9cc8-487a-ae18-d46fdc1dceca None admin] PATH_INFO: `/v3/groups/a8505ee3a8444d09b41523ea8381cf48` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:49.050517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365002e-9cc8-487a-ae18-d46fdc1dceca None admin] RBAC: Authorizing `identity:get_group(group_id=a8505ee3a8444d09b41523ea8381cf48)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:49.050517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365002e-9cc8-487a-ae18-d46fdc1dceca None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:49.050517 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 193/392] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Oct 1 20:32:48 2022] GET /identity/v3/groups/a8505ee3a8444d09b41523ea8381cf48 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:49.059680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480f460-b5cf-425a-8067-03de67eb4d1a None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:49.169131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b0525-6a56-45eb-b41f-6304e1a9ba67 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:49.169739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b0525-6a56-45eb-b41f-6304e1a9ba67 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:49.170202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b0525-6a56-45eb-b41f-6304e1a9ba67 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:49.172216 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 194/393] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:49.177239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faef06-ebfd-4d51-8d58-7412739c4547 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:49.177527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faef06-ebfd-4d51-8d58-7412739c4547 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:49.177804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faef06-ebfd-4d51-8d58-7412739c4547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:49.187921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480f460-b5cf-425a-8067-03de67eb4d1a None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:49.215531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c9faef06-ebfd-4d51-8d58-7412739c4547 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:49.216737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c9faef06-ebfd-4d51-8d58-7412739c4547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:49.232816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e809c0-5d19-4596-b717-e1eb7cb8bcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:49.233457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e809c0-5d19-4596-b717-e1eb7cb8bcd8 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:49.233736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e809c0-5d19-4596-b717-e1eb7cb8bcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:49.234006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e809c0-5d19-4596-b717-e1eb7cb8bcd8 None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:49.235383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e809c0-5d19-4596-b717-e1eb7cb8bcd8 None admin] RBAC: Authorizing `identity:get_project(project_id=a503020af843467a992a8df7f46d3fec)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:49.236904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e809c0-5d19-4596-b717-e1eb7cb8bcd8 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:49.241677 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 200/394] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:49 2022] GET /identity/v3/projects/a503020af843467a992a8df7f46d3fec => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:49.248370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e809c0-5d19-4596-b717-e1eb7cb8bcd8 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:49.254051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:49.287587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:49.322678 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 195/395] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:32:49.432122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e809c0-5d19-4596-b717-e1eb7cb8bcd8 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:49.478005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee6197-a6e8-4ccf-85fd-2fe44bb2881e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:49.478946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee6197-a6e8-4ccf-85fd-2fe44bb2881e None admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:49.479368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee6197-a6e8-4ccf-85fd-2fe44bb2881e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:49.479783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee6197-a6e8-4ccf-85fd-2fe44bb2881e None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec/groups/a8505ee3a8444d09b41523ea8381cf48/roles/8b7b6e0028b04be8912abda33d3ef729` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:49.482228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee6197-a6e8-4ccf-85fd-2fe44bb2881e None admin] RBAC: Authorizing `identity:create_grant(project_id=a503020af843467a992a8df7f46d3fec, group_id=a8505ee3a8444d09b41523ea8381cf48, role_id=8b7b6e0028b04be8912abda33d3ef729)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:49.483893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee6197-a6e8-4ccf-85fd-2fe44bb2881e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:49.512807 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 201/396] 10.222.0.17 () {58 vars in 1442 bytes} [Sat Oct 1 20:32:49 2022] PUT /identity/v3/projects/a503020af843467a992a8df7f46d3fec/groups/a8505ee3a8444d09b41523ea8381cf48/roles/8b7b6e0028b04be8912abda33d3ef729 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:32:49.737876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faef06-ebfd-4d51-8d58-7412739c4547 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:49.842263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d675f0-2da5-4243-900e-b7cc1a667541 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:49.842835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d675f0-2da5-4243-900e-b7cc1a667541 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:49.842835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d675f0-2da5-4243-900e-b7cc1a667541 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:49.844697 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 202/397] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:49.850323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a649d4e-912d-4428-8aff-db051bb62c69 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:49.850704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a649d4e-912d-4428-8aff-db051bb62c69 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:49.851006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a649d4e-912d-4428-8aff-db051bb62c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:49.877604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faef06-ebfd-4d51-8d58-7412739c4547 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:49.887396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6a649d4e-912d-4428-8aff-db051bb62c69 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:49.888828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6a649d4e-912d-4428-8aff-db051bb62c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:49.921699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46357a04-c827-4847-8ff5-1acbe0ed6297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:49.923271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46357a04-c827-4847-8ff5-1acbe0ed6297 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:49.923713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46357a04-c827-4847-8ff5-1acbe0ed6297 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:49.924195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46357a04-c827-4847-8ff5-1acbe0ed6297 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:49.937278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46357a04-c827-4847-8ff5-1acbe0ed6297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:49.945211 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-46357a04-c827-4847-8ff5-1acbe0ed6297 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:49.945211 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:49.945211 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:49.945211 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:49.945211 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:49.945211 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:49.945211 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:49.945211 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:49.945211 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:49.945211 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:49.945211 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:49.945211 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:49.945211 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:49.945211 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:49.945211 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:49.945211 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:49.946135 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:49.946135 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:49.946135 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:49.946135 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:49.946135 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:49.946135 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:49.946135 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:49.946135 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:49.946135 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:49.946135 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:49.946135 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:49.946135 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:49.946135 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:49.946135 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:49.946135 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:49.946135 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:49.951354 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:49.951354 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:49.951354 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:49.951354 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:49.951354 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:49.951354 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:49.951354 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:49.951354 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:49.951354 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:49.951354 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:49.951354 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:49.951354 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:32:49.951354 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 196/398] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:49.955482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46357a04-c827-4847-8ff5-1acbe0ed6297 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:49.973313 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 203/399] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:32:49.984615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:50.035276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff6981-02c5-4ecb-af65-d83e3d6684a7 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:50.035558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff6981-02c5-4ecb-af65-d83e3d6684a7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:50.035754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff6981-02c5-4ecb-af65-d83e3d6684a7 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:50.037329 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 204/400] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:32:50.042553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5e059-05bc-4ca8-a772-9db85fb46e7c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:50.042853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5e059-05bc-4ca8-a772-9db85fb46e7c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:50.043131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5e059-05bc-4ca8-a772-9db85fb46e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:50.086568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-61f5e059-05bc-4ca8-a772-9db85fb46e7c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:50.087776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-61f5e059-05bc-4ca8-a772-9db85fb46e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:50.094932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46357a04-c827-4847-8ff5-1acbe0ed6297 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:50.099662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:50.143812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37022891-c66e-4301-9c07-38133eefffcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:50.145665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37022891-c66e-4301-9c07-38133eefffcf None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:50.147385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37022891-c66e-4301-9c07-38133eefffcf None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:50.148104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37022891-c66e-4301-9c07-38133eefffcf None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:50.149971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37022891-c66e-4301-9c07-38133eefffcf None admin] RBAC: Adding query filter params (name=member) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:50.150479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37022891-c66e-4301-9c07-38133eefffcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:50.152971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37022891-c66e-4301-9c07-38133eefffcf None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:50.175413 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 197/401] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:50.181397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37022891-c66e-4301-9c07-38133eefffcf None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:50.229751 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 205/402] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:32:50.294144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:50.326295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37022891-c66e-4301-9c07-38133eefffcf None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:50.376091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab819c5f-c6c3-43bb-aec3-41826db3ea1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:50.376721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab819c5f-c6c3-43bb-aec3-41826db3ea1b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:50.376999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab819c5f-c6c3-43bb-aec3-41826db3ea1b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:50.377363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab819c5f-c6c3-43bb-aec3-41826db3ea1b None admin] PATH_INFO: `/v3/groups/a8505ee3a8444d09b41523ea8381cf48` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:50.378511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab819c5f-c6c3-43bb-aec3-41826db3ea1b None admin] RBAC: Authorizing `identity:get_group(group_id=a8505ee3a8444d09b41523ea8381cf48)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:50.379959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab819c5f-c6c3-43bb-aec3-41826db3ea1b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:50.381524 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 198/403] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Oct 1 20:32:50 2022] GET /identity/v3/groups/a8505ee3a8444d09b41523ea8381cf48 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:50.387236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5e059-05bc-4ca8-a772-9db85fb46e7c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:50.401954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab819c5f-c6c3-43bb-aec3-41826db3ea1b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:50.551336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5e059-05bc-4ca8-a772-9db85fb46e7c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:50.594095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab819c5f-c6c3-43bb-aec3-41826db3ea1b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:50.617153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e859f3-a9cd-473d-be45-41fab7b3047b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:50.617991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e859f3-a9cd-473d-be45-41fab7b3047b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:50.618278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e859f3-a9cd-473d-be45-41fab7b3047b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:50.618580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e859f3-a9cd-473d-be45-41fab7b3047b None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:50.619867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e859f3-a9cd-473d-be45-41fab7b3047b None admin] RBAC: Authorizing `identity:get_project(project_id=5acc2b4d60e244298e1c3401d66e7636)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:50.621417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e859f3-a9cd-473d-be45-41fab7b3047b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:50.623564 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 206/404] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:50 2022] GET /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:50.632191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e859f3-a9cd-473d-be45-41fab7b3047b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:50.649837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:50.666949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348316b0-144c-44cb-bfa9-bb25899d8cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:50.667879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348316b0-144c-44cb-bfa9-bb25899d8cf7 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:50.668901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348316b0-144c-44cb-bfa9-bb25899d8cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:50.669394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348316b0-144c-44cb-bfa9-bb25899d8cf7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:50.689128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348316b0-144c-44cb-bfa9-bb25899d8cf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:50.697119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-348316b0-144c-44cb-bfa9-bb25899d8cf7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:50.697119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:50.697119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:50.697119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:50.697119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:50.697119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:50.697119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:50.697119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:50.697119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:50.697119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:50.697119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:50.697119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:50.697119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:50.697119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:50.697119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:50.697119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:50.698858 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:50.698858 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:50.698858 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:50.698858 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:50.698858 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:50.698858 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:50.698858 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:50.698858 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:50.698858 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:50.698858 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:50.698858 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:50.698858 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:50.698858 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:50.698858 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:50.698858 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:50.698858 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:50.700944 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:50.700944 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:50.700944 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:50.700944 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:50.700944 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:50.700944 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:50.700944 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:50.700944 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:50.700944 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:50.700944 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:50.700944 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:50.700944 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:32:50.704532 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 199/405] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:32:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:32:50.704532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348316b0-144c-44cb-bfa9-bb25899d8cf7 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:50.716433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:50.776287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e859f3-a9cd-473d-be45-41fab7b3047b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:50.812494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219cb3d2-357f-43f9-8149-96f685a6a6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:50.813113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cb3d2-357f-43f9-8149-96f685a6a6af None admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:50.813387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cb3d2-357f-43f9-8149-96f685a6a6af None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:50.813659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cb3d2-357f-43f9-8149-96f685a6a6af None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636/groups/a8505ee3a8444d09b41523ea8381cf48/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:50.816308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219cb3d2-357f-43f9-8149-96f685a6a6af None admin] RBAC: Authorizing `identity:create_grant(project_id=5acc2b4d60e244298e1c3401d66e7636, group_id=a8505ee3a8444d09b41523ea8381cf48, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:50.817778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219cb3d2-357f-43f9-8149-96f685a6a6af None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:50.836306 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 207/406] 10.222.0.17 () {58 vars in 1442 bytes} [Sat Oct 1 20:32:50 2022] PUT /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636/groups/a8505ee3a8444d09b41523ea8381cf48/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:32:50.836904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219cb3d2-357f-43f9-8149-96f685a6a6af None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:50.843863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348316b0-144c-44cb-bfa9-bb25899d8cf7 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:50.898550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849c57b-3ad6-4dca-b80f-5a40c567594c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:50.899897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849c57b-3ad6-4dca-b80f-5a40c567594c None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:50.900601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849c57b-3ad6-4dca-b80f-5a40c567594c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:50.901352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849c57b-3ad6-4dca-b80f-5a40c567594c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:50.917296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849c57b-3ad6-4dca-b80f-5a40c567594c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:50.924643 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-a849c57b-3ad6-4dca-b80f-5a40c567594c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:32:50.924643 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:50.924643 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:50.924643 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:50.924643 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:50.924643 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:50.924643 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:50.924643 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:50.924643 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:50.924643 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:50.924643 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:50.924643 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:50.924643 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:50.924643 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:50.924643 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:50.924643 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:50.925609 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:50.925609 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:50.925609 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:50.925609 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:50.925609 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:50.925609 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:50.925609 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:50.925609 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:50.925609 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:50.925609 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:50.925609 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:50.925609 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:50.925609 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:50.925609 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:50.925609 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:50.925609 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:50.926400 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:50.926400 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:50.926400 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:50.926400 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:50.926400 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:50.926400 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:50.926400 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:50.926400 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:50.926400 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:50.926400 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:50.926400 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:32:50.926400 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:32:50.934731 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 200/407] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:32:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:50.936856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849c57b-3ad6-4dca-b80f-5a40c567594c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:50.998213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:51.045519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219cb3d2-357f-43f9-8149-96f685a6a6af None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:51.063025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:51.090454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1fe8f-3411-43da-a6f6-d07ae90414d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:51.091233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1fe8f-3411-43da-a6f6-d07ae90414d0 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:51.091582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1fe8f-3411-43da-a6f6-d07ae90414d0 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:51.091926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1fe8f-3411-43da-a6f6-d07ae90414d0 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:51.092681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1fe8f-3411-43da-a6f6-d07ae90414d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:51.093052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1fe8f-3411-43da-a6f6-d07ae90414d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:51.094519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1fe8f-3411-43da-a6f6-d07ae90414d0 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:51.108954 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 208/408] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:32:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:51.110346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1fe8f-3411-43da-a6f6-d07ae90414d0 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:51.135219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849c57b-3ad6-4dca-b80f-5a40c567594c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:51.173851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8ea03-bebe-444b-8bfe-8cf82f1d2a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:51.175471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8ea03-bebe-444b-8bfe-8cf82f1d2a7b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:51.175956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8ea03-bebe-444b-8bfe-8cf82f1d2a7b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:51.176414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8ea03-bebe-444b-8bfe-8cf82f1d2a7b None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:51.177268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8ea03-bebe-444b-8bfe-8cf82f1d2a7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:51.177758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8ea03-bebe-444b-8bfe-8cf82f1d2a7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:51.179914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8ea03-bebe-444b-8bfe-8cf82f1d2a7b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:51.189917 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 201/409] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:51.196527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8ea03-bebe-444b-8bfe-8cf82f1d2a7b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:51.237440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1fe8f-3411-43da-a6f6-d07ae90414d0 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:51.268731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:51.276167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6545529-3968-476a-a631-76699e9e5442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:51.276805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6545529-3968-476a-a631-76699e9e5442 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:51.277087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6545529-3968-476a-a631-76699e9e5442 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:51.277362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6545529-3968-476a-a631-76699e9e5442 None admin] PATH_INFO: `/v3/groups/a8505ee3a8444d09b41523ea8381cf48` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:51.278514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6545529-3968-476a-a631-76699e9e5442 None admin] RBAC: Authorizing `identity:get_group(group_id=a8505ee3a8444d09b41523ea8381cf48)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:51.279944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6545529-3968-476a-a631-76699e9e5442 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:51.286400 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 209/410] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Oct 1 20:32:51 2022] GET /identity/v3/groups/a8505ee3a8444d09b41523ea8381cf48 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:51.287753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6545529-3968-476a-a631-76699e9e5442 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:51.372095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8ea03-bebe-444b-8bfe-8cf82f1d2a7b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:51.396351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6545529-3968-476a-a631-76699e9e5442 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:51.433082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22e58f-162b-4e28-a776-8fc3d1c44b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:51.433816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22e58f-162b-4e28-a776-8fc3d1c44b6b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:51.434166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22e58f-162b-4e28-a776-8fc3d1c44b6b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:51.434566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22e58f-162b-4e28-a776-8fc3d1c44b6b None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:51.434881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:51.435900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22e58f-162b-4e28-a776-8fc3d1c44b6b None admin] RBAC: Authorizing `identity:get_project(project_id=5acc2b4d60e244298e1c3401d66e7636)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:51.437661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22e58f-162b-4e28-a776-8fc3d1c44b6b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:51.445683 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 210/411] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:51 2022] GET /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:51.447441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22e58f-162b-4e28-a776-8fc3d1c44b6b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:51.463188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2b0d0-283b-466c-8375-faba1065e033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:51.464201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2b0d0-283b-466c-8375-faba1065e033 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:51.464651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2b0d0-283b-466c-8375-faba1065e033 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:51.465089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2b0d0-283b-466c-8375-faba1065e033 None admin] PATH_INFO: `/v3/groups/3b4e4f0428644954ae373c4bfaf05a8f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:51.466413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2b0d0-283b-466c-8375-faba1065e033 None admin] RBAC: Authorizing `identity:get_group(group_id=3b4e4f0428644954ae373c4bfaf05a8f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:51.468060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2b0d0-283b-466c-8375-faba1065e033 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:51.475921 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 202/412] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Oct 1 20:32:51 2022] GET /identity/v3/groups/3b4e4f0428644954ae373c4bfaf05a8f => generated 229 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:51.492888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2b0d0-283b-466c-8375-faba1065e033 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:51.525446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:51.571019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22e58f-162b-4e28-a776-8fc3d1c44b6b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:51.617459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db439d-4a3d-4d00-8cc7-8ed82e074e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:51.618211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db439d-4a3d-4d00-8cc7-8ed82e074e8c None admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:51.618585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db439d-4a3d-4d00-8cc7-8ed82e074e8c None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:51.619281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db439d-4a3d-4d00-8cc7-8ed82e074e8c None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636/groups/a8505ee3a8444d09b41523ea8381cf48/roles/8b7b6e0028b04be8912abda33d3ef729` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:51.622365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db439d-4a3d-4d00-8cc7-8ed82e074e8c None admin] RBAC: Authorizing `identity:create_grant(project_id=5acc2b4d60e244298e1c3401d66e7636, group_id=a8505ee3a8444d09b41523ea8381cf48, role_id=8b7b6e0028b04be8912abda33d3ef729)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:51.622738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2b0d0-283b-466c-8375-faba1065e033 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:51.623924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db439d-4a3d-4d00-8cc7-8ed82e074e8c None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:51.650636 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 211/413] 10.222.0.17 () {58 vars in 1442 bytes} [Sat Oct 1 20:32:51 2022] PUT /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636/groups/a8505ee3a8444d09b41523ea8381cf48/roles/8b7b6e0028b04be8912abda33d3ef729 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:32:51.652245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424cd7a-bc41-4aa4-94ba-03a2ef8d9e75 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:51.652661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424cd7a-bc41-4aa4-94ba-03a2ef8d9e75 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:51.653009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424cd7a-bc41-4aa4-94ba-03a2ef8d9e75 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:51.654830 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 212/414] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:51.660862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a767fe5-cf45-4c33-92d7-27c3ec36361a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:51.661190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a767fe5-cf45-4c33-92d7-27c3ec36361a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:51.661473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a767fe5-cf45-4c33-92d7-27c3ec36361a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:51.679288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0f0c0-452e-4e0f-8706-e6c3fd617900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:51.680183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0f0c0-452e-4e0f-8706-e6c3fd617900 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:51.680464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0f0c0-452e-4e0f-8706-e6c3fd617900 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:51.680883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0f0c0-452e-4e0f-8706-e6c3fd617900 None admin] PATH_INFO: `/v3/projects/549fa0b49ffd4207961a9dc3298560fa` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:51.682204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0f0c0-452e-4e0f-8706-e6c3fd617900 None admin] RBAC: Authorizing `identity:get_project(project_id=549fa0b49ffd4207961a9dc3298560fa)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:51.685111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0f0c0-452e-4e0f-8706-e6c3fd617900 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:51.687351 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 203/415] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:51 2022] GET /identity/v3/projects/549fa0b49ffd4207961a9dc3298560fa => generated 344 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:51.698587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0f0c0-452e-4e0f-8706-e6c3fd617900 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:51.703223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3a767fe5-cf45-4c33-92d7-27c3ec36361a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:51.704424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3a767fe5-cf45-4c33-92d7-27c3ec36361a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:51.787262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:51.835651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0f0c0-452e-4e0f-8706-e6c3fd617900 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:51.889699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f40482-4ae5-4a13-89f6-60aa802e8af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:51.890316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f40482-4ae5-4a13-89f6-60aa802e8af6 None admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:51.890602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f40482-4ae5-4a13-89f6-60aa802e8af6 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:51.893356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f40482-4ae5-4a13-89f6-60aa802e8af6 None admin] PATH_INFO: `/v3/projects/549fa0b49ffd4207961a9dc3298560fa/groups/3b4e4f0428644954ae373c4bfaf05a8f/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:51.896317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f40482-4ae5-4a13-89f6-60aa802e8af6 None admin] RBAC: Authorizing `identity:create_grant(project_id=549fa0b49ffd4207961a9dc3298560fa, group_id=3b4e4f0428644954ae373c4bfaf05a8f, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:51.901773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f40482-4ae5-4a13-89f6-60aa802e8af6 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:51.926814 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 213/416] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:32:51.943368 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 204/417] 10.222.0.17 () {58 vars in 1442 bytes} [Sat Oct 1 20:32:51 2022] PUT /identity/v3/projects/549fa0b49ffd4207961a9dc3298560fa/groups/3b4e4f0428644954ae373c4bfaf05a8f/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:32:52.278959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a767fe5-cf45-4c33-92d7-27c3ec36361a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:52.341514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:52.392151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a767fe5-cf45-4c33-92d7-27c3ec36361a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:52.429093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a5f16-502f-4dc5-86d1-6e5c75338514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:52.429723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a5f16-502f-4dc5-86d1-6e5c75338514 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:52.430005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a5f16-502f-4dc5-86d1-6e5c75338514 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:52.430274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a5f16-502f-4dc5-86d1-6e5c75338514 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:52.441565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a5f16-502f-4dc5-86d1-6e5c75338514 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:52.446662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-ed1a5f16-502f-4dc5-86d1-6e5c75338514 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:52.446662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:52.446662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:52.446662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:52.446662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:52.446662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:52.446662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:52.446662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:52.446662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:52.446662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:52.446662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:52.446662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:52.446662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:52.446662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:52.446662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:52.446662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:52.447674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:52.447674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:52.447674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:52.447674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:52.447674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:52.447674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:52.447674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:52.447674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:52.447674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:52.447674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:52.447674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:52.447674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:52.447674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:52.447674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:52.447674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:52.447674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:52.448521 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:52.448521 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:52.448521 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:52.448521 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:52.448521 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:52.448521 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:52.448521 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:52.448521 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:52.448521 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:52.448521 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:52.448521 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:52.448521 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:52.448521 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 214/418] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:52.453880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f40482-4ae5-4a13-89f6-60aa802e8af6 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:52.571030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f40482-4ae5-4a13-89f6-60aa802e8af6 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:52.602745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:52.615243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fce925-84e5-4e5c-9873-050fc236d49e None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:52.615551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fce925-84e5-4e5c-9873-050fc236d49e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:52.615702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e0e65-ba4d-49ff-b34c-d40858e1e904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:52.615889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fce925-84e5-4e5c-9873-050fc236d49e None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:52.616175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e0e65-ba4d-49ff-b34c-d40858e1e904 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:52.616453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e0e65-ba4d-49ff-b34c-d40858e1e904 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:52.616720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e0e65-ba4d-49ff-b34c-d40858e1e904 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:52.617385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e0e65-ba4d-49ff-b34c-d40858e1e904 None admin] RBAC: Adding query filter params (name=member) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:52.617552 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 215/419] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:32:52.617702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e0e65-ba4d-49ff-b34c-d40858e1e904 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:52.619086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e0e65-ba4d-49ff-b34c-d40858e1e904 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:52.622579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445b4b6-b6bc-4943-861d-7dfd6f266151 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:52.622864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445b4b6-b6bc-4943-861d-7dfd6f266151 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:52.623138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445b4b6-b6bc-4943-861d-7dfd6f266151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:52.633853 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 205/420] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:52.640315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e0e65-ba4d-49ff-b34c-d40858e1e904 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:52.649109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c445b4b6-b6bc-4943-861d-7dfd6f266151 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:52.650305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c445b4b6-b6bc-4943-861d-7dfd6f266151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:52.731931 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 216/421] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:32:52.816688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e0e65-ba4d-49ff-b34c-d40858e1e904 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:52.866726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f33b1e-85fb-47b3-bd7b-1852577488a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:52.867257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f33b1e-85fb-47b3-bd7b-1852577488a4 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:52.867536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f33b1e-85fb-47b3-bd7b-1852577488a4 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:52.867860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f33b1e-85fb-47b3-bd7b-1852577488a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:52.934318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:52.950380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f33b1e-85fb-47b3-bd7b-1852577488a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:52.951838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f33b1e-85fb-47b3-bd7b-1852577488a4 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:53.055244 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 206/422] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 01 20:32:53.060966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445b4b6-b6bc-4943-861d-7dfd6f266151 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:53.110217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:53.169328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445b4b6-b6bc-4943-861d-7dfd6f266151 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:53.205236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b2f3e-e74e-45eb-9b98-45a11423ab68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:53.205862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b2f3e-e74e-45eb-9b98-45a11423ab68 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:53.206141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b2f3e-e74e-45eb-9b98-45a11423ab68 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:53.206413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b2f3e-e74e-45eb-9b98-45a11423ab68 None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:53.207550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b2f3e-e74e-45eb-9b98-45a11423ab68 None admin] RBAC: Authorizing `identity:get_project(project_id=a503020af843467a992a8df7f46d3fec)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:53.208977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b2f3e-e74e-45eb-9b98-45a11423ab68 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:53.210970 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 217/423] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:53 2022] GET /identity/v3/projects/a503020af843467a992a8df7f46d3fec => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:53.216542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f33b1e-85fb-47b3-bd7b-1852577488a4 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:53.323397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:53.345018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f33b1e-85fb-47b3-bd7b-1852577488a4 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:53.383532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f523f-25f2-4214-be0e-5e3cbe5d79be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:53.384161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f523f-25f2-4214-be0e-5e3cbe5d79be None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:53.384435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f523f-25f2-4214-be0e-5e3cbe5d79be None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:53.384704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f523f-25f2-4214-be0e-5e3cbe5d79be None admin] PATH_INFO: `/v3/groups/a8505ee3a8444d09b41523ea8381cf48` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:53.385904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f523f-25f2-4214-be0e-5e3cbe5d79be None admin] RBAC: Authorizing `identity:get_group(group_id=a8505ee3a8444d09b41523ea8381cf48)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:53.387369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f523f-25f2-4214-be0e-5e3cbe5d79be None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:53.389249 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 207/424] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Oct 1 20:32:53 2022] GET /identity/v3/groups/a8505ee3a8444d09b41523ea8381cf48 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:53.394831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b2f3e-e74e-45eb-9b98-45a11423ab68 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:53.412403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f523f-25f2-4214-be0e-5e3cbe5d79be None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:53.523766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f523f-25f2-4214-be0e-5e3cbe5d79be None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:53.564449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b2f3e-e74e-45eb-9b98-45a11423ab68 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:53.579650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:53.604963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0de21-7478-4913-80f7-ddc384cd95b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:53.606609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0de21-7478-4913-80f7-ddc384cd95b8 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:53.607475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0de21-7478-4913-80f7-ddc384cd95b8 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:53.608750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0de21-7478-4913-80f7-ddc384cd95b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:53.610108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c0de21-7478-4913-80f7-ddc384cd95b8 None admin] RBAC: Adding query filter params (group.id=a8505ee3a8444d09b41523ea8381cf48, role.id=17f3aeacc5034f3891cc6556da651c1b, scope.project.id=a503020af843467a992a8df7f46d3fec) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:53.611012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c0de21-7478-4913-80f7-ddc384cd95b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:53.612920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c0de21-7478-4913-80f7-ddc384cd95b8 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:53.614184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd0676-e189-4f4f-a17c-7f4c232ac796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:53.614986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd0676-e189-4f4f-a17c-7f4c232ac796 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:53.616203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd0676-e189-4f4f-a17c-7f4c232ac796 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:53.616642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd0676-e189-4f4f-a17c-7f4c232ac796 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:53.624829 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 208/425] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Oct 1 20:32:53 2022] GET /identity/v3/role_assignments?group.id=a8505ee3a8444d09b41523ea8381cf48&scope.project.id=a503020af843467a992a8df7f46d3fec&role.id=17f3aeacc5034f3891cc6556da651c1b => generated 611 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:53.639131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd0676-e189-4f4f-a17c-7f4c232ac796 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:53.645491 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-33cd0676-e189-4f4f-a17c-7f4c232ac796 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:53.645491 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:53.645491 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:53.645491 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:53.645491 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:53.645491 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:53.645491 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:53.645491 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:53.645491 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:53.645491 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:53.645491 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:53.645491 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:53.645491 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:53.645491 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:53.645491 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:53.645491 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:53.646604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:53.646604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:53.646604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:53.646604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:53.646604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:53.646604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:53.646604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:53.646604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:53.646604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:53.646604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:53.646604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:53.646604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:53.646604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:53.646604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:53.646604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:53.646604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:53.647599 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:53.647599 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:53.647599 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:53.647599 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:53.647599 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:53.647599 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:53.647599 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:53.647599 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:53.647599 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:53.647599 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:53.647599 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:53.647599 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:53.649860 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 218/426] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:32:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:53.655480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0de21-7478-4913-80f7-ddc384cd95b8 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:53.692914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:53.815508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0de21-7478-4913-80f7-ddc384cd95b8 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:53.862706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac136ca1-773c-41bf-882f-f00a681a4653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:53.863316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac136ca1-773c-41bf-882f-f00a681a4653 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:53.863594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac136ca1-773c-41bf-882f-f00a681a4653 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:53.863864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac136ca1-773c-41bf-882f-f00a681a4653 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:53.864530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac136ca1-773c-41bf-882f-f00a681a4653 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:53.864829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac136ca1-773c-41bf-882f-f00a681a4653 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:53.866192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac136ca1-773c-41bf-882f-f00a681a4653 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:53.878598 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 209/427] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:32:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:53.884253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd0676-e189-4f4f-a17c-7f4c232ac796 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:53.970627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97aac1-9246-4afd-a4e3-cf46d0d87aab None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:53.972087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97aac1-9246-4afd-a4e3-cf46d0d87aab None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:53.972570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97aac1-9246-4afd-a4e3-cf46d0d87aab None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:53.974457 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 210/428] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:32:53.979921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b1272-a87b-4f8c-bf09-eb4bd8f6c5af None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:53.980423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b1272-a87b-4f8c-bf09-eb4bd8f6c5af None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:53.980869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b1272-a87b-4f8c-bf09-eb4bd8f6c5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:54.000703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-590b1272-a87b-4f8c-bf09-eb4bd8f6c5af None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:54.002320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-590b1272-a87b-4f8c-bf09-eb4bd8f6c5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:54.030889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd0676-e189-4f4f-a17c-7f4c232ac796 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:54.073519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d631e9d-eaea-40b9-8828-c0ac6153dfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:54.074049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d631e9d-eaea-40b9-8828-c0ac6153dfb3 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:54.074365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d631e9d-eaea-40b9-8828-c0ac6153dfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:54.074897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d631e9d-eaea-40b9-8828-c0ac6153dfb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:54.086463 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 211/429] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:32:54.123724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:54.154738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d631e9d-eaea-40b9-8828-c0ac6153dfb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:54.156195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d631e9d-eaea-40b9-8828-c0ac6153dfb3 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:54.269159 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 219/430] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:54.274335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b1272-a87b-4f8c-bf09-eb4bd8f6c5af None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:54.292879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:54.400861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b1272-a87b-4f8c-bf09-eb4bd8f6c5af None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:54.443108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6103be8-e33a-48e8-80e6-ed532c6618cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:54.443748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6103be8-e33a-48e8-80e6-ed532c6618cb None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:54.444030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6103be8-e33a-48e8-80e6-ed532c6618cb None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:54.444302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6103be8-e33a-48e8-80e6-ed532c6618cb None admin] PATH_INFO: `/v3/projects/a503020af843467a992a8df7f46d3fec` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:54.445421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6103be8-e33a-48e8-80e6-ed532c6618cb None admin] RBAC: Authorizing `identity:get_project(project_id=a503020af843467a992a8df7f46d3fec)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:54.447158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6103be8-e33a-48e8-80e6-ed532c6618cb None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:54.449134 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 212/431] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:54 2022] GET /identity/v3/projects/a503020af843467a992a8df7f46d3fec => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:54.454404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d631e9d-eaea-40b9-8828-c0ac6153dfb3 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:54.476448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6103be8-e33a-48e8-80e6-ed532c6618cb None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:54.542414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:54.565499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:54.570752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d631e9d-eaea-40b9-8828-c0ac6153dfb3 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:54.590288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6103be8-e33a-48e8-80e6-ed532c6618cb None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:54.611172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8613261-e26b-4654-a8aa-42e56431e865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:54.611922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8613261-e26b-4654-a8aa-42e56431e865 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:54.612198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8613261-e26b-4654-a8aa-42e56431e865 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:54.612467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8613261-e26b-4654-a8aa-42e56431e865 None admin] PATH_INFO: `/v3/groups/a8505ee3a8444d09b41523ea8381cf48` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:54.613946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8613261-e26b-4654-a8aa-42e56431e865 None admin] RBAC: Authorizing `identity:get_group(group_id=a8505ee3a8444d09b41523ea8381cf48)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:54.615884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8613261-e26b-4654-a8aa-42e56431e865 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:54.617902 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 220/432] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Oct 1 20:32:54 2022] GET /identity/v3/groups/a8505ee3a8444d09b41523ea8381cf48 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:54.625878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8613261-e26b-4654-a8aa-42e56431e865 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:54.641370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77387a17-b105-4c8c-97be-99846dbd3001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:54.642635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77387a17-b105-4c8c-97be-99846dbd3001 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:54.643113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77387a17-b105-4c8c-97be-99846dbd3001 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:54.643594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77387a17-b105-4c8c-97be-99846dbd3001 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:54.659987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77387a17-b105-4c8c-97be-99846dbd3001 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:54.667859 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-77387a17-b105-4c8c-97be-99846dbd3001 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:54.667859 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:54.667859 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:54.667859 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:54.667859 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:54.667859 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:54.667859 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:54.667859 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:54.667859 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:54.667859 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:54.667859 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:54.667859 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:54.667859 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:54.667859 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:54.667859 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:54.667859 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:54.668906 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:54.668906 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:54.668906 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:54.668906 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:54.668906 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:54.668906 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:54.668906 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:54.668906 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:54.668906 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:54.668906 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:54.668906 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:54.668906 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:54.668906 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:54.668906 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:54.668906 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:54.668906 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:54.669889 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:54.669889 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:54.669889 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:54.669889 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:54.669889 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:54.669889 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:54.669889 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:54.669889 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:54.669889 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:54.669889 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:54.669889 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 01 20:32:54.669889 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:32:54.672134 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 213/433] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:32:54.679051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77387a17-b105-4c8c-97be-99846dbd3001 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:54.767228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8613261-e26b-4654-a8aa-42e56431e865 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:54.804876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77387a17-b105-4c8c-97be-99846dbd3001 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:54.818871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c987585b-26f8-489f-b2d6-8c59afb468ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:54.819065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987585b-26f8-489f-b2d6-8c59afb468ef None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:54.819317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987585b-26f8-489f-b2d6-8c59afb468ef None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:54.819654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987585b-26f8-489f-b2d6-8c59afb468ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:54.820333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c987585b-26f8-489f-b2d6-8c59afb468ef None admin] RBAC: Adding query filter params (group.id=a8505ee3a8444d09b41523ea8381cf48, role.id=8b7b6e0028b04be8912abda33d3ef729, scope.project.id=a503020af843467a992a8df7f46d3fec) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:54.820647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c987585b-26f8-489f-b2d6-8c59afb468ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:54.823274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c987585b-26f8-489f-b2d6-8c59afb468ef None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:54.831945 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 221/434] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Oct 1 20:32:54 2022] GET /identity/v3/role_assignments?group.id=a8505ee3a8444d09b41523ea8381cf48&scope.project.id=a503020af843467a992a8df7f46d3fec&role.id=8b7b6e0028b04be8912abda33d3ef729 => generated 611 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:54.833183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506ebec-86df-4471-b150-514855e5e919 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:54.833466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506ebec-86df-4471-b150-514855e5e919 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:54.833737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506ebec-86df-4471-b150-514855e5e919 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:54.835658 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 222/435] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:54.843061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ea675-531b-412b-862e-003694fbdf60 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:54.843659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ea675-531b-412b-862e-003694fbdf60 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:54.844328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ea675-531b-412b-862e-003694fbdf60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:54.861511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:54.869249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b6d3f-2afa-4746-bf11-8d6e47db73e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:54.869932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d36ea675-531b-412b-862e-003694fbdf60 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:54.870148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b6d3f-2afa-4746-bf11-8d6e47db73e6 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:54.870542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b6d3f-2afa-4746-bf11-8d6e47db73e6 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:54.870928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b6d3f-2afa-4746-bf11-8d6e47db73e6 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:54.871728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b6d3f-2afa-4746-bf11-8d6e47db73e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:54.872075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d36ea675-531b-412b-862e-003694fbdf60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:54.872243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b6d3f-2afa-4746-bf11-8d6e47db73e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:54.873776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b6d3f-2afa-4746-bf11-8d6e47db73e6 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:54.886701 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 214/436] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:32:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:54.887926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:54.893131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b6d3f-2afa-4746-bf11-8d6e47db73e6 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:54.982915 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 223/437] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:32:55.039045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b6d3f-2afa-4746-bf11-8d6e47db73e6 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:55.091999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550ecec-770f-453a-a824-a4b1ae29fbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:55.092624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550ecec-770f-453a-a824-a4b1ae29fbe3 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:55.092906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550ecec-770f-453a-a824-a4b1ae29fbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:55.093185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550ecec-770f-453a-a824-a4b1ae29fbe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:55.115318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7213db-3b07-435e-9c6e-0d7f26f027a3 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:55.115616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7213db-3b07-435e-9c6e-0d7f26f027a3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:55.115869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7213db-3b07-435e-9c6e-0d7f26f027a3 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:55.117469 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 224/438] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:32:55.122461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91431e-df4e-436d-aee5-b5e67dbe112d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:55.122776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91431e-df4e-436d-aee5-b5e67dbe112d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:55.123041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91431e-df4e-436d-aee5-b5e67dbe112d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:55.143980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8d91431e-df4e-436d-aee5-b5e67dbe112d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:55.145175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8d91431e-df4e-436d-aee5-b5e67dbe112d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:55.169720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:55.172743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550ecec-770f-453a-a824-a4b1ae29fbe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:55.175486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550ecec-770f-453a-a824-a4b1ae29fbe3 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:55.247870 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 225/439] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:32:55.293248 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 215/440] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 01 20:32:55.298809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91431e-df4e-436d-aee5-b5e67dbe112d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:55.308454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550ecec-770f-453a-a824-a4b1ae29fbe3 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:55.386752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:55.429448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91431e-df4e-436d-aee5-b5e67dbe112d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:55.432171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:55.435419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550ecec-770f-453a-a824-a4b1ae29fbe3 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:55.466583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841c663-efa1-4f84-a060-649cb7e85f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:55.467301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841c663-efa1-4f84-a060-649cb7e85f9b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:55.467657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841c663-efa1-4f84-a060-649cb7e85f9b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:55.467932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841c663-efa1-4f84-a060-649cb7e85f9b None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:55.469071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841c663-efa1-4f84-a060-649cb7e85f9b None admin] RBAC: Authorizing `identity:get_project(project_id=5acc2b4d60e244298e1c3401d66e7636)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:55.470147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d3420-5b12-4502-8122-02e780473b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:55.470453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841c663-efa1-4f84-a060-649cb7e85f9b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:55.471312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d3420-5b12-4502-8122-02e780473b3b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:55.471589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d3420-5b12-4502-8122-02e780473b3b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:55.471855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d3420-5b12-4502-8122-02e780473b3b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:55.472848 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 226/441] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:55 2022] GET /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:55.478120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841c663-efa1-4f84-a060-649cb7e85f9b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:55.482400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d3420-5b12-4502-8122-02e780473b3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:55.487929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-217d3420-5b12-4502-8122-02e780473b3b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:55.487929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:55.487929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:55.487929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:55.487929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:55.487929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:55.487929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:55.487929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:55.487929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:55.487929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:55.487929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:55.487929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:55.487929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:55.487929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:55.487929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:55.487929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:55.488870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:55.488870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:55.488870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:55.488870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:55.488870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:55.488870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:55.488870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:55.488870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:55.488870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:55.488870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:55.488870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:55.488870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:55.488870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:55.488870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:55.488870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:55.488870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:55.491633 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:55.491633 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:55.491633 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:55.491633 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:55.491633 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:55.491633 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:55.491633 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:55.491633 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:55.491633 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:55.491633 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:55.491633 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:32:55.491633 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:32:55.491633 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 216/442] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:32:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:32:55.494765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d3420-5b12-4502-8122-02e780473b3b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:55.584967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841c663-efa1-4f84-a060-649cb7e85f9b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:55.598732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d3420-5b12-4502-8122-02e780473b3b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:55.618907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f7d2d-ba64-40eb-a2a3-dc911d99927c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:55.619637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f7d2d-ba64-40eb-a2a3-dc911d99927c None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:55.619815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f7d2d-ba64-40eb-a2a3-dc911d99927c None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:55.620090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f7d2d-ba64-40eb-a2a3-dc911d99927c None admin] PATH_INFO: `/v3/groups/a8505ee3a8444d09b41523ea8381cf48` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:55.621207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f7d2d-ba64-40eb-a2a3-dc911d99927c None admin] RBAC: Authorizing `identity:get_group(group_id=a8505ee3a8444d09b41523ea8381cf48)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:55.622665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f7d2d-ba64-40eb-a2a3-dc911d99927c None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:55.624212 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 227/443] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Oct 1 20:32:55 2022] GET /identity/v3/groups/a8505ee3a8444d09b41523ea8381cf48 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:55.625034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f7d2d-ba64-40eb-a2a3-dc911d99927c None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:55.628088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:55.637544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd0107-fe8e-4957-812a-d39ad7646811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:55.638460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd0107-fe8e-4957-812a-d39ad7646811 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:55.638795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd0107-fe8e-4957-812a-d39ad7646811 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:55.639069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd0107-fe8e-4957-812a-d39ad7646811 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:55.639735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd0107-fe8e-4957-812a-d39ad7646811 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:55.640038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd0107-fe8e-4957-812a-d39ad7646811 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:55.641409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd0107-fe8e-4957-812a-d39ad7646811 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:55.649242 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 217/444] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:32:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:55.650001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd0107-fe8e-4957-812a-d39ad7646811 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:55.674463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:55.729185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f7d2d-ba64-40eb-a2a3-dc911d99927c None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:55.751573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd0107-fe8e-4957-812a-d39ad7646811 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:55.762145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351573c2-df01-45a3-b01b-44702161b66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:55.762813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351573c2-df01-45a3-b01b-44702161b66d None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:55.763094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351573c2-df01-45a3-b01b-44702161b66d None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:55.763364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351573c2-df01-45a3-b01b-44702161b66d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:55.773836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351573c2-df01-45a3-b01b-44702161b66d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:32:55.779000 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-351573c2-df01-45a3-b01b-44702161b66d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:32:55.779000 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:32:55.779000 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:32:55.779000 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:32:55.779000 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:32:55.779000 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:32:55.779000 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:32:55.779000 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:32:55.779000 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:32:55.779000 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:32:55.779000 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:32:55.779000 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:32:55.779000 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:32:55.779000 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:32:55.779000 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:32:55.779000 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:32:55.780065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:32:55.780065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:32:55.780065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:32:55.780065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:32:55.780065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:32:55.780065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:32:55.780065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:32:55.780065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:32:55.780065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:32:55.780065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:32:55.780065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:32:55.780065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:32:55.780065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:32:55.780065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:32:55.780065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:32:55.780065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:32:55.780879 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:32:55.780879 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:32:55.780879 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:32:55.780879 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:32:55.780879 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:32:55.780879 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:32:55.780879 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:32:55.780879 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:32:55.780879 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:32:55.780879 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:32:55.780879 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:32:55.780879 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:32:55.780879 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 228/445] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:32:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:32:55.781531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351573c2-df01-45a3-b01b-44702161b66d None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:55.788279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfeb5e9-d01f-463f-abf4-ec030bba6b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:55.788887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeb5e9-d01f-463f-abf4-ec030bba6b1c None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:55.789162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeb5e9-d01f-463f-abf4-ec030bba6b1c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:55.789436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeb5e9-d01f-463f-abf4-ec030bba6b1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:55.790154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfeb5e9-d01f-463f-abf4-ec030bba6b1c None admin] RBAC: Adding query filter params (group.id=a8505ee3a8444d09b41523ea8381cf48, role.id=17f3aeacc5034f3891cc6556da651c1b, scope.project.id=5acc2b4d60e244298e1c3401d66e7636) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:55.790454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfeb5e9-d01f-463f-abf4-ec030bba6b1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:55.791900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfeb5e9-d01f-463f-abf4-ec030bba6b1c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:55.797627 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 218/446] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Oct 1 20:32:55 2022] GET /identity/v3/role_assignments?group.id=a8505ee3a8444d09b41523ea8381cf48&scope.project.id=5acc2b4d60e244298e1c3401d66e7636&role.id=17f3aeacc5034f3891cc6556da651c1b => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:55.798378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfeb5e9-d01f-463f-abf4-ec030bba6b1c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:55.852001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:55.895746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:55.899683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351573c2-df01-45a3-b01b-44702161b66d None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:55.913397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfeb5e9-d01f-463f-abf4-ec030bba6b1c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:55.938971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716edc8-2bee-42d6-9843-d6dc72124b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:55.940115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716edc8-2bee-42d6-9843-d6dc72124b60 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:55.940317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716edc8-2bee-42d6-9843-d6dc72124b60 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:55.940579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716edc8-2bee-42d6-9843-d6dc72124b60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:55.955391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a6d62-00ac-44af-8409-17e6717a2890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:55.956010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a6d62-00ac-44af-8409-17e6717a2890 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:55.956663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a6d62-00ac-44af-8409-17e6717a2890 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:55.956942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a6d62-00ac-44af-8409-17e6717a2890 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:55.957924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a6d62-00ac-44af-8409-17e6717a2890 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:55.958242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a6d62-00ac-44af-8409-17e6717a2890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:55.959721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a6d62-00ac-44af-8409-17e6717a2890 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:55.975265 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 219/447] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:32:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:55.977174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a6d62-00ac-44af-8409-17e6717a2890 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:56.025512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716edc8-2bee-42d6-9843-d6dc72124b60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:56.027533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716edc8-2bee-42d6-9843-d6dc72124b60 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:56.094989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a6d62-00ac-44af-8409-17e6717a2890 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:56.108743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:56.137456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526379e-b305-4931-8dba-cff16d86467f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:56.138068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526379e-b305-4931-8dba-cff16d86467f None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:56.138348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526379e-b305-4931-8dba-cff16d86467f None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:56.138656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526379e-b305-4931-8dba-cff16d86467f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:56.153602 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 229/448] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:32:56.160779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716edc8-2bee-42d6-9843-d6dc72124b60 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:56.164174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:56.224869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526379e-b305-4931-8dba-cff16d86467f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:56.226658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526379e-b305-4931-8dba-cff16d86467f None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:56.272831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716edc8-2bee-42d6-9843-d6dc72124b60 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:56.306040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c3a3fe-94ea-4405-8d97-14fadafd28d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:56.306682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3a3fe-94ea-4405-8d97-14fadafd28d5 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:56.306961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3a3fe-94ea-4405-8d97-14fadafd28d5 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:56.307230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3a3fe-94ea-4405-8d97-14fadafd28d5 None admin] PATH_INFO: `/v3/projects/5acc2b4d60e244298e1c3401d66e7636` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:56.308334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c3a3fe-94ea-4405-8d97-14fadafd28d5 None admin] RBAC: Authorizing `identity:get_project(project_id=5acc2b4d60e244298e1c3401d66e7636)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:56.309744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c3a3fe-94ea-4405-8d97-14fadafd28d5 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:56.311357 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 230/449] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:56 2022] GET /identity/v3/projects/5acc2b4d60e244298e1c3401d66e7636 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:56.316865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c3a3fe-94ea-4405-8d97-14fadafd28d5 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:56.324565 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 220/450] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:32:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 01 20:32:56.330514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526379e-b305-4931-8dba-cff16d86467f None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:56.348542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:56.379979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:56.417649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c3a3fe-94ea-4405-8d97-14fadafd28d5 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:56.429809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526379e-b305-4931-8dba-cff16d86467f None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:56.451530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d98f1-655f-4cfa-b032-2897144e4cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:56.452147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d98f1-655f-4cfa-b032-2897144e4cdb None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:56.452424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d98f1-655f-4cfa-b032-2897144e4cdb None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:56.452697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d98f1-655f-4cfa-b032-2897144e4cdb None admin] PATH_INFO: `/v3/groups/a8505ee3a8444d09b41523ea8381cf48` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:56.453821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d98f1-655f-4cfa-b032-2897144e4cdb None admin] RBAC: Authorizing `identity:get_group(group_id=a8505ee3a8444d09b41523ea8381cf48)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:56.455779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d98f1-655f-4cfa-b032-2897144e4cdb None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:56.457495 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 231/451] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Oct 1 20:32:56 2022] GET /identity/v3/groups/a8505ee3a8444d09b41523ea8381cf48 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:56.462999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d98f1-655f-4cfa-b032-2897144e4cdb None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:56.463778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ba3f5-9707-48a6-9274-6d3c35768933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:56.464403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ba3f5-9707-48a6-9274-6d3c35768933 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:56.464679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ba3f5-9707-48a6-9274-6d3c35768933 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:56.464949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ba3f5-9707-48a6-9274-6d3c35768933 None admin] PATH_INFO: `/v3/projects/549fa0b49ffd4207961a9dc3298560fa` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:56.466089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ba3f5-9707-48a6-9274-6d3c35768933 None admin] RBAC: Authorizing `identity:get_project(project_id=549fa0b49ffd4207961a9dc3298560fa)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:56.467559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ba3f5-9707-48a6-9274-6d3c35768933 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:56.469276 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 221/452] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:32:56 2022] GET /identity/v3/projects/549fa0b49ffd4207961a9dc3298560fa => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:56.474427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ba3f5-9707-48a6-9274-6d3c35768933 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:56.556889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:56.560490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d98f1-655f-4cfa-b032-2897144e4cdb None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:56.571125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ba3f5-9707-48a6-9274-6d3c35768933 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:56.588725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:56.593942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a68acd-d5a8-4fdb-8c2f-64e8bea97723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:56.594569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a68acd-d5a8-4fdb-8c2f-64e8bea97723 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:56.594850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a68acd-d5a8-4fdb-8c2f-64e8bea97723 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:56.595118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a68acd-d5a8-4fdb-8c2f-64e8bea97723 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:56.596401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a68acd-d5a8-4fdb-8c2f-64e8bea97723 None admin] RBAC: Adding query filter params (group.id=a8505ee3a8444d09b41523ea8381cf48, role.id=8b7b6e0028b04be8912abda33d3ef729, scope.project.id=5acc2b4d60e244298e1c3401d66e7636) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:56.596703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a68acd-d5a8-4fdb-8c2f-64e8bea97723 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:56.598116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a68acd-d5a8-4fdb-8c2f-64e8bea97723 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:56.603814 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 232/453] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Oct 1 20:32:56 2022] GET /identity/v3/role_assignments?group.id=a8505ee3a8444d09b41523ea8381cf48&scope.project.id=5acc2b4d60e244298e1c3401d66e7636&role.id=8b7b6e0028b04be8912abda33d3ef729 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:56.615974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911f7db-8269-4afd-af03-509bdf5cfa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:56.616756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911f7db-8269-4afd-af03-509bdf5cfa62 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:56.617103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911f7db-8269-4afd-af03-509bdf5cfa62 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:56.617444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911f7db-8269-4afd-af03-509bdf5cfa62 None admin] PATH_INFO: `/v3/groups/3b4e4f0428644954ae373c4bfaf05a8f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:56.618753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911f7db-8269-4afd-af03-509bdf5cfa62 None admin] RBAC: Authorizing `identity:get_group(group_id=3b4e4f0428644954ae373c4bfaf05a8f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:56.620232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911f7db-8269-4afd-af03-509bdf5cfa62 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:56.621877 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 222/454] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Oct 1 20:32:56 2022] GET /identity/v3/groups/3b4e4f0428644954ae373c4bfaf05a8f => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:32:56.628508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a68acd-d5a8-4fdb-8c2f-64e8bea97723 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:32:56.733884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a68acd-d5a8-4fdb-8c2f-64e8bea97723 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:32:56.769145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1adbc7-4aec-4157-a875-bea0cbc58055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:32:56.769757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1adbc7-4aec-4157-a875-bea0cbc58055 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:56.770035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1adbc7-4aec-4157-a875-bea0cbc58055 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:56.770303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1adbc7-4aec-4157-a875-bea0cbc58055 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:56.771070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1adbc7-4aec-4157-a875-bea0cbc58055 None admin] RBAC: Adding query filter params (group.id=3b4e4f0428644954ae373c4bfaf05a8f, role.id=1c50896511054a5f994bee3d99fa0619, scope.project.id=549fa0b49ffd4207961a9dc3298560fa) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:32:56.771383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1adbc7-4aec-4157-a875-bea0cbc58055 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:32:56.772829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1adbc7-4aec-4157-a875-bea0cbc58055 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:32:56.779346 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 233/455] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Oct 1 20:32:56 2022] GET /identity/v3/role_assignments?group.id=3b4e4f0428644954ae373c4bfaf05a8f&scope.project.id=549fa0b49ffd4207961a9dc3298560fa&role.id=1c50896511054a5f994bee3d99fa0619 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:32:59.747390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f71695-80d6-40c2-b743-d3829b139a31 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:59.747976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f71695-80d6-40c2-b743-d3829b139a31 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:59.747976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f71695-80d6-40c2-b743-d3829b139a31 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:59.749524 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 223/456] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:32:59.755166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500fd29-5b2f-4416-a9e8-4fb057538f9a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:32:59.755562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500fd29-5b2f-4416-a9e8-4fb057538f9a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:32:59.755728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500fd29-5b2f-4416-a9e8-4fb057538f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:32:59.779693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4500fd29-5b2f-4416-a9e8-4fb057538f9a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:32:59.780863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4500fd29-5b2f-4416-a9e8-4fb057538f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:32:59.810294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:32:59.867829 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 234/457] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:32:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:00.014318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a819b93-3209-4224-a888-a28c5b4004f3 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:00.014632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a819b93-3209-4224-a888-a28c5b4004f3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:00.014821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a819b93-3209-4224-a888-a28c5b4004f3 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:00.016411 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 224/458] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:00.028153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38c2b5-ab8f-4539-a017-41f5c3ff5d07 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:00.028153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38c2b5-ab8f-4539-a017-41f5c3ff5d07 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:00.028153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38c2b5-ab8f-4539-a017-41f5c3ff5d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:00.056325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2c38c2b5-ab8f-4539-a017-41f5c3ff5d07 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:00.057530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2c38c2b5-ab8f-4539-a017-41f5c3ff5d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:00.158219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4a448-b31a-477e-8986-3ab026768072 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:00.158642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4a448-b31a-477e-8986-3ab026768072 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:00.158794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4a448-b31a-477e-8986-3ab026768072 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:00.160574 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 225/460] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:00.160702 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 235/460] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:00.166203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a2bce-45fa-4e13-aa6c-22de5881a27f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:00.166736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a2bce-45fa-4e13-aa6c-22de5881a27f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:00.167234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a2bce-45fa-4e13-aa6c-22de5881a27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:00.205854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d75a2bce-45fa-4e13-aa6c-22de5881a27f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:00.207383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d75a2bce-45fa-4e13-aa6c-22de5881a27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:00.318264 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 236/461] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:00.365882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4a448-b31a-477e-8986-3ab026768072 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:00.499519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4a448-b31a-477e-8986-3ab026768072 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:00.523481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01552821-1318-47a2-a045-2cc94dc067dd None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:00.524073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01552821-1318-47a2-a045-2cc94dc067dd None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:00.524568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01552821-1318-47a2-a045-2cc94dc067dd None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:00.526797 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 237/462] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:33:00.531849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537703f-bc7a-4226-8e7b-2fbe924c2b7a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:00.532161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537703f-bc7a-4226-8e7b-2fbe924c2b7a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:00.532426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537703f-bc7a-4226-8e7b-2fbe924c2b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:00.545863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865b5e0-e953-4bb8-ab7e-b467d263046c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:00.546526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865b5e0-e953-4bb8-ab7e-b467d263046c None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:00.546800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865b5e0-e953-4bb8-ab7e-b467d263046c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:00.547074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865b5e0-e953-4bb8-ab7e-b467d263046c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:00.559561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865b5e0-e953-4bb8-ab7e-b467d263046c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:00.561153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865b5e0-e953-4bb8-ab7e-b467d263046c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:00.561458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2537703f-bc7a-4226-8e7b-2fbe924c2b7a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:00.563149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2537703f-bc7a-4226-8e7b-2fbe924c2b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:00.575836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:00.581824 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-2865b5e0-e953-4bb8-ab7e-b467d263046c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:00.581824 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:00.581824 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:00.581824 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:00.581824 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:00.581824 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:00.581824 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:00.581824 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:00.581824 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:00.581824 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:00.581824 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:00.581824 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:00.581824 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:00.581824 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:00.581824 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:00.581824 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:00.582970 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:00.582970 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:00.582970 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:00.582970 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:00.582970 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:00.582970 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:00.582970 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:00.582970 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:00.582970 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:00.582970 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:00.582970 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:00.582970 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:00.582970 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:00.582970 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:00.582970 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:00.582970 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:00.582970 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:00.584268 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:00.584268 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:00.584268 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:00.584268 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 226/463] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:00.584268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865b5e0-e953-4bb8-ab7e-b467d263046c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:00.638866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:00.639589 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 238/464] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:00.640756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537703f-bc7a-4226-8e7b-2fbe924c2b7a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:00.709753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865b5e0-e953-4bb8-ab7e-b467d263046c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:00.753596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fd862-f5bd-4311-8139-b28bc01b9c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:00.754357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fd862-f5bd-4311-8139-b28bc01b9c1c None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:00.754851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fd862-f5bd-4311-8139-b28bc01b9c1c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:00.755107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fd862-f5bd-4311-8139-b28bc01b9c1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:00.770303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fd862-f5bd-4311-8139-b28bc01b9c1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:00.772351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fd862-f5bd-4311-8139-b28bc01b9c1c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:00.789680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537703f-bc7a-4226-8e7b-2fbe924c2b7a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:00.791538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-434fd862-f5bd-4311-8139-b28bc01b9c1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:00.791538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:00.791538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:00.791538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:00.791538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:00.791538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:00.791538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:00.791538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:00.791538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:00.791538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:00.791538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:00.791538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:00.791538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:00.791538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:00.791538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:00.791538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:00.792402 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:00.792402 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:00.792402 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:00.792402 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:00.792402 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:00.792402 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:00.792402 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:00.792402 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:00.792402 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:00.792402 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:00.792402 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:00.792402 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:00.792402 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:00.792402 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:00.792402 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:00.792402 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:00.792402 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:00.793236 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:00.793236 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:00.793236 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:00.794443 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 227/465] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:00.794658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fd862-f5bd-4311-8139-b28bc01b9c1c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:00.827063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704f8104-0aa3-4c05-a0c9-cf17131f2c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:00.828123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f8104-0aa3-4c05-a0c9-cf17131f2c5c None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:00.828747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f8104-0aa3-4c05-a0c9-cf17131f2c5c None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:00.829032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f8104-0aa3-4c05-a0c9-cf17131f2c5c None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:00.830691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704f8104-0aa3-4c05-a0c9-cf17131f2c5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:00.831066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704f8104-0aa3-4c05-a0c9-cf17131f2c5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:00.833335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704f8104-0aa3-4c05-a0c9-cf17131f2c5c None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:00.848336 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 239/466] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:00.849297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704f8104-0aa3-4c05-a0c9-cf17131f2c5c None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:00.859511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:00.908728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:00.925416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fd862-f5bd-4311-8139-b28bc01b9c1c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:00.974206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704f8104-0aa3-4c05-a0c9-cf17131f2c5c None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:00.980616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fea10-e3e3-4fe3-b5e8-b04aa3be42f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:00.981488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fea10-e3e3-4fe3-b5e8-b04aa3be42f6 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:00.981970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fea10-e3e3-4fe3-b5e8-b04aa3be42f6 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:00.982463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fea10-e3e3-4fe3-b5e8-b04aa3be42f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:00.998476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fea10-e3e3-4fe3-b5e8-b04aa3be42f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:01.000669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fea10-e3e3-4fe3-b5e8-b04aa3be42f6 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:01.012419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604bdc52-3963-4fa6-ac16-60d2a27d19a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:01.012823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604bdc52-3963-4fa6-ac16-60d2a27d19a4 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:01.013182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604bdc52-3963-4fa6-ac16-60d2a27d19a4 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:01.013526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604bdc52-3963-4fa6-ac16-60d2a27d19a4 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:01.014246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604bdc52-3963-4fa6-ac16-60d2a27d19a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:01.014657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604bdc52-3963-4fa6-ac16-60d2a27d19a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:01.016079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604bdc52-3963-4fa6-ac16-60d2a27d19a4 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:01.021538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-762fea10-e3e3-4fe3-b5e8-b04aa3be42f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:01.021538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:01.021538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:01.021538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:01.021538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:01.021538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:01.021538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:01.021538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:01.021538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:01.021538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:01.021538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:01.021538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:01.021538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:01.021538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:01.021538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:01.021538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:01.022743 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:01.022743 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:01.022743 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:01.022743 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:01.022743 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:01.022743 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:01.022743 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:01.022743 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:01.022743 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:01.022743 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:01.022743 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:01.022743 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:01.022743 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:01.022743 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:01.022743 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:01.022743 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:01.022743 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:01.023862 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:01.023862 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:01.023862 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:01.031023 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 228/467] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:01.031023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fea10-e3e3-4fe3-b5e8-b04aa3be42f6 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:01.038938 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 240/468] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:01.040028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604bdc52-3963-4fa6-ac16-60d2a27d19a4 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:01.110721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:01.142326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fea10-e3e3-4fe3-b5e8-b04aa3be42f6 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:01.149293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604bdc52-3963-4fa6-ac16-60d2a27d19a4 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:01.176969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd875c14-122a-45b6-b893-9be711dbf6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:01.177588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd875c14-122a-45b6-b893-9be711dbf6b4 None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:01.177866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd875c14-122a-45b6-b893-9be711dbf6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:01.178137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd875c14-122a-45b6-b893-9be711dbf6b4 None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:01.178806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd875c14-122a-45b6-b893-9be711dbf6b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:01.180261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd875c14-122a-45b6-b893-9be711dbf6b4 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:01.186182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417681ca-6661-4b6e-9d8a-a9dad528e72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:01.186871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417681ca-6661-4b6e-9d8a-a9dad528e72c None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:01.187150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417681ca-6661-4b6e-9d8a-a9dad528e72c None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:01.187417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417681ca-6661-4b6e-9d8a-a9dad528e72c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:01.193495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:01.201000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417681ca-6661-4b6e-9d8a-a9dad528e72c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:01.202760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417681ca-6661-4b6e-9d8a-a9dad528e72c None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:01.205272 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 229/469] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Oct 1 20:33:01 2022] POST /identity/v3/users => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:33:01.206395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd875c14-122a-45b6-b893-9be711dbf6b4 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:01.230206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-417681ca-6661-4b6e-9d8a-a9dad528e72c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:01.230206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:01.230206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:01.230206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:01.230206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:01.230206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:01.230206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:01.230206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:01.230206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:01.230206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:01.230206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:01.230206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:01.230206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:01.230206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:01.230206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:01.230206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:01.231309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:01.231309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:01.231309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:01.231309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:01.231309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:01.231309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:01.231309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:01.231309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:01.231309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:01.231309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:01.231309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:01.231309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:01.231309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:01.231309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:01.231309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:01.231309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:01.231309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:01.232167 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:01.232167 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:01.232167 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:01.234766 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 241/470] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:01.235929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417681ca-6661-4b6e-9d8a-a9dad528e72c None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:01.348780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd875c14-122a-45b6-b893-9be711dbf6b4 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:01.373101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417681ca-6661-4b6e-9d8a-a9dad528e72c None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:01.394070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:01.397372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36d6a2-ffc4-4a50-879f-c6d95d4834b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:01.397372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36d6a2-ffc4-4a50-879f-c6d95d4834b0 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:01.397372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36d6a2-ffc4-4a50-879f-c6d95d4834b0 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:01.397372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36d6a2-ffc4-4a50-879f-c6d95d4834b0 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:01.397858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36d6a2-ffc4-4a50-879f-c6d95d4834b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:01.398096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36d6a2-ffc4-4a50-879f-c6d95d4834b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:01.399601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36d6a2-ffc4-4a50-879f-c6d95d4834b0 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:01.417553 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 230/471] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:01.417553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36d6a2-ffc4-4a50-879f-c6d95d4834b0 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:01.419504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404176ce-de5e-47e1-bf82-15dd09baffd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:01.419504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404176ce-de5e-47e1-bf82-15dd09baffd8 None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:01.419707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404176ce-de5e-47e1-bf82-15dd09baffd8 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:01.419965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404176ce-de5e-47e1-bf82-15dd09baffd8 None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:01.420665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404176ce-de5e-47e1-bf82-15dd09baffd8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:01.422241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404176ce-de5e-47e1-bf82-15dd09baffd8 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:01.447920 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 242/472] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Oct 1 20:33:01 2022] POST /identity/v3/users => generated 248 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:33:01.449142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfbbfa-0a70-444f-9a8b-8dcd280fe849 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:01.449428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfbbfa-0a70-444f-9a8b-8dcd280fe849 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:01.449760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfbbfa-0a70-444f-9a8b-8dcd280fe849 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:01.451273 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 243/473] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:01.452171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfbbfa-0a70-444f-9a8b-8dcd280fe849 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:01.490885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:01.543802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36d6a2-ffc4-4a50-879f-c6d95d4834b0 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:01.591468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225be18-ebe0-49ab-bf8c-d687a0bf41dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:01.591946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfbbfa-0a70-444f-9a8b-8dcd280fe849 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:01.592079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225be18-ebe0-49ab-bf8c-d687a0bf41dd None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:01.592367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225be18-ebe0-49ab-bf8c-d687a0bf41dd None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:01.592639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225be18-ebe0-49ab-bf8c-d687a0bf41dd None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:01.593297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225be18-ebe0-49ab-bf8c-d687a0bf41dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:01.593911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225be18-ebe0-49ab-bf8c-d687a0bf41dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:01.596984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225be18-ebe0-49ab-bf8c-d687a0bf41dd None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:01.613205 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 231/474] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:01.614533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32068f7-76de-4929-bc97-0c4157a6e958 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:01.614828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32068f7-76de-4929-bc97-0c4157a6e958 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:01.615101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32068f7-76de-4929-bc97-0c4157a6e958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:01.637827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be97373-ceae-4825-8921-841effe86b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:01.638376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be97373-ceae-4825-8921-841effe86b6a None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:01.638747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be97373-ceae-4825-8921-841effe86b6a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:01.638849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e32068f7-76de-4929-bc97-0c4157a6e958 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:01.639025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be97373-ceae-4825-8921-841effe86b6a None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:01.639730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be97373-ceae-4825-8921-841effe86b6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:01.640102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e32068f7-76de-4929-bc97-0c4157a6e958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:01.641324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be97373-ceae-4825-8921-841effe86b6a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:01.669647 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 244/475] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Oct 1 20:33:01 2022] POST /identity/v3/users => generated 248 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:33:01.670630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be97373-ceae-4825-8921-841effe86b6a None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:01.714382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:01.746604 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 232/476] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:01.806525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be97373-ceae-4825-8921-841effe86b6a None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:01.847397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2062fef-7545-4fd9-bea4-655f9ebae024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:01.848371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2062fef-7545-4fd9-bea4-655f9ebae024 None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:01.848851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2062fef-7545-4fd9-bea4-655f9ebae024 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:01.849371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2062fef-7545-4fd9-bea4-655f9ebae024 None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:01.850305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2062fef-7545-4fd9-bea4-655f9ebae024 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:01.852147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2062fef-7545-4fd9-bea4-655f9ebae024 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:01.880586 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 245/477] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Oct 1 20:33:01 2022] POST /identity/v3/users => generated 249 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:33:02.162308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32068f7-76de-4929-bc97-0c4157a6e958 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:02.212753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:02.278427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32068f7-76de-4929-bc97-0c4157a6e958 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:02.318810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32990fd-64f4-47f0-b5e9-5a2286f3027c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:02.319345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32990fd-64f4-47f0-b5e9-5a2286f3027c None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:02.319628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32990fd-64f4-47f0-b5e9-5a2286f3027c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:02.319961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32990fd-64f4-47f0-b5e9-5a2286f3027c None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:02.320635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32990fd-64f4-47f0-b5e9-5a2286f3027c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:02.322258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32990fd-64f4-47f0-b5e9-5a2286f3027c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:02.335124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.sql.core [None req-a32990fd-64f4-47f0-b5e9-5a2286f3027c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 01 20:33:02.335124 ncdv-master-1 devstack@keystone.service[99140]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 01 20:33:02.335124 ncdv-master-1 devstack@keystone.service[99140]: [parameters: {'id': 'e11c96f49cd148aa9fd001e208c29268', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 01 20:33:02.335124 ncdv-master-1 devstack@keystone.service[99140]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99140) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 01 20:33:02.351123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-a32990fd-64f4-47f0-b5e9-5a2286f3027c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 01 20:33:02.351123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:02.351123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 01 20:33:02.351123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 01 20:33:02.351123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 01 20:33:02.351123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 01 20:33:02.351123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 01 20:33:02.351123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application result = self._query(query) Oct 01 20:33:02.351123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 01 20:33:02.351123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application conn.query(q) Oct 01 20:33:02.351123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 01 20:33:02.351123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 01 20:33:02.351123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 01 20:33:02.351123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application result.read() Oct 01 20:33:02.351123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 01 20:33:02.352001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 01 20:33:02.352001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 01 20:33:02.352001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 01 20:33:02.352001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 01 20:33:02.352001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 01 20:33:02.352001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 01 20:33:02.352001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 01 20:33:02.352001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 01 20:33:02.352001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:02.352001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 01 20:33:02.352001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:02.352001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:02.352001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 01 20:33:02.352001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 01 20:33:02.352001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 01 20:33:02.352001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return ref.to_dict() Oct 01 20:33:02.352001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 01 20:33:02.352001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application next(self.gen) Oct 01 20:33:02.352901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 01 20:33:02.352901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application yield resource Oct 01 20:33:02.352901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 01 20:33:02.352901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application next(self.gen) Oct 01 20:33:02.352901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 01 20:33:02.352901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.session.rollback() Oct 01 20:33:02.352901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 01 20:33:02.352901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.force_reraise() Oct 01 20:33:02.352901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 01 20:33:02.352901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise self.value Oct 01 20:33:02.352901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 01 20:33:02.352901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 01 20:33:02.352901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 01 20:33:02.352901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application session.commit() Oct 01 20:33:02.352901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 01 20:33:02.352901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 01 20:33:02.352901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 01 20:33:02.353733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._prepare_impl() Oct 01 20:33:02.353733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 01 20:33:02.353733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.session.flush() Oct 01 20:33:02.353733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 01 20:33:02.353733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._flush(objects) Oct 01 20:33:02.353733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 01 20:33:02.353733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 01 20:33:02.353733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 01 20:33:02.353733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application compat.raise_( Oct 01 20:33:02.353733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 01 20:33:02.353733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception Oct 01 20:33:02.353733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 01 20:33:02.353733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application flush_context.execute() Oct 01 20:33:02.353733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 01 20:33:02.353733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rec.execute(self) Oct 01 20:33:02.353733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 01 20:33:02.353733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 01 20:33:02.355591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 01 20:33:02.355591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 01 20:33:02.355591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 01 20:33:02.355591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 01 20:33:02.355591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 01 20:33:02.355591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 01 20:33:02.355591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 01 20:33:02.355591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 01 20:33:02.355591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 01 20:33:02.355591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 01 20:33:02.355591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 01 20:33:02.355591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 01 20:33:02.355591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 01 20:33:02.355591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 01 20:33:02.355591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 01 20:33:02.355591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception Oct 01 20:33:02.359084 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 01 20:33:02.359084 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 01 20:33:02.359084 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 01 20:33:02.359084 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 01 20:33:02.359084 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 01 20:33:02.359084 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application result = self._query(query) Oct 01 20:33:02.359084 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 01 20:33:02.359084 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application conn.query(q) Oct 01 20:33:02.359084 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 01 20:33:02.359084 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 01 20:33:02.359084 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 01 20:33:02.359084 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application result.read() Oct 01 20:33:02.359084 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 01 20:33:02.359084 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 01 20:33:02.359084 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 01 20:33:02.359084 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 01 20:33:02.361606 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 01 20:33:02.361606 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 01 20:33:02.361606 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 01 20:33:02.361606 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 01 20:33:02.361606 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 01 20:33:02.361606 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 01 20:33:02.361606 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [parameters: {'id': 'e11c96f49cd148aa9fd001e208c29268', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 01 20:33:02.361606 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 01 20:33:02.361606 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:02.361606 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:33:02.361606 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:02.361606 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:02.361606 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:02.361606 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:02.361606 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:02.361606 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:02.363637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:02.363637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:02.363637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:02.363637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:02.363637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:02.363637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:02.363637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 01 20:33:02.363637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 01 20:33:02.363637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:02.363637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:02.363637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 01 20:33:02.363637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 01 20:33:02.363637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 01 20:33:02.363637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 01 20:33:02.363637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 01 20:33:02.363637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:02.365684 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 233/478] 10.222.0.17 () {60 vars in 1131 bytes} [Sat Oct 1 20:33:02 2022] POST /identity/v3/roles => generated 144 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:02.365827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2062fef-7545-4fd9-bea4-655f9ebae024 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:02.469405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:02.493235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2062fef-7545-4fd9-bea4-655f9ebae024 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:02.538059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a469d17-bf22-49ec-8405-60285221bcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:02.539075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a469d17-bf22-49ec-8405-60285221bcef None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:02.539595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a469d17-bf22-49ec-8405-60285221bcef None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:02.540070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a469d17-bf22-49ec-8405-60285221bcef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:02.556114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a469d17-bf22-49ec-8405-60285221bcef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:02.562512 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-6a469d17-bf22-49ec-8405-60285221bcef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:33:02.562512 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:02.562512 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:02.562512 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:02.562512 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:02.562512 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:02.562512 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:02.562512 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:02.562512 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:02.562512 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:02.562512 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:02.562512 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:02.562512 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:02.562512 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:02.562512 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:02.562512 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:02.563440 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:02.563440 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:02.563440 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:02.563440 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:02.563440 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:02.563440 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:02.563440 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:02.563440 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:02.563440 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:02.563440 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:02.563440 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:02.563440 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:02.563440 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:02.563440 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:02.563440 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:02.563440 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:02.564342 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:02.564342 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:02.564342 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:02.564342 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:02.564342 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:02.564342 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:02.564342 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:02.564342 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:02.564342 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:02.564342 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:02.564342 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 01 20:33:02.564342 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:02.570656 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 246/479] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:33:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:02.573545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32990fd-64f4-47f0-b5e9-5a2286f3027c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:02.717864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32990fd-64f4-47f0-b5e9-5a2286f3027c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:02.739180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:02.763537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b9f4f-d455-443c-bfae-b0c28eb330f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:02.764154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b9f4f-d455-443c-bfae-b0c28eb330f6 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:02.764434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b9f4f-d455-443c-bfae-b0c28eb330f6 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:02.764708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b9f4f-d455-443c-bfae-b0c28eb330f6 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:02.765376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b9f4f-d455-443c-bfae-b0c28eb330f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:02.765679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b9f4f-d455-443c-bfae-b0c28eb330f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:02.767080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b9f4f-d455-443c-bfae-b0c28eb330f6 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:02.775399 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 234/480] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:04.021175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe60e7-de4c-4e5c-8aba-939087218819 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:04.021697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe60e7-de4c-4e5c-8aba-939087218819 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:04.021774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe60e7-de4c-4e5c-8aba-939087218819 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:04.023358 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 247/481] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:04.033727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee544f6d-daf3-4bcf-9745-584e6a20c263 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:04.034171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee544f6d-daf3-4bcf-9745-584e6a20c263 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:04.034246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee544f6d-daf3-4bcf-9745-584e6a20c263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:04.060228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ee544f6d-daf3-4bcf-9745-584e6a20c263 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:04.061419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ee544f6d-daf3-4bcf-9745-584e6a20c263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:04.168033 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 235/482] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:04.556092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe60e7-de4c-4e5c-8aba-939087218819 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:04.697930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe60e7-de4c-4e5c-8aba-939087218819 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:04.741294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49389ae0-554e-4ac9-85f7-e93598dc7a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:04.741927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49389ae0-554e-4ac9-85f7-e93598dc7a58 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:04.742210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49389ae0-554e-4ac9-85f7-e93598dc7a58 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:04.742483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49389ae0-554e-4ac9-85f7-e93598dc7a58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:04.761480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49389ae0-554e-4ac9-85f7-e93598dc7a58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:04.766259 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-49389ae0-554e-4ac9-85f7-e93598dc7a58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:04.766259 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:04.766259 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:04.766259 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:04.766259 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:04.766259 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:04.766259 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:04.766259 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:04.766259 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:04.766259 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:04.766259 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:04.766259 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:04.766259 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:04.766259 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:04.766259 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:04.766259 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:04.767769 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:04.767769 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:04.767769 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:04.767769 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:04.767769 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:04.767769 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:04.767769 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:04.767769 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:04.767769 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:04.767769 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:04.767769 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:04.767769 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:04.767769 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:04.767769 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:04.767769 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:04.767769 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:04.769005 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:04.769005 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:04.769005 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:04.769005 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:04.769005 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:04.769005 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:04.769005 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:04.769005 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:04.769005 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:04.769005 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:04.769005 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:04.769005 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:04.769005 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 248/483] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:04.775774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee544f6d-daf3-4bcf-9745-584e6a20c263 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:04.779246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0105750-2c6f-46a4-87aa-41c806f7c287 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:04.779532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0105750-2c6f-46a4-87aa-41c806f7c287 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:04.779808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0105750-2c6f-46a4-87aa-41c806f7c287 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:04.781381 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 249/484] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:04.786442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6b7cb-16b1-42b8-b5d1-ab9d1a264d4a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:04.786754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6b7cb-16b1-42b8-b5d1-ab9d1a264d4a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:04.787025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6b7cb-16b1-42b8-b5d1-ab9d1a264d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:04.805753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:04.815076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8ea6b7cb-16b1-42b8-b5d1-ab9d1a264d4a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:04.815076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8ea6b7cb-16b1-42b8-b5d1-ab9d1a264d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:04.901872 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 250/485] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:33:04.906921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39aed4a-a707-4af8-bef3-971fe1122f5d None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:04.907212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39aed4a-a707-4af8-bef3-971fe1122f5d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:04.907482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39aed4a-a707-4af8-bef3-971fe1122f5d None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:04.910435 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 251/486] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:04.919952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03801ab1-4c5d-4c8b-afd1-2d0d726c76b7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:04.920228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03801ab1-4c5d-4c8b-afd1-2d0d726c76b7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:04.920500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03801ab1-4c5d-4c8b-afd1-2d0d726c76b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:04.949392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee544f6d-daf3-4bcf-9745-584e6a20c263 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:04.952292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-03801ab1-4c5d-4c8b-afd1-2d0d726c76b7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:04.952407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:04.953474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-03801ab1-4c5d-4c8b-afd1-2d0d726c76b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:05.029876 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 252/487] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 01 20:33:05.051203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f46ace6-14e0-44cd-b9f5-f03a53bd1fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:05.051829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46ace6-14e0-44cd-b9f5-f03a53bd1fc8 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:05.052109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46ace6-14e0-44cd-b9f5-f03a53bd1fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:05.052381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46ace6-14e0-44cd-b9f5-f03a53bd1fc8 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:05.053056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f46ace6-14e0-44cd-b9f5-f03a53bd1fc8 None admin] RBAC: Adding query filter params (name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:05.053356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f46ace6-14e0-44cd-b9f5-f03a53bd1fc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:05.054777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f46ace6-14e0-44cd-b9f5-f03a53bd1fc8 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:05.069051 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 236/488] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:33:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:05.075246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03801ab1-4c5d-4c8b-afd1-2d0d726c76b7 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:05.174406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:05.180052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d484a2-9f13-4d7c-b4f9-2a02fb416d50 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:05.180334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d484a2-9f13-4d7c-b4f9-2a02fb416d50 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:05.180610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d484a2-9f13-4d7c-b4f9-2a02fb416d50 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:05.182007 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 237/489] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:05.191799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fa0ce-648c-44e7-9161-8163cd9c5cfe None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:05.192076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fa0ce-648c-44e7-9161-8163cd9c5cfe None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:05.192349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fa0ce-648c-44e7-9161-8163cd9c5cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:05.211343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03801ab1-4c5d-4c8b-afd1-2d0d726c76b7 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:05.231471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3a4fa0ce-648c-44e7-9161-8163cd9c5cfe None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:05.232659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3a4fa0ce-648c-44e7-9161-8163cd9c5cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:05.247115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2ce54-e280-4569-82c5-6fcf944fe21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:05.248016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2ce54-e280-4569-82c5-6fcf944fe21f None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:05.248487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2ce54-e280-4569-82c5-6fcf944fe21f None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:05.248969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2ce54-e280-4569-82c5-6fcf944fe21f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:05.260120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2ce54-e280-4569-82c5-6fcf944fe21f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:05.261794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2ce54-e280-4569-82c5-6fcf944fe21f None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:05.277225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-c9a2ce54-e280-4569-82c5-6fcf944fe21f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:05.277225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:05.277225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:05.277225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:05.277225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:05.277225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:05.277225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:05.277225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:05.277225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:05.277225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:05.277225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:05.277225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:05.277225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:05.277225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:05.277225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:05.277225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:05.278133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:05.278133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:05.278133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:05.278133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:05.278133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:05.278133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:05.278133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:05.278133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:05.278133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:05.278133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:05.278133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:05.278133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:05.278133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:05.278133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:05.278133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:05.278133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:05.278133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:05.279145 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:05.279145 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:05.279145 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:05.280817 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 253/490] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:05.290580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2ce54-e280-4569-82c5-6fcf944fe21f None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:05.382831 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 238/491] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:05.383633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fa0ce-648c-44e7-9161-8163cd9c5cfe None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:05.406282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2ce54-e280-4569-82c5-6fcf944fe21f None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:05.424460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:05.443514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af4422-fe34-4b91-a7e6-5fa32105743d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:05.444210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af4422-fe34-4b91-a7e6-5fa32105743d None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:05.444560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af4422-fe34-4b91-a7e6-5fa32105743d None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:05.444910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af4422-fe34-4b91-a7e6-5fa32105743d None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:05.445625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af4422-fe34-4b91-a7e6-5fa32105743d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:05.446018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af4422-fe34-4b91-a7e6-5fa32105743d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:05.447791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af4422-fe34-4b91-a7e6-5fa32105743d None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:05.460647 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 254/492] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:05.461546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af4422-fe34-4b91-a7e6-5fa32105743d None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:05.502249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:05.510549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fa0ce-648c-44e7-9161-8163cd9c5cfe None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:05.559833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0c389-3ce2-4b00-bee2-633201984ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:05.561203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0c389-3ce2-4b00-bee2-633201984ab0 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:05.561661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0c389-3ce2-4b00-bee2-633201984ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:05.562639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0c389-3ce2-4b00-bee2-633201984ab0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:05.575332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0c389-3ce2-4b00-bee2-633201984ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:05.581040 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-3bd0c389-3ce2-4b00-bee2-633201984ab0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:05.581040 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:05.581040 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:05.581040 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:05.581040 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:05.581040 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:05.581040 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:05.581040 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:05.581040 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:05.581040 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:05.581040 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:05.581040 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:05.581040 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:05.581040 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:05.581040 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:05.581040 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:05.581910 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:05.581910 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:05.581910 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:05.581910 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:05.581910 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:05.581910 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:05.581910 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:05.581910 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:05.581910 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:05.581910 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:05.581910 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:05.581910 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:05.581910 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:05.581910 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:05.581910 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:05.581910 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:05.582720 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:05.582720 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:05.582720 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:05.582720 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:05.582720 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:05.582720 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:05.582720 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:05.582720 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:05.582720 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:05.582720 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:05.582720 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:05.582720 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:05.585220 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 239/493] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:05.586190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0c389-3ce2-4b00-bee2-633201984ab0 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:05.588220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af4422-fe34-4b91-a7e6-5fa32105743d None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:05.627368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a73bae-69c7-46cf-b859-ec8a4476e263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:05.628300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a73bae-69c7-46cf-b859-ec8a4476e263 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:05.628438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a73bae-69c7-46cf-b859-ec8a4476e263 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:05.628708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a73bae-69c7-46cf-b859-ec8a4476e263 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:05.640042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a73bae-69c7-46cf-b859-ec8a4476e263 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:05.645048 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-a9a73bae-69c7-46cf-b859-ec8a4476e263 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:05.645048 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:05.645048 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:05.645048 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:05.645048 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:05.645048 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:05.645048 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:05.645048 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:05.645048 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:05.645048 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:05.645048 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:05.645048 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:05.645048 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:05.645048 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:05.645048 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:05.645048 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:05.645953 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:05.645953 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:05.645953 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:05.645953 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:05.645953 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:05.645953 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:05.645953 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:05.645953 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:05.645953 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:05.645953 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:05.645953 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:05.645953 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:05.645953 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:05.645953 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:05.645953 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:05.645953 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:05.646766 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:05.646766 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:05.646766 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:05.646766 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:05.646766 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:05.646766 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:05.646766 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:05.646766 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:05.646766 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:05.646766 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:05.646766 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:05.646766 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:05.650780 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 255/494] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:05.651721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a73bae-69c7-46cf-b859-ec8a4476e263 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:05.694540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0c389-3ce2-4b00-bee2-633201984ab0 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:05.697245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:05.731089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11853fa-c7f3-41e6-b7ee-431cf40ecce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:05.732066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11853fa-c7f3-41e6-b7ee-431cf40ecce3 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:05.732562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11853fa-c7f3-41e6-b7ee-431cf40ecce3 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:05.733038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11853fa-c7f3-41e6-b7ee-431cf40ecce3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:05.740581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11853fa-c7f3-41e6-b7ee-431cf40ecce3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:05.742024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11853fa-c7f3-41e6-b7ee-431cf40ecce3 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:05.755528 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-d11853fa-c7f3-41e6-b7ee-431cf40ecce3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 01 20:33:05.755528 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:05.755528 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:05.755528 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:05.755528 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:05.755528 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:05.755528 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:05.755528 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:05.755528 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:05.755528 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:05.755528 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:05.755528 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:05.755528 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:05.755528 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:05.755528 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:05.755528 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:05.756448 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:05.756448 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:05.756448 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:05.756448 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:05.756448 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:05.756448 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:05.756448 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:05.756448 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:05.756448 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:05.756448 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:05.756448 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:05.756448 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:05.756448 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:05.756448 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:05.756448 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:05.756448 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:05.756448 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:05.756448 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:05.757344 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:05.757344 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:05.757344 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:05.757344 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:05.757344 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:05.757344 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:05.757344 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:05.757344 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:05.757344 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 01 20:33:05.757344 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:05.757344 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 240/495] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Oct 1 20:33:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:05.757344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11853fa-c7f3-41e6-b7ee-431cf40ecce3 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:05.772012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a73bae-69c7-46cf-b859-ec8a4476e263 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:05.774776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:05.810483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e4829-fcca-477e-81d2-7d268c5a3c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:05.811499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e4829-fcca-477e-81d2-7d268c5a3c53 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:05.811786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e4829-fcca-477e-81d2-7d268c5a3c53 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:05.812064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e4829-fcca-477e-81d2-7d268c5a3c53 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:05.812768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e4829-fcca-477e-81d2-7d268c5a3c53 None admin] RBAC: Adding query filter params (name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:05.813076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e4829-fcca-477e-81d2-7d268c5a3c53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:05.815397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e4829-fcca-477e-81d2-7d268c5a3c53 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:05.824580 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 256/496] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:33:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:05.827656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e4829-fcca-477e-81d2-7d268c5a3c53 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:05.862543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11853fa-c7f3-41e6-b7ee-431cf40ecce3 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:05.898840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18832be3-1f13-435c-9e30-3be33210067e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:05.899364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18832be3-1f13-435c-9e30-3be33210067e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:05.899640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18832be3-1f13-435c-9e30-3be33210067e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:05.899908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18832be3-1f13-435c-9e30-3be33210067e None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:05.900565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18832be3-1f13-435c-9e30-3be33210067e None admin] RBAC: Adding query filter params (name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:05.900863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18832be3-1f13-435c-9e30-3be33210067e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:05.902245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18832be3-1f13-435c-9e30-3be33210067e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:05.915358 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 241/497] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:33:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:05.915358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18832be3-1f13-435c-9e30-3be33210067e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:05.942433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:05.945446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e4829-fcca-477e-81d2-7d268c5a3c53 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:05.987830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76165412-dc5b-49bc-8095-1775321507e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:05.988454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76165412-dc5b-49bc-8095-1775321507e8 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:05.988734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76165412-dc5b-49bc-8095-1775321507e8 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:05.989004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76165412-dc5b-49bc-8095-1775321507e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:05.999036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76165412-dc5b-49bc-8095-1775321507e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:05.999542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:06.004833 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-76165412-dc5b-49bc-8095-1775321507e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:06.004833 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:06.004833 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:06.004833 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:06.004833 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:06.004833 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:06.004833 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:06.004833 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:06.004833 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:06.004833 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:06.004833 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:06.004833 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:06.004833 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:06.004833 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:06.004833 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:06.004833 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:06.005948 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:06.005948 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:06.005948 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:06.005948 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:06.005948 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:06.005948 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:06.005948 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:06.005948 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:06.005948 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:06.005948 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:06.005948 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:06.005948 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:06.005948 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:06.005948 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:06.005948 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:06.005948 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:06.007053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:06.007053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:06.007053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:06.007053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:06.007053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:06.007053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:06.007053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:06.007053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:06.007053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:06.007053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:06.007053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:06.007053 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:06.007053 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 257/498] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:06.007053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76165412-dc5b-49bc-8095-1775321507e8 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:06.029158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18832be3-1f13-435c-9e30-3be33210067e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:06.064963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55a909-35f0-4f01-820f-0b04088f1f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:06.065571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55a909-35f0-4f01-820f-0b04088f1f2d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:06.065849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55a909-35f0-4f01-820f-0b04088f1f2d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:06.066140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55a909-35f0-4f01-820f-0b04088f1f2d None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:06.066874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55a909-35f0-4f01-820f-0b04088f1f2d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:06.067182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55a909-35f0-4f01-820f-0b04088f1f2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:06.068609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55a909-35f0-4f01-820f-0b04088f1f2d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:06.105504 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 242/499] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Oct 1 20:33:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:06.106339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55a909-35f0-4f01-820f-0b04088f1f2d None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:06.113975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76165412-dc5b-49bc-8095-1775321507e8 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:06.147350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e499353-ac9a-48aa-bf01-4c7db1dfa57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:06.147965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e499353-ac9a-48aa-bf01-4c7db1dfa57d None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:06.148250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e499353-ac9a-48aa-bf01-4c7db1dfa57d None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:06.148516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e499353-ac9a-48aa-bf01-4c7db1dfa57d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:06.159244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e499353-ac9a-48aa-bf01-4c7db1dfa57d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:06.160621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e499353-ac9a-48aa-bf01-4c7db1dfa57d None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:06.174135 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-3e499353-ac9a-48aa-bf01-4c7db1dfa57d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:06.174135 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:06.174135 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:06.174135 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:06.174135 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:06.174135 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:06.174135 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:06.174135 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:06.174135 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:06.174135 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:06.174135 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:06.174135 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:06.174135 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:06.174135 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:06.174135 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:06.174135 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:06.175148 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:06.175148 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:06.175148 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:06.175148 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:06.175148 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:06.175148 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:06.175148 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:06.175148 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:06.175148 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:06.175148 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:06.175148 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:06.175148 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:06.175148 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:06.175148 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:06.175148 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:06.175148 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:06.175148 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:06.176007 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:06.176007 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:06.176007 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:06.176007 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 258/500] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:06.176561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6393001-8e5b-45c3-874f-82f0a465b0e2 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:06.176561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6393001-8e5b-45c3-874f-82f0a465b0e2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:06.176835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6393001-8e5b-45c3-874f-82f0a465b0e2 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:06.178545 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 259/501] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:06.179018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6393001-8e5b-45c3-874f-82f0a465b0e2 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:06.195981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:06.206579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55a909-35f0-4f01-820f-0b04088f1f2d None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:06.240367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c5b86-b0de-4703-b54c-c47406ef04b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:06.240991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c5b86-b0de-4703-b54c-c47406ef04b2 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:06.241270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c5b86-b0de-4703-b54c-c47406ef04b2 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:06.241542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c5b86-b0de-4703-b54c-c47406ef04b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:06.243420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:06.252697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c5b86-b0de-4703-b54c-c47406ef04b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:06.254144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c5b86-b0de-4703-b54c-c47406ef04b2 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:06.268826 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-649c5b86-b0de-4703-b54c-c47406ef04b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:06.268826 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:06.268826 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:06.268826 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:06.268826 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:06.268826 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:06.268826 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:06.268826 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:06.268826 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:06.268826 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:06.268826 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:06.268826 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:06.268826 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:06.268826 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:06.268826 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:06.268826 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:06.269898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:06.269898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:06.269898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:06.269898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:06.269898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:06.269898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:06.269898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:06.269898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:06.269898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:06.269898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:06.269898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:06.269898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:06.269898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:06.269898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:06.269898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:06.269898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:06.269898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:06.270764 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:06.270764 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:06.270764 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:06.270764 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 243/502] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:06.270764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c5b86-b0de-4703-b54c-c47406ef04b2 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:06.287822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6393001-8e5b-45c3-874f-82f0a465b0e2 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:06.322449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cde623-cea0-4511-a8ae-cbbfae3fc734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:06.323081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cde623-cea0-4511-a8ae-cbbfae3fc734 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:06.323359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cde623-cea0-4511-a8ae-cbbfae3fc734 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:06.323631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cde623-cea0-4511-a8ae-cbbfae3fc734 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:06.324295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cde623-cea0-4511-a8ae-cbbfae3fc734 None admin] RBAC: Adding query filter params (name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:06.324599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cde623-cea0-4511-a8ae-cbbfae3fc734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:06.325953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cde623-cea0-4511-a8ae-cbbfae3fc734 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:06.333735 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 260/503] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:33:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:06.334530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cde623-cea0-4511-a8ae-cbbfae3fc734 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:06.372464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c5b86-b0de-4703-b54c-c47406ef04b2 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:06.405623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebc11b-2519-447a-8e62-6b5b0d90bb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:06.406226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebc11b-2519-447a-8e62-6b5b0d90bb42 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:06.406538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebc11b-2519-447a-8e62-6b5b0d90bb42 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:06.406815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebc11b-2519-447a-8e62-6b5b0d90bb42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:06.407607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:06.433776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cde623-cea0-4511-a8ae-cbbfae3fc734 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:06.461232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:06.467910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2afeb-ffae-4ca5-be1e-0a4a9a38805c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:06.468510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2afeb-ffae-4ca5-be1e-0a4a9a38805c None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:06.468792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2afeb-ffae-4ca5-be1e-0a4a9a38805c None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:06.469066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2afeb-ffae-4ca5-be1e-0a4a9a38805c None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:06.469707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2afeb-ffae-4ca5-be1e-0a4a9a38805c None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:06.470013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2afeb-ffae-4ca5-be1e-0a4a9a38805c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:06.471433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2afeb-ffae-4ca5-be1e-0a4a9a38805c None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:06.473621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aebc11b-2519-447a-8e62-6b5b0d90bb42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:06.475082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aebc11b-2519-447a-8e62-6b5b0d90bb42 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:06.487397 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 261/504] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:06.487397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2a558-43a4-4180-8ac2-07ba9c206846 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:06.487397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2a558-43a4-4180-8ac2-07ba9c206846 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:06.487397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2a558-43a4-4180-8ac2-07ba9c206846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:06.502744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-eef2a558-43a4-4180-8ac2-07ba9c206846 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:06.503936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-eef2a558-43a4-4180-8ac2-07ba9c206846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:06.568831 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 262/505] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:06.569949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2a558-43a4-4180-8ac2-07ba9c206846 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:06.574328 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 244/506] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:33:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:33:06.575374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebc11b-2519-447a-8e62-6b5b0d90bb42 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:06.635781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:06.667349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:06.677482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2a558-43a4-4180-8ac2-07ba9c206846 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:06.683255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebc11b-2519-447a-8e62-6b5b0d90bb42 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:06.711435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09bce72-5f77-4999-a0ba-a06d985a3321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:06.712039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bce72-5f77-4999-a0ba-a06d985a3321 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:06.712346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bce72-5f77-4999-a0ba-a06d985a3321 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:06.712618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bce72-5f77-4999-a0ba-a06d985a3321 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:06.713258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09bce72-5f77-4999-a0ba-a06d985a3321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:06.713560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09bce72-5f77-4999-a0ba-a06d985a3321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:06.714951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09bce72-5f77-4999-a0ba-a06d985a3321 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:06.719708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ff3dd-7d69-47a2-8d67-698decc8c3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:06.721043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ff3dd-7d69-47a2-8d67-698decc8c3f6 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:06.721591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ff3dd-7d69-47a2-8d67-698decc8c3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:06.721867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ff3dd-7d69-47a2-8d67-698decc8c3f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:06.726646 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 263/507] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:06.727602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09bce72-5f77-4999-a0ba-a06d985a3321 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:06.735309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ff3dd-7d69-47a2-8d67-698decc8c3f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:06.736842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ff3dd-7d69-47a2-8d67-698decc8c3f6 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:06.751366 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-639ff3dd-7d69-47a2-8d67-698decc8c3f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:06.751366 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:06.751366 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:06.751366 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:06.751366 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:06.751366 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:06.751366 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:06.751366 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:06.751366 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:06.751366 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:06.751366 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:06.751366 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:06.751366 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:06.751366 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:06.751366 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:06.751366 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:06.752294 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:06.752294 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:06.752294 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:06.752294 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:06.752294 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:06.752294 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:06.752294 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:06.752294 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:06.752294 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:06.752294 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:06.752294 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:06.752294 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:06.752294 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:06.752294 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:06.752294 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:06.752294 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:06.752294 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:06.753135 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:06.753135 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:06.753135 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:06.753135 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 245/508] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:06.753760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ff3dd-7d69-47a2-8d67-698decc8c3f6 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:06.840980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09bce72-5f77-4999-a0ba-a06d985a3321 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:06.857286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ff3dd-7d69-47a2-8d67-698decc8c3f6 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:06.865598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:06.877068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba2eb70-94c0-4323-a816-96e5fac12417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:06.877781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2eb70-94c0-4323-a816-96e5fac12417 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:06.877963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2eb70-94c0-4323-a816-96e5fac12417 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:06.878234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2eb70-94c0-4323-a816-96e5fac12417 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:06.885839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba2eb70-94c0-4323-a816-96e5fac12417 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:06.887308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba2eb70-94c0-4323-a816-96e5fac12417 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:06.898482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150cd014-a4ba-436d-96e0-e55a184c55d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:06.899189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150cd014-a4ba-436d-96e0-e55a184c55d1 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:06.899466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150cd014-a4ba-436d-96e0-e55a184c55d1 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:06.899738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150cd014-a4ba-436d-96e0-e55a184c55d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:06.900492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-7ba2eb70-94c0-4323-a816-96e5fac12417 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 01 20:33:06.900492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:06.900492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:06.900492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:06.900492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:06.900492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:06.900492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:06.900492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:06.900492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:06.900492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:06.900492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:06.900492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:06.900492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:06.900492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:06.900492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:06.900492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:06.901373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:06.901373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:06.901373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:06.901373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:06.901373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:06.901373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:06.901373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:06.901373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:06.901373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:06.901373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:06.901373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:06.901373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:06.901373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:06.901373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:06.901373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:06.901373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:06.901373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:06.901373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:06.902286 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:06.902286 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:06.902286 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:06.902286 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:06.902286 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:06.902286 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:06.902286 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:06.902286 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:06.902286 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 01 20:33:06.902286 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:06.902286 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 264/509] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:33:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:06.902929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba2eb70-94c0-4323-a816-96e5fac12417 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:06.911327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:06.915189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150cd014-a4ba-436d-96e0-e55a184c55d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:06.917340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150cd014-a4ba-436d-96e0-e55a184c55d1 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:06.931872 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-150cd014-a4ba-436d-96e0-e55a184c55d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:06.931872 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:06.931872 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:06.931872 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:06.931872 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:06.931872 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:06.931872 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:06.931872 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:06.931872 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:06.931872 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:06.931872 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:06.931872 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:06.931872 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:06.931872 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:06.931872 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:06.931872 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:06.932807 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:06.932807 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:06.932807 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:06.932807 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:06.932807 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:06.932807 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:06.932807 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:06.932807 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:06.932807 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:06.932807 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:06.932807 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:06.932807 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:06.932807 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:06.932807 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:06.932807 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:06.932807 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:06.932807 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:06.933644 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:06.933644 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:06.933644 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:06.933644 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 246/510] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:06.933911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150cd014-a4ba-436d-96e0-e55a184c55d1 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:07.007358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba2eb70-94c0-4323-a816-96e5fac12417 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:07.037177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150cd014-a4ba-436d-96e0-e55a184c55d1 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:07.041420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13994607-7f1e-4ede-958c-cf101ed75405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:07.042048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13994607-7f1e-4ede-958c-cf101ed75405 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:07.042361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13994607-7f1e-4ede-958c-cf101ed75405 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:07.042652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13994607-7f1e-4ede-958c-cf101ed75405 None admin] PATH_INFO: `/v3/users/glance` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:07.050414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13994607-7f1e-4ede-958c-cf101ed75405 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:07.051881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13994607-7f1e-4ede-958c-cf101ed75405 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:07.062548 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-13994607-7f1e-4ede-958c-cf101ed75405 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 01 20:33:07.062548 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:07.062548 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:07.062548 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:07.062548 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:07.062548 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:07.062548 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:07.062548 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:07.062548 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:07.062548 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:07.062548 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:07.062548 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:07.062548 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:07.062548 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:07.062548 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:07.062548 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:07.063603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:07.063603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:07.063603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:07.063603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:07.063603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:07.063603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:07.063603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:07.063603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:07.063603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:07.063603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:07.063603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:07.063603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:07.063603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:07.063603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:07.063603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:07.063603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:07.063603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:07.063603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:07.064912 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:07.064912 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:07.064912 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:07.064912 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:07.064912 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:07.064912 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:07.064912 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:07.064912 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:07.064912 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 01 20:33:07.064912 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:07.064912 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 265/511] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:33:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:07.067296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13994607-7f1e-4ede-958c-cf101ed75405 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:07.076628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d93317-e2bf-45fc-9d8c-cfd246a4f98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:07.077653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d93317-e2bf-45fc-9d8c-cfd246a4f98f None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:07.077934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d93317-e2bf-45fc-9d8c-cfd246a4f98f None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:07.078209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d93317-e2bf-45fc-9d8c-cfd246a4f98f None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:07.078887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d93317-e2bf-45fc-9d8c-cfd246a4f98f None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:07.079189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d93317-e2bf-45fc-9d8c-cfd246a4f98f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:07.081089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d93317-e2bf-45fc-9d8c-cfd246a4f98f None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:07.092578 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 247/512] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:07.093363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d93317-e2bf-45fc-9d8c-cfd246a4f98f None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:07.115219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:07.145965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:07.172390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13994607-7f1e-4ede-958c-cf101ed75405 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:07.195924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d93317-e2bf-45fc-9d8c-cfd246a4f98f None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:07.205988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e957413-9299-451c-bc33-3b0fb0779927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:07.206655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e957413-9299-451c-bc33-3b0fb0779927 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:07.206932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e957413-9299-451c-bc33-3b0fb0779927 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:07.207203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e957413-9299-451c-bc33-3b0fb0779927 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:07.217364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e957413-9299-451c-bc33-3b0fb0779927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:07.218823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e957413-9299-451c-bc33-3b0fb0779927 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:07.229841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cdd906-3bb2-42a4-924e-5c4be91b22f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:07.230432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cdd906-3bb2-42a4-924e-5c4be91b22f9 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:07.230738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cdd906-3bb2-42a4-924e-5c4be91b22f9 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:07.231008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cdd906-3bb2-42a4-924e-5c4be91b22f9 None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:07.231719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cdd906-3bb2-42a4-924e-5c4be91b22f9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:07.232023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cdd906-3bb2-42a4-924e-5c4be91b22f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:07.232489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-5e957413-9299-451c-bc33-3b0fb0779927 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:07.232489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:07.232489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:07.232489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:07.232489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:07.232489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:07.232489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:07.232489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:07.232489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:07.232489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:07.232489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:07.232489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:07.232489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:07.232489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:07.232489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:07.232489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:07.233518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:07.233518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:07.233518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:07.233518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:07.233518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:07.233518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:07.233518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:07.233518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:07.233518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:07.233518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:07.233518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:07.233518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:07.233518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:07.233518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:07.233518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:07.233518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:07.233518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:07.234373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:07.234373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:07.234373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:07.234565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cdd906-3bb2-42a4-924e-5c4be91b22f9 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:07.234644 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 266/513] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:07.235311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e957413-9299-451c-bc33-3b0fb0779927 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:07.257531 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 248/514] 10.222.0.17 () {56 vars in 1137 bytes} [Sat Oct 1 20:33:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:07.257531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cdd906-3bb2-42a4-924e-5c4be91b22f9 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:07.346539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:07.353539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e957413-9299-451c-bc33-3b0fb0779927 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:07.362706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cdd906-3bb2-42a4-924e-5c4be91b22f9 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:07.371269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:07.393266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc5d49-843b-4822-9cce-d166d935c11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:07.393872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc5d49-843b-4822-9cce-d166d935c11a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:07.394146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc5d49-843b-4822-9cce-d166d935c11a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:07.394414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc5d49-843b-4822-9cce-d166d935c11a None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:07.395082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc5d49-843b-4822-9cce-d166d935c11a None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:07.395381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc5d49-843b-4822-9cce-d166d935c11a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:07.396731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc5d49-843b-4822-9cce-d166d935c11a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:07.398952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f28347-70ce-47a0-adce-90262a689911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:07.399555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f28347-70ce-47a0-adce-90262a689911 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:07.399832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f28347-70ce-47a0-adce-90262a689911 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:07.400101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f28347-70ce-47a0-adce-90262a689911 None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:07.400816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f28347-70ce-47a0-adce-90262a689911 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:07.401119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f28347-70ce-47a0-adce-90262a689911 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:07.402546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f28347-70ce-47a0-adce-90262a689911 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:07.409624 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 267/515] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:07.410481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc5d49-843b-4822-9cce-d166d935c11a None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:07.425141 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 249/516] 10.222.0.17 () {56 vars in 1137 bytes} [Sat Oct 1 20:33:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:07.426048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f28347-70ce-47a0-adce-90262a689911 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:07.518799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc5d49-843b-4822-9cce-d166d935c11a None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:07.529749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f28347-70ce-47a0-adce-90262a689911 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:07.557173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5489627-9ee3-47aa-bdc9-3b2c328065a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:07.557802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5489627-9ee3-47aa-bdc9-3b2c328065a6 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:07.558083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5489627-9ee3-47aa-bdc9-3b2c328065a6 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:07.558355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5489627-9ee3-47aa-bdc9-3b2c328065a6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:07.566367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5489627-9ee3-47aa-bdc9-3b2c328065a6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:07.567236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5b7e6-a4b0-4e68-bb9b-81e36163fda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:07.567833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5489627-9ee3-47aa-bdc9-3b2c328065a6 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:07.567930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5b7e6-a4b0-4e68-bb9b-81e36163fda4 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:07.568109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5b7e6-a4b0-4e68-bb9b-81e36163fda4 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:07.568383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5b7e6-a4b0-4e68-bb9b-81e36163fda4 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:07.569021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5b7e6-a4b0-4e68-bb9b-81e36163fda4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:07.569324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5b7e6-a4b0-4e68-bb9b-81e36163fda4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:07.570742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5b7e6-a4b0-4e68-bb9b-81e36163fda4 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:07.579363 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-a5489627-9ee3-47aa-bdc9-3b2c328065a6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 01 20:33:07.579363 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:07.579363 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:07.579363 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:07.579363 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:07.579363 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:07.579363 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:07.579363 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:07.579363 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:07.579363 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:07.579363 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:07.579363 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:07.579363 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:07.579363 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:07.579363 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:07.579363 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:07.580914 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:07.580914 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:07.580914 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:07.580914 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:07.580914 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:07.580914 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:07.580914 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:07.580914 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:07.580914 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:07.580914 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:07.580914 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:07.580914 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:07.580914 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:07.580914 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:07.580914 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:07.580914 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:07.580914 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:07.580914 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:07.581813 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:07.581813 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:07.581813 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:07.581813 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:07.581813 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:07.581813 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:07.581813 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:07.581813 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:07.581813 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 01 20:33:07.581813 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:07.581813 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 268/517] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:07.582367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5489627-9ee3-47aa-bdc9-3b2c328065a6 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:07.582845 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 250/518] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:07.587405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5b7e6-a4b0-4e68-bb9b-81e36163fda4 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:07.603427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:07.615356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:07.683636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5489627-9ee3-47aa-bdc9-3b2c328065a6 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:07.686197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5b7e6-a4b0-4e68-bb9b-81e36163fda4 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:07.717202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c1e1d-86fc-411b-b076-ce8da8832802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:07.717797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c1e1d-86fc-411b-b076-ce8da8832802 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:07.718153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c1e1d-86fc-411b-b076-ce8da8832802 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:07.718350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c1e1d-86fc-411b-b076-ce8da8832802 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:07.719131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f8a5a-0375-4265-b66b-88891502e5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:07.719749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f8a5a-0375-4265-b66b-88891502e5f0 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:07.720028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f8a5a-0375-4265-b66b-88891502e5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:07.720900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f8a5a-0375-4265-b66b-88891502e5f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:07.727369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f8a5a-0375-4265-b66b-88891502e5f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:07.728803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f8a5a-0375-4265-b66b-88891502e5f0 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:07.740442 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-bc9f8a5a-0375-4265-b66b-88891502e5f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:07.740442 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:07.740442 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:07.740442 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:07.740442 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:07.740442 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:07.740442 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:07.740442 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:07.740442 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:07.740442 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:07.740442 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:07.740442 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:07.740442 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:07.740442 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:07.740442 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:07.741460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:07.741460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:07.741460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:07.741460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:07.741460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:07.741460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:07.741460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:07.741460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:07.741460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:07.741460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:07.741460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:07.741460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:07.741460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:07.741460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:07.741460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:07.741460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:07.741460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:07.742521 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:07.742521 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:07.742521 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:07.742521 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:07.742521 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:07.742521 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:07.742521 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:07.742521 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:07.742521 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 251/519] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:07.742521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f8a5a-0375-4265-b66b-88891502e5f0 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:07.789523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c1e1d-86fc-411b-b076-ce8da8832802 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:07.790929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c1e1d-86fc-411b-b076-ce8da8832802 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:07.807826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:07.825545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:07.840947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f8a5a-0375-4265-b66b-88891502e5f0 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:07.873490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7151754-73d5-4bb7-82aa-ae68001b50de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:07.874080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7151754-73d5-4bb7-82aa-ae68001b50de None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:07.874361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7151754-73d5-4bb7-82aa-ae68001b50de None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:07.874659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7151754-73d5-4bb7-82aa-ae68001b50de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:07.889331 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 269/520] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:33:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:33:07.890142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c1e1d-86fc-411b-b076-ce8da8832802 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:07.947053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7151754-73d5-4bb7-82aa-ae68001b50de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:07.948594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7151754-73d5-4bb7-82aa-ae68001b50de None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:07.985958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c1e1d-86fc-411b-b076-ce8da8832802 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:08.002405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:08.020479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0ab61-e719-4ff8-b7fa-e635eb96d8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:08.021070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0ab61-e719-4ff8-b7fa-e635eb96d8af None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:08.021346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0ab61-e719-4ff8-b7fa-e635eb96d8af None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:08.021615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0ab61-e719-4ff8-b7fa-e635eb96d8af None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:08.022332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0ab61-e719-4ff8-b7fa-e635eb96d8af None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:08.022668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0ab61-e719-4ff8-b7fa-e635eb96d8af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:08.024072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0ab61-e719-4ff8-b7fa-e635eb96d8af None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:08.030722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:08.046984 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 252/521] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:33:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:33:08.048145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7151754-73d5-4bb7-82aa-ae68001b50de None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:08.068509 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 270/522] 10.222.0.17 () {56 vars in 1139 bytes} [Sat Oct 1 20:33:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:08.069343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0ab61-e719-4ff8-b7fa-e635eb96d8af None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:08.148244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7151754-73d5-4bb7-82aa-ae68001b50de None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:08.172300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0ab61-e719-4ff8-b7fa-e635eb96d8af None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:08.181653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e01e5d-152a-45d5-8dd1-424fe5a8a1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:08.182247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e01e5d-152a-45d5-8dd1-424fe5a8a1ce None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:08.182559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e01e5d-152a-45d5-8dd1-424fe5a8a1ce None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:08.182831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e01e5d-152a-45d5-8dd1-424fe5a8a1ce None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:08.183456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e01e5d-152a-45d5-8dd1-424fe5a8a1ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:08.184914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e01e5d-152a-45d5-8dd1-424fe5a8a1ce None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:08.206035 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 253/523] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Oct 1 20:33:08 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:33:08.207196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e01e5d-152a-45d5-8dd1-424fe5a8a1ce None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:08.208438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f7d71-723f-4272-a569-1e6bd2dd03cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:08.209054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f7d71-723f-4272-a569-1e6bd2dd03cb None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:08.209337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f7d71-723f-4272-a569-1e6bd2dd03cb None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:08.209612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f7d71-723f-4272-a569-1e6bd2dd03cb None admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:08.210525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f7d71-723f-4272-a569-1e6bd2dd03cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:08.211871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f7d71-723f-4272-a569-1e6bd2dd03cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:08.213370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f7d71-723f-4272-a569-1e6bd2dd03cb None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:08.241626 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 271/524] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:08.242872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f7d71-723f-4272-a569-1e6bd2dd03cb None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:08.257470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:08.284020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:08.324712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e01e5d-152a-45d5-8dd1-424fe5a8a1ce None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:08.358778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f7d71-723f-4272-a569-1e6bd2dd03cb None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:08.362081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c1e63-1ad9-40fb-856a-8e53952d031c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:08.362755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c1e63-1ad9-40fb-856a-8e53952d031c None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:08.363089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c1e63-1ad9-40fb-856a-8e53952d031c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:08.363369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c1e63-1ad9-40fb-856a-8e53952d031c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:08.376828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c1e63-1ad9-40fb-856a-8e53952d031c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:08.378323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c1e63-1ad9-40fb-856a-8e53952d031c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:08.397421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-265c1e63-1ad9-40fb-856a-8e53952d031c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:08.397421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:08.397421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:08.397421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:08.397421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:08.397421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:08.397421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:08.397421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:08.397421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:08.397421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:08.397421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:08.397421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:08.397421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:08.397421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:08.397421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:08.397421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:08.398514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:08.398514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:08.398514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:08.398514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:08.398514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:08.398514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:08.398514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:08.398514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:08.398514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:08.398514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:08.398514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:08.398514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:08.398514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:08.398514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:08.398514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:08.398514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:08.398514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:08.399368 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:08.399368 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:08.399368 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:08.399537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37841a-6dda-48e5-9641-72788254f300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:08.399537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37841a-6dda-48e5-9641-72788254f300 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:08.399537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37841a-6dda-48e5-9641-72788254f300 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:08.399740 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 254/525] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:08.400359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c1e63-1ad9-40fb-856a-8e53952d031c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:08.403640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37841a-6dda-48e5-9641-72788254f300 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:08.415186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe37841a-6dda-48e5-9641-72788254f300 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:08.416361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe37841a-6dda-48e5-9641-72788254f300 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:08.431469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-fe37841a-6dda-48e5-9641-72788254f300 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:08.431469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:08.431469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:08.431469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:08.431469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:08.431469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:08.431469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:08.431469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:08.431469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:08.431469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:08.431469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:08.431469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:08.431469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:08.431469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:08.431469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:08.431469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:08.432435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:08.432435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:08.432435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:08.432435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:08.432435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:08.432435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:08.432435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:08.432435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:08.432435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:08.432435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:08.432435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:08.432435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:08.432435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:08.432435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:08.432435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:08.432435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:08.432435 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:08.433358 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:08.433358 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:08.433358 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:08.433358 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 272/526] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:08.433715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37841a-6dda-48e5-9641-72788254f300 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:08.512384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:08.522058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c1e63-1ad9-40fb-856a-8e53952d031c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:08.530793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:08.548332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37841a-6dda-48e5-9641-72788254f300 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:08.558367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d566b6d-fb8d-4441-b761-0b118773c886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:08.558728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d566b6d-fb8d-4441-b761-0b118773c886 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:08.559007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d566b6d-fb8d-4441-b761-0b118773c886 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:08.559277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d566b6d-fb8d-4441-b761-0b118773c886 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:08.584539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5e827-8078-4085-8c95-04f766b97148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:08.585142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5e827-8078-4085-8c95-04f766b97148 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:08.585423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5e827-8078-4085-8c95-04f766b97148 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:08.585696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5e827-8078-4085-8c95-04f766b97148 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:08.586422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5e827-8078-4085-8c95-04f766b97148 None admin] RBAC: Adding query filter params (role.id=ef7bd7293369475a9e7b3444fac30cda, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=7ab975dafb564aa2b357311c4a384d53) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:08.586756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5e827-8078-4085-8c95-04f766b97148 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:08.588189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5e827-8078-4085-8c95-04f766b97148 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:08.594161 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 273/527] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:33:08 2022] GET /identity/v3/role_assignments?user.id=7ab975dafb564aa2b357311c4a384d53&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=ef7bd7293369475a9e7b3444fac30cda => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:08.595283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5e827-8078-4085-8c95-04f766b97148 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:08.656040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d566b6d-fb8d-4441-b761-0b118773c886 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:08.658991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d566b6d-fb8d-4441-b761-0b118773c886 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:08.725958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5e827-8078-4085-8c95-04f766b97148 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:08.771655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1048a9-2009-4a20-88f9-004d82719656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:08.772951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1048a9-2009-4a20-88f9-004d82719656 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:08.773440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1048a9-2009-4a20-88f9-004d82719656 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:08.773905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1048a9-2009-4a20-88f9-004d82719656 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:08.774789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1048a9-2009-4a20-88f9-004d82719656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:08.775292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1048a9-2009-4a20-88f9-004d82719656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:08.777060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1048a9-2009-4a20-88f9-004d82719656 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:08.783474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:08.791776 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 255/528] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:33:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:33:08.792678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d566b6d-fb8d-4441-b761-0b118773c886 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:08.795064 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 274/529] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:08.800239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1048a9-2009-4a20-88f9-004d82719656 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:08.816953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:08.925358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d566b6d-fb8d-4441-b761-0b118773c886 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:08.938630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1048a9-2009-4a20-88f9-004d82719656 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:09.018516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275911d4-ef60-40bb-8664-ab02a8bdb3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:09.018516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275911d4-ef60-40bb-8664-ab02a8bdb3f1 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:09.018516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275911d4-ef60-40bb-8664-ab02a8bdb3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:09.018516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275911d4-ef60-40bb-8664-ab02a8bdb3f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:09.026514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f02e3-f262-4b2c-8ba7-c9f1033ef062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:09.026514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f02e3-f262-4b2c-8ba7-c9f1033ef062 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:09.026514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f02e3-f262-4b2c-8ba7-c9f1033ef062 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:09.026514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f02e3-f262-4b2c-8ba7-c9f1033ef062 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:09.026894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f02e3-f262-4b2c-8ba7-c9f1033ef062 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:09.026894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f02e3-f262-4b2c-8ba7-c9f1033ef062 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:09.026894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f02e3-f262-4b2c-8ba7-c9f1033ef062 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:09.038516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275911d4-ef60-40bb-8664-ab02a8bdb3f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:09.038516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275911d4-ef60-40bb-8664-ab02a8bdb3f1 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:09.058948 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 256/530] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:09.062123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f02e3-f262-4b2c-8ba7-c9f1033ef062 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:09.072388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-275911d4-ef60-40bb-8664-ab02a8bdb3f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:09.072388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:09.072388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:09.072388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:09.072388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:09.072388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:09.072388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:09.072388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:09.072388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:09.072388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:09.072388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:09.072388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:09.072388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:09.072388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:09.072388 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:09.073298 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:09.073298 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:09.073298 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:09.073298 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:09.073298 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:09.073298 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:09.073298 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:09.073298 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:09.073298 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:09.073298 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:09.073298 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:09.073298 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:09.073298 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:09.073298 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:09.073298 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:09.073298 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:09.073298 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:09.074157 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:09.074157 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:09.074157 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:09.074157 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:09.074157 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:09.074157 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:09.074157 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:09.074157 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:09.075469 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 275/531] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:09.076991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275911d4-ef60-40bb-8664-ab02a8bdb3f1 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:09.128066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:09.153575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:09.186321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f02e3-f262-4b2c-8ba7-c9f1033ef062 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:09.188928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275911d4-ef60-40bb-8664-ab02a8bdb3f1 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:09.224687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaef1e-da85-47ad-97b4-64d2d5da8814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:09.225325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaef1e-da85-47ad-97b4-64d2d5da8814 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:09.225610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaef1e-da85-47ad-97b4-64d2d5da8814 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:09.225897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaef1e-da85-47ad-97b4-64d2d5da8814 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:09.226765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc613ca3-27cf-4233-bf81-34577faad6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:09.227630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc613ca3-27cf-4233-bf81-34577faad6ad None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:09.228125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc613ca3-27cf-4233-bf81-34577faad6ad None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:09.228596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc613ca3-27cf-4233-bf81-34577faad6ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:09.236279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc613ca3-27cf-4233-bf81-34577faad6ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:09.237393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afaef1e-da85-47ad-97b4-64d2d5da8814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:09.238221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc613ca3-27cf-4233-bf81-34577faad6ad None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:09.238939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afaef1e-da85-47ad-97b4-64d2d5da8814 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:09.250393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-dc613ca3-27cf-4233-bf81-34577faad6ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:09.250393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:09.250393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:09.250393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:09.250393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:09.250393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:09.250393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:09.250393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:09.250393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:09.250393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:09.250393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:09.250393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:09.250393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:09.250393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:09.250393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:09.251451 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:09.251451 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:09.251451 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:09.251451 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:09.251451 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:09.251451 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:09.251451 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:09.251451 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:09.251451 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:09.251451 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:09.251451 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:09.251451 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:09.251451 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:09.251451 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:09.251451 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:09.251451 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:09.251451 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:09.252297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:09.252297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:09.252297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:09.252297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:09.252297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:09.252297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:09.252297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:09.252297 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:09.253551 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-8afaef1e-da85-47ad-97b4-64d2d5da8814 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:09.253551 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:09.253551 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:09.253551 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:09.253551 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:09.253551 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:09.253551 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:09.253551 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:09.253551 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:09.253551 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:09.253551 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:09.253551 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:09.253551 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:09.253551 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:09.253551 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:09.253551 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:09.254434 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:09.254434 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:09.254434 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:09.254434 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:09.254434 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:09.254434 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:09.254434 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:09.254434 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:09.254434 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:09.254434 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:09.254434 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:09.254434 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:09.254434 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:09.254434 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:09.254434 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:09.254434 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:09.254434 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:09.255495 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:09.255495 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:09.255495 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:09.255495 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 257/532] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:09.255495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaef1e-da85-47ad-97b4-64d2d5da8814 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:09.256879 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 276/533] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:09.265120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc613ca3-27cf-4233-bf81-34577faad6ad None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:09.363293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaef1e-da85-47ad-97b4-64d2d5da8814 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:09.371100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:09.381142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc613ca3-27cf-4233-bf81-34577faad6ad None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:09.401223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2502d-9801-4d09-a7a2-76892afd6e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:09.401836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2502d-9801-4d09-a7a2-76892afd6e28 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:09.402118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2502d-9801-4d09-a7a2-76892afd6e28 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:09.402394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2502d-9801-4d09-a7a2-76892afd6e28 None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:09.403127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2502d-9801-4d09-a7a2-76892afd6e28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:09.403437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2502d-9801-4d09-a7a2-76892afd6e28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:09.404883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2502d-9801-4d09-a7a2-76892afd6e28 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:09.414894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:09.420812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f1eee-768b-4fd8-8fb9-be55146b7382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:09.421667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f1eee-768b-4fd8-8fb9-be55146b7382 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:09.422136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f1eee-768b-4fd8-8fb9-be55146b7382 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:09.422658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f1eee-768b-4fd8-8fb9-be55146b7382 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:09.423904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f1eee-768b-4fd8-8fb9-be55146b7382 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:09.424153 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 258/534] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:09.424664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f1eee-768b-4fd8-8fb9-be55146b7382 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:09.425110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2502d-9801-4d09-a7a2-76892afd6e28 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:09.426510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f1eee-768b-4fd8-8fb9-be55146b7382 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:09.441663 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 277/535] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:09.442816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f1eee-768b-4fd8-8fb9-be55146b7382 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:09.536861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2502d-9801-4d09-a7a2-76892afd6e28 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:09.554352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f1eee-768b-4fd8-8fb9-be55146b7382 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:09.573245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae854de-bc55-4f3f-aac8-2e76d239dc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:09.573859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae854de-bc55-4f3f-aac8-2e76d239dc9b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:09.574142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae854de-bc55-4f3f-aac8-2e76d239dc9b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:09.574412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae854de-bc55-4f3f-aac8-2e76d239dc9b None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:09.575388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae854de-bc55-4f3f-aac8-2e76d239dc9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:09.575701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae854de-bc55-4f3f-aac8-2e76d239dc9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:09.577449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae854de-bc55-4f3f-aac8-2e76d239dc9b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:09.597105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861e55a-d6f1-41b9-94e7-a0a92e0645e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:09.598033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861e55a-d6f1-41b9-94e7-a0a92e0645e0 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:09.598558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861e55a-d6f1-41b9-94e7-a0a92e0645e0 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:09.598664 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 259/536] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:09.599140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861e55a-d6f1-41b9-94e7-a0a92e0645e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:09.599528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae854de-bc55-4f3f-aac8-2e76d239dc9b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:09.600298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861e55a-d6f1-41b9-94e7-a0a92e0645e0 None admin] RBAC: Adding query filter params (role.id=ef7bd7293369475a9e7b3444fac30cda, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=197a0ae6b71f4635a069ac0238e017c9) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:09.600805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861e55a-d6f1-41b9-94e7-a0a92e0645e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:09.602443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861e55a-d6f1-41b9-94e7-a0a92e0645e0 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:09.608705 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 278/537] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:33:09 2022] GET /identity/v3/role_assignments?user.id=197a0ae6b71f4635a069ac0238e017c9&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=ef7bd7293369475a9e7b3444fac30cda => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:09.609830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861e55a-d6f1-41b9-94e7-a0a92e0645e0 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:09.634666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:09.729011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae854de-bc55-4f3f-aac8-2e76d239dc9b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:09.770533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d7405b-5aff-417f-aa19-ffc4b228b929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:09.771197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7405b-5aff-417f-aa19-ffc4b228b929 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:09.771410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:09.771526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7405b-5aff-417f-aa19-ffc4b228b929 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:09.771741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7405b-5aff-417f-aa19-ffc4b228b929 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:09.781319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d7405b-5aff-417f-aa19-ffc4b228b929 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:09.783072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d7405b-5aff-417f-aa19-ffc4b228b929 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:09.796987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-83d7405b-5aff-417f-aa19-ffc4b228b929 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:09.796987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:09.796987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:09.796987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:09.796987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:09.796987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:09.796987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:09.796987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:09.796987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:09.796987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:09.796987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:09.796987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:09.796987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:09.796987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:09.796987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:09.797929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:09.797929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:09.797929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:09.797929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:09.797929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:09.797929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:09.797929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:09.797929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:09.797929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:09.797929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:09.797929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:09.797929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:09.797929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:09.797929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:09.797929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:09.797929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:09.797929 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:09.800332 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:09.800332 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:09.800332 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:09.800332 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:09.800332 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:09.800332 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:09.800332 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:09.800332 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:09.802872 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 260/538] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:09.806246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d7405b-5aff-417f-aa19-ffc4b228b929 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:09.896484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861e55a-d6f1-41b9-94e7-a0a92e0645e0 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:09.915287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:09.939948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834d1e5-8adb-4437-ac9b-5e424d3a3669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:09.940719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d7405b-5aff-417f-aa19-ffc4b228b929 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:09.940985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834d1e5-8adb-4437-ac9b-5e424d3a3669 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:09.941481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834d1e5-8adb-4437-ac9b-5e424d3a3669 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:09.941904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834d1e5-8adb-4437-ac9b-5e424d3a3669 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:09.942909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6834d1e5-8adb-4437-ac9b-5e424d3a3669 None admin] RBAC: Adding query filter params (role.id=ef7bd7293369475a9e7b3444fac30cda, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=e2927203ce394fabab76dd52a68c2436) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:09.943369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6834d1e5-8adb-4437-ac9b-5e424d3a3669 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:09.945362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6834d1e5-8adb-4437-ac9b-5e424d3a3669 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:09.953592 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 279/539] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:33:09 2022] GET /identity/v3/role_assignments?user.id=e2927203ce394fabab76dd52a68c2436&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=ef7bd7293369475a9e7b3444fac30cda => generated 259 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:09.985142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d82f0-6318-47e2-adf7-b8b3536b9d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:09.986755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d82f0-6318-47e2-adf7-b8b3536b9d35 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:09.987184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d82f0-6318-47e2-adf7-b8b3536b9d35 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:09.987663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d82f0-6318-47e2-adf7-b8b3536b9d35 None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:09.988542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d82f0-6318-47e2-adf7-b8b3536b9d35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:09.989000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d82f0-6318-47e2-adf7-b8b3536b9d35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:09.990899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d82f0-6318-47e2-adf7-b8b3536b9d35 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:10.006375 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 261/540] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:10.015736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834d1e5-8adb-4437-ac9b-5e424d3a3669 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:10.180179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834d1e5-8adb-4437-ac9b-5e424d3a3669 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:10.204146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:10.224085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e7bb8-f1be-448d-8a52-9fb6adc6758e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:10.224744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e7bb8-f1be-448d-8a52-9fb6adc6758e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:10.225048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e7bb8-f1be-448d-8a52-9fb6adc6758e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:10.225310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e7bb8-f1be-448d-8a52-9fb6adc6758e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:10.226047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e7bb8-f1be-448d-8a52-9fb6adc6758e None admin] RBAC: Adding query filter params (role.id=ef7bd7293369475a9e7b3444fac30cda, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=e17a026f305248878f18eec35e1e02dd) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:10.226359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e7bb8-f1be-448d-8a52-9fb6adc6758e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:10.227966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e7bb8-f1be-448d-8a52-9fb6adc6758e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:10.237127 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 280/541] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:33:10 2022] GET /identity/v3/role_assignments?user.id=e17a026f305248878f18eec35e1e02dd&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=ef7bd7293369475a9e7b3444fac30cda => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:10.776475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa0f8b-acd3-40c1-a331-8797e13830cc None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:10.776983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa0f8b-acd3-40c1-a331-8797e13830cc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:10.777057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa0f8b-acd3-40c1-a331-8797e13830cc None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:10.778416 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 262/542] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:10.784104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bccc01-95b8-41fb-a1da-a28f3b044b25 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:10.784393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bccc01-95b8-41fb-a1da-a28f3b044b25 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:10.784668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bccc01-95b8-41fb-a1da-a28f3b044b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:10.811731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-49bccc01-95b8-41fb-a1da-a28f3b044b25 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:10.812931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-49bccc01-95b8-41fb-a1da-a28f3b044b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:10.904546 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 281/543] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:11.422324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa0f8b-acd3-40c1-a331-8797e13830cc None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:11.483273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8ec1b-af20-43c4-895c-dee47db3d57f None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:11.484240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8ec1b-af20-43c4-895c-dee47db3d57f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:11.484731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8ec1b-af20-43c4-895c-dee47db3d57f None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:11.486433 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 282/544] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:11.491770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2e54a-3094-4e24-8da0-98526d368802 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:11.492399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2e54a-3094-4e24-8da0-98526d368802 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:11.492903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2e54a-3094-4e24-8da0-98526d368802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:11.516549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-94d2e54a-3094-4e24-8da0-98526d368802 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:11.519276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-94d2e54a-3094-4e24-8da0-98526d368802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:11.589568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa0f8b-acd3-40c1-a331-8797e13830cc None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:11.616050 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 283/545] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:11.647460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d8de5-d068-4254-9118-7a2d9fa4e45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:11.648467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d8de5-d068-4254-9118-7a2d9fa4e45c None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:11.648936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d8de5-d068-4254-9118-7a2d9fa4e45c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:11.649392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d8de5-d068-4254-9118-7a2d9fa4e45c None admin] PATH_INFO: `/v3/roles/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:11.667634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d8de5-d068-4254-9118-7a2d9fa4e45c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:11.672731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-6f8d8de5-d068-4254-9118-7a2d9fa4e45c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:11.672731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:11.672731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:11.672731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:11.672731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:11.672731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:11.672731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:11.672731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:11.672731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:11.672731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:11.672731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:11.672731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:11.672731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:11.672731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:11.672731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:11.672731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:11.673624 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:11.673624 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:11.673624 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:11.673624 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:11.673624 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:11.673624 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:11.673624 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:11.673624 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:11.673624 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:11.673624 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:11.673624 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:11.673624 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:11.673624 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:11.673624 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:11.673624 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:11.673624 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:11.674438 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:11.674438 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:11.674438 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:11.674438 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:11.674438 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:11.674438 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:11.674438 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:11.674438 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:11.674438 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:11.674438 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:11.674438 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:11.674438 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:11.676948 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 263/546] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:11.682849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2e54a-3094-4e24-8da0-98526d368802 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:11.733815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:11.806003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2e54a-3094-4e24-8da0-98526d368802 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:11.856917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16458d20-e560-4f99-85ae-acce180be9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:11.857583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16458d20-e560-4f99-85ae-acce180be9ef None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:11.857866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16458d20-e560-4f99-85ae-acce180be9ef None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:11.858136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16458d20-e560-4f99-85ae-acce180be9ef None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:11.858843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16458d20-e560-4f99-85ae-acce180be9ef None admin] RBAC: Adding query filter params (name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:11.859153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16458d20-e560-4f99-85ae-acce180be9ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:11.860523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16458d20-e560-4f99-85ae-acce180be9ef None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:11.873975 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 284/547] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:33:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:11.879996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d8de5-d068-4254-9118-7a2d9fa4e45c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:11.889983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:11.974676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16458d20-e560-4f99-85ae-acce180be9ef None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:12.015538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d8de5-d068-4254-9118-7a2d9fa4e45c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:12.087740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a3b68-739f-489d-b863-c840ca89c417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:12.088370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a3b68-739f-489d-b863-c840ca89c417 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:12.088646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a3b68-739f-489d-b863-c840ca89c417 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:12.088916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a3b68-739f-489d-b863-c840ca89c417 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:12.108141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a3b68-739f-489d-b863-c840ca89c417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:12.109599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a3b68-739f-489d-b863-c840ca89c417 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:12.136110 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-975a3b68-739f-489d-b863-c840ca89c417 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:12.136110 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:12.136110 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:12.136110 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:12.136110 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:12.136110 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:12.136110 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:12.136110 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:12.136110 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:12.136110 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:12.136110 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:12.136110 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:12.136110 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:12.136110 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:12.136110 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:12.136110 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:12.137092 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:12.137092 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:12.137092 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:12.137092 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:12.137092 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:12.137092 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:12.137092 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:12.137092 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:12.137092 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:12.137092 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:12.137092 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:12.137092 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:12.137092 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:12.137092 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:12.137092 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:12.137092 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:12.137092 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:12.138446 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:12.138446 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:12.138446 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:12.138446 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 264/548] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:12.147717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a3b68-739f-489d-b863-c840ca89c417 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:12.156457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16458d20-e560-4f99-85ae-acce180be9ef None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:12.159680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:12.210526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7312c74-36ed-430f-9bfa-c082ab02b64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:12.212089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7312c74-36ed-430f-9bfa-c082ab02b64c None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:12.212368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7312c74-36ed-430f-9bfa-c082ab02b64c None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:12.212641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7312c74-36ed-430f-9bfa-c082ab02b64c None admin] PATH_INFO: `/v3/roles/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:12.234597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7312c74-36ed-430f-9bfa-c082ab02b64c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:12.244690 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-d7312c74-36ed-430f-9bfa-c082ab02b64c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:12.244690 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:12.244690 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:12.244690 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:12.244690 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:12.244690 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:12.244690 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:12.244690 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:12.244690 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:12.244690 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:12.244690 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:12.244690 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:12.244690 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:12.244690 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:12.244690 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:12.244690 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:12.247043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:12.247043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:12.247043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:12.247043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:12.247043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:12.247043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:12.247043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:12.247043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:12.247043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:12.247043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:12.247043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:12.247043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:12.247043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:12.247043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:12.247043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:12.247043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:12.248229 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:12.248229 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:12.248229 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:12.248229 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:12.248229 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:12.248229 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:12.248229 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:12.248229 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:12.248229 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:12.248229 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:12.248229 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:12.248229 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:12.248229 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 285/549] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:12.254926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7312c74-36ed-430f-9bfa-c082ab02b64c None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:12.258273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:12.341288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a3b68-739f-489d-b863-c840ca89c417 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:12.406952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa404d-ddef-402f-a424-acae2f354d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:12.408087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa404d-ddef-402f-a424-acae2f354d28 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:12.408580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa404d-ddef-402f-a424-acae2f354d28 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:12.408864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa404d-ddef-402f-a424-acae2f354d28 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:12.410813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa404d-ddef-402f-a424-acae2f354d28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:12.411133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa404d-ddef-402f-a424-acae2f354d28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:12.413186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa404d-ddef-402f-a424-acae2f354d28 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:12.437358 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 265/550] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:12.443558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa404d-ddef-402f-a424-acae2f354d28 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:12.455014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7312c74-36ed-430f-9bfa-c082ab02b64c None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:12.515578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa1741-2253-40d4-a450-5d129be6d875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:12.516306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa1741-2253-40d4-a450-5d129be6d875 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:12.516466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa1741-2253-40d4-a450-5d129be6d875 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:12.516616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa1741-2253-40d4-a450-5d129be6d875 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:12.517328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa1741-2253-40d4-a450-5d129be6d875 None admin] RBAC: Adding query filter params (name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:12.517632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa1741-2253-40d4-a450-5d129be6d875 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:12.519321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa1741-2253-40d4-a450-5d129be6d875 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:12.540760 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 286/551] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:33:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:12.543878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa1741-2253-40d4-a450-5d129be6d875 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:12.592990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:12.616064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa404d-ddef-402f-a424-acae2f354d28 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:12.637426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:12.658423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af027547-f95b-4f88-9d04-ee6db98fa700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:12.659478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af027547-f95b-4f88-9d04-ee6db98fa700 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:12.659976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af027547-f95b-4f88-9d04-ee6db98fa700 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:12.660530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af027547-f95b-4f88-9d04-ee6db98fa700 None admin] PATH_INFO: `/v3/users/swift` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:12.668819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af027547-f95b-4f88-9d04-ee6db98fa700 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:12.670852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af027547-f95b-4f88-9d04-ee6db98fa700 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:12.685175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-af027547-f95b-4f88-9d04-ee6db98fa700 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 01 20:33:12.685175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:12.685175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:12.685175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:12.685175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:12.685175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:12.685175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:12.685175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:12.685175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:12.685175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:12.685175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:12.685175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:12.685175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:12.685175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:12.685175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:12.685175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:12.686227 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:12.686227 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:12.686227 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:12.686227 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:12.686227 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:12.686227 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:12.686227 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:12.686227 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:12.686227 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:12.686227 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:12.686227 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:12.686227 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:12.686227 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:12.686227 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:12.686227 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:12.686227 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:12.686227 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:12.686227 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:12.687205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:12.687205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:12.687205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:12.687205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:12.687205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:12.687205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:12.687205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:12.687205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:12.687205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 01 20:33:12.687205 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:12.689761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa1741-2253-40d4-a450-5d129be6d875 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:12.690538 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 266/552] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:33:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:12.691904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f555358-51a8-47b4-bad8-9979a2a4ab78 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:12.692382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f555358-51a8-47b4-bad8-9979a2a4ab78 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:12.692876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f555358-51a8-47b4-bad8-9979a2a4ab78 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:12.694572 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 267/553] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:12.695801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f555358-51a8-47b4-bad8-9979a2a4ab78 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:12.746283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d456c06-e7e1-4821-8874-dcc8644a571f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:12.746934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d456c06-e7e1-4821-8874-dcc8644a571f None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:12.747209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d456c06-e7e1-4821-8874-dcc8644a571f None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:12.747474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d456c06-e7e1-4821-8874-dcc8644a571f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:12.762908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d456c06-e7e1-4821-8874-dcc8644a571f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:12.764081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d456c06-e7e1-4821-8874-dcc8644a571f None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:12.779019 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-2d456c06-e7e1-4821-8874-dcc8644a571f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:12.779019 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:12.779019 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:12.779019 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:12.779019 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:12.779019 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:12.779019 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:12.779019 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:12.779019 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:12.779019 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:12.779019 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:12.779019 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:12.779019 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:12.779019 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:12.779019 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:12.779019 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:12.780023 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:12.780023 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:12.780023 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:12.780023 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:12.780023 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:12.780023 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:12.780023 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:12.780023 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:12.780023 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:12.780023 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:12.780023 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:12.780023 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:12.780023 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:12.780023 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:12.780023 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:12.780023 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:12.780023 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:12.780921 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:12.780921 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:12.780921 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:12.780921 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 287/554] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:12.781728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843097a-ea3d-4573-bc63-d99de85d496c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:12.781940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843097a-ea3d-4573-bc63-d99de85d496c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:12.782215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843097a-ea3d-4573-bc63-d99de85d496c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:12.801188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7843097a-ea3d-4573-bc63-d99de85d496c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:12.802476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7843097a-ea3d-4573-bc63-d99de85d496c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:12.845059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f555358-51a8-47b4-bad8-9979a2a4ab78 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:12.882759 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 288/555] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:12.883797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843097a-ea3d-4573-bc63-d99de85d496c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:12.895241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427f14b-2eb6-4187-af06-604d8167e194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:12.896559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427f14b-2eb6-4187-af06-604d8167e194 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:12.897036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427f14b-2eb6-4187-af06-604d8167e194 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:12.897505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427f14b-2eb6-4187-af06-604d8167e194 None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:12.898438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427f14b-2eb6-4187-af06-604d8167e194 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:12.899553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427f14b-2eb6-4187-af06-604d8167e194 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:12.901209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427f14b-2eb6-4187-af06-604d8167e194 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:12.905536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:12.926369 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 268/556] 10.222.0.17 () {56 vars in 1135 bytes} [Sat Oct 1 20:33:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:12.930939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427f14b-2eb6-4187-af06-604d8167e194 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:13.008653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:13.050875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843097a-ea3d-4573-bc63-d99de85d496c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:13.092457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec5a60-9729-4b9e-8844-dee53edee22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:13.093416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec5a60-9729-4b9e-8844-dee53edee22d None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:13.093843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec5a60-9729-4b9e-8844-dee53edee22d None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:13.094284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec5a60-9729-4b9e-8844-dee53edee22d None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:13.095675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec5a60-9729-4b9e-8844-dee53edee22d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:13.096385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec5a60-9729-4b9e-8844-dee53edee22d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:13.098667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec5a60-9729-4b9e-8844-dee53edee22d None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:13.112568 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 289/557] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:13.114106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075db586-fa84-436e-bd18-daa14358d749 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:13.114560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075db586-fa84-436e-bd18-daa14358d749 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:13.114980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075db586-fa84-436e-bd18-daa14358d749 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:13.116726 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 290/558] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:13.118796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075db586-fa84-436e-bd18-daa14358d749 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:13.125276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427f14b-2eb6-4187-af06-604d8167e194 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:13.167180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53322772-210b-473b-9f55-7f95123712e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:13.168463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53322772-210b-473b-9f55-7f95123712e4 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:13.168885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53322772-210b-473b-9f55-7f95123712e4 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:13.169257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53322772-210b-473b-9f55-7f95123712e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:13.210008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:13.253265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53322772-210b-473b-9f55-7f95123712e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:13.253877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075db586-fa84-436e-bd18-daa14358d749 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:13.255761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53322772-210b-473b-9f55-7f95123712e4 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:13.295934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:13.299804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650654e-0254-4d77-a887-4c1f0a484fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:13.300809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650654e-0254-4d77-a887-4c1f0a484fd4 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:13.301279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650654e-0254-4d77-a887-4c1f0a484fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:13.302827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650654e-0254-4d77-a887-4c1f0a484fd4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:13.312174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650654e-0254-4d77-a887-4c1f0a484fd4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:13.314398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650654e-0254-4d77-a887-4c1f0a484fd4 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:13.326697 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-c650654e-0254-4d77-a887-4c1f0a484fd4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 01 20:33:13.326697 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:13.326697 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:13.326697 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:13.326697 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:13.326697 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:13.326697 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:13.326697 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:13.326697 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:13.326697 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:13.326697 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:13.326697 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:13.326697 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:13.326697 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:13.326697 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:13.326697 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:13.327869 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:13.327869 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:13.327869 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:13.327869 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:13.327869 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:13.327869 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:13.327869 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:13.327869 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:13.327869 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:13.327869 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:13.327869 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:13.327869 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:13.327869 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:13.327869 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:13.327869 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:13.327869 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:13.327869 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:13.327869 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:13.328796 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:13.328796 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:13.328796 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:13.328796 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:13.328796 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:13.328796 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:13.328796 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:13.328796 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:13.328796 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 01 20:33:13.328796 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:13.328796 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 291/559] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Oct 1 20:33:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:13.329977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52f35f-1131-42f6-a80a-bb4cec5186b0 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:13.330053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52f35f-1131-42f6-a80a-bb4cec5186b0 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:13.330528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52f35f-1131-42f6-a80a-bb4cec5186b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:13.347667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7f52f35f-1131-42f6-a80a-bb4cec5186b0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:13.348851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7f52f35f-1131-42f6-a80a-bb4cec5186b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:13.367535 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 269/560] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:33:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 01 20:33:13.368361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53322772-210b-473b-9f55-7f95123712e4 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:13.417608 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 292/561] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:13.418872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac210f-a12a-4ab8-a1a0-e653a5569726 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:13.419157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac210f-a12a-4ab8-a1a0-e653a5569726 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:13.419424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac210f-a12a-4ab8-a1a0-e653a5569726 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:13.420768 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 293/562] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:13.425369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac210f-a12a-4ab8-a1a0-e653a5569726 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:13.470781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53322772-210b-473b-9f55-7f95123712e4 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:13.495213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:13.507800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1bca4-fdfb-46fe-8bfc-cb585c36fc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:13.508430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1bca4-fdfb-46fe-8bfc-cb585c36fc2f None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:13.508702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1bca4-fdfb-46fe-8bfc-cb585c36fc2f None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:13.508975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1bca4-fdfb-46fe-8bfc-cb585c36fc2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:13.511550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:13.520669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1bca4-fdfb-46fe-8bfc-cb585c36fc2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:13.525364 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-84e1bca4-fdfb-46fe-8bfc-cb585c36fc2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:13.525364 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:13.525364 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:13.525364 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:13.525364 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:13.525364 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:13.525364 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:13.525364 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:13.525364 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:13.525364 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:13.525364 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:13.525364 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:13.525364 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:13.525364 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:13.525364 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:13.525364 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:13.526284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:13.526284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:13.526284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:13.526284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:13.526284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:13.526284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:13.526284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:13.526284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:13.526284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:13.526284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:13.526284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:13.526284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:13.526284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:13.526284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:13.526284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:13.526284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:13.527175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:13.527175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:13.527175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:13.527175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:13.527175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:13.527175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:13.527175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:13.527175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:13.527175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:13.527175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:13.527175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:13.527175 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:13.527175 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 270/563] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:13.527868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1bca4-fdfb-46fe-8bfc-cb585c36fc2f None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:13.546596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac210f-a12a-4ab8-a1a0-e653a5569726 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:13.590062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5950e-33d3-40c2-bd25-db42419e0a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:13.590907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5950e-33d3-40c2-bd25-db42419e0a41 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:13.591205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5950e-33d3-40c2-bd25-db42419e0a41 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:13.591474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5950e-33d3-40c2-bd25-db42419e0a41 None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:13.592195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5950e-33d3-40c2-bd25-db42419e0a41 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:13.592495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5950e-33d3-40c2-bd25-db42419e0a41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:13.593902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5950e-33d3-40c2-bd25-db42419e0a41 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:13.611386 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 294/564] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Oct 1 20:33:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:13.612546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e70c64-bf85-4d81-b93b-ae2b62a499b5 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:13.612825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e70c64-bf85-4d81-b93b-ae2b62a499b5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:13.613096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e70c64-bf85-4d81-b93b-ae2b62a499b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:13.630093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f1e70c64-bf85-4d81-b93b-ae2b62a499b5 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:13.631306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f1e70c64-bf85-4d81-b93b-ae2b62a499b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:13.637067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1bca4-fdfb-46fe-8bfc-cb585c36fc2f None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:13.674034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4530343-ef25-47cd-a297-cc4853a29be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:13.674963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4530343-ef25-47cd-a297-cc4853a29be4 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:13.675444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4530343-ef25-47cd-a297-cc4853a29be4 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:13.675893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4530343-ef25-47cd-a297-cc4853a29be4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:13.690058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4530343-ef25-47cd-a297-cc4853a29be4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:13.691517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4530343-ef25-47cd-a297-cc4853a29be4 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:13.704920 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 295/565] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:13.705915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e70c64-bf85-4d81-b93b-ae2b62a499b5 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:13.707022 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-d4530343-ef25-47cd-a297-cc4853a29be4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:13.707022 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:13.707022 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:13.707022 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:13.707022 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:13.707022 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:13.707022 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:13.707022 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:13.707022 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:13.707022 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:13.707022 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:13.707022 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:13.707022 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:13.707022 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:13.707022 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:13.707022 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:13.707939 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:13.707939 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:13.707939 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:13.707939 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:13.707939 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:13.707939 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:13.707939 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:13.707939 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:13.707939 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:13.707939 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:13.707939 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:13.707939 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:13.707939 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:13.707939 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:13.707939 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:13.707939 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:13.707939 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:13.708818 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:13.708818 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:13.708818 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:13.712593 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 271/566] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:13.713391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4530343-ef25-47cd-a297-cc4853a29be4 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:13.764146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:13.778944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:13.823488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e70c64-bf85-4d81-b93b-ae2b62a499b5 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:13.833233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4530343-ef25-47cd-a297-cc4853a29be4 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:13.860837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733b3d7-2f60-4c37-9764-e8f83160964a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:13.861371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733b3d7-2f60-4c37-9764-e8f83160964a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:13.861653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733b3d7-2f60-4c37-9764-e8f83160964a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:13.861919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733b3d7-2f60-4c37-9764-e8f83160964a None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:13.862601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733b3d7-2f60-4c37-9764-e8f83160964a None admin] RBAC: Adding query filter params (name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:13.862905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733b3d7-2f60-4c37-9764-e8f83160964a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:13.864248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733b3d7-2f60-4c37-9764-e8f83160964a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:13.865547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e27748-7f38-4459-99a1-cfbf2d0aa47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:13.866165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e27748-7f38-4459-99a1-cfbf2d0aa47e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:13.866445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e27748-7f38-4459-99a1-cfbf2d0aa47e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:13.866796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e27748-7f38-4459-99a1-cfbf2d0aa47e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:13.877565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e27748-7f38-4459-99a1-cfbf2d0aa47e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:13.877880 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 296/567] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:33:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:13.878909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733b3d7-2f60-4c37-9764-e8f83160964a None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:13.879036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e27748-7f38-4459-99a1-cfbf2d0aa47e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:13.893626 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-c2e27748-7f38-4459-99a1-cfbf2d0aa47e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:13.893626 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:13.893626 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:13.893626 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:13.893626 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:13.893626 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:13.893626 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:13.893626 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:13.893626 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:13.893626 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:13.893626 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:13.893626 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:13.893626 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:13.893626 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:13.893626 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:13.893626 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:13.894602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:13.894602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:13.894602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:13.894602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:13.894602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:13.894602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:13.894602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:13.894602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:13.894602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:13.894602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:13.894602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:13.894602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:13.894602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:13.894602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:13.894602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:13.894602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:13.894602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:13.895932 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:13.895932 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:13.895932 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:13.895932 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 272/568] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:13.895932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e27748-7f38-4459-99a1-cfbf2d0aa47e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:13.989552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733b3d7-2f60-4c37-9764-e8f83160964a None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:13.996224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:13.999446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:13.999642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e27748-7f38-4459-99a1-cfbf2d0aa47e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:14.031497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726099c-d424-4bc4-8436-6b5017b759d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:14.032106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726099c-d424-4bc4-8436-6b5017b759d2 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:14.032519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726099c-d424-4bc4-8436-6b5017b759d2 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:14.032892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726099c-d424-4bc4-8436-6b5017b759d2 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:14.033694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726099c-d424-4bc4-8436-6b5017b759d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:14.034076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726099c-d424-4bc4-8436-6b5017b759d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:14.035478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726099c-d424-4bc4-8436-6b5017b759d2 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:14.035908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbaa01-af38-47b6-aadc-a9f16d913d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:14.036538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbaa01-af38-47b6-aadc-a9f16d913d0e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:14.036823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbaa01-af38-47b6-aadc-a9f16d913d0e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:14.037269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbaa01-af38-47b6-aadc-a9f16d913d0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:14.047572 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 297/569] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:14.048256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726099c-d424-4bc4-8436-6b5017b759d2 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:14.048675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dbaa01-af38-47b6-aadc-a9f16d913d0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:14.054730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-c8dbaa01-af38-47b6-aadc-a9f16d913d0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:14.054730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:14.054730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:14.054730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:14.054730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:14.054730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:14.054730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:14.054730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:14.054730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:14.054730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:14.054730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:14.054730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:14.054730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:14.054730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:14.054730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:14.054730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:14.055866 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:14.055866 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:14.055866 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:14.055866 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:14.055866 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:14.055866 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:14.055866 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:14.055866 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:14.055866 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:14.055866 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:14.055866 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:14.055866 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:14.055866 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:14.055866 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:14.055866 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:14.055866 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:14.056896 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:14.056896 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:14.056896 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:14.056896 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:14.056896 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:14.056896 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:14.056896 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:14.056896 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:14.056896 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:14.056896 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:14.056896 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:14.056896 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:14.058637 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 273/570] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:14.059831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbaa01-af38-47b6-aadc-a9f16d913d0e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:14.148391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726099c-d424-4bc4-8436-6b5017b759d2 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:14.157214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbaa01-af38-47b6-aadc-a9f16d913d0e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:14.180987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b321da3-666c-4225-a144-614cf8e7e0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:14.181607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b321da3-666c-4225-a144-614cf8e7e0c5 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:14.181878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b321da3-666c-4225-a144-614cf8e7e0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:14.182148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b321da3-666c-4225-a144-614cf8e7e0c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:14.189843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966ddec-1e3f-4c10-b160-01279b269535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:14.190440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966ddec-1e3f-4c10-b160-01279b269535 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:14.190768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966ddec-1e3f-4c10-b160-01279b269535 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:14.191043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966ddec-1e3f-4c10-b160-01279b269535 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:14.191677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966ddec-1e3f-4c10-b160-01279b269535 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:14.191981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966ddec-1e3f-4c10-b160-01279b269535 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:14.192618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b321da3-666c-4225-a144-614cf8e7e0c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:14.193334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966ddec-1e3f-4c10-b160-01279b269535 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:14.194034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b321da3-666c-4225-a144-614cf8e7e0c5 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:14.211087 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 274/571] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:14.211087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966ddec-1e3f-4c10-b160-01279b269535 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:14.211323 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-9b321da3-666c-4225-a144-614cf8e7e0c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:14.211323 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:14.211323 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:14.211323 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:14.211323 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:14.211323 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:14.211323 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:14.211323 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:14.211323 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:14.211323 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:14.211323 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:14.211323 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:14.211323 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:14.211323 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:14.211323 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:14.211323 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:14.212225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:14.212225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:14.212225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:14.212225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:14.212225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:14.212225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:14.212225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:14.212225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:14.212225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:14.212225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:14.212225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:14.212225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:14.212225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:14.212225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:14.212225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:14.212225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:14.212225 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:14.213099 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:14.213099 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:14.213099 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:14.213099 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 298/572] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:14.213099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b321da3-666c-4225-a144-614cf8e7e0c5 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:14.228661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:14.233584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:14.306411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966ddec-1e3f-4c10-b160-01279b269535 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:14.310719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b321da3-666c-4225-a144-614cf8e7e0c5 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:14.340111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7f3ed-9bd0-4d17-875b-f94e467b7d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:14.340736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7f3ed-9bd0-4d17-875b-f94e467b7d01 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:14.341020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7f3ed-9bd0-4d17-875b-f94e467b7d01 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:14.341296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7f3ed-9bd0-4d17-875b-f94e467b7d01 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:14.347013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc62aba-2747-455e-84b4-2f3ba9c4e626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:14.347627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc62aba-2747-455e-84b4-2f3ba9c4e626 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:14.347898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc62aba-2747-455e-84b4-2f3ba9c4e626 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:14.348173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc62aba-2747-455e-84b4-2f3ba9c4e626 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:14.351582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7f3ed-9bd0-4d17-875b-f94e467b7d01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:14.354916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc62aba-2747-455e-84b4-2f3ba9c4e626 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:14.356340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc62aba-2747-455e-84b4-2f3ba9c4e626 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:14.356439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-f1a7f3ed-9bd0-4d17-875b-f94e467b7d01 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:14.356439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:14.356439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:14.356439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:14.356439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:14.356439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:14.356439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:14.356439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:14.356439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:14.356439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:14.356439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:14.356439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:14.356439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:14.356439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:14.356439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:14.356439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:14.357497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:14.357497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:14.357497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:14.357497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:14.357497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:14.357497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:14.357497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:14.357497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:14.357497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:14.357497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:14.357497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:14.357497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:14.357497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:14.357497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:14.357497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:14.357497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:14.358538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:14.358538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:14.358538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:14.358538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:14.358538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:14.358538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:14.358538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:14.358538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:14.358538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:14.358538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:14.358538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:14.358538 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:14.358538 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 275/573] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:14.358538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7f3ed-9bd0-4d17-875b-f94e467b7d01 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:14.371294 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-8dc62aba-2747-455e-84b4-2f3ba9c4e626 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:14.371294 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:14.371294 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:14.371294 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:14.371294 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:14.371294 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:14.371294 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:14.371294 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:14.371294 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:14.371294 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:14.371294 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:14.371294 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:14.371294 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:14.371294 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:14.371294 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:14.372273 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:14.372273 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:14.372273 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:14.372273 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:14.372273 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:14.372273 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:14.372273 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:14.372273 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:14.372273 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:14.372273 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:14.372273 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:14.372273 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:14.372273 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:14.372273 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:14.372273 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:14.372273 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:14.372273 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:14.373215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:14.373215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:14.373215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:14.373215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:14.373215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:14.373215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:14.373215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:14.373215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:14.373215 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 299/574] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:14.373215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc62aba-2747-455e-84b4-2f3ba9c4e626 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:14.446450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:14.452374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:14.466236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7f3ed-9bd0-4d17-875b-f94e467b7d01 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:14.473058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc62aba-2747-455e-84b4-2f3ba9c4e626 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:14.500046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07248ae9-516c-4339-aded-4b3878c3df9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:14.500651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07248ae9-516c-4339-aded-4b3878c3df9e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:14.500963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07248ae9-516c-4339-aded-4b3878c3df9e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:14.501208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07248ae9-516c-4339-aded-4b3878c3df9e None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:14.501907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07248ae9-516c-4339-aded-4b3878c3df9e None admin] RBAC: Adding query filter params (name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:14.502211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07248ae9-516c-4339-aded-4b3878c3df9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:14.503618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07248ae9-516c-4339-aded-4b3878c3df9e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:14.505873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36360c2b-d22b-46f3-a034-8aa0ec28281b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:14.506508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36360c2b-d22b-46f3-a034-8aa0ec28281b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:14.506796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36360c2b-d22b-46f3-a034-8aa0ec28281b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:14.507068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36360c2b-d22b-46f3-a034-8aa0ec28281b None admin] PATH_INFO: `/v3/projects/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:14.511549 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 276/575] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:33:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:14.512329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07248ae9-516c-4339-aded-4b3878c3df9e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:14.513574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36360c2b-d22b-46f3-a034-8aa0ec28281b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:14.515031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36360c2b-d22b-46f3-a034-8aa0ec28281b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:14.526648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-36360c2b-d22b-46f3-a034-8aa0ec28281b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:14.526648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:14.526648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:14.526648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:14.526648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:14.526648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:14.526648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:14.526648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:14.526648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:14.526648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:14.526648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:14.526648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:14.526648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:14.526648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:14.526648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:14.527535 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:14.527535 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:14.527535 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:14.527535 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:14.527535 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:14.527535 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:14.527535 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:14.527535 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:14.527535 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:14.527535 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:14.527535 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:14.527535 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:14.527535 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:14.527535 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:14.527535 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:14.527535 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:14.527535 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:14.528455 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:14.528455 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:14.528455 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:14.528455 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:14.528455 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:14.528455 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:14.528455 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:14.528455 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:14.528455 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 300/576] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:14.528455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36360c2b-d22b-46f3-a034-8aa0ec28281b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:14.613537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07248ae9-516c-4339-aded-4b3878c3df9e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:14.627131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36360c2b-d22b-46f3-a034-8aa0ec28281b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:14.646759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f893a5b-3c43-4174-8c2c-895c06650592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:14.647355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f893a5b-3c43-4174-8c2c-895c06650592 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:14.647635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f893a5b-3c43-4174-8c2c-895c06650592 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:14.647908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f893a5b-3c43-4174-8c2c-895c06650592 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:14.648545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f893a5b-3c43-4174-8c2c-895c06650592 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:14.648847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f893a5b-3c43-4174-8c2c-895c06650592 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:14.650210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f893a5b-3c43-4174-8c2c-895c06650592 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:14.660200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f3b7d-d297-40f0-8596-e94c3a28b781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:14.660789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f3b7d-d297-40f0-8596-e94c3a28b781 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:14.661064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f3b7d-d297-40f0-8596-e94c3a28b781 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:14.661333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f3b7d-d297-40f0-8596-e94c3a28b781 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:14.661984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f3b7d-d297-40f0-8596-e94c3a28b781 None admin] RBAC: Adding query filter params (name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:14.662094 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 277/577] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:14.662289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f3b7d-d297-40f0-8596-e94c3a28b781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:14.662794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f893a5b-3c43-4174-8c2c-895c06650592 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:14.663671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f3b7d-d297-40f0-8596-e94c3a28b781 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:14.668279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:14.676623 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 301/578] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:33:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:14.676623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f3b7d-d297-40f0-8596-e94c3a28b781 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:14.676623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:14.767085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f893a5b-3c43-4174-8c2c-895c06650592 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:14.775341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f3b7d-d297-40f0-8596-e94c3a28b781 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:14.800261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f15dee-e87d-4b7d-bfe0-c64880ea0a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:14.800858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f15dee-e87d-4b7d-bfe0-c64880ea0a15 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:14.801142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f15dee-e87d-4b7d-bfe0-c64880ea0a15 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:14.801427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f15dee-e87d-4b7d-bfe0-c64880ea0a15 None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:14.802161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f15dee-e87d-4b7d-bfe0-c64880ea0a15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:14.802513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f15dee-e87d-4b7d-bfe0-c64880ea0a15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:14.803974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f15dee-e87d-4b7d-bfe0-c64880ea0a15 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:14.808254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802ac58-e35d-4ea4-b65e-52397161a925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:14.808876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802ac58-e35d-4ea4-b65e-52397161a925 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:14.809155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802ac58-e35d-4ea4-b65e-52397161a925 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:14.809422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802ac58-e35d-4ea4-b65e-52397161a925 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:14.815656 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 278/579] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:14.816439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f15dee-e87d-4b7d-bfe0-c64880ea0a15 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:14.824671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802ac58-e35d-4ea4-b65e-52397161a925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:14.826110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802ac58-e35d-4ea4-b65e-52397161a925 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:14.840332 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-7802ac58-e35d-4ea4-b65e-52397161a925 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:14.840332 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:14.840332 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:14.840332 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:14.840332 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:14.840332 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:14.840332 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:14.840332 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:14.840332 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:14.840332 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:14.840332 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:14.840332 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:14.840332 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:14.840332 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:14.840332 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:14.840332 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:14.841393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:14.841393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:14.841393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:14.841393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:14.841393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:14.841393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:14.841393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:14.841393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:14.841393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:14.841393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:14.841393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:14.841393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:14.841393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:14.841393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:14.841393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:14.841393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:14.841393 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:14.842292 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:14.842292 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:14.842292 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:14.842292 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 302/580] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:14.842649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802ac58-e35d-4ea4-b65e-52397161a925 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:14.882009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:14.901378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:14.920642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f15dee-e87d-4b7d-bfe0-c64880ea0a15 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:14.946832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802ac58-e35d-4ea4-b65e-52397161a925 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:14.953702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b447d3-6f05-438a-b8e8-3601fba3d372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:14.954321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b447d3-6f05-438a-b8e8-3601fba3d372 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:14.954627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b447d3-6f05-438a-b8e8-3601fba3d372 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:14.954898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b447d3-6f05-438a-b8e8-3601fba3d372 None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:14.955579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b447d3-6f05-438a-b8e8-3601fba3d372 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:14.955879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b447d3-6f05-438a-b8e8-3601fba3d372 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:14.957300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b447d3-6f05-438a-b8e8-3601fba3d372 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:14.969191 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 279/581] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:14.970610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b447d3-6f05-438a-b8e8-3601fba3d372 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:14.984996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03a5f3-0035-4fb6-8cc3-4ab8c817a08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:14.986259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03a5f3-0035-4fb6-8cc3-4ab8c817a08e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:14.986477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03a5f3-0035-4fb6-8cc3-4ab8c817a08e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:14.986780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03a5f3-0035-4fb6-8cc3-4ab8c817a08e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:14.994124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d03a5f3-0035-4fb6-8cc3-4ab8c817a08e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:14.995581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d03a5f3-0035-4fb6-8cc3-4ab8c817a08e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:15.006439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-9d03a5f3-0035-4fb6-8cc3-4ab8c817a08e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 01 20:33:15.006439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:15.006439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:15.006439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:15.006439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:15.006439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:15.006439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:15.006439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:15.006439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:15.006439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:15.006439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:15.006439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:15.006439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:15.006439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:15.006439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:15.006439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:15.012301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:15.012301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:15.012301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:15.012301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:15.012301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:15.012301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:15.012301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:15.012301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:15.012301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:15.012301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:15.012301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:15.012301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:15.012301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:15.012301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:15.012301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:15.012301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:15.012301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:15.012301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:15.013528 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:15.013528 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:15.013528 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:15.013528 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:15.013528 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:15.013528 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:15.013528 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:15.013528 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:15.013528 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 01 20:33:15.013528 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:15.013528 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 303/582] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:33:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:15.013528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03a5f3-0035-4fb6-8cc3-4ab8c817a08e None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:15.078384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b447d3-6f05-438a-b8e8-3601fba3d372 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:15.099751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:15.109872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03a5f3-0035-4fb6-8cc3-4ab8c817a08e None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:15.113410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d35c46-0c2b-416e-b6e6-696ce53fcedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:15.114041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d35c46-0c2b-416e-b6e6-696ce53fcedc None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:15.114319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d35c46-0c2b-416e-b6e6-696ce53fcedc None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:15.114613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d35c46-0c2b-416e-b6e6-696ce53fcedc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:15.124411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:15.125478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d35c46-0c2b-416e-b6e6-696ce53fcedc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:15.127170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d35c46-0c2b-416e-b6e6-696ce53fcedc None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:15.141282 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-97d35c46-0c2b-416e-b6e6-696ce53fcedc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:15.141282 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:15.141282 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:15.141282 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:15.141282 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:15.141282 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:15.141282 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:15.141282 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:15.141282 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:15.141282 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:15.141282 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:15.141282 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:15.141282 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:15.141282 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:15.141282 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:15.141282 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:15.142921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:15.142921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:15.142921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:15.142921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:15.142921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:15.142921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:15.142921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:15.142921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:15.142921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:15.142921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:15.142921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:15.142921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:15.142921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:15.142921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:15.142921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:15.142921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:15.142921 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:15.144183 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:15.144183 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:15.144183 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:15.144183 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 280/583] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:15.144183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d35c46-0c2b-416e-b6e6-696ce53fcedc None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:15.146878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c78334-02ee-4b0c-ac9f-e373c77f443e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:15.147622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c78334-02ee-4b0c-ac9f-e373c77f443e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:15.147751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c78334-02ee-4b0c-ac9f-e373c77f443e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:15.148016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c78334-02ee-4b0c-ac9f-e373c77f443e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:15.148740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c78334-02ee-4b0c-ac9f-e373c77f443e None admin] RBAC: Adding query filter params (role.id=ef7bd7293369475a9e7b3444fac30cda, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=c4dc51e2b4474d6ba9fd5d2bc3702826) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:15.149047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c78334-02ee-4b0c-ac9f-e373c77f443e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:15.150448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c78334-02ee-4b0c-ac9f-e373c77f443e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:15.155810 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 304/584] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:33:15 2022] GET /identity/v3/role_assignments?user.id=c4dc51e2b4474d6ba9fd5d2bc3702826&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=ef7bd7293369475a9e7b3444fac30cda => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:15.156543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c78334-02ee-4b0c-ac9f-e373c77f443e None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:15.251938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d35c46-0c2b-416e-b6e6-696ce53fcedc None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:15.279283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c78334-02ee-4b0c-ac9f-e373c77f443e None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:15.286815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f94c7e-d8ae-4a4a-bce2-884fe06eff2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:15.287436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f94c7e-d8ae-4a4a-bce2-884fe06eff2d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:15.287716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f94c7e-d8ae-4a4a-bce2-884fe06eff2d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:15.287992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f94c7e-d8ae-4a4a-bce2-884fe06eff2d None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:15.288646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f94c7e-d8ae-4a4a-bce2-884fe06eff2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:15.288951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f94c7e-d8ae-4a4a-bce2-884fe06eff2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:15.290337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f94c7e-d8ae-4a4a-bce2-884fe06eff2d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:15.303665 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 281/585] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:15.304292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f94c7e-d8ae-4a4a-bce2-884fe06eff2d None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:15.325014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead16697-e942-4c40-b3ca-0f4150e48fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:15.325694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead16697-e942-4c40-b3ca-0f4150e48fea None admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:15.325977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead16697-e942-4c40-b3ca-0f4150e48fea None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:15.326255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead16697-e942-4c40-b3ca-0f4150e48fea None admin] PATH_INFO: `/v3/projects/b7875946723844198353bbfcf22516fb/users/7ab975dafb564aa2b357311c4a384d53/roles/ef7bd7293369475a9e7b3444fac30cda` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:15.343597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead16697-e942-4c40-b3ca-0f4150e48fea None admin] RBAC: Authorizing `identity:create_grant(project_id=b7875946723844198353bbfcf22516fb, user_id=7ab975dafb564aa2b357311c4a384d53, role_id=ef7bd7293369475a9e7b3444fac30cda)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:15.345138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead16697-e942-4c40-b3ca-0f4150e48fea None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:15.346598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:15.366525 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 305/586] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:33:15 2022] PUT /identity/v3/projects/b7875946723844198353bbfcf22516fb/users/7ab975dafb564aa2b357311c4a384d53/roles/ef7bd7293369475a9e7b3444fac30cda => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:33:15.366525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead16697-e942-4c40-b3ca-0f4150e48fea None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:15.405760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:15.439480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f94c7e-d8ae-4a4a-bce2-884fe06eff2d None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:15.491604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347ba81-db9c-4830-a721-414e71540da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:15.492592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347ba81-db9c-4830-a721-414e71540da1 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:15.493024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347ba81-db9c-4830-a721-414e71540da1 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:15.493759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347ba81-db9c-4830-a721-414e71540da1 None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:15.494913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347ba81-db9c-4830-a721-414e71540da1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:15.495317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347ba81-db9c-4830-a721-414e71540da1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:15.497498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347ba81-db9c-4830-a721-414e71540da1 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:15.499972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead16697-e942-4c40-b3ca-0f4150e48fea None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:15.526537 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 282/587] 10.222.0.17 () {56 vars in 1137 bytes} [Sat Oct 1 20:33:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:15.527659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347ba81-db9c-4830-a721-414e71540da1 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:15.549324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d887085-e883-4f56-a8cf-3e95c51bbde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:15.549944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d887085-e883-4f56-a8cf-3e95c51bbde1 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:15.550230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d887085-e883-4f56-a8cf-3e95c51bbde1 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:15.550529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d887085-e883-4f56-a8cf-3e95c51bbde1 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:15.551213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d887085-e883-4f56-a8cf-3e95c51bbde1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:15.551519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d887085-e883-4f56-a8cf-3e95c51bbde1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:15.553550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d887085-e883-4f56-a8cf-3e95c51bbde1 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:15.571306 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 306/588] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:15.572114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d887085-e883-4f56-a8cf-3e95c51bbde1 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:15.639796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:15.650532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347ba81-db9c-4830-a721-414e71540da1 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:15.679885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:15.690969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6080d54-ef60-47dd-bb32-4889e7708a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:15.691714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6080d54-ef60-47dd-bb32-4889e7708a44 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:15.692091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6080d54-ef60-47dd-bb32-4889e7708a44 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:15.692432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6080d54-ef60-47dd-bb32-4889e7708a44 None admin] PATH_INFO: `/v3/users/glance` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:15.697288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d887085-e883-4f56-a8cf-3e95c51bbde1 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:15.700628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6080d54-ef60-47dd-bb32-4889e7708a44 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:15.702143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6080d54-ef60-47dd-bb32-4889e7708a44 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:15.714258 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-c6080d54-ef60-47dd-bb32-4889e7708a44 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 01 20:33:15.714258 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:15.714258 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:15.714258 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:15.714258 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:15.714258 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:15.714258 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:15.714258 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:15.714258 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:15.714258 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:15.714258 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:15.714258 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:15.714258 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:15.714258 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:15.714258 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:15.714258 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:15.715426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:15.715426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:15.715426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:15.715426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:15.715426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:15.715426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:15.715426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:15.715426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:15.715426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:15.715426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:15.715426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:15.715426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:15.715426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:15.715426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:15.715426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:15.715426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:15.715426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:15.715426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:15.716577 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:15.716577 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:15.716577 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:15.716577 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:15.716577 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:15.716577 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:15.716577 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:15.716577 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:15.716577 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 01 20:33:15.716577 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:15.717915 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 283/589] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:33:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:15.723770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6080d54-ef60-47dd-bb32-4889e7708a44 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:15.745811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46637ab2-a766-4dc7-9aaf-57447c752151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:15.746617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46637ab2-a766-4dc7-9aaf-57447c752151 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:15.746972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46637ab2-a766-4dc7-9aaf-57447c752151 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:15.747502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46637ab2-a766-4dc7-9aaf-57447c752151 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:15.761688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46637ab2-a766-4dc7-9aaf-57447c752151 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:15.763763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46637ab2-a766-4dc7-9aaf-57447c752151 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:15.782409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-46637ab2-a766-4dc7-9aaf-57447c752151 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:15.782409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:15.782409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:15.782409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:15.782409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:15.782409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:15.782409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:15.782409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:15.782409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:15.782409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:15.782409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:15.782409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:15.782409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:15.782409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:15.782409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:15.782409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:15.783608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:15.783608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:15.783608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:15.783608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:15.783608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:15.783608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:15.783608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:15.783608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:15.783608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:15.783608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:15.783608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:15.783608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:15.783608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:15.783608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:15.783608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:15.783608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:15.783608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:15.784704 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:15.784704 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:15.784704 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:15.785728 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 307/590] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:15.786831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46637ab2-a766-4dc7-9aaf-57447c752151 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:15.853605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6080d54-ef60-47dd-bb32-4889e7708a44 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:15.900113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5b903-50ad-413b-814e-0c8481626d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:15.901001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5b903-50ad-413b-814e-0c8481626d8f None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:15.901423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5b903-50ad-413b-814e-0c8481626d8f None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:15.901838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5b903-50ad-413b-814e-0c8481626d8f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:15.912994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5b903-50ad-413b-814e-0c8481626d8f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:15.915609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5b903-50ad-413b-814e-0c8481626d8f None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:15.917338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46637ab2-a766-4dc7-9aaf-57447c752151 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:15.932242 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-a3d5b903-50ad-413b-814e-0c8481626d8f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 01 20:33:15.932242 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:15.932242 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:15.932242 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:15.932242 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:15.932242 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:15.932242 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:15.932242 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:15.932242 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:15.932242 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:15.932242 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:15.932242 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:15.932242 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:15.932242 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:15.932242 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:15.932242 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:15.933774 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:15.933774 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:15.933774 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:15.933774 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:15.933774 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:15.933774 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:15.933774 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:15.933774 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:15.933774 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:15.933774 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:15.933774 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:15.933774 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:15.933774 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:15.933774 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:15.933774 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:15.933774 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:15.933774 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:15.933774 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:15.934959 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:15.934959 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:15.934959 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:15.934959 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:15.934959 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:15.934959 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:15.934959 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:15.934959 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:15.934959 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 01 20:33:15.934959 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:15.937867 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 284/591] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:15.939905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5b903-50ad-413b-814e-0c8481626d8f None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:15.962295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:15.974365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c4d57-75c3-495a-9063-649d4b9a8c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:15.975664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c4d57-75c3-495a-9063-649d4b9a8c3e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:15.976056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c4d57-75c3-495a-9063-649d4b9a8c3e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:15.976430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c4d57-75c3-495a-9063-649d4b9a8c3e None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:15.977275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c4d57-75c3-495a-9063-649d4b9a8c3e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:15.977880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c4d57-75c3-495a-9063-649d4b9a8c3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:15.980109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c4d57-75c3-495a-9063-649d4b9a8c3e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:15.994181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:16.011313 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 308/592] 10.222.0.17 () {56 vars in 1137 bytes} [Sat Oct 1 20:33:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:16.012401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c4d57-75c3-495a-9063-649d4b9a8c3e None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:16.078152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5b903-50ad-413b-814e-0c8481626d8f None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:16.113884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5fed73-9afb-448f-9add-fa3da4d0444e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:16.114817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fed73-9afb-448f-9add-fa3da4d0444e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:16.115234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fed73-9afb-448f-9add-fa3da4d0444e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:16.115644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fed73-9afb-448f-9add-fa3da4d0444e None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:16.116413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5fed73-9afb-448f-9add-fa3da4d0444e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:16.116854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5fed73-9afb-448f-9add-fa3da4d0444e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:16.118369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5fed73-9afb-448f-9add-fa3da4d0444e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:16.132901 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 285/593] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:16.133712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5fed73-9afb-448f-9add-fa3da4d0444e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:16.149819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c4d57-75c3-495a-9063-649d4b9a8c3e None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:16.199148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88548860-c527-4d47-84ed-a54ab37e9f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:16.200671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88548860-c527-4d47-84ed-a54ab37e9f8f None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:16.201140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88548860-c527-4d47-84ed-a54ab37e9f8f None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:16.201846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88548860-c527-4d47-84ed-a54ab37e9f8f None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:16.202825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88548860-c527-4d47-84ed-a54ab37e9f8f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:16.203332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88548860-c527-4d47-84ed-a54ab37e9f8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:16.205938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88548860-c527-4d47-84ed-a54ab37e9f8f None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:16.210003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:16.232564 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 309/594] 10.222.0.17 () {56 vars in 1139 bytes} [Sat Oct 1 20:33:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:16.234114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88548860-c527-4d47-84ed-a54ab37e9f8f None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:16.248915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5fed73-9afb-448f-9add-fa3da4d0444e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:16.293649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad4276-e11d-4553-884b-99489a6f62d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:16.295272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad4276-e11d-4553-884b-99489a6f62d7 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:16.295738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad4276-e11d-4553-884b-99489a6f62d7 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:16.296188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad4276-e11d-4553-884b-99489a6f62d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:16.308713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:16.311888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad4276-e11d-4553-884b-99489a6f62d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:16.314853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad4276-e11d-4553-884b-99489a6f62d7 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:16.336517 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-7cad4276-e11d-4553-884b-99489a6f62d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:16.336517 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:16.336517 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:16.336517 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:16.336517 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:16.336517 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:16.336517 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:16.336517 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:16.336517 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:16.336517 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:16.336517 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:16.336517 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:16.336517 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:16.336517 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:16.336517 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:16.336517 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:16.337508 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:16.337508 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:16.337508 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:16.337508 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:16.337508 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:16.337508 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:16.337508 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:16.337508 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:16.337508 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:16.337508 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:16.337508 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:16.337508 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:16.337508 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:16.337508 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:16.337508 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:16.337508 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:16.337508 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:16.338406 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:16.338406 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:16.338406 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:16.339734 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 286/595] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:16.340520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad4276-e11d-4553-884b-99489a6f62d7 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:16.361288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88548860-c527-4d47-84ed-a54ab37e9f8f None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:16.398151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21899606-7c5c-4bd1-a80f-22506b5a252f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:16.398750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21899606-7c5c-4bd1-a80f-22506b5a252f None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:16.399028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21899606-7c5c-4bd1-a80f-22506b5a252f None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:16.399297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21899606-7c5c-4bd1-a80f-22506b5a252f None admin] PATH_INFO: `/v3/projects/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:16.406404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21899606-7c5c-4bd1-a80f-22506b5a252f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:16.408517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21899606-7c5c-4bd1-a80f-22506b5a252f None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:16.420133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-21899606-7c5c-4bd1-a80f-22506b5a252f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:16.420133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:16.420133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:16.420133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:16.420133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:16.420133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:16.420133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:16.420133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:16.420133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:16.420133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:16.420133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:16.420133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:16.420133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:16.420133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:16.420133 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:16.421043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:16.421043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:16.421043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:16.421043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:16.421043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:16.421043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:16.421043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:16.421043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:16.421043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:16.421043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:16.421043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:16.421043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:16.421043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:16.421043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:16.421043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:16.421043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:16.421043 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:16.421936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:16.421936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:16.421936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:16.421936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:16.421936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:16.421936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:16.421936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:16.421936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:16.421936 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 310/596] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:16.422414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21899606-7c5c-4bd1-a80f-22506b5a252f None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:16.482547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad4276-e11d-4553-884b-99489a6f62d7 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:16.507493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:16.520284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fc160-4a1c-4c1a-9b28-384582ee6407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:16.521239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fc160-4a1c-4c1a-9b28-384582ee6407 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:16.521745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fc160-4a1c-4c1a-9b28-384582ee6407 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:16.522245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fc160-4a1c-4c1a-9b28-384582ee6407 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:16.529946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21899606-7c5c-4bd1-a80f-22506b5a252f None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:16.536567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fc160-4a1c-4c1a-9b28-384582ee6407 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:16.538612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fc160-4a1c-4c1a-9b28-384582ee6407 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:16.547790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:16.556450 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-e44fc160-4a1c-4c1a-9b28-384582ee6407 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:16.556450 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:16.556450 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:16.556450 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:16.556450 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:16.556450 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:16.556450 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:16.556450 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:16.556450 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:16.556450 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:16.556450 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:16.556450 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:16.556450 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:16.556450 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:16.556450 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:16.556450 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:16.557395 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:16.557395 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:16.557395 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:16.557395 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:16.557395 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:16.557395 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:16.557395 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:16.557395 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:16.557395 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:16.557395 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:16.557395 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:16.557395 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:16.557395 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:16.557395 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:16.557395 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:16.557395 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:16.557395 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:16.558389 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:16.558389 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:16.558389 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:16.560195 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 287/597] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:16.561245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fc160-4a1c-4c1a-9b28-384582ee6407 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:16.569205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c99cd-e7e6-4cde-917f-625e22a0fdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:16.569812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c99cd-e7e6-4cde-917f-625e22a0fdf5 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:16.570139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c99cd-e7e6-4cde-917f-625e22a0fdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:16.570412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c99cd-e7e6-4cde-917f-625e22a0fdf5 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:16.571489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c99cd-e7e6-4cde-917f-625e22a0fdf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:16.571790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c99cd-e7e6-4cde-917f-625e22a0fdf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:16.573245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c99cd-e7e6-4cde-917f-625e22a0fdf5 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:16.586541 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 311/598] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:16.587244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c99cd-e7e6-4cde-917f-625e22a0fdf5 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:16.679003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fc160-4a1c-4c1a-9b28-384582ee6407 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:16.693292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c99cd-e7e6-4cde-917f-625e22a0fdf5 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:16.718288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5edbcf-75d9-4a67-8d63-c1eebf517f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:16.719192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5edbcf-75d9-4a67-8d63-c1eebf517f0e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:16.719476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5edbcf-75d9-4a67-8d63-c1eebf517f0e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:16.719751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5edbcf-75d9-4a67-8d63-c1eebf517f0e None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:16.720765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5edbcf-75d9-4a67-8d63-c1eebf517f0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:16.721522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5edbcf-75d9-4a67-8d63-c1eebf517f0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:16.723520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5edbcf-75d9-4a67-8d63-c1eebf517f0e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:16.729699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6587fa2-bc98-4d82-9590-ec4599950ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:16.730386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6587fa2-bc98-4d82-9590-ec4599950ec9 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:16.730761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6587fa2-bc98-4d82-9590-ec4599950ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:16.731172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6587fa2-bc98-4d82-9590-ec4599950ec9 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:16.731875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6587fa2-bc98-4d82-9590-ec4599950ec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:16.732293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6587fa2-bc98-4d82-9590-ec4599950ec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:16.733703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6587fa2-bc98-4d82-9590-ec4599950ec9 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:16.739449 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 288/599] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:16.740302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5edbcf-75d9-4a67-8d63-c1eebf517f0e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:16.747058 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 312/600] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:16.747784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6587fa2-bc98-4d82-9590-ec4599950ec9 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:16.786786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:16.789938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:16.859452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6587fa2-bc98-4d82-9590-ec4599950ec9 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:16.862124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5edbcf-75d9-4a67-8d63-c1eebf517f0e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:16.896532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c4fc6-1445-4c1f-bc53-9893c5bc240e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:16.897187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c4fc6-1445-4c1f-bc53-9893c5bc240e None admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:16.897513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c4fc6-1445-4c1f-bc53-9893c5bc240e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:16.897787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c4fc6-1445-4c1f-bc53-9893c5bc240e None admin] PATH_INFO: `/v3/projects/b7875946723844198353bbfcf22516fb/users/197a0ae6b71f4635a069ac0238e017c9/roles/ef7bd7293369475a9e7b3444fac30cda` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:16.899808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ccccb-2572-4dcb-91e6-7ed85e7c26d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:16.900699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ccccb-2572-4dcb-91e6-7ed85e7c26d1 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:16.901236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ccccb-2572-4dcb-91e6-7ed85e7c26d1 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:16.901666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ccccb-2572-4dcb-91e6-7ed85e7c26d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:16.909458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72ccccb-2572-4dcb-91e6-7ed85e7c26d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:16.911353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72ccccb-2572-4dcb-91e6-7ed85e7c26d1 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:16.913947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c4fc6-1445-4c1f-bc53-9893c5bc240e None admin] RBAC: Authorizing `identity:create_grant(project_id=b7875946723844198353bbfcf22516fb, user_id=197a0ae6b71f4635a069ac0238e017c9, role_id=ef7bd7293369475a9e7b3444fac30cda)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:16.915852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c4fc6-1445-4c1f-bc53-9893c5bc240e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:16.925228 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-c72ccccb-2572-4dcb-91e6-7ed85e7c26d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:16.925228 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:16.925228 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:16.925228 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:16.925228 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:16.925228 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:16.925228 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:16.925228 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:16.925228 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:16.925228 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:16.925228 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:16.925228 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:16.925228 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:16.925228 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:16.925228 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:16.926349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:16.926349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:16.926349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:16.926349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:16.926349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:16.926349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:16.926349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:16.926349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:16.926349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:16.926349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:16.926349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:16.926349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:16.926349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:16.926349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:16.926349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:16.926349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:16.926349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:16.927309 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:16.927309 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:16.927309 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:16.927309 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:16.927309 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:16.927309 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:16.927309 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:16.927309 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:16.929668 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 289/601] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:16.930521 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 313/602] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:33:16 2022] PUT /identity/v3/projects/b7875946723844198353bbfcf22516fb/users/197a0ae6b71f4635a069ac0238e017c9/roles/ef7bd7293369475a9e7b3444fac30cda => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:33:16.930889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ccccb-2572-4dcb-91e6-7ed85e7c26d1 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:16.934920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c4fc6-1445-4c1f-bc53-9893c5bc240e None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:17.050045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:17.068251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c4fc6-1445-4c1f-bc53-9893c5bc240e None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:17.115927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e259c-55c5-4bb4-839b-729a0e08ff95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:17.116707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e259c-55c5-4bb4-839b-729a0e08ff95 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:17.117082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e259c-55c5-4bb4-839b-729a0e08ff95 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:17.117447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e259c-55c5-4bb4-839b-729a0e08ff95 None admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:17.118245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e259c-55c5-4bb4-839b-729a0e08ff95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:17.118689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e259c-55c5-4bb4-839b-729a0e08ff95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:17.121417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e259c-55c5-4bb4-839b-729a0e08ff95 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:17.145217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:17.146381 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 314/603] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:17.152671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e259c-55c5-4bb4-839b-729a0e08ff95 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:17.158237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ccccb-2572-4dcb-91e6-7ed85e7c26d1 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:17.211547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96024fe7-2ff8-4350-93b1-98d1af03e40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:17.212325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96024fe7-2ff8-4350-93b1-98d1af03e40a None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:17.212692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96024fe7-2ff8-4350-93b1-98d1af03e40a None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:17.213057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96024fe7-2ff8-4350-93b1-98d1af03e40a None admin] PATH_INFO: `/v3/projects/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:17.224350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96024fe7-2ff8-4350-93b1-98d1af03e40a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:17.225954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96024fe7-2ff8-4350-93b1-98d1af03e40a None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:17.244648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-96024fe7-2ff8-4350-93b1-98d1af03e40a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:17.244648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:17.244648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:17.244648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:17.244648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:17.244648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:17.244648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:17.244648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:17.244648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:17.244648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:17.244648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:17.244648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:17.244648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:17.244648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:17.244648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:17.245798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:17.245798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:17.245798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:17.245798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:17.245798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:17.245798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:17.245798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:17.245798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:17.245798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:17.245798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:17.245798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:17.245798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:17.245798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:17.245798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:17.245798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:17.245798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:17.245798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:17.246943 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:17.246943 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:17.246943 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:17.246943 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:17.246943 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:17.246943 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:17.246943 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:17.246943 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:17.249012 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 290/604] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:17.263118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96024fe7-2ff8-4350-93b1-98d1af03e40a None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:17.313430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e259c-55c5-4bb4-839b-729a0e08ff95 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:17.352751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973b4d5-e2e6-4a9c-94e9-6bfb4c4a0f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:17.353459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973b4d5-e2e6-4a9c-94e9-6bfb4c4a0f02 None admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:17.353740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973b4d5-e2e6-4a9c-94e9-6bfb4c4a0f02 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:17.354015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973b4d5-e2e6-4a9c-94e9-6bfb4c4a0f02 None admin] PATH_INFO: `/v3/projects/b7875946723844198353bbfcf22516fb/users/e2927203ce394fabab76dd52a68c2436/roles/ef7bd7293369475a9e7b3444fac30cda` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:17.385021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7973b4d5-e2e6-4a9c-94e9-6bfb4c4a0f02 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7875946723844198353bbfcf22516fb, user_id=e2927203ce394fabab76dd52a68c2436, role_id=ef7bd7293369475a9e7b3444fac30cda)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:17.386481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7973b4d5-e2e6-4a9c-94e9-6bfb4c4a0f02 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:17.390577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:17.409869 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 315/605] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:33:17 2022] PUT /identity/v3/projects/b7875946723844198353bbfcf22516fb/users/e2927203ce394fabab76dd52a68c2436/roles/ef7bd7293369475a9e7b3444fac30cda => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:33:17.448109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96024fe7-2ff8-4350-93b1-98d1af03e40a None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:17.488127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90257e29-eb3b-4e55-a251-cfe28d9b1d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:17.488713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90257e29-eb3b-4e55-a251-cfe28d9b1d91 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:17.488999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90257e29-eb3b-4e55-a251-cfe28d9b1d91 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:17.489319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90257e29-eb3b-4e55-a251-cfe28d9b1d91 None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:17.490088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90257e29-eb3b-4e55-a251-cfe28d9b1d91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:17.490404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90257e29-eb3b-4e55-a251-cfe28d9b1d91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:17.492085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90257e29-eb3b-4e55-a251-cfe28d9b1d91 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:17.506764 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 291/606] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:17.515172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973b4d5-e2e6-4a9c-94e9-6bfb4c4a0f02 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:17.642365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973b4d5-e2e6-4a9c-94e9-6bfb4c4a0f02 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:17.678764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca596ee7-8ac7-4332-b9ec-5fb9301b7123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:17.679448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca596ee7-8ac7-4332-b9ec-5fb9301b7123 None admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:17.679740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca596ee7-8ac7-4332-b9ec-5fb9301b7123 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:17.680055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca596ee7-8ac7-4332-b9ec-5fb9301b7123 None admin] PATH_INFO: `/v3/projects/b7875946723844198353bbfcf22516fb/users/e17a026f305248878f18eec35e1e02dd/roles/ef7bd7293369475a9e7b3444fac30cda` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:17.704584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca596ee7-8ac7-4332-b9ec-5fb9301b7123 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7875946723844198353bbfcf22516fb, user_id=e17a026f305248878f18eec35e1e02dd, role_id=ef7bd7293369475a9e7b3444fac30cda)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:17.706162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca596ee7-8ac7-4332-b9ec-5fb9301b7123 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:17.732151 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 316/607] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:33:17 2022] PUT /identity/v3/projects/b7875946723844198353bbfcf22516fb/users/e17a026f305248878f18eec35e1e02dd/roles/ef7bd7293369475a9e7b3444fac30cda => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:33:17.885021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b131c32-cf5f-4307-a308-6bfb4ee59b97 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:17.885196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b131c32-cf5f-4307-a308-6bfb4ee59b97 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:17.885488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b131c32-cf5f-4307-a308-6bfb4ee59b97 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:17.886995 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 292/608] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:17.892797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d2a3a-7509-4c46-ab46-e102052537c1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:17.893087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d2a3a-7509-4c46-ab46-e102052537c1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:17.893340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d2a3a-7509-4c46-ab46-e102052537c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:17.914941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d29d2a3a-7509-4c46-ab46-e102052537c1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:17.916156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d29d2a3a-7509-4c46-ab46-e102052537c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:17.976266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:18.020152 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 317/609] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:18.031226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b353b-8d46-4ead-892f-be37e1d29113 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:18.031510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b353b-8d46-4ead-892f-be37e1d29113 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:18.031783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b353b-8d46-4ead-892f-be37e1d29113 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:18.033526 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 293/610] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:18.046554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d799ee-8a19-4d70-81af-16ef91f35702 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:18.046554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d799ee-8a19-4d70-81af-16ef91f35702 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:18.046554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d799ee-8a19-4d70-81af-16ef91f35702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:18.064076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-08d799ee-8a19-4d70-81af-16ef91f35702 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:18.065269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-08d799ee-8a19-4d70-81af-16ef91f35702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:18.136531 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 318/611] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:18.382424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b353b-8d46-4ead-892f-be37e1d29113 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:18.406435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:18.521610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b353b-8d46-4ead-892f-be37e1d29113 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:18.565948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d4f27-e52d-4d48-a43e-3ddf99b442ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:18.566644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d4f27-e52d-4d48-a43e-3ddf99b442ca None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:18.566921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d4f27-e52d-4d48-a43e-3ddf99b442ca None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:18.567186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d4f27-e52d-4d48-a43e-3ddf99b442ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:18.576586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d799ee-8a19-4d70-81af-16ef91f35702 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:18.582732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d4f27-e52d-4d48-a43e-3ddf99b442ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:18.587493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-169d4f27-e52d-4d48-a43e-3ddf99b442ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:18.587493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:18.587493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:18.587493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:18.587493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:18.587493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:18.587493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:18.587493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:18.587493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:18.587493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:18.587493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:18.587493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:18.587493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:18.587493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:18.587493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:18.587493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:18.588422 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:18.588422 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:18.588422 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:18.588422 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:18.588422 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:18.588422 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:18.588422 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:18.588422 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:18.588422 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:18.588422 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:18.588422 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:18.588422 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:18.588422 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:18.588422 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:18.588422 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:18.588422 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:18.589869 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:18.589869 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:18.589869 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:18.589869 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:18.589869 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:18.589869 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:18.589869 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:18.589869 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:18.589869 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:18.589869 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:18.589869 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:18.589869 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:18.589869 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 294/612] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:18.595724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d4f27-e52d-4d48-a43e-3ddf99b442ca None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:18.708963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d799ee-8a19-4d70-81af-16ef91f35702 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:18.717569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:18.744806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:18.752381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c022b07-89e6-4fb2-8221-9c4d11aefa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:18.753859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c022b07-89e6-4fb2-8221-9c4d11aefa36 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:18.754137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c022b07-89e6-4fb2-8221-9c4d11aefa36 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:18.754409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c022b07-89e6-4fb2-8221-9c4d11aefa36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:18.772463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c022b07-89e6-4fb2-8221-9c4d11aefa36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:18.777809 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-1c022b07-89e6-4fb2-8221-9c4d11aefa36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:18.777809 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:18.777809 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:18.777809 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:18.777809 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:18.777809 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:18.777809 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:18.777809 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:18.777809 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:18.777809 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:18.777809 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:18.777809 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:18.777809 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:18.777809 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:18.777809 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:18.777809 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:18.778782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:18.778782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:18.778782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:18.778782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:18.778782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:18.778782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:18.778782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:18.778782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:18.778782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:18.778782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:18.778782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:18.778782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:18.778782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:18.778782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:18.778782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:18.778782 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:18.779612 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:18.779612 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:18.779612 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:18.779612 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:18.779612 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:18.779612 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:18.779612 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:18.779612 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:18.779612 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:18.779612 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:18.779612 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:18.779612 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:18.782692 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 319/613] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:18.791465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c022b07-89e6-4fb2-8221-9c4d11aefa36 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:18.812677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d4f27-e52d-4d48-a43e-3ddf99b442ca None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:18.904679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c022b07-89e6-4fb2-8221-9c4d11aefa36 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:18.911729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd62761-7b18-4056-8b75-920a96c6c66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:18.912342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd62761-7b18-4056-8b75-920a96c6c66c None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:18.912623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd62761-7b18-4056-8b75-920a96c6c66c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:18.912895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd62761-7b18-4056-8b75-920a96c6c66c None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:18.913571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd62761-7b18-4056-8b75-920a96c6c66c None admin] RBAC: Adding query filter params (name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:18.913875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd62761-7b18-4056-8b75-920a96c6c66c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:18.915275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd62761-7b18-4056-8b75-920a96c6c66c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:18.933410 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 295/614] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:33:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:18.939016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd62761-7b18-4056-8b75-920a96c6c66c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:18.946299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce9290-542c-46d8-b2d9-3c99538f617e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:18.947269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce9290-542c-46d8-b2d9-3c99538f617e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:18.953027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce9290-542c-46d8-b2d9-3c99538f617e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:18.953541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce9290-542c-46d8-b2d9-3c99538f617e None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:18.954963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce9290-542c-46d8-b2d9-3c99538f617e None admin] RBAC: Adding query filter params (name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:18.955470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce9290-542c-46d8-b2d9-3c99538f617e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:18.957027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce9290-542c-46d8-b2d9-3c99538f617e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:18.973711 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 320/615] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:33:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:18.982532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce9290-542c-46d8-b2d9-3c99538f617e None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:19.040236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:19.143642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd62761-7b18-4056-8b75-920a96c6c66c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:19.158469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce9290-542c-46d8-b2d9-3c99538f617e None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:19.205853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18aee39-b148-411f-b786-ad8ac4ae33ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:19.206508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aee39-b148-411f-b786-ad8ac4ae33ca None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:19.206795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aee39-b148-411f-b786-ad8ac4ae33ca None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:19.207069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18aee39-b148-411f-b786-ad8ac4ae33ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:19.221661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385bb87-958e-4f3c-b7ed-dc527c7fe6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:19.222302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385bb87-958e-4f3c-b7ed-dc527c7fe6d7 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:19.222613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385bb87-958e-4f3c-b7ed-dc527c7fe6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:19.222950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385bb87-958e-4f3c-b7ed-dc527c7fe6d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:19.224006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:19.234011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18aee39-b148-411f-b786-ad8ac4ae33ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:19.235841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18aee39-b148-411f-b786-ad8ac4ae33ca None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:19.243877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385bb87-958e-4f3c-b7ed-dc527c7fe6d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:19.245316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385bb87-958e-4f3c-b7ed-dc527c7fe6d7 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:19.262791 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-a18aee39-b148-411f-b786-ad8ac4ae33ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:19.262791 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:19.262791 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:19.262791 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:19.262791 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:19.262791 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:19.262791 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:19.262791 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:19.262791 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:19.262791 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:19.262791 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:19.262791 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:19.262791 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:19.262791 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:19.262791 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:19.262791 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:19.263827 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:19.263827 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:19.263827 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:19.263827 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:19.263827 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:19.263827 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:19.263827 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:19.263827 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:19.263827 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:19.263827 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:19.263827 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:19.263827 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:19.263827 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:19.263827 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:19.263827 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:19.263827 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:19.263827 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:19.264716 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:19.264716 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:19.264716 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:19.267214 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 296/616] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:19.271405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18aee39-b148-411f-b786-ad8ac4ae33ca None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:19.274596 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-b385bb87-958e-4f3c-b7ed-dc527c7fe6d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:19.274596 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:19.274596 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:19.274596 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:19.274596 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:19.274596 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:19.274596 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:19.274596 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:19.274596 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:19.274596 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:19.274596 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:19.274596 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:19.274596 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:19.274596 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:19.274596 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:19.274596 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:19.275574 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:19.275574 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:19.275574 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:19.275574 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:19.275574 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:19.275574 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:19.275574 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:19.275574 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:19.275574 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:19.275574 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:19.275574 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:19.275574 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:19.275574 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:19.275574 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:19.275574 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:19.275574 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:19.275574 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:19.277144 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:19.277144 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:19.277144 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:19.277144 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 321/617] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:19.284277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385bb87-958e-4f3c-b7ed-dc527c7fe6d7 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:19.429022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:19.464276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18aee39-b148-411f-b786-ad8ac4ae33ca None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:19.468311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385bb87-958e-4f3c-b7ed-dc527c7fe6d7 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:19.530091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ef8ac-133b-4b40-bb26-eb12ba3e694e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:19.530643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ef8ac-133b-4b40-bb26-eb12ba3e694e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:19.530922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ef8ac-133b-4b40-bb26-eb12ba3e694e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:19.531193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ef8ac-133b-4b40-bb26-eb12ba3e694e None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:19.531834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ef8ac-133b-4b40-bb26-eb12ba3e694e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:19.532133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ef8ac-133b-4b40-bb26-eb12ba3e694e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:19.534123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ef8ac-133b-4b40-bb26-eb12ba3e694e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:19.537881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a799bf-d24a-41f2-abd5-b7063943ef52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:19.538511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a799bf-d24a-41f2-abd5-b7063943ef52 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:19.544963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a799bf-d24a-41f2-abd5-b7063943ef52 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:19.545268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a799bf-d24a-41f2-abd5-b7063943ef52 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:19.545894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a799bf-d24a-41f2-abd5-b7063943ef52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:19.546198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a799bf-d24a-41f2-abd5-b7063943ef52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:19.547647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a799bf-d24a-41f2-abd5-b7063943ef52 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:19.558540 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 297/618] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:19.565865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ef8ac-133b-4b40-bb26-eb12ba3e694e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:19.567836 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 322/619] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:19.576368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a799bf-d24a-41f2-abd5-b7063943ef52 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:19.675368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:19.716535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a799bf-d24a-41f2-abd5-b7063943ef52 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:19.760882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ef8ac-133b-4b40-bb26-eb12ba3e694e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:19.799590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe771df4-5940-498b-b83f-ddcd59650000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:19.800146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe771df4-5940-498b-b83f-ddcd59650000 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:19.800426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe771df4-5940-498b-b83f-ddcd59650000 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:19.800749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe771df4-5940-498b-b83f-ddcd59650000 None admin] PATH_INFO: `/v3/users/swift` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:19.807196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5cbc76-5194-46cd-9561-7f94d5fef08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:19.808124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cbc76-5194-46cd-9561-7f94d5fef08c None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:19.808438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe771df4-5940-498b-b83f-ddcd59650000 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:19.808837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cbc76-5194-46cd-9561-7f94d5fef08c None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:19.809310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cbc76-5194-46cd-9561-7f94d5fef08c None admin] PATH_INFO: `/v3/users/nova` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:19.809903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe771df4-5940-498b-b83f-ddcd59650000 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:19.823604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5cbc76-5194-46cd-9561-7f94d5fef08c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:19.825394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5cbc76-5194-46cd-9561-7f94d5fef08c None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:19.832409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:19.839534 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-fe771df4-5940-498b-b83f-ddcd59650000 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 01 20:33:19.839534 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:19.839534 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:19.839534 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:19.839534 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:19.839534 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:19.839534 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:19.839534 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:19.839534 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:19.839534 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:19.839534 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:19.839534 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:19.839534 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:19.839534 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:19.839534 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:19.839534 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:19.840497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:19.840497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:19.840497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:19.840497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:19.840497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:19.840497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:19.840497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:19.840497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:19.840497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:19.840497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:19.840497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:19.840497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:19.840497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:19.840497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:19.840497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:19.840497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:19.840497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:19.840497 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:19.841965 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:19.841965 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:19.841965 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:19.841965 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:19.841965 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:19.841965 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:19.841965 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:19.841965 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:19.841965 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 01 20:33:19.841965 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:19.844490 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 298/620] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:33:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:19.845900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d28943-69c4-4097-abd8-378ac7593956 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:19.846391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d28943-69c4-4097-abd8-378ac7593956 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:19.846893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d28943-69c4-4097-abd8-378ac7593956 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:19.849045 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 299/621] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:19.850022 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-2c5cbc76-5194-46cd-9561-7f94d5fef08c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 01 20:33:19.850022 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:19.850022 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:19.850022 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:19.850022 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:19.850022 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:19.850022 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:19.850022 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:19.850022 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:19.850022 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:19.850022 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:19.850022 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:19.850022 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:19.850022 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:19.850022 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:19.850022 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:19.850981 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:19.850981 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:19.850981 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:19.850981 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:19.850981 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:19.850981 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:19.850981 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:19.850981 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:19.850981 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:19.850981 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:19.850981 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:19.850981 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:19.850981 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:19.850981 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:19.850981 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:19.850981 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:19.850981 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:19.850981 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:19.852371 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:19.852371 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:19.852371 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:19.852371 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:19.852371 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:19.852371 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:19.852371 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:19.852371 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:19.852371 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 01 20:33:19.852371 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:19.853849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d28943-69c4-4097-abd8-378ac7593956 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:19.856390 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 323/622] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Oct 1 20:33:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:19.857867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbf745-b41c-4091-9977-05eb4927480d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:19.858347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbf745-b41c-4091-9977-05eb4927480d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:19.858874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbf745-b41c-4091-9977-05eb4927480d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:19.882733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-06bbf745-b41c-4091-9977-05eb4927480d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:19.883953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-06bbf745-b41c-4091-9977-05eb4927480d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:19.960465 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 324/623] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:19.961871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbf745-b41c-4091-9977-05eb4927480d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:20.019375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d28943-69c4-4097-abd8-378ac7593956 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:20.056544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:20.083103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbf745-b41c-4091-9977-05eb4927480d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:20.115673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:20.122536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078621aa-e3d9-409d-b499-0c846b34467c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:20.123546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078621aa-e3d9-409d-b499-0c846b34467c None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:20.124066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078621aa-e3d9-409d-b499-0c846b34467c None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:20.124552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078621aa-e3d9-409d-b499-0c846b34467c None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:20.125527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078621aa-e3d9-409d-b499-0c846b34467c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:20.126593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078621aa-e3d9-409d-b499-0c846b34467c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:20.127221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f8a70-7101-4e71-8e57-edf0b0878f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:20.127845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f8a70-7101-4e71-8e57-edf0b0878f38 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:20.128124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f8a70-7101-4e71-8e57-edf0b0878f38 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:20.128395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f8a70-7101-4e71-8e57-edf0b0878f38 None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:20.129131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f8a70-7101-4e71-8e57-edf0b0878f38 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:20.129373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078621aa-e3d9-409d-b499-0c846b34467c None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:20.129475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f8a70-7101-4e71-8e57-edf0b0878f38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:20.130874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f8a70-7101-4e71-8e57-edf0b0878f38 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:20.149902 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 325/624] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Oct 1 20:33:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:20.155040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078621aa-e3d9-409d-b499-0c846b34467c None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:20.170455 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 300/625] 10.222.0.17 () {56 vars in 1135 bytes} [Sat Oct 1 20:33:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:20.175794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f8a70-7101-4e71-8e57-edf0b0878f38 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:20.262868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078621aa-e3d9-409d-b499-0c846b34467c None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:20.311033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94282747-abc1-4dd5-8505-c689a4814c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:20.312189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94282747-abc1-4dd5-8505-c689a4814c29 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:20.313549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94282747-abc1-4dd5-8505-c689a4814c29 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:20.315282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94282747-abc1-4dd5-8505-c689a4814c29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:20.385576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:20.394612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94282747-abc1-4dd5-8505-c689a4814c29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:20.396449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94282747-abc1-4dd5-8505-c689a4814c29 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:20.410862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f8a70-7101-4e71-8e57-edf0b0878f38 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:20.463192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d78da-2646-4708-84f5-32d0dfe36296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:20.463831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d78da-2646-4708-84f5-32d0dfe36296 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:20.464111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d78da-2646-4708-84f5-32d0dfe36296 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:20.464376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d78da-2646-4708-84f5-32d0dfe36296 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:20.477768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d78da-2646-4708-84f5-32d0dfe36296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:20.479513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d78da-2646-4708-84f5-32d0dfe36296 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:20.495798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-ff9d78da-2646-4708-84f5-32d0dfe36296 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:20.495798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:20.495798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:20.495798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:20.495798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:20.495798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:20.495798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:20.495798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:20.495798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:20.495798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:20.495798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:20.495798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:20.495798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:20.495798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:20.495798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:20.495798 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:20.496792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:20.496792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:20.496792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:20.496792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:20.496792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:20.496792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:20.496792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:20.496792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:20.496792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:20.496792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:20.496792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:20.496792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:20.496792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:20.496792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:20.496792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:20.496792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:20.496792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:20.497686 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:20.497686 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:20.497686 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:20.499974 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 301/626] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:20.501197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d78da-2646-4708-84f5-32d0dfe36296 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:20.511715 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 326/627] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:33:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 01 20:33:20.512936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94282747-abc1-4dd5-8505-c689a4814c29 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:20.529593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:20.610091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:20.620173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d78da-2646-4708-84f5-32d0dfe36296 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:20.623099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94282747-abc1-4dd5-8505-c689a4814c29 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:20.656125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0149706-b770-4c0a-a691-c44f2adb534b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:20.656755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0149706-b770-4c0a-a691-c44f2adb534b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:20.657038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0149706-b770-4c0a-a691-c44f2adb534b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:20.657889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0149706-b770-4c0a-a691-c44f2adb534b None admin] PATH_INFO: `/v3/roles/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:20.660006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbb193-218e-4e95-afca-f41dc2237ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:20.660648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbb193-218e-4e95-afca-f41dc2237ac9 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:20.660884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbb193-218e-4e95-afca-f41dc2237ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:20.661153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbb193-218e-4e95-afca-f41dc2237ac9 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:20.661788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcbb193-218e-4e95-afca-f41dc2237ac9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:20.662086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcbb193-218e-4e95-afca-f41dc2237ac9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:20.663485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcbb193-218e-4e95-afca-f41dc2237ac9 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:20.669269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0149706-b770-4c0a-a691-c44f2adb534b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:20.674898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-f0149706-b770-4c0a-a691-c44f2adb534b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:20.674898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:20.674898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:20.674898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:20.674898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:20.674898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:20.674898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:20.674898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:20.674898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:20.674898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:20.674898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:20.674898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:20.674898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:20.674898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:20.674898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:20.674898 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:20.684668 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:20.684668 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:20.684668 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:20.684668 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:20.684668 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:20.684668 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:20.684668 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:20.684668 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:20.684668 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:20.684668 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:20.684668 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:20.684668 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:20.684668 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:20.684668 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:20.684668 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:20.684668 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:20.686342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:20.686342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:20.686342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:20.686342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:20.686342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:20.686342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:20.686342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:20.686342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:20.686342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:20.686342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:20.686342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:20.686342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:20.687223 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 327/628] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:20.687301 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 302/629] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:20.687371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbb193-218e-4e95-afca-f41dc2237ac9 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:20.687448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ad2f0-8bae-4dd5-91c0-5003dcbef11d None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:20.687448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ad2f0-8bae-4dd5-91c0-5003dcbef11d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:20.687448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ad2f0-8bae-4dd5-91c0-5003dcbef11d None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:20.687448 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 303/630] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:20.687448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ad2f0-8bae-4dd5-91c0-5003dcbef11d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:20.770849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:20.793183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ad2f0-8bae-4dd5-91c0-5003dcbef11d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:20.806909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbb193-218e-4e95-afca-f41dc2237ac9 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:20.827054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb442f-3cfd-4d12-8153-e1dad15b0184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:20.827680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb442f-3cfd-4d12-8153-e1dad15b0184 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:20.827961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb442f-3cfd-4d12-8153-e1dad15b0184 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:20.828230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb442f-3cfd-4d12-8153-e1dad15b0184 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:20.834794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb442f-3cfd-4d12-8153-e1dad15b0184 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:20.836245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb442f-3cfd-4d12-8153-e1dad15b0184 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:20.841035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3fad6-0c42-45c4-9788-e353f79dd538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:20.841649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3fad6-0c42-45c4-9788-e353f79dd538 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:20.841924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3fad6-0c42-45c4-9788-e353f79dd538 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:20.842269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3fad6-0c42-45c4-9788-e353f79dd538 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:20.847840 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-1dbb442f-3cfd-4d12-8153-e1dad15b0184 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:20.847840 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:20.847840 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:20.847840 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:20.847840 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:20.847840 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:20.847840 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:20.847840 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:20.847840 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:20.847840 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:20.847840 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:20.847840 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:20.847840 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:20.847840 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:20.847840 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:20.848902 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:20.848902 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:20.848902 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:20.848902 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:20.848902 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:20.848902 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:20.848902 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:20.848902 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:20.848902 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:20.848902 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:20.848902 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:20.848902 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:20.848902 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:20.848902 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:20.848902 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:20.848902 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:20.848902 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:20.849805 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:20.849805 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:20.849805 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:20.849805 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:20.849805 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:20.849805 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:20.849805 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:20.849805 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:20.849805 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 304/631] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:20.850305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead3e8b-21f3-4739-a61b-adcbec1bfa28 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:20.850676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead3e8b-21f3-4739-a61b-adcbec1bfa28 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:20.850900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead3e8b-21f3-4739-a61b-adcbec1bfa28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:20.857732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3fad6-0c42-45c4-9788-e353f79dd538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:20.859296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3fad6-0c42-45c4-9788-e353f79dd538 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:20.867770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:20.869061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9ead3e8b-21f3-4739-a61b-adcbec1bfa28 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:20.870641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9ead3e8b-21f3-4739-a61b-adcbec1bfa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:20.874216 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-d0a3fad6-0c42-45c4-9788-e353f79dd538 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:20.874216 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:20.874216 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:20.874216 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:20.874216 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:20.874216 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:20.874216 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:20.874216 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:20.874216 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:20.874216 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:20.874216 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:20.874216 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:20.874216 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:20.874216 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:20.874216 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:20.874216 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:20.875243 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:20.875243 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:20.875243 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:20.875243 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:20.875243 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:20.875243 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:20.875243 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:20.875243 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:20.875243 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:20.875243 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:20.875243 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:20.875243 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:20.875243 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:20.875243 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:20.875243 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:20.875243 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:20.875243 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:20.876713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:20.876713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:20.876713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:20.876713 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 328/632] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:20.876713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3fad6-0c42-45c4-9788-e353f79dd538 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:20.937639 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 305/633] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:20.938935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890b400-b1b5-48df-8a74-f0658c179813 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:20.939222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890b400-b1b5-48df-8a74-f0658c179813 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:20.939495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890b400-b1b5-48df-8a74-f0658c179813 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:20.940710 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 306/634] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:20.941401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890b400-b1b5-48df-8a74-f0658c179813 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:20.992792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3fad6-0c42-45c4-9788-e353f79dd538 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:21.015875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:21.033885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686cd69-714c-4930-a250-d2f18b0e8b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:21.034534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686cd69-714c-4930-a250-d2f18b0e8b04 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:21.034825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686cd69-714c-4930-a250-d2f18b0e8b04 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:21.035101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686cd69-714c-4930-a250-d2f18b0e8b04 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:21.035770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686cd69-714c-4930-a250-d2f18b0e8b04 None admin] RBAC: Adding query filter params (name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:21.036076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686cd69-714c-4930-a250-d2f18b0e8b04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:21.037437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686cd69-714c-4930-a250-d2f18b0e8b04 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:21.052836 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 329/635] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:33:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:21.052836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686cd69-714c-4930-a250-d2f18b0e8b04 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:21.058150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890b400-b1b5-48df-8a74-f0658c179813 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:21.093418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bdc552-7b21-4056-b39f-c2a70d4a7623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:21.094107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdc552-7b21-4056-b39f-c2a70d4a7623 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:21.094452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdc552-7b21-4056-b39f-c2a70d4a7623 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:21.095375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdc552-7b21-4056-b39f-c2a70d4a7623 None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:21.096050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bdc552-7b21-4056-b39f-c2a70d4a7623 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:21.096352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bdc552-7b21-4056-b39f-c2a70d4a7623 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:21.097789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bdc552-7b21-4056-b39f-c2a70d4a7623 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:21.109753 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 307/636] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:21.110971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e299f8-2f5e-4230-8e2d-63957da11b8f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:21.111251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e299f8-2f5e-4230-8e2d-63957da11b8f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:21.111518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e299f8-2f5e-4230-8e2d-63957da11b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:21.126402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:21.130878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a0e299f8-2f5e-4230-8e2d-63957da11b8f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:21.132076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a0e299f8-2f5e-4230-8e2d-63957da11b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:21.172512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686cd69-714c-4930-a250-d2f18b0e8b04 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:21.205665 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 308/637] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:21.206368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e299f8-2f5e-4230-8e2d-63957da11b8f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:21.219639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c08852-4ac8-4e94-b18e-91dabeb285b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:21.221105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c08852-4ac8-4e94-b18e-91dabeb285b9 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:21.221603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c08852-4ac8-4e94-b18e-91dabeb285b9 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:21.222128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c08852-4ac8-4e94-b18e-91dabeb285b9 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:21.223486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c08852-4ac8-4e94-b18e-91dabeb285b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:21.224016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c08852-4ac8-4e94-b18e-91dabeb285b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:21.226285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c08852-4ac8-4e94-b18e-91dabeb285b9 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:21.240309 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 330/638] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:21.241687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c08852-4ac8-4e94-b18e-91dabeb285b9 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:21.287001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:21.334883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e299f8-2f5e-4230-8e2d-63957da11b8f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:21.354366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c08852-4ac8-4e94-b18e-91dabeb285b9 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:21.369949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d73bbf-55ce-4643-af3c-6b0924447a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:21.370595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d73bbf-55ce-4643-af3c-6b0924447a4e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:21.370873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d73bbf-55ce-4643-af3c-6b0924447a4e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:21.371142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d73bbf-55ce-4643-af3c-6b0924447a4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:21.378427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:21.383956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d73bbf-55ce-4643-af3c-6b0924447a4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:21.385510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d73bbf-55ce-4643-af3c-6b0924447a4e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:21.390622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c403a-5e27-499d-b6e7-16536c6aef66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:21.391279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c403a-5e27-499d-b6e7-16536c6aef66 None admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:21.391586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c403a-5e27-499d-b6e7-16536c6aef66 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:21.391932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c403a-5e27-499d-b6e7-16536c6aef66 None admin] PATH_INFO: `/v3/projects/b7875946723844198353bbfcf22516fb/users/c4dc51e2b4474d6ba9fd5d2bc3702826/roles/ef7bd7293369475a9e7b3444fac30cda` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:21.400349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-a7d73bbf-55ce-4643-af3c-6b0924447a4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:21.400349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:21.400349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:21.400349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:21.400349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:21.400349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:21.400349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:21.400349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:21.400349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:21.400349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:21.400349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:21.400349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:21.400349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:21.400349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:21.400349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:21.400349 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:21.401329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:21.401329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:21.401329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:21.401329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:21.401329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:21.401329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:21.401329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:21.401329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:21.401329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:21.401329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:21.401329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:21.401329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:21.401329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:21.401329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:21.401329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:21.401329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:21.401329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:21.407848 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:21.407848 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:21.407848 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:21.407848 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 309/639] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:21.407848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d73bbf-55ce-4643-af3c-6b0924447a4e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:21.413506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c403a-5e27-499d-b6e7-16536c6aef66 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7875946723844198353bbfcf22516fb, user_id=c4dc51e2b4474d6ba9fd5d2bc3702826, role_id=ef7bd7293369475a9e7b3444fac30cda)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:21.415002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c403a-5e27-499d-b6e7-16536c6aef66 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:21.430158 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 331/640] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:33:21 2022] PUT /identity/v3/projects/b7875946723844198353bbfcf22516fb/users/c4dc51e2b4474d6ba9fd5d2bc3702826/roles/ef7bd7293369475a9e7b3444fac30cda => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:33:21.431596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c403a-5e27-499d-b6e7-16536c6aef66 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:21.530702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d73bbf-55ce-4643-af3c-6b0924447a4e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:21.546620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:21.572808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c403a-5e27-499d-b6e7-16536c6aef66 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:21.573353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439e86d-1940-4aa7-ae28-686a7e20f1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:21.574278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439e86d-1940-4aa7-ae28-686a7e20f1cf None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:21.574736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439e86d-1940-4aa7-ae28-686a7e20f1cf None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:21.575273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439e86d-1940-4aa7-ae28-686a7e20f1cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:21.583085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439e86d-1940-4aa7-ae28-686a7e20f1cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:21.584577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439e86d-1940-4aa7-ae28-686a7e20f1cf None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:21.598178 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-3439e86d-1940-4aa7-ae28-686a7e20f1cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:21.598178 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:21.598178 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:21.598178 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:21.598178 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:21.598178 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:21.598178 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:21.598178 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:21.598178 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:21.598178 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:21.598178 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:21.598178 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:21.598178 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:21.598178 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:21.598178 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:21.599384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:21.599384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:21.599384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:21.599384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:21.599384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:21.599384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:21.599384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:21.599384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:21.599384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:21.599384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:21.599384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:21.599384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:21.599384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:21.599384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:21.599384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:21.599384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:21.599384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:21.600284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:21.600284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:21.600284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:21.600284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:21.600284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:21.600284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:21.600284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:21.600284 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:21.602976 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 310/641] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:21.604341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439e86d-1940-4aa7-ae28-686a7e20f1cf None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:21.728921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439e86d-1940-4aa7-ae28-686a7e20f1cf None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:21.737252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bce82c-f3a5-4ece-bf1a-a8fe1a269137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:21.737885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bce82c-f3a5-4ece-bf1a-a8fe1a269137 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:21.738168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bce82c-f3a5-4ece-bf1a-a8fe1a269137 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:21.738442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bce82c-f3a5-4ece-bf1a-a8fe1a269137 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:21.750155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bce82c-f3a5-4ece-bf1a-a8fe1a269137 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:21.756469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-26bce82c-f3a5-4ece-bf1a-a8fe1a269137 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:21.756469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:21.756469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:21.756469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:21.756469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:21.756469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:21.756469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:21.756469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:21.756469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:21.756469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:21.756469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:21.756469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:21.756469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:21.756469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:21.756469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:21.756469 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:21.757495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:21.757495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:21.757495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:21.757495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:21.757495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:21.757495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:21.757495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:21.757495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:21.757495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:21.757495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:21.757495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:21.757495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:21.757495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:21.757495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:21.757495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:21.757495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:21.758348 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:21.758348 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:21.758348 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:21.758348 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:21.758348 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:21.758348 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:21.758348 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:21.758348 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:21.758348 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:21.758348 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:21.758348 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:21.758348 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:21.758348 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 332/642] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:21.758348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bce82c-f3a5-4ece-bf1a-a8fe1a269137 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:21.769925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553766f5-4319-4d98-92a9-474ca33b3d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:21.770584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553766f5-4319-4d98-92a9-474ca33b3d61 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:21.770866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553766f5-4319-4d98-92a9-474ca33b3d61 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:21.771137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553766f5-4319-4d98-92a9-474ca33b3d61 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:21.771781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553766f5-4319-4d98-92a9-474ca33b3d61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:21.772080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553766f5-4319-4d98-92a9-474ca33b3d61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:21.773584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553766f5-4319-4d98-92a9-474ca33b3d61 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:21.786016 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 311/643] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:21.786847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553766f5-4319-4d98-92a9-474ca33b3d61 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:21.798971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:21.823204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:21.876298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bce82c-f3a5-4ece-bf1a-a8fe1a269137 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:21.902971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553766f5-4319-4d98-92a9-474ca33b3d61 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:21.913824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21709e2-77d5-483c-a332-3a03a58414e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:21.914453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21709e2-77d5-483c-a332-3a03a58414e7 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:21.914759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21709e2-77d5-483c-a332-3a03a58414e7 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:21.915029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21709e2-77d5-483c-a332-3a03a58414e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:21.925966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21709e2-77d5-483c-a332-3a03a58414e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:21.930746 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-b21709e2-77d5-483c-a332-3a03a58414e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:21.930746 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:21.930746 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:21.930746 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:21.930746 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:21.930746 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:21.930746 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:21.930746 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:21.930746 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:21.930746 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:21.930746 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:21.930746 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:21.930746 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:21.930746 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:21.930746 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:21.930746 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:21.931688 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:21.931688 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:21.931688 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:21.931688 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:21.931688 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:21.931688 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:21.931688 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:21.931688 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:21.931688 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:21.931688 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:21.931688 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:21.931688 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:21.931688 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:21.931688 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:21.931688 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:21.931688 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:21.932608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:21.932608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:21.932608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:21.932608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:21.932608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:21.932608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:21.932608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:21.932608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:21.932608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:21.932608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:21.932608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:21.932608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:21.932608 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 333/644] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:21.933424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21709e2-77d5-483c-a332-3a03a58414e7 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:21.944686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec541ac-7b0b-4f97-94ed-c3dff6d1382e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:21.945292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec541ac-7b0b-4f97-94ed-c3dff6d1382e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:21.945571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec541ac-7b0b-4f97-94ed-c3dff6d1382e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:21.945840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec541ac-7b0b-4f97-94ed-c3dff6d1382e None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:21.946823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec541ac-7b0b-4f97-94ed-c3dff6d1382e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:21.947129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec541ac-7b0b-4f97-94ed-c3dff6d1382e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:21.948553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec541ac-7b0b-4f97-94ed-c3dff6d1382e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:21.960641 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 312/645] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:21.961451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec541ac-7b0b-4f97-94ed-c3dff6d1382e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:22.046252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:22.053679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21709e2-77d5-483c-a332-3a03a58414e7 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:22.067975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:22.080994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec541ac-7b0b-4f97-94ed-c3dff6d1382e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:22.091598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa9063-b887-429b-92f2-e0267c591527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:22.092225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa9063-b887-429b-92f2-e0267c591527 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:22.092500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa9063-b887-429b-92f2-e0267c591527 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:22.092770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa9063-b887-429b-92f2-e0267c591527 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:22.093430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa9063-b887-429b-92f2-e0267c591527 None admin] RBAC: Adding query filter params (name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:22.093847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa9063-b887-429b-92f2-e0267c591527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:22.095386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa9063-b887-429b-92f2-e0267c591527 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:22.108857 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 334/646] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:33:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:22.108857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa9063-b887-429b-92f2-e0267c591527 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:22.120436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021610d6-75eb-438d-a17a-419dbd5e81d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:22.121066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021610d6-75eb-438d-a17a-419dbd5e81d5 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:22.121348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021610d6-75eb-438d-a17a-419dbd5e81d5 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:22.121626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021610d6-75eb-438d-a17a-419dbd5e81d5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:22.130198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021610d6-75eb-438d-a17a-419dbd5e81d5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:22.131777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021610d6-75eb-438d-a17a-419dbd5e81d5 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:22.142984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-021610d6-75eb-438d-a17a-419dbd5e81d5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 01 20:33:22.142984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:22.142984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:22.142984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:22.142984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:22.142984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:22.142984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:22.142984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:22.142984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:22.142984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:22.142984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:22.142984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:22.142984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:22.142984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:22.142984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:22.142984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:22.144114 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:22.144114 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:22.144114 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:22.144114 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:22.144114 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:22.144114 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:22.144114 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:22.144114 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:22.144114 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:22.144114 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:22.144114 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:22.144114 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:22.144114 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:22.144114 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:22.144114 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:22.144114 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:22.144114 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:22.144114 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:22.145355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:22.145355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:22.145355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:22.145355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:22.145355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:22.145355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:22.145355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:22.145355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:22.145355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 01 20:33:22.145355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:22.148672 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 313/647] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:33:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:22.149395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021610d6-75eb-438d-a17a-419dbd5e81d5 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:22.222827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa9063-b887-429b-92f2-e0267c591527 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:22.252978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021610d6-75eb-438d-a17a-419dbd5e81d5 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:22.258174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f405aa-8eda-4fe3-b29e-da0066c7952b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:22.258807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f405aa-8eda-4fe3-b29e-da0066c7952b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:22.259087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f405aa-8eda-4fe3-b29e-da0066c7952b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:22.259357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f405aa-8eda-4fe3-b29e-da0066c7952b None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:22.260019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f405aa-8eda-4fe3-b29e-da0066c7952b None admin] RBAC: Adding query filter params (name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:22.260315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f405aa-8eda-4fe3-b29e-da0066c7952b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:22.261664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f405aa-8eda-4fe3-b29e-da0066c7952b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:22.275332 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 335/648] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:33:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:22.276704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f405aa-8eda-4fe3-b29e-da0066c7952b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:22.306973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1538c6e-ec70-45b3-a80f-3f54d0feeff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:22.307424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1538c6e-ec70-45b3-a80f-3f54d0feeff6 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:22.307986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:22.308773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1538c6e-ec70-45b3-a80f-3f54d0feeff6 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:22.309093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1538c6e-ec70-45b3-a80f-3f54d0feeff6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:22.311029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1538c6e-ec70-45b3-a80f-3f54d0feeff6 None admin] RBAC: Adding query filter params (role.id=ef7bd7293369475a9e7b3444fac30cda, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=7ab975dafb564aa2b357311c4a384d53) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:22.311384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1538c6e-ec70-45b3-a80f-3f54d0feeff6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:22.314684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1538c6e-ec70-45b3-a80f-3f54d0feeff6 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:22.334788 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 314/649] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:33:22 2022] GET /identity/v3/role_assignments?user.id=7ab975dafb564aa2b357311c4a384d53&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=ef7bd7293369475a9e7b3444fac30cda => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:22.334788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1538c6e-ec70-45b3-a80f-3f54d0feeff6 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:22.372026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:22.463290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f405aa-8eda-4fe3-b29e-da0066c7952b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:22.509180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f6285-80b8-464a-9fec-f0c66c2487f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:22.509832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f6285-80b8-464a-9fec-f0c66c2487f1 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:22.510113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f6285-80b8-464a-9fec-f0c66c2487f1 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:22.510385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f6285-80b8-464a-9fec-f0c66c2487f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:22.521597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1538c6e-ec70-45b3-a80f-3f54d0feeff6 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:22.524417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f6285-80b8-464a-9fec-f0c66c2487f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:22.526260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f6285-80b8-464a-9fec-f0c66c2487f1 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:22.546422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-310f6285-80b8-464a-9fec-f0c66c2487f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:22.546422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:22.546422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:22.546422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:22.546422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:22.546422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:22.546422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:22.546422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:22.546422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:22.546422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:22.546422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:22.546422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:22.546422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:22.546422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:22.546422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:22.546422 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:22.547486 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:22.547486 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:22.547486 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:22.547486 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:22.547486 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:22.547486 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:22.547486 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:22.547486 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:22.547486 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:22.547486 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:22.547486 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:22.547486 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:22.547486 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:22.547486 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:22.547486 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:22.547486 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:22.547486 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:22.549211 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:22.549211 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:22.549211 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:22.550648 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 336/650] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:22.551734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f6285-80b8-464a-9fec-f0c66c2487f1 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:22.580833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6754362-e3a8-4241-ac51-4f6016311df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:22.581477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6754362-e3a8-4241-ac51-4f6016311df3 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:22.581648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6754362-e3a8-4241-ac51-4f6016311df3 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:22.581922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6754362-e3a8-4241-ac51-4f6016311df3 None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:22.583307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6754362-e3a8-4241-ac51-4f6016311df3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:22.583624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6754362-e3a8-4241-ac51-4f6016311df3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:22.585403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6754362-e3a8-4241-ac51-4f6016311df3 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:22.609619 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 315/651] 10.222.0.17 () {56 vars in 1137 bytes} [Sat Oct 1 20:33:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:22.609619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6754362-e3a8-4241-ac51-4f6016311df3 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:22.640103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:22.683927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f6285-80b8-464a-9fec-f0c66c2487f1 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:22.709917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:22.724474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301751d5-819e-4a96-8385-1146be4e08d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:22.725098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301751d5-819e-4a96-8385-1146be4e08d6 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:22.725513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301751d5-819e-4a96-8385-1146be4e08d6 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:22.725788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301751d5-819e-4a96-8385-1146be4e08d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:22.737129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301751d5-819e-4a96-8385-1146be4e08d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:22.738592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301751d5-819e-4a96-8385-1146be4e08d6 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:22.744733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6754362-e3a8-4241-ac51-4f6016311df3 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:22.753759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-301751d5-819e-4a96-8385-1146be4e08d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:22.753759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:22.753759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:22.753759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:22.753759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:22.753759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:22.753759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:22.753759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:22.753759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:22.753759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:22.753759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:22.753759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:22.753759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:22.753759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:22.753759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:22.753759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:22.754960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:22.754960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:22.754960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:22.754960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:22.754960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:22.754960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:22.754960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:22.754960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:22.754960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:22.754960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:22.754960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:22.754960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:22.754960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:22.754960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:22.754960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:22.754960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:22.754960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:22.755876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:22.755876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:22.755876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:22.755876 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 337/652] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:22.755876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301751d5-819e-4a96-8385-1146be4e08d6 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:22.798117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28f9ce-c42d-49d2-ba50-a0786cb3c17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:22.798958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28f9ce-c42d-49d2-ba50-a0786cb3c17d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:22.799339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28f9ce-c42d-49d2-ba50-a0786cb3c17d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:22.799716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28f9ce-c42d-49d2-ba50-a0786cb3c17d None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:22.800491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad28f9ce-c42d-49d2-ba50-a0786cb3c17d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:22.800858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad28f9ce-c42d-49d2-ba50-a0786cb3c17d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:22.802808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad28f9ce-c42d-49d2-ba50-a0786cb3c17d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:22.818090 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 316/653] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:22.822944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28f9ce-c42d-49d2-ba50-a0786cb3c17d None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:22.869807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301751d5-819e-4a96-8385-1146be4e08d6 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:22.896735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:22.912717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bcdd3f-4e3b-4094-993a-d6a90364ac94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:22.913329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcdd3f-4e3b-4094-993a-d6a90364ac94 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:22.913609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcdd3f-4e3b-4094-993a-d6a90364ac94 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:22.913882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcdd3f-4e3b-4094-993a-d6a90364ac94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:22.952134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28f9ce-c42d-49d2-ba50-a0786cb3c17d None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:22.989135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bcdd3f-4e3b-4094-993a-d6a90364ac94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:22.990805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bcdd3f-4e3b-4094-993a-d6a90364ac94 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:22.998340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00136fc6-3b73-49ea-ac55-6775ed758cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:22.999878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00136fc6-3b73-49ea-ac55-6775ed758cfd None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:23.000370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00136fc6-3b73-49ea-ac55-6775ed758cfd None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:23.001801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00136fc6-3b73-49ea-ac55-6775ed758cfd None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:23.002683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00136fc6-3b73-49ea-ac55-6775ed758cfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:23.003542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00136fc6-3b73-49ea-ac55-6775ed758cfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:23.005645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00136fc6-3b73-49ea-ac55-6775ed758cfd None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:23.008165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:23.026717 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 317/654] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:23.028051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00136fc6-3b73-49ea-ac55-6775ed758cfd None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:23.127291 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 338/655] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:33:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:33:23.128266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bcdd3f-4e3b-4094-993a-d6a90364ac94 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:23.153002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00136fc6-3b73-49ea-ac55-6775ed758cfd None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:23.154998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:23.192764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dad7f-3830-445f-8bd3-3210ad6aed5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:23.193566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dad7f-3830-445f-8bd3-3210ad6aed5f None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:23.193957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dad7f-3830-445f-8bd3-3210ad6aed5f None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:23.194367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dad7f-3830-445f-8bd3-3210ad6aed5f None admin] PATH_INFO: `/v3/users/glance` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:23.202524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188dad7f-3830-445f-8bd3-3210ad6aed5f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:23.204796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188dad7f-3830-445f-8bd3-3210ad6aed5f None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:23.217335 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-188dad7f-3830-445f-8bd3-3210ad6aed5f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 01 20:33:23.217335 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:23.217335 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:23.217335 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:23.217335 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:23.217335 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:23.217335 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:23.217335 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:23.217335 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:23.217335 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:23.217335 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:23.217335 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:23.217335 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:23.217335 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:23.217335 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:23.217335 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:23.218298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:23.218298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:23.218298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:23.218298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:23.218298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:23.218298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:23.218298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:23.218298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:23.218298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:23.218298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:23.218298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:23.218298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:23.218298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:23.218298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:23.218298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:23.218298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:23.218298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:23.218298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:23.219298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:23.219298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:23.219298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:23.219298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:23.219298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:23.219298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:23.219298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:23.219298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:23.219298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 01 20:33:23.219298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:23.219298 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 318/656] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:33:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:23.219298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dad7f-3830-445f-8bd3-3210ad6aed5f None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:23.252189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bcdd3f-4e3b-4094-993a-d6a90364ac94 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:23.281501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:23.296154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d12c5a-0ac5-4a20-9a68-e31affd94f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:23.297339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d12c5a-0ac5-4a20-9a68-e31affd94f92 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:23.298585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d12c5a-0ac5-4a20-9a68-e31affd94f92 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:23.299257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d12c5a-0ac5-4a20-9a68-e31affd94f92 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:23.307858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d12c5a-0ac5-4a20-9a68-e31affd94f92 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:23.309878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d12c5a-0ac5-4a20-9a68-e31affd94f92 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:23.330525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-b6d12c5a-0ac5-4a20-9a68-e31affd94f92 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 01 20:33:23.330525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:23.330525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:23.330525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:23.330525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:23.330525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:23.330525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:23.330525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:23.330525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:23.330525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:23.330525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:23.330525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:23.330525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:23.330525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:23.330525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:23.330525 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:23.331731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:23.331731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:23.331731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:23.331731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:23.331731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:23.331731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:23.331731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:23.331731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:23.331731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:23.331731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:23.331731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:23.331731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:23.331731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:23.331731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:23.331731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:23.331731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:23.331731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:23.331731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:23.332897 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:23.332897 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:23.332897 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:23.332897 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:23.332897 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:23.332897 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:23.332897 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:23.332897 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:23.332897 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 01 20:33:23.332897 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:23.338093 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 339/657] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:23.338093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d12c5a-0ac5-4a20-9a68-e31affd94f92 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:23.351778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dad7f-3830-445f-8bd3-3210ad6aed5f None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:23.393287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743bd41-0065-416e-b674-03cee9e35477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:23.394302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743bd41-0065-416e-b674-03cee9e35477 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:23.394801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743bd41-0065-416e-b674-03cee9e35477 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:23.395265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743bd41-0065-416e-b674-03cee9e35477 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:23.406410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743bd41-0065-416e-b674-03cee9e35477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:23.408313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743bd41-0065-416e-b674-03cee9e35477 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:23.423911 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-f743bd41-0065-416e-b674-03cee9e35477 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:23.423911 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:23.423911 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:23.423911 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:23.423911 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:23.423911 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:23.423911 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:23.423911 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:23.423911 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:23.423911 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:23.423911 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:23.423911 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:23.423911 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:23.423911 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:23.423911 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:23.423911 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:23.424887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:23.424887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:23.424887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:23.424887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:23.424887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:23.424887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:23.424887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:23.424887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:23.424887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:23.424887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:23.424887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:23.424887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:23.424887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:23.424887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:23.424887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:23.424887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:23.424887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:23.425797 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:23.425797 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:23.425797 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:23.427972 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 319/658] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:23.429219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743bd41-0065-416e-b674-03cee9e35477 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:23.430915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:23.447080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d12c5a-0ac5-4a20-9a68-e31affd94f92 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:23.490372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01546c-6185-49ef-ac6e-736f158af1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:23.491056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01546c-6185-49ef-ac6e-736f158af1ca None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:23.491334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01546c-6185-49ef-ac6e-736f158af1ca None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:23.491604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01546c-6185-49ef-ac6e-736f158af1ca None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:23.492662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01546c-6185-49ef-ac6e-736f158af1ca None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:23.492970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01546c-6185-49ef-ac6e-736f158af1ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:23.494433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01546c-6185-49ef-ac6e-736f158af1ca None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:23.519980 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 340/659] 10.222.0.17 () {56 vars in 1137 bytes} [Sat Oct 1 20:33:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:23.520877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01546c-6185-49ef-ac6e-736f158af1ca None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:23.543572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743bd41-0065-416e-b674-03cee9e35477 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:23.551593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:23.582026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1242ea-d499-4043-a987-d030fbf8378f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:23.583500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1242ea-d499-4043-a987-d030fbf8378f None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:23.584001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1242ea-d499-4043-a987-d030fbf8378f None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:23.584467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1242ea-d499-4043-a987-d030fbf8378f None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:23.585604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1242ea-d499-4043-a987-d030fbf8378f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:23.586103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1242ea-d499-4043-a987-d030fbf8378f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:23.587820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1242ea-d499-4043-a987-d030fbf8378f None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:23.610711 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 320/660] 10.222.0.17 () {56 vars in 1139 bytes} [Sat Oct 1 20:33:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:23.611861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1242ea-d499-4043-a987-d030fbf8378f None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:23.632932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01546c-6185-49ef-ac6e-736f158af1ca None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:23.674247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb3689-897d-4b31-b9f7-be701b0f186a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:23.674809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb3689-897d-4b31-b9f7-be701b0f186a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:23.675087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb3689-897d-4b31-b9f7-be701b0f186a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:23.675359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb3689-897d-4b31-b9f7-be701b0f186a None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:23.676120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb3689-897d-4b31-b9f7-be701b0f186a None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:23.676406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb3689-897d-4b31-b9f7-be701b0f186a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:23.677919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb3689-897d-4b31-b9f7-be701b0f186a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:23.687702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:23.697289 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 341/661] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:23.698148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb3689-897d-4b31-b9f7-be701b0f186a None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:23.725709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1242ea-d499-4043-a987-d030fbf8378f None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:23.761921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35025203-a969-49e1-84cb-fe89f2f30da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:23.763109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35025203-a969-49e1-84cb-fe89f2f30da8 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:23.763585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35025203-a969-49e1-84cb-fe89f2f30da8 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:23.764060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35025203-a969-49e1-84cb-fe89f2f30da8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:23.799735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:23.826429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb3689-897d-4b31-b9f7-be701b0f186a None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:23.852628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35025203-a969-49e1-84cb-fe89f2f30da8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:23.854283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35025203-a969-49e1-84cb-fe89f2f30da8 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:23.867132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab7623-1859-4b80-b169-8d8ece8bca7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:23.868503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab7623-1859-4b80-b169-8d8ece8bca7c None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:23.869225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab7623-1859-4b80-b169-8d8ece8bca7c None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:23.869792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab7623-1859-4b80-b169-8d8ece8bca7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:23.951104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:23.955884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab7623-1859-4b80-b169-8d8ece8bca7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:23.957740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab7623-1859-4b80-b169-8d8ece8bca7c None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:23.976015 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 321/662] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:33:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:33:23.976857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35025203-a969-49e1-84cb-fe89f2f30da8 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:24.056045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:24.067264 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 342/663] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:33:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:33:24.068356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066b7f1-0554-4daf-9a55-25f2f89b5318 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:24.068644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066b7f1-0554-4daf-9a55-25f2f89b5318 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:24.068919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066b7f1-0554-4daf-9a55-25f2f89b5318 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:24.070265 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 343/664] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:24.075168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066b7f1-0554-4daf-9a55-25f2f89b5318 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:24.102541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35025203-a969-49e1-84cb-fe89f2f30da8 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:24.138265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47948031-3f87-4a25-9067-88e75268193d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:24.139104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47948031-3f87-4a25-9067-88e75268193d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:24.139490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47948031-3f87-4a25-9067-88e75268193d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:24.139761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47948031-3f87-4a25-9067-88e75268193d None admin] PATH_INFO: `/v3/projects/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:24.146162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47948031-3f87-4a25-9067-88e75268193d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:24.147650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47948031-3f87-4a25-9067-88e75268193d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:24.158468 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-47948031-3f87-4a25-9067-88e75268193d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:24.158468 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:24.158468 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:24.158468 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:24.158468 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:24.158468 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:24.158468 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:24.158468 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:24.158468 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:24.158468 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:24.158468 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:24.158468 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:24.158468 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:24.158468 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:24.158468 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:24.159574 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:24.159574 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:24.159574 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:24.159574 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:24.159574 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:24.159574 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:24.159574 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:24.159574 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:24.159574 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:24.159574 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:24.159574 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:24.159574 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:24.159574 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:24.159574 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:24.159574 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:24.159574 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:24.159574 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:24.160487 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:24.160487 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:24.160487 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:24.160487 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:24.160487 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:24.160487 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:24.160487 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:24.160487 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:24.160487 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 322/665] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:24.160487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47948031-3f87-4a25-9067-88e75268193d None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:24.182508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:24.188133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066b7f1-0554-4daf-9a55-25f2f89b5318 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:24.224314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402a6d4-6733-4d96-8574-d080a048b0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:24.224862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402a6d4-6733-4d96-8574-d080a048b0b4 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:24.225139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402a6d4-6733-4d96-8574-d080a048b0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:24.225411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402a6d4-6733-4d96-8574-d080a048b0b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:24.235849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402a6d4-6733-4d96-8574-d080a048b0b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:24.237264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402a6d4-6733-4d96-8574-d080a048b0b4 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:24.252380 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-5402a6d4-6733-4d96-8574-d080a048b0b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:24.252380 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:24.252380 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:24.252380 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:24.252380 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:24.252380 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:24.252380 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:24.252380 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:24.252380 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:24.252380 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:24.252380 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:24.252380 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:24.252380 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:24.252380 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:24.252380 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:24.252380 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:24.253339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:24.253339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:24.253339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:24.253339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:24.253339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:24.253339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:24.253339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:24.253339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:24.253339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:24.253339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:24.253339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:24.253339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:24.253339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:24.253339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:24.253339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:24.253339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:24.253339 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:24.254250 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:24.254250 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:24.254250 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:24.254250 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 344/666] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:24.255024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f074f0f-73d4-4b0f-b5e6-0695ca24ec6d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:24.255276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f074f0f-73d4-4b0f-b5e6-0695ca24ec6d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:24.255634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f074f0f-73d4-4b0f-b5e6-0695ca24ec6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:24.274679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1f074f0f-73d4-4b0f-b5e6-0695ca24ec6d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:24.275353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47948031-3f87-4a25-9067-88e75268193d None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:24.275896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1f074f0f-73d4-4b0f-b5e6-0695ca24ec6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:24.300166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:24.311644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819411ec-b83c-49f5-a15a-f20f2e4b030f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:24.312506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819411ec-b83c-49f5-a15a-f20f2e4b030f None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:24.312905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819411ec-b83c-49f5-a15a-f20f2e4b030f None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:24.313414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819411ec-b83c-49f5-a15a-f20f2e4b030f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:24.328070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819411ec-b83c-49f5-a15a-f20f2e4b030f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:24.330083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819411ec-b83c-49f5-a15a-f20f2e4b030f None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:24.344495 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-819411ec-b83c-49f5-a15a-f20f2e4b030f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:24.344495 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:24.344495 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:24.344495 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:24.344495 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:24.344495 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:24.344495 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:24.344495 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:24.344495 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:24.344495 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:24.344495 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:24.344495 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:24.344495 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:24.344495 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:24.344495 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:24.344495 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:24.346045 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:24.346045 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:24.346045 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:24.346045 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:24.346045 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:24.346045 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:24.346045 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:24.346045 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:24.346045 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:24.346045 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:24.346045 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:24.346045 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:24.346045 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:24.346045 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:24.346045 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:24.346045 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:24.346045 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:24.347697 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:24.347697 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:24.347697 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:24.347697 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 323/667] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:24.347697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819411ec-b83c-49f5-a15a-f20f2e4b030f None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:24.366617 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 345/668] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:24.367836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f074f0f-73d4-4b0f-b5e6-0695ca24ec6d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:24.459017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819411ec-b83c-49f5-a15a-f20f2e4b030f None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:24.463999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:24.488476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f074f0f-73d4-4b0f-b5e6-0695ca24ec6d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:24.496849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1b177-da9d-46ca-89ee-c836bdb9f2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:24.497453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1b177-da9d-46ca-89ee-c836bdb9f2ab None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:24.497735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1b177-da9d-46ca-89ee-c836bdb9f2ab None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:24.498009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1b177-da9d-46ca-89ee-c836bdb9f2ab None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:24.499082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af1b177-da9d-46ca-89ee-c836bdb9f2ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:24.499211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af1b177-da9d-46ca-89ee-c836bdb9f2ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:24.500549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af1b177-da9d-46ca-89ee-c836bdb9f2ab None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:24.518676 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 324/669] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:24.518676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1b177-da9d-46ca-89ee-c836bdb9f2ab None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:24.539143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eadf11-2c69-407e-8895-1d37463edc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:24.540476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eadf11-2c69-407e-8895-1d37463edc63 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:24.540931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eadf11-2c69-407e-8895-1d37463edc63 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:24.541345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eadf11-2c69-407e-8895-1d37463edc63 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:24.542181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eadf11-2c69-407e-8895-1d37463edc63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:24.542704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eadf11-2c69-407e-8895-1d37463edc63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:24.545135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eadf11-2c69-407e-8895-1d37463edc63 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:24.558077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:24.561144 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 346/670] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:24.562236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eadf11-2c69-407e-8895-1d37463edc63 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:24.632998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1b177-da9d-46ca-89ee-c836bdb9f2ab None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:24.670649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d590345-9e69-4936-8ff4-89db92ac56b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:24.671440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d590345-9e69-4936-8ff4-89db92ac56b9 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:24.671818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d590345-9e69-4936-8ff4-89db92ac56b9 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:24.672193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d590345-9e69-4936-8ff4-89db92ac56b9 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:24.672980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d590345-9e69-4936-8ff4-89db92ac56b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:24.673383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d590345-9e69-4936-8ff4-89db92ac56b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:24.674884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d590345-9e69-4936-8ff4-89db92ac56b9 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:24.682370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eadf11-2c69-407e-8895-1d37463edc63 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:24.689247 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 325/671] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:24.690254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d590345-9e69-4936-8ff4-89db92ac56b9 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:24.724331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b5e57-46bf-47fb-bb3a-674a67873bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:24.724933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b5e57-46bf-47fb-bb3a-674a67873bed None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:24.725210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b5e57-46bf-47fb-bb3a-674a67873bed None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:24.725540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b5e57-46bf-47fb-bb3a-674a67873bed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:24.726226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b5e57-46bf-47fb-bb3a-674a67873bed None admin] RBAC: Adding query filter params (role.id=ef7bd7293369475a9e7b3444fac30cda, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=197a0ae6b71f4635a069ac0238e017c9) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:24.726556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b5e57-46bf-47fb-bb3a-674a67873bed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:24.727984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b5e57-46bf-47fb-bb3a-674a67873bed None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:24.733952 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 347/672] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:33:24 2022] GET /identity/v3/role_assignments?user.id=197a0ae6b71f4635a069ac0238e017c9&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=ef7bd7293369475a9e7b3444fac30cda => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:24.734762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b5e57-46bf-47fb-bb3a-674a67873bed None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:24.744140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:24.813436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:24.823731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d590345-9e69-4936-8ff4-89db92ac56b9 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:24.870708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12fd07-7fe3-4069-9f9b-ce9c4f7afbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:24.871859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12fd07-7fe3-4069-9f9b-ce9c4f7afbb0 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:24.872237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12fd07-7fe3-4069-9f9b-ce9c4f7afbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:24.872512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12fd07-7fe3-4069-9f9b-ce9c4f7afbb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:24.884440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b5e57-46bf-47fb-bb3a-674a67873bed None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:24.887188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12fd07-7fe3-4069-9f9b-ce9c4f7afbb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:24.889893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12fd07-7fe3-4069-9f9b-ce9c4f7afbb0 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:24.905163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-2a12fd07-7fe3-4069-9f9b-ce9c4f7afbb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:24.905163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:24.905163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:24.905163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:24.905163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:24.905163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:24.905163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:24.905163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:24.905163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:24.905163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:24.905163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:24.905163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:24.905163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:24.905163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:24.905163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:24.906273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:24.906273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:24.906273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:24.906273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:24.906273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:24.906273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:24.906273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:24.906273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:24.906273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:24.906273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:24.906273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:24.906273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:24.906273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:24.906273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:24.906273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:24.906273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:24.906273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:24.907315 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:24.907315 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:24.907315 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:24.907315 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:24.907315 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:24.907315 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:24.907315 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:24.907315 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:24.907315 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 326/673] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:24.907840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12fd07-7fe3-4069-9f9b-ce9c4f7afbb0 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:24.932042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76c826-a2af-41d8-a526-cae23b923111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:24.933674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76c826-a2af-41d8-a526-cae23b923111 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:24.934168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76c826-a2af-41d8-a526-cae23b923111 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:24.934675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76c826-a2af-41d8-a526-cae23b923111 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:24.944437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76c826-a2af-41d8-a526-cae23b923111 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:24.947392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76c826-a2af-41d8-a526-cae23b923111 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:24.962989 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-8b76c826-a2af-41d8-a526-cae23b923111 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:24.962989 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:24.962989 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:24.962989 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:24.962989 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:24.962989 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:24.962989 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:24.962989 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:24.962989 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:24.962989 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:24.962989 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:24.962989 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:24.962989 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:24.962989 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:24.962989 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:24.964543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:24.964543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:24.964543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:24.964543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:24.964543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:24.964543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:24.964543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:24.964543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:24.964543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:24.964543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:24.964543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:24.964543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:24.964543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:24.964543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:24.964543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:24.964543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:24.964543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:24.965492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:24.965492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:24.965492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:24.965492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:24.965492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:24.965492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:24.965492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:24.965492 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:24.968267 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 348/674] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:24.969378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76c826-a2af-41d8-a526-cae23b923111 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:25.049220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12fd07-7fe3-4069-9f9b-ce9c4f7afbb0 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:25.054005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:25.093623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb053f-d18b-4253-98ad-83a1180e2dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:25.095257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb053f-d18b-4253-98ad-83a1180e2dd8 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:25.095816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb053f-d18b-4253-98ad-83a1180e2dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:25.096092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb053f-d18b-4253-98ad-83a1180e2dd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:25.097480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76c826-a2af-41d8-a526-cae23b923111 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:25.108252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb053f-d18b-4253-98ad-83a1180e2dd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:25.113813 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-35eb053f-d18b-4253-98ad-83a1180e2dd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:25.113813 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:25.113813 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:25.113813 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:25.113813 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:25.113813 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:25.113813 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:25.113813 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:25.113813 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:25.113813 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:25.113813 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:25.113813 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:25.113813 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:25.113813 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:25.113813 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:25.113813 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:25.114792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:25.114792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:25.114792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:25.114792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:25.114792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:25.114792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:25.114792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:25.114792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:25.114792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:25.114792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:25.114792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:25.114792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:25.114792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:25.114792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:25.114792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:25.114792 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:25.115676 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:25.115676 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:25.115676 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:25.115676 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:25.115676 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:25.115676 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:25.115676 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:25.115676 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:25.115676 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:25.115676 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:25.115676 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:25.115676 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:25.115676 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 327/675] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:25.116359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb053f-d18b-4253-98ad-83a1180e2dd8 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:25.136085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:25.141252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a9240-9c63-4738-a987-ecb1534100ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:25.141984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a9240-9c63-4738-a987-ecb1534100ea None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:25.142259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a9240-9c63-4738-a987-ecb1534100ea None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:25.142429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a9240-9c63-4738-a987-ecb1534100ea None admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:25.143223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a9240-9c63-4738-a987-ecb1534100ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:25.143700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a9240-9c63-4738-a987-ecb1534100ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:25.145736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a9240-9c63-4738-a987-ecb1534100ea None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:25.160526 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 349/676] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:25.161627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dabfc-8a3d-49d1-8a72-9c588cb7e40f None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:25.161920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dabfc-8a3d-49d1-8a72-9c588cb7e40f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:25.162197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dabfc-8a3d-49d1-8a72-9c588cb7e40f None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:25.163569 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 350/677] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:25.164106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9dabfc-8a3d-49d1-8a72-9c588cb7e40f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:25.233246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb053f-d18b-4253-98ad-83a1180e2dd8 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:25.268444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b281662-2903-4666-b864-210201ac962a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:25.269043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b281662-2903-4666-b864-210201ac962a None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:25.269379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b281662-2903-4666-b864-210201ac962a None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:25.269576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b281662-2903-4666-b864-210201ac962a None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:25.270256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b281662-2903-4666-b864-210201ac962a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:25.270590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b281662-2903-4666-b864-210201ac962a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:25.270892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9dabfc-8a3d-49d1-8a72-9c588cb7e40f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:25.272031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b281662-2903-4666-b864-210201ac962a None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:25.289723 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 328/678] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:25.289723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b281662-2903-4666-b864-210201ac962a None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:25.301519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:25.315179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae095f-dd75-4235-ac3e-d49b8624d004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:25.315881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae095f-dd75-4235-ac3e-d49b8624d004 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:25.316238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae095f-dd75-4235-ac3e-d49b8624d004 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:25.317167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae095f-dd75-4235-ac3e-d49b8624d004 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:25.317282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae095f-dd75-4235-ac3e-d49b8624d004 None admin] RBAC: Adding query filter params (name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:25.317607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae095f-dd75-4235-ac3e-d49b8624d004 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:25.319287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae095f-dd75-4235-ac3e-d49b8624d004 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:25.332869 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 351/679] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:33:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:25.338637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d62ecc-76af-4fc1-baa4-051917ad4c01 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:25.338929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d62ecc-76af-4fc1-baa4-051917ad4c01 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:25.339210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d62ecc-76af-4fc1-baa4-051917ad4c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:25.357872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-17d62ecc-76af-4fc1-baa4-051917ad4c01 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:25.358818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-17d62ecc-76af-4fc1-baa4-051917ad4c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:25.375529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:25.400216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b281662-2903-4666-b864-210201ac962a None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:25.431240 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 352/680] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:25.436259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97d16b-b6c2-4a44-be40-5cc8fc34e229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:25.437063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97d16b-b6c2-4a44-be40-5cc8fc34e229 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:25.437339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97d16b-b6c2-4a44-be40-5cc8fc34e229 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:25.437606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97d16b-b6c2-4a44-be40-5cc8fc34e229 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:25.438867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97d16b-b6c2-4a44-be40-5cc8fc34e229 None admin] RBAC: Adding query filter params (role.id=ef7bd7293369475a9e7b3444fac30cda, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=e2927203ce394fabab76dd52a68c2436) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:25.439165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97d16b-b6c2-4a44-be40-5cc8fc34e229 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:25.440583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97d16b-b6c2-4a44-be40-5cc8fc34e229 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:25.441787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d62ecc-76af-4fc1-baa4-051917ad4c01 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:25.454872 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 329/681] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:33:25 2022] GET /identity/v3/role_assignments?user.id=e2927203ce394fabab76dd52a68c2436&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=ef7bd7293369475a9e7b3444fac30cda => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:25.454872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97d16b-b6c2-4a44-be40-5cc8fc34e229 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:25.653906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97d16b-b6c2-4a44-be40-5cc8fc34e229 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:25.660414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d62ecc-76af-4fc1-baa4-051917ad4c01 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:25.693433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:25.727709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c561a57-d0ed-434f-9d72-e608b293b5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:25.728331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c561a57-d0ed-434f-9d72-e608b293b5ac None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:25.728607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c561a57-d0ed-434f-9d72-e608b293b5ac None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:25.728877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c561a57-d0ed-434f-9d72-e608b293b5ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:25.729630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c561a57-d0ed-434f-9d72-e608b293b5ac None admin] RBAC: Adding query filter params (role.id=ef7bd7293369475a9e7b3444fac30cda, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=e17a026f305248878f18eec35e1e02dd) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:25.729927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c561a57-d0ed-434f-9d72-e608b293b5ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:25.731388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c561a57-d0ed-434f-9d72-e608b293b5ac None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:25.734415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07476a9a-9d25-4ae7-9417-3fe829a93ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:25.735673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07476a9a-9d25-4ae7-9417-3fe829a93ee0 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:25.735950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07476a9a-9d25-4ae7-9417-3fe829a93ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:25.736215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07476a9a-9d25-4ae7-9417-3fe829a93ee0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:25.746792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:25.750246 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 353/682] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:33:25 2022] GET /identity/v3/role_assignments?user.id=e17a026f305248878f18eec35e1e02dd&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=ef7bd7293369475a9e7b3444fac30cda => generated 608 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:25.752284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07476a9a-9d25-4ae7-9417-3fe829a93ee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:25.753753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07476a9a-9d25-4ae7-9417-3fe829a93ee0 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:25.762781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c561a57-d0ed-434f-9d72-e608b293b5ac None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:25.779568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-07476a9a-9d25-4ae7-9417-3fe829a93ee0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:25.779568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:25.779568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:25.779568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:25.779568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:25.779568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:25.779568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:25.779568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:25.779568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:25.779568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:25.779568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:25.779568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:25.779568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:25.779568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:25.779568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:25.779568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:25.780771 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:25.780771 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:25.780771 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:25.780771 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:25.780771 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:25.780771 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:25.780771 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:25.780771 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:25.780771 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:25.780771 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:25.780771 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:25.780771 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:25.780771 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:25.780771 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:25.780771 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:25.780771 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:25.780771 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:25.781917 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:25.781917 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:25.781917 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:25.784577 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 330/683] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:25.791654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07476a9a-9d25-4ae7-9417-3fe829a93ee0 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:25.912676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c561a57-d0ed-434f-9d72-e608b293b5ac None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:25.955787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07476a9a-9d25-4ae7-9417-3fe829a93ee0 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:25.970561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e530ce7-5340-41c9-8cef-ab9f5836c2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:25.972386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e530ce7-5340-41c9-8cef-ab9f5836c2fd None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:25.972868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e530ce7-5340-41c9-8cef-ab9f5836c2fd None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:25.973632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e530ce7-5340-41c9-8cef-ab9f5836c2fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:25.988469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e530ce7-5340-41c9-8cef-ab9f5836c2fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:25.995856 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-8e530ce7-5340-41c9-8cef-ab9f5836c2fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:33:25.995856 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:25.995856 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:25.995856 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:25.995856 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:25.995856 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:25.995856 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:25.995856 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:25.995856 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:25.995856 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:25.995856 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:25.995856 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:25.995856 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:25.995856 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:25.995856 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:25.995856 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:25.996824 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:25.996824 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:25.996824 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:25.996824 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:25.996824 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:25.996824 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:25.996824 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:25.996824 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:25.996824 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:25.996824 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:25.996824 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:25.996824 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:25.996824 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:25.996824 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:25.996824 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:25.996824 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:25.997703 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:25.997703 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:25.997703 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:25.997703 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:25.997703 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:25.997703 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:25.997703 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:25.997703 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:25.997703 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:25.997703 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:25.997703 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:33:25.997703 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:26.000437 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 354/684] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:33:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:26.008001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e530ce7-5340-41c9-8cef-ab9f5836c2fd None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:26.008464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796ec8c-e894-48f8-b7d0-d5d3e75ec7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:26.009315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796ec8c-e894-48f8-b7d0-d5d3e75ec7d1 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:26.009758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796ec8c-e894-48f8-b7d0-d5d3e75ec7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:26.010191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796ec8c-e894-48f8-b7d0-d5d3e75ec7d1 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:26.011053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796ec8c-e894-48f8-b7d0-d5d3e75ec7d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:26.011522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796ec8c-e894-48f8-b7d0-d5d3e75ec7d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:26.013268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796ec8c-e894-48f8-b7d0-d5d3e75ec7d1 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:26.030254 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 331/685] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:26.036075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796ec8c-e894-48f8-b7d0-d5d3e75ec7d1 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:26.064495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:26.105200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:26.151069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e530ce7-5340-41c9-8cef-ab9f5836c2fd None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:26.166521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796ec8c-e894-48f8-b7d0-d5d3e75ec7d1 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:26.200487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b963b5d-46ec-4997-882d-191e89b4504b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:26.201514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b963b5d-46ec-4997-882d-191e89b4504b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:26.201971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b963b5d-46ec-4997-882d-191e89b4504b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:26.202458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b963b5d-46ec-4997-882d-191e89b4504b None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:26.203312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b963b5d-46ec-4997-882d-191e89b4504b None admin] RBAC: Adding query filter params (name=admin) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:26.203763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b963b5d-46ec-4997-882d-191e89b4504b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:26.205345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b963b5d-46ec-4997-882d-191e89b4504b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:26.206304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b5d0b-ba4f-44fc-8d1e-092765ee6e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:26.206972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b5d0b-ba4f-44fc-8d1e-092765ee6e7e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:26.207246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b5d0b-ba4f-44fc-8d1e-092765ee6e7e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:26.207523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b5d0b-ba4f-44fc-8d1e-092765ee6e7e None admin] PATH_INFO: `/v3/users/swift` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:26.218597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b5d0b-ba4f-44fc-8d1e-092765ee6e7e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:26.220825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b5d0b-ba4f-44fc-8d1e-092765ee6e7e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:26.222776 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 355/686] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:33:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:26.228603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b963b5d-46ec-4997-882d-191e89b4504b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:26.246976 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-d91b5d0b-ba4f-44fc-8d1e-092765ee6e7e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 01 20:33:26.246976 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:26.246976 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:26.246976 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:26.246976 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:26.246976 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:26.246976 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:26.246976 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:26.246976 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:26.246976 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:26.246976 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:26.246976 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:26.246976 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:26.246976 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:26.246976 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:26.246976 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:26.248788 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:26.248788 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:26.248788 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:26.248788 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:26.248788 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:26.248788 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:26.248788 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:26.248788 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:26.248788 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:26.248788 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:26.248788 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:26.248788 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:26.248788 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:26.248788 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:26.248788 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:26.248788 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:26.248788 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:26.248788 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:26.250648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:26.250648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:26.250648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:26.250648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:26.250648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:26.250648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:26.250648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:26.250648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:26.250648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 01 20:33:26.250648 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:26.254581 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 332/687] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:33:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:26.256676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b5d0b-ba4f-44fc-8d1e-092765ee6e7e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:26.418699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b963b5d-46ec-4997-882d-191e89b4504b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:26.420406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:26.435635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b5d0b-ba4f-44fc-8d1e-092765ee6e7e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:26.456677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:26.484690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c3dc2-1343-45d5-848b-5dbee7f04b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:26.485863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c3dc2-1343-45d5-848b-5dbee7f04b7f None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:26.486149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c3dc2-1343-45d5-848b-5dbee7f04b7f None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:26.486410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c3dc2-1343-45d5-848b-5dbee7f04b7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:26.496901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab98dc3-7840-427e-81cd-da8e9c9c63c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:26.497514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab98dc3-7840-427e-81cd-da8e9c9c63c6 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:26.497789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab98dc3-7840-427e-81cd-da8e9c9c63c6 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:26.498058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab98dc3-7840-427e-81cd-da8e9c9c63c6 None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:26.498814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab98dc3-7840-427e-81cd-da8e9c9c63c6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:26.499115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab98dc3-7840-427e-81cd-da8e9c9c63c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:26.500543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab98dc3-7840-427e-81cd-da8e9c9c63c6 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:26.502314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c3dc2-1343-45d5-848b-5dbee7f04b7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:26.503791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c3dc2-1343-45d5-848b-5dbee7f04b7f None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:26.520199 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 333/688] 10.222.0.17 () {56 vars in 1135 bytes} [Sat Oct 1 20:33:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:26.525637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab98dc3-7840-427e-81cd-da8e9c9c63c6 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:26.527509 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-fc8c3dc2-1343-45d5-848b-5dbee7f04b7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:26.527509 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:26.527509 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:26.527509 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:26.527509 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:26.527509 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:26.527509 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:26.527509 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:26.527509 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:26.527509 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:26.527509 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:26.527509 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:26.527509 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:26.527509 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:26.527509 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:26.527509 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:26.528633 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:26.528633 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:26.528633 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:26.528633 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:26.528633 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:26.528633 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:26.528633 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:26.528633 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:26.528633 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:26.528633 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:26.528633 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:26.528633 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:26.528633 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:26.528633 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:26.528633 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:26.528633 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:26.528633 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:26.529706 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:26.529706 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:26.529706 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:26.529970 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 356/689] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:26.539577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c3dc2-1343-45d5-848b-5dbee7f04b7f None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:26.700924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab98dc3-7840-427e-81cd-da8e9c9c63c6 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:26.737254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af46c47c-214a-4293-98d5-b407402aff30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:26.737865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46c47c-214a-4293-98d5-b407402aff30 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:26.738139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46c47c-214a-4293-98d5-b407402aff30 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:26.738410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46c47c-214a-4293-98d5-b407402aff30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:26.741850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c3dc2-1343-45d5-848b-5dbee7f04b7f None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:26.789706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:26.800695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a8a50-14ca-4e5e-83b6-fc07ca64217e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:26.801310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a8a50-14ca-4e5e-83b6-fc07ca64217e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:26.801590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a8a50-14ca-4e5e-83b6-fc07ca64217e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:26.801861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a8a50-14ca-4e5e-83b6-fc07ca64217e None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:26.802515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a8a50-14ca-4e5e-83b6-fc07ca64217e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:26.806481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a8a50-14ca-4e5e-83b6-fc07ca64217e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:26.807882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a8a50-14ca-4e5e-83b6-fc07ca64217e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:26.838802 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 357/690] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:26.845966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af46c47c-214a-4293-98d5-b407402aff30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:26.847499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af46c47c-214a-4293-98d5-b407402aff30 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:26.854098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a8a50-14ca-4e5e-83b6-fc07ca64217e None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:26.878529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:26.975792 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 334/691] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:33:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 01 20:33:26.985835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af46c47c-214a-4293-98d5-b407402aff30 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:27.029548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a8a50-14ca-4e5e-83b6-fc07ca64217e None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:27.067544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3890a64-b252-4548-9894-ad7bcfa23c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:27.068205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3890a64-b252-4548-9894-ad7bcfa23c09 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:27.068582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3890a64-b252-4548-9894-ad7bcfa23c09 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:27.068861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3890a64-b252-4548-9894-ad7bcfa23c09 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:27.087264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3890a64-b252-4548-9894-ad7bcfa23c09 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:27.088714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3890a64-b252-4548-9894-ad7bcfa23c09 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:27.107996 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-f3890a64-b252-4548-9894-ad7bcfa23c09 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 01 20:33:27.107996 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:27.107996 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:27.107996 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:27.107996 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:27.107996 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:27.107996 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:27.107996 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:27.107996 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:27.107996 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:27.107996 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:27.107996 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:27.107996 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:27.107996 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:27.107996 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:27.107996 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:27.109252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:27.109252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:27.109252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:27.109252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:27.109252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:27.109252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:27.109252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:27.109252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:27.109252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:27.109252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:27.109252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:27.109252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:27.109252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:27.109252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:27.109252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:27.109252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:27.109252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:27.109252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:27.110790 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:27.110790 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:27.110790 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:27.110790 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:27.110790 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:27.110790 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:27.110790 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:27.110790 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:27.110790 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 01 20:33:27.110790 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:27.110790 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 358/692] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Oct 1 20:33:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:27.123120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3890a64-b252-4548-9894-ad7bcfa23c09 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:27.123635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:27.203495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af46c47c-214a-4293-98d5-b407402aff30 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:27.205892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:27.245791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3890a64-b252-4548-9894-ad7bcfa23c09 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:27.273765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74366b57-0f60-4bd8-b46d-263ae5208b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:27.276218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74366b57-0f60-4bd8-b46d-263ae5208b82 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:27.276644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74366b57-0f60-4bd8-b46d-263ae5208b82 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:27.277825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74366b57-0f60-4bd8-b46d-263ae5208b82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:27.299448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6397a8-4dd3-48b6-a11a-3febd9f00fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:27.300190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6397a8-4dd3-48b6-a11a-3febd9f00fb4 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:27.300584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6397a8-4dd3-48b6-a11a-3febd9f00fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:27.300949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6397a8-4dd3-48b6-a11a-3febd9f00fb4 None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:27.301770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6397a8-4dd3-48b6-a11a-3febd9f00fb4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:27.302170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6397a8-4dd3-48b6-a11a-3febd9f00fb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:27.303931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6397a8-4dd3-48b6-a11a-3febd9f00fb4 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:27.311336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74366b57-0f60-4bd8-b46d-263ae5208b82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:27.312807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74366b57-0f60-4bd8-b46d-263ae5208b82 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:27.325256 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 359/693] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Oct 1 20:33:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:27.332871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6397a8-4dd3-48b6-a11a-3febd9f00fb4 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:27.345270 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-74366b57-0f60-4bd8-b46d-263ae5208b82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:27.345270 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:27.345270 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:27.345270 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:27.345270 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:27.345270 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:27.345270 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:27.345270 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:27.345270 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:27.345270 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:27.345270 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:27.345270 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:27.345270 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:27.345270 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:27.345270 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:27.345270 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:27.346472 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:27.346472 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:27.346472 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:27.346472 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:27.346472 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:27.346472 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:27.346472 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:27.346472 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:27.346472 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:27.346472 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:27.346472 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:27.346472 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:27.346472 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:27.346472 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:27.346472 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:27.346472 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:27.346472 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:27.347454 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:27.347454 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:27.347454 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:27.351236 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 335/694] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:27.359444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74366b57-0f60-4bd8-b46d-263ae5208b82 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:27.508423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6397a8-4dd3-48b6-a11a-3febd9f00fb4 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:27.530176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74366b57-0f60-4bd8-b46d-263ae5208b82 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:27.545167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:27.557009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:27.560405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25712288-a3e9-4136-b16a-d64351b36e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:27.563043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25712288-a3e9-4136-b16a-d64351b36e96 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:27.564323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25712288-a3e9-4136-b16a-d64351b36e96 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:27.564803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25712288-a3e9-4136-b16a-d64351b36e96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:27.571770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd922a77-6652-488e-8f06-6b8c8a48d74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:27.572682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd922a77-6652-488e-8f06-6b8c8a48d74e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:27.573125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd922a77-6652-488e-8f06-6b8c8a48d74e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:27.573561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd922a77-6652-488e-8f06-6b8c8a48d74e None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:27.574385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd922a77-6652-488e-8f06-6b8c8a48d74e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:27.574952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd922a77-6652-488e-8f06-6b8c8a48d74e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:27.576658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd922a77-6652-488e-8f06-6b8c8a48d74e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:27.591589 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 336/695] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:27.593157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377ff5b-701a-419d-8434-d4244b7f4958 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:27.593701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377ff5b-701a-419d-8434-d4244b7f4958 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:27.594161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377ff5b-701a-419d-8434-d4244b7f4958 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:27.599919 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 337/696] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:27.601003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377ff5b-701a-419d-8434-d4244b7f4958 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:27.643656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25712288-a3e9-4136-b16a-d64351b36e96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:27.645227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25712288-a3e9-4136-b16a-d64351b36e96 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:27.720251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377ff5b-701a-419d-8434-d4244b7f4958 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:27.758236 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 360/697] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:33:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:33:27.759103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c5543-3ec7-45e6-8bf3-d6fc1d2cc153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:27.759676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7490fba-8835-4f92-9c86-e6f5820ee9b9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:27.760035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7490fba-8835-4f92-9c86-e6f5820ee9b9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:27.760357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c5543-3ec7-45e6-8bf3-d6fc1d2cc153 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:27.760472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7490fba-8835-4f92-9c86-e6f5820ee9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:27.760911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c5543-3ec7-45e6-8bf3-d6fc1d2cc153 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:27.761361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c5543-3ec7-45e6-8bf3-d6fc1d2cc153 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:27.769767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c5543-3ec7-45e6-8bf3-d6fc1d2cc153 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:27.771525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c5543-3ec7-45e6-8bf3-d6fc1d2cc153 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:27.780116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a7490fba-8835-4f92-9c86-e6f5820ee9b9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:27.781375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a7490fba-8835-4f92-9c86-e6f5820ee9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:27.784591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-b17c5543-3ec7-45e6-8bf3-d6fc1d2cc153 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:27.784591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:27.784591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:27.784591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:27.784591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:27.784591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:27.784591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:27.784591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:27.784591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:27.784591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:27.784591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:27.784591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:27.784591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:27.784591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:27.784591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:27.785489 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:27.785489 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:27.785489 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:27.785489 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:27.785489 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:27.785489 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:27.785489 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:27.785489 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:27.785489 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:27.785489 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:27.785489 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:27.785489 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:27.785489 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:27.785489 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:27.785489 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:27.785489 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:27.785489 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:27.786394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:27.786394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:27.786394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:27.786394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:27.786394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:27.786394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:27.786394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:27.786394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:27.788495 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 338/698] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:27.789516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c5543-3ec7-45e6-8bf3-d6fc1d2cc153 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:27.810083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:27.840019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:27.861892 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 361/699] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:27.861892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7490fba-8835-4f92-9c86-e6f5820ee9b9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:27.940450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c5543-3ec7-45e6-8bf3-d6fc1d2cc153 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:27.993493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7490fba-8835-4f92-9c86-e6f5820ee9b9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:27.999857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602aa62e-e444-4632-be2f-717bcc6da03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:28.003776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602aa62e-e444-4632-be2f-717bcc6da03e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:28.004220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602aa62e-e444-4632-be2f-717bcc6da03e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:28.004647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602aa62e-e444-4632-be2f-717bcc6da03e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:28.021872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602aa62e-e444-4632-be2f-717bcc6da03e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:28.023557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602aa62e-e444-4632-be2f-717bcc6da03e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:28.054091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-602aa62e-e444-4632-be2f-717bcc6da03e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:28.054091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:28.054091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:28.054091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:28.054091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:28.054091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:28.054091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:28.054091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:28.054091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:28.054091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:28.054091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:28.054091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:28.054091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:28.054091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:28.054091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:28.054091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:28.055302 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:28.055302 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:28.055302 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:28.055302 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:28.055302 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:28.055302 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:28.055302 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:28.055302 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:28.055302 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:28.055302 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:28.055302 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:28.055302 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:28.055302 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:28.055302 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:28.055302 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:28.055302 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:28.055302 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:28.056221 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:28.056221 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:28.056221 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:28.058382 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 339/700] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:28.070454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602aa62e-e444-4632-be2f-717bcc6da03e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:28.084298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc6d59-d613-408e-88f0-7a226292851e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:28.085117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc6d59-d613-408e-88f0-7a226292851e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:28.085502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc6d59-d613-408e-88f0-7a226292851e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:28.085834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc6d59-d613-408e-88f0-7a226292851e None admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:28.086560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc6d59-d613-408e-88f0-7a226292851e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:28.086869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc6d59-d613-408e-88f0-7a226292851e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:28.088298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc6d59-d613-408e-88f0-7a226292851e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:28.102468 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 362/701] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:28.108169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc6d59-d613-408e-88f0-7a226292851e None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:28.170855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:28.243439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:28.255664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602aa62e-e444-4632-be2f-717bcc6da03e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:28.304414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc6d59-d613-408e-88f0-7a226292851e None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:28.305738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf62ac-b791-4843-aa19-024c34ea4772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:28.306368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf62ac-b791-4843-aa19-024c34ea4772 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:28.306690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf62ac-b791-4843-aa19-024c34ea4772 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:28.306968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf62ac-b791-4843-aa19-024c34ea4772 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:28.307636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf62ac-b791-4843-aa19-024c34ea4772 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:28.308011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf62ac-b791-4843-aa19-024c34ea4772 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:28.309809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf62ac-b791-4843-aa19-024c34ea4772 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:28.324597 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 340/702] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:28.325586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf62ac-b791-4843-aa19-024c34ea4772 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:28.353891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8466da-a94b-4639-8af1-92d9c630b636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:28.355152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8466da-a94b-4639-8af1-92d9c630b636 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:28.355687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8466da-a94b-4639-8af1-92d9c630b636 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:28.356217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8466da-a94b-4639-8af1-92d9c630b636 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:28.357238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8466da-a94b-4639-8af1-92d9c630b636 None admin] RBAC: Adding query filter params (role.id=ef7bd7293369475a9e7b3444fac30cda, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=c4dc51e2b4474d6ba9fd5d2bc3702826) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:28.357769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8466da-a94b-4639-8af1-92d9c630b636 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:28.360254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8466da-a94b-4639-8af1-92d9c630b636 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:28.368373 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 363/703] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:33:28 2022] GET /identity/v3/role_assignments?user.id=c4dc51e2b4474d6ba9fd5d2bc3702826&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=ef7bd7293369475a9e7b3444fac30cda => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:28.369600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8466da-a94b-4639-8af1-92d9c630b636 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:28.442318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf62ac-b791-4843-aa19-024c34ea4772 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:28.479431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77139761-0270-45cf-8e34-b20952aa0141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:28.480075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77139761-0270-45cf-8e34-b20952aa0141 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:28.480452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77139761-0270-45cf-8e34-b20952aa0141 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:28.480655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77139761-0270-45cf-8e34-b20952aa0141 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:28.481200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77139761-0270-45cf-8e34-b20952aa0141 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:28.482606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77139761-0270-45cf-8e34-b20952aa0141 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:28.486945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:28.494394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-77139761-0270-45cf-8e34-b20952aa0141 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 01 20:33:28.494394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:28.494394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:33:28.494394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:33:28.494394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:33:28.494394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:33:28.494394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:33:28.494394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:28.494394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 01 20:33:28.494394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:28.494394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:33:28.494394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:28.494394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:28.494394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:28.494394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:28.494394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:28.494394 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:28.495591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:28.495591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:28.495591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:28.495591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:28.495591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:28.495591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:28.495591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:33:28.495591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:33:28.495591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:33:28.495591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:33:28.495591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:28.495591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:28.495591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:28.495591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:28.495591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:28.495591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:28.496503 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:28.496503 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:28.496503 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:28.496503 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:28.496503 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:28.496503 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:28.496503 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:28.496503 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:28.496503 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:28.496503 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:28.496503 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:33:28.496503 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:28.496503 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 01 20:33:28.496503 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:28.496503 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 341/704] 10.222.0.17 () {56 vars in 1129 bytes} [Sat Oct 1 20:33:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:28.501576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77139761-0270-45cf-8e34-b20952aa0141 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:28.511909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8466da-a94b-4639-8af1-92d9c630b636 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:28.523184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:28.559625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38de1c2-36c6-4484-9c73-c0c12712a672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:28.560716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38de1c2-36c6-4484-9c73-c0c12712a672 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:28.561203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38de1c2-36c6-4484-9c73-c0c12712a672 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:28.561679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38de1c2-36c6-4484-9c73-c0c12712a672 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:28.572143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38de1c2-36c6-4484-9c73-c0c12712a672 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:28.574193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38de1c2-36c6-4484-9c73-c0c12712a672 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:28.591373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-f38de1c2-36c6-4484-9c73-c0c12712a672 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:28.591373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:28.591373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:28.591373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:28.591373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:28.591373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:28.591373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:28.591373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:28.591373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:28.591373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:28.591373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:28.591373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:28.591373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:28.591373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:28.591373 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:28.592335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:28.592335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:28.592335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:28.592335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:28.592335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:28.592335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:28.592335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:28.592335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:28.592335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:28.592335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:28.592335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:28.592335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:28.592335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:28.592335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:28.592335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:28.592335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:28.592335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:28.593224 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:28.593224 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:28.593224 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:28.593224 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:28.593224 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:28.593224 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:28.593224 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:28.593224 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:28.596378 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 364/705] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:28.604302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38de1c2-36c6-4484-9c73-c0c12712a672 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:28.628828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77139761-0270-45cf-8e34-b20952aa0141 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:28.676330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f2e56-c19c-4a0f-ab04-2f9066adf010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:28.677540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f2e56-c19c-4a0f-ab04-2f9066adf010 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:28.678220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f2e56-c19c-4a0f-ab04-2f9066adf010 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:28.678958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f2e56-c19c-4a0f-ab04-2f9066adf010 None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:28.681650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f2e56-c19c-4a0f-ab04-2f9066adf010 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:28.682942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f2e56-c19c-4a0f-ab04-2f9066adf010 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:28.687539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f2e56-c19c-4a0f-ab04-2f9066adf010 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:28.698881 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 342/706] 10.222.0.17 () {56 vars in 1124 bytes} [Sat Oct 1 20:33:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:28.701247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f2e56-c19c-4a0f-ab04-2f9066adf010 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:28.801584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:28.802195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38de1c2-36c6-4484-9c73-c0c12712a672 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:28.821507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f2e56-c19c-4a0f-ab04-2f9066adf010 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:28.841248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1523be6-7dfa-48b1-8df1-5acc1609ecee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:28.841860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1523be6-7dfa-48b1-8df1-5acc1609ecee None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:28.842140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1523be6-7dfa-48b1-8df1-5acc1609ecee None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:28.842409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1523be6-7dfa-48b1-8df1-5acc1609ecee None admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:28.843143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1523be6-7dfa-48b1-8df1-5acc1609ecee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:28.843453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1523be6-7dfa-48b1-8df1-5acc1609ecee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:28.844943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1523be6-7dfa-48b1-8df1-5acc1609ecee None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:28.856913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8ceea-f045-4948-add7-4bf1659cbecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:28.857542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8ceea-f045-4948-add7-4bf1659cbecc None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:28.857821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8ceea-f045-4948-add7-4bf1659cbecc None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:28.858090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8ceea-f045-4948-add7-4bf1659cbecc None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:28.858740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8ceea-f045-4948-add7-4bf1659cbecc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:28.859041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8ceea-f045-4948-add7-4bf1659cbecc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:28.860421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8ceea-f045-4948-add7-4bf1659cbecc None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:28.861345 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 365/707] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:28.862852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9986e22-c8d6-4f0c-a42f-d5471c7d47aa None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:28.863138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9986e22-c8d6-4f0c-a42f-d5471c7d47aa None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:28.863413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9986e22-c8d6-4f0c-a42f-d5471c7d47aa None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:28.864768 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 366/708] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:28.865597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ff381-5afd-4b66-9fd7-5cbf3526bdd9 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:28.865877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ff381-5afd-4b66-9fd7-5cbf3526bdd9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:28.866141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ff381-5afd-4b66-9fd7-5cbf3526bdd9 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:28.867609 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 367/709] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:28.870847 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 343/710] 10.222.0.17 () {56 vars in 1124 bytes} [Sat Oct 1 20:33:28 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:28.872684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a217c68-fd74-4df8-aa72-9cde14b418c0 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:28.872884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5532fb-5f38-46af-a951-869e6ccd69c8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:28.872986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a217c68-fd74-4df8-aa72-9cde14b418c0 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:28.873141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5532fb-5f38-46af-a951-869e6ccd69c8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:28.873236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a217c68-fd74-4df8-aa72-9cde14b418c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:28.873423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5532fb-5f38-46af-a951-869e6ccd69c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:28.894195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9f5532fb-5f38-46af-a951-869e6ccd69c8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:28.895533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9f5532fb-5f38-46af-a951-869e6ccd69c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:28.895651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7a217c68-fd74-4df8-aa72-9cde14b418c0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:28.896852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7a217c68-fd74-4df8-aa72-9cde14b418c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:28.899508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:28.976689 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 344/711] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:28.977945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5532fb-5f38-46af-a951-869e6ccd69c8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:29.006795 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 368/712] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:29.113524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:29.141079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5532fb-5f38-46af-a951-869e6ccd69c8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:29.179972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2b9bd-73cc-4ce7-8a69-5c13506c994c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:29.180581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2b9bd-73cc-4ce7-8a69-5c13506c994c None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:29.180865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2b9bd-73cc-4ce7-8a69-5c13506c994c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:29.181142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2b9bd-73cc-4ce7-8a69-5c13506c994c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:29.181877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2b9bd-73cc-4ce7-8a69-5c13506c994c None admin] RBAC: Adding query filter params (role.id=1c50896511054a5f994bee3d99fa0619, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=7ab975dafb564aa2b357311c4a384d53) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:29.182185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2b9bd-73cc-4ce7-8a69-5c13506c994c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:29.183777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2b9bd-73cc-4ce7-8a69-5c13506c994c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:29.190510 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 345/713] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:33:28 2022] GET /identity/v3/role_assignments?user.id=7ab975dafb564aa2b357311c4a384d53&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=1c50896511054a5f994bee3d99fa0619 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:29.377902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a217c68-fd74-4df8-aa72-9cde14b418c0 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:29.461930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2b9bd-73cc-4ce7-8a69-5c13506c994c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:29.490287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a217c68-fd74-4df8-aa72-9cde14b418c0 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:29.533493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcede95b-6077-4ea4-84a3-a05743798db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:29.534617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcede95b-6077-4ea4-84a3-a05743798db2 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:29.535116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcede95b-6077-4ea4-84a3-a05743798db2 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:29.535641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcede95b-6077-4ea4-84a3-a05743798db2 None admin] PATH_INFO: `/v3/services/network` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:29.536406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcede95b-6077-4ea4-84a3-a05743798db2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:29.538183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcede95b-6077-4ea4-84a3-a05743798db2 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:29.548554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:29.558187 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-dcede95b-6077-4ea4-84a3-a05743798db2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 01 20:33:29.558187 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:29.558187 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:33:29.558187 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:33:29.558187 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:33:29.558187 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:33:29.558187 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:33:29.558187 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:29.558187 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 01 20:33:29.558187 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:29.558187 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:33:29.558187 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:29.558187 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:29.558187 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:29.558187 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:29.558187 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:29.558187 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:29.559321 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:29.559321 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:29.559321 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:29.559321 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:29.559321 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:29.559321 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:29.559321 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:33:29.559321 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:33:29.559321 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:33:29.559321 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:33:29.559321 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:29.559321 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:29.559321 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:29.559321 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:29.559321 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:29.559321 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:29.560167 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:29.560167 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:29.560167 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:29.560167 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:29.560167 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:29.560167 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:29.560167 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:29.560167 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:29.560167 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:29.560167 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:29.560167 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:33:29.560167 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:29.560167 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 01 20:33:29.560167 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:29.562143 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 369/714] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:29 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:29.569775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcede95b-6077-4ea4-84a3-a05743798db2 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:29.611117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2b9bd-73cc-4ce7-8a69-5c13506c994c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:29.652667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfc79f-a691-4f54-b027-419a7abba825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:29.654390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfc79f-a691-4f54-b027-419a7abba825 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:29.655429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfc79f-a691-4f54-b027-419a7abba825 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:29.656052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfc79f-a691-4f54-b027-419a7abba825 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:29.674826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:29.677997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcfc79f-a691-4f54-b027-419a7abba825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:29.680719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcfc79f-a691-4f54-b027-419a7abba825 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:29.709403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-9bcfc79f-a691-4f54-b027-419a7abba825 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:29.709403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:29.709403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:29.709403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:29.709403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:29.709403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:29.709403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:29.709403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:29.709403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:29.709403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:29.709403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:29.709403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:29.709403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:29.709403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:29.709403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:29.709403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:29.710426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:29.710426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:29.710426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:29.710426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:29.710426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:29.710426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:29.710426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:29.710426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:29.710426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:29.710426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:29.710426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:29.710426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:29.710426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:29.710426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:29.710426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:29.710426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:29.710426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:29.711352 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:29.711352 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:29.711352 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:29.711952 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 346/715] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:29.717900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfc79f-a691-4f54-b027-419a7abba825 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:29.798766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcede95b-6077-4ea4-84a3-a05743798db2 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:29.853335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985a883-e34d-45f2-94cf-3532759e0c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:29.853948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985a883-e34d-45f2-94cf-3532759e0c43 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:29.854225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985a883-e34d-45f2-94cf-3532759e0c43 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:29.854514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985a883-e34d-45f2-94cf-3532759e0c43 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:29.855189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985a883-e34d-45f2-94cf-3532759e0c43 None admin] RBAC: Adding query filter params (name=network) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:29.855492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985a883-e34d-45f2-94cf-3532759e0c43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:29.857596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985a883-e34d-45f2-94cf-3532759e0c43 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:29.863246 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 370/716] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Oct 1 20:33:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:29.868308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985a883-e34d-45f2-94cf-3532759e0c43 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:29.909012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfc79f-a691-4f54-b027-419a7abba825 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:29.971863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:29.980123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad760cdd-ca9f-4449-877e-51419e6f7e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:29.980770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad760cdd-ca9f-4449-877e-51419e6f7e54 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:29.981058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad760cdd-ca9f-4449-877e-51419e6f7e54 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:29.981366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad760cdd-ca9f-4449-877e-51419e6f7e54 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:29.983113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad760cdd-ca9f-4449-877e-51419e6f7e54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:29.983113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad760cdd-ca9f-4449-877e-51419e6f7e54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:29.984341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad760cdd-ca9f-4449-877e-51419e6f7e54 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:30.004289 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 347/717] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:30.011353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad760cdd-ca9f-4449-877e-51419e6f7e54 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:30.020328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985a883-e34d-45f2-94cf-3532759e0c43 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:30.067775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56d8d4-4f60-4534-91bf-932a1b56e1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:30.068636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56d8d4-4f60-4534-91bf-932a1b56e1b9 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:30.069115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56d8d4-4f60-4534-91bf-932a1b56e1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:30.069539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56d8d4-4f60-4534-91bf-932a1b56e1b9 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:30.070339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56d8d4-4f60-4534-91bf-932a1b56e1b9 None admin] RBAC: Adding query filter params (type=network) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:30.070828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56d8d4-4f60-4534-91bf-932a1b56e1b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:30.072689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56d8d4-4f60-4534-91bf-932a1b56e1b9 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:30.079713 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 371/718] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Oct 1 20:33:29 2022] GET /identity/v3/services?type=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:30.104386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:30.158910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad760cdd-ca9f-4449-877e-51419e6f7e54 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:30.209817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243a7d3-c4f5-4000-aa45-70f9d6496581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:30.210755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243a7d3-c4f5-4000-aa45-70f9d6496581 None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:30.211201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243a7d3-c4f5-4000-aa45-70f9d6496581 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:30.211620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243a7d3-c4f5-4000-aa45-70f9d6496581 None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:30.212432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243a7d3-c4f5-4000-aa45-70f9d6496581 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:30.214058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243a7d3-c4f5-4000-aa45-70f9d6496581 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:30.249549 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 348/719] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Oct 1 20:33:30 2022] POST /identity/v3/users => generated 254 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:33:31.496026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f54ce0-4c45-4db4-84e5-120dd83d1ba1 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:31.496570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f54ce0-4c45-4db4-84e5-120dd83d1ba1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:31.496570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f54ce0-4c45-4db4-84e5-120dd83d1ba1 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:31.498178 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 372/720] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:31.503596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58cf84-b650-459d-b10e-ce1dd7de8a05 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:31.504003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58cf84-b650-459d-b10e-ce1dd7de8a05 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:31.504174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58cf84-b650-459d-b10e-ce1dd7de8a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:31.531609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9e58cf84-b650-459d-b10e-ce1dd7de8a05 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:31.533421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9e58cf84-b650-459d-b10e-ce1dd7de8a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:31.624930 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 349/721] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:31.704459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56c580-01be-4dcb-b402-870ed66944b5 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:31.704875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56c580-01be-4dcb-b402-870ed66944b5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:31.705067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56c580-01be-4dcb-b402-870ed66944b5 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:31.706620 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 373/722] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:31.711884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb25d63-206b-4d66-8901-6f45d6ba4d7b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:31.712173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb25d63-206b-4d66-8901-6f45d6ba4d7b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:31.712447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb25d63-206b-4d66-8901-6f45d6ba4d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:31.739773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1cb25d63-206b-4d66-8901-6f45d6ba4d7b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:31.740977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1cb25d63-206b-4d66-8901-6f45d6ba4d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:31.784062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:31.840226 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 350/723] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:32.085652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56c580-01be-4dcb-b402-870ed66944b5 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:32.201964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56c580-01be-4dcb-b402-870ed66944b5 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:32.229078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:32.236800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2df3ad-33d8-45bb-a6be-6d5a95e828c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:32.237434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2df3ad-33d8-45bb-a6be-6d5a95e828c1 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:32.237705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2df3ad-33d8-45bb-a6be-6d5a95e828c1 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:32.237976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2df3ad-33d8-45bb-a6be-6d5a95e828c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:32.248692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2df3ad-33d8-45bb-a6be-6d5a95e828c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:32.253454 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-bb2df3ad-33d8-45bb-a6be-6d5a95e828c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:33:32.253454 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:32.253454 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:32.253454 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:32.253454 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:32.253454 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:32.253454 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:32.253454 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:32.253454 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:32.253454 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:32.253454 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:32.253454 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:32.253454 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:32.253454 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:32.253454 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:32.253454 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:32.254379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:32.254379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:32.254379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:32.254379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:32.254379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:32.254379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:32.254379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:32.254379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:32.254379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:32.254379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:32.254379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:32.254379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:32.254379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:32.254379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:32.254379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:32.254379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:32.255281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:32.255281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:32.255281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:32.255281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:32.255281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:32.255281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:32.255281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:32.255281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:32.255281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:32.255281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:32.255281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:33:32.255281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:32.255281 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 374/724] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:33:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:32.263983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb25d63-206b-4d66-8901-6f45d6ba4d7b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:32.313496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74be1e-1ba5-40af-b290-206d837c97b6 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:32.313780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74be1e-1ba5-40af-b290-206d837c97b6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:32.313962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74be1e-1ba5-40af-b290-206d837c97b6 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:32.315308 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 375/725] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:32.326226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab18b8-e276-439e-ba4b-b101b66c6611 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:32.326533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab18b8-e276-439e-ba4b-b101b66c6611 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:32.326799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab18b8-e276-439e-ba4b-b101b66c6611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:32.352990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2bab18b8-e276-439e-ba4b-b101b66c6611 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:32.354893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2bab18b8-e276-439e-ba4b-b101b66c6611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:32.416959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb25d63-206b-4d66-8901-6f45d6ba4d7b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:32.442200 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 376/726] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:33:32.448200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab18b8-e276-439e-ba4b-b101b66c6611 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:32.475403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7018c3-badc-412e-a09f-2f967a5270f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:32.476018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7018c3-badc-412e-a09f-2f967a5270f5 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:32.476302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7018c3-badc-412e-a09f-2f967a5270f5 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:32.476582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7018c3-badc-412e-a09f-2f967a5270f5 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:32.477260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7018c3-badc-412e-a09f-2f967a5270f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:32.477624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7018c3-badc-412e-a09f-2f967a5270f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:32.479039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7018c3-badc-412e-a09f-2f967a5270f5 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:32.495657 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 351/727] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:33:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:32.499964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7018c3-badc-412e-a09f-2f967a5270f5 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:32.559890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:32.642360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab18b8-e276-439e-ba4b-b101b66c6611 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:32.666400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7018c3-badc-412e-a09f-2f967a5270f5 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:32.677980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:32.710284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155061a6-d064-4007-bd97-1dfcd0d73497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:32.710955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155061a6-d064-4007-bd97-1dfcd0d73497 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:32.711235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155061a6-d064-4007-bd97-1dfcd0d73497 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:32.711510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155061a6-d064-4007-bd97-1dfcd0d73497 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:32.723236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c09328-e65e-4293-8dc8-4cf327a2efe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:32.723844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c09328-e65e-4293-8dc8-4cf327a2efe1 None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:32.724117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c09328-e65e-4293-8dc8-4cf327a2efe1 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:32.724380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c09328-e65e-4293-8dc8-4cf327a2efe1 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:32.725026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c09328-e65e-4293-8dc8-4cf327a2efe1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:32.726389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c09328-e65e-4293-8dc8-4cf327a2efe1 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:32.730652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155061a6-d064-4007-bd97-1dfcd0d73497 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:32.732072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155061a6-d064-4007-bd97-1dfcd0d73497 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:32.753554 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-155061a6-d064-4007-bd97-1dfcd0d73497 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:32.753554 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:32.753554 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:32.753554 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:32.753554 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:32.753554 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:32.753554 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:32.753554 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:32.753554 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:32.753554 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:32.753554 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:32.753554 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:32.753554 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:32.753554 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:32.753554 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:32.753554 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:32.754764 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:32.754764 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:32.754764 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:32.754764 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:32.754764 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:32.754764 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:32.754764 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:32.754764 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:32.754764 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:32.754764 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:32.754764 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:32.754764 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:32.754764 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:32.754764 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:32.754764 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:32.754764 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:32.754764 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:32.755841 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:32.755841 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:32.755841 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:32.755841 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 352/728] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:32.759229 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 377/729] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Oct 1 20:33:32 2022] POST /identity/v3/services => generated 251 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:33:32.769608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155061a6-d064-4007-bd97-1dfcd0d73497 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:32.879958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:32.912742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155061a6-d064-4007-bd97-1dfcd0d73497 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:32.933872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c09328-e65e-4293-8dc8-4cf327a2efe1 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:32.962664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd2735-1625-46c8-8b71-7744b39346ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:32.963567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd2735-1625-46c8-8b71-7744b39346ad None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:32.963991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd2735-1625-46c8-8b71-7744b39346ad None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:32.964413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd2735-1625-46c8-8b71-7744b39346ad None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:32.965206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd2735-1625-46c8-8b71-7744b39346ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:32.965666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd2735-1625-46c8-8b71-7744b39346ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:32.967226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd2735-1625-46c8-8b71-7744b39346ad None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:32.980672 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 353/730] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:32.987431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd2735-1625-46c8-8b71-7744b39346ad None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:33.094918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c09328-e65e-4293-8dc8-4cf327a2efe1 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:33.097408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd2735-1625-46c8-8b71-7744b39346ad None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:33.135432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b105d8-e3ec-4f6c-ad9c-5548310464e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:33.135753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b105d8-e3ec-4f6c-ad9c-5548310464e9 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:33.136707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b105d8-e3ec-4f6c-ad9c-5548310464e9 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:33.136707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b105d8-e3ec-4f6c-ad9c-5548310464e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:33.136898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21df29-3c86-4fc9-9d78-325ee9f59262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:33.137369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21df29-3c86-4fc9-9d78-325ee9f59262 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:33.137646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21df29-3c86-4fc9-9d78-325ee9f59262 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:33.137914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21df29-3c86-4fc9-9d78-325ee9f59262 None admin] PATH_INFO: `/v3/users/swift` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:33.148438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b105d8-e3ec-4f6c-ad9c-5548310464e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:33.149429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21df29-3c86-4fc9-9d78-325ee9f59262 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:33.150919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21df29-3c86-4fc9-9d78-325ee9f59262 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:33.153990 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-70b105d8-e3ec-4f6c-ad9c-5548310464e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:33:33.153990 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:33.153990 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:33.153990 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:33.153990 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:33.153990 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:33.153990 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:33.153990 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:33.153990 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:33.153990 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:33.153990 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:33.153990 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:33.153990 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:33.153990 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:33.153990 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:33.153990 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:33.154952 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:33.154952 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:33.154952 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:33.154952 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:33.154952 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:33.154952 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:33.154952 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:33.154952 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:33.154952 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:33.154952 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:33.154952 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:33.154952 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:33.154952 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:33.154952 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:33.154952 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:33.154952 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:33.155783 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:33.155783 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:33.155783 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:33.155783 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:33.155783 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:33.155783 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:33.155783 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:33.155783 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:33.155783 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:33.155783 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:33.155783 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:33:33.155783 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:33.155783 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 378/731] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:33:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:33.156495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b1510-a589-4a17-920d-77240f1e9ba5 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:33.157696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b1510-a589-4a17-920d-77240f1e9ba5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:33.157696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b1510-a589-4a17-920d-77240f1e9ba5 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:33.159068 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 379/732] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:33.159493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:33.164223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b1510-a589-4a17-920d-77240f1e9ba5 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:33.173691 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-2f21df29-3c86-4fc9-9d78-325ee9f59262 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 01 20:33:33.173691 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:33.173691 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:33.173691 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:33.173691 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:33.173691 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:33.173691 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:33.173691 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:33.173691 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:33.173691 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:33.173691 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:33.173691 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:33.173691 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:33.173691 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:33.173691 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:33.173691 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:33.174730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:33.174730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:33.174730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:33.174730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:33.174730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:33.174730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:33.174730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:33.174730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:33.174730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:33.174730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:33.174730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:33.174730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:33.174730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:33.174730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:33.174730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:33.174730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:33.174730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:33.174730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:33.176185 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:33.176185 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:33.176185 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:33.176185 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:33.176185 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:33.176185 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:33.176185 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:33.176185 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:33.176185 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 01 20:33:33.176185 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:33.176185 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 354/733] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:33:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:33.182269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4460b-2952-48a1-bdfc-6d5e19780b6c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:33.182734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4460b-2952-48a1-bdfc-6d5e19780b6c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:33.183015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4460b-2952-48a1-bdfc-6d5e19780b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:33.208209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-01f4460b-2952-48a1-bdfc-6d5e19780b6c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:33.209432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-01f4460b-2952-48a1-bdfc-6d5e19780b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:33.220321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:33.288559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b1510-a589-4a17-920d-77240f1e9ba5 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:33.308135 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 355/734] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:33 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:33:33.309126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4460b-2952-48a1-bdfc-6d5e19780b6c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:33.326406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f64fe-558a-4f50-82c6-f6cac06bfd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:33.327381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f64fe-558a-4f50-82c6-f6cac06bfd32 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:33.327972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f64fe-558a-4f50-82c6-f6cac06bfd32 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:33.328340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f64fe-558a-4f50-82c6-f6cac06bfd32 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:33.330062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f64fe-558a-4f50-82c6-f6cac06bfd32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:33.330462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f64fe-558a-4f50-82c6-f6cac06bfd32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:33.331952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f64fe-558a-4f50-82c6-f6cac06bfd32 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:33.352813 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 380/735] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:33:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:33.358406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f64fe-558a-4f50-82c6-f6cac06bfd32 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:33.458573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4460b-2952-48a1-bdfc-6d5e19780b6c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:33.504460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:33.519707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f64fe-558a-4f50-82c6-f6cac06bfd32 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:33.521668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:33.522549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea4fae-4a6a-4f79-b5e4-624fac00d71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:33.523164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea4fae-4a6a-4f79-b5e4-624fac00d71e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:33.523431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea4fae-4a6a-4f79-b5e4-624fac00d71e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:33.523700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea4fae-4a6a-4f79-b5e4-624fac00d71e None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:33.524425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea4fae-4a6a-4f79-b5e4-624fac00d71e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:33.524726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea4fae-4a6a-4f79-b5e4-624fac00d71e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:33.526197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea4fae-4a6a-4f79-b5e4-624fac00d71e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:33.555927 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 356/736] 10.222.0.17 () {56 vars in 1135 bytes} [Sat Oct 1 20:33:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:33.561816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea4fae-4a6a-4f79-b5e4-624fac00d71e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:33.600427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d94cd3-afd4-4335-acaf-61dfdd20d7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:33.601058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d94cd3-afd4-4335-acaf-61dfdd20d7c3 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:33.601375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d94cd3-afd4-4335-acaf-61dfdd20d7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:33.601653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d94cd3-afd4-4335-acaf-61dfdd20d7c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:33.622402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d94cd3-afd4-4335-acaf-61dfdd20d7c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:33.623865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d94cd3-afd4-4335-acaf-61dfdd20d7c3 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:33.651792 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-01d94cd3-afd4-4335-acaf-61dfdd20d7c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:33.651792 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:33.651792 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:33.651792 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:33.651792 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:33.651792 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:33.651792 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:33.651792 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:33.651792 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:33.651792 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:33.651792 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:33.651792 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:33.651792 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:33.651792 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:33.651792 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:33.651792 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:33.654087 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:33.654087 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:33.654087 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:33.654087 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:33.654087 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:33.654087 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:33.654087 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:33.654087 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:33.654087 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:33.654087 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:33.654087 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:33.654087 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:33.654087 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:33.654087 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:33.654087 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:33.654087 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:33.654087 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:33.656228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:33.656228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:33.656228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:33.656228 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 381/737] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:33.659222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d94cd3-afd4-4335-acaf-61dfdd20d7c3 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:33.726156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea4fae-4a6a-4f79-b5e4-624fac00d71e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:33.763118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19bf42-9feb-4906-a625-0b1fa96114e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:33.763873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19bf42-9feb-4906-a625-0b1fa96114e3 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:33.764155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19bf42-9feb-4906-a625-0b1fa96114e3 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:33.764425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19bf42-9feb-4906-a625-0b1fa96114e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:33.786528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d94cd3-afd4-4335-acaf-61dfdd20d7c3 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:33.819664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:33.825265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5dd51-f9cd-4984-a8ed-cfe949a4abbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:33.825872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5dd51-f9cd-4984-a8ed-cfe949a4abbd None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:33.826174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5dd51-f9cd-4984-a8ed-cfe949a4abbd None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:33.826481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5dd51-f9cd-4984-a8ed-cfe949a4abbd None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:33.827076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5dd51-f9cd-4984-a8ed-cfe949a4abbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:33.827376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5dd51-f9cd-4984-a8ed-cfe949a4abbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:33.829254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5dd51-f9cd-4984-a8ed-cfe949a4abbd None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:33.836539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19bf42-9feb-4906-a625-0b1fa96114e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:33.838425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19bf42-9feb-4906-a625-0b1fa96114e3 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:33.844858 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 382/738] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:33.844858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5dd51-f9cd-4984-a8ed-cfe949a4abbd None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:33.860605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:33.946237 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 357/739] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:33:33 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:33:33.947971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2b930-cc6d-404b-91ff-c9859853be6d None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:33.948492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2b930-cc6d-404b-91ff-c9859853be6d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:33.949007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2b930-cc6d-404b-91ff-c9859853be6d None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:33.950716 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 358/740] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:33.951603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2b930-cc6d-404b-91ff-c9859853be6d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:33.964444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5dd51-f9cd-4984-a8ed-cfe949a4abbd None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:33.999746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8004e-4d91-42e8-ad6c-3a01e273b9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:34.000287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8004e-4d91-42e8-ad6c-3a01e273b9a4 None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:34.000630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8004e-4d91-42e8-ad6c-3a01e273b9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:34.001413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8004e-4d91-42e8-ad6c-3a01e273b9a4 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:34.002067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f8004e-4d91-42e8-ad6c-3a01e273b9a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:34.003522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f8004e-4d91-42e8-ad6c-3a01e273b9a4 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:34.028185 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 383/741] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Oct 1 20:33:33 2022] POST /identity/v3/services => generated 240 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:33:34.028185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8004e-4d91-42e8-ad6c-3a01e273b9a4 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:34.065953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:34.095510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2b930-cc6d-404b-91ff-c9859853be6d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:34.139893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d2536-4718-4bf9-b86f-4efa84d08df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:34.140626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:34.140741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d2536-4718-4bf9-b86f-4efa84d08df6 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:34.141085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d2536-4718-4bf9-b86f-4efa84d08df6 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:34.142323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d2536-4718-4bf9-b86f-4efa84d08df6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:34.153009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d2536-4718-4bf9-b86f-4efa84d08df6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:34.154834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d2536-4718-4bf9-b86f-4efa84d08df6 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:34.170587 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-6a8d2536-4718-4bf9-b86f-4efa84d08df6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 01 20:33:34.170587 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:34.170587 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:34.170587 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:34.170587 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:34.170587 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:34.170587 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:34.170587 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:34.170587 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:34.170587 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:34.170587 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:34.170587 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:34.170587 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:34.170587 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:34.170587 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:34.170587 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:34.171607 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:34.171607 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:34.171607 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:34.171607 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:34.171607 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:34.171607 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:34.171607 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:34.171607 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:34.171607 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:34.171607 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:34.171607 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:34.171607 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:34.171607 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:34.171607 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:34.171607 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:34.171607 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:34.171607 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:34.171607 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:34.172634 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:34.172634 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:34.172634 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:34.172634 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:34.172634 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:34.172634 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:34.172634 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:34.172634 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:34.172634 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 01 20:33:34.172634 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:34.177274 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 359/742] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Oct 1 20:33:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:34.178887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5efa0-24c5-4375-ab71-2fa85224eb9b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:34.179242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5efa0-24c5-4375-ab71-2fa85224eb9b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:34.179590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5efa0-24c5-4375-ab71-2fa85224eb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:34.184847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8004e-4d91-42e8-ad6c-3a01e273b9a4 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:34.201780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-54a5efa0-24c5-4375-ab71-2fa85224eb9b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:34.203397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-54a5efa0-24c5-4375-ab71-2fa85224eb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:34.233725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f55bd-87d3-4ee8-9533-2daf4901fb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:34.235019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f55bd-87d3-4ee8-9533-2daf4901fb9e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:34.235485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f55bd-87d3-4ee8-9533-2daf4901fb9e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:34.236334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f55bd-87d3-4ee8-9533-2daf4901fb9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:34.249955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f55bd-87d3-4ee8-9533-2daf4901fb9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:34.251902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f55bd-87d3-4ee8-9533-2daf4901fb9e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:34.270180 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-b80f55bd-87d3-4ee8-9533-2daf4901fb9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:34.270180 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:34.270180 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:34.270180 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:34.270180 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:34.270180 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:34.270180 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:34.270180 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:34.270180 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:34.270180 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:34.270180 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:34.270180 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:34.270180 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:34.270180 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:34.270180 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:34.270180 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:34.271206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:34.271206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:34.271206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:34.271206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:34.271206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:34.271206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:34.271206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:34.271206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:34.271206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:34.271206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:34.271206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:34.271206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:34.271206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:34.271206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:34.271206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:34.271206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:34.271206 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:34.272114 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:34.272114 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:34.272114 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:34.272114 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 384/743] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:34.272114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f55bd-87d3-4ee8-9533-2daf4901fb9e None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:34.294842 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 360/744] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:33:34.295776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a5efa0-24c5-4375-ab71-2fa85224eb9b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:34.389646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f55bd-87d3-4ee8-9533-2daf4901fb9e None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:34.415557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:34.420056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:34.434646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e34fc-1562-457a-a5b9-69e91154e72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:34.435629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e34fc-1562-457a-a5b9-69e91154e72d None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:34.436428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e34fc-1562-457a-a5b9-69e91154e72d None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:34.437194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e34fc-1562-457a-a5b9-69e91154e72d None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:34.438181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e34fc-1562-457a-a5b9-69e91154e72d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:34.438735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e34fc-1562-457a-a5b9-69e91154e72d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:34.440369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e34fc-1562-457a-a5b9-69e91154e72d None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:34.461273 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 385/745] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Oct 1 20:33:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:34.467869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e34fc-1562-457a-a5b9-69e91154e72d None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:34.499270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a5efa0-24c5-4375-ab71-2fa85224eb9b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:34.585149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e34fc-1562-457a-a5b9-69e91154e72d None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:34.591768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8d95e-0dec-4793-83bc-136c9f810b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:34.592379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8d95e-0dec-4793-83bc-136c9f810b66 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:34.592658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8d95e-0dec-4793-83bc-136c9f810b66 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:34.592931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8d95e-0dec-4793-83bc-136c9f810b66 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:34.593656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8d95e-0dec-4793-83bc-136c9f810b66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:34.593967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8d95e-0dec-4793-83bc-136c9f810b66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:34.595593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8d95e-0dec-4793-83bc-136c9f810b66 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:34.617543 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 361/746] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:34.622948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8d95e-0dec-4793-83bc-136c9f810b66 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:34.628015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64be2b-1b31-4cd4-b441-99237ec6163e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:34.629075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64be2b-1b31-4cd4-b441-99237ec6163e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:34.629561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64be2b-1b31-4cd4-b441-99237ec6163e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:34.630069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64be2b-1b31-4cd4-b441-99237ec6163e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:34.641879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64be2b-1b31-4cd4-b441-99237ec6163e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:34.643726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64be2b-1b31-4cd4-b441-99237ec6163e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:34.660680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-6c64be2b-1b31-4cd4-b441-99237ec6163e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:34.660680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:34.660680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:34.660680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:34.660680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:34.660680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:34.660680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:34.660680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:34.660680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:34.660680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:34.660680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:34.660680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:34.660680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:34.660680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:34.660680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:34.660680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:34.662099 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:34.662099 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:34.662099 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:34.662099 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:34.662099 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:34.662099 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:34.662099 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:34.662099 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:34.662099 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:34.662099 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:34.662099 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:34.662099 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:34.662099 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:34.662099 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:34.662099 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:34.662099 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:34.662099 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:34.663096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:34.663096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:34.663096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:34.665773 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 386/747] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:34.671534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64be2b-1b31-4cd4-b441-99237ec6163e None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:34.711742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:34.785228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64be2b-1b31-4cd4-b441-99237ec6163e None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:34.823691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc1ef5-53df-43ea-846e-b4f2e7020198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:34.824651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc1ef5-53df-43ea-846e-b4f2e7020198 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:34.824898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8d95e-0dec-4793-83bc-136c9f810b66 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:34.825415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc1ef5-53df-43ea-846e-b4f2e7020198 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:34.825891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc1ef5-53df-43ea-846e-b4f2e7020198 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:34.826888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc1ef5-53df-43ea-846e-b4f2e7020198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:34.827479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc1ef5-53df-43ea-846e-b4f2e7020198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:34.829613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc1ef5-53df-43ea-846e-b4f2e7020198 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:34.843124 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 387/748] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:34.849155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc1ef5-53df-43ea-846e-b4f2e7020198 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:34.859954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:34.866695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875f76c-95a9-463d-a1a9-17c007e63ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:34.867330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875f76c-95a9-463d-a1a9-17c007e63ebf None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:34.867610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875f76c-95a9-463d-a1a9-17c007e63ebf None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:34.867881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875f76c-95a9-463d-a1a9-17c007e63ebf None admin] PATH_INFO: `/v3/projects/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:34.877608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875f76c-95a9-463d-a1a9-17c007e63ebf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:34.879393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875f76c-95a9-463d-a1a9-17c007e63ebf None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:34.972371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:34.980702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc1ef5-53df-43ea-846e-b4f2e7020198 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:35.023590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486ea34-bb85-4a0b-ab2f-20829fadadab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:35.024228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486ea34-bb85-4a0b-ab2f-20829fadadab None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:35.026826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486ea34-bb85-4a0b-ab2f-20829fadadab None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:35.027332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486ea34-bb85-4a0b-ab2f-20829fadadab None admin] PATH_INFO: `/v3/roles/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:35.034342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-3875f76c-95a9-463d-a1a9-17c007e63ebf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:35.034342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:35.034342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:35.034342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:35.034342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:35.034342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:35.034342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:35.034342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:35.034342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:35.034342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:35.034342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:35.034342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:35.034342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:35.034342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:35.034342 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:35.035304 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:35.035304 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:35.035304 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:35.035304 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:35.035304 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:35.035304 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:35.035304 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:35.035304 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:35.035304 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:35.035304 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:35.035304 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:35.035304 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:35.035304 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:35.035304 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:35.035304 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:35.035304 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:35.035304 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:35.036297 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:35.036297 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:35.036297 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:35.036297 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:35.036297 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:35.036297 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:35.036297 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:35.036297 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:35.038466 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 362/749] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:35.046743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875f76c-95a9-463d-a1a9-17c007e63ebf None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:35.049208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486ea34-bb85-4a0b-ab2f-20829fadadab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:35.055185 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-a486ea34-bb85-4a0b-ab2f-20829fadadab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:35.055185 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:35.055185 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:35.055185 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:35.055185 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:35.055185 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:35.055185 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:35.055185 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:35.055185 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:35.055185 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:35.055185 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:35.055185 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:35.055185 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:35.055185 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:35.055185 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:35.055185 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:35.056221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:35.056221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:35.056221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:35.056221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:35.056221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:35.056221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:35.056221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:35.056221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:35.056221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:35.056221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:35.056221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:35.056221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:35.056221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:35.056221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:35.056221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:35.056221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:35.057103 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:35.057103 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:35.057103 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:35.057103 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:35.057103 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:35.057103 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:35.057103 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:35.057103 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:35.057103 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:35.057103 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:35.057103 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:35.057103 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:35.061120 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 388/750] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:35.061978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486ea34-bb85-4a0b-ab2f-20829fadadab None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:35.178585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875f76c-95a9-463d-a1a9-17c007e63ebf None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:35.198305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486ea34-bb85-4a0b-ab2f-20829fadadab None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:35.223065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789b034-f968-4dca-af0c-402097a17f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:35.224226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789b034-f968-4dca-af0c-402097a17f2a None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:35.224689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789b034-f968-4dca-af0c-402097a17f2a None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:35.225328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789b034-f968-4dca-af0c-402097a17f2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:35.234517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8789b034-f968-4dca-af0c-402097a17f2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:35.236934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8789b034-f968-4dca-af0c-402097a17f2a None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:35.244974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea388bc3-0c22-436a-9c26-d3a71da43e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:35.245714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea388bc3-0c22-436a-9c26-d3a71da43e8d None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:35.246095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea388bc3-0c22-436a-9c26-d3a71da43e8d None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:35.246468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea388bc3-0c22-436a-9c26-d3a71da43e8d None admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:35.247282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea388bc3-0c22-436a-9c26-d3a71da43e8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:35.247688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea388bc3-0c22-436a-9c26-d3a71da43e8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:35.249264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea388bc3-0c22-436a-9c26-d3a71da43e8d None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:35.251486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-8789b034-f968-4dca-af0c-402097a17f2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:35.251486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:35.251486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:35.251486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:35.251486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:35.251486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:35.251486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:35.251486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:35.251486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:35.251486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:35.251486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:35.251486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:35.251486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:35.251486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:35.251486 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:35.252564 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:35.252564 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:35.252564 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:35.252564 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:35.252564 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:35.252564 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:35.252564 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:35.252564 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:35.252564 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:35.252564 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:35.252564 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:35.252564 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:35.252564 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:35.252564 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:35.252564 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:35.252564 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:35.252564 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:35.253689 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:35.253689 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:35.253689 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:35.253689 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:35.253689 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:35.253689 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:35.253689 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:35.253689 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:35.260262 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 363/751] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:35.260262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789b034-f968-4dca-af0c-402097a17f2a None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:35.270584 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 389/752] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:35.271771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea388bc3-0c22-436a-9c26-d3a71da43e8d None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:35.293481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:35.337411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:35.400514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789b034-f968-4dca-af0c-402097a17f2a None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:35.407145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea388bc3-0c22-436a-9c26-d3a71da43e8d None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:35.443290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3451f4-a10f-4315-b324-a985469d703d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:35.443815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3451f4-a10f-4315-b324-a985469d703d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:35.444097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3451f4-a10f-4315-b324-a985469d703d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:35.444362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3451f4-a10f-4315-b324-a985469d703d None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:35.445014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3451f4-a10f-4315-b324-a985469d703d None admin] RBAC: Adding query filter params (name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:35.445315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3451f4-a10f-4315-b324-a985469d703d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:35.446981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3451f4-a10f-4315-b324-a985469d703d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:35.449873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8945dd-e9ea-48cc-a1bf-07e415cba47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:35.450485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8945dd-e9ea-48cc-a1bf-07e415cba47a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:35.450797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8945dd-e9ea-48cc-a1bf-07e415cba47a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:35.451074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8945dd-e9ea-48cc-a1bf-07e415cba47a None admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:35.451831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8945dd-e9ea-48cc-a1bf-07e415cba47a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:35.452391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8945dd-e9ea-48cc-a1bf-07e415cba47a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:35.453546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8945dd-e9ea-48cc-a1bf-07e415cba47a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:35.460281 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 364/753] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:33:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:35.460281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3451f4-a10f-4315-b324-a985469d703d None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:35.473051 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 390/754] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:35.479001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10dec87-5b56-45f4-a645-3e0b99d9d7f6 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:35.479286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10dec87-5b56-45f4-a645-3e0b99d9d7f6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:35.479556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10dec87-5b56-45f4-a645-3e0b99d9d7f6 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:35.480986 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 391/755] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:35.481501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10dec87-5b56-45f4-a645-3e0b99d9d7f6 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:35.561694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:35.576515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3451f4-a10f-4315-b324-a985469d703d None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:35.589517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10dec87-5b56-45f4-a645-3e0b99d9d7f6 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:35.591679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:35.612629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471b267-8f61-44ed-a4fd-3a4fcec9e66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:35.613897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471b267-8f61-44ed-a4fd-3a4fcec9e66d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:35.614332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471b267-8f61-44ed-a4fd-3a4fcec9e66d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:35.614639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471b267-8f61-44ed-a4fd-3a4fcec9e66d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:35.615363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471b267-8f61-44ed-a4fd-3a4fcec9e66d None admin] RBAC: Adding query filter params (role.id=1c50896511054a5f994bee3d99fa0619, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=c4dc51e2b4474d6ba9fd5d2bc3702826) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:35.615676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471b267-8f61-44ed-a4fd-3a4fcec9e66d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:35.617092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471b267-8f61-44ed-a4fd-3a4fcec9e66d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:35.622548 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 365/756] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:33:35 2022] GET /identity/v3/role_assignments?user.id=c4dc51e2b4474d6ba9fd5d2bc3702826&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=1c50896511054a5f994bee3d99fa0619 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:35.627638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471b267-8f61-44ed-a4fd-3a4fcec9e66d None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:35.628558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a56d7-f088-4063-958e-f3cf55052e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:35.629387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a56d7-f088-4063-958e-f3cf55052e04 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:35.629860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a56d7-f088-4063-958e-f3cf55052e04 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:35.630267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a56d7-f088-4063-958e-f3cf55052e04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:35.643078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a56d7-f088-4063-958e-f3cf55052e04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:35.644953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a56d7-f088-4063-958e-f3cf55052e04 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:35.663767 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-082a56d7-f088-4063-958e-f3cf55052e04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:35.663767 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:35.663767 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:35.663767 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:35.663767 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:35.663767 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:35.663767 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:35.663767 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:35.663767 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:35.663767 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:35.663767 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:35.663767 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:35.663767 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:35.663767 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:35.663767 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:35.663767 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:35.665281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:35.665281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:35.665281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:35.665281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:35.665281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:35.665281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:35.665281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:35.665281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:35.665281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:35.665281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:35.665281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:35.665281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:35.665281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:35.665281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:35.665281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:35.665281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:35.665281 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:35.666168 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:35.666168 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:35.666168 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:35.669042 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 392/757] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:35.670723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47960a-83f1-4343-9f51-ae908beefc3b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:35.671224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47960a-83f1-4343-9f51-ae908beefc3b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:35.671690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47960a-83f1-4343-9f51-ae908beefc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:35.693485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4e47960a-83f1-4343-9f51-ae908beefc3b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:35.695230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4e47960a-83f1-4343-9f51-ae908beefc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:35.767208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471b267-8f61-44ed-a4fd-3a4fcec9e66d None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:35.790313 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 393/758] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:33:35.790313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47960a-83f1-4343-9f51-ae908beefc3b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:35.832990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820557a-89ca-45a6-9a69-eeefd38a9909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:35.834056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820557a-89ca-45a6-9a69-eeefd38a9909 None admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:35.834624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820557a-89ca-45a6-9a69-eeefd38a9909 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:35.835110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820557a-89ca-45a6-9a69-eeefd38a9909 None admin] PATH_INFO: `/v3/projects/b7875946723844198353bbfcf22516fb/users/7ab975dafb564aa2b357311c4a384d53/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:35.839915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820557a-89ca-45a6-9a69-eeefd38a9909 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7875946723844198353bbfcf22516fb, user_id=7ab975dafb564aa2b357311c4a384d53, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:35.841737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820557a-89ca-45a6-9a69-eeefd38a9909 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:35.854831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:35.869038 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 366/759] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:33:35 2022] PUT /identity/v3/projects/b7875946723844198353bbfcf22516fb/users/7ab975dafb564aa2b357311c4a384d53/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:33:35.924968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47960a-83f1-4343-9f51-ae908beefc3b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:35.977208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197decb-09c4-459e-9668-3ff809d9b5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:35.977827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197decb-09c4-459e-9668-3ff809d9b5eb None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:35.978106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197decb-09c4-459e-9668-3ff809d9b5eb None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:35.978378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197decb-09c4-459e-9668-3ff809d9b5eb None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:35.979057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197decb-09c4-459e-9668-3ff809d9b5eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:35.979359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197decb-09c4-459e-9668-3ff809d9b5eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:35.980724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197decb-09c4-459e-9668-3ff809d9b5eb None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:35.997847 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 394/760] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:36.003353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820557a-89ca-45a6-9a69-eeefd38a9909 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:36.071551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:36.165696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820557a-89ca-45a6-9a69-eeefd38a9909 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:36.209504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5002991-e7a3-4ff1-bb7f-daa08d621d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:36.210563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5002991-e7a3-4ff1-bb7f-daa08d621d8d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:36.211064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5002991-e7a3-4ff1-bb7f-daa08d621d8d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:36.211529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5002991-e7a3-4ff1-bb7f-daa08d621d8d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:36.216657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197decb-09c4-459e-9668-3ff809d9b5eb None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:36.221271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5002991-e7a3-4ff1-bb7f-daa08d621d8d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:36.223135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5002991-e7a3-4ff1-bb7f-daa08d621d8d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:36.236559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-b5002991-e7a3-4ff1-bb7f-daa08d621d8d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 01 20:33:36.236559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:36.236559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:36.236559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:36.236559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:36.236559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:36.236559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:36.236559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:36.236559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:36.236559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:36.236559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:36.236559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:36.236559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:36.236559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:36.236559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:36.236559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:36.237537 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:36.237537 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:36.237537 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:36.237537 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:36.237537 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:36.237537 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:36.237537 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:36.237537 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:36.237537 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:36.237537 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:36.237537 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:36.237537 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:36.237537 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:36.237537 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:36.237537 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:36.237537 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:36.237537 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:36.237537 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:36.238505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:36.238505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:36.238505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:36.238505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:36.238505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:36.238505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:36.238505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:36.238505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:36.238505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 01 20:33:36.238505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:36.240609 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 367/761] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Oct 1 20:33:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:36.247319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5002991-e7a3-4ff1-bb7f-daa08d621d8d None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:36.424552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197decb-09c4-459e-9668-3ff809d9b5eb None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:36.442912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:36.450074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5002991-e7a3-4ff1-bb7f-daa08d621d8d None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:36.479123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:36.505438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b40d7b-7f0c-404a-b86e-afb57e3c6de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:36.506401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b40d7b-7f0c-404a-b86e-afb57e3c6de3 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:36.506946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b40d7b-7f0c-404a-b86e-afb57e3c6de3 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:36.507393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b40d7b-7f0c-404a-b86e-afb57e3c6de3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:36.508185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b40d7b-7f0c-404a-b86e-afb57e3c6de3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:36.512013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b40d7b-7f0c-404a-b86e-afb57e3c6de3 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:36.525987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-88b40d7b-7f0c-404a-b86e-afb57e3c6de3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 01 20:33:36.525987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:36.525987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:33:36.525987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:33:36.525987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:33:36.525987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:33:36.525987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:33:36.525987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:36.525987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 01 20:33:36.525987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:36.525987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:33:36.525987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:36.525987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:36.525987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:36.525987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:36.525987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:36.525987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:36.527309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:36.527309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:36.527309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:36.527309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:36.527309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:36.527309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:36.527309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:33:36.527309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:33:36.527309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:33:36.527309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:33:36.527309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:36.527309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:36.527309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:36.527309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:36.527309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:36.527309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:36.528408 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:36.528408 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:36.528408 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:36.528408 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:36.528408 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:36.528408 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:36.528408 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:36.528408 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:36.528408 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:36.528408 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:36.528408 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:33:36.528408 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:36.528408 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 01 20:33:36.528408 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:36.536543 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 395/762] 10.222.0.17 () {56 vars in 1129 bytes} [Sat Oct 1 20:33:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:36.542643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b40d7b-7f0c-404a-b86e-afb57e3c6de3 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:36.546930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b847402-3634-44b2-9efc-caf92428f840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:36.547958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b847402-3634-44b2-9efc-caf92428f840 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:36.548451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b847402-3634-44b2-9efc-caf92428f840 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:36.548911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b847402-3634-44b2-9efc-caf92428f840 None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:36.549834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b847402-3634-44b2-9efc-caf92428f840 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:36.551153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b847402-3634-44b2-9efc-caf92428f840 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:36.557260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b847402-3634-44b2-9efc-caf92428f840 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:36.579834 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 368/763] 10.222.0.17 () {56 vars in 1149 bytes} [Sat Oct 1 20:33:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:36.585068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b847402-3634-44b2-9efc-caf92428f840 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:36.747098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b40d7b-7f0c-404a-b86e-afb57e3c6de3 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:36.747727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b847402-3634-44b2-9efc-caf92428f840 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:36.786745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934aa276-818d-49d8-90fe-2a6938fc4b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:36.787277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934aa276-818d-49d8-90fe-2a6938fc4b15 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:36.787556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934aa276-818d-49d8-90fe-2a6938fc4b15 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:36.787903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934aa276-818d-49d8-90fe-2a6938fc4b15 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:36.788459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934aa276-818d-49d8-90fe-2a6938fc4b15 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:36.788761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934aa276-818d-49d8-90fe-2a6938fc4b15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:36.790125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934aa276-818d-49d8-90fe-2a6938fc4b15 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:36.790981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22120f97-29a4-4b9d-bbde-9e3b3657fe76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:36.791674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22120f97-29a4-4b9d-bbde-9e3b3657fe76 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:36.792051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22120f97-29a4-4b9d-bbde-9e3b3657fe76 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:36.792322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22120f97-29a4-4b9d-bbde-9e3b3657fe76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:36.795267 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 396/764] 10.222.0.17 () {56 vars in 1124 bytes} [Sat Oct 1 20:33:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:36.796649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37c01d-dc8c-4f33-9b59-8b252e83ff85 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:36.796939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37c01d-dc8c-4f33-9b59-8b252e83ff85 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:36.797241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37c01d-dc8c-4f33-9b59-8b252e83ff85 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:36.802669 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 397/765] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:36.803593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37c01d-dc8c-4f33-9b59-8b252e83ff85 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:36.818222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:36.854982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:36.865821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22120f97-29a4-4b9d-bbde-9e3b3657fe76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:36.867418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22120f97-29a4-4b9d-bbde-9e3b3657fe76 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:36.913305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37c01d-dc8c-4f33-9b59-8b252e83ff85 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:36.954575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21a0fe-c680-414d-ae2a-0676fa8b48e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:36.955454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21a0fe-c680-414d-ae2a-0676fa8b48e1 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:36.956096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21a0fe-c680-414d-ae2a-0676fa8b48e1 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:36.956526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21a0fe-c680-414d-ae2a-0676fa8b48e1 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:36.957585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21a0fe-c680-414d-ae2a-0676fa8b48e1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:36.958047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21a0fe-c680-414d-ae2a-0676fa8b48e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:36.959587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21a0fe-c680-414d-ae2a-0676fa8b48e1 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:36.966528 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 398/766] 10.222.0.17 () {56 vars in 1124 bytes} [Sat Oct 1 20:33:36 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:36.968172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2170b-b0fd-4654-bf56-4cfb622eff65 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:36.968780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2170b-b0fd-4654-bf56-4cfb622eff65 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:36.969764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2170b-b0fd-4654-bf56-4cfb622eff65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:36.973997 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 369/767] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:33:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 01 20:33:36.975022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22120f97-29a4-4b9d-bbde-9e3b3657fe76 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:36.993535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d6a2170b-b0fd-4654-bf56-4cfb622eff65 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:36.995300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d6a2170b-b0fd-4654-bf56-4cfb622eff65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:37.036360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:37.089886 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 399/768] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:33:37.091375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2170b-b0fd-4654-bf56-4cfb622eff65 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:37.095963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22120f97-29a4-4b9d-bbde-9e3b3657fe76 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:37.153519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9b7a6-4cb5-4c27-a4ce-5a9702c2eeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:37.155087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9b7a6-4cb5-4c27-a4ce-5a9702c2eeb3 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:37.155772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9b7a6-4cb5-4c27-a4ce-5a9702c2eeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:37.157111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9b7a6-4cb5-4c27-a4ce-5a9702c2eeb3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:37.158972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9b7a6-4cb5-4c27-a4ce-5a9702c2eeb3 None admin] RBAC: Adding query filter params (interface=public, service_id=6634ff4dddce4620b3294944edeb14c6, region_id=RegionOne) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:37.160757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9b7a6-4cb5-4c27-a4ce-5a9702c2eeb3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:37.162766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9b7a6-4cb5-4c27-a4ce-5a9702c2eeb3 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:37.176552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:37.182374 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 370/769] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Oct 1 20:33:36 2022] GET /identity/v3/endpoints?service_id=6634ff4dddce4620b3294944edeb14c6&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:37.188314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9b7a6-4cb5-4c27-a4ce-5a9702c2eeb3 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:37.294468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2170b-b0fd-4654-bf56-4cfb622eff65 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:37.342894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e95a6-8328-436c-ada7-d69d4b64c1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:37.343449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e95a6-8328-436c-ada7-d69d4b64c1f2 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:37.343727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e95a6-8328-436c-ada7-d69d4b64c1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:37.343996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e95a6-8328-436c-ada7-d69d4b64c1f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:37.354954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e95a6-8328-436c-ada7-d69d4b64c1f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:37.356513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e95a6-8328-436c-ada7-d69d4b64c1f2 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:37.371832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-5f9e95a6-8328-436c-ada7-d69d4b64c1f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:37.371832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:37.371832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:37.371832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:37.371832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:37.371832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:37.371832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:37.371832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:37.371832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:37.371832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:37.371832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:37.371832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:37.371832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:37.371832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:37.371832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:37.371832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:37.373097 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:37.373097 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:37.373097 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:37.373097 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:37.373097 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:37.373097 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:37.373097 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:37.373097 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:37.373097 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:37.373097 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:37.373097 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:37.373097 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:37.373097 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:37.373097 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:37.373097 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:37.373097 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:37.373097 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:37.374304 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:37.374304 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:37.374304 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:37.374304 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 400/770] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:37.379244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e95a6-8328-436c-ada7-d69d4b64c1f2 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:37.420436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9b7a6-4cb5-4c27-a4ce-5a9702c2eeb3 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:37.450340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:37.481380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35b099-7259-41d2-b0ee-ad6001aaf0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:37.482044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35b099-7259-41d2-b0ee-ad6001aaf0ed None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:37.482331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35b099-7259-41d2-b0ee-ad6001aaf0ed None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:37.482636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35b099-7259-41d2-b0ee-ad6001aaf0ed None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:37.483241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b35b099-7259-41d2-b0ee-ad6001aaf0ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:37.484616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b35b099-7259-41d2-b0ee-ad6001aaf0ed None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:37.495274 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 371/771] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Oct 1 20:33:37 2022] GET /identity/v3/services => generated 772 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:37.496201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35b099-7259-41d2-b0ee-ad6001aaf0ed None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:37.503094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:37.519169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e95a6-8328-436c-ada7-d69d4b64c1f2 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:37.567952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecadaa6c-e877-42c4-9e84-b5667feceae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:37.569072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadaa6c-e877-42c4-9e84-b5667feceae7 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:37.569622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadaa6c-e877-42c4-9e84-b5667feceae7 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:37.569920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadaa6c-e877-42c4-9e84-b5667feceae7 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:37.570642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecadaa6c-e877-42c4-9e84-b5667feceae7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:37.571472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecadaa6c-e877-42c4-9e84-b5667feceae7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:37.573748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecadaa6c-e877-42c4-9e84-b5667feceae7 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:37.591405 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 401/772] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:37.601244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecadaa6c-e877-42c4-9e84-b5667feceae7 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:37.643932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35b099-7259-41d2-b0ee-ad6001aaf0ed None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:37.695527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ce2cb-598c-4c20-8927-16509d47c418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:37.696473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ce2cb-598c-4c20-8927-16509d47c418 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:37.696749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ce2cb-598c-4c20-8927-16509d47c418 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:37.697020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ce2cb-598c-4c20-8927-16509d47c418 None admin] PATH_INFO: `/v3/services/network` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:37.697614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ce2cb-598c-4c20-8927-16509d47c418 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:37.699130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ce2cb-598c-4c20-8927-16509d47c418 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:37.711432 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-539ce2cb-598c-4c20-8927-16509d47c418 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 01 20:33:37.711432 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:37.711432 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:33:37.711432 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:33:37.711432 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:33:37.711432 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:33:37.711432 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:33:37.711432 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:37.711432 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 01 20:33:37.711432 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:37.711432 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:33:37.711432 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:37.711432 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:37.711432 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:37.711432 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:37.711432 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:37.711432 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:37.712541 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:37.712541 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:37.712541 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:37.712541 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:37.712541 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:37.712541 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:37.712541 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:33:37.712541 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:33:37.712541 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:33:37.712541 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:33:37.712541 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:37.712541 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:37.712541 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:37.712541 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:37.712541 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:37.712541 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:37.713427 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:37.713427 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:37.713427 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:37.713427 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:37.713427 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:37.713427 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:37.713427 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:37.713427 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:37.713427 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:37.713427 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:37.713427 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:33:37.713427 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:37.713427 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 01 20:33:37.713427 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:37.713427 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 372/773] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:37 2022] GET /identity/v3/services/network => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:37.718849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ce2cb-598c-4c20-8927-16509d47c418 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:37.789809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:37.837270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecadaa6c-e877-42c4-9e84-b5667feceae7 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:37.855756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ce2cb-598c-4c20-8927-16509d47c418 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:37.897368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b123e8-4d32-47ee-83f9-e83f4ed42a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:37.898000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b123e8-4d32-47ee-83f9-e83f4ed42a80 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:37.898274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b123e8-4d32-47ee-83f9-e83f4ed42a80 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:37.898571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b123e8-4d32-47ee-83f9-e83f4ed42a80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:37.906330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b123e8-4d32-47ee-83f9-e83f4ed42a80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:37.908471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b123e8-4d32-47ee-83f9-e83f4ed42a80 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:37.920169 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-d7b123e8-4d32-47ee-83f9-e83f4ed42a80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:37.920169 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:37.920169 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:37.920169 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:37.920169 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:37.920169 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:37.920169 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:37.920169 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:37.920169 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:37.920169 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:37.920169 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:37.920169 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:37.920169 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:37.920169 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:37.920169 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:37.921106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:37.921106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:37.921106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:37.921106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:37.921106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:37.921106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:37.921106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:37.921106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:37.921106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:37.921106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:37.921106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:37.921106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:37.921106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:37.921106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:37.921106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:37.921106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:37.921106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:37.921997 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:37.921997 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:37.921997 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:37.921997 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:37.921997 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:37.921997 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:37.921997 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:37.921997 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:37.921997 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 402/774] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:37.926750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f6d36-cc76-455d-b756-9ce5514bc360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:37.927244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b123e8-4d32-47ee-83f9-e83f4ed42a80 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:37.927363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f6d36-cc76-455d-b756-9ce5514bc360 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:37.927629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f6d36-cc76-455d-b756-9ce5514bc360 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:37.927898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f6d36-cc76-455d-b756-9ce5514bc360 None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:37.928535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f6d36-cc76-455d-b756-9ce5514bc360 None admin] RBAC: Adding query filter params (name=network) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:37.928839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f6d36-cc76-455d-b756-9ce5514bc360 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:37.930220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f6d36-cc76-455d-b756-9ce5514bc360 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:37.948185 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 373/775] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Oct 1 20:33:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:37.948777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:37.953677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f6d36-cc76-455d-b756-9ce5514bc360 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:38.039524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b123e8-4d32-47ee-83f9-e83f4ed42a80 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:38.079591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd041bfd-dafa-43ff-93a5-c76b792a93ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:38.080855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd041bfd-dafa-43ff-93a5-c76b792a93ef None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:38.081372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd041bfd-dafa-43ff-93a5-c76b792a93ef None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:38.081913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd041bfd-dafa-43ff-93a5-c76b792a93ef None admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:38.082903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd041bfd-dafa-43ff-93a5-c76b792a93ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:38.083444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd041bfd-dafa-43ff-93a5-c76b792a93ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:38.085112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd041bfd-dafa-43ff-93a5-c76b792a93ef None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:38.099863 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 403/776] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:38.105355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd041bfd-dafa-43ff-93a5-c76b792a93ef None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:38.205241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f6d36-cc76-455d-b756-9ce5514bc360 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:38.235406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:38.252969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:38.278420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c885db-53ad-48e5-af0f-5887aab8bc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:38.279048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c885db-53ad-48e5-af0f-5887aab8bc47 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:38.279324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c885db-53ad-48e5-af0f-5887aab8bc47 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:38.279590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c885db-53ad-48e5-af0f-5887aab8bc47 None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:38.280219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c885db-53ad-48e5-af0f-5887aab8bc47 None admin] RBAC: Adding query filter params (type=network) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:38.280521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c885db-53ad-48e5-af0f-5887aab8bc47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:38.282519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c885db-53ad-48e5-af0f-5887aab8bc47 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:38.296081 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 374/777] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Oct 1 20:33:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:38.301664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c885db-53ad-48e5-af0f-5887aab8bc47 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:38.313534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd041bfd-dafa-43ff-93a5-c76b792a93ef None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:38.378425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3caf0-e728-4f61-9b47-d7016f028cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:38.379426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3caf0-e728-4f61-9b47-d7016f028cf8 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:38.379982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3caf0-e728-4f61-9b47-d7016f028cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:38.380453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3caf0-e728-4f61-9b47-d7016f028cf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:38.381439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3caf0-e728-4f61-9b47-d7016f028cf8 None admin] RBAC: Adding query filter params (role.id=ef7bd7293369475a9e7b3444fac30cda, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=917fc96fa3504b19b1a56990d4747439) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:38.384420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3caf0-e728-4f61-9b47-d7016f028cf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:38.386138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3caf0-e728-4f61-9b47-d7016f028cf8 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:38.397310 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 404/778] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:33:38 2022] GET /identity/v3/role_assignments?user.id=917fc96fa3504b19b1a56990d4747439&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=ef7bd7293369475a9e7b3444fac30cda => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:38.456613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c885db-53ad-48e5-af0f-5887aab8bc47 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:38.508674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3785b-d57a-46b5-969f-292830ff1a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:38.509297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3785b-d57a-46b5-969f-292830ff1a42 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:38.509578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3785b-d57a-46b5-969f-292830ff1a42 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:38.509855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3785b-d57a-46b5-969f-292830ff1a42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:38.510595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d3785b-d57a-46b5-969f-292830ff1a42 None admin] RBAC: Adding query filter params (interface=public, service_id=a1c7ed98fe8a429dbbe75d1e36e135a9, region_id=RegionOne) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:38.510910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d3785b-d57a-46b5-969f-292830ff1a42 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:38.512315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d3785b-d57a-46b5-969f-292830ff1a42 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:38.523503 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 375/779] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Oct 1 20:33:38 2022] GET /identity/v3/endpoints?service_id=a1c7ed98fe8a429dbbe75d1e36e135a9&interface=public®ion_id=RegionOne => generated 194 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:38.529208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3caf0-e728-4f61-9b47-d7016f028cf8 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:38.610087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a126dbe-6745-4f00-b23e-aeaef05ce85c None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:38.610458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a126dbe-6745-4f00-b23e-aeaef05ce85c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:38.610733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a126dbe-6745-4f00-b23e-aeaef05ce85c None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:38.612161 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 376/780] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:38.622531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db51151-4109-42ee-9721-d37d9ce66df8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:38.622531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db51151-4109-42ee-9721-d37d9ce66df8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:38.622531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db51151-4109-42ee-9721-d37d9ce66df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:38.648403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5db51151-4109-42ee-9721-d37d9ce66df8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:38.649732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5db51151-4109-42ee-9721-d37d9ce66df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:38.673759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3caf0-e728-4f61-9b47-d7016f028cf8 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:38.698234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:38.717535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72af47-abc7-4182-9e2e-b7f4d8cc478a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:38.718163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72af47-abc7-4182-9e2e-b7f4d8cc478a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:38.718553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72af47-abc7-4182-9e2e-b7f4d8cc478a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:38.719138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72af47-abc7-4182-9e2e-b7f4d8cc478a None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:38.719830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f72af47-abc7-4182-9e2e-b7f4d8cc478a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:38.721276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f72af47-abc7-4182-9e2e-b7f4d8cc478a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:38.722766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:38.728612 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 405/781] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Oct 1 20:33:38 2022] GET /identity/v3/services => generated 772 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:38.729921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca94b84-f48a-4681-aff5-9157991cf445 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:38.730322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca94b84-f48a-4681-aff5-9157991cf445 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:38.730689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca94b84-f48a-4681-aff5-9157991cf445 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:38.732259 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 406/782] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:38.738692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b7b37-7add-46fc-afea-3ac49b82cf9f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:38.739131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b7b37-7add-46fc-afea-3ac49b82cf9f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:38.739504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b7b37-7add-46fc-afea-3ac49b82cf9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:38.752502 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 377/783] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:33:38.762385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fe4b7b37-7add-46fc-afea-3ac49b82cf9f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:38.763677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fe4b7b37-7add-46fc-afea-3ac49b82cf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:38.840718 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 407/784] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:33:39.180209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db51151-4109-42ee-9721-d37d9ce66df8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:39.309018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b7b37-7add-46fc-afea-3ac49b82cf9f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:39.316793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db51151-4109-42ee-9721-d37d9ce66df8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:39.352902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4aec06-16c5-4a21-8af5-2a791b28ae1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:39.353777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4aec06-16c5-4a21-8af5-2a791b28ae1a None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:39.354262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4aec06-16c5-4a21-8af5-2a791b28ae1a None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:39.354748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4aec06-16c5-4a21-8af5-2a791b28ae1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:39.365904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4aec06-16c5-4a21-8af5-2a791b28ae1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:39.371460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-cd4aec06-16c5-4a21-8af5-2a791b28ae1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:33:39.371460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:39.371460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:39.371460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:39.371460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:39.371460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:39.371460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:39.371460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:39.371460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:39.371460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:39.371460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:39.371460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:39.371460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:39.371460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:39.371460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:39.371460 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:39.372408 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:39.372408 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:39.372408 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:39.372408 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:39.372408 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:39.372408 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:39.372408 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:39.372408 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:39.372408 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:39.372408 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:39.372408 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:39.372408 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:39.372408 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:39.372408 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:39.372408 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:39.372408 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:39.373260 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:39.373260 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:39.373260 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:39.373260 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:39.373260 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:39.373260 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:39.373260 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:39.373260 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:39.373260 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:39.373260 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:39.373260 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:33:39.373260 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:39.376544 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 378/785] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:33:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:39.382003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4aec06-16c5-4a21-8af5-2a791b28ae1a None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:39.414865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:39.475220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b7b37-7add-46fc-afea-3ac49b82cf9f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:39.503531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:39.513730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4aec06-16c5-4a21-8af5-2a791b28ae1a None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:39.527185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ba17d-5fc6-4f78-92ce-ed8721e21697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:39.527820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ba17d-5fc6-4f78-92ce-ed8721e21697 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:39.528100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ba17d-5fc6-4f78-92ce-ed8721e21697 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:39.528372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ba17d-5fc6-4f78-92ce-ed8721e21697 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:39.547875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ba17d-5fc6-4f78-92ce-ed8721e21697 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:39.552998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfa6c8-48d7-4a28-81ad-90258aad478c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:39.553620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfa6c8-48d7-4a28-81ad-90258aad478c None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:39.553904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfa6c8-48d7-4a28-81ad-90258aad478c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:39.554183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfa6c8-48d7-4a28-81ad-90258aad478c None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:39.555036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dfa6c8-48d7-4a28-81ad-90258aad478c None admin] RBAC: Adding query filter params (name=admin) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:39.555436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dfa6c8-48d7-4a28-81ad-90258aad478c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:39.557461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dfa6c8-48d7-4a28-81ad-90258aad478c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:39.561049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-c28ba17d-5fc6-4f78-92ce-ed8721e21697 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:33:39.561049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:39.561049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:39.561049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:39.561049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:39.561049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:39.561049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:39.561049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:39.561049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:39.561049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:39.561049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:39.561049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:39.561049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:39.561049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:39.561049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:39.561049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:39.562072 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:39.562072 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:39.562072 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:39.562072 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:39.562072 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:39.562072 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:39.562072 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:39.562072 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:39.562072 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:39.562072 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:39.562072 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:39.562072 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:39.562072 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:39.562072 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:39.562072 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:39.562072 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:39.563603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:39.563603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:39.563603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:39.563603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:39.563603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:39.563603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:39.563603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:39.563603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:39.563603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:39.563603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:39.563603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:33:39.563603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:39.563603 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 408/786] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:33:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:39.574053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ba17d-5fc6-4f78-92ce-ed8721e21697 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:39.583466 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 379/787] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:33:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:39.588802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfa6c8-48d7-4a28-81ad-90258aad478c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:39.760232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ba17d-5fc6-4f78-92ce-ed8721e21697 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:39.763861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:39.780207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfa6c8-48d7-4a28-81ad-90258aad478c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:39.821257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96d279-7937-4637-839f-4638877232dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:39.822934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96d279-7937-4637-839f-4638877232dc None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:39.823217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96d279-7937-4637-839f-4638877232dc None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:39.823490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96d279-7937-4637-839f-4638877232dc None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:39.824166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96d279-7937-4637-839f-4638877232dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:39.824816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96d279-7937-4637-839f-4638877232dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:39.830434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96d279-7937-4637-839f-4638877232dc None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:39.851528 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 409/788] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:33:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:39.852834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2d85e-49e5-41e3-a6bd-686f6fb23051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:39.853494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2d85e-49e5-41e3-a6bd-686f6fb23051 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:39.853772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2d85e-49e5-41e3-a6bd-686f6fb23051 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:39.854041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2d85e-49e5-41e3-a6bd-686f6fb23051 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:39.856418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96d279-7937-4637-839f-4638877232dc None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:39.876748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb2d85e-49e5-41e3-a6bd-686f6fb23051 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:39.878352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb2d85e-49e5-41e3-a6bd-686f6fb23051 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:39.895469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-fbb2d85e-49e5-41e3-a6bd-686f6fb23051 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:39.895469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:39.895469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:39.895469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:39.895469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:39.895469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:39.895469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:39.895469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:39.895469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:39.895469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:39.895469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:39.895469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:39.895469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:39.895469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:39.895469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:39.895469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:39.896591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:39.896591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:39.896591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:39.896591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:39.896591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:39.896591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:39.896591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:39.896591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:39.896591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:39.896591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:39.896591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:39.896591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:39.896591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:39.896591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:39.896591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:39.896591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:39.896591 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:39.897526 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:39.897526 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:39.897526 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:39.901688 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 380/789] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:39.902932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2d85e-49e5-41e3-a6bd-686f6fb23051 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:39.964999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:40.075073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96d279-7937-4637-839f-4638877232dc None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:40.091666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2d85e-49e5-41e3-a6bd-686f6fb23051 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:40.147131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405d7b1-f41e-4f7d-a3e2-6f05b0657693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:40.147789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405d7b1-f41e-4f7d-a3e2-6f05b0657693 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:40.148073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405d7b1-f41e-4f7d-a3e2-6f05b0657693 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:40.148342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405d7b1-f41e-4f7d-a3e2-6f05b0657693 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:40.163023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144f143-310b-4046-81b1-7bf63dfc876c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:40.163630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144f143-310b-4046-81b1-7bf63dfc876c None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:40.163904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144f143-310b-4046-81b1-7bf63dfc876c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:40.164170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144f143-310b-4046-81b1-7bf63dfc876c None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:40.164821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144f143-310b-4046-81b1-7bf63dfc876c None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:40.165124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144f143-310b-4046-81b1-7bf63dfc876c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:40.166585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144f143-310b-4046-81b1-7bf63dfc876c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:40.175621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405d7b1-f41e-4f7d-a3e2-6f05b0657693 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:40.177050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405d7b1-f41e-4f7d-a3e2-6f05b0657693 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:40.191434 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 381/790] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:40.199583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144f143-310b-4046-81b1-7bf63dfc876c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:40.207007 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-7405d7b1-f41e-4f7d-a3e2-6f05b0657693 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:40.207007 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:40.207007 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:40.207007 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:40.207007 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:40.207007 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:40.207007 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:40.207007 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:40.207007 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:40.207007 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:40.207007 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:40.207007 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:40.207007 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:40.207007 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:40.207007 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:40.207007 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:40.207932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:40.207932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:40.207932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:40.207932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:40.207932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:40.207932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:40.207932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:40.207932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:40.207932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:40.207932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:40.207932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:40.207932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:40.207932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:40.207932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:40.207932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:40.207932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:40.207932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:40.208825 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:40.208825 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:40.208825 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:40.208825 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 410/791] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:40.210096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:40.225241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405d7b1-f41e-4f7d-a3e2-6f05b0657693 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:40.320390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144f143-310b-4046-81b1-7bf63dfc876c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:40.359655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498044f-297c-4e63-9a39-8370c0b6d938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:40.360236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498044f-297c-4e63-9a39-8370c0b6d938 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:40.360511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498044f-297c-4e63-9a39-8370c0b6d938 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:40.360775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498044f-297c-4e63-9a39-8370c0b6d938 None admin] PATH_INFO: `/v3/users/swift` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:40.373729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498044f-297c-4e63-9a39-8370c0b6d938 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:40.375722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498044f-297c-4e63-9a39-8370c0b6d938 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:40.388306 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-7498044f-297c-4e63-9a39-8370c0b6d938 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 01 20:33:40.388306 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:40.388306 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:40.388306 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:40.388306 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:40.388306 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:40.388306 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:40.388306 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:40.388306 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:40.388306 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:40.388306 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:40.388306 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:40.388306 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:40.388306 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:40.388306 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:40.388306 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:40.389280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:40.389280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:40.389280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:40.389280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:40.389280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:40.389280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:40.389280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:40.389280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:40.389280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:40.389280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:40.389280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:40.389280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:40.389280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:40.389280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:40.389280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:40.389280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:40.389280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:40.389280 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:40.390248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:40.390248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:40.390248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:40.390248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:40.390248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:40.390248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:40.390248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:40.390248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:40.390248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 01 20:33:40.390248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:40.390248 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 382/792] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:33:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:40.392527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54840258-78b3-4b87-9f1d-676172aa926d None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:40.392527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54840258-78b3-4b87-9f1d-676172aa926d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:40.392527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54840258-78b3-4b87-9f1d-676172aa926d None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:40.398515 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 383/793] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:40.398653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405d7b1-f41e-4f7d-a3e2-6f05b0657693 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:40.398764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54840258-78b3-4b87-9f1d-676172aa926d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:40.399311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:40.434427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a45965-627b-41e8-a1a3-8f7d565f8d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:40.435254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a45965-627b-41e8-a1a3-8f7d565f8d32 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:40.435602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a45965-627b-41e8-a1a3-8f7d565f8d32 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:40.435967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a45965-627b-41e8-a1a3-8f7d565f8d32 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:40.436769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a45965-627b-41e8-a1a3-8f7d565f8d32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:40.437140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a45965-627b-41e8-a1a3-8f7d565f8d32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:40.439230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a45965-627b-41e8-a1a3-8f7d565f8d32 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:40.462526 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 411/794] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:40.462526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0514c7f-4575-4846-aa4a-646e4576b7e7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:40.462526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0514c7f-4575-4846-aa4a-646e4576b7e7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:40.462526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0514c7f-4575-4846-aa4a-646e4576b7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:40.476219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:40.481330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a0514c7f-4575-4846-aa4a-646e4576b7e7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:40.482820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a0514c7f-4575-4846-aa4a-646e4576b7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:40.528569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54840258-78b3-4b87-9f1d-676172aa926d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:40.562911 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 412/795] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:33:40.564086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0514c7f-4575-4846-aa4a-646e4576b7e7 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:40.582102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41fc0a-42bb-4049-8247-15220704097f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:40.583212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41fc0a-42bb-4049-8247-15220704097f None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:40.583369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41fc0a-42bb-4049-8247-15220704097f None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:40.583674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41fc0a-42bb-4049-8247-15220704097f None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:40.584447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41fc0a-42bb-4049-8247-15220704097f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:40.584800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41fc0a-42bb-4049-8247-15220704097f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:40.598540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41fc0a-42bb-4049-8247-15220704097f None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:40.629707 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 384/796] 10.222.0.17 () {56 vars in 1135 bytes} [Sat Oct 1 20:33:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:40.635231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41fc0a-42bb-4049-8247-15220704097f None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:40.719532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:40.735766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0514c7f-4575-4846-aa4a-646e4576b7e7 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:40.803183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af73343-d524-456a-86f4-d28c2f7f892e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:40.803810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af73343-d524-456a-86f4-d28c2f7f892e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:40.804085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af73343-d524-456a-86f4-d28c2f7f892e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:40.804353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af73343-d524-456a-86f4-d28c2f7f892e None admin] PATH_INFO: `/v3/users/nova` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:40.815760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41fc0a-42bb-4049-8247-15220704097f None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:40.818777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af73343-d524-456a-86f4-d28c2f7f892e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:40.820186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af73343-d524-456a-86f4-d28c2f7f892e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:40.837662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-4af73343-d524-456a-86f4-d28c2f7f892e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 01 20:33:40.837662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:40.837662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:40.837662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:40.837662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:40.837662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:40.837662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:40.837662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:40.837662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:40.837662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:40.837662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:40.837662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:40.837662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:40.837662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:40.837662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:40.837662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:40.838698 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:40.838698 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:40.838698 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:40.838698 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:40.838698 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:40.838698 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:40.838698 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:40.838698 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:40.838698 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:40.838698 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:40.838698 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:40.838698 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:40.838698 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:40.838698 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:40.838698 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:40.838698 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:40.838698 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:40.838698 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:40.839644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:40.839644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:40.839644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:40.839644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:40.839644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:40.839644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:40.839644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:40.839644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:40.839644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 01 20:33:40.839644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:40.839644 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 413/797] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Oct 1 20:33:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:40.844516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af73343-d524-456a-86f4-d28c2f7f892e None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:40.853075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:40.875963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c40762-9564-47af-9d41-1b7e13c31f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:40.878857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c40762-9564-47af-9d41-1b7e13c31f11 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:40.879587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c40762-9564-47af-9d41-1b7e13c31f11 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:40.880116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c40762-9564-47af-9d41-1b7e13c31f11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:40.900789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c40762-9564-47af-9d41-1b7e13c31f11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:40.902656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c40762-9564-47af-9d41-1b7e13c31f11 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:40.936630 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-a7c40762-9564-47af-9d41-1b7e13c31f11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:40.936630 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:40.936630 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:40.936630 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:40.936630 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:40.936630 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:40.936630 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:40.936630 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:40.936630 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:40.936630 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:40.936630 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:40.936630 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:40.936630 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:40.936630 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:40.936630 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:40.936630 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:40.937635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:40.937635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:40.937635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:40.937635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:40.937635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:40.937635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:40.937635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:40.937635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:40.937635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:40.937635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:40.937635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:40.937635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:40.937635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:40.937635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:40.937635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:40.937635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:40.937635 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:40.939097 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:40.939097 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:40.939097 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:40.947325 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 385/798] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:40.953801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c40762-9564-47af-9d41-1b7e13c31f11 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:40.966353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af73343-d524-456a-86f4-d28c2f7f892e None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:41.011770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3f36b-2c82-4105-bd28-2632c120dd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:41.012376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3f36b-2c82-4105-bd28-2632c120dd52 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:41.012653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3f36b-2c82-4105-bd28-2632c120dd52 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:41.012922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3f36b-2c82-4105-bd28-2632c120dd52 None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:41.013646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3f36b-2c82-4105-bd28-2632c120dd52 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:41.013947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3f36b-2c82-4105-bd28-2632c120dd52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:41.015392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3f36b-2c82-4105-bd28-2632c120dd52 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:41.047785 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 414/799] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Oct 1 20:33:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:41.048622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3f36b-2c82-4105-bd28-2632c120dd52 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:41.073869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:41.113890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c40762-9564-47af-9d41-1b7e13c31f11 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:41.150769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcca489-e2af-4ab0-b3fe-7dd5e7846888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:41.151361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcca489-e2af-4ab0-b3fe-7dd5e7846888 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:41.151640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcca489-e2af-4ab0-b3fe-7dd5e7846888 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:41.151911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcca489-e2af-4ab0-b3fe-7dd5e7846888 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:41.152558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcca489-e2af-4ab0-b3fe-7dd5e7846888 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:41.152858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcca489-e2af-4ab0-b3fe-7dd5e7846888 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:41.154282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcca489-e2af-4ab0-b3fe-7dd5e7846888 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:41.167392 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 386/800] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:41.168233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcca489-e2af-4ab0-b3fe-7dd5e7846888 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:41.196893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3f36b-2c82-4105-bd28-2632c120dd52 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:41.203933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:41.240147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6c6d0-3dab-4faa-8ae3-6bfb32211254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:41.240759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6c6d0-3dab-4faa-8ae3-6bfb32211254 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:41.240983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6c6d0-3dab-4faa-8ae3-6bfb32211254 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:41.241256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6c6d0-3dab-4faa-8ae3-6bfb32211254 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:41.241835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6c6d0-3dab-4faa-8ae3-6bfb32211254 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:41.243212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6c6d0-3dab-4faa-8ae3-6bfb32211254 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:41.253141 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-5bd6c6d0-3dab-4faa-8ae3-6bfb32211254 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 01 20:33:41.253141 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:41.253141 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:33:41.253141 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:33:41.253141 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:33:41.253141 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:33:41.253141 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:33:41.253141 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:41.253141 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 01 20:33:41.253141 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:41.253141 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:33:41.253141 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:41.253141 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:41.253141 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:41.253141 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:41.253141 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:41.253141 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:41.254161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:41.254161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:41.254161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:41.254161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:41.254161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:41.254161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:41.254161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:33:41.254161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:33:41.254161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:33:41.254161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:33:41.254161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:41.254161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:41.254161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:41.254161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:41.254161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:41.254161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:41.255669 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:41.255669 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:41.255669 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:41.255669 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:41.255669 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:41.255669 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:41.255669 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:41.255669 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:41.255669 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:41.255669 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:41.255669 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:33:41.255669 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:41.255669 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 01 20:33:41.255669 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:41.255669 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 415/801] 10.222.0.17 () {56 vars in 1129 bytes} [Sat Oct 1 20:33:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:41.256466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6c6d0-3dab-4faa-8ae3-6bfb32211254 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:41.297423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcca489-e2af-4ab0-b3fe-7dd5e7846888 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:41.347491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:41.351672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e4821-2869-4144-a029-cf6f26b4e0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:41.352953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e4821-2869-4144-a029-cf6f26b4e0cd None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:41.353539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e4821-2869-4144-a029-cf6f26b4e0cd None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:41.354421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e4821-2869-4144-a029-cf6f26b4e0cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:41.373492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6c6d0-3dab-4faa-8ae3-6bfb32211254 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:41.410844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a099c-1763-4a7c-b7da-845c759c887a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:41.411671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a099c-1763-4a7c-b7da-845c759c887a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:41.411954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a099c-1763-4a7c-b7da-845c759c887a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:41.412227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a099c-1763-4a7c-b7da-845c759c887a None admin] PATH_INFO: `/v3/projects/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:41.419409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a099c-1763-4a7c-b7da-845c759c887a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:41.420836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a099c-1763-4a7c-b7da-845c759c887a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:41.431670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e4821-2869-4144-a029-cf6f26b4e0cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:41.431983 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-353a099c-1763-4a7c-b7da-845c759c887a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:41.431983 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:41.431983 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:41.431983 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:41.431983 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:41.431983 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:41.431983 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:41.431983 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:41.431983 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:41.431983 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:41.431983 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:41.431983 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:41.431983 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:41.431983 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:41.431983 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:41.433037 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:41.433037 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:41.433037 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:41.433037 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:41.433037 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:41.433037 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:41.433037 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:41.433037 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:41.433037 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:41.433037 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:41.433037 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:41.433037 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:41.433037 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:41.433037 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:41.433037 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:41.433037 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:41.433037 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:41.434149 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:41.434149 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:41.434149 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:41.434149 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:41.434149 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:41.434149 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:41.434149 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:41.434149 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:41.434149 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 416/802] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:41.434149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a099c-1763-4a7c-b7da-845c759c887a None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:41.435909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e4821-2869-4144-a029-cf6f26b4e0cd None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:41.472470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:41.544293 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 387/803] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:33:41 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 01 20:33:41.545658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c45f1-c753-405c-bf11-860c624fe6a1 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:41.545766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a099c-1763-4a7c-b7da-845c759c887a None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:41.545987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c45f1-c753-405c-bf11-860c624fe6a1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:41.546223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c45f1-c753-405c-bf11-860c624fe6a1 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:41.548251 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 388/804] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:41.548883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c45f1-c753-405c-bf11-860c624fe6a1 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:41.584982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b624f2a-ba44-4f89-9620-e55e740f3567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:41.585584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b624f2a-ba44-4f89-9620-e55e740f3567 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:41.585868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b624f2a-ba44-4f89-9620-e55e740f3567 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:41.586148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b624f2a-ba44-4f89-9620-e55e740f3567 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:41.586974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b624f2a-ba44-4f89-9620-e55e740f3567 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:41.587281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b624f2a-ba44-4f89-9620-e55e740f3567 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:41.588643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b624f2a-ba44-4f89-9620-e55e740f3567 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:41.589716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:41.598657 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 417/805] 10.222.0.17 () {56 vars in 1124 bytes} [Sat Oct 1 20:33:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:41.599943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c421e2-34d4-44cb-be88-383a4131c3fd None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:41.600228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c421e2-34d4-44cb-be88-383a4131c3fd None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:41.600525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c421e2-34d4-44cb-be88-383a4131c3fd None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:41.601836 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 418/806] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:41.602601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c421e2-34d4-44cb-be88-383a4131c3fd None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:41.659057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c45f1-c753-405c-bf11-860c624fe6a1 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:41.692525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d657ad0-f0ce-452a-8727-2e666e40a750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:41.693060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d657ad0-f0ce-452a-8727-2e666e40a750 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:41.693334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d657ad0-f0ce-452a-8727-2e666e40a750 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:41.693598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d657ad0-f0ce-452a-8727-2e666e40a750 None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:41.694272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d657ad0-f0ce-452a-8727-2e666e40a750 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:41.694600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d657ad0-f0ce-452a-8727-2e666e40a750 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:41.696271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d657ad0-f0ce-452a-8727-2e666e40a750 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:41.697811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:41.706209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c421e2-34d4-44cb-be88-383a4131c3fd None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:41.709507 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 389/807] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:41.710708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67489769-f4ab-495b-a15f-e678af48e9f5 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:41.711001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67489769-f4ab-495b-a15f-e678af48e9f5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:41.711275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67489769-f4ab-495b-a15f-e678af48e9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:41.730314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-67489769-f4ab-495b-a15f-e678af48e9f5 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:41.731543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-67489769-f4ab-495b-a15f-e678af48e9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:41.741557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59e897-2cfe-4b75-b459-b2ba4c25e39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:41.742171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59e897-2cfe-4b75-b459-b2ba4c25e39b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:41.742445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59e897-2cfe-4b75-b459-b2ba4c25e39b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:41.742753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59e897-2cfe-4b75-b459-b2ba4c25e39b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:41.753036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59e897-2cfe-4b75-b459-b2ba4c25e39b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:41.754440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59e897-2cfe-4b75-b459-b2ba4c25e39b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:41.768887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-ab59e897-2cfe-4b75-b459-b2ba4c25e39b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:41.768887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:41.768887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:41.768887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:41.768887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:41.768887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:41.768887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:41.768887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:41.768887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:41.768887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:41.768887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:41.768887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:41.768887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:41.768887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:41.768887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:41.768887 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:41.769998 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:41.769998 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:41.769998 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:41.769998 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:41.769998 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:41.769998 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:41.769998 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:41.769998 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:41.769998 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:41.769998 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:41.769998 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:41.769998 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:41.769998 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:41.769998 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:41.769998 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:41.769998 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:41.769998 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:41.770976 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:41.770976 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:41.770976 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:41.770976 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 419/808] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:41.770976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59e897-2cfe-4b75-b459-b2ba4c25e39b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:41.804220 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 390/809] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:33:41.805487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18007c00-81ca-40ab-8e01-ddfab0341ebf None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:41.805785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18007c00-81ca-40ab-8e01-ddfab0341ebf None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:41.806047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18007c00-81ca-40ab-8e01-ddfab0341ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:41.830993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:41.834882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-18007c00-81ca-40ab-8e01-ddfab0341ebf None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:41.836088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-18007c00-81ca-40ab-8e01-ddfab0341ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:41.877118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59e897-2cfe-4b75-b459-b2ba4c25e39b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:41.906945 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 391/810] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:33:41.908057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18007c00-81ca-40ab-8e01-ddfab0341ebf None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:41.921956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d5798-73c7-4732-a0ed-0a7bc52e2511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:41.923125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d5798-73c7-4732-a0ed-0a7bc52e2511 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:41.924060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d5798-73c7-4732-a0ed-0a7bc52e2511 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:41.924960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d5798-73c7-4732-a0ed-0a7bc52e2511 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:41.926082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d5798-73c7-4732-a0ed-0a7bc52e2511 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:41.926683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d5798-73c7-4732-a0ed-0a7bc52e2511 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:41.928681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d5798-73c7-4732-a0ed-0a7bc52e2511 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:41.936703 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 420/811] 10.222.0.17 () {56 vars in 1124 bytes} [Sat Oct 1 20:33:41 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:41.938604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d5798-73c7-4732-a0ed-0a7bc52e2511 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:41.966837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:42.013936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18007c00-81ca-40ab-8e01-ddfab0341ebf None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:42.062001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77e704-e1cb-4166-8182-1450190649f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:42.062792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77e704-e1cb-4166-8182-1450190649f3 None admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:42.063694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77e704-e1cb-4166-8182-1450190649f3 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:42.063979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77e704-e1cb-4166-8182-1450190649f3 None admin] PATH_INFO: `/v3/projects/b7875946723844198353bbfcf22516fb/users/c4dc51e2b4474d6ba9fd5d2bc3702826/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:42.066051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77e704-e1cb-4166-8182-1450190649f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7875946723844198353bbfcf22516fb, user_id=c4dc51e2b4474d6ba9fd5d2bc3702826, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:42.067575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77e704-e1cb-4166-8182-1450190649f3 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:42.073941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d5798-73c7-4732-a0ed-0a7bc52e2511 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:42.090549 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 392/812] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:33:41 2022] PUT /identity/v3/projects/b7875946723844198353bbfcf22516fb/users/c4dc51e2b4474d6ba9fd5d2bc3702826/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:33:42.090549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77e704-e1cb-4166-8182-1450190649f3 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:42.102879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:42.151187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91a7c7-c9fd-4c1b-ad6a-be29b31c3a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:42.153172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91a7c7-c9fd-4c1b-ad6a-be29b31c3a55 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:42.153451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91a7c7-c9fd-4c1b-ad6a-be29b31c3a55 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:42.153722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91a7c7-c9fd-4c1b-ad6a-be29b31c3a55 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:42.156459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91a7c7-c9fd-4c1b-ad6a-be29b31c3a55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:42.159230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91a7c7-c9fd-4c1b-ad6a-be29b31c3a55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:42.162218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91a7c7-c9fd-4c1b-ad6a-be29b31c3a55 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:42.177970 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 421/813] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:42.179068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91a7c7-c9fd-4c1b-ad6a-be29b31c3a55 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:42.263244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77e704-e1cb-4166-8182-1450190649f3 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:42.280650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:42.300588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91a7c7-c9fd-4c1b-ad6a-be29b31c3a55 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:42.308129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9b51b-daaa-486f-ac8b-3f85581999b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:42.308806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9b51b-daaa-486f-ac8b-3f85581999b7 None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:42.309085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9b51b-daaa-486f-ac8b-3f85581999b7 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:42.309355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9b51b-daaa-486f-ac8b-3f85581999b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:42.310017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9b51b-daaa-486f-ac8b-3f85581999b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:42.311422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9b51b-daaa-486f-ac8b-3f85581999b7 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:42.340705 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 393/814] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:33:42 2022] POST /identity/v3/endpoints => generated 346 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:33:42.341610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9b51b-daaa-486f-ac8b-3f85581999b7 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:42.373331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7f269-885a-4c10-ba95-9e34ec614112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:42.374627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7f269-885a-4c10-ba95-9e34ec614112 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:42.375619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7f269-885a-4c10-ba95-9e34ec614112 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:42.376373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7f269-885a-4c10-ba95-9e34ec614112 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:42.393321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd7f269-885a-4c10-ba95-9e34ec614112 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:42.400489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-8bd7f269-885a-4c10-ba95-9e34ec614112 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:42.400489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:42.400489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:42.400489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:42.400489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:42.400489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:42.400489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:42.400489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:42.400489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:42.400489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:42.400489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:42.400489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:42.400489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:42.400489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:42.400489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:42.400489 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:42.401480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:42.401480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:42.401480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:42.401480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:42.401480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:42.401480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:42.401480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:42.401480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:42.401480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:42.401480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:42.401480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:42.401480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:42.401480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:42.401480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:42.401480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:42.401480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:42.402319 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:42.402319 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:42.402319 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:42.402319 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:42.402319 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:42.402319 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:42.402319 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:42.402319 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:42.402319 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:42.402319 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:42.402319 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:42.402319 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:42.404069 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 422/815] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:42.410553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7f269-885a-4c10-ba95-9e34ec614112 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:42.454678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:42.478168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9b51b-daaa-486f-ac8b-3f85581999b7 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:42.521982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2705b26-5549-44f1-b6e9-f9fa0c19db7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:42.523051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2705b26-5549-44f1-b6e9-f9fa0c19db7f None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:42.523549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2705b26-5549-44f1-b6e9-f9fa0c19db7f None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:42.524033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2705b26-5549-44f1-b6e9-f9fa0c19db7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:42.534430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2705b26-5549-44f1-b6e9-f9fa0c19db7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:42.536577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2705b26-5549-44f1-b6e9-f9fa0c19db7f None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:42.558682 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-b2705b26-5549-44f1-b6e9-f9fa0c19db7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:42.558682 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:42.558682 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:42.558682 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:42.558682 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:42.558682 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:42.558682 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:42.558682 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:42.558682 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:42.558682 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:42.558682 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:42.558682 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:42.558682 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:42.558682 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:42.558682 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:42.559665 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:42.559665 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:42.559665 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:42.559665 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:42.559665 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:42.559665 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:42.559665 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:42.559665 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:42.559665 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:42.559665 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:42.559665 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:42.559665 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:42.559665 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:42.559665 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:42.559665 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:42.559665 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:42.559665 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:42.560558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:42.560558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:42.560558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:42.560558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:42.560558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:42.560558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:42.560558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:42.560558 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:42.560558 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 394/816] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:42.561649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2705b26-5549-44f1-b6e9-f9fa0c19db7f None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:42.575777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7f269-885a-4c10-ba95-9e34ec614112 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:42.596123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:42.625633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f91d17-7f83-48f2-9184-f83e25bde505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:42.627093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f91d17-7f83-48f2-9184-f83e25bde505 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:42.627857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f91d17-7f83-48f2-9184-f83e25bde505 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:42.628328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f91d17-7f83-48f2-9184-f83e25bde505 None admin] PATH_INFO: `/v3/services/network` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:42.629365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f91d17-7f83-48f2-9184-f83e25bde505 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:42.631911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f91d17-7f83-48f2-9184-f83e25bde505 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:42.643945 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-20f91d17-7f83-48f2-9184-f83e25bde505 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 01 20:33:42.643945 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:42.643945 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:33:42.643945 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:33:42.643945 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:33:42.643945 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:33:42.643945 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:33:42.643945 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:42.643945 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 01 20:33:42.643945 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:42.643945 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:33:42.643945 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:42.643945 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:42.643945 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:42.643945 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:42.643945 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:42.643945 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:42.644978 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:42.644978 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:42.644978 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:42.644978 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:42.644978 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:42.644978 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:42.644978 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:33:42.644978 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:33:42.644978 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:33:42.644978 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:33:42.644978 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:42.644978 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:42.644978 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:42.644978 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:42.644978 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:42.644978 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:42.645845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:42.645845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:42.645845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:42.645845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:42.645845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:42.645845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:42.645845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:42.645845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:42.645845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:42.645845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:42.645845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:33:42.645845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:42.645845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 01 20:33:42.645845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:42.645845 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 423/817] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:42 2022] GET /identity/v3/services/network => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:42.646684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f91d17-7f83-48f2-9184-f83e25bde505 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:42.685736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2705b26-5549-44f1-b6e9-f9fa0c19db7f None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:42.729003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce8941-9737-4a4d-a3ec-826001f1dca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:42.729964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce8941-9737-4a4d-a3ec-826001f1dca5 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:42.730439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce8941-9737-4a4d-a3ec-826001f1dca5 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:42.730971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce8941-9737-4a4d-a3ec-826001f1dca5 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:42.731985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce8941-9737-4a4d-a3ec-826001f1dca5 None admin] RBAC: Adding query filter params (name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:42.732497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce8941-9737-4a4d-a3ec-826001f1dca5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:42.735155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce8941-9737-4a4d-a3ec-826001f1dca5 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:42.751877 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 395/818] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:33:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:42.751877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce8941-9737-4a4d-a3ec-826001f1dca5 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:42.758158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:42.765873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f91d17-7f83-48f2-9184-f83e25bde505 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:42.805025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc968b1-0ab9-416b-86da-21714f3f265b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:42.806319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc968b1-0ab9-416b-86da-21714f3f265b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:42.806827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc968b1-0ab9-416b-86da-21714f3f265b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:42.807299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc968b1-0ab9-416b-86da-21714f3f265b None admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:42.808802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc968b1-0ab9-416b-86da-21714f3f265b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:42.809331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc968b1-0ab9-416b-86da-21714f3f265b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:42.811585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc968b1-0ab9-416b-86da-21714f3f265b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:42.830010 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 424/819] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:42.831705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc968b1-0ab9-416b-86da-21714f3f265b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:42.862867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:42.870560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce8941-9737-4a4d-a3ec-826001f1dca5 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:42.911646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f4e89-6cb9-4deb-9309-6b56be17fd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:42.912264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f4e89-6cb9-4deb-9309-6b56be17fd9f None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:42.912548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f4e89-6cb9-4deb-9309-6b56be17fd9f None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:42.912823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f4e89-6cb9-4deb-9309-6b56be17fd9f None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:42.913458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f4e89-6cb9-4deb-9309-6b56be17fd9f None admin] RBAC: Adding query filter params (name=network) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:42.913760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f4e89-6cb9-4deb-9309-6b56be17fd9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:42.915442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f4e89-6cb9-4deb-9309-6b56be17fd9f None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:42.922461 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 396/820] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Oct 1 20:33:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:42.923380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f4e89-6cb9-4deb-9309-6b56be17fd9f None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:42.955489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc968b1-0ab9-416b-86da-21714f3f265b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:43.002455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b38c5b-e1ea-411d-bd5b-cc8246ff810e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:43.003251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b38c5b-e1ea-411d-bd5b-cc8246ff810e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:43.003640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b38c5b-e1ea-411d-bd5b-cc8246ff810e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:43.004005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b38c5b-e1ea-411d-bd5b-cc8246ff810e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:43.018337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b38c5b-e1ea-411d-bd5b-cc8246ff810e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:43.019915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b38c5b-e1ea-411d-bd5b-cc8246ff810e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:43.045910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-f9b38c5b-e1ea-411d-bd5b-cc8246ff810e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:43.045910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:43.045910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:43.045910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:43.045910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:43.045910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:43.045910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:43.045910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:43.045910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:43.045910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:43.045910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:43.045910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:43.045910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:43.045910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:43.045910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:43.045910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:43.047638 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:43.047638 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:43.047638 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:43.047638 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:43.047638 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:43.047638 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:43.047638 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:43.047638 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:43.047638 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:43.047638 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:43.047638 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:43.047638 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:43.047638 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:43.047638 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:43.047638 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:43.047638 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:43.047638 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:43.048765 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:43.048765 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:43.048765 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:43.054521 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 425/821] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:43.055525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b38c5b-e1ea-411d-bd5b-cc8246ff810e None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:43.058811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f4e89-6cb9-4deb-9309-6b56be17fd9f None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:43.065528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:43.107352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3a065-a4d7-4397-8231-c14d4f130703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:43.108497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3a065-a4d7-4397-8231-c14d4f130703 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:43.108926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3a065-a4d7-4397-8231-c14d4f130703 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:43.109390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3a065-a4d7-4397-8231-c14d4f130703 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:43.110848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3a065-a4d7-4397-8231-c14d4f130703 None admin] RBAC: Adding query filter params (role.id=1c50896511054a5f994bee3d99fa0619, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=7ab975dafb564aa2b357311c4a384d53) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:43.111334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3a065-a4d7-4397-8231-c14d4f130703 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:43.112916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3a065-a4d7-4397-8231-c14d4f130703 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:43.121135 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 397/822] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:33:42 2022] GET /identity/v3/role_assignments?user.id=7ab975dafb564aa2b357311c4a384d53&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=1c50896511054a5f994bee3d99fa0619 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:43.122345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3a065-a4d7-4397-8231-c14d4f130703 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:43.167542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:43.203601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b38c5b-e1ea-411d-bd5b-cc8246ff810e None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:43.286793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655aefa-d71c-4c10-80b7-b5a154f2d187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:43.290577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655aefa-d71c-4c10-80b7-b5a154f2d187 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:43.290868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655aefa-d71c-4c10-80b7-b5a154f2d187 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:43.291148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655aefa-d71c-4c10-80b7-b5a154f2d187 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:43.291875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655aefa-d71c-4c10-80b7-b5a154f2d187 None admin] RBAC: Adding query filter params (type=network) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:43.292249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655aefa-d71c-4c10-80b7-b5a154f2d187 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:43.293657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655aefa-d71c-4c10-80b7-b5a154f2d187 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:43.306319 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 426/823] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Oct 1 20:33:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:43.311855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655aefa-d71c-4c10-80b7-b5a154f2d187 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:43.355989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3a065-a4d7-4397-8231-c14d4f130703 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:43.421034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3ec65-ec1b-477d-ae04-2bb9c8af9c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:43.421667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3ec65-ec1b-477d-ae04-2bb9c8af9c92 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:43.421975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3ec65-ec1b-477d-ae04-2bb9c8af9c92 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:43.422286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3ec65-ec1b-477d-ae04-2bb9c8af9c92 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:43.422963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3ec65-ec1b-477d-ae04-2bb9c8af9c92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:43.423264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3ec65-ec1b-477d-ae04-2bb9c8af9c92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:43.424620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3ec65-ec1b-477d-ae04-2bb9c8af9c92 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:43.429880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:43.440889 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 398/824] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:43.446693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3ec65-ec1b-477d-ae04-2bb9c8af9c92 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:43.477699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655aefa-d71c-4c10-80b7-b5a154f2d187 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:43.518682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cb04d-1c7a-4bcc-92cd-f1b5edf614e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:43.519436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cb04d-1c7a-4bcc-92cd-f1b5edf614e3 None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:43.519810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cb04d-1c7a-4bcc-92cd-f1b5edf614e3 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:43.520179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cb04d-1c7a-4bcc-92cd-f1b5edf614e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:43.521043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272cb04d-1c7a-4bcc-92cd-f1b5edf614e3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:43.522607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272cb04d-1c7a-4bcc-92cd-f1b5edf614e3 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:43.539655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:43.551800 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 427/825] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:33:43 2022] POST /identity/v3/endpoints => generated 337 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:33:43.567123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3ec65-ec1b-477d-ae04-2bb9c8af9c92 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:43.619767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91e0db-f763-4ef9-b61f-3b0559b9db65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:43.620419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91e0db-f763-4ef9-b61f-3b0559b9db65 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:43.620875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91e0db-f763-4ef9-b61f-3b0559b9db65 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:43.621147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91e0db-f763-4ef9-b61f-3b0559b9db65 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:43.632672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91e0db-f763-4ef9-b61f-3b0559b9db65 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:43.632672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91e0db-f763-4ef9-b61f-3b0559b9db65 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:43.646585 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-fa91e0db-f763-4ef9-b61f-3b0559b9db65 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 01 20:33:43.646585 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:43.646585 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:43.646585 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:43.646585 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:43.646585 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:43.646585 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:43.646585 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:43.646585 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:43.646585 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:43.646585 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:43.646585 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:43.646585 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:43.646585 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:43.646585 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:43.646585 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:43.647638 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:43.647638 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:43.647638 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:43.647638 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:43.647638 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:43.647638 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:43.647638 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:43.647638 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:43.647638 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:43.647638 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:43.647638 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:43.647638 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:43.647638 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:43.647638 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:43.647638 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:43.647638 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:43.647638 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:43.647638 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:43.648580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:43.648580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:43.648580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:43.648580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:43.648580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:43.648580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:43.648580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:43.648580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:43.648580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 01 20:33:43.648580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:43.648580 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 399/826] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Oct 1 20:33:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:43.653687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cb04d-1c7a-4bcc-92cd-f1b5edf614e3 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:43.778786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cb04d-1c7a-4bcc-92cd-f1b5edf614e3 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:43.808642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:43.819995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07755403-a527-40bb-9f7f-2a01c8ca911d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:43.821291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07755403-a527-40bb-9f7f-2a01c8ca911d None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:43.821850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07755403-a527-40bb-9f7f-2a01c8ca911d None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:43.822326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07755403-a527-40bb-9f7f-2a01c8ca911d None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:43.823354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07755403-a527-40bb-9f7f-2a01c8ca911d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:43.823931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07755403-a527-40bb-9f7f-2a01c8ca911d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:43.825741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07755403-a527-40bb-9f7f-2a01c8ca911d None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:43.846270 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 428/827] 10.222.0.17 () {56 vars in 1149 bytes} [Sat Oct 1 20:33:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:43.854523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91e0db-f763-4ef9-b61f-3b0559b9db65 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:43.984356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91e0db-f763-4ef9-b61f-3b0559b9db65 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:44.019385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4025056-dca6-49b1-95d6-40019a2bd0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:44.020023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4025056-dca6-49b1-95d6-40019a2bd0d7 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:44.020301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4025056-dca6-49b1-95d6-40019a2bd0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:44.020572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4025056-dca6-49b1-95d6-40019a2bd0d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:44.023489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:44.032694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4025056-dca6-49b1-95d6-40019a2bd0d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:44.034142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4025056-dca6-49b1-95d6-40019a2bd0d7 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:44.049757 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-a4025056-dca6-49b1-95d6-40019a2bd0d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:44.049757 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:44.049757 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:44.049757 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:44.049757 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:44.049757 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:44.049757 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:44.049757 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:44.049757 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:44.049757 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:44.049757 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:44.049757 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:44.049757 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:44.049757 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:44.049757 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:44.049757 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:44.050728 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:44.050728 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:44.050728 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:44.050728 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:44.050728 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:44.050728 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:44.050728 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:44.050728 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:44.050728 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:44.050728 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:44.050728 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:44.050728 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:44.050728 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:44.050728 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:44.050728 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:44.050728 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:44.050728 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:44.051632 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:44.051632 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:44.051632 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:44.051632 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 400/828] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:44.059568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07755403-a527-40bb-9f7f-2a01c8ca911d None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:44.177008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07755403-a527-40bb-9f7f-2a01c8ca911d None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:44.211727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686def6-cd6e-4e15-9144-ce3243221acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:44.212381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686def6-cd6e-4e15-9144-ce3243221acc None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:44.212606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686def6-cd6e-4e15-9144-ce3243221acc None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:44.212862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686def6-cd6e-4e15-9144-ce3243221acc None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:44.213511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686def6-cd6e-4e15-9144-ce3243221acc None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:44.213812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686def6-cd6e-4e15-9144-ce3243221acc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:44.215227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686def6-cd6e-4e15-9144-ce3243221acc None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:44.228850 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 429/829] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:44.235702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4025056-dca6-49b1-95d6-40019a2bd0d7 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:44.345449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4025056-dca6-49b1-95d6-40019a2bd0d7 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:44.379891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60645e1c-e7b7-45ce-bae2-cd95ad132aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:44.380563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60645e1c-e7b7-45ce-bae2-cd95ad132aa9 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:44.380846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60645e1c-e7b7-45ce-bae2-cd95ad132aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:44.381039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60645e1c-e7b7-45ce-bae2-cd95ad132aa9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:44.388123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60645e1c-e7b7-45ce-bae2-cd95ad132aa9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:44.389567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60645e1c-e7b7-45ce-bae2-cd95ad132aa9 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:44.401288 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-60645e1c-e7b7-45ce-bae2-cd95ad132aa9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:44.401288 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:44.401288 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:44.401288 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:44.401288 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:44.401288 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:44.401288 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:44.401288 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:44.401288 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:44.401288 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:44.401288 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:44.401288 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:44.401288 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:44.401288 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:44.401288 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:44.402264 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:44.402264 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:44.402264 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:44.402264 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:44.402264 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:44.402264 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:44.402264 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:44.402264 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:44.402264 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:44.402264 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:44.402264 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:44.402264 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:44.402264 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:44.402264 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:44.402264 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:44.402264 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:44.402264 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:44.403209 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:44.403209 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:44.403209 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:44.403209 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:44.403209 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:44.403209 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:44.403209 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:44.403209 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:44.403209 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 401/830] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:44.411599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686def6-cd6e-4e15-9144-ce3243221acc None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:44.459546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:44.514285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686def6-cd6e-4e15-9144-ce3243221acc None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:44.548790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1231a-c253-4000-87d6-7e28a630ac3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:44.549404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1231a-c253-4000-87d6-7e28a630ac3d None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:44.549682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1231a-c253-4000-87d6-7e28a630ac3d None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:44.549952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1231a-c253-4000-87d6-7e28a630ac3d None admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:44.550675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1231a-c253-4000-87d6-7e28a630ac3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:44.550980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1231a-c253-4000-87d6-7e28a630ac3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:44.552399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1231a-c253-4000-87d6-7e28a630ac3d None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:44.568921 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 430/831] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:44.574974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60645e1c-e7b7-45ce-bae2-cd95ad132aa9 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:44.633409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:44.673854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c7787-2cb6-498b-9af1-34e453f11a94 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:44.674212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c7787-2cb6-498b-9af1-34e453f11a94 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:44.674407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c7787-2cb6-498b-9af1-34e453f11a94 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:44.675969 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 431/832] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:44.682191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f12af-8596-4a72-8960-3554c83acb98 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:44.682772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f12af-8596-4a72-8960-3554c83acb98 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:44.683488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f12af-8596-4a72-8960-3554c83acb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:44.702051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60645e1c-e7b7-45ce-bae2-cd95ad132aa9 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:44.709350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-979f12af-8596-4a72-8960-3554c83acb98 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:44.711084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-979f12af-8596-4a72-8960-3554c83acb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:44.738268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75212ec5-fa94-4a1a-94e5-d6e3b07e0f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:44.738959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75212ec5-fa94-4a1a-94e5-d6e3b07e0f5c None admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:44.739238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75212ec5-fa94-4a1a-94e5-d6e3b07e0f5c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:44.739514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75212ec5-fa94-4a1a-94e5-d6e3b07e0f5c None admin] PATH_INFO: `/v3/projects/b7875946723844198353bbfcf22516fb/users/917fc96fa3504b19b1a56990d4747439/roles/ef7bd7293369475a9e7b3444fac30cda` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:44.755057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75212ec5-fa94-4a1a-94e5-d6e3b07e0f5c None admin] RBAC: Authorizing `identity:create_grant(project_id=b7875946723844198353bbfcf22516fb, user_id=917fc96fa3504b19b1a56990d4747439, role_id=ef7bd7293369475a9e7b3444fac30cda)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:44.756496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75212ec5-fa94-4a1a-94e5-d6e3b07e0f5c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:44.771365 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 402/833] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:33:44 2022] PUT /identity/v3/projects/b7875946723844198353bbfcf22516fb/users/917fc96fa3504b19b1a56990d4747439/roles/ef7bd7293369475a9e7b3444fac30cda => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:33:44.794292 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 432/834] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 01 20:33:45.122832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75212ec5-fa94-4a1a-94e5-d6e3b07e0f5c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:45.128993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b06ab-63e1-41b0-8f1d-162e038407de None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:45.129283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b06ab-63e1-41b0-8f1d-162e038407de None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:45.129553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b06ab-63e1-41b0-8f1d-162e038407de None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:45.131164 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 433/835] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:33:45.135902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0962dfc-d660-4e71-a30c-4ac46cf9935c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:45.136185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0962dfc-d660-4e71-a30c-4ac46cf9935c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:45.136460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0962dfc-d660-4e71-a30c-4ac46cf9935c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:45.154832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e0962dfc-d660-4e71-a30c-4ac46cf9935c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:45.156034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e0962dfc-d660-4e71-a30c-4ac46cf9935c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:45.192024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:45.238001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:45.244139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75212ec5-fa94-4a1a-94e5-d6e3b07e0f5c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:45.250721 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 434/836] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 01 20:33:45.281606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40fb94-18e6-4c90-8665-d8b80ade61c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:45.282237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40fb94-18e6-4c90-8665-d8b80ade61c0 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:45.282548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40fb94-18e6-4c90-8665-d8b80ade61c0 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:45.282823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40fb94-18e6-4c90-8665-d8b80ade61c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:45.293066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd40fb94-18e6-4c90-8665-d8b80ade61c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:45.298343 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-bd40fb94-18e6-4c90-8665-d8b80ade61c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:33:45.298343 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:45.298343 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:45.298343 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:45.298343 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:45.298343 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:45.298343 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:45.298343 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:45.298343 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:45.298343 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:45.298343 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:45.298343 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:45.298343 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:45.298343 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:45.298343 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:45.298343 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:45.299298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:45.299298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:45.299298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:45.299298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:45.299298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:45.299298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:45.299298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:45.299298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:45.299298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:45.299298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:45.299298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:45.299298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:45.299298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:45.299298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:45.299298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:45.299298 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:45.300157 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:45.300157 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:45.300157 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:45.300157 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:45.300157 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:45.300157 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:45.300157 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:45.300157 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:45.300157 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:45.300157 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:45.300157 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:33:45.300157 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:45.300157 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 403/837] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:33:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:45.307622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0962dfc-d660-4e71-a30c-4ac46cf9935c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:45.411272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0962dfc-d660-4e71-a30c-4ac46cf9935c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:45.445405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301d4c3-7d26-4317-b603-1d0c64bb3713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:45.446329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301d4c3-7d26-4317-b603-1d0c64bb3713 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:45.446643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301d4c3-7d26-4317-b603-1d0c64bb3713 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:45.446994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301d4c3-7d26-4317-b603-1d0c64bb3713 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:45.447664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f301d4c3-7d26-4317-b603-1d0c64bb3713 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:45.447967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f301d4c3-7d26-4317-b603-1d0c64bb3713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:45.449331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f301d4c3-7d26-4317-b603-1d0c64bb3713 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:45.462613 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 435/838] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:33:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:45.464302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40fb94-18e6-4c90-8665-d8b80ade61c0 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:45.568603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40fb94-18e6-4c90-8665-d8b80ade61c0 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:45.596256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301d4c3-7d26-4317-b603-1d0c64bb3713 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:45.608634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd690c5-2ebc-4434-a24a-a8a00a247b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:45.609405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd690c5-2ebc-4434-a24a-a8a00a247b0e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:45.609696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd690c5-2ebc-4434-a24a-a8a00a247b0e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:45.610022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd690c5-2ebc-4434-a24a-a8a00a247b0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:45.621408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd690c5-2ebc-4434-a24a-a8a00a247b0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:45.623020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd690c5-2ebc-4434-a24a-a8a00a247b0e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:45.626566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:45.644159 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-bdd690c5-2ebc-4434-a24a-a8a00a247b0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:45.644159 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:45.644159 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:45.644159 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:45.644159 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:45.644159 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:45.644159 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:45.644159 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:45.644159 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:45.644159 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:45.644159 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:45.644159 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:45.644159 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:45.644159 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:45.644159 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:45.644159 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:45.645172 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:45.645172 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:45.645172 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:45.645172 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:45.645172 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:45.645172 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:45.645172 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:45.645172 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:45.645172 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:45.645172 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:45.645172 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:45.645172 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:45.645172 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:45.645172 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:45.645172 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:45.645172 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:45.645172 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:45.646084 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:45.646084 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:45.646084 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:45.650558 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 404/839] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:45.652074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd690c5-2ebc-4434-a24a-a8a00a247b0e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:45.671215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:45.721294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301d4c3-7d26-4317-b603-1d0c64bb3713 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:45.762679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b151432-e1fa-4d69-b3ee-f26dc885a870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:45.763337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b151432-e1fa-4d69-b3ee-f26dc885a870 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:45.763620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b151432-e1fa-4d69-b3ee-f26dc885a870 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:45.763898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b151432-e1fa-4d69-b3ee-f26dc885a870 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:45.764820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b151432-e1fa-4d69-b3ee-f26dc885a870 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:45.767529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b151432-e1fa-4d69-b3ee-f26dc885a870 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:45.779394 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-8b151432-e1fa-4d69-b3ee-f26dc885a870 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 01 20:33:45.779394 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:45.779394 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:33:45.779394 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:33:45.779394 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:33:45.779394 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:33:45.779394 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:33:45.779394 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:45.779394 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 01 20:33:45.779394 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:45.779394 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:33:45.779394 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:45.779394 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:45.779394 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:45.779394 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:45.779394 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:45.779394 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:45.780495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:45.780495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:45.780495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:45.780495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:45.780495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:45.780495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:45.780495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:33:45.780495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:33:45.780495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:33:45.780495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:33:45.780495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:45.780495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:45.780495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:45.780495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:45.780495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:45.780495 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:45.786955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:45.786955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:45.786955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:45.786955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:45.786955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:45.786955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:45.786955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:45.786955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:45.786955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:45.786955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:45.786955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:33:45.786955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:45.786955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 01 20:33:45.786955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:45.786955 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 436/840] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:33:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:45.787844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b151432-e1fa-4d69-b3ee-f26dc885a870 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:45.794729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd690c5-2ebc-4434-a24a-a8a00a247b0e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:45.840856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb05e46-ff5b-4e33-84e6-08f1ee3db3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:45.841396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb05e46-ff5b-4e33-84e6-08f1ee3db3f8 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:45.841679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb05e46-ff5b-4e33-84e6-08f1ee3db3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:45.841959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb05e46-ff5b-4e33-84e6-08f1ee3db3f8 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:45.842644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb05e46-ff5b-4e33-84e6-08f1ee3db3f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:45.842954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb05e46-ff5b-4e33-84e6-08f1ee3db3f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:45.845608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb05e46-ff5b-4e33-84e6-08f1ee3db3f8 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:45.863993 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 405/841] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:45.871083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb05e46-ff5b-4e33-84e6-08f1ee3db3f8 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:45.909187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:45.924556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b151432-e1fa-4d69-b3ee-f26dc885a870 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:45.964212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:45.973354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8a5f1-086a-4a33-b185-551d536c6037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:45.973976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8a5f1-086a-4a33-b185-551d536c6037 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:45.974263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8a5f1-086a-4a33-b185-551d536c6037 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:45.974568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8a5f1-086a-4a33-b185-551d536c6037 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:45.975211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8a5f1-086a-4a33-b185-551d536c6037 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:45.975518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8a5f1-086a-4a33-b185-551d536c6037 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:45.977356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8a5f1-086a-4a33-b185-551d536c6037 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:45.989072 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 437/842] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:33:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:45.990083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8a5f1-086a-4a33-b185-551d536c6037 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:46.008698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb05e46-ff5b-4e33-84e6-08f1ee3db3f8 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:46.057428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be49bb39-00cd-4bf9-bceb-b711433f6289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:46.058351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49bb39-00cd-4bf9-bceb-b711433f6289 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:46.058672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49bb39-00cd-4bf9-bceb-b711433f6289 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:46.058950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49bb39-00cd-4bf9-bceb-b711433f6289 None admin] PATH_INFO: `/v3/users/swift` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:46.069668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be49bb39-00cd-4bf9-bceb-b711433f6289 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:46.071768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be49bb39-00cd-4bf9-bceb-b711433f6289 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:46.086887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-be49bb39-00cd-4bf9-bceb-b711433f6289 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 01 20:33:46.086887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:46.086887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:46.086887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:46.086887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:46.086887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:46.086887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:46.086887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:46.086887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:46.086887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:46.086887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:46.086887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:46.086887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:46.086887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:46.086887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:46.086887 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:46.087865 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:46.087865 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:46.087865 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:46.087865 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:46.087865 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:46.087865 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:46.087865 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:46.087865 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:46.087865 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:46.087865 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:46.087865 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:46.087865 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:46.087865 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:46.087865 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:46.087865 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:46.087865 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:46.087865 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:46.087865 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:46.088810 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:46.088810 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:46.088810 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:46.088810 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:46.088810 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:46.088810 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:46.088810 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:46.088810 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:46.088810 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 01 20:33:46.088810 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:46.091488 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 406/843] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:33:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:46.099306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be49bb39-00cd-4bf9-bceb-b711433f6289 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:46.111106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8a5f1-086a-4a33-b185-551d536c6037 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:46.159166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030099d-d652-4a20-a18a-2597638970f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:46.159823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030099d-d652-4a20-a18a-2597638970f3 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:46.160109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030099d-d652-4a20-a18a-2597638970f3 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:46.160386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030099d-d652-4a20-a18a-2597638970f3 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:46.161303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030099d-d652-4a20-a18a-2597638970f3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:46.161613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030099d-d652-4a20-a18a-2597638970f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:46.163275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030099d-d652-4a20-a18a-2597638970f3 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:46.170532 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 438/844] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:33:45 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:46.188897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea3e1e-f9f5-449b-8f29-77735e668fdb None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:46.189183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea3e1e-f9f5-449b-8f29-77735e668fdb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:46.189460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea3e1e-f9f5-449b-8f29-77735e668fdb None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:46.191014 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 439/845] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:33:46.196865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1cf7c-17b5-406d-bfe4-a62bf871167e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:46.197177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1cf7c-17b5-406d-bfe4-a62bf871167e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:46.197420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1cf7c-17b5-406d-bfe4-a62bf871167e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:46.218255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-79a1cf7c-17b5-406d-bfe4-a62bf871167e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:46.220167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-79a1cf7c-17b5-406d-bfe4-a62bf871167e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:46.229367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be49bb39-00cd-4bf9-bceb-b711433f6289 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:46.246903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:46.273651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:46.281057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ede596-9418-4589-a982-46ff81c4dd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:46.281704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ede596-9418-4589-a982-46ff81c4dd80 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:46.281981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ede596-9418-4589-a982-46ff81c4dd80 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:46.282248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ede596-9418-4589-a982-46ff81c4dd80 None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:46.283339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ede596-9418-4589-a982-46ff81c4dd80 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:46.283339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ede596-9418-4589-a982-46ff81c4dd80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:46.284864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ede596-9418-4589-a982-46ff81c4dd80 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:46.308737 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 407/846] 10.222.0.17 () {56 vars in 1135 bytes} [Sat Oct 1 20:33:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:46.309798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ede596-9418-4589-a982-46ff81c4dd80 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:46.335112 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 440/847] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:33:46.423042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ede596-9418-4589-a982-46ff81c4dd80 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:46.459354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230f231-7bb1-4943-914c-40a35470f81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:46.460299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230f231-7bb1-4943-914c-40a35470f81c None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:46.460792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230f231-7bb1-4943-914c-40a35470f81c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:46.461256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230f231-7bb1-4943-914c-40a35470f81c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:46.519224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:46.532201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1230f231-7bb1-4943-914c-40a35470f81c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:46.533651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1230f231-7bb1-4943-914c-40a35470f81c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:46.638708 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 408/848] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:33:46 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 01 20:33:46.644704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1cf7c-17b5-406d-bfe4-a62bf871167e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:46.659985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230f231-7bb1-4943-914c-40a35470f81c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:46.750907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:46.754017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1cf7c-17b5-406d-bfe4-a62bf871167e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:46.765296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230f231-7bb1-4943-914c-40a35470f81c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:46.790988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7b8f8-13fb-4806-b7cc-6aa271c60d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:46.791634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7b8f8-13fb-4806-b7cc-6aa271c60d0d None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:46.791911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7b8f8-13fb-4806-b7cc-6aa271c60d0d None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:46.792349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7b8f8-13fb-4806-b7cc-6aa271c60d0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:46.807476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d91bd9-1798-49a0-90cb-a28687ab9703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:46.808492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d91bd9-1798-49a0-90cb-a28687ab9703 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:46.808990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d91bd9-1798-49a0-90cb-a28687ab9703 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:46.809479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d91bd9-1798-49a0-90cb-a28687ab9703 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:46.809705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7b8f8-13fb-4806-b7cc-6aa271c60d0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:46.810518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d91bd9-1798-49a0-90cb-a28687ab9703 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:46.811151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7b8f8-13fb-4806-b7cc-6aa271c60d0d None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:46.812352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d91bd9-1798-49a0-90cb-a28687ab9703 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:46.821612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:46.823163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-87d91bd9-1798-49a0-90cb-a28687ab9703 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 01 20:33:46.823163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:46.823163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:33:46.823163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:33:46.823163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:33:46.823163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:33:46.823163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:33:46.823163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:46.823163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 01 20:33:46.823163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:46.823163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:33:46.823163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:46.823163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:46.823163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:46.823163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:46.823163 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:46.825194 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:46.825194 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:46.825194 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:46.825194 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:46.825194 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:46.825194 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:46.825194 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:46.825194 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:33:46.825194 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:33:46.825194 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:33:46.825194 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:33:46.825194 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:46.825194 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:46.825194 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:46.825194 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:46.825194 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:46.825194 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:46.826109 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:46.826109 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:46.826109 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:46.826109 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:46.826109 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:46.826109 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:46.826109 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:46.826109 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:46.826109 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:46.826109 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:46.826109 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:33:46.826109 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:46.826109 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 01 20:33:46.826109 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:46.832910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-5ba7b8f8-13fb-4806-b7cc-6aa271c60d0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:46.832910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:46.832910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:46.832910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:46.832910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:46.832910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:46.832910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:46.832910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:46.832910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:46.832910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:46.832910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:46.832910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:46.832910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:46.832910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:46.832910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:46.832910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:46.833845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:46.833845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:46.833845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:46.833845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:46.833845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:46.833845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:46.833845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:46.833845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:46.833845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:46.833845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:46.833845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:46.833845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:46.833845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:46.833845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:46.833845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:46.833845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:46.833845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:46.834767 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:46.834767 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:46.834767 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:46.836237 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 409/850] 10.222.0.17 () {56 vars in 1132 bytes} [Sat Oct 1 20:33:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:46.836884 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 441/850] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:46.837709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7b8f8-13fb-4806-b7cc-6aa271c60d0d None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:46.838012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d91bd9-1798-49a0-90cb-a28687ab9703 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:46.942223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d91bd9-1798-49a0-90cb-a28687ab9703 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:46.944401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7b8f8-13fb-4806-b7cc-6aa271c60d0d None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:46.978456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c92b3e-1920-4121-8b62-637c3f736e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:46.979144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c92b3e-1920-4121-8b62-637c3f736e5b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:46.979435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c92b3e-1920-4121-8b62-637c3f736e5b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:46.979744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c92b3e-1920-4121-8b62-637c3f736e5b None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:46.980477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c92b3e-1920-4121-8b62-637c3f736e5b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:46.980788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c92b3e-1920-4121-8b62-637c3f736e5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:46.982161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c92b3e-1920-4121-8b62-637c3f736e5b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:46.987077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03ed4c-83e8-46d5-b0d3-f3e6a81e86bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:46.987682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03ed4c-83e8-46d5-b0d3-f3e6a81e86bb None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:46.987960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03ed4c-83e8-46d5-b0d3-f3e6a81e86bb None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:46.988230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03ed4c-83e8-46d5-b0d3-f3e6a81e86bb None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:46.988875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c03ed4c-83e8-46d5-b0d3-f3e6a81e86bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:46.989173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c03ed4c-83e8-46d5-b0d3-f3e6a81e86bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:46.990544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c03ed4c-83e8-46d5-b0d3-f3e6a81e86bb None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:46.991811 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 410/851] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Oct 1 20:33:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:46.993041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c92b3e-1920-4121-8b62-637c3f736e5b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:47.008815 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 442/852] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:47.009614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03ed4c-83e8-46d5-b0d3-f3e6a81e86bb None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:47.020731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:47.078352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:47.102443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c92b3e-1920-4121-8b62-637c3f736e5b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:47.121169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03ed4c-83e8-46d5-b0d3-f3e6a81e86bb None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:47.140397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc271df-7b04-44ab-8eda-8a93d06b0994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:47.141903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc271df-7b04-44ab-8eda-8a93d06b0994 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:47.142416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc271df-7b04-44ab-8eda-8a93d06b0994 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:47.142981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc271df-7b04-44ab-8eda-8a93d06b0994 None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:47.143830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc271df-7b04-44ab-8eda-8a93d06b0994 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:47.144358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc271df-7b04-44ab-8eda-8a93d06b0994 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:47.146117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc271df-7b04-44ab-8eda-8a93d06b0994 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:47.156976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b53fbd-15cf-4502-b5d9-95657db0b25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:47.157442 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 411/853] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Oct 1 20:33:46 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:47.157617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b53fbd-15cf-4502-b5d9-95657db0b25b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:47.157901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b53fbd-15cf-4502-b5d9-95657db0b25b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:47.158175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b53fbd-15cf-4502-b5d9-95657db0b25b None admin] PATH_INFO: `/v3/projects/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:47.166116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b53fbd-15cf-4502-b5d9-95657db0b25b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:47.167612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b53fbd-15cf-4502-b5d9-95657db0b25b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:47.180642 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-62b53fbd-15cf-4502-b5d9-95657db0b25b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:47.180642 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:47.180642 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:47.180642 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:47.180642 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:47.180642 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:47.180642 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:47.180642 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:47.180642 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:47.180642 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:47.180642 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:47.180642 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:47.180642 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:47.180642 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:47.180642 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:47.181628 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:47.181628 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:47.181628 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:47.181628 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:47.181628 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:47.181628 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:47.181628 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:47.181628 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:47.181628 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:47.181628 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:47.181628 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:47.181628 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:47.181628 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:47.181628 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:47.181628 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:47.181628 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:47.181628 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:47.182969 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:47.182969 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:47.182969 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:47.182969 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:47.182969 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:47.182969 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:47.182969 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:47.182969 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:47.182969 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 443/854] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:47.188753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc271df-7b04-44ab-8eda-8a93d06b0994 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:47.240771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a770b-ab7b-4e5b-8ec1-e593959cc311 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:47.241102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a770b-ab7b-4e5b-8ec1-e593959cc311 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:47.241328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a770b-ab7b-4e5b-8ec1-e593959cc311 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:47.243049 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 444/855] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:47.250245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb48b4d-64cb-41b2-afe7-f53dfe941622 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:47.250619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb48b4d-64cb-41b2-afe7-f53dfe941622 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:47.250843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb48b4d-64cb-41b2-afe7-f53dfe941622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:47.271648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fcb48b4d-64cb-41b2-afe7-f53dfe941622 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:47.272779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fcb48b4d-64cb-41b2-afe7-f53dfe941622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:47.296153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:47.324582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc271df-7b04-44ab-8eda-8a93d06b0994 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:47.354939 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 445/856] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 01 20:33:47.374474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed4851-b342-4a81-82ce-739dc1d5d32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:47.376064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed4851-b342-4a81-82ce-739dc1d5d32d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:47.376542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed4851-b342-4a81-82ce-739dc1d5d32d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:47.377035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed4851-b342-4a81-82ce-739dc1d5d32d None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:47.377932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed4851-b342-4a81-82ce-739dc1d5d32d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:47.378464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed4851-b342-4a81-82ce-739dc1d5d32d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:47.380209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed4851-b342-4a81-82ce-739dc1d5d32d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:47.398533 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 412/857] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:47.404894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb48b4d-64cb-41b2-afe7-f53dfe941622 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:47.458869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:47.510697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb48b4d-64cb-41b2-afe7-f53dfe941622 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:47.546065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee79bef-c0e9-40c1-b590-b7f58313a37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:47.546623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee79bef-c0e9-40c1-b590-b7f58313a37e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:47.547467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee79bef-c0e9-40c1-b590-b7f58313a37e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:47.547745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee79bef-c0e9-40c1-b590-b7f58313a37e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:47.548479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee79bef-c0e9-40c1-b590-b7f58313a37e None admin] RBAC: Adding query filter params (role.id=1c50896511054a5f994bee3d99fa0619, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=c4dc51e2b4474d6ba9fd5d2bc3702826) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:47.548788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee79bef-c0e9-40c1-b590-b7f58313a37e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:47.550289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee79bef-c0e9-40c1-b590-b7f58313a37e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:47.560600 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 446/858] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:33:47 2022] GET /identity/v3/role_assignments?user.id=c4dc51e2b4474d6ba9fd5d2bc3702826&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=1c50896511054a5f994bee3d99fa0619 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:47.697175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed4851-b342-4a81-82ce-739dc1d5d32d None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:47.807979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed4851-b342-4a81-82ce-739dc1d5d32d None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:47.848594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc459980-fa1a-419d-88e9-5dd9d50590c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:47.849232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc459980-fa1a-419d-88e9-5dd9d50590c3 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:47.849593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc459980-fa1a-419d-88e9-5dd9d50590c3 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:47.849874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc459980-fa1a-419d-88e9-5dd9d50590c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:47.858079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:47.861537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc459980-fa1a-419d-88e9-5dd9d50590c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:47.867439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-dc459980-fa1a-419d-88e9-5dd9d50590c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:47.867439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:47.867439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:47.867439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:47.867439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:47.867439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:47.867439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:47.867439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:47.867439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:47.867439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:47.867439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:47.867439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:47.867439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:47.867439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:47.867439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:47.867439 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:47.868559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:47.868559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:47.868559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:47.868559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:47.868559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:47.868559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:47.868559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:47.868559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:47.868559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:47.868559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:47.868559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:47.868559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:47.868559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:47.868559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:47.868559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:47.868559 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:47.869580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:47.869580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:47.869580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:47.869580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:47.869580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:47.869580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:47.869580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:47.869580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:47.869580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:47.869580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:47.869580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:33:47.869580 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:47.870313 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 413/859] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:33:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:47.876222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee79bef-c0e9-40c1-b590-b7f58313a37e None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:47.979703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee79bef-c0e9-40c1-b590-b7f58313a37e None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:47.994762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:48.015937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ed929-dcba-4e3c-8991-346f816d4f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:48.016545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ed929-dcba-4e3c-8991-346f816d4f40 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:48.016829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ed929-dcba-4e3c-8991-346f816d4f40 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:48.017104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ed929-dcba-4e3c-8991-346f816d4f40 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:48.017772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ed929-dcba-4e3c-8991-346f816d4f40 None admin] RBAC: Adding query filter params (name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:48.018076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ed929-dcba-4e3c-8991-346f816d4f40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:48.019464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ed929-dcba-4e3c-8991-346f816d4f40 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:48.028521 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 447/860] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:33:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:48.036416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc459980-fa1a-419d-88e9-5dd9d50590c3 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:48.150566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc459980-fa1a-419d-88e9-5dd9d50590c3 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:48.186172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045fb6e-d507-4ff2-abc9-4fd9e0cb345b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:48.186830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045fb6e-d507-4ff2-abc9-4fd9e0cb345b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:48.187109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045fb6e-d507-4ff2-abc9-4fd9e0cb345b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:48.187405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045fb6e-d507-4ff2-abc9-4fd9e0cb345b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:48.198105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045fb6e-d507-4ff2-abc9-4fd9e0cb345b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:48.199600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045fb6e-d507-4ff2-abc9-4fd9e0cb345b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:48.215863 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-4045fb6e-d507-4ff2-abc9-4fd9e0cb345b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:48.215863 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:48.215863 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:48.215863 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:48.215863 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:48.215863 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:48.215863 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:48.215863 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:48.215863 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:48.215863 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:48.215863 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:48.215863 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:48.215863 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:48.215863 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:48.215863 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:48.215863 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:48.217031 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:48.217031 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:48.217031 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:48.217031 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:48.217031 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:48.217031 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:48.217031 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:48.217031 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:48.217031 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:48.217031 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:48.217031 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:48.217031 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:48.217031 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:48.217031 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:48.217031 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:48.217031 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:48.217031 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:48.218123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:48.218123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:48.218123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:48.218123 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 414/861] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:48.224689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ed929-dcba-4e3c-8991-346f816d4f40 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:48.333019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ed929-dcba-4e3c-8991-346f816d4f40 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:48.367790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5544176-5f17-44c2-b985-f894fb2e9d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:48.368461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5544176-5f17-44c2-b985-f894fb2e9d55 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:48.368686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5544176-5f17-44c2-b985-f894fb2e9d55 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:48.368960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5544176-5f17-44c2-b985-f894fb2e9d55 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:48.369631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5544176-5f17-44c2-b985-f894fb2e9d55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:48.369936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5544176-5f17-44c2-b985-f894fb2e9d55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:48.371376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5544176-5f17-44c2-b985-f894fb2e9d55 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:48.385051 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 448/862] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:48.391840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045fb6e-d507-4ff2-abc9-4fd9e0cb345b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:48.452019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:48.505511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045fb6e-d507-4ff2-abc9-4fd9e0cb345b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:48.540833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e499e-5115-4a88-b59d-77eddfcd863e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:48.541695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e499e-5115-4a88-b59d-77eddfcd863e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:48.542140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e499e-5115-4a88-b59d-77eddfcd863e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:48.542582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e499e-5115-4a88-b59d-77eddfcd863e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:48.550656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e499e-5115-4a88-b59d-77eddfcd863e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:48.552581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e499e-5115-4a88-b59d-77eddfcd863e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:48.565119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-ec5e499e-5115-4a88-b59d-77eddfcd863e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 01 20:33:48.565119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:48.565119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:48.565119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:48.565119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:48.565119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:48.565119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:48.565119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:48.565119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:48.565119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:48.565119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:48.565119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:48.565119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:48.565119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:48.565119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:48.565119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:48.566291 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:48.566291 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:48.566291 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:48.566291 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:48.566291 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:48.566291 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:48.566291 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:48.566291 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:48.566291 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:48.566291 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:48.566291 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:48.566291 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:48.566291 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:48.566291 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:48.566291 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:48.566291 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:48.566291 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:48.566291 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:48.567513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:48.567513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:48.567513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:48.567513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:48.567513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:48.567513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:48.567513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:48.567513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:48.567513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 01 20:33:48.567513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:48.569615 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 415/863] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Oct 1 20:33:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:48.579595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5544176-5f17-44c2-b985-f894fb2e9d55 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:48.615832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:48.684816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5544176-5f17-44c2-b985-f894fb2e9d55 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:48.710031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bc6d4-8635-4894-af64-b544728c2fbf None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:48.710892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bc6d4-8635-4894-af64-b544728c2fbf None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:48.711314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bc6d4-8635-4894-af64-b544728c2fbf None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:48.713051 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 416/864] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:48.717760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080ee40-042d-455c-b963-455bdd1ecfd8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:48.718045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080ee40-042d-455c-b963-455bdd1ecfd8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:48.718325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080ee40-042d-455c-b963-455bdd1ecfd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:48.723787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9a9af-de8e-492f-8520-ed33f8c54499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:48.724397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9a9af-de8e-492f-8520-ed33f8c54499 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:48.724679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9a9af-de8e-492f-8520-ed33f8c54499 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:48.725010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9a9af-de8e-492f-8520-ed33f8c54499 None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:48.726473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9a9af-de8e-492f-8520-ed33f8c54499 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:48.726473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9a9af-de8e-492f-8520-ed33f8c54499 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:48.727438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9a9af-de8e-492f-8520-ed33f8c54499 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:48.737527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1080ee40-042d-455c-b963-455bdd1ecfd8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:48.738762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1080ee40-042d-455c-b963-455bdd1ecfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:48.746078 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 449/865] 10.222.0.17 () {56 vars in 1149 bytes} [Sat Oct 1 20:33:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:48.751566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9a9af-de8e-492f-8520-ed33f8c54499 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:48.828932 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 417/866] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 01 20:33:48.863082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:48.870688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9a9af-de8e-492f-8520-ed33f8c54499 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:48.906291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f565c46-7233-4f9e-bdba-20fcef7fbfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:48.906912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f565c46-7233-4f9e-bdba-20fcef7fbfc8 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:48.907189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f565c46-7233-4f9e-bdba-20fcef7fbfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:48.907460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f565c46-7233-4f9e-bdba-20fcef7fbfc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:48.978823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f565c46-7233-4f9e-bdba-20fcef7fbfc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:48.980314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f565c46-7233-4f9e-bdba-20fcef7fbfc8 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:49.089951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:49.098387 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 450/867] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:33:48 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 01 20:33:49.104347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080ee40-042d-455c-b963-455bdd1ecfd8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:49.150431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:49.154039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f565c46-7233-4f9e-bdba-20fcef7fbfc8 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:49.225560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080ee40-042d-455c-b963-455bdd1ecfd8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:49.263725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f565c46-7233-4f9e-bdba-20fcef7fbfc8 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:49.269202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574269b-074f-4d7c-97de-83fd2838a04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:49.269837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574269b-074f-4d7c-97de-83fd2838a04e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:49.270116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574269b-074f-4d7c-97de-83fd2838a04e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:49.270388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574269b-074f-4d7c-97de-83fd2838a04e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:49.281484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0574269b-074f-4d7c-97de-83fd2838a04e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:49.283081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0574269b-074f-4d7c-97de-83fd2838a04e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:49.299359 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-0574269b-074f-4d7c-97de-83fd2838a04e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:49.299359 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:49.299359 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:49.299359 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:49.299359 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:49.299359 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:49.299359 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:49.299359 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:49.299359 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:49.299359 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:49.299359 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:49.299359 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:49.299359 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:49.299359 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:49.299359 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:49.299359 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:49.300387 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:49.300387 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:49.300387 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:49.300387 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:49.300387 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:49.300387 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:49.300387 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:49.300387 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:49.300387 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:49.300387 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:49.300387 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:49.300387 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:49.300387 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:49.300387 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:49.300387 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:49.300387 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:49.300387 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:49.301283 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:49.301283 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:49.301283 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:49.301283 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 418/868] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:49.306593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574269b-074f-4d7c-97de-83fd2838a04e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:49.312193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08291b98-60b8-4895-ac53-413a7b056df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:49.313367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08291b98-60b8-4895-ac53-413a7b056df3 None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:49.313648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08291b98-60b8-4895-ac53-413a7b056df3 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:49.313922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08291b98-60b8-4895-ac53-413a7b056df3 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:49.314632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08291b98-60b8-4895-ac53-413a7b056df3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:49.316051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08291b98-60b8-4895-ac53-413a7b056df3 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:49.336627 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 451/869] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Oct 1 20:33:49 2022] POST /identity/v3/services => generated 251 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:33:49.423511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:49.431322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574269b-074f-4d7c-97de-83fd2838a04e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:49.477412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edfae91-bc9c-40b9-b4fe-37db06c3c8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:49.478029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfae91-bc9c-40b9-b4fe-37db06c3c8ba None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:49.478309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfae91-bc9c-40b9-b4fe-37db06c3c8ba None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:49.478599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfae91-bc9c-40b9-b4fe-37db06c3c8ba None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:49.479252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edfae91-bc9c-40b9-b4fe-37db06c3c8ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:49.479555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edfae91-bc9c-40b9-b4fe-37db06c3c8ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:49.480961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edfae91-bc9c-40b9-b4fe-37db06c3c8ba None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:49.494940 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 419/870] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:49.500611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08291b98-60b8-4895-ac53-413a7b056df3 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:49.560524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:49.609961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08291b98-60b8-4895-ac53-413a7b056df3 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:49.645788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad2bca8-c6e9-4d53-8693-548987ed5bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:49.646425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2bca8-c6e9-4d53-8693-548987ed5bf7 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:49.646710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2bca8-c6e9-4d53-8693-548987ed5bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:49.646984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2bca8-c6e9-4d53-8693-548987ed5bf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:49.653949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad2bca8-c6e9-4d53-8693-548987ed5bf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:49.655415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad2bca8-c6e9-4d53-8693-548987ed5bf7 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:49.666966 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-dad2bca8-c6e9-4d53-8693-548987ed5bf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:49.666966 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:49.666966 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:49.666966 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:49.666966 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:49.666966 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:49.666966 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:49.666966 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:49.666966 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:49.666966 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:49.666966 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:49.666966 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:49.666966 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:49.666966 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:49.666966 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:49.667874 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:49.667874 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:49.667874 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:49.667874 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:49.667874 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:49.667874 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:49.667874 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:49.667874 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:49.667874 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:49.667874 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:49.667874 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:49.667874 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:49.667874 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:49.667874 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:49.667874 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:49.667874 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:49.667874 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:49.668768 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:49.668768 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:49.668768 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:49.668768 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:49.668768 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:49.668768 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:49.668768 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:49.668768 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:49.668768 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 452/871] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:49.674759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edfae91-bc9c-40b9-b4fe-37db06c3c8ba None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:49.756965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da4e0a-ae3f-437f-b336-5f95d8b7b2b5 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:49.757330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da4e0a-ae3f-437f-b336-5f95d8b7b2b5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:49.757615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da4e0a-ae3f-437f-b336-5f95d8b7b2b5 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:49.759219 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 453/872] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:33:49.768774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a92600-d164-4b9f-a889-563998deb669 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:49.769061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a92600-d164-4b9f-a889-563998deb669 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:49.769320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a92600-d164-4b9f-a889-563998deb669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:49.793209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edfae91-bc9c-40b9-b4fe-37db06c3c8ba None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:49.793868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-40a92600-d164-4b9f-a889-563998deb669 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:49.795057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-40a92600-d164-4b9f-a889-563998deb669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:49.833706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5fa49-9d96-44c3-9606-66ace3a64c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:49.834728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5fa49-9d96-44c3-9606-66ace3a64c82 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:49.835083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5fa49-9d96-44c3-9606-66ace3a64c82 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:49.835432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5fa49-9d96-44c3-9606-66ace3a64c82 None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:49.836216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5fa49-9d96-44c3-9606-66ace3a64c82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:49.836595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5fa49-9d96-44c3-9606-66ace3a64c82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:49.838154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5fa49-9d96-44c3-9606-66ace3a64c82 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:49.855881 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 420/873] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:49.857237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5fa49-9d96-44c3-9606-66ace3a64c82 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:49.871530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:49.894124 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 454/874] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:49 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:33:49.975640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5fa49-9d96-44c3-9606-66ace3a64c82 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:50.011698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31596359-57e7-4652-89a2-fd41aa6b402e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:50.012307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31596359-57e7-4652-89a2-fd41aa6b402e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:50.012589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31596359-57e7-4652-89a2-fd41aa6b402e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:50.012914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31596359-57e7-4652-89a2-fd41aa6b402e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:50.013644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31596359-57e7-4652-89a2-fd41aa6b402e None admin] RBAC: Adding query filter params (role.id=ef7bd7293369475a9e7b3444fac30cda, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=917fc96fa3504b19b1a56990d4747439) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:50.013994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31596359-57e7-4652-89a2-fd41aa6b402e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:50.015657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31596359-57e7-4652-89a2-fd41aa6b402e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:50.024136 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 421/875] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:33:49 2022] GET /identity/v3/role_assignments?user.id=917fc96fa3504b19b1a56990d4747439&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=ef7bd7293369475a9e7b3444fac30cda => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:50.233946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a92600-d164-4b9f-a889-563998deb669 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:50.246310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b129ae4-675c-497e-b65d-31680cd62d73 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:50.247896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b129ae4-675c-497e-b65d-31680cd62d73 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:50.248178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b129ae4-675c-497e-b65d-31680cd62d73 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:50.249641 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 422/876] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:50.256458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba9552-bcae-4f69-a501-64ddfdf071ae None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:50.257236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba9552-bcae-4f69-a501-64ddfdf071ae None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:50.258106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba9552-bcae-4f69-a501-64ddfdf071ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:50.279757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5eba9552-bcae-4f69-a501-64ddfdf071ae None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:50.280959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5eba9552-bcae-4f69-a501-64ddfdf071ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:50.289210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:50.355308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a92600-d164-4b9f-a889-563998deb669 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:50.356234 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 423/877] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:50 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 01 20:33:50.393297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d9873-9837-4a22-9c6c-c2f2bf176e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:50.393906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d9873-9837-4a22-9c6c-c2f2bf176e0b None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:50.394190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d9873-9837-4a22-9c6c-c2f2bf176e0b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:50.394465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d9873-9837-4a22-9c6c-c2f2bf176e0b None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:50.395158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d9873-9837-4a22-9c6c-c2f2bf176e0b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:50.396515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d9873-9837-4a22-9c6c-c2f2bf176e0b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:50.413272 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 455/878] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Oct 1 20:33:50 2022] POST /identity/v3/services => generated 269 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:33:50.683574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba9552-bcae-4f69-a501-64ddfdf071ae None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:50.687024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:50.793639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba9552-bcae-4f69-a501-64ddfdf071ae None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:50.836091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6a504-332e-41c5-ac43-a6919a229802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:50.836723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6a504-332e-41c5-ac43-a6919a229802 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:50.837002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6a504-332e-41c5-ac43-a6919a229802 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:50.837276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6a504-332e-41c5-ac43-a6919a229802 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:50.837857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6a504-332e-41c5-ac43-a6919a229802 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:50.839256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6a504-332e-41c5-ac43-a6919a229802 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:50.849386 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-30d6a504-332e-41c5-ac43-a6919a229802 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 01 20:33:50.849386 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:50.849386 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:33:50.849386 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:33:50.849386 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:33:50.849386 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:33:50.849386 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:33:50.849386 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:50.849386 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 01 20:33:50.849386 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:50.849386 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:33:50.849386 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:50.849386 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:50.849386 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:50.849386 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:50.849386 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:50.849386 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:50.850403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:50.850403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:50.850403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:50.850403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:50.850403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:50.850403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:50.850403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:33:50.850403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:33:50.850403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:33:50.850403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:33:50.850403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:50.850403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:50.850403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:50.850403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:50.850403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:50.850403 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:50.851275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:50.851275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:50.851275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:50.851275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:50.851275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:50.851275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:50.851275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:50.851275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:50.851275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:50.851275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:50.851275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:33:50.851275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:50.851275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 01 20:33:50.851275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:50.851275 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 424/879] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Oct 1 20:33:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:50.856367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d9873-9837-4a22-9c6c-c2f2bf176e0b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:50.960182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d9873-9837-4a22-9c6c-c2f2bf176e0b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:50.962665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:50.997253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e791df-1540-4bec-a3ef-916a2bf43a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:50.997859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e791df-1540-4bec-a3ef-916a2bf43a82 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:50.998135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e791df-1540-4bec-a3ef-916a2bf43a82 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:50.998403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e791df-1540-4bec-a3ef-916a2bf43a82 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:50.999061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e791df-1540-4bec-a3ef-916a2bf43a82 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:50.999356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e791df-1540-4bec-a3ef-916a2bf43a82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:51.000848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e791df-1540-4bec-a3ef-916a2bf43a82 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:51.006094 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 456/880] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Oct 1 20:33:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:51.011990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6a504-332e-41c5-ac43-a6919a229802 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:51.084496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:51.122794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6a504-332e-41c5-ac43-a6919a229802 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:51.156906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab1897-c8c4-487a-97f0-6bfc2248a486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:51.157501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab1897-c8c4-487a-97f0-6bfc2248a486 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:51.157776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab1897-c8c4-487a-97f0-6bfc2248a486 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:51.158041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab1897-c8c4-487a-97f0-6bfc2248a486 None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:51.158689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab1897-c8c4-487a-97f0-6bfc2248a486 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:51.158987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab1897-c8c4-487a-97f0-6bfc2248a486 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:51.160340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab1897-c8c4-487a-97f0-6bfc2248a486 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:51.165209 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 425/881] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Oct 1 20:33:51 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:51.981513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80827db9-9d2d-46ab-af03-3e52bf338b01 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:51.982559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80827db9-9d2d-46ab-af03-3e52bf338b01 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:51.983048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80827db9-9d2d-46ab-af03-3e52bf338b01 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:51.985347 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 457/882] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:51.990650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954199cb-e8c2-4884-be32-b7f0a9bff756 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:51.991438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954199cb-e8c2-4884-be32-b7f0a9bff756 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:51.991920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954199cb-e8c2-4884-be32-b7f0a9bff756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:52.013082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-954199cb-e8c2-4884-be32-b7f0a9bff756 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:52.014249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-954199cb-e8c2-4884-be32-b7f0a9bff756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:52.084132 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 426/883] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:51 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:33:52.416798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80827db9-9d2d-46ab-af03-3e52bf338b01 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:52.531655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80827db9-9d2d-46ab-af03-3e52bf338b01 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:52.567125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05705b-0ccc-47a9-8f30-c66db2b1ce86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:52.568047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05705b-0ccc-47a9-8f30-c66db2b1ce86 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:52.568527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05705b-0ccc-47a9-8f30-c66db2b1ce86 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:52.568967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05705b-0ccc-47a9-8f30-c66db2b1ce86 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:52.569723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05705b-0ccc-47a9-8f30-c66db2b1ce86 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:52.571553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05705b-0ccc-47a9-8f30-c66db2b1ce86 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:52.581070 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-fd05705b-0ccc-47a9-8f30-c66db2b1ce86 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 01 20:33:52.581070 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:52.581070 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:33:52.581070 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:33:52.581070 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:33:52.581070 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:33:52.581070 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:33:52.581070 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:52.581070 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 01 20:33:52.581070 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:52.581070 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:33:52.581070 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:52.581070 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:52.581070 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:52.581070 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:52.581070 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:52.581070 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:52.582046 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:52.582046 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:52.582046 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:52.582046 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:52.582046 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:52.582046 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:52.582046 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:33:52.582046 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:33:52.582046 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:33:52.582046 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:33:52.582046 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:52.582046 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:52.582046 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:52.582046 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:52.582046 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:52.582046 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:52.582926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:52.582926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:52.582926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:52.582926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:52.582926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:52.582926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:52.582926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:52.582926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:52.582926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:52.582926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:52.582926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:33:52.582926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:52.582926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 01 20:33:52.582926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:52.585259 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 458/884] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:33:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:52.591139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954199cb-e8c2-4884-be32-b7f0a9bff756 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:52.654689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:52.703540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b664eb-ce41-43f4-95fd-2a97d33d7601 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:52.704157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b664eb-ce41-43f4-95fd-2a97d33d7601 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:52.704644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b664eb-ce41-43f4-95fd-2a97d33d7601 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:52.706528 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 459/885] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:33:52.707082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954199cb-e8c2-4884-be32-b7f0a9bff756 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:52.711092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103b72a-d9e2-4e31-9ab1-32a71aa311d4 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:52.711608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103b72a-d9e2-4e31-9ab1-32a71aa311d4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:52.712071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103b72a-d9e2-4e31-9ab1-32a71aa311d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:52.731899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f103b72a-d9e2-4e31-9ab1-32a71aa311d4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:52.733401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f103b72a-d9e2-4e31-9ab1-32a71aa311d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:52.742614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95132cd0-28b3-4734-bbd1-f103dbefbb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:52.743224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95132cd0-28b3-4734-bbd1-f103dbefbb27 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:52.743502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95132cd0-28b3-4734-bbd1-f103dbefbb27 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:52.743772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95132cd0-28b3-4734-bbd1-f103dbefbb27 None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:52.744403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95132cd0-28b3-4734-bbd1-f103dbefbb27 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:52.744702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95132cd0-28b3-4734-bbd1-f103dbefbb27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:52.746055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95132cd0-28b3-4734-bbd1-f103dbefbb27 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:52.751139 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 427/886] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:33:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:52.759411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95132cd0-28b3-4734-bbd1-f103dbefbb27 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:52.764981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:52.821302 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 460/887] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:52 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:33:52.879191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95132cd0-28b3-4734-bbd1-f103dbefbb27 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:52.890596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a6f24-e90c-4fcc-ad74-8d36d8345c4d None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:52.890945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a6f24-e90c-4fcc-ad74-8d36d8345c4d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:52.891227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a6f24-e90c-4fcc-ad74-8d36d8345c4d None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:52.893021 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 461/888] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:33:52.897523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc568eae-1b4b-4595-84ff-4f8d60551e3c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:52.897822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc568eae-1b4b-4595-84ff-4f8d60551e3c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:52.898103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc568eae-1b4b-4595-84ff-4f8d60551e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:52.904787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:52.917287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-bc568eae-1b4b-4595-84ff-4f8d60551e3c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:52.918468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bc568eae-1b4b-4595-84ff-4f8d60551e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:52.920641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b61acb-19b5-4680-beeb-66857f465e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:52.922303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b61acb-19b5-4680-beeb-66857f465e1b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:52.922639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b61acb-19b5-4680-beeb-66857f465e1b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:52.922915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b61acb-19b5-4680-beeb-66857f465e1b None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:52.923541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b61acb-19b5-4680-beeb-66857f465e1b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:52.923841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b61acb-19b5-4680-beeb-66857f465e1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:52.925193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b61acb-19b5-4680-beeb-66857f465e1b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:52.932037 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 428/889] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:33:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:52.938365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b61acb-19b5-4680-beeb-66857f465e1b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:52.995806 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 462/890] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:52 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 01 20:33:53.067679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b61acb-19b5-4680-beeb-66857f465e1b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:53.103400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fb794-79d1-4db4-8a70-0d8bbdae005b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:53.104006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fb794-79d1-4db4-8a70-0d8bbdae005b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:53.104284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fb794-79d1-4db4-8a70-0d8bbdae005b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:53.104556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fb794-79d1-4db4-8a70-0d8bbdae005b None admin] PATH_INFO: `/v3/endpoints` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:53.105307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fb794-79d1-4db4-8a70-0d8bbdae005b None admin] RBAC: Adding query filter params (interface=public, service_id=f4cd6ddb12914304a52cc1c21b022963, region_id=RegionOne) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:53.105611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fb794-79d1-4db4-8a70-0d8bbdae005b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:53.107217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fb794-79d1-4db4-8a70-0d8bbdae005b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:53.112771 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 429/891] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Oct 1 20:33:52 2022] GET /identity/v3/endpoints?service_id=f4cd6ddb12914304a52cc1c21b022963&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:53.119860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc568eae-1b4b-4595-84ff-4f8d60551e3c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:53.190431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fb794-79d1-4db4-8a70-0d8bbdae005b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:53.224787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:53.235080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc568eae-1b4b-4595-84ff-4f8d60551e3c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:53.260828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:53.277336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de555a-0503-48cc-8001-cb55c219d0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:53.278418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de555a-0503-48cc-8001-cb55c219d0bd None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:53.278952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de555a-0503-48cc-8001-cb55c219d0bd None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:53.279420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de555a-0503-48cc-8001-cb55c219d0bd None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:53.280358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de555a-0503-48cc-8001-cb55c219d0bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:53.282208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de555a-0503-48cc-8001-cb55c219d0bd None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:53.294742 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 463/892] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Oct 1 20:33:53 2022] GET /identity/v3/services => generated 1268 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:33:53.326826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fb794-79d1-4db4-8a70-0d8bbdae005b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:53.330887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de555a-0503-48cc-8001-cb55c219d0bd None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:53.377099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69db718-1f8d-4bad-9bff-0b825906069e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:53.377757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69db718-1f8d-4bad-9bff-0b825906069e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:53.378094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69db718-1f8d-4bad-9bff-0b825906069e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:53.378363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69db718-1f8d-4bad-9bff-0b825906069e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:53.398246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69db718-1f8d-4bad-9bff-0b825906069e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:53.403212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-d69db718-1f8d-4bad-9bff-0b825906069e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 01 20:33:53.403212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:53.403212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:53.403212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:53.403212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:53.403212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:53.403212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:53.403212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:53.403212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:53.403212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:53.403212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:53.403212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:53.403212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:53.403212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:53.403212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:53.403212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:53.404212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:53.404212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:53.404212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:53.404212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:53.404212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:53.404212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:53.404212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:53.404212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:53.404212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:53.404212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:53.404212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:53.404212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:53.404212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:53.404212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:53.404212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:53.404212 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:53.405054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:53.405054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:53.405054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:53.405054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:53.405054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:53.405054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:53.405054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:53.405054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:53.405054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:53.405054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:53.405054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 01 20:33:53.405054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:53.405054 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 430/893] 10.222.0.17 () {56 vars in 1120 bytes} [Sat Oct 1 20:33:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:53.411222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69db718-1f8d-4bad-9bff-0b825906069e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:53.450507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de555a-0503-48cc-8001-cb55c219d0bd None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:53.489519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f5706-c7d3-410a-bfe0-753ad4632f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:53.490327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f5706-c7d3-410a-bfe0-753ad4632f94 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:53.490752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f5706-c7d3-410a-bfe0-753ad4632f94 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:53.491122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f5706-c7d3-410a-bfe0-753ad4632f94 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:53.491814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f5706-c7d3-410a-bfe0-753ad4632f94 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:53.493301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f5706-c7d3-410a-bfe0-753ad4632f94 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:53.503013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-9b6f5706-c7d3-410a-bfe0-753ad4632f94 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 01 20:33:53.503013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:53.503013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:33:53.503013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:33:53.503013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:33:53.503013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:33:53.503013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:33:53.503013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:53.503013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 01 20:33:53.503013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:53.503013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:33:53.503013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:53.503013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:53.503013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:53.503013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:53.503013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:53.504198 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:53.504198 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:53.504198 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:53.504198 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:53.504198 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:53.504198 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:53.504198 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:53.504198 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:33:53.504198 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:33:53.504198 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:33:53.504198 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:33:53.504198 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:53.504198 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:53.504198 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:53.504198 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:53.504198 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:53.504198 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:53.505311 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:53.505311 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:53.505311 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:53.505311 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:53.505311 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:53.505311 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:53.505311 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:53.505311 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:53.505311 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:53.505311 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:53.505311 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:33:53.505311 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:53.505311 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 01 20:33:53.505311 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:53.511114 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 464/894] 10.222.0.17 () {56 vars in 1132 bytes} [Sat Oct 1 20:33:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:53.512051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f5706-c7d3-410a-bfe0-753ad4632f94 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:53.536916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69db718-1f8d-4bad-9bff-0b825906069e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:53.541598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:53.549673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:53.583212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8baa2c1-ea37-4b6b-9e3f-4a240165d335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:53.584269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baa2c1-ea37-4b6b-9e3f-4a240165d335 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:53.584550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baa2c1-ea37-4b6b-9e3f-4a240165d335 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:53.584965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baa2c1-ea37-4b6b-9e3f-4a240165d335 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:53.585764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8baa2c1-ea37-4b6b-9e3f-4a240165d335 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:53.586074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8baa2c1-ea37-4b6b-9e3f-4a240165d335 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:53.588206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8baa2c1-ea37-4b6b-9e3f-4a240165d335 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:53.599775 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 431/895] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Oct 1 20:33:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:53.605286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8baa2c1-ea37-4b6b-9e3f-4a240165d335 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:53.634966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f5706-c7d3-410a-bfe0-753ad4632f94 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:53.670323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e32b74-9568-4ad4-ba34-148373cf9d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:53.671050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e32b74-9568-4ad4-ba34-148373cf9d83 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:53.671401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e32b74-9568-4ad4-ba34-148373cf9d83 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:53.671976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e32b74-9568-4ad4-ba34-148373cf9d83 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:53.672708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e32b74-9568-4ad4-ba34-148373cf9d83 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:53.673057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e32b74-9568-4ad4-ba34-148373cf9d83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:53.674511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e32b74-9568-4ad4-ba34-148373cf9d83 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:53.679329 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 465/896] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Oct 1 20:33:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:53.685062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e32b74-9568-4ad4-ba34-148373cf9d83 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:53.718276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8baa2c1-ea37-4b6b-9e3f-4a240165d335 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:53.758390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af1544-b72a-43b1-af19-92cd076fb99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:53.759053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af1544-b72a-43b1-af19-92cd076fb99d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:53.759331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af1544-b72a-43b1-af19-92cd076fb99d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:53.759599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af1544-b72a-43b1-af19-92cd076fb99d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:53.770889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af1544-b72a-43b1-af19-92cd076fb99d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:53.773502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af1544-b72a-43b1-af19-92cd076fb99d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:53.783364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:53.790384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-e8af1544-b72a-43b1-af19-92cd076fb99d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:53.790384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:53.790384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:53.790384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:53.790384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:53.790384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:53.790384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:53.790384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:53.790384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:53.790384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:53.790384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:53.790384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:53.790384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:53.790384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:53.790384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:53.790384 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:53.791542 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:53.791542 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:53.791542 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:53.791542 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:53.791542 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:53.791542 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:53.791542 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:53.791542 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:53.791542 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:53.791542 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:53.791542 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:53.791542 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:53.791542 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:53.791542 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:53.791542 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:53.791542 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:53.791542 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:53.792447 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:53.792447 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:53.792447 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:53.792447 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 432/897] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:53.793193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9afef-83e8-40bf-be9c-3a83e24a71fd None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:53.793193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9afef-83e8-40bf-be9c-3a83e24a71fd None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:53.793364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9afef-83e8-40bf-be9c-3a83e24a71fd None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:53.794735 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 433/898] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:53.799845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9afef-83e8-40bf-be9c-3a83e24a71fd None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:53.806041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e32b74-9568-4ad4-ba34-148373cf9d83 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:53.828604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:53.846041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0394eb-a1f6-44a2-b98e-1a3d7acfafcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:53.846684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0394eb-a1f6-44a2-b98e-1a3d7acfafcc None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:53.846966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0394eb-a1f6-44a2-b98e-1a3d7acfafcc None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:53.847243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0394eb-a1f6-44a2-b98e-1a3d7acfafcc None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:53.848032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0394eb-a1f6-44a2-b98e-1a3d7acfafcc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:53.848340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0394eb-a1f6-44a2-b98e-1a3d7acfafcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:53.849701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0394eb-a1f6-44a2-b98e-1a3d7acfafcc None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:53.855366 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 466/899] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Oct 1 20:33:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:53.861154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf44810-7878-49e5-b440-3998b311ee8c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:53.861452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf44810-7878-49e5-b440-3998b311ee8c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:53.861708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf44810-7878-49e5-b440-3998b311ee8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:53.880781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1cf44810-7878-49e5-b440-3998b311ee8c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:53.881970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1cf44810-7878-49e5-b440-3998b311ee8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:53.913324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9afef-83e8-40bf-be9c-3a83e24a71fd None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:53.949768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a91e7-8765-4708-acc0-11a1c23a46a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:53.950383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a91e7-8765-4708-acc0-11a1c23a46a1 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:53.950692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a91e7-8765-4708-acc0-11a1c23a46a1 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:53.950970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a91e7-8765-4708-acc0-11a1c23a46a1 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:53.952018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a91e7-8765-4708-acc0-11a1c23a46a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:53.952018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a91e7-8765-4708-acc0-11a1c23a46a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:53.953924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a91e7-8765-4708-acc0-11a1c23a46a1 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:53.966942 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 434/900] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:53.968076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a91e7-8765-4708-acc0-11a1c23a46a1 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:53.976897 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 467/901] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:53 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:33:53.978142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf44810-7878-49e5-b440-3998b311ee8c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:54.085767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:54.088782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:54.091309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a91e7-8765-4708-acc0-11a1c23a46a1 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:54.117660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf44810-7878-49e5-b440-3998b311ee8c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:54.135753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3becc47-b974-48e5-a725-398031c9b402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:54.136866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3becc47-b974-48e5-a725-398031c9b402 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:54.137311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3becc47-b974-48e5-a725-398031c9b402 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:54.137995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3becc47-b974-48e5-a725-398031c9b402 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:54.139260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3becc47-b974-48e5-a725-398031c9b402 None admin] RBAC: Adding query filter params (interface=public, service_id=bc74777c999e4c6e98b85e2c61065965, region_id=RegionOne) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:54.139718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3becc47-b974-48e5-a725-398031c9b402 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:54.141820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3becc47-b974-48e5-a725-398031c9b402 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:54.154607 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 435/902] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Oct 1 20:33:53 2022] GET /identity/v3/endpoints?service_id=bc74777c999e4c6e98b85e2c61065965&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:54.155453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3becc47-b974-48e5-a725-398031c9b402 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:54.164849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec9798-2918-4243-9ce0-eeb22580f90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:54.165487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec9798-2918-4243-9ce0-eeb22580f90c None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:54.165773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec9798-2918-4243-9ce0-eeb22580f90c None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:54.166051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec9798-2918-4243-9ce0-eeb22580f90c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:54.175289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec9798-2918-4243-9ce0-eeb22580f90c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:54.177009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec9798-2918-4243-9ce0-eeb22580f90c None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:54.190065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-22ec9798-2918-4243-9ce0-eeb22580f90c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 01 20:33:54.190065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:54.190065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:54.190065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:54.190065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:54.190065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:54.190065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:54.190065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:54.190065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:54.190065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:54.190065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:54.190065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:54.190065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:54.190065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:54.190065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:54.190065 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:54.191096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:54.191096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:54.191096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:54.191096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:54.191096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:54.191096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:54.191096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:54.191096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:54.191096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:54.191096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:54.191096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:54.191096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:54.191096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:54.191096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:54.191096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:54.191096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:54.191096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:54.191096 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:54.192064 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:54.192064 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:54.192064 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:54.192064 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:54.192064 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:54.192064 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:54.192064 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:54.192064 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:54.192064 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 01 20:33:54.192064 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:54.194558 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 468/903] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Oct 1 20:33:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:54.197651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec9798-2918-4243-9ce0-eeb22580f90c None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:54.295704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3becc47-b974-48e5-a725-398031c9b402 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:54.324159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec9798-2918-4243-9ce0-eeb22580f90c None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:54.340286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57ca6b-7a96-4c82-b28b-60eba4530e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:54.340896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57ca6b-7a96-4c82-b28b-60eba4530e34 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:54.341173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57ca6b-7a96-4c82-b28b-60eba4530e34 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:54.341435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57ca6b-7a96-4c82-b28b-60eba4530e34 None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:54.342040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57ca6b-7a96-4c82-b28b-60eba4530e34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:54.344449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57ca6b-7a96-4c82-b28b-60eba4530e34 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:54.350431 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 436/904] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Oct 1 20:33:54 2022] GET /identity/v3/services => generated 1268 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:33:54.351229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57ca6b-7a96-4c82-b28b-60eba4530e34 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:54.367628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:54.392509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24274fd3-c0d3-40cd-b190-17e6484ca296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:54.393558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24274fd3-c0d3-40cd-b190-17e6484ca296 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:54.394071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24274fd3-c0d3-40cd-b190-17e6484ca296 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:54.394578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24274fd3-c0d3-40cd-b190-17e6484ca296 None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:54.395721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24274fd3-c0d3-40cd-b190-17e6484ca296 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:54.396249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24274fd3-c0d3-40cd-b190-17e6484ca296 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:54.398271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24274fd3-c0d3-40cd-b190-17e6484ca296 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:54.419068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:54.421909 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 469/905] 10.222.0.17 () {56 vars in 1149 bytes} [Sat Oct 1 20:33:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:54.427820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24274fd3-c0d3-40cd-b190-17e6484ca296 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:54.500897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57ca6b-7a96-4c82-b28b-60eba4530e34 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:54.552033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86204383-cffb-4172-befb-911b99918f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:54.552868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86204383-cffb-4172-befb-911b99918f0c None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:54.553290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86204383-cffb-4172-befb-911b99918f0c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:54.553574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86204383-cffb-4172-befb-911b99918f0c None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:54.554268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86204383-cffb-4172-befb-911b99918f0c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:54.556040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86204383-cffb-4172-befb-911b99918f0c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:54.568679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24274fd3-c0d3-40cd-b190-17e6484ca296 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:54.582555 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 437/906] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Oct 1 20:33:54 2022] POST /identity/v3/services => generated 241 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:33:54.627718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c078953-9e07-45c4-83af-82725fba4fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:54.628351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c078953-9e07-45c4-83af-82725fba4fbc None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:54.628635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c078953-9e07-45c4-83af-82725fba4fbc None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:54.628912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c078953-9e07-45c4-83af-82725fba4fbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:54.715409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:54.728648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c078953-9e07-45c4-83af-82725fba4fbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:54.730564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c078953-9e07-45c4-83af-82725fba4fbc None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:54.842731 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 470/907] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:33:54 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 01 20:33:54.849653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86204383-cffb-4172-befb-911b99918f0c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:54.932948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:54.968402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86204383-cffb-4172-befb-911b99918f0c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:55.004309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de106c2c-e49c-4e54-8450-8df0ab1535ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:55.004944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de106c2c-e49c-4e54-8450-8df0ab1535ca None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:55.005223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de106c2c-e49c-4e54-8450-8df0ab1535ca None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:55.005496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de106c2c-e49c-4e54-8450-8df0ab1535ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:55.016830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de106c2c-e49c-4e54-8450-8df0ab1535ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:55.018277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de106c2c-e49c-4e54-8450-8df0ab1535ca None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:55.033879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-de106c2c-e49c-4e54-8450-8df0ab1535ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:55.033879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:55.033879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:55.033879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:55.033879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:55.033879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:55.033879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:55.033879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:55.033879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:55.033879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:55.033879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:55.033879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:55.033879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:55.033879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:55.033879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:55.033879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:55.034901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:55.034901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:55.034901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:55.034901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:55.034901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:55.034901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:55.034901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:55.034901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:55.034901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:55.034901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:55.034901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:55.034901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:55.034901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:55.034901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:55.034901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:55.034901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:55.034901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:55.036378 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:55.036378 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:55.036378 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:55.036378 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 438/908] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:55.045154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c078953-9e07-45c4-83af-82725fba4fbc None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:55.146396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:55.157733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c078953-9e07-45c4-83af-82725fba4fbc None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:55.196631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e899d5-1fce-4b37-a4b0-f07fc3dfa4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:55.197241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e899d5-1fce-4b37-a4b0-f07fc3dfa4dc None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:55.197518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e899d5-1fce-4b37-a4b0-f07fc3dfa4dc None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:55.197790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e899d5-1fce-4b37-a4b0-f07fc3dfa4dc None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:55.198433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e899d5-1fce-4b37-a4b0-f07fc3dfa4dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:55.198769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e899d5-1fce-4b37-a4b0-f07fc3dfa4dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:55.200191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e899d5-1fce-4b37-a4b0-f07fc3dfa4dc None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:55.212637 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 471/909] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:55.217934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de106c2c-e49c-4e54-8450-8df0ab1535ca None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:55.328041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de106c2c-e49c-4e54-8450-8df0ab1535ca None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:55.353823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:55.365815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707fbbb6-12f4-4ae6-8d42-f547a322ff9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:55.366451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707fbbb6-12f4-4ae6-8d42-f547a322ff9a None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:55.366763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707fbbb6-12f4-4ae6-8d42-f547a322ff9a None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:55.367035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707fbbb6-12f4-4ae6-8d42-f547a322ff9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:55.373856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707fbbb6-12f4-4ae6-8d42-f547a322ff9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:55.375877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707fbbb6-12f4-4ae6-8d42-f547a322ff9a None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:55.387149 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-707fbbb6-12f4-4ae6-8d42-f547a322ff9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:55.387149 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:55.387149 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:55.387149 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:55.387149 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:55.387149 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:55.387149 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:55.387149 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:55.387149 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:55.387149 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:55.387149 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:55.387149 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:55.387149 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:33:55.387149 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:33:55.387149 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:33:55.388066 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:33:55.388066 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:55.388066 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:55.388066 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:55.388066 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:55.388066 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:55.388066 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:55.388066 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:55.388066 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:55.388066 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:55.388066 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:55.388066 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:55.388066 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:55.388066 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:55.388066 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:55.388066 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:55.388066 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:55.389065 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:33:55.389065 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:33:55.389065 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:33:55.389065 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:33:55.389065 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:33:55.389065 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:33:55.389065 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:33:55.389065 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:55.393433 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 439/910] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:33:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:55.395590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e899d5-1fce-4b37-a4b0-f07fc3dfa4dc None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:55.509147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e899d5-1fce-4b37-a4b0-f07fc3dfa4dc None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:55.545151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187368ae-18fd-4b61-b267-8e289242d539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:55.545809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187368ae-18fd-4b61-b267-8e289242d539 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:55.546104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187368ae-18fd-4b61-b267-8e289242d539 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:55.546379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187368ae-18fd-4b61-b267-8e289242d539 None admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:55.547167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187368ae-18fd-4b61-b267-8e289242d539 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:55.547478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187368ae-18fd-4b61-b267-8e289242d539 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:55.548907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187368ae-18fd-4b61-b267-8e289242d539 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:55.561644 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 472/911] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:33:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:55.567504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707fbbb6-12f4-4ae6-8d42-f547a322ff9a None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:55.674343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707fbbb6-12f4-4ae6-8d42-f547a322ff9a None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:55.709962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05e7c0-8292-4596-af8a-af807b1625fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:55.710595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05e7c0-8292-4596-af8a-af807b1625fe None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:55.710879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05e7c0-8292-4596-af8a-af807b1625fe None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:55.711154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05e7c0-8292-4596-af8a-af807b1625fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:55.711882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a05e7c0-8292-4596-af8a-af807b1625fe None admin] RBAC: Adding query filter params (role.id=fcfc5f73645d423d8475708ef191dacb, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=917fc96fa3504b19b1a56990d4747439) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:55.712196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a05e7c0-8292-4596-af8a-af807b1625fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:55.713685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a05e7c0-8292-4596-af8a-af807b1625fe None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:55.722626 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 440/912] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:33:55 2022] GET /identity/v3/role_assignments?user.id=917fc96fa3504b19b1a56990d4747439&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=fcfc5f73645d423d8475708ef191dacb => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:55.900874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e7f4e-e5c5-4c7f-be47-316083ff0193 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:55.901553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e7f4e-e5c5-4c7f-be47-316083ff0193 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:55.902062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e7f4e-e5c5-4c7f-be47-316083ff0193 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:55.903999 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 473/913] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:55.910412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adcb46-e847-46af-95c0-7bf527d5d810 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:55.911055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adcb46-e847-46af-95c0-7bf527d5d810 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:55.911530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adcb46-e847-46af-95c0-7bf527d5d810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:55.933889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-94adcb46-e847-46af-95c0-7bf527d5d810 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:55.935314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-94adcb46-e847-46af-95c0-7bf527d5d810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:55.982401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:56.008401 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 441/914] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:33:56.347613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e7f4e-e5c5-4c7f-be47-316083ff0193 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:56.363381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:56.457584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e7f4e-e5c5-4c7f-be47-316083ff0193 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:56.494511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090affb1-d03c-4492-ac59-ed85051406fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:56.495203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090affb1-d03c-4492-ac59-ed85051406fd None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:56.495490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090affb1-d03c-4492-ac59-ed85051406fd None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:56.495761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090affb1-d03c-4492-ac59-ed85051406fd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:56.496413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090affb1-d03c-4492-ac59-ed85051406fd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:56.497817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090affb1-d03c-4492-ac59-ed85051406fd None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:56.507644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-090affb1-d03c-4492-ac59-ed85051406fd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 01 20:33:56.507644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:56.507644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:33:56.507644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:33:56.507644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:33:56.507644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:33:56.507644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:33:56.507644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:56.507644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 01 20:33:56.507644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:56.507644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:33:56.507644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:56.507644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:56.507644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:56.507644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:56.507644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:56.507644 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:56.508680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:56.508680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:56.508680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:56.508680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:56.508680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:56.508680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:56.508680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:33:56.508680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:33:56.508680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:33:56.508680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:33:56.508680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:56.508680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:56.508680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:56.508680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:56.508680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:56.508680 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:56.510817 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:56.510817 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:56.510817 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:56.510817 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:56.510817 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:56.510817 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:56.510817 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:56.510817 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:56.510817 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:56.510817 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:56.510817 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:33:56.510817 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:56.510817 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 01 20:33:56.510817 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:56.510817 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 474/915] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:33:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:56.519965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adcb46-e847-46af-95c0-7bf527d5d810 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:56.629345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adcb46-e847-46af-95c0-7bf527d5d810 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:56.665931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839aee10-36a6-4577-8696-1768c974c89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:56.666636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839aee10-36a6-4577-8696-1768c974c89a None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:56.666918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839aee10-36a6-4577-8696-1768c974c89a None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:56.667190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839aee10-36a6-4577-8696-1768c974c89a None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:56.667818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839aee10-36a6-4577-8696-1768c974c89a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:56.668122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839aee10-36a6-4577-8696-1768c974c89a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:56.669491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839aee10-36a6-4577-8696-1768c974c89a None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:56.674963 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 442/916] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:33:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:56.680749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090affb1-d03c-4492-ac59-ed85051406fd None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:56.762082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:56.786441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090affb1-d03c-4492-ac59-ed85051406fd None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:56.821555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ac780-b0ea-4aac-a815-5a1e99aa322b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:56.822168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ac780-b0ea-4aac-a815-5a1e99aa322b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:56.822523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ac780-b0ea-4aac-a815-5a1e99aa322b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:56.822808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ac780-b0ea-4aac-a815-5a1e99aa322b None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:56.823442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ac780-b0ea-4aac-a815-5a1e99aa322b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:56.823744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ac780-b0ea-4aac-a815-5a1e99aa322b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:56.825102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ac780-b0ea-4aac-a815-5a1e99aa322b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:56.830587 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 475/917] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Oct 1 20:33:56 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:56.836384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839aee10-36a6-4577-8696-1768c974c89a None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:56.888438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:56.948719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839aee10-36a6-4577-8696-1768c974c89a None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:56.961853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88cb83-1deb-4bb8-91a4-1200450a62e2 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:56.962133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88cb83-1deb-4bb8-91a4-1200450a62e2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:56.962404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88cb83-1deb-4bb8-91a4-1200450a62e2 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:56.964022 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 476/918] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:33:56.969246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edfe81-efde-4874-9481-51ab008533af None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:56.969516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edfe81-efde-4874-9481-51ab008533af None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:56.969792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edfe81-efde-4874-9481-51ab008533af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:56.987097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c8f71-4dd3-4619-bbf5-01a5576c3349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:56.987712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c8f71-4dd3-4619-bbf5-01a5576c3349 None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:56.987994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c8f71-4dd3-4619-bbf5-01a5576c3349 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:56.988266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c8f71-4dd3-4619-bbf5-01a5576c3349 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:56.988945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c8f71-4dd3-4619-bbf5-01a5576c3349 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:56.989250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-38edfe81-efde-4874-9481-51ab008533af None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:56.990321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c8f71-4dd3-4619-bbf5-01a5576c3349 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:56.990433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-38edfe81-efde-4874-9481-51ab008533af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:57.014223 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 443/919] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:33:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:33:57.080054 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 477/920] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 01 20:33:57.328656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a83f20-fd6d-4685-b38e-9a4be7b3b7b8 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:57.329041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a83f20-fd6d-4685-b38e-9a4be7b3b7b8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:57.329248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a83f20-fd6d-4685-b38e-9a4be7b3b7b8 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:57.330850 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 444/921] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:33:57.335998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbb041-eaf9-44ca-9a4d-6bdf561d5814 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:57.336291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbb041-eaf9-44ca-9a4d-6bdf561d5814 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:57.336569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbb041-eaf9-44ca-9a4d-6bdf561d5814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:57.355026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c0dbb041-eaf9-44ca-9a4d-6bdf561d5814 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:57.356231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c0dbb041-eaf9-44ca-9a4d-6bdf561d5814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:57.419405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a83f20-fd6d-4685-b38e-9a4be7b3b7b8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:57.424864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:57.431878 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 478/922] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:33:57.529173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:57.534994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a83f20-fd6d-4685-b38e-9a4be7b3b7b8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:57.571524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bc73d-6940-4a0f-9f3c-3eb7e0c554c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:57.572155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bc73d-6940-4a0f-9f3c-3eb7e0c554c0 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:57.572429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bc73d-6940-4a0f-9f3c-3eb7e0c554c0 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:57.572747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bc73d-6940-4a0f-9f3c-3eb7e0c554c0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:57.573330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087bc73d-6940-4a0f-9f3c-3eb7e0c554c0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:57.574710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087bc73d-6940-4a0f-9f3c-3eb7e0c554c0 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:57.584829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-087bc73d-6940-4a0f-9f3c-3eb7e0c554c0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 01 20:33:57.584829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:57.584829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:33:57.584829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:33:57.584829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:33:57.584829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:33:57.584829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:33:57.584829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:57.584829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 01 20:33:57.584829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:57.584829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:33:57.584829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:57.584829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:57.584829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:57.584829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:57.584829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:57.587226 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:57.587226 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:57.587226 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:57.587226 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:57.587226 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:57.587226 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:57.587226 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:57.587226 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:33:57.587226 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:33:57.587226 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:33:57.587226 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:33:57.587226 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:57.587226 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:57.587226 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:57.587226 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:57.587226 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:57.587226 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:57.589482 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:57.589482 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:57.589482 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:57.589482 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:57.589482 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:57.589482 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:57.589482 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:57.589482 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:57.589482 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:57.589482 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:57.589482 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:33:57.589482 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:57.589482 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 01 20:33:57.589482 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:57.589482 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 445/923] 10.222.0.17 () {56 vars in 1132 bytes} [Sat Oct 1 20:33:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:57.595615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbb041-eaf9-44ca-9a4d-6bdf561d5814 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:57.698458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbb041-eaf9-44ca-9a4d-6bdf561d5814 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:57.733040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224aac4-4358-419e-98b1-c31e3117bd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:57.733695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224aac4-4358-419e-98b1-c31e3117bd7f None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:57.734060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224aac4-4358-419e-98b1-c31e3117bd7f None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:57.734312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224aac4-4358-419e-98b1-c31e3117bd7f None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:57.734968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224aac4-4358-419e-98b1-c31e3117bd7f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:57.735268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224aac4-4358-419e-98b1-c31e3117bd7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:57.736618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224aac4-4358-419e-98b1-c31e3117bd7f None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:57.741811 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 479/924] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Oct 1 20:33:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:57.747165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bc73d-6940-4a0f-9f3c-3eb7e0c554c0 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:57.803519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224aac4-4358-419e-98b1-c31e3117bd7f None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:57.879003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bc73d-6940-4a0f-9f3c-3eb7e0c554c0 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:57.891373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:57.923997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146415ad-4837-4d0b-a5a8-e688078d7f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:57.924601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146415ad-4837-4d0b-a5a8-e688078d7f05 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:57.924881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146415ad-4837-4d0b-a5a8-e688078d7f05 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:57.925154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146415ad-4837-4d0b-a5a8-e688078d7f05 None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:57.925786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146415ad-4837-4d0b-a5a8-e688078d7f05 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:57.926089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146415ad-4837-4d0b-a5a8-e688078d7f05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:57.927509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146415ad-4837-4d0b-a5a8-e688078d7f05 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:57.938555 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 446/925] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Oct 1 20:33:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:57.940672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146415ad-4837-4d0b-a5a8-e688078d7f05 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:57.943316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224aac4-4358-419e-98b1-c31e3117bd7f None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:57.970736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:57.986297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd5b7d-6dff-42ed-aea0-cfcc52b74d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:57.986967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd5b7d-6dff-42ed-aea0-cfcc52b74d33 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:57.987249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd5b7d-6dff-42ed-aea0-cfcc52b74d33 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:57.987528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd5b7d-6dff-42ed-aea0-cfcc52b74d33 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:57.988112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd5b7d-6dff-42ed-aea0-cfcc52b74d33 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:57.989919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd5b7d-6dff-42ed-aea0-cfcc52b74d33 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:58.001150 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-45cd5b7d-6dff-42ed-aea0-cfcc52b74d33 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 01 20:33:58.001150 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:58.001150 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:33:58.001150 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:33:58.001150 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:33:58.001150 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:33:58.001150 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:33:58.001150 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:58.001150 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 01 20:33:58.001150 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:58.001150 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:33:58.001150 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:58.001150 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:58.001150 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:58.001150 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:58.001150 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:58.001150 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:58.002183 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:58.002183 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:58.002183 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:58.002183 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:58.002183 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:58.002183 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:58.002183 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:33:58.002183 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:33:58.002183 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:33:58.002183 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:33:58.002183 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:58.002183 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:58.002183 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:58.002183 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:58.002183 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:58.002183 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:58.007626 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:58.007626 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:58.007626 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:58.007626 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:58.007626 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:58.007626 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:58.007626 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:58.007626 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:58.007626 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:58.007626 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:58.007626 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:33:58.007626 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:33:58.007626 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 01 20:33:58.007626 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:33:58.007626 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 480/926] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Oct 1 20:33:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:33:58.010536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd5b7d-6dff-42ed-aea0-cfcc52b74d33 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:58.073539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146415ad-4837-4d0b-a5a8-e688078d7f05 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:58.118463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34390fb-8104-43b4-a426-a72e399018b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:58.120126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34390fb-8104-43b4-a426-a72e399018b2 None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:58.120896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34390fb-8104-43b4-a426-a72e399018b2 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:58.121871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34390fb-8104-43b4-a426-a72e399018b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:58.123026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34390fb-8104-43b4-a426-a72e399018b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:58.125362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34390fb-8104-43b4-a426-a72e399018b2 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:58.132098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd5b7d-6dff-42ed-aea0-cfcc52b74d33 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:58.159059 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 447/927] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:33:57 2022] POST /identity/v3/endpoints => generated 347 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:33:58.165496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:58.184364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f87682-2889-4efd-8a7a-dd96ce6b2a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:58.185036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f87682-2889-4efd-8a7a-dd96ce6b2a51 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:58.185321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f87682-2889-4efd-8a7a-dd96ce6b2a51 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:58.185593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f87682-2889-4efd-8a7a-dd96ce6b2a51 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:58.186290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f87682-2889-4efd-8a7a-dd96ce6b2a51 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:58.186642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f87682-2889-4efd-8a7a-dd96ce6b2a51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:58.188617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f87682-2889-4efd-8a7a-dd96ce6b2a51 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:58.194859 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 481/928] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Oct 1 20:33:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:58.202130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34390fb-8104-43b4-a426-a72e399018b2 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:58.288861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b6726-1aa5-4670-a0ee-033ff71e32c7 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:58.289216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b6726-1aa5-4670-a0ee-033ff71e32c7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:58.289430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b6726-1aa5-4670-a0ee-033ff71e32c7 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:58.294637 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 482/929] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:33:58.296805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29725264-cc76-4290-b011-376560a1f780 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:58.297089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29725264-cc76-4290-b011-376560a1f780 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:58.297368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29725264-cc76-4290-b011-376560a1f780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:58.313981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:58.316954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-29725264-cc76-4290-b011-376560a1f780 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:58.318181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-29725264-cc76-4290-b011-376560a1f780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:58.332030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34390fb-8104-43b4-a426-a72e399018b2 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:58.372338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228353c-f1e6-4d79-9343-720f7f8493ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:58.372599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228353c-f1e6-4d79-9343-720f7f8493ef None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:58.372933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228353c-f1e6-4d79-9343-720f7f8493ef None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:58.373304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228353c-f1e6-4d79-9343-720f7f8493ef None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:58.374081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228353c-f1e6-4d79-9343-720f7f8493ef None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:58.375049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228353c-f1e6-4d79-9343-720f7f8493ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:58.376587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228353c-f1e6-4d79-9343-720f7f8493ef None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:58.384448 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 448/930] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Oct 1 20:33:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:58.389883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228353c-f1e6-4d79-9343-720f7f8493ef None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:58.427740 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 483/931] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 01 20:33:58.517088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228353c-f1e6-4d79-9343-720f7f8493ef None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:58.565956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4f748-4a97-4146-84a0-7afc9b14244d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:58.567762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4f748-4a97-4146-84a0-7afc9b14244d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:58.568268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4f748-4a97-4146-84a0-7afc9b14244d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:58.568742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4f748-4a97-4146-84a0-7afc9b14244d None admin] PATH_INFO: `/v3/endpoints` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:58.569757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4f748-4a97-4146-84a0-7afc9b14244d None admin] RBAC: Adding query filter params (interface=public, service_id=2cdb69f5f1cb4ffabd171e2bbc67a7de, region_id=RegionOne) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:58.570272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4f748-4a97-4146-84a0-7afc9b14244d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:58.572520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4f748-4a97-4146-84a0-7afc9b14244d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:58.581104 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 449/932] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Oct 1 20:33:58 2022] GET /identity/v3/endpoints?service_id=2cdb69f5f1cb4ffabd171e2bbc67a7de&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:33:58.587194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29725264-cc76-4290-b011-376560a1f780 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:58.695456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29725264-cc76-4290-b011-376560a1f780 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:58.709388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:58.729867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8d2b3-42d2-4043-8703-7ca802f6c6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:58.730465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8d2b3-42d2-4043-8703-7ca802f6c6b6 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:58.730924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8d2b3-42d2-4043-8703-7ca802f6c6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:58.731205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8d2b3-42d2-4043-8703-7ca802f6c6b6 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:58.731807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e8d2b3-42d2-4043-8703-7ca802f6c6b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:58.733156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e8d2b3-42d2-4043-8703-7ca802f6c6b6 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:58.739949 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 484/933] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Oct 1 20:33:58 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:33:58.754508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4f748-4a97-4146-84a0-7afc9b14244d None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:58.769715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:58.870149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4f748-4a97-4146-84a0-7afc9b14244d None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:58.909962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7fb62-56fe-4f3f-be5f-f0ed98de001e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:58.910689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7fb62-56fe-4f3f-be5f-f0ed98de001e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:58.910967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7fb62-56fe-4f3f-be5f-f0ed98de001e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:58.911234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7fb62-56fe-4f3f-be5f-f0ed98de001e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:58.922943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7fb62-56fe-4f3f-be5f-f0ed98de001e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:33:58.928334 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-17b7fb62-56fe-4f3f-be5f-f0ed98de001e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 01 20:33:58.928334 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:58.928334 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:58.928334 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:58.928334 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:58.928334 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:58.928334 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:58.928334 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:58.928334 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:58.928334 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:58.928334 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:58.928334 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:58.928334 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:33:58.928334 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:33:58.928334 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:33:58.928334 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:33:58.929854 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:33:58.929854 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:33:58.929854 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:33:58.929854 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:33:58.929854 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:58.929854 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:58.929854 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:58.929854 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:58.929854 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:58.929854 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:58.929854 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:58.929854 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:58.929854 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:58.929854 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:58.929854 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:58.929854 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:58.931346 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:58.931346 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:58.931346 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:58.931346 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:58.931346 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:33:58.931346 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:33:58.931346 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:33:58.931346 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:33:58.931346 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:33:58.931346 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:33:58.931346 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 01 20:33:58.931346 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:58.934982 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 450/934] 10.222.0.17 () {56 vars in 1120 bytes} [Sat Oct 1 20:33:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:58.936261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8d2b3-42d2-4043-8703-7ca802f6c6b6 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:59.052771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8d2b3-42d2-4043-8703-7ca802f6c6b6 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:59.093441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005eb583-dc9a-4ada-9da7-c5e9446b092a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:59.094068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005eb583-dc9a-4ada-9da7-c5e9446b092a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:59.094346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005eb583-dc9a-4ada-9da7-c5e9446b092a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:59.094664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005eb583-dc9a-4ada-9da7-c5e9446b092a None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:59.095350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005eb583-dc9a-4ada-9da7-c5e9446b092a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:59.095664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005eb583-dc9a-4ada-9da7-c5e9446b092a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:59.097049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005eb583-dc9a-4ada-9da7-c5e9446b092a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:59.110634 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 485/935] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Oct 1 20:33:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:59.113522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7fb62-56fe-4f3f-be5f-f0ed98de001e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:59.191318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:59.224865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7fb62-56fe-4f3f-be5f-f0ed98de001e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:59.262919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ecbb74-284e-4c66-9e2c-0ffe0a0fd66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:59.263554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecbb74-284e-4c66-9e2c-0ffe0a0fd66f None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:59.263826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecbb74-284e-4c66-9e2c-0ffe0a0fd66f None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:59.264098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecbb74-284e-4c66-9e2c-0ffe0a0fd66f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:59.275382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ecbb74-284e-4c66-9e2c-0ffe0a0fd66f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:59.276817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ecbb74-284e-4c66-9e2c-0ffe0a0fd66f None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:59.291455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-57ecbb74-284e-4c66-9e2c-0ffe0a0fd66f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:59.291455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:59.291455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:59.291455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:59.291455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:59.291455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:59.291455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:59.291455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:59.291455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:59.291455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:59.291455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:59.291455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:59.291455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:33:59.291455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:33:59.291455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:33:59.291455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:33:59.292469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:59.292469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:59.292469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:59.292469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:59.292469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:59.292469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:59.292469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:59.292469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:59.292469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:59.292469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:59.292469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:59.292469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:59.292469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:59.292469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:59.292469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:59.292469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:59.292469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:33:59.293381 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:33:59.293381 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:33:59.293381 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:59.297987 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 451/936] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:59.299382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005eb583-dc9a-4ada-9da7-c5e9446b092a None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:59.342673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:59.403840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005eb583-dc9a-4ada-9da7-c5e9446b092a None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:59.438995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d9817-9a2b-4959-9ffe-e05128b020a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:59.439620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d9817-9a2b-4959-9ffe-e05128b020a8 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:59.439865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d9817-9a2b-4959-9ffe-e05128b020a8 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:59.440139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d9817-9a2b-4959-9ffe-e05128b020a8 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:59.440783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d9817-9a2b-4959-9ffe-e05128b020a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:59.441086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d9817-9a2b-4959-9ffe-e05128b020a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:59.442565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d9817-9a2b-4959-9ffe-e05128b020a8 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:59.457265 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 486/937] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:59.463456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ecbb74-284e-4c66-9e2c-0ffe0a0fd66f None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:59.514961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70763dfa-4e23-472c-9fa4-18144f5cf08d None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:59.515336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70763dfa-4e23-472c-9fa4-18144f5cf08d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:59.515539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70763dfa-4e23-472c-9fa4-18144f5cf08d None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:59.517276 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 487/938] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:33:59.523949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a87614-7d18-4e58-91b2-ec34f17fe41b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:59.524303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a87614-7d18-4e58-91b2-ec34f17fe41b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:59.524594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a87614-7d18-4e58-91b2-ec34f17fe41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:59.543695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-39a87614-7d18-4e58-91b2-ec34f17fe41b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:33:59.545032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-39a87614-7d18-4e58-91b2-ec34f17fe41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:33:59.573820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ecbb74-284e-4c66-9e2c-0ffe0a0fd66f None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:59.605163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:59.621547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193515a-00de-465e-922e-ce430f94729c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:59.622117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193515a-00de-465e-922e-ce430f94729c None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:59.622391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193515a-00de-465e-922e-ce430f94729c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:59.622688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193515a-00de-465e-922e-ce430f94729c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:59.623167 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 488/939] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:33:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 01 20:33:59.638676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b193515a-00de-465e-922e-ce430f94729c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:59.640204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b193515a-00de-465e-922e-ce430f94729c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:59.652121 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-b193515a-00de-465e-922e-ce430f94729c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 01 20:33:59.652121 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:33:59.652121 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:33:59.652121 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:33:59.652121 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:33:59.652121 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:33:59.652121 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:33:59.652121 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:33:59.652121 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:33:59.652121 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:33:59.652121 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:33:59.652121 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:33:59.652121 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:33:59.652121 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:33:59.652121 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:33:59.652121 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:33:59.653111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:33:59.653111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:33:59.653111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:33:59.653111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:59.653111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:33:59.653111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:33:59.653111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:33:59.653111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:33:59.653111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:33:59.653111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:33:59.653111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:33:59.653111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:33:59.653111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:33:59.653111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:33:59.653111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:33:59.653111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:33:59.653111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:33:59.653111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:33:59.654075 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:33:59.654075 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:33:59.654075 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:33:59.654075 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:33:59.654075 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:33:59.654075 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:33:59.654075 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:33:59.654075 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:33:59.654075 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 01 20:33:59.654075 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:33:59.654075 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 452/940] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Oct 1 20:33:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:33:59.661432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a87614-7d18-4e58-91b2-ec34f17fe41b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:59.694015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:33:59.771448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a87614-7d18-4e58-91b2-ec34f17fe41b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:59.806041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d4501-d6a4-4656-a5c7-e480ebaee3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:59.806674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d4501-d6a4-4656-a5c7-e480ebaee3e3 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:59.806952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d4501-d6a4-4656-a5c7-e480ebaee3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:59.807220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d4501-d6a4-4656-a5c7-e480ebaee3e3 None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:59.807943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d4501-d6a4-4656-a5c7-e480ebaee3e3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:33:59.808247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d4501-d6a4-4656-a5c7-e480ebaee3e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:59.809655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d4501-d6a4-4656-a5c7-e480ebaee3e3 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:33:59.827404 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 489/941] 10.222.0.17 () {56 vars in 1149 bytes} [Sat Oct 1 20:33:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:33:59.833008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193515a-00de-465e-922e-ce430f94729c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:59.939714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193515a-00de-465e-922e-ce430f94729c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:33:59.975080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa0710-98e0-42aa-b10a-66db34536d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:33:59.975695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa0710-98e0-42aa-b10a-66db34536d48 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:33:59.975972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa0710-98e0-42aa-b10a-66db34536d48 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:33:59.976242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa0710-98e0-42aa-b10a-66db34536d48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:33:59.984102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d4501-d6a4-4656-a5c7-e480ebaee3e3 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:33:59.990179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa0710-98e0-42aa-b10a-66db34536d48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:33:59.991674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa0710-98e0-42aa-b10a-66db34536d48 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:00.008974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-15aa0710-98e0-42aa-b10a-66db34536d48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:34:00.008974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:00.008974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:00.008974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:00.008974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:00.008974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:00.008974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:00.008974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:00.008974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:00.008974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:00.008974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:00.008974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:00.008974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:34:00.008974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:34:00.008974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:34:00.008974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:34:00.009997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:00.009997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:00.009997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:00.009997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:00.009997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:00.009997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:00.009997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:00.009997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:00.009997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:00.009997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:00.009997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:00.009997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:00.009997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:00.009997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:00.009997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:00.009997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:00.009997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:34:00.010947 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:34:00.010947 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:34:00.010947 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:00.018206 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 453/942] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:33:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:00.019428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa0710-98e0-42aa-b10a-66db34536d48 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:00.059392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:00.094689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:00.109568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d4501-d6a4-4656-a5c7-e480ebaee3e3 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:00.137289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa0710-98e0-42aa-b10a-66db34536d48 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:00.156966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658671c-c0b7-4265-be70-1e48345caa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:00.157585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658671c-c0b7-4265-be70-1e48345caa12 None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:00.157865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658671c-c0b7-4265-be70-1e48345caa12 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:00.158135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658671c-c0b7-4265-be70-1e48345caa12 None admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:00.159077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658671c-c0b7-4265-be70-1e48345caa12 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:00.161326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658671c-c0b7-4265-be70-1e48345caa12 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:00.164222 ncdv-master-1 devstack@keystone.service[99139]: WARNING keystone.server.flask.common [None req-5658671c-c0b7-4265-be70-1e48345caa12 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 01 20:34:00.182198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105078b0-b2cf-4541-b6ab-234d32af4ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:00.182915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105078b0-b2cf-4541-b6ab-234d32af4ff4 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:00.183300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105078b0-b2cf-4541-b6ab-234d32af4ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:00.183614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105078b0-b2cf-4541-b6ab-234d32af4ff4 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:00.184368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105078b0-b2cf-4541-b6ab-234d32af4ff4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:00.184704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105078b0-b2cf-4541-b6ab-234d32af4ff4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:00.185487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.sql.core [None req-5658671c-c0b7-4265-be70-1e48345caa12 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 01 20:34:00.185487 ncdv-master-1 devstack@keystone.service[99139]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 01 20:34:00.185487 ncdv-master-1 devstack@keystone.service[99139]: [parameters: {'id': '406708219e63435a9513dcc5c6b2efa0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 01 20:34:00.185487 ncdv-master-1 devstack@keystone.service[99139]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99139) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 01 20:34:00.186305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105078b0-b2cf-4541-b6ab-234d32af4ff4 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:00.199472 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 454/943] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:00.205230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105078b0-b2cf-4541-b6ab-234d32af4ff4 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:00.206021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-5658671c-c0b7-4265-be70-1e48345caa12 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 406708219e63435a9513dcc5c6b2efa0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 406708219e63435a9513dcc5c6b2efa0. Oct 01 20:34:00.206021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:00.206021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 01 20:34:00.206021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 01 20:34:00.206021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 01 20:34:00.206021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 01 20:34:00.206021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 01 20:34:00.206021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application result = self._query(query) Oct 01 20:34:00.206021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 01 20:34:00.206021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application conn.query(q) Oct 01 20:34:00.206021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 01 20:34:00.206021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 01 20:34:00.206021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 01 20:34:00.206909 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application result.read() Oct 01 20:34:00.206909 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 01 20:34:00.206909 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 01 20:34:00.206909 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 01 20:34:00.206909 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 01 20:34:00.206909 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 01 20:34:00.206909 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 01 20:34:00.206909 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 01 20:34:00.206909 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 01 20:34:00.206909 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 01 20:34:00.206909 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:00.206909 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 01 20:34:00.206909 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:00.206909 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:00.206909 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 01 20:34:00.206909 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 01 20:34:00.206909 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 01 20:34:00.206909 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 01 20:34:00.207866 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 01 20:34:00.207866 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application next(self.gen) Oct 01 20:34:00.207866 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 01 20:34:00.207866 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application yield resource Oct 01 20:34:00.207866 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 01 20:34:00.207866 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application next(self.gen) Oct 01 20:34:00.207866 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 01 20:34:00.207866 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.session.rollback() Oct 01 20:34:00.207866 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 01 20:34:00.207866 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.force_reraise() Oct 01 20:34:00.207866 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 01 20:34:00.207866 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise self.value Oct 01 20:34:00.207866 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 01 20:34:00.207866 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 01 20:34:00.207866 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 01 20:34:00.207866 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application session.commit() Oct 01 20:34:00.207866 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 01 20:34:00.208802 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 01 20:34:00.208802 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 01 20:34:00.208802 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._prepare_impl() Oct 01 20:34:00.208802 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 01 20:34:00.208802 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.session.flush() Oct 01 20:34:00.208802 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 01 20:34:00.208802 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._flush(objects) Oct 01 20:34:00.208802 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 01 20:34:00.208802 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 01 20:34:00.208802 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 01 20:34:00.208802 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application compat.raise_( Oct 01 20:34:00.208802 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 01 20:34:00.208802 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception Oct 01 20:34:00.208802 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 01 20:34:00.208802 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application flush_context.execute() Oct 01 20:34:00.208802 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 01 20:34:00.208802 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rec.execute(self) Oct 01 20:34:00.209729 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 01 20:34:00.209729 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 01 20:34:00.209729 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 01 20:34:00.209729 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 01 20:34:00.209729 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 01 20:34:00.209729 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 01 20:34:00.209729 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 01 20:34:00.209729 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 01 20:34:00.209729 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 01 20:34:00.209729 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 01 20:34:00.209729 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 01 20:34:00.209729 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 01 20:34:00.209729 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 01 20:34:00.209729 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 01 20:34:00.209729 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 01 20:34:00.210581 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 01 20:34:00.210581 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 01 20:34:00.210581 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception Oct 01 20:34:00.210581 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 01 20:34:00.210581 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 01 20:34:00.210581 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 01 20:34:00.210581 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 01 20:34:00.210581 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 01 20:34:00.210581 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application result = self._query(query) Oct 01 20:34:00.210581 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 01 20:34:00.210581 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application conn.query(q) Oct 01 20:34:00.210581 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 01 20:34:00.210581 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 01 20:34:00.210581 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 01 20:34:00.210581 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application result.read() Oct 01 20:34:00.210581 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 01 20:34:00.210581 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 01 20:34:00.211480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 01 20:34:00.211480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 01 20:34:00.211480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 01 20:34:00.211480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 01 20:34:00.211480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 01 20:34:00.211480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 01 20:34:00.211480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 01 20:34:00.211480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 01 20:34:00.211480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [parameters: {'id': '406708219e63435a9513dcc5c6b2efa0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 01 20:34:00.211480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 01 20:34:00.211480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:00.211480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:34:00.211480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:00.211480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:00.211480 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 01 20:34:00.212272 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 01 20:34:00.212272 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 01 20:34:00.212272 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 01 20:34:00.212272 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 01 20:34:00.212272 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:00.212272 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:34:00.212272 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:00.212272 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:00.212272 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:00.212272 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:00.212272 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:00.212272 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:00.212272 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:00.212272 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:00.212272 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:00.212272 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:00.212272 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:00.213203 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:00.213203 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 01 20:34:00.213203 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 01 20:34:00.213203 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:00.213203 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:00.213203 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 01 20:34:00.213203 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 01 20:34:00.213203 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 406708219e63435a9513dcc5c6b2efa0. Oct 01 20:34:00.213203 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:00.213203 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 490/944] 10.222.0.17 () {60 vars in 1140 bytes} [Sat Oct 1 20:33:59 2022] POST /identity/v3/projects => generated 262 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:34:00.221833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658671c-c0b7-4265-be70-1e48345caa12 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:00.335738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658671c-c0b7-4265-be70-1e48345caa12 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:00.337378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:00.347173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105078b0-b2cf-4541-b6ab-234d32af4ff4 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:00.380345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51383906-77da-47e5-920b-dc83e9b93b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:00.381385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51383906-77da-47e5-920b-dc83e9b93b3e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:00.382142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51383906-77da-47e5-920b-dc83e9b93b3e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:00.382578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51383906-77da-47e5-920b-dc83e9b93b3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:00.387220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437179ac-4656-4cab-8380-0db1ead1534f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:00.387851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437179ac-4656-4cab-8380-0db1ead1534f None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:00.388140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437179ac-4656-4cab-8380-0db1ead1534f None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:00.388411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437179ac-4656-4cab-8380-0db1ead1534f None admin] PATH_INFO: `/v3/projects/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:00.390819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51383906-77da-47e5-920b-dc83e9b93b3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:00.392663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51383906-77da-47e5-920b-dc83e9b93b3e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:00.395673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437179ac-4656-4cab-8380-0db1ead1534f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:00.397107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437179ac-4656-4cab-8380-0db1ead1534f None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:00.406409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-51383906-77da-47e5-920b-dc83e9b93b3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:34:00.406409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:00.406409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:00.406409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:00.406409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:00.406409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:00.406409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:00.406409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:00.406409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:00.406409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:00.406409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:00.406409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:00.406409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:34:00.406409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:34:00.406409 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:34:00.407556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:34:00.407556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:00.407556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:00.407556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:00.407556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:00.407556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:00.407556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:00.407556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:00.407556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:00.407556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:00.407556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:00.407556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:00.407556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:00.407556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:00.407556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:00.407556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:00.407556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:00.409465 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:34:00.409465 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:34:00.409465 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:34:00.409465 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:34:00.409465 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:34:00.409465 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:34:00.409465 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:34:00.409465 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:00.410355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-437179ac-4656-4cab-8380-0db1ead1534f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:34:00.410355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:00.410355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:00.410355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:00.410355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:00.410355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:00.410355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:00.410355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:00.410355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:00.410355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:00.410355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:00.410355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:00.410355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:34:00.410355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:34:00.410355 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:34:00.412067 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:34:00.412067 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:00.412067 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:00.412067 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:00.412067 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:00.412067 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:00.412067 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:00.412067 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:00.412067 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:00.412067 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:00.412067 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:00.412067 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:00.412067 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:00.412067 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:00.412067 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:00.412067 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:00.412067 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:00.418762 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:34:00.418762 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:34:00.418762 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:34:00.418762 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:34:00.418762 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:34:00.418762 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:34:00.418762 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:34:00.418762 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:00.419793 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 491/945] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:34:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:34:00.419960 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 455/946] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:34:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:00.423670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51383906-77da-47e5-920b-dc83e9b93b3e None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:00.425100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437179ac-4656-4cab-8380-0db1ead1534f None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:00.429356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:00.539060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437179ac-4656-4cab-8380-0db1ead1534f None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:00.560359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51383906-77da-47e5-920b-dc83e9b93b3e None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:00.574023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174590f8-2e2d-4d18-86d1-f3f60b5c9e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:00.575459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174590f8-2e2d-4d18-86d1-f3f60b5c9e0a None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:00.575993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174590f8-2e2d-4d18-86d1-f3f60b5c9e0a None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:00.576473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174590f8-2e2d-4d18-86d1-f3f60b5c9e0a None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:00.577397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174590f8-2e2d-4d18-86d1-f3f60b5c9e0a None admin] RBAC: Adding query filter params (name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:00.578023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174590f8-2e2d-4d18-86d1-f3f60b5c9e0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:00.579932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174590f8-2e2d-4d18-86d1-f3f60b5c9e0a None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:00.603290 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 456/947] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Oct 1 20:34:00 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:00.607972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099ec1e-1865-4764-80ae-2d39bf90a83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:00.608606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099ec1e-1865-4764-80ae-2d39bf90a83e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:00.608997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099ec1e-1865-4764-80ae-2d39bf90a83e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:00.609271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099ec1e-1865-4764-80ae-2d39bf90a83e None admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:00.609983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099ec1e-1865-4764-80ae-2d39bf90a83e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:00.610358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099ec1e-1865-4764-80ae-2d39bf90a83e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:00.611886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099ec1e-1865-4764-80ae-2d39bf90a83e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:00.630158 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 492/948] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:34:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:00.636729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174590f8-2e2d-4d18-86d1-f3f60b5c9e0a None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:00.654416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:00.752944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174590f8-2e2d-4d18-86d1-f3f60b5c9e0a None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:00.793950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe6360-3fc9-4ba0-9bb3-7609f85254ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:00.794667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe6360-3fc9-4ba0-9bb3-7609f85254ab None admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:00.794948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe6360-3fc9-4ba0-9bb3-7609f85254ab None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:00.795221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe6360-3fc9-4ba0-9bb3-7609f85254ab None admin] PATH_INFO: `/v3/projects/b7875946723844198353bbfcf22516fb/users/917fc96fa3504b19b1a56990d4747439/roles/fcfc5f73645d423d8475708ef191dacb` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:00.797550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe6360-3fc9-4ba0-9bb3-7609f85254ab None admin] RBAC: Authorizing `identity:create_grant(project_id=b7875946723844198353bbfcf22516fb, user_id=917fc96fa3504b19b1a56990d4747439, role_id=fcfc5f73645d423d8475708ef191dacb)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:00.799054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe6360-3fc9-4ba0-9bb3-7609f85254ab None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:00.821425 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 457/949] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:34:00 2022] PUT /identity/v3/projects/b7875946723844198353bbfcf22516fb/users/917fc96fa3504b19b1a56990d4747439/roles/fcfc5f73645d423d8475708ef191dacb => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:34:00.868548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae914cde-7c64-45e3-9c18-0ebadb921164 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:00.868865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae914cde-7c64-45e3-9c18-0ebadb921164 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:00.869075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae914cde-7c64-45e3-9c18-0ebadb921164 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:00.870977 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 493/950] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:00.877223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bcedc-a1cf-4a56-a09a-df2a5d696aa0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:00.877806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bcedc-a1cf-4a56-a09a-df2a5d696aa0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:00.878302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bcedc-a1cf-4a56-a09a-df2a5d696aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:00.898768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d75bcedc-a1cf-4a56-a09a-df2a5d696aa0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:00.900332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d75bcedc-a1cf-4a56-a09a-df2a5d696aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:00.981116 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 458/951] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:01.323912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae914cde-7c64-45e3-9c18-0ebadb921164 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:01.392285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:01.422305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1e8cf-cb24-497a-bdc8-e30757706e75 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:01.422665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1e8cf-cb24-497a-bdc8-e30757706e75 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:01.422894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1e8cf-cb24-497a-bdc8-e30757706e75 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:01.425490 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 459/952] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:01.429862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb29215-118d-4bdf-b4c5-efa3024878cc None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:01.430167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb29215-118d-4bdf-b4c5-efa3024878cc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:01.430446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb29215-118d-4bdf-b4c5-efa3024878cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:01.434711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae914cde-7c64-45e3-9c18-0ebadb921164 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:01.440596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:01.454759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8cb29215-118d-4bdf-b4c5-efa3024878cc None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:01.456770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8cb29215-118d-4bdf-b4c5-efa3024878cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:01.480922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b14bb-4089-4226-8499-610b83b89055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:01.481561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b14bb-4089-4226-8499-610b83b89055 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:01.481851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b14bb-4089-4226-8499-610b83b89055 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:01.482124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b14bb-4089-4226-8499-610b83b89055 None admin] PATH_INFO: `/v3/services/compute` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:01.482737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b14bb-4089-4226-8499-610b83b89055 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:01.484755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b14bb-4089-4226-8499-610b83b89055 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:01.496603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-089b14bb-4089-4226-8499-610b83b89055 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 01 20:34:01.496603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:01.496603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:34:01.496603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:34:01.496603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:34:01.496603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:34:01.496603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:34:01.496603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:34:01.496603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 01 20:34:01.496603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:01.496603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:34:01.496603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:01.496603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:01.496603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:01.496603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:01.496603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:01.496603 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:01.498239 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:01.498239 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:01.498239 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:01.498239 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:01.498239 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:01.498239 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:01.498239 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:34:01.498239 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:34:01.498239 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:34:01.498239 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:34:01.498239 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:01.498239 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:01.498239 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:01.498239 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:01.498239 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:01.498239 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:01.499390 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:01.499390 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:01.499390 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:01.499390 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:01.499390 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:01.499390 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:01.499390 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:01.499390 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:01.499390 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:01.499390 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:01.499390 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:34:01.499390 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:34:01.499390 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 01 20:34:01.499390 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:01.499390 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 494/953] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:34:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:01.507592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b14bb-4089-4226-8499-610b83b89055 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:01.552962 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 460/954] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:01.622463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b14bb-4089-4226-8499-610b83b89055 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:01.650206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:01.659840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e43b7-6241-4554-905b-8810ef9c60c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:01.660763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e43b7-6241-4554-905b-8810ef9c60c8 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:01.661407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e43b7-6241-4554-905b-8810ef9c60c8 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:01.661929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e43b7-6241-4554-905b-8810ef9c60c8 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:01.663021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e43b7-6241-4554-905b-8810ef9c60c8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:01.663324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e43b7-6241-4554-905b-8810ef9c60c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:01.664732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e43b7-6241-4554-905b-8810ef9c60c8 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:01.669918 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 495/955] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Oct 1 20:34:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:01.675561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb29215-118d-4bdf-b4c5-efa3024878cc None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:01.784014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb29215-118d-4bdf-b4c5-efa3024878cc None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:01.819429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523452d5-e00e-4859-9b62-052f84747565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:01.820340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523452d5-e00e-4859-9b62-052f84747565 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:01.821086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523452d5-e00e-4859-9b62-052f84747565 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:01.821556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523452d5-e00e-4859-9b62-052f84747565 None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:01.822378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523452d5-e00e-4859-9b62-052f84747565 None admin] RBAC: Adding query filter params (type=compute) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:01.822900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523452d5-e00e-4859-9b62-052f84747565 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:01.824479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523452d5-e00e-4859-9b62-052f84747565 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:01.825875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:01.834514 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 461/956] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Oct 1 20:34:01 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:01.894250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e43b7-6241-4554-905b-8810ef9c60c8 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:02.012709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e43b7-6241-4554-905b-8810ef9c60c8 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:02.051586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea9c0a-fdb4-4767-a2b8-a201d436f483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:02.052274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea9c0a-fdb4-4767-a2b8-a201d436f483 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:02.052556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea9c0a-fdb4-4767-a2b8-a201d436f483 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:02.052823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea9c0a-fdb4-4767-a2b8-a201d436f483 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:02.053406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea9c0a-fdb4-4767-a2b8-a201d436f483 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:02.054813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea9c0a-fdb4-4767-a2b8-a201d436f483 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:02.064517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-cbea9c0a-fdb4-4767-a2b8-a201d436f483 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 01 20:34:02.064517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:02.064517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:34:02.064517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:34:02.064517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:34:02.064517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:34:02.064517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:34:02.064517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:34:02.064517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 01 20:34:02.064517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:02.064517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:34:02.064517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:02.064517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:02.064517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:02.064517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:02.064517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:02.064517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:02.065595 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:02.065595 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:02.065595 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:02.065595 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:02.065595 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:02.065595 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:02.065595 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:34:02.065595 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:34:02.065595 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:34:02.065595 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:34:02.065595 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:02.065595 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:02.065595 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:02.065595 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:02.065595 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:02.065595 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:02.066449 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:02.066449 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:02.066449 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:02.066449 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:02.066449 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:02.066449 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:02.066449 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:02.066449 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:02.066449 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:02.066449 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:02.066449 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:34:02.066449 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:34:02.066449 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 01 20:34:02.066449 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:02.066449 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 496/957] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Oct 1 20:34:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:02.072000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523452d5-e00e-4859-9b62-052f84747565 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:02.179861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523452d5-e00e-4859-9b62-052f84747565 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:02.215804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eecf4d6-ee2b-49d2-bc59-7285237ff7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:02.216408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecf4d6-ee2b-49d2-bc59-7285237ff7dd None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:02.216719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecf4d6-ee2b-49d2-bc59-7285237ff7dd None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:02.216996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecf4d6-ee2b-49d2-bc59-7285237ff7dd None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:02.217618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eecf4d6-ee2b-49d2-bc59-7285237ff7dd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:02.217920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eecf4d6-ee2b-49d2-bc59-7285237ff7dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:02.219819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eecf4d6-ee2b-49d2-bc59-7285237ff7dd None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:02.224883 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 462/958] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Oct 1 20:34:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:02.230219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea9c0a-fdb4-4767-a2b8-a201d436f483 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:02.291497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:02.339270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea9c0a-fdb4-4767-a2b8-a201d436f483 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:02.375291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da5514-044b-4f03-a4f5-416eaee80826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:02.375898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da5514-044b-4f03-a4f5-416eaee80826 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:02.376184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da5514-044b-4f03-a4f5-416eaee80826 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:02.376456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da5514-044b-4f03-a4f5-416eaee80826 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:02.377086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da5514-044b-4f03-a4f5-416eaee80826 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:02.377391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da5514-044b-4f03-a4f5-416eaee80826 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:02.378766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da5514-044b-4f03-a4f5-416eaee80826 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:02.384114 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 497/959] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Oct 1 20:34:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:02.390511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eecf4d6-ee2b-49d2-bc59-7285237ff7dd None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:02.455817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:02.497234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eecf4d6-ee2b-49d2-bc59-7285237ff7dd None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:02.536542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea840f-edaa-42ef-b142-81208e672e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:02.537144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea840f-edaa-42ef-b142-81208e672e7d None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:02.537420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea840f-edaa-42ef-b142-81208e672e7d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:02.537687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea840f-edaa-42ef-b142-81208e672e7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:02.538349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea840f-edaa-42ef-b142-81208e672e7d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:02.539739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea840f-edaa-42ef-b142-81208e672e7d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:02.560893 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 463/960] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:34:02 2022] POST /identity/v3/endpoints => generated 349 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:34:03.241121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2faea-9656-4b19-a377-f18b383b6fde None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:03.241667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2faea-9656-4b19-a377-f18b383b6fde None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:03.241667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2faea-9656-4b19-a377-f18b383b6fde None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:03.243368 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 498/961] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:03.248693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cb0ef-43b0-44dd-92fa-f9bcb75aafd6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:03.249190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cb0ef-43b0-44dd-92fa-f9bcb75aafd6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:03.249420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cb0ef-43b0-44dd-92fa-f9bcb75aafd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:03.268116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f01cb0ef-43b0-44dd-92fa-f9bcb75aafd6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:03.269311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f01cb0ef-43b0-44dd-92fa-f9bcb75aafd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:03.350744 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 464/962] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:03.361109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14342a6-1c0b-48fc-9c3e-64b200f37e81 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:03.361696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14342a6-1c0b-48fc-9c3e-64b200f37e81 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:03.362175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14342a6-1c0b-48fc-9c3e-64b200f37e81 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:03.364008 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 499/963] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:03.369489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c026d4-70bf-4e71-9123-c9a143f0e34b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:03.370021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c026d4-70bf-4e71-9123-c9a143f0e34b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:03.370525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c026d4-70bf-4e71-9123-c9a143f0e34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:03.389276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-74c026d4-70bf-4e71-9123-c9a143f0e34b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:03.390872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-74c026d4-70bf-4e71-9123-c9a143f0e34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:03.454299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:03.465276 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 465/964] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:03.680684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14342a6-1c0b-48fc-9c3e-64b200f37e81 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:03.786592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14342a6-1c0b-48fc-9c3e-64b200f37e81 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:03.796482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c026d4-70bf-4e71-9123-c9a143f0e34b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:03.822905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:03.830523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8cd00-ee86-4e50-b38c-41ff3df377f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:03.831621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8cd00-ee86-4e50-b38c-41ff3df377f1 None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:03.832425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8cd00-ee86-4e50-b38c-41ff3df377f1 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:03.832896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8cd00-ee86-4e50-b38c-41ff3df377f1 None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:03.834421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b8cd00-ee86-4e50-b38c-41ff3df377f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:03.836133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b8cd00-ee86-4e50-b38c-41ff3df377f1 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:03.856298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.sql.core [None req-b4b8cd00-ee86-4e50-b38c-41ff3df377f1 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 01 20:34:03.856298 ncdv-master-1 devstack@keystone.service[99139]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 01 20:34:03.856298 ncdv-master-1 devstack@keystone.service[99139]: [parameters: {'user_id': '0dadd6dca367409fb6a8715969b1582a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 01 20:34:03.856298 ncdv-master-1 devstack@keystone.service[99139]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99139) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 01 20:34:03.866773 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-b4b8cd00-ee86-4e50-b38c-41ff3df377f1 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 01 20:34:03.866773 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:03.866773 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 01 20:34:03.866773 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 01 20:34:03.866773 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 01 20:34:03.866773 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 01 20:34:03.866773 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 01 20:34:03.866773 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application result = self._query(query) Oct 01 20:34:03.866773 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 01 20:34:03.866773 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application conn.query(q) Oct 01 20:34:03.866773 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 01 20:34:03.866773 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 01 20:34:03.866773 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 01 20:34:03.866773 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application result.read() Oct 01 20:34:03.866773 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 01 20:34:03.867682 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 01 20:34:03.867682 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 01 20:34:03.867682 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 01 20:34:03.867682 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 01 20:34:03.867682 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 01 20:34:03.867682 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 01 20:34:03.867682 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 01 20:34:03.867682 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 01 20:34:03.867682 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:03.867682 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 01 20:34:03.867682 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:03.867682 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:03.867682 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 01 20:34:03.867682 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 01 20:34:03.867682 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 01 20:34:03.867682 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 01 20:34:03.867682 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 01 20:34:03.867682 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application next(self.gen) Oct 01 20:34:03.868621 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 01 20:34:03.868621 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application yield resource Oct 01 20:34:03.868621 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 01 20:34:03.868621 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application next(self.gen) Oct 01 20:34:03.868621 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 01 20:34:03.868621 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.session.rollback() Oct 01 20:34:03.868621 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 01 20:34:03.868621 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.force_reraise() Oct 01 20:34:03.868621 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 01 20:34:03.868621 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise self.value Oct 01 20:34:03.868621 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 01 20:34:03.868621 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 01 20:34:03.868621 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 01 20:34:03.868621 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application session.commit() Oct 01 20:34:03.868621 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 01 20:34:03.868621 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 01 20:34:03.868621 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 01 20:34:03.869517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._prepare_impl() Oct 01 20:34:03.869517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 01 20:34:03.869517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.session.flush() Oct 01 20:34:03.869517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 01 20:34:03.869517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._flush(objects) Oct 01 20:34:03.869517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 01 20:34:03.869517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 01 20:34:03.869517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 01 20:34:03.869517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application compat.raise_( Oct 01 20:34:03.869517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 01 20:34:03.869517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception Oct 01 20:34:03.869517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 01 20:34:03.869517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application flush_context.execute() Oct 01 20:34:03.869517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 01 20:34:03.869517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rec.execute(self) Oct 01 20:34:03.869517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 01 20:34:03.869517 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 01 20:34:03.870402 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 01 20:34:03.870402 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 01 20:34:03.870402 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 01 20:34:03.870402 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 01 20:34:03.870402 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 01 20:34:03.870402 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 01 20:34:03.870402 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 01 20:34:03.870402 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 01 20:34:03.870402 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 01 20:34:03.870402 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 01 20:34:03.870402 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 01 20:34:03.870402 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 01 20:34:03.870402 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 01 20:34:03.870402 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 01 20:34:03.870402 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 01 20:34:03.870402 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception Oct 01 20:34:03.871302 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 01 20:34:03.871302 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 01 20:34:03.871302 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 01 20:34:03.871302 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 01 20:34:03.871302 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 01 20:34:03.871302 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application result = self._query(query) Oct 01 20:34:03.871302 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 01 20:34:03.871302 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application conn.query(q) Oct 01 20:34:03.871302 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 01 20:34:03.871302 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 01 20:34:03.871302 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 01 20:34:03.871302 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application result.read() Oct 01 20:34:03.871302 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 01 20:34:03.871302 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 01 20:34:03.871302 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 01 20:34:03.871302 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 01 20:34:03.874987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 01 20:34:03.874987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 01 20:34:03.874987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 01 20:34:03.874987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 01 20:34:03.874987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 01 20:34:03.874987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 01 20:34:03.874987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [parameters: {'user_id': '0dadd6dca367409fb6a8715969b1582a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 01 20:34:03.874987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 01 20:34:03.874987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:03.874987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:34:03.874987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:03.874987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:03.874987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:03.874987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:03.874987 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:03.877370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:03.877370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:03.877370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:03.877370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:03.877370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:03.877370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:03.877370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:03.877370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 01 20:34:03.877370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 01 20:34:03.877370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:03.877370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:03.877370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:34:03.877370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:34:03.877370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:34:03.877370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:34:03.877370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 01 20:34:03.877370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 01 20:34:03.879556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 01 20:34:03.879556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 01 20:34:03.879556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 01 20:34:03.879556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 01 20:34:03.879556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 01 20:34:03.879556 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:03.879556 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 500/965] 10.222.0.17 () {60 vars in 1131 bytes} [Sat Oct 1 20:34:03 2022] POST /identity/v3/users => generated 160 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:03.879556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8cd00-ee86-4e50-b38c-41ff3df377f1 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:03.925349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c026d4-70bf-4e71-9123-c9a143f0e34b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:03.963076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554428e-b5ec-4209-b11b-df274d0f2c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:03.963653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554428e-b5ec-4209-b11b-df274d0f2c2b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:03.963932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554428e-b5ec-4209-b11b-df274d0f2c2b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:03.964210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554428e-b5ec-4209-b11b-df274d0f2c2b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:03.976014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554428e-b5ec-4209-b11b-df274d0f2c2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:34:03.981117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-2554428e-b5ec-4209-b11b-df274d0f2c2b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 01 20:34:03.981117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:03.981117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:03.981117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:03.981117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:03.981117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:03.981117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:03.981117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:03.981117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:03.981117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:03.981117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:03.981117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:03.981117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:34:03.981117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:34:03.981117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:34:03.981117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:34:03.982073 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:34:03.982073 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:34:03.982073 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:34:03.982073 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:34:03.982073 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:03.982073 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:03.982073 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:03.982073 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:03.982073 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:03.982073 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:03.982073 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:03.982073 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:03.982073 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:03.982073 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:03.982073 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:03.982073 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:03.982984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:03.982984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:03.982984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:03.982984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:03.982984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:34:03.982984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:34:03.982984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:34:03.982984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:34:03.982984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:34:03.982984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:34:03.982984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 01 20:34:03.982984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:03.986962 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 466/966] 10.222.0.17 () {56 vars in 1120 bytes} [Sat Oct 1 20:34:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:03.988533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554428e-b5ec-4209-b11b-df274d0f2c2b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:04.019250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8cd00-ee86-4e50-b38c-41ff3df377f1 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:04.038177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:04.073559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d65527-c8ff-4415-bba5-4d8bbdaae3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:04.074362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d65527-c8ff-4415-bba5-4d8bbdaae3ce None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:04.075069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d65527-c8ff-4415-bba5-4d8bbdaae3ce None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:04.075452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d65527-c8ff-4415-bba5-4d8bbdaae3ce None admin] PATH_INFO: `/v3/users/cinder` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:04.086926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d65527-c8ff-4415-bba5-4d8bbdaae3ce None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:04.088946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d65527-c8ff-4415-bba5-4d8bbdaae3ce None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:04.107865 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-68d65527-c8ff-4415-bba5-4d8bbdaae3ce None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 01 20:34:04.107865 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:04.107865 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:04.107865 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:04.107865 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:04.107865 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:04.107865 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:04.107865 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:04.107865 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:04.107865 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:04.107865 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:04.107865 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:04.107865 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:34:04.107865 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:34:04.107865 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:34:04.107865 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:34:04.109178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:04.109178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:04.109178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:34:04.109178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:34:04.109178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:34:04.109178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:34:04.109178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:04.109178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:04.109178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:04.109178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:04.109178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:04.109178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:04.109178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:04.109178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:04.109178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:04.109178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:04.109178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:04.109178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:04.110391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:04.110391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:04.110391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:34:04.110391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:34:04.110391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:34:04.110391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:34:04.110391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:34:04.110391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:34:04.110391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 01 20:34:04.110391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:04.112109 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 501/967] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:34:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:34:04.117468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d65527-c8ff-4415-bba5-4d8bbdaae3ce None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:04.129494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554428e-b5ec-4209-b11b-df274d0f2c2b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:04.165915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60b800-528f-44ce-9511-0bf6005f9103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:04.166897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60b800-528f-44ce-9511-0bf6005f9103 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:04.167358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60b800-528f-44ce-9511-0bf6005f9103 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:04.167809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60b800-528f-44ce-9511-0bf6005f9103 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:04.168706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60b800-528f-44ce-9511-0bf6005f9103 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:04.169219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60b800-528f-44ce-9511-0bf6005f9103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:04.170530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:04.172095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60b800-528f-44ce-9511-0bf6005f9103 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:04.188507 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 467/968] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Oct 1 20:34:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:34:04.191037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60b800-528f-44ce-9511-0bf6005f9103 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:04.234894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d65527-c8ff-4415-bba5-4d8bbdaae3ce None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:04.272777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a04287-eefb-4807-bb5d-62c80ee5c122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:04.273423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a04287-eefb-4807-bb5d-62c80ee5c122 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:04.273666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a04287-eefb-4807-bb5d-62c80ee5c122 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:04.273911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a04287-eefb-4807-bb5d-62c80ee5c122 None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:04.274648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a04287-eefb-4807-bb5d-62c80ee5c122 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:04.274950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a04287-eefb-4807-bb5d-62c80ee5c122 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:04.277306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a04287-eefb-4807-bb5d-62c80ee5c122 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:04.305517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:04.314479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60b800-528f-44ce-9511-0bf6005f9103 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:04.326746 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 502/969] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Oct 1 20:34:04 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:34:04.359192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d24c28-eef9-40aa-881d-a24cc8d4431d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:04.360260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d24c28-eef9-40aa-881d-a24cc8d4431d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:04.360746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d24c28-eef9-40aa-881d-a24cc8d4431d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:04.361220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d24c28-eef9-40aa-881d-a24cc8d4431d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:04.374054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d24c28-eef9-40aa-881d-a24cc8d4431d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:04.376546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d24c28-eef9-40aa-881d-a24cc8d4431d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:04.392700 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-52d24c28-eef9-40aa-881d-a24cc8d4431d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:34:04.392700 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:04.392700 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:04.392700 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:04.392700 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:04.392700 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:04.392700 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:04.392700 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:04.392700 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:04.392700 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:04.392700 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:04.392700 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:04.392700 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:34:04.392700 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:34:04.392700 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:34:04.392700 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:34:04.393732 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:04.393732 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:04.393732 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:04.393732 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:04.393732 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:04.393732 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:04.393732 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:04.393732 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:04.393732 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:04.393732 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:04.393732 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:04.393732 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:04.393732 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:04.393732 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:04.393732 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:04.393732 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:04.393732 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:34:04.394707 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:34:04.394707 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:34:04.394707 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:04.397057 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 468/970] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:34:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:04.403495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a04287-eefb-4807-bb5d-62c80ee5c122 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:04.469129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f1ab7-96c6-46c1-8497-4d3322f4c17e None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:04.469478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f1ab7-96c6-46c1-8497-4d3322f4c17e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:04.469680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f1ab7-96c6-46c1-8497-4d3322f4c17e None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:04.471336 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 469/971] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:04.476905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04335b4-a696-4502-b499-9931fcbcf798 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:04.477205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04335b4-a696-4502-b499-9931fcbcf798 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:04.477445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04335b4-a696-4502-b499-9931fcbcf798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:04.505635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c04335b4-a696-4502-b499-9931fcbcf798 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:04.507299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c04335b4-a696-4502-b499-9931fcbcf798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:04.521906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:04.527627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a04287-eefb-4807-bb5d-62c80ee5c122 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:04.573113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ee4e6-4f73-452a-a265-6d1dabaf3339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:04.573324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ee4e6-4f73-452a-a265-6d1dabaf3339 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:04.573546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ee4e6-4f73-452a-a265-6d1dabaf3339 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:04.573782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ee4e6-4f73-452a-a265-6d1dabaf3339 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:04.574431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1ee4e6-4f73-452a-a265-6d1dabaf3339 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:04.574777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1ee4e6-4f73-452a-a265-6d1dabaf3339 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:04.576129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1ee4e6-4f73-452a-a265-6d1dabaf3339 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:04.591659 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 503/972] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:34:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:04.596848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ee4e6-4f73-452a-a265-6d1dabaf3339 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:04.604289 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 470/973] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 01 20:34:04.717136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ee4e6-4f73-452a-a265-6d1dabaf3339 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:04.752344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3bda1-9de5-4dff-868b-6ab884854e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:04.752966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3bda1-9de5-4dff-868b-6ab884854e40 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:04.753244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3bda1-9de5-4dff-868b-6ab884854e40 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:04.753517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3bda1-9de5-4dff-868b-6ab884854e40 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:04.761149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3bda1-9de5-4dff-868b-6ab884854e40 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:04.762530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3bda1-9de5-4dff-868b-6ab884854e40 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:04.773514 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-cfa3bda1-9de5-4dff-868b-6ab884854e40 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 01 20:34:04.773514 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:04.773514 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:04.773514 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:04.773514 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:04.773514 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:04.773514 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:04.773514 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:04.773514 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:04.773514 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:04.773514 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:04.773514 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:04.773514 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:34:04.773514 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:34:04.773514 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:34:04.773514 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:34:04.774478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:04.774478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:04.774478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:34:04.774478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:34:04.774478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:34:04.774478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:34:04.774478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:04.774478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:04.774478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:04.774478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:04.774478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:04.774478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:04.774478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:04.774478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:04.774478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:04.774478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:04.774478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:04.774478 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:04.775564 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:04.775564 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:04.775564 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:34:04.775564 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:34:04.775564 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:34:04.775564 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:34:04.775564 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:34:04.775564 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:34:04.775564 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 01 20:34:04.775564 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:04.775564 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 504/974] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Oct 1 20:34:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:34:04.780265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04335b4-a696-4502-b499-9931fcbcf798 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:04.859919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:04.882209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04335b4-a696-4502-b499-9931fcbcf798 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:04.917327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57c514-2e9d-44fd-8f29-2b6ec968a49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:04.917931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57c514-2e9d-44fd-8f29-2b6ec968a49c None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:04.918210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57c514-2e9d-44fd-8f29-2b6ec968a49c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:04.918482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57c514-2e9d-44fd-8f29-2b6ec968a49c None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:04.919236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57c514-2e9d-44fd-8f29-2b6ec968a49c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:04.919542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57c514-2e9d-44fd-8f29-2b6ec968a49c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:04.921052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57c514-2e9d-44fd-8f29-2b6ec968a49c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:04.938196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3bda1-9de5-4dff-868b-6ab884854e40 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:04.939460 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 471/975] 10.222.0.17 () {56 vars in 1149 bytes} [Sat Oct 1 20:34:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:04.945979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57c514-2e9d-44fd-8f29-2b6ec968a49c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:04.962423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:05.052441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3bda1-9de5-4dff-868b-6ab884854e40 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:05.058566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57c514-2e9d-44fd-8f29-2b6ec968a49c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:05.089090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54011c8c-4431-4adc-bb9d-57eaf0541907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:05.089700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54011c8c-4431-4adc-bb9d-57eaf0541907 None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:05.089985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54011c8c-4431-4adc-bb9d-57eaf0541907 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:05.090256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54011c8c-4431-4adc-bb9d-57eaf0541907 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:05.090949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54011c8c-4431-4adc-bb9d-57eaf0541907 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:05.092312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54011c8c-4431-4adc-bb9d-57eaf0541907 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:05.093735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cd486-7a03-4c5a-a44c-ebb92570687d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:05.094335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cd486-7a03-4c5a-a44c-ebb92570687d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:05.094653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cd486-7a03-4c5a-a44c-ebb92570687d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:05.094923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cd486-7a03-4c5a-a44c-ebb92570687d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:05.098290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:05.107657 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 505/976] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Oct 1 20:34:04 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:34:05.109507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e293edd-b8e3-4d20-8879-c98e7ccb1ad6 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:05.109798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e293edd-b8e3-4d20-8879-c98e7ccb1ad6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:05.110073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e293edd-b8e3-4d20-8879-c98e7ccb1ad6 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:05.111374 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 506/977] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:05.121110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163dc85a-a0d1-4b77-8153-a54ed00b4e8c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:05.121346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163dc85a-a0d1-4b77-8153-a54ed00b4e8c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:05.121605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163dc85a-a0d1-4b77-8153-a54ed00b4e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:05.142279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-163dc85a-a0d1-4b77-8153-a54ed00b4e8c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:05.143963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-163dc85a-a0d1-4b77-8153-a54ed00b4e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:05.175453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4cd486-7a03-4c5a-a44c-ebb92570687d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:05.177679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4cd486-7a03-4c5a-a44c-ebb92570687d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:05.242519 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 507/978] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:05 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:05.302904 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 472/979] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:34:04 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 01 20:34:05.308671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163dc85a-a0d1-4b77-8153-a54ed00b4e8c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:05.317706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:05.414152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163dc85a-a0d1-4b77-8153-a54ed00b4e8c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:05.450463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae521980-9758-4b50-83c5-ff50a3cd4670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:05.451127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae521980-9758-4b50-83c5-ff50a3cd4670 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:05.451411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae521980-9758-4b50-83c5-ff50a3cd4670 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:05.451687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae521980-9758-4b50-83c5-ff50a3cd4670 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:05.463885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae521980-9758-4b50-83c5-ff50a3cd4670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:05.465722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae521980-9758-4b50-83c5-ff50a3cd4670 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:05.482101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-ae521980-9758-4b50-83c5-ff50a3cd4670 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:34:05.482101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:05.482101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:05.482101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:05.482101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:05.482101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:05.482101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:05.482101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:05.482101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:05.482101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:05.482101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:05.482101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:05.482101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:34:05.482101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:34:05.482101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:34:05.482101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:34:05.483250 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:05.483250 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:05.483250 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:05.483250 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:05.483250 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:05.483250 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:05.483250 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:05.483250 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:05.483250 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:05.483250 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:05.483250 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:05.483250 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:05.483250 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:05.483250 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:05.483250 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:05.483250 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:05.483250 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:34:05.484171 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:34:05.484171 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:34:05.484171 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:05.484171 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 508/980] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:34:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:34:05.489201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cd486-7a03-4c5a-a44c-ebb92570687d None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:05.524609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:05.568033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae521980-9758-4b50-83c5-ff50a3cd4670 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:05.593901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cd486-7a03-4c5a-a44c-ebb92570687d None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:05.630026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f803c-5da5-4620-b708-b15576e514e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:05.630673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f803c-5da5-4620-b708-b15576e514e4 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:05.630955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f803c-5da5-4620-b708-b15576e514e4 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:05.631228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f803c-5da5-4620-b708-b15576e514e4 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:05.631895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f803c-5da5-4620-b708-b15576e514e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:05.632202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f803c-5da5-4620-b708-b15576e514e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:05.633540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f803c-5da5-4620-b708-b15576e514e4 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:05.633790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:05.650392 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 473/981] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:05.651724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f803c-5da5-4620-b708-b15576e514e4 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:05.680435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae521980-9758-4b50-83c5-ff50a3cd4670 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:05.719852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a07f9c6-707b-455b-bcf7-3c5d95dd62de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:05.720948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07f9c6-707b-455b-bcf7-3c5d95dd62de None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:05.721364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07f9c6-707b-455b-bcf7-3c5d95dd62de None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:05.721636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07f9c6-707b-455b-bcf7-3c5d95dd62de None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:05.722224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a07f9c6-707b-455b-bcf7-3c5d95dd62de None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:05.723604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a07f9c6-707b-455b-bcf7-3c5d95dd62de None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:05.733220 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-6a07f9c6-707b-455b-bcf7-3c5d95dd62de None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 01 20:34:05.733220 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:05.733220 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:34:05.733220 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:34:05.733220 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:34:05.733220 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:34:05.733220 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:34:05.733220 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:34:05.733220 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 01 20:34:05.733220 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:05.733220 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:34:05.733220 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:05.733220 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:05.733220 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:05.733220 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:05.733220 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:05.733220 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:05.734573 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:05.734573 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:05.734573 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:05.734573 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:05.734573 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:05.734573 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:05.734573 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:34:05.734573 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:34:05.734573 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:34:05.734573 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:34:05.734573 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:05.734573 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:05.734573 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:05.734573 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:05.734573 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:05.734573 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:05.735664 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:05.735664 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:05.735664 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:05.735664 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:05.735664 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:05.735664 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:05.735664 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:05.735664 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:05.735664 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:05.735664 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:05.735664 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:34:05.735664 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:34:05.735664 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 01 20:34:05.735664 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:05.735664 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 509/982] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Oct 1 20:34:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:34:05.740160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a07f9c6-707b-455b-bcf7-3c5d95dd62de None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:05.766449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:05.773385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f803c-5da5-4620-b708-b15576e514e4 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:05.808790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08d19d-d161-404b-b1fa-ece1881c6482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:05.809648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08d19d-d161-404b-b1fa-ece1881c6482 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:05.809944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08d19d-d161-404b-b1fa-ece1881c6482 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:05.810234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08d19d-d161-404b-b1fa-ece1881c6482 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:05.817638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08d19d-d161-404b-b1fa-ece1881c6482 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:05.819172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08d19d-d161-404b-b1fa-ece1881c6482 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:05.831305 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-ac08d19d-d161-404b-b1fa-ece1881c6482 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:34:05.831305 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:05.831305 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:05.831305 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:05.831305 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:05.831305 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:05.831305 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:05.831305 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:05.831305 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:05.831305 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:05.831305 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:05.831305 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:05.831305 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:34:05.831305 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:34:05.831305 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:34:05.832221 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:34:05.832221 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:05.832221 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:05.832221 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:05.832221 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:05.832221 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:05.832221 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:05.832221 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:05.832221 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:05.832221 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:05.832221 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:05.832221 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:05.832221 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:05.832221 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:05.832221 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:05.832221 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:05.832221 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:05.833119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:34:05.833119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:34:05.833119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:34:05.833119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:34:05.833119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:34:05.833119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:34:05.833119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:34:05.833119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:05.833119 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 474/983] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:34:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:34:05.838194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08d19d-d161-404b-b1fa-ece1881c6482 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:05.857545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a07f9c6-707b-455b-bcf7-3c5d95dd62de None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:05.881686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:05.894890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc133b-7faf-4eec-bb1f-8bc8fedab361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:05.895503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc133b-7faf-4eec-bb1f-8bc8fedab361 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:05.895786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc133b-7faf-4eec-bb1f-8bc8fedab361 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:05.896067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc133b-7faf-4eec-bb1f-8bc8fedab361 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:05.896705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc133b-7faf-4eec-bb1f-8bc8fedab361 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:05.897015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc133b-7faf-4eec-bb1f-8bc8fedab361 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:05.898385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc133b-7faf-4eec-bb1f-8bc8fedab361 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:05.903933 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 510/984] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Oct 1 20:34:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:34:05.910554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc133b-7faf-4eec-bb1f-8bc8fedab361 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:05.950126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08d19d-d161-404b-b1fa-ece1881c6482 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:05.988414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445c706-2690-442c-948d-f703f80ab2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:05.989046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445c706-2690-442c-948d-f703f80ab2f7 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:05.989320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445c706-2690-442c-948d-f703f80ab2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:05.989586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445c706-2690-442c-948d-f703f80ab2f7 None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:05.990319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445c706-2690-442c-948d-f703f80ab2f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:05.990662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445c706-2690-442c-948d-f703f80ab2f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:05.992083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445c706-2690-442c-948d-f703f80ab2f7 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:06.009889 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 475/985] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:34:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:34:06.010936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445c706-2690-442c-948d-f703f80ab2f7 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:06.023879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc133b-7faf-4eec-bb1f-8bc8fedab361 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:06.030619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:06.066281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221402d5-daee-4ee9-8932-f18a255a1335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:06.066934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221402d5-daee-4ee9-8932-f18a255a1335 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:06.067206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221402d5-daee-4ee9-8932-f18a255a1335 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:06.067484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221402d5-daee-4ee9-8932-f18a255a1335 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:06.068105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221402d5-daee-4ee9-8932-f18a255a1335 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:06.068407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221402d5-daee-4ee9-8932-f18a255a1335 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:06.069808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221402d5-daee-4ee9-8932-f18a255a1335 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:06.075058 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 511/986] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Oct 1 20:34:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:34:06.081079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221402d5-daee-4ee9-8932-f18a255a1335 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:06.120453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445c706-2690-442c-948d-f703f80ab2f7 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:06.132406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:06.158328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eaa4a4-7fad-4ec8-b40c-9cdf358e8f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:06.159338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eaa4a4-7fad-4ec8-b40c-9cdf358e8f87 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:06.159831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eaa4a4-7fad-4ec8-b40c-9cdf358e8f87 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:06.160287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eaa4a4-7fad-4ec8-b40c-9cdf358e8f87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:06.161253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eaa4a4-7fad-4ec8-b40c-9cdf358e8f87 None admin] RBAC: Adding query filter params (role.id=fcfc5f73645d423d8475708ef191dacb, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=917fc96fa3504b19b1a56990d4747439) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:06.161767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eaa4a4-7fad-4ec8-b40c-9cdf358e8f87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:06.163456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eaa4a4-7fad-4ec8-b40c-9cdf358e8f87 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:06.169789 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 476/987] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:34:06 2022] GET /identity/v3/role_assignments?user.id=917fc96fa3504b19b1a56990d4747439&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=fcfc5f73645d423d8475708ef191dacb => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:34:06.193565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221402d5-daee-4ee9-8932-f18a255a1335 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:06.238429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f4e34-9750-4662-9b2b-d48d2251e389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:06.239096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f4e34-9750-4662-9b2b-d48d2251e389 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:06.239374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f4e34-9750-4662-9b2b-d48d2251e389 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:06.239644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f4e34-9750-4662-9b2b-d48d2251e389 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:06.240377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f4e34-9750-4662-9b2b-d48d2251e389 None admin] RBAC: Adding query filter params (interface=admin, service_id=2cdb69f5f1cb4ffabd171e2bbc67a7de, region_id=RegionOne) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:06.240687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f4e34-9750-4662-9b2b-d48d2251e389 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:06.242170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f4e34-9750-4662-9b2b-d48d2251e389 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:06.254863 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 512/988] 10.222.0.17 () {56 vars in 1249 bytes} [Sat Oct 1 20:34:06 2022] GET /identity/v3/endpoints?service_id=2cdb69f5f1cb4ffabd171e2bbc67a7de&interface=admin®ion_id=RegionOne => generated 193 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:34:06.257253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eaa4a4-7fad-4ec8-b40c-9cdf358e8f87 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:06.348060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:06.371164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eaa4a4-7fad-4ec8-b40c-9cdf358e8f87 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:06.408916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4d454-384e-481d-8f88-079fb3781856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:06.409528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4d454-384e-481d-8f88-079fb3781856 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:06.409809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4d454-384e-481d-8f88-079fb3781856 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:06.410082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4d454-384e-481d-8f88-079fb3781856 None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:06.410792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4d454-384e-481d-8f88-079fb3781856 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:06.412167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4d454-384e-481d-8f88-079fb3781856 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:06.418595 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 477/989] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Oct 1 20:34:06 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:34:07.575389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47fe0d-0576-4e3a-a6d5-1d69006847cf None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:07.576021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47fe0d-0576-4e3a-a6d5-1d69006847cf None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:07.576021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47fe0d-0576-4e3a-a6d5-1d69006847cf None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:07.577817 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 513/990] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:07.583563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f03db1-c125-499c-b71c-2440076db189 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:07.584082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f03db1-c125-499c-b71c-2440076db189 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:07.584170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f03db1-c125-499c-b71c-2440076db189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:07.603362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c9f03db1-c125-499c-b71c-2440076db189 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:07.604651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c9f03db1-c125-499c-b71c-2440076db189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:07.678965 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 478/991] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:07 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:08.002206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47fe0d-0576-4e3a-a6d5-1d69006847cf None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:08.113877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47fe0d-0576-4e3a-a6d5-1d69006847cf None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:08.128667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:08.149694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e62cd2-750b-4e2e-b84d-ad7386e93a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:08.150316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e62cd2-750b-4e2e-b84d-ad7386e93a90 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:08.150625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e62cd2-750b-4e2e-b84d-ad7386e93a90 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:08.150898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e62cd2-750b-4e2e-b84d-ad7386e93a90 None admin] PATH_INFO: `/v3/services/compute` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:08.151476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e62cd2-750b-4e2e-b84d-ad7386e93a90 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:08.152832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e62cd2-750b-4e2e-b84d-ad7386e93a90 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:08.161597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-e0e62cd2-750b-4e2e-b84d-ad7386e93a90 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 01 20:34:08.161597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:08.161597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:34:08.161597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:34:08.161597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:34:08.161597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:34:08.161597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:34:08.161597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:34:08.161597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 01 20:34:08.161597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:08.161597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:34:08.161597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:08.161597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:08.161597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:08.161597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:08.161597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:08.161597 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:08.162701 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:08.162701 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:08.162701 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:08.162701 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:08.162701 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:08.162701 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:08.162701 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:34:08.162701 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:34:08.162701 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:34:08.162701 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:34:08.162701 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:08.162701 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:08.162701 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:08.162701 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:08.162701 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:08.162701 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:08.163707 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:08.163707 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:08.163707 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:08.163707 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:08.163707 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:08.163707 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:08.163707 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:08.163707 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:08.163707 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:08.163707 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:08.163707 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:34:08.163707 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:34:08.163707 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 01 20:34:08.163707 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:08.165185 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 514/992] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:34:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:08.168551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f03db1-c125-499c-b71c-2440076db189 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:08.247790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:08.273931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f03db1-c125-499c-b71c-2440076db189 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:08.313209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e90f6-aeff-4a81-8f4c-29c1d06ceb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:08.313833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e90f6-aeff-4a81-8f4c-29c1d06ceb11 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:08.314116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e90f6-aeff-4a81-8f4c-29c1d06ceb11 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:08.314392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e90f6-aeff-4a81-8f4c-29c1d06ceb11 None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:08.315072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e90f6-aeff-4a81-8f4c-29c1d06ceb11 None admin] RBAC: Adding query filter params (name=compute) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:08.315381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e90f6-aeff-4a81-8f4c-29c1d06ceb11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:08.316773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e90f6-aeff-4a81-8f4c-29c1d06ceb11 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:08.327296 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 479/993] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Oct 1 20:34:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:08.328286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e62cd2-750b-4e2e-b84d-ad7386e93a90 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:08.432936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e62cd2-750b-4e2e-b84d-ad7386e93a90 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:08.470728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066a808-0bf3-460a-815c-513de466c4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:08.471334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066a808-0bf3-460a-815c-513de466c4cf None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:08.471623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066a808-0bf3-460a-815c-513de466c4cf None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:08.471900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066a808-0bf3-460a-815c-513de466c4cf None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:08.472530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066a808-0bf3-460a-815c-513de466c4cf None admin] RBAC: Adding query filter params (type=compute) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:08.472837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066a808-0bf3-460a-815c-513de466c4cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:08.474218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066a808-0bf3-460a-815c-513de466c4cf None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:08.484439 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 515/994] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Oct 1 20:34:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:08.485452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e90f6-aeff-4a81-8f4c-29c1d06ceb11 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:08.586546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e90f6-aeff-4a81-8f4c-29c1d06ceb11 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:08.622037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb59e4-6f20-4073-ac18-8b161d26c65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:08.622685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb59e4-6f20-4073-ac18-8b161d26c65a None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:08.622970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb59e4-6f20-4073-ac18-8b161d26c65a None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:08.623242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb59e4-6f20-4073-ac18-8b161d26c65a None admin] PATH_INFO: `/v3/endpoints` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:08.623940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb59e4-6f20-4073-ac18-8b161d26c65a None admin] RBAC: Adding query filter params (interface=public, service_id=8707fb62b1134ee18c5e769447e378b3, region_id=RegionOne) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:08.624298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb59e4-6f20-4073-ac18-8b161d26c65a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:08.625662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb59e4-6f20-4073-ac18-8b161d26c65a None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:08.637334 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 480/995] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Oct 1 20:34:08 2022] GET /identity/v3/endpoints?service_id=8707fb62b1134ee18c5e769447e378b3&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:34:08.637514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066a808-0bf3-460a-815c-513de466c4cf None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:08.681602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:08.720044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7ab6b-c004-47fc-a373-8810d4aad161 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:08.720326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7ab6b-c004-47fc-a373-8810d4aad161 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:08.720600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7ab6b-c004-47fc-a373-8810d4aad161 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:08.722071 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 481/996] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:34:08.726838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6decc-232b-4a15-8cb0-7b8f1b257585 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:08.727122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6decc-232b-4a15-8cb0-7b8f1b257585 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:08.727395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6decc-232b-4a15-8cb0-7b8f1b257585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:08.734287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:08.747849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7df6decc-232b-4a15-8cb0-7b8f1b257585 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:08.749036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7df6decc-232b-4a15-8cb0-7b8f1b257585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:08.751486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066a808-0bf3-460a-815c-513de466c4cf None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:08.789700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d9121-8b94-4fb9-9bdd-694709dfe614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:08.790892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d9121-8b94-4fb9-9bdd-694709dfe614 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:08.791175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d9121-8b94-4fb9-9bdd-694709dfe614 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:08.791446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d9121-8b94-4fb9-9bdd-694709dfe614 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:08.792047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d9121-8b94-4fb9-9bdd-694709dfe614 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:08.794262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d9121-8b94-4fb9-9bdd-694709dfe614 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:08.810537 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 516/997] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Oct 1 20:34:08 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:34:08.824393 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 482/998] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:08 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:08.865241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1f31e-b989-4a49-bc44-27e33bee0611 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:08.865442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1f31e-b989-4a49-bc44-27e33bee0611 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:08.865716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1f31e-b989-4a49-bc44-27e33bee0611 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:08.867327 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 517/999] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:34:08.873208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c53b2a-2f47-4ee4-9f58-9bb54ed551b9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:08.873527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c53b2a-2f47-4ee4-9f58-9bb54ed551b9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:08.873756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c53b2a-2f47-4ee4-9f58-9bb54ed551b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:08.892797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-25c53b2a-2f47-4ee4-9f58-9bb54ed551b9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:08.894025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-25c53b2a-2f47-4ee4-9f58-9bb54ed551b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:08.968786 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 483/1000] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:08 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:09.148753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1f31e-b989-4a49-bc44-27e33bee0611 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:09.257008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1f31e-b989-4a49-bc44-27e33bee0611 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:09.264620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:09.292600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7571a9-0f7f-411d-85e8-a97e7df074af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:09.293225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7571a9-0f7f-411d-85e8-a97e7df074af None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:09.293499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7571a9-0f7f-411d-85e8-a97e7df074af None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:09.293765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7571a9-0f7f-411d-85e8-a97e7df074af None admin] PATH_INFO: `/v3/services/image` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:09.294406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7571a9-0f7f-411d-85e8-a97e7df074af None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:09.295363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c53b2a-2f47-4ee4-9f58-9bb54ed551b9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:09.295785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7571a9-0f7f-411d-85e8-a97e7df074af None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:09.305985 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-0a7571a9-0f7f-411d-85e8-a97e7df074af None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 01 20:34:09.305985 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:09.305985 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:34:09.305985 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:34:09.305985 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:34:09.305985 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:34:09.305985 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:34:09.305985 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:34:09.305985 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 01 20:34:09.305985 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:09.305985 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:34:09.305985 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:09.305985 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:09.305985 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:09.305985 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:09.305985 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:09.305985 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:09.307094 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:09.307094 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:09.307094 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:09.307094 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:09.307094 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:09.307094 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:09.307094 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:34:09.307094 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:34:09.307094 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:34:09.307094 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:34:09.307094 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:09.307094 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:09.307094 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:09.307094 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:09.307094 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:09.307094 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:09.308288 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:09.308288 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:09.308288 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:09.308288 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:09.308288 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:09.308288 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:09.308288 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:09.308288 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:09.308288 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:09.308288 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:09.308288 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:34:09.308288 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:34:09.308288 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 01 20:34:09.308288 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:09.308288 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 518/1001] 10.222.0.17 () {56 vars in 1105 bytes} [Sat Oct 1 20:34:09 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:09.312927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7571a9-0f7f-411d-85e8-a97e7df074af None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:09.364854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:09.405973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c53b2a-2f47-4ee4-9f58-9bb54ed551b9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:09.415659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7571a9-0f7f-411d-85e8-a97e7df074af None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:09.442862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72368f8-4dab-4495-afd0-aff2d5422f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:09.443774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72368f8-4dab-4495-afd0-aff2d5422f9a None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:09.444276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72368f8-4dab-4495-afd0-aff2d5422f9a None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:09.444744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72368f8-4dab-4495-afd0-aff2d5422f9a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:09.445523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72368f8-4dab-4495-afd0-aff2d5422f9a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:09.447361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72368f8-4dab-4495-afd0-aff2d5422f9a None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:09.450131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e58b5-1af2-40b0-8433-1f046e476790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:09.450763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e58b5-1af2-40b0-8433-1f046e476790 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:09.451041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e58b5-1af2-40b0-8433-1f046e476790 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:09.451313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e58b5-1af2-40b0-8433-1f046e476790 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:09.451934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e58b5-1af2-40b0-8433-1f046e476790 None admin] RBAC: Adding query filter params (name=image) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:09.452856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e58b5-1af2-40b0-8433-1f046e476790 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:09.454189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e58b5-1af2-40b0-8433-1f046e476790 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:09.458195 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-b72368f8-4dab-4495-afd0-aff2d5422f9a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 01 20:34:09.458195 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:09.458195 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:34:09.458195 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:34:09.458195 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:34:09.458195 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:34:09.458195 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:34:09.458195 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:34:09.458195 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 01 20:34:09.458195 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:09.458195 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:34:09.458195 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:09.458195 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:09.458195 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:09.458195 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:09.458195 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:09.458195 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:09.459474 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:09.459474 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:09.459474 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:09.459474 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:09.459474 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:09.459474 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:09.459474 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:34:09.459474 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:34:09.459474 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:34:09.459474 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:34:09.459474 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:09.459474 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:09.459474 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:09.459474 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:09.459474 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:09.459474 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:09.460613 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:09.460613 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:09.460613 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:09.460613 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:09.460613 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:09.460613 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:09.460613 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:09.460613 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:09.460613 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:09.460613 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:09.460613 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:34:09.460613 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:34:09.460613 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 01 20:34:09.460613 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:09.461555 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 519/1002] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:34:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:34:09.468055 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 484/1003] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Oct 1 20:34:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:34:09.468237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e58b5-1af2-40b0-8433-1f046e476790 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:09.469527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72368f8-4dab-4495-afd0-aff2d5422f9a None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:09.504471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:09.576370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72368f8-4dab-4495-afd0-aff2d5422f9a None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:09.579972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e58b5-1af2-40b0-8433-1f046e476790 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:09.604798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:09.613814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14606d2-d935-4c98-b23d-6e7b836478ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:09.614419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14606d2-d935-4c98-b23d-6e7b836478ed None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:09.614729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14606d2-d935-4c98-b23d-6e7b836478ed None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:09.615003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14606d2-d935-4c98-b23d-6e7b836478ed None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:09.615633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14606d2-d935-4c98-b23d-6e7b836478ed None admin] RBAC: Adding query filter params (type=image) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:09.615846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fdf36-450c-4f01-be71-e91dcb36c937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:09.615971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14606d2-d935-4c98-b23d-6e7b836478ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:09.616720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fdf36-450c-4f01-be71-e91dcb36c937 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:09.617234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fdf36-450c-4f01-be71-e91dcb36c937 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:09.617350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14606d2-d935-4c98-b23d-6e7b836478ed None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:09.617762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fdf36-450c-4f01-be71-e91dcb36c937 None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:09.618768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6fdf36-450c-4f01-be71-e91dcb36c937 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:09.619402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6fdf36-450c-4f01-be71-e91dcb36c937 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:09.620930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6fdf36-450c-4f01-be71-e91dcb36c937 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:09.621612 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 520/1004] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:34:09 2022] GET /identity/v3/services?type=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:34:09.626783 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 485/1005] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Oct 1 20:34:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:34:09.632531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14606d2-d935-4c98-b23d-6e7b836478ed None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:09.742707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:09.747253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14606d2-d935-4c98-b23d-6e7b836478ed None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:09.784689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767293fa-f108-4244-af51-220a045fd226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:09.785311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767293fa-f108-4244-af51-220a045fd226 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:09.785589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767293fa-f108-4244-af51-220a045fd226 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:09.785858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767293fa-f108-4244-af51-220a045fd226 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:09.786575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767293fa-f108-4244-af51-220a045fd226 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:09.786880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767293fa-f108-4244-af51-220a045fd226 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:09.788243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767293fa-f108-4244-af51-220a045fd226 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:09.793718 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 521/1006] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Oct 1 20:34:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:09.800141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fdf36-450c-4f01-be71-e91dcb36c937 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:09.906460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fdf36-450c-4f01-be71-e91dcb36c937 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:09.942573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b5c5c-9534-445b-9699-36066f56d4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:09.942573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b5c5c-9534-445b-9699-36066f56d4b2 None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:09.942860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b5c5c-9534-445b-9699-36066f56d4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:09.943129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b5c5c-9534-445b-9699-36066f56d4b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:09.943869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b5c5c-9534-445b-9699-36066f56d4b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:09.946115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b5c5c-9534-445b-9699-36066f56d4b2 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:09.972940 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 486/1007] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:34:09 2022] POST /identity/v3/endpoints => generated 325 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:34:11.293709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324569b1-31ff-44e6-8921-d6fa222b98f6 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:11.294349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324569b1-31ff-44e6-8921-d6fa222b98f6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:11.294349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324569b1-31ff-44e6-8921-d6fa222b98f6 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:11.295782 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 522/1008] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:11.301484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79fca2-d18e-41e5-b9f9-3600c235d3b3 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:11.301903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79fca2-d18e-41e5-b9f9-3600c235d3b3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:11.302082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79fca2-d18e-41e5-b9f9-3600c235d3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:11.323140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fd79fca2-d18e-41e5-b9f9-3600c235d3b3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:11.324337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fd79fca2-d18e-41e5-b9f9-3600c235d3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:11.376239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:11.407282 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 487/1009] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:11 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:11.755850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324569b1-31ff-44e6-8921-d6fa222b98f6 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:11.863932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324569b1-31ff-44e6-8921-d6fa222b98f6 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:11.898317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f07f6-72dd-4668-bc69-fe280289374f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:11.898977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f07f6-72dd-4668-bc69-fe280289374f None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:11.899256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f07f6-72dd-4668-bc69-fe280289374f None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:11.899529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f07f6-72dd-4668-bc69-fe280289374f None admin] PATH_INFO: `/v3/services/compute` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:11.900229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f07f6-72dd-4668-bc69-fe280289374f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:11.901588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f07f6-72dd-4668-bc69-fe280289374f None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:11.910376 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-908f07f6-72dd-4668-bc69-fe280289374f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 01 20:34:11.910376 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:11.910376 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:34:11.910376 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:34:11.910376 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:34:11.910376 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:34:11.910376 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:34:11.910376 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:34:11.910376 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 01 20:34:11.910376 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:11.910376 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:34:11.910376 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:11.910376 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:11.910376 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:11.910376 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:11.910376 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:11.910376 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:11.911389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:11.911389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:11.911389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:11.911389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:11.911389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:11.911389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:11.911389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:34:11.911389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:34:11.911389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:34:11.911389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:34:11.911389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:11.911389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:11.911389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:11.911389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:11.911389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:11.911389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:11.912257 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:11.912257 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:11.912257 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:11.912257 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:11.912257 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:11.912257 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:11.912257 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:11.912257 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:11.912257 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:11.912257 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:11.912257 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:34:11.912257 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:34:11.912257 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 01 20:34:11.912257 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:11.914840 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 523/1010] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:34:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:11.920970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79fca2-d18e-41e5-b9f9-3600c235d3b3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:12.039331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79fca2-d18e-41e5-b9f9-3600c235d3b3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:12.077519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf292d-1afc-4784-a04c-94833af8f280 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:12.077735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec088379-5c9d-450c-963f-078bbb49ba3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:12.077836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf292d-1afc-4784-a04c-94833af8f280 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:12.078090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf292d-1afc-4784-a04c-94833af8f280 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:12.078284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec088379-5c9d-450c-963f-078bbb49ba3c None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:12.078592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec088379-5c9d-450c-963f-078bbb49ba3c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:12.078865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec088379-5c9d-450c-963f-078bbb49ba3c None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:12.079555 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 524/1011] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:34:12.079666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec088379-5c9d-450c-963f-078bbb49ba3c None admin] RBAC: Adding query filter params (name=compute) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:12.079752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec088379-5c9d-450c-963f-078bbb49ba3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:12.081119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec088379-5c9d-450c-963f-078bbb49ba3c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:12.084574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42d664-7eff-4585-8f9c-499a4faad1aa None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:12.084914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42d664-7eff-4585-8f9c-499a4faad1aa None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:12.085237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42d664-7eff-4585-8f9c-499a4faad1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:12.090585 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 488/1012] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Oct 1 20:34:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:12.091417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec088379-5c9d-450c-963f-078bbb49ba3c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:12.105726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-cc42d664-7eff-4585-8f9c-499a4faad1aa None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:12.106967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cc42d664-7eff-4585-8f9c-499a4faad1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:12.128870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:12.137941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:12.180624 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 525/1013] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:12 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:12.197709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec088379-5c9d-450c-963f-078bbb49ba3c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:12.232022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c8da8-0bce-485f-bb93-0ed04090028e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:12.232624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c8da8-0bce-485f-bb93-0ed04090028e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:12.232900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c8da8-0bce-485f-bb93-0ed04090028e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:12.233170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c8da8-0bce-485f-bb93-0ed04090028e None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:12.233822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c8da8-0bce-485f-bb93-0ed04090028e None admin] RBAC: Adding query filter params (type=compute) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:12.234122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c8da8-0bce-485f-bb93-0ed04090028e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:12.235775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c8da8-0bce-485f-bb93-0ed04090028e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:12.245472 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 489/1014] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Oct 1 20:34:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:12.246259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42d664-7eff-4585-8f9c-499a4faad1aa None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:12.349812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42d664-7eff-4585-8f9c-499a4faad1aa None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:12.384323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c4e6a-368b-4ba5-89b3-35881afff4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:12.384943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c4e6a-368b-4ba5-89b3-35881afff4c9 None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:12.385203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c4e6a-368b-4ba5-89b3-35881afff4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:12.385468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c4e6a-368b-4ba5-89b3-35881afff4c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:12.386114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c4e6a-368b-4ba5-89b3-35881afff4c9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:12.387498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c4e6a-368b-4ba5-89b3-35881afff4c9 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:12.410524 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 526/1015] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:34:12 2022] POST /identity/v3/endpoints => generated 334 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:34:12.490889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807a4ef-3e3d-4168-bfbd-258e22f68d49 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:12.491476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807a4ef-3e3d-4168-bfbd-258e22f68d49 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:12.491476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807a4ef-3e3d-4168-bfbd-258e22f68d49 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:12.492826 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 490/1016] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:12.498868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5435c-aa5a-4e9c-8d7b-1ba5e7034ac0 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:12.499699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5435c-aa5a-4e9c-8d7b-1ba5e7034ac0 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:12.500022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5435c-aa5a-4e9c-8d7b-1ba5e7034ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:12.511730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807a4ef-3e3d-4168-bfbd-258e22f68d49 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:12.520386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-78f5435c-aa5a-4e9c-8d7b-1ba5e7034ac0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:12.521607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-78f5435c-aa5a-4e9c-8d7b-1ba5e7034ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:12.530716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:12.614606 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 527/1017] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:12 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:12.646741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807a4ef-3e3d-4168-bfbd-258e22f68d49 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:12.649137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:12.682849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373b58b-1ca4-4239-af50-b7561a473a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:12.683457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373b58b-1ca4-4239-af50-b7561a473a07 None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:12.683800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373b58b-1ca4-4239-af50-b7561a473a07 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:12.684073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373b58b-1ca4-4239-af50-b7561a473a07 None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:12.684716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373b58b-1ca4-4239-af50-b7561a473a07 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:12.686153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373b58b-1ca4-4239-af50-b7561a473a07 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:12.703685 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 491/1018] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Oct 1 20:34:12 2022] POST /identity/v3/services => generated 242 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:34:12.938004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5435c-aa5a-4e9c-8d7b-1ba5e7034ac0 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:13.051049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5435c-aa5a-4e9c-8d7b-1ba5e7034ac0 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:13.094376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf65d7-3669-4848-aa0a-51d2d65a477a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:13.095047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf65d7-3669-4848-aa0a-51d2d65a477a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:13.095329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf65d7-3669-4848-aa0a-51d2d65a477a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:13.095602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf65d7-3669-4848-aa0a-51d2d65a477a None admin] PATH_INFO: `/v3/domains/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:13.096701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf65d7-3669-4848-aa0a-51d2d65a477a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:13.098119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf65d7-3669-4848-aa0a-51d2d65a477a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:13.099933 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 528/1019] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:34:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:13.105408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373b58b-1ca4-4239-af50-b7561a473a07 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:13.208529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373b58b-1ca4-4239-af50-b7561a473a07 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:13.241982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f78ae-327d-44e7-b01e-cbd2a5560893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:13.242608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f78ae-327d-44e7-b01e-cbd2a5560893 None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:13.242886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f78ae-327d-44e7-b01e-cbd2a5560893 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:13.243153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f78ae-327d-44e7-b01e-cbd2a5560893 None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:13.243802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f78ae-327d-44e7-b01e-cbd2a5560893 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:13.245227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f78ae-327d-44e7-b01e-cbd2a5560893 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:13.284843 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 492/1020] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Oct 1 20:34:13 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:34:15.023528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdf722-dbfd-4b00-b7ca-45e19dada8fb None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:15.024081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdf722-dbfd-4b00-b7ca-45e19dada8fb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:15.024081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdf722-dbfd-4b00-b7ca-45e19dada8fb None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:15.028161 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 529/1021] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:15.031337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a180da-501f-4874-9691-67b50af6b9dd None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:15.031758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a180da-501f-4874-9691-67b50af6b9dd None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:15.031924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a180da-501f-4874-9691-67b50af6b9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:15.053475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-84a180da-501f-4874-9691-67b50af6b9dd None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:15.054701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-84a180da-501f-4874-9691-67b50af6b9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:15.066175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:15.128026 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 493/1022] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:15.188663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086afc0d-276e-4556-beb3-a174ec2eba03 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:15.188988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086afc0d-276e-4556-beb3-a174ec2eba03 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:15.189213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086afc0d-276e-4556-beb3-a174ec2eba03 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:15.190728 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 530/1023] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:15.196496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3289ba-ce35-4ee4-9245-d4fdb5db10fa None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:15.196927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3289ba-ce35-4ee4-9245-d4fdb5db10fa None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:15.197086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3289ba-ce35-4ee4-9245-d4fdb5db10fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:15.216181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7d3289ba-ce35-4ee4-9245-d4fdb5db10fa None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:15.217358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7d3289ba-ce35-4ee4-9245-d4fdb5db10fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:15.290141 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 494/1024] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:15.455306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086afc0d-276e-4556-beb3-a174ec2eba03 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:15.458216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:15.559892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086afc0d-276e-4556-beb3-a174ec2eba03 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:15.594578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fbb32b-d78d-4e4c-bb4a-ef38374aab4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:15.595207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fbb32b-d78d-4e4c-bb4a-ef38374aab4e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:15.595489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fbb32b-d78d-4e4c-bb4a-ef38374aab4e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:15.595766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fbb32b-d78d-4e4c-bb4a-ef38374aab4e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:15.605900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fbb32b-d78d-4e4c-bb4a-ef38374aab4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:34:15.610778 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-e8fbb32b-d78d-4e4c-bb4a-ef38374aab4e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 01 20:34:15.610778 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:15.610778 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:15.610778 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:15.610778 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:15.610778 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:15.610778 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:15.610778 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:15.610778 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:15.610778 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:15.610778 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:15.610778 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:15.610778 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:34:15.610778 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:34:15.610778 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:34:15.610778 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:34:15.611731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:34:15.611731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:34:15.611731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:34:15.611731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:34:15.611731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:15.611731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:15.611731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:15.611731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:15.611731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:15.611731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:15.611731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:15.611731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:15.611731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:15.611731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:15.611731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:15.611731 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:15.612604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:15.612604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:15.612604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:15.612604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:15.612604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:34:15.612604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:34:15.612604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:34:15.612604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:34:15.612604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:34:15.612604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:34:15.612604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 01 20:34:15.612604 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:15.614789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3289ba-ce35-4ee4-9245-d4fdb5db10fa None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:15.614914 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 531/1025] 10.222.0.17 () {56 vars in 1120 bytes} [Sat Oct 1 20:34:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:15.617842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fbb32b-d78d-4e4c-bb4a-ef38374aab4e None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:15.682658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:15.697126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:15.722848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3289ba-ce35-4ee4-9245-d4fdb5db10fa None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:15.724339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fbb32b-d78d-4e4c-bb4a-ef38374aab4e None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:15.758328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b9658-6ce0-4ca2-8243-47114162247a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:15.758819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447cf27-d42b-47b2-8414-6cf264e74b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:15.758987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b9658-6ce0-4ca2-8243-47114162247a None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:15.759272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b9658-6ce0-4ca2-8243-47114162247a None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:15.759547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b9658-6ce0-4ca2-8243-47114162247a None admin] PATH_INFO: `/v3/services/image` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:15.760027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447cf27-d42b-47b2-8414-6cf264e74b0a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:15.760151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b9658-6ce0-4ca2-8243-47114162247a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:15.760627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447cf27-d42b-47b2-8414-6cf264e74b0a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:15.761048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447cf27-d42b-47b2-8414-6cf264e74b0a None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:15.761492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b9658-6ce0-4ca2-8243-47114162247a None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:15.762045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6447cf27-d42b-47b2-8414-6cf264e74b0a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:15.762587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6447cf27-d42b-47b2-8414-6cf264e74b0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:15.764081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6447cf27-d42b-47b2-8414-6cf264e74b0a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:15.771123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-9f1b9658-6ce0-4ca2-8243-47114162247a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 01 20:34:15.771123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:15.771123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:34:15.771123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:34:15.771123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:34:15.771123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:34:15.771123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:34:15.771123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:34:15.771123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 01 20:34:15.771123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:15.771123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:34:15.771123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:15.771123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:15.771123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:15.771123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:15.771123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:15.771123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:15.772119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:15.772119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:15.772119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:15.772119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:15.772119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:15.772119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:15.772119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:34:15.772119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:34:15.772119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:34:15.772119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:34:15.772119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:15.772119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:15.772119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:15.772119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:15.772119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:15.772119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:15.772960 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:15.772960 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:15.772960 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:15.772960 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:15.772960 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:15.772960 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:15.772960 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:15.772960 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:15.772960 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:15.772960 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:15.772960 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:34:15.772960 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:34:15.772960 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 01 20:34:15.772960 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:15.772960 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 495/1026] 10.222.0.17 () {56 vars in 1105 bytes} [Sat Oct 1 20:34:15 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:15.773771 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 532/1027] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Oct 1 20:34:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:34:15.775007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288b5cf-2b90-4e7f-b3cf-c8a8c71115c3 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:15.775297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288b5cf-2b90-4e7f-b3cf-c8a8c71115c3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:15.775570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288b5cf-2b90-4e7f-b3cf-c8a8c71115c3 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:15.776853 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 496/1028] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:15.778310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447cf27-d42b-47b2-8414-6cf264e74b0a None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:15.778812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288b5cf-2b90-4e7f-b3cf-c8a8c71115c3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:15.879356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447cf27-d42b-47b2-8414-6cf264e74b0a None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:15.881252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288b5cf-2b90-4e7f-b3cf-c8a8c71115c3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:15.902828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:15.914752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d9eee-1fd6-405d-8111-d7eaffc8d59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:15.914937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6333ad-eaed-4029-9976-29068f9825db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:15.915379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6333ad-eaed-4029-9976-29068f9825db None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:15.915490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d9eee-1fd6-405d-8111-d7eaffc8d59b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:15.915653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d9eee-1fd6-405d-8111-d7eaffc8d59b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:15.915747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6333ad-eaed-4029-9976-29068f9825db None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:15.915924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6333ad-eaed-4029-9976-29068f9825db None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:15.916013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d9eee-1fd6-405d-8111-d7eaffc8d59b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:15.916554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6333ad-eaed-4029-9976-29068f9825db None admin] RBAC: Adding query filter params (name=image) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:15.916860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6333ad-eaed-4029-9976-29068f9825db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:15.918214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6333ad-eaed-4029-9976-29068f9825db None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:15.923455 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 533/1029] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:34:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:15.924573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bafbf7-0198-499d-ab99-9b063ea9539d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:15.924857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bafbf7-0198-499d-ab99-9b063ea9539d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:15.925133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bafbf7-0198-499d-ab99-9b063ea9539d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:15.929852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:15.932116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d9eee-1fd6-405d-8111-d7eaffc8d59b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:15.934113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d9eee-1fd6-405d-8111-d7eaffc8d59b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:15.943804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f0bafbf7-0198-499d-ab99-9b063ea9539d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:15.945238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f0bafbf7-0198-499d-ab99-9b063ea9539d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:15.949768 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-0d6d9eee-1fd6-405d-8111-d7eaffc8d59b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:34:15.949768 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:15.949768 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:15.949768 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:15.949768 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:15.949768 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:15.949768 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:15.949768 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:15.949768 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:15.949768 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:15.949768 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:15.949768 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:15.949768 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:34:15.949768 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:34:15.949768 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:34:15.949768 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:34:15.950871 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:15.950871 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:15.950871 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:15.950871 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:15.950871 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:15.950871 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:15.950871 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:15.950871 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:15.950871 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:15.950871 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:15.950871 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:15.950871 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:15.950871 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:15.950871 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:15.950871 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:15.950871 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:15.950871 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:34:15.952400 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:34:15.952400 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:34:15.952400 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:15.952400 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 497/1030] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:34:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:34:15.953075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d9eee-1fd6-405d-8111-d7eaffc8d59b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:16.012651 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 534/1031] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:16.013481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bafbf7-0198-499d-ab99-9b063ea9539d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:16.066111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d9eee-1fd6-405d-8111-d7eaffc8d59b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:16.102211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98717a83-5268-4b25-b70b-9dffdd2d17f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:16.102852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98717a83-5268-4b25-b70b-9dffdd2d17f3 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:16.103130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98717a83-5268-4b25-b70b-9dffdd2d17f3 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:16.103400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98717a83-5268-4b25-b70b-9dffdd2d17f3 None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:16.104027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98717a83-5268-4b25-b70b-9dffdd2d17f3 None admin] RBAC: Adding query filter params (type=image) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:16.104332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98717a83-5268-4b25-b70b-9dffdd2d17f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:16.105693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98717a83-5268-4b25-b70b-9dffdd2d17f3 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:16.110641 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 498/1032] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:34:15 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:16.116212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98717a83-5268-4b25-b70b-9dffdd2d17f3 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:16.120356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bafbf7-0198-499d-ab99-9b063ea9539d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:16.138760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:16.157565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5175f85-e73a-437b-a5ad-e18f039c73ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:16.158173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5175f85-e73a-437b-a5ad-e18f039c73ac None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:16.158457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5175f85-e73a-437b-a5ad-e18f039c73ac None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:16.158753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5175f85-e73a-437b-a5ad-e18f039c73ac None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:16.159664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5175f85-e73a-437b-a5ad-e18f039c73ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:16.159664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5175f85-e73a-437b-a5ad-e18f039c73ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:16.161039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5175f85-e73a-437b-a5ad-e18f039c73ac None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:16.173260 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 535/1033] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:34:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:16.177412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:16.178840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5175f85-e73a-437b-a5ad-e18f039c73ac None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:16.224451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98717a83-5268-4b25-b70b-9dffdd2d17f3 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:16.260531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5eda60-d7be-417e-b5fa-ed2961f2950e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:16.261143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5eda60-d7be-417e-b5fa-ed2961f2950e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:16.261422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5eda60-d7be-417e-b5fa-ed2961f2950e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:16.261703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5eda60-d7be-417e-b5fa-ed2961f2950e None admin] PATH_INFO: `/v3/endpoints` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:16.262401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5eda60-d7be-417e-b5fa-ed2961f2950e None admin] RBAC: Adding query filter params (interface=public, service_id=6fbf2679e0bc41b3bc29411391451fbc, region_id=RegionOne) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:16.262742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5eda60-d7be-417e-b5fa-ed2961f2950e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:16.264110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5eda60-d7be-417e-b5fa-ed2961f2950e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:16.269559 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 499/1034] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Oct 1 20:34:16 2022] GET /identity/v3/endpoints?service_id=6fbf2679e0bc41b3bc29411391451fbc&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:16.275357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5eda60-d7be-417e-b5fa-ed2961f2950e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:16.292851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5175f85-e73a-437b-a5ad-e18f039c73ac None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:16.327357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bc05f-0fac-436a-bc90-cdf49d51db35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:16.327980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bc05f-0fac-436a-bc90-cdf49d51db35 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:16.328260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bc05f-0fac-436a-bc90-cdf49d51db35 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:16.328530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bc05f-0fac-436a-bc90-cdf49d51db35 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:16.336761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bc05f-0fac-436a-bc90-cdf49d51db35 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:16.338217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bc05f-0fac-436a-bc90-cdf49d51db35 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:16.350774 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-3b0bc05f-0fac-436a-bc90-cdf49d51db35 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 01 20:34:16.350774 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:16.350774 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:16.350774 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:16.350774 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:16.350774 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:16.350774 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:16.350774 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:16.350774 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:16.350774 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:16.350774 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:16.350774 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:16.350774 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:34:16.350774 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:34:16.350774 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:34:16.350774 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:34:16.351759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:16.351759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:16.351759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:34:16.351759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:34:16.351759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:34:16.351759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:34:16.351759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:16.351759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:16.351759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:16.351759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:16.351759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:16.351759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:16.351759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:16.351759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:16.351759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:16.351759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:16.351759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:16.351759 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:16.352736 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:16.352736 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:16.352736 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:34:16.352736 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:34:16.352736 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:34:16.352736 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:34:16.352736 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:34:16.352736 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:34:16.352736 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 01 20:34:16.352736 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:16.352736 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 536/1035] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Oct 1 20:34:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:34:16.352736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bc05f-0fac-436a-bc90-cdf49d51db35 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:16.387096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5eda60-d7be-417e-b5fa-ed2961f2950e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:16.392795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:16.406837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:16.421236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3d8a5-6308-4673-94c7-2cd8526cf8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:16.421832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3d8a5-6308-4673-94c7-2cd8526cf8fd None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:16.422115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3d8a5-6308-4673-94c7-2cd8526cf8fd None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:16.422383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3d8a5-6308-4673-94c7-2cd8526cf8fd None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:16.423012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3d8a5-6308-4673-94c7-2cd8526cf8fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:16.424363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3d8a5-6308-4673-94c7-2cd8526cf8fd None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:16.430824 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 500/1036] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Oct 1 20:34:16 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:34:16.443991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3d8a5-6308-4673-94c7-2cd8526cf8fd None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:16.455074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bc05f-0fac-436a-bc90-cdf49d51db35 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:16.496084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78406dc0-8b57-4819-a3e2-a93e10a90646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:16.497342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78406dc0-8b57-4819-a3e2-a93e10a90646 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:16.497623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78406dc0-8b57-4819-a3e2-a93e10a90646 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:16.497900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78406dc0-8b57-4819-a3e2-a93e10a90646 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:16.499643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78406dc0-8b57-4819-a3e2-a93e10a90646 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:16.501600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78406dc0-8b57-4819-a3e2-a93e10a90646 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:16.503749 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 537/1037] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:34:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:16.509647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78406dc0-8b57-4819-a3e2-a93e10a90646 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:16.565948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3d8a5-6308-4673-94c7-2cd8526cf8fd None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:16.604012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49137370-3e97-423f-aef2-4af5182571f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:16.604950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49137370-3e97-423f-aef2-4af5182571f8 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:16.605463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49137370-3e97-423f-aef2-4af5182571f8 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:16.605737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49137370-3e97-423f-aef2-4af5182571f8 None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:16.606477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49137370-3e97-423f-aef2-4af5182571f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:16.606816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49137370-3e97-423f-aef2-4af5182571f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:16.608297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49137370-3e97-423f-aef2-4af5182571f8 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:16.622142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78406dc0-8b57-4819-a3e2-a93e10a90646 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:16.627195 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 501/1038] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Oct 1 20:34:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:16.630984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:16.633708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49137370-3e97-423f-aef2-4af5182571f8 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:16.668536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5fc79-8ae0-403d-8650-01d700c15d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:16.669119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5fc79-8ae0-403d-8650-01d700c15d9d None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:16.669397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5fc79-8ae0-403d-8650-01d700c15d9d None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:16.669699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5fc79-8ae0-403d-8650-01d700c15d9d None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:16.670344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5fc79-8ae0-403d-8650-01d700c15d9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:16.672342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5fc79-8ae0-403d-8650-01d700c15d9d None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:16.686358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:16.696260 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 538/1039] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Oct 1 20:34:16 2022] POST /identity/v3/users => generated 285 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:34:16.753550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49137370-3e97-423f-aef2-4af5182571f8 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:16.792974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a38ec5-0da8-4b28-b3e6-834d4f2243fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:16.794011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a38ec5-0da8-4b28-b3e6-834d4f2243fb None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:16.794543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a38ec5-0da8-4b28-b3e6-834d4f2243fb None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:16.795019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a38ec5-0da8-4b28-b3e6-834d4f2243fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:16.871704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a38ec5-0da8-4b28-b3e6-834d4f2243fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:16.873582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a38ec5-0da8-4b28-b3e6-834d4f2243fb None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:16.921027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:16.980377 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 502/1040] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:34:16 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 01 20:34:16.986474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5fc79-8ae0-403d-8650-01d700c15d9d None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:17.099644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5fc79-8ae0-403d-8650-01d700c15d9d None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:17.132603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a782cd-58fe-47ba-bfea-c3d8c977abdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:17.133227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a782cd-58fe-47ba-bfea-c3d8c977abdb None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:17.133487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a782cd-58fe-47ba-bfea-c3d8c977abdb None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:17.133753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a782cd-58fe-47ba-bfea-c3d8c977abdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:17.145077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a782cd-58fe-47ba-bfea-c3d8c977abdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:17.146735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a782cd-58fe-47ba-bfea-c3d8c977abdb None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:17.160608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-e0a782cd-58fe-47ba-bfea-c3d8c977abdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:34:17.160608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:17.160608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:17.160608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:17.160608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:17.160608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:17.160608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:17.160608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:17.160608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:17.160608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:17.160608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:17.160608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:17.160608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:34:17.160608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:34:17.160608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:34:17.160608 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:34:17.161591 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:17.161591 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:17.161591 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:17.161591 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:17.161591 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:17.161591 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:17.161591 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:17.161591 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:17.161591 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:17.161591 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:17.161591 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:17.161591 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:17.161591 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:17.161591 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:17.161591 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:17.161591 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:17.161591 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:34:17.167710 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:34:17.167710 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:34:17.167710 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:17.167710 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 539/1041] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:34:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:34:17.169395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a38ec5-0da8-4b28-b3e6-834d4f2243fb None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:17.269335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a38ec5-0da8-4b28-b3e6-834d4f2243fb None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:17.306163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97660030-fe46-4b1e-9724-9b29fb9d3cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:17.306802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97660030-fe46-4b1e-9724-9b29fb9d3cb5 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:17.307083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97660030-fe46-4b1e-9724-9b29fb9d3cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:17.307357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97660030-fe46-4b1e-9724-9b29fb9d3cb5 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:17.307987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97660030-fe46-4b1e-9724-9b29fb9d3cb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:17.308291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97660030-fe46-4b1e-9724-9b29fb9d3cb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:17.309659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97660030-fe46-4b1e-9724-9b29fb9d3cb5 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:17.325511 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 503/1042] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:17.327827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a782cd-58fe-47ba-bfea-c3d8c977abdb None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:17.344775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:17.434061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a782cd-58fe-47ba-bfea-c3d8c977abdb None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:17.468471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa360d83-d4c5-48ec-ae50-a6f34c757331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:17.469092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa360d83-d4c5-48ec-ae50-a6f34c757331 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:17.469373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa360d83-d4c5-48ec-ae50-a6f34c757331 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:17.469714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa360d83-d4c5-48ec-ae50-a6f34c757331 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:17.476542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa360d83-d4c5-48ec-ae50-a6f34c757331 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:17.477977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa360d83-d4c5-48ec-ae50-a6f34c757331 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:17.489832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-fa360d83-d4c5-48ec-ae50-a6f34c757331 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:34:17.489832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:17.489832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:17.489832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:17.489832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:17.489832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:17.489832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:17.489832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:17.489832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:17.489832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:17.489832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:17.489832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:17.489832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:34:17.489832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:34:17.489832 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:34:17.491003 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:34:17.491003 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:17.491003 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:17.491003 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:17.491003 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:17.491003 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:17.491003 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:17.491003 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:17.491003 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:17.491003 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:17.491003 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:17.491003 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:17.491003 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:17.491003 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:17.491003 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:17.491003 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:17.491003 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:17.491934 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:34:17.491934 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:34:17.491934 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:34:17.491934 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:34:17.491934 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:34:17.491934 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:34:17.491934 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:34:17.491934 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:17.494564 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 540/1043] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:34:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:17.497105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97660030-fe46-4b1e-9724-9b29fb9d3cb5 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:17.500309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:17.606697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97660030-fe46-4b1e-9724-9b29fb9d3cb5 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:17.641260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7283cfd-3f34-4a02-a70b-08a966d604e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:17.641858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7283cfd-3f34-4a02-a70b-08a966d604e7 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:17.642132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7283cfd-3f34-4a02-a70b-08a966d604e7 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:17.642400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7283cfd-3f34-4a02-a70b-08a966d604e7 None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:17.643112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7283cfd-3f34-4a02-a70b-08a966d604e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:17.643413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7283cfd-3f34-4a02-a70b-08a966d604e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:17.644828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7283cfd-3f34-4a02-a70b-08a966d604e7 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:17.662605 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 504/1044] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:34:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:17.664263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa360d83-d4c5-48ec-ae50-a6f34c757331 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:17.743685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:17.769652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa360d83-d4c5-48ec-ae50-a6f34c757331 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:17.804117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3374588f-947b-43a2-9791-ae4b9f0b7d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:17.804719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374588f-947b-43a2-9791-ae4b9f0b7d53 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:17.805003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374588f-947b-43a2-9791-ae4b9f0b7d53 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:17.805276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374588f-947b-43a2-9791-ae4b9f0b7d53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:17.806007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3374588f-947b-43a2-9791-ae4b9f0b7d53 None admin] RBAC: Adding query filter params (role.id=fcfc5f73645d423d8475708ef191dacb, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=7ab975dafb564aa2b357311c4a384d53) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:17.806315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3374588f-947b-43a2-9791-ae4b9f0b7d53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:17.807759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3374588f-947b-43a2-9791-ae4b9f0b7d53 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:17.813356 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 541/1045] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:34:17 2022] GET /identity/v3/role_assignments?user.id=7ab975dafb564aa2b357311c4a384d53&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=fcfc5f73645d423d8475708ef191dacb => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:18.854373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40900d08-8753-494a-a29c-8d4165f6cd43 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:18.854973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40900d08-8753-494a-a29c-8d4165f6cd43 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:18.854973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40900d08-8753-494a-a29c-8d4165f6cd43 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:18.856573 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 505/1046] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:18.862690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20326327-ae95-46e8-8986-8d8639b6ebbf None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:18.863162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20326327-ae95-46e8-8986-8d8639b6ebbf None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:18.863259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20326327-ae95-46e8-8986-8d8639b6ebbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:18.881132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-20326327-ae95-46e8-8986-8d8639b6ebbf None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:18.882314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-20326327-ae95-46e8-8986-8d8639b6ebbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:18.951631 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 542/1047] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:19.184795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23299478-1f44-4104-836e-6a88fbf57a1b None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:19.184970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23299478-1f44-4104-836e-6a88fbf57a1b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:19.185231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23299478-1f44-4104-836e-6a88fbf57a1b None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:19.187270 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 506/1048] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:19.192199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f71708-11a6-4734-8c41-ebe19f071661 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:19.192490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f71708-11a6-4734-8c41-ebe19f071661 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:19.192764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f71708-11a6-4734-8c41-ebe19f071661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:19.211216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-81f71708-11a6-4734-8c41-ebe19f071661 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:19.212396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-81f71708-11a6-4734-8c41-ebe19f071661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:19.276132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:19.277541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23299478-1f44-4104-836e-6a88fbf57a1b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:19.283193 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 543/1049] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 01 20:34:19.344929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:19.384908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23299478-1f44-4104-836e-6a88fbf57a1b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:19.418746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0104e4-2314-45f6-8972-8d117288f945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:19.419370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0104e4-2314-45f6-8972-8d117288f945 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:19.419647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0104e4-2314-45f6-8972-8d117288f945 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:19.419920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0104e4-2314-45f6-8972-8d117288f945 None admin] PATH_INFO: `/v3/services/image` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:19.420491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0104e4-2314-45f6-8972-8d117288f945 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:19.421930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0104e4-2314-45f6-8972-8d117288f945 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:19.430505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-2e0104e4-2314-45f6-8972-8d117288f945 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 01 20:34:19.430505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:19.430505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:34:19.430505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:34:19.430505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:34:19.430505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:34:19.430505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:34:19.430505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:34:19.430505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 01 20:34:19.430505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:19.430505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:34:19.430505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:19.430505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:19.430505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:19.430505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:19.430505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:19.430505 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:19.431513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:19.431513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:19.431513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:19.431513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:19.431513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:19.431513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:19.431513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:34:19.431513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:34:19.431513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:34:19.431513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:34:19.431513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:19.431513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:19.431513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:19.431513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:19.431513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:19.431513 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:19.432393 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:19.432393 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:19.432393 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:19.432393 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:19.432393 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:19.432393 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:19.432393 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:19.432393 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:19.432393 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:19.432393 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:19.432393 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:34:19.432393 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:34:19.432393 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 01 20:34:19.432393 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:19.433947 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 507/1050] 10.222.0.17 () {56 vars in 1105 bytes} [Sat Oct 1 20:34:19 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:34:19.438107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f71708-11a6-4734-8c41-ebe19f071661 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:19.538082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f71708-11a6-4734-8c41-ebe19f071661 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:19.571933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe2d8a-0cb9-4949-948c-96d10967a2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:19.572538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe2d8a-0cb9-4949-948c-96d10967a2c8 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:19.572807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe2d8a-0cb9-4949-948c-96d10967a2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:19.573078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe2d8a-0cb9-4949-948c-96d10967a2c8 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:19.573696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe2d8a-0cb9-4949-948c-96d10967a2c8 None admin] RBAC: Adding query filter params (name=image) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:19.573993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe2d8a-0cb9-4949-948c-96d10967a2c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:19.575356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe2d8a-0cb9-4949-948c-96d10967a2c8 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:19.580880 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 544/1051] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:34:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:19.589811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0104e4-2314-45f6-8972-8d117288f945 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:19.603190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe2d8a-0cb9-4949-948c-96d10967a2c8 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:19.666656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:19.692921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0104e4-2314-45f6-8972-8d117288f945 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:19.708308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe2d8a-0cb9-4949-948c-96d10967a2c8 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:19.719952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:19.729749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977447b-d3bf-4956-99a3-d8c5c6a0ee37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:19.730353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977447b-d3bf-4956-99a3-d8c5c6a0ee37 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:19.730666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977447b-d3bf-4956-99a3-d8c5c6a0ee37 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:19.730937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977447b-d3bf-4956-99a3-d8c5c6a0ee37 None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:19.731568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977447b-d3bf-4956-99a3-d8c5c6a0ee37 None admin] RBAC: Adding query filter params (type=image) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:19.731869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977447b-d3bf-4956-99a3-d8c5c6a0ee37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:19.733452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977447b-d3bf-4956-99a3-d8c5c6a0ee37 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:19.738469 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 508/1052] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:34:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:19.743484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb2c46-e8b8-4069-9252-74e69792d96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:19.743887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977447b-d3bf-4956-99a3-d8c5c6a0ee37 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:19.744110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb2c46-e8b8-4069-9252-74e69792d96a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:19.744387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb2c46-e8b8-4069-9252-74e69792d96a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:19.744720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb2c46-e8b8-4069-9252-74e69792d96a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:19.756375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb2c46-e8b8-4069-9252-74e69792d96a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:34:19.761374 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-8efb2c46-e8b8-4069-9252-74e69792d96a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:34:19.761374 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:19.761374 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:19.761374 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:19.761374 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:19.761374 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:19.761374 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:19.761374 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:19.761374 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:19.761374 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:19.761374 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:19.761374 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:19.761374 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:34:19.761374 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:34:19.761374 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:34:19.761374 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:34:19.762415 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:34:19.762415 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:34:19.762415 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:34:19.762415 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:34:19.762415 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:19.762415 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:19.762415 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:19.762415 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:19.762415 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:19.762415 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:19.762415 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:19.762415 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:19.762415 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:19.762415 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:19.762415 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:19.762415 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:19.763474 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:19.763474 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:19.763474 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:19.763474 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:19.763474 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:34:19.763474 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:34:19.763474 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:34:19.763474 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:34:19.763474 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:34:19.763474 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:34:19.763474 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:34:19.763474 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:19.768232 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 545/1053] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:34:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:34:19.769213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb2c46-e8b8-4069-9252-74e69792d96a None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:19.853595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977447b-d3bf-4956-99a3-d8c5c6a0ee37 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:19.879572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb2c46-e8b8-4069-9252-74e69792d96a None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:19.892078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc53d3-6040-4c27-8dfe-6e270fcd461c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:19.893287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc53d3-6040-4c27-8dfe-6e270fcd461c None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:19.893627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc53d3-6040-4c27-8dfe-6e270fcd461c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:19.893905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc53d3-6040-4c27-8dfe-6e270fcd461c None admin] PATH_INFO: `/v3/endpoints` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:19.894591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc53d3-6040-4c27-8dfe-6e270fcd461c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:19.896065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc53d3-6040-4c27-8dfe-6e270fcd461c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:19.902531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:19.917094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ce4f0-1e48-4c6a-ae1e-a877626b6857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:19.917752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ce4f0-1e48-4c6a-ae1e-a877626b6857 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:19.918034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ce4f0-1e48-4c6a-ae1e-a877626b6857 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:19.918303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ce4f0-1e48-4c6a-ae1e-a877626b6857 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:19.919025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ce4f0-1e48-4c6a-ae1e-a877626b6857 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:19.919345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ce4f0-1e48-4c6a-ae1e-a877626b6857 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:19.920757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ce4f0-1e48-4c6a-ae1e-a877626b6857 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:19.926927 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 509/1054] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:34:19 2022] POST /identity/v3/endpoints => generated 327 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:34:19.937005 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 546/1055] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:34:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:19.937866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc53d3-6040-4c27-8dfe-6e270fcd461c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:19.996740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:20.071028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc53d3-6040-4c27-8dfe-6e270fcd461c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:20.110743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99daff59-61ab-45e2-a898-49fc83e4152f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:20.111400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99daff59-61ab-45e2-a898-49fc83e4152f None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:20.111684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99daff59-61ab-45e2-a898-49fc83e4152f None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:20.111962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99daff59-61ab-45e2-a898-49fc83e4152f None admin] PATH_INFO: `/v3/users/7cd47367c67447caa84e2cdac5da6214` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:20.127179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99daff59-61ab-45e2-a898-49fc83e4152f None admin] RBAC: Authorizing `identity:get_user(user_id=7cd47367c67447caa84e2cdac5da6214)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:20.128646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99daff59-61ab-45e2-a898-49fc83e4152f None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:20.130510 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 510/1056] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:34:19 2022] GET /identity/v3/users/7cd47367c67447caa84e2cdac5da6214 => generated 285 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:20.136465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ce4f0-1e48-4c6a-ae1e-a877626b6857 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:20.216601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad31b9-29fe-4f76-8247-b425d3e5a820 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:20.216926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad31b9-29fe-4f76-8247-b425d3e5a820 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:20.217153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad31b9-29fe-4f76-8247-b425d3e5a820 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:20.219183 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 511/1057] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:20.223815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d765f-4d49-4279-ae91-1507328f854c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:20.224088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d765f-4d49-4279-ae91-1507328f854c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:20.224356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d765f-4d49-4279-ae91-1507328f854c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:20.241836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ce4f0-1e48-4c6a-ae1e-a877626b6857 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:20.242257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-671d765f-4d49-4279-ae91-1507328f854c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:20.243468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-671d765f-4d49-4279-ae91-1507328f854c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:20.279245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601c90d-7c0d-4dac-b625-7cd87dc0c368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:20.280366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601c90d-7c0d-4dac-b625-7cd87dc0c368 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:20.280889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601c90d-7c0d-4dac-b625-7cd87dc0c368 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:20.281488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601c90d-7c0d-4dac-b625-7cd87dc0c368 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:20.329714 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 512/1058] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 01 20:34:20.342103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:20.359896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601c90d-7c0d-4dac-b625-7cd87dc0c368 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:20.361344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601c90d-7c0d-4dac-b625-7cd87dc0c368 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:20.465606 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 547/1059] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:34:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 01 20:34:20.466923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d765f-4d49-4279-ae91-1507328f854c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:20.498480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:20.569260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d765f-4d49-4279-ae91-1507328f854c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:20.603051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59eeb9-6a38-4d3c-942e-e171f9fa1681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:20.603676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59eeb9-6a38-4d3c-942e-e171f9fa1681 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:20.603956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59eeb9-6a38-4d3c-942e-e171f9fa1681 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:20.604229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59eeb9-6a38-4d3c-942e-e171f9fa1681 None admin] PATH_INFO: `/v3/projects/eb9cc39ed8da4ca492983162ec9d7af0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:20.605331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59eeb9-6a38-4d3c-942e-e171f9fa1681 None admin] RBAC: Authorizing `identity:get_project(project_id=eb9cc39ed8da4ca492983162ec9d7af0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:20.606803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59eeb9-6a38-4d3c-942e-e171f9fa1681 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:20.609131 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 513/1060] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:34:20 2022] GET /identity/v3/projects/eb9cc39ed8da4ca492983162ec9d7af0 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:34:20.615151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601c90d-7c0d-4dac-b625-7cd87dc0c368 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:20.649193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59eeb9-6a38-4d3c-942e-e171f9fa1681 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:20.705230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:20.724776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601c90d-7c0d-4dac-b625-7cd87dc0c368 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:20.750133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:20.753973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59eeb9-6a38-4d3c-942e-e171f9fa1681 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:20.763144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c4e22-707f-449d-9c55-951be41517e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:20.763754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c4e22-707f-449d-9c55-951be41517e6 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:20.764037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c4e22-707f-449d-9c55-951be41517e6 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:20.764310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c4e22-707f-449d-9c55-951be41517e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:20.765038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c4e22-707f-449d-9c55-951be41517e6 None admin] RBAC: Adding query filter params (role.id=1c50896511054a5f994bee3d99fa0619, scope.project.id=eb9cc39ed8da4ca492983162ec9d7af0, user.id=7cd47367c67447caa84e2cdac5da6214) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:20.765348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c4e22-707f-449d-9c55-951be41517e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:20.766787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c4e22-707f-449d-9c55-951be41517e6 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:20.778530 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 548/1061] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:34:20 2022] GET /identity/v3/role_assignments?user.id=7cd47367c67447caa84e2cdac5da6214&scope.project.id=eb9cc39ed8da4ca492983162ec9d7af0&role.id=1c50896511054a5f994bee3d99fa0619 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:20.791511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed37322-8953-4279-a0ea-00b31b404c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:20.792163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed37322-8953-4279-a0ea-00b31b404c2e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:20.792441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed37322-8953-4279-a0ea-00b31b404c2e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:20.792712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed37322-8953-4279-a0ea-00b31b404c2e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:20.805149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed37322-8953-4279-a0ea-00b31b404c2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:34:20.810078 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-6ed37322-8953-4279-a0ea-00b31b404c2e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 01 20:34:20.810078 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:20.810078 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:20.810078 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:20.810078 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:20.810078 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:20.810078 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:20.810078 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:20.810078 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:20.810078 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:20.810078 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:20.810078 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:20.810078 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:34:20.810078 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:34:20.810078 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:34:20.810078 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:34:20.811141 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:34:20.811141 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:34:20.811141 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:34:20.811141 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:34:20.811141 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:20.811141 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:20.811141 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:20.811141 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:20.811141 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:20.811141 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:20.811141 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:20.811141 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:20.811141 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:20.811141 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:20.811141 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:20.811141 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:20.811997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:20.811997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:20.811997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:20.811997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:20.811997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:34:20.811997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:34:20.811997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:34:20.811997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:34:20.811997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:34:20.811997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:34:20.811997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 01 20:34:20.811997 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:20.811997 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 514/1062] 10.222.0.17 () {56 vars in 1120 bytes} [Sat Oct 1 20:34:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:34:20.817561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c4e22-707f-449d-9c55-951be41517e6 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:20.923757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c4e22-707f-449d-9c55-951be41517e6 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:20.959104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f501f5-14bc-4f31-86ad-8570f99a0f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:20.959724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f501f5-14bc-4f31-86ad-8570f99a0f34 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:20.960003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f501f5-14bc-4f31-86ad-8570f99a0f34 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:20.960275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f501f5-14bc-4f31-86ad-8570f99a0f34 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:20.960954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f501f5-14bc-4f31-86ad-8570f99a0f34 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:20.961264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f501f5-14bc-4f31-86ad-8570f99a0f34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:20.962665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f501f5-14bc-4f31-86ad-8570f99a0f34 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:20.977926 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 549/1063] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Oct 1 20:34:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:20.979167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed37322-8953-4279-a0ea-00b31b404c2e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:21.088573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed37322-8953-4279-a0ea-00b31b404c2e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:21.124538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85ed8a-6c5d-42ea-a1e8-e0580379cc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:21.125165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85ed8a-6c5d-42ea-a1e8-e0580379cc37 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:21.125446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85ed8a-6c5d-42ea-a1e8-e0580379cc37 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:21.125718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85ed8a-6c5d-42ea-a1e8-e0580379cc37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:21.136146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85ed8a-6c5d-42ea-a1e8-e0580379cc37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:21.137601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85ed8a-6c5d-42ea-a1e8-e0580379cc37 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:21.153273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-ab85ed8a-6c5d-42ea-a1e8-e0580379cc37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:34:21.153273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:21.153273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:21.153273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:21.153273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:21.153273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:21.153273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:21.153273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:21.153273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:21.153273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:21.153273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:21.153273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:21.153273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:34:21.153273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:34:21.153273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:34:21.153273 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:34:21.154331 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:21.154331 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:21.154331 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:21.154331 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:21.154331 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:21.154331 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:21.154331 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:21.154331 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:21.154331 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:21.154331 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:21.154331 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:21.154331 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:21.154331 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:21.154331 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:21.154331 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:21.154331 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:21.154331 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:34:21.155900 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:34:21.155900 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:34:21.155900 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:21.155900 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 515/1064] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:34:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:21.161873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f501f5-14bc-4f31-86ad-8570f99a0f34 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:21.174410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:21.265479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f501f5-14bc-4f31-86ad-8570f99a0f34 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:21.303911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43b55f-4d97-48a8-9eba-072db47b4786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:21.303911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43b55f-4d97-48a8-9eba-072db47b4786 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:21.304216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43b55f-4d97-48a8-9eba-072db47b4786 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:21.304441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43b55f-4d97-48a8-9eba-072db47b4786 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:21.305759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43b55f-4d97-48a8-9eba-072db47b4786 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:21.305759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43b55f-4d97-48a8-9eba-072db47b4786 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:21.306741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43b55f-4d97-48a8-9eba-072db47b4786 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:21.318736 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 550/1065] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:21.324570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85ed8a-6c5d-42ea-a1e8-e0580379cc37 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:21.337838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:21.428348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85ed8a-6c5d-42ea-a1e8-e0580379cc37 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:21.461180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e5c6e-44df-4865-b51c-6bf1b4547b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:21.461807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e5c6e-44df-4865-b51c-6bf1b4547b97 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:21.462092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e5c6e-44df-4865-b51c-6bf1b4547b97 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:21.462364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e5c6e-44df-4865-b51c-6bf1b4547b97 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:21.470385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e5c6e-44df-4865-b51c-6bf1b4547b97 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:21.472157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e5c6e-44df-4865-b51c-6bf1b4547b97 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:21.483181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-e56e5c6e-44df-4865-b51c-6bf1b4547b97 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 01 20:34:21.483181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:21.483181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:21.483181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:21.483181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:21.483181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:21.483181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:21.483181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:21.483181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:21.483181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:21.483181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:21.483181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:21.483181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:34:21.483181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:34:21.483181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:34:21.483181 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:34:21.484117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:21.484117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:21.484117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:34:21.484117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:34:21.484117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:34:21.484117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:34:21.484117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:21.484117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:21.484117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:21.484117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:21.484117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:21.484117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:21.484117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:21.484117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:21.484117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:21.484117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:21.484117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:21.484117 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:21.485119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:21.485119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:21.485119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:34:21.485119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:34:21.485119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:34:21.485119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:34:21.485119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:34:21.485119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:34:21.485119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 01 20:34:21.485119 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:21.486804 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 516/1066] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Oct 1 20:34:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:21.490329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43b55f-4d97-48a8-9eba-072db47b4786 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:21.562039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:21.595375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43b55f-4d97-48a8-9eba-072db47b4786 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:21.629481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70c147-a91c-496b-a217-669da86a591c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:21.630216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70c147-a91c-496b-a217-669da86a591c None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:21.630637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70c147-a91c-496b-a217-669da86a591c None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:21.631026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70c147-a91c-496b-a217-669da86a591c None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:21.631842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70c147-a91c-496b-a217-669da86a591c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:21.632254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70c147-a91c-496b-a217-669da86a591c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:21.633778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70c147-a91c-496b-a217-669da86a591c None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:21.650798 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 551/1067] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Oct 1 20:34:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:34:21.656782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e5c6e-44df-4865-b51c-6bf1b4547b97 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:21.729854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:21.762153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e5c6e-44df-4865-b51c-6bf1b4547b97 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:21.796067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8446ba-da91-402d-8162-202815fad986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:21.796838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8446ba-da91-402d-8162-202815fad986 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:21.797214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8446ba-da91-402d-8162-202815fad986 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:21.797583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8446ba-da91-402d-8162-202815fad986 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:21.808186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8446ba-da91-402d-8162-202815fad986 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:21.809718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8446ba-da91-402d-8162-202815fad986 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:21.823848 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-2f8446ba-da91-402d-8162-202815fad986 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:34:21.823848 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:21.823848 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:21.823848 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:21.823848 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:21.823848 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:21.823848 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:21.823848 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:21.823848 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:21.823848 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:21.823848 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:21.823848 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:21.823848 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:34:21.823848 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:34:21.823848 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:34:21.823848 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:34:21.824989 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:21.824989 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:21.824989 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:21.824989 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:21.824989 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:21.824989 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:21.824989 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:21.824989 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:21.824989 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:21.824989 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:21.824989 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:21.824989 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:21.824989 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:21.824989 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:21.824989 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:21.824989 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:21.824989 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:34:21.826091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:34:21.826091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:34:21.826091 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:21.827821 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 517/1068] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:34:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:21.833494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70c147-a91c-496b-a217-669da86a591c None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:21.940863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70c147-a91c-496b-a217-669da86a591c None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:21.969517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:21.978679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27057827-31c9-4a31-81f6-b686b122c78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:21.979290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27057827-31c9-4a31-81f6-b686b122c78b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:21.979565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27057827-31c9-4a31-81f6-b686b122c78b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:21.979831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27057827-31c9-4a31-81f6-b686b122c78b None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:21.980525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27057827-31c9-4a31-81f6-b686b122c78b None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:21.980771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27057827-31c9-4a31-81f6-b686b122c78b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:21.982113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27057827-31c9-4a31-81f6-b686b122c78b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:21.998137 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 552/1069] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:34:21.999949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8446ba-da91-402d-8162-202815fad986 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:22.111091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8446ba-da91-402d-8162-202815fad986 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:22.134075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:22.147149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5648f-11aa-4658-89cc-8d538c587852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:22.147774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5648f-11aa-4658-89cc-8d538c587852 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:22.148051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5648f-11aa-4658-89cc-8d538c587852 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:22.148318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5648f-11aa-4658-89cc-8d538c587852 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:22.155007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5648f-11aa-4658-89cc-8d538c587852 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:22.156440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5648f-11aa-4658-89cc-8d538c587852 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:22.167499 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-48a5648f-11aa-4658-89cc-8d538c587852 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:34:22.167499 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:22.167499 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:22.167499 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:22.167499 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:22.167499 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:22.167499 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:22.167499 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:22.167499 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:22.167499 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:22.167499 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:22.167499 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:22.167499 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:34:22.167499 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:34:22.167499 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:34:22.168375 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:34:22.168375 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:22.168375 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:22.168375 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:22.168375 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:22.168375 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:22.168375 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:22.168375 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:22.168375 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:22.168375 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:22.168375 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:22.168375 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:22.168375 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:22.168375 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:22.168375 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:22.168375 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:22.168375 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:22.169287 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:34:22.169287 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:34:22.169287 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:34:22.169287 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:34:22.169287 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:34:22.169287 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:34:22.169287 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:34:22.169287 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:22.170809 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 518/1070] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:34:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:22.174520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27057827-31c9-4a31-81f6-b686b122c78b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:22.278996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27057827-31c9-4a31-81f6-b686b122c78b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:22.318142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44f226b-01c5-47cc-97be-003ef7dfcc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:22.318783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f226b-01c5-47cc-97be-003ef7dfcc56 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:22.319068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f226b-01c5-47cc-97be-003ef7dfcc56 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:22.319342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f226b-01c5-47cc-97be-003ef7dfcc56 None admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:22.320042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44f226b-01c5-47cc-97be-003ef7dfcc56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:22.320349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44f226b-01c5-47cc-97be-003ef7dfcc56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:22.322040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44f226b-01c5-47cc-97be-003ef7dfcc56 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:22.334931 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 553/1071] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:34:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:22.340688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5648f-11aa-4658-89cc-8d538c587852 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:22.347249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd57d6-e8cd-4bc0-bcb4-a82064c7cb7b None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:22.347537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd57d6-e8cd-4bc0-bcb4-a82064c7cb7b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:22.347813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd57d6-e8cd-4bc0-bcb4-a82064c7cb7b None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:22.349230 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 554/1072] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:34:22.353792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64037a2c-547c-4e1c-b8f4-04df5cf44530 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:22.354077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64037a2c-547c-4e1c-b8f4-04df5cf44530 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:22.354350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64037a2c-547c-4e1c-b8f4-04df5cf44530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:22.371863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-64037a2c-547c-4e1c-b8f4-04df5cf44530 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:22.373047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-64037a2c-547c-4e1c-b8f4-04df5cf44530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:22.421371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:22.450541 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 555/1073] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 01 20:34:22.453275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5648f-11aa-4658-89cc-8d538c587852 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:22.487972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2594c-e70a-4930-ba0e-50372c7374da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:22.488656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2594c-e70a-4930-ba0e-50372c7374da None admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:22.488937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2594c-e70a-4930-ba0e-50372c7374da None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:22.489212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2594c-e70a-4930-ba0e-50372c7374da None admin] PATH_INFO: `/v3/projects/b7875946723844198353bbfcf22516fb/users/7ab975dafb564aa2b357311c4a384d53/roles/fcfc5f73645d423d8475708ef191dacb` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:22.491379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2594c-e70a-4930-ba0e-50372c7374da None admin] RBAC: Authorizing `identity:create_grant(project_id=b7875946723844198353bbfcf22516fb, user_id=7ab975dafb564aa2b357311c4a384d53, role_id=fcfc5f73645d423d8475708ef191dacb)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:22.492808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2594c-e70a-4930-ba0e-50372c7374da None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:22.507039 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 519/1074] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:34:22 2022] PUT /identity/v3/projects/b7875946723844198353bbfcf22516fb/users/7ab975dafb564aa2b357311c4a384d53/roles/fcfc5f73645d423d8475708ef191dacb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:34:22.773588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64037a2c-547c-4e1c-b8f4-04df5cf44530 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:22.883511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64037a2c-547c-4e1c-b8f4-04df5cf44530 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:22.920356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce77eb-a153-456f-ab5d-7fdee07da7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:22.920993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce77eb-a153-456f-ab5d-7fdee07da7fe None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:22.921277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce77eb-a153-456f-ab5d-7fdee07da7fe None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:22.921549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce77eb-a153-456f-ab5d-7fdee07da7fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:22.967709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:22.991023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce77eb-a153-456f-ab5d-7fdee07da7fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:22.992479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce77eb-a153-456f-ab5d-7fdee07da7fe None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:23.105628 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 556/1075] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:34:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 01 20:34:23.107161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2594c-e70a-4930-ba0e-50372c7374da None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:23.151610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:23.212122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2594c-e70a-4930-ba0e-50372c7374da None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:23.221958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54d15e-53e6-42a4-a37d-bc0760cdee36 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:23.222247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54d15e-53e6-42a4-a37d-bc0760cdee36 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:23.222555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54d15e-53e6-42a4-a37d-bc0760cdee36 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:23.224054 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 557/1076] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:23.228741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21d5eb-8a22-46ea-a72b-62b9cf665829 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:23.229013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21d5eb-8a22-46ea-a72b-62b9cf665829 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:23.229291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21d5eb-8a22-46ea-a72b-62b9cf665829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:23.248440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0b21d5eb-8a22-46ea-a72b-62b9cf665829 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:23.250302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0b21d5eb-8a22-46ea-a72b-62b9cf665829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:23.250585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e742484-521e-400e-af6c-4d06a9f247ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:23.251224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e742484-521e-400e-af6c-4d06a9f247ea None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:23.251503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e742484-521e-400e-af6c-4d06a9f247ea None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:23.251775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e742484-521e-400e-af6c-4d06a9f247ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:23.262918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e742484-521e-400e-af6c-4d06a9f247ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:23.264391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e742484-521e-400e-af6c-4d06a9f247ea None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:23.278695 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-6e742484-521e-400e-af6c-4d06a9f247ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:34:23.278695 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:23.278695 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:23.278695 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:23.278695 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:23.278695 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:23.278695 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:23.278695 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:23.278695 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:23.278695 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:23.278695 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:23.278695 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:23.278695 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:34:23.278695 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:34:23.278695 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:34:23.278695 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:34:23.279693 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:23.279693 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:23.279693 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:23.279693 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:23.279693 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:23.279693 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:23.279693 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:23.279693 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:23.279693 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:23.279693 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:23.279693 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:23.279693 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:23.279693 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:23.279693 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:23.279693 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:23.279693 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:23.279693 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:34:23.280664 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:34:23.280664 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:34:23.280664 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:23.280664 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 520/1077] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:34:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:34:23.285716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e742484-521e-400e-af6c-4d06a9f247ea None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:23.339565 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 558/1078] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 01 20:34:23.393134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:23.400434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e742484-521e-400e-af6c-4d06a9f247ea None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:23.434215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9919d9-a808-49c4-ad06-038afd78a867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:23.434839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9919d9-a808-49c4-ad06-038afd78a867 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:23.435117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9919d9-a808-49c4-ad06-038afd78a867 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:23.435387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9919d9-a808-49c4-ad06-038afd78a867 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:23.436588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9919d9-a808-49c4-ad06-038afd78a867 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:23.436868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9919d9-a808-49c4-ad06-038afd78a867 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:23.438226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9919d9-a808-49c4-ad06-038afd78a867 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:23.458779 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 521/1079] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:34:23.677567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21d5eb-8a22-46ea-a72b-62b9cf665829 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:23.688470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:23.789016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21d5eb-8a22-46ea-a72b-62b9cf665829 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:23.824377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427cc54-06ff-45e3-a024-a8aa809ffd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:23.825000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427cc54-06ff-45e3-a024-a8aa809ffd1b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:23.825279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427cc54-06ff-45e3-a024-a8aa809ffd1b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:23.825549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427cc54-06ff-45e3-a024-a8aa809ffd1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:23.836389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427cc54-06ff-45e3-a024-a8aa809ffd1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:34:23.841134 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-3427cc54-06ff-45e3-a024-a8aa809ffd1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:34:23.841134 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:23.841134 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:23.841134 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:23.841134 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:23.841134 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:23.841134 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:23.841134 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:23.841134 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:23.841134 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:23.841134 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:23.841134 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:23.841134 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:34:23.841134 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:34:23.841134 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:34:23.841134 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:34:23.842109 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:34:23.842109 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:34:23.842109 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:34:23.842109 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:34:23.842109 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:23.842109 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:23.842109 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:23.842109 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:23.842109 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:23.842109 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:23.842109 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:23.842109 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:23.842109 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:23.842109 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:23.842109 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:23.842109 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:23.843068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:23.843068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:23.843068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:23.843068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:23.843068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:34:23.843068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:34:23.843068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:34:23.843068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:34:23.843068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:34:23.843068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:34:23.843068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:34:23.843068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:23.843068 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 559/1080] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:34:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:23.848240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9919d9-a808-49c4-ad06-038afd78a867 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:23.943867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9919d9-a808-49c4-ad06-038afd78a867 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:23.977033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ff6cc-aa10-4103-97e3-e421885a162c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:23.977629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ff6cc-aa10-4103-97e3-e421885a162c None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:23.977900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ff6cc-aa10-4103-97e3-e421885a162c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:23.978167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ff6cc-aa10-4103-97e3-e421885a162c None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:23.978858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ff6cc-aa10-4103-97e3-e421885a162c None admin] RBAC: Adding query filter params (name=admin) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:23.979163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ff6cc-aa10-4103-97e3-e421885a162c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:23.980511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ff6cc-aa10-4103-97e3-e421885a162c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:23.988773 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 522/1081] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:34:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:23.994519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427cc54-06ff-45e3-a024-a8aa809ffd1b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:24.064774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:24.099023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427cc54-06ff-45e3-a024-a8aa809ffd1b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:24.132791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb86a20-4f7b-4841-9e2f-616c3232f5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:24.133406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb86a20-4f7b-4841-9e2f-616c3232f5f4 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:24.133682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb86a20-4f7b-4841-9e2f-616c3232f5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:24.133950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb86a20-4f7b-4841-9e2f-616c3232f5f4 None admin] PATH_INFO: `/v3/users/7cd47367c67447caa84e2cdac5da6214` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:24.135073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb86a20-4f7b-4841-9e2f-616c3232f5f4 None admin] RBAC: Authorizing `identity:get_user(user_id=7cd47367c67447caa84e2cdac5da6214)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:24.136486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb86a20-4f7b-4841-9e2f-616c3232f5f4 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:24.138099 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 560/1082] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:34:23 2022] GET /identity/v3/users/7cd47367c67447caa84e2cdac5da6214 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:34:24.143851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ff6cc-aa10-4103-97e3-e421885a162c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:24.161821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:24.241430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ff6cc-aa10-4103-97e3-e421885a162c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:24.273472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873f60e-abb7-4c95-8a83-4f534f82eb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:24.274254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873f60e-abb7-4c95-8a83-4f534f82eb97 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:24.274570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873f60e-abb7-4c95-8a83-4f534f82eb97 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:24.274841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873f60e-abb7-4c95-8a83-4f534f82eb97 None admin] PATH_INFO: `/v3/projects/eb9cc39ed8da4ca492983162ec9d7af0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:24.275939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5873f60e-abb7-4c95-8a83-4f534f82eb97 None admin] RBAC: Authorizing `identity:get_project(project_id=eb9cc39ed8da4ca492983162ec9d7af0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:24.277360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5873f60e-abb7-4c95-8a83-4f534f82eb97 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:24.279004 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 523/1083] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:34:24 2022] GET /identity/v3/projects/eb9cc39ed8da4ca492983162ec9d7af0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:34:24.284856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb86a20-4f7b-4841-9e2f-616c3232f5f4 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:24.385906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb86a20-4f7b-4841-9e2f-616c3232f5f4 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:24.419272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ca74a-cfa6-45c2-893d-bfd917466d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:24.419914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ca74a-cfa6-45c2-893d-bfd917466d24 None admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:24.420190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ca74a-cfa6-45c2-893d-bfd917466d24 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:24.421037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ca74a-cfa6-45c2-893d-bfd917466d24 None admin] PATH_INFO: `/v3/projects/eb9cc39ed8da4ca492983162ec9d7af0/users/7cd47367c67447caa84e2cdac5da6214/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:24.422930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ca74a-cfa6-45c2-893d-bfd917466d24 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb9cc39ed8da4ca492983162ec9d7af0, user_id=7cd47367c67447caa84e2cdac5da6214, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:24.424336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ca74a-cfa6-45c2-893d-bfd917466d24 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:24.427272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:24.438207 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 561/1084] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:34:24 2022] PUT /identity/v3/projects/eb9cc39ed8da4ca492983162ec9d7af0/users/7cd47367c67447caa84e2cdac5da6214/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:34:24.952341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128497f6-5ed7-43b3-9678-fbd194985144 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:24.952623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128497f6-5ed7-43b3-9678-fbd194985144 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:24.953957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128497f6-5ed7-43b3-9678-fbd194985144 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:24.955395 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 524/1085] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:24.961421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66410087-a54d-44f6-9511-9375d9fe1d85 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:24.961708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66410087-a54d-44f6-9511-9375d9fe1d85 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:24.961985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66410087-a54d-44f6-9511-9375d9fe1d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:24.980418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-66410087-a54d-44f6-9511-9375d9fe1d85 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:24.981583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-66410087-a54d-44f6-9511-9375d9fe1d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:25.049465 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 562/1086] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:25.371859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128497f6-5ed7-43b3-9678-fbd194985144 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:25.455630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:25.472595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128497f6-5ed7-43b3-9678-fbd194985144 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:25.508259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8053e1-6f97-4361-aa22-a5f865017e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:25.508892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8053e1-6f97-4361-aa22-a5f865017e89 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:25.509172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8053e1-6f97-4361-aa22-a5f865017e89 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:25.509444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8053e1-6f97-4361-aa22-a5f865017e89 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:25.519747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8053e1-6f97-4361-aa22-a5f865017e89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:34:25.524462 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-0a8053e1-6f97-4361-aa22-a5f865017e89 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 01 20:34:25.524462 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:25.524462 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:25.524462 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:25.524462 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:25.524462 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:25.524462 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:25.524462 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:25.524462 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:25.524462 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:25.524462 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:25.524462 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:25.524462 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:34:25.524462 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:34:25.524462 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:34:25.524462 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:34:25.525568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:34:25.525568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:34:25.525568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:34:25.525568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:34:25.525568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:25.525568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:25.525568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:25.525568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:25.525568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:25.525568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:25.525568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:25.525568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:25.525568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:25.525568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:25.525568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:25.525568 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:25.526428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:25.526428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:25.526428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:25.526428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:25.526428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:34:25.526428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:34:25.526428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:34:25.526428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:34:25.526428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:34:25.526428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:34:25.526428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 01 20:34:25.526428 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:25.526428 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 525/1087] 10.222.0.17 () {56 vars in 1120 bytes} [Sat Oct 1 20:34:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:25.531772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66410087-a54d-44f6-9511-9375d9fe1d85 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:25.633507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66410087-a54d-44f6-9511-9375d9fe1d85 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:25.658771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:25.667566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3075799-4285-41f0-a3ee-af76e79efef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:25.668430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3075799-4285-41f0-a3ee-af76e79efef2 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:25.668934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3075799-4285-41f0-a3ee-af76e79efef2 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:25.669219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3075799-4285-41f0-a3ee-af76e79efef2 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:25.669887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3075799-4285-41f0-a3ee-af76e79efef2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:25.670190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3075799-4285-41f0-a3ee-af76e79efef2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:25.672076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3075799-4285-41f0-a3ee-af76e79efef2 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:25.680211 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 563/1088] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Oct 1 20:34:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:25.686904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8053e1-6f97-4361-aa22-a5f865017e89 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:25.784893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8053e1-6f97-4361-aa22-a5f865017e89 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:25.818052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6a2eb-aa3f-4ff1-afda-42be50960af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:25.818706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6a2eb-aa3f-4ff1-afda-42be50960af6 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:25.818987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6a2eb-aa3f-4ff1-afda-42be50960af6 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:25.819259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6a2eb-aa3f-4ff1-afda-42be50960af6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:25.830138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6a2eb-aa3f-4ff1-afda-42be50960af6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:25.831612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6a2eb-aa3f-4ff1-afda-42be50960af6 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:25.834102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:25.846662 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-b0d6a2eb-aa3f-4ff1-afda-42be50960af6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:34:25.846662 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:25.846662 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:25.846662 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:25.846662 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:25.846662 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:25.846662 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:25.846662 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:25.846662 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:25.846662 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:25.846662 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:25.846662 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:25.846662 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:34:25.846662 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:34:25.846662 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:34:25.846662 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:34:25.847717 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:25.847717 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:25.847717 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:25.847717 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:25.847717 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:25.847717 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:25.847717 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:25.847717 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:25.847717 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:25.847717 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:25.847717 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:25.847717 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:25.847717 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:25.847717 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:25.847717 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:25.847717 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:25.847717 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:34:25.849003 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:34:25.849003 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:34:25.849003 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:25.849003 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 526/1089] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:25.853462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3075799-4285-41f0-a3ee-af76e79efef2 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:25.951050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3075799-4285-41f0-a3ee-af76e79efef2 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:25.983604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dcc41e-e999-40a3-8873-6db99c8da750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:25.984256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcc41e-e999-40a3-8873-6db99c8da750 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:25.984535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcc41e-e999-40a3-8873-6db99c8da750 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:25.984802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcc41e-e999-40a3-8873-6db99c8da750 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:25.985430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dcc41e-e999-40a3-8873-6db99c8da750 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:25.985725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dcc41e-e999-40a3-8873-6db99c8da750 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:25.987102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dcc41e-e999-40a3-8873-6db99c8da750 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:26.003436 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 564/1090] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:26.009153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6a2eb-aa3f-4ff1-afda-42be50960af6 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:26.114312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6a2eb-aa3f-4ff1-afda-42be50960af6 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:26.145895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea4faa-d1ca-470f-b768-27fe408a8ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:26.146526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea4faa-d1ca-470f-b768-27fe408a8ae8 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:26.146808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea4faa-d1ca-470f-b768-27fe408a8ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:26.147079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea4faa-d1ca-470f-b768-27fe408a8ae8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:26.154136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea4faa-d1ca-470f-b768-27fe408a8ae8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:26.156112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea4faa-d1ca-470f-b768-27fe408a8ae8 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:26.166846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-7bea4faa-d1ca-470f-b768-27fe408a8ae8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 01 20:34:26.166846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:26.166846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:26.166846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:26.166846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:26.166846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:26.166846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:26.166846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:26.166846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:26.166846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:26.166846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:26.166846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:26.166846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:34:26.166846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:34:26.166846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:34:26.166846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:34:26.168002 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:26.168002 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:26.168002 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:34:26.168002 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:34:26.168002 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:34:26.168002 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:34:26.168002 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:26.168002 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:26.168002 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:26.168002 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:26.168002 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:26.168002 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:26.168002 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:26.168002 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:26.168002 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:26.168002 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:26.168002 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:26.168002 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:26.169197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:26.169197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:26.169197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:34:26.169197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:34:26.169197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:34:26.169197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:34:26.169197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:34:26.169197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:34:26.169197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 01 20:34:26.169197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:26.169197 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 527/1091] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Oct 1 20:34:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:34:26.176324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dcc41e-e999-40a3-8873-6db99c8da750 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:26.225127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:26.276724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dcc41e-e999-40a3-8873-6db99c8da750 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:26.310324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136933f0-c0be-4bac-b35a-991e7697e5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:26.310978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136933f0-c0be-4bac-b35a-991e7697e5d7 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:26.311264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136933f0-c0be-4bac-b35a-991e7697e5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:26.311540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136933f0-c0be-4bac-b35a-991e7697e5d7 None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:26.312266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136933f0-c0be-4bac-b35a-991e7697e5d7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:26.312575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136933f0-c0be-4bac-b35a-991e7697e5d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:26.314000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136933f0-c0be-4bac-b35a-991e7697e5d7 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:26.332545 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 565/1092] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Oct 1 20:34:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:26.338247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea4faa-d1ca-470f-b768-27fe408a8ae8 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:26.400739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:26.447449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea4faa-d1ca-470f-b768-27fe408a8ae8 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:26.481062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae7f62-b02a-4f69-992e-f91c24279af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:26.481588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae7f62-b02a-4f69-992e-f91c24279af7 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:26.481862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae7f62-b02a-4f69-992e-f91c24279af7 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:26.482129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae7f62-b02a-4f69-992e-f91c24279af7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:26.547848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ae7f62-b02a-4f69-992e-f91c24279af7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:26.549291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ae7f62-b02a-4f69-992e-f91c24279af7 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:26.602709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:26.646734 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 528/1093] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:34:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 01 20:34:26.652450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136933f0-c0be-4bac-b35a-991e7697e5d7 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:26.750270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136933f0-c0be-4bac-b35a-991e7697e5d7 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:26.757331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:26.783729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2f056-9cc4-4ca8-959e-f8c8d68d874a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:26.784345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2f056-9cc4-4ca8-959e-f8c8d68d874a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:26.785024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2f056-9cc4-4ca8-959e-f8c8d68d874a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:26.785024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2f056-9cc4-4ca8-959e-f8c8d68d874a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:26.794569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb2f056-9cc4-4ca8-959e-f8c8d68d874a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:26.796153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb2f056-9cc4-4ca8-959e-f8c8d68d874a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:26.810197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-abb2f056-9cc4-4ca8-959e-f8c8d68d874a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:34:26.810197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:26.810197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:26.810197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:26.810197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:26.810197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:26.810197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:26.810197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:26.810197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:26.810197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:26.810197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:26.810197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:26.810197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:34:26.810197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:34:26.810197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:34:26.810197 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:34:26.811855 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:26.811855 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:26.811855 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:26.811855 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:26.811855 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:26.811855 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:26.811855 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:26.811855 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:26.811855 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:26.811855 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:26.811855 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:26.811855 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:26.811855 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:26.811855 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:26.811855 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:26.811855 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:26.811855 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:34:26.812918 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:34:26.812918 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:34:26.812918 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:26.812918 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 566/1094] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:34:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:34:26.817600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae7f62-b02a-4f69-992e-f91c24279af7 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:26.823940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a004e-d429-4ba2-8c57-a06425dcc434 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:26.824221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a004e-d429-4ba2-8c57-a06425dcc434 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:26.824489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a004e-d429-4ba2-8c57-a06425dcc434 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:26.825906 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 567/1095] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:34:26.830700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0589143-4d50-4f6c-92df-9fe50577d85b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:26.830985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0589143-4d50-4f6c-92df-9fe50577d85b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:26.831264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0589143-4d50-4f6c-92df-9fe50577d85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:26.849240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e0589143-4d50-4f6c-92df-9fe50577d85b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:26.850423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e0589143-4d50-4f6c-92df-9fe50577d85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:26.914797 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 568/1096] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 01 20:34:26.915572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae7f62-b02a-4f69-992e-f91c24279af7 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:26.949023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce086980-109b-45ef-9e1d-1dec9d1f4063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:26.949618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce086980-109b-45ef-9e1d-1dec9d1f4063 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:26.949896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce086980-109b-45ef-9e1d-1dec9d1f4063 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:26.950167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce086980-109b-45ef-9e1d-1dec9d1f4063 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:26.950839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce086980-109b-45ef-9e1d-1dec9d1f4063 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:26.951144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce086980-109b-45ef-9e1d-1dec9d1f4063 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:26.953105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce086980-109b-45ef-9e1d-1dec9d1f4063 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:26.963929 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 529/1097] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:26.969553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0589143-4d50-4f6c-92df-9fe50577d85b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:27.054417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:27.067143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0589143-4d50-4f6c-92df-9fe50577d85b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:27.099594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f729fc-b7f7-420c-a338-ffcfd7116f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:27.100213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f729fc-b7f7-420c-a338-ffcfd7116f78 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:27.100491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f729fc-b7f7-420c-a338-ffcfd7116f78 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:27.100761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f729fc-b7f7-420c-a338-ffcfd7116f78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:27.107083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f729fc-b7f7-420c-a338-ffcfd7116f78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:27.109025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f729fc-b7f7-420c-a338-ffcfd7116f78 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:27.119677 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-c2f729fc-b7f7-420c-a338-ffcfd7116f78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:34:27.119677 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:27.119677 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:27.119677 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:27.119677 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:27.119677 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:27.119677 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:27.119677 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:27.119677 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:27.119677 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:27.119677 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:27.119677 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:27.119677 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:34:27.119677 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:34:27.119677 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:34:27.120716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:34:27.120716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:27.120716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:27.120716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:27.120716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:27.120716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:27.120716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:27.120716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:27.120716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:27.120716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:27.120716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:27.120716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:27.120716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:27.120716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:27.120716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:27.120716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:27.120716 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:27.121648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:34:27.121648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:34:27.121648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:34:27.121648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:34:27.121648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:34:27.121648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:34:27.121648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:34:27.121648 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:27.121648 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 569/1098] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:34:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:34:27.127238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce086980-109b-45ef-9e1d-1dec9d1f4063 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:27.145346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:27.224200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce086980-109b-45ef-9e1d-1dec9d1f4063 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:27.240548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f729fc-b7f7-420c-a338-ffcfd7116f78 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:27.257257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0f6bd-f454-4a92-8126-ecb8cb7800e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:27.257863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0f6bd-f454-4a92-8126-ecb8cb7800e8 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:27.258140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0f6bd-f454-4a92-8126-ecb8cb7800e8 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:27.258410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0f6bd-f454-4a92-8126-ecb8cb7800e8 None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:27.259122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0f6bd-f454-4a92-8126-ecb8cb7800e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:27.259427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0f6bd-f454-4a92-8126-ecb8cb7800e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:27.260846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0f6bd-f454-4a92-8126-ecb8cb7800e8 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:27.273583 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 530/1099] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:34:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:27.279608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0f6bd-f454-4a92-8126-ecb8cb7800e8 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:27.345184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f729fc-b7f7-420c-a338-ffcfd7116f78 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:27.357202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:27.378314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1eed7-349a-4a6e-aa2f-802acfeb2f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:27.379047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1eed7-349a-4a6e-aa2f-802acfeb2f71 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:27.379326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1eed7-349a-4a6e-aa2f-802acfeb2f71 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:27.379598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1eed7-349a-4a6e-aa2f-802acfeb2f71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:27.380734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0f6bd-f454-4a92-8126-ecb8cb7800e8 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:27.390750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c1eed7-349a-4a6e-aa2f-802acfeb2f71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:34:27.395448 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-06c1eed7-349a-4a6e-aa2f-802acfeb2f71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:34:27.395448 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:27.395448 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:27.395448 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:27.395448 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:27.395448 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:27.395448 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:27.395448 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:27.395448 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:27.395448 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:27.395448 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:27.395448 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:27.395448 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:34:27.395448 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:34:27.395448 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:34:27.395448 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:34:27.396442 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:34:27.396442 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:34:27.396442 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:34:27.396442 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:34:27.396442 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:27.396442 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:27.396442 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:27.396442 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:27.396442 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:27.396442 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:27.396442 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:27.396442 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:27.396442 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:27.396442 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:27.396442 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:27.396442 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:27.397341 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:27.397341 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:27.397341 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:27.397341 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:27.397341 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:34:27.397341 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:34:27.397341 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:34:27.397341 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:34:27.397341 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:34:27.397341 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:34:27.397341 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:34:27.397341 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:27.397341 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 570/1100] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:34:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:34:27.402567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1eed7-349a-4a6e-aa2f-802acfeb2f71 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:27.414224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:27.415061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda7fa1-63e4-43ea-8ffd-839b2f6fef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:27.415663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda7fa1-63e4-43ea-8ffd-839b2f6fef00 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:27.415958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda7fa1-63e4-43ea-8ffd-839b2f6fef00 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:27.416191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda7fa1-63e4-43ea-8ffd-839b2f6fef00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:27.416913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dda7fa1-63e4-43ea-8ffd-839b2f6fef00 None admin] RBAC: Adding query filter params (role.id=fcfc5f73645d423d8475708ef191dacb, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=7ab975dafb564aa2b357311c4a384d53) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:27.417222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dda7fa1-63e4-43ea-8ffd-839b2f6fef00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:27.418661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dda7fa1-63e4-43ea-8ffd-839b2f6fef00 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:27.431408 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 531/1101] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:34:27 2022] GET /identity/v3/role_assignments?user.id=7ab975dafb564aa2b357311c4a384d53&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=fcfc5f73645d423d8475708ef191dacb => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:27.511189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1eed7-349a-4a6e-aa2f-802acfeb2f71 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:27.546569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e772d2-b044-412b-bbe6-34f67d25292a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:27.547191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e772d2-b044-412b-bbe6-34f67d25292a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:27.547471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e772d2-b044-412b-bbe6-34f67d25292a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:27.547745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e772d2-b044-412b-bbe6-34f67d25292a None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:27.548420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e772d2-b044-412b-bbe6-34f67d25292a None admin] RBAC: Adding query filter params (name=admin) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:27.548725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e772d2-b044-412b-bbe6-34f67d25292a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:27.550120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e772d2-b044-412b-bbe6-34f67d25292a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:27.559802 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 571/1102] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:34:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:34:27.566536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda7fa1-63e4-43ea-8ffd-839b2f6fef00 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:27.672711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda7fa1-63e4-43ea-8ffd-839b2f6fef00 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:27.709804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcc18c-9d4a-4f76-b85b-2ce79e003cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:27.710273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcc18c-9d4a-4f76-b85b-2ce79e003cf9 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:27.710579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcc18c-9d4a-4f76-b85b-2ce79e003cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:27.710877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcc18c-9d4a-4f76-b85b-2ce79e003cf9 None admin] PATH_INFO: `/v3/users/7cd47367c67447caa84e2cdac5da6214` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:27.711983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbcc18c-9d4a-4f76-b85b-2ce79e003cf9 None admin] RBAC: Authorizing `identity:get_user(user_id=7cd47367c67447caa84e2cdac5da6214)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:27.713412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbcc18c-9d4a-4f76-b85b-2ce79e003cf9 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:27.715020 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 532/1103] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:34:27 2022] GET /identity/v3/users/7cd47367c67447caa84e2cdac5da6214 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:27.721144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e772d2-b044-412b-bbe6-34f67d25292a None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:27.791734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:27.823074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e772d2-b044-412b-bbe6-34f67d25292a None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:27.858125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff38bad-4437-41eb-9650-adf920a29145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:27.858760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff38bad-4437-41eb-9650-adf920a29145 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:27.859149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff38bad-4437-41eb-9650-adf920a29145 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:27.859426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff38bad-4437-41eb-9650-adf920a29145 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:27.927150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff38bad-4437-41eb-9650-adf920a29145 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:27.928525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff38bad-4437-41eb-9650-adf920a29145 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:28.005629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:28.039194 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 572/1104] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:34:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 01 20:34:28.045843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcc18c-9d4a-4f76-b85b-2ce79e003cf9 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:28.058299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:28.155635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcc18c-9d4a-4f76-b85b-2ce79e003cf9 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:28.190749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843a181-2549-4eff-bfa1-741f2be33493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:28.191299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843a181-2549-4eff-bfa1-741f2be33493 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:28.191581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843a181-2549-4eff-bfa1-741f2be33493 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:28.191848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843a181-2549-4eff-bfa1-741f2be33493 None admin] PATH_INFO: `/v3/projects/eb9cc39ed8da4ca492983162ec9d7af0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:28.192962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843a181-2549-4eff-bfa1-741f2be33493 None admin] RBAC: Authorizing `identity:get_project(project_id=eb9cc39ed8da4ca492983162ec9d7af0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:28.194399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843a181-2549-4eff-bfa1-741f2be33493 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:28.196307 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 533/1105] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:34:28 2022] GET /identity/v3/projects/eb9cc39ed8da4ca492983162ec9d7af0 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:28.202730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff38bad-4437-41eb-9650-adf920a29145 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:28.303359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff38bad-4437-41eb-9650-adf920a29145 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:28.338037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb92b4-bcdc-4dcb-b14d-274eeb331e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:28.341656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb92b4-bcdc-4dcb-b14d-274eeb331e9e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:28.341656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb92b4-bcdc-4dcb-b14d-274eeb331e9e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:28.341656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb92b4-bcdc-4dcb-b14d-274eeb331e9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:28.341656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb92b4-bcdc-4dcb-b14d-274eeb331e9e None admin] RBAC: Adding query filter params (role.id=1c50896511054a5f994bee3d99fa0619, scope.project.id=eb9cc39ed8da4ca492983162ec9d7af0, user.id=7cd47367c67447caa84e2cdac5da6214) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:28.341656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb92b4-bcdc-4dcb-b14d-274eeb331e9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:28.341656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb92b4-bcdc-4dcb-b14d-274eeb331e9e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:28.350953 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 573/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:34:28 2022] GET /identity/v3/role_assignments?user.id=7cd47367c67447caa84e2cdac5da6214&scope.project.id=eb9cc39ed8da4ca492983162ec9d7af0&role.id=1c50896511054a5f994bee3d99fa0619 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:30.745860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cf5ad-2f87-4850-b858-9e7a6e6248d7 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:30.746416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cf5ad-2f87-4850-b858-9e7a6e6248d7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:30.746416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cf5ad-2f87-4850-b858-9e7a6e6248d7 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:30.747746 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 534/1107] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:30.752877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017db40-2dab-4fff-ad7f-c87028dd03b1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:30.753296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017db40-2dab-4fff-ad7f-c87028dd03b1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:30.753433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017db40-2dab-4fff-ad7f-c87028dd03b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:30.771707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c017db40-2dab-4fff-ad7f-c87028dd03b1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:30.772900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c017db40-2dab-4fff-ad7f-c87028dd03b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:30.799320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:30.837925 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 574/1108] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:31.163475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cf5ad-2f87-4850-b858-9e7a6e6248d7 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:31.238781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:31.262371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cf5ad-2f87-4850-b858-9e7a6e6248d7 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:31.294558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92883ee7-df72-479f-aff4-cad3455138e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:31.295138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92883ee7-df72-479f-aff4-cad3455138e4 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:31.295418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92883ee7-df72-479f-aff4-cad3455138e4 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:31.295686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92883ee7-df72-479f-aff4-cad3455138e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:31.296775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92883ee7-df72-479f-aff4-cad3455138e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:31.298191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92883ee7-df72-479f-aff4-cad3455138e4 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:31.299754 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 535/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:34:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:31.305343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017db40-2dab-4fff-ad7f-c87028dd03b1 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:31.403810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017db40-2dab-4fff-ad7f-c87028dd03b1 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:31.435544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c0de8-1a17-486c-9a6b-273a87a1c00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:31.436626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c0de8-1a17-486c-9a6b-273a87a1c00f None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:31.436902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c0de8-1a17-486c-9a6b-273a87a1c00f None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:31.437167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c0de8-1a17-486c-9a6b-273a87a1c00f None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:31.437810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c0de8-1a17-486c-9a6b-273a87a1c00f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:31.439238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c0de8-1a17-486c-9a6b-273a87a1c00f None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:31.459450 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 575/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Oct 1 20:34:31 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:34:33.874000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1188c1c-55b1-4ee1-8791-e5be92d549fb None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:33.874551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1188c1c-55b1-4ee1-8791-e5be92d549fb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:33.874551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1188c1c-55b1-4ee1-8791-e5be92d549fb None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:33.875887 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 536/1111] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:33.881116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c683f-c68f-4964-a7ae-6fd7f29a5537 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:33.881551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c683f-c68f-4964-a7ae-6fd7f29a5537 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:33.881668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c683f-c68f-4964-a7ae-6fd7f29a5537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:33.899861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-008c683f-c68f-4964-a7ae-6fd7f29a5537 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:33.901040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-008c683f-c68f-4964-a7ae-6fd7f29a5537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:33.915144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:33.967953 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 576/1112] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:34.289394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1188c1c-55b1-4ee1-8791-e5be92d549fb None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:34.391668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1188c1c-55b1-4ee1-8791-e5be92d549fb None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:34.424191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790bcfd4-37e7-4666-9a4b-5efddf5ea418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:34.424809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bcfd4-37e7-4666-9a4b-5efddf5ea418 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:34.425088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bcfd4-37e7-4666-9a4b-5efddf5ea418 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:34.425359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bcfd4-37e7-4666-9a4b-5efddf5ea418 None admin] PATH_INFO: `/v3/roles/8b7b6e0028b04be8912abda33d3ef729` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:34.426763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790bcfd4-37e7-4666-9a4b-5efddf5ea418 None admin] RBAC: Authorizing `identity:get_role(role_id=8b7b6e0028b04be8912abda33d3ef729)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:34.428130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790bcfd4-37e7-4666-9a4b-5efddf5ea418 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:34.429429 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 537/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:34:34 2022] GET /identity/v3/roles/8b7b6e0028b04be8912abda33d3ef729 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:34.435425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c683f-c68f-4964-a7ae-6fd7f29a5537 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:34.531172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c683f-c68f-4964-a7ae-6fd7f29a5537 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:34.564292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe1e5d-0500-486a-9026-3cfa0308abdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:34.564844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe1e5d-0500-486a-9026-3cfa0308abdf None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:34.565122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe1e5d-0500-486a-9026-3cfa0308abdf None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:34.565390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe1e5d-0500-486a-9026-3cfa0308abdf None admin] PATH_INFO: `/v3/users/21ffcddfa5124dbd87b96e1c1c4a5da7` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:34.578185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe1e5d-0500-486a-9026-3cfa0308abdf None admin] RBAC: Authorizing `identity:get_user(user_id=21ffcddfa5124dbd87b96e1c1c4a5da7)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:34.579625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe1e5d-0500-486a-9026-3cfa0308abdf None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:34.581192 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 577/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:34:34 2022] GET /identity/v3/users/21ffcddfa5124dbd87b96e1c1c4a5da7 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:34.586839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790bcfd4-37e7-4666-9a4b-5efddf5ea418 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:34.601579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:34.684374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790bcfd4-37e7-4666-9a4b-5efddf5ea418 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:34.716822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788786fc-da27-48e9-a4c0-872b9f711de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:34.717917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788786fc-da27-48e9-a4c0-872b9f711de1 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:34.718194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788786fc-da27-48e9-a4c0-872b9f711de1 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:34.718464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788786fc-da27-48e9-a4c0-872b9f711de1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:34.782761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788786fc-da27-48e9-a4c0-872b9f711de1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:34.784174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788786fc-da27-48e9-a4c0-872b9f711de1 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:34.796939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:34.878554 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 538/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:34:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 01 20:34:34.884550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe1e5d-0500-486a-9026-3cfa0308abdf None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:34.895807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:34.981484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe1e5d-0500-486a-9026-3cfa0308abdf None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:35.015047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733ce3a-9ca5-46f8-9fc4-186a5f87e657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:35.015662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733ce3a-9ca5-46f8-9fc4-186a5f87e657 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:35.015943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733ce3a-9ca5-46f8-9fc4-186a5f87e657 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:35.016216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733ce3a-9ca5-46f8-9fc4-186a5f87e657 None admin] PATH_INFO: `/v3/projects/eb9cc39ed8da4ca492983162ec9d7af0` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:35.017320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733ce3a-9ca5-46f8-9fc4-186a5f87e657 None admin] RBAC: Authorizing `identity:get_project(project_id=eb9cc39ed8da4ca492983162ec9d7af0)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:35.018758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733ce3a-9ca5-46f8-9fc4-186a5f87e657 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:35.020360 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 578/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:34:34 2022] GET /identity/v3/projects/eb9cc39ed8da4ca492983162ec9d7af0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:35.026796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788786fc-da27-48e9-a4c0-872b9f711de1 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:35.123566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788786fc-da27-48e9-a4c0-872b9f711de1 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:35.136160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:35.157480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042607e5-e59c-443f-9bb8-2a8968d013a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:35.158075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042607e5-e59c-443f-9bb8-2a8968d013a6 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:35.158354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042607e5-e59c-443f-9bb8-2a8968d013a6 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:35.158667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042607e5-e59c-443f-9bb8-2a8968d013a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:35.159385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042607e5-e59c-443f-9bb8-2a8968d013a6 None admin] RBAC: Adding query filter params (role.id=8b7b6e0028b04be8912abda33d3ef729, scope.project.id=eb9cc39ed8da4ca492983162ec9d7af0, user.id=21ffcddfa5124dbd87b96e1c1c4a5da7) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:35.159694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042607e5-e59c-443f-9bb8-2a8968d013a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:35.161110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042607e5-e59c-443f-9bb8-2a8968d013a6 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:35.166290 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 539/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:34:35 2022] GET /identity/v3/role_assignments?user.id=21ffcddfa5124dbd87b96e1c1c4a5da7&scope.project.id=eb9cc39ed8da4ca492983162ec9d7af0&role.id=8b7b6e0028b04be8912abda33d3ef729 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:37.503249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b68f28-07a2-4218-af1b-0071af464643 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:37.503763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b68f28-07a2-4218-af1b-0071af464643 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:37.503852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b68f28-07a2-4218-af1b-0071af464643 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:37.505182 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 579/1118] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:37.510206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29072b5f-e8ea-4f67-8cc2-750e2ea4e7c2 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:37.510644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29072b5f-e8ea-4f67-8cc2-750e2ea4e7c2 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:37.510782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29072b5f-e8ea-4f67-8cc2-750e2ea4e7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:37.529025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-29072b5f-e8ea-4f67-8cc2-750e2ea4e7c2 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:37.530843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-29072b5f-e8ea-4f67-8cc2-750e2ea4e7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:37.595175 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 540/1119] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:37.917765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b68f28-07a2-4218-af1b-0071af464643 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:37.992709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:38.020378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b68f28-07a2-4218-af1b-0071af464643 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:38.061803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0a7da-ac43-4902-ae39-a67228269b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:38.062426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0a7da-ac43-4902-ae39-a67228269b31 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:38.062744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0a7da-ac43-4902-ae39-a67228269b31 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:38.063016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0a7da-ac43-4902-ae39-a67228269b31 None admin] PATH_INFO: `/v3/roles/8b7b6e0028b04be8912abda33d3ef729` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:38.064392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0a7da-ac43-4902-ae39-a67228269b31 None admin] RBAC: Authorizing `identity:get_role(role_id=8b7b6e0028b04be8912abda33d3ef729)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:38.065748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0a7da-ac43-4902-ae39-a67228269b31 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:38.067119 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 580/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:34:37 2022] GET /identity/v3/roles/8b7b6e0028b04be8912abda33d3ef729 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:38.073096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29072b5f-e8ea-4f67-8cc2-750e2ea4e7c2 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:38.171691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29072b5f-e8ea-4f67-8cc2-750e2ea4e7c2 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:38.178672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:38.206003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f011b-cca1-4cbd-8fff-bfc5ec6fee6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:38.206658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f011b-cca1-4cbd-8fff-bfc5ec6fee6e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:38.206943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f011b-cca1-4cbd-8fff-bfc5ec6fee6e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:38.207220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f011b-cca1-4cbd-8fff-bfc5ec6fee6e None admin] PATH_INFO: `/v3/users/21ffcddfa5124dbd87b96e1c1c4a5da7` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:38.208327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f011b-cca1-4cbd-8fff-bfc5ec6fee6e None admin] RBAC: Authorizing `identity:get_user(user_id=21ffcddfa5124dbd87b96e1c1c4a5da7)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:38.209760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f011b-cca1-4cbd-8fff-bfc5ec6fee6e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:38.211354 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 541/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:34:38 2022] GET /identity/v3/users/21ffcddfa5124dbd87b96e1c1c4a5da7 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:38.217169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0a7da-ac43-4902-ae39-a67228269b31 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:38.314615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0a7da-ac43-4902-ae39-a67228269b31 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:38.346746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c4edc5-8c84-4d6f-b1d0-29abd9441089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:38.347356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4edc5-8c84-4d6f-b1d0-29abd9441089 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:38.347638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4edc5-8c84-4d6f-b1d0-29abd9441089 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:38.347913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4edc5-8c84-4d6f-b1d0-29abd9441089 None admin] PATH_INFO: `/v3/projects/eb9cc39ed8da4ca492983162ec9d7af0` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:38.349020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c4edc5-8c84-4d6f-b1d0-29abd9441089 None admin] RBAC: Authorizing `identity:get_project(project_id=eb9cc39ed8da4ca492983162ec9d7af0)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:38.350423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c4edc5-8c84-4d6f-b1d0-29abd9441089 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:38.352051 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 581/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:34:38 2022] GET /identity/v3/projects/eb9cc39ed8da4ca492983162ec9d7af0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:38.358233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f011b-cca1-4cbd-8fff-bfc5ec6fee6e None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:38.457260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f011b-cca1-4cbd-8fff-bfc5ec6fee6e None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:38.490146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a97d4-ec98-4f27-b774-4eec5abb6b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:38.490820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a97d4-ec98-4f27-b774-4eec5abb6b3e None admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:38.491101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a97d4-ec98-4f27-b774-4eec5abb6b3e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:38.491374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a97d4-ec98-4f27-b774-4eec5abb6b3e None admin] PATH_INFO: `/v3/projects/eb9cc39ed8da4ca492983162ec9d7af0/users/21ffcddfa5124dbd87b96e1c1c4a5da7/roles/8b7b6e0028b04be8912abda33d3ef729` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:38.493511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a97d4-ec98-4f27-b774-4eec5abb6b3e None admin] RBAC: Authorizing `identity:create_grant(project_id=eb9cc39ed8da4ca492983162ec9d7af0, user_id=21ffcddfa5124dbd87b96e1c1c4a5da7, role_id=8b7b6e0028b04be8912abda33d3ef729)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:38.494986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a97d4-ec98-4f27-b774-4eec5abb6b3e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:38.507683 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 542/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:34:38 2022] PUT /identity/v3/projects/eb9cc39ed8da4ca492983162ec9d7af0/users/21ffcddfa5124dbd87b96e1c1c4a5da7/roles/8b7b6e0028b04be8912abda33d3ef729 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:34:40.883676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb86863-e45e-404f-8d5e-5ddf11740624 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:40.884226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb86863-e45e-404f-8d5e-5ddf11740624 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:40.884226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb86863-e45e-404f-8d5e-5ddf11740624 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:40.885577 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 582/1124] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:40.891016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5edde-934f-4be2-a88b-9e937ea043c8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:40.891449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5edde-934f-4be2-a88b-9e937ea043c8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:40.891572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5edde-934f-4be2-a88b-9e937ea043c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:40.910601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f2a5edde-934f-4be2-a88b-9e937ea043c8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:40.911782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f2a5edde-934f-4be2-a88b-9e937ea043c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:40.937736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:40.977142 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 543/1125] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:41.306303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb86863-e45e-404f-8d5e-5ddf11740624 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:41.321747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:41.406035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb86863-e45e-404f-8d5e-5ddf11740624 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:41.440314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1474b15-500d-4417-8bb8-bf8728322fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:41.440934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1474b15-500d-4417-8bb8-bf8728322fb7 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:41.441220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1474b15-500d-4417-8bb8-bf8728322fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:41.441500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1474b15-500d-4417-8bb8-bf8728322fb7 None admin] PATH_INFO: `/v3/roles/8b7b6e0028b04be8912abda33d3ef729` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:41.442929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1474b15-500d-4417-8bb8-bf8728322fb7 None admin] RBAC: Authorizing `identity:get_role(role_id=8b7b6e0028b04be8912abda33d3ef729)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:41.444285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1474b15-500d-4417-8bb8-bf8728322fb7 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:41.445847 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 583/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:34:41 2022] GET /identity/v3/roles/8b7b6e0028b04be8912abda33d3ef729 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:41.451628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5edde-934f-4be2-a88b-9e937ea043c8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:41.548046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5edde-934f-4be2-a88b-9e937ea043c8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:41.580620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d365b-d148-4e51-8eeb-09495fdfb209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:41.581173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d365b-d148-4e51-8eeb-09495fdfb209 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:41.581448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d365b-d148-4e51-8eeb-09495fdfb209 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:41.581715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d365b-d148-4e51-8eeb-09495fdfb209 None admin] PATH_INFO: `/v3/users/21ffcddfa5124dbd87b96e1c1c4a5da7` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:41.582832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d365b-d148-4e51-8eeb-09495fdfb209 None admin] RBAC: Authorizing `identity:get_user(user_id=21ffcddfa5124dbd87b96e1c1c4a5da7)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:41.584243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d365b-d148-4e51-8eeb-09495fdfb209 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:41.585796 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 544/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:34:41 2022] GET /identity/v3/users/21ffcddfa5124dbd87b96e1c1c4a5da7 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:41.592044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1474b15-500d-4417-8bb8-bf8728322fb7 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:41.669249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:41.688684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1474b15-500d-4417-8bb8-bf8728322fb7 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:41.722707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9408a03-c303-4193-82a8-443eaf65a968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:41.723304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9408a03-c303-4193-82a8-443eaf65a968 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:41.723583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9408a03-c303-4193-82a8-443eaf65a968 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:41.723852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9408a03-c303-4193-82a8-443eaf65a968 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:41.787725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9408a03-c303-4193-82a8-443eaf65a968 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:41.789140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9408a03-c303-4193-82a8-443eaf65a968 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:41.864912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:41.884006 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 584/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:34:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 01 20:34:41.889928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d365b-d148-4e51-8eeb-09495fdfb209 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:41.920513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:41.986675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d365b-d148-4e51-8eeb-09495fdfb209 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:42.021464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22227aa8-e7eb-4358-b0ca-cb2757ba57da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:42.022088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22227aa8-e7eb-4358-b0ca-cb2757ba57da None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:42.022366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22227aa8-e7eb-4358-b0ca-cb2757ba57da None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:42.022663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22227aa8-e7eb-4358-b0ca-cb2757ba57da None admin] PATH_INFO: `/v3/projects/eb9cc39ed8da4ca492983162ec9d7af0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:42.023762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22227aa8-e7eb-4358-b0ca-cb2757ba57da None admin] RBAC: Authorizing `identity:get_project(project_id=eb9cc39ed8da4ca492983162ec9d7af0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:42.025180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22227aa8-e7eb-4358-b0ca-cb2757ba57da None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:42.026839 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 545/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:34:41 2022] GET /identity/v3/projects/eb9cc39ed8da4ca492983162ec9d7af0 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:42.033709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9408a03-c303-4193-82a8-443eaf65a968 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:42.128334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9408a03-c303-4193-82a8-443eaf65a968 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:42.161051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f15064-6f9b-4379-9b2a-91722fcd010d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:42.161627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f15064-6f9b-4379-9b2a-91722fcd010d None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:42.161903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f15064-6f9b-4379-9b2a-91722fcd010d None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:42.162172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f15064-6f9b-4379-9b2a-91722fcd010d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:42.162913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f15064-6f9b-4379-9b2a-91722fcd010d None admin] RBAC: Adding query filter params (role.id=8b7b6e0028b04be8912abda33d3ef729, scope.project.id=eb9cc39ed8da4ca492983162ec9d7af0, user.id=21ffcddfa5124dbd87b96e1c1c4a5da7) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:42.163238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f15064-6f9b-4379-9b2a-91722fcd010d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:42.164623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f15064-6f9b-4379-9b2a-91722fcd010d None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:42.170132 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 585/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:34:42 2022] GET /identity/v3/role_assignments?user.id=21ffcddfa5124dbd87b96e1c1c4a5da7&scope.project.id=eb9cc39ed8da4ca492983162ec9d7af0&role.id=8b7b6e0028b04be8912abda33d3ef729 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:44.571389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0122b34-8d0a-41b3-a5f6-8df9eac3722d None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:44.571940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0122b34-8d0a-41b3-a5f6-8df9eac3722d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:44.571940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0122b34-8d0a-41b3-a5f6-8df9eac3722d None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:44.573301 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 546/1131] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:44.578708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1384ea2-1177-49ac-9d9f-209d95682076 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:44.579141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1384ea2-1177-49ac-9d9f-209d95682076 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:44.579253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1384ea2-1177-49ac-9d9f-209d95682076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:44.597324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e1384ea2-1177-49ac-9d9f-209d95682076 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:44.598517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e1384ea2-1177-49ac-9d9f-209d95682076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:44.637461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:44.663654 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 586/1132] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:44.994260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0122b34-8d0a-41b3-a5f6-8df9eac3722d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:45.090125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:45.094943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0122b34-8d0a-41b3-a5f6-8df9eac3722d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:45.129392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae36f1-ae70-49d7-9a89-d9fce9252441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:45.130011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae36f1-ae70-49d7-9a89-d9fce9252441 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:45.130292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae36f1-ae70-49d7-9a89-d9fce9252441 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:45.130596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae36f1-ae70-49d7-9a89-d9fce9252441 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:45.131699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae36f1-ae70-49d7-9a89-d9fce9252441 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:45.133137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae36f1-ae70-49d7-9a89-d9fce9252441 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:45.134733 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 547/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:34:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:45.141177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1384ea2-1177-49ac-9d9f-209d95682076 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:45.237629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1384ea2-1177-49ac-9d9f-209d95682076 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:45.270274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ef9b2-b521-42fb-9dea-caa51fcf1756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:45.270835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ef9b2-b521-42fb-9dea-caa51fcf1756 None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:45.271117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ef9b2-b521-42fb-9dea-caa51fcf1756 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:45.271389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ef9b2-b521-42fb-9dea-caa51fcf1756 None admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:45.272035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ef9b2-b521-42fb-9dea-caa51fcf1756 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:45.273447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ef9b2-b521-42fb-9dea-caa51fcf1756 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:45.300006 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 587/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Oct 1 20:34:45 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:34:47.720670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ea2b-c864-4530-95e6-f86b798d24fb None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:47.721198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ea2b-c864-4530-95e6-f86b798d24fb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:47.721198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ea2b-c864-4530-95e6-f86b798d24fb None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:47.722560 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 548/1135] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:47.728888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21265fbd-4df7-4d63-a81f-3052c372d8f7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:47.729312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21265fbd-4df7-4d63-a81f-3052c372d8f7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:47.729547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21265fbd-4df7-4d63-a81f-3052c372d8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:47.748121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-21265fbd-4df7-4d63-a81f-3052c372d8f7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:47.749302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-21265fbd-4df7-4d63-a81f-3052c372d8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:47.769525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:47.815284 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 588/1136] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:48.139963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8ea2b-c864-4530-95e6-f86b798d24fb None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:48.237917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8ea2b-c864-4530-95e6-f86b798d24fb None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:48.270849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178dcc40-8077-4c12-a7ce-db85aa8a67d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:48.271456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178dcc40-8077-4c12-a7ce-db85aa8a67d2 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:48.271731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178dcc40-8077-4c12-a7ce-db85aa8a67d2 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:48.271999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178dcc40-8077-4c12-a7ce-db85aa8a67d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:48.273090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178dcc40-8077-4c12-a7ce-db85aa8a67d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:48.274537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178dcc40-8077-4c12-a7ce-db85aa8a67d2 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:48.276063 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 549/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:34:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:48.281745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21265fbd-4df7-4d63-a81f-3052c372d8f7 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:48.377376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21265fbd-4df7-4d63-a81f-3052c372d8f7 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:48.410368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ee6a3-987f-4be3-99cb-0c5516863052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:48.410993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ee6a3-987f-4be3-99cb-0c5516863052 None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:48.411275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ee6a3-987f-4be3-99cb-0c5516863052 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:48.411550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ee6a3-987f-4be3-99cb-0c5516863052 None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:48.412190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ee6a3-987f-4be3-99cb-0c5516863052 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:48.413596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ee6a3-987f-4be3-99cb-0c5516863052 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:48.434769 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 589/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Oct 1 20:34:48 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:34:50.885970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4156d5c-f7f0-4fc6-a2f1-df6e8dee3a19 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:50.886528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4156d5c-f7f0-4fc6-a2f1-df6e8dee3a19 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:50.886528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4156d5c-f7f0-4fc6-a2f1-df6e8dee3a19 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:50.887948 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 550/1139] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:50.893785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213670f7-205e-4468-b7a3-46f87a26e45a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:50.894204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213670f7-205e-4468-b7a3-46f87a26e45a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:50.894327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213670f7-205e-4468-b7a3-46f87a26e45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:50.912071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-213670f7-205e-4468-b7a3-46f87a26e45a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:50.913245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-213670f7-205e-4468-b7a3-46f87a26e45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:50.920452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:50.978250 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 590/1140] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:51.305898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4156d5c-f7f0-4fc6-a2f1-df6e8dee3a19 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:51.340689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:51.403768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4156d5c-f7f0-4fc6-a2f1-df6e8dee3a19 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:51.435983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe27437-6119-4ea8-948a-b302292b841c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:51.436588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe27437-6119-4ea8-948a-b302292b841c None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:51.436862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe27437-6119-4ea8-948a-b302292b841c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:51.437128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe27437-6119-4ea8-948a-b302292b841c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:51.447006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe27437-6119-4ea8-948a-b302292b841c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:34:51.452239 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-9fe27437-6119-4ea8-948a-b302292b841c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:34:51.452239 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:51.452239 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:51.452239 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:51.452239 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:51.452239 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:51.452239 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:51.452239 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:51.452239 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:51.452239 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:51.452239 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:51.452239 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:51.452239 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:34:51.452239 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:34:51.452239 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:34:51.452239 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:34:51.453453 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:34:51.453453 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:34:51.453453 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:34:51.453453 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:34:51.453453 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:51.453453 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:51.453453 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:51.453453 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:51.453453 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:51.453453 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:51.453453 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:51.453453 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:51.453453 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:51.453453 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:51.453453 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:51.453453 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:51.454777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:51.454777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:51.454777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:51.454777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:51.454777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:34:51.454777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:34:51.454777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:34:51.454777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:34:51.454777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:34:51.454777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:34:51.454777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:34:51.454777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:51.454777 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 551/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:34:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:51.459256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213670f7-205e-4468-b7a3-46f87a26e45a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:51.555974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213670f7-205e-4468-b7a3-46f87a26e45a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:51.588818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906c848-61b7-4c43-aceb-dbccb59af291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:51.589405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906c848-61b7-4c43-aceb-dbccb59af291 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:51.589683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906c848-61b7-4c43-aceb-dbccb59af291 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:51.589953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906c848-61b7-4c43-aceb-dbccb59af291 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:51.590637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906c848-61b7-4c43-aceb-dbccb59af291 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:51.590938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906c848-61b7-4c43-aceb-dbccb59af291 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:51.592312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906c848-61b7-4c43-aceb-dbccb59af291 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:51.599991 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 591/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:34:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:51.605868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe27437-6119-4ea8-948a-b302292b841c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:51.697845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:51.704590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe27437-6119-4ea8-948a-b302292b841c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:51.736762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07af347-9da7-490c-b3e9-b7c4bf777b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:51.737367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07af347-9da7-490c-b3e9-b7c4bf777b76 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:51.737643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07af347-9da7-490c-b3e9-b7c4bf777b76 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:51.737914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07af347-9da7-490c-b3e9-b7c4bf777b76 None admin] PATH_INFO: `/v3/users/d38ce6f5f2a94371a4ec7285a83537a4` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:51.751257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07af347-9da7-490c-b3e9-b7c4bf777b76 None admin] RBAC: Authorizing `identity:get_user(user_id=d38ce6f5f2a94371a4ec7285a83537a4)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:51.752679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07af347-9da7-490c-b3e9-b7c4bf777b76 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:51.754235 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 552/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:34:51 2022] GET /identity/v3/users/d38ce6f5f2a94371a4ec7285a83537a4 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:51.762915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906c848-61b7-4c43-aceb-dbccb59af291 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:51.766752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:51.861085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906c848-61b7-4c43-aceb-dbccb59af291 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:51.893791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5e0b8-2f4e-4afe-98a7-6deab2bd073d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:51.894387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5e0b8-2f4e-4afe-98a7-6deab2bd073d None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:51.894703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5e0b8-2f4e-4afe-98a7-6deab2bd073d None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:51.894975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5e0b8-2f4e-4afe-98a7-6deab2bd073d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:51.959906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e5e0b8-2f4e-4afe-98a7-6deab2bd073d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:51.961336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e5e0b8-2f4e-4afe-98a7-6deab2bd073d None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:51.963827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:52.056184 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 592/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:34:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 01 20:34:52.062298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07af347-9da7-490c-b3e9-b7c4bf777b76 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:52.157651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07af347-9da7-490c-b3e9-b7c4bf777b76 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:52.189596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608f0fd-ee7d-488b-a3e5-613ec43b77da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:52.190199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608f0fd-ee7d-488b-a3e5-613ec43b77da None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:52.190474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608f0fd-ee7d-488b-a3e5-613ec43b77da None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:52.190773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608f0fd-ee7d-488b-a3e5-613ec43b77da None admin] PATH_INFO: `/v3/projects/78e2215e2fac497d88dfba39fd4289c7` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:52.191859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608f0fd-ee7d-488b-a3e5-613ec43b77da None admin] RBAC: Authorizing `identity:get_project(project_id=78e2215e2fac497d88dfba39fd4289c7)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:52.193271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608f0fd-ee7d-488b-a3e5-613ec43b77da None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:52.194884 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 553/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:34:52 2022] GET /identity/v3/projects/78e2215e2fac497d88dfba39fd4289c7 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:52.201755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5e0b8-2f4e-4afe-98a7-6deab2bd073d None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:52.297017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5e0b8-2f4e-4afe-98a7-6deab2bd073d None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:52.299273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:52.330771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a6432-a884-4ee4-b1a9-05431fe12742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:52.331361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a6432-a884-4ee4-b1a9-05431fe12742 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:52.331640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a6432-a884-4ee4-b1a9-05431fe12742 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:52.331911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a6432-a884-4ee4-b1a9-05431fe12742 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:52.332626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a6432-a884-4ee4-b1a9-05431fe12742 None admin] RBAC: Adding query filter params (role.id=1c50896511054a5f994bee3d99fa0619, scope.project.id=78e2215e2fac497d88dfba39fd4289c7, user.id=d38ce6f5f2a94371a4ec7285a83537a4) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:52.332932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a6432-a884-4ee4-b1a9-05431fe12742 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:52.334333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a6432-a884-4ee4-b1a9-05431fe12742 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:52.339502 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 593/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:34:52 2022] GET /identity/v3/role_assignments?user.id=d38ce6f5f2a94371a4ec7285a83537a4&scope.project.id=78e2215e2fac497d88dfba39fd4289c7&role.id=1c50896511054a5f994bee3d99fa0619 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:54.694686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8318f-9a3c-410a-b43e-627dcaf92842 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:54.695255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8318f-9a3c-410a-b43e-627dcaf92842 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:54.695255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8318f-9a3c-410a-b43e-627dcaf92842 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:54.696849 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 554/1147] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:54.701870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ecc40-23c8-4959-b78f-b26f2cd61adb None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:54.702294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ecc40-23c8-4959-b78f-b26f2cd61adb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:54.702425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ecc40-23c8-4959-b78f-b26f2cd61adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:54.720844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6f3ecc40-23c8-4959-b78f-b26f2cd61adb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:54.722027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6f3ecc40-23c8-4959-b78f-b26f2cd61adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:54.787151 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 594/1148] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:55.114050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8318f-9a3c-410a-b43e-627dcaf92842 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:55.138001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:55.212633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8318f-9a3c-410a-b43e-627dcaf92842 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:55.245830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba750d-fd5f-4f7e-861b-d259ee82c6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:55.246437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba750d-fd5f-4f7e-861b-d259ee82c6bc None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:55.246744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba750d-fd5f-4f7e-861b-d259ee82c6bc None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:55.247016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba750d-fd5f-4f7e-861b-d259ee82c6bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:55.256836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba750d-fd5f-4f7e-861b-d259ee82c6bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:34:55.261524 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-45ba750d-fd5f-4f7e-861b-d259ee82c6bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:34:55.261524 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:55.261524 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:55.261524 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:55.261524 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:55.261524 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:55.261524 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:55.261524 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:55.261524 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:55.261524 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:55.261524 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:55.261524 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:55.261524 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:34:55.261524 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:34:55.261524 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:34:55.261524 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:34:55.262650 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:34:55.262650 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:34:55.262650 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:34:55.262650 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:34:55.262650 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:55.262650 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:55.262650 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:55.262650 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:55.262650 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:55.262650 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:55.262650 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:55.262650 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:55.262650 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:55.262650 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:55.262650 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:55.262650 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:55.263555 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:55.263555 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:55.263555 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:55.263555 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:55.263555 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:34:55.263555 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:34:55.263555 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:34:55.263555 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:34:55.263555 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:34:55.263555 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:34:55.263555 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:34:55.263555 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:34:55.263555 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 555/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:34:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:55.268584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ecc40-23c8-4959-b78f-b26f2cd61adb None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:55.359939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:55.363373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ecc40-23c8-4959-b78f-b26f2cd61adb None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:55.396522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632317f8-393e-4f36-b030-d4a33b118a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:55.397112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632317f8-393e-4f36-b030-d4a33b118a2c None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:55.397388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632317f8-393e-4f36-b030-d4a33b118a2c None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:55.397659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632317f8-393e-4f36-b030-d4a33b118a2c None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:55.398309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632317f8-393e-4f36-b030-d4a33b118a2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:55.398642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632317f8-393e-4f36-b030-d4a33b118a2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:55.399987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632317f8-393e-4f36-b030-d4a33b118a2c None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:55.408158 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 595/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:34:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:34:55.413978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba750d-fd5f-4f7e-861b-d259ee82c6bc None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:55.493663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:55.511480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba750d-fd5f-4f7e-861b-d259ee82c6bc None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:55.546467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd72f9f-a80d-49d8-bcac-119a6ef980b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:55.547118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd72f9f-a80d-49d8-bcac-119a6ef980b8 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:55.547400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd72f9f-a80d-49d8-bcac-119a6ef980b8 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:55.547675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd72f9f-a80d-49d8-bcac-119a6ef980b8 None admin] PATH_INFO: `/v3/users/d38ce6f5f2a94371a4ec7285a83537a4` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:55.548808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd72f9f-a80d-49d8-bcac-119a6ef980b8 None admin] RBAC: Authorizing `identity:get_user(user_id=d38ce6f5f2a94371a4ec7285a83537a4)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:55.550350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd72f9f-a80d-49d8-bcac-119a6ef980b8 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:55.551963 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 556/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:34:55 2022] GET /identity/v3/users/d38ce6f5f2a94371a4ec7285a83537a4 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:55.557860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632317f8-393e-4f36-b030-d4a33b118a2c None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:55.654303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632317f8-393e-4f36-b030-d4a33b118a2c None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:55.685503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78a153-3db6-415e-9201-a6783d590f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:55.686147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78a153-3db6-415e-9201-a6783d590f91 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:55.686424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78a153-3db6-415e-9201-a6783d590f91 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:55.686725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78a153-3db6-415e-9201-a6783d590f91 None admin] PATH_INFO: `/v3/projects/78e2215e2fac497d88dfba39fd4289c7` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:55.687818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78a153-3db6-415e-9201-a6783d590f91 None admin] RBAC: Authorizing `identity:get_project(project_id=78e2215e2fac497d88dfba39fd4289c7)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:55.689214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78a153-3db6-415e-9201-a6783d590f91 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:55.690820 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 596/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:34:55 2022] GET /identity/v3/projects/78e2215e2fac497d88dfba39fd4289c7 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:55.696809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd72f9f-a80d-49d8-bcac-119a6ef980b8 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:55.792753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd72f9f-a80d-49d8-bcac-119a6ef980b8 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:55.826455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82630ea8-5210-4337-b6cd-93287cd10c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:55.827120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82630ea8-5210-4337-b6cd-93287cd10c81 None admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:55.827395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82630ea8-5210-4337-b6cd-93287cd10c81 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:55.827665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82630ea8-5210-4337-b6cd-93287cd10c81 None admin] PATH_INFO: `/v3/projects/78e2215e2fac497d88dfba39fd4289c7/users/d38ce6f5f2a94371a4ec7285a83537a4/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:55.829517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82630ea8-5210-4337-b6cd-93287cd10c81 None admin] RBAC: Authorizing `identity:create_grant(project_id=78e2215e2fac497d88dfba39fd4289c7, user_id=d38ce6f5f2a94371a4ec7285a83537a4, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:55.830966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82630ea8-5210-4337-b6cd-93287cd10c81 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:55.844661 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 557/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:34:55 2022] PUT /identity/v3/projects/78e2215e2fac497d88dfba39fd4289c7/users/d38ce6f5f2a94371a4ec7285a83537a4/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:34:58.201076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774feda-b34d-496f-97e6-2cf5310f646d None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:58.201620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774feda-b34d-496f-97e6-2cf5310f646d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:58.201620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774feda-b34d-496f-97e6-2cf5310f646d None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:58.202984 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 597/1154] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:34:58.208114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070e55b-65e4-4f1a-9dc5-9d6566efc428 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:58.208481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070e55b-65e4-4f1a-9dc5-9d6566efc428 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:58.208647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070e55b-65e4-4f1a-9dc5-9d6566efc428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:58.227444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1070e55b-65e4-4f1a-9dc5-9d6566efc428 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:34:58.228611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1070e55b-65e4-4f1a-9dc5-9d6566efc428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:34:58.231323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:58.293263 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 558/1155] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:34:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:34:58.615698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774feda-b34d-496f-97e6-2cf5310f646d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:58.644651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:58.717620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774feda-b34d-496f-97e6-2cf5310f646d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:58.750621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ecc671-b5b5-41cd-8323-a76a8f31241c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:58.751245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ecc671-b5b5-41cd-8323-a76a8f31241c None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:58.751527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ecc671-b5b5-41cd-8323-a76a8f31241c None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:58.751798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ecc671-b5b5-41cd-8323-a76a8f31241c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:58.761541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ecc671-b5b5-41cd-8323-a76a8f31241c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:34:58.766808 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-07ecc671-b5b5-41cd-8323-a76a8f31241c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:34:58.766808 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:34:58.766808 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:34:58.766808 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:34:58.766808 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:34:58.766808 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:34:58.766808 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:34:58.766808 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:34:58.766808 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:34:58.766808 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:34:58.766808 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:34:58.766808 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:34:58.766808 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:34:58.766808 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:34:58.766808 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:34:58.766808 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:34:58.767926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:34:58.767926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:34:58.767926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:34:58.767926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:34:58.767926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:34:58.767926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:34:58.767926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:34:58.767926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:34:58.767926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:34:58.767926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:34:58.767926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:34:58.767926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:34:58.767926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:34:58.767926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:34:58.767926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:34:58.767926 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:34:58.768960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:34:58.768960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:34:58.768960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:34:58.768960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:34:58.768960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:34:58.768960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:34:58.768960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:34:58.768960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:34:58.768960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:34:58.768960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:34:58.768960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:34:58.768960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:34:58.768960 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 598/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:34:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:34:58.774194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1070e55b-65e4-4f1a-9dc5-9d6566efc428 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:58.868628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1070e55b-65e4-4f1a-9dc5-9d6566efc428 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:58.901173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d68de8-43d4-4b45-a5d6-754267fc5c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:58.901755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d68de8-43d4-4b45-a5d6-754267fc5c1b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:58.902030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d68de8-43d4-4b45-a5d6-754267fc5c1b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:58.902299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d68de8-43d4-4b45-a5d6-754267fc5c1b None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:58.902966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d68de8-43d4-4b45-a5d6-754267fc5c1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:58.903264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d68de8-43d4-4b45-a5d6-754267fc5c1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:58.905141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d68de8-43d4-4b45-a5d6-754267fc5c1b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:58.913024 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 559/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:34:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:58.918892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ecc671-b5b5-41cd-8323-a76a8f31241c None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:59.004396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:59.016195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ecc671-b5b5-41cd-8323-a76a8f31241c None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:59.050155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbbdca-566f-4855-ac39-c6da817f4dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:59.050816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbbdca-566f-4855-ac39-c6da817f4dd5 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:59.051102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbbdca-566f-4855-ac39-c6da817f4dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:59.051377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbbdca-566f-4855-ac39-c6da817f4dd5 None admin] PATH_INFO: `/v3/users/d38ce6f5f2a94371a4ec7285a83537a4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:59.052499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbbdca-566f-4855-ac39-c6da817f4dd5 None admin] RBAC: Authorizing `identity:get_user(user_id=d38ce6f5f2a94371a4ec7285a83537a4)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:59.053920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbbdca-566f-4855-ac39-c6da817f4dd5 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:59.055549 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 599/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:34:58 2022] GET /identity/v3/users/d38ce6f5f2a94371a4ec7285a83537a4 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:59.061277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d68de8-43d4-4b45-a5d6-754267fc5c1b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:59.064094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:59.157870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d68de8-43d4-4b45-a5d6-754267fc5c1b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:59.189671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005c6dc-056d-4038-88af-0ab9481e86ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:59.190253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005c6dc-056d-4038-88af-0ab9481e86ba None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:59.190556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005c6dc-056d-4038-88af-0ab9481e86ba None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:59.190829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005c6dc-056d-4038-88af-0ab9481e86ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:59.255241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005c6dc-056d-4038-88af-0ab9481e86ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:59.256671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005c6dc-056d-4038-88af-0ab9481e86ba None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:59.258384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:59.353381 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 560/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:34:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 01 20:34:59.359342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbbdca-566f-4855-ac39-c6da817f4dd5 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:59.456649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbbdca-566f-4855-ac39-c6da817f4dd5 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:59.489344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3091f11-787f-4bb3-8a9a-95d7f18a09d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:59.489960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3091f11-787f-4bb3-8a9a-95d7f18a09d1 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:59.490242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3091f11-787f-4bb3-8a9a-95d7f18a09d1 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:59.490541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3091f11-787f-4bb3-8a9a-95d7f18a09d1 None admin] PATH_INFO: `/v3/projects/78e2215e2fac497d88dfba39fd4289c7` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:59.491644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3091f11-787f-4bb3-8a9a-95d7f18a09d1 None admin] RBAC: Authorizing `identity:get_project(project_id=78e2215e2fac497d88dfba39fd4289c7)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:59.493126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3091f11-787f-4bb3-8a9a-95d7f18a09d1 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:59.494760 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 600/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:34:59 2022] GET /identity/v3/projects/78e2215e2fac497d88dfba39fd4289c7 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:34:59.501327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005c6dc-056d-4038-88af-0ab9481e86ba None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:34:59.598106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005c6dc-056d-4038-88af-0ab9481e86ba None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:34:59.599619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:34:59.632523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167a037-91d0-4e24-ac8f-5ac143316f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:34:59.633116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167a037-91d0-4e24-ac8f-5ac143316f5f None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:34:59.633396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167a037-91d0-4e24-ac8f-5ac143316f5f None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:34:59.633666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167a037-91d0-4e24-ac8f-5ac143316f5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:34:59.634377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167a037-91d0-4e24-ac8f-5ac143316f5f None admin] RBAC: Adding query filter params (role.id=1c50896511054a5f994bee3d99fa0619, scope.project.id=78e2215e2fac497d88dfba39fd4289c7, user.id=d38ce6f5f2a94371a4ec7285a83537a4) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:34:59.634727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167a037-91d0-4e24-ac8f-5ac143316f5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:34:59.636142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167a037-91d0-4e24-ac8f-5ac143316f5f None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:34:59.641651 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 561/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:34:59 2022] GET /identity/v3/role_assignments?user.id=d38ce6f5f2a94371a4ec7285a83537a4&scope.project.id=78e2215e2fac497d88dfba39fd4289c7&role.id=1c50896511054a5f994bee3d99fa0619 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:35:02.060171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f40ef-5b5f-4dc3-bbef-a102276d514e None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:02.060631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f40ef-5b5f-4dc3-bbef-a102276d514e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:02.060722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f40ef-5b5f-4dc3-bbef-a102276d514e None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:02.062036 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 601/1162] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:35:02.067394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc24bd-a346-46b7-9c7e-9ca271103db5 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:02.067773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc24bd-a346-46b7-9c7e-9ca271103db5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:02.067945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc24bd-a346-46b7-9c7e-9ca271103db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:02.086439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e0cc24bd-a346-46b7-9c7e-9ca271103db5 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:35:02.087644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e0cc24bd-a346-46b7-9c7e-9ca271103db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:35:02.152639 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 562/1163] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:35:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:35:02.481186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f40ef-5b5f-4dc3-bbef-a102276d514e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:02.507346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:02.582769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f40ef-5b5f-4dc3-bbef-a102276d514e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:02.615764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e0d6d-ec9f-4c08-9967-0db7c4c0b202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:02.616350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e0d6d-ec9f-4c08-9967-0db7c4c0b202 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:02.616626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e0d6d-ec9f-4c08-9967-0db7c4c0b202 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:02.616896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e0d6d-ec9f-4c08-9967-0db7c4c0b202 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:02.681787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e0d6d-ec9f-4c08-9967-0db7c4c0b202 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:02.683254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e0d6d-ec9f-4c08-9967-0db7c4c0b202 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:02.706806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:02.779594 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 602/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:35:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 01 20:35:02.785638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc24bd-a346-46b7-9c7e-9ca271103db5 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:02.875655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:02.881775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc24bd-a346-46b7-9c7e-9ca271103db5 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:02.914372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c3e50-14d1-494e-8262-eef80bb74597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:02.915015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c3e50-14d1-494e-8262-eef80bb74597 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:02.915291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c3e50-14d1-494e-8262-eef80bb74597 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:02.915561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c3e50-14d1-494e-8262-eef80bb74597 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:02.922512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c3e50-14d1-494e-8262-eef80bb74597 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:02.923954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c3e50-14d1-494e-8262-eef80bb74597 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:02.934823 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-9e9c3e50-14d1-494e-8262-eef80bb74597 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 01 20:35:02.934823 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:35:02.934823 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 01 20:35:02.934823 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 01 20:35:02.934823 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:35:02.934823 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:35:02.934823 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:35:02.934823 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:35:02.934823 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 01 20:35:02.934823 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:35:02.934823 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:35:02.934823 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:35:02.934823 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:35:02.934823 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:35:02.934823 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:35:02.934823 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:35:02.934823 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:35:02.935974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:35:02.935974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:35:02.935974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:35:02.935974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:35:02.935974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:35:02.935974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:35:02.935974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:35:02.935974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:35:02.935974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:35:02.935974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:35:02.935974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:35:02.935974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:35:02.935974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:35:02.935974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:35:02.935974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:35:02.935974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:35:02.937014 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:35:02.937014 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:35:02.937014 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:35:02.937014 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:35:02.937014 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:35:02.937014 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:35:02.937014 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:35:02.937014 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:35:02.937014 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:35:02.937014 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:35:02.937014 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 01 20:35:02.937014 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:35:02.937014 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 01 20:35:02.937014 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:35:02.937014 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 563/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Oct 1 20:35:02 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:35:02.941854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e0d6d-ec9f-4c08-9967-0db7c4c0b202 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:03.037666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e0d6d-ec9f-4c08-9967-0db7c4c0b202 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:03.060338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:03.071625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25c18a-fed3-4c8d-be32-1a9783bc9a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:03.072226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25c18a-fed3-4c8d-be32-1a9783bc9a1f None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:03.072509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25c18a-fed3-4c8d-be32-1a9783bc9a1f None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:03.072786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25c18a-fed3-4c8d-be32-1a9783bc9a1f None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:03.073455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25c18a-fed3-4c8d-be32-1a9783bc9a1f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:35:03.073760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25c18a-fed3-4c8d-be32-1a9783bc9a1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:03.075674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25c18a-fed3-4c8d-be32-1a9783bc9a1f None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:03.082632 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 603/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Oct 1 20:35:02 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:35:05.419363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c5672-589e-4955-858b-f5d701231344 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:05.419878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c5672-589e-4955-858b-f5d701231344 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:05.419878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c5672-589e-4955-858b-f5d701231344 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:05.421120 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 564/1167] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:35:05.426362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bfe28-6b2b-43cf-9a62-246e40becc46 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:05.426826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bfe28-6b2b-43cf-9a62-246e40becc46 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:05.426947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bfe28-6b2b-43cf-9a62-246e40becc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:05.444640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a04bfe28-6b2b-43cf-9a62-246e40becc46 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:35:05.446072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a04bfe28-6b2b-43cf-9a62-246e40becc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:35:05.509588 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 604/1168] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:35:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 01 20:35:05.835323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c5672-589e-4955-858b-f5d701231344 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:05.933179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c5672-589e-4955-858b-f5d701231344 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:05.966274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb702a-fbfb-4c2b-88c8-3856e8e03d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:05.966890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb702a-fbfb-4c2b-88c8-3856e8e03d5a None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:05.967169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb702a-fbfb-4c2b-88c8-3856e8e03d5a None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:05.967438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb702a-fbfb-4c2b-88c8-3856e8e03d5a None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:05.968095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb702a-fbfb-4c2b-88c8-3856e8e03d5a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:05.969443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb702a-fbfb-4c2b-88c8-3856e8e03d5a None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:05.985673 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 565/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Sat Oct 1 20:35:05 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:35:08.415106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333d3c6-f537-4e32-94ab-a76383c5d83c None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:08.415623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333d3c6-f537-4e32-94ab-a76383c5d83c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:08.415623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333d3c6-f537-4e32-94ab-a76383c5d83c None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:08.416895 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 605/1170] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:35:08.422070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6798f9-e65d-4cf3-9331-a45c658f08ed None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:08.422517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6798f9-e65d-4cf3-9331-a45c658f08ed None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:08.422630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6798f9-e65d-4cf3-9331-a45c658f08ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:08.440464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8a6798f9-e65d-4cf3-9331-a45c658f08ed None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:35:08.441642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8a6798f9-e65d-4cf3-9331-a45c658f08ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:35:08.452703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:08.508362 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 566/1171] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:35:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:35:08.839621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333d3c6-f537-4e32-94ab-a76383c5d83c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:08.938599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333d3c6-f537-4e32-94ab-a76383c5d83c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:08.955125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:08.971893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf62123-d8a4-4893-8b1a-283c9eabd2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:08.972500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf62123-d8a4-4893-8b1a-283c9eabd2df None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:08.972776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf62123-d8a4-4893-8b1a-283c9eabd2df None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:08.973048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf62123-d8a4-4893-8b1a-283c9eabd2df None admin] PATH_INFO: `/v3/domains/93635a706fa3480bad0f0738bfe2224d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:08.983587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf62123-d8a4-4893-8b1a-283c9eabd2df None admin] RBAC: Authorizing `identity:get_domain(domain_id=93635a706fa3480bad0f0738bfe2224d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:08.985004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf62123-d8a4-4893-8b1a-283c9eabd2df None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:08.986576 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 606/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Sat Oct 1 20:35:08 2022] GET /identity/v3/domains/93635a706fa3480bad0f0738bfe2224d => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:35:08.992402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6798f9-e65d-4cf3-9331-a45c658f08ed None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:09.090451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6798f9-e65d-4cf3-9331-a45c658f08ed None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:09.122984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b184d9-ac55-4e6e-af55-2ec288289e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:09.123577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b184d9-ac55-4e6e-af55-2ec288289e73 None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:09.123854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b184d9-ac55-4e6e-af55-2ec288289e73 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:09.124126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b184d9-ac55-4e6e-af55-2ec288289e73 None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:09.124764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b184d9-ac55-4e6e-af55-2ec288289e73 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:09.126196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b184d9-ac55-4e6e-af55-2ec288289e73 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:09.151643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:09.166070 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 567/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Oct 1 20:35:08 2022] POST /identity/v3/projects => generated 361 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:35:11.577930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4f277-9897-40f0-a98d-e4e5a4336b73 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:11.578479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4f277-9897-40f0-a98d-e4e5a4336b73 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:11.578479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4f277-9897-40f0-a98d-e4e5a4336b73 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:11.579842 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 607/1174] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:35:11.585157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd332547-f300-426d-a9e4-b0ceb4597642 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:11.585567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd332547-f300-426d-a9e4-b0ceb4597642 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:11.585703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd332547-f300-426d-a9e4-b0ceb4597642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:11.604152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-cd332547-f300-426d-a9e4-b0ceb4597642 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:35:11.605323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cd332547-f300-426d-a9e4-b0ceb4597642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:35:11.670629 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 568/1175] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:35:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:35:12.000229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4f277-9897-40f0-a98d-e4e5a4336b73 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:12.097873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4f277-9897-40f0-a98d-e4e5a4336b73 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:12.130982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af1b68-fd16-4595-9ce6-24a2f224b62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:12.131590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af1b68-fd16-4595-9ce6-24a2f224b62e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:12.131869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af1b68-fd16-4595-9ce6-24a2f224b62e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:12.132164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af1b68-fd16-4595-9ce6-24a2f224b62e None admin] PATH_INFO: `/v3/domains/93635a706fa3480bad0f0738bfe2224d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:12.133250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af1b68-fd16-4595-9ce6-24a2f224b62e None admin] RBAC: Authorizing `identity:get_domain(domain_id=93635a706fa3480bad0f0738bfe2224d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:12.134680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af1b68-fd16-4595-9ce6-24a2f224b62e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:12.136214 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 608/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Sat Oct 1 20:35:11 2022] GET /identity/v3/domains/93635a706fa3480bad0f0738bfe2224d => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:35:12.142044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd332547-f300-426d-a9e4-b0ceb4597642 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:12.237780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd332547-f300-426d-a9e4-b0ceb4597642 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:12.263351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:12.271129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75b696-bc52-4625-bf7a-1cff24f1da1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:12.271721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75b696-bc52-4625-bf7a-1cff24f1da1b None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:12.271999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75b696-bc52-4625-bf7a-1cff24f1da1b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:12.272270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75b696-bc52-4625-bf7a-1cff24f1da1b None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:12.272911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75b696-bc52-4625-bf7a-1cff24f1da1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:12.274320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75b696-bc52-4625-bf7a-1cff24f1da1b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:12.295025 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 569/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Oct 1 20:35:12 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:35:14.719869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab4cb7-7778-4d76-bfde-0b016c898369 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:14.720438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab4cb7-7778-4d76-bfde-0b016c898369 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:14.720438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab4cb7-7778-4d76-bfde-0b016c898369 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:14.721738 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 609/1178] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:35:14.726846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9da5e-c4fd-4345-90b1-786c2693b57a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:14.727263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9da5e-c4fd-4345-90b1-786c2693b57a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:14.727401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9da5e-c4fd-4345-90b1-786c2693b57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:14.745309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fef9da5e-c4fd-4345-90b1-786c2693b57a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:35:14.747012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fef9da5e-c4fd-4345-90b1-786c2693b57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:35:14.811100 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 570/1179] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:35:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:35:15.133447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab4cb7-7778-4d76-bfde-0b016c898369 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:15.180098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:15.232923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab4cb7-7778-4d76-bfde-0b016c898369 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:15.266036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ba596-a5bd-4867-bc85-f5e8272b96b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:15.266675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ba596-a5bd-4867-bc85-f5e8272b96b6 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:15.266956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ba596-a5bd-4867-bc85-f5e8272b96b6 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:15.267226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ba596-a5bd-4867-bc85-f5e8272b96b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:15.276903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ba596-a5bd-4867-bc85-f5e8272b96b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:35:15.282122 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-cc2ba596-a5bd-4867-bc85-f5e8272b96b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:35:15.282122 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:35:15.282122 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:35:15.282122 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:35:15.282122 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:35:15.282122 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:35:15.282122 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:35:15.282122 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:35:15.282122 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:35:15.282122 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:35:15.282122 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:35:15.282122 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:35:15.282122 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:35:15.282122 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:35:15.282122 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:35:15.282122 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:35:15.283284 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:35:15.283284 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:35:15.283284 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:35:15.283284 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:35:15.283284 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:35:15.283284 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:35:15.283284 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:35:15.283284 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:35:15.283284 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:35:15.283284 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:35:15.283284 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:35:15.283284 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:35:15.283284 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:35:15.283284 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:35:15.283284 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:35:15.283284 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:35:15.284221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:35:15.284221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:35:15.284221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:35:15.284221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:35:15.284221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:35:15.284221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:35:15.284221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:35:15.284221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:35:15.284221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:35:15.284221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:35:15.284221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:35:15.284221 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:35:15.284221 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 610/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:35:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:35:15.289328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9da5e-c4fd-4345-90b1-786c2693b57a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:15.385243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9da5e-c4fd-4345-90b1-786c2693b57a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:15.399129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:15.419048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ba00f-d2c2-4051-b0dc-61e3342d78cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:15.419639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ba00f-d2c2-4051-b0dc-61e3342d78cd None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:15.419918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ba00f-d2c2-4051-b0dc-61e3342d78cd None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:15.420184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ba00f-d2c2-4051-b0dc-61e3342d78cd None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:15.420836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ba00f-d2c2-4051-b0dc-61e3342d78cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:35:15.421137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ba00f-d2c2-4051-b0dc-61e3342d78cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:15.422509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ba00f-d2c2-4051-b0dc-61e3342d78cd None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:15.430223 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 571/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:35:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:35:15.436619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ba596-a5bd-4867-bc85-f5e8272b96b6 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:15.540912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ba596-a5bd-4867-bc85-f5e8272b96b6 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:15.548125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:15.576573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c6fc5-6051-4edb-819a-3b31aab62a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:15.577195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c6fc5-6051-4edb-819a-3b31aab62a1f None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:15.577472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c6fc5-6051-4edb-819a-3b31aab62a1f None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:15.577744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c6fc5-6051-4edb-819a-3b31aab62a1f None admin] PATH_INFO: `/v3/users/bb7085047538470a90e0e30aa2757547` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:15.590917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c6fc5-6051-4edb-819a-3b31aab62a1f None admin] RBAC: Authorizing `identity:get_user(user_id=bb7085047538470a90e0e30aa2757547)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:15.592334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c6fc5-6051-4edb-819a-3b31aab62a1f None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:15.593905 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 611/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:35:15 2022] GET /identity/v3/users/bb7085047538470a90e0e30aa2757547 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:35:15.599783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ba00f-d2c2-4051-b0dc-61e3342d78cd None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:15.697813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ba00f-d2c2-4051-b0dc-61e3342d78cd None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:15.730301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74e4cb-00d6-48e1-b130-a23b1d2bcffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:15.730997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74e4cb-00d6-48e1-b130-a23b1d2bcffa None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:15.731249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74e4cb-00d6-48e1-b130-a23b1d2bcffa None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:15.731520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74e4cb-00d6-48e1-b130-a23b1d2bcffa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:15.773988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:15.796919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74e4cb-00d6-48e1-b130-a23b1d2bcffa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:15.798361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74e4cb-00d6-48e1-b130-a23b1d2bcffa None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:15.902680 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 572/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:35:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 01 20:35:15.909126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c6fc5-6051-4edb-819a-3b31aab62a1f None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:16.004056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c6fc5-6051-4edb-819a-3b31aab62a1f None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:16.037526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9554bf-d81e-4652-bb4c-b9167e3a2dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:16.038151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9554bf-d81e-4652-bb4c-b9167e3a2dd4 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:16.038438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9554bf-d81e-4652-bb4c-b9167e3a2dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:16.039183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9554bf-d81e-4652-bb4c-b9167e3a2dd4 None admin] PATH_INFO: `/v3/projects/7ed94ab8cdaf48eaa630ea66cbc139df` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:16.040313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9554bf-d81e-4652-bb4c-b9167e3a2dd4 None admin] RBAC: Authorizing `identity:get_project(project_id=7ed94ab8cdaf48eaa630ea66cbc139df)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:16.041728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9554bf-d81e-4652-bb4c-b9167e3a2dd4 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:16.043699 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 612/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:35:15 2022] GET /identity/v3/projects/7ed94ab8cdaf48eaa630ea66cbc139df => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:35:16.049467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74e4cb-00d6-48e1-b130-a23b1d2bcffa None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:16.123362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:16.145597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74e4cb-00d6-48e1-b130-a23b1d2bcffa None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:16.181495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a4aee-ad59-461c-9e4f-fa1838004eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:16.182128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a4aee-ad59-461c-9e4f-fa1838004eb0 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:16.182432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a4aee-ad59-461c-9e4f-fa1838004eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:16.182754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a4aee-ad59-461c-9e4f-fa1838004eb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:16.183531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a4aee-ad59-461c-9e4f-fa1838004eb0 None admin] RBAC: Adding query filter params (role.id=1c50896511054a5f994bee3d99fa0619, scope.project.id=7ed94ab8cdaf48eaa630ea66cbc139df, user.id=bb7085047538470a90e0e30aa2757547) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:35:16.183841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a4aee-ad59-461c-9e4f-fa1838004eb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:16.185302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a4aee-ad59-461c-9e4f-fa1838004eb0 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:16.191091 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 573/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:35:16 2022] GET /identity/v3/role_assignments?user.id=bb7085047538470a90e0e30aa2757547&scope.project.id=7ed94ab8cdaf48eaa630ea66cbc139df&role.id=1c50896511054a5f994bee3d99fa0619 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:35:18.552780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6f62f-28cd-40a8-b465-ff26f329423b None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:18.553306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6f62f-28cd-40a8-b465-ff26f329423b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:18.553306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6f62f-28cd-40a8-b465-ff26f329423b None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:18.554687 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 613/1186] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:35:18.559744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33daff7-4aa3-46b9-a8f2-7f280d537f8e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:18.560190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33daff7-4aa3-46b9-a8f2-7f280d537f8e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:18.560292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33daff7-4aa3-46b9-a8f2-7f280d537f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:18.578263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b33daff7-4aa3-46b9-a8f2-7f280d537f8e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:35:18.579489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b33daff7-4aa3-46b9-a8f2-7f280d537f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:35:18.643651 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 574/1187] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:35:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:35:18.974922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6f62f-28cd-40a8-b465-ff26f329423b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:19.005124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:19.073289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6f62f-28cd-40a8-b465-ff26f329423b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:19.105972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a02ac-ca5c-4264-bc4e-1e059fcd08fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:19.106610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a02ac-ca5c-4264-bc4e-1e059fcd08fa None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:19.106917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a02ac-ca5c-4264-bc4e-1e059fcd08fa None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:19.107188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a02ac-ca5c-4264-bc4e-1e059fcd08fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:19.116797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a02ac-ca5c-4264-bc4e-1e059fcd08fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:35:19.121936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-c90a02ac-ca5c-4264-bc4e-1e059fcd08fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:35:19.121936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:35:19.121936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:35:19.121936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:35:19.121936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:35:19.121936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:35:19.121936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:35:19.121936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:35:19.121936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:35:19.121936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:35:19.121936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:35:19.121936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:35:19.121936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:35:19.121936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:35:19.121936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:35:19.121936 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:35:19.123073 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:35:19.123073 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:35:19.123073 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:35:19.123073 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:35:19.123073 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:35:19.123073 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:35:19.123073 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:35:19.123073 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:35:19.123073 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:35:19.123073 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:35:19.123073 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:35:19.123073 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:35:19.123073 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:35:19.123073 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:35:19.123073 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:35:19.123073 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:35:19.124267 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:35:19.124267 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:35:19.124267 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:35:19.124267 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:35:19.124267 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:35:19.124267 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:35:19.124267 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:35:19.124267 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:35:19.124267 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:35:19.124267 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:35:19.124267 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:35:19.124267 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:35:19.124267 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 614/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:35:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:35:19.129053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33daff7-4aa3-46b9-a8f2-7f280d537f8e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:19.194762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:19.224918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33daff7-4aa3-46b9-a8f2-7f280d537f8e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:19.257151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fd8d6-ce82-4979-804e-70ec07d25176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:19.257683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fd8d6-ce82-4979-804e-70ec07d25176 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:19.257961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fd8d6-ce82-4979-804e-70ec07d25176 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:19.258229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fd8d6-ce82-4979-804e-70ec07d25176 None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:19.258904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5fd8d6-ce82-4979-804e-70ec07d25176 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:35:19.259204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5fd8d6-ce82-4979-804e-70ec07d25176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:19.260545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5fd8d6-ce82-4979-804e-70ec07d25176 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:19.268135 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 575/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:35:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:35:19.273812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a02ac-ca5c-4264-bc4e-1e059fcd08fa None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:19.361434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:19.370531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a02ac-ca5c-4264-bc4e-1e059fcd08fa None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:19.403289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd0b1a-c429-4250-a9b9-f48fb1755164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:19.403898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd0b1a-c429-4250-a9b9-f48fb1755164 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:19.404178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd0b1a-c429-4250-a9b9-f48fb1755164 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:19.404450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd0b1a-c429-4250-a9b9-f48fb1755164 None admin] PATH_INFO: `/v3/users/bb7085047538470a90e0e30aa2757547` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:19.405558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd0b1a-c429-4250-a9b9-f48fb1755164 None admin] RBAC: Authorizing `identity:get_user(user_id=bb7085047538470a90e0e30aa2757547)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:19.407002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd0b1a-c429-4250-a9b9-f48fb1755164 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:19.408546 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 615/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:35:19 2022] GET /identity/v3/users/bb7085047538470a90e0e30aa2757547 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:35:19.414531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fd8d6-ce82-4979-804e-70ec07d25176 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:19.510861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fd8d6-ce82-4979-804e-70ec07d25176 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:19.539165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:19.544200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306a3b4-e096-41dd-a87f-ddead8f739b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:19.544813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306a3b4-e096-41dd-a87f-ddead8f739b7 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:19.545090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306a3b4-e096-41dd-a87f-ddead8f739b7 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:19.545358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306a3b4-e096-41dd-a87f-ddead8f739b7 None admin] PATH_INFO: `/v3/projects/7ed94ab8cdaf48eaa630ea66cbc139df` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:19.546455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1306a3b4-e096-41dd-a87f-ddead8f739b7 None admin] RBAC: Authorizing `identity:get_project(project_id=7ed94ab8cdaf48eaa630ea66cbc139df)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:19.547897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1306a3b4-e096-41dd-a87f-ddead8f739b7 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:19.549527 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 576/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:35:19 2022] GET /identity/v3/projects/7ed94ab8cdaf48eaa630ea66cbc139df => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:35:19.555357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd0b1a-c429-4250-a9b9-f48fb1755164 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:19.656020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd0b1a-c429-4250-a9b9-f48fb1755164 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:19.687659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda9f39-5dc7-4724-9e12-d519b13e9f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:19.688295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda9f39-5dc7-4724-9e12-d519b13e9f5f None admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:19.688571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda9f39-5dc7-4724-9e12-d519b13e9f5f None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:19.688841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda9f39-5dc7-4724-9e12-d519b13e9f5f None admin] PATH_INFO: `/v3/projects/7ed94ab8cdaf48eaa630ea66cbc139df/users/bb7085047538470a90e0e30aa2757547/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:19.690717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda9f39-5dc7-4724-9e12-d519b13e9f5f None admin] RBAC: Authorizing `identity:create_grant(project_id=7ed94ab8cdaf48eaa630ea66cbc139df, user_id=bb7085047538470a90e0e30aa2757547, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:19.692131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda9f39-5dc7-4724-9e12-d519b13e9f5f None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:19.705425 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 616/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:35:19 2022] PUT /identity/v3/projects/7ed94ab8cdaf48eaa630ea66cbc139df/users/bb7085047538470a90e0e30aa2757547/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:35:22.071711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91f854-6557-4672-bb1c-67cda380b7b5 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:22.072166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91f854-6557-4672-bb1c-67cda380b7b5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:22.072279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91f854-6557-4672-bb1c-67cda380b7b5 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:22.073583 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 577/1193] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:35:22.079475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a835a-80bd-4a46-8acf-aa47091a94e2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:22.079844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a835a-80bd-4a46-8acf-aa47091a94e2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:22.080032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a835a-80bd-4a46-8acf-aa47091a94e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:22.098302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-184a835a-80bd-4a46-8acf-aa47091a94e2 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:35:22.099514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-184a835a-80bd-4a46-8acf-aa47091a94e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:35:22.112981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:22.164392 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 617/1194] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:35:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:35:22.489269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d91f854-6557-4672-bb1c-67cda380b7b5 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:22.588847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d91f854-6557-4672-bb1c-67cda380b7b5 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:22.620748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad224c-9c5f-439e-a8f4-8a02c890282d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:22.621384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad224c-9c5f-439e-a8f4-8a02c890282d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:22.621662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad224c-9c5f-439e-a8f4-8a02c890282d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:22.621931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad224c-9c5f-439e-a8f4-8a02c890282d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:22.632066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad224c-9c5f-439e-a8f4-8a02c890282d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:35:22.637279 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-abad224c-9c5f-439e-a8f4-8a02c890282d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:35:22.637279 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:35:22.637279 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:35:22.637279 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:35:22.637279 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:35:22.637279 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:35:22.637279 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:35:22.637279 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:35:22.637279 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:35:22.637279 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:35:22.637279 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:35:22.637279 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:35:22.637279 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:35:22.637279 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:35:22.637279 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:35:22.637279 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:35:22.638421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:35:22.638421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:35:22.638421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:35:22.638421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:35:22.638421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:35:22.638421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:35:22.638421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:35:22.638421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:35:22.638421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:35:22.638421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:35:22.638421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:35:22.638421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:35:22.638421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:35:22.638421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:35:22.638421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:35:22.638421 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:35:22.639493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:35:22.639493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:35:22.639493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:35:22.639493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:35:22.639493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:35:22.639493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:35:22.639493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:35:22.639493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:35:22.639493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:35:22.639493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:35:22.639493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:35:22.639493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:35:22.639493 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 578/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:35:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:35:22.644283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a835a-80bd-4a46-8acf-aa47091a94e2 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:22.739125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a835a-80bd-4a46-8acf-aa47091a94e2 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:22.771926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c22993-2fe2-4494-96d0-3cb11e948059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:22.772468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c22993-2fe2-4494-96d0-3cb11e948059 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:22.772752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c22993-2fe2-4494-96d0-3cb11e948059 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:22.773025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c22993-2fe2-4494-96d0-3cb11e948059 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:22.773683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c22993-2fe2-4494-96d0-3cb11e948059 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:35:22.773985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c22993-2fe2-4494-96d0-3cb11e948059 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:22.775365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c22993-2fe2-4494-96d0-3cb11e948059 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:22.783662 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 618/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:35:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:35:22.789421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad224c-9c5f-439e-a8f4-8a02c890282d None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:22.847025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:22.888479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad224c-9c5f-439e-a8f4-8a02c890282d None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:22.920304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9198e9b-c3fb-4028-b6dc-a17a2dafbe38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:22.920915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9198e9b-c3fb-4028-b6dc-a17a2dafbe38 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:22.921194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9198e9b-c3fb-4028-b6dc-a17a2dafbe38 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:22.921468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9198e9b-c3fb-4028-b6dc-a17a2dafbe38 None admin] PATH_INFO: `/v3/users/bb7085047538470a90e0e30aa2757547` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:22.922592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9198e9b-c3fb-4028-b6dc-a17a2dafbe38 None admin] RBAC: Authorizing `identity:get_user(user_id=bb7085047538470a90e0e30aa2757547)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:22.924012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9198e9b-c3fb-4028-b6dc-a17a2dafbe38 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:22.925544 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 579/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Oct 1 20:35:22 2022] GET /identity/v3/users/bb7085047538470a90e0e30aa2757547 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:35:22.931319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c22993-2fe2-4494-96d0-3cb11e948059 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:22.946739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:23.030730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c22993-2fe2-4494-96d0-3cb11e948059 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:23.063566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea7785-2013-4bef-b0d2-d04129bed968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:23.064162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea7785-2013-4bef-b0d2-d04129bed968 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:23.064442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea7785-2013-4bef-b0d2-d04129bed968 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:23.064714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea7785-2013-4bef-b0d2-d04129bed968 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:23.129514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea7785-2013-4bef-b0d2-d04129bed968 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:23.131038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea7785-2013-4bef-b0d2-d04129bed968 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:23.143917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:23.225405 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 619/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:35:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 01 20:35:23.231387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9198e9b-c3fb-4028-b6dc-a17a2dafbe38 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:23.327573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9198e9b-c3fb-4028-b6dc-a17a2dafbe38 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:23.352570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:23.360630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c6169-c9ef-4d25-b4b9-3da47e0c9e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:23.361238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c6169-c9ef-4d25-b4b9-3da47e0c9e78 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:23.361511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c6169-c9ef-4d25-b4b9-3da47e0c9e78 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:23.361778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c6169-c9ef-4d25-b4b9-3da47e0c9e78 None admin] PATH_INFO: `/v3/projects/7ed94ab8cdaf48eaa630ea66cbc139df` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:23.362897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c6169-c9ef-4d25-b4b9-3da47e0c9e78 None admin] RBAC: Authorizing `identity:get_project(project_id=7ed94ab8cdaf48eaa630ea66cbc139df)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:23.364312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c6169-c9ef-4d25-b4b9-3da47e0c9e78 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:23.365895 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 580/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Oct 1 20:35:23 2022] GET /identity/v3/projects/7ed94ab8cdaf48eaa630ea66cbc139df => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:35:23.371942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea7785-2013-4bef-b0d2-d04129bed968 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:35:23.467256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea7785-2013-4bef-b0d2-d04129bed968 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:35:23.480182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:23.500538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fa76a-c275-41c3-ab11-2eedabc76a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:35:23.501129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fa76a-c275-41c3-ab11-2eedabc76a8a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:35:23.501408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fa76a-c275-41c3-ab11-2eedabc76a8a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:35:23.501680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fa76a-c275-41c3-ab11-2eedabc76a8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:35:23.502401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fa76a-c275-41c3-ab11-2eedabc76a8a None admin] RBAC: Adding query filter params (role.id=1c50896511054a5f994bee3d99fa0619, scope.project.id=7ed94ab8cdaf48eaa630ea66cbc139df, user.id=bb7085047538470a90e0e30aa2757547) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:35:23.502736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fa76a-c275-41c3-ab11-2eedabc76a8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:35:23.504138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fa76a-c275-41c3-ab11-2eedabc76a8a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:35:23.509623 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 620/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:35:23 2022] GET /identity/v3/role_assignments?user.id=bb7085047538470a90e0e30aa2757547&scope.project.id=7ed94ab8cdaf48eaa630ea66cbc139df&role.id=1c50896511054a5f994bee3d99fa0619 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:35:33.356706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:35:33.507145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=12 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:26.441049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86257010-eb89-4d73-a67e-cfa950ccbcf4 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:26.441619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86257010-eb89-4d73-a67e-cfa950ccbcf4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:26.441619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86257010-eb89-4d73-a67e-cfa950ccbcf4 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:26.443081 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 581/1201] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:36:26.449480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff3e11-761d-4c91-a96c-96c5d1527dee None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:26.449937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff3e11-761d-4c91-a96c-96c5d1527dee None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:26.450023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff3e11-761d-4c91-a96c-96c5d1527dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:26.470221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-89ff3e11-761d-4c91-a96c-96c5d1527dee None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:36:26.471523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-89ff3e11-761d-4c91-a96c-96c5d1527dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:36:26.557873 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 621/1202] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:36:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:36:26.915933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86257010-eb89-4d73-a67e-cfa950ccbcf4 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:27.044006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86257010-eb89-4d73-a67e-cfa950ccbcf4 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:27.085657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c3bdb-d00f-4392-9924-4186346f9c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:27.086794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c3bdb-d00f-4392-9924-4186346f9c5d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:27.087108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c3bdb-d00f-4392-9924-4186346f9c5d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:27.087385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c3bdb-d00f-4392-9924-4186346f9c5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:27.102000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35c3bdb-d00f-4392-9924-4186346f9c5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:27.104645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35c3bdb-d00f-4392-9924-4186346f9c5d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:27.125308 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-f35c3bdb-d00f-4392-9924-4186346f9c5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:27.125308 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:27.125308 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:27.125308 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:27.125308 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:27.125308 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:27.125308 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:27.125308 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:27.125308 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:27.125308 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:27.125308 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:27.125308 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:27.125308 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:36:27.125308 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:36:27.125308 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:36:27.125308 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:36:27.126589 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:27.126589 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:27.126589 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:27.126589 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:27.126589 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:27.126589 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:27.126589 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:27.126589 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:27.126589 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:27.126589 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:27.126589 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:27.126589 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:27.126589 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:27.126589 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:27.126589 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:27.126589 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:27.126589 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:36:27.127590 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:36:27.127590 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:27.127590 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:36:27.127922 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 582/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:36:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:27.133912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff3e11-761d-4c91-a96c-96c5d1527dee None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:27.277197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff3e11-761d-4c91-a96c-96c5d1527dee None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:27.323549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:27.330638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72985c40-5619-4839-befb-b926566aa442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:27.330638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72985c40-5619-4839-befb-b926566aa442 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:27.330638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72985c40-5619-4839-befb-b926566aa442 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:27.330638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72985c40-5619-4839-befb-b926566aa442 None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:27.331107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72985c40-5619-4839-befb-b926566aa442 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:27.331107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72985c40-5619-4839-befb-b926566aa442 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:27.331107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72985c40-5619-4839-befb-b926566aa442 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:27.360126 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 622/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:27.360358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c3bdb-d00f-4392-9924-4186346f9c5d None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:27.450289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:27.534382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c3bdb-d00f-4392-9924-4186346f9c5d None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:27.591003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da996bc5-8731-4b85-8578-73d540614a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:27.592240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da996bc5-8731-4b85-8578-73d540614a8d None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:27.592600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da996bc5-8731-4b85-8578-73d540614a8d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:27.592880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da996bc5-8731-4b85-8578-73d540614a8d None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:27.593849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da996bc5-8731-4b85-8578-73d540614a8d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:27.595936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da996bc5-8731-4b85-8578-73d540614a8d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:27.638650 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 583/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Oct 1 20:36:27 2022] POST /identity/v3/users => generated 251 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:36:30.541857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9fe03-7bf3-40c0-a95a-1bd21d350e5e None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:30.542454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9fe03-7bf3-40c0-a95a-1bd21d350e5e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:30.542454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9fe03-7bf3-40c0-a95a-1bd21d350e5e None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:30.543832 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 623/1206] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:36:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:36:30.549943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8601a-1bd3-44b2-83f5-dd1a25e56e92 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:30.550641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8601a-1bd3-44b2-83f5-dd1a25e56e92 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:30.550641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8601a-1bd3-44b2-83f5-dd1a25e56e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:30.574133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-94e8601a-1bd3-44b2-83f5-dd1a25e56e92 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:36:30.575443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-94e8601a-1bd3-44b2-83f5-dd1a25e56e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:36:30.686795 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 584/1207] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:36:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:36:31.044760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9fe03-7bf3-40c0-a95a-1bd21d350e5e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:31.161989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9fe03-7bf3-40c0-a95a-1bd21d350e5e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:31.203257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b47f9-9f42-4631-8394-ba8c6bfddcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:31.204358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b47f9-9f42-4631-8394-ba8c6bfddcd5 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:31.204954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b47f9-9f42-4631-8394-ba8c6bfddcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:31.205496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b47f9-9f42-4631-8394-ba8c6bfddcd5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:31.217553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b47f9-9f42-4631-8394-ba8c6bfddcd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:36:31.222758 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-368b47f9-9f42-4631-8394-ba8c6bfddcd5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:36:31.222758 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:31.222758 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:31.222758 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:31.222758 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:31.222758 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:31.222758 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:31.222758 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:31.222758 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:31.222758 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:31.222758 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:31.222758 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:31.222758 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:36:31.222758 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:36:31.222758 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:36:31.222758 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:36:31.223922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:36:31.223922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:36:31.223922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:36:31.223922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:36:31.223922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:31.223922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:31.223922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:31.223922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:31.223922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:31.223922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:31.223922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:31.223922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:31.223922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:31.223922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:31.223922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:31.223922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:31.225061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:31.225061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:31.225061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:31.225061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:31.225061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:36:31.225061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:36:31.225061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:36:31.225061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:36:31.225061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:36:31.225061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:36:31.225061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:36:31.225061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:36:31.230273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8601a-1bd3-44b2-83f5-dd1a25e56e92 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:31.232319 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 624/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:36:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:31.277282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:31.347233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8601a-1bd3-44b2-83f5-dd1a25e56e92 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:31.383274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cce907-c352-45ed-843e-1ad97ef73b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:31.384093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cce907-c352-45ed-843e-1ad97ef73b7c None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:31.384549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cce907-c352-45ed-843e-1ad97ef73b7c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:31.385234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cce907-c352-45ed-843e-1ad97ef73b7c None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:31.386396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cce907-c352-45ed-843e-1ad97ef73b7c None admin] RBAC: Adding query filter params (name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:31.386915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cce907-c352-45ed-843e-1ad97ef73b7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:31.389364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cce907-c352-45ed-843e-1ad97ef73b7c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:31.403640 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 585/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:36:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:31.405094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b47f9-9f42-4631-8394-ba8c6bfddcd5 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:31.468548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:31.517249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b47f9-9f42-4631-8394-ba8c6bfddcd5 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:31.554682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389f767-3f24-48b5-9526-10f478510759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:31.555415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389f767-3f24-48b5-9526-10f478510759 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:31.555616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389f767-3f24-48b5-9526-10f478510759 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:31.555886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389f767-3f24-48b5-9526-10f478510759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:31.566829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389f767-3f24-48b5-9526-10f478510759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:31.568258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389f767-3f24-48b5-9526-10f478510759 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:31.584705 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-c389f767-3f24-48b5-9526-10f478510759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:31.584705 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:31.584705 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:31.584705 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:31.584705 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:31.584705 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:31.584705 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:31.584705 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:31.584705 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:31.584705 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:31.584705 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:31.584705 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:31.584705 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:36:31.584705 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:36:31.584705 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:36:31.584705 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:36:31.586687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:31.586687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:31.586687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:31.586687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:31.586687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:31.586687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:31.586687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:31.586687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:31.586687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:31.586687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:31.586687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:31.586687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:31.586687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:31.586687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:31.586687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:31.586687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:31.586687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:36:31.587778 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:36:31.587778 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:31.587778 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:36:31.587778 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 625/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:36:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:31.592145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cce907-c352-45ed-843e-1ad97ef73b7c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:31.710484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:31.714917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cce907-c352-45ed-843e-1ad97ef73b7c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:31.756671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d229c7-a747-42cf-aaac-0d5d50a42f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:31.757389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d229c7-a747-42cf-aaac-0d5d50a42f29 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:31.757655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d229c7-a747-42cf-aaac-0d5d50a42f29 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:31.757979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d229c7-a747-42cf-aaac-0d5d50a42f29 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:31.758676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d229c7-a747-42cf-aaac-0d5d50a42f29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:31.759398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d229c7-a747-42cf-aaac-0d5d50a42f29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:31.760597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d229c7-a747-42cf-aaac-0d5d50a42f29 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:31.778703 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 586/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:36:31.779907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389f767-3f24-48b5-9526-10f478510759 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:31.893561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389f767-3f24-48b5-9526-10f478510759 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:31.895278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:31.931096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c51de-6f67-4f1a-b8a0-b50489ba2775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:31.931745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c51de-6f67-4f1a-b8a0-b50489ba2775 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:31.932553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c51de-6f67-4f1a-b8a0-b50489ba2775 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:31.932890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c51de-6f67-4f1a-b8a0-b50489ba2775 None admin] PATH_INFO: `/v3/users/placement` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:31.941189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c51de-6f67-4f1a-b8a0-b50489ba2775 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:31.942922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c51de-6f67-4f1a-b8a0-b50489ba2775 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:31.956228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-7d7c51de-6f67-4f1a-b8a0-b50489ba2775 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 01 20:36:31.956228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:31.956228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:31.956228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:31.956228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:31.956228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:31.956228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:31.956228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:31.956228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:31.956228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:31.956228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:31.956228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:31.956228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:36:31.956228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:36:31.956228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:36:31.956228 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:36:31.957242 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:31.957242 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:31.957242 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:36:31.957242 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:36:31.957242 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:36:31.957242 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:36:31.957242 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:31.957242 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:31.957242 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:31.957242 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:31.957242 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:31.957242 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:31.957242 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:31.957242 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:31.957242 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:31.957242 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:31.957242 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:31.957242 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:31.958212 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:31.958212 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:31.958212 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:36:31.958212 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:36:31.958212 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:36:31.958212 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:36:31.958212 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:36:31.958212 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:36:31.958212 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 01 20:36:31.958212 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:36:31.958212 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 626/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:36:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:31.963915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d229c7-a747-42cf-aaac-0d5d50a42f29 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:32.080616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d229c7-a747-42cf-aaac-0d5d50a42f29 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:32.122222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975c751-1fd7-4850-a0a0-5a2510102a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:32.122870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975c751-1fd7-4850-a0a0-5a2510102a4d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:32.123148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975c751-1fd7-4850-a0a0-5a2510102a4d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:32.123473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975c751-1fd7-4850-a0a0-5a2510102a4d None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:32.124155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a975c751-1fd7-4850-a0a0-5a2510102a4d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:32.124466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a975c751-1fd7-4850-a0a0-5a2510102a4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:32.125897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a975c751-1fd7-4850-a0a0-5a2510102a4d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:32.147256 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 587/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Sat Oct 1 20:36:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:32.153447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c51de-6f67-4f1a-b8a0-b50489ba2775 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:32.274660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c51de-6f67-4f1a-b8a0-b50489ba2775 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:32.323134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f974c-ee31-4e23-b3a8-8f7a933c81ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:32.324467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f974c-ee31-4e23-b3a8-8f7a933c81ad None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:32.324972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f974c-ee31-4e23-b3a8-8f7a933c81ad None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:32.325453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f974c-ee31-4e23-b3a8-8f7a933c81ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:32.370412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:32.425098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f974c-ee31-4e23-b3a8-8f7a933c81ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:32.426511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f974c-ee31-4e23-b3a8-8f7a933c81ad None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:32.544713 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 627/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:36:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 01 20:36:32.546059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975c751-1fd7-4850-a0a0-5a2510102a4d None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:32.567029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:32.657747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975c751-1fd7-4850-a0a0-5a2510102a4d None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:32.695698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44188bb2-e30c-4a45-b4ab-1a5172574810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:32.696545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44188bb2-e30c-4a45-b4ab-1a5172574810 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:32.696884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44188bb2-e30c-4a45-b4ab-1a5172574810 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:32.697288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44188bb2-e30c-4a45-b4ab-1a5172574810 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:32.710179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44188bb2-e30c-4a45-b4ab-1a5172574810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:32.712006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44188bb2-e30c-4a45-b4ab-1a5172574810 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:32.732622 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-44188bb2-e30c-4a45-b4ab-1a5172574810 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:32.732622 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:32.732622 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:32.732622 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:32.732622 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:32.732622 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:32.732622 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:32.732622 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:32.732622 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:32.732622 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:32.732622 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:32.732622 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:32.732622 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:36:32.732622 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:36:32.732622 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:36:32.732622 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:36:32.739055 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:32.739055 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:32.739055 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:32.739055 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:32.739055 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:32.739055 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:32.739055 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:32.739055 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:32.739055 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:32.739055 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:32.739055 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:32.739055 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:32.739055 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:32.739055 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:32.739055 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:32.739055 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:32.739055 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:36:32.740179 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:36:32.740179 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:32.740179 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:36:32.740179 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 588/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:36:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:32.740473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f974c-ee31-4e23-b3a8-8f7a933c81ad None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:32.807865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:32.849272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f974c-ee31-4e23-b3a8-8f7a933c81ad None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:32.884930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1342a-633f-4027-9dcd-f589ed84adfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:32.885536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1342a-633f-4027-9dcd-f589ed84adfb None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:32.885816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1342a-633f-4027-9dcd-f589ed84adfb None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:32.886085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1342a-633f-4027-9dcd-f589ed84adfb None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:32.886812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb1342a-633f-4027-9dcd-f589ed84adfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:32.887115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb1342a-633f-4027-9dcd-f589ed84adfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:32.888458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb1342a-633f-4027-9dcd-f589ed84adfb None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:32.899788 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 628/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:36:32.905729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44188bb2-e30c-4a45-b4ab-1a5172574810 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:32.980739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:33.015715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44188bb2-e30c-4a45-b4ab-1a5172574810 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:33.065947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93103989-75f4-4e7f-bbcd-ab5a7d7dd9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:33.066613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93103989-75f4-4e7f-bbcd-ab5a7d7dd9d1 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:33.066895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93103989-75f4-4e7f-bbcd-ab5a7d7dd9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:33.067166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93103989-75f4-4e7f-bbcd-ab5a7d7dd9d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:33.075519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93103989-75f4-4e7f-bbcd-ab5a7d7dd9d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:33.077047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93103989-75f4-4e7f-bbcd-ab5a7d7dd9d1 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:33.092170 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-93103989-75f4-4e7f-bbcd-ab5a7d7dd9d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:36:33.092170 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:33.092170 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:33.092170 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:33.092170 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:33.092170 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:33.092170 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:33.092170 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:33.092170 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:33.092170 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:33.092170 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:33.092170 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:33.092170 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:36:33.092170 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:36:33.092170 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:36:33.093420 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:36:33.093420 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:33.093420 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:33.093420 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:33.093420 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:33.093420 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:33.093420 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:33.093420 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:33.093420 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:33.093420 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:33.093420 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:33.093420 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:33.093420 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:33.093420 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:33.093420 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:33.093420 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:33.093420 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:33.094649 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:36:33.094649 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:36:33.094649 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:36:33.094649 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:36:33.094649 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:36:33.094649 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:36:33.094649 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:36:33.094649 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:36:33.094649 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 589/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:36:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:33.099427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1342a-633f-4027-9dcd-f589ed84adfb None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:33.220998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1342a-633f-4027-9dcd-f589ed84adfb None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:33.248183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:33.263855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487366e-2f31-4c31-85b5-ea4bf17b89c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:33.264548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487366e-2f31-4c31-85b5-ea4bf17b89c3 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:33.264895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487366e-2f31-4c31-85b5-ea4bf17b89c3 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:33.265231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487366e-2f31-4c31-85b5-ea4bf17b89c3 None admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:33.266029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487366e-2f31-4c31-85b5-ea4bf17b89c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:33.266338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487366e-2f31-4c31-85b5-ea4bf17b89c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:33.267984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487366e-2f31-4c31-85b5-ea4bf17b89c3 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:33.283969 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 629/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:36:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:33.290106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93103989-75f4-4e7f-bbcd-ab5a7d7dd9d1 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:33.427703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93103989-75f4-4e7f-bbcd-ab5a7d7dd9d1 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:33.461742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:33.475133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d315f4b-7cd2-473b-997e-e4ba63501391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:33.475742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d315f4b-7cd2-473b-997e-e4ba63501391 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:33.476016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d315f4b-7cd2-473b-997e-e4ba63501391 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:33.476283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d315f4b-7cd2-473b-997e-e4ba63501391 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:33.477017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d315f4b-7cd2-473b-997e-e4ba63501391 None admin] RBAC: Adding query filter params (role.id=ef7bd7293369475a9e7b3444fac30cda, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=1e5f6284cd7d4973adffefa98291dbdb) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:33.477314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d315f4b-7cd2-473b-997e-e4ba63501391 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:33.478909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d315f4b-7cd2-473b-997e-e4ba63501391 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:33.489308 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 590/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:36:33 2022] GET /identity/v3/role_assignments?user.id=1e5f6284cd7d4973adffefa98291dbdb&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=ef7bd7293369475a9e7b3444fac30cda => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:35.941889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916e1aa-bb76-4071-9648-ac8e953da994 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:35.942467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916e1aa-bb76-4071-9648-ac8e953da994 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:35.942467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916e1aa-bb76-4071-9648-ac8e953da994 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:35.943809 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 630/1220] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:36:35.950363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b297ea-c395-4588-8282-88cd6c0060fc None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:35.950888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b297ea-c395-4588-8282-88cd6c0060fc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:35.951034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b297ea-c395-4588-8282-88cd6c0060fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:35.971124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a8b297ea-c395-4588-8282-88cd6c0060fc None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:36:35.972694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a8b297ea-c395-4588-8282-88cd6c0060fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:36:36.047930 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 591/1221] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:36:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:36:36.373816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916e1aa-bb76-4071-9648-ac8e953da994 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:36.488799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916e1aa-bb76-4071-9648-ac8e953da994 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:36.526801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00bd8f-8a06-413b-8cf4-dbf48ef0e095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:36.527569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00bd8f-8a06-413b-8cf4-dbf48ef0e095 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:36.527842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00bd8f-8a06-413b-8cf4-dbf48ef0e095 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:36.528177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00bd8f-8a06-413b-8cf4-dbf48ef0e095 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:36.540505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00bd8f-8a06-413b-8cf4-dbf48ef0e095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:36:36.546106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-af00bd8f-8a06-413b-8cf4-dbf48ef0e095 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:36:36.546106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:36.546106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:36.546106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:36.546106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:36.546106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:36.546106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:36.546106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:36.546106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:36.546106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:36.546106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:36.546106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:36.546106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:36:36.546106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:36:36.546106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:36:36.546106 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:36:36.547309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:36:36.547309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:36:36.547309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:36:36.547309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:36:36.547309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:36.547309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:36.547309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:36.547309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:36.547309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:36.547309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:36.547309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:36.547309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:36.547309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:36.547309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:36.547309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:36.547309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:36.548146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:36.548146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:36.548146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:36.548146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:36.548146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:36:36.548146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:36:36.548146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:36:36.548146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:36:36.548146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:36:36.548146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:36:36.548146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:36:36.548146 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:36:36.548146 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 631/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:36:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:36.553346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b297ea-c395-4588-8282-88cd6c0060fc None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:36.664182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b297ea-c395-4588-8282-88cd6c0060fc None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:36.688272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:36.700931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a97ca1-ceb4-4f06-8414-5e8919ed2f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:36.701556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a97ca1-ceb4-4f06-8414-5e8919ed2f9b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:36.701869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a97ca1-ceb4-4f06-8414-5e8919ed2f9b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:36.702157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a97ca1-ceb4-4f06-8414-5e8919ed2f9b None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:36.702873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a97ca1-ceb4-4f06-8414-5e8919ed2f9b None admin] RBAC: Adding query filter params (name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:36.703225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a97ca1-ceb4-4f06-8414-5e8919ed2f9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:36.704583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a97ca1-ceb4-4f06-8414-5e8919ed2f9b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:36.713666 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 592/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:36:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:36.719899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00bd8f-8a06-413b-8cf4-dbf48ef0e095 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:36.773352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:36.838980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00bd8f-8a06-413b-8cf4-dbf48ef0e095 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:36.876699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759a637-607c-4741-84a2-933882ae3c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:36.877332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759a637-607c-4741-84a2-933882ae3c72 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:36.877684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759a637-607c-4741-84a2-933882ae3c72 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:36.878032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759a637-607c-4741-84a2-933882ae3c72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:36.889471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759a637-607c-4741-84a2-933882ae3c72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:36.891002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759a637-607c-4741-84a2-933882ae3c72 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:36.905654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-c759a637-607c-4741-84a2-933882ae3c72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:36.905654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:36.905654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:36.905654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:36.905654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:36.905654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:36.905654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:36.905654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:36.905654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:36.905654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:36.905654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:36.905654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:36.905654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:36:36.905654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:36:36.905654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:36:36.905654 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:36:36.906687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:36.906687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:36.906687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:36.906687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:36.906687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:36.906687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:36.906687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:36.906687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:36.906687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:36.906687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:36.906687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:36.906687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:36.906687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:36.906687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:36.906687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:36.906687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:36.906687 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:36:36.907643 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:36:36.907643 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:36.907643 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:36:36.907643 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 632/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:36:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:36:36.912691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a97ca1-ceb4-4f06-8414-5e8919ed2f9b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:37.019988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a97ca1-ceb4-4f06-8414-5e8919ed2f9b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:37.057557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f01870-c628-4fc6-b392-aadabfef5281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:37.058279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f01870-c628-4fc6-b392-aadabfef5281 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:37.058600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f01870-c628-4fc6-b392-aadabfef5281 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:37.058898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f01870-c628-4fc6-b392-aadabfef5281 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:37.059559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f01870-c628-4fc6-b392-aadabfef5281 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:37.059867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f01870-c628-4fc6-b392-aadabfef5281 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:37.061470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f01870-c628-4fc6-b392-aadabfef5281 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:37.075827 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 593/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:37.081291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759a637-607c-4741-84a2-933882ae3c72 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:37.181059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:37.188524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759a637-607c-4741-84a2-933882ae3c72 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:37.223133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7181fcf5-b512-4b36-bce3-b16c9e9ab9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:37.223754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181fcf5-b512-4b36-bce3-b16c9e9ab9cb None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:37.224034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181fcf5-b512-4b36-bce3-b16c9e9ab9cb None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:37.224306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181fcf5-b512-4b36-bce3-b16c9e9ab9cb None admin] PATH_INFO: `/v3/users/placement` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:37.232941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7181fcf5-b512-4b36-bce3-b16c9e9ab9cb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:37.234640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7181fcf5-b512-4b36-bce3-b16c9e9ab9cb None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:37.246483 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-7181fcf5-b512-4b36-bce3-b16c9e9ab9cb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 01 20:36:37.246483 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:37.246483 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:37.246483 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:37.246483 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:37.246483 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:37.246483 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:37.246483 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:37.246483 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:37.246483 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:37.246483 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:37.246483 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:37.246483 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:36:37.246483 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:36:37.246483 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:36:37.246483 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:36:37.247727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:37.247727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:37.247727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:36:37.247727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:36:37.247727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:36:37.247727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:36:37.247727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:37.247727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:37.247727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:37.247727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:37.247727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:37.247727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:37.247727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:37.247727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:37.247727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:37.247727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:37.247727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:37.247727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:37.248991 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:37.248991 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:37.248991 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:36:37.248991 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:36:37.248991 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:36:37.248991 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:36:37.248991 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:36:37.248991 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:36:37.248991 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 01 20:36:37.248991 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:36:37.248991 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 633/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:36:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 01 20:36:37.255815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f01870-c628-4fc6-b392-aadabfef5281 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:37.306291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:37.364989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f01870-c628-4fc6-b392-aadabfef5281 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:37.402736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f787cc5-4286-4712-9cef-08e4617619cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:37.403339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f787cc5-4286-4712-9cef-08e4617619cc None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:37.403616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f787cc5-4286-4712-9cef-08e4617619cc None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:37.403885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f787cc5-4286-4712-9cef-08e4617619cc None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:37.404601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f787cc5-4286-4712-9cef-08e4617619cc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:37.404903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f787cc5-4286-4712-9cef-08e4617619cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:37.406322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f787cc5-4286-4712-9cef-08e4617619cc None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:37.424422 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 594/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Sat Oct 1 20:36:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:37.430551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7181fcf5-b512-4b36-bce3-b16c9e9ab9cb None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:37.551326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7181fcf5-b512-4b36-bce3-b16c9e9ab9cb None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:37.590166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeac427-245f-4d97-a5a9-734722e8e74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:37.590738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeac427-245f-4d97-a5a9-734722e8e74c None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:37.591021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeac427-245f-4d97-a5a9-734722e8e74c None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:37.591292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeac427-245f-4d97-a5a9-734722e8e74c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:37.601996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeac427-245f-4d97-a5a9-734722e8e74c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:37.603615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeac427-245f-4d97-a5a9-734722e8e74c None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:37.617851 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-8eeac427-245f-4d97-a5a9-734722e8e74c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:37.617851 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:37.617851 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:37.617851 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:37.617851 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:37.617851 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:37.617851 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:37.617851 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:37.617851 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:37.617851 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:37.617851 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:37.617851 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:37.617851 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:36:37.617851 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:36:37.617851 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:36:37.617851 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:36:37.619061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:37.619061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:37.619061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:37.619061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:37.619061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:37.619061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:37.619061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:37.619061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:37.619061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:37.619061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:37.619061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:37.619061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:37.619061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:37.619061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:37.619061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:37.619061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:37.619061 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:36:37.620017 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:36:37.620017 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:37.620017 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:36:37.620017 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 634/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:36:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:37.624933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f787cc5-4286-4712-9cef-08e4617619cc None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:37.728859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f787cc5-4286-4712-9cef-08e4617619cc None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:37.731723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:37.767448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb835b-ff08-4289-bde2-585b1c86ac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:37.768053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb835b-ff08-4289-bde2-585b1c86ac57 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:37.768329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb835b-ff08-4289-bde2-585b1c86ac57 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:37.768598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb835b-ff08-4289-bde2-585b1c86ac57 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:37.769237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb835b-ff08-4289-bde2-585b1c86ac57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:37.769535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb835b-ff08-4289-bde2-585b1c86ac57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:37.770935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb835b-ff08-4289-bde2-585b1c86ac57 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:37.782897 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 595/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:36:37.789121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeac427-245f-4d97-a5a9-734722e8e74c None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:37.799260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:37.896164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeac427-245f-4d97-a5a9-734722e8e74c None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:37.935738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3c5a5-646e-4425-92b5-71c441a3a698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:37.936494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3c5a5-646e-4425-92b5-71c441a3a698 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:37.936921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3c5a5-646e-4425-92b5-71c441a3a698 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:37.937268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3c5a5-646e-4425-92b5-71c441a3a698 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:37.948342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3c5a5-646e-4425-92b5-71c441a3a698 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:37.949769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3c5a5-646e-4425-92b5-71c441a3a698 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:37.960867 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-64a3c5a5-646e-4425-92b5-71c441a3a698 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:36:37.960867 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:37.960867 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:37.960867 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:37.960867 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:37.960867 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:37.960867 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:37.960867 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:37.960867 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:37.960867 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:37.960867 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:37.960867 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:37.960867 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:36:37.960867 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:36:37.960867 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:36:37.961835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:36:37.961835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:37.961835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:37.961835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:37.961835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:37.961835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:37.961835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:37.961835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:37.961835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:37.961835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:37.961835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:37.961835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:37.961835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:37.961835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:37.961835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:37.961835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:37.961835 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:37.962764 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:36:37.962764 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:36:37.962764 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:36:37.962764 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:36:37.962764 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:36:37.962764 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:36:37.962764 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:36:37.962764 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:36:37.962764 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 635/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:36:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:37.967783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb835b-ff08-4289-bde2-585b1c86ac57 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:38.080377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb835b-ff08-4289-bde2-585b1c86ac57 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:38.116485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988bda8-2cfd-44af-94b4-756152b939b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:38.117088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988bda8-2cfd-44af-94b4-756152b939b1 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:38.117369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988bda8-2cfd-44af-94b4-756152b939b1 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:38.117645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988bda8-2cfd-44af-94b4-756152b939b1 None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:38.118330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988bda8-2cfd-44af-94b4-756152b939b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:38.118674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988bda8-2cfd-44af-94b4-756152b939b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:38.120632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988bda8-2cfd-44af-94b4-756152b939b1 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:38.132502 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 596/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:36:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:38.139224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3c5a5-646e-4425-92b5-71c441a3a698 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:38.208917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:38.246319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3c5a5-646e-4425-92b5-71c441a3a698 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:38.282123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247993e-98ec-4127-b059-b054af23ea2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:38.282820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247993e-98ec-4127-b059-b054af23ea2c None admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:38.283108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247993e-98ec-4127-b059-b054af23ea2c None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:38.283384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247993e-98ec-4127-b059-b054af23ea2c None admin] PATH_INFO: `/v3/projects/b7875946723844198353bbfcf22516fb/users/1e5f6284cd7d4973adffefa98291dbdb/roles/ef7bd7293369475a9e7b3444fac30cda` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:38.297958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247993e-98ec-4127-b059-b054af23ea2c None admin] RBAC: Authorizing `identity:create_grant(project_id=b7875946723844198353bbfcf22516fb, user_id=1e5f6284cd7d4973adffefa98291dbdb, role_id=ef7bd7293369475a9e7b3444fac30cda)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:38.299435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247993e-98ec-4127-b059-b054af23ea2c None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:38.312282 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 636/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:36:38 2022] PUT /identity/v3/projects/b7875946723844198353bbfcf22516fb/users/1e5f6284cd7d4973adffefa98291dbdb/roles/ef7bd7293369475a9e7b3444fac30cda => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:36:40.775766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a4741-492d-4862-b360-217122be1740 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:40.776327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a4741-492d-4862-b360-217122be1740 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:40.776569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a4741-492d-4862-b360-217122be1740 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:40.778341 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 597/1233] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:36:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:36:40.783755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca43afe-bc01-4640-9315-11240c21ad94 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:40.784156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca43afe-bc01-4640-9315-11240c21ad94 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:40.784312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca43afe-bc01-4640-9315-11240c21ad94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:40.803721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5ca43afe-bc01-4640-9315-11240c21ad94 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:36:40.804975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5ca43afe-bc01-4640-9315-11240c21ad94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:36:40.880528 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 637/1234] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:36:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 01 20:36:41.221835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a4741-492d-4862-b360-217122be1740 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:41.251594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:41.341764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a4741-492d-4862-b360-217122be1740 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:41.379514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dfbd65-2905-4d10-8ff6-0679a0367f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:41.380177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfbd65-2905-4d10-8ff6-0679a0367f60 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:41.380457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfbd65-2905-4d10-8ff6-0679a0367f60 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:41.380830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfbd65-2905-4d10-8ff6-0679a0367f60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:41.391645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dfbd65-2905-4d10-8ff6-0679a0367f60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:36:41.396469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-48dfbd65-2905-4d10-8ff6-0679a0367f60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:36:41.396469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:41.396469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:41.396469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:41.396469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:41.396469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:41.396469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:41.396469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:41.396469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:41.396469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:41.396469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:41.396469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:41.396469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:36:41.396469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:36:41.396469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:36:41.396469 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:36:41.397843 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:36:41.397843 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:36:41.397843 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:36:41.397843 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:36:41.397843 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:41.397843 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:41.397843 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:41.397843 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:41.397843 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:41.397843 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:41.397843 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:41.397843 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:41.397843 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:41.397843 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:41.397843 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:41.397843 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:41.398805 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:41.398805 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:41.398805 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:41.398805 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:41.398805 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:36:41.398805 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:36:41.398805 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:36:41.398805 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:36:41.398805 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:36:41.398805 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:36:41.398805 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 01 20:36:41.398805 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:36:41.398805 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 598/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Oct 1 20:36:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:41.404169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca43afe-bc01-4640-9315-11240c21ad94 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:41.456743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:41.516774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca43afe-bc01-4640-9315-11240c21ad94 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:41.559526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a079d3-a573-4e56-8cdd-1ccfe063d843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:41.560181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a079d3-a573-4e56-8cdd-1ccfe063d843 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:41.560445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a079d3-a573-4e56-8cdd-1ccfe063d843 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:41.560722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a079d3-a573-4e56-8cdd-1ccfe063d843 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:41.561391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a079d3-a573-4e56-8cdd-1ccfe063d843 None admin] RBAC: Adding query filter params (name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:41.561714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a079d3-a573-4e56-8cdd-1ccfe063d843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:41.563115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a079d3-a573-4e56-8cdd-1ccfe063d843 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:41.571420 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 638/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Oct 1 20:36:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:41.577383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dfbd65-2905-4d10-8ff6-0679a0367f60 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:41.669088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:41.688306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dfbd65-2905-4d10-8ff6-0679a0367f60 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:41.727516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4511e7-d577-4ebf-87b8-10c58bc0a8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:41.728078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4511e7-d577-4ebf-87b8-10c58bc0a8ba None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:41.728405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4511e7-d577-4ebf-87b8-10c58bc0a8ba None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:41.729320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4511e7-d577-4ebf-87b8-10c58bc0a8ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:41.742500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4511e7-d577-4ebf-87b8-10c58bc0a8ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:41.744022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4511e7-d577-4ebf-87b8-10c58bc0a8ba None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:41.759267 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-5c4511e7-d577-4ebf-87b8-10c58bc0a8ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:41.759267 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:41.759267 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:41.759267 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:41.759267 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:41.759267 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:41.759267 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:41.759267 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:41.759267 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:41.759267 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:41.759267 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:41.759267 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:41.759267 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:36:41.759267 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:36:41.759267 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:36:41.759267 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:36:41.760945 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:41.760945 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:41.760945 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:41.760945 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:41.760945 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:41.760945 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:41.760945 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:41.760945 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:41.760945 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:41.760945 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:41.760945 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:41.760945 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:41.760945 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:41.760945 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:41.760945 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:41.760945 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:41.760945 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:36:41.761994 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:36:41.761994 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:41.761994 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:36:41.761994 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 599/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:36:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:41.766689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a079d3-a573-4e56-8cdd-1ccfe063d843 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:41.872695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a079d3-a573-4e56-8cdd-1ccfe063d843 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:41.882693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:41.913926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09a24d-e2c6-426b-81ff-6fcbda6b6d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:41.914692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09a24d-e2c6-426b-81ff-6fcbda6b6d3a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:41.915070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09a24d-e2c6-426b-81ff-6fcbda6b6d3a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:41.915347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09a24d-e2c6-426b-81ff-6fcbda6b6d3a None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:41.916623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09a24d-e2c6-426b-81ff-6fcbda6b6d3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:41.916927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09a24d-e2c6-426b-81ff-6fcbda6b6d3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:41.918292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09a24d-e2c6-426b-81ff-6fcbda6b6d3a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:41.931050 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 639/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:36:41.936914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4511e7-d577-4ebf-87b8-10c58bc0a8ba None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:42.052487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4511e7-d577-4ebf-87b8-10c58bc0a8ba None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:42.093250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb287e69-0df1-4cdf-bcf9-1788bb455ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:42.094015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb287e69-0df1-4cdf-bcf9-1788bb455ea4 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:42.094374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb287e69-0df1-4cdf-bcf9-1788bb455ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:42.094755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb287e69-0df1-4cdf-bcf9-1788bb455ea4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:42.097483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:42.106629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb287e69-0df1-4cdf-bcf9-1788bb455ea4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:42.108665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb287e69-0df1-4cdf-bcf9-1788bb455ea4 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:42.121491 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-eb287e69-0df1-4cdf-bcf9-1788bb455ea4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 01 20:36:42.121491 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:42.121491 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:42.121491 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:42.121491 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:42.121491 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:42.121491 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:42.121491 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:42.121491 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:42.121491 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:42.121491 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:42.121491 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:42.121491 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:36:42.121491 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:36:42.121491 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:36:42.121491 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:36:42.122735 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:42.122735 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:42.122735 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:36:42.122735 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:36:42.122735 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:36:42.122735 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:36:42.122735 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:42.122735 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:42.122735 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:42.122735 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:42.122735 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:42.122735 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:42.122735 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:42.122735 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:42.122735 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:42.122735 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:42.122735 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:42.122735 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:42.123729 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:42.123729 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:42.123729 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:36:42.123729 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:36:42.123729 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:36:42.123729 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:36:42.123729 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:36:42.123729 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:36:42.123729 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 01 20:36:42.123729 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:36:42.123729 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 600/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:36:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:42.128574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09a24d-e2c6-426b-81ff-6fcbda6b6d3a None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:42.237469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09a24d-e2c6-426b-81ff-6fcbda6b6d3a None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:42.274858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453be30-b82e-4223-b899-5b109e98119c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:42.275543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453be30-b82e-4223-b899-5b109e98119c None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:42.275798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453be30-b82e-4223-b899-5b109e98119c None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:42.276067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453be30-b82e-4223-b899-5b109e98119c None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:42.276794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453be30-b82e-4223-b899-5b109e98119c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:42.277099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453be30-b82e-4223-b899-5b109e98119c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:42.278533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453be30-b82e-4223-b899-5b109e98119c None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:42.297063 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 640/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Sat Oct 1 20:36:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:42.303049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb287e69-0df1-4cdf-bcf9-1788bb455ea4 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:42.414742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb287e69-0df1-4cdf-bcf9-1788bb455ea4 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:42.453981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba0e8f-8c05-4c39-9f4c-c9dd8e8c6a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:42.454669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba0e8f-8c05-4c39-9f4c-c9dd8e8c6a9b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:42.454958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba0e8f-8c05-4c39-9f4c-c9dd8e8c6a9b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:42.455244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba0e8f-8c05-4c39-9f4c-c9dd8e8c6a9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:42.531866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:42.539636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba0e8f-8c05-4c39-9f4c-c9dd8e8c6a9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:42.541119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba0e8f-8c05-4c39-9f4c-c9dd8e8c6a9b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:42.646070 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 601/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:36:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 01 20:36:42.653941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453be30-b82e-4223-b899-5b109e98119c None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:42.683411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:42.779774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453be30-b82e-4223-b899-5b109e98119c None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:42.819292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137b830-00f0-403f-a69b-f4b4c79fda6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:42.819806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137b830-00f0-403f-a69b-f4b4c79fda6e None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:42.820090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137b830-00f0-403f-a69b-f4b4c79fda6e None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:42.820357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137b830-00f0-403f-a69b-f4b4c79fda6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:42.831933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137b830-00f0-403f-a69b-f4b4c79fda6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:42.833701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137b830-00f0-403f-a69b-f4b4c79fda6e None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:42.852537 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-4137b830-00f0-403f-a69b-f4b4c79fda6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:42.852537 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:42.852537 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:42.852537 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:42.852537 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:42.852537 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:42.852537 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:42.852537 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:42.852537 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:42.852537 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:42.852537 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:42.852537 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:42.852537 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:36:42.852537 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:36:42.852537 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:36:42.852537 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:36:42.853794 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:42.853794 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:42.853794 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:42.853794 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:42.853794 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:42.853794 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:42.853794 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:42.853794 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:42.853794 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:42.853794 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:42.853794 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:42.853794 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:42.853794 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:42.853794 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:42.853794 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:42.853794 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:42.853794 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:36:42.855038 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:36:42.855038 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:42.855038 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:36:42.859712 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 641/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:36:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:42.860652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba0e8f-8c05-4c39-9f4c-c9dd8e8c6a9b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:42.968883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:42.976091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba0e8f-8c05-4c39-9f4c-c9dd8e8c6a9b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:43.015132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22240e9b-9952-4289-a674-8e697856d04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:43.015742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22240e9b-9952-4289-a674-8e697856d04d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:43.016119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22240e9b-9952-4289-a674-8e697856d04d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:43.016478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22240e9b-9952-4289-a674-8e697856d04d None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:43.017257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22240e9b-9952-4289-a674-8e697856d04d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:43.017557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22240e9b-9952-4289-a674-8e697856d04d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:43.019145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22240e9b-9952-4289-a674-8e697856d04d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:43.039241 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 602/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:36:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:43.040610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137b830-00f0-403f-a69b-f4b4c79fda6e None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:43.132427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:43.163752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137b830-00f0-403f-a69b-f4b4c79fda6e None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:43.211495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4dd19-9f3c-40fa-b989-3c823820b280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:43.212448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4dd19-9f3c-40fa-b989-3c823820b280 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:43.213192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4dd19-9f3c-40fa-b989-3c823820b280 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:43.213594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4dd19-9f3c-40fa-b989-3c823820b280 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:43.222812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4dd19-9f3c-40fa-b989-3c823820b280 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:43.224251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4dd19-9f3c-40fa-b989-3c823820b280 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:43.241740 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-e1c4dd19-9f3c-40fa-b989-3c823820b280 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:36:43.241740 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:43.241740 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:43.241740 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:43.241740 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:43.241740 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:43.241740 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:43.241740 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:43.241740 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:43.241740 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:43.241740 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:43.241740 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:43.241740 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:36:43.241740 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:36:43.241740 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:36:43.242974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:36:43.242974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:43.242974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:43.242974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:43.242974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:43.242974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:43.242974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:43.242974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:43.242974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:43.242974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:43.242974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:43.242974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:43.242974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:43.242974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:43.242974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:43.242974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:43.242974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:43.244639 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:36:43.244639 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:36:43.244639 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:36:43.244639 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:36:43.244639 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:36:43.244639 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:36:43.244639 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:36:43.244639 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:36:43.244639 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 642/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:36:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:43.249601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22240e9b-9952-4289-a674-8e697856d04d None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:43.366110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22240e9b-9952-4289-a674-8e697856d04d None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:43.404541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6a6c9-3c80-476f-bb91-a5acf46c8f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:43.405067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6a6c9-3c80-476f-bb91-a5acf46c8f8d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:43.405350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6a6c9-3c80-476f-bb91-a5acf46c8f8d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:43.405626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6a6c9-3c80-476f-bb91-a5acf46c8f8d None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:43.406324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6a6c9-3c80-476f-bb91-a5acf46c8f8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:43.406664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6a6c9-3c80-476f-bb91-a5acf46c8f8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:43.408190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6a6c9-3c80-476f-bb91-a5acf46c8f8d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:43.421620 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 603/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:36:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:43.428120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4dd19-9f3c-40fa-b989-3c823820b280 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:43.547364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4dd19-9f3c-40fa-b989-3c823820b280 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:43.587197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3517e9d-0841-4d45-af60-8217e30e02d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:43.587860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3517e9d-0841-4d45-af60-8217e30e02d1 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:43.588689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3517e9d-0841-4d45-af60-8217e30e02d1 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:43.588969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3517e9d-0841-4d45-af60-8217e30e02d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:43.589879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3517e9d-0841-4d45-af60-8217e30e02d1 None admin] RBAC: Adding query filter params (role.id=ef7bd7293369475a9e7b3444fac30cda, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=1e5f6284cd7d4973adffefa98291dbdb) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:43.590175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3517e9d-0841-4d45-af60-8217e30e02d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:43.591813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3517e9d-0841-4d45-af60-8217e30e02d1 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:43.598732 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 643/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:36:43 2022] GET /identity/v3/role_assignments?user.id=1e5f6284cd7d4973adffefa98291dbdb&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=ef7bd7293369475a9e7b3444fac30cda => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:46.185413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efcea4-7e16-4f45-89c2-85135d535932 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:46.185946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efcea4-7e16-4f45-89c2-85135d535932 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:46.185946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efcea4-7e16-4f45-89c2-85135d535932 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:46.187447 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 604/1247] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:36:46.192463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc931a-c44e-41de-ab9e-817d7d049449 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:46.192861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc931a-c44e-41de-ab9e-817d7d049449 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:46.193021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc931a-c44e-41de-ab9e-817d7d049449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:46.199741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:46.213694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4bcc931a-c44e-41de-ab9e-817d7d049449 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:36:46.214928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4bcc931a-c44e-41de-ab9e-817d7d049449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:36:46.281016 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 644/1248] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:36:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:36:46.611629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efcea4-7e16-4f45-89c2-85135d535932 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:46.629412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:46.730224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efcea4-7e16-4f45-89c2-85135d535932 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:46.774836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88e9d3-f135-4c1c-8c5b-c5f90bda21c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:46.776070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88e9d3-f135-4c1c-8c5b-c5f90bda21c8 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:46.776351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88e9d3-f135-4c1c-8c5b-c5f90bda21c8 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:46.776627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88e9d3-f135-4c1c-8c5b-c5f90bda21c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:46.786762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88e9d3-f135-4c1c-8c5b-c5f90bda21c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:36:46.791511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-de88e9d3-f135-4c1c-8c5b-c5f90bda21c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:36:46.791511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:46.791511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:46.791511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:46.791511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:46.791511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:46.791511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:46.791511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:46.791511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:46.791511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:46.791511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:46.791511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:46.791511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:36:46.791511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:36:46.791511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:36:46.791511 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:36:46.792602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:36:46.792602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:36:46.792602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:36:46.792602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:36:46.792602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:46.792602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:46.792602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:46.792602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:46.792602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:46.792602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:46.792602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:46.792602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:46.792602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:46.792602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:46.792602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:46.792602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:46.793501 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:46.793501 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:46.793501 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:46.793501 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:46.793501 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:36:46.793501 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:36:46.793501 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:36:46.793501 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:36:46.793501 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:36:46.793501 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:36:46.793501 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:36:46.793501 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:36:46.793501 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 605/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:36:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:46.798663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc931a-c44e-41de-ab9e-817d7d049449 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:46.894689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc931a-c44e-41de-ab9e-817d7d049449 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:46.922198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:46.927400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228d4ad-8d63-4fd7-a0e7-f62f3ce42948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:46.927991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228d4ad-8d63-4fd7-a0e7-f62f3ce42948 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:46.928269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228d4ad-8d63-4fd7-a0e7-f62f3ce42948 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:46.928541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228d4ad-8d63-4fd7-a0e7-f62f3ce42948 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:46.929198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228d4ad-8d63-4fd7-a0e7-f62f3ce42948 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:46.929498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228d4ad-8d63-4fd7-a0e7-f62f3ce42948 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:46.930886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228d4ad-8d63-4fd7-a0e7-f62f3ce42948 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:46.939150 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 645/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:36:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:46.944885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88e9d3-f135-4c1c-8c5b-c5f90bda21c8 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:47.019910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:47.044856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88e9d3-f135-4c1c-8c5b-c5f90bda21c8 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:47.078484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b8b88-69a8-4cfa-9d4f-b12444ecde94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:47.079128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b8b88-69a8-4cfa-9d4f-b12444ecde94 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:47.079407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b8b88-69a8-4cfa-9d4f-b12444ecde94 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:47.079677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b8b88-69a8-4cfa-9d4f-b12444ecde94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:47.089831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b8b88-69a8-4cfa-9d4f-b12444ecde94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:47.091371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b8b88-69a8-4cfa-9d4f-b12444ecde94 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:47.104984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-907b8b88-69a8-4cfa-9d4f-b12444ecde94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:47.104984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:47.104984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:47.104984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:47.104984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:47.104984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:47.104984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:47.104984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:47.104984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:47.104984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:47.104984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:47.104984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:47.104984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:36:47.104984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:36:47.104984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:36:47.104984 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:36:47.106132 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:47.106132 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:47.106132 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:47.106132 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:47.106132 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:47.106132 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:47.106132 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:47.106132 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:47.106132 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:47.106132 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:47.106132 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:47.106132 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:47.106132 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:47.106132 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:47.106132 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:47.106132 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:47.106132 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:36:47.107306 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:36:47.107306 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:47.107306 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:36:47.107306 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 606/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:36:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:47.113427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228d4ad-8d63-4fd7-a0e7-f62f3ce42948 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:47.211864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228d4ad-8d63-4fd7-a0e7-f62f3ce42948 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:47.243642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3113128-5871-4f6e-8a61-f50656abf8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:47.244860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3113128-5871-4f6e-8a61-f50656abf8ab None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:47.245064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3113128-5871-4f6e-8a61-f50656abf8ab None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:47.245334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3113128-5871-4f6e-8a61-f50656abf8ab None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:47.245964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3113128-5871-4f6e-8a61-f50656abf8ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:47.246267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3113128-5871-4f6e-8a61-f50656abf8ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:47.247692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3113128-5871-4f6e-8a61-f50656abf8ab None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:47.258843 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 646/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:47.264342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b8b88-69a8-4cfa-9d4f-b12444ecde94 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:47.359674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b8b88-69a8-4cfa-9d4f-b12444ecde94 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:47.384385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:47.392636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c339a5-a90d-4d5a-b6f7-2e37573f7c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:47.393240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c339a5-a90d-4d5a-b6f7-2e37573f7c83 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:47.393515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c339a5-a90d-4d5a-b6f7-2e37573f7c83 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:47.393784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c339a5-a90d-4d5a-b6f7-2e37573f7c83 None admin] PATH_INFO: `/v3/users/placement` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:47.401029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c339a5-a90d-4d5a-b6f7-2e37573f7c83 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:47.402448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c339a5-a90d-4d5a-b6f7-2e37573f7c83 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:47.413134 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-b4c339a5-a90d-4d5a-b6f7-2e37573f7c83 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 01 20:36:47.413134 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:47.413134 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:47.413134 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:47.413134 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:47.413134 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:47.413134 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:47.413134 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:47.413134 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:47.413134 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:47.413134 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:47.413134 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:47.413134 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:36:47.413134 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:36:47.413134 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:36:47.413134 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:36:47.414225 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:47.414225 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:47.414225 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:36:47.414225 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:36:47.414225 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:36:47.414225 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:36:47.414225 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:47.414225 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:47.414225 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:47.414225 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:47.414225 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:47.414225 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:47.414225 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:47.414225 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:47.414225 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:47.414225 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:47.414225 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:47.414225 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:47.415388 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:47.415388 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:47.415388 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:36:47.415388 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:36:47.415388 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:36:47.415388 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:36:47.415388 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:36:47.415388 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:36:47.415388 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 01 20:36:47.415388 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:36:47.415388 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 607/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:36:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:47.420413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3113128-5871-4f6e-8a61-f50656abf8ab None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:47.474251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:47.518230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3113128-5871-4f6e-8a61-f50656abf8ab None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:47.551304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d983366-26b9-466d-b2c0-d6dfa0f84232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:47.551897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d983366-26b9-466d-b2c0-d6dfa0f84232 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:47.552179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d983366-26b9-466d-b2c0-d6dfa0f84232 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:47.552454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d983366-26b9-466d-b2c0-d6dfa0f84232 None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:47.553171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d983366-26b9-466d-b2c0-d6dfa0f84232 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:47.553479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d983366-26b9-466d-b2c0-d6dfa0f84232 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:47.554943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d983366-26b9-466d-b2c0-d6dfa0f84232 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:47.576600 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 647/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Sat Oct 1 20:36:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:47.582433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c339a5-a90d-4d5a-b6f7-2e37573f7c83 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:47.685971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c339a5-a90d-4d5a-b6f7-2e37573f7c83 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:47.728135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b570a-27bc-4371-8fb3-9a4fa6fc6576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:47.729305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b570a-27bc-4371-8fb3-9a4fa6fc6576 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:47.729592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b570a-27bc-4371-8fb3-9a4fa6fc6576 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:47.729864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b570a-27bc-4371-8fb3-9a4fa6fc6576 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:47.789503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:47.802538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b570a-27bc-4371-8fb3-9a4fa6fc6576 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:47.803981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b570a-27bc-4371-8fb3-9a4fa6fc6576 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:47.897467 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 608/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:36:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 01 20:36:47.903776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d983366-26b9-466d-b2c0-d6dfa0f84232 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:47.999009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d983366-26b9-466d-b2c0-d6dfa0f84232 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:48.011878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:48.032890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72220ab9-6946-490a-ac7f-7d9c06811dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:48.033509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72220ab9-6946-490a-ac7f-7d9c06811dab None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:48.033789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72220ab9-6946-490a-ac7f-7d9c06811dab None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:48.034057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72220ab9-6946-490a-ac7f-7d9c06811dab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:48.043913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72220ab9-6946-490a-ac7f-7d9c06811dab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:48.045329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72220ab9-6946-490a-ac7f-7d9c06811dab None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:48.059552 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-72220ab9-6946-490a-ac7f-7d9c06811dab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:48.059552 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:48.059552 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:48.059552 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:48.059552 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:48.059552 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:48.059552 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:48.059552 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:48.059552 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:48.059552 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:48.059552 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:48.059552 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:48.059552 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:36:48.059552 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:36:48.059552 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:36:48.059552 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:36:48.060704 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:48.060704 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:48.060704 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:48.060704 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:48.060704 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:48.060704 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:48.060704 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:48.060704 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:48.060704 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:48.060704 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:48.060704 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:48.060704 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:48.060704 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:48.060704 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:48.060704 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:48.060704 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:48.060704 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:36:48.061635 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:36:48.061635 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:48.061635 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:36:48.061635 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 648/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:36:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:48.066710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b570a-27bc-4371-8fb3-9a4fa6fc6576 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:48.153819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:48.166195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b570a-27bc-4371-8fb3-9a4fa6fc6576 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:48.199413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b129d-e504-4c39-8108-d1c6d92a4e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:48.200005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b129d-e504-4c39-8108-d1c6d92a4e7a None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:48.200283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b129d-e504-4c39-8108-d1c6d92a4e7a None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:48.200555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b129d-e504-4c39-8108-d1c6d92a4e7a None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:48.201194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b129d-e504-4c39-8108-d1c6d92a4e7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:48.201497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b129d-e504-4c39-8108-d1c6d92a4e7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:48.202884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b129d-e504-4c39-8108-d1c6d92a4e7a None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:48.214204 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 609/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:48.220020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72220ab9-6946-490a-ac7f-7d9c06811dab None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:48.316187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72220ab9-6946-490a-ac7f-7d9c06811dab None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:48.348932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd03271-03f6-4c0e-b055-a58abf3ceeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:48.349535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd03271-03f6-4c0e-b055-a58abf3ceeb2 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:48.349815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd03271-03f6-4c0e-b055-a58abf3ceeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:48.350088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd03271-03f6-4c0e-b055-a58abf3ceeb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:48.356459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd03271-03f6-4c0e-b055-a58abf3ceeb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:48.357892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd03271-03f6-4c0e-b055-a58abf3ceeb2 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:48.368665 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-bbd03271-03f6-4c0e-b055-a58abf3ceeb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:36:48.368665 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:48.368665 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:48.368665 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:48.368665 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:48.368665 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:48.368665 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:48.368665 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:48.368665 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:48.368665 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:48.368665 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:48.368665 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:48.368665 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:36:48.368665 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:36:48.368665 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:36:48.370211 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:36:48.370211 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:48.370211 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:48.370211 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:48.370211 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:48.370211 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:48.370211 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:48.370211 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:48.370211 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:48.370211 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:48.370211 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:48.370211 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:48.370211 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:48.370211 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:48.370211 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:48.370211 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:48.370211 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:48.371379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:36:48.371379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:36:48.371379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:36:48.371379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:36:48.371379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:36:48.371379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:36:48.371379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:36:48.371379 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:36:48.371379 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 649/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:36:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:48.376053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b129d-e504-4c39-8108-d1c6d92a4e7a None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:48.473656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b129d-e504-4c39-8108-d1c6d92a4e7a None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:48.505640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb3e8d-f38a-4bd0-9275-d21b13b3d89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:48.506235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb3e8d-f38a-4bd0-9275-d21b13b3d89e None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:48.506534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb3e8d-f38a-4bd0-9275-d21b13b3d89e None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:48.506813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb3e8d-f38a-4bd0-9275-d21b13b3d89e None admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:48.507494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb3e8d-f38a-4bd0-9275-d21b13b3d89e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:48.507798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb3e8d-f38a-4bd0-9275-d21b13b3d89e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:48.509234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb3e8d-f38a-4bd0-9275-d21b13b3d89e None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:48.521239 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 610/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:36:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:48.527345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd03271-03f6-4c0e-b055-a58abf3ceeb2 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:48.552642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:48.628201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd03271-03f6-4c0e-b055-a58abf3ceeb2 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:48.660293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed59a8-9d90-4d1a-ab2c-dbfd929e41d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:48.660881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed59a8-9d90-4d1a-ab2c-dbfd929e41d1 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:48.661158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed59a8-9d90-4d1a-ab2c-dbfd929e41d1 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:48.661427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed59a8-9d90-4d1a-ab2c-dbfd929e41d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:48.662139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ed59a8-9d90-4d1a-ab2c-dbfd929e41d1 None admin] RBAC: Adding query filter params (role.id=1c50896511054a5f994bee3d99fa0619, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=1e5f6284cd7d4973adffefa98291dbdb) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:48.662444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ed59a8-9d90-4d1a-ab2c-dbfd929e41d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:48.663874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ed59a8-9d90-4d1a-ab2c-dbfd929e41d1 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:48.669074 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 650/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:36:48 2022] GET /identity/v3/role_assignments?user.id=1e5f6284cd7d4973adffefa98291dbdb&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=1c50896511054a5f994bee3d99fa0619 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:36:51.024886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194cc4f-2d4f-4662-8c42-68d0981ed633 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:51.025487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194cc4f-2d4f-4662-8c42-68d0981ed633 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:51.025487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194cc4f-2d4f-4662-8c42-68d0981ed633 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:51.027419 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 611/1261] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:36:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:36:51.032688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ca7f9-c6e0-4025-beae-be0253272fe1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:51.033090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ca7f9-c6e0-4025-beae-be0253272fe1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:51.033256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ca7f9-c6e0-4025-beae-be0253272fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:51.051007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ac9ca7f9-c6e0-4025-beae-be0253272fe1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:36:51.052178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ac9ca7f9-c6e0-4025-beae-be0253272fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:36:51.115818 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 651/1262] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:36:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 01 20:36:51.438449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194cc4f-2d4f-4662-8c42-68d0981ed633 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:51.449523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:51.537338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194cc4f-2d4f-4662-8c42-68d0981ed633 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:51.569831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d836798-6c49-4eca-be46-1a2abd6fceea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:51.570440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d836798-6c49-4eca-be46-1a2abd6fceea None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:51.571012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d836798-6c49-4eca-be46-1a2abd6fceea None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:51.571282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d836798-6c49-4eca-be46-1a2abd6fceea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:51.581048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d836798-6c49-4eca-be46-1a2abd6fceea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:36:51.585770 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-1d836798-6c49-4eca-be46-1a2abd6fceea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:36:51.585770 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:51.585770 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:51.585770 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:51.585770 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:51.585770 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:51.585770 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:51.585770 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:51.585770 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:51.585770 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:51.585770 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:51.585770 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:51.585770 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:36:51.585770 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:36:51.585770 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:36:51.585770 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:36:51.587137 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:36:51.587137 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:36:51.587137 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:36:51.587137 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:36:51.587137 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:51.587137 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:51.587137 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:51.587137 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:51.587137 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:51.587137 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:51.587137 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:51.587137 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:51.587137 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:51.587137 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:51.587137 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:51.587137 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:51.588855 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:51.588855 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:51.588855 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:51.588855 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:51.588855 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:36:51.588855 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:36:51.588855 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:36:51.588855 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:36:51.588855 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:36:51.588855 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:36:51.588855 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:36:51.588855 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:36:51.588855 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 612/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:36:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:51.593329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ca7f9-c6e0-4025-beae-be0253272fe1 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:51.612107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:51.690233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ca7f9-c6e0-4025-beae-be0253272fe1 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:51.726042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181a6a7-a2fc-4089-9f3a-31d49306ae42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:51.726635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181a6a7-a2fc-4089-9f3a-31d49306ae42 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:51.726913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181a6a7-a2fc-4089-9f3a-31d49306ae42 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:51.727193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181a6a7-a2fc-4089-9f3a-31d49306ae42 None admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:51.727855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181a6a7-a2fc-4089-9f3a-31d49306ae42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:51.728165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181a6a7-a2fc-4089-9f3a-31d49306ae42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:51.730086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181a6a7-a2fc-4089-9f3a-31d49306ae42 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:51.738217 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 652/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:36:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:51.744178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d836798-6c49-4eca-be46-1a2abd6fceea None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:51.809941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:51.841930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d836798-6c49-4eca-be46-1a2abd6fceea None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:51.875136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88454c3-b792-4173-b364-2309972a68cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:51.875747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88454c3-b792-4173-b364-2309972a68cb None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:51.876028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88454c3-b792-4173-b364-2309972a68cb None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:51.876300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88454c3-b792-4173-b364-2309972a68cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:51.885904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88454c3-b792-4173-b364-2309972a68cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:51.887369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88454c3-b792-4173-b364-2309972a68cb None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:51.901531 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-f88454c3-b792-4173-b364-2309972a68cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:51.901531 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:51.901531 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:51.901531 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:51.901531 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:51.901531 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:51.901531 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:51.901531 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:51.901531 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:51.901531 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:51.901531 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:51.901531 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:51.901531 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:36:51.901531 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:36:51.901531 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:36:51.901531 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:36:51.902692 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:51.902692 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:51.902692 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:51.902692 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:51.902692 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:51.902692 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:51.902692 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:51.902692 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:51.902692 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:51.902692 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:51.902692 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:51.902692 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:51.902692 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:51.902692 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:51.902692 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:51.902692 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:51.902692 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:36:51.903952 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:36:51.903952 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:51.903952 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:36:51.903952 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 613/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:36:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:51.908317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181a6a7-a2fc-4089-9f3a-31d49306ae42 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:51.987169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:52.006909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181a6a7-a2fc-4089-9f3a-31d49306ae42 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:52.041267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e329ab9-bf6a-481b-ba11-474ad729c94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:52.041854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e329ab9-bf6a-481b-ba11-474ad729c94c None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:52.042138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e329ab9-bf6a-481b-ba11-474ad729c94c None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:52.042414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e329ab9-bf6a-481b-ba11-474ad729c94c None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:52.043086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e329ab9-bf6a-481b-ba11-474ad729c94c None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:52.043397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e329ab9-bf6a-481b-ba11-474ad729c94c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:52.044752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e329ab9-bf6a-481b-ba11-474ad729c94c None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:52.056394 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 653/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:52.062619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88454c3-b792-4173-b364-2309972a68cb None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:52.160253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88454c3-b792-4173-b364-2309972a68cb None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:52.179255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:52.192955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b02ed-4796-4654-937f-dc621e80a243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:52.193560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b02ed-4796-4654-937f-dc621e80a243 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:52.193835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b02ed-4796-4654-937f-dc621e80a243 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:52.194104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b02ed-4796-4654-937f-dc621e80a243 None admin] PATH_INFO: `/v3/users/placement` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:52.201421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b02ed-4796-4654-937f-dc621e80a243 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:52.202867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b02ed-4796-4654-937f-dc621e80a243 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:52.214188 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-c47b02ed-4796-4654-937f-dc621e80a243 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 01 20:36:52.214188 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:52.214188 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:52.214188 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:52.214188 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:52.214188 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:52.214188 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:52.214188 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:52.214188 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:52.214188 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:52.214188 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:52.214188 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:52.214188 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:36:52.214188 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:36:52.214188 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:36:52.214188 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:36:52.215371 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:52.215371 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:52.215371 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:36:52.215371 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:36:52.215371 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:36:52.215371 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:36:52.215371 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:52.215371 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:52.215371 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:52.215371 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:52.215371 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:52.215371 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:52.215371 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:52.215371 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:52.215371 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:52.215371 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:52.215371 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:52.215371 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:52.216575 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:52.216575 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:52.216575 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:36:52.216575 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:36:52.216575 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:36:52.216575 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:36:52.216575 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:36:52.216575 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:36:52.216575 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 01 20:36:52.216575 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:36:52.216575 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 614/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:36:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:52.221131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e329ab9-bf6a-481b-ba11-474ad729c94c None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:52.317021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e329ab9-bf6a-481b-ba11-474ad729c94c None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:52.349780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590cede-c553-4a51-a298-0aa5a525a30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:52.350368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590cede-c553-4a51-a298-0aa5a525a30b None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:52.350683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590cede-c553-4a51-a298-0aa5a525a30b None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:52.350956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590cede-c553-4a51-a298-0aa5a525a30b None admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:52.351675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590cede-c553-4a51-a298-0aa5a525a30b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:52.351982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590cede-c553-4a51-a298-0aa5a525a30b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:52.353947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590cede-c553-4a51-a298-0aa5a525a30b None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:52.362242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:52.370760 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 654/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Sat Oct 1 20:36:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:36:52.376398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b02ed-4796-4654-937f-dc621e80a243 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:52.474595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b02ed-4796-4654-937f-dc621e80a243 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:52.507327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6841ff-09ef-4a53-b2dd-a5d31d2ea60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:52.507936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6841ff-09ef-4a53-b2dd-a5d31d2ea60d None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:52.508217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6841ff-09ef-4a53-b2dd-a5d31d2ea60d None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:52.508490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6841ff-09ef-4a53-b2dd-a5d31d2ea60d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:52.519120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6841ff-09ef-4a53-b2dd-a5d31d2ea60d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:52.520551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6841ff-09ef-4a53-b2dd-a5d31d2ea60d None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:52.534846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-0d6841ff-09ef-4a53-b2dd-a5d31d2ea60d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:52.534846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:52.534846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:52.534846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:52.534846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:52.534846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:52.534846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:52.534846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:52.534846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:52.534846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:52.534846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:52.534846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:52.534846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:36:52.534846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:36:52.534846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:36:52.534846 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:36:52.540664 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:52.540664 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:52.540664 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:52.540664 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:52.540664 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:52.540664 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:52.540664 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:52.540664 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:52.540664 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:52.540664 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:52.540664 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:52.540664 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:52.540664 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:52.540664 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:52.540664 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:52.540664 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:52.540664 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:36:52.541853 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:36:52.541853 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:52.541853 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:36:52.541853 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 615/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:36:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:52.542152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590cede-c553-4a51-a298-0aa5a525a30b None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:52.644599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590cede-c553-4a51-a298-0aa5a525a30b None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:52.677381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e87afe-6b84-43e6-a23a-e391f801434a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:52.677985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e87afe-6b84-43e6-a23a-e391f801434a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:52.678268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e87afe-6b84-43e6-a23a-e391f801434a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:52.678577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e87afe-6b84-43e6-a23a-e391f801434a None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:52.679219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e87afe-6b84-43e6-a23a-e391f801434a None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:52.679526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e87afe-6b84-43e6-a23a-e391f801434a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:52.680899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e87afe-6b84-43e6-a23a-e391f801434a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:52.691732 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 655/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:52.698253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6841ff-09ef-4a53-b2dd-a5d31d2ea60d None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:52.736221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:52.803679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6841ff-09ef-4a53-b2dd-a5d31d2ea60d None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:52.836139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca24bb-2bdc-4daf-9b06-b1812557bd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:52.836745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca24bb-2bdc-4daf-9b06-b1812557bd2b None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:52.837020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca24bb-2bdc-4daf-9b06-b1812557bd2b None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:52.837287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca24bb-2bdc-4daf-9b06-b1812557bd2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:52.844147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca24bb-2bdc-4daf-9b06-b1812557bd2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:52.845576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca24bb-2bdc-4daf-9b06-b1812557bd2b None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:52.857152 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-78ca24bb-2bdc-4daf-9b06-b1812557bd2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:36:52.857152 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:52.857152 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:52.857152 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:52.857152 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:52.857152 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:52.857152 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:52.857152 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:52.857152 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:52.857152 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:52.857152 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:52.857152 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:52.857152 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:36:52.857152 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:36:52.857152 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:36:52.858167 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:36:52.858167 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:52.858167 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:52.858167 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:52.858167 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:52.858167 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:52.858167 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:52.858167 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:52.858167 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:52.858167 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:52.858167 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:52.858167 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:52.858167 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:52.858167 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:52.858167 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:52.858167 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:52.858167 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:52.859303 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:36:52.859303 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:36:52.859303 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:36:52.859303 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:36:52.859303 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:36:52.859303 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:36:52.859303 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:36:52.859303 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:36:52.859303 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 616/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:36:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:52.864107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e87afe-6b84-43e6-a23a-e391f801434a None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:52.921201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:52.961711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e87afe-6b84-43e6-a23a-e391f801434a None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:52.994577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eea0d2-61c0-4972-9e48-01bb4ba902d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:52.995173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eea0d2-61c0-4972-9e48-01bb4ba902d0 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:52.995454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eea0d2-61c0-4972-9e48-01bb4ba902d0 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:52.995729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eea0d2-61c0-4972-9e48-01bb4ba902d0 None admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:52.996415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eea0d2-61c0-4972-9e48-01bb4ba902d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:52.996724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eea0d2-61c0-4972-9e48-01bb4ba902d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:52.998145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eea0d2-61c0-4972-9e48-01bb4ba902d0 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:53.009948 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 656/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:36:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:53.015799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca24bb-2bdc-4daf-9b06-b1812557bd2b None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:53.105636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:53.114750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca24bb-2bdc-4daf-9b06-b1812557bd2b None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:53.147873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f760b-b946-4a2a-bc62-769398015bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:53.148522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f760b-b946-4a2a-bc62-769398015bd5 None admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:53.148799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f760b-b946-4a2a-bc62-769398015bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:53.149070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f760b-b946-4a2a-bc62-769398015bd5 None admin] PATH_INFO: `/v3/projects/b7875946723844198353bbfcf22516fb/users/1e5f6284cd7d4973adffefa98291dbdb/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:53.150957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f760b-b946-4a2a-bc62-769398015bd5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7875946723844198353bbfcf22516fb, user_id=1e5f6284cd7d4973adffefa98291dbdb, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:53.152374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f760b-b946-4a2a-bc62-769398015bd5 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:53.164134 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 617/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Oct 1 20:36:53 2022] PUT /identity/v3/projects/b7875946723844198353bbfcf22516fb/users/1e5f6284cd7d4973adffefa98291dbdb/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:36:55.522779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07475606-0654-44d6-94dc-d8934b0dfcef None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:55.523305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07475606-0654-44d6-94dc-d8934b0dfcef None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:55.523305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07475606-0654-44d6-94dc-d8934b0dfcef None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:55.524664 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 657/1274] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:36:55.530443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52990a72-7489-497d-beef-8d672a0cdfa7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:55.530878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52990a72-7489-497d-beef-8d672a0cdfa7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:55.531029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52990a72-7489-497d-beef-8d672a0cdfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:55.548881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-52990a72-7489-497d-beef-8d672a0cdfa7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:36:55.550051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-52990a72-7489-497d-beef-8d672a0cdfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:36:55.614645 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 618/1275] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:36:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:36:55.963057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07475606-0654-44d6-94dc-d8934b0dfcef None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:56.063007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07475606-0654-44d6-94dc-d8934b0dfcef None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:56.079860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:56.096758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04164d7-309e-4604-be8b-33336f50c161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:56.097380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04164d7-309e-4604-be8b-33336f50c161 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:56.097663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04164d7-309e-4604-be8b-33336f50c161 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:56.097938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04164d7-309e-4604-be8b-33336f50c161 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:56.108395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04164d7-309e-4604-be8b-33336f50c161 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:36:56.113194 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-d04164d7-309e-4604-be8b-33336f50c161 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:36:56.113194 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:56.113194 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:56.113194 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:56.113194 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:56.113194 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:56.113194 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:56.113194 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:56.113194 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:56.113194 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:56.113194 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:56.113194 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:56.113194 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 01 20:36:56.113194 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 01 20:36:56.113194 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 01 20:36:56.113194 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 01 20:36:56.114266 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:36:56.114266 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:36:56.114266 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:36:56.114266 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:36:56.114266 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:56.114266 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:56.114266 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:56.114266 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:56.114266 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:56.114266 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:56.114266 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:56.114266 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:56.114266 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:56.114266 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:56.114266 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:56.114266 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:56.115335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:56.115335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:56.115335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:56.115335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:56.115335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 20:36:56.115335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 20:36:56.115335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 20:36:56.115335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 20:36:56.115335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 20:36:56.115335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 20:36:56.115335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 01 20:36:56.115335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:36:56.115335 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 658/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Oct 1 20:36:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:56.122258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52990a72-7489-497d-beef-8d672a0cdfa7 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:56.201148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:56.219911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52990a72-7489-497d-beef-8d672a0cdfa7 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:56.252507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f58f1-bfa9-47e7-b40d-ba451470e11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:56.253098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f58f1-bfa9-47e7-b40d-ba451470e11c None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:56.253408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f58f1-bfa9-47e7-b40d-ba451470e11c None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:56.253686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f58f1-bfa9-47e7-b40d-ba451470e11c None admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:56.254345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f58f1-bfa9-47e7-b40d-ba451470e11c None admin] RBAC: Adding query filter params (name=admin) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:56.254683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f58f1-bfa9-47e7-b40d-ba451470e11c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:56.256042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f58f1-bfa9-47e7-b40d-ba451470e11c None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:56.264292 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 619/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Oct 1 20:36:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:56.270027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04164d7-309e-4604-be8b-33336f50c161 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:56.365671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04164d7-309e-4604-be8b-33336f50c161 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:56.399052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec212f-2c89-48b0-89c0-26b61bd42374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:56.399665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec212f-2c89-48b0-89c0-26b61bd42374 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:56.400562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec212f-2c89-48b0-89c0-26b61bd42374 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:56.400848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec212f-2c89-48b0-89c0-26b61bd42374 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:56.410424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec212f-2c89-48b0-89c0-26b61bd42374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:56.411883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec212f-2c89-48b0-89c0-26b61bd42374 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:56.425727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-e5ec212f-2c89-48b0-89c0-26b61bd42374 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:56.425727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:56.425727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:56.425727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:56.425727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:56.425727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:56.425727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:56.425727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:56.425727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:56.425727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:56.425727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:56.425727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:56.425727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:36:56.425727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:36:56.425727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:36:56.425727 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:36:56.426842 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:56.426842 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:56.426842 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:56.426842 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:56.426842 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:56.426842 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:56.426842 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:56.426842 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:56.426842 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:56.426842 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:56.426842 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:56.426842 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:56.426842 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:56.426842 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:56.426842 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:56.426842 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:56.426842 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:36:56.427758 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:36:56.427758 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:56.427758 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:36:56.427758 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 659/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:36:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:56.432731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f58f1-bfa9-47e7-b40d-ba451470e11c None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:56.529673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f58f1-bfa9-47e7-b40d-ba451470e11c None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:56.562019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41d5ef-647c-4cdd-a969-14f3024ca125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:56.562707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41d5ef-647c-4cdd-a969-14f3024ca125 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:56.562993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41d5ef-647c-4cdd-a969-14f3024ca125 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:56.563263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41d5ef-647c-4cdd-a969-14f3024ca125 None admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:56.563893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be41d5ef-647c-4cdd-a969-14f3024ca125 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:56.564191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be41d5ef-647c-4cdd-a969-14f3024ca125 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:56.565538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be41d5ef-647c-4cdd-a969-14f3024ca125 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:56.577590 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 620/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:56.583258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec212f-2c89-48b0-89c0-26b61bd42374 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:56.614778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:56.683670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec212f-2c89-48b0-89c0-26b61bd42374 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:56.718997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666deec-726a-4c29-8eac-d8651065a842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:56.719617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666deec-726a-4c29-8eac-d8651065a842 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:56.719896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666deec-726a-4c29-8eac-d8651065a842 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:56.720166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666deec-726a-4c29-8eac-d8651065a842 None admin] PATH_INFO: `/v3/users/placement` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:56.727310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b666deec-726a-4c29-8eac-d8651065a842 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:56.728735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b666deec-726a-4c29-8eac-d8651065a842 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:56.739354 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-b666deec-726a-4c29-8eac-d8651065a842 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 01 20:36:56.739354 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:56.739354 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:56.739354 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:56.739354 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:56.739354 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:56.739354 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:56.739354 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:56.739354 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:56.739354 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:56.739354 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:56.739354 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:56.739354 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 20:36:56.739354 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 20:36:56.739354 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 20:36:56.739354 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 20:36:56.740549 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:56.740549 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:56.740549 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:36:56.740549 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:36:56.740549 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 20:36:56.740549 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:36:56.740549 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:56.740549 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:56.740549 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:56.740549 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:56.740549 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:56.740549 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:56.740549 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:56.740549 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:56.740549 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:56.740549 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:56.740549 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:56.740549 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:56.741893 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:56.741893 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:56.741893 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 20:36:56.741893 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 20:36:56.741893 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 20:36:56.741893 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 20:36:56.741893 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 20:36:56.741893 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 20:36:56.741893 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 01 20:36:56.741893 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:36:56.741893 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 660/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:36:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:56.752556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41d5ef-647c-4cdd-a969-14f3024ca125 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:56.752556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:56.845024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41d5ef-647c-4cdd-a969-14f3024ca125 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:56.877691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc3b9c-2c54-4d64-818f-7712c45dc09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:56.878277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc3b9c-2c54-4d64-818f-7712c45dc09f None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:56.878590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc3b9c-2c54-4d64-818f-7712c45dc09f None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:56.878864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc3b9c-2c54-4d64-818f-7712c45dc09f None admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:56.879578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc3b9c-2c54-4d64-818f-7712c45dc09f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:56.879882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc3b9c-2c54-4d64-818f-7712c45dc09f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:56.881302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc3b9c-2c54-4d64-818f-7712c45dc09f None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:56.898239 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 621/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Sat Oct 1 20:36:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:56.904001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666deec-726a-4c29-8eac-d8651065a842 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:56.990238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:57.000577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666deec-726a-4c29-8eac-d8651065a842 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:57.032366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabd0c0-a74e-48a7-bfea-c70b1ffc0826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:57.032963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabd0c0-a74e-48a7-bfea-c70b1ffc0826 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:57.033244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabd0c0-a74e-48a7-bfea-c70b1ffc0826 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:57.033518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabd0c0-a74e-48a7-bfea-c70b1ffc0826 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:57.099460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabd0c0-a74e-48a7-bfea-c70b1ffc0826 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:57.100827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabd0c0-a74e-48a7-bfea-c70b1ffc0826 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:57.187394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:57.198191 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 661/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Oct 1 20:36:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 01 20:36:57.204094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc3b9c-2c54-4d64-818f-7712c45dc09f None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:57.263107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:57.300641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc3b9c-2c54-4d64-818f-7712c45dc09f None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:57.333819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd26fe96-d427-4ae9-b477-53def9aae1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:57.334431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26fe96-d427-4ae9-b477-53def9aae1d6 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:57.334737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26fe96-d427-4ae9-b477-53def9aae1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:57.335011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26fe96-d427-4ae9-b477-53def9aae1d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:57.345149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd26fe96-d427-4ae9-b477-53def9aae1d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:57.346606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd26fe96-d427-4ae9-b477-53def9aae1d6 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:57.361275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-cd26fe96-d427-4ae9-b477-53def9aae1d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:57.361275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:57.361275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:57.361275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:57.361275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:57.361275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:57.361275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:57.361275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:57.361275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:57.361275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:57.361275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:57.361275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:57.361275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 20:36:57.361275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 20:36:57.361275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 20:36:57.361275 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 20:36:57.362398 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:57.362398 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:57.362398 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:57.362398 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:57.362398 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:57.362398 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:57.362398 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:57.362398 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:57.362398 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:57.362398 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:57.362398 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:57.362398 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:57.362398 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:57.362398 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:57.362398 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:57.362398 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:57.362398 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 01 20:36:57.363682 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 20:36:57.363682 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 01 20:36:57.363682 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:36:57.363682 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 622/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:36:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:57.367941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabd0c0-a74e-48a7-bfea-c70b1ffc0826 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:57.463353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabd0c0-a74e-48a7-bfea-c70b1ffc0826 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:57.495615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcb9a3-a07e-4c14-a843-cf3e63aaaaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:57.496200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcb9a3-a07e-4c14-a843-cf3e63aaaaec None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:57.496475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcb9a3-a07e-4c14-a843-cf3e63aaaaec None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:57.496739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcb9a3-a07e-4c14-a843-cf3e63aaaaec None admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:57.497366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcb9a3-a07e-4c14-a843-cf3e63aaaaec None admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:57.497662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcb9a3-a07e-4c14-a843-cf3e63aaaaec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:57.499041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcb9a3-a07e-4c14-a843-cf3e63aaaaec None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:57.509653 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 662/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Oct 1 20:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:36:57.515300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd26fe96-d427-4ae9-b477-53def9aae1d6 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:57.613455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd26fe96-d427-4ae9-b477-53def9aae1d6 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:57.626929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:57.646696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa6bf99-1562-464c-8a61-2db3a0d7a368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:57.647320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6bf99-1562-464c-8a61-2db3a0d7a368 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:57.647599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6bf99-1562-464c-8a61-2db3a0d7a368 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:57.647867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6bf99-1562-464c-8a61-2db3a0d7a368 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:57.654871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa6bf99-1562-464c-8a61-2db3a0d7a368 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:57.656301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa6bf99-1562-464c-8a61-2db3a0d7a368 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:57.667870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-7fa6bf99-1562-464c-8a61-2db3a0d7a368 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:36:57.667870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:36:57.667870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:36:57.667870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:36:57.667870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:36:57.667870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:36:57.667870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:36:57.667870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:36:57.667870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:36:57.667870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:36:57.667870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:36:57.667870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:36:57.667870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 01 20:36:57.667870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 01 20:36:57.667870 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 01 20:36:57.668955 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 01 20:36:57.668955 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:36:57.668955 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:36:57.668955 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:36:57.668955 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:36:57.668955 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:36:57.668955 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:36:57.668955 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:36:57.668955 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 20:36:57.668955 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:36:57.668955 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:36:57.668955 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:36:57.668955 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:36:57.668955 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:36:57.668955 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:36:57.668955 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:36:57.668955 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:36:57.670099 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 01 20:36:57.670099 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 01 20:36:57.670099 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:36:57.670099 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:36:57.670099 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:36:57.670099 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:36:57.670099 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 01 20:36:57.670099 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:36:57.670099 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 623/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Oct 1 20:36:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:36:57.674808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcb9a3-a07e-4c14-a843-cf3e63aaaaec None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:57.727133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:36:57.777151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcb9a3-a07e-4c14-a843-cf3e63aaaaec None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:57.809923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1703f1-d9a3-48e6-a538-81153cc250d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:57.810538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1703f1-d9a3-48e6-a538-81153cc250d4 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:57.810821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1703f1-d9a3-48e6-a538-81153cc250d4 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:57.811093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1703f1-d9a3-48e6-a538-81153cc250d4 None admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:57.811773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1703f1-d9a3-48e6-a538-81153cc250d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:57.812077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1703f1-d9a3-48e6-a538-81153cc250d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:57.813489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1703f1-d9a3-48e6-a538-81153cc250d4 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:57.825167 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 663/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Oct 1 20:36:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:36:57.831181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa6bf99-1562-464c-8a61-2db3a0d7a368 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:36:57.927387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa6bf99-1562-464c-8a61-2db3a0d7a368 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:36:57.959721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e25af-fd20-499d-9d64-620f422e9c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:36:57.960302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e25af-fd20-499d-9d64-620f422e9c32 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:36:57.960581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e25af-fd20-499d-9d64-620f422e9c32 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:36:57.960853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e25af-fd20-499d-9d64-620f422e9c32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:36:57.961562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e25af-fd20-499d-9d64-620f422e9c32 None admin] RBAC: Adding query filter params (role.id=1c50896511054a5f994bee3d99fa0619, scope.project.id=b7875946723844198353bbfcf22516fb, user.id=1e5f6284cd7d4973adffefa98291dbdb) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:36:57.961867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e25af-fd20-499d-9d64-620f422e9c32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:36:57.963331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e25af-fd20-499d-9d64-620f422e9c32 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:36:57.968871 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 624/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Oct 1 20:36:57 2022] GET /identity/v3/role_assignments?user.id=1e5f6284cd7d4973adffefa98291dbdb&scope.project.id=b7875946723844198353bbfcf22516fb&role.id=1c50896511054a5f994bee3d99fa0619 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:37:00.344184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc20de6-a8f7-4baf-baec-d181e730da15 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:37:00.344688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc20de6-a8f7-4baf-baec-d181e730da15 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:37:00.344688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc20de6-a8f7-4baf-baec-d181e730da15 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:37:00.345980 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 664/1288] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:37:00.351129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34286032-0a5a-4a45-aec5-4a8156afcb53 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:37:00.351513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34286032-0a5a-4a45-aec5-4a8156afcb53 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:37:00.351679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34286032-0a5a-4a45-aec5-4a8156afcb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:37:00.369883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-34286032-0a5a-4a45-aec5-4a8156afcb53 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:37:00.371085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-34286032-0a5a-4a45-aec5-4a8156afcb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:37:00.399882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:37:00.436206 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 625/1289] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:37:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:37:00.769402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc20de6-a8f7-4baf-baec-d181e730da15 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:37:00.869466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc20de6-a8f7-4baf-baec-d181e730da15 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:37:00.879696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:37:00.902552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d720d4d-089d-4ff8-8797-14bda9fd65ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:37:00.903162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d720d4d-089d-4ff8-8797-14bda9fd65ae None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:37:00.903438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d720d4d-089d-4ff8-8797-14bda9fd65ae None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:37:00.903706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d720d4d-089d-4ff8-8797-14bda9fd65ae None admin] PATH_INFO: `/v3/services/placement` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:37:00.904276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d720d4d-089d-4ff8-8797-14bda9fd65ae None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:37:00.905632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d720d4d-089d-4ff8-8797-14bda9fd65ae None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:37:00.914013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-0d720d4d-089d-4ff8-8797-14bda9fd65ae None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 01 20:37:00.914013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:37:00.914013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:37:00.914013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:37:00.914013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:37:00.914013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:37:00.914013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:37:00.914013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:37:00.914013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 01 20:37:00.914013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:37:00.914013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:37:00.914013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:37:00.914013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:37:00.914013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:37:00.914013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:37:00.914013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:37:00.914013 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:37:00.915253 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:37:00.915253 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:37:00.915253 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:37:00.915253 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:37:00.915253 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:37:00.915253 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:37:00.915253 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:37:00.915253 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:37:00.915253 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:37:00.915253 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:37:00.915253 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:37:00.915253 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:37:00.915253 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:37:00.915253 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:37:00.915253 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:37:00.915253 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:37:00.916781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:37:00.916781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:37:00.916781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:37:00.916781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:37:00.916781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:37:00.916781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:37:00.916781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:37:00.916781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:37:00.916781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:37:00.916781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:37:00.916781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:37:00.916781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:37:00.916781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 01 20:37:00.916781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:37:00.916781 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 665/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Oct 1 20:37:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:37:00.921793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34286032-0a5a-4a45-aec5-4a8156afcb53 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:37:01.018506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34286032-0a5a-4a45-aec5-4a8156afcb53 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:37:01.051755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcda556-1b45-4f24-b525-0a63a2c5269a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:37:01.052361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcda556-1b45-4f24-b525-0a63a2c5269a None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:37:01.052642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcda556-1b45-4f24-b525-0a63a2c5269a None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:37:01.052915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcda556-1b45-4f24-b525-0a63a2c5269a None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:37:01.053541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcda556-1b45-4f24-b525-0a63a2c5269a None admin] RBAC: Adding query filter params (name=placement) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:37:01.053863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcda556-1b45-4f24-b525-0a63a2c5269a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:37:01.055245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcda556-1b45-4f24-b525-0a63a2c5269a None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:37:01.060568 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 626/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Sat Oct 1 20:37:00 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:37:01.066227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d720d4d-089d-4ff8-8797-14bda9fd65ae None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:37:01.164462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d720d4d-089d-4ff8-8797-14bda9fd65ae None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:37:01.197172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9b079-f8e6-4a4a-86df-1a4b3b10ca58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:37:01.197763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9b079-f8e6-4a4a-86df-1a4b3b10ca58 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:37:01.198044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9b079-f8e6-4a4a-86df-1a4b3b10ca58 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:37:01.198317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9b079-f8e6-4a4a-86df-1a4b3b10ca58 None admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:37:01.198983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9b079-f8e6-4a4a-86df-1a4b3b10ca58 None admin] RBAC: Adding query filter params (type=placement) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:37:01.199282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9b079-f8e6-4a4a-86df-1a4b3b10ca58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:37:01.200643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9b079-f8e6-4a4a-86df-1a4b3b10ca58 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:37:01.204913 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 666/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Sat Oct 1 20:37:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:37:03.546537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2cd84-029d-4d29-b079-1dee5fc52d6a None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:37:03.547147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2cd84-029d-4d29-b079-1dee5fc52d6a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:37:03.547147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2cd84-029d-4d29-b079-1dee5fc52d6a None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:37:03.548762 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 627/1293] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:37:03.553795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb604793-1144-46ef-a3ee-a99e69f319a6 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:37:03.554164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb604793-1144-46ef-a3ee-a99e69f319a6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:37:03.554342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb604793-1144-46ef-a3ee-a99e69f319a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:37:03.572647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-bb604793-1144-46ef-a3ee-a99e69f319a6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:37:03.573673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bb604793-1144-46ef-a3ee-a99e69f319a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:37:03.602403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:37:03.640220 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 667/1294] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:37:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:37:03.963051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2cd84-029d-4d29-b079-1dee5fc52d6a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:37:03.993468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:37:04.062158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2cd84-029d-4d29-b079-1dee5fc52d6a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:37:04.096513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931defa-4446-48a0-97e5-c30b84af0335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:37:04.097277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931defa-4446-48a0-97e5-c30b84af0335 None admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:37:04.097758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931defa-4446-48a0-97e5-c30b84af0335 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:37:04.098241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931defa-4446-48a0-97e5-c30b84af0335 None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:37:04.099435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4931defa-4446-48a0-97e5-c30b84af0335 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:37:04.101779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4931defa-4446-48a0-97e5-c30b84af0335 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:37:04.116630 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 628/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Oct 1 20:37:03 2022] POST /identity/v3/services => generated 246 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:37:06.490583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515b6f5-da7c-4b3e-a331-d79f7b468ccf None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:37:06.491118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515b6f5-da7c-4b3e-a331-d79f7b468ccf None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:37:06.491118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515b6f5-da7c-4b3e-a331-d79f7b468ccf None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:37:06.492466 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 668/1296] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:37:06.498208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb2f67-cdcb-42ce-ad4d-d125861acec7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:37:06.498669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb2f67-cdcb-42ce-ad4d-d125861acec7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:37:06.498763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb2f67-cdcb-42ce-ad4d-d125861acec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:37:06.517116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-04bb2f67-cdcb-42ce-ad4d-d125861acec7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:37:06.518291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-04bb2f67-cdcb-42ce-ad4d-d125861acec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:37:06.588000 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 629/1297] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:37:06 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:37:06.912192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515b6f5-da7c-4b3e-a331-d79f7b468ccf None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:37:07.009946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515b6f5-da7c-4b3e-a331-d79f7b468ccf None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:37:07.042756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ca8a6-d2a9-4a62-9603-aec48c8c2c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:37:07.043696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ca8a6-d2a9-4a62-9603-aec48c8c2c1a None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:37:07.043696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ca8a6-d2a9-4a62-9603-aec48c8c2c1a None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:37:07.043893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ca8a6-d2a9-4a62-9603-aec48c8c2c1a None admin] PATH_INFO: `/v3/services/placement` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:37:07.044448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ca8a6-d2a9-4a62-9603-aec48c8c2c1a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:37:07.045786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ca8a6-d2a9-4a62-9603-aec48c8c2c1a None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:37:07.054309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-267ca8a6-d2a9-4a62-9603-aec48c8c2c1a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 01 20:37:07.054309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:37:07.054309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:37:07.054309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:37:07.054309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:37:07.054309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:37:07.054309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:37:07.054309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:37:07.054309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 01 20:37:07.054309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:37:07.054309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:37:07.054309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:37:07.054309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:37:07.054309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:37:07.054309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:37:07.054309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:37:07.054309 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:37:07.055580 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:37:07.055580 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:37:07.055580 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:37:07.055580 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:37:07.055580 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:37:07.055580 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:37:07.055580 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:37:07.055580 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:37:07.055580 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:37:07.055580 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:37:07.055580 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:37:07.055580 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:37:07.055580 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:37:07.055580 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:37:07.055580 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:37:07.055580 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:37:07.056743 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:37:07.056743 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:37:07.056743 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:37:07.056743 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:37:07.056743 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:37:07.056743 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:37:07.056743 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:37:07.056743 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:37:07.056743 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:37:07.056743 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:37:07.056743 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:37:07.056743 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:37:07.056743 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 01 20:37:07.056743 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:37:07.056743 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 669/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Oct 1 20:37:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:37:07.061328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb2f67-cdcb-42ce-ad4d-d125861acec7 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:37:07.079838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:37:07.159712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb2f67-cdcb-42ce-ad4d-d125861acec7 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:37:07.192379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6eff5-1e16-4024-8e0b-d4f5a0da48d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:37:07.192917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6eff5-1e16-4024-8e0b-d4f5a0da48d9 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:37:07.193195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6eff5-1e16-4024-8e0b-d4f5a0da48d9 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:37:07.193465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6eff5-1e16-4024-8e0b-d4f5a0da48d9 None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:37:07.194082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6eff5-1e16-4024-8e0b-d4f5a0da48d9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:37:07.194380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6eff5-1e16-4024-8e0b-d4f5a0da48d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:37:07.196029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6eff5-1e16-4024-8e0b-d4f5a0da48d9 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:37:07.201527 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 630/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Sat Oct 1 20:37:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:37:07.207525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ca8a6-d2a9-4a62-9603-aec48c8c2c1a None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:37:07.237687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:37:07.303775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ca8a6-d2a9-4a62-9603-aec48c8c2c1a None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:37:07.336342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b65203-5476-4799-8b24-d6a2a8265fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:37:07.336921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b65203-5476-4799-8b24-d6a2a8265fa0 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:37:07.337224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b65203-5476-4799-8b24-d6a2a8265fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:37:07.337494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b65203-5476-4799-8b24-d6a2a8265fa0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:37:07.338187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b65203-5476-4799-8b24-d6a2a8265fa0 None admin] RBAC: Adding query filter params (interface=public, service_id=f09731c4642645598c2bcf4a2b64cc4d, region_id=RegionOne) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:37:07.338515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b65203-5476-4799-8b24-d6a2a8265fa0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:37:07.339846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b65203-5476-4799-8b24-d6a2a8265fa0 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:37:07.344602 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 670/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Oct 1 20:37:07 2022] GET /identity/v3/endpoints?service_id=f09731c4642645598c2bcf4a2b64cc4d&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:37:07.350261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6eff5-1e16-4024-8e0b-d4f5a0da48d9 None admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:37:07.428416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:37:07.447612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6eff5-1e16-4024-8e0b-d4f5a0da48d9 None admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:37:07.480016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc76fd-1c63-46bd-84cd-9464ff0100ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:37:07.480605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc76fd-1c63-46bd-84cd-9464ff0100ce None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:37:07.480884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc76fd-1c63-46bd-84cd-9464ff0100ce None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:37:07.481155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc76fd-1c63-46bd-84cd-9464ff0100ce None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:37:07.481755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc76fd-1c63-46bd-84cd-9464ff0100ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:37:07.483143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc76fd-1c63-46bd-84cd-9464ff0100ce None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:37:07.489601 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 631/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Oct 1 20:37:07 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:37:09.859628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70798c6c-76db-4425-a90e-c57ae46214c2 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:37:09.860178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70798c6c-76db-4425-a90e-c57ae46214c2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:37:09.860178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70798c6c-76db-4425-a90e-c57ae46214c2 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:37:09.861504 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 671/1302] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:37:09.866701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08612b33-199d-4087-a36c-4f9e12c17072 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:37:09.867089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08612b33-199d-4087-a36c-4f9e12c17072 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:37:09.867246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08612b33-199d-4087-a36c-4f9e12c17072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:37:09.885482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-08612b33-199d-4087-a36c-4f9e12c17072 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:37:09.886678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-08612b33-199d-4087-a36c-4f9e12c17072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:37:09.950551 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 632/1303] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:37:09 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 01 20:37:10.273084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70798c6c-76db-4425-a90e-c57ae46214c2 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:37:10.364808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:37:10.372113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70798c6c-76db-4425-a90e-c57ae46214c2 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:37:10.404643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5a8a9-df63-4922-8bce-fe705dcb7f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:37:10.405250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5a8a9-df63-4922-8bce-fe705dcb7f85 None admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:37:10.405528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5a8a9-df63-4922-8bce-fe705dcb7f85 None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:37:10.405796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5a8a9-df63-4922-8bce-fe705dcb7f85 None admin] PATH_INFO: `/v3/services/placement` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:37:10.406369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b5a8a9-df63-4922-8bce-fe705dcb7f85 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:37:10.407744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b5a8a9-df63-4922-8bce-fe705dcb7f85 None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:37:10.416184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-a3b5a8a9-df63-4922-8bce-fe705dcb7f85 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 01 20:37:10.416184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:37:10.416184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 20:37:10.416184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 20:37:10.416184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 20:37:10.416184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 20:37:10.416184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 20:37:10.416184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:37:10.416184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 01 20:37:10.416184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:37:10.416184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:37:10.416184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:37:10.416184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:37:10.416184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:37:10.416184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:37:10.416184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:37:10.416184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:37:10.417317 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:37:10.417317 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:37:10.417317 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:37:10.417317 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:37:10.417317 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:37:10.417317 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:37:10.417317 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 20:37:10.417317 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 20:37:10.417317 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 20:37:10.417317 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 20:37:10.417317 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:37:10.417317 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:37:10.417317 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:37:10.417317 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:37:10.417317 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:37:10.417317 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:37:10.418463 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:37:10.418463 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:37:10.418463 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:37:10.418463 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:37:10.418463 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:37:10.418463 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:37:10.418463 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:37:10.418463 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:37:10.418463 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:37:10.418463 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:37:10.418463 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 20:37:10.418463 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 20:37:10.418463 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 01 20:37:10.418463 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:37:10.418463 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 672/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Oct 1 20:37:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:37:10.424371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08612b33-199d-4087-a36c-4f9e12c17072 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:37:10.497667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:37:10.519683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08612b33-199d-4087-a36c-4f9e12c17072 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:37:10.551953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad3489-9678-4f6c-adc2-4a8c006dc7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:37:10.552491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad3489-9678-4f6c-adc2-4a8c006dc7d1 None admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:37:10.552766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad3489-9678-4f6c-adc2-4a8c006dc7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:37:10.553038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad3489-9678-4f6c-adc2-4a8c006dc7d1 None admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:37:10.553650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad3489-9678-4f6c-adc2-4a8c006dc7d1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:37:10.553949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad3489-9678-4f6c-adc2-4a8c006dc7d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:37:10.555332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad3489-9678-4f6c-adc2-4a8c006dc7d1 None admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:37:10.560705 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 633/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Sat Oct 1 20:37:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:37:10.566253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5a8a9-df63-4922-8bce-fe705dcb7f85 None admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:37:10.662232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5a8a9-df63-4922-8bce-fe705dcb7f85 None admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:37:10.697032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc36b1b-e47f-4703-a2ad-8dd8d79c90fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:37:10.697625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc36b1b-e47f-4703-a2ad-8dd8d79c90fd None admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:37:10.697988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc36b1b-e47f-4703-a2ad-8dd8d79c90fd None admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:37:10.698262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc36b1b-e47f-4703-a2ad-8dd8d79c90fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:37:10.699047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc36b1b-e47f-4703-a2ad-8dd8d79c90fd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:37:10.700388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc36b1b-e47f-4703-a2ad-8dd8d79c90fd None admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:37:10.719216 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 673/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:37:10 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:37:20.557890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=30 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:37:20.715212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:23.758124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1ecba-c618-4c80-b1cc-bd0d2f6632a7 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:23.758736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1ecba-c618-4c80-b1cc-bd0d2f6632a7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:23.758736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1ecba-c618-4c80-b1cc-bd0d2f6632a7 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:23.760223 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 634/1307] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:38:23.765486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60269cb-c6ec-47ed-88b4-f60493256dcd None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:23.765929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60269cb-c6ec-47ed-88b4-f60493256dcd None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:23.766015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60269cb-c6ec-47ed-88b4-f60493256dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:23.927553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f60269cb-c6ec-47ed-88b4-f60493256dcd None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:38:23.935056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f60269cb-c6ec-47ed-88b4-f60493256dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:24.141980 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 674/1308] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:38:24.514377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b2579-aebb-428a-8281-875ed4424778 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:24.514848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b2579-aebb-428a-8281-875ed4424778 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:24.515211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b2579-aebb-428a-8281-875ed4424778 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:24.517194 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 635/1309] 10.222.0.17 () {54 vars in 964 bytes} [Sat Oct 1 20:38:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:38:24.531019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2cd18-120e-44ed-a069-c2fb08f31b4f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:24.531389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2cd18-120e-44ed-a069-c2fb08f31b4f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:24.531784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2cd18-120e-44ed-a069-c2fb08f31b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:24.643079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9cd2cd18-120e-44ed-a069-c2fb08f31b4f None None] MFA Rules not processed for user `e17a026f305248878f18eec35e1e02dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:38:24.644668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9cd2cd18-120e-44ed-a069-c2fb08f31b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:24.671183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:24.751933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b2579-aebb-428a-8281-875ed4424778 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:38:24.753829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9c5b2579-aebb-428a-8281-875ed4424778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:24.758607 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 675/1310] 10.222.0.17 () {58 vars in 1061 bytes} [Sat Oct 1 20:38:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:38:24.876334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b2579-aebb-428a-8281-875ed4424778 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:38:24.963581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32dd497-9719-492a-992c-6aaa40ea0da4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:38:24.964191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dd497-9719-492a-992c-6aaa40ea0da4 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:24.964469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dd497-9719-492a-992c-6aaa40ea0da4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:24.964738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dd497-9719-492a-992c-6aaa40ea0da4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:24.966394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b32dd497-9719-492a-992c-6aaa40ea0da4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:25.033450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:25.092476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32dd497-9719-492a-992c-6aaa40ea0da4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:38:25.094098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32dd497-9719-492a-992c-6aaa40ea0da4 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:38:25.113531 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 636/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:38:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:38:27.894728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963af2ef-14b9-4e3f-ab86-43e8497cef84 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:27.895297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963af2ef-14b9-4e3f-ab86-43e8497cef84 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:27.895297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963af2ef-14b9-4e3f-ab86-43e8497cef84 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:27.896614 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 676/1312] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:38:27.902570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc12e45-1582-4d03-8806-2d1db4e7b798 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:27.902970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc12e45-1582-4d03-8806-2d1db4e7b798 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:27.903110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc12e45-1582-4d03-8806-2d1db4e7b798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:27.974086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-cfc12e45-1582-4d03-8806-2d1db4e7b798 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:38:27.975349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cfc12e45-1582-4d03-8806-2d1db4e7b798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:28.075898 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 637/1313] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:38:28.414119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963af2ef-14b9-4e3f-ab86-43e8497cef84 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:38:28.415728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-963af2ef-14b9-4e3f-ab86-43e8497cef84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:28.519610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963af2ef-14b9-4e3f-ab86-43e8497cef84 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:38:28.593064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:28.595221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92a371-9a5b-4e49-a2c3-7ed4a5172ad0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:38:28.595939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92a371-9a5b-4e49-a2c3-7ed4a5172ad0 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:28.596230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92a371-9a5b-4e49-a2c3-7ed4a5172ad0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:28.596759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92a371-9a5b-4e49-a2c3-7ed4a5172ad0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:28.598848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7d92a371-9a5b-4e49-a2c3-7ed4a5172ad0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:28.707862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d92a371-9a5b-4e49-a2c3-7ed4a5172ad0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:38:28.709328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d92a371-9a5b-4e49-a2c3-7ed4a5172ad0 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:38:28.728094 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 677/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:38:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:38:31.470819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532db9a2-b490-44e1-a037-3c69e0caa284 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:31.471391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532db9a2-b490-44e1-a037-3c69e0caa284 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:31.471391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532db9a2-b490-44e1-a037-3c69e0caa284 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:31.472676 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 638/1315] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:38:31.478328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058bd78-fb55-44f6-ae03-3ec90af25ce3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:31.478768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058bd78-fb55-44f6-ae03-3ec90af25ce3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:31.478917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058bd78-fb55-44f6-ae03-3ec90af25ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:31.544429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8058bd78-fb55-44f6-ae03-3ec90af25ce3 None None] MFA Rules not processed for user `2e606e5ac9f84891835962715092f667`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:38:31.545620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8058bd78-fb55-44f6-ae03-3ec90af25ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:31.596474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:31.641464 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 678/1316] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:38:31 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:38:31.989267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532db9a2-b490-44e1-a037-3c69e0caa284 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:38:31.991358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-532db9a2-b490-44e1-a037-3c69e0caa284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:32.001001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:32.095367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532db9a2-b490-44e1-a037-3c69e0caa284 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:38:32.167497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030f427-47c1-480f-9e3f-9fa99efa1c41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:38:32.168090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030f427-47c1-480f-9e3f-9fa99efa1c41 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:32.168363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030f427-47c1-480f-9e3f-9fa99efa1c41 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:32.168628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030f427-47c1-480f-9e3f-9fa99efa1c41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:32.170185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5030f427-47c1-480f-9e3f-9fa99efa1c41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:32.233809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:32.285203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5030f427-47c1-480f-9e3f-9fa99efa1c41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:38:32.286963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5030f427-47c1-480f-9e3f-9fa99efa1c41 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:38:32.304834 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 639/1317] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:38:31 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:38:35.212789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc041e5-c8a8-44eb-936d-9b008ff6f183 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:35.213348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc041e5-c8a8-44eb-936d-9b008ff6f183 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:35.213348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc041e5-c8a8-44eb-936d-9b008ff6f183 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:35.214725 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 679/1318] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:38:35.219904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525038ee-fb92-4c47-bb3b-1f2457b67484 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:35.220339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525038ee-fb92-4c47-bb3b-1f2457b67484 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:35.220436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525038ee-fb92-4c47-bb3b-1f2457b67484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:35.284986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-525038ee-fb92-4c47-bb3b-1f2457b67484 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:38:35.286180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-525038ee-fb92-4c47-bb3b-1f2457b67484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:35.384844 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 640/1319] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:38:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:38:35.742752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ed75e-b436-4f07-b593-a77a1148fdfe None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:35.743032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ed75e-b436-4f07-b593-a77a1148fdfe None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:35.743304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ed75e-b436-4f07-b593-a77a1148fdfe None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:35.744678 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 680/1320] 10.222.0.17 () {54 vars in 964 bytes} [Sat Oct 1 20:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:38:35.750577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dff1b-7a21-4a66-b43a-36f719325c7b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:35.750858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dff1b-7a21-4a66-b43a-36f719325c7b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:35.751127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dff1b-7a21-4a66-b43a-36f719325c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:35.773706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:35.826421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a83dff1b-7a21-4a66-b43a-36f719325c7b None None] MFA Rules not processed for user `e17a026f305248878f18eec35e1e02dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:38:35.827640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a83dff1b-7a21-4a66-b43a-36f719325c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:35.900742 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 641/1321] 10.222.0.17 () {58 vars in 1061 bytes} [Sat Oct 1 20:38:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:38:35.909728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ed75e-b436-4f07-b593-a77a1148fdfe None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:38:35.911268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3c3ed75e-b436-4f07-b593-a77a1148fdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:36.014048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ed75e-b436-4f07-b593-a77a1148fdfe None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:38:36.093301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f681c6-7a0b-44c9-aa58-76fa33b05a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:38:36.093900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f681c6-7a0b-44c9-aa58-76fa33b05a19 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:36.094176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f681c6-7a0b-44c9-aa58-76fa33b05a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:36.094447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f681c6-7a0b-44c9-aa58-76fa33b05a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:36.096061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-69f681c6-7a0b-44c9-aa58-76fa33b05a19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:36.103020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:36.202720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f681c6-7a0b-44c9-aa58-76fa33b05a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:38:36.204152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f681c6-7a0b-44c9-aa58-76fa33b05a19 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:38:36.222475 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 681/1322] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:38:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:38:39.684757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59adee1f-4196-4417-9dff-844543f1f4b3 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:39.685263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59adee1f-4196-4417-9dff-844543f1f4b3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:39.685263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59adee1f-4196-4417-9dff-844543f1f4b3 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:39.686554 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 642/1323] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:38:39.691696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b0e1e-c846-483b-873a-3e48c5305c97 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:39.692087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b0e1e-c846-483b-873a-3e48c5305c97 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:39.692249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b0e1e-c846-483b-873a-3e48c5305c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:39.759292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f32b0e1e-c846-483b-873a-3e48c5305c97 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:38:39.760477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f32b0e1e-c846-483b-873a-3e48c5305c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:39.816320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:39.856380 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 682/1324] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:38:40.190555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59adee1f-4196-4417-9dff-844543f1f4b3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:38:40.192074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-59adee1f-4196-4417-9dff-844543f1f4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:40.296451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59adee1f-4196-4417-9dff-844543f1f4b3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:38:40.308247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:40.373272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9680d70-135f-455e-b5e2-a38fa84e6103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:38:40.373998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9680d70-135f-455e-b5e2-a38fa84e6103 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:40.374284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9680d70-135f-455e-b5e2-a38fa84e6103 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:40.374598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9680d70-135f-455e-b5e2-a38fa84e6103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:40.376252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d9680d70-135f-455e-b5e2-a38fa84e6103 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:40.488908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9680d70-135f-455e-b5e2-a38fa84e6103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:38:40.490358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9680d70-135f-455e-b5e2-a38fa84e6103 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:38:40.508972 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 643/1325] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:38:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:38:42.978726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60527da1-4412-46f7-83b4-ca017f5aa3e4 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:42.979286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60527da1-4412-46f7-83b4-ca017f5aa3e4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:42.979286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60527da1-4412-46f7-83b4-ca017f5aa3e4 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:42.980602 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 683/1326] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:38:42.985817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97ad8d-bf13-4ccd-a595-b96478e67b9f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:42.986198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97ad8d-bf13-4ccd-a595-b96478e67b9f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:42.986373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97ad8d-bf13-4ccd-a595-b96478e67b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:43.036894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:43.057290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7f97ad8d-bf13-4ccd-a595-b96478e67b9f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:38:43.058528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7f97ad8d-bf13-4ccd-a595-b96478e67b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:43.156530 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 644/1327] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:38:43.494926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60527da1-4412-46f7-83b4-ca017f5aa3e4 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:38:43.496537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-60527da1-4412-46f7-83b4-ca017f5aa3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:43.599332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60527da1-4412-46f7-83b4-ca017f5aa3e4 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:38:43.671021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c7071-224e-4e29-8f03-a64572d64300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:38:43.671687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c7071-224e-4e29-8f03-a64572d64300 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:43.671969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c7071-224e-4e29-8f03-a64572d64300 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:43.672230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c7071-224e-4e29-8f03-a64572d64300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:43.673798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-430c7071-224e-4e29-8f03-a64572d64300 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:43.691603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:43.785331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c7071-224e-4e29-8f03-a64572d64300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:38:43.786719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c7071-224e-4e29-8f03-a64572d64300 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:38:43.804851 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 684/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:38:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:38:46.254390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8fd97-1a61-4e0a-8466-be2f01e7d945 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:46.255017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8fd97-1a61-4e0a-8466-be2f01e7d945 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:46.255017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8fd97-1a61-4e0a-8466-be2f01e7d945 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:46.256926 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 645/1329] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:38:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:38:46.262163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe6a84-5db3-457c-84fc-cb1bc51e37a8 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:46.262609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe6a84-5db3-457c-84fc-cb1bc51e37a8 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:46.262870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe6a84-5db3-457c-84fc-cb1bc51e37a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:46.328942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-25fe6a84-5db3-457c-84fc-cb1bc51e37a8 None None] MFA Rules not processed for user `2e606e5ac9f84891835962715092f667`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:38:46.330063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-25fe6a84-5db3-457c-84fc-cb1bc51e37a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:46.397763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:46.429653 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 685/1330] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:38:46 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:38:46.765755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8fd97-1a61-4e0a-8466-be2f01e7d945 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:38:46.767345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7be8fd97-1a61-4e0a-8466-be2f01e7d945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:46.870066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8fd97-1a61-4e0a-8466-be2f01e7d945 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:38:46.891816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:46.944952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf481de-1e6e-488c-ba3e-8b2d43eee7fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:38:46.945554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf481de-1e6e-488c-ba3e-8b2d43eee7fe service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:46.946088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf481de-1e6e-488c-ba3e-8b2d43eee7fe service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:46.946357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf481de-1e6e-488c-ba3e-8b2d43eee7fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:46.947987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1cf481de-1e6e-488c-ba3e-8b2d43eee7fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:47.061323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf481de-1e6e-488c-ba3e-8b2d43eee7fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:38:47.062816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf481de-1e6e-488c-ba3e-8b2d43eee7fe service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:38:47.080757 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 646/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:38:46 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:38:49.744862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5249d2-950d-4c17-8bd7-c717bdefb77e None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:49.745366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5249d2-950d-4c17-8bd7-c717bdefb77e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:49.745366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5249d2-950d-4c17-8bd7-c717bdefb77e None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:49.746792 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 686/1332] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:38:49.752118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99393921-4c53-4d7c-af90-30490aa95af2 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:49.752576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99393921-4c53-4d7c-af90-30490aa95af2 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:49.752672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99393921-4c53-4d7c-af90-30490aa95af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:49.817274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:49.820849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-99393921-4c53-4d7c-af90-30490aa95af2 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:38:49.822699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-99393921-4c53-4d7c-af90-30490aa95af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:49.918925 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 647/1333] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:38:50.255427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5249d2-950d-4c17-8bd7-c717bdefb77e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:38:50.257512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2b5249d2-950d-4c17-8bd7-c717bdefb77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:50.377160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5249d2-950d-4c17-8bd7-c717bdefb77e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:38:50.462521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77e41c-c2ad-496f-937f-040caa6de17b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:38:50.463123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77e41c-c2ad-496f-937f-040caa6de17b service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:50.463407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77e41c-c2ad-496f-937f-040caa6de17b service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:50.463676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77e41c-c2ad-496f-937f-040caa6de17b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:50.465254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fa77e41c-c2ad-496f-937f-040caa6de17b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:50.496149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:50.578872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77e41c-c2ad-496f-937f-040caa6de17b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:38:50.580260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77e41c-c2ad-496f-937f-040caa6de17b service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:38:50.598617 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 687/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:38:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:38:53.047456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f25b5-f323-42f9-9f8d-c79537a32f56 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:53.047942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f25b5-f323-42f9-9f8d-c79537a32f56 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:53.048037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f25b5-f323-42f9-9f8d-c79537a32f56 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:53.049321 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 648/1335] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:38:53.055068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8bd36-7e71-42c1-97b9-db989d6f7423 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:53.055446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8bd36-7e71-42c1-97b9-db989d6f7423 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:53.055625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8bd36-7e71-42c1-97b9-db989d6f7423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:53.120793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-cbc8bd36-7e71-42c1-97b9-db989d6f7423 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:38:53.121975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cbc8bd36-7e71-42c1-97b9-db989d6f7423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:53.196048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:53.218111 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 688/1336] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:38:53.557676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f25b5-f323-42f9-9f8d-c79537a32f56 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:38:53.559284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-262f25b5-f323-42f9-9f8d-c79537a32f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:53.659340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f25b5-f323-42f9-9f8d-c79537a32f56 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:38:53.687254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:53.733267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e6b3a-8eb5-4bb0-9f59-624439e2a8dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:38:53.733857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e6b3a-8eb5-4bb0-9f59-624439e2a8dc service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:53.734131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e6b3a-8eb5-4bb0-9f59-624439e2a8dc service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:53.734397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e6b3a-8eb5-4bb0-9f59-624439e2a8dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:53.735987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-705e6b3a-8eb5-4bb0-9f59-624439e2a8dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:53.840858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e6b3a-8eb5-4bb0-9f59-624439e2a8dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:38:53.842213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e6b3a-8eb5-4bb0-9f59-624439e2a8dc service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:38:53.860759 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 649/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:38:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:38:56.588093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f488e7f-6f8b-4b30-b886-c0b932b8ddf4 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:56.588667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f488e7f-6f8b-4b30-b886-c0b932b8ddf4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:56.588667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f488e7f-6f8b-4b30-b886-c0b932b8ddf4 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:56.589990 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 689/1338] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:38:56.595021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2541b16-09bf-4279-b9f1-c6734456f38f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:56.595452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2541b16-09bf-4279-b9f1-c6734456f38f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:56.595548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2541b16-09bf-4279-b9f1-c6734456f38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:56.660686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a2541b16-09bf-4279-b9f1-c6734456f38f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:38:56.662471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a2541b16-09bf-4279-b9f1-c6734456f38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:56.670892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:56.763649 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 650/1339] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:38:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:38:57.103020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f488e7f-6f8b-4b30-b886-c0b932b8ddf4 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:38:57.104760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5f488e7f-6f8b-4b30-b886-c0b932b8ddf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:57.205828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f488e7f-6f8b-4b30-b886-c0b932b8ddf4 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:38:57.282774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c03968-7ac0-40d5-8cc0-521237e722ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:38:57.283385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c03968-7ac0-40d5-8cc0-521237e722ae service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:38:57.283662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c03968-7ac0-40d5-8cc0-521237e722ae service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:38:57.283930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c03968-7ac0-40d5-8cc0-521237e722ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:38:57.285506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b3c03968-7ac0-40d5-8cc0-521237e722ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:38:57.321523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:38:57.396329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c03968-7ac0-40d5-8cc0-521237e722ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:38:57.397756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c03968-7ac0-40d5-8cc0-521237e722ae service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:38:57.416362 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 690/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:38:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:39:01.742714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58f8df-eb7b-4fc8-8d58-5c41bd098b74 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:01.743285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58f8df-eb7b-4fc8-8d58-5c41bd098b74 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:01.743285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58f8df-eb7b-4fc8-8d58-5c41bd098b74 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:01.744625 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 651/1341] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:39:01.749799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c260bb-eee3-4c8e-894a-0d57e15a4e92 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:01.750215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c260bb-eee3-4c8e-894a-0d57e15a4e92 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:01.750359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c260bb-eee3-4c8e-894a-0d57e15a4e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:01.815987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-05c260bb-eee3-4c8e-894a-0d57e15a4e92 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:01.817186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-05c260bb-eee3-4c8e-894a-0d57e15a4e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:01.904921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:01.916794 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 691/1342] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:39:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:39:02.250945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58f8df-eb7b-4fc8-8d58-5c41bd098b74 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:39:02.252463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-be58f8df-eb7b-4fc8-8d58-5c41bd098b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:02.357624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58f8df-eb7b-4fc8-8d58-5c41bd098b74 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:39:02.395442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:02.437377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b4961-8a13-4575-8cbd-3392f5658aee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:39:02.437990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b4961-8a13-4575-8cbd-3392f5658aee service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:02.438270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b4961-8a13-4575-8cbd-3392f5658aee service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:02.438574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b4961-8a13-4575-8cbd-3392f5658aee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:02.440202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6e8b4961-8a13-4575-8cbd-3392f5658aee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:02.550504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b4961-8a13-4575-8cbd-3392f5658aee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:39:02.551968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b4961-8a13-4575-8cbd-3392f5658aee service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:39:02.571042 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 652/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:39:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:39:05.572956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045de19f-e571-4bd0-bcb0-8422a079218e None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:05.573550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045de19f-e571-4bd0-bcb0-8422a079218e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:05.573550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045de19f-e571-4bd0-bcb0-8422a079218e None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:05.574960 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 692/1344] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:39:05.580396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918ce4b-a548-42f6-8799-9aa954be2397 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:05.580850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918ce4b-a548-42f6-8799-9aa954be2397 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:05.581049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918ce4b-a548-42f6-8799-9aa954be2397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:05.647225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1918ce4b-a548-42f6-8799-9aa954be2397 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:05.648444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1918ce4b-a548-42f6-8799-9aa954be2397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:05.666536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:05.746412 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 653/1345] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:39:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:39:06.085771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045de19f-e571-4bd0-bcb0-8422a079218e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:39:06.087623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-045de19f-e571-4bd0-bcb0-8422a079218e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:06.190509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045de19f-e571-4bd0-bcb0-8422a079218e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:39:06.264503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ecb25-6d33-4cd1-a95c-a66d5af77010 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:39:06.265418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ecb25-6d33-4cd1-a95c-a66d5af77010 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:06.265694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ecb25-6d33-4cd1-a95c-a66d5af77010 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:06.265982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ecb25-6d33-4cd1-a95c-a66d5af77010 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:06.267628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ec8ecb25-6d33-4cd1-a95c-a66d5af77010 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:06.311569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:06.382556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ecb25-6d33-4cd1-a95c-a66d5af77010 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:39:06.383981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ecb25-6d33-4cd1-a95c-a66d5af77010 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:39:06.402917 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 693/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:39:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:39:11.068784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7553c-c845-4e80-b7ae-850d51b0c3e5 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:11.069326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7553c-c845-4e80-b7ae-850d51b0c3e5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:11.069409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7553c-c845-4e80-b7ae-850d51b0c3e5 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:11.071074 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 654/1347] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:39:11.076869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed849fa6-a73f-431b-bc86-f38969cae986 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:11.077381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed849fa6-a73f-431b-bc86-f38969cae986 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:11.077503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed849fa6-a73f-431b-bc86-f38969cae986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:11.148561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ed849fa6-a73f-431b-bc86-f38969cae986 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:11.149779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ed849fa6-a73f-431b-bc86-f38969cae986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:11.245875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:11.248931 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 694/1348] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:39:11.595785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7553c-c845-4e80-b7ae-850d51b0c3e5 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:39:11.597330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cdf7553c-c845-4e80-b7ae-850d51b0c3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:11.712401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7553c-c845-4e80-b7ae-850d51b0c3e5 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:39:11.766089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:11.792459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500e0ef-9862-465b-b073-d127354977c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:39:11.793178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500e0ef-9862-465b-b073-d127354977c5 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:11.793464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500e0ef-9862-465b-b073-d127354977c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:11.793739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500e0ef-9862-465b-b073-d127354977c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:11.795456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5500e0ef-9862-465b-b073-d127354977c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:11.906268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500e0ef-9862-465b-b073-d127354977c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:39:11.907751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500e0ef-9862-465b-b073-d127354977c5 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:39:11.925559 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 655/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:39:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:39:17.403970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b555abc-5437-4d51-a329-0665601718c7 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:17.404491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b555abc-5437-4d51-a329-0665601718c7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:17.404732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b555abc-5437-4d51-a329-0665601718c7 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:17.406226 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 695/1350] 10.222.0.17 () {54 vars in 921 bytes} [Sat Oct 1 20:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:39:17.416408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edf91d-2304-4309-a007-2a2b2b8effcd None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:17.416844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edf91d-2304-4309-a007-2a2b2b8effcd None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:17.417039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edf91d-2304-4309-a007-2a2b2b8effcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:17.573389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-85edf91d-2304-4309-a007-2a2b2b8effcd None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:17.574645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-85edf91d-2304-4309-a007-2a2b2b8effcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:17.603046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:17.708969 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 656/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 20:39:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:39:22.254912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdab792-0c0c-4061-84a5-a403e9be03bc None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:22.255811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdab792-0c0c-4061-84a5-a403e9be03bc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:22.256079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdab792-0c0c-4061-84a5-a403e9be03bc None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:22.257923 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 696/1352] 10.222.0.17 () {54 vars in 921 bytes} [Sat Oct 1 20:39:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:39:22.265282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5f0d8-adf0-4e3f-aa04-95ac7ead0e43 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:22.265871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5f0d8-adf0-4e3f-aa04-95ac7ead0e43 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:22.266053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5f0d8-adf0-4e3f-aa04-95ac7ead0e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:22.361002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e6d5f0d8-adf0-4e3f-aa04-95ac7ead0e43 None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:22.362213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e6d5f0d8-adf0-4e3f-aa04-95ac7ead0e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:22.448208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:22.489290 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 657/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 20:39:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:39:23.839519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed98d90-ded4-4c27-949f-d1b2b93043b1 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:23.840063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed98d90-ded4-4c27-949f-d1b2b93043b1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:23.840063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed98d90-ded4-4c27-949f-d1b2b93043b1 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:23.841412 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 697/1354] 10.222.0.17 () {54 vars in 921 bytes} [Sat Oct 1 20:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:39:23.846306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f77b7-ea7b-402f-acdc-b0583e50c40e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:23.846790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f77b7-ea7b-402f-acdc-b0583e50c40e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:23.846936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f77b7-ea7b-402f-acdc-b0583e50c40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:23.924033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bd3f77b7-ea7b-402f-acdc-b0583e50c40e None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:23.925231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bd3f77b7-ea7b-402f-acdc-b0583e50c40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:24.041752 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 658/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 20:39:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:39:27.904592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e6b1b-8e82-48c9-97a6-a68d052c947c None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:27.905158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e6b1b-8e82-48c9-97a6-a68d052c947c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:27.905158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e6b1b-8e82-48c9-97a6-a68d052c947c None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:27.906375 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 698/1356] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:39:27.911805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb6f6a-4b37-4fe5-8bbf-9ba5c36e86bd None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:27.912230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb6f6a-4b37-4fe5-8bbf-9ba5c36e86bd None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:27.912342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb6f6a-4b37-4fe5-8bbf-9ba5c36e86bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:27.949384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:27.984647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5dfb6f6a-4b37-4fe5-8bbf-9ba5c36e86bd None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:27.986059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5dfb6f6a-4b37-4fe5-8bbf-9ba5c36e86bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:28.106473 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 659/1357] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:39:28.464064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb471e-3947-4929-b091-7540e9116be6 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:28.464905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb471e-3947-4929-b091-7540e9116be6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:28.465983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb471e-3947-4929-b091-7540e9116be6 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:28.470345 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 699/1358] 10.222.0.17 () {54 vars in 961 bytes} [Sat Oct 1 20:39:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:39:28.484034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646d7e9-419a-48e1-a8b8-e44b9c82d4ab None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:28.484815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646d7e9-419a-48e1-a8b8-e44b9c82d4ab None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:28.485465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646d7e9-419a-48e1-a8b8-e44b9c82d4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:28.574090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2646d7e9-419a-48e1-a8b8-e44b9c82d4ab None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:28.575517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2646d7e9-419a-48e1-a8b8-e44b9c82d4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:28.631075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:28.704886 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 660/1359] 10.222.0.17 () {58 vars in 1058 bytes} [Sat Oct 1 20:39:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:39:28.711233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb471e-3947-4929-b091-7540e9116be6 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:39:28.712881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6feb471e-3947-4929-b091-7540e9116be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:28.894262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb471e-3947-4929-b091-7540e9116be6 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:39:28.978176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:29.021124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295db1c-3af9-4dad-befe-9ef2babe818f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:39:29.021730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295db1c-3af9-4dad-befe-9ef2babe818f service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:29.022015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295db1c-3af9-4dad-befe-9ef2babe818f service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:29.022283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295db1c-3af9-4dad-befe-9ef2babe818f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:29.023907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f295db1c-3af9-4dad-befe-9ef2babe818f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:29.250823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295db1c-3af9-4dad-befe-9ef2babe818f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:39:29.252327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295db1c-3af9-4dad-befe-9ef2babe818f service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:39:29.272037 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 700/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:39:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:39:31.967529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec56877-25e5-44fa-a5bb-2461989e9445 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:31.968097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec56877-25e5-44fa-a5bb-2461989e9445 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:31.968097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec56877-25e5-44fa-a5bb-2461989e9445 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:31.973985 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 661/1361] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:39:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:39:31.975410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7149f-6177-48af-bbd7-a2694f620f3a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:31.975838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7149f-6177-48af-bbd7-a2694f620f3a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:31.975990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7149f-6177-48af-bbd7-a2694f620f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:32.050033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c4b7149f-6177-48af-bbd7-a2694f620f3a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:32.051301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c4b7149f-6177-48af-bbd7-a2694f620f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:32.093652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:32.165433 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 701/1362] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:39:32.509651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec56877-25e5-44fa-a5bb-2461989e9445 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:39:32.511291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6ec56877-25e5-44fa-a5bb-2461989e9445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:32.669949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec56877-25e5-44fa-a5bb-2461989e9445 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:39:32.680009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:32.794256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83930f-703e-4119-ab93-531c0896baad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:39:32.794894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83930f-703e-4119-ab93-531c0896baad service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:32.795169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83930f-703e-4119-ab93-531c0896baad service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:32.795434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83930f-703e-4119-ab93-531c0896baad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:32.797001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4b83930f-703e-4119-ab93-531c0896baad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:32.900708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83930f-703e-4119-ab93-531c0896baad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:39:32.902305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83930f-703e-4119-ab93-531c0896baad service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:39:32.923483 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 662/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:39:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:39:35.642977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34571f0-118e-4b4e-b1bd-a96c2fdc3151 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:35.644000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34571f0-118e-4b4e-b1bd-a96c2fdc3151 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:35.644429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34571f0-118e-4b4e-b1bd-a96c2fdc3151 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:35.646457 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 702/1364] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:39:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:39:35.652508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925d853-22de-47e6-9827-bb9b6e18aaae None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:35.652966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925d853-22de-47e6-9827-bb9b6e18aaae None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:35.653061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925d853-22de-47e6-9827-bb9b6e18aaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:35.678966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:35.739343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7925d853-22de-47e6-9827-bb9b6e18aaae None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:35.740631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7925d853-22de-47e6-9827-bb9b6e18aaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:35.902828 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 663/1365] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:39:36.298703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46555f2-ee0b-4b4c-995a-615ee6cb7042 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:36.299003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46555f2-ee0b-4b4c-995a-615ee6cb7042 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:36.299268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46555f2-ee0b-4b4c-995a-615ee6cb7042 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:36.300868 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 703/1366] 10.222.0.17 () {54 vars in 961 bytes} [Sat Oct 1 20:39:36 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:39:36.310136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c0f91-1a9e-4cd3-bd82-843a7edecd77 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:36.311019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c0f91-1a9e-4cd3-bd82-843a7edecd77 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:36.311455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c0f91-1a9e-4cd3-bd82-843a7edecd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:36.418797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-859c0f91-1a9e-4cd3-bd82-843a7edecd77 None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:36.420020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-859c0f91-1a9e-4cd3-bd82-843a7edecd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:36.481524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:36.622781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46555f2-ee0b-4b4c-995a-615ee6cb7042 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:39:36.625876 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 664/1367] 10.222.0.17 () {58 vars in 1058 bytes} [Sat Oct 1 20:39:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:39:36.628099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d46555f2-ee0b-4b4c-995a-615ee6cb7042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:36.854091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46555f2-ee0b-4b4c-995a-615ee6cb7042 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:39:36.935042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:37.039085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d538111-e247-4483-8447-e00c80034abd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:39:37.039867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d538111-e247-4483-8447-e00c80034abd service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:37.040248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d538111-e247-4483-8447-e00c80034abd service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:37.040622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d538111-e247-4483-8447-e00c80034abd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:37.044833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1d538111-e247-4483-8447-e00c80034abd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:37.154818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d538111-e247-4483-8447-e00c80034abd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:39:37.156419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d538111-e247-4483-8447-e00c80034abd service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:39:37.366664 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 704/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:39:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:39:40.007297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f515e1-07f1-4c1b-b784-7ad075e6971e None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:40.007846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f515e1-07f1-4c1b-b784-7ad075e6971e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:40.007846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f515e1-07f1-4c1b-b784-7ad075e6971e None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:40.014377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8213d-0137-4623-b21c-606df233fed3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:40.014850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8213d-0137-4623-b21c-606df233fed3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:40.014946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8213d-0137-4623-b21c-606df233fed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:40.018893 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 665/1369] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:39:40 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:39:40.061908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:40.102744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-48f8213d-0137-4623-b21c-606df233fed3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:40.104082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-48f8213d-0137-4623-b21c-606df233fed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:40.223448 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 705/1370] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:39:40.755625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f687162-ef31-4ada-9034-762469220ef0 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:40.755914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f687162-ef31-4ada-9034-762469220ef0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:40.756368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f687162-ef31-4ada-9034-762469220ef0 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:40.757856 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 666/1371] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:39:40.762756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8213d-0137-4623-b21c-606df233fed3 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:39:40.763365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7d8ff-1d58-4315-b21a-3ff86fb1e8c9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:40.763645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7d8ff-1d58-4315-b21a-3ff86fb1e8c9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:40.763915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7d8ff-1d58-4315-b21a-3ff86fb1e8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:40.764464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-48f8213d-0137-4623-b21c-606df233fed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:40.846957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6da7d8ff-1d58-4315-b21a-3ff86fb1e8c9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:40.848861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6da7d8ff-1d58-4315-b21a-3ff86fb1e8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:40.909134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:40.921643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8213d-0137-4623-b21c-606df233fed3 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:39:40.968552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:40.978544 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 667/1372] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:39:41.068542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2c08f-9cd9-4ede-bfc7-c7f6b097d64f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:39:41.069147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2c08f-9cd9-4ede-bfc7-c7f6b097d64f service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:41.069416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2c08f-9cd9-4ede-bfc7-c7f6b097d64f service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:41.069891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2c08f-9cd9-4ede-bfc7-c7f6b097d64f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:41.071790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8fc2c08f-9cd9-4ede-bfc7-c7f6b097d64f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:41.168992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2c08f-9cd9-4ede-bfc7-c7f6b097d64f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:39:41.170653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2c08f-9cd9-4ede-bfc7-c7f6b097d64f service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:39:41.187471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:41.195094 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 706/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:39:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:39:41.412672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d882c-108a-47b6-b4a8-0b0224f34e28 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:41.413664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d882c-108a-47b6-b4a8-0b0224f34e28 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:41.414129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d882c-108a-47b6-b4a8-0b0224f34e28 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:41.415829 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 668/1374] 10.222.0.17 () {54 vars in 963 bytes} [Sat Oct 1 20:39:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:39:41.421842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46296aec-19bf-464f-a9a1-fa3a53e0b66b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:41.422397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46296aec-19bf-464f-a9a1-fa3a53e0b66b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:41.422901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46296aec-19bf-464f-a9a1-fa3a53e0b66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:41.510114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-46296aec-19bf-464f-a9a1-fa3a53e0b66b None None] MFA Rules not processed for user `197a0ae6b71f4635a069ac0238e017c9`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:41.511500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-46296aec-19bf-464f-a9a1-fa3a53e0b66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:41.592277 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 707/1375] 10.222.0.17 () {58 vars in 1060 bytes} [Sat Oct 1 20:39:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:39:41.599905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d882c-108a-47b6-b4a8-0b0224f34e28 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:39:41.601522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ed5d882c-108a-47b6-b4a8-0b0224f34e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:41.726082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d882c-108a-47b6-b4a8-0b0224f34e28 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:39:41.813958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62338b09-bf4d-47eb-9363-323eaa1bb295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:39:41.814607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62338b09-bf4d-47eb-9363-323eaa1bb295 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:41.814887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62338b09-bf4d-47eb-9363-323eaa1bb295 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:41.815159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62338b09-bf4d-47eb-9363-323eaa1bb295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:41.816853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-62338b09-bf4d-47eb-9363-323eaa1bb295 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:41.868440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:41.934958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62338b09-bf4d-47eb-9363-323eaa1bb295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:39:41.936571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62338b09-bf4d-47eb-9363-323eaa1bb295 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:39:41.955986 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 669/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:39:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:39:43.959529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399b782-6708-42ca-959b-62f6e0903920 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:43.960108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399b782-6708-42ca-959b-62f6e0903920 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:43.960108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399b782-6708-42ca-959b-62f6e0903920 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:43.961458 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 708/1377] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:39:43.966920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1a6d7-083d-4d2e-a5ad-fc2b0c189e4a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:43.967455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1a6d7-083d-4d2e-a5ad-fc2b0c189e4a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:43.967455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1a6d7-083d-4d2e-a5ad-fc2b0c189e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:44.035447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0dd1a6d7-083d-4d2e-a5ad-fc2b0c189e4a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:44.036664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0dd1a6d7-083d-4d2e-a5ad-fc2b0c189e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:44.141689 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 670/1378] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:39:44.490162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399b782-6708-42ca-959b-62f6e0903920 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:39:44.491698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4399b782-6708-42ca-959b-62f6e0903920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:44.581755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:44.643509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399b782-6708-42ca-959b-62f6e0903920 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:39:44.763447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb733d30-4adb-44ef-9c1d-a7dc61e9035f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:39:44.764114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb733d30-4adb-44ef-9c1d-a7dc61e9035f service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:44.764602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb733d30-4adb-44ef-9c1d-a7dc61e9035f service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:44.764730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb733d30-4adb-44ef-9c1d-a7dc61e9035f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:44.766312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cb733d30-4adb-44ef-9c1d-a7dc61e9035f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:44.825123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:44.864268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb733d30-4adb-44ef-9c1d-a7dc61e9035f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:39:44.866542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb733d30-4adb-44ef-9c1d-a7dc61e9035f service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:39:44.895280 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 709/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:39:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:39:47.377497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d26732-d912-4023-b490-050079c606a1 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:47.378203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d26732-d912-4023-b490-050079c606a1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:47.378203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d26732-d912-4023-b490-050079c606a1 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:47.382619 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 671/1380] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:39:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:39:47.385379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2e1ed-ec4d-42d1-8d2c-f8f787e34d7b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:47.385845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2e1ed-ec4d-42d1-8d2c-f8f787e34d7b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:47.385954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2e1ed-ec4d-42d1-8d2c-f8f787e34d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:47.454204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c6d2e1ed-ec4d-42d1-8d2c-f8f787e34d7b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:47.455441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c6d2e1ed-ec4d-42d1-8d2c-f8f787e34d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:47.555302 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 710/1381] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:39:47.911174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d26732-d912-4023-b490-050079c606a1 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:39:47.912973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d2d26732-d912-4023-b490-050079c606a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:47.916049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:48.079910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d26732-d912-4023-b490-050079c606a1 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:39:48.171522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:48.225573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea7eb2-be40-460c-bb95-b22ebe758efb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:39:48.226263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea7eb2-be40-460c-bb95-b22ebe758efb service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:48.226568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea7eb2-be40-460c-bb95-b22ebe758efb service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:48.226949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea7eb2-be40-460c-bb95-b22ebe758efb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:48.228543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-41ea7eb2-be40-460c-bb95-b22ebe758efb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:48.327287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea7eb2-be40-460c-bb95-b22ebe758efb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:39:48.328806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea7eb2-be40-460c-bb95-b22ebe758efb service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:39:48.348192 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 672/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:39:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:39:50.959117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33592568-a774-4722-8fd0-98d907213860 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:50.959730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33592568-a774-4722-8fd0-98d907213860 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:50.959841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33592568-a774-4722-8fd0-98d907213860 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:50.961678 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 711/1383] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:39:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:39:50.967481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7656727-ff93-4e59-8e97-15626e95860f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:50.967985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7656727-ff93-4e59-8e97-15626e95860f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:50.968097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7656727-ff93-4e59-8e97-15626e95860f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:51.046078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c7656727-ff93-4e59-8e97-15626e95860f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:51.047336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c7656727-ff93-4e59-8e97-15626e95860f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:51.090201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:51.155824 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 673/1384] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:39:51.502746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33592568-a774-4722-8fd0-98d907213860 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:39:51.504348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-33592568-a774-4722-8fd0-98d907213860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:51.538688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:51.667640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33592568-a774-4722-8fd0-98d907213860 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:39:51.798276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:51.812744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7822156-cf10-4530-a6e1-52c36542bfe6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:39:51.813354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7822156-cf10-4530-a6e1-52c36542bfe6 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:51.813635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7822156-cf10-4530-a6e1-52c36542bfe6 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:51.813928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7822156-cf10-4530-a6e1-52c36542bfe6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:51.815611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a7822156-cf10-4530-a6e1-52c36542bfe6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:51.910827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7822156-cf10-4530-a6e1-52c36542bfe6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:39:51.912430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7822156-cf10-4530-a6e1-52c36542bfe6 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:39:51.931463 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 712/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:39:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:39:54.477527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d6069-3898-47ea-b6a6-8a804b544723 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:54.478314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d6069-3898-47ea-b6a6-8a804b544723 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:54.478314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d6069-3898-47ea-b6a6-8a804b544723 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:54.479431 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 674/1386] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:39:54.484660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9db049-6fb7-470b-9934-bcb670643bee None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:54.485162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9db049-6fb7-470b-9934-bcb670643bee None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:54.485580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9db049-6fb7-470b-9934-bcb670643bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:54.572333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1f9db049-6fb7-470b-9934-bcb670643bee None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:54.574025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1f9db049-6fb7-470b-9934-bcb670643bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:54.650133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:54.692943 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 713/1387] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:39:55.056135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d6069-3898-47ea-b6a6-8a804b544723 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:39:55.057659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cc9d6069-3898-47ea-b6a6-8a804b544723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:55.239055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d6069-3898-47ea-b6a6-8a804b544723 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:39:55.262201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:55.368136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aab8aa-ba5d-4ec9-b8ad-460eceaefbce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:39:55.368659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aab8aa-ba5d-4ec9-b8ad-460eceaefbce service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:55.368939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aab8aa-ba5d-4ec9-b8ad-460eceaefbce service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:55.369213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aab8aa-ba5d-4ec9-b8ad-460eceaefbce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:55.370812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c5aab8aa-ba5d-4ec9-b8ad-460eceaefbce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:55.464778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aab8aa-ba5d-4ec9-b8ad-460eceaefbce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:39:55.466405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aab8aa-ba5d-4ec9-b8ad-460eceaefbce service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:39:55.484585 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 675/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:39:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:39:58.076319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19530cc-0cd9-4bea-b947-20b7e3bcd982 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:58.077384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19530cc-0cd9-4bea-b947-20b7e3bcd982 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:58.077741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19530cc-0cd9-4bea-b947-20b7e3bcd982 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:58.079252 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 714/1389] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:39:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:39:58.085420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5d414-d664-4f6c-bf2c-b2f6ff268316 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:58.085907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5d414-d664-4f6c-bf2c-b2f6ff268316 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:58.085996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5d414-d664-4f6c-bf2c-b2f6ff268316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:58.120981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:58.152520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-26c5d414-d664-4f6c-bf2c-b2f6ff268316 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:58.153740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-26c5d414-d664-4f6c-bf2c-b2f6ff268316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:58.176995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181b950-6fcf-41cd-8e2c-07977239d3e9 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:58.177274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181b950-6fcf-41cd-8e2c-07977239d3e9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:58.177546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181b950-6fcf-41cd-8e2c-07977239d3e9 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:58.178917 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 715/1390] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:39:58.184594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9178509-f32a-47e5-986d-9ed851b2c6d3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:58.184881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9178509-f32a-47e5-986d-9ed851b2c6d3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:58.185153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9178509-f32a-47e5-986d-9ed851b2c6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:58.251727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a9178509-f32a-47e5-986d-9ed851b2c6d3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:58.252962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a9178509-f32a-47e5-986d-9ed851b2c6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:58.257086 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 676/1391] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:39:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:39:58.353779 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 716/1392] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:39:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:39:58.603229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5d414-d664-4f6c-bf2c-b2f6ff268316 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:39:58.604496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-26c5d414-d664-4f6c-bf2c-b2f6ff268316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:58.712775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b3acb-c73b-41c8-ae28-88b7b72857bf None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:58.713060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b3acb-c73b-41c8-ae28-88b7b72857bf None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:58.713334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b3acb-c73b-41c8-ae28-88b7b72857bf None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:58.714708 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 717/1393] 10.222.0.17 () {54 vars in 962 bytes} [Sat Oct 1 20:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:39:58.715120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:58.720013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b4dce-f6cd-4991-869d-a0feb602fb7f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:58.720370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b4dce-f6cd-4991-869d-a0feb602fb7f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:58.720647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b4dce-f6cd-4991-869d-a0feb602fb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:58.723839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5d414-d664-4f6c-bf2c-b2f6ff268316 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:39:58.782206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:58.805356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ef4b4dce-f6cd-4991-869d-a0feb602fb7f None None] MFA Rules not processed for user `e2927203ce394fabab76dd52a68c2436`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:58.806599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ef4b4dce-f6cd-4991-869d-a0feb602fb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:58.844376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db418ad7-d547-462e-b3e2-16d15d34bec3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:39:58.844981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db418ad7-d547-462e-b3e2-16d15d34bec3 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:58.845257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db418ad7-d547-462e-b3e2-16d15d34bec3 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:58.845531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db418ad7-d547-462e-b3e2-16d15d34bec3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:58.847157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-db418ad7-d547-462e-b3e2-16d15d34bec3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:58.884320 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 718/1394] 10.222.0.17 () {58 vars in 1059 bytes} [Sat Oct 1 20:39:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:39:58.892391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b4dce-f6cd-4991-869d-a0feb602fb7f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:39:58.902032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b4dce-f6cd-4991-869d-a0feb602fb7f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:39:58.941722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db418ad7-d547-462e-b3e2-16d15d34bec3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:39:58.943392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db418ad7-d547-462e-b3e2-16d15d34bec3 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:39:58.960027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:58.963870 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 677/1395] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:39:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:39:58.980816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f939aef-d864-4b14-a8f5-d2a8c0beeba9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:39:58.981389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f939aef-d864-4b14-a8f5-d2a8c0beeba9 service glance] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:58.981681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f939aef-d864-4b14-a8f5-d2a8c0beeba9 service glance] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:58.981926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f939aef-d864-4b14-a8f5-d2a8c0beeba9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:58.983537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5f939aef-d864-4b14-a8f5-d2a8c0beeba9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:59.059156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:59.114048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f939aef-d864-4b14-a8f5-d2a8c0beeba9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:39:59.115560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f939aef-d864-4b14-a8f5-d2a8c0beeba9 service glance] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:39:59.136423 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 719/1396] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:39:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:39:59.744695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1883ac-bfed-4d53-aa87-8c54145ff84b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:59.745162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1883ac-bfed-4d53-aa87-8c54145ff84b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:59.745383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1883ac-bfed-4d53-aa87-8c54145ff84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:59.812721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7b1883ac-bfed-4d53-aa87-8c54145ff84b None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:39:59.813932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7b1883ac-bfed-4d53-aa87-8c54145ff84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:39:59.902758 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 678/1397] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:39:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:39:59.950076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a3103-c590-4514-b55d-f0a2fc707306 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:59.950367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a3103-c590-4514-b55d-f0a2fc707306 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:59.950673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a3103-c590-4514-b55d-f0a2fc707306 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:39:59.952012 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 720/1398] 10.222.0.17 () {54 vars in 943 bytes} [Sat Oct 1 20:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:39:59.958610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7111ea-af20-4900-84ca-27d6efc9b7a4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:39:59.958898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7111ea-af20-4900-84ca-27d6efc9b7a4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:39:59.959170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7111ea-af20-4900-84ca-27d6efc9b7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:00.038136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2f7111ea-af20-4900-84ca-27d6efc9b7a4 None None] MFA Rules not processed for user `c4dc51e2b4474d6ba9fd5d2bc3702826`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:40:00.039385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2f7111ea-af20-4900-84ca-27d6efc9b7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:00.073858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:00.150083 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 679/1399] 10.222.0.17 () {58 vars in 1040 bytes} [Sat Oct 1 20:39:59 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:40:00.159303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a3103-c590-4514-b55d-f0a2fc707306 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:40:00.160816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b22a3103-c590-4514-b55d-f0a2fc707306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:00.300065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a3103-c590-4514-b55d-f0a2fc707306 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:40:00.334557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:00.406234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ccb70-b369-42fa-a639-2e44b58d01d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:40:00.406872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ccb70-b369-42fa-a639-2e44b58d01d5 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:00.407154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ccb70-b369-42fa-a639-2e44b58d01d5 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:00.407427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ccb70-b369-42fa-a639-2e44b58d01d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:00.409046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c60ccb70-b369-42fa-a639-2e44b58d01d5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:00.492172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ccb70-b369-42fa-a639-2e44b58d01d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:40:00.493852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ccb70-b369-42fa-a639-2e44b58d01d5 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:40:00.501833 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 721/1400] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:40:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 20:40:01.476179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9456627-cff1-444b-a7c9-9b605b8d103a None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:01.476778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9456627-cff1-444b-a7c9-9b605b8d103a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:01.476778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9456627-cff1-444b-a7c9-9b605b8d103a None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:01.478092 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 680/1401] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:40:01.483380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cb5a3-9ca9-457a-8387-fbd87cbf3820 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:01.483811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cb5a3-9ca9-457a-8387-fbd87cbf3820 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:01.484063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cb5a3-9ca9-457a-8387-fbd87cbf3820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:01.556107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b84cb5a3-9ca9-457a-8387-fbd87cbf3820 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:40:01.557322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b84cb5a3-9ca9-457a-8387-fbd87cbf3820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:01.572993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:01.664288 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 722/1402] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:40:02.015888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9456627-cff1-444b-a7c9-9b605b8d103a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:40:02.017400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f9456627-cff1-444b-a7c9-9b605b8d103a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:02.190974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9456627-cff1-444b-a7c9-9b605b8d103a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:40:02.195240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:02.332168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df538a3a-8a1c-4ad0-a4cb-b72f1479c91e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:40:02.332980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df538a3a-8a1c-4ad0-a4cb-b72f1479c91e service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:02.333485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df538a3a-8a1c-4ad0-a4cb-b72f1479c91e service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:02.333846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df538a3a-8a1c-4ad0-a4cb-b72f1479c91e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:02.338879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-df538a3a-8a1c-4ad0-a4cb-b72f1479c91e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:02.439760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df538a3a-8a1c-4ad0-a4cb-b72f1479c91e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:40:02.441900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df538a3a-8a1c-4ad0-a4cb-b72f1479c91e service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:40:02.461936 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 681/1403] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:40:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:40:04.939201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38b57d-244b-4b9b-87ed-cf007538acb8 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:04.939904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38b57d-244b-4b9b-87ed-cf007538acb8 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:04.939904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38b57d-244b-4b9b-87ed-cf007538acb8 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:04.941112 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 723/1404] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:40:04.947734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df2df8-e1a1-4a5e-a550-d0a39f1a2883 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:04.947734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df2df8-e1a1-4a5e-a550-d0a39f1a2883 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:04.947734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df2df8-e1a1-4a5e-a550-d0a39f1a2883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:04.960086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:05.027483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-42df2df8-e1a1-4a5e-a550-d0a39f1a2883 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:40:05.037169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-42df2df8-e1a1-4a5e-a550-d0a39f1a2883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:05.149984 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 682/1405] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:40:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:40:05.496511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38b57d-244b-4b9b-87ed-cf007538acb8 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:40:05.498017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0e38b57d-244b-4b9b-87ed-cf007538acb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:05.662190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:05.666454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38b57d-244b-4b9b-87ed-cf007538acb8 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:40:05.813338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925839d5-661d-40ab-94f1-463cebe50788 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:40:05.815572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925839d5-661d-40ab-94f1-463cebe50788 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:05.815572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925839d5-661d-40ab-94f1-463cebe50788 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:05.815572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925839d5-661d-40ab-94f1-463cebe50788 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:05.816859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-925839d5-661d-40ab-94f1-463cebe50788 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:05.924741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925839d5-661d-40ab-94f1-463cebe50788 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:40:05.926523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925839d5-661d-40ab-94f1-463cebe50788 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:40:05.946911 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 724/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:40:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:40:08.436883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def813bf-a879-46c2-8d7f-5692b61ea0c4 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:08.437439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def813bf-a879-46c2-8d7f-5692b61ea0c4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:08.437439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def813bf-a879-46c2-8d7f-5692b61ea0c4 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:08.438692 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 683/1407] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:40:08.443873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dda75b-1ade-488f-8d1c-59e2881c310a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:08.444320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dda75b-1ade-488f-8d1c-59e2881c310a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:08.444413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dda75b-1ade-488f-8d1c-59e2881c310a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:08.449179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:08.518197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-96dda75b-1ade-488f-8d1c-59e2881c310a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:40:08.519466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-96dda75b-1ade-488f-8d1c-59e2881c310a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:08.626448 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 725/1408] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:40:08.979549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def813bf-a879-46c2-8d7f-5692b61ea0c4 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:40:08.981412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-def813bf-a879-46c2-8d7f-5692b61ea0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:09.088449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:09.156583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def813bf-a879-46c2-8d7f-5692b61ea0c4 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:40:09.311256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230bfb63-d16e-4254-b28a-a6bb7c2595ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:40:09.311790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bfb63-d16e-4254-b28a-a6bb7c2595ca service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:09.312065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bfb63-d16e-4254-b28a-a6bb7c2595ca service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:09.312334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bfb63-d16e-4254-b28a-a6bb7c2595ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:09.313960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-230bfb63-d16e-4254-b28a-a6bb7c2595ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:09.384053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:09.455933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230bfb63-d16e-4254-b28a-a6bb7c2595ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:40:09.455933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230bfb63-d16e-4254-b28a-a6bb7c2595ca service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:40:09.480617 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 684/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:40:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:40:18.624973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=64 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:19.478216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=23 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:24.441221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0be3de-135c-4189-b41c-c9dd9ead0369 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:24.441760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0be3de-135c-4189-b41c-c9dd9ead0369 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:24.441760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0be3de-135c-4189-b41c-c9dd9ead0369 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:24.443061 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 726/1410] 10.222.0.17 () {54 vars in 918 bytes} [Sat Oct 1 20:40:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:40:24.448595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c069e-3e68-478a-812d-31f3bffb4736 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:24.449000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c069e-3e68-478a-812d-31f3bffb4736 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:24.449151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c069e-3e68-478a-812d-31f3bffb4736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:24.523253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bb3c069e-3e68-478a-812d-31f3bffb4736 None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:40:24.524435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bb3c069e-3e68-478a-812d-31f3bffb4736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:24.631844 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 685/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Sat Oct 1 20:40:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:40:24.647355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f4f82-2878-4dbb-a2b3-2759365fea4f None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:24.647638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f4f82-2878-4dbb-a2b3-2759365fea4f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:24.647961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f4f82-2878-4dbb-a2b3-2759365fea4f None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:24.649321 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 727/1412] 10.222.0.17 () {54 vars in 918 bytes} [Sat Oct 1 20:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:40:24.654188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cc92b-f476-48ea-b165-89363f4bb225 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:24.654511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cc92b-f476-48ea-b165-89363f4bb225 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:24.654809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cc92b-f476-48ea-b165-89363f4bb225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:24.727044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:24.730860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-175cc92b-f476-48ea-b165-89363f4bb225 None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:40:24.732970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-175cc92b-f476-48ea-b165-89363f4bb225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:24.851982 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 686/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Sat Oct 1 20:40:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:40:27.689042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe277015-d08b-4b69-af34-6b238a010b75 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:27.689611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe277015-d08b-4b69-af34-6b238a010b75 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:27.689611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe277015-d08b-4b69-af34-6b238a010b75 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:27.690964 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 728/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 1 20:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:40:27.696779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648562b0-60e8-40ba-8928-e0446668bce0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:27.697225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648562b0-60e8-40ba-8928-e0446668bce0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:27.697333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648562b0-60e8-40ba-8928-e0446668bce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:27.763509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-648562b0-60e8-40ba-8928-e0446668bce0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:40:27.764705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-648562b0-60e8-40ba-8928-e0446668bce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:27.822204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:27.867755 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 687/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 1 20:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:40:28.225197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe277015-d08b-4b69-af34-6b238a010b75 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:40:28.226754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fe277015-d08b-4b69-af34-6b238a010b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:28.370468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe277015-d08b-4b69-af34-6b238a010b75 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:40:28.443248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:28.483061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d80f55-6019-4915-a4ce-318550678e31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:40:28.483662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d80f55-6019-4915-a4ce-318550678e31 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:28.483939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d80f55-6019-4915-a4ce-318550678e31 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:28.484211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d80f55-6019-4915-a4ce-318550678e31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:28.485794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a6d80f55-6019-4915-a4ce-318550678e31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:28.573071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d80f55-6019-4915-a4ce-318550678e31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:40:28.574679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d80f55-6019-4915-a4ce-318550678e31 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:40:28.592593 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 729/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:40:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:40:30.304692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b2a85-034c-4349-9a2a-8c838b635869 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:30.305217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b2a85-034c-4349-9a2a-8c838b635869 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:30.305217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b2a85-034c-4349-9a2a-8c838b635869 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:30.306463 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 688/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 1 20:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:40:30.311311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa434372-652c-442e-a8a7-8fc691e96bf6 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:30.311738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa434372-652c-442e-a8a7-8fc691e96bf6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:30.311849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa434372-652c-442e-a8a7-8fc691e96bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:30.388500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-aa434372-652c-442e-a8a7-8fc691e96bf6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:40:30.389713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-aa434372-652c-442e-a8a7-8fc691e96bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:30.416222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:30.486857 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 730/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 1 20:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:40:30.719018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b2a85-034c-4349-9a2a-8c838b635869 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:40:30.720544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b64b2a85-034c-4349-9a2a-8c838b635869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:30.864979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b2a85-034c-4349-9a2a-8c838b635869 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:40:30.905310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:30.977582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec96cc-7069-40ec-aa45-c49ac8fe680e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:40:30.978112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec96cc-7069-40ec-aa45-c49ac8fe680e service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:30.978388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec96cc-7069-40ec-aa45-c49ac8fe680e service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:30.978696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec96cc-7069-40ec-aa45-c49ac8fe680e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:30.980283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cfec96cc-7069-40ec-aa45-c49ac8fe680e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:31.068541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec96cc-7069-40ec-aa45-c49ac8fe680e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:40:31.070154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec96cc-7069-40ec-aa45-c49ac8fe680e service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:40:31.088766 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 689/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:40:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:40:32.807978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7cd00-d880-498d-9571-60143af9db8b None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:32.808468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7cd00-d880-498d-9571-60143af9db8b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:32.808588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7cd00-d880-498d-9571-60143af9db8b None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:32.809895 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 731/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 1 20:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:40:32.814693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c3e01-d65a-43b9-860e-d9addee20550 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:32.815677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c3e01-d65a-43b9-860e-d9addee20550 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:32.815848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c3e01-d65a-43b9-860e-d9addee20550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:32.870799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:32.893990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-010c3e01-d65a-43b9-860e-d9addee20550 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:40:32.895234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-010c3e01-d65a-43b9-860e-d9addee20550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:32.990523 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 690/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 1 20:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:40:33.149928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7cd00-d880-498d-9571-60143af9db8b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:40:33.151471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-77a7cd00-d880-498d-9571-60143af9db8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:33.294562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7cd00-d880-498d-9571-60143af9db8b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:40:33.307851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:33.411611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c41d3-2f8c-4e30-a969-6dcf59199c84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:40:33.412241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c41d3-2f8c-4e30-a969-6dcf59199c84 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:33.412521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c41d3-2f8c-4e30-a969-6dcf59199c84 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:33.412793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c41d3-2f8c-4e30-a969-6dcf59199c84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:33.414373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4e1c41d3-2f8c-4e30-a969-6dcf59199c84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:33.502104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c41d3-2f8c-4e30-a969-6dcf59199c84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:40:33.504301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c41d3-2f8c-4e30-a969-6dcf59199c84 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:40:33.521690 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 732/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:40:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:40:35.182809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89970b94-f3eb-447e-8af3-d32b3002bf0a None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:35.183393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89970b94-f3eb-447e-8af3-d32b3002bf0a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:35.183393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89970b94-f3eb-447e-8af3-d32b3002bf0a None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:35.184698 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 691/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 1 20:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:40:35.189604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a18fd4-a17f-47f8-9650-64dabe4a5321 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:35.190644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a18fd4-a17f-47f8-9650-64dabe4a5321 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:35.190825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a18fd4-a17f-47f8-9650-64dabe4a5321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:35.210410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:35.269756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e2a18fd4-a17f-47f8-9650-64dabe4a5321 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:40:35.271003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e2a18fd4-a17f-47f8-9650-64dabe4a5321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:35.368347 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 733/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 1 20:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:40:35.530622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89970b94-f3eb-447e-8af3-d32b3002bf0a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:40:35.532134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-89970b94-f3eb-447e-8af3-d32b3002bf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:35.643406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:35.678822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89970b94-f3eb-447e-8af3-d32b3002bf0a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:40:35.789614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd44bd9-ede8-405d-a880-a0ca811a0e49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:40:35.790219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd44bd9-ede8-405d-a880-a0ca811a0e49 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:35.790515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd44bd9-ede8-405d-a880-a0ca811a0e49 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:35.790802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd44bd9-ede8-405d-a880-a0ca811a0e49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:35.792377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bdd44bd9-ede8-405d-a880-a0ca811a0e49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:35.868020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:35.881788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd44bd9-ede8-405d-a880-a0ca811a0e49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:40:35.883438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd44bd9-ede8-405d-a880-a0ca811a0e49 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:40:35.901204 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 692/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:40:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:40:37.573366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0807e7b-58d1-463b-813a-7c742f40666d None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:37.573938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0807e7b-58d1-463b-813a-7c742f40666d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:37.573938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0807e7b-58d1-463b-813a-7c742f40666d None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:37.575288 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 734/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 1 20:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:40:37.580241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e9efd-e8b9-4112-953f-6b24d149fb36 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:37.580693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e9efd-e8b9-4112-953f-6b24d149fb36 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:37.580806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e9efd-e8b9-4112-953f-6b24d149fb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:37.656045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2d1e9efd-e8b9-4112-953f-6b24d149fb36 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:40:37.657272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2d1e9efd-e8b9-4112-953f-6b24d149fb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:37.752997 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 693/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 1 20:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:40:37.914164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0807e7b-58d1-463b-813a-7c742f40666d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:40:37.915642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c0807e7b-58d1-463b-813a-7c742f40666d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:37.999411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:38.069220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0807e7b-58d1-463b-813a-7c742f40666d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:40:38.183380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb50c8-0853-4597-98d2-ce131e80267c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:40:38.183910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb50c8-0853-4597-98d2-ce131e80267c service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:38.184188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb50c8-0853-4597-98d2-ce131e80267c service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:38.184457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb50c8-0853-4597-98d2-ce131e80267c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:38.186051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a4cb50c8-0853-4597-98d2-ce131e80267c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:38.234106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:38.276456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb50c8-0853-4597-98d2-ce131e80267c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:40:38.278040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb50c8-0853-4597-98d2-ce131e80267c service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:40:38.295576 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 735/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:40:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:40:39.903695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd2ee0-3f36-433f-8a9f-acc8b2411235 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:39.904207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd2ee0-3f36-433f-8a9f-acc8b2411235 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:39.904207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd2ee0-3f36-433f-8a9f-acc8b2411235 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:39.905517 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 694/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 1 20:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:40:39.910904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663635f-64ce-47b1-9b04-f40bb8c7cc6a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:39.911348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663635f-64ce-47b1-9b04-f40bb8c7cc6a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:39.911448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663635f-64ce-47b1-9b04-f40bb8c7cc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:39.985905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a663635f-64ce-47b1-9b04-f40bb8c7cc6a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:40:39.987757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a663635f-64ce-47b1-9b04-f40bb8c7cc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:40.083824 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 736/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 1 20:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:40:40.247894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd2ee0-3f36-433f-8a9f-acc8b2411235 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:40:40.249458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-56cd2ee0-3f36-433f-8a9f-acc8b2411235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:40.298630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:40.395554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd2ee0-3f36-433f-8a9f-acc8b2411235 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:40:40.514141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c124e-79fb-443e-b12b-d510f52ca02c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:40:40.514691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c124e-79fb-443e-b12b-d510f52ca02c service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:40.514967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c124e-79fb-443e-b12b-d510f52ca02c service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:40.515232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c124e-79fb-443e-b12b-d510f52ca02c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:40.516836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-578c124e-79fb-443e-b12b-d510f52ca02c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:40.523837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:40.607713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c124e-79fb-443e-b12b-d510f52ca02c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:40:40.609336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c124e-79fb-443e-b12b-d510f52ca02c service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:40:40.627555 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 695/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:40:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:40:42.232094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d038a8b-57a6-46b7-b2b3-e2dcabcdc8bf None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:42.232624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d038a8b-57a6-46b7-b2b3-e2dcabcdc8bf None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:42.232624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d038a8b-57a6-46b7-b2b3-e2dcabcdc8bf None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:42.234051 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 737/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 1 20:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:40:42.239446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec94d1-642b-4c37-a10c-5e4be01edb64 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:42.239853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec94d1-642b-4c37-a10c-5e4be01edb64 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:42.239967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec94d1-642b-4c37-a10c-5e4be01edb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:42.315610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c2ec94d1-642b-4c37-a10c-5e4be01edb64 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:40:42.316720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c2ec94d1-642b-4c37-a10c-5e4be01edb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:42.383573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:42.412843 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 696/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 1 20:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:40:42.579641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d038a8b-57a6-46b7-b2b3-e2dcabcdc8bf None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:40:42.581646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6d038a8b-57a6-46b7-b2b3-e2dcabcdc8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:42.599754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:42.728562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d038a8b-57a6-46b7-b2b3-e2dcabcdc8bf None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:40:42.823944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:42.851316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded259f-d871-44c7-9fed-bfa61d9417eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:40:42.851849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded259f-d871-44c7-9fed-bfa61d9417eb service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:42.852127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded259f-d871-44c7-9fed-bfa61d9417eb service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:42.852395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded259f-d871-44c7-9fed-bfa61d9417eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:42.854078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5ded259f-d871-44c7-9fed-bfa61d9417eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:42.942803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded259f-d871-44c7-9fed-bfa61d9417eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:40:42.944389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded259f-d871-44c7-9fed-bfa61d9417eb service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:40:42.962909 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 738/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:40:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:40:44.574887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81988e-8d49-4862-8925-9ad69b5733b7 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:44.575472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81988e-8d49-4862-8925-9ad69b5733b7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:44.575472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81988e-8d49-4862-8925-9ad69b5733b7 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:44.576764 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 697/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 1 20:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:40:44.582073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffee034-de05-47ec-820f-ffb96a6459fe None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:44.582475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffee034-de05-47ec-820f-ffb96a6459fe None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:44.582666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffee034-de05-47ec-820f-ffb96a6459fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:44.657384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7ffee034-de05-47ec-820f-ffb96a6459fe None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:40:44.659275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7ffee034-de05-47ec-820f-ffb96a6459fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:44.699800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:44.755620 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 739/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 1 20:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:40:44.914950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81988e-8d49-4862-8925-9ad69b5733b7 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:40:44.916390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2b81988e-8d49-4862-8925-9ad69b5733b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:45.059627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81988e-8d49-4862-8925-9ad69b5733b7 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:40:45.114344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:45.172256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9f377-057a-4306-b799-f823b8898e68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:40:45.172792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9f377-057a-4306-b799-f823b8898e68 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:45.173066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9f377-057a-4306-b799-f823b8898e68 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:45.173333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9f377-057a-4306-b799-f823b8898e68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:45.174966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-42c9f377-057a-4306-b799-f823b8898e68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:45.263070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9f377-057a-4306-b799-f823b8898e68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:40:45.264687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9f377-057a-4306-b799-f823b8898e68 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:40:45.282944 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 698/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:40:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:40:46.873191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba8c5a-bd9a-4ec8-8645-f9f6a051d936 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:46.873759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba8c5a-bd9a-4ec8-8645-f9f6a051d936 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:46.873759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba8c5a-bd9a-4ec8-8645-f9f6a051d936 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:46.875098 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 740/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 1 20:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:40:46.880541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17948af-20db-49d0-a438-3cbad6169ef9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:46.880971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17948af-20db-49d0-a438-3cbad6169ef9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:46.881096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17948af-20db-49d0-a438-3cbad6169ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:46.956328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a17948af-20db-49d0-a438-3cbad6169ef9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:40:46.957548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a17948af-20db-49d0-a438-3cbad6169ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:46.959605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:47.053742 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 699/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 1 20:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:40:47.213740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba8c5a-bd9a-4ec8-8645-f9f6a051d936 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:40:47.215330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-40ba8c5a-bd9a-4ec8-8645-f9f6a051d936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:47.361909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba8c5a-bd9a-4ec8-8645-f9f6a051d936 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:40:47.388315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:47.477010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961c524-5cc1-4598-bb49-349a145c16f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:40:47.477525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961c524-5cc1-4598-bb49-349a145c16f8 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:47.477803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961c524-5cc1-4598-bb49-349a145c16f8 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:47.478075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961c524-5cc1-4598-bb49-349a145c16f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:47.479695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b961c524-5cc1-4598-bb49-349a145c16f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:47.568132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961c524-5cc1-4598-bb49-349a145c16f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:40:47.569737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961c524-5cc1-4598-bb49-349a145c16f8 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:40:47.588525 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 741/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:40:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:40:49.893922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1ebff-ef64-466a-b473-07b15a4b593d None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:49.894535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1ebff-ef64-466a-b473-07b15a4b593d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:49.894535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1ebff-ef64-466a-b473-07b15a4b593d None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:49.895861 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 700/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 1 20:40:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:40:49.901149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0cca5-5d3d-497f-a534-18d53ae2cb83 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:49.901593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0cca5-5d3d-497f-a534-18d53ae2cb83 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:49.901697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0cca5-5d3d-497f-a534-18d53ae2cb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:49.942236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:49.967480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8db0cca5-5d3d-497f-a534-18d53ae2cb83 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:40:49.968672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8db0cca5-5d3d-497f-a534-18d53ae2cb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:50.064555 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 742/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 1 20:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:40:50.407975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1ebff-ef64-466a-b473-07b15a4b593d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:40:50.409543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0cd1ebff-ef64-466a-b473-07b15a4b593d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:50.510846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1ebff-ef64-466a-b473-07b15a4b593d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:40:50.542254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:50.585133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d8ffbd-21fd-493c-abc4-017d5e9c2232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:40:50.585662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8ffbd-21fd-493c-abc4-017d5e9c2232 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:50.585942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8ffbd-21fd-493c-abc4-017d5e9c2232 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:50.586277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8ffbd-21fd-493c-abc4-017d5e9c2232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:50.588077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-37d8ffbd-21fd-493c-abc4-017d5e9c2232 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:50.696766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d8ffbd-21fd-493c-abc4-017d5e9c2232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:40:50.698194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d8ffbd-21fd-493c-abc4-017d5e9c2232 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:40:50.716023 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 701/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:40:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:40:53.260692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0175c24-de14-4a23-8603-a494d50a7c85 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:53.261250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0175c24-de14-4a23-8603-a494d50a7c85 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:53.261250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0175c24-de14-4a23-8603-a494d50a7c85 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:53.262747 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 743/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 1 20:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:40:53.267881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bf0d8-a824-41bc-aeae-6c2baa3083d3 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:53.268272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bf0d8-a824-41bc-aeae-6c2baa3083d3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:53.268427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bf0d8-a824-41bc-aeae-6c2baa3083d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:53.336194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-be9bf0d8-a824-41bc-aeae-6c2baa3083d3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:40:53.337424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-be9bf0d8-a824-41bc-aeae-6c2baa3083d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:53.350715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:53.435343 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 702/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 1 20:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:40:53.781128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0175c24-de14-4a23-8603-a494d50a7c85 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:40:53.782692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f0175c24-de14-4a23-8603-a494d50a7c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:53.886997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0175c24-de14-4a23-8603-a494d50a7c85 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:40:53.900114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:53.962112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9949810-46a9-463f-bf9e-c689ec5400ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:40:53.962740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9949810-46a9-463f-bf9e-c689ec5400ed service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:53.963030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9949810-46a9-463f-bf9e-c689ec5400ed service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:53.963306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9949810-46a9-463f-bf9e-c689ec5400ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:53.964900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c9949810-46a9-463f-bf9e-c689ec5400ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:54.073110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9949810-46a9-463f-bf9e-c689ec5400ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:40:54.074563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9949810-46a9-463f-bf9e-c689ec5400ed service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:40:54.092721 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 744/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:40:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:40:56.818692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068df5c3-2563-45aa-97d6-e5e10c8dbfed None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:56.819233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068df5c3-2563-45aa-97d6-e5e10c8dbfed None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:56.819233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068df5c3-2563-45aa-97d6-e5e10c8dbfed None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:56.820822 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 703/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 1 20:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:40:56.826194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50bef5-4d6e-4197-8f12-30641a8c7671 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:56.826588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50bef5-4d6e-4197-8f12-30641a8c7671 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:56.826793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50bef5-4d6e-4197-8f12-30641a8c7671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:56.874987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:56.893635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7d50bef5-4d6e-4197-8f12-30641a8c7671 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:40:56.895160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7d50bef5-4d6e-4197-8f12-30641a8c7671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:56.991688 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 745/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 1 20:40:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:40:57.337385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068df5c3-2563-45aa-97d6-e5e10c8dbfed None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:40:57.338986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-068df5c3-2563-45aa-97d6-e5e10c8dbfed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:57.440969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068df5c3-2563-45aa-97d6-e5e10c8dbfed None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:40:57.481189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:57.515274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dea21-f24f-477a-89cf-1cbc6834fdd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:40:57.515799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dea21-f24f-477a-89cf-1cbc6834fdd7 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:40:57.516072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dea21-f24f-477a-89cf-1cbc6834fdd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:40:57.516338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dea21-f24f-477a-89cf-1cbc6834fdd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:40:57.518396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-355dea21-f24f-477a-89cf-1cbc6834fdd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:40:57.627788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355dea21-f24f-477a-89cf-1cbc6834fdd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:40:57.629222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355dea21-f24f-477a-89cf-1cbc6834fdd7 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:40:57.647278 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 704/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:40:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:41:00.235013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c2e59-972e-4a38-94db-ff4664278d0a None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:00.235535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c2e59-972e-4a38-94db-ff4664278d0a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:00.235630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c2e59-972e-4a38-94db-ff4664278d0a None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:00.237585 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 746/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 1 20:41:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:41:00.242986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695098e5-0394-4666-9032-445857f23d33 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:00.243387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695098e5-0394-4666-9032-445857f23d33 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:00.243559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695098e5-0394-4666-9032-445857f23d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:00.309667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-695098e5-0394-4666-9032-445857f23d33 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:41:00.310822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-695098e5-0394-4666-9032-445857f23d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:00.331589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:00.406088 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 705/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 1 20:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:41:00.755284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c2e59-972e-4a38-94db-ff4664278d0a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:41:00.756892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2b0c2e59-972e-4a38-94db-ff4664278d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:00.861863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c2e59-972e-4a38-94db-ff4664278d0a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:41:00.885296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:00.940309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f76348-c7b3-4720-a326-6c38b2a2ba6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:41:00.940913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f76348-c7b3-4720-a326-6c38b2a2ba6f service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:00.941192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f76348-c7b3-4720-a326-6c38b2a2ba6f service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:00.941459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f76348-c7b3-4720-a326-6c38b2a2ba6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:00.943081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-20f76348-c7b3-4720-a326-6c38b2a2ba6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:01.053200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f76348-c7b3-4720-a326-6c38b2a2ba6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:41:01.054668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f76348-c7b3-4720-a326-6c38b2a2ba6f service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:41:01.073289 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 747/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:41:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:41:03.535260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebd42b-2e48-4a56-8543-a895bc6b253c None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:03.535819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebd42b-2e48-4a56-8543-a895bc6b253c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:03.535819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebd42b-2e48-4a56-8543-a895bc6b253c None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:03.537170 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 706/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 1 20:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:41:03.542762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cc33c-1ae8-4931-927f-ec8d9dd82b3f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:03.543163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cc33c-1ae8-4931-927f-ec8d9dd82b3f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:03.543326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cc33c-1ae8-4931-927f-ec8d9dd82b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:03.606660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:03.609818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-dc8cc33c-1ae8-4931-927f-ec8d9dd82b3f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:41:03.611469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-dc8cc33c-1ae8-4931-927f-ec8d9dd82b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:03.709954 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 748/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 1 20:41:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:41:04.061294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebd42b-2e48-4a56-8543-a895bc6b253c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:41:04.062896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1eebd42b-2e48-4a56-8543-a895bc6b253c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:04.166130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebd42b-2e48-4a56-8543-a895bc6b253c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:41:04.215560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:04.240499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35178aad-cf41-4d45-9357-a1cf699df906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:41:04.241099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35178aad-cf41-4d45-9357-a1cf699df906 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:04.241379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35178aad-cf41-4d45-9357-a1cf699df906 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:04.241652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35178aad-cf41-4d45-9357-a1cf699df906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:04.243273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-35178aad-cf41-4d45-9357-a1cf699df906 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:04.352167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35178aad-cf41-4d45-9357-a1cf699df906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:41:04.353517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35178aad-cf41-4d45-9357-a1cf699df906 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:41:04.371677 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 707/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:41:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:41:06.851884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23f9f9-e1a2-4911-8eb4-c208a7784ce5 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:06.852511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23f9f9-e1a2-4911-8eb4-c208a7784ce5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:06.852511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23f9f9-e1a2-4911-8eb4-c208a7784ce5 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:06.853775 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 749/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 1 20:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:41:06.859199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ba444-58f8-4772-915a-e2adfdcb4ef0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:06.859630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ba444-58f8-4772-915a-e2adfdcb4ef0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:06.859738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ba444-58f8-4772-915a-e2adfdcb4ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:06.926266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ab0ba444-58f8-4772-915a-e2adfdcb4ef0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:41:06.927522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ab0ba444-58f8-4772-915a-e2adfdcb4ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:06.956934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:07.023861 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 708/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 1 20:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:41:07.366284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23f9f9-e1a2-4911-8eb4-c208a7784ce5 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:41:07.367872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9d23f9f9-e1a2-4911-8eb4-c208a7784ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:07.470869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23f9f9-e1a2-4911-8eb4-c208a7784ce5 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:41:07.500003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:07.545763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd80cf5-d33d-4364-9ad3-a27e9709a4a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:41:07.546365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd80cf5-d33d-4364-9ad3-a27e9709a4a0 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:07.546681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd80cf5-d33d-4364-9ad3-a27e9709a4a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:07.546952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd80cf5-d33d-4364-9ad3-a27e9709a4a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:07.549194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5fd80cf5-d33d-4364-9ad3-a27e9709a4a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:07.659938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd80cf5-d33d-4364-9ad3-a27e9709a4a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:41:07.661365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd80cf5-d33d-4364-9ad3-a27e9709a4a0 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:41:07.679636 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 750/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:41:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:41:10.122012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee061a7e-5339-4b6f-afa7-1c7a7dcfe9e0 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:10.122433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee061a7e-5339-4b6f-afa7-1c7a7dcfe9e0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:10.122594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee061a7e-5339-4b6f-afa7-1c7a7dcfe9e0 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:10.123965 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 709/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 1 20:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:41:10.129888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3007189-f0a1-4bfa-a7d7-863a67bab505 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:10.130291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3007189-f0a1-4bfa-a7d7-863a67bab505 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:10.130443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3007189-f0a1-4bfa-a7d7-863a67bab505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:10.196428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a3007189-f0a1-4bfa-a7d7-863a67bab505 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:41:10.197640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a3007189-f0a1-4bfa-a7d7-863a67bab505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:10.206002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:10.293925 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 751/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 1 20:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:41:10.646442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee061a7e-5339-4b6f-afa7-1c7a7dcfe9e0 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:41:10.648052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ee061a7e-5339-4b6f-afa7-1c7a7dcfe9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:10.753343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee061a7e-5339-4b6f-afa7-1c7a7dcfe9e0 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:41:10.808350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:10.826913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a304b01-cc1b-4bcf-89fa-52ddad11adf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:41:10.827514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a304b01-cc1b-4bcf-89fa-52ddad11adf1 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:10.827796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a304b01-cc1b-4bcf-89fa-52ddad11adf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:10.828071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a304b01-cc1b-4bcf-89fa-52ddad11adf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:10.830158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8a304b01-cc1b-4bcf-89fa-52ddad11adf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:10.940837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a304b01-cc1b-4bcf-89fa-52ddad11adf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:41:10.942210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a304b01-cc1b-4bcf-89fa-52ddad11adf1 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:41:10.961020 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 710/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:41:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:41:13.410613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431dff96-f467-41d7-86b1-15e21b547b59 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:13.411198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431dff96-f467-41d7-86b1-15e21b547b59 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:13.411198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431dff96-f467-41d7-86b1-15e21b547b59 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:13.412497 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 752/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 1 20:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:41:13.417682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a30ef-a7c9-42a3-aed7-e4a12f80eb46 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:13.418116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a30ef-a7c9-42a3-aed7-e4a12f80eb46 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:13.418211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a30ef-a7c9-42a3-aed7-e4a12f80eb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:13.486085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-135a30ef-a7c9-42a3-aed7-e4a12f80eb46 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:41:13.487348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-135a30ef-a7c9-42a3-aed7-e4a12f80eb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:13.526037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:13.583849 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 711/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 1 20:41:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:41:13.922021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431dff96-f467-41d7-86b1-15e21b547b59 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:41:13.924139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-431dff96-f467-41d7-86b1-15e21b547b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:14.027082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431dff96-f467-41d7-86b1-15e21b547b59 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:41:14.065646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:14.101762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd2fb5-bb47-4963-9f2d-3db9c46d9327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:41:14.102369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd2fb5-bb47-4963-9f2d-3db9c46d9327 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:14.102688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd2fb5-bb47-4963-9f2d-3db9c46d9327 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:14.102959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd2fb5-bb47-4963-9f2d-3db9c46d9327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:14.104549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-eedd2fb5-bb47-4963-9f2d-3db9c46d9327 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:14.213896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd2fb5-bb47-4963-9f2d-3db9c46d9327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:41:14.215337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd2fb5-bb47-4963-9f2d-3db9c46d9327 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:41:14.233194 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 753/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:41:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:41:16.646439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68b083-3901-4215-9d95-bc1e31c51397 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:16.647023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68b083-3901-4215-9d95-bc1e31c51397 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:16.647023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68b083-3901-4215-9d95-bc1e31c51397 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:16.648337 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 712/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 1 20:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:41:16.653572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037456c-12bf-4d5f-85d2-f826073cc9b0 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:16.654033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037456c-12bf-4d5f-85d2-f826073cc9b0 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:16.654142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037456c-12bf-4d5f-85d2-f826073cc9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:16.722155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c037456c-12bf-4d5f-85d2-f826073cc9b0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:41:16.723467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c037456c-12bf-4d5f-85d2-f826073cc9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:16.742668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:16.819212 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 754/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 1 20:41:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:41:17.161699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68b083-3901-4215-9d95-bc1e31c51397 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:41:17.163276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ba68b083-3901-4215-9d95-bc1e31c51397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:17.264217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68b083-3901-4215-9d95-bc1e31c51397 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:41:17.330866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:17.339290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f639d13-2e11-40f0-90d1-20dc7be2eb67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:41:17.339893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f639d13-2e11-40f0-90d1-20dc7be2eb67 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:17.340180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f639d13-2e11-40f0-90d1-20dc7be2eb67 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:17.340450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f639d13-2e11-40f0-90d1-20dc7be2eb67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:17.342019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6f639d13-2e11-40f0-90d1-20dc7be2eb67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:17.448090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f639d13-2e11-40f0-90d1-20dc7be2eb67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:41:17.449516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f639d13-2e11-40f0-90d1-20dc7be2eb67 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:41:17.467453 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 713/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:41:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:41:19.888867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082346c-aae9-4c2e-85a4-b7f61033b1d5 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:19.889396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082346c-aae9-4c2e-85a4-b7f61033b1d5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:19.889396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082346c-aae9-4c2e-85a4-b7f61033b1d5 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:19.890780 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 755/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 1 20:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:41:19.896112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46544b6d-bb95-4d99-935a-2661347bd835 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:19.896559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46544b6d-bb95-4d99-935a-2661347bd835 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:19.896663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46544b6d-bb95-4d99-935a-2661347bd835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:19.961616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-46544b6d-bb95-4d99-935a-2661347bd835 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:41:19.962821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-46544b6d-bb95-4d99-935a-2661347bd835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:20.010907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:20.060420 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 714/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 1 20:41:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:41:20.397784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082346c-aae9-4c2e-85a4-b7f61033b1d5 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:41:20.399381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3082346c-aae9-4c2e-85a4-b7f61033b1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:20.503132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082346c-aae9-4c2e-85a4-b7f61033b1d5 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:41:20.550795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:20.577232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf97cd-0be0-4f68-892f-f8f0f9a67f68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:41:20.577860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf97cd-0be0-4f68-892f-f8f0f9a67f68 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:20.578141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf97cd-0be0-4f68-892f-f8f0f9a67f68 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:20.578417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf97cd-0be0-4f68-892f-f8f0f9a67f68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:20.580641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b7bf97cd-0be0-4f68-892f-f8f0f9a67f68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:20.690686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf97cd-0be0-4f68-892f-f8f0f9a67f68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:41:20.692114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf97cd-0be0-4f68-892f-f8f0f9a67f68 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:41:20.710311 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 756/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:41:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:41:30.058907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=22 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:30.708094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=66 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:45.220823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d77a78-dbdd-4a98-9de9-6a033b3da53f None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:45.221414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d77a78-dbdd-4a98-9de9-6a033b3da53f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:45.221414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d77a78-dbdd-4a98-9de9-6a033b3da53f None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:45.222747 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 715/1471] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:41:45.227947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5e4d9-f5fb-4f37-8b08-2960fd072382 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:45.228390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5e4d9-f5fb-4f37-8b08-2960fd072382 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:45.228567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5e4d9-f5fb-4f37-8b08-2960fd072382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:45.294593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-35a5e4d9-f5fb-4f37-8b08-2960fd072382 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:41:45.295719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-35a5e4d9-f5fb-4f37-8b08-2960fd072382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:45.393630 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 757/1472] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:41:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:41:45.746657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728fbf7-15b3-4abe-a519-2d08d9a29214 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:45.746970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728fbf7-15b3-4abe-a519-2d08d9a29214 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:45.747246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728fbf7-15b3-4abe-a519-2d08d9a29214 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:45.748582 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 716/1473] 10.222.0.17 () {54 vars in 962 bytes} [Sat Oct 1 20:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:41:45.754144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a434326-27b5-4d1f-9453-635aff9cd772 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:45.754431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a434326-27b5-4d1f-9453-635aff9cd772 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:45.754744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a434326-27b5-4d1f-9453-635aff9cd772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:45.831393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4a434326-27b5-4d1f-9453-635aff9cd772 None None] MFA Rules not processed for user `e2927203ce394fabab76dd52a68c2436`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:41:45.832639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4a434326-27b5-4d1f-9453-635aff9cd772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:45.853075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:45.908140 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 758/1474] 10.222.0.17 () {58 vars in 1059 bytes} [Sat Oct 1 20:41:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:41:45.916074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728fbf7-15b3-4abe-a519-2d08d9a29214 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:41:45.917567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2728fbf7-15b3-4abe-a519-2d08d9a29214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:46.019709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728fbf7-15b3-4abe-a519-2d08d9a29214 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:41:46.092908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5f3d3-2fee-4456-a85c-4b963e8b578b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:41:46.093509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5f3d3-2fee-4456-a85c-4b963e8b578b service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:46.093786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5f3d3-2fee-4456-a85c-4b963e8b578b service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:46.094055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5f3d3-2fee-4456-a85c-4b963e8b578b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:46.095683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a8c5f3d3-2fee-4456-a85c-4b963e8b578b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:46.148840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:46.203954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c5f3d3-2fee-4456-a85c-4b963e8b578b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:41:46.205398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c5f3d3-2fee-4456-a85c-4b963e8b578b service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:41:46.223338 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 717/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:41:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:41:48.756727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067d535-c463-4332-945d-a30d3003ab6c None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:48.757299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067d535-c463-4332-945d-a30d3003ab6c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:48.757299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067d535-c463-4332-945d-a30d3003ab6c None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:48.758652 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 759/1476] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:41:48.763870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbe2d6-2fe4-4eb5-84f8-19226da15f88 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:48.764280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbe2d6-2fe4-4eb5-84f8-19226da15f88 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:48.764443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbe2d6-2fe4-4eb5-84f8-19226da15f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:48.829078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8fdbe2d6-2fe4-4eb5-84f8-19226da15f88 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:41:48.830270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8fdbe2d6-2fe4-4eb5-84f8-19226da15f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:49.057780 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 718/1477] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:41:49.388348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067d535-c463-4332-945d-a30d3003ab6c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:41:49.389817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3067d535-c463-4332-945d-a30d3003ab6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:49.515138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067d535-c463-4332-945d-a30d3003ab6c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:41:49.566028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:49.611639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9be9fa-35b1-4a5e-a5a9-8f20cc6d05d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:41:49.612245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9be9fa-35b1-4a5e-a5a9-8f20cc6d05d0 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:49.612527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9be9fa-35b1-4a5e-a5a9-8f20cc6d05d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:49.612797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9be9fa-35b1-4a5e-a5a9-8f20cc6d05d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:49.613430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9be9fa-35b1-4a5e-a5a9-8f20cc6d05d0 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:41:49.615186 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-5f9be9fa-35b1-4a5e-a5a9-8f20cc6d05d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:41:49.615186 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 20:41:49.615186 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:41:49.616324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9be9fa-35b1-4a5e-a5a9-8f20cc6d05d0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:41:49.634926 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 760/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Oct 1 20:41:49 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:41:52.030091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14a97b-62f1-46ac-ba2f-3b8b1be0d4a1 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:52.030671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14a97b-62f1-46ac-ba2f-3b8b1be0d4a1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:52.030671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14a97b-62f1-46ac-ba2f-3b8b1be0d4a1 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:52.032005 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 719/1479] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:41:52.037380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ccf58-8864-4e40-93e4-48c0c1c613e6 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:52.037833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ccf58-8864-4e40-93e4-48c0c1c613e6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:52.037950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ccf58-8864-4e40-93e4-48c0c1c613e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:52.107787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-bc9ccf58-8864-4e40-93e4-48c0c1c613e6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:41:52.109537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bc9ccf58-8864-4e40-93e4-48c0c1c613e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:52.205218 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 761/1480] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:41:52.556484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14a97b-62f1-46ac-ba2f-3b8b1be0d4a1 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:41:52.558101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bc14a97b-62f1-46ac-ba2f-3b8b1be0d4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:52.567652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:52.708033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14a97b-62f1-46ac-ba2f-3b8b1be0d4a1 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:41:52.790263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:52.822516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71d790-3363-4502-9bc7-f6d5f7d22f10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:41:52.823046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71d790-3363-4502-9bc7-f6d5f7d22f10 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:52.823318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71d790-3363-4502-9bc7-f6d5f7d22f10 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:52.823582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71d790-3363-4502-9bc7-f6d5f7d22f10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:52.825150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5b71d790-3363-4502-9bc7-f6d5f7d22f10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:52.913710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71d790-3363-4502-9bc7-f6d5f7d22f10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:41:52.915418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71d790-3363-4502-9bc7-f6d5f7d22f10 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:41:52.933256 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 720/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:41:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:41:55.540712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7baf2-8d1d-4cea-a001-c3f9071228f6 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:55.541253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7baf2-8d1d-4cea-a001-c3f9071228f6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:55.541369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7baf2-8d1d-4cea-a001-c3f9071228f6 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:55.542686 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 762/1482] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:41:55.548342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18628861-cd8d-43f3-8fa1-50df2e032044 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:55.548794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18628861-cd8d-43f3-8fa1-50df2e032044 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:55.548899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18628861-cd8d-43f3-8fa1-50df2e032044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:55.613584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-18628861-cd8d-43f3-8fa1-50df2e032044 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:41:55.614822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-18628861-cd8d-43f3-8fa1-50df2e032044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:55.661357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:55.711526 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 721/1483] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:41:56.054224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7baf2-8d1d-4cea-a001-c3f9071228f6 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:41:56.055852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9ce7baf2-8d1d-4cea-a001-c3f9071228f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:56.080208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:56.160680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7baf2-8d1d-4cea-a001-c3f9071228f6 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:41:56.233233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943bc68b-05df-4b9f-872d-8c37a29f7817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:41:56.233857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bc68b-05df-4b9f-872d-8c37a29f7817 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:56.234138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bc68b-05df-4b9f-872d-8c37a29f7817 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:56.234409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bc68b-05df-4b9f-872d-8c37a29f7817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:56.236036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-943bc68b-05df-4b9f-872d-8c37a29f7817 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:56.312809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:56.348212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943bc68b-05df-4b9f-872d-8c37a29f7817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:41:56.349570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943bc68b-05df-4b9f-872d-8c37a29f7817 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:41:56.367090 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 763/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:41:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:41:58.809431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60192f7d-993b-4a94-9af5-f199dce48606 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:58.809970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60192f7d-993b-4a94-9af5-f199dce48606 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:58.809970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60192f7d-993b-4a94-9af5-f199dce48606 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:58.811328 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 722/1485] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:41:58.816556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa859a04-aa95-45eb-9847-07ad92d87ec2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:58.816982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa859a04-aa95-45eb-9847-07ad92d87ec2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:58.817113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa859a04-aa95-45eb-9847-07ad92d87ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:58.882226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-aa859a04-aa95-45eb-9847-07ad92d87ec2 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:41:58.883458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-aa859a04-aa95-45eb-9847-07ad92d87ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:58.977510 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 764/1486] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:41:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:41:59.316038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60192f7d-993b-4a94-9af5-f199dce48606 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:41:59.317555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-60192f7d-993b-4a94-9af5-f199dce48606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:59.418445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60192f7d-993b-4a94-9af5-f199dce48606 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:41:59.493081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:59.494573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ee06f-5018-4f00-b6d3-2e7b49c07c83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:41:59.495253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ee06f-5018-4f00-b6d3-2e7b49c07c83 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:41:59.495718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ee06f-5018-4f00-b6d3-2e7b49c07c83 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:41:59.496315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ee06f-5018-4f00-b6d3-2e7b49c07c83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:41:59.498185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e82ee06f-5018-4f00-b6d3-2e7b49c07c83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:41:59.612245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ee06f-5018-4f00-b6d3-2e7b49c07c83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:41:59.613598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ee06f-5018-4f00-b6d3-2e7b49c07c83 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:41:59.631983 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 723/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:41:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:42:02.157525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c324da-605d-488a-a51b-7f0f4c12c326 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:42:02.157947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c324da-605d-488a-a51b-7f0f4c12c326 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:42:02.158065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c324da-605d-488a-a51b-7f0f4c12c326 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:42:02.159450 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 765/1488] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:42:02.164709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1940f37-2b70-4bce-9e2e-1893c85db411 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:42:02.165151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1940f37-2b70-4bce-9e2e-1893c85db411 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:42:02.165273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1940f37-2b70-4bce-9e2e-1893c85db411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:42:02.230799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f1940f37-2b70-4bce-9e2e-1893c85db411 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:42:02.231989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f1940f37-2b70-4bce-9e2e-1893c85db411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:42:02.283448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:42:02.327859 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 724/1489] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:42:02.670270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c324da-605d-488a-a51b-7f0f4c12c326 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:42:02.671857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e5c324da-605d-488a-a51b-7f0f4c12c326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:42:02.707583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:42:02.776900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c324da-605d-488a-a51b-7f0f4c12c326 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:42:02.851620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbd52d-0a12-4165-b7cc-12439092525f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:42:02.852226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbd52d-0a12-4165-b7cc-12439092525f service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:42:02.852501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbd52d-0a12-4165-b7cc-12439092525f service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:42:02.852769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbd52d-0a12-4165-b7cc-12439092525f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:42:02.854342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-47dbd52d-0a12-4165-b7cc-12439092525f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:42:02.934123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:42:02.964142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbd52d-0a12-4165-b7cc-12439092525f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:42:02.965709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbd52d-0a12-4165-b7cc-12439092525f service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:42:02.984188 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 766/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:42:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:42:05.518809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ce0d7-404b-4a8a-aa7d-9d5b499f083b None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:42:05.519429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ce0d7-404b-4a8a-aa7d-9d5b499f083b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:42:05.519429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ce0d7-404b-4a8a-aa7d-9d5b499f083b None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:42:05.520743 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 725/1491] 10.222.0.17 () {54 vars in 927 bytes} [Sat Oct 1 20:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:42:05.525941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe3969-7984-4029-a6d5-207c093d8074 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:42:05.526363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe3969-7984-4029-a6d5-207c093d8074 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:42:05.526482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe3969-7984-4029-a6d5-207c093d8074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:42:05.593433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f3fe3969-7984-4029-a6d5-207c093d8074 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:42:05.594724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f3fe3969-7984-4029-a6d5-207c093d8074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:42:05.690411 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 767/1492] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Oct 1 20:42:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:42:06.038905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ce0d7-404b-4a8a-aa7d-9d5b499f083b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:42:06.040481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-205ce0d7-404b-4a8a-aa7d-9d5b499f083b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:42:06.142860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ce0d7-404b-4a8a-aa7d-9d5b499f083b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:42:06.216074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df2002-3159-49df-9bd8-ef42600d3660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:42:06.216673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df2002-3159-49df-9bd8-ef42600d3660 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:42:06.216953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df2002-3159-49df-9bd8-ef42600d3660 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:42:06.217224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df2002-3159-49df-9bd8-ef42600d3660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:42:06.218828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-23df2002-3159-49df-9bd8-ef42600d3660 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:42:06.228019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:42:06.329446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df2002-3159-49df-9bd8-ef42600d3660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:42:06.330838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df2002-3159-49df-9bd8-ef42600d3660 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:42:06.348175 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 726/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:42:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:42:15.688929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=82 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:42:16.345999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=50 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:43:20.657862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92624c6-50d4-4ed4-aecd-e5b905656f3c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:43:20.658439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92624c6-50d4-4ed4-aecd-e5b905656f3c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:43:20.658439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92624c6-50d4-4ed4-aecd-e5b905656f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:43:20.735181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d92624c6-50d4-4ed4-aecd-e5b905656f3c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:43:20.736403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d92624c6-50d4-4ed4-aecd-e5b905656f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:43:20.832217 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 768/1494] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:43:20.840338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c8ec3-a1af-418c-819c-64fd6a6bc748 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:43:20.840849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c8ec3-a1af-418c-819c-64fd6a6bc748 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:43:20.840952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c8ec3-a1af-418c-819c-64fd6a6bc748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:43:20.915603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e40c8ec3-a1af-418c-819c-64fd6a6bc748 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:43:20.916704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e40c8ec3-a1af-418c-819c-64fd6a6bc748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:43:21.012451 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 727/1495] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:43:21.018065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92624c6-50d4-4ed4-aecd-e5b905656f3c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:43:21.019860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d92624c6-50d4-4ed4-aecd-e5b905656f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:43:21.031703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92624c6-50d4-4ed4-aecd-e5b905656f3c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:43:21.104178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:43:21.137158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a3dbf-e2e9-477a-a803-c394498fd575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:43:21.137760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a3dbf-e2e9-477a-a803-c394498fd575 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:43:21.138724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a3dbf-e2e9-477a-a803-c394498fd575 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:43:21.139001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a3dbf-e2e9-477a-a803-c394498fd575 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:43:21.139641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a3dbf-e2e9-477a-a803-c394498fd575 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:43:21.139945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a3dbf-e2e9-477a-a803-c394498fd575 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:43:21.140940 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-886a3dbf-e2e9-477a-a803-c394498fd575 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:43:21.140940 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 20:43:21.140940 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:43:21.141826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a3dbf-e2e9-477a-a803-c394498fd575 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:43:21.153550 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 769/1496] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:43:21.161808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c8ec3-a1af-418c-819c-64fd6a6bc748 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:43:21.170995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c8ec3-a1af-418c-819c-64fd6a6bc748 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:43:21.231837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:43:21.266069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae2507-101c-4090-80e4-51675093c008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:43:21.266703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae2507-101c-4090-80e4-51675093c008 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:43:21.266980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae2507-101c-4090-80e4-51675093c008 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:43:21.267250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae2507-101c-4090-80e4-51675093c008 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:43:21.267877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae2507-101c-4090-80e4-51675093c008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:43:21.269597 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-32ae2507-101c-4090-80e4-51675093c008 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:43:21.269597 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:43:21.269597 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:43:21.270813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae2507-101c-4090-80e4-51675093c008 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:43:21.331172 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 728/1497] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:43:21 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:43:21.335505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a3dbf-e2e9-477a-a803-c394498fd575 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:43:21.337025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-886a3dbf-e2e9-477a-a803-c394498fd575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:43:21.369580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a3dbf-e2e9-477a-a803-c394498fd575 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:43:21.465706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a8515-e5b5-467b-9d19-7ca62665a602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:43:21.466339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a8515-e5b5-467b-9d19-7ca62665a602 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:43:21.466656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a8515-e5b5-467b-9d19-7ca62665a602 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:43:21.466928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a8515-e5b5-467b-9d19-7ca62665a602 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:43:21.467549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a8515-e5b5-467b-9d19-7ca62665a602 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:43:21.468552 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-719a8515-e5b5-467b-9d19-7ca62665a602 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:43:21.468552 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 20:43:21.468552 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:43:21.469746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a8515-e5b5-467b-9d19-7ca62665a602 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:43:21.511714 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 770/1498] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:43:21 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:43:21.516491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae2507-101c-4090-80e4-51675093c008 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:43:21.518093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-32ae2507-101c-4090-80e4-51675093c008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:43:21.579055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae2507-101c-4090-80e4-51675093c008 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:43:21.661070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:43:21.672940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61dd26-77ff-41f3-8311-7a1523bd4709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:43:21.673537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61dd26-77ff-41f3-8311-7a1523bd4709 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:43:21.673812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61dd26-77ff-41f3-8311-7a1523bd4709 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:43:21.674081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61dd26-77ff-41f3-8311-7a1523bd4709 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:43:21.674752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd61dd26-77ff-41f3-8311-7a1523bd4709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:43:21.675762 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-fd61dd26-77ff-41f3-8311-7a1523bd4709 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:43:21.675762 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:43:21.675762 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:43:21.676651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd61dd26-77ff-41f3-8311-7a1523bd4709 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:43:21.689402 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 729/1499] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:43:21.693472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a8515-e5b5-467b-9d19-7ca62665a602 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:43:21.695268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-719a8515-e5b5-467b-9d19-7ca62665a602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:43:21.706739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a8515-e5b5-467b-9d19-7ca62665a602 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:43:21.731851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:43:21.802460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251eff3-7b45-45ba-9e5d-a660d55ae8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:43:21.803147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251eff3-7b45-45ba-9e5d-a660d55ae8fb admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:43:21.803425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251eff3-7b45-45ba-9e5d-a660d55ae8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:43:21.803760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251eff3-7b45-45ba-9e5d-a660d55ae8fb admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:43:21.804432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251eff3-7b45-45ba-9e5d-a660d55ae8fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:43:21.805438 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-6251eff3-7b45-45ba-9e5d-a660d55ae8fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:43:21.805438 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 20:43:21.805438 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:43:21.806329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251eff3-7b45-45ba-9e5d-a660d55ae8fb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:43:21.819349 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 771/1500] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:43:21.824090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61dd26-77ff-41f3-8311-7a1523bd4709 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:43:21.826196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fd61dd26-77ff-41f3-8311-7a1523bd4709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:43:21.859265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61dd26-77ff-41f3-8311-7a1523bd4709 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:43:21.952729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d63eac0-ed4e-4501-9f87-7dca5ab7e7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:43:21.953375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63eac0-ed4e-4501-9f87-7dca5ab7e7a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:43:21.953656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63eac0-ed4e-4501-9f87-7dca5ab7e7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:43:21.953929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63eac0-ed4e-4501-9f87-7dca5ab7e7a9 admin admin] PATH_INFO: `/v3/projects/899b038b3b47461f88ba2e2f30423fec/users/c8dfcb1f9ee043f68b8a2fb7eee78be3/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:43:21.975616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d63eac0-ed4e-4501-9f87-7dca5ab7e7a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=899b038b3b47461f88ba2e2f30423fec, user_id=c8dfcb1f9ee043f68b8a2fb7eee78be3, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:43:21.976633 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-4d63eac0-ed4e-4501-9f87-7dca5ab7e7a9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:43:21.976633 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:43:21.976633 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:43:21.978591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d63eac0-ed4e-4501-9f87-7dca5ab7e7a9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:43:22.004073 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 730/1501] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:43:21 2022] PUT /identity/v3/projects/899b038b3b47461f88ba2e2f30423fec/users/c8dfcb1f9ee043f68b8a2fb7eee78be3/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:43:22.016671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b457c-397e-442f-9003-7b5c5b0ceb87 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:43:22.016984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b457c-397e-442f-9003-7b5c5b0ceb87 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:43:22.017259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b457c-397e-442f-9003-7b5c5b0ceb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:43:22.075418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-164b457c-397e-442f-9003-7b5c5b0ceb87 None None] MFA Rules not processed for user `c8dfcb1f9ee043f68b8a2fb7eee78be3`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:43:22.076622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-164b457c-397e-442f-9003-7b5c5b0ceb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:43:22.153188 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 772/1502] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:43:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:43:22.166674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e7a0a-80a7-4bf2-9505-6e51633b8a02 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:43:22.166957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e7a0a-80a7-4bf2-9505-6e51633b8a02 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:43:22.167226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e7a0a-80a7-4bf2-9505-6e51633b8a02 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:43:22.168536 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 731/1503] 10.222.0.17 () {54 vars in 963 bytes} [Sat Oct 1 20:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:43:22.173939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786ac54-a685-4f84-93f1-8d655f308bdb None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:43:22.174229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786ac54-a685-4f84-93f1-8d655f308bdb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:43:22.174529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786ac54-a685-4f84-93f1-8d655f308bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:43:22.206441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:43:22.251148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f786ac54-a685-4f84-93f1-8d655f308bdb None None] MFA Rules not processed for user `197a0ae6b71f4635a069ac0238e017c9`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:43:22.252360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f786ac54-a685-4f84-93f1-8d655f308bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:43:22.327425 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 773/1504] 10.222.0.17 () {58 vars in 1060 bytes} [Sat Oct 1 20:43:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:43:22.335175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e7a0a-80a7-4bf2-9505-6e51633b8a02 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:43:22.336679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0e7e7a0a-80a7-4bf2-9505-6e51633b8a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:43:22.387199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:43:22.442171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e7a0a-80a7-4bf2-9505-6e51633b8a02 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:43:22.513893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8a855-8ffe-489b-892a-b5f81a8be3b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:43:22.514484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8a855-8ffe-489b-892a-b5f81a8be3b2 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:43:22.514791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8a855-8ffe-489b-892a-b5f81a8be3b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:43:22.515061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8a855-8ffe-489b-892a-b5f81a8be3b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:43:22.516637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-21f8a855-8ffe-489b-892a-b5f81a8be3b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:43:22.613311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f8a855-8ffe-489b-892a-b5f81a8be3b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:43:22.614692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f8a855-8ffe-489b-892a-b5f81a8be3b2 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:43:22.618073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:43:22.632919 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 732/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:43:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:43:22.678319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9b856-8a3b-4c07-860e-4a75c69b2c85 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:43:22.678712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9b856-8a3b-4c07-860e-4a75c69b2c85 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:43:22.679060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9b856-8a3b-4c07-860e-4a75c69b2c85 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:43:22.680380 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 774/1506] 10.222.0.17 () {52 vars in 830 bytes} [Sat Oct 1 20:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:43:22.684509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8a855-8ffe-489b-892a-b5f81a8be3b2 service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:43:22.686241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-21f8a855-8ffe-489b-892a-b5f81a8be3b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:43:22.786683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8a855-8ffe-489b-892a-b5f81a8be3b2 service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:43:22.881123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d88e59-7af8-4134-a747-e815307d6078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:43:22.881811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d88e59-7af8-4134-a747-e815307d6078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:43:22.881936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d88e59-7af8-4134-a747-e815307d6078 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:43:22.882183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d88e59-7af8-4134-a747-e815307d6078 admin admin] PATH_INFO: `/v3/users/c8dfcb1f9ee043f68b8a2fb7eee78be3` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:43:22.895534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d88e59-7af8-4134-a747-e815307d6078 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8dfcb1f9ee043f68b8a2fb7eee78be3)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:43:22.896556 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-35d88e59-7af8-4134-a747-e815307d6078 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:43:22.896556 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:43:22.896556 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:43:22.897774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d88e59-7af8-4134-a747-e815307d6078 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:43:22.908138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:43:22.963578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-35d88e59-7af8-4134-a747-e815307d6078 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8dfcb1f9ee043f68b8a2fb7eee78be3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:43:22.964004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-35d88e59-7af8-4134-a747-e815307d6078 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8dfcb1f9ee043f68b8a2fb7eee78be3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:43:22.964517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-35d88e59-7af8-4134-a747-e815307d6078 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8dfcb1f9ee043f68b8a2fb7eee78be3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:43:22.974289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-35d88e59-7af8-4134-a747-e815307d6078 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8dfcb1f9ee043f68b8a2fb7eee78be3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:43:22.996915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-35d88e59-7af8-4134-a747-e815307d6078 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8dfcb1f9ee043f68b8a2fb7eee78be3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:43:23.015835 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 733/1507] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:43:22 2022] DELETE /identity/v3/users/c8dfcb1f9ee043f68b8a2fb7eee78be3 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:43:23.020316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b52084-718c-461b-893d-3ee4325d9afb None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:43:23.020820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b52084-718c-461b-893d-3ee4325d9afb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:43:23.020940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b52084-718c-461b-893d-3ee4325d9afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:43:23.078436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e7b52084-718c-461b-893d-3ee4325d9afb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:43:23.079661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e7b52084-718c-461b-893d-3ee4325d9afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:43:23.155126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:43:23.168270 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 775/1508] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:43:23.179838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d88e59-7af8-4134-a747-e815307d6078 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:43:23.181058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-35d88e59-7af8-4134-a747-e815307d6078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:43:23.277843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d88e59-7af8-4134-a747-e815307d6078 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:43:23.333158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:43:23.354216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a85361-fd6c-4763-b2cc-5f807adebad0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:43:23.354865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a85361-fd6c-4763-b2cc-5f807adebad0 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:43:23.355145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a85361-fd6c-4763-b2cc-5f807adebad0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:43:23.355429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a85361-fd6c-4763-b2cc-5f807adebad0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:43:23.357666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-06a85361-fd6c-4763-b2cc-5f807adebad0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:43:23.464382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a85361-fd6c-4763-b2cc-5f807adebad0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:43:23.465829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a85361-fd6c-4763-b2cc-5f807adebad0 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:43:23.486602 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 734/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:43:23.848892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b52084-718c-461b-893d-3ee4325d9afb None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:43:23.850242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e7b52084-718c-461b-893d-3ee4325d9afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:43:23.862144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b52084-718c-461b-893d-3ee4325d9afb None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:43:23.971392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff875902-df3e-498a-b470-2094b26bac5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:43:23.972025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff875902-df3e-498a-b470-2094b26bac5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:43:23.972352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff875902-df3e-498a-b470-2094b26bac5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:43:23.972571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff875902-df3e-498a-b470-2094b26bac5f admin admin] PATH_INFO: `/v3/projects/899b038b3b47461f88ba2e2f30423fec` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:43:23.983038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff875902-df3e-498a-b470-2094b26bac5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=899b038b3b47461f88ba2e2f30423fec)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:43:23.984059 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-ff875902-df3e-498a-b470-2094b26bac5f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:43:23.984059 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 20:43:23.984059 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:43:23.985251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff875902-df3e-498a-b470-2094b26bac5f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:43:24.044410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ff875902-df3e-498a-b470-2094b26bac5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '899b038b3b47461f88ba2e2f30423fec', 'request_id': 'req-ff875902-df3e-498a-b470-2094b26bac5f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:43:24.057714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ff875902-df3e-498a-b470-2094b26bac5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '899b038b3b47461f88ba2e2f30423fec', 'request_id': 'req-ff875902-df3e-498a-b470-2094b26bac5f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:43:24.064293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ff875902-df3e-498a-b470-2094b26bac5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '899b038b3b47461f88ba2e2f30423fec', 'request_id': 'req-ff875902-df3e-498a-b470-2094b26bac5f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:43:24.091649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ff875902-df3e-498a-b470-2094b26bac5f admin admin] The token cache is being invalidate because project 899b038b3b47461f88ba2e2f30423fec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:43:24.091940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ff875902-df3e-498a-b470-2094b26bac5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:43:24.092604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ff875902-df3e-498a-b470-2094b26bac5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:43:24.094215 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 776/1510] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:43:23 2022] DELETE /identity/v3/projects/899b038b3b47461f88ba2e2f30423fec => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:43:33.484343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=62 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:43:34.089421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:46:24.369417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba964943-3e50-480c-8462-40e4d32561f8 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:46:24.369975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba964943-3e50-480c-8462-40e4d32561f8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:46:24.369975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba964943-3e50-480c-8462-40e4d32561f8 None None] PATH_INFO: `/v3` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:46:24.371194 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 735/1511] 10.222.0.39 () {54 vars in 925 bytes} [Sat Oct 1 20:46:24 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:46:24.377280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5a0a0-595b-4f68-8629-5234d8407118 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:46:24.377842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5a0a0-595b-4f68-8629-5234d8407118 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:46:24.377842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5a0a0-595b-4f68-8629-5234d8407118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:46:24.448868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15281d6-e017-4c6a-aae5-e810fca465f0 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:46:24.449329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15281d6-e017-4c6a-aae5-e810fca465f0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:46:24.449614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15281d6-e017-4c6a-aae5-e810fca465f0 None None] PATH_INFO: `/v3` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:46:24.450928 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 736/1512] 10.222.0.27 () {54 vars in 925 bytes} [Sat Oct 1 20:46:24 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:46:24.456460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccc53b-af63-4720-980e-3385880855da None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:46:24.456744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccc53b-af63-4720-980e-3385880855da None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:46:24.457019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccc53b-af63-4720-980e-3385880855da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:46:24.458957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f9d5a0a0-595b-4f68-8629-5234d8407118 None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:46:24.460248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f9d5a0a0-595b-4f68-8629-5234d8407118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:46:24.476454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bdccc53b-af63-4720-980e-3385880855da None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:46:24.477621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bdccc53b-af63-4720-980e-3385880855da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:46:24.554705 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 777/1513] 10.222.0.39 () {58 vars in 1015 bytes} [Sat Oct 1 20:46:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:46:24.555298 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 737/1514] 10.222.0.27 () {58 vars in 1015 bytes} [Sat Oct 1 20:46:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:46:28.189791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453611e1-330f-49d4-8847-b857331e6ee5 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:46:28.190380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453611e1-330f-49d4-8847-b857331e6ee5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:46:28.190380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453611e1-330f-49d4-8847-b857331e6ee5 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:46:28.190601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61eea9-98a1-4aac-b2fe-b51442e52d3c None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:46:28.190961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61eea9-98a1-4aac-b2fe-b51442e52d3c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:46:28.191113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61eea9-98a1-4aac-b2fe-b51442e52d3c None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:46:28.191852 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 738/1515] 10.222.0.17 () {54 vars in 961 bytes} [Sat Oct 1 20:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:46:28.192976 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 778/1516] 10.222.0.17 () {54 vars in 961 bytes} [Sat Oct 1 20:46:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:46:28.197144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46bc4b-719e-4d22-bd66-4071987d5648 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:46:28.197425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46bc4b-719e-4d22-bd66-4071987d5648 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:46:28.197726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46bc4b-719e-4d22-bd66-4071987d5648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:46:28.198096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654eb28-0494-4bb7-b474-8043fa97c404 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:46:28.198385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654eb28-0494-4bb7-b474-8043fa97c404 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:46:28.198703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654eb28-0494-4bb7-b474-8043fa97c404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:46:28.217237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6c46bc4b-719e-4d22-bd66-4071987d5648 None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:46:28.218278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e654eb28-0494-4bb7-b474-8043fa97c404 None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:46:28.218407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6c46bc4b-719e-4d22-bd66-4071987d5648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:46:28.219581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e654eb28-0494-4bb7-b474-8043fa97c404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:46:28.226963 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 739/1517] 10.222.0.17 () {58 vars in 1058 bytes} [Sat Oct 1 20:46:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:46:28.228482 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 779/1518] 10.222.0.17 () {58 vars in 1058 bytes} [Sat Oct 1 20:46:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:46:28.234736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c46bc4b-719e-4d22-bd66-4071987d5648 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:46:28.236165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654eb28-0494-4bb7-b474-8043fa97c404 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:46:28.244213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c46bc4b-719e-4d22-bd66-4071987d5648 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:46:28.246276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654eb28-0494-4bb7-b474-8043fa97c404 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:46:28.250015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ff4a5-404d-4cb2-a927-667bfaf9b818 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:46:28.250693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ff4a5-404d-4cb2-a927-667bfaf9b818 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:46:28.250965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ff4a5-404d-4cb2-a927-667bfaf9b818 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:46:28.251241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ff4a5-404d-4cb2-a927-667bfaf9b818 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:46:28.252021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0183ae0-d488-405b-a014-841e42e8be4a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:46:28.252597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0183ae0-d488-405b-a014-841e42e8be4a service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:46:28.252937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0183ae0-d488-405b-a014-841e42e8be4a service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:46:28.253222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0183ae0-d488-405b-a014-841e42e8be4a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:46:28.259486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13ff4a5-404d-4cb2-a927-667bfaf9b818 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:46:28.261050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13ff4a5-404d-4cb2-a927-667bfaf9b818 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:46:28.261712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0183ae0-d488-405b-a014-841e42e8be4a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:46:28.263425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0183ae0-d488-405b-a014-841e42e8be4a service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:46:28.271318 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 740/1519] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:46:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:46:28.273296 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 780/1520] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:46:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:46:38.266644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=42 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:46:38.269058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=72 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:47:05.605483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454131b4-894a-4932-bfa1-1e702b9c5e2a None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:47:05.606076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454131b4-894a-4932-bfa1-1e702b9c5e2a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:47:05.606076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454131b4-894a-4932-bfa1-1e702b9c5e2a None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:47:05.607455 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 741/1521] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 1 20:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:47:05.612334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafac21-1a80-47f8-a153-ad9a620c7cd1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:47:05.612738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafac21-1a80-47f8-a153-ad9a620c7cd1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:47:05.612879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafac21-1a80-47f8-a153-ad9a620c7cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:47:05.659942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8dafac21-1a80-47f8-a153-ad9a620c7cd1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:47:05.661617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8dafac21-1a80-47f8-a153-ad9a620c7cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:47:05.732208 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 781/1522] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 1 20:47:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:47:05.895028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454131b4-894a-4932-bfa1-1e702b9c5e2a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:47:05.896747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-454131b4-894a-4932-bfa1-1e702b9c5e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:47:05.997449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454131b4-894a-4932-bfa1-1e702b9c5e2a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:47:06.002929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f02e530-52e5-4a46-984f-f600f8401b56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:47:06.003555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02e530-52e5-4a46-984f-f600f8401b56 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:47:06.003833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02e530-52e5-4a46-984f-f600f8401b56 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:47:06.004101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02e530-52e5-4a46-984f-f600f8401b56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:47:06.012033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f02e530-52e5-4a46-984f-f600f8401b56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:47:06.013604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f02e530-52e5-4a46-984f-f600f8401b56 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:47:06.023258 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 742/1523] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:47:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:47:07.685599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5ab2f-c402-4aa8-b4f4-67d249fd4e06 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:47:07.686252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5ab2f-c402-4aa8-b4f4-67d249fd4e06 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:47:07.686252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5ab2f-c402-4aa8-b4f4-67d249fd4e06 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:47:07.687504 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 782/1524] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 1 20:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:47:07.692559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332db173-01d0-436b-8b37-5b1e03fb56dc None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:47:07.693299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332db173-01d0-436b-8b37-5b1e03fb56dc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:47:07.693427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332db173-01d0-436b-8b37-5b1e03fb56dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:47:07.713407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-332db173-01d0-436b-8b37-5b1e03fb56dc None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:47:07.714623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-332db173-01d0-436b-8b37-5b1e03fb56dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:47:07.722250 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 743/1525] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 1 20:47:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:47:07.881784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5ab2f-c402-4aa8-b4f4-67d249fd4e06 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:47:07.896637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5ab2f-c402-4aa8-b4f4-67d249fd4e06 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:47:07.902091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4991278-3121-448e-833a-067505e079ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:47:07.902718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4991278-3121-448e-833a-067505e079ad service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:47:07.902997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4991278-3121-448e-833a-067505e079ad service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:47:07.903266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4991278-3121-448e-833a-067505e079ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:47:07.911285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4991278-3121-448e-833a-067505e079ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:47:07.912855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4991278-3121-448e-833a-067505e079ad service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:47:07.922057 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 783/1526] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:47:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:47:12.105918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c76c4-8a25-465b-809f-f45bee6b4ddd None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:47:12.106469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c76c4-8a25-465b-809f-f45bee6b4ddd None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:47:12.106469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c76c4-8a25-465b-809f-f45bee6b4ddd None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:47:12.107875 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 744/1527] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 1 20:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:47:12.113281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234b57b-48b7-4cfb-bf08-88eb3b8bfbaf None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:47:12.113683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234b57b-48b7-4cfb-bf08-88eb3b8bfbaf None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:47:12.113832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234b57b-48b7-4cfb-bf08-88eb3b8bfbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:47:12.133327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2234b57b-48b7-4cfb-bf08-88eb3b8bfbaf None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:47:12.134554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2234b57b-48b7-4cfb-bf08-88eb3b8bfbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:47:12.142416 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 784/1528] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 1 20:47:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:47:12.490444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c76c4-8a25-465b-809f-f45bee6b4ddd None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:47:12.492331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ff9c76c4-8a25-465b-809f-f45bee6b4ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:47:12.570860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c76c4-8a25-465b-809f-f45bee6b4ddd None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:47:12.574337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5582e1b-cfff-4d31-aa87-667e36288d09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:47:12.574971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5582e1b-cfff-4d31-aa87-667e36288d09 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:47:12.575247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5582e1b-cfff-4d31-aa87-667e36288d09 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:47:12.575516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5582e1b-cfff-4d31-aa87-667e36288d09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:47:12.584437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5582e1b-cfff-4d31-aa87-667e36288d09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:47:12.585861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5582e1b-cfff-4d31-aa87-667e36288d09 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:47:12.595169 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 745/1529] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:47:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:47:15.011120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3c5d8-5d05-4b6b-b4ab-c566eb96767f None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:47:15.011778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3c5d8-5d05-4b6b-b4ab-c566eb96767f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:47:15.011778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3c5d8-5d05-4b6b-b4ab-c566eb96767f None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:47:15.012997 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 785/1530] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 1 20:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:47:15.018155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba79c28-fd29-47fe-ba1e-12b80daa27aa None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:47:15.018604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba79c28-fd29-47fe-ba1e-12b80daa27aa None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:47:15.018764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba79c28-fd29-47fe-ba1e-12b80daa27aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:47:15.038277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-cba79c28-fd29-47fe-ba1e-12b80daa27aa None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:47:15.039560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cba79c28-fd29-47fe-ba1e-12b80daa27aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:47:15.047338 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 746/1531] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 1 20:47:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:47:15.390408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3c5d8-5d05-4b6b-b4ab-c566eb96767f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:47:15.403508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3c5d8-5d05-4b6b-b4ab-c566eb96767f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:47:15.407046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f658bea-d698-4449-8b66-215b709478bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:47:15.407638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f658bea-d698-4449-8b66-215b709478bb service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:47:15.407928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f658bea-d698-4449-8b66-215b709478bb service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:47:15.408205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f658bea-d698-4449-8b66-215b709478bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:47:15.417237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f658bea-d698-4449-8b66-215b709478bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:47:15.418701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f658bea-d698-4449-8b66-215b709478bb service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:47:15.428070 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 786/1532] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:47:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:47:25.033141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=87 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:47:25.423998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=56 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:04.079651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e6b95-afd6-4613-bee0-44d67c0c2b92 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:04.080211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e6b95-afd6-4613-bee0-44d67c0c2b92 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:04.080379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e6b95-afd6-4613-bee0-44d67c0c2b92 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:04.081796 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 747/1533] 10.222.0.17 () {54 vars in 921 bytes} [Sat Oct 1 20:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:48:04.087093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f2735-06af-451e-ada2-aff713a07a38 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:04.087520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f2735-06af-451e-ada2-aff713a07a38 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:04.087771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f2735-06af-451e-ada2-aff713a07a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:04.108473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4d7f2735-06af-451e-ada2-aff713a07a38 None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:04.109672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4d7f2735-06af-451e-ada2-aff713a07a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:04.118229 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 787/1534] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 20:48:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:14.103208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:37.981431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fb50d-7117-40c6-b556-101847255e37 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:37.982509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fb50d-7117-40c6-b556-101847255e37 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:37.983004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fb50d-7117-40c6-b556-101847255e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.007496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0d2fb50d-7117-40c6-b556-101847255e37 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:38.007689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db870ec7-604f-4818-8fa7-dca8102c63bf None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.008125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db870ec7-604f-4818-8fa7-dca8102c63bf None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.008198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db870ec7-604f-4818-8fa7-dca8102c63bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.008712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0d2fb50d-7117-40c6-b556-101847255e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:38.027102 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 748/1535] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:38.036132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76ebba-5d89-409c-8bf1-c8748771f089 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.036433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76ebba-5d89-409c-8bf1-c8748771f089 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.036692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76ebba-5d89-409c-8bf1-c8748771f089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.042601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-db870ec7-604f-4818-8fa7-dca8102c63bf None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:38.042601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-db870ec7-604f-4818-8fa7-dca8102c63bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:38.077340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fa76ebba-5d89-409c-8bf1-c8748771f089 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:38.079971 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 788/1536] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:38.081543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40ae6d-f000-42d9-9810-82ef479ec561 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.081937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40ae6d-f000-42d9-9810-82ef479ec561 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.082620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40ae6d-f000-42d9-9810-82ef479ec561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.084372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fa76ebba-5d89-409c-8bf1-c8748771f089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:38.102398 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 749/1537] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:38.103875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436718f-157f-467b-b945-f9b4765954a8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.104503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436718f-157f-467b-b945-f9b4765954a8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.104927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436718f-157f-467b-b945-f9b4765954a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.125831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-cc40ae6d-f000-42d9-9810-82ef479ec561 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:38.127160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cc40ae6d-f000-42d9-9810-82ef479ec561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:38.140012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4436718f-157f-467b-b945-f9b4765954a8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:38.141430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4436718f-157f-467b-b945-f9b4765954a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:38.141912 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 789/1538] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:38.146341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40ae6d-f000-42d9-9810-82ef479ec561 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:38.157442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40ae6d-f000-42d9-9810-82ef479ec561 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:38.160019 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 750/1539] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:38.161549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e52a2-3c98-4fce-887e-7b416207dc2c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.162028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e52a2-3c98-4fce-887e-7b416207dc2c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.162477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e52a2-3c98-4fce-887e-7b416207dc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.175213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e1040-e221-48e8-bf77-ea06bd795890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:38.175983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e1040-e221-48e8-bf77-ea06bd795890 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.176272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e1040-e221-48e8-bf77-ea06bd795890 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.176554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e1040-e221-48e8-bf77-ea06bd795890 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.177212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e1040-e221-48e8-bf77-ea06bd795890 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:48:38.177524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e1040-e221-48e8-bf77-ea06bd795890 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:38.179173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e1040-e221-48e8-bf77-ea06bd795890 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:38.185752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-128e52a2-3c98-4fce-887e-7b416207dc2c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:38.187483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-128e52a2-3c98-4fce-887e-7b416207dc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:38.196256 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 751/1540] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:38.197962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87f6fb-5884-46b6-8075-7c216933f10e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.198347 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 790/1541] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:38.202544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677fd27-f8c8-4b24-aa30-f1494403f005 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.202544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677fd27-f8c8-4b24-aa30-f1494403f005 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.202544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677fd27-f8c8-4b24-aa30-f1494403f005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.203140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87f6fb-5884-46b6-8075-7c216933f10e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.203667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87f6fb-5884-46b6-8075-7c216933f10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.234102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2e87f6fb-5884-46b6-8075-7c216933f10e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:38.235714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2e87f6fb-5884-46b6-8075-7c216933f10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:38.235926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c677fd27-f8c8-4b24-aa30-f1494403f005 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:38.237270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c677fd27-f8c8-4b24-aa30-f1494403f005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:38.245372 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 752/1542] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:38.247180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87f6fb-5884-46b6-8075-7c216933f10e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:38.248698 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 791/1543] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:38.249976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f7416-0e03-4b4d-98df-33d2a6baa84c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.250267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f7416-0e03-4b4d-98df-33d2a6baa84c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.250562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f7416-0e03-4b4d-98df-33d2a6baa84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.262980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87f6fb-5884-46b6-8075-7c216933f10e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:38.268612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f42517-ead0-4d02-a85d-968db77fc66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:38.269323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f42517-ead0-4d02-a85d-968db77fc66c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.269767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f42517-ead0-4d02-a85d-968db77fc66c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.269993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f42517-ead0-4d02-a85d-968db77fc66c admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.270679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f42517-ead0-4d02-a85d-968db77fc66c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:48:38.270798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ba8f7416-0e03-4b4d-98df-33d2a6baa84c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:38.271010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f42517-ead0-4d02-a85d-968db77fc66c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:38.271908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ba8f7416-0e03-4b4d-98df-33d2a6baa84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:38.272022 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-02f42517-ead0-4d02-a85d-968db77fc66c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:48:38.272022 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:48:38.272022 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:48:38.272890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f42517-ead0-4d02-a85d-968db77fc66c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:38.280041 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 792/1544] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:38.280899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f7416-0e03-4b4d-98df-33d2a6baa84c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:38.287098 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 753/1545] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:38.288187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f62a44-36b7-4b50-8d4c-0b328d57c403 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.288471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f62a44-36b7-4b50-8d4c-0b328d57c403 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.288761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f62a44-36b7-4b50-8d4c-0b328d57c403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.298423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f7416-0e03-4b4d-98df-33d2a6baa84c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:38.303161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8aef9-2ab9-4eb0-8a1b-a654c0e8e45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:38.303771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8aef9-2ab9-4eb0-8a1b-a654c0e8e45b admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.304050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8aef9-2ab9-4eb0-8a1b-a654c0e8e45b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.304321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8aef9-2ab9-4eb0-8a1b-a654c0e8e45b admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.304945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8aef9-2ab9-4eb0-8a1b-a654c0e8e45b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:38.305947 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-f0a8aef9-2ab9-4eb0-8a1b-a654c0e8e45b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:48:38.305947 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 20:48:38.305947 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:48:38.307172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8aef9-2ab9-4eb0-8a1b-a654c0e8e45b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:38.309207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-65f62a44-36b7-4b50-8d4c-0b328d57c403 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:38.310398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-65f62a44-36b7-4b50-8d4c-0b328d57c403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:38.319859 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 754/1546] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:38.321255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1df917-ba34-4481-bbab-c22da753fe68 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.321538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1df917-ba34-4481-bbab-c22da753fe68 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.321813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1df917-ba34-4481-bbab-c22da753fe68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.350796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1f1df917-ba34-4481-bbab-c22da753fe68 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:38.352018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1f1df917-ba34-4481-bbab-c22da753fe68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:38.357410 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 793/1547] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/projects => generated 379 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:38.358519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab874e-cb47-438f-80bb-a156d8bb914b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.361829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab874e-cb47-438f-80bb-a156d8bb914b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.362115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab874e-cb47-438f-80bb-a156d8bb914b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.365566 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 755/1548] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:38.366283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1df917-ba34-4481-bbab-c22da753fe68 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:38.378879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1df917-ba34-4481-bbab-c22da753fe68 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:38.385405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4eab874e-cb47-438f-80bb-a156d8bb914b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:38.386850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4eab874e-cb47-438f-80bb-a156d8bb914b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:38.436959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0516c0-02fb-4ddd-a483-d687d0bf429a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:38.437557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0516c0-02fb-4ddd-a483-d687d0bf429a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.437836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0516c0-02fb-4ddd-a483-d687d0bf429a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.438110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0516c0-02fb-4ddd-a483-d687d0bf429a admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.438786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0516c0-02fb-4ddd-a483-d687d0bf429a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:48:38.439205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0516c0-02fb-4ddd-a483-d687d0bf429a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:38.442416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0516c0-02fb-4ddd-a483-d687d0bf429a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:38.451006 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 794/1549] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:38.452224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3c8d6-a5be-44a5-9ebc-f816bc128ec3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.452513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3c8d6-a5be-44a5-9ebc-f816bc128ec3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.452788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3c8d6-a5be-44a5-9ebc-f816bc128ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.454184 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 756/1550] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:38.462097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0516c0-02fb-4ddd-a483-d687d0bf429a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:38.472352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0516c0-02fb-4ddd-a483-d687d0bf429a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:38.476171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-09f3c8d6-a5be-44a5-9ebc-f816bc128ec3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:38.477388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-09f3c8d6-a5be-44a5-9ebc-f816bc128ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:38.477512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b418772-7643-46ea-8521-159ec9aacdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:38.478079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b418772-7643-46ea-8521-159ec9aacdbc admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.478356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b418772-7643-46ea-8521-159ec9aacdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.478662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b418772-7643-46ea-8521-159ec9aacdbc admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.479336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b418772-7643-46ea-8521-159ec9aacdbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:48:38.479646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b418772-7643-46ea-8521-159ec9aacdbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:38.481885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b418772-7643-46ea-8521-159ec9aacdbc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:38.493925 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 795/1551] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:38.494554 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 757/1552] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:38.494815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3c8d6-a5be-44a5-9ebc-f816bc128ec3 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:38.495389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4248d-1a09-4743-aa96-733cfd95991f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.495670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4248d-1a09-4743-aa96-733cfd95991f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.495970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4248d-1a09-4743-aa96-733cfd95991f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.512410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3c8d6-a5be-44a5-9ebc-f816bc128ec3 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:38.517151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca00da-5748-4fe0-8555-bd18c60111ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:38.517756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca00da-5748-4fe0-8555-bd18c60111ca admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.518059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca00da-5748-4fe0-8555-bd18c60111ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.518337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca00da-5748-4fe0-8555-bd18c60111ca admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.518999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca00da-5748-4fe0-8555-bd18c60111ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:38.520897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca00da-5748-4fe0-8555-bd18c60111ca admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:38.521839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-65a4248d-1a09-4743-aa96-733cfd95991f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:38.523091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-65a4248d-1a09-4743-aa96-733cfd95991f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:38.531393 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 758/1553] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:38.532198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4248d-1a09-4743-aa96-733cfd95991f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:38.543822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4248d-1a09-4743-aa96-733cfd95991f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:38.548623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216350d4-145f-45d2-b112-fc7b9b5dc9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:38.549226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216350d4-145f-45d2-b112-fc7b9b5dc9ff admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.549614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216350d4-145f-45d2-b112-fc7b9b5dc9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.549885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216350d4-145f-45d2-b112-fc7b9b5dc9ff admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.550595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216350d4-145f-45d2-b112-fc7b9b5dc9ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:38.550898 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 796/1554] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:38.551616 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-216350d4-145f-45d2-b112-fc7b9b5dc9ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:48:38.551616 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:48:38.551616 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:48:38.552170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb046030-09dd-4e26-b52f-730c98e638fe None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.552462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb046030-09dd-4e26-b52f-730c98e638fe None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.552741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb046030-09dd-4e26-b52f-730c98e638fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.552826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216350d4-145f-45d2-b112-fc7b9b5dc9ff admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:38.573885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-eb046030-09dd-4e26-b52f-730c98e638fe None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:38.577456 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 759/1555] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/users => generated 290 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:38.577603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-eb046030-09dd-4e26-b52f-730c98e638fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:38.577700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22000a32-37b0-4e10-9929-17bb4ef676c8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.577700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22000a32-37b0-4e10-9929-17bb4ef676c8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.577700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22000a32-37b0-4e10-9929-17bb4ef676c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.595723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-22000a32-37b0-4e10-9929-17bb4ef676c8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:38.596920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-22000a32-37b0-4e10-9929-17bb4ef676c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:38.636220 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 797/1556] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:38.639574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb046030-09dd-4e26-b52f-730c98e638fe None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:38.644436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:38.646772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:38.651340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb046030-09dd-4e26-b52f-730c98e638fe None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:38.657414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026db72d-fba1-4b56-abd4-e1ba2f7136f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:38.657546 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 760/1557] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:38.658020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026db72d-fba1-4b56-abd4-e1ba2f7136f8 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.658249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22000a32-37b0-4e10-9929-17bb4ef676c8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:38.658342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026db72d-fba1-4b56-abd4-e1ba2f7136f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.658596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026db72d-fba1-4b56-abd4-e1ba2f7136f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.659235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026db72d-fba1-4b56-abd4-e1ba2f7136f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:38.661379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026db72d-fba1-4b56-abd4-e1ba2f7136f8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:38.667384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22000a32-37b0-4e10-9929-17bb4ef676c8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:38.672526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc248f-0885-4171-8605-4fe088529a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:38.673115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc248f-0885-4171-8605-4fe088529a90 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.673395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc248f-0885-4171-8605-4fe088529a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.673964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc248f-0885-4171-8605-4fe088529a90 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.674300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc248f-0885-4171-8605-4fe088529a90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:48:38.674634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc248f-0885-4171-8605-4fe088529a90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:38.676223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc248f-0885-4171-8605-4fe088529a90 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:38.688454 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 761/1558] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:38.693792 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 798/1559] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/projects => generated 401 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:38.693792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026db72d-fba1-4b56-abd4-e1ba2f7136f8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:38.695716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc248f-0885-4171-8605-4fe088529a90 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:38.706349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc248f-0885-4171-8605-4fe088529a90 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:38.752945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026db72d-fba1-4b56-abd4-e1ba2f7136f8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:38.757890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525783d-6386-45fb-8786-d4a275ca6475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:38.758511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525783d-6386-45fb-8786-d4a275ca6475 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.758798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525783d-6386-45fb-8786-d4a275ca6475 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.759072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525783d-6386-45fb-8786-d4a275ca6475 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.759720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525783d-6386-45fb-8786-d4a275ca6475 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:48:38.760025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525783d-6386-45fb-8786-d4a275ca6475 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:38.762905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc2596-9718-4867-9b5b-a05b3ed9a84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:38.763199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525783d-6386-45fb-8786-d4a275ca6475 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:38.763855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc2596-9718-4867-9b5b-a05b3ed9a84e admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.764451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc2596-9718-4867-9b5b-a05b3ed9a84e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.764751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc2596-9718-4867-9b5b-a05b3ed9a84e admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.765434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc2596-9718-4867-9b5b-a05b3ed9a84e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:38.768389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc2596-9718-4867-9b5b-a05b3ed9a84e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:38.777354 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 799/1560] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:38.778607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525783d-6386-45fb-8786-d4a275ca6475 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:38.789468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525783d-6386-45fb-8786-d4a275ca6475 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:38.795057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b60afa-7e3a-4ab3-8bfc-80c7c7967f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:38.795694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b60afa-7e3a-4ab3-8bfc-80c7c7967f46 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.796025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b60afa-7e3a-4ab3-8bfc-80c7c7967f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.796367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b60afa-7e3a-4ab3-8bfc-80c7c7967f46 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.797016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b60afa-7e3a-4ab3-8bfc-80c7c7967f46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:38.799176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b60afa-7e3a-4ab3-8bfc-80c7c7967f46 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:38.809938 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 762/1561] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:38.809938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc2596-9718-4867-9b5b-a05b3ed9a84e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:38.828446 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 800/1562] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/users => generated 287 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:38.829111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b60afa-7e3a-4ab3-8bfc-80c7c7967f46 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:38.839244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b60afa-7e3a-4ab3-8bfc-80c7c7967f46 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:38.875377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc2596-9718-4867-9b5b-a05b3ed9a84e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:38.880258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bd3f0-893e-4b9a-85e2-87d1a5290236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:38.880651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bd3f0-893e-4b9a-85e2-87d1a5290236 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.880931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bd3f0-893e-4b9a-85e2-87d1a5290236 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.881200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bd3f0-893e-4b9a-85e2-87d1a5290236 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.881839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bd3f0-893e-4b9a-85e2-87d1a5290236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:38.883562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bd3f0-893e-4b9a-85e2-87d1a5290236 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:38.893583 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 763/1563] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:38.896723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bd3f0-893e-4b9a-85e2-87d1a5290236 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:38.897535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84680d-58bf-4d02-b312-44f955991dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:38.898359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84680d-58bf-4d02-b312-44f955991dbc admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.898770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84680d-58bf-4d02-b312-44f955991dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.899156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84680d-58bf-4d02-b312-44f955991dbc admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.900020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a84680d-58bf-4d02-b312-44f955991dbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:48:38.900331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a84680d-58bf-4d02-b312-44f955991dbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:38.901932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a84680d-58bf-4d02-b312-44f955991dbc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:38.904269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bd3f0-893e-4b9a-85e2-87d1a5290236 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:38.908775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89512a0e-dc63-425a-82df-37804f3cce7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:38.909367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89512a0e-dc63-425a-82df-37804f3cce7b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.909627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89512a0e-dc63-425a-82df-37804f3cce7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.909896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89512a0e-dc63-425a-82df-37804f3cce7b admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.910541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89512a0e-dc63-425a-82df-37804f3cce7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:48:38.910841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89512a0e-dc63-425a-82df-37804f3cce7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:38.912471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89512a0e-dc63-425a-82df-37804f3cce7b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:38.913548 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 801/1564] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:38.914325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84680d-58bf-4d02-b312-44f955991dbc admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:38.932187 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 764/1565] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:38.932187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89512a0e-dc63-425a-82df-37804f3cce7b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:38.932441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84680d-58bf-4d02-b312-44f955991dbc admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:38.937275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02117f9-b378-4886-ae1b-147477ded4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:38.937883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02117f9-b378-4886-ae1b-147477ded4ce admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.938164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02117f9-b378-4886-ae1b-147477ded4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.938515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02117f9-b378-4886-ae1b-147477ded4ce admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.939122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02117f9-b378-4886-ae1b-147477ded4ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:38.941030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02117f9-b378-4886-ae1b-147477ded4ce admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:38.942774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89512a0e-dc63-425a-82df-37804f3cce7b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:38.947428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441057b-0be6-4efc-8c2e-dc4267fbb45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:38.948012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441057b-0be6-4efc-8c2e-dc4267fbb45c admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.948292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441057b-0be6-4efc-8c2e-dc4267fbb45c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.948565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441057b-0be6-4efc-8c2e-dc4267fbb45c admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.949264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441057b-0be6-4efc-8c2e-dc4267fbb45c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:38.951455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441057b-0be6-4efc-8c2e-dc4267fbb45c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:38.962598 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 802/1566] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/users => generated 301 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:38.963402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02117f9-b378-4886-ae1b-147477ded4ce admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:38.974106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02117f9-b378-4886-ae1b-147477ded4ce admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:38.978848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82553c-5db8-4455-afe9-afb5a26a01b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:38.979431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82553c-5db8-4455-afe9-afb5a26a01b5 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:38.979712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82553c-5db8-4455-afe9-afb5a26a01b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:38.980111 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 765/1567] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/projects => generated 395 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:38.980728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82553c-5db8-4455-afe9-afb5a26a01b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:38.981249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de82553c-5db8-4455-afe9-afb5a26a01b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:38.981526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441057b-0be6-4efc-8c2e-dc4267fbb45c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:38.983197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de82553c-5db8-4455-afe9-afb5a26a01b5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.012489 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 803/1568] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:39.013263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82553c-5db8-4455-afe9-afb5a26a01b5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.017106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:39.027586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:39.052123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441057b-0be6-4efc-8c2e-dc4267fbb45c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.079066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82553c-5db8-4455-afe9-afb5a26a01b5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.083717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba8d16-20e3-40dd-b37d-88a754ff6195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.084322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba8d16-20e3-40dd-b37d-88a754ff6195 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.084601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba8d16-20e3-40dd-b37d-88a754ff6195 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.084963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba8d16-20e3-40dd-b37d-88a754ff6195 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.085608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba8d16-20e3-40dd-b37d-88a754ff6195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.087232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba8d16-20e3-40dd-b37d-88a754ff6195 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.097156 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 804/1569] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:39.098032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba8d16-20e3-40dd-b37d-88a754ff6195 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.108875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba8d16-20e3-40dd-b37d-88a754ff6195 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.111549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d184535-425b-469b-80d9-6bd1119bd453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.112188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d184535-425b-469b-80d9-6bd1119bd453 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.112468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d184535-425b-469b-80d9-6bd1119bd453 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.112739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d184535-425b-469b-80d9-6bd1119bd453 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.113356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d184535-425b-469b-80d9-6bd1119bd453 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.113620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3a754-c6c3-4b97-8227-a761eb039605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.114223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3a754-c6c3-4b97-8227-a761eb039605 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.114542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3a754-c6c3-4b97-8227-a761eb039605 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.114821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3a754-c6c3-4b97-8227-a761eb039605 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.115353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d184535-425b-469b-80d9-6bd1119bd453 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.115447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3a754-c6c3-4b97-8227-a761eb039605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.117024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3a754-c6c3-4b97-8227-a761eb039605 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.131112 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 805/1570] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:39.131112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3a754-c6c3-4b97-8227-a761eb039605 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.139600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3a754-c6c3-4b97-8227-a761eb039605 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.145129 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 766/1571] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:38 2022] POST /identity/v3/users => generated 291 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:39.145129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d184535-425b-469b-80d9-6bd1119bd453 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.145328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1fd3c-8c03-405e-83f8-a8ae26040bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.145328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1fd3c-8c03-405e-83f8-a8ae26040bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.145328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1fd3c-8c03-405e-83f8-a8ae26040bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.145530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1fd3c-8c03-405e-83f8-a8ae26040bb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.146078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1fd3c-8c03-405e-83f8-a8ae26040bb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.147989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1fd3c-8c03-405e-83f8-a8ae26040bb0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.152986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d184535-425b-469b-80d9-6bd1119bd453 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.157535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033d5a1-e315-4667-9506-ae85b95b8956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.158105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033d5a1-e315-4667-9506-ae85b95b8956 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.158382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033d5a1-e315-4667-9506-ae85b95b8956 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.158685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033d5a1-e315-4667-9506-ae85b95b8956 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.159292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033d5a1-e315-4667-9506-ae85b95b8956 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.161165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033d5a1-e315-4667-9506-ae85b95b8956 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.176851 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 806/1572] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:39 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:39.177583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1fd3c-8c03-405e-83f8-a8ae26040bb0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.196445 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 767/1573] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:39 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:39.196445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033d5a1-e315-4667-9506-ae85b95b8956 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.241061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1fd3c-8c03-405e-83f8-a8ae26040bb0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.255658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033d5a1-e315-4667-9506-ae85b95b8956 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.260311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e23887-2be1-43e0-8896-90397a856ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.260912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e23887-2be1-43e0-8896-90397a856ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.261187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e23887-2be1-43e0-8896-90397a856ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.261468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e23887-2be1-43e0-8896-90397a856ea1 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.262083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e23887-2be1-43e0-8896-90397a856ea1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.264109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e23887-2be1-43e0-8896-90397a856ea1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.286187 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 768/1574] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:39 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:39.287400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e23887-2be1-43e0-8896-90397a856ea1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.299712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa4d83a-d2cf-4272-9d30-2da61c9f8881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.300317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa4d83a-d2cf-4272-9d30-2da61c9f8881 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.300495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e23887-2be1-43e0-8896-90397a856ea1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.300596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa4d83a-d2cf-4272-9d30-2da61c9f8881 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.300853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa4d83a-d2cf-4272-9d30-2da61c9f8881 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.301494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa4d83a-d2cf-4272-9d30-2da61c9f8881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.303109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa4d83a-d2cf-4272-9d30-2da61c9f8881 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.305358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e61a4-7e62-4920-bae5-b6d280acd8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.305940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e61a4-7e62-4920-bae5-b6d280acd8e2 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.306219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e61a4-7e62-4920-bae5-b6d280acd8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.306505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e61a4-7e62-4920-bae5-b6d280acd8e2 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.307162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e61a4-7e62-4920-bae5-b6d280acd8e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.309134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e61a4-7e62-4920-bae5-b6d280acd8e2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.315782 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 807/1575] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:39.315782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa4d83a-d2cf-4272-9d30-2da61c9f8881 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.326834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa4d83a-d2cf-4272-9d30-2da61c9f8881 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.331384 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 769/1576] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:39 2022] POST /identity/v3/users => generated 291 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:39.332024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e61a4-7e62-4920-bae5-b6d280acd8e2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.332235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d99ef-0989-4ad9-b77b-3c48791b6858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.332833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d99ef-0989-4ad9-b77b-3c48791b6858 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.333105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d99ef-0989-4ad9-b77b-3c48791b6858 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.333376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d99ef-0989-4ad9-b77b-3c48791b6858 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.334140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d99ef-0989-4ad9-b77b-3c48791b6858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.336322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d99ef-0989-4ad9-b77b-3c48791b6858 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.342353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e61a4-7e62-4920-bae5-b6d280acd8e2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.347038 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 808/1577] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:39.347157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5c7bd-ea7e-419c-b526-d08a1dfdcb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.347578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5c7bd-ea7e-419c-b526-d08a1dfdcb23 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.347807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d99ef-0989-4ad9-b77b-3c48791b6858 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.347897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5c7bd-ea7e-419c-b526-d08a1dfdcb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.348077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5c7bd-ea7e-419c-b526-d08a1dfdcb23 admin admin] PATH_INFO: `/v3/projects/8fb1cc925cc241d291826e2a79e9c2db/users/6ba3f051cd7346779d878f1694e761ea/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.358432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d99ef-0989-4ad9-b77b-3c48791b6858 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.363128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b89a84-6297-4331-b361-9f0ff981dd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.363343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5c7bd-ea7e-419c-b526-d08a1dfdcb23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fb1cc925cc241d291826e2a79e9c2db, user_id=6ba3f051cd7346779d878f1694e761ea, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.363703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b89a84-6297-4331-b361-9f0ff981dd33 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.363989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b89a84-6297-4331-b361-9f0ff981dd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.364252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b89a84-6297-4331-b361-9f0ff981dd33 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.364877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b89a84-6297-4331-b361-9f0ff981dd33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.366014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5c7bd-ea7e-419c-b526-d08a1dfdcb23 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.366421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b89a84-6297-4331-b361-9f0ff981dd33 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.367718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:39.368191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:39.380037 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 809/1578] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:39.380830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b89a84-6297-4331-b361-9f0ff981dd33 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.384763 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 770/1579] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:39 2022] PUT /identity/v3/projects/8fb1cc925cc241d291826e2a79e9c2db/users/6ba3f051cd7346779d878f1694e761ea/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:39.386411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5c7bd-ea7e-419c-b526-d08a1dfdcb23 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.391321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b89a84-6297-4331-b361-9f0ff981dd33 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.449638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604d067-88fd-493c-b51f-a43f2d493cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.450241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604d067-88fd-493c-b51f-a43f2d493cdf admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.450533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5c7bd-ea7e-419c-b526-d08a1dfdcb23 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.450662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604d067-88fd-493c-b51f-a43f2d493cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.450805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604d067-88fd-493c-b51f-a43f2d493cdf admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.451451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604d067-88fd-493c-b51f-a43f2d493cdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.453347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604d067-88fd-493c-b51f-a43f2d493cdf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.455334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f79732-0b1b-4bf0-a947-b4f216dd39e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.455955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f79732-0b1b-4bf0-a947-b4f216dd39e2 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.456268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f79732-0b1b-4bf0-a947-b4f216dd39e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.456540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f79732-0b1b-4bf0-a947-b4f216dd39e2 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.457156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f79732-0b1b-4bf0-a947-b4f216dd39e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.459108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f79732-0b1b-4bf0-a947-b4f216dd39e2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.476351 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 810/1580] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:39 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:39.479864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604d067-88fd-493c-b51f-a43f2d493cdf admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.483960 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 771/1581] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:39 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:39.483960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f79732-0b1b-4bf0-a947-b4f216dd39e2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.487197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604d067-88fd-493c-b51f-a43f2d493cdf admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.492100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586888c-f70f-46e8-9a53-b42c19a1499c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.492622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f79732-0b1b-4bf0-a947-b4f216dd39e2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.492732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586888c-f70f-46e8-9a53-b42c19a1499c admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.492953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586888c-f70f-46e8-9a53-b42c19a1499c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.493228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586888c-f70f-46e8-9a53-b42c19a1499c admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.493858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586888c-f70f-46e8-9a53-b42c19a1499c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.495480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586888c-f70f-46e8-9a53-b42c19a1499c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.497307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33e4a1-3878-4246-a33c-90b487aaa601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.497898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33e4a1-3878-4246-a33c-90b487aaa601 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.498166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33e4a1-3878-4246-a33c-90b487aaa601 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.498442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33e4a1-3878-4246-a33c-90b487aaa601 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.499112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33e4a1-3878-4246-a33c-90b487aaa601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.500692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33e4a1-3878-4246-a33c-90b487aaa601 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.505392 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 811/1582] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:39.506170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586888c-f70f-46e8-9a53-b42c19a1499c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.513487 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 772/1583] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:39.514188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33e4a1-3878-4246-a33c-90b487aaa601 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.515296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586888c-f70f-46e8-9a53-b42c19a1499c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.519798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8e8b1-56d5-477c-95ce-424928ec0a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.520373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8e8b1-56d5-477c-95ce-424928ec0a81 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.520651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8e8b1-56d5-477c-95ce-424928ec0a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.520922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8e8b1-56d5-477c-95ce-424928ec0a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.521563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8e8b1-56d5-477c-95ce-424928ec0a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.523149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8e8b1-56d5-477c-95ce-424928ec0a81 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.523780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33e4a1-3878-4246-a33c-90b487aaa601 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.528890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a085584-fef8-46ad-8fc6-08e657440ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.529516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a085584-fef8-46ad-8fc6-08e657440ae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.529798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a085584-fef8-46ad-8fc6-08e657440ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.530074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a085584-fef8-46ad-8fc6-08e657440ae6 admin admin] PATH_INFO: `/v3/projects/aea1e8351c4343088c92a487903a78b6/users/481eb4e6b7d744d6825977ddc547938a/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.534415 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 812/1584] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:39.535422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8e8b1-56d5-477c-95ce-424928ec0a81 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.545395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a085584-fef8-46ad-8fc6-08e657440ae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aea1e8351c4343088c92a487903a78b6, user_id=481eb4e6b7d744d6825977ddc547938a, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.545657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8e8b1-56d5-477c-95ce-424928ec0a81 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.548084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a085584-fef8-46ad-8fc6-08e657440ae6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.550127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2b09f-94b3-4792-80d3-82b90807fe20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.550736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2b09f-94b3-4792-80d3-82b90807fe20 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.551350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2b09f-94b3-4792-80d3-82b90807fe20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.551350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2b09f-94b3-4792-80d3-82b90807fe20 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.551887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2b09f-94b3-4792-80d3-82b90807fe20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.553707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2b09f-94b3-4792-80d3-82b90807fe20 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.562315 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 773/1585] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:39 2022] PUT /identity/v3/projects/aea1e8351c4343088c92a487903a78b6/users/481eb4e6b7d744d6825977ddc547938a/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:39.563415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992317c-9047-400c-96af-570222c74f57 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.563700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992317c-9047-400c-96af-570222c74f57 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.564101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992317c-9047-400c-96af-570222c74f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.568916 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 813/1586] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:39.568916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2b09f-94b3-4792-80d3-82b90807fe20 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.580792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2b09f-94b3-4792-80d3-82b90807fe20 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.603819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e992317c-9047-400c-96af-570222c74f57 None None] MFA Rules not processed for user `6ba3f051cd7346779d878f1694e761ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:39.605059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e992317c-9047-400c-96af-570222c74f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:39.641700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c537b-02bf-409a-aef0-25e4afb71faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.642309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c537b-02bf-409a-aef0-25e4afb71faa admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.642627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c537b-02bf-409a-aef0-25e4afb71faa admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.642902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c537b-02bf-409a-aef0-25e4afb71faa admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.643536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c537b-02bf-409a-aef0-25e4afb71faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.645110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c537b-02bf-409a-aef0-25e4afb71faa admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.655310 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 814/1587] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:39.656101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c537b-02bf-409a-aef0-25e4afb71faa admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.665582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c537b-02bf-409a-aef0-25e4afb71faa admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.670582 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 774/1588] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:39.670859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b04db-359d-4dc7-bdd8-4f7c2c5c7dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.671436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b04db-359d-4dc7-bdd8-4f7c2c5c7dbd admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.671678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992317c-9047-400c-96af-570222c74f57 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.671788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b04db-359d-4dc7-bdd8-4f7c2c5c7dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.671978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b04db-359d-4dc7-bdd8-4f7c2c5c7dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.672605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b04db-359d-4dc7-bdd8-4f7c2c5c7dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.674163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b04db-359d-4dc7-bdd8-4f7c2c5c7dbd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.684417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992317c-9047-400c-96af-570222c74f57 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.685201 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 815/1589] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:39.685958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b04db-359d-4dc7-bdd8-4f7c2c5c7dbd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.689257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3bcd6-176b-465e-85f7-ccc65dd6256e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.689862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3bcd6-176b-465e-85f7-ccc65dd6256e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.690143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3bcd6-176b-465e-85f7-ccc65dd6256e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.690419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3bcd6-176b-465e-85f7-ccc65dd6256e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.691184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3bcd6-176b-465e-85f7-ccc65dd6256e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.693037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3bcd6-176b-465e-85f7-ccc65dd6256e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.695648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b04db-359d-4dc7-bdd8-4f7c2c5c7dbd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.700334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529e1e0-3ec9-4f96-9eac-bbf0888438d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.700953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529e1e0-3ec9-4f96-9eac-bbf0888438d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.701249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529e1e0-3ec9-4f96-9eac-bbf0888438d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.701507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529e1e0-3ec9-4f96-9eac-bbf0888438d5 admin admin] PATH_INFO: `/v3/projects/6741e5e1d01343dbb0889a1de15d6302/users/f40f50783d9f417a8f84fab11b0d10d0/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.703313 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 775/1590] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:39.704099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3bcd6-176b-465e-85f7-ccc65dd6256e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.713418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3bcd6-176b-465e-85f7-ccc65dd6256e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.718383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594e892-f24b-4068-bf84-a1db7fc99621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.718986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594e892-f24b-4068-bf84-a1db7fc99621 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.719092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b529e1e0-3ec9-4f96-9eac-bbf0888438d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6741e5e1d01343dbb0889a1de15d6302, user_id=f40f50783d9f417a8f84fab11b0d10d0, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.719270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594e892-f24b-4068-bf84-a1db7fc99621 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.719545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594e892-f24b-4068-bf84-a1db7fc99621 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.720162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594e892-f24b-4068-bf84-a1db7fc99621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.720312 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-b529e1e0-3ec9-4f96-9eac-bbf0888438d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:48:39.720312 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 20:48:39.720312 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:48:39.721718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594e892-f24b-4068-bf84-a1db7fc99621 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.722230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b529e1e0-3ec9-4f96-9eac-bbf0888438d5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.734539 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 776/1591] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:39.734539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594e892-f24b-4068-bf84-a1db7fc99621 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.737336 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 816/1592] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:39 2022] PUT /identity/v3/projects/6741e5e1d01343dbb0889a1de15d6302/users/f40f50783d9f417a8f84fab11b0d10d0/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:39.738675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd186c1-1143-4b46-8635-2cbe369939fe None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.738964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd186c1-1143-4b46-8635-2cbe369939fe None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.739236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd186c1-1143-4b46-8635-2cbe369939fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.744818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594e892-f24b-4068-bf84-a1db7fc99621 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.758222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:39.765111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:39.769472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-abd186c1-1143-4b46-8635-2cbe369939fe None None] MFA Rules not processed for user `481eb4e6b7d744d6825977ddc547938a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:39.771852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-abd186c1-1143-4b46-8635-2cbe369939fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:39.807859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e936525-d9f0-466e-bf0d-532e6e47c3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.809432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e936525-d9f0-466e-bf0d-532e6e47c3b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.809856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e936525-d9f0-466e-bf0d-532e6e47c3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.810269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e936525-d9f0-466e-bf0d-532e6e47c3b0 admin admin] PATH_INFO: `/v3/projects/207636159f8a4318b3d141a828b16260/users/07638141e5f34ee783ebfe4f577be214/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.825873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e936525-d9f0-466e-bf0d-532e6e47c3b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=207636159f8a4318b3d141a828b16260, user_id=07638141e5f34ee783ebfe4f577be214, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.828564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e936525-d9f0-466e-bf0d-532e6e47c3b0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.841669 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 777/1593] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:39 2022] PUT /identity/v3/projects/207636159f8a4318b3d141a828b16260/users/07638141e5f34ee783ebfe4f577be214/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:39.842126 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 817/1594] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:39.843031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd186c1-1143-4b46-8635-2cbe369939fe None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.843557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e936525-d9f0-466e-bf0d-532e6e47c3b0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.863058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd186c1-1143-4b46-8635-2cbe369939fe None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.907187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e936525-d9f0-466e-bf0d-532e6e47c3b0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.911923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383cd1b0-1a20-4597-99c4-64560e00f5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.912542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cd1b0-1a20-4597-99c4-64560e00f5c4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.912836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cd1b0-1a20-4597-99c4-64560e00f5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.913092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cd1b0-1a20-4597-99c4-64560e00f5c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.913723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383cd1b0-1a20-4597-99c4-64560e00f5c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.915375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383cd1b0-1a20-4597-99c4-64560e00f5c4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.919914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce22b9-d907-4a2e-904f-eb6a02cfa4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.920527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce22b9-d907-4a2e-904f-eb6a02cfa4fd admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.920814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce22b9-d907-4a2e-904f-eb6a02cfa4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.921263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce22b9-d907-4a2e-904f-eb6a02cfa4fd admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.921900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce22b9-d907-4a2e-904f-eb6a02cfa4fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.923831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce22b9-d907-4a2e-904f-eb6a02cfa4fd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.925468 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 778/1595] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:39.926256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383cd1b0-1a20-4597-99c4-64560e00f5c4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.936032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383cd1b0-1a20-4597-99c4-64560e00f5c4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:39.940939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893b5e3-4cd8-426d-85d6-abfac3f63d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:39.941651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893b5e3-4cd8-426d-85d6-abfac3f63d1e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:39.941835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893b5e3-4cd8-426d-85d6-abfac3f63d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:39.942387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893b5e3-4cd8-426d-85d6-abfac3f63d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:39.943042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893b5e3-4cd8-426d-85d6-abfac3f63d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:39.944603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893b5e3-4cd8-426d-85d6-abfac3f63d1e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:39.955685 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 779/1596] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:39.955956 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 818/1597] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:39 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:39.956940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce22b9-d907-4a2e-904f-eb6a02cfa4fd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.959391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893b5e3-4cd8-426d-85d6-abfac3f63d1e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:39.968616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893b5e3-4cd8-426d-85d6-abfac3f63d1e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:40.020662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce22b9-d907-4a2e-904f-eb6a02cfa4fd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:40.025536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cfd4ae-5e7f-434d-9c9d-5fb39ef9100e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:40.026190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfd4ae-5e7f-434d-9c9d-5fb39ef9100e admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:40.026468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfd4ae-5e7f-434d-9c9d-5fb39ef9100e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:40.026767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfd4ae-5e7f-434d-9c9d-5fb39ef9100e admin admin] PATH_INFO: `/v3/projects/b6ead845a07d465f92f43f1d284658cf/users/26c35587c0e940bcb7f6e3e88a52aa47/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:40.031141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f339fce-8870-4bad-a8be-2cb694c64075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:40.031808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f339fce-8870-4bad-a8be-2cb694c64075 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:40.032095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f339fce-8870-4bad-a8be-2cb694c64075 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:40.032372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f339fce-8870-4bad-a8be-2cb694c64075 admin admin] PATH_INFO: `/v3/projects/2b38d7f557db4bfeacdd4e958cec9557/users/d658c70488c74cedbee2bafdde3fe047/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:40.042532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cfd4ae-5e7f-434d-9c9d-5fb39ef9100e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6ead845a07d465f92f43f1d284658cf, user_id=26c35587c0e940bcb7f6e3e88a52aa47, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:40.043893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cfd4ae-5e7f-434d-9c9d-5fb39ef9100e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:40.047653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f339fce-8870-4bad-a8be-2cb694c64075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b38d7f557db4bfeacdd4e958cec9557, user_id=d658c70488c74cedbee2bafdde3fe047, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:40.051117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f339fce-8870-4bad-a8be-2cb694c64075 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:40.057127 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 819/1598] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:39 2022] PUT /identity/v3/projects/b6ead845a07d465f92f43f1d284658cf/users/26c35587c0e940bcb7f6e3e88a52aa47/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:40.058974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043a9f7-6563-4124-91cf-09d5b243d60b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:40.059331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043a9f7-6563-4124-91cf-09d5b243d60b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:40.059675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043a9f7-6563-4124-91cf-09d5b243d60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:40.076998 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 780/1599] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:39 2022] PUT /identity/v3/projects/2b38d7f557db4bfeacdd4e958cec9557/users/d658c70488c74cedbee2bafdde3fe047/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:40.079316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8043a9f7-6563-4124-91cf-09d5b243d60b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:40.079436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f339fce-8870-4bad-a8be-2cb694c64075 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:40.080520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8043a9f7-6563-4124-91cf-09d5b243d60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:40.087266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:40.089215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:40.144824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f339fce-8870-4bad-a8be-2cb694c64075 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:40.145365 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 820/1600] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:40.146823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf97c4-7b0e-49d8-8198-a9faa8af0cdd None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:40.147114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf97c4-7b0e-49d8-8198-a9faa8af0cdd None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:40.147392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf97c4-7b0e-49d8-8198-a9faa8af0cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:40.152859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f6b9b-53b1-4822-bc05-638cfd91e641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:40.153476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f6b9b-53b1-4822-bc05-638cfd91e641 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:40.153877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f6b9b-53b1-4822-bc05-638cfd91e641 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:40.154156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f6b9b-53b1-4822-bc05-638cfd91e641 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:40.154845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f6b9b-53b1-4822-bc05-638cfd91e641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:40.156804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f6b9b-53b1-4822-bc05-638cfd91e641 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:40.176774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e8cf97c4-7b0e-49d8-8198-a9faa8af0cdd None None] MFA Rules not processed for user `07638141e5f34ee783ebfe4f577be214`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:40.178013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e8cf97c4-7b0e-49d8-8198-a9faa8af0cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:40.184167 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 781/1601] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:40 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:40.184962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f6b9b-53b1-4822-bc05-638cfd91e641 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:40.243966 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 821/1602] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:40.244801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf97c4-7b0e-49d8-8198-a9faa8af0cdd None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:40.250824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f6b9b-53b1-4822-bc05-638cfd91e641 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:40.255175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf97c4-7b0e-49d8-8198-a9faa8af0cdd None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:40.255558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c74997-9640-4962-b8a8-04fa97f49db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:40.256241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c74997-9640-4962-b8a8-04fa97f49db5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:40.256549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c74997-9640-4962-b8a8-04fa97f49db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:40.256910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c74997-9640-4962-b8a8-04fa97f49db5 admin admin] PATH_INFO: `/v3/projects/646008f76eaa4a6398c1a63f6ee13bd1/users/610291b77fa747e9a04aa1abf6afb9c7/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:40.259684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907f863-4e24-4bba-a198-124d063aa343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:40.260261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907f863-4e24-4bba-a198-124d063aa343 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:40.260536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907f863-4e24-4bba-a198-124d063aa343 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:40.260805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907f863-4e24-4bba-a198-124d063aa343 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:40.262026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907f863-4e24-4bba-a198-124d063aa343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:40.263954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907f863-4e24-4bba-a198-124d063aa343 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:40.273050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c74997-9640-4962-b8a8-04fa97f49db5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=646008f76eaa4a6398c1a63f6ee13bd1, user_id=610291b77fa747e9a04aa1abf6afb9c7, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:40.276178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c74997-9640-4962-b8a8-04fa97f49db5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:40.287689 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 822/1603] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:40 2022] POST /identity/v3/users => generated 296 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:40.288766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907f863-4e24-4bba-a198-124d063aa343 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:40.289999 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 782/1604] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:40 2022] PUT /identity/v3/projects/646008f76eaa4a6398c1a63f6ee13bd1/users/610291b77fa747e9a04aa1abf6afb9c7/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:40.291565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aac07a-4595-4bb3-b619-eaa8f79ed250 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:40.291854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aac07a-4595-4bb3-b619-eaa8f79ed250 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:40.292146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aac07a-4595-4bb3-b619-eaa8f79ed250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:40.300456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907f863-4e24-4bba-a198-124d063aa343 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:40.324280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-68aac07a-4595-4bb3-b619-eaa8f79ed250 None None] MFA Rules not processed for user `26c35587c0e940bcb7f6e3e88a52aa47`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:40.325571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-68aac07a-4595-4bb3-b619-eaa8f79ed250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:40.359309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2251b280-3599-4375-8a6a-f7bd41684398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:40.359974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251b280-3599-4375-8a6a-f7bd41684398 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:40.360257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251b280-3599-4375-8a6a-f7bd41684398 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:40.360540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251b280-3599-4375-8a6a-f7bd41684398 admin admin] PATH_INFO: `/v3/projects/c9b87cefcd1d49f48a7f3167f14af7b4/users/a8c7e3fb07754e46b7ea90a7f4508404/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:40.375622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2251b280-3599-4375-8a6a-f7bd41684398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b87cefcd1d49f48a7f3167f14af7b4, user_id=a8c7e3fb07754e46b7ea90a7f4508404, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:40.378318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2251b280-3599-4375-8a6a-f7bd41684398 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:40.392009 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 823/1605] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:40 2022] PUT /identity/v3/projects/c9b87cefcd1d49f48a7f3167f14af7b4/users/a8c7e3fb07754e46b7ea90a7f4508404/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:40.392502 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 783/1606] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:40.393341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aac07a-4595-4bb3-b619-eaa8f79ed250 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:40.394143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d16f7-792f-4fb5-8be1-18351447287c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:40.394430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d16f7-792f-4fb5-8be1-18351447287c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:40.394744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d16f7-792f-4fb5-8be1-18351447287c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:40.404438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-68aac07a-4595-4bb3-b619-eaa8f79ed250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:40.409290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:40.412390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:40.425923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-287d16f7-792f-4fb5-8be1-18351447287c None None] MFA Rules not processed for user `d658c70488c74cedbee2bafdde3fe047`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:40.427191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-287d16f7-792f-4fb5-8be1-18351447287c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:40.470078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aac07a-4595-4bb3-b619-eaa8f79ed250 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:40.493958 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 824/1607] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:40 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:40.494997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d16f7-792f-4fb5-8be1-18351447287c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:40.528942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1319c4-45e5-45fa-8a82-04b4d3aca020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:40.529468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1319c4-45e5-45fa-8a82-04b4d3aca020 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:40.529749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1319c4-45e5-45fa-8a82-04b4d3aca020 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:40.530024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1319c4-45e5-45fa-8a82-04b4d3aca020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:40.560243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d16f7-792f-4fb5-8be1-18351447287c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:40.565366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0b035-adc6-4f59-8283-86708dfbe23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:40.566152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0b035-adc6-4f59-8283-86708dfbe23a admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:40.566442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0b035-adc6-4f59-8283-86708dfbe23a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:40.566759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0b035-adc6-4f59-8283-86708dfbe23a admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:40.568108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0b035-adc6-4f59-8283-86708dfbe23a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:40.570033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0b035-adc6-4f59-8283-86708dfbe23a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:40.591578 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 825/1608] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:40 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:40.595413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0b035-adc6-4f59-8283-86708dfbe23a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:40.598472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1319c4-45e5-45fa-8a82-04b4d3aca020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:40.599933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1319c4-45e5-45fa-8a82-04b4d3aca020 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:40.604767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0b035-adc6-4f59-8283-86708dfbe23a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:40.609908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd9a05-b88e-4a2a-90e0-4a8dcd86b420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:40.610273 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 784/1609] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:40.610847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd9a05-b88e-4a2a-90e0-4a8dcd86b420 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:40.611121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1319c4-45e5-45fa-8a82-04b4d3aca020 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:40.611232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd9a05-b88e-4a2a-90e0-4a8dcd86b420 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:40.611392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd9a05-b88e-4a2a-90e0-4a8dcd86b420 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:40.612146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bd9a05-b88e-4a2a-90e0-4a8dcd86b420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:40.614033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bd9a05-b88e-4a2a-90e0-4a8dcd86b420 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:40.620539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1319c4-45e5-45fa-8a82-04b4d3aca020 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:40.626694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666f28d-221e-40b8-9d3f-37f3f96f781c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:40.627307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666f28d-221e-40b8-9d3f-37f3f96f781c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:40.627757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666f28d-221e-40b8-9d3f-37f3f96f781c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:40.628038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666f28d-221e-40b8-9d3f-37f3f96f781c admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:40.628681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666f28d-221e-40b8-9d3f-37f3f96f781c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:40.630288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666f28d-221e-40b8-9d3f-37f3f96f781c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:40.641907 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 785/1610] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:40.643380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c07e02-5474-4138-8838-cc88f5d9f12b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:40.643669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c07e02-5474-4138-8838-cc88f5d9f12b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:40.643945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c07e02-5474-4138-8838-cc88f5d9f12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:40.647726 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 826/1611] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:40 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:40.649129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2968f8-fa3e-4977-831b-74098615354f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:40.649418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2968f8-fa3e-4977-831b-74098615354f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:40.649753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2968f8-fa3e-4977-831b-74098615354f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:40.676970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e4c07e02-5474-4138-8838-cc88f5d9f12b None None] MFA Rules not processed for user `610291b77fa747e9a04aa1abf6afb9c7`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:40.678358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e4c07e02-5474-4138-8838-cc88f5d9f12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:40.680960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6c2968f8-fa3e-4977-831b-74098615354f None None] MFA Rules not processed for user `a8c7e3fb07754e46b7ea90a7f4508404`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:40.682169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6c2968f8-fa3e-4977-831b-74098615354f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:40.735123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:40.752926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:40.756131 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 786/1612] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:40.757130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c07e02-5474-4138-8838-cc88f5d9f12b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:40.766553 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 827/1613] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:40.767263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2968f8-fa3e-4977-831b-74098615354f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:40.844258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c07e02-5474-4138-8838-cc88f5d9f12b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:40.848255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b900d-65e1-4483-bb05-2e852e0bfea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:40.848860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b900d-65e1-4483-bb05-2e852e0bfea9 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:40.849137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b900d-65e1-4483-bb05-2e852e0bfea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:40.849905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b900d-65e1-4483-bb05-2e852e0bfea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:40.854970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2968f8-fa3e-4977-831b-74098615354f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:40.862344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c04bc5-0fb3-4344-a32c-4db7f86ecf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:40.863110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c04bc5-0fb3-4344-a32c-4db7f86ecf23 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:40.863392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c04bc5-0fb3-4344-a32c-4db7f86ecf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:40.863664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c04bc5-0fb3-4344-a32c-4db7f86ecf23 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:40.864296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c04bc5-0fb3-4344-a32c-4db7f86ecf23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:40.872892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c04bc5-0fb3-4344-a32c-4db7f86ecf23 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:40.907120 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 828/1614] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:40 2022] POST /identity/v3/projects => generated 395 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:40.908465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c04bc5-0fb3-4344-a32c-4db7f86ecf23 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:40.923908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b900d-65e1-4483-bb05-2e852e0bfea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:40.925552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b900d-65e1-4483-bb05-2e852e0bfea9 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:40.934791 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 787/1615] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:40.935714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b900d-65e1-4483-bb05-2e852e0bfea9 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:40.947785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b900d-65e1-4483-bb05-2e852e0bfea9 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:41.001480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c04bc5-0fb3-4344-a32c-4db7f86ecf23 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:41.007878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0cd90-09c2-4dff-9c51-f27ffb706bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:41.008633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0cd90-09c2-4dff-9c51-f27ffb706bcd admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:41.008917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0cd90-09c2-4dff-9c51-f27ffb706bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:41.009190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0cd90-09c2-4dff-9c51-f27ffb706bcd admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:41.009845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0cd90-09c2-4dff-9c51-f27ffb706bcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:41.011528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0cd90-09c2-4dff-9c51-f27ffb706bcd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:41.023729 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 829/1616] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:41.024560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0cd90-09c2-4dff-9c51-f27ffb706bcd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:41.053773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0cd90-09c2-4dff-9c51-f27ffb706bcd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:41.060136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd054b9-3edb-4933-a4a9-53a2be4a070f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:41.060840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd054b9-3edb-4933-a4a9-53a2be4a070f admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:41.061024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd054b9-3edb-4933-a4a9-53a2be4a070f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:41.061302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd054b9-3edb-4933-a4a9-53a2be4a070f admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:41.061941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd054b9-3edb-4933-a4a9-53a2be4a070f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:41.064260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd054b9-3edb-4933-a4a9-53a2be4a070f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:41.068101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef4d8a-867f-498f-abfa-7267aeec397f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:41.068777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef4d8a-867f-498f-abfa-7267aeec397f admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:41.069062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef4d8a-867f-498f-abfa-7267aeec397f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:41.070658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef4d8a-867f-498f-abfa-7267aeec397f admin admin] PATH_INFO: `/v3/projects/c8eaa857c05240d394707c400cf87116/users/7f57761e58f444548d909bc8784b80d5/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:41.097733 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 830/1617] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:41 2022] POST /identity/v3/users => generated 298 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:41.097959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef4d8a-867f-498f-abfa-7267aeec397f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8eaa857c05240d394707c400cf87116, user_id=7f57761e58f444548d909bc8784b80d5, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:41.098558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd054b9-3edb-4933-a4a9-53a2be4a070f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:41.101751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef4d8a-867f-498f-abfa-7267aeec397f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:41.114513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd054b9-3edb-4933-a4a9-53a2be4a070f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:41.117236 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 788/1618] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:40 2022] PUT /identity/v3/projects/c8eaa857c05240d394707c400cf87116/users/7f57761e58f444548d909bc8784b80d5/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:41.118374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef4d8a-867f-498f-abfa-7267aeec397f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:41.155666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:41.189683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:41.197054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef4d8a-867f-498f-abfa-7267aeec397f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:41.200101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a722f1c-b12c-43b1-a78a-3e7bfd3cd1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:41.200806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a722f1c-b12c-43b1-a78a-3e7bfd3cd1c4 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:41.201551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a722f1c-b12c-43b1-a78a-3e7bfd3cd1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:41.202385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a722f1c-b12c-43b1-a78a-3e7bfd3cd1c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:41.203314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a722f1c-b12c-43b1-a78a-3e7bfd3cd1c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:41.205425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaabfbf-f133-4d86-ab66-acd3fb3f3b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:41.205580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a722f1c-b12c-43b1-a78a-3e7bfd3cd1c4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:41.205928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaabfbf-f133-4d86-ab66-acd3fb3f3b54 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:41.206270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaabfbf-f133-4d86-ab66-acd3fb3f3b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:41.206644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaabfbf-f133-4d86-ab66-acd3fb3f3b54 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:41.207363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaabfbf-f133-4d86-ab66-acd3fb3f3b54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:41.209826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaabfbf-f133-4d86-ab66-acd3fb3f3b54 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:41.244019 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 789/1619] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:41 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:41.244954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaabfbf-f133-4d86-ab66-acd3fb3f3b54 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:41.248318 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 831/1620] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:41 2022] POST /identity/v3/projects => generated 393 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:41.249581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a722f1c-b12c-43b1-a78a-3e7bfd3cd1c4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:41.323186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a722f1c-b12c-43b1-a78a-3e7bfd3cd1c4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:41.325274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaabfbf-f133-4d86-ab66-acd3fb3f3b54 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:41.329247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc0c75e-4d7b-48a7-a18d-5f39cc34cd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:41.329939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0c75e-4d7b-48a7-a18d-5f39cc34cd34 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:41.330223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0c75e-4d7b-48a7-a18d-5f39cc34cd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:41.330327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908d5da-915f-47d4-9c50-99f50b7b07bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:41.331085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908d5da-915f-47d4-9c50-99f50b7b07bd admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:41.331428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908d5da-915f-47d4-9c50-99f50b7b07bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:41.332028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908d5da-915f-47d4-9c50-99f50b7b07bd admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:41.332139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0c75e-4d7b-48a7-a18d-5f39cc34cd34 admin admin] PATH_INFO: `/v3/projects/f063d1149d2f49d488112e2765440f3b/users/c7aab644bc2d4a9089d9ad72eff3e187/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:41.332767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908d5da-915f-47d4-9c50-99f50b7b07bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:41.335061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908d5da-915f-47d4-9c50-99f50b7b07bd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:41.354161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc0c75e-4d7b-48a7-a18d-5f39cc34cd34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f063d1149d2f49d488112e2765440f3b, user_id=c7aab644bc2d4a9089d9ad72eff3e187, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:41.357426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc0c75e-4d7b-48a7-a18d-5f39cc34cd34 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:41.359208 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 790/1621] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:41 2022] POST /identity/v3/users => generated 304 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:41.360034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908d5da-915f-47d4-9c50-99f50b7b07bd admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:41.370859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908d5da-915f-47d4-9c50-99f50b7b07bd admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:41.378569 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 832/1622] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:41 2022] PUT /identity/v3/projects/f063d1149d2f49d488112e2765440f3b/users/c7aab644bc2d4a9089d9ad72eff3e187/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:41.378569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc0c75e-4d7b-48a7-a18d-5f39cc34cd34 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:41.446355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fad4c5-6e13-4bca-bedc-d5ce4566b595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:41.447015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fad4c5-6e13-4bca-bedc-d5ce4566b595 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:41.447297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fad4c5-6e13-4bca-bedc-d5ce4566b595 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:41.447632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fad4c5-6e13-4bca-bedc-d5ce4566b595 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:41.448291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fad4c5-6e13-4bca-bedc-d5ce4566b595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:41.450404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fad4c5-6e13-4bca-bedc-d5ce4566b595 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:41.456253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc0c75e-4d7b-48a7-a18d-5f39cc34cd34 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:41.461312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a5827-0569-4164-8ec6-c2cd0be4e178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:41.462155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a5827-0569-4164-8ec6-c2cd0be4e178 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:41.462334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a5827-0569-4164-8ec6-c2cd0be4e178 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:41.462667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a5827-0569-4164-8ec6-c2cd0be4e178 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:41.463347 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 791/1623] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:41.463451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a5827-0569-4164-8ec6-c2cd0be4e178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:41.465088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fad4c5-6e13-4bca-bedc-d5ce4566b595 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:41.465699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a5827-0569-4164-8ec6-c2cd0be4e178 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:41.481242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fad4c5-6e13-4bca-bedc-d5ce4566b595 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:41.485601 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 833/1624] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:41.486680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a5827-0569-4164-8ec6-c2cd0be4e178 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:41.486783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce04eb-3678-44ef-a236-d417b2ccc7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:41.488076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce04eb-3678-44ef-a236-d417b2ccc7e7 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:41.488327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce04eb-3678-44ef-a236-d417b2ccc7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:41.488723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce04eb-3678-44ef-a236-d417b2ccc7e7 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:41.489405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce04eb-3678-44ef-a236-d417b2ccc7e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:41.492180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce04eb-3678-44ef-a236-d417b2ccc7e7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:41.501206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a5827-0569-4164-8ec6-c2cd0be4e178 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:41.507749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76b94d-0a97-4763-bf3b-24eb120448cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:41.508461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76b94d-0a97-4763-bf3b-24eb120448cf admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:41.514700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76b94d-0a97-4763-bf3b-24eb120448cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:41.515177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76b94d-0a97-4763-bf3b-24eb120448cf admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:41.515865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76b94d-0a97-4763-bf3b-24eb120448cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:41.518532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76b94d-0a97-4763-bf3b-24eb120448cf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:41.524380 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 792/1625] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:41 2022] POST /identity/v3/users => generated 293 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:41.525268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce04eb-3678-44ef-a236-d417b2ccc7e7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:41.537510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce04eb-3678-44ef-a236-d417b2ccc7e7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:41.545863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b67ff4-c724-4f10-8507-34cf23a5d4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:41.546761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b67ff4-c724-4f10-8507-34cf23a5d4f4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:41.547186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b67ff4-c724-4f10-8507-34cf23a5d4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:41.547472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b67ff4-c724-4f10-8507-34cf23a5d4f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:41.548159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b67ff4-c724-4f10-8507-34cf23a5d4f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:41.551255 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 834/1626] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:41 2022] POST /identity/v3/users => generated 303 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:41.551379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b67ff4-c724-4f10-8507-34cf23a5d4f4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:41.551453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76b94d-0a97-4763-bf3b-24eb120448cf admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:41.561268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76b94d-0a97-4763-bf3b-24eb120448cf admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:41.561672 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 793/1627] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:41.564032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b67ff4-c724-4f10-8507-34cf23a5d4f4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:41.566897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef5d7b-d97c-415c-8ef3-86fda5bc4969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:41.567971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef5d7b-d97c-415c-8ef3-86fda5bc4969 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:41.568373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef5d7b-d97c-415c-8ef3-86fda5bc4969 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:41.568672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef5d7b-d97c-415c-8ef3-86fda5bc4969 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:41.569432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef5d7b-d97c-415c-8ef3-86fda5bc4969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:41.571333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef5d7b-d97c-415c-8ef3-86fda5bc4969 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:41.575550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b67ff4-c724-4f10-8507-34cf23a5d4f4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:41.584004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20646a81-dee5-4c53-b054-fd3abcfec20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:41.584648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20646a81-dee5-4c53-b054-fd3abcfec20f admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:41.585101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20646a81-dee5-4c53-b054-fd3abcfec20f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:41.585518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20646a81-dee5-4c53-b054-fd3abcfec20f admin admin] PATH_INFO: `/v3/projects/215471dd429642b3974741aade3aaba0/users/7408e981e5404f7faeb1b018e345f741/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:41.586579 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 835/1628] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:41.587597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef5d7b-d97c-415c-8ef3-86fda5bc4969 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:41.595887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:41.599600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef5d7b-d97c-415c-8ef3-86fda5bc4969 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:41.604844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db59f7-3200-4de6-b70f-1e71cfbea03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:41.606102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db59f7-3200-4de6-b70f-1e71cfbea03d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:41.606524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db59f7-3200-4de6-b70f-1e71cfbea03d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:41.606860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db59f7-3200-4de6-b70f-1e71cfbea03d admin admin] PATH_INFO: `/v3/users/26c35587c0e940bcb7f6e3e88a52aa47` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:41.608508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db59f7-3200-4de6-b70f-1e71cfbea03d admin admin] RBAC: Authorizing `identity:delete_user(user_id=26c35587c0e940bcb7f6e3e88a52aa47)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:41.610236 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-83db59f7-3200-4de6-b70f-1e71cfbea03d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:48:41.610236 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 20:48:41.610236 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:48:41.611870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db59f7-3200-4de6-b70f-1e71cfbea03d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:41.622884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20646a81-dee5-4c53-b054-fd3abcfec20f admin admin] RBAC: Authorizing `identity:create_grant(project_id=215471dd429642b3974741aade3aaba0, user_id=7408e981e5404f7faeb1b018e345f741, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:41.628487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20646a81-dee5-4c53-b054-fd3abcfec20f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:41.647559 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 794/1629] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:41 2022] PUT /identity/v3/projects/215471dd429642b3974741aade3aaba0/users/7408e981e5404f7faeb1b018e345f741/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:41.651931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20646a81-dee5-4c53-b054-fd3abcfec20f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:41.661374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:41.693306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-83db59f7-3200-4de6-b70f-1e71cfbea03d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26c35587c0e940bcb7f6e3e88a52aa47'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:41.725617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-83db59f7-3200-4de6-b70f-1e71cfbea03d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26c35587c0e940bcb7f6e3e88a52aa47'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:41.753121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-83db59f7-3200-4de6-b70f-1e71cfbea03d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26c35587c0e940bcb7f6e3e88a52aa47'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:41.754483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-83db59f7-3200-4de6-b70f-1e71cfbea03d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26c35587c0e940bcb7f6e3e88a52aa47'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:41.756372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20646a81-dee5-4c53-b054-fd3abcfec20f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:41.762322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d1bd8-cab3-4549-92bd-2e3cd37bdaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:41.763411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d1bd8-cab3-4549-92bd-2e3cd37bdaf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:41.763754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d1bd8-cab3-4549-92bd-2e3cd37bdaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:41.764243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d1bd8-cab3-4549-92bd-2e3cd37bdaf7 admin admin] PATH_INFO: `/v3/projects/c8eaa857c05240d394707c400cf87116/users/7f57761e58f444548d909bc8784b80d5/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:41.766168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-83db59f7-3200-4de6-b70f-1e71cfbea03d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26c35587c0e940bcb7f6e3e88a52aa47'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:41.766725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d1bd8-cab3-4549-92bd-2e3cd37bdaf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8eaa857c05240d394707c400cf87116, user_id=7f57761e58f444548d909bc8784b80d5, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:41.770655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d1bd8-cab3-4549-92bd-2e3cd37bdaf7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:41.775726 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 836/1630] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:48:41 2022] DELETE /identity/v3/users/26c35587c0e940bcb7f6e3e88a52aa47 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:41.777462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db59f7-3200-4de6-b70f-1e71cfbea03d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:41.780381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-83db59f7-3200-4de6-b70f-1e71cfbea03d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:41.796978 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 795/1631] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:41 2022] PUT /identity/v3/projects/c8eaa857c05240d394707c400cf87116/users/7f57761e58f444548d909bc8784b80d5/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:41.796978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d1bd8-cab3-4549-92bd-2e3cd37bdaf7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:41.801126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7b3d1bd8-cab3-4549-92bd-2e3cd37bdaf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:41.882142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db59f7-3200-4de6-b70f-1e71cfbea03d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:41.927720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d1bd8-cab3-4549-92bd-2e3cd37bdaf7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:41.942332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75edb4-c7b1-4dd8-973d-287cd39e45e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:41.943162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75edb4-c7b1-4dd8-973d-287cd39e45e6 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:41.943445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75edb4-c7b1-4dd8-973d-287cd39e45e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:41.943715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75edb4-c7b1-4dd8-973d-287cd39e45e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:41.944514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75edb4-c7b1-4dd8-973d-287cd39e45e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:41.946249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75edb4-c7b1-4dd8-973d-287cd39e45e6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:41.959011 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 796/1632] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:41.959857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75edb4-c7b1-4dd8-973d-287cd39e45e6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:41.961738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fd75edb4-c7b1-4dd8-973d-287cd39e45e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:41.970126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df4468-17e3-4a0b-a774-aed299062e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:41.971405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df4468-17e3-4a0b-a774-aed299062e48 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:41.972176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df4468-17e3-4a0b-a774-aed299062e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:41.972671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df4468-17e3-4a0b-a774-aed299062e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:41.973463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df4468-17e3-4a0b-a774-aed299062e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:41.975328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75edb4-c7b1-4dd8-973d-287cd39e45e6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:41.975739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df4468-17e3-4a0b-a774-aed299062e48 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:41.981051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b70db5-c3c3-4b4a-b471-72dd6b4576dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:41.981698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b70db5-c3c3-4b4a-b471-72dd6b4576dc admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:41.982057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b70db5-c3c3-4b4a-b471-72dd6b4576dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:41.983047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b70db5-c3c3-4b4a-b471-72dd6b4576dc admin admin] PATH_INFO: `/v3/projects/08b6602ad13646c195ab1f3be7aa0e72/users/3e0503d20ac74f9ebe516009ddf320a5/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:41.997418 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 837/1633] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:41.998573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df4468-17e3-4a0b-a774-aed299062e48 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:42.001420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-93df4468-17e3-4a0b-a774-aed299062e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:42.009694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b70db5-c3c3-4b4a-b471-72dd6b4576dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=08b6602ad13646c195ab1f3be7aa0e72, user_id=3e0503d20ac74f9ebe516009ddf320a5, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:42.013000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b70db5-c3c3-4b4a-b471-72dd6b4576dc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:42.016934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df4468-17e3-4a0b-a774-aed299062e48 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:42.028698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2008405-d47b-4458-b1f2-6127f27ce44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:42.028698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2008405-d47b-4458-b1f2-6127f27ce44c admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:42.028698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2008405-d47b-4458-b1f2-6127f27ce44c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:42.028698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2008405-d47b-4458-b1f2-6127f27ce44c admin admin] PATH_INFO: `/v3/projects/f063d1149d2f49d488112e2765440f3b/users/c7aab644bc2d4a9089d9ad72eff3e187/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:42.032102 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 797/1634] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:41 2022] PUT /identity/v3/projects/08b6602ad13646c195ab1f3be7aa0e72/users/3e0503d20ac74f9ebe516009ddf320a5/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:42.033220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b70db5-c3c3-4b4a-b471-72dd6b4576dc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:42.035319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2008405-d47b-4458-b1f2-6127f27ce44c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f063d1149d2f49d488112e2765440f3b, user_id=c7aab644bc2d4a9089d9ad72eff3e187, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:42.037304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a5b70db5-c3c3-4b4a-b471-72dd6b4576dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:42.038157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2008405-d47b-4458-b1f2-6127f27ce44c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:42.056899 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 838/1635] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:41 2022] PUT /identity/v3/projects/f063d1149d2f49d488112e2765440f3b/users/c7aab644bc2d4a9089d9ad72eff3e187/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:42.058892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b5ef4-4157-46fd-9fc2-bc38468febfc None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:42.059196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b5ef4-4157-46fd-9fc2-bc38468febfc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:42.059469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b5ef4-4157-46fd-9fc2-bc38468febfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:42.086406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-910b5ef4-4157-46fd-9fc2-bc38468febfc None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:42.087878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-910b5ef4-4157-46fd-9fc2-bc38468febfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:42.122519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:42.147107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b70db5-c3c3-4b4a-b471-72dd6b4576dc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:42.156064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:42.198535 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 839/1636] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:42.204269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276662d-166b-4dd8-a2a5-417180d920dc None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:42.204654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276662d-166b-4dd8-a2a5-417180d920dc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:42.205220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276662d-166b-4dd8-a2a5-417180d920dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:42.241419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a6f14-dcfd-4c8b-953f-9ef2fb684fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:42.241735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6276662d-166b-4dd8-a2a5-417180d920dc None None] MFA Rules not processed for user `7f57761e58f444548d909bc8784b80d5`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:42.242044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a6f14-dcfd-4c8b-953f-9ef2fb684fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:42.242325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a6f14-dcfd-4c8b-953f-9ef2fb684fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:42.242629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a6f14-dcfd-4c8b-953f-9ef2fb684fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:42.243281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a6f14-dcfd-4c8b-953f-9ef2fb684fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:42.243525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6276662d-166b-4dd8-a2a5-417180d920dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:42.245642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a6f14-dcfd-4c8b-953f-9ef2fb684fc5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:42.267214 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 798/1637] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:42.268847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a6f14-dcfd-4c8b-953f-9ef2fb684fc5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:42.285682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a6f14-dcfd-4c8b-953f-9ef2fb684fc5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:42.292641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f981e-c9ec-4b22-86da-ad08a9633fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:42.293427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f981e-c9ec-4b22-86da-ad08a9633fbe admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:42.293714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f981e-c9ec-4b22-86da-ad08a9633fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:42.293991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f981e-c9ec-4b22-86da-ad08a9633fbe admin admin] PATH_INFO: `/v3/projects/c24c2cfdf0d94150a5b849f4c0f2c052/users/fdce7dd18be74b80a9b5cf080e303d5e/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:42.323418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f981e-c9ec-4b22-86da-ad08a9633fbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c24c2cfdf0d94150a5b849f4c0f2c052, user_id=fdce7dd18be74b80a9b5cf080e303d5e, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:42.326790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f981e-c9ec-4b22-86da-ad08a9633fbe admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:42.361348 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 799/1638] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:42 2022] PUT /identity/v3/projects/c24c2cfdf0d94150a5b849f4c0f2c052/users/fdce7dd18be74b80a9b5cf080e303d5e/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:42.362190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f981e-c9ec-4b22-86da-ad08a9633fbe admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:42.394639 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 840/1639] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:42 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:42.396580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276662d-166b-4dd8-a2a5-417180d920dc None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:42.422832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276662d-166b-4dd8-a2a5-417180d920dc None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:42.462743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f981e-c9ec-4b22-86da-ad08a9633fbe admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:42.470296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95726519-e3f9-44e2-b1b3-c468c29e0016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:42.471309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95726519-e3f9-44e2-b1b3-c468c29e0016 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:42.471763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95726519-e3f9-44e2-b1b3-c468c29e0016 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:42.472175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95726519-e3f9-44e2-b1b3-c468c29e0016 admin admin] PATH_INFO: `/v3/projects/c70de3f14b4f430ebe701ae7543b7e9f/users/efe2a89621104cc587add706d8de1586/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:42.492791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95726519-e3f9-44e2-b1b3-c468c29e0016 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c70de3f14b4f430ebe701ae7543b7e9f, user_id=efe2a89621104cc587add706d8de1586, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:42.496930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95726519-e3f9-44e2-b1b3-c468c29e0016 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:42.510378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5e010-6a23-4748-8c91-02d730212bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:42.511248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5e010-6a23-4748-8c91-02d730212bac admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:42.511592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5e010-6a23-4748-8c91-02d730212bac admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:42.511863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5e010-6a23-4748-8c91-02d730212bac admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:42.512516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5e010-6a23-4748-8c91-02d730212bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:42.514137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5e010-6a23-4748-8c91-02d730212bac admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:42.528693 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 841/1640] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:42.528906 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 800/1641] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:42 2022] PUT /identity/v3/projects/c70de3f14b4f430ebe701ae7543b7e9f/users/efe2a89621104cc587add706d8de1586/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:42.530368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94cf95-dc4b-484d-965b-9f2167ab9bf8 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:42.530677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95726519-e3f9-44e2-b1b3-c468c29e0016 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:42.530804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94cf95-dc4b-484d-965b-9f2167ab9bf8 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:42.530970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94cf95-dc4b-484d-965b-9f2167ab9bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:42.535957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-95726519-e3f9-44e2-b1b3-c468c29e0016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:42.568080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7f94cf95-dc4b-484d-965b-9f2167ab9bf8 None None] MFA Rules not processed for user `c7aab644bc2d4a9089d9ad72eff3e187`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:42.569475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7f94cf95-dc4b-484d-965b-9f2167ab9bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:42.661377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:42.679646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95726519-e3f9-44e2-b1b3-c468c29e0016 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:42.683639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2988cf-6807-44a9-ab40-0ea45b58c7ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:42.684375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2988cf-6807-44a9-ab40-0ea45b58c7ea service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:42.684774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2988cf-6807-44a9-ab40-0ea45b58c7ea service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:42.685133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2988cf-6807-44a9-ab40-0ea45b58c7ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:42.692449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:42.711726 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 842/1642] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:42.711726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94cf95-dc4b-484d-965b-9f2167ab9bf8 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:42.797099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2988cf-6807-44a9-ab40-0ea45b58c7ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:42.798681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2988cf-6807-44a9-ab40-0ea45b58c7ea service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:42.812434 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 801/1643] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:42.813881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2988cf-6807-44a9-ab40-0ea45b58c7ea service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:42.817705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94cf95-dc4b-484d-965b-9f2167ab9bf8 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:42.823008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8dce6-9eb3-4755-9978-be2e04b89d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:42.823686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8dce6-9eb3-4755-9978-be2e04b89d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:42.824047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8dce6-9eb3-4755-9978-be2e04b89d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:42.824342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8dce6-9eb3-4755-9978-be2e04b89d48 admin admin] PATH_INFO: `/v3/projects/215471dd429642b3974741aade3aaba0/users/7408e981e5404f7faeb1b018e345f741/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:42.825488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2988cf-6807-44a9-ab40-0ea45b58c7ea service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:42.826604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f8dce6-9eb3-4755-9978-be2e04b89d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=215471dd429642b3974741aade3aaba0, user_id=7408e981e5404f7faeb1b018e345f741, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:42.829392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f8dce6-9eb3-4755-9978-be2e04b89d48 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:42.844028 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 843/1644] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:42 2022] PUT /identity/v3/projects/215471dd429642b3974741aade3aaba0/users/7408e981e5404f7faeb1b018e345f741/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:42.845321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7886b4-f067-41d4-ba1f-e092bcd54b28 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:42.845673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7886b4-f067-41d4-ba1f-e092bcd54b28 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:42.846437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7886b4-f067-41d4-ba1f-e092bcd54b28 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:42.847783 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 844/1645] 10.222.0.17 () {54 vars in 961 bytes} [Sat Oct 1 20:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:48:42.848497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7886b4-f067-41d4-ba1f-e092bcd54b28 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:42.863338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041ad08-79e5-47de-9a98-f922f7cf5440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:42.865266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041ad08-79e5-47de-9a98-f922f7cf5440 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:42.865749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041ad08-79e5-47de-9a98-f922f7cf5440 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:42.866035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041ad08-79e5-47de-9a98-f922f7cf5440 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:42.866805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041ad08-79e5-47de-9a98-f922f7cf5440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:42.868679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041ad08-79e5-47de-9a98-f922f7cf5440 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:42.886969 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 802/1646] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:42.886969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041ad08-79e5-47de-9a98-f922f7cf5440 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:42.897241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041ad08-79e5-47de-9a98-f922f7cf5440 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:42.949557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7886b4-f067-41d4-ba1f-e092bcd54b28 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:42.967520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd326d2e-3a64-4cbf-82c1-2cd5016fa9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:42.968212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd326d2e-3a64-4cbf-82c1-2cd5016fa9e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:42.968495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd326d2e-3a64-4cbf-82c1-2cd5016fa9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:42.968774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd326d2e-3a64-4cbf-82c1-2cd5016fa9e1 admin admin] PATH_INFO: `/v3/projects/08b6602ad13646c195ab1f3be7aa0e72/users/3e0503d20ac74f9ebe516009ddf320a5/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:42.971073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd326d2e-3a64-4cbf-82c1-2cd5016fa9e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08b6602ad13646c195ab1f3be7aa0e72, user_id=3e0503d20ac74f9ebe516009ddf320a5, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:42.974602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd326d2e-3a64-4cbf-82c1-2cd5016fa9e1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:42.984082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85322e-2d14-4b51-a71a-ef2514ad8947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:42.985475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85322e-2d14-4b51-a71a-ef2514ad8947 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:42.986091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85322e-2d14-4b51-a71a-ef2514ad8947 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:42.986660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85322e-2d14-4b51-a71a-ef2514ad8947 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:42.987429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85322e-2d14-4b51-a71a-ef2514ad8947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:42.989186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85322e-2d14-4b51-a71a-ef2514ad8947 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:42.994709 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 845/1647] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:42 2022] PUT /identity/v3/projects/08b6602ad13646c195ab1f3be7aa0e72/users/3e0503d20ac74f9ebe516009ddf320a5/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:42.996560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08be6d-2c79-42b9-8714-4b1f8142a815 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:43.000132 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 803/1648] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:43.001398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ba508-4ce6-4897-9c39-0b280d732a8a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:43.001684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ba508-4ce6-4897-9c39-0b280d732a8a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:43.001966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ba508-4ce6-4897-9c39-0b280d732a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:43.004197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08be6d-2c79-42b9-8714-4b1f8142a815 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:43.004943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08be6d-2c79-42b9-8714-4b1f8142a815 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:43.006947 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 846/1649] 10.222.0.17 () {54 vars in 961 bytes} [Sat Oct 1 20:48:42 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:48:43.008252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c245531-1dd8-4843-b174-6444f731edf2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:43.008689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c245531-1dd8-4843-b174-6444f731edf2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:43.009357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c245531-1dd8-4843-b174-6444f731edf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:43.034979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d16ba508-4ce6-4897-9c39-0b280d732a8a None None] MFA Rules not processed for user `7408e981e5404f7faeb1b018e345f741`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:43.036857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d16ba508-4ce6-4897-9c39-0b280d732a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:43.086415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0c245531-1dd8-4843-b174-6444f731edf2 None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:43.088771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0c245531-1dd8-4843-b174-6444f731edf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:43.142960 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 804/1650] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:43.143901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ba508-4ce6-4897-9c39-0b280d732a8a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:43.158681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:43.161966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:43.193622 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 847/1651] 10.222.0.17 () {58 vars in 1058 bytes} [Sat Oct 1 20:48:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:43.195242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b12d8-05ad-4f1e-acb1-bacb92c67573 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:43.196174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b12d8-05ad-4f1e-acb1-bacb92c67573 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:43.196528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b12d8-05ad-4f1e-acb1-bacb92c67573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:43.252834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-553b12d8-05ad-4f1e-acb1-bacb92c67573 None None] MFA Rules not processed for user `3e0503d20ac74f9ebe516009ddf320a5`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:43.254604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-553b12d8-05ad-4f1e-acb1-bacb92c67573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:43.257814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ba508-4ce6-4897-9c39-0b280d732a8a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:43.267842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f166ed4-9cf7-4984-910e-790a6487da04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:43.268446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f166ed4-9cf7-4984-910e-790a6487da04 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:43.268749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f166ed4-9cf7-4984-910e-790a6487da04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:43.268997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f166ed4-9cf7-4984-910e-790a6487da04 admin admin] PATH_INFO: `/v3/projects/c24c2cfdf0d94150a5b849f4c0f2c052/users/fdce7dd18be74b80a9b5cf080e303d5e/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:43.271414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f166ed4-9cf7-4984-910e-790a6487da04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c24c2cfdf0d94150a5b849f4c0f2c052, user_id=fdce7dd18be74b80a9b5cf080e303d5e, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:43.274113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f166ed4-9cf7-4984-910e-790a6487da04 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:43.292148 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 805/1652] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:43 2022] PUT /identity/v3/projects/c24c2cfdf0d94150a5b849f4c0f2c052/users/fdce7dd18be74b80a9b5cf080e303d5e/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:43.293667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f166ed4-9cf7-4984-910e-790a6487da04 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:43.378875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f166ed4-9cf7-4984-910e-790a6487da04 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:43.382257 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 848/1653] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:43 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:43.384109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574cbf37-425a-41da-b0f4-d3bea68699f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:43.384313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a18888-16ff-4356-8228-61a0ba6d5bc6 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:43.384738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a18888-16ff-4356-8228-61a0ba6d5bc6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:43.385006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cbf37-425a-41da-b0f4-d3bea68699f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:43.385298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cbf37-425a-41da-b0f4-d3bea68699f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:43.385629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a18888-16ff-4356-8228-61a0ba6d5bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:43.385994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cbf37-425a-41da-b0f4-d3bea68699f7 admin admin] PATH_INFO: `/v3/projects/c70de3f14b4f430ebe701ae7543b7e9f/users/efe2a89621104cc587add706d8de1586/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:43.389009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574cbf37-425a-41da-b0f4-d3bea68699f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c70de3f14b4f430ebe701ae7543b7e9f, user_id=efe2a89621104cc587add706d8de1586, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:43.392933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574cbf37-425a-41da-b0f4-d3bea68699f7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:43.419969 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 806/1654] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:43 2022] PUT /identity/v3/projects/c70de3f14b4f430ebe701ae7543b7e9f/users/efe2a89621104cc587add706d8de1586/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:43.420176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-67a18888-16ff-4356-8228-61a0ba6d5bc6 None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:43.420317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574cbf37-425a-41da-b0f4-d3bea68699f7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:43.420412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-67a18888-16ff-4356-8228-61a0ba6d5bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:43.509717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574cbf37-425a-41da-b0f4-d3bea68699f7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:43.520499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddc6de-2044-4bba-9c03-a3e294a6fac1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:43.521493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddc6de-2044-4bba-9c03-a3e294a6fac1 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:43.521972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddc6de-2044-4bba-9c03-a3e294a6fac1 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:43.522469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddc6de-2044-4bba-9c03-a3e294a6fac1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:43.527966 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 849/1655] 10.222.0.17 () {58 vars in 1058 bytes} [Sat Oct 1 20:48:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:43.529205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a18888-16ff-4356-8228-61a0ba6d5bc6 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:43.531310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-67a18888-16ff-4356-8228-61a0ba6d5bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:43.580141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:43.614598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caddc6de-2044-4bba-9c03-a3e294a6fac1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:43.616351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caddc6de-2044-4bba-9c03-a3e294a6fac1 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:43.632435 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 807/1656] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:43 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:43.633728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3248b84-4ece-49c6-b5a8-ebfc45e29692 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:43.634115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3248b84-4ece-49c6-b5a8-ebfc45e29692 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:43.634667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3248b84-4ece-49c6-b5a8-ebfc45e29692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:43.656061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a18888-16ff-4356-8228-61a0ba6d5bc6 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:43.660940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05f887-9ecf-4f97-b49d-9faac2591a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:43.661720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05f887-9ecf-4f97-b49d-9faac2591a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:43.662007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05f887-9ecf-4f97-b49d-9faac2591a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:43.662281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05f887-9ecf-4f97-b49d-9faac2591a67 admin admin] PATH_INFO: `/v3/projects/b6ead845a07d465f92f43f1d284658cf` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:43.663507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05f887-9ecf-4f97-b49d-9faac2591a67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6ead845a07d465f92f43f1d284658cf)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:43.665612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05f887-9ecf-4f97-b49d-9faac2591a67 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:43.675324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:43.713537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b3248b84-4ece-49c6-b5a8-ebfc45e29692 None None] MFA Rules not processed for user `fdce7dd18be74b80a9b5cf080e303d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:43.716011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b3248b84-4ece-49c6-b5a8-ebfc45e29692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:43.744205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fb05f887-9ecf-4f97-b49d-9faac2591a67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6ead845a07d465f92f43f1d284658cf', 'request_id': 'req-fb05f887-9ecf-4f97-b49d-9faac2591a67'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:43.774424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fb05f887-9ecf-4f97-b49d-9faac2591a67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6ead845a07d465f92f43f1d284658cf', 'request_id': 'req-fb05f887-9ecf-4f97-b49d-9faac2591a67'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:43.790576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fb05f887-9ecf-4f97-b49d-9faac2591a67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6ead845a07d465f92f43f1d284658cf', 'request_id': 'req-fb05f887-9ecf-4f97-b49d-9faac2591a67'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:43.816312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fb05f887-9ecf-4f97-b49d-9faac2591a67 admin admin] The token cache is being invalidate because project b6ead845a07d465f92f43f1d284658cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:48:43.816609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fb05f887-9ecf-4f97-b49d-9faac2591a67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:43.818537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fb05f887-9ecf-4f97-b49d-9faac2591a67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:43.822534 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 850/1657] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:48:43 2022] DELETE /identity/v3/projects/b6ead845a07d465f92f43f1d284658cf => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:43.826538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05f887-9ecf-4f97-b49d-9faac2591a67 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:43.828881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fb05f887-9ecf-4f97-b49d-9faac2591a67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:43.904218 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 808/1658] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:43 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:43.905594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db53e2-ee55-4cb5-bb14-342b4232df1f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:43.905880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db53e2-ee55-4cb5-bb14-342b4232df1f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:43.906155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db53e2-ee55-4cb5-bb14-342b4232df1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:43.961739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f7db53e2-ee55-4cb5-bb14-342b4232df1f None None] MFA Rules not processed for user `efe2a89621104cc587add706d8de1586`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:43.963296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f7db53e2-ee55-4cb5-bb14-342b4232df1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:43.986280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05f887-9ecf-4f97-b49d-9faac2591a67 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:43.995883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88adc1-095e-4b7c-85f4-d086bf60605e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:43.997178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88adc1-095e-4b7c-85f4-d086bf60605e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:43.998834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88adc1-095e-4b7c-85f4-d086bf60605e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:43.999269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88adc1-095e-4b7c-85f4-d086bf60605e admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:44.000055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88adc1-095e-4b7c-85f4-d086bf60605e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:44.004718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88adc1-095e-4b7c-85f4-d086bf60605e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:44.072450 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 851/1659] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:43 2022] POST /identity/v3/projects => generated 395 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:44.075335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88adc1-095e-4b7c-85f4-d086bf60605e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:44.079205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3b88adc1-095e-4b7c-85f4-d086bf60605e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:44.125400 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 809/1660] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:44.136047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a24c2a-bb6c-40c4-bf3f-eb9ad04e4ec8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:44.136801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a24c2a-bb6c-40c4-bf3f-eb9ad04e4ec8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:44.141929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a24c2a-bb6c-40c4-bf3f-eb9ad04e4ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:44.191393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-42a24c2a-bb6c-40c4-bf3f-eb9ad04e4ec8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:44.193980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-42a24c2a-bb6c-40c4-bf3f-eb9ad04e4ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:44.230531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88adc1-095e-4b7c-85f4-d086bf60605e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:44.239084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a269171-ca8d-4561-bdc8-42b9470f105c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:44.239758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a269171-ca8d-4561-bdc8-42b9470f105c service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:44.240042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a269171-ca8d-4561-bdc8-42b9470f105c service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:44.240315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a269171-ca8d-4561-bdc8-42b9470f105c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:44.245183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8a269171-ca8d-4561-bdc8-42b9470f105c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:44.296005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:44.300383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:44.368657 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 810/1661] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:44.369646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a24c2a-bb6c-40c4-bf3f-eb9ad04e4ec8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:44.371164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-42a24c2a-bb6c-40c4-bf3f-eb9ad04e4ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:44.398232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a24c2a-bb6c-40c4-bf3f-eb9ad04e4ec8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:44.406740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0acf1c-4857-44d2-90d9-9fb813b9aceb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:44.407300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a269171-ca8d-4561-bdc8-42b9470f105c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:44.407622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0acf1c-4857-44d2-90d9-9fb813b9aceb service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:44.408001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0acf1c-4857-44d2-90d9-9fb813b9aceb service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:44.408374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0acf1c-4857-44d2-90d9-9fb813b9aceb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:44.409795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a269171-ca8d-4561-bdc8-42b9470f105c service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:44.410750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8b0acf1c-4857-44d2-90d9-9fb813b9aceb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:44.449375 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 852/1662] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:44.456903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a269171-ca8d-4561-bdc8-42b9470f105c service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:44.493424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a269171-ca8d-4561-bdc8-42b9470f105c service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:44.494781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0acf1c-4857-44d2-90d9-9fb813b9aceb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:44.497337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0acf1c-4857-44d2-90d9-9fb813b9aceb service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:44.510334 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 811/1663] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:44.512287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b91b8-8150-4f68-b9da-8c3912f3c0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:44.512511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22e5f8-6cce-4878-a809-a20bdb0a623d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:44.512956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22e5f8-6cce-4878-a809-a20bdb0a623d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:44.513269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22e5f8-6cce-4878-a809-a20bdb0a623d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:44.514591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b91b8-8150-4f68-b9da-8c3912f3c0b0 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:44.514889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b91b8-8150-4f68-b9da-8c3912f3c0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:44.515167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b91b8-8150-4f68-b9da-8c3912f3c0b0 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:44.515814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b91b8-8150-4f68-b9da-8c3912f3c0b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:44.521446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b91b8-8150-4f68-b9da-8c3912f3c0b0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:44.543334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2e22e5f8-6cce-4878-a809-a20bdb0a623d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:44.544760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2e22e5f8-6cce-4878-a809-a20bdb0a623d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:44.560669 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 812/1664] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:44.561734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22e5f8-6cce-4878-a809-a20bdb0a623d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:44.569977 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 853/1665] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:44 2022] POST /identity/v3/users => generated 298 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:44.571292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b91b8-8150-4f68-b9da-8c3912f3c0b0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:44.580711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22e5f8-6cce-4878-a809-a20bdb0a623d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:44.587140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea69418-6b36-493a-b788-437882c63002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:44.588561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea69418-6b36-493a-b788-437882c63002 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:44.588848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea69418-6b36-493a-b788-437882c63002 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:44.589120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea69418-6b36-493a-b788-437882c63002 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:44.589878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea69418-6b36-493a-b788-437882c63002 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:48:44.590185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea69418-6b36-493a-b788-437882c63002 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:44.592395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea69418-6b36-493a-b788-437882c63002 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:44.598159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b91b8-8150-4f68-b9da-8c3912f3c0b0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:44.607722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30809bb7-a5f0-47ac-988d-8e427644d628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:44.608359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30809bb7-a5f0-47ac-988d-8e427644d628 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:44.608644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30809bb7-a5f0-47ac-988d-8e427644d628 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:44.608921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30809bb7-a5f0-47ac-988d-8e427644d628 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:44.609580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30809bb7-a5f0-47ac-988d-8e427644d628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:44.611565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30809bb7-a5f0-47ac-988d-8e427644d628 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:44.614457 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 813/1666] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:44.615500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea69418-6b36-493a-b788-437882c63002 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:44.629487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea69418-6b36-493a-b788-437882c63002 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:44.635197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff455066-25ce-4969-b485-5c2b6361a69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:44.635895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff455066-25ce-4969-b485-5c2b6361a69b admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:44.636099 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 854/1667] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:44.636200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff455066-25ce-4969-b485-5c2b6361a69b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:44.637172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff455066-25ce-4969-b485-5c2b6361a69b admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:44.637559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30809bb7-a5f0-47ac-988d-8e427644d628 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:44.637737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff455066-25ce-4969-b485-5c2b6361a69b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:44.640080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff455066-25ce-4969-b485-5c2b6361a69b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:44.648794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30809bb7-a5f0-47ac-988d-8e427644d628 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:44.654827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036e16a-329b-4cf9-905a-08708b311350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:44.655451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036e16a-329b-4cf9-905a-08708b311350 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:44.655752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036e16a-329b-4cf9-905a-08708b311350 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:44.656064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036e16a-329b-4cf9-905a-08708b311350 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:44.656871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036e16a-329b-4cf9-905a-08708b311350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:44.658643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036e16a-329b-4cf9-905a-08708b311350 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:44.676502 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 855/1668] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:44.677234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036e16a-329b-4cf9-905a-08708b311350 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:44.688581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036e16a-329b-4cf9-905a-08708b311350 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:44.693186 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 814/1669] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:44 2022] POST /identity/v3/projects => generated 383 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:44.694352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b3e7a-3d02-459a-b9b8-b0331c00bd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:44.694647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff455066-25ce-4969-b485-5c2b6361a69b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:44.695198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b3e7a-3d02-459a-b9b8-b0331c00bd4f admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:44.695550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b3e7a-3d02-459a-b9b8-b0331c00bd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:44.695984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b3e7a-3d02-459a-b9b8-b0331c00bd4f admin admin] PATH_INFO: `/v3/projects/02be8fe6f40b42a684054e8a3a7ebf9e/users/6debf86cca3a4c81a036c8e7f243e60a/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:44.713446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b3e7a-3d02-459a-b9b8-b0331c00bd4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=02be8fe6f40b42a684054e8a3a7ebf9e, user_id=6debf86cca3a4c81a036c8e7f243e60a, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:44.716888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b3e7a-3d02-459a-b9b8-b0331c00bd4f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:44.736914 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 856/1670] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:44 2022] PUT /identity/v3/projects/02be8fe6f40b42a684054e8a3a7ebf9e/users/6debf86cca3a4c81a036c8e7f243e60a/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:44.748296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b3e7a-3d02-459a-b9b8-b0331c00bd4f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:44.818474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff455066-25ce-4969-b485-5c2b6361a69b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:44.823138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b3e7a-3d02-459a-b9b8-b0331c00bd4f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:44.829510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960a1ae-e5ca-42da-9ef8-7cf8684a77eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:44.830280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960a1ae-e5ca-42da-9ef8-7cf8684a77eb admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:44.830622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960a1ae-e5ca-42da-9ef8-7cf8684a77eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:44.830906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960a1ae-e5ca-42da-9ef8-7cf8684a77eb admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:44.831550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960a1ae-e5ca-42da-9ef8-7cf8684a77eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:44.833507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960a1ae-e5ca-42da-9ef8-7cf8684a77eb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:44.862472 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 857/1671] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:44 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:44.864017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a376aca-1467-4d3b-b80a-869ec695ff38 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:44.864315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a376aca-1467-4d3b-b80a-869ec695ff38 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:44.864596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a376aca-1467-4d3b-b80a-869ec695ff38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:44.893108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:44.903908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5a376aca-1467-4d3b-b80a-869ec695ff38 None None] MFA Rules not processed for user `6debf86cca3a4c81a036c8e7f243e60a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:44.905823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5a376aca-1467-4d3b-b80a-869ec695ff38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:44.970637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:44.991514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a2a1ef-dfe6-41e7-9cab-7c2cb594a50c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:44.992143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2a1ef-dfe6-41e7-9cab-7c2cb594a50c service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:44.992419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2a1ef-dfe6-41e7-9cab-7c2cb594a50c service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:44.992688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2a1ef-dfe6-41e7-9cab-7c2cb594a50c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:45.047605 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 858/1672] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:44 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:45.048872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a376aca-1467-4d3b-b80a-869ec695ff38 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:45.062673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a376aca-1467-4d3b-b80a-869ec695ff38 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:45.069123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04a652-4169-4c43-8245-0011d876c59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:45.069801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04a652-4169-4c43-8245-0011d876c59c admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:45.070094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04a652-4169-4c43-8245-0011d876c59c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:45.070426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04a652-4169-4c43-8245-0011d876c59c admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:45.071144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c04a652-4169-4c43-8245-0011d876c59c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:45.072828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c04a652-4169-4c43-8245-0011d876c59c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:45.090198 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 859/1673] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:45.093839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04a652-4169-4c43-8245-0011d876c59c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:45.118452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04a652-4169-4c43-8245-0011d876c59c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:45.132028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6abc7c-40a7-4fb8-9c35-71fbe8c2b952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:45.132651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6abc7c-40a7-4fb8-9c35-71fbe8c2b952 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:45.132928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6abc7c-40a7-4fb8-9c35-71fbe8c2b952 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:45.133201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6abc7c-40a7-4fb8-9c35-71fbe8c2b952 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:45.134361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2a1ef-dfe6-41e7-9cab-7c2cb594a50c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:45.135988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2a1ef-dfe6-41e7-9cab-7c2cb594a50c service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:45.136831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6abc7c-40a7-4fb8-9c35-71fbe8c2b952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:45.139366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6abc7c-40a7-4fb8-9c35-71fbe8c2b952 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:45.153635 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 815/1674] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:44 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:45.155273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5b2ab-db1c-4b06-ba97-f611a222334b None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:45.155666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5b2ab-db1c-4b06-ba97-f611a222334b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:45.156049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5b2ab-db1c-4b06-ba97-f611a222334b None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:45.157819 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 816/1675] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:48:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:48:45.160942 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 860/1676] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:45.163597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5b2ab-db1c-4b06-ba97-f611a222334b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:45.163717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6abc7c-40a7-4fb8-9c35-71fbe8c2b952 admin admin] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:48:45.179013 ncdv-master-1 devstack@keystone.service[99139]: WARNING keystone.server.flask.request_processing.middleware.auth_context [None req-2b6abc7c-40a7-4fb8-9c35-71fbe8c2b952 admin admin] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. Oct 01 20:48:45.179237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5b2ab-db1c-4b06-ba97-f611a222334b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:45.179319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6abc7c-40a7-4fb8-9c35-71fbe8c2b952 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:45.181155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2b6abc7c-40a7-4fb8-9c35-71fbe8c2b952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:45.185018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc09cfd-ef2b-4090-b6f8-919432a186af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:45.185813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc09cfd-ef2b-4090-b6f8-919432a186af admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:45.186170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc09cfd-ef2b-4090-b6f8-919432a186af admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:45.186541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc09cfd-ef2b-4090-b6f8-919432a186af admin admin] PATH_INFO: `/v3/projects/94df346c1d9645d3831ede450f4da091/users/f921d61f3bbb49a39a5bb998f6e09012/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:45.206894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc09cfd-ef2b-4090-b6f8-919432a186af admin admin] RBAC: Authorizing `identity:create_grant(project_id=94df346c1d9645d3831ede450f4da091, user_id=f921d61f3bbb49a39a5bb998f6e09012, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:45.210704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc09cfd-ef2b-4090-b6f8-919432a186af admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:45.235481 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 817/1677] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:45 2022] PUT /identity/v3/projects/94df346c1d9645d3831ede450f4da091/users/f921d61f3bbb49a39a5bb998f6e09012/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:45.248244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181ec4c-e354-487d-ab70-486d8970c43b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:45.248582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181ec4c-e354-487d-ab70-486d8970c43b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:45.248807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181ec4c-e354-487d-ab70-486d8970c43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:45.288566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6abc7c-40a7-4fb8-9c35-71fbe8c2b952 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:45.293706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6181ec4c-e354-487d-ab70-486d8970c43b None None] MFA Rules not processed for user `f921d61f3bbb49a39a5bb998f6e09012`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:45.295377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6181ec4c-e354-487d-ab70-486d8970c43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:45.364681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58b7f2-40ae-4d86-87bd-15d1b89558a0 tempest-FlavorsAccessTestJSON-1602123528 tempest-FlavorsAccessTestJSON-1602123528-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f57761e58f444548d909bc8784b80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8eaa857c05240d394707c400cf87116', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:45.365337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58b7f2-40ae-4d86-87bd-15d1b89558a0 tempest-FlavorsAccessTestJSON-1602123528 tempest-FlavorsAccessTestJSON-1602123528-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:45.365629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58b7f2-40ae-4d86-87bd-15d1b89558a0 tempest-FlavorsAccessTestJSON-1602123528 tempest-FlavorsAccessTestJSON-1602123528-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:45.366025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58b7f2-40ae-4d86-87bd-15d1b89558a0 tempest-FlavorsAccessTestJSON-1602123528 tempest-FlavorsAccessTestJSON-1602123528-project-admin] PATH_INFO: `/v3/projects/8fb1cc925cc241d291826e2a79e9c2db` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:45.367460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58b7f2-40ae-4d86-87bd-15d1b89558a0 tempest-FlavorsAccessTestJSON-1602123528 tempest-FlavorsAccessTestJSON-1602123528-project-admin] RBAC: Authorizing `identity:get_project(project_id=8fb1cc925cc241d291826e2a79e9c2db)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:45.382549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58b7f2-40ae-4d86-87bd-15d1b89558a0 tempest-FlavorsAccessTestJSON-1602123528 tempest-FlavorsAccessTestJSON-1602123528-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:45.388312 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 861/1678] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:48:45 2022] GET /identity/v3/projects/8fb1cc925cc241d291826e2a79e9c2db => generated 379 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:45.389773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58b7f2-40ae-4d86-87bd-15d1b89558a0 tempest-FlavorsAccessTestJSON-1602123528 tempest-FlavorsAccessTestJSON-1602123528-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:45.404503 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 818/1679] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:48:45.409354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181ec4c-e354-487d-ab70-486d8970c43b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:45.467088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:45.497607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:45.503136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58b7f2-40ae-4d86-87bd-15d1b89558a0 tempest-FlavorsAccessTestJSON-1602123528 tempest-FlavorsAccessTestJSON-1602123528-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:45.512774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61cfdb-85e9-47c8-b075-e86b81ca4055 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:45.514054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61cfdb-85e9-47c8-b075-e86b81ca4055 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:45.514617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61cfdb-85e9-47c8-b075-e86b81ca4055 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:45.514904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61cfdb-85e9-47c8-b075-e86b81ca4055 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:45.536954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181ec4c-e354-487d-ab70-486d8970c43b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:45.543773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce94cfd-ef7f-404c-a23b-ba82b0b05585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:45.544953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce94cfd-ef7f-404c-a23b-ba82b0b05585 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:45.545328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce94cfd-ef7f-404c-a23b-ba82b0b05585 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:45.545702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce94cfd-ef7f-404c-a23b-ba82b0b05585 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:45.546444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce94cfd-ef7f-404c-a23b-ba82b0b05585 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:45.548605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce94cfd-ef7f-404c-a23b-ba82b0b05585 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:45.589576 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 819/1680] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:45 2022] POST /identity/v3/projects => generated 383 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:48:45.591071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce94cfd-ef7f-404c-a23b-ba82b0b05585 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:45.614257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61cfdb-85e9-47c8-b075-e86b81ca4055 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:45.616362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61cfdb-85e9-47c8-b075-e86b81ca4055 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:45.643575 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 862/1681] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:45.665332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61cfdb-85e9-47c8-b075-e86b81ca4055 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:45.683707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61cfdb-85e9-47c8-b075-e86b81ca4055 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:45.715014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce94cfd-ef7f-404c-a23b-ba82b0b05585 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:45.721316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261c918-925a-4c23-8ece-a68ae8ccdb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:45.722180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261c918-925a-4c23-8ece-a68ae8ccdb5c admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:45.722557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261c918-925a-4c23-8ece-a68ae8ccdb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:45.722917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261c918-925a-4c23-8ece-a68ae8ccdb5c admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:45.723629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261c918-925a-4c23-8ece-a68ae8ccdb5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:45.726162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261c918-925a-4c23-8ece-a68ae8ccdb5c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:45.763507 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 820/1682] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:45 2022] POST /identity/v3/users => generated 298 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:45.764466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261c918-925a-4c23-8ece-a68ae8ccdb5c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:45.794239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8743413-dd8d-4fe7-b4c1-38bb7f97dd88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:45.794564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8743413-dd8d-4fe7-b4c1-38bb7f97dd88 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:45.794849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8743413-dd8d-4fe7-b4c1-38bb7f97dd88 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:45.795172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8743413-dd8d-4fe7-b4c1-38bb7f97dd88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:45.797972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b8743413-dd8d-4fe7-b4c1-38bb7f97dd88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:45.863701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261c918-925a-4c23-8ece-a68ae8ccdb5c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:45.869473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f801e-c09b-4668-ad0c-7cff6aa1fc18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:45.870159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f801e-c09b-4668-ad0c-7cff6aa1fc18 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:45.870596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f801e-c09b-4668-ad0c-7cff6aa1fc18 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:45.871548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f801e-c09b-4668-ad0c-7cff6aa1fc18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:45.873448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1f5f801e-c09b-4668-ad0c-7cff6aa1fc18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:45.875711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8743413-dd8d-4fe7-b4c1-38bb7f97dd88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:45.877411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8743413-dd8d-4fe7-b4c1-38bb7f97dd88 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:45.907664 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 863/1683] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:45 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:48:45.908517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8743413-dd8d-4fe7-b4c1-38bb7f97dd88 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:45.938124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:45.951683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8743413-dd8d-4fe7-b4c1-38bb7f97dd88 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:45.966871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f2d62-9c42-4ac2-b4a5-fa59cb546338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:45.967981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f2d62-9c42-4ac2-b4a5-fa59cb546338 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:45.969244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f2d62-9c42-4ac2-b4a5-fa59cb546338 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:45.969745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f2d62-9c42-4ac2-b4a5-fa59cb546338 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:45.970957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f2d62-9c42-4ac2-b4a5-fa59cb546338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:45.971645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f801e-c09b-4668-ad0c-7cff6aa1fc18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:45.973443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f801e-c09b-4668-ad0c-7cff6aa1fc18 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:45.976786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f2d62-9c42-4ac2-b4a5-fa59cb546338 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:45.997524 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 864/1684] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:45.998866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f2d62-9c42-4ac2-b4a5-fa59cb546338 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:46.007875 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 821/1685] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:46.019700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f2d62-9c42-4ac2-b4a5-fa59cb546338 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:46.032423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81831433-55b7-402a-a175-9e37b67401ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:46.033499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81831433-55b7-402a-a175-9e37b67401ff admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:46.034597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81831433-55b7-402a-a175-9e37b67401ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:46.034757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81831433-55b7-402a-a175-9e37b67401ff admin admin] PATH_INFO: `/v3/projects/c1bdf2dd709e4d44b587f35be0b0f43f/users/8a423ddf823744069284c149f96107c9/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:46.052596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:46.080588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f801e-c09b-4668-ad0c-7cff6aa1fc18 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:46.084496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81831433-55b7-402a-a175-9e37b67401ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1bdf2dd709e4d44b587f35be0b0f43f, user_id=8a423ddf823744069284c149f96107c9, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:46.087762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81831433-55b7-402a-a175-9e37b67401ff admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:46.106906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f801e-c09b-4668-ad0c-7cff6aa1fc18 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:46.114566 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 865/1686] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:45 2022] PUT /identity/v3/projects/c1bdf2dd709e4d44b587f35be0b0f43f/users/8a423ddf823744069284c149f96107c9/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:46.118162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81831433-55b7-402a-a175-9e37b67401ff admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:46.248690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81831433-55b7-402a-a175-9e37b67401ff admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:46.260278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9708abd-fa92-4109-b3f1-65bc2ebd92ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:46.262046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9708abd-fa92-4109-b3f1-65bc2ebd92ed service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:46.262521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9708abd-fa92-4109-b3f1-65bc2ebd92ed service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:46.262805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9708abd-fa92-4109-b3f1-65bc2ebd92ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:46.263390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2ec01-b0e4-41e6-9bfe-2a1cc2f0c0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:46.264733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b9708abd-fa92-4109-b3f1-65bc2ebd92ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:46.264837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2ec01-b0e4-41e6-9bfe-2a1cc2f0c0f7 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:46.265143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2ec01-b0e4-41e6-9bfe-2a1cc2f0c0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:46.265492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2ec01-b0e4-41e6-9bfe-2a1cc2f0c0f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:46.266264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2ec01-b0e4-41e6-9bfe-2a1cc2f0c0f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:46.268114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2ec01-b0e4-41e6-9bfe-2a1cc2f0c0f7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:46.290153 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 866/1687] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:46.296406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2ec01-b0e4-41e6-9bfe-2a1cc2f0c0f7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:46.307499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2ec01-b0e4-41e6-9bfe-2a1cc2f0c0f7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:46.317413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17925f07-8a50-4b45-8f08-4dbc55abd000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:46.318721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17925f07-8a50-4b45-8f08-4dbc55abd000 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:46.319310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17925f07-8a50-4b45-8f08-4dbc55abd000 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:46.319729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17925f07-8a50-4b45-8f08-4dbc55abd000 admin admin] PATH_INFO: `/v3/projects/c1bdf2dd709e4d44b587f35be0b0f43f/users/8a423ddf823744069284c149f96107c9/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:46.324504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17925f07-8a50-4b45-8f08-4dbc55abd000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1bdf2dd709e4d44b587f35be0b0f43f, user_id=8a423ddf823744069284c149f96107c9, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:46.328766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17925f07-8a50-4b45-8f08-4dbc55abd000 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:46.350363 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 867/1688] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:46 2022] PUT /identity/v3/projects/c1bdf2dd709e4d44b587f35be0b0f43f/users/8a423ddf823744069284c149f96107c9/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:46.365804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c4b30-d519-4931-bd89-49aba2597c97 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:46.366166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c4b30-d519-4931-bd89-49aba2597c97 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:46.367279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c4b30-d519-4931-bd89-49aba2597c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:46.394614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9708abd-fa92-4109-b3f1-65bc2ebd92ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:46.399116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9708abd-fa92-4109-b3f1-65bc2ebd92ed service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:46.413874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2a2c4b30-d519-4931-bd89-49aba2597c97 None None] MFA Rules not processed for user `8a423ddf823744069284c149f96107c9`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:46.415337 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 822/1689] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:48:46.416644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3126356-ef8c-40c3-ad05-b0bdf706aa09 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:46.416927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3126356-ef8c-40c3-ad05-b0bdf706aa09 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:46.417268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3126356-ef8c-40c3-ad05-b0bdf706aa09 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:46.426528 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 823/1690] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:48:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:48:46.426763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2a2c4b30-d519-4931-bd89-49aba2597c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:46.428404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb22622-4fd6-49c8-aa32-f8494d2a3339 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:46.439203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb22622-4fd6-49c8-aa32-f8494d2a3339 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:46.439494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb22622-4fd6-49c8-aa32-f8494d2a3339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:46.490532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-edb22622-4fd6-49c8-aa32-f8494d2a3339 None None] MFA Rules not processed for user `f40f50783d9f417a8f84fab11b0d10d0`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:46.491779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-edb22622-4fd6-49c8-aa32-f8494d2a3339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:46.537525 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 868/1691] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:46 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:46.542778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c4b30-d519-4931-bd89-49aba2597c97 None None] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:48:46.642196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:46.653974 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 824/1692] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:46 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:46.661151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb22622-4fd6-49c8-aa32-f8494d2a3339 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:46.664532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-edb22622-4fd6-49c8-aa32-f8494d2a3339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:46.696567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:46.696888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c4b30-d519-4931-bd89-49aba2597c97 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:46.790451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb22622-4fd6-49c8-aa32-f8494d2a3339 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:46.796343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607dcaf5-1787-4dfc-8b2c-71f2e0955daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:46.798303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dcaf5-1787-4dfc-8b2c-71f2e0955daf admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:46.798677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dcaf5-1787-4dfc-8b2c-71f2e0955daf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:46.799263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dcaf5-1787-4dfc-8b2c-71f2e0955daf admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:46.800138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607dcaf5-1787-4dfc-8b2c-71f2e0955daf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:46.805239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607dcaf5-1787-4dfc-8b2c-71f2e0955daf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:46.846061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c4b30-d519-4931-bd89-49aba2597c97 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:46.850429 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 825/1693] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:46 2022] POST /identity/v3/projects => generated 401 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:46.850429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c800c13-92ab-4906-8192-a9dd24b29c8f None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:46.850429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c800c13-92ab-4906-8192-a9dd24b29c8f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:46.850429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c800c13-92ab-4906-8192-a9dd24b29c8f None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:46.855069 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 826/1694] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:48:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:48:46.855971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c800c13-92ab-4906-8192-a9dd24b29c8f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:46.923222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489eb9ab-994d-458b-a44d-427d855e0bde tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e0503d20ac74f9ebe516009ddf320a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08b6602ad13646c195ab1f3be7aa0e72', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:46.923870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489eb9ab-994d-458b-a44d-427d855e0bde tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:46.924226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489eb9ab-994d-458b-a44d-427d855e0bde tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:46.924570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489eb9ab-994d-458b-a44d-427d855e0bde tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] PATH_INFO: `/v3/projects/2b38d7f557db4bfeacdd4e958cec9557` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:46.925811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489eb9ab-994d-458b-a44d-427d855e0bde tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] RBAC: Authorizing `identity:get_project(project_id=2b38d7f557db4bfeacdd4e958cec9557)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:46.927987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489eb9ab-994d-458b-a44d-427d855e0bde tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:46.929732 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 869/1695] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:48:46 2022] GET /identity/v3/projects/2b38d7f557db4bfeacdd4e958cec9557 => generated 395 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:46.930959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b894fd7-5642-4597-9f7d-ff18c18c87e5 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:46.931478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b894fd7-5642-4597-9f7d-ff18c18c87e5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:46.931743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b894fd7-5642-4597-9f7d-ff18c18c87e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:46.935712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c800c13-92ab-4906-8192-a9dd24b29c8f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:46.940426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36515560-f2c4-493f-8baf-7c8c12f782f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:46.941137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36515560-f2c4-493f-8baf-7c8c12f782f3 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:46.941484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36515560-f2c4-493f-8baf-7c8c12f782f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:46.941827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36515560-f2c4-493f-8baf-7c8c12f782f3 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:46.942949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36515560-f2c4-493f-8baf-7c8c12f782f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:46.945179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36515560-f2c4-493f-8baf-7c8c12f782f3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:46.956681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3b894fd7-5642-4597-9f7d-ff18c18c87e5 None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:46.958626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3b894fd7-5642-4597-9f7d-ff18c18c87e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:46.981460 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 827/1696] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:46 2022] POST /identity/v3/users => generated 307 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:46.982689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36515560-f2c4-493f-8baf-7c8c12f782f3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:46.996507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36515560-f2c4-493f-8baf-7c8c12f782f3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:47.003138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782644c8-763d-4d6c-83b2-893268795a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:47.003138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782644c8-763d-4d6c-83b2-893268795a1c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:47.003138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782644c8-763d-4d6c-83b2-893268795a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:47.003138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782644c8-763d-4d6c-83b2-893268795a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:47.003755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782644c8-763d-4d6c-83b2-893268795a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:47.005605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782644c8-763d-4d6c-83b2-893268795a1c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:47.021425 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 828/1697] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:47.022738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069f576-6b6c-4701-bc60-604d62271781 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:47.023085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069f576-6b6c-4701-bc60-604d62271781 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:47.023560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069f576-6b6c-4701-bc60-604d62271781 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:47.029632 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 829/1698] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:48:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:48:47.029632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069f576-6b6c-4701-bc60-604d62271781 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:47.050550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069f576-6b6c-4701-bc60-604d62271781 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:47.051334 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 870/1699] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:48:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:47.055417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b894fd7-5642-4597-9f7d-ff18c18c87e5 None None] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:48:47.055769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4464c1-1299-4dea-bf9b-a014bd5323e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:47.059175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4464c1-1299-4dea-bf9b-a014bd5323e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:47.059466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4464c1-1299-4dea-bf9b-a014bd5323e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:47.059750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4464c1-1299-4dea-bf9b-a014bd5323e0 admin admin] PATH_INFO: `/v3/projects/ce548a968fa74341965cfe34485d3291/users/6aa6de193f6c41b9b9a2c3fd1155b489/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:47.086834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4464c1-1299-4dea-bf9b-a014bd5323e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce548a968fa74341965cfe34485d3291, user_id=6aa6de193f6c41b9b9a2c3fd1155b489, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:47.090006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4464c1-1299-4dea-bf9b-a014bd5323e0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:47.100066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:47.105572 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 830/1700] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:47 2022] PUT /identity/v3/projects/ce548a968fa74341965cfe34485d3291/users/6aa6de193f6c41b9b9a2c3fd1155b489/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:47.106506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4464c1-1299-4dea-bf9b-a014bd5323e0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:47.109415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fb4464c1-1299-4dea-bf9b-a014bd5323e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:47.139247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b894fd7-5642-4597-9f7d-ff18c18c87e5 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:47.165424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:47.186374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4464c1-1299-4dea-bf9b-a014bd5323e0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:47.191408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccec49e-6070-4f03-934c-208fdb93c9c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:47.192266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccec49e-6070-4f03-934c-208fdb93c9c0 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:47.192549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccec49e-6070-4f03-934c-208fdb93c9c0 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:47.192823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccec49e-6070-4f03-934c-208fdb93c9c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:47.201094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b894fd7-5642-4597-9f7d-ff18c18c87e5 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:47.201213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccec49e-6070-4f03-934c-208fdb93c9c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:47.202746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccec49e-6070-4f03-934c-208fdb93c9c0 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:47.212341 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 831/1701] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:47.213228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccec49e-6070-4f03-934c-208fdb93c9c0 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:47.263360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af96ae6-4a3c-4ba8-868e-73ffef9a0023 tempest-FlavorsAccessTestJSON-1602123528 tempest-FlavorsAccessTestJSON-1602123528-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f57761e58f444548d909bc8784b80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8eaa857c05240d394707c400cf87116', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:47.264114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af96ae6-4a3c-4ba8-868e-73ffef9a0023 tempest-FlavorsAccessTestJSON-1602123528 tempest-FlavorsAccessTestJSON-1602123528-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:47.264399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af96ae6-4a3c-4ba8-868e-73ffef9a0023 tempest-FlavorsAccessTestJSON-1602123528 tempest-FlavorsAccessTestJSON-1602123528-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:47.264754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af96ae6-4a3c-4ba8-868e-73ffef9a0023 tempest-FlavorsAccessTestJSON-1602123528 tempest-FlavorsAccessTestJSON-1602123528-project-admin] PATH_INFO: `/v3/projects/8fb1cc925cc241d291826e2a79e9c2db` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:47.265888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af96ae6-4a3c-4ba8-868e-73ffef9a0023 tempest-FlavorsAccessTestJSON-1602123528 tempest-FlavorsAccessTestJSON-1602123528-project-admin] RBAC: Authorizing `identity:get_project(project_id=8fb1cc925cc241d291826e2a79e9c2db)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:47.268078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af96ae6-4a3c-4ba8-868e-73ffef9a0023 tempest-FlavorsAccessTestJSON-1602123528 tempest-FlavorsAccessTestJSON-1602123528-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:47.269871 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 871/1702] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:48:47 2022] GET /identity/v3/projects/8fb1cc925cc241d291826e2a79e9c2db => generated 379 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:47.283796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccec49e-6070-4f03-934c-208fdb93c9c0 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:47.292894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0c5e3-6bd2-46ea-9637-f03a70ee2870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:47.293600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0c5e3-6bd2-46ea-9637-f03a70ee2870 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:47.293892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0c5e3-6bd2-46ea-9637-f03a70ee2870 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:47.294202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0c5e3-6bd2-46ea-9637-f03a70ee2870 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:47.294843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d0c5e3-6bd2-46ea-9637-f03a70ee2870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:47.296674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d0c5e3-6bd2-46ea-9637-f03a70ee2870 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:47.307556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af96ae6-4a3c-4ba8-868e-73ffef9a0023 tempest-FlavorsAccessTestJSON-1602123528 tempest-FlavorsAccessTestJSON-1602123528-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:47.312822 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 832/1703] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:47.314083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0c5e3-6bd2-46ea-9637-f03a70ee2870 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:47.331782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0c5e3-6bd2-46ea-9637-f03a70ee2870 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:47.341042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d317e-5abf-4704-b95d-7c968a470e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:47.341817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d317e-5abf-4704-b95d-7c968a470e9f admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:47.342343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d317e-5abf-4704-b95d-7c968a470e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:47.342617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d317e-5abf-4704-b95d-7c968a470e9f admin admin] PATH_INFO: `/v3/projects/ce548a968fa74341965cfe34485d3291/users/6aa6de193f6c41b9b9a2c3fd1155b489/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:47.345721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d317e-5abf-4704-b95d-7c968a470e9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce548a968fa74341965cfe34485d3291, user_id=6aa6de193f6c41b9b9a2c3fd1155b489, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:47.349652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d317e-5abf-4704-b95d-7c968a470e9f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:47.366137 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 833/1704] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:47 2022] PUT /identity/v3/projects/ce548a968fa74341965cfe34485d3291/users/6aa6de193f6c41b9b9a2c3fd1155b489/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:48:47.367656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbb3a8-d693-4e45-bdbc-a592ebdf4095 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:47.368028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbb3a8-d693-4e45-bdbc-a592ebdf4095 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:47.368380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbb3a8-d693-4e45-bdbc-a592ebdf4095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:47.402850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-74bbb3a8-d693-4e45-bdbc-a592ebdf4095 None None] MFA Rules not processed for user `07638141e5f34ee783ebfe4f577be214`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:47.411280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-74bbb3a8-d693-4e45-bdbc-a592ebdf4095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:47.421884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af96ae6-4a3c-4ba8-868e-73ffef9a0023 tempest-FlavorsAccessTestJSON-1602123528 tempest-FlavorsAccessTestJSON-1602123528-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:47.434535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:47.477959 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 834/1705] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:47.477959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbb3a8-d693-4e45-bdbc-a592ebdf4095 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:47.477959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-74bbb3a8-d693-4e45-bdbc-a592ebdf4095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:47.509724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2b043-8742-42ad-8cce-6a5053f7f946 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:47.510512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2b043-8742-42ad-8cce-6a5053f7f946 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:47.510962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2b043-8742-42ad-8cce-6a5053f7f946 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:47.511857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2b043-8742-42ad-8cce-6a5053f7f946 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:47.561291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbb3a8-d693-4e45-bdbc-a592ebdf4095 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:47.567775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27e2dc-95eb-4bf7-be91-68b091612ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:47.568633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27e2dc-95eb-4bf7-be91-68b091612ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:47.569008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27e2dc-95eb-4bf7-be91-68b091612ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:47.569456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27e2dc-95eb-4bf7-be91-68b091612ddf admin admin] PATH_INFO: `/v3/users/610291b77fa747e9a04aa1abf6afb9c7` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:47.571320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27e2dc-95eb-4bf7-be91-68b091612ddf admin admin] RBAC: Authorizing `identity:delete_user(user_id=610291b77fa747e9a04aa1abf6afb9c7)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:47.573417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27e2dc-95eb-4bf7-be91-68b091612ddf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:47.578249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2b043-8742-42ad-8cce-6a5053f7f946 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:47.580094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2b043-8742-42ad-8cce-6a5053f7f946 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:47.594668 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 872/1706] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:48:47.596257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7950925-85d7-4554-931c-57a46b9364ab None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:47.596551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7950925-85d7-4554-931c-57a46b9364ab None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:47.596834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7950925-85d7-4554-931c-57a46b9364ab None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:47.598659 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 873/1707] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:48:47.607269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7950925-85d7-4554-931c-57a46b9364ab None None] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:48:47.611296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:47.633694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7950925-85d7-4554-931c-57a46b9364ab None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:47.682734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6a27e2dc-95eb-4bf7-be91-68b091612ddf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '610291b77fa747e9a04aa1abf6afb9c7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:47.684856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6a27e2dc-95eb-4bf7-be91-68b091612ddf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '610291b77fa747e9a04aa1abf6afb9c7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:47.686750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6a27e2dc-95eb-4bf7-be91-68b091612ddf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '610291b77fa747e9a04aa1abf6afb9c7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:47.699916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6a27e2dc-95eb-4bf7-be91-68b091612ddf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '610291b77fa747e9a04aa1abf6afb9c7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:47.716572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6a27e2dc-95eb-4bf7-be91-68b091612ddf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '610291b77fa747e9a04aa1abf6afb9c7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:47.746623 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 835/1708] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:48:47 2022] DELETE /identity/v3/users/610291b77fa747e9a04aa1abf6afb9c7 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:47.746623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27e2dc-95eb-4bf7-be91-68b091612ddf admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:47.747517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7950925-85d7-4554-931c-57a46b9364ab None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:47.747785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6a27e2dc-95eb-4bf7-be91-68b091612ddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:47.809138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:47.823457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a627b-7873-4af8-86f3-4acec9571e70 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e0503d20ac74f9ebe516009ddf320a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08b6602ad13646c195ab1f3be7aa0e72', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:47.825259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a627b-7873-4af8-86f3-4acec9571e70 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:47.825666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a627b-7873-4af8-86f3-4acec9571e70 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:47.825996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a627b-7873-4af8-86f3-4acec9571e70 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] PATH_INFO: `/v3/projects/2b38d7f557db4bfeacdd4e958cec9557` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:47.829914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a627b-7873-4af8-86f3-4acec9571e70 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] RBAC: Authorizing `identity:get_project(project_id=2b38d7f557db4bfeacdd4e958cec9557)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:47.832630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a627b-7873-4af8-86f3-4acec9571e70 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:47.835284 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 874/1709] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:48:47 2022] GET /identity/v3/projects/2b38d7f557db4bfeacdd4e958cec9557 => generated 395 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:47.836856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a627b-7873-4af8-86f3-4acec9571e70 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:47.841459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a02a627b-7873-4af8-86f3-4acec9571e70 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:47.843202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27e2dc-95eb-4bf7-be91-68b091612ddf admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:47.848622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024e4dd-3dcb-48d1-a4ae-2c54c94975b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:47.849372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024e4dd-3dcb-48d1-a4ae-2c54c94975b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:47.849657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024e4dd-3dcb-48d1-a4ae-2c54c94975b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:47.849984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024e4dd-3dcb-48d1-a4ae-2c54c94975b7 admin admin] PATH_INFO: `/v3/users/f921d61f3bbb49a39a5bb998f6e09012` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:47.851955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024e4dd-3dcb-48d1-a4ae-2c54c94975b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f921d61f3bbb49a39a5bb998f6e09012)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:47.854343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024e4dd-3dcb-48d1-a4ae-2c54c94975b7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:47.943062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a627b-7873-4af8-86f3-4acec9571e70 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:47.952775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2827cba-2eec-404b-81b2-277393211b3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:47.953397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2827cba-2eec-404b-81b2-277393211b3f service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:47.953856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2827cba-2eec-404b-81b2-277393211b3f service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:47.954134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2827cba-2eec-404b-81b2-277393211b3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:47.958401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c2827cba-2eec-404b-81b2-277393211b3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:47.971701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6024e4dd-3dcb-48d1-a4ae-2c54c94975b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f921d61f3bbb49a39a5bb998f6e09012'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:47.972830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6024e4dd-3dcb-48d1-a4ae-2c54c94975b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f921d61f3bbb49a39a5bb998f6e09012'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:47.974971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6024e4dd-3dcb-48d1-a4ae-2c54c94975b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f921d61f3bbb49a39a5bb998f6e09012'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:47.991242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6024e4dd-3dcb-48d1-a4ae-2c54c94975b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f921d61f3bbb49a39a5bb998f6e09012'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:48.019236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6024e4dd-3dcb-48d1-a4ae-2c54c94975b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f921d61f3bbb49a39a5bb998f6e09012'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:48.047209 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 836/1710] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:48:47 2022] DELETE /identity/v3/users/f921d61f3bbb49a39a5bb998f6e09012 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:48.048769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e44cd6-930a-46f0-8682-a9c38259796b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:48.049062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e44cd6-930a-46f0-8682-a9c38259796b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:48.049334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e44cd6-930a-46f0-8682-a9c38259796b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:48.068427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2827cba-2eec-404b-81b2-277393211b3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:48.070735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2827cba-2eec-404b-81b2-277393211b3f service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:48.078108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:48.086751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-42e44cd6-930a-46f0-8682-a9c38259796b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:48.088518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-42e44cd6-930a-46f0-8682-a9c38259796b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:48.089263 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 875/1711] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:48.091500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58aa1f4-2e67-411d-bccb-c16841083998 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:48.091975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58aa1f4-2e67-411d-bccb-c16841083998 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:48.092939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58aa1f4-2e67-411d-bccb-c16841083998 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:48.094410 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 876/1712] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:48:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:48:48.100220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17cd3c-bd9e-49a9-ad51-8d75a17bb61c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:48.100710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17cd3c-bd9e-49a9-ad51-8d75a17bb61c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:48.101184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17cd3c-bd9e-49a9-ad51-8d75a17bb61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:48.133273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ff17cd3c-bd9e-49a9-ad51-8d75a17bb61c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:48.136907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ff17cd3c-bd9e-49a9-ad51-8d75a17bb61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:48.176429 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 837/1713] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:48.186877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e44cd6-930a-46f0-8682-a9c38259796b None None] Authenticating service token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:48:48.188300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-42e44cd6-930a-46f0-8682-a9c38259796b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:48.257533 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 877/1714] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:48.258448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17cd3c-bd9e-49a9-ad51-8d75a17bb61c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:48.259930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ff17cd3c-bd9e-49a9-ad51-8d75a17bb61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:48.303778 ncdv-master-1 devstack@keystone.service[99140]: WARNING keystone.server.flask.request_processing.middleware.auth_context [None req-42e44cd6-930a-46f0-8682-a9c38259796b None None] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. Oct 01 20:48:48.304346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e44cd6-930a-46f0-8682-a9c38259796b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:48.305784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-42e44cd6-930a-46f0-8682-a9c38259796b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:48.315874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:48.344655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17cd3c-bd9e-49a9-ad51-8d75a17bb61c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:48.350185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb5331-6a84-4d6c-92ea-59c320a03c91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:48.350831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb5331-6a84-4d6c-92ea-59c320a03c91 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:48.351120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb5331-6a84-4d6c-92ea-59c320a03c91 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:48.351402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb5331-6a84-4d6c-92ea-59c320a03c91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:48.386091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e44cd6-930a-46f0-8682-a9c38259796b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:48.387116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb5331-6a84-4d6c-92ea-59c320a03c91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:48.391155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb5331-6a84-4d6c-92ea-59c320a03c91 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:48.391261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290c7de-6077-4e81-9496-85c3d0315827 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e0503d20ac74f9ebe516009ddf320a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08b6602ad13646c195ab1f3be7aa0e72', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:48.392028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290c7de-6077-4e81-9496-85c3d0315827 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:48.392326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290c7de-6077-4e81-9496-85c3d0315827 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:48.392607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290c7de-6077-4e81-9496-85c3d0315827 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] PATH_INFO: `/v3/projects/2b38d7f557db4bfeacdd4e958cec9557` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:48.393762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290c7de-6077-4e81-9496-85c3d0315827 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] RBAC: Authorizing `identity:get_project(project_id=2b38d7f557db4bfeacdd4e958cec9557)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:48.395863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290c7de-6077-4e81-9496-85c3d0315827 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:48.401058 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 838/1715] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:48:48 2022] GET /identity/v3/projects/2b38d7f557db4bfeacdd4e958cec9557 => generated 395 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:48.403202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81a5e0-29c9-4976-93b5-0775f16bf34b None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:48.403483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81a5e0-29c9-4976-93b5-0775f16bf34b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:48.403759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81a5e0-29c9-4976-93b5-0775f16bf34b None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:48.406091 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 839/1716] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:48:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:48:48.406880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81a5e0-29c9-4976-93b5-0775f16bf34b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:48.409194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1f81a5e0-29c9-4976-93b5-0775f16bf34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:48.428475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81a5e0-29c9-4976-93b5-0775f16bf34b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:48.435029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26372b-96dc-4156-9ffd-8f09a81b0a01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:48.435797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26372b-96dc-4156-9ffd-8f09a81b0a01 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:48.436146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26372b-96dc-4156-9ffd-8f09a81b0a01 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:48.436488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26372b-96dc-4156-9ffd-8f09a81b0a01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:48.442265 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 878/1717] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:48.443715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb9dd1-ecc0-4de2-aaf6-469a61c65f03 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:48.448229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26372b-96dc-4156-9ffd-8f09a81b0a01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:48.454690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26372b-96dc-4156-9ffd-8f09a81b0a01 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:48.454908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb9dd1-ecc0-4de2-aaf6-469a61c65f03 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:48.454908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb9dd1-ecc0-4de2-aaf6-469a61c65f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:48.482982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-09eb9dd1-ecc0-4de2-aaf6-469a61c65f03 None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:48.484301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-09eb9dd1-ecc0-4de2-aaf6-469a61c65f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:48.485271 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 840/1718] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:48.500654 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 879/1719] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:48:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:48.515855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26372b-96dc-4156-9ffd-8f09a81b0a01 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:48.517720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-aa26372b-96dc-4156-9ffd-8f09a81b0a01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:48.575439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:48.615993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb9dd1-ecc0-4de2-aaf6-469a61c65f03 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:48.617871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-09eb9dd1-ecc0-4de2-aaf6-469a61c65f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:48.635418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb9dd1-ecc0-4de2-aaf6-469a61c65f03 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:48.643985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb54fd-f398-4461-a4ca-1774ad003041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:48.644905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb54fd-f398-4461-a4ca-1774ad003041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:48.645725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb54fd-f398-4461-a4ca-1774ad003041 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:48.646102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb54fd-f398-4461-a4ca-1774ad003041 admin admin] PATH_INFO: `/v3/users/481eb4e6b7d744d6825977ddc547938a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:48.648309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb54fd-f398-4461-a4ca-1774ad003041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=481eb4e6b7d744d6825977ddc547938a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:48.651852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb54fd-f398-4461-a4ca-1774ad003041 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:48.724969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26372b-96dc-4156-9ffd-8f09a81b0a01 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:48.735318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04edeb-df25-4a44-a85b-43897251d2b0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:48.737876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04edeb-df25-4a44-a85b-43897251d2b0 service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:48.738163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04edeb-df25-4a44-a85b-43897251d2b0 service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:48.739135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04edeb-df25-4a44-a85b-43897251d2b0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:48.744558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77cb54fd-f398-4461-a4ca-1774ad003041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '481eb4e6b7d744d6825977ddc547938a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:48.770613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77cb54fd-f398-4461-a4ca-1774ad003041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '481eb4e6b7d744d6825977ddc547938a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:48.772428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04edeb-df25-4a44-a85b-43897251d2b0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:48.775609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04edeb-df25-4a44-a85b-43897251d2b0 service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:48.816549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77cb54fd-f398-4461-a4ca-1774ad003041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '481eb4e6b7d744d6825977ddc547938a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:48.819634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77cb54fd-f398-4461-a4ca-1774ad003041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '481eb4e6b7d744d6825977ddc547938a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:48.821121 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 841/1720] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:48.840427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77cb54fd-f398-4461-a4ca-1774ad003041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '481eb4e6b7d744d6825977ddc547938a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:48.856560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136270a9-f968-4741-9d97-078144c62f26 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:48.856850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136270a9-f968-4741-9d97-078144c62f26 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:48.857708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136270a9-f968-4741-9d97-078144c62f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:48.861822 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 880/1721] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:48:48 2022] DELETE /identity/v3/users/481eb4e6b7d744d6825977ddc547938a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:48.892414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-136270a9-f968-4741-9d97-078144c62f26 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:48.893842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-136270a9-f968-4741-9d97-078144c62f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:48.986276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb54fd-f398-4461-a4ca-1774ad003041 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:48.987852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-77cb54fd-f398-4461-a4ca-1774ad003041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:49.022098 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 842/1722] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:49.035924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136270a9-f968-4741-9d97-078144c62f26 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:49.037765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-136270a9-f968-4741-9d97-078144c62f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:49.157688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb54fd-f398-4461-a4ca-1774ad003041 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:49.169581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d215ed-66c9-4380-997d-c0756ab74e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:49.169581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d215ed-66c9-4380-997d-c0756ab74e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:49.169581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d215ed-66c9-4380-997d-c0756ab74e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:49.169581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d215ed-66c9-4380-997d-c0756ab74e3a admin admin] PATH_INFO: `/v3/users/6ba3f051cd7346779d878f1694e761ea` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:49.169908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d215ed-66c9-4380-997d-c0756ab74e3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ba3f051cd7346779d878f1694e761ea)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:49.170509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d215ed-66c9-4380-997d-c0756ab74e3a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:49.187641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136270a9-f968-4741-9d97-078144c62f26 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:49.197321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7d89e-72ad-40f8-b998-47154e83274c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:49.198094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7d89e-72ad-40f8-b998-47154e83274c service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:49.198381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7d89e-72ad-40f8-b998-47154e83274c service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:49.199240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7d89e-72ad-40f8-b998-47154e83274c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:49.220663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa7d89e-72ad-40f8-b998-47154e83274c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:49.222210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa7d89e-72ad-40f8-b998-47154e83274c service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:49.243084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:49.248118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-94d215ed-66c9-4380-997d-c0756ab74e3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ba3f051cd7346779d878f1694e761ea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:49.254698 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 843/1723] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:48:49.256481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7d89e-72ad-40f8-b998-47154e83274c service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:49.260145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-94d215ed-66c9-4380-997d-c0756ab74e3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ba3f051cd7346779d878f1694e761ea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:49.262550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-faa7d89e-72ad-40f8-b998-47154e83274c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:49.278807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-94d215ed-66c9-4380-997d-c0756ab74e3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ba3f051cd7346779d878f1694e761ea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:49.279573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-94d215ed-66c9-4380-997d-c0756ab74e3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ba3f051cd7346779d878f1694e761ea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:49.293514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-94d215ed-66c9-4380-997d-c0756ab74e3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ba3f051cd7346779d878f1694e761ea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:49.307423 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 881/1724] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:48:48 2022] DELETE /identity/v3/users/6ba3f051cd7346779d878f1694e761ea => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:48:49.310338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe3b35-ad58-4639-8e7d-a816577a1b27 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:49.310659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe3b35-ad58-4639-8e7d-a816577a1b27 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:49.311241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe3b35-ad58-4639-8e7d-a816577a1b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:49.338627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9efe3b35-ad58-4639-8e7d-a816577a1b27 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:49.340008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9efe3b35-ad58-4639-8e7d-a816577a1b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:49.403289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:49.435503 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 882/1725] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:49.438913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe3b35-ad58-4639-8e7d-a816577a1b27 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:49.438913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9efe3b35-ad58-4639-8e7d-a816577a1b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:49.448507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7d89e-72ad-40f8-b998-47154e83274c service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:49.452951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe3b35-ad58-4639-8e7d-a816577a1b27 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:49.461765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739e197-5efa-4d5c-a494-a73996dd7396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:49.462514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739e197-5efa-4d5c-a494-a73996dd7396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:49.463060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739e197-5efa-4d5c-a494-a73996dd7396 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:49.463345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739e197-5efa-4d5c-a494-a73996dd7396 admin admin] PATH_INFO: `/v3/projects/646008f76eaa4a6398c1a63f6ee13bd1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:49.464825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739e197-5efa-4d5c-a494-a73996dd7396 admin admin] RBAC: Authorizing `identity:delete_project(project_id=646008f76eaa4a6398c1a63f6ee13bd1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:49.468136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739e197-5efa-4d5c-a494-a73996dd7396 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:49.580212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a739e197-5efa-4d5c-a494-a73996dd7396 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '646008f76eaa4a6398c1a63f6ee13bd1', 'request_id': 'req-a739e197-5efa-4d5c-a494-a73996dd7396'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:49.591680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a739e197-5efa-4d5c-a494-a73996dd7396 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '646008f76eaa4a6398c1a63f6ee13bd1', 'request_id': 'req-a739e197-5efa-4d5c-a494-a73996dd7396'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:49.599172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a739e197-5efa-4d5c-a494-a73996dd7396 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '646008f76eaa4a6398c1a63f6ee13bd1', 'request_id': 'req-a739e197-5efa-4d5c-a494-a73996dd7396'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:49.631366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ca7f7-da49-4159-8479-171e9bef3fac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:49.632037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ca7f7-da49-4159-8479-171e9bef3fac service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:49.632656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ca7f7-da49-4159-8479-171e9bef3fac service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:49.633696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ca7f7-da49-4159-8479-171e9bef3fac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:49.637790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4c0ca7f7-da49-4159-8479-171e9bef3fac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:49.650699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a739e197-5efa-4d5c-a494-a73996dd7396 admin admin] The token cache is being invalidate because project 646008f76eaa4a6398c1a63f6ee13bd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:48:49.651226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a739e197-5efa-4d5c-a494-a73996dd7396 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:49.652778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a739e197-5efa-4d5c-a494-a73996dd7396 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:49.659352 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 883/1726] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:48:49 2022] DELETE /identity/v3/projects/646008f76eaa4a6398c1a63f6ee13bd1 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:49.660693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739e197-5efa-4d5c-a494-a73996dd7396 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:49.663065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a739e197-5efa-4d5c-a494-a73996dd7396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:49.734653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:49.784453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739e197-5efa-4d5c-a494-a73996dd7396 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:49.804932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fd7e1-8bb6-4dbc-bd3e-1b1822fa8524 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:49.806390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fd7e1-8bb6-4dbc-bd3e-1b1822fa8524 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:49.809443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fd7e1-8bb6-4dbc-bd3e-1b1822fa8524 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:49.809823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fd7e1-8bb6-4dbc-bd3e-1b1822fa8524 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:49.812900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5f2fd7e1-8bb6-4dbc-bd3e-1b1822fa8524 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:49.817891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ca7f7-da49-4159-8479-171e9bef3fac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:49.820070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ca7f7-da49-4159-8479-171e9bef3fac service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:49.832120 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 844/1727] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:49 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:49.833287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ca7f7-da49-4159-8479-171e9bef3fac service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:49.834795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4c0ca7f7-da49-4159-8479-171e9bef3fac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:49.852558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ca7f7-da49-4159-8479-171e9bef3fac service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:49.954549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2fd7e1-8bb6-4dbc-bd3e-1b1822fa8524 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:49.955989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2fd7e1-8bb6-4dbc-bd3e-1b1822fa8524 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:49.972282 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 884/1728] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:49.973620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fd7e1-8bb6-4dbc-bd3e-1b1822fa8524 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:49.977682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5f2fd7e1-8bb6-4dbc-bd3e-1b1822fa8524 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:49.981879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85acd3-6bd9-4a35-8dad-e15e5bb2ba9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:49.982633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85acd3-6bd9-4a35-8dad-e15e5bb2ba9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:49.982920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85acd3-6bd9-4a35-8dad-e15e5bb2ba9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:49.983182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85acd3-6bd9-4a35-8dad-e15e5bb2ba9e admin admin] PATH_INFO: `/v3/projects/94df346c1d9645d3831ede450f4da091` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:49.986077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85acd3-6bd9-4a35-8dad-e15e5bb2ba9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=94df346c1d9645d3831ede450f4da091)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:49.987148 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-ff85acd3-6bd9-4a35-8dad-e15e5bb2ba9e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:48:49.987148 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:48:49.987148 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:48:49.990554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85acd3-6bd9-4a35-8dad-e15e5bb2ba9e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:50.004290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fd7e1-8bb6-4dbc-bd3e-1b1822fa8524 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:50.014070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70584e-469c-46a3-a7b2-b703ae9ab7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:50.014701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70584e-469c-46a3-a7b2-b703ae9ab7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:50.015919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70584e-469c-46a3-a7b2-b703ae9ab7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:50.016191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70584e-469c-46a3-a7b2-b703ae9ab7ca admin admin] PATH_INFO: `/v3/users/fdce7dd18be74b80a9b5cf080e303d5e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:50.019248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70584e-469c-46a3-a7b2-b703ae9ab7ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdce7dd18be74b80a9b5cf080e303d5e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:50.021119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70584e-469c-46a3-a7b2-b703ae9ab7ca admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:50.111643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:50.147464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2b70584e-469c-46a3-a7b2-b703ae9ab7ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdce7dd18be74b80a9b5cf080e303d5e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:50.156962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2b70584e-469c-46a3-a7b2-b703ae9ab7ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdce7dd18be74b80a9b5cf080e303d5e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:50.171693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ff85acd3-6bd9-4a35-8dad-e15e5bb2ba9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94df346c1d9645d3831ede450f4da091', 'request_id': 'req-ff85acd3-6bd9-4a35-8dad-e15e5bb2ba9e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:50.186845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2b70584e-469c-46a3-a7b2-b703ae9ab7ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdce7dd18be74b80a9b5cf080e303d5e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:50.188275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2b70584e-469c-46a3-a7b2-b703ae9ab7ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdce7dd18be74b80a9b5cf080e303d5e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:50.189813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ff85acd3-6bd9-4a35-8dad-e15e5bb2ba9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94df346c1d9645d3831ede450f4da091', 'request_id': 'req-ff85acd3-6bd9-4a35-8dad-e15e5bb2ba9e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:50.202038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2b70584e-469c-46a3-a7b2-b703ae9ab7ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdce7dd18be74b80a9b5cf080e303d5e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:50.216777 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 885/1729] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:48:49 2022] DELETE /identity/v3/users/fdce7dd18be74b80a9b5cf080e303d5e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:50.216777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70584e-469c-46a3-a7b2-b703ae9ab7ca admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:50.218124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2b70584e-469c-46a3-a7b2-b703ae9ab7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:50.219912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ff85acd3-6bd9-4a35-8dad-e15e5bb2ba9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94df346c1d9645d3831ede450f4da091', 'request_id': 'req-ff85acd3-6bd9-4a35-8dad-e15e5bb2ba9e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:50.265763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ff85acd3-6bd9-4a35-8dad-e15e5bb2ba9e admin admin] The token cache is being invalidate because project 94df346c1d9645d3831ede450f4da091 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:48:50.266182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ff85acd3-6bd9-4a35-8dad-e15e5bb2ba9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:50.269130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ff85acd3-6bd9-4a35-8dad-e15e5bb2ba9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:50.273671 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 845/1730] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:48:49 2022] DELETE /identity/v3/projects/94df346c1d9645d3831ede450f4da091 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:50.274593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2c7b6-3856-4386-8b4b-51eff673fbb7 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:50.276148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2c7b6-3856-4386-8b4b-51eff673fbb7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:50.276561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2c7b6-3856-4386-8b4b-51eff673fbb7 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:50.279088 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 846/1731] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:48:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:48:50.281805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2c7b6-3856-4386-8b4b-51eff673fbb7 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:50.285667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a1b2c7b6-3856-4386-8b4b-51eff673fbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:50.374906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70584e-469c-46a3-a7b2-b703ae9ab7ca admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:50.382654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1c5df-9054-4aa9-8a62-ade60f540388 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:50.383240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1c5df-9054-4aa9-8a62-ade60f540388 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:50.383726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1c5df-9054-4aa9-8a62-ade60f540388 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:50.384861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1c5df-9054-4aa9-8a62-ade60f540388 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:50.386821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-00b1c5df-9054-4aa9-8a62-ade60f540388 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:50.399771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:50.405215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1c5df-9054-4aa9-8a62-ade60f540388 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:50.410582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1c5df-9054-4aa9-8a62-ade60f540388 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:50.439350 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 886/1732] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:50.441377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1c5df-9054-4aa9-8a62-ade60f540388 service placement] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:48:50.443410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-00b1c5df-9054-4aa9-8a62-ade60f540388 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:50.461390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2c7b6-3856-4386-8b4b-51eff673fbb7 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:50.472383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dccb93-cb92-403e-ab29-c53c5c967a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:50.474397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dccb93-cb92-403e-ab29-c53c5c967a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:50.474708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dccb93-cb92-403e-ab29-c53c5c967a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:50.476408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dccb93-cb92-403e-ab29-c53c5c967a1d admin admin] PATH_INFO: `/v3/users/8a423ddf823744069284c149f96107c9` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:50.479687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dccb93-cb92-403e-ab29-c53c5c967a1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a423ddf823744069284c149f96107c9)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:50.489208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dccb93-cb92-403e-ab29-c53c5c967a1d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:50.570531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1c5df-9054-4aa9-8a62-ade60f540388 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:50.571642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-00b1c5df-9054-4aa9-8a62-ade60f540388 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:50.608913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-12dccb93-cb92-403e-ab29-c53c5c967a1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a423ddf823744069284c149f96107c9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:50.610204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-12dccb93-cb92-403e-ab29-c53c5c967a1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a423ddf823744069284c149f96107c9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:50.611119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-12dccb93-cb92-403e-ab29-c53c5c967a1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a423ddf823744069284c149f96107c9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:50.623556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-12dccb93-cb92-403e-ab29-c53c5c967a1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a423ddf823744069284c149f96107c9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:50.658223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-12dccb93-cb92-403e-ab29-c53c5c967a1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a423ddf823744069284c149f96107c9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:50.676118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1c5df-9054-4aa9-8a62-ade60f540388 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:50.681516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211c989-65a0-4fc9-925f-b7973a72c5d7 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e0503d20ac74f9ebe516009ddf320a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08b6602ad13646c195ab1f3be7aa0e72', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:50.682217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211c989-65a0-4fc9-925f-b7973a72c5d7 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:50.682552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211c989-65a0-4fc9-925f-b7973a72c5d7 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:50.683388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211c989-65a0-4fc9-925f-b7973a72c5d7 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] PATH_INFO: `/v3/projects/2b38d7f557db4bfeacdd4e958cec9557` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:50.684566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211c989-65a0-4fc9-925f-b7973a72c5d7 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] RBAC: Authorizing `identity:get_project(project_id=2b38d7f557db4bfeacdd4e958cec9557)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:50.687164 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 847/1733] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:48:50 2022] DELETE /identity/v3/users/8a423ddf823744069284c149f96107c9 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:50.688099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dccb93-cb92-403e-ab29-c53c5c967a1d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:50.691384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-12dccb93-cb92-403e-ab29-c53c5c967a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:50.701726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211c989-65a0-4fc9-925f-b7973a72c5d7 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:50.707613 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 887/1734] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:48:50 2022] GET /identity/v3/projects/2b38d7f557db4bfeacdd4e958cec9557 => generated 395 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:50.709404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211c989-65a0-4fc9-925f-b7973a72c5d7 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:50.715529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7211c989-65a0-4fc9-925f-b7973a72c5d7 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:50.746194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211c989-65a0-4fc9-925f-b7973a72c5d7 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:50.861296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:50.910706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dccb93-cb92-403e-ab29-c53c5c967a1d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:50.917864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc91ecb-6478-4418-8bbf-f70b8273ef2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:50.918899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc91ecb-6478-4418-8bbf-f70b8273ef2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:50.919627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc91ecb-6478-4418-8bbf-f70b8273ef2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:50.920112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc91ecb-6478-4418-8bbf-f70b8273ef2b admin admin] PATH_INFO: `/v3/projects/aea1e8351c4343088c92a487903a78b6` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:50.921370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8687c-b4ff-4f4b-ae7c-f6edc8f1809f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:50.921737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc91ecb-6478-4418-8bbf-f70b8273ef2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=aea1e8351c4343088c92a487903a78b6)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:50.922090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8687c-b4ff-4f4b-ae7c-f6edc8f1809f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:50.922383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8687c-b4ff-4f4b-ae7c-f6edc8f1809f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:50.925192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc91ecb-6478-4418-8bbf-f70b8273ef2b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:50.926915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8687c-b4ff-4f4b-ae7c-f6edc8f1809f admin admin] PATH_INFO: `/v3/projects/8fb1cc925cc241d291826e2a79e9c2db` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:50.932655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8687c-b4ff-4f4b-ae7c-f6edc8f1809f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fb1cc925cc241d291826e2a79e9c2db)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:50.938436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8687c-b4ff-4f4b-ae7c-f6edc8f1809f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:51.052508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:51.075334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4ab8687c-b4ff-4f4b-ae7c-f6edc8f1809f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fb1cc925cc241d291826e2a79e9c2db', 'request_id': 'req-4ab8687c-b4ff-4f4b-ae7c-f6edc8f1809f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:51.078169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fcc91ecb-6478-4418-8bbf-f70b8273ef2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aea1e8351c4343088c92a487903a78b6', 'request_id': 'req-fcc91ecb-6478-4418-8bbf-f70b8273ef2b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:51.096924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fcc91ecb-6478-4418-8bbf-f70b8273ef2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aea1e8351c4343088c92a487903a78b6', 'request_id': 'req-fcc91ecb-6478-4418-8bbf-f70b8273ef2b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:51.105485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4ab8687c-b4ff-4f4b-ae7c-f6edc8f1809f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fb1cc925cc241d291826e2a79e9c2db', 'request_id': 'req-4ab8687c-b4ff-4f4b-ae7c-f6edc8f1809f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:51.109437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fcc91ecb-6478-4418-8bbf-f70b8273ef2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aea1e8351c4343088c92a487903a78b6', 'request_id': 'req-fcc91ecb-6478-4418-8bbf-f70b8273ef2b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:51.129565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fcc91ecb-6478-4418-8bbf-f70b8273ef2b admin admin] The token cache is being invalidate because project aea1e8351c4343088c92a487903a78b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:48:51.143066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fcc91ecb-6478-4418-8bbf-f70b8273ef2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:51.146964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fcc91ecb-6478-4418-8bbf-f70b8273ef2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:51.154980 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 848/1735] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:48:50 2022] DELETE /identity/v3/projects/aea1e8351c4343088c92a487903a78b6 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:51.154980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec685d3-c2ec-4354-a176-beaf356adbc5 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:51.154980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec685d3-c2ec-4354-a176-beaf356adbc5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:51.155276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4ab8687c-b4ff-4f4b-ae7c-f6edc8f1809f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fb1cc925cc241d291826e2a79e9c2db', 'request_id': 'req-4ab8687c-b4ff-4f4b-ae7c-f6edc8f1809f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:51.156119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec685d3-c2ec-4354-a176-beaf356adbc5 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:51.161262 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 849/1736] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:48:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:48:51.164823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec685d3-c2ec-4354-a176-beaf356adbc5 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:51.179909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6ec685d3-c2ec-4354-a176-beaf356adbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:51.186658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4ab8687c-b4ff-4f4b-ae7c-f6edc8f1809f admin admin] The token cache is being invalidate because project 8fb1cc925cc241d291826e2a79e9c2db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:48:51.186955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4ab8687c-b4ff-4f4b-ae7c-f6edc8f1809f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:51.187698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4ab8687c-b4ff-4f4b-ae7c-f6edc8f1809f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:51.189923 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 888/1737] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:48:50 2022] DELETE /identity/v3/projects/8fb1cc925cc241d291826e2a79e9c2db => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:51.190462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8687c-b4ff-4f4b-ae7c-f6edc8f1809f admin admin] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:48:51.194860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4ab8687c-b4ff-4f4b-ae7c-f6edc8f1809f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:51.400697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec685d3-c2ec-4354-a176-beaf356adbc5 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:51.418310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058a483-d83b-4ab4-8610-ed9516fdf76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:51.419058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058a483-d83b-4ab4-8610-ed9516fdf76a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:51.419334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058a483-d83b-4ab4-8610-ed9516fdf76a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:51.419601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058a483-d83b-4ab4-8610-ed9516fdf76a admin admin] PATH_INFO: `/v3/users/c7aab644bc2d4a9089d9ad72eff3e187` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:51.422009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058a483-d83b-4ab4-8610-ed9516fdf76a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7aab644bc2d4a9089d9ad72eff3e187)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:51.426344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8687c-b4ff-4f4b-ae7c-f6edc8f1809f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:51.426827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058a483-d83b-4ab4-8610-ed9516fdf76a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:51.433312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4ab8687c-b4ff-4f4b-ae7c-f6edc8f1809f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:51.521379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f058a483-d83b-4ab4-8610-ed9516fdf76a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7aab644bc2d4a9089d9ad72eff3e187'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:51.522162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f058a483-d83b-4ab4-8610-ed9516fdf76a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7aab644bc2d4a9089d9ad72eff3e187'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:51.522930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f058a483-d83b-4ab4-8610-ed9516fdf76a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7aab644bc2d4a9089d9ad72eff3e187'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:51.544738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f058a483-d83b-4ab4-8610-ed9516fdf76a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7aab644bc2d4a9089d9ad72eff3e187'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:51.569109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f058a483-d83b-4ab4-8610-ed9516fdf76a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7aab644bc2d4a9089d9ad72eff3e187'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:51.572232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:51.616751 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 850/1738] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:48:51 2022] DELETE /identity/v3/users/c7aab644bc2d4a9089d9ad72eff3e187 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:51.617849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058a483-d83b-4ab4-8610-ed9516fdf76a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:51.619410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f058a483-d83b-4ab4-8610-ed9516fdf76a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:51.623679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8687c-b4ff-4f4b-ae7c-f6edc8f1809f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:51.739603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058a483-d83b-4ab4-8610-ed9516fdf76a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:51.751382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de05bc-6b3d-4d05-aad6-7669b2b31f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:51.752784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de05bc-6b3d-4d05-aad6-7669b2b31f73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:51.754408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de05bc-6b3d-4d05-aad6-7669b2b31f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:51.754732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de05bc-6b3d-4d05-aad6-7669b2b31f73 admin admin] PATH_INFO: `/v3/users/7f57761e58f444548d909bc8784b80d5` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:51.757256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de05bc-6b3d-4d05-aad6-7669b2b31f73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f57761e58f444548d909bc8784b80d5)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:51.759542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a2dab-a365-461d-acf0-d00c1287cdd9 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e0503d20ac74f9ebe516009ddf320a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08b6602ad13646c195ab1f3be7aa0e72', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:51.762966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a2dab-a365-461d-acf0-d00c1287cdd9 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:51.763262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a2dab-a365-461d-acf0-d00c1287cdd9 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:51.763543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a2dab-a365-461d-acf0-d00c1287cdd9 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] PATH_INFO: `/v3/projects/2b38d7f557db4bfeacdd4e958cec9557` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:51.764343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de05bc-6b3d-4d05-aad6-7669b2b31f73 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:51.766056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a2dab-a365-461d-acf0-d00c1287cdd9 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] RBAC: Authorizing `identity:get_project(project_id=2b38d7f557db4bfeacdd4e958cec9557)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:51.770771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a2dab-a365-461d-acf0-d00c1287cdd9 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:51.776170 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 889/1739] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:48:51 2022] GET /identity/v3/projects/2b38d7f557db4bfeacdd4e958cec9557 => generated 395 bytes in 585 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:51.777825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a2dab-a365-461d-acf0-d00c1287cdd9 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:51.784388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ec8a2dab-a365-461d-acf0-d00c1287cdd9 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:51.827713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a2dab-a365-461d-acf0-d00c1287cdd9 tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:51.848197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4db96-3085-4d26-bdba-11f49d33f8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:51.848851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4db96-3085-4d26-bdba-11f49d33f8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:51.849168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4db96-3085-4d26-bdba-11f49d33f8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:51.849758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4db96-3085-4d26-bdba-11f49d33f8dc admin admin] PATH_INFO: `/v3/projects/c24c2cfdf0d94150a5b849f4c0f2c052` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:51.882264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4db96-3085-4d26-bdba-11f49d33f8dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c24c2cfdf0d94150a5b849f4c0f2c052)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:51.889358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4db96-3085-4d26-bdba-11f49d33f8dc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:51.900450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:51.930073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f2de05bc-6b3d-4d05-aad6-7669b2b31f73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f57761e58f444548d909bc8784b80d5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:51.932285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f2de05bc-6b3d-4d05-aad6-7669b2b31f73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f57761e58f444548d909bc8784b80d5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:51.933598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f2de05bc-6b3d-4d05-aad6-7669b2b31f73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f57761e58f444548d909bc8784b80d5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:51.952078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f2de05bc-6b3d-4d05-aad6-7669b2b31f73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f57761e58f444548d909bc8784b80d5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:51.977028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f2de05bc-6b3d-4d05-aad6-7669b2b31f73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f57761e58f444548d909bc8784b80d5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:52.026374 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 851/1740] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:48:51 2022] DELETE /identity/v3/users/7f57761e58f444548d909bc8784b80d5 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:52.028574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de05bc-6b3d-4d05-aad6-7669b2b31f73 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:52.030836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f2de05bc-6b3d-4d05-aad6-7669b2b31f73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:52.030953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-89a4db96-3085-4d26-bdba-11f49d33f8dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c24c2cfdf0d94150a5b849f4c0f2c052', 'request_id': 'req-89a4db96-3085-4d26-bdba-11f49d33f8dc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:52.054605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-89a4db96-3085-4d26-bdba-11f49d33f8dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c24c2cfdf0d94150a5b849f4c0f2c052', 'request_id': 'req-89a4db96-3085-4d26-bdba-11f49d33f8dc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:52.083839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-89a4db96-3085-4d26-bdba-11f49d33f8dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c24c2cfdf0d94150a5b849f4c0f2c052', 'request_id': 'req-89a4db96-3085-4d26-bdba-11f49d33f8dc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:52.135611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-89a4db96-3085-4d26-bdba-11f49d33f8dc admin admin] The token cache is being invalidate because project c24c2cfdf0d94150a5b849f4c0f2c052 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:48:52.136673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-89a4db96-3085-4d26-bdba-11f49d33f8dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:52.139505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-89a4db96-3085-4d26-bdba-11f49d33f8dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:52.143830 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 890/1741] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:48:51 2022] DELETE /identity/v3/projects/c24c2cfdf0d94150a5b849f4c0f2c052 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:52.145216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e5452-f561-4bdc-a6a5-a9fbc94084ee None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:52.145500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e5452-f561-4bdc-a6a5-a9fbc94084ee None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:52.145779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e5452-f561-4bdc-a6a5-a9fbc94084ee None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:52.164718 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 891/1742] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:48:52 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:48:52.168613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd9d0d-5506-4312-aa82-89db90e59f66 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:52.169086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd9d0d-5506-4312-aa82-89db90e59f66 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:52.171090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd9d0d-5506-4312-aa82-89db90e59f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:52.214605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de05bc-6b3d-4d05-aad6-7669b2b31f73 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:52.220801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8db4f9-7133-48ee-87f2-a5af7e803802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:52.221654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8db4f9-7133-48ee-87f2-a5af7e803802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:52.222014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8db4f9-7133-48ee-87f2-a5af7e803802 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:52.222338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8db4f9-7133-48ee-87f2-a5af7e803802 admin admin] PATH_INFO: `/v3/projects/c1bdf2dd709e4d44b587f35be0b0f43f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:52.223747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8db4f9-7133-48ee-87f2-a5af7e803802 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1bdf2dd709e4d44b587f35be0b0f43f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:52.228166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8db4f9-7133-48ee-87f2-a5af7e803802 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:52.236145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0abd9d0d-5506-4312-aa82-89db90e59f66 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:52.237500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0abd9d0d-5506-4312-aa82-89db90e59f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:52.255418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:52.271568 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 892/1743] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:52.273371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd9d0d-5506-4312-aa82-89db90e59f66 None None] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:48:52.279997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0abd9d0d-5506-4312-aa82-89db90e59f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:52.339479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ed8db4f9-7133-48ee-87f2-a5af7e803802 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1bdf2dd709e4d44b587f35be0b0f43f', 'request_id': 'req-ed8db4f9-7133-48ee-87f2-a5af7e803802'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:52.389705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ed8db4f9-7133-48ee-87f2-a5af7e803802 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1bdf2dd709e4d44b587f35be0b0f43f', 'request_id': 'req-ed8db4f9-7133-48ee-87f2-a5af7e803802'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:52.410364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ed8db4f9-7133-48ee-87f2-a5af7e803802 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1bdf2dd709e4d44b587f35be0b0f43f', 'request_id': 'req-ed8db4f9-7133-48ee-87f2-a5af7e803802'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:52.436539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ed8db4f9-7133-48ee-87f2-a5af7e803802 admin admin] The token cache is being invalidate because project c1bdf2dd709e4d44b587f35be0b0f43f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:48:52.437125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ed8db4f9-7133-48ee-87f2-a5af7e803802 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:52.439783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ed8db4f9-7133-48ee-87f2-a5af7e803802 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:52.446194 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 852/1744] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:48:52 2022] DELETE /identity/v3/projects/c1bdf2dd709e4d44b587f35be0b0f43f => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:52.446194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747d376-1785-492f-aaf4-21e8f2afebf3 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:52.446194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747d376-1785-492f-aaf4-21e8f2afebf3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:52.446655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747d376-1785-492f-aaf4-21e8f2afebf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:52.487997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9747d376-1785-492f-aaf4-21e8f2afebf3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:52.489700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9747d376-1785-492f-aaf4-21e8f2afebf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:52.562375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd9d0d-5506-4312-aa82-89db90e59f66 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:52.566182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0abd9d0d-5506-4312-aa82-89db90e59f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:52.697338 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 853/1745] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:52.697338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747d376-1785-492f-aaf4-21e8f2afebf3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:52.697886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9747d376-1785-492f-aaf4-21e8f2afebf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:52.720009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747d376-1785-492f-aaf4-21e8f2afebf3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:52.739968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf07d3-7f7b-48eb-a2ab-bff3a1afa037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:52.740642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf07d3-7f7b-48eb-a2ab-bff3a1afa037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:52.743545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf07d3-7f7b-48eb-a2ab-bff3a1afa037 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:52.744089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf07d3-7f7b-48eb-a2ab-bff3a1afa037 admin admin] PATH_INFO: `/v3/projects/f063d1149d2f49d488112e2765440f3b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:52.746218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf07d3-7f7b-48eb-a2ab-bff3a1afa037 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f063d1149d2f49d488112e2765440f3b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:52.753395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf07d3-7f7b-48eb-a2ab-bff3a1afa037 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:52.885441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd9d0d-5506-4312-aa82-89db90e59f66 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:52.887264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-abcf07d3-7f7b-48eb-a2ab-bff3a1afa037 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f063d1149d2f49d488112e2765440f3b', 'request_id': 'req-abcf07d3-7f7b-48eb-a2ab-bff3a1afa037'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:52.913638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-abcf07d3-7f7b-48eb-a2ab-bff3a1afa037 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f063d1149d2f49d488112e2765440f3b', 'request_id': 'req-abcf07d3-7f7b-48eb-a2ab-bff3a1afa037'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:52.932039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-abcf07d3-7f7b-48eb-a2ab-bff3a1afa037 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f063d1149d2f49d488112e2765440f3b', 'request_id': 'req-abcf07d3-7f7b-48eb-a2ab-bff3a1afa037'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:52.958802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:52.963787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-abcf07d3-7f7b-48eb-a2ab-bff3a1afa037 admin admin] The token cache is being invalidate because project f063d1149d2f49d488112e2765440f3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:48:52.964894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-abcf07d3-7f7b-48eb-a2ab-bff3a1afa037 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:52.966798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-abcf07d3-7f7b-48eb-a2ab-bff3a1afa037 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:52.970972 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 854/1746] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:48:52 2022] DELETE /identity/v3/projects/f063d1149d2f49d488112e2765440f3b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:52.978636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022688ad-871b-4915-8bea-4d9e37edbf24 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:52.979237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022688ad-871b-4915-8bea-4d9e37edbf24 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:52.980512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022688ad-871b-4915-8bea-4d9e37edbf24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:53.039578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-022688ad-871b-4915-8bea-4d9e37edbf24 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:53.044356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-022688ad-871b-4915-8bea-4d9e37edbf24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:53.078761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0727c76-8348-4d3a-a4d2-9733614c618e tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e0503d20ac74f9ebe516009ddf320a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08b6602ad13646c195ab1f3be7aa0e72', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:53.079432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0727c76-8348-4d3a-a4d2-9733614c618e tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:53.079721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0727c76-8348-4d3a-a4d2-9733614c618e tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:53.080007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0727c76-8348-4d3a-a4d2-9733614c618e tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] PATH_INFO: `/v3/projects/02be8fe6f40b42a684054e8a3a7ebf9e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:53.081196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0727c76-8348-4d3a-a4d2-9733614c618e tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] RBAC: Authorizing `identity:get_project(project_id=02be8fe6f40b42a684054e8a3a7ebf9e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:53.092335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0727c76-8348-4d3a-a4d2-9733614c618e tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:53.098588 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 893/1747] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:48:52 2022] GET /identity/v3/projects/02be8fe6f40b42a684054e8a3a7ebf9e => generated 395 bytes in 826 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:53.100620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0727c76-8348-4d3a-a4d2-9733614c618e tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:53.101770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:53.105212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c0727c76-8348-4d3a-a4d2-9733614c618e tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:53.256290 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 855/1748] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:53.260107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc0740-5f9c-47bd-b28f-d6a9a3538ab6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:53.263225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc0740-5f9c-47bd-b28f-d6a9a3538ab6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:53.263516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc0740-5f9c-47bd-b28f-d6a9a3538ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:53.310557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-86dc0740-5f9c-47bd-b28f-d6a9a3538ab6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:53.314433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-86dc0740-5f9c-47bd-b28f-d6a9a3538ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:53.354662 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 856/1749] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:53.356379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f869626-6896-41e5-a14e-d869c86e2502 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:53.356672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f869626-6896-41e5-a14e-d869c86e2502 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:53.357017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f869626-6896-41e5-a14e-d869c86e2502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:53.389554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9f869626-6896-41e5-a14e-d869c86e2502 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:53.390870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9f869626-6896-41e5-a14e-d869c86e2502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:53.391549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0727c76-8348-4d3a-a4d2-9733614c618e tempest-FlavorsAccessNegativeTestJSON-1821884549 tempest-FlavorsAccessNegativeTestJSON-1821884549-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:53.407758 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 857/1750] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:53.407758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f869626-6896-41e5-a14e-d869c86e2502 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:53.410067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9f869626-6896-41e5-a14e-d869c86e2502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:53.413518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a96f8-5280-4013-ad70-743c65ce5eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:53.414444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a96f8-5280-4013-ad70-743c65ce5eac admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:53.414988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a96f8-5280-4013-ad70-743c65ce5eac admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:53.415407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a96f8-5280-4013-ad70-743c65ce5eac admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:53.417372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a96f8-5280-4013-ad70-743c65ce5eac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:48:53.418355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a96f8-5280-4013-ad70-743c65ce5eac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:53.420419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a96f8-5280-4013-ad70-743c65ce5eac admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:53.426366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f869626-6896-41e5-a14e-d869c86e2502 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:53.434353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8417029-2f6f-45af-b74d-471e1b8bec55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:53.435223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8417029-2f6f-45af-b74d-471e1b8bec55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:53.437003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8417029-2f6f-45af-b74d-471e1b8bec55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:53.437696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8417029-2f6f-45af-b74d-471e1b8bec55 admin admin] PATH_INFO: `/v3/users/d658c70488c74cedbee2bafdde3fe047` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:53.444092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8417029-2f6f-45af-b74d-471e1b8bec55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d658c70488c74cedbee2bafdde3fe047)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:53.444844 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 894/1751] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:53.446103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a96f8-5280-4013-ad70-743c65ce5eac admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:53.447534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8417029-2f6f-45af-b74d-471e1b8bec55 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:53.450451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2c4a96f8-5280-4013-ad70-743c65ce5eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:53.476933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a96f8-5280-4013-ad70-743c65ce5eac admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:53.500648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87a237-828a-4b0c-9b82-2a9d165fb906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:53.502540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87a237-828a-4b0c-9b82-2a9d165fb906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:53.503021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87a237-828a-4b0c-9b82-2a9d165fb906 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:53.503521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87a237-828a-4b0c-9b82-2a9d165fb906 admin admin] PATH_INFO: `/v3/projects/c8eaa857c05240d394707c400cf87116` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:53.505036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87a237-828a-4b0c-9b82-2a9d165fb906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8eaa857c05240d394707c400cf87116)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:53.507295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87a237-828a-4b0c-9b82-2a9d165fb906 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:53.588353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c8417029-2f6f-45af-b74d-471e1b8bec55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd658c70488c74cedbee2bafdde3fe047'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:53.591692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c8417029-2f6f-45af-b74d-471e1b8bec55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd658c70488c74cedbee2bafdde3fe047'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:53.593452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c8417029-2f6f-45af-b74d-471e1b8bec55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd658c70488c74cedbee2bafdde3fe047'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:53.617904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c8417029-2f6f-45af-b74d-471e1b8bec55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd658c70488c74cedbee2bafdde3fe047'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:53.645899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c8417029-2f6f-45af-b74d-471e1b8bec55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd658c70488c74cedbee2bafdde3fe047'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:53.666760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dc87a237-828a-4b0c-9b82-2a9d165fb906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8eaa857c05240d394707c400cf87116', 'request_id': 'req-dc87a237-828a-4b0c-9b82-2a9d165fb906'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:53.679138 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 858/1752] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:48:53 2022] DELETE /identity/v3/users/d658c70488c74cedbee2bafdde3fe047 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:53.680885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacb9b7-ad17-4d72-8399-3835ce5f76c8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:53.681097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacb9b7-ad17-4d72-8399-3835ce5f76c8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:53.681188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dc87a237-828a-4b0c-9b82-2a9d165fb906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8eaa857c05240d394707c400cf87116', 'request_id': 'req-dc87a237-828a-4b0c-9b82-2a9d165fb906'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:53.681675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacb9b7-ad17-4d72-8399-3835ce5f76c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:53.690132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dc87a237-828a-4b0c-9b82-2a9d165fb906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8eaa857c05240d394707c400cf87116', 'request_id': 'req-dc87a237-828a-4b0c-9b82-2a9d165fb906'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:53.712093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dc87a237-828a-4b0c-9b82-2a9d165fb906 admin admin] The token cache is being invalidate because project c8eaa857c05240d394707c400cf87116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:48:53.712385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dc87a237-828a-4b0c-9b82-2a9d165fb906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:53.713126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dc87a237-828a-4b0c-9b82-2a9d165fb906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:53.715998 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 895/1753] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:48:53 2022] DELETE /identity/v3/projects/c8eaa857c05240d394707c400cf87116 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:53.722392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87a237-828a-4b0c-9b82-2a9d165fb906 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:53.724124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-dc87a237-828a-4b0c-9b82-2a9d165fb906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:53.728808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bdacb9b7-ad17-4d72-8399-3835ce5f76c8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:53.730321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bdacb9b7-ad17-4d72-8399-3835ce5f76c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:53.879166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:53.912372 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 859/1754] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:53.913327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdacb9b7-ad17-4d72-8399-3835ce5f76c8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:53.926068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bdacb9b7-ad17-4d72-8399-3835ce5f76c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:53.952378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87a237-828a-4b0c-9b82-2a9d165fb906 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:53.959931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdacb9b7-ad17-4d72-8399-3835ce5f76c8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:53.973593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab615e4e-cc05-48e4-a2a5-ee6b58f46ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:53.974240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab615e4e-cc05-48e4-a2a5-ee6b58f46ebf admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:53.974540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab615e4e-cc05-48e4-a2a5-ee6b58f46ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:53.974816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab615e4e-cc05-48e4-a2a5-ee6b58f46ebf admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:53.975477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab615e4e-cc05-48e4-a2a5-ee6b58f46ebf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:48:53.975787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab615e4e-cc05-48e4-a2a5-ee6b58f46ebf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:53.977137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e45c2-18b6-4ce7-b465-41a8f8712379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:53.977420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab615e4e-cc05-48e4-a2a5-ee6b58f46ebf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:53.978186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e45c2-18b6-4ce7-b465-41a8f8712379 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:53.978690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e45c2-18b6-4ce7-b465-41a8f8712379 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:53.979176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e45c2-18b6-4ce7-b465-41a8f8712379 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:53.980581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e45c2-18b6-4ce7-b465-41a8f8712379 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:53.982768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e45c2-18b6-4ce7-b465-41a8f8712379 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:54.039580 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 896/1755] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:54.039580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df796ad-aeb0-4997-8812-627968ea2578 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:54.039580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df796ad-aeb0-4997-8812-627968ea2578 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:54.045755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df796ad-aeb0-4997-8812-627968ea2578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:54.065730 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 860/1756] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:53 2022] POST /identity/v3/projects => generated 375 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:54.070051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934eca0b-f571-4671-a47b-6c6c2c7ba5f6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:54.070641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934eca0b-f571-4671-a47b-6c6c2c7ba5f6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:54.071549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934eca0b-f571-4671-a47b-6c6c2c7ba5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:54.090836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1df796ad-aeb0-4997-8812-627968ea2578 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:54.092093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1df796ad-aeb0-4997-8812-627968ea2578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:54.093213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:54.146522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-934eca0b-f571-4671-a47b-6c6c2c7ba5f6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:54.160167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-934eca0b-f571-4671-a47b-6c6c2c7ba5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:54.281901 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 897/1757] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:54.281901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df796ad-aeb0-4997-8812-627968ea2578 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:54.297538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df796ad-aeb0-4997-8812-627968ea2578 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:54.309955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3e4ed-1481-4b87-b67c-5a38d6a3f010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:54.310777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3e4ed-1481-4b87-b67c-5a38d6a3f010 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:54.310859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3e4ed-1481-4b87-b67c-5a38d6a3f010 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:54.311127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3e4ed-1481-4b87-b67c-5a38d6a3f010 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:54.311784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3e4ed-1481-4b87-b67c-5a38d6a3f010 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:48:54.315320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3e4ed-1481-4b87-b67c-5a38d6a3f010 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:54.320217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3e4ed-1481-4b87-b67c-5a38d6a3f010 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:54.331524 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 861/1758] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:54.332729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934eca0b-f571-4671-a47b-6c6c2c7ba5f6 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:54.346941 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 898/1759] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:54.348137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3e4ed-1481-4b87-b67c-5a38d6a3f010 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:54.360480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934eca0b-f571-4671-a47b-6c6c2c7ba5f6 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:54.362522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3e4ed-1481-4b87-b67c-5a38d6a3f010 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:54.368753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6083293-26c4-4300-a319-f7710fd22d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:54.369636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6083293-26c4-4300-a319-f7710fd22d82 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:54.370083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6083293-26c4-4300-a319-f7710fd22d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:54.370698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6083293-26c4-4300-a319-f7710fd22d82 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:54.371448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6083293-26c4-4300-a319-f7710fd22d82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:54.374606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6083293-26c4-4300-a319-f7710fd22d82 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:54.377265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca75b0-8cc2-44db-87eb-de0d765df438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:54.377892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca75b0-8cc2-44db-87eb-de0d765df438 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:54.378168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca75b0-8cc2-44db-87eb-de0d765df438 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:54.378615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca75b0-8cc2-44db-87eb-de0d765df438 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:54.379262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca75b0-8cc2-44db-87eb-de0d765df438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:54.381314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca75b0-8cc2-44db-87eb-de0d765df438 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:54.411222 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 899/1760] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:54 2022] POST /identity/v3/users => generated 288 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:54.412733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a099b44-17b1-41af-b31a-8263b85763df None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:54.413670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a099b44-17b1-41af-b31a-8263b85763df None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:54.413670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a099b44-17b1-41af-b31a-8263b85763df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:54.462439 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 862/1761] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:54 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:54.469906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca75b0-8cc2-44db-87eb-de0d765df438 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:54.473868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a7ca75b0-8cc2-44db-87eb-de0d765df438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:54.506711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8a099b44-17b1-41af-b31a-8263b85763df None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:54.509787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8a099b44-17b1-41af-b31a-8263b85763df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:54.625982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca75b0-8cc2-44db-87eb-de0d765df438 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:54.637181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24035b4f-5882-4062-9716-1d1392619eaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:54.637827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24035b4f-5882-4062-9716-1d1392619eaa service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:54.638103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24035b4f-5882-4062-9716-1d1392619eaa service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:54.638373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24035b4f-5882-4062-9716-1d1392619eaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:54.733279 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 900/1762] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:48:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:54.735955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788a586-a910-40be-9644-4de7ef52f001 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:54.736456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788a586-a910-40be-9644-4de7ef52f001 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:54.736868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788a586-a910-40be-9644-4de7ef52f001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:54.772369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b788a586-a910-40be-9644-4de7ef52f001 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:54.775028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b788a586-a910-40be-9644-4de7ef52f001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:54.800763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:54.829094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24035b4f-5882-4062-9716-1d1392619eaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:54.832258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24035b4f-5882-4062-9716-1d1392619eaa service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:54.859645 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 863/1763] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:54.860718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24035b4f-5882-4062-9716-1d1392619eaa service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:54.882059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24035b4f-5882-4062-9716-1d1392619eaa service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:54.891914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaba132-c472-48c1-a505-f486478d583f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:54.893746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaba132-c472-48c1-a505-f486478d583f admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:54.894836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaba132-c472-48c1-a505-f486478d583f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:54.894920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaba132-c472-48c1-a505-f486478d583f admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:54.895624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaba132-c472-48c1-a505-f486478d583f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:54.902232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaba132-c472-48c1-a505-f486478d583f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:54.909333 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 901/1764] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:54.911010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788a586-a910-40be-9644-4de7ef52f001 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:54.926220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788a586-a910-40be-9644-4de7ef52f001 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:54.945092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77706b0f-f25c-4794-a1a0-a4cbe4f928c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:54.945841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77706b0f-f25c-4794-a1a0-a4cbe4f928c5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:54.946025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77706b0f-f25c-4794-a1a0-a4cbe4f928c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:54.946265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77706b0f-f25c-4794-a1a0-a4cbe4f928c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:54.946985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77706b0f-f25c-4794-a1a0-a4cbe4f928c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:54.949133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77706b0f-f25c-4794-a1a0-a4cbe4f928c5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:54.966862 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 864/1765] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:54 2022] POST /identity/v3/projects => generated 389 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:54.967993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaba132-c472-48c1-a505-f486478d583f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:54.971418 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 902/1766] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:54.972812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d8ce5-36bf-43d2-9258-25fc300488dd None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:54.973109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d8ce5-36bf-43d2-9258-25fc300488dd None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:54.973823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d8ce5-36bf-43d2-9258-25fc300488dd None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:54.975485 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 903/1767] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:48:54.976118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d8ce5-36bf-43d2-9258-25fc300488dd None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:55.003269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:55.158618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaba132-c472-48c1-a505-f486478d583f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:55.163372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d8ce5-36bf-43d2-9258-25fc300488dd None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:55.173887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac536b-71d7-4dfe-9fe4-bd02220cb08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:55.174821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac536b-71d7-4dfe-9fe4-bd02220cb08f admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:55.174980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d93bcb-e663-44ba-b210-6c02442885f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:55.175444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac536b-71d7-4dfe-9fe4-bd02220cb08f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:55.175885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac536b-71d7-4dfe-9fe4-bd02220cb08f admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:55.176986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d93bcb-e663-44ba-b210-6c02442885f7 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:55.177220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac536b-71d7-4dfe-9fe4-bd02220cb08f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:55.177320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d93bcb-e663-44ba-b210-6c02442885f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:55.177533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d93bcb-e663-44ba-b210-6c02442885f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:55.181937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d93bcb-e663-44ba-b210-6c02442885f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:48:55.182270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d93bcb-e663-44ba-b210-6c02442885f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:55.182741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac536b-71d7-4dfe-9fe4-bd02220cb08f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:55.184266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d93bcb-e663-44ba-b210-6c02442885f7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:55.216952 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 904/1768] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:48:55.220461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d93bcb-e663-44ba-b210-6c02442885f7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:55.234168 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 865/1769] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:54 2022] POST /identity/v3/users => generated 294 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:55.235460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac536b-71d7-4dfe-9fe4-bd02220cb08f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:55.239136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d93bcb-e663-44ba-b210-6c02442885f7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:55.245797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89117429-a707-4eba-a10a-0d48b4c79770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:55.246846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89117429-a707-4eba-a10a-0d48b4c79770 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:55.247290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89117429-a707-4eba-a10a-0d48b4c79770 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:55.247913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89117429-a707-4eba-a10a-0d48b4c79770 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:55.249223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89117429-a707-4eba-a10a-0d48b4c79770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:55.251508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89117429-a707-4eba-a10a-0d48b4c79770 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:55.256773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac536b-71d7-4dfe-9fe4-bd02220cb08f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:55.272248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25cb6-8cb3-43b4-8681-b3f47c7bc5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:55.272981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25cb6-8cb3-43b4-8681-b3f47c7bc5e0 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:55.273334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25cb6-8cb3-43b4-8681-b3f47c7bc5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:55.273682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25cb6-8cb3-43b4-8681-b3f47c7bc5e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:55.274569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd25cb6-8cb3-43b4-8681-b3f47c7bc5e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:55.277003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd25cb6-8cb3-43b4-8681-b3f47c7bc5e0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:55.288148 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 905/1770] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:55 2022] POST /identity/v3/users => generated 295 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:55.291992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85de4e4-1050-4785-af27-a2380a3af972 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:55.296520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85de4e4-1050-4785-af27-a2380a3af972 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:55.298572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85de4e4-1050-4785-af27-a2380a3af972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:55.308545 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 866/1771] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:55.312658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25cb6-8cb3-43b4-8681-b3f47c7bc5e0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:55.321590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5cd25cb6-8cb3-43b4-8681-b3f47c7bc5e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:55.365467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f85de4e4-1050-4785-af27-a2380a3af972 None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:55.368075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f85de4e4-1050-4785-af27-a2380a3af972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:55.482426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25cb6-8cb3-43b4-8681-b3f47c7bc5e0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:55.491875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004da57-0f2b-4e25-9e49-fa491320b6b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:55.493000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004da57-0f2b-4e25-9e49-fa491320b6b4 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:55.494158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004da57-0f2b-4e25-9e49-fa491320b6b4 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:55.494808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004da57-0f2b-4e25-9e49-fa491320b6b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:55.579662 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 906/1772] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:48:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:55.581021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85de4e4-1050-4785-af27-a2380a3af972 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:55.599498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85de4e4-1050-4785-af27-a2380a3af972 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:55.604037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004da57-0f2b-4e25-9e49-fa491320b6b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:55.606152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1593b8-e4ba-4a72-851b-4cea0fcdda2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:55.606964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1593b8-e4ba-4a72-851b-4cea0fcdda2a admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:55.607410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1593b8-e4ba-4a72-851b-4cea0fcdda2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:55.607810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1593b8-e4ba-4a72-851b-4cea0fcdda2a admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:55.608726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1593b8-e4ba-4a72-851b-4cea0fcdda2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:55.608972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004da57-0f2b-4e25-9e49-fa491320b6b4 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:55.611354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1593b8-e4ba-4a72-851b-4cea0fcdda2a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:55.647289 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 867/1773] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:48:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 20:48:55.648510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004da57-0f2b-4e25-9e49-fa491320b6b4 service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:55.648620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:55.675207 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 907/1774] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:55 2022] POST /identity/v3/projects => generated 391 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:55.676796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1593b8-e4ba-4a72-851b-4cea0fcdda2a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:55.705445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004da57-0f2b-4e25-9e49-fa491320b6b4 service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:55.720264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:55.835992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7212cd-7800-4167-a813-33324fb42bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:55.836849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7212cd-7800-4167-a813-33324fb42bfe admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:55.837719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7212cd-7800-4167-a813-33324fb42bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:55.837719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7212cd-7800-4167-a813-33324fb42bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:55.838275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7212cd-7800-4167-a813-33324fb42bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:55.841407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7212cd-7800-4167-a813-33324fb42bfe admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:55.891564 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 868/1775] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:55.891755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7212cd-7800-4167-a813-33324fb42bfe admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:55.927252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7212cd-7800-4167-a813-33324fb42bfe admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:55.944372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c679a2-898a-4d71-bf1a-a0ca211b2205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:55.945267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c679a2-898a-4d71-bf1a-a0ca211b2205 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:55.945699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c679a2-898a-4d71-bf1a-a0ca211b2205 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:55.945986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c679a2-898a-4d71-bf1a-a0ca211b2205 admin admin] PATH_INFO: `/v3/projects/598c3f5d2363441fb1a2fc8f2589544f/users/d6d3ae72992a48709e3d58930ccd6b52/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:55.968756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1593b8-e4ba-4a72-851b-4cea0fcdda2a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:55.983676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c679a2-898a-4d71-bf1a-a0ca211b2205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=598c3f5d2363441fb1a2fc8f2589544f, user_id=d6d3ae72992a48709e3d58930ccd6b52, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:55.988939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9d6eb-51f5-424d-be5c-dd095a0ddbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:55.989357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c679a2-898a-4d71-bf1a-a0ca211b2205 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:55.993758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9d6eb-51f5-424d-be5c-dd095a0ddbb5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:55.996662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9d6eb-51f5-424d-be5c-dd095a0ddbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:55.997243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9d6eb-51f5-424d-be5c-dd095a0ddbb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:55.998474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9d6eb-51f5-424d-be5c-dd095a0ddbb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:56.000502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9d6eb-51f5-424d-be5c-dd095a0ddbb5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:56.028661 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 869/1776] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:55 2022] PUT /identity/v3/projects/598c3f5d2363441fb1a2fc8f2589544f/users/d6d3ae72992a48709e3d58930ccd6b52/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:56.031082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c679a2-898a-4d71-bf1a-a0ca211b2205 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:56.048045 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 908/1777] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:56.050256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9d6eb-51f5-424d-be5c-dd095a0ddbb5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:56.075285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9d6eb-51f5-424d-be5c-dd095a0ddbb5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:56.188683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c679a2-898a-4d71-bf1a-a0ca211b2205 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:56.211140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17da0d-3726-416e-a4ad-e5f64c1cedb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:56.214748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17da0d-3726-416e-a4ad-e5f64c1cedb3 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:56.216580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17da0d-3726-416e-a4ad-e5f64c1cedb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:56.217815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17da0d-3726-416e-a4ad-e5f64c1cedb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:56.219506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7f7d6-e324-4e20-85db-12bdad151559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:56.220752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7f7d6-e324-4e20-85db-12bdad151559 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:56.221163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7f7d6-e324-4e20-85db-12bdad151559 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:56.221570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7f7d6-e324-4e20-85db-12bdad151559 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:56.222466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7f7d6-e324-4e20-85db-12bdad151559 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:56.225574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7f7d6-e324-4e20-85db-12bdad151559 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:56.281402 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 909/1778] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:56 2022] POST /identity/v3/users => generated 296 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:56.299277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7f7d6-e324-4e20-85db-12bdad151559 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:56.345816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7f7d6-e324-4e20-85db-12bdad151559 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:56.351437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b125ea12-6d3b-436b-9b23-958418c7cd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:56.363840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125ea12-6d3b-436b-9b23-958418c7cd7d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:56.363840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125ea12-6d3b-436b-9b23-958418c7cd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:56.374622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125ea12-6d3b-436b-9b23-958418c7cd7d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:56.375488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b125ea12-6d3b-436b-9b23-958418c7cd7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:56.377191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b125ea12-6d3b-436b-9b23-958418c7cd7d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:56.404356 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 910/1779] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:56.405827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506dd7a-3d4c-46e1-a62d-a0b59aecf599 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:56.406114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506dd7a-3d4c-46e1-a62d-a0b59aecf599 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:56.406392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506dd7a-3d4c-46e1-a62d-a0b59aecf599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:56.484358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0506dd7a-3d4c-46e1-a62d-a0b59aecf599 None None] MFA Rules not processed for user `d6d3ae72992a48709e3d58930ccd6b52`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:56.485913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0506dd7a-3d4c-46e1-a62d-a0b59aecf599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:56.488556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17da0d-3726-416e-a4ad-e5f64c1cedb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:56.492212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17da0d-3726-416e-a4ad-e5f64c1cedb3 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:56.529602 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 870/1780] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:56.539699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17da0d-3726-416e-a4ad-e5f64c1cedb3 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:56.591246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17da0d-3726-416e-a4ad-e5f64c1cedb3 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:56.629545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc3c9e-f387-4207-9e1b-c5daa917c700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:56.630484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc3c9e-f387-4207-9e1b-c5daa917c700 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:56.631593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc3c9e-f387-4207-9e1b-c5daa917c700 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:56.642440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc3c9e-f387-4207-9e1b-c5daa917c700 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:56.648127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc3c9e-f387-4207-9e1b-c5daa917c700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:56.655522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc3c9e-f387-4207-9e1b-c5daa917c700 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:56.677386 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 871/1781] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:56.677386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f7caa-31fa-4748-b8d0-ab241cd76613 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:56.677386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f7caa-31fa-4748-b8d0-ab241cd76613 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:56.677386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f7caa-31fa-4748-b8d0-ab241cd76613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:56.742284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:56.767957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-334f7caa-31fa-4748-b8d0-ab241cd76613 None None] MFA Rules not processed for user `6aa6de193f6c41b9b9a2c3fd1155b489`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:56.769341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-334f7caa-31fa-4748-b8d0-ab241cd76613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:56.783523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:56.819147 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 911/1782] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:56.820196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506dd7a-3d4c-46e1-a62d-a0b59aecf599 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:56.834924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506dd7a-3d4c-46e1-a62d-a0b59aecf599 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:56.848744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58112ce7-f408-47e3-9902-26cc2cc0f0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:56.850270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58112ce7-f408-47e3-9902-26cc2cc0f0c2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:56.851275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58112ce7-f408-47e3-9902-26cc2cc0f0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:56.851542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58112ce7-f408-47e3-9902-26cc2cc0f0c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:56.854064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58112ce7-f408-47e3-9902-26cc2cc0f0c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:56.857239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58112ce7-f408-47e3-9902-26cc2cc0f0c2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:56.881671 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 912/1783] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:56.890543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58112ce7-f408-47e3-9902-26cc2cc0f0c2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:56.910940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58112ce7-f408-47e3-9902-26cc2cc0f0c2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:56.917800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fc1e3-bb66-4fcb-b2ef-a10209c5a94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:56.919218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fc1e3-bb66-4fcb-b2ef-a10209c5a94c admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:56.919493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fc1e3-bb66-4fcb-b2ef-a10209c5a94c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:56.919817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fc1e3-bb66-4fcb-b2ef-a10209c5a94c admin admin] PATH_INFO: `/v3/projects/e93dcf30e02b41178c6cde6e50a41e17/users/7ef35c15f9d541ac90239c3713f9f5e6/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:56.947130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908fc1e3-bb66-4fcb-b2ef-a10209c5a94c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e93dcf30e02b41178c6cde6e50a41e17, user_id=7ef35c15f9d541ac90239c3713f9f5e6, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:56.951423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908fc1e3-bb66-4fcb-b2ef-a10209c5a94c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:56.977733 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 913/1784] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:56 2022] PUT /identity/v3/projects/e93dcf30e02b41178c6cde6e50a41e17/users/7ef35c15f9d541ac90239c3713f9f5e6/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:56.978815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fc1e3-bb66-4fcb-b2ef-a10209c5a94c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:56.981408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-908fc1e3-bb66-4fcb-b2ef-a10209c5a94c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:57.021521 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 872/1785] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:56 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:57.022405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f7caa-31fa-4748-b8d0-ab241cd76613 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:57.039338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f7caa-31fa-4748-b8d0-ab241cd76613 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:57.141858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914ae1d-ce00-4e1e-9f28-b1f28ca75785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:57.142763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914ae1d-ce00-4e1e-9f28-b1f28ca75785 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:57.143143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914ae1d-ce00-4e1e-9f28-b1f28ca75785 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:57.143514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914ae1d-ce00-4e1e-9f28-b1f28ca75785 admin admin] PATH_INFO: `/v3/projects/b331e32eb5964145bde96b9dce962819/users/35e45ec47cd648649829660999769d0c/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:57.167828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d914ae1d-ce00-4e1e-9f28-b1f28ca75785 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b331e32eb5964145bde96b9dce962819, user_id=35e45ec47cd648649829660999769d0c, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:57.184589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d914ae1d-ce00-4e1e-9f28-b1f28ca75785 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:57.196592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fc1e3-bb66-4fcb-b2ef-a10209c5a94c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:57.211691 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 873/1786] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:57 2022] PUT /identity/v3/projects/b331e32eb5964145bde96b9dce962819/users/35e45ec47cd648649829660999769d0c/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:57.219486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0488141-a12c-4c64-9dec-e932840564e8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:57.219796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0488141-a12c-4c64-9dec-e932840564e8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:57.220043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0488141-a12c-4c64-9dec-e932840564e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:57.223115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d656a65-c455-4297-8adb-a9f9c9277928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:57.223609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d656a65-c455-4297-8adb-a9f9c9277928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:57.223896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d656a65-c455-4297-8adb-a9f9c9277928 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:57.224265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d656a65-c455-4297-8adb-a9f9c9277928 admin admin] PATH_INFO: `/v3/projects/2b38d7f557db4bfeacdd4e958cec9557` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:57.225315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d656a65-c455-4297-8adb-a9f9c9277928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b38d7f557db4bfeacdd4e958cec9557)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:57.227296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d656a65-c455-4297-8adb-a9f9c9277928 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:57.332052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b0488141-a12c-4c64-9dec-e932840564e8 None None] MFA Rules not processed for user `e17a026f305248878f18eec35e1e02dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:57.336442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b0488141-a12c-4c64-9dec-e932840564e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:57.359302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3d656a65-c455-4297-8adb-a9f9c9277928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b38d7f557db4bfeacdd4e958cec9557', 'request_id': 'req-3d656a65-c455-4297-8adb-a9f9c9277928'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:57.382156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3d656a65-c455-4297-8adb-a9f9c9277928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b38d7f557db4bfeacdd4e958cec9557', 'request_id': 'req-3d656a65-c455-4297-8adb-a9f9c9277928'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:57.411471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3d656a65-c455-4297-8adb-a9f9c9277928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b38d7f557db4bfeacdd4e958cec9557', 'request_id': 'req-3d656a65-c455-4297-8adb-a9f9c9277928'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:57.436651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3d656a65-c455-4297-8adb-a9f9c9277928 admin admin] The token cache is being invalidate because project 2b38d7f557db4bfeacdd4e958cec9557 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:48:57.436982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3d656a65-c455-4297-8adb-a9f9c9277928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:57.438467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3d656a65-c455-4297-8adb-a9f9c9277928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:57.441601 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 914/1787] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:48:56 2022] DELETE /identity/v3/projects/2b38d7f557db4bfeacdd4e958cec9557 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:57.442996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d656a65-c455-4297-8adb-a9f9c9277928 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:57.446175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3d656a65-c455-4297-8adb-a9f9c9277928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:57.468625 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 874/1788] 10.222.0.27 () {58 vars in 1015 bytes} [Sat Oct 1 20:48:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:57.470082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0488141-a12c-4c64-9dec-e932840564e8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:57.472471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b0488141-a12c-4c64-9dec-e932840564e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:57.513575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:57.596833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:57.618364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0488141-a12c-4c64-9dec-e932840564e8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:57.621704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d656a65-c455-4297-8adb-a9f9c9277928 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:57.648375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadae1ff-45e4-4a2a-a94b-92991ad641f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:57.650396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadae1ff-45e4-4a2a-a94b-92991ad641f4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:57.651763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadae1ff-45e4-4a2a-a94b-92991ad641f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:57.652041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadae1ff-45e4-4a2a-a94b-92991ad641f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:57.652767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadae1ff-45e4-4a2a-a94b-92991ad641f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:57.654367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadae1ff-45e4-4a2a-a94b-92991ad641f4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:57.660164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2e7dd-1914-49fd-98de-aadc9ffbf113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:57.661158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2e7dd-1914-49fd-98de-aadc9ffbf113 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:57.661623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2e7dd-1914-49fd-98de-aadc9ffbf113 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:57.662258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2e7dd-1914-49fd-98de-aadc9ffbf113 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:57.663422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2e7dd-1914-49fd-98de-aadc9ffbf113 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:57.665581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2e7dd-1914-49fd-98de-aadc9ffbf113 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:57.683830 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 875/1789] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:57.685437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1798b8-50cc-4db2-9182-27cb2797643b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:57.685723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1798b8-50cc-4db2-9182-27cb2797643b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:57.685996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1798b8-50cc-4db2-9182-27cb2797643b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:57.703515 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 915/1790] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:57 2022] POST /identity/v3/projects => generated 377 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:57.707466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2e7dd-1914-49fd-98de-aadc9ffbf113 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:57.709661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2bb2e7dd-1914-49fd-98de-aadc9ffbf113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:57.757760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9a1798b8-50cc-4db2-9182-27cb2797643b None None] MFA Rules not processed for user `7ef35c15f9d541ac90239c3713f9f5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:57.759596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9a1798b8-50cc-4db2-9182-27cb2797643b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:57.877192 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 876/1791] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:57.877192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8952a7-c61a-46b1-9f4e-8b61a51d28bb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:57.877192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8952a7-c61a-46b1-9f4e-8b61a51d28bb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:57.877529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8952a7-c61a-46b1-9f4e-8b61a51d28bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:57.903095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2e7dd-1914-49fd-98de-aadc9ffbf113 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:57.915077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92883ffc-9031-42fa-9ea0-5cd080a6dc27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:57.916100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92883ffc-9031-42fa-9ea0-5cd080a6dc27 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:57.916591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92883ffc-9031-42fa-9ea0-5cd080a6dc27 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:57.917633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92883ffc-9031-42fa-9ea0-5cd080a6dc27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:57.919808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-92883ffc-9031-42fa-9ea0-5cd080a6dc27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:57.925125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5e8952a7-c61a-46b1-9f4e-8b61a51d28bb None None] MFA Rules not processed for user `35e45ec47cd648649829660999769d0c`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:57.926581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5e8952a7-c61a-46b1-9f4e-8b61a51d28bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:58.010105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92883ffc-9031-42fa-9ea0-5cd080a6dc27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:58.012123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92883ffc-9031-42fa-9ea0-5cd080a6dc27 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:58.022687 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 877/1792] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:57 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:58.023561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8952a7-c61a-46b1-9f4e-8b61a51d28bb None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:58.029894 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 916/1793] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:57 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:58.034435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92883ffc-9031-42fa-9ea0-5cd080a6dc27 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:58.039890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5e8952a7-c61a-46b1-9f4e-8b61a51d28bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:58.140803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:58.210273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92883ffc-9031-42fa-9ea0-5cd080a6dc27 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:58.221117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8952a7-c61a-46b1-9f4e-8b61a51d28bb None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:58.222282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491bb0fa-78b6-474a-9b09-43d8ddf3d27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:58.223057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bb0fa-78b6-474a-9b09-43d8ddf3d27d admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:58.223313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bb0fa-78b6-474a-9b09-43d8ddf3d27d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:58.223568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bb0fa-78b6-474a-9b09-43d8ddf3d27d admin admin] PATH_INFO: `/v3/projects/e82c68b8617a4b3cbc43b18d52f56842/users/64bded24bc1c4d6f832416230591e98f/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:58.239241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1c2c2-7e37-467e-9d5e-8ba3b498ce91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:58.239901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1c2c2-7e37-467e-9d5e-8ba3b498ce91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:58.240172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1c2c2-7e37-467e-9d5e-8ba3b498ce91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:58.240452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1c2c2-7e37-467e-9d5e-8ba3b498ce91 admin admin] PATH_INFO: `/v3/users/3e0503d20ac74f9ebe516009ddf320a5` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:58.244762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f1c2c2-7e37-467e-9d5e-8ba3b498ce91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e0503d20ac74f9ebe516009ddf320a5)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:58.258624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f1c2c2-7e37-467e-9d5e-8ba3b498ce91 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:58.261641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491bb0fa-78b6-474a-9b09-43d8ddf3d27d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e82c68b8617a4b3cbc43b18d52f56842, user_id=64bded24bc1c4d6f832416230591e98f, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:58.264380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491bb0fa-78b6-474a-9b09-43d8ddf3d27d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:58.282648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:58.294238 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 917/1794] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:58 2022] PUT /identity/v3/projects/e82c68b8617a4b3cbc43b18d52f56842/users/64bded24bc1c4d6f832416230591e98f/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:58.295187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491bb0fa-78b6-474a-9b09-43d8ddf3d27d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:58.380973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-16f1c2c2-7e37-467e-9d5e-8ba3b498ce91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e0503d20ac74f9ebe516009ddf320a5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:58.382832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-16f1c2c2-7e37-467e-9d5e-8ba3b498ce91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e0503d20ac74f9ebe516009ddf320a5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:58.384002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-16f1c2c2-7e37-467e-9d5e-8ba3b498ce91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e0503d20ac74f9ebe516009ddf320a5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:58.393736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-16f1c2c2-7e37-467e-9d5e-8ba3b498ce91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e0503d20ac74f9ebe516009ddf320a5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:58.411731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-16f1c2c2-7e37-467e-9d5e-8ba3b498ce91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e0503d20ac74f9ebe516009ddf320a5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:58.427039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491bb0fa-78b6-474a-9b09-43d8ddf3d27d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:58.432007 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 878/1795] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:48:58 2022] DELETE /identity/v3/users/3e0503d20ac74f9ebe516009ddf320a5 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:58.432953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa93695-87d0-4358-80fc-58a6512fa4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:58.433095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1c2c2-7e37-467e-9d5e-8ba3b498ce91 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:58.433776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa93695-87d0-4358-80fc-58a6512fa4e8 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:58.434159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa93695-87d0-4358-80fc-58a6512fa4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:58.434575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa93695-87d0-4358-80fc-58a6512fa4e8 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:58.435383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa93695-87d0-4358-80fc-58a6512fa4e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:58.435619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-16f1c2c2-7e37-467e-9d5e-8ba3b498ce91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:58.437629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa93695-87d0-4358-80fc-58a6512fa4e8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:58.489035 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 918/1796] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:58 2022] POST /identity/v3/users => generated 295 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:58.496714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa93695-87d0-4358-80fc-58a6512fa4e8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:58.502857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3fa93695-87d0-4358-80fc-58a6512fa4e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:58.542532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa93695-87d0-4358-80fc-58a6512fa4e8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:58.560047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1c2c2-7e37-467e-9d5e-8ba3b498ce91 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:58.567662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d216cf5-4367-45a0-a975-e411f88d3bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:58.568522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d216cf5-4367-45a0-a975-e411f88d3bec admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:58.574844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d216cf5-4367-45a0-a975-e411f88d3bec admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:58.575245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d216cf5-4367-45a0-a975-e411f88d3bec admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:58.576487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d216cf5-4367-45a0-a975-e411f88d3bec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:58.578609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d216cf5-4367-45a0-a975-e411f88d3bec admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:58.634635 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 879/1797] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:58 2022] POST /identity/v3/projects => generated 385 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:58.636729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d216cf5-4367-45a0-a975-e411f88d3bec admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:58.639882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1d216cf5-4367-45a0-a975-e411f88d3bec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:58.684763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f48be0-e1e8-444e-98e0-33fc1035ab4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:58.685670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f48be0-e1e8-444e-98e0-33fc1035ab4d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:58.685972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f48be0-e1e8-444e-98e0-33fc1035ab4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:58.686239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f48be0-e1e8-444e-98e0-33fc1035ab4d admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:58.686921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f48be0-e1e8-444e-98e0-33fc1035ab4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:58.688987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f48be0-e1e8-444e-98e0-33fc1035ab4d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:58.711173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:58.754322 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 919/1798] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:58 2022] POST /identity/v3/projects => generated 389 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:58.755668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65201dae-0880-492b-b814-33d1fc3138a9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:58.755961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65201dae-0880-492b-b814-33d1fc3138a9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:58.756238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65201dae-0880-492b-b814-33d1fc3138a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:58.804686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-65201dae-0880-492b-b814-33d1fc3138a9 None None] MFA Rules not processed for user `64bded24bc1c4d6f832416230591e98f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:48:58.805615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d216cf5-4367-45a0-a975-e411f88d3bec admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:58.805914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-65201dae-0880-492b-b814-33d1fc3138a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:58.911943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bdfbe2-16d6-461e-b3c3-b3d7fa6e87d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:58.912594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bdfbe2-16d6-461e-b3c3-b3d7fa6e87d7 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:58.912881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bdfbe2-16d6-461e-b3c3-b3d7fa6e87d7 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:58.913155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bdfbe2-16d6-461e-b3c3-b3d7fa6e87d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:58.915020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-79bdfbe2-16d6-461e-b3c3-b3d7fa6e87d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:58.915266 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 920/1799] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:48:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:48:58.916987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65201dae-0880-492b-b814-33d1fc3138a9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:58.923247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-65201dae-0880-492b-b814-33d1fc3138a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:58.936522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:59.035655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bdfbe2-16d6-461e-b3c3-b3d7fa6e87d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:59.037615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bdfbe2-16d6-461e-b3c3-b3d7fa6e87d7 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:59.056342 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 880/1800] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:48:58 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:48:59.071003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bdfbe2-16d6-461e-b3c3-b3d7fa6e87d7 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:59.076380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65201dae-0880-492b-b814-33d1fc3138a9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:59.084755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfd93d-aa73-4055-a539-36a2d26c00a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:59.085748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfd93d-aa73-4055-a539-36a2d26c00a4 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:59.085928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfd93d-aa73-4055-a539-36a2d26c00a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:59.086294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfd93d-aa73-4055-a539-36a2d26c00a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:59.086985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bfd93d-aa73-4055-a539-36a2d26c00a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:59.089007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bfd93d-aa73-4055-a539-36a2d26c00a4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:59.103091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bdfbe2-16d6-461e-b3c3-b3d7fa6e87d7 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:59.107941 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 921/1801] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:58 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:59.107941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfd93d-aa73-4055-a539-36a2d26c00a4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:59.125482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfd93d-aa73-4055-a539-36a2d26c00a4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:59.131477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48b89c-c1d7-4054-850b-1428c91bf1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:59.132237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48b89c-c1d7-4054-850b-1428c91bf1b2 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:59.132517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48b89c-c1d7-4054-850b-1428c91bf1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:59.132861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48b89c-c1d7-4054-850b-1428c91bf1b2 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:59.133569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48b89c-c1d7-4054-850b-1428c91bf1b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:59.135797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48b89c-c1d7-4054-850b-1428c91bf1b2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:59.143418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cd6b5-0943-464e-a18c-1fc35f42032f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:59.144155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cd6b5-0943-464e-a18c-1fc35f42032f admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:59.144575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cd6b5-0943-464e-a18c-1fc35f42032f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:59.144868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cd6b5-0943-464e-a18c-1fc35f42032f admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:59.145642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cd6b5-0943-464e-a18c-1fc35f42032f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:59.148045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cd6b5-0943-464e-a18c-1fc35f42032f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:59.173449 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 922/1802] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:59 2022] POST /identity/v3/users => generated 301 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:59.174335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48b89c-c1d7-4054-850b-1428c91bf1b2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:59.178983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7b48b89c-c1d7-4054-850b-1428c91bf1b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:59.196910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48b89c-c1d7-4054-850b-1428c91bf1b2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:59.204194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3c882-9cb5-42cc-ab2e-8297c9265521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:59.204949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3c882-9cb5-42cc-ab2e-8297c9265521 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:59.205306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3c882-9cb5-42cc-ab2e-8297c9265521 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:59.205582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3c882-9cb5-42cc-ab2e-8297c9265521 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:59.205949 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 881/1803] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:59 2022] POST /identity/v3/users => generated 299 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:59.206810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb3c882-9cb5-42cc-ab2e-8297c9265521 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:59.207013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cd6b5-0943-464e-a18c-1fc35f42032f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:59.209393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb3c882-9cb5-42cc-ab2e-8297c9265521 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:59.209903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-314cd6b5-0943-464e-a18c-1fc35f42032f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:59.237251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cd6b5-0943-464e-a18c-1fc35f42032f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:59.255483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b856834-c59f-47ad-99ac-30815aa5b6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:59.256067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b856834-c59f-47ad-99ac-30815aa5b6b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:59.256349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b856834-c59f-47ad-99ac-30815aa5b6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:59.256650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b856834-c59f-47ad-99ac-30815aa5b6b3 admin admin] PATH_INFO: `/v3/projects/08b6602ad13646c195ab1f3be7aa0e72` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:59.257912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b856834-c59f-47ad-99ac-30815aa5b6b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08b6602ad13646c195ab1f3be7aa0e72)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:59.259905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b856834-c59f-47ad-99ac-30815aa5b6b3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:59.265820 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 923/1804] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:48:59 2022] POST /identity/v3/projects => generated 387 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:59.265820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3c882-9cb5-42cc-ab2e-8297c9265521 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:59.378329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3c882-9cb5-42cc-ab2e-8297c9265521 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:59.380057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:59.384936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6b856834-c59f-47ad-99ac-30815aa5b6b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08b6602ad13646c195ab1f3be7aa0e72', 'request_id': 'req-6b856834-c59f-47ad-99ac-30815aa5b6b3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:59.400538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6b856834-c59f-47ad-99ac-30815aa5b6b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08b6602ad13646c195ab1f3be7aa0e72', 'request_id': 'req-6b856834-c59f-47ad-99ac-30815aa5b6b3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:59.421096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6b856834-c59f-47ad-99ac-30815aa5b6b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08b6602ad13646c195ab1f3be7aa0e72', 'request_id': 'req-6b856834-c59f-47ad-99ac-30815aa5b6b3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:59.453887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6b856834-c59f-47ad-99ac-30815aa5b6b3 admin admin] The token cache is being invalidate because project 08b6602ad13646c195ab1f3be7aa0e72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:48:59.454314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6b856834-c59f-47ad-99ac-30815aa5b6b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:59.464983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6b856834-c59f-47ad-99ac-30815aa5b6b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:59.469790 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 882/1805] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:48:59 2022] DELETE /identity/v3/projects/08b6602ad13646c195ab1f3be7aa0e72 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:59.469790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b856834-c59f-47ad-99ac-30815aa5b6b3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:59.469790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6b856834-c59f-47ad-99ac-30815aa5b6b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:59.496090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e78037-6dd2-40d3-a97a-45e77f13b8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:59.496886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e78037-6dd2-40d3-a97a-45e77f13b8a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:59.497175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e78037-6dd2-40d3-a97a-45e77f13b8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:59.497447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e78037-6dd2-40d3-a97a-45e77f13b8a5 admin admin] PATH_INFO: `/v3/projects/51f59d1267df48309fe64b4d960720bb/users/6cbdb512aa12495b96d80f245efb6012/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:59.528507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:59.535117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e78037-6dd2-40d3-a97a-45e77f13b8a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51f59d1267df48309fe64b4d960720bb, user_id=6cbdb512aa12495b96d80f245efb6012, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:59.541553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e78037-6dd2-40d3-a97a-45e77f13b8a5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:59.570165 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 924/1806] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:59 2022] PUT /identity/v3/projects/51f59d1267df48309fe64b4d960720bb/users/6cbdb512aa12495b96d80f245efb6012/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:48:59.571370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e78037-6dd2-40d3-a97a-45e77f13b8a5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:59.585996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-93e78037-6dd2-40d3-a97a-45e77f13b8a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:59.610508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b856834-c59f-47ad-99ac-30815aa5b6b3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:59.723280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e78037-6dd2-40d3-a97a-45e77f13b8a5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:59.729906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84cfb2-7d69-41c5-aa4e-f2c90422c715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:59.732072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84cfb2-7d69-41c5-aa4e-f2c90422c715 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:59.732511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84cfb2-7d69-41c5-aa4e-f2c90422c715 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:59.732717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84cfb2-7d69-41c5-aa4e-f2c90422c715 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:59.737069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c84cfb2-7d69-41c5-aa4e-f2c90422c715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:59.737069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c84cfb2-7d69-41c5-aa4e-f2c90422c715 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:59.760409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29ab4a-194b-411e-a00d-f3c1ebd62fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:59.763849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29ab4a-194b-411e-a00d-f3c1ebd62fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:59.764140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29ab4a-194b-411e-a00d-f3c1ebd62fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:59.764417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29ab4a-194b-411e-a00d-f3c1ebd62fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:59.765080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29ab4a-194b-411e-a00d-f3c1ebd62fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:59.767859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29ab4a-194b-411e-a00d-f3c1ebd62fa5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:59.772069 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 925/1807] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:59.772971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84cfb2-7d69-41c5-aa4e-f2c90422c715 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:59.776551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5c84cfb2-7d69-41c5-aa4e-f2c90422c715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:59.790106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84cfb2-7d69-41c5-aa4e-f2c90422c715 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:59.795912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33d28e-01bd-4ab0-bf6d-82e2de6b7b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:59.796568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33d28e-01bd-4ab0-bf6d-82e2de6b7b2b admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:59.796867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33d28e-01bd-4ab0-bf6d-82e2de6b7b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:59.796984 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 883/1808] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:59.797148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33d28e-01bd-4ab0-bf6d-82e2de6b7b2b admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:59.798121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33d28e-01bd-4ab0-bf6d-82e2de6b7b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:59.798248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29ab4a-194b-411e-a00d-f3c1ebd62fa5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:59.800204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33d28e-01bd-4ab0-bf6d-82e2de6b7b2b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:59.800328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7e29ab4a-194b-411e-a00d-f3c1ebd62fa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:59.832052 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 926/1809] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:48:59 2022] POST /identity/v3/users => generated 300 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:48:59.832462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33d28e-01bd-4ab0-bf6d-82e2de6b7b2b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:59.837625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bb33d28e-01bd-4ab0-bf6d-82e2de6b7b2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:48:59.838121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29ab4a-194b-411e-a00d-f3c1ebd62fa5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:59.850449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e3427-5e1e-4b80-86fe-850282a62284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:59.851217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e3427-5e1e-4b80-86fe-850282a62284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:59.851499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e3427-5e1e-4b80-86fe-850282a62284 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:59.851780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e3427-5e1e-4b80-86fe-850282a62284 admin admin] PATH_INFO: `/v3/users/6debf86cca3a4c81a036c8e7f243e60a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:59.853220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07e3427-5e1e-4b80-86fe-850282a62284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6debf86cca3a4c81a036c8e7f243e60a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:59.858052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07e3427-5e1e-4b80-86fe-850282a62284 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:59.876272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33d28e-01bd-4ab0-bf6d-82e2de6b7b2b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:59.891811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6eea7b-3a1e-43da-ac9b-a6da1ab9db1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:59.893012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6eea7b-3a1e-43da-ac9b-a6da1ab9db1d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:59.893365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6eea7b-3a1e-43da-ac9b-a6da1ab9db1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:59.893810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6eea7b-3a1e-43da-ac9b-a6da1ab9db1d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:59.896105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6eea7b-3a1e-43da-ac9b-a6da1ab9db1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:59.899336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6eea7b-3a1e-43da-ac9b-a6da1ab9db1d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:48:59.926313 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 927/1810] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:48:59.926896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6eea7b-3a1e-43da-ac9b-a6da1ab9db1d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:48:59.950946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6eea7b-3a1e-43da-ac9b-a6da1ab9db1d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:48:59.956341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e07e3427-5e1e-4b80-86fe-850282a62284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6debf86cca3a4c81a036c8e7f243e60a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:59.957295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e07e3427-5e1e-4b80-86fe-850282a62284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6debf86cca3a4c81a036c8e7f243e60a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:59.958101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e07e3427-5e1e-4b80-86fe-850282a62284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6debf86cca3a4c81a036c8e7f243e60a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:59.966801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eff349-9825-4cc8-8bc0-ba3d0c8a2f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:48:59.967712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eff349-9825-4cc8-8bc0-ba3d0c8a2f67 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:48:59.968033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eff349-9825-4cc8-8bc0-ba3d0c8a2f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:48:59.968384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eff349-9825-4cc8-8bc0-ba3d0c8a2f67 admin admin] PATH_INFO: `/v3/projects/a10db1d5e60c44a6819b67914849b844/users/4697a57e25c14284ae92262df0793cb5/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:48:59.976338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e07e3427-5e1e-4b80-86fe-850282a62284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6debf86cca3a4c81a036c8e7f243e60a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:48:59.990503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eff349-9825-4cc8-8bc0-ba3d0c8a2f67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a10db1d5e60c44a6819b67914849b844, user_id=4697a57e25c14284ae92262df0793cb5, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:48:59.997333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eff349-9825-4cc8-8bc0-ba3d0c8a2f67 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:00.005837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:00.008004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e07e3427-5e1e-4b80-86fe-850282a62284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6debf86cca3a4c81a036c8e7f243e60a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:00.022354 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 928/1811] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:48:59 2022] PUT /identity/v3/projects/a10db1d5e60c44a6819b67914849b844/users/4697a57e25c14284ae92262df0793cb5/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:00.026619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eff349-9825-4cc8-8bc0-ba3d0c8a2f67 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:00.029497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-40eff349-9825-4cc8-8bc0-ba3d0c8a2f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:00.038468 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 884/1812] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:48:59 2022] DELETE /identity/v3/users/6debf86cca3a4c81a036c8e7f243e60a => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:00.054019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e3427-5e1e-4b80-86fe-850282a62284 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:00.060801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e07e3427-5e1e-4b80-86fe-850282a62284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:00.192110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eff349-9825-4cc8-8bc0-ba3d0c8a2f67 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:00.251363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e3427-5e1e-4b80-86fe-850282a62284 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:00.263142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeeb38b-f002-4fab-852c-ecda9497030b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:00.263856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeeb38b-f002-4fab-852c-ecda9497030b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:00.264235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeeb38b-f002-4fab-852c-ecda9497030b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:00.264613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeeb38b-f002-4fab-852c-ecda9497030b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:00.265392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeeb38b-f002-4fab-852c-ecda9497030b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:00.267128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeeb38b-f002-4fab-852c-ecda9497030b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:00.268607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:00.296419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643a184-e36e-4998-8e55-7c524d9e3fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:00.297131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643a184-e36e-4998-8e55-7c524d9e3fea admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:00.297578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643a184-e36e-4998-8e55-7c524d9e3fea admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:00.297963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643a184-e36e-4998-8e55-7c524d9e3fea admin admin] PATH_INFO: `/v3/projects/479f3dd136484b2f8cf66d951ce0ee58/users/4e27033a708c40c8b262464105e126e6/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:00.313180 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 885/1813] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:00.320287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeeb38b-f002-4fab-852c-ecda9497030b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:00.329941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643a184-e36e-4998-8e55-7c524d9e3fea admin admin] RBAC: Authorizing `identity:create_grant(project_id=479f3dd136484b2f8cf66d951ce0ee58, user_id=4e27033a708c40c8b262464105e126e6, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:00.332670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5eeeb38b-f002-4fab-852c-ecda9497030b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:00.333121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643a184-e36e-4998-8e55-7c524d9e3fea admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:00.353700 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 929/1814] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:00 2022] PUT /identity/v3/projects/479f3dd136484b2f8cf66d951ce0ee58/users/4e27033a708c40c8b262464105e126e6/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:00.355969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643a184-e36e-4998-8e55-7c524d9e3fea admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:00.357910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8643a184-e36e-4998-8e55-7c524d9e3fea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:00.375495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeeb38b-f002-4fab-852c-ecda9497030b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:00.476595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643a184-e36e-4998-8e55-7c524d9e3fea admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:00.484289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b465270-d92a-4f93-99e6-d3d0384d9bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:00.485056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b465270-d92a-4f93-99e6-d3d0384d9bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:00.485597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b465270-d92a-4f93-99e6-d3d0384d9bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:00.485926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b465270-d92a-4f93-99e6-d3d0384d9bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:00.487501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b465270-d92a-4f93-99e6-d3d0384d9bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:00.489362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b465270-d92a-4f93-99e6-d3d0384d9bc2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:00.506656 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 930/1815] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:00.508092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b465270-d92a-4f93-99e6-d3d0384d9bc2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:00.516170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a43e8-97ab-46fb-95b5-51ce9d541c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:00.516952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a43e8-97ab-46fb-95b5-51ce9d541c50 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:00.517221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a43e8-97ab-46fb-95b5-51ce9d541c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:00.517505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a43e8-97ab-46fb-95b5-51ce9d541c50 admin admin] PATH_INFO: `/v3/projects/51f59d1267df48309fe64b4d960720bb/users/6cbdb512aa12495b96d80f245efb6012/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:00.519852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a43e8-97ab-46fb-95b5-51ce9d541c50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51f59d1267df48309fe64b4d960720bb, user_id=6cbdb512aa12495b96d80f245efb6012, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:00.523111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b465270-d92a-4f93-99e6-d3d0384d9bc2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:00.523593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a43e8-97ab-46fb-95b5-51ce9d541c50 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:00.529922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c14b8d-ff29-47f3-83e5-f1935b3e1e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:00.531025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c14b8d-ff29-47f3-83e5-f1935b3e1e68 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:00.531708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c14b8d-ff29-47f3-83e5-f1935b3e1e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:00.532468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c14b8d-ff29-47f3-83e5-f1935b3e1e68 admin admin] PATH_INFO: `/v3/projects/bde2c758b4b4486881f4026201c90359/users/c3523161149c444f826a7c9d332cb6b8/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:00.555009 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 886/1816] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:00 2022] PUT /identity/v3/projects/51f59d1267df48309fe64b4d960720bb/users/6cbdb512aa12495b96d80f245efb6012/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:00.555009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a43e8-97ab-46fb-95b5-51ce9d541c50 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:00.569482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c14b8d-ff29-47f3-83e5-f1935b3e1e68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde2c758b4b4486881f4026201c90359, user_id=c3523161149c444f826a7c9d332cb6b8, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:00.572876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c14b8d-ff29-47f3-83e5-f1935b3e1e68 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:00.599727 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 931/1817] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:00 2022] PUT /identity/v3/projects/bde2c758b4b4486881f4026201c90359/users/c3523161149c444f826a7c9d332cb6b8/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:00.600646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c14b8d-ff29-47f3-83e5-f1935b3e1e68 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:00.696460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c14b8d-ff29-47f3-83e5-f1935b3e1e68 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:00.702835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb999a-9383-4d45-8993-dadaf79ae0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:00.703328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb999a-9383-4d45-8993-dadaf79ae0be admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:00.703703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb999a-9383-4d45-8993-dadaf79ae0be admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:00.703985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb999a-9383-4d45-8993-dadaf79ae0be admin admin] PATH_INFO: `/v3/projects/a10db1d5e60c44a6819b67914849b844/users/4697a57e25c14284ae92262df0793cb5/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:00.706041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb999a-9383-4d45-8993-dadaf79ae0be admin admin] RBAC: Authorizing `identity:create_grant(project_id=a10db1d5e60c44a6819b67914849b844, user_id=4697a57e25c14284ae92262df0793cb5, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:00.708829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb999a-9383-4d45-8993-dadaf79ae0be admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:00.730577 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 932/1818] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:00 2022] PUT /identity/v3/projects/a10db1d5e60c44a6819b67914849b844/users/4697a57e25c14284ae92262df0793cb5/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:00.730577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533ec0c-f477-465c-a483-b47ccf81a547 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:00.730577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533ec0c-f477-465c-a483-b47ccf81a547 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:00.730577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533ec0c-f477-465c-a483-b47ccf81a547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:00.730975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a43e8-97ab-46fb-95b5-51ce9d541c50 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:00.770236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9533ec0c-f477-465c-a483-b47ccf81a547 None None] MFA Rules not processed for user `6cbdb512aa12495b96d80f245efb6012`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:00.773782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9533ec0c-f477-465c-a483-b47ccf81a547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:00.804658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:00.807170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:00.905867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f5ee4-cb75-4e24-b3d9-aa393ef97709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:00.907983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f5ee4-cb75-4e24-b3d9-aa393ef97709 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:00.908288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f5ee4-cb75-4e24-b3d9-aa393ef97709 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:00.908574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f5ee4-cb75-4e24-b3d9-aa393ef97709 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:00.911903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f5ee4-cb75-4e24-b3d9-aa393ef97709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:00.913664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f5ee4-cb75-4e24-b3d9-aa393ef97709 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:00.938849 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 933/1819] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:00.939865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533ec0c-f477-465c-a483-b47ccf81a547 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:00.944791 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 887/1820] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:00.946581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a175358-4e22-46bd-be58-91d267c5cffc None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:00.946957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a175358-4e22-46bd-be58-91d267c5cffc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:00.947165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a175358-4e22-46bd-be58-91d267c5cffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:00.969260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533ec0c-f477-465c-a483-b47ccf81a547 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:00.984913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a23d00-2966-4ae4-8f65-f64382415c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:00.987039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a23d00-2966-4ae4-8f65-f64382415c46 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:00.987754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a23d00-2966-4ae4-8f65-f64382415c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:00.988208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a23d00-2966-4ae4-8f65-f64382415c46 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:00.991161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a23d00-2966-4ae4-8f65-f64382415c46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:00.994083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3a175358-4e22-46bd-be58-91d267c5cffc None None] MFA Rules not processed for user `4697a57e25c14284ae92262df0793cb5`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:00.995854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a23d00-2966-4ae4-8f65-f64382415c46 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:01.009903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3a175358-4e22-46bd-be58-91d267c5cffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:01.035442 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 934/1821] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:01.036541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a23d00-2966-4ae4-8f65-f64382415c46 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:01.041631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b4a23d00-2966-4ae4-8f65-f64382415c46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:01.200487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a23d00-2966-4ae4-8f65-f64382415c46 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:01.204699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcd5f8-1d29-43b7-b900-c5ad3e5a8a05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:01.205392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcd5f8-1d29-43b7-b900-c5ad3e5a8a05 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:01.205680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcd5f8-1d29-43b7-b900-c5ad3e5a8a05 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:01.206003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcd5f8-1d29-43b7-b900-c5ad3e5a8a05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:01.208454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-aedcd5f8-1d29-43b7-b900-c5ad3e5a8a05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:01.220385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedcd5f8-1d29-43b7-b900-c5ad3e5a8a05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:01.221830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedcd5f8-1d29-43b7-b900-c5ad3e5a8a05 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:01.245937 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 935/1822] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:01.249471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcd5f8-1d29-43b7-b900-c5ad3e5a8a05 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:01.253094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-aedcd5f8-1d29-43b7-b900-c5ad3e5a8a05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:01.274712 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 888/1823] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:01.274712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a175358-4e22-46bd-be58-91d267c5cffc None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:01.296770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcd5f8-1d29-43b7-b900-c5ad3e5a8a05 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:01.300027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a175358-4e22-46bd-be58-91d267c5cffc None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:01.311876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23758dc7-df64-4c80-9212-3d61bec2ec72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:01.313717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23758dc7-df64-4c80-9212-3d61bec2ec72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:01.319351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0210998-320f-48d8-9a04-a5080001569a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:01.319714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23758dc7-df64-4c80-9212-3d61bec2ec72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:01.320159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0210998-320f-48d8-9a04-a5080001569a admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:01.320263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23758dc7-df64-4c80-9212-3d61bec2ec72 admin admin] PATH_INFO: `/v3/projects/02be8fe6f40b42a684054e8a3a7ebf9e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:01.320463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0210998-320f-48d8-9a04-a5080001569a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:01.320763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0210998-320f-48d8-9a04-a5080001569a admin admin] PATH_INFO: `/v3/projects/479f3dd136484b2f8cf66d951ce0ee58/users/4e27033a708c40c8b262464105e126e6/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:01.330090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23758dc7-df64-4c80-9212-3d61bec2ec72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02be8fe6f40b42a684054e8a3a7ebf9e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:01.334146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23758dc7-df64-4c80-9212-3d61bec2ec72 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:01.336009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0210998-320f-48d8-9a04-a5080001569a admin admin] RBAC: Authorizing `identity:create_grant(project_id=479f3dd136484b2f8cf66d951ce0ee58, user_id=4e27033a708c40c8b262464105e126e6, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:01.340547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0210998-320f-48d8-9a04-a5080001569a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:01.376050 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 889/1824] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:01 2022] PUT /identity/v3/projects/479f3dd136484b2f8cf66d951ce0ee58/users/4e27033a708c40c8b262464105e126e6/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:01.377750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0210998-320f-48d8-9a04-a5080001569a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:01.384576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b0210998-320f-48d8-9a04-a5080001569a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:01.471481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-23758dc7-df64-4c80-9212-3d61bec2ec72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02be8fe6f40b42a684054e8a3a7ebf9e', 'request_id': 'req-23758dc7-df64-4c80-9212-3d61bec2ec72'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:01.486716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-23758dc7-df64-4c80-9212-3d61bec2ec72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02be8fe6f40b42a684054e8a3a7ebf9e', 'request_id': 'req-23758dc7-df64-4c80-9212-3d61bec2ec72'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:01.505518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-23758dc7-df64-4c80-9212-3d61bec2ec72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02be8fe6f40b42a684054e8a3a7ebf9e', 'request_id': 'req-23758dc7-df64-4c80-9212-3d61bec2ec72'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:01.539094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-23758dc7-df64-4c80-9212-3d61bec2ec72 admin admin] The token cache is being invalidate because project 02be8fe6f40b42a684054e8a3a7ebf9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:01.539437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-23758dc7-df64-4c80-9212-3d61bec2ec72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:01.541347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-23758dc7-df64-4c80-9212-3d61bec2ec72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:01.553497 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 936/1825] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:01 2022] DELETE /identity/v3/projects/02be8fe6f40b42a684054e8a3a7ebf9e => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:01.553497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23758dc7-df64-4c80-9212-3d61bec2ec72 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:01.553497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-23758dc7-df64-4c80-9212-3d61bec2ec72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:01.610386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0210998-320f-48d8-9a04-a5080001569a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:01.659287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:01.703153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:01.746714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23758dc7-df64-4c80-9212-3d61bec2ec72 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:01.759580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af9fed-8aad-412a-8194-027568fe878c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:01.761854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af9fed-8aad-412a-8194-027568fe878c admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:01.762140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af9fed-8aad-412a-8194-027568fe878c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:01.762420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af9fed-8aad-412a-8194-027568fe878c admin admin] PATH_INFO: `/v3/projects/bde2c758b4b4486881f4026201c90359/users/c3523161149c444f826a7c9d332cb6b8/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:01.763719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3432e41-5542-4be1-92cb-5be72a7036bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:01.778714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3432e41-5542-4be1-92cb-5be72a7036bc service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:01.779430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98af9fed-8aad-412a-8194-027568fe878c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde2c758b4b4486881f4026201c90359, user_id=c3523161149c444f826a7c9d332cb6b8, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:01.780140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3432e41-5542-4be1-92cb-5be72a7036bc service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:01.783038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3432e41-5542-4be1-92cb-5be72a7036bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:01.788733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98af9fed-8aad-412a-8194-027568fe878c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:01.790328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b3432e41-5542-4be1-92cb-5be72a7036bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:01.836154 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 937/1826] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:01 2022] PUT /identity/v3/projects/bde2c758b4b4486881f4026201c90359/users/c3523161149c444f826a7c9d332cb6b8/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:01.837909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a75730-1bcb-4349-ad38-edab9ef99095 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:01.838242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a75730-1bcb-4349-ad38-edab9ef99095 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:01.838555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a75730-1bcb-4349-ad38-edab9ef99095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:01.876340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c1a75730-1bcb-4349-ad38-edab9ef99095 None None] MFA Rules not processed for user `7ef35c15f9d541ac90239c3713f9f5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:01.877568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c1a75730-1bcb-4349-ad38-edab9ef99095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:01.968014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3432e41-5542-4be1-92cb-5be72a7036bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:01.969921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3432e41-5542-4be1-92cb-5be72a7036bc service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:01.991523 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 938/1827] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:01.991523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b52f1-6a9c-469a-90d7-49f94ba0ca4c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:01.991523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b52f1-6a9c-469a-90d7-49f94ba0ca4c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:01.991523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b52f1-6a9c-469a-90d7-49f94ba0ca4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:02.000821 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 890/1828] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:02.008266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee99f95-9fc5-4d5e-a3b7-8c466f114520 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:02.015935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee99f95-9fc5-4d5e-a3b7-8c466f114520 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:02.016338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee99f95-9fc5-4d5e-a3b7-8c466f114520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:02.074483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3ee99f95-9fc5-4d5e-a3b7-8c466f114520 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:02.076730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3ee99f95-9fc5-4d5e-a3b7-8c466f114520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:02.095136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3a6b52f1-6a9c-469a-90d7-49f94ba0ca4c None None] MFA Rules not processed for user `4e27033a708c40c8b262464105e126e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:02.097204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3a6b52f1-6a9c-469a-90d7-49f94ba0ca4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:02.243869 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 891/1829] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:02.246884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edef522-621b-4c65-92ec-f02bb8f986bb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:02.247178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edef522-621b-4c65-92ec-f02bb8f986bb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:02.247516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edef522-621b-4c65-92ec-f02bb8f986bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:02.307575 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 939/1830] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:01 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:02.309237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b52f1-6a9c-469a-90d7-49f94ba0ca4c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:02.314767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3a6b52f1-6a9c-469a-90d7-49f94ba0ca4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:02.348333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2edef522-621b-4c65-92ec-f02bb8f986bb None None] MFA Rules not processed for user `c3523161149c444f826a7c9d332cb6b8`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:02.354422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2edef522-621b-4c65-92ec-f02bb8f986bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:02.425521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:02.456943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b52f1-6a9c-469a-90d7-49f94ba0ca4c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:02.463986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b1eb6-c6ce-4c55-87c8-ab636f6a2b1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:02.465913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b1eb6-c6ce-4c55-87c8-ab636f6a2b1a service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:02.466694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b1eb6-c6ce-4c55-87c8-ab636f6a2b1a service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:02.467190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b1eb6-c6ce-4c55-87c8-ab636f6a2b1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:02.469934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-676b1eb6-c6ce-4c55-87c8-ab636f6a2b1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:02.487517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b1eb6-c6ce-4c55-87c8-ab636f6a2b1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:02.492385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b1eb6-c6ce-4c55-87c8-ab636f6a2b1a service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:02.524089 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 940/1831] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:02.536135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44721f1-7c2d-4f5c-8b2d-3a2405a37d72 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:02.536355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44721f1-7c2d-4f5c-8b2d-3a2405a37d72 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:02.536438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44721f1-7c2d-4f5c-8b2d-3a2405a37d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:02.547287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:02.587401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e44721f1-7c2d-4f5c-8b2d-3a2405a37d72 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:02.590939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e44721f1-7c2d-4f5c-8b2d-3a2405a37d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:02.594633 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 892/1832] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:02.632665 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 941/1833] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:02.637030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edef522-621b-4c65-92ec-f02bb8f986bb None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:02.682447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edef522-621b-4c65-92ec-f02bb8f986bb None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:02.726123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f2e052-6358-4070-9b03-48f58109eb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:02.727148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2e052-6358-4070-9b03-48f58109eb99 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:02.728531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2e052-6358-4070-9b03-48f58109eb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:02.730136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2e052-6358-4070-9b03-48f58109eb99 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:02.732376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f2e052-6358-4070-9b03-48f58109eb99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:49:02.733083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f2e052-6358-4070-9b03-48f58109eb99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:02.735145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f2e052-6358-4070-9b03-48f58109eb99 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:02.794729 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 893/1834] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:49:02.801854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44721f1-7c2d-4f5c-8b2d-3a2405a37d72 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:02.814949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44721f1-7c2d-4f5c-8b2d-3a2405a37d72 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:02.849830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfc716-2735-4b6e-9120-9395608ee897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:02.850725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfc716-2735-4b6e-9120-9395608ee897 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:02.852428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfc716-2735-4b6e-9120-9395608ee897 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:02.853334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfc716-2735-4b6e-9120-9395608ee897 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:02.855477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bfc716-2735-4b6e-9120-9395608ee897 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:02.861937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bfc716-2735-4b6e-9120-9395608ee897 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:02.896303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23876e-08e0-40b8-9223-411a9ddee513 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:02.896526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23876e-08e0-40b8-9223-411a9ddee513 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:02.896776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23876e-08e0-40b8-9223-411a9ddee513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:02.952352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bc23876e-08e0-40b8-9223-411a9ddee513 None None] MFA Rules not processed for user `f40f50783d9f417a8f84fab11b0d10d0`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:02.956342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bc23876e-08e0-40b8-9223-411a9ddee513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:02.976790 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 942/1835] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:02 2022] POST /identity/v3/projects => generated 403 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:02.982564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfc716-2735-4b6e-9120-9395608ee897 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:03.149534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfc716-2735-4b6e-9120-9395608ee897 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:03.156468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4af74-c72f-43a1-9ee1-cc1373f263db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:03.157312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4af74-c72f-43a1-9ee1-cc1373f263db admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:03.157868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4af74-c72f-43a1-9ee1-cc1373f263db admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:03.158282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4af74-c72f-43a1-9ee1-cc1373f263db admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:03.159172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4af74-c72f-43a1-9ee1-cc1373f263db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:03.161193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4af74-c72f-43a1-9ee1-cc1373f263db admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:03.196238 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 943/1836] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:02 2022] POST /identity/v3/users => generated 302 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:49:03.198438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4af74-c72f-43a1-9ee1-cc1373f263db admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:03.208211 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 894/1837] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:02 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:03.216676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4af74-c72f-43a1-9ee1-cc1373f263db admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:03.222651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477d854-b16c-47e6-9dad-79826b2dded2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:03.223275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477d854-b16c-47e6-9dad-79826b2dded2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:03.223556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477d854-b16c-47e6-9dad-79826b2dded2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:03.223822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477d854-b16c-47e6-9dad-79826b2dded2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:03.224472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477d854-b16c-47e6-9dad-79826b2dded2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:03.226125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477d854-b16c-47e6-9dad-79826b2dded2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:03.242124 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 944/1838] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:03.242325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23876e-08e0-40b8-9223-411a9ddee513 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:03.258273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23876e-08e0-40b8-9223-411a9ddee513 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:03.264222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231ba87-c4c9-414e-994c-dc9650a9a160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:03.265843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231ba87-c4c9-414e-994c-dc9650a9a160 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:03.266137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231ba87-c4c9-414e-994c-dc9650a9a160 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:03.267095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231ba87-c4c9-414e-994c-dc9650a9a160 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:03.267817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3231ba87-c4c9-414e-994c-dc9650a9a160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:03.273405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3231ba87-c4c9-414e-994c-dc9650a9a160 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:03.309532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477d854-b16c-47e6-9dad-79826b2dded2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:03.322465 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 895/1839] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:03.329819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477d854-b16c-47e6-9dad-79826b2dded2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:03.341284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769851ba-6cf8-4409-bdd7-0f4381f78f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:03.342714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769851ba-6cf8-4409-bdd7-0f4381f78f32 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:03.343150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769851ba-6cf8-4409-bdd7-0f4381f78f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:03.343564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769851ba-6cf8-4409-bdd7-0f4381f78f32 admin admin] PATH_INFO: `/v3/projects/9e5c6bc240b64b589182038270f9f088/users/7a686affea6c4a75a559464b1bd7abac/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:03.368190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769851ba-6cf8-4409-bdd7-0f4381f78f32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e5c6bc240b64b589182038270f9f088, user_id=7a686affea6c4a75a559464b1bd7abac, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:03.371964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769851ba-6cf8-4409-bdd7-0f4381f78f32 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:03.405486 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 945/1840] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:03 2022] PUT /identity/v3/projects/9e5c6bc240b64b589182038270f9f088/users/7a686affea6c4a75a559464b1bd7abac/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:03.419096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b24503-38d4-4e7e-ae8b-5c8671c60430 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:03.419519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b24503-38d4-4e7e-ae8b-5c8671c60430 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:03.419883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b24503-38d4-4e7e-ae8b-5c8671c60430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:03.496490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e4b24503-38d4-4e7e-ae8b-5c8671c60430 None None] MFA Rules not processed for user `7a686affea6c4a75a559464b1bd7abac`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:03.499923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e4b24503-38d4-4e7e-ae8b-5c8671c60430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:03.552863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769851ba-6cf8-4409-bdd7-0f4381f78f32 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:03.555363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-769851ba-6cf8-4409-bdd7-0f4381f78f32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:03.660222 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 896/1841] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:03 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:03.661781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b24503-38d4-4e7e-ae8b-5c8671c60430 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:03.733629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:03.749152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769851ba-6cf8-4409-bdd7-0f4381f78f32 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:03.768261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a4f27-0dce-43c0-9aee-6ac312222cfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:03.770917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a4f27-0dce-43c0-9aee-6ac312222cfb service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:03.771536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a4f27-0dce-43c0-9aee-6ac312222cfb service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:03.772267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a4f27-0dce-43c0-9aee-6ac312222cfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:03.799973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b24503-38d4-4e7e-ae8b-5c8671c60430 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:03.809408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8403b-f982-4620-ab9f-d31cea8840e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:03.809408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8403b-f982-4620-ab9f-d31cea8840e8 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:03.809702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8403b-f982-4620-ab9f-d31cea8840e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:03.810142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8403b-f982-4620-ab9f-d31cea8840e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:03.811506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8403b-f982-4620-ab9f-d31cea8840e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:03.813070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8403b-f982-4620-ab9f-d31cea8840e8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:03.820638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:03.858913 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 897/1842] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:03 2022] POST /identity/v3/projects => generated 401 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:49:03.860454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d9487-be25-461d-8705-1cf363b22614 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:03.860787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d9487-be25-461d-8705-1cf363b22614 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:03.861102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d9487-be25-461d-8705-1cf363b22614 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:03.863328 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 898/1843] 10.222.0.17 () {54 vars in 921 bytes} [Sat Oct 1 20:49:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:49:03.864170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d9487-be25-461d-8705-1cf363b22614 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:03.887193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a4f27-0dce-43c0-9aee-6ac312222cfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:03.889666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a4f27-0dce-43c0-9aee-6ac312222cfb service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:03.903152 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 946/1844] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:03 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:49:03.918266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134f7da-8281-4dc1-9839-2edf20b554fc None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:03.918266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134f7da-8281-4dc1-9839-2edf20b554fc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:03.920756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134f7da-8281-4dc1-9839-2edf20b554fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:03.955942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d9487-be25-461d-8705-1cf363b22614 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:03.962078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb667dd6-4bc2-478c-ac67-9d17fcb7b62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:03.962688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb667dd6-4bc2-478c-ac67-9d17fcb7b62d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:03.963467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb667dd6-4bc2-478c-ac67-9d17fcb7b62d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:03.963467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb667dd6-4bc2-478c-ac67-9d17fcb7b62d admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:03.973197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb667dd6-4bc2-478c-ac67-9d17fcb7b62d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:03.979371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb667dd6-4bc2-478c-ac67-9d17fcb7b62d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:03.985810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a134f7da-8281-4dc1-9839-2edf20b554fc None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:03.989480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a134f7da-8281-4dc1-9839-2edf20b554fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:04.018617 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 899/1845] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:03 2022] POST /identity/v3/users => generated 307 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:04.019641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb667dd6-4bc2-478c-ac67-9d17fcb7b62d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:04.187602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb667dd6-4bc2-478c-ac67-9d17fcb7b62d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:04.207923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb526c-989a-4015-a839-d48e59c8f955 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:04.209164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb526c-989a-4015-a839-d48e59c8f955 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:04.210095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb526c-989a-4015-a839-d48e59c8f955 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:04.210653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb526c-989a-4015-a839-d48e59c8f955 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:04.238288 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 947/1846] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 20:49:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:04.239090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134f7da-8281-4dc1-9839-2edf20b554fc None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:04.265735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134f7da-8281-4dc1-9839-2edf20b554fc None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:04.276836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3494c9f-cdfe-4e1e-a229-93a9c38092cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:04.277758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3494c9f-cdfe-4e1e-a229-93a9c38092cf admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:04.278325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3494c9f-cdfe-4e1e-a229-93a9c38092cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:04.279012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3494c9f-cdfe-4e1e-a229-93a9c38092cf admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:04.280183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3494c9f-cdfe-4e1e-a229-93a9c38092cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:04.282615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3494c9f-cdfe-4e1e-a229-93a9c38092cf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:04.315834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb526c-989a-4015-a839-d48e59c8f955 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:04.316305 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 948/1847] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:04.321914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb526c-989a-4015-a839-d48e59c8f955 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:04.324193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3494c9f-cdfe-4e1e-a229-93a9c38092cf admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:04.341554 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 900/1848] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:04 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:04.361786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3494c9f-cdfe-4e1e-a229-93a9c38092cf admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:04.378160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4796778-f188-4639-b104-6043ae6251cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:04.378884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4796778-f188-4639-b104-6043ae6251cb admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:04.379170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4796778-f188-4639-b104-6043ae6251cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:04.381885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4796778-f188-4639-b104-6043ae6251cb admin admin] PATH_INFO: `/v3/projects/6de3b6487f874ce2a574f1ff93b7fb06/users/f21b913012c84275bf815435145d15a9/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:04.430663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4796778-f188-4639-b104-6043ae6251cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6de3b6487f874ce2a574f1ff93b7fb06, user_id=f21b913012c84275bf815435145d15a9, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:04.439115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4796778-f188-4639-b104-6043ae6251cb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:04.478114 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 949/1849] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:04 2022] PUT /identity/v3/projects/6de3b6487f874ce2a574f1ff93b7fb06/users/f21b913012c84275bf815435145d15a9/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:04.482830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb526c-989a-4015-a839-d48e59c8f955 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:04.507448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb526c-989a-4015-a839-d48e59c8f955 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:04.519915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:04.645503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed36ac3-0417-42e7-8792-a2f2571918ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:04.647523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed36ac3-0417-42e7-8792-a2f2571918ad admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:04.647820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed36ac3-0417-42e7-8792-a2f2571918ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:04.648093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed36ac3-0417-42e7-8792-a2f2571918ad admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:04.648832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed36ac3-0417-42e7-8792-a2f2571918ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:04.650581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed36ac3-0417-42e7-8792-a2f2571918ad admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:04.666583 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 901/1850] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:04.667745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4796778-f188-4639-b104-6043ae6251cb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:04.695542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4796778-f188-4639-b104-6043ae6251cb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:04.702975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea1759-566c-4842-8d81-76f1f755251d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:04.714292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea1759-566c-4842-8d81-76f1f755251d admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:04.714755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea1759-566c-4842-8d81-76f1f755251d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:04.716087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea1759-566c-4842-8d81-76f1f755251d admin admin] PATH_INFO: `/v3/projects/6de3b6487f874ce2a574f1ff93b7fb06/users/f21b913012c84275bf815435145d15a9/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:04.721495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea1759-566c-4842-8d81-76f1f755251d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6de3b6487f874ce2a574f1ff93b7fb06, user_id=f21b913012c84275bf815435145d15a9, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:04.734276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea1759-566c-4842-8d81-76f1f755251d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:04.781981 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 950/1851] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:04 2022] PUT /identity/v3/projects/6de3b6487f874ce2a574f1ff93b7fb06/users/f21b913012c84275bf815435145d15a9/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:04.793197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05fcc9-1d4b-4866-bd5c-11d07c0cb89f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:04.793565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05fcc9-1d4b-4866-bd5c-11d07c0cb89f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:04.793747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05fcc9-1d4b-4866-bd5c-11d07c0cb89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:04.868223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea1759-566c-4842-8d81-76f1f755251d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:04.906258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ed05fcc9-1d4b-4866-bd5c-11d07c0cb89f None None] MFA Rules not processed for user `f21b913012c84275bf815435145d15a9`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:04.915497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ed05fcc9-1d4b-4866-bd5c-11d07c0cb89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:05.038326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:05.135934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea1759-566c-4842-8d81-76f1f755251d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:05.159342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fef7e-6aab-4250-8153-c64b5cb2706c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:05.161104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fef7e-6aab-4250-8153-c64b5cb2706c service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:05.161355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fef7e-6aab-4250-8153-c64b5cb2706c service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:05.161635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fef7e-6aab-4250-8153-c64b5cb2706c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:05.168996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ab6fef7e-6aab-4250-8153-c64b5cb2706c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:05.178886 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 902/1852] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:04 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:05.367128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fef7e-6aab-4250-8153-c64b5cb2706c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:05.368885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fef7e-6aab-4250-8153-c64b5cb2706c service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:05.394724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0afa0-0843-4f90-9648-797d1142baad None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:05.395853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0afa0-0843-4f90-9648-797d1142baad None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:05.396351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0afa0-0843-4f90-9648-797d1142baad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:05.403164 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 951/1853] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:04 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:05.467230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-19e0afa0-0843-4f90-9648-797d1142baad None None] MFA Rules not processed for user `e17a026f305248878f18eec35e1e02dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:05.470661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-19e0afa0-0843-4f90-9648-797d1142baad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:05.568467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fef7e-6aab-4250-8153-c64b5cb2706c service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:05.585709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fef7e-6aab-4250-8153-c64b5cb2706c service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:05.603075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae5152-3fa8-400b-a05c-05bab7fdaaaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:05.603704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae5152-3fa8-400b-a05c-05bab7fdaaaf service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:05.604375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae5152-3fa8-400b-a05c-05bab7fdaaaf service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:05.604609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae5152-3fa8-400b-a05c-05bab7fdaaaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:05.625327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae5152-3fa8-400b-a05c-05bab7fdaaaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:05.625327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae5152-3fa8-400b-a05c-05bab7fdaaaf service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:05.647957 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 903/1854] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:49:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:05.655974 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 952/1855] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:49:05.661910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0afa0-0843-4f90-9648-797d1142baad None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:05.663417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-19e0afa0-0843-4f90-9648-797d1142baad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:05.718894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0afa0-0843-4f90-9648-797d1142baad None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:05.723138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126a657-3bd2-4368-bce1-213338beb744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:05.723756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126a657-3bd2-4368-bce1-213338beb744 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:05.724034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126a657-3bd2-4368-bce1-213338beb744 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:05.724306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126a657-3bd2-4368-bce1-213338beb744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:05.744209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126a657-3bd2-4368-bce1-213338beb744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:05.749901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126a657-3bd2-4368-bce1-213338beb744 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:05.760142 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 904/1856] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:06.160170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae5152-3fa8-400b-a05c-05bab7fdaaaf service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:06.177784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae5152-3fa8-400b-a05c-05bab7fdaaaf service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:06.185130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d2078-f6bc-492e-8dd5-b4e274853941 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:06.185746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d2078-f6bc-492e-8dd5-b4e274853941 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:06.186029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d2078-f6bc-492e-8dd5-b4e274853941 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:06.186310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d2078-f6bc-492e-8dd5-b4e274853941 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:06.195183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d2078-f6bc-492e-8dd5-b4e274853941 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:06.196743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d2078-f6bc-492e-8dd5-b4e274853941 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:06.207942 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 953/1857] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:06 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:06.335711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126a657-3bd2-4368-bce1-213338beb744 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:06.374534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:06.461005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d2078-f6bc-492e-8dd5-b4e274853941 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:06.485183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d2078-f6bc-492e-8dd5-b4e274853941 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:06.487565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126a657-3bd2-4368-bce1-213338beb744 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:06.497663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e668eb-7539-419c-8df7-6d9b029d247b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:06.498593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e668eb-7539-419c-8df7-6d9b029d247b service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:06.499034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e668eb-7539-419c-8df7-6d9b029d247b service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:06.499461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e668eb-7539-419c-8df7-6d9b029d247b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:06.502899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca833f15-caad-40f7-b31e-bd3a5634bcf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:06.503792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca833f15-caad-40f7-b31e-bd3a5634bcf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:06.504177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca833f15-caad-40f7-b31e-bd3a5634bcf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:06.504841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca833f15-caad-40f7-b31e-bd3a5634bcf0 admin admin] PATH_INFO: `/v3/users/64bded24bc1c4d6f832416230591e98f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:06.508571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca833f15-caad-40f7-b31e-bd3a5634bcf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64bded24bc1c4d6f832416230591e98f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:06.516364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca833f15-caad-40f7-b31e-bd3a5634bcf0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:06.627141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ca833f15-caad-40f7-b31e-bd3a5634bcf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64bded24bc1c4d6f832416230591e98f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:06.630901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ca833f15-caad-40f7-b31e-bd3a5634bcf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64bded24bc1c4d6f832416230591e98f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:06.630901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ca833f15-caad-40f7-b31e-bd3a5634bcf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64bded24bc1c4d6f832416230591e98f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:06.650571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ca833f15-caad-40f7-b31e-bd3a5634bcf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64bded24bc1c4d6f832416230591e98f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:06.682456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ca833f15-caad-40f7-b31e-bd3a5634bcf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64bded24bc1c4d6f832416230591e98f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:06.699915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e668eb-7539-419c-8df7-6d9b029d247b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:06.701393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e668eb-7539-419c-8df7-6d9b029d247b service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:06.714111 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 954/1858] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:06 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:06.722126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e668eb-7539-419c-8df7-6d9b029d247b service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:06.724422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c8e668eb-7539-419c-8df7-6d9b029d247b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:06.734018 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 905/1859] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:06 2022] DELETE /identity/v3/users/64bded24bc1c4d6f832416230591e98f => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:06.738863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b6ea9-7d08-4a19-acb7-c28baae8c6fd None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:06.739144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b6ea9-7d08-4a19-acb7-c28baae8c6fd None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:06.739425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b6ea9-7d08-4a19-acb7-c28baae8c6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:06.766162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e668eb-7539-419c-8df7-6d9b029d247b service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:06.810249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e41b6ea9-7d08-4a19-acb7-c28baae8c6fd None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:06.811445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e41b6ea9-7d08-4a19-acb7-c28baae8c6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:06.940934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:06.981907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149257d4-51f4-480f-a000-357aaeb9118b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:06.988672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149257d4-51f4-480f-a000-357aaeb9118b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:06.989455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149257d4-51f4-480f-a000-357aaeb9118b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:06.989751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149257d4-51f4-480f-a000-357aaeb9118b admin admin] PATH_INFO: `/v3/users/d6d3ae72992a48709e3d58930ccd6b52` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:06.994208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149257d4-51f4-480f-a000-357aaeb9118b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6d3ae72992a48709e3d58930ccd6b52)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:07.001200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149257d4-51f4-480f-a000-357aaeb9118b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:07.141281 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 906/1860] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:07.179464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b6ea9-7d08-4a19-acb7-c28baae8c6fd None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:07.188922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e41b6ea9-7d08-4a19-acb7-c28baae8c6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:07.220631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-149257d4-51f4-480f-a000-357aaeb9118b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6d3ae72992a48709e3d58930ccd6b52'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:07.239659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-149257d4-51f4-480f-a000-357aaeb9118b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6d3ae72992a48709e3d58930ccd6b52'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:07.282902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-149257d4-51f4-480f-a000-357aaeb9118b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6d3ae72992a48709e3d58930ccd6b52'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:07.283970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-149257d4-51f4-480f-a000-357aaeb9118b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6d3ae72992a48709e3d58930ccd6b52'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:07.302884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-149257d4-51f4-480f-a000-357aaeb9118b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6d3ae72992a48709e3d58930ccd6b52'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:07.324612 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 955/1861] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:06 2022] DELETE /identity/v3/users/d6d3ae72992a48709e3d58930ccd6b52 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:49:07.327201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149257d4-51f4-480f-a000-357aaeb9118b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:07.347034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-149257d4-51f4-480f-a000-357aaeb9118b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:07.403425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:07.442741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b6ea9-7d08-4a19-acb7-c28baae8c6fd None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:07.556896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149257d4-51f4-480f-a000-357aaeb9118b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:07.564499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a349566-d08c-49a6-acc0-2a645b313e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:07.567933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a349566-d08c-49a6-acc0-2a645b313e3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:07.568585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a349566-d08c-49a6-acc0-2a645b313e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:07.570037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a349566-d08c-49a6-acc0-2a645b313e3c admin admin] PATH_INFO: `/v3/users/35e45ec47cd648649829660999769d0c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:07.572081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a349566-d08c-49a6-acc0-2a645b313e3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=35e45ec47cd648649829660999769d0c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:07.575974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a349566-d08c-49a6-acc0-2a645b313e3c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:07.607445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260c8a5-a567-4801-80cf-3bd00e823683 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:07.610084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260c8a5-a567-4801-80cf-3bd00e823683 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:07.610084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260c8a5-a567-4801-80cf-3bd00e823683 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:07.610084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260c8a5-a567-4801-80cf-3bd00e823683 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:07.616047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b260c8a5-a567-4801-80cf-3bd00e823683 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:07.645992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b260c8a5-a567-4801-80cf-3bd00e823683 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:07.648161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b260c8a5-a567-4801-80cf-3bd00e823683 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:07.674534 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 907/1862] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:07.677478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2d96a-a7dc-477e-af63-8902cedcb689 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:07.677817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2d96a-a7dc-477e-af63-8902cedcb689 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:07.678587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2d96a-a7dc-477e-af63-8902cedcb689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:07.683293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1a349566-d08c-49a6-acc0-2a645b313e3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35e45ec47cd648649829660999769d0c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:07.699388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1a349566-d08c-49a6-acc0-2a645b313e3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35e45ec47cd648649829660999769d0c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:07.719299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:07.721536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1a349566-d08c-49a6-acc0-2a645b313e3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35e45ec47cd648649829660999769d0c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:07.722595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1a349566-d08c-49a6-acc0-2a645b313e3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35e45ec47cd648649829660999769d0c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:07.723164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fce2d96a-a7dc-477e-af63-8902cedcb689 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:07.724520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fce2d96a-a7dc-477e-af63-8902cedcb689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:07.734889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1a349566-d08c-49a6-acc0-2a645b313e3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35e45ec47cd648649829660999769d0c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:07.737629 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 908/1863] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:07.755341 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 956/1864] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:07 2022] DELETE /identity/v3/users/35e45ec47cd648649829660999769d0c => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:07.761991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d505b-0c1f-4198-a558-12b114eeee0b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:07.762286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d505b-0c1f-4198-a558-12b114eeee0b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:07.762611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d505b-0c1f-4198-a558-12b114eeee0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:07.776885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a349566-d08c-49a6-acc0-2a645b313e3c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:07.779535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1a349566-d08c-49a6-acc0-2a645b313e3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:07.810388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b89d505b-0c1f-4198-a558-12b114eeee0b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:07.814852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b89d505b-0c1f-4198-a558-12b114eeee0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:07.924114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a349566-d08c-49a6-acc0-2a645b313e3c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:07.931236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d171c2-c9f4-4fd7-a7d6-5dee2e50ad15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:07.932966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d171c2-c9f4-4fd7-a7d6-5dee2e50ad15 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:07.932966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d171c2-c9f4-4fd7-a7d6-5dee2e50ad15 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:07.932966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d171c2-c9f4-4fd7-a7d6-5dee2e50ad15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:08.036129 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 909/1865] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:08.037632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d505b-0c1f-4198-a558-12b114eeee0b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:08.039452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b89d505b-0c1f-4198-a558-12b114eeee0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:08.053761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d171c2-c9f4-4fd7-a7d6-5dee2e50ad15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:08.055602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d171c2-c9f4-4fd7-a7d6-5dee2e50ad15 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:08.058728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d505b-0c1f-4198-a558-12b114eeee0b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:08.064432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62601716-67ba-47df-9602-e1816d45af89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:08.065173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62601716-67ba-47df-9602-e1816d45af89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:08.065466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62601716-67ba-47df-9602-e1816d45af89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:08.065730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62601716-67ba-47df-9602-e1816d45af89 admin admin] PATH_INFO: `/v3/users/7a686affea6c4a75a559464b1bd7abac` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:08.067329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62601716-67ba-47df-9602-e1816d45af89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a686affea6c4a75a559464b1bd7abac)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:08.073716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62601716-67ba-47df-9602-e1816d45af89 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:08.077441 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 957/1866] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:08.078422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d171c2-c9f4-4fd7-a7d6-5dee2e50ad15 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:08.079897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-27d171c2-c9f4-4fd7-a7d6-5dee2e50ad15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:08.091911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d171c2-c9f4-4fd7-a7d6-5dee2e50ad15 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:08.096087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d86d4f-690a-4125-85cc-0ef061ddd90e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:08.096874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d86d4f-690a-4125-85cc-0ef061ddd90e service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:08.097178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d86d4f-690a-4125-85cc-0ef061ddd90e service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:08.097382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d86d4f-690a-4125-85cc-0ef061ddd90e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:08.113422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d86d4f-690a-4125-85cc-0ef061ddd90e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:08.114950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d86d4f-690a-4125-85cc-0ef061ddd90e service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:08.125522 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 958/1867] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:08.142999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-62601716-67ba-47df-9602-e1816d45af89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a686affea6c4a75a559464b1bd7abac'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:08.143693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-62601716-67ba-47df-9602-e1816d45af89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a686affea6c4a75a559464b1bd7abac'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:08.144323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-62601716-67ba-47df-9602-e1816d45af89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a686affea6c4a75a559464b1bd7abac'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:08.153400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-62601716-67ba-47df-9602-e1816d45af89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a686affea6c4a75a559464b1bd7abac'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:08.176944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-62601716-67ba-47df-9602-e1816d45af89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a686affea6c4a75a559464b1bd7abac'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:08.199149 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 910/1868] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:08 2022] DELETE /identity/v3/users/7a686affea6c4a75a559464b1bd7abac => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:08.204173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20140c-3b2f-4a10-b625-ea4b7eccd081 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:08.204727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20140c-3b2f-4a10-b625-ea4b7eccd081 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:08.205207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20140c-3b2f-4a10-b625-ea4b7eccd081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:08.237777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fc20140c-3b2f-4a10-b625-ea4b7eccd081 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:08.241652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fc20140c-3b2f-4a10-b625-ea4b7eccd081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:08.334239 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 959/1869] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:08.363660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62601716-67ba-47df-9602-e1816d45af89 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:08.365986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-62601716-67ba-47df-9602-e1816d45af89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:08.381832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:08.494477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62601716-67ba-47df-9602-e1816d45af89 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:08.499612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc82c5c-4a78-4439-97bc-741d99360b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:08.500238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc82c5c-4a78-4439-97bc-741d99360b55 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:08.500692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc82c5c-4a78-4439-97bc-741d99360b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:08.500806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc82c5c-4a78-4439-97bc-741d99360b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:08.501614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20140c-3b2f-4a10-b625-ea4b7eccd081 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:08.503118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fc20140c-3b2f-4a10-b625-ea4b7eccd081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:08.531644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20140c-3b2f-4a10-b625-ea4b7eccd081 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:08.532868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc82c5c-4a78-4439-97bc-741d99360b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:08.535910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc82c5c-4a78-4439-97bc-741d99360b55 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:08.544181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6cd6a8-a698-47f3-bf4b-41e8d5c94bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:08.544842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cd6a8-a698-47f3-bf4b-41e8d5c94bec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:08.545126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cd6a8-a698-47f3-bf4b-41e8d5c94bec admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:08.545401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cd6a8-a698-47f3-bf4b-41e8d5c94bec admin admin] PATH_INFO: `/v3/projects/e82c68b8617a4b3cbc43b18d52f56842` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:08.548146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6cd6a8-a698-47f3-bf4b-41e8d5c94bec admin admin] RBAC: Authorizing `identity:delete_project(project_id=e82c68b8617a4b3cbc43b18d52f56842)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:08.550088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6cd6a8-a698-47f3-bf4b-41e8d5c94bec admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:08.555762 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 911/1870] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:08.583026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:08.618337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7d6cd6a8-a698-47f3-bf4b-41e8d5c94bec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e82c68b8617a4b3cbc43b18d52f56842', 'request_id': 'req-7d6cd6a8-a698-47f3-bf4b-41e8d5c94bec'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:08.630752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7d6cd6a8-a698-47f3-bf4b-41e8d5c94bec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e82c68b8617a4b3cbc43b18d52f56842', 'request_id': 'req-7d6cd6a8-a698-47f3-bf4b-41e8d5c94bec'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:08.638168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7d6cd6a8-a698-47f3-bf4b-41e8d5c94bec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e82c68b8617a4b3cbc43b18d52f56842', 'request_id': 'req-7d6cd6a8-a698-47f3-bf4b-41e8d5c94bec'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:08.660191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7d6cd6a8-a698-47f3-bf4b-41e8d5c94bec admin admin] The token cache is being invalidate because project e82c68b8617a4b3cbc43b18d52f56842 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:08.660534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7d6cd6a8-a698-47f3-bf4b-41e8d5c94bec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:08.661281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7d6cd6a8-a698-47f3-bf4b-41e8d5c94bec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:08.663336 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 960/1871] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:08 2022] DELETE /identity/v3/projects/e82c68b8617a4b3cbc43b18d52f56842 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:49:08.667324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc82c5c-4a78-4439-97bc-741d99360b55 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:08.683499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc82c5c-4a78-4439-97bc-741d99360b55 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:08.735785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6cd6a8-a698-47f3-bf4b-41e8d5c94bec admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:08.737493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7d6cd6a8-a698-47f3-bf4b-41e8d5c94bec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:08.795436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef2743-a75d-4d0f-af4d-9a7970d89610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:08.796492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef2743-a75d-4d0f-af4d-9a7970d89610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:08.796813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef2743-a75d-4d0f-af4d-9a7970d89610 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:08.797167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef2743-a75d-4d0f-af4d-9a7970d89610 admin admin] PATH_INFO: `/v3/users/c3523161149c444f826a7c9d332cb6b8` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:08.798700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef2743-a75d-4d0f-af4d-9a7970d89610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3523161149c444f826a7c9d332cb6b8)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:08.801677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef2743-a75d-4d0f-af4d-9a7970d89610 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:08.897098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a5ef2743-a75d-4d0f-af4d-9a7970d89610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3523161149c444f826a7c9d332cb6b8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:08.900469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a5ef2743-a75d-4d0f-af4d-9a7970d89610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3523161149c444f826a7c9d332cb6b8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:08.901653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a5ef2743-a75d-4d0f-af4d-9a7970d89610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3523161149c444f826a7c9d332cb6b8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:08.912624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6cd6a8-a698-47f3-bf4b-41e8d5c94bec admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:08.918531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a5ef2743-a75d-4d0f-af4d-9a7970d89610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3523161149c444f826a7c9d332cb6b8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:08.924417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb9650-0fb6-4d5e-9f7c-d6236ac25b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:08.925087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb9650-0fb6-4d5e-9f7c-d6236ac25b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:08.926235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb9650-0fb6-4d5e-9f7c-d6236ac25b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:08.926967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb9650-0fb6-4d5e-9f7c-d6236ac25b66 admin admin] PATH_INFO: `/v3/projects/b331e32eb5964145bde96b9dce962819` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:08.928280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb9650-0fb6-4d5e-9f7c-d6236ac25b66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b331e32eb5964145bde96b9dce962819)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:08.937078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb9650-0fb6-4d5e-9f7c-d6236ac25b66 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:08.950903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a5ef2743-a75d-4d0f-af4d-9a7970d89610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3523161149c444f826a7c9d332cb6b8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:08.973759 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 912/1872] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:08 2022] DELETE /identity/v3/users/c3523161149c444f826a7c9d332cb6b8 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:09.048595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef2743-a75d-4d0f-af4d-9a7970d89610 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:09.051261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a5ef2743-a75d-4d0f-af4d-9a7970d89610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:09.051735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-44eb9650-0fb6-4d5e-9f7c-d6236ac25b66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b331e32eb5964145bde96b9dce962819', 'request_id': 'req-44eb9650-0fb6-4d5e-9f7c-d6236ac25b66'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.066032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-44eb9650-0fb6-4d5e-9f7c-d6236ac25b66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b331e32eb5964145bde96b9dce962819', 'request_id': 'req-44eb9650-0fb6-4d5e-9f7c-d6236ac25b66'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.076807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-44eb9650-0fb6-4d5e-9f7c-d6236ac25b66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b331e32eb5964145bde96b9dce962819', 'request_id': 'req-44eb9650-0fb6-4d5e-9f7c-d6236ac25b66'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.118951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-44eb9650-0fb6-4d5e-9f7c-d6236ac25b66 admin admin] The token cache is being invalidate because project b331e32eb5964145bde96b9dce962819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:09.119257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-44eb9650-0fb6-4d5e-9f7c-d6236ac25b66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.120916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-44eb9650-0fb6-4d5e-9f7c-d6236ac25b66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.125897 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 961/1873] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:08 2022] DELETE /identity/v3/projects/b331e32eb5964145bde96b9dce962819 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:49:09.140724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb9650-0fb6-4d5e-9f7c-d6236ac25b66 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:09.146609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-44eb9650-0fb6-4d5e-9f7c-d6236ac25b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:09.149768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:09.204146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef2743-a75d-4d0f-af4d-9a7970d89610 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:09.281504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00277686-a571-4dd1-97f4-28b128f10b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:09.282454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00277686-a571-4dd1-97f4-28b128f10b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:09.282868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00277686-a571-4dd1-97f4-28b128f10b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:09.283255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00277686-a571-4dd1-97f4-28b128f10b2e admin admin] PATH_INFO: `/v3/projects/598c3f5d2363441fb1a2fc8f2589544f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:09.283481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:09.285592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00277686-a571-4dd1-97f4-28b128f10b2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=598c3f5d2363441fb1a2fc8f2589544f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:09.299544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00277686-a571-4dd1-97f4-28b128f10b2e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:09.315391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb9650-0fb6-4d5e-9f7c-d6236ac25b66 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:09.323290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e249a-0b2e-42f9-9574-732f3985ecb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:09.324535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e249a-0b2e-42f9-9574-732f3985ecb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:09.325043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e249a-0b2e-42f9-9574-732f3985ecb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:09.325759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e249a-0b2e-42f9-9574-732f3985ecb7 admin admin] PATH_INFO: `/v3/users/4e27033a708c40c8b262464105e126e6` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:09.328026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e249a-0b2e-42f9-9574-732f3985ecb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e27033a708c40c8b262464105e126e6)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:09.345462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e249a-0b2e-42f9-9574-732f3985ecb7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:09.408025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00277686-a571-4dd1-97f4-28b128f10b2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '598c3f5d2363441fb1a2fc8f2589544f', 'request_id': 'req-00277686-a571-4dd1-97f4-28b128f10b2e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.418279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00277686-a571-4dd1-97f4-28b128f10b2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '598c3f5d2363441fb1a2fc8f2589544f', 'request_id': 'req-00277686-a571-4dd1-97f4-28b128f10b2e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.425956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00277686-a571-4dd1-97f4-28b128f10b2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '598c3f5d2363441fb1a2fc8f2589544f', 'request_id': 'req-00277686-a571-4dd1-97f4-28b128f10b2e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.445845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-922e249a-0b2e-42f9-9574-732f3985ecb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e27033a708c40c8b262464105e126e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.453161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00277686-a571-4dd1-97f4-28b128f10b2e admin admin] The token cache is being invalidate because project 598c3f5d2363441fb1a2fc8f2589544f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:09.453487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00277686-a571-4dd1-97f4-28b128f10b2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.454208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00277686-a571-4dd1-97f4-28b128f10b2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.456078 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 913/1874] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:09 2022] DELETE /identity/v3/projects/598c3f5d2363441fb1a2fc8f2589544f => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:09.462215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00277686-a571-4dd1-97f4-28b128f10b2e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:09.465229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-00277686-a571-4dd1-97f4-28b128f10b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:09.466611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-922e249a-0b2e-42f9-9574-732f3985ecb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e27033a708c40c8b262464105e126e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.493435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-922e249a-0b2e-42f9-9574-732f3985ecb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e27033a708c40c8b262464105e126e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.495232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-922e249a-0b2e-42f9-9574-732f3985ecb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e27033a708c40c8b262464105e126e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.506205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-922e249a-0b2e-42f9-9574-732f3985ecb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e27033a708c40c8b262464105e126e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.521208 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 962/1875] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:09 2022] DELETE /identity/v3/users/4e27033a708c40c8b262464105e126e6 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:09.522248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e249a-0b2e-42f9-9574-732f3985ecb7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:09.525037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-922e249a-0b2e-42f9-9574-732f3985ecb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:09.640412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00277686-a571-4dd1-97f4-28b128f10b2e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:09.668773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e249a-0b2e-42f9-9574-732f3985ecb7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:09.669333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:09.676500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79e315-f874-447d-b9c5-f4adc1354f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:09.677158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79e315-f874-447d-b9c5-f4adc1354f99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:09.677444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79e315-f874-447d-b9c5-f4adc1354f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:09.678258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79e315-f874-447d-b9c5-f4adc1354f99 admin admin] PATH_INFO: `/v3/projects/9e5c6bc240b64b589182038270f9f088` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:09.679871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79e315-f874-447d-b9c5-f4adc1354f99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e5c6bc240b64b589182038270f9f088)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:09.682009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79e315-f874-447d-b9c5-f4adc1354f99 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:09.753677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1d79e315-f874-447d-b9c5-f4adc1354f99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e5c6bc240b64b589182038270f9f088', 'request_id': 'req-1d79e315-f874-447d-b9c5-f4adc1354f99'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.771617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1d79e315-f874-447d-b9c5-f4adc1354f99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e5c6bc240b64b589182038270f9f088', 'request_id': 'req-1d79e315-f874-447d-b9c5-f4adc1354f99'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.780620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:09.782932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1d79e315-f874-447d-b9c5-f4adc1354f99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e5c6bc240b64b589182038270f9f088', 'request_id': 'req-1d79e315-f874-447d-b9c5-f4adc1354f99'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.800180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018c36a-7c39-4290-b39c-b6b8d542014c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:09.801014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018c36a-7c39-4290-b39c-b6b8d542014c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:09.801362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018c36a-7c39-4290-b39c-b6b8d542014c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:09.801850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018c36a-7c39-4290-b39c-b6b8d542014c admin admin] PATH_INFO: `/v3/users/6cbdb512aa12495b96d80f245efb6012` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:09.803272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018c36a-7c39-4290-b39c-b6b8d542014c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cbdb512aa12495b96d80f245efb6012)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:09.803628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1d79e315-f874-447d-b9c5-f4adc1354f99 admin admin] The token cache is being invalidate because project 9e5c6bc240b64b589182038270f9f088 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:09.804013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1d79e315-f874-447d-b9c5-f4adc1354f99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.804753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1d79e315-f874-447d-b9c5-f4adc1354f99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.805661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018c36a-7c39-4290-b39c-b6b8d542014c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:09.810059 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 963/1876] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:09 2022] DELETE /identity/v3/projects/9e5c6bc240b64b589182038270f9f088 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:09.812633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79e315-f874-447d-b9c5-f4adc1354f99 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:09.815015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1d79e315-f874-447d-b9c5-f4adc1354f99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:09.887940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5018c36a-7c39-4290-b39c-b6b8d542014c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cbdb512aa12495b96d80f245efb6012'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.888871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5018c36a-7c39-4290-b39c-b6b8d542014c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cbdb512aa12495b96d80f245efb6012'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.889573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5018c36a-7c39-4290-b39c-b6b8d542014c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cbdb512aa12495b96d80f245efb6012'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.902302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5018c36a-7c39-4290-b39c-b6b8d542014c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cbdb512aa12495b96d80f245efb6012'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.920878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79e315-f874-447d-b9c5-f4adc1354f99 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:09.925473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5018c36a-7c39-4290-b39c-b6b8d542014c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cbdb512aa12495b96d80f245efb6012'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:09.926185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15fbfb-166d-4f2f-8511-58deda325415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:09.926987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15fbfb-166d-4f2f-8511-58deda325415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:09.927716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15fbfb-166d-4f2f-8511-58deda325415 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:09.928032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15fbfb-166d-4f2f-8511-58deda325415 admin admin] PATH_INFO: `/v3/users/f21b913012c84275bf815435145d15a9` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:09.929700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d15fbfb-166d-4f2f-8511-58deda325415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f21b913012c84275bf815435145d15a9)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:09.932730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d15fbfb-166d-4f2f-8511-58deda325415 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:09.949941 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 914/1877] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:09 2022] DELETE /identity/v3/users/6cbdb512aa12495b96d80f245efb6012 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:09.981274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018c36a-7c39-4290-b39c-b6b8d542014c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:09.983014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5018c36a-7c39-4290-b39c-b6b8d542014c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:10.019991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3d15fbfb-166d-4f2f-8511-58deda325415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f21b913012c84275bf815435145d15a9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:10.035288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3d15fbfb-166d-4f2f-8511-58deda325415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f21b913012c84275bf815435145d15a9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:10.059011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3d15fbfb-166d-4f2f-8511-58deda325415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f21b913012c84275bf815435145d15a9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:10.061755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3d15fbfb-166d-4f2f-8511-58deda325415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f21b913012c84275bf815435145d15a9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:10.078396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3d15fbfb-166d-4f2f-8511-58deda325415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f21b913012c84275bf815435145d15a9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:10.096384 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 964/1878] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:09 2022] DELETE /identity/v3/users/f21b913012c84275bf815435145d15a9 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:49:10.102986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018c36a-7c39-4290-b39c-b6b8d542014c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:10.150357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15fbfb-166d-4f2f-8511-58deda325415 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:10.152236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3d15fbfb-166d-4f2f-8511-58deda325415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:10.168164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:10.210195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e4a5a-d7b1-4a4e-8387-f8c059a878e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:10.210961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e4a5a-d7b1-4a4e-8387-f8c059a878e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:10.211251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e4a5a-d7b1-4a4e-8387-f8c059a878e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:10.211527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e4a5a-d7b1-4a4e-8387-f8c059a878e7 admin admin] PATH_INFO: `/v3/projects/479f3dd136484b2f8cf66d951ce0ee58` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:10.214668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e4a5a-d7b1-4a4e-8387-f8c059a878e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=479f3dd136484b2f8cf66d951ce0ee58)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:10.217643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e4a5a-d7b1-4a4e-8387-f8c059a878e7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:10.320299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15fbfb-166d-4f2f-8511-58deda325415 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:10.329654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-215e4a5a-d7b1-4a4e-8387-f8c059a878e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '479f3dd136484b2f8cf66d951ce0ee58', 'request_id': 'req-215e4a5a-d7b1-4a4e-8387-f8c059a878e7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:10.342124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:10.347640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-215e4a5a-d7b1-4a4e-8387-f8c059a878e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '479f3dd136484b2f8cf66d951ce0ee58', 'request_id': 'req-215e4a5a-d7b1-4a4e-8387-f8c059a878e7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:10.364175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-215e4a5a-d7b1-4a4e-8387-f8c059a878e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '479f3dd136484b2f8cf66d951ce0ee58', 'request_id': 'req-215e4a5a-d7b1-4a4e-8387-f8c059a878e7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:10.398170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-215e4a5a-d7b1-4a4e-8387-f8c059a878e7 admin admin] The token cache is being invalidate because project 479f3dd136484b2f8cf66d951ce0ee58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:10.398571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-215e4a5a-d7b1-4a4e-8387-f8c059a878e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:10.400976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-215e4a5a-d7b1-4a4e-8387-f8c059a878e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:10.403530 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 915/1879] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:09 2022] DELETE /identity/v3/projects/479f3dd136484b2f8cf66d951ce0ee58 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:10.408081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7d531-2195-4590-a23c-8899fa644881 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:10.408368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7d531-2195-4590-a23c-8899fa644881 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:10.408647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7d531-2195-4590-a23c-8899fa644881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:10.416033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fad3ff-b8a3-45a0-86f3-abff84feadeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:10.417871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fad3ff-b8a3-45a0-86f3-abff84feadeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:10.418397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fad3ff-b8a3-45a0-86f3-abff84feadeb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:10.420115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fad3ff-b8a3-45a0-86f3-abff84feadeb admin admin] PATH_INFO: `/v3/projects/bde2c758b4b4486881f4026201c90359` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:10.421640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fad3ff-b8a3-45a0-86f3-abff84feadeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=bde2c758b4b4486881f4026201c90359)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:10.425181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fad3ff-b8a3-45a0-86f3-abff84feadeb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:10.443778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7ce7d531-2195-4590-a23c-8899fa644881 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:10.445112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7ce7d531-2195-4590-a23c-8899fa644881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:10.461638 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 916/1880] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:49:10.470414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac83cdf-52d4-4168-b786-f33e5e48ef76 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:10.470989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac83cdf-52d4-4168-b786-f33e5e48ef76 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:10.471312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac83cdf-52d4-4168-b786-f33e5e48ef76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:10.501706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5ac83cdf-52d4-4168-b786-f33e5e48ef76 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:10.502804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5ac83cdf-52d4-4168-b786-f33e5e48ef76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:10.504448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-03fad3ff-b8a3-45a0-86f3-abff84feadeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bde2c758b4b4486881f4026201c90359', 'request_id': 'req-03fad3ff-b8a3-45a0-86f3-abff84feadeb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:10.515510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-03fad3ff-b8a3-45a0-86f3-abff84feadeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bde2c758b4b4486881f4026201c90359', 'request_id': 'req-03fad3ff-b8a3-45a0-86f3-abff84feadeb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:10.528173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-03fad3ff-b8a3-45a0-86f3-abff84feadeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bde2c758b4b4486881f4026201c90359', 'request_id': 'req-03fad3ff-b8a3-45a0-86f3-abff84feadeb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:10.584397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-03fad3ff-b8a3-45a0-86f3-abff84feadeb admin admin] The token cache is being invalidate because project bde2c758b4b4486881f4026201c90359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:10.586315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-03fad3ff-b8a3-45a0-86f3-abff84feadeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:10.588125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-03fad3ff-b8a3-45a0-86f3-abff84feadeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:10.598532 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 965/1881] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:10 2022] DELETE /identity/v3/projects/bde2c758b4b4486881f4026201c90359 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:49:10.600522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461e369-109f-4308-84d5-bef3037c7e2a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:10.600955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461e369-109f-4308-84d5-bef3037c7e2a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:10.601526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461e369-109f-4308-84d5-bef3037c7e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:10.630172 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 917/1882] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:10.631231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac83cdf-52d4-4168-b786-f33e5e48ef76 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:10.660989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac83cdf-52d4-4168-b786-f33e5e48ef76 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:10.667919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0461e369-109f-4308-84d5-bef3037c7e2a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:10.669714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0461e369-109f-4308-84d5-bef3037c7e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:10.676946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f2ab6-d35d-441b-ba46-2c801eec8159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:10.677595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f2ab6-d35d-441b-ba46-2c801eec8159 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:10.677876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f2ab6-d35d-441b-ba46-2c801eec8159 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:10.678149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f2ab6-d35d-441b-ba46-2c801eec8159 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:10.678922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f2ab6-d35d-441b-ba46-2c801eec8159 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:49:10.679468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f2ab6-d35d-441b-ba46-2c801eec8159 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:10.680841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f2ab6-d35d-441b-ba46-2c801eec8159 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:10.694044 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 966/1883] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:49:10.695938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ac1de-8aba-453f-8e20-acdca519fd01 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:10.695938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ac1de-8aba-453f-8e20-acdca519fd01 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:10.696113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ac1de-8aba-453f-8e20-acdca519fd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:10.699917 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 918/1884] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:49:10.704495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f2ab6-d35d-441b-ba46-2c801eec8159 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:10.728682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-682ac1de-8aba-453f-8e20-acdca519fd01 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:10.732361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-682ac1de-8aba-453f-8e20-acdca519fd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:10.740555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f2ab6-d35d-441b-ba46-2c801eec8159 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:10.752162 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 967/1885] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:10.754976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6e5dc-30c8-408b-ab5a-c622973e51ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:10.755945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6e5dc-30c8-408b-ab5a-c622973e51ea admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:10.756299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6e5dc-30c8-408b-ab5a-c622973e51ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:10.756556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ac1de-8aba-453f-8e20-acdca519fd01 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:10.756664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6e5dc-30c8-408b-ab5a-c622973e51ea admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:10.757264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d6e5dc-30c8-408b-ab5a-c622973e51ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:10.759313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d6e5dc-30c8-408b-ab5a-c622973e51ea admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:10.773855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ac1de-8aba-453f-8e20-acdca519fd01 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:10.781833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e78e2-f32c-4021-896a-a599456778bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:10.782679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e78e2-f32c-4021-896a-a599456778bf admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:10.782997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e78e2-f32c-4021-896a-a599456778bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:10.784085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e78e2-f32c-4021-896a-a599456778bf admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:10.784356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e78e2-f32c-4021-896a-a599456778bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:49:10.785312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e78e2-f32c-4021-896a-a599456778bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:10.786727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e78e2-f32c-4021-896a-a599456778bf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:10.817618 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 968/1886] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:49:10.819050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e78e2-f32c-4021-896a-a599456778bf admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:10.820682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-399e78e2-f32c-4021-896a-a599456778bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:10.848058 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 919/1887] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:10 2022] POST /identity/v3/projects => generated 391 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:10.849932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6e5dc-30c8-408b-ab5a-c622973e51ea admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:10.853096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e78e2-f32c-4021-896a-a599456778bf admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:11.005737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a750d51-61c4-4425-b4a4-2473396664df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:11.006593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a750d51-61c4-4425-b4a4-2473396664df admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:11.006883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a750d51-61c4-4425-b4a4-2473396664df admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:11.007485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a750d51-61c4-4425-b4a4-2473396664df admin admin] PATH_INFO: `/v3/projects/51f59d1267df48309fe64b4d960720bb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:11.010671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a750d51-61c4-4425-b4a4-2473396664df admin admin] RBAC: Authorizing `identity:delete_project(project_id=51f59d1267df48309fe64b4d960720bb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:11.013671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a750d51-61c4-4425-b4a4-2473396664df admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:11.049499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:11.076693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:11.097822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a750d51-61c4-4425-b4a4-2473396664df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51f59d1267df48309fe64b4d960720bb', 'request_id': 'req-9a750d51-61c4-4425-b4a4-2473396664df'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:11.121715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a750d51-61c4-4425-b4a4-2473396664df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51f59d1267df48309fe64b4d960720bb', 'request_id': 'req-9a750d51-61c4-4425-b4a4-2473396664df'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:11.145083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6e5dc-30c8-408b-ab5a-c622973e51ea admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:11.151598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a750d51-61c4-4425-b4a4-2473396664df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51f59d1267df48309fe64b4d960720bb', 'request_id': 'req-9a750d51-61c4-4425-b4a4-2473396664df'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:11.189457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a750d51-61c4-4425-b4a4-2473396664df admin admin] The token cache is being invalidate because project 51f59d1267df48309fe64b4d960720bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:11.189915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a750d51-61c4-4425-b4a4-2473396664df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:11.190900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a750d51-61c4-4425-b4a4-2473396664df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:11.195359 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 969/1888] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:10 2022] DELETE /identity/v3/projects/51f59d1267df48309fe64b4d960720bb => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:11.198573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a750d51-61c4-4425-b4a4-2473396664df admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:11.203246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9a750d51-61c4-4425-b4a4-2473396664df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:11.261093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b55ab-d031-49fd-acd0-a370fc2824f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:11.261968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b55ab-d031-49fd-acd0-a370fc2824f8 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:11.262408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b55ab-d031-49fd-acd0-a370fc2824f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:11.262859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b55ab-d031-49fd-acd0-a370fc2824f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:11.263671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b55ab-d031-49fd-acd0-a370fc2824f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:11.265604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b55ab-d031-49fd-acd0-a370fc2824f8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:11.299445 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 920/1889] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:10 2022] POST /identity/v3/projects => generated 391 bytes in 451 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:11.302736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b55ab-d031-49fd-acd0-a370fc2824f8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:11.304460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-517b55ab-d031-49fd-acd0-a370fc2824f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:11.326524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a750d51-61c4-4425-b4a4-2473396664df admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:11.384008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40822f2-1d9e-4adf-8c59-d5f413f31635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:11.385756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40822f2-1d9e-4adf-8c59-d5f413f31635 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:11.386412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40822f2-1d9e-4adf-8c59-d5f413f31635 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:11.386913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40822f2-1d9e-4adf-8c59-d5f413f31635 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:11.387771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40822f2-1d9e-4adf-8c59-d5f413f31635 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:11.389949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40822f2-1d9e-4adf-8c59-d5f413f31635 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:11.413350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b55ab-d031-49fd-acd0-a370fc2824f8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:11.420036 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 970/1890] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:11 2022] POST /identity/v3/users => generated 296 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:11.421504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710421b-fdec-4bfa-afdb-1cc91395e942 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:11.421846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710421b-fdec-4bfa-afdb-1cc91395e942 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:11.422127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710421b-fdec-4bfa-afdb-1cc91395e942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:11.423793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d45ac8-bd79-48df-b08e-bee743c4fe5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:11.424683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d45ac8-bd79-48df-b08e-bee743c4fe5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:11.425218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d45ac8-bd79-48df-b08e-bee743c4fe5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:11.425324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d45ac8-bd79-48df-b08e-bee743c4fe5d admin admin] PATH_INFO: `/v3/projects/6de3b6487f874ce2a574f1ff93b7fb06` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:11.426714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d45ac8-bd79-48df-b08e-bee743c4fe5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6de3b6487f874ce2a574f1ff93b7fb06)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:11.428774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d45ac8-bd79-48df-b08e-bee743c4fe5d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:11.446887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7710421b-fdec-4bfa-afdb-1cc91395e942 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:11.448800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7710421b-fdec-4bfa-afdb-1cc91395e942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:11.457667 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 971/1891] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:11.458942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710421b-fdec-4bfa-afdb-1cc91395e942 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:11.460342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7710421b-fdec-4bfa-afdb-1cc91395e942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:11.474907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710421b-fdec-4bfa-afdb-1cc91395e942 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:11.481017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160d45c-0268-4334-aecd-43e52e6151a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:11.481860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160d45c-0268-4334-aecd-43e52e6151a6 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:11.482073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160d45c-0268-4334-aecd-43e52e6151a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:11.482351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160d45c-0268-4334-aecd-43e52e6151a6 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:11.483369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b160d45c-0268-4334-aecd-43e52e6151a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:11.486139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b160d45c-0268-4334-aecd-43e52e6151a6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:11.498699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e7d45ac8-bd79-48df-b08e-bee743c4fe5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6de3b6487f874ce2a574f1ff93b7fb06', 'request_id': 'req-e7d45ac8-bd79-48df-b08e-bee743c4fe5d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:11.508203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e7d45ac8-bd79-48df-b08e-bee743c4fe5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6de3b6487f874ce2a574f1ff93b7fb06', 'request_id': 'req-e7d45ac8-bd79-48df-b08e-bee743c4fe5d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:11.515962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e7d45ac8-bd79-48df-b08e-bee743c4fe5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6de3b6487f874ce2a574f1ff93b7fb06', 'request_id': 'req-e7d45ac8-bd79-48df-b08e-bee743c4fe5d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:11.517180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:11.522069 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 972/1892] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:11 2022] POST /identity/v3/users => generated 296 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:11.523020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160d45c-0268-4334-aecd-43e52e6151a6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:11.532369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e7d45ac8-bd79-48df-b08e-bee743c4fe5d admin admin] The token cache is being invalidate because project 6de3b6487f874ce2a574f1ff93b7fb06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:11.532369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e7d45ac8-bd79-48df-b08e-bee743c4fe5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:11.532985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e7d45ac8-bd79-48df-b08e-bee743c4fe5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:11.534730 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 921/1893] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:11 2022] DELETE /identity/v3/projects/6de3b6487f874ce2a574f1ff93b7fb06 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:11.538538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160d45c-0268-4334-aecd-43e52e6151a6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:11.539963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e4164-baf4-4212-bc11-f16d7f74ec00 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:11.540320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e4164-baf4-4212-bc11-f16d7f74ec00 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:11.540702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e4164-baf4-4212-bc11-f16d7f74ec00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:11.564484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1e4e4164-baf4-4212-bc11-f16d7f74ec00 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:11.565861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1e4e4164-baf4-4212-bc11-f16d7f74ec00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:11.570040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=7,keystone:SELECT=86,keystone:INSERT=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:11.626279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae89b3-2da0-4649-90e3-fdaa8d191ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:11.627383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae89b3-2da0-4649-90e3-fdaa8d191ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:11.627695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae89b3-2da0-4649-90e3-fdaa8d191ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:11.628067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae89b3-2da0-4649-90e3-fdaa8d191ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:11.628827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae89b3-2da0-4649-90e3-fdaa8d191ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:11.630569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae89b3-2da0-4649-90e3-fdaa8d191ae9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:11.648051 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 973/1894] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:11.650063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae89b3-2da0-4649-90e3-fdaa8d191ae9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:11.657080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ebae89b3-2da0-4649-90e3-fdaa8d191ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:11.672316 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 922/1895] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:11.674031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f60b71-62a5-4955-a873-875b66edae61 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:11.675827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f60b71-62a5-4955-a873-875b66edae61 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:11.676260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f60b71-62a5-4955-a873-875b66edae61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:11.677040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae89b3-2da0-4649-90e3-fdaa8d191ae9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:11.684565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffa02c-a33e-4d4b-b465-93c804267dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:11.685421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffa02c-a33e-4d4b-b465-93c804267dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:11.685805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffa02c-a33e-4d4b-b465-93c804267dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:11.686988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffa02c-a33e-4d4b-b465-93c804267dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:11.687973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffa02c-a33e-4d4b-b465-93c804267dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:11.691227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffa02c-a33e-4d4b-b465-93c804267dc5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:11.709854 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 974/1896] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:11.710763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffa02c-a33e-4d4b-b465-93c804267dc5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:11.714325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-84ffa02c-a33e-4d4b-b465-93c804267dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:11.716538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-40f60b71-62a5-4955-a873-875b66edae61 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:11.719060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-40f60b71-62a5-4955-a873-875b66edae61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:11.733482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffa02c-a33e-4d4b-b465-93c804267dc5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:11.740282 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 923/1897] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:11.741292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f60b71-62a5-4955-a873-875b66edae61 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:11.751139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e4e85-b9f7-4470-958b-e78d3310b1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:11.751734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e4e85-b9f7-4470-958b-e78d3310b1e7 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:11.754617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e4e85-b9f7-4470-958b-e78d3310b1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:11.754617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e4e85-b9f7-4470-958b-e78d3310b1e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:11.754617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e4e85-b9f7-4470-958b-e78d3310b1e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:11.755755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e4e85-b9f7-4470-958b-e78d3310b1e7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:11.761814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f60b71-62a5-4955-a873-875b66edae61 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:11.777902 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 975/1898] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:11.778313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d97a5d-6fe0-4f0f-bdf5-8eedcb06e894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:11.778529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e4e85-b9f7-4470-958b-e78d3310b1e7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:11.778631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d97a5d-6fe0-4f0f-bdf5-8eedcb06e894 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:11.778631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d97a5d-6fe0-4f0f-bdf5-8eedcb06e894 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:11.778631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d97a5d-6fe0-4f0f-bdf5-8eedcb06e894 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:11.778631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d97a5d-6fe0-4f0f-bdf5-8eedcb06e894 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:49:11.778631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d97a5d-6fe0-4f0f-bdf5-8eedcb06e894 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:11.779422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d97a5d-6fe0-4f0f-bdf5-8eedcb06e894 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:11.797921 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 924/1899] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:49:11.797921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8fc04-1e45-4e70-b260-5dc53c972ee3 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:11.797921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8fc04-1e45-4e70-b260-5dc53c972ee3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:11.797921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8fc04-1e45-4e70-b260-5dc53c972ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:11.799308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e4e85-b9f7-4470-958b-e78d3310b1e7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:11.806580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886c89f-726a-41b2-b2e7-b4bdba58c0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:11.808169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886c89f-726a-41b2-b2e7-b4bdba58c0cd admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:11.808454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886c89f-726a-41b2-b2e7-b4bdba58c0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:11.808729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886c89f-726a-41b2-b2e7-b4bdba58c0cd admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:11.809550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886c89f-726a-41b2-b2e7-b4bdba58c0cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:11.811467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886c89f-726a-41b2-b2e7-b4bdba58c0cd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:11.828132 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 976/1900] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:11.828132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886c89f-726a-41b2-b2e7-b4bdba58c0cd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:11.828428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-87c8fc04-1e45-4e70-b260-5dc53c972ee3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:11.828873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-87c8fc04-1e45-4e70-b260-5dc53c972ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:11.839479 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 925/1901] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:11.840623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8fc04-1e45-4e70-b260-5dc53c972ee3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:11.842317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886c89f-726a-41b2-b2e7-b4bdba58c0cd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:11.850417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaff6a-433a-4f3e-a297-d4507ebcd3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:11.851439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaff6a-433a-4f3e-a297-d4507ebcd3d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:11.851830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaff6a-433a-4f3e-a297-d4507ebcd3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:11.852188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaff6a-433a-4f3e-a297-d4507ebcd3d4 admin admin] PATH_INFO: `/v3/projects/b2b9de2ad3ad459794059338dd371e0b/users/b61faa86ae0d4eedb371407380c22bfe/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:11.852385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8fc04-1e45-4e70-b260-5dc53c972ee3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:11.858361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40566168-8406-41d7-a02c-13416610e416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:11.860000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40566168-8406-41d7-a02c-13416610e416 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:11.860289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40566168-8406-41d7-a02c-13416610e416 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:11.860564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40566168-8406-41d7-a02c-13416610e416 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:11.861601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40566168-8406-41d7-a02c-13416610e416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:11.863538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40566168-8406-41d7-a02c-13416610e416 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:11.879855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeaff6a-433a-4f3e-a297-d4507ebcd3d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2b9de2ad3ad459794059338dd371e0b, user_id=b61faa86ae0d4eedb371407380c22bfe, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:11.885649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeaff6a-433a-4f3e-a297-d4507ebcd3d4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:11.915018 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 977/1902] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:11 2022] PUT /identity/v3/projects/b2b9de2ad3ad459794059338dd371e0b/users/b61faa86ae0d4eedb371407380c22bfe/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:11.925481 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 926/1903] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:11 2022] POST /identity/v3/projects => generated 385 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:11.925481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40566168-8406-41d7-a02c-13416610e416 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:11.925735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaff6a-433a-4f3e-a297-d4507ebcd3d4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:12.008623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40566168-8406-41d7-a02c-13416610e416 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:12.014174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c457d0b-ac23-485c-ab53-43c8e09eb441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:12.015133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c457d0b-ac23-485c-ab53-43c8e09eb441 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:12.015565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c457d0b-ac23-485c-ab53-43c8e09eb441 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:12.016469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c457d0b-ac23-485c-ab53-43c8e09eb441 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:12.017111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c457d0b-ac23-485c-ab53-43c8e09eb441 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:49:12.017414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c457d0b-ac23-485c-ab53-43c8e09eb441 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:12.019664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c457d0b-ac23-485c-ab53-43c8e09eb441 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:12.022240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaff6a-433a-4f3e-a297-d4507ebcd3d4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:12.028466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce5c71-f78e-469b-b4ec-186233b682f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:12.029880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce5c71-f78e-469b-b4ec-186233b682f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:12.030160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce5c71-f78e-469b-b4ec-186233b682f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:12.030467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce5c71-f78e-469b-b4ec-186233b682f6 admin admin] PATH_INFO: `/v3/projects/f7d124b2b3db49328d393be7be2e82d6/users/0980a1e24cdc491aa1a0202afc4fcd64/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:12.035442 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 927/1904] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:49:12.036823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a9927-ed4f-41a0-aa15-4ba8f68ba4f1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:12.037213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a9927-ed4f-41a0-aa15-4ba8f68ba4f1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:12.037625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a9927-ed4f-41a0-aa15-4ba8f68ba4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:12.053141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce5c71-f78e-469b-b4ec-186233b682f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d124b2b3db49328d393be7be2e82d6, user_id=0980a1e24cdc491aa1a0202afc4fcd64, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:12.056907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce5c71-f78e-469b-b4ec-186233b682f6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:12.078249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7a7a9927-ed4f-41a0-aa15-4ba8f68ba4f1 None None] MFA Rules not processed for user `b61faa86ae0d4eedb371407380c22bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:12.078907 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 978/1905] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:11 2022] PUT /identity/v3/projects/f7d124b2b3db49328d393be7be2e82d6/users/0980a1e24cdc491aa1a0202afc4fcd64/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:12.079863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce5c71-f78e-469b-b4ec-186233b682f6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:12.089240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7a7a9927-ed4f-41a0-aa15-4ba8f68ba4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:12.120360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:12.138843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:12.158669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce5c71-f78e-469b-b4ec-186233b682f6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:12.164383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9457bea7-2474-49e1-aca6-e626d3276176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:12.165056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457bea7-2474-49e1-aca6-e626d3276176 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:12.165338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457bea7-2474-49e1-aca6-e626d3276176 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:12.165619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457bea7-2474-49e1-aca6-e626d3276176 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:12.166385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9457bea7-2474-49e1-aca6-e626d3276176 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:12.168447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9457bea7-2474-49e1-aca6-e626d3276176 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:12.172750 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 928/1906] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:12.172750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ff772-5a93-4005-ba0f-ab233202d31a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:12.172750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ff772-5a93-4005-ba0f-ab233202d31a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:12.172750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ff772-5a93-4005-ba0f-ab233202d31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:12.196008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-412ff772-5a93-4005-ba0f-ab233202d31a None None] MFA Rules not processed for user `e17a026f305248878f18eec35e1e02dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:12.197471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-412ff772-5a93-4005-ba0f-ab233202d31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:12.212690 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 979/1907] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:12 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:12.216437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9457bea7-2474-49e1-aca6-e626d3276176 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:12.234381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9457bea7-2474-49e1-aca6-e626d3276176 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:12.239750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e594af-d8f8-4556-98de-2d302e777b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:12.240431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e594af-d8f8-4556-98de-2d302e777b28 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:12.240716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e594af-d8f8-4556-98de-2d302e777b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:12.240994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e594af-d8f8-4556-98de-2d302e777b28 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:12.241686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e594af-d8f8-4556-98de-2d302e777b28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:12.243657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e594af-d8f8-4556-98de-2d302e777b28 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:12.286821 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 980/1908] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:12 2022] POST /identity/v3/projects => generated 391 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:12.289268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351ffa5-cf32-48b6-a8bb-3e284af27ad5 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:12.289555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351ffa5-cf32-48b6-a8bb-3e284af27ad5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:12.289839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351ffa5-cf32-48b6-a8bb-3e284af27ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:12.293658 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 929/1909] 10.222.0.39 () {58 vars in 1015 bytes} [Sat Oct 1 20:49:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:12.294690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ff772-5a93-4005-ba0f-ab233202d31a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:12.307967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ff772-5a93-4005-ba0f-ab233202d31a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:12.331927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5351ffa5-cf32-48b6-a8bb-3e284af27ad5 None None] MFA Rules not processed for user `0980a1e24cdc491aa1a0202afc4fcd64`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:12.333680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5351ffa5-cf32-48b6-a8bb-3e284af27ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:12.400160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c09c03-b7fa-4583-9a7d-6e6460dd7b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:12.401957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c09c03-b7fa-4583-9a7d-6e6460dd7b69 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:12.402992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c09c03-b7fa-4583-9a7d-6e6460dd7b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:12.403385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c09c03-b7fa-4583-9a7d-6e6460dd7b69 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:12.404260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c09c03-b7fa-4583-9a7d-6e6460dd7b69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:12.408190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c09c03-b7fa-4583-9a7d-6e6460dd7b69 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:12.429224 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 981/1910] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:12.430431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351ffa5-cf32-48b6-a8bb-3e284af27ad5 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:12.445147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351ffa5-cf32-48b6-a8bb-3e284af27ad5 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:12.451427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8420da2-4855-4c8a-8168-95443ee96e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:12.452194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8420da2-4855-4c8a-8168-95443ee96e24 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:12.452481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8420da2-4855-4c8a-8168-95443ee96e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:12.452836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8420da2-4855-4c8a-8168-95443ee96e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:12.453719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8420da2-4855-4c8a-8168-95443ee96e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:12.455644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8420da2-4855-4c8a-8168-95443ee96e24 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:12.461762 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 930/1911] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:12 2022] POST /identity/v3/projects => generated 393 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:12.465833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c09c03-b7fa-4583-9a7d-6e6460dd7b69 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:12.470931 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 982/1912] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:12.471821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8420da2-4855-4c8a-8168-95443ee96e24 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:12.473646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c8420da2-4855-4c8a-8168-95443ee96e24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:12.552312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8420da2-4855-4c8a-8168-95443ee96e24 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:12.560041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c09c03-b7fa-4583-9a7d-6e6460dd7b69 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:12.565738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49591aa-3fa5-4655-b13b-d3a2df5a4c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:12.565915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49591aa-3fa5-4655-b13b-d3a2df5a4c6b admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:12.566113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49591aa-3fa5-4655-b13b-d3a2df5a4c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:12.566382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49591aa-3fa5-4655-b13b-d3a2df5a4c6b admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:12.567318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49591aa-3fa5-4655-b13b-d3a2df5a4c6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:12.569313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49591aa-3fa5-4655-b13b-d3a2df5a4c6b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:12.586072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:12.602310 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 931/1913] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:12 2022] POST /identity/v3/users => generated 296 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:12.603447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49591aa-3fa5-4655-b13b-d3a2df5a4c6b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:12.613354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:12.615124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49591aa-3fa5-4655-b13b-d3a2df5a4c6b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:12.619840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c38b2-a05f-48a5-93c8-f2d39ba73f81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:12.620592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c38b2-a05f-48a5-93c8-f2d39ba73f81 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:12.621166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51ef36-4a56-44d8-8bfe-f7a3c1dc979b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:12.621319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c38b2-a05f-48a5-93c8-f2d39ba73f81 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:12.621770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c38b2-a05f-48a5-93c8-f2d39ba73f81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:12.621883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51ef36-4a56-44d8-8bfe-f7a3c1dc979b admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:12.622289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51ef36-4a56-44d8-8bfe-f7a3c1dc979b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:12.622584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51ef36-4a56-44d8-8bfe-f7a3c1dc979b admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:12.623716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51ef36-4a56-44d8-8bfe-f7a3c1dc979b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:12.625475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51ef36-4a56-44d8-8bfe-f7a3c1dc979b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:12.630784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c38b2-a05f-48a5-93c8-f2d39ba73f81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:12.632279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c38b2-a05f-48a5-93c8-f2d39ba73f81 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:12.642833 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 983/1914] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:12.647632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c38b2-a05f-48a5-93c8-f2d39ba73f81 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:12.670455 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 932/1915] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:12 2022] POST /identity/v3/projects => generated 393 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:12.670455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51ef36-4a56-44d8-8bfe-f7a3c1dc979b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:12.685066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c38b2-a05f-48a5-93c8-f2d39ba73f81 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:12.789676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0958a22-e444-47e0-b406-9f94f8424f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:12.789676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0958a22-e444-47e0-b406-9f94f8424f8d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:12.789676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0958a22-e444-47e0-b406-9f94f8424f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:12.790840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0958a22-e444-47e0-b406-9f94f8424f8d admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:12.792774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0958a22-e444-47e0-b406-9f94f8424f8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:12.797085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0958a22-e444-47e0-b406-9f94f8424f8d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:12.843171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51ef36-4a56-44d8-8bfe-f7a3c1dc979b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:12.886942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066c92d-6b80-432c-9895-ecae6dad6294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:12.887644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066c92d-6b80-432c-9895-ecae6dad6294 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:12.887852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066c92d-6b80-432c-9895-ecae6dad6294 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:12.888131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066c92d-6b80-432c-9895-ecae6dad6294 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:12.888799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d066c92d-6b80-432c-9895-ecae6dad6294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:12.890626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d066c92d-6b80-432c-9895-ecae6dad6294 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:12.914424 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 984/1916] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:12 2022] POST /identity/v3/users => generated 303 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:12.915417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0958a22-e444-47e0-b406-9f94f8424f8d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:12.930171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0958a22-e444-47e0-b406-9f94f8424f8d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:12.933107 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 933/1917] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:12.933970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066c92d-6b80-432c-9895-ecae6dad6294 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:12.945382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17c4ba-8d5e-4759-9850-e243c92415be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:12.946016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17c4ba-8d5e-4759-9850-e243c92415be admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:12.946297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17c4ba-8d5e-4759-9850-e243c92415be admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:12.946639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17c4ba-8d5e-4759-9850-e243c92415be admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:12.948249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17c4ba-8d5e-4759-9850-e243c92415be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:12.949913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17c4ba-8d5e-4759-9850-e243c92415be admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:12.964580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066c92d-6b80-432c-9895-ecae6dad6294 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:12.982561 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 985/1918] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:12.982561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17c4ba-8d5e-4759-9850-e243c92415be admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:12.982937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526872a8-0e03-4648-80fd-19a0a4287442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:12.982937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526872a8-0e03-4648-80fd-19a0a4287442 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:12.982937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526872a8-0e03-4648-80fd-19a0a4287442 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:12.982937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526872a8-0e03-4648-80fd-19a0a4287442 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:12.983585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526872a8-0e03-4648-80fd-19a0a4287442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:12.985251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526872a8-0e03-4648-80fd-19a0a4287442 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:13.009302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17c4ba-8d5e-4759-9850-e243c92415be admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:13.016653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cb9ef-0671-4a1d-baea-38553e9f0a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:13.017359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cb9ef-0671-4a1d-baea-38553e9f0a83 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:13.017689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cb9ef-0671-4a1d-baea-38553e9f0a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:13.017972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cb9ef-0671-4a1d-baea-38553e9f0a83 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:13.018887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06cb9ef-0671-4a1d-baea-38553e9f0a83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:13.021067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06cb9ef-0671-4a1d-baea-38553e9f0a83 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:13.032545 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 934/1919] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:12 2022] POST /identity/v3/users => generated 303 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:13.032766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526872a8-0e03-4648-80fd-19a0a4287442 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:13.042738 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 986/1920] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:13.042928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cb9ef-0671-4a1d-baea-38553e9f0a83 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:13.055001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cb9ef-0671-4a1d-baea-38553e9f0a83 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:13.058439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526872a8-0e03-4648-80fd-19a0a4287442 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:13.060742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892666b-f576-4a15-baa0-f079e7f757e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:13.061501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892666b-f576-4a15-baa0-f079e7f757e4 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:13.061788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892666b-f576-4a15-baa0-f079e7f757e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:13.062062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892666b-f576-4a15-baa0-f079e7f757e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:13.062918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892666b-f576-4a15-baa0-f079e7f757e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:13.064489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892666b-f576-4a15-baa0-f079e7f757e4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:13.066980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3da718-be50-4347-996f-a8f8c714b857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:13.067940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3da718-be50-4347-996f-a8f8c714b857 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:13.068445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3da718-be50-4347-996f-a8f8c714b857 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:13.069000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3da718-be50-4347-996f-a8f8c714b857 admin admin] PATH_INFO: `/v3/projects/d8ce7a62c5cc4bc1bc4ab07e6138d84f/users/67c49f6c3eb04a709545b361fa4b4708/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:13.085779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3da718-be50-4347-996f-a8f8c714b857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8ce7a62c5cc4bc1bc4ab07e6138d84f, user_id=67c49f6c3eb04a709545b361fa4b4708, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:13.089610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3da718-be50-4347-996f-a8f8c714b857 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:13.092888 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 987/1921] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:13.095873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892666b-f576-4a15-baa0-f079e7f757e4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:13.113002 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 935/1922] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:13 2022] PUT /identity/v3/projects/d8ce7a62c5cc4bc1bc4ab07e6138d84f/users/67c49f6c3eb04a709545b361fa4b4708/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:13.113723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3da718-be50-4347-996f-a8f8c714b857 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:13.114217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892666b-f576-4a15-baa0-f079e7f757e4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:13.244257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f509e-5f50-4e77-95fb-3d1c4bed1c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:13.245525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f509e-5f50-4e77-95fb-3d1c4bed1c59 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:13.245898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f509e-5f50-4e77-95fb-3d1c4bed1c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:13.246267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f509e-5f50-4e77-95fb-3d1c4bed1c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:13.249410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3da718-be50-4347-996f-a8f8c714b857 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:13.251168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f509e-5f50-4e77-95fb-3d1c4bed1c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:13.253323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f509e-5f50-4e77-95fb-3d1c4bed1c59 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:13.259681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a1d33-0cb2-4970-ba47-252db460b35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:13.260936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a1d33-0cb2-4970-ba47-252db460b35f admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:13.261217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a1d33-0cb2-4970-ba47-252db460b35f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:13.261493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a1d33-0cb2-4970-ba47-252db460b35f admin admin] PATH_INFO: `/v3/projects/a11d92672f1d40a0ab70d046f73b5fb8/users/a433d354781c4c199b855cefe099eba8/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:13.270256 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 988/1923] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:13.271919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f509e-5f50-4e77-95fb-3d1c4bed1c59 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:13.283898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f509e-5f50-4e77-95fb-3d1c4bed1c59 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:13.284572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a1d33-0cb2-4970-ba47-252db460b35f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a11d92672f1d40a0ab70d046f73b5fb8, user_id=a433d354781c4c199b855cefe099eba8, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:13.287794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a1d33-0cb2-4970-ba47-252db460b35f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:13.289685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba826a73-f6d1-42c6-b286-bcb2951c6505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:13.290670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba826a73-f6d1-42c6-b286-bcb2951c6505 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:13.291048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba826a73-f6d1-42c6-b286-bcb2951c6505 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:13.291953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba826a73-f6d1-42c6-b286-bcb2951c6505 admin admin] PATH_INFO: `/v3/projects/1048abb881ff4ede869168468c185f28/users/98ff2a059d104df5aa783553f72fbdcf/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:13.310914 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 936/1924] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:13 2022] PUT /identity/v3/projects/a11d92672f1d40a0ab70d046f73b5fb8/users/a433d354781c4c199b855cefe099eba8/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:13.316007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13642932-f9e5-475e-882b-5137f170645f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:13.316329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13642932-f9e5-475e-882b-5137f170645f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:13.316505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13642932-f9e5-475e-882b-5137f170645f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:13.317200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba826a73-f6d1-42c6-b286-bcb2951c6505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1048abb881ff4ede869168468c185f28, user_id=98ff2a059d104df5aa783553f72fbdcf, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:13.320412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba826a73-f6d1-42c6-b286-bcb2951c6505 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:13.338083 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 989/1925] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:13 2022] PUT /identity/v3/projects/1048abb881ff4ede869168468c185f28/users/98ff2a059d104df5aa783553f72fbdcf/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:13.339698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba826a73-f6d1-42c6-b286-bcb2951c6505 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:13.353408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:13.383779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-13642932-f9e5-475e-882b-5137f170645f None None] MFA Rules not processed for user `67c49f6c3eb04a709545b361fa4b4708`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:13.384873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-13642932-f9e5-475e-882b-5137f170645f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:13.386872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:13.449358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba826a73-f6d1-42c6-b286-bcb2951c6505 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:13.454946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d211e0-fed1-4cd5-8bbd-d69aabb30538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:13.455703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d211e0-fed1-4cd5-8bbd-d69aabb30538 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:13.456059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d211e0-fed1-4cd5-8bbd-d69aabb30538 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:13.456412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d211e0-fed1-4cd5-8bbd-d69aabb30538 admin admin] PATH_INFO: `/v3/projects/219a096e44d4400c9513ada7e5acf85b/users/c964ff61874f4e8f8713261701c34991/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:13.474461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d211e0-fed1-4cd5-8bbd-d69aabb30538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=219a096e44d4400c9513ada7e5acf85b, user_id=c964ff61874f4e8f8713261701c34991, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:13.477809 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 937/1926] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:13.478732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d211e0-fed1-4cd5-8bbd-d69aabb30538 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:13.479073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13642932-f9e5-475e-882b-5137f170645f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:13.493932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13642932-f9e5-475e-882b-5137f170645f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:13.503138 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 990/1927] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:13 2022] PUT /identity/v3/projects/219a096e44d4400c9513ada7e5acf85b/users/c964ff61874f4e8f8713261701c34991/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:13.504354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba561a-4892-4649-817b-5d042ae2548e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:13.504833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba561a-4892-4649-817b-5d042ae2548e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:13.505335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba561a-4892-4649-817b-5d042ae2548e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:13.541018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5bba561a-4892-4649-817b-5d042ae2548e None None] MFA Rules not processed for user `98ff2a059d104df5aa783553f72fbdcf`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:13.542654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5bba561a-4892-4649-817b-5d042ae2548e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:13.567411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a44cb-6584-4453-940d-75aacd60e9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:13.568108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a44cb-6584-4453-940d-75aacd60e9a3 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:13.569059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a44cb-6584-4453-940d-75aacd60e9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:13.569589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a44cb-6584-4453-940d-75aacd60e9a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:13.575952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a44cb-6584-4453-940d-75aacd60e9a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:13.578362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a44cb-6584-4453-940d-75aacd60e9a3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:13.593980 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 938/1928] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:13.595068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a44cb-6584-4453-940d-75aacd60e9a3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:13.608143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a44cb-6584-4453-940d-75aacd60e9a3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:13.613021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d778f-ae69-43c3-b09d-31ca60092f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:13.613817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d778f-ae69-43c3-b09d-31ca60092f69 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:13.614285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d778f-ae69-43c3-b09d-31ca60092f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:13.614764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d778f-ae69-43c3-b09d-31ca60092f69 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:13.615490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d778f-ae69-43c3-b09d-31ca60092f69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:13.617760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d778f-ae69-43c3-b09d-31ca60092f69 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:13.622945 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 991/1929] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:13.623802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba561a-4892-4649-817b-5d042ae2548e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:13.635834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba561a-4892-4649-817b-5d042ae2548e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:13.641516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ed056-92ce-4df3-869b-702b0ec411f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:13.642337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ed056-92ce-4df3-869b-702b0ec411f2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:13.642953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ed056-92ce-4df3-869b-702b0ec411f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:13.643552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ed056-92ce-4df3-869b-702b0ec411f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:13.644202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ed056-92ce-4df3-869b-702b0ec411f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:13.645819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ed056-92ce-4df3-869b-702b0ec411f2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:13.652802 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 939/1930] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:13 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:13.652802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d778f-ae69-43c3-b09d-31ca60092f69 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:13.658233 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 992/1931] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:13.659281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ed056-92ce-4df3-869b-702b0ec411f2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:13.670131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ed056-92ce-4df3-869b-702b0ec411f2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:13.729117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d778f-ae69-43c3-b09d-31ca60092f69 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:13.734711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa18791-95f2-494b-8acf-24e34bb3b698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:13.735369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa18791-95f2-494b-8acf-24e34bb3b698 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:13.735646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa18791-95f2-494b-8acf-24e34bb3b698 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:13.735923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa18791-95f2-494b-8acf-24e34bb3b698 admin admin] PATH_INFO: `/v3/projects/a11d92672f1d40a0ab70d046f73b5fb8/users/a433d354781c4c199b855cefe099eba8/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:13.737890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa18791-95f2-494b-8acf-24e34bb3b698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a11d92672f1d40a0ab70d046f73b5fb8, user_id=a433d354781c4c199b855cefe099eba8, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:13.738528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df478561-a07c-4918-9546-853ba90b302b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:13.739294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df478561-a07c-4918-9546-853ba90b302b admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:13.739401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df478561-a07c-4918-9546-853ba90b302b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:13.739606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df478561-a07c-4918-9546-853ba90b302b admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:13.740233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df478561-a07c-4918-9546-853ba90b302b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:13.740639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa18791-95f2-494b-8acf-24e34bb3b698 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:13.742129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df478561-a07c-4918-9546-853ba90b302b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:13.752454 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 940/1932] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:13 2022] PUT /identity/v3/projects/a11d92672f1d40a0ab70d046f73b5fb8/users/a433d354781c4c199b855cefe099eba8/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:13.753597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61a8a0-3e1f-4f01-89ba-ec353572faf3 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:13.753888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61a8a0-3e1f-4f01-89ba-ec353572faf3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:13.754158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61a8a0-3e1f-4f01-89ba-ec353572faf3 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:13.755644 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 941/1933] 10.222.0.17 () {54 vars in 921 bytes} [Sat Oct 1 20:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:49:13.756328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61a8a0-3e1f-4f01-89ba-ec353572faf3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:13.772535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:13.783298 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 993/1934] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:13 2022] POST /identity/v3/projects => generated 391 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:13.784349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df478561-a07c-4918-9546-853ba90b302b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:13.796216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:13.872939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61a8a0-3e1f-4f01-89ba-ec353572faf3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:13.878243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625519cc-4fef-4279-a6bd-ed08eac490a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:13.878443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df478561-a07c-4918-9546-853ba90b302b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:13.879164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625519cc-4fef-4279-a6bd-ed08eac490a4 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:13.879603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625519cc-4fef-4279-a6bd-ed08eac490a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:13.880568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625519cc-4fef-4279-a6bd-ed08eac490a4 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:13.881318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625519cc-4fef-4279-a6bd-ed08eac490a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:13.883731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625519cc-4fef-4279-a6bd-ed08eac490a4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:13.884558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c75db-51f2-4488-9d4d-ef291f2f4fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:13.885413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c75db-51f2-4488-9d4d-ef291f2f4fa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:13.885723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c75db-51f2-4488-9d4d-ef291f2f4fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:13.885969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c75db-51f2-4488-9d4d-ef291f2f4fa8 admin admin] PATH_INFO: `/v3/projects/219a096e44d4400c9513ada7e5acf85b/users/c964ff61874f4e8f8713261701c34991/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:13.888008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c75db-51f2-4488-9d4d-ef291f2f4fa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=219a096e44d4400c9513ada7e5acf85b, user_id=c964ff61874f4e8f8713261701c34991, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:13.891154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c75db-51f2-4488-9d4d-ef291f2f4fa8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:13.906300 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 994/1935] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:13 2022] PUT /identity/v3/projects/219a096e44d4400c9513ada7e5acf85b/users/c964ff61874f4e8f8713261701c34991/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:13.907875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8080e-31c7-41d2-b606-f4f80f50c111 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:13.908458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8080e-31c7-41d2-b606-f4f80f50c111 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:13.908576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8080e-31c7-41d2-b606-f4f80f50c111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:13.919165 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 942/1936] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:13 2022] POST /identity/v3/users => generated 299 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:13.920060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625519cc-4fef-4279-a6bd-ed08eac490a4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:13.931768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625519cc-4fef-4279-a6bd-ed08eac490a4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:13.954671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b6d8080e-31c7-41d2-b606-f4f80f50c111 None None] MFA Rules not processed for user `a433d354781c4c199b855cefe099eba8`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:13.955914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b6d8080e-31c7-41d2-b606-f4f80f50c111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:14.023537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a26de6-7238-4024-9f56-03188c3dccc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:14.024198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a26de6-7238-4024-9f56-03188c3dccc5 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:14.025170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a26de6-7238-4024-9f56-03188c3dccc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:14.031341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a26de6-7238-4024-9f56-03188c3dccc5 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:14.031341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a26de6-7238-4024-9f56-03188c3dccc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:14.031341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a26de6-7238-4024-9f56-03188c3dccc5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:14.073304 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 943/1937] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:13 2022] POST /identity/v3/users => generated 302 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:14.073304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcfa21-aaa5-47c1-860a-2289d93716f4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:14.073304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcfa21-aaa5-47c1-860a-2289d93716f4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:14.074822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcfa21-aaa5-47c1-860a-2289d93716f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:14.079713 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 995/1938] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:13 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:14.084487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808cb77-3a7d-4a67-b73e-57ed968a30ab None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:14.084786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808cb77-3a7d-4a67-b73e-57ed968a30ab None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:14.085066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808cb77-3a7d-4a67-b73e-57ed968a30ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:14.101697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-36fcfa21-aaa5-47c1-860a-2289d93716f4 None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:14.103313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-36fcfa21-aaa5-47c1-860a-2289d93716f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:14.119598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a808cb77-3a7d-4a67-b73e-57ed968a30ab None None] MFA Rules not processed for user `c964ff61874f4e8f8713261701c34991`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:14.120875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a808cb77-3a7d-4a67-b73e-57ed968a30ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:14.188383 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 944/1939] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 20:49:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:14.189467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcfa21-aaa5-47c1-860a-2289d93716f4 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:14.201632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcfa21-aaa5-47c1-860a-2289d93716f4 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:14.211395 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 996/1940] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:14 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:14.212699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808cb77-3a7d-4a67-b73e-57ed968a30ab None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:14.216604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7186f-0c05-400a-9021-8166f8b3950f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:14.217250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7186f-0c05-400a-9021-8166f8b3950f admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:14.217712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7186f-0c05-400a-9021-8166f8b3950f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:14.217976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7186f-0c05-400a-9021-8166f8b3950f admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:14.218675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7186f-0c05-400a-9021-8166f8b3950f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:14.220447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7186f-0c05-400a-9021-8166f8b3950f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:14.225479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808cb77-3a7d-4a67-b73e-57ed968a30ab None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:14.233686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e767121-8ff7-4ae3-a374-df77d3792a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:14.234312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e767121-8ff7-4ae3-a374-df77d3792a25 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:14.234798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e767121-8ff7-4ae3-a374-df77d3792a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:14.235078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e767121-8ff7-4ae3-a374-df77d3792a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:14.235731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e767121-8ff7-4ae3-a374-df77d3792a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:14.237594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e767121-8ff7-4ae3-a374-df77d3792a25 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:14.240281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:14.250194 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 945/1941] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:14.251520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef174e-52e7-4726-beb9-d25d3f3a20c4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:14.251805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef174e-52e7-4726-beb9-d25d3f3a20c4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:14.252080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef174e-52e7-4726-beb9-d25d3f3a20c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:14.258122 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 997/1942] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:14.259880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ba341-69d4-4964-985f-1b8a5b459183 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:14.260442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ba341-69d4-4964-985f-1b8a5b459183 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:14.260721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ba341-69d4-4964-985f-1b8a5b459183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:14.278001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:14.293208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-294ba341-69d4-4964-985f-1b8a5b459183 None None] MFA Rules not processed for user `0980a1e24cdc491aa1a0202afc4fcd64`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:14.294705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0aef174e-52e7-4726-beb9-d25d3f3a20c4 None None] MFA Rules not processed for user `b61faa86ae0d4eedb371407380c22bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:14.294851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-294ba341-69d4-4964-985f-1b8a5b459183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:14.295914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0aef174e-52e7-4726-beb9-d25d3f3a20c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:14.381637 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 946/1943] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:14.381637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef174e-52e7-4726-beb9-d25d3f3a20c4 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:14.388108 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 998/1944] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:14.388108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ba341-69d4-4964-985f-1b8a5b459183 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:14.391640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef174e-52e7-4726-beb9-d25d3f3a20c4 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:14.397888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f1545-b0c1-49a6-aeaa-36c3fa8d0acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:14.398613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f1545-b0c1-49a6-aeaa-36c3fa8d0acc admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:14.398897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f1545-b0c1-49a6-aeaa-36c3fa8d0acc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:14.399175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f1545-b0c1-49a6-aeaa-36c3fa8d0acc admin admin] PATH_INFO: `/v3/projects/f390bd8ba99a4e928d54c15eb3ab2bb5/users/920704d8a66c47a7b8f6c35bab9dc398/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:14.400365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ba341-69d4-4964-985f-1b8a5b459183 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:14.410895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761ef1d-9634-42d6-9639-7dd77744534b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:14.411893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761ef1d-9634-42d6-9639-7dd77744534b admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:14.412759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761ef1d-9634-42d6-9639-7dd77744534b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:14.413144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761ef1d-9634-42d6-9639-7dd77744534b admin admin] PATH_INFO: `/v3/projects/88450b008137436c8bd03cc66fd43637/users/1763387ee81341c6bb757ba986f0e39f/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:14.425916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f1545-b0c1-49a6-aeaa-36c3fa8d0acc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f390bd8ba99a4e928d54c15eb3ab2bb5, user_id=920704d8a66c47a7b8f6c35bab9dc398, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:14.433000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f1545-b0c1-49a6-aeaa-36c3fa8d0acc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:14.445046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761ef1d-9634-42d6-9639-7dd77744534b admin admin] RBAC: Authorizing `identity:create_grant(project_id=88450b008137436c8bd03cc66fd43637, user_id=1763387ee81341c6bb757ba986f0e39f, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:14.450773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761ef1d-9634-42d6-9639-7dd77744534b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:14.454721 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 947/1945] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:14 2022] PUT /identity/v3/projects/f390bd8ba99a4e928d54c15eb3ab2bb5/users/920704d8a66c47a7b8f6c35bab9dc398/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:14.454721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f1545-b0c1-49a6-aeaa-36c3fa8d0acc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:14.457666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-df9f1545-b0c1-49a6-aeaa-36c3fa8d0acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:14.480575 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 999/1946] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:14 2022] PUT /identity/v3/projects/88450b008137436c8bd03cc66fd43637/users/1763387ee81341c6bb757ba986f0e39f/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:14.481714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761ef1d-9634-42d6-9639-7dd77744534b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:14.558548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f1545-b0c1-49a6-aeaa-36c3fa8d0acc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:14.579530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761ef1d-9634-42d6-9639-7dd77744534b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:14.584150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b130856-d2c3-469c-8c26-f5660ab2848a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:14.585024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b130856-d2c3-469c-8c26-f5660ab2848a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:14.585459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b130856-d2c3-469c-8c26-f5660ab2848a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:14.586153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b130856-d2c3-469c-8c26-f5660ab2848a admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:14.586616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b130856-d2c3-469c-8c26-f5660ab2848a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:14.588383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b130856-d2c3-469c-8c26-f5660ab2848a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:14.607668 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1000/1947] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:14.608979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fdc54-82f9-49c8-960e-486903ccafc2 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:14.609267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fdc54-82f9-49c8-960e-486903ccafc2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:14.609550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fdc54-82f9-49c8-960e-486903ccafc2 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:14.610930 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1001/1948] 10.222.0.17 () {54 vars in 921 bytes} [Sat Oct 1 20:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:49:14.611497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fdc54-82f9-49c8-960e-486903ccafc2 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:14.626049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fdc54-82f9-49c8-960e-486903ccafc2 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:14.631070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1ce84-20ae-4d92-a254-d5c7cce17797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:14.631719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1ce84-20ae-4d92-a254-d5c7cce17797 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:14.632002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1ce84-20ae-4d92-a254-d5c7cce17797 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:14.632270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1ce84-20ae-4d92-a254-d5c7cce17797 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:14.632902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1ce84-20ae-4d92-a254-d5c7cce17797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:14.634480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1ce84-20ae-4d92-a254-d5c7cce17797 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:14.637624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf2aa0-1d8f-4285-a033-de303ad80cac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:14.638235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf2aa0-1d8f-4285-a033-de303ad80cac service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:14.638550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf2aa0-1d8f-4285-a033-de303ad80cac service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:14.638819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf2aa0-1d8f-4285-a033-de303ad80cac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:14.648471 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1002/1949] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:14.648471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1ce84-20ae-4d92-a254-d5c7cce17797 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:14.648471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8da1ce84-20ae-4d92-a254-d5c7cce17797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:14.652083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf2aa0-1d8f-4285-a033-de303ad80cac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:14.653674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf2aa0-1d8f-4285-a033-de303ad80cac service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:14.663878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1ce84-20ae-4d92-a254-d5c7cce17797 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:14.665855 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 948/1950] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:14.666936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf2aa0-1d8f-4285-a033-de303ad80cac service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:14.685014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf2aa0-1d8f-4285-a033-de303ad80cac service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:14.687219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa72db7-a852-4cde-952f-58ab791b9b87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:14.687858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa72db7-a852-4cde-952f-58ab791b9b87 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:14.688143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa72db7-a852-4cde-952f-58ab791b9b87 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:14.688414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa72db7-a852-4cde-952f-58ab791b9b87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:14.694991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d96ca-34bd-4116-8f2d-debe9220663d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:14.696055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d96ca-34bd-4116-8f2d-debe9220663d admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:14.696411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d96ca-34bd-4116-8f2d-debe9220663d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:14.696687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d96ca-34bd-4116-8f2d-debe9220663d admin admin] PATH_INFO: `/v3/projects/f390bd8ba99a4e928d54c15eb3ab2bb5/users/920704d8a66c47a7b8f6c35bab9dc398/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:14.700223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d96ca-34bd-4116-8f2d-debe9220663d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f390bd8ba99a4e928d54c15eb3ab2bb5, user_id=920704d8a66c47a7b8f6c35bab9dc398, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:14.703909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d96ca-34bd-4116-8f2d-debe9220663d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:14.710820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:14.720154 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 949/1951] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:14 2022] PUT /identity/v3/projects/f390bd8ba99a4e928d54c15eb3ab2bb5/users/920704d8a66c47a7b8f6c35bab9dc398/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:14.720154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c297849-9d18-496c-be09-19271a4ca665 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:14.728502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c297849-9d18-496c-be09-19271a4ca665 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:14.728502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c297849-9d18-496c-be09-19271a4ca665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:14.747644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3c297849-9d18-496c-be09-19271a4ca665 None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:14.748883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3c297849-9d18-496c-be09-19271a4ca665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:14.767648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa72db7-a852-4cde-952f-58ab791b9b87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:14.770013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa72db7-a852-4cde-952f-58ab791b9b87 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:14.784171 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1003/1952] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:14.785262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa72db7-a852-4cde-952f-58ab791b9b87 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:14.790614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:14.800806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa72db7-a852-4cde-952f-58ab791b9b87 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:14.846297 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 950/1953] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 20:49:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:14.847674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90747658-ceb8-4792-acc6-d23a1dbd9267 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:14.847963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90747658-ceb8-4792-acc6-d23a1dbd9267 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:14.848241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90747658-ceb8-4792-acc6-d23a1dbd9267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:14.888219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-90747658-ceb8-4792-acc6-d23a1dbd9267 None None] MFA Rules not processed for user `920704d8a66c47a7b8f6c35bab9dc398`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:14.889587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-90747658-ceb8-4792-acc6-d23a1dbd9267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:14.890166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f066a8-bf31-45de-a46f-e7e1da643fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:14.890958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f066a8-bf31-45de-a46f-e7e1da643fdc admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:14.891391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f066a8-bf31-45de-a46f-e7e1da643fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:14.891735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f066a8-bf31-45de-a46f-e7e1da643fdc admin admin] PATH_INFO: `/v3/projects/88450b008137436c8bd03cc66fd43637/users/1763387ee81341c6bb757ba986f0e39f/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:14.894626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f066a8-bf31-45de-a46f-e7e1da643fdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=88450b008137436c8bd03cc66fd43637, user_id=1763387ee81341c6bb757ba986f0e39f, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:14.897859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f066a8-bf31-45de-a46f-e7e1da643fdc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:14.918912 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1004/1954] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:14 2022] PUT /identity/v3/projects/88450b008137436c8bd03cc66fd43637/users/1763387ee81341c6bb757ba986f0e39f/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:14.918912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f066a8-bf31-45de-a46f-e7e1da643fdc admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:15.000486 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 951/1955] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:15.000486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479ae94-cfd0-48c8-b4b2-ce4622e0ef99 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:15.000486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479ae94-cfd0-48c8-b4b2-ce4622e0ef99 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:15.000486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479ae94-cfd0-48c8-b4b2-ce4622e0ef99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:15.029340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f066a8-bf31-45de-a46f-e7e1da643fdc admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:15.036393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7edb1d-3811-49e2-8b28-f79959288b7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:15.037118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7edb1d-3811-49e2-8b28-f79959288b7f service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:15.037472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7edb1d-3811-49e2-8b28-f79959288b7f service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:15.037926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7edb1d-3811-49e2-8b28-f79959288b7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:15.042632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e479ae94-cfd0-48c8-b4b2-ce4622e0ef99 None None] MFA Rules not processed for user `1763387ee81341c6bb757ba986f0e39f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:15.043792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e479ae94-cfd0-48c8-b4b2-ce4622e0ef99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:15.112690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7edb1d-3811-49e2-8b28-f79959288b7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:15.114288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7edb1d-3811-49e2-8b28-f79959288b7f service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:15.124482 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1005/1956] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:15.125151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7edb1d-3811-49e2-8b28-f79959288b7f service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:15.135542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:15.137038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7edb1d-3811-49e2-8b28-f79959288b7f service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:15.142087 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 952/1957] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:14 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:15.148371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9982c-6683-4765-8909-b6de6ad59383 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:15.149031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9982c-6683-4765-8909-b6de6ad59383 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:15.149433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9982c-6683-4765-8909-b6de6ad59383 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:15.150400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9982c-6683-4765-8909-b6de6ad59383 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:15.160972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9982c-6683-4765-8909-b6de6ad59383 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:15.163048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9982c-6683-4765-8909-b6de6ad59383 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:15.176880 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1006/1958] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:15.227593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479ae94-cfd0-48c8-b4b2-ce4622e0ef99 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:15.244472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479ae94-cfd0-48c8-b4b2-ce4622e0ef99 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:15.250750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29102d02-2993-470a-a719-246a7a0bcbcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:15.251530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29102d02-2993-470a-a719-246a7a0bcbcc service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:15.251814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29102d02-2993-470a-a719-246a7a0bcbcc service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:15.252090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29102d02-2993-470a-a719-246a7a0bcbcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:15.348389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29102d02-2993-470a-a719-246a7a0bcbcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:15.350008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29102d02-2993-470a-a719-246a7a0bcbcc service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:15.364416 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 953/1959] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:15 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:15.904635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9982c-6683-4765-8909-b6de6ad59383 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:15.932427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:16.019291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9982c-6683-4765-8909-b6de6ad59383 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:16.033812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393c28d-85c1-4b3b-9345-941c482593fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:16.036776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393c28d-85c1-4b3b-9345-941c482593fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:16.038764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393c28d-85c1-4b3b-9345-941c482593fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:16.045613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393c28d-85c1-4b3b-9345-941c482593fd admin admin] PATH_INFO: `/v3/users/98ff2a059d104df5aa783553f72fbdcf` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:16.047682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393c28d-85c1-4b3b-9345-941c482593fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ff2a059d104df5aa783553f72fbdcf)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:16.049416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393c28d-85c1-4b3b-9345-941c482593fd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:16.120113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5393c28d-85c1-4b3b-9345-941c482593fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ff2a059d104df5aa783553f72fbdcf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:16.128868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5393c28d-85c1-4b3b-9345-941c482593fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ff2a059d104df5aa783553f72fbdcf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:16.152915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5393c28d-85c1-4b3b-9345-941c482593fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ff2a059d104df5aa783553f72fbdcf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:16.154479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5393c28d-85c1-4b3b-9345-941c482593fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ff2a059d104df5aa783553f72fbdcf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:16.164807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5393c28d-85c1-4b3b-9345-941c482593fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ff2a059d104df5aa783553f72fbdcf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:16.174612 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1007/1960] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:15 2022] DELETE /identity/v3/users/98ff2a059d104df5aa783553f72fbdcf => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:16.179200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a5455-736c-49c2-b813-1eee4ae01914 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:16.179717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a5455-736c-49c2-b813-1eee4ae01914 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:16.179824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a5455-736c-49c2-b813-1eee4ae01914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:16.209371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-860a5455-736c-49c2-b813-1eee4ae01914 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:16.210858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-860a5455-736c-49c2-b813-1eee4ae01914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:16.392292 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 954/1961] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:16.399961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393c28d-85c1-4b3b-9345-941c482593fd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:16.402325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5393c28d-85c1-4b3b-9345-941c482593fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:16.485800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a5455-736c-49c2-b813-1eee4ae01914 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:16.488296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-860a5455-736c-49c2-b813-1eee4ae01914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:16.509376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a5455-736c-49c2-b813-1eee4ae01914 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:16.520716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6392c-fb8f-4126-aa9a-8437837c7875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:16.523156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6392c-fb8f-4126-aa9a-8437837c7875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:16.524946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6392c-fb8f-4126-aa9a-8437837c7875 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:16.525331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6392c-fb8f-4126-aa9a-8437837c7875 admin admin] PATH_INFO: `/v3/users/67c49f6c3eb04a709545b361fa4b4708` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:16.528613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6392c-fb8f-4126-aa9a-8437837c7875 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67c49f6c3eb04a709545b361fa4b4708)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:16.538686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6392c-fb8f-4126-aa9a-8437837c7875 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:16.556010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393c28d-85c1-4b3b-9345-941c482593fd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:16.573690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e67147-dedc-4bbc-abbe-ae2c2807df0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:16.574356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e67147-dedc-4bbc-abbe-ae2c2807df0a service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:16.574734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e67147-dedc-4bbc-abbe-ae2c2807df0a service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:16.575023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e67147-dedc-4bbc-abbe-ae2c2807df0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:16.602994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e67147-dedc-4bbc-abbe-ae2c2807df0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:16.605602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e67147-dedc-4bbc-abbe-ae2c2807df0a service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:16.618760 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1008/1962] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:16.646805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1fe6392c-fb8f-4126-aa9a-8437837c7875 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67c49f6c3eb04a709545b361fa4b4708'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:16.647768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1fe6392c-fb8f-4126-aa9a-8437837c7875 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67c49f6c3eb04a709545b361fa4b4708'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:16.648542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1fe6392c-fb8f-4126-aa9a-8437837c7875 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67c49f6c3eb04a709545b361fa4b4708'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:16.662730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1fe6392c-fb8f-4126-aa9a-8437837c7875 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67c49f6c3eb04a709545b361fa4b4708'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:16.693789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1fe6392c-fb8f-4126-aa9a-8437837c7875 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67c49f6c3eb04a709545b361fa4b4708'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:16.709515 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 955/1963] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:16 2022] DELETE /identity/v3/users/67c49f6c3eb04a709545b361fa4b4708 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:16.714860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a36b7-aa6c-458c-ad73-1aa388ff6823 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:16.715146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a36b7-aa6c-458c-ad73-1aa388ff6823 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:16.716122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a36b7-aa6c-458c-ad73-1aa388ff6823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:16.739627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-961a36b7-aa6c-458c-ad73-1aa388ff6823 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:16.740615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-961a36b7-aa6c-458c-ad73-1aa388ff6823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:16.809638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:16.860286 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1009/1964] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:16.871673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6392c-fb8f-4126-aa9a-8437837c7875 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:16.873658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1fe6392c-fb8f-4126-aa9a-8437837c7875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:16.896743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:16.975027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6392c-fb8f-4126-aa9a-8437837c7875 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:16.979056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7564c14-8ad8-48e4-9a5a-d4704640fbdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:16.979895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7564c14-8ad8-48e4-9a5a-d4704640fbdc service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:16.980189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7564c14-8ad8-48e4-9a5a-d4704640fbdc service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:16.980520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7564c14-8ad8-48e4-9a5a-d4704640fbdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:16.992145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7564c14-8ad8-48e4-9a5a-d4704640fbdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:16.993782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7564c14-8ad8-48e4-9a5a-d4704640fbdc service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:17.006278 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 956/1965] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:17.091089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a36b7-aa6c-458c-ad73-1aa388ff6823 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:17.096317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-961a36b7-aa6c-458c-ad73-1aa388ff6823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:17.123556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a36b7-aa6c-458c-ad73-1aa388ff6823 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:17.139302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc6235-14b0-467e-93d5-cc2ec60fe179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:17.140128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc6235-14b0-467e-93d5-cc2ec60fe179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:17.140370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc6235-14b0-467e-93d5-cc2ec60fe179 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:17.140647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc6235-14b0-467e-93d5-cc2ec60fe179 admin admin] PATH_INFO: `/v3/projects/1048abb881ff4ede869168468c185f28` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:17.141813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc6235-14b0-467e-93d5-cc2ec60fe179 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1048abb881ff4ede869168468c185f28)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:17.144437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc6235-14b0-467e-93d5-cc2ec60fe179 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:17.200293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eedc6235-14b0-467e-93d5-cc2ec60fe179 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1048abb881ff4ede869168468c185f28', 'request_id': 'req-eedc6235-14b0-467e-93d5-cc2ec60fe179'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.206761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eedc6235-14b0-467e-93d5-cc2ec60fe179 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1048abb881ff4ede869168468c185f28', 'request_id': 'req-eedc6235-14b0-467e-93d5-cc2ec60fe179'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.214126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eedc6235-14b0-467e-93d5-cc2ec60fe179 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1048abb881ff4ede869168468c185f28', 'request_id': 'req-eedc6235-14b0-467e-93d5-cc2ec60fe179'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.227195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eedc6235-14b0-467e-93d5-cc2ec60fe179 admin admin] The token cache is being invalidate because project 1048abb881ff4ede869168468c185f28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:17.227691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eedc6235-14b0-467e-93d5-cc2ec60fe179 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.228498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eedc6235-14b0-467e-93d5-cc2ec60fe179 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.231032 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1010/1966] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:17 2022] DELETE /identity/v3/projects/1048abb881ff4ede869168468c185f28 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:49:17.235706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7564c14-8ad8-48e4-9a5a-d4704640fbdc service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:17.246228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7564c14-8ad8-48e4-9a5a-d4704640fbdc service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:17.325574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f4381-c48b-4ade-a102-09dc085a7d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:17.326249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f4381-c48b-4ade-a102-09dc085a7d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:17.326714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f4381-c48b-4ade-a102-09dc085a7d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:17.327055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f4381-c48b-4ade-a102-09dc085a7d49 admin admin] PATH_INFO: `/v3/users/1763387ee81341c6bb757ba986f0e39f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:17.328138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f4381-c48b-4ade-a102-09dc085a7d49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1763387ee81341c6bb757ba986f0e39f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:17.330080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f4381-c48b-4ade-a102-09dc085a7d49 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:17.387229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7a6f4381-c48b-4ade-a102-09dc085a7d49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1763387ee81341c6bb757ba986f0e39f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.387951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7a6f4381-c48b-4ade-a102-09dc085a7d49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1763387ee81341c6bb757ba986f0e39f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.390053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7a6f4381-c48b-4ade-a102-09dc085a7d49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1763387ee81341c6bb757ba986f0e39f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.401339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7a6f4381-c48b-4ade-a102-09dc085a7d49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1763387ee81341c6bb757ba986f0e39f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.411940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc6235-14b0-467e-93d5-cc2ec60fe179 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:17.413893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-eedc6235-14b0-467e-93d5-cc2ec60fe179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:17.424854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7a6f4381-c48b-4ade-a102-09dc085a7d49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1763387ee81341c6bb757ba986f0e39f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.428654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc6235-14b0-467e-93d5-cc2ec60fe179 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:17.436045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7ed38-f2ad-4d56-aa2c-41a1e392b54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:17.436695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7ed38-f2ad-4d56-aa2c-41a1e392b54f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:17.437528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7ed38-f2ad-4d56-aa2c-41a1e392b54f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:17.437528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7ed38-f2ad-4d56-aa2c-41a1e392b54f admin admin] PATH_INFO: `/v3/projects/d8ce7a62c5cc4bc1bc4ab07e6138d84f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:17.438595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7ed38-f2ad-4d56-aa2c-41a1e392b54f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8ce7a62c5cc4bc1bc4ab07e6138d84f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:17.440515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7ed38-f2ad-4d56-aa2c-41a1e392b54f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:17.456688 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 957/1967] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:17 2022] DELETE /identity/v3/users/1763387ee81341c6bb757ba986f0e39f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:17.510440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2de7ed38-f2ad-4d56-aa2c-41a1e392b54f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8ce7a62c5cc4bc1bc4ab07e6138d84f', 'request_id': 'req-2de7ed38-f2ad-4d56-aa2c-41a1e392b54f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.521293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2de7ed38-f2ad-4d56-aa2c-41a1e392b54f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8ce7a62c5cc4bc1bc4ab07e6138d84f', 'request_id': 'req-2de7ed38-f2ad-4d56-aa2c-41a1e392b54f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.527612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2de7ed38-f2ad-4d56-aa2c-41a1e392b54f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8ce7a62c5cc4bc1bc4ab07e6138d84f', 'request_id': 'req-2de7ed38-f2ad-4d56-aa2c-41a1e392b54f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.540850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2de7ed38-f2ad-4d56-aa2c-41a1e392b54f admin admin] The token cache is being invalidate because project d8ce7a62c5cc4bc1bc4ab07e6138d84f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:17.541852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2de7ed38-f2ad-4d56-aa2c-41a1e392b54f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.542576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2de7ed38-f2ad-4d56-aa2c-41a1e392b54f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.544326 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1011/1968] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:17 2022] DELETE /identity/v3/projects/d8ce7a62c5cc4bc1bc4ab07e6138d84f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:49:17.548509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f4381-c48b-4ade-a102-09dc085a7d49 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:17.550014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7a6f4381-c48b-4ade-a102-09dc085a7d49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:17.589559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:17.634813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f4381-c48b-4ade-a102-09dc085a7d49 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:17.756141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e92ca-f5a7-485e-bd21-47810aeff0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:17.756790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e92ca-f5a7-485e-bd21-47810aeff0e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:17.757065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e92ca-f5a7-485e-bd21-47810aeff0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:17.757841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e92ca-f5a7-485e-bd21-47810aeff0e2 admin admin] PATH_INFO: `/v3/users/920704d8a66c47a7b8f6c35bab9dc398` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:17.758987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571e92ca-f5a7-485e-bd21-47810aeff0e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=920704d8a66c47a7b8f6c35bab9dc398)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:17.760671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571e92ca-f5a7-485e-bd21-47810aeff0e2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:17.811606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-571e92ca-f5a7-485e-bd21-47810aeff0e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '920704d8a66c47a7b8f6c35bab9dc398'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.812299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-571e92ca-f5a7-485e-bd21-47810aeff0e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '920704d8a66c47a7b8f6c35bab9dc398'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.812932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-571e92ca-f5a7-485e-bd21-47810aeff0e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '920704d8a66c47a7b8f6c35bab9dc398'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.819043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7ed38-f2ad-4d56-aa2c-41a1e392b54f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:17.820753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2de7ed38-f2ad-4d56-aa2c-41a1e392b54f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:17.823766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-571e92ca-f5a7-485e-bd21-47810aeff0e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '920704d8a66c47a7b8f6c35bab9dc398'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.836019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7ed38-f2ad-4d56-aa2c-41a1e392b54f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:17.839738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-571e92ca-f5a7-485e-bd21-47810aeff0e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '920704d8a66c47a7b8f6c35bab9dc398'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.841704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726912d-93ef-4b8c-94be-f56f5b7998f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:17.842344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726912d-93ef-4b8c-94be-f56f5b7998f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:17.842807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726912d-93ef-4b8c-94be-f56f5b7998f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:17.843075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726912d-93ef-4b8c-94be-f56f5b7998f2 admin admin] PATH_INFO: `/v3/projects/88450b008137436c8bd03cc66fd43637` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:17.844218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a726912d-93ef-4b8c-94be-f56f5b7998f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88450b008137436c8bd03cc66fd43637)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:17.846144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a726912d-93ef-4b8c-94be-f56f5b7998f2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:17.866023 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 958/1969] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:17 2022] DELETE /identity/v3/users/920704d8a66c47a7b8f6c35bab9dc398 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:17.905166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:17.909857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a726912d-93ef-4b8c-94be-f56f5b7998f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88450b008137436c8bd03cc66fd43637', 'request_id': 'req-a726912d-93ef-4b8c-94be-f56f5b7998f2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.919019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a726912d-93ef-4b8c-94be-f56f5b7998f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88450b008137436c8bd03cc66fd43637', 'request_id': 'req-a726912d-93ef-4b8c-94be-f56f5b7998f2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.926155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a726912d-93ef-4b8c-94be-f56f5b7998f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88450b008137436c8bd03cc66fd43637', 'request_id': 'req-a726912d-93ef-4b8c-94be-f56f5b7998f2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.946741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a726912d-93ef-4b8c-94be-f56f5b7998f2 admin admin] The token cache is being invalidate because project 88450b008137436c8bd03cc66fd43637 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:17.947154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a726912d-93ef-4b8c-94be-f56f5b7998f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.948532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a726912d-93ef-4b8c-94be-f56f5b7998f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:17.950594 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1012/1970] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:17 2022] DELETE /identity/v3/projects/88450b008137436c8bd03cc66fd43637 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:49:17.956068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f087a69-14dc-4817-a38f-a504bd3c370f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:17.956355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f087a69-14dc-4817-a38f-a504bd3c370f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:17.956627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f087a69-14dc-4817-a38f-a504bd3c370f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:17.977379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0f087a69-14dc-4817-a38f-a504bd3c370f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:17.978714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0f087a69-14dc-4817-a38f-a504bd3c370f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:18.048028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:18.062314 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 959/1971] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:18.071738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4599516-9f8e-4551-9b72-dc40a8ab4267 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:18.072983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4599516-9f8e-4551-9b72-dc40a8ab4267 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:18.073256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4599516-9f8e-4551-9b72-dc40a8ab4267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:18.095009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e4599516-9f8e-4551-9b72-dc40a8ab4267 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:18.096305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e4599516-9f8e-4551-9b72-dc40a8ab4267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:18.104812 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1013/1972] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:49:18.109041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f087a69-14dc-4817-a38f-a504bd3c370f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:18.118577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f087a69-14dc-4817-a38f-a504bd3c370f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:18.123887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09509179-d04d-4918-babb-ddd108e618ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:18.124494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09509179-d04d-4918-babb-ddd108e618ec admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:18.124774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09509179-d04d-4918-babb-ddd108e618ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:18.125047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09509179-d04d-4918-babb-ddd108e618ec admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:18.125693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09509179-d04d-4918-babb-ddd108e618ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:49:18.125994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09509179-d04d-4918-babb-ddd108e618ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:18.127634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09509179-d04d-4918-babb-ddd108e618ec admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:18.151418 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 960/1973] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:49:18.155836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4599516-9f8e-4551-9b72-dc40a8ab4267 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:18.165284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4599516-9f8e-4551-9b72-dc40a8ab4267 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:18.173345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09509179-d04d-4918-babb-ddd108e618ec admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:18.174700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f028f7f-ead3-4cc7-88f2-fd1d73fdb298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:18.176078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-09509179-d04d-4918-babb-ddd108e618ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:18.176894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f028f7f-ead3-4cc7-88f2-fd1d73fdb298 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:18.177552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f028f7f-ead3-4cc7-88f2-fd1d73fdb298 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:18.178074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f028f7f-ead3-4cc7-88f2-fd1d73fdb298 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:18.178921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f028f7f-ead3-4cc7-88f2-fd1d73fdb298 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:18.181113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f028f7f-ead3-4cc7-88f2-fd1d73fdb298 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:18.190290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09509179-d04d-4918-babb-ddd108e618ec admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:18.196370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db1b65-b1b6-4e96-b6ac-ea99dd5de43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:18.196712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db1b65-b1b6-4e96-b6ac-ea99dd5de43d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:18.196993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db1b65-b1b6-4e96-b6ac-ea99dd5de43d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:18.197267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db1b65-b1b6-4e96-b6ac-ea99dd5de43d admin admin] PATH_INFO: `/v3/projects/f390bd8ba99a4e928d54c15eb3ab2bb5` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:18.200727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db1b65-b1b6-4e96-b6ac-ea99dd5de43d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f390bd8ba99a4e928d54c15eb3ab2bb5)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:18.202720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db1b65-b1b6-4e96-b6ac-ea99dd5de43d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:18.223937 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1014/1974] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:18 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:18.227484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f028f7f-ead3-4cc7-88f2-fd1d73fdb298 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:18.294312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-96db1b65-b1b6-4e96-b6ac-ea99dd5de43d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f390bd8ba99a4e928d54c15eb3ab2bb5', 'request_id': 'req-96db1b65-b1b6-4e96-b6ac-ea99dd5de43d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:18.305986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-96db1b65-b1b6-4e96-b6ac-ea99dd5de43d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f390bd8ba99a4e928d54c15eb3ab2bb5', 'request_id': 'req-96db1b65-b1b6-4e96-b6ac-ea99dd5de43d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:18.314857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-96db1b65-b1b6-4e96-b6ac-ea99dd5de43d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f390bd8ba99a4e928d54c15eb3ab2bb5', 'request_id': 'req-96db1b65-b1b6-4e96-b6ac-ea99dd5de43d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:18.339135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-96db1b65-b1b6-4e96-b6ac-ea99dd5de43d admin admin] The token cache is being invalidate because project f390bd8ba99a4e928d54c15eb3ab2bb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:18.339658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-96db1b65-b1b6-4e96-b6ac-ea99dd5de43d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:18.340570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-96db1b65-b1b6-4e96-b6ac-ea99dd5de43d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:18.343554 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 961/1975] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:18 2022] DELETE /identity/v3/projects/f390bd8ba99a4e928d54c15eb3ab2bb5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:49:18.359194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7fdb4-04fd-426a-9c35-d2480f11269c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:18.359194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7fdb4-04fd-426a-9c35-d2480f11269c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:18.359194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7fdb4-04fd-426a-9c35-d2480f11269c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:18.364500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f028f7f-ead3-4cc7-88f2-fd1d73fdb298 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:18.380387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4ca7fdb4-04fd-426a-9c35-d2480f11269c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:18.381983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4ca7fdb4-04fd-426a-9c35-d2480f11269c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:18.440754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e01f5-c744-4f08-9880-f97215ea3387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:18.441322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e01f5-c744-4f08-9880-f97215ea3387 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:18.441847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e01f5-c744-4f08-9880-f97215ea3387 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:18.443503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e01f5-c744-4f08-9880-f97215ea3387 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:18.443807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e01f5-c744-4f08-9880-f97215ea3387 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:18.445752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e01f5-c744-4f08-9880-f97215ea3387 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:18.479818 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1015/1976] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:18 2022] POST /identity/v3/users => generated 291 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:18.481119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e01f5-c744-4f08-9880-f97215ea3387 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:18.483097 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 962/1977] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:18.483215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e82e01f5-c744-4f08-9880-f97215ea3387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:18.490994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c4423-c317-4b09-9eaf-7784052c5d21 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:18.491687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c4423-c317-4b09-9eaf-7784052c5d21 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:18.492193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c4423-c317-4b09-9eaf-7784052c5d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:18.514681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e01f5-c744-4f08-9880-f97215ea3387 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:18.526161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0e084-5c68-4bba-9f2c-3ec8195dedf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:18.527039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0e084-5c68-4bba-9f2c-3ec8195dedf0 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:18.528655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0e084-5c68-4bba-9f2c-3ec8195dedf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:18.529237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0e084-5c68-4bba-9f2c-3ec8195dedf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:18.530019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0e084-5c68-4bba-9f2c-3ec8195dedf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:18.534933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0e084-5c68-4bba-9f2c-3ec8195dedf0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:18.538504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e63c4423-c317-4b09-9eaf-7784052c5d21 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:18.543131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e63c4423-c317-4b09-9eaf-7784052c5d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:18.555197 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1016/1978] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:49:18.559570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0e084-5c68-4bba-9f2c-3ec8195dedf0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:18.567505 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 963/1979] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:18.572099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c4423-c317-4b09-9eaf-7784052c5d21 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:18.586405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0e084-5c68-4bba-9f2c-3ec8195dedf0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:18.590802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c4423-c317-4b09-9eaf-7784052c5d21 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:18.597466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239704a-d873-440f-9ed7-92d5668879c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:18.599289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239704a-d873-440f-9ed7-92d5668879c5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:18.600010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239704a-d873-440f-9ed7-92d5668879c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:18.601314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239704a-d873-440f-9ed7-92d5668879c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:18.602249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0beb3-f34e-4152-bd72-88df20863141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:18.603090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239704a-d873-440f-9ed7-92d5668879c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:18.614986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0beb3-f34e-4152-bd72-88df20863141 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:18.615433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0beb3-f34e-4152-bd72-88df20863141 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:18.615629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0beb3-f34e-4152-bd72-88df20863141 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:18.616315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0beb3-f34e-4152-bd72-88df20863141 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:49:18.616622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0beb3-f34e-4152-bd72-88df20863141 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:18.618263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0beb3-f34e-4152-bd72-88df20863141 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:18.621157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239704a-d873-440f-9ed7-92d5668879c5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:18.644715 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1017/1980] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:18.648711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239704a-d873-440f-9ed7-92d5668879c5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:18.666926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239704a-d873-440f-9ed7-92d5668879c5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:18.668345 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 964/1981] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:49:18.671712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0beb3-f34e-4152-bd72-88df20863141 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:18.673538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3293a51-39e5-458d-9031-39bc60ea6e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:18.674829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3293a51-39e5-458d-9031-39bc60ea6e51 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:18.675187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3293a51-39e5-458d-9031-39bc60ea6e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:18.675652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3293a51-39e5-458d-9031-39bc60ea6e51 admin admin] PATH_INFO: `/v3/projects/71114ee297854826be0cdd490ddb3184/users/4aeedfffb4d24b9e956335741f00ee27/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:18.695417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3293a51-39e5-458d-9031-39bc60ea6e51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71114ee297854826be0cdd490ddb3184, user_id=4aeedfffb4d24b9e956335741f00ee27, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:18.695924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0beb3-f34e-4152-bd72-88df20863141 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:18.701036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3293a51-39e5-458d-9031-39bc60ea6e51 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:18.703844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:18.724331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b06d5-4f57-4e81-9b80-34dc83e43cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:18.726470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b06d5-4f57-4e81-9b80-34dc83e43cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:18.726853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b06d5-4f57-4e81-9b80-34dc83e43cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:18.728816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b06d5-4f57-4e81-9b80-34dc83e43cc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:18.731184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b06d5-4f57-4e81-9b80-34dc83e43cc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:18.739409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b06d5-4f57-4e81-9b80-34dc83e43cc3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:18.752360 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1018/1982] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:18 2022] PUT /identity/v3/projects/71114ee297854826be0cdd490ddb3184/users/4aeedfffb4d24b9e956335741f00ee27/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:18.764252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f503760-2746-48ef-a9d6-8e3b4884c040 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:18.764670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f503760-2746-48ef-a9d6-8e3b4884c040 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:18.765139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f503760-2746-48ef-a9d6-8e3b4884c040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:18.853205 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 965/1983] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:18 2022] POST /identity/v3/projects => generated 387 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:18.858387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b06d5-4f57-4e81-9b80-34dc83e43cc3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:18.881214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0f503760-2746-48ef-a9d6-8e3b4884c040 None None] MFA Rules not processed for user `4aeedfffb4d24b9e956335741f00ee27`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:18.884441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0f503760-2746-48ef-a9d6-8e3b4884c040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:18.943269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:19.021051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b06d5-4f57-4e81-9b80-34dc83e43cc3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:19.035208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af6b9f-666d-4645-af4e-d654c5808c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:19.036409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af6b9f-666d-4645-af4e-d654c5808c63 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:19.036824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af6b9f-666d-4645-af4e-d654c5808c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:19.037260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af6b9f-666d-4645-af4e-d654c5808c63 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:19.038069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af6b9f-666d-4645-af4e-d654c5808c63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:19.041423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af6b9f-666d-4645-af4e-d654c5808c63 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:19.053521 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1019/1984] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:49:19.068638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f503760-2746-48ef-a9d6-8e3b4884c040 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:19.072986 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 966/1985] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:18 2022] POST /identity/v3/users => generated 294 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:19.074119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af6b9f-666d-4645-af4e-d654c5808c63 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:19.084066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f503760-2746-48ef-a9d6-8e3b4884c040 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:19.086831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af6b9f-666d-4645-af4e-d654c5808c63 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:19.090000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2d7ae-0760-41fd-8e9f-8748abd96e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:19.090722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2d7ae-0760-41fd-8e9f-8748abd96e96 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:19.091001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2d7ae-0760-41fd-8e9f-8748abd96e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:19.091279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2d7ae-0760-41fd-8e9f-8748abd96e96 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:19.091925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e2d7ae-0760-41fd-8e9f-8748abd96e96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:19.092821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545726f1-0e15-46de-a8bc-945ef952f0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:19.093651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545726f1-0e15-46de-a8bc-945ef952f0ea admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:19.093936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e2d7ae-0760-41fd-8e9f-8748abd96e96 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:19.094109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545726f1-0e15-46de-a8bc-945ef952f0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:19.094478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545726f1-0e15-46de-a8bc-945ef952f0ea admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:19.095506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545726f1-0e15-46de-a8bc-945ef952f0ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:19.097254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545726f1-0e15-46de-a8bc-945ef952f0ea admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:19.117711 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 967/1986] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:19.121905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545726f1-0e15-46de-a8bc-945ef952f0ea admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:19.154393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545726f1-0e15-46de-a8bc-945ef952f0ea admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:19.163960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5881592-85c7-4e40-83a3-f3b43817e36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:19.164730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5881592-85c7-4e40-83a3-f3b43817e36e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:19.165020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5881592-85c7-4e40-83a3-f3b43817e36e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:19.165300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5881592-85c7-4e40-83a3-f3b43817e36e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:19.166175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5881592-85c7-4e40-83a3-f3b43817e36e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:19.168152 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1020/1987] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:19 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:19.168638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5881592-85c7-4e40-83a3-f3b43817e36e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:19.183912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2d7ae-0760-41fd-8e9f-8748abd96e96 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:19.188938 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 968/1988] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:19.201318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5881592-85c7-4e40-83a3-f3b43817e36e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:19.227414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5881592-85c7-4e40-83a3-f3b43817e36e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:19.304211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2d7ae-0760-41fd-8e9f-8748abd96e96 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:19.305829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc29ed6-3835-489b-9d15-37337af42636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:19.306688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc29ed6-3835-489b-9d15-37337af42636 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:19.307554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc29ed6-3835-489b-9d15-37337af42636 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:19.307830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc29ed6-3835-489b-9d15-37337af42636 admin admin] PATH_INFO: `/v3/projects/b8fac8bdf98d46fd979e2c1bc15cf7ce/users/acc2b63414674809aa8268df78b7c8e1/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:19.311501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8bba6-143c-4659-917c-22ecfcf58c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:19.312255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8bba6-143c-4659-917c-22ecfcf58c6f admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:19.312638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8bba6-143c-4659-917c-22ecfcf58c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:19.313013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8bba6-143c-4659-917c-22ecfcf58c6f admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:19.313746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8bba6-143c-4659-917c-22ecfcf58c6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:19.316483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8bba6-143c-4659-917c-22ecfcf58c6f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:19.324819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc29ed6-3835-489b-9d15-37337af42636 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8fac8bdf98d46fd979e2c1bc15cf7ce, user_id=acc2b63414674809aa8268df78b7c8e1, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:19.327694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc29ed6-3835-489b-9d15-37337af42636 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:19.346121 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1021/1990] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:19 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:19.349819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8bba6-143c-4659-917c-22ecfcf58c6f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:19.354727 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 969/1990] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:19 2022] PUT /identity/v3/projects/b8fac8bdf98d46fd979e2c1bc15cf7ce/users/acc2b63414674809aa8268df78b7c8e1/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:19.356987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279b0cb-8227-410a-8ed1-11f8c9a0d083 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:19.359916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279b0cb-8227-410a-8ed1-11f8c9a0d083 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:19.360355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279b0cb-8227-410a-8ed1-11f8c9a0d083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:19.362144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8bba6-143c-4659-917c-22ecfcf58c6f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:19.442288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1279b0cb-8227-410a-8ed1-11f8c9a0d083 None None] MFA Rules not processed for user `acc2b63414674809aa8268df78b7c8e1`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:19.446675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1279b0cb-8227-410a-8ed1-11f8c9a0d083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:19.471566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:19.484597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22c242-d0e1-4c63-a393-2413b7023ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:19.485819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22c242-d0e1-4c63-a393-2413b7023ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:19.486314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22c242-d0e1-4c63-a393-2413b7023ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:19.486844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22c242-d0e1-4c63-a393-2413b7023ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:19.487753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22c242-d0e1-4c63-a393-2413b7023ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:19.489708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22c242-d0e1-4c63-a393-2413b7023ae9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:19.502455 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1022/1991] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:19.507440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22c242-d0e1-4c63-a393-2413b7023ae9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:19.533268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22c242-d0e1-4c63-a393-2413b7023ae9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:19.557353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563d5ee-fb5f-4d5a-b509-d6f12dc492b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:19.558106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563d5ee-fb5f-4d5a-b509-d6f12dc492b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:19.560311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563d5ee-fb5f-4d5a-b509-d6f12dc492b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:19.563343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563d5ee-fb5f-4d5a-b509-d6f12dc492b4 admin admin] PATH_INFO: `/v3/projects/8122a44aaea0440594916941ab6a3038/users/b30d939c4aca4419b6e71010bbe143de/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:19.598379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563d5ee-fb5f-4d5a-b509-d6f12dc492b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8122a44aaea0440594916941ab6a3038, user_id=b30d939c4aca4419b6e71010bbe143de, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:19.601322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563d5ee-fb5f-4d5a-b509-d6f12dc492b4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:19.617169 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 970/1992] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:19.634260 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1023/1993] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:19 2022] PUT /identity/v3/projects/8122a44aaea0440594916941ab6a3038/users/b30d939c4aca4419b6e71010bbe143de/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:19.639675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279b0cb-8227-410a-8ed1-11f8c9a0d083 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:19.639805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563d5ee-fb5f-4d5a-b509-d6f12dc492b4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:19.649142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:19.657291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279b0cb-8227-410a-8ed1-11f8c9a0d083 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:19.756821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29864499-6d75-4fdd-b34f-d63b53be1a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:19.758810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29864499-6d75-4fdd-b34f-d63b53be1a1b admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:19.759300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29864499-6d75-4fdd-b34f-d63b53be1a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:19.759969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29864499-6d75-4fdd-b34f-d63b53be1a1b admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:19.761901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29864499-6d75-4fdd-b34f-d63b53be1a1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:19.765225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29864499-6d75-4fdd-b34f-d63b53be1a1b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:19.783395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563d5ee-fb5f-4d5a-b509-d6f12dc492b4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:19.799258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e360d-ddd8-4c31-864b-46da0f534a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:19.802625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e360d-ddd8-4c31-864b-46da0f534a6b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:19.803607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e360d-ddd8-4c31-864b-46da0f534a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:19.803607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e360d-ddd8-4c31-864b-46da0f534a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:19.804273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e360d-ddd8-4c31-864b-46da0f534a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:19.807494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e360d-ddd8-4c31-864b-46da0f534a6b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:19.812011 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 971/1994] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:19 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:19.815811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29864499-6d75-4fdd-b34f-d63b53be1a1b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:19.842717 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1024/1995] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:19.842717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e360d-ddd8-4c31-864b-46da0f534a6b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:19.863306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e360d-ddd8-4c31-864b-46da0f534a6b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:19.937932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29864499-6d75-4fdd-b34f-d63b53be1a1b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:19.944756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e849e9-42f8-46ae-89dc-f432dfef621a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:19.945640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e849e9-42f8-46ae-89dc-f432dfef621a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:19.945992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e849e9-42f8-46ae-89dc-f432dfef621a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:19.946337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e849e9-42f8-46ae-89dc-f432dfef621a admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:19.947073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e849e9-42f8-46ae-89dc-f432dfef621a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:19.953724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e849e9-42f8-46ae-89dc-f432dfef621a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:19.983701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db735c-0096-47e6-b877-846cc0b6c49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:19.984181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db735c-0096-47e6-b877-846cc0b6c49c admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:19.984464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db735c-0096-47e6-b877-846cc0b6c49c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:19.984735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db735c-0096-47e6-b877-846cc0b6c49c admin admin] PATH_INFO: `/v3/projects/8122a44aaea0440594916941ab6a3038/users/b30d939c4aca4419b6e71010bbe143de/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:19.988047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db735c-0096-47e6-b877-846cc0b6c49c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8122a44aaea0440594916941ab6a3038, user_id=b30d939c4aca4419b6e71010bbe143de, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:19.993053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db735c-0096-47e6-b877-846cc0b6c49c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:19.995729 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 972/1996] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:19 2022] POST /identity/v3/users => generated 299 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:19.999296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e849e9-42f8-46ae-89dc-f432dfef621a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:20.013853 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1025/1997] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:19 2022] PUT /identity/v3/projects/8122a44aaea0440594916941ab6a3038/users/b30d939c4aca4419b6e71010bbe143de/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:20.023060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e849e9-42f8-46ae-89dc-f432dfef621a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:20.028652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e9519-c146-48fb-bc27-83f992309bae None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:20.028850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e9519-c146-48fb-bc27-83f992309bae None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:20.029008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e9519-c146-48fb-bc27-83f992309bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:20.081524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ad4e9519-c146-48fb-bc27-83f992309bae None None] MFA Rules not processed for user `b30d939c4aca4419b6e71010bbe143de`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:20.084440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ad4e9519-c146-48fb-bc27-83f992309bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:20.141425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:20.169829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b73352-88dc-4764-aa2d-552096141edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:20.170801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b73352-88dc-4764-aa2d-552096141edf admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:20.171045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b73352-88dc-4764-aa2d-552096141edf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:20.171433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b73352-88dc-4764-aa2d-552096141edf admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:20.173190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b73352-88dc-4764-aa2d-552096141edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:20.176106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b73352-88dc-4764-aa2d-552096141edf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:20.186639 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1026/1998] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:20.195620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e9519-c146-48fb-bc27-83f992309bae None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:20.196597 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 973/1999] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:49:20.198064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ad4e9519-c146-48fb-bc27-83f992309bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:20.200835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b73352-88dc-4764-aa2d-552096141edf admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:20.214384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b73352-88dc-4764-aa2d-552096141edf admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:20.221220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd0702-dfd6-466e-a1b9-10c5e49d3390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:20.222047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd0702-dfd6-466e-a1b9-10c5e49d3390 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:20.223573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd0702-dfd6-466e-a1b9-10c5e49d3390 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:20.224062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd0702-dfd6-466e-a1b9-10c5e49d3390 admin admin] PATH_INFO: `/v3/projects/87f93a0a006a40f2bb0617a9431f402f/users/dcf027b75c09410ab02ca4a9c7f4c514/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:20.242880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:20.248860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd0702-dfd6-466e-a1b9-10c5e49d3390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87f93a0a006a40f2bb0617a9431f402f, user_id=dcf027b75c09410ab02ca4a9c7f4c514, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:20.253962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd0702-dfd6-466e-a1b9-10c5e49d3390 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:20.277297 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 974/2000] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:20 2022] PUT /identity/v3/projects/87f93a0a006a40f2bb0617a9431f402f/users/dcf027b75c09410ab02ca4a9c7f4c514/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:20.281085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd0702-dfd6-466e-a1b9-10c5e49d3390 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:20.294963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e9519-c146-48fb-bc27-83f992309bae None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:20.396844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd0702-dfd6-466e-a1b9-10c5e49d3390 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:20.405382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6e34f-6777-444f-845b-c2771204aea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:20.406761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6e34f-6777-444f-845b-c2771204aea2 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:20.407893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6e34f-6777-444f-845b-c2771204aea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:20.408364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6e34f-6777-444f-845b-c2771204aea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:20.409081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6e34f-6777-444f-845b-c2771204aea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:20.409725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6fc7d-d893-4504-8b9b-45e100dc9842 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:20.410508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6fc7d-d893-4504-8b9b-45e100dc9842 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:20.410798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6e34f-6777-444f-845b-c2771204aea2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:20.410943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6fc7d-d893-4504-8b9b-45e100dc9842 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:20.411317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6fc7d-d893-4504-8b9b-45e100dc9842 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:20.430942 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 975/2001] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:49:20.430942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6e34f-6777-444f-845b-c2771204aea2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:20.432920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e3a6e34f-6777-444f-845b-c2771204aea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:20.456889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6e34f-6777-444f-845b-c2771204aea2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:20.467560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e971c6-2d16-4843-a7ac-487ebe963848 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:20.468290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e971c6-2d16-4843-a7ac-487ebe963848 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:20.468663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e971c6-2d16-4843-a7ac-487ebe963848 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:20.469153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e971c6-2d16-4843-a7ac-487ebe963848 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:20.471652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-06e971c6-2d16-4843-a7ac-487ebe963848 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:20.502895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6fc7d-d893-4504-8b9b-45e100dc9842 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:20.505255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6fc7d-d893-4504-8b9b-45e100dc9842 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:20.519612 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1027/2002] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:20 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:20.520750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6fc7d-d893-4504-8b9b-45e100dc9842 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:20.535138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6fc7d-d893-4504-8b9b-45e100dc9842 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:20.543040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18820e1d-f09f-4623-8789-caf23f638550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:20.544019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18820e1d-f09f-4623-8789-caf23f638550 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:20.544331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18820e1d-f09f-4623-8789-caf23f638550 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:20.544586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18820e1d-f09f-4623-8789-caf23f638550 admin admin] PATH_INFO: `/v3/projects/87f93a0a006a40f2bb0617a9431f402f/users/dcf027b75c09410ab02ca4a9c7f4c514/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:20.546758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18820e1d-f09f-4623-8789-caf23f638550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87f93a0a006a40f2bb0617a9431f402f, user_id=dcf027b75c09410ab02ca4a9c7f4c514, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:20.550565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18820e1d-f09f-4623-8789-caf23f638550 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:20.554833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:20.579849 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1028/2003] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:20 2022] PUT /identity/v3/projects/87f93a0a006a40f2bb0617a9431f402f/users/dcf027b75c09410ab02ca4a9c7f4c514/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:20.583203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ad893-f68e-4bf4-82ed-b8d02038ca14 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:20.583511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ad893-f68e-4bf4-82ed-b8d02038ca14 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:20.583793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ad893-f68e-4bf4-82ed-b8d02038ca14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:20.594973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e971c6-2d16-4843-a7ac-487ebe963848 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:20.596941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e971c6-2d16-4843-a7ac-487ebe963848 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:20.612206 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 976/2004] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:20 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:20.628195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b29ad893-f68e-4bf4-82ed-b8d02038ca14 None None] MFA Rules not processed for user `dcf027b75c09410ab02ca4a9c7f4c514`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:20.628195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b29ad893-f68e-4bf4-82ed-b8d02038ca14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:20.741284 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1029/2005] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:20 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:20.815796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e971c6-2d16-4843-a7ac-487ebe963848 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:20.829855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e971c6-2d16-4843-a7ac-487ebe963848 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:20.949042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7839f-d306-4e26-9dd4-3d65207d94fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:20.949787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7839f-d306-4e26-9dd4-3d65207d94fe service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:20.950168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7839f-d306-4e26-9dd4-3d65207d94fe service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:20.950575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7839f-d306-4e26-9dd4-3d65207d94fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:20.961591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7839f-d306-4e26-9dd4-3d65207d94fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:20.963795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7839f-d306-4e26-9dd4-3d65207d94fe service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:21.001941 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 977/2006] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:22.464406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ad893-f68e-4bf4-82ed-b8d02038ca14 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:22.601145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ad893-f68e-4bf4-82ed-b8d02038ca14 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:22.608255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a0f83-9d74-42ae-98b5-6511856ab6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:22.608907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a0f83-9d74-42ae-98b5-6511856ab6e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:22.609189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a0f83-9d74-42ae-98b5-6511856ab6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:22.609458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a0f83-9d74-42ae-98b5-6511856ab6e6 admin admin] PATH_INFO: `/v3/users/acc2b63414674809aa8268df78b7c8e1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:22.610812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a0f83-9d74-42ae-98b5-6511856ab6e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acc2b63414674809aa8268df78b7c8e1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:22.612724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a0f83-9d74-42ae-98b5-6511856ab6e6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:22.735453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2c9a0f83-9d74-42ae-98b5-6511856ab6e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acc2b63414674809aa8268df78b7c8e1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:22.759554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2c9a0f83-9d74-42ae-98b5-6511856ab6e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acc2b63414674809aa8268df78b7c8e1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:22.784249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2c9a0f83-9d74-42ae-98b5-6511856ab6e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acc2b63414674809aa8268df78b7c8e1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:22.786106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2c9a0f83-9d74-42ae-98b5-6511856ab6e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acc2b63414674809aa8268df78b7c8e1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:22.828185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2c9a0f83-9d74-42ae-98b5-6511856ab6e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acc2b63414674809aa8268df78b7c8e1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:22.851089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:22.864071 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1030/2007] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:22 2022] DELETE /identity/v3/users/acc2b63414674809aa8268df78b7c8e1 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:22.867515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ddb81-e3a7-48f3-a65c-5199e403b330 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:22.868078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ddb81-e3a7-48f3-a65c-5199e403b330 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:22.868307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ddb81-e3a7-48f3-a65c-5199e403b330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:22.889643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:22.901383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-048ddb81-e3a7-48f3-a65c-5199e403b330 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:22.903247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-048ddb81-e3a7-48f3-a65c-5199e403b330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:22.978604 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 978/2008] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:22.988595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a0f83-9d74-42ae-98b5-6511856ab6e6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:22.990194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2c9a0f83-9d74-42ae-98b5-6511856ab6e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:22.995154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ddb81-e3a7-48f3-a65c-5199e403b330 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:22.997194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-048ddb81-e3a7-48f3-a65c-5199e403b330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:23.086333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a0f83-9d74-42ae-98b5-6511856ab6e6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:23.090713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73267495-bf46-47e3-ba1a-c7655e1f02eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:23.091378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73267495-bf46-47e3-ba1a-c7655e1f02eb service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:23.091902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73267495-bf46-47e3-ba1a-c7655e1f02eb service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:23.092025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73267495-bf46-47e3-ba1a-c7655e1f02eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:23.102322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73267495-bf46-47e3-ba1a-c7655e1f02eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:23.104424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73267495-bf46-47e3-ba1a-c7655e1f02eb service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:23.113651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ddb81-e3a7-48f3-a65c-5199e403b330 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:23.117606 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1031/2009] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:23.118517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73267495-bf46-47e3-ba1a-c7655e1f02eb service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:23.120446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-73267495-bf46-47e3-ba1a-c7655e1f02eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:23.121666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e052bec-f1bb-4700-9301-c069ff96a13a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:23.122371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e052bec-f1bb-4700-9301-c069ff96a13a service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:23.122664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e052bec-f1bb-4700-9301-c069ff96a13a service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:23.122931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e052bec-f1bb-4700-9301-c069ff96a13a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:23.125248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4e052bec-f1bb-4700-9301-c069ff96a13a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:23.135038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73267495-bf46-47e3-ba1a-c7655e1f02eb service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:23.141585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331205d1-f9ea-4fb2-a4d5-e1aa0cfd9e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:23.142237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331205d1-f9ea-4fb2-a4d5-e1aa0cfd9e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:23.142552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331205d1-f9ea-4fb2-a4d5-e1aa0cfd9e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:23.142833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331205d1-f9ea-4fb2-a4d5-e1aa0cfd9e8e admin admin] PATH_INFO: `/v3/users/4aeedfffb4d24b9e956335741f00ee27` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:23.144041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331205d1-f9ea-4fb2-a4d5-e1aa0cfd9e8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aeedfffb4d24b9e956335741f00ee27)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:23.146552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331205d1-f9ea-4fb2-a4d5-e1aa0cfd9e8e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:23.210359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-331205d1-f9ea-4fb2-a4d5-e1aa0cfd9e8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aeedfffb4d24b9e956335741f00ee27'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:23.219916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-331205d1-f9ea-4fb2-a4d5-e1aa0cfd9e8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aeedfffb4d24b9e956335741f00ee27'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:23.237964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-331205d1-f9ea-4fb2-a4d5-e1aa0cfd9e8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aeedfffb4d24b9e956335741f00ee27'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:23.239162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-331205d1-f9ea-4fb2-a4d5-e1aa0cfd9e8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aeedfffb4d24b9e956335741f00ee27'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:23.251324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-331205d1-f9ea-4fb2-a4d5-e1aa0cfd9e8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aeedfffb4d24b9e956335741f00ee27'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:23.263164 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1032/2010] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:23 2022] DELETE /identity/v3/users/4aeedfffb4d24b9e956335741f00ee27 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:23.268657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4174f3-f5fa-4d5c-ac18-68f31bccab1a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:23.268963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4174f3-f5fa-4d5c-ac18-68f31bccab1a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:23.269310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4174f3-f5fa-4d5c-ac18-68f31bccab1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:23.289327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e052bec-f1bb-4700-9301-c069ff96a13a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:23.292094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e052bec-f1bb-4700-9301-c069ff96a13a service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:23.310186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-bc4174f3-f5fa-4d5c-ac18-68f31bccab1a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:23.312484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bc4174f3-f5fa-4d5c-ac18-68f31bccab1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:23.318891 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 979/2011] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:22 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:23.420628 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1033/2012] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:23.438231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e052bec-f1bb-4700-9301-c069ff96a13a service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:23.439891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4e052bec-f1bb-4700-9301-c069ff96a13a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:23.462727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e052bec-f1bb-4700-9301-c069ff96a13a service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:23.531586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:23.547914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d69171-8e17-4b4c-833e-290821728695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:23.549465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d69171-8e17-4b4c-833e-290821728695 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:23.550157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d69171-8e17-4b4c-833e-290821728695 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:23.550667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d69171-8e17-4b4c-833e-290821728695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:23.563748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d69171-8e17-4b4c-833e-290821728695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:23.566053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d69171-8e17-4b4c-833e-290821728695 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:23.582450 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 980/2013] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:23.873990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4174f3-f5fa-4d5c-ac18-68f31bccab1a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:23.876349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bc4174f3-f5fa-4d5c-ac18-68f31bccab1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:23.893260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4174f3-f5fa-4d5c-ac18-68f31bccab1a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:23.910999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ec73c-4da2-48e1-80e8-c2e115b5ceab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:23.911456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ec73c-4da2-48e1-80e8-c2e115b5ceab admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:23.912192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ec73c-4da2-48e1-80e8-c2e115b5ceab admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:23.912396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ec73c-4da2-48e1-80e8-c2e115b5ceab admin admin] PATH_INFO: `/v3/projects/b8fac8bdf98d46fd979e2c1bc15cf7ce` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:23.913573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ec73c-4da2-48e1-80e8-c2e115b5ceab admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8fac8bdf98d46fd979e2c1bc15cf7ce)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:23.915870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ec73c-4da2-48e1-80e8-c2e115b5ceab admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:23.929630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:23.986676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac9ec73c-4da2-48e1-80e8-c2e115b5ceab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8fac8bdf98d46fd979e2c1bc15cf7ce', 'request_id': 'req-ac9ec73c-4da2-48e1-80e8-c2e115b5ceab'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:23.991077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac9ec73c-4da2-48e1-80e8-c2e115b5ceab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8fac8bdf98d46fd979e2c1bc15cf7ce', 'request_id': 'req-ac9ec73c-4da2-48e1-80e8-c2e115b5ceab'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:23.997527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac9ec73c-4da2-48e1-80e8-c2e115b5ceab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8fac8bdf98d46fd979e2c1bc15cf7ce', 'request_id': 'req-ac9ec73c-4da2-48e1-80e8-c2e115b5ceab'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.013773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac9ec73c-4da2-48e1-80e8-c2e115b5ceab admin admin] The token cache is being invalidate because project b8fac8bdf98d46fd979e2c1bc15cf7ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:24.014128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac9ec73c-4da2-48e1-80e8-c2e115b5ceab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.014983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac9ec73c-4da2-48e1-80e8-c2e115b5ceab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.018174 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1034/2014] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:23 2022] DELETE /identity/v3/projects/b8fac8bdf98d46fd979e2c1bc15cf7ce => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:24.021966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d69171-8e17-4b4c-833e-290821728695 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:24.043512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d69171-8e17-4b4c-833e-290821728695 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:24.114273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfa500-51dc-4bf4-8b11-d2a07fb3cc59 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:24.114723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfa500-51dc-4bf4-8b11-d2a07fb3cc59 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:24.115150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfa500-51dc-4bf4-8b11-d2a07fb3cc59 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:24.117026 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1035/2015] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:49:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:49:24.122747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81b7e6-7b40-42f3-8767-e75983c76488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:24.123412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81b7e6-7b40-42f3-8767-e75983c76488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:24.123886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81b7e6-7b40-42f3-8767-e75983c76488 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:24.124161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81b7e6-7b40-42f3-8767-e75983c76488 admin admin] PATH_INFO: `/v3/users/dcf027b75c09410ab02ca4a9c7f4c514` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:24.125061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe3dcc-baa7-445c-9195-c463c9dfc576 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:24.125353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81b7e6-7b40-42f3-8767-e75983c76488 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcf027b75c09410ab02ca4a9c7f4c514)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:24.125518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe3dcc-baa7-445c-9195-c463c9dfc576 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:24.126087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe3dcc-baa7-445c-9195-c463c9dfc576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:24.127752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81b7e6-7b40-42f3-8767-e75983c76488 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:24.149597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7fbe3dcc-baa7-445c-9195-c463c9dfc576 None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:24.153115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7fbe3dcc-baa7-445c-9195-c463c9dfc576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:24.184862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fb81b7e6-7b40-42f3-8767-e75983c76488 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcf027b75c09410ab02ca4a9c7f4c514'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.185780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fb81b7e6-7b40-42f3-8767-e75983c76488 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcf027b75c09410ab02ca4a9c7f4c514'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.186628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fb81b7e6-7b40-42f3-8767-e75983c76488 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcf027b75c09410ab02ca4a9c7f4c514'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.196966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fb81b7e6-7b40-42f3-8767-e75983c76488 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcf027b75c09410ab02ca4a9c7f4c514'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.217980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fb81b7e6-7b40-42f3-8767-e75983c76488 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcf027b75c09410ab02ca4a9c7f4c514'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.233011 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1036/2016] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:49:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:24.237292 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 981/2017] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:24 2022] DELETE /identity/v3/users/dcf027b75c09410ab02ca4a9c7f4c514 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:24.243609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe3dcc-baa7-445c-9195-c463c9dfc576 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:24.252521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7fbe3dcc-baa7-445c-9195-c463c9dfc576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:24.265558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81b7e6-7b40-42f3-8767-e75983c76488 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:24.268552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fb81b7e6-7b40-42f3-8767-e75983c76488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:24.292593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe3dcc-baa7-445c-9195-c463c9dfc576 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:24.384229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:24.391321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa918d-27ed-4ffb-aa92-2488d8731bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:24.392659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa918d-27ed-4ffb-aa92-2488d8731bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:24.393262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa918d-27ed-4ffb-aa92-2488d8731bba admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:24.393814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa918d-27ed-4ffb-aa92-2488d8731bba admin admin] PATH_INFO: `/v3/projects/71114ee297854826be0cdd490ddb3184` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:24.395427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa918d-27ed-4ffb-aa92-2488d8731bba admin admin] RBAC: Authorizing `identity:delete_project(project_id=71114ee297854826be0cdd490ddb3184)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:24.398819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa918d-27ed-4ffb-aa92-2488d8731bba admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:24.407139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81b7e6-7b40-42f3-8767-e75983c76488 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:24.425325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca2360-9a17-410a-812f-7ad9ab171095 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:24.425974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca2360-9a17-410a-812f-7ad9ab171095 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:24.426256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca2360-9a17-410a-812f-7ad9ab171095 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:24.429126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca2360-9a17-410a-812f-7ad9ab171095 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:24.450784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:24.451686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca2360-9a17-410a-812f-7ad9ab171095 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:24.454392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca2360-9a17-410a-812f-7ad9ab171095 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:24.461720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05fa918d-27ed-4ffb-aa92-2488d8731bba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71114ee297854826be0cdd490ddb3184', 'request_id': 'req-05fa918d-27ed-4ffb-aa92-2488d8731bba'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.470247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05fa918d-27ed-4ffb-aa92-2488d8731bba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71114ee297854826be0cdd490ddb3184', 'request_id': 'req-05fa918d-27ed-4ffb-aa92-2488d8731bba'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.477696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05fa918d-27ed-4ffb-aa92-2488d8731bba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71114ee297854826be0cdd490ddb3184', 'request_id': 'req-05fa918d-27ed-4ffb-aa92-2488d8731bba'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.478886 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 982/2018] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:24.498608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05fa918d-27ed-4ffb-aa92-2488d8731bba admin admin] The token cache is being invalidate because project 71114ee297854826be0cdd490ddb3184 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:24.498754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05fa918d-27ed-4ffb-aa92-2488d8731bba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.503140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05fa918d-27ed-4ffb-aa92-2488d8731bba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.505410 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1037/2019] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:24 2022] DELETE /identity/v3/projects/71114ee297854826be0cdd490ddb3184 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:24.509211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca2360-9a17-410a-812f-7ad9ab171095 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:24.534168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca2360-9a17-410a-812f-7ad9ab171095 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:24.671653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925eab7c-c91f-4c0d-b228-577fdf1b8336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:24.672314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925eab7c-c91f-4c0d-b228-577fdf1b8336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:24.672599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925eab7c-c91f-4c0d-b228-577fdf1b8336 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:24.672881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925eab7c-c91f-4c0d-b228-577fdf1b8336 admin admin] PATH_INFO: `/v3/users/b30d939c4aca4419b6e71010bbe143de` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:24.674049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925eab7c-c91f-4c0d-b228-577fdf1b8336 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b30d939c4aca4419b6e71010bbe143de)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:24.676022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925eab7c-c91f-4c0d-b228-577fdf1b8336 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:24.702840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa918d-27ed-4ffb-aa92-2488d8731bba admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:24.704489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-05fa918d-27ed-4ffb-aa92-2488d8731bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:24.724986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa918d-27ed-4ffb-aa92-2488d8731bba admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:24.744995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843272d8-5759-4b54-82d7-9ede95704a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:24.745679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843272d8-5759-4b54-82d7-9ede95704a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:24.746981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843272d8-5759-4b54-82d7-9ede95704a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:24.747270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843272d8-5759-4b54-82d7-9ede95704a3b admin admin] PATH_INFO: `/v3/projects/87f93a0a006a40f2bb0617a9431f402f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:24.749864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843272d8-5759-4b54-82d7-9ede95704a3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=87f93a0a006a40f2bb0617a9431f402f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:24.755529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843272d8-5759-4b54-82d7-9ede95704a3b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:24.778084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-925eab7c-c91f-4c0d-b228-577fdf1b8336 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b30d939c4aca4419b6e71010bbe143de'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.782360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-925eab7c-c91f-4c0d-b228-577fdf1b8336 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b30d939c4aca4419b6e71010bbe143de'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.783192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-925eab7c-c91f-4c0d-b228-577fdf1b8336 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b30d939c4aca4419b6e71010bbe143de'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.808496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-925eab7c-c91f-4c0d-b228-577fdf1b8336 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b30d939c4aca4419b6e71010bbe143de'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.836219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-925eab7c-c91f-4c0d-b228-577fdf1b8336 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b30d939c4aca4419b6e71010bbe143de'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.855695 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 983/2020] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:24 2022] DELETE /identity/v3/users/b30d939c4aca4419b6e71010bbe143de => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:24.925754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-843272d8-5759-4b54-82d7-9ede95704a3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87f93a0a006a40f2bb0617a9431f402f', 'request_id': 'req-843272d8-5759-4b54-82d7-9ede95704a3b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.950716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-843272d8-5759-4b54-82d7-9ede95704a3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87f93a0a006a40f2bb0617a9431f402f', 'request_id': 'req-843272d8-5759-4b54-82d7-9ede95704a3b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.959912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-843272d8-5759-4b54-82d7-9ede95704a3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87f93a0a006a40f2bb0617a9431f402f', 'request_id': 'req-843272d8-5759-4b54-82d7-9ede95704a3b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.989173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-843272d8-5759-4b54-82d7-9ede95704a3b admin admin] The token cache is being invalidate because project 87f93a0a006a40f2bb0617a9431f402f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:24.990084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-843272d8-5759-4b54-82d7-9ede95704a3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.991116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-843272d8-5759-4b54-82d7-9ede95704a3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:24.995692 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1038/2021] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:24 2022] DELETE /identity/v3/projects/87f93a0a006a40f2bb0617a9431f402f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:49:25.000923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275031a9-47cd-4c73-b673-617d543c610e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:25.001223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275031a9-47cd-4c73-b673-617d543c610e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:25.001509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275031a9-47cd-4c73-b673-617d543c610e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:25.031465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-275031a9-47cd-4c73-b673-617d543c610e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:25.034061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-275031a9-47cd-4c73-b673-617d543c610e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:25.118509 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 984/2022] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:25.126323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe28a1-b0ff-4ba1-8a5b-2895dc1a7597 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:25.126645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe28a1-b0ff-4ba1-8a5b-2895dc1a7597 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:25.126927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe28a1-b0ff-4ba1-8a5b-2895dc1a7597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:25.148940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-cebe28a1-b0ff-4ba1-8a5b-2895dc1a7597 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:25.151956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cebe28a1-b0ff-4ba1-8a5b-2895dc1a7597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:25.166561 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1039/2023] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:25.166788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275031a9-47cd-4c73-b673-617d543c610e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:25.179861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275031a9-47cd-4c73-b673-617d543c610e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:25.187209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752baa9-6ca4-4947-aa40-c97fa9436dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:25.189207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752baa9-6ca4-4947-aa40-c97fa9436dbb admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:25.189728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752baa9-6ca4-4947-aa40-c97fa9436dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:25.191457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752baa9-6ca4-4947-aa40-c97fa9436dbb admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:25.193612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752baa9-6ca4-4947-aa40-c97fa9436dbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:49:25.194135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752baa9-6ca4-4947-aa40-c97fa9436dbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:25.198389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752baa9-6ca4-4947-aa40-c97fa9436dbb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:25.215012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:25.240052 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 985/2024] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:49:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:49:25.241033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe28a1-b0ff-4ba1-8a5b-2895dc1a7597 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:25.250597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe28a1-b0ff-4ba1-8a5b-2895dc1a7597 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:25.255860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052642e4-ceb5-4f89-9e44-4d6c0ddb7955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:25.256470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052642e4-ceb5-4f89-9e44-4d6c0ddb7955 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:25.256929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052642e4-ceb5-4f89-9e44-4d6c0ddb7955 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:25.257149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052642e4-ceb5-4f89-9e44-4d6c0ddb7955 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:25.257790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052642e4-ceb5-4f89-9e44-4d6c0ddb7955 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:25.259787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052642e4-ceb5-4f89-9e44-4d6c0ddb7955 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:25.302783 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1040/2025] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:25 2022] POST /identity/v3/projects => generated 393 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:25.306003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752baa9-6ca4-4947-aa40-c97fa9436dbb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:25.326711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752baa9-6ca4-4947-aa40-c97fa9436dbb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:25.434176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052642e4-ceb5-4f89-9e44-4d6c0ddb7955 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:25.435964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-052642e4-ceb5-4f89-9e44-4d6c0ddb7955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:25.528183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb60c8cb-6737-4d92-8430-5e5e01114dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:25.528816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60c8cb-6737-4d92-8430-5e5e01114dfc admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:25.529885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60c8cb-6737-4d92-8430-5e5e01114dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:25.529885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60c8cb-6737-4d92-8430-5e5e01114dfc admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:25.530089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb60c8cb-6737-4d92-8430-5e5e01114dfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:25.532087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb60c8cb-6737-4d92-8430-5e5e01114dfc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:25.574652 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 986/2026] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:25 2022] POST /identity/v3/users => generated 297 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:25.578454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb60c8cb-6737-4d92-8430-5e5e01114dfc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:25.589931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052642e4-ceb5-4f89-9e44-4d6c0ddb7955 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:25.599140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb60c8cb-6737-4d92-8430-5e5e01114dfc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:25.604128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbece4e-faf6-4223-b2ea-8106d8bf2baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:25.604785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbece4e-faf6-4223-b2ea-8106d8bf2baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:25.605071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbece4e-faf6-4223-b2ea-8106d8bf2baf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:25.605458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbece4e-faf6-4223-b2ea-8106d8bf2baf admin admin] PATH_INFO: `/v3/projects/8122a44aaea0440594916941ab6a3038` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:25.607126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbece4e-faf6-4223-b2ea-8106d8bf2baf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8122a44aaea0440594916941ab6a3038)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:25.609031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbece4e-faf6-4223-b2ea-8106d8bf2baf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:25.619224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d033204-ab71-4767-af27-55f1fbc1fffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:25.619847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d033204-ab71-4767-af27-55f1fbc1fffb admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:25.620127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d033204-ab71-4767-af27-55f1fbc1fffb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:25.620400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d033204-ab71-4767-af27-55f1fbc1fffb admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:25.621055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d033204-ab71-4767-af27-55f1fbc1fffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:25.623223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d033204-ab71-4767-af27-55f1fbc1fffb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:25.640636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:25.649409 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 987/2027] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:25.655521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d033204-ab71-4767-af27-55f1fbc1fffb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:25.686076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dcbece4e-faf6-4223-b2ea-8106d8bf2baf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8122a44aaea0440594916941ab6a3038', 'request_id': 'req-dcbece4e-faf6-4223-b2ea-8106d8bf2baf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:25.694698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dcbece4e-faf6-4223-b2ea-8106d8bf2baf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8122a44aaea0440594916941ab6a3038', 'request_id': 'req-dcbece4e-faf6-4223-b2ea-8106d8bf2baf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:25.696933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d033204-ab71-4767-af27-55f1fbc1fffb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:25.703246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dcbece4e-faf6-4223-b2ea-8106d8bf2baf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8122a44aaea0440594916941ab6a3038', 'request_id': 'req-dcbece4e-faf6-4223-b2ea-8106d8bf2baf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:25.725001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dcbece4e-faf6-4223-b2ea-8106d8bf2baf admin admin] The token cache is being invalidate because project 8122a44aaea0440594916941ab6a3038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:25.725394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dcbece4e-faf6-4223-b2ea-8106d8bf2baf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:25.726155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dcbece4e-faf6-4223-b2ea-8106d8bf2baf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:25.731488 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1041/2028] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:25 2022] DELETE /identity/v3/projects/8122a44aaea0440594916941ab6a3038 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:49:25.733132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a414fc4-24bf-4817-83f6-36a3ebf75d8c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:25.733735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a414fc4-24bf-4817-83f6-36a3ebf75d8c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:25.734049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a414fc4-24bf-4817-83f6-36a3ebf75d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:25.779699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1a414fc4-24bf-4817-83f6-36a3ebf75d8c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:25.780977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1a414fc4-24bf-4817-83f6-36a3ebf75d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:25.816907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3013fb-fdf2-48fa-94ac-619fbb965ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:25.817524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3013fb-fdf2-48fa-94ac-619fbb965ebc admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:25.817788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3013fb-fdf2-48fa-94ac-619fbb965ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:25.818273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3013fb-fdf2-48fa-94ac-619fbb965ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:25.819006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3013fb-fdf2-48fa-94ac-619fbb965ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:25.820690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3013fb-fdf2-48fa-94ac-619fbb965ebc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:25.832642 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 988/2029] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:25.836978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3013fb-fdf2-48fa-94ac-619fbb965ebc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:25.840319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-aa3013fb-fdf2-48fa-94ac-619fbb965ebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:25.865066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3013fb-fdf2-48fa-94ac-619fbb965ebc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:25.871123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7596cf-41a6-4486-9ae5-f62fdc3e0a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:25.872326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7596cf-41a6-4486-9ae5-f62fdc3e0a15 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:25.872752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7596cf-41a6-4486-9ae5-f62fdc3e0a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:25.873168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7596cf-41a6-4486-9ae5-f62fdc3e0a15 admin admin] PATH_INFO: `/v3/projects/158034e03ee1475fa3ef2fcd49191642/users/12d267ccfaad43b58d3fcfe77b295147/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:25.888664 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1042/2030] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:25.898126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e6713-014e-4108-b555-05b4747e184c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:25.898731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7596cf-41a6-4486-9ae5-f62fdc3e0a15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=158034e03ee1475fa3ef2fcd49191642, user_id=12d267ccfaad43b58d3fcfe77b295147, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:25.899055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e6713-014e-4108-b555-05b4747e184c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:25.899772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e6713-014e-4108-b555-05b4747e184c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:25.903223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7596cf-41a6-4486-9ae5-f62fdc3e0a15 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:25.923453 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 989/2031] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:25 2022] PUT /identity/v3/projects/158034e03ee1475fa3ef2fcd49191642/users/12d267ccfaad43b58d3fcfe77b295147/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:49:25.949707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdc098-347c-4432-a83d-1345978de9df None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:25.950338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdc098-347c-4432-a83d-1345978de9df None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:25.950656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdc098-347c-4432-a83d-1345978de9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:25.953565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-631e6713-014e-4108-b555-05b4747e184c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:25.955402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-631e6713-014e-4108-b555-05b4747e184c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:26.046413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d4fdc098-347c-4432-a83d-1345978de9df None None] MFA Rules not processed for user `12d267ccfaad43b58d3fcfe77b295147`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:26.053360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d4fdc098-347c-4432-a83d-1345978de9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:26.082319 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1043/2032] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:26.086289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e6713-014e-4108-b555-05b4747e184c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:26.107390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e6713-014e-4108-b555-05b4747e184c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:26.123697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:26.133081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f881d72-56ca-4e2a-af1c-5df07df51680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:26.135748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f881d72-56ca-4e2a-af1c-5df07df51680 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:26.136294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f881d72-56ca-4e2a-af1c-5df07df51680 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:26.136670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f881d72-56ca-4e2a-af1c-5df07df51680 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:26.137711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f881d72-56ca-4e2a-af1c-5df07df51680 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:49:26.138219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f881d72-56ca-4e2a-af1c-5df07df51680 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:26.142416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f881d72-56ca-4e2a-af1c-5df07df51680 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:26.168965 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 990/2033] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:26.175970 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1044/2034] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:49:26.177270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fdc098-347c-4432-a83d-1345978de9df None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:26.182462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f881d72-56ca-4e2a-af1c-5df07df51680 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:26.193829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f881d72-56ca-4e2a-af1c-5df07df51680 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:26.201159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50f239-dbfb-4d00-bcd5-2b34eb89be86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:26.201781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50f239-dbfb-4d00-bcd5-2b34eb89be86 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:26.202063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50f239-dbfb-4d00-bcd5-2b34eb89be86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:26.202343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50f239-dbfb-4d00-bcd5-2b34eb89be86 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:26.206465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50f239-dbfb-4d00-bcd5-2b34eb89be86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:26.207159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fdc098-347c-4432-a83d-1345978de9df None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:26.208455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50f239-dbfb-4d00-bcd5-2b34eb89be86 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:26.219488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4461410-d284-490b-bae1-a8b7376d4e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:26.220112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4461410-d284-490b-bae1-a8b7376d4e33 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:26.220555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4461410-d284-490b-bae1-a8b7376d4e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:26.220774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4461410-d284-490b-bae1-a8b7376d4e33 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:26.221419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4461410-d284-490b-bae1-a8b7376d4e33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:26.223538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4461410-d284-490b-bae1-a8b7376d4e33 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:26.277543 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1045/2035] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:26 2022] POST /identity/v3/projects => generated 399 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:26.278658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50f239-dbfb-4d00-bcd5-2b34eb89be86 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:26.282962 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 991/2036] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:26 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:26.286604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4461410-d284-490b-bae1-a8b7376d4e33 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:26.307438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:26.430679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50f239-dbfb-4d00-bcd5-2b34eb89be86 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:26.448304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4461410-d284-490b-bae1-a8b7376d4e33 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:26.453124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d54d49-17ff-44eb-b215-720bac5f4410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:26.453762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d54d49-17ff-44eb-b215-720bac5f4410 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:26.454047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d54d49-17ff-44eb-b215-720bac5f4410 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:26.454325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d54d49-17ff-44eb-b215-720bac5f4410 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:26.455181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d54d49-17ff-44eb-b215-720bac5f4410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:26.457026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d54d49-17ff-44eb-b215-720bac5f4410 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:26.462186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c42794-4bba-40fd-a8ae-81bd3bbda33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:26.462955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c42794-4bba-40fd-a8ae-81bd3bbda33e admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:26.463252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c42794-4bba-40fd-a8ae-81bd3bbda33e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:26.463535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c42794-4bba-40fd-a8ae-81bd3bbda33e admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:26.464163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c42794-4bba-40fd-a8ae-81bd3bbda33e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:26.466115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c42794-4bba-40fd-a8ae-81bd3bbda33e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:26.507108 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1046/2037] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:26 2022] POST /identity/v3/users => generated 300 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:26.511619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d54d49-17ff-44eb-b215-720bac5f4410 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:26.532166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d54d49-17ff-44eb-b215-720bac5f4410 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:26.547228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d468e3-3059-4021-8c3f-d716776ad921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:26.547852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d468e3-3059-4021-8c3f-d716776ad921 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:26.548210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d468e3-3059-4021-8c3f-d716776ad921 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:26.548500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d468e3-3059-4021-8c3f-d716776ad921 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:26.549144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d468e3-3059-4021-8c3f-d716776ad921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:26.550566 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 992/2038] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:26 2022] POST /identity/v3/users => generated 300 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:26.552759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d468e3-3059-4021-8c3f-d716776ad921 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:26.557614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c42794-4bba-40fd-a8ae-81bd3bbda33e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:26.579122 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1047/2039] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:26.582689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c42794-4bba-40fd-a8ae-81bd3bbda33e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:26.585982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d468e3-3059-4021-8c3f-d716776ad921 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:26.595441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8220f-5a56-4281-b075-89ac58915f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:26.596741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8220f-5a56-4281-b075-89ac58915f0d admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:26.596741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8220f-5a56-4281-b075-89ac58915f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:26.596741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8220f-5a56-4281-b075-89ac58915f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:26.597922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8220f-5a56-4281-b075-89ac58915f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:26.602092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d468e3-3059-4021-8c3f-d716776ad921 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:26.607626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8220f-5a56-4281-b075-89ac58915f0d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:26.616998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61644d1-32bb-4efa-99e1-988a132c9b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:26.617804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61644d1-32bb-4efa-99e1-988a132c9b68 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:26.618096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61644d1-32bb-4efa-99e1-988a132c9b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:26.618680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61644d1-32bb-4efa-99e1-988a132c9b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:26.619927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61644d1-32bb-4efa-99e1-988a132c9b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:26.621549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61644d1-32bb-4efa-99e1-988a132c9b68 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:26.625016 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 993/2040] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:26.643551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8220f-5a56-4281-b075-89ac58915f0d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:26.655702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8220f-5a56-4281-b075-89ac58915f0d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:26.663558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abbe45-1c2d-4e73-93bf-041528ab0997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:26.663558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abbe45-1c2d-4e73-93bf-041528ab0997 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:26.665890 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1048/2041] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:26.667319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61644d1-32bb-4efa-99e1-988a132c9b68 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:26.667622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abbe45-1c2d-4e73-93bf-041528ab0997 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:26.667983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abbe45-1c2d-4e73-93bf-041528ab0997 admin admin] PATH_INFO: `/v3/projects/36ed5b8bf96d46889380bc1bd386552f/users/212bd8ae4d5f41afb5793106cbf97170/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:26.691086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61644d1-32bb-4efa-99e1-988a132c9b68 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:26.693632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abbe45-1c2d-4e73-93bf-041528ab0997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36ed5b8bf96d46889380bc1bd386552f, user_id=212bd8ae4d5f41afb5793106cbf97170, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:26.696872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abbe45-1c2d-4e73-93bf-041528ab0997 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:26.702940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7c807-9341-49c6-a457-e74ec603a4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:26.703917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7c807-9341-49c6-a457-e74ec603a4da admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:26.704608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7c807-9341-49c6-a457-e74ec603a4da admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:26.705102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7c807-9341-49c6-a457-e74ec603a4da admin admin] PATH_INFO: `/v3/projects/d23d8ef101154d23bda42171941e90b3/users/b2b4c0f4f5d14963b4d64e87f7952152/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:26.733620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7c807-9341-49c6-a457-e74ec603a4da admin admin] RBAC: Authorizing `identity:create_grant(project_id=d23d8ef101154d23bda42171941e90b3, user_id=b2b4c0f4f5d14963b4d64e87f7952152, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:26.743242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7c807-9341-49c6-a457-e74ec603a4da admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:26.745429 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 994/2042] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:26 2022] PUT /identity/v3/projects/36ed5b8bf96d46889380bc1bd386552f/users/212bd8ae4d5f41afb5793106cbf97170/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:26.749559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abbe45-1c2d-4e73-93bf-041528ab0997 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:26.806423 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1049/2043] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:26 2022] PUT /identity/v3/projects/d23d8ef101154d23bda42171941e90b3/users/b2b4c0f4f5d14963b4d64e87f7952152/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:26.810079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd09b9-73f9-41ce-a8fa-143d086faf2d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:26.810374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd09b9-73f9-41ce-a8fa-143d086faf2d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:26.810727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd09b9-73f9-41ce-a8fa-143d086faf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:26.901837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abbe45-1c2d-4e73-93bf-041528ab0997 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:26.904914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-32fd09b9-73f9-41ce-a8fa-143d086faf2d None None] MFA Rules not processed for user `b2b4c0f4f5d14963b4d64e87f7952152`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:26.908007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-32fd09b9-73f9-41ce-a8fa-143d086faf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:26.936649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:27.060548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79959bed-1fc2-4b55-803f-d6eef871c862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:27.060548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79959bed-1fc2-4b55-803f-d6eef871c862 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:27.060548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79959bed-1fc2-4b55-803f-d6eef871c862 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:27.060548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79959bed-1fc2-4b55-803f-d6eef871c862 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:27.061718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79959bed-1fc2-4b55-803f-d6eef871c862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:27.061718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79959bed-1fc2-4b55-803f-d6eef871c862 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:27.078117 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 995/2044] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:27.090705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79959bed-1fc2-4b55-803f-d6eef871c862 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:27.095617 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1050/2045] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:26 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:27.096554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd09b9-73f9-41ce-a8fa-143d086faf2d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:27.116894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd09b9-73f9-41ce-a8fa-143d086faf2d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:27.118227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79959bed-1fc2-4b55-803f-d6eef871c862 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:27.125550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016a481-fb8c-48e5-a6df-a93963e895ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:27.126168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016a481-fb8c-48e5-a6df-a93963e895ea admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:27.126452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016a481-fb8c-48e5-a6df-a93963e895ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:27.126855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016a481-fb8c-48e5-a6df-a93963e895ea admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:27.127310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86367d-19b3-461b-b9f4-aa39deff9bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:27.127791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016a481-fb8c-48e5-a6df-a93963e895ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:27.127999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86367d-19b3-461b-b9f4-aa39deff9bf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:27.128356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86367d-19b3-461b-b9f4-aa39deff9bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:27.128816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86367d-19b3-461b-b9f4-aa39deff9bf7 admin admin] PATH_INFO: `/v3/projects/36ed5b8bf96d46889380bc1bd386552f/users/212bd8ae4d5f41afb5793106cbf97170/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:27.129705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016a481-fb8c-48e5-a6df-a93963e895ea admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:27.130960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae86367d-19b3-461b-b9f4-aa39deff9bf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36ed5b8bf96d46889380bc1bd386552f, user_id=212bd8ae4d5f41afb5793106cbf97170, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:27.136116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae86367d-19b3-461b-b9f4-aa39deff9bf7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:27.153052 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 996/2046] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:27 2022] PUT /identity/v3/projects/36ed5b8bf96d46889380bc1bd386552f/users/212bd8ae4d5f41afb5793106cbf97170/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:27.164164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189b7ba-f991-4da1-aa6a-8dafaa0f04da None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:27.164530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189b7ba-f991-4da1-aa6a-8dafaa0f04da None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:27.164891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189b7ba-f991-4da1-aa6a-8dafaa0f04da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:27.183453 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1051/2047] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:27 2022] POST /identity/v3/projects => generated 397 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:27.184455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016a481-fb8c-48e5-a6df-a93963e895ea admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:27.223573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b189b7ba-f991-4da1-aa6a-8dafaa0f04da None None] MFA Rules not processed for user `212bd8ae4d5f41afb5793106cbf97170`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:27.224961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b189b7ba-f991-4da1-aa6a-8dafaa0f04da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:27.282871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:27.346182 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 997/2048] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:49:27.418269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016a481-fb8c-48e5-a6df-a93963e895ea admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:27.429889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bbcded-4d21-434b-bc65-12fc5c1a2e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:27.433193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbcded-4d21-434b-bc65-12fc5c1a2e53 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:27.433545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbcded-4d21-434b-bc65-12fc5c1a2e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:27.433832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbcded-4d21-434b-bc65-12fc5c1a2e53 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:27.434980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bbcded-4d21-434b-bc65-12fc5c1a2e53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:27.437927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bbcded-4d21-434b-bc65-12fc5c1a2e53 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:27.454897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189b7ba-f991-4da1-aa6a-8dafaa0f04da None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:27.457581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b189b7ba-f991-4da1-aa6a-8dafaa0f04da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:27.480819 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1052/2049] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:27 2022] POST /identity/v3/users => generated 305 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:27.482197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bbcded-4d21-434b-bc65-12fc5c1a2e53 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:27.498969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bbcded-4d21-434b-bc65-12fc5c1a2e53 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:27.509903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d1f49-0733-4e18-b083-a4d6794c3a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:27.511090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d1f49-0733-4e18-b083-a4d6794c3a32 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:27.511384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d1f49-0733-4e18-b083-a4d6794c3a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:27.512141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d1f49-0733-4e18-b083-a4d6794c3a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:27.514150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d1f49-0733-4e18-b083-a4d6794c3a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:27.521597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d1f49-0733-4e18-b083-a4d6794c3a32 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:27.538142 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1053/2050] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:27.540449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d1f49-0733-4e18-b083-a4d6794c3a32 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:27.557181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d1f49-0733-4e18-b083-a4d6794c3a32 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:27.563850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938f8a4-1cb1-480d-8685-7fc2b67ec7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:27.564520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938f8a4-1cb1-480d-8685-7fc2b67ec7ed admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:27.564849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938f8a4-1cb1-480d-8685-7fc2b67ec7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:27.565192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938f8a4-1cb1-480d-8685-7fc2b67ec7ed admin admin] PATH_INFO: `/v3/projects/a68a43f9a1a44afdbe35ce9ce10e2279/users/fa8ed9d260d64c6689104e50497dcfc9/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:27.587559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938f8a4-1cb1-480d-8685-7fc2b67ec7ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68a43f9a1a44afdbe35ce9ce10e2279, user_id=fa8ed9d260d64c6689104e50497dcfc9, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:27.591397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938f8a4-1cb1-480d-8685-7fc2b67ec7ed admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:27.605390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189b7ba-f991-4da1-aa6a-8dafaa0f04da None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:27.611941 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1054/2051] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:27 2022] PUT /identity/v3/projects/a68a43f9a1a44afdbe35ce9ce10e2279/users/fa8ed9d260d64c6689104e50497dcfc9/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:27.616576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938f8a4-1cb1-480d-8685-7fc2b67ec7ed admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:27.667472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:27.722191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938f8a4-1cb1-480d-8685-7fc2b67ec7ed admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:27.740096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c84361-b79c-4648-8a90-ec2981967fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:27.741094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c84361-b79c-4648-8a90-ec2981967fcf admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:27.741905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c84361-b79c-4648-8a90-ec2981967fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:27.742699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c84361-b79c-4648-8a90-ec2981967fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:27.743645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c84361-b79c-4648-8a90-ec2981967fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:27.746606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c84361-b79c-4648-8a90-ec2981967fcf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:27.754849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6c95c-ff06-4618-be3e-0143e45cd853 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:27.755484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6c95c-ff06-4618-be3e-0143e45cd853 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:27.756652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6c95c-ff06-4618-be3e-0143e45cd853 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:27.757001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6c95c-ff06-4618-be3e-0143e45cd853 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:27.759419 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1055/2052] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:27.766331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c84361-b79c-4648-8a90-ec2981967fcf admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:27.781228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c84361-b79c-4648-8a90-ec2981967fcf admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:27.791693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5643b93-af84-43ce-a9bb-310423c4cf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:27.792842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5643b93-af84-43ce-a9bb-310423c4cf3a admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:27.792842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5643b93-af84-43ce-a9bb-310423c4cf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:27.793740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5643b93-af84-43ce-a9bb-310423c4cf3a admin admin] PATH_INFO: `/v3/projects/a68a43f9a1a44afdbe35ce9ce10e2279/users/fa8ed9d260d64c6689104e50497dcfc9/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:27.796330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5643b93-af84-43ce-a9bb-310423c4cf3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68a43f9a1a44afdbe35ce9ce10e2279, user_id=fa8ed9d260d64c6689104e50497dcfc9, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:27.800252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5643b93-af84-43ce-a9bb-310423c4cf3a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:27.828019 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1056/2053] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:27 2022] PUT /identity/v3/projects/a68a43f9a1a44afdbe35ce9ce10e2279/users/fa8ed9d260d64c6689104e50497dcfc9/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:27.830703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbb776-ca3a-4ddb-90e4-3a3f137b2bf9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:27.831012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbb776-ca3a-4ddb-90e4-3a3f137b2bf9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:27.831297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbb776-ca3a-4ddb-90e4-3a3f137b2bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:27.863485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6c95c-ff06-4618-be3e-0143e45cd853 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:27.865884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6c95c-ff06-4618-be3e-0143e45cd853 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:27.877839 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 998/2054] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:27.893803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7fcbb776-ca3a-4ddb-90e4-3a3f137b2bf9 None None] MFA Rules not processed for user `fa8ed9d260d64c6689104e50497dcfc9`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:27.895226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7fcbb776-ca3a-4ddb-90e4-3a3f137b2bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:27.945918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6c95c-ff06-4618-be3e-0143e45cd853 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:27.958476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6c95c-ff06-4618-be3e-0143e45cd853 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:27.994643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:28.029129 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1057/2055] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:27 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:28.059037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511050a3-2bb5-4544-8c12-1768154bbdbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:28.059954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511050a3-2bb5-4544-8c12-1768154bbdbf service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:28.063242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511050a3-2bb5-4544-8c12-1768154bbdbf service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:28.064874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511050a3-2bb5-4544-8c12-1768154bbdbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:28.136507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:28.141404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511050a3-2bb5-4544-8c12-1768154bbdbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:28.143048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511050a3-2bb5-4544-8c12-1768154bbdbf service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:28.153799 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 999/2056] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:27 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:49:28.179784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbb776-ca3a-4ddb-90e4-3a3f137b2bf9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:28.186812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511050a3-2bb5-4544-8c12-1768154bbdbf service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:28.190449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-511050a3-2bb5-4544-8c12-1768154bbdbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:28.225601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511050a3-2bb5-4544-8c12-1768154bbdbf service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:28.235254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de6890-79ca-43bb-8236-25866a91faed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:28.236317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de6890-79ca-43bb-8236-25866a91faed service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:28.236611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de6890-79ca-43bb-8236-25866a91faed service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:28.236890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de6890-79ca-43bb-8236-25866a91faed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:28.265575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbb776-ca3a-4ddb-90e4-3a3f137b2bf9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:28.270786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ff36d-d6a2-47e4-8bde-7c1fe3125fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:28.271846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ff36d-d6a2-47e4-8bde-7c1fe3125fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:28.272271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ff36d-d6a2-47e4-8bde-7c1fe3125fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:28.272686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ff36d-d6a2-47e4-8bde-7c1fe3125fd1 admin admin] PATH_INFO: `/v3/users/12d267ccfaad43b58d3fcfe77b295147` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:28.273856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ff36d-d6a2-47e4-8bde-7c1fe3125fd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12d267ccfaad43b58d3fcfe77b295147)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:28.275976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ff36d-d6a2-47e4-8bde-7c1fe3125fd1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:28.311175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de6890-79ca-43bb-8236-25866a91faed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:28.313329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de6890-79ca-43bb-8236-25866a91faed service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:28.324141 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1000/2057] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:28 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:28.331488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a84ff36d-d6a2-47e4-8bde-7c1fe3125fd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12d267ccfaad43b58d3fcfe77b295147'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:28.339368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a84ff36d-d6a2-47e4-8bde-7c1fe3125fd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12d267ccfaad43b58d3fcfe77b295147'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:28.356389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a84ff36d-d6a2-47e4-8bde-7c1fe3125fd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12d267ccfaad43b58d3fcfe77b295147'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:28.357248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a84ff36d-d6a2-47e4-8bde-7c1fe3125fd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12d267ccfaad43b58d3fcfe77b295147'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:28.365577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a84ff36d-d6a2-47e4-8bde-7c1fe3125fd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12d267ccfaad43b58d3fcfe77b295147'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:28.373500 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1058/2058] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:28 2022] DELETE /identity/v3/users/12d267ccfaad43b58d3fcfe77b295147 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:28.377951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16e87c-e238-49a6-821b-cebd15122304 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:28.378244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16e87c-e238-49a6-821b-cebd15122304 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:28.379059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16e87c-e238-49a6-821b-cebd15122304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:28.427698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0d16e87c-e238-49a6-821b-cebd15122304 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:28.429432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0d16e87c-e238-49a6-821b-cebd15122304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:28.530928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ff36d-d6a2-47e4-8bde-7c1fe3125fd1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:28.532626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a84ff36d-d6a2-47e4-8bde-7c1fe3125fd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:28.564981 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1001/2059] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:28.576294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16e87c-e238-49a6-821b-cebd15122304 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:28.580656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0d16e87c-e238-49a6-821b-cebd15122304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:28.661408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16e87c-e238-49a6-821b-cebd15122304 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:28.661712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ff36d-d6a2-47e4-8bde-7c1fe3125fd1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:28.671843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07964e43-8baf-4e29-bf55-63723124efb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:28.672967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d03b2a-dc45-4dfb-8d1f-11c473b7de62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:28.673999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d03b2a-dc45-4dfb-8d1f-11c473b7de62 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:28.673999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d03b2a-dc45-4dfb-8d1f-11c473b7de62 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:28.674175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d03b2a-dc45-4dfb-8d1f-11c473b7de62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:28.674708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07964e43-8baf-4e29-bf55-63723124efb4 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:28.674997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07964e43-8baf-4e29-bf55-63723124efb4 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:28.675293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07964e43-8baf-4e29-bf55-63723124efb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:28.677050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-07964e43-8baf-4e29-bf55-63723124efb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:28.682852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:28.689725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d03b2a-dc45-4dfb-8d1f-11c473b7de62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:28.691781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d03b2a-dc45-4dfb-8d1f-11c473b7de62 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:28.708680 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1002/2060] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:28.724058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cb868-a0fd-4197-855c-56280d13930b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:28.724357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cb868-a0fd-4197-855c-56280d13930b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:28.724638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cb868-a0fd-4197-855c-56280d13930b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:28.751721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-676cb868-a0fd-4197-855c-56280d13930b None None] MFA Rules not processed for user `7ef35c15f9d541ac90239c3713f9f5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:28.753794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-676cb868-a0fd-4197-855c-56280d13930b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:28.767647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07964e43-8baf-4e29-bf55-63723124efb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:28.769536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07964e43-8baf-4e29-bf55-63723124efb4 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:28.784465 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1059/2061] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:28 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:28.838412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:28.902987 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1003/2062] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:29.129054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07964e43-8baf-4e29-bf55-63723124efb4 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:29.129784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-07964e43-8baf-4e29-bf55-63723124efb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:29.151834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07964e43-8baf-4e29-bf55-63723124efb4 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:29.157140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8273b0a-911e-46e2-b497-6736f7eb8eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:29.157784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8273b0a-911e-46e2-b497-6736f7eb8eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:29.158060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8273b0a-911e-46e2-b497-6736f7eb8eef admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:29.158339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8273b0a-911e-46e2-b497-6736f7eb8eef admin admin] PATH_INFO: `/v3/projects/158034e03ee1475fa3ef2fcd49191642` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:29.159763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8273b0a-911e-46e2-b497-6736f7eb8eef admin admin] RBAC: Authorizing `identity:delete_project(project_id=158034e03ee1475fa3ef2fcd49191642)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:29.161677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8273b0a-911e-46e2-b497-6736f7eb8eef admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:29.221291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8273b0a-911e-46e2-b497-6736f7eb8eef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '158034e03ee1475fa3ef2fcd49191642', 'request_id': 'req-d8273b0a-911e-46e2-b497-6736f7eb8eef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:29.226041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8273b0a-911e-46e2-b497-6736f7eb8eef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '158034e03ee1475fa3ef2fcd49191642', 'request_id': 'req-d8273b0a-911e-46e2-b497-6736f7eb8eef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:29.232474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8273b0a-911e-46e2-b497-6736f7eb8eef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '158034e03ee1475fa3ef2fcd49191642', 'request_id': 'req-d8273b0a-911e-46e2-b497-6736f7eb8eef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:29.248010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8273b0a-911e-46e2-b497-6736f7eb8eef admin admin] The token cache is being invalidate because project 158034e03ee1475fa3ef2fcd49191642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:29.248307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8273b0a-911e-46e2-b497-6736f7eb8eef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:29.249026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8273b0a-911e-46e2-b497-6736f7eb8eef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:29.251375 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1060/2063] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:29 2022] DELETE /identity/v3/projects/158034e03ee1475fa3ef2fcd49191642 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:29.255304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cb868-a0fd-4197-855c-56280d13930b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:29.269409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cb868-a0fd-4197-855c-56280d13930b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:29.345614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8273b0a-911e-46e2-b497-6736f7eb8eef admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:29.354836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d8273b0a-911e-46e2-b497-6736f7eb8eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:29.366966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839b12a-e9a8-444f-98df-b8ddfec6a04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:29.367635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839b12a-e9a8-444f-98df-b8ddfec6a04a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:29.367921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839b12a-e9a8-444f-98df-b8ddfec6a04a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:29.368199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839b12a-e9a8-444f-98df-b8ddfec6a04a admin admin] PATH_INFO: `/v3/users/212bd8ae4d5f41afb5793106cbf97170` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:29.372320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8273b0a-911e-46e2-b497-6736f7eb8eef admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:29.374262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839b12a-e9a8-444f-98df-b8ddfec6a04a admin admin] RBAC: Authorizing `identity:delete_user(user_id=212bd8ae4d5f41afb5793106cbf97170)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:29.378635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cb211-4029-4d1f-90a0-348719068f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:29.379763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cb211-4029-4d1f-90a0-348719068f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:29.380148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839b12a-e9a8-444f-98df-b8ddfec6a04a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:29.392804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cb211-4029-4d1f-90a0-348719068f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:29.394001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cb211-4029-4d1f-90a0-348719068f52 admin admin] PATH_INFO: `/v3/users/a8c7e3fb07754e46b7ea90a7f4508404` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:29.399602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cb211-4029-4d1f-90a0-348719068f52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8c7e3fb07754e46b7ea90a7f4508404)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:29.429328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cb211-4029-4d1f-90a0-348719068f52 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:29.496925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c839b12a-e9a8-444f-98df-b8ddfec6a04a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '212bd8ae4d5f41afb5793106cbf97170'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:29.497939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c839b12a-e9a8-444f-98df-b8ddfec6a04a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '212bd8ae4d5f41afb5793106cbf97170'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:29.502293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c839b12a-e9a8-444f-98df-b8ddfec6a04a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '212bd8ae4d5f41afb5793106cbf97170'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:29.515527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c839b12a-e9a8-444f-98df-b8ddfec6a04a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '212bd8ae4d5f41afb5793106cbf97170'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:29.528485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ff3cb211-4029-4d1f-90a0-348719068f52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8c7e3fb07754e46b7ea90a7f4508404'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:29.546607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c839b12a-e9a8-444f-98df-b8ddfec6a04a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '212bd8ae4d5f41afb5793106cbf97170'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:29.547691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ff3cb211-4029-4d1f-90a0-348719068f52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8c7e3fb07754e46b7ea90a7f4508404'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:29.561463 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1004/2064] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:29 2022] DELETE /identity/v3/users/212bd8ae4d5f41afb5793106cbf97170 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:49:29.565282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839b12a-e9a8-444f-98df-b8ddfec6a04a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:29.568035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c839b12a-e9a8-444f-98df-b8ddfec6a04a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:29.619155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ff3cb211-4029-4d1f-90a0-348719068f52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8c7e3fb07754e46b7ea90a7f4508404'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:29.628982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ff3cb211-4029-4d1f-90a0-348719068f52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8c7e3fb07754e46b7ea90a7f4508404'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:29.645007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:29.655340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ff3cb211-4029-4d1f-90a0-348719068f52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8c7e3fb07754e46b7ea90a7f4508404'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:29.662316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839b12a-e9a8-444f-98df-b8ddfec6a04a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:29.675793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099bc00-38a5-4f97-834e-92f010ab2f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:29.676420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099bc00-38a5-4f97-834e-92f010ab2f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:29.677021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099bc00-38a5-4f97-834e-92f010ab2f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:29.677589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099bc00-38a5-4f97-834e-92f010ab2f7e admin admin] PATH_INFO: `/v3/users/b2b4c0f4f5d14963b4d64e87f7952152` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:29.681070 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1061/2065] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:29 2022] DELETE /identity/v3/users/a8c7e3fb07754e46b7ea90a7f4508404 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:29.681250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099bc00-38a5-4f97-834e-92f010ab2f7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2b4c0f4f5d14963b4d64e87f7952152)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:29.682945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68744edf-da81-4d7a-b7c3-156bc3248e6c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:29.683238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68744edf-da81-4d7a-b7c3-156bc3248e6c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:29.683449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099bc00-38a5-4f97-834e-92f010ab2f7e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:29.683554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68744edf-da81-4d7a-b7c3-156bc3248e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:29.685698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:29.749010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-68744edf-da81-4d7a-b7c3-156bc3248e6c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:29.750306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-68744edf-da81-4d7a-b7c3-156bc3248e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:29.777704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c099bc00-38a5-4f97-834e-92f010ab2f7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2b4c0f4f5d14963b4d64e87f7952152'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:29.779173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c099bc00-38a5-4f97-834e-92f010ab2f7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2b4c0f4f5d14963b4d64e87f7952152'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:29.780555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c099bc00-38a5-4f97-834e-92f010ab2f7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2b4c0f4f5d14963b4d64e87f7952152'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:29.792999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c099bc00-38a5-4f97-834e-92f010ab2f7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2b4c0f4f5d14963b4d64e87f7952152'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:29.818134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c099bc00-38a5-4f97-834e-92f010ab2f7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2b4c0f4f5d14963b4d64e87f7952152'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:29.838781 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1005/2066] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:29 2022] DELETE /identity/v3/users/b2b4c0f4f5d14963b4d64e87f7952152 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:29.860518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88cfee-ee8c-4433-a016-70f7f72e880c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:29.860939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88cfee-ee8c-4433-a016-70f7f72e880c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:29.861251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88cfee-ee8c-4433-a016-70f7f72e880c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:29.928320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3e88cfee-ee8c-4433-a016-70f7f72e880c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:29.928582 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1062/2067] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:29.929563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3e88cfee-ee8c-4433-a016-70f7f72e880c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:29.937081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68744edf-da81-4d7a-b7c3-156bc3248e6c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:29.938839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-68744edf-da81-4d7a-b7c3-156bc3248e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:30.065153 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1006/2068] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:30.086194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88cfee-ee8c-4433-a016-70f7f72e880c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:30.092113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68744edf-da81-4d7a-b7c3-156bc3248e6c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:30.180703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fae5d4-5320-4f2b-937c-1e8b8c1acc9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:30.181865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fae5d4-5320-4f2b-937c-1e8b8c1acc9b service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:30.182157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fae5d4-5320-4f2b-937c-1e8b8c1acc9b service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:30.182435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fae5d4-5320-4f2b-937c-1e8b8c1acc9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:30.204897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fae5d4-5320-4f2b-937c-1e8b8c1acc9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:30.207109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fae5d4-5320-4f2b-937c-1e8b8c1acc9b service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:30.218150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88cfee-ee8c-4433-a016-70f7f72e880c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:30.221245 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1063/2069] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:49:30.222279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fae5d4-5320-4f2b-937c-1e8b8c1acc9b service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:30.223845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-73fae5d4-5320-4f2b-937c-1e8b8c1acc9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:30.237797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fae5d4-5320-4f2b-937c-1e8b8c1acc9b service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:30.241055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db76ff7-af73-43f6-b09b-5da758055923 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:30.242765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db76ff7-af73-43f6-b09b-5da758055923 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:30.245214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d3719-ac86-430a-b9f2-d083d86840c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:30.245378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db76ff7-af73-43f6-b09b-5da758055923 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:30.245688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db76ff7-af73-43f6-b09b-5da758055923 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:30.245841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d3719-ac86-430a-b9f2-d083d86840c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:30.246178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d3719-ac86-430a-b9f2-d083d86840c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:30.246460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d3719-ac86-430a-b9f2-d083d86840c6 admin admin] PATH_INFO: `/v3/projects/36ed5b8bf96d46889380bc1bd386552f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:30.247656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d3719-ac86-430a-b9f2-d083d86840c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36ed5b8bf96d46889380bc1bd386552f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:30.249882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d3719-ac86-430a-b9f2-d083d86840c6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:30.280376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db76ff7-af73-43f6-b09b-5da758055923 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:30.283514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db76ff7-af73-43f6-b09b-5da758055923 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:30.297353 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1007/2070] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:30.319112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5b7d3719-ac86-430a-b9f2-d083d86840c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36ed5b8bf96d46889380bc1bd386552f', 'request_id': 'req-5b7d3719-ac86-430a-b9f2-d083d86840c6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:30.320724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:30.329899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5b7d3719-ac86-430a-b9f2-d083d86840c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36ed5b8bf96d46889380bc1bd386552f', 'request_id': 'req-5b7d3719-ac86-430a-b9f2-d083d86840c6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:30.337210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5b7d3719-ac86-430a-b9f2-d083d86840c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36ed5b8bf96d46889380bc1bd386552f', 'request_id': 'req-5b7d3719-ac86-430a-b9f2-d083d86840c6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:30.352743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5b7d3719-ac86-430a-b9f2-d083d86840c6 admin admin] The token cache is being invalidate because project 36ed5b8bf96d46889380bc1bd386552f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:30.352743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5b7d3719-ac86-430a-b9f2-d083d86840c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:30.353211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5b7d3719-ac86-430a-b9f2-d083d86840c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:30.361405 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1064/2071] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:30 2022] DELETE /identity/v3/projects/36ed5b8bf96d46889380bc1bd386552f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:30.363229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199088a-d63d-4097-8c8e-1b6007b3662f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:30.363519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199088a-d63d-4097-8c8e-1b6007b3662f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:30.363807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199088a-d63d-4097-8c8e-1b6007b3662f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:30.399187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1199088a-d63d-4097-8c8e-1b6007b3662f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:30.402165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1199088a-d63d-4097-8c8e-1b6007b3662f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:30.437371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:30.522306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d3719-ac86-430a-b9f2-d083d86840c6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:30.526431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5b7d3719-ac86-430a-b9f2-d083d86840c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:30.560239 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1008/2072] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:30.568452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc37ac-16de-4566-b77b-1886e264a20b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:30.571445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc37ac-16de-4566-b77b-1886e264a20b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:30.571445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc37ac-16de-4566-b77b-1886e264a20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:30.597408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f2fc37ac-16de-4566-b77b-1886e264a20b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:30.598819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f2fc37ac-16de-4566-b77b-1886e264a20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:30.610597 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1009/2073] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:30.613704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc37ac-16de-4566-b77b-1886e264a20b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:30.627527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc37ac-16de-4566-b77b-1886e264a20b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:30.645010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed53be-76d4-4627-89e7-9da262a8be50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:30.647112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed53be-76d4-4627-89e7-9da262a8be50 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:30.647559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed53be-76d4-4627-89e7-9da262a8be50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:30.647774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed53be-76d4-4627-89e7-9da262a8be50 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:30.648433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed53be-76d4-4627-89e7-9da262a8be50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:49:30.648735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed53be-76d4-4627-89e7-9da262a8be50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:30.651206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed53be-76d4-4627-89e7-9da262a8be50 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:30.658323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d3719-ac86-430a-b9f2-d083d86840c6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:30.665254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080c2eb-1517-47f4-9b5e-5dbec531d3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:30.665694 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1010/2074] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:49:30.669889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed53be-76d4-4627-89e7-9da262a8be50 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:30.670883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080c2eb-1517-47f4-9b5e-5dbec531d3ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:30.671179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080c2eb-1517-47f4-9b5e-5dbec531d3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:30.671461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080c2eb-1517-47f4-9b5e-5dbec531d3ac admin admin] PATH_INFO: `/v3/users/07638141e5f34ee783ebfe4f577be214` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:30.673310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080c2eb-1517-47f4-9b5e-5dbec531d3ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=07638141e5f34ee783ebfe4f577be214)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:30.675053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080c2eb-1517-47f4-9b5e-5dbec531d3ac admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:30.684821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed53be-76d4-4627-89e7-9da262a8be50 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:30.690948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026028d3-a9ff-4d5c-8b52-9f2a45b3aef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:30.692228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026028d3-a9ff-4d5c-8b52-9f2a45b3aef1 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:30.692952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026028d3-a9ff-4d5c-8b52-9f2a45b3aef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:30.693519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026028d3-a9ff-4d5c-8b52-9f2a45b3aef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:30.694370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026028d3-a9ff-4d5c-8b52-9f2a45b3aef1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:30.697677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026028d3-a9ff-4d5c-8b52-9f2a45b3aef1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:30.740317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e080c2eb-1517-47f4-9b5e-5dbec531d3ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07638141e5f34ee783ebfe4f577be214'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:30.744948 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1011/2075] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:30 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:49:30.744948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026028d3-a9ff-4d5c-8b52-9f2a45b3aef1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:30.749035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-026028d3-a9ff-4d5c-8b52-9f2a45b3aef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:30.751284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e080c2eb-1517-47f4-9b5e-5dbec531d3ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07638141e5f34ee783ebfe4f577be214'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:30.781267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e080c2eb-1517-47f4-9b5e-5dbec531d3ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07638141e5f34ee783ebfe4f577be214'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:30.782580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e080c2eb-1517-47f4-9b5e-5dbec531d3ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07638141e5f34ee783ebfe4f577be214'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:30.797492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e080c2eb-1517-47f4-9b5e-5dbec531d3ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07638141e5f34ee783ebfe4f577be214'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:30.809444 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1065/2076] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:30 2022] DELETE /identity/v3/users/07638141e5f34ee783ebfe4f577be214 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:30.810013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080c2eb-1517-47f4-9b5e-5dbec531d3ac admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:30.813150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e080c2eb-1517-47f4-9b5e-5dbec531d3ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:30.876113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026028d3-a9ff-4d5c-8b52-9f2a45b3aef1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:30.900752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:30.908407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080c2eb-1517-47f4-9b5e-5dbec531d3ac admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:30.916912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6faf75-96d6-4a7f-bbfc-dc6dc96ff0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:30.917438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6faf75-96d6-4a7f-bbfc-dc6dc96ff0b2 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:30.918056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6faf75-96d6-4a7f-bbfc-dc6dc96ff0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:30.918850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6faf75-96d6-4a7f-bbfc-dc6dc96ff0b2 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:30.919774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6faf75-96d6-4a7f-bbfc-dc6dc96ff0b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:30.924870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6faf75-96d6-4a7f-bbfc-dc6dc96ff0b2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:30.955942 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1066/2077] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:30 2022] POST /identity/v3/users => generated 288 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:30.958138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7f2fc-d865-4f21-8db5-32efc5fd1d73 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:30.958436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7f2fc-d865-4f21-8db5-32efc5fd1d73 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:30.958944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7f2fc-d865-4f21-8db5-32efc5fd1d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:30.994876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1ee7f2fc-d865-4f21-8db5-32efc5fd1d73 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:30.996418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1ee7f2fc-d865-4f21-8db5-32efc5fd1d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:31.010821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991dce6-363c-46fc-95bb-730af84ab367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:31.011394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991dce6-363c-46fc-95bb-730af84ab367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:31.011696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991dce6-363c-46fc-95bb-730af84ab367 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:31.011975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991dce6-363c-46fc-95bb-730af84ab367 admin admin] PATH_INFO: `/v3/projects/c9b87cefcd1d49f48a7f3167f14af7b4` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:31.016180 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1067/2078] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:31.016319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991dce6-363c-46fc-95bb-730af84ab367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9b87cefcd1d49f48a7f3167f14af7b4)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:31.016398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7f2fc-d865-4f21-8db5-32efc5fd1d73 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:31.016470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991dce6-363c-46fc-95bb-730af84ab367 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:31.016540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1ee7f2fc-d865-4f21-8db5-32efc5fd1d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:31.041691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7f2fc-d865-4f21-8db5-32efc5fd1d73 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:31.055263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009638d-c865-4648-a515-52e253e89483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:31.056051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009638d-c865-4648-a515-52e253e89483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:31.056192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009638d-c865-4648-a515-52e253e89483 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:31.057144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009638d-c865-4648-a515-52e253e89483 admin admin] PATH_INFO: `/v3/projects/d23d8ef101154d23bda42171941e90b3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:31.059307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009638d-c865-4648-a515-52e253e89483 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d23d8ef101154d23bda42171941e90b3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:31.061888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009638d-c865-4648-a515-52e253e89483 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:31.074031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:31.094985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9991dce6-363c-46fc-95bb-730af84ab367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9b87cefcd1d49f48a7f3167f14af7b4', 'request_id': 'req-9991dce6-363c-46fc-95bb-730af84ab367'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:31.106301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9991dce6-363c-46fc-95bb-730af84ab367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9b87cefcd1d49f48a7f3167f14af7b4', 'request_id': 'req-9991dce6-363c-46fc-95bb-730af84ab367'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:31.114551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9991dce6-363c-46fc-95bb-730af84ab367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9b87cefcd1d49f48a7f3167f14af7b4', 'request_id': 'req-9991dce6-363c-46fc-95bb-730af84ab367'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:31.126244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f009638d-c865-4648-a515-52e253e89483 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd23d8ef101154d23bda42171941e90b3', 'request_id': 'req-f009638d-c865-4648-a515-52e253e89483'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:31.130448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9991dce6-363c-46fc-95bb-730af84ab367 admin admin] The token cache is being invalidate because project c9b87cefcd1d49f48a7f3167f14af7b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:31.130856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9991dce6-363c-46fc-95bb-730af84ab367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:31.131673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9991dce6-363c-46fc-95bb-730af84ab367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:31.134308 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1012/2079] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:30 2022] DELETE /identity/v3/projects/c9b87cefcd1d49f48a7f3167f14af7b4 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:31.136333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991dce6-363c-46fc-95bb-730af84ab367 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:31.138461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9991dce6-363c-46fc-95bb-730af84ab367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:31.149133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f009638d-c865-4648-a515-52e253e89483 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd23d8ef101154d23bda42171941e90b3', 'request_id': 'req-f009638d-c865-4648-a515-52e253e89483'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:31.157598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f009638d-c865-4648-a515-52e253e89483 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd23d8ef101154d23bda42171941e90b3', 'request_id': 'req-f009638d-c865-4648-a515-52e253e89483'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:31.173650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f009638d-c865-4648-a515-52e253e89483 admin admin] The token cache is being invalidate because project d23d8ef101154d23bda42171941e90b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:31.173968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f009638d-c865-4648-a515-52e253e89483 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:31.175135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f009638d-c865-4648-a515-52e253e89483 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:31.178540 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1068/2080] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:31 2022] DELETE /identity/v3/projects/d23d8ef101154d23bda42171941e90b3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:31.178685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009638d-c865-4648-a515-52e253e89483 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:31.179957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f009638d-c865-4648-a515-52e253e89483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:31.223925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991dce6-363c-46fc-95bb-730af84ab367 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:31.250888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009638d-c865-4648-a515-52e253e89483 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:31.255292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d3bac-f428-4c07-a0cf-460d2e30efb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:31.256127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d3bac-f428-4c07-a0cf-460d2e30efb7 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:31.256556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d3bac-f428-4c07-a0cf-460d2e30efb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:31.257220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d3bac-f428-4c07-a0cf-460d2e30efb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:31.259519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-de9d3bac-f428-4c07-a0cf-460d2e30efb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:31.309103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d0f71-42cf-4c20-b9f6-0ba7e4e7d3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:31.310140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d0f71-42cf-4c20-b9f6-0ba7e4e7d3e6 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:31.310432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d0f71-42cf-4c20-b9f6-0ba7e4e7d3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:31.310741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d0f71-42cf-4c20-b9f6-0ba7e4e7d3e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:31.311752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d0f71-42cf-4c20-b9f6-0ba7e4e7d3e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:31.313483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d0f71-42cf-4c20-b9f6-0ba7e4e7d3e6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:31.325288 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1013/2081] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:31.325953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d0f71-42cf-4c20-b9f6-0ba7e4e7d3e6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:31.329567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9f6d0f71-42cf-4c20-b9f6-0ba7e4e7d3e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:31.340789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d3bac-f428-4c07-a0cf-460d2e30efb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:31.342380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d3bac-f428-4c07-a0cf-460d2e30efb7 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:31.343442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d0f71-42cf-4c20-b9f6-0ba7e4e7d3e6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:31.348636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5f87e-2171-4480-8cae-33e86f9c0bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:31.349288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5f87e-2171-4480-8cae-33e86f9c0bdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:31.349574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5f87e-2171-4480-8cae-33e86f9c0bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:31.350029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5f87e-2171-4480-8cae-33e86f9c0bdd admin admin] PATH_INFO: `/v3/users/efe2a89621104cc587add706d8de1586` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:31.351267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5f87e-2171-4480-8cae-33e86f9c0bdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=efe2a89621104cc587add706d8de1586)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:31.353766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5f87e-2171-4480-8cae-33e86f9c0bdd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:31.354457 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1069/2082] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:31.355270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d3bac-f428-4c07-a0cf-460d2e30efb7 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:31.356707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-de9d3bac-f428-4c07-a0cf-460d2e30efb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:31.372471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d3bac-f428-4c07-a0cf-460d2e30efb7 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:31.378991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4da11-b15b-44fe-b25b-81bcd1bd8fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:31.379644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4da11-b15b-44fe-b25b-81bcd1bd8fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:31.379932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4da11-b15b-44fe-b25b-81bcd1bd8fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:31.380212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4da11-b15b-44fe-b25b-81bcd1bd8fc3 admin admin] PATH_INFO: `/v3/users/fa8ed9d260d64c6689104e50497dcfc9` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:31.381709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4da11-b15b-44fe-b25b-81bcd1bd8fc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa8ed9d260d64c6689104e50497dcfc9)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:31.383565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4da11-b15b-44fe-b25b-81bcd1bd8fc3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:31.392109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:31.437758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9de5f87e-2171-4480-8cae-33e86f9c0bdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efe2a89621104cc587add706d8de1586'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:31.438582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9de5f87e-2171-4480-8cae-33e86f9c0bdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efe2a89621104cc587add706d8de1586'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:31.439214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9de5f87e-2171-4480-8cae-33e86f9c0bdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efe2a89621104cc587add706d8de1586'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:31.449887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9de5f87e-2171-4480-8cae-33e86f9c0bdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efe2a89621104cc587add706d8de1586'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:31.452372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:31.467769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9de5f87e-2171-4480-8cae-33e86f9c0bdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efe2a89621104cc587add706d8de1586'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:31.470039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ade4da11-b15b-44fe-b25b-81bcd1bd8fc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa8ed9d260d64c6689104e50497dcfc9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:31.481239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ade4da11-b15b-44fe-b25b-81bcd1bd8fc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa8ed9d260d64c6689104e50497dcfc9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:31.488032 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1014/2083] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:31 2022] DELETE /identity/v3/users/efe2a89621104cc587add706d8de1586 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:31.488032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5f87e-2171-4480-8cae-33e86f9c0bdd admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:31.488032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9de5f87e-2171-4480-8cae-33e86f9c0bdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:31.502478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ade4da11-b15b-44fe-b25b-81bcd1bd8fc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa8ed9d260d64c6689104e50497dcfc9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:31.503898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ade4da11-b15b-44fe-b25b-81bcd1bd8fc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa8ed9d260d64c6689104e50497dcfc9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:31.518634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ade4da11-b15b-44fe-b25b-81bcd1bd8fc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa8ed9d260d64c6689104e50497dcfc9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:31.529835 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1070/2084] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:31 2022] DELETE /identity/v3/users/fa8ed9d260d64c6689104e50497dcfc9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:31.531510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4da11-b15b-44fe-b25b-81bcd1bd8fc3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:31.533032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ade4da11-b15b-44fe-b25b-81bcd1bd8fc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:31.586290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5f87e-2171-4480-8cae-33e86f9c0bdd admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:31.626167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4da11-b15b-44fe-b25b-81bcd1bd8fc3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:31.631550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a305f-db43-42ec-9a6b-e105932f95d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:31.632166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a305f-db43-42ec-9a6b-e105932f95d2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:31.632517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a305f-db43-42ec-9a6b-e105932f95d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:31.634962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a305f-db43-42ec-9a6b-e105932f95d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:31.635436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a305f-db43-42ec-9a6b-e105932f95d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:31.640040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a305f-db43-42ec-9a6b-e105932f95d2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:31.660131 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1071/2085] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:31.664145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a305f-db43-42ec-9a6b-e105932f95d2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:31.695033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a305f-db43-42ec-9a6b-e105932f95d2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:31.706753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69867439-d21a-4df8-a5c6-66f2d745b83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:31.707563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69867439-d21a-4df8-a5c6-66f2d745b83e admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:31.707979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69867439-d21a-4df8-a5c6-66f2d745b83e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:31.708487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69867439-d21a-4df8-a5c6-66f2d745b83e admin admin] PATH_INFO: `/v3/projects/70acee88008f4d83af21de726cafe09a/users/e34abaa1d265489fa35c25a3eae6f052/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:31.725597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce123a9a-2538-4954-bf1a-0da0eedd812a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:31.727448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce123a9a-2538-4954-bf1a-0da0eedd812a service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:31.727776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce123a9a-2538-4954-bf1a-0da0eedd812a service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:31.728063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce123a9a-2538-4954-bf1a-0da0eedd812a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:31.731243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ce123a9a-2538-4954-bf1a-0da0eedd812a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:31.735914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69867439-d21a-4df8-a5c6-66f2d745b83e admin admin] RBAC: Authorizing `identity:create_grant(project_id=70acee88008f4d83af21de726cafe09a, user_id=e34abaa1d265489fa35c25a3eae6f052, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:31.739688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69867439-d21a-4df8-a5c6-66f2d745b83e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:31.768972 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1072/2086] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:31 2022] PUT /identity/v3/projects/70acee88008f4d83af21de726cafe09a/users/e34abaa1d265489fa35c25a3eae6f052/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:31.773996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69867439-d21a-4df8-a5c6-66f2d745b83e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:31.779176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-69867439-d21a-4df8-a5c6-66f2d745b83e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:31.873352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce123a9a-2538-4954-bf1a-0da0eedd812a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:31.875401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce123a9a-2538-4954-bf1a-0da0eedd812a service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:31.886595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:31.905724 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1015/2087] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:31 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:31.910201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69867439-d21a-4df8-a5c6-66f2d745b83e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:31.911569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377dd8e-8943-4bf5-909c-f735f67f86d6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:31.911872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377dd8e-8943-4bf5-909c-f735f67f86d6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:31.912313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377dd8e-8943-4bf5-909c-f735f67f86d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:31.924505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d92b13-a2ac-4dff-9537-147cf8532fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:31.925173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d92b13-a2ac-4dff-9537-147cf8532fea admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:31.925489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d92b13-a2ac-4dff-9537-147cf8532fea admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:31.926371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d92b13-a2ac-4dff-9537-147cf8532fea admin admin] PATH_INFO: `/v3/projects/207636159f8a4318b3d141a828b16260` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:31.929437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d92b13-a2ac-4dff-9537-147cf8532fea admin admin] RBAC: Authorizing `identity:delete_project(project_id=207636159f8a4318b3d141a828b16260)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:31.939505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d92b13-a2ac-4dff-9537-147cf8532fea admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:31.943361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:31.956048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6377dd8e-8943-4bf5-909c-f735f67f86d6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:31.957440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6377dd8e-8943-4bf5-909c-f735f67f86d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:31.975745 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1016/2088] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:31.983074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d29349-08db-46cc-be40-aaeea40eff69 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:31.983514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d29349-08db-46cc-be40-aaeea40eff69 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:31.983953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d29349-08db-46cc-be40-aaeea40eff69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:32.008570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d8d29349-08db-46cc-be40-aaeea40eff69 None None] MFA Rules not processed for user `0980a1e24cdc491aa1a0202afc4fcd64`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:32.010468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d8d29349-08db-46cc-be40-aaeea40eff69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:32.017260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d5d92b13-a2ac-4dff-9537-147cf8532fea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '207636159f8a4318b3d141a828b16260', 'request_id': 'req-d5d92b13-a2ac-4dff-9537-147cf8532fea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:32.026149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d5d92b13-a2ac-4dff-9537-147cf8532fea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '207636159f8a4318b3d141a828b16260', 'request_id': 'req-d5d92b13-a2ac-4dff-9537-147cf8532fea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:32.044817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d5d92b13-a2ac-4dff-9537-147cf8532fea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '207636159f8a4318b3d141a828b16260', 'request_id': 'req-d5d92b13-a2ac-4dff-9537-147cf8532fea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:32.069733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d5d92b13-a2ac-4dff-9537-147cf8532fea admin admin] The token cache is being invalidate because project 207636159f8a4318b3d141a828b16260 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:32.070031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d5d92b13-a2ac-4dff-9537-147cf8532fea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:32.070958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d5d92b13-a2ac-4dff-9537-147cf8532fea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:32.074152 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1073/2089] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:31 2022] DELETE /identity/v3/projects/207636159f8a4318b3d141a828b16260 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:32.074702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d92b13-a2ac-4dff-9537-147cf8532fea admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:32.075942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d5d92b13-a2ac-4dff-9537-147cf8532fea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:32.113307 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1017/2090] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:32.114729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d29349-08db-46cc-be40-aaeea40eff69 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:32.116421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d8d29349-08db-46cc-be40-aaeea40eff69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:32.202219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d92b13-a2ac-4dff-9537-147cf8532fea admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:32.209399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd0bc5-853f-4534-b318-56cef5cb7950 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:32.210201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd0bc5-853f-4534-b318-56cef5cb7950 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:32.213734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd0bc5-853f-4534-b318-56cef5cb7950 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:32.214040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd0bc5-853f-4534-b318-56cef5cb7950 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:32.216904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0cfd0bc5-853f-4534-b318-56cef5cb7950 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:32.236676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d29349-08db-46cc-be40-aaeea40eff69 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:32.241747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea604450-2759-4021-a319-610a87bb1e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:32.242568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea604450-2759-4021-a319-610a87bb1e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:32.243140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea604450-2759-4021-a319-610a87bb1e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:32.243306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea604450-2759-4021-a319-610a87bb1e8e admin admin] PATH_INFO: `/v3/users/7408e981e5404f7faeb1b018e345f741` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:32.244652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea604450-2759-4021-a319-610a87bb1e8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7408e981e5404f7faeb1b018e345f741)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:32.247268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea604450-2759-4021-a319-610a87bb1e8e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:32.322767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd0bc5-853f-4534-b318-56cef5cb7950 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:32.324477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd0bc5-853f-4534-b318-56cef5cb7950 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:32.333026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ea604450-2759-4021-a319-610a87bb1e8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7408e981e5404f7faeb1b018e345f741'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:32.334311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ea604450-2759-4021-a319-610a87bb1e8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7408e981e5404f7faeb1b018e345f741'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:32.335418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ea604450-2759-4021-a319-610a87bb1e8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7408e981e5404f7faeb1b018e345f741'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:32.337407 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1074/2091] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:32.347382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ea604450-2759-4021-a319-610a87bb1e8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7408e981e5404f7faeb1b018e345f741'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:32.350111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd0bc5-853f-4534-b318-56cef5cb7950 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:32.354828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0cfd0bc5-853f-4534-b318-56cef5cb7950 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:32.370691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ea604450-2759-4021-a319-610a87bb1e8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7408e981e5404f7faeb1b018e345f741'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:32.387118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd0bc5-853f-4534-b318-56cef5cb7950 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:32.399906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a8c85-2429-456d-9845-b757bd3b83f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:32.403083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a8c85-2429-456d-9845-b757bd3b83f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:32.407067 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1018/2092] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:32 2022] DELETE /identity/v3/users/7408e981e5404f7faeb1b018e345f741 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:32.407212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a8c85-2429-456d-9845-b757bd3b83f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:32.407212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a8c85-2429-456d-9845-b757bd3b83f8 admin admin] PATH_INFO: `/v3/projects/c70de3f14b4f430ebe701ae7543b7e9f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:32.414894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a8c85-2429-456d-9845-b757bd3b83f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c70de3f14b4f430ebe701ae7543b7e9f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:32.417547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a8c85-2429-456d-9845-b757bd3b83f8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:32.485651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:32.502772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-636a8c85-2429-456d-9845-b757bd3b83f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c70de3f14b4f430ebe701ae7543b7e9f', 'request_id': 'req-636a8c85-2429-456d-9845-b757bd3b83f8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:32.512192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-636a8c85-2429-456d-9845-b757bd3b83f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c70de3f14b4f430ebe701ae7543b7e9f', 'request_id': 'req-636a8c85-2429-456d-9845-b757bd3b83f8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:32.517597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea604450-2759-4021-a319-610a87bb1e8e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:32.521087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-636a8c85-2429-456d-9845-b757bd3b83f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c70de3f14b4f430ebe701ae7543b7e9f', 'request_id': 'req-636a8c85-2429-456d-9845-b757bd3b83f8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:32.521220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ea604450-2759-4021-a319-610a87bb1e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:32.541519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-636a8c85-2429-456d-9845-b757bd3b83f8 admin admin] The token cache is being invalidate because project c70de3f14b4f430ebe701ae7543b7e9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:32.541826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-636a8c85-2429-456d-9845-b757bd3b83f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:32.542713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-636a8c85-2429-456d-9845-b757bd3b83f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:32.545109 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1075/2093] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:32 2022] DELETE /identity/v3/projects/c70de3f14b4f430ebe701ae7543b7e9f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:32.551241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef654b69-b968-45cb-aafe-5b399d778984 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:32.553508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef654b69-b968-45cb-aafe-5b399d778984 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:32.553879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef654b69-b968-45cb-aafe-5b399d778984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:32.599044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:32.614437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ef654b69-b968-45cb-aafe-5b399d778984 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:32.615905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ef654b69-b968-45cb-aafe-5b399d778984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:32.701909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea604450-2759-4021-a319-610a87bb1e8e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:32.767661 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1076/2094] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:32.777993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92eb16c-28ee-4b0f-8255-85ddf91ad3e5 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:32.778397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92eb16c-28ee-4b0f-8255-85ddf91ad3e5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:32.778694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92eb16c-28ee-4b0f-8255-85ddf91ad3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:32.808978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedb8b3-7e3f-4606-8ca0-f17ddce70437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:32.809810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedb8b3-7e3f-4606-8ca0-f17ddce70437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:32.810097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedb8b3-7e3f-4606-8ca0-f17ddce70437 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:32.810452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedb8b3-7e3f-4606-8ca0-f17ddce70437 admin admin] PATH_INFO: `/v3/projects/a68a43f9a1a44afdbe35ce9ce10e2279` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:32.811750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eedb8b3-7e3f-4606-8ca0-f17ddce70437 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a68a43f9a1a44afdbe35ce9ce10e2279)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:32.814102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eedb8b3-7e3f-4606-8ca0-f17ddce70437 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:32.830624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b92eb16c-28ee-4b0f-8255-85ddf91ad3e5 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:32.835127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b92eb16c-28ee-4b0f-8255-85ddf91ad3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:32.872159 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1077/2095] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:49:32.876910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92eb16c-28ee-4b0f-8255-85ddf91ad3e5 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:32.896010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92eb16c-28ee-4b0f-8255-85ddf91ad3e5 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:32.910857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e147f3e-c384-47da-9021-ff3eadcf5dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:32.912186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e147f3e-c384-47da-9021-ff3eadcf5dba admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:32.913376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e147f3e-c384-47da-9021-ff3eadcf5dba admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:32.913875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e147f3e-c384-47da-9021-ff3eadcf5dba admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:32.919192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e147f3e-c384-47da-9021-ff3eadcf5dba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:49:32.920446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e147f3e-c384-47da-9021-ff3eadcf5dba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:32.923907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e147f3e-c384-47da-9021-ff3eadcf5dba admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:32.961270 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1078/2096] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:49:32.973126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2eedb8b3-7e3f-4606-8ca0-f17ddce70437 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a68a43f9a1a44afdbe35ce9ce10e2279', 'request_id': 'req-2eedb8b3-7e3f-4606-8ca0-f17ddce70437'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:32.978600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e147f3e-c384-47da-9021-ff3eadcf5dba admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:33.008312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2eedb8b3-7e3f-4606-8ca0-f17ddce70437 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a68a43f9a1a44afdbe35ce9ce10e2279', 'request_id': 'req-2eedb8b3-7e3f-4606-8ca0-f17ddce70437'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:33.019696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e147f3e-c384-47da-9021-ff3eadcf5dba admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:33.022072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2eedb8b3-7e3f-4606-8ca0-f17ddce70437 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a68a43f9a1a44afdbe35ce9ce10e2279', 'request_id': 'req-2eedb8b3-7e3f-4606-8ca0-f17ddce70437'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:33.059344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2eedb8b3-7e3f-4606-8ca0-f17ddce70437 admin admin] The token cache is being invalidate because project a68a43f9a1a44afdbe35ce9ce10e2279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:33.059547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2eedb8b3-7e3f-4606-8ca0-f17ddce70437 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:33.060853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2eedb8b3-7e3f-4606-8ca0-f17ddce70437 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:33.069108 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1019/2097] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:32 2022] DELETE /identity/v3/projects/a68a43f9a1a44afdbe35ce9ce10e2279 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:33.071187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedb8b3-7e3f-4606-8ca0-f17ddce70437 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:33.084305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2eedb8b3-7e3f-4606-8ca0-f17ddce70437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:33.178341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd8542d-7a06-4bd1-8f56-258eaeeb96ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:33.179228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8542d-7a06-4bd1-8f56-258eaeeb96ab admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:33.179352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8542d-7a06-4bd1-8f56-258eaeeb96ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:33.179530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8542d-7a06-4bd1-8f56-258eaeeb96ab admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:33.183008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd8542d-7a06-4bd1-8f56-258eaeeb96ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:33.187765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd8542d-7a06-4bd1-8f56-258eaeeb96ab admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:33.192308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedb8b3-7e3f-4606-8ca0-f17ddce70437 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:33.202263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960b7ba-6643-49b4-a4db-e049258c6479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:33.203182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960b7ba-6643-49b4-a4db-e049258c6479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:33.203615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960b7ba-6643-49b4-a4db-e049258c6479 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:33.204098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960b7ba-6643-49b4-a4db-e049258c6479 admin admin] PATH_INFO: `/v3/projects/215471dd429642b3974741aade3aaba0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:33.205398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960b7ba-6643-49b4-a4db-e049258c6479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=215471dd429642b3974741aade3aaba0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:33.207826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960b7ba-6643-49b4-a4db-e049258c6479 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:33.210177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:33.288744 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1079/2098] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:32 2022] POST /identity/v3/projects => generated 395 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:49:33.292783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbe717-984f-4bc3-b962-dd83da233056 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:33.292783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbe717-984f-4bc3-b962-dd83da233056 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:33.294842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbe717-984f-4bc3-b962-dd83da233056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:33.328795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f960b7ba-6643-49b4-a4db-e049258c6479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '215471dd429642b3974741aade3aaba0', 'request_id': 'req-f960b7ba-6643-49b4-a4db-e049258c6479'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:33.341010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f960b7ba-6643-49b4-a4db-e049258c6479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '215471dd429642b3974741aade3aaba0', 'request_id': 'req-f960b7ba-6643-49b4-a4db-e049258c6479'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:33.348059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f960b7ba-6643-49b4-a4db-e049258c6479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '215471dd429642b3974741aade3aaba0', 'request_id': 'req-f960b7ba-6643-49b4-a4db-e049258c6479'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:33.353112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-cfcbe717-984f-4bc3-b962-dd83da233056 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:33.354370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cfcbe717-984f-4bc3-b962-dd83da233056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:33.364989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f960b7ba-6643-49b4-a4db-e049258c6479 admin admin] The token cache is being invalidate because project 215471dd429642b3974741aade3aaba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:33.365374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f960b7ba-6643-49b4-a4db-e049258c6479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:33.371051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f960b7ba-6643-49b4-a4db-e049258c6479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:33.372546 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1020/2099] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:33 2022] DELETE /identity/v3/projects/215471dd429642b3974741aade3aaba0 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:33.377491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960b7ba-6643-49b4-a4db-e049258c6479 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:33.379032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f960b7ba-6643-49b4-a4db-e049258c6479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:33.398685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:33.496600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960b7ba-6643-49b4-a4db-e049258c6479 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:33.505648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70dec6-11c1-42a1-aec7-6ea81cc32ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:33.506327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70dec6-11c1-42a1-aec7-6ea81cc32ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:33.506647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70dec6-11c1-42a1-aec7-6ea81cc32ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:33.507169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70dec6-11c1-42a1-aec7-6ea81cc32ba0 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:33.507829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70dec6-11c1-42a1-aec7-6ea81cc32ba0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:33.508440 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1080/2100] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:33.510981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70dec6-11c1-42a1-aec7-6ea81cc32ba0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:33.511096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435cdc9-973e-4623-85b6-c356a9121b0a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:33.511492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435cdc9-973e-4623-85b6-c356a9121b0a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:33.512016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435cdc9-973e-4623-85b6-c356a9121b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:33.534312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1435cdc9-973e-4623-85b6-c356a9121b0a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:33.535951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1435cdc9-973e-4623-85b6-c356a9121b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:33.547929 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1081/2101] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:33.550846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552d563-60e9-4cd6-a8cf-ebc103d4567e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:33.551332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552d563-60e9-4cd6-a8cf-ebc103d4567e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:33.551757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552d563-60e9-4cd6-a8cf-ebc103d4567e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:33.559143 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1021/2102] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:33 2022] POST /identity/v3/users => generated 298 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:33.562169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abdd12-ef1a-45d6-b7c6-a8d68deeb1eb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:33.565560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abdd12-ef1a-45d6-b7c6-a8d68deeb1eb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:33.566113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abdd12-ef1a-45d6-b7c6-a8d68deeb1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:33.584998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8552d563-60e9-4cd6-a8cf-ebc103d4567e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:33.586583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8552d563-60e9-4cd6-a8cf-ebc103d4567e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:33.609741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f9abdd12-ef1a-45d6-b7c6-a8d68deeb1eb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:33.611490 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1082/2103] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:33.611908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f9abdd12-ef1a-45d6-b7c6-a8d68deeb1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:33.612942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552d563-60e9-4cd6-a8cf-ebc103d4567e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:33.625799 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1022/2104] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:33.627416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9abdd12-ef1a-45d6-b7c6-a8d68deeb1eb None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:33.629278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552d563-60e9-4cd6-a8cf-ebc103d4567e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:33.636118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97bec0-55f7-46d2-a452-1cdc1c19de81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:33.636934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97bec0-55f7-46d2-a452-1cdc1c19de81 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:33.637362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97bec0-55f7-46d2-a452-1cdc1c19de81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:33.637787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97bec0-55f7-46d2-a452-1cdc1c19de81 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:33.638706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97bec0-55f7-46d2-a452-1cdc1c19de81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:33.640459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97bec0-55f7-46d2-a452-1cdc1c19de81 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:33.641278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9abdd12-ef1a-45d6-b7c6-a8d68deeb1eb None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:33.648425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5734a-aaf9-4776-ad16-50772f37b7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:33.649807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5734a-aaf9-4776-ad16-50772f37b7b3 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:33.650818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5734a-aaf9-4776-ad16-50772f37b7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:33.651360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5734a-aaf9-4776-ad16-50772f37b7b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:33.652642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5734a-aaf9-4776-ad16-50772f37b7b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:49:33.653951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5734a-aaf9-4776-ad16-50772f37b7b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:33.657811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5734a-aaf9-4776-ad16-50772f37b7b3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:33.664380 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1083/2105] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:33.665647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97bec0-55f7-46d2-a452-1cdc1c19de81 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:33.681738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97bec0-55f7-46d2-a452-1cdc1c19de81 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:33.694918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea45e1-28e3-4fc1-86b5-fbb4c09b8286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:33.696353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea45e1-28e3-4fc1-86b5-fbb4c09b8286 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:33.696481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea45e1-28e3-4fc1-86b5-fbb4c09b8286 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:33.696677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea45e1-28e3-4fc1-86b5-fbb4c09b8286 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:33.699495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea45e1-28e3-4fc1-86b5-fbb4c09b8286 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:49:33.699762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea45e1-28e3-4fc1-86b5-fbb4c09b8286 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:33.702131 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1023/2106] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:49:33.704496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5734a-aaf9-4776-ad16-50772f37b7b3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:33.705383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea45e1-28e3-4fc1-86b5-fbb4c09b8286 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:33.760964 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1084/2107] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:49:33.761823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea45e1-28e3-4fc1-86b5-fbb4c09b8286 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:33.763185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5734a-aaf9-4776-ad16-50772f37b7b3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:33.771321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f759ad-73cf-489d-8224-5c4f5cc28f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:33.773368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f759ad-73cf-489d-8224-5c4f5cc28f7c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:33.774600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f759ad-73cf-489d-8224-5c4f5cc28f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:33.775821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f759ad-73cf-489d-8224-5c4f5cc28f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:33.777692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f759ad-73cf-489d-8224-5c4f5cc28f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:33.780572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f759ad-73cf-489d-8224-5c4f5cc28f7c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:33.794920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea45e1-28e3-4fc1-86b5-fbb4c09b8286 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:33.796973 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1024/2108] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:33.798477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f759ad-73cf-489d-8224-5c4f5cc28f7c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:33.812204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f759ad-73cf-489d-8224-5c4f5cc28f7c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:33.817212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6fd35d-c5b4-4944-8fe9-107df8ca87c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:33.818473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fd35d-c5b4-4944-8fe9-107df8ca87c7 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:33.819727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cc695-fe53-4e51-86fc-33bd66af5587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:33.820810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fd35d-c5b4-4944-8fe9-107df8ca87c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:33.821023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cc695-fe53-4e51-86fc-33bd66af5587 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:33.821135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fd35d-c5b4-4944-8fe9-107df8ca87c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:33.821737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cc695-fe53-4e51-86fc-33bd66af5587 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:33.822893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cc695-fe53-4e51-86fc-33bd66af5587 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:33.823784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3cc695-fe53-4e51-86fc-33bd66af5587 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:33.824429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6fd35d-c5b4-4944-8fe9-107df8ca87c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:33.826259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3cc695-fe53-4e51-86fc-33bd66af5587 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:33.831349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6fd35d-c5b4-4944-8fe9-107df8ca87c7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:33.893455 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1025/2109] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:33 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:33.901060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cc695-fe53-4e51-86fc-33bd66af5587 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:33.903901 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1085/2110] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:33 2022] POST /identity/v3/projects => generated 375 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:33.908897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6fd35d-c5b4-4944-8fe9-107df8ca87c7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:33.974053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:34.032541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cc695-fe53-4e51-86fc-33bd66af5587 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:34.078555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6fd35d-c5b4-4944-8fe9-107df8ca87c7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:34.101492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07a425-6716-40fe-bd91-c0c77c0a8203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:34.106087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07a425-6716-40fe-bd91-c0c77c0a8203 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:34.106385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07a425-6716-40fe-bd91-c0c77c0a8203 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:34.106697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07a425-6716-40fe-bd91-c0c77c0a8203 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:34.108559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb07a425-6716-40fe-bd91-c0c77c0a8203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:34.112367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb07a425-6716-40fe-bd91-c0c77c0a8203 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:34.113725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c51e8f-fde7-4ae3-99e0-9e9911851cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:34.115147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c51e8f-fde7-4ae3-99e0-9e9911851cb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:34.115537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c51e8f-fde7-4ae3-99e0-9e9911851cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:34.115938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c51e8f-fde7-4ae3-99e0-9e9911851cb5 admin admin] PATH_INFO: `/v3/projects/56cc6eafd5c246c398c881900345d3b7/users/129ab6b2ad664992af2149c447676971/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:34.149617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c51e8f-fde7-4ae3-99e0-9e9911851cb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56cc6eafd5c246c398c881900345d3b7, user_id=129ab6b2ad664992af2149c447676971, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:34.157673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c51e8f-fde7-4ae3-99e0-9e9911851cb5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:34.192630 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1086/2112] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:33 2022] POST /identity/v3/users => generated 291 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:34.192630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07a425-6716-40fe-bd91-c0c77c0a8203 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:34.198184 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1026/2112] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:33 2022] PUT /identity/v3/projects/56cc6eafd5c246c398c881900345d3b7/users/129ab6b2ad664992af2149c447676971/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:34.199845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b245dbe-a17b-4c0f-922b-7ecabe57831d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:34.200388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b245dbe-a17b-4c0f-922b-7ecabe57831d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:34.200771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b245dbe-a17b-4c0f-922b-7ecabe57831d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:34.218763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07a425-6716-40fe-bd91-c0c77c0a8203 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:34.247476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0b245dbe-a17b-4c0f-922b-7ecabe57831d None None] MFA Rules not processed for user `129ab6b2ad664992af2149c447676971`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:34.261674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0b245dbe-a17b-4c0f-922b-7ecabe57831d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:34.352440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:34.369104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09525b9e-72d5-4a74-9921-46021af6a85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:34.369790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09525b9e-72d5-4a74-9921-46021af6a85d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:34.370097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09525b9e-72d5-4a74-9921-46021af6a85d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:34.370378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09525b9e-72d5-4a74-9921-46021af6a85d admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:34.371059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09525b9e-72d5-4a74-9921-46021af6a85d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:34.379079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09525b9e-72d5-4a74-9921-46021af6a85d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:34.399487 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1027/2113] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:34.401055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b245dbe-a17b-4c0f-922b-7ecabe57831d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:34.417197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b245dbe-a17b-4c0f-922b-7ecabe57831d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:34.424969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5397f-0635-47ee-8503-21689991bd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:34.425487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5397f-0635-47ee-8503-21689991bd0b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:34.427176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5397f-0635-47ee-8503-21689991bd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:34.427176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5397f-0635-47ee-8503-21689991bd0b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:34.427876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f5397f-0635-47ee-8503-21689991bd0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:34.429790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f5397f-0635-47ee-8503-21689991bd0b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:34.430887 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1087/2114] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:34 2022] POST /identity/v3/users => generated 288 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:34.431799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09525b9e-72d5-4a74-9921-46021af6a85d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:34.448740 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1028/2115] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:34.449807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5397f-0635-47ee-8503-21689991bd0b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:34.455131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09525b9e-72d5-4a74-9921-46021af6a85d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:34.460814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf3fad-3496-4d0f-a397-ed39827689b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:34.462185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf3fad-3496-4d0f-a397-ed39827689b7 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:34.463304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf3fad-3496-4d0f-a397-ed39827689b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:34.463304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf3fad-3496-4d0f-a397-ed39827689b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:34.463940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf3fad-3496-4d0f-a397-ed39827689b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:34.465496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5397f-0635-47ee-8503-21689991bd0b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:34.466729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf3fad-3496-4d0f-a397-ed39827689b7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:34.481354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384db1b3-3292-496e-9341-97a4982c036d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:34.485477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384db1b3-3292-496e-9341-97a4982c036d admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:34.485811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384db1b3-3292-496e-9341-97a4982c036d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:34.486099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384db1b3-3292-496e-9341-97a4982c036d admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:34.488282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384db1b3-3292-496e-9341-97a4982c036d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:34.491934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384db1b3-3292-496e-9341-97a4982c036d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:34.516527 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1088/2116] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:34 2022] POST /identity/v3/projects => generated 395 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:34.517747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf3fad-3496-4d0f-a397-ed39827689b7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:34.526540 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1029/2117] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:34.527531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384db1b3-3292-496e-9341-97a4982c036d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:34.559098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384db1b3-3292-496e-9341-97a4982c036d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:34.589465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:34.604589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf3fad-3496-4d0f-a397-ed39827689b7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:34.614958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014772fb-8afd-4912-93ed-f2893b90407d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:34.615786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014772fb-8afd-4912-93ed-f2893b90407d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:34.616339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014772fb-8afd-4912-93ed-f2893b90407d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:34.616753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014772fb-8afd-4912-93ed-f2893b90407d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:34.617638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014772fb-8afd-4912-93ed-f2893b90407d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:34.620091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014772fb-8afd-4912-93ed-f2893b90407d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:34.637523 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1089/2118] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:34.638804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014772fb-8afd-4912-93ed-f2893b90407d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:34.656767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014772fb-8afd-4912-93ed-f2893b90407d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:34.659196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07854a8-7447-4b74-8090-00fea1fa4d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:34.660837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07854a8-7447-4b74-8090-00fea1fa4d12 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:34.661982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07854a8-7447-4b74-8090-00fea1fa4d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:34.662325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07854a8-7447-4b74-8090-00fea1fa4d12 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:34.666858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07854a8-7447-4b74-8090-00fea1fa4d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:34.667240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99d25f-bd61-4be0-8249-b3b1dfcfdd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:34.668240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99d25f-bd61-4be0-8249-b3b1dfcfdd65 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:34.668706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99d25f-bd61-4be0-8249-b3b1dfcfdd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:34.669255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99d25f-bd61-4be0-8249-b3b1dfcfdd65 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:34.669364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07854a8-7447-4b74-8090-00fea1fa4d12 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:34.671283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99d25f-bd61-4be0-8249-b3b1dfcfdd65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:34.673123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99d25f-bd61-4be0-8249-b3b1dfcfdd65 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:34.699134 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1090/2119] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:34.702245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99d25f-bd61-4be0-8249-b3b1dfcfdd65 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:34.713623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99d25f-bd61-4be0-8249-b3b1dfcfdd65 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:34.719682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea644a6d-bfc1-427e-98f7-8c4bba126eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:34.720430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea644a6d-bfc1-427e-98f7-8c4bba126eac admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:34.720797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea644a6d-bfc1-427e-98f7-8c4bba126eac admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:34.721185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea644a6d-bfc1-427e-98f7-8c4bba126eac admin admin] PATH_INFO: `/v3/projects/8b016dd4b31c436ca22895223dc4d28e/users/aab0ce57b5964e5582bc85b52e16f71c/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:34.738755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea644a6d-bfc1-427e-98f7-8c4bba126eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b016dd4b31c436ca22895223dc4d28e, user_id=aab0ce57b5964e5582bc85b52e16f71c, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:34.741649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea644a6d-bfc1-427e-98f7-8c4bba126eac admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:34.763243 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1030/2120] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:34 2022] POST /identity/v3/users => generated 304 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:34.763831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07854a8-7447-4b74-8090-00fea1fa4d12 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:34.770040 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1091/2121] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:34 2022] PUT /identity/v3/projects/8b016dd4b31c436ca22895223dc4d28e/users/aab0ce57b5964e5582bc85b52e16f71c/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:34.773506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea644a6d-bfc1-427e-98f7-8c4bba126eac admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:34.805954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07854a8-7447-4b74-8090-00fea1fa4d12 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:34.913792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea644a6d-bfc1-427e-98f7-8c4bba126eac admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:34.924045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fd08a-c4e1-4a27-8056-9d8adc2e0c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:34.925501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fd08a-c4e1-4a27-8056-9d8adc2e0c62 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:34.926125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fd08a-c4e1-4a27-8056-9d8adc2e0c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:34.927233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fd08a-c4e1-4a27-8056-9d8adc2e0c62 admin admin] PATH_INFO: `/v3/projects/08fae3c0a9324c68b16dd495478a89e3/users/e5bfbc1e512a445ca7527f3d79e40d16/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:34.928200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92361e0e-ef64-43fc-9f09-0979b904fc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:34.930143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92361e0e-ef64-43fc-9f09-0979b904fc02 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:34.932479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92361e0e-ef64-43fc-9f09-0979b904fc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:34.932866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92361e0e-ef64-43fc-9f09-0979b904fc02 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:34.933742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92361e0e-ef64-43fc-9f09-0979b904fc02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:34.935539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92361e0e-ef64-43fc-9f09-0979b904fc02 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:34.952286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fd08a-c4e1-4a27-8056-9d8adc2e0c62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08fae3c0a9324c68b16dd495478a89e3, user_id=e5bfbc1e512a445ca7527f3d79e40d16, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:34.954090 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1092/2122] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:34.955594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededa483-7a7c-4fec-9376-f110544d86e9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:34.955951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededa483-7a7c-4fec-9376-f110544d86e9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:34.956294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededa483-7a7c-4fec-9376-f110544d86e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:34.958902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fd08a-c4e1-4a27-8056-9d8adc2e0c62 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:34.994760 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1031/2123] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:34 2022] PUT /identity/v3/projects/08fae3c0a9324c68b16dd495478a89e3/users/e5bfbc1e512a445ca7527f3d79e40d16/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:34.996819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fd08a-c4e1-4a27-8056-9d8adc2e0c62 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:35.045879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ededa483-7a7c-4fec-9376-f110544d86e9 None None] MFA Rules not processed for user `aab0ce57b5964e5582bc85b52e16f71c`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:35.047448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ededa483-7a7c-4fec-9376-f110544d86e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:35.071296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:35.188573 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1093/2124] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:35.190222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4ffa0-ce34-458e-a209-6a6c0a37c895 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:35.190531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4ffa0-ce34-458e-a209-6a6c0a37c895 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:35.190809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4ffa0-ce34-458e-a209-6a6c0a37c895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:35.225213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fd08a-c4e1-4a27-8056-9d8adc2e0c62 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:35.236371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-85e4ffa0-ce34-458e-a209-6a6c0a37c895 None None] MFA Rules not processed for user `e5bfbc1e512a445ca7527f3d79e40d16`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:35.237258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f062dc-30af-4ba0-84f1-c097441de2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:35.237595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-85e4ffa0-ce34-458e-a209-6a6c0a37c895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:35.238413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f062dc-30af-4ba0-84f1-c097441de2a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:35.239118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f062dc-30af-4ba0-84f1-c097441de2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:35.239630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f062dc-30af-4ba0-84f1-c097441de2a7 admin admin] PATH_INFO: `/v3/projects/3815398f22e144bd9be64ac1890ba4cc/users/f9b8da34c44a4817989ed79b3d4c4326/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:35.269424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f062dc-30af-4ba0-84f1-c097441de2a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3815398f22e144bd9be64ac1890ba4cc, user_id=f9b8da34c44a4817989ed79b3d4c4326, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:35.272698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f062dc-30af-4ba0-84f1-c097441de2a7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:35.295802 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1032/2125] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:34 2022] PUT /identity/v3/projects/3815398f22e144bd9be64ac1890ba4cc/users/f9b8da34c44a4817989ed79b3d4c4326/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:35.295802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f062dc-30af-4ba0-84f1-c097441de2a7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:35.324671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:35.342104 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1094/2126] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:35.342104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4ffa0-ce34-458e-a209-6a6c0a37c895 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:35.355009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4ffa0-ce34-458e-a209-6a6c0a37c895 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:35.427738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f062dc-30af-4ba0-84f1-c097441de2a7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:35.441880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be70cb-dc8b-424d-8d44-e2ca7d22417b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:35.442600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be70cb-dc8b-424d-8d44-e2ca7d22417b admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:35.443086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be70cb-dc8b-424d-8d44-e2ca7d22417b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:35.444643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be70cb-dc8b-424d-8d44-e2ca7d22417b admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:35.445724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be70cb-dc8b-424d-8d44-e2ca7d22417b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:35.447684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be70cb-dc8b-424d-8d44-e2ca7d22417b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:35.501531 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1033/2127] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:35 2022] POST /identity/v3/projects => generated 381 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:35.502399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be70cb-dc8b-424d-8d44-e2ca7d22417b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:35.513789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4707bb-dd18-45b8-917f-78ad30c8d3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:35.515013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4707bb-dd18-45b8-917f-78ad30c8d3df admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:35.515666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4707bb-dd18-45b8-917f-78ad30c8d3df admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:35.516107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4707bb-dd18-45b8-917f-78ad30c8d3df admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:35.518675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4707bb-dd18-45b8-917f-78ad30c8d3df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:35.521052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4707bb-dd18-45b8-917f-78ad30c8d3df admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:35.533126 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1095/2128] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:35.534240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4707bb-dd18-45b8-917f-78ad30c8d3df admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:35.546174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4707bb-dd18-45b8-917f-78ad30c8d3df admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:35.593214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be70cb-dc8b-424d-8d44-e2ca7d22417b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:35.598738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f2103-b93c-487b-81fc-e12bd823c15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:35.599433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f2103-b93c-487b-81fc-e12bd823c15e admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:35.599719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f2103-b93c-487b-81fc-e12bd823c15e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:35.599995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f2103-b93c-487b-81fc-e12bd823c15e admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:35.600658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f2103-b93c-487b-81fc-e12bd823c15e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:35.601723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:35.602639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f2103-b93c-487b-81fc-e12bd823c15e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:35.633826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945abf0-b805-4cc0-be95-faf5e5561515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:35.635168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945abf0-b805-4cc0-be95-faf5e5561515 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:35.635511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945abf0-b805-4cc0-be95-faf5e5561515 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:35.635847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945abf0-b805-4cc0-be95-faf5e5561515 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:35.636542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f945abf0-b805-4cc0-be95-faf5e5561515 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:35.638643 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1034/2129] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:35 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:35.638976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f945abf0-b805-4cc0-be95-faf5e5561515 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:35.639864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f2103-b93c-487b-81fc-e12bd823c15e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:35.662335 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1096/2130] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:35 2022] POST /identity/v3/users => generated 297 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:35.663416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945abf0-b805-4cc0-be95-faf5e5561515 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:35.678319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945abf0-b805-4cc0-be95-faf5e5561515 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:35.724791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f2103-b93c-487b-81fc-e12bd823c15e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:35.731356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d02540-2cb2-4a7d-b344-04e51fafcc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:35.732580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d02540-2cb2-4a7d-b344-04e51fafcc78 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:35.732869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d02540-2cb2-4a7d-b344-04e51fafcc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:35.733149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d02540-2cb2-4a7d-b344-04e51fafcc78 admin admin] PATH_INFO: `/v3/projects/3815398f22e144bd9be64ac1890ba4cc/users/f9b8da34c44a4817989ed79b3d4c4326/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:35.735948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d02540-2cb2-4a7d-b344-04e51fafcc78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3815398f22e144bd9be64ac1890ba4cc, user_id=f9b8da34c44a4817989ed79b3d4c4326, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:35.738947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d02540-2cb2-4a7d-b344-04e51fafcc78 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:35.753286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8c1df-96f3-47d8-be2a-7fa6e9f9f3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:35.754149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8c1df-96f3-47d8-be2a-7fa6e9f9f3bc admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:35.754436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8c1df-96f3-47d8-be2a-7fa6e9f9f3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:35.754794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8c1df-96f3-47d8-be2a-7fa6e9f9f3bc admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:35.755725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b8c1df-96f3-47d8-be2a-7fa6e9f9f3bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:35.758364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b8c1df-96f3-47d8-be2a-7fa6e9f9f3bc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:35.766607 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1035/2131] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:35 2022] PUT /identity/v3/projects/3815398f22e144bd9be64ac1890ba4cc/users/f9b8da34c44a4817989ed79b3d4c4326/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:35.767540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d02540-2cb2-4a7d-b344-04e51fafcc78 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:35.805585 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1097/2132] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:35 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:35.805585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cc8ec-13cc-48a3-aa5f-8c11ef5dcc4f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:35.805585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cc8ec-13cc-48a3-aa5f-8c11ef5dcc4f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:35.805585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cc8ec-13cc-48a3-aa5f-8c11ef5dcc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:35.826813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:35.837329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b33cc8ec-13cc-48a3-aa5f-8c11ef5dcc4f None None] MFA Rules not processed for user `f9b8da34c44a4817989ed79b3d4c4326`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:35.839621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b33cc8ec-13cc-48a3-aa5f-8c11ef5dcc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:35.882129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d02540-2cb2-4a7d-b344-04e51fafcc78 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:35.888259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbde64-2293-4ad2-ab3a-331d515df6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:35.888854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbde64-2293-4ad2-ab3a-331d515df6a6 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:35.889132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbde64-2293-4ad2-ab3a-331d515df6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:35.889406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbde64-2293-4ad2-ab3a-331d515df6a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:35.890329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dbde64-2293-4ad2-ab3a-331d515df6a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:35.892283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dbde64-2293-4ad2-ab3a-331d515df6a6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:35.910957 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1036/2133] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:35.912374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbde64-2293-4ad2-ab3a-331d515df6a6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:35.924711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbde64-2293-4ad2-ab3a-331d515df6a6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:35.931138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d95707-5008-4995-8eff-b7ba24123fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:35.932092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d95707-5008-4995-8eff-b7ba24123fdc admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:35.932656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d95707-5008-4995-8eff-b7ba24123fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:35.933082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d95707-5008-4995-8eff-b7ba24123fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:35.933887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d95707-5008-4995-8eff-b7ba24123fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:35.936421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d95707-5008-4995-8eff-b7ba24123fdc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:35.943309 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1098/2134] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:35 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:35.944358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cc8ec-13cc-48a3-aa5f-8c11ef5dcc4f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:35.969543 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1037/2135] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:35.970921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d95707-5008-4995-8eff-b7ba24123fdc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:35.973587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cc8ec-13cc-48a3-aa5f-8c11ef5dcc4f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:35.974937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-84d95707-5008-4995-8eff-b7ba24123fdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:35.985911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c8c98-f3c4-42f9-a4c4-4a2f26271a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:35.986861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c8c98-f3c4-42f9-a4c4-4a2f26271a75 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:35.988438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c8c98-f3c4-42f9-a4c4-4a2f26271a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:35.988852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c8c98-f3c4-42f9-a4c4-4a2f26271a75 admin admin] PATH_INFO: `/v3/projects/96a30b1ca3d64339b9a17dad76c210ff/users/060f0868eca247f8a34244ef7f8e15f3/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:36.030291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831c8c98-f3c4-42f9-a4c4-4a2f26271a75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96a30b1ca3d64339b9a17dad76c210ff, user_id=060f0868eca247f8a34244ef7f8e15f3, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:36.033444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831c8c98-f3c4-42f9-a4c4-4a2f26271a75 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:36.074628 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1099/2136] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:35 2022] PUT /identity/v3/projects/96a30b1ca3d64339b9a17dad76c210ff/users/060f0868eca247f8a34244ef7f8e15f3/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:36.079452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c8c98-f3c4-42f9-a4c4-4a2f26271a75 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:36.125004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:36.161574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d95707-5008-4995-8eff-b7ba24123fdc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:36.226690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c8c98-f3c4-42f9-a4c4-4a2f26271a75 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:36.234537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0816da4-71c5-4940-aaca-48db70869035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:36.234972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0816da4-71c5-4940-aaca-48db70869035 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:36.235320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0816da4-71c5-4940-aaca-48db70869035 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:36.235737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0816da4-71c5-4940-aaca-48db70869035 admin admin] PATH_INFO: `/v3/projects/9efda885854e4211bc1145a488270fbc/users/b12f27914a4544cf82521abd3e7db9a7/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:36.273122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0816da4-71c5-4940-aaca-48db70869035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9efda885854e4211bc1145a488270fbc, user_id=b12f27914a4544cf82521abd3e7db9a7, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:36.276681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0816da4-71c5-4940-aaca-48db70869035 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:36.301366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151a1f7-7724-4f88-96da-5bf828bc77e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:36.307144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151a1f7-7724-4f88-96da-5bf828bc77e9 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:36.307144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151a1f7-7724-4f88-96da-5bf828bc77e9 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:36.307144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151a1f7-7724-4f88-96da-5bf828bc77e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:36.329486 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1100/2137] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:36 2022] PUT /identity/v3/projects/9efda885854e4211bc1145a488270fbc/users/b12f27914a4544cf82521abd3e7db9a7/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:36.331596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0816da4-71c5-4940-aaca-48db70869035 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:36.396673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:36.427802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151a1f7-7724-4f88-96da-5bf828bc77e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:36.430287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151a1f7-7724-4f88-96da-5bf828bc77e9 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:36.487032 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1038/2138] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:35 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:36.488037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151a1f7-7724-4f88-96da-5bf828bc77e9 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:36.501375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0816da4-71c5-4940-aaca-48db70869035 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:36.519709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151a1f7-7724-4f88-96da-5bf828bc77e9 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:36.527137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9e4e9-7b7a-4942-94b9-195cf2873de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:36.527707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9e4e9-7b7a-4942-94b9-195cf2873de2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:36.527989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9e4e9-7b7a-4942-94b9-195cf2873de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:36.528261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9e4e9-7b7a-4942-94b9-195cf2873de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:36.530901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9e4e9-7b7a-4942-94b9-195cf2873de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:36.532975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9e4e9-7b7a-4942-94b9-195cf2873de2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:36.549489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45222596-1e8f-4443-b08f-b3c7f4e9b849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:36.550739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45222596-1e8f-4443-b08f-b3c7f4e9b849 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:36.551028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45222596-1e8f-4443-b08f-b3c7f4e9b849 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:36.551308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45222596-1e8f-4443-b08f-b3c7f4e9b849 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:36.551969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45222596-1e8f-4443-b08f-b3c7f4e9b849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:36.558579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45222596-1e8f-4443-b08f-b3c7f4e9b849 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:36.585759 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1039/2139] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:36.591482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45222596-1e8f-4443-b08f-b3c7f4e9b849 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:36.606351 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1101/2140] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:36.608060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9e4e9-7b7a-4942-94b9-195cf2873de2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:36.629230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45222596-1e8f-4443-b08f-b3c7f4e9b849 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:36.634725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9e4e9-7b7a-4942-94b9-195cf2873de2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:36.645113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961af419-396b-4ca9-8ee7-17a40c1f0d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:36.645876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961af419-396b-4ca9-8ee7-17a40c1f0d20 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:36.646180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961af419-396b-4ca9-8ee7-17a40c1f0d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:36.646462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961af419-396b-4ca9-8ee7-17a40c1f0d20 admin admin] PATH_INFO: `/v3/projects/9efda885854e4211bc1145a488270fbc/users/b12f27914a4544cf82521abd3e7db9a7/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:36.656494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961af419-396b-4ca9-8ee7-17a40c1f0d20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9efda885854e4211bc1145a488270fbc, user_id=b12f27914a4544cf82521abd3e7db9a7, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:36.660364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961af419-396b-4ca9-8ee7-17a40c1f0d20 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:36.666201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139d96b-1ae9-43b6-b524-b07256df962a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:36.667243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139d96b-1ae9-43b6-b524-b07256df962a admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:36.669078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139d96b-1ae9-43b6-b524-b07256df962a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:36.669581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139d96b-1ae9-43b6-b524-b07256df962a admin admin] PATH_INFO: `/v3/projects/96a30b1ca3d64339b9a17dad76c210ff/users/060f0868eca247f8a34244ef7f8e15f3/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:36.674466 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1040/2141] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:36 2022] PUT /identity/v3/projects/9efda885854e4211bc1145a488270fbc/users/b12f27914a4544cf82521abd3e7db9a7/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:36.675747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961af419-396b-4ca9-8ee7-17a40c1f0d20 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:36.676380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139d96b-1ae9-43b6-b524-b07256df962a admin admin] RBAC: Authorizing `identity:create_grant(project_id=96a30b1ca3d64339b9a17dad76c210ff, user_id=060f0868eca247f8a34244ef7f8e15f3, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:36.683625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139d96b-1ae9-43b6-b524-b07256df962a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:36.732869 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1102/2142] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:36 2022] PUT /identity/v3/projects/96a30b1ca3d64339b9a17dad76c210ff/users/060f0868eca247f8a34244ef7f8e15f3/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:36.740467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168c4e4-2d6a-4ce4-a93c-3645524d3109 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:36.740994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168c4e4-2d6a-4ce4-a93c-3645524d3109 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:36.741607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168c4e4-2d6a-4ce4-a93c-3645524d3109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:36.802642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e168c4e4-2d6a-4ce4-a93c-3645524d3109 None None] MFA Rules not processed for user `b12f27914a4544cf82521abd3e7db9a7`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:36.803271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961af419-396b-4ca9-8ee7-17a40c1f0d20 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:36.804288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e168c4e4-2d6a-4ce4-a93c-3645524d3109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:36.897333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d46d2-512f-4f3a-a688-c1f881159e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:36.897973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d46d2-512f-4f3a-a688-c1f881159e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:36.898386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d46d2-512f-4f3a-a688-c1f881159e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:36.898711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d46d2-512f-4f3a-a688-c1f881159e47 admin admin] PATH_INFO: `/v3/users/129ab6b2ad664992af2149c447676971` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:36.900507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d46d2-512f-4f3a-a688-c1f881159e47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=129ab6b2ad664992af2149c447676971)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:36.902764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d46d2-512f-4f3a-a688-c1f881159e47 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:36.974591 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1103/2143] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:36.976889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531945e-0d63-4832-890d-bfd4ce6cfe8a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:36.977097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531945e-0d63-4832-890d-bfd4ce6cfe8a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:36.977380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531945e-0d63-4832-890d-bfd4ce6cfe8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:36.985766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ef3d46d2-512f-4f3a-a688-c1f881159e47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '129ab6b2ad664992af2149c447676971'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:36.986458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ef3d46d2-512f-4f3a-a688-c1f881159e47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '129ab6b2ad664992af2149c447676971'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:36.987397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ef3d46d2-512f-4f3a-a688-c1f881159e47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '129ab6b2ad664992af2149c447676971'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:37.012667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ef3d46d2-512f-4f3a-a688-c1f881159e47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '129ab6b2ad664992af2149c447676971'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:37.018631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:37.051243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4531945e-0d63-4832-890d-bfd4ce6cfe8a None None] MFA Rules not processed for user `060f0868eca247f8a34244ef7f8e15f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:37.052457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4531945e-0d63-4832-890d-bfd4ce6cfe8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:37.068326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:37.083236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ef3d46d2-512f-4f3a-a688-c1f881159e47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '129ab6b2ad664992af2149c447676971'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:37.115193 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1041/2144] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:36 2022] DELETE /identity/v3/users/129ab6b2ad664992af2149c447676971 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:37.122736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507375c-a775-46b9-b3a8-d3bd53d45503 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:37.129257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507375c-a775-46b9-b3a8-d3bd53d45503 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:37.129582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507375c-a775-46b9-b3a8-d3bd53d45503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:37.199139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d507375c-a775-46b9-b3a8-d3bd53d45503 None None] MFA Rules not processed for user `e5bfbc1e512a445ca7527f3d79e40d16`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:37.206104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d507375c-a775-46b9-b3a8-d3bd53d45503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:37.286737 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1104/2145] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:36 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:37.288791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642145f3-4f67-4532-90f6-48de92a5f03e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:37.289027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642145f3-4f67-4532-90f6-48de92a5f03e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:37.289305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642145f3-4f67-4532-90f6-48de92a5f03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:37.337201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-642145f3-4f67-4532-90f6-48de92a5f03e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:37.338538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-642145f3-4f67-4532-90f6-48de92a5f03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:37.391089 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1042/2146] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:37.391089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507375c-a775-46b9-b3a8-d3bd53d45503 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:37.391089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d507375c-a775-46b9-b3a8-d3bd53d45503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:37.471697 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1105/2147] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:37.501126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642145f3-4f67-4532-90f6-48de92a5f03e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:37.503532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-642145f3-4f67-4532-90f6-48de92a5f03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:37.542225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507375c-a775-46b9-b3a8-d3bd53d45503 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:37.560935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03872d41-ad0a-44ff-96b4-d35ea0790731 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:37.560935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03872d41-ad0a-44ff-96b4-d35ea0790731 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:37.561375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03872d41-ad0a-44ff-96b4-d35ea0790731 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:37.561662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03872d41-ad0a-44ff-96b4-d35ea0790731 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:37.632540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642145f3-4f67-4532-90f6-48de92a5f03e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:37.645340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c36e3-8ebb-4b12-86fe-89b11faf8e82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:37.646975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c36e3-8ebb-4b12-86fe-89b11faf8e82 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:37.650723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c36e3-8ebb-4b12-86fe-89b11faf8e82 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:37.651116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c36e3-8ebb-4b12-86fe-89b11faf8e82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:37.675466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c36e3-8ebb-4b12-86fe-89b11faf8e82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:37.677068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c36e3-8ebb-4b12-86fe-89b11faf8e82 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:37.704463 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1106/2148] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:49:37.735924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03872d41-ad0a-44ff-96b4-d35ea0790731 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:37.739849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03872d41-ad0a-44ff-96b4-d35ea0790731 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:37.775924 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1043/2149] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:37 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:37.868350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c36e3-8ebb-4b12-86fe-89b11faf8e82 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:37.883625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c36e3-8ebb-4b12-86fe-89b11faf8e82 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:37.890850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd144652-03bf-4013-83b9-7c74ccef875f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:37.891677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd144652-03bf-4013-83b9-7c74ccef875f service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:37.892051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd144652-03bf-4013-83b9-7c74ccef875f service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:37.892423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd144652-03bf-4013-83b9-7c74ccef875f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:37.894505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cd144652-03bf-4013-83b9-7c74ccef875f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:37.907427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd144652-03bf-4013-83b9-7c74ccef875f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:37.909538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd144652-03bf-4013-83b9-7c74ccef875f service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:37.936029 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1107/2150] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:38.405639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03872d41-ad0a-44ff-96b4-d35ea0790731 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:38.407207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-03872d41-ad0a-44ff-96b4-d35ea0790731 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:38.428364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03872d41-ad0a-44ff-96b4-d35ea0790731 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:38.437661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72677921-e07a-44e3-8a24-bd9d775688e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:38.442478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72677921-e07a-44e3-8a24-bd9d775688e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:38.443315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72677921-e07a-44e3-8a24-bd9d775688e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:38.443691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72677921-e07a-44e3-8a24-bd9d775688e9 admin admin] PATH_INFO: `/v3/projects/56cc6eafd5c246c398c881900345d3b7` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:38.445061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72677921-e07a-44e3-8a24-bd9d775688e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56cc6eafd5c246c398c881900345d3b7)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:38.447763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72677921-e07a-44e3-8a24-bd9d775688e9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:38.496217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:38.549394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-72677921-e07a-44e3-8a24-bd9d775688e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56cc6eafd5c246c398c881900345d3b7', 'request_id': 'req-72677921-e07a-44e3-8a24-bd9d775688e9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:38.579426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-72677921-e07a-44e3-8a24-bd9d775688e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56cc6eafd5c246c398c881900345d3b7', 'request_id': 'req-72677921-e07a-44e3-8a24-bd9d775688e9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:38.600782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-72677921-e07a-44e3-8a24-bd9d775688e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56cc6eafd5c246c398c881900345d3b7', 'request_id': 'req-72677921-e07a-44e3-8a24-bd9d775688e9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:38.679672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-72677921-e07a-44e3-8a24-bd9d775688e9 admin admin] The token cache is being invalidate because project 56cc6eafd5c246c398c881900345d3b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:38.683009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-72677921-e07a-44e3-8a24-bd9d775688e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:38.685360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-72677921-e07a-44e3-8a24-bd9d775688e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:38.688095 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1044/2151] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:38 2022] DELETE /identity/v3/projects/56cc6eafd5c246c398c881900345d3b7 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:38.692366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd144652-03bf-4013-83b9-7c74ccef875f service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:38.713847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd144652-03bf-4013-83b9-7c74ccef875f service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:38.817508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:38.954337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb868f-2212-4576-9375-424806b4d120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:38.954901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb868f-2212-4576-9375-424806b4d120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:38.955184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb868f-2212-4576-9375-424806b4d120 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:38.955453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb868f-2212-4576-9375-424806b4d120 admin admin] PATH_INFO: `/v3/users/f9b8da34c44a4817989ed79b3d4c4326` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:38.958762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb868f-2212-4576-9375-424806b4d120 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9b8da34c44a4817989ed79b3d4c4326)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:38.960708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb868f-2212-4576-9375-424806b4d120 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:39.073311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-23eb868f-2212-4576-9375-424806b4d120 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9b8da34c44a4817989ed79b3d4c4326'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:39.096300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-23eb868f-2212-4576-9375-424806b4d120 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9b8da34c44a4817989ed79b3d4c4326'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:39.144676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-23eb868f-2212-4576-9375-424806b4d120 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9b8da34c44a4817989ed79b3d4c4326'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:39.145700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-23eb868f-2212-4576-9375-424806b4d120 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9b8da34c44a4817989ed79b3d4c4326'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:39.163615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-23eb868f-2212-4576-9375-424806b4d120 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9b8da34c44a4817989ed79b3d4c4326'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:39.178514 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1108/2152] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:38 2022] DELETE /identity/v3/users/f9b8da34c44a4817989ed79b3d4c4326 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:39.988574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72677921-e07a-44e3-8a24-bd9d775688e9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:39.990473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-72677921-e07a-44e3-8a24-bd9d775688e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:40.021808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72677921-e07a-44e3-8a24-bd9d775688e9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:40.071514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd557c9-76e3-4157-9158-bbb93e3c8451 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:40.072026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd557c9-76e3-4157-9158-bbb93e3c8451 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:40.072433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd557c9-76e3-4157-9158-bbb93e3c8451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:40.116735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-bcd557c9-76e3-4157-9158-bbb93e3c8451 None None] MFA Rules not processed for user `e34abaa1d265489fa35c25a3eae6f052`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:40.128686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bcd557c9-76e3-4157-9158-bbb93e3c8451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:40.169945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea610be4-e2dd-4019-9ab8-e28d66cdf56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:40.172287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea610be4-e2dd-4019-9ab8-e28d66cdf56d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:40.173510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea610be4-e2dd-4019-9ab8-e28d66cdf56d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:40.173879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea610be4-e2dd-4019-9ab8-e28d66cdf56d admin admin] PATH_INFO: `/v3/users/aab0ce57b5964e5582bc85b52e16f71c` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:40.176796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea610be4-e2dd-4019-9ab8-e28d66cdf56d admin admin] RBAC: Authorizing `identity:delete_user(user_id=aab0ce57b5964e5582bc85b52e16f71c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:40.183167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea610be4-e2dd-4019-9ab8-e28d66cdf56d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:40.250301 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1109/2153] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:49:40.252087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd557c9-76e3-4157-9158-bbb93e3c8451 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:40.263594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bcd557c9-76e3-4157-9158-bbb93e3c8451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:40.283300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ea610be4-e2dd-4019-9ab8-e28d66cdf56d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aab0ce57b5964e5582bc85b52e16f71c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:40.284599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ea610be4-e2dd-4019-9ab8-e28d66cdf56d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aab0ce57b5964e5582bc85b52e16f71c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:40.285776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ea610be4-e2dd-4019-9ab8-e28d66cdf56d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aab0ce57b5964e5582bc85b52e16f71c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:40.299800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ea610be4-e2dd-4019-9ab8-e28d66cdf56d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aab0ce57b5964e5582bc85b52e16f71c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:40.311017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd557c9-76e3-4157-9158-bbb93e3c8451 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:40.323230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ea610be4-e2dd-4019-9ab8-e28d66cdf56d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aab0ce57b5964e5582bc85b52e16f71c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:40.327163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160c744-854f-427c-b79d-3405e64f6fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:40.327928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160c744-854f-427c-b79d-3405e64f6fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:40.330626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160c744-854f-427c-b79d-3405e64f6fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:40.330917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160c744-854f-427c-b79d-3405e64f6fb6 admin admin] PATH_INFO: `/v3/projects/3815398f22e144bd9be64ac1890ba4cc` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:40.334947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160c744-854f-427c-b79d-3405e64f6fb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3815398f22e144bd9be64ac1890ba4cc)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:40.339345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160c744-854f-427c-b79d-3405e64f6fb6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:40.345295 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1045/2154] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:39 2022] DELETE /identity/v3/users/aab0ce57b5964e5582bc85b52e16f71c => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:40.347068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea610be4-e2dd-4019-9ab8-e28d66cdf56d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:40.350116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ea610be4-e2dd-4019-9ab8-e28d66cdf56d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:40.445154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:40.468085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:40.497216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2160c744-854f-427c-b79d-3405e64f6fb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3815398f22e144bd9be64ac1890ba4cc', 'request_id': 'req-2160c744-854f-427c-b79d-3405e64f6fb6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:40.506336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2160c744-854f-427c-b79d-3405e64f6fb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3815398f22e144bd9be64ac1890ba4cc', 'request_id': 'req-2160c744-854f-427c-b79d-3405e64f6fb6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:40.514286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2160c744-854f-427c-b79d-3405e64f6fb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3815398f22e144bd9be64ac1890ba4cc', 'request_id': 'req-2160c744-854f-427c-b79d-3405e64f6fb6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:40.524546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea610be4-e2dd-4019-9ab8-e28d66cdf56d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:40.549378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2160c744-854f-427c-b79d-3405e64f6fb6 admin admin] The token cache is being invalidate because project 3815398f22e144bd9be64ac1890ba4cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:40.549755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2160c744-854f-427c-b79d-3405e64f6fb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:40.551052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2160c744-854f-427c-b79d-3405e64f6fb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:40.552496 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1110/2155] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:40 2022] DELETE /identity/v3/projects/3815398f22e144bd9be64ac1890ba4cc => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:40.553748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6be513-0681-4d4e-b894-3eafe8847284 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:40.554030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6be513-0681-4d4e-b894-3eafe8847284 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:40.554305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6be513-0681-4d4e-b894-3eafe8847284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:40.583845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fd6be513-0681-4d4e-b894-3eafe8847284 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:40.585602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fd6be513-0681-4d4e-b894-3eafe8847284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:40.638281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35434703-635f-4653-a491-3f8bd5f9a9d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:40.638934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35434703-635f-4653-a491-3f8bd5f9a9d5 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:40.639214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35434703-635f-4653-a491-3f8bd5f9a9d5 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:40.639487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35434703-635f-4653-a491-3f8bd5f9a9d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:40.641518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-35434703-635f-4653-a491-3f8bd5f9a9d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:40.700443 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1111/2156] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:40.711512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf36c3e-6f3e-4988-8358-654c536d6295 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:40.711939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf36c3e-6f3e-4988-8358-654c536d6295 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:40.712270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf36c3e-6f3e-4988-8358-654c536d6295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:40.719068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35434703-635f-4653-a491-3f8bd5f9a9d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:40.721641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35434703-635f-4653-a491-3f8bd5f9a9d5 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:40.739461 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1046/2157] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:40.742019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35434703-635f-4653-a491-3f8bd5f9a9d5 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:40.744727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-35434703-635f-4653-a491-3f8bd5f9a9d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:40.754571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-faf36c3e-6f3e-4988-8358-654c536d6295 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:40.758253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-faf36c3e-6f3e-4988-8358-654c536d6295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:40.779555 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1112/2158] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:40.788456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c9fc3-864f-40db-b853-83f0a57f7ff8 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:40.788714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c9fc3-864f-40db-b853-83f0a57f7ff8 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:40.788983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c9fc3-864f-40db-b853-83f0a57f7ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:40.840097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-408c9fc3-864f-40db-b853-83f0a57f7ff8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:40.841394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-408c9fc3-864f-40db-b853-83f0a57f7ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:40.859320 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1113/2159] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:40.860387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c9fc3-864f-40db-b853-83f0a57f7ff8 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:40.885001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35434703-635f-4653-a491-3f8bd5f9a9d5 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:40.891001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91628d4-7f8a-4778-b9d4-05cd0138d6da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:40.891201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c9fc3-864f-40db-b853-83f0a57f7ff8 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:40.892223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91628d4-7f8a-4778-b9d4-05cd0138d6da service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:40.892770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91628d4-7f8a-4778-b9d4-05cd0138d6da service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:40.893248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91628d4-7f8a-4778-b9d4-05cd0138d6da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:40.901556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa851277-3c36-443c-96f9-a9285aaf9c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:40.902177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa851277-3c36-443c-96f9-a9285aaf9c77 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:40.902456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa851277-3c36-443c-96f9-a9285aaf9c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:40.902771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa851277-3c36-443c-96f9-a9285aaf9c77 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:40.904755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa851277-3c36-443c-96f9-a9285aaf9c77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:49:40.905071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa851277-3c36-443c-96f9-a9285aaf9c77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:40.906522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91628d4-7f8a-4778-b9d4-05cd0138d6da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:40.908152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa851277-3c36-443c-96f9-a9285aaf9c77 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:40.908418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91628d4-7f8a-4778-b9d4-05cd0138d6da service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:40.915107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:40.926272 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1114/2160] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:49:40.931622 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1047/2161] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:40.932199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa851277-3c36-443c-96f9-a9285aaf9c77 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:40.948965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa851277-3c36-443c-96f9-a9285aaf9c77 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:40.957501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846da85a-4c8c-443e-a3da-9a96550d923e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:40.958128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846da85a-4c8c-443e-a3da-9a96550d923e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:40.958418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846da85a-4c8c-443e-a3da-9a96550d923e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:40.958781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846da85a-4c8c-443e-a3da-9a96550d923e admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:40.959419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846da85a-4c8c-443e-a3da-9a96550d923e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:40.961770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846da85a-4c8c-443e-a3da-9a96550d923e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:41.007406 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1115/2162] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:40 2022] POST /identity/v3/projects => generated 395 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:41.009087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91628d4-7f8a-4778-b9d4-05cd0138d6da service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:41.022445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91628d4-7f8a-4778-b9d4-05cd0138d6da service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:41.155848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d2088-be1a-4feb-85af-9371b529ec30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:41.156477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d2088-be1a-4feb-85af-9371b529ec30 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:41.157881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d2088-be1a-4feb-85af-9371b529ec30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:41.158149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d2088-be1a-4feb-85af-9371b529ec30 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:41.158826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d2088-be1a-4feb-85af-9371b529ec30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:41.163334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d2088-be1a-4feb-85af-9371b529ec30 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:41.198183 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1048/2163] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:41 2022] POST /identity/v3/users => generated 298 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:41.202250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846da85a-4c8c-443e-a3da-9a96550d923e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:41.219765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846da85a-4c8c-443e-a3da-9a96550d923e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:41.231949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa276429-3dbc-4a50-a083-ddb9f335b1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:41.233239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa276429-3dbc-4a50-a083-ddb9f335b1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:41.234024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa276429-3dbc-4a50-a083-ddb9f335b1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:41.237514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa276429-3dbc-4a50-a083-ddb9f335b1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:41.238432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa276429-3dbc-4a50-a083-ddb9f335b1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:41.242439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa276429-3dbc-4a50-a083-ddb9f335b1e8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:41.265923 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1116/2164] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:41.270416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d2088-be1a-4feb-85af-9371b529ec30 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:41.301410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d2088-be1a-4feb-85af-9371b529ec30 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:41.315948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e0d41-63ef-461d-9a8b-d007a0472207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:41.315948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e0d41-63ef-461d-9a8b-d007a0472207 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:41.315948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e0d41-63ef-461d-9a8b-d007a0472207 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:41.315948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e0d41-63ef-461d-9a8b-d007a0472207 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:41.317484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e0d41-63ef-461d-9a8b-d007a0472207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:41.319552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e0d41-63ef-461d-9a8b-d007a0472207 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:41.327560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa276429-3dbc-4a50-a083-ddb9f335b1e8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:41.329698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-aa276429-3dbc-4a50-a083-ddb9f335b1e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:41.336218 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1049/2165] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:41.337160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e0d41-63ef-461d-9a8b-d007a0472207 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:41.348987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e0d41-63ef-461d-9a8b-d007a0472207 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:41.355444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a168bd-f706-4a6c-a8d7-c062b28385be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:41.356618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a168bd-f706-4a6c-a8d7-c062b28385be admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:41.356902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a168bd-f706-4a6c-a8d7-c062b28385be admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:41.357175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a168bd-f706-4a6c-a8d7-c062b28385be admin admin] PATH_INFO: `/v3/projects/76e0a9843d9c41f7b2fae591dc09d9d2/users/aa72870453cd4b958725801b842c7a28/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:41.360657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa276429-3dbc-4a50-a083-ddb9f335b1e8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:41.369607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a509c41-0b41-448c-9922-f38cffcf6ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:41.370463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a509c41-0b41-448c-9922-f38cffcf6ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:41.370853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a509c41-0b41-448c-9922-f38cffcf6ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:41.371363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a509c41-0b41-448c-9922-f38cffcf6ba2 admin admin] PATH_INFO: `/v3/projects/8b016dd4b31c436ca22895223dc4d28e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:41.372896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a509c41-0b41-448c-9922-f38cffcf6ba2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b016dd4b31c436ca22895223dc4d28e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:41.375902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a509c41-0b41-448c-9922-f38cffcf6ba2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:41.382090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a168bd-f706-4a6c-a8d7-c062b28385be admin admin] RBAC: Authorizing `identity:create_grant(project_id=76e0a9843d9c41f7b2fae591dc09d9d2, user_id=aa72870453cd4b958725801b842c7a28, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:41.385798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a168bd-f706-4a6c-a8d7-c062b28385be admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:41.446765 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1050/2166] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:41 2022] PUT /identity/v3/projects/76e0a9843d9c41f7b2fae591dc09d9d2/users/aa72870453cd4b958725801b842c7a28/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:41.451167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78339d25-692d-4e73-bedc-261e532fb9ff None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:41.451456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78339d25-692d-4e73-bedc-261e532fb9ff None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:41.451721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78339d25-692d-4e73-bedc-261e532fb9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:41.485775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a509c41-0b41-448c-9922-f38cffcf6ba2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b016dd4b31c436ca22895223dc4d28e', 'request_id': 'req-6a509c41-0b41-448c-9922-f38cffcf6ba2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:41.492407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:41.493695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-78339d25-692d-4e73-bedc-261e532fb9ff None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:41.495201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-78339d25-692d-4e73-bedc-261e532fb9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:41.511350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a509c41-0b41-448c-9922-f38cffcf6ba2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b016dd4b31c436ca22895223dc4d28e', 'request_id': 'req-6a509c41-0b41-448c-9922-f38cffcf6ba2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:41.523493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a509c41-0b41-448c-9922-f38cffcf6ba2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b016dd4b31c436ca22895223dc4d28e', 'request_id': 'req-6a509c41-0b41-448c-9922-f38cffcf6ba2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:41.543652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a509c41-0b41-448c-9922-f38cffcf6ba2 admin admin] The token cache is being invalidate because project 8b016dd4b31c436ca22895223dc4d28e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:41.544182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a509c41-0b41-448c-9922-f38cffcf6ba2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:41.545277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a509c41-0b41-448c-9922-f38cffcf6ba2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:41.548653 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1117/2167] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:41 2022] DELETE /identity/v3/projects/8b016dd4b31c436ca22895223dc4d28e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:49:41.554827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a509c41-0b41-448c-9922-f38cffcf6ba2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:41.563797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6a509c41-0b41-448c-9922-f38cffcf6ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:41.597214 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1051/2168] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:41.607587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78339d25-692d-4e73-bedc-261e532fb9ff None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:41.609521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-78339d25-692d-4e73-bedc-261e532fb9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:41.665333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a509c41-0b41-448c-9922-f38cffcf6ba2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:41.671146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b903b5-af70-435c-9c59-403b37cd5b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:41.671845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b903b5-af70-435c-9c59-403b37cd5b10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:41.672131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b903b5-af70-435c-9c59-403b37cd5b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:41.672404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b903b5-af70-435c-9c59-403b37cd5b10 admin admin] PATH_INFO: `/v3/users/060f0868eca247f8a34244ef7f8e15f3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:41.673572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b903b5-af70-435c-9c59-403b37cd5b10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=060f0868eca247f8a34244ef7f8e15f3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:41.675557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b903b5-af70-435c-9c59-403b37cd5b10 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:41.751179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78339d25-692d-4e73-bedc-261e532fb9ff None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:41.755450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df33195-5cf9-4163-8b54-5c9e4ac45301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:41.757119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df33195-5cf9-4163-8b54-5c9e4ac45301 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:41.757465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df33195-5cf9-4163-8b54-5c9e4ac45301 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:41.757807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df33195-5cf9-4163-8b54-5c9e4ac45301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:41.763835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b3b903b5-af70-435c-9c59-403b37cd5b10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '060f0868eca247f8a34244ef7f8e15f3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:41.773211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df33195-5cf9-4163-8b54-5c9e4ac45301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:41.775673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df33195-5cf9-4163-8b54-5c9e4ac45301 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:41.780670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b3b903b5-af70-435c-9c59-403b37cd5b10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '060f0868eca247f8a34244ef7f8e15f3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:41.783708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:41.796962 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1052/2169] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:49:41.802528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b3b903b5-af70-435c-9c59-403b37cd5b10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '060f0868eca247f8a34244ef7f8e15f3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:41.802528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b3b903b5-af70-435c-9c59-403b37cd5b10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '060f0868eca247f8a34244ef7f8e15f3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:41.817434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b3b903b5-af70-435c-9c59-403b37cd5b10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '060f0868eca247f8a34244ef7f8e15f3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:41.831090 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1118/2170] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:41 2022] DELETE /identity/v3/users/060f0868eca247f8a34244ef7f8e15f3 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:42.435173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df33195-5cf9-4163-8b54-5c9e4ac45301 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:42.445262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df33195-5cf9-4163-8b54-5c9e4ac45301 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:42.526134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e90270-e7a9-4074-b797-22bfb1c7262e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:42.526885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e90270-e7a9-4074-b797-22bfb1c7262e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:42.527536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e90270-e7a9-4074-b797-22bfb1c7262e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:42.527664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e90270-e7a9-4074-b797-22bfb1c7262e admin admin] PATH_INFO: `/v3/projects/96a30b1ca3d64339b9a17dad76c210ff` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:42.529004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e90270-e7a9-4074-b797-22bfb1c7262e admin admin] RBAC: Authorizing `identity:delete_project(project_id=96a30b1ca3d64339b9a17dad76c210ff)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:42.531323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e90270-e7a9-4074-b797-22bfb1c7262e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:42.598198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-51e90270-e7a9-4074-b797-22bfb1c7262e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96a30b1ca3d64339b9a17dad76c210ff', 'request_id': 'req-51e90270-e7a9-4074-b797-22bfb1c7262e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:42.607181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-51e90270-e7a9-4074-b797-22bfb1c7262e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96a30b1ca3d64339b9a17dad76c210ff', 'request_id': 'req-51e90270-e7a9-4074-b797-22bfb1c7262e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:42.613567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-51e90270-e7a9-4074-b797-22bfb1c7262e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96a30b1ca3d64339b9a17dad76c210ff', 'request_id': 'req-51e90270-e7a9-4074-b797-22bfb1c7262e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:42.632646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-51e90270-e7a9-4074-b797-22bfb1c7262e admin admin] The token cache is being invalidate because project 96a30b1ca3d64339b9a17dad76c210ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:42.632936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-51e90270-e7a9-4074-b797-22bfb1c7262e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:42.635378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-51e90270-e7a9-4074-b797-22bfb1c7262e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:42.646999 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1053/2171] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:42 2022] DELETE /identity/v3/projects/96a30b1ca3d64339b9a17dad76c210ff => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:42.652262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab32fe7-bd6e-40ff-a4c3-967988ba92b3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:42.652741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab32fe7-bd6e-40ff-a4c3-967988ba92b3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:42.652828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab32fe7-bd6e-40ff-a4c3-967988ba92b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:42.677483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7ab32fe7-bd6e-40ff-a4c3-967988ba92b3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:42.678989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7ab32fe7-bd6e-40ff-a4c3-967988ba92b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:42.763568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:42.776156 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1119/2172] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:42.788613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c5f6e-a864-4324-8176-43ea693e14e1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:42.790007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c5f6e-a864-4324-8176-43ea693e14e1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:42.791163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c5f6e-a864-4324-8176-43ea693e14e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:42.818352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a63c5f6e-a864-4324-8176-43ea693e14e1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:42.819419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a63c5f6e-a864-4324-8176-43ea693e14e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:42.834938 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1054/2173] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:42.843356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab32fe7-bd6e-40ff-a4c3-967988ba92b3 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:42.855579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab32fe7-bd6e-40ff-a4c3-967988ba92b3 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:42.861369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac2d79-2e0a-431e-9e98-c9faf84d234f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:42.862475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac2d79-2e0a-431e-9e98-c9faf84d234f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:42.862842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac2d79-2e0a-431e-9e98-c9faf84d234f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:42.863120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac2d79-2e0a-431e-9e98-c9faf84d234f admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:42.863786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac2d79-2e0a-431e-9e98-c9faf84d234f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:49:42.864591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac2d79-2e0a-431e-9e98-c9faf84d234f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:42.865695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac2d79-2e0a-431e-9e98-c9faf84d234f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:42.888276 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1120/2174] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:49:42.890663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c5f6e-a864-4324-8176-43ea693e14e1 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:42.904790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c5f6e-a864-4324-8176-43ea693e14e1 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:42.911228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b9a5c-4684-4314-ac3c-bd61700f812b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:42.911937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b9a5c-4684-4314-ac3c-bd61700f812b admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:42.912285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b9a5c-4684-4314-ac3c-bd61700f812b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:42.912738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b9a5c-4684-4314-ac3c-bd61700f812b admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:42.913572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b9a5c-4684-4314-ac3c-bd61700f812b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:42.915916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b9a5c-4684-4314-ac3c-bd61700f812b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:42.966582 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1055/2175] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:42 2022] POST /identity/v3/projects => generated 399 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:42.967829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac2d79-2e0a-431e-9e98-c9faf84d234f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:42.979488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac2d79-2e0a-431e-9e98-c9faf84d234f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:43.056934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420dcb1-8929-4631-901f-693c3ac491b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:43.057572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420dcb1-8929-4631-901f-693c3ac491b2 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:43.057852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420dcb1-8929-4631-901f-693c3ac491b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:43.058127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420dcb1-8929-4631-901f-693c3ac491b2 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:43.058878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420dcb1-8929-4631-901f-693c3ac491b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:43.060786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420dcb1-8929-4631-901f-693c3ac491b2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:43.093611 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1121/2176] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:42 2022] POST /identity/v3/users => generated 300 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:43.095383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b9a5c-4684-4314-ac3c-bd61700f812b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:43.113739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b9a5c-4684-4314-ac3c-bd61700f812b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:43.136992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304b60b-bf48-45eb-a55a-799872a0ce50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:43.138445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304b60b-bf48-45eb-a55a-799872a0ce50 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:43.138445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304b60b-bf48-45eb-a55a-799872a0ce50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:43.138774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304b60b-bf48-45eb-a55a-799872a0ce50 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:43.139954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304b60b-bf48-45eb-a55a-799872a0ce50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:43.141595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304b60b-bf48-45eb-a55a-799872a0ce50 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:43.159961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420dcb1-8929-4631-901f-693c3ac491b2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:43.163827 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1056/2177] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:43.177957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420dcb1-8929-4631-901f-693c3ac491b2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:43.190107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9ba19-3089-4912-a6ed-d52dd6ec8722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:43.192199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9ba19-3089-4912-a6ed-d52dd6ec8722 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:43.192621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9ba19-3089-4912-a6ed-d52dd6ec8722 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:43.193010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9ba19-3089-4912-a6ed-d52dd6ec8722 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:43.193961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9ba19-3089-4912-a6ed-d52dd6ec8722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:43.195913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9ba19-3089-4912-a6ed-d52dd6ec8722 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:43.209919 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1122/2178] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:43.214122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304b60b-bf48-45eb-a55a-799872a0ce50 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:43.227594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304b60b-bf48-45eb-a55a-799872a0ce50 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:43.232763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be495e71-cb78-475f-8f45-c6f56bf40c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:43.233437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be495e71-cb78-475f-8f45-c6f56bf40c38 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:43.233714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be495e71-cb78-475f-8f45-c6f56bf40c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:43.233982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be495e71-cb78-475f-8f45-c6f56bf40c38 admin admin] PATH_INFO: `/v3/projects/d4cb176cab2a4f9d9c6e33d62dd91ddc/users/fac54082682041db8cb3cf87fdea889f/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:43.270862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be495e71-cb78-475f-8f45-c6f56bf40c38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4cb176cab2a4f9d9c6e33d62dd91ddc, user_id=fac54082682041db8cb3cf87fdea889f, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:43.284200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be495e71-cb78-475f-8f45-c6f56bf40c38 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:43.312027 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1057/2179] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:43 2022] PUT /identity/v3/projects/d4cb176cab2a4f9d9c6e33d62dd91ddc/users/fac54082682041db8cb3cf87fdea889f/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:43.325932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b4dd5-415d-4337-8be3-3fb138b37b29 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:43.326301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b4dd5-415d-4337-8be3-3fb138b37b29 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:43.326688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b4dd5-415d-4337-8be3-3fb138b37b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:43.374781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4e5b4dd5-415d-4337-8be3-3fb138b37b29 None None] MFA Rules not processed for user `fac54082682041db8cb3cf87fdea889f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:43.374781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4e5b4dd5-415d-4337-8be3-3fb138b37b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:43.464134 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1123/2180] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:43 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:43.468414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be495e71-cb78-475f-8f45-c6f56bf40c38 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:43.507172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:43.554397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be495e71-cb78-475f-8f45-c6f56bf40c38 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:43.563908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6af901-0b00-4a77-9335-460e70590f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:43.564234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6af901-0b00-4a77-9335-460e70590f29 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:43.564515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6af901-0b00-4a77-9335-460e70590f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:43.564787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6af901-0b00-4a77-9335-460e70590f29 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:43.565437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6af901-0b00-4a77-9335-460e70590f29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:43.567871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6af901-0b00-4a77-9335-460e70590f29 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:43.618132 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1058/2181] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:43 2022] POST /identity/v3/projects => generated 399 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:49:43.620436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b4dd5-415d-4337-8be3-3fb138b37b29 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:43.634452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b4dd5-415d-4337-8be3-3fb138b37b29 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:43.716461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:43.727697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c0207-0db4-461d-9d7c-a7cba51adab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:43.729270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c0207-0db4-461d-9d7c-a7cba51adab2 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:43.730248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c0207-0db4-461d-9d7c-a7cba51adab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:43.730818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c0207-0db4-461d-9d7c-a7cba51adab2 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:43.731781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c0207-0db4-461d-9d7c-a7cba51adab2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:43.734952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c0207-0db4-461d-9d7c-a7cba51adab2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:43.772468 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1124/2182] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:43 2022] POST /identity/v3/users => generated 306 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:43.773967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6af901-0b00-4a77-9335-460e70590f29 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:43.795653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6af901-0b00-4a77-9335-460e70590f29 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:43.815371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db36ae3-0ad1-489d-a170-01c6d3befbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:43.816005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db36ae3-0ad1-489d-a170-01c6d3befbb4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:43.816288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db36ae3-0ad1-489d-a170-01c6d3befbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:43.816917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db36ae3-0ad1-489d-a170-01c6d3befbb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:43.817722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db36ae3-0ad1-489d-a170-01c6d3befbb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:43.819472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db36ae3-0ad1-489d-a170-01c6d3befbb4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:43.855961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c0207-0db4-461d-9d7c-a7cba51adab2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:43.856918 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1059/2183] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:43.869548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c0207-0db4-461d-9d7c-a7cba51adab2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:43.881589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b125b4c-77d0-49b8-ac61-b01ce253303c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:43.882319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b125b4c-77d0-49b8-ac61-b01ce253303c admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:43.882643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b125b4c-77d0-49b8-ac61-b01ce253303c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:43.882919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b125b4c-77d0-49b8-ac61-b01ce253303c admin admin] PATH_INFO: `/v3/projects/4803d0561d104d7c8298e6782b8cb18d/users/35503a37685d4e11a514195dfaf29f38/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:43.901775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b125b4c-77d0-49b8-ac61-b01ce253303c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4803d0561d104d7c8298e6782b8cb18d, user_id=35503a37685d4e11a514195dfaf29f38, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:43.904578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b125b4c-77d0-49b8-ac61-b01ce253303c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:43.930031 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1125/2184] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:43 2022] PUT /identity/v3/projects/4803d0561d104d7c8298e6782b8cb18d/users/35503a37685d4e11a514195dfaf29f38/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:49:43.932057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db36ae3-0ad1-489d-a170-01c6d3befbb4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:43.948708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db36ae3-0ad1-489d-a170-01c6d3befbb4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:44.072321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf8a9b-bf06-4724-a501-9586aa36967f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:44.073283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf8a9b-bf06-4724-a501-9586aa36967f admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:44.073661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf8a9b-bf06-4724-a501-9586aa36967f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:44.074062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf8a9b-bf06-4724-a501-9586aa36967f admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:44.074890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf8a9b-bf06-4724-a501-9586aa36967f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:44.076868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf8a9b-bf06-4724-a501-9586aa36967f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:44.099702 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1060/2185] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:44.104425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b125b4c-77d0-49b8-ac61-b01ce253303c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:44.126801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b125b4c-77d0-49b8-ac61-b01ce253303c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:44.135861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbeea4-8f4b-474f-84a5-0c92e6538366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:44.136642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbeea4-8f4b-474f-84a5-0c92e6538366 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:44.137026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbeea4-8f4b-474f-84a5-0c92e6538366 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:44.137373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbeea4-8f4b-474f-84a5-0c92e6538366 admin admin] PATH_INFO: `/v3/projects/4803d0561d104d7c8298e6782b8cb18d/users/35503a37685d4e11a514195dfaf29f38/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:44.139521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dbeea4-8f4b-474f-84a5-0c92e6538366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4803d0561d104d7c8298e6782b8cb18d, user_id=35503a37685d4e11a514195dfaf29f38, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:44.142691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dbeea4-8f4b-474f-84a5-0c92e6538366 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:44.173459 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1126/2186] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:44 2022] PUT /identity/v3/projects/4803d0561d104d7c8298e6782b8cb18d/users/35503a37685d4e11a514195dfaf29f38/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:44.176035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e61225-9c6c-4663-9b8d-675f8de71305 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:44.176656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e61225-9c6c-4663-9b8d-675f8de71305 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:44.177010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e61225-9c6c-4663-9b8d-675f8de71305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:44.232212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-20e61225-9c6c-4663-9b8d-675f8de71305 None None] MFA Rules not processed for user `35503a37685d4e11a514195dfaf29f38`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:44.233611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-20e61225-9c6c-4663-9b8d-675f8de71305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:44.385865 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1061/2187] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:44 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:45.161552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbeea4-8f4b-474f-84a5-0c92e6538366 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:45.161552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-76dbeea4-8f4b-474f-84a5-0c92e6538366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:45.440086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbeea4-8f4b-474f-84a5-0c92e6538366 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:45.461006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b04716-30f9-4b2a-b920-548967b2b9c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:45.464965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b04716-30f9-4b2a-b920-548967b2b9c0 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:45.467861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b04716-30f9-4b2a-b920-548967b2b9c0 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:45.468427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b04716-30f9-4b2a-b920-548967b2b9c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:45.485305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b04716-30f9-4b2a-b920-548967b2b9c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:45.487628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b04716-30f9-4b2a-b920-548967b2b9c0 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:45.502616 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1127/2188] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:45 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:48.437720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6e3bf-9a29-46df-98b6-4ede4a05c6de None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:48.438246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6e3bf-9a29-46df-98b6-4ede4a05c6de None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:48.438246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6e3bf-9a29-46df-98b6-4ede4a05c6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:48.503059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-15e6e3bf-9a29-46df-98b6-4ede4a05c6de None None] MFA Rules not processed for user `aa72870453cd4b958725801b842c7a28`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:48.504409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-15e6e3bf-9a29-46df-98b6-4ede4a05c6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:48.518479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:48.661436 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1062/2189] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:48 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:48.662546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b04716-30f9-4b2a-b920-548967b2b9c0 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:48.665240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-91b04716-30f9-4b2a-b920-548967b2b9c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:48.787134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b04716-30f9-4b2a-b920-548967b2b9c0 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:48.795053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d97cf7-29e7-4473-823e-1964da3a1f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:48.795830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d97cf7-29e7-4473-823e-1964da3a1f68 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:48.796271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d97cf7-29e7-4473-823e-1964da3a1f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:48.796551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d97cf7-29e7-4473-823e-1964da3a1f68 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:48.797270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d97cf7-29e7-4473-823e-1964da3a1f68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:48.799445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d97cf7-29e7-4473-823e-1964da3a1f68 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:48.866668 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1128/2190] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:48 2022] POST /identity/v3/projects => generated 397 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:48.870630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6e3bf-9a29-46df-98b6-4ede4a05c6de None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:48.883220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6e3bf-9a29-46df-98b6-4ede4a05c6de None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:48.961640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84200650-cae0-4a52-a037-4fe6f41d78b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:48.962304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84200650-cae0-4a52-a037-4fe6f41d78b2 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:48.962643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84200650-cae0-4a52-a037-4fe6f41d78b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:48.962921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84200650-cae0-4a52-a037-4fe6f41d78b2 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:48.963556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84200650-cae0-4a52-a037-4fe6f41d78b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:48.965862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84200650-cae0-4a52-a037-4fe6f41d78b2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:48.999653 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1063/2191] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:49:48 2022] POST /identity/v3/users => generated 305 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:49:49.003385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d97cf7-29e7-4473-823e-1964da3a1f68 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:49.015401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d97cf7-29e7-4473-823e-1964da3a1f68 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:49.022103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fdc1a-7469-4116-85e3-d4bf538ec903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:49.022776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fdc1a-7469-4116-85e3-d4bf538ec903 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:49.023059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fdc1a-7469-4116-85e3-d4bf538ec903 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:49.023336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fdc1a-7469-4116-85e3-d4bf538ec903 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:49.024081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3fdc1a-7469-4116-85e3-d4bf538ec903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:49.026074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3fdc1a-7469-4116-85e3-d4bf538ec903 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:49.043458 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1129/2192] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:49.057468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84200650-cae0-4a52-a037-4fe6f41d78b2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:49.077844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84200650-cae0-4a52-a037-4fe6f41d78b2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:49.084714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29311686-5d33-4ab3-ac77-983f1dbee4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:49.085432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29311686-5d33-4ab3-ac77-983f1dbee4b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:49.085719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29311686-5d33-4ab3-ac77-983f1dbee4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:49.085998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29311686-5d33-4ab3-ac77-983f1dbee4b6 admin admin] PATH_INFO: `/v3/projects/9e40f93ce5d54331901307bd6ced39a9/users/bddb1c3a5b754575a5e6cd717c7c593f/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:49.106333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29311686-5d33-4ab3-ac77-983f1dbee4b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e40f93ce5d54331901307bd6ced39a9, user_id=bddb1c3a5b754575a5e6cd717c7c593f, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:49.109478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29311686-5d33-4ab3-ac77-983f1dbee4b6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:49.129835 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1064/2193] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:49 2022] PUT /identity/v3/projects/9e40f93ce5d54331901307bd6ced39a9/users/bddb1c3a5b754575a5e6cd717c7c593f/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:49.131206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fdc1a-7469-4116-85e3-d4bf538ec903 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:49.143569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fdc1a-7469-4116-85e3-d4bf538ec903 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:49.148323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:49.207267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957060c-b6a6-4ce7-9792-4e07dd0f13ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:49.207989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957060c-b6a6-4ce7-9792-4e07dd0f13ab admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:49.208341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957060c-b6a6-4ce7-9792-4e07dd0f13ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:49.208688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957060c-b6a6-4ce7-9792-4e07dd0f13ab admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:49.209405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957060c-b6a6-4ce7-9792-4e07dd0f13ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:49.211106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957060c-b6a6-4ce7-9792-4e07dd0f13ab admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:49.221129 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1130/2194] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:49:49.225457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29311686-5d33-4ab3-ac77-983f1dbee4b6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:49.235861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29311686-5d33-4ab3-ac77-983f1dbee4b6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:49.240657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d421886-17c5-4c64-ae37-a6aa33efae48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:49.241301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d421886-17c5-4c64-ae37-a6aa33efae48 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:49.241574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d421886-17c5-4c64-ae37-a6aa33efae48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:49.241845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d421886-17c5-4c64-ae37-a6aa33efae48 admin admin] PATH_INFO: `/v3/projects/9e40f93ce5d54331901307bd6ced39a9/users/bddb1c3a5b754575a5e6cd717c7c593f/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:49.244437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d421886-17c5-4c64-ae37-a6aa33efae48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e40f93ce5d54331901307bd6ced39a9, user_id=bddb1c3a5b754575a5e6cd717c7c593f, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:49.247096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d421886-17c5-4c64-ae37-a6aa33efae48 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:49.257331 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1065/2195] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:49:49 2022] PUT /identity/v3/projects/9e40f93ce5d54331901307bd6ced39a9/users/bddb1c3a5b754575a5e6cd717c7c593f/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:49.686533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957060c-b6a6-4ce7-9792-4e07dd0f13ab admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:49.815020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957060c-b6a6-4ce7-9792-4e07dd0f13ab admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:49.896127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8de59-3b24-42e9-ac01-c84699905408 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:49.897286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8de59-3b24-42e9-ac01-c84699905408 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:49.897810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8de59-3b24-42e9-ac01-c84699905408 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:49.898153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8de59-3b24-42e9-ac01-c84699905408 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:49.947793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:49.967457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8de59-3b24-42e9-ac01-c84699905408 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:49.969644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8de59-3b24-42e9-ac01-c84699905408 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:49.981200 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1131/2196] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:49 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:51.534591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28ecef-8175-4d76-af12-9371162d7aa7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:51.537219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28ecef-8175-4d76-af12-9371162d7aa7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:51.538072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28ecef-8175-4d76-af12-9371162d7aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:51.607511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fe28ecef-8175-4d76-af12-9371162d7aa7 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:51.611997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fe28ecef-8175-4d76-af12-9371162d7aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:51.739966 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1066/2197] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:49:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:49:51.760760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8de59-3b24-42e9-ac01-c84699905408 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:51.762850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-48d8de59-3b24-42e9-ac01-c84699905408 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:51.863170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8de59-3b24-42e9-ac01-c84699905408 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:51.870126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da538a-ed18-40fe-bc5f-9ca3219ab8be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:51.870817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da538a-ed18-40fe-bc5f-9ca3219ab8be service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:51.871094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da538a-ed18-40fe-bc5f-9ca3219ab8be service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:51.871530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da538a-ed18-40fe-bc5f-9ca3219ab8be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:51.881641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da538a-ed18-40fe-bc5f-9ca3219ab8be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:51.883973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da538a-ed18-40fe-bc5f-9ca3219ab8be service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:51.894732 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1132/2198] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:49:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 20:49:56.670673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f597b-e890-4275-8b29-14672fbadaaf None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:56.671197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f597b-e890-4275-8b29-14672fbadaaf None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:56.671290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f597b-e890-4275-8b29-14672fbadaaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:56.690981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:56.726778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-443f597b-e890-4275-8b29-14672fbadaaf None None] MFA Rules not processed for user `bddb1c3a5b754575a5e6cd717c7c593f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:56.730923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-443f597b-e890-4275-8b29-14672fbadaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:56.731896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da538a-ed18-40fe-bc5f-9ca3219ab8be service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:56.733833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-37da538a-ed18-40fe-bc5f-9ca3219ab8be service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:56.931661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da538a-ed18-40fe-bc5f-9ca3219ab8be service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:56.951235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89c207-e2e2-4014-97f3-7b250cf85b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:56.952285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89c207-e2e2-4014-97f3-7b250cf85b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:56.952817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89c207-e2e2-4014-97f3-7b250cf85b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:56.953311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89c207-e2e2-4014-97f3-7b250cf85b9b admin admin] PATH_INFO: `/v3/users/7ef35c15f9d541ac90239c3713f9f5e6` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:56.959332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89c207-e2e2-4014-97f3-7b250cf85b9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ef35c15f9d541ac90239c3713f9f5e6)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:56.962176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89c207-e2e2-4014-97f3-7b250cf85b9b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:56.969233 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1067/2199] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:56 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:56.970918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f4112-5efd-4afb-be26-c7b38300616a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:56.971208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f4112-5efd-4afb-be26-c7b38300616a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:56.971510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f4112-5efd-4afb-be26-c7b38300616a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:57.063775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-581f4112-5efd-4afb-be26-c7b38300616a None None] MFA Rules not processed for user `aa72870453cd4b958725801b842c7a28`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:57.069300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-581f4112-5efd-4afb-be26-c7b38300616a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:57.125457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a89c207-e2e2-4014-97f3-7b250cf85b9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ef35c15f9d541ac90239c3713f9f5e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:57.151484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a89c207-e2e2-4014-97f3-7b250cf85b9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ef35c15f9d541ac90239c3713f9f5e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:57.268414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:57.275742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a89c207-e2e2-4014-97f3-7b250cf85b9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ef35c15f9d541ac90239c3713f9f5e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:57.277942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a89c207-e2e2-4014-97f3-7b250cf85b9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ef35c15f9d541ac90239c3713f9f5e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:57.297334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a89c207-e2e2-4014-97f3-7b250cf85b9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ef35c15f9d541ac90239c3713f9f5e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:57.325374 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1133/2200] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:56 2022] DELETE /identity/v3/users/7ef35c15f9d541ac90239c3713f9f5e6 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:57.339857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb656448-c418-4605-893a-8e5a35495a38 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:57.340163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb656448-c418-4605-893a-8e5a35495a38 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:57.340480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb656448-c418-4605-893a-8e5a35495a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:57.363956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-eb656448-c418-4605-893a-8e5a35495a38 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:57.367528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-eb656448-c418-4605-893a-8e5a35495a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:57.379569 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1068/2201] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:56 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:57.420711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f4112-5efd-4afb-be26-c7b38300616a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:57.422379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-581f4112-5efd-4afb-be26-c7b38300616a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:57.455964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f4112-5efd-4afb-be26-c7b38300616a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:57.488915 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1134/2202] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:57.501048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb656448-c418-4605-893a-8e5a35495a38 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:57.502703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-eb656448-c418-4605-893a-8e5a35495a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:57.654734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1e3d9-bfd9-4e55-a2a3-27bc27d3bb16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:57.655390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1e3d9-bfd9-4e55-a2a3-27bc27d3bb16 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:57.655666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1e3d9-bfd9-4e55-a2a3-27bc27d3bb16 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:57.655945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1e3d9-bfd9-4e55-a2a3-27bc27d3bb16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:57.656774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb656448-c418-4605-893a-8e5a35495a38 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:57.659352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6eb1e3d9-bfd9-4e55-a2a3-27bc27d3bb16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:57.682599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8168cc-9de5-4a28-9947-eab75f99db5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:57.683147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8168cc-9de5-4a28-9947-eab75f99db5f service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:57.684117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8168cc-9de5-4a28-9947-eab75f99db5f service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:57.684410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8168cc-9de5-4a28-9947-eab75f99db5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:57.698929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8168cc-9de5-4a28-9947-eab75f99db5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:57.703318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8168cc-9de5-4a28-9947-eab75f99db5f service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:57.729135 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1135/2203] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:49:57.745540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:57.812030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1e3d9-bfd9-4e55-a2a3-27bc27d3bb16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:57.814121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1e3d9-bfd9-4e55-a2a3-27bc27d3bb16 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:57.840951 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1069/2204] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:57 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:49:58.159444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8168cc-9de5-4a28-9947-eab75f99db5f service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:58.161797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5a8168cc-9de5-4a28-9947-eab75f99db5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:58.180063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8168cc-9de5-4a28-9947-eab75f99db5f service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:58.188432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a69fb-f410-49bb-a8f0-1e14d258cacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:58.189192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a69fb-f410-49bb-a8f0-1e14d258cacc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:58.189488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a69fb-f410-49bb-a8f0-1e14d258cacc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:58.189763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a69fb-f410-49bb-a8f0-1e14d258cacc admin admin] PATH_INFO: `/v3/projects/e93dcf30e02b41178c6cde6e50a41e17` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:58.194372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a69fb-f410-49bb-a8f0-1e14d258cacc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e93dcf30e02b41178c6cde6e50a41e17)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:58.196490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a69fb-f410-49bb-a8f0-1e14d258cacc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:58.274987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-226a69fb-f410-49bb-a8f0-1e14d258cacc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e93dcf30e02b41178c6cde6e50a41e17', 'request_id': 'req-226a69fb-f410-49bb-a8f0-1e14d258cacc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:58.306471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-226a69fb-f410-49bb-a8f0-1e14d258cacc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e93dcf30e02b41178c6cde6e50a41e17', 'request_id': 'req-226a69fb-f410-49bb-a8f0-1e14d258cacc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:58.313602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-226a69fb-f410-49bb-a8f0-1e14d258cacc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e93dcf30e02b41178c6cde6e50a41e17', 'request_id': 'req-226a69fb-f410-49bb-a8f0-1e14d258cacc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:58.338806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-226a69fb-f410-49bb-a8f0-1e14d258cacc admin admin] The token cache is being invalidate because project e93dcf30e02b41178c6cde6e50a41e17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:58.339180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-226a69fb-f410-49bb-a8f0-1e14d258cacc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:58.343726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-226a69fb-f410-49bb-a8f0-1e14d258cacc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:58.350773 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1136/2205] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:58 2022] DELETE /identity/v3/projects/e93dcf30e02b41178c6cde6e50a41e17 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:58.355658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1e3d9-bfd9-4e55-a2a3-27bc27d3bb16 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:58.372370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1e3d9-bfd9-4e55-a2a3-27bc27d3bb16 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:58.560829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c53dc-c6fe-4936-a34d-5280726c4699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:58.561477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c53dc-c6fe-4936-a34d-5280726c4699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:58.561755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c53dc-c6fe-4936-a34d-5280726c4699 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:58.562054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c53dc-c6fe-4936-a34d-5280726c4699 admin admin] PATH_INFO: `/v3/users/4697a57e25c14284ae92262df0793cb5` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:58.563193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c53dc-c6fe-4936-a34d-5280726c4699 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4697a57e25c14284ae92262df0793cb5)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:58.565906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c53dc-c6fe-4936-a34d-5280726c4699 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:58.646943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-354c53dc-c6fe-4936-a34d-5280726c4699 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4697a57e25c14284ae92262df0793cb5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:58.650222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-354c53dc-c6fe-4936-a34d-5280726c4699 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4697a57e25c14284ae92262df0793cb5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:58.650912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-354c53dc-c6fe-4936-a34d-5280726c4699 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4697a57e25c14284ae92262df0793cb5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:58.663292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-354c53dc-c6fe-4936-a34d-5280726c4699 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4697a57e25c14284ae92262df0793cb5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:58.681265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-354c53dc-c6fe-4936-a34d-5280726c4699 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4697a57e25c14284ae92262df0793cb5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:58.697012 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1070/2206] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:49:58 2022] DELETE /identity/v3/users/4697a57e25c14284ae92262df0793cb5 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:49:59.106329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5503e-a819-46b3-8bc6-f150eab0a9cc None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:59.108014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5503e-a819-46b3-8bc6-f150eab0a9cc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:59.108014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5503e-a819-46b3-8bc6-f150eab0a9cc None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:59.109310 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1137/2207] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:49:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:49:59.116152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b20260-aeab-4bb5-8e80-5b5a28d64a9d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:59.116664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b20260-aeab-4bb5-8e80-5b5a28d64a9d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:59.116768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b20260-aeab-4bb5-8e80-5b5a28d64a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:59.174005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5503e-a819-46b3-8bc6-f150eab0a9cc None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:59.176454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d5f5503e-a819-46b3-8bc6-f150eab0a9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:59.206992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-71b20260-aeab-4bb5-8e80-5b5a28d64a9d None None] MFA Rules not processed for user `e17a026f305248878f18eec35e1e02dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:59.213095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:59.214036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-71b20260-aeab-4bb5-8e80-5b5a28d64a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:59.327351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5503e-a819-46b3-8bc6-f150eab0a9cc None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:59.333093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820701e-ed0c-4459-b9ee-21e629a65e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:59.333918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820701e-ed0c-4459-b9ee-21e629a65e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:59.334338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820701e-ed0c-4459-b9ee-21e629a65e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:59.334965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820701e-ed0c-4459-b9ee-21e629a65e7b admin admin] PATH_INFO: `/v3/projects/a10db1d5e60c44a6819b67914849b844` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:59.336897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c820701e-ed0c-4459-b9ee-21e629a65e7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a10db1d5e60c44a6819b67914849b844)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:59.338996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c820701e-ed0c-4459-b9ee-21e629a65e7b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:59.374647 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1071/2208] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:49:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:59.431790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b20260-aeab-4bb5-8e80-5b5a28d64a9d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:59.433222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-71b20260-aeab-4bb5-8e80-5b5a28d64a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:59.441740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:59.467597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c820701e-ed0c-4459-b9ee-21e629a65e7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a10db1d5e60c44a6819b67914849b844', 'request_id': 'req-c820701e-ed0c-4459-b9ee-21e629a65e7b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:59.472858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b20260-aeab-4bb5-8e80-5b5a28d64a9d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:59.495609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c820701e-ed0c-4459-b9ee-21e629a65e7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a10db1d5e60c44a6819b67914849b844', 'request_id': 'req-c820701e-ed0c-4459-b9ee-21e629a65e7b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:59.509510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c820701e-ed0c-4459-b9ee-21e629a65e7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a10db1d5e60c44a6819b67914849b844', 'request_id': 'req-c820701e-ed0c-4459-b9ee-21e629a65e7b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:59.547841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c820701e-ed0c-4459-b9ee-21e629a65e7b admin admin] The token cache is being invalidate because project a10db1d5e60c44a6819b67914849b844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:49:59.550716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c820701e-ed0c-4459-b9ee-21e629a65e7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:59.553116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c820701e-ed0c-4459-b9ee-21e629a65e7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:49:59.558432 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1138/2209] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:49:59 2022] DELETE /identity/v3/projects/a10db1d5e60c44a6819b67914849b844 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:49:59.570160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9c834-ca33-44e1-81c5-6522290b818e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:59.570846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9c834-ca33-44e1-81c5-6522290b818e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:59.572447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9c834-ca33-44e1-81c5-6522290b818e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:59.588174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca289a8-bedd-4d47-b182-8fb9e3436751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:59.588844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca289a8-bedd-4d47-b182-8fb9e3436751 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:59.589163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca289a8-bedd-4d47-b182-8fb9e3436751 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:59.589437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca289a8-bedd-4d47-b182-8fb9e3436751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:59.596334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3ca289a8-bedd-4d47-b182-8fb9e3436751 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:59.604485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-83a9c834-ca33-44e1-81c5-6522290b818e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:59.609802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-83a9c834-ca33-44e1-81c5-6522290b818e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:59.625689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca289a8-bedd-4d47-b182-8fb9e3436751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:59.627302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca289a8-bedd-4d47-b182-8fb9e3436751 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:59.657824 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1072/2210] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:49:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:49:59.702715 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1139/2211] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:59.707692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee5fef-2ebe-4e3c-9559-73aebf918b45 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:59.708406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee5fef-2ebe-4e3c-9559-73aebf918b45 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:59.708683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee5fef-2ebe-4e3c-9559-73aebf918b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:59.736163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0fee5fef-2ebe-4e3c-9559-73aebf918b45 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:49:59.736342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0fee5fef-2ebe-4e3c-9559-73aebf918b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:49:59.786635 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1073/2212] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:49:59.787141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9c834-ca33-44e1-81c5-6522290b818e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:59.817388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9c834-ca33-44e1-81c5-6522290b818e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:59.847424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c53d0d-6eb6-4048-a468-76fd1699f6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:59.848071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c53d0d-6eb6-4048-a468-76fd1699f6ed admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:59.848355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c53d0d-6eb6-4048-a468-76fd1699f6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:59.848682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c53d0d-6eb6-4048-a468-76fd1699f6ed admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:59.849301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c53d0d-6eb6-4048-a468-76fd1699f6ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:49:59.849607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c53d0d-6eb6-4048-a468-76fd1699f6ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:59.851331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c53d0d-6eb6-4048-a468-76fd1699f6ed admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:49:59.894652 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1140/2213] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:49:59.908811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee5fef-2ebe-4e3c-9559-73aebf918b45 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:49:59.927205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee5fef-2ebe-4e3c-9559-73aebf918b45 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:49:59.950672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b79ae-0d68-42fd-964d-79726f08aa25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:49:59.951321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b79ae-0d68-42fd-964d-79726f08aa25 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:49:59.951608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b79ae-0d68-42fd-964d-79726f08aa25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:49:59.952207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b79ae-0d68-42fd-964d-79726f08aa25 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:49:59.952532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b79ae-0d68-42fd-964d-79726f08aa25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:49:59.954620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b79ae-0d68-42fd-964d-79726f08aa25 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:00.019204 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1074/2214] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:49:59 2022] POST /identity/v3/projects => generated 409 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:00.030043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c53d0d-6eb6-4048-a468-76fd1699f6ed admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:00.059606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c53d0d-6eb6-4048-a468-76fd1699f6ed admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:00.119642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:00.198016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01e306-cc57-4f7c-aa7a-715f1285c070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:00.199334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01e306-cc57-4f7c-aa7a-715f1285c070 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:00.200636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01e306-cc57-4f7c-aa7a-715f1285c070 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:00.203538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01e306-cc57-4f7c-aa7a-715f1285c070 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:00.206135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01e306-cc57-4f7c-aa7a-715f1285c070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:00.210167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01e306-cc57-4f7c-aa7a-715f1285c070 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:00.267401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940567a-4f07-47e3-80bf-4d0585e4413b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:00.272851 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1141/2215] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:50:00 2022] POST /identity/v3/users => generated 305 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:50:00.273028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940567a-4f07-47e3-80bf-4d0585e4413b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:00.273028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940567a-4f07-47e3-80bf-4d0585e4413b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:00.274291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01e306-cc57-4f7c-aa7a-715f1285c070 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:00.303332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01e306-cc57-4f7c-aa7a-715f1285c070 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:00.317627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f940567a-4f07-47e3-80bf-4d0585e4413b None None] MFA Rules not processed for user `f40f50783d9f417a8f84fab11b0d10d0`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:00.318527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebfd0c9-aee0-4c70-a6a1-f0a2b6c462aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:00.319410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfd0c9-aee0-4c70-a6a1-f0a2b6c462aa admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:00.319782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfd0c9-aee0-4c70-a6a1-f0a2b6c462aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:00.320152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfd0c9-aee0-4c70-a6a1-f0a2b6c462aa admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:00.322960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f940567a-4f07-47e3-80bf-4d0585e4413b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:00.323342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebfd0c9-aee0-4c70-a6a1-f0a2b6c462aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:00.325275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebfd0c9-aee0-4c70-a6a1-f0a2b6c462aa admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:00.356139 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1142/2216] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:00.357400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebfd0c9-aee0-4c70-a6a1-f0a2b6c462aa admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:00.391502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebfd0c9-aee0-4c70-a6a1-f0a2b6c462aa admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:00.402605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56254865-0c03-4f53-bcae-13b913fe7fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:00.404914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56254865-0c03-4f53-bcae-13b913fe7fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:00.404914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56254865-0c03-4f53-bcae-13b913fe7fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:00.405147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56254865-0c03-4f53-bcae-13b913fe7fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:00.410150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56254865-0c03-4f53-bcae-13b913fe7fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:00.412676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56254865-0c03-4f53-bcae-13b913fe7fe5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:00.442644 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1143/2217] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:00.443813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56254865-0c03-4f53-bcae-13b913fe7fe5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:00.465602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56254865-0c03-4f53-bcae-13b913fe7fe5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:00.476392 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1075/2218] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:00 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:00.487406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9589bd-36ff-4a93-b0a3-6f2c883b8a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:00.488097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9589bd-36ff-4a93-b0a3-6f2c883b8a91 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:00.489465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9589bd-36ff-4a93-b0a3-6f2c883b8a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:00.489756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9589bd-36ff-4a93-b0a3-6f2c883b8a91 admin admin] PATH_INFO: `/v3/projects/7f1f2ee4738b4b74bbcb7c685d12feb4/users/5df768a2d0f64f08953559660694beef/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:00.518411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9589bd-36ff-4a93-b0a3-6f2c883b8a91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f1f2ee4738b4b74bbcb7c685d12feb4, user_id=5df768a2d0f64f08953559660694beef, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:00.524467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9589bd-36ff-4a93-b0a3-6f2c883b8a91 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:00.561279 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1144/2219] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:00 2022] PUT /identity/v3/projects/7f1f2ee4738b4b74bbcb7c685d12feb4/users/5df768a2d0f64f08953559660694beef/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:00.562821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229a599-46af-4d5f-ad97-4781479942b0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:00.563271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229a599-46af-4d5f-ad97-4781479942b0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:00.563779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229a599-46af-4d5f-ad97-4781479942b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:00.622412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4229a599-46af-4d5f-ad97-4781479942b0 None None] MFA Rules not processed for user `5df768a2d0f64f08953559660694beef`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:00.628912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4229a599-46af-4d5f-ad97-4781479942b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:00.774901 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1076/2220] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:00 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:00.779374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9589bd-36ff-4a93-b0a3-6f2c883b8a91 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:00.949451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9589bd-36ff-4a93-b0a3-6f2c883b8a91 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:00.962185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e03a88-9f29-431f-81af-41d97687d685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:00.963591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e03a88-9f29-431f-81af-41d97687d685 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:00.963951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e03a88-9f29-431f-81af-41d97687d685 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:00.964851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e03a88-9f29-431f-81af-41d97687d685 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:00.965899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e03a88-9f29-431f-81af-41d97687d685 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:00.970394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e03a88-9f29-431f-81af-41d97687d685 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:01.042079 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1145/2221] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:50:00 2022] POST /identity/v3/projects => generated 409 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:01.047172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229a599-46af-4d5f-ad97-4781479942b0 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:01.073708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229a599-46af-4d5f-ad97-4781479942b0 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:01.111291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:01.204262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed47ec-35d1-417b-965b-16a79831d062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:01.205040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed47ec-35d1-417b-965b-16a79831d062 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:01.205311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed47ec-35d1-417b-965b-16a79831d062 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:01.205598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed47ec-35d1-417b-965b-16a79831d062 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:01.206197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed47ec-35d1-417b-965b-16a79831d062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:01.208215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed47ec-35d1-417b-965b-16a79831d062 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:01.247759 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1077/2222] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:50:01 2022] POST /identity/v3/users => generated 311 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:01.252146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e03a88-9f29-431f-81af-41d97687d685 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:01.281454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e03a88-9f29-431f-81af-41d97687d685 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:01.299240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d1560-10b0-4219-8b01-661513e4696c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:01.300204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d1560-10b0-4219-8b01-661513e4696c admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:01.300690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d1560-10b0-4219-8b01-661513e4696c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:01.301065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d1560-10b0-4219-8b01-661513e4696c admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:01.335478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d1560-10b0-4219-8b01-661513e4696c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:01.339257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d1560-10b0-4219-8b01-661513e4696c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:01.364472 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1146/2223] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:01.369512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed47ec-35d1-417b-965b-16a79831d062 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:01.383834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed47ec-35d1-417b-965b-16a79831d062 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:01.395166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5df9ee-ac2f-4676-a8a1-9172f62efc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:01.401111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5df9ee-ac2f-4676-a8a1-9172f62efc70 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:01.401397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5df9ee-ac2f-4676-a8a1-9172f62efc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:01.401675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5df9ee-ac2f-4676-a8a1-9172f62efc70 admin admin] PATH_INFO: `/v3/projects/cb7f6c9003734fcaac1124e7879370d7/users/3dd3804e3e0a43ffa8fda4161c7b77c7/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:01.474842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5df9ee-ac2f-4676-a8a1-9172f62efc70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb7f6c9003734fcaac1124e7879370d7, user_id=3dd3804e3e0a43ffa8fda4161c7b77c7, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:01.477908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5df9ee-ac2f-4676-a8a1-9172f62efc70 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:01.515482 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1078/2224] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:01 2022] PUT /identity/v3/projects/cb7f6c9003734fcaac1124e7879370d7/users/3dd3804e3e0a43ffa8fda4161c7b77c7/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:01.518958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d1560-10b0-4219-8b01-661513e4696c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:01.543002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d1560-10b0-4219-8b01-661513e4696c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:01.696371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:01.729725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39e57d-7f75-46ad-8377-cfe133415425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:01.732298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39e57d-7f75-46ad-8377-cfe133415425 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:01.732583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39e57d-7f75-46ad-8377-cfe133415425 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:01.733058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39e57d-7f75-46ad-8377-cfe133415425 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:01.733739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc39e57d-7f75-46ad-8377-cfe133415425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:01.742805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc39e57d-7f75-46ad-8377-cfe133415425 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:01.765997 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1147/2225] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:50:01.770878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5df9ee-ac2f-4676-a8a1-9172f62efc70 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:01.815517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5df9ee-ac2f-4676-a8a1-9172f62efc70 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:01.861516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d8001-5e7f-41a6-acf2-202f98af1542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:01.862248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d8001-5e7f-41a6-acf2-202f98af1542 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:01.862721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d8001-5e7f-41a6-acf2-202f98af1542 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:01.863491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d8001-5e7f-41a6-acf2-202f98af1542 admin admin] PATH_INFO: `/v3/projects/cb7f6c9003734fcaac1124e7879370d7/users/3dd3804e3e0a43ffa8fda4161c7b77c7/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:01.873620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d8001-5e7f-41a6-acf2-202f98af1542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb7f6c9003734fcaac1124e7879370d7, user_id=3dd3804e3e0a43ffa8fda4161c7b77c7, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:01.883469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d8001-5e7f-41a6-acf2-202f98af1542 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:01.942681 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1079/2226] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:01 2022] PUT /identity/v3/projects/cb7f6c9003734fcaac1124e7879370d7/users/3dd3804e3e0a43ffa8fda4161c7b77c7/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:01.953045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4526e-b48f-4d97-b2d1-6fae3ca19056 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:01.953693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4526e-b48f-4d97-b2d1-6fae3ca19056 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:01.954347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4526e-b48f-4d97-b2d1-6fae3ca19056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:02.037328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ccb4526e-b48f-4d97-b2d1-6fae3ca19056 None None] MFA Rules not processed for user `3dd3804e3e0a43ffa8fda4161c7b77c7`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:02.050136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ccb4526e-b48f-4d97-b2d1-6fae3ca19056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:02.232172 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1148/2227] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:01 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:02.834232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d8001-5e7f-41a6-acf2-202f98af1542 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:02.837286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ac7d8001-5e7f-41a6-acf2-202f98af1542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:03.038085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d8001-5e7f-41a6-acf2-202f98af1542 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:03.058027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b4824b-63dd-48f4-9539-b3e6aa1318c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:03.059111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4824b-63dd-48f4-9539-b3e6aa1318c9 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:03.059490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4824b-63dd-48f4-9539-b3e6aa1318c9 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:03.059874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4824b-63dd-48f4-9539-b3e6aa1318c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:03.078983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b4824b-63dd-48f4-9539-b3e6aa1318c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:03.081024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b4824b-63dd-48f4-9539-b3e6aa1318c9 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:03.096561 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1080/2228] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:02 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:04.353088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4526e-b48f-4d97-b2d1-6fae3ca19056 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:04.370431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4526e-b48f-4d97-b2d1-6fae3ca19056 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:04.379789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ec96d-1fcc-4845-b3df-f4cdbef950cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:04.380664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ec96d-1fcc-4845-b3df-f4cdbef950cb service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:04.381090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ec96d-1fcc-4845-b3df-f4cdbef950cb service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:04.381526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ec96d-1fcc-4845-b3df-f4cdbef950cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:04.466429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ec96d-1fcc-4845-b3df-f4cdbef950cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:04.468329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ec96d-1fcc-4845-b3df-f4cdbef950cb service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:04.482121 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1149/2229] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:04 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:04.843642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b4824b-63dd-48f4-9539-b3e6aa1318c9 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:04.882213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b4824b-63dd-48f4-9539-b3e6aa1318c9 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:04.903808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33330029-29b5-4e58-9709-31f53f6b42e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:04.904351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33330029-29b5-4e58-9709-31f53f6b42e4 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:04.904797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33330029-29b5-4e58-9709-31f53f6b42e4 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:04.905216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33330029-29b5-4e58-9709-31f53f6b42e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:04.909694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-33330029-29b5-4e58-9709-31f53f6b42e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:05.036283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:05.104227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33330029-29b5-4e58-9709-31f53f6b42e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:05.111859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33330029-29b5-4e58-9709-31f53f6b42e4 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:05.124468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ec96d-1fcc-4845-b3df-f4cdbef950cb service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:05.154298 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1081/2230] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:05.296044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ec96d-1fcc-4845-b3df-f4cdbef950cb service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:05.305031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2846e1-6537-4c30-8e16-9c6c50617fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:05.309942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2846e1-6537-4c30-8e16-9c6c50617fd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:05.309942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2846e1-6537-4c30-8e16-9c6c50617fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:05.309942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2846e1-6537-4c30-8e16-9c6c50617fd6 admin admin] PATH_INFO: `/v3/users/5df768a2d0f64f08953559660694beef` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:05.312438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2846e1-6537-4c30-8e16-9c6c50617fd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5df768a2d0f64f08953559660694beef)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:05.322784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2846e1-6537-4c30-8e16-9c6c50617fd6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:05.349193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:05.499879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0a2846e1-6537-4c30-8e16-9c6c50617fd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5df768a2d0f64f08953559660694beef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:05.542582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0a2846e1-6537-4c30-8e16-9c6c50617fd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5df768a2d0f64f08953559660694beef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:05.591182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0a2846e1-6537-4c30-8e16-9c6c50617fd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5df768a2d0f64f08953559660694beef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:05.592386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0a2846e1-6537-4c30-8e16-9c6c50617fd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5df768a2d0f64f08953559660694beef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:05.607819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0a2846e1-6537-4c30-8e16-9c6c50617fd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5df768a2d0f64f08953559660694beef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:05.621687 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1150/2231] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:50:05 2022] DELETE /identity/v3/users/5df768a2d0f64f08953559660694beef => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:50:05.626004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542108e6-3442-4502-890e-14a6a2e86f98 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:05.626290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542108e6-3442-4502-890e-14a6a2e86f98 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:05.626591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542108e6-3442-4502-890e-14a6a2e86f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:05.694224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-542108e6-3442-4502-890e-14a6a2e86f98 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:05.706527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-542108e6-3442-4502-890e-14a6a2e86f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:06.159473 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1082/2232] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 534 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:06.183547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2846e1-6537-4c30-8e16-9c6c50617fd6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:06.185214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0a2846e1-6537-4c30-8e16-9c6c50617fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:06.307690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2846e1-6537-4c30-8e16-9c6c50617fd6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:06.327839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad9bef-74e2-4599-996e-ee152dc9cab9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:06.328486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad9bef-74e2-4599-996e-ee152dc9cab9 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:06.328771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad9bef-74e2-4599-996e-ee152dc9cab9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:06.329047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad9bef-74e2-4599-996e-ee152dc9cab9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:06.400476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad9bef-74e2-4599-996e-ee152dc9cab9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:06.406113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad9bef-74e2-4599-996e-ee152dc9cab9 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:06.452616 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1151/2233] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:07.939113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542108e6-3442-4502-890e-14a6a2e86f98 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:07.940587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-542108e6-3442-4502-890e-14a6a2e86f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:07.997221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542108e6-3442-4502-890e-14a6a2e86f98 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:08.012626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e52f3-4685-406f-940f-816e350d870f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:08.014620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e52f3-4685-406f-940f-816e350d870f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:08.015125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e52f3-4685-406f-940f-816e350d870f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:08.015704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e52f3-4685-406f-940f-816e350d870f admin admin] PATH_INFO: `/v3/projects/7f1f2ee4738b4b74bbcb7c685d12feb4` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:08.020425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e52f3-4685-406f-940f-816e350d870f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f1f2ee4738b4b74bbcb7c685d12feb4)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:08.037880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e52f3-4685-406f-940f-816e350d870f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:08.173598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1d4e52f3-4685-406f-940f-816e350d870f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f1f2ee4738b4b74bbcb7c685d12feb4', 'request_id': 'req-1d4e52f3-4685-406f-940f-816e350d870f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:08.193099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1d4e52f3-4685-406f-940f-816e350d870f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f1f2ee4738b4b74bbcb7c685d12feb4', 'request_id': 'req-1d4e52f3-4685-406f-940f-816e350d870f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:08.204448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1d4e52f3-4685-406f-940f-816e350d870f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f1f2ee4738b4b74bbcb7c685d12feb4', 'request_id': 'req-1d4e52f3-4685-406f-940f-816e350d870f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:08.233495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1d4e52f3-4685-406f-940f-816e350d870f admin admin] The token cache is being invalidate because project 7f1f2ee4738b4b74bbcb7c685d12feb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:50:08.233792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1d4e52f3-4685-406f-940f-816e350d870f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:08.235628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1d4e52f3-4685-406f-940f-816e350d870f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:08.243648 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1083/2234] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:50:07 2022] DELETE /identity/v3/projects/7f1f2ee4738b4b74bbcb7c685d12feb4 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:08.244870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad9bef-74e2-4599-996e-ee152dc9cab9 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:08.264753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad9bef-74e2-4599-996e-ee152dc9cab9 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:08.381830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a42fc7-0720-4b23-9bbd-f4a0fbe25129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:08.382621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a42fc7-0720-4b23-9bbd-f4a0fbe25129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:08.384298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a42fc7-0720-4b23-9bbd-f4a0fbe25129 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:08.389703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a42fc7-0720-4b23-9bbd-f4a0fbe25129 admin admin] PATH_INFO: `/v3/users/3dd3804e3e0a43ffa8fda4161c7b77c7` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:08.390392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a42fc7-0720-4b23-9bbd-f4a0fbe25129 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dd3804e3e0a43ffa8fda4161c7b77c7)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:08.392637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a42fc7-0720-4b23-9bbd-f4a0fbe25129 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:08.468407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:08.490688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-67a42fc7-0720-4b23-9bbd-f4a0fbe25129 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dd3804e3e0a43ffa8fda4161c7b77c7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:08.506174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-67a42fc7-0720-4b23-9bbd-f4a0fbe25129 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dd3804e3e0a43ffa8fda4161c7b77c7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:08.535997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-67a42fc7-0720-4b23-9bbd-f4a0fbe25129 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dd3804e3e0a43ffa8fda4161c7b77c7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:08.538470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-67a42fc7-0720-4b23-9bbd-f4a0fbe25129 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dd3804e3e0a43ffa8fda4161c7b77c7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:08.556992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-67a42fc7-0720-4b23-9bbd-f4a0fbe25129 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dd3804e3e0a43ffa8fda4161c7b77c7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:08.571052 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1152/2235] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:50:08 2022] DELETE /identity/v3/users/3dd3804e3e0a43ffa8fda4161c7b77c7 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:10.023204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe67e99-4c35-45f4-8749-092cb40c566c None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:10.024303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe67e99-4c35-45f4-8749-092cb40c566c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:10.024303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe67e99-4c35-45f4-8749-092cb40c566c None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:10.025830 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1084/2236] 10.222.0.17 () {54 vars in 921 bytes} [Sat Oct 1 20:50:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:50:10.038152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e27459-fb6e-46a9-91b4-9538e6ebd88e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:10.039720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e27459-fb6e-46a9-91b4-9538e6ebd88e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:10.040140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e27459-fb6e-46a9-91b4-9538e6ebd88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:10.102620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-16e27459-fb6e-46a9-91b4-9538e6ebd88e None None] MFA Rules not processed for user `e17a026f305248878f18eec35e1e02dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:10.103947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-16e27459-fb6e-46a9-91b4-9538e6ebd88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:10.232794 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1153/2237] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 20:50:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:10.257335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe67e99-4c35-45f4-8749-092cb40c566c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:10.261219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-abe67e99-4c35-45f4-8749-092cb40c566c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:10.285509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe67e99-4c35-45f4-8749-092cb40c566c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:10.303230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98333a09-3f25-4592-9dbd-e465a9da1ff0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:10.304385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98333a09-3f25-4592-9dbd-e465a9da1ff0 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:10.304923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98333a09-3f25-4592-9dbd-e465a9da1ff0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:10.305342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98333a09-3f25-4592-9dbd-e465a9da1ff0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:10.322936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98333a09-3f25-4592-9dbd-e465a9da1ff0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:10.326283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98333a09-3f25-4592-9dbd-e465a9da1ff0 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:10.338270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e27459-fb6e-46a9-91b4-9538e6ebd88e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:10.342302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-16e27459-fb6e-46a9-91b4-9538e6ebd88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:10.393080 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1085/2238] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:10.395063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cc9a0-f918-4afe-88b4-f63e2ac5d93a None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:10.395513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cc9a0-f918-4afe-88b4-f63e2ac5d93a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:10.395944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cc9a0-f918-4afe-88b4-f63e2ac5d93a None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:10.399174 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1086/2239] 10.222.0.17 () {54 vars in 921 bytes} [Sat Oct 1 20:50:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:50:10.495271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e27459-fb6e-46a9-91b4-9538e6ebd88e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:10.507241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19dd3e-30d2-48fc-8c97-a3715c031937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:10.508797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19dd3e-30d2-48fc-8c97-a3715c031937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:10.509236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19dd3e-30d2-48fc-8c97-a3715c031937 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:10.509617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19dd3e-30d2-48fc-8c97-a3715c031937 admin admin] PATH_INFO: `/v3/projects/cb7f6c9003734fcaac1124e7879370d7` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:10.511405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d19dd3e-30d2-48fc-8c97-a3715c031937 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb7f6c9003734fcaac1124e7879370d7)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:10.514628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d19dd3e-30d2-48fc-8c97-a3715c031937 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:10.592888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cc9a0-f918-4afe-88b4-f63e2ac5d93a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:10.598025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ee0cc9a0-f918-4afe-88b4-f63e2ac5d93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:10.614611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4d19dd3e-30d2-48fc-8c97-a3715c031937 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb7f6c9003734fcaac1124e7879370d7', 'request_id': 'req-4d19dd3e-30d2-48fc-8c97-a3715c031937'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:10.636413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4d19dd3e-30d2-48fc-8c97-a3715c031937 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb7f6c9003734fcaac1124e7879370d7', 'request_id': 'req-4d19dd3e-30d2-48fc-8c97-a3715c031937'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:10.656549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:10.667572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4d19dd3e-30d2-48fc-8c97-a3715c031937 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb7f6c9003734fcaac1124e7879370d7', 'request_id': 'req-4d19dd3e-30d2-48fc-8c97-a3715c031937'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:10.688127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4d19dd3e-30d2-48fc-8c97-a3715c031937 admin admin] The token cache is being invalidate because project cb7f6c9003734fcaac1124e7879370d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:50:10.689715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4d19dd3e-30d2-48fc-8c97-a3715c031937 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:10.690727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4d19dd3e-30d2-48fc-8c97-a3715c031937 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:10.693298 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1154/2240] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:50:10 2022] DELETE /identity/v3/projects/cb7f6c9003734fcaac1124e7879370d7 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:10.694890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba42e5-6350-4347-b2a7-42054ffd46e9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:10.696138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba42e5-6350-4347-b2a7-42054ffd46e9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:10.696653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba42e5-6350-4347-b2a7-42054ffd46e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:10.698471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:10.738160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-45ba42e5-6350-4347-b2a7-42054ffd46e9 None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:10.739843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-45ba42e5-6350-4347-b2a7-42054ffd46e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:10.764840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cc9a0-f918-4afe-88b4-f63e2ac5d93a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:10.786131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c787b-365f-438d-89a2-bab5a6392e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:10.791459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c787b-365f-438d-89a2-bab5a6392e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:10.791793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c787b-365f-438d-89a2-bab5a6392e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:10.792072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c787b-365f-438d-89a2-bab5a6392e67 admin admin] PATH_INFO: `/v3/users/e34abaa1d265489fa35c25a3eae6f052` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:10.793640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c787b-365f-438d-89a2-bab5a6392e67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e34abaa1d265489fa35c25a3eae6f052)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:10.801447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c787b-365f-438d-89a2-bab5a6392e67 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:10.920951 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1155/2241] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 20:50:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:10.923035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba42e5-6350-4347-b2a7-42054ffd46e9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:10.924874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-45ba42e5-6350-4347-b2a7-42054ffd46e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:10.942871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba42e5-6350-4347-b2a7-42054ffd46e9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:10.949551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b07e5d-2473-4b37-bd0c-c351507ffd01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:10.950786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b07e5d-2473-4b37-bd0c-c351507ffd01 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:10.951302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b07e5d-2473-4b37-bd0c-c351507ffd01 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:10.951876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b07e5d-2473-4b37-bd0c-c351507ffd01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:10.954816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e6b07e5d-2473-4b37-bd0c-c351507ffd01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:10.983636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-816c787b-365f-438d-89a2-bab5a6392e67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e34abaa1d265489fa35c25a3eae6f052'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:10.984631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-816c787b-365f-438d-89a2-bab5a6392e67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e34abaa1d265489fa35c25a3eae6f052'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:10.985377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-816c787b-365f-438d-89a2-bab5a6392e67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e34abaa1d265489fa35c25a3eae6f052'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:10.987678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b07e5d-2473-4b37-bd0c-c351507ffd01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:10.992696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b07e5d-2473-4b37-bd0c-c351507ffd01 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:11.014415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-816c787b-365f-438d-89a2-bab5a6392e67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e34abaa1d265489fa35c25a3eae6f052'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:11.076498 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1156/2242] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:11.087819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88227e-f7fa-4df6-aa26-42c9803f3e30 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:11.092774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88227e-f7fa-4df6-aa26-42c9803f3e30 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:11.094833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88227e-f7fa-4df6-aa26-42c9803f3e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:11.110023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-816c787b-365f-438d-89a2-bab5a6392e67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e34abaa1d265489fa35c25a3eae6f052'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:11.204595 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1087/2243] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:50:10 2022] DELETE /identity/v3/users/e34abaa1d265489fa35c25a3eae6f052 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:11.205550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c787b-365f-438d-89a2-bab5a6392e67 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:11.222597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2e88227e-f7fa-4df6-aa26-42c9803f3e30 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:11.225703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-816c787b-365f-438d-89a2-bab5a6392e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:11.238558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2e88227e-f7fa-4df6-aa26-42c9803f3e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:11.538820 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1157/2244] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:11.580082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bf144-a4ee-4c64-87f1-4dcc931f0237 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:11.580379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bf144-a4ee-4c64-87f1-4dcc931f0237 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:11.580664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bf144-a4ee-4c64-87f1-4dcc931f0237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:11.584108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c787b-365f-438d-89a2-bab5a6392e67 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:11.611611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5c04a-52af-40ed-8d20-3166ad743bf1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:11.612046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a69bf144-a4ee-4c64-87f1-4dcc931f0237 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:11.612283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5c04a-52af-40ed-8d20-3166ad743bf1 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:11.612575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5c04a-52af-40ed-8d20-3166ad743bf1 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:11.612854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5c04a-52af-40ed-8d20-3166ad743bf1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:11.614952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a69bf144-a4ee-4c64-87f1-4dcc931f0237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:11.623483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f2f5c04a-52af-40ed-8d20-3166ad743bf1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:11.645641 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1158/2245] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:11.648103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bf144-a4ee-4c64-87f1-4dcc931f0237 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:11.677718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bf144-a4ee-4c64-87f1-4dcc931f0237 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:11.678019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5c04a-52af-40ed-8d20-3166ad743bf1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:11.680330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5c04a-52af-40ed-8d20-3166ad743bf1 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:11.691659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f94e5-c92d-4b1b-aa3f-25a9b45ce948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:11.693389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f94e5-c92d-4b1b-aa3f-25a9b45ce948 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:11.694646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f94e5-c92d-4b1b-aa3f-25a9b45ce948 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:11.708560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f94e5-c92d-4b1b-aa3f-25a9b45ce948 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:11.709571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f94e5-c92d-4b1b-aa3f-25a9b45ce948 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:50:11.711148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f94e5-c92d-4b1b-aa3f-25a9b45ce948 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:11.718835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f94e5-c92d-4b1b-aa3f-25a9b45ce948 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:11.738957 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1088/2246] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:11.740618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae74e6-3782-4299-86c7-810d03d7535b None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:11.740751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae74e6-3782-4299-86c7-810d03d7535b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:11.740998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae74e6-3782-4299-86c7-810d03d7535b None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:11.753295 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1089/2247] 10.222.0.17 () {54 vars in 921 bytes} [Sat Oct 1 20:50:11 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:50:11.759671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e52e1f-e2e0-4366-853e-47220763479d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:11.759989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e52e1f-e2e0-4366-853e-47220763479d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:11.760199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e52e1f-e2e0-4366-853e-47220763479d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:11.769035 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1159/2248] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:50:11.776831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f94e5-c92d-4b1b-aa3f-25a9b45ce948 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:11.807385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f94e5-c92d-4b1b-aa3f-25a9b45ce948 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:11.817853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc8760-167a-40bc-911d-ecd5fcd5f060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:11.818751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc8760-167a-40bc-911d-ecd5fcd5f060 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:11.819040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc8760-167a-40bc-911d-ecd5fcd5f060 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:11.819320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc8760-167a-40bc-911d-ecd5fcd5f060 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:11.820039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc8760-167a-40bc-911d-ecd5fcd5f060 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:11.822470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc8760-167a-40bc-911d-ecd5fcd5f060 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:11.849769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-34e52e1f-e2e0-4366-853e-47220763479d None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:11.854272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-34e52e1f-e2e0-4366-853e-47220763479d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:11.888090 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1160/2249] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:50:11 2022] POST /identity/v3/projects => generated 403 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:50:11.888090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc8760-167a-40bc-911d-ecd5fcd5f060 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:11.890382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-51cc8760-167a-40bc-911d-ecd5fcd5f060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:11.962046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:11.992773 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1090/2250] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 20:50:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 01 20:50:11.993810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e52e1f-e2e0-4366-853e-47220763479d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:11.996070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:12.037710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e52e1f-e2e0-4366-853e-47220763479d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:12.048541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc8760-167a-40bc-911d-ecd5fcd5f060 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:12.053753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a4203-9bd9-4c20-b4ef-426feb41282d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:12.054419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a4203-9bd9-4c20-b4ef-426feb41282d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:12.054970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a4203-9bd9-4c20-b4ef-426feb41282d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:12.055534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a4203-9bd9-4c20-b4ef-426feb41282d admin admin] PATH_INFO: `/v3/projects/70acee88008f4d83af21de726cafe09a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:12.056726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a4203-9bd9-4c20-b4ef-426feb41282d admin admin] RBAC: Authorizing `identity:delete_project(project_id=70acee88008f4d83af21de726cafe09a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:12.059081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a4203-9bd9-4c20-b4ef-426feb41282d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:12.059245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c8a5c-0440-4d60-8313-51ce682651c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:12.060975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c8a5c-0440-4d60-8313-51ce682651c7 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:12.063717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c8a5c-0440-4d60-8313-51ce682651c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:12.064521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c8a5c-0440-4d60-8313-51ce682651c7 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:12.069518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c8a5c-0440-4d60-8313-51ce682651c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:12.072242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c8a5c-0440-4d60-8313-51ce682651c7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:12.146558 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1091/2251] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:50:11 2022] POST /identity/v3/users => generated 302 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:12.149009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c8a5c-0440-4d60-8313-51ce682651c7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:12.160048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-050c8a5c-0440-4d60-8313-51ce682651c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:12.189213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-095a4203-9bd9-4c20-b4ef-426feb41282d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70acee88008f4d83af21de726cafe09a', 'request_id': 'req-095a4203-9bd9-4c20-b4ef-426feb41282d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:12.203705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-095a4203-9bd9-4c20-b4ef-426feb41282d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70acee88008f4d83af21de726cafe09a', 'request_id': 'req-095a4203-9bd9-4c20-b4ef-426feb41282d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:12.224218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-095a4203-9bd9-4c20-b4ef-426feb41282d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70acee88008f4d83af21de726cafe09a', 'request_id': 'req-095a4203-9bd9-4c20-b4ef-426feb41282d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:12.269224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-095a4203-9bd9-4c20-b4ef-426feb41282d admin admin] The token cache is being invalidate because project 70acee88008f4d83af21de726cafe09a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:50:12.270019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-095a4203-9bd9-4c20-b4ef-426feb41282d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:12.270469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-095a4203-9bd9-4c20-b4ef-426feb41282d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:12.273281 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1161/2252] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:50:11 2022] DELETE /identity/v3/projects/70acee88008f4d83af21de726cafe09a => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:12.274348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a4203-9bd9-4c20-b4ef-426feb41282d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:12.276878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-095a4203-9bd9-4c20-b4ef-426feb41282d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:12.403725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c8a5c-0440-4d60-8313-51ce682651c7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:12.482072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a4203-9bd9-4c20-b4ef-426feb41282d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:12.490191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e979e-1025-430b-a9f1-e22ee8d64672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:12.491079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e979e-1025-430b-a9f1-e22ee8d64672 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:12.491469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e979e-1025-430b-a9f1-e22ee8d64672 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:12.491828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e979e-1025-430b-a9f1-e22ee8d64672 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:12.492938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e979e-1025-430b-a9f1-e22ee8d64672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:12.494762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e979e-1025-430b-a9f1-e22ee8d64672 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:12.534252 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1162/2253] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:12.537059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f9e53-8808-4715-be46-476342a0a82d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:12.537866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f9e53-8808-4715-be46-476342a0a82d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:12.538443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f9e53-8808-4715-be46-476342a0a82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:12.554456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4958229-f66e-44c7-ab64-e405727e9cbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:12.555200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4958229-f66e-44c7-ab64-e405727e9cbe service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:12.555563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4958229-f66e-44c7-ab64-e405727e9cbe service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:12.555934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4958229-f66e-44c7-ab64-e405727e9cbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:12.558038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d4958229-f66e-44c7-ab64-e405727e9cbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:12.599168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c38f9e53-8808-4715-be46-476342a0a82d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:12.600541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c38f9e53-8808-4715-be46-476342a0a82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:12.630571 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1163/2254] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:12.631896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f9e53-8808-4715-be46-476342a0a82d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:12.651560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f9e53-8808-4715-be46-476342a0a82d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:12.664497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404cdb4-03bf-4641-acad-74d7c4861adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:12.665115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404cdb4-03bf-4641-acad-74d7c4861adb admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:12.665403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404cdb4-03bf-4641-acad-74d7c4861adb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:12.665951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404cdb4-03bf-4641-acad-74d7c4861adb admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:12.666483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404cdb4-03bf-4641-acad-74d7c4861adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:12.668347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404cdb4-03bf-4641-acad-74d7c4861adb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:12.697281 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1164/2255] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:12.705591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe144c8-ffbb-4bf1-95a3-2ab26cafaacf None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:12.706016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe144c8-ffbb-4bf1-95a3-2ab26cafaacf None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:12.706305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe144c8-ffbb-4bf1-95a3-2ab26cafaacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:12.744672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4958229-f66e-44c7-ab64-e405727e9cbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:12.750682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4958229-f66e-44c7-ab64-e405727e9cbe service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:12.777768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8fe144c8-ffbb-4bf1-95a3-2ab26cafaacf None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:12.782647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8fe144c8-ffbb-4bf1-95a3-2ab26cafaacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:12.786378 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1092/2256] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:12.787984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4958229-f66e-44c7-ab64-e405727e9cbe service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:12.809409 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1165/2257] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:12.813699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe144c8-ffbb-4bf1-95a3-2ab26cafaacf None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:12.826302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4958229-f66e-44c7-ab64-e405727e9cbe service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:12.841099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe144c8-ffbb-4bf1-95a3-2ab26cafaacf None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:12.842665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334cb9b-2716-4bf7-82b3-52a90f555b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:12.845724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334cb9b-2716-4bf7-82b3-52a90f555b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:12.846609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334cb9b-2716-4bf7-82b3-52a90f555b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:12.847639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334cb9b-2716-4bf7-82b3-52a90f555b6d admin admin] PATH_INFO: `/v3/projects/4f67f67302a44fcaa504527da580532e/users/a022d16da7234783b36d85cfe717628f/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:12.867235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9b82c-08ce-4145-9360-53f3ba5a41f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:12.868365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9b82c-08ce-4145-9360-53f3ba5a41f7 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:12.868737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9b82c-08ce-4145-9360-53f3ba5a41f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:12.868902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9b82c-08ce-4145-9360-53f3ba5a41f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:12.880047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9b82c-08ce-4145-9360-53f3ba5a41f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:50:12.880390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9b82c-08ce-4145-9360-53f3ba5a41f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:12.893548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9b82c-08ce-4145-9360-53f3ba5a41f7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:12.919412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334cb9b-2716-4bf7-82b3-52a90f555b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f67f67302a44fcaa504527da580532e, user_id=a022d16da7234783b36d85cfe717628f, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:12.931792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334cb9b-2716-4bf7-82b3-52a90f555b6d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:12.942503 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1166/2258] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:50:12.943581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9b82c-08ce-4145-9360-53f3ba5a41f7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:12.962851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9b82c-08ce-4145-9360-53f3ba5a41f7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:12.978421 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1093/2259] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:12 2022] PUT /identity/v3/projects/4f67f67302a44fcaa504527da580532e/users/a022d16da7234783b36d85cfe717628f/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:12.981797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6220c56-e1c3-4f22-b540-39ae1e90cd22 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:12.982092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6220c56-e1c3-4f22-b540-39ae1e90cd22 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:12.983978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6220c56-e1c3-4f22-b540-39ae1e90cd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:13.010034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:13.048350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f6220c56-e1c3-4f22-b540-39ae1e90cd22 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:13.053913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f6220c56-e1c3-4f22-b540-39ae1e90cd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:13.064130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:13.195926 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1094/2260] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:13.200204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e90fb6-986a-481a-93ea-edf0c65d861a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:13.201173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e90fb6-986a-481a-93ea-edf0c65d861a admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:13.206793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6220c56-e1c3-4f22-b540-39ae1e90cd22 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:13.207306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e90fb6-986a-481a-93ea-edf0c65d861a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:13.208713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e90fb6-986a-481a-93ea-edf0c65d861a admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:13.208859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e90fb6-986a-481a-93ea-edf0c65d861a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:13.211741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e90fb6-986a-481a-93ea-edf0c65d861a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:13.212641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f6220c56-e1c3-4f22-b540-39ae1e90cd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:13.290203 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1167/2261] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:50:12 2022] POST /identity/v3/projects => generated 391 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:13.292346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e90fb6-986a-481a-93ea-edf0c65d861a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:13.380117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6220c56-e1c3-4f22-b540-39ae1e90cd22 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:13.479358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e90fb6-986a-481a-93ea-edf0c65d861a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:13.489659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df8213-de1d-4d59-9610-88fe00850ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:13.490397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df8213-de1d-4d59-9610-88fe00850ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:13.490870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df8213-de1d-4d59-9610-88fe00850ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:13.491270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df8213-de1d-4d59-9610-88fe00850ab4 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:13.491992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df8213-de1d-4d59-9610-88fe00850ab4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:13.494070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df8213-de1d-4d59-9610-88fe00850ab4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:13.517045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6349f7-48b9-4bd9-9164-3130c37e90ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:13.518337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6349f7-48b9-4bd9-9164-3130c37e90ee service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:13.518729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6349f7-48b9-4bd9-9164-3130c37e90ee service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:13.519071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6349f7-48b9-4bd9-9164-3130c37e90ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:13.540265 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1168/2262] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:50:13 2022] POST /identity/v3/users => generated 296 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:13.547952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df8213-de1d-4d59-9610-88fe00850ab4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:13.559655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6349f7-48b9-4bd9-9164-3130c37e90ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:13.561368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6349f7-48b9-4bd9-9164-3130c37e90ee service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:13.582662 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1095/2263] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:13.622323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df8213-de1d-4d59-9610-88fe00850ab4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:13.642614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca3109-7807-4876-ad6f-f97e7187e609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:13.643806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca3109-7807-4876-ad6f-f97e7187e609 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:13.644251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca3109-7807-4876-ad6f-f97e7187e609 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:13.644703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca3109-7807-4876-ad6f-f97e7187e609 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:13.645516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca3109-7807-4876-ad6f-f97e7187e609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:13.652514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca3109-7807-4876-ad6f-f97e7187e609 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:13.685800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6349f7-48b9-4bd9-9164-3130c37e90ee service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:13.699723 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1169/2264] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:13.702978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6349f7-48b9-4bd9-9164-3130c37e90ee service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:13.717930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46251d3-c6b2-41b1-89ea-aad11fc4bf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:13.718830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46251d3-c6b2-41b1-89ea-aad11fc4bf71 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:13.719270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46251d3-c6b2-41b1-89ea-aad11fc4bf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:13.719726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46251d3-c6b2-41b1-89ea-aad11fc4bf71 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:13.720526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46251d3-c6b2-41b1-89ea-aad11fc4bf71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:13.725870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46251d3-c6b2-41b1-89ea-aad11fc4bf71 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:13.749829 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1096/2265] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:13.754900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca3109-7807-4876-ad6f-f97e7187e609 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:13.780636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca3109-7807-4876-ad6f-f97e7187e609 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:13.794764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b2ea7-a9c3-4675-933c-7ba4c8f28a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:13.794764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b2ea7-a9c3-4675-933c-7ba4c8f28a11 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:13.794764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b2ea7-a9c3-4675-933c-7ba4c8f28a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:13.794764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b2ea7-a9c3-4675-933c-7ba4c8f28a11 admin admin] PATH_INFO: `/v3/projects/3b1e74abd17d4102b622bf3ec2d4f715/users/0e76f88bb60c4f4eb2657a929098f991/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:13.863462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b2ea7-a9c3-4675-933c-7ba4c8f28a11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b1e74abd17d4102b622bf3ec2d4f715, user_id=0e76f88bb60c4f4eb2657a929098f991, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:13.874808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b2ea7-a9c3-4675-933c-7ba4c8f28a11 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:13.909139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46251d3-c6b2-41b1-89ea-aad11fc4bf71 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:13.911276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a46251d3-c6b2-41b1-89ea-aad11fc4bf71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:13.940399 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1170/2266] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:13 2022] PUT /identity/v3/projects/3b1e74abd17d4102b622bf3ec2d4f715/users/0e76f88bb60c4f4eb2657a929098f991/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:13.948229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23833606-1f90-42da-96e9-8ca3968d4097 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:13.949876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46251d3-c6b2-41b1-89ea-aad11fc4bf71 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:13.951591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23833606-1f90-42da-96e9-8ca3968d4097 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:13.951897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23833606-1f90-42da-96e9-8ca3968d4097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:14.004932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-23833606-1f90-42da-96e9-8ca3968d4097 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:14.006907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-23833606-1f90-42da-96e9-8ca3968d4097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:14.135304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:14.147311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:14.158940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1143b38-60ba-4cd6-8b37-5d7463fe066b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:14.159609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1143b38-60ba-4cd6-8b37-5d7463fe066b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:14.161336 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1171/2267] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:14.162968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1143b38-60ba-4cd6-8b37-5d7463fe066b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:14.163505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1143b38-60ba-4cd6-8b37-5d7463fe066b admin admin] PATH_INFO: `/v3/users/0980a1e24cdc491aa1a0202afc4fcd64` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:14.165626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1143b38-60ba-4cd6-8b37-5d7463fe066b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0980a1e24cdc491aa1a0202afc4fcd64)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:14.168093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1143b38-60ba-4cd6-8b37-5d7463fe066b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:14.189202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23833606-1f90-42da-96e9-8ca3968d4097 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:14.286821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f1143b38-60ba-4cd6-8b37-5d7463fe066b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0980a1e24cdc491aa1a0202afc4fcd64'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:14.288201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f1143b38-60ba-4cd6-8b37-5d7463fe066b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0980a1e24cdc491aa1a0202afc4fcd64'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:14.290534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f1143b38-60ba-4cd6-8b37-5d7463fe066b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0980a1e24cdc491aa1a0202afc4fcd64'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:14.307438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23833606-1f90-42da-96e9-8ca3968d4097 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:14.309720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f1143b38-60ba-4cd6-8b37-5d7463fe066b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0980a1e24cdc491aa1a0202afc4fcd64'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:14.314832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfb24c-4967-45db-bd62-76753b66c7dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:14.316009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfb24c-4967-45db-bd62-76753b66c7dc service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:14.316831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfb24c-4967-45db-bd62-76753b66c7dc service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:14.317331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfb24c-4967-45db-bd62-76753b66c7dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:14.320814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-86cfb24c-4967-45db-bd62-76753b66c7dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:14.333678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f1143b38-60ba-4cd6-8b37-5d7463fe066b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0980a1e24cdc491aa1a0202afc4fcd64'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:14.359855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cfb24c-4967-45db-bd62-76753b66c7dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:14.363981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cfb24c-4967-45db-bd62-76753b66c7dc service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:14.374774 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1097/2268] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:50:13 2022] DELETE /identity/v3/users/0980a1e24cdc491aa1a0202afc4fcd64 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:14.378932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f81ff-7bb5-466d-b044-fb3445ae4480 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:14.379747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f81ff-7bb5-466d-b044-fb3445ae4480 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:14.380026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f81ff-7bb5-466d-b044-fb3445ae4480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:14.382195 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1172/2269] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:50:14.417765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-909f81ff-7bb5-466d-b044-fb3445ae4480 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:14.419208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-909f81ff-7bb5-466d-b044-fb3445ae4480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:14.562286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfb24c-4967-45db-bd62-76753b66c7dc service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:14.565121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-86cfb24c-4967-45db-bd62-76753b66c7dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:14.598341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfb24c-4967-45db-bd62-76753b66c7dc service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:14.629242 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1098/2270] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:14.648124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f81ff-7bb5-466d-b044-fb3445ae4480 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:14.649883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-909f81ff-7bb5-466d-b044-fb3445ae4480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:14.785722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df40310-3bd9-4afa-90b1-a3d700009a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:14.786407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df40310-3bd9-4afa-90b1-a3d700009a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:14.786778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df40310-3bd9-4afa-90b1-a3d700009a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:14.787087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df40310-3bd9-4afa-90b1-a3d700009a27 admin admin] PATH_INFO: `/v3/users/fac54082682041db8cb3cf87fdea889f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:14.789683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df40310-3bd9-4afa-90b1-a3d700009a27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fac54082682041db8cb3cf87fdea889f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:14.802975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df40310-3bd9-4afa-90b1-a3d700009a27 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:14.844878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f81ff-7bb5-466d-b044-fb3445ae4480 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:14.858075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf8e5d-359f-473d-803a-4e3ae356bb2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:14.868740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf8e5d-359f-473d-803a-4e3ae356bb2a service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:14.868740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf8e5d-359f-473d-803a-4e3ae356bb2a service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:14.869105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf8e5d-359f-473d-803a-4e3ae356bb2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:14.932117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf8e5d-359f-473d-803a-4e3ae356bb2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:14.933757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf8e5d-359f-473d-803a-4e3ae356bb2a service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:14.974082 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1099/2271] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:15.184692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3df40310-3bd9-4afa-90b1-a3d700009a27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fac54082682041db8cb3cf87fdea889f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:15.217360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3df40310-3bd9-4afa-90b1-a3d700009a27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fac54082682041db8cb3cf87fdea889f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:15.249965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3df40310-3bd9-4afa-90b1-a3d700009a27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fac54082682041db8cb3cf87fdea889f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:15.252669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3df40310-3bd9-4afa-90b1-a3d700009a27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fac54082682041db8cb3cf87fdea889f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:15.270381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3df40310-3bd9-4afa-90b1-a3d700009a27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fac54082682041db8cb3cf87fdea889f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:15.283810 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1173/2272] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:50:14 2022] DELETE /identity/v3/users/fac54082682041db8cb3cf87fdea889f => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:50:15.288236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48648c15-49df-4d78-a52f-5f646a4e3511 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:15.289916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48648c15-49df-4d78-a52f-5f646a4e3511 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:15.290409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48648c15-49df-4d78-a52f-5f646a4e3511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:15.366479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-48648c15-49df-4d78-a52f-5f646a4e3511 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:15.372815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-48648c15-49df-4d78-a52f-5f646a4e3511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:15.408836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:15.521533 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1100/2273] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:15.594601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df40310-3bd9-4afa-90b1-a3d700009a27 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:15.596144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3df40310-3bd9-4afa-90b1-a3d700009a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:15.669269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:15.753068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df40310-3bd9-4afa-90b1-a3d700009a27 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:15.763951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19228d-d460-4dd6-9bd6-1ebd86fbb80d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:15.764873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19228d-d460-4dd6-9bd6-1ebd86fbb80d service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:15.765910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19228d-d460-4dd6-9bd6-1ebd86fbb80d service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:15.766447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19228d-d460-4dd6-9bd6-1ebd86fbb80d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:15.793421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19228d-d460-4dd6-9bd6-1ebd86fbb80d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:15.796368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19228d-d460-4dd6-9bd6-1ebd86fbb80d service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:15.809992 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1174/2274] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:16.056861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48648c15-49df-4d78-a52f-5f646a4e3511 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:16.060617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-48648c15-49df-4d78-a52f-5f646a4e3511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:16.088487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48648c15-49df-4d78-a52f-5f646a4e3511 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:16.111461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b84c35-469e-4b81-8f6e-f01f62db38f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:16.112036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b84c35-469e-4b81-8f6e-f01f62db38f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:16.112231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b84c35-469e-4b81-8f6e-f01f62db38f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:16.112527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b84c35-469e-4b81-8f6e-f01f62db38f7 admin admin] PATH_INFO: `/v3/projects/f7d124b2b3db49328d393be7be2e82d6` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:16.113779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b84c35-469e-4b81-8f6e-f01f62db38f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7d124b2b3db49328d393be7be2e82d6)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:16.115759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b84c35-469e-4b81-8f6e-f01f62db38f7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:16.221190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a6b84c35-469e-4b81-8f6e-f01f62db38f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7d124b2b3db49328d393be7be2e82d6', 'request_id': 'req-a6b84c35-469e-4b81-8f6e-f01f62db38f7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:16.244439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a6b84c35-469e-4b81-8f6e-f01f62db38f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7d124b2b3db49328d393be7be2e82d6', 'request_id': 'req-a6b84c35-469e-4b81-8f6e-f01f62db38f7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:16.253471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a6b84c35-469e-4b81-8f6e-f01f62db38f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7d124b2b3db49328d393be7be2e82d6', 'request_id': 'req-a6b84c35-469e-4b81-8f6e-f01f62db38f7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:16.268588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a6b84c35-469e-4b81-8f6e-f01f62db38f7 admin admin] The token cache is being invalidate because project f7d124b2b3db49328d393be7be2e82d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:50:16.268898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a6b84c35-469e-4b81-8f6e-f01f62db38f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:16.269581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a6b84c35-469e-4b81-8f6e-f01f62db38f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:16.271298 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1101/2275] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:50:16 2022] DELETE /identity/v3/projects/f7d124b2b3db49328d393be7be2e82d6 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:50:16.276180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19228d-d460-4dd6-9bd6-1ebd86fbb80d service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:16.306534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19228d-d460-4dd6-9bd6-1ebd86fbb80d service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:16.465898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a222a7-b3ab-4560-8f83-1500e5df3ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:16.466592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a222a7-b3ab-4560-8f83-1500e5df3ce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:16.466884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a222a7-b3ab-4560-8f83-1500e5df3ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:16.467160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a222a7-b3ab-4560-8f83-1500e5df3ce9 admin admin] PATH_INFO: `/v3/users/c964ff61874f4e8f8713261701c34991` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:16.473731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a222a7-b3ab-4560-8f83-1500e5df3ce9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c964ff61874f4e8f8713261701c34991)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:16.475758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a222a7-b3ab-4560-8f83-1500e5df3ce9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:16.581309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f7a222a7-b3ab-4560-8f83-1500e5df3ce9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c964ff61874f4e8f8713261701c34991'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:16.596054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f7a222a7-b3ab-4560-8f83-1500e5df3ce9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c964ff61874f4e8f8713261701c34991'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:16.616561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f7a222a7-b3ab-4560-8f83-1500e5df3ce9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c964ff61874f4e8f8713261701c34991'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:16.620003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f7a222a7-b3ab-4560-8f83-1500e5df3ce9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c964ff61874f4e8f8713261701c34991'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:16.638392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f7a222a7-b3ab-4560-8f83-1500e5df3ce9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c964ff61874f4e8f8713261701c34991'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:16.649345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b84c35-469e-4b81-8f6e-f01f62db38f7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:16.651018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a6b84c35-469e-4b81-8f6e-f01f62db38f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:16.655669 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1175/2276] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:50:16 2022] DELETE /identity/v3/users/c964ff61874f4e8f8713261701c34991 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:16.668500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b84c35-469e-4b81-8f6e-f01f62db38f7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:16.829144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b3ceb-89b9-424e-b007-b739b9dfe2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:16.830581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b3ceb-89b9-424e-b007-b739b9dfe2cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:16.831237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b3ceb-89b9-424e-b007-b739b9dfe2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:16.831967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b3ceb-89b9-424e-b007-b739b9dfe2cf admin admin] PATH_INFO: `/v3/projects/d4cb176cab2a4f9d9c6e33d62dd91ddc` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:16.834322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b3ceb-89b9-424e-b007-b739b9dfe2cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4cb176cab2a4f9d9c6e33d62dd91ddc)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:16.837708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b3ceb-89b9-424e-b007-b739b9dfe2cf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:16.944857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:16.958085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a82b3ceb-89b9-424e-b007-b739b9dfe2cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4cb176cab2a4f9d9c6e33d62dd91ddc', 'request_id': 'req-a82b3ceb-89b9-424e-b007-b739b9dfe2cf'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:16.968996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a82b3ceb-89b9-424e-b007-b739b9dfe2cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4cb176cab2a4f9d9c6e33d62dd91ddc', 'request_id': 'req-a82b3ceb-89b9-424e-b007-b739b9dfe2cf'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:16.979338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a82b3ceb-89b9-424e-b007-b739b9dfe2cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4cb176cab2a4f9d9c6e33d62dd91ddc', 'request_id': 'req-a82b3ceb-89b9-424e-b007-b739b9dfe2cf'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:17.004386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a82b3ceb-89b9-424e-b007-b739b9dfe2cf admin admin] The token cache is being invalidate because project d4cb176cab2a4f9d9c6e33d62dd91ddc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:50:17.005029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a82b3ceb-89b9-424e-b007-b739b9dfe2cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:17.008555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a82b3ceb-89b9-424e-b007-b739b9dfe2cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:17.012174 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1102/2277] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:50:16 2022] DELETE /identity/v3/projects/d4cb176cab2a4f9d9c6e33d62dd91ddc => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:50:17.016322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a222a7-b3ab-4560-8f83-1500e5df3ce9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:17.017916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f7a222a7-b3ab-4560-8f83-1500e5df3ce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:17.044734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a222a7-b3ab-4560-8f83-1500e5df3ce9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:17.169952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:17.208880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c343d-22c8-44b3-b475-8c05314a52da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:17.210426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c343d-22c8-44b3-b475-8c05314a52da admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:17.211664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c343d-22c8-44b3-b475-8c05314a52da admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:17.212126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c343d-22c8-44b3-b475-8c05314a52da admin admin] PATH_INFO: `/v3/users/35503a37685d4e11a514195dfaf29f38` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:17.213809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c343d-22c8-44b3-b475-8c05314a52da admin admin] RBAC: Authorizing `identity:delete_user(user_id=35503a37685d4e11a514195dfaf29f38)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:17.217629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c343d-22c8-44b3-b475-8c05314a52da admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:17.365682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cb4c343d-22c8-44b3-b475-8c05314a52da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35503a37685d4e11a514195dfaf29f38'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:17.383844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cb4c343d-22c8-44b3-b475-8c05314a52da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35503a37685d4e11a514195dfaf29f38'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:17.411497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cb4c343d-22c8-44b3-b475-8c05314a52da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35503a37685d4e11a514195dfaf29f38'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:17.412641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cb4c343d-22c8-44b3-b475-8c05314a52da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35503a37685d4e11a514195dfaf29f38'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:17.427580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cb4c343d-22c8-44b3-b475-8c05314a52da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35503a37685d4e11a514195dfaf29f38'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:17.440149 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1176/2278] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:50:17 2022] DELETE /identity/v3/users/35503a37685d4e11a514195dfaf29f38 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:17.890586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b3ceb-89b9-424e-b007-b739b9dfe2cf admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:17.892108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a82b3ceb-89b9-424e-b007-b739b9dfe2cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:17.921764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b3ceb-89b9-424e-b007-b739b9dfe2cf admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:18.090457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a464f5e-bc25-40d5-a1ff-1d197795d7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:18.091229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a464f5e-bc25-40d5-a1ff-1d197795d7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:18.091503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a464f5e-bc25-40d5-a1ff-1d197795d7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:18.091850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a464f5e-bc25-40d5-a1ff-1d197795d7ca admin admin] PATH_INFO: `/v3/projects/219a096e44d4400c9513ada7e5acf85b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:18.094721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a464f5e-bc25-40d5-a1ff-1d197795d7ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=219a096e44d4400c9513ada7e5acf85b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:18.102341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a464f5e-bc25-40d5-a1ff-1d197795d7ca admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:18.189832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0a464f5e-bc25-40d5-a1ff-1d197795d7ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '219a096e44d4400c9513ada7e5acf85b', 'request_id': 'req-0a464f5e-bc25-40d5-a1ff-1d197795d7ca'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:18.200694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0a464f5e-bc25-40d5-a1ff-1d197795d7ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '219a096e44d4400c9513ada7e5acf85b', 'request_id': 'req-0a464f5e-bc25-40d5-a1ff-1d197795d7ca'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:18.222751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0a464f5e-bc25-40d5-a1ff-1d197795d7ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '219a096e44d4400c9513ada7e5acf85b', 'request_id': 'req-0a464f5e-bc25-40d5-a1ff-1d197795d7ca'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:18.239444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0a464f5e-bc25-40d5-a1ff-1d197795d7ca admin admin] The token cache is being invalidate because project 219a096e44d4400c9513ada7e5acf85b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:50:18.240269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0a464f5e-bc25-40d5-a1ff-1d197795d7ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:18.241354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0a464f5e-bc25-40d5-a1ff-1d197795d7ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:18.244203 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1103/2279] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:50:17 2022] DELETE /identity/v3/projects/219a096e44d4400c9513ada7e5acf85b => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:18.250046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05ef7f-846b-41f6-8a56-645f5c14b099 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:18.251110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05ef7f-846b-41f6-8a56-645f5c14b099 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:18.251616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05ef7f-846b-41f6-8a56-645f5c14b099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:18.299205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7c05ef7f-846b-41f6-8a56-645f5c14b099 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:18.302982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7c05ef7f-846b-41f6-8a56-645f5c14b099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:18.430107 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1177/2280] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:18.438507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f7a4d-4abe-41a4-badb-c324185a08cc None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:18.439104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f7a4d-4abe-41a4-badb-c324185a08cc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:18.439619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f7a4d-4abe-41a4-badb-c324185a08cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:18.467682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-429f7a4d-4abe-41a4-badb-c324185a08cc None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:18.469314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-429f7a4d-4abe-41a4-badb-c324185a08cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:18.479812 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1104/2281] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:18.484537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05ef7f-846b-41f6-8a56-645f5c14b099 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:18.497499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05ef7f-846b-41f6-8a56-645f5c14b099 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:18.514666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1013bd8-5108-4f49-846d-c01d1fcf6c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:18.516058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1013bd8-5108-4f49-846d-c01d1fcf6c64 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:18.517102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1013bd8-5108-4f49-846d-c01d1fcf6c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:18.517694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1013bd8-5108-4f49-846d-c01d1fcf6c64 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:18.522049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1013bd8-5108-4f49-846d-c01d1fcf6c64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:50:18.523274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1013bd8-5108-4f49-846d-c01d1fcf6c64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:18.526438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1013bd8-5108-4f49-846d-c01d1fcf6c64 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:18.546274 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1178/2282] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:50:18.555646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f7a4d-4abe-41a4-badb-c324185a08cc None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:18.567198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f7a4d-4abe-41a4-badb-c324185a08cc None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:18.583936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769aa3b1-e0c6-4484-a13f-abfccf6fcaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:18.584712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769aa3b1-e0c6-4484-a13f-abfccf6fcaab admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:18.585174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769aa3b1-e0c6-4484-a13f-abfccf6fcaab admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:18.585762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769aa3b1-e0c6-4484-a13f-abfccf6fcaab admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:18.586679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769aa3b1-e0c6-4484-a13f-abfccf6fcaab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:18.588800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769aa3b1-e0c6-4484-a13f-abfccf6fcaab admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:18.665819 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1105/2283] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:50:18 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:18.667054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1013bd8-5108-4f49-846d-c01d1fcf6c64 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:18.678845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1013bd8-5108-4f49-846d-c01d1fcf6c64 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:18.768035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769aa3b1-e0c6-4484-a13f-abfccf6fcaab admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:18.771539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-769aa3b1-e0c6-4484-a13f-abfccf6fcaab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:18.822507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:18.867573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ca86a-72a8-4d9d-9d38-f801058189b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:18.870181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:18.872238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ca86a-72a8-4d9d-9d38-f801058189b7 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:18.873481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ca86a-72a8-4d9d-9d38-f801058189b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:18.875567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ca86a-72a8-4d9d-9d38-f801058189b7 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:18.877248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ca86a-72a8-4d9d-9d38-f801058189b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:18.884547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ca86a-72a8-4d9d-9d38-f801058189b7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:18.941360 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1179/2284] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:50:18 2022] POST /identity/v3/users => generated 286 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:18.941591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769aa3b1-e0c6-4484-a13f-abfccf6fcaab admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:18.945245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ca86a-72a8-4d9d-9d38-f801058189b7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:18.958037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb44694-f578-43c3-8324-3681a2e7281c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:18.958732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb44694-f578-43c3-8324-3681a2e7281c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:18.959014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb44694-f578-43c3-8324-3681a2e7281c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:18.959288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb44694-f578-43c3-8324-3681a2e7281c admin admin] PATH_INFO: `/v3/projects/4803d0561d104d7c8298e6782b8cb18d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:18.960221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ca86a-72a8-4d9d-9d38-f801058189b7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:18.962058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb44694-f578-43c3-8324-3681a2e7281c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4803d0561d104d7c8298e6782b8cb18d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:18.964826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb44694-f578-43c3-8324-3681a2e7281c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:18.967243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e8f77-5581-4b2e-b475-13a2d6b8f420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:18.967970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e8f77-5581-4b2e-b475-13a2d6b8f420 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:18.968388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e8f77-5581-4b2e-b475-13a2d6b8f420 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:18.969179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e8f77-5581-4b2e-b475-13a2d6b8f420 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:18.969962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e8f77-5581-4b2e-b475-13a2d6b8f420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:18.971843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e8f77-5581-4b2e-b475-13a2d6b8f420 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:18.986505 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1180/2285] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:18.990075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e8f77-5581-4b2e-b475-13a2d6b8f420 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:19.004866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e8f77-5581-4b2e-b475-13a2d6b8f420 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:19.027750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c56ee6-0eb9-45fd-b1c0-cb12f8bfc30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:19.028541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c56ee6-0eb9-45fd-b1c0-cb12f8bfc30b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:19.030504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c56ee6-0eb9-45fd-b1c0-cb12f8bfc30b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:19.031205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c56ee6-0eb9-45fd-b1c0-cb12f8bfc30b admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:19.033501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c56ee6-0eb9-45fd-b1c0-cb12f8bfc30b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:19.040513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c56ee6-0eb9-45fd-b1c0-cb12f8bfc30b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:19.075920 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1181/2286] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:19.079938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c56ee6-0eb9-45fd-b1c0-cb12f8bfc30b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:19.119046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c56ee6-0eb9-45fd-b1c0-cb12f8bfc30b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:19.127261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-aeb44694-f578-43c3-8324-3681a2e7281c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4803d0561d104d7c8298e6782b8cb18d', 'request_id': 'req-aeb44694-f578-43c3-8324-3681a2e7281c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:19.147018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-aeb44694-f578-43c3-8324-3681a2e7281c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4803d0561d104d7c8298e6782b8cb18d', 'request_id': 'req-aeb44694-f578-43c3-8324-3681a2e7281c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:19.160853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-aeb44694-f578-43c3-8324-3681a2e7281c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4803d0561d104d7c8298e6782b8cb18d', 'request_id': 'req-aeb44694-f578-43c3-8324-3681a2e7281c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:19.195783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-aeb44694-f578-43c3-8324-3681a2e7281c admin admin] The token cache is being invalidate because project 4803d0561d104d7c8298e6782b8cb18d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:50:19.196244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-aeb44694-f578-43c3-8324-3681a2e7281c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:19.197148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-aeb44694-f578-43c3-8324-3681a2e7281c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:19.200891 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1106/2287] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:50:18 2022] DELETE /identity/v3/projects/4803d0561d104d7c8298e6782b8cb18d => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:19.206115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d99dd-db5b-4d61-b77a-4405cc6d63db None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:19.206597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d99dd-db5b-4d61-b77a-4405cc6d63db None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:19.207092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d99dd-db5b-4d61-b77a-4405cc6d63db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:19.247821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-472d99dd-db5b-4d61-b77a-4405cc6d63db None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:19.249542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-472d99dd-db5b-4d61-b77a-4405cc6d63db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:19.288746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b39a60-5f48-45e4-b248-a5697cd9efb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:19.291249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b39a60-5f48-45e4-b248-a5697cd9efb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:19.292130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b39a60-5f48-45e4-b248-a5697cd9efb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:19.292590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b39a60-5f48-45e4-b248-a5697cd9efb6 admin admin] PATH_INFO: `/v3/projects/63cb41305a594f9086e0238fbb7859ed/users/6de55f058a884fb2bb75153f49975eef/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:19.342651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b39a60-5f48-45e4-b248-a5697cd9efb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63cb41305a594f9086e0238fbb7859ed, user_id=6de55f058a884fb2bb75153f49975eef, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:19.345954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b39a60-5f48-45e4-b248-a5697cd9efb6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:19.374876 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1182/2288] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:19 2022] PUT /identity/v3/projects/63cb41305a594f9086e0238fbb7859ed/users/6de55f058a884fb2bb75153f49975eef/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:19.385122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c4826-7adb-4a41-b642-d5e4c8079819 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:19.385455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c4826-7adb-4a41-b642-d5e4c8079819 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:19.385693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c4826-7adb-4a41-b642-d5e4c8079819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:19.397411 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1107/2289] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:50:19.406533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d6bab-9d9e-4f1f-8f87-bf9f3cc8beaa None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:19.406887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d6bab-9d9e-4f1f-8f87-bf9f3cc8beaa None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:19.407093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d6bab-9d9e-4f1f-8f87-bf9f3cc8beaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:19.469877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6b7c4826-7adb-4a41-b642-d5e4c8079819 None None] MFA Rules not processed for user `6de55f058a884fb2bb75153f49975eef`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:19.474665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6b7c4826-7adb-4a41-b642-d5e4c8079819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:19.477208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3b5d6bab-9d9e-4f1f-8f87-bf9f3cc8beaa None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:19.481281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3b5d6bab-9d9e-4f1f-8f87-bf9f3cc8beaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:19.662763 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1183/2290] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:50:19.687427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c4826-7adb-4a41-b642-d5e4c8079819 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:19.691392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6b7c4826-7adb-4a41-b642-d5e4c8079819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:19.697804 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1108/2291] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:19.698263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d6bab-9d9e-4f1f-8f87-bf9f3cc8beaa None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:19.714705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d6bab-9d9e-4f1f-8f87-bf9f3cc8beaa None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:19.731199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a37d2-0358-4e70-aec7-87aab86b132d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:19.731831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a37d2-0358-4e70-aec7-87aab86b132d admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:19.732105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a37d2-0358-4e70-aec7-87aab86b132d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:19.732382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a37d2-0358-4e70-aec7-87aab86b132d admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:19.733041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a37d2-0358-4e70-aec7-87aab86b132d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:50:19.733346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a37d2-0358-4e70-aec7-87aab86b132d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:19.735088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a37d2-0358-4e70-aec7-87aab86b132d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:19.737060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:19.762981 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1109/2292] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:50:19.767637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a37d2-0358-4e70-aec7-87aab86b132d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:19.776111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:19.794728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a37d2-0358-4e70-aec7-87aab86b132d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:19.812859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0fc843-d923-470c-bcc1-6a0f7b9cf40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:19.813784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fc843-d923-470c-bcc1-6a0f7b9cf40d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:19.814928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fc843-d923-470c-bcc1-6a0f7b9cf40d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:19.815382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fc843-d923-470c-bcc1-6a0f7b9cf40d admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:19.821135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0fc843-d923-470c-bcc1-6a0f7b9cf40d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:19.824337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0fc843-d923-470c-bcc1-6a0f7b9cf40d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:19.850276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c4826-7adb-4a41-b642-d5e4c8079819 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:19.877913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf340b2b-a8f7-4bea-9f2b-5df3ce50f2a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:19.879101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf340b2b-a8f7-4bea-9f2b-5df3ce50f2a7 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:19.879715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf340b2b-a8f7-4bea-9f2b-5df3ce50f2a7 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:19.881247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf340b2b-a8f7-4bea-9f2b-5df3ce50f2a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:19.890946 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1110/2293] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:50:19 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:19.892036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0fc843-d923-470c-bcc1-6a0f7b9cf40d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:19.892350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf340b2b-a8f7-4bea-9f2b-5df3ce50f2a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:19.897464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf340b2b-a8f7-4bea-9f2b-5df3ce50f2a7 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:19.916162 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1184/2294] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:19.991551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0fc843-d923-470c-bcc1-6a0f7b9cf40d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:19.999445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfca8e-20bd-47ef-a718-dfe8a5dddc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:20.000425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfca8e-20bd-47ef-a718-dfe8a5dddc44 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:20.000933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfca8e-20bd-47ef-a718-dfe8a5dddc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:20.001534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfca8e-20bd-47ef-a718-dfe8a5dddc44 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:20.004266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dfca8e-20bd-47ef-a718-dfe8a5dddc44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:20.007235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dfca8e-20bd-47ef-a718-dfe8a5dddc44 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:20.051668 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1111/2295] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:50:19 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:20.055834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf340b2b-a8f7-4bea-9f2b-5df3ce50f2a7 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:20.075243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf340b2b-a8f7-4bea-9f2b-5df3ce50f2a7 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:20.087285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45c543-41bd-4732-8673-195869446fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:20.087911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45c543-41bd-4732-8673-195869446fcc admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:20.088191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45c543-41bd-4732-8673-195869446fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:20.088464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45c543-41bd-4732-8673-195869446fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:20.089117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc45c543-41bd-4732-8673-195869446fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:20.099642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc45c543-41bd-4732-8673-195869446fcc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:20.154108 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1185/2296] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:20.158172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfca8e-20bd-47ef-a718-dfe8a5dddc44 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:20.172272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfca8e-20bd-47ef-a718-dfe8a5dddc44 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:20.183894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395464b7-fdde-4b59-8da6-04c6b603cad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:20.184529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395464b7-fdde-4b59-8da6-04c6b603cad4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:20.184809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395464b7-fdde-4b59-8da6-04c6b603cad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:20.185080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395464b7-fdde-4b59-8da6-04c6b603cad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:20.185731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395464b7-fdde-4b59-8da6-04c6b603cad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:20.187398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395464b7-fdde-4b59-8da6-04c6b603cad4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:20.212557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45c543-41bd-4732-8673-195869446fcc admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:20.219460 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1112/2297] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:20.224134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45c543-41bd-4732-8673-195869446fcc admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:20.237074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ee58e-8d12-4bc2-bd05-549c766d98ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:20.237759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ee58e-8d12-4bc2-bd05-549c766d98ec admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:20.238040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ee58e-8d12-4bc2-bd05-549c766d98ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:20.238480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ee58e-8d12-4bc2-bd05-549c766d98ec admin admin] PATH_INFO: `/v3/projects/f0c9140437fb44329168b6369b12cfa3/users/bb40ef4a263f476a842083f29b3c35d3/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:20.280588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ee58e-8d12-4bc2-bd05-549c766d98ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0c9140437fb44329168b6369b12cfa3, user_id=bb40ef4a263f476a842083f29b3c35d3, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:20.283525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ee58e-8d12-4bc2-bd05-549c766d98ec admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:20.322931 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1186/2298] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:20 2022] PUT /identity/v3/projects/f0c9140437fb44329168b6369b12cfa3/users/bb40ef4a263f476a842083f29b3c35d3/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:20.335663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9d6d8-51c4-431c-9c4c-aa72db88dd8c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:20.335933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9d6d8-51c4-431c-9c4c-aa72db88dd8c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:20.336172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9d6d8-51c4-431c-9c4c-aa72db88dd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:20.410014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b1b9d6d8-51c4-431c-9c4c-aa72db88dd8c None None] MFA Rules not processed for user `bb40ef4a263f476a842083f29b3c35d3`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:20.413497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b1b9d6d8-51c4-431c-9c4c-aa72db88dd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:20.571508 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1113/2299] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:20.575731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ee58e-8d12-4bc2-bd05-549c766d98ec admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:20.756114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ee58e-8d12-4bc2-bd05-549c766d98ec admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:20.778802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f819f-c0b5-4ceb-be1d-b25688f7ebe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:20.779815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f819f-c0b5-4ceb-be1d-b25688f7ebe9 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:20.780199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f819f-c0b5-4ceb-be1d-b25688f7ebe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:20.780577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f819f-c0b5-4ceb-be1d-b25688f7ebe9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:20.781320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f819f-c0b5-4ceb-be1d-b25688f7ebe9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:20.787153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f819f-c0b5-4ceb-be1d-b25688f7ebe9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:20.854299 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1187/2300] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:50:20 2022] POST /identity/v3/projects => generated 369 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:20.855957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9d6d8-51c4-431c-9c4c-aa72db88dd8c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:20.873621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9d6d8-51c4-431c-9c4c-aa72db88dd8c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:20.940288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:20.966266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856c2f1-96ec-4951-b02f-4f88974af258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:20.966992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856c2f1-96ec-4951-b02f-4f88974af258 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:20.967419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856c2f1-96ec-4951-b02f-4f88974af258 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:20.967700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856c2f1-96ec-4951-b02f-4f88974af258 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:20.968334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4856c2f1-96ec-4951-b02f-4f88974af258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:20.971212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4856c2f1-96ec-4951-b02f-4f88974af258 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:21.004976 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1114/2301] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:50:20 2022] POST /identity/v3/users => generated 291 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:21.009579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f819f-c0b5-4ceb-be1d-b25688f7ebe9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:21.026256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f819f-c0b5-4ceb-be1d-b25688f7ebe9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:21.032925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c93b6c-5c5f-46b5-a186-86a4990b0b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:21.034770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c93b6c-5c5f-46b5-a186-86a4990b0b92 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:21.035472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c93b6c-5c5f-46b5-a186-86a4990b0b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:21.036114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c93b6c-5c5f-46b5-a186-86a4990b0b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:21.037141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c93b6c-5c5f-46b5-a186-86a4990b0b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:21.039801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c93b6c-5c5f-46b5-a186-86a4990b0b92 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:21.056662 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1188/2302] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:21.061355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856c2f1-96ec-4951-b02f-4f88974af258 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:21.078105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856c2f1-96ec-4951-b02f-4f88974af258 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:21.085458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0080085-dd22-45e6-adfc-e4544ed2c602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:21.086542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0080085-dd22-45e6-adfc-e4544ed2c602 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:21.087492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0080085-dd22-45e6-adfc-e4544ed2c602 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:21.087968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0080085-dd22-45e6-adfc-e4544ed2c602 admin admin] PATH_INFO: `/v3/projects/1e41c249bb334415a434e2f5b2e1e1f9/users/9e454af927e44d8085ecbc8332320741/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:21.109950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0080085-dd22-45e6-adfc-e4544ed2c602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e41c249bb334415a434e2f5b2e1e1f9, user_id=9e454af927e44d8085ecbc8332320741, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:21.116139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0080085-dd22-45e6-adfc-e4544ed2c602 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:21.149254 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1115/2303] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:21 2022] PUT /identity/v3/projects/1e41c249bb334415a434e2f5b2e1e1f9/users/9e454af927e44d8085ecbc8332320741/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:21.149703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c93b6c-5c5f-46b5-a186-86a4990b0b92 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:21.163642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c93b6c-5c5f-46b5-a186-86a4990b0b92 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:21.251467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:21.262541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf8c3e-23fe-4ae6-bf60-432d60840a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:21.263652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf8c3e-23fe-4ae6-bf60-432d60840a28 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:21.271756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf8c3e-23fe-4ae6-bf60-432d60840a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:21.272162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf8c3e-23fe-4ae6-bf60-432d60840a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:21.273754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf8c3e-23fe-4ae6-bf60-432d60840a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:21.275711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf8c3e-23fe-4ae6-bf60-432d60840a28 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:21.322961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0080085-dd22-45e6-adfc-e4544ed2c602 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:21.343027 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1189/2304] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:50:21.358295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0080085-dd22-45e6-adfc-e4544ed2c602 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:21.376970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1adac-8f2b-4bdb-97c5-322783f4ee1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:21.381884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1adac-8f2b-4bdb-97c5-322783f4ee1c admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:21.382253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1adac-8f2b-4bdb-97c5-322783f4ee1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:21.383116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1adac-8f2b-4bdb-97c5-322783f4ee1c admin admin] PATH_INFO: `/v3/projects/1e41c249bb334415a434e2f5b2e1e1f9/users/9e454af927e44d8085ecbc8332320741/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:21.387216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1adac-8f2b-4bdb-97c5-322783f4ee1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e41c249bb334415a434e2f5b2e1e1f9, user_id=9e454af927e44d8085ecbc8332320741, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:21.393866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1adac-8f2b-4bdb-97c5-322783f4ee1c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:21.442592 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1116/2305] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:21 2022] PUT /identity/v3/projects/1e41c249bb334415a434e2f5b2e1e1f9/users/9e454af927e44d8085ecbc8332320741/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:21.450675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a92be-c08d-4322-a2d7-db2d8fb7ef60 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:21.451047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a92be-c08d-4322-a2d7-db2d8fb7ef60 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:21.451383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a92be-c08d-4322-a2d7-db2d8fb7ef60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:21.517174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fe0a92be-c08d-4322-a2d7-db2d8fb7ef60 None None] MFA Rules not processed for user `9e454af927e44d8085ecbc8332320741`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:21.520410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fe0a92be-c08d-4322-a2d7-db2d8fb7ef60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:21.698617 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1190/2306] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:21 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:22.239360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1adac-8f2b-4bdb-97c5-322783f4ee1c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:22.242680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-84d1adac-8f2b-4bdb-97c5-322783f4ee1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:22.353582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1adac-8f2b-4bdb-97c5-322783f4ee1c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:22.366567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4b17d-0a05-4293-96bd-4516eb2e3c00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:22.369400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4b17d-0a05-4293-96bd-4516eb2e3c00 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:22.369922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4b17d-0a05-4293-96bd-4516eb2e3c00 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:22.370374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4b17d-0a05-4293-96bd-4516eb2e3c00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:22.392210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4b17d-0a05-4293-96bd-4516eb2e3c00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:22.395622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4b17d-0a05-4293-96bd-4516eb2e3c00 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:22.421033 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1117/2307] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:22 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:22.599411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a92be-c08d-4322-a2d7-db2d8fb7ef60 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:22.678875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4b17d-0a05-4293-96bd-4516eb2e3c00 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:22.681174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-11d4b17d-0a05-4293-96bd-4516eb2e3c00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:22.746020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a92be-c08d-4322-a2d7-db2d8fb7ef60 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:22.771855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454898d-2a67-49a4-9f31-e9e436b569f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:22.772789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454898d-2a67-49a4-9f31-e9e436b569f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:22.774915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454898d-2a67-49a4-9f31-e9e436b569f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:22.775301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454898d-2a67-49a4-9f31-e9e436b569f9 admin admin] PATH_INFO: `/v3/users/bb40ef4a263f476a842083f29b3c35d3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:22.778932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454898d-2a67-49a4-9f31-e9e436b569f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb40ef4a263f476a842083f29b3c35d3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:22.781466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454898d-2a67-49a4-9f31-e9e436b569f9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:22.878336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4b17d-0a05-4293-96bd-4516eb2e3c00 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:22.908455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7454898d-2a67-49a4-9f31-e9e436b569f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb40ef4a263f476a842083f29b3c35d3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:22.912529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71827802-a89a-4da6-8853-2c7d1e90f028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:22.921261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71827802-a89a-4da6-8853-2c7d1e90f028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:22.921659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71827802-a89a-4da6-8853-2c7d1e90f028 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:22.922398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71827802-a89a-4da6-8853-2c7d1e90f028 admin admin] PATH_INFO: `/v3/users/6de55f058a884fb2bb75153f49975eef` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:22.925491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7454898d-2a67-49a4-9f31-e9e436b569f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb40ef4a263f476a842083f29b3c35d3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:22.933203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71827802-a89a-4da6-8853-2c7d1e90f028 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6de55f058a884fb2bb75153f49975eef)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:22.936988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71827802-a89a-4da6-8853-2c7d1e90f028 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:22.955070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7454898d-2a67-49a4-9f31-e9e436b569f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb40ef4a263f476a842083f29b3c35d3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:22.956638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7454898d-2a67-49a4-9f31-e9e436b569f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb40ef4a263f476a842083f29b3c35d3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:22.962269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:22.972641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7454898d-2a67-49a4-9f31-e9e436b569f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb40ef4a263f476a842083f29b3c35d3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:22.990606 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1191/2308] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:50:22 2022] DELETE /identity/v3/users/bb40ef4a263f476a842083f29b3c35d3 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:22.991363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe0fb1-e00e-4fa8-98dd-b57914e61ed7 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:22.991363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe0fb1-e00e-4fa8-98dd-b57914e61ed7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:22.991545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe0fb1-e00e-4fa8-98dd-b57914e61ed7 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:22.997301 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1192/2309] 10.222.0.17 () {54 vars in 921 bytes} [Sat Oct 1 20:50:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:50:22.999693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a614a8-716e-47c1-82ab-1a62c8446ec9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:22.999977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a614a8-716e-47c1-82ab-1a62c8446ec9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:23.000250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a614a8-716e-47c1-82ab-1a62c8446ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:23.013431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:23.059034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d5a614a8-716e-47c1-82ab-1a62c8446ec9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:23.062437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d5a614a8-716e-47c1-82ab-1a62c8446ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:23.069134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-71827802-a89a-4da6-8853-2c7d1e90f028 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6de55f058a884fb2bb75153f49975eef'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:23.070972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-71827802-a89a-4da6-8853-2c7d1e90f028 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6de55f058a884fb2bb75153f49975eef'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:23.071594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-71827802-a89a-4da6-8853-2c7d1e90f028 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6de55f058a884fb2bb75153f49975eef'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:23.087914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-71827802-a89a-4da6-8853-2c7d1e90f028 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6de55f058a884fb2bb75153f49975eef'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:23.115508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-71827802-a89a-4da6-8853-2c7d1e90f028 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6de55f058a884fb2bb75153f49975eef'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:23.140530 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1118/2310] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:50:22 2022] DELETE /identity/v3/users/6de55f058a884fb2bb75153f49975eef => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:23.140530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14a95a-b8c4-463a-a495-b5fa10eac5c1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:23.140530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14a95a-b8c4-463a-a495-b5fa10eac5c1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:23.140530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14a95a-b8c4-463a-a495-b5fa10eac5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:23.191924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bb14a95a-b8c4-463a-a495-b5fa10eac5c1 None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:23.208296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bb14a95a-b8c4-463a-a495-b5fa10eac5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:23.273042 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1193/2311] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:23.274837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b7386-b24a-4529-b329-89b8c4e53fda None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:23.275126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b7386-b24a-4529-b329-89b8c4e53fda None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:23.276032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b7386-b24a-4529-b329-89b8c4e53fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:23.307951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-973b7386-b24a-4529-b329-89b8c4e53fda None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:23.315455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-973b7386-b24a-4529-b329-89b8c4e53fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:23.371231 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1119/2312] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 20:50:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:23.375375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a81d9b-7182-4bae-973b-14b2191b8c10 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:23.378314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a81d9b-7182-4bae-973b-14b2191b8c10 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:23.378870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a81d9b-7182-4bae-973b-14b2191b8c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:23.434429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f2a81d9b-7182-4bae-973b-14b2191b8c10 None None] MFA Rules not processed for user `0e76f88bb60c4f4eb2657a929098f991`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:23.438665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f2a81d9b-7182-4bae-973b-14b2191b8c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:23.439715 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1194/2313] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:23.440819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b7386-b24a-4529-b329-89b8c4e53fda None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:23.442854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-973b7386-b24a-4529-b329-89b8c4e53fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:23.546259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b7386-b24a-4529-b329-89b8c4e53fda None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:23.573914 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1120/2314] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:23.575493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a81d9b-7182-4bae-973b-14b2191b8c10 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:23.580132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98d42b-8e51-4d6e-a5c6-dc69b8c13426 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:23.580756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98d42b-8e51-4d6e-a5c6-dc69b8c13426 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:23.581043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98d42b-8e51-4d6e-a5c6-dc69b8c13426 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:23.581358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98d42b-8e51-4d6e-a5c6-dc69b8c13426 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:23.590184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f2a81d9b-7182-4bae-973b-14b2191b8c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:23.600398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98d42b-8e51-4d6e-a5c6-dc69b8c13426 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:23.602470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98d42b-8e51-4d6e-a5c6-dc69b8c13426 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:23.625681 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1195/2315] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:23.627174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98d42b-8e51-4d6e-a5c6-dc69b8c13426 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:23.637964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:23.651144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98d42b-8e51-4d6e-a5c6-dc69b8c13426 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:23.665834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd17a12-5973-40e3-952d-7fdab0c413f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:23.666791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd17a12-5973-40e3-952d-7fdab0c413f4 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:23.667989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd17a12-5973-40e3-952d-7fdab0c413f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:23.668486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd17a12-5973-40e3-952d-7fdab0c413f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:23.682300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd17a12-5973-40e3-952d-7fdab0c413f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:23.684168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd17a12-5973-40e3-952d-7fdab0c413f4 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:23.703260 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1196/2316] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:23.704919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd17a12-5973-40e3-952d-7fdab0c413f4 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:23.706720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2cd17a12-5973-40e3-952d-7fdab0c413f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:23.708001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a81d9b-7182-4bae-973b-14b2191b8c10 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:23.714384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8da85-2178-428e-a77e-e563d633624d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:23.715208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8da85-2178-428e-a77e-e563d633624d service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:23.715593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8da85-2178-428e-a77e-e563d633624d service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:23.716471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8da85-2178-428e-a77e-e563d633624d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:23.728103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd17a12-5973-40e3-952d-7fdab0c413f4 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:23.744978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea2c90-c3fe-4e32-af1c-61723c4cd5cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:23.746030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd8da85-2178-428e-a77e-e563d633624d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:23.746847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea2c90-c3fe-4e32-af1c-61723c4cd5cc service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:23.747303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea2c90-c3fe-4e32-af1c-61723c4cd5cc service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:23.749047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd8da85-2178-428e-a77e-e563d633624d service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:23.749503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea2c90-c3fe-4e32-af1c-61723c4cd5cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:23.767892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea2c90-c3fe-4e32-af1c-61723c4cd5cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:23.769509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea2c90-c3fe-4e32-af1c-61723c4cd5cc service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:23.779168 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1121/2317] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:23.828383 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1197/2318] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:23.913085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ecd37-d102-4f00-ad22-1898cdbfaf6a None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:23.913346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ecd37-d102-4f00-ad22-1898cdbfaf6a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:23.914208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ecd37-d102-4f00-ad22-1898cdbfaf6a None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:23.916345 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1122/2319] 10.222.0.17 () {54 vars in 921 bytes} [Sat Oct 1 20:50:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:50:23.925201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9db30-6a59-4797-a17c-78958386323f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:23.925494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9db30-6a59-4797-a17c-78958386323f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:23.929162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9db30-6a59-4797-a17c-78958386323f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:23.968862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-bfb9db30-6a59-4797-a17c-78958386323f None None] MFA Rules not processed for user `e17a026f305248878f18eec35e1e02dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:23.970111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bfb9db30-6a59-4797-a17c-78958386323f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:23.978117 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1198/2320] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 20:50:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:50:24.012501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ecd37-d102-4f00-ad22-1898cdbfaf6a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:24.014362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-488ecd37-d102-4f00-ad22-1898cdbfaf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:24.065238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ecd37-d102-4f00-ad22-1898cdbfaf6a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:24.069581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8696d9-646a-46ed-95b1-1558f9cb7342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:24.070832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8696d9-646a-46ed-95b1-1558f9cb7342 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:24.071108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8696d9-646a-46ed-95b1-1558f9cb7342 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:24.071385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8696d9-646a-46ed-95b1-1558f9cb7342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:24.096027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8696d9-646a-46ed-95b1-1558f9cb7342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:24.097545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8696d9-646a-46ed-95b1-1558f9cb7342 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:24.124160 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1123/2321] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:24.325897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9db30-6a59-4797-a17c-78958386323f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:24.327437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bfb9db30-6a59-4797-a17c-78958386323f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:24.432311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9db30-6a59-4797-a17c-78958386323f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:24.443063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37c123-4f32-4ca6-9bea-3e33780d5122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:24.443719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37c123-4f32-4ca6-9bea-3e33780d5122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:24.444003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37c123-4f32-4ca6-9bea-3e33780d5122 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:24.444283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37c123-4f32-4ca6-9bea-3e33780d5122 admin admin] PATH_INFO: `/v3/projects/f0c9140437fb44329168b6369b12cfa3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:24.451889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d37c123-4f32-4ca6-9bea-3e33780d5122 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0c9140437fb44329168b6369b12cfa3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:24.454533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d37c123-4f32-4ca6-9bea-3e33780d5122 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:24.455687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8696d9-646a-46ed-95b1-1558f9cb7342 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:24.456477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:24.458905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cf8696d9-646a-46ed-95b1-1558f9cb7342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:24.493416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8696d9-646a-46ed-95b1-1558f9cb7342 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:24.516418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e64286-fe4d-4225-8419-22f28cd7c03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:24.517505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e64286-fe4d-4225-8419-22f28cd7c03d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:24.517969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e64286-fe4d-4225-8419-22f28cd7c03d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:24.519102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e64286-fe4d-4225-8419-22f28cd7c03d admin admin] PATH_INFO: `/v3/projects/63cb41305a594f9086e0238fbb7859ed` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:24.522553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e64286-fe4d-4225-8419-22f28cd7c03d admin admin] RBAC: Authorizing `identity:delete_project(project_id=63cb41305a594f9086e0238fbb7859ed)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:24.527289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e64286-fe4d-4225-8419-22f28cd7c03d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:24.545586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8d37c123-4f32-4ca6-9bea-3e33780d5122 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0c9140437fb44329168b6369b12cfa3', 'request_id': 'req-8d37c123-4f32-4ca6-9bea-3e33780d5122'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:24.559587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8d37c123-4f32-4ca6-9bea-3e33780d5122 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0c9140437fb44329168b6369b12cfa3', 'request_id': 'req-8d37c123-4f32-4ca6-9bea-3e33780d5122'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:24.576626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8d37c123-4f32-4ca6-9bea-3e33780d5122 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0c9140437fb44329168b6369b12cfa3', 'request_id': 'req-8d37c123-4f32-4ca6-9bea-3e33780d5122'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:24.617478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8d37c123-4f32-4ca6-9bea-3e33780d5122 admin admin] The token cache is being invalidate because project f0c9140437fb44329168b6369b12cfa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:50:24.618087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8d37c123-4f32-4ca6-9bea-3e33780d5122 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:24.619250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8d37c123-4f32-4ca6-9bea-3e33780d5122 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:24.621838 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1199/2322] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:50:24 2022] DELETE /identity/v3/projects/f0c9140437fb44329168b6369b12cfa3 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:24.625726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37c123-4f32-4ca6-9bea-3e33780d5122 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:24.631866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8d37c123-4f32-4ca6-9bea-3e33780d5122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:24.634482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f9e64286-fe4d-4225-8419-22f28cd7c03d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63cb41305a594f9086e0238fbb7859ed', 'request_id': 'req-f9e64286-fe4d-4225-8419-22f28cd7c03d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:24.646482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f9e64286-fe4d-4225-8419-22f28cd7c03d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63cb41305a594f9086e0238fbb7859ed', 'request_id': 'req-f9e64286-fe4d-4225-8419-22f28cd7c03d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:24.658217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f9e64286-fe4d-4225-8419-22f28cd7c03d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63cb41305a594f9086e0238fbb7859ed', 'request_id': 'req-f9e64286-fe4d-4225-8419-22f28cd7c03d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:24.678367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f9e64286-fe4d-4225-8419-22f28cd7c03d admin admin] The token cache is being invalidate because project 63cb41305a594f9086e0238fbb7859ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:50:24.678776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f9e64286-fe4d-4225-8419-22f28cd7c03d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:24.679636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f9e64286-fe4d-4225-8419-22f28cd7c03d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:24.683082 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1124/2323] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:50:24 2022] DELETE /identity/v3/projects/63cb41305a594f9086e0238fbb7859ed => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:24.688134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc205d47-ce8a-428d-a924-fe3953ff464c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:24.688594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc205d47-ce8a-428d-a924-fe3953ff464c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:24.689013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc205d47-ce8a-428d-a924-fe3953ff464c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:24.723560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-dc205d47-ce8a-428d-a924-fe3953ff464c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:24.723871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-dc205d47-ce8a-428d-a924-fe3953ff464c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:24.829105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37c123-4f32-4ca6-9bea-3e33780d5122 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:24.837856 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1125/2324] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:24.849592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b409c-ae40-46cd-85d3-e11a2abfd0e9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:24.850280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b409c-ae40-46cd-85d3-e11a2abfd0e9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:24.850826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0ca14-9429-4561-b55b-cedd2177153b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:24.852038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b409c-ae40-46cd-85d3-e11a2abfd0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:24.852274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0ca14-9429-4561-b55b-cedd2177153b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:24.852594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0ca14-9429-4561-b55b-cedd2177153b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:24.852964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0ca14-9429-4561-b55b-cedd2177153b admin admin] PATH_INFO: `/v3/users/9e454af927e44d8085ecbc8332320741` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:24.857476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0ca14-9429-4561-b55b-cedd2177153b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e454af927e44d8085ecbc8332320741)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:24.860105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0ca14-9429-4561-b55b-cedd2177153b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:24.923618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d24b409c-ae40-46cd-85d3-e11a2abfd0e9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:24.923618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d24b409c-ae40-46cd-85d3-e11a2abfd0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:24.956743 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1126/2325] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:24.956743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b409c-ae40-46cd-85d3-e11a2abfd0e9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:24.969933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-23c0ca14-9429-4561-b55b-cedd2177153b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e454af927e44d8085ecbc8332320741'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:24.970173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b409c-ae40-46cd-85d3-e11a2abfd0e9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:24.985078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634c338-5d5d-48e3-8be7-8926f6870095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:24.985725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634c338-5d5d-48e3-8be7-8926f6870095 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:24.986193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634c338-5d5d-48e3-8be7-8926f6870095 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:24.986479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634c338-5d5d-48e3-8be7-8926f6870095 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:24.987463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634c338-5d5d-48e3-8be7-8926f6870095 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:50:24.987783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634c338-5d5d-48e3-8be7-8926f6870095 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:24.989995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634c338-5d5d-48e3-8be7-8926f6870095 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:24.999005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-23c0ca14-9429-4561-b55b-cedd2177153b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e454af927e44d8085ecbc8332320741'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:25.012416 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1127/2326] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:50:25.013725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634c338-5d5d-48e3-8be7-8926f6870095 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:25.026518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-23c0ca14-9429-4561-b55b-cedd2177153b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e454af927e44d8085ecbc8332320741'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:25.030904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-23c0ca14-9429-4561-b55b-cedd2177153b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e454af927e44d8085ecbc8332320741'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:25.051732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-23c0ca14-9429-4561-b55b-cedd2177153b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e454af927e44d8085ecbc8332320741'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:25.064623 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1200/2327] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:50:24 2022] DELETE /identity/v3/users/9e454af927e44d8085ecbc8332320741 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:50:25.073282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634c338-5d5d-48e3-8be7-8926f6870095 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:25.084284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:25.213771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21620ea-d241-44ae-81a7-d62ea4efe000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:25.214411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21620ea-d241-44ae-81a7-d62ea4efe000 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:25.216556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21620ea-d241-44ae-81a7-d62ea4efe000 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:25.216671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21620ea-d241-44ae-81a7-d62ea4efe000 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:25.217245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21620ea-d241-44ae-81a7-d62ea4efe000 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:25.223129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21620ea-d241-44ae-81a7-d62ea4efe000 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:25.307251 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1128/2328] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:50:25 2022] POST /identity/v3/projects => generated 379 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:25.311377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0ca14-9429-4561-b55b-cedd2177153b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:25.313062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-23c0ca14-9429-4561-b55b-cedd2177153b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:25.330184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0ca14-9429-4561-b55b-cedd2177153b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:25.337831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:25.433607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d283f44-e6dc-45f4-b2ff-1c9068f26869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:25.434698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d283f44-e6dc-45f4-b2ff-1c9068f26869 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:25.435089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d283f44-e6dc-45f4-b2ff-1c9068f26869 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:25.435469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d283f44-e6dc-45f4-b2ff-1c9068f26869 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:25.436425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d283f44-e6dc-45f4-b2ff-1c9068f26869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:25.438775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d283f44-e6dc-45f4-b2ff-1c9068f26869 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:25.492293 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1201/2329] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:50:25 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:25.493854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21620ea-d241-44ae-81a7-d62ea4efe000 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:25.507026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21620ea-d241-44ae-81a7-d62ea4efe000 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:25.513767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e1926-8f72-4c41-99f0-cd38a4510f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:25.514482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e1926-8f72-4c41-99f0-cd38a4510f47 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:25.514880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e1926-8f72-4c41-99f0-cd38a4510f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:25.515265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e1926-8f72-4c41-99f0-cd38a4510f47 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:25.516208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e1926-8f72-4c41-99f0-cd38a4510f47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:25.517987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e1926-8f72-4c41-99f0-cd38a4510f47 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:25.541023 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1129/2330] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:25.543097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d283f44-e6dc-45f4-b2ff-1c9068f26869 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:25.586259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d283f44-e6dc-45f4-b2ff-1c9068f26869 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:25.594892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea59cb-857d-4405-8db8-68f82e47917f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:25.595521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea59cb-857d-4405-8db8-68f82e47917f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:25.595927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea59cb-857d-4405-8db8-68f82e47917f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:25.596370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea59cb-857d-4405-8db8-68f82e47917f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:25.597791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea59cb-857d-4405-8db8-68f82e47917f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:25.599504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea59cb-857d-4405-8db8-68f82e47917f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:25.627072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e1926-8f72-4c41-99f0-cd38a4510f47 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:25.634968 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1202/2331] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:25.638946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e1926-8f72-4c41-99f0-cd38a4510f47 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:25.645176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be568bcb-5746-43ee-8318-1ba21510c4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:25.646578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be568bcb-5746-43ee-8318-1ba21510c4a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:25.646873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be568bcb-5746-43ee-8318-1ba21510c4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:25.647173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be568bcb-5746-43ee-8318-1ba21510c4a7 admin admin] PATH_INFO: `/v3/projects/b775e6b549b54415b76066373ff165b4/users/485f457f31154f9aad86634ae69d14bb/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:25.698009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be568bcb-5746-43ee-8318-1ba21510c4a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b775e6b549b54415b76066373ff165b4, user_id=485f457f31154f9aad86634ae69d14bb, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:25.700925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be568bcb-5746-43ee-8318-1ba21510c4a7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:25.758144 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1130/2332] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:25 2022] PUT /identity/v3/projects/b775e6b549b54415b76066373ff165b4/users/485f457f31154f9aad86634ae69d14bb/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:25.772089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d701f0-4d82-44f2-aaca-84075731a2c7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:25.772405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d701f0-4d82-44f2-aaca-84075731a2c7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:25.772685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d701f0-4d82-44f2-aaca-84075731a2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:25.849306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a0d701f0-4d82-44f2-aaca-84075731a2c7 None None] MFA Rules not processed for user `485f457f31154f9aad86634ae69d14bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:25.850633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a0d701f0-4d82-44f2-aaca-84075731a2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:26.038955 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1203/2333] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:26.052351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be568bcb-5746-43ee-8318-1ba21510c4a7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:26.054286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-be568bcb-5746-43ee-8318-1ba21510c4a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:26.222697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be568bcb-5746-43ee-8318-1ba21510c4a7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:26.241615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5027e8-cedc-4031-8606-3ace3a949cc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:26.242252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5027e8-cedc-4031-8606-3ace3a949cc3 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:26.242606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5027e8-cedc-4031-8606-3ace3a949cc3 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:26.243030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5027e8-cedc-4031-8606-3ace3a949cc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:26.252817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5027e8-cedc-4031-8606-3ace3a949cc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:26.254894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5027e8-cedc-4031-8606-3ace3a949cc3 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:26.285735 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1131/2334] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:26.551215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5ad75-c1ac-4a9c-a69f-fe18e9c4eac9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:26.551560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5ad75-c1ac-4a9c-a69f-fe18e9c4eac9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:26.552045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5ad75-c1ac-4a9c-a69f-fe18e9c4eac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:26.590017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-cfa5ad75-c1ac-4a9c-a69f-fe18e9c4eac9 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:26.591315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cfa5ad75-c1ac-4a9c-a69f-fe18e9c4eac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:26.715714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5027e8-cedc-4031-8606-3ace3a949cc3 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:26.717401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4e5027e8-cedc-4031-8606-3ace3a949cc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:26.757542 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1204/2335] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:50:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:26.771235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5ad75-c1ac-4a9c-a69f-fe18e9c4eac9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:26.774027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cfa5ad75-c1ac-4a9c-a69f-fe18e9c4eac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:26.875722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:26.891001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:26.933252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5027e8-cedc-4031-8606-3ace3a949cc3 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:26.947220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf04d69-719e-4712-acc1-a0a48ea9f8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:26.947944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf04d69-719e-4712-acc1-a0a48ea9f8c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:26.948159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf04d69-719e-4712-acc1-a0a48ea9f8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:26.948413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf04d69-719e-4712-acc1-a0a48ea9f8c3 admin admin] PATH_INFO: `/v3/projects/1e41c249bb334415a434e2f5b2e1e1f9` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:26.950772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf04d69-719e-4712-acc1-a0a48ea9f8c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e41c249bb334415a434e2f5b2e1e1f9)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:26.952734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf04d69-719e-4712-acc1-a0a48ea9f8c3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:27.029160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5ad75-c1ac-4a9c-a69f-fe18e9c4eac9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:27.053276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf482b2-b965-4dd2-866f-3c566aee049c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:27.054241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf482b2-b965-4dd2-866f-3c566aee049c service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:27.055317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf482b2-b965-4dd2-866f-3c566aee049c service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:27.055317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf482b2-b965-4dd2-866f-3c566aee049c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:27.087053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf482b2-b965-4dd2-866f-3c566aee049c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:27.089248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf482b2-b965-4dd2-866f-3c566aee049c service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:27.114458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dcf04d69-719e-4712-acc1-a0a48ea9f8c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e41c249bb334415a434e2f5b2e1e1f9', 'request_id': 'req-dcf04d69-719e-4712-acc1-a0a48ea9f8c3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:27.122901 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1205/2336] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:50:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 20:50:27.124305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f69c3-a63d-4630-b29f-d0f0efb6128c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:27.124604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f69c3-a63d-4630-b29f-d0f0efb6128c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:27.129265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f69c3-a63d-4630-b29f-d0f0efb6128c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:27.139421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dcf04d69-719e-4712-acc1-a0a48ea9f8c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e41c249bb334415a434e2f5b2e1e1f9', 'request_id': 'req-dcf04d69-719e-4712-acc1-a0a48ea9f8c3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:27.157036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dcf04d69-719e-4712-acc1-a0a48ea9f8c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e41c249bb334415a434e2f5b2e1e1f9', 'request_id': 'req-dcf04d69-719e-4712-acc1-a0a48ea9f8c3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:27.177398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-759f69c3-a63d-4630-b29f-d0f0efb6128c None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:27.180540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-759f69c3-a63d-4630-b29f-d0f0efb6128c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:27.202735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dcf04d69-719e-4712-acc1-a0a48ea9f8c3 admin admin] The token cache is being invalidate because project 1e41c249bb334415a434e2f5b2e1e1f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:50:27.203018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dcf04d69-719e-4712-acc1-a0a48ea9f8c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:27.209406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dcf04d69-719e-4712-acc1-a0a48ea9f8c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:27.213760 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1132/2337] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:50:26 2022] DELETE /identity/v3/projects/1e41c249bb334415a434e2f5b2e1e1f9 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:27.220079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0824932-d13b-4ae8-9800-132eb033189e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:27.220519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0824932-d13b-4ae8-9800-132eb033189e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:27.220997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0824932-d13b-4ae8-9800-132eb033189e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:27.290448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e0824932-d13b-4ae8-9800-132eb033189e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:27.292490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e0824932-d13b-4ae8-9800-132eb033189e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:27.330997 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1206/2338] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:50:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:27.396464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f69c3-a63d-4630-b29f-d0f0efb6128c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:27.399211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-759f69c3-a63d-4630-b29f-d0f0efb6128c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:27.472511 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1133/2339] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:27.481180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45cb10-793b-4207-8c33-7cf94635847b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:27.481384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45cb10-793b-4207-8c33-7cf94635847b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:27.481757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45cb10-793b-4207-8c33-7cf94635847b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:27.529963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ae45cb10-793b-4207-8c33-7cf94635847b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:27.531226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ae45cb10-793b-4207-8c33-7cf94635847b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:27.560692 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1134/2340] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:50:27.561986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7388929-db22-459c-8cf3-9a9d60829f13 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:27.562268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7388929-db22-459c-8cf3-9a9d60829f13 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:27.562557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7388929-db22-459c-8cf3-9a9d60829f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:27.586127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f69c3-a63d-4630-b29f-d0f0efb6128c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:27.594009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56ed98-8f4c-48ec-9783-9c12f3fecde4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:27.594840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56ed98-8f4c-48ec-9783-9c12f3fecde4 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:27.595125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56ed98-8f4c-48ec-9783-9c12f3fecde4 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:27.595398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56ed98-8f4c-48ec-9783-9c12f3fecde4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:27.606076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56ed98-8f4c-48ec-9783-9c12f3fecde4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:27.607773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b7388929-db22-459c-8cf3-9a9d60829f13 None None] MFA Rules not processed for user `485f457f31154f9aad86634ae69d14bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:27.608330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56ed98-8f4c-48ec-9783-9c12f3fecde4 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:27.609003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b7388929-db22-459c-8cf3-9a9d60829f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:27.618930 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1207/2341] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:50:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 20:50:27.622092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56ed98-8f4c-48ec-9783-9c12f3fecde4 service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:27.639676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56ed98-8f4c-48ec-9783-9c12f3fecde4 service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:27.660757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75f447-56dd-4ef5-bf1a-08c281e70084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:27.661714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75f447-56dd-4ef5-bf1a-08c281e70084 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:27.662192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75f447-56dd-4ef5-bf1a-08c281e70084 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:27.662754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75f447-56dd-4ef5-bf1a-08c281e70084 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:27.665711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75f447-56dd-4ef5-bf1a-08c281e70084 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:50:27.666195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75f447-56dd-4ef5-bf1a-08c281e70084 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:27.671175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75f447-56dd-4ef5-bf1a-08c281e70084 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:27.744121 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1208/2342] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:50:27.745308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75f447-56dd-4ef5-bf1a-08c281e70084 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:27.778844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75f447-56dd-4ef5-bf1a-08c281e70084 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:27.807280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b8b30-9865-4532-ad9d-6c8ea6103546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:27.810374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b8b30-9865-4532-ad9d-6c8ea6103546 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:27.812690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b8b30-9865-4532-ad9d-6c8ea6103546 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:27.816659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b8b30-9865-4532-ad9d-6c8ea6103546 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:27.823088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0b8b30-9865-4532-ad9d-6c8ea6103546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:27.825979 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1135/2343] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:27.837151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0b8b30-9865-4532-ad9d-6c8ea6103546 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:27.918410 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1209/2344] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:50:27 2022] POST /identity/v3/projects => generated 375 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:27.918567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7388929-db22-459c-8cf3-9a9d60829f13 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:27.955063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7388929-db22-459c-8cf3-9a9d60829f13 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:27.994731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:28.217084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85634b-7923-4c0a-9e06-ca291905a0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:28.218139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85634b-7923-4c0a-9e06-ca291905a0b9 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:28.218665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85634b-7923-4c0a-9e06-ca291905a0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:28.219141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85634b-7923-4c0a-9e06-ca291905a0b9 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:28.219994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85634b-7923-4c0a-9e06-ca291905a0b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:28.222146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85634b-7923-4c0a-9e06-ca291905a0b9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:28.302696 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1136/2345] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:50:27 2022] POST /identity/v3/users => generated 288 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:50:28.307542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b8b30-9865-4532-ad9d-6c8ea6103546 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:28.319895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b8b30-9865-4532-ad9d-6c8ea6103546 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:28.339321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc004306-c557-4bb9-900f-a79775c7cdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:28.342279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc004306-c557-4bb9-900f-a79775c7cdf4 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:28.350049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc004306-c557-4bb9-900f-a79775c7cdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:28.350405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc004306-c557-4bb9-900f-a79775c7cdf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:28.355515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc004306-c557-4bb9-900f-a79775c7cdf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:28.358616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc004306-c557-4bb9-900f-a79775c7cdf4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:28.385764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:28.407260 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1210/2346] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:28.412041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85634b-7923-4c0a-9e06-ca291905a0b9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:28.442260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85634b-7923-4c0a-9e06-ca291905a0b9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:28.466939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a902b-f5cf-45a4-bc8d-5844d816ada6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:28.472543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a902b-f5cf-45a4-bc8d-5844d816ada6 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:28.472543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a902b-f5cf-45a4-bc8d-5844d816ada6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:28.472543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a902b-f5cf-45a4-bc8d-5844d816ada6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:28.473115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a902b-f5cf-45a4-bc8d-5844d816ada6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:28.484026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a902b-f5cf-45a4-bc8d-5844d816ada6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:28.545514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc004306-c557-4bb9-900f-a79775c7cdf4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:28.557452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc004306-c557-4bb9-900f-a79775c7cdf4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:28.558428 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1137/2347] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:28.573463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138b992-c823-471e-a814-595611807107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:28.587512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138b992-c823-471e-a814-595611807107 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:28.587512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138b992-c823-471e-a814-595611807107 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:28.587512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138b992-c823-471e-a814-595611807107 admin admin] PATH_INFO: `/v3/projects/de2a660566d947f9a2a2977cf3a32d80/users/b99f9f102b064fd68a8222a23948c813/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:28.639971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138b992-c823-471e-a814-595611807107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de2a660566d947f9a2a2977cf3a32d80, user_id=b99f9f102b064fd68a8222a23948c813, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:28.642685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138b992-c823-471e-a814-595611807107 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:28.701147 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1211/2348] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:28 2022] PUT /identity/v3/projects/de2a660566d947f9a2a2977cf3a32d80/users/b99f9f102b064fd68a8222a23948c813/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:50:28.704996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dc4c8-1712-4d60-80d3-59df06ac3446 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:28.705284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dc4c8-1712-4d60-80d3-59df06ac3446 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:28.705572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dc4c8-1712-4d60-80d3-59df06ac3446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:28.795616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a90dc4c8-1712-4d60-80d3-59df06ac3446 None None] MFA Rules not processed for user `b99f9f102b064fd68a8222a23948c813`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:28.799974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a90dc4c8-1712-4d60-80d3-59df06ac3446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:28.958329 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1138/2349] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:28.963343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138b992-c823-471e-a814-595611807107 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:29.230463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138b992-c823-471e-a814-595611807107 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:29.238785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10e74f-fd43-4069-ac23-bbfec0bfa0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:29.239900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10e74f-fd43-4069-ac23-bbfec0bfa0e1 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:29.240654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10e74f-fd43-4069-ac23-bbfec0bfa0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:29.241289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10e74f-fd43-4069-ac23-bbfec0bfa0e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:29.242648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b10e74f-fd43-4069-ac23-bbfec0bfa0e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:29.244859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b10e74f-fd43-4069-ac23-bbfec0bfa0e1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:29.302158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90dc4c8-1712-4d60-80d3-59df06ac3446 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:29.303308 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1212/2350] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:50:28 2022] POST /identity/v3/projects => generated 377 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:29.324771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90dc4c8-1712-4d60-80d3-59df06ac3446 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:29.377176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10e74f-fd43-4069-ac23-bbfec0bfa0e1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:29.379506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2b10e74f-fd43-4069-ac23-bbfec0bfa0e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:29.438192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f66f8-9563-4a8d-8ee8-8dfb8bfa664b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:29.438937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f66f8-9563-4a8d-8ee8-8dfb8bfa664b admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:29.439239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f66f8-9563-4a8d-8ee8-8dfb8bfa664b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:29.439513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f66f8-9563-4a8d-8ee8-8dfb8bfa664b admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:29.440163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f66f8-9563-4a8d-8ee8-8dfb8bfa664b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:29.442444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f66f8-9563-4a8d-8ee8-8dfb8bfa664b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:29.469354 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1139/2351] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:50:29 2022] POST /identity/v3/users => generated 295 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:50:29.473101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f66f8-9563-4a8d-8ee8-8dfb8bfa664b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:29.484882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f66f8-9563-4a8d-8ee8-8dfb8bfa664b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:29.491354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fad1d8-85b1-4fcd-bdca-fe955f82195d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:29.492974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fad1d8-85b1-4fcd-bdca-fe955f82195d admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:29.493457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fad1d8-85b1-4fcd-bdca-fe955f82195d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:29.493788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fad1d8-85b1-4fcd-bdca-fe955f82195d admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:29.494431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fad1d8-85b1-4fcd-bdca-fe955f82195d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:29.496313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fad1d8-85b1-4fcd-bdca-fe955f82195d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:29.525589 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1140/2352] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:29.526835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fad1d8-85b1-4fcd-bdca-fe955f82195d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:29.537122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:29.543883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fad1d8-85b1-4fcd-bdca-fe955f82195d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:29.553774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbb223-b658-462b-aaef-8f37a105bccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:29.555206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbb223-b658-462b-aaef-8f37a105bccb admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:29.555855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbb223-b658-462b-aaef-8f37a105bccb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:29.556197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbb223-b658-462b-aaef-8f37a105bccb admin admin] PATH_INFO: `/v3/projects/979c07d50d2f456a9fd6b8a56ffa5d34/users/27e519e61f0d4ce19a06021f6c67fb0f/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:29.579366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbb223-b658-462b-aaef-8f37a105bccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=979c07d50d2f456a9fd6b8a56ffa5d34, user_id=27e519e61f0d4ce19a06021f6c67fb0f, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:29.582726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbb223-b658-462b-aaef-8f37a105bccb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:29.637457 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1141/2353] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:29 2022] PUT /identity/v3/projects/979c07d50d2f456a9fd6b8a56ffa5d34/users/27e519e61f0d4ce19a06021f6c67fb0f/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:29.638909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbb223-b658-462b-aaef-8f37a105bccb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:29.770553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10e74f-fd43-4069-ac23-bbfec0bfa0e1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:29.841427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbb223-b658-462b-aaef-8f37a105bccb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:29.860321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef685257-2bcc-46be-a57b-3a6d079c1df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:29.863131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef685257-2bcc-46be-a57b-3a6d079c1df0 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:29.863429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef685257-2bcc-46be-a57b-3a6d079c1df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:29.863700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef685257-2bcc-46be-a57b-3a6d079c1df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:29.865023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef685257-2bcc-46be-a57b-3a6d079c1df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:29.868746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef685257-2bcc-46be-a57b-3a6d079c1df0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:29.907646 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1142/2354] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:50:29.909079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62b3ca-9d64-4fe7-aa35-09ff4fae0666 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:29.909360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62b3ca-9d64-4fe7-aa35-09ff4fae0666 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:29.909631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62b3ca-9d64-4fe7-aa35-09ff4fae0666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:29.984985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:30.006390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3d62b3ca-9d64-4fe7-aa35-09ff4fae0666 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:30.007689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3d62b3ca-9d64-4fe7-aa35-09ff4fae0666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:30.024156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f72f0e-80a9-4765-b770-ac60666a0c3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:30.024797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f72f0e-80a9-4765-b770-ac60666a0c3e service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:30.026842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f72f0e-80a9-4765-b770-ac60666a0c3e service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:30.027124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f72f0e-80a9-4765-b770-ac60666a0c3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:30.035705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-98f72f0e-80a9-4765-b770-ac60666a0c3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:30.203172 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1143/2355] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 20:50:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:30.205044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62b3ca-9d64-4fe7-aa35-09ff4fae0666 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:30.235015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62b3ca-9d64-4fe7-aa35-09ff4fae0666 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:30.244507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f72f0e-80a9-4765-b770-ac60666a0c3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:30.245973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f72f0e-80a9-4765-b770-ac60666a0c3e service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:30.255778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e9650-069f-429f-b405-21d6bb16b020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:30.261969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e9650-069f-429f-b405-21d6bb16b020 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:30.263292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e9650-069f-429f-b405-21d6bb16b020 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:30.263896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e9650-069f-429f-b405-21d6bb16b020 admin admin] PATH_INFO: `/v3/projects/979c07d50d2f456a9fd6b8a56ffa5d34/users/27e519e61f0d4ce19a06021f6c67fb0f/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:30.269487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e9650-069f-429f-b405-21d6bb16b020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=979c07d50d2f456a9fd6b8a56ffa5d34, user_id=27e519e61f0d4ce19a06021f6c67fb0f, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:30.280115 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1213/2356] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:29 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 903 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:50:30.280811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e9650-069f-429f-b405-21d6bb16b020 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:30.281074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f72f0e-80a9-4765-b770-ac60666a0c3e service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:30.327040 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1144/2357] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:30 2022] PUT /identity/v3/projects/979c07d50d2f456a9fd6b8a56ffa5d34/users/27e519e61f0d4ce19a06021f6c67fb0f/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:30.339007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d715ca-9d7b-4d8e-a6dc-d5dc07f976c8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:30.339972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d715ca-9d7b-4d8e-a6dc-d5dc07f976c8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:30.340523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d715ca-9d7b-4d8e-a6dc-d5dc07f976c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:30.476981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-08d715ca-9d7b-4d8e-a6dc-d5dc07f976c8 None None] MFA Rules not processed for user `27e519e61f0d4ce19a06021f6c67fb0f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:30.479160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-08d715ca-9d7b-4d8e-a6dc-d5dc07f976c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:30.552568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f72f0e-80a9-4765-b770-ac60666a0c3e service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:30.634041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:30.698158 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1145/2358] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:30.835423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf596c-01f8-4575-af93-ad2a4c7c2b2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:30.837113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf596c-01f8-4575-af93-ad2a4c7c2b2f service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:30.839854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf596c-01f8-4575-af93-ad2a4c7c2b2f service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:30.840312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf596c-01f8-4575-af93-ad2a4c7c2b2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:30.840452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d715ca-9d7b-4d8e-a6dc-d5dc07f976c8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:30.986295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:31.045237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf596c-01f8-4575-af93-ad2a4c7c2b2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:31.047207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf596c-01f8-4575-af93-ad2a4c7c2b2f service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:31.069273 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1214/2359] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:50:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 778 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 20:50:31.124003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d715ca-9d7b-4d8e-a6dc-d5dc07f976c8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:31.154736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c514b-5c71-46d0-be6b-292d2b9353c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:31.155484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c514b-5c71-46d0-be6b-292d2b9353c8 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:31.155867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c514b-5c71-46d0-be6b-292d2b9353c8 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:31.156238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c514b-5c71-46d0-be6b-292d2b9353c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:31.178378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c514b-5c71-46d0-be6b-292d2b9353c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:31.183302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c514b-5c71-46d0-be6b-292d2b9353c8 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:31.206808 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1146/2360] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:50:31.385415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf596c-01f8-4575-af93-ad2a4c7c2b2f service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:31.387630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-17bf596c-01f8-4575-af93-ad2a4c7c2b2f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:31.527857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf596c-01f8-4575-af93-ad2a4c7c2b2f service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:31.544485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e206e-5e67-4a30-be47-ffc5025c5957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:31.545532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e206e-5e67-4a30-be47-ffc5025c5957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:31.546013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e206e-5e67-4a30-be47-ffc5025c5957 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:31.546340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e206e-5e67-4a30-be47-ffc5025c5957 admin admin] PATH_INFO: `/v3/users/485f457f31154f9aad86634ae69d14bb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:31.555574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e206e-5e67-4a30-be47-ffc5025c5957 admin admin] RBAC: Authorizing `identity:delete_user(user_id=485f457f31154f9aad86634ae69d14bb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:31.568846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e206e-5e67-4a30-be47-ffc5025c5957 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:31.813554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fc6e206e-5e67-4a30-be47-ffc5025c5957 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '485f457f31154f9aad86634ae69d14bb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:31.830803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fc6e206e-5e67-4a30-be47-ffc5025c5957 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '485f457f31154f9aad86634ae69d14bb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:31.862668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fc6e206e-5e67-4a30-be47-ffc5025c5957 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '485f457f31154f9aad86634ae69d14bb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:31.863786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fc6e206e-5e67-4a30-be47-ffc5025c5957 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '485f457f31154f9aad86634ae69d14bb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:31.889116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fc6e206e-5e67-4a30-be47-ffc5025c5957 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '485f457f31154f9aad86634ae69d14bb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:31.896711 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1215/2361] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:50:31 2022] DELETE /identity/v3/users/485f457f31154f9aad86634ae69d14bb => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:31.901040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c3ebc-1601-4b23-8599-73d0f55e636a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:31.901583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c3ebc-1601-4b23-8599-73d0f55e636a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:31.902107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c3ebc-1601-4b23-8599-73d0f55e636a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:31.930724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-993c3ebc-1601-4b23-8599-73d0f55e636a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:31.953822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-993c3ebc-1601-4b23-8599-73d0f55e636a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:32.166006 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1147/2362] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:32.200323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e206e-5e67-4a30-be47-ffc5025c5957 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:32.202956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fc6e206e-5e67-4a30-be47-ffc5025c5957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:32.367587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e206e-5e67-4a30-be47-ffc5025c5957 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:32.378055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218260eb-5b40-43b5-b8b2-a96adc0b15f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:32.378838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218260eb-5b40-43b5-b8b2-a96adc0b15f6 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:32.379727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218260eb-5b40-43b5-b8b2-a96adc0b15f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:32.380696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218260eb-5b40-43b5-b8b2-a96adc0b15f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:32.403382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218260eb-5b40-43b5-b8b2-a96adc0b15f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:32.414190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218260eb-5b40-43b5-b8b2-a96adc0b15f6 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:32.444710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:32.474442 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1216/2363] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:33.753582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53627c-921d-44be-bd57-a065f1887280 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:33.754202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53627c-921d-44be-bd57-a065f1887280 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:33.754423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53627c-921d-44be-bd57-a065f1887280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:33.826689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4b53627c-921d-44be-bd57-a065f1887280 None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:33.828091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4b53627c-921d-44be-bd57-a065f1887280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:34.023645 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1148/2364] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 20:50:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:34.061215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218260eb-5b40-43b5-b8b2-a96adc0b15f6 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:34.064055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-218260eb-5b40-43b5-b8b2-a96adc0b15f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:34.083958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218260eb-5b40-43b5-b8b2-a96adc0b15f6 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:34.095026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c444c13-0637-4c87-8f4c-e168e8935354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:34.097004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c444c13-0637-4c87-8f4c-e168e8935354 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:34.097739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c444c13-0637-4c87-8f4c-e168e8935354 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:34.098236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c444c13-0637-4c87-8f4c-e168e8935354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:34.128081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c444c13-0637-4c87-8f4c-e168e8935354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:34.129668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c444c13-0637-4c87-8f4c-e168e8935354 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:34.163874 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1217/2365] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:50:35.135724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c488e-9b92-4bd3-99da-413a674a0908 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:35.136205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c488e-9b92-4bd3-99da-413a674a0908 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:35.137206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c488e-9b92-4bd3-99da-413a674a0908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:35.199823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:35.229445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ff4c488e-9b92-4bd3-99da-413a674a0908 None None] MFA Rules not processed for user `a022d16da7234783b36d85cfe717628f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:35.231442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ff4c488e-9b92-4bd3-99da-413a674a0908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:35.435357 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1149/2366] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:35 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:35.439408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c444c13-0637-4c87-8f4c-e168e8935354 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:35.442352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5c444c13-0637-4c87-8f4c-e168e8935354 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:35.490799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c444c13-0637-4c87-8f4c-e168e8935354 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:35.503179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc87ff9-a77c-43bd-aed3-f3a10250c7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:35.505006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc87ff9-a77c-43bd-aed3-f3a10250c7e0 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:35.505190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc87ff9-a77c-43bd-aed3-f3a10250c7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:35.505512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc87ff9-a77c-43bd-aed3-f3a10250c7e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:35.508471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc87ff9-a77c-43bd-aed3-f3a10250c7e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:35.515424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc87ff9-a77c-43bd-aed3-f3a10250c7e0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:35.584521 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1218/2367] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:50:35 2022] POST /identity/v3/projects => generated 403 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:50:35.588538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c488e-9b92-4bd3-99da-413a674a0908 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:35.613981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c488e-9b92-4bd3-99da-413a674a0908 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:35.838804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e0b26-85e4-4026-ac86-5799c3fc0e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:35.839956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e0b26-85e4-4026-ac86-5799c3fc0e1b admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:35.840662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e0b26-85e4-4026-ac86-5799c3fc0e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:35.842305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e0b26-85e4-4026-ac86-5799c3fc0e1b admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:35.843433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e0b26-85e4-4026-ac86-5799c3fc0e1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:35.851625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e0b26-85e4-4026-ac86-5799c3fc0e1b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:35.943136 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1150/2368] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:50:35 2022] POST /identity/v3/users => generated 308 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:35.947392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc87ff9-a77c-43bd-aed3-f3a10250c7e0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:35.948562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e0b26-85e4-4026-ac86-5799c3fc0e1b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:35.955056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3bc87ff9-a77c-43bd-aed3-f3a10250c7e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:35.983579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc87ff9-a77c-43bd-aed3-f3a10250c7e0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:35.986426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e0b26-85e4-4026-ac86-5799c3fc0e1b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:35.993355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2cfd81-de6f-46cb-b4e3-6f43d16c2e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:35.994372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cfd81-de6f-46cb-b4e3-6f43d16c2e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:35.994778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cfd81-de6f-46cb-b4e3-6f43d16c2e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:35.995289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cfd81-de6f-46cb-b4e3-6f43d16c2e56 admin admin] PATH_INFO: `/v3/projects/b775e6b549b54415b76066373ff165b4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:35.997601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2cfd81-de6f-46cb-b4e3-6f43d16c2e56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b775e6b549b54415b76066373ff165b4)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:36.000703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2cfd81-de6f-46cb-b4e3-6f43d16c2e56 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:36.031165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f691b-3025-4c4f-a6a9-e78915cc2377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:36.033775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f691b-3025-4c4f-a6a9-e78915cc2377 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:36.035090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f691b-3025-4c4f-a6a9-e78915cc2377 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:36.037547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f691b-3025-4c4f-a6a9-e78915cc2377 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:36.041460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f691b-3025-4c4f-a6a9-e78915cc2377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:36.052258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f691b-3025-4c4f-a6a9-e78915cc2377 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:36.106064 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1151/2369] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:36.108634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f691b-3025-4c4f-a6a9-e78915cc2377 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:36.145873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f691b-3025-4c4f-a6a9-e78915cc2377 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:36.162540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2c2cfd81-de6f-46cb-b4e3-6f43d16c2e56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b775e6b549b54415b76066373ff165b4', 'request_id': 'req-2c2cfd81-de6f-46cb-b4e3-6f43d16c2e56'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:36.182579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2c2cfd81-de6f-46cb-b4e3-6f43d16c2e56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b775e6b549b54415b76066373ff165b4', 'request_id': 'req-2c2cfd81-de6f-46cb-b4e3-6f43d16c2e56'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:36.199487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2c2cfd81-de6f-46cb-b4e3-6f43d16c2e56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b775e6b549b54415b76066373ff165b4', 'request_id': 'req-2c2cfd81-de6f-46cb-b4e3-6f43d16c2e56'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:36.250727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2c2cfd81-de6f-46cb-b4e3-6f43d16c2e56 admin admin] The token cache is being invalidate because project b775e6b549b54415b76066373ff165b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:50:36.251014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2c2cfd81-de6f-46cb-b4e3-6f43d16c2e56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:36.251699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2c2cfd81-de6f-46cb-b4e3-6f43d16c2e56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:36.253560 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1219/2370] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:50:35 2022] DELETE /identity/v3/projects/b775e6b549b54415b76066373ff165b4 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:36.259767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14265667-fca1-4658-b8ef-99e30324433d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:36.260695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14265667-fca1-4658-b8ef-99e30324433d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:36.261142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14265667-fca1-4658-b8ef-99e30324433d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:36.305589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-14265667-fca1-4658-b8ef-99e30324433d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:36.309133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-14265667-fca1-4658-b8ef-99e30324433d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:36.376673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f07015-56c2-4e83-880e-02cfe56045db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:36.378572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f07015-56c2-4e83-880e-02cfe56045db admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:36.380117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f07015-56c2-4e83-880e-02cfe56045db admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:36.380416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f07015-56c2-4e83-880e-02cfe56045db admin admin] PATH_INFO: `/v3/projects/ed69b3e55ed64d0bb5a0cd02f2d42d99/users/2be566d906f947f7abedf3dedb733529/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:36.430066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f07015-56c2-4e83-880e-02cfe56045db admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed69b3e55ed64d0bb5a0cd02f2d42d99, user_id=2be566d906f947f7abedf3dedb733529, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:36.430066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f07015-56c2-4e83-880e-02cfe56045db admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:36.454731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:36.467816 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1152/2371] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:36 2022] PUT /identity/v3/projects/ed69b3e55ed64d0bb5a0cd02f2d42d99/users/2be566d906f947f7abedf3dedb733529/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:50:36.469733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f07015-56c2-4e83-880e-02cfe56045db admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:36.476892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b7f07015-56c2-4e83-880e-02cfe56045db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:36.650142 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1220/2372] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:50:36.658255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094a0d2-abf8-4354-98ef-dd7f639cc797 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:36.659026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094a0d2-abf8-4354-98ef-dd7f639cc797 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:36.661238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094a0d2-abf8-4354-98ef-dd7f639cc797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:36.685289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f07015-56c2-4e83-880e-02cfe56045db admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:36.710548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8ee41-7ba1-4850-a378-1bbba54b035b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:36.715371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8ee41-7ba1-4850-a378-1bbba54b035b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:36.718454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8ee41-7ba1-4850-a378-1bbba54b035b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:36.719954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8ee41-7ba1-4850-a378-1bbba54b035b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:36.723418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8ee41-7ba1-4850-a378-1bbba54b035b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:36.739866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8ee41-7ba1-4850-a378-1bbba54b035b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:36.800004 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1153/2373] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:36.801967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8ee41-7ba1-4850-a378-1bbba54b035b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:36.886456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8ee41-7ba1-4850-a378-1bbba54b035b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:36.938096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f67aa2-1fc2-45df-93aa-62f29b873ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:36.938831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f67aa2-1fc2-45df-93aa-62f29b873ed9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:36.939112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f67aa2-1fc2-45df-93aa-62f29b873ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:36.939388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f67aa2-1fc2-45df-93aa-62f29b873ed9 admin admin] PATH_INFO: `/v3/projects/ed69b3e55ed64d0bb5a0cd02f2d42d99/users/2be566d906f947f7abedf3dedb733529/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:36.948251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f67aa2-1fc2-45df-93aa-62f29b873ed9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed69b3e55ed64d0bb5a0cd02f2d42d99, user_id=2be566d906f947f7abedf3dedb733529, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:36.951011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f67aa2-1fc2-45df-93aa-62f29b873ed9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:37.027805 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1154/2374] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:36 2022] PUT /identity/v3/projects/ed69b3e55ed64d0bb5a0cd02f2d42d99/users/2be566d906f947f7abedf3dedb733529/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:37.057220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f094a0d2-abf8-4354-98ef-dd7f639cc797 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:37.058540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f094a0d2-abf8-4354-98ef-dd7f639cc797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:37.121755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:37.155806 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1221/2375] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 499 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:37.160338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f67aa2-1fc2-45df-93aa-62f29b873ed9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:37.191251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f67aa2-1fc2-45df-93aa-62f29b873ed9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:37.200254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2bc875-95ed-4fe2-86bf-eb7dac712f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:37.203903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2bc875-95ed-4fe2-86bf-eb7dac712f92 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:37.204580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2bc875-95ed-4fe2-86bf-eb7dac712f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:37.205057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2bc875-95ed-4fe2-86bf-eb7dac712f92 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:37.206250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2bc875-95ed-4fe2-86bf-eb7dac712f92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:50:37.207291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2bc875-95ed-4fe2-86bf-eb7dac712f92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:37.209764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2bc875-95ed-4fe2-86bf-eb7dac712f92 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:37.234670 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1155/2376] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:50:37.238724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094a0d2-abf8-4354-98ef-dd7f639cc797 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:37.258474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094a0d2-abf8-4354-98ef-dd7f639cc797 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:37.275258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d254d-b50d-429e-976d-e8daa0cfd2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:37.275902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d254d-b50d-429e-976d-e8daa0cfd2ec admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:37.276198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d254d-b50d-429e-976d-e8daa0cfd2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:37.276677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d254d-b50d-429e-976d-e8daa0cfd2ec admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:37.277246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d254d-b50d-429e-976d-e8daa0cfd2ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:37.281495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d254d-b50d-429e-976d-e8daa0cfd2ec admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:37.329364 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1222/2377] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:50:37 2022] POST /identity/v3/projects => generated 411 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:37.333832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2bc875-95ed-4fe2-86bf-eb7dac712f92 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:37.349436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2bc875-95ed-4fe2-86bf-eb7dac712f92 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:37.469337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9caaf1f-b463-40cf-9278-8ead4c489487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:37.469892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9caaf1f-b463-40cf-9278-8ead4c489487 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:37.470178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9caaf1f-b463-40cf-9278-8ead4c489487 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:37.470548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9caaf1f-b463-40cf-9278-8ead4c489487 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:37.471391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9caaf1f-b463-40cf-9278-8ead4c489487 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:37.473355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9caaf1f-b463-40cf-9278-8ead4c489487 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:37.529138 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1156/2378] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:50:37 2022] POST /identity/v3/users => generated 306 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:37.533562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d254d-b50d-429e-976d-e8daa0cfd2ec admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:37.552025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d254d-b50d-429e-976d-e8daa0cfd2ec admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:37.566842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8935f7-19f6-40a1-917d-162dda56d7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:37.567470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8935f7-19f6-40a1-917d-162dda56d7ef admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:37.567743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8935f7-19f6-40a1-917d-162dda56d7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:37.568007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8935f7-19f6-40a1-917d-162dda56d7ef admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:37.568657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8935f7-19f6-40a1-917d-162dda56d7ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:37.570248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8935f7-19f6-40a1-917d-162dda56d7ef admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:37.588689 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1223/2379] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:37.592695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9caaf1f-b463-40cf-9278-8ead4c489487 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:37.623374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9caaf1f-b463-40cf-9278-8ead4c489487 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:37.635431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3931e5-6984-45b9-a8bd-4aeaa7488f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:37.636460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3931e5-6984-45b9-a8bd-4aeaa7488f6c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:37.638090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3931e5-6984-45b9-a8bd-4aeaa7488f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:37.638625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3931e5-6984-45b9-a8bd-4aeaa7488f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:37.640773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3931e5-6984-45b9-a8bd-4aeaa7488f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:37.642693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3931e5-6984-45b9-a8bd-4aeaa7488f6c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:37.683997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8935f7-19f6-40a1-917d-162dda56d7ef admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:37.686569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4d8935f7-19f6-40a1-917d-162dda56d7ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:37.705168 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1157/2380] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:37.710711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3931e5-6984-45b9-a8bd-4aeaa7488f6c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:37.714267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8935f7-19f6-40a1-917d-162dda56d7ef admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:37.736729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455e989-e1c5-42c5-90f0-3e4830b84846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:37.742356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455e989-e1c5-42c5-90f0-3e4830b84846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:37.744446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3931e5-6984-45b9-a8bd-4aeaa7488f6c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:37.745068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455e989-e1c5-42c5-90f0-3e4830b84846 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:37.745562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455e989-e1c5-42c5-90f0-3e4830b84846 admin admin] PATH_INFO: `/v3/users/0e76f88bb60c4f4eb2657a929098f991` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:37.750152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a455e989-e1c5-42c5-90f0-3e4830b84846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e76f88bb60c4f4eb2657a929098f991)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:37.754472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a455e989-e1c5-42c5-90f0-3e4830b84846 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:37.761581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729acae-df14-4cc7-b33d-c51f90910e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:37.762627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729acae-df14-4cc7-b33d-c51f90910e0f admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:37.763130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729acae-df14-4cc7-b33d-c51f90910e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:37.763651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729acae-df14-4cc7-b33d-c51f90910e0f admin admin] PATH_INFO: `/v3/projects/bbd3b538b14a4f77a6a50e9fbe849427/users/50d13e9e2afc44fd91caf096922460ed/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:37.797878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729acae-df14-4cc7-b33d-c51f90910e0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbd3b538b14a4f77a6a50e9fbe849427, user_id=50d13e9e2afc44fd91caf096922460ed, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:37.801165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729acae-df14-4cc7-b33d-c51f90910e0f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:37.845097 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1158/2381] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:37 2022] PUT /identity/v3/projects/bbd3b538b14a4f77a6a50e9fbe849427/users/50d13e9e2afc44fd91caf096922460ed/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:37.853642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37999e0e-34c0-4ac1-ac62-541effc7c227 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:37.854256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37999e0e-34c0-4ac1-ac62-541effc7c227 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:37.855252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37999e0e-34c0-4ac1-ac62-541effc7c227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:37.884116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a455e989-e1c5-42c5-90f0-3e4830b84846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e76f88bb60c4f4eb2657a929098f991'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:37.907333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a455e989-e1c5-42c5-90f0-3e4830b84846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e76f88bb60c4f4eb2657a929098f991'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:37.922550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-37999e0e-34c0-4ac1-ac62-541effc7c227 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:37.925176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-37999e0e-34c0-4ac1-ac62-541effc7c227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:37.972932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a455e989-e1c5-42c5-90f0-3e4830b84846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e76f88bb60c4f4eb2657a929098f991'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:37.982310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:37.982607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a455e989-e1c5-42c5-90f0-3e4830b84846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e76f88bb60c4f4eb2657a929098f991'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:38.010476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a455e989-e1c5-42c5-90f0-3e4830b84846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e76f88bb60c4f4eb2657a929098f991'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:38.046585 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1224/2382] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:50:37 2022] DELETE /identity/v3/users/0e76f88bb60c4f4eb2657a929098f991 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:38.112620 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1159/2383] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:38.138027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455e989-e1c5-42c5-90f0-3e4830b84846 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:38.139654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a455e989-e1c5-42c5-90f0-3e4830b84846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:38.345028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455e989-e1c5-42c5-90f0-3e4830b84846 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:38.357695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b90883-d1bd-4178-a34e-a7edb95daeec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:38.359403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b90883-d1bd-4178-a34e-a7edb95daeec service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:38.360588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b90883-d1bd-4178-a34e-a7edb95daeec service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:38.364684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b90883-d1bd-4178-a34e-a7edb95daeec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:38.457963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:38.469622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37999e0e-34c0-4ac1-ac62-541effc7c227 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:38.471111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-37999e0e-34c0-4ac1-ac62-541effc7c227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:38.499059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37999e0e-34c0-4ac1-ac62-541effc7c227 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:38.516187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7606d-ce65-4fe8-9368-dc2b82c2d0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:38.516691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7606d-ce65-4fe8-9368-dc2b82c2d0d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:38.516819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7606d-ce65-4fe8-9368-dc2b82c2d0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:38.517101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7606d-ce65-4fe8-9368-dc2b82c2d0d8 admin admin] PATH_INFO: `/v3/projects/3b1e74abd17d4102b622bf3ec2d4f715` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:38.527397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a7606d-ce65-4fe8-9368-dc2b82c2d0d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b1e74abd17d4102b622bf3ec2d4f715)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:38.529456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a7606d-ce65-4fe8-9368-dc2b82c2d0d8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:38.529900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b90883-d1bd-4178-a34e-a7edb95daeec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:38.535446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b90883-d1bd-4178-a34e-a7edb95daeec service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:38.575436 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1225/2384] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:38.653815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b8a7606d-ce65-4fe8-9368-dc2b82c2d0d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b1e74abd17d4102b622bf3ec2d4f715', 'request_id': 'req-b8a7606d-ce65-4fe8-9368-dc2b82c2d0d8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:38.670429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b8a7606d-ce65-4fe8-9368-dc2b82c2d0d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b1e74abd17d4102b622bf3ec2d4f715', 'request_id': 'req-b8a7606d-ce65-4fe8-9368-dc2b82c2d0d8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:38.679470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b8a7606d-ce65-4fe8-9368-dc2b82c2d0d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b1e74abd17d4102b622bf3ec2d4f715', 'request_id': 'req-b8a7606d-ce65-4fe8-9368-dc2b82c2d0d8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:38.707695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b8a7606d-ce65-4fe8-9368-dc2b82c2d0d8 admin admin] The token cache is being invalidate because project 3b1e74abd17d4102b622bf3ec2d4f715 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:50:38.708409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b8a7606d-ce65-4fe8-9368-dc2b82c2d0d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:38.709525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b8a7606d-ce65-4fe8-9368-dc2b82c2d0d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:38.713974 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1160/2385] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:50:38 2022] DELETE /identity/v3/projects/3b1e74abd17d4102b622bf3ec2d4f715 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:50:38.719365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a17a9d-d9d3-4b98-b925-f0e8b0137394 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:38.720529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a17a9d-d9d3-4b98-b925-f0e8b0137394 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:38.721087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a17a9d-d9d3-4b98-b925-f0e8b0137394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:38.772535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-18a17a9d-d9d3-4b98-b925-f0e8b0137394 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:38.774042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-18a17a9d-d9d3-4b98-b925-f0e8b0137394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:38.818129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7606d-ce65-4fe8-9368-dc2b82c2d0d8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:38.820328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b8a7606d-ce65-4fe8-9368-dc2b82c2d0d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:39.014587 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1226/2386] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:39.023833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d4ef8-9f24-4152-bd65-4e975ebd0cd9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:39.024174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d4ef8-9f24-4152-bd65-4e975ebd0cd9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:39.024391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d4ef8-9f24-4152-bd65-4e975ebd0cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:39.083894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7606d-ce65-4fe8-9368-dc2b82c2d0d8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:39.090569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7d2d4ef8-9f24-4152-bd65-4e975ebd0cd9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:39.092079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7d2d4ef8-9f24-4152-bd65-4e975ebd0cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:39.098752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e32df-8373-4609-943c-8c1f4ca7f9d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:39.099383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e32df-8373-4609-943c-8c1f4ca7f9d3 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:39.099661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e32df-8373-4609-943c-8c1f4ca7f9d3 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:39.099931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e32df-8373-4609-943c-8c1f4ca7f9d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:39.106207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e29e32df-8373-4609-943c-8c1f4ca7f9d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:39.108117 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1227/2387] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:39.109880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d4ef8-9f24-4152-bd65-4e975ebd0cd9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:39.143215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d4ef8-9f24-4152-bd65-4e975ebd0cd9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:39.160899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50212c1-60d6-4f52-96b2-42a4a96d4755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:39.161802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50212c1-60d6-4f52-96b2-42a4a96d4755 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:39.162184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50212c1-60d6-4f52-96b2-42a4a96d4755 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:39.163249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50212c1-60d6-4f52-96b2-42a4a96d4755 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:39.164063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50212c1-60d6-4f52-96b2-42a4a96d4755 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:50:39.165422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50212c1-60d6-4f52-96b2-42a4a96d4755 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:39.170603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50212c1-60d6-4f52-96b2-42a4a96d4755 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:39.211526 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1228/2388] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:50:39.214337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50212c1-60d6-4f52-96b2-42a4a96d4755 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:39.227938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:39.236611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50212c1-60d6-4f52-96b2-42a4a96d4755 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:39.255782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e32df-8373-4609-943c-8c1f4ca7f9d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:39.259235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e32df-8373-4609-943c-8c1f4ca7f9d3 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:39.296148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1aa7c-bd59-4175-8fa1-0863d1591047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:39.296780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1aa7c-bd59-4175-8fa1-0863d1591047 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:39.297179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1aa7c-bd59-4175-8fa1-0863d1591047 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:39.297549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1aa7c-bd59-4175-8fa1-0863d1591047 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:39.304910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c1aa7c-bd59-4175-8fa1-0863d1591047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:39.307424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c1aa7c-bd59-4175-8fa1-0863d1591047 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:39.325519 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1161/2389] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:39.425609 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1229/2390] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:50:39 2022] POST /identity/v3/projects => generated 387 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:39.435602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e32df-8373-4609-943c-8c1f4ca7f9d3 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:39.479106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e32df-8373-4609-943c-8c1f4ca7f9d3 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:39.718930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755ba26-4dbc-4eee-9a81-bb6a669acf3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:39.719736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755ba26-4dbc-4eee-9a81-bb6a669acf3c admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:39.720128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755ba26-4dbc-4eee-9a81-bb6a669acf3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:39.720523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755ba26-4dbc-4eee-9a81-bb6a669acf3c admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:39.721371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755ba26-4dbc-4eee-9a81-bb6a669acf3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:39.723474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755ba26-4dbc-4eee-9a81-bb6a669acf3c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:39.766107 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1162/2391] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:50:39 2022] POST /identity/v3/users => generated 294 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:39.767907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1aa7c-bd59-4175-8fa1-0863d1591047 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:39.780369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1aa7c-bd59-4175-8fa1-0863d1591047 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:39.786197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aabeab-ccde-43bc-bb91-fe35624a6756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:39.787087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aabeab-ccde-43bc-bb91-fe35624a6756 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:39.787538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aabeab-ccde-43bc-bb91-fe35624a6756 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:39.788694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aabeab-ccde-43bc-bb91-fe35624a6756 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:39.789909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aabeab-ccde-43bc-bb91-fe35624a6756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:39.791913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aabeab-ccde-43bc-bb91-fe35624a6756 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:39.810331 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1230/2392] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:39.811554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755ba26-4dbc-4eee-9a81-bb6a669acf3c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:39.833325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755ba26-4dbc-4eee-9a81-bb6a669acf3c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:39.854288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665d446-1e6b-4f1c-957e-6f0d71448498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:39.855447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665d446-1e6b-4f1c-957e-6f0d71448498 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:39.856263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665d446-1e6b-4f1c-957e-6f0d71448498 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:39.858530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665d446-1e6b-4f1c-957e-6f0d71448498 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:39.862079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665d446-1e6b-4f1c-957e-6f0d71448498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:39.865826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665d446-1e6b-4f1c-957e-6f0d71448498 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:39.883876 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1163/2393] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:39.885463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aabeab-ccde-43bc-bb91-fe35624a6756 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:39.900401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aabeab-ccde-43bc-bb91-fe35624a6756 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:39.916724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5bb16-6571-4f51-83d1-6c9fa9c8c7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:39.917876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5bb16-6571-4f51-83d1-6c9fa9c8c7f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:39.918336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5bb16-6571-4f51-83d1-6c9fa9c8c7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:39.918809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5bb16-6571-4f51-83d1-6c9fa9c8c7f6 admin admin] PATH_INFO: `/v3/projects/bb36a4c325b54c8a964113ba5e52f47b/users/2aa4f468a15546eb9f72cb2b632e67bb/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:39.954373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5bb16-6571-4f51-83d1-6c9fa9c8c7f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb36a4c325b54c8a964113ba5e52f47b, user_id=2aa4f468a15546eb9f72cb2b632e67bb, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:39.965523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5bb16-6571-4f51-83d1-6c9fa9c8c7f6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:39.992481 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1231/2394] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:39 2022] PUT /identity/v3/projects/bb36a4c325b54c8a964113ba5e52f47b/users/2aa4f468a15546eb9f72cb2b632e67bb/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:39.998300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589b828-9e4c-43d0-98d9-e5cf1b129093 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:39.998680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589b828-9e4c-43d0-98d9-e5cf1b129093 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:39.998879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589b828-9e4c-43d0-98d9-e5cf1b129093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:40.083048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0589b828-9e4c-43d0-98d9-e5cf1b129093 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:40.085417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0589b828-9e4c-43d0-98d9-e5cf1b129093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:40.267037 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1164/2395] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:40.361284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5bb16-6571-4f51-83d1-6c9fa9c8c7f6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:40.367458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-82a5bb16-6571-4f51-83d1-6c9fa9c8c7f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:40.531742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5bb16-6571-4f51-83d1-6c9fa9c8c7f6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:40.551833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c7ce6-28ce-4682-894c-20001414f0bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:40.552713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c7ce6-28ce-4682-894c-20001414f0bf service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:40.553004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c7ce6-28ce-4682-894c-20001414f0bf service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:40.559606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c7ce6-28ce-4682-894c-20001414f0bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:40.582299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c7ce6-28ce-4682-894c-20001414f0bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:40.584065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c7ce6-28ce-4682-894c-20001414f0bf service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:40.615591 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1232/2396] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:43.342658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589b828-9e4c-43d0-98d9-e5cf1b129093 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:43.344420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0589b828-9e4c-43d0-98d9-e5cf1b129093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:43.372512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589b828-9e4c-43d0-98d9-e5cf1b129093 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:43.381160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0b8e1-883d-4549-bd4c-5c15f1cb4659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:43.382806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0b8e1-883d-4549-bd4c-5c15f1cb4659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:43.383223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0b8e1-883d-4549-bd4c-5c15f1cb4659 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:43.383572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0b8e1-883d-4549-bd4c-5c15f1cb4659 admin admin] PATH_INFO: `/v3/users/b99f9f102b064fd68a8222a23948c813` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:43.384959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa0b8e1-883d-4549-bd4c-5c15f1cb4659 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b99f9f102b064fd68a8222a23948c813)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:43.387074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa0b8e1-883d-4549-bd4c-5c15f1cb4659 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:43.462308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1fa0b8e1-883d-4549-bd4c-5c15f1cb4659 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b99f9f102b064fd68a8222a23948c813'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:43.464235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1fa0b8e1-883d-4549-bd4c-5c15f1cb4659 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b99f9f102b064fd68a8222a23948c813'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:43.465014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1fa0b8e1-883d-4549-bd4c-5c15f1cb4659 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b99f9f102b064fd68a8222a23948c813'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:43.475959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1fa0b8e1-883d-4549-bd4c-5c15f1cb4659 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b99f9f102b064fd68a8222a23948c813'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:43.482778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:43.502850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1fa0b8e1-883d-4549-bd4c-5c15f1cb4659 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b99f9f102b064fd68a8222a23948c813'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:43.526529 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1165/2397] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:50:43 2022] DELETE /identity/v3/users/b99f9f102b064fd68a8222a23948c813 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:43.528341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb950447-501d-4a83-9a70-f1c3e1d43cec None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:43.528864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb950447-501d-4a83-9a70-f1c3e1d43cec None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:43.528969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb950447-501d-4a83-9a70-f1c3e1d43cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:43.536125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:43.550435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fb950447-501d-4a83-9a70-f1c3e1d43cec None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:43.552361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fb950447-501d-4a83-9a70-f1c3e1d43cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:43.655478 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1233/2398] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:43.669435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0b8e1-883d-4549-bd4c-5c15f1cb4659 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:43.671033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1fa0b8e1-883d-4549-bd4c-5c15f1cb4659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:43.826346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0b8e1-883d-4549-bd4c-5c15f1cb4659 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:43.832320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64535b35-af9d-413a-9bd6-e1e2d99be21f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:43.834324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64535b35-af9d-413a-9bd6-e1e2d99be21f service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:43.834845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64535b35-af9d-413a-9bd6-e1e2d99be21f service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:43.835270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64535b35-af9d-413a-9bd6-e1e2d99be21f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:43.851594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64535b35-af9d-413a-9bd6-e1e2d99be21f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:43.854133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64535b35-af9d-413a-9bd6-e1e2d99be21f service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:43.885673 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1166/2399] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:45.475197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0759671-1f80-4289-ae41-109429ce24bf None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:45.476185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0759671-1f80-4289-ae41-109429ce24bf None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:45.476634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0759671-1f80-4289-ae41-109429ce24bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:45.518751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e0759671-1f80-4289-ae41-109429ce24bf None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:45.522942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e0759671-1f80-4289-ae41-109429ce24bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:45.686159 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1234/2400] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 20:50:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:45.701877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64535b35-af9d-413a-9bd6-e1e2d99be21f service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:45.704235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-64535b35-af9d-413a-9bd6-e1e2d99be21f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:45.871481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64535b35-af9d-413a-9bd6-e1e2d99be21f service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:45.910322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c98154-64ab-47d8-8252-a4787cbc57a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:45.911017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c98154-64ab-47d8-8252-a4787cbc57a7 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:45.911299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c98154-64ab-47d8-8252-a4787cbc57a7 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:45.911575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c98154-64ab-47d8-8252-a4787cbc57a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:45.924934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c98154-64ab-47d8-8252-a4787cbc57a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:45.927757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c98154-64ab-47d8-8252-a4787cbc57a7 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:45.945601 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1167/2401] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:50:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 20:50:48.110253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0759671-1f80-4289-ae41-109429ce24bf None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:48.114805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e0759671-1f80-4289-ae41-109429ce24bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:48.140128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0759671-1f80-4289-ae41-109429ce24bf None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:48.148523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968724a3-76b8-42f1-bb7a-2c4ae4f85dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:48.149442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968724a3-76b8-42f1-bb7a-2c4ae4f85dbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:48.150142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968724a3-76b8-42f1-bb7a-2c4ae4f85dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:48.150710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968724a3-76b8-42f1-bb7a-2c4ae4f85dbf admin admin] PATH_INFO: `/v3/projects/de2a660566d947f9a2a2977cf3a32d80` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:48.152198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968724a3-76b8-42f1-bb7a-2c4ae4f85dbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=de2a660566d947f9a2a2977cf3a32d80)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:48.155394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968724a3-76b8-42f1-bb7a-2c4ae4f85dbf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:48.248637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-968724a3-76b8-42f1-bb7a-2c4ae4f85dbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de2a660566d947f9a2a2977cf3a32d80', 'request_id': 'req-968724a3-76b8-42f1-bb7a-2c4ae4f85dbf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:48.259391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-968724a3-76b8-42f1-bb7a-2c4ae4f85dbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de2a660566d947f9a2a2977cf3a32d80', 'request_id': 'req-968724a3-76b8-42f1-bb7a-2c4ae4f85dbf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:48.292676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-968724a3-76b8-42f1-bb7a-2c4ae4f85dbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de2a660566d947f9a2a2977cf3a32d80', 'request_id': 'req-968724a3-76b8-42f1-bb7a-2c4ae4f85dbf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:48.318721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-968724a3-76b8-42f1-bb7a-2c4ae4f85dbf admin admin] The token cache is being invalidate because project de2a660566d947f9a2a2977cf3a32d80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:50:48.319925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-968724a3-76b8-42f1-bb7a-2c4ae4f85dbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:48.321232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-968724a3-76b8-42f1-bb7a-2c4ae4f85dbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:48.325646 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1235/2402] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:50:48 2022] DELETE /identity/v3/projects/de2a660566d947f9a2a2977cf3a32d80 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:48.329697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c98154-64ab-47d8-8252-a4787cbc57a7 service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:48.367552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c98154-64ab-47d8-8252-a4787cbc57a7 service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:48.548168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009c78d-f51a-477c-83e2-8babf46db03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:48.548852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009c78d-f51a-477c-83e2-8babf46db03b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:48.549119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009c78d-f51a-477c-83e2-8babf46db03b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:48.549391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009c78d-f51a-477c-83e2-8babf46db03b admin admin] PATH_INFO: `/v3/users/27e519e61f0d4ce19a06021f6c67fb0f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:48.550593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009c78d-f51a-477c-83e2-8babf46db03b admin admin] RBAC: Authorizing `identity:delete_user(user_id=27e519e61f0d4ce19a06021f6c67fb0f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:48.552543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009c78d-f51a-477c-83e2-8babf46db03b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:48.581155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:48.682111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b009c78d-f51a-477c-83e2-8babf46db03b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27e519e61f0d4ce19a06021f6c67fb0f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:48.687184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b009c78d-f51a-477c-83e2-8babf46db03b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27e519e61f0d4ce19a06021f6c67fb0f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:48.687915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b009c78d-f51a-477c-83e2-8babf46db03b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27e519e61f0d4ce19a06021f6c67fb0f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:48.711793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b009c78d-f51a-477c-83e2-8babf46db03b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27e519e61f0d4ce19a06021f6c67fb0f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:48.749525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b009c78d-f51a-477c-83e2-8babf46db03b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27e519e61f0d4ce19a06021f6c67fb0f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:48.808153 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1168/2403] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:50:48 2022] DELETE /identity/v3/users/27e519e61f0d4ce19a06021f6c67fb0f => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:49.948722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67aa80f-0e88-4e02-93f3-91148f1decbb None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:49.949253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67aa80f-0e88-4e02-93f3-91148f1decbb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:49.949253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67aa80f-0e88-4e02-93f3-91148f1decbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:49.985677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b67aa80f-0e88-4e02-93f3-91148f1decbb None None] MFA Rules not processed for user `2aa4f468a15546eb9f72cb2b632e67bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:49.987948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b67aa80f-0e88-4e02-93f3-91148f1decbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:50.051563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009c78d-f51a-477c-83e2-8babf46db03b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:50.053015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b009c78d-f51a-477c-83e2-8babf46db03b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:50.054163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:50.201854 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1236/2404] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:50.210748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67aa80f-0e88-4e02-93f3-91148f1decbb None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:50.212653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b67aa80f-0e88-4e02-93f3-91148f1decbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:50.232004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009c78d-f51a-477c-83e2-8babf46db03b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:50.243039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26438d86-f0ec-4cca-9319-27c01087c52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:50.244522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26438d86-f0ec-4cca-9319-27c01087c52f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:50.244950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26438d86-f0ec-4cca-9319-27c01087c52f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:50.245366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26438d86-f0ec-4cca-9319-27c01087c52f admin admin] PATH_INFO: `/v3/projects/979c07d50d2f456a9fd6b8a56ffa5d34` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:50.245876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67aa80f-0e88-4e02-93f3-91148f1decbb None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:50.249243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26438d86-f0ec-4cca-9319-27c01087c52f admin admin] RBAC: Authorizing `identity:delete_project(project_id=979c07d50d2f456a9fd6b8a56ffa5d34)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:50.251752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb991115-ca08-4853-a153-949f8aa5ea97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:50.252999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb991115-ca08-4853-a153-949f8aa5ea97 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:50.253456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb991115-ca08-4853-a153-949f8aa5ea97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:50.253897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb991115-ca08-4853-a153-949f8aa5ea97 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:50.254702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26438d86-f0ec-4cca-9319-27c01087c52f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:50.255059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb991115-ca08-4853-a153-949f8aa5ea97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:50.261349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb991115-ca08-4853-a153-949f8aa5ea97 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:50.316708 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1237/2405] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:50:50 2022] POST /identity/v3/projects => generated 389 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:50.319501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb991115-ca08-4853-a153-949f8aa5ea97 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:50.348300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-26438d86-f0ec-4cca-9319-27c01087c52f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '979c07d50d2f456a9fd6b8a56ffa5d34', 'request_id': 'req-26438d86-f0ec-4cca-9319-27c01087c52f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:50.367602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-26438d86-f0ec-4cca-9319-27c01087c52f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '979c07d50d2f456a9fd6b8a56ffa5d34', 'request_id': 'req-26438d86-f0ec-4cca-9319-27c01087c52f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:50.380300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-26438d86-f0ec-4cca-9319-27c01087c52f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '979c07d50d2f456a9fd6b8a56ffa5d34', 'request_id': 'req-26438d86-f0ec-4cca-9319-27c01087c52f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:50.396035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-26438d86-f0ec-4cca-9319-27c01087c52f admin admin] The token cache is being invalidate because project 979c07d50d2f456a9fd6b8a56ffa5d34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:50:50.396686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-26438d86-f0ec-4cca-9319-27c01087c52f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:50.398774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-26438d86-f0ec-4cca-9319-27c01087c52f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:50.400864 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1169/2406] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:50:50 2022] DELETE /identity/v3/projects/979c07d50d2f456a9fd6b8a56ffa5d34 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:50.407217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33590d-490d-4bec-a2a6-3ef11f38a82d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:50.407782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33590d-490d-4bec-a2a6-3ef11f38a82d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:50.408826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33590d-490d-4bec-a2a6-3ef11f38a82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:50.414540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb991115-ca08-4853-a153-949f8aa5ea97 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:50.437456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7a33590d-490d-4bec-a2a6-3ef11f38a82d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:50.438917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7a33590d-490d-4bec-a2a6-3ef11f38a82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:50.459999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:50.529163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08b935-3800-4821-b040-8c45dcdb61de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:50.530032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08b935-3800-4821-b040-8c45dcdb61de admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:50.530521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08b935-3800-4821-b040-8c45dcdb61de admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:50.530947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08b935-3800-4821-b040-8c45dcdb61de admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:50.532053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08b935-3800-4821-b040-8c45dcdb61de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:50.534236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08b935-3800-4821-b040-8c45dcdb61de admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:50.558367 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1170/2407] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:50.562553 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1238/2408] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:50:50 2022] POST /identity/v3/users => generated 301 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:50:50.572674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790cdbb-665c-4b62-b2d6-3f98fa69307e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:50.572674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790cdbb-665c-4b62-b2d6-3f98fa69307e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:50.573007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790cdbb-665c-4b62-b2d6-3f98fa69307e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:50.578863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08b935-3800-4821-b040-8c45dcdb61de admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:50.583445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9b08b935-3800-4821-b040-8c45dcdb61de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:50.606591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1790cdbb-665c-4b62-b2d6-3f98fa69307e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:50.606591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1790cdbb-665c-4b62-b2d6-3f98fa69307e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:50.636807 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1171/2409] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:50.637016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08b935-3800-4821-b040-8c45dcdb61de admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:50.637919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790cdbb-665c-4b62-b2d6-3f98fa69307e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:50.658908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7dee9-297b-476f-bdfd-2740bf42205f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:50.659520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7dee9-297b-476f-bdfd-2740bf42205f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:50.659804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7dee9-297b-476f-bdfd-2740bf42205f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:50.660080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7dee9-297b-476f-bdfd-2740bf42205f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:50.660734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7dee9-297b-476f-bdfd-2740bf42205f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:50.663748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790cdbb-665c-4b62-b2d6-3f98fa69307e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:50.665869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7dee9-297b-476f-bdfd-2740bf42205f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:50.675032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e485ebf-d606-40e1-afb0-e1248fe1f167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:50.675652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e485ebf-d606-40e1-afb0-e1248fe1f167 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:50.675929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e485ebf-d606-40e1-afb0-e1248fe1f167 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:50.676201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e485ebf-d606-40e1-afb0-e1248fe1f167 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:50.676856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e485ebf-d606-40e1-afb0-e1248fe1f167 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:50:50.677157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e485ebf-d606-40e1-afb0-e1248fe1f167 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:50.678829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e485ebf-d606-40e1-afb0-e1248fe1f167 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:50.687337 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1239/2410] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:50.696654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7dee9-297b-476f-bdfd-2740bf42205f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:50.715510 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1172/2411] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:50:50.715751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7dee9-297b-476f-bdfd-2740bf42205f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:50.716677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e485ebf-d606-40e1-afb0-e1248fe1f167 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:50.728916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e485ebf-d606-40e1-afb0-e1248fe1f167 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:50.730164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a1b28-476b-4bd3-b319-1cd61b89c61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:50.731339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a1b28-476b-4bd3-b319-1cd61b89c61a admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:50.733394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a1b28-476b-4bd3-b319-1cd61b89c61a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:50.734055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a1b28-476b-4bd3-b319-1cd61b89c61a admin admin] PATH_INFO: `/v3/projects/520ed1a4ca304a9b9a47c5a9908bcb0b/users/85ba386a1dad4f43b6eb3f36b588c09e/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:50.736608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6415f6-6178-44a1-8a50-c8d14d36c718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:50.737227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6415f6-6178-44a1-8a50-c8d14d36c718 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:50.737611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6415f6-6178-44a1-8a50-c8d14d36c718 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:50.737889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6415f6-6178-44a1-8a50-c8d14d36c718 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:50.738749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6415f6-6178-44a1-8a50-c8d14d36c718 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:50.741369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6415f6-6178-44a1-8a50-c8d14d36c718 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:50.778095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a1b28-476b-4bd3-b319-1cd61b89c61a admin admin] RBAC: Authorizing `identity:create_grant(project_id=520ed1a4ca304a9b9a47c5a9908bcb0b, user_id=85ba386a1dad4f43b6eb3f36b588c09e, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:50.784225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a1b28-476b-4bd3-b319-1cd61b89c61a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:50.786764 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1173/2412] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:50:50 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:50.791876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6415f6-6178-44a1-8a50-c8d14d36c718 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:50.806467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:50.826747 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1240/2413] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:50 2022] PUT /identity/v3/projects/520ed1a4ca304a9b9a47c5a9908bcb0b/users/85ba386a1dad4f43b6eb3f36b588c09e/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:50.827852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a1b28-476b-4bd3-b319-1cd61b89c61a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:50.932492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6415f6-6178-44a1-8a50-c8d14d36c718 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:50.958190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a1b28-476b-4bd3-b319-1cd61b89c61a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:50.966428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8150de-f574-4ad9-99b5-5ffb0b55aec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:50.968090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8150de-f574-4ad9-99b5-5ffb0b55aec7 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:50.968713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8150de-f574-4ad9-99b5-5ffb0b55aec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:50.970656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8150de-f574-4ad9-99b5-5ffb0b55aec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:50.973616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8150de-f574-4ad9-99b5-5ffb0b55aec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:50.977841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8150de-f574-4ad9-99b5-5ffb0b55aec7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:51.002224 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1241/2414] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:51.004846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8150de-f574-4ad9-99b5-5ffb0b55aec7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:51.036454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f8e9d-772f-4b1f-8ea2-5f49084566b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:51.044448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f8e9d-772f-4b1f-8ea2-5f49084566b7 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:51.044448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f8e9d-772f-4b1f-8ea2-5f49084566b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:51.048898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8150de-f574-4ad9-99b5-5ffb0b55aec7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:51.054638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f8e9d-772f-4b1f-8ea2-5f49084566b7 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:51.055452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f8e9d-772f-4b1f-8ea2-5f49084566b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:51.056217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339c132-b744-49a8-9267-58d328fd05b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:51.057164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339c132-b744-49a8-9267-58d328fd05b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:51.057426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f8e9d-772f-4b1f-8ea2-5f49084566b7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:51.057758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339c132-b744-49a8-9267-58d328fd05b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:51.058176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339c132-b744-49a8-9267-58d328fd05b7 admin admin] PATH_INFO: `/v3/projects/520ed1a4ca304a9b9a47c5a9908bcb0b/users/85ba386a1dad4f43b6eb3f36b588c09e/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:51.060616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7339c132-b744-49a8-9267-58d328fd05b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=520ed1a4ca304a9b9a47c5a9908bcb0b, user_id=85ba386a1dad4f43b6eb3f36b588c09e, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:51.063746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7339c132-b744-49a8-9267-58d328fd05b7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:51.083076 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1242/2415] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:51 2022] PUT /identity/v3/projects/520ed1a4ca304a9b9a47c5a9908bcb0b/users/85ba386a1dad4f43b6eb3f36b588c09e/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:51.091396 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1174/2416] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:50:50 2022] POST /identity/v3/users => generated 292 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:51.096084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339c132-b744-49a8-9267-58d328fd05b7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:51.174468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339c132-b744-49a8-9267-58d328fd05b7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:51.181214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327bbb9-cfac-4380-8528-d6757f219b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:51.181843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327bbb9-cfac-4380-8528-d6757f219b04 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:51.182124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327bbb9-cfac-4380-8528-d6757f219b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:51.182587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327bbb9-cfac-4380-8528-d6757f219b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:51.183260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f327bbb9-cfac-4380-8528-d6757f219b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:51.185328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f327bbb9-cfac-4380-8528-d6757f219b04 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:51.212046 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1243/2417] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:51.217167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f8e9d-772f-4b1f-8ea2-5f49084566b7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:51.233351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f8e9d-772f-4b1f-8ea2-5f49084566b7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:51.241497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62a3f0-7920-4847-8bf1-1c60e43de1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:51.242483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62a3f0-7920-4847-8bf1-1c60e43de1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:51.243115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62a3f0-7920-4847-8bf1-1c60e43de1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:51.243687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62a3f0-7920-4847-8bf1-1c60e43de1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:51.244544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f62a3f0-7920-4847-8bf1-1c60e43de1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:51.246512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f62a3f0-7920-4847-8bf1-1c60e43de1a9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:51.275272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327bbb9-cfac-4380-8528-d6757f219b04 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:51.288228 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1175/2418] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:51.293506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327bbb9-cfac-4380-8528-d6757f219b04 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:51.302070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a53b0-fc3c-43f1-afd9-edeab2877c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:51.303096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a53b0-fc3c-43f1-afd9-edeab2877c36 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:51.303635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a53b0-fc3c-43f1-afd9-edeab2877c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:51.304063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a53b0-fc3c-43f1-afd9-edeab2877c36 admin admin] PATH_INFO: `/v3/projects/a993171008c84bdaa3f81aacd4d4e94f/users/137cdb175c6549939c41d5b42ce5c19d/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:51.326237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a53b0-fc3c-43f1-afd9-edeab2877c36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a993171008c84bdaa3f81aacd4d4e94f, user_id=137cdb175c6549939c41d5b42ce5c19d, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:51.329330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a53b0-fc3c-43f1-afd9-edeab2877c36 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:51.386143 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1244/2419] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:51 2022] PUT /identity/v3/projects/a993171008c84bdaa3f81aacd4d4e94f/users/137cdb175c6549939c41d5b42ce5c19d/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:51.401430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13c27b-32ea-457b-801c-98b2875cf06b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:51.401797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13c27b-32ea-457b-801c-98b2875cf06b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:51.402146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13c27b-32ea-457b-801c-98b2875cf06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:51.413788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:51.462954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8a13c27b-32ea-457b-801c-98b2875cf06b None None] MFA Rules not processed for user `137cdb175c6549939c41d5b42ce5c19d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:51.466439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8a13c27b-32ea-457b-801c-98b2875cf06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:51.621785 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1176/2420] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:51.622885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a53b0-fc3c-43f1-afd9-edeab2877c36 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:51.771258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:51.791591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a53b0-fc3c-43f1-afd9-edeab2877c36 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:51.813029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7e8a0-3c4f-4670-9a94-5691210447e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:51.813814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7e8a0-3c4f-4670-9a94-5691210447e2 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:51.814205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7e8a0-3c4f-4670-9a94-5691210447e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:51.814641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7e8a0-3c4f-4670-9a94-5691210447e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:51.816407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e7e8a0-3c4f-4670-9a94-5691210447e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:51.818540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e7e8a0-3c4f-4670-9a94-5691210447e2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:51.891895 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1245/2421] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:50:51 2022] POST /identity/v3/projects => generated 383 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:50:51.893481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a13c27b-32ea-457b-801c-98b2875cf06b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:51.910717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a13c27b-32ea-457b-801c-98b2875cf06b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:52.124692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63798c20-bf8e-4a32-9635-290d83f8f963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:52.125966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63798c20-bf8e-4a32-9635-290d83f8f963 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:52.130358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63798c20-bf8e-4a32-9635-290d83f8f963 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:52.130912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63798c20-bf8e-4a32-9635-290d83f8f963 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:52.132070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63798c20-bf8e-4a32-9635-290d83f8f963 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:52.134394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63798c20-bf8e-4a32-9635-290d83f8f963 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:52.174615 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1177/2422] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:50:51 2022] POST /identity/v3/users => generated 298 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:50:52.174807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7e8a0-3c4f-4670-9a94-5691210447e2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:52.189632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7e8a0-3c4f-4670-9a94-5691210447e2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:52.212765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd3ba0-a13d-4276-ab39-d47b594c1776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:52.213759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd3ba0-a13d-4276-ab39-d47b594c1776 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:52.214253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd3ba0-a13d-4276-ab39-d47b594c1776 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:52.214798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd3ba0-a13d-4276-ab39-d47b594c1776 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:52.215673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd3ba0-a13d-4276-ab39-d47b594c1776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:52.221118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd3ba0-a13d-4276-ab39-d47b594c1776 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:52.273876 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1246/2423] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:50:52.280068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63798c20-bf8e-4a32-9635-290d83f8f963 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:52.304931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63798c20-bf8e-4a32-9635-290d83f8f963 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:52.327346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed47a4-36a2-4162-a824-70cfc48e7c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:52.331374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed47a4-36a2-4162-a824-70cfc48e7c89 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:52.332274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed47a4-36a2-4162-a824-70cfc48e7c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:52.333445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed47a4-36a2-4162-a824-70cfc48e7c89 admin admin] PATH_INFO: `/v3/projects/54d73537de704dbcba9db6d0cd53c944/users/2e8bc010e22248b3993b61cb97b362f1/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:52.371003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed47a4-36a2-4162-a824-70cfc48e7c89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54d73537de704dbcba9db6d0cd53c944, user_id=2e8bc010e22248b3993b61cb97b362f1, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:52.377206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed47a4-36a2-4162-a824-70cfc48e7c89 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:52.425509 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1178/2424] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:52 2022] PUT /identity/v3/projects/54d73537de704dbcba9db6d0cd53c944/users/2e8bc010e22248b3993b61cb97b362f1/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:52.431161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd3ba0-a13d-4276-ab39-d47b594c1776 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:52.485590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd3ba0-a13d-4276-ab39-d47b594c1776 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:52.758159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd222eee-61f7-4253-b961-f6c5ff81d148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:52.759570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd222eee-61f7-4253-b961-f6c5ff81d148 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:52.760072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd222eee-61f7-4253-b961-f6c5ff81d148 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:52.761044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd222eee-61f7-4253-b961-f6c5ff81d148 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:52.761937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd222eee-61f7-4253-b961-f6c5ff81d148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:52.765098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd222eee-61f7-4253-b961-f6c5ff81d148 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:52.785209 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1247/2425] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:50:52.789669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed47a4-36a2-4162-a824-70cfc48e7c89 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:52.811229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed47a4-36a2-4162-a824-70cfc48e7c89 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:52.828926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241bdc0-095a-40fe-a1a0-e62a0892235b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:52.829690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241bdc0-095a-40fe-a1a0-e62a0892235b admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:52.829975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241bdc0-095a-40fe-a1a0-e62a0892235b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:52.830254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241bdc0-095a-40fe-a1a0-e62a0892235b admin admin] PATH_INFO: `/v3/projects/54d73537de704dbcba9db6d0cd53c944/users/2e8bc010e22248b3993b61cb97b362f1/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:52.833083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d241bdc0-095a-40fe-a1a0-e62a0892235b admin admin] RBAC: Authorizing `identity:create_grant(project_id=54d73537de704dbcba9db6d0cd53c944, user_id=2e8bc010e22248b3993b61cb97b362f1, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:52.836292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d241bdc0-095a-40fe-a1a0-e62a0892235b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:52.890045 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1179/2426] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:50:52 2022] PUT /identity/v3/projects/54d73537de704dbcba9db6d0cd53c944/users/2e8bc010e22248b3993b61cb97b362f1/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:50:52.899028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbda42-69d4-4341-a6f9-7da767d34fbd None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:52.900091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbda42-69d4-4341-a6f9-7da767d34fbd None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:52.900573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbda42-69d4-4341-a6f9-7da767d34fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:52.977122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-92dbda42-69d4-4341-a6f9-7da767d34fbd None None] MFA Rules not processed for user `2e8bc010e22248b3993b61cb97b362f1`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:52.980674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-92dbda42-69d4-4341-a6f9-7da767d34fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:53.230946 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1248/2427] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:53.863628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241bdc0-095a-40fe-a1a0-e62a0892235b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:53.871110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d241bdc0-095a-40fe-a1a0-e62a0892235b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:53.970765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:54.028924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241bdc0-095a-40fe-a1a0-e62a0892235b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:54.049233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fcbf3-2fad-483a-918f-cd92cefcc48f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:54.050189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fcbf3-2fad-483a-918f-cd92cefcc48f service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:54.050855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fcbf3-2fad-483a-918f-cd92cefcc48f service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:54.051385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fcbf3-2fad-483a-918f-cd92cefcc48f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:54.095039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4fcbf3-2fad-483a-918f-cd92cefcc48f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:54.096564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4fcbf3-2fad-483a-918f-cd92cefcc48f service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:54.122574 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1180/2428] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:54.811506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbda42-69d4-4341-a6f9-7da767d34fbd None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:54.921471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:54.956493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbda42-69d4-4341-a6f9-7da767d34fbd None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:54.963736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8b9cc-ebc0-48b4-9bfb-bf216fa14e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:54.965322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8b9cc-ebc0-48b4-9bfb-bf216fa14e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:54.965764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8b9cc-ebc0-48b4-9bfb-bf216fa14e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:54.968429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8b9cc-ebc0-48b4-9bfb-bf216fa14e76 admin admin] PATH_INFO: `/v3/users/137cdb175c6549939c41d5b42ce5c19d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:54.971856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8b9cc-ebc0-48b4-9bfb-bf216fa14e76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=137cdb175c6549939c41d5b42ce5c19d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:54.983282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8b9cc-ebc0-48b4-9bfb-bf216fa14e76 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:55.097361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96d8b9cc-ebc0-48b4-9bfb-bf216fa14e76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '137cdb175c6549939c41d5b42ce5c19d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:55.115049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96d8b9cc-ebc0-48b4-9bfb-bf216fa14e76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '137cdb175c6549939c41d5b42ce5c19d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:55.152096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96d8b9cc-ebc0-48b4-9bfb-bf216fa14e76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '137cdb175c6549939c41d5b42ce5c19d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:55.153727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96d8b9cc-ebc0-48b4-9bfb-bf216fa14e76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '137cdb175c6549939c41d5b42ce5c19d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:55.170472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96d8b9cc-ebc0-48b4-9bfb-bf216fa14e76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '137cdb175c6549939c41d5b42ce5c19d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:55.186889 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1249/2429] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:50:54 2022] DELETE /identity/v3/users/137cdb175c6549939c41d5b42ce5c19d => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:50:55.191804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e05709-6da2-4d32-bd77-9604bec911fe None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:55.192634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e05709-6da2-4d32-bd77-9604bec911fe None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:55.193061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e05709-6da2-4d32-bd77-9604bec911fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:55.249300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-14e05709-6da2-4d32-bd77-9604bec911fe None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:55.250581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-14e05709-6da2-4d32-bd77-9604bec911fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:55.352244 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1181/2430] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:55.616273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8b9cc-ebc0-48b4-9bfb-bf216fa14e76 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:55.618591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-96d8b9cc-ebc0-48b4-9bfb-bf216fa14e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:55.755918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8b9cc-ebc0-48b4-9bfb-bf216fa14e76 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:55.762295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779c399-8efa-4707-9273-29e68c16c2c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:55.763212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779c399-8efa-4707-9273-29e68c16c2c2 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:55.763754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779c399-8efa-4707-9273-29e68c16c2c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:55.764215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779c399-8efa-4707-9273-29e68c16c2c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:55.779995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779c399-8efa-4707-9273-29e68c16c2c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:55.782022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779c399-8efa-4707-9273-29e68c16c2c2 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:55.806353 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1250/2431] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:50:57.119459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e07d26-488b-4b84-9948-d91298e6b1dc None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:57.120022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e07d26-488b-4b84-9948-d91298e6b1dc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:57.120022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e07d26-488b-4b84-9948-d91298e6b1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:57.215453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e7e07d26-488b-4b84-9948-d91298e6b1dc None None] MFA Rules not processed for user `2be566d906f947f7abedf3dedb733529`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:57.222477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e7e07d26-488b-4b84-9948-d91298e6b1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:57.404615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445ba1e-fa6e-4687-a73a-4346e5130961 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:57.407377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445ba1e-fa6e-4687-a73a-4346e5130961 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:57.412101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445ba1e-fa6e-4687-a73a-4346e5130961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:57.505313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0445ba1e-fa6e-4687-a73a-4346e5130961 None None] MFA Rules not processed for user `50d13e9e2afc44fd91caf096922460ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:57.508130 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1182/2432] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:57 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:57.509121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0445ba1e-fa6e-4687-a73a-4346e5130961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:57.686163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e07d26-488b-4b84-9948-d91298e6b1dc None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:57.691011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e7e07d26-488b-4b84-9948-d91298e6b1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:57.711523 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1251/2433] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:57 2022] POST /identity/v3/auth/tokens => generated 3288 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:50:57.721824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb2963-20e5-4b93-beba-c640fe4d7cbc None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:57.723489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb2963-20e5-4b93-beba-c640fe4d7cbc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:57.723732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb2963-20e5-4b93-beba-c640fe4d7cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:57.731443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e07d26-488b-4b84-9948-d91298e6b1dc None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:57.745146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2be87-68f2-462f-afae-56c091e918b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:57.745928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2be87-68f2-462f-afae-56c091e918b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:57.746485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2be87-68f2-462f-afae-56c091e918b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:57.747010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2be87-68f2-462f-afae-56c091e918b6 admin admin] PATH_INFO: `/v3/projects/a993171008c84bdaa3f81aacd4d4e94f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:57.748387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:57.749804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b2be87-68f2-462f-afae-56c091e918b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a993171008c84bdaa3f81aacd4d4e94f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:57.754731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b2be87-68f2-462f-afae-56c091e918b6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:57.793688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e5fb2963-20e5-4b93-beba-c640fe4d7cbc None None] MFA Rules not processed for user `50d13e9e2afc44fd91caf096922460ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:50:57.795241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e5fb2963-20e5-4b93-beba-c640fe4d7cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:57.810941 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1252/2434] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:50:57 2022] POST /identity/v3/auth/tokens => generated 3288 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:50:57.868782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d4b2be87-68f2-462f-afae-56c091e918b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a993171008c84bdaa3f81aacd4d4e94f', 'request_id': 'req-d4b2be87-68f2-462f-afae-56c091e918b6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:57.872299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:57.889138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d4b2be87-68f2-462f-afae-56c091e918b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a993171008c84bdaa3f81aacd4d4e94f', 'request_id': 'req-d4b2be87-68f2-462f-afae-56c091e918b6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:57.906156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d4b2be87-68f2-462f-afae-56c091e918b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a993171008c84bdaa3f81aacd4d4e94f', 'request_id': 'req-d4b2be87-68f2-462f-afae-56c091e918b6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:57.938009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d4b2be87-68f2-462f-afae-56c091e918b6 admin admin] The token cache is being invalidate because project a993171008c84bdaa3f81aacd4d4e94f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:50:57.938401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d4b2be87-68f2-462f-afae-56c091e918b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:57.942404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d4b2be87-68f2-462f-afae-56c091e918b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:57.946423 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1183/2435] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:50:57 2022] DELETE /identity/v3/projects/a993171008c84bdaa3f81aacd4d4e94f => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:57.950715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb2963-20e5-4b93-beba-c640fe4d7cbc None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:57.972004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb2963-20e5-4b93-beba-c640fe4d7cbc None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:58.166409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73702f3-4848-4535-893d-2f9c41001666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:58.167723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73702f3-4848-4535-893d-2f9c41001666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:58.167723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73702f3-4848-4535-893d-2f9c41001666 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:58.167723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73702f3-4848-4535-893d-2f9c41001666 admin admin] PATH_INFO: `/v3/users/2e8bc010e22248b3993b61cb97b362f1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:58.169168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73702f3-4848-4535-893d-2f9c41001666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e8bc010e22248b3993b61cb97b362f1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:58.171181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73702f3-4848-4535-893d-2f9c41001666 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:58.289805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f73702f3-4848-4535-893d-2f9c41001666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e8bc010e22248b3993b61cb97b362f1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:58.301894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f73702f3-4848-4535-893d-2f9c41001666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e8bc010e22248b3993b61cb97b362f1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:58.367195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f73702f3-4848-4535-893d-2f9c41001666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e8bc010e22248b3993b61cb97b362f1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:58.371379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f73702f3-4848-4535-893d-2f9c41001666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e8bc010e22248b3993b61cb97b362f1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:58.390681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f73702f3-4848-4535-893d-2f9c41001666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e8bc010e22248b3993b61cb97b362f1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:50:58.405560 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1253/2436] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:50:57 2022] DELETE /identity/v3/users/2e8bc010e22248b3993b61cb97b362f1 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:50:58.577748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2be87-68f2-462f-afae-56c091e918b6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:50:58.580480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d4b2be87-68f2-462f-afae-56c091e918b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:58.760445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2be87-68f2-462f-afae-56c091e918b6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:50:58.865018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc6033-7777-405d-a7ef-d82d7b0e242f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:50:58.865872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc6033-7777-405d-a7ef-d82d7b0e242f service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:50:58.866433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc6033-7777-405d-a7ef-d82d7b0e242f service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:50:58.866942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc6033-7777-405d-a7ef-d82d7b0e242f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:50:58.875417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b1fc6033-7777-405d-a7ef-d82d7b0e242f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:50:59.066197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc6033-7777-405d-a7ef-d82d7b0e242f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:50:59.068264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc6033-7777-405d-a7ef-d82d7b0e242f service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:50:59.091883 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1184/2437] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:50:58 2022] GET /identity/v3/auth/tokens => generated 3288 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:51:00.338576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73702f3-4848-4535-893d-2f9c41001666 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:00.341121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f73702f3-4848-4535-893d-2f9c41001666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:00.481494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73702f3-4848-4535-893d-2f9c41001666 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:00.487907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43495524-3570-4b48-acb0-6e19ae6da6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:00.488675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43495524-3570-4b48-acb0-6e19ae6da6bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:00.489751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43495524-3570-4b48-acb0-6e19ae6da6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:00.490191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43495524-3570-4b48-acb0-6e19ae6da6bc admin admin] PATH_INFO: `/v3/projects/54d73537de704dbcba9db6d0cd53c944` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:00.491959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43495524-3570-4b48-acb0-6e19ae6da6bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=54d73537de704dbcba9db6d0cd53c944)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:00.495286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43495524-3570-4b48-acb0-6e19ae6da6bc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:00.524785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:00.596003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-43495524-3570-4b48-acb0-6e19ae6da6bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54d73537de704dbcba9db6d0cd53c944', 'request_id': 'req-43495524-3570-4b48-acb0-6e19ae6da6bc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:00.606838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-43495524-3570-4b48-acb0-6e19ae6da6bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54d73537de704dbcba9db6d0cd53c944', 'request_id': 'req-43495524-3570-4b48-acb0-6e19ae6da6bc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:00.627858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-43495524-3570-4b48-acb0-6e19ae6da6bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54d73537de704dbcba9db6d0cd53c944', 'request_id': 'req-43495524-3570-4b48-acb0-6e19ae6da6bc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:00.668976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-43495524-3570-4b48-acb0-6e19ae6da6bc admin admin] The token cache is being invalidate because project 54d73537de704dbcba9db6d0cd53c944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:51:00.669296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-43495524-3570-4b48-acb0-6e19ae6da6bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:00.670202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-43495524-3570-4b48-acb0-6e19ae6da6bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:00.675362 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1254/2438] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:51:00 2022] DELETE /identity/v3/projects/54d73537de704dbcba9db6d0cd53c944 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:51:00.677090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5528fc5-69d3-42c1-8fa8-e28207e1f586 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:00.677586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5528fc5-69d3-42c1-8fa8-e28207e1f586 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:00.677773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5528fc5-69d3-42c1-8fa8-e28207e1f586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:00.721872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c5528fc5-69d3-42c1-8fa8-e28207e1f586 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:00.723177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c5528fc5-69d3-42c1-8fa8-e28207e1f586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:00.748611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:00.861905 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1185/2439] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:51:00.869566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bcf242-9f4b-468a-8f04-6bd3d8cc3d82 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:00.870207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bcf242-9f4b-468a-8f04-6bd3d8cc3d82 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:00.870620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bcf242-9f4b-468a-8f04-6bd3d8cc3d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:00.914982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-65bcf242-9f4b-468a-8f04-6bd3d8cc3d82 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:00.918254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-65bcf242-9f4b-468a-8f04-6bd3d8cc3d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:00.932273 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1255/2440] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:51:00.936668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5528fc5-69d3-42c1-8fa8-e28207e1f586 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:00.950850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5528fc5-69d3-42c1-8fa8-e28207e1f586 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:00.959797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e80a67-ce52-497d-8172-1bce81e69a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:00.960578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e80a67-ce52-497d-8172-1bce81e69a5c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:00.960892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e80a67-ce52-497d-8172-1bce81e69a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:00.961193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e80a67-ce52-497d-8172-1bce81e69a5c admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:00.961953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e80a67-ce52-497d-8172-1bce81e69a5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:51:00.962270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e80a67-ce52-497d-8172-1bce81e69a5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:00.964051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e80a67-ce52-497d-8172-1bce81e69a5c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:01.014266 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1186/2441] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:51:01.019035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bcf242-9f4b-468a-8f04-6bd3d8cc3d82 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:01.034167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bcf242-9f4b-468a-8f04-6bd3d8cc3d82 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:01.051702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b468a1-a6b3-4aa6-b219-8e818d65e7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:01.053882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b468a1-a6b3-4aa6-b219-8e818d65e7d5 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:01.055784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b468a1-a6b3-4aa6-b219-8e818d65e7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:01.056140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b468a1-a6b3-4aa6-b219-8e818d65e7d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:01.057584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b468a1-a6b3-4aa6-b219-8e818d65e7d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:01.081068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b468a1-a6b3-4aa6-b219-8e818d65e7d5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:01.164931 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1256/2442] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:51:01 2022] POST /identity/v3/projects => generated 381 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:51:01.169532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e80a67-ce52-497d-8172-1bce81e69a5c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:01.187170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e80a67-ce52-497d-8172-1bce81e69a5c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:01.297494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b104e-3528-4e44-bdcc-646727ab2282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:01.298331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b104e-3528-4e44-bdcc-646727ab2282 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:01.298666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b104e-3528-4e44-bdcc-646727ab2282 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:01.299158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b104e-3528-4e44-bdcc-646727ab2282 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:01.300164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429b104e-3528-4e44-bdcc-646727ab2282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:01.302307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429b104e-3528-4e44-bdcc-646727ab2282 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:01.336638 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1187/2443] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:51:01 2022] POST /identity/v3/users => generated 291 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:51:01.341127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b468a1-a6b3-4aa6-b219-8e818d65e7d5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:01.362847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b468a1-a6b3-4aa6-b219-8e818d65e7d5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:01.369443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b671f-864b-46e1-abc3-109f9b4671dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:01.370438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b671f-864b-46e1-abc3-109f9b4671dc admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:01.370610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b671f-864b-46e1-abc3-109f9b4671dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:01.370881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b671f-864b-46e1-abc3-109f9b4671dc admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:01.371538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b671f-864b-46e1-abc3-109f9b4671dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:01.373211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b671f-864b-46e1-abc3-109f9b4671dc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:01.387330 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1257/2444] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:51:01.391371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b104e-3528-4e44-bdcc-646727ab2282 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:01.405949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b104e-3528-4e44-bdcc-646727ab2282 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:01.413135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b2e6e-0ae0-4bfa-8228-75b6d544d429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:01.414079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b2e6e-0ae0-4bfa-8228-75b6d544d429 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:01.414631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b2e6e-0ae0-4bfa-8228-75b6d544d429 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:01.416142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b2e6e-0ae0-4bfa-8228-75b6d544d429 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:01.418331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b2e6e-0ae0-4bfa-8228-75b6d544d429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:01.420231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b2e6e-0ae0-4bfa-8228-75b6d544d429 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:01.436413 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1188/2445] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:51:01.440407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b671f-864b-46e1-abc3-109f9b4671dc admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:01.467332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b671f-864b-46e1-abc3-109f9b4671dc admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:01.477601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369aaf4b-a982-4239-bb5b-3943863f813d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:01.478629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369aaf4b-a982-4239-bb5b-3943863f813d admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:01.479418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369aaf4b-a982-4239-bb5b-3943863f813d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:01.480171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369aaf4b-a982-4239-bb5b-3943863f813d admin admin] PATH_INFO: `/v3/projects/06d6d25327a2419eb77e8f1d2bd7e453/users/3f80ddd768754a45ab8509fe42bea142/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:01.534860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369aaf4b-a982-4239-bb5b-3943863f813d admin admin] RBAC: Authorizing `identity:create_grant(project_id=06d6d25327a2419eb77e8f1d2bd7e453, user_id=3f80ddd768754a45ab8509fe42bea142, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:01.541912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369aaf4b-a982-4239-bb5b-3943863f813d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:01.579283 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1258/2446] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:51:01 2022] PUT /identity/v3/projects/06d6d25327a2419eb77e8f1d2bd7e453/users/3f80ddd768754a45ab8509fe42bea142/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:51:01.591113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b65ad7-7834-49fd-adad-9815d179f445 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:01.591967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b65ad7-7834-49fd-adad-9815d179f445 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:01.592288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b65ad7-7834-49fd-adad-9815d179f445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:01.684130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-52b65ad7-7834-49fd-adad-9815d179f445 None None] MFA Rules not processed for user `3f80ddd768754a45ab8509fe42bea142`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:01.691629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-52b65ad7-7834-49fd-adad-9815d179f445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:02.040227 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1189/2447] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:51:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:51:02.044372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369aaf4b-a982-4239-bb5b-3943863f813d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:02.168445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369aaf4b-a982-4239-bb5b-3943863f813d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:02.180922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e81b7c-f881-4d5e-bf1d-94c2b74862ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:02.183081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e81b7c-f881-4d5e-bf1d-94c2b74862ac admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:02.183721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e81b7c-f881-4d5e-bf1d-94c2b74862ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:02.185359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e81b7c-f881-4d5e-bf1d-94c2b74862ac admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:02.186083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e81b7c-f881-4d5e-bf1d-94c2b74862ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:02.191643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e81b7c-f881-4d5e-bf1d-94c2b74862ac admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:02.261999 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1259/2448] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:51:02 2022] POST /identity/v3/projects => generated 381 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:51:02.263330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b65ad7-7834-49fd-adad-9815d179f445 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:02.284381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b65ad7-7834-49fd-adad-9815d179f445 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:02.302273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:02.304105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd527e-70b1-49ec-832d-6e356b6750b7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:02.304299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd527e-70b1-49ec-832d-6e356b6750b7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:02.304584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd527e-70b1-49ec-832d-6e356b6750b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:02.388208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-41bd527e-70b1-49ec-832d-6e356b6750b7 None None] MFA Rules not processed for user `85ba386a1dad4f43b6eb3f36b588c09e`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:02.389439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-41bd527e-70b1-49ec-832d-6e356b6750b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:02.404147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4662f2-3c03-4db3-9fe9-435727154a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:02.404854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4662f2-3c03-4db3-9fe9-435727154a74 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:02.405209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4662f2-3c03-4db3-9fe9-435727154a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:02.405587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4662f2-3c03-4db3-9fe9-435727154a74 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:02.406319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4662f2-3c03-4db3-9fe9-435727154a74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:02.408628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4662f2-3c03-4db3-9fe9-435727154a74 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:02.441509 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1190/2449] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:51:02 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:51:02.442868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4662f2-3c03-4db3-9fe9-435727154a74 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:02.456582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4662f2-3c03-4db3-9fe9-435727154a74 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:02.458800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:02.466056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9116f-c259-4205-abd2-8d12c3a642e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:02.466978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9116f-c259-4205-abd2-8d12c3a642e4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:02.467326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9116f-c259-4205-abd2-8d12c3a642e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:02.467665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9116f-c259-4205-abd2-8d12c3a642e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:02.485928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9116f-c259-4205-abd2-8d12c3a642e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:02.490102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9116f-c259-4205-abd2-8d12c3a642e4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:02.505409 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1191/2450] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:51:02.511625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9116f-c259-4205-abd2-8d12c3a642e4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:02.535636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9116f-c259-4205-abd2-8d12c3a642e4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:02.551736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a7653-507a-438b-a03f-5aed9f7663a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:02.554422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a7653-507a-438b-a03f-5aed9f7663a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:02.558662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a7653-507a-438b-a03f-5aed9f7663a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:02.558662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a7653-507a-438b-a03f-5aed9f7663a1 admin admin] PATH_INFO: `/v3/projects/6cab2353a6114f3683b8d074d3e7280c/users/81bf7466ba31404c919c9a80075bdc05/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:02.603646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a7653-507a-438b-a03f-5aed9f7663a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cab2353a6114f3683b8d074d3e7280c, user_id=81bf7466ba31404c919c9a80075bdc05, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:02.610067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a7653-507a-438b-a03f-5aed9f7663a1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:02.644456 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1192/2451] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:51:02 2022] PUT /identity/v3/projects/6cab2353a6114f3683b8d074d3e7280c/users/81bf7466ba31404c919c9a80075bdc05/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:51:02.645987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a7653-507a-438b-a03f-5aed9f7663a1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:02.661433 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1260/2452] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:51:02 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:51:02.677971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd527e-70b1-49ec-832d-6e356b6750b7 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:02.680127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-41bd527e-70b1-49ec-832d-6e356b6750b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:02.837503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd527e-70b1-49ec-832d-6e356b6750b7 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:02.843656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a7653-507a-438b-a03f-5aed9f7663a1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:02.875075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef576fb-0cfe-4f31-a788-0d9c788d6120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:02.877977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef576fb-0cfe-4f31-a788-0d9c788d6120 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:02.879369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef576fb-0cfe-4f31-a788-0d9c788d6120 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:02.879762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef576fb-0cfe-4f31-a788-0d9c788d6120 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:02.883353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef576fb-0cfe-4f31-a788-0d9c788d6120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:02.888671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef576fb-0cfe-4f31-a788-0d9c788d6120 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:02.921104 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1193/2453] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:51:02.923484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef576fb-0cfe-4f31-a788-0d9c788d6120 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:02.968818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef576fb-0cfe-4f31-a788-0d9c788d6120 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:02.985721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125ee01-ed8f-4ac0-af91-f22c6b0eec17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:02.986866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125ee01-ed8f-4ac0-af91-f22c6b0eec17 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:02.989085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125ee01-ed8f-4ac0-af91-f22c6b0eec17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:02.989634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125ee01-ed8f-4ac0-af91-f22c6b0eec17 admin admin] PATH_INFO: `/v3/projects/6cab2353a6114f3683b8d074d3e7280c/users/81bf7466ba31404c919c9a80075bdc05/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:02.996236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f125ee01-ed8f-4ac0-af91-f22c6b0eec17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cab2353a6114f3683b8d074d3e7280c, user_id=81bf7466ba31404c919c9a80075bdc05, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:02.999621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f125ee01-ed8f-4ac0-af91-f22c6b0eec17 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:03.026749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112857a9-f199-434a-b1a8-f7de49da39b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:03.032932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112857a9-f199-434a-b1a8-f7de49da39b8 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:03.034584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112857a9-f199-434a-b1a8-f7de49da39b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:03.035118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112857a9-f199-434a-b1a8-f7de49da39b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:03.038371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-112857a9-f199-434a-b1a8-f7de49da39b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:03.060558 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1194/2454] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:51:02 2022] PUT /identity/v3/projects/6cab2353a6114f3683b8d074d3e7280c/users/81bf7466ba31404c919c9a80075bdc05/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:51:03.062642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48709daf-454c-401a-9adb-38f3d970968c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:03.063281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48709daf-454c-401a-9adb-38f3d970968c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:03.063281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48709daf-454c-401a-9adb-38f3d970968c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:03.160076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-48709daf-454c-401a-9adb-38f3d970968c None None] MFA Rules not processed for user `81bf7466ba31404c919c9a80075bdc05`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:03.163989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-48709daf-454c-401a-9adb-38f3d970968c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:03.183331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112857a9-f199-434a-b1a8-f7de49da39b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:03.184794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112857a9-f199-434a-b1a8-f7de49da39b8 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:03.203733 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1261/2455] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:51:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:51:03.298303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:03.372747 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1195/2456] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:51:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:51:04.126156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112857a9-f199-434a-b1a8-f7de49da39b8 service cinder] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:04.128285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-112857a9-f199-434a-b1a8-f7de49da39b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:04.136625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:04.431510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112857a9-f199-434a-b1a8-f7de49da39b8 service cinder] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:04.642208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25093550-8d74-4137-9400-61befb3baac3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:04.644913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25093550-8d74-4137-9400-61befb3baac3 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:04.645289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25093550-8d74-4137-9400-61befb3baac3 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:04.645695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25093550-8d74-4137-9400-61befb3baac3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:04.671889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25093550-8d74-4137-9400-61befb3baac3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:04.674763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25093550-8d74-4137-9400-61befb3baac3 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:04.710972 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1262/2457] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:51:04 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:51:05.462058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eea3f1-1f38-4e6f-8b4c-ec451e9fb1e6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:05.462630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eea3f1-1f38-4e6f-8b4c-ec451e9fb1e6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:05.463163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eea3f1-1f38-4e6f-8b4c-ec451e9fb1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:05.493399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-32eea3f1-1f38-4e6f-8b4c-ec451e9fb1e6 None None] MFA Rules not processed for user `aa72870453cd4b958725801b842c7a28`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:05.494913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-32eea3f1-1f38-4e6f-8b4c-ec451e9fb1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:05.509031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25093550-8d74-4137-9400-61befb3baac3 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:05.754860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25093550-8d74-4137-9400-61befb3baac3 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:05.783283 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1196/2458] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:51:05 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:51:05.783503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e10ca9-468c-441b-9c3f-474b7bc56b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:05.783503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e10ca9-468c-441b-9c3f-474b7bc56b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:05.783503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e10ca9-468c-441b-9c3f-474b7bc56b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:05.783503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e10ca9-468c-441b-9c3f-474b7bc56b4d admin admin] PATH_INFO: `/v3/users/3f80ddd768754a45ab8509fe42bea142` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:05.790296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e10ca9-468c-441b-9c3f-474b7bc56b4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f80ddd768754a45ab8509fe42bea142)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:05.817333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e10ca9-468c-441b-9c3f-474b7bc56b4d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:05.995933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:06.066682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a6e10ca9-468c-441b-9c3f-474b7bc56b4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f80ddd768754a45ab8509fe42bea142'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:06.092213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a6e10ca9-468c-441b-9c3f-474b7bc56b4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f80ddd768754a45ab8509fe42bea142'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:06.121148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a6e10ca9-468c-441b-9c3f-474b7bc56b4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f80ddd768754a45ab8509fe42bea142'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:06.122417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a6e10ca9-468c-441b-9c3f-474b7bc56b4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f80ddd768754a45ab8509fe42bea142'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:06.154724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a6e10ca9-468c-441b-9c3f-474b7bc56b4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f80ddd768754a45ab8509fe42bea142'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:06.177598 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1263/2459] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:51:05 2022] DELETE /identity/v3/users/3f80ddd768754a45ab8509fe42bea142 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:51:06.182780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d8b3f-c714-4062-9330-3dc67280bbb3 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:06.182976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d8b3f-c714-4062-9330-3dc67280bbb3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:06.183273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d8b3f-c714-4062-9330-3dc67280bbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:06.217768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5a4d8b3f-c714-4062-9330-3dc67280bbb3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:06.218969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5a4d8b3f-c714-4062-9330-3dc67280bbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:06.369476 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1197/2460] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:51:06.470898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e10ca9-468c-441b-9c3f-474b7bc56b4d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:06.472374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a6e10ca9-468c-441b-9c3f-474b7bc56b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:06.616933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e10ca9-468c-441b-9c3f-474b7bc56b4d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:06.627211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf150c-0e35-440b-baf4-8b2c42b42e90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:06.627840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf150c-0e35-440b-baf4-8b2c42b42e90 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:06.628121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf150c-0e35-440b-baf4-8b2c42b42e90 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:06.628399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf150c-0e35-440b-baf4-8b2c42b42e90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:06.652179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf150c-0e35-440b-baf4-8b2c42b42e90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:06.653642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf150c-0e35-440b-baf4-8b2c42b42e90 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:06.682856 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1264/2461] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:51:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:51:08.123689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d8b3f-c714-4062-9330-3dc67280bbb3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:08.125452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5a4d8b3f-c714-4062-9330-3dc67280bbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:08.167398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d8b3f-c714-4062-9330-3dc67280bbb3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:08.176268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed323d1-e40f-4062-96fc-0dcbc209a813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:08.178910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed323d1-e40f-4062-96fc-0dcbc209a813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:08.179676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed323d1-e40f-4062-96fc-0dcbc209a813 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:08.180167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed323d1-e40f-4062-96fc-0dcbc209a813 admin admin] PATH_INFO: `/v3/projects/06d6d25327a2419eb77e8f1d2bd7e453` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:08.181921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed323d1-e40f-4062-96fc-0dcbc209a813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06d6d25327a2419eb77e8f1d2bd7e453)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:08.184637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed323d1-e40f-4062-96fc-0dcbc209a813 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:08.349722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b86d9-3421-4193-9d99-a218ca070b08 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:08.351068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b86d9-3421-4193-9d99-a218ca070b08 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:08.351584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b86d9-3421-4193-9d99-a218ca070b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:08.355592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9ed323d1-e40f-4062-96fc-0dcbc209a813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06d6d25327a2419eb77e8f1d2bd7e453', 'request_id': 'req-9ed323d1-e40f-4062-96fc-0dcbc209a813'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:08.377171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9ed323d1-e40f-4062-96fc-0dcbc209a813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06d6d25327a2419eb77e8f1d2bd7e453', 'request_id': 'req-9ed323d1-e40f-4062-96fc-0dcbc209a813'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:08.392334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b07b86d9-3421-4193-9d99-a218ca070b08 None None] MFA Rules not processed for user `e5bfbc1e512a445ca7527f3d79e40d16`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:08.393992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b07b86d9-3421-4193-9d99-a218ca070b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:08.410206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9ed323d1-e40f-4062-96fc-0dcbc209a813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06d6d25327a2419eb77e8f1d2bd7e453', 'request_id': 'req-9ed323d1-e40f-4062-96fc-0dcbc209a813'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:08.452177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:08.476615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9ed323d1-e40f-4062-96fc-0dcbc209a813 admin admin] The token cache is being invalidate because project 06d6d25327a2419eb77e8f1d2bd7e453 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:51:08.487870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9ed323d1-e40f-4062-96fc-0dcbc209a813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:08.490678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9ed323d1-e40f-4062-96fc-0dcbc209a813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:08.496661 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1198/2462] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:51:08 2022] DELETE /identity/v3/projects/06d6d25327a2419eb77e8f1d2bd7e453 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:51:08.498023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed323d1-e40f-4062-96fc-0dcbc209a813 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:08.500023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9ed323d1-e40f-4062-96fc-0dcbc209a813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:08.531233 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1265/2463] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:51:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:51:08.642359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed323d1-e40f-4062-96fc-0dcbc209a813 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:08.653843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef3b61-47f9-4cae-b896-bcf519305127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:08.654845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef3b61-47f9-4cae-b896-bcf519305127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:08.655342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef3b61-47f9-4cae-b896-bcf519305127 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:08.655863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef3b61-47f9-4cae-b896-bcf519305127 admin admin] PATH_INFO: `/v3/users/81bf7466ba31404c919c9a80075bdc05` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:08.657897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef3b61-47f9-4cae-b896-bcf519305127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81bf7466ba31404c919c9a80075bdc05)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:08.660568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef3b61-47f9-4cae-b896-bcf519305127 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:08.742101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-33ef3b61-47f9-4cae-b896-bcf519305127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81bf7466ba31404c919c9a80075bdc05'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:08.743105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-33ef3b61-47f9-4cae-b896-bcf519305127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81bf7466ba31404c919c9a80075bdc05'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:08.747265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-33ef3b61-47f9-4cae-b896-bcf519305127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81bf7466ba31404c919c9a80075bdc05'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:08.759495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-33ef3b61-47f9-4cae-b896-bcf519305127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81bf7466ba31404c919c9a80075bdc05'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:08.776271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-33ef3b61-47f9-4cae-b896-bcf519305127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81bf7466ba31404c919c9a80075bdc05'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:08.793493 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1199/2464] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:51:08 2022] DELETE /identity/v3/users/81bf7466ba31404c919c9a80075bdc05 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:51:09.564323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b86d9-3421-4193-9d99-a218ca070b08 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:09.579853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b86d9-3421-4193-9d99-a218ca070b08 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:09.781834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:09.819168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdde02-ee43-4750-85a1-3b3921edb40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:09.820439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdde02-ee43-4750-85a1-3b3921edb40a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:09.820439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdde02-ee43-4750-85a1-3b3921edb40a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:09.820439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdde02-ee43-4750-85a1-3b3921edb40a admin admin] PATH_INFO: `/v3/projects/6cab2353a6114f3683b8d074d3e7280c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:09.829550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fdde02-ee43-4750-85a1-3b3921edb40a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cab2353a6114f3683b8d074d3e7280c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:09.831585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fdde02-ee43-4750-85a1-3b3921edb40a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:09.979119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-49fdde02-ee43-4750-85a1-3b3921edb40a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cab2353a6114f3683b8d074d3e7280c', 'request_id': 'req-49fdde02-ee43-4750-85a1-3b3921edb40a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:09.999372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-49fdde02-ee43-4750-85a1-3b3921edb40a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cab2353a6114f3683b8d074d3e7280c', 'request_id': 'req-49fdde02-ee43-4750-85a1-3b3921edb40a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:10.015693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-49fdde02-ee43-4750-85a1-3b3921edb40a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cab2353a6114f3683b8d074d3e7280c', 'request_id': 'req-49fdde02-ee43-4750-85a1-3b3921edb40a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:10.077827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-49fdde02-ee43-4750-85a1-3b3921edb40a admin admin] The token cache is being invalidate because project 6cab2353a6114f3683b8d074d3e7280c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:51:10.078957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-49fdde02-ee43-4750-85a1-3b3921edb40a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:10.081170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-49fdde02-ee43-4750-85a1-3b3921edb40a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:10.086661 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1266/2465] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:51:09 2022] DELETE /identity/v3/projects/6cab2353a6114f3683b8d074d3e7280c => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:51:10.091451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ee4c9-1e17-4698-a9e9-e07e357378ae None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:10.092484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ee4c9-1e17-4698-a9e9-e07e357378ae None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:10.092958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ee4c9-1e17-4698-a9e9-e07e357378ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:10.137759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-aa5ee4c9-1e17-4698-a9e9-e07e357378ae None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:10.140075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-aa5ee4c9-1e17-4698-a9e9-e07e357378ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:10.481270 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1200/2466] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:51:10.489455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fedd91-baba-455c-a6a9-ed5d8459c601 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:10.490028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fedd91-baba-455c-a6a9-ed5d8459c601 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:10.490720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fedd91-baba-455c-a6a9-ed5d8459c601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:10.542365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b0fedd91-baba-455c-a6a9-ed5d8459c601 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:10.544887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b0fedd91-baba-455c-a6a9-ed5d8459c601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:10.571374 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1267/2467] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:51:10.578087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ee4c9-1e17-4698-a9e9-e07e357378ae None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:10.597096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ee4c9-1e17-4698-a9e9-e07e357378ae None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:10.621887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bef15-841b-4b0b-8779-b57ac2a88397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:10.622851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bef15-841b-4b0b-8779-b57ac2a88397 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:10.623683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bef15-841b-4b0b-8779-b57ac2a88397 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:10.624463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bef15-841b-4b0b-8779-b57ac2a88397 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:10.625851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bef15-841b-4b0b-8779-b57ac2a88397 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:51:10.626726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bef15-841b-4b0b-8779-b57ac2a88397 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:10.633247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bef15-841b-4b0b-8779-b57ac2a88397 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:10.653985 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1201/2468] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:51:10.657904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fedd91-baba-455c-a6a9-ed5d8459c601 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:10.674861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fedd91-baba-455c-a6a9-ed5d8459c601 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:10.686597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb920c4f-5b10-45fb-8813-47e57adf6249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:10.688402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb920c4f-5b10-45fb-8813-47e57adf6249 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:10.688848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb920c4f-5b10-45fb-8813-47e57adf6249 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:10.689276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb920c4f-5b10-45fb-8813-47e57adf6249 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:10.690096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb920c4f-5b10-45fb-8813-47e57adf6249 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:10.694362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb920c4f-5b10-45fb-8813-47e57adf6249 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:10.758388 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1268/2469] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:51:10 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:51:10.761993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bef15-841b-4b0b-8779-b57ac2a88397 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:10.782949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bef15-841b-4b0b-8779-b57ac2a88397 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:10.789678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:10.925077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528517e-249b-42b6-ab44-cf0aa5e77f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:10.926251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528517e-249b-42b6-ab44-cf0aa5e77f44 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:10.926826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528517e-249b-42b6-ab44-cf0aa5e77f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:10.927196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528517e-249b-42b6-ab44-cf0aa5e77f44 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:10.928516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528517e-249b-42b6-ab44-cf0aa5e77f44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:10.930775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528517e-249b-42b6-ab44-cf0aa5e77f44 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:10.973515 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1202/2470] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:51:10 2022] POST /identity/v3/users => generated 286 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:51:10.977574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb920c4f-5b10-45fb-8813-47e57adf6249 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:10.992654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb920c4f-5b10-45fb-8813-47e57adf6249 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:11.001539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f44bf-e4f7-4b98-93d3-cfd2b988d9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:11.002167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f44bf-e4f7-4b98-93d3-cfd2b988d9df admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:11.002591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f44bf-e4f7-4b98-93d3-cfd2b988d9df admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:11.002887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f44bf-e4f7-4b98-93d3-cfd2b988d9df admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:11.003705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f44bf-e4f7-4b98-93d3-cfd2b988d9df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:11.005326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f44bf-e4f7-4b98-93d3-cfd2b988d9df admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:11.023768 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1269/2471] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:51:11.025637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528517e-249b-42b6-ab44-cf0aa5e77f44 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:11.039177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528517e-249b-42b6-ab44-cf0aa5e77f44 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:11.044945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177c829-e1cd-43bd-bbfd-d8096361ab71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:11.045591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177c829-e1cd-43bd-bbfd-d8096361ab71 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:11.045872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177c829-e1cd-43bd-bbfd-d8096361ab71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:11.046215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177c829-e1cd-43bd-bbfd-d8096361ab71 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:11.046916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177c829-e1cd-43bd-bbfd-d8096361ab71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:11.048806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177c829-e1cd-43bd-bbfd-d8096361ab71 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:11.070446 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1203/2472] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:51:11.075344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f44bf-e4f7-4b98-93d3-cfd2b988d9df admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:11.091764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f44bf-e4f7-4b98-93d3-cfd2b988d9df admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:11.095096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38673892-0c1e-493f-9056-e928b87e5b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:11.095898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38673892-0c1e-493f-9056-e928b87e5b49 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:11.096476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38673892-0c1e-493f-9056-e928b87e5b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:11.096817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38673892-0c1e-493f-9056-e928b87e5b49 admin admin] PATH_INFO: `/v3/projects/237c6bad691343e7a7e64304f55e01e9/users/a749a74841e840e2b8de67fb9562bf51/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:11.132769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38673892-0c1e-493f-9056-e928b87e5b49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=237c6bad691343e7a7e64304f55e01e9, user_id=a749a74841e840e2b8de67fb9562bf51, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:11.143416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38673892-0c1e-493f-9056-e928b87e5b49 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:11.168864 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1270/2473] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:51:11 2022] PUT /identity/v3/projects/237c6bad691343e7a7e64304f55e01e9/users/a749a74841e840e2b8de67fb9562bf51/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:51:11.176625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dacae5-41ac-44b4-bc6d-faa0c4b99303 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:11.177162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dacae5-41ac-44b4-bc6d-faa0c4b99303 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:11.177162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dacae5-41ac-44b4-bc6d-faa0c4b99303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:11.248552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-04dacae5-41ac-44b4-bc6d-faa0c4b99303 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:11.252080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-04dacae5-41ac-44b4-bc6d-faa0c4b99303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:11.498261 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1204/2474] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:51:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:51:11.609418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38673892-0c1e-493f-9056-e928b87e5b49 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:11.611647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-38673892-0c1e-493f-9056-e928b87e5b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:11.834608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38673892-0c1e-493f-9056-e928b87e5b49 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:11.853261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab15514-6a27-4ce2-97bc-672f980744f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:11.854044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab15514-6a27-4ce2-97bc-672f980744f5 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:11.854479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab15514-6a27-4ce2-97bc-672f980744f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:11.854887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab15514-6a27-4ce2-97bc-672f980744f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:11.879576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab15514-6a27-4ce2-97bc-672f980744f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:11.883316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab15514-6a27-4ce2-97bc-672f980744f5 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:11.907688 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1271/2475] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:51:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:51:16.675694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b6666-386a-4275-ad93-94abd593f070 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:16.676336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b6666-386a-4275-ad93-94abd593f070 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:16.676336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b6666-386a-4275-ad93-94abd593f070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:16.715416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-240b6666-386a-4275-ad93-94abd593f070 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:16.717242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-240b6666-386a-4275-ad93-94abd593f070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:16.905783 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1205/2476] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:51:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:51:16.933021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab15514-6a27-4ce2-97bc-672f980744f5 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:16.935517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3ab15514-6a27-4ce2-97bc-672f980744f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:16.980098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:17.141668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab15514-6a27-4ce2-97bc-672f980744f5 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:17.154936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2f93f-78b2-4b62-bdb6-ebe7ccc725c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:17.155942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2f93f-78b2-4b62-bdb6-ebe7ccc725c7 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:17.157408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2f93f-78b2-4b62-bdb6-ebe7ccc725c7 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:17.157690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2f93f-78b2-4b62-bdb6-ebe7ccc725c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:17.191053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c2f93f-78b2-4b62-bdb6-ebe7ccc725c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:17.194319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c2f93f-78b2-4b62-bdb6-ebe7ccc725c7 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:17.218709 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1272/2477] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:51:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 20:51:22.560174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf71fa-06dc-4821-8ca9-a59c6f398972 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:22.560800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf71fa-06dc-4821-8ca9-a59c6f398972 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:22.560800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf71fa-06dc-4821-8ca9-a59c6f398972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:22.650211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4ddf71fa-06dc-4821-8ca9-a59c6f398972 None None] MFA Rules not processed for user `a749a74841e840e2b8de67fb9562bf51`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:22.652260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4ddf71fa-06dc-4821-8ca9-a59c6f398972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:22.743163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:23.043770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2f93f-78b2-4b62-bdb6-ebe7ccc725c7 service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:23.044857 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1206/2478] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:51:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 485 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:51:23.068545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2f93f-78b2-4b62-bdb6-ebe7ccc725c7 service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:23.084837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e23335-e2ba-4567-9cc5-6198b538a144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:23.085476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e23335-e2ba-4567-9cc5-6198b538a144 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:23.085759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e23335-e2ba-4567-9cc5-6198b538a144 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:23.086037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e23335-e2ba-4567-9cc5-6198b538a144 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:23.086693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e23335-e2ba-4567-9cc5-6198b538a144 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:23.102261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e23335-e2ba-4567-9cc5-6198b538a144 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:23.219070 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1273/2479] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:51:23 2022] POST /identity/v3/projects => generated 369 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:51:23.223126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf71fa-06dc-4821-8ca9-a59c6f398972 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:23.255883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf71fa-06dc-4821-8ca9-a59c6f398972 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:23.466084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa08c0-1735-4c07-a8c5-b650f0e32c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:23.466891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa08c0-1735-4c07-a8c5-b650f0e32c79 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:23.467203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa08c0-1735-4c07-a8c5-b650f0e32c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:23.467482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa08c0-1735-4c07-a8c5-b650f0e32c79 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:23.468132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa08c0-1735-4c07-a8c5-b650f0e32c79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:23.470127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa08c0-1735-4c07-a8c5-b650f0e32c79 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:23.525145 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1207/2480] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:51:23 2022] POST /identity/v3/users => generated 291 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:51:23.529924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e23335-e2ba-4567-9cc5-6198b538a144 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:23.545704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e23335-e2ba-4567-9cc5-6198b538a144 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:23.569916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4801580-8bc6-4735-bcdb-042b0115844c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:23.571097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4801580-8bc6-4735-bcdb-042b0115844c admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:23.571386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4801580-8bc6-4735-bcdb-042b0115844c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:23.571667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4801580-8bc6-4735-bcdb-042b0115844c admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:23.572557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4801580-8bc6-4735-bcdb-042b0115844c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:23.574785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4801580-8bc6-4735-bcdb-042b0115844c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:23.603270 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1274/2481] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:51:23.607770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa08c0-1735-4c07-a8c5-b650f0e32c79 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:23.621477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa08c0-1735-4c07-a8c5-b650f0e32c79 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:23.664746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcf335-c2c8-4d57-a002-c0dccd0225c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:23.668073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcf335-c2c8-4d57-a002-c0dccd0225c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:23.669230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcf335-c2c8-4d57-a002-c0dccd0225c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:23.672054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcf335-c2c8-4d57-a002-c0dccd0225c4 admin admin] PATH_INFO: `/v3/projects/0c14d930ddf943b598625b5352a03d8d/users/511c9feaf07741cc906e2465214edbf4/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:23.724985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fcf335-c2c8-4d57-a002-c0dccd0225c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c14d930ddf943b598625b5352a03d8d, user_id=511c9feaf07741cc906e2465214edbf4, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:23.728722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fcf335-c2c8-4d57-a002-c0dccd0225c4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:23.757646 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1208/2482] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:51:23 2022] PUT /identity/v3/projects/0c14d930ddf943b598625b5352a03d8d/users/511c9feaf07741cc906e2465214edbf4/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:51:23.762056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4801580-8bc6-4735-bcdb-042b0115844c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:23.776176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4801580-8bc6-4735-bcdb-042b0115844c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:23.922202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a21196-bbd0-4306-90e7-dc64dcddb817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:23.922870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a21196-bbd0-4306-90e7-dc64dcddb817 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:23.923154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a21196-bbd0-4306-90e7-dc64dcddb817 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:23.923431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a21196-bbd0-4306-90e7-dc64dcddb817 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:23.924094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a21196-bbd0-4306-90e7-dc64dcddb817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:23.935516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a21196-bbd0-4306-90e7-dc64dcddb817 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:23.962283 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1275/2483] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:51:23.966647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcf335-c2c8-4d57-a002-c0dccd0225c4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:23.981726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcf335-c2c8-4d57-a002-c0dccd0225c4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:24.010184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddc281-35b9-4d6d-8d67-a76e82356561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:24.014308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddc281-35b9-4d6d-8d67-a76e82356561 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:24.014579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddc281-35b9-4d6d-8d67-a76e82356561 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:24.014865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddc281-35b9-4d6d-8d67-a76e82356561 admin admin] PATH_INFO: `/v3/projects/0c14d930ddf943b598625b5352a03d8d/users/511c9feaf07741cc906e2465214edbf4/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:24.034822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bddc281-35b9-4d6d-8d67-a76e82356561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c14d930ddf943b598625b5352a03d8d, user_id=511c9feaf07741cc906e2465214edbf4, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:24.047550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bddc281-35b9-4d6d-8d67-a76e82356561 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:24.094341 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1209/2484] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:51:23 2022] PUT /identity/v3/projects/0c14d930ddf943b598625b5352a03d8d/users/511c9feaf07741cc906e2465214edbf4/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:51:28.106967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a21196-bbd0-4306-90e7-dc64dcddb817 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:28.108994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-16a21196-bbd0-4306-90e7-dc64dcddb817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:28.266017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a21196-bbd0-4306-90e7-dc64dcddb817 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:28.340521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:28.491214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90278a-fb4f-4fc8-88a3-5a85a272724e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:28.491834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90278a-fb4f-4fc8-88a3-5a85a272724e service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:28.492112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90278a-fb4f-4fc8-88a3-5a85a272724e service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:28.492384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90278a-fb4f-4fc8-88a3-5a85a272724e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:28.497725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-eb90278a-fb4f-4fc8-88a3-5a85a272724e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:28.527578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb90278a-fb4f-4fc8-88a3-5a85a272724e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:28.529092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb90278a-fb4f-4fc8-88a3-5a85a272724e service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:28.554618 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1276/2485] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:51:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:51:34.075950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:34.383817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddc281-35b9-4d6d-8d67-a76e82356561 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:34.389674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8bddc281-35b9-4d6d-8d67-a76e82356561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:34.427336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddc281-35b9-4d6d-8d67-a76e82356561 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:34.444355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0764c95f-be2d-42a1-b0c4-d05e3501fd13 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:34.447008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764c95f-be2d-42a1-b0c4-d05e3501fd13 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:34.450616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764c95f-be2d-42a1-b0c4-d05e3501fd13 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:34.451177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764c95f-be2d-42a1-b0c4-d05e3501fd13 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:34.454043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0764c95f-be2d-42a1-b0c4-d05e3501fd13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:34.491061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0764c95f-be2d-42a1-b0c4-d05e3501fd13 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:34.495382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0764c95f-be2d-42a1-b0c4-d05e3501fd13 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:34.530059 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1210/2486] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:51:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:51:37.698933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90278a-fb4f-4fc8-88a3-5a85a272724e service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:37.700651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-eb90278a-fb4f-4fc8-88a3-5a85a272724e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:38.022059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90278a-fb4f-4fc8-88a3-5a85a272724e service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:38.068865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011c557-f121-48d2-b610-0d303f31d505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:38.069528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011c557-f121-48d2-b610-0d303f31d505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:38.069908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011c557-f121-48d2-b610-0d303f31d505 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:38.070219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011c557-f121-48d2-b610-0d303f31d505 admin admin] PATH_INFO: `/v3/users/a022d16da7234783b36d85cfe717628f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:38.082017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011c557-f121-48d2-b610-0d303f31d505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a022d16da7234783b36d85cfe717628f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:38.084113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011c557-f121-48d2-b610-0d303f31d505 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:38.305427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7011c557-f121-48d2-b610-0d303f31d505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a022d16da7234783b36d85cfe717628f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:38.335271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7011c557-f121-48d2-b610-0d303f31d505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a022d16da7234783b36d85cfe717628f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:38.406456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7011c557-f121-48d2-b610-0d303f31d505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a022d16da7234783b36d85cfe717628f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:38.407266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7011c557-f121-48d2-b610-0d303f31d505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a022d16da7234783b36d85cfe717628f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:38.421482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7011c557-f121-48d2-b610-0d303f31d505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a022d16da7234783b36d85cfe717628f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:38.432243 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1277/2487] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:51:37 2022] DELETE /identity/v3/users/a022d16da7234783b36d85cfe717628f => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:51:39.641825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0764c95f-be2d-42a1-b0c4-d05e3501fd13 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:39.666334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0764c95f-be2d-42a1-b0c4-d05e3501fd13 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:39.885166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab431f8-ee7f-4cec-af76-4eee55b023c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:39.887528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab431f8-ee7f-4cec-af76-4eee55b023c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:39.889172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab431f8-ee7f-4cec-af76-4eee55b023c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:39.889678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab431f8-ee7f-4cec-af76-4eee55b023c9 admin admin] PATH_INFO: `/v3/projects/4f67f67302a44fcaa504527da580532e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:39.893166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab431f8-ee7f-4cec-af76-4eee55b023c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f67f67302a44fcaa504527da580532e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:39.902349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab431f8-ee7f-4cec-af76-4eee55b023c9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:40.042225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cab431f8-ee7f-4cec-af76-4eee55b023c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f67f67302a44fcaa504527da580532e', 'request_id': 'req-cab431f8-ee7f-4cec-af76-4eee55b023c9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:40.067678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cab431f8-ee7f-4cec-af76-4eee55b023c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f67f67302a44fcaa504527da580532e', 'request_id': 'req-cab431f8-ee7f-4cec-af76-4eee55b023c9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:40.081073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cab431f8-ee7f-4cec-af76-4eee55b023c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f67f67302a44fcaa504527da580532e', 'request_id': 'req-cab431f8-ee7f-4cec-af76-4eee55b023c9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:40.112275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cab431f8-ee7f-4cec-af76-4eee55b023c9 admin admin] The token cache is being invalidate because project 4f67f67302a44fcaa504527da580532e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:51:40.114369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cab431f8-ee7f-4cec-af76-4eee55b023c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:40.116385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cab431f8-ee7f-4cec-af76-4eee55b023c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:40.120210 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1211/2488] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:51:39 2022] DELETE /identity/v3/projects/4f67f67302a44fcaa504527da580532e => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:51:40.124269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011c557-f121-48d2-b610-0d303f31d505 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:40.125847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7011c557-f121-48d2-b610-0d303f31d505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:40.149972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011c557-f121-48d2-b610-0d303f31d505 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:40.221548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:40.319859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a107e99-acd7-4b0e-bf31-3b37b5c7b76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:40.320488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a107e99-acd7-4b0e-bf31-3b37b5c7b76e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:40.320770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a107e99-acd7-4b0e-bf31-3b37b5c7b76e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:40.321042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a107e99-acd7-4b0e-bf31-3b37b5c7b76e admin admin] PATH_INFO: `/v3/users/2be566d906f947f7abedf3dedb733529` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:40.325789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a107e99-acd7-4b0e-bf31-3b37b5c7b76e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2be566d906f947f7abedf3dedb733529)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:40.327791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a107e99-acd7-4b0e-bf31-3b37b5c7b76e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:40.599894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a107e99-acd7-4b0e-bf31-3b37b5c7b76e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2be566d906f947f7abedf3dedb733529'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:40.624008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a107e99-acd7-4b0e-bf31-3b37b5c7b76e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2be566d906f947f7abedf3dedb733529'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:40.656084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23ae3a-8bf5-4035-b9d6-83f29260d4ae None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:40.656601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23ae3a-8bf5-4035-b9d6-83f29260d4ae None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:40.656711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23ae3a-8bf5-4035-b9d6-83f29260d4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:40.659082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a107e99-acd7-4b0e-bf31-3b37b5c7b76e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2be566d906f947f7abedf3dedb733529'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:40.659838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a107e99-acd7-4b0e-bf31-3b37b5c7b76e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2be566d906f947f7abedf3dedb733529'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:40.677413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a107e99-acd7-4b0e-bf31-3b37b5c7b76e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2be566d906f947f7abedf3dedb733529'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:40.685774 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1278/2489] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:51:40 2022] DELETE /identity/v3/users/2be566d906f947f7abedf3dedb733529 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:51:40.757936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1d23ae3a-8bf5-4035-b9d6-83f29260d4ae None None] MFA Rules not processed for user `511c9feaf07741cc906e2465214edbf4`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:40.760052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1d23ae3a-8bf5-4035-b9d6-83f29260d4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:41.239761 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1212/2490] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:51:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 584 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:51:41.247194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726815d6-191d-459f-b81f-ac3fae1c47c3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:41.247750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726815d6-191d-459f-b81f-ac3fae1c47c3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:41.247750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726815d6-191d-459f-b81f-ac3fae1c47c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:41.313594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-726815d6-191d-459f-b81f-ac3fae1c47c3 None None] MFA Rules not processed for user `a749a74841e840e2b8de67fb9562bf51`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:41.314907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-726815d6-191d-459f-b81f-ac3fae1c47c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:41.594709 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1279/2491] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:51:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:51:42.234669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23ae3a-8bf5-4035-b9d6-83f29260d4ae None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:42.237133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1d23ae3a-8bf5-4035-b9d6-83f29260d4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:42.242350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:42.376348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23ae3a-8bf5-4035-b9d6-83f29260d4ae None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:42.388421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21891af-bb59-4213-a7a7-8b4ab417ec93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:42.389157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21891af-bb59-4213-a7a7-8b4ab417ec93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:42.389333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21891af-bb59-4213-a7a7-8b4ab417ec93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:42.389603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21891af-bb59-4213-a7a7-8b4ab417ec93 admin admin] PATH_INFO: `/v3/projects/ed69b3e55ed64d0bb5a0cd02f2d42d99` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:42.391487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21891af-bb59-4213-a7a7-8b4ab417ec93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed69b3e55ed64d0bb5a0cd02f2d42d99)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:42.392683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21891af-bb59-4213-a7a7-8b4ab417ec93 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:42.453760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726815d6-191d-459f-b81f-ac3fae1c47c3 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:42.455201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-726815d6-191d-459f-b81f-ac3fae1c47c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:42.539489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f21891af-bb59-4213-a7a7-8b4ab417ec93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed69b3e55ed64d0bb5a0cd02f2d42d99', 'request_id': 'req-f21891af-bb59-4213-a7a7-8b4ab417ec93'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:42.554066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f21891af-bb59-4213-a7a7-8b4ab417ec93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed69b3e55ed64d0bb5a0cd02f2d42d99', 'request_id': 'req-f21891af-bb59-4213-a7a7-8b4ab417ec93'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:42.563225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f21891af-bb59-4213-a7a7-8b4ab417ec93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed69b3e55ed64d0bb5a0cd02f2d42d99', 'request_id': 'req-f21891af-bb59-4213-a7a7-8b4ab417ec93'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:42.575012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:42.600419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f21891af-bb59-4213-a7a7-8b4ab417ec93 admin admin] The token cache is being invalidate because project ed69b3e55ed64d0bb5a0cd02f2d42d99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:51:42.600738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f21891af-bb59-4213-a7a7-8b4ab417ec93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:42.602356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f21891af-bb59-4213-a7a7-8b4ab417ec93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:51:42.609871 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1213/2492] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:51:42 2022] DELETE /identity/v3/projects/ed69b3e55ed64d0bb5a0cd02f2d42d99 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:51:42.613072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49ec69-0b5c-4fc7-b049-aa56c4d07764 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:42.613389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49ec69-0b5c-4fc7-b049-aa56c4d07764 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:42.614552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49ec69-0b5c-4fc7-b049-aa56c4d07764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:42.636756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726815d6-191d-459f-b81f-ac3fae1c47c3 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:42.667000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-dc49ec69-0b5c-4fc7-b049-aa56c4d07764 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:42.670368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-dc49ec69-0b5c-4fc7-b049-aa56c4d07764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:42.802561 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1214/2493] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:51:42.810233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394e291-7397-421b-9cdb-a83ac684009b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:42.810787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394e291-7397-421b-9cdb-a83ac684009b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:42.811267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394e291-7397-421b-9cdb-a83ac684009b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:42.856386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9394e291-7397-421b-9cdb-a83ac684009b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:42.857892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9394e291-7397-421b-9cdb-a83ac684009b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:42.866946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1057c-df5f-4883-973f-c480eedd7466 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:42.868704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1057c-df5f-4883-973f-c480eedd7466 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:42.868992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1057c-df5f-4883-973f-c480eedd7466 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:42.869270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1057c-df5f-4883-973f-c480eedd7466 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:42.875757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f3c1057c-df5f-4883-973f-c480eedd7466 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:42.882345 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1215/2494] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:51:42.885185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394e291-7397-421b-9cdb-a83ac684009b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:42.921861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394e291-7397-421b-9cdb-a83ac684009b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:42.928982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d42c16-f2de-43a4-bc1f-24bdb36756fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:42.929763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d42c16-f2de-43a4-bc1f-24bdb36756fb admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:42.930033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d42c16-f2de-43a4-bc1f-24bdb36756fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:42.930370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d42c16-f2de-43a4-bc1f-24bdb36756fb admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:42.932656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d42c16-f2de-43a4-bc1f-24bdb36756fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:51:42.933062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d42c16-f2de-43a4-bc1f-24bdb36756fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:42.936727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d42c16-f2de-43a4-bc1f-24bdb36756fb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:42.969148 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1216/2495] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:51:42.975586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d42c16-f2de-43a4-bc1f-24bdb36756fb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:42.994004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d42c16-f2de-43a4-bc1f-24bdb36756fb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:43.012095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925dd90-357e-4c0e-895c-082e468ec969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:43.012712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925dd90-357e-4c0e-895c-082e468ec969 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:43.013006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925dd90-357e-4c0e-895c-082e468ec969 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:43.013251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925dd90-357e-4c0e-895c-082e468ec969 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:43.013882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1925dd90-357e-4c0e-895c-082e468ec969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:43.015972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1925dd90-357e-4c0e-895c-082e468ec969 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:43.024731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c1057c-df5f-4883-973f-c480eedd7466 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:43.026604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c1057c-df5f-4883-973f-c480eedd7466 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:43.039262 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1280/2496] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:51:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:51:43.052724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:43.090355 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1217/2497] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:51:42 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:51:43.091562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1057c-df5f-4883-973f-c480eedd7466 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:43.107644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1057c-df5f-4883-973f-c480eedd7466 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:43.267012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1952ddd-4759-4abb-8d0b-04c0217143b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:43.267635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1952ddd-4759-4abb-8d0b-04c0217143b6 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:43.267912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1952ddd-4759-4abb-8d0b-04c0217143b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:43.268184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1952ddd-4759-4abb-8d0b-04c0217143b6 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:43.269473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1952ddd-4759-4abb-8d0b-04c0217143b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:43.271743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1952ddd-4759-4abb-8d0b-04c0217143b6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:43.314147 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1281/2498] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:51:43 2022] POST /identity/v3/users => generated 294 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:51:43.317641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925dd90-357e-4c0e-895c-082e468ec969 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:43.334597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925dd90-357e-4c0e-895c-082e468ec969 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:43.351500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d60b4-607f-40d1-b353-f5ceeaaafb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:43.352462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d60b4-607f-40d1-b353-f5ceeaaafb0a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:43.353415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d60b4-607f-40d1-b353-f5ceeaaafb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:43.355617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d60b4-607f-40d1-b353-f5ceeaaafb0a admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:43.358000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d60b4-607f-40d1-b353-f5ceeaaafb0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:43.367740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d60b4-607f-40d1-b353-f5ceeaaafb0a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:43.412893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1952ddd-4759-4abb-8d0b-04c0217143b6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:43.414397 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1218/2499] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:51:43.435262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1952ddd-4759-4abb-8d0b-04c0217143b6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:43.459485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37557c9-588e-4344-8a44-e6ee476f2976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:43.463543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37557c9-588e-4344-8a44-e6ee476f2976 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:43.465460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37557c9-588e-4344-8a44-e6ee476f2976 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:43.466706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37557c9-588e-4344-8a44-e6ee476f2976 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:43.470963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37557c9-588e-4344-8a44-e6ee476f2976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:43.476708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37557c9-588e-4344-8a44-e6ee476f2976 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:43.529719 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1282/2500] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:51:43.534118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d60b4-607f-40d1-b353-f5ceeaaafb0a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:43.550831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d60b4-607f-40d1-b353-f5ceeaaafb0a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:43.566968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e3363-3cee-467e-ba59-f5caede30c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:43.567652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e3363-3cee-467e-ba59-f5caede30c1c admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:43.567933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e3363-3cee-467e-ba59-f5caede30c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:43.568212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e3363-3cee-467e-ba59-f5caede30c1c admin admin] PATH_INFO: `/v3/projects/963f525498ef4ebdb9db327df62bd68a/users/04add76a65d74165b65189a5fd1f47ed/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:43.606708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e3363-3cee-467e-ba59-f5caede30c1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=963f525498ef4ebdb9db327df62bd68a, user_id=04add76a65d74165b65189a5fd1f47ed, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:43.609548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e3363-3cee-467e-ba59-f5caede30c1c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:43.636768 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1219/2501] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:51:43 2022] PUT /identity/v3/projects/963f525498ef4ebdb9db327df62bd68a/users/04add76a65d74165b65189a5fd1f47ed/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:51:43.648618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274286b7-ca0f-413c-848d-0cc2da8cc69a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:43.648727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274286b7-ca0f-413c-848d-0cc2da8cc69a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:43.649564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274286b7-ca0f-413c-848d-0cc2da8cc69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:43.658748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:43.762246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-274286b7-ca0f-413c-848d-0cc2da8cc69a None None] MFA Rules not processed for user `04add76a65d74165b65189a5fd1f47ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:43.762879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-274286b7-ca0f-413c-848d-0cc2da8cc69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:43.901924 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1283/2502] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:51:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:51:43.902916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e3363-3cee-467e-ba59-f5caede30c1c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:44.177566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e3363-3cee-467e-ba59-f5caede30c1c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:44.194636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41531554-ed29-4a98-b9a6-50000d414c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:44.195518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41531554-ed29-4a98-b9a6-50000d414c39 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:44.195807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41531554-ed29-4a98-b9a6-50000d414c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:44.196078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41531554-ed29-4a98-b9a6-50000d414c39 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:44.196776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41531554-ed29-4a98-b9a6-50000d414c39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:44.199340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41531554-ed29-4a98-b9a6-50000d414c39 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:44.284247 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1220/2503] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:51:43 2022] POST /identity/v3/projects => generated 387 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:51:44.289917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274286b7-ca0f-413c-848d-0cc2da8cc69a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:44.318442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274286b7-ca0f-413c-848d-0cc2da8cc69a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:44.477102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20081c-280e-4910-8893-e3b0770ee28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:44.477890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20081c-280e-4910-8893-e3b0770ee28d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:44.477977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20081c-280e-4910-8893-e3b0770ee28d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:44.478247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20081c-280e-4910-8893-e3b0770ee28d admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:44.478911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e20081c-280e-4910-8893-e3b0770ee28d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:44.480846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e20081c-280e-4910-8893-e3b0770ee28d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:44.525796 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1284/2504] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:51:44 2022] POST /identity/v3/users => generated 300 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:51:44.530115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41531554-ed29-4a98-b9a6-50000d414c39 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:44.553174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41531554-ed29-4a98-b9a6-50000d414c39 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:44.572364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794893d-a97a-424e-a4d8-dbb5ff55816b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:44.573184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794893d-a97a-424e-a4d8-dbb5ff55816b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:44.573620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794893d-a97a-424e-a4d8-dbb5ff55816b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:44.574055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794893d-a97a-424e-a4d8-dbb5ff55816b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:44.574908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794893d-a97a-424e-a4d8-dbb5ff55816b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:44.579019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794893d-a97a-424e-a4d8-dbb5ff55816b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:44.640582 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1221/2505] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:51:44.641683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20081c-280e-4910-8893-e3b0770ee28d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:44.661185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20081c-280e-4910-8893-e3b0770ee28d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:44.669472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410eb20-9332-482a-90b4-2ae77c7b4f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:44.670132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410eb20-9332-482a-90b4-2ae77c7b4f7f admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:44.670415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410eb20-9332-482a-90b4-2ae77c7b4f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:44.670768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410eb20-9332-482a-90b4-2ae77c7b4f7f admin admin] PATH_INFO: `/v3/projects/1ab469e2d1a04a368860f8a96492940f/users/dfd53627e013481fb9fb43b94d43db47/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:44.723001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410eb20-9332-482a-90b4-2ae77c7b4f7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ab469e2d1a04a368860f8a96492940f, user_id=dfd53627e013481fb9fb43b94d43db47, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:44.725852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410eb20-9332-482a-90b4-2ae77c7b4f7f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:44.789148 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1285/2506] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:51:44 2022] PUT /identity/v3/projects/1ab469e2d1a04a368860f8a96492940f/users/dfd53627e013481fb9fb43b94d43db47/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:51:44.790435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794893d-a97a-424e-a4d8-dbb5ff55816b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:44.809745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794893d-a97a-424e-a4d8-dbb5ff55816b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:44.938608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e1ca5-60f1-4fc8-af67-6e8e021d530c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:44.966638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e1ca5-60f1-4fc8-af67-6e8e021d530c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:44.966638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e1ca5-60f1-4fc8-af67-6e8e021d530c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:44.966638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e1ca5-60f1-4fc8-af67-6e8e021d530c admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:44.966638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e1ca5-60f1-4fc8-af67-6e8e021d530c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:44.978698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e1ca5-60f1-4fc8-af67-6e8e021d530c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:45.043704 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1222/2507] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:51:45.045894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410eb20-9332-482a-90b4-2ae77c7b4f7f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:45.099053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410eb20-9332-482a-90b4-2ae77c7b4f7f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:45.110580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c93186-51f8-49b7-b10e-b8368a8eabea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:45.111487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c93186-51f8-49b7-b10e-b8368a8eabea admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:45.111899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c93186-51f8-49b7-b10e-b8368a8eabea admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:45.112448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c93186-51f8-49b7-b10e-b8368a8eabea admin admin] PATH_INFO: `/v3/projects/1ab469e2d1a04a368860f8a96492940f/users/dfd53627e013481fb9fb43b94d43db47/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:45.119736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c93186-51f8-49b7-b10e-b8368a8eabea admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ab469e2d1a04a368860f8a96492940f, user_id=dfd53627e013481fb9fb43b94d43db47, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:45.122702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c93186-51f8-49b7-b10e-b8368a8eabea admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:45.162761 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1286/2508] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:51:45 2022] PUT /identity/v3/projects/1ab469e2d1a04a368860f8a96492940f/users/dfd53627e013481fb9fb43b94d43db47/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:51:45.164778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bb41d-b0de-4e05-b520-e94c0ec6ea8a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:45.166860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bb41d-b0de-4e05-b520-e94c0ec6ea8a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:45.169411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bb41d-b0de-4e05-b520-e94c0ec6ea8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:45.413814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d54bb41d-b0de-4e05-b520-e94c0ec6ea8a None None] MFA Rules not processed for user `dfd53627e013481fb9fb43b94d43db47`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:45.417631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d54bb41d-b0de-4e05-b520-e94c0ec6ea8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:45.498574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:45.816174 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1223/2509] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:51:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 653 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:51:45.825426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae15d7d-0003-404e-938c-4e90274e97b3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:45.826017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae15d7d-0003-404e-938c-4e90274e97b3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:45.826851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae15d7d-0003-404e-938c-4e90274e97b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:46.032013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3ae15d7d-0003-404e-938c-4e90274e97b3 None None] MFA Rules not processed for user `04add76a65d74165b65189a5fd1f47ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:46.052168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3ae15d7d-0003-404e-938c-4e90274e97b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:46.294558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:46.351357 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1287/2510] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:51:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 527 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:51:47.247667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bb41d-b0de-4e05-b520-e94c0ec6ea8a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:47.249100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d54bb41d-b0de-4e05-b520-e94c0ec6ea8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:47.382993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bb41d-b0de-4e05-b520-e94c0ec6ea8a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:47.389333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a58ef-dff8-4a37-adcb-577624de49d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:47.390010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a58ef-dff8-4a37-adcb-577624de49d3 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:47.390296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a58ef-dff8-4a37-adcb-577624de49d3 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:47.390641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a58ef-dff8-4a37-adcb-577624de49d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:47.402674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a58ef-dff8-4a37-adcb-577624de49d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:47.404626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a58ef-dff8-4a37-adcb-577624de49d3 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:47.424740 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1224/2511] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:51:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:51:50.070376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00867fa4-e29a-4cd1-aa00-4a625f3c1267 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:50.072162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00867fa4-e29a-4cd1-aa00-4a625f3c1267 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:50.072595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00867fa4-e29a-4cd1-aa00-4a625f3c1267 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:50.074523 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1288/2512] 10.222.0.17 () {54 vars in 921 bytes} [Sat Oct 1 20:51:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:51:50.089369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcc9f5-61bd-4d37-813b-d71c891dd26e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:50.089369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcc9f5-61bd-4d37-813b-d71c891dd26e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:50.089369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcc9f5-61bd-4d37-813b-d71c891dd26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:50.132096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c1bcc9f5-61bd-4d37-813b-d71c891dd26e None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:50.133545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c1bcc9f5-61bd-4d37-813b-d71c891dd26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:50.150887 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1225/2513] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 20:51:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:51:50.165614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00867fa4-e29a-4cd1-aa00-4a625f3c1267 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:50.167606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-00867fa4-e29a-4cd1-aa00-4a625f3c1267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:50.231625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00867fa4-e29a-4cd1-aa00-4a625f3c1267 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:50.235978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c97a0f-7cd0-4e43-993a-2cb5c52f89e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:50.236640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c97a0f-7cd0-4e43-993a-2cb5c52f89e8 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:50.236921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c97a0f-7cd0-4e43-993a-2cb5c52f89e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:50.237242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c97a0f-7cd0-4e43-993a-2cb5c52f89e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:50.249036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c97a0f-7cd0-4e43-993a-2cb5c52f89e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:50.250463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c97a0f-7cd0-4e43-993a-2cb5c52f89e8 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:50.275395 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1289/2514] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:51:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:51:55.376564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f43e0-7bdd-463b-9ba7-1e3b8a13243d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:55.377164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f43e0-7bdd-463b-9ba7-1e3b8a13243d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:55.377264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f43e0-7bdd-463b-9ba7-1e3b8a13243d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:55.411788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-691f43e0-7bdd-463b-9ba7-1e3b8a13243d None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:51:55.413083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-691f43e0-7bdd-463b-9ba7-1e3b8a13243d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:55.496664 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1226/2515] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:51:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:51:55.518864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c97a0f-7cd0-4e43-993a-2cb5c52f89e8 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:51:55.521023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-65c97a0f-7cd0-4e43-993a-2cb5c52f89e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:51:55.712925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c97a0f-7cd0-4e43-993a-2cb5c52f89e8 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:51:55.733714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9607eca-a885-4967-b3bc-7fd8d097b25d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:51:55.734577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9607eca-a885-4967-b3bc-7fd8d097b25d service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:51:55.735073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9607eca-a885-4967-b3bc-7fd8d097b25d service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:51:55.735440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9607eca-a885-4967-b3bc-7fd8d097b25d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:51:55.762317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9607eca-a885-4967-b3bc-7fd8d097b25d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:51:55.764216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9607eca-a885-4967-b3bc-7fd8d097b25d service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:51:55.777267 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1290/2516] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:51:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 20:52:05.485247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=84 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:05.772414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=62 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:09.047355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f43e0-7bdd-463b-9ba7-1e3b8a13243d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:09.048812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-691f43e0-7bdd-463b-9ba7-1e3b8a13243d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:09.073852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f43e0-7bdd-463b-9ba7-1e3b8a13243d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:09.084378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d48fd-7058-478b-a91b-6d7b3d94d613 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:09.086182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d48fd-7058-478b-a91b-6d7b3d94d613 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:09.086710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d48fd-7058-478b-a91b-6d7b3d94d613 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:09.087887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d48fd-7058-478b-a91b-6d7b3d94d613 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:09.092198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ca3d48fd-7058-478b-a91b-6d7b3d94d613 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:09.245203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d48fd-7058-478b-a91b-6d7b3d94d613 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:09.245368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885d8ac-6a7a-4d7d-a3a4-85ab50d42eb6 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:09.245368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885d8ac-6a7a-4d7d-a3a4-85ab50d42eb6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:09.245368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885d8ac-6a7a-4d7d-a3a4-85ab50d42eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:09.246678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d48fd-7058-478b-a91b-6d7b3d94d613 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:09.261949 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1227/2517] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:09.282791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f885d8ac-6a7a-4d7d-a3a4-85ab50d42eb6 None None] MFA Rules not processed for user `2aa4f468a15546eb9f72cb2b632e67bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:09.284153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f885d8ac-6a7a-4d7d-a3a4-85ab50d42eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:09.411784 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1291/2518] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:09.427655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4f586-4847-4828-b7f2-dbfc378a078a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:09.427940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4f586-4847-4828-b7f2-dbfc378a078a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:09.428231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4f586-4847-4828-b7f2-dbfc378a078a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:09.448986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d8f4f586-4847-4828-b7f2-dbfc378a078a None None] MFA Rules not processed for user `2aa4f468a15546eb9f72cb2b632e67bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:09.450211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d8f4f586-4847-4828-b7f2-dbfc378a078a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:09.460639 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1228/2519] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:10.025205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c87366-76f9-4c44-a36f-c29ad93d223e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:10.025498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c87366-76f9-4c44-a36f-c29ad93d223e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:10.025778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c87366-76f9-4c44-a36f-c29ad93d223e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:10.083797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-16c87366-76f9-4c44-a36f-c29ad93d223e None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:10.085870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-16c87366-76f9-4c44-a36f-c29ad93d223e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:10.106396 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1292/2520] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 20:52:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:10.116077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4f586-4847-4828-b7f2-dbfc378a078a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:10.131592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4f586-4847-4828-b7f2-dbfc378a078a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:10.137906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf8726-23a4-4f10-a38b-58c4f91aab2a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:10.138671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf8726-23a4-4f10-a38b-58c4f91aab2a service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:10.139057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf8726-23a4-4f10-a38b-58c4f91aab2a service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:10.139765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf8726-23a4-4f10-a38b-58c4f91aab2a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:10.151476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf8726-23a4-4f10-a38b-58c4f91aab2a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:10.153877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf8726-23a4-4f10-a38b-58c4f91aab2a service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:10.165130 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1229/2521] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:52:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 20:52:10.396014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61035b09-e570-4983-8db2-c21e1fd33700 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:10.396497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61035b09-e570-4983-8db2-c21e1fd33700 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:10.396595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61035b09-e570-4983-8db2-c21e1fd33700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:10.421662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-61035b09-e570-4983-8db2-c21e1fd33700 None None] MFA Rules not processed for user `aa72870453cd4b958725801b842c7a28`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:10.423387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-61035b09-e570-4983-8db2-c21e1fd33700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:10.528165 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1293/2522] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:10 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:11.420822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5478c4d-0e5f-46b1-b323-cc669021f8a9 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:11.421391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5478c4d-0e5f-46b1-b323-cc669021f8a9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:11.421391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5478c4d-0e5f-46b1-b323-cc669021f8a9 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:11.422723 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1230/2523] 10.222.0.17 () {54 vars in 921 bytes} [Sat Oct 1 20:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:52:11.446223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceffb00-894f-47ba-aa94-26bd7e85eabe None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:11.446809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceffb00-894f-47ba-aa94-26bd7e85eabe None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:11.447090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceffb00-894f-47ba-aa94-26bd7e85eabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:11.467463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3ceffb00-894f-47ba-aa94-26bd7e85eabe None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:11.468653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3ceffb00-894f-47ba-aa94-26bd7e85eabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:11.538699 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1294/2524] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 20:52:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:11.559003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5478c4d-0e5f-46b1-b323-cc669021f8a9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:11.560683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a5478c4d-0e5f-46b1-b323-cc669021f8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:11.625897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5478c4d-0e5f-46b1-b323-cc669021f8a9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:11.633122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceffb00-894f-47ba-aa94-26bd7e85eabe None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:11.637583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3ceffb00-894f-47ba-aa94-26bd7e85eabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:11.639300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e47e9e-0c1d-4e16-bf89-1bafc4fd833e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:11.640109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e47e9e-0c1d-4e16-bf89-1bafc4fd833e service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:11.640562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e47e9e-0c1d-4e16-bf89-1bafc4fd833e service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:11.640980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e47e9e-0c1d-4e16-bf89-1bafc4fd833e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:11.652838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e47e9e-0c1d-4e16-bf89-1bafc4fd833e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:11.656927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e47e9e-0c1d-4e16-bf89-1bafc4fd833e service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:11.682952 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1231/2525] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:11.703240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:11.731071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceffb00-894f-47ba-aa94-26bd7e85eabe None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:11.735592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e75a88-829b-415c-b94e-453c484c5f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:11.736223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e75a88-829b-415c-b94e-453c484c5f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:11.736508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e75a88-829b-415c-b94e-453c484c5f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:11.736785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e75a88-829b-415c-b94e-453c484c5f90 admin admin] PATH_INFO: `/v3/users/f40f50783d9f417a8f84fab11b0d10d0` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:11.737953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e75a88-829b-415c-b94e-453c484c5f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f40f50783d9f417a8f84fab11b0d10d0)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:11.739910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e75a88-829b-415c-b94e-453c484c5f90 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:11.800618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48e75a88-829b-415c-b94e-453c484c5f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f40f50783d9f417a8f84fab11b0d10d0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:11.811100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48e75a88-829b-415c-b94e-453c484c5f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f40f50783d9f417a8f84fab11b0d10d0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:11.827317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48e75a88-829b-415c-b94e-453c484c5f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f40f50783d9f417a8f84fab11b0d10d0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:11.828407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48e75a88-829b-415c-b94e-453c484c5f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f40f50783d9f417a8f84fab11b0d10d0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:11.841851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48e75a88-829b-415c-b94e-453c484c5f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f40f50783d9f417a8f84fab11b0d10d0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:11.850291 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1295/2526] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:52:11 2022] DELETE /identity/v3/users/f40f50783d9f417a8f84fab11b0d10d0 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:12.131424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e47e9e-0c1d-4e16-bf89-1bafc4fd833e service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:12.152196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e47e9e-0c1d-4e16-bf89-1bafc4fd833e service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:12.225150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded8a3be-2388-447b-876f-eb8b97cb1935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:12.225789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8a3be-2388-447b-876f-eb8b97cb1935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:12.226074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8a3be-2388-447b-876f-eb8b97cb1935 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:12.226346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8a3be-2388-447b-876f-eb8b97cb1935 admin admin] PATH_INFO: `/v3/projects/6741e5e1d01343dbb0889a1de15d6302` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:12.227519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded8a3be-2388-447b-876f-eb8b97cb1935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6741e5e1d01343dbb0889a1de15d6302)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:12.229474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded8a3be-2388-447b-876f-eb8b97cb1935 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:12.280026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ded8a3be-2388-447b-876f-eb8b97cb1935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6741e5e1d01343dbb0889a1de15d6302', 'request_id': 'req-ded8a3be-2388-447b-876f-eb8b97cb1935'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:12.288777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ded8a3be-2388-447b-876f-eb8b97cb1935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6741e5e1d01343dbb0889a1de15d6302', 'request_id': 'req-ded8a3be-2388-447b-876f-eb8b97cb1935'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:12.293911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:12.296550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ded8a3be-2388-447b-876f-eb8b97cb1935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6741e5e1d01343dbb0889a1de15d6302', 'request_id': 'req-ded8a3be-2388-447b-876f-eb8b97cb1935'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:12.315518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ded8a3be-2388-447b-876f-eb8b97cb1935 admin admin] The token cache is being invalidate because project 6741e5e1d01343dbb0889a1de15d6302 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:52:12.315888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ded8a3be-2388-447b-876f-eb8b97cb1935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:12.316667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ded8a3be-2388-447b-876f-eb8b97cb1935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:12.320054 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1232/2527] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:52:12 2022] DELETE /identity/v3/projects/6741e5e1d01343dbb0889a1de15d6302 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:12.324057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e75a88-829b-415c-b94e-453c484c5f90 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:12.325539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-48e75a88-829b-415c-b94e-453c484c5f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:12.341507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e75a88-829b-415c-b94e-453c484c5f90 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:12.433538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdd6da-1778-450f-8167-ed66c0f17c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:12.434722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdd6da-1778-450f-8167-ed66c0f17c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:12.435117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdd6da-1778-450f-8167-ed66c0f17c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:12.435531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdd6da-1778-450f-8167-ed66c0f17c4e admin admin] PATH_INFO: `/v3/users/6aa6de193f6c41b9b9a2c3fd1155b489` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:12.436903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cdd6da-1778-450f-8167-ed66c0f17c4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aa6de193f6c41b9b9a2c3fd1155b489)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:12.439186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cdd6da-1778-450f-8167-ed66c0f17c4e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:12.494167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-55cdd6da-1778-450f-8167-ed66c0f17c4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aa6de193f6c41b9b9a2c3fd1155b489'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:12.505127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-55cdd6da-1778-450f-8167-ed66c0f17c4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aa6de193f6c41b9b9a2c3fd1155b489'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:12.510485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:12.521830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-55cdd6da-1778-450f-8167-ed66c0f17c4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aa6de193f6c41b9b9a2c3fd1155b489'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:12.522607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-55cdd6da-1778-450f-8167-ed66c0f17c4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aa6de193f6c41b9b9a2c3fd1155b489'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:12.530572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-55cdd6da-1778-450f-8167-ed66c0f17c4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aa6de193f6c41b9b9a2c3fd1155b489'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:12.537953 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1296/2528] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:52:12 2022] DELETE /identity/v3/users/6aa6de193f6c41b9b9a2c3fd1155b489 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:12.721489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded8a3be-2388-447b-876f-eb8b97cb1935 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:12.723143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ded8a3be-2388-447b-876f-eb8b97cb1935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:12.753576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded8a3be-2388-447b-876f-eb8b97cb1935 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:12.814956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce3402-f4b2-4d13-ab74-d1ce3028b4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:12.816522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce3402-f4b2-4d13-ab74-d1ce3028b4d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:12.816866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce3402-f4b2-4d13-ab74-d1ce3028b4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:12.817163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce3402-f4b2-4d13-ab74-d1ce3028b4d1 admin admin] PATH_INFO: `/v3/projects/ce548a968fa74341965cfe34485d3291` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:12.818409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce3402-f4b2-4d13-ab74-d1ce3028b4d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce548a968fa74341965cfe34485d3291)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:12.820406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce3402-f4b2-4d13-ab74-d1ce3028b4d1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:12.899091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6cce3402-f4b2-4d13-ab74-d1ce3028b4d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce548a968fa74341965cfe34485d3291', 'request_id': 'req-6cce3402-f4b2-4d13-ab74-d1ce3028b4d1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:12.915526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6cce3402-f4b2-4d13-ab74-d1ce3028b4d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce548a968fa74341965cfe34485d3291', 'request_id': 'req-6cce3402-f4b2-4d13-ab74-d1ce3028b4d1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:12.929379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6cce3402-f4b2-4d13-ab74-d1ce3028b4d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce548a968fa74341965cfe34485d3291', 'request_id': 'req-6cce3402-f4b2-4d13-ab74-d1ce3028b4d1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:12.942434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6cce3402-f4b2-4d13-ab74-d1ce3028b4d1 admin admin] The token cache is being invalidate because project ce548a968fa74341965cfe34485d3291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:52:12.942792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6cce3402-f4b2-4d13-ab74-d1ce3028b4d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:12.943464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6cce3402-f4b2-4d13-ab74-d1ce3028b4d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:12.948388 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1233/2529] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:52:12 2022] DELETE /identity/v3/projects/ce548a968fa74341965cfe34485d3291 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:12.950994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c104e4e-ce5e-4e92-806a-55c7cc2a9c41 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:12.951349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c104e4e-ce5e-4e92-806a-55c7cc2a9c41 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:12.951690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c104e4e-ce5e-4e92-806a-55c7cc2a9c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:12.979712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1c104e4e-ce5e-4e92-806a-55c7cc2a9c41 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:12.981792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1c104e4e-ce5e-4e92-806a-55c7cc2a9c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:13.058233 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1297/2530] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:13.067199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de6cfc-1a2f-4d3e-a657-c4aac268d73d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:13.067487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de6cfc-1a2f-4d3e-a657-c4aac268d73d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:13.067769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de6cfc-1a2f-4d3e-a657-c4aac268d73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:13.092670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-90de6cfc-1a2f-4d3e-a657-c4aac268d73d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:13.094323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-90de6cfc-1a2f-4d3e-a657-c4aac268d73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:13.124819 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1234/2531] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:52:13.126453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c104e4e-ce5e-4e92-806a-55c7cc2a9c41 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:13.139679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c104e4e-ce5e-4e92-806a-55c7cc2a9c41 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:13.148552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69137049-ab42-4d16-bf2e-1bc92e21d6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:13.149878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69137049-ab42-4d16-bf2e-1bc92e21d6aa admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:13.150617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69137049-ab42-4d16-bf2e-1bc92e21d6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:13.151235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69137049-ab42-4d16-bf2e-1bc92e21d6aa admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:13.152567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69137049-ab42-4d16-bf2e-1bc92e21d6aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:52:13.153160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69137049-ab42-4d16-bf2e-1bc92e21d6aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:13.155959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69137049-ab42-4d16-bf2e-1bc92e21d6aa admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:13.177108 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1298/2532] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:52:13.178287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de6cfc-1a2f-4d3e-a657-c4aac268d73d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:13.188398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de6cfc-1a2f-4d3e-a657-c4aac268d73d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:13.194855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd2fa80-22f3-4655-9ae7-ea59dfbf28ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:13.203177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2fa80-22f3-4655-9ae7-ea59dfbf28ab admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:13.203656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2fa80-22f3-4655-9ae7-ea59dfbf28ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:13.204085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2fa80-22f3-4655-9ae7-ea59dfbf28ab admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:13.204877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd2fa80-22f3-4655-9ae7-ea59dfbf28ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:13.207865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd2fa80-22f3-4655-9ae7-ea59dfbf28ab admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:13.278561 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1235/2533] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:52:13 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:13.283203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69137049-ab42-4d16-bf2e-1bc92e21d6aa admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:13.296968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69137049-ab42-4d16-bf2e-1bc92e21d6aa admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:13.390150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d8204-df19-4a87-8a0c-d5d1cd168edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:13.390709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d8204-df19-4a87-8a0c-d5d1cd168edc admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:13.390991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d8204-df19-4a87-8a0c-d5d1cd168edc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:13.394640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d8204-df19-4a87-8a0c-d5d1cd168edc admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:13.394640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d8204-df19-4a87-8a0c-d5d1cd168edc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:13.394640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d8204-df19-4a87-8a0c-d5d1cd168edc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:13.419905 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1299/2534] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:52:13 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:13.424852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd2fa80-22f3-4655-9ae7-ea59dfbf28ab admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:13.440292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd2fa80-22f3-4655-9ae7-ea59dfbf28ab admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:13.453044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68108062-a2c1-4184-ba94-6ddcd58d77f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:13.454442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68108062-a2c1-4184-ba94-6ddcd58d77f4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:13.456951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68108062-a2c1-4184-ba94-6ddcd58d77f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:13.457239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68108062-a2c1-4184-ba94-6ddcd58d77f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:13.458131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68108062-a2c1-4184-ba94-6ddcd58d77f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:13.462352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68108062-a2c1-4184-ba94-6ddcd58d77f4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:13.490525 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1236/2535] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:52:13.491536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d8204-df19-4a87-8a0c-d5d1cd168edc admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:13.523585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d8204-df19-4a87-8a0c-d5d1cd168edc admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:13.539679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e6057-f367-4381-b0ec-c5193ca55578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:13.540629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e6057-f367-4381-b0ec-c5193ca55578 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:13.540957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e6057-f367-4381-b0ec-c5193ca55578 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:13.541368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e6057-f367-4381-b0ec-c5193ca55578 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:13.542166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e6057-f367-4381-b0ec-c5193ca55578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:13.546324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e6057-f367-4381-b0ec-c5193ca55578 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:13.570816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68108062-a2c1-4184-ba94-6ddcd58d77f4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:13.589206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68108062-a2c1-4184-ba94-6ddcd58d77f4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:13.590262 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1300/2536] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:52:13.595368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c303b-5b4d-4bbb-9fb0-66865744908a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:13.596057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c303b-5b4d-4bbb-9fb0-66865744908a admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:13.596415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c303b-5b4d-4bbb-9fb0-66865744908a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:13.596774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c303b-5b4d-4bbb-9fb0-66865744908a admin admin] PATH_INFO: `/v3/projects/baf5302894a8431f9cb39838a629f2ff/users/1cd9db29440d47cc84756871260d32b6/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:13.628683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c303b-5b4d-4bbb-9fb0-66865744908a admin admin] RBAC: Authorizing `identity:create_grant(project_id=baf5302894a8431f9cb39838a629f2ff, user_id=1cd9db29440d47cc84756871260d32b6, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:13.630878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c303b-5b4d-4bbb-9fb0-66865744908a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:13.649851 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1237/2537] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:52:13 2022] PUT /identity/v3/projects/baf5302894a8431f9cb39838a629f2ff/users/1cd9db29440d47cc84756871260d32b6/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:52:13.660445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf5cd9-078a-4bcd-841c-ba7652b4ea72 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:13.660729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf5cd9-078a-4bcd-841c-ba7652b4ea72 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:13.661003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf5cd9-078a-4bcd-841c-ba7652b4ea72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:13.699054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a7bf5cd9-078a-4bcd-841c-ba7652b4ea72 None None] MFA Rules not processed for user `1cd9db29440d47cc84756871260d32b6`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:13.700647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a7bf5cd9-078a-4bcd-841c-ba7652b4ea72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:13.753088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:13.799479 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1301/2538] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:13.804927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c303b-5b4d-4bbb-9fb0-66865744908a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:13.834984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:13.895981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c303b-5b4d-4bbb-9fb0-66865744908a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:13.902991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ede2e-86ef-4e98-8593-608b565a0995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:13.904239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ede2e-86ef-4e98-8593-608b565a0995 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:13.904837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ede2e-86ef-4e98-8593-608b565a0995 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:13.905330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ede2e-86ef-4e98-8593-608b565a0995 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:13.906145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ede2e-86ef-4e98-8593-608b565a0995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:13.908108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ede2e-86ef-4e98-8593-608b565a0995 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:13.960474 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1238/2539] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:52:13 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:13.964414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf5cd9-078a-4bcd-841c-ba7652b4ea72 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:13.976872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf5cd9-078a-4bcd-841c-ba7652b4ea72 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:14.067118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ede2e-86ef-4e98-8593-608b565a0995 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:14.067855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fa3ede2e-86ef-4e98-8593-608b565a0995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:14.090419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20149af-1352-492e-91e3-d9de83aa4578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:14.091374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20149af-1352-492e-91e3-d9de83aa4578 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:14.091806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20149af-1352-492e-91e3-d9de83aa4578 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:14.092227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20149af-1352-492e-91e3-d9de83aa4578 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:14.093021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20149af-1352-492e-91e3-d9de83aa4578 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:14.095129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20149af-1352-492e-91e3-d9de83aa4578 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:14.128364 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1302/2540] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:52:13 2022] POST /identity/v3/users => generated 299 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:14.131947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20149af-1352-492e-91e3-d9de83aa4578 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:14.144296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20149af-1352-492e-91e3-d9de83aa4578 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:14.152232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650da0c-da89-445b-8bf3-5912f73ddd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:14.153403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650da0c-da89-445b-8bf3-5912f73ddd2c admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:14.153558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650da0c-da89-445b-8bf3-5912f73ddd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:14.153811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650da0c-da89-445b-8bf3-5912f73ddd2c admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:14.154621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650da0c-da89-445b-8bf3-5912f73ddd2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:14.156217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650da0c-da89-445b-8bf3-5912f73ddd2c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:14.169555 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1303/2541] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:52:14.173700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650da0c-da89-445b-8bf3-5912f73ddd2c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:14.182170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ede2e-86ef-4e98-8593-608b565a0995 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:14.187894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650da0c-da89-445b-8bf3-5912f73ddd2c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:14.193225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d007716-1c14-494f-bddc-917b028d241b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:14.194287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d007716-1c14-494f-bddc-917b028d241b admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:14.194287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d007716-1c14-494f-bddc-917b028d241b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:14.194482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d007716-1c14-494f-bddc-917b028d241b admin admin] PATH_INFO: `/v3/projects/e5882c021bbf42ee98b9e769a5cd1e92/users/06630999e423468abf0784062b11688a/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:14.196377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8f1fb-de5b-4374-aedd-353efb58fefa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:14.196999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8f1fb-de5b-4374-aedd-353efb58fefa service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:14.197733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8f1fb-de5b-4374-aedd-353efb58fefa service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:14.197733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8f1fb-de5b-4374-aedd-353efb58fefa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:14.204114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b2a8f1fb-de5b-4374-aedd-353efb58fefa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:14.215157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d007716-1c14-494f-bddc-917b028d241b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5882c021bbf42ee98b9e769a5cd1e92, user_id=06630999e423468abf0784062b11688a, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:14.219288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d007716-1c14-494f-bddc-917b028d241b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:14.242387 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1304/2542] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:52:14 2022] PUT /identity/v3/projects/e5882c021bbf42ee98b9e769a5cd1e92/users/06630999e423468abf0784062b11688a/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:52:14.243742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d007716-1c14-494f-bddc-917b028d241b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:14.308761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a8f1fb-de5b-4374-aedd-353efb58fefa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:14.310474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a8f1fb-de5b-4374-aedd-353efb58fefa service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:14.325649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d007716-1c14-494f-bddc-917b028d241b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:14.325883 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1239/2543] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:14 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:52:14.332708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c965c6-e219-421b-b62b-699a306dceab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:14.333336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c965c6-e219-421b-b62b-699a306dceab admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:14.333615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c965c6-e219-421b-b62b-699a306dceab admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:14.333886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c965c6-e219-421b-b62b-699a306dceab admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:14.334687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c965c6-e219-421b-b62b-699a306dceab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:14.340119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c965c6-e219-421b-b62b-699a306dceab admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:14.354464 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1305/2544] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:52:14.358991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8f1fb-de5b-4374-aedd-353efb58fefa service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:14.374111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8f1fb-de5b-4374-aedd-353efb58fefa service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:14.381403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df8c69-6d9d-464a-9b9e-04d65da75114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:14.382155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df8c69-6d9d-464a-9b9e-04d65da75114 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:14.382530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df8c69-6d9d-464a-9b9e-04d65da75114 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:14.382954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df8c69-6d9d-464a-9b9e-04d65da75114 admin admin] PATH_INFO: `/v3/projects/e5882c021bbf42ee98b9e769a5cd1e92/users/06630999e423468abf0784062b11688a/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:14.385138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df8c69-6d9d-464a-9b9e-04d65da75114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5882c021bbf42ee98b9e769a5cd1e92, user_id=06630999e423468abf0784062b11688a, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:14.387919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df8c69-6d9d-464a-9b9e-04d65da75114 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:14.400222 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1240/2545] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:52:14 2022] PUT /identity/v3/projects/e5882c021bbf42ee98b9e769a5cd1e92/users/06630999e423468abf0784062b11688a/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:14.411887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be5a8a-9726-4587-8725-c9127279fe6f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:14.412176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be5a8a-9726-4587-8725-c9127279fe6f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:14.412527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be5a8a-9726-4587-8725-c9127279fe6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:14.459268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b3be5a8a-9726-4587-8725-c9127279fe6f None None] MFA Rules not processed for user `06630999e423468abf0784062b11688a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:14.461201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b3be5a8a-9726-4587-8725-c9127279fe6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:14.506233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:14.625830 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1306/2546] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:15.522356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df8c69-6d9d-464a-9b9e-04d65da75114 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:15.524683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d6df8c69-6d9d-464a-9b9e-04d65da75114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:15.543239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be2b70-6e99-40f9-943a-88a4e0fa4513 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:15.543532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be2b70-6e99-40f9-943a-88a4e0fa4513 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:15.543803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be2b70-6e99-40f9-943a-88a4e0fa4513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:15.557039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:15.579059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-08be2b70-6e99-40f9-943a-88a4e0fa4513 None None] MFA Rules not processed for user `04add76a65d74165b65189a5fd1f47ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:15.582987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-08be2b70-6e99-40f9-943a-88a4e0fa4513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:15.641393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df8c69-6d9d-464a-9b9e-04d65da75114 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:15.649739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab8479-360e-4c04-880e-fc32e20f6f2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:15.650599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab8479-360e-4c04-880e-fc32e20f6f2e service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:15.651013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab8479-360e-4c04-880e-fc32e20f6f2e service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:15.651760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab8479-360e-4c04-880e-fc32e20f6f2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:15.665201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab8479-360e-4c04-880e-fc32e20f6f2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:15.668446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab8479-360e-4c04-880e-fc32e20f6f2e service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:15.693734 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1307/2547] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:15.696717 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1241/2548] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:15 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:18.513385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be2b70-6e99-40f9-943a-88a4e0fa4513 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:18.515751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-08be2b70-6e99-40f9-943a-88a4e0fa4513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:18.651660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be2b70-6e99-40f9-943a-88a4e0fa4513 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:18.668128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbda48-c34c-45de-874a-6efc3c14795c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:18.669090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbda48-c34c-45de-874a-6efc3c14795c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:18.669435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbda48-c34c-45de-874a-6efc3c14795c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:18.669632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbda48-c34c-45de-874a-6efc3c14795c admin admin] PATH_INFO: `/v3/users/50d13e9e2afc44fd91caf096922460ed` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:18.672268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fbda48-c34c-45de-874a-6efc3c14795c admin admin] RBAC: Authorizing `identity:delete_user(user_id=50d13e9e2afc44fd91caf096922460ed)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:18.675876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fbda48-c34c-45de-874a-6efc3c14795c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:18.786803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-91fbda48-c34c-45de-874a-6efc3c14795c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50d13e9e2afc44fd91caf096922460ed'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:18.804192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-91fbda48-c34c-45de-874a-6efc3c14795c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50d13e9e2afc44fd91caf096922460ed'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:18.807988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:18.835615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-91fbda48-c34c-45de-874a-6efc3c14795c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50d13e9e2afc44fd91caf096922460ed'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:18.837531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-91fbda48-c34c-45de-874a-6efc3c14795c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50d13e9e2afc44fd91caf096922460ed'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:18.854317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-91fbda48-c34c-45de-874a-6efc3c14795c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50d13e9e2afc44fd91caf096922460ed'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:18.863942 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1308/2549] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:52:18 2022] DELETE /identity/v3/users/50d13e9e2afc44fd91caf096922460ed => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:19.208457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab8479-360e-4c04-880e-fc32e20f6f2e service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:19.225120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab8479-360e-4c04-880e-fc32e20f6f2e service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:19.353650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95329588-525a-4277-99a3-d60d05d6b284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:19.354295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95329588-525a-4277-99a3-d60d05d6b284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:19.354625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95329588-525a-4277-99a3-d60d05d6b284 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:19.354904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95329588-525a-4277-99a3-d60d05d6b284 admin admin] PATH_INFO: `/v3/projects/bbd3b538b14a4f77a6a50e9fbe849427` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:19.356048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95329588-525a-4277-99a3-d60d05d6b284 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbd3b538b14a4f77a6a50e9fbe849427)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:19.358108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95329588-525a-4277-99a3-d60d05d6b284 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:19.423883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95329588-525a-4277-99a3-d60d05d6b284 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbd3b538b14a4f77a6a50e9fbe849427', 'request_id': 'req-95329588-525a-4277-99a3-d60d05d6b284'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:19.435976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95329588-525a-4277-99a3-d60d05d6b284 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbd3b538b14a4f77a6a50e9fbe849427', 'request_id': 'req-95329588-525a-4277-99a3-d60d05d6b284'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:19.444654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95329588-525a-4277-99a3-d60d05d6b284 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbd3b538b14a4f77a6a50e9fbe849427', 'request_id': 'req-95329588-525a-4277-99a3-d60d05d6b284'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:19.462731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95329588-525a-4277-99a3-d60d05d6b284 admin admin] The token cache is being invalidate because project bbd3b538b14a4f77a6a50e9fbe849427 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:52:19.463025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95329588-525a-4277-99a3-d60d05d6b284 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:19.463906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95329588-525a-4277-99a3-d60d05d6b284 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:19.466621 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1242/2550] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:52:19 2022] DELETE /identity/v3/projects/bbd3b538b14a4f77a6a50e9fbe849427 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:19.472128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972953f-429b-45ea-afac-6fa216e13286 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:19.472510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972953f-429b-45ea-afac-6fa216e13286 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:19.472918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972953f-429b-45ea-afac-6fa216e13286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:19.498762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4972953f-429b-45ea-afac-6fa216e13286 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:19.500933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4972953f-429b-45ea-afac-6fa216e13286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:19.545343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95329588-525a-4277-99a3-d60d05d6b284 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:19.547490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-95329588-525a-4277-99a3-d60d05d6b284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:19.614630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:19.622733 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1309/2551] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:19.631920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0b403-7169-497a-ad7f-11246191ee08 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:19.632226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0b403-7169-497a-ad7f-11246191ee08 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:19.632510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0b403-7169-497a-ad7f-11246191ee08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:19.667708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-52a0b403-7169-497a-ad7f-11246191ee08 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:19.669326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-52a0b403-7169-497a-ad7f-11246191ee08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:19.674819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95329588-525a-4277-99a3-d60d05d6b284 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:19.681601 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1310/2552] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:52:19.685599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0b403-7169-497a-ad7f-11246191ee08 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:19.690962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d44e64-336d-4bdc-b16f-e20e56196f66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:19.691977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d44e64-336d-4bdc-b16f-e20e56196f66 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:19.692476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d44e64-336d-4bdc-b16f-e20e56196f66 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:19.692962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d44e64-336d-4bdc-b16f-e20e56196f66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:19.695112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f8d44e64-336d-4bdc-b16f-e20e56196f66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:19.702107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0b403-7169-497a-ad7f-11246191ee08 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:19.709372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ad878-e40e-4bad-b412-54080da069ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:19.711427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ad878-e40e-4bad-b412-54080da069ac admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:19.711783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ad878-e40e-4bad-b412-54080da069ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:19.712154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ad878-e40e-4bad-b412-54080da069ac admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:19.712894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ad878-e40e-4bad-b412-54080da069ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:52:19.713398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ad878-e40e-4bad-b412-54080da069ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:19.716014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ad878-e40e-4bad-b412-54080da069ac admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:19.732725 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1311/2553] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:52:19.736998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ad878-e40e-4bad-b412-54080da069ac admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:19.752421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ad878-e40e-4bad-b412-54080da069ac admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:19.759005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1513d71-d6ba-41ed-ae31-5a2722974354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:19.759864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1513d71-d6ba-41ed-ae31-5a2722974354 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:19.760356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1513d71-d6ba-41ed-ae31-5a2722974354 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:19.760802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1513d71-d6ba-41ed-ae31-5a2722974354 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:19.761430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1513d71-d6ba-41ed-ae31-5a2722974354 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:19.765479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1513d71-d6ba-41ed-ae31-5a2722974354 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:19.791857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d44e64-336d-4bdc-b16f-e20e56196f66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:19.793582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d44e64-336d-4bdc-b16f-e20e56196f66 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:19.807153 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1243/2554] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:19.824676 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1312/2555] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:52:19 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:19.825948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d44e64-336d-4bdc-b16f-e20e56196f66 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:19.843359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d44e64-336d-4bdc-b16f-e20e56196f66 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:19.908261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1513d71-d6ba-41ed-ae31-5a2722974354 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:19.910086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b1513d71-d6ba-41ed-ae31-5a2722974354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:19.972314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24905a87-7de6-4da8-9c98-edef0daf2079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:19.972933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24905a87-7de6-4da8-9c98-edef0daf2079 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:19.973321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24905a87-7de6-4da8-9c98-edef0daf2079 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:19.973471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24905a87-7de6-4da8-9c98-edef0daf2079 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:19.974108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24905a87-7de6-4da8-9c98-edef0daf2079 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:19.976110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24905a87-7de6-4da8-9c98-edef0daf2079 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:20.025045 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1244/2556] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:52:19 2022] POST /identity/v3/users => generated 286 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:20.029094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24905a87-7de6-4da8-9c98-edef0daf2079 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:20.051128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24905a87-7de6-4da8-9c98-edef0daf2079 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:20.067236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfa76d-ffef-421b-9431-04c0261bcd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:20.068216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfa76d-ffef-421b-9431-04c0261bcd80 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:20.068345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1513d71-d6ba-41ed-ae31-5a2722974354 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:20.068447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfa76d-ffef-421b-9431-04c0261bcd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:20.068447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfa76d-ffef-421b-9431-04c0261bcd80 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:20.069346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebfa76d-ffef-421b-9431-04c0261bcd80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:20.071089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebfa76d-ffef-421b-9431-04c0261bcd80 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:20.083286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3c870-71fb-4623-881b-702650bbacab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:20.083786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3c870-71fb-4623-881b-702650bbacab admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:20.083786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3c870-71fb-4623-881b-702650bbacab admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:20.083786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3c870-71fb-4623-881b-702650bbacab admin admin] PATH_INFO: `/v3/users/b61faa86ae0d4eedb371407380c22bfe` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:20.088105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d3c870-71fb-4623-881b-702650bbacab admin admin] RBAC: Authorizing `identity:delete_user(user_id=b61faa86ae0d4eedb371407380c22bfe)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:20.090261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d3c870-71fb-4623-881b-702650bbacab admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:20.100836 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1245/2557] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:52:20.104565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfa76d-ffef-421b-9431-04c0261bcd80 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:20.131211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfa76d-ffef-421b-9431-04c0261bcd80 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:20.153390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700b556-5f00-41e1-8aa7-db0ea9f40abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:20.154485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700b556-5f00-41e1-8aa7-db0ea9f40abd admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:20.154947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700b556-5f00-41e1-8aa7-db0ea9f40abd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:20.156325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700b556-5f00-41e1-8aa7-db0ea9f40abd admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:20.158971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700b556-5f00-41e1-8aa7-db0ea9f40abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:20.163583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700b556-5f00-41e1-8aa7-db0ea9f40abd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:20.184282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:20.191665 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1246/2558] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:52:20.195696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700b556-5f00-41e1-8aa7-db0ea9f40abd admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:20.204437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-69d3c870-71fb-4623-881b-702650bbacab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b61faa86ae0d4eedb371407380c22bfe'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:20.210818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700b556-5f00-41e1-8aa7-db0ea9f40abd admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:20.217196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-69d3c870-71fb-4623-881b-702650bbacab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b61faa86ae0d4eedb371407380c22bfe'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:20.226431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddaa014-c4b4-4e0b-a58c-9a9770ac60d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:20.228025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddaa014-c4b4-4e0b-a58c-9a9770ac60d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:20.228494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddaa014-c4b4-4e0b-a58c-9a9770ac60d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:20.228944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddaa014-c4b4-4e0b-a58c-9a9770ac60d9 admin admin] PATH_INFO: `/v3/projects/4ae105e248b648f182d75d1eb8db51df/users/bba86a7a9dab49d191e553647250f279/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:20.230801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-69d3c870-71fb-4623-881b-702650bbacab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b61faa86ae0d4eedb371407380c22bfe'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:20.231602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-69d3c870-71fb-4623-881b-702650bbacab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b61faa86ae0d4eedb371407380c22bfe'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:20.245465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-69d3c870-71fb-4623-881b-702650bbacab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b61faa86ae0d4eedb371407380c22bfe'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:20.251530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddaa014-c4b4-4e0b-a58c-9a9770ac60d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ae105e248b648f182d75d1eb8db51df, user_id=bba86a7a9dab49d191e553647250f279, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:20.256870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddaa014-c4b4-4e0b-a58c-9a9770ac60d9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:20.257813 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1313/2559] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:52:19 2022] DELETE /identity/v3/users/b61faa86ae0d4eedb371407380c22bfe => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:52:20.262461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bb117-17e2-4b2b-a5fc-0242d3d53e9a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:20.262774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bb117-17e2-4b2b-a5fc-0242d3d53e9a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:20.263066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bb117-17e2-4b2b-a5fc-0242d3d53e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:20.275744 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1247/2560] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:52:20 2022] PUT /identity/v3/projects/4ae105e248b648f182d75d1eb8db51df/users/bba86a7a9dab49d191e553647250f279/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:20.287971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df4b45-dcbe-48c5-b46a-8ec77740966b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:20.289857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df4b45-dcbe-48c5-b46a-8ec77740966b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:20.294152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df4b45-dcbe-48c5-b46a-8ec77740966b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:20.329992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ae2bb117-17e2-4b2b-a5fc-0242d3d53e9a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:20.331906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ae2bb117-17e2-4b2b-a5fc-0242d3d53e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:20.344265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c6df4b45-dcbe-48c5-b46a-8ec77740966b None None] MFA Rules not processed for user `bba86a7a9dab49d191e553647250f279`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:20.345964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c6df4b45-dcbe-48c5-b46a-8ec77740966b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:20.353111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:20.470388 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1314/2561] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:20.472444 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1248/2562] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:20.557821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bb117-17e2-4b2b-a5fc-0242d3d53e9a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:20.559314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ae2bb117-17e2-4b2b-a5fc-0242d3d53e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:20.673906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bb117-17e2-4b2b-a5fc-0242d3d53e9a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:20.687923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0c090-47d8-49be-817a-41444957d6ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:20.692244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0c090-47d8-49be-817a-41444957d6ca service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:20.692786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0c090-47d8-49be-817a-41444957d6ca service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:20.693546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0c090-47d8-49be-817a-41444957d6ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:20.711185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f0c090-47d8-49be-817a-41444957d6ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:20.716021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f0c090-47d8-49be-817a-41444957d6ca service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:20.730691 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1315/2563] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:20.808895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df4b45-dcbe-48c5-b46a-8ec77740966b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:20.810366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c6df4b45-dcbe-48c5-b46a-8ec77740966b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:20.924950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df4b45-dcbe-48c5-b46a-8ec77740966b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:20.933538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8002bd-f64a-4002-8d96-8b4d525aa013 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:20.934772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8002bd-f64a-4002-8d96-8b4d525aa013 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:20.935144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8002bd-f64a-4002-8d96-8b4d525aa013 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:20.935530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8002bd-f64a-4002-8d96-8b4d525aa013 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:20.953607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8002bd-f64a-4002-8d96-8b4d525aa013 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:20.955816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8002bd-f64a-4002-8d96-8b4d525aa013 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:20.969509 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1249/2564] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:21.483649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0c090-47d8-49be-817a-41444957d6ca service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:21.486357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e9f0c090-47d8-49be-817a-41444957d6ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:21.517765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0c090-47d8-49be-817a-41444957d6ca service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:21.525712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86255635-966e-43ed-9f94-ab6035b4c5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:21.526797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86255635-966e-43ed-9f94-ab6035b4c5c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:21.527162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86255635-966e-43ed-9f94-ab6035b4c5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:21.527513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86255635-966e-43ed-9f94-ab6035b4c5c8 admin admin] PATH_INFO: `/v3/projects/b2b9de2ad3ad459794059338dd371e0b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:21.529030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86255635-966e-43ed-9f94-ab6035b4c5c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2b9de2ad3ad459794059338dd371e0b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:21.541478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86255635-966e-43ed-9f94-ab6035b4c5c8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:21.616830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-86255635-966e-43ed-9f94-ab6035b4c5c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2b9de2ad3ad459794059338dd371e0b', 'request_id': 'req-86255635-966e-43ed-9f94-ab6035b4c5c8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:21.619212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:21.635512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-86255635-966e-43ed-9f94-ab6035b4c5c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2b9de2ad3ad459794059338dd371e0b', 'request_id': 'req-86255635-966e-43ed-9f94-ab6035b4c5c8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:21.646298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-86255635-966e-43ed-9f94-ab6035b4c5c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2b9de2ad3ad459794059338dd371e0b', 'request_id': 'req-86255635-966e-43ed-9f94-ab6035b4c5c8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:21.692582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-86255635-966e-43ed-9f94-ab6035b4c5c8 admin admin] The token cache is being invalidate because project b2b9de2ad3ad459794059338dd371e0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:52:21.693220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-86255635-966e-43ed-9f94-ab6035b4c5c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:21.693681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-86255635-966e-43ed-9f94-ab6035b4c5c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:21.704198 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1316/2565] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:52:21 2022] DELETE /identity/v3/projects/b2b9de2ad3ad459794059338dd371e0b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:21.704407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8002bd-f64a-4002-8d96-8b4d525aa013 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:21.723556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8002bd-f64a-4002-8d96-8b4d525aa013 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:21.890398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71be97-fb25-4333-91df-89cd5a6b63e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:21.891321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71be97-fb25-4333-91df-89cd5a6b63e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:21.891666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71be97-fb25-4333-91df-89cd5a6b63e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:21.892004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71be97-fb25-4333-91df-89cd5a6b63e2 admin admin] PATH_INFO: `/v3/users/a433d354781c4c199b855cefe099eba8` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:21.895132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71be97-fb25-4333-91df-89cd5a6b63e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a433d354781c4c199b855cefe099eba8)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:21.897598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71be97-fb25-4333-91df-89cd5a6b63e2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:21.975643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:21.989280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b71be97-fb25-4333-91df-89cd5a6b63e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a433d354781c4c199b855cefe099eba8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:21.990118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b71be97-fb25-4333-91df-89cd5a6b63e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a433d354781c4c199b855cefe099eba8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:21.990950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b71be97-fb25-4333-91df-89cd5a6b63e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a433d354781c4c199b855cefe099eba8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:22.001033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b71be97-fb25-4333-91df-89cd5a6b63e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a433d354781c4c199b855cefe099eba8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:22.029827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b71be97-fb25-4333-91df-89cd5a6b63e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a433d354781c4c199b855cefe099eba8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:22.052240 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1250/2566] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:52:21 2022] DELETE /identity/v3/users/a433d354781c4c199b855cefe099eba8 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:22.711976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86255635-966e-43ed-9f94-ab6035b4c5c8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:22.713572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-86255635-966e-43ed-9f94-ab6035b4c5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:22.735441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86255635-966e-43ed-9f94-ab6035b4c5c8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:22.884846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc63fe-ea2f-4dfd-90e1-d51acdd60358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:22.891182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc63fe-ea2f-4dfd-90e1-d51acdd60358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:22.891489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc63fe-ea2f-4dfd-90e1-d51acdd60358 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:22.891769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc63fe-ea2f-4dfd-90e1-d51acdd60358 admin admin] PATH_INFO: `/v3/projects/a11d92672f1d40a0ab70d046f73b5fb8` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:22.897584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc63fe-ea2f-4dfd-90e1-d51acdd60358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a11d92672f1d40a0ab70d046f73b5fb8)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:22.900342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc63fe-ea2f-4dfd-90e1-d51acdd60358 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:23.059647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-72cc63fe-ea2f-4dfd-90e1-d51acdd60358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a11d92672f1d40a0ab70d046f73b5fb8', 'request_id': 'req-72cc63fe-ea2f-4dfd-90e1-d51acdd60358'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:23.075781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-72cc63fe-ea2f-4dfd-90e1-d51acdd60358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a11d92672f1d40a0ab70d046f73b5fb8', 'request_id': 'req-72cc63fe-ea2f-4dfd-90e1-d51acdd60358'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:23.083779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-72cc63fe-ea2f-4dfd-90e1-d51acdd60358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a11d92672f1d40a0ab70d046f73b5fb8', 'request_id': 'req-72cc63fe-ea2f-4dfd-90e1-d51acdd60358'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:23.102958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-72cc63fe-ea2f-4dfd-90e1-d51acdd60358 admin admin] The token cache is being invalidate because project a11d92672f1d40a0ab70d046f73b5fb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:52:23.103502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-72cc63fe-ea2f-4dfd-90e1-d51acdd60358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:23.104261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-72cc63fe-ea2f-4dfd-90e1-d51acdd60358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:23.105902 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1317/2567] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:52:22 2022] DELETE /identity/v3/projects/a11d92672f1d40a0ab70d046f73b5fb8 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:23.112740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a3270-2b3e-4e4e-b23e-b97124a148e3 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:23.113330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a3270-2b3e-4e4e-b23e-b97124a148e3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:23.113504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a3270-2b3e-4e4e-b23e-b97124a148e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:23.152709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5c3a3270-2b3e-4e4e-b23e-b97124a148e3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:23.171565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5c3a3270-2b3e-4e4e-b23e-b97124a148e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:23.262878 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1251/2568] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:23.272107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d45a18-6ceb-4b59-acc8-1a82d41d51f7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:23.272395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d45a18-6ceb-4b59-acc8-1a82d41d51f7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:23.272675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d45a18-6ceb-4b59-acc8-1a82d41d51f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:23.294825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f6d45a18-6ceb-4b59-acc8-1a82d41d51f7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:23.296094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f6d45a18-6ceb-4b59-acc8-1a82d41d51f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:23.304484 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1318/2569] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:52:23.309081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a3270-2b3e-4e4e-b23e-b97124a148e3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:23.320985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a3270-2b3e-4e4e-b23e-b97124a148e3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:23.327593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011c5eb-9a1c-4e50-8060-ad35a3bff99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:23.328205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011c5eb-9a1c-4e50-8060-ad35a3bff99c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:23.328601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011c5eb-9a1c-4e50-8060-ad35a3bff99c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:23.328952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011c5eb-9a1c-4e50-8060-ad35a3bff99c admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:23.330170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011c5eb-9a1c-4e50-8060-ad35a3bff99c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:52:23.330170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011c5eb-9a1c-4e50-8060-ad35a3bff99c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:23.332380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011c5eb-9a1c-4e50-8060-ad35a3bff99c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:23.344719 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1252/2570] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:52:23.350888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d45a18-6ceb-4b59-acc8-1a82d41d51f7 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:23.362177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d45a18-6ceb-4b59-acc8-1a82d41d51f7 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:23.368274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed39bc-cfad-4943-ade4-c88cea14ecc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:23.369018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed39bc-cfad-4943-ade4-c88cea14ecc3 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:23.369378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed39bc-cfad-4943-ade4-c88cea14ecc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:23.370156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed39bc-cfad-4943-ade4-c88cea14ecc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:23.370842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ed39bc-cfad-4943-ade4-c88cea14ecc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:23.372922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ed39bc-cfad-4943-ade4-c88cea14ecc3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:23.405158 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1319/2571] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:52:23 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:23.409619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011c5eb-9a1c-4e50-8060-ad35a3bff99c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:23.424775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011c5eb-9a1c-4e50-8060-ad35a3bff99c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:23.509485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1d112-58c0-4baf-a746-9074f2c7dbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:23.510183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1d112-58c0-4baf-a746-9074f2c7dbda admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:23.510470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1d112-58c0-4baf-a746-9074f2c7dbda admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:23.510805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1d112-58c0-4baf-a746-9074f2c7dbda admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:23.512108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1d112-58c0-4baf-a746-9074f2c7dbda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:23.514391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1d112-58c0-4baf-a746-9074f2c7dbda admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:23.549195 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1253/2572] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:52:23 2022] POST /identity/v3/users => generated 288 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:23.553277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed39bc-cfad-4943-ade4-c88cea14ecc3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:23.564483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed39bc-cfad-4943-ade4-c88cea14ecc3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:23.571024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86f128-6632-4624-b88e-59a45bd9134d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:23.571683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86f128-6632-4624-b88e-59a45bd9134d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:23.572519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86f128-6632-4624-b88e-59a45bd9134d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:23.572890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86f128-6632-4624-b88e-59a45bd9134d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:23.578944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f86f128-6632-4624-b88e-59a45bd9134d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:23.578944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f86f128-6632-4624-b88e-59a45bd9134d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:23.591433 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1320/2573] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:52:23.595273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1d112-58c0-4baf-a746-9074f2c7dbda admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:23.608753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1d112-58c0-4baf-a746-9074f2c7dbda admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:23.616260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de7e26-0fab-4ced-b977-f439aa65ac81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:23.617133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de7e26-0fab-4ced-b977-f439aa65ac81 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:23.617642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de7e26-0fab-4ced-b977-f439aa65ac81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:23.618117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de7e26-0fab-4ced-b977-f439aa65ac81 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:23.618988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de7e26-0fab-4ced-b977-f439aa65ac81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:23.621263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de7e26-0fab-4ced-b977-f439aa65ac81 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:23.641971 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1254/2574] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:52:23.646713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86f128-6632-4624-b88e-59a45bd9134d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:23.665530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86f128-6632-4624-b88e-59a45bd9134d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:23.676609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0aa10e-0d6f-4ac5-99cb-b0da09079681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:23.677623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0aa10e-0d6f-4ac5-99cb-b0da09079681 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:23.678960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0aa10e-0d6f-4ac5-99cb-b0da09079681 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:23.679381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0aa10e-0d6f-4ac5-99cb-b0da09079681 admin admin] PATH_INFO: `/v3/projects/a1c660e5e0f14f16b4b2b43e89609bb2/users/3245b8304f604cfc97da73b4dc884bd5/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:23.703059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0aa10e-0d6f-4ac5-99cb-b0da09079681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1c660e5e0f14f16b4b2b43e89609bb2, user_id=3245b8304f604cfc97da73b4dc884bd5, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:23.718877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0aa10e-0d6f-4ac5-99cb-b0da09079681 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:23.734884 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1321/2575] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:52:23 2022] PUT /identity/v3/projects/a1c660e5e0f14f16b4b2b43e89609bb2/users/3245b8304f604cfc97da73b4dc884bd5/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:23.747757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca9daf-5b0a-4530-b3a3-87eb77e3df23 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:23.748302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca9daf-5b0a-4530-b3a3-87eb77e3df23 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:23.748526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca9daf-5b0a-4530-b3a3-87eb77e3df23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:23.786089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-79ca9daf-5b0a-4530-b3a3-87eb77e3df23 None None] MFA Rules not processed for user `3245b8304f604cfc97da73b4dc884bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:23.787475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-79ca9daf-5b0a-4530-b3a3-87eb77e3df23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:23.794167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:23.898058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0aa10e-0d6f-4ac5-99cb-b0da09079681 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:23.900862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4e0aa10e-0d6f-4ac5-99cb-b0da09079681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:23.909571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:23.961094 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1255/2576] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:23.965573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca9daf-5b0a-4530-b3a3-87eb77e3df23 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:24.039742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0aa10e-0d6f-4ac5-99cb-b0da09079681 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:24.051986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6885a0-5833-4d07-bc70-13d4de1af7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:24.053036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6885a0-5833-4d07-bc70-13d4de1af7d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:24.053491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6885a0-5833-4d07-bc70-13d4de1af7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:24.053920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6885a0-5833-4d07-bc70-13d4de1af7d6 admin admin] PATH_INFO: `/v3/users/1cd9db29440d47cc84756871260d32b6` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:24.058923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6885a0-5833-4d07-bc70-13d4de1af7d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cd9db29440d47cc84756871260d32b6)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:24.061449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6885a0-5833-4d07-bc70-13d4de1af7d6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:24.121702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca9daf-5b0a-4530-b3a3-87eb77e3df23 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:24.137014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594dd402-c70a-4708-b18c-0e4dc9ab5275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:24.137915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dd402-c70a-4708-b18c-0e4dc9ab5275 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:24.138312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dd402-c70a-4708-b18c-0e4dc9ab5275 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:24.138885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dd402-c70a-4708-b18c-0e4dc9ab5275 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:24.139654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594dd402-c70a-4708-b18c-0e4dc9ab5275 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:24.141784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594dd402-c70a-4708-b18c-0e4dc9ab5275 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:24.154195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ca6885a0-5833-4d07-bc70-13d4de1af7d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cd9db29440d47cc84756871260d32b6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:24.175401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ca6885a0-5833-4d07-bc70-13d4de1af7d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cd9db29440d47cc84756871260d32b6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:24.201582 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1256/2577] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:52:23 2022] POST /identity/v3/projects => generated 375 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:24.203064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594dd402-c70a-4708-b18c-0e4dc9ab5275 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:24.209202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ca6885a0-5833-4d07-bc70-13d4de1af7d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cd9db29440d47cc84756871260d32b6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:24.210966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ca6885a0-5833-4d07-bc70-13d4de1af7d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cd9db29440d47cc84756871260d32b6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:24.236266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ca6885a0-5833-4d07-bc70-13d4de1af7d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cd9db29440d47cc84756871260d32b6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:24.246510 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1322/2578] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:52:23 2022] DELETE /identity/v3/users/1cd9db29440d47cc84756871260d32b6 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:52:24.251230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969172c-78a3-4171-bfbc-6c95d03ab156 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:24.251230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969172c-78a3-4171-bfbc-6c95d03ab156 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:24.251640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969172c-78a3-4171-bfbc-6c95d03ab156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:24.284211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f969172c-78a3-4171-bfbc-6c95d03ab156 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:24.285535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f969172c-78a3-4171-bfbc-6c95d03ab156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:24.344070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594dd402-c70a-4708-b18c-0e4dc9ab5275 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:24.417374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:24.441515 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1323/2579] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:52:24.467896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969172c-78a3-4171-bfbc-6c95d03ab156 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:24.468673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a0a66-bc75-43e0-a4e6-a0ba3b91b3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:24.469575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a0a66-bc75-43e0-a4e6-a0ba3b91b3d2 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:24.469768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a0a66-bc75-43e0-a4e6-a0ba3b91b3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:24.470976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a0a66-bc75-43e0-a4e6-a0ba3b91b3d2 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:24.472002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a0a66-bc75-43e0-a4e6-a0ba3b91b3d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:24.472358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f969172c-78a3-4171-bfbc-6c95d03ab156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:24.474750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a0a66-bc75-43e0-a4e6-a0ba3b91b3d2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:24.520391 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1257/2580] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:52:24 2022] POST /identity/v3/users => generated 294 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:52:24.530403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a0a66-bc75-43e0-a4e6-a0ba3b91b3d2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:24.538320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:24.541435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ab9a0a66-bc75-43e0-a4e6-a0ba3b91b3d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:24.571874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a0a66-bc75-43e0-a4e6-a0ba3b91b3d2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:24.579322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969172c-78a3-4171-bfbc-6c95d03ab156 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:24.583691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308c2c5-752b-4ca0-90ca-e9e046420820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:24.584315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308c2c5-752b-4ca0-90ca-e9e046420820 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:24.584750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308c2c5-752b-4ca0-90ca-e9e046420820 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:24.587436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308c2c5-752b-4ca0-90ca-e9e046420820 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:24.587928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2308c2c5-752b-4ca0-90ca-e9e046420820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:24.596105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2308c2c5-752b-4ca0-90ca-e9e046420820 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:24.600029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e69eb3-d9b2-4354-8528-efce89451a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:24.601208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e69eb3-d9b2-4354-8528-efce89451a3d service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:24.601498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e69eb3-d9b2-4354-8528-efce89451a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:24.601774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e69eb3-d9b2-4354-8528-efce89451a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:24.617169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e69eb3-d9b2-4354-8528-efce89451a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:24.618747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e69eb3-d9b2-4354-8528-efce89451a3d service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:24.625449 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1258/2581] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:52:24.630147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308c2c5-752b-4ca0-90ca-e9e046420820 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:24.648596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308c2c5-752b-4ca0-90ca-e9e046420820 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:24.652463 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1324/2582] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:24.655751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e570a53-c9bd-4ac4-9523-c94fde0f3028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:24.656503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e570a53-c9bd-4ac4-9523-c94fde0f3028 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:24.657032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e570a53-c9bd-4ac4-9523-c94fde0f3028 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:24.657442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e570a53-c9bd-4ac4-9523-c94fde0f3028 admin admin] PATH_INFO: `/v3/projects/9d8830defffa4b8dbacf6bbbd7ab4205/users/e31b54cd54d54f5ebfb064edac6dfe3b/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:24.679508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e570a53-c9bd-4ac4-9523-c94fde0f3028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d8830defffa4b8dbacf6bbbd7ab4205, user_id=e31b54cd54d54f5ebfb064edac6dfe3b, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:24.682498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e570a53-c9bd-4ac4-9523-c94fde0f3028 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:24.703128 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1259/2583] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:52:24 2022] PUT /identity/v3/projects/9d8830defffa4b8dbacf6bbbd7ab4205/users/e31b54cd54d54f5ebfb064edac6dfe3b/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:52:24.706893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e69eb3-d9b2-4354-8528-efce89451a3d service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:24.719014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e69eb3-d9b2-4354-8528-efce89451a3d service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:24.790439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546e54b-524d-44a8-b75b-e3279eb4815c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:24.791129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546e54b-524d-44a8-b75b-e3279eb4815c admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:24.791658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546e54b-524d-44a8-b75b-e3279eb4815c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:24.791658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546e54b-524d-44a8-b75b-e3279eb4815c admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:24.792317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546e54b-524d-44a8-b75b-e3279eb4815c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:24.794056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546e54b-524d-44a8-b75b-e3279eb4815c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:24.811237 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1325/2584] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:52:24.816617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e570a53-c9bd-4ac4-9523-c94fde0f3028 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:24.830218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e570a53-c9bd-4ac4-9523-c94fde0f3028 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:24.837534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d777e-0987-47e0-8605-9edd2185dda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:24.838445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d777e-0987-47e0-8605-9edd2185dda9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:24.839181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d777e-0987-47e0-8605-9edd2185dda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:24.839321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d777e-0987-47e0-8605-9edd2185dda9 admin admin] PATH_INFO: `/v3/projects/9d8830defffa4b8dbacf6bbbd7ab4205/users/e31b54cd54d54f5ebfb064edac6dfe3b/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:24.841734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d777e-0987-47e0-8605-9edd2185dda9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d8830defffa4b8dbacf6bbbd7ab4205, user_id=e31b54cd54d54f5ebfb064edac6dfe3b, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:24.845063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d777e-0987-47e0-8605-9edd2185dda9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:24.858119 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1260/2585] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:52:24 2022] PUT /identity/v3/projects/9d8830defffa4b8dbacf6bbbd7ab4205/users/e31b54cd54d54f5ebfb064edac6dfe3b/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:24.870258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abc35a-cc7c-4fd9-94be-0eaca91d557f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:24.870258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abc35a-cc7c-4fd9-94be-0eaca91d557f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:24.870258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abc35a-cc7c-4fd9-94be-0eaca91d557f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:24.904028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-11abc35a-cc7c-4fd9-94be-0eaca91d557f None None] MFA Rules not processed for user `e31b54cd54d54f5ebfb064edac6dfe3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:24.905713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-11abc35a-cc7c-4fd9-94be-0eaca91d557f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:25.018906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d401b2a-78d1-4951-b6ec-2c338aba1337 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:25.019293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d401b2a-78d1-4951-b6ec-2c338aba1337 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:25.019650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d401b2a-78d1-4951-b6ec-2c338aba1337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:25.023963 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1326/2586] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:25.061162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8d401b2a-78d1-4951-b6ec-2c338aba1337 None None] MFA Rules not processed for user `3245b8304f604cfc97da73b4dc884bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:25.064192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8d401b2a-78d1-4951-b6ec-2c338aba1337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:25.072029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abc35a-cc7c-4fd9-94be-0eaca91d557f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:25.073662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-11abc35a-cc7c-4fd9-94be-0eaca91d557f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:25.159363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:25.172319 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1261/2587] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:25.189138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d401b2a-78d1-4951-b6ec-2c338aba1337 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:25.191757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8d401b2a-78d1-4951-b6ec-2c338aba1337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:25.192043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abc35a-cc7c-4fd9-94be-0eaca91d557f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:25.198269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c298aea-7112-444a-83f0-7ba5387875c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:25.199347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c298aea-7112-444a-83f0-7ba5387875c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:25.199347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c298aea-7112-444a-83f0-7ba5387875c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:25.199801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c298aea-7112-444a-83f0-7ba5387875c4 admin admin] PATH_INFO: `/v3/users/bba86a7a9dab49d191e553647250f279` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:25.201045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c298aea-7112-444a-83f0-7ba5387875c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bba86a7a9dab49d191e553647250f279)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:25.203112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c298aea-7112-444a-83f0-7ba5387875c4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:25.270563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7c298aea-7112-444a-83f0-7ba5387875c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bba86a7a9dab49d191e553647250f279'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:25.280635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7c298aea-7112-444a-83f0-7ba5387875c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bba86a7a9dab49d191e553647250f279'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:25.297243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7c298aea-7112-444a-83f0-7ba5387875c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bba86a7a9dab49d191e553647250f279'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:25.297467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d401b2a-78d1-4951-b6ec-2c338aba1337 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:25.298161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7c298aea-7112-444a-83f0-7ba5387875c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bba86a7a9dab49d191e553647250f279'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:25.305893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342267d5-5d53-4f96-ae6e-2645176b21a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:25.306899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342267d5-5d53-4f96-ae6e-2645176b21a6 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:25.307313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342267d5-5d53-4f96-ae6e-2645176b21a6 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:25.307688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342267d5-5d53-4f96-ae6e-2645176b21a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:25.309877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7c298aea-7112-444a-83f0-7ba5387875c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bba86a7a9dab49d191e553647250f279'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:25.310727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-342267d5-5d53-4f96-ae6e-2645176b21a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:25.319087 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1327/2588] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:52:25 2022] DELETE /identity/v3/users/bba86a7a9dab49d191e553647250f279 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:52:25.323744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d1bdd-76ab-4ef6-85b8-02698c7fa749 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:25.324119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d1bdd-76ab-4ef6-85b8-02698c7fa749 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:25.324472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d1bdd-76ab-4ef6-85b8-02698c7fa749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:25.325684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342267d5-5d53-4f96-ae6e-2645176b21a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:25.327867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342267d5-5d53-4f96-ae6e-2645176b21a6 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:25.352313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e81d1bdd-76ab-4ef6-85b8-02698c7fa749 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:25.354483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e81d1bdd-76ab-4ef6-85b8-02698c7fa749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:25.361076 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1262/2589] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:25.434562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342267d5-5d53-4f96-ae6e-2645176b21a6 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:25.437603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-342267d5-5d53-4f96-ae6e-2645176b21a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:25.450320 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1328/2590] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:52:25.461019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342267d5-5d53-4f96-ae6e-2645176b21a6 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:25.467184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1935e9-4c78-429d-8389-d312b8a78767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:25.467940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1935e9-4c78-429d-8389-d312b8a78767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:25.468341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1935e9-4c78-429d-8389-d312b8a78767 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:25.468710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1935e9-4c78-429d-8389-d312b8a78767 admin admin] PATH_INFO: `/v3/projects/baf5302894a8431f9cb39838a629f2ff` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:25.470673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1935e9-4c78-429d-8389-d312b8a78767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baf5302894a8431f9cb39838a629f2ff)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:25.472792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1935e9-4c78-429d-8389-d312b8a78767 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:25.476542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d1bdd-76ab-4ef6-85b8-02698c7fa749 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:25.478199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e81d1bdd-76ab-4ef6-85b8-02698c7fa749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:25.481112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:25.541486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9d1935e9-4c78-429d-8389-d312b8a78767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baf5302894a8431f9cb39838a629f2ff', 'request_id': 'req-9d1935e9-4c78-429d-8389-d312b8a78767'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:25.554039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9d1935e9-4c78-429d-8389-d312b8a78767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baf5302894a8431f9cb39838a629f2ff', 'request_id': 'req-9d1935e9-4c78-429d-8389-d312b8a78767'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:25.572705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9d1935e9-4c78-429d-8389-d312b8a78767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baf5302894a8431f9cb39838a629f2ff', 'request_id': 'req-9d1935e9-4c78-429d-8389-d312b8a78767'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:25.595252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9d1935e9-4c78-429d-8389-d312b8a78767 admin admin] The token cache is being invalidate because project baf5302894a8431f9cb39838a629f2ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:52:25.595842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9d1935e9-4c78-429d-8389-d312b8a78767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:25.597820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9d1935e9-4c78-429d-8389-d312b8a78767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:25.599735 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1263/2591] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:52:25 2022] DELETE /identity/v3/projects/baf5302894a8431f9cb39838a629f2ff => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:52:25.604310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1935e9-4c78-429d-8389-d312b8a78767 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:25.605754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9d1935e9-4c78-429d-8389-d312b8a78767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:25.631525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d1bdd-76ab-4ef6-85b8-02698c7fa749 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:25.670443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:25.721537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1935e9-4c78-429d-8389-d312b8a78767 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:25.724153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c793239-93a9-4cee-95c0-3fe53d4579e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:25.725610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c793239-93a9-4cee-95c0-3fe53d4579e0 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:25.726133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c793239-93a9-4cee-95c0-3fe53d4579e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:25.727258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c793239-93a9-4cee-95c0-3fe53d4579e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:25.729026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c46f42-3822-4329-8d2d-55434b82d710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:25.729849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c46f42-3822-4329-8d2d-55434b82d710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:25.730269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c46f42-3822-4329-8d2d-55434b82d710 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:25.730904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c46f42-3822-4329-8d2d-55434b82d710 admin admin] PATH_INFO: `/v3/users/06630999e423468abf0784062b11688a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:25.732003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1c793239-93a9-4cee-95c0-3fe53d4579e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:25.732444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c46f42-3822-4329-8d2d-55434b82d710 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06630999e423468abf0784062b11688a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:25.735245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c46f42-3822-4329-8d2d-55434b82d710 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:25.766718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c793239-93a9-4cee-95c0-3fe53d4579e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:25.773238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c793239-93a9-4cee-95c0-3fe53d4579e0 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:25.802812 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1329/2592] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:25.858272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d9c46f42-3822-4329-8d2d-55434b82d710 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06630999e423468abf0784062b11688a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:25.859798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d9c46f42-3822-4329-8d2d-55434b82d710 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06630999e423468abf0784062b11688a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:25.860609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d9c46f42-3822-4329-8d2d-55434b82d710 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06630999e423468abf0784062b11688a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:25.874930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d9c46f42-3822-4329-8d2d-55434b82d710 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06630999e423468abf0784062b11688a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:25.894733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d9c46f42-3822-4329-8d2d-55434b82d710 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06630999e423468abf0784062b11688a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:25.921567 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1264/2593] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:52:25 2022] DELETE /identity/v3/users/06630999e423468abf0784062b11688a => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:26.358365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c793239-93a9-4cee-95c0-3fe53d4579e0 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:26.359910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1c793239-93a9-4cee-95c0-3fe53d4579e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:26.381735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c793239-93a9-4cee-95c0-3fe53d4579e0 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:26.492090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb670c3b-ab18-4b7c-9a62-c89151a22f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:26.496166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb670c3b-ab18-4b7c-9a62-c89151a22f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:26.496953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb670c3b-ab18-4b7c-9a62-c89151a22f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:26.497739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb670c3b-ab18-4b7c-9a62-c89151a22f48 admin admin] PATH_INFO: `/v3/projects/4ae105e248b648f182d75d1eb8db51df` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:26.500642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb670c3b-ab18-4b7c-9a62-c89151a22f48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ae105e248b648f182d75d1eb8db51df)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:26.503753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb670c3b-ab18-4b7c-9a62-c89151a22f48 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:26.644153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c46f42-3822-4329-8d2d-55434b82d710 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:26.646376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d9c46f42-3822-4329-8d2d-55434b82d710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:26.667222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eb670c3b-ab18-4b7c-9a62-c89151a22f48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ae105e248b648f182d75d1eb8db51df', 'request_id': 'req-eb670c3b-ab18-4b7c-9a62-c89151a22f48'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:26.668212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c46f42-3822-4329-8d2d-55434b82d710 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:26.675464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eb670c3b-ab18-4b7c-9a62-c89151a22f48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ae105e248b648f182d75d1eb8db51df', 'request_id': 'req-eb670c3b-ab18-4b7c-9a62-c89151a22f48'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:26.682361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:26.684179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eb670c3b-ab18-4b7c-9a62-c89151a22f48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ae105e248b648f182d75d1eb8db51df', 'request_id': 'req-eb670c3b-ab18-4b7c-9a62-c89151a22f48'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:26.716189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eb670c3b-ab18-4b7c-9a62-c89151a22f48 admin admin] The token cache is being invalidate because project 4ae105e248b648f182d75d1eb8db51df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:52:26.716513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eb670c3b-ab18-4b7c-9a62-c89151a22f48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:26.717416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eb670c3b-ab18-4b7c-9a62-c89151a22f48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:26.724493 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1330/2594] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:52:26 2022] DELETE /identity/v3/projects/4ae105e248b648f182d75d1eb8db51df => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:26.731034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372bab2-724c-4cf9-a22e-35871846b795 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:26.731448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372bab2-724c-4cf9-a22e-35871846b795 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:26.732085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372bab2-724c-4cf9-a22e-35871846b795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:26.755766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74351640-96ba-4327-b1e4-39ea168cb210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:26.756424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74351640-96ba-4327-b1e4-39ea168cb210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:26.757191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74351640-96ba-4327-b1e4-39ea168cb210 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:26.757311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74351640-96ba-4327-b1e4-39ea168cb210 admin admin] PATH_INFO: `/v3/projects/e5882c021bbf42ee98b9e769a5cd1e92` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:26.758788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74351640-96ba-4327-b1e4-39ea168cb210 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5882c021bbf42ee98b9e769a5cd1e92)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:26.761880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74351640-96ba-4327-b1e4-39ea168cb210 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:26.806776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f372bab2-724c-4cf9-a22e-35871846b795 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:26.808864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f372bab2-724c-4cf9-a22e-35871846b795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:26.837656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-74351640-96ba-4327-b1e4-39ea168cb210 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5882c021bbf42ee98b9e769a5cd1e92', 'request_id': 'req-74351640-96ba-4327-b1e4-39ea168cb210'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:26.852378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-74351640-96ba-4327-b1e4-39ea168cb210 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5882c021bbf42ee98b9e769a5cd1e92', 'request_id': 'req-74351640-96ba-4327-b1e4-39ea168cb210'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:26.864297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-74351640-96ba-4327-b1e4-39ea168cb210 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5882c021bbf42ee98b9e769a5cd1e92', 'request_id': 'req-74351640-96ba-4327-b1e4-39ea168cb210'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:26.919208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:26.934557 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1331/2595] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:52:26.938104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-74351640-96ba-4327-b1e4-39ea168cb210 admin admin] The token cache is being invalidate because project e5882c021bbf42ee98b9e769a5cd1e92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:52:26.938433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-74351640-96ba-4327-b1e4-39ea168cb210 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:26.941935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00323cb-9999-4991-8e07-68a26250feb5 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:26.942618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00323cb-9999-4991-8e07-68a26250feb5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:26.943040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00323cb-9999-4991-8e07-68a26250feb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:26.944773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-74351640-96ba-4327-b1e4-39ea168cb210 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:26.956764 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1265/2596] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:52:26 2022] DELETE /identity/v3/projects/e5882c021bbf42ee98b9e769a5cd1e92 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:26.958375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe4d03-7856-4484-8e73-c71ebd357cc9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:26.958659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe4d03-7856-4484-8e73-c71ebd357cc9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:26.958941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe4d03-7856-4484-8e73-c71ebd357cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:26.968342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f00323cb-9999-4991-8e07-68a26250feb5 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:26.969957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f00323cb-9999-4991-8e07-68a26250feb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:26.995932 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1332/2597] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:52:26.999867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00323cb-9999-4991-8e07-68a26250feb5 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:27.015437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00323cb-9999-4991-8e07-68a26250feb5 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:27.021976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d488f-c882-4c3a-b9c6-3a1e212d9935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:27.023039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d488f-c882-4c3a-b9c6-3a1e212d9935 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:27.023562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d488f-c882-4c3a-b9c6-3a1e212d9935 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:27.024051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d488f-c882-4c3a-b9c6-3a1e212d9935 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:27.025105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d488f-c882-4c3a-b9c6-3a1e212d9935 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:52:27.025644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d488f-c882-4c3a-b9c6-3a1e212d9935 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:27.027534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d488f-c882-4c3a-b9c6-3a1e212d9935 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:27.033413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ffbe4d03-7856-4484-8e73-c71ebd357cc9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:27.035414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ffbe4d03-7856-4484-8e73-c71ebd357cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:27.044379 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1333/2598] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:52:27.048562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d488f-c882-4c3a-b9c6-3a1e212d9935 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:27.066359 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1266/2599] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:27.070163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d488f-c882-4c3a-b9c6-3a1e212d9935 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:27.073910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6de11-9c27-45cc-a709-9942248fba7b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:27.074235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6de11-9c27-45cc-a709-9942248fba7b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:27.074451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6de11-9c27-45cc-a709-9942248fba7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:27.078123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9696359-e2ef-46f1-8294-908f8a6e858b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:27.078857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9696359-e2ef-46f1-8294-908f8a6e858b admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:27.079236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9696359-e2ef-46f1-8294-908f8a6e858b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:27.079607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9696359-e2ef-46f1-8294-908f8a6e858b admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:27.080837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9696359-e2ef-46f1-8294-908f8a6e858b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:27.083962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9696359-e2ef-46f1-8294-908f8a6e858b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:27.106151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fcf6de11-9c27-45cc-a709-9942248fba7b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:27.107563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fcf6de11-9c27-45cc-a709-9942248fba7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:27.117451 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1267/2600] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:27.122347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6de11-9c27-45cc-a709-9942248fba7b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:27.133391 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1334/2601] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:52:27 2022] POST /identity/v3/projects => generated 379 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:52:27.138013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9696359-e2ef-46f1-8294-908f8a6e858b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:27.139005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6de11-9c27-45cc-a709-9942248fba7b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:27.239278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9696359-e2ef-46f1-8294-908f8a6e858b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:27.246808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28ab4a-0bd5-41e2-b4b5-d8cbf5a88a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:27.247485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28ab4a-0bd5-41e2-b4b5-d8cbf5a88a4b admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:27.247776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28ab4a-0bd5-41e2-b4b5-d8cbf5a88a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:27.248056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28ab4a-0bd5-41e2-b4b5-d8cbf5a88a4b admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:27.248760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28ab4a-0bd5-41e2-b4b5-d8cbf5a88a4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:27.252557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28ab4a-0bd5-41e2-b4b5-d8cbf5a88a4b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:27.268331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2f921-1b2f-45ba-b53b-2ad2754f06f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:27.268965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2f921-1b2f-45ba-b53b-2ad2754f06f4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:27.272114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2f921-1b2f-45ba-b53b-2ad2754f06f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:27.272401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2f921-1b2f-45ba-b53b-2ad2754f06f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:27.273239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2f921-1b2f-45ba-b53b-2ad2754f06f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:52:27.273551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2f921-1b2f-45ba-b53b-2ad2754f06f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:27.278678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2f921-1b2f-45ba-b53b-2ad2754f06f4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:27.282526 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1335/2602] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:52:27 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:27.284917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28ab4a-0bd5-41e2-b4b5-d8cbf5a88a4b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:27.298130 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1268/2603] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:52:27.300584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2f921-1b2f-45ba-b53b-2ad2754f06f4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:27.302576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28ab4a-0bd5-41e2-b4b5-d8cbf5a88a4b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:27.319832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635e63b-8739-4b78-8020-538ca354ec9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:27.320459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635e63b-8739-4b78-8020-538ca354ec9d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:27.320740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635e63b-8739-4b78-8020-538ca354ec9d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:27.321014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635e63b-8739-4b78-8020-538ca354ec9d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:27.321664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635e63b-8739-4b78-8020-538ca354ec9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:27.323279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635e63b-8739-4b78-8020-538ca354ec9d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:27.328633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2f921-1b2f-45ba-b53b-2ad2754f06f4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:27.340913 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1336/2604] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:52:27.342371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9db4f-1ed3-408e-af2e-bec3afc88733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:27.343123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9db4f-1ed3-408e-af2e-bec3afc88733 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:27.343472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9db4f-1ed3-408e-af2e-bec3afc88733 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:27.343844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9db4f-1ed3-408e-af2e-bec3afc88733 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:27.344509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9db4f-1ed3-408e-af2e-bec3afc88733 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:27.344730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635e63b-8739-4b78-8020-538ca354ec9d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:27.348924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9db4f-1ed3-408e-af2e-bec3afc88733 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:27.355299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635e63b-8739-4b78-8020-538ca354ec9d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:27.360284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0b950-0dbc-4107-b51e-da6618374c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:27.360882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0b950-0dbc-4107-b51e-da6618374c23 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:27.361158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0b950-0dbc-4107-b51e-da6618374c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:27.361483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0b950-0dbc-4107-b51e-da6618374c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:27.362169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0b950-0dbc-4107-b51e-da6618374c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:27.363973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0b950-0dbc-4107-b51e-da6618374c23 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:27.375471 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1337/2605] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:52:27.380284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0b950-0dbc-4107-b51e-da6618374c23 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:27.394317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0b950-0dbc-4107-b51e-da6618374c23 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:27.398085 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1269/2606] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:52:27 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:27.402388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9db4f-1ed3-408e-af2e-bec3afc88733 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:27.407769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d1a350-81d9-4ce7-af95-a8f92a757d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:27.410346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1a350-81d9-4ce7-af95-a8f92a757d0c admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:27.410775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1a350-81d9-4ce7-af95-a8f92a757d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:27.411133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1a350-81d9-4ce7-af95-a8f92a757d0c admin admin] PATH_INFO: `/v3/projects/ec1c1a8f4f8745388760db1428e4c1b5/users/b3cecdbd13d74f4397e0f3f34b9d64f0/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:27.415883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:27.441923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d1a350-81d9-4ce7-af95-a8f92a757d0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec1c1a8f4f8745388760db1428e4c1b5, user_id=b3cecdbd13d74f4397e0f3f34b9d64f0, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:27.446415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d1a350-81d9-4ce7-af95-a8f92a757d0c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:27.473073 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1338/2607] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:52:27 2022] PUT /identity/v3/projects/ec1c1a8f4f8745388760db1428e4c1b5/users/b3cecdbd13d74f4397e0f3f34b9d64f0/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:52:27.485240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8ef5f-d995-4e4e-b89c-068c7ba07860 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:27.485527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8ef5f-d995-4e4e-b89c-068c7ba07860 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:27.485796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8ef5f-d995-4e4e-b89c-068c7ba07860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:27.549307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-04a8ef5f-d995-4e4e-b89c-068c7ba07860 None None] MFA Rules not processed for user `b3cecdbd13d74f4397e0f3f34b9d64f0`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:27.552710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-04a8ef5f-d995-4e4e-b89c-068c7ba07860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:27.575300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9db4f-1ed3-408e-af2e-bec3afc88733 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:27.655454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:27.688515 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1339/2608] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:52:27.702533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfb169-bbd1-4e9d-933a-c3b1ffafd0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:27.703672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfb169-bbd1-4e9d-933a-c3b1ffafd0aa admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:27.703952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfb169-bbd1-4e9d-933a-c3b1ffafd0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:27.704122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d3385-74c2-42a1-aefb-40d833669b23 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:27.704400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfb169-bbd1-4e9d-933a-c3b1ffafd0aa admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:27.705127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bfb169-bbd1-4e9d-933a-c3b1ffafd0aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:27.706375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d3385-74c2-42a1-aefb-40d833669b23 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:27.706901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d3385-74c2-42a1-aefb-40d833669b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:27.708745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bfb169-bbd1-4e9d-933a-c3b1ffafd0aa admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:27.742984 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1270/2609] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:52:27 2022] POST /identity/v3/users => generated 287 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:27.744012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfb169-bbd1-4e9d-933a-c3b1ffafd0aa admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:27.759489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ac7d3385-74c2-42a1-aefb-40d833669b23 None None] MFA Rules not processed for user `b3cecdbd13d74f4397e0f3f34b9d64f0`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:27.760310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfb169-bbd1-4e9d-933a-c3b1ffafd0aa admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:27.760821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ac7d3385-74c2-42a1-aefb-40d833669b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:27.772310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9ff8c-35c9-4644-84c4-e1e700377e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:27.774011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9ff8c-35c9-4644-84c4-e1e700377e20 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:27.774293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9ff8c-35c9-4644-84c4-e1e700377e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:27.774584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9ff8c-35c9-4644-84c4-e1e700377e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:27.776384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9ff8c-35c9-4644-84c4-e1e700377e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:27.782638 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1340/2610] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:27.787439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9ff8c-35c9-4644-84c4-e1e700377e20 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:27.814290 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1271/2611] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:52:27.821519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d3385-74c2-42a1-aefb-40d833669b23 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:27.842453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d3385-74c2-42a1-aefb-40d833669b23 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:27.856394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb1e41-c1b8-4145-b0a3-031286335389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:27.856721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb1e41-c1b8-4145-b0a3-031286335389 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:27.857005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb1e41-c1b8-4145-b0a3-031286335389 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:27.857322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb1e41-c1b8-4145-b0a3-031286335389 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:27.858032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb1e41-c1b8-4145-b0a3-031286335389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:27.860770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb1e41-c1b8-4145-b0a3-031286335389 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:27.881268 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1341/2612] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:52:27.885668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9ff8c-35c9-4644-84c4-e1e700377e20 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:27.901496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9ff8c-35c9-4644-84c4-e1e700377e20 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:27.915520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e491db-652b-477d-a9e8-abfb272e69fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:27.916409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e491db-652b-477d-a9e8-abfb272e69fb admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:27.916583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e491db-652b-477d-a9e8-abfb272e69fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:27.916945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e491db-652b-477d-a9e8-abfb272e69fb admin admin] PATH_INFO: `/v3/projects/31902a3ed806467fadc434c0a5f73642/users/22090f1819eb4d5eaba6391cdddbf6c2/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:27.956140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e491db-652b-477d-a9e8-abfb272e69fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=31902a3ed806467fadc434c0a5f73642, user_id=22090f1819eb4d5eaba6391cdddbf6c2, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:27.960258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e491db-652b-477d-a9e8-abfb272e69fb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:27.989115 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1272/2613] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:52:27 2022] PUT /identity/v3/projects/31902a3ed806467fadc434c0a5f73642/users/22090f1819eb4d5eaba6391cdddbf6c2/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:28.001691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2acbc9-ef47-4af4-a7da-6f48212ff27c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:28.002461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2acbc9-ef47-4af4-a7da-6f48212ff27c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:28.002770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2acbc9-ef47-4af4-a7da-6f48212ff27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:28.044098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9e2acbc9-ef47-4af4-a7da-6f48212ff27c None None] MFA Rules not processed for user `22090f1819eb4d5eaba6391cdddbf6c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:28.046113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9e2acbc9-ef47-4af4-a7da-6f48212ff27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:28.197733 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1342/2614] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:28.198997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e491db-652b-477d-a9e8-abfb272e69fb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:28.308011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2acbc9-ef47-4af4-a7da-6f48212ff27c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:28.309737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9e2acbc9-ef47-4af4-a7da-6f48212ff27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:28.411435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e491db-652b-477d-a9e8-abfb272e69fb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:28.421430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd926d2-d14a-40e5-a1c3-5d693817ec5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:28.422115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd926d2-d14a-40e5-a1c3-5d693817ec5f admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:28.422435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd926d2-d14a-40e5-a1c3-5d693817ec5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:28.422766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd926d2-d14a-40e5-a1c3-5d693817ec5f admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:28.423479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd926d2-d14a-40e5-a1c3-5d693817ec5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:28.425522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd926d2-d14a-40e5-a1c3-5d693817ec5f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:28.439426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:28.463048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2acbc9-ef47-4af4-a7da-6f48212ff27c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:28.472964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798db863-3522-4875-af4c-229d2e07d31d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:28.473816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798db863-3522-4875-af4c-229d2e07d31d service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:28.474176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798db863-3522-4875-af4c-229d2e07d31d service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:28.475692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798db863-3522-4875-af4c-229d2e07d31d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:28.490744 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1273/2615] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:52:28 2022] POST /identity/v3/projects => generated 371 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:28.491642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd926d2-d14a-40e5-a1c3-5d693817ec5f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:28.607239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd926d2-d14a-40e5-a1c3-5d693817ec5f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:28.612671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d1186-b60a-4bdc-8d5e-bb5097e54f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:28.613912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d1186-b60a-4bdc-8d5e-bb5097e54f09 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:28.614463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d1186-b60a-4bdc-8d5e-bb5097e54f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:28.614984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d1186-b60a-4bdc-8d5e-bb5097e54f09 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:28.615827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d1186-b60a-4bdc-8d5e-bb5097e54f09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:28.617993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d1186-b60a-4bdc-8d5e-bb5097e54f09 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:28.651877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798db863-3522-4875-af4c-229d2e07d31d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:28.655138 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1274/2616] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:52:28 2022] POST /identity/v3/users => generated 292 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:28.655964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798db863-3522-4875-af4c-229d2e07d31d service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:28.660489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d1186-b60a-4bdc-8d5e-bb5097e54f09 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:28.663464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:28.690143 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1343/2617] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:52:28.691108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d1186-b60a-4bdc-8d5e-bb5097e54f09 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:28.707033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a780fb3-28a1-4485-bbe3-61e0250967a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:28.707786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a780fb3-28a1-4485-bbe3-61e0250967a0 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:28.708024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a780fb3-28a1-4485-bbe3-61e0250967a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:28.708307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a780fb3-28a1-4485-bbe3-61e0250967a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:28.708966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a780fb3-28a1-4485-bbe3-61e0250967a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:28.710861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a780fb3-28a1-4485-bbe3-61e0250967a0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:28.736910 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1275/2618] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:52:28.741236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798db863-3522-4875-af4c-229d2e07d31d service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:28.764406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798db863-3522-4875-af4c-229d2e07d31d service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:28.783415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e9d6f-4b2d-4d20-b201-4716ff4a84a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:28.785642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e9d6f-4b2d-4d20-b201-4716ff4a84a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:28.785931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e9d6f-4b2d-4d20-b201-4716ff4a84a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:28.786212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e9d6f-4b2d-4d20-b201-4716ff4a84a2 admin admin] PATH_INFO: `/v3/projects/98b4a953d9054ef79793059b903f240d/users/2902035de7b74b018dfffb4d8ce99dcd/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:28.825739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e9d6f-4b2d-4d20-b201-4716ff4a84a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b4a953d9054ef79793059b903f240d, user_id=2902035de7b74b018dfffb4d8ce99dcd, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:28.829335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e9d6f-4b2d-4d20-b201-4716ff4a84a2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:28.855526 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1344/2619] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:52:28 2022] PUT /identity/v3/projects/98b4a953d9054ef79793059b903f240d/users/2902035de7b74b018dfffb4d8ce99dcd/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:52:28.859923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a780fb3-28a1-4485-bbe3-61e0250967a0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:28.875191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a780fb3-28a1-4485-bbe3-61e0250967a0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:29.014684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f51fc-f9e6-4f93-8715-93efe432a2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:29.016194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f51fc-f9e6-4f93-8715-93efe432a2a4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:29.016484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f51fc-f9e6-4f93-8715-93efe432a2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:29.016760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f51fc-f9e6-4f93-8715-93efe432a2a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:29.018750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f51fc-f9e6-4f93-8715-93efe432a2a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:29.024824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f51fc-f9e6-4f93-8715-93efe432a2a4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:29.038169 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1276/2620] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:52:29.047905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e9d6f-4b2d-4d20-b201-4716ff4a84a2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:29.065874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e9d6f-4b2d-4d20-b201-4716ff4a84a2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:29.072753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04860d08-f409-4f80-97d2-95170458f748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:29.073562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04860d08-f409-4f80-97d2-95170458f748 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:29.073912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04860d08-f409-4f80-97d2-95170458f748 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:29.074315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04860d08-f409-4f80-97d2-95170458f748 admin admin] PATH_INFO: `/v3/projects/98b4a953d9054ef79793059b903f240d/users/2902035de7b74b018dfffb4d8ce99dcd/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:29.076751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04860d08-f409-4f80-97d2-95170458f748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b4a953d9054ef79793059b903f240d, user_id=2902035de7b74b018dfffb4d8ce99dcd, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:29.080061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04860d08-f409-4f80-97d2-95170458f748 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:29.094567 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1345/2621] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:52:29 2022] PUT /identity/v3/projects/98b4a953d9054ef79793059b903f240d/users/2902035de7b74b018dfffb4d8ce99dcd/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:29.109684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcdf49-18a8-4b1e-8d39-da7f275705b2 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:29.110560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcdf49-18a8-4b1e-8d39-da7f275705b2 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:29.110851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcdf49-18a8-4b1e-8d39-da7f275705b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:29.192658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6abcdf49-18a8-4b1e-8d39-da7f275705b2 None None] MFA Rules not processed for user `2902035de7b74b018dfffb4d8ce99dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:29.194025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6abcdf49-18a8-4b1e-8d39-da7f275705b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:29.307062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:29.365774 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1277/2622] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:29 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:29.946881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04860d08-f409-4f80-97d2-95170458f748 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:29.952631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-04860d08-f409-4f80-97d2-95170458f748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:30.179132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04860d08-f409-4f80-97d2-95170458f748 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:30.190610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f6ef7-0d78-41e5-b4f6-73e2146ba069 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:30.191682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f6ef7-0d78-41e5-b4f6-73e2146ba069 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:30.191971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f6ef7-0d78-41e5-b4f6-73e2146ba069 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:30.192251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f6ef7-0d78-41e5-b4f6-73e2146ba069 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:30.207113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f6ef7-0d78-41e5-b4f6-73e2146ba069 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:30.208806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f6ef7-0d78-41e5-b4f6-73e2146ba069 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:30.222571 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1346/2623] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:30.834832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcdf49-18a8-4b1e-8d39-da7f275705b2 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:30.849983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcdf49-18a8-4b1e-8d39-da7f275705b2 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:30.856178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55ae34-6189-4157-8361-63de10020a7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:30.856787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55ae34-6189-4157-8361-63de10020a7c service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:30.857063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55ae34-6189-4157-8361-63de10020a7c service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:30.857414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55ae34-6189-4157-8361-63de10020a7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:30.919307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55ae34-6189-4157-8361-63de10020a7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:30.920931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55ae34-6189-4157-8361-63de10020a7c service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:30.931120 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1278/2624] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:32.433070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f6ef7-0d78-41e5-b4f6-73e2146ba069 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:32.590758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f6ef7-0d78-41e5-b4f6-73e2146ba069 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:32.603440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4d44f-0920-45c6-9b12-976e2f48d3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:32.604312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4d44f-0920-45c6-9b12-976e2f48d3f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:32.604772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4d44f-0920-45c6-9b12-976e2f48d3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:32.605199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4d44f-0920-45c6-9b12-976e2f48d3f2 admin admin] PATH_INFO: `/v3/users/22090f1819eb4d5eaba6391cdddbf6c2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:32.611230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c4d44f-0920-45c6-9b12-976e2f48d3f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22090f1819eb4d5eaba6391cdddbf6c2)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:32.613605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c4d44f-0920-45c6-9b12-976e2f48d3f2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:32.741203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-30c4d44f-0920-45c6-9b12-976e2f48d3f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22090f1819eb4d5eaba6391cdddbf6c2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:32.750244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:32.765625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-30c4d44f-0920-45c6-9b12-976e2f48d3f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22090f1819eb4d5eaba6391cdddbf6c2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:32.786178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-30c4d44f-0920-45c6-9b12-976e2f48d3f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22090f1819eb4d5eaba6391cdddbf6c2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:32.786863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-30c4d44f-0920-45c6-9b12-976e2f48d3f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22090f1819eb4d5eaba6391cdddbf6c2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:32.797879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-30c4d44f-0920-45c6-9b12-976e2f48d3f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22090f1819eb4d5eaba6391cdddbf6c2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:32.813430 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1347/2625] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:52:32 2022] DELETE /identity/v3/users/22090f1819eb4d5eaba6391cdddbf6c2 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:32.816351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7e415-913b-46cf-b957-be916b3407e0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:32.818830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7e415-913b-46cf-b957-be916b3407e0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:32.819317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7e415-913b-46cf-b957-be916b3407e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:32.853796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bde7e415-913b-46cf-b957-be916b3407e0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:32.855562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bde7e415-913b-46cf-b957-be916b3407e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:32.934694 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1279/2626] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:32.945708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4d44f-0920-45c6-9b12-976e2f48d3f2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:32.947190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-30c4d44f-0920-45c6-9b12-976e2f48d3f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:33.066416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4d44f-0920-45c6-9b12-976e2f48d3f2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:33.072581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03709e81-c032-4e63-8b74-aea11c6531d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:33.073205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03709e81-c032-4e63-8b74-aea11c6531d0 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:33.074457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03709e81-c032-4e63-8b74-aea11c6531d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:33.074979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03709e81-c032-4e63-8b74-aea11c6531d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:33.092369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03709e81-c032-4e63-8b74-aea11c6531d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:33.094147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03709e81-c032-4e63-8b74-aea11c6531d0 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:33.106684 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1348/2627] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:52:34.012286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7e415-913b-46cf-b957-be916b3407e0 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:34.013761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bde7e415-913b-46cf-b957-be916b3407e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:34.036619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7e415-913b-46cf-b957-be916b3407e0 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:34.050743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f974b8-1e42-4f78-ad52-83fb688d2ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:34.051593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f974b8-1e42-4f78-ad52-83fb688d2ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:34.052033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f974b8-1e42-4f78-ad52-83fb688d2ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:34.052862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f974b8-1e42-4f78-ad52-83fb688d2ae9 admin admin] PATH_INFO: `/v3/projects/31902a3ed806467fadc434c0a5f73642` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:34.054301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f974b8-1e42-4f78-ad52-83fb688d2ae9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31902a3ed806467fadc434c0a5f73642)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:34.057088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f974b8-1e42-4f78-ad52-83fb688d2ae9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:34.122120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95f974b8-1e42-4f78-ad52-83fb688d2ae9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31902a3ed806467fadc434c0a5f73642', 'request_id': 'req-95f974b8-1e42-4f78-ad52-83fb688d2ae9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:34.142309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95f974b8-1e42-4f78-ad52-83fb688d2ae9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31902a3ed806467fadc434c0a5f73642', 'request_id': 'req-95f974b8-1e42-4f78-ad52-83fb688d2ae9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:34.161401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95f974b8-1e42-4f78-ad52-83fb688d2ae9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31902a3ed806467fadc434c0a5f73642', 'request_id': 'req-95f974b8-1e42-4f78-ad52-83fb688d2ae9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:34.178193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95f974b8-1e42-4f78-ad52-83fb688d2ae9 admin admin] The token cache is being invalidate because project 31902a3ed806467fadc434c0a5f73642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:52:34.178801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95f974b8-1e42-4f78-ad52-83fb688d2ae9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:34.179651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95f974b8-1e42-4f78-ad52-83fb688d2ae9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:34.181859 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1280/2628] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:52:34 2022] DELETE /identity/v3/projects/31902a3ed806467fadc434c0a5f73642 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:34.185713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03709e81-c032-4e63-8b74-aea11c6531d0 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:34.198834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03709e81-c032-4e63-8b74-aea11c6531d0 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:34.276772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf9361-bc1c-44d1-a646-9064c906403f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:34.277408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf9361-bc1c-44d1-a646-9064c906403f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:34.277969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf9361-bc1c-44d1-a646-9064c906403f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:34.278136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf9361-bc1c-44d1-a646-9064c906403f admin admin] PATH_INFO: `/v3/users/2902035de7b74b018dfffb4d8ce99dcd` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:34.279513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf9361-bc1c-44d1-a646-9064c906403f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2902035de7b74b018dfffb4d8ce99dcd)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:34.281424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf9361-bc1c-44d1-a646-9064c906403f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:34.353590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-27cf9361-bc1c-44d1-a646-9064c906403f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2902035de7b74b018dfffb4d8ce99dcd'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:34.363448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-27cf9361-bc1c-44d1-a646-9064c906403f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2902035de7b74b018dfffb4d8ce99dcd'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:34.407118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:34.415608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-27cf9361-bc1c-44d1-a646-9064c906403f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2902035de7b74b018dfffb4d8ce99dcd'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:34.420110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-27cf9361-bc1c-44d1-a646-9064c906403f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2902035de7b74b018dfffb4d8ce99dcd'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:34.439142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-27cf9361-bc1c-44d1-a646-9064c906403f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2902035de7b74b018dfffb4d8ce99dcd'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:34.452040 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1349/2629] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:52:34 2022] DELETE /identity/v3/users/2902035de7b74b018dfffb4d8ce99dcd => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:34.907666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f974b8-1e42-4f78-ad52-83fb688d2ae9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:34.909894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-95f974b8-1e42-4f78-ad52-83fb688d2ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:34.924619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f974b8-1e42-4f78-ad52-83fb688d2ae9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:34.933507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:35.015854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5ec6b-9d01-44d0-9b75-9a1719f9ae71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:35.016477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5ec6b-9d01-44d0-9b75-9a1719f9ae71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:35.016767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5ec6b-9d01-44d0-9b75-9a1719f9ae71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:35.017047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5ec6b-9d01-44d0-9b75-9a1719f9ae71 admin admin] PATH_INFO: `/v3/projects/98b4a953d9054ef79793059b903f240d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:35.018224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5ec6b-9d01-44d0-9b75-9a1719f9ae71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98b4a953d9054ef79793059b903f240d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:35.020408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5ec6b-9d01-44d0-9b75-9a1719f9ae71 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:35.110003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-efd5ec6b-9d01-44d0-9b75-9a1719f9ae71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98b4a953d9054ef79793059b903f240d', 'request_id': 'req-efd5ec6b-9d01-44d0-9b75-9a1719f9ae71'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:35.118867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-efd5ec6b-9d01-44d0-9b75-9a1719f9ae71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98b4a953d9054ef79793059b903f240d', 'request_id': 'req-efd5ec6b-9d01-44d0-9b75-9a1719f9ae71'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:35.125319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-efd5ec6b-9d01-44d0-9b75-9a1719f9ae71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98b4a953d9054ef79793059b903f240d', 'request_id': 'req-efd5ec6b-9d01-44d0-9b75-9a1719f9ae71'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:35.141157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-efd5ec6b-9d01-44d0-9b75-9a1719f9ae71 admin admin] The token cache is being invalidate because project 98b4a953d9054ef79793059b903f240d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:52:35.141712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-efd5ec6b-9d01-44d0-9b75-9a1719f9ae71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:35.143129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-efd5ec6b-9d01-44d0-9b75-9a1719f9ae71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:35.150406 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1281/2630] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:52:34 2022] DELETE /identity/v3/projects/98b4a953d9054ef79793059b903f240d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:35.151251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1bc73-732b-4642-8fe6-532d9344cca5 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:35.151616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1bc73-732b-4642-8fe6-532d9344cca5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:35.152127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1bc73-732b-4642-8fe6-532d9344cca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:35.175994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ecb1bc73-732b-4642-8fe6-532d9344cca5 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:35.177334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ecb1bc73-732b-4642-8fe6-532d9344cca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:35.272519 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1350/2631] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:35.274635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f879f-11e0-4fb3-b696-088dc4adf034 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:35.274767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f879f-11e0-4fb3-b696-088dc4adf034 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:35.274977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f879f-11e0-4fb3-b696-088dc4adf034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:35.311561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1a8f879f-11e0-4fb3-b696-088dc4adf034 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:35.313689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1a8f879f-11e0-4fb3-b696-088dc4adf034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:35.333553 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1282/2632] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:35.333989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1bc73-732b-4642-8fe6-532d9344cca5 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:35.344608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1bc73-732b-4642-8fe6-532d9344cca5 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:35.353237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82642d-6ec4-4dbb-9b08-d7346aee1906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:35.353908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82642d-6ec4-4dbb-9b08-d7346aee1906 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:35.354183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82642d-6ec4-4dbb-9b08-d7346aee1906 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:35.354460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82642d-6ec4-4dbb-9b08-d7346aee1906 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:35.355153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82642d-6ec4-4dbb-9b08-d7346aee1906 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:52:35.355453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82642d-6ec4-4dbb-9b08-d7346aee1906 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:35.357219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82642d-6ec4-4dbb-9b08-d7346aee1906 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:35.381969 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1351/2633] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:52:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:52:35.382141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f879f-11e0-4fb3-b696-088dc4adf034 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:35.392859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f879f-11e0-4fb3-b696-088dc4adf034 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:35.405718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e70d81-0abb-4617-bfcd-4304b4035024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:35.406832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e70d81-0abb-4617-bfcd-4304b4035024 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:35.407330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e70d81-0abb-4617-bfcd-4304b4035024 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:35.407992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e70d81-0abb-4617-bfcd-4304b4035024 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:35.409178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e70d81-0abb-4617-bfcd-4304b4035024 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:35.411451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e70d81-0abb-4617-bfcd-4304b4035024 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:35.448061 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1283/2634] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:52:35 2022] POST /identity/v3/projects => generated 409 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:35.453017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82642d-6ec4-4dbb-9b08-d7346aee1906 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:35.463568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82642d-6ec4-4dbb-9b08-d7346aee1906 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:35.528078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d2ef5-49c7-41fc-8ddd-e88eed814709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:35.528803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d2ef5-49c7-41fc-8ddd-e88eed814709 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:35.529213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d2ef5-49c7-41fc-8ddd-e88eed814709 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:35.529557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d2ef5-49c7-41fc-8ddd-e88eed814709 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:35.535333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d2ef5-49c7-41fc-8ddd-e88eed814709 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:35.537609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d2ef5-49c7-41fc-8ddd-e88eed814709 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:35.565152 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1352/2635] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:52:35 2022] POST /identity/v3/users => generated 305 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:35.572841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e70d81-0abb-4617-bfcd-4304b4035024 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:35.585785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e70d81-0abb-4617-bfcd-4304b4035024 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:35.600841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb452ad-e1c7-4be4-80d2-3fc4a4ca85be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:35.601634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb452ad-e1c7-4be4-80d2-3fc4a4ca85be admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:35.601951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb452ad-e1c7-4be4-80d2-3fc4a4ca85be admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:35.602629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb452ad-e1c7-4be4-80d2-3fc4a4ca85be admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:35.603410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb452ad-e1c7-4be4-80d2-3fc4a4ca85be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:35.606456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb452ad-e1c7-4be4-80d2-3fc4a4ca85be admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:35.622298 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1284/2636] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:52:35.626105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d2ef5-49c7-41fc-8ddd-e88eed814709 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:35.640188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d2ef5-49c7-41fc-8ddd-e88eed814709 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:35.646088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e36db1-c7a3-4ba0-8ce4-b1fec500c6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:35.646764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e36db1-c7a3-4ba0-8ce4-b1fec500c6c5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:35.647049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e36db1-c7a3-4ba0-8ce4-b1fec500c6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:35.647331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e36db1-c7a3-4ba0-8ce4-b1fec500c6c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:35.647987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e36db1-c7a3-4ba0-8ce4-b1fec500c6c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:35.649669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e36db1-c7a3-4ba0-8ce4-b1fec500c6c5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:35.662735 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1353/2637] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:52:35.667612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb452ad-e1c7-4be4-80d2-3fc4a4ca85be admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:35.682043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb452ad-e1c7-4be4-80d2-3fc4a4ca85be admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:35.687963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ad46b-3c23-4bfa-bf34-9d4ee11d7a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:35.688652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ad46b-3c23-4bfa-bf34-9d4ee11d7a31 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:35.688934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ad46b-3c23-4bfa-bf34-9d4ee11d7a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:35.689207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ad46b-3c23-4bfa-bf34-9d4ee11d7a31 admin admin] PATH_INFO: `/v3/projects/65ae697418f542acb416e10094c204c3/users/bb9e693c893b44aeafd7aee7cf30613d/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:35.706420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ad46b-3c23-4bfa-bf34-9d4ee11d7a31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65ae697418f542acb416e10094c204c3, user_id=bb9e693c893b44aeafd7aee7cf30613d, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:35.709113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ad46b-3c23-4bfa-bf34-9d4ee11d7a31 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:35.723478 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1285/2638] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:52:35 2022] PUT /identity/v3/projects/65ae697418f542acb416e10094c204c3/users/bb9e693c893b44aeafd7aee7cf30613d/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:35.733785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dcb38-623d-4ccd-97fa-839f14160ccc None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:35.734186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dcb38-623d-4ccd-97fa-839f14160ccc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:35.734562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dcb38-623d-4ccd-97fa-839f14160ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:35.765682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-707dcb38-623d-4ccd-97fa-839f14160ccc None None] MFA Rules not processed for user `bb9e693c893b44aeafd7aee7cf30613d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:35.766964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-707dcb38-623d-4ccd-97fa-839f14160ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:35.813704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:35.844953 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1354/2639] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:35 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:35.849971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ad46b-3c23-4bfa-bf34-9d4ee11d7a31 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:35.894290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:35.918163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ad46b-3c23-4bfa-bf34-9d4ee11d7a31 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:35.923017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd91134-4b98-41c1-bf59-7a0b3573d89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:35.923666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd91134-4b98-41c1-bf59-7a0b3573d89d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:35.923978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd91134-4b98-41c1-bf59-7a0b3573d89d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:35.924323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd91134-4b98-41c1-bf59-7a0b3573d89d admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:35.924958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd91134-4b98-41c1-bf59-7a0b3573d89d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:35.927049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd91134-4b98-41c1-bf59-7a0b3573d89d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:35.960737 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1286/2640] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:52:35 2022] POST /identity/v3/projects => generated 407 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:35.962116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dcb38-623d-4ccd-97fa-839f14160ccc None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:35.979336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dcb38-623d-4ccd-97fa-839f14160ccc None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:36.055792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fde6f3-d862-40dd-b7e6-12d559ba6f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:36.058626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fde6f3-d862-40dd-b7e6-12d559ba6f79 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:36.058626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fde6f3-d862-40dd-b7e6-12d559ba6f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:36.058626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fde6f3-d862-40dd-b7e6-12d559ba6f79 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:36.058626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fde6f3-d862-40dd-b7e6-12d559ba6f79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:36.059892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fde6f3-d862-40dd-b7e6-12d559ba6f79 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:36.092151 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1355/2641] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:52:35 2022] POST /identity/v3/users => generated 310 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:36.097079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd91134-4b98-41c1-bf59-7a0b3573d89d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:36.117211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd91134-4b98-41c1-bf59-7a0b3573d89d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:36.121112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf664da6-70d3-4bf7-bc71-abfe95078327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:36.121867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf664da6-70d3-4bf7-bc71-abfe95078327 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:36.122335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf664da6-70d3-4bf7-bc71-abfe95078327 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:36.122646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf664da6-70d3-4bf7-bc71-abfe95078327 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:36.123294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf664da6-70d3-4bf7-bc71-abfe95078327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:36.125048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf664da6-70d3-4bf7-bc71-abfe95078327 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:36.138602 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1287/2642] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:52:36.143327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fde6f3-d862-40dd-b7e6-12d559ba6f79 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:36.153923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fde6f3-d862-40dd-b7e6-12d559ba6f79 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:36.159827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c5798-5aaf-446d-8971-1a0cb0062c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:36.160620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c5798-5aaf-446d-8971-1a0cb0062c4f admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:36.161006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c5798-5aaf-446d-8971-1a0cb0062c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:36.161384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c5798-5aaf-446d-8971-1a0cb0062c4f admin admin] PATH_INFO: `/v3/projects/308124c6e0594e5e8b0e8955f5bf9f35/users/b627dc12e22b4c369eea7aab80e1d11b/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:36.179366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c5798-5aaf-446d-8971-1a0cb0062c4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=308124c6e0594e5e8b0e8955f5bf9f35, user_id=b627dc12e22b4c369eea7aab80e1d11b, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:36.182132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c5798-5aaf-446d-8971-1a0cb0062c4f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:36.199418 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1356/2643] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:52:36 2022] PUT /identity/v3/projects/308124c6e0594e5e8b0e8955f5bf9f35/users/b627dc12e22b4c369eea7aab80e1d11b/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:36.203108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf664da6-70d3-4bf7-bc71-abfe95078327 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:36.213924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf664da6-70d3-4bf7-bc71-abfe95078327 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:36.286657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efcfdc-dd4b-4266-9fb4-8c09554f64b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:36.287306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efcfdc-dd4b-4266-9fb4-8c09554f64b0 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:36.287592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efcfdc-dd4b-4266-9fb4-8c09554f64b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:36.287880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efcfdc-dd4b-4266-9fb4-8c09554f64b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:36.288522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efcfdc-dd4b-4266-9fb4-8c09554f64b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:36.290118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efcfdc-dd4b-4266-9fb4-8c09554f64b0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:36.300712 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1288/2644] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:52:36.305106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c5798-5aaf-446d-8971-1a0cb0062c4f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:36.319361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c5798-5aaf-446d-8971-1a0cb0062c4f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:36.325865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642acf15-b4f2-4753-ad23-e39ad62818e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:36.326571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642acf15-b4f2-4753-ad23-e39ad62818e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:36.326851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642acf15-b4f2-4753-ad23-e39ad62818e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:36.327120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642acf15-b4f2-4753-ad23-e39ad62818e5 admin admin] PATH_INFO: `/v3/projects/308124c6e0594e5e8b0e8955f5bf9f35/users/b627dc12e22b4c369eea7aab80e1d11b/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:36.329087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642acf15-b4f2-4753-ad23-e39ad62818e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=308124c6e0594e5e8b0e8955f5bf9f35, user_id=b627dc12e22b4c369eea7aab80e1d11b, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:36.332695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642acf15-b4f2-4753-ad23-e39ad62818e5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:36.355815 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1357/2645] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:52:36 2022] PUT /identity/v3/projects/308124c6e0594e5e8b0e8955f5bf9f35/users/b627dc12e22b4c369eea7aab80e1d11b/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:36.360795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8c803-4f0d-4a9c-8c82-20882e75d0b8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:36.361103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8c803-4f0d-4a9c-8c82-20882e75d0b8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:36.361420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8c803-4f0d-4a9c-8c82-20882e75d0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:36.397176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-43c8c803-4f0d-4a9c-8c82-20882e75d0b8 None None] MFA Rules not processed for user `b627dc12e22b4c369eea7aab80e1d11b`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:36.398460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-43c8c803-4f0d-4a9c-8c82-20882e75d0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:36.490038 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1289/2646] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:36 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:37.132227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642acf15-b4f2-4753-ad23-e39ad62818e5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:37.137638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091d278-c4bb-45a2-9ee2-8b5c89a84f5b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:37.138727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091d278-c4bb-45a2-9ee2-8b5c89a84f5b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:37.139247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091d278-c4bb-45a2-9ee2-8b5c89a84f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:37.143593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-642acf15-b4f2-4753-ad23-e39ad62818e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:37.165420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e091d278-c4bb-45a2-9ee2-8b5c89a84f5b None None] MFA Rules not processed for user `e5bfbc1e512a445ca7527f3d79e40d16`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:37.167140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e091d278-c4bb-45a2-9ee2-8b5c89a84f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:37.212556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:37.266128 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1290/2647] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:37.285792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642acf15-b4f2-4753-ad23-e39ad62818e5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:37.292821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9dcb4f-2b2a-4230-88fe-3a0c8852efca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:37.293438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dcb4f-2b2a-4230-88fe-3a0c8852efca service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:37.293724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dcb4f-2b2a-4230-88fe-3a0c8852efca service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:37.293993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dcb4f-2b2a-4230-88fe-3a0c8852efca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:37.309028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9dcb4f-2b2a-4230-88fe-3a0c8852efca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:37.310915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9dcb4f-2b2a-4230-88fe-3a0c8852efca service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:37.323484 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1358/2648] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:37 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:38.718933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091d278-c4bb-45a2-9ee2-8b5c89a84f5b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:38.741039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091d278-c4bb-45a2-9ee2-8b5c89a84f5b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:38.746356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8633d6-7d3f-45dd-9be4-ffc6578de0f7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:38.747398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8633d6-7d3f-45dd-9be4-ffc6578de0f7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:38.747831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8633d6-7d3f-45dd-9be4-ffc6578de0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:38.760698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a1159-fa76-4096-8a59-2c59c99ae605 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:38.761741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a1159-fa76-4096-8a59-2c59c99ae605 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:38.763121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a1159-fa76-4096-8a59-2c59c99ae605 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:38.763630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a1159-fa76-4096-8a59-2c59c99ae605 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:38.791748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5f8633d6-7d3f-45dd-9be4-ffc6578de0f7 None None] MFA Rules not processed for user `a749a74841e840e2b8de67fb9562bf51`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:38.795637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5f8633d6-7d3f-45dd-9be4-ffc6578de0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:38.826040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:38.855543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6a1159-fa76-4096-8a59-2c59c99ae605 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:38.857597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6a1159-fa76-4096-8a59-2c59c99ae605 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:38.869719 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1291/2649] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:38 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:38.918453 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1359/2650] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:41.897069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a1159-fa76-4096-8a59-2c59c99ae605 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:42.079961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a1159-fa76-4096-8a59-2c59c99ae605 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:42.091497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c04c6b-9dae-4033-9793-ebab8130a5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:42.092679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c04c6b-9dae-4033-9793-ebab8130a5c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:42.095729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c04c6b-9dae-4033-9793-ebab8130a5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:42.096394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c04c6b-9dae-4033-9793-ebab8130a5c0 admin admin] PATH_INFO: `/v3/users/bb9e693c893b44aeafd7aee7cf30613d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:42.099508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c04c6b-9dae-4033-9793-ebab8130a5c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb9e693c893b44aeafd7aee7cf30613d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:42.105508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c04c6b-9dae-4033-9793-ebab8130a5c0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:42.243803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-55c04c6b-9dae-4033-9793-ebab8130a5c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb9e693c893b44aeafd7aee7cf30613d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:42.244502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-55c04c6b-9dae-4033-9793-ebab8130a5c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb9e693c893b44aeafd7aee7cf30613d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:42.245195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-55c04c6b-9dae-4033-9793-ebab8130a5c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb9e693c893b44aeafd7aee7cf30613d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:42.266754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-55c04c6b-9dae-4033-9793-ebab8130a5c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb9e693c893b44aeafd7aee7cf30613d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:42.313160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-55c04c6b-9dae-4033-9793-ebab8130a5c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb9e693c893b44aeafd7aee7cf30613d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:42.334025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:42.362416 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1292/2651] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:52:41 2022] DELETE /identity/v3/users/bb9e693c893b44aeafd7aee7cf30613d => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:42.364346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1c5cc-8ab9-45cd-acef-b0bbbbcd2b9e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:42.365533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1c5cc-8ab9-45cd-acef-b0bbbbcd2b9e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:42.366198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1c5cc-8ab9-45cd-acef-b0bbbbcd2b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:42.435314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-30d1c5cc-8ab9-45cd-acef-b0bbbbcd2b9e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:42.447514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-30d1c5cc-8ab9-45cd-acef-b0bbbbcd2b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:42.579650 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1360/2652] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:42.605391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c04c6b-9dae-4033-9793-ebab8130a5c0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:42.608431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-55c04c6b-9dae-4033-9793-ebab8130a5c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:42.689356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c04c6b-9dae-4033-9793-ebab8130a5c0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:42.694243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b9981-0403-4443-b019-ba046faf083a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:42.695477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b9981-0403-4443-b019-ba046faf083a service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:42.695931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b9981-0403-4443-b019-ba046faf083a service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:42.696380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b9981-0403-4443-b019-ba046faf083a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:42.712345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b9981-0403-4443-b019-ba046faf083a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:42.714012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b9981-0403-4443-b019-ba046faf083a service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:42.726700 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1293/2653] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:43.355950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1c5cc-8ab9-45cd-acef-b0bbbbcd2b9e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:43.357996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-30d1c5cc-8ab9-45cd-acef-b0bbbbcd2b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:43.386713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1c5cc-8ab9-45cd-acef-b0bbbbcd2b9e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:43.395967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d65523-51d4-4062-bd59-0818505848b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:43.396835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d65523-51d4-4062-bd59-0818505848b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:43.407075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d65523-51d4-4062-bd59-0818505848b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:43.407460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d65523-51d4-4062-bd59-0818505848b5 admin admin] PATH_INFO: `/v3/projects/65ae697418f542acb416e10094c204c3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:43.415862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d65523-51d4-4062-bd59-0818505848b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65ae697418f542acb416e10094c204c3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:43.417814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d65523-51d4-4062-bd59-0818505848b5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:43.505895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f0d65523-51d4-4062-bd59-0818505848b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65ae697418f542acb416e10094c204c3', 'request_id': 'req-f0d65523-51d4-4062-bd59-0818505848b5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:43.515794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f0d65523-51d4-4062-bd59-0818505848b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65ae697418f542acb416e10094c204c3', 'request_id': 'req-f0d65523-51d4-4062-bd59-0818505848b5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:43.525581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f0d65523-51d4-4062-bd59-0818505848b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65ae697418f542acb416e10094c204c3', 'request_id': 'req-f0d65523-51d4-4062-bd59-0818505848b5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:43.575485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f0d65523-51d4-4062-bd59-0818505848b5 admin admin] The token cache is being invalidate because project 65ae697418f542acb416e10094c204c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:52:43.575977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f0d65523-51d4-4062-bd59-0818505848b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:43.578106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f0d65523-51d4-4062-bd59-0818505848b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:43.581347 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1361/2654] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:52:43 2022] DELETE /identity/v3/projects/65ae697418f542acb416e10094c204c3 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:43.585283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b9981-0403-4443-b019-ba046faf083a service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:43.596730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b9981-0403-4443-b019-ba046faf083a service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:43.661963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89075391-80ea-44e1-8c6b-012310788905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:43.662792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89075391-80ea-44e1-8c6b-012310788905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:43.663076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89075391-80ea-44e1-8c6b-012310788905 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:43.663556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89075391-80ea-44e1-8c6b-012310788905 admin admin] PATH_INFO: `/v3/users/b627dc12e22b4c369eea7aab80e1d11b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:43.664707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89075391-80ea-44e1-8c6b-012310788905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b627dc12e22b4c369eea7aab80e1d11b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:43.666672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89075391-80ea-44e1-8c6b-012310788905 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:43.728703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89075391-80ea-44e1-8c6b-012310788905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b627dc12e22b4c369eea7aab80e1d11b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:43.729635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89075391-80ea-44e1-8c6b-012310788905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b627dc12e22b4c369eea7aab80e1d11b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:43.731689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89075391-80ea-44e1-8c6b-012310788905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b627dc12e22b4c369eea7aab80e1d11b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:43.744752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89075391-80ea-44e1-8c6b-012310788905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b627dc12e22b4c369eea7aab80e1d11b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:43.759429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89075391-80ea-44e1-8c6b-012310788905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b627dc12e22b4c369eea7aab80e1d11b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:43.774141 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1294/2655] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:52:43 2022] DELETE /identity/v3/users/b627dc12e22b4c369eea7aab80e1d11b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:44.174603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d65523-51d4-4062-bd59-0818505848b5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:44.177117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f0d65523-51d4-4062-bd59-0818505848b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:44.203986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d65523-51d4-4062-bd59-0818505848b5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:44.253414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89075391-80ea-44e1-8c6b-012310788905 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:44.255007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-89075391-80ea-44e1-8c6b-012310788905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:44.283536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:44.297856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:44.330947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b8848-6e61-4f0b-86cf-75aeb7214c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:44.331587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b8848-6e61-4f0b-86cf-75aeb7214c95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:44.332880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b8848-6e61-4f0b-86cf-75aeb7214c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:44.333953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b8848-6e61-4f0b-86cf-75aeb7214c95 admin admin] PATH_INFO: `/v3/users/b3cecdbd13d74f4397e0f3f34b9d64f0` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:44.337003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b8848-6e61-4f0b-86cf-75aeb7214c95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3cecdbd13d74f4397e0f3f34b9d64f0)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:44.342182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b8848-6e61-4f0b-86cf-75aeb7214c95 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:44.392811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89075391-80ea-44e1-8c6b-012310788905 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:44.402156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d021e0-4c01-4c92-bd2e-74cd12969348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:44.403059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d021e0-4c01-4c92-bd2e-74cd12969348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:44.403392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d021e0-4c01-4c92-bd2e-74cd12969348 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:44.403668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d021e0-4c01-4c92-bd2e-74cd12969348 admin admin] PATH_INFO: `/v3/projects/308124c6e0594e5e8b0e8955f5bf9f35` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:44.405494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d021e0-4c01-4c92-bd2e-74cd12969348 admin admin] RBAC: Authorizing `identity:delete_project(project_id=308124c6e0594e5e8b0e8955f5bf9f35)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:44.409827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d021e0-4c01-4c92-bd2e-74cd12969348 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:44.464387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-468b8848-6e61-4f0b-86cf-75aeb7214c95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3cecdbd13d74f4397e0f3f34b9d64f0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:44.481369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-468b8848-6e61-4f0b-86cf-75aeb7214c95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3cecdbd13d74f4397e0f3f34b9d64f0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:44.502289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-76d021e0-4c01-4c92-bd2e-74cd12969348 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '308124c6e0594e5e8b0e8955f5bf9f35', 'request_id': 'req-76d021e0-4c01-4c92-bd2e-74cd12969348'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:44.505665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-468b8848-6e61-4f0b-86cf-75aeb7214c95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3cecdbd13d74f4397e0f3f34b9d64f0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:44.506896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-468b8848-6e61-4f0b-86cf-75aeb7214c95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3cecdbd13d74f4397e0f3f34b9d64f0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:44.515472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-76d021e0-4c01-4c92-bd2e-74cd12969348 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '308124c6e0594e5e8b0e8955f5bf9f35', 'request_id': 'req-76d021e0-4c01-4c92-bd2e-74cd12969348'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:44.517062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-468b8848-6e61-4f0b-86cf-75aeb7214c95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3cecdbd13d74f4397e0f3f34b9d64f0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:44.527054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-76d021e0-4c01-4c92-bd2e-74cd12969348 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '308124c6e0594e5e8b0e8955f5bf9f35', 'request_id': 'req-76d021e0-4c01-4c92-bd2e-74cd12969348'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:44.528933 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1362/2656] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:52:44 2022] DELETE /identity/v3/users/b3cecdbd13d74f4397e0f3f34b9d64f0 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:52:44.533788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2440d1-8e9b-4af6-8464-4ea3e3329562 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:44.534417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2440d1-8e9b-4af6-8464-4ea3e3329562 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:44.534861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2440d1-8e9b-4af6-8464-4ea3e3329562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:44.544966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-76d021e0-4c01-4c92-bd2e-74cd12969348 admin admin] The token cache is being invalidate because project 308124c6e0594e5e8b0e8955f5bf9f35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:52:44.545667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-76d021e0-4c01-4c92-bd2e-74cd12969348 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:44.547069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-76d021e0-4c01-4c92-bd2e-74cd12969348 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:44.562596 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1295/2657] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:52:44 2022] DELETE /identity/v3/projects/308124c6e0594e5e8b0e8955f5bf9f35 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:44.565102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca058d94-c6cc-4718-a207-81b5c8af9b10 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:44.565494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca058d94-c6cc-4718-a207-81b5c8af9b10 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:44.565872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca058d94-c6cc-4718-a207-81b5c8af9b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:44.566004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8b2440d1-8e9b-4af6-8464-4ea3e3329562 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:44.567249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8b2440d1-8e9b-4af6-8464-4ea3e3329562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:44.596455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ca058d94-c6cc-4718-a207-81b5c8af9b10 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:44.597799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ca058d94-c6cc-4718-a207-81b5c8af9b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:44.656208 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1363/2658] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:52:44.666223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2440d1-8e9b-4af6-8464-4ea3e3329562 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:44.668179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8b2440d1-8e9b-4af6-8464-4ea3e3329562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:44.675715 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1296/2659] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:44.679234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5c322-1505-4ca1-bfdb-5596add7e537 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:44.680938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5c322-1505-4ca1-bfdb-5596add7e537 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:44.681175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5c322-1505-4ca1-bfdb-5596add7e537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:44.702945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-72d5c322-1505-4ca1-bfdb-5596add7e537 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:44.704178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-72d5c322-1505-4ca1-bfdb-5596add7e537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:44.716799 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1297/2660] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:44.717893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5c322-1505-4ca1-bfdb-5596add7e537 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:44.728119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5c322-1505-4ca1-bfdb-5596add7e537 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:44.734048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813c70a-4fd3-4fe0-9ea6-100f2082a039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:44.734678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813c70a-4fd3-4fe0-9ea6-100f2082a039 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:44.734963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813c70a-4fd3-4fe0-9ea6-100f2082a039 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:44.735239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813c70a-4fd3-4fe0-9ea6-100f2082a039 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:44.736485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813c70a-4fd3-4fe0-9ea6-100f2082a039 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:52:44.736978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813c70a-4fd3-4fe0-9ea6-100f2082a039 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:44.738832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813c70a-4fd3-4fe0-9ea6-100f2082a039 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:44.743057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2440d1-8e9b-4af6-8464-4ea3e3329562 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:44.756226 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1298/2661] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:52:44.760619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813c70a-4fd3-4fe0-9ea6-100f2082a039 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:44.762413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51cb10d-fca7-49c5-881f-90bfdc182f83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:44.763184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51cb10d-fca7-49c5-881f-90bfdc182f83 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:44.763577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51cb10d-fca7-49c5-881f-90bfdc182f83 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:44.764117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51cb10d-fca7-49c5-881f-90bfdc182f83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:44.775006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51cb10d-fca7-49c5-881f-90bfdc182f83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:44.776041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813c70a-4fd3-4fe0-9ea6-100f2082a039 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:44.777741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51cb10d-fca7-49c5-881f-90bfdc182f83 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:44.785310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a123cf-4d09-4d3b-94ef-da7b10f860e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:44.787032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a123cf-4d09-4d3b-94ef-da7b10f860e3 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:44.787501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a123cf-4d09-4d3b-94ef-da7b10f860e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:44.787987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a123cf-4d09-4d3b-94ef-da7b10f860e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:44.789304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a123cf-4d09-4d3b-94ef-da7b10f860e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:44.792077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a123cf-4d09-4d3b-94ef-da7b10f860e3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:44.795602 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1364/2662] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:52:44.834557 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1299/2663] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:52:44 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:44.839456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51cb10d-fca7-49c5-881f-90bfdc182f83 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:44.852398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51cb10d-fca7-49c5-881f-90bfdc182f83 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:44.863981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:44.948436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73bc5b-0c03-46f4-89ad-f4aa43118fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:44.949059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73bc5b-0c03-46f4-89ad-f4aa43118fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:44.949338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73bc5b-0c03-46f4-89ad-f4aa43118fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:44.949610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73bc5b-0c03-46f4-89ad-f4aa43118fd1 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:44.950244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73bc5b-0c03-46f4-89ad-f4aa43118fd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:44.953454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73bc5b-0c03-46f4-89ad-f4aa43118fd1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:45.007046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a123cf-4d09-4d3b-94ef-da7b10f860e3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:45.007243 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1365/2664] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:52:44 2022] POST /identity/v3/users => generated 288 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:45.025117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a123cf-4d09-4d3b-94ef-da7b10f860e3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:45.046604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42934520-4022-462a-89c5-a767088873ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:45.047587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42934520-4022-462a-89c5-a767088873ab admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:45.048158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42934520-4022-462a-89c5-a767088873ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:45.049667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42934520-4022-462a-89c5-a767088873ab admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:45.050590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42934520-4022-462a-89c5-a767088873ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:45.053519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42934520-4022-462a-89c5-a767088873ab admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:45.056068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:45.074295 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1300/2665] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:52:45.075462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73bc5b-0c03-46f4-89ad-f4aa43118fd1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:45.088271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73bc5b-0c03-46f4-89ad-f4aa43118fd1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:45.097306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea74978-e6a3-4867-8e30-22ee7b2257bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:45.098019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea74978-e6a3-4867-8e30-22ee7b2257bb admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:45.098357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea74978-e6a3-4867-8e30-22ee7b2257bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:45.098761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea74978-e6a3-4867-8e30-22ee7b2257bb admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:45.099619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea74978-e6a3-4867-8e30-22ee7b2257bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:45.101387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea74978-e6a3-4867-8e30-22ee7b2257bb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:45.120415 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1366/2666] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:52:45.122955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42934520-4022-462a-89c5-a767088873ab admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:45.149770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42934520-4022-462a-89c5-a767088873ab admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:45.155878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8e3b8-2771-4a80-8ac0-e508473901d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:45.156803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8e3b8-2771-4a80-8ac0-e508473901d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:45.157083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8e3b8-2771-4a80-8ac0-e508473901d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:45.157398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8e3b8-2771-4a80-8ac0-e508473901d5 admin admin] PATH_INFO: `/v3/projects/f0e7756c4e2a408693ace3c9c3fc2680/users/f4390915c556409cb6b718e515d822c8/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:45.177507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8e3b8-2771-4a80-8ac0-e508473901d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e7756c4e2a408693ace3c9c3fc2680, user_id=f4390915c556409cb6b718e515d822c8, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:45.180550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8e3b8-2771-4a80-8ac0-e508473901d5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:45.196001 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1301/2667] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:52:45 2022] PUT /identity/v3/projects/f0e7756c4e2a408693ace3c9c3fc2680/users/f4390915c556409cb6b718e515d822c8/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:45.209786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b855df5-b91f-47f3-a58a-4ca064904336 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:45.210136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b855df5-b91f-47f3-a58a-4ca064904336 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:45.210355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b855df5-b91f-47f3-a58a-4ca064904336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:45.247140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3b855df5-b91f-47f3-a58a-4ca064904336 None None] MFA Rules not processed for user `f4390915c556409cb6b718e515d822c8`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:45.247140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3b855df5-b91f-47f3-a58a-4ca064904336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:45.328635 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1367/2668] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:45.333599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8e3b8-2771-4a80-8ac0-e508473901d5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:45.409553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8e3b8-2771-4a80-8ac0-e508473901d5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:45.415840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ba1bd-8e68-4269-9b73-d5b016201709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:45.416607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ba1bd-8e68-4269-9b73-d5b016201709 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:45.416957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ba1bd-8e68-4269-9b73-d5b016201709 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:45.417301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ba1bd-8e68-4269-9b73-d5b016201709 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:45.417952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ba1bd-8e68-4269-9b73-d5b016201709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:45.420003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ba1bd-8e68-4269-9b73-d5b016201709 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:45.454307 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1302/2669] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:52:45 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:52:45.456165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b855df5-b91f-47f3-a58a-4ca064904336 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:45.472152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b855df5-b91f-47f3-a58a-4ca064904336 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:45.555148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e9326-3f2c-40d9-9de7-e6e7c7827384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:45.555774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e9326-3f2c-40d9-9de7-e6e7c7827384 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:45.556058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e9326-3f2c-40d9-9de7-e6e7c7827384 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:45.556480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e9326-3f2c-40d9-9de7-e6e7c7827384 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:45.557139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e9326-3f2c-40d9-9de7-e6e7c7827384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:45.559346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e9326-3f2c-40d9-9de7-e6e7c7827384 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:45.593385 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1368/2670] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:52:45 2022] POST /identity/v3/users => generated 294 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:45.597686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ba1bd-8e68-4269-9b73-d5b016201709 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:45.626283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ba1bd-8e68-4269-9b73-d5b016201709 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:45.643945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db687bc-1b05-4c29-b1bc-a280ffbd89b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:45.644873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db687bc-1b05-4c29-b1bc-a280ffbd89b2 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:45.645358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db687bc-1b05-4c29-b1bc-a280ffbd89b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:45.646635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db687bc-1b05-4c29-b1bc-a280ffbd89b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:45.648132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db687bc-1b05-4c29-b1bc-a280ffbd89b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:45.650009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db687bc-1b05-4c29-b1bc-a280ffbd89b2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:45.670168 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1303/2671] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:52:45.677041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e9326-3f2c-40d9-9de7-e6e7c7827384 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:45.703358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e9326-3f2c-40d9-9de7-e6e7c7827384 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:45.711801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019b22f-a8f6-4243-a322-2623b2eb20c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:45.713239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019b22f-a8f6-4243-a322-2623b2eb20c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:45.713732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019b22f-a8f6-4243-a322-2623b2eb20c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:45.714236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019b22f-a8f6-4243-a322-2623b2eb20c1 admin admin] PATH_INFO: `/v3/projects/09e4b27d45544465bcb556504c44cfbc/users/d5d5c826ff90477bbffefd37bbbac4d0/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:45.718386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:45.735357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019b22f-a8f6-4243-a322-2623b2eb20c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09e4b27d45544465bcb556504c44cfbc, user_id=d5d5c826ff90477bbffefd37bbbac4d0, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:45.739206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019b22f-a8f6-4243-a322-2623b2eb20c1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:45.755363 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1369/2672] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:52:45 2022] PUT /identity/v3/projects/09e4b27d45544465bcb556504c44cfbc/users/d5d5c826ff90477bbffefd37bbbac4d0/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:52:45.759234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db687bc-1b05-4c29-b1bc-a280ffbd89b2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:45.771038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db687bc-1b05-4c29-b1bc-a280ffbd89b2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:45.844541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427cf63-7775-42e7-aec2-4c3959e4f988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:45.845917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427cf63-7775-42e7-aec2-4c3959e4f988 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:45.846385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427cf63-7775-42e7-aec2-4c3959e4f988 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:45.846786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427cf63-7775-42e7-aec2-4c3959e4f988 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:45.852195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427cf63-7775-42e7-aec2-4c3959e4f988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:45.854657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427cf63-7775-42e7-aec2-4c3959e4f988 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:45.869552 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1304/2673] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:52:45.873739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019b22f-a8f6-4243-a322-2623b2eb20c1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:45.885971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019b22f-a8f6-4243-a322-2623b2eb20c1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:45.891202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25d30e-2cd8-4066-af46-0375dc0c7c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:45.891883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25d30e-2cd8-4066-af46-0375dc0c7c4b admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:45.892670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25d30e-2cd8-4066-af46-0375dc0c7c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:45.892967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25d30e-2cd8-4066-af46-0375dc0c7c4b admin admin] PATH_INFO: `/v3/projects/09e4b27d45544465bcb556504c44cfbc/users/d5d5c826ff90477bbffefd37bbbac4d0/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:45.894994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25d30e-2cd8-4066-af46-0375dc0c7c4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=09e4b27d45544465bcb556504c44cfbc, user_id=d5d5c826ff90477bbffefd37bbbac4d0, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:45.897640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25d30e-2cd8-4066-af46-0375dc0c7c4b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:45.909592 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1370/2674] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:52:45 2022] PUT /identity/v3/projects/09e4b27d45544465bcb556504c44cfbc/users/d5d5c826ff90477bbffefd37bbbac4d0/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:45.920121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150bafb-7d18-40e1-b4f8-11aa673190fb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:45.920408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150bafb-7d18-40e1-b4f8-11aa673190fb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:45.920684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150bafb-7d18-40e1-b4f8-11aa673190fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:45.953315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8150bafb-7d18-40e1-b4f8-11aa673190fb None None] MFA Rules not processed for user `d5d5c826ff90477bbffefd37bbbac4d0`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:45.955469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8150bafb-7d18-40e1-b4f8-11aa673190fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:45.998433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:46.075857 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1305/2675] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:45 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:52:46.083677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25d30e-2cd8-4066-af46-0375dc0c7c4b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:46.106685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25d30e-2cd8-4066-af46-0375dc0c7c4b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:46.115118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9705f8-b8d5-4765-a0fc-e1a72133df6c tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d5c826ff90477bbffefd37bbbac4d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '09e4b27d45544465bcb556504c44cfbc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:46.115118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9705f8-b8d5-4765-a0fc-e1a72133df6c tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:46.115118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9705f8-b8d5-4765-a0fc-e1a72133df6c tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:46.115510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9705f8-b8d5-4765-a0fc-e1a72133df6c tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:46.115510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9705f8-b8d5-4765-a0fc-e1a72133df6c tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:52:46.115510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9705f8-b8d5-4765-a0fc-e1a72133df6c tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:46.116913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9705f8-b8d5-4765-a0fc-e1a72133df6c tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:46.150679 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1371/2676] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:52:46.155325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150bafb-7d18-40e1-b4f8-11aa673190fb None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:46.166732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150bafb-7d18-40e1-b4f8-11aa673190fb None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:46.171530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67053dd0-4fbe-40e0-b70b-937fbfc4018e tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d5c826ff90477bbffefd37bbbac4d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '09e4b27d45544465bcb556504c44cfbc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:46.172222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67053dd0-4fbe-40e0-b70b-937fbfc4018e tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:46.172573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67053dd0-4fbe-40e0-b70b-937fbfc4018e tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:46.172917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67053dd0-4fbe-40e0-b70b-937fbfc4018e tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:46.173663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67053dd0-4fbe-40e0-b70b-937fbfc4018e tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:46.175837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67053dd0-4fbe-40e0-b70b-937fbfc4018e tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:46.211061 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1306/2677] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:52:46 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:46.211830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9705f8-b8d5-4765-a0fc-e1a72133df6c tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:46.224401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9705f8-b8d5-4765-a0fc-e1a72133df6c tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:46.259448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67053dd0-4fbe-40e0-b70b-937fbfc4018e tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:46.261232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-67053dd0-4fbe-40e0-b70b-937fbfc4018e tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:46.316428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7dd1d-5191-461e-8e97-ddc9a3800b47 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d5c826ff90477bbffefd37bbbac4d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '09e4b27d45544465bcb556504c44cfbc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:46.317747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7dd1d-5191-461e-8e97-ddc9a3800b47 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:46.318788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7dd1d-5191-461e-8e97-ddc9a3800b47 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:46.319077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7dd1d-5191-461e-8e97-ddc9a3800b47 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:46.320116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7dd1d-5191-461e-8e97-ddc9a3800b47 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:52:46.320431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7dd1d-5191-461e-8e97-ddc9a3800b47 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:46.322727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7dd1d-5191-461e-8e97-ddc9a3800b47 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:46.347393 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1372/2678] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:52:46.364973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67053dd0-4fbe-40e0-b70b-937fbfc4018e tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:46.370609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc19a4a-dcec-4c2f-889d-b7b11bb7372d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:46.371497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc19a4a-dcec-4c2f-889d-b7b11bb7372d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:46.371992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc19a4a-dcec-4c2f-889d-b7b11bb7372d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:46.372470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc19a4a-dcec-4c2f-889d-b7b11bb7372d admin admin] PATH_INFO: `/v3/projects/ec1c1a8f4f8745388760db1428e4c1b5` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:46.373806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc19a4a-dcec-4c2f-889d-b7b11bb7372d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec1c1a8f4f8745388760db1428e4c1b5)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:46.376139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc19a4a-dcec-4c2f-889d-b7b11bb7372d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:46.432869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-afc19a4a-dcec-4c2f-889d-b7b11bb7372d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec1c1a8f4f8745388760db1428e4c1b5', 'request_id': 'req-afc19a4a-dcec-4c2f-889d-b7b11bb7372d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:46.441553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-afc19a4a-dcec-4c2f-889d-b7b11bb7372d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec1c1a8f4f8745388760db1428e4c1b5', 'request_id': 'req-afc19a4a-dcec-4c2f-889d-b7b11bb7372d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:46.448365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-afc19a4a-dcec-4c2f-889d-b7b11bb7372d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec1c1a8f4f8745388760db1428e4c1b5', 'request_id': 'req-afc19a4a-dcec-4c2f-889d-b7b11bb7372d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:46.460050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-afc19a4a-dcec-4c2f-889d-b7b11bb7372d admin admin] The token cache is being invalidate because project ec1c1a8f4f8745388760db1428e4c1b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:52:46.460344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-afc19a4a-dcec-4c2f-889d-b7b11bb7372d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:46.461040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-afc19a4a-dcec-4c2f-889d-b7b11bb7372d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:46.462572 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1307/2679] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:52:46 2022] DELETE /identity/v3/projects/ec1c1a8f4f8745388760db1428e4c1b5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:46.467470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f469e44-530e-4944-9fae-dcbb3c9ba197 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:46.467758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f469e44-530e-4944-9fae-dcbb3c9ba197 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:46.468036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f469e44-530e-4944-9fae-dcbb3c9ba197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:46.495607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5f469e44-530e-4944-9fae-dcbb3c9ba197 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:46.496918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5f469e44-530e-4944-9fae-dcbb3c9ba197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:46.546886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc19a4a-dcec-4c2f-889d-b7b11bb7372d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:46.550992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-afc19a4a-dcec-4c2f-889d-b7b11bb7372d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:46.646407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:46.657745 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1373/2680] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:46.659215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850f42e-eedb-4433-911f-2d6e915cfced None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:46.659586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850f42e-eedb-4433-911f-2d6e915cfced None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:46.660087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850f42e-eedb-4433-911f-2d6e915cfced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:46.682458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0850f42e-eedb-4433-911f-2d6e915cfced None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:46.683834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0850f42e-eedb-4433-911f-2d6e915cfced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:46.696548 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1374/2681] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:46.703533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850f42e-eedb-4433-911f-2d6e915cfced None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:46.716590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850f42e-eedb-4433-911f-2d6e915cfced None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:46.722892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad438ab-7b6e-4f80-bff4-795f98d7359e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:46.723510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad438ab-7b6e-4f80-bff4-795f98d7359e admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:46.723791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad438ab-7b6e-4f80-bff4-795f98d7359e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:46.724066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad438ab-7b6e-4f80-bff4-795f98d7359e admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:46.724722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad438ab-7b6e-4f80-bff4-795f98d7359e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:52:46.725025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad438ab-7b6e-4f80-bff4-795f98d7359e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:46.726730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad438ab-7b6e-4f80-bff4-795f98d7359e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:46.744198 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1375/2682] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:52:46.747680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc19a4a-dcec-4c2f-889d-b7b11bb7372d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:46.748479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad438ab-7b6e-4f80-bff4-795f98d7359e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:46.755371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d03fdc-6464-47cd-8589-8bf79c3b2886 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:46.756925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d03fdc-6464-47cd-8589-8bf79c3b2886 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:46.756925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d03fdc-6464-47cd-8589-8bf79c3b2886 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:46.757483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d03fdc-6464-47cd-8589-8bf79c3b2886 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:46.760790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-56d03fdc-6464-47cd-8589-8bf79c3b2886 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:46.763345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad438ab-7b6e-4f80-bff4-795f98d7359e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:46.771052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5365aa6-2e36-42da-b926-884ad14bc505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:46.772645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5365aa6-2e36-42da-b926-884ad14bc505 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:46.773123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5365aa6-2e36-42da-b926-884ad14bc505 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:46.773451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5365aa6-2e36-42da-b926-884ad14bc505 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:46.774621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5365aa6-2e36-42da-b926-884ad14bc505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:46.793958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5365aa6-2e36-42da-b926-884ad14bc505 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:46.843634 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1376/2683] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:52:46 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:46.848303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5365aa6-2e36-42da-b926-884ad14bc505 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:46.870339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:46.889797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d03fdc-6464-47cd-8589-8bf79c3b2886 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:46.891486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d03fdc-6464-47cd-8589-8bf79c3b2886 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:46.903407 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1308/2684] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:46 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:46.931311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed3333-4718-4577-b37f-7f7144052cfb None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:46.931798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed3333-4718-4577-b37f-7f7144052cfb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:46.932106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed3333-4718-4577-b37f-7f7144052cfb None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:46.933990 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1309/2685] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:52:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 01 20:52:46.938418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5365aa6-2e36-42da-b926-884ad14bc505 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:46.938843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed3333-4718-4577-b37f-7f7144052cfb None None] Authenticating service token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:52:46.945054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda1f40-c0e7-42fd-9686-bfa71f1afa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:46.946009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda1f40-c0e7-42fd-9686-bfa71f1afa69 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:46.946483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda1f40-c0e7-42fd-9686-bfa71f1afa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:46.947041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda1f40-c0e7-42fd-9686-bfa71f1afa69 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:46.947965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda1f40-c0e7-42fd-9686-bfa71f1afa69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:46.950540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda1f40-c0e7-42fd-9686-bfa71f1afa69 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:46.981659 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1377/2686] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:52:46 2022] POST /identity/v3/users => generated 290 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:46.985164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda1f40-c0e7-42fd-9686-bfa71f1afa69 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:46.998904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda1f40-c0e7-42fd-9686-bfa71f1afa69 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:47.006142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff5902-ffbc-4683-8324-1c42313d561e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:47.006797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff5902-ffbc-4683-8324-1c42313d561e admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:47.007098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff5902-ffbc-4683-8324-1c42313d561e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:47.007340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff5902-ffbc-4683-8324-1c42313d561e admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:47.008177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff5902-ffbc-4683-8324-1c42313d561e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:47.009698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff5902-ffbc-4683-8324-1c42313d561e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:47.022738 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1378/2687] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:52:47.034809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff5902-ffbc-4683-8324-1c42313d561e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:47.040757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed3333-4718-4577-b37f-7f7144052cfb None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:47.057580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff5902-ffbc-4683-8324-1c42313d561e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:47.074599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1def0-2b86-42f2-b355-50aa744292ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:47.075223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1def0-2b86-42f2-b355-50aa744292ec admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:47.075498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1def0-2b86-42f2-b355-50aa744292ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:47.075768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1def0-2b86-42f2-b355-50aa744292ec admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:47.076410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1def0-2b86-42f2-b355-50aa744292ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:47.078079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1def0-2b86-42f2-b355-50aa744292ec admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:47.089958 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1379/2688] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:52:47.094432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1def0-2b86-42f2-b355-50aa744292ec admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:47.107643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1def0-2b86-42f2-b355-50aa744292ec admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:47.115052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d1cc4-f39e-488f-a30d-7a9f25df1fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:47.115725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d1cc4-f39e-488f-a30d-7a9f25df1fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:47.116304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d1cc4-f39e-488f-a30d-7a9f25df1fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:47.116304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d1cc4-f39e-488f-a30d-7a9f25df1fb2 admin admin] PATH_INFO: `/v3/projects/caba560a5eaa46c1b816e09230b5a1d9/users/d270f04a32ac44bc88fc57f8e765111e/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:47.137070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d1cc4-f39e-488f-a30d-7a9f25df1fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caba560a5eaa46c1b816e09230b5a1d9, user_id=d270f04a32ac44bc88fc57f8e765111e, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:47.138010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed3333-4718-4577-b37f-7f7144052cfb None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:47.139965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d1cc4-f39e-488f-a30d-7a9f25df1fb2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:47.144647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a62ab3-9b43-4092-a2a3-2f5a89a79571 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d5c826ff90477bbffefd37bbbac4d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '09e4b27d45544465bcb556504c44cfbc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:47.145349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a62ab3-9b43-4092-a2a3-2f5a89a79571 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:47.145792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a62ab3-9b43-4092-a2a3-2f5a89a79571 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:47.146072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a62ab3-9b43-4092-a2a3-2f5a89a79571 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] PATH_INFO: `/v3/projects/27fa2a1f8926420c8c7cdaa9048527c8` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:47.147463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a62ab3-9b43-4092-a2a3-2f5a89a79571 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorizing `identity:get_project(project_id=27fa2a1f8926420c8c7cdaa9048527c8)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:47.149681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a62ab3-9b43-4092-a2a3-2f5a89a79571 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:47.151561 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1310/2689] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:52:46 2022] GET /identity/v3/projects/27fa2a1f8926420c8c7cdaa9048527c8 => generated 369 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:52:47.159675 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1380/2690] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:52:47 2022] PUT /identity/v3/projects/caba560a5eaa46c1b816e09230b5a1d9/users/d270f04a32ac44bc88fc57f8e765111e/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:47.164956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2f553-dd01-44b8-b185-8d981913352e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:47.165499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2f553-dd01-44b8-b185-8d981913352e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:47.166020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2f553-dd01-44b8-b185-8d981913352e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:47.178050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:47.193981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ced2f553-dd01-44b8-b185-8d981913352e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:47.195843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ced2f553-dd01-44b8-b185-8d981913352e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:47.280481 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1311/2691] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:47.295866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d1cc4-f39e-488f-a30d-7a9f25df1fb2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:47.299921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-315d1cc4-f39e-488f-a30d-7a9f25df1fb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:47.371618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d1cc4-f39e-488f-a30d-7a9f25df1fb2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:47.375561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8527967-9ffd-4e43-9063-d9ab008df911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:47.376174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8527967-9ffd-4e43-9063-d9ab008df911 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:47.376522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8527967-9ffd-4e43-9063-d9ab008df911 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:47.377303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8527967-9ffd-4e43-9063-d9ab008df911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:47.387490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8527967-9ffd-4e43-9063-d9ab008df911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:47.388916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8527967-9ffd-4e43-9063-d9ab008df911 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:47.400421 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1381/2692] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:47.740958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88905d14-8206-41fe-ab73-5942231cb0ea None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:47.742082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88905d14-8206-41fe-ab73-5942231cb0ea None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:47.742906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88905d14-8206-41fe-ab73-5942231cb0ea None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:47.744957 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1312/2693] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:52:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:52:47.751101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8527967-9ffd-4e43-9063-d9ab008df911 service neutron] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:52:47.753337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b8527967-9ffd-4e43-9063-d9ab008df911 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:47.855262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:47.885557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8527967-9ffd-4e43-9063-d9ab008df911 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:47.960770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8527967-9ffd-4e43-9063-d9ab008df911 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:47.967148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0a62c-6c09-439a-9d84-084e4ca10970 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d5c826ff90477bbffefd37bbbac4d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '09e4b27d45544465bcb556504c44cfbc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:47.967535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0a62c-6c09-439a-9d84-084e4ca10970 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:47.967816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0a62c-6c09-439a-9d84-084e4ca10970 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:47.968384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0a62c-6c09-439a-9d84-084e4ca10970 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] PATH_INFO: `/v3/projects/27fa2a1f8926420c8c7cdaa9048527c8` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:47.969411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0a62c-6c09-439a-9d84-084e4ca10970 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorizing `identity:get_project(project_id=27fa2a1f8926420c8c7cdaa9048527c8)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:47.972296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0a62c-6c09-439a-9d84-084e4ca10970 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:47.974284 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1382/2694] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:52:47 2022] GET /identity/v3/projects/27fa2a1f8926420c8c7cdaa9048527c8 => generated 369 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:52:48.037285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88905d14-8206-41fe-ab73-5942231cb0ea None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:48.072710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88905d14-8206-41fe-ab73-5942231cb0ea None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:48.090835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9cab2-2659-42ae-9388-5ed0f4c3e1cf tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d5c826ff90477bbffefd37bbbac4d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '09e4b27d45544465bcb556504c44cfbc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:48.091738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9cab2-2659-42ae-9388-5ed0f4c3e1cf tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:48.092325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9cab2-2659-42ae-9388-5ed0f4c3e1cf tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:48.092817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9cab2-2659-42ae-9388-5ed0f4c3e1cf tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:48.094403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9cab2-2659-42ae-9388-5ed0f4c3e1cf tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:52:48.095030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9cab2-2659-42ae-9388-5ed0f4c3e1cf tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:48.097588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9cab2-2659-42ae-9388-5ed0f4c3e1cf tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:48.112589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0a62c-6c09-439a-9d84-084e4ca10970 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:48.144221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0a62c-6c09-439a-9d84-084e4ca10970 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:48.154221 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1313/2695] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:52:48.160589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9cab2-2659-42ae-9388-5ed0f4c3e1cf tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:48.168073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099843c3-aecc-4a57-8fe5-1f15ed34e1ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:48.168703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099843c3-aecc-4a57-8fe5-1f15ed34e1ee service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:48.169560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099843c3-aecc-4a57-8fe5-1f15ed34e1ee service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:48.169560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099843c3-aecc-4a57-8fe5-1f15ed34e1ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:48.174323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-099843c3-aecc-4a57-8fe5-1f15ed34e1ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:48.201407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9cab2-2659-42ae-9388-5ed0f4c3e1cf tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:48.209957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1553f-cb85-4d32-944e-754f5af40795 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d5c826ff90477bbffefd37bbbac4d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '09e4b27d45544465bcb556504c44cfbc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:48.210705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1553f-cb85-4d32-944e-754f5af40795 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:48.210904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1553f-cb85-4d32-944e-754f5af40795 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:48.211245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1553f-cb85-4d32-944e-754f5af40795 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:48.211902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1553f-cb85-4d32-944e-754f5af40795 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:48.214702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1553f-cb85-4d32-944e-754f5af40795 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:48.279486 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1314/2696] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:52:48 2022] POST /identity/v3/users => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:48.280488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1553f-cb85-4d32-944e-754f5af40795 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:48.299459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099843c3-aecc-4a57-8fe5-1f15ed34e1ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:48.302017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099843c3-aecc-4a57-8fe5-1f15ed34e1ee service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:48.304858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1553f-cb85-4d32-944e-754f5af40795 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:48.316641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98023f50-7993-490c-a8c1-98ab4d4a8e78 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d5c826ff90477bbffefd37bbbac4d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '09e4b27d45544465bcb556504c44cfbc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:48.318019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98023f50-7993-490c-a8c1-98ab4d4a8e78 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:48.318320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98023f50-7993-490c-a8c1-98ab4d4a8e78 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:48.318672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98023f50-7993-490c-a8c1-98ab4d4a8e78 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:48.319415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98023f50-7993-490c-a8c1-98ab4d4a8e78 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:52:48.319697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98023f50-7993-490c-a8c1-98ab4d4a8e78 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:48.322006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98023f50-7993-490c-a8c1-98ab4d4a8e78 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:48.351839 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1315/2698] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:52:48.354196 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1383/2698] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:48.507019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816da3d-a7ec-40f1-a05d-d5985858e9ae None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:48.507651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816da3d-a7ec-40f1-a05d-d5985858e9ae None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:48.508018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816da3d-a7ec-40f1-a05d-d5985858e9ae None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:48.509789 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1316/2699] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:52:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:52:48.518703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099843c3-aecc-4a57-8fe5-1f15ed34e1ee service nova] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:52:48.536801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099843c3-aecc-4a57-8fe5-1f15ed34e1ee service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:48.549495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099843c3-aecc-4a57-8fe5-1f15ed34e1ee service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:48.555265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878662a5-96ad-42b3-9cb3-5cb2630688c4 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d5c826ff90477bbffefd37bbbac4d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '09e4b27d45544465bcb556504c44cfbc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:48.565447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878662a5-96ad-42b3-9cb3-5cb2630688c4 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:48.565743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878662a5-96ad-42b3-9cb3-5cb2630688c4 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:48.566054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878662a5-96ad-42b3-9cb3-5cb2630688c4 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] PATH_INFO: `/v3/projects/27fa2a1f8926420c8c7cdaa9048527c8` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:48.568490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878662a5-96ad-42b3-9cb3-5cb2630688c4 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorizing `identity:get_project(project_id=27fa2a1f8926420c8c7cdaa9048527c8)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:48.570738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878662a5-96ad-42b3-9cb3-5cb2630688c4 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:48.574562 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1384/2700] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:52:48 2022] GET /identity/v3/projects/27fa2a1f8926420c8c7cdaa9048527c8 => generated 369 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:52:48.852261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a45f9-4fa2-49f8-94fa-ae9139a0235f None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:48.852895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a45f9-4fa2-49f8-94fa-ae9139a0235f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:48.852895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a45f9-4fa2-49f8-94fa-ae9139a0235f None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:48.854252 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1317/2701] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:52:48.863352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878662a5-96ad-42b3-9cb3-5cb2630688c4 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:52:48.892711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878662a5-96ad-42b3-9cb3-5cb2630688c4 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:48.924298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878662a5-96ad-42b3-9cb3-5cb2630688c4 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:48.930941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403ece4-7fef-42d0-8fcf-4c8e6c467c31 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d5c826ff90477bbffefd37bbbac4d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '09e4b27d45544465bcb556504c44cfbc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:48.932384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403ece4-7fef-42d0-8fcf-4c8e6c467c31 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:48.932851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403ece4-7fef-42d0-8fcf-4c8e6c467c31 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:48.933295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403ece4-7fef-42d0-8fcf-4c8e6c467c31 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] PATH_INFO: `/v3/projects/27fa2a1f8926420c8c7cdaa9048527c8` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:48.934650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403ece4-7fef-42d0-8fcf-4c8e6c467c31 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorizing `identity:get_project(project_id=27fa2a1f8926420c8c7cdaa9048527c8)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:48.939419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403ece4-7fef-42d0-8fcf-4c8e6c467c31 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:48.942087 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1385/2702] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:52:48 2022] GET /identity/v3/projects/27fa2a1f8926420c8c7cdaa9048527c8 => generated 369 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:52:48.979258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a45f9-4fa2-49f8-94fa-ae9139a0235f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:49.003161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a45f9-4fa2-49f8-94fa-ae9139a0235f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:49.018645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4f609-6079-490a-b964-a3ace93aff68 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d5c826ff90477bbffefd37bbbac4d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '09e4b27d45544465bcb556504c44cfbc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:49.019806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4f609-6079-490a-b964-a3ace93aff68 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:49.020100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4f609-6079-490a-b964-a3ace93aff68 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:49.020435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4f609-6079-490a-b964-a3ace93aff68 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] PATH_INFO: `/v3/users/fd267ab95c8a412aae5da7e8e4036f22` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:49.066696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4f609-6079-490a-b964-a3ace93aff68 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorizing `identity:delete_user(user_id=fd267ab95c8a412aae5da7e8e4036f22)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:49.069714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4f609-6079-490a-b964-a3ace93aff68 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:49.167547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-76f4f609-6079-490a-b964-a3ace93aff68 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd267ab95c8a412aae5da7e8e4036f22'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:49.170283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-76f4f609-6079-490a-b964-a3ace93aff68 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd267ab95c8a412aae5da7e8e4036f22'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:49.175519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-76f4f609-6079-490a-b964-a3ace93aff68 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd267ab95c8a412aae5da7e8e4036f22'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:49.197974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-76f4f609-6079-490a-b964-a3ace93aff68 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd267ab95c8a412aae5da7e8e4036f22'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:49.241257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-76f4f609-6079-490a-b964-a3ace93aff68 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd267ab95c8a412aae5da7e8e4036f22'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:49.261559 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1318/2703] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:52:48 2022] DELETE /identity/v3/users/fd267ab95c8a412aae5da7e8e4036f22 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:49.265523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403ece4-7fef-42d0-8fcf-4c8e6c467c31 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:49.267385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8403ece4-7fef-42d0-8fcf-4c8e6c467c31 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:49.294398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403ece4-7fef-42d0-8fcf-4c8e6c467c31 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:49.436894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59462e52-d88d-469a-8c20-67d05162d33d tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5d5c826ff90477bbffefd37bbbac4d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '09e4b27d45544465bcb556504c44cfbc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:49.438475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59462e52-d88d-469a-8c20-67d05162d33d tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:49.438821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59462e52-d88d-469a-8c20-67d05162d33d tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:49.439105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59462e52-d88d-469a-8c20-67d05162d33d tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] PATH_INFO: `/v3/projects/27fa2a1f8926420c8c7cdaa9048527c8` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:49.441102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59462e52-d88d-469a-8c20-67d05162d33d tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorizing `identity:delete_project(project_id=27fa2a1f8926420c8c7cdaa9048527c8)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:49.444560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59462e52-d88d-469a-8c20-67d05162d33d tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:49.475856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:49.557731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-59462e52-d88d-469a-8c20-67d05162d33d tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27fa2a1f8926420c8c7cdaa9048527c8', 'request_id': 'req-59462e52-d88d-469a-8c20-67d05162d33d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:49.572057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-59462e52-d88d-469a-8c20-67d05162d33d tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27fa2a1f8926420c8c7cdaa9048527c8', 'request_id': 'req-59462e52-d88d-469a-8c20-67d05162d33d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:49.582303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-59462e52-d88d-469a-8c20-67d05162d33d tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27fa2a1f8926420c8c7cdaa9048527c8', 'request_id': 'req-59462e52-d88d-469a-8c20-67d05162d33d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:49.627462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-59462e52-d88d-469a-8c20-67d05162d33d tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] The token cache is being invalidate because project 27fa2a1f8926420c8c7cdaa9048527c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:52:49.627868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-59462e52-d88d-469a-8c20-67d05162d33d tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:49.629176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-59462e52-d88d-469a-8c20-67d05162d33d tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:49.636894 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1386/2704] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:52:49 2022] DELETE /identity/v3/projects/27fa2a1f8926420c8c7cdaa9048527c8 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:49.643612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4f609-6079-490a-b964-a3ace93aff68 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:49.645190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-76f4f609-6079-490a-b964-a3ace93aff68 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:49.708376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:49.779219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4f609-6079-490a-b964-a3ace93aff68 tempest-QuotasAdminTestV236-2041601533 tempest-QuotasAdminTestV236-2041601533-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:49.868626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6961bb4-8b06-4a05-8292-dd06c2c47dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:49.870475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6961bb4-8b06-4a05-8292-dd06c2c47dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:49.870947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6961bb4-8b06-4a05-8292-dd06c2c47dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:49.871414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6961bb4-8b06-4a05-8292-dd06c2c47dfe admin admin] PATH_INFO: `/v3/users/f4390915c556409cb6b718e515d822c8` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:49.873295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6961bb4-8b06-4a05-8292-dd06c2c47dfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4390915c556409cb6b718e515d822c8)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:49.876000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6961bb4-8b06-4a05-8292-dd06c2c47dfe admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:49.992298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a6961bb4-8b06-4a05-8292-dd06c2c47dfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4390915c556409cb6b718e515d822c8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:49.996471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a6961bb4-8b06-4a05-8292-dd06c2c47dfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4390915c556409cb6b718e515d822c8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:49.997375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a6961bb4-8b06-4a05-8292-dd06c2c47dfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4390915c556409cb6b718e515d822c8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:50.021824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a6961bb4-8b06-4a05-8292-dd06c2c47dfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4390915c556409cb6b718e515d822c8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:50.075226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a6961bb4-8b06-4a05-8292-dd06c2c47dfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4390915c556409cb6b718e515d822c8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:50.115052 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1319/2705] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:52:49 2022] DELETE /identity/v3/users/f4390915c556409cb6b718e515d822c8 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:50.116888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3c720-9eb9-423e-8199-66c09fe953d5 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:50.119317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3c720-9eb9-423e-8199-66c09fe953d5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:50.119973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3c720-9eb9-423e-8199-66c09fe953d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:50.172093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a4b3c720-9eb9-423e-8199-66c09fe953d5 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:50.176230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a4b3c720-9eb9-423e-8199-66c09fe953d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:50.397694 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1387/2706] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:50.460175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6961bb4-8b06-4a05-8292-dd06c2c47dfe admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:50.462029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a6961bb4-8b06-4a05-8292-dd06c2c47dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:50.630254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:50.647294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6961bb4-8b06-4a05-8292-dd06c2c47dfe admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:50.661079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8124eb-6c4e-4010-8b1a-aa7ffd120305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:50.661671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8124eb-6c4e-4010-8b1a-aa7ffd120305 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:50.663817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8124eb-6c4e-4010-8b1a-aa7ffd120305 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:50.664104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8124eb-6c4e-4010-8b1a-aa7ffd120305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:50.696689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8124eb-6c4e-4010-8b1a-aa7ffd120305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:50.702434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8124eb-6c4e-4010-8b1a-aa7ffd120305 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:50.723580 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1320/2707] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:53.466739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3c720-9eb9-423e-8199-66c09fe953d5 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:53.469962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a4b3c720-9eb9-423e-8199-66c09fe953d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:53.485435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3c720-9eb9-423e-8199-66c09fe953d5 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:53.493969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c264c-2ddf-4c8a-9e88-58892609b454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:53.495258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c264c-2ddf-4c8a-9e88-58892609b454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:53.495723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c264c-2ddf-4c8a-9e88-58892609b454 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:53.496157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c264c-2ddf-4c8a-9e88-58892609b454 admin admin] PATH_INFO: `/v3/projects/f0e7756c4e2a408693ace3c9c3fc2680` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:53.498285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c264c-2ddf-4c8a-9e88-58892609b454 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0e7756c4e2a408693ace3c9c3fc2680)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:53.501968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c264c-2ddf-4c8a-9e88-58892609b454 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:53.564129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0b9c264c-2ddf-4c8a-9e88-58892609b454 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0e7756c4e2a408693ace3c9c3fc2680', 'request_id': 'req-0b9c264c-2ddf-4c8a-9e88-58892609b454'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:53.572109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0b9c264c-2ddf-4c8a-9e88-58892609b454 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0e7756c4e2a408693ace3c9c3fc2680', 'request_id': 'req-0b9c264c-2ddf-4c8a-9e88-58892609b454'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:53.578808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0b9c264c-2ddf-4c8a-9e88-58892609b454 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0e7756c4e2a408693ace3c9c3fc2680', 'request_id': 'req-0b9c264c-2ddf-4c8a-9e88-58892609b454'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:53.622651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0b9c264c-2ddf-4c8a-9e88-58892609b454 admin admin] The token cache is being invalidate because project f0e7756c4e2a408693ace3c9c3fc2680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:52:53.622882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0b9c264c-2ddf-4c8a-9e88-58892609b454 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:53.631904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0b9c264c-2ddf-4c8a-9e88-58892609b454 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:53.636693 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1388/2708] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:52:53 2022] DELETE /identity/v3/projects/f0e7756c4e2a408693ace3c9c3fc2680 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:53.640659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8124eb-6c4e-4010-8b1a-aa7ffd120305 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:53.651982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8124eb-6c4e-4010-8b1a-aa7ffd120305 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:53.788559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c264c-2ddf-4c8a-9e88-58892609b454 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:53.790547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0b9c264c-2ddf-4c8a-9e88-58892609b454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:53.811238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259bae6-14b2-420f-a4fd-b8d73e6a0614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:53.812114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259bae6-14b2-420f-a4fd-b8d73e6a0614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:53.812575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259bae6-14b2-420f-a4fd-b8d73e6a0614 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:53.813059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259bae6-14b2-420f-a4fd-b8d73e6a0614 admin admin] PATH_INFO: `/v3/users/d5d5c826ff90477bbffefd37bbbac4d0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:53.815696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259bae6-14b2-420f-a4fd-b8d73e6a0614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5d5c826ff90477bbffefd37bbbac4d0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:53.817612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259bae6-14b2-420f-a4fd-b8d73e6a0614 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:53.866792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:53.902775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f259bae6-14b2-420f-a4fd-b8d73e6a0614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5d5c826ff90477bbffefd37bbbac4d0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:53.903851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f259bae6-14b2-420f-a4fd-b8d73e6a0614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5d5c826ff90477bbffefd37bbbac4d0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:53.904561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f259bae6-14b2-420f-a4fd-b8d73e6a0614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5d5c826ff90477bbffefd37bbbac4d0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:53.915639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f259bae6-14b2-420f-a4fd-b8d73e6a0614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5d5c826ff90477bbffefd37bbbac4d0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:53.932831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c264c-2ddf-4c8a-9e88-58892609b454 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:53.947413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc20b3-ecb2-427d-96d9-be3a88b82006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:53.948025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f259bae6-14b2-420f-a4fd-b8d73e6a0614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5d5c826ff90477bbffefd37bbbac4d0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:53.948153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc20b3-ecb2-427d-96d9-be3a88b82006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:53.948338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc20b3-ecb2-427d-96d9-be3a88b82006 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:53.948607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc20b3-ecb2-427d-96d9-be3a88b82006 admin admin] PATH_INFO: `/v3/users/3245b8304f604cfc97da73b4dc884bd5` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:53.952004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc20b3-ecb2-427d-96d9-be3a88b82006 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3245b8304f604cfc97da73b4dc884bd5)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:53.956124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc20b3-ecb2-427d-96d9-be3a88b82006 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:53.982325 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1321/2709] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:52:53 2022] DELETE /identity/v3/users/d5d5c826ff90477bbffefd37bbbac4d0 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:54.100353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-10fc20b3-ecb2-427d-96d9-be3a88b82006 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3245b8304f604cfc97da73b4dc884bd5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:54.114025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-10fc20b3-ecb2-427d-96d9-be3a88b82006 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3245b8304f604cfc97da73b4dc884bd5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:54.133787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-10fc20b3-ecb2-427d-96d9-be3a88b82006 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3245b8304f604cfc97da73b4dc884bd5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:54.134843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-10fc20b3-ecb2-427d-96d9-be3a88b82006 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3245b8304f604cfc97da73b4dc884bd5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:54.163333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-10fc20b3-ecb2-427d-96d9-be3a88b82006 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3245b8304f604cfc97da73b4dc884bd5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:54.172665 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1389/2710] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:52:53 2022] DELETE /identity/v3/users/3245b8304f604cfc97da73b4dc884bd5 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:54.176708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efddec2-ff11-472e-91bb-1cc067a5da51 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:54.176998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efddec2-ff11-472e-91bb-1cc067a5da51 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:54.177276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efddec2-ff11-472e-91bb-1cc067a5da51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:54.202335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5efddec2-ff11-472e-91bb-1cc067a5da51 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:54.203719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5efddec2-ff11-472e-91bb-1cc067a5da51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:54.330298 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1322/2711] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:54.353142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc20b3-ecb2-427d-96d9-be3a88b82006 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:54.354725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-10fc20b3-ecb2-427d-96d9-be3a88b82006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:54.512334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc20b3-ecb2-427d-96d9-be3a88b82006 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:54.519558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839a44d-b2cd-4477-a5ba-34dc75524534 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:54.520432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839a44d-b2cd-4477-a5ba-34dc75524534 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:54.521376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839a44d-b2cd-4477-a5ba-34dc75524534 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:54.521930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839a44d-b2cd-4477-a5ba-34dc75524534 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:54.543201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efddec2-ff11-472e-91bb-1cc067a5da51 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:54.545638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5efddec2-ff11-472e-91bb-1cc067a5da51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:54.557961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d839a44d-b2cd-4477-a5ba-34dc75524534 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:54.565409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d839a44d-b2cd-4477-a5ba-34dc75524534 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:54.570547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efddec2-ff11-472e-91bb-1cc067a5da51 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:54.576918 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1390/2712] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:54.593760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dfa6a0-93c7-45cc-b630-c800434bca1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:54.594410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dfa6a0-93c7-45cc-b630-c800434bca1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:54.594731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dfa6a0-93c7-45cc-b630-c800434bca1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:54.595043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dfa6a0-93c7-45cc-b630-c800434bca1b admin admin] PATH_INFO: `/v3/projects/09e4b27d45544465bcb556504c44cfbc` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:54.599822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dfa6a0-93c7-45cc-b630-c800434bca1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=09e4b27d45544465bcb556504c44cfbc)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:54.602138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dfa6a0-93c7-45cc-b630-c800434bca1b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:54.618931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:54.705295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d8dfa6a0-93c7-45cc-b630-c800434bca1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09e4b27d45544465bcb556504c44cfbc', 'request_id': 'req-d8dfa6a0-93c7-45cc-b630-c800434bca1b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:54.724877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d8dfa6a0-93c7-45cc-b630-c800434bca1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09e4b27d45544465bcb556504c44cfbc', 'request_id': 'req-d8dfa6a0-93c7-45cc-b630-c800434bca1b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:54.734316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d8dfa6a0-93c7-45cc-b630-c800434bca1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09e4b27d45544465bcb556504c44cfbc', 'request_id': 'req-d8dfa6a0-93c7-45cc-b630-c800434bca1b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:54.749301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d8dfa6a0-93c7-45cc-b630-c800434bca1b admin admin] The token cache is being invalidate because project 09e4b27d45544465bcb556504c44cfbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:52:54.750523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d8dfa6a0-93c7-45cc-b630-c800434bca1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:54.751110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d8dfa6a0-93c7-45cc-b630-c800434bca1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:54.758782 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1323/2713] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:52:54 2022] DELETE /identity/v3/projects/09e4b27d45544465bcb556504c44cfbc => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:52:54.760663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46452ff3-5771-41cf-ad10-013c14ceeb8f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:54.761355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46452ff3-5771-41cf-ad10-013c14ceeb8f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:54.761355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46452ff3-5771-41cf-ad10-013c14ceeb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:54.828297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-46452ff3-5771-41cf-ad10-013c14ceeb8f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:54.834372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-46452ff3-5771-41cf-ad10-013c14ceeb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:54.885994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dfa6a0-93c7-45cc-b630-c800434bca1b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:54.888487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d8dfa6a0-93c7-45cc-b630-c800434bca1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:54.935308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:54.966542 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1391/2714] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:52:54.973748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f1391-a60a-44f8-8da0-3096ae507a41 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:54.974546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f1391-a60a-44f8-8da0-3096ae507a41 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:54.974546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f1391-a60a-44f8-8da0-3096ae507a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:55.016442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a59f1391-a60a-44f8-8da0-3096ae507a41 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:55.018598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a59f1391-a60a-44f8-8da0-3096ae507a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:55.030397 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1392/2715] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:52:55.031513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f1391-a60a-44f8-8da0-3096ae507a41 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:55.041690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f1391-a60a-44f8-8da0-3096ae507a41 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:55.054903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64851787-9a98-4ada-a611-6cbcf840ce19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:55.055670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64851787-9a98-4ada-a611-6cbcf840ce19 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:55.055909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64851787-9a98-4ada-a611-6cbcf840ce19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:55.056088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64851787-9a98-4ada-a611-6cbcf840ce19 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:55.056730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64851787-9a98-4ada-a611-6cbcf840ce19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:52:55.057043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64851787-9a98-4ada-a611-6cbcf840ce19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:55.061682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64851787-9a98-4ada-a611-6cbcf840ce19 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:55.065997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dfa6a0-93c7-45cc-b630-c800434bca1b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:55.077098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca745fa-1cf6-4453-80f8-48a91b9541f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:55.077302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca745fa-1cf6-4453-80f8-48a91b9541f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:55.077679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca745fa-1cf6-4453-80f8-48a91b9541f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:55.077815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca745fa-1cf6-4453-80f8-48a91b9541f8 admin admin] PATH_INFO: `/v3/projects/a1c660e5e0f14f16b4b2b43e89609bb2` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:55.080127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca745fa-1cf6-4453-80f8-48a91b9541f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1c660e5e0f14f16b4b2b43e89609bb2)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:55.087397 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1393/2716] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:52:55.087977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64851787-9a98-4ada-a611-6cbcf840ce19 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:55.088464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca745fa-1cf6-4453-80f8-48a91b9541f8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:55.107408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64851787-9a98-4ada-a611-6cbcf840ce19 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:55.113740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb939a-b6e1-4190-8041-c8f1f8d0c57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:55.114517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb939a-b6e1-4190-8041-c8f1f8d0c57e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:55.114867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb939a-b6e1-4190-8041-c8f1f8d0c57e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:55.115440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb939a-b6e1-4190-8041-c8f1f8d0c57e admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:55.116209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb939a-b6e1-4190-8041-c8f1f8d0c57e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:55.118238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb939a-b6e1-4190-8041-c8f1f8d0c57e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:55.184323 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1394/2717] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:52:55 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:55.185278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb939a-b6e1-4190-8041-c8f1f8d0c57e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:55.191655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cca745fa-1cf6-4453-80f8-48a91b9541f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1c660e5e0f14f16b4b2b43e89609bb2', 'request_id': 'req-cca745fa-1cf6-4453-80f8-48a91b9541f8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:55.200994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cca745fa-1cf6-4453-80f8-48a91b9541f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1c660e5e0f14f16b4b2b43e89609bb2', 'request_id': 'req-cca745fa-1cf6-4453-80f8-48a91b9541f8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:55.213031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cca745fa-1cf6-4453-80f8-48a91b9541f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1c660e5e0f14f16b4b2b43e89609bb2', 'request_id': 'req-cca745fa-1cf6-4453-80f8-48a91b9541f8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:55.240795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cca745fa-1cf6-4453-80f8-48a91b9541f8 admin admin] The token cache is being invalidate because project a1c660e5e0f14f16b4b2b43e89609bb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:52:55.241249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cca745fa-1cf6-4453-80f8-48a91b9541f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:55.242731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cca745fa-1cf6-4453-80f8-48a91b9541f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:55.249384 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1324/2718] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:52:54 2022] DELETE /identity/v3/projects/a1c660e5e0f14f16b4b2b43e89609bb2 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:52:55.256320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca745fa-1cf6-4453-80f8-48a91b9541f8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:55.258241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cca745fa-1cf6-4453-80f8-48a91b9541f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:55.309107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:55.325578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb939a-b6e1-4190-8041-c8f1f8d0c57e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:55.332393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf55ac0-97f9-4cc4-9e05-6d5fbf782e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:55.333394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf55ac0-97f9-4cc4-9e05-6d5fbf782e76 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:55.333802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf55ac0-97f9-4cc4-9e05-6d5fbf782e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:55.334184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf55ac0-97f9-4cc4-9e05-6d5fbf782e76 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:55.335053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf55ac0-97f9-4cc4-9e05-6d5fbf782e76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:55.342276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf55ac0-97f9-4cc4-9e05-6d5fbf782e76 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:55.373536 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1395/2719] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:52:55 2022] POST /identity/v3/users => generated 294 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:55.374882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf55ac0-97f9-4cc4-9e05-6d5fbf782e76 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:55.376201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca745fa-1cf6-4453-80f8-48a91b9541f8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:55.376735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-faf55ac0-97f9-4cc4-9e05-6d5fbf782e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:55.393821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c791e-2cdb-45c2-9aef-9bc2ab1227ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:55.394513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c791e-2cdb-45c2-9aef-9bc2ab1227ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:55.394897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c791e-2cdb-45c2-9aef-9bc2ab1227ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:55.395210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c791e-2cdb-45c2-9aef-9bc2ab1227ac admin admin] PATH_INFO: `/v3/users/e31b54cd54d54f5ebfb064edac6dfe3b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:55.396641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c791e-2cdb-45c2-9aef-9bc2ab1227ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=e31b54cd54d54f5ebfb064edac6dfe3b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:55.400537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c791e-2cdb-45c2-9aef-9bc2ab1227ac admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:55.411802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf55ac0-97f9-4cc4-9e05-6d5fbf782e76 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:55.421307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641195c2-353a-4a5c-9ea6-3a15e80c9da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:55.422352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641195c2-353a-4a5c-9ea6-3a15e80c9da4 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:55.423915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641195c2-353a-4a5c-9ea6-3a15e80c9da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:55.424339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641195c2-353a-4a5c-9ea6-3a15e80c9da4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:55.425910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641195c2-353a-4a5c-9ea6-3a15e80c9da4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:55.427718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641195c2-353a-4a5c-9ea6-3a15e80c9da4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:55.444864 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1396/2720] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:52:55.448091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641195c2-353a-4a5c-9ea6-3a15e80c9da4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:55.463983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641195c2-353a-4a5c-9ea6-3a15e80c9da4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:55.476204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b05cd-9694-4e88-a8eb-8383552029ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:55.476865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b05cd-9694-4e88-a8eb-8383552029ea admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:55.477117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b05cd-9694-4e88-a8eb-8383552029ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:55.477391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b05cd-9694-4e88-a8eb-8383552029ea admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:55.478062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b05cd-9694-4e88-a8eb-8383552029ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:55.479530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-361c791e-2cdb-45c2-9aef-9bc2ab1227ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e31b54cd54d54f5ebfb064edac6dfe3b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:55.479848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b05cd-9694-4e88-a8eb-8383552029ea admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:55.480810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-361c791e-2cdb-45c2-9aef-9bc2ab1227ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e31b54cd54d54f5ebfb064edac6dfe3b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:55.481860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-361c791e-2cdb-45c2-9aef-9bc2ab1227ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e31b54cd54d54f5ebfb064edac6dfe3b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:55.496693 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1397/2721] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:52:55.496874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-361c791e-2cdb-45c2-9aef-9bc2ab1227ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e31b54cd54d54f5ebfb064edac6dfe3b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:55.498196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b05cd-9694-4e88-a8eb-8383552029ea admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:55.500501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5f4b05cd-9694-4e88-a8eb-8383552029ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:55.515582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-361c791e-2cdb-45c2-9aef-9bc2ab1227ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e31b54cd54d54f5ebfb064edac6dfe3b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:55.516318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b05cd-9694-4e88-a8eb-8383552029ea admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:55.523499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce64226-2beb-4950-8f1f-23cf4ca0d594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:55.524333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce64226-2beb-4950-8f1f-23cf4ca0d594 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:55.524724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce64226-2beb-4950-8f1f-23cf4ca0d594 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:55.525014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce64226-2beb-4950-8f1f-23cf4ca0d594 admin admin] PATH_INFO: `/v3/projects/9ddc17f805ab4817923b61c801bbbed3/users/2fb4ea88f36345d89b985350e0be98b7/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:55.538564 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1325/2722] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:52:55 2022] DELETE /identity/v3/users/e31b54cd54d54f5ebfb064edac6dfe3b => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:55.546158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce64226-2beb-4950-8f1f-23cf4ca0d594 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ddc17f805ab4817923b61c801bbbed3, user_id=2fb4ea88f36345d89b985350e0be98b7, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:55.550241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce64226-2beb-4950-8f1f-23cf4ca0d594 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:55.570598 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1398/2723] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:52:55 2022] PUT /identity/v3/projects/9ddc17f805ab4817923b61c801bbbed3/users/2fb4ea88f36345d89b985350e0be98b7/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:55.576182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e89e1-365d-4378-9de9-53bb86bfa571 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:55.576554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e89e1-365d-4378-9de9-53bb86bfa571 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:55.576899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e89e1-365d-4378-9de9-53bb86bfa571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:55.633395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-cc1e89e1-365d-4378-9de9-53bb86bfa571 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:55.638226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cc1e89e1-365d-4378-9de9-53bb86bfa571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:55.781741 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1326/2724] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:52:55.808950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce64226-2beb-4950-8f1f-23cf4ca0d594 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:55.811484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3ce64226-2beb-4950-8f1f-23cf4ca0d594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:55.904442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:55.916577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce64226-2beb-4950-8f1f-23cf4ca0d594 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:55.923595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1dc350-31d8-4714-b681-254ea4da79d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:55.924107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dc350-31d8-4714-b681-254ea4da79d6 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:55.924399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dc350-31d8-4714-b681-254ea4da79d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:55.924675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dc350-31d8-4714-b681-254ea4da79d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:55.939289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1dc350-31d8-4714-b681-254ea4da79d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:55.940938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1dc350-31d8-4714-b681-254ea4da79d6 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:55.958886 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1399/2725] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:56.139619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e89e1-365d-4378-9de9-53bb86bfa571 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:56.141429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cc1e89e1-365d-4378-9de9-53bb86bfa571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:56.163944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e89e1-365d-4378-9de9-53bb86bfa571 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:56.178769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd912c-da9f-491f-98a3-51a7394651e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:56.179436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd912c-da9f-491f-98a3-51a7394651e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:56.179719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd912c-da9f-491f-98a3-51a7394651e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:56.179996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd912c-da9f-491f-98a3-51a7394651e1 admin admin] PATH_INFO: `/v3/projects/9d8830defffa4b8dbacf6bbbd7ab4205` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:56.184523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd912c-da9f-491f-98a3-51a7394651e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d8830defffa4b8dbacf6bbbd7ab4205)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:56.186827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd912c-da9f-491f-98a3-51a7394651e1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:56.300429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:56.323179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-39cd912c-da9f-491f-98a3-51a7394651e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d8830defffa4b8dbacf6bbbd7ab4205', 'request_id': 'req-39cd912c-da9f-491f-98a3-51a7394651e1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:56.333725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-39cd912c-da9f-491f-98a3-51a7394651e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d8830defffa4b8dbacf6bbbd7ab4205', 'request_id': 'req-39cd912c-da9f-491f-98a3-51a7394651e1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:56.342180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-39cd912c-da9f-491f-98a3-51a7394651e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d8830defffa4b8dbacf6bbbd7ab4205', 'request_id': 'req-39cd912c-da9f-491f-98a3-51a7394651e1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:56.364524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-39cd912c-da9f-491f-98a3-51a7394651e1 admin admin] The token cache is being invalidate because project 9d8830defffa4b8dbacf6bbbd7ab4205 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:52:56.364822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-39cd912c-da9f-491f-98a3-51a7394651e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:56.365982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-39cd912c-da9f-491f-98a3-51a7394651e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:56.372371 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1327/2726] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:52:56 2022] DELETE /identity/v3/projects/9d8830defffa4b8dbacf6bbbd7ab4205 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:56.374716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9f17d-84e2-4e57-b9b1-c502dfb266ed None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:56.375009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9f17d-84e2-4e57-b9b1-c502dfb266ed None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:56.375311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9f17d-84e2-4e57-b9b1-c502dfb266ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:56.414972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-26f9f17d-84e2-4e57-b9b1-c502dfb266ed None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:56.416270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-26f9f17d-84e2-4e57-b9b1-c502dfb266ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:56.621250 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1400/2727] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:56.629394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10800774-207a-4dec-848f-3fc7c872ad4c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:56.629687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10800774-207a-4dec-848f-3fc7c872ad4c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:56.630266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10800774-207a-4dec-848f-3fc7c872ad4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:56.660382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-10800774-207a-4dec-848f-3fc7c872ad4c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:56.662390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-10800774-207a-4dec-848f-3fc7c872ad4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:56.676286 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1328/2728] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:56.676858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9f17d-84e2-4e57-b9b1-c502dfb266ed None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:56.698293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9f17d-84e2-4e57-b9b1-c502dfb266ed None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:56.715164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055e683-37a6-4679-aa16-81407592a52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:56.717993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055e683-37a6-4679-aa16-81407592a52f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:56.718979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055e683-37a6-4679-aa16-81407592a52f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:56.719768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055e683-37a6-4679-aa16-81407592a52f admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:56.722142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055e683-37a6-4679-aa16-81407592a52f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:52:56.722601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055e683-37a6-4679-aa16-81407592a52f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:56.727221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055e683-37a6-4679-aa16-81407592a52f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:56.765187 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1401/2729] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:52:56.771402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10800774-207a-4dec-848f-3fc7c872ad4c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:56.803914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10800774-207a-4dec-848f-3fc7c872ad4c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:56.817717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b3148-3837-42ec-ac3d-97cea703e120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:56.819064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b3148-3837-42ec-ac3d-97cea703e120 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:56.819689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b3148-3837-42ec-ac3d-97cea703e120 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:56.820247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b3148-3837-42ec-ac3d-97cea703e120 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:56.821164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b3148-3837-42ec-ac3d-97cea703e120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:56.824647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b3148-3837-42ec-ac3d-97cea703e120 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:56.867469 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1329/2730] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:52:56 2022] POST /identity/v3/projects => generated 395 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:52:56.871783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055e683-37a6-4679-aa16-81407592a52f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:56.889178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055e683-37a6-4679-aa16-81407592a52f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:57.053281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358424c-963f-4270-ac4d-267bafd1088f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:57.053998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358424c-963f-4270-ac4d-267bafd1088f admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:57.054187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358424c-963f-4270-ac4d-267bafd1088f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:57.054445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358424c-963f-4270-ac4d-267bafd1088f admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:57.059606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358424c-963f-4270-ac4d-267bafd1088f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:57.061674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358424c-963f-4270-ac4d-267bafd1088f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:57.102101 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1402/2731] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:52:56 2022] POST /identity/v3/users => generated 298 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:52:57.107030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b3148-3837-42ec-ac3d-97cea703e120 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:57.123541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b3148-3837-42ec-ac3d-97cea703e120 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:57.132520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd5c15-7f97-4ccf-95ac-10e087bfd5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:57.133770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd5c15-7f97-4ccf-95ac-10e087bfd5ea admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:57.134419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd5c15-7f97-4ccf-95ac-10e087bfd5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:57.134731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd5c15-7f97-4ccf-95ac-10e087bfd5ea admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:57.135826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd5c15-7f97-4ccf-95ac-10e087bfd5ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:57.137291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd5c15-7f97-4ccf-95ac-10e087bfd5ea admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:57.156596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358424c-963f-4270-ac4d-267bafd1088f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:57.157780 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1330/2732] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:52:57.169369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358424c-963f-4270-ac4d-267bafd1088f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:57.175371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e63a10-acca-4883-8db4-52cd6077d9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:57.176151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e63a10-acca-4883-8db4-52cd6077d9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:57.176367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e63a10-acca-4883-8db4-52cd6077d9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:57.176646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e63a10-acca-4883-8db4-52cd6077d9f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:57.177526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e63a10-acca-4883-8db4-52cd6077d9f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:57.179621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e63a10-acca-4883-8db4-52cd6077d9f2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:57.199354 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1403/2733] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:52:57.201298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd5c15-7f97-4ccf-95ac-10e087bfd5ea admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:57.213301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd5c15-7f97-4ccf-95ac-10e087bfd5ea admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:57.220835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ceedcc-1bef-45ef-8ee5-239937e03f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:57.221763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ceedcc-1bef-45ef-8ee5-239937e03f5c admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:57.222196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ceedcc-1bef-45ef-8ee5-239937e03f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:57.222539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ceedcc-1bef-45ef-8ee5-239937e03f5c admin admin] PATH_INFO: `/v3/projects/1a2694c8af60455aa9db5997c53adbdc/users/f4de9e6274c041da9c366ea00c0bf769/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:57.267152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ceedcc-1bef-45ef-8ee5-239937e03f5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a2694c8af60455aa9db5997c53adbdc, user_id=f4de9e6274c041da9c366ea00c0bf769, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:57.271990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ceedcc-1bef-45ef-8ee5-239937e03f5c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:57.302004 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1331/2734] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:52:57 2022] PUT /identity/v3/projects/1a2694c8af60455aa9db5997c53adbdc/users/f4de9e6274c041da9c366ea00c0bf769/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:52:57.303757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb268a-3a71-46e5-9343-ed5a4dcc77f1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:57.304050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb268a-3a71-46e5-9343-ed5a4dcc77f1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:57.304325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb268a-3a71-46e5-9343-ed5a4dcc77f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:57.342635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-20eb268a-3a71-46e5-9343-ed5a4dcc77f1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:57.346850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-20eb268a-3a71-46e5-9343-ed5a4dcc77f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:57.497774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:57.520710 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1404/2735] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:57.587905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ceedcc-1bef-45ef-8ee5-239937e03f5c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:57.590032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-79ceedcc-1bef-45ef-8ee5-239937e03f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:57.707641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ceedcc-1bef-45ef-8ee5-239937e03f5c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:57.713672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4638dc80-d570-4f85-ac7b-f24a537a38b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:57.715245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638dc80-d570-4f85-ac7b-f24a537a38b2 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:57.716122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638dc80-d570-4f85-ac7b-f24a537a38b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:57.716606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638dc80-d570-4f85-ac7b-f24a537a38b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:57.733767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4638dc80-d570-4f85-ac7b-f24a537a38b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:57.736938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4638dc80-d570-4f85-ac7b-f24a537a38b2 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:57.778286 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1332/2736] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:52:58.978405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb268a-3a71-46e5-9343-ed5a4dcc77f1 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:58.980647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-20eb268a-3a71-46e5-9343-ed5a4dcc77f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:58.999601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb268a-3a71-46e5-9343-ed5a4dcc77f1 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:59.012896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354a253-1162-4871-8e9c-e26ff8323c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:59.013811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354a253-1162-4871-8e9c-e26ff8323c36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:59.014306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354a253-1162-4871-8e9c-e26ff8323c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:59.014774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354a253-1162-4871-8e9c-e26ff8323c36 admin admin] PATH_INFO: `/v3/users/04add76a65d74165b65189a5fd1f47ed` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:59.018585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7354a253-1162-4871-8e9c-e26ff8323c36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04add76a65d74165b65189a5fd1f47ed)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:59.021416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7354a253-1162-4871-8e9c-e26ff8323c36 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:59.099604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7354a253-1162-4871-8e9c-e26ff8323c36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04add76a65d74165b65189a5fd1f47ed'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:59.117806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7354a253-1162-4871-8e9c-e26ff8323c36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04add76a65d74165b65189a5fd1f47ed'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:59.146115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7354a253-1162-4871-8e9c-e26ff8323c36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04add76a65d74165b65189a5fd1f47ed'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:59.147655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7354a253-1162-4871-8e9c-e26ff8323c36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04add76a65d74165b65189a5fd1f47ed'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:59.158820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7354a253-1162-4871-8e9c-e26ff8323c36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04add76a65d74165b65189a5fd1f47ed'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:52:59.171313 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1405/2737] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:52:58 2022] DELETE /identity/v3/users/04add76a65d74165b65189a5fd1f47ed => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:52:59.173321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c1eda-f32d-4066-8205-a5e627c32d04 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:59.173768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c1eda-f32d-4066-8205-a5e627c32d04 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:59.173867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c1eda-f32d-4066-8205-a5e627c32d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:59.205993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d59c1eda-f32d-4066-8205-a5e627c32d04 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:52:59.209496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d59c1eda-f32d-4066-8205-a5e627c32d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:59.321998 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1333/2738] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:52:59.344714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354a253-1162-4871-8e9c-e26ff8323c36 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:52:59.347026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7354a253-1162-4871-8e9c-e26ff8323c36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:52:59.469003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354a253-1162-4871-8e9c-e26ff8323c36 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:52:59.478977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cd849-5628-4d6f-ac80-6573ec129f04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:52:59.480009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cd849-5628-4d6f-ac80-6573ec129f04 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:52:59.480565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cd849-5628-4d6f-ac80-6573ec129f04 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:52:59.481050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cd849-5628-4d6f-ac80-6573ec129f04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:52:59.504013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cd849-5628-4d6f-ac80-6573ec129f04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:52:59.506321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cd849-5628-4d6f-ac80-6573ec129f04 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:52:59.521685 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1406/2739] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:52:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:53:00.144581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20822dff-5356-46aa-a389-70d5d4b23fa2 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:00.144939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20822dff-5356-46aa-a389-70d5d4b23fa2 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:00.145160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20822dff-5356-46aa-a389-70d5d4b23fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:00.159316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:00.181489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-20822dff-5356-46aa-a389-70d5d4b23fa2 None None] MFA Rules not processed for user `d270f04a32ac44bc88fc57f8e765111e`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:00.182748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-20822dff-5356-46aa-a389-70d5d4b23fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:00.303910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24f864-ff48-4420-a8f5-4325a5a02b72 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:00.305363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24f864-ff48-4420-a8f5-4325a5a02b72 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:00.305795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24f864-ff48-4420-a8f5-4325a5a02b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:00.310656 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1334/2740] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:00.347046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5c24f864-ff48-4420-a8f5-4325a5a02b72 None None] MFA Rules not processed for user `d270f04a32ac44bc88fc57f8e765111e`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:00.348435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5c24f864-ff48-4420-a8f5-4325a5a02b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:00.374112 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1407/2741] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:00.643579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20822dff-5356-46aa-a389-70d5d4b23fa2 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:00.645124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-20822dff-5356-46aa-a389-70d5d4b23fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:00.671441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20822dff-5356-46aa-a389-70d5d4b23fa2 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:00.683601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529d2b9-ad98-4458-aaf7-09bb6e78e080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:00.685595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529d2b9-ad98-4458-aaf7-09bb6e78e080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:00.685945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529d2b9-ad98-4458-aaf7-09bb6e78e080 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:00.686211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529d2b9-ad98-4458-aaf7-09bb6e78e080 admin admin] PATH_INFO: `/v3/projects/963f525498ef4ebdb9db327df62bd68a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:00.688910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529d2b9-ad98-4458-aaf7-09bb6e78e080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=963f525498ef4ebdb9db327df62bd68a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:00.695350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529d2b9-ad98-4458-aaf7-09bb6e78e080 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:00.798557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f529d2b9-ad98-4458-aaf7-09bb6e78e080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '963f525498ef4ebdb9db327df62bd68a', 'request_id': 'req-f529d2b9-ad98-4458-aaf7-09bb6e78e080'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:00.809917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f529d2b9-ad98-4458-aaf7-09bb6e78e080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '963f525498ef4ebdb9db327df62bd68a', 'request_id': 'req-f529d2b9-ad98-4458-aaf7-09bb6e78e080'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:00.826727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f529d2b9-ad98-4458-aaf7-09bb6e78e080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '963f525498ef4ebdb9db327df62bd68a', 'request_id': 'req-f529d2b9-ad98-4458-aaf7-09bb6e78e080'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:00.867050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f529d2b9-ad98-4458-aaf7-09bb6e78e080 admin admin] The token cache is being invalidate because project 963f525498ef4ebdb9db327df62bd68a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:53:00.869051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f529d2b9-ad98-4458-aaf7-09bb6e78e080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:00.872096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f529d2b9-ad98-4458-aaf7-09bb6e78e080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:00.881195 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1335/2742] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:53:00 2022] DELETE /identity/v3/projects/963f525498ef4ebdb9db327df62bd68a => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:53:00.885064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24f864-ff48-4420-a8f5-4325a5a02b72 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:00.904370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24f864-ff48-4420-a8f5-4325a5a02b72 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:01.023333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529d2b9-ad98-4458-aaf7-09bb6e78e080 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:01.024936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f529d2b9-ad98-4458-aaf7-09bb6e78e080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:01.113210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1b6f5-d075-4ccd-aa22-78efaae70a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:01.113866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1b6f5-d075-4ccd-aa22-78efaae70a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:01.114148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1b6f5-d075-4ccd-aa22-78efaae70a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:01.114424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1b6f5-d075-4ccd-aa22-78efaae70a75 admin admin] PATH_INFO: `/v3/users/dfd53627e013481fb9fb43b94d43db47` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:01.118348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c1b6f5-d075-4ccd-aa22-78efaae70a75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfd53627e013481fb9fb43b94d43db47)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:01.122659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c1b6f5-d075-4ccd-aa22-78efaae70a75 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:01.140381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:01.233099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529d2b9-ad98-4458-aaf7-09bb6e78e080 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:01.239603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a5c1b6f5-d075-4ccd-aa22-78efaae70a75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfd53627e013481fb9fb43b94d43db47'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:01.247114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f0e2e-e147-48c1-939a-2b4798434e45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:01.247955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f0e2e-e147-48c1-939a-2b4798434e45 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:01.248275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a5c1b6f5-d075-4ccd-aa22-78efaae70a75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfd53627e013481fb9fb43b94d43db47'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:01.248565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f0e2e-e147-48c1-939a-2b4798434e45 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:01.248988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f0e2e-e147-48c1-939a-2b4798434e45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:01.253999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-828f0e2e-e147-48c1-939a-2b4798434e45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:01.287169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a5c1b6f5-d075-4ccd-aa22-78efaae70a75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfd53627e013481fb9fb43b94d43db47'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:01.288481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a5c1b6f5-d075-4ccd-aa22-78efaae70a75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfd53627e013481fb9fb43b94d43db47'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:01.291572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:01.304574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a5c1b6f5-d075-4ccd-aa22-78efaae70a75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfd53627e013481fb9fb43b94d43db47'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:01.327912 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1408/2743] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:53:00 2022] DELETE /identity/v3/users/dfd53627e013481fb9fb43b94d43db47 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:53:01.403395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f0e2e-e147-48c1-939a-2b4798434e45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:01.405271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f0e2e-e147-48c1-939a-2b4798434e45 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:01.419093 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1336/2744] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:53:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:53:02.680221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1b6f5-d075-4ccd-aa22-78efaae70a75 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:02.681726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a5c1b6f5-d075-4ccd-aa22-78efaae70a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:02.782610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1b6f5-d075-4ccd-aa22-78efaae70a75 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:02.807135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39e639-a66e-4b46-82d5-58ef2993782b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:02.808145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39e639-a66e-4b46-82d5-58ef2993782b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:02.808712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39e639-a66e-4b46-82d5-58ef2993782b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:02.810236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39e639-a66e-4b46-82d5-58ef2993782b admin admin] PATH_INFO: `/v3/projects/1ab469e2d1a04a368860f8a96492940f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:02.811681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39e639-a66e-4b46-82d5-58ef2993782b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ab469e2d1a04a368860f8a96492940f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:02.813820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39e639-a66e-4b46-82d5-58ef2993782b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:02.884521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bf39e639-a66e-4b46-82d5-58ef2993782b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ab469e2d1a04a368860f8a96492940f', 'request_id': 'req-bf39e639-a66e-4b46-82d5-58ef2993782b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:02.896295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bf39e639-a66e-4b46-82d5-58ef2993782b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ab469e2d1a04a368860f8a96492940f', 'request_id': 'req-bf39e639-a66e-4b46-82d5-58ef2993782b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:02.905454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bf39e639-a66e-4b46-82d5-58ef2993782b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ab469e2d1a04a368860f8a96492940f', 'request_id': 'req-bf39e639-a66e-4b46-82d5-58ef2993782b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:02.937688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bf39e639-a66e-4b46-82d5-58ef2993782b admin admin] The token cache is being invalidate because project 1ab469e2d1a04a368860f8a96492940f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:53:02.938316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bf39e639-a66e-4b46-82d5-58ef2993782b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:02.939285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bf39e639-a66e-4b46-82d5-58ef2993782b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:02.944192 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1409/2745] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:53:02 2022] DELETE /identity/v3/projects/1ab469e2d1a04a368860f8a96492940f => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:53:02.948722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0d571-c98c-4dff-bf60-77c1c427b5a9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:02.949293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0d571-c98c-4dff-bf60-77c1c427b5a9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:02.949293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0d571-c98c-4dff-bf60-77c1c427b5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:02.991825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6ef0d571-c98c-4dff-bf60-77c1c427b5a9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:02.994215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6ef0d571-c98c-4dff-bf60-77c1c427b5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:03.159715 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1337/2746] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:03.161416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce86e32-1eb0-4ec2-bb5e-86eb508dfd78 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:03.161698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce86e32-1eb0-4ec2-bb5e-86eb508dfd78 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:03.161972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce86e32-1eb0-4ec2-bb5e-86eb508dfd78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:03.210732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3ce86e32-1eb0-4ec2-bb5e-86eb508dfd78 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:03.217087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3ce86e32-1eb0-4ec2-bb5e-86eb508dfd78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:03.256205 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1410/2747] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:03.260890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0d571-c98c-4dff-bf60-77c1c427b5a9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:03.274061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0d571-c98c-4dff-bf60-77c1c427b5a9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:03.295136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bdf6b-9070-43d2-98cf-dcbf4a48c5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:03.295992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bdf6b-9070-43d2-98cf-dcbf4a48c5e7 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:03.296368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bdf6b-9070-43d2-98cf-dcbf4a48c5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:03.296812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bdf6b-9070-43d2-98cf-dcbf4a48c5e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:03.298228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bdf6b-9070-43d2-98cf-dcbf4a48c5e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:53:03.298693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bdf6b-9070-43d2-98cf-dcbf4a48c5e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:03.300624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bdf6b-9070-43d2-98cf-dcbf4a48c5e7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:03.334713 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1338/2748] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:53:03.335943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce86e32-1eb0-4ec2-bb5e-86eb508dfd78 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:03.348319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce86e32-1eb0-4ec2-bb5e-86eb508dfd78 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:03.373674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3edefe0-4e51-42c3-a4c9-0a4341002625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:03.374570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edefe0-4e51-42c3-a4c9-0a4341002625 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:03.374570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edefe0-4e51-42c3-a4c9-0a4341002625 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:03.374853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edefe0-4e51-42c3-a4c9-0a4341002625 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:03.376501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3edefe0-4e51-42c3-a4c9-0a4341002625 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:03.380884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3edefe0-4e51-42c3-a4c9-0a4341002625 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:03.395630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:03.487251 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1411/2749] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:53:03 2022] POST /identity/v3/projects => generated 361 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:53:03.488708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bdf6b-9070-43d2-98cf-dcbf4a48c5e7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:03.507921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bdf6b-9070-43d2-98cf-dcbf4a48c5e7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:03.637323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539609b-acf3-4e35-9ed3-f5fdc5a5ad19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:03.638682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539609b-acf3-4e35-9ed3-f5fdc5a5ad19 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:03.643369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539609b-acf3-4e35-9ed3-f5fdc5a5ad19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:03.643929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539609b-acf3-4e35-9ed3-f5fdc5a5ad19 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:03.644764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539609b-acf3-4e35-9ed3-f5fdc5a5ad19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:03.646901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539609b-acf3-4e35-9ed3-f5fdc5a5ad19 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:03.697032 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1339/2750] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:53:03 2022] POST /identity/v3/users => generated 281 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:53:03.698426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3edefe0-4e51-42c3-a4c9-0a4341002625 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:03.713962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3edefe0-4e51-42c3-a4c9-0a4341002625 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:03.727298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca029a06-fa5a-4ea0-a148-52bfece23c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:03.728041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca029a06-fa5a-4ea0-a148-52bfece23c68 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:03.728435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca029a06-fa5a-4ea0-a148-52bfece23c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:03.728819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca029a06-fa5a-4ea0-a148-52bfece23c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:03.729600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca029a06-fa5a-4ea0-a148-52bfece23c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:03.734935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca029a06-fa5a-4ea0-a148-52bfece23c68 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:03.762934 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1412/2751] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:53:03.764479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539609b-acf3-4e35-9ed3-f5fdc5a5ad19 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:03.780308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539609b-acf3-4e35-9ed3-f5fdc5a5ad19 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:03.788136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef91b2b-5ff6-4016-9378-09408f7badea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:03.789105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef91b2b-5ff6-4016-9378-09408f7badea admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:03.789429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef91b2b-5ff6-4016-9378-09408f7badea admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:03.789808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef91b2b-5ff6-4016-9378-09408f7badea admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:03.790827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef91b2b-5ff6-4016-9378-09408f7badea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:03.793157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef91b2b-5ff6-4016-9378-09408f7badea admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:03.827404 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1340/2752] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:53:03.829398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca029a06-fa5a-4ea0-a148-52bfece23c68 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:03.848967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca029a06-fa5a-4ea0-a148-52bfece23c68 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:03.854922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2d1fa-9d21-4533-8417-0840f105e82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:03.861518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2d1fa-9d21-4533-8417-0840f105e82a admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:03.861929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2d1fa-9d21-4533-8417-0840f105e82a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:03.862318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2d1fa-9d21-4533-8417-0840f105e82a admin admin] PATH_INFO: `/v3/projects/0e16bd1c74884610adc40a8a8153f0a0/users/3538ba50b6104ecea13b0efb7b4b39f6/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:03.879940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2d1fa-9d21-4533-8417-0840f105e82a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e16bd1c74884610adc40a8a8153f0a0, user_id=3538ba50b6104ecea13b0efb7b4b39f6, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:03.894617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2d1fa-9d21-4533-8417-0840f105e82a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:03.932048 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1413/2753] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:53:03 2022] PUT /identity/v3/projects/0e16bd1c74884610adc40a8a8153f0a0/users/3538ba50b6104ecea13b0efb7b4b39f6/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:53:03.937269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6b830-e9c9-4a53-ac54-818ddd3fb5e4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:03.937559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6b830-e9c9-4a53-ac54-818ddd3fb5e4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:03.937974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6b830-e9c9-4a53-ac54-818ddd3fb5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:03.991652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ffe6b830-e9c9-4a53-ac54-818ddd3fb5e4 None None] MFA Rules not processed for user `3538ba50b6104ecea13b0efb7b4b39f6`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:03.994990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ffe6b830-e9c9-4a53-ac54-818ddd3fb5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:04.008404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:04.112574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2d1fa-9d21-4533-8417-0840f105e82a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:04.112723 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1341/2754] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:04.254061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2d1fa-9d21-4533-8417-0840f105e82a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:04.261766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca266292-a2c3-4fa3-99eb-64423fae769e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:04.262589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca266292-a2c3-4fa3-99eb-64423fae769e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:04.262984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca266292-a2c3-4fa3-99eb-64423fae769e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:04.263434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca266292-a2c3-4fa3-99eb-64423fae769e admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:04.264157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca266292-a2c3-4fa3-99eb-64423fae769e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:04.266277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca266292-a2c3-4fa3-99eb-64423fae769e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:04.344783 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1414/2755] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:53:04 2022] POST /identity/v3/projects => generated 359 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:53:04.349092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6b830-e9c9-4a53-ac54-818ddd3fb5e4 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:04.369842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6b830-e9c9-4a53-ac54-818ddd3fb5e4 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:04.479452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dab0fe-3428-4008-9925-d34ce775f80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:04.480125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dab0fe-3428-4008-9925-d34ce775f80d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:04.480413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dab0fe-3428-4008-9925-d34ce775f80d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:04.480694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dab0fe-3428-4008-9925-d34ce775f80d admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:04.481646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dab0fe-3428-4008-9925-d34ce775f80d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:04.503038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dab0fe-3428-4008-9925-d34ce775f80d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:04.569282 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1342/2756] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:53:04 2022] POST /identity/v3/users => generated 286 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:53:04.573072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca266292-a2c3-4fa3-99eb-64423fae769e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:04.596338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca266292-a2c3-4fa3-99eb-64423fae769e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:04.611460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0dec4-26f4-4c31-9714-23976fd32f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:04.614555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0dec4-26f4-4c31-9714-23976fd32f85 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:04.614848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0dec4-26f4-4c31-9714-23976fd32f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:04.615127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0dec4-26f4-4c31-9714-23976fd32f85 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:04.615797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0dec4-26f4-4c31-9714-23976fd32f85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:04.617410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0dec4-26f4-4c31-9714-23976fd32f85 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:04.669646 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1415/2757] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:53:04.670033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dab0fe-3428-4008-9925-d34ce775f80d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:04.681341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dab0fe-3428-4008-9925-d34ce775f80d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:04.686598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60713a8d-9955-41ce-8a5b-05007c72a4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:04.687260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60713a8d-9955-41ce-8a5b-05007c72a4d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:04.687549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60713a8d-9955-41ce-8a5b-05007c72a4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:04.687989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60713a8d-9955-41ce-8a5b-05007c72a4d2 admin admin] PATH_INFO: `/v3/projects/47d0691d44a148d98362a13f0ff3a322/users/5583f22ca6bc4c71aafb646a962cc182/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:04.731578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60713a8d-9955-41ce-8a5b-05007c72a4d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47d0691d44a148d98362a13f0ff3a322, user_id=5583f22ca6bc4c71aafb646a962cc182, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:04.737918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60713a8d-9955-41ce-8a5b-05007c72a4d2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:04.786347 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1343/2758] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:53:04 2022] PUT /identity/v3/projects/47d0691d44a148d98362a13f0ff3a322/users/5583f22ca6bc4c71aafb646a962cc182/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:53:04.790806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0dec4-26f4-4c31-9714-23976fd32f85 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:04.812574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0dec4-26f4-4c31-9714-23976fd32f85 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:04.983768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d548c-d3d5-49a1-9a35-8c98dc7e55ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:04.984633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d548c-d3d5-49a1-9a35-8c98dc7e55ab admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:04.985795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d548c-d3d5-49a1-9a35-8c98dc7e55ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:04.986287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d548c-d3d5-49a1-9a35-8c98dc7e55ab admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:04.987220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d548c-d3d5-49a1-9a35-8c98dc7e55ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:04.989548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d548c-d3d5-49a1-9a35-8c98dc7e55ab admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:05.020764 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1416/2759] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:53:05.025231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60713a8d-9955-41ce-8a5b-05007c72a4d2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:05.043203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60713a8d-9955-41ce-8a5b-05007c72a4d2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:05.059764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7384120-28bf-4dd6-a15b-05bee82fa0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:05.066115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7384120-28bf-4dd6-a15b-05bee82fa0c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:05.066115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7384120-28bf-4dd6-a15b-05bee82fa0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:05.066343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7384120-28bf-4dd6-a15b-05bee82fa0c7 admin admin] PATH_INFO: `/v3/projects/47d0691d44a148d98362a13f0ff3a322/users/5583f22ca6bc4c71aafb646a962cc182/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:05.068502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7384120-28bf-4dd6-a15b-05bee82fa0c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47d0691d44a148d98362a13f0ff3a322, user_id=5583f22ca6bc4c71aafb646a962cc182, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:05.074081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7384120-28bf-4dd6-a15b-05bee82fa0c7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:05.089043 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1344/2760] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:53:05 2022] PUT /identity/v3/projects/47d0691d44a148d98362a13f0ff3a322/users/5583f22ca6bc4c71aafb646a962cc182/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:53:05.101324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e857cad-f4cd-406e-99bf-e39a122a4784 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:05.103430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e857cad-f4cd-406e-99bf-e39a122a4784 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:05.103920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e857cad-f4cd-406e-99bf-e39a122a4784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:05.171490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7e857cad-f4cd-406e-99bf-e39a122a4784 None None] MFA Rules not processed for user `5583f22ca6bc4c71aafb646a962cc182`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:05.172770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7e857cad-f4cd-406e-99bf-e39a122a4784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:05.196465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:05.289606 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1417/2761] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:05 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:05.297148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7384120-28bf-4dd6-a15b-05bee82fa0c7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:05.299545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a7384120-28bf-4dd6-a15b-05bee82fa0c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:05.405952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:05.431301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7384120-28bf-4dd6-a15b-05bee82fa0c7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:05.448360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c5098-39c6-4406-ad1c-488fe14c25ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:05.449011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c5098-39c6-4406-ad1c-488fe14c25ec service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:05.451087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c5098-39c6-4406-ad1c-488fe14c25ec service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:05.451087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c5098-39c6-4406-ad1c-488fe14c25ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:05.474440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c5098-39c6-4406-ad1c-488fe14c25ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:05.476088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c5098-39c6-4406-ad1c-488fe14c25ec service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:05.497394 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1345/2762] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:53:05 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:53:08.500515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbcddf-41e7-4447-9fbf-2c8db6d07913 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:08.501069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbcddf-41e7-4447-9fbf-2c8db6d07913 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:08.501069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbcddf-41e7-4447-9fbf-2c8db6d07913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:08.561077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-bddbcddf-41e7-4447-9fbf-2c8db6d07913 None None] MFA Rules not processed for user `2fb4ea88f36345d89b985350e0be98b7`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:08.563111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bddbcddf-41e7-4447-9fbf-2c8db6d07913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:08.693868 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1418/2763] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:09.270153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c5098-39c6-4406-ad1c-488fe14c25ec service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:09.378664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c5098-39c6-4406-ad1c-488fe14c25ec service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:09.388474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55624ca-3160-4b0d-afb0-49601f1462b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:09.390254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55624ca-3160-4b0d-afb0-49601f1462b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:09.390814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55624ca-3160-4b0d-afb0-49601f1462b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:09.392131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55624ca-3160-4b0d-afb0-49601f1462b2 admin admin] PATH_INFO: `/v3/users/3538ba50b6104ecea13b0efb7b4b39f6` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:09.394020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55624ca-3160-4b0d-afb0-49601f1462b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3538ba50b6104ecea13b0efb7b4b39f6)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:09.397402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55624ca-3160-4b0d-afb0-49601f1462b2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:09.415213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42706bb2-7dd5-4f69-8ed4-680ede8f17d4 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:09.415509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42706bb2-7dd5-4f69-8ed4-680ede8f17d4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:09.415789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42706bb2-7dd5-4f69-8ed4-680ede8f17d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:09.456063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-42706bb2-7dd5-4f69-8ed4-680ede8f17d4 None None] MFA Rules not processed for user `f4de9e6274c041da9c366ea00c0bf769`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:09.457710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-42706bb2-7dd5-4f69-8ed4-680ede8f17d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:09.501322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e55624ca-3160-4b0d-afb0-49601f1462b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3538ba50b6104ecea13b0efb7b4b39f6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:09.502288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e55624ca-3160-4b0d-afb0-49601f1462b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3538ba50b6104ecea13b0efb7b4b39f6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:09.503288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e55624ca-3160-4b0d-afb0-49601f1462b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3538ba50b6104ecea13b0efb7b4b39f6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:09.515403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e55624ca-3160-4b0d-afb0-49601f1462b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3538ba50b6104ecea13b0efb7b4b39f6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:09.545311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e55624ca-3160-4b0d-afb0-49601f1462b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3538ba50b6104ecea13b0efb7b4b39f6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:09.556099 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1419/2764] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:09 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:09.567317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a84c9-dad3-4da4-a1d5-54831290f91d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:09.568408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a84c9-dad3-4da4-a1d5-54831290f91d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:09.568905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a84c9-dad3-4da4-a1d5-54831290f91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:09.572206 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1346/2765] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:53:09 2022] DELETE /identity/v3/users/3538ba50b6104ecea13b0efb7b4b39f6 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:53:09.577224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c6a61-6cd8-4c49-b3e5-8d7bab1524a8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:09.577680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c6a61-6cd8-4c49-b3e5-8d7bab1524a8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:09.578412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c6a61-6cd8-4c49-b3e5-8d7bab1524a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:09.598752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-769a84c9-dad3-4da4-a1d5-54831290f91d None None] MFA Rules not processed for user `f4de9e6274c041da9c366ea00c0bf769`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:09.601218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-769a84c9-dad3-4da4-a1d5-54831290f91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:09.605246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5a6c6a61-6cd8-4c49-b3e5-8d7bab1524a8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:09.606867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5a6c6a61-6cd8-4c49-b3e5-8d7bab1524a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:09.638584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:09.684522 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1347/2766] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:09.694942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c6a61-6cd8-4c49-b3e5-8d7bab1524a8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:09.696469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5a6c6a61-6cd8-4c49-b3e5-8d7bab1524a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:09.720524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:09.728460 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1420/2767] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:09 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:53:09.779436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c6a61-6cd8-4c49-b3e5-8d7bab1524a8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:09.785643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed691d4f-396c-42cb-af0a-0748ce0d3703 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:09.786406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed691d4f-396c-42cb-af0a-0748ce0d3703 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:09.786811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed691d4f-396c-42cb-af0a-0748ce0d3703 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:09.787181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed691d4f-396c-42cb-af0a-0748ce0d3703 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:09.797399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed691d4f-396c-42cb-af0a-0748ce0d3703 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:09.799030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed691d4f-396c-42cb-af0a-0748ce0d3703 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:09.809593 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1348/2768] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:53:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:53:10.059863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a84c9-dad3-4da4-a1d5-54831290f91d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:10.061453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-769a84c9-dad3-4da4-a1d5-54831290f91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:10.211277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a84c9-dad3-4da4-a1d5-54831290f91d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:10.218818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f276b34-984b-453a-8c3b-cbeeef461cf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:10.219786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f276b34-984b-453a-8c3b-cbeeef461cf2 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:10.220427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f276b34-984b-453a-8c3b-cbeeef461cf2 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:10.220867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f276b34-984b-453a-8c3b-cbeeef461cf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:10.225695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6f276b34-984b-453a-8c3b-cbeeef461cf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:10.225860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed691d4f-396c-42cb-af0a-0748ce0d3703 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:10.227622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ed691d4f-396c-42cb-af0a-0748ce0d3703 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:10.264937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed691d4f-396c-42cb-af0a-0748ce0d3703 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:10.275988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332255f9-7e64-4f88-950f-8844b4cc030b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:10.276632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332255f9-7e64-4f88-950f-8844b4cc030b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:10.277112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332255f9-7e64-4f88-950f-8844b4cc030b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:10.277562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332255f9-7e64-4f88-950f-8844b4cc030b admin admin] PATH_INFO: `/v3/projects/0e16bd1c74884610adc40a8a8153f0a0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:10.278977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332255f9-7e64-4f88-950f-8844b4cc030b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e16bd1c74884610adc40a8a8153f0a0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:10.280884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332255f9-7e64-4f88-950f-8844b4cc030b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:10.338534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f276b34-984b-453a-8c3b-cbeeef461cf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:10.340290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f276b34-984b-453a-8c3b-cbeeef461cf2 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:10.366040 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1421/2769] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:53:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:53:10.381011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-332255f9-7e64-4f88-950f-8844b4cc030b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e16bd1c74884610adc40a8a8153f0a0', 'request_id': 'req-332255f9-7e64-4f88-950f-8844b4cc030b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:10.393567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-332255f9-7e64-4f88-950f-8844b4cc030b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e16bd1c74884610adc40a8a8153f0a0', 'request_id': 'req-332255f9-7e64-4f88-950f-8844b4cc030b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:10.402074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-332255f9-7e64-4f88-950f-8844b4cc030b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e16bd1c74884610adc40a8a8153f0a0', 'request_id': 'req-332255f9-7e64-4f88-950f-8844b4cc030b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:10.417238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-332255f9-7e64-4f88-950f-8844b4cc030b admin admin] The token cache is being invalidate because project 0e16bd1c74884610adc40a8a8153f0a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:53:10.417238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-332255f9-7e64-4f88-950f-8844b4cc030b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:10.417820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-332255f9-7e64-4f88-950f-8844b4cc030b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:10.422683 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1349/2770] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:53:10 2022] DELETE /identity/v3/projects/0e16bd1c74884610adc40a8a8153f0a0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:53:10.424238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f276b34-984b-453a-8c3b-cbeeef461cf2 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:10.444334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f276b34-984b-453a-8c3b-cbeeef461cf2 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:10.516611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc870e67-943b-4669-be6e-50ed56cd409c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:10.517171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc870e67-943b-4669-be6e-50ed56cd409c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:10.517456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc870e67-943b-4669-be6e-50ed56cd409c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:10.517773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc870e67-943b-4669-be6e-50ed56cd409c admin admin] PATH_INFO: `/v3/users/5583f22ca6bc4c71aafb646a962cc182` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:10.523299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc870e67-943b-4669-be6e-50ed56cd409c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5583f22ca6bc4c71aafb646a962cc182)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:10.525711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc870e67-943b-4669-be6e-50ed56cd409c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:10.545040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:10.598440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cc870e67-943b-4669-be6e-50ed56cd409c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5583f22ca6bc4c71aafb646a962cc182'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:10.606092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cc870e67-943b-4669-be6e-50ed56cd409c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5583f22ca6bc4c71aafb646a962cc182'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:10.625064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cc870e67-943b-4669-be6e-50ed56cd409c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5583f22ca6bc4c71aafb646a962cc182'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:10.626674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cc870e67-943b-4669-be6e-50ed56cd409c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5583f22ca6bc4c71aafb646a962cc182'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:10.639185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cc870e67-943b-4669-be6e-50ed56cd409c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5583f22ca6bc4c71aafb646a962cc182'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:10.651097 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1422/2771] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:53:10 2022] DELETE /identity/v3/users/5583f22ca6bc4c71aafb646a962cc182 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:53:10.772517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332255f9-7e64-4f88-950f-8844b4cc030b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:10.774335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-332255f9-7e64-4f88-950f-8844b4cc030b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:10.919720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332255f9-7e64-4f88-950f-8844b4cc030b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:11.006275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:11.085791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5906b-f29c-43a6-8f4e-0f57a1ab1ccc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:11.088161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5906b-f29c-43a6-8f4e-0f57a1ab1ccc service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:11.088391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5906b-f29c-43a6-8f4e-0f57a1ab1ccc service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:11.088640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5906b-f29c-43a6-8f4e-0f57a1ab1ccc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:11.092453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-37c5906b-f29c-43a6-8f4e-0f57a1ab1ccc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:11.144346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc870e67-943b-4669-be6e-50ed56cd409c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:11.146170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cc870e67-943b-4669-be6e-50ed56cd409c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:11.199283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5906b-f29c-43a6-8f4e-0f57a1ab1ccc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:11.201458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5906b-f29c-43a6-8f4e-0f57a1ab1ccc service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:11.214046 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1350/2772] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:53:10 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:53:11.294578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc870e67-943b-4669-be6e-50ed56cd409c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:11.303186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ab456-ed45-4a53-abaf-6fa9602f0e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:11.303843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ab456-ed45-4a53-abaf-6fa9602f0e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:11.304126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ab456-ed45-4a53-abaf-6fa9602f0e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:11.304402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ab456-ed45-4a53-abaf-6fa9602f0e11 admin admin] PATH_INFO: `/v3/projects/47d0691d44a148d98362a13f0ff3a322` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:11.307429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ab456-ed45-4a53-abaf-6fa9602f0e11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47d0691d44a148d98362a13f0ff3a322)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:11.309414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ab456-ed45-4a53-abaf-6fa9602f0e11 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:11.425767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-226ab456-ed45-4a53-abaf-6fa9602f0e11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47d0691d44a148d98362a13f0ff3a322', 'request_id': 'req-226ab456-ed45-4a53-abaf-6fa9602f0e11'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:11.434898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-226ab456-ed45-4a53-abaf-6fa9602f0e11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47d0691d44a148d98362a13f0ff3a322', 'request_id': 'req-226ab456-ed45-4a53-abaf-6fa9602f0e11'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:11.455298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-226ab456-ed45-4a53-abaf-6fa9602f0e11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47d0691d44a148d98362a13f0ff3a322', 'request_id': 'req-226ab456-ed45-4a53-abaf-6fa9602f0e11'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:11.485921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-226ab456-ed45-4a53-abaf-6fa9602f0e11 admin admin] The token cache is being invalidate because project 47d0691d44a148d98362a13f0ff3a322 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:53:11.486401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-226ab456-ed45-4a53-abaf-6fa9602f0e11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:11.487272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-226ab456-ed45-4a53-abaf-6fa9602f0e11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:11.493911 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1423/2773] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:53:11 2022] DELETE /identity/v3/projects/47d0691d44a148d98362a13f0ff3a322 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:53:11.496115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dc4d4-aaa0-46c1-a580-3bc619f198da None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:11.496598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dc4d4-aaa0-46c1-a580-3bc619f198da None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:11.496598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dc4d4-aaa0-46c1-a580-3bc619f198da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:11.534357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bc8dc4d4-aaa0-46c1-a580-3bc619f198da None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:11.541787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bc8dc4d4-aaa0-46c1-a580-3bc619f198da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:11.652364 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1351/2774] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:11.729978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f80ad9-bf59-4933-a8a4-b4ba3364dcba None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:11.730878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f80ad9-bf59-4933-a8a4-b4ba3364dcba None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:11.730878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f80ad9-bf59-4933-a8a4-b4ba3364dcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:11.753635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d9f80ad9-bf59-4933-a8a4-b4ba3364dcba None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:11.755266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d9f80ad9-bf59-4933-a8a4-b4ba3364dcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:11.767552 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1424/2775] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:11.768092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dc4d4-aaa0-46c1-a580-3bc619f198da None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:11.787090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dc4d4-aaa0-46c1-a580-3bc619f198da None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:11.794796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e0896-0683-4e75-bacb-84e19e0cc493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:11.796028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e0896-0683-4e75-bacb-84e19e0cc493 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:11.796520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e0896-0683-4e75-bacb-84e19e0cc493 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:11.797009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e0896-0683-4e75-bacb-84e19e0cc493 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:11.800126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e0896-0683-4e75-bacb-84e19e0cc493 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:53:11.800610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e0896-0683-4e75-bacb-84e19e0cc493 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:11.802728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e0896-0683-4e75-bacb-84e19e0cc493 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:11.824409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f80ad9-bf59-4933-a8a4-b4ba3364dcba None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:11.833132 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1352/2776] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:53:11.839065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f80ad9-bf59-4933-a8a4-b4ba3364dcba None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:11.846344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8c963-4ee2-4fb7-b670-2d30e3e38f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:11.848324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8c963-4ee2-4fb7-b670-2d30e3e38f17 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:11.848895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8c963-4ee2-4fb7-b670-2d30e3e38f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:11.849354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8c963-4ee2-4fb7-b670-2d30e3e38f17 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:11.851109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8c963-4ee2-4fb7-b670-2d30e3e38f17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:11.854479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8c963-4ee2-4fb7-b670-2d30e3e38f17 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:11.878994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:11.913365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e0896-0683-4e75-bacb-84e19e0cc493 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:11.924779 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1425/2777] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:53:11 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:53:11.928282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e0896-0683-4e75-bacb-84e19e0cc493 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:12.023714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:12.032937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60a193-b628-4529-bd0f-b27e0f9bf5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:12.033884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60a193-b628-4529-bd0f-b27e0f9bf5b0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:12.034234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60a193-b628-4529-bd0f-b27e0f9bf5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:12.034680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60a193-b628-4529-bd0f-b27e0f9bf5b0 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:12.035390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e60a193-b628-4529-bd0f-b27e0f9bf5b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:12.037805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e60a193-b628-4529-bd0f-b27e0f9bf5b0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:12.068927 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1353/2778] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:53:11 2022] POST /identity/v3/users => generated 290 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:53:12.072345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8c963-4ee2-4fb7-b670-2d30e3e38f17 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:12.083944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8c963-4ee2-4fb7-b670-2d30e3e38f17 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:12.089921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460dce4-37b4-4bde-9991-0ce1fd22bd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:12.090594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460dce4-37b4-4bde-9991-0ce1fd22bd3f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:12.090881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460dce4-37b4-4bde-9991-0ce1fd22bd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:12.091142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460dce4-37b4-4bde-9991-0ce1fd22bd3f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:12.091802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3460dce4-37b4-4bde-9991-0ce1fd22bd3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:12.093597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3460dce4-37b4-4bde-9991-0ce1fd22bd3f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:12.111629 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1426/2779] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:53:12.112784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60a193-b628-4529-bd0f-b27e0f9bf5b0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:12.124901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60a193-b628-4529-bd0f-b27e0f9bf5b0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:12.131941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e624d8b-778e-4a8b-9aae-05ed95061c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:12.132674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e624d8b-778e-4a8b-9aae-05ed95061c30 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:12.133022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e624d8b-778e-4a8b-9aae-05ed95061c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:12.133367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e624d8b-778e-4a8b-9aae-05ed95061c30 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:12.134183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e624d8b-778e-4a8b-9aae-05ed95061c30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:12.136088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e624d8b-778e-4a8b-9aae-05ed95061c30 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:12.152344 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1354/2780] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:53:12.155681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460dce4-37b4-4bde-9991-0ce1fd22bd3f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:12.169161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460dce4-37b4-4bde-9991-0ce1fd22bd3f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:12.176981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc19b9-7477-4e65-84c0-9018c2e17cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:12.177977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc19b9-7477-4e65-84c0-9018c2e17cd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:12.178990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc19b9-7477-4e65-84c0-9018c2e17cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:12.179519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc19b9-7477-4e65-84c0-9018c2e17cd6 admin admin] PATH_INFO: `/v3/projects/44683518fe3d46c58b139727cf0c9515/users/e82ed96a3e794a1b99662cec0969b7ec/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:12.203401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc19b9-7477-4e65-84c0-9018c2e17cd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44683518fe3d46c58b139727cf0c9515, user_id=e82ed96a3e794a1b99662cec0969b7ec, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:12.208264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc19b9-7477-4e65-84c0-9018c2e17cd6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:12.233708 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1427/2781] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:53:12 2022] PUT /identity/v3/projects/44683518fe3d46c58b139727cf0c9515/users/e82ed96a3e794a1b99662cec0969b7ec/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:53:12.236962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4df650-f998-48d6-8415-b53eb51a8871 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:12.237257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4df650-f998-48d6-8415-b53eb51a8871 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:12.237599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4df650-f998-48d6-8415-b53eb51a8871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:12.271220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0b4df650-f998-48d6-8415-b53eb51a8871 None None] MFA Rules not processed for user `e82ed96a3e794a1b99662cec0969b7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:12.272608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0b4df650-f998-48d6-8415-b53eb51a8871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:12.381900 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1355/2782] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:12.385964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc19b9-7477-4e65-84c0-9018c2e17cd6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:12.508192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc19b9-7477-4e65-84c0-9018c2e17cd6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:12.517519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eeaf8a-32f5-4dc1-a096-4c4c994c7cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:12.518764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eeaf8a-32f5-4dc1-a096-4c4c994c7cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:12.519276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eeaf8a-32f5-4dc1-a096-4c4c994c7cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:12.519658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eeaf8a-32f5-4dc1-a096-4c4c994c7cf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:12.520419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eeaf8a-32f5-4dc1-a096-4c4c994c7cf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:12.522477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eeaf8a-32f5-4dc1-a096-4c4c994c7cf5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:12.577632 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1428/2783] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:53:12 2022] POST /identity/v3/projects => generated 379 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:53:12.578354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4df650-f998-48d6-8415-b53eb51a8871 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:12.597712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4df650-f998-48d6-8415-b53eb51a8871 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:12.688810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2973e6-cbb2-46ab-b2fd-1102f1b70595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:12.690039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2973e6-cbb2-46ab-b2fd-1102f1b70595 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:12.690483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2973e6-cbb2-46ab-b2fd-1102f1b70595 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:12.690994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2973e6-cbb2-46ab-b2fd-1102f1b70595 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:12.691867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2973e6-cbb2-46ab-b2fd-1102f1b70595 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:12.693990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2973e6-cbb2-46ab-b2fd-1102f1b70595 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:12.736293 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1356/2784] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:53:12 2022] POST /identity/v3/users => generated 296 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:53:12.739863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eeaf8a-32f5-4dc1-a096-4c4c994c7cf5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:12.751191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eeaf8a-32f5-4dc1-a096-4c4c994c7cf5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:12.768267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae65dc4-404a-4bce-bb8e-4c92969661aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:12.769566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae65dc4-404a-4bce-bb8e-4c92969661aa admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:12.769942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae65dc4-404a-4bce-bb8e-4c92969661aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:12.770644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae65dc4-404a-4bce-bb8e-4c92969661aa admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:12.771718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae65dc4-404a-4bce-bb8e-4c92969661aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:12.773792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae65dc4-404a-4bce-bb8e-4c92969661aa admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:12.793728 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1429/2785] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:53:12.798001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2973e6-cbb2-46ab-b2fd-1102f1b70595 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:12.812616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2973e6-cbb2-46ab-b2fd-1102f1b70595 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:12.820049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492e625-5f77-428e-84e9-54d77a2cc1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:12.821121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492e625-5f77-428e-84e9-54d77a2cc1f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:12.821678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492e625-5f77-428e-84e9-54d77a2cc1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:12.822169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492e625-5f77-428e-84e9-54d77a2cc1f9 admin admin] PATH_INFO: `/v3/projects/9fc6bcd8cd654af7bf1534e88709ef66/users/9f0ebdbb11df4a0399f4e63ea0cd1eda/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:12.855569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1492e625-5f77-428e-84e9-54d77a2cc1f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fc6bcd8cd654af7bf1534e88709ef66, user_id=9f0ebdbb11df4a0399f4e63ea0cd1eda, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:12.861174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1492e625-5f77-428e-84e9-54d77a2cc1f9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:12.894441 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1357/2786] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:53:12 2022] PUT /identity/v3/projects/9fc6bcd8cd654af7bf1534e88709ef66/users/9f0ebdbb11df4a0399f4e63ea0cd1eda/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:53:12.898249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae65dc4-404a-4bce-bb8e-4c92969661aa admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:12.913127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae65dc4-404a-4bce-bb8e-4c92969661aa admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:13.088736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d543898-5ecb-4975-aeae-39e054a1b0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:13.089452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d543898-5ecb-4975-aeae-39e054a1b0bb admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:13.089861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d543898-5ecb-4975-aeae-39e054a1b0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:13.090202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d543898-5ecb-4975-aeae-39e054a1b0bb admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:13.091111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d543898-5ecb-4975-aeae-39e054a1b0bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:13.094686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d543898-5ecb-4975-aeae-39e054a1b0bb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:13.106970 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1430/2787] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:53:13.111859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492e625-5f77-428e-84e9-54d77a2cc1f9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:13.125932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492e625-5f77-428e-84e9-54d77a2cc1f9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:13.137377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c5463-d012-4f36-b50f-560b9f438614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:13.138145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c5463-d012-4f36-b50f-560b9f438614 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:13.138429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c5463-d012-4f36-b50f-560b9f438614 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:13.138741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c5463-d012-4f36-b50f-560b9f438614 admin admin] PATH_INFO: `/v3/projects/9fc6bcd8cd654af7bf1534e88709ef66/users/9f0ebdbb11df4a0399f4e63ea0cd1eda/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:13.140703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c5463-d012-4f36-b50f-560b9f438614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fc6bcd8cd654af7bf1534e88709ef66, user_id=9f0ebdbb11df4a0399f4e63ea0cd1eda, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:13.144396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c5463-d012-4f36-b50f-560b9f438614 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:13.160658 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1358/2788] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:53:13 2022] PUT /identity/v3/projects/9fc6bcd8cd654af7bf1534e88709ef66/users/9f0ebdbb11df4a0399f4e63ea0cd1eda/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:53:13.170967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163f193-e643-4583-bf72-5501aa717cd3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:13.171259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163f193-e643-4583-bf72-5501aa717cd3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:13.171914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163f193-e643-4583-bf72-5501aa717cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:13.227848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4163f193-e643-4583-bf72-5501aa717cd3 None None] MFA Rules not processed for user `9f0ebdbb11df4a0399f4e63ea0cd1eda`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:13.229098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4163f193-e643-4583-bf72-5501aa717cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:13.255116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:13.404228 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1431/2789] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:13.414937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5271257-b2ae-46a4-b4e9-5ee13a961e0a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:13.415228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5271257-b2ae-46a4-b4e9-5ee13a961e0a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:13.415507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5271257-b2ae-46a4-b4e9-5ee13a961e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:13.444026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d5271257-b2ae-46a4-b4e9-5ee13a961e0a None None] MFA Rules not processed for user `e82ed96a3e794a1b99662cec0969b7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:13.445528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d5271257-b2ae-46a4-b4e9-5ee13a961e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:13.462674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:13.557539 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1359/2790] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:14.673679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163f193-e643-4583-bf72-5501aa717cd3 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:14.675259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4163f193-e643-4583-bf72-5501aa717cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:14.785557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163f193-e643-4583-bf72-5501aa717cd3 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:14.791840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34147a7-8dc5-4496-ad95-7335be1d5ad3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:14.792484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34147a7-8dc5-4496-ad95-7335be1d5ad3 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:14.792765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34147a7-8dc5-4496-ad95-7335be1d5ad3 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:14.793046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34147a7-8dc5-4496-ad95-7335be1d5ad3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:14.807772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34147a7-8dc5-4496-ad95-7335be1d5ad3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:14.809734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34147a7-8dc5-4496-ad95-7335be1d5ad3 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:14.836922 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1432/2791] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:53:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:53:23.542407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=17 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:24.817117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=57 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:27.031222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc3c84-077c-463c-ac67-67a1966e2faa None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:27.031222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc3c84-077c-463c-ac67-67a1966e2faa None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:27.031794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc3c84-077c-463c-ac67-67a1966e2faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:27.209674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5edc3c84-077c-463c-ac67-67a1966e2faa None None] MFA Rules not processed for user `a749a74841e840e2b8de67fb9562bf51`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:27.210545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5edc3c84-077c-463c-ac67-67a1966e2faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:27.322674 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1360/2792] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:37.316645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=33 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:38.164667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34147a7-8dc5-4496-ad95-7335be1d5ad3 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:38.167384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f34147a7-8dc5-4496-ad95-7335be1d5ad3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:38.215919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34147a7-8dc5-4496-ad95-7335be1d5ad3 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:38.223002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717733a3-1ba6-4f2b-accb-baea15fcd9ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:38.223750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717733a3-1ba6-4f2b-accb-baea15fcd9ac service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:38.224099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717733a3-1ba6-4f2b-accb-baea15fcd9ac service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:38.224435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717733a3-1ba6-4f2b-accb-baea15fcd9ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:38.233598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717733a3-1ba6-4f2b-accb-baea15fcd9ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:38.235400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717733a3-1ba6-4f2b-accb-baea15fcd9ac service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:38.249099 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1433/2793] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:53:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:53:40.173118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519c559-f7a0-4076-9349-005a15b85acc None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:40.173582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519c559-f7a0-4076-9349-005a15b85acc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:40.173752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519c559-f7a0-4076-9349-005a15b85acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:40.199255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0519c559-f7a0-4076-9349-005a15b85acc None None] MFA Rules not processed for user `e82ed96a3e794a1b99662cec0969b7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:40.200504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0519c559-f7a0-4076-9349-005a15b85acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:40.210581 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1361/2794] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:44.417597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a66b0-4999-4ec6-8fda-38103f44ece2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:44.418812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a66b0-4999-4ec6-8fda-38103f44ece2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:44.419158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a66b0-4999-4ec6-8fda-38103f44ece2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:44.464922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-714a66b0-4999-4ec6-8fda-38103f44ece2 None None] MFA Rules not processed for user `f4de9e6274c041da9c366ea00c0bf769`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:44.470389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-714a66b0-4999-4ec6-8fda-38103f44ece2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:44.568622 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1434/2795] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:44 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:49.327380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519c559-f7a0-4076-9349-005a15b85acc None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:49.329123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0519c559-f7a0-4076-9349-005a15b85acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:49.360546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a66b0-4999-4ec6-8fda-38103f44ece2 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:49.362019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-714a66b0-4999-4ec6-8fda-38103f44ece2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:49.480477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519c559-f7a0-4076-9349-005a15b85acc None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:49.485317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed5df2-9baa-44b4-af89-57f6ed020dd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:49.485935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed5df2-9baa-44b4-af89-57f6ed020dd9 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:49.486211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed5df2-9baa-44b4-af89-57f6ed020dd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:49.486478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed5df2-9baa-44b4-af89-57f6ed020dd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:49.488266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-13ed5df2-9baa-44b4-af89-57f6ed020dd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:49.514700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed5df2-9baa-44b4-af89-57f6ed020dd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:49.517308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed5df2-9baa-44b4-af89-57f6ed020dd9 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:49.550583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a66b0-4999-4ec6-8fda-38103f44ece2 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:49.554245 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1362/2796] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:53:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:53:49.564056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bb84e1-acd8-4db1-89bd-55ac7c845c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:49.564784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb84e1-acd8-4db1-89bd-55ac7c845c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:49.566170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb84e1-acd8-4db1-89bd-55ac7c845c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:49.566675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb84e1-acd8-4db1-89bd-55ac7c845c4a admin admin] PATH_INFO: `/v3/users/2fb4ea88f36345d89b985350e0be98b7` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:49.584513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bb84e1-acd8-4db1-89bd-55ac7c845c4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fb4ea88f36345d89b985350e0be98b7)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:49.593882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bb84e1-acd8-4db1-89bd-55ac7c845c4a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:49.694952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20bb84e1-acd8-4db1-89bd-55ac7c845c4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fb4ea88f36345d89b985350e0be98b7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:49.704178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:49.711670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20bb84e1-acd8-4db1-89bd-55ac7c845c4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fb4ea88f36345d89b985350e0be98b7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:49.734931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20bb84e1-acd8-4db1-89bd-55ac7c845c4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fb4ea88f36345d89b985350e0be98b7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:49.742731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20bb84e1-acd8-4db1-89bd-55ac7c845c4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fb4ea88f36345d89b985350e0be98b7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:49.765795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20bb84e1-acd8-4db1-89bd-55ac7c845c4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fb4ea88f36345d89b985350e0be98b7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:49.773350 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1435/2797] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:53:49 2022] DELETE /identity/v3/users/2fb4ea88f36345d89b985350e0be98b7 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:53:51.597501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed5df2-9baa-44b4-af89-57f6ed020dd9 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:51.619837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed5df2-9baa-44b4-af89-57f6ed020dd9 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:51.766380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af20b7c-0d60-4191-a218-fab89bf642ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:51.768154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af20b7c-0d60-4191-a218-fab89bf642ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:51.768902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af20b7c-0d60-4191-a218-fab89bf642ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:51.769301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af20b7c-0d60-4191-a218-fab89bf642ca admin admin] PATH_INFO: `/v3/projects/9ddc17f805ab4817923b61c801bbbed3` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:51.771269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af20b7c-0d60-4191-a218-fab89bf642ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ddc17f805ab4817923b61c801bbbed3)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:51.774209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af20b7c-0d60-4191-a218-fab89bf642ca admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:51.834577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8af20b7c-0d60-4191-a218-fab89bf642ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ddc17f805ab4817923b61c801bbbed3', 'request_id': 'req-8af20b7c-0d60-4191-a218-fab89bf642ca'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:51.851534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8af20b7c-0d60-4191-a218-fab89bf642ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ddc17f805ab4817923b61c801bbbed3', 'request_id': 'req-8af20b7c-0d60-4191-a218-fab89bf642ca'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:51.863270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8af20b7c-0d60-4191-a218-fab89bf642ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ddc17f805ab4817923b61c801bbbed3', 'request_id': 'req-8af20b7c-0d60-4191-a218-fab89bf642ca'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:51.888042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8af20b7c-0d60-4191-a218-fab89bf642ca admin admin] The token cache is being invalidate because project 9ddc17f805ab4817923b61c801bbbed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:53:51.888536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8af20b7c-0d60-4191-a218-fab89bf642ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:51.889761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8af20b7c-0d60-4191-a218-fab89bf642ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:51.892408 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1363/2798] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:53:51 2022] DELETE /identity/v3/projects/9ddc17f805ab4817923b61c801bbbed3 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:53:51.897714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c54330-8b9d-48c3-a152-f4a6b3b5f623 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:51.898751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c54330-8b9d-48c3-a152-f4a6b3b5f623 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:51.899104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c54330-8b9d-48c3-a152-f4a6b3b5f623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:51.928269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e2c54330-8b9d-48c3-a152-f4a6b3b5f623 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:51.929895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e2c54330-8b9d-48c3-a152-f4a6b3b5f623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:52.115573 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1436/2799] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:52.118669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac64c8-c362-4a2e-b488-73f7983edd4c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:52.119544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac64c8-c362-4a2e-b488-73f7983edd4c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:52.119933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac64c8-c362-4a2e-b488-73f7983edd4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:52.166141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-55ac64c8-c362-4a2e-b488-73f7983edd4c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:52.171662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-55ac64c8-c362-4a2e-b488-73f7983edd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:52.199771 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1364/2800] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:52.200298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c54330-8b9d-48c3-a152-f4a6b3b5f623 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:52.214153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c54330-8b9d-48c3-a152-f4a6b3b5f623 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:52.223916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45992a9f-01ab-4014-a713-8a94f39319b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:52.224664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45992a9f-01ab-4014-a713-8a94f39319b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:52.224947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45992a9f-01ab-4014-a713-8a94f39319b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:52.225298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45992a9f-01ab-4014-a713-8a94f39319b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:52.226050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45992a9f-01ab-4014-a713-8a94f39319b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:53:52.226459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45992a9f-01ab-4014-a713-8a94f39319b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:52.228616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45992a9f-01ab-4014-a713-8a94f39319b6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:52.246213 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1437/2801] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:53:52.251729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac64c8-c362-4a2e-b488-73f7983edd4c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:52.264773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac64c8-c362-4a2e-b488-73f7983edd4c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:52.270393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25514a-3670-4f9d-a5fb-fc02d9c9a2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:52.271216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25514a-3670-4f9d-a5fb-fc02d9c9a2f5 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:52.271502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25514a-3670-4f9d-a5fb-fc02d9c9a2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:52.271772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25514a-3670-4f9d-a5fb-fc02d9c9a2f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:52.272687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25514a-3670-4f9d-a5fb-fc02d9c9a2f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:52.275548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25514a-3670-4f9d-a5fb-fc02d9c9a2f5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:52.311803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:52.325913 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1365/2802] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:53:52 2022] POST /identity/v3/projects => generated 363 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:53:52.327338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45992a9f-01ab-4014-a713-8a94f39319b6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:52.342387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45992a9f-01ab-4014-a713-8a94f39319b6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:52.419708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c68f0-9dd2-49f5-8468-cf3132a653b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:52.421096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c68f0-9dd2-49f5-8468-cf3132a653b1 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:52.421314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c68f0-9dd2-49f5-8468-cf3132a653b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:52.421586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c68f0-9dd2-49f5-8468-cf3132a653b1 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:52.422479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c68f0-9dd2-49f5-8468-cf3132a653b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:52.424562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c68f0-9dd2-49f5-8468-cf3132a653b1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:52.460893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25514a-3670-4f9d-a5fb-fc02d9c9a2f5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:52.476635 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1438/2803] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:53:52 2022] POST /identity/v3/users => generated 282 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:53:52.486478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25514a-3670-4f9d-a5fb-fc02d9c9a2f5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:52.496924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c0fac-d2f4-4a2c-bdbf-d31f6aa0bf4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:52.497897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c0fac-d2f4-4a2c-bdbf-d31f6aa0bf4c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:52.498224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c0fac-d2f4-4a2c-bdbf-d31f6aa0bf4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:52.499266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c0fac-d2f4-4a2c-bdbf-d31f6aa0bf4c admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:52.500135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c0fac-d2f4-4a2c-bdbf-d31f6aa0bf4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:52.502189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c0fac-d2f4-4a2c-bdbf-d31f6aa0bf4c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:52.519151 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1366/2804] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:53:52.519850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c68f0-9dd2-49f5-8468-cf3132a653b1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:52.540426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c68f0-9dd2-49f5-8468-cf3132a653b1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:52.575261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a9b59-f8f8-485a-b214-6bd2d31a919a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:52.575653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a9b59-f8f8-485a-b214-6bd2d31a919a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:52.576059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a9b59-f8f8-485a-b214-6bd2d31a919a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:52.576461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a9b59-f8f8-485a-b214-6bd2d31a919a admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:52.577477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a9b59-f8f8-485a-b214-6bd2d31a919a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:52.598263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a9b59-f8f8-485a-b214-6bd2d31a919a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:52.640729 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1439/2805] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:53:52.642527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c0fac-d2f4-4a2c-bdbf-d31f6aa0bf4c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:52.656371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c0fac-d2f4-4a2c-bdbf-d31f6aa0bf4c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:52.661865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0260f37-ae03-4df0-9ce4-19028cbedebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:52.663409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0260f37-ae03-4df0-9ce4-19028cbedebe admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:52.663918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0260f37-ae03-4df0-9ce4-19028cbedebe admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:52.664407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0260f37-ae03-4df0-9ce4-19028cbedebe admin admin] PATH_INFO: `/v3/projects/9c0b38cea2c345c68c51c4d8cb9c07e7/users/1d180849a694494e89843c0c315acd6d/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:52.682767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0260f37-ae03-4df0-9ce4-19028cbedebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c0b38cea2c345c68c51c4d8cb9c07e7, user_id=1d180849a694494e89843c0c315acd6d, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:52.686805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0260f37-ae03-4df0-9ce4-19028cbedebe admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:52.706308 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1367/2806] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:53:52 2022] PUT /identity/v3/projects/9c0b38cea2c345c68c51c4d8cb9c07e7/users/1d180849a694494e89843c0c315acd6d/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:53:52.711855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb92294-24d8-4a15-84d5-05e6a50fd139 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:52.712510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb92294-24d8-4a15-84d5-05e6a50fd139 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:52.712653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb92294-24d8-4a15-84d5-05e6a50fd139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:52.744179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1eb92294-24d8-4a15-84d5-05e6a50fd139 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:52.745733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1eb92294-24d8-4a15-84d5-05e6a50fd139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:52.802362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:52.863687 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1440/2807] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:52.911915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0260f37-ae03-4df0-9ce4-19028cbedebe admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:52.914067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a0260f37-ae03-4df0-9ce4-19028cbedebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:52.994164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0260f37-ae03-4df0-9ce4-19028cbedebe admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:53.000899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32df4b-0d36-4a32-80ea-d79d4932b211 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:53.006751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32df4b-0d36-4a32-80ea-d79d4932b211 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:53.007147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32df4b-0d36-4a32-80ea-d79d4932b211 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:53.007465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32df4b-0d36-4a32-80ea-d79d4932b211 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:53.031405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c32df4b-0d36-4a32-80ea-d79d4932b211 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:53.033223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c32df4b-0d36-4a32-80ea-d79d4932b211 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:53.055255 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1368/2808] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:53:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:53:53.149445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb92294-24d8-4a15-84d5-05e6a50fd139 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:53.151048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1eb92294-24d8-4a15-84d5-05e6a50fd139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:53.174879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb92294-24d8-4a15-84d5-05e6a50fd139 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:53.182820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edf897-41e6-4c2b-8128-b81ece9986b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:53.189001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edf897-41e6-4c2b-8128-b81ece9986b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:53.189778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edf897-41e6-4c2b-8128-b81ece9986b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:53.189778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edf897-41e6-4c2b-8128-b81ece9986b0 admin admin] PATH_INFO: `/v3/users/d270f04a32ac44bc88fc57f8e765111e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:53.193441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79edf897-41e6-4c2b-8128-b81ece9986b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d270f04a32ac44bc88fc57f8e765111e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:53.200115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79edf897-41e6-4c2b-8128-b81ece9986b0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:53.380700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-79edf897-41e6-4c2b-8128-b81ece9986b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd270f04a32ac44bc88fc57f8e765111e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:53.394172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-79edf897-41e6-4c2b-8128-b81ece9986b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd270f04a32ac44bc88fc57f8e765111e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:53.415502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-79edf897-41e6-4c2b-8128-b81ece9986b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd270f04a32ac44bc88fc57f8e765111e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:53.416597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-79edf897-41e6-4c2b-8128-b81ece9986b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd270f04a32ac44bc88fc57f8e765111e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:53.431952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-79edf897-41e6-4c2b-8128-b81ece9986b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd270f04a32ac44bc88fc57f8e765111e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:53.440894 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1441/2809] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:53:53 2022] DELETE /identity/v3/users/d270f04a32ac44bc88fc57f8e765111e => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:53:54.440536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32df4b-0d36-4a32-80ea-d79d4932b211 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:54.461160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32df4b-0d36-4a32-80ea-d79d4932b211 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:54.542288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e112d-3ad5-46f7-93f1-332651c1522d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:54.542549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e112d-3ad5-46f7-93f1-332651c1522d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:54.542876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e112d-3ad5-46f7-93f1-332651c1522d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:54.543382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e112d-3ad5-46f7-93f1-332651c1522d admin admin] PATH_INFO: `/v3/projects/caba560a5eaa46c1b816e09230b5a1d9` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:54.547326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e112d-3ad5-46f7-93f1-332651c1522d admin admin] RBAC: Authorizing `identity:delete_project(project_id=caba560a5eaa46c1b816e09230b5a1d9)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:54.549544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e112d-3ad5-46f7-93f1-332651c1522d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:54.608565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4f7e112d-3ad5-46f7-93f1-332651c1522d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caba560a5eaa46c1b816e09230b5a1d9', 'request_id': 'req-4f7e112d-3ad5-46f7-93f1-332651c1522d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:54.619618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4f7e112d-3ad5-46f7-93f1-332651c1522d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caba560a5eaa46c1b816e09230b5a1d9', 'request_id': 'req-4f7e112d-3ad5-46f7-93f1-332651c1522d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:54.628532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4f7e112d-3ad5-46f7-93f1-332651c1522d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caba560a5eaa46c1b816e09230b5a1d9', 'request_id': 'req-4f7e112d-3ad5-46f7-93f1-332651c1522d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:54.632364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:54.652112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4f7e112d-3ad5-46f7-93f1-332651c1522d admin admin] The token cache is being invalidate because project caba560a5eaa46c1b816e09230b5a1d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:53:54.655202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4f7e112d-3ad5-46f7-93f1-332651c1522d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:54.656718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4f7e112d-3ad5-46f7-93f1-332651c1522d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:53:54.659664 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1369/2810] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:53:54 2022] DELETE /identity/v3/projects/caba560a5eaa46c1b816e09230b5a1d9 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:53:54.664675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef099c-4f16-4667-bbcf-c9b997d5ed9b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:54.665694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef099c-4f16-4667-bbcf-c9b997d5ed9b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:54.666074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef099c-4f16-4667-bbcf-c9b997d5ed9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:54.687230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e9ef099c-4f16-4667-bbcf-c9b997d5ed9b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:54.689121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e9ef099c-4f16-4667-bbcf-c9b997d5ed9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:54.927547 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1442/2811] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:53:54.939196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4615717-4a60-4e60-8392-0007d6c54ae5 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:54.939778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4615717-4a60-4e60-8392-0007d6c54ae5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:54.940305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4615717-4a60-4e60-8392-0007d6c54ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:54.986108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e4615717-4a60-4e60-8392-0007d6c54ae5 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:54.988519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e4615717-4a60-4e60-8392-0007d6c54ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:55.004091 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1370/2812] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:55.008446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef099c-4f16-4667-bbcf-c9b997d5ed9b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:55.022427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef099c-4f16-4667-bbcf-c9b997d5ed9b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:55.045781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563185c-ff91-4348-a28d-8e1137e1e99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:55.047494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563185c-ff91-4348-a28d-8e1137e1e99f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:55.047989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563185c-ff91-4348-a28d-8e1137e1e99f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:55.048597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563185c-ff91-4348-a28d-8e1137e1e99f admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:55.055795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9563185c-ff91-4348-a28d-8e1137e1e99f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:53:55.058163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9563185c-ff91-4348-a28d-8e1137e1e99f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:55.060069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9563185c-ff91-4348-a28d-8e1137e1e99f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:55.090099 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1443/2813] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:53:55.095449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4615717-4a60-4e60-8392-0007d6c54ae5 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:55.111720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4615717-4a60-4e60-8392-0007d6c54ae5 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:55.121801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891be73a-9670-4cd5-a7f6-ead629f04ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:55.122582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891be73a-9670-4cd5-a7f6-ead629f04ce0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:55.122949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891be73a-9670-4cd5-a7f6-ead629f04ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:55.124124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891be73a-9670-4cd5-a7f6-ead629f04ce0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:55.124995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891be73a-9670-4cd5-a7f6-ead629f04ce0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:55.128028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891be73a-9670-4cd5-a7f6-ead629f04ce0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:55.176773 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1371/2814] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:53:55 2022] POST /identity/v3/projects => generated 377 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:53:55.183377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563185c-ff91-4348-a28d-8e1137e1e99f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:55.205757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563185c-ff91-4348-a28d-8e1137e1e99f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:55.252257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:55.296388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4a0a5-505e-4624-ac6a-3f9727c79494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:55.297421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4a0a5-505e-4624-ac6a-3f9727c79494 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:55.298654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4a0a5-505e-4624-ac6a-3f9727c79494 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:55.299174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4a0a5-505e-4624-ac6a-3f9727c79494 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:55.300036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4a0a5-505e-4624-ac6a-3f9727c79494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:55.302460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4a0a5-505e-4624-ac6a-3f9727c79494 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:55.345289 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1444/2815] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:53:55 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:53:55.349009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891be73a-9670-4cd5-a7f6-ead629f04ce0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:55.387714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891be73a-9670-4cd5-a7f6-ead629f04ce0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:55.415541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f059fe6-7c54-4566-bff0-612e55378594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:55.416369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f059fe6-7c54-4566-bff0-612e55378594 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:55.416659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f059fe6-7c54-4566-bff0-612e55378594 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:55.417682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f059fe6-7c54-4566-bff0-612e55378594 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:55.418422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f059fe6-7c54-4566-bff0-612e55378594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:55.422465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f059fe6-7c54-4566-bff0-612e55378594 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:55.443161 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1372/2816] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:53:55.445540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4a0a5-505e-4624-ac6a-3f9727c79494 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:55.465606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4a0a5-505e-4624-ac6a-3f9727c79494 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:55.472791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33108cff-fab4-442f-bf6a-e0be7bcdab99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:55.473783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33108cff-fab4-442f-bf6a-e0be7bcdab99 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:55.473991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33108cff-fab4-442f-bf6a-e0be7bcdab99 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:55.474211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33108cff-fab4-442f-bf6a-e0be7bcdab99 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:55.474808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33108cff-fab4-442f-bf6a-e0be7bcdab99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:55.476591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33108cff-fab4-442f-bf6a-e0be7bcdab99 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:55.490231 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1445/2817] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:53:55.495723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f059fe6-7c54-4566-bff0-612e55378594 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:55.515268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f059fe6-7c54-4566-bff0-612e55378594 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:55.523222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b5fe6-2b53-445e-b6ad-cd940348e652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:55.524192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b5fe6-2b53-445e-b6ad-cd940348e652 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:55.525024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b5fe6-2b53-445e-b6ad-cd940348e652 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:55.525531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b5fe6-2b53-445e-b6ad-cd940348e652 admin admin] PATH_INFO: `/v3/projects/bcaab3f0817043c682737e94bb1dd1ac/users/725f68beba264c2eafc951f8300eba32/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:55.543261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b5fe6-2b53-445e-b6ad-cd940348e652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcaab3f0817043c682737e94bb1dd1ac, user_id=725f68beba264c2eafc951f8300eba32, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:55.548507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b5fe6-2b53-445e-b6ad-cd940348e652 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:55.570372 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1373/2818] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:53:55 2022] PUT /identity/v3/projects/bcaab3f0817043c682737e94bb1dd1ac/users/725f68beba264c2eafc951f8300eba32/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:53:55.576441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3b5a6-6b8f-4547-be11-bbe380a9b7f8 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:55.578607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3b5a6-6b8f-4547-be11-bbe380a9b7f8 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:55.580069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3b5a6-6b8f-4547-be11-bbe380a9b7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:55.629782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-09d3b5a6-6b8f-4547-be11-bbe380a9b7f8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:55.632085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-09d3b5a6-6b8f-4547-be11-bbe380a9b7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:55.787717 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1446/2819] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:55.830530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b5fe6-2b53-445e-b6ad-cd940348e652 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:55.834700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d14b5fe6-2b53-445e-b6ad-cd940348e652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:55.958782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b5fe6-2b53-445e-b6ad-cd940348e652 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:55.965367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d017f3-6546-4449-85e5-f157cb62a4f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:55.968326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d017f3-6546-4449-85e5-f157cb62a4f9 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:55.970234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d017f3-6546-4449-85e5-f157cb62a4f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:55.970879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d017f3-6546-4449-85e5-f157cb62a4f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:55.989802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d017f3-6546-4449-85e5-f157cb62a4f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:55.991629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d017f3-6546-4449-85e5-f157cb62a4f9 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:56.008966 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1374/2820] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:53:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:53:56.599979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3b5a6-6b8f-4547-be11-bbe380a9b7f8 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:56.602044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-09d3b5a6-6b8f-4547-be11-bbe380a9b7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:56.626228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3b5a6-6b8f-4547-be11-bbe380a9b7f8 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:56.631561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824dc756-3aea-4e17-97be-eb878e84c7a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:56.632366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dc756-3aea-4e17-97be-eb878e84c7a9 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:56.632654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dc756-3aea-4e17-97be-eb878e84c7a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:56.632937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dc756-3aea-4e17-97be-eb878e84c7a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:56.634837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-824dc756-3aea-4e17-97be-eb878e84c7a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:56.842393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824dc756-3aea-4e17-97be-eb878e84c7a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:56.844919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824dc756-3aea-4e17-97be-eb878e84c7a9 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:56.869475 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1447/2821] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:53:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:53:58.088712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13041c-85b1-41d7-b371-6f2f4bc1cc98 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:58.089259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13041c-85b1-41d7-b371-6f2f4bc1cc98 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:58.089839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13041c-85b1-41d7-b371-6f2f4bc1cc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:58.116620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8a13041c-85b1-41d7-b371-6f2f4bc1cc98 None None] MFA Rules not processed for user `e5bfbc1e512a445ca7527f3d79e40d16`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:53:58.125047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8a13041c-85b1-41d7-b371-6f2f4bc1cc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:58.230430 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1375/2822] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:53:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:53:58.979605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824dc756-3aea-4e17-97be-eb878e84c7a9 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:53:58.981429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-824dc756-3aea-4e17-97be-eb878e84c7a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:59.124738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:59.211187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824dc756-3aea-4e17-97be-eb878e84c7a9 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:53:59.231981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc4113-7e2a-421f-983d-03601fa736a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:53:59.235219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc4113-7e2a-421f-983d-03601fa736a8 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:53:59.236107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc4113-7e2a-421f-983d-03601fa736a8 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:53:59.236627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc4113-7e2a-421f-983d-03601fa736a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:53:59.244125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2dfc4113-7e2a-421f-983d-03601fa736a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:53:59.264942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc4113-7e2a-421f-983d-03601fa736a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:53:59.267789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc4113-7e2a-421f-983d-03601fa736a8 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:53:59.282356 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1448/2823] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:53:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:54:00.691343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a13041c-85b1-41d7-b371-6f2f4bc1cc98 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:54:00.706976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a13041c-85b1-41d7-b371-6f2f4bc1cc98 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:54:00.715238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252ffda-03af-491d-ac90-6a17e26241cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:54:00.716150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252ffda-03af-491d-ac90-6a17e26241cc service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:00.716669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252ffda-03af-491d-ac90-6a17e26241cc service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:00.717161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252ffda-03af-491d-ac90-6a17e26241cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:00.720068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1252ffda-03af-491d-ac90-6a17e26241cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:00.731572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252ffda-03af-491d-ac90-6a17e26241cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:54:00.733928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252ffda-03af-491d-ac90-6a17e26241cc service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:54:00.747995 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1376/2824] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:54:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:54:04.647686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c650122-0247-4a6b-86c8-97f5eaabeea9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:04.648651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c650122-0247-4a6b-86c8-97f5eaabeea9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:04.649145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c650122-0247-4a6b-86c8-97f5eaabeea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:04.700375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7c650122-0247-4a6b-86c8-97f5eaabeea9 None None] MFA Rules not processed for user `725f68beba264c2eafc951f8300eba32`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:54:04.707689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7c650122-0247-4a6b-86c8-97f5eaabeea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:04.881840 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1449/2825] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:54:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:54:04.892383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e401091-97d2-49e3-8ad8-66249f5fed17 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:04.892982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e401091-97d2-49e3-8ad8-66249f5fed17 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:04.893073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e401091-97d2-49e3-8ad8-66249f5fed17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:04.923368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5e401091-97d2-49e3-8ad8-66249f5fed17 None None] MFA Rules not processed for user `725f68beba264c2eafc951f8300eba32`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:54:04.925535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5e401091-97d2-49e3-8ad8-66249f5fed17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:04.940448 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1377/2826] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:54:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:54:04.949011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c650122-0247-4a6b-86c8-97f5eaabeea9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:54:04.950746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7c650122-0247-4a6b-86c8-97f5eaabeea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:04.973026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c650122-0247-4a6b-86c8-97f5eaabeea9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:54:04.986423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e299254-f491-4950-9150-d3be1187b838 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:54:04.987366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e299254-f491-4950-9150-d3be1187b838 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:04.987648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e299254-f491-4950-9150-d3be1187b838 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:04.987956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e299254-f491-4950-9150-d3be1187b838 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:04.999102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e299254-f491-4950-9150-d3be1187b838 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:54:05.001558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e299254-f491-4950-9150-d3be1187b838 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:54:05.024498 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1450/2827] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:54:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:54:05.943038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4cb27c-b600-415e-b507-941e355c4471 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:05.943456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4cb27c-b600-415e-b507-941e355c4471 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:05.943558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4cb27c-b600-415e-b507-941e355c4471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:05.984328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-be4cb27c-b600-415e-b507-941e355c4471 None None] MFA Rules not processed for user `1d180849a694494e89843c0c315acd6d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:54:05.986823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-be4cb27c-b600-415e-b507-941e355c4471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:06.010788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:54:06.169488 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1378/2828] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:54:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:54:06.184579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109753ee-add9-4d19-8783-9bf049d7a220 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:06.187246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109753ee-add9-4d19-8783-9bf049d7a220 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:06.187808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109753ee-add9-4d19-8783-9bf049d7a220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:06.246624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-109753ee-add9-4d19-8783-9bf049d7a220 None None] MFA Rules not processed for user `1d180849a694494e89843c0c315acd6d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:54:06.247910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-109753ee-add9-4d19-8783-9bf049d7a220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:06.275783 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1451/2829] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:54:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:54:06.661897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4cb27c-b600-415e-b507-941e355c4471 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:54:06.696440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4cb27c-b600-415e-b507-941e355c4471 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:54:06.710242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bcf5d-5d9f-4b8a-b923-b97b02c7fafe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:54:06.711266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bcf5d-5d9f-4b8a-b923-b97b02c7fafe service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:06.712340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bcf5d-5d9f-4b8a-b923-b97b02c7fafe service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:06.712656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2a72e-3136-4b0c-a4fc-f4bd5e271e65 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:06.713239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bcf5d-5d9f-4b8a-b923-b97b02c7fafe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:06.713383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2a72e-3136-4b0c-a4fc-f4bd5e271e65 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:06.713534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2a72e-3136-4b0c-a4fc-f4bd5e271e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:06.750225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bcf5d-5d9f-4b8a-b923-b97b02c7fafe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:54:06.752161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bcf5d-5d9f-4b8a-b923-b97b02c7fafe service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:54:06.759944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-40c2a72e-3136-4b0c-a4fc-f4bd5e271e65 None None] MFA Rules not processed for user `aa72870453cd4b958725801b842c7a28`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:54:06.773526 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1379/2830] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:54:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:54:06.781231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-40c2a72e-3136-4b0c-a4fc-f4bd5e271e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:06.890557 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1452/2831] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:54:06 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:54:08.835747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bcf5d-5d9f-4b8a-b923-b97b02c7fafe service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:54:08.837295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ba3bcf5d-5d9f-4b8a-b923-b97b02c7fafe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:08.881411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bcf5d-5d9f-4b8a-b923-b97b02c7fafe service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:54:08.902263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78992faa-c65b-48ff-aa46-56afab123877 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:54:08.906531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78992faa-c65b-48ff-aa46-56afab123877 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:08.906823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78992faa-c65b-48ff-aa46-56afab123877 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:08.907309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78992faa-c65b-48ff-aa46-56afab123877 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:08.912173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-78992faa-c65b-48ff-aa46-56afab123877 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:08.957775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78992faa-c65b-48ff-aa46-56afab123877 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:54:08.961635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78992faa-c65b-48ff-aa46-56afab123877 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:54:09.014207 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1380/2832] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:54:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:54:11.602634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2a72e-3136-4b0c-a4fc-f4bd5e271e65 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:54:11.617097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2a72e-3136-4b0c-a4fc-f4bd5e271e65 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:54:11.624686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ac916-915f-4dda-9b63-6a94b7210226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:54:11.626651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ac916-915f-4dda-9b63-6a94b7210226 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:11.627224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ac916-915f-4dda-9b63-6a94b7210226 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:11.627712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ac916-915f-4dda-9b63-6a94b7210226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:11.631067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7d5ac916-915f-4dda-9b63-6a94b7210226 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:11.650763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ac916-915f-4dda-9b63-6a94b7210226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:54:11.652205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ac916-915f-4dda-9b63-6a94b7210226 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:54:11.664888 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1453/2833] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:54:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:54:16.530371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78992faa-c65b-48ff-aa46-56afab123877 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:54:16.542927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78992faa-c65b-48ff-aa46-56afab123877 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:54:16.549160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e49cc-11f4-4874-9e31-47aebd97f325 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:54:16.549673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e49cc-11f4-4874-9e31-47aebd97f325 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:16.549957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e49cc-11f4-4874-9e31-47aebd97f325 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:16.550233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e49cc-11f4-4874-9e31-47aebd97f325 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:16.552039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a87e49cc-11f4-4874-9e31-47aebd97f325 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:16.565879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87e49cc-11f4-4874-9e31-47aebd97f325 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:54:16.567497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87e49cc-11f4-4874-9e31-47aebd97f325 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:54:16.594304 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1381/2834] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:54:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:54:18.021113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ac916-915f-4dda-9b63-6a94b7210226 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:54:18.053531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ac916-915f-4dda-9b63-6a94b7210226 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:54:18.059557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcaa725-c54d-4d3b-937f-2b746f847c91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:54:18.061353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaa725-c54d-4d3b-937f-2b746f847c91 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:18.061937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaa725-c54d-4d3b-937f-2b746f847c91 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:18.062436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaa725-c54d-4d3b-937f-2b746f847c91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:18.066309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8bcaa725-c54d-4d3b-937f-2b746f847c91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:18.098886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcaa725-c54d-4d3b-937f-2b746f847c91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:54:18.100586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcaa725-c54d-4d3b-937f-2b746f847c91 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:54:18.115401 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1454/2835] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:54:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:54:18.225050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb606084-d986-45c0-ae7e-8545c90204ec None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:18.225501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb606084-d986-45c0-ae7e-8545c90204ec None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:18.225588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb606084-d986-45c0-ae7e-8545c90204ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:18.257208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bb606084-d986-45c0-ae7e-8545c90204ec None None] MFA Rules not processed for user `e82ed96a3e794a1b99662cec0969b7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:54:18.261478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bb606084-d986-45c0-ae7e-8545c90204ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:18.355558 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1382/2836] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:54:18.671920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcaa725-c54d-4d3b-937f-2b746f847c91 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:54:18.685696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcaa725-c54d-4d3b-937f-2b746f847c91 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:54:18.692615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6db19-b66a-4338-910d-60467710eb03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:54:18.693442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6db19-b66a-4338-910d-60467710eb03 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:18.693884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6db19-b66a-4338-910d-60467710eb03 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:18.694390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6db19-b66a-4338-910d-60467710eb03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:18.696649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-84f6db19-b66a-4338-910d-60467710eb03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:18.710643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6db19-b66a-4338-910d-60467710eb03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:54:18.712489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6db19-b66a-4338-910d-60467710eb03 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:54:18.739533 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1455/2837] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:54:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:54:26.266424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb606084-d986-45c0-ae7e-8545c90204ec None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:54:26.268016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bb606084-d986-45c0-ae7e-8545c90204ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:26.290556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb606084-d986-45c0-ae7e-8545c90204ec None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:54:26.305182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab6d1e-1f1a-4040-a28c-6e7bdddc778f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:54:26.307069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab6d1e-1f1a-4040-a28c-6e7bdddc778f service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:26.307354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab6d1e-1f1a-4040-a28c-6e7bdddc778f service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:26.307638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab6d1e-1f1a-4040-a28c-6e7bdddc778f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:26.313721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fcab6d1e-1f1a-4040-a28c-6e7bdddc778f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:26.336123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab6d1e-1f1a-4040-a28c-6e7bdddc778f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:54:26.337873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab6d1e-1f1a-4040-a28c-6e7bdddc778f service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:54:26.363209 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1383/2838] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:54:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:54:28.720691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:54:36.346930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=72 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:54:37.790364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b83d7e-b6d1-410e-a9e6-1ca5c3ec7336 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:37.791354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b83d7e-b6d1-410e-a9e6-1ca5c3ec7336 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:37.791633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b83d7e-b6d1-410e-a9e6-1ca5c3ec7336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:37.819230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-21b83d7e-b6d1-410e-a9e6-1ca5c3ec7336 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:54:37.821993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-21b83d7e-b6d1-410e-a9e6-1ca5c3ec7336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:37.896889 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1456/2839] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:54:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:54:37.960777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab6d1e-1f1a-4040-a28c-6e7bdddc778f service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:54:37.962561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fcab6d1e-1f1a-4040-a28c-6e7bdddc778f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:38.076053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab6d1e-1f1a-4040-a28c-6e7bdddc778f service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:54:38.083404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f157e2-8680-4727-a61c-1233d72ef3ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:54:38.084435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f157e2-8680-4727-a61c-1233d72ef3ec service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:38.085218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f157e2-8680-4727-a61c-1233d72ef3ec service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:38.085218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f157e2-8680-4727-a61c-1233d72ef3ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:38.098211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f157e2-8680-4727-a61c-1233d72ef3ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:54:38.099840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f157e2-8680-4727-a61c-1233d72ef3ec service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:54:38.121358 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1384/2840] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:54:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 20:54:40.391181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b83d7e-b6d1-410e-a9e6-1ca5c3ec7336 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:54:40.410254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b83d7e-b6d1-410e-a9e6-1ca5c3ec7336 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:54:40.418422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580883b6-069b-4ca9-9a55-bc1ca04601d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:54:40.420176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580883b6-069b-4ca9-9a55-bc1ca04601d0 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:40.420946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580883b6-069b-4ca9-9a55-bc1ca04601d0 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:40.421440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580883b6-069b-4ca9-9a55-bc1ca04601d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:40.424949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-580883b6-069b-4ca9-9a55-bc1ca04601d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:40.501834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580883b6-069b-4ca9-9a55-bc1ca04601d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:54:40.508933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580883b6-069b-4ca9-9a55-bc1ca04601d0 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:54:40.598211 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1457/2841] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:54:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:54:48.118393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=32 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:54:50.533845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=31 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:54:50.853197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251502c-1297-4400-ab94-26a1dc7dee99 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:50.854240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251502c-1297-4400-ab94-26a1dc7dee99 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:50.854773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251502c-1297-4400-ab94-26a1dc7dee99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:50.892240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4251502c-1297-4400-ab94-26a1dc7dee99 None None] MFA Rules not processed for user `e82ed96a3e794a1b99662cec0969b7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:54:50.893688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4251502c-1297-4400-ab94-26a1dc7dee99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:50.905354 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1385/2842] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:54:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:54:54.962798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d879042-ca03-4362-9b24-6b7ae3aa045a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:54.963390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d879042-ca03-4362-9b24-6b7ae3aa045a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:54.963514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d879042-ca03-4362-9b24-6b7ae3aa045a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:54.986792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1d879042-ca03-4362-9b24-6b7ae3aa045a None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:54:54.988484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1d879042-ca03-4362-9b24-6b7ae3aa045a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:54.996674 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1458/2843] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 20:54:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:54:55.011973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251502c-1297-4400-ab94-26a1dc7dee99 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:54:55.029141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251502c-1297-4400-ab94-26a1dc7dee99 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:54:55.037813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba3630-6d5a-4501-a22a-363f0db3ea54 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:54:55.038853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba3630-6d5a-4501-a22a-363f0db3ea54 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:55.039319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba3630-6d5a-4501-a22a-363f0db3ea54 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:55.039751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba3630-6d5a-4501-a22a-363f0db3ea54 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:55.058785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba3630-6d5a-4501-a22a-363f0db3ea54 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:54:55.060407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba3630-6d5a-4501-a22a-363f0db3ea54 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:54:55.070947 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1386/2844] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:54:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 20:54:55.912719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471f159-a50a-4f03-9feb-f18541a7cbf9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:55.913214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471f159-a50a-4f03-9feb-f18541a7cbf9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:55.913489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471f159-a50a-4f03-9feb-f18541a7cbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:55.976339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c471f159-a50a-4f03-9feb-f18541a7cbf9 None None] MFA Rules not processed for user `1d180849a694494e89843c0c315acd6d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:54:55.981939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c471f159-a50a-4f03-9feb-f18541a7cbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:56.010385 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1459/2845] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:54:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:54:56.703036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba3630-6d5a-4501-a22a-363f0db3ea54 service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:54:56.715373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba3630-6d5a-4501-a22a-363f0db3ea54 service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:54:56.719890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf519e-5d4d-42af-b4a7-e4e00ee3193f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:54:56.721367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf519e-5d4d-42af-b4a7-e4e00ee3193f service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:56.721592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf519e-5d4d-42af-b4a7-e4e00ee3193f service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:56.722200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf519e-5d4d-42af-b4a7-e4e00ee3193f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:56.724731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6dbf519e-5d4d-42af-b4a7-e4e00ee3193f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:56.742979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf519e-5d4d-42af-b4a7-e4e00ee3193f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:54:56.745912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf519e-5d4d-42af-b4a7-e4e00ee3193f service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:54:56.759458 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1387/2846] 10.222.0.17 () {60 vars in 1396 bytes} [Sat Oct 1 20:54:56 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3272 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:54:57.329611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69052f08-f7c9-4968-8eca-e9ebb84b1a5c None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:57.330201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69052f08-f7c9-4968-8eca-e9ebb84b1a5c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:57.330201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69052f08-f7c9-4968-8eca-e9ebb84b1a5c None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:57.331868 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1460/2847] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:54:57.338133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565b7ac-0577-4aa4-b218-6fb25b3f6679 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:57.338420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565b7ac-0577-4aa4-b218-6fb25b3f6679 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:57.339134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565b7ac-0577-4aa4-b218-6fb25b3f6679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:57.372848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6565b7ac-0577-4aa4-b218-6fb25b3f6679 None None] MFA Rules not processed for user `e17a026f305248878f18eec35e1e02dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:54:57.374529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6565b7ac-0577-4aa4-b218-6fb25b3f6679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:54:57.385377 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1388/2848] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:54:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:54:57.412375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69052f08-f7c9-4968-8eca-e9ebb84b1a5c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:54:57.442515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69052f08-f7c9-4968-8eca-e9ebb84b1a5c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:54:57.442515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d238a-5199-43ad-acd3-c52129d259b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:54:57.442515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d238a-5199-43ad-acd3-c52129d259b1 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:54:57.442926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d238a-5199-43ad-acd3-c52129d259b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:54:57.443030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d238a-5199-43ad-acd3-c52129d259b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:54:57.460870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d238a-5199-43ad-acd3-c52129d259b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:54:57.462475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d238a-5199-43ad-acd3-c52129d259b1 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:54:57.482664 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1461/2849] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:54:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:55:02.282202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565b7ac-0577-4aa4-b218-6fb25b3f6679 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:02.285994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6565b7ac-0577-4aa4-b218-6fb25b3f6679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:02.344117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565b7ac-0577-4aa4-b218-6fb25b3f6679 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:02.372627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4e9ed-34ed-4e70-9645-57007ee0d041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:02.374482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4e9ed-34ed-4e70-9645-57007ee0d041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:02.385189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4e9ed-34ed-4e70-9645-57007ee0d041 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:02.385948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4e9ed-34ed-4e70-9645-57007ee0d041 admin admin] PATH_INFO: `/v3/users/f4de9e6274c041da9c366ea00c0bf769` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:02.388958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4e9ed-34ed-4e70-9645-57007ee0d041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4de9e6274c041da9c366ea00c0bf769)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:02.393533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4e9ed-34ed-4e70-9645-57007ee0d041 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:02.545549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d2c4e9ed-34ed-4e70-9645-57007ee0d041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4de9e6274c041da9c366ea00c0bf769'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:02.546895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d2c4e9ed-34ed-4e70-9645-57007ee0d041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4de9e6274c041da9c366ea00c0bf769'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:02.547968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d2c4e9ed-34ed-4e70-9645-57007ee0d041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4de9e6274c041da9c366ea00c0bf769'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:02.561631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d2c4e9ed-34ed-4e70-9645-57007ee0d041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4de9e6274c041da9c366ea00c0bf769'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:02.580489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d2c4e9ed-34ed-4e70-9645-57007ee0d041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4de9e6274c041da9c366ea00c0bf769'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:02.603980 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1389/2850] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:55:02 2022] DELETE /identity/v3/users/f4de9e6274c041da9c366ea00c0bf769 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:55:02.831651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d238a-5199-43ad-acd3-c52129d259b1 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:02.848470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d238a-5199-43ad-acd3-c52129d259b1 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:02.992528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be498d9-ca02-4417-a786-ef064818b5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:02.993430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be498d9-ca02-4417-a786-ef064818b5de admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:02.993858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be498d9-ca02-4417-a786-ef064818b5de admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:02.994327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be498d9-ca02-4417-a786-ef064818b5de admin admin] PATH_INFO: `/v3/projects/1a2694c8af60455aa9db5997c53adbdc` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:02.995901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be498d9-ca02-4417-a786-ef064818b5de admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a2694c8af60455aa9db5997c53adbdc)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:02.997975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be498d9-ca02-4417-a786-ef064818b5de admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:03.071167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4be498d9-ca02-4417-a786-ef064818b5de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a2694c8af60455aa9db5997c53adbdc', 'request_id': 'req-4be498d9-ca02-4417-a786-ef064818b5de'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:03.083014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4be498d9-ca02-4417-a786-ef064818b5de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a2694c8af60455aa9db5997c53adbdc', 'request_id': 'req-4be498d9-ca02-4417-a786-ef064818b5de'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:03.092631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4be498d9-ca02-4417-a786-ef064818b5de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a2694c8af60455aa9db5997c53adbdc', 'request_id': 'req-4be498d9-ca02-4417-a786-ef064818b5de'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:03.126419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4be498d9-ca02-4417-a786-ef064818b5de admin admin] The token cache is being invalidate because project 1a2694c8af60455aa9db5997c53adbdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:55:03.126787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4be498d9-ca02-4417-a786-ef064818b5de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:03.127941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4be498d9-ca02-4417-a786-ef064818b5de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:03.138743 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1462/2851] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:55:02 2022] DELETE /identity/v3/projects/1a2694c8af60455aa9db5997c53adbdc => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:55:03.140371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9db519-029e-46c6-80a6-38e8a5d9cf06 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:03.140866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9db519-029e-46c6-80a6-38e8a5d9cf06 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:03.141359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9db519-029e-46c6-80a6-38e8a5d9cf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:03.197404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4e9db519-029e-46c6-80a6-38e8a5d9cf06 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:03.200000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4e9db519-029e-46c6-80a6-38e8a5d9cf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:03.339312 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1390/2852] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:03.340241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4a9aa-bf7f-4188-ad2c-6ebb5080a675 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:03.340533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4a9aa-bf7f-4188-ad2c-6ebb5080a675 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:03.340852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4a9aa-bf7f-4188-ad2c-6ebb5080a675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:03.374379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b4f4a9aa-bf7f-4188-ad2c-6ebb5080a675 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:03.375994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b4f4a9aa-bf7f-4188-ad2c-6ebb5080a675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:03.385592 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1463/2853] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:03.391355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9db519-029e-46c6-80a6-38e8a5d9cf06 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:03.403716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9db519-029e-46c6-80a6-38e8a5d9cf06 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:03.411602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fcaf2-cb83-4931-a267-f1f834cf97bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:03.412600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fcaf2-cb83-4931-a267-f1f834cf97bd admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:03.412843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fcaf2-cb83-4931-a267-f1f834cf97bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:03.413115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fcaf2-cb83-4931-a267-f1f834cf97bd admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:03.414029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0fcaf2-cb83-4931-a267-f1f834cf97bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:55:03.414539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0fcaf2-cb83-4931-a267-f1f834cf97bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:03.416345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0fcaf2-cb83-4931-a267-f1f834cf97bd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:03.434538 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1391/2854] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:55:03.436048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4a9aa-bf7f-4188-ad2c-6ebb5080a675 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:03.446582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4a9aa-bf7f-4188-ad2c-6ebb5080a675 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:03.451981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397dcfc-f96a-4501-87f6-e48fc27b6b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:03.452594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397dcfc-f96a-4501-87f6-e48fc27b6b16 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:03.452881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397dcfc-f96a-4501-87f6-e48fc27b6b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:03.453159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397dcfc-f96a-4501-87f6-e48fc27b6b16 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:03.453814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397dcfc-f96a-4501-87f6-e48fc27b6b16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:03.455949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397dcfc-f96a-4501-87f6-e48fc27b6b16 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:03.496314 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1464/2855] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:55:03 2022] POST /identity/v3/projects => generated 397 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:55:03.498204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fcaf2-cb83-4931-a267-f1f834cf97bd admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:03.515062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fcaf2-cb83-4931-a267-f1f834cf97bd admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:03.545691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:03.608663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f6902-b8c4-4084-93ac-b4d648de23ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:03.609204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f6902-b8c4-4084-93ac-b4d648de23ad admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:03.609923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f6902-b8c4-4084-93ac-b4d648de23ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:03.610208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f6902-b8c4-4084-93ac-b4d648de23ad admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:03.610899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f6902-b8c4-4084-93ac-b4d648de23ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:03.613549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f6902-b8c4-4084-93ac-b4d648de23ad admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:03.646079 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1392/2856] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:55:03 2022] POST /identity/v3/users => generated 299 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:55:03.647435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397dcfc-f96a-4501-87f6-e48fc27b6b16 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:03.660298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397dcfc-f96a-4501-87f6-e48fc27b6b16 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:03.668538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d27c4-8ce5-4b88-8ad1-2b83d0d05aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:03.669646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d27c4-8ce5-4b88-8ad1-2b83d0d05aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:03.669646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d27c4-8ce5-4b88-8ad1-2b83d0d05aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:03.669888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d27c4-8ce5-4b88-8ad1-2b83d0d05aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:03.671121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d27c4-8ce5-4b88-8ad1-2b83d0d05aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:03.673813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d27c4-8ce5-4b88-8ad1-2b83d0d05aa2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:03.686680 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1465/2857] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:55:03.690874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f6902-b8c4-4084-93ac-b4d648de23ad admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:03.708840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f6902-b8c4-4084-93ac-b4d648de23ad admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:03.717011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc7b832-2b77-4d4f-9e66-3ec4a6738f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:03.717924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7b832-2b77-4d4f-9e66-3ec4a6738f7a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:03.718976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7b832-2b77-4d4f-9e66-3ec4a6738f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:03.720543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7b832-2b77-4d4f-9e66-3ec4a6738f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:03.721750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc7b832-2b77-4d4f-9e66-3ec4a6738f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:03.723484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc7b832-2b77-4d4f-9e66-3ec4a6738f7a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:03.740596 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1393/2858] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:55:03.742241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d27c4-8ce5-4b88-8ad1-2b83d0d05aa2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:03.754470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d27c4-8ce5-4b88-8ad1-2b83d0d05aa2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:03.759653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130413d-1b61-409e-a28f-31357fec0866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:03.760541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130413d-1b61-409e-a28f-31357fec0866 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:03.761548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130413d-1b61-409e-a28f-31357fec0866 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:03.761828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130413d-1b61-409e-a28f-31357fec0866 admin admin] PATH_INFO: `/v3/projects/0f3af8127d0c4659a2dfd3da10fb480d/users/0358348983a84a7d80a29622a021fa28/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:03.793577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d130413d-1b61-409e-a28f-31357fec0866 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f3af8127d0c4659a2dfd3da10fb480d, user_id=0358348983a84a7d80a29622a021fa28, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:03.799386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d130413d-1b61-409e-a28f-31357fec0866 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:03.816117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:03.851234 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1466/2859] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:55:03 2022] PUT /identity/v3/projects/0f3af8127d0c4659a2dfd3da10fb480d/users/0358348983a84a7d80a29622a021fa28/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:55:03.857999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d94c7-51e0-4ddb-8669-a19b8fcdcd66 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:03.858296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d94c7-51e0-4ddb-8669-a19b8fcdcd66 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:03.858649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d94c7-51e0-4ddb-8669-a19b8fcdcd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:03.927394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e98d94c7-51e0-4ddb-8669-a19b8fcdcd66 None None] MFA Rules not processed for user `0358348983a84a7d80a29622a021fa28`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:03.928666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e98d94c7-51e0-4ddb-8669-a19b8fcdcd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:04.100684 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1394/2860] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:03 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:04.586705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130413d-1b61-409e-a28f-31357fec0866 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:04.591210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d130413d-1b61-409e-a28f-31357fec0866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:04.718090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130413d-1b61-409e-a28f-31357fec0866 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:04.726469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d88ea-3caa-44e4-a450-82c56c823e24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:04.739082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d88ea-3caa-44e4-a450-82c56c823e24 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:04.740622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d88ea-3caa-44e4-a450-82c56c823e24 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:04.741372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d88ea-3caa-44e4-a450-82c56c823e24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:04.767933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d88ea-3caa-44e4-a450-82c56c823e24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:04.771915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d88ea-3caa-44e4-a450-82c56c823e24 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:04.799551 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1467/2861] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:55:04 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:55:04.978346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904ed42-7e10-4b5b-8c9a-df85e957b4e0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:04.979527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904ed42-7e10-4b5b-8c9a-df85e957b4e0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:04.979527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904ed42-7e10-4b5b-8c9a-df85e957b4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:05.013050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6904ed42-7e10-4b5b-8c9a-df85e957b4e0 None None] MFA Rules not processed for user `0358348983a84a7d80a29622a021fa28`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:05.018311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6904ed42-7e10-4b5b-8c9a-df85e957b4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:05.050181 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1395/2862] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:04 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:05.975461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d88ea-3caa-44e4-a450-82c56c823e24 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:06.051974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d88ea-3caa-44e4-a450-82c56c823e24 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:06.056783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67527f-2505-4442-ab22-cbdcd97967d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:06.057456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67527f-2505-4442-ab22-cbdcd97967d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:06.057742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67527f-2505-4442-ab22-cbdcd97967d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:06.058837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67527f-2505-4442-ab22-cbdcd97967d7 admin admin] PATH_INFO: `/v3/users/0358348983a84a7d80a29622a021fa28` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:06.060358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67527f-2505-4442-ab22-cbdcd97967d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0358348983a84a7d80a29622a021fa28)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:06.062287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67527f-2505-4442-ab22-cbdcd97967d7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:06.178856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5a67527f-2505-4442-ab22-cbdcd97967d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0358348983a84a7d80a29622a021fa28'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:06.189024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5a67527f-2505-4442-ab22-cbdcd97967d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0358348983a84a7d80a29622a021fa28'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:06.208085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5a67527f-2505-4442-ab22-cbdcd97967d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0358348983a84a7d80a29622a021fa28'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:06.208856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5a67527f-2505-4442-ab22-cbdcd97967d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0358348983a84a7d80a29622a021fa28'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:06.218366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5a67527f-2505-4442-ab22-cbdcd97967d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0358348983a84a7d80a29622a021fa28'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:06.224885 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1468/2863] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:55:05 2022] DELETE /identity/v3/users/0358348983a84a7d80a29622a021fa28 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:55:06.229454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db416ecc-5419-4e98-a99b-da39f9cd5092 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:06.230269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db416ecc-5419-4e98-a99b-da39f9cd5092 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:06.230617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db416ecc-5419-4e98-a99b-da39f9cd5092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:06.251159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-db416ecc-5419-4e98-a99b-da39f9cd5092 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:06.252357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-db416ecc-5419-4e98-a99b-da39f9cd5092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:06.363619 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1396/2864] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:55:06.374807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67527f-2505-4442-ab22-cbdcd97967d7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:06.377012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5a67527f-2505-4442-ab22-cbdcd97967d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:06.389004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:06.480132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67527f-2505-4442-ab22-cbdcd97967d7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:06.486952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c747ff-ccaf-41ac-b5a0-796373823d5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:06.487926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c747ff-ccaf-41ac-b5a0-796373823d5b service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:06.498890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c747ff-ccaf-41ac-b5a0-796373823d5b service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:06.501875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c747ff-ccaf-41ac-b5a0-796373823d5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:06.523836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c747ff-ccaf-41ac-b5a0-796373823d5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:06.525604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c747ff-ccaf-41ac-b5a0-796373823d5b service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:06.553993 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1469/2865] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:55:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:55:06.943061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db416ecc-5419-4e98-a99b-da39f9cd5092 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:06.944855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-db416ecc-5419-4e98-a99b-da39f9cd5092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:06.970827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db416ecc-5419-4e98-a99b-da39f9cd5092 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:06.978517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6445ba-25d9-48b5-8f47-6a214e0a3367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:06.979598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6445ba-25d9-48b5-8f47-6a214e0a3367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:06.980005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6445ba-25d9-48b5-8f47-6a214e0a3367 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:06.980372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6445ba-25d9-48b5-8f47-6a214e0a3367 admin admin] PATH_INFO: `/v3/projects/0f3af8127d0c4659a2dfd3da10fb480d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:06.990649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6445ba-25d9-48b5-8f47-6a214e0a3367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f3af8127d0c4659a2dfd3da10fb480d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:06.994772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6445ba-25d9-48b5-8f47-6a214e0a3367 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:07.032141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:07.071256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4e6445ba-25d9-48b5-8f47-6a214e0a3367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f3af8127d0c4659a2dfd3da10fb480d', 'request_id': 'req-4e6445ba-25d9-48b5-8f47-6a214e0a3367'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:07.080421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4e6445ba-25d9-48b5-8f47-6a214e0a3367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f3af8127d0c4659a2dfd3da10fb480d', 'request_id': 'req-4e6445ba-25d9-48b5-8f47-6a214e0a3367'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:07.086923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4e6445ba-25d9-48b5-8f47-6a214e0a3367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f3af8127d0c4659a2dfd3da10fb480d', 'request_id': 'req-4e6445ba-25d9-48b5-8f47-6a214e0a3367'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:07.099506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4e6445ba-25d9-48b5-8f47-6a214e0a3367 admin admin] The token cache is being invalidate because project 0f3af8127d0c4659a2dfd3da10fb480d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:55:07.099823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4e6445ba-25d9-48b5-8f47-6a214e0a3367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:07.100479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4e6445ba-25d9-48b5-8f47-6a214e0a3367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:07.101959 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1397/2866] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:55:06 2022] DELETE /identity/v3/projects/0f3af8127d0c4659a2dfd3da10fb480d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:55:07.108355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8a7c3-af33-4dac-9e98-f640b1e68e94 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:07.108894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8a7c3-af33-4dac-9e98-f640b1e68e94 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:07.109011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8a7c3-af33-4dac-9e98-f640b1e68e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:07.130348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-77e8a7c3-af33-4dac-9e98-f640b1e68e94 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:07.132057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-77e8a7c3-af33-4dac-9e98-f640b1e68e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:07.206616 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1470/2867] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:07.214060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce757839-2afb-4e89-83f7-0a7e3bf0ec45 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:07.214428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce757839-2afb-4e89-83f7-0a7e3bf0ec45 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:07.214788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce757839-2afb-4e89-83f7-0a7e3bf0ec45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:07.234962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ce757839-2afb-4e89-83f7-0a7e3bf0ec45 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:07.236173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ce757839-2afb-4e89-83f7-0a7e3bf0ec45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:07.244852 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1398/2868] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:55:07.249807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8a7c3-af33-4dac-9e98-f640b1e68e94 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:07.259878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8a7c3-af33-4dac-9e98-f640b1e68e94 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:07.265190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b736e8-f696-4934-b0b1-f646afe8e44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:07.265799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b736e8-f696-4934-b0b1-f646afe8e44b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:07.266085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b736e8-f696-4934-b0b1-f646afe8e44b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:07.266362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b736e8-f696-4934-b0b1-f646afe8e44b admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:07.267082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b736e8-f696-4934-b0b1-f646afe8e44b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:55:07.267391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b736e8-f696-4934-b0b1-f646afe8e44b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:07.268980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b736e8-f696-4934-b0b1-f646afe8e44b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:07.281207 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1471/2869] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:55:07.285777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce757839-2afb-4e89-83f7-0a7e3bf0ec45 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:07.295668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce757839-2afb-4e89-83f7-0a7e3bf0ec45 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:07.301308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c94e9-2372-4a67-8d19-74763c8f20d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:07.302054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c94e9-2372-4a67-8d19-74763c8f20d9 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:07.302360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c94e9-2372-4a67-8d19-74763c8f20d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:07.302672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c94e9-2372-4a67-8d19-74763c8f20d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:07.303399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c94e9-2372-4a67-8d19-74763c8f20d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:07.305343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c94e9-2372-4a67-8d19-74763c8f20d9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:07.337467 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1399/2870] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:55:07 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:55:07.341706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b736e8-f696-4934-b0b1-f646afe8e44b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:07.353043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b736e8-f696-4934-b0b1-f646afe8e44b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:07.426953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb15f86-f0cc-4503-8950-1ff8534c735d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:07.427551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb15f86-f0cc-4503-8950-1ff8534c735d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:07.427923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb15f86-f0cc-4503-8950-1ff8534c735d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:07.428085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb15f86-f0cc-4503-8950-1ff8534c735d admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:07.428727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb15f86-f0cc-4503-8950-1ff8534c735d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:07.430615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb15f86-f0cc-4503-8950-1ff8534c735d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:07.453822 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1472/2871] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:55:07 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:55:07.457912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c94e9-2372-4a67-8d19-74763c8f20d9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:07.468242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c94e9-2372-4a67-8d19-74763c8f20d9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:07.473042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7055e0f-ba6b-4bc5-a6c4-5e91459a8604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:07.473669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7055e0f-ba6b-4bc5-a6c4-5e91459a8604 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:07.473945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7055e0f-ba6b-4bc5-a6c4-5e91459a8604 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:07.474212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7055e0f-ba6b-4bc5-a6c4-5e91459a8604 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:07.474886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7055e0f-ba6b-4bc5-a6c4-5e91459a8604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:07.476465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7055e0f-ba6b-4bc5-a6c4-5e91459a8604 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:07.491066 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1400/2872] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:55:07.492779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb15f86-f0cc-4503-8950-1ff8534c735d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:07.503981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb15f86-f0cc-4503-8950-1ff8534c735d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:07.510334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c996f8e-368a-47bb-bd4a-260c44559863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:07.511717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c996f8e-368a-47bb-bd4a-260c44559863 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:07.511885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c996f8e-368a-47bb-bd4a-260c44559863 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:07.512407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c996f8e-368a-47bb-bd4a-260c44559863 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:07.513179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c996f8e-368a-47bb-bd4a-260c44559863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:07.514800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c996f8e-368a-47bb-bd4a-260c44559863 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:07.528681 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1473/2873] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:55:07.533319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7055e0f-ba6b-4bc5-a6c4-5e91459a8604 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:07.544707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7055e0f-ba6b-4bc5-a6c4-5e91459a8604 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:07.549993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9793e0c-4206-48eb-a0b1-da64ecd9e81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:07.550761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9793e0c-4206-48eb-a0b1-da64ecd9e81a admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:07.551042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9793e0c-4206-48eb-a0b1-da64ecd9e81a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:07.551311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9793e0c-4206-48eb-a0b1-da64ecd9e81a admin admin] PATH_INFO: `/v3/projects/0edc386b86af4e77bcc8e6663b966c7c/users/7d21433c5d434e6f85d17aad6e21f4f0/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:07.574987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9793e0c-4206-48eb-a0b1-da64ecd9e81a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0edc386b86af4e77bcc8e6663b966c7c, user_id=7d21433c5d434e6f85d17aad6e21f4f0, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:07.578871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9793e0c-4206-48eb-a0b1-da64ecd9e81a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:07.602865 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1401/2874] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:55:07 2022] PUT /identity/v3/projects/0edc386b86af4e77bcc8e6663b966c7c/users/7d21433c5d434e6f85d17aad6e21f4f0/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:55:07.608328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482620fa-25de-4a09-ab8b-d41e679a6d57 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:07.608621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482620fa-25de-4a09-ab8b-d41e679a6d57 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:07.608873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482620fa-25de-4a09-ab8b-d41e679a6d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:07.629200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-482620fa-25de-4a09-ab8b-d41e679a6d57 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:07.630394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-482620fa-25de-4a09-ab8b-d41e679a6d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:07.639351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:07.711366 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1474/2875] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:07.721541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9793e0c-4206-48eb-a0b1-da64ecd9e81a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:07.723306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b9793e0c-4206-48eb-a0b1-da64ecd9e81a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:07.788279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9793e0c-4206-48eb-a0b1-da64ecd9e81a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:07.792596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d127973-b722-4479-97f9-b7d90e4cef85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:07.793285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d127973-b722-4479-97f9-b7d90e4cef85 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:07.793570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d127973-b722-4479-97f9-b7d90e4cef85 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:07.793842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d127973-b722-4479-97f9-b7d90e4cef85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:07.803804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d127973-b722-4479-97f9-b7d90e4cef85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:07.805366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d127973-b722-4479-97f9-b7d90e4cef85 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:07.816185 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1402/2876] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:55:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:55:09.168499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fb698-1af8-4660-b137-898e7d123f22 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:09.169046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fb698-1af8-4660-b137-898e7d123f22 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:09.169758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fb698-1af8-4660-b137-898e7d123f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:09.203001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f80fb698-1af8-4660-b137-898e7d123f22 None None] MFA Rules not processed for user `725f68beba264c2eafc951f8300eba32`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:09.204575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f80fb698-1af8-4660-b137-898e7d123f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:09.286594 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1475/2877] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:09.753508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cb036-2e41-41fc-92e8-99f127fbac6c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:09.753996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cb036-2e41-41fc-92e8-99f127fbac6c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:09.754101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cb036-2e41-41fc-92e8-99f127fbac6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:09.783423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e17cb036-2e41-41fc-92e8-99f127fbac6c None None] MFA Rules not processed for user `e82ed96a3e794a1b99662cec0969b7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:09.785133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e17cb036-2e41-41fc-92e8-99f127fbac6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:09.840718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:09.886672 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1403/2878] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:09.920663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3bd44-a633-4bcb-8941-269437962e88 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:09.920890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3bd44-a633-4bcb-8941-269437962e88 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:09.921174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3bd44-a633-4bcb-8941-269437962e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:09.955190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d1e3bd44-a633-4bcb-8941-269437962e88 None None] MFA Rules not processed for user `7d21433c5d434e6f85d17aad6e21f4f0`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:09.956440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d1e3bd44-a633-4bcb-8941-269437962e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:10.043734 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1476/2879] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:10.119478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e8908-2b1a-4dc7-a603-d4f84f3d56f8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:10.120324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e8908-2b1a-4dc7-a603-d4f84f3d56f8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:10.120546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e8908-2b1a-4dc7-a603-d4f84f3d56f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:10.157274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-717e8908-2b1a-4dc7-a603-d4f84f3d56f8 None None] MFA Rules not processed for user `7d21433c5d434e6f85d17aad6e21f4f0`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:10.159162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-717e8908-2b1a-4dc7-a603-d4f84f3d56f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:10.183013 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1404/2880] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:10.189338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16b40b-9529-4d59-bd0a-691b1d80f94a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:10.190440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16b40b-9529-4d59-bd0a-691b1d80f94a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:10.190937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16b40b-9529-4d59-bd0a-691b1d80f94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:10.226718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6e16b40b-9529-4d59-bd0a-691b1d80f94a None None] MFA Rules not processed for user `7d21433c5d434e6f85d17aad6e21f4f0`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:10.235003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6e16b40b-9529-4d59-bd0a-691b1d80f94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:10.252837 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1477/2881] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:55:10.596711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e8908-2b1a-4dc7-a603-d4f84f3d56f8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:10.599510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-717e8908-2b1a-4dc7-a603-d4f84f3d56f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:10.613842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e8908-2b1a-4dc7-a603-d4f84f3d56f8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:10.619745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478fbc7-283c-4a27-a9ae-f896f8acc80b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:10.620478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478fbc7-283c-4a27-a9ae-f896f8acc80b service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:10.621365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478fbc7-283c-4a27-a9ae-f896f8acc80b service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:10.631936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478fbc7-283c-4a27-a9ae-f896f8acc80b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:10.636781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9478fbc7-283c-4a27-a9ae-f896f8acc80b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:10.654700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478fbc7-283c-4a27-a9ae-f896f8acc80b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:10.667915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478fbc7-283c-4a27-a9ae-f896f8acc80b service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:10.692335 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1405/2882] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:55:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:55:11.171131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16b40b-9529-4d59-bd0a-691b1d80f94a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:11.173315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6e16b40b-9529-4d59-bd0a-691b1d80f94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:11.292885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:11.394651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16b40b-9529-4d59-bd0a-691b1d80f94a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:11.429288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70648954-5bdc-4107-ab76-b63bcd983ef9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:11.430364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70648954-5bdc-4107-ab76-b63bcd983ef9 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:11.430896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70648954-5bdc-4107-ab76-b63bcd983ef9 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:11.431401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70648954-5bdc-4107-ab76-b63bcd983ef9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:11.457550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70648954-5bdc-4107-ab76-b63bcd983ef9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:11.459347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70648954-5bdc-4107-ab76-b63bcd983ef9 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:11.497032 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1478/2883] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:55:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:55:13.014860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6bfcb-3366-49f0-9b64-fa68fe9a443c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:13.015795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6bfcb-3366-49f0-9b64-fa68fe9a443c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:13.015795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6bfcb-3366-49f0-9b64-fa68fe9a443c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:13.080023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-05a6bfcb-3366-49f0-9b64-fa68fe9a443c None None] MFA Rules not processed for user `7d21433c5d434e6f85d17aad6e21f4f0`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:13.082459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-05a6bfcb-3366-49f0-9b64-fa68fe9a443c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:13.108078 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1406/2884] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:15.271345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bef6f-6bb4-4ac8-9c8b-0089713480c7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:15.271966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bef6f-6bb4-4ac8-9c8b-0089713480c7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:15.272154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bef6f-6bb4-4ac8-9c8b-0089713480c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:15.313537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-081bef6f-6bb4-4ac8-9c8b-0089713480c7 None None] MFA Rules not processed for user `7d21433c5d434e6f85d17aad6e21f4f0`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:15.316921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-081bef6f-6bb4-4ac8-9c8b-0089713480c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:15.343539 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1479/2885] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:17.808848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbf769-24eb-4e04-94fa-5b575dcae7a4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:17.809396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbf769-24eb-4e04-94fa-5b575dcae7a4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:17.809396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbf769-24eb-4e04-94fa-5b575dcae7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:17.829448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3cdbf769-24eb-4e04-94fa-5b575dcae7a4 None None] MFA Rules not processed for user `a749a74841e840e2b8de67fb9562bf51`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:17.830701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3cdbf769-24eb-4e04-94fa-5b575dcae7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:17.888793 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1407/2886] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:25.296768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=18 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:27.881729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=42 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:29.585669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63957295-9622-4258-9ffc-26a85b858763 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:29.586434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63957295-9622-4258-9ffc-26a85b858763 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:29.586747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63957295-9622-4258-9ffc-26a85b858763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:29.647344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-63957295-9622-4258-9ffc-26a85b858763 None None] MFA Rules not processed for user `e82ed96a3e794a1b99662cec0969b7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:29.648787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-63957295-9622-4258-9ffc-26a85b858763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:29.674687 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1480/2887] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:30.103427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbf769-24eb-4e04-94fa-5b575dcae7a4 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:30.105401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3cdbf769-24eb-4e04-94fa-5b575dcae7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:30.121941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbf769-24eb-4e04-94fa-5b575dcae7a4 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:30.128809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c33b3-6587-4a38-8258-5c4e4e2a87e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:30.129840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c33b3-6587-4a38-8258-5c4e4e2a87e1 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:30.130279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c33b3-6587-4a38-8258-5c4e4e2a87e1 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:30.130759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c33b3-6587-4a38-8258-5c4e4e2a87e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:30.144286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ab2c33b3-6587-4a38-8258-5c4e4e2a87e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:30.270851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c33b3-6587-4a38-8258-5c4e4e2a87e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:30.273260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c33b3-6587-4a38-8258-5c4e4e2a87e1 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:30.284760 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1408/2888] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:55:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:55:36.351543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63957295-9622-4258-9ffc-26a85b858763 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:36.365693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63957295-9622-4258-9ffc-26a85b858763 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:36.376006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c84cc0-f625-4b82-af43-d6822cd46917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:36.376644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c84cc0-f625-4b82-af43-d6822cd46917 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:36.376934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c84cc0-f625-4b82-af43-d6822cd46917 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:36.377216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c84cc0-f625-4b82-af43-d6822cd46917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:36.386532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-77c84cc0-f625-4b82-af43-d6822cd46917 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:36.405099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c84cc0-f625-4b82-af43-d6822cd46917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:36.407091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c84cc0-f625-4b82-af43-d6822cd46917 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:36.419760 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1481/2889] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:55:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:55:40.280080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=30 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:41.007028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c33b3-6587-4a38-8258-5c4e4e2a87e1 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:41.008519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ab2c33b3-6587-4a38-8258-5c4e4e2a87e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:41.124891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c33b3-6587-4a38-8258-5c4e4e2a87e1 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:41.142868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcc76a-53a7-44dc-ad89-88859945c8a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:41.143542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcc76a-53a7-44dc-ad89-88859945c8a4 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:41.143843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcc76a-53a7-44dc-ad89-88859945c8a4 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:41.144110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcc76a-53a7-44dc-ad89-88859945c8a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:41.147679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e8bcc76a-53a7-44dc-ad89-88859945c8a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:41.161919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bcc76a-53a7-44dc-ad89-88859945c8a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:41.164388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bcc76a-53a7-44dc-ad89-88859945c8a4 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:41.176644 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1409/2890] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:55:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:55:42.444929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c84cc0-f625-4b82-af43-d6822cd46917 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:42.457366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c84cc0-f625-4b82-af43-d6822cd46917 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:42.474756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca53c99-05d1-4450-9599-4dc690db4397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:42.475565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca53c99-05d1-4450-9599-4dc690db4397 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:42.475895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca53c99-05d1-4450-9599-4dc690db4397 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:42.476333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca53c99-05d1-4450-9599-4dc690db4397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:42.479078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-eca53c99-05d1-4450-9599-4dc690db4397 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:42.541349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca53c99-05d1-4450-9599-4dc690db4397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:42.550714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca53c99-05d1-4450-9599-4dc690db4397 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:42.580009 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1482/2891] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:55:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:55:42.679166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcc76a-53a7-44dc-ad89-88859945c8a4 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:42.694849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcc76a-53a7-44dc-ad89-88859945c8a4 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:42.698926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0b2af-a753-4386-bdd9-c173224e2004 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:42.699749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0b2af-a753-4386-bdd9-c173224e2004 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:42.700176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0b2af-a753-4386-bdd9-c173224e2004 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:42.700601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0b2af-a753-4386-bdd9-c173224e2004 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:42.702528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7ca0b2af-a753-4386-bdd9-c173224e2004 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:42.730689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0b2af-a753-4386-bdd9-c173224e2004 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:42.732752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0b2af-a753-4386-bdd9-c173224e2004 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:42.743484 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1410/2892] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:55:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:55:43.007193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca53c99-05d1-4450-9599-4dc690db4397 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:43.010742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-eca53c99-05d1-4450-9599-4dc690db4397 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:43.047793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca53c99-05d1-4450-9599-4dc690db4397 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:43.053238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c151d-e12b-420e-931e-ff42448d6bd1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:43.053982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c151d-e12b-420e-931e-ff42448d6bd1 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:43.054335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c151d-e12b-420e-931e-ff42448d6bd1 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:43.054730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c151d-e12b-420e-931e-ff42448d6bd1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:43.056826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-274c151d-e12b-420e-931e-ff42448d6bd1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:43.074702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c151d-e12b-420e-931e-ff42448d6bd1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:43.076799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c151d-e12b-420e-931e-ff42448d6bd1 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:43.091242 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1483/2893] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:55:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:55:47.318113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0b2af-a753-4386-bdd9-c173224e2004 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:47.320205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7ca0b2af-a753-4386-bdd9-c173224e2004 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:47.338733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0b2af-a753-4386-bdd9-c173224e2004 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:47.345153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d13ab-4298-47ba-ac10-2c5ee0e7c5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:47.345951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d13ab-4298-47ba-ac10-2c5ee0e7c5d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:47.346482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d13ab-4298-47ba-ac10-2c5ee0e7c5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:47.346789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d13ab-4298-47ba-ac10-2c5ee0e7c5d7 admin admin] PATH_INFO: `/v3/users/aa72870453cd4b958725801b842c7a28` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:47.347948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d13ab-4298-47ba-ac10-2c5ee0e7c5d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa72870453cd4b958725801b842c7a28)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:47.349864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d13ab-4298-47ba-ac10-2c5ee0e7c5d7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:47.404024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-151d13ab-4298-47ba-ac10-2c5ee0e7c5d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa72870453cd4b958725801b842c7a28'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:47.404784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-151d13ab-4298-47ba-ac10-2c5ee0e7c5d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa72870453cd4b958725801b842c7a28'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:47.405453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-151d13ab-4298-47ba-ac10-2c5ee0e7c5d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa72870453cd4b958725801b842c7a28'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:47.419144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-151d13ab-4298-47ba-ac10-2c5ee0e7c5d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa72870453cd4b958725801b842c7a28'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:47.434662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-151d13ab-4298-47ba-ac10-2c5ee0e7c5d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa72870453cd4b958725801b842c7a28'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:47.447698 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1411/2894] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:55:47 2022] DELETE /identity/v3/users/aa72870453cd4b958725801b842c7a28 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:55:47.651169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c151d-e12b-420e-931e-ff42448d6bd1 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:47.667496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c151d-e12b-420e-931e-ff42448d6bd1 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:47.733583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d97881-5e7c-44a3-951f-e487b4ba48d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:47.734226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d97881-5e7c-44a3-951f-e487b4ba48d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:47.734533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d97881-5e7c-44a3-951f-e487b4ba48d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:47.734817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d97881-5e7c-44a3-951f-e487b4ba48d5 admin admin] PATH_INFO: `/v3/projects/76e0a9843d9c41f7b2fae591dc09d9d2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:47.736158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d97881-5e7c-44a3-951f-e487b4ba48d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76e0a9843d9c41f7b2fae591dc09d9d2)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:47.738121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d97881-5e7c-44a3-951f-e487b4ba48d5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:47.798457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-82d97881-5e7c-44a3-951f-e487b4ba48d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76e0a9843d9c41f7b2fae591dc09d9d2', 'request_id': 'req-82d97881-5e7c-44a3-951f-e487b4ba48d5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:47.809011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-82d97881-5e7c-44a3-951f-e487b4ba48d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76e0a9843d9c41f7b2fae591dc09d9d2', 'request_id': 'req-82d97881-5e7c-44a3-951f-e487b4ba48d5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:47.814857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-82d97881-5e7c-44a3-951f-e487b4ba48d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76e0a9843d9c41f7b2fae591dc09d9d2', 'request_id': 'req-82d97881-5e7c-44a3-951f-e487b4ba48d5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:47.836541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-82d97881-5e7c-44a3-951f-e487b4ba48d5 admin admin] The token cache is being invalidate because project 76e0a9843d9c41f7b2fae591dc09d9d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:55:47.837566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-82d97881-5e7c-44a3-951f-e487b4ba48d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:47.838538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-82d97881-5e7c-44a3-951f-e487b4ba48d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:47.840312 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1484/2895] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:55:47 2022] DELETE /identity/v3/projects/76e0a9843d9c41f7b2fae591dc09d9d2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:55:47.844335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d13ab-4298-47ba-ac10-2c5ee0e7c5d7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:47.846335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-151d13ab-4298-47ba-ac10-2c5ee0e7c5d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:47.865727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d13ab-4298-47ba-ac10-2c5ee0e7c5d7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:47.920729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:47.937360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33506652-7325-4b02-8d00-48141fc58d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:47.938012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33506652-7325-4b02-8d00-48141fc58d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:47.938295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33506652-7325-4b02-8d00-48141fc58d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:47.938606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33506652-7325-4b02-8d00-48141fc58d8e admin admin] PATH_INFO: `/v3/users/bddb1c3a5b754575a5e6cd717c7c593f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:47.939788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33506652-7325-4b02-8d00-48141fc58d8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bddb1c3a5b754575a5e6cd717c7c593f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:47.941727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33506652-7325-4b02-8d00-48141fc58d8e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:48.001184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-33506652-7325-4b02-8d00-48141fc58d8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bddb1c3a5b754575a5e6cd717c7c593f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:48.001903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-33506652-7325-4b02-8d00-48141fc58d8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bddb1c3a5b754575a5e6cd717c7c593f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:48.002551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-33506652-7325-4b02-8d00-48141fc58d8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bddb1c3a5b754575a5e6cd717c7c593f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:48.012835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-33506652-7325-4b02-8d00-48141fc58d8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bddb1c3a5b754575a5e6cd717c7c593f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:48.032605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-33506652-7325-4b02-8d00-48141fc58d8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bddb1c3a5b754575a5e6cd717c7c593f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:48.056301 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1412/2896] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:55:47 2022] DELETE /identity/v3/users/bddb1c3a5b754575a5e6cd717c7c593f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:55:48.274708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d97881-5e7c-44a3-951f-e487b4ba48d5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:48.276711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-82d97881-5e7c-44a3-951f-e487b4ba48d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:48.291332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d97881-5e7c-44a3-951f-e487b4ba48d5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:48.335740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:48.354948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc560b-1f31-45df-844f-a4caf0e6de38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:48.355578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc560b-1f31-45df-844f-a4caf0e6de38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:48.355856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc560b-1f31-45df-844f-a4caf0e6de38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:48.356132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc560b-1f31-45df-844f-a4caf0e6de38 admin admin] PATH_INFO: `/v3/projects/9e40f93ce5d54331901307bd6ced39a9` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:48.357303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc560b-1f31-45df-844f-a4caf0e6de38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e40f93ce5d54331901307bd6ced39a9)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:48.359252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc560b-1f31-45df-844f-a4caf0e6de38 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:48.406245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f7cc560b-1f31-45df-844f-a4caf0e6de38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e40f93ce5d54331901307bd6ced39a9', 'request_id': 'req-f7cc560b-1f31-45df-844f-a4caf0e6de38'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:48.412379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f7cc560b-1f31-45df-844f-a4caf0e6de38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e40f93ce5d54331901307bd6ced39a9', 'request_id': 'req-f7cc560b-1f31-45df-844f-a4caf0e6de38'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:48.418602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f7cc560b-1f31-45df-844f-a4caf0e6de38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e40f93ce5d54331901307bd6ced39a9', 'request_id': 'req-f7cc560b-1f31-45df-844f-a4caf0e6de38'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:48.435464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f7cc560b-1f31-45df-844f-a4caf0e6de38 admin admin] The token cache is being invalidate because project 9e40f93ce5d54331901307bd6ced39a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:55:48.435674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f7cc560b-1f31-45df-844f-a4caf0e6de38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:48.436332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f7cc560b-1f31-45df-844f-a4caf0e6de38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:48.438104 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1485/2897] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:55:48 2022] DELETE /identity/v3/projects/9e40f93ce5d54331901307bd6ced39a9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:55:48.443549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71231073-47cf-458e-8c8d-3531abebf037 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:48.444135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71231073-47cf-458e-8c8d-3531abebf037 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:48.444257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71231073-47cf-458e-8c8d-3531abebf037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:48.465542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-71231073-47cf-458e-8c8d-3531abebf037 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:48.466946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-71231073-47cf-458e-8c8d-3531abebf037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:48.533922 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1413/2898] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:48.542420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443b975-2203-4ebc-a8ed-b78869db70a8 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:48.542745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443b975-2203-4ebc-a8ed-b78869db70a8 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:48.543023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443b975-2203-4ebc-a8ed-b78869db70a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:48.566721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0443b975-2203-4ebc-a8ed-b78869db70a8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:48.568013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0443b975-2203-4ebc-a8ed-b78869db70a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:48.581196 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1486/2899] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:48.582860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71231073-47cf-458e-8c8d-3531abebf037 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:48.595667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71231073-47cf-458e-8c8d-3531abebf037 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:48.600911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d103b5-0d05-48a2-a742-6f733dd91a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:48.601780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d103b5-0d05-48a2-a742-6f733dd91a4e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:48.602290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d103b5-0d05-48a2-a742-6f733dd91a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:48.603205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d103b5-0d05-48a2-a742-6f733dd91a4e admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:48.604096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d103b5-0d05-48a2-a742-6f733dd91a4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:55:48.604494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d103b5-0d05-48a2-a742-6f733dd91a4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:48.606411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d103b5-0d05-48a2-a742-6f733dd91a4e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:48.622543 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1414/2900] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:55:48.625154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443b975-2203-4ebc-a8ed-b78869db70a8 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:48.635282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443b975-2203-4ebc-a8ed-b78869db70a8 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:48.640171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49283ed6-5da6-4ee2-ac0f-2be75cce0a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:48.640782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49283ed6-5da6-4ee2-ac0f-2be75cce0a0d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:48.641063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49283ed6-5da6-4ee2-ac0f-2be75cce0a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:48.641338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49283ed6-5da6-4ee2-ac0f-2be75cce0a0d admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:48.642159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49283ed6-5da6-4ee2-ac0f-2be75cce0a0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:48.644340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49283ed6-5da6-4ee2-ac0f-2be75cce0a0d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:48.681301 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1487/2901] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:55:48 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:55:48.687379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d103b5-0d05-48a2-a742-6f733dd91a4e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:48.699130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d103b5-0d05-48a2-a742-6f733dd91a4e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:48.753731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:48.763021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d49b43-442f-4993-a6db-864110af5014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:48.763828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d49b43-442f-4993-a6db-864110af5014 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:48.764046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d49b43-442f-4993-a6db-864110af5014 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:48.764606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d49b43-442f-4993-a6db-864110af5014 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:48.765456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d49b43-442f-4993-a6db-864110af5014 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:48.767690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d49b43-442f-4993-a6db-864110af5014 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:48.794554 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1415/2902] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:55:48 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:55:48.796148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49283ed6-5da6-4ee2-ac0f-2be75cce0a0d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:48.808289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49283ed6-5da6-4ee2-ac0f-2be75cce0a0d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:48.814207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0aaf7-4311-44cb-9e9d-c3f7dfb9452d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:48.814851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0aaf7-4311-44cb-9e9d-c3f7dfb9452d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:48.815132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0aaf7-4311-44cb-9e9d-c3f7dfb9452d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:48.815435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0aaf7-4311-44cb-9e9d-c3f7dfb9452d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:48.816091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be0aaf7-4311-44cb-9e9d-c3f7dfb9452d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:48.817680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be0aaf7-4311-44cb-9e9d-c3f7dfb9452d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:48.830534 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1488/2903] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:55:48.832815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d49b43-442f-4993-a6db-864110af5014 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:48.843790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d49b43-442f-4993-a6db-864110af5014 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:48.851293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989747cd-a7e7-4684-b004-9515d0d5bd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:48.851902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989747cd-a7e7-4684-b004-9515d0d5bd91 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:48.852277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989747cd-a7e7-4684-b004-9515d0d5bd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:48.852637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989747cd-a7e7-4684-b004-9515d0d5bd91 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:48.853283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989747cd-a7e7-4684-b004-9515d0d5bd91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:48.854919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989747cd-a7e7-4684-b004-9515d0d5bd91 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:48.879057 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1416/2904] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:55:48.879322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0aaf7-4311-44cb-9e9d-c3f7dfb9452d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:48.890335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0aaf7-4311-44cb-9e9d-c3f7dfb9452d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:48.896514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1546df-ba05-41e1-83a3-b414ca1dac3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:48.897182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1546df-ba05-41e1-83a3-b414ca1dac3b admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:48.897469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1546df-ba05-41e1-83a3-b414ca1dac3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:48.897748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1546df-ba05-41e1-83a3-b414ca1dac3b admin admin] PATH_INFO: `/v3/projects/55363ebfb9024c75bf8249b8f088320e/users/877f6cb8e4864e09940ad582e55008f8/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:48.916901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1546df-ba05-41e1-83a3-b414ca1dac3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=55363ebfb9024c75bf8249b8f088320e, user_id=877f6cb8e4864e09940ad582e55008f8, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:48.919598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1546df-ba05-41e1-83a3-b414ca1dac3b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:48.933752 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1489/2905] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:55:48 2022] PUT /identity/v3/projects/55363ebfb9024c75bf8249b8f088320e/users/877f6cb8e4864e09940ad582e55008f8/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:55:48.945494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c90a93-5b6d-463b-8627-453c8fd64086 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:48.945681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c90a93-5b6d-463b-8627-453c8fd64086 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:48.945949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c90a93-5b6d-463b-8627-453c8fd64086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:48.986144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-96c90a93-5b6d-463b-8627-453c8fd64086 None None] MFA Rules not processed for user `877f6cb8e4864e09940ad582e55008f8`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:48.987161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-96c90a93-5b6d-463b-8627-453c8fd64086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:49.087876 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1417/2906] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:49.092487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1546df-ba05-41e1-83a3-b414ca1dac3b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:49.173605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1546df-ba05-41e1-83a3-b414ca1dac3b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:49.178409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58611b30-7e22-4361-aaf5-5754556fbfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:49.179086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58611b30-7e22-4361-aaf5-5754556fbfe6 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:49.179366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58611b30-7e22-4361-aaf5-5754556fbfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:49.179634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58611b30-7e22-4361-aaf5-5754556fbfe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:49.180299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58611b30-7e22-4361-aaf5-5754556fbfe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:49.182911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58611b30-7e22-4361-aaf5-5754556fbfe6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:49.199117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:49.214347 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1490/2907] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:55:49 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:55:49.218280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c90a93-5b6d-463b-8627-453c8fd64086 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:49.229788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c90a93-5b6d-463b-8627-453c8fd64086 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:49.309534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624ee36-0b0b-4186-9f4d-c8f6962f70bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:49.309880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624ee36-0b0b-4186-9f4d-c8f6962f70bc admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:49.310163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624ee36-0b0b-4186-9f4d-c8f6962f70bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:49.310437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624ee36-0b0b-4186-9f4d-c8f6962f70bc admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:49.311126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624ee36-0b0b-4186-9f4d-c8f6962f70bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:49.313459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624ee36-0b0b-4186-9f4d-c8f6962f70bc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:49.341498 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1418/2908] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:55:49 2022] POST /identity/v3/users => generated 294 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:55:49.345567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58611b30-7e22-4361-aaf5-5754556fbfe6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:49.356886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58611b30-7e22-4361-aaf5-5754556fbfe6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:49.361836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ab353-72a7-4e47-9b73-d34d3cae92e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:49.362439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ab353-72a7-4e47-9b73-d34d3cae92e7 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:49.362741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ab353-72a7-4e47-9b73-d34d3cae92e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:49.363028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ab353-72a7-4e47-9b73-d34d3cae92e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:49.363650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ab353-72a7-4e47-9b73-d34d3cae92e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:49.365233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ab353-72a7-4e47-9b73-d34d3cae92e7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:49.377255 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1491/2909] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:55:49.381644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624ee36-0b0b-4186-9f4d-c8f6962f70bc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:49.393836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624ee36-0b0b-4186-9f4d-c8f6962f70bc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:49.400977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcdc239-13f6-4b15-a387-da46c69e4f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:49.401941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcdc239-13f6-4b15-a387-da46c69e4f0e admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:49.402320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcdc239-13f6-4b15-a387-da46c69e4f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:49.402734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcdc239-13f6-4b15-a387-da46c69e4f0e admin admin] PATH_INFO: `/v3/projects/7c2e9b5442ab48c9af69ab0d0e94b55e/users/ccdeb939d99a40dd84b2930a8b05214f/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:49.422160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcdc239-13f6-4b15-a387-da46c69e4f0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c2e9b5442ab48c9af69ab0d0e94b55e, user_id=ccdeb939d99a40dd84b2930a8b05214f, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:49.424903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcdc239-13f6-4b15-a387-da46c69e4f0e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:49.439916 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1419/2910] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:55:49 2022] PUT /identity/v3/projects/7c2e9b5442ab48c9af69ab0d0e94b55e/users/ccdeb939d99a40dd84b2930a8b05214f/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:55:49.444252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ab353-72a7-4e47-9b73-d34d3cae92e7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:49.454837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ab353-72a7-4e47-9b73-d34d3cae92e7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:49.510437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ae32b-d626-431a-b0c3-20c8224278a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:49.511156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ae32b-d626-431a-b0c3-20c8224278a5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:49.511432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ae32b-d626-431a-b0c3-20c8224278a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:49.511700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ae32b-d626-431a-b0c3-20c8224278a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:49.512490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ae32b-d626-431a-b0c3-20c8224278a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:49.514066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ae32b-d626-431a-b0c3-20c8224278a5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:49.524163 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1492/2911] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:55:49.528984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcdc239-13f6-4b15-a387-da46c69e4f0e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:49.540614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcdc239-13f6-4b15-a387-da46c69e4f0e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:49.545278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d11b1-d985-40a5-b9be-a19a67c12cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:49.545969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d11b1-d985-40a5-b9be-a19a67c12cc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:49.546241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d11b1-d985-40a5-b9be-a19a67c12cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:49.546538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d11b1-d985-40a5-b9be-a19a67c12cc6 admin admin] PATH_INFO: `/v3/projects/7c2e9b5442ab48c9af69ab0d0e94b55e/users/ccdeb939d99a40dd84b2930a8b05214f/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:49.548422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d11b1-d985-40a5-b9be-a19a67c12cc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c2e9b5442ab48c9af69ab0d0e94b55e, user_id=ccdeb939d99a40dd84b2930a8b05214f, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:49.551067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d11b1-d985-40a5-b9be-a19a67c12cc6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:49.570169 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1420/2912] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:55:49 2022] PUT /identity/v3/projects/7c2e9b5442ab48c9af69ab0d0e94b55e/users/ccdeb939d99a40dd84b2930a8b05214f/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:55:49.580380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aeddfe-463a-4b84-9d37-ae9dea3a7ed5 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:49.580667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aeddfe-463a-4b84-9d37-ae9dea3a7ed5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:49.580946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aeddfe-463a-4b84-9d37-ae9dea3a7ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:49.609857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e1aeddfe-463a-4b84-9d37-ae9dea3a7ed5 None None] MFA Rules not processed for user `ccdeb939d99a40dd84b2930a8b05214f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:49.611107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e1aeddfe-463a-4b84-9d37-ae9dea3a7ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:49.678090 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1493/2913] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:49.685549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d11b1-d985-40a5-b9be-a19a67c12cc6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:49.696999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d11b1-d985-40a5-b9be-a19a67c12cc6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:49.701486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f551e-0491-437e-bfc2-0c1f82662965 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccdeb939d99a40dd84b2930a8b05214f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7c2e9b5442ab48c9af69ab0d0e94b55e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:49.702074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f551e-0491-437e-bfc2-0c1f82662965 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:49.702355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f551e-0491-437e-bfc2-0c1f82662965 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:49.702662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f551e-0491-437e-bfc2-0c1f82662965 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:49.703307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f551e-0491-437e-bfc2-0c1f82662965 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:55:49.703609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f551e-0491-437e-bfc2-0c1f82662965 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:49.705646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f551e-0491-437e-bfc2-0c1f82662965 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:49.718243 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1421/2914] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:55:49.722516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aeddfe-463a-4b84-9d37-ae9dea3a7ed5 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:49.731740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aeddfe-463a-4b84-9d37-ae9dea3a7ed5 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:49.736408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf1f7f-7fbb-447d-abf0-749d7b7247cd tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccdeb939d99a40dd84b2930a8b05214f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7c2e9b5442ab48c9af69ab0d0e94b55e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:49.737035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf1f7f-7fbb-447d-abf0-749d7b7247cd tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:49.737322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf1f7f-7fbb-447d-abf0-749d7b7247cd tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:49.737585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf1f7f-7fbb-447d-abf0-749d7b7247cd tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:49.738211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf1f7f-7fbb-447d-abf0-749d7b7247cd tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:49.740138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf1f7f-7fbb-447d-abf0-749d7b7247cd tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:49.774603 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1494/2915] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:55:49 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:55:49.776112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f551e-0491-437e-bfc2-0c1f82662965 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:49.782781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:49.789177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f551e-0491-437e-bfc2-0c1f82662965 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:49.850324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d1943-cf23-4d36-bc79-f1e6a802eabf tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccdeb939d99a40dd84b2930a8b05214f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7c2e9b5442ab48c9af69ab0d0e94b55e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:49.850980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d1943-cf23-4d36-bc79-f1e6a802eabf tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:49.851303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d1943-cf23-4d36-bc79-f1e6a802eabf tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:49.851536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d1943-cf23-4d36-bc79-f1e6a802eabf tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:49.852185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d1943-cf23-4d36-bc79-f1e6a802eabf tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:55:49.853008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d1943-cf23-4d36-bc79-f1e6a802eabf tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:49.854644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d1943-cf23-4d36-bc79-f1e6a802eabf tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:49.874548 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1422/2916] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:55:49.887831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf1f7f-7fbb-447d-abf0-749d7b7247cd tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:49.890153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-09cf1f7f-7fbb-447d-abf0-749d7b7247cd tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:49.941834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:49.970846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf1f7f-7fbb-447d-abf0-749d7b7247cd tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:49.976610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f22a91-128a-416d-a170-ebbc9233805b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:49.977223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f22a91-128a-416d-a170-ebbc9233805b service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:49.978029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f22a91-128a-416d-a170-ebbc9233805b service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:49.978339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f22a91-128a-416d-a170-ebbc9233805b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:49.987012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f22a91-128a-416d-a170-ebbc9233805b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:49.988829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f22a91-128a-416d-a170-ebbc9233805b service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:49.999440 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1495/2917] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:55:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:55:50.015988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94681c-a696-4866-95d9-3cdf75b69826 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:50.016269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94681c-a696-4866-95d9-3cdf75b69826 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:50.016540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94681c-a696-4866-95d9-3cdf75b69826 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:50.017863 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1423/2918] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:55:50.022648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f22a91-128a-416d-a170-ebbc9233805b service nova] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:55:50.034696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f22a91-128a-416d-a170-ebbc9233805b service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:50.044708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f22a91-128a-416d-a170-ebbc9233805b service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:50.049560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9ebf9-0bca-403d-a57d-b8ae5ba836ad tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccdeb939d99a40dd84b2930a8b05214f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7c2e9b5442ab48c9af69ab0d0e94b55e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:50.050192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9ebf9-0bca-403d-a57d-b8ae5ba836ad tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:50.050480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9ebf9-0bca-403d-a57d-b8ae5ba836ad tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:50.050788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9ebf9-0bca-403d-a57d-b8ae5ba836ad tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] PATH_INFO: `/v3/projects/51030f254d594921b621dc3651ea9173` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:50.051974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d9ebf9-0bca-403d-a57d-b8ae5ba836ad tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorizing `identity:get_project(project_id=51030f254d594921b621dc3651ea9173)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:50.054041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d9ebf9-0bca-403d-a57d-b8ae5ba836ad tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:50.056376 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1496/2919] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:55:50 2022] GET /identity/v3/projects/51030f254d594921b621dc3651ea9173 => generated 369 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:55:50.180156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e16f1-5996-49c5-9955-9b62f9f29f72 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:50.180437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e16f1-5996-49c5-9955-9b62f9f29f72 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:50.180712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e16f1-5996-49c5-9955-9b62f9f29f72 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:50.182008 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1424/2920] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:55:50.187175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9ebf9-0bca-403d-a57d-b8ae5ba836ad tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:55:50.189367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f9d9ebf9-0bca-403d-a57d-b8ae5ba836ad tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:50.206320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9ebf9-0bca-403d-a57d-b8ae5ba836ad tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:50.215951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9ebf9-0bca-403d-a57d-b8ae5ba836ad tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:50.220656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4919fb-57a0-4ce6-955e-ebc52190bd05 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccdeb939d99a40dd84b2930a8b05214f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7c2e9b5442ab48c9af69ab0d0e94b55e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:50.221345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4919fb-57a0-4ce6-955e-ebc52190bd05 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:50.221569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4919fb-57a0-4ce6-955e-ebc52190bd05 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:50.221847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4919fb-57a0-4ce6-955e-ebc52190bd05 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] PATH_INFO: `/v3/projects/51030f254d594921b621dc3651ea9173` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:50.223031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4919fb-57a0-4ce6-955e-ebc52190bd05 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorizing `identity:get_project(project_id=51030f254d594921b621dc3651ea9173)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:50.225075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4919fb-57a0-4ce6-955e-ebc52190bd05 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:50.226786 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1497/2921] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:55:50 2022] GET /identity/v3/projects/51030f254d594921b621dc3651ea9173 => generated 369 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:55:50.289551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e16f1-5996-49c5-9955-9b62f9f29f72 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:50.306323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e16f1-5996-49c5-9955-9b62f9f29f72 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:50.311375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2f75b-d8fa-45b2-8f74-af0a1d6f90ca tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccdeb939d99a40dd84b2930a8b05214f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7c2e9b5442ab48c9af69ab0d0e94b55e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:50.312387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2f75b-d8fa-45b2-8f74-af0a1d6f90ca tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:50.312670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2f75b-d8fa-45b2-8f74-af0a1d6f90ca tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:50.313557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2f75b-d8fa-45b2-8f74-af0a1d6f90ca tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:50.314217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2f75b-d8fa-45b2-8f74-af0a1d6f90ca tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:55:50.314557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2f75b-d8fa-45b2-8f74-af0a1d6f90ca tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:50.316337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2f75b-d8fa-45b2-8f74-af0a1d6f90ca tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:50.330241 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1425/2922] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:55:50.335837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4919fb-57a0-4ce6-955e-ebc52190bd05 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:50.346720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4919fb-57a0-4ce6-955e-ebc52190bd05 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:50.356698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4095a-f784-445e-b999-c00c69430479 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccdeb939d99a40dd84b2930a8b05214f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7c2e9b5442ab48c9af69ab0d0e94b55e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:50.357353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4095a-f784-445e-b999-c00c69430479 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:50.357634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4095a-f784-445e-b999-c00c69430479 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:50.357911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4095a-f784-445e-b999-c00c69430479 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:50.358600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b4095a-f784-445e-b999-c00c69430479 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:50.360753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b4095a-f784-445e-b999-c00c69430479 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:50.385376 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1498/2923] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:55:50 2022] POST /identity/v3/users => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:55:50.390347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2f75b-d8fa-45b2-8f74-af0a1d6f90ca tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:50.425253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2f75b-d8fa-45b2-8f74-af0a1d6f90ca tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:50.432384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325b187-4cf8-45d6-ba98-a1ef39389b0d tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccdeb939d99a40dd84b2930a8b05214f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7c2e9b5442ab48c9af69ab0d0e94b55e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:50.433948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325b187-4cf8-45d6-ba98-a1ef39389b0d tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:50.434476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325b187-4cf8-45d6-ba98-a1ef39389b0d tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:50.435097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325b187-4cf8-45d6-ba98-a1ef39389b0d tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:50.436627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325b187-4cf8-45d6-ba98-a1ef39389b0d tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:55:50.437545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325b187-4cf8-45d6-ba98-a1ef39389b0d tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:50.439933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325b187-4cf8-45d6-ba98-a1ef39389b0d tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:50.454376 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1426/2924] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:55:50.743657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b141af-040a-45c9-a28d-72e1c93efbf2 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:50.744194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b141af-040a-45c9-a28d-72e1c93efbf2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:50.744194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b141af-040a-45c9-a28d-72e1c93efbf2 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:50.745782 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1499/2925] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:55:50.750256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325b187-4cf8-45d6-ba98-a1ef39389b0d tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Authenticating service token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:55:50.763905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325b187-4cf8-45d6-ba98-a1ef39389b0d tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:50.774052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325b187-4cf8-45d6-ba98-a1ef39389b0d tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:50.779780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee023c-a631-452c-9685-3083b692d71d tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccdeb939d99a40dd84b2930a8b05214f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7c2e9b5442ab48c9af69ab0d0e94b55e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:50.780456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee023c-a631-452c-9685-3083b692d71d tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:50.780771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee023c-a631-452c-9685-3083b692d71d tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:50.781055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee023c-a631-452c-9685-3083b692d71d tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] PATH_INFO: `/v3/projects/51030f254d594921b621dc3651ea9173` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:50.782367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee023c-a631-452c-9685-3083b692d71d tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorizing `identity:get_project(project_id=51030f254d594921b621dc3651ea9173)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:50.784642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee023c-a631-452c-9685-3083b692d71d tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:50.786359 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1427/2926] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:55:50 2022] GET /identity/v3/projects/51030f254d594921b621dc3651ea9173 => generated 369 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:55:51.223354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ca377-1501-46f7-a5dd-20e0fa5a547a None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:51.223666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ca377-1501-46f7-a5dd-20e0fa5a547a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:51.223947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ca377-1501-46f7-a5dd-20e0fa5a547a None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:51.228072 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1500/2927] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:55:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:55:51.232480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee023c-a631-452c-9685-3083b692d71d tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Authenticating service token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:55:51.253536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee023c-a631-452c-9685-3083b692d71d tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:51.274836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee023c-a631-452c-9685-3083b692d71d tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:51.286482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b268bc-78f3-469e-8503-0e62c82b467b tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccdeb939d99a40dd84b2930a8b05214f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7c2e9b5442ab48c9af69ab0d0e94b55e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:51.287156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b268bc-78f3-469e-8503-0e62c82b467b tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:51.287438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b268bc-78f3-469e-8503-0e62c82b467b tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:51.287714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b268bc-78f3-469e-8503-0e62c82b467b tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] PATH_INFO: `/v3/projects/51030f254d594921b621dc3651ea9173` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:51.288846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b268bc-78f3-469e-8503-0e62c82b467b tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorizing `identity:get_project(project_id=51030f254d594921b621dc3651ea9173)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:51.291528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b268bc-78f3-469e-8503-0e62c82b467b tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:51.293610 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1428/2928] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:55:51 2022] GET /identity/v3/projects/51030f254d594921b621dc3651ea9173 => generated 369 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:55:51.328044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ca377-1501-46f7-a5dd-20e0fa5a547a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:51.346350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ca377-1501-46f7-a5dd-20e0fa5a547a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:51.353599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d62c16-dee4-4679-b696-5cea872e73a6 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccdeb939d99a40dd84b2930a8b05214f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7c2e9b5442ab48c9af69ab0d0e94b55e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:51.354385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d62c16-dee4-4679-b696-5cea872e73a6 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:51.354805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d62c16-dee4-4679-b696-5cea872e73a6 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:51.355179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d62c16-dee4-4679-b696-5cea872e73a6 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] PATH_INFO: `/v3/users/dabd0acd01284edc8077eb8311a790d4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:51.389400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d62c16-dee4-4679-b696-5cea872e73a6 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorizing `identity:delete_user(user_id=dabd0acd01284edc8077eb8311a790d4)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:51.397084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d62c16-dee4-4679-b696-5cea872e73a6 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:51.465835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20d62c16-dee4-4679-b696-5cea872e73a6 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dabd0acd01284edc8077eb8311a790d4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:51.473095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20d62c16-dee4-4679-b696-5cea872e73a6 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dabd0acd01284edc8077eb8311a790d4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:51.486235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20d62c16-dee4-4679-b696-5cea872e73a6 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dabd0acd01284edc8077eb8311a790d4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:51.487427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20d62c16-dee4-4679-b696-5cea872e73a6 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dabd0acd01284edc8077eb8311a790d4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:51.496253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20d62c16-dee4-4679-b696-5cea872e73a6 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dabd0acd01284edc8077eb8311a790d4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:51.503255 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1501/2929] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:55:51 2022] DELETE /identity/v3/users/dabd0acd01284edc8077eb8311a790d4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:55:51.507252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b268bc-78f3-469e-8503-0e62c82b467b tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:51.509936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-13b268bc-78f3-469e-8503-0e62c82b467b tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:51.522286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b268bc-78f3-469e-8503-0e62c82b467b tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:51.579244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342babd-6aac-456c-8e8b-9161210e8347 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ccdeb939d99a40dd84b2930a8b05214f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7c2e9b5442ab48c9af69ab0d0e94b55e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:51.579789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342babd-6aac-456c-8e8b-9161210e8347 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:51.580068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342babd-6aac-456c-8e8b-9161210e8347 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:51.580338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342babd-6aac-456c-8e8b-9161210e8347 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] PATH_INFO: `/v3/projects/51030f254d594921b621dc3651ea9173` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:51.581429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342babd-6aac-456c-8e8b-9161210e8347 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorizing `identity:delete_project(project_id=51030f254d594921b621dc3651ea9173)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:51.583381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342babd-6aac-456c-8e8b-9161210e8347 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:51.630989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2342babd-6aac-456c-8e8b-9161210e8347 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51030f254d594921b621dc3651ea9173', 'request_id': 'req-2342babd-6aac-456c-8e8b-9161210e8347'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:51.635338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:51.640825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2342babd-6aac-456c-8e8b-9161210e8347 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51030f254d594921b621dc3651ea9173', 'request_id': 'req-2342babd-6aac-456c-8e8b-9161210e8347'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:51.647758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2342babd-6aac-456c-8e8b-9161210e8347 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51030f254d594921b621dc3651ea9173', 'request_id': 'req-2342babd-6aac-456c-8e8b-9161210e8347'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:51.659940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2342babd-6aac-456c-8e8b-9161210e8347 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] The token cache is being invalidate because project 51030f254d594921b621dc3651ea9173 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:55:51.660237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2342babd-6aac-456c-8e8b-9161210e8347 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:51.660882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2342babd-6aac-456c-8e8b-9161210e8347 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:51.662349 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1429/2930] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:55:51 2022] DELETE /identity/v3/projects/51030f254d594921b621dc3651ea9173 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:55:51.668382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d62c16-dee4-4679-b696-5cea872e73a6 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:51.669797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-20d62c16-dee4-4679-b696-5cea872e73a6 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:51.734781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:51.741701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d62c16-dee4-4679-b696-5cea872e73a6 tempest-QuotasAdminTestV257-1281127633 tempest-QuotasAdminTestV257-1281127633-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:51.808952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c7f19-65fc-4f79-a3a3-0e1612b01276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:51.809701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c7f19-65fc-4f79-a3a3-0e1612b01276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:51.809860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c7f19-65fc-4f79-a3a3-0e1612b01276 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:51.810139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c7f19-65fc-4f79-a3a3-0e1612b01276 admin admin] PATH_INFO: `/v3/users/877f6cb8e4864e09940ad582e55008f8` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:51.811309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c7f19-65fc-4f79-a3a3-0e1612b01276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=877f6cb8e4864e09940ad582e55008f8)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:51.813481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c7f19-65fc-4f79-a3a3-0e1612b01276 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:51.864677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-208c7f19-65fc-4f79-a3a3-0e1612b01276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '877f6cb8e4864e09940ad582e55008f8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:51.872187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-208c7f19-65fc-4f79-a3a3-0e1612b01276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '877f6cb8e4864e09940ad582e55008f8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:51.885124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-208c7f19-65fc-4f79-a3a3-0e1612b01276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '877f6cb8e4864e09940ad582e55008f8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:51.885862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-208c7f19-65fc-4f79-a3a3-0e1612b01276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '877f6cb8e4864e09940ad582e55008f8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:51.894889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-208c7f19-65fc-4f79-a3a3-0e1612b01276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '877f6cb8e4864e09940ad582e55008f8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:51.902522 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1502/2931] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:55:51 2022] DELETE /identity/v3/users/877f6cb8e4864e09940ad582e55008f8 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:55:51.906909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9fe0e-8b7a-4a73-83f7-107713314aa1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:51.907394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9fe0e-8b7a-4a73-83f7-107713314aa1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:51.907503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9fe0e-8b7a-4a73-83f7-107713314aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:51.926364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-08c9fe0e-8b7a-4a73-83f7-107713314aa1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:51.927587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-08c9fe0e-8b7a-4a73-83f7-107713314aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:51.985365 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1430/2932] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:55:51.995215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c7f19-65fc-4f79-a3a3-0e1612b01276 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:51.996606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-208c7f19-65fc-4f79-a3a3-0e1612b01276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:52.054645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c7f19-65fc-4f79-a3a3-0e1612b01276 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:52.058297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec86786-c4ec-4594-8946-0d0a52bf91d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:52.058943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec86786-c4ec-4594-8946-0d0a52bf91d0 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:52.059222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec86786-c4ec-4594-8946-0d0a52bf91d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:52.059493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec86786-c4ec-4594-8946-0d0a52bf91d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:52.068617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec86786-c4ec-4594-8946-0d0a52bf91d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:52.070041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec86786-c4ec-4594-8946-0d0a52bf91d0 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:52.079849 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1503/2933] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:55:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:55:52.518431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9fe0e-8b7a-4a73-83f7-107713314aa1 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:52.520336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-08c9fe0e-8b7a-4a73-83f7-107713314aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:52.547589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9fe0e-8b7a-4a73-83f7-107713314aa1 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:52.558975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0f080-d810-4875-9bb3-2a59307b26f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:52.559613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0f080-d810-4875-9bb3-2a59307b26f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:52.559894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0f080-d810-4875-9bb3-2a59307b26f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:52.560164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0f080-d810-4875-9bb3-2a59307b26f7 admin admin] PATH_INFO: `/v3/projects/55363ebfb9024c75bf8249b8f088320e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:52.562160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce0f080-d810-4875-9bb3-2a59307b26f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55363ebfb9024c75bf8249b8f088320e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:52.564285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce0f080-d810-4875-9bb3-2a59307b26f7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:52.626915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4ce0f080-d810-4875-9bb3-2a59307b26f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55363ebfb9024c75bf8249b8f088320e', 'request_id': 'req-4ce0f080-d810-4875-9bb3-2a59307b26f7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:52.635717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4ce0f080-d810-4875-9bb3-2a59307b26f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55363ebfb9024c75bf8249b8f088320e', 'request_id': 'req-4ce0f080-d810-4875-9bb3-2a59307b26f7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:52.643972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4ce0f080-d810-4875-9bb3-2a59307b26f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55363ebfb9024c75bf8249b8f088320e', 'request_id': 'req-4ce0f080-d810-4875-9bb3-2a59307b26f7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:52.667359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4ce0f080-d810-4875-9bb3-2a59307b26f7 admin admin] The token cache is being invalidate because project 55363ebfb9024c75bf8249b8f088320e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:55:52.667654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4ce0f080-d810-4875-9bb3-2a59307b26f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:52.669398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4ce0f080-d810-4875-9bb3-2a59307b26f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:52.671252 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1431/2934] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:55:52 2022] DELETE /identity/v3/projects/55363ebfb9024c75bf8249b8f088320e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:55:52.675050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec86786-c4ec-4594-8946-0d0a52bf91d0 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:52.686089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec86786-c4ec-4594-8946-0d0a52bf91d0 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:52.710480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:52.773208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cbb434-b172-4ac9-af38-948f0e0ff387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:52.774172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cbb434-b172-4ac9-af38-948f0e0ff387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:52.774899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cbb434-b172-4ac9-af38-948f0e0ff387 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:52.775201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cbb434-b172-4ac9-af38-948f0e0ff387 admin admin] PATH_INFO: `/v3/users/ccdeb939d99a40dd84b2930a8b05214f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:52.776723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cbb434-b172-4ac9-af38-948f0e0ff387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccdeb939d99a40dd84b2930a8b05214f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:52.779640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cbb434-b172-4ac9-af38-948f0e0ff387 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:52.839371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d2cbb434-b172-4ac9-af38-948f0e0ff387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccdeb939d99a40dd84b2930a8b05214f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:52.850869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d2cbb434-b172-4ac9-af38-948f0e0ff387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccdeb939d99a40dd84b2930a8b05214f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:52.867197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d2cbb434-b172-4ac9-af38-948f0e0ff387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccdeb939d99a40dd84b2930a8b05214f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:52.868860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d2cbb434-b172-4ac9-af38-948f0e0ff387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccdeb939d99a40dd84b2930a8b05214f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:52.878747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d2cbb434-b172-4ac9-af38-948f0e0ff387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccdeb939d99a40dd84b2930a8b05214f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:52.886649 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1504/2935] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:55:52 2022] DELETE /identity/v3/users/ccdeb939d99a40dd84b2930a8b05214f => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:55:53.266743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0f080-d810-4875-9bb3-2a59307b26f7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:53.268112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4ce0f080-d810-4875-9bb3-2a59307b26f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:53.287990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0f080-d810-4875-9bb3-2a59307b26f7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:53.389313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0965db3-714b-4bfa-b5b5-c47e0208316d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:53.389959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0965db3-714b-4bfa-b5b5-c47e0208316d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:53.390275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0965db3-714b-4bfa-b5b5-c47e0208316d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:53.390606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0965db3-714b-4bfa-b5b5-c47e0208316d admin admin] PATH_INFO: `/v3/projects/7c2e9b5442ab48c9af69ab0d0e94b55e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:53.391840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0965db3-714b-4bfa-b5b5-c47e0208316d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c2e9b5442ab48c9af69ab0d0e94b55e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:53.393811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0965db3-714b-4bfa-b5b5-c47e0208316d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:53.404710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:53.473704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b0965db3-714b-4bfa-b5b5-c47e0208316d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c2e9b5442ab48c9af69ab0d0e94b55e', 'request_id': 'req-b0965db3-714b-4bfa-b5b5-c47e0208316d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:53.495684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b0965db3-714b-4bfa-b5b5-c47e0208316d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c2e9b5442ab48c9af69ab0d0e94b55e', 'request_id': 'req-b0965db3-714b-4bfa-b5b5-c47e0208316d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:53.512217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b0965db3-714b-4bfa-b5b5-c47e0208316d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c2e9b5442ab48c9af69ab0d0e94b55e', 'request_id': 'req-b0965db3-714b-4bfa-b5b5-c47e0208316d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:53.529131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b0965db3-714b-4bfa-b5b5-c47e0208316d admin admin] The token cache is being invalidate because project 7c2e9b5442ab48c9af69ab0d0e94b55e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:55:53.530019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b0965db3-714b-4bfa-b5b5-c47e0208316d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:53.531358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b0965db3-714b-4bfa-b5b5-c47e0208316d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:53.533915 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1432/2936] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:55:53 2022] DELETE /identity/v3/projects/7c2e9b5442ab48c9af69ab0d0e94b55e => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:55:53.540369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd7a98-02f1-497a-9958-e6113687d518 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:53.540369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd7a98-02f1-497a-9958-e6113687d518 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:53.540621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd7a98-02f1-497a-9958-e6113687d518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:53.585580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-03fd7a98-02f1-497a-9958-e6113687d518 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:53.588085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-03fd7a98-02f1-497a-9958-e6113687d518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:53.724995 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1505/2937] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:53.733927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2c391-67d4-4b0a-b2c7-6477beddd0be None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:53.734190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2c391-67d4-4b0a-b2c7-6477beddd0be None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:53.734467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2c391-67d4-4b0a-b2c7-6477beddd0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:53.760796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3ce2c391-67d4-4b0a-b2c7-6477beddd0be None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:53.762451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3ce2c391-67d4-4b0a-b2c7-6477beddd0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:53.772407 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1433/2938] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:53.777944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd7a98-02f1-497a-9958-e6113687d518 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:53.791416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd7a98-02f1-497a-9958-e6113687d518 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:53.798473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bcd04-7484-41c8-969b-3c1b937f0a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:53.799709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bcd04-7484-41c8-969b-3c1b937f0a2c admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:53.800169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bcd04-7484-41c8-969b-3c1b937f0a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:53.800615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bcd04-7484-41c8-969b-3c1b937f0a2c admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:53.801469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6bcd04-7484-41c8-969b-3c1b937f0a2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:55:53.802074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6bcd04-7484-41c8-969b-3c1b937f0a2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:53.804172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6bcd04-7484-41c8-969b-3c1b937f0a2c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:53.823648 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1506/2939] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:55:53.827904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2c391-67d4-4b0a-b2c7-6477beddd0be None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:53.841069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2c391-67d4-4b0a-b2c7-6477beddd0be None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:53.846794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c25a3-7603-490c-abf2-1ab41fbf8cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:53.847941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c25a3-7603-490c-abf2-1ab41fbf8cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:53.848285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c25a3-7603-490c-abf2-1ab41fbf8cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:53.848555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c25a3-7603-490c-abf2-1ab41fbf8cb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:53.849375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c25a3-7603-490c-abf2-1ab41fbf8cb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:53.852076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c25a3-7603-490c-abf2-1ab41fbf8cb5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:53.907566 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1434/2940] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:55:53 2022] POST /identity/v3/projects => generated 379 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:55:53.911558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bcd04-7484-41c8-969b-3c1b937f0a2c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:53.931496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bcd04-7484-41c8-969b-3c1b937f0a2c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:53.996078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:54.082281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8290c4a-da09-4852-81e8-39b864a26bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:54.083211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8290c4a-da09-4852-81e8-39b864a26bcd admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:54.084456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8290c4a-da09-4852-81e8-39b864a26bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:54.085531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8290c4a-da09-4852-81e8-39b864a26bcd admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:54.086689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8290c4a-da09-4852-81e8-39b864a26bcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:54.091493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8290c4a-da09-4852-81e8-39b864a26bcd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:54.124326 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1507/2941] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:55:53 2022] POST /identity/v3/users => generated 290 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:55:54.128766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c25a3-7603-490c-abf2-1ab41fbf8cb5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:54.151384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c25a3-7603-490c-abf2-1ab41fbf8cb5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:54.158144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067ffb2-7e81-4ea5-a1d7-9b0c82dd6407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:54.159022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067ffb2-7e81-4ea5-a1d7-9b0c82dd6407 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:54.159413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067ffb2-7e81-4ea5-a1d7-9b0c82dd6407 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:54.160156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067ffb2-7e81-4ea5-a1d7-9b0c82dd6407 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:54.162096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067ffb2-7e81-4ea5-a1d7-9b0c82dd6407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:54.163978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067ffb2-7e81-4ea5-a1d7-9b0c82dd6407 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:54.177621 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1435/2942] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:55:54.182682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8290c4a-da09-4852-81e8-39b864a26bcd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:54.204412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8290c4a-da09-4852-81e8-39b864a26bcd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:54.212149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada7f0e-cad6-4635-998f-f133d6f664fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:54.212884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada7f0e-cad6-4635-998f-f133d6f664fb admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:54.213241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada7f0e-cad6-4635-998f-f133d6f664fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:54.213645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada7f0e-cad6-4635-998f-f133d6f664fb admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:54.214376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada7f0e-cad6-4635-998f-f133d6f664fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:54.216644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada7f0e-cad6-4635-998f-f133d6f664fb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:54.235913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067ffb2-7e81-4ea5-a1d7-9b0c82dd6407 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:54.248106 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1508/2943] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:55:54.251574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067ffb2-7e81-4ea5-a1d7-9b0c82dd6407 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:54.259625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87700dd3-fba4-4866-91bd-e9ebcfc0b270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:54.259968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87700dd3-fba4-4866-91bd-e9ebcfc0b270 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:54.260463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87700dd3-fba4-4866-91bd-e9ebcfc0b270 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:54.261147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87700dd3-fba4-4866-91bd-e9ebcfc0b270 admin admin] PATH_INFO: `/v3/projects/29a8000bfb634471b905328904f6bb07/users/1b9aed06520543b5bad8fb49d9e0cfb6/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:54.321850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87700dd3-fba4-4866-91bd-e9ebcfc0b270 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29a8000bfb634471b905328904f6bb07, user_id=1b9aed06520543b5bad8fb49d9e0cfb6, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:54.325036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87700dd3-fba4-4866-91bd-e9ebcfc0b270 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:54.356553 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1436/2944] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:55:54 2022] PUT /identity/v3/projects/29a8000bfb634471b905328904f6bb07/users/1b9aed06520543b5bad8fb49d9e0cfb6/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:55:54.373050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64226232-527b-48cb-b72a-7261c7b541f6 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:54.373050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64226232-527b-48cb-b72a-7261c7b541f6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:54.373050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64226232-527b-48cb-b72a-7261c7b541f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:54.416597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-64226232-527b-48cb-b72a-7261c7b541f6 None None] MFA Rules not processed for user `1b9aed06520543b5bad8fb49d9e0cfb6`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:54.419070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-64226232-527b-48cb-b72a-7261c7b541f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:54.553830 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1509/2945] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:54.558687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87700dd3-fba4-4866-91bd-e9ebcfc0b270 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:54.730414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87700dd3-fba4-4866-91bd-e9ebcfc0b270 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:54.735576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b1463-a81f-4879-b119-5f443dccbf8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:54.736157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b1463-a81f-4879-b119-5f443dccbf8d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:54.736539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b1463-a81f-4879-b119-5f443dccbf8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:54.737121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b1463-a81f-4879-b119-5f443dccbf8d admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:54.737866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b1463-a81f-4879-b119-5f443dccbf8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:54.740010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b1463-a81f-4879-b119-5f443dccbf8d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:54.795643 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1437/2946] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:55:54 2022] POST /identity/v3/projects => generated 381 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:55:54.799646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64226232-527b-48cb-b72a-7261c7b541f6 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:54.815168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64226232-527b-48cb-b72a-7261c7b541f6 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:54.903521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd73348-82d9-4370-9c7d-74f16ad6531d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:54.903521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd73348-82d9-4370-9c7d-74f16ad6531d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:54.903521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd73348-82d9-4370-9c7d-74f16ad6531d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:54.903521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd73348-82d9-4370-9c7d-74f16ad6531d admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:54.903876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd73348-82d9-4370-9c7d-74f16ad6531d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:54.911142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd73348-82d9-4370-9c7d-74f16ad6531d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:54.956260 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1510/2947] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:55:54 2022] POST /identity/v3/users => generated 297 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:55:54.957423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b1463-a81f-4879-b119-5f443dccbf8d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:54.968750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b1463-a81f-4879-b119-5f443dccbf8d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:54.973453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de32347-c62c-439e-b610-e55c0d0b8748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:54.974328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de32347-c62c-439e-b610-e55c0d0b8748 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:54.975066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de32347-c62c-439e-b610-e55c0d0b8748 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:54.975066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de32347-c62c-439e-b610-e55c0d0b8748 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:54.975889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de32347-c62c-439e-b610-e55c0d0b8748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:54.978852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de32347-c62c-439e-b610-e55c0d0b8748 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:54.992078 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1438/2948] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:55:54.997170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd73348-82d9-4370-9c7d-74f16ad6531d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:55.009232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd73348-82d9-4370-9c7d-74f16ad6531d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:55.014330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f42c1-be33-4e95-baf1-05328369e2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:55.015185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f42c1-be33-4e95-baf1-05328369e2dc admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:55.015566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f42c1-be33-4e95-baf1-05328369e2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:55.016337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f42c1-be33-4e95-baf1-05328369e2dc admin admin] PATH_INFO: `/v3/projects/acf7d3c47743482c97c2d776a05f2ff1/users/09da6bb64857421eaf68a40d23a3b3e7/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:55.037060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f42c1-be33-4e95-baf1-05328369e2dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=acf7d3c47743482c97c2d776a05f2ff1, user_id=09da6bb64857421eaf68a40d23a3b3e7, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:55.044978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f42c1-be33-4e95-baf1-05328369e2dc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:55.059442 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1511/2949] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:55:54 2022] PUT /identity/v3/projects/acf7d3c47743482c97c2d776a05f2ff1/users/09da6bb64857421eaf68a40d23a3b3e7/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:55:55.063885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de32347-c62c-439e-b610-e55c0d0b8748 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:55.074469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de32347-c62c-439e-b610-e55c0d0b8748 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:55.079040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:55.145552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc1c38-fdd4-4dac-91c3-8c66a1d31e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:55.146456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc1c38-fdd4-4dac-91c3-8c66a1d31e3a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:55.146862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc1c38-fdd4-4dac-91c3-8c66a1d31e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:55.147232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc1c38-fdd4-4dac-91c3-8c66a1d31e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:55.147974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc1c38-fdd4-4dac-91c3-8c66a1d31e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:55.151690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc1c38-fdd4-4dac-91c3-8c66a1d31e3a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:55.163830 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1439/2950] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:55:55.168037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f42c1-be33-4e95-baf1-05328369e2dc admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:55.175937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:55.180097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f42c1-be33-4e95-baf1-05328369e2dc admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:55.187033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b39aff-cd43-42e6-a754-4e4ba2e0b7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:55.187875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b39aff-cd43-42e6-a754-4e4ba2e0b7d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:55.188334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b39aff-cd43-42e6-a754-4e4ba2e0b7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:55.188609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b39aff-cd43-42e6-a754-4e4ba2e0b7d1 admin admin] PATH_INFO: `/v3/projects/acf7d3c47743482c97c2d776a05f2ff1/users/09da6bb64857421eaf68a40d23a3b3e7/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:55.190611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b39aff-cd43-42e6-a754-4e4ba2e0b7d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acf7d3c47743482c97c2d776a05f2ff1, user_id=09da6bb64857421eaf68a40d23a3b3e7, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:55.193304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b39aff-cd43-42e6-a754-4e4ba2e0b7d1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:55.204471 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1512/2951] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:55:55 2022] PUT /identity/v3/projects/acf7d3c47743482c97c2d776a05f2ff1/users/09da6bb64857421eaf68a40d23a3b3e7/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:55:55.215434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f866f0-40fe-4128-bd84-ed68f655facb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:55.215723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f866f0-40fe-4128-bd84-ed68f655facb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:55.215998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f866f0-40fe-4128-bd84-ed68f655facb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:55.244507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b1f866f0-40fe-4128-bd84-ed68f655facb None None] MFA Rules not processed for user `09da6bb64857421eaf68a40d23a3b3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:55.245749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b1f866f0-40fe-4128-bd84-ed68f655facb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:55.315851 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1440/2952] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:55.327883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b39aff-cd43-42e6-a754-4e4ba2e0b7d1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:55.329577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-72b39aff-cd43-42e6-a754-4e4ba2e0b7d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:55.412456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b39aff-cd43-42e6-a754-4e4ba2e0b7d1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:55.418166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab41497-3359-476d-84c0-b893b1d4b69d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:55.418806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab41497-3359-476d-84c0-b893b1d4b69d service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:55.419085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab41497-3359-476d-84c0-b893b1d4b69d service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:55.419359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab41497-3359-476d-84c0-b893b1d4b69d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:55.428179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab41497-3359-476d-84c0-b893b1d4b69d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:55.429753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab41497-3359-476d-84c0-b893b1d4b69d service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:55.440169 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1513/2953] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:55:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:55:55.455827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134683f-d712-4538-8332-97ceb2e1aec3 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:55.456338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134683f-d712-4538-8332-97ceb2e1aec3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:55.456451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134683f-d712-4538-8332-97ceb2e1aec3 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:55.457709 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1441/2954] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:55:55.462588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab41497-3359-476d-84c0-b893b1d4b69d service nova] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:55:55.473909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab41497-3359-476d-84c0-b893b1d4b69d service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:55.484756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab41497-3359-476d-84c0-b893b1d4b69d service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:55.489674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdf002-cc93-4604-a4b2-378dc5c73c33 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09da6bb64857421eaf68a40d23a3b3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'acf7d3c47743482c97c2d776a05f2ff1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:55.490296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdf002-cc93-4604-a4b2-378dc5c73c33 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:55.490618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdf002-cc93-4604-a4b2-378dc5c73c33 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:55.490900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdf002-cc93-4604-a4b2-378dc5c73c33 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] PATH_INFO: `/v3/projects/29a8000bfb634471b905328904f6bb07` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:55.492064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdf002-cc93-4604-a4b2-378dc5c73c33 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: Authorizing `identity:get_project(project_id=29a8000bfb634471b905328904f6bb07)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:55.494659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdf002-cc93-4604-a4b2-378dc5c73c33 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:55.496411 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1514/2955] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:55:55 2022] GET /identity/v3/projects/29a8000bfb634471b905328904f6bb07 => generated 379 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:55:55.563078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd7ef8-6024-4bc6-8437-76fcb2c53fa1 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:55.563478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd7ef8-6024-4bc6-8437-76fcb2c53fa1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:55.563777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd7ef8-6024-4bc6-8437-76fcb2c53fa1 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:55.565527 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1442/2956] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:55:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:55:55.570895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdf002-cc93-4604-a4b2-378dc5c73c33 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:55:55.584203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdf002-cc93-4604-a4b2-378dc5c73c33 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:55.593938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdf002-cc93-4604-a4b2-378dc5c73c33 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:55.600558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28404b16-08dd-496d-a72b-df6ecfeb6293 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09da6bb64857421eaf68a40d23a3b3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'acf7d3c47743482c97c2d776a05f2ff1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:55.601195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28404b16-08dd-496d-a72b-df6ecfeb6293 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:55.601479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28404b16-08dd-496d-a72b-df6ecfeb6293 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:55.601753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28404b16-08dd-496d-a72b-df6ecfeb6293 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] PATH_INFO: `/v3/projects/29a8000bfb634471b905328904f6bb07` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:55.603598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28404b16-08dd-496d-a72b-df6ecfeb6293 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: Authorizing `identity:get_project(project_id=29a8000bfb634471b905328904f6bb07)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:55.605637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28404b16-08dd-496d-a72b-df6ecfeb6293 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:55.607492 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1515/2957] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:55:55 2022] GET /identity/v3/projects/29a8000bfb634471b905328904f6bb07 => generated 379 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:55:55.692057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499ffbb-25ed-4332-be1a-803cad3c2787 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:55.692459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499ffbb-25ed-4332-be1a-803cad3c2787 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:55.692854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499ffbb-25ed-4332-be1a-803cad3c2787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:55.717062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3499ffbb-25ed-4332-be1a-803cad3c2787 None None] MFA Rules not processed for user `1b9aed06520543b5bad8fb49d9e0cfb6`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:55.717974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3499ffbb-25ed-4332-be1a-803cad3c2787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:55.851157 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1443/2958] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:55.865978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28404b16-08dd-496d-a72b-df6ecfeb6293 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:55.883094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28404b16-08dd-496d-a72b-df6ecfeb6293 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:55.890517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d54c9-0f4b-486e-83df-bad8b12a40c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:55.891276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d54c9-0f4b-486e-83df-bad8b12a40c1 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:55.891625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d54c9-0f4b-486e-83df-bad8b12a40c1 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:55.891967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d54c9-0f4b-486e-83df-bad8b12a40c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:55.902545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d54c9-0f4b-486e-83df-bad8b12a40c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:55.904570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d54c9-0f4b-486e-83df-bad8b12a40c1 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:55.922702 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1516/2959] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:55:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:55:56.867252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ece2e-7c2e-455c-9c38-000688ba7c30 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:56.868347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ece2e-7c2e-455c-9c38-000688ba7c30 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:56.868347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ece2e-7c2e-455c-9c38-000688ba7c30 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:56.869887 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1444/2960] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:55:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:55:56.874730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d54c9-0f4b-486e-83df-bad8b12a40c1 service nova] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:55:56.887271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b61d54c9-0f4b-486e-83df-bad8b12a40c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:56.903212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d54c9-0f4b-486e-83df-bad8b12a40c1 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:56.914452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d54c9-0f4b-486e-83df-bad8b12a40c1 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:56.922115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c533a-dcde-4b4b-9b74-173896fc71de tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09da6bb64857421eaf68a40d23a3b3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'acf7d3c47743482c97c2d776a05f2ff1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:56.922984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c533a-dcde-4b4b-9b74-173896fc71de tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:56.923539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c533a-dcde-4b4b-9b74-173896fc71de tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:56.923827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c533a-dcde-4b4b-9b74-173896fc71de tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] PATH_INFO: `/v3/projects/29a8000bfb634471b905328904f6bb07` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:56.925078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c533a-dcde-4b4b-9b74-173896fc71de tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: Authorizing `identity:get_project(project_id=29a8000bfb634471b905328904f6bb07)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:56.927262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c533a-dcde-4b4b-9b74-173896fc71de tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:56.929584 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1517/2961] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:55:56 2022] GET /identity/v3/projects/29a8000bfb634471b905328904f6bb07 => generated 379 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:55:57.060873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55adabe-8819-4aec-97a6-9332db905d94 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:57.061160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55adabe-8819-4aec-97a6-9332db905d94 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:57.061438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55adabe-8819-4aec-97a6-9332db905d94 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:57.063343 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1445/2962] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:55:57.068182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c533a-dcde-4b4b-9b74-173896fc71de tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:55:57.081601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c533a-dcde-4b4b-9b74-173896fc71de tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:57.091789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c533a-dcde-4b4b-9b74-173896fc71de tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:57.096925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87166123-1a0a-417a-ac44-d6727f6023c0 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09da6bb64857421eaf68a40d23a3b3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'acf7d3c47743482c97c2d776a05f2ff1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:57.097699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87166123-1a0a-417a-ac44-d6727f6023c0 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:57.098170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87166123-1a0a-417a-ac44-d6727f6023c0 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:57.098438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87166123-1a0a-417a-ac44-d6727f6023c0 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] PATH_INFO: `/v3/projects/29a8000bfb634471b905328904f6bb07` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:57.099633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87166123-1a0a-417a-ac44-d6727f6023c0 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: Authorizing `identity:get_project(project_id=29a8000bfb634471b905328904f6bb07)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:57.101658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87166123-1a0a-417a-ac44-d6727f6023c0 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:57.104097 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1518/2963] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:55:57 2022] GET /identity/v3/projects/29a8000bfb634471b905328904f6bb07 => generated 379 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:55:57.282642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8e9ed-4ffb-4382-84c4-acaa1e1774e2 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:57.283009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8e9ed-4ffb-4382-84c4-acaa1e1774e2 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:57.283227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8e9ed-4ffb-4382-84c4-acaa1e1774e2 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:57.284546 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1446/2964] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:55:57.290103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87166123-1a0a-417a-ac44-d6727f6023c0 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:55:57.304647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87166123-1a0a-417a-ac44-d6727f6023c0 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:57.318091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87166123-1a0a-417a-ac44-d6727f6023c0 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:57.323883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3d6b7-7809-411a-85f0-00106468411f tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09da6bb64857421eaf68a40d23a3b3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'acf7d3c47743482c97c2d776a05f2ff1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:57.324460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3d6b7-7809-411a-85f0-00106468411f tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:57.324792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3d6b7-7809-411a-85f0-00106468411f tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:57.325029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3d6b7-7809-411a-85f0-00106468411f tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] PATH_INFO: `/v3/projects/29a8000bfb634471b905328904f6bb07` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:57.326324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3d6b7-7809-411a-85f0-00106468411f tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: Authorizing `identity:get_project(project_id=29a8000bfb634471b905328904f6bb07)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:57.328419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3d6b7-7809-411a-85f0-00106468411f tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:57.330297 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1519/2965] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:55:57 2022] GET /identity/v3/projects/29a8000bfb634471b905328904f6bb07 => generated 379 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:55:57.419048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f219c-655d-432a-a37a-39122c0af1a5 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:57.419606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f219c-655d-432a-a37a-39122c0af1a5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:57.419977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f219c-655d-432a-a37a-39122c0af1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:57.440794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d84f219c-655d-432a-a37a-39122c0af1a5 None None] MFA Rules not processed for user `1b9aed06520543b5bad8fb49d9e0cfb6`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:57.442204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d84f219c-655d-432a-a37a-39122c0af1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:57.449973 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1447/2966] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:55:58.010532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab42522-6665-411b-b0e9-d1261349196f None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:58.011508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab42522-6665-411b-b0e9-d1261349196f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:58.011867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab42522-6665-411b-b0e9-d1261349196f None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:58.013329 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1520/2967] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:55:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:55:58.018046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f219c-655d-432a-a37a-39122c0af1a5 None None] Authenticating service token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:55:58.036805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f219c-655d-432a-a37a-39122c0af1a5 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:58.047521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f219c-655d-432a-a37a-39122c0af1a5 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:58.052135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c330541-0df7-4655-a7a2-4edb126e275d tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09da6bb64857421eaf68a40d23a3b3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'acf7d3c47743482c97c2d776a05f2ff1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:58.052793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c330541-0df7-4655-a7a2-4edb126e275d tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:58.053076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c330541-0df7-4655-a7a2-4edb126e275d tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:58.053350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c330541-0df7-4655-a7a2-4edb126e275d tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] PATH_INFO: `/v3/projects/29a8000bfb634471b905328904f6bb07` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:58.054516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c330541-0df7-4655-a7a2-4edb126e275d tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: Authorizing `identity:get_project(project_id=29a8000bfb634471b905328904f6bb07)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:58.056621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c330541-0df7-4655-a7a2-4edb126e275d tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:58.058240 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1448/2968] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:55:58 2022] GET /identity/v3/projects/29a8000bfb634471b905328904f6bb07 => generated 379 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:55:58.155811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d11d49-d8a1-41bf-9111-12a126f307d6 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:58.156284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d11d49-d8a1-41bf-9111-12a126f307d6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:58.156630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d11d49-d8a1-41bf-9111-12a126f307d6 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:58.159178 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1521/2969] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:55:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:55:58.166064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c330541-0df7-4655-a7a2-4edb126e275d tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] Authenticating service token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:55:58.180274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c330541-0df7-4655-a7a2-4edb126e275d tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:58.192536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c330541-0df7-4655-a7a2-4edb126e275d tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:58.197823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d9da6-7e30-465e-b869-2be91bc4cbc9 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09da6bb64857421eaf68a40d23a3b3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'acf7d3c47743482c97c2d776a05f2ff1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:58.198449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d9da6-7e30-465e-b869-2be91bc4cbc9 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:58.198766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d9da6-7e30-465e-b869-2be91bc4cbc9 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:58.199042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d9da6-7e30-465e-b869-2be91bc4cbc9 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] PATH_INFO: `/v3/projects/29a8000bfb634471b905328904f6bb07` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:58.200204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d9da6-7e30-465e-b869-2be91bc4cbc9 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: Authorizing `identity:get_project(project_id=29a8000bfb634471b905328904f6bb07)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:58.202251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d9da6-7e30-465e-b869-2be91bc4cbc9 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:58.203924 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1449/2970] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:55:58 2022] GET /identity/v3/projects/29a8000bfb634471b905328904f6bb07 => generated 379 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:55:58.244570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25b69d-03ff-4cf0-af2d-042c8aa59f11 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:58.244902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25b69d-03ff-4cf0-af2d-042c8aa59f11 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:58.245180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25b69d-03ff-4cf0-af2d-042c8aa59f11 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:58.247181 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1522/2971] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:55:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:55:58.252322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d9da6-7e30-465e-b869-2be91bc4cbc9 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] Authenticating service token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:55:58.266533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d9da6-7e30-465e-b869-2be91bc4cbc9 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:58.272100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:58.276874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d9da6-7e30-465e-b869-2be91bc4cbc9 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:58.282450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db92fc-cc3c-41de-a2c1-1313bb7d72ee tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09da6bb64857421eaf68a40d23a3b3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'acf7d3c47743482c97c2d776a05f2ff1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:58.283196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db92fc-cc3c-41de-a2c1-1313bb7d72ee tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:58.283555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db92fc-cc3c-41de-a2c1-1313bb7d72ee tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:58.283897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db92fc-cc3c-41de-a2c1-1313bb7d72ee tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] PATH_INFO: `/v3/projects/29a8000bfb634471b905328904f6bb07` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:58.285463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db92fc-cc3c-41de-a2c1-1313bb7d72ee tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: Authorizing `identity:get_project(project_id=29a8000bfb634471b905328904f6bb07)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:58.287718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db92fc-cc3c-41de-a2c1-1313bb7d72ee tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:58.289389 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1450/2972] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:55:58 2022] GET /identity/v3/projects/29a8000bfb634471b905328904f6bb07 => generated 379 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:55:58.395263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3446d1-f49b-45f9-ba7b-f810e58851ad None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:58.395596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3446d1-f49b-45f9-ba7b-f810e58851ad None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:58.395975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3446d1-f49b-45f9-ba7b-f810e58851ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:58.415177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-df3446d1-f49b-45f9-ba7b-f810e58851ad None None] MFA Rules not processed for user `1b9aed06520543b5bad8fb49d9e0cfb6`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:58.416382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-df3446d1-f49b-45f9-ba7b-f810e58851ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:58.423846 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1523/2973] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:58.813429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8fdbd-bb7a-4cac-852a-5531851ec128 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:58.813713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8fdbd-bb7a-4cac-852a-5531851ec128 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:58.813989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8fdbd-bb7a-4cac-852a-5531851ec128 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:58.815333 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1451/2974] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:55:58.819931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3446d1-f49b-45f9-ba7b-f810e58851ad None None] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:55:58.832634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3446d1-f49b-45f9-ba7b-f810e58851ad None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:58.841838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3446d1-f49b-45f9-ba7b-f810e58851ad None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:58.846722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614adc15-1bf0-4e12-ab3d-28e467b4e490 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09da6bb64857421eaf68a40d23a3b3e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'acf7d3c47743482c97c2d776a05f2ff1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:58.847696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614adc15-1bf0-4e12-ab3d-28e467b4e490 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:58.847910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614adc15-1bf0-4e12-ab3d-28e467b4e490 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:58.848460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614adc15-1bf0-4e12-ab3d-28e467b4e490 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] PATH_INFO: `/v3/projects/29a8000bfb634471b905328904f6bb07` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:58.849414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614adc15-1bf0-4e12-ab3d-28e467b4e490 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: Authorizing `identity:get_project(project_id=29a8000bfb634471b905328904f6bb07)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:58.851598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614adc15-1bf0-4e12-ab3d-28e467b4e490 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:58.853721 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1524/2975] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:55:58 2022] GET /identity/v3/projects/29a8000bfb634471b905328904f6bb07 => generated 379 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:55:58.960019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8fdbd-bb7a-4cac-852a-5531851ec128 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:59.027758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8fdbd-bb7a-4cac-852a-5531851ec128 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:59.033549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c433da2-4632-4aca-b087-46a0a6e46197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:59.034187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c433da2-4632-4aca-b087-46a0a6e46197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:59.034464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c433da2-4632-4aca-b087-46a0a6e46197 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:59.034874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c433da2-4632-4aca-b087-46a0a6e46197 admin admin] PATH_INFO: `/v3/users/1b9aed06520543b5bad8fb49d9e0cfb6` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:59.036439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c433da2-4632-4aca-b087-46a0a6e46197 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b9aed06520543b5bad8fb49d9e0cfb6)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:59.038335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c433da2-4632-4aca-b087-46a0a6e46197 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:59.069416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614adc15-1bf0-4e12-ab3d-28e467b4e490 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:59.071286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-614adc15-1bf0-4e12-ab3d-28e467b4e490 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:59.087751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614adc15-1bf0-4e12-ab3d-28e467b4e490 tempest-QuotasAdminNegativeTest-148632111 tempest-QuotasAdminNegativeTest-148632111-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:59.096047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb42d5a-5bfd-4b34-a3e1-e9670079dd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:59.096875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb42d5a-5bfd-4b34-a3e1-e9670079dd0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:59.097343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb42d5a-5bfd-4b34-a3e1-e9670079dd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:59.098399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb42d5a-5bfd-4b34-a3e1-e9670079dd0d admin admin] PATH_INFO: `/v3/users/e5bfbc1e512a445ca7527f3d79e40d16` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:59.099763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb42d5a-5bfd-4b34-a3e1-e9670079dd0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5bfbc1e512a445ca7527f3d79e40d16)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:59.102609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb42d5a-5bfd-4b34-a3e1-e9670079dd0d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:59.130829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0c433da2-4632-4aca-b087-46a0a6e46197 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b9aed06520543b5bad8fb49d9e0cfb6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:59.132668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0c433da2-4632-4aca-b087-46a0a6e46197 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b9aed06520543b5bad8fb49d9e0cfb6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:59.133741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0c433da2-4632-4aca-b087-46a0a6e46197 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b9aed06520543b5bad8fb49d9e0cfb6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:59.145895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0c433da2-4632-4aca-b087-46a0a6e46197 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b9aed06520543b5bad8fb49d9e0cfb6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:59.160936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0c433da2-4632-4aca-b087-46a0a6e46197 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b9aed06520543b5bad8fb49d9e0cfb6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:59.169589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7cb42d5a-5bfd-4b34-a3e1-e9670079dd0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5bfbc1e512a445ca7527f3d79e40d16'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:59.178451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7cb42d5a-5bfd-4b34-a3e1-e9670079dd0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5bfbc1e512a445ca7527f3d79e40d16'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:59.178962 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1452/2976] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:55:58 2022] DELETE /identity/v3/users/1b9aed06520543b5bad8fb49d9e0cfb6 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:55:59.180603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf55c4-651b-443c-bc92-b70f7b0885b9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:59.180976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf55c4-651b-443c-bc92-b70f7b0885b9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:59.181171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:59.181482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf55c4-651b-443c-bc92-b70f7b0885b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:59.195490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7cb42d5a-5bfd-4b34-a3e1-e9670079dd0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5bfbc1e512a445ca7527f3d79e40d16'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:59.196601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7cb42d5a-5bfd-4b34-a3e1-e9670079dd0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5bfbc1e512a445ca7527f3d79e40d16'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:59.204824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7cb42d5a-5bfd-4b34-a3e1-e9670079dd0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5bfbc1e512a445ca7527f3d79e40d16'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:55:59.206386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-87bf55c4-651b-443c-bc92-b70f7b0885b9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:59.208651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-87bf55c4-651b-443c-bc92-b70f7b0885b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:59.213408 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1525/2977] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:55:59 2022] DELETE /identity/v3/users/e5bfbc1e512a445ca7527f3d79e40d16 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:55:59.218127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd8f10-36dc-4d6a-8dc0-f5aa50fd8d13 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:59.218541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd8f10-36dc-4d6a-8dc0-f5aa50fd8d13 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:59.218887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd8f10-36dc-4d6a-8dc0-f5aa50fd8d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:59.241901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3dcd8f10-36dc-4d6a-8dc0-f5aa50fd8d13 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:55:59.243275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3dcd8f10-36dc-4d6a-8dc0-f5aa50fd8d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:59.283688 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1453/2978] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:55:59.291272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf55c4-651b-443c-bc92-b70f7b0885b9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:59.292820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-87bf55c4-651b-443c-bc92-b70f7b0885b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:59.316617 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1526/2979] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:55:59.318960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:59.327217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd8f10-36dc-4d6a-8dc0-f5aa50fd8d13 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:59.330301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3dcd8f10-36dc-4d6a-8dc0-f5aa50fd8d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:59.363220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf55c4-651b-443c-bc92-b70f7b0885b9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:59.420840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd8f10-36dc-4d6a-8dc0-f5aa50fd8d13 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:59.430809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a36502-73c7-421e-9466-a2ad40bee0e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:59.431447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a36502-73c7-421e-9466-a2ad40bee0e7 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:59.431794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a36502-73c7-421e-9466-a2ad40bee0e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:59.432133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a36502-73c7-421e-9466-a2ad40bee0e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:59.437777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966860d-d8de-488f-ae03-17914410b2a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:59.438394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966860d-d8de-488f-ae03-17914410b2a1 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:59.438846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966860d-d8de-488f-ae03-17914410b2a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:59.438967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966860d-d8de-488f-ae03-17914410b2a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:59.442560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a36502-73c7-421e-9466-a2ad40bee0e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:59.444141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a36502-73c7-421e-9466-a2ad40bee0e7 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:59.451149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966860d-d8de-488f-ae03-17914410b2a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:59.452080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966860d-d8de-488f-ae03-17914410b2a1 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:55:59.456411 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1454/2980] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:55:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:55:59.462297 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1527/2981] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:55:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:55:59.869914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a36502-73c7-421e-9466-a2ad40bee0e7 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:55:59.872668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b1a36502-73c7-421e-9466-a2ad40bee0e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:55:59.910535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a36502-73c7-421e-9466-a2ad40bee0e7 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:55:59.915911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862039f-e858-49ba-b20b-3be247998c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:55:59.916556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862039f-e858-49ba-b20b-3be247998c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:55:59.916831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862039f-e858-49ba-b20b-3be247998c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:55:59.917108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862039f-e858-49ba-b20b-3be247998c57 admin admin] PATH_INFO: `/v3/projects/08fae3c0a9324c68b16dd495478a89e3` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:55:59.918250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862039f-e858-49ba-b20b-3be247998c57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08fae3c0a9324c68b16dd495478a89e3)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:55:59.920790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862039f-e858-49ba-b20b-3be247998c57 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:00.006345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a862039f-e858-49ba-b20b-3be247998c57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08fae3c0a9324c68b16dd495478a89e3', 'request_id': 'req-a862039f-e858-49ba-b20b-3be247998c57'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.021843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a862039f-e858-49ba-b20b-3be247998c57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08fae3c0a9324c68b16dd495478a89e3', 'request_id': 'req-a862039f-e858-49ba-b20b-3be247998c57'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.035185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a862039f-e858-49ba-b20b-3be247998c57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08fae3c0a9324c68b16dd495478a89e3', 'request_id': 'req-a862039f-e858-49ba-b20b-3be247998c57'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.054364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a862039f-e858-49ba-b20b-3be247998c57 admin admin] The token cache is being invalidate because project 08fae3c0a9324c68b16dd495478a89e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:56:00.054703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a862039f-e858-49ba-b20b-3be247998c57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.055402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a862039f-e858-49ba-b20b-3be247998c57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.057672 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1455/2982] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:55:59 2022] DELETE /identity/v3/projects/08fae3c0a9324c68b16dd495478a89e3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:00.061510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966860d-d8de-488f-ae03-17914410b2a1 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:00.084721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966860d-d8de-488f-ae03-17914410b2a1 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:00.115636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862039f-e858-49ba-b20b-3be247998c57 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:00.116974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a862039f-e858-49ba-b20b-3be247998c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:00.215460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862039f-e858-49ba-b20b-3be247998c57 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:00.221824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e674f7f-4fff-4836-9644-ecfddec6a50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:00.223548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e674f7f-4fff-4836-9644-ecfddec6a50e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:00.224086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e674f7f-4fff-4836-9644-ecfddec6a50e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:00.224588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e674f7f-4fff-4836-9644-ecfddec6a50e admin admin] PATH_INFO: `/v3/projects/29a8000bfb634471b905328904f6bb07` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:00.226044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e674f7f-4fff-4836-9644-ecfddec6a50e admin admin] RBAC: Authorizing `identity:delete_project(project_id=29a8000bfb634471b905328904f6bb07)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:00.229353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e674f7f-4fff-4836-9644-ecfddec6a50e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:00.234191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:00.234417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194610c7-203c-4a1f-af3b-f45cf7592f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:00.234980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194610c7-203c-4a1f-af3b-f45cf7592f13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:00.235320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194610c7-203c-4a1f-af3b-f45cf7592f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:00.235928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194610c7-203c-4a1f-af3b-f45cf7592f13 admin admin] PATH_INFO: `/v3/users/b12f27914a4544cf82521abd3e7db9a7` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:00.237141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194610c7-203c-4a1f-af3b-f45cf7592f13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b12f27914a4544cf82521abd3e7db9a7)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:00.239321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194610c7-203c-4a1f-af3b-f45cf7592f13 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:00.297049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:00.324979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1e674f7f-4fff-4836-9644-ecfddec6a50e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29a8000bfb634471b905328904f6bb07', 'request_id': 'req-1e674f7f-4fff-4836-9644-ecfddec6a50e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.339039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1e674f7f-4fff-4836-9644-ecfddec6a50e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29a8000bfb634471b905328904f6bb07', 'request_id': 'req-1e674f7f-4fff-4836-9644-ecfddec6a50e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.351344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-194610c7-203c-4a1f-af3b-f45cf7592f13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b12f27914a4544cf82521abd3e7db9a7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.353435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1e674f7f-4fff-4836-9644-ecfddec6a50e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29a8000bfb634471b905328904f6bb07', 'request_id': 'req-1e674f7f-4fff-4836-9644-ecfddec6a50e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.368855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-194610c7-203c-4a1f-af3b-f45cf7592f13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b12f27914a4544cf82521abd3e7db9a7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.374081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1e674f7f-4fff-4836-9644-ecfddec6a50e admin admin] The token cache is being invalidate because project 29a8000bfb634471b905328904f6bb07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:56:00.374643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1e674f7f-4fff-4836-9644-ecfddec6a50e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.377864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1e674f7f-4fff-4836-9644-ecfddec6a50e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.382404 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1456/2983] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:56:00 2022] DELETE /identity/v3/projects/29a8000bfb634471b905328904f6bb07 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:00.384002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e674f7f-4fff-4836-9644-ecfddec6a50e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:00.385513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1e674f7f-4fff-4836-9644-ecfddec6a50e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:00.422728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-194610c7-203c-4a1f-af3b-f45cf7592f13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b12f27914a4544cf82521abd3e7db9a7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.429494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-194610c7-203c-4a1f-af3b-f45cf7592f13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b12f27914a4544cf82521abd3e7db9a7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.450702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-194610c7-203c-4a1f-af3b-f45cf7592f13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b12f27914a4544cf82521abd3e7db9a7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.470222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e674f7f-4fff-4836-9644-ecfddec6a50e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:00.472235 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1528/2984] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:56:00 2022] DELETE /identity/v3/users/b12f27914a4544cf82521abd3e7db9a7 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:00.580874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69831f9a-62c5-4e41-9869-d8f7ba6f9aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:00.582347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69831f9a-62c5-4e41-9869-d8f7ba6f9aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:00.582847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69831f9a-62c5-4e41-9869-d8f7ba6f9aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:00.583308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69831f9a-62c5-4e41-9869-d8f7ba6f9aa6 admin admin] PATH_INFO: `/v3/users/09da6bb64857421eaf68a40d23a3b3e7` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:00.584767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69831f9a-62c5-4e41-9869-d8f7ba6f9aa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09da6bb64857421eaf68a40d23a3b3e7)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:00.589062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69831f9a-62c5-4e41-9869-d8f7ba6f9aa6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:00.660759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:00.678817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-69831f9a-62c5-4e41-9869-d8f7ba6f9aa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09da6bb64857421eaf68a40d23a3b3e7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.679504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-69831f9a-62c5-4e41-9869-d8f7ba6f9aa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09da6bb64857421eaf68a40d23a3b3e7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.680678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-69831f9a-62c5-4e41-9869-d8f7ba6f9aa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09da6bb64857421eaf68a40d23a3b3e7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.690257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-69831f9a-62c5-4e41-9869-d8f7ba6f9aa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09da6bb64857421eaf68a40d23a3b3e7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.705879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-69831f9a-62c5-4e41-9869-d8f7ba6f9aa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09da6bb64857421eaf68a40d23a3b3e7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.711430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194610c7-203c-4a1f-af3b-f45cf7592f13 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:00.712859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-194610c7-203c-4a1f-af3b-f45cf7592f13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:00.731537 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1457/2985] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:56:00 2022] DELETE /identity/v3/users/09da6bb64857421eaf68a40d23a3b3e7 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:00.760270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194610c7-203c-4a1f-af3b-f45cf7592f13 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:00.840576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c49298-b0fb-46e2-aedf-712ff8b7f781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:00.841316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c49298-b0fb-46e2-aedf-712ff8b7f781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:00.841728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c49298-b0fb-46e2-aedf-712ff8b7f781 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:00.842014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c49298-b0fb-46e2-aedf-712ff8b7f781 admin admin] PATH_INFO: `/v3/projects/9efda885854e4211bc1145a488270fbc` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:00.843656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c49298-b0fb-46e2-aedf-712ff8b7f781 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9efda885854e4211bc1145a488270fbc)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:00.846141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c49298-b0fb-46e2-aedf-712ff8b7f781 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:00.967836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05c49298-b0fb-46e2-aedf-712ff8b7f781 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9efda885854e4211bc1145a488270fbc', 'request_id': 'req-05c49298-b0fb-46e2-aedf-712ff8b7f781'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.987984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05c49298-b0fb-46e2-aedf-712ff8b7f781 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9efda885854e4211bc1145a488270fbc', 'request_id': 'req-05c49298-b0fb-46e2-aedf-712ff8b7f781'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:00.995188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05c49298-b0fb-46e2-aedf-712ff8b7f781 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9efda885854e4211bc1145a488270fbc', 'request_id': 'req-05c49298-b0fb-46e2-aedf-712ff8b7f781'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:01.014700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05c49298-b0fb-46e2-aedf-712ff8b7f781 admin admin] The token cache is being invalidate because project 9efda885854e4211bc1145a488270fbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:56:01.015461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05c49298-b0fb-46e2-aedf-712ff8b7f781 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:01.016490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05c49298-b0fb-46e2-aedf-712ff8b7f781 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:01.019759 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1529/2986] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:56:00 2022] DELETE /identity/v3/projects/9efda885854e4211bc1145a488270fbc => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:56:01.025321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5923ac0-a214-4fe1-94be-1012da5de10a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:01.025614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5923ac0-a214-4fe1-94be-1012da5de10a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:01.025888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5923ac0-a214-4fe1-94be-1012da5de10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:01.055228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c5923ac0-a214-4fe1-94be-1012da5de10a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:01.057493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c5923ac0-a214-4fe1-94be-1012da5de10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:01.136371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c49298-b0fb-46e2-aedf-712ff8b7f781 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:01.136843 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1458/2987] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:56:01.138123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-05c49298-b0fb-46e2-aedf-712ff8b7f781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:01.145218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02183b13-3b52-4662-b908-1988e37e8fad None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:01.145627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02183b13-3b52-4662-b908-1988e37e8fad None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:01.146553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02183b13-3b52-4662-b908-1988e37e8fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:01.163449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c49298-b0fb-46e2-aedf-712ff8b7f781 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:01.175059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfcea9-da01-428d-af19-082493e72103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:01.182921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-02183b13-3b52-4662-b908-1988e37e8fad None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:01.184393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-02183b13-3b52-4662-b908-1988e37e8fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:01.185317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfcea9-da01-428d-af19-082493e72103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:01.185655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfcea9-da01-428d-af19-082493e72103 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:01.186233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfcea9-da01-428d-af19-082493e72103 admin admin] PATH_INFO: `/v3/projects/acf7d3c47743482c97c2d776a05f2ff1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:01.188792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfcea9-da01-428d-af19-082493e72103 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acf7d3c47743482c97c2d776a05f2ff1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:01.191100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfcea9-da01-428d-af19-082493e72103 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:01.203436 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1459/2988] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:56:01.204955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02183b13-3b52-4662-b908-1988e37e8fad None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:01.220781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02183b13-3b52-4662-b908-1988e37e8fad None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:01.228234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:01.228469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a90f91-af8c-4b0c-98dc-45dbd9700ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:01.229144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a90f91-af8c-4b0c-98dc-45dbd9700ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:01.229395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a90f91-af8c-4b0c-98dc-45dbd9700ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:01.229675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a90f91-af8c-4b0c-98dc-45dbd9700ac6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:01.231377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a90f91-af8c-4b0c-98dc-45dbd9700ac6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:56:01.231772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a90f91-af8c-4b0c-98dc-45dbd9700ac6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:01.233502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a90f91-af8c-4b0c-98dc-45dbd9700ac6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:01.251337 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1460/2989] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:01.252412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a90f91-af8c-4b0c-98dc-45dbd9700ac6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:01.264298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ddbfcea9-da01-428d-af19-082493e72103 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acf7d3c47743482c97c2d776a05f2ff1', 'request_id': 'req-ddbfcea9-da01-428d-af19-082493e72103'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:01.264727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a90f91-af8c-4b0c-98dc-45dbd9700ac6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:01.276628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ddbfcea9-da01-428d-af19-082493e72103 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acf7d3c47743482c97c2d776a05f2ff1', 'request_id': 'req-ddbfcea9-da01-428d-af19-082493e72103'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:01.285799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ddbfcea9-da01-428d-af19-082493e72103 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acf7d3c47743482c97c2d776a05f2ff1', 'request_id': 'req-ddbfcea9-da01-428d-af19-082493e72103'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:01.308600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ddbfcea9-da01-428d-af19-082493e72103 admin admin] The token cache is being invalidate because project acf7d3c47743482c97c2d776a05f2ff1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:56:01.309005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ddbfcea9-da01-428d-af19-082493e72103 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:01.310328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ddbfcea9-da01-428d-af19-082493e72103 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:01.312868 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1530/2990] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:56:01 2022] DELETE /identity/v3/projects/acf7d3c47743482c97c2d776a05f2ff1 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:01.320465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffd0b3-31a8-4d80-a9c0-2a70aeacbda0 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:01.322089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffd0b3-31a8-4d80-a9c0-2a70aeacbda0 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:01.323058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffd0b3-31a8-4d80-a9c0-2a70aeacbda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:01.335292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa08218-39ce-40c0-ba3c-7e8b661c4a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:01.335815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa08218-39ce-40c0-ba3c-7e8b661c4a7d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:01.336126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa08218-39ce-40c0-ba3c-7e8b661c4a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:01.337014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa08218-39ce-40c0-ba3c-7e8b661c4a7d admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:01.337715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa08218-39ce-40c0-ba3c-7e8b661c4a7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:01.339696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa08218-39ce-40c0-ba3c-7e8b661c4a7d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:01.355172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5dffd0b3-31a8-4d80-a9c0-2a70aeacbda0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:01.356851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5dffd0b3-31a8-4d80-a9c0-2a70aeacbda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:01.370403 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1531/2991] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:01.378459 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1461/2992] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:56:01 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:01.390541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffd0b3-31a8-4d80-a9c0-2a70aeacbda0 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:01.395900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5dffd0b3-31a8-4d80-a9c0-2a70aeacbda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:01.416367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffd0b3-31a8-4d80-a9c0-2a70aeacbda0 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:01.452150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab409a-8d44-48b2-ac1d-25792b5a6391 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:01.452451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab409a-8d44-48b2-ac1d-25792b5a6391 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:01.452712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab409a-8d44-48b2-ac1d-25792b5a6391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:01.480385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-06ab409a-8d44-48b2-ac1d-25792b5a6391 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:01.481806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-06ab409a-8d44-48b2-ac1d-25792b5a6391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:01.510291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:01.559320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c7455-be30-43d6-9904-30e5c6c41947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:01.560935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c7455-be30-43d6-9904-30e5c6c41947 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:01.565168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c7455-be30-43d6-9904-30e5c6c41947 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:01.567798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c7455-be30-43d6-9904-30e5c6c41947 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:01.568848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c7455-be30-43d6-9904-30e5c6c41947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:01.572978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c7455-be30-43d6-9904-30e5c6c41947 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:01.583338 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1462/2993] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:56:01.584381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab409a-8d44-48b2-ac1d-25792b5a6391 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:01.597382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab409a-8d44-48b2-ac1d-25792b5a6391 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:01.604145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f14778-8bb9-41e5-be00-d5ff0df97990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:01.605227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f14778-8bb9-41e5-be00-d5ff0df97990 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:01.605453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f14778-8bb9-41e5-be00-d5ff0df97990 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:01.606330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f14778-8bb9-41e5-be00-d5ff0df97990 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:01.606654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f14778-8bb9-41e5-be00-d5ff0df97990 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:56:01.608248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f14778-8bb9-41e5-be00-d5ff0df97990 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:01.610482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f14778-8bb9-41e5-be00-d5ff0df97990 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:01.626123 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1532/2994] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:56:01 2022] POST /identity/v3/users => generated 287 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:01.630705 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1463/2995] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:56:01.630917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c7455-be30-43d6-9904-30e5c6c41947 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:01.635313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f14778-8bb9-41e5-be00-d5ff0df97990 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:01.649231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f14778-8bb9-41e5-be00-d5ff0df97990 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:01.656192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e549ff-79b0-4537-b6a1-e86baf3bd979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:01.656813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e549ff-79b0-4537-b6a1-e86baf3bd979 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:01.657124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e549ff-79b0-4537-b6a1-e86baf3bd979 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:01.657396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e549ff-79b0-4537-b6a1-e86baf3bd979 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:01.658985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e549ff-79b0-4537-b6a1-e86baf3bd979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:01.660064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c7455-be30-43d6-9904-30e5c6c41947 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:01.661759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e549ff-79b0-4537-b6a1-e86baf3bd979 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:01.681595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c6bfd-4594-4c43-ac77-053d8aa76fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:01.682515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c6bfd-4594-4c43-ac77-053d8aa76fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:01.682902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c6bfd-4594-4c43-ac77-053d8aa76fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:01.683026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c6bfd-4594-4c43-ac77-053d8aa76fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:01.683681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c6bfd-4594-4c43-ac77-053d8aa76fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:01.685366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c6bfd-4594-4c43-ac77-053d8aa76fd3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:01.723110 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1533/2996] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:56:01.723539 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1464/2997] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:56:01 2022] POST /identity/v3/projects => generated 407 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:56:01.727559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e549ff-79b0-4537-b6a1-e86baf3bd979 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:01.728267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c6bfd-4594-4c43-ac77-053d8aa76fd3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:01.749363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c6bfd-4594-4c43-ac77-053d8aa76fd3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:01.857417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e549ff-79b0-4537-b6a1-e86baf3bd979 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:01.868367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac88401-24a3-41f1-ba09-ac47259987ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:01.868992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac88401-24a3-41f1-ba09-ac47259987ea admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:01.869357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac88401-24a3-41f1-ba09-ac47259987ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:01.869766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac88401-24a3-41f1-ba09-ac47259987ea admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:01.871170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac88401-24a3-41f1-ba09-ac47259987ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:01.872676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac88401-24a3-41f1-ba09-ac47259987ea admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:01.879912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144feb0a-01bd-4b9f-b21e-4806fd724628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:01.881332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144feb0a-01bd-4b9f-b21e-4806fd724628 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:01.881520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144feb0a-01bd-4b9f-b21e-4806fd724628 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:01.881869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144feb0a-01bd-4b9f-b21e-4806fd724628 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:01.882598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144feb0a-01bd-4b9f-b21e-4806fd724628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:01.884394 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1465/2998] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:56:01.887060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144feb0a-01bd-4b9f-b21e-4806fd724628 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:01.888956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac88401-24a3-41f1-ba09-ac47259987ea admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:01.901583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac88401-24a3-41f1-ba09-ac47259987ea admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:01.907190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb694d1-1854-40e0-9ecb-954a9a13e48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:01.907952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb694d1-1854-40e0-9ecb-954a9a13e48a admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:01.908293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb694d1-1854-40e0-9ecb-954a9a13e48a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:01.908587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb694d1-1854-40e0-9ecb-954a9a13e48a admin admin] PATH_INFO: `/v3/projects/6f9e92bfb756442983da09a9adfe2e88/users/da040526fe7b490784ea27f2572f0b7b/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:01.925253 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1534/2999] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:56:01 2022] POST /identity/v3/users => generated 304 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:01.927076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144feb0a-01bd-4b9f-b21e-4806fd724628 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:01.932303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb694d1-1854-40e0-9ecb-954a9a13e48a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f9e92bfb756442983da09a9adfe2e88, user_id=da040526fe7b490784ea27f2572f0b7b, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:01.935459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb694d1-1854-40e0-9ecb-954a9a13e48a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:01.942595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144feb0a-01bd-4b9f-b21e-4806fd724628 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:01.952192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61c01d-103e-42dd-9a89-40453a896532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:01.953246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61c01d-103e-42dd-9a89-40453a896532 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:01.953387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61c01d-103e-42dd-9a89-40453a896532 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:01.953788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61c01d-103e-42dd-9a89-40453a896532 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:01.954569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61c01d-103e-42dd-9a89-40453a896532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:01.956585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61c01d-103e-42dd-9a89-40453a896532 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:01.963099 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1466/3000] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:01 2022] PUT /identity/v3/projects/6f9e92bfb756442983da09a9adfe2e88/users/da040526fe7b490784ea27f2572f0b7b/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:01.975240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d56b16-8a93-40a7-8dee-b8652d7e25ca None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:01.976223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d56b16-8a93-40a7-8dee-b8652d7e25ca None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:01.976223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d56b16-8a93-40a7-8dee-b8652d7e25ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:01.980681 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1535/3001] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:56:01.982801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61c01d-103e-42dd-9a89-40453a896532 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:01.993054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=87,keystone:DELETE=6,keystone:INSERT=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:02.004708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61c01d-103e-42dd-9a89-40453a896532 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:02.043379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d2d56b16-8a93-40a7-8dee-b8652d7e25ca None None] MFA Rules not processed for user `da040526fe7b490784ea27f2572f0b7b`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:02.045069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d2d56b16-8a93-40a7-8dee-b8652d7e25ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:02.095551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce976d-f0f7-491d-abde-1337b84ff5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:02.098116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce976d-f0f7-491d-abde-1337b84ff5eb admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:02.098676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce976d-f0f7-491d-abde-1337b84ff5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:02.100031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce976d-f0f7-491d-abde-1337b84ff5eb admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:02.101181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce976d-f0f7-491d-abde-1337b84ff5eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:02.104065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce976d-f0f7-491d-abde-1337b84ff5eb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:02.114387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:02.120703 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1536/3002] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:56:02.126252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce976d-f0f7-491d-abde-1337b84ff5eb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:02.141040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce976d-f0f7-491d-abde-1337b84ff5eb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:02.148107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe002df-c633-4218-82fa-658c2be46aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:02.149596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe002df-c633-4218-82fa-658c2be46aed admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:02.149992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe002df-c633-4218-82fa-658c2be46aed admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:02.150725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe002df-c633-4218-82fa-658c2be46aed admin admin] PATH_INFO: `/v3/projects/7054880d930f47089965ef038e7e7fcd/users/75d801ac314947efa34662acf7838a15/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:02.155226 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1467/3003] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:56:02.158898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d56b16-8a93-40a7-8dee-b8652d7e25ca None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:02.172445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d56b16-8a93-40a7-8dee-b8652d7e25ca None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:02.180096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ad439-7816-4e9a-a7ef-67aeb8f850a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:02.180997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ad439-7816-4e9a-a7ef-67aeb8f850a4 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:02.181292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ad439-7816-4e9a-a7ef-67aeb8f850a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:02.181572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ad439-7816-4e9a-a7ef-67aeb8f850a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:02.182209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ad439-7816-4e9a-a7ef-67aeb8f850a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:02.183805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe002df-c633-4218-82fa-658c2be46aed admin admin] RBAC: Authorizing `identity:create_grant(project_id=7054880d930f47089965ef038e7e7fcd, user_id=75d801ac314947efa34662acf7838a15, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:02.185328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ad439-7816-4e9a-a7ef-67aeb8f850a4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:02.189190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe002df-c633-4218-82fa-658c2be46aed admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:02.208290 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1537/3004] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:02 2022] PUT /identity/v3/projects/7054880d930f47089965ef038e7e7fcd/users/75d801ac314947efa34662acf7838a15/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:02.221571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e799e-c4c5-48ba-bc68-c5d2b8869a11 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:02.221985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e799e-c4c5-48ba-bc68-c5d2b8869a11 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:02.222363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e799e-c4c5-48ba-bc68-c5d2b8869a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:02.223680 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1468/3005] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:56:02 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:02.228670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ad439-7816-4e9a-a7ef-67aeb8f850a4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:02.257862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ab7e799e-c4c5-48ba-bc68-c5d2b8869a11 None None] MFA Rules not processed for user `75d801ac314947efa34662acf7838a15`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:02.260149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ab7e799e-c4c5-48ba-bc68-c5d2b8869a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:02.310657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ad439-7816-4e9a-a7ef-67aeb8f850a4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:02.317613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ed68a-6ee3-43a4-8a71-1184771d8529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:02.317916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ed68a-6ee3-43a4-8a71-1184771d8529 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:02.318210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ed68a-6ee3-43a4-8a71-1184771d8529 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:02.318416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ed68a-6ee3-43a4-8a71-1184771d8529 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:02.319096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ed68a-6ee3-43a4-8a71-1184771d8529 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:02.321277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ed68a-6ee3-43a4-8a71-1184771d8529 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:02.345374 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1469/3006] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:56:02 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:02.349537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ed68a-6ee3-43a4-8a71-1184771d8529 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:02.358004 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1538/3007] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:02 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:02.360513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e799e-c4c5-48ba-bc68-c5d2b8869a11 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:02.363272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ed68a-6ee3-43a4-8a71-1184771d8529 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:02.369414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a5f46-4e78-4d3b-9ff8-a32f70846dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:02.370248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a5f46-4e78-4d3b-9ff8-a32f70846dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:02.370605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a5f46-4e78-4d3b-9ff8-a32f70846dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:02.370889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a5f46-4e78-4d3b-9ff8-a32f70846dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:02.371597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a5f46-4e78-4d3b-9ff8-a32f70846dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:02.371755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e799e-c4c5-48ba-bc68-c5d2b8869a11 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:02.374082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a5f46-4e78-4d3b-9ff8-a32f70846dd5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:02.376604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60b7c7-c4d1-4af4-b63e-b33258dd1aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:02.377214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60b7c7-c4d1-4af4-b63e-b33258dd1aee admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:02.377527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60b7c7-c4d1-4af4-b63e-b33258dd1aee admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:02.377896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60b7c7-c4d1-4af4-b63e-b33258dd1aee admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:02.378393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed60b7c7-c4d1-4af4-b63e-b33258dd1aee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:02.380313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed60b7c7-c4d1-4af4-b63e-b33258dd1aee admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:02.387941 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1470/3008] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:56:02.392912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a5f46-4e78-4d3b-9ff8-a32f70846dd5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:02.406454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a5f46-4e78-4d3b-9ff8-a32f70846dd5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:02.410426 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1539/3009] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:56:02 2022] POST /identity/v3/projects => generated 409 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:56:02.411922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a1a89-93f9-430b-be35-e7dee6d85c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:02.412593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a1a89-93f9-430b-be35-e7dee6d85c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:02.412875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a1a89-93f9-430b-be35-e7dee6d85c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:02.413181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a1a89-93f9-430b-be35-e7dee6d85c70 admin admin] PATH_INFO: `/v3/projects/217f0757a4e54c09b7a0f9ae01d575be/users/662e58b7335544d69d980be75c13e8aa/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:02.415167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60b7c7-c4d1-4af4-b63e-b33258dd1aee admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:02.430954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a1a89-93f9-430b-be35-e7dee6d85c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=217f0757a4e54c09b7a0f9ae01d575be, user_id=662e58b7335544d69d980be75c13e8aa, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:02.433629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a1a89-93f9-430b-be35-e7dee6d85c70 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:02.449835 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1471/3010] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:02 2022] PUT /identity/v3/projects/217f0757a4e54c09b7a0f9ae01d575be/users/662e58b7335544d69d980be75c13e8aa/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:56:02.453988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a1a89-93f9-430b-be35-e7dee6d85c70 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:02.470020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:02.496100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60b7c7-c4d1-4af4-b63e-b33258dd1aee admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:02.538743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a1a89-93f9-430b-be35-e7dee6d85c70 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:02.547207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1c379-b15e-4ce1-8bda-cf43084b8392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:02.548263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1c379-b15e-4ce1-8bda-cf43084b8392 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:02.548263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1c379-b15e-4ce1-8bda-cf43084b8392 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:02.548518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1c379-b15e-4ce1-8bda-cf43084b8392 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:02.549090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e1c379-b15e-4ce1-8bda-cf43084b8392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:02.550820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e1c379-b15e-4ce1-8bda-cf43084b8392 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:02.570158 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1472/3011] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:56:02.571080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbf130-4062-4e12-a4f8-5e02cf902deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:02.571793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbf130-4062-4e12-a4f8-5e02cf902deb admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:02.572081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbf130-4062-4e12-a4f8-5e02cf902deb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:02.572484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbf130-4062-4e12-a4f8-5e02cf902deb admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:02.573123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccbf130-4062-4e12-a4f8-5e02cf902deb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:02.575008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1c379-b15e-4ce1-8bda-cf43084b8392 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:02.575277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccbf130-4062-4e12-a4f8-5e02cf902deb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:02.588197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1c379-b15e-4ce1-8bda-cf43084b8392 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:02.595052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924e623-c232-47ce-91fb-a62b83465472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:02.595780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924e623-c232-47ce-91fb-a62b83465472 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:02.596079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924e623-c232-47ce-91fb-a62b83465472 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:02.596328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924e623-c232-47ce-91fb-a62b83465472 admin admin] PATH_INFO: `/v3/projects/217f0757a4e54c09b7a0f9ae01d575be/users/662e58b7335544d69d980be75c13e8aa/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:02.598235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924e623-c232-47ce-91fb-a62b83465472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=217f0757a4e54c09b7a0f9ae01d575be, user_id=662e58b7335544d69d980be75c13e8aa, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:02.599779 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1540/3012] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:56:02 2022] POST /identity/v3/users => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:56:02.600917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924e623-c232-47ce-91fb-a62b83465472 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:02.603922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbf130-4062-4e12-a4f8-5e02cf902deb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:02.616746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbf130-4062-4e12-a4f8-5e02cf902deb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:02.624287 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1473/3013] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:02 2022] PUT /identity/v3/projects/217f0757a4e54c09b7a0f9ae01d575be/users/662e58b7335544d69d980be75c13e8aa/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:56:02.636932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036d74d-4cea-496d-a3fc-1ec1c98cee32 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:02.638091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036d74d-4cea-496d-a3fc-1ec1c98cee32 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:02.638901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036d74d-4cea-496d-a3fc-1ec1c98cee32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:02.659161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:02.677013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b036d74d-4cea-496d-a3fc-1ec1c98cee32 None None] MFA Rules not processed for user `662e58b7335544d69d980be75c13e8aa`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:02.678439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b036d74d-4cea-496d-a3fc-1ec1c98cee32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:02.692452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dddf5b-931a-4e8d-8c01-aa20e7d040c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:02.693162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dddf5b-931a-4e8d-8c01-aa20e7d040c9 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:02.693516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dddf5b-931a-4e8d-8c01-aa20e7d040c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:02.693886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dddf5b-931a-4e8d-8c01-aa20e7d040c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:02.694680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dddf5b-931a-4e8d-8c01-aa20e7d040c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:02.696882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dddf5b-931a-4e8d-8c01-aa20e7d040c9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:02.707861 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1541/3014] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:56:02.712471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dddf5b-931a-4e8d-8c01-aa20e7d040c9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:02.725351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dddf5b-931a-4e8d-8c01-aa20e7d040c9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:02.730542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee50ad-fee0-43c3-99b8-40ce0f38eec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:02.731880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee50ad-fee0-43c3-99b8-40ce0f38eec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:02.732118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee50ad-fee0-43c3-99b8-40ce0f38eec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:02.732395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee50ad-fee0-43c3-99b8-40ce0f38eec5 admin admin] PATH_INFO: `/v3/projects/d816c76fc476424c8b86b3860e650665/users/74d06155c4f1415fb7db26bba621e7f7/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:02.752249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee50ad-fee0-43c3-99b8-40ce0f38eec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d816c76fc476424c8b86b3860e650665, user_id=74d06155c4f1415fb7db26bba621e7f7, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:02.758866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee50ad-fee0-43c3-99b8-40ce0f38eec5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:02.774564 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1542/3015] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:02 2022] PUT /identity/v3/projects/d816c76fc476424c8b86b3860e650665/users/74d06155c4f1415fb7db26bba621e7f7/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:56:02.777152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee50ad-fee0-43c3-99b8-40ce0f38eec5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:02.784690 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1474/3016] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:02.807528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be4572-e1b5-4879-ab88-05f721153a7a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:02.807898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be4572-e1b5-4879-ab88-05f721153a7a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:02.808831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be4572-e1b5-4879-ab88-05f721153a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:02.829305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-94be4572-e1b5-4879-ab88-05f721153a7a None None] MFA Rules not processed for user `da040526fe7b490784ea27f2572f0b7b`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:02.830616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-94be4572-e1b5-4879-ab88-05f721153a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:02.847165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:02.861140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee50ad-fee0-43c3-99b8-40ce0f38eec5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:02.866543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb8804-06aa-44a6-89aa-1d3ba073799d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:02.867955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb8804-06aa-44a6-89aa-1d3ba073799d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:02.868270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb8804-06aa-44a6-89aa-1d3ba073799d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:02.868645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb8804-06aa-44a6-89aa-1d3ba073799d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:02.869678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb8804-06aa-44a6-89aa-1d3ba073799d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:02.871985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb8804-06aa-44a6-89aa-1d3ba073799d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:02.884420 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1543/3017] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:56:02.888735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb8804-06aa-44a6-89aa-1d3ba073799d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:02.900638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb8804-06aa-44a6-89aa-1d3ba073799d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:02.905783 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1475/3018] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:56:02.907241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb5880-1b72-4e50-8f45-2240998e5b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:02.907976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb5880-1b72-4e50-8f45-2240998e5b86 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:02.908437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb5880-1b72-4e50-8f45-2240998e5b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:02.908775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb5880-1b72-4e50-8f45-2240998e5b86 admin admin] PATH_INFO: `/v3/projects/d816c76fc476424c8b86b3860e650665/users/74d06155c4f1415fb7db26bba621e7f7/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:02.910820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb5880-1b72-4e50-8f45-2240998e5b86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d816c76fc476424c8b86b3860e650665, user_id=74d06155c4f1415fb7db26bba621e7f7, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:02.913425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb5880-1b72-4e50-8f45-2240998e5b86 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:02.917296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be4572-e1b5-4879-ab88-05f721153a7a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:02.919294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-94be4572-e1b5-4879-ab88-05f721153a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:02.925697 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1544/3019] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:02 2022] PUT /identity/v3/projects/d816c76fc476424c8b86b3860e650665/users/74d06155c4f1415fb7db26bba621e7f7/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:02.937532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea050dc-7c1c-4be4-a0aa-bc352c9dcac4 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:02.937936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea050dc-7c1c-4be4-a0aa-bc352c9dcac4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:02.938262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea050dc-7c1c-4be4-a0aa-bc352c9dcac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:02.987537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0ea050dc-7c1c-4be4-a0aa-bc352c9dcac4 None None] MFA Rules not processed for user `74d06155c4f1415fb7db26bba621e7f7`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:02.987537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0ea050dc-7c1c-4be4-a0aa-bc352c9dcac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:03.051945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be4572-e1b5-4879-ab88-05f721153a7a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:03.074056 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1545/3020] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:02 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:03.081140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea050dc-7c1c-4be4-a0aa-bc352c9dcac4 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:03.088647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:03.125819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3531cc4-8a02-490b-8bd1-bd959a2eca14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:03.127243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3531cc4-8a02-490b-8bd1-bd959a2eca14 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:03.127604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3531cc4-8a02-490b-8bd1-bd959a2eca14 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:03.128409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3531cc4-8a02-490b-8bd1-bd959a2eca14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:03.157579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea050dc-7c1c-4be4-a0aa-bc352c9dcac4 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:03.165103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce9164-6945-46b8-9598-fd5bf9e10305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:03.165743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce9164-6945-46b8-9598-fd5bf9e10305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:03.166046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce9164-6945-46b8-9598-fd5bf9e10305 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:03.166278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce9164-6945-46b8-9598-fd5bf9e10305 admin admin] PATH_INFO: `/v3/users/75d801ac314947efa34662acf7838a15` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:03.167466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce9164-6945-46b8-9598-fd5bf9e10305 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75d801ac314947efa34662acf7838a15)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:03.169545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce9164-6945-46b8-9598-fd5bf9e10305 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:03.201225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3531cc4-8a02-490b-8bd1-bd959a2eca14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:03.203507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3531cc4-8a02-490b-8bd1-bd959a2eca14 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:03.216441 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1476/3021] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:56:03.244783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3ce9164-6945-46b8-9598-fd5bf9e10305 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75d801ac314947efa34662acf7838a15'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:03.255972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3ce9164-6945-46b8-9598-fd5bf9e10305 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75d801ac314947efa34662acf7838a15'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:03.262198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:03.274540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3ce9164-6945-46b8-9598-fd5bf9e10305 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75d801ac314947efa34662acf7838a15'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:03.274938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3ce9164-6945-46b8-9598-fd5bf9e10305 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75d801ac314947efa34662acf7838a15'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:03.284061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3ce9164-6945-46b8-9598-fd5bf9e10305 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75d801ac314947efa34662acf7838a15'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:03.290978 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1546/3022] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:56:03 2022] DELETE /identity/v3/users/75d801ac314947efa34662acf7838a15 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:56:03.295513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74e1a8-407f-4b50-b01f-b83522818bb1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:03.295809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74e1a8-407f-4b50-b01f-b83522818bb1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:03.296113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74e1a8-407f-4b50-b01f-b83522818bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:03.323935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ab74e1a8-407f-4b50-b01f-b83522818bb1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:03.325575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ab74e1a8-407f-4b50-b01f-b83522818bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:03.406365 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1477/3023] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:03.441467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce9164-6945-46b8-9598-fd5bf9e10305 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:03.442822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e3ce9164-6945-46b8-9598-fd5bf9e10305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:03.529618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce9164-6945-46b8-9598-fd5bf9e10305 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:03.533654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c1727-45b9-4099-9e1d-6c9177722a45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:03.534300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c1727-45b9-4099-9e1d-6c9177722a45 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:03.534615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c1727-45b9-4099-9e1d-6c9177722a45 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:03.534896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c1727-45b9-4099-9e1d-6c9177722a45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:03.550525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c1727-45b9-4099-9e1d-6c9177722a45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:03.552146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c1727-45b9-4099-9e1d-6c9177722a45 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:03.563580 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1547/3024] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:56:04.104013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74e1a8-407f-4b50-b01f-b83522818bb1 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:04.106680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ab74e1a8-407f-4b50-b01f-b83522818bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:04.140055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74e1a8-407f-4b50-b01f-b83522818bb1 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:04.151718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d30a1d-4107-4692-b156-1936030c0916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:04.152733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d30a1d-4107-4692-b156-1936030c0916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:04.153388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d30a1d-4107-4692-b156-1936030c0916 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:04.153664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d30a1d-4107-4692-b156-1936030c0916 admin admin] PATH_INFO: `/v3/projects/7054880d930f47089965ef038e7e7fcd` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:04.156417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d30a1d-4107-4692-b156-1936030c0916 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7054880d930f47089965ef038e7e7fcd)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:04.165125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d30a1d-4107-4692-b156-1936030c0916 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:04.297795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a3d30a1d-4107-4692-b156-1936030c0916 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7054880d930f47089965ef038e7e7fcd', 'request_id': 'req-a3d30a1d-4107-4692-b156-1936030c0916'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:04.312093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a3d30a1d-4107-4692-b156-1936030c0916 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7054880d930f47089965ef038e7e7fcd', 'request_id': 'req-a3d30a1d-4107-4692-b156-1936030c0916'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:04.324659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a3d30a1d-4107-4692-b156-1936030c0916 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7054880d930f47089965ef038e7e7fcd', 'request_id': 'req-a3d30a1d-4107-4692-b156-1936030c0916'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:04.327762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:04.360044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a3d30a1d-4107-4692-b156-1936030c0916 admin admin] The token cache is being invalidate because project 7054880d930f47089965ef038e7e7fcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:56:04.360732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a3d30a1d-4107-4692-b156-1936030c0916 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:04.361710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a3d30a1d-4107-4692-b156-1936030c0916 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:04.363470 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1478/3025] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:56:04 2022] DELETE /identity/v3/projects/7054880d930f47089965ef038e7e7fcd => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:56:04.369403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c1727-45b9-4099-9e1d-6c9177722a45 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:04.388578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c1727-45b9-4099-9e1d-6c9177722a45 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:04.501210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59708b-4e45-4dc8-b94e-1dd18e5d00f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:04.503619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59708b-4e45-4dc8-b94e-1dd18e5d00f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:04.504178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59708b-4e45-4dc8-b94e-1dd18e5d00f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:04.504462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59708b-4e45-4dc8-b94e-1dd18e5d00f8 admin admin] PATH_INFO: `/v3/users/74d06155c4f1415fb7db26bba621e7f7` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:04.507003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59708b-4e45-4dc8-b94e-1dd18e5d00f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74d06155c4f1415fb7db26bba621e7f7)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:04.509376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59708b-4e45-4dc8-b94e-1dd18e5d00f8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:04.571059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a59708b-4e45-4dc8-b94e-1dd18e5d00f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74d06155c4f1415fb7db26bba621e7f7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:04.585513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a59708b-4e45-4dc8-b94e-1dd18e5d00f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74d06155c4f1415fb7db26bba621e7f7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:04.608887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a59708b-4e45-4dc8-b94e-1dd18e5d00f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74d06155c4f1415fb7db26bba621e7f7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:04.610940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a59708b-4e45-4dc8-b94e-1dd18e5d00f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74d06155c4f1415fb7db26bba621e7f7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:04.622521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a59708b-4e45-4dc8-b94e-1dd18e5d00f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74d06155c4f1415fb7db26bba621e7f7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:04.626445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:04.637495 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1548/3026] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:56:04 2022] DELETE /identity/v3/users/74d06155c4f1415fb7db26bba621e7f7 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:05.721041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d30a1d-4107-4692-b156-1936030c0916 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:05.723452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a3d30a1d-4107-4692-b156-1936030c0916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:05.752799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d30a1d-4107-4692-b156-1936030c0916 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:05.891779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81874bd5-dd12-4a0c-b8e1-811d0a7f3c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:05.892606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81874bd5-dd12-4a0c-b8e1-811d0a7f3c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:05.893268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81874bd5-dd12-4a0c-b8e1-811d0a7f3c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:05.893553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81874bd5-dd12-4a0c-b8e1-811d0a7f3c2b admin admin] PATH_INFO: `/v3/projects/d816c76fc476424c8b86b3860e650665` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:05.895098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81874bd5-dd12-4a0c-b8e1-811d0a7f3c2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d816c76fc476424c8b86b3860e650665)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:05.897345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81874bd5-dd12-4a0c-b8e1-811d0a7f3c2b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:05.987748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-81874bd5-dd12-4a0c-b8e1-811d0a7f3c2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd816c76fc476424c8b86b3860e650665', 'request_id': 'req-81874bd5-dd12-4a0c-b8e1-811d0a7f3c2b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:06.000112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-81874bd5-dd12-4a0c-b8e1-811d0a7f3c2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd816c76fc476424c8b86b3860e650665', 'request_id': 'req-81874bd5-dd12-4a0c-b8e1-811d0a7f3c2b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:06.008890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-81874bd5-dd12-4a0c-b8e1-811d0a7f3c2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd816c76fc476424c8b86b3860e650665', 'request_id': 'req-81874bd5-dd12-4a0c-b8e1-811d0a7f3c2b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:06.036657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-81874bd5-dd12-4a0c-b8e1-811d0a7f3c2b admin admin] The token cache is being invalidate because project d816c76fc476424c8b86b3860e650665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:56:06.037178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-81874bd5-dd12-4a0c-b8e1-811d0a7f3c2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:06.037677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-81874bd5-dd12-4a0c-b8e1-811d0a7f3c2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:06.039750 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1479/3027] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:56:05 2022] DELETE /identity/v3/projects/d816c76fc476424c8b86b3860e650665 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:06.046751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff07a55-b707-47f1-96d2-427921e601f9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:06.047298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff07a55-b707-47f1-96d2-427921e601f9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:06.047298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff07a55-b707-47f1-96d2-427921e601f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:06.073775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9ff07a55-b707-47f1-96d2-427921e601f9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:06.076321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9ff07a55-b707-47f1-96d2-427921e601f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:06.189773 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1549/3028] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:06.202305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3eed26-afd7-438f-a315-ebdace3a044f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:06.202543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3eed26-afd7-438f-a315-ebdace3a044f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:06.202814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3eed26-afd7-438f-a315-ebdace3a044f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:06.228557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5b3eed26-afd7-438f-a315-ebdace3a044f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:06.230051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5b3eed26-afd7-438f-a315-ebdace3a044f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:06.245344 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1480/3029] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:56:06.246231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff07a55-b707-47f1-96d2-427921e601f9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:06.257537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff07a55-b707-47f1-96d2-427921e601f9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:06.265987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81bd28-c704-43c7-b6ac-b4f3e71546eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:06.266689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81bd28-c704-43c7-b6ac-b4f3e71546eb admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:06.267121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81bd28-c704-43c7-b6ac-b4f3e71546eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:06.267694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81bd28-c704-43c7-b6ac-b4f3e71546eb admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:06.268377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81bd28-c704-43c7-b6ac-b4f3e71546eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:56:06.268688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81bd28-c704-43c7-b6ac-b4f3e71546eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:06.270996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81bd28-c704-43c7-b6ac-b4f3e71546eb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:06.297456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3eed26-afd7-438f-a315-ebdace3a044f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:06.304698 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1550/3030] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:06.312147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3eed26-afd7-438f-a315-ebdace3a044f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:06.319974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fc84a-098d-4da5-a666-0e16dc1670e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:06.321149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fc84a-098d-4da5-a666-0e16dc1670e2 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:06.321429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fc84a-098d-4da5-a666-0e16dc1670e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:06.321701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fc84a-098d-4da5-a666-0e16dc1670e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:06.322338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46fc84a-098d-4da5-a666-0e16dc1670e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:06.324467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46fc84a-098d-4da5-a666-0e16dc1670e2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:06.366532 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1481/3031] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:56:06 2022] POST /identity/v3/projects => generated 391 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:56:06.368556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81bd28-c704-43c7-b6ac-b4f3e71546eb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:06.380964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81bd28-c704-43c7-b6ac-b4f3e71546eb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:06.451818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b238477-1f43-455e-bae5-7aca1ad44079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:06.452434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b238477-1f43-455e-bae5-7aca1ad44079 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:06.452735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b238477-1f43-455e-bae5-7aca1ad44079 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:06.453072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b238477-1f43-455e-bae5-7aca1ad44079 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:06.453682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b238477-1f43-455e-bae5-7aca1ad44079 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:06.455643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b238477-1f43-455e-bae5-7aca1ad44079 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:06.479779 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1551/3032] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:56:06 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:56:06.484033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fc84a-098d-4da5-a666-0e16dc1670e2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:06.496380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fc84a-098d-4da5-a666-0e16dc1670e2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:06.512235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec300242-7bbf-492f-a588-21c9407265b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:06.512971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec300242-7bbf-492f-a588-21c9407265b3 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:06.513309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec300242-7bbf-492f-a588-21c9407265b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:06.513662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec300242-7bbf-492f-a588-21c9407265b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:06.514562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec300242-7bbf-492f-a588-21c9407265b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:06.516835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec300242-7bbf-492f-a588-21c9407265b3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:06.532106 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1482/3033] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:56:06.536539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b238477-1f43-455e-bae5-7aca1ad44079 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:06.549244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b238477-1f43-455e-bae5-7aca1ad44079 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:06.555779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7b990-e513-432e-9e42-ee1f5c2796b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:06.556486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7b990-e513-432e-9e42-ee1f5c2796b1 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:06.556827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7b990-e513-432e-9e42-ee1f5c2796b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:06.557071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7b990-e513-432e-9e42-ee1f5c2796b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:06.557753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7b990-e513-432e-9e42-ee1f5c2796b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:06.564244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7b990-e513-432e-9e42-ee1f5c2796b1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:06.577778 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1552/3034] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:56:06.583046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec300242-7bbf-492f-a588-21c9407265b3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:06.594752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec300242-7bbf-492f-a588-21c9407265b3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:06.601238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da174d-21fd-4b37-b758-78567cf337ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:06.602129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da174d-21fd-4b37-b758-78567cf337ea admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:06.602413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da174d-21fd-4b37-b758-78567cf337ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:06.602744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da174d-21fd-4b37-b758-78567cf337ea admin admin] PATH_INFO: `/v3/projects/ef52ce9c74be428d92e51a3db71d83b0/users/8a5c96e82db44cd39a218d6ce3575e18/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:06.621998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da174d-21fd-4b37-b758-78567cf337ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef52ce9c74be428d92e51a3db71d83b0, user_id=8a5c96e82db44cd39a218d6ce3575e18, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:06.624923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da174d-21fd-4b37-b758-78567cf337ea admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:06.642042 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1483/3035] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:06 2022] PUT /identity/v3/projects/ef52ce9c74be428d92e51a3db71d83b0/users/8a5c96e82db44cd39a218d6ce3575e18/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:06.654111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a7404-96f7-4cd8-8dfd-a724180ea59f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:06.655418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a7404-96f7-4cd8-8dfd-a724180ea59f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:06.655802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a7404-96f7-4cd8-8dfd-a724180ea59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:06.691991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7b7a7404-96f7-4cd8-8dfd-a724180ea59f None None] MFA Rules not processed for user `8a5c96e82db44cd39a218d6ce3575e18`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:06.693565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7b7a7404-96f7-4cd8-8dfd-a724180ea59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:06.758727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:06.769625 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1553/3036] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:06.774222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da174d-21fd-4b37-b758-78567cf337ea admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:06.799963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:06.839651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da174d-21fd-4b37-b758-78567cf337ea admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:06.846169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a714be-fe94-41c9-9319-2efde548c9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:06.846841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a714be-fe94-41c9-9319-2efde548c9ce admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:06.847160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a714be-fe94-41c9-9319-2efde548c9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:06.847559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a714be-fe94-41c9-9319-2efde548c9ce admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:06.848484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a714be-fe94-41c9-9319-2efde548c9ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:06.851787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a714be-fe94-41c9-9319-2efde548c9ce admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:06.885916 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1484/3037] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:56:06 2022] POST /identity/v3/projects => generated 389 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:06.890684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a7404-96f7-4cd8-8dfd-a724180ea59f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:06.903339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a7404-96f7-4cd8-8dfd-a724180ea59f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:06.965846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7084fc3-f1cc-4b4c-a5a1-36defc76fb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:06.966458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7084fc3-f1cc-4b4c-a5a1-36defc76fb84 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:06.966770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7084fc3-f1cc-4b4c-a5a1-36defc76fb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:06.967046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7084fc3-f1cc-4b4c-a5a1-36defc76fb84 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:06.967675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7084fc3-f1cc-4b4c-a5a1-36defc76fb84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:06.969582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7084fc3-f1cc-4b4c-a5a1-36defc76fb84 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:06.990344 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1554/3038] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:56:06 2022] POST /identity/v3/users => generated 301 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:06.995078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a714be-fe94-41c9-9319-2efde548c9ce admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:07.005494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a714be-fe94-41c9-9319-2efde548c9ce admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:07.011299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858a518-c198-4448-af14-1496ac9c60a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:07.011883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858a518-c198-4448-af14-1496ac9c60a8 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:07.012158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858a518-c198-4448-af14-1496ac9c60a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:07.012425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858a518-c198-4448-af14-1496ac9c60a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:07.013050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858a518-c198-4448-af14-1496ac9c60a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:07.014662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858a518-c198-4448-af14-1496ac9c60a8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:07.025317 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1485/3039] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:56:07.029723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7084fc3-f1cc-4b4c-a5a1-36defc76fb84 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:07.041133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7084fc3-f1cc-4b4c-a5a1-36defc76fb84 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:07.046947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c38ba-83bb-4a06-9cf0-846134c9b20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:07.047650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c38ba-83bb-4a06-9cf0-846134c9b20a admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:07.047879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c38ba-83bb-4a06-9cf0-846134c9b20a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:07.048152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c38ba-83bb-4a06-9cf0-846134c9b20a admin admin] PATH_INFO: `/v3/projects/7b22c35747b84ed69c900e69faf00341/users/1d74da9f2da54efaa57b33b5ba496388/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:07.069653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c38ba-83bb-4a06-9cf0-846134c9b20a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b22c35747b84ed69c900e69faf00341, user_id=1d74da9f2da54efaa57b33b5ba496388, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:07.073007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c38ba-83bb-4a06-9cf0-846134c9b20a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:07.088060 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1555/3040] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:07 2022] PUT /identity/v3/projects/7b22c35747b84ed69c900e69faf00341/users/1d74da9f2da54efaa57b33b5ba496388/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:56:07.092171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858a518-c198-4448-af14-1496ac9c60a8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:07.112153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858a518-c198-4448-af14-1496ac9c60a8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:07.185912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613afc16-3b40-4410-b471-cb772d19968a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:07.187194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613afc16-3b40-4410-b471-cb772d19968a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:07.187693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613afc16-3b40-4410-b471-cb772d19968a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:07.188867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613afc16-3b40-4410-b471-cb772d19968a admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:07.188867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613afc16-3b40-4410-b471-cb772d19968a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:07.191276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613afc16-3b40-4410-b471-cb772d19968a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:07.206425 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1486/3041] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:56:07.210744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c38ba-83bb-4a06-9cf0-846134c9b20a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:07.221345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c38ba-83bb-4a06-9cf0-846134c9b20a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:07.225926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642bc89-ff1d-49e0-9259-73235916a5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:07.227246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642bc89-ff1d-49e0-9259-73235916a5fd admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:07.227530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642bc89-ff1d-49e0-9259-73235916a5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:07.227804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642bc89-ff1d-49e0-9259-73235916a5fd admin admin] PATH_INFO: `/v3/projects/7b22c35747b84ed69c900e69faf00341/users/1d74da9f2da54efaa57b33b5ba496388/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:07.229844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9642bc89-ff1d-49e0-9259-73235916a5fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b22c35747b84ed69c900e69faf00341, user_id=1d74da9f2da54efaa57b33b5ba496388, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:07.232518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9642bc89-ff1d-49e0-9259-73235916a5fd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:07.244257 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1556/3042] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:07 2022] PUT /identity/v3/projects/7b22c35747b84ed69c900e69faf00341/users/1d74da9f2da54efaa57b33b5ba496388/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:56:07.255277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320843e-1e0d-47ff-9ddb-dd11faac3b15 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:07.255563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320843e-1e0d-47ff-9ddb-dd11faac3b15 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:07.255838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320843e-1e0d-47ff-9ddb-dd11faac3b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:07.301867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1320843e-1e0d-47ff-9ddb-dd11faac3b15 None None] MFA Rules not processed for user `1d74da9f2da54efaa57b33b5ba496388`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:07.303135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1320843e-1e0d-47ff-9ddb-dd11faac3b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:07.380989 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1487/3043] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:07 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:07.492396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642bc89-ff1d-49e0-9259-73235916a5fd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:07.494718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9642bc89-ff1d-49e0-9259-73235916a5fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:07.597960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642bc89-ff1d-49e0-9259-73235916a5fd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:07.605659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf05e84-8279-4791-8200-334bc82b0947 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:07.608073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf05e84-8279-4791-8200-334bc82b0947 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:07.608073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf05e84-8279-4791-8200-334bc82b0947 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:07.608073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf05e84-8279-4791-8200-334bc82b0947 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:07.668823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:07.683209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf05e84-8279-4791-8200-334bc82b0947 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:07.685098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf05e84-8279-4791-8200-334bc82b0947 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:07.700104 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1557/3044] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:56:09.201092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320843e-1e0d-47ff-9ddb-dd11faac3b15 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:09.202574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1320843e-1e0d-47ff-9ddb-dd11faac3b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:09.222656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320843e-1e0d-47ff-9ddb-dd11faac3b15 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:09.229545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b70877-4722-44fb-a97b-96f02909c8b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:09.230323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b70877-4722-44fb-a97b-96f02909c8b1 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:09.230710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b70877-4722-44fb-a97b-96f02909c8b1 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:09.231145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b70877-4722-44fb-a97b-96f02909c8b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:09.242762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b70877-4722-44fb-a97b-96f02909c8b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:09.244668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b70877-4722-44fb-a97b-96f02909c8b1 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:09.255853 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1488/3045] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:09 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:56:11.701297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8427d8-c294-437c-a045-dc7c9a6cb922 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:11.701831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8427d8-c294-437c-a045-dc7c9a6cb922 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:11.701831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8427d8-c294-437c-a045-dc7c9a6cb922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:11.724924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1b8427d8-c294-437c-a045-dc7c9a6cb922 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:11.734571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1b8427d8-c294-437c-a045-dc7c9a6cb922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:11.838581 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1558/3046] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:56:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:11.917420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b70877-4722-44fb-a97b-96f02909c8b1 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:11.919636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-05b70877-4722-44fb-a97b-96f02909c8b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:11.928904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:12.058109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b70877-4722-44fb-a97b-96f02909c8b1 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:12.069582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db22fc5-b20b-4cca-a115-288b1d19d43b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:12.070943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db22fc5-b20b-4cca-a115-288b1d19d43b service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:12.072139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db22fc5-b20b-4cca-a115-288b1d19d43b service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:12.072635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db22fc5-b20b-4cca-a115-288b1d19d43b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:12.087701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db22fc5-b20b-4cca-a115-288b1d19d43b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:12.090707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db22fc5-b20b-4cca-a115-288b1d19d43b service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:12.110410 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1489/3047] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:56:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 20:56:16.946673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8427d8-c294-437c-a045-dc7c9a6cb922 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:17.115265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8427d8-c294-437c-a045-dc7c9a6cb922 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:17.122389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c96f7-6247-475e-82cc-02047c4cb10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:17.123091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c96f7-6247-475e-82cc-02047c4cb10d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:17.123372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c96f7-6247-475e-82cc-02047c4cb10d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:17.123883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c96f7-6247-475e-82cc-02047c4cb10d admin admin] PATH_INFO: `/v3/users/8a5c96e82db44cd39a218d6ce3575e18` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:17.125129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c96f7-6247-475e-82cc-02047c4cb10d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a5c96e82db44cd39a218d6ce3575e18)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:17.127098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c96f7-6247-475e-82cc-02047c4cb10d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:17.201851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-253c96f7-6247-475e-82cc-02047c4cb10d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a5c96e82db44cd39a218d6ce3575e18'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:17.210139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-253c96f7-6247-475e-82cc-02047c4cb10d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a5c96e82db44cd39a218d6ce3575e18'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:17.229540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-253c96f7-6247-475e-82cc-02047c4cb10d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a5c96e82db44cd39a218d6ce3575e18'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:17.230515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-253c96f7-6247-475e-82cc-02047c4cb10d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a5c96e82db44cd39a218d6ce3575e18'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:17.267121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-253c96f7-6247-475e-82cc-02047c4cb10d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a5c96e82db44cd39a218d6ce3575e18'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:17.277899 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1559/3048] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:56:16 2022] DELETE /identity/v3/users/8a5c96e82db44cd39a218d6ce3575e18 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:17.282715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f169ece-4237-416e-9e9c-84aa456f8c99 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:17.283701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f169ece-4237-416e-9e9c-84aa456f8c99 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:17.284153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f169ece-4237-416e-9e9c-84aa456f8c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:17.312738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2f169ece-4237-416e-9e9c-84aa456f8c99 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:17.314468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2f169ece-4237-416e-9e9c-84aa456f8c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:17.398867 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1490/3049] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:17.408559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c96f7-6247-475e-82cc-02047c4cb10d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:17.410622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-253c96f7-6247-475e-82cc-02047c4cb10d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:17.431092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:17.509207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c96f7-6247-475e-82cc-02047c4cb10d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:17.519596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f13900-ecf9-4627-9b3a-174690d96b7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:17.520814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f13900-ecf9-4627-9b3a-174690d96b7c service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:17.521104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f13900-ecf9-4627-9b3a-174690d96b7c service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:17.521386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f13900-ecf9-4627-9b3a-174690d96b7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:17.549723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f13900-ecf9-4627-9b3a-174690d96b7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:17.555489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f13900-ecf9-4627-9b3a-174690d96b7c service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:17.572553 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1560/3050] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:56:17.906901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f169ece-4237-416e-9e9c-84aa456f8c99 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:17.908452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2f169ece-4237-416e-9e9c-84aa456f8c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:17.929197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f169ece-4237-416e-9e9c-84aa456f8c99 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:17.937053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10110674-6baa-4f00-9495-af60f1423178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:17.937947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10110674-6baa-4f00-9495-af60f1423178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:17.938290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10110674-6baa-4f00-9495-af60f1423178 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:17.938813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10110674-6baa-4f00-9495-af60f1423178 admin admin] PATH_INFO: `/v3/projects/ef52ce9c74be428d92e51a3db71d83b0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:17.940559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10110674-6baa-4f00-9495-af60f1423178 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef52ce9c74be428d92e51a3db71d83b0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:17.943911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10110674-6baa-4f00-9495-af60f1423178 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:18.056597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-10110674-6baa-4f00-9495-af60f1423178 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef52ce9c74be428d92e51a3db71d83b0', 'request_id': 'req-10110674-6baa-4f00-9495-af60f1423178'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:18.071699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-10110674-6baa-4f00-9495-af60f1423178 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef52ce9c74be428d92e51a3db71d83b0', 'request_id': 'req-10110674-6baa-4f00-9495-af60f1423178'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:18.081742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-10110674-6baa-4f00-9495-af60f1423178 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef52ce9c74be428d92e51a3db71d83b0', 'request_id': 'req-10110674-6baa-4f00-9495-af60f1423178'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:18.104898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-10110674-6baa-4f00-9495-af60f1423178 admin admin] The token cache is being invalidate because project ef52ce9c74be428d92e51a3db71d83b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:56:18.106625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-10110674-6baa-4f00-9495-af60f1423178 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:18.107773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-10110674-6baa-4f00-9495-af60f1423178 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:18.111143 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1491/3051] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:56:17 2022] DELETE /identity/v3/projects/ef52ce9c74be428d92e51a3db71d83b0 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:18.115196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f13900-ecf9-4627-9b3a-174690d96b7c service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:18.137141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f13900-ecf9-4627-9b3a-174690d96b7c service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:18.272029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ddbfd-ff5f-44b5-b8ec-fa50778aa0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:18.272689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ddbfd-ff5f-44b5-b8ec-fa50778aa0e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:18.273022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ddbfd-ff5f-44b5-b8ec-fa50778aa0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:18.273300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ddbfd-ff5f-44b5-b8ec-fa50778aa0e3 admin admin] PATH_INFO: `/v3/users/1d74da9f2da54efaa57b33b5ba496388` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:18.274458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ddbfd-ff5f-44b5-b8ec-fa50778aa0e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d74da9f2da54efaa57b33b5ba496388)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:18.277769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ddbfd-ff5f-44b5-b8ec-fa50778aa0e3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:18.389459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9c0ddbfd-ff5f-44b5-b8ec-fa50778aa0e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d74da9f2da54efaa57b33b5ba496388'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:18.398533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9c0ddbfd-ff5f-44b5-b8ec-fa50778aa0e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d74da9f2da54efaa57b33b5ba496388'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:18.425392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9c0ddbfd-ff5f-44b5-b8ec-fa50778aa0e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d74da9f2da54efaa57b33b5ba496388'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:18.426992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9c0ddbfd-ff5f-44b5-b8ec-fa50778aa0e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d74da9f2da54efaa57b33b5ba496388'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:18.448156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9c0ddbfd-ff5f-44b5-b8ec-fa50778aa0e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d74da9f2da54efaa57b33b5ba496388'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:18.459204 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1561/3052] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:56:18 2022] DELETE /identity/v3/users/1d74da9f2da54efaa57b33b5ba496388 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:18.699099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10110674-6baa-4f00-9495-af60f1423178 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:18.701089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-10110674-6baa-4f00-9495-af60f1423178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:18.723466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10110674-6baa-4f00-9495-af60f1423178 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:18.739875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:18.848752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670e17f-b968-41ea-b6d9-22da05e6fe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:18.849897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670e17f-b968-41ea-b6d9-22da05e6fe39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:18.850842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670e17f-b968-41ea-b6d9-22da05e6fe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:18.851726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670e17f-b968-41ea-b6d9-22da05e6fe39 admin admin] PATH_INFO: `/v3/projects/7b22c35747b84ed69c900e69faf00341` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:18.857538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670e17f-b968-41ea-b6d9-22da05e6fe39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b22c35747b84ed69c900e69faf00341)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:18.860008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670e17f-b968-41ea-b6d9-22da05e6fe39 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:18.952160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e670e17f-b968-41ea-b6d9-22da05e6fe39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b22c35747b84ed69c900e69faf00341', 'request_id': 'req-e670e17f-b968-41ea-b6d9-22da05e6fe39'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:18.970873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e670e17f-b968-41ea-b6d9-22da05e6fe39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b22c35747b84ed69c900e69faf00341', 'request_id': 'req-e670e17f-b968-41ea-b6d9-22da05e6fe39'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:18.980427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e670e17f-b968-41ea-b6d9-22da05e6fe39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b22c35747b84ed69c900e69faf00341', 'request_id': 'req-e670e17f-b968-41ea-b6d9-22da05e6fe39'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:19.012884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e670e17f-b968-41ea-b6d9-22da05e6fe39 admin admin] The token cache is being invalidate because project 7b22c35747b84ed69c900e69faf00341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:56:19.013474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e670e17f-b968-41ea-b6d9-22da05e6fe39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:19.014408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e670e17f-b968-41ea-b6d9-22da05e6fe39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:19.018815 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1492/3053] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:56:18 2022] DELETE /identity/v3/projects/7b22c35747b84ed69c900e69faf00341 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:19.023674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d409d1-e9de-43e8-9485-060415bf7da9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:19.023964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d409d1-e9de-43e8-9485-060415bf7da9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:19.024323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d409d1-e9de-43e8-9485-060415bf7da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:19.065490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a8d409d1-e9de-43e8-9485-060415bf7da9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:19.066758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a8d409d1-e9de-43e8-9485-060415bf7da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:19.118061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:19.188056 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1562/3054] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:19.199085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45d5b9-63b5-4f59-8eb8-5e549c46fb16 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:19.199767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45d5b9-63b5-4f59-8eb8-5e549c46fb16 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:19.200375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45d5b9-63b5-4f59-8eb8-5e549c46fb16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:19.239903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5c45d5b9-63b5-4f59-8eb8-5e549c46fb16 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:19.242468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5c45d5b9-63b5-4f59-8eb8-5e549c46fb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:19.252318 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1493/3055] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:19.256449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d409d1-e9de-43e8-9485-060415bf7da9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:19.271242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d409d1-e9de-43e8-9485-060415bf7da9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:19.278421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0ce77-e66c-44b6-b9c3-28baf3a2d066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:19.279364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0ce77-e66c-44b6-b9c3-28baf3a2d066 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:19.280381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0ce77-e66c-44b6-b9c3-28baf3a2d066 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:19.280871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0ce77-e66c-44b6-b9c3-28baf3a2d066 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:19.281767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0ce77-e66c-44b6-b9c3-28baf3a2d066 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:56:19.282389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0ce77-e66c-44b6-b9c3-28baf3a2d066 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:19.284336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0ce77-e66c-44b6-b9c3-28baf3a2d066 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:19.333540 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1563/3056] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:56:19.336680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45d5b9-63b5-4f59-8eb8-5e549c46fb16 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:19.348264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45d5b9-63b5-4f59-8eb8-5e549c46fb16 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:19.371452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e666e0-a34c-41c3-93d1-21fc2a921baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:19.373332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e666e0-a34c-41c3-93d1-21fc2a921baf admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:19.377152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e666e0-a34c-41c3-93d1-21fc2a921baf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:19.377673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e666e0-a34c-41c3-93d1-21fc2a921baf admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:19.380854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e666e0-a34c-41c3-93d1-21fc2a921baf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:19.388261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e666e0-a34c-41c3-93d1-21fc2a921baf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:19.448486 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1494/3057] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:56:19 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:19.456842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0ce77-e66c-44b6-b9c3-28baf3a2d066 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:19.484607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0ce77-e66c-44b6-b9c3-28baf3a2d066 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:19.625844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691f429-c087-47e5-a006-b33838f73cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:19.626784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691f429-c087-47e5-a006-b33838f73cdd admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:19.627849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691f429-c087-47e5-a006-b33838f73cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:19.628310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691f429-c087-47e5-a006-b33838f73cdd admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:19.629116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691f429-c087-47e5-a006-b33838f73cdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:19.633669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691f429-c087-47e5-a006-b33838f73cdd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:19.704421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e666e0-a34c-41c3-93d1-21fc2a921baf admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:19.717007 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1564/3058] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:56:19 2022] POST /identity/v3/users => generated 288 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:19.723119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e666e0-a34c-41c3-93d1-21fc2a921baf admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:19.731322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ad968-2f0e-4929-916c-3ca0a395bdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:19.732269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ad968-2f0e-4929-916c-3ca0a395bdb6 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:19.732819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ad968-2f0e-4929-916c-3ca0a395bdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:19.733357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ad968-2f0e-4929-916c-3ca0a395bdb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:19.734615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ad968-2f0e-4929-916c-3ca0a395bdb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:19.736763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ad968-2f0e-4929-916c-3ca0a395bdb6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:19.759177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691f429-c087-47e5-a006-b33838f73cdd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:19.766035 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1495/3059] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:56:19.770928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691f429-c087-47e5-a006-b33838f73cdd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:19.780471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1c575-f576-4324-b224-20b6647b24fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:19.781506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1c575-f576-4324-b224-20b6647b24fa admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:19.782005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1c575-f576-4324-b224-20b6647b24fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:19.782481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1c575-f576-4324-b224-20b6647b24fa admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:19.783339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a1c575-f576-4324-b224-20b6647b24fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:19.785787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a1c575-f576-4324-b224-20b6647b24fa admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:19.807033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ad968-2f0e-4929-916c-3ca0a395bdb6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:19.814670 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1565/3060] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:56:19.825598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ad968-2f0e-4929-916c-3ca0a395bdb6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:19.839952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22241b89-ec45-408a-a10b-fab18b8b1e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:19.843097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22241b89-ec45-408a-a10b-fab18b8b1e7c admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:19.844689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22241b89-ec45-408a-a10b-fab18b8b1e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:19.846342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22241b89-ec45-408a-a10b-fab18b8b1e7c admin admin] PATH_INFO: `/v3/projects/a6bdd72ba854403e8fef7f651a3ef106/users/123a8a1d0b65407098b5a8e91091b9ca/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:19.886995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22241b89-ec45-408a-a10b-fab18b8b1e7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6bdd72ba854403e8fef7f651a3ef106, user_id=123a8a1d0b65407098b5a8e91091b9ca, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:19.889811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22241b89-ec45-408a-a10b-fab18b8b1e7c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:19.928566 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1496/3061] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:19 2022] PUT /identity/v3/projects/a6bdd72ba854403e8fef7f651a3ef106/users/123a8a1d0b65407098b5a8e91091b9ca/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:56:19.942102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ec545-e418-4880-b9ef-1652c65ca7d7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:19.942102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ec545-e418-4880-b9ef-1652c65ca7d7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:19.942102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ec545-e418-4880-b9ef-1652c65ca7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:20.036407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b98ec545-e418-4880-b9ef-1652c65ca7d7 None None] MFA Rules not processed for user `123a8a1d0b65407098b5a8e91091b9ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:20.037838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b98ec545-e418-4880-b9ef-1652c65ca7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:20.321035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22241b89-ec45-408a-a10b-fab18b8b1e7c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:20.333702 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1566/3062] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:20.445356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:20.511248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22241b89-ec45-408a-a10b-fab18b8b1e7c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:20.523687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f7402-a03d-43cb-bb10-243d18713b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:20.524306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f7402-a03d-43cb-bb10-243d18713b33 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:20.524579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f7402-a03d-43cb-bb10-243d18713b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:20.524844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f7402-a03d-43cb-bb10-243d18713b33 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:20.525472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f7402-a03d-43cb-bb10-243d18713b33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:20.527470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f7402-a03d-43cb-bb10-243d18713b33 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:20.598727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ec545-e418-4880-b9ef-1652c65ca7d7 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:20.615837 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1497/3063] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:56:20 2022] POST /identity/v3/projects => generated 375 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:20.620971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ec545-e418-4880-b9ef-1652c65ca7d7 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:20.709952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:20.798108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5816ec5-b0ba-447b-8912-8c8b81354b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:20.798898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5816ec5-b0ba-447b-8912-8c8b81354b88 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:20.799307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5816ec5-b0ba-447b-8912-8c8b81354b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:20.800393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5816ec5-b0ba-447b-8912-8c8b81354b88 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:20.801266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5816ec5-b0ba-447b-8912-8c8b81354b88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:20.809525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5816ec5-b0ba-447b-8912-8c8b81354b88 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:20.864860 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1567/3064] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:56:20 2022] POST /identity/v3/users => generated 294 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:20.870790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f7402-a03d-43cb-bb10-243d18713b33 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:20.887352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f7402-a03d-43cb-bb10-243d18713b33 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:20.899022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be878887-d756-4a70-99b2-eb23b23fd684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:20.900490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be878887-d756-4a70-99b2-eb23b23fd684 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:20.901117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be878887-d756-4a70-99b2-eb23b23fd684 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:20.901630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be878887-d756-4a70-99b2-eb23b23fd684 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:20.902508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be878887-d756-4a70-99b2-eb23b23fd684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:20.907671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be878887-d756-4a70-99b2-eb23b23fd684 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:20.934921 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1498/3065] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:56:20.940403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5816ec5-b0ba-447b-8912-8c8b81354b88 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:20.957565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5816ec5-b0ba-447b-8912-8c8b81354b88 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:20.984581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786462dc-dbca-44bb-8ee5-6227ef28fa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:20.984876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786462dc-dbca-44bb-8ee5-6227ef28fa97 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:20.985135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786462dc-dbca-44bb-8ee5-6227ef28fa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:20.985326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786462dc-dbca-44bb-8ee5-6227ef28fa97 admin admin] PATH_INFO: `/v3/projects/6e41d13569c94212af2211abe1d8ba99/users/a4be5d569e63435382d1bf274799a23b/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:21.082791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786462dc-dbca-44bb-8ee5-6227ef28fa97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e41d13569c94212af2211abe1d8ba99, user_id=a4be5d569e63435382d1bf274799a23b, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:21.087918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786462dc-dbca-44bb-8ee5-6227ef28fa97 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:21.140199 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1568/3066] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:20 2022] PUT /identity/v3/projects/6e41d13569c94212af2211abe1d8ba99/users/a4be5d569e63435382d1bf274799a23b/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:21.146301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be878887-d756-4a70-99b2-eb23b23fd684 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:21.162058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be878887-d756-4a70-99b2-eb23b23fd684 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:21.249811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ccc42-afdd-4b4f-8d6e-c7378b8b0138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:21.250693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ccc42-afdd-4b4f-8d6e-c7378b8b0138 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:21.251068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ccc42-afdd-4b4f-8d6e-c7378b8b0138 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:21.251452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ccc42-afdd-4b4f-8d6e-c7378b8b0138 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:21.252183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ccc42-afdd-4b4f-8d6e-c7378b8b0138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:21.254106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ccc42-afdd-4b4f-8d6e-c7378b8b0138 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:21.268106 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1499/3067] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:56:21.272162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786462dc-dbca-44bb-8ee5-6227ef28fa97 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:21.284768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786462dc-dbca-44bb-8ee5-6227ef28fa97 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:21.290528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125793d-1c27-4547-ba1c-db8813eef098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:21.291498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125793d-1c27-4547-ba1c-db8813eef098 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:21.291940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125793d-1c27-4547-ba1c-db8813eef098 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:21.292387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125793d-1c27-4547-ba1c-db8813eef098 admin admin] PATH_INFO: `/v3/projects/6e41d13569c94212af2211abe1d8ba99/users/a4be5d569e63435382d1bf274799a23b/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:21.295125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6125793d-1c27-4547-ba1c-db8813eef098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e41d13569c94212af2211abe1d8ba99, user_id=a4be5d569e63435382d1bf274799a23b, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:21.298433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6125793d-1c27-4547-ba1c-db8813eef098 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:21.309938 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1569/3068] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:21 2022] PUT /identity/v3/projects/6e41d13569c94212af2211abe1d8ba99/users/a4be5d569e63435382d1bf274799a23b/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:21.323654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729881fd-dba3-47c2-bda8-d1d526bd053e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:21.323928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729881fd-dba3-47c2-bda8-d1d526bd053e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:21.324270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729881fd-dba3-47c2-bda8-d1d526bd053e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:21.356646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-729881fd-dba3-47c2-bda8-d1d526bd053e None None] MFA Rules not processed for user `a4be5d569e63435382d1bf274799a23b`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:21.357956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-729881fd-dba3-47c2-bda8-d1d526bd053e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:21.464931 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1500/3069] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:21.475003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f124197-34d8-40d6-85c6-156125e94e00 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:21.475585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f124197-34d8-40d6-85c6-156125e94e00 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:21.475722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f124197-34d8-40d6-85c6-156125e94e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:21.503638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0f124197-34d8-40d6-85c6-156125e94e00 None None] MFA Rules not processed for user `123a8a1d0b65407098b5a8e91091b9ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:21.515583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0f124197-34d8-40d6-85c6-156125e94e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:21.683921 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1570/3070] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:21.855456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729881fd-dba3-47c2-bda8-d1d526bd053e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:21.857035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-729881fd-dba3-47c2-bda8-d1d526bd053e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:21.911185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:21.954807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729881fd-dba3-47c2-bda8-d1d526bd053e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:21.961019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71691a-3b88-4da6-980a-0f387f5bfc69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:21.961740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71691a-3b88-4da6-980a-0f387f5bfc69 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:21.962213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71691a-3b88-4da6-980a-0f387f5bfc69 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:21.962698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71691a-3b88-4da6-980a-0f387f5bfc69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:21.971274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71691a-3b88-4da6-980a-0f387f5bfc69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:21.972998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71691a-3b88-4da6-980a-0f387f5bfc69 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:21.983905 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1501/3071] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:56:22.534700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd3854-611d-4503-a664-aaafefa6b2df None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:22.535866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd3854-611d-4503-a664-aaafefa6b2df None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:22.536342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd3854-611d-4503-a664-aaafefa6b2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:22.574202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-09dd3854-611d-4503-a664-aaafefa6b2df None None] MFA Rules not processed for user `123a8a1d0b65407098b5a8e91091b9ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:22.575469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-09dd3854-611d-4503-a664-aaafefa6b2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:22.597592 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1571/3072] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:29.092625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43287ab0-bb13-465d-837b-e84a97406bbe None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:29.093138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43287ab0-bb13-465d-837b-e84a97406bbe None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:29.093303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43287ab0-bb13-465d-837b-e84a97406bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:29.148501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-43287ab0-bb13-465d-837b-e84a97406bbe None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:29.149996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-43287ab0-bb13-465d-837b-e84a97406bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:29.228525 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1502/3073] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 20:56:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:29.241232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd3854-611d-4503-a664-aaafefa6b2df None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:29.242668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-09dd3854-611d-4503-a664-aaafefa6b2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:29.320273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd3854-611d-4503-a664-aaafefa6b2df None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:29.325225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9518ef4-b0f3-4ba1-81d8-0735c6687209 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:29.325858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9518ef4-b0f3-4ba1-81d8-0735c6687209 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:29.326141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9518ef4-b0f3-4ba1-81d8-0735c6687209 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:29.326416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9518ef4-b0f3-4ba1-81d8-0735c6687209 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:29.334376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9518ef4-b0f3-4ba1-81d8-0735c6687209 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:29.336490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9518ef4-b0f3-4ba1-81d8-0735c6687209 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:29.339976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:29.346149 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1572/3074] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:56:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 20:56:29.567088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43287ab0-bb13-465d-837b-e84a97406bbe None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:29.568691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-43287ab0-bb13-465d-837b-e84a97406bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:29.634541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43287ab0-bb13-465d-837b-e84a97406bbe None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:29.638510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85711076-1d8b-41e5-a7b6-11d6a3ab670b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:29.639230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85711076-1d8b-41e5-a7b6-11d6a3ab670b service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:29.639629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85711076-1d8b-41e5-a7b6-11d6a3ab670b service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:29.640062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85711076-1d8b-41e5-a7b6-11d6a3ab670b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:29.641842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-85711076-1d8b-41e5-a7b6-11d6a3ab670b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:29.699766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:29.737308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85711076-1d8b-41e5-a7b6-11d6a3ab670b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:29.742962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85711076-1d8b-41e5-a7b6-11d6a3ab670b service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:29.762427 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1503/3075] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:56:29.874057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480c22d-7aee-492d-a3e7-18a0fa37e95d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:29.874338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480c22d-7aee-492d-a3e7-18a0fa37e95d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:29.875266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480c22d-7aee-492d-a3e7-18a0fa37e95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:29.914028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4480c22d-7aee-492d-a3e7-18a0fa37e95d None None] MFA Rules not processed for user `1d180849a694494e89843c0c315acd6d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:29.915406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4480c22d-7aee-492d-a3e7-18a0fa37e95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:30.019965 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1573/3076] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:33.483649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85711076-1d8b-41e5-a7b6-11d6a3ab670b service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:33.487196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-85711076-1d8b-41e5-a7b6-11d6a3ab670b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:33.664251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85711076-1d8b-41e5-a7b6-11d6a3ab670b service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:33.674339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b51afd-f1b7-4fb6-a3b7-2d9fffae9bbe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:33.675884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b51afd-f1b7-4fb6-a3b7-2d9fffae9bbe service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:33.676308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b51afd-f1b7-4fb6-a3b7-2d9fffae9bbe service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:33.676659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b51afd-f1b7-4fb6-a3b7-2d9fffae9bbe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:33.680567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-06b51afd-f1b7-4fb6-a3b7-2d9fffae9bbe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:33.708620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b51afd-f1b7-4fb6-a3b7-2d9fffae9bbe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:33.710799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b51afd-f1b7-4fb6-a3b7-2d9fffae9bbe service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:33.756540 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1504/3077] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:56:33.994303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480c22d-7aee-492d-a3e7-18a0fa37e95d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:33.995860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4480c22d-7aee-492d-a3e7-18a0fa37e95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:34.027833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480c22d-7aee-492d-a3e7-18a0fa37e95d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:34.046989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8dbf6-6d8b-4d66-9f41-96ca987d9fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:34.047806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8dbf6-6d8b-4d66-9f41-96ca987d9fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:34.048011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8dbf6-6d8b-4d66-9f41-96ca987d9fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:34.048328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8dbf6-6d8b-4d66-9f41-96ca987d9fb9 admin admin] PATH_INFO: `/v3/users/e82ed96a3e794a1b99662cec0969b7ec` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:34.050908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8dbf6-6d8b-4d66-9f41-96ca987d9fb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e82ed96a3e794a1b99662cec0969b7ec)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:34.058680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8dbf6-6d8b-4d66-9f41-96ca987d9fb9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:34.175061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e7e8dbf6-6d8b-4d66-9f41-96ca987d9fb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e82ed96a3e794a1b99662cec0969b7ec'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:34.185220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e7e8dbf6-6d8b-4d66-9f41-96ca987d9fb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e82ed96a3e794a1b99662cec0969b7ec'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:34.202587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e7e8dbf6-6d8b-4d66-9f41-96ca987d9fb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e82ed96a3e794a1b99662cec0969b7ec'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:34.203649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e7e8dbf6-6d8b-4d66-9f41-96ca987d9fb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e82ed96a3e794a1b99662cec0969b7ec'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:34.239960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e7e8dbf6-6d8b-4d66-9f41-96ca987d9fb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e82ed96a3e794a1b99662cec0969b7ec'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:34.285511 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1574/3078] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:56:33 2022] DELETE /identity/v3/users/e82ed96a3e794a1b99662cec0969b7ec => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:34.287188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a3e8f-e545-4cab-9ea0-e230e5063131 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:34.287483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a3e8f-e545-4cab-9ea0-e230e5063131 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:34.287765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a3e8f-e545-4cab-9ea0-e230e5063131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:34.327456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-cf1a3e8f-e545-4cab-9ea0-e230e5063131 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:34.329236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cf1a3e8f-e545-4cab-9ea0-e230e5063131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:34.467671 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1505/3079] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:34.564667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8dbf6-6d8b-4d66-9f41-96ca987d9fb9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:34.567965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e7e8dbf6-6d8b-4d66-9f41-96ca987d9fb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:34.679166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a3e8f-e545-4cab-9ea0-e230e5063131 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:34.681401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cf1a3e8f-e545-4cab-9ea0-e230e5063131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:34.772089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8dbf6-6d8b-4d66-9f41-96ca987d9fb9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:34.786560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1cbd2-d76c-4a03-b956-e0a1c12f9363 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:34.787197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1cbd2-d76c-4a03-b956-e0a1c12f9363 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:34.787556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1cbd2-d76c-4a03-b956-e0a1c12f9363 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:34.789062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1cbd2-d76c-4a03-b956-e0a1c12f9363 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:34.822330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1cbd2-d76c-4a03-b956-e0a1c12f9363 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:34.825723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1cbd2-d76c-4a03-b956-e0a1c12f9363 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:34.864823 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1575/3080] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:56:34.872957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:34.925476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a3e8f-e545-4cab-9ea0-e230e5063131 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:34.950866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4cd1f-a317-4be4-b495-19ac01a1f288 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:34.951514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4cd1f-a317-4be4-b495-19ac01a1f288 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:34.951798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4cd1f-a317-4be4-b495-19ac01a1f288 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:34.952074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4cd1f-a317-4be4-b495-19ac01a1f288 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:34.959718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a9c4cd1f-a317-4be4-b495-19ac01a1f288 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:34.981748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c4cd1f-a317-4be4-b495-19ac01a1f288 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:34.983497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c4cd1f-a317-4be4-b495-19ac01a1f288 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:35.004500 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1506/3081] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:56:35.585829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1cbd2-d76c-4a03-b956-e0a1c12f9363 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:35.592410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-15b1cbd2-d76c-4a03-b956-e0a1c12f9363 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:35.633806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1cbd2-d76c-4a03-b956-e0a1c12f9363 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:35.659818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417b2f0-04f3-4d9a-a7f0-54d8a2f7525e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:35.664505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417b2f0-04f3-4d9a-a7f0-54d8a2f7525e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:35.665185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417b2f0-04f3-4d9a-a7f0-54d8a2f7525e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:35.671287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417b2f0-04f3-4d9a-a7f0-54d8a2f7525e admin admin] PATH_INFO: `/v3/projects/44683518fe3d46c58b139727cf0c9515` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:35.680707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417b2f0-04f3-4d9a-a7f0-54d8a2f7525e admin admin] RBAC: Authorizing `identity:delete_project(project_id=44683518fe3d46c58b139727cf0c9515)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:35.684272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417b2f0-04f3-4d9a-a7f0-54d8a2f7525e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:35.728254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:35.882151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f417b2f0-04f3-4d9a-a7f0-54d8a2f7525e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44683518fe3d46c58b139727cf0c9515', 'request_id': 'req-f417b2f0-04f3-4d9a-a7f0-54d8a2f7525e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:35.911617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f417b2f0-04f3-4d9a-a7f0-54d8a2f7525e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44683518fe3d46c58b139727cf0c9515', 'request_id': 'req-f417b2f0-04f3-4d9a-a7f0-54d8a2f7525e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:35.934476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f417b2f0-04f3-4d9a-a7f0-54d8a2f7525e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44683518fe3d46c58b139727cf0c9515', 'request_id': 'req-f417b2f0-04f3-4d9a-a7f0-54d8a2f7525e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:35.999008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f417b2f0-04f3-4d9a-a7f0-54d8a2f7525e admin admin] The token cache is being invalidate because project 44683518fe3d46c58b139727cf0c9515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:56:36.002178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f417b2f0-04f3-4d9a-a7f0-54d8a2f7525e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:36.003685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f417b2f0-04f3-4d9a-a7f0-54d8a2f7525e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:36.008398 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1576/3082] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:56:35 2022] DELETE /identity/v3/projects/44683518fe3d46c58b139727cf0c9515 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:36.012331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4cd1f-a317-4be4-b495-19ac01a1f288 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:36.061625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4cd1f-a317-4be4-b495-19ac01a1f288 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:36.154943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae75ed8-c6c7-4455-a1b5-f6665b7837fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:36.155996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae75ed8-c6c7-4455-a1b5-f6665b7837fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:36.156407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae75ed8-c6c7-4455-a1b5-f6665b7837fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:36.156953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae75ed8-c6c7-4455-a1b5-f6665b7837fe admin admin] PATH_INFO: `/v3/users/9f0ebdbb11df4a0399f4e63ea0cd1eda` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:36.158374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae75ed8-c6c7-4455-a1b5-f6665b7837fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f0ebdbb11df4a0399f4e63ea0cd1eda)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:36.160449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae75ed8-c6c7-4455-a1b5-f6665b7837fe admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:36.228619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8ae75ed8-c6c7-4455-a1b5-f6665b7837fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f0ebdbb11df4a0399f4e63ea0cd1eda'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:36.229592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8ae75ed8-c6c7-4455-a1b5-f6665b7837fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f0ebdbb11df4a0399f4e63ea0cd1eda'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:36.230231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8ae75ed8-c6c7-4455-a1b5-f6665b7837fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f0ebdbb11df4a0399f4e63ea0cd1eda'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:36.259426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8ae75ed8-c6c7-4455-a1b5-f6665b7837fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f0ebdbb11df4a0399f4e63ea0cd1eda'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:36.276173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8ae75ed8-c6c7-4455-a1b5-f6665b7837fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f0ebdbb11df4a0399f4e63ea0cd1eda'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:36.317389 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1507/3083] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:56:36 2022] DELETE /identity/v3/users/9f0ebdbb11df4a0399f4e63ea0cd1eda => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:37.375788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417b2f0-04f3-4d9a-a7f0-54d8a2f7525e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:37.377646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f417b2f0-04f3-4d9a-a7f0-54d8a2f7525e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:37.405975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417b2f0-04f3-4d9a-a7f0-54d8a2f7525e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:37.584600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f38b78-8571-4ad6-8dff-a08961e9fd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:37.585823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f38b78-8571-4ad6-8dff-a08961e9fd06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:37.586767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f38b78-8571-4ad6-8dff-a08961e9fd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:37.587797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f38b78-8571-4ad6-8dff-a08961e9fd06 admin admin] PATH_INFO: `/v3/projects/9fc6bcd8cd654af7bf1534e88709ef66` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:37.589484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f38b78-8571-4ad6-8dff-a08961e9fd06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fc6bcd8cd654af7bf1534e88709ef66)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:37.592220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f38b78-8571-4ad6-8dff-a08961e9fd06 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:37.676011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-99f38b78-8571-4ad6-8dff-a08961e9fd06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fc6bcd8cd654af7bf1534e88709ef66', 'request_id': 'req-99f38b78-8571-4ad6-8dff-a08961e9fd06'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:37.683726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-99f38b78-8571-4ad6-8dff-a08961e9fd06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fc6bcd8cd654af7bf1534e88709ef66', 'request_id': 'req-99f38b78-8571-4ad6-8dff-a08961e9fd06'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:37.693558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-99f38b78-8571-4ad6-8dff-a08961e9fd06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fc6bcd8cd654af7bf1534e88709ef66', 'request_id': 'req-99f38b78-8571-4ad6-8dff-a08961e9fd06'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:37.726523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-99f38b78-8571-4ad6-8dff-a08961e9fd06 admin admin] The token cache is being invalidate because project 9fc6bcd8cd654af7bf1534e88709ef66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:56:37.726523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-99f38b78-8571-4ad6-8dff-a08961e9fd06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:37.726523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-99f38b78-8571-4ad6-8dff-a08961e9fd06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:37.726523 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1577/3084] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:56:37 2022] DELETE /identity/v3/projects/9fc6bcd8cd654af7bf1534e88709ef66 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:37.727068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221e930-1905-46b6-9b9f-65ae7440c414 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:37.727068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221e930-1905-46b6-9b9f-65ae7440c414 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:37.727068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221e930-1905-46b6-9b9f-65ae7440c414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:37.807886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9221e930-1905-46b6-9b9f-65ae7440c414 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:37.809033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9221e930-1905-46b6-9b9f-65ae7440c414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:37.897028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:37.909894 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1508/3085] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:37.918086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8899c5b-d266-4e7d-8916-c703e4b0fb0b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:37.918380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8899c5b-d266-4e7d-8916-c703e4b0fb0b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:37.918693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8899c5b-d266-4e7d-8916-c703e4b0fb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:37.945679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d8899c5b-d266-4e7d-8916-c703e4b0fb0b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:37.946922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d8899c5b-d266-4e7d-8916-c703e4b0fb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:37.956724 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1578/3086] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:56:37.960521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221e930-1905-46b6-9b9f-65ae7440c414 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:37.970273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221e930-1905-46b6-9b9f-65ae7440c414 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:37.986821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eeee20-fd52-44b4-aa74-6cd6e3c5fffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:37.987520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eeee20-fd52-44b4-aa74-6cd6e3c5fffc admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:37.987871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eeee20-fd52-44b4-aa74-6cd6e3c5fffc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:37.988215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eeee20-fd52-44b4-aa74-6cd6e3c5fffc admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:37.988957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eeee20-fd52-44b4-aa74-6cd6e3c5fffc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:56:37.989375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eeee20-fd52-44b4-aa74-6cd6e3c5fffc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:37.992394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eeee20-fd52-44b4-aa74-6cd6e3c5fffc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:38.026368 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1509/3087] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:56:38.032807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8899c5b-d266-4e7d-8916-c703e4b0fb0b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:38.045274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8899c5b-d266-4e7d-8916-c703e4b0fb0b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:38.054531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a514b1-5a6a-451f-b144-7e65086982e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:38.055163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a514b1-5a6a-451f-b144-7e65086982e1 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:38.055444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a514b1-5a6a-451f-b144-7e65086982e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:38.056439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a514b1-5a6a-451f-b144-7e65086982e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:38.057406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a514b1-5a6a-451f-b144-7e65086982e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:38.064299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a514b1-5a6a-451f-b144-7e65086982e1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:38.101016 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1579/3088] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:56:38 2022] POST /identity/v3/projects => generated 393 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:38.104759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eeee20-fd52-44b4-aa74-6cd6e3c5fffc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:38.115675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eeee20-fd52-44b4-aa74-6cd6e3c5fffc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:38.191691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107324b3-ada0-4d34-a692-c324590e4ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:38.192502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107324b3-ada0-4d34-a692-c324590e4ac3 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:38.192854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107324b3-ada0-4d34-a692-c324590e4ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:38.193186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107324b3-ada0-4d34-a692-c324590e4ac3 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:38.193923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107324b3-ada0-4d34-a692-c324590e4ac3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:38.195998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107324b3-ada0-4d34-a692-c324590e4ac3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:38.225004 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1510/3089] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:56:38 2022] POST /identity/v3/users => generated 297 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:38.229390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a514b1-5a6a-451f-b144-7e65086982e1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:38.235178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:38.243557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a514b1-5a6a-451f-b144-7e65086982e1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:38.253195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b36146-9de4-42d5-8b3e-97ec5525e6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:38.254121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b36146-9de4-42d5-8b3e-97ec5525e6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:38.256324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b36146-9de4-42d5-8b3e-97ec5525e6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:38.256849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b36146-9de4-42d5-8b3e-97ec5525e6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:38.257713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b36146-9de4-42d5-8b3e-97ec5525e6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:38.259638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b36146-9de4-42d5-8b3e-97ec5525e6d5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:38.271426 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1580/3090] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:56:38.275928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107324b3-ada0-4d34-a692-c324590e4ac3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:38.286340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107324b3-ada0-4d34-a692-c324590e4ac3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:38.292235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99127f9-4350-4f13-adeb-074024526ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:38.292865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99127f9-4350-4f13-adeb-074024526ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:38.293167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99127f9-4350-4f13-adeb-074024526ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:38.293435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99127f9-4350-4f13-adeb-074024526ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:38.294175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99127f9-4350-4f13-adeb-074024526ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:38.295889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99127f9-4350-4f13-adeb-074024526ce9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:38.311490 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1511/3091] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:56:38.312106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b36146-9de4-42d5-8b3e-97ec5525e6d5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:38.323366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b36146-9de4-42d5-8b3e-97ec5525e6d5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:38.328740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a48c97-6338-4776-aa7d-5e4218a724d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:38.329429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a48c97-6338-4776-aa7d-5e4218a724d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:38.329705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a48c97-6338-4776-aa7d-5e4218a724d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:38.329974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a48c97-6338-4776-aa7d-5e4218a724d8 admin admin] PATH_INFO: `/v3/projects/8a666f410e6146daa5b86d414a196cd8/users/7e48c730d06a4cc8b1c55c1290d02437/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:38.345846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a48c97-6338-4776-aa7d-5e4218a724d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a666f410e6146daa5b86d414a196cd8, user_id=7e48c730d06a4cc8b1c55c1290d02437, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:38.348556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a48c97-6338-4776-aa7d-5e4218a724d8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:38.362805 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1581/3092] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:38 2022] PUT /identity/v3/projects/8a666f410e6146daa5b86d414a196cd8/users/7e48c730d06a4cc8b1c55c1290d02437/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:38.373054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ae482-c777-4b18-8f67-37086ed151d9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:38.373409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ae482-c777-4b18-8f67-37086ed151d9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:38.373766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ae482-c777-4b18-8f67-37086ed151d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:38.401878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6f1ae482-c777-4b18-8f67-37086ed151d9 None None] MFA Rules not processed for user `7e48c730d06a4cc8b1c55c1290d02437`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:38.403342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6f1ae482-c777-4b18-8f67-37086ed151d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:38.469906 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1512/3093] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:38.474446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a48c97-6338-4776-aa7d-5e4218a724d8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:38.539411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a48c97-6338-4776-aa7d-5e4218a724d8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:38.544395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042d50f-6e02-4003-ac64-35a74b508efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:38.545252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042d50f-6e02-4003-ac64-35a74b508efe admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:38.546163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042d50f-6e02-4003-ac64-35a74b508efe admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:38.546163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042d50f-6e02-4003-ac64-35a74b508efe admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:38.546637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042d50f-6e02-4003-ac64-35a74b508efe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:38.548649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042d50f-6e02-4003-ac64-35a74b508efe admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:38.577262 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1582/3094] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:56:38 2022] POST /identity/v3/projects => generated 393 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:56:38.581636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ae482-c777-4b18-8f67-37086ed151d9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:38.595438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ae482-c777-4b18-8f67-37086ed151d9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:38.653403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:38.663840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e48a8-65eb-45be-a1b9-0aba7772105a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:38.664515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e48a8-65eb-45be-a1b9-0aba7772105a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:38.665327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e48a8-65eb-45be-a1b9-0aba7772105a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:38.666036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e48a8-65eb-45be-a1b9-0aba7772105a admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:38.666803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e48a8-65eb-45be-a1b9-0aba7772105a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:38.668922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e48a8-65eb-45be-a1b9-0aba7772105a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:38.700711 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1513/3095] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:56:38 2022] POST /identity/v3/users => generated 303 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:56:38.704060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042d50f-6e02-4003-ac64-35a74b508efe admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:38.717401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042d50f-6e02-4003-ac64-35a74b508efe admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:38.722725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21bd3b-ac99-43ed-945c-9c1a31cf8f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:38.723337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21bd3b-ac99-43ed-945c-9c1a31cf8f33 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:38.723620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21bd3b-ac99-43ed-945c-9c1a31cf8f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:38.723893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21bd3b-ac99-43ed-945c-9c1a31cf8f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:38.724663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f21bd3b-ac99-43ed-945c-9c1a31cf8f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:38.726187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f21bd3b-ac99-43ed-945c-9c1a31cf8f33 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:38.737687 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1583/3096] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:56:38.741875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e48a8-65eb-45be-a1b9-0aba7772105a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:38.757389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e48a8-65eb-45be-a1b9-0aba7772105a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:38.762564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4fb4c-2329-4c32-8b68-aa2525f09493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:38.763328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4fb4c-2329-4c32-8b68-aa2525f09493 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:38.763713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4fb4c-2329-4c32-8b68-aa2525f09493 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:38.764088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4fb4c-2329-4c32-8b68-aa2525f09493 admin admin] PATH_INFO: `/v3/projects/4ff6d6b9ced94d54819a76df40458e90/users/2b0eeb50021a4c84b2bf98e3bc719f70/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:38.796725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4fb4c-2329-4c32-8b68-aa2525f09493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ff6d6b9ced94d54819a76df40458e90, user_id=2b0eeb50021a4c84b2bf98e3bc719f70, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:38.800808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4fb4c-2329-4c32-8b68-aa2525f09493 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:38.827971 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1514/3097] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:38 2022] PUT /identity/v3/projects/4ff6d6b9ced94d54819a76df40458e90/users/2b0eeb50021a4c84b2bf98e3bc719f70/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:38.831789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21bd3b-ac99-43ed-945c-9c1a31cf8f33 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:38.843040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21bd3b-ac99-43ed-945c-9c1a31cf8f33 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:38.929981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c495104-59f6-4d62-9ed8-bbe4ec9d294e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:38.930707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c495104-59f6-4d62-9ed8-bbe4ec9d294e admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:38.930988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c495104-59f6-4d62-9ed8-bbe4ec9d294e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:38.931262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c495104-59f6-4d62-9ed8-bbe4ec9d294e admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:38.931908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c495104-59f6-4d62-9ed8-bbe4ec9d294e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:38.933529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c495104-59f6-4d62-9ed8-bbe4ec9d294e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:38.944779 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1584/3098] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:56:38.948944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4fb4c-2329-4c32-8b68-aa2525f09493 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:38.959338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4fb4c-2329-4c32-8b68-aa2525f09493 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:38.964200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671654d2-8833-428c-87b4-098128329028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:38.964836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671654d2-8833-428c-87b4-098128329028 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:38.965135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671654d2-8833-428c-87b4-098128329028 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:38.965412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671654d2-8833-428c-87b4-098128329028 admin admin] PATH_INFO: `/v3/projects/4ff6d6b9ced94d54819a76df40458e90/users/2b0eeb50021a4c84b2bf98e3bc719f70/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:38.967379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671654d2-8833-428c-87b4-098128329028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ff6d6b9ced94d54819a76df40458e90, user_id=2b0eeb50021a4c84b2bf98e3bc719f70, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:38.970008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671654d2-8833-428c-87b4-098128329028 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:38.981169 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1515/3099] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:38 2022] PUT /identity/v3/projects/4ff6d6b9ced94d54819a76df40458e90/users/2b0eeb50021a4c84b2bf98e3bc719f70/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:38.992798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75cbcb-d730-4279-8fdc-f9f024d279f0 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:38.993085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75cbcb-d730-4279-8fdc-f9f024d279f0 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:38.993356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75cbcb-d730-4279-8fdc-f9f024d279f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:39.023205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4a75cbcb-d730-4279-8fdc-f9f024d279f0 None None] MFA Rules not processed for user `2b0eeb50021a4c84b2bf98e3bc719f70`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:39.031299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4a75cbcb-d730-4279-8fdc-f9f024d279f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:39.057662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:39.104122 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1585/3100] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:38 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:39.263056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671654d2-8833-428c-87b4-098128329028 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:39.264016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-671654d2-8833-428c-87b4-098128329028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:39.367771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671654d2-8833-428c-87b4-098128329028 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:39.381251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617a7cc-977a-4adf-a45b-83ca7ce98972 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:39.382418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617a7cc-977a-4adf-a45b-83ca7ce98972 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:39.382951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617a7cc-977a-4adf-a45b-83ca7ce98972 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:39.383467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617a7cc-977a-4adf-a45b-83ca7ce98972 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:39.396484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617a7cc-977a-4adf-a45b-83ca7ce98972 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:39.398653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617a7cc-977a-4adf-a45b-83ca7ce98972 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:39.421802 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1516/3101] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:39 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:56:40.925514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75cbcb-d730-4279-8fdc-f9f024d279f0 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:40.939056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75cbcb-d730-4279-8fdc-f9f024d279f0 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:40.945388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39317d8-784a-496c-b16d-8177594e67fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:40.945997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39317d8-784a-496c-b16d-8177594e67fd service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:40.946284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39317d8-784a-496c-b16d-8177594e67fd service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:40.946591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39317d8-784a-496c-b16d-8177594e67fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:41.005396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39317d8-784a-496c-b16d-8177594e67fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:41.007840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39317d8-784a-496c-b16d-8177594e67fd service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:41.017315 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1586/3102] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:40 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:56:41.036662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617a7cc-977a-4adf-a45b-83ca7ce98972 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:41.047053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39317d8-784a-496c-b16d-8177594e67fd service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:41.059465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39317d8-784a-496c-b16d-8177594e67fd service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:41.066629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc1a34-5007-47a3-b85b-6579dfe3adcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:41.067363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc1a34-5007-47a3-b85b-6579dfe3adcb service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:41.067816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc1a34-5007-47a3-b85b-6579dfe3adcb service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:41.068100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc1a34-5007-47a3-b85b-6579dfe3adcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:41.071416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-abbc1a34-5007-47a3-b85b-6579dfe3adcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:41.139429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617a7cc-977a-4adf-a45b-83ca7ce98972 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:41.145177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275df03-d06e-4223-acde-3bb8053cc278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:41.145883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275df03-d06e-4223-acde-3bb8053cc278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:41.146250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275df03-d06e-4223-acde-3bb8053cc278 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:41.146634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275df03-d06e-4223-acde-3bb8053cc278 admin admin] PATH_INFO: `/v3/users/7e48c730d06a4cc8b1c55c1290d02437` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:41.147937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275df03-d06e-4223-acde-3bb8053cc278 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e48c730d06a4cc8b1c55c1290d02437)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:41.150335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275df03-d06e-4223-acde-3bb8053cc278 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:41.166637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc1a34-5007-47a3-b85b-6579dfe3adcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:41.182637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc1a34-5007-47a3-b85b-6579dfe3adcb service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:41.207390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:41.212987 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1587/3103] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:41 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:56:41.235112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9275df03-d06e-4223-acde-3bb8053cc278 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e48c730d06a4cc8b1c55c1290d02437'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:41.235872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9275df03-d06e-4223-acde-3bb8053cc278 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e48c730d06a4cc8b1c55c1290d02437'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:41.236718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9275df03-d06e-4223-acde-3bb8053cc278 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e48c730d06a4cc8b1c55c1290d02437'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:41.258627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9275df03-d06e-4223-acde-3bb8053cc278 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e48c730d06a4cc8b1c55c1290d02437'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:41.296552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9275df03-d06e-4223-acde-3bb8053cc278 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e48c730d06a4cc8b1c55c1290d02437'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:41.314222 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1517/3104] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:56:41 2022] DELETE /identity/v3/users/7e48c730d06a4cc8b1c55c1290d02437 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:41.315778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2558778-588f-4d38-b7dc-149bb732d5d4 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:41.316067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2558778-588f-4d38-b7dc-149bb732d5d4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:41.316360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2558778-588f-4d38-b7dc-149bb732d5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:41.348061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a2558778-588f-4d38-b7dc-149bb732d5d4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:41.349317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a2558778-588f-4d38-b7dc-149bb732d5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:41.437201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:41.469585 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1588/3105] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:56:41.503913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275df03-d06e-4223-acde-3bb8053cc278 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:41.505506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9275df03-d06e-4223-acde-3bb8053cc278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:41.674296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275df03-d06e-4223-acde-3bb8053cc278 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:41.684024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9d4fd-6ee8-450d-86f1-c50963315149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:41.684651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9d4fd-6ee8-450d-86f1-c50963315149 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:41.688517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9d4fd-6ee8-450d-86f1-c50963315149 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:41.688813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9d4fd-6ee8-450d-86f1-c50963315149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:41.711947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff9d4fd-6ee8-450d-86f1-c50963315149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:41.716268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff9d4fd-6ee8-450d-86f1-c50963315149 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:41.748828 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1518/3106] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:56:42.444013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2558778-588f-4d38-b7dc-149bb732d5d4 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:42.445504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a2558778-588f-4d38-b7dc-149bb732d5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:42.527666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2558778-588f-4d38-b7dc-149bb732d5d4 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:42.557302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e601a6b-da69-4191-959d-fd2420d6de6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:42.571392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e601a6b-da69-4191-959d-fd2420d6de6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:42.573562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e601a6b-da69-4191-959d-fd2420d6de6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:42.574025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e601a6b-da69-4191-959d-fd2420d6de6f admin admin] PATH_INFO: `/v3/projects/8a666f410e6146daa5b86d414a196cd8` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:42.579198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e601a6b-da69-4191-959d-fd2420d6de6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a666f410e6146daa5b86d414a196cd8)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:42.581349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e601a6b-da69-4191-959d-fd2420d6de6f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:42.744317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3e601a6b-da69-4191-959d-fd2420d6de6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a666f410e6146daa5b86d414a196cd8', 'request_id': 'req-3e601a6b-da69-4191-959d-fd2420d6de6f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:42.751535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3e601a6b-da69-4191-959d-fd2420d6de6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a666f410e6146daa5b86d414a196cd8', 'request_id': 'req-3e601a6b-da69-4191-959d-fd2420d6de6f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:42.759510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3e601a6b-da69-4191-959d-fd2420d6de6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a666f410e6146daa5b86d414a196cd8', 'request_id': 'req-3e601a6b-da69-4191-959d-fd2420d6de6f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:42.775978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3e601a6b-da69-4191-959d-fd2420d6de6f admin admin] The token cache is being invalidate because project 8a666f410e6146daa5b86d414a196cd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:56:42.776594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3e601a6b-da69-4191-959d-fd2420d6de6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:42.777646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3e601a6b-da69-4191-959d-fd2420d6de6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:42.782743 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1589/3107] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:56:42 2022] DELETE /identity/v3/projects/8a666f410e6146daa5b86d414a196cd8 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:42.786409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9d4fd-6ee8-450d-86f1-c50963315149 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:42.799061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9d4fd-6ee8-450d-86f1-c50963315149 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:42.889396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63370deb-fcfb-4a31-bc3c-16938277b890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:42.890319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63370deb-fcfb-4a31-bc3c-16938277b890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:42.891452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63370deb-fcfb-4a31-bc3c-16938277b890 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:42.891934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63370deb-fcfb-4a31-bc3c-16938277b890 admin admin] PATH_INFO: `/v3/users/2b0eeb50021a4c84b2bf98e3bc719f70` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:42.893940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63370deb-fcfb-4a31-bc3c-16938277b890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b0eeb50021a4c84b2bf98e3bc719f70)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:42.897847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63370deb-fcfb-4a31-bc3c-16938277b890 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:43.011280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-63370deb-fcfb-4a31-bc3c-16938277b890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b0eeb50021a4c84b2bf98e3bc719f70'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:43.012293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-63370deb-fcfb-4a31-bc3c-16938277b890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b0eeb50021a4c84b2bf98e3bc719f70'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:43.013355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-63370deb-fcfb-4a31-bc3c-16938277b890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b0eeb50021a4c84b2bf98e3bc719f70'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:43.055733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-63370deb-fcfb-4a31-bc3c-16938277b890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b0eeb50021a4c84b2bf98e3bc719f70'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:43.059144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:43.083466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-63370deb-fcfb-4a31-bc3c-16938277b890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b0eeb50021a4c84b2bf98e3bc719f70'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:43.105863 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1519/3108] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:56:42 2022] DELETE /identity/v3/users/2b0eeb50021a4c84b2bf98e3bc719f70 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:43.734232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0f3bd-679b-433b-9b06-c480bb4e9767 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:43.734848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0f3bd-679b-433b-9b06-c480bb4e9767 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:43.734848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0f3bd-679b-433b-9b06-c480bb4e9767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:43.772406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-89a0f3bd-679b-433b-9b06-c480bb4e9767 None None] MFA Rules not processed for user `123a8a1d0b65407098b5a8e91091b9ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:43.775743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-89a0f3bd-679b-433b-9b06-c480bb4e9767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:43.914467 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1590/3109] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:44.067189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63370deb-fcfb-4a31-bc3c-16938277b890 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:44.069674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-63370deb-fcfb-4a31-bc3c-16938277b890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:44.176418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63370deb-fcfb-4a31-bc3c-16938277b890 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:44.182144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164cf5f-2aef-431d-8b43-5f34f90317e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:44.185194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164cf5f-2aef-431d-8b43-5f34f90317e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:44.185405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164cf5f-2aef-431d-8b43-5f34f90317e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:44.187464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164cf5f-2aef-431d-8b43-5f34f90317e2 admin admin] PATH_INFO: `/v3/projects/4ff6d6b9ced94d54819a76df40458e90` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:44.193391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164cf5f-2aef-431d-8b43-5f34f90317e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ff6d6b9ced94d54819a76df40458e90)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:44.193391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164cf5f-2aef-431d-8b43-5f34f90317e2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:44.261454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9164cf5f-2aef-431d-8b43-5f34f90317e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ff6d6b9ced94d54819a76df40458e90', 'request_id': 'req-9164cf5f-2aef-431d-8b43-5f34f90317e2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:44.269543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9164cf5f-2aef-431d-8b43-5f34f90317e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ff6d6b9ced94d54819a76df40458e90', 'request_id': 'req-9164cf5f-2aef-431d-8b43-5f34f90317e2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:44.277369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9164cf5f-2aef-431d-8b43-5f34f90317e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ff6d6b9ced94d54819a76df40458e90', 'request_id': 'req-9164cf5f-2aef-431d-8b43-5f34f90317e2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:44.290127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9164cf5f-2aef-431d-8b43-5f34f90317e2 admin admin] The token cache is being invalidate because project 4ff6d6b9ced94d54819a76df40458e90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:56:44.290433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9164cf5f-2aef-431d-8b43-5f34f90317e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:44.291406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9164cf5f-2aef-431d-8b43-5f34f90317e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:44.293272 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1520/3110] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:56:44 2022] DELETE /identity/v3/projects/4ff6d6b9ced94d54819a76df40458e90 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:44.302406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e8b65-7792-4ffe-886a-cd49e7dd79ec None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:44.302733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e8b65-7792-4ffe-886a-cd49e7dd79ec None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:44.303543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e8b65-7792-4ffe-886a-cd49e7dd79ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:44.334814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-572e8b65-7792-4ffe-886a-cd49e7dd79ec None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:44.336217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-572e8b65-7792-4ffe-886a-cd49e7dd79ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:44.421438 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1591/3111] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:44.422179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164cf5f-2aef-431d-8b43-5f34f90317e2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:44.422280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db156a9-a185-4a58-84a3-c25ce44ca79c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:44.422280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db156a9-a185-4a58-84a3-c25ce44ca79c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:44.422469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9164cf5f-2aef-431d-8b43-5f34f90317e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:44.422587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db156a9-a185-4a58-84a3-c25ce44ca79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:44.442558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164cf5f-2aef-431d-8b43-5f34f90317e2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:44.450900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1db156a9-a185-4a58-84a3-c25ce44ca79c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:44.451803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7f938-33cb-40cf-a18f-ff9d17654802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:44.453350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1db156a9-a185-4a58-84a3-c25ce44ca79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:44.454876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7f938-33cb-40cf-a18f-ff9d17654802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:44.455243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7f938-33cb-40cf-a18f-ff9d17654802 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:44.455509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7f938-33cb-40cf-a18f-ff9d17654802 admin admin] PATH_INFO: `/v3/users/a749a74841e840e2b8de67fb9562bf51` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:44.456778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7f938-33cb-40cf-a18f-ff9d17654802 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a749a74841e840e2b8de67fb9562bf51)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:44.458727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7f938-33cb-40cf-a18f-ff9d17654802 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:44.467520 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1592/3112] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:56:44.471696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db156a9-a185-4a58-84a3-c25ce44ca79c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:44.483460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db156a9-a185-4a58-84a3-c25ce44ca79c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:44.491538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73780cb-21b8-42b8-8954-ed239a737c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:44.492164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73780cb-21b8-42b8-8954-ed239a737c92 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:44.492465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73780cb-21b8-42b8-8954-ed239a737c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:44.492704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73780cb-21b8-42b8-8954-ed239a737c92 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:44.493358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73780cb-21b8-42b8-8954-ed239a737c92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:56:44.493664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73780cb-21b8-42b8-8954-ed239a737c92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:44.495396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73780cb-21b8-42b8-8954-ed239a737c92 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:44.514452 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1593/3113] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:56:44.516063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73780cb-21b8-42b8-8954-ed239a737c92 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:44.522606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-41e7f938-33cb-40cf-a18f-ff9d17654802 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a749a74841e840e2b8de67fb9562bf51'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:44.523628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-41e7f938-33cb-40cf-a18f-ff9d17654802 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a749a74841e840e2b8de67fb9562bf51'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:44.524407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-41e7f938-33cb-40cf-a18f-ff9d17654802 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a749a74841e840e2b8de67fb9562bf51'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:44.525441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:44.531861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73780cb-21b8-42b8-8954-ed239a737c92 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:44.550522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-41e7f938-33cb-40cf-a18f-ff9d17654802 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a749a74841e840e2b8de67fb9562bf51'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:44.550522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:44.553605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec16c39-7183-487b-a290-e40b7f62f09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:44.556914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec16c39-7183-487b-a290-e40b7f62f09a admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:44.556914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec16c39-7183-487b-a290-e40b7f62f09a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:44.556914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec16c39-7183-487b-a290-e40b7f62f09a admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:44.556914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec16c39-7183-487b-a290-e40b7f62f09a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:44.560283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec16c39-7183-487b-a290-e40b7f62f09a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:44.588212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-41e7f938-33cb-40cf-a18f-ff9d17654802 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a749a74841e840e2b8de67fb9562bf51'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:44.627483 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1521/3114] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:56:44 2022] DELETE /identity/v3/users/a749a74841e840e2b8de67fb9562bf51 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:44.630218 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1594/3115] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:56:44 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:44.635049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7f938-33cb-40cf-a18f-ff9d17654802 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:44.638162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-41e7f938-33cb-40cf-a18f-ff9d17654802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:44.778327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7f938-33cb-40cf-a18f-ff9d17654802 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:44.886147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda63ae3-feba-4365-b1a4-987be3512a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:44.888373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda63ae3-feba-4365-b1a4-987be3512a03 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:44.889603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda63ae3-feba-4365-b1a4-987be3512a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:44.889794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda63ae3-feba-4365-b1a4-987be3512a03 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:44.890592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda63ae3-feba-4365-b1a4-987be3512a03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:44.893772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda63ae3-feba-4365-b1a4-987be3512a03 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:44.932264 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1522/3116] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:56:44 2022] POST /identity/v3/users => generated 292 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:44.935832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec16c39-7183-487b-a290-e40b7f62f09a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:44.948751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec16c39-7183-487b-a290-e40b7f62f09a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:44.953560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b7257-8d96-432a-b81b-ae530df187c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:44.954172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b7257-8d96-432a-b81b-ae530df187c5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:44.954450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b7257-8d96-432a-b81b-ae530df187c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:44.954780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b7257-8d96-432a-b81b-ae530df187c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:44.955483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b7257-8d96-432a-b81b-ae530df187c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:44.956970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b7257-8d96-432a-b81b-ae530df187c5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:44.967865 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1595/3117] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:56:44.972356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda63ae3-feba-4365-b1a4-987be3512a03 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:44.987707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda63ae3-feba-4365-b1a4-987be3512a03 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:44.994687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58e1a3-a2e0-47ea-ac9c-5256fab5228c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:44.995764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58e1a3-a2e0-47ea-ac9c-5256fab5228c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:44.996202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58e1a3-a2e0-47ea-ac9c-5256fab5228c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:44.996536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58e1a3-a2e0-47ea-ac9c-5256fab5228c admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:44.997797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58e1a3-a2e0-47ea-ac9c-5256fab5228c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:44.999681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58e1a3-a2e0-47ea-ac9c-5256fab5228c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:45.021508 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1523/3118] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:56:45.023331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b7257-8d96-432a-b81b-ae530df187c5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:45.047233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b7257-8d96-432a-b81b-ae530df187c5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:45.055433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338ab59-bae1-4259-b2f7-a6a52ccaa588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:45.056183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338ab59-bae1-4259-b2f7-a6a52ccaa588 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:45.056533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338ab59-bae1-4259-b2f7-a6a52ccaa588 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:45.056879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338ab59-bae1-4259-b2f7-a6a52ccaa588 admin admin] PATH_INFO: `/v3/projects/90234df69d5e4546983a5d3f14143a7e/users/1f9c07411c404bc3bc680e597f6c2e00/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:45.076424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338ab59-bae1-4259-b2f7-a6a52ccaa588 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90234df69d5e4546983a5d3f14143a7e, user_id=1f9c07411c404bc3bc680e597f6c2e00, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:45.079770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338ab59-bae1-4259-b2f7-a6a52ccaa588 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:45.098733 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1596/3119] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:45 2022] PUT /identity/v3/projects/90234df69d5e4546983a5d3f14143a7e/users/1f9c07411c404bc3bc680e597f6c2e00/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:45.109491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c2233-7e69-45a3-8f29-2276f7d961a6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:45.109991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c2233-7e69-45a3-8f29-2276f7d961a6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:45.110253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c2233-7e69-45a3-8f29-2276f7d961a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:45.197372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ed8c2233-7e69-45a3-8f29-2276f7d961a6 None None] MFA Rules not processed for user `1f9c07411c404bc3bc680e597f6c2e00`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:45.198625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ed8c2233-7e69-45a3-8f29-2276f7d961a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:45.286602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338ab59-bae1-4259-b2f7-a6a52ccaa588 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:45.290101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a338ab59-bae1-4259-b2f7-a6a52ccaa588 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:45.311873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:45.355000 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1524/3120] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:45.371469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c2233-7e69-45a3-8f29-2276f7d961a6 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:45.375595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ed8c2233-7e69-45a3-8f29-2276f7d961a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:45.442301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338ab59-bae1-4259-b2f7-a6a52ccaa588 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:45.451074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02367c3-88bd-436f-9169-36381aaf5d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:45.451713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02367c3-88bd-436f-9169-36381aaf5d34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:45.451990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02367c3-88bd-436f-9169-36381aaf5d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:45.452262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02367c3-88bd-436f-9169-36381aaf5d34 admin admin] PATH_INFO: `/v3/projects/237c6bad691343e7a7e64304f55e01e9` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:45.460556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02367c3-88bd-436f-9169-36381aaf5d34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=237c6bad691343e7a7e64304f55e01e9)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:45.463758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02367c3-88bd-436f-9169-36381aaf5d34 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:45.479892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c2233-7e69-45a3-8f29-2276f7d961a6 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:45.493572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7138a26-a567-4af3-a695-cfa1673b3aa5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:45.494202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7138a26-a567-4af3-a695-cfa1673b3aa5 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:45.494482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7138a26-a567-4af3-a695-cfa1673b3aa5 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:45.494777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7138a26-a567-4af3-a695-cfa1673b3aa5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:45.515189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7138a26-a567-4af3-a695-cfa1673b3aa5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:45.518800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7138a26-a567-4af3-a695-cfa1673b3aa5 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:45.543123 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1525/3121] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:56:45.554751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f02367c3-88bd-436f-9169-36381aaf5d34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '237c6bad691343e7a7e64304f55e01e9', 'request_id': 'req-f02367c3-88bd-436f-9169-36381aaf5d34'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:45.563697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f02367c3-88bd-436f-9169-36381aaf5d34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '237c6bad691343e7a7e64304f55e01e9', 'request_id': 'req-f02367c3-88bd-436f-9169-36381aaf5d34'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:45.570217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f02367c3-88bd-436f-9169-36381aaf5d34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '237c6bad691343e7a7e64304f55e01e9', 'request_id': 'req-f02367c3-88bd-436f-9169-36381aaf5d34'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:45.587370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f02367c3-88bd-436f-9169-36381aaf5d34 admin admin] The token cache is being invalidate because project 237c6bad691343e7a7e64304f55e01e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:56:45.587670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f02367c3-88bd-436f-9169-36381aaf5d34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:45.588369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f02367c3-88bd-436f-9169-36381aaf5d34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:45.590347 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1597/3122] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:56:45 2022] DELETE /identity/v3/projects/237c6bad691343e7a7e64304f55e01e9 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:56:45.594345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7138a26-a567-4af3-a695-cfa1673b3aa5 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:45.607522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7138a26-a567-4af3-a695-cfa1673b3aa5 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:45.653996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02367c3-88bd-436f-9169-36381aaf5d34 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:45.655800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f02367c3-88bd-436f-9169-36381aaf5d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:45.722222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95845da-911b-4408-9a97-bd28bbab6886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:45.722684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95845da-911b-4408-9a97-bd28bbab6886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:45.722841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95845da-911b-4408-9a97-bd28bbab6886 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:45.723108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95845da-911b-4408-9a97-bd28bbab6886 admin admin] PATH_INFO: `/v3/users/511c9feaf07741cc906e2465214edbf4` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:45.725521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95845da-911b-4408-9a97-bd28bbab6886 admin admin] RBAC: Authorizing `identity:delete_user(user_id=511c9feaf07741cc906e2465214edbf4)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:45.728600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95845da-911b-4408-9a97-bd28bbab6886 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:45.749576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:45.808581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02367c3-88bd-436f-9169-36381aaf5d34 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:45.819570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f137af6-45b7-4a20-9edc-c036ba9a3f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:45.820735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f137af6-45b7-4a20-9edc-c036ba9a3f77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:45.821902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f137af6-45b7-4a20-9edc-c036ba9a3f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:45.822620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f137af6-45b7-4a20-9edc-c036ba9a3f77 admin admin] PATH_INFO: `/v3/users/725f68beba264c2eafc951f8300eba32` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:45.823972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f137af6-45b7-4a20-9edc-c036ba9a3f77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=725f68beba264c2eafc951f8300eba32)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:45.826191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f137af6-45b7-4a20-9edc-c036ba9a3f77 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:45.845076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a95845da-911b-4408-9a97-bd28bbab6886 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '511c9feaf07741cc906e2465214edbf4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:45.845850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a95845da-911b-4408-9a97-bd28bbab6886 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '511c9feaf07741cc906e2465214edbf4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:45.846876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a95845da-911b-4408-9a97-bd28bbab6886 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '511c9feaf07741cc906e2465214edbf4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:45.864811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a95845da-911b-4408-9a97-bd28bbab6886 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '511c9feaf07741cc906e2465214edbf4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:45.898010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a95845da-911b-4408-9a97-bd28bbab6886 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '511c9feaf07741cc906e2465214edbf4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:45.905530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:45.921261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3f137af6-45b7-4a20-9edc-c036ba9a3f77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '725f68beba264c2eafc951f8300eba32'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:45.930846 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1526/3123] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:56:45 2022] DELETE /identity/v3/users/511c9feaf07741cc906e2465214edbf4 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:45.941936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3f137af6-45b7-4a20-9edc-c036ba9a3f77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '725f68beba264c2eafc951f8300eba32'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:46.005015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3f137af6-45b7-4a20-9edc-c036ba9a3f77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '725f68beba264c2eafc951f8300eba32'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:46.009447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3f137af6-45b7-4a20-9edc-c036ba9a3f77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '725f68beba264c2eafc951f8300eba32'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:46.032856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3f137af6-45b7-4a20-9edc-c036ba9a3f77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '725f68beba264c2eafc951f8300eba32'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:46.043754 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1598/3124] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:56:45 2022] DELETE /identity/v3/users/725f68beba264c2eafc951f8300eba32 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:46.209781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95845da-911b-4408-9a97-bd28bbab6886 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:46.216164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a95845da-911b-4408-9a97-bd28bbab6886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:46.322898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f137af6-45b7-4a20-9edc-c036ba9a3f77 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:46.324587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3f137af6-45b7-4a20-9edc-c036ba9a3f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:46.385486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95845da-911b-4408-9a97-bd28bbab6886 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:46.494738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f137af6-45b7-4a20-9edc-c036ba9a3f77 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:46.509695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f07582-fe02-4906-8336-0702b6690d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:46.510480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f07582-fe02-4906-8336-0702b6690d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:46.510797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f07582-fe02-4906-8336-0702b6690d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:46.511076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f07582-fe02-4906-8336-0702b6690d3d admin admin] PATH_INFO: `/v3/projects/bcaab3f0817043c682737e94bb1dd1ac` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:46.521275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f07582-fe02-4906-8336-0702b6690d3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcaab3f0817043c682737e94bb1dd1ac)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:46.521833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be245ae0-2083-4bbc-aa74-d68f4dec21f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:46.521980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f07582-fe02-4906-8336-0702b6690d3d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:46.522119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be245ae0-2083-4bbc-aa74-d68f4dec21f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:46.522119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be245ae0-2083-4bbc-aa74-d68f4dec21f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:46.522119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be245ae0-2083-4bbc-aa74-d68f4dec21f6 admin admin] PATH_INFO: `/v3/projects/0c14d930ddf943b598625b5352a03d8d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:46.522119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be245ae0-2083-4bbc-aa74-d68f4dec21f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c14d930ddf943b598625b5352a03d8d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:46.524362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be245ae0-2083-4bbc-aa74-d68f4dec21f6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:46.583286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b1f07582-fe02-4906-8336-0702b6690d3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcaab3f0817043c682737e94bb1dd1ac', 'request_id': 'req-b1f07582-fe02-4906-8336-0702b6690d3d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:46.591800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b1f07582-fe02-4906-8336-0702b6690d3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcaab3f0817043c682737e94bb1dd1ac', 'request_id': 'req-b1f07582-fe02-4906-8336-0702b6690d3d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:46.601262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:46.606805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b1f07582-fe02-4906-8336-0702b6690d3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcaab3f0817043c682737e94bb1dd1ac', 'request_id': 'req-b1f07582-fe02-4906-8336-0702b6690d3d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:46.609666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-be245ae0-2083-4bbc-aa74-d68f4dec21f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c14d930ddf943b598625b5352a03d8d', 'request_id': 'req-be245ae0-2083-4bbc-aa74-d68f4dec21f6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:46.625241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-be245ae0-2083-4bbc-aa74-d68f4dec21f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c14d930ddf943b598625b5352a03d8d', 'request_id': 'req-be245ae0-2083-4bbc-aa74-d68f4dec21f6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:46.627430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b1f07582-fe02-4906-8336-0702b6690d3d admin admin] The token cache is being invalidate because project bcaab3f0817043c682737e94bb1dd1ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:56:46.627795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b1f07582-fe02-4906-8336-0702b6690d3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:46.628650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b1f07582-fe02-4906-8336-0702b6690d3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:46.630251 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1599/3125] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:56:46 2022] DELETE /identity/v3/projects/bcaab3f0817043c682737e94bb1dd1ac => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:56:46.635747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-be245ae0-2083-4bbc-aa74-d68f4dec21f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c14d930ddf943b598625b5352a03d8d', 'request_id': 'req-be245ae0-2083-4bbc-aa74-d68f4dec21f6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:46.644649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7c5ca-60bc-4cb5-971d-a4aa29d6a4e5 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:46.645028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7c5ca-60bc-4cb5-971d-a4aa29d6a4e5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:46.645952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7c5ca-60bc-4cb5-971d-a4aa29d6a4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:46.671141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-be245ae0-2083-4bbc-aa74-d68f4dec21f6 admin admin] The token cache is being invalidate because project 0c14d930ddf943b598625b5352a03d8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:56:46.671418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-be245ae0-2083-4bbc-aa74-d68f4dec21f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:46.672160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-be245ae0-2083-4bbc-aa74-d68f4dec21f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:46.674844 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1527/3126] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:56:46 2022] DELETE /identity/v3/projects/0c14d930ddf943b598625b5352a03d8d => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:46.679132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-18a7c5ca-60bc-4cb5-971d-a4aa29d6a4e5 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:46.680744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b075e-72a0-4cf2-87a3-c6b7abaeafce None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:46.681059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-18a7c5ca-60bc-4cb5-971d-a4aa29d6a4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:46.681414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b075e-72a0-4cf2-87a3-c6b7abaeafce None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:46.681886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b075e-72a0-4cf2-87a3-c6b7abaeafce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:46.711392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ac4b075e-72a0-4cf2-87a3-c6b7abaeafce None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:46.720491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ac4b075e-72a0-4cf2-87a3-c6b7abaeafce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:46.776597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:46.828573 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1600/3127] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:56:46.831315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b4dac-f1f1-4066-be81-d68bab0a279e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:46.831770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b4dac-f1f1-4066-be81-d68bab0a279e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:46.832598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b4dac-f1f1-4066-be81-d68bab0a279e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:46.846534 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1528/3128] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:46.848971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613565ed-b5b1-4580-ba1d-1308f3ff8bd0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:46.849258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613565ed-b5b1-4580-ba1d-1308f3ff8bd0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:46.850336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613565ed-b5b1-4580-ba1d-1308f3ff8bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:46.870802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b90b4dac-f1f1-4066-be81-d68bab0a279e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:46.885924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b90b4dac-f1f1-4066-be81-d68bab0a279e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:46.899655 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1601/3129] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:46.901294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b4dac-f1f1-4066-be81-d68bab0a279e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:46.916660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-613565ed-b5b1-4580-ba1d-1308f3ff8bd0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:46.922110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-613565ed-b5b1-4580-ba1d-1308f3ff8bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:46.933294 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1529/3130] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:46.936824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613565ed-b5b1-4580-ba1d-1308f3ff8bd0 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:46.936966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b4dac-f1f1-4066-be81-d68bab0a279e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:46.937086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-613565ed-b5b1-4580-ba1d-1308f3ff8bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:46.949155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5478ee-542f-4f2b-9ba4-8e384a7a79f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:46.949809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5478ee-542f-4f2b-9ba4-8e384a7a79f8 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:46.950098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5478ee-542f-4f2b-9ba4-8e384a7a79f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:46.950376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5478ee-542f-4f2b-9ba4-8e384a7a79f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:46.951071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5478ee-542f-4f2b-9ba4-8e384a7a79f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:56:46.951376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5478ee-542f-4f2b-9ba4-8e384a7a79f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:46.952799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613565ed-b5b1-4580-ba1d-1308f3ff8bd0 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:46.956369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5478ee-542f-4f2b-9ba4-8e384a7a79f8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:46.957609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa157f0f-b0b3-45e1-a0d6-14a3d41765cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:46.958322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa157f0f-b0b3-45e1-a0d6-14a3d41765cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:46.958707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa157f0f-b0b3-45e1-a0d6-14a3d41765cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:46.959876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa157f0f-b0b3-45e1-a0d6-14a3d41765cc admin admin] PATH_INFO: `/v3/users/1f9c07411c404bc3bc680e597f6c2e00` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:46.961480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa157f0f-b0b3-45e1-a0d6-14a3d41765cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f9c07411c404bc3bc680e597f6c2e00)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:46.963840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa157f0f-b0b3-45e1-a0d6-14a3d41765cc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:46.985497 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1602/3131] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:46.991487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5478ee-542f-4f2b-9ba4-8e384a7a79f8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:47.008770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5478ee-542f-4f2b-9ba4-8e384a7a79f8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:47.015387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1856e-c422-4a1b-b31b-3988ee92868f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:47.016003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1856e-c422-4a1b-b31b-3988ee92868f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:47.016295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1856e-c422-4a1b-b31b-3988ee92868f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:47.016710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1856e-c422-4a1b-b31b-3988ee92868f admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:47.017420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1856e-c422-4a1b-b31b-3988ee92868f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:56:47.017729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1856e-c422-4a1b-b31b-3988ee92868f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:47.019465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1856e-c422-4a1b-b31b-3988ee92868f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:47.040758 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1603/3132] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:47.041633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1856e-c422-4a1b-b31b-3988ee92868f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:47.054132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1856e-c422-4a1b-b31b-3988ee92868f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:47.062377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e5368-1189-4533-8ac1-6bfd7156163e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:47.063297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e5368-1189-4533-8ac1-6bfd7156163e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:47.063430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e5368-1189-4533-8ac1-6bfd7156163e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:47.063618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e5368-1189-4533-8ac1-6bfd7156163e admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:47.064236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e5368-1189-4533-8ac1-6bfd7156163e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:47.066255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e5368-1189-4533-8ac1-6bfd7156163e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:47.067083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-aa157f0f-b0b3-45e1-a0d6-14a3d41765cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f9c07411c404bc3bc680e597f6c2e00'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:47.070105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-aa157f0f-b0b3-45e1-a0d6-14a3d41765cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f9c07411c404bc3bc680e597f6c2e00'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:47.070936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-aa157f0f-b0b3-45e1-a0d6-14a3d41765cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f9c07411c404bc3bc680e597f6c2e00'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:47.085925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-aa157f0f-b0b3-45e1-a0d6-14a3d41765cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f9c07411c404bc3bc680e597f6c2e00'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:47.106382 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1604/3133] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:56:47 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:47.106382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e5368-1189-4533-8ac1-6bfd7156163e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:47.107661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5a2e5368-1189-4533-8ac1-6bfd7156163e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:47.113502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-aa157f0f-b0b3-45e1-a0d6-14a3d41765cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f9c07411c404bc3bc680e597f6c2e00'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:47.141227 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1530/3134] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:56:46 2022] DELETE /identity/v3/users/1f9c07411c404bc3bc680e597f6c2e00 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:47.142445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa157f0f-b0b3-45e1-a0d6-14a3d41765cc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:47.144301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-aa157f0f-b0b3-45e1-a0d6-14a3d41765cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:47.205026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e5368-1189-4533-8ac1-6bfd7156163e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:47.233079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa157f0f-b0b3-45e1-a0d6-14a3d41765cc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:47.238669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ea130-b03a-4ec8-8015-f51271806838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:47.239163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ea130-b03a-4ec8-8015-f51271806838 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:47.240481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ea130-b03a-4ec8-8015-f51271806838 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:47.240681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ea130-b03a-4ec8-8015-f51271806838 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:47.241571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ea130-b03a-4ec8-8015-f51271806838 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:47.243611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ea130-b03a-4ec8-8015-f51271806838 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:47.266408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:47.273059 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1531/3135] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:56:47 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:47.274481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654d69e-e7f1-4348-8aee-969273af14ad None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:47.274807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654d69e-e7f1-4348-8aee-969273af14ad None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:47.275098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654d69e-e7f1-4348-8aee-969273af14ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:47.297189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a145a-d149-4811-83b0-bbc06429c124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:47.297874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a145a-d149-4811-83b0-bbc06429c124 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:47.298372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a145a-d149-4811-83b0-bbc06429c124 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:47.298756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a145a-d149-4811-83b0-bbc06429c124 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:47.299367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5654d69e-e7f1-4348-8aee-969273af14ad None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:47.299494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a145a-d149-4811-83b0-bbc06429c124 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:47.300766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5654d69e-e7f1-4348-8aee-969273af14ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:47.301265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a145a-d149-4811-83b0-bbc06429c124 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:47.311657 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1532/3136] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:47.312647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654d69e-e7f1-4348-8aee-969273af14ad None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:47.326962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654d69e-e7f1-4348-8aee-969273af14ad None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:47.333898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f1d32-c2cf-4905-8297-34be4d826583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:47.335411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f1d32-c2cf-4905-8297-34be4d826583 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:47.335815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f1d32-c2cf-4905-8297-34be4d826583 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:47.336207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f1d32-c2cf-4905-8297-34be4d826583 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:47.336970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f1d32-c2cf-4905-8297-34be4d826583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:47.338785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f1d32-c2cf-4905-8297-34be4d826583 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:47.342403 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1605/3137] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:56:47 2022] POST /identity/v3/projects => generated 371 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:47.342403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a145a-d149-4811-83b0-bbc06429c124 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:47.343906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bd1a145a-d149-4811-83b0-bbc06429c124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:47.354708 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1533/3138] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:56:47.357226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f1d32-c2cf-4905-8297-34be4d826583 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:47.370891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f1d32-c2cf-4905-8297-34be4d826583 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:47.377475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:47.419304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a145a-d149-4811-83b0-bbc06429c124 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:47.424456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c61db-7386-4627-bc41-64f6e0c34e74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:47.425320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c61db-7386-4627-bc41-64f6e0c34e74 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:47.426132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c61db-7386-4627-bc41-64f6e0c34e74 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:47.426698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c61db-7386-4627-bc41-64f6e0c34e74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:47.454404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0789bb5-8e37-4ab1-9a9a-c3991ee1b1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:47.455224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0789bb5-8e37-4ab1-9a9a-c3991ee1b1f9 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:47.455975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0789bb5-8e37-4ab1-9a9a-c3991ee1b1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:47.455975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0789bb5-8e37-4ab1-9a9a-c3991ee1b1f9 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:47.456801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0789bb5-8e37-4ab1-9a9a-c3991ee1b1f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:47.458888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0789bb5-8e37-4ab1-9a9a-c3991ee1b1f9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:47.483420 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1534/3139] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:56:47 2022] POST /identity/v3/users => generated 286 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:47.484413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0789bb5-8e37-4ab1-9a9a-c3991ee1b1f9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:47.495091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0789bb5-8e37-4ab1-9a9a-c3991ee1b1f9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:47.501294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeebde2-8725-42db-8cd1-74fc7b4856ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:47.502018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeebde2-8725-42db-8cd1-74fc7b4856ab admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:47.502372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeebde2-8725-42db-8cd1-74fc7b4856ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:47.502746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeebde2-8725-42db-8cd1-74fc7b4856ab admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:47.503467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeebde2-8725-42db-8cd1-74fc7b4856ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:47.505296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeebde2-8725-42db-8cd1-74fc7b4856ab admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:47.509194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c61db-7386-4627-bc41-64f6e0c34e74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:47.510845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c61db-7386-4627-bc41-64f6e0c34e74 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:47.525214 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1606/3140] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:56:47.526097 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1535/3141] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:56:47.526889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c61db-7386-4627-bc41-64f6e0c34e74 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:47.531661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeebde2-8725-42db-8cd1-74fc7b4856ab admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:47.539399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c61db-7386-4627-bc41-64f6e0c34e74 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:47.544698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeebde2-8725-42db-8cd1-74fc7b4856ab admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:47.546311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be339cb1-3774-40b0-a5f0-9943476bccf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:47.547365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be339cb1-3774-40b0-a5f0-9943476bccf9 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:47.548498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be339cb1-3774-40b0-a5f0-9943476bccf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:47.548869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be339cb1-3774-40b0-a5f0-9943476bccf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:47.549686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be339cb1-3774-40b0-a5f0-9943476bccf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:47.551983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be339cb1-3774-40b0-a5f0-9943476bccf9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:47.553132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b6a88-85ff-49c3-ac75-a0f83bad89ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:47.554415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b6a88-85ff-49c3-ac75-a0f83bad89ad admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:47.554925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b6a88-85ff-49c3-ac75-a0f83bad89ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:47.555580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b6a88-85ff-49c3-ac75-a0f83bad89ad admin admin] PATH_INFO: `/v3/projects/49706c672bdd47d39405f8d7b80636ca/users/a418c7b850684796bf4022c5b05e936d/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:47.574918 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1607/3142] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:56:47.576190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b6a88-85ff-49c3-ac75-a0f83bad89ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=49706c672bdd47d39405f8d7b80636ca, user_id=a418c7b850684796bf4022c5b05e936d, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:47.577744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be339cb1-3774-40b0-a5f0-9943476bccf9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:47.579340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b6a88-85ff-49c3-ac75-a0f83bad89ad admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:47.593289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be339cb1-3774-40b0-a5f0-9943476bccf9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:47.606361 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1536/3143] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:47 2022] PUT /identity/v3/projects/49706c672bdd47d39405f8d7b80636ca/users/a418c7b850684796bf4022c5b05e936d/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:47.616556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83950b2c-1b6e-453a-ac51-bd8b2a4c64ea None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:47.617194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83950b2c-1b6e-453a-ac51-bd8b2a4c64ea None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:47.617194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83950b2c-1b6e-453a-ac51-bd8b2a4c64ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:47.671709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-83950b2c-1b6e-453a-ac51-bd8b2a4c64ea None None] MFA Rules not processed for user `a418c7b850684796bf4022c5b05e936d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:47.678759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-83950b2c-1b6e-453a-ac51-bd8b2a4c64ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:47.702057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ec441-8f10-49b0-b75c-ddcd19aee323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:47.702785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ec441-8f10-49b0-b75c-ddcd19aee323 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:47.703136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ec441-8f10-49b0-b75c-ddcd19aee323 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:47.703486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ec441-8f10-49b0-b75c-ddcd19aee323 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:47.704260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ec441-8f10-49b0-b75c-ddcd19aee323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:47.706776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ec441-8f10-49b0-b75c-ddcd19aee323 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:47.714348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:47.726633 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1608/3144] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:56:47.727569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ec441-8f10-49b0-b75c-ddcd19aee323 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:47.742250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ec441-8f10-49b0-b75c-ddcd19aee323 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:47.748891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7915d6-3049-47e7-bd78-d7f7f1800f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:47.749700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7915d6-3049-47e7-bd78-d7f7f1800f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:47.749983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7915d6-3049-47e7-bd78-d7f7f1800f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:47.750259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7915d6-3049-47e7-bd78-d7f7f1800f83 admin admin] PATH_INFO: `/v3/projects/9777f22c6fa9415292d22267bc555cc5/users/d54d53f1f120428f84b0fb8cb34254f7/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:47.771048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7915d6-3049-47e7-bd78-d7f7f1800f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9777f22c6fa9415292d22267bc555cc5, user_id=d54d53f1f120428f84b0fb8cb34254f7, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:47.776666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7915d6-3049-47e7-bd78-d7f7f1800f83 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:47.795104 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1609/3145] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:47 2022] PUT /identity/v3/projects/9777f22c6fa9415292d22267bc555cc5/users/d54d53f1f120428f84b0fb8cb34254f7/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:47.805544 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1537/3146] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:47.806549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d6f26-ad46-410a-84d5-2676f23e8270 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:47.806848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d6f26-ad46-410a-84d5-2676f23e8270 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:47.807125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d6f26-ad46-410a-84d5-2676f23e8270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:47.807251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbbaca-487b-4928-b925-308d15dd0424 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:47.807563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbbaca-487b-4928-b925-308d15dd0424 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:47.807851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbbaca-487b-4928-b925-308d15dd0424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:47.864653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6adbbaca-487b-4928-b925-308d15dd0424 None None] MFA Rules not processed for user `a418c7b850684796bf4022c5b05e936d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:47.866267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6adbbaca-487b-4928-b925-308d15dd0424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:47.892770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d37d6f26-ad46-410a-84d5-2676f23e8270 None None] MFA Rules not processed for user `d54d53f1f120428f84b0fb8cb34254f7`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:47.893997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d37d6f26-ad46-410a-84d5-2676f23e8270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:47.977348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:48.008005 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1538/3147] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:48.012906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbbaca-487b-4928-b925-308d15dd0424 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:48.014748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6adbbaca-487b-4928-b925-308d15dd0424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:48.071175 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1610/3148] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:48.072240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d6f26-ad46-410a-84d5-2676f23e8270 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:48.073854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d37d6f26-ad46-410a-84d5-2676f23e8270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:48.158075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbbaca-487b-4928-b925-308d15dd0424 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:48.171380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059efa99-fb6b-4512-bc38-054abc23ef55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:48.173188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059efa99-fb6b-4512-bc38-054abc23ef55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:48.173529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059efa99-fb6b-4512-bc38-054abc23ef55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:48.173905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059efa99-fb6b-4512-bc38-054abc23ef55 admin admin] PATH_INFO: `/v3/projects/90234df69d5e4546983a5d3f14143a7e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:48.176055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059efa99-fb6b-4512-bc38-054abc23ef55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90234df69d5e4546983a5d3f14143a7e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:48.180464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059efa99-fb6b-4512-bc38-054abc23ef55 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:48.238570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d6f26-ad46-410a-84d5-2676f23e8270 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:48.253765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867cace-f3fd-42aa-abaa-d935e6615656 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:48.254388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867cace-f3fd-42aa-abaa-d935e6615656 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:48.255308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867cace-f3fd-42aa-abaa-d935e6615656 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:48.255543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867cace-f3fd-42aa-abaa-d935e6615656 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:48.276705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867cace-f3fd-42aa-abaa-d935e6615656 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:48.280418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867cace-f3fd-42aa-abaa-d935e6615656 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:48.285010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-059efa99-fb6b-4512-bc38-054abc23ef55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90234df69d5e4546983a5d3f14143a7e', 'request_id': 'req-059efa99-fb6b-4512-bc38-054abc23ef55'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:48.298138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-059efa99-fb6b-4512-bc38-054abc23ef55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90234df69d5e4546983a5d3f14143a7e', 'request_id': 'req-059efa99-fb6b-4512-bc38-054abc23ef55'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:48.303248 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1611/3149] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:56:48.304126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867cace-f3fd-42aa-abaa-d935e6615656 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:48.308579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-059efa99-fb6b-4512-bc38-054abc23ef55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90234df69d5e4546983a5d3f14143a7e', 'request_id': 'req-059efa99-fb6b-4512-bc38-054abc23ef55'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:48.316012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867cace-f3fd-42aa-abaa-d935e6615656 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:48.382949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-059efa99-fb6b-4512-bc38-054abc23ef55 admin admin] The token cache is being invalidate because project 90234df69d5e4546983a5d3f14143a7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:56:48.383211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-059efa99-fb6b-4512-bc38-054abc23ef55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:48.386330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-059efa99-fb6b-4512-bc38-054abc23ef55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:48.397889 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1539/3150] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:56:48 2022] DELETE /identity/v3/projects/90234df69d5e4546983a5d3f14143a7e => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:56:48.407636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831bf35-aaaa-4ccf-be49-46e355e5f6e4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:48.407925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831bf35-aaaa-4ccf-be49-46e355e5f6e4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:48.408502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831bf35-aaaa-4ccf-be49-46e355e5f6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:48.418120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:48.432871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea184ecb-2da1-497c-9007-c973e620e4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:48.433569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea184ecb-2da1-497c-9007-c973e620e4ca admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:48.434013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea184ecb-2da1-497c-9007-c973e620e4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:48.434447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea184ecb-2da1-497c-9007-c973e620e4ca admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:48.435222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea184ecb-2da1-497c-9007-c973e620e4ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:48.437655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea184ecb-2da1-497c-9007-c973e620e4ca admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:48.454122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7831bf35-aaaa-4ccf-be49-46e355e5f6e4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:48.458698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7831bf35-aaaa-4ccf-be49-46e355e5f6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:48.479393 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1540/3151] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:48.489107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d35f73-0b9e-45ed-a204-2df69324e2aa None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:48.489738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d35f73-0b9e-45ed-a204-2df69324e2aa None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:48.490144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d35f73-0b9e-45ed-a204-2df69324e2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:48.502261 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1612/3152] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:56:48 2022] POST /identity/v3/projects => generated 375 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:48.503487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea184ecb-2da1-497c-9007-c973e620e4ca admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:48.508980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ea184ecb-2da1-497c-9007-c973e620e4ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:48.547057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-87d35f73-0b9e-45ed-a204-2df69324e2aa None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:48.548301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-87d35f73-0b9e-45ed-a204-2df69324e2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:48.659739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea184ecb-2da1-497c-9007-c973e620e4ca admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:48.667633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31e393-7a88-4716-90f3-9b841195b354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:48.668344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31e393-7a88-4716-90f3-9b841195b354 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:48.668683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31e393-7a88-4716-90f3-9b841195b354 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:48.669128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31e393-7a88-4716-90f3-9b841195b354 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:48.669826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31e393-7a88-4716-90f3-9b841195b354 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:48.674359 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1541/3153] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:48.674559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31e393-7a88-4716-90f3-9b841195b354 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:48.675436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d35f73-0b9e-45ed-a204-2df69324e2aa None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:48.694559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d35f73-0b9e-45ed-a204-2df69324e2aa None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:48.703394 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1613/3154] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:56:48 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:48.704467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31e393-7a88-4716-90f3-9b841195b354 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:48.705867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea9028-3f15-4ebd-9415-079b9745445e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:48.706707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea9028-3f15-4ebd-9415-079b9745445e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:48.707083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea9028-3f15-4ebd-9415-079b9745445e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:48.707646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea9028-3f15-4ebd-9415-079b9745445e admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:48.713423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea9028-3f15-4ebd-9415-079b9745445e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:56:48.713747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea9028-3f15-4ebd-9415-079b9745445e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:48.718119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea9028-3f15-4ebd-9415-079b9745445e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:48.720390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31e393-7a88-4716-90f3-9b841195b354 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:48.727905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52d197-1f41-4113-a7c6-9ccc2db63c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:48.728589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52d197-1f41-4113-a7c6-9ccc2db63c28 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:48.728878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52d197-1f41-4113-a7c6-9ccc2db63c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:48.729339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52d197-1f41-4113-a7c6-9ccc2db63c28 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:48.730018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52d197-1f41-4113-a7c6-9ccc2db63c28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:48.732261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52d197-1f41-4113-a7c6-9ccc2db63c28 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:48.747762 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1542/3155] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:48.752577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea9028-3f15-4ebd-9415-079b9745445e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:48.757327 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1614/3156] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:56:48.758444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52d197-1f41-4113-a7c6-9ccc2db63c28 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:48.785345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea9028-3f15-4ebd-9415-079b9745445e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:48.786765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52d197-1f41-4113-a7c6-9ccc2db63c28 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:48.794334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b59fe9-6c64-44c9-9988-557ade17b253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:48.794995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b59fe9-6c64-44c9-9988-557ade17b253 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:48.795353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b59fe9-6c64-44c9-9988-557ade17b253 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:48.795629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b59fe9-6c64-44c9-9988-557ade17b253 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:48.796372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b59fe9-6c64-44c9-9988-557ade17b253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:48.799502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd7412-d470-4b97-8faa-8e525f687ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:48.800368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd7412-d470-4b97-8faa-8e525f687ab3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:48.800810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd7412-d470-4b97-8faa-8e525f687ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:48.801407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd7412-d470-4b97-8faa-8e525f687ab3 admin admin] PATH_INFO: `/v3/projects/7b69297bbdcf4d3bb2076c739ec16adb/users/c56a46a5f20248f6bfb72cc01ac1ae9a/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:48.801549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b59fe9-6c64-44c9-9988-557ade17b253 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:48.820900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:48.822578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd7412-d470-4b97-8faa-8e525f687ab3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b69297bbdcf4d3bb2076c739ec16adb, user_id=c56a46a5f20248f6bfb72cc01ac1ae9a, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:48.831145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd7412-d470-4b97-8faa-8e525f687ab3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:48.863351 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1615/3157] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:48 2022] PUT /identity/v3/projects/7b69297bbdcf4d3bb2076c739ec16adb/users/c56a46a5f20248f6bfb72cc01ac1ae9a/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:48.868703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd7412-d470-4b97-8faa-8e525f687ab3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:48.887036 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1543/3158] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:56:48 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:56:48.888601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b59fe9-6c64-44c9-9988-557ade17b253 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:49.035453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd7412-d470-4b97-8faa-8e525f687ab3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:49.057986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b924d-d480-48e4-9d75-89a0afd2a16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:49.058677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b924d-d480-48e4-9d75-89a0afd2a16f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:49.058979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b924d-d480-48e4-9d75-89a0afd2a16f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:49.059336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b924d-d480-48e4-9d75-89a0afd2a16f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:49.059903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b924d-d480-48e4-9d75-89a0afd2a16f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:49.064169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b59fe9-6c64-44c9-9988-557ade17b253 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:49.069684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b924d-d480-48e4-9d75-89a0afd2a16f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:49.084773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d778be-cb57-4635-9fcf-d66f0a76835b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:49.088498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d778be-cb57-4635-9fcf-d66f0a76835b admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:49.088786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d778be-cb57-4635-9fcf-d66f0a76835b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:49.089066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d778be-cb57-4635-9fcf-d66f0a76835b admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:49.092093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d778be-cb57-4635-9fcf-d66f0a76835b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:49.096556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d778be-cb57-4635-9fcf-d66f0a76835b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:49.102882 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1616/3159] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:56:49.107908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b924d-d480-48e4-9d75-89a0afd2a16f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:49.132890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b924d-d480-48e4-9d75-89a0afd2a16f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:49.146196 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1544/3160] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:56:48 2022] POST /identity/v3/users => generated 290 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:49.153136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f786b-5a27-4fd1-b53d-0dc55b355242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:49.157377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f786b-5a27-4fd1-b53d-0dc55b355242 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:49.157679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f786b-5a27-4fd1-b53d-0dc55b355242 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:49.158021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d778be-cb57-4635-9fcf-d66f0a76835b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:49.159262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f786b-5a27-4fd1-b53d-0dc55b355242 admin admin] PATH_INFO: `/v3/projects/7b69297bbdcf4d3bb2076c739ec16adb/users/c56a46a5f20248f6bfb72cc01ac1ae9a/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:49.163504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f786b-5a27-4fd1-b53d-0dc55b355242 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b69297bbdcf4d3bb2076c739ec16adb, user_id=c56a46a5f20248f6bfb72cc01ac1ae9a, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:49.169420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f786b-5a27-4fd1-b53d-0dc55b355242 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:49.186500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d778be-cb57-4635-9fcf-d66f0a76835b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:49.188098 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1617/3161] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:49 2022] PUT /identity/v3/projects/7b69297bbdcf4d3bb2076c739ec16adb/users/c56a46a5f20248f6bfb72cc01ac1ae9a/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:49.202769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6531c93-4f7e-4181-bcaa-e037680eb593 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:49.206038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6531c93-4f7e-4181-bcaa-e037680eb593 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:49.206733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6531c93-4f7e-4181-bcaa-e037680eb593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:49.297064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:49.303630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f6531c93-4f7e-4181-bcaa-e037680eb593 None None] MFA Rules not processed for user `c56a46a5f20248f6bfb72cc01ac1ae9a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:49.307748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f6531c93-4f7e-4181-bcaa-e037680eb593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:49.406876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7082c-ed2b-4031-9fb2-70affac96051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:49.407716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7082c-ed2b-4031-9fb2-70affac96051 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:49.408000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7082c-ed2b-4031-9fb2-70affac96051 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:49.408284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7082c-ed2b-4031-9fb2-70affac96051 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:49.409252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7082c-ed2b-4031-9fb2-70affac96051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:49.414053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7082c-ed2b-4031-9fb2-70affac96051 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:49.456286 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1545/3162] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:56:49.461466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7082c-ed2b-4031-9fb2-70affac96051 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:49.493572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7082c-ed2b-4031-9fb2-70affac96051 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:49.500827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4d658-17cb-4a0f-99c3-32c66740638a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:49.501736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4d658-17cb-4a0f-99c3-32c66740638a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:49.502231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4d658-17cb-4a0f-99c3-32c66740638a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:49.502889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4d658-17cb-4a0f-99c3-32c66740638a admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:49.504466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b4d658-17cb-4a0f-99c3-32c66740638a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:49.507823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b4d658-17cb-4a0f-99c3-32c66740638a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:49.512219 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1618/3163] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:49.518960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6531c93-4f7e-4181-bcaa-e037680eb593 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:49.535021 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1546/3164] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:56:49.538952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4d658-17cb-4a0f-99c3-32c66740638a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:49.557765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4d658-17cb-4a0f-99c3-32c66740638a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:49.560980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6531c93-4f7e-4181-bcaa-e037680eb593 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:49.565269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dfe918-5697-45f6-bdd4-26efa31f01b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:49.566807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfe918-5697-45f6-bdd4-26efa31f01b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:49.567376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfe918-5697-45f6-bdd4-26efa31f01b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:49.567848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfe918-5697-45f6-bdd4-26efa31f01b5 admin admin] PATH_INFO: `/v3/projects/a4961525068042aba372c046eb589a7a/users/56b4d78c7c6c450fb5fc1d3ac587fb26/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:49.594535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca82d0-54b6-4e7f-88f1-635ae6a58b77 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:49.594820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dfe918-5697-45f6-bdd4-26efa31f01b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4961525068042aba372c046eb589a7a, user_id=56b4d78c7c6c450fb5fc1d3ac587fb26, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:49.595115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca82d0-54b6-4e7f-88f1-635ae6a58b77 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:49.595536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca82d0-54b6-4e7f-88f1-635ae6a58b77 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:49.595671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca82d0-54b6-4e7f-88f1-635ae6a58b77 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:49.600617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dfe918-5697-45f6-bdd4-26efa31f01b5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:49.600832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca82d0-54b6-4e7f-88f1-635ae6a58b77 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:56:49.601219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca82d0-54b6-4e7f-88f1-635ae6a58b77 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:49.610319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca82d0-54b6-4e7f-88f1-635ae6a58b77 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:49.654222 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1547/3165] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:56:49 2022] PUT /identity/v3/projects/a4961525068042aba372c046eb589a7a/users/56b4d78c7c6c450fb5fc1d3ac587fb26/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:49.662536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba075c-9e59-4a53-88d0-de6a834f48c6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:49.662885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba075c-9e59-4a53-88d0-de6a834f48c6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:49.664614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba075c-9e59-4a53-88d0-de6a834f48c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:49.684495 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1619/3166] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:49.696535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca82d0-54b6-4e7f-88f1-635ae6a58b77 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:49.717677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca82d0-54b6-4e7f-88f1-635ae6a58b77 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:49.731625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6fba075c-9e59-4a53-88d0-de6a834f48c6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:49.734629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6fba075c-9e59-4a53-88d0-de6a834f48c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:49.883162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df00d93-03e7-4a8e-9be6-fc47224d42b2 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:49.883774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df00d93-03e7-4a8e-9be6-fc47224d42b2 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:49.884257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df00d93-03e7-4a8e-9be6-fc47224d42b2 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:49.884544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df00d93-03e7-4a8e-9be6-fc47224d42b2 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:49.885292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df00d93-03e7-4a8e-9be6-fc47224d42b2 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:49.887414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df00d93-03e7-4a8e-9be6-fc47224d42b2 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:49.887890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:49.963144 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1620/3167] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:56:49 2022] POST /identity/v3/projects => generated 369 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:49.966015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df00d93-03e7-4a8e-9be6-fc47224d42b2 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:49.978252 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1548/3168] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:56:49.988517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba075c-9e59-4a53-88d0-de6a834f48c6 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:49.991832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6fba075c-9e59-4a53-88d0-de6a834f48c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:50.107826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba075c-9e59-4a53-88d0-de6a834f48c6 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:50.110872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df00d93-03e7-4a8e-9be6-fc47224d42b2 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:50.118182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5455396-fb81-45ae-ad72-88c946161848 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:50.119413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5455396-fb81-45ae-ad72-88c946161848 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:50.120057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5455396-fb81-45ae-ad72-88c946161848 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:50.120639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5455396-fb81-45ae-ad72-88c946161848 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:50.121304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5455396-fb81-45ae-ad72-88c946161848 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:56:50.122963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5455396-fb81-45ae-ad72-88c946161848 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:50.123612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5455396-fb81-45ae-ad72-88c946161848 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:50.140150 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1621/3169] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:50.148438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5455396-fb81-45ae-ad72-88c946161848 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:50.150288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d5455396-fb81-45ae-ad72-88c946161848 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:50.158836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:50.183575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bebc4b-2144-4750-bd3e-ddb0839e0fd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:50.184675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bebc4b-2144-4750-bd3e-ddb0839e0fd4 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:50.185323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bebc4b-2144-4750-bd3e-ddb0839e0fd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:50.187193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bebc4b-2144-4750-bd3e-ddb0839e0fd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:50.251783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5455396-fb81-45ae-ad72-88c946161848 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:50.258231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2df9b-6e5f-4a23-abd3-c8b8ee273818 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:50.258599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2df9b-6e5f-4a23-abd3-c8b8ee273818 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:50.258891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2df9b-6e5f-4a23-abd3-c8b8ee273818 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:50.259169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2df9b-6e5f-4a23-abd3-c8b8ee273818 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:50.269583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f2df9b-6e5f-4a23-abd3-c8b8ee273818 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:50.272671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f2df9b-6e5f-4a23-abd3-c8b8ee273818 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:50.285690 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1622/3170] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:56:50.286004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bebc4b-2144-4750-bd3e-ddb0839e0fd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:50.288442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bebc4b-2144-4750-bd3e-ddb0839e0fd4 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:50.299452 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1549/3171] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:56:50.313956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d3c8a-d10c-4cf3-9075-e396870a7d85 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:50.314330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d3c8a-d10c-4cf3-9075-e396870a7d85 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:50.314717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d3c8a-d10c-4cf3-9075-e396870a7d85 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:50.316135 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1623/3172] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:56:50.323155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bebc4b-2144-4750-bd3e-ddb0839e0fd4 service neutron] Authenticating service token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:56:50.342596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bebc4b-2144-4750-bd3e-ddb0839e0fd4 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:50.356484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bebc4b-2144-4750-bd3e-ddb0839e0fd4 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:50.363132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4875ca-3bcd-4c51-a8ed-0e6248691ba3 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:50.363945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4875ca-3bcd-4c51-a8ed-0e6248691ba3 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:50.364302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4875ca-3bcd-4c51-a8ed-0e6248691ba3 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:50.364650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4875ca-3bcd-4c51-a8ed-0e6248691ba3 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/projects/731dff07d63c4d4286e4cec5f8b7b868` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:50.366649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4875ca-3bcd-4c51-a8ed-0e6248691ba3 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:get_project(project_id=731dff07d63c4d4286e4cec5f8b7b868)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:50.368947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4875ca-3bcd-4c51-a8ed-0e6248691ba3 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:50.371471 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1550/3173] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:56:50 2022] GET /identity/v3/projects/731dff07d63c4d4286e4cec5f8b7b868 => generated 369 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:50.456200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635d21c-b6b6-4c6f-85d0-f15ee921598f None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:50.456668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635d21c-b6b6-4c6f-85d0-f15ee921598f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:50.456873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635d21c-b6b6-4c6f-85d0-f15ee921598f None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:50.459745 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1624/3174] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:56:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:56:50.484695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4875ca-3bcd-4c51-a8ed-0e6248691ba3 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating service token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:56:50.522422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4875ca-3bcd-4c51-a8ed-0e6248691ba3 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:50.532508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4875ca-3bcd-4c51-a8ed-0e6248691ba3 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:50.538557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203cc4d-241d-4813-a3ce-c7b3040cb32f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:50.539214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203cc4d-241d-4813-a3ce-c7b3040cb32f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:50.539453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203cc4d-241d-4813-a3ce-c7b3040cb32f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:50.539742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203cc4d-241d-4813-a3ce-c7b3040cb32f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/projects/731dff07d63c4d4286e4cec5f8b7b868` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:50.541337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203cc4d-241d-4813-a3ce-c7b3040cb32f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:get_project(project_id=731dff07d63c4d4286e4cec5f8b7b868)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:50.543952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203cc4d-241d-4813-a3ce-c7b3040cb32f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:50.545747 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1551/3175] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:56:50 2022] GET /identity/v3/projects/731dff07d63c4d4286e4cec5f8b7b868 => generated 369 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:50.720460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd2701-0ac3-4890-a613-793a2e4de894 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:50.720816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd2701-0ac3-4890-a613-793a2e4de894 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:50.721010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd2701-0ac3-4890-a613-793a2e4de894 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:50.722374 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1625/3176] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:56:50.728648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203cc4d-241d-4813-a3ce-c7b3040cb32f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating service token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:56:50.743332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203cc4d-241d-4813-a3ce-c7b3040cb32f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:50.753310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203cc4d-241d-4813-a3ce-c7b3040cb32f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:50.759380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600fd40-c11b-43bd-b653-c4bf628fdd73 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:50.760382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600fd40-c11b-43bd-b653-c4bf628fdd73 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:50.760677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600fd40-c11b-43bd-b653-c4bf628fdd73 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:50.760957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600fd40-c11b-43bd-b653-c4bf628fdd73 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/projects/731dff07d63c4d4286e4cec5f8b7b868` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:50.762140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600fd40-c11b-43bd-b653-c4bf628fdd73 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:get_project(project_id=731dff07d63c4d4286e4cec5f8b7b868)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:50.764892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600fd40-c11b-43bd-b653-c4bf628fdd73 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:50.774242 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1552/3177] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:56:50 2022] GET /identity/v3/projects/731dff07d63c4d4286e4cec5f8b7b868 => generated 369 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:50.823412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd2701-0ac3-4890-a613-793a2e4de894 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:50.849466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd2701-0ac3-4890-a613-793a2e4de894 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:50.875435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd28769-9583-4306-ae5c-47b83d61b6d4 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:50.876221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd28769-9583-4306-ae5c-47b83d61b6d4 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:50.876509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd28769-9583-4306-ae5c-47b83d61b6d4 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:50.876782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd28769-9583-4306-ae5c-47b83d61b6d4 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/projects/731dff07d63c4d4286e4cec5f8b7b868` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:50.881332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd28769-9583-4306-ae5c-47b83d61b6d4 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:delete_project(project_id=731dff07d63c4d4286e4cec5f8b7b868)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:50.899353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd28769-9583-4306-ae5c-47b83d61b6d4 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:51.011666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ecd28769-9583-4306-ae5c-47b83d61b6d4 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '731dff07d63c4d4286e4cec5f8b7b868', 'request_id': 'req-ecd28769-9583-4306-ae5c-47b83d61b6d4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:51.027164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ecd28769-9583-4306-ae5c-47b83d61b6d4 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '731dff07d63c4d4286e4cec5f8b7b868', 'request_id': 'req-ecd28769-9583-4306-ae5c-47b83d61b6d4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:51.038646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ecd28769-9583-4306-ae5c-47b83d61b6d4 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '731dff07d63c4d4286e4cec5f8b7b868', 'request_id': 'req-ecd28769-9583-4306-ae5c-47b83d61b6d4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:51.061226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ecd28769-9583-4306-ae5c-47b83d61b6d4 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] The token cache is being invalidate because project 731dff07d63c4d4286e4cec5f8b7b868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:56:51.061517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ecd28769-9583-4306-ae5c-47b83d61b6d4 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:51.062252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ecd28769-9583-4306-ae5c-47b83d61b6d4 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:51.072402 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1626/3178] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:56:50 2022] DELETE /identity/v3/projects/731dff07d63c4d4286e4cec5f8b7b868 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:51.087012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092a9b5-3695-41d9-8f65-ce8495518789 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:51.087313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092a9b5-3695-41d9-8f65-ce8495518789 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:51.088564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092a9b5-3695-41d9-8f65-ce8495518789 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:51.089287 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1553/3179] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:56:51.096129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd28769-9583-4306-ae5c-47b83d61b6d4 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:56:51.097651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ecd28769-9583-4306-ae5c-47b83d61b6d4 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:51.196142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd28769-9583-4306-ae5c-47b83d61b6d4 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:51.197736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ecd28769-9583-4306-ae5c-47b83d61b6d4 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:51.208621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:51.289339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd28769-9583-4306-ae5c-47b83d61b6d4 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:51.298772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44102e-ac44-4ba7-951c-b9d6e7446971 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:51.299440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44102e-ac44-4ba7-951c-b9d6e7446971 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:51.299811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44102e-ac44-4ba7-951c-b9d6e7446971 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:51.300153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44102e-ac44-4ba7-951c-b9d6e7446971 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/projects/9777f22c6fa9415292d22267bc555cc5` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:51.301425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44102e-ac44-4ba7-951c-b9d6e7446971 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:get_project(project_id=9777f22c6fa9415292d22267bc555cc5)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:51.304630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44102e-ac44-4ba7-951c-b9d6e7446971 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:51.306802 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1627/3180] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:56:51 2022] GET /identity/v3/projects/9777f22c6fa9415292d22267bc555cc5 => generated 371 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:51.331780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092a9b5-3695-41d9-8f65-ce8495518789 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:51.357376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092a9b5-3695-41d9-8f65-ce8495518789 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:51.367248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ababb5c-1380-4846-89a5-dab1172a761f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:51.368468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ababb5c-1380-4846-89a5-dab1172a761f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:51.368999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ababb5c-1380-4846-89a5-dab1172a761f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:51.371351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ababb5c-1380-4846-89a5-dab1172a761f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:51.372209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ababb5c-1380-4846-89a5-dab1172a761f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:56:51.377765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ababb5c-1380-4846-89a5-dab1172a761f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:51.382957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ababb5c-1380-4846-89a5-dab1172a761f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:51.407305 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1554/3181] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:51.411786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44102e-ac44-4ba7-951c-b9d6e7446971 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:51.428086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44102e-ac44-4ba7-951c-b9d6e7446971 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:51.440428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884a80e-f8bf-428e-a297-ce0fd19b38f6 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:51.441251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884a80e-f8bf-428e-a297-ce0fd19b38f6 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:51.442247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884a80e-f8bf-428e-a297-ce0fd19b38f6 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:51.442719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884a80e-f8bf-428e-a297-ce0fd19b38f6 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:51.444791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884a80e-f8bf-428e-a297-ce0fd19b38f6 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:51.448706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884a80e-f8bf-428e-a297-ce0fd19b38f6 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:51.505593 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1628/3182] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:56:51 2022] POST /identity/v3/projects => generated 369 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:51.507903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ababb5c-1380-4846-89a5-dab1172a761f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:51.524214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:51.534519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ababb5c-1380-4846-89a5-dab1172a761f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:51.634236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ce647-0089-4b26-a79e-aff415483433 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:51.635348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ce647-0089-4b26-a79e-aff415483433 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:51.635913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ce647-0089-4b26-a79e-aff415483433 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:51.636464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ce647-0089-4b26-a79e-aff415483433 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:51.637344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ce647-0089-4b26-a79e-aff415483433 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:56:51.637866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ce647-0089-4b26-a79e-aff415483433 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:51.639841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ce647-0089-4b26-a79e-aff415483433 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:51.656368 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1555/3183] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:51.786812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4179a0-e887-4187-b3c5-ec8de9c251a4 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:51.787170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4179a0-e887-4187-b3c5-ec8de9c251a4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:51.787447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4179a0-e887-4187-b3c5-ec8de9c251a4 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:51.788832 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1629/3184] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:56:51.794094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ce647-0089-4b26-a79e-aff415483433 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating service token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:56:51.874125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ce647-0089-4b26-a79e-aff415483433 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:51.884383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ce647-0089-4b26-a79e-aff415483433 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:51.889862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f554e9-d681-4f48-aa02-ba2299c6441b tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:51.890659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f554e9-d681-4f48-aa02-ba2299c6441b tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:51.890983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f554e9-d681-4f48-aa02-ba2299c6441b tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:51.891523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f554e9-d681-4f48-aa02-ba2299c6441b tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/projects/2c5ebc4b698040f6bed9392c55617867` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:51.892801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f554e9-d681-4f48-aa02-ba2299c6441b tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:get_project(project_id=2c5ebc4b698040f6bed9392c55617867)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:51.895123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f554e9-d681-4f48-aa02-ba2299c6441b tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:51.897164 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1556/3185] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:56:51 2022] GET /identity/v3/projects/2c5ebc4b698040f6bed9392c55617867 => generated 369 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:52.104881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf911206-c5a4-44a4-a39a-92239c7acd24 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:52.105533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf911206-c5a4-44a4-a39a-92239c7acd24 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:52.105826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf911206-c5a4-44a4-a39a-92239c7acd24 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:52.108703 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1630/3186] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:56:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:56:52.114242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f554e9-d681-4f48-aa02-ba2299c6441b tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating service token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:56:52.135171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f554e9-d681-4f48-aa02-ba2299c6441b tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:52.153055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f554e9-d681-4f48-aa02-ba2299c6441b tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:52.160766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a57fb-40a5-41aa-ba55-abf855ae2d7a tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:52.161592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a57fb-40a5-41aa-ba55-abf855ae2d7a tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:52.162577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a57fb-40a5-41aa-ba55-abf855ae2d7a tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:52.162916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a57fb-40a5-41aa-ba55-abf855ae2d7a tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/projects/2c5ebc4b698040f6bed9392c55617867` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:52.164521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a57fb-40a5-41aa-ba55-abf855ae2d7a tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:get_project(project_id=2c5ebc4b698040f6bed9392c55617867)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:52.168219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a57fb-40a5-41aa-ba55-abf855ae2d7a tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:52.171185 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1557/3187] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:56:52 2022] GET /identity/v3/projects/2c5ebc4b698040f6bed9392c55617867 => generated 369 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:52.234315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf911206-c5a4-44a4-a39a-92239c7acd24 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:52.278066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf911206-c5a4-44a4-a39a-92239c7acd24 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:52.290670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6389b-2a52-4e65-916b-22d0c43eeae5 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:52.291674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6389b-2a52-4e65-916b-22d0c43eeae5 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:52.294862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6389b-2a52-4e65-916b-22d0c43eeae5 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:52.295281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6389b-2a52-4e65-916b-22d0c43eeae5 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:52.296685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6389b-2a52-4e65-916b-22d0c43eeae5 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:56:52.297094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6389b-2a52-4e65-916b-22d0c43eeae5 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:52.298864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6389b-2a52-4e65-916b-22d0c43eeae5 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:52.319664 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1631/3188] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:52.321616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a57fb-40a5-41aa-ba55-abf855ae2d7a tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:52.336027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a57fb-40a5-41aa-ba55-abf855ae2d7a tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:52.342479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fac593-8094-40b1-bef9-a18859347f4e tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:52.343803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fac593-8094-40b1-bef9-a18859347f4e tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:52.344089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fac593-8094-40b1-bef9-a18859347f4e tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:52.344514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fac593-8094-40b1-bef9-a18859347f4e tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:52.345179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fac593-8094-40b1-bef9-a18859347f4e tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:52.348181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fac593-8094-40b1-bef9-a18859347f4e tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:52.381796 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1558/3189] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:56:52 2022] POST /identity/v3/users => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:56:52.382381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6389b-2a52-4e65-916b-22d0c43eeae5 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:52.394301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6389b-2a52-4e65-916b-22d0c43eeae5 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:52.401457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161fd0f-0a7e-4915-b612-a8a701de525d tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:52.402883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161fd0f-0a7e-4915-b612-a8a701de525d tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:52.403386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161fd0f-0a7e-4915-b612-a8a701de525d tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:52.404528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161fd0f-0a7e-4915-b612-a8a701de525d tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:52.405479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161fd0f-0a7e-4915-b612-a8a701de525d tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:56:52.406681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161fd0f-0a7e-4915-b612-a8a701de525d tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:52.408807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161fd0f-0a7e-4915-b612-a8a701de525d tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:52.428337 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1632/3190] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:56:52.451098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a77660-1d8f-4285-8cc9-cb525c979b4c None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:52.451823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a77660-1d8f-4285-8cc9-cb525c979b4c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:52.452340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a77660-1d8f-4285-8cc9-cb525c979b4c None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:52.455467 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1559/3191] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:56:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:56:52.466145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161fd0f-0a7e-4915-b612-a8a701de525d tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:56:52.468688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f161fd0f-0a7e-4915-b612-a8a701de525d tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:52.521126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161fd0f-0a7e-4915-b612-a8a701de525d tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:52.534161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161fd0f-0a7e-4915-b612-a8a701de525d tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:52.548471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86df0ae-17dd-4fad-bcff-ee0b0bfa4d37 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:52.549275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86df0ae-17dd-4fad-bcff-ee0b0bfa4d37 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:52.549657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86df0ae-17dd-4fad-bcff-ee0b0bfa4d37 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:52.550031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86df0ae-17dd-4fad-bcff-ee0b0bfa4d37 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/projects/2c5ebc4b698040f6bed9392c55617867` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:52.554539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86df0ae-17dd-4fad-bcff-ee0b0bfa4d37 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:get_project(project_id=2c5ebc4b698040f6bed9392c55617867)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:52.556616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86df0ae-17dd-4fad-bcff-ee0b0bfa4d37 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:52.558362 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1633/3192] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:56:52 2022] GET /identity/v3/projects/2c5ebc4b698040f6bed9392c55617867 => generated 369 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:52.795533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34b97b-a85e-4b48-8f50-7ca649278a2c None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:52.796097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34b97b-a85e-4b48-8f50-7ca649278a2c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:52.796097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34b97b-a85e-4b48-8f50-7ca649278a2c None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:52.797756 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1560/3193] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:56:52.804078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86df0ae-17dd-4fad-bcff-ee0b0bfa4d37 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:56:52.822170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86df0ae-17dd-4fad-bcff-ee0b0bfa4d37 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:52.835055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86df0ae-17dd-4fad-bcff-ee0b0bfa4d37 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:52.843214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125cccf-6c48-402a-90f6-a5fb7989c731 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:52.846905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125cccf-6c48-402a-90f6-a5fb7989c731 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:52.847298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125cccf-6c48-402a-90f6-a5fb7989c731 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:52.848499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125cccf-6c48-402a-90f6-a5fb7989c731 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/projects/2c5ebc4b698040f6bed9392c55617867` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:52.852042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125cccf-6c48-402a-90f6-a5fb7989c731 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:get_project(project_id=2c5ebc4b698040f6bed9392c55617867)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:52.857848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125cccf-6c48-402a-90f6-a5fb7989c731 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:52.864057 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1634/3194] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:56:52 2022] GET /identity/v3/projects/2c5ebc4b698040f6bed9392c55617867 => generated 369 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:52.912889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34b97b-a85e-4b48-8f50-7ca649278a2c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:52.930307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34b97b-a85e-4b48-8f50-7ca649278a2c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:52.936315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11758a-f73a-46bd-8ac1-ae38ed32746e tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:52.937582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11758a-f73a-46bd-8ac1-ae38ed32746e tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:52.938095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11758a-f73a-46bd-8ac1-ae38ed32746e tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:52.938558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11758a-f73a-46bd-8ac1-ae38ed32746e tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/users/b95f61b5a9c04888b73c7a10a3fbe123` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:52.958634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11758a-f73a-46bd-8ac1-ae38ed32746e tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:delete_user(user_id=b95f61b5a9c04888b73c7a10a3fbe123)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:52.961562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11758a-f73a-46bd-8ac1-ae38ed32746e tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:53.057972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:53.071169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ac11758a-f73a-46bd-8ac1-ae38ed32746e tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b95f61b5a9c04888b73c7a10a3fbe123'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:53.071169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ac11758a-f73a-46bd-8ac1-ae38ed32746e tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b95f61b5a9c04888b73c7a10a3fbe123'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:53.071169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ac11758a-f73a-46bd-8ac1-ae38ed32746e tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b95f61b5a9c04888b73c7a10a3fbe123'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:53.086771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ac11758a-f73a-46bd-8ac1-ae38ed32746e tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b95f61b5a9c04888b73c7a10a3fbe123'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:53.107427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ac11758a-f73a-46bd-8ac1-ae38ed32746e tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b95f61b5a9c04888b73c7a10a3fbe123'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:53.144530 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1561/3195] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:56:52 2022] DELETE /identity/v3/users/b95f61b5a9c04888b73c7a10a3fbe123 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:53.148182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125cccf-6c48-402a-90f6-a5fb7989c731 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:53.150114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c125cccf-6c48-402a-90f6-a5fb7989c731 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:53.168440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125cccf-6c48-402a-90f6-a5fb7989c731 tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:53.359823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5e20c-10fc-4452-ba80-24465b69d00c tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:53.360604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5e20c-10fc-4452-ba80-24465b69d00c tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:53.360993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5e20c-10fc-4452-ba80-24465b69d00c tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:53.361396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5e20c-10fc-4452-ba80-24465b69d00c tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/projects/2c5ebc4b698040f6bed9392c55617867` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:53.366206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf5e20c-10fc-4452-ba80-24465b69d00c tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2c5ebc4b698040f6bed9392c55617867)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:53.371985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf5e20c-10fc-4452-ba80-24465b69d00c tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:53.472715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:53.482024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eaf5e20c-10fc-4452-ba80-24465b69d00c tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c5ebc4b698040f6bed9392c55617867', 'request_id': 'req-eaf5e20c-10fc-4452-ba80-24465b69d00c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:53.493286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eaf5e20c-10fc-4452-ba80-24465b69d00c tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c5ebc4b698040f6bed9392c55617867', 'request_id': 'req-eaf5e20c-10fc-4452-ba80-24465b69d00c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:53.501083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eaf5e20c-10fc-4452-ba80-24465b69d00c tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c5ebc4b698040f6bed9392c55617867', 'request_id': 'req-eaf5e20c-10fc-4452-ba80-24465b69d00c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:53.523442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eaf5e20c-10fc-4452-ba80-24465b69d00c tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] The token cache is being invalidate because project 2c5ebc4b698040f6bed9392c55617867 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:56:53.523742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eaf5e20c-10fc-4452-ba80-24465b69d00c tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:53.524643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eaf5e20c-10fc-4452-ba80-24465b69d00c tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:53.532359 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1635/3196] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:56:53 2022] DELETE /identity/v3/projects/2c5ebc4b698040f6bed9392c55617867 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:56:53.549109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8772d95-ac83-477c-818e-1ab3df0c21b7 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:53.549399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8772d95-ac83-477c-818e-1ab3df0c21b7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:53.549681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8772d95-ac83-477c-818e-1ab3df0c21b7 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:53.559413 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1562/3197] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:56:53 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:56:53.565273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5e20c-10fc-4452-ba80-24465b69d00c tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:56:53.566950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-eaf5e20c-10fc-4452-ba80-24465b69d00c tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:53.682327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5e20c-10fc-4452-ba80-24465b69d00c tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:53.684182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-eaf5e20c-10fc-4452-ba80-24465b69d00c tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:53.790007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5e20c-10fc-4452-ba80-24465b69d00c tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:53.818410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3cc081-876b-445a-b22d-32a0079c253a tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:53.819514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cc081-876b-445a-b22d-32a0079c253a tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:53.820154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cc081-876b-445a-b22d-32a0079c253a tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:53.821288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cc081-876b-445a-b22d-32a0079c253a tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/projects/9777f22c6fa9415292d22267bc555cc5` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:53.823732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3cc081-876b-445a-b22d-32a0079c253a tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:get_project(project_id=9777f22c6fa9415292d22267bc555cc5)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:53.828679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3cc081-876b-445a-b22d-32a0079c253a tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:53.846001 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1636/3198] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:56:53 2022] GET /identity/v3/projects/9777f22c6fa9415292d22267bc555cc5 => generated 371 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:53.895986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120f73a-a419-4cb4-a7b7-8b923bbdfbf8 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:53.897226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120f73a-a419-4cb4-a7b7-8b923bbdfbf8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:53.897226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120f73a-a419-4cb4-a7b7-8b923bbdfbf8 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:53.898919 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1563/3199] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:56:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:56:53.914392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3cc081-876b-445a-b22d-32a0079c253a tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:56:53.930826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3cc081-876b-445a-b22d-32a0079c253a tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:53.942581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3cc081-876b-445a-b22d-32a0079c253a tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:53.948699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed189b-cc3f-4aab-b163-cdfca556f24f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:53.949832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed189b-cc3f-4aab-b163-cdfca556f24f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:53.950121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed189b-cc3f-4aab-b163-cdfca556f24f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:53.950398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed189b-cc3f-4aab-b163-cdfca556f24f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/projects/9777f22c6fa9415292d22267bc555cc5` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:53.951730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed189b-cc3f-4aab-b163-cdfca556f24f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:get_project(project_id=9777f22c6fa9415292d22267bc555cc5)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:53.953795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed189b-cc3f-4aab-b163-cdfca556f24f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:53.970101 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1637/3200] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:56:53 2022] GET /identity/v3/projects/9777f22c6fa9415292d22267bc555cc5 => generated 371 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:53.973102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8ce61-b8f5-49ca-aaf1-d76250d63ddb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:53.973281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8ce61-b8f5-49ca-aaf1-d76250d63ddb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:53.973744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8ce61-b8f5-49ca-aaf1-d76250d63ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:54.003044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-17a8ce61-b8f5-49ca-aaf1-d76250d63ddb None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:54.005365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-17a8ce61-b8f5-49ca-aaf1-d76250d63ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:54.186609 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1564/3201] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:56:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:54.204816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed189b-cc3f-4aab-b163-cdfca556f24f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:54.207367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-12ed189b-cc3f-4aab-b163-cdfca556f24f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:54.387287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:54.408689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed189b-cc3f-4aab-b163-cdfca556f24f tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:54.427378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ea9a8-d6b7-45fb-84df-e4bce80ea6a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:54.428596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ea9a8-d6b7-45fb-84df-e4bce80ea6a5 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:54.429191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ea9a8-d6b7-45fb-84df-e4bce80ea6a5 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:54.430459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ea9a8-d6b7-45fb-84df-e4bce80ea6a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:54.453489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ea9a8-d6b7-45fb-84df-e4bce80ea6a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:54.455926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ea9a8-d6b7-45fb-84df-e4bce80ea6a5 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:54.473977 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1638/3202] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:56:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 20:56:54.651107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f9a8c-1cb0-4fab-a274-545177ae82e3 None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:54.651700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f9a8c-1cb0-4fab-a274-545177ae82e3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:54.652131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f9a8c-1cb0-4fab-a274-545177ae82e3 None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:54.653634 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1565/3203] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:56:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:56:54.661038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ea9a8-d6b7-45fb-84df-e4bce80ea6a5 service swift] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:56:54.665128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5e6ea9a8-d6b7-45fb-84df-e4bce80ea6a5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:54.700858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ea9a8-d6b7-45fb-84df-e4bce80ea6a5 service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:54.756140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ea9a8-d6b7-45fb-84df-e4bce80ea6a5 service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:54.779447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d47068-7a7c-4032-9cae-b5c0c058c25d tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56a46a5f20248f6bfb72cc01ac1ae9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b69297bbdcf4d3bb2076c739ec16adb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:54.780101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d47068-7a7c-4032-9cae-b5c0c058c25d tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:54.780381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d47068-7a7c-4032-9cae-b5c0c058c25d tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:54.780848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d47068-7a7c-4032-9cae-b5c0c058c25d tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] PATH_INFO: `/v3/projects/9777f22c6fa9415292d22267bc555cc5` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:54.784709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d47068-7a7c-4032-9cae-b5c0c058c25d tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorizing `identity:get_project(project_id=9777f22c6fa9415292d22267bc555cc5)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:54.786821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d47068-7a7c-4032-9cae-b5c0c058c25d tempest-QuotasAdminTestJSON-1447010991 tempest-QuotasAdminTestJSON-1447010991-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:54.803211 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1639/3204] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:56:54 2022] GET /identity/v3/projects/9777f22c6fa9415292d22267bc555cc5 => generated 371 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:55.584952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f9a8c-1cb0-4fab-a274-545177ae82e3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:55.587331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-991f9a8c-1cb0-4fab-a274-545177ae82e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:55.784659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f9a8c-1cb0-4fab-a274-545177ae82e3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:55.806931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5af8a8-66fc-45ad-ad55-8df0c7056808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:55.808399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5af8a8-66fc-45ad-ad55-8df0c7056808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:55.810113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5af8a8-66fc-45ad-ad55-8df0c7056808 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:55.810750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5af8a8-66fc-45ad-ad55-8df0c7056808 admin admin] PATH_INFO: `/v3/users/d54d53f1f120428f84b0fb8cb34254f7` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:55.814324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5af8a8-66fc-45ad-ad55-8df0c7056808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d54d53f1f120428f84b0fb8cb34254f7)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:55.820954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5af8a8-66fc-45ad-ad55-8df0c7056808 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:55.946714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ef5af8a8-66fc-45ad-ad55-8df0c7056808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd54d53f1f120428f84b0fb8cb34254f7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:55.947943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ef5af8a8-66fc-45ad-ad55-8df0c7056808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd54d53f1f120428f84b0fb8cb34254f7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:55.948788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ef5af8a8-66fc-45ad-ad55-8df0c7056808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd54d53f1f120428f84b0fb8cb34254f7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:55.963852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ef5af8a8-66fc-45ad-ad55-8df0c7056808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd54d53f1f120428f84b0fb8cb34254f7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:55.982303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:55.985313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ef5af8a8-66fc-45ad-ad55-8df0c7056808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd54d53f1f120428f84b0fb8cb34254f7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:56.047798 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1566/3205] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:56:55 2022] DELETE /identity/v3/users/d54d53f1f120428f84b0fb8cb34254f7 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:56.052781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c144b1e-5ba7-44eb-8303-022da992aa7b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:56.053748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c144b1e-5ba7-44eb-8303-022da992aa7b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:56.054256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c144b1e-5ba7-44eb-8303-022da992aa7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:56.109487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6c144b1e-5ba7-44eb-8303-022da992aa7b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:56.112183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6c144b1e-5ba7-44eb-8303-022da992aa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:56.333430 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1640/3206] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:56.344927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5af8a8-66fc-45ad-ad55-8df0c7056808 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:56.346910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ef5af8a8-66fc-45ad-ad55-8df0c7056808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:56.541090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5af8a8-66fc-45ad-ad55-8df0c7056808 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:56.557708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f35d5-4b61-41b9-9d00-71f30ae067d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:56.558645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f35d5-4b61-41b9-9d00-71f30ae067d5 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:56.559133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f35d5-4b61-41b9-9d00-71f30ae067d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:56.559752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f35d5-4b61-41b9-9d00-71f30ae067d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:56.577130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f35d5-4b61-41b9-9d00-71f30ae067d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:56.579389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f35d5-4b61-41b9-9d00-71f30ae067d5 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:56.598199 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1567/3207] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:56:57.609730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c144b1e-5ba7-44eb-8303-022da992aa7b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:57.612244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6c144b1e-5ba7-44eb-8303-022da992aa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:57.643953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c144b1e-5ba7-44eb-8303-022da992aa7b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:57.651789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54542faa-cf97-4a0c-b5b1-fc7e9fe7bd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:57.653135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54542faa-cf97-4a0c-b5b1-fc7e9fe7bd06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:57.653580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54542faa-cf97-4a0c-b5b1-fc7e9fe7bd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:57.654018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54542faa-cf97-4a0c-b5b1-fc7e9fe7bd06 admin admin] PATH_INFO: `/v3/projects/9777f22c6fa9415292d22267bc555cc5` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:57.655707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54542faa-cf97-4a0c-b5b1-fc7e9fe7bd06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9777f22c6fa9415292d22267bc555cc5)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:57.657780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54542faa-cf97-4a0c-b5b1-fc7e9fe7bd06 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:57.775696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-54542faa-cf97-4a0c-b5b1-fc7e9fe7bd06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9777f22c6fa9415292d22267bc555cc5', 'request_id': 'req-54542faa-cf97-4a0c-b5b1-fc7e9fe7bd06'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:57.787220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-54542faa-cf97-4a0c-b5b1-fc7e9fe7bd06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9777f22c6fa9415292d22267bc555cc5', 'request_id': 'req-54542faa-cf97-4a0c-b5b1-fc7e9fe7bd06'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:57.795225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-54542faa-cf97-4a0c-b5b1-fc7e9fe7bd06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9777f22c6fa9415292d22267bc555cc5', 'request_id': 'req-54542faa-cf97-4a0c-b5b1-fc7e9fe7bd06'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:57.822201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-54542faa-cf97-4a0c-b5b1-fc7e9fe7bd06 admin admin] The token cache is being invalidate because project 9777f22c6fa9415292d22267bc555cc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:56:57.822541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-54542faa-cf97-4a0c-b5b1-fc7e9fe7bd06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:57.823215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-54542faa-cf97-4a0c-b5b1-fc7e9fe7bd06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:57.825611 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1641/3208] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:56:57 2022] DELETE /identity/v3/projects/9777f22c6fa9415292d22267bc555cc5 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:57.829572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f35d5-4b61-41b9-9d00-71f30ae067d5 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:57.844295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f35d5-4b61-41b9-9d00-71f30ae067d5 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:58.010765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6cdf2-c331-480d-9a34-430f467d3258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:58.011905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6cdf2-c331-480d-9a34-430f467d3258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:58.012875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6cdf2-c331-480d-9a34-430f467d3258 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:58.013982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6cdf2-c331-480d-9a34-430f467d3258 admin admin] PATH_INFO: `/v3/users/c56a46a5f20248f6bfb72cc01ac1ae9a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:58.017360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6cdf2-c331-480d-9a34-430f467d3258 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c56a46a5f20248f6bfb72cc01ac1ae9a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:58.022152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6cdf2-c331-480d-9a34-430f467d3258 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:58.238900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-43a6cdf2-c331-480d-9a34-430f467d3258 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c56a46a5f20248f6bfb72cc01ac1ae9a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:58.242610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-43a6cdf2-c331-480d-9a34-430f467d3258 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c56a46a5f20248f6bfb72cc01ac1ae9a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:58.243767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-43a6cdf2-c331-480d-9a34-430f467d3258 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c56a46a5f20248f6bfb72cc01ac1ae9a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:58.272716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-43a6cdf2-c331-480d-9a34-430f467d3258 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c56a46a5f20248f6bfb72cc01ac1ae9a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:58.298443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-43a6cdf2-c331-480d-9a34-430f467d3258 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c56a46a5f20248f6bfb72cc01ac1ae9a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:58.320465 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1568/3209] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:56:57 2022] DELETE /identity/v3/users/c56a46a5f20248f6bfb72cc01ac1ae9a => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:58.984641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54542faa-cf97-4a0c-b5b1-fc7e9fe7bd06 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:58.989650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-54542faa-cf97-4a0c-b5b1-fc7e9fe7bd06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:59.014527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54542faa-cf97-4a0c-b5b1-fc7e9fe7bd06 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:59.149682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcab3b4-d292-49f6-89d7-da2d8b8b5d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:59.150317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcab3b4-d292-49f6-89d7-da2d8b8b5d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:59.150633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcab3b4-d292-49f6-89d7-da2d8b8b5d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:59.150909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcab3b4-d292-49f6-89d7-da2d8b8b5d2b admin admin] PATH_INFO: `/v3/projects/7b69297bbdcf4d3bb2076c739ec16adb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:59.156893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcab3b4-d292-49f6-89d7-da2d8b8b5d2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b69297bbdcf4d3bb2076c739ec16adb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:59.168816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcab3b4-d292-49f6-89d7-da2d8b8b5d2b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:59.191978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:59.285874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9bcab3b4-d292-49f6-89d7-da2d8b8b5d2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b69297bbdcf4d3bb2076c739ec16adb', 'request_id': 'req-9bcab3b4-d292-49f6-89d7-da2d8b8b5d2b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:59.301127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9bcab3b4-d292-49f6-89d7-da2d8b8b5d2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b69297bbdcf4d3bb2076c739ec16adb', 'request_id': 'req-9bcab3b4-d292-49f6-89d7-da2d8b8b5d2b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:59.316074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9bcab3b4-d292-49f6-89d7-da2d8b8b5d2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b69297bbdcf4d3bb2076c739ec16adb', 'request_id': 'req-9bcab3b4-d292-49f6-89d7-da2d8b8b5d2b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:59.339110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9bcab3b4-d292-49f6-89d7-da2d8b8b5d2b admin admin] The token cache is being invalidate because project 7b69297bbdcf4d3bb2076c739ec16adb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:56:59.339849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9bcab3b4-d292-49f6-89d7-da2d8b8b5d2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:59.340765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9bcab3b4-d292-49f6-89d7-da2d8b8b5d2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:56:59.342963 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1642/3210] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:56:58 2022] DELETE /identity/v3/projects/7b69297bbdcf4d3bb2076c739ec16adb => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:56:59.349189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553c895-83d9-4eec-b58a-cea572d69303 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:59.351427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553c895-83d9-4eec-b58a-cea572d69303 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:59.351723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553c895-83d9-4eec-b58a-cea572d69303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:59.410626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3553c895-83d9-4eec-b58a-cea572d69303 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:59.414270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3553c895-83d9-4eec-b58a-cea572d69303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:59.421560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:59.652510 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1569/3211] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:59.664263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e41b3-8c88-487c-86df-76eda7597216 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:59.664560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e41b3-8c88-487c-86df-76eda7597216 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:59.664855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e41b3-8c88-487c-86df-76eda7597216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:59.700026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c94e41b3-8c88-487c-86df-76eda7597216 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:56:59.702058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c94e41b3-8c88-487c-86df-76eda7597216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:59.723279 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1643/3212] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:56:59.724692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553c895-83d9-4eec-b58a-cea572d69303 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:59.743377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553c895-83d9-4eec-b58a-cea572d69303 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:59.768046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ce804-41e9-41e8-9230-8d10111e01c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:59.770215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ce804-41e9-41e8-9230-8d10111e01c0 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:59.770626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ce804-41e9-41e8-9230-8d10111e01c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:59.771782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ce804-41e9-41e8-9230-8d10111e01c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:59.773470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ce804-41e9-41e8-9230-8d10111e01c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:56:59.773880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ce804-41e9-41e8-9230-8d10111e01c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:59.776948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ce804-41e9-41e8-9230-8d10111e01c0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:59.792162 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1570/3213] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:56:59.797437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e41b3-8c88-487c-86df-76eda7597216 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:59.827747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ce804-41e9-41e8-9230-8d10111e01c0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:59.830340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1a4ce804-41e9-41e8-9230-8d10111e01c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:56:59.831040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e41b3-8c88-487c-86df-76eda7597216 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:56:59.839285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5abae-28c9-4971-8a26-4d7e86866279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:56:59.839912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5abae-28c9-4971-8a26-4d7e86866279 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:56:59.840194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5abae-28c9-4971-8a26-4d7e86866279 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:56:59.840465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5abae-28c9-4971-8a26-4d7e86866279 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:56:59.841161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5abae-28c9-4971-8a26-4d7e86866279 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:56:59.844205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5abae-28c9-4971-8a26-4d7e86866279 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:56:59.952357 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1644/3214] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:56:59 2022] POST /identity/v3/projects => generated 395 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:56:59.953718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5abae-28c9-4971-8a26-4d7e86866279 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:56:59.956349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d5e5abae-28c9-4971-8a26-4d7e86866279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:00.011817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ce804-41e9-41e8-9230-8d10111e01c0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:00.163753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b42890-e0e1-4a83-82fa-5dd8e1df4d88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:00.164895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5abae-28c9-4971-8a26-4d7e86866279 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:00.166057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b42890-e0e1-4a83-82fa-5dd8e1df4d88 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:00.166634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b42890-e0e1-4a83-82fa-5dd8e1df4d88 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:00.167155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b42890-e0e1-4a83-82fa-5dd8e1df4d88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:00.169785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-02b42890-e0e1-4a83-82fa-5dd8e1df4d88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:00.178115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4fdd7-8aae-4caa-9252-aa4424e9598f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:00.179113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4fdd7-8aae-4caa-9252-aa4424e9598f service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:00.179403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4fdd7-8aae-4caa-9252-aa4424e9598f service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:00.179683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4fdd7-8aae-4caa-9252-aa4424e9598f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:00.182152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2bb4fdd7-8aae-4caa-9252-aa4424e9598f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:00.210928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:00.279410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b42890-e0e1-4a83-82fa-5dd8e1df4d88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:00.283623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b42890-e0e1-4a83-82fa-5dd8e1df4d88 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:00.299805 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1571/3215] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:57:00.301324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b42890-e0e1-4a83-82fa-5dd8e1df4d88 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:00.305692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4fdd7-8aae-4caa-9252-aa4424e9598f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:00.307288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4fdd7-8aae-4caa-9252-aa4424e9598f service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:00.330672 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1645/3216] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:56:59 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:00.359653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c2204-3ace-4fe4-8762-126b851c14f8 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:00.359790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c2204-3ace-4fe4-8762-126b851c14f8 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:00.360062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c2204-3ace-4fe4-8762-126b851c14f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:00.372507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:00.449944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-128c2204-3ace-4fe4-8762-126b851c14f8 None None] MFA Rules not processed for user `56b4d78c7c6c450fb5fc1d3ac587fb26`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:00.456507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-128c2204-3ace-4fe4-8762-126b851c14f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:00.486455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b42890-e0e1-4a83-82fa-5dd8e1df4d88 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:00.501682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d54f9f-792a-4c01-882f-4b961e71e698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:00.502518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d54f9f-792a-4c01-882f-4b961e71e698 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:00.502950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d54f9f-792a-4c01-882f-4b961e71e698 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:00.504121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d54f9f-792a-4c01-882f-4b961e71e698 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:00.506643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d54f9f-792a-4c01-882f-4b961e71e698 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:00.512985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d54f9f-792a-4c01-882f-4b961e71e698 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:00.571623 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1572/3217] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:57:00 2022] POST /identity/v3/users => generated 298 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:00.575533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d54f9f-792a-4c01-882f-4b961e71e698 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:00.624922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d54f9f-792a-4c01-882f-4b961e71e698 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:00.640338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989bce4-1bf4-4739-bad2-0bf8e729c95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:00.645254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989bce4-1bf4-4739-bad2-0bf8e729c95a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:00.645593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989bce4-1bf4-4739-bad2-0bf8e729c95a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:00.646704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989bce4-1bf4-4739-bad2-0bf8e729c95a admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:00.649327 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1646/3218] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:00.649514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989bce4-1bf4-4739-bad2-0bf8e729c95a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:00.651123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989bce4-1bf4-4739-bad2-0bf8e729c95a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:00.667676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c2204-3ace-4fe4-8762-126b851c14f8 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:00.673533 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1573/3219] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:00.683369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c2204-3ace-4fe4-8762-126b851c14f8 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:00.689088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd561be6-09ed-4a7e-ae7e-01ee3694ff5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:00.690055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd561be6-09ed-4a7e-ae7e-01ee3694ff5c admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:00.690633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd561be6-09ed-4a7e-ae7e-01ee3694ff5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:00.691013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd561be6-09ed-4a7e-ae7e-01ee3694ff5c admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:00.691860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd561be6-09ed-4a7e-ae7e-01ee3694ff5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:00.693596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd561be6-09ed-4a7e-ae7e-01ee3694ff5c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:00.730287 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1647/3220] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:57:00.734568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989bce4-1bf4-4739-bad2-0bf8e729c95a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:00.753924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989bce4-1bf4-4739-bad2-0bf8e729c95a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:00.768023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8066a-ec00-4bb5-b5c8-4b6a23c35aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:00.768209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8066a-ec00-4bb5-b5c8-4b6a23c35aa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:00.768397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8066a-ec00-4bb5-b5c8-4b6a23c35aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:00.770071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8066a-ec00-4bb5-b5c8-4b6a23c35aa2 admin admin] PATH_INFO: `/v3/projects/ff3a0720d242499694d57ffe3b405f21/users/00d15c074f9344b2b2b5df4411df66d1/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:00.845853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8066a-ec00-4bb5-b5c8-4b6a23c35aa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff3a0720d242499694d57ffe3b405f21, user_id=00d15c074f9344b2b2b5df4411df66d1, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:00.856082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8066a-ec00-4bb5-b5c8-4b6a23c35aa2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:00.899157 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1574/3221] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:00 2022] PUT /identity/v3/projects/ff3a0720d242499694d57ffe3b405f21/users/00d15c074f9344b2b2b5df4411df66d1/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:57:00.911893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c85329-d03d-404b-81bd-e916285d67c1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:00.912610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c85329-d03d-404b-81bd-e916285d67c1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:00.913119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c85329-d03d-404b-81bd-e916285d67c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:00.973312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-24c85329-d03d-404b-81bd-e916285d67c1 None None] MFA Rules not processed for user `00d15c074f9344b2b2b5df4411df66d1`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:00.974945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-24c85329-d03d-404b-81bd-e916285d67c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:01.064993 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1648/3222] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:00 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:01.066443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8066a-ec00-4bb5-b5c8-4b6a23c35aa2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:01.181787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8066a-ec00-4bb5-b5c8-4b6a23c35aa2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:01.201471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7cabd-4ec6-4f0f-9cd0-9cefebc41c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:01.205316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7cabd-4ec6-4f0f-9cd0-9cefebc41c6e admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:01.206250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7cabd-4ec6-4f0f-9cd0-9cefebc41c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:01.209006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7cabd-4ec6-4f0f-9cd0-9cefebc41c6e admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:01.210159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a7cabd-4ec6-4f0f-9cd0-9cefebc41c6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:01.210596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c85329-d03d-404b-81bd-e916285d67c1 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:01.219900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a7cabd-4ec6-4f0f-9cd0-9cefebc41c6e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:01.289368 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1575/3223] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:57:01 2022] POST /identity/v3/projects => generated 395 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:01.293419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7cabd-4ec6-4f0f-9cd0-9cefebc41c6e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:01.321810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:01.408151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c85329-d03d-404b-81bd-e916285d67c1 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:01.439022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:01.494601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7cabd-4ec6-4f0f-9cd0-9cefebc41c6e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:01.512785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8cd4e-07e5-43d4-aacb-1b3f625f036f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:01.513487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8cd4e-07e5-43d4-aacb-1b3f625f036f admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:01.513858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8cd4e-07e5-43d4-aacb-1b3f625f036f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:01.514205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8cd4e-07e5-43d4-aacb-1b3f625f036f admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:01.514937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8cd4e-07e5-43d4-aacb-1b3f625f036f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:01.520435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8cd4e-07e5-43d4-aacb-1b3f625f036f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:01.574900 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1576/3224] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:57:01 2022] POST /identity/v3/users => generated 304 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:01.579266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8cd4e-07e5-43d4-aacb-1b3f625f036f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:01.615637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8cd4e-07e5-43d4-aacb-1b3f625f036f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:01.617414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b2812-55b6-45bc-a964-204ba2f916c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:01.618874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b2812-55b6-45bc-a964-204ba2f916c4 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:01.620666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b2812-55b6-45bc-a964-204ba2f916c4 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:01.621211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b2812-55b6-45bc-a964-204ba2f916c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:01.622643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e546f4a-9bb4-41ef-bea3-2e54f6d06a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:01.623398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e546f4a-9bb4-41ef-bea3-2e54f6d06a38 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:01.623659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e546f4a-9bb4-41ef-bea3-2e54f6d06a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:01.623924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e546f4a-9bb4-41ef-bea3-2e54f6d06a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:01.624578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e546f4a-9bb4-41ef-bea3-2e54f6d06a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:01.626215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e546f4a-9bb4-41ef-bea3-2e54f6d06a38 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:01.651589 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1577/3225] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:01.655539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e546f4a-9bb4-41ef-bea3-2e54f6d06a38 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:01.675793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e546f4a-9bb4-41ef-bea3-2e54f6d06a38 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:01.687675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b5f53-6215-451e-8695-29714906527e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:01.688305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b5f53-6215-451e-8695-29714906527e admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:01.688592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b5f53-6215-451e-8695-29714906527e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:01.688853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b5f53-6215-451e-8695-29714906527e admin admin] PATH_INFO: `/v3/projects/bda0a4af0431450f9787ed47eea6aa47/users/95df3199fded4e42980aa91ee239b663/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:01.726615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b5f53-6215-451e-8695-29714906527e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bda0a4af0431450f9787ed47eea6aa47, user_id=95df3199fded4e42980aa91ee239b663, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:01.729500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b5f53-6215-451e-8695-29714906527e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:01.754029 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1578/3226] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:01 2022] PUT /identity/v3/projects/bda0a4af0431450f9787ed47eea6aa47/users/95df3199fded4e42980aa91ee239b663/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:01.758147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b5f53-6215-451e-8695-29714906527e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:01.783378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b2812-55b6-45bc-a964-204ba2f916c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:01.790194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b2812-55b6-45bc-a964-204ba2f916c4 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:01.820861 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1649/3227] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:57:01.900136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b5f53-6215-451e-8695-29714906527e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:01.920243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54794fbb-cc5c-4b4a-9eb6-8191a744a7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:01.923039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54794fbb-cc5c-4b4a-9eb6-8191a744a7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:01.929305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54794fbb-cc5c-4b4a-9eb6-8191a744a7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:01.929601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54794fbb-cc5c-4b4a-9eb6-8191a744a7a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:01.930440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54794fbb-cc5c-4b4a-9eb6-8191a744a7a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:01.932552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54794fbb-cc5c-4b4a-9eb6-8191a744a7a5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:01.954185 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1579/3228] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:57:01.955444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b2812-55b6-45bc-a964-204ba2f916c4 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:02.073888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b2812-55b6-45bc-a964-204ba2f916c4 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:02.082841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93719b27-90ed-498a-a1e8-94f99371877d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:02.084306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93719b27-90ed-498a-a1e8-94f99371877d admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:02.084816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93719b27-90ed-498a-a1e8-94f99371877d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:02.085423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93719b27-90ed-498a-a1e8-94f99371877d admin admin] PATH_INFO: `/v3/projects/bda0a4af0431450f9787ed47eea6aa47/users/95df3199fded4e42980aa91ee239b663/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:02.089070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93719b27-90ed-498a-a1e8-94f99371877d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bda0a4af0431450f9787ed47eea6aa47, user_id=95df3199fded4e42980aa91ee239b663, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:02.092538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93719b27-90ed-498a-a1e8-94f99371877d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:02.137931 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1650/3229] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:01 2022] PUT /identity/v3/projects/bda0a4af0431450f9787ed47eea6aa47/users/95df3199fded4e42980aa91ee239b663/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:02.148822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a518df0-b83f-4f38-b7a4-2c32d1a5dade None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:02.149117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a518df0-b83f-4f38-b7a4-2c32d1a5dade None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:02.149395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a518df0-b83f-4f38-b7a4-2c32d1a5dade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:02.256574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4a518df0-b83f-4f38-b7a4-2c32d1a5dade None None] MFA Rules not processed for user `95df3199fded4e42980aa91ee239b663`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:02.279217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4a518df0-b83f-4f38-b7a4-2c32d1a5dade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:02.485646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:02.554528 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1580/3230] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:02 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:57:02.556879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f848ca-97bb-4eba-b91f-f0aeed7cb60f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:02.557716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f848ca-97bb-4eba-b91f-f0aeed7cb60f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:02.558525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f848ca-97bb-4eba-b91f-f0aeed7cb60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:02.596847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e5f848ca-97bb-4eba-b91f-f0aeed7cb60f None None] MFA Rules not processed for user `00d15c074f9344b2b2b5df4411df66d1`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:02.598358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e5f848ca-97bb-4eba-b91f-f0aeed7cb60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:02.749939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a518df0-b83f-4f38-b7a4-2c32d1a5dade None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:02.751846 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1651/3231] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:02 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:57:02.760376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4a518df0-b83f-4f38-b7a4-2c32d1a5dade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:02.781837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f848ca-97bb-4eba-b91f-f0aeed7cb60f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:02.889049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:02.942275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a518df0-b83f-4f38-b7a4-2c32d1a5dade None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:02.948607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f848ca-97bb-4eba-b91f-f0aeed7cb60f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:02.954681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4932a0-93bd-4c8b-ad0b-570ff133e3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:02.955340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4932a0-93bd-4c8b-ad0b-570ff133e3eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:02.955622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4932a0-93bd-4c8b-ad0b-570ff133e3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:02.963169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4932a0-93bd-4c8b-ad0b-570ff133e3eb admin admin] PATH_INFO: `/v3/users/7d21433c5d434e6f85d17aad6e21f4f0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:02.963169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4932a0-93bd-4c8b-ad0b-570ff133e3eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d21433c5d434e6f85d17aad6e21f4f0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:02.965913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2df4a9-d074-4c26-9d85-679216876ce3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:02.967043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2df4a9-d074-4c26-9d85-679216876ce3 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:02.967043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2df4a9-d074-4c26-9d85-679216876ce3 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:02.967481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2df4a9-d074-4c26-9d85-679216876ce3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:02.969579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4932a0-93bd-4c8b-ad0b-570ff133e3eb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:02.992416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2df4a9-d074-4c26-9d85-679216876ce3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:02.994536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2df4a9-d074-4c26-9d85-679216876ce3 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:03.018306 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1652/3232] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:02 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:57:03.118791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fa4932a0-93bd-4c8b-ad0b-570ff133e3eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d21433c5d434e6f85d17aad6e21f4f0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:03.119665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fa4932a0-93bd-4c8b-ad0b-570ff133e3eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d21433c5d434e6f85d17aad6e21f4f0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:03.121358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fa4932a0-93bd-4c8b-ad0b-570ff133e3eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d21433c5d434e6f85d17aad6e21f4f0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:03.135528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fa4932a0-93bd-4c8b-ad0b-570ff133e3eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d21433c5d434e6f85d17aad6e21f4f0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:03.172553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fa4932a0-93bd-4c8b-ad0b-570ff133e3eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d21433c5d434e6f85d17aad6e21f4f0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:03.188614 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1581/3233] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:57:02 2022] DELETE /identity/v3/users/7d21433c5d434e6f85d17aad6e21f4f0 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:57:03.612528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2df4a9-d074-4c26-9d85-679216876ce3 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:03.636369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2df4a9-d074-4c26-9d85-679216876ce3 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:03.822336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ba70e-9e7f-486c-89b4-266da4412e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:03.828699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ba70e-9e7f-486c-89b4-266da4412e09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:03.828990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ba70e-9e7f-486c-89b4-266da4412e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:03.831791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ba70e-9e7f-486c-89b4-266da4412e09 admin admin] PATH_INFO: `/v3/projects/0edc386b86af4e77bcc8e6663b966c7c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:03.836269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ba70e-9e7f-486c-89b4-266da4412e09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0edc386b86af4e77bcc8e6663b966c7c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:03.845208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ba70e-9e7f-486c-89b4-266da4412e09 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:03.993841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-350ba70e-9e7f-486c-89b4-266da4412e09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0edc386b86af4e77bcc8e6663b966c7c', 'request_id': 'req-350ba70e-9e7f-486c-89b4-266da4412e09'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:04.001836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-350ba70e-9e7f-486c-89b4-266da4412e09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0edc386b86af4e77bcc8e6663b966c7c', 'request_id': 'req-350ba70e-9e7f-486c-89b4-266da4412e09'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:04.016269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-350ba70e-9e7f-486c-89b4-266da4412e09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0edc386b86af4e77bcc8e6663b966c7c', 'request_id': 'req-350ba70e-9e7f-486c-89b4-266da4412e09'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:04.053424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-350ba70e-9e7f-486c-89b4-266da4412e09 admin admin] The token cache is being invalidate because project 0edc386b86af4e77bcc8e6663b966c7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:57:04.053795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-350ba70e-9e7f-486c-89b4-266da4412e09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:04.054587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-350ba70e-9e7f-486c-89b4-266da4412e09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:04.057401 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1653/3234] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:57:03 2022] DELETE /identity/v3/projects/0edc386b86af4e77bcc8e6663b966c7c => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:04.062252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd4c0b-4a8f-468e-b394-cffab2dea5d6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:04.063204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd4c0b-4a8f-468e-b394-cffab2dea5d6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:04.063603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd4c0b-4a8f-468e-b394-cffab2dea5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:04.102514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-85cd4c0b-4a8f-468e-b394-cffab2dea5d6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:04.106022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-85cd4c0b-4a8f-468e-b394-cffab2dea5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:04.258166 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1582/3235] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:04.260399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d999620-8942-4635-a784-5b4b75b7275a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:04.260785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d999620-8942-4635-a784-5b4b75b7275a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:04.261174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d999620-8942-4635-a784-5b4b75b7275a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:04.316320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1d999620-8942-4635-a784-5b4b75b7275a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:04.317937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1d999620-8942-4635-a784-5b4b75b7275a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:04.335195 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1654/3236] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:04.339530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd4c0b-4a8f-468e-b394-cffab2dea5d6 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:04.345364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:04.367235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd4c0b-4a8f-468e-b394-cffab2dea5d6 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:04.380940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda5a4d-235c-47f8-ba63-81a23c170fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:04.381892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda5a4d-235c-47f8-ba63-81a23c170fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:04.382346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda5a4d-235c-47f8-ba63-81a23c170fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:04.383717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda5a4d-235c-47f8-ba63-81a23c170fe5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:04.384554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda5a4d-235c-47f8-ba63-81a23c170fe5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:57:04.387134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda5a4d-235c-47f8-ba63-81a23c170fe5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:04.391483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda5a4d-235c-47f8-ba63-81a23c170fe5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:04.431582 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1583/3237] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:57:04.436529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d999620-8942-4635-a784-5b4b75b7275a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:04.451000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d999620-8942-4635-a784-5b4b75b7275a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:04.462438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78cb7e-8e01-4221-9d3a-5723fd714d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:04.463606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78cb7e-8e01-4221-9d3a-5723fd714d33 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:04.464987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78cb7e-8e01-4221-9d3a-5723fd714d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:04.470961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78cb7e-8e01-4221-9d3a-5723fd714d33 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:04.472353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b78cb7e-8e01-4221-9d3a-5723fd714d33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:04.476881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b78cb7e-8e01-4221-9d3a-5723fd714d33 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:04.544529 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1655/3238] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:57:04 2022] POST /identity/v3/projects => generated 395 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:04.548435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda5a4d-235c-47f8-ba63-81a23c170fe5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:04.571024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda5a4d-235c-47f8-ba63-81a23c170fe5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:04.745178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0142d6-9e56-4696-807f-bfd2743bdabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:04.745866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0142d6-9e56-4696-807f-bfd2743bdabf admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:04.746182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0142d6-9e56-4696-807f-bfd2743bdabf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:04.746445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0142d6-9e56-4696-807f-bfd2743bdabf admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:04.756165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0142d6-9e56-4696-807f-bfd2743bdabf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:04.760305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0142d6-9e56-4696-807f-bfd2743bdabf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:04.822138 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1584/3239] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:57:04 2022] POST /identity/v3/users => generated 298 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:04.827244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78cb7e-8e01-4221-9d3a-5723fd714d33 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:04.874027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78cb7e-8e01-4221-9d3a-5723fd714d33 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:04.906058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd0982-6566-4c26-88d5-4080cd3b8652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:04.906717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd0982-6566-4c26-88d5-4080cd3b8652 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:04.907006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd0982-6566-4c26-88d5-4080cd3b8652 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:04.907285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd0982-6566-4c26-88d5-4080cd3b8652 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:04.910982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd0982-6566-4c26-88d5-4080cd3b8652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:04.916734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd0982-6566-4c26-88d5-4080cd3b8652 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:04.958912 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1656/3240] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:04.962458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0142d6-9e56-4696-807f-bfd2743bdabf admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:04.979968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0142d6-9e56-4696-807f-bfd2743bdabf admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:04.994122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9a9bc-9cb0-46fb-82fc-a9797eb4bb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:04.997173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9a9bc-9cb0-46fb-82fc-a9797eb4bb83 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:04.999235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9a9bc-9cb0-46fb-82fc-a9797eb4bb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:05.000033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9a9bc-9cb0-46fb-82fc-a9797eb4bb83 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:05.002891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9a9bc-9cb0-46fb-82fc-a9797eb4bb83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:05.010198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9a9bc-9cb0-46fb-82fc-a9797eb4bb83 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:05.061779 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1585/3241] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:05.063025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd0982-6566-4c26-88d5-4080cd3b8652 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:05.082200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd0982-6566-4c26-88d5-4080cd3b8652 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:05.092619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f1e87-6c11-4e5c-829c-5185f4e9b746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:05.093906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f1e87-6c11-4e5c-829c-5185f4e9b746 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:05.094422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f1e87-6c11-4e5c-829c-5185f4e9b746 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:05.094940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f1e87-6c11-4e5c-829c-5185f4e9b746 admin admin] PATH_INFO: `/v3/projects/e112b4debcc94e7791f047e7b9d7f06c/users/21c60b2c9b224f4e9750fa3eb6aa0b47/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:05.120825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:05.130303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f1e87-6c11-4e5c-829c-5185f4e9b746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e112b4debcc94e7791f047e7b9d7f06c, user_id=21c60b2c9b224f4e9750fa3eb6aa0b47, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:05.149652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f1e87-6c11-4e5c-829c-5185f4e9b746 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:05.179658 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1657/3242] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:05 2022] PUT /identity/v3/projects/e112b4debcc94e7791f047e7b9d7f06c/users/21c60b2c9b224f4e9750fa3eb6aa0b47/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:05.190240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd020a9-801d-41eb-8bab-bfbc667904ef None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:05.191202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd020a9-801d-41eb-8bab-bfbc667904ef None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:05.191455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd020a9-801d-41eb-8bab-bfbc667904ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:05.249092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-efd020a9-801d-41eb-8bab-bfbc667904ef None None] MFA Rules not processed for user `21c60b2c9b224f4e9750fa3eb6aa0b47`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:05.250776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-efd020a9-801d-41eb-8bab-bfbc667904ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:05.381450 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1586/3243] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:05 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:05.392882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ba585-ed34-4468-a6ea-a41fd34c3690 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:05.393256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ba585-ed34-4468-a6ea-a41fd34c3690 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:05.393442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ba585-ed34-4468-a6ea-a41fd34c3690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:05.471032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ba9ba585-ed34-4468-a6ea-a41fd34c3690 None None] MFA Rules not processed for user `21c60b2c9b224f4e9750fa3eb6aa0b47`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:05.472414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ba9ba585-ed34-4468-a6ea-a41fd34c3690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:05.500374 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1658/3244] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:05 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:05.558738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd020a9-801d-41eb-8bab-bfbc667904ef None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:05.560477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-efd020a9-801d-41eb-8bab-bfbc667904ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:05.741984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:05.792268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd020a9-801d-41eb-8bab-bfbc667904ef None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:05.803247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f750a-1fb9-473a-b16a-9a1e8f357bcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:05.804197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f750a-1fb9-473a-b16a-9a1e8f357bcc service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:05.804763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f750a-1fb9-473a-b16a-9a1e8f357bcc service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:05.805149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f750a-1fb9-473a-b16a-9a1e8f357bcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:05.839797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f750a-1fb9-473a-b16a-9a1e8f357bcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:05.841158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f750a-1fb9-473a-b16a-9a1e8f357bcc service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:05.880563 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1587/3245] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:05 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:05.926304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e738da-e452-4ea4-907a-840f6f3f967e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:05.927449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e738da-e452-4ea4-907a-840f6f3f967e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:05.927921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e738da-e452-4ea4-907a-840f6f3f967e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:05.992470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d5e738da-e452-4ea4-907a-840f6f3f967e None None] MFA Rules not processed for user `21c60b2c9b224f4e9750fa3eb6aa0b47`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:05.995531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d5e738da-e452-4ea4-907a-840f6f3f967e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:06.018709 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1659/3246] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:05 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:06.290030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d90750-5de2-4996-ba8a-be1cf1b0f737 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:06.290523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d90750-5de2-4996-ba8a-be1cf1b0f737 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:06.290757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d90750-5de2-4996-ba8a-be1cf1b0f737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:06.322789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-00d90750-5de2-4996-ba8a-be1cf1b0f737 None None] MFA Rules not processed for user `21c60b2c9b224f4e9750fa3eb6aa0b47`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:06.324399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-00d90750-5de2-4996-ba8a-be1cf1b0f737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:06.341947 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1588/3247] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:06 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:07.293507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dadf8-21ff-467f-8305-8ea73e44467f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:07.293997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dadf8-21ff-467f-8305-8ea73e44467f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:07.294117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dadf8-21ff-467f-8305-8ea73e44467f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:07.341936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-097dadf8-21ff-467f-8305-8ea73e44467f None None] MFA Rules not processed for user `21c60b2c9b224f4e9750fa3eb6aa0b47`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:07.343238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-097dadf8-21ff-467f-8305-8ea73e44467f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:07.364280 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1660/3248] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:07 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:07.948598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eeedb5-19e2-4164-adae-8a8421f8fa7c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:07.949611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eeedb5-19e2-4164-adae-8a8421f8fa7c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:07.950251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eeedb5-19e2-4164-adae-8a8421f8fa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:07.998726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b9eeedb5-19e2-4164-adae-8a8421f8fa7c None None] MFA Rules not processed for user `21c60b2c9b224f4e9750fa3eb6aa0b47`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:08.000250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b9eeedb5-19e2-4164-adae-8a8421f8fa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:08.019751 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1589/3249] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:07 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:08.174573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dadf8-21ff-467f-8305-8ea73e44467f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:08.351774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dadf8-21ff-467f-8305-8ea73e44467f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:08.365182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43487eff-9f5e-4eeb-a306-92c16697d277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:08.365861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43487eff-9f5e-4eeb-a306-92c16697d277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:08.366116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43487eff-9f5e-4eeb-a306-92c16697d277 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:08.368877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43487eff-9f5e-4eeb-a306-92c16697d277 admin admin] PATH_INFO: `/v3/users/21c60b2c9b224f4e9750fa3eb6aa0b47` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:08.373417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43487eff-9f5e-4eeb-a306-92c16697d277 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21c60b2c9b224f4e9750fa3eb6aa0b47)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:08.375774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43487eff-9f5e-4eeb-a306-92c16697d277 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:08.442220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabf021-f259-48f0-8be3-d8df582ad395 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:08.443600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabf021-f259-48f0-8be3-d8df582ad395 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:08.443977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabf021-f259-48f0-8be3-d8df582ad395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:08.513998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-cbabf021-f259-48f0-8be3-d8df582ad395 None None] MFA Rules not processed for user `56b4d78c7c6c450fb5fc1d3ac587fb26`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:08.515459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cbabf021-f259-48f0-8be3-d8df582ad395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:08.527267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-43487eff-9f5e-4eeb-a306-92c16697d277 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21c60b2c9b224f4e9750fa3eb6aa0b47'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:08.551688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-43487eff-9f5e-4eeb-a306-92c16697d277 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21c60b2c9b224f4e9750fa3eb6aa0b47'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:08.631791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-43487eff-9f5e-4eeb-a306-92c16697d277 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21c60b2c9b224f4e9750fa3eb6aa0b47'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:08.640817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-43487eff-9f5e-4eeb-a306-92c16697d277 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21c60b2c9b224f4e9750fa3eb6aa0b47'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:08.664777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-43487eff-9f5e-4eeb-a306-92c16697d277 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21c60b2c9b224f4e9750fa3eb6aa0b47'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:08.675625 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1661/3250] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:57:08 2022] DELETE /identity/v3/users/21c60b2c9b224f4e9750fa3eb6aa0b47 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:08.683072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05598a7f-5bdd-4d8c-8320-d9fdee028ace None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:08.683564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05598a7f-5bdd-4d8c-8320-d9fdee028ace None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:08.684055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05598a7f-5bdd-4d8c-8320-d9fdee028ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:08.748905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-05598a7f-5bdd-4d8c-8320-d9fdee028ace None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:08.750125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-05598a7f-5bdd-4d8c-8320-d9fdee028ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:08.778591 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1590/3251] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:08.819978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:08.896745 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1662/3252] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:09.047905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabf021-f259-48f0-8be3-d8df582ad395 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:09.049677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cbabf021-f259-48f0-8be3-d8df582ad395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:09.175430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabf021-f259-48f0-8be3-d8df582ad395 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:09.274657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68535c9-b35b-41cd-95ae-ba5487ef0f21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:09.275646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68535c9-b35b-41cd-95ae-ba5487ef0f21 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:09.276134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68535c9-b35b-41cd-95ae-ba5487ef0f21 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:09.276612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68535c9-b35b-41cd-95ae-ba5487ef0f21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:09.300753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68535c9-b35b-41cd-95ae-ba5487ef0f21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:09.305820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68535c9-b35b-41cd-95ae-ba5487ef0f21 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:09.332447 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1591/3253] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:10.300705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05598a7f-5bdd-4d8c-8320-d9fdee028ace None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:10.302910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-05598a7f-5bdd-4d8c-8320-d9fdee028ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:10.322432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05598a7f-5bdd-4d8c-8320-d9fdee028ace None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:10.333277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a661e-084c-4593-94ec-cc475cdb6b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:10.336076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a661e-084c-4593-94ec-cc475cdb6b51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:10.336546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a661e-084c-4593-94ec-cc475cdb6b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:10.337390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a661e-084c-4593-94ec-cc475cdb6b51 admin admin] PATH_INFO: `/v3/projects/e112b4debcc94e7791f047e7b9d7f06c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:10.339376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a661e-084c-4593-94ec-cc475cdb6b51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e112b4debcc94e7791f047e7b9d7f06c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:10.344063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a661e-084c-4593-94ec-cc475cdb6b51 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:10.483578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-188a661e-084c-4593-94ec-cc475cdb6b51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e112b4debcc94e7791f047e7b9d7f06c', 'request_id': 'req-188a661e-084c-4593-94ec-cc475cdb6b51'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:10.515121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-188a661e-084c-4593-94ec-cc475cdb6b51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e112b4debcc94e7791f047e7b9d7f06c', 'request_id': 'req-188a661e-084c-4593-94ec-cc475cdb6b51'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:10.533079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-188a661e-084c-4593-94ec-cc475cdb6b51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e112b4debcc94e7791f047e7b9d7f06c', 'request_id': 'req-188a661e-084c-4593-94ec-cc475cdb6b51'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:10.562251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-188a661e-084c-4593-94ec-cc475cdb6b51 admin admin] The token cache is being invalidate because project e112b4debcc94e7791f047e7b9d7f06c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:57:10.563022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-188a661e-084c-4593-94ec-cc475cdb6b51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:10.566075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-188a661e-084c-4593-94ec-cc475cdb6b51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:10.571438 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1663/3254] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:57:10 2022] DELETE /identity/v3/projects/e112b4debcc94e7791f047e7b9d7f06c => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:10.576733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb8318-fa6f-4ae8-93fb-7446ebdd5b8c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:10.577281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb8318-fa6f-4ae8-93fb-7446ebdd5b8c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:10.577402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb8318-fa6f-4ae8-93fb-7446ebdd5b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:10.664407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-09bb8318-fa6f-4ae8-93fb-7446ebdd5b8c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:10.690728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-09bb8318-fa6f-4ae8-93fb-7446ebdd5b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:10.731844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:10.849565 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1592/3255] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:10.857082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e47ad5-8a60-4807-a666-7555f43a206e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:10.857761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e47ad5-8a60-4807-a666-7555f43a206e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:10.858617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e47ad5-8a60-4807-a666-7555f43a206e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:10.944073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-72e47ad5-8a60-4807-a666-7555f43a206e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:10.952951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-72e47ad5-8a60-4807-a666-7555f43a206e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:10.988534 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1664/3256] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:10.991448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb8318-fa6f-4ae8-93fb-7446ebdd5b8c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:11.008525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb8318-fa6f-4ae8-93fb-7446ebdd5b8c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:11.038894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70df425-9368-41f9-a931-ef07b50b4cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:11.056131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70df425-9368-41f9-a931-ef07b50b4cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:11.066627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70df425-9368-41f9-a931-ef07b50b4cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:11.068722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70df425-9368-41f9-a931-ef07b50b4cf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:11.072407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70df425-9368-41f9-a931-ef07b50b4cf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:57:11.072848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70df425-9368-41f9-a931-ef07b50b4cf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:11.076326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70df425-9368-41f9-a931-ef07b50b4cf9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:11.127147 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1593/3257] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:57:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:57:11.129500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e47ad5-8a60-4807-a666-7555f43a206e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:11.145259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e47ad5-8a60-4807-a666-7555f43a206e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:11.154792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2b120-f75b-4cda-a00f-36a34a07f08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:11.156228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2b120-f75b-4cda-a00f-36a34a07f08a admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:11.156611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2b120-f75b-4cda-a00f-36a34a07f08a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:11.157081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2b120-f75b-4cda-a00f-36a34a07f08a admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:11.157901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2b120-f75b-4cda-a00f-36a34a07f08a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:11.160343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2b120-f75b-4cda-a00f-36a34a07f08a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:11.228204 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1665/3258] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:57:11 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:11.231584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70df425-9368-41f9-a931-ef07b50b4cf9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:11.255463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70df425-9368-41f9-a931-ef07b50b4cf9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:11.432461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b52d5-23e4-48da-9ff4-22e50b556280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:11.433101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b52d5-23e4-48da-9ff4-22e50b556280 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:11.445923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b52d5-23e4-48da-9ff4-22e50b556280 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:11.446458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b52d5-23e4-48da-9ff4-22e50b556280 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:11.450083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b52d5-23e4-48da-9ff4-22e50b556280 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:11.452668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b52d5-23e4-48da-9ff4-22e50b556280 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:11.518442 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1594/3259] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:57:11 2022] POST /identity/v3/users => generated 291 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:11.522975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2b120-f75b-4cda-a00f-36a34a07f08a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:11.548896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2b120-f75b-4cda-a00f-36a34a07f08a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:11.570781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1cb1f-61cf-4ada-b96b-32f7531dd9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:11.571580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1cb1f-61cf-4ada-b96b-32f7531dd9b2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:11.571979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1cb1f-61cf-4ada-b96b-32f7531dd9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:11.572162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1cb1f-61cf-4ada-b96b-32f7531dd9b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:11.582066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1cb1f-61cf-4ada-b96b-32f7531dd9b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:11.583674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1cb1f-61cf-4ada-b96b-32f7531dd9b2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:11.611170 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1666/3260] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:11.613651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b52d5-23e4-48da-9ff4-22e50b556280 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:11.634110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b52d5-23e4-48da-9ff4-22e50b556280 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:11.648624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf84349-d330-4bab-8a97-2b461b3b9189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:11.652724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf84349-d330-4bab-8a97-2b461b3b9189 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:11.653016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf84349-d330-4bab-8a97-2b461b3b9189 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:11.653502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf84349-d330-4bab-8a97-2b461b3b9189 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:11.655251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf84349-d330-4bab-8a97-2b461b3b9189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:11.657989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf84349-d330-4bab-8a97-2b461b3b9189 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:11.691478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1cb1f-61cf-4ada-b96b-32f7531dd9b2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:11.693091 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1595/3261] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:11.712119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1cb1f-61cf-4ada-b96b-32f7531dd9b2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:11.727078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b52c5-592b-4ab8-97af-cdb48d351ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:11.727767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b52c5-592b-4ab8-97af-cdb48d351ed5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:11.728058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b52c5-592b-4ab8-97af-cdb48d351ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:11.728328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b52c5-592b-4ab8-97af-cdb48d351ed5 admin admin] PATH_INFO: `/v3/projects/c007c403f0b8414eb5bb14d11897c71a/users/60d08233105349009c9dcf5f25b82229/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:11.767891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b52c5-592b-4ab8-97af-cdb48d351ed5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c007c403f0b8414eb5bb14d11897c71a, user_id=60d08233105349009c9dcf5f25b82229, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:11.771537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b52c5-592b-4ab8-97af-cdb48d351ed5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:11.816613 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1667/3262] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:11 2022] PUT /identity/v3/projects/c007c403f0b8414eb5bb14d11897c71a/users/60d08233105349009c9dcf5f25b82229/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:11.819529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a099d54-2c4e-4c06-a74b-f387c83fed4d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:11.820271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a099d54-2c4e-4c06-a74b-f387c83fed4d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:11.820788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a099d54-2c4e-4c06-a74b-f387c83fed4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:11.887884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7a099d54-2c4e-4c06-a74b-f387c83fed4d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:11.891157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7a099d54-2c4e-4c06-a74b-f387c83fed4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:11.997435 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1596/3263] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:12.012165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b52c5-592b-4ab8-97af-cdb48d351ed5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:12.014690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0c8b52c5-592b-4ab8-97af-cdb48d351ed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:12.171132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b52c5-592b-4ab8-97af-cdb48d351ed5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:12.175825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a967e7-eb3f-484c-96df-b5212fbf2ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:12.176600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a967e7-eb3f-484c-96df-b5212fbf2ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:12.177497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a967e7-eb3f-484c-96df-b5212fbf2ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:12.177879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a967e7-eb3f-484c-96df-b5212fbf2ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:12.200322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a967e7-eb3f-484c-96df-b5212fbf2ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:12.202167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a967e7-eb3f-484c-96df-b5212fbf2ef4 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:12.209660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:12.233210 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1668/3264] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:17.821920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b18082-edcb-418d-bf59-e1ff8b59eb55 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:17.822567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b18082-edcb-418d-bf59-e1ff8b59eb55 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:17.822567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b18082-edcb-418d-bf59-e1ff8b59eb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:17.842609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e9b18082-edcb-418d-bf59-e1ff8b59eb55 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:17.843895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e9b18082-edcb-418d-bf59-e1ff8b59eb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:17.866011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:17.915581 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1597/3265] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 20:57:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:57:17.928571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a967e7-eb3f-484c-96df-b5212fbf2ef4 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:17.930618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a6a967e7-eb3f-484c-96df-b5212fbf2ef4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:18.035181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a967e7-eb3f-484c-96df-b5212fbf2ef4 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:18.044812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80cf42-dcc2-4e68-980d-1be45f1d39ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:18.045805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80cf42-dcc2-4e68-980d-1be45f1d39ef service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:18.046090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80cf42-dcc2-4e68-980d-1be45f1d39ef service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:18.046528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80cf42-dcc2-4e68-980d-1be45f1d39ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:18.056956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80cf42-dcc2-4e68-980d-1be45f1d39ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:18.058828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80cf42-dcc2-4e68-980d-1be45f1d39ef service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:18.069342 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1669/3266] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:57:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 20:57:19.165563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351fe45-cdfc-4940-b2d5-b946a873a7e8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:19.166137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351fe45-cdfc-4940-b2d5-b946a873a7e8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:19.166137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351fe45-cdfc-4940-b2d5-b946a873a7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:19.204455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4351fe45-cdfc-4940-b2d5-b946a873a7e8 None None] MFA Rules not processed for user `1d180849a694494e89843c0c315acd6d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:19.206321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4351fe45-cdfc-4940-b2d5-b946a873a7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:19.284633 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1598/3267] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:23.408133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80cf42-dcc2-4e68-980d-1be45f1d39ef service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:23.409560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cd80cf42-dcc2-4e68-980d-1be45f1d39ef service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:23.453292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80cf42-dcc2-4e68-980d-1be45f1d39ef service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:23.458931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa3e7c-05c7-4697-8ddb-87bd50c2066e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:23.459917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa3e7c-05c7-4697-8ddb-87bd50c2066e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:23.460359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa3e7c-05c7-4697-8ddb-87bd50c2066e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:23.460765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa3e7c-05c7-4697-8ddb-87bd50c2066e admin admin] PATH_INFO: `/v3/users/da040526fe7b490784ea27f2572f0b7b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:23.462065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa3e7c-05c7-4697-8ddb-87bd50c2066e admin admin] RBAC: Authorizing `identity:delete_user(user_id=da040526fe7b490784ea27f2572f0b7b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:23.464339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa3e7c-05c7-4697-8ddb-87bd50c2066e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:23.558518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f9aa3e7c-05c7-4697-8ddb-87bd50c2066e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da040526fe7b490784ea27f2572f0b7b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:23.575814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f9aa3e7c-05c7-4697-8ddb-87bd50c2066e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da040526fe7b490784ea27f2572f0b7b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:23.592021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f9aa3e7c-05c7-4697-8ddb-87bd50c2066e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da040526fe7b490784ea27f2572f0b7b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:23.592771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f9aa3e7c-05c7-4697-8ddb-87bd50c2066e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da040526fe7b490784ea27f2572f0b7b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:23.602416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f9aa3e7c-05c7-4697-8ddb-87bd50c2066e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da040526fe7b490784ea27f2572f0b7b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:23.635929 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1670/3268] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:57:23 2022] DELETE /identity/v3/users/da040526fe7b490784ea27f2572f0b7b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:23.640415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e1222-3eae-49e0-a9b3-780c556f82be None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:23.641508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e1222-3eae-49e0-a9b3-780c556f82be None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:23.641922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e1222-3eae-49e0-a9b3-780c556f82be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:23.667292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a48166-3439-4475-84d6-74cdea3efc64 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:23.667292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a48166-3439-4475-84d6-74cdea3efc64 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:23.667636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a48166-3439-4475-84d6-74cdea3efc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:23.678803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4c1e1222-3eae-49e0-a9b3-780c556f82be None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:23.681161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4c1e1222-3eae-49e0-a9b3-780c556f82be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:23.735222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-75a48166-3439-4475-84d6-74cdea3efc64 None None] MFA Rules not processed for user `60d08233105349009c9dcf5f25b82229`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:23.736553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-75a48166-3439-4475-84d6-74cdea3efc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:23.819475 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1599/3269] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:23.831261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e1222-3eae-49e0-a9b3-780c556f82be None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:23.832908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4c1e1222-3eae-49e0-a9b3-780c556f82be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:23.905661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:23.960203 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1671/3270] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:57:23.962151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed09a60-13a5-4bb5-ad12-30c15817df28 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:23.962750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed09a60-13a5-4bb5-ad12-30c15817df28 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:23.963035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed09a60-13a5-4bb5-ad12-30c15817df28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:23.997801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:24.001123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e1222-3eae-49e0-a9b3-780c556f82be None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:24.002452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3ed09a60-13a5-4bb5-ad12-30c15817df28 None None] MFA Rules not processed for user `60d08233105349009c9dcf5f25b82229`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:24.004247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3ed09a60-13a5-4bb5-ad12-30c15817df28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:24.012821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a0ddf-e3e7-413b-84a5-2729161c0d95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:24.013722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a0ddf-e3e7-413b-84a5-2729161c0d95 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:24.014386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a0ddf-e3e7-413b-84a5-2729161c0d95 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:24.014632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a0ddf-e3e7-413b-84a5-2729161c0d95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:24.017062 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1672/3271] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:24.026174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed09a60-13a5-4bb5-ad12-30c15817df28 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:24.033827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3ed09a60-13a5-4bb5-ad12-30c15817df28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:24.044441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a0ddf-e3e7-413b-84a5-2729161c0d95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:24.046208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a0ddf-e3e7-413b-84a5-2729161c0d95 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:24.064376 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1600/3272] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:24.238241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed09a60-13a5-4bb5-ad12-30c15817df28 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:24.256789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f895572-14d0-45c9-8292-0c2eb01ae5e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:24.257885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f895572-14d0-45c9-8292-0c2eb01ae5e9 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:24.258195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f895572-14d0-45c9-8292-0c2eb01ae5e9 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:24.258560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f895572-14d0-45c9-8292-0c2eb01ae5e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:24.284022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f895572-14d0-45c9-8292-0c2eb01ae5e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:24.293552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f895572-14d0-45c9-8292-0c2eb01ae5e9 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:24.311732 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1673/3273] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:25.121367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a0ddf-e3e7-413b-84a5-2729161c0d95 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:25.123288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-592a0ddf-e3e7-413b-84a5-2729161c0d95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:25.140391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a0ddf-e3e7-413b-84a5-2729161c0d95 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:25.147081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506d24b-4900-4339-8046-b4926930e1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:25.147924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506d24b-4900-4339-8046-b4926930e1d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:25.148310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506d24b-4900-4339-8046-b4926930e1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:25.148696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506d24b-4900-4339-8046-b4926930e1d7 admin admin] PATH_INFO: `/v3/projects/6f9e92bfb756442983da09a9adfe2e88` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:25.149917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506d24b-4900-4339-8046-b4926930e1d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f9e92bfb756442983da09a9adfe2e88)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:25.152244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506d24b-4900-4339-8046-b4926930e1d7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:25.214892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6506d24b-4900-4339-8046-b4926930e1d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f9e92bfb756442983da09a9adfe2e88', 'request_id': 'req-6506d24b-4900-4339-8046-b4926930e1d7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:25.238977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6506d24b-4900-4339-8046-b4926930e1d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f9e92bfb756442983da09a9adfe2e88', 'request_id': 'req-6506d24b-4900-4339-8046-b4926930e1d7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:25.251983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6506d24b-4900-4339-8046-b4926930e1d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f9e92bfb756442983da09a9adfe2e88', 'request_id': 'req-6506d24b-4900-4339-8046-b4926930e1d7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:25.289484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6506d24b-4900-4339-8046-b4926930e1d7 admin admin] The token cache is being invalidate because project 6f9e92bfb756442983da09a9adfe2e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:57:25.289983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6506d24b-4900-4339-8046-b4926930e1d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:25.290868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6506d24b-4900-4339-8046-b4926930e1d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:25.292820 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1601/3274] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:57:25 2022] DELETE /identity/v3/projects/6f9e92bfb756442983da09a9adfe2e88 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:25.296438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f895572-14d0-45c9-8292-0c2eb01ae5e9 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:25.307516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f895572-14d0-45c9-8292-0c2eb01ae5e9 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:25.374814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e760fd2-c642-4d69-9e67-ae3d09fb05cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:25.375875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e760fd2-c642-4d69-9e67-ae3d09fb05cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:25.376333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e760fd2-c642-4d69-9e67-ae3d09fb05cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:25.376950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e760fd2-c642-4d69-9e67-ae3d09fb05cf admin admin] PATH_INFO: `/v3/users/662e58b7335544d69d980be75c13e8aa` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:25.378294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e760fd2-c642-4d69-9e67-ae3d09fb05cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=662e58b7335544d69d980be75c13e8aa)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:25.381465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e760fd2-c642-4d69-9e67-ae3d09fb05cf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:25.477513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4e760fd2-c642-4d69-9e67-ae3d09fb05cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '662e58b7335544d69d980be75c13e8aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:25.481137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:25.495479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4e760fd2-c642-4d69-9e67-ae3d09fb05cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '662e58b7335544d69d980be75c13e8aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:25.529622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4e760fd2-c642-4d69-9e67-ae3d09fb05cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '662e58b7335544d69d980be75c13e8aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:25.530886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4e760fd2-c642-4d69-9e67-ae3d09fb05cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '662e58b7335544d69d980be75c13e8aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:25.552944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4e760fd2-c642-4d69-9e67-ae3d09fb05cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '662e58b7335544d69d980be75c13e8aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:25.565403 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1674/3275] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:57:25 2022] DELETE /identity/v3/users/662e58b7335544d69d980be75c13e8aa => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:57:25.914447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506d24b-4900-4339-8046-b4926930e1d7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:25.916420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6506d24b-4900-4339-8046-b4926930e1d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:25.940250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506d24b-4900-4339-8046-b4926930e1d7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:26.066077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc283f-df4a-40d8-a32c-f9c98c8cf223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:26.066773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc283f-df4a-40d8-a32c-f9c98c8cf223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:26.067169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc283f-df4a-40d8-a32c-f9c98c8cf223 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:26.067445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc283f-df4a-40d8-a32c-f9c98c8cf223 admin admin] PATH_INFO: `/v3/projects/217f0757a4e54c09b7a0f9ae01d575be` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:26.068591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc283f-df4a-40d8-a32c-f9c98c8cf223 admin admin] RBAC: Authorizing `identity:delete_project(project_id=217f0757a4e54c09b7a0f9ae01d575be)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:26.070511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc283f-df4a-40d8-a32c-f9c98c8cf223 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:26.126373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-78dc283f-df4a-40d8-a32c-f9c98c8cf223 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '217f0757a4e54c09b7a0f9ae01d575be', 'request_id': 'req-78dc283f-df4a-40d8-a32c-f9c98c8cf223'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:26.136827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-78dc283f-df4a-40d8-a32c-f9c98c8cf223 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '217f0757a4e54c09b7a0f9ae01d575be', 'request_id': 'req-78dc283f-df4a-40d8-a32c-f9c98c8cf223'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:26.143980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-78dc283f-df4a-40d8-a32c-f9c98c8cf223 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '217f0757a4e54c09b7a0f9ae01d575be', 'request_id': 'req-78dc283f-df4a-40d8-a32c-f9c98c8cf223'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:26.156731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-78dc283f-df4a-40d8-a32c-f9c98c8cf223 admin admin] The token cache is being invalidate because project 217f0757a4e54c09b7a0f9ae01d575be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:57:26.157075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-78dc283f-df4a-40d8-a32c-f9c98c8cf223 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:26.157778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-78dc283f-df4a-40d8-a32c-f9c98c8cf223 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:26.159447 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1602/3276] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:57:25 2022] DELETE /identity/v3/projects/217f0757a4e54c09b7a0f9ae01d575be => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:26.166185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a1541-fddb-4d50-87d4-8c5dca402d59 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:26.166481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a1541-fddb-4d50-87d4-8c5dca402d59 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:26.166949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a1541-fddb-4d50-87d4-8c5dca402d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:26.191515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1f5a1541-fddb-4d50-87d4-8c5dca402d59 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:26.193078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1f5a1541-fddb-4d50-87d4-8c5dca402d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:26.266287 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1675/3277] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:26.275808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ff3f8-3116-4e3e-a16d-36a1fc29775d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:26.276299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ff3f8-3116-4e3e-a16d-36a1fc29775d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:26.276752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ff3f8-3116-4e3e-a16d-36a1fc29775d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:26.307691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:26.323094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c78ff3f8-3116-4e3e-a16d-36a1fc29775d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:26.324519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c78ff3f8-3116-4e3e-a16d-36a1fc29775d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:26.334615 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1603/3278] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:26.338481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a1541-fddb-4d50-87d4-8c5dca402d59 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:26.350473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a1541-fddb-4d50-87d4-8c5dca402d59 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:26.356844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfec82-ca10-4f37-850e-51fae0cdc0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:26.358004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfec82-ca10-4f37-850e-51fae0cdc0a1 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:26.358442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfec82-ca10-4f37-850e-51fae0cdc0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:26.359237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfec82-ca10-4f37-850e-51fae0cdc0a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:26.360513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dfec82-ca10-4f37-850e-51fae0cdc0a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:57:26.360986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dfec82-ca10-4f37-850e-51fae0cdc0a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:26.363489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dfec82-ca10-4f37-850e-51fae0cdc0a1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:26.376982 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1676/3279] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:57:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:57:26.381864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ff3f8-3116-4e3e-a16d-36a1fc29775d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:26.396368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ff3f8-3116-4e3e-a16d-36a1fc29775d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:26.401851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e69ac1-d766-4812-a3e8-3573c5dabe4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:26.402575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e69ac1-d766-4812-a3e8-3573c5dabe4c admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:26.403017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e69ac1-d766-4812-a3e8-3573c5dabe4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:26.403294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e69ac1-d766-4812-a3e8-3573c5dabe4c admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:26.403932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e69ac1-d766-4812-a3e8-3573c5dabe4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:26.405845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e69ac1-d766-4812-a3e8-3573c5dabe4c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:26.465849 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1604/3280] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:57:26 2022] POST /identity/v3/projects => generated 377 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:26.470234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfec82-ca10-4f37-850e-51fae0cdc0a1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:26.489302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfec82-ca10-4f37-850e-51fae0cdc0a1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:26.615182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb4802-14a5-4506-8e46-c535767d932a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:26.615921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb4802-14a5-4506-8e46-c535767d932a admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:26.616207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb4802-14a5-4506-8e46-c535767d932a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:26.616676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb4802-14a5-4506-8e46-c535767d932a admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:26.617382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb4802-14a5-4506-8e46-c535767d932a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:26.622265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb4802-14a5-4506-8e46-c535767d932a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:26.710938 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1677/3281] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:57:26 2022] POST /identity/v3/users => generated 289 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:26.711805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e69ac1-d766-4812-a3e8-3573c5dabe4c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:26.728473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e69ac1-d766-4812-a3e8-3573c5dabe4c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:26.741296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60e2d7-aefb-4be2-a8e6-58bb208a2b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:26.741980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60e2d7-aefb-4be2-a8e6-58bb208a2b5c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:26.742264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60e2d7-aefb-4be2-a8e6-58bb208a2b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:26.742573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60e2d7-aefb-4be2-a8e6-58bb208a2b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:26.746671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60e2d7-aefb-4be2-a8e6-58bb208a2b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:26.755697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60e2d7-aefb-4be2-a8e6-58bb208a2b5c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:26.806052 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1605/3282] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:26.810335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb4802-14a5-4506-8e46-c535767d932a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:26.837376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb4802-14a5-4506-8e46-c535767d932a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:26.856642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eceb879-c894-455a-8b3a-cb8405f6b9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:26.857393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eceb879-c894-455a-8b3a-cb8405f6b9d2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:26.857776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eceb879-c894-455a-8b3a-cb8405f6b9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:26.858300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eceb879-c894-455a-8b3a-cb8405f6b9d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:26.861467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eceb879-c894-455a-8b3a-cb8405f6b9d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:26.870856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eceb879-c894-455a-8b3a-cb8405f6b9d2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:26.919616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60e2d7-aefb-4be2-a8e6-58bb208a2b5c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:26.920291 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1678/3283] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:26.933039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60e2d7-aefb-4be2-a8e6-58bb208a2b5c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:26.952743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29daac3-f666-4ccf-9d79-f7b3308e40ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:26.953438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29daac3-f666-4ccf-9d79-f7b3308e40ec admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:26.953718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29daac3-f666-4ccf-9d79-f7b3308e40ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:26.954280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29daac3-f666-4ccf-9d79-f7b3308e40ec admin admin] PATH_INFO: `/v3/projects/4b9c23f1ff084a4e98e24993dd923c64/users/b5c4d1842a2b41ab823715823cbe075d/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:26.979373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29daac3-f666-4ccf-9d79-f7b3308e40ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b9c23f1ff084a4e98e24993dd923c64, user_id=b5c4d1842a2b41ab823715823cbe075d, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:26.985289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29daac3-f666-4ccf-9d79-f7b3308e40ec admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:27.024073 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1606/3284] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:26 2022] PUT /identity/v3/projects/4b9c23f1ff084a4e98e24993dd923c64/users/b5c4d1842a2b41ab823715823cbe075d/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:27.038194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67214b-6b5f-429b-8c9c-12f71a628965 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:27.038684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67214b-6b5f-429b-8c9c-12f71a628965 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:27.039500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67214b-6b5f-429b-8c9c-12f71a628965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:27.112557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5b67214b-6b5f-429b-8c9c-12f71a628965 None None] MFA Rules not processed for user `b5c4d1842a2b41ab823715823cbe075d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:27.116691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5b67214b-6b5f-429b-8c9c-12f71a628965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:27.185133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:27.282773 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1679/3285] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:27.288339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29daac3-f666-4ccf-9d79-f7b3308e40ec admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:27.423812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29daac3-f666-4ccf-9d79-f7b3308e40ec admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:27.448274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854f4b6-a665-4a08-89bd-34305cbe61de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:27.449073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854f4b6-a665-4a08-89bd-34305cbe61de admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:27.449499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854f4b6-a665-4a08-89bd-34305cbe61de admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:27.449931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854f4b6-a665-4a08-89bd-34305cbe61de admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:27.451897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854f4b6-a665-4a08-89bd-34305cbe61de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:27.456818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854f4b6-a665-4a08-89bd-34305cbe61de admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:27.515785 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1607/3286] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:57:27 2022] POST /identity/v3/projects => generated 379 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:27.519733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67214b-6b5f-429b-8c9c-12f71a628965 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:27.536102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67214b-6b5f-429b-8c9c-12f71a628965 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:27.588259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854f4b6-a665-4a08-89bd-34305cbe61de admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:27.590091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b854f4b6-a665-4a08-89bd-34305cbe61de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:27.659338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3cdf2-0206-4620-9a45-043ffb0099b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:27.660796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3cdf2-0206-4620-9a45-043ffb0099b0 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:27.661729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3cdf2-0206-4620-9a45-043ffb0099b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:27.662030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3cdf2-0206-4620-9a45-043ffb0099b0 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:27.666600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3cdf2-0206-4620-9a45-043ffb0099b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:27.668893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3cdf2-0206-4620-9a45-043ffb0099b0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:27.741203 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1680/3287] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:57:27 2022] POST /identity/v3/users => generated 296 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:27.741741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854f4b6-a665-4a08-89bd-34305cbe61de admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:27.742578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3cdf2-0206-4620-9a45-043ffb0099b0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:27.754278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23430990-8e12-44dc-bd66-36b9f57d2602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:27.754969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23430990-8e12-44dc-bd66-36b9f57d2602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:27.755253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23430990-8e12-44dc-bd66-36b9f57d2602 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:27.755532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23430990-8e12-44dc-bd66-36b9f57d2602 admin admin] PATH_INFO: `/v3/users/00d15c074f9344b2b2b5df4411df66d1` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:27.759609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23430990-8e12-44dc-bd66-36b9f57d2602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00d15c074f9344b2b2b5df4411df66d1)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:27.762185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23430990-8e12-44dc-bd66-36b9f57d2602 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:27.778384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3cdf2-0206-4620-9a45-043ffb0099b0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:27.792780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7563208-1be5-4263-b87f-38b6bdbb32d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:27.793617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7563208-1be5-4263-b87f-38b6bdbb32d0 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:27.795695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7563208-1be5-4263-b87f-38b6bdbb32d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:27.797278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7563208-1be5-4263-b87f-38b6bdbb32d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:27.799251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7563208-1be5-4263-b87f-38b6bdbb32d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:27.804111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7563208-1be5-4263-b87f-38b6bdbb32d0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:27.845031 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1681/3288] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:27.848655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7563208-1be5-4263-b87f-38b6bdbb32d0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:27.869110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:27.871284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7563208-1be5-4263-b87f-38b6bdbb32d0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:27.889573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464c4d7-fd66-4fd3-91f9-c2058a75cba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:27.892506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464c4d7-fd66-4fd3-91f9-c2058a75cba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:27.893628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464c4d7-fd66-4fd3-91f9-c2058a75cba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:27.893948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464c4d7-fd66-4fd3-91f9-c2058a75cba8 admin admin] PATH_INFO: `/v3/projects/6c6517b87d044904b21ab9f8104211ea/users/e0497ba03f2541b2bd1c724d473fe3e7/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:27.907229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-23430990-8e12-44dc-bd66-36b9f57d2602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00d15c074f9344b2b2b5df4411df66d1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:27.908810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-23430990-8e12-44dc-bd66-36b9f57d2602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00d15c074f9344b2b2b5df4411df66d1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:27.909463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-23430990-8e12-44dc-bd66-36b9f57d2602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00d15c074f9344b2b2b5df4411df66d1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:27.923934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-23430990-8e12-44dc-bd66-36b9f57d2602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00d15c074f9344b2b2b5df4411df66d1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:27.924517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464c4d7-fd66-4fd3-91f9-c2058a75cba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c6517b87d044904b21ab9f8104211ea, user_id=e0497ba03f2541b2bd1c724d473fe3e7, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:27.928532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464c4d7-fd66-4fd3-91f9-c2058a75cba8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:27.960720 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1682/3289] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:27 2022] PUT /identity/v3/projects/6c6517b87d044904b21ab9f8104211ea/users/e0497ba03f2541b2bd1c724d473fe3e7/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:27.960923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-23430990-8e12-44dc-bd66-36b9f57d2602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00d15c074f9344b2b2b5df4411df66d1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:27.962207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464c4d7-fd66-4fd3-91f9-c2058a75cba8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:27.970184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9464c4d7-fd66-4fd3-91f9-c2058a75cba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:27.985548 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1608/3290] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:57:27 2022] DELETE /identity/v3/users/00d15c074f9344b2b2b5df4411df66d1 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:27.990268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593d31c-8302-4d9a-a346-4e048b6dc883 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:27.990591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593d31c-8302-4d9a-a346-4e048b6dc883 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:27.990875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593d31c-8302-4d9a-a346-4e048b6dc883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:28.045017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a593d31c-8302-4d9a-a346-4e048b6dc883 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:28.046208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a593d31c-8302-4d9a-a346-4e048b6dc883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:28.205586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464c4d7-fd66-4fd3-91f9-c2058a75cba8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:28.206467 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1609/3291] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:28.212687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05eecd2-5b61-435b-96a4-2b178cf3a349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:28.213315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05eecd2-5b61-435b-96a4-2b178cf3a349 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:28.213600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05eecd2-5b61-435b-96a4-2b178cf3a349 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:28.213968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05eecd2-5b61-435b-96a4-2b178cf3a349 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:28.214653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05eecd2-5b61-435b-96a4-2b178cf3a349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:28.218717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05eecd2-5b61-435b-96a4-2b178cf3a349 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:28.238669 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1683/3292] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:28.240811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593d31c-8302-4d9a-a346-4e048b6dc883 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:28.253460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593d31c-8302-4d9a-a346-4e048b6dc883 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:28.269254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5140d-e90b-405f-a6fc-a25b70c7c2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:28.270769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5140d-e90b-405f-a6fc-a25b70c7c2bf admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:28.271115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5140d-e90b-405f-a6fc-a25b70c7c2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:28.271396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5140d-e90b-405f-a6fc-a25b70c7c2bf admin admin] PATH_INFO: `/v3/projects/6c6517b87d044904b21ab9f8104211ea/users/e0497ba03f2541b2bd1c724d473fe3e7/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:28.273478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5140d-e90b-405f-a6fc-a25b70c7c2bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c6517b87d044904b21ab9f8104211ea, user_id=e0497ba03f2541b2bd1c724d473fe3e7, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:28.277200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5140d-e90b-405f-a6fc-a25b70c7c2bf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:28.303787 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1610/3293] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:28 2022] PUT /identity/v3/projects/6c6517b87d044904b21ab9f8104211ea/users/e0497ba03f2541b2bd1c724d473fe3e7/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:28.315536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1cd06-5405-4ccb-b8a2-d04a75ee19ad None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:28.317333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1cd06-5405-4ccb-b8a2-d04a75ee19ad None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:28.317824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1cd06-5405-4ccb-b8a2-d04a75ee19ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:28.321028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5140d-e90b-405f-a6fc-a25b70c7c2bf admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:28.325697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fac5140d-e90b-405f-a6fc-a25b70c7c2bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:28.400204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:28.417761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-87e1cd06-5405-4ccb-b8a2-d04a75ee19ad None None] MFA Rules not processed for user `e0497ba03f2541b2bd1c724d473fe3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:28.421578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-87e1cd06-5405-4ccb-b8a2-d04a75ee19ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:28.515806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5140d-e90b-405f-a6fc-a25b70c7c2bf admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:28.527424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbec9b-ac93-4cd7-a0d8-01eb0d40f2a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:28.528115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbec9b-ac93-4cd7-a0d8-01eb0d40f2a8 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:28.528482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbec9b-ac93-4cd7-a0d8-01eb0d40f2a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:28.533736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbec9b-ac93-4cd7-a0d8-01eb0d40f2a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:28.659342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:28.669670 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1684/3294] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:28.703490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbec9b-ac93-4cd7-a0d8-01eb0d40f2a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:28.712425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbec9b-ac93-4cd7-a0d8-01eb0d40f2a8 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:28.734717 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1611/3295] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:29.036361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1cd06-5405-4ccb-b8a2-d04a75ee19ad None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:29.038591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-87e1cd06-5405-4ccb-b8a2-d04a75ee19ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:29.153136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1cd06-5405-4ccb-b8a2-d04a75ee19ad None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:29.165612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24b590-fd7a-4091-99f9-b09ade1ebe1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:29.167260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24b590-fd7a-4091-99f9-b09ade1ebe1e service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:29.167390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24b590-fd7a-4091-99f9-b09ade1ebe1e service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:29.170466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24b590-fd7a-4091-99f9-b09ade1ebe1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:29.183136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24b590-fd7a-4091-99f9-b09ade1ebe1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:29.194638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24b590-fd7a-4091-99f9-b09ade1ebe1e service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:29.209651 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1685/3296] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:29 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:29.458933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbec9b-ac93-4cd7-a0d8-01eb0d40f2a8 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:29.460670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ebcbec9b-ac93-4cd7-a0d8-01eb0d40f2a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:29.498617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbec9b-ac93-4cd7-a0d8-01eb0d40f2a8 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:29.515061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c981de-18a3-4ec2-8a1b-ee69bc9f031a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:29.515893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c981de-18a3-4ec2-8a1b-ee69bc9f031a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:29.516311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c981de-18a3-4ec2-8a1b-ee69bc9f031a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:29.517511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c981de-18a3-4ec2-8a1b-ee69bc9f031a admin admin] PATH_INFO: `/v3/projects/ff3a0720d242499694d57ffe3b405f21` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:29.520459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c981de-18a3-4ec2-8a1b-ee69bc9f031a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff3a0720d242499694d57ffe3b405f21)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:29.524545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c981de-18a3-4ec2-8a1b-ee69bc9f031a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:29.645096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24b590-fd7a-4091-99f9-b09ade1ebe1e service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:29.655586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b8c981de-18a3-4ec2-8a1b-ee69bc9f031a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff3a0720d242499694d57ffe3b405f21', 'request_id': 'req-b8c981de-18a3-4ec2-8a1b-ee69bc9f031a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:29.672499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b8c981de-18a3-4ec2-8a1b-ee69bc9f031a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff3a0720d242499694d57ffe3b405f21', 'request_id': 'req-b8c981de-18a3-4ec2-8a1b-ee69bc9f031a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:29.674523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24b590-fd7a-4091-99f9-b09ade1ebe1e service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:29.699777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b8c981de-18a3-4ec2-8a1b-ee69bc9f031a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff3a0720d242499694d57ffe3b405f21', 'request_id': 'req-b8c981de-18a3-4ec2-8a1b-ee69bc9f031a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:29.733323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b8c981de-18a3-4ec2-8a1b-ee69bc9f031a admin admin] The token cache is being invalidate because project ff3a0720d242499694d57ffe3b405f21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:57:29.734023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b8c981de-18a3-4ec2-8a1b-ee69bc9f031a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:29.735002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b8c981de-18a3-4ec2-8a1b-ee69bc9f031a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:29.738178 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1612/3297] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:57:29 2022] DELETE /identity/v3/projects/ff3a0720d242499694d57ffe3b405f21 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:29.743318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c981de-18a3-4ec2-8a1b-ee69bc9f031a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:29.748025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b8c981de-18a3-4ec2-8a1b-ee69bc9f031a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:29.784777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da871066-4897-40a8-8380-6868b8e31f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:29.786175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da871066-4897-40a8-8380-6868b8e31f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:29.786902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da871066-4897-40a8-8380-6868b8e31f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:29.787178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da871066-4897-40a8-8380-6868b8e31f6c admin admin] PATH_INFO: `/v3/users/b5c4d1842a2b41ab823715823cbe075d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:29.788414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da871066-4897-40a8-8380-6868b8e31f6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5c4d1842a2b41ab823715823cbe075d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:29.791437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da871066-4897-40a8-8380-6868b8e31f6c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:29.851395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:29.919520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-da871066-4897-40a8-8380-6868b8e31f6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5c4d1842a2b41ab823715823cbe075d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:29.938210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c981de-18a3-4ec2-8a1b-ee69bc9f031a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:29.948266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-da871066-4897-40a8-8380-6868b8e31f6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5c4d1842a2b41ab823715823cbe075d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:29.948657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840704e-bd55-4292-b632-6ca149a1f364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:29.949989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840704e-bd55-4292-b632-6ca149a1f364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:29.950419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840704e-bd55-4292-b632-6ca149a1f364 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:29.950827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840704e-bd55-4292-b632-6ca149a1f364 admin admin] PATH_INFO: `/v3/users/95df3199fded4e42980aa91ee239b663` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:29.952389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840704e-bd55-4292-b632-6ca149a1f364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95df3199fded4e42980aa91ee239b663)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:29.955362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840704e-bd55-4292-b632-6ca149a1f364 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:30.004319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-da871066-4897-40a8-8380-6868b8e31f6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5c4d1842a2b41ab823715823cbe075d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:30.004974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-da871066-4897-40a8-8380-6868b8e31f6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5c4d1842a2b41ab823715823cbe075d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:30.028539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-da871066-4897-40a8-8380-6868b8e31f6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5c4d1842a2b41ab823715823cbe075d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:30.043803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0840704e-bd55-4292-b632-6ca149a1f364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95df3199fded4e42980aa91ee239b663'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:30.047207 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1686/3298] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:57:29 2022] DELETE /identity/v3/users/b5c4d1842a2b41ab823715823cbe075d => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:57:30.048775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0840704e-bd55-4292-b632-6ca149a1f364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95df3199fded4e42980aa91ee239b663'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:30.049513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83bd72-5d9b-4c1c-9c78-4148bbd4c70f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:30.049955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83bd72-5d9b-4c1c-9c78-4148bbd4c70f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:30.050236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83bd72-5d9b-4c1c-9c78-4148bbd4c70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:30.050606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0840704e-bd55-4292-b632-6ca149a1f364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95df3199fded4e42980aa91ee239b663'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:30.063934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0840704e-bd55-4292-b632-6ca149a1f364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95df3199fded4e42980aa91ee239b663'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:30.102527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5c83bd72-5d9b-4c1c-9c78-4148bbd4c70f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:30.106893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5c83bd72-5d9b-4c1c-9c78-4148bbd4c70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:30.120936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:30.128709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0840704e-bd55-4292-b632-6ca149a1f364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95df3199fded4e42980aa91ee239b663'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:30.192672 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1613/3299] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:57:29 2022] DELETE /identity/v3/users/95df3199fded4e42980aa91ee239b663 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:30.257714 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1687/3300] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:30.280340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840704e-bd55-4292-b632-6ca149a1f364 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:30.282471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0840704e-bd55-4292-b632-6ca149a1f364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:30.407127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840704e-bd55-4292-b632-6ca149a1f364 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:30.413268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2352a6a-cd62-43ca-a5b8-02ffb099bd81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:30.414726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2352a6a-cd62-43ca-a5b8-02ffb099bd81 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:30.415114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2352a6a-cd62-43ca-a5b8-02ffb099bd81 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:30.415489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2352a6a-cd62-43ca-a5b8-02ffb099bd81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:30.589049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2352a6a-cd62-43ca-a5b8-02ffb099bd81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:30.592165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2352a6a-cd62-43ca-a5b8-02ffb099bd81 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:30.608103 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1614/3301] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:31.239363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83bd72-5d9b-4c1c-9c78-4148bbd4c70f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:31.240863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5c83bd72-5d9b-4c1c-9c78-4148bbd4c70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:31.266289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83bd72-5d9b-4c1c-9c78-4148bbd4c70f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:31.284695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245d24a-c91f-495b-aef6-92f38f4d6669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:31.286122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245d24a-c91f-495b-aef6-92f38f4d6669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:31.286465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245d24a-c91f-495b-aef6-92f38f4d6669 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:31.286766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245d24a-c91f-495b-aef6-92f38f4d6669 admin admin] PATH_INFO: `/v3/projects/bda0a4af0431450f9787ed47eea6aa47` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:31.288098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245d24a-c91f-495b-aef6-92f38f4d6669 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bda0a4af0431450f9787ed47eea6aa47)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:31.290030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245d24a-c91f-495b-aef6-92f38f4d6669 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:31.392581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2245d24a-c91f-495b-aef6-92f38f4d6669 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bda0a4af0431450f9787ed47eea6aa47', 'request_id': 'req-2245d24a-c91f-495b-aef6-92f38f4d6669'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:31.400959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2245d24a-c91f-495b-aef6-92f38f4d6669 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bda0a4af0431450f9787ed47eea6aa47', 'request_id': 'req-2245d24a-c91f-495b-aef6-92f38f4d6669'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:31.408446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2245d24a-c91f-495b-aef6-92f38f4d6669 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bda0a4af0431450f9787ed47eea6aa47', 'request_id': 'req-2245d24a-c91f-495b-aef6-92f38f4d6669'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:31.427699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2245d24a-c91f-495b-aef6-92f38f4d6669 admin admin] The token cache is being invalidate because project bda0a4af0431450f9787ed47eea6aa47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:57:31.428137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2245d24a-c91f-495b-aef6-92f38f4d6669 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:31.428866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2245d24a-c91f-495b-aef6-92f38f4d6669 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:31.435691 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1688/3302] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:57:31 2022] DELETE /identity/v3/projects/bda0a4af0431450f9787ed47eea6aa47 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:31.437796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33044e4-a1a9-4b1b-a2ba-92213b0474a2 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:31.438233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33044e4-a1a9-4b1b-a2ba-92213b0474a2 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:31.438669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33044e4-a1a9-4b1b-a2ba-92213b0474a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:31.490518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e33044e4-a1a9-4b1b-a2ba-92213b0474a2 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:31.491997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e33044e4-a1a9-4b1b-a2ba-92213b0474a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:31.567980 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1615/3303] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:31.570148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f1bc4-7141-49c2-b30e-913b37200c1c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:31.570469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f1bc4-7141-49c2-b30e-913b37200c1c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:31.570769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f1bc4-7141-49c2-b30e-913b37200c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:31.577334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33044e4-a1a9-4b1b-a2ba-92213b0474a2 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:31.581082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e33044e4-a1a9-4b1b-a2ba-92213b0474a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:31.604902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33044e4-a1a9-4b1b-a2ba-92213b0474a2 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:31.626835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-711f1bc4-7141-49c2-b30e-913b37200c1c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:31.628049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-711f1bc4-7141-49c2-b30e-913b37200c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:31.634885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d1b96-617d-40ce-8a65-a998bd89f01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:31.635547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d1b96-617d-40ce-8a65-a998bd89f01a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:31.635838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d1b96-617d-40ce-8a65-a998bd89f01a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:31.636156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d1b96-617d-40ce-8a65-a998bd89f01a admin admin] PATH_INFO: `/v3/projects/4b9c23f1ff084a4e98e24993dd923c64` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:31.639801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d1b96-617d-40ce-8a65-a998bd89f01a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b9c23f1ff084a4e98e24993dd923c64)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:31.646243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d1b96-617d-40ce-8a65-a998bd89f01a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:31.670042 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1689/3304] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:31.675970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f1bc4-7141-49c2-b30e-913b37200c1c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:31.705261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f1bc4-7141-49c2-b30e-913b37200c1c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:31.737865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcba1ef-ce55-4117-b20f-94fc89440ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:31.738732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcba1ef-ce55-4117-b20f-94fc89440ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:31.739081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcba1ef-ce55-4117-b20f-94fc89440ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:31.739362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcba1ef-ce55-4117-b20f-94fc89440ad7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:31.740050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcba1ef-ce55-4117-b20f-94fc89440ad7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:57:31.740362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcba1ef-ce55-4117-b20f-94fc89440ad7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:31.741969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcba1ef-ce55-4117-b20f-94fc89440ad7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:31.788338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:31.798477 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1690/3305] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:57:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:57:31.808054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcba1ef-ce55-4117-b20f-94fc89440ad7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:31.809597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8e3d1b96-617d-40ce-8a65-a998bd89f01a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b9c23f1ff084a4e98e24993dd923c64', 'request_id': 'req-8e3d1b96-617d-40ce-8a65-a998bd89f01a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:31.825754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8e3d1b96-617d-40ce-8a65-a998bd89f01a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b9c23f1ff084a4e98e24993dd923c64', 'request_id': 'req-8e3d1b96-617d-40ce-8a65-a998bd89f01a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:31.835046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8e3d1b96-617d-40ce-8a65-a998bd89f01a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b9c23f1ff084a4e98e24993dd923c64', 'request_id': 'req-8e3d1b96-617d-40ce-8a65-a998bd89f01a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:31.843520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcba1ef-ce55-4117-b20f-94fc89440ad7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:31.857474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8e3d1b96-617d-40ce-8a65-a998bd89f01a admin admin] The token cache is being invalidate because project 4b9c23f1ff084a4e98e24993dd923c64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:57:31.857856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8e3d1b96-617d-40ce-8a65-a998bd89f01a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:31.858905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8e3d1b96-617d-40ce-8a65-a998bd89f01a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:31.861643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:31.862455 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1616/3306] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:57:31 2022] DELETE /identity/v3/projects/4b9c23f1ff084a4e98e24993dd923c64 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:57:31.865580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d1b96-617d-40ce-8a65-a998bd89f01a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:31.867751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8e3d1b96-617d-40ce-8a65-a998bd89f01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:31.993665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2784b-f5f3-43a1-b482-da88b67dae17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:31.994337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2784b-f5f3-43a1-b482-da88b67dae17 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:31.994657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2784b-f5f3-43a1-b482-da88b67dae17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:31.994932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2784b-f5f3-43a1-b482-da88b67dae17 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:31.995571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2784b-f5f3-43a1-b482-da88b67dae17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:32.005554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2784b-f5f3-43a1-b482-da88b67dae17 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:32.020247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d1b96-617d-40ce-8a65-a998bd89f01a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:32.027555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e705b5f-c619-4b02-adfc-fb0115894629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:32.028201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e705b5f-c619-4b02-adfc-fb0115894629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:32.028528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e705b5f-c619-4b02-adfc-fb0115894629 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:32.028807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e705b5f-c619-4b02-adfc-fb0115894629 admin admin] PATH_INFO: `/v3/users/e0497ba03f2541b2bd1c724d473fe3e7` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:32.030155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e705b5f-c619-4b02-adfc-fb0115894629 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0497ba03f2541b2bd1c724d473fe3e7)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:32.032937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e705b5f-c619-4b02-adfc-fb0115894629 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:32.086855 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1691/3307] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:57:31 2022] POST /identity/v3/projects => generated 397 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:32.088897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2784b-f5f3-43a1-b482-da88b67dae17 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:32.097331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-78b2784b-f5f3-43a1-b482-da88b67dae17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:32.124290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0e705b5f-c619-4b02-adfc-fb0115894629 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0497ba03f2541b2bd1c724d473fe3e7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:32.125455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0e705b5f-c619-4b02-adfc-fb0115894629 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0497ba03f2541b2bd1c724d473fe3e7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:32.127481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0e705b5f-c619-4b02-adfc-fb0115894629 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0497ba03f2541b2bd1c724d473fe3e7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:32.144375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0e705b5f-c619-4b02-adfc-fb0115894629 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0497ba03f2541b2bd1c724d473fe3e7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:32.164319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0e705b5f-c619-4b02-adfc-fb0115894629 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0497ba03f2541b2bd1c724d473fe3e7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:32.185866 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1617/3308] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:57:31 2022] DELETE /identity/v3/users/e0497ba03f2541b2bd1c724d473fe3e7 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:32.185866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e705b5f-c619-4b02-adfc-fb0115894629 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:32.187212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0e705b5f-c619-4b02-adfc-fb0115894629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:32.213141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2784b-f5f3-43a1-b482-da88b67dae17 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:32.285139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:32.356683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58c5c5-1bc7-4510-a112-f9dd649d5bc5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:32.357344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58c5c5-1bc7-4510-a112-f9dd649d5bc5 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:32.357580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58c5c5-1bc7-4510-a112-f9dd649d5bc5 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:32.357864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58c5c5-1bc7-4510-a112-f9dd649d5bc5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:32.359820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e705b5f-c619-4b02-adfc-fb0115894629 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:32.359934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9b58c5c5-1bc7-4510-a112-f9dd649d5bc5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:32.372544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58c5c5-1bc7-4510-a112-f9dd649d5bc5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:32.374166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58c5c5-1bc7-4510-a112-f9dd649d5bc5 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:32.375231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e84ae4-9670-4acd-855b-bc7ae5748524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:32.375869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e84ae4-9670-4acd-855b-bc7ae5748524 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:32.376151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e84ae4-9670-4acd-855b-bc7ae5748524 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:32.376422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e84ae4-9670-4acd-855b-bc7ae5748524 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:32.377055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e84ae4-9670-4acd-855b-bc7ae5748524 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:32.380159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e84ae4-9670-4acd-855b-bc7ae5748524 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:32.386138 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1692/3309] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:32.443012 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1618/3310] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:57:32 2022] POST /identity/v3/users => generated 299 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:32.444203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58c5c5-1bc7-4510-a112-f9dd649d5bc5 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:32.464118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58c5c5-1bc7-4510-a112-f9dd649d5bc5 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:32.470477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00533010-5743-4db7-ae37-341a6c93f723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:32.472154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00533010-5743-4db7-ae37-341a6c93f723 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:32.472154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00533010-5743-4db7-ae37-341a6c93f723 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:32.472154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00533010-5743-4db7-ae37-341a6c93f723 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:32.472442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00533010-5743-4db7-ae37-341a6c93f723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:32.474298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00533010-5743-4db7-ae37-341a6c93f723 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:32.490950 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1693/3311] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:32.492262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e84ae4-9670-4acd-855b-bc7ae5748524 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:32.505438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e84ae4-9670-4acd-855b-bc7ae5748524 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:32.512325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fafa65-9802-4d26-8b37-d9a529ec65b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:32.513080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fafa65-9802-4d26-8b37-d9a529ec65b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:32.513457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fafa65-9802-4d26-8b37-d9a529ec65b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:32.513844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fafa65-9802-4d26-8b37-d9a529ec65b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:32.514632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fafa65-9802-4d26-8b37-d9a529ec65b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:32.516343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fafa65-9802-4d26-8b37-d9a529ec65b6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:32.529279 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1619/3312] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:32.533332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00533010-5743-4db7-ae37-341a6c93f723 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:32.553655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00533010-5743-4db7-ae37-341a6c93f723 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:32.560145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a23a8-e173-436b-9192-d26594570b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:32.561374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a23a8-e173-436b-9192-d26594570b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:32.561909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a23a8-e173-436b-9192-d26594570b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:32.562441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a23a8-e173-436b-9192-d26594570b6b admin admin] PATH_INFO: `/v3/projects/cdcdda9ef3a04b76af2a643c0aa26009/users/a29014c4674d44d5aa2b98c4b3f1730d/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:32.582306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:32.591608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a23a8-e173-436b-9192-d26594570b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdcdda9ef3a04b76af2a643c0aa26009, user_id=a29014c4674d44d5aa2b98c4b3f1730d, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:32.596677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a23a8-e173-436b-9192-d26594570b6b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:32.639215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af10e1d-eb0b-48ec-9f5a-ebb17fbf013a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:32.639749 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1694/3313] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:32 2022] PUT /identity/v3/projects/cdcdda9ef3a04b76af2a643c0aa26009/users/a29014c4674d44d5aa2b98c4b3f1730d/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:57:32.640459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af10e1d-eb0b-48ec-9f5a-ebb17fbf013a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:32.640807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af10e1d-eb0b-48ec-9f5a-ebb17fbf013a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:32.685999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8af10e1d-eb0b-48ec-9f5a-ebb17fbf013a None None] MFA Rules not processed for user `a29014c4674d44d5aa2b98c4b3f1730d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:32.687429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8af10e1d-eb0b-48ec-9f5a-ebb17fbf013a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:32.786685 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1620/3314] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:32 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:57:32.796654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a23a8-e173-436b-9192-d26594570b6b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:32.807558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-de4a23a8-e173-436b-9192-d26594570b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:32.995365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a23a8-e173-436b-9192-d26594570b6b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:33.006770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7ce14-84e6-41e6-9f5c-9091e1804c06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:33.008365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7ce14-84e6-41e6-9f5c-9091e1804c06 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:33.008754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7ce14-84e6-41e6-9f5c-9091e1804c06 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:33.009125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7ce14-84e6-41e6-9f5c-9091e1804c06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:33.018854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda7ce14-84e6-41e6-9f5c-9091e1804c06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:33.020637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda7ce14-84e6-41e6-9f5c-9091e1804c06 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:33.051604 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1695/3315] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:32 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:33.538070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af10e1d-eb0b-48ec-9f5a-ebb17fbf013a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:33.539763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8af10e1d-eb0b-48ec-9f5a-ebb17fbf013a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:33.706426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af10e1d-eb0b-48ec-9f5a-ebb17fbf013a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:33.714619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4102632-1553-4b8e-8bb0-c46a0e9b2132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:33.715230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4102632-1553-4b8e-8bb0-c46a0e9b2132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:33.715513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4102632-1553-4b8e-8bb0-c46a0e9b2132 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:33.715785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4102632-1553-4b8e-8bb0-c46a0e9b2132 admin admin] PATH_INFO: `/v3/projects/6c6517b87d044904b21ab9f8104211ea` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:33.717090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4102632-1553-4b8e-8bb0-c46a0e9b2132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c6517b87d044904b21ab9f8104211ea)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:33.719197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4102632-1553-4b8e-8bb0-c46a0e9b2132 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:33.782864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:33.784425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e4102632-1553-4b8e-8bb0-c46a0e9b2132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c6517b87d044904b21ab9f8104211ea', 'request_id': 'req-e4102632-1553-4b8e-8bb0-c46a0e9b2132'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:33.810872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e4102632-1553-4b8e-8bb0-c46a0e9b2132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c6517b87d044904b21ab9f8104211ea', 'request_id': 'req-e4102632-1553-4b8e-8bb0-c46a0e9b2132'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:33.810872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e4102632-1553-4b8e-8bb0-c46a0e9b2132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c6517b87d044904b21ab9f8104211ea', 'request_id': 'req-e4102632-1553-4b8e-8bb0-c46a0e9b2132'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:33.824151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e4102632-1553-4b8e-8bb0-c46a0e9b2132 admin admin] The token cache is being invalidate because project 6c6517b87d044904b21ab9f8104211ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:57:33.824449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e4102632-1553-4b8e-8bb0-c46a0e9b2132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:33.825166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e4102632-1553-4b8e-8bb0-c46a0e9b2132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:33.826976 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1621/3316] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:57:33 2022] DELETE /identity/v3/projects/6c6517b87d044904b21ab9f8104211ea => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:33.832985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b5847-3d02-48db-9ac0-1e803e6356dc None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:33.833492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b5847-3d02-48db-9ac0-1e803e6356dc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:33.833492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b5847-3d02-48db-9ac0-1e803e6356dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:33.862685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ea5b5847-3d02-48db-9ac0-1e803e6356dc None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:33.862685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ea5b5847-3d02-48db-9ac0-1e803e6356dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:33.959065 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1696/3317] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:33.967705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831ab48-528a-4570-86e8-925641020d34 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:33.967995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831ab48-528a-4570-86e8-925641020d34 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:33.968274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831ab48-528a-4570-86e8-925641020d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:34.033129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b831ab48-528a-4570-86e8-925641020d34 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:34.034673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b831ab48-528a-4570-86e8-925641020d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:34.067122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b5847-3d02-48db-9ac0-1e803e6356dc None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:34.098148 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1622/3318] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:34.101130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b5847-3d02-48db-9ac0-1e803e6356dc None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:34.122398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056dbfb1-9830-4358-86be-2e7ab0df159c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:34.123062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056dbfb1-9830-4358-86be-2e7ab0df159c admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:34.126250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056dbfb1-9830-4358-86be-2e7ab0df159c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:34.129891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056dbfb1-9830-4358-86be-2e7ab0df159c admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:34.130692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056dbfb1-9830-4358-86be-2e7ab0df159c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:57:34.131023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056dbfb1-9830-4358-86be-2e7ab0df159c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:34.140991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056dbfb1-9830-4358-86be-2e7ab0df159c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:34.183836 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1697/3319] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:57:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:57:34.188190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831ab48-528a-4570-86e8-925641020d34 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:34.208035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831ab48-528a-4570-86e8-925641020d34 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:34.224286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbae57-4851-4702-8d92-8df45bea4b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:34.225188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbae57-4851-4702-8d92-8df45bea4b40 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:34.225607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbae57-4851-4702-8d92-8df45bea4b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:34.226017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbae57-4851-4702-8d92-8df45bea4b40 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:34.226809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbae57-4851-4702-8d92-8df45bea4b40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:34.228847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbae57-4851-4702-8d92-8df45bea4b40 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:34.281350 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1623/3320] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:57:34 2022] POST /identity/v3/projects => generated 403 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:34.287336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056dbfb1-9830-4358-86be-2e7ab0df159c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:34.301950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056dbfb1-9830-4358-86be-2e7ab0df159c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:34.456724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82538dce-6508-4969-bdc1-9a44689e575e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:34.457848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82538dce-6508-4969-bdc1-9a44689e575e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:34.458125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82538dce-6508-4969-bdc1-9a44689e575e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:34.458397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82538dce-6508-4969-bdc1-9a44689e575e admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:34.459064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82538dce-6508-4969-bdc1-9a44689e575e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:34.461912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82538dce-6508-4969-bdc1-9a44689e575e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:34.480938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:34.505515 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1698/3321] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:57:34 2022] POST /identity/v3/users => generated 302 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:34.510064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbae57-4851-4702-8d92-8df45bea4b40 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:34.524542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbae57-4851-4702-8d92-8df45bea4b40 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:34.537194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800bc8a8-e179-49ed-8673-c58bfa4eac08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:34.537831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bc8a8-e179-49ed-8673-c58bfa4eac08 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:34.538114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bc8a8-e179-49ed-8673-c58bfa4eac08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:34.539074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bc8a8-e179-49ed-8673-c58bfa4eac08 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:34.539742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800bc8a8-e179-49ed-8673-c58bfa4eac08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:34.541382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800bc8a8-e179-49ed-8673-c58bfa4eac08 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:34.560793 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1624/3322] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:34.565026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82538dce-6508-4969-bdc1-9a44689e575e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:34.577371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82538dce-6508-4969-bdc1-9a44689e575e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:34.588491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2abdf-8aed-4852-98b8-0444335f1797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:34.589214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2abdf-8aed-4852-98b8-0444335f1797 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:34.589408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2abdf-8aed-4852-98b8-0444335f1797 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:34.589969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2abdf-8aed-4852-98b8-0444335f1797 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:34.591112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2abdf-8aed-4852-98b8-0444335f1797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:34.592910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2abdf-8aed-4852-98b8-0444335f1797 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:34.611378 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1699/3323] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:57:34.616058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800bc8a8-e179-49ed-8673-c58bfa4eac08 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:34.634019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800bc8a8-e179-49ed-8673-c58bfa4eac08 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:34.652041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e25f38-9ed4-4d4f-b2db-6d43a2a4d423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:34.655179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e25f38-9ed4-4d4f-b2db-6d43a2a4d423 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:34.655720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e25f38-9ed4-4d4f-b2db-6d43a2a4d423 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:34.656302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e25f38-9ed4-4d4f-b2db-6d43a2a4d423 admin admin] PATH_INFO: `/v3/projects/f5e9bf185e0e40219dc7deefa296173d/users/66e0ab3f29ca4b1dafbd5a3e30587c36/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:34.690580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e25f38-9ed4-4d4f-b2db-6d43a2a4d423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5e9bf185e0e40219dc7deefa296173d, user_id=66e0ab3f29ca4b1dafbd5a3e30587c36, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:34.694445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e25f38-9ed4-4d4f-b2db-6d43a2a4d423 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:34.724337 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1625/3324] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:34 2022] PUT /identity/v3/projects/f5e9bf185e0e40219dc7deefa296173d/users/66e0ab3f29ca4b1dafbd5a3e30587c36/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:34.735227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6592e4b-05ee-4683-821c-097d18e9f281 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:34.735515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6592e4b-05ee-4683-821c-097d18e9f281 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:34.736077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6592e4b-05ee-4683-821c-097d18e9f281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:34.790635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a6592e4b-05ee-4683-821c-097d18e9f281 None None] MFA Rules not processed for user `66e0ab3f29ca4b1dafbd5a3e30587c36`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:34.791932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a6592e4b-05ee-4683-821c-097d18e9f281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:34.947883 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1700/3325] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:34 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:34.953014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e25f38-9ed4-4d4f-b2db-6d43a2a4d423 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:35.045237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e25f38-9ed4-4d4f-b2db-6d43a2a4d423 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:35.050320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d277248-1824-45de-9bdb-8b190856d8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:35.052561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d277248-1824-45de-9bdb-8b190856d8c7 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:35.052932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d277248-1824-45de-9bdb-8b190856d8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:35.053656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d277248-1824-45de-9bdb-8b190856d8c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:35.057488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d277248-1824-45de-9bdb-8b190856d8c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:35.061562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d277248-1824-45de-9bdb-8b190856d8c7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:35.097177 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1626/3326] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:57:34 2022] POST /identity/v3/projects => generated 403 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:35.101535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6592e4b-05ee-4683-821c-097d18e9f281 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:35.112807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6592e4b-05ee-4683-821c-097d18e9f281 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:35.189706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d277248-1824-45de-9bdb-8b190856d8c7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:35.191733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0d277248-1824-45de-9bdb-8b190856d8c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:35.207697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9882f-72ee-4655-9da9-433c43ebf359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:35.207697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9882f-72ee-4655-9da9-433c43ebf359 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:35.207697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9882f-72ee-4655-9da9-433c43ebf359 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:35.207697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9882f-72ee-4655-9da9-433c43ebf359 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:35.208114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9882f-72ee-4655-9da9-433c43ebf359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:35.208767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9882f-72ee-4655-9da9-433c43ebf359 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:35.271505 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1701/3327] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:57:35 2022] POST /identity/v3/users => generated 308 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:57:35.273017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9882f-72ee-4655-9da9-433c43ebf359 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:35.295443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9882f-72ee-4655-9da9-433c43ebf359 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:35.301759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278a635-5d79-4c53-b5ff-dac158bfae0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:35.302578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278a635-5d79-4c53-b5ff-dac158bfae0a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:35.302886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278a635-5d79-4c53-b5ff-dac158bfae0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:35.303231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278a635-5d79-4c53-b5ff-dac158bfae0a admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:35.304176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278a635-5d79-4c53-b5ff-dac158bfae0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:35.304997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:35.307283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278a635-5d79-4c53-b5ff-dac158bfae0a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:35.309223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d277248-1824-45de-9bdb-8b190856d8c7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:35.316993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babcb755-1e91-4f57-9148-b8a0d2d89a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:35.318539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcb755-1e91-4f57-9148-b8a0d2d89a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:35.318823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcb755-1e91-4f57-9148-b8a0d2d89a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:35.319113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcb755-1e91-4f57-9148-b8a0d2d89a06 admin admin] PATH_INFO: `/v3/users/a29014c4674d44d5aa2b98c4b3f1730d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:35.320501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babcb755-1e91-4f57-9148-b8a0d2d89a06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a29014c4674d44d5aa2b98c4b3f1730d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:35.322552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babcb755-1e91-4f57-9148-b8a0d2d89a06 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:35.325237 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1702/3328] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:35.327618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278a635-5d79-4c53-b5ff-dac158bfae0a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:35.341526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278a635-5d79-4c53-b5ff-dac158bfae0a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:35.348922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fe935-ffbd-472c-bd7d-b0a7bd1ba147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:35.349908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fe935-ffbd-472c-bd7d-b0a7bd1ba147 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:35.350298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fe935-ffbd-472c-bd7d-b0a7bd1ba147 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:35.351126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fe935-ffbd-472c-bd7d-b0a7bd1ba147 admin admin] PATH_INFO: `/v3/projects/3f9cbc3e76fc414fa7e6b3aa869668d5/users/dc06764d7ab8496e8cd24290f70f6fd1/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:35.373985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6fe935-ffbd-472c-bd7d-b0a7bd1ba147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f9cbc3e76fc414fa7e6b3aa869668d5, user_id=dc06764d7ab8496e8cd24290f70f6fd1, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:35.377844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6fe935-ffbd-472c-bd7d-b0a7bd1ba147 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:35.398622 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1703/3329] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:35 2022] PUT /identity/v3/projects/3f9cbc3e76fc414fa7e6b3aa869668d5/users/dc06764d7ab8496e8cd24290f70f6fd1/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:35.399967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fe935-ffbd-472c-bd7d-b0a7bd1ba147 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:35.417345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-babcb755-1e91-4f57-9148-b8a0d2d89a06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a29014c4674d44d5aa2b98c4b3f1730d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:35.431919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-babcb755-1e91-4f57-9148-b8a0d2d89a06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a29014c4674d44d5aa2b98c4b3f1730d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:35.432981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-babcb755-1e91-4f57-9148-b8a0d2d89a06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a29014c4674d44d5aa2b98c4b3f1730d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:35.439795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:35.446472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-babcb755-1e91-4f57-9148-b8a0d2d89a06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a29014c4674d44d5aa2b98c4b3f1730d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:35.464780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-babcb755-1e91-4f57-9148-b8a0d2d89a06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a29014c4674d44d5aa2b98c4b3f1730d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:35.485937 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1627/3330] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:57:35 2022] DELETE /identity/v3/users/a29014c4674d44d5aa2b98c4b3f1730d => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:57:35.486096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fe935-ffbd-472c-bd7d-b0a7bd1ba147 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:35.488051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb165181-5743-4ad2-80e8-48fdfdc94032 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:35.488937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb165181-5743-4ad2-80e8-48fdfdc94032 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:35.489264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb165181-5743-4ad2-80e8-48fdfdc94032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:35.514176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fb165181-5743-4ad2-80e8-48fdfdc94032 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:35.515461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fb165181-5743-4ad2-80e8-48fdfdc94032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:35.604225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ccb0ca-2dc0-4536-ae7d-877f3b26ed20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:35.604881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccb0ca-2dc0-4536-ae7d-877f3b26ed20 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:35.606819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccb0ca-2dc0-4536-ae7d-877f3b26ed20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:35.607242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccb0ca-2dc0-4536-ae7d-877f3b26ed20 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:35.610414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ccb0ca-2dc0-4536-ae7d-877f3b26ed20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:35.612761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ccb0ca-2dc0-4536-ae7d-877f3b26ed20 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:35.617983 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1628/3331] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:57:35.619770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb165181-5743-4ad2-80e8-48fdfdc94032 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:35.621891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fb165181-5743-4ad2-80e8-48fdfdc94032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:35.654997 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1704/3332] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:35.657842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ccb0ca-2dc0-4536-ae7d-877f3b26ed20 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:35.662981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e9ccb0ca-2dc0-4536-ae7d-877f3b26ed20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:35.685301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ccb0ca-2dc0-4536-ae7d-877f3b26ed20 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:35.693591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf94c81-2a2a-4c24-be97-d6e2c2d4b98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:35.694673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf94c81-2a2a-4c24-be97-d6e2c2d4b98c admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:35.695164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf94c81-2a2a-4c24-be97-d6e2c2d4b98c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:35.695543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf94c81-2a2a-4c24-be97-d6e2c2d4b98c admin admin] PATH_INFO: `/v3/projects/3f9cbc3e76fc414fa7e6b3aa869668d5/users/dc06764d7ab8496e8cd24290f70f6fd1/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:35.698739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf94c81-2a2a-4c24-be97-d6e2c2d4b98c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f9cbc3e76fc414fa7e6b3aa869668d5, user_id=dc06764d7ab8496e8cd24290f70f6fd1, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:35.703262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf94c81-2a2a-4c24-be97-d6e2c2d4b98c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:35.729483 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1705/3333] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:35 2022] PUT /identity/v3/projects/3f9cbc3e76fc414fa7e6b3aa869668d5/users/dc06764d7ab8496e8cd24290f70f6fd1/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:35.729671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb165181-5743-4ad2-80e8-48fdfdc94032 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:35.740566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c67598-57c4-49ca-a66e-c70fd266ed68 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:35.740904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c67598-57c4-49ca-a66e-c70fd266ed68 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:35.741446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c67598-57c4-49ca-a66e-c70fd266ed68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:35.783905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:35.786340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-26c67598-57c4-49ca-a66e-c70fd266ed68 None None] MFA Rules not processed for user `dc06764d7ab8496e8cd24290f70f6fd1`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:35.788656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-26c67598-57c4-49ca-a66e-c70fd266ed68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:35.815371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1996b-12a9-40c4-82f7-a6bd6cb0073c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:35.817388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1996b-12a9-40c4-82f7-a6bd6cb0073c service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:35.818050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1996b-12a9-40c4-82f7-a6bd6cb0073c service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:35.818518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1996b-12a9-40c4-82f7-a6bd6cb0073c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:35.944297 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1706/3334] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:35 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:57:35.950258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1996b-12a9-40c4-82f7-a6bd6cb0073c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:35.951912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1996b-12a9-40c4-82f7-a6bd6cb0073c service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:35.957842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c67598-57c4-49ca-a66e-c70fd266ed68 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:35.959711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-26c67598-57c4-49ca-a66e-c70fd266ed68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:35.966691 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1629/3335] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:36.017686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:36.076586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c67598-57c4-49ca-a66e-c70fd266ed68 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:36.083155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44bb66-2502-4b73-9d9a-0b222156991e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:36.083877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44bb66-2502-4b73-9d9a-0b222156991e service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:36.084139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44bb66-2502-4b73-9d9a-0b222156991e service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:36.084579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44bb66-2502-4b73-9d9a-0b222156991e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:36.097432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a44bb66-2502-4b73-9d9a-0b222156991e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:36.099140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a44bb66-2502-4b73-9d9a-0b222156991e service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:36.122156 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1707/3336] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:35 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:57:36.346537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1996b-12a9-40c4-82f7-a6bd6cb0073c service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:36.348706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c7c1996b-12a9-40c4-82f7-a6bd6cb0073c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:36.371701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1996b-12a9-40c4-82f7-a6bd6cb0073c service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:36.385146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb2312-ee38-4df9-8822-7cb8c79feaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:36.385879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb2312-ee38-4df9-8822-7cb8c79feaf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:36.386160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb2312-ee38-4df9-8822-7cb8c79feaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:36.386435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb2312-ee38-4df9-8822-7cb8c79feaf3 admin admin] PATH_INFO: `/v3/projects/cdcdda9ef3a04b76af2a643c0aa26009` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:36.389228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb2312-ee38-4df9-8822-7cb8c79feaf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdcdda9ef3a04b76af2a643c0aa26009)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:36.391266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb2312-ee38-4df9-8822-7cb8c79feaf3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:36.491501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-96fb2312-ee38-4df9-8822-7cb8c79feaf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdcdda9ef3a04b76af2a643c0aa26009', 'request_id': 'req-96fb2312-ee38-4df9-8822-7cb8c79feaf3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:36.514406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-96fb2312-ee38-4df9-8822-7cb8c79feaf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdcdda9ef3a04b76af2a643c0aa26009', 'request_id': 'req-96fb2312-ee38-4df9-8822-7cb8c79feaf3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:36.522475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-96fb2312-ee38-4df9-8822-7cb8c79feaf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdcdda9ef3a04b76af2a643c0aa26009', 'request_id': 'req-96fb2312-ee38-4df9-8822-7cb8c79feaf3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:36.555501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-96fb2312-ee38-4df9-8822-7cb8c79feaf3 admin admin] The token cache is being invalidate because project cdcdda9ef3a04b76af2a643c0aa26009 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:57:36.556019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-96fb2312-ee38-4df9-8822-7cb8c79feaf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:36.557716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-96fb2312-ee38-4df9-8822-7cb8c79feaf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:36.560337 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1630/3337] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:57:36 2022] DELETE /identity/v3/projects/cdcdda9ef3a04b76af2a643c0aa26009 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:36.565336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0a4e5-56db-4be0-8672-956f41614d00 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:36.565625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0a4e5-56db-4be0-8672-956f41614d00 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:36.565904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0a4e5-56db-4be0-8672-956f41614d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:36.603537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c6e0a4e5-56db-4be0-8672-956f41614d00 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:36.605975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c6e0a4e5-56db-4be0-8672-956f41614d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:36.780117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf741769-aa3e-46c8-a633-544079f470e2 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:36.780677 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1708/3338] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:36.780755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf741769-aa3e-46c8-a633-544079f470e2 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:36.780823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf741769-aa3e-46c8-a633-544079f470e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:36.826529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bf741769-aa3e-46c8-a633-544079f470e2 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:36.828430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bf741769-aa3e-46c8-a633-544079f470e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:36.843307 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1631/3339] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:36.847275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0a4e5-56db-4be0-8672-956f41614d00 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:36.870736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0a4e5-56db-4be0-8672-956f41614d00 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:36.885611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b87e0-8c98-4965-8372-ac78d774dace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:36.886352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b87e0-8c98-4965-8372-ac78d774dace admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:36.886676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b87e0-8c98-4965-8372-ac78d774dace admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:36.887303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b87e0-8c98-4965-8372-ac78d774dace admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:36.888303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b87e0-8c98-4965-8372-ac78d774dace admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:57:36.888702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b87e0-8c98-4965-8372-ac78d774dace admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:36.891167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b87e0-8c98-4965-8372-ac78d774dace admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:36.918982 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1709/3340] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:57:36.920952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf741769-aa3e-46c8-a633-544079f470e2 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:36.939882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf741769-aa3e-46c8-a633-544079f470e2 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:36.956714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3d406-0789-458f-895b-3d271f8ccf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:36.958065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3d406-0789-458f-895b-3d271f8ccf91 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:36.958970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3d406-0789-458f-895b-3d271f8ccf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:36.959455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3d406-0789-458f-895b-3d271f8ccf91 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:36.961884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3d406-0789-458f-895b-3d271f8ccf91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:36.966810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3d406-0789-458f-895b-3d271f8ccf91 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:37.043170 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1632/3341] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:57:36 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:37.047294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b87e0-8c98-4965-8372-ac78d774dace admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:37.076314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b87e0-8c98-4965-8372-ac78d774dace admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:37.091738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3d406-0789-458f-895b-3d271f8ccf91 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:37.093785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b6b3d406-0789-458f-895b-3d271f8ccf91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:37.153315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:37.189535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb2732-9039-4aa2-86b8-cdf21e81110c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:37.190804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb2732-9039-4aa2-86b8-cdf21e81110c admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:37.192165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb2732-9039-4aa2-86b8-cdf21e81110c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:37.192616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb2732-9039-4aa2-86b8-cdf21e81110c admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:37.193394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb2732-9039-4aa2-86b8-cdf21e81110c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:37.195679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb2732-9039-4aa2-86b8-cdf21e81110c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:37.219979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3d406-0789-458f-895b-3d271f8ccf91 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:37.228760 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1710/3342] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:57:37 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:37.231833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6631ab7-ec21-4dc9-b6bd-d954d148d958 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:37.232884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb2732-9039-4aa2-86b8-cdf21e81110c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:37.237985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6631ab7-ec21-4dc9-b6bd-d954d148d958 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:37.238893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6631ab7-ec21-4dc9-b6bd-d954d148d958 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:37.239180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6631ab7-ec21-4dc9-b6bd-d954d148d958 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:37.241908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d6631ab7-ec21-4dc9-b6bd-d954d148d958 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:37.248829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb2732-9039-4aa2-86b8-cdf21e81110c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:37.255326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7f6ca-dd69-41d5-91dd-f9aade196ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:37.256436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7f6ca-dd69-41d5-91dd-f9aade196ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:37.256436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7f6ca-dd69-41d5-91dd-f9aade196ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:37.256765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7f6ca-dd69-41d5-91dd-f9aade196ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:37.258457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7f6ca-dd69-41d5-91dd-f9aade196ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:37.261947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7f6ca-dd69-41d5-91dd-f9aade196ba6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:37.275357 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1711/3343] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:57:37.279694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7f6ca-dd69-41d5-91dd-f9aade196ba6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:37.310799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7f6ca-dd69-41d5-91dd-f9aade196ba6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:37.326526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe80b14-6ea9-4cf3-8831-88285b4dc093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:37.327188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe80b14-6ea9-4cf3-8831-88285b4dc093 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:37.328128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe80b14-6ea9-4cf3-8831-88285b4dc093 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:37.328412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe80b14-6ea9-4cf3-8831-88285b4dc093 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:37.332035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe80b14-6ea9-4cf3-8831-88285b4dc093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:37.334783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe80b14-6ea9-4cf3-8831-88285b4dc093 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:37.365628 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1712/3344] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:37.374876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe80b14-6ea9-4cf3-8831-88285b4dc093 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:37.381348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6631ab7-ec21-4dc9-b6bd-d954d148d958 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:37.383155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6631ab7-ec21-4dc9-b6bd-d954d148d958 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:37.388895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe80b14-6ea9-4cf3-8831-88285b4dc093 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:37.395501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546b0e5-6366-4207-8053-f34b45bdd563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:37.396099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546b0e5-6366-4207-8053-f34b45bdd563 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:37.396379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546b0e5-6366-4207-8053-f34b45bdd563 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:37.396653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546b0e5-6366-4207-8053-f34b45bdd563 admin admin] PATH_INFO: `/v3/projects/3cf38b1a4bc4458199fea3373632117d/users/4e76b153982141d8964ecbb118d19eab/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:37.414118 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1633/3345] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:37 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:37.421100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546b0e5-6366-4207-8053-f34b45bdd563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cf38b1a4bc4458199fea3373632117d, user_id=4e76b153982141d8964ecbb118d19eab, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:37.424255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546b0e5-6366-4207-8053-f34b45bdd563 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:37.427887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:37.441970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6631ab7-ec21-4dc9-b6bd-d954d148d958 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:37.444226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d6631ab7-ec21-4dc9-b6bd-d954d148d958 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:37.455763 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1713/3346] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:37 2022] PUT /identity/v3/projects/3cf38b1a4bc4458199fea3373632117d/users/4e76b153982141d8964ecbb118d19eab/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:37.463185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6631ab7-ec21-4dc9-b6bd-d954d148d958 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:37.469819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6c299-b045-4422-99bd-55e94a9e45c5 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:37.470118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6c299-b045-4422-99bd-55e94a9e45c5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:37.470757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6c299-b045-4422-99bd-55e94a9e45c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:37.511805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-07c6c299-b045-4422-99bd-55e94a9e45c5 None None] MFA Rules not processed for user `4e76b153982141d8964ecbb118d19eab`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:37.511805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-07c6c299-b045-4422-99bd-55e94a9e45c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:37.542733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19814c7e-e36b-4100-ad33-7d076ccec9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:37.543541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19814c7e-e36b-4100-ad33-7d076ccec9c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:37.543984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19814c7e-e36b-4100-ad33-7d076ccec9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:37.544353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19814c7e-e36b-4100-ad33-7d076ccec9c8 admin admin] PATH_INFO: `/v3/users/66e0ab3f29ca4b1dafbd5a3e30587c36` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:37.545680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19814c7e-e36b-4100-ad33-7d076ccec9c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66e0ab3f29ca4b1dafbd5a3e30587c36)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:37.547967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19814c7e-e36b-4100-ad33-7d076ccec9c8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:37.621176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-19814c7e-e36b-4100-ad33-7d076ccec9c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66e0ab3f29ca4b1dafbd5a3e30587c36'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:37.622195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-19814c7e-e36b-4100-ad33-7d076ccec9c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66e0ab3f29ca4b1dafbd5a3e30587c36'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:37.622761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-19814c7e-e36b-4100-ad33-7d076ccec9c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66e0ab3f29ca4b1dafbd5a3e30587c36'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:37.634868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-19814c7e-e36b-4100-ad33-7d076ccec9c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66e0ab3f29ca4b1dafbd5a3e30587c36'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:37.658704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-19814c7e-e36b-4100-ad33-7d076ccec9c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66e0ab3f29ca4b1dafbd5a3e30587c36'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:37.665123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:37.682800 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1714/3348] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:37.684165 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1634/3348] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:57:37 2022] DELETE /identity/v3/users/66e0ab3f29ca4b1dafbd5a3e30587c36 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:57:37.684580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6c299-b045-4422-99bd-55e94a9e45c5 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:37.686244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f78455-c741-4fbe-a802-96e5d8e5c0f6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:37.686851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f78455-c741-4fbe-a802-96e5d8e5c0f6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:37.687290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-07c6c299-b045-4422-99bd-55e94a9e45c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:37.687567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f78455-c741-4fbe-a802-96e5d8e5c0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:37.724641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-71f78455-c741-4fbe-a802-96e5d8e5c0f6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:37.725973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-71f78455-c741-4fbe-a802-96e5d8e5c0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:37.788933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6c299-b045-4422-99bd-55e94a9e45c5 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:37.822177 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1635/3349] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:37.824153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac0e84-2dc1-45ce-836d-bd4476d4d058 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:37.824153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac0e84-2dc1-45ce-836d-bd4476d4d058 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:37.824153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac0e84-2dc1-45ce-836d-bd4476d4d058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:37.844991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-98ac0e84-2dc1-45ce-836d-bd4476d4d058 None None] MFA Rules not processed for user `4e76b153982141d8964ecbb118d19eab`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:37.846218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-98ac0e84-2dc1-45ce-836d-bd4476d4d058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:37.858318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467fb6f-01b5-4b85-bb6c-24bd1fd85db6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:37.858963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467fb6f-01b5-4b85-bb6c-24bd1fd85db6 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:37.859251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467fb6f-01b5-4b85-bb6c-24bd1fd85db6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:37.859496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467fb6f-01b5-4b85-bb6c-24bd1fd85db6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:37.862578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a467fb6f-01b5-4b85-bb6c-24bd1fd85db6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:37.882608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a467fb6f-01b5-4b85-bb6c-24bd1fd85db6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:37.885275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a467fb6f-01b5-4b85-bb6c-24bd1fd85db6 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:37.897712 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1715/3350] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:37.898536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467fb6f-01b5-4b85-bb6c-24bd1fd85db6 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:37.900090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a467fb6f-01b5-4b85-bb6c-24bd1fd85db6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:37.909092 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1636/3351] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:37.912557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467fb6f-01b5-4b85-bb6c-24bd1fd85db6 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:37.916639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5273b6-f07c-42f9-94a5-032c2d306e2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:37.917250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5273b6-f07c-42f9-94a5-032c2d306e2f service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:37.917536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5273b6-f07c-42f9-94a5-032c2d306e2f service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:37.917817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5273b6-f07c-42f9-94a5-032c2d306e2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:37.927914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5273b6-f07c-42f9-94a5-032c2d306e2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:37.929332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5273b6-f07c-42f9-94a5-032c2d306e2f service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:37.939150 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1716/3352] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:38.053461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac0e84-2dc1-45ce-836d-bd4476d4d058 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:38.055508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-98ac0e84-2dc1-45ce-836d-bd4476d4d058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:38.149151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac0e84-2dc1-45ce-836d-bd4476d4d058 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:38.156259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7fe445-e5aa-4cfd-bd57-6ac5cd750e39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:38.157165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fe445-e5aa-4cfd-bd57-6ac5cd750e39 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:38.157165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fe445-e5aa-4cfd-bd57-6ac5cd750e39 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:38.157445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fe445-e5aa-4cfd-bd57-6ac5cd750e39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:38.170452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7fe445-e5aa-4cfd-bd57-6ac5cd750e39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:38.172296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7fe445-e5aa-4cfd-bd57-6ac5cd750e39 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:38.183533 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1637/3353] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:38.247695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5273b6-f07c-42f9-94a5-032c2d306e2f service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:38.249083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5f5273b6-f07c-42f9-94a5-032c2d306e2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:38.267465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5273b6-f07c-42f9-94a5-032c2d306e2f service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:38.273029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8f05b-2929-46cc-8a4f-b9f7350de7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:38.273728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8f05b-2929-46cc-8a4f-b9f7350de7f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:38.274214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8f05b-2929-46cc-8a4f-b9f7350de7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:38.274589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8f05b-2929-46cc-8a4f-b9f7350de7f1 admin admin] PATH_INFO: `/v3/projects/f5e9bf185e0e40219dc7deefa296173d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:38.275854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e8f05b-2929-46cc-8a4f-b9f7350de7f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5e9bf185e0e40219dc7deefa296173d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:38.277881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e8f05b-2929-46cc-8a4f-b9f7350de7f1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:38.319169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:38.339977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b9e8f05b-2929-46cc-8a4f-b9f7350de7f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5e9bf185e0e40219dc7deefa296173d', 'request_id': 'req-b9e8f05b-2929-46cc-8a4f-b9f7350de7f1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:38.347545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b9e8f05b-2929-46cc-8a4f-b9f7350de7f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5e9bf185e0e40219dc7deefa296173d', 'request_id': 'req-b9e8f05b-2929-46cc-8a4f-b9f7350de7f1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:38.354638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b9e8f05b-2929-46cc-8a4f-b9f7350de7f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5e9bf185e0e40219dc7deefa296173d', 'request_id': 'req-b9e8f05b-2929-46cc-8a4f-b9f7350de7f1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:38.371474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b9e8f05b-2929-46cc-8a4f-b9f7350de7f1 admin admin] The token cache is being invalidate because project f5e9bf185e0e40219dc7deefa296173d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:57:38.371765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b9e8f05b-2929-46cc-8a4f-b9f7350de7f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:38.372406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b9e8f05b-2929-46cc-8a4f-b9f7350de7f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:38.374231 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1717/3354] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:57:38 2022] DELETE /identity/v3/projects/f5e9bf185e0e40219dc7deefa296173d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:38.377933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7fe445-e5aa-4cfd-bd57-6ac5cd750e39 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:38.389663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7fe445-e5aa-4cfd-bd57-6ac5cd750e39 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:38.415217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:38.454898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c655de-e742-4788-91f1-91cb62991d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:38.455624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c655de-e742-4788-91f1-91cb62991d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:38.455922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c655de-e742-4788-91f1-91cb62991d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:38.456192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c655de-e742-4788-91f1-91cb62991d31 admin admin] PATH_INFO: `/v3/users/dc06764d7ab8496e8cd24290f70f6fd1` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:38.457785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c655de-e742-4788-91f1-91cb62991d31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc06764d7ab8496e8cd24290f70f6fd1)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:38.459743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c655de-e742-4788-91f1-91cb62991d31 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:38.510522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00c655de-e742-4788-91f1-91cb62991d31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc06764d7ab8496e8cd24290f70f6fd1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:38.511184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00c655de-e742-4788-91f1-91cb62991d31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc06764d7ab8496e8cd24290f70f6fd1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:38.511786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00c655de-e742-4788-91f1-91cb62991d31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc06764d7ab8496e8cd24290f70f6fd1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:38.519351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00c655de-e742-4788-91f1-91cb62991d31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc06764d7ab8496e8cd24290f70f6fd1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:38.532246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00c655de-e742-4788-91f1-91cb62991d31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc06764d7ab8496e8cd24290f70f6fd1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:38.545171 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1638/3355] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:57:38 2022] DELETE /identity/v3/users/dc06764d7ab8496e8cd24290f70f6fd1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:38.951354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8f05b-2929-46cc-8a4f-b9f7350de7f1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:38.953615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b9e8f05b-2929-46cc-8a4f-b9f7350de7f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:38.976958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8f05b-2929-46cc-8a4f-b9f7350de7f1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:39.102546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65007a0d-9b1c-48fc-8e98-630e6cfb9734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:39.103191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65007a0d-9b1c-48fc-8e98-630e6cfb9734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:39.103486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65007a0d-9b1c-48fc-8e98-630e6cfb9734 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:39.103749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65007a0d-9b1c-48fc-8e98-630e6cfb9734 admin admin] PATH_INFO: `/v3/projects/3f9cbc3e76fc414fa7e6b3aa869668d5` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:39.105419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65007a0d-9b1c-48fc-8e98-630e6cfb9734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f9cbc3e76fc414fa7e6b3aa869668d5)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:39.107750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65007a0d-9b1c-48fc-8e98-630e6cfb9734 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:39.182378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-65007a0d-9b1c-48fc-8e98-630e6cfb9734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f9cbc3e76fc414fa7e6b3aa869668d5', 'request_id': 'req-65007a0d-9b1c-48fc-8e98-630e6cfb9734'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:39.192366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-65007a0d-9b1c-48fc-8e98-630e6cfb9734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f9cbc3e76fc414fa7e6b3aa869668d5', 'request_id': 'req-65007a0d-9b1c-48fc-8e98-630e6cfb9734'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:39.200165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-65007a0d-9b1c-48fc-8e98-630e6cfb9734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f9cbc3e76fc414fa7e6b3aa869668d5', 'request_id': 'req-65007a0d-9b1c-48fc-8e98-630e6cfb9734'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:39.245744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-65007a0d-9b1c-48fc-8e98-630e6cfb9734 admin admin] The token cache is being invalidate because project 3f9cbc3e76fc414fa7e6b3aa869668d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:57:39.247376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-65007a0d-9b1c-48fc-8e98-630e6cfb9734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:39.247752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-65007a0d-9b1c-48fc-8e98-630e6cfb9734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:39.255606 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1718/3356] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:57:38 2022] DELETE /identity/v3/projects/3f9cbc3e76fc414fa7e6b3aa869668d5 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:39.263274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bf755-8a07-489b-bada-e36c88402f82 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:39.264010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bf755-8a07-489b-bada-e36c88402f82 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:39.264292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bf755-8a07-489b-bada-e36c88402f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:39.304328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f09bf755-8a07-489b-bada-e36c88402f82 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:39.305594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f09bf755-8a07-489b-bada-e36c88402f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:39.381471 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1639/3357] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:39.388631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ba607-3892-4696-b7ec-182b0725f91e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:39.389028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ba607-3892-4696-b7ec-182b0725f91e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:39.389257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ba607-3892-4696-b7ec-182b0725f91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:39.410808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-af2ba607-3892-4696-b7ec-182b0725f91e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:39.411889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-af2ba607-3892-4696-b7ec-182b0725f91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:39.424747 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1719/3358] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:57:39.433147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09bf755-8a07-489b-bada-e36c88402f82 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:39.463144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09bf755-8a07-489b-bada-e36c88402f82 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:39.472316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350a638-a9d2-4284-b0e8-ffc59be5b19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:39.473065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350a638-a9d2-4284-b0e8-ffc59be5b19a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:39.473476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350a638-a9d2-4284-b0e8-ffc59be5b19a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:39.473795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350a638-a9d2-4284-b0e8-ffc59be5b19a admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:39.474579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350a638-a9d2-4284-b0e8-ffc59be5b19a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:57:39.474973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350a638-a9d2-4284-b0e8-ffc59be5b19a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:39.476798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350a638-a9d2-4284-b0e8-ffc59be5b19a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:39.494867 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1640/3359] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:57:39.495886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ba607-3892-4696-b7ec-182b0725f91e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:39.507290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ba607-3892-4696-b7ec-182b0725f91e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:39.516887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bf136-cd1c-4f3a-b8f1-e821977a7a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:39.517841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bf136-cd1c-4f3a-b8f1-e821977a7a87 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:39.517963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bf136-cd1c-4f3a-b8f1-e821977a7a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:39.518365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bf136-cd1c-4f3a-b8f1-e821977a7a87 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:39.518918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35bf136-cd1c-4f3a-b8f1-e821977a7a87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:39.521486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35bf136-cd1c-4f3a-b8f1-e821977a7a87 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:39.560835 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1720/3360] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:57:39 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:39.564931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350a638-a9d2-4284-b0e8-ffc59be5b19a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:39.576505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350a638-a9d2-4284-b0e8-ffc59be5b19a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:39.612645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:39.656246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d68222-317b-4e79-8a2c-0f32a9fccde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:39.656946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d68222-317b-4e79-8a2c-0f32a9fccde0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:39.657259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d68222-317b-4e79-8a2c-0f32a9fccde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:39.657429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d68222-317b-4e79-8a2c-0f32a9fccde0 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:39.658165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d68222-317b-4e79-8a2c-0f32a9fccde0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:39.660869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d68222-317b-4e79-8a2c-0f32a9fccde0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:39.695890 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1641/3361] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:57:39 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:39.697569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bf136-cd1c-4f3a-b8f1-e821977a7a87 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:39.712892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bf136-cd1c-4f3a-b8f1-e821977a7a87 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:39.720873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a6e4b-a4f8-4011-acef-573591b5bc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:39.721667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a6e4b-a4f8-4011-acef-573591b5bc71 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:39.721973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a6e4b-a4f8-4011-acef-573591b5bc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:39.722316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a6e4b-a4f8-4011-acef-573591b5bc71 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:39.723727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a6e4b-a4f8-4011-acef-573591b5bc71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:39.726804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a6e4b-a4f8-4011-acef-573591b5bc71 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:39.752003 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1721/3362] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:57:39.753960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d68222-317b-4e79-8a2c-0f32a9fccde0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:39.765497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d68222-317b-4e79-8a2c-0f32a9fccde0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:39.773758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a342dc-ccc6-4f62-bb83-3cc1909c16c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:39.774853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a342dc-ccc6-4f62-bb83-3cc1909c16c1 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:39.775136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a342dc-ccc6-4f62-bb83-3cc1909c16c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:39.775409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a342dc-ccc6-4f62-bb83-3cc1909c16c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:39.776470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a342dc-ccc6-4f62-bb83-3cc1909c16c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:39.778161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a342dc-ccc6-4f62-bb83-3cc1909c16c1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:39.794367 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1642/3363] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:39.795154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a6e4b-a4f8-4011-acef-573591b5bc71 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:39.802901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:39.807287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a6e4b-a4f8-4011-acef-573591b5bc71 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:39.814990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531237a-03f6-462b-9308-3e07c0d10201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:39.815681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531237a-03f6-462b-9308-3e07c0d10201 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:39.816311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531237a-03f6-462b-9308-3e07c0d10201 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:39.816916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531237a-03f6-462b-9308-3e07c0d10201 admin admin] PATH_INFO: `/v3/projects/bcdb77df347145639f2de60eb820a3ca/users/ccbf2c814364466baca8dfcabd2c13f9/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:39.841593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531237a-03f6-462b-9308-3e07c0d10201 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcdb77df347145639f2de60eb820a3ca, user_id=ccbf2c814364466baca8dfcabd2c13f9, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:39.844357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531237a-03f6-462b-9308-3e07c0d10201 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:39.861102 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1722/3364] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:39 2022] PUT /identity/v3/projects/bcdb77df347145639f2de60eb820a3ca/users/ccbf2c814364466baca8dfcabd2c13f9/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:39.866399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a0e49-d168-4a69-a91b-826dc1beb51a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:39.866728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a0e49-d168-4a69-a91b-826dc1beb51a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:39.867004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a0e49-d168-4a69-a91b-826dc1beb51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:39.892058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a74a0e49-d168-4a69-a91b-826dc1beb51a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:39.895558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a74a0e49-d168-4a69-a91b-826dc1beb51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:39.986313 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1643/3365] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:39.995288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531237a-03f6-462b-9308-3e07c0d10201 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:39.997417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1531237a-03f6-462b-9308-3e07c0d10201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:40.010576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a0e49-d168-4a69-a91b-826dc1beb51a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:40.012052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a74a0e49-d168-4a69-a91b-826dc1beb51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:40.026357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a0e49-d168-4a69-a91b-826dc1beb51a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:40.036060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbccf8-4da1-4409-a595-d9d84ee1e41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:40.036717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbccf8-4da1-4409-a595-d9d84ee1e41b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:40.036998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbccf8-4da1-4409-a595-d9d84ee1e41b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:40.037273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbccf8-4da1-4409-a595-d9d84ee1e41b admin admin] PATH_INFO: `/v3/users/a418c7b850684796bf4022c5b05e936d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:40.038483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bbccf8-4da1-4409-a595-d9d84ee1e41b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a418c7b850684796bf4022c5b05e936d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:40.049180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bbccf8-4da1-4409-a595-d9d84ee1e41b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:40.113151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-15bbccf8-4da1-4409-a595-d9d84ee1e41b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a418c7b850684796bf4022c5b05e936d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:40.114239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-15bbccf8-4da1-4409-a595-d9d84ee1e41b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a418c7b850684796bf4022c5b05e936d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:40.115010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-15bbccf8-4da1-4409-a595-d9d84ee1e41b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a418c7b850684796bf4022c5b05e936d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:40.125505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-15bbccf8-4da1-4409-a595-d9d84ee1e41b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a418c7b850684796bf4022c5b05e936d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:40.141540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-15bbccf8-4da1-4409-a595-d9d84ee1e41b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a418c7b850684796bf4022c5b05e936d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:40.150432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531237a-03f6-462b-9308-3e07c0d10201 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:40.156725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4527699-76d5-4cd5-b0e5-7189fb9430e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:40.157540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4527699-76d5-4cd5-b0e5-7189fb9430e2 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:40.157923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4527699-76d5-4cd5-b0e5-7189fb9430e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:40.158332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4527699-76d5-4cd5-b0e5-7189fb9430e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:40.160218 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1644/3366] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:57:40 2022] DELETE /identity/v3/users/a418c7b850684796bf4022c5b05e936d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:40.163951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e4527699-76d5-4cd5-b0e5-7189fb9430e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:40.164892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f7aaa-fe0b-430c-937a-6beb1db38003 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:40.165554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f7aaa-fe0b-430c-937a-6beb1db38003 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:40.165554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f7aaa-fe0b-430c-937a-6beb1db38003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:40.177375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4527699-76d5-4cd5-b0e5-7189fb9430e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:40.178946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4527699-76d5-4cd5-b0e5-7189fb9430e2 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:40.187874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c76f7aaa-fe0b-430c-937a-6beb1db38003 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:40.189247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c76f7aaa-fe0b-430c-937a-6beb1db38003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:40.189837 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1723/3367] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:40.216186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:40.277160 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1645/3368] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:40.291239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4527699-76d5-4cd5-b0e5-7189fb9430e2 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:40.293621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e4527699-76d5-4cd5-b0e5-7189fb9430e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:40.309051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4527699-76d5-4cd5-b0e5-7189fb9430e2 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:40.382100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689981a6-82db-4e39-9487-bd7592e93bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:40.382933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689981a6-82db-4e39-9487-bd7592e93bce service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:40.383325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689981a6-82db-4e39-9487-bd7592e93bce service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:40.383697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689981a6-82db-4e39-9487-bd7592e93bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:40.396545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689981a6-82db-4e39-9487-bd7592e93bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:40.399619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689981a6-82db-4e39-9487-bd7592e93bce service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:40.423284 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1724/3369] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:40.878271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f7aaa-fe0b-430c-937a-6beb1db38003 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:40.881519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c76f7aaa-fe0b-430c-937a-6beb1db38003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:40.905474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f7aaa-fe0b-430c-937a-6beb1db38003 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:40.915283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb3c01-62d3-4e70-b87b-bb424dd52b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:40.916630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb3c01-62d3-4e70-b87b-bb424dd52b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:40.916630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb3c01-62d3-4e70-b87b-bb424dd52b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:40.916630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb3c01-62d3-4e70-b87b-bb424dd52b09 admin admin] PATH_INFO: `/v3/projects/49706c672bdd47d39405f8d7b80636ca` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:40.918452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb3c01-62d3-4e70-b87b-bb424dd52b09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49706c672bdd47d39405f8d7b80636ca)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:40.919647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb3c01-62d3-4e70-b87b-bb424dd52b09 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:40.986040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-79eb3c01-62d3-4e70-b87b-bb424dd52b09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49706c672bdd47d39405f8d7b80636ca', 'request_id': 'req-79eb3c01-62d3-4e70-b87b-bb424dd52b09'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:40.998018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-79eb3c01-62d3-4e70-b87b-bb424dd52b09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49706c672bdd47d39405f8d7b80636ca', 'request_id': 'req-79eb3c01-62d3-4e70-b87b-bb424dd52b09'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:41.006321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-79eb3c01-62d3-4e70-b87b-bb424dd52b09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49706c672bdd47d39405f8d7b80636ca', 'request_id': 'req-79eb3c01-62d3-4e70-b87b-bb424dd52b09'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:41.022815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-79eb3c01-62d3-4e70-b87b-bb424dd52b09 admin admin] The token cache is being invalidate because project 49706c672bdd47d39405f8d7b80636ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:57:41.023111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-79eb3c01-62d3-4e70-b87b-bb424dd52b09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:41.024021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-79eb3c01-62d3-4e70-b87b-bb424dd52b09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:41.025712 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1646/3370] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:57:40 2022] DELETE /identity/v3/projects/49706c672bdd47d39405f8d7b80636ca => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:41.033745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c672e-0b04-45c2-9fea-dc19c7473ad2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:41.035494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c672e-0b04-45c2-9fea-dc19c7473ad2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:41.036097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c672e-0b04-45c2-9fea-dc19c7473ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:41.060096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-716c672e-0b04-45c2-9fea-dc19c7473ad2 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:41.061869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-716c672e-0b04-45c2-9fea-dc19c7473ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:41.139017 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1725/3371] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:41.139211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cfe95-e146-4104-b5ce-3b2983a73110 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:41.139211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cfe95-e146-4104-b5ce-3b2983a73110 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:41.139211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cfe95-e146-4104-b5ce-3b2983a73110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:41.163032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a92cfe95-e146-4104-b5ce-3b2983a73110 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:41.164778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a92cfe95-e146-4104-b5ce-3b2983a73110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:41.174656 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1647/3372] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:41.180192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c672e-0b04-45c2-9fea-dc19c7473ad2 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:41.194707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c672e-0b04-45c2-9fea-dc19c7473ad2 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:41.200872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6e902-0cf5-4eb1-8b75-9cad61ff6f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:41.201596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6e902-0cf5-4eb1-8b75-9cad61ff6f1a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:41.201892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6e902-0cf5-4eb1-8b75-9cad61ff6f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:41.202168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6e902-0cf5-4eb1-8b75-9cad61ff6f1a admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:41.202866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6e902-0cf5-4eb1-8b75-9cad61ff6f1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:57:41.203214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6e902-0cf5-4eb1-8b75-9cad61ff6f1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:41.204811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6e902-0cf5-4eb1-8b75-9cad61ff6f1a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:41.224666 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1726/3373] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:57:41.225593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92cfe95-e146-4104-b5ce-3b2983a73110 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:41.251166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92cfe95-e146-4104-b5ce-3b2983a73110 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:41.271324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e94ac-46d9-4a22-8b8f-6d19a486f922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:41.272224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e94ac-46d9-4a22-8b8f-6d19a486f922 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:41.272697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e94ac-46d9-4a22-8b8f-6d19a486f922 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:41.273157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e94ac-46d9-4a22-8b8f-6d19a486f922 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:41.274648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e94ac-46d9-4a22-8b8f-6d19a486f922 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:41.279432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e94ac-46d9-4a22-8b8f-6d19a486f922 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:41.348190 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1648/3374] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:57:41 2022] POST /identity/v3/projects => generated 363 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:57:41.349626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6e902-0cf5-4eb1-8b75-9cad61ff6f1a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:41.362264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6e902-0cf5-4eb1-8b75-9cad61ff6f1a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:41.374385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:41.439960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cfd9a-f04f-44e8-926b-41fa10a9d705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:41.440784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cfd9a-f04f-44e8-926b-41fa10a9d705 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:41.441151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cfd9a-f04f-44e8-926b-41fa10a9d705 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:41.441423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cfd9a-f04f-44e8-926b-41fa10a9d705 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:41.442100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5cfd9a-f04f-44e8-926b-41fa10a9d705 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:41.444406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5cfd9a-f04f-44e8-926b-41fa10a9d705 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:41.470256 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1727/3375] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:57:41 2022] POST /identity/v3/users => generated 282 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:41.474975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e94ac-46d9-4a22-8b8f-6d19a486f922 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:41.486970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e94ac-46d9-4a22-8b8f-6d19a486f922 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:41.548965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3cd2a-b851-49f3-aa95-249a19798818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:41.551813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3cd2a-b851-49f3-aa95-249a19798818 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:41.552101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3cd2a-b851-49f3-aa95-249a19798818 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:41.552529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3cd2a-b851-49f3-aa95-249a19798818 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:41.554899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3cd2a-b851-49f3-aa95-249a19798818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:41.559873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3cd2a-b851-49f3-aa95-249a19798818 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:41.591928 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1649/3376] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:41.595928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cfd9a-f04f-44e8-926b-41fa10a9d705 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:41.611945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cfd9a-f04f-44e8-926b-41fa10a9d705 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:41.625603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224b476-e93a-4279-b0c1-c9c17b5aabf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:41.629205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224b476-e93a-4279-b0c1-c9c17b5aabf9 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:41.629502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224b476-e93a-4279-b0c1-c9c17b5aabf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:41.629782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224b476-e93a-4279-b0c1-c9c17b5aabf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:41.631883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7224b476-e93a-4279-b0c1-c9c17b5aabf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:41.637130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7224b476-e93a-4279-b0c1-c9c17b5aabf9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:41.678962 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1728/3377] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:41.683544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3cd2a-b851-49f3-aa95-249a19798818 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:41.704548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3cd2a-b851-49f3-aa95-249a19798818 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:41.712568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5283c27-662d-4db0-a1fd-c4898da2f7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:41.713573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5283c27-662d-4db0-a1fd-c4898da2f7ef admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:41.713859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5283c27-662d-4db0-a1fd-c4898da2f7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:41.714132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5283c27-662d-4db0-a1fd-c4898da2f7ef admin admin] PATH_INFO: `/v3/projects/5e3d28e616bc412da6a6698f461dd104/users/6249e0b4b3954668ad9313180a72cba5/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:41.748608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5283c27-662d-4db0-a1fd-c4898da2f7ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e3d28e616bc412da6a6698f461dd104, user_id=6249e0b4b3954668ad9313180a72cba5, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:41.757971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5283c27-662d-4db0-a1fd-c4898da2f7ef admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:41.791729 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1650/3378] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:41 2022] PUT /identity/v3/projects/5e3d28e616bc412da6a6698f461dd104/users/6249e0b4b3954668ad9313180a72cba5/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:41.805940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d69a4-222f-48df-abbf-654a48a57e76 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:41.806573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d69a4-222f-48df-abbf-654a48a57e76 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:41.806998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d69a4-222f-48df-abbf-654a48a57e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:41.918510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3e6d69a4-222f-48df-abbf-654a48a57e76 None None] MFA Rules not processed for user `6249e0b4b3954668ad9313180a72cba5`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:41.921457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3e6d69a4-222f-48df-abbf-654a48a57e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:42.070054 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1729/3379] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:42.080744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5283c27-662d-4db0-a1fd-c4898da2f7ef admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:42.082995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f5283c27-662d-4db0-a1fd-c4898da2f7ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:42.160365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:42.221804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5283c27-662d-4db0-a1fd-c4898da2f7ef admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:42.237114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b895fe-9b21-4906-b3a7-1506ee25e27b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:42.237758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b895fe-9b21-4906-b3a7-1506ee25e27b service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:42.238613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b895fe-9b21-4906-b3a7-1506ee25e27b service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:42.239320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b895fe-9b21-4906-b3a7-1506ee25e27b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:42.267021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b895fe-9b21-4906-b3a7-1506ee25e27b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:42.268676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b895fe-9b21-4906-b3a7-1506ee25e27b service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:42.288674 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1651/3380] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:42.305067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9642f-4eaf-4b01-9c29-ca6a33787d4f None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:42.305628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9642f-4eaf-4b01-9c29-ca6a33787d4f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:42.306068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9642f-4eaf-4b01-9c29-ca6a33787d4f None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:42.307833 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1730/3381] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:57:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:57:42.315439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b895fe-9b21-4906-b3a7-1506ee25e27b service nova] Authenticating service token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:57:42.331548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b895fe-9b21-4906-b3a7-1506ee25e27b service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:42.351538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b895fe-9b21-4906-b3a7-1506ee25e27b service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:42.364135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c80e8f-d00a-406b-b2f3-c26bdb8aa3fc tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6249e0b4b3954668ad9313180a72cba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e3d28e616bc412da6a6698f461dd104', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:42.364791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c80e8f-d00a-406b-b2f3-c26bdb8aa3fc tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:42.365074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c80e8f-d00a-406b-b2f3-c26bdb8aa3fc tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:42.365354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c80e8f-d00a-406b-b2f3-c26bdb8aa3fc tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] PATH_INFO: `/v3/projects/5e3d28e616bc412da6a6698f461dd104` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:42.367519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c80e8f-d00a-406b-b2f3-c26bdb8aa3fc tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] RBAC: Authorizing `identity:get_project(project_id=5e3d28e616bc412da6a6698f461dd104)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:42.374534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c80e8f-d00a-406b-b2f3-c26bdb8aa3fc tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:42.381814 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1652/3382] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:57:42 2022] GET /identity/v3/projects/5e3d28e616bc412da6a6698f461dd104 => generated 363 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:57:42.692305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f453d-e845-439c-949a-b2e44612f888 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:42.693784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f453d-e845-439c-949a-b2e44612f888 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:42.694227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f453d-e845-439c-949a-b2e44612f888 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:42.696362 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1731/3383] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:57:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:57:42.701753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c80e8f-d00a-406b-b2f3-c26bdb8aa3fc tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] Authenticating service token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:57:42.704384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-36c80e8f-d00a-406b-b2f3-c26bdb8aa3fc tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:42.726518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c80e8f-d00a-406b-b2f3-c26bdb8aa3fc tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:42.748202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c80e8f-d00a-406b-b2f3-c26bdb8aa3fc tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:42.756472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ec943-153d-476e-9b71-82c167434826 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6249e0b4b3954668ad9313180a72cba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e3d28e616bc412da6a6698f461dd104', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:42.757439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ec943-153d-476e-9b71-82c167434826 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:42.758039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ec943-153d-476e-9b71-82c167434826 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:42.758529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ec943-153d-476e-9b71-82c167434826 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] PATH_INFO: `/v3/projects/5e3d28e616bc412da6a6698f461dd104` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:42.761260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ec943-153d-476e-9b71-82c167434826 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] RBAC: Authorizing `identity:get_project(project_id=5e3d28e616bc412da6a6698f461dd104)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:42.766481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ec943-153d-476e-9b71-82c167434826 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:42.768997 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1653/3384] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:57:42 2022] GET /identity/v3/projects/5e3d28e616bc412da6a6698f461dd104 => generated 363 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:57:43.006358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafb288-0913-4ccd-a7a3-e47fd886f7d1 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:43.006725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafb288-0913-4ccd-a7a3-e47fd886f7d1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:43.006977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafb288-0913-4ccd-a7a3-e47fd886f7d1 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:43.008455 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1732/3385] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:57:43.019957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ec943-153d-476e-9b71-82c167434826 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] Authenticating service token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:57:43.051580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ec943-153d-476e-9b71-82c167434826 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:43.079159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ec943-153d-476e-9b71-82c167434826 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:43.090339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeae290-4aef-435f-b21b-7116fbce3654 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6249e0b4b3954668ad9313180a72cba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e3d28e616bc412da6a6698f461dd104', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:43.093971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeae290-4aef-435f-b21b-7116fbce3654 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:43.094261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeae290-4aef-435f-b21b-7116fbce3654 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:43.094558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeae290-4aef-435f-b21b-7116fbce3654 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] PATH_INFO: `/v3/projects/5e3d28e616bc412da6a6698f461dd104` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:43.095980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeae290-4aef-435f-b21b-7116fbce3654 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] RBAC: Authorizing `identity:get_project(project_id=5e3d28e616bc412da6a6698f461dd104)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:43.098462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeae290-4aef-435f-b21b-7116fbce3654 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:43.125588 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1654/3386] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:57:43 2022] GET /identity/v3/projects/5e3d28e616bc412da6a6698f461dd104 => generated 363 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:57:43.522440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340166dc-8aef-45d2-a5e1-85b2ea19f765 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:43.522808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340166dc-8aef-45d2-a5e1-85b2ea19f765 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:43.523102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340166dc-8aef-45d2-a5e1-85b2ea19f765 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:43.525175 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1733/3387] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:57:43.534102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeae290-4aef-435f-b21b-7116fbce3654 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] Authenticating service token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:57:43.579913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeae290-4aef-435f-b21b-7116fbce3654 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:43.601129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeae290-4aef-435f-b21b-7116fbce3654 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:43.606267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837717e-8390-4e95-a163-e9e5ade363ca tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6249e0b4b3954668ad9313180a72cba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e3d28e616bc412da6a6698f461dd104', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:43.607138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837717e-8390-4e95-a163-e9e5ade363ca tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:43.608706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837717e-8390-4e95-a163-e9e5ade363ca tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:43.609279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837717e-8390-4e95-a163-e9e5ade363ca tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] PATH_INFO: `/v3/projects/5e3d28e616bc412da6a6698f461dd104` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:43.612522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837717e-8390-4e95-a163-e9e5ade363ca tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] RBAC: Authorizing `identity:get_project(project_id=5e3d28e616bc412da6a6698f461dd104)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:43.619696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837717e-8390-4e95-a163-e9e5ade363ca tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:43.624196 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1655/3388] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:57:43 2022] GET /identity/v3/projects/5e3d28e616bc412da6a6698f461dd104 => generated 363 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:57:43.815399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed3272-ca43-480b-aa19-297472b0b9e8 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:43.815851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed3272-ca43-480b-aa19-297472b0b9e8 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:43.816824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed3272-ca43-480b-aa19-297472b0b9e8 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:43.818314 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1734/3389] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 20:57:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 20:57:43.827134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837717e-8390-4e95-a163-e9e5ade363ca tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] Authenticating service token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 20:57:43.873615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837717e-8390-4e95-a163-e9e5ade363ca tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:43.917320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837717e-8390-4e95-a163-e9e5ade363ca tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:43.932085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962fabd-aff2-4731-b945-7745bd2ca3a8 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6249e0b4b3954668ad9313180a72cba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e3d28e616bc412da6a6698f461dd104', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:43.932738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962fabd-aff2-4731-b945-7745bd2ca3a8 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:43.933036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962fabd-aff2-4731-b945-7745bd2ca3a8 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:43.933287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962fabd-aff2-4731-b945-7745bd2ca3a8 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] PATH_INFO: `/v3/projects/5e3d28e616bc412da6a6698f461dd104` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:43.936073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9962fabd-aff2-4731-b945-7745bd2ca3a8 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] RBAC: Authorizing `identity:get_project(project_id=5e3d28e616bc412da6a6698f461dd104)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:43.941592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9962fabd-aff2-4731-b945-7745bd2ca3a8 tempest-QuotasTestJSON-303158974 tempest-QuotasTestJSON-303158974-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:43.947236 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1656/3390] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 20:57:43 2022] GET /identity/v3/projects/5e3d28e616bc412da6a6698f461dd104 => generated 363 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:57:44.023782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed3272-ca43-480b-aa19-297472b0b9e8 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:44.186832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed3272-ca43-480b-aa19-297472b0b9e8 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:44.195980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39599ca-c3ac-4da3-90ff-40dc75116848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:44.199512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39599ca-c3ac-4da3-90ff-40dc75116848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:44.199914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39599ca-c3ac-4da3-90ff-40dc75116848 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:44.204610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39599ca-c3ac-4da3-90ff-40dc75116848 admin admin] PATH_INFO: `/v3/users/6249e0b4b3954668ad9313180a72cba5` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:44.208209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39599ca-c3ac-4da3-90ff-40dc75116848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6249e0b4b3954668ad9313180a72cba5)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:44.213181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39599ca-c3ac-4da3-90ff-40dc75116848 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:44.262841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:44.318915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c39599ca-c3ac-4da3-90ff-40dc75116848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6249e0b4b3954668ad9313180a72cba5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:44.328679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c39599ca-c3ac-4da3-90ff-40dc75116848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6249e0b4b3954668ad9313180a72cba5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:44.356245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c39599ca-c3ac-4da3-90ff-40dc75116848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6249e0b4b3954668ad9313180a72cba5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:44.357068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c39599ca-c3ac-4da3-90ff-40dc75116848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6249e0b4b3954668ad9313180a72cba5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:44.383985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c39599ca-c3ac-4da3-90ff-40dc75116848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6249e0b4b3954668ad9313180a72cba5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:44.412233 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1735/3391] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:57:44 2022] DELETE /identity/v3/users/6249e0b4b3954668ad9313180a72cba5 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:57:44.416516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e2d6e-bc49-4647-99e3-26260838b589 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:44.416798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e2d6e-bc49-4647-99e3-26260838b589 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:44.417076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e2d6e-bc49-4647-99e3-26260838b589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:44.449212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f81e2d6e-bc49-4647-99e3-26260838b589 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:44.450616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f81e2d6e-bc49-4647-99e3-26260838b589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:44.541684 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1657/3392] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:44.612548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39599ca-c3ac-4da3-90ff-40dc75116848 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:44.615246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c39599ca-c3ac-4da3-90ff-40dc75116848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:44.731353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39599ca-c3ac-4da3-90ff-40dc75116848 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:44.736174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8864828-0e90-45b5-a1e2-c436e2c6dcf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:44.737370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8864828-0e90-45b5-a1e2-c436e2c6dcf4 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:44.738435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8864828-0e90-45b5-a1e2-c436e2c6dcf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:44.738911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8864828-0e90-45b5-a1e2-c436e2c6dcf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:44.751951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8864828-0e90-45b5-a1e2-c436e2c6dcf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:44.755175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8864828-0e90-45b5-a1e2-c436e2c6dcf4 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:44.769742 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1736/3393] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:45.528738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e2d6e-bc49-4647-99e3-26260838b589 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:45.530166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f81e2d6e-bc49-4647-99e3-26260838b589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:45.543919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e2d6e-bc49-4647-99e3-26260838b589 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:45.551087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73aa42-1fc0-4d0a-b4fd-8bca3d51d6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:45.551808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73aa42-1fc0-4d0a-b4fd-8bca3d51d6b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:45.552157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73aa42-1fc0-4d0a-b4fd-8bca3d51d6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:45.552681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73aa42-1fc0-4d0a-b4fd-8bca3d51d6b7 admin admin] PATH_INFO: `/v3/projects/5e3d28e616bc412da6a6698f461dd104` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:45.553903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73aa42-1fc0-4d0a-b4fd-8bca3d51d6b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e3d28e616bc412da6a6698f461dd104)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:45.556727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73aa42-1fc0-4d0a-b4fd-8bca3d51d6b7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:45.624329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ab73aa42-1fc0-4d0a-b4fd-8bca3d51d6b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e3d28e616bc412da6a6698f461dd104', 'request_id': 'req-ab73aa42-1fc0-4d0a-b4fd-8bca3d51d6b7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:45.633801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ab73aa42-1fc0-4d0a-b4fd-8bca3d51d6b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e3d28e616bc412da6a6698f461dd104', 'request_id': 'req-ab73aa42-1fc0-4d0a-b4fd-8bca3d51d6b7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:45.646023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ab73aa42-1fc0-4d0a-b4fd-8bca3d51d6b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e3d28e616bc412da6a6698f461dd104', 'request_id': 'req-ab73aa42-1fc0-4d0a-b4fd-8bca3d51d6b7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:45.662342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ab73aa42-1fc0-4d0a-b4fd-8bca3d51d6b7 admin admin] The token cache is being invalidate because project 5e3d28e616bc412da6a6698f461dd104 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:57:45.662884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ab73aa42-1fc0-4d0a-b4fd-8bca3d51d6b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:45.664842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ab73aa42-1fc0-4d0a-b4fd-8bca3d51d6b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:45.666763 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1658/3394] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:57:45 2022] DELETE /identity/v3/projects/5e3d28e616bc412da6a6698f461dd104 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:45.675179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30b3f0-4743-4130-be03-8936cded62ec None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:45.676454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30b3f0-4743-4130-be03-8936cded62ec None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:45.677059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30b3f0-4743-4130-be03-8936cded62ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:45.718967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6d30b3f0-4743-4130-be03-8936cded62ec None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:45.721436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6d30b3f0-4743-4130-be03-8936cded62ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:45.821165 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1737/3395] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:45.829475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7134a5a-2456-41d8-9798-416efbbce2d4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:45.829768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7134a5a-2456-41d8-9798-416efbbce2d4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:45.830047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7134a5a-2456-41d8-9798-416efbbce2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:45.849627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:45.858136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f7134a5a-2456-41d8-9798-416efbbce2d4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:45.861120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f7134a5a-2456-41d8-9798-416efbbce2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:45.875429 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1659/3396] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:45.879473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d30b3f0-4743-4130-be03-8936cded62ec None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:45.892606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d30b3f0-4743-4130-be03-8936cded62ec None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:45.900346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29ddae-d634-4093-ab04-0b79ef6a9f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:45.901520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29ddae-d634-4093-ab04-0b79ef6a9f18 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:45.903133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29ddae-d634-4093-ab04-0b79ef6a9f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:45.903442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29ddae-d634-4093-ab04-0b79ef6a9f18 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:45.904123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29ddae-d634-4093-ab04-0b79ef6a9f18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:57:45.904902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29ddae-d634-4093-ab04-0b79ef6a9f18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:45.907313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29ddae-d634-4093-ab04-0b79ef6a9f18 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:45.928155 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1738/3397] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:57:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:57:45.928837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7134a5a-2456-41d8-9798-416efbbce2d4 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:45.939686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7134a5a-2456-41d8-9798-416efbbce2d4 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:45.944358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925323e-e220-4b36-b034-abf6ff17aae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:45.944970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925323e-e220-4b36-b034-abf6ff17aae0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:45.945253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925323e-e220-4b36-b034-abf6ff17aae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:45.945530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925323e-e220-4b36-b034-abf6ff17aae0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:45.946321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925323e-e220-4b36-b034-abf6ff17aae0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:45.948447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925323e-e220-4b36-b034-abf6ff17aae0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:45.986963 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1660/3398] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:57:45 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:45.991055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29ddae-d634-4093-ab04-0b79ef6a9f18 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:46.005812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29ddae-d634-4093-ab04-0b79ef6a9f18 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:46.080341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:46.162431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45207dd0-7920-41d3-85fd-6043a8323feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:46.164333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45207dd0-7920-41d3-85fd-6043a8323feb admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:46.164723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45207dd0-7920-41d3-85fd-6043a8323feb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:46.166111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45207dd0-7920-41d3-85fd-6043a8323feb admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:46.169468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45207dd0-7920-41d3-85fd-6043a8323feb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:46.178665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45207dd0-7920-41d3-85fd-6043a8323feb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:46.224071 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1739/3399] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:57:45 2022] POST /identity/v3/users => generated 294 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:46.224366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925323e-e220-4b36-b034-abf6ff17aae0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:46.237069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925323e-e220-4b36-b034-abf6ff17aae0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:46.254679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553172b-395b-4023-aa3f-df920cec08c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:46.255422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553172b-395b-4023-aa3f-df920cec08c1 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:46.255783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553172b-395b-4023-aa3f-df920cec08c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:46.256277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553172b-395b-4023-aa3f-df920cec08c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:46.257024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553172b-395b-4023-aa3f-df920cec08c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:46.259791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553172b-395b-4023-aa3f-df920cec08c1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:46.276259 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1661/3400] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:57:46.280119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45207dd0-7920-41d3-85fd-6043a8323feb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:46.302195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45207dd0-7920-41d3-85fd-6043a8323feb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:46.310132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76cd43-c2ab-411b-8385-d3f88e1aebe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:46.311039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76cd43-c2ab-411b-8385-d3f88e1aebe5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:46.311321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76cd43-c2ab-411b-8385-d3f88e1aebe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:46.311597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76cd43-c2ab-411b-8385-d3f88e1aebe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:46.312505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76cd43-c2ab-411b-8385-d3f88e1aebe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:46.314264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76cd43-c2ab-411b-8385-d3f88e1aebe5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:46.332748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553172b-395b-4023-aa3f-df920cec08c1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:46.345781 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1740/3401] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:46.346458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553172b-395b-4023-aa3f-df920cec08c1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:46.354533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50443ff2-e94f-413c-99e5-679d9846a0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:46.355853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50443ff2-e94f-413c-99e5-679d9846a0c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:46.356199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50443ff2-e94f-413c-99e5-679d9846a0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:46.356481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50443ff2-e94f-413c-99e5-679d9846a0c7 admin admin] PATH_INFO: `/v3/projects/991a33671da640269295c9eafd6f167a/users/b7d945310c5b48278b73125c2d328f0a/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:46.377786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50443ff2-e94f-413c-99e5-679d9846a0c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=991a33671da640269295c9eafd6f167a, user_id=b7d945310c5b48278b73125c2d328f0a, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:46.381140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50443ff2-e94f-413c-99e5-679d9846a0c7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:46.398702 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1662/3402] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:46 2022] PUT /identity/v3/projects/991a33671da640269295c9eafd6f167a/users/b7d945310c5b48278b73125c2d328f0a/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:57:46.403806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d84af-45be-4477-b30f-449b46b272ec None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:46.404094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d84af-45be-4477-b30f-449b46b272ec None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:46.404421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d84af-45be-4477-b30f-449b46b272ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:46.436427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-630d84af-45be-4477-b30f-449b46b272ec None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:46.437662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-630d84af-45be-4477-b30f-449b46b272ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:46.537059 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1741/3403] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:57:46.549173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50443ff2-e94f-413c-99e5-679d9846a0c7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:46.551740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-50443ff2-e94f-413c-99e5-679d9846a0c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:46.638861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50443ff2-e94f-413c-99e5-679d9846a0c7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:46.643333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763445c-3808-41b1-a50a-b8c997cdfe0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:46.644225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763445c-3808-41b1-a50a-b8c997cdfe0c service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:46.646197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763445c-3808-41b1-a50a-b8c997cdfe0c service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:46.646755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763445c-3808-41b1-a50a-b8c997cdfe0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:46.657560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5763445c-3808-41b1-a50a-b8c997cdfe0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:46.659378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5763445c-3808-41b1-a50a-b8c997cdfe0c service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:46.672643 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1663/3404] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:57:47.372281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f73a5c-f86f-4e7d-9382-157577926488 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:47.375263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f73a5c-f86f-4e7d-9382-157577926488 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:47.375825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f73a5c-f86f-4e7d-9382-157577926488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:47.467121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b5f73a5c-f86f-4e7d-9382-157577926488 None None] MFA Rules not processed for user `b7d945310c5b48278b73125c2d328f0a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:47.468752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b5f73a5c-f86f-4e7d-9382-157577926488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:47.601809 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1742/3405] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:57:47.614025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf193ad-8f2d-4159-a243-0d18367f1e04 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:47.615054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf193ad-8f2d-4159-a243-0d18367f1e04 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:47.615474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf193ad-8f2d-4159-a243-0d18367f1e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:47.647679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-daf193ad-8f2d-4159-a243-0d18367f1e04 None None] MFA Rules not processed for user `b7d945310c5b48278b73125c2d328f0a`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:47.651621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-daf193ad-8f2d-4159-a243-0d18367f1e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:47.678596 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1664/3406] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:47.977840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f73a5c-f86f-4e7d-9382-157577926488 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:47.979581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b5f73a5c-f86f-4e7d-9382-157577926488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:48.086949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:48.128050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f73a5c-f86f-4e7d-9382-157577926488 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:48.135087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66f42b-2006-4638-a544-f7b8eb364a36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:48.135907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66f42b-2006-4638-a544-f7b8eb364a36 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:48.136291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66f42b-2006-4638-a544-f7b8eb364a36 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:48.136669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66f42b-2006-4638-a544-f7b8eb364a36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:48.145923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66f42b-2006-4638-a544-f7b8eb364a36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:48.147780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66f42b-2006-4638-a544-f7b8eb364a36 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:48.160215 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1743/3407] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:49.965912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf193ad-8f2d-4159-a243-0d18367f1e04 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:49.982644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf193ad-8f2d-4159-a243-0d18367f1e04 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:49.999647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbe080-1ddd-4b0a-95ab-c8f8243f81d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:50.000698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbe080-1ddd-4b0a-95ab-c8f8243f81d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:50.001267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbe080-1ddd-4b0a-95ab-c8f8243f81d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:50.001734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbe080-1ddd-4b0a-95ab-c8f8243f81d2 admin admin] PATH_INFO: `/v3/users/b7d945310c5b48278b73125c2d328f0a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:50.004919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbe080-1ddd-4b0a-95ab-c8f8243f81d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7d945310c5b48278b73125c2d328f0a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:50.014448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbe080-1ddd-4b0a-95ab-c8f8243f81d2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:50.106410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a8bbe080-1ddd-4b0a-95ab-c8f8243f81d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7d945310c5b48278b73125c2d328f0a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:50.108366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a8bbe080-1ddd-4b0a-95ab-c8f8243f81d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7d945310c5b48278b73125c2d328f0a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:50.109125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a8bbe080-1ddd-4b0a-95ab-c8f8243f81d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7d945310c5b48278b73125c2d328f0a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:50.119830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a8bbe080-1ddd-4b0a-95ab-c8f8243f81d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7d945310c5b48278b73125c2d328f0a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:50.138296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a8bbe080-1ddd-4b0a-95ab-c8f8243f81d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7d945310c5b48278b73125c2d328f0a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:50.154149 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1665/3408] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:57:49 2022] DELETE /identity/v3/users/b7d945310c5b48278b73125c2d328f0a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:50.552122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66f42b-2006-4638-a544-f7b8eb364a36 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:50.576847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66f42b-2006-4638-a544-f7b8eb364a36 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:50.751197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aef2ff-e48f-48c1-aac3-4a448d6f9a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:50.752320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aef2ff-e48f-48c1-aac3-4a448d6f9a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:50.752841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aef2ff-e48f-48c1-aac3-4a448d6f9a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:50.753879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aef2ff-e48f-48c1-aac3-4a448d6f9a71 admin admin] PATH_INFO: `/v3/projects/991a33671da640269295c9eafd6f167a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:50.755327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aef2ff-e48f-48c1-aac3-4a448d6f9a71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=991a33671da640269295c9eafd6f167a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:50.757757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aef2ff-e48f-48c1-aac3-4a448d6f9a71 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:50.831195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d9aef2ff-e48f-48c1-aac3-4a448d6f9a71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '991a33671da640269295c9eafd6f167a', 'request_id': 'req-d9aef2ff-e48f-48c1-aac3-4a448d6f9a71'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:50.838999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d9aef2ff-e48f-48c1-aac3-4a448d6f9a71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '991a33671da640269295c9eafd6f167a', 'request_id': 'req-d9aef2ff-e48f-48c1-aac3-4a448d6f9a71'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:50.850455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d9aef2ff-e48f-48c1-aac3-4a448d6f9a71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '991a33671da640269295c9eafd6f167a', 'request_id': 'req-d9aef2ff-e48f-48c1-aac3-4a448d6f9a71'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:50.868972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d9aef2ff-e48f-48c1-aac3-4a448d6f9a71 admin admin] The token cache is being invalidate because project 991a33671da640269295c9eafd6f167a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:57:50.869648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d9aef2ff-e48f-48c1-aac3-4a448d6f9a71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:50.871087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d9aef2ff-e48f-48c1-aac3-4a448d6f9a71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:50.874016 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1744/3409] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:57:50 2022] DELETE /identity/v3/projects/991a33671da640269295c9eafd6f167a => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:50.883693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4452c42-7df4-42b8-b0fd-926f8028db4b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:50.884520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4452c42-7df4-42b8-b0fd-926f8028db4b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:50.884920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4452c42-7df4-42b8-b0fd-926f8028db4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:50.934519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c4452c42-7df4-42b8-b0fd-926f8028db4b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:50.936218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c4452c42-7df4-42b8-b0fd-926f8028db4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:50.953049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:51.031204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aef2ff-e48f-48c1-aac3-4a448d6f9a71 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:51.032887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d9aef2ff-e48f-48c1-aac3-4a448d6f9a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:51.106541 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1666/3410] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:51.113951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be845ed-1b3a-486e-b4be-04d8c7abf3df None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:51.114510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be845ed-1b3a-486e-b4be-04d8c7abf3df None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:51.114964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be845ed-1b3a-486e-b4be-04d8c7abf3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:51.169204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7be845ed-1b3a-486e-b4be-04d8c7abf3df None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:51.171277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7be845ed-1b3a-486e-b4be-04d8c7abf3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:51.171887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:51.181530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aef2ff-e48f-48c1-aac3-4a448d6f9a71 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:51.196977 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1667/3411] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:51.197199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efdddd-e50e-428c-b8ea-188f22a68d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:51.197668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efdddd-e50e-428c-b8ea-188f22a68d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:51.197956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efdddd-e50e-428c-b8ea-188f22a68d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:51.198236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efdddd-e50e-428c-b8ea-188f22a68d15 admin admin] PATH_INFO: `/v3/users/2aa4f468a15546eb9f72cb2b632e67bb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:51.199475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1efdddd-e50e-428c-b8ea-188f22a68d15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aa4f468a15546eb9f72cb2b632e67bb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:51.200529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be845ed-1b3a-486e-b4be-04d8c7abf3df None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:51.208385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1efdddd-e50e-428c-b8ea-188f22a68d15 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:51.223501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be845ed-1b3a-486e-b4be-04d8c7abf3df None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:51.235133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7628640-4d2b-44ba-923c-d098b9cf4d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:51.236251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7628640-4d2b-44ba-923c-d098b9cf4d7c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:51.236737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7628640-4d2b-44ba-923c-d098b9cf4d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:51.237213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7628640-4d2b-44ba-923c-d098b9cf4d7c admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:51.238634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7628640-4d2b-44ba-923c-d098b9cf4d7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:57:51.239152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7628640-4d2b-44ba-923c-d098b9cf4d7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:51.241694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7628640-4d2b-44ba-923c-d098b9cf4d7c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:51.291885 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1668/3412] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:57:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:57:51.295432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7628640-4d2b-44ba-923c-d098b9cf4d7c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:51.301640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b1efdddd-e50e-428c-b8ea-188f22a68d15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aa4f468a15546eb9f72cb2b632e67bb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:51.317747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b1efdddd-e50e-428c-b8ea-188f22a68d15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aa4f468a15546eb9f72cb2b632e67bb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:51.336136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7628640-4d2b-44ba-923c-d098b9cf4d7c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:51.345641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b1efdddd-e50e-428c-b8ea-188f22a68d15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aa4f468a15546eb9f72cb2b632e67bb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:51.346370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b1efdddd-e50e-428c-b8ea-188f22a68d15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aa4f468a15546eb9f72cb2b632e67bb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:51.354262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5762ef1-7ba6-4aa7-ac99-f47f01e5541b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:51.355554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5762ef1-7ba6-4aa7-ac99-f47f01e5541b admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:51.355842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5762ef1-7ba6-4aa7-ac99-f47f01e5541b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:51.359549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5762ef1-7ba6-4aa7-ac99-f47f01e5541b admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:51.361497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5762ef1-7ba6-4aa7-ac99-f47f01e5541b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:51.368283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b1efdddd-e50e-428c-b8ea-188f22a68d15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aa4f468a15546eb9f72cb2b632e67bb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:51.370565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5762ef1-7ba6-4aa7-ac99-f47f01e5541b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:51.376953 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1745/3413] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:57:51 2022] DELETE /identity/v3/users/2aa4f468a15546eb9f72cb2b632e67bb => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:57:51.457513 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1669/3414] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:57:51 2022] POST /identity/v3/projects => generated 371 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:51.466674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efdddd-e50e-428c-b8ea-188f22a68d15 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:51.469549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b1efdddd-e50e-428c-b8ea-188f22a68d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:51.580127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efdddd-e50e-428c-b8ea-188f22a68d15 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:51.713118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbc138-94f2-458e-a5b5-d1bde039b5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:51.713770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbc138-94f2-458e-a5b5-d1bde039b5d3 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:51.714008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbc138-94f2-458e-a5b5-d1bde039b5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:51.714291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbc138-94f2-458e-a5b5-d1bde039b5d3 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:51.714990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bbc138-94f2-458e-a5b5-d1bde039b5d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:51.738067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bbc138-94f2-458e-a5b5-d1bde039b5d3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:51.778552 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1746/3415] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:57:51 2022] POST /identity/v3/users => generated 286 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:51.779870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5762ef1-7ba6-4aa7-ac99-f47f01e5541b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:51.791578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5762ef1-7ba6-4aa7-ac99-f47f01e5541b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:51.805989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359c613-7b4c-4763-bd12-72a9b3298d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:51.807303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359c613-7b4c-4763-bd12-72a9b3298d4f admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:51.807584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359c613-7b4c-4763-bd12-72a9b3298d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:51.807859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359c613-7b4c-4763-bd12-72a9b3298d4f admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:51.808516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359c613-7b4c-4763-bd12-72a9b3298d4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:51.810429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359c613-7b4c-4763-bd12-72a9b3298d4f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:51.852573 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1670/3416] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:51.854920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbc138-94f2-458e-a5b5-d1bde039b5d3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:51.867622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbc138-94f2-458e-a5b5-d1bde039b5d3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:51.873001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9cee57-9a7a-4068-bc0d-a83d79d2d00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:51.873759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9cee57-9a7a-4068-bc0d-a83d79d2d00b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:51.874136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9cee57-9a7a-4068-bc0d-a83d79d2d00b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:51.874589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9cee57-9a7a-4068-bc0d-a83d79d2d00b admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:51.875312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9cee57-9a7a-4068-bc0d-a83d79d2d00b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:51.880645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9cee57-9a7a-4068-bc0d-a83d79d2d00b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:51.889407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb50697-bef8-4958-b5ba-3a2dfe7e9af0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:51.889721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb50697-bef8-4958-b5ba-3a2dfe7e9af0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:51.890193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb50697-bef8-4958-b5ba-3a2dfe7e9af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:51.901815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:51.913599 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1747/3417] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:51.914996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9cee57-9a7a-4068-bc0d-a83d79d2d00b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:51.928796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9cee57-9a7a-4068-bc0d-a83d79d2d00b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:51.944792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955fea4d-483f-40a7-9b37-61f27bdfe14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:51.945966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fea4d-483f-40a7-9b37-61f27bdfe14f admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:51.946431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fea4d-483f-40a7-9b37-61f27bdfe14f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:51.946934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fea4d-483f-40a7-9b37-61f27bdfe14f admin admin] PATH_INFO: `/v3/projects/3e6f03ca2bb44f89b6f0b3c3adce1558/users/46af47ed34344f09ad9c54ed16e392ee/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:51.970620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955fea4d-483f-40a7-9b37-61f27bdfe14f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e6f03ca2bb44f89b6f0b3c3adce1558, user_id=46af47ed34344f09ad9c54ed16e392ee, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:51.973991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2bb50697-bef8-4958-b5ba-3a2dfe7e9af0 None None] MFA Rules not processed for user `ccbf2c814364466baca8dfcabd2c13f9`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:51.975136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955fea4d-483f-40a7-9b37-61f27bdfe14f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:51.975393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2bb50697-bef8-4958-b5ba-3a2dfe7e9af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:52.010636 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1748/3418] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:51 2022] PUT /identity/v3/projects/3e6f03ca2bb44f89b6f0b3c3adce1558/users/46af47ed34344f09ad9c54ed16e392ee/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:52.013713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de305e87-8a4f-4950-9906-adeb3fd26463 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:52.013713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de305e87-8a4f-4950-9906-adeb3fd26463 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:52.013713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de305e87-8a4f-4950-9906-adeb3fd26463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:52.114236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-de305e87-8a4f-4950-9906-adeb3fd26463 None None] MFA Rules not processed for user `46af47ed34344f09ad9c54ed16e392ee`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:52.115509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-de305e87-8a4f-4950-9906-adeb3fd26463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:52.132362 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1671/3419] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:57:52.137386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb50697-bef8-4958-b5ba-3a2dfe7e9af0 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:52.140526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2bb50697-bef8-4958-b5ba-3a2dfe7e9af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:52.169835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb50697-bef8-4958-b5ba-3a2dfe7e9af0 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:52.270466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:52.294458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac84e25-514d-4b56-83b6-fc0f4e377cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:52.295622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac84e25-514d-4b56-83b6-fc0f4e377cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:52.296074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac84e25-514d-4b56-83b6-fc0f4e377cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:52.296482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac84e25-514d-4b56-83b6-fc0f4e377cc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:52.298851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac84e25-514d-4b56-83b6-fc0f4e377cc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:52.301798 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1749/3420] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:52.302360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac84e25-514d-4b56-83b6-fc0f4e377cc2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:52.303439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de305e87-8a4f-4950-9906-adeb3fd26463 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:52.304838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-de305e87-8a4f-4950-9906-adeb3fd26463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:52.322068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de305e87-8a4f-4950-9906-adeb3fd26463 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:52.328362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65f902-5c6b-45b2-b9fb-020389e48b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:52.329375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65f902-5c6b-45b2-b9fb-020389e48b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:52.329689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65f902-5c6b-45b2-b9fb-020389e48b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:52.330076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65f902-5c6b-45b2-b9fb-020389e48b1c admin admin] PATH_INFO: `/v3/projects/bb36a4c325b54c8a964113ba5e52f47b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:52.331495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65f902-5c6b-45b2-b9fb-020389e48b1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb36a4c325b54c8a964113ba5e52f47b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:52.333952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65f902-5c6b-45b2-b9fb-020389e48b1c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:52.342537 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1672/3421] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:57:52 2022] POST /identity/v3/projects => generated 383 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:57:52.343340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac84e25-514d-4b56-83b6-fc0f4e377cc2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:52.411280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2d65f902-5c6b-45b2-b9fb-020389e48b1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb36a4c325b54c8a964113ba5e52f47b', 'request_id': 'req-2d65f902-5c6b-45b2-b9fb-020389e48b1c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:52.420231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2d65f902-5c6b-45b2-b9fb-020389e48b1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb36a4c325b54c8a964113ba5e52f47b', 'request_id': 'req-2d65f902-5c6b-45b2-b9fb-020389e48b1c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:52.422253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac84e25-514d-4b56-83b6-fc0f4e377cc2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:52.427873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2d65f902-5c6b-45b2-b9fb-020389e48b1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb36a4c325b54c8a964113ba5e52f47b', 'request_id': 'req-2d65f902-5c6b-45b2-b9fb-020389e48b1c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:52.458668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2d65f902-5c6b-45b2-b9fb-020389e48b1c admin admin] The token cache is being invalidate because project bb36a4c325b54c8a964113ba5e52f47b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:57:52.459156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2d65f902-5c6b-45b2-b9fb-020389e48b1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:52.462099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2d65f902-5c6b-45b2-b9fb-020389e48b1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:52.466138 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1750/3422] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:57:52 2022] DELETE /identity/v3/projects/bb36a4c325b54c8a964113ba5e52f47b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:52.466871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65f902-5c6b-45b2-b9fb-020389e48b1c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:52.468420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2d65f902-5c6b-45b2-b9fb-020389e48b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:52.506246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4546a61-c33c-4a08-b1ec-4582ae8b2f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:52.507264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4546a61-c33c-4a08-b1ec-4582ae8b2f08 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:52.507743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4546a61-c33c-4a08-b1ec-4582ae8b2f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:52.508283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4546a61-c33c-4a08-b1ec-4582ae8b2f08 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:52.509139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4546a61-c33c-4a08-b1ec-4582ae8b2f08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:52.511458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4546a61-c33c-4a08-b1ec-4582ae8b2f08 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:52.555366 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1673/3423] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:57:52 2022] POST /identity/v3/projects => generated 371 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:52.558448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4546a61-c33c-4a08-b1ec-4582ae8b2f08 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:52.563502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e4546a61-c33c-4a08-b1ec-4582ae8b2f08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:52.582621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:52.617601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65f902-5c6b-45b2-b9fb-020389e48b1c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:52.695733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4546a61-c33c-4a08-b1ec-4582ae8b2f08 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:52.711797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b97f6-761c-499c-b98b-67a4f6f39c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:52.712757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b97f6-761c-499c-b98b-67a4f6f39c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:52.713080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b97f6-761c-499c-b98b-67a4f6f39c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:52.713361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b97f6-761c-499c-b98b-67a4f6f39c2d admin admin] PATH_INFO: `/v3/users/85ba386a1dad4f43b6eb3f36b588c09e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:52.729062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b97f6-761c-499c-b98b-67a4f6f39c2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=85ba386a1dad4f43b6eb3f36b588c09e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:52.732293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecefdfe8-68ee-4142-b8f8-d61cd0da6069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:52.733903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefdfe8-68ee-4142-b8f8-d61cd0da6069 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:52.734405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefdfe8-68ee-4142-b8f8-d61cd0da6069 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:52.735481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b97f6-761c-499c-b98b-67a4f6f39c2d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:52.735719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefdfe8-68ee-4142-b8f8-d61cd0da6069 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:52.736666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecefdfe8-68ee-4142-b8f8-d61cd0da6069 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:52.740261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecefdfe8-68ee-4142-b8f8-d61cd0da6069 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:52.748808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:52.785187 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1751/3424] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:57:52 2022] POST /identity/v3/users => generated 298 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:52.786388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecefdfe8-68ee-4142-b8f8-d61cd0da6069 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:52.790132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ecefdfe8-68ee-4142-b8f8-d61cd0da6069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:52.812285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecefdfe8-68ee-4142-b8f8-d61cd0da6069 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:52.836442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5ca64-38df-4cd6-9f5a-ce95e060e7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:52.843195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5ca64-38df-4cd6-9f5a-ce95e060e7b0 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:52.845194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5ca64-38df-4cd6-9f5a-ce95e060e7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:52.845632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5ca64-38df-4cd6-9f5a-ce95e060e7b0 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:52.847581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d5ca64-38df-4cd6-9f5a-ce95e060e7b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:52.849871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d5ca64-38df-4cd6-9f5a-ce95e060e7b0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:52.852511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8b3b97f6-761c-499c-b98b-67a4f6f39c2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85ba386a1dad4f43b6eb3f36b588c09e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:52.853828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8b3b97f6-761c-499c-b98b-67a4f6f39c2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85ba386a1dad4f43b6eb3f36b588c09e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:52.855625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8b3b97f6-761c-499c-b98b-67a4f6f39c2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85ba386a1dad4f43b6eb3f36b588c09e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:52.871673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8b3b97f6-761c-499c-b98b-67a4f6f39c2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85ba386a1dad4f43b6eb3f36b588c09e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:52.886743 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1752/3425] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:57:52 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:52.888039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5ca64-38df-4cd6-9f5a-ce95e060e7b0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:52.890633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d2d5ca64-38df-4cd6-9f5a-ce95e060e7b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:52.907511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8b3b97f6-761c-499c-b98b-67a4f6f39c2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85ba386a1dad4f43b6eb3f36b588c09e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:52.909380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5ca64-38df-4cd6-9f5a-ce95e060e7b0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:52.916627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7a629-9e1a-44ff-80cf-c4989d4e3dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:52.917689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7a629-9e1a-44ff-80cf-c4989d4e3dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:52.918014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7a629-9e1a-44ff-80cf-c4989d4e3dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:52.918350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7a629-9e1a-44ff-80cf-c4989d4e3dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:52.919174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7a629-9e1a-44ff-80cf-c4989d4e3dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:52.922393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7a629-9e1a-44ff-80cf-c4989d4e3dd5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:52.937350 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1753/3426] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:52.938129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7a629-9e1a-44ff-80cf-c4989d4e3dd5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:52.947663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ced7a629-9e1a-44ff-80cf-c4989d4e3dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:52.955709 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1674/3427] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:57:52 2022] DELETE /identity/v3/users/85ba386a1dad4f43b6eb3f36b588c09e => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:52.959864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b97f6-761c-499c-b98b-67a4f6f39c2d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:52.983826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7a629-9e1a-44ff-80cf-c4989d4e3dd5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:53.085830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b97f6-761c-499c-b98b-67a4f6f39c2d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:53.096889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fc089-98e0-4c64-afba-4aae59e88722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:53.099483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fc089-98e0-4c64-afba-4aae59e88722 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:53.100012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fc089-98e0-4c64-afba-4aae59e88722 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:53.101371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fc089-98e0-4c64-afba-4aae59e88722 admin admin] PATH_INFO: `/v3/projects/24a9d88d60164b2785bacc9f8bfa3c05/users/0284a8ec6c4e41f0a1a88e2f11a4faa8/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:53.149837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f046b57-dbb1-48b5-860f-7ed735d2db8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:53.150549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f046b57-dbb1-48b5-860f-7ed735d2db8a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:53.150841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f046b57-dbb1-48b5-860f-7ed735d2db8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:53.151116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f046b57-dbb1-48b5-860f-7ed735d2db8a admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:53.156865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fc089-98e0-4c64-afba-4aae59e88722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24a9d88d60164b2785bacc9f8bfa3c05, user_id=0284a8ec6c4e41f0a1a88e2f11a4faa8, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:53.157542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f046b57-dbb1-48b5-860f-7ed735d2db8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:53.163606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f046b57-dbb1-48b5-860f-7ed735d2db8a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:53.166073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fc089-98e0-4c64-afba-4aae59e88722 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:53.198598 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1754/3428] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:53.200478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f046b57-dbb1-48b5-860f-7ed735d2db8a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:53.205621 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1675/3429] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:52 2022] PUT /identity/v3/projects/24a9d88d60164b2785bacc9f8bfa3c05/users/0284a8ec6c4e41f0a1a88e2f11a4faa8/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:53.210971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fc089-98e0-4c64-afba-4aae59e88722 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:53.247133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f046b57-dbb1-48b5-860f-7ed735d2db8a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:53.349416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:53.351224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fc089-98e0-4c64-afba-4aae59e88722 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:53.359247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e7ab1-c5dc-44f3-991b-53a039f589d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:53.359873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e7ab1-c5dc-44f3-991b-53a039f589d7 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:53.360156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e7ab1-c5dc-44f3-991b-53a039f589d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:53.360429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e7ab1-c5dc-44f3-991b-53a039f589d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:53.361261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e7ab1-c5dc-44f3-991b-53a039f589d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:53.364285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e7ab1-c5dc-44f3-991b-53a039f589d7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:53.365517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0a041-3f79-4143-afd9-d6fa3244bb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:53.366111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0a041-3f79-4143-afd9-d6fa3244bb08 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:53.366292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0a041-3f79-4143-afd9-d6fa3244bb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:53.366748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0a041-3f79-4143-afd9-d6fa3244bb08 admin admin] PATH_INFO: `/v3/projects/1557c02911274e9ab1b6a5e95b045f91/users/33704d3fe1c14d8ea89aa999b2cb5d0c/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:53.387883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0a041-3f79-4143-afd9-d6fa3244bb08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1557c02911274e9ab1b6a5e95b045f91, user_id=33704d3fe1c14d8ea89aa999b2cb5d0c, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:53.388109 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1676/3430] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:53 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:53.388965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e7ab1-c5dc-44f3-991b-53a039f589d7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:53.391445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0a041-3f79-4143-afd9-d6fa3244bb08 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:53.391570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-dd7e7ab1-c5dc-44f3-991b-53a039f589d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:53.403046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:53.409932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e7ab1-c5dc-44f3-991b-53a039f589d7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:53.421453 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1755/3431] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:53 2022] PUT /identity/v3/projects/1557c02911274e9ab1b6a5e95b045f91/users/33704d3fe1c14d8ea89aa999b2cb5d0c/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:53.422547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0a041-3f79-4143-afd9-d6fa3244bb08 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:53.503929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0de250-d606-4700-9544-cf47c2c8befd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:53.504749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0de250-d606-4700-9544-cf47c2c8befd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:53.505136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0de250-d606-4700-9544-cf47c2c8befd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:53.505534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0de250-d606-4700-9544-cf47c2c8befd admin admin] PATH_INFO: `/v3/projects/520ed1a4ca304a9b9a47c5a9908bcb0b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:53.506979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0de250-d606-4700-9544-cf47c2c8befd admin admin] RBAC: Authorizing `identity:delete_project(project_id=520ed1a4ca304a9b9a47c5a9908bcb0b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:53.509249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0de250-d606-4700-9544-cf47c2c8befd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:53.520543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0a041-3f79-4143-afd9-d6fa3244bb08 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:53.527091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b995629-9e51-4349-bb98-9475c1487e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:53.527698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b995629-9e51-4349-bb98-9475c1487e4a admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:53.527978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b995629-9e51-4349-bb98-9475c1487e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:53.528800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b995629-9e51-4349-bb98-9475c1487e4a admin admin] PATH_INFO: `/v3/projects/24a9d88d60164b2785bacc9f8bfa3c05/users/0284a8ec6c4e41f0a1a88e2f11a4faa8/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:53.531286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b995629-9e51-4349-bb98-9475c1487e4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=24a9d88d60164b2785bacc9f8bfa3c05, user_id=0284a8ec6c4e41f0a1a88e2f11a4faa8, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:53.534449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b995629-9e51-4349-bb98-9475c1487e4a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:53.549483 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1756/3432] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:53 2022] PUT /identity/v3/projects/24a9d88d60164b2785bacc9f8bfa3c05/users/0284a8ec6c4e41f0a1a88e2f11a4faa8/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:53.550683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b995629-9e51-4349-bb98-9475c1487e4a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:53.574904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8d0de250-d606-4700-9544-cf47c2c8befd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '520ed1a4ca304a9b9a47c5a9908bcb0b', 'request_id': 'req-8d0de250-d606-4700-9544-cf47c2c8befd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:53.591590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8d0de250-d606-4700-9544-cf47c2c8befd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '520ed1a4ca304a9b9a47c5a9908bcb0b', 'request_id': 'req-8d0de250-d606-4700-9544-cf47c2c8befd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:53.608873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8d0de250-d606-4700-9544-cf47c2c8befd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '520ed1a4ca304a9b9a47c5a9908bcb0b', 'request_id': 'req-8d0de250-d606-4700-9544-cf47c2c8befd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:53.642524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8d0de250-d606-4700-9544-cf47c2c8befd admin admin] The token cache is being invalidate because project 520ed1a4ca304a9b9a47c5a9908bcb0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:57:53.642860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8d0de250-d606-4700-9544-cf47c2c8befd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:53.644714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8d0de250-d606-4700-9544-cf47c2c8befd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:53.647999 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1677/3433] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:57:53 2022] DELETE /identity/v3/projects/520ed1a4ca304a9b9a47c5a9908bcb0b => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:53.653324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39232c62-65c5-4ca9-9646-295258334e54 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:53.654181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39232c62-65c5-4ca9-9646-295258334e54 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:53.654684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39232c62-65c5-4ca9-9646-295258334e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:53.682002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b995629-9e51-4349-bb98-9475c1487e4a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:53.711552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-39232c62-65c5-4ca9-9646-295258334e54 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:53.715290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-39232c62-65c5-4ca9-9646-295258334e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:53.788833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf069a95-2933-4153-98de-11375381f993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:53.790795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf069a95-2933-4153-98de-11375381f993 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:53.791079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf069a95-2933-4153-98de-11375381f993 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:53.791415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf069a95-2933-4153-98de-11375381f993 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:53.792041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf069a95-2933-4153-98de-11375381f993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:53.793599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf069a95-2933-4153-98de-11375381f993 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:53.827067 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1757/3434] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:53 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:53.829993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf069a95-2933-4153-98de-11375381f993 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:53.834769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bf069a95-2933-4153-98de-11375381f993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:53.835924 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1678/3435] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:57:53.854676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c17bce-2cbb-40ed-a7d3-df549b439226 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:53.856054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c17bce-2cbb-40ed-a7d3-df549b439226 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:53.856341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c17bce-2cbb-40ed-a7d3-df549b439226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:53.887467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf069a95-2933-4153-98de-11375381f993 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:53.898584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5b0fb-b003-4884-be2c-54d1a3c53939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:53.900827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5b0fb-b003-4884-be2c-54d1a3c53939 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:53.901329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5b0fb-b003-4884-be2c-54d1a3c53939 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:53.901614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5b0fb-b003-4884-be2c-54d1a3c53939 admin admin] PATH_INFO: `/v3/projects/1557c02911274e9ab1b6a5e95b045f91/users/33704d3fe1c14d8ea89aa999b2cb5d0c/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:53.907208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5b0fb-b003-4884-be2c-54d1a3c53939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1557c02911274e9ab1b6a5e95b045f91, user_id=33704d3fe1c14d8ea89aa999b2cb5d0c, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:53.908405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b8c17bce-2cbb-40ed-a7d3-df549b439226 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:53.909736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b8c17bce-2cbb-40ed-a7d3-df549b439226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:53.913109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5b0fb-b003-4884-be2c-54d1a3c53939 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:53.928665 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1679/3436] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:53.931988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:53.934796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c17bce-2cbb-40ed-a7d3-df549b439226 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:53.945769 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1758/3437] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:53 2022] PUT /identity/v3/projects/1557c02911274e9ab1b6a5e95b045f91/users/33704d3fe1c14d8ea89aa999b2cb5d0c/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:53.954805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c17bce-2cbb-40ed-a7d3-df549b439226 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:53.961944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e060a2-a127-4601-b77f-8456c719e476 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:53.962258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e060a2-a127-4601-b77f-8456c719e476 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:53.962723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e060a2-a127-4601-b77f-8456c719e476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:53.996809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:54.036350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-14e060a2-a127-4601-b77f-8456c719e476 None None] MFA Rules not processed for user `33704d3fe1c14d8ea89aa999b2cb5d0c`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:54.038040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-14e060a2-a127-4601-b77f-8456c719e476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:54.068709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c91aa-01ac-4b30-8c98-5594481cfdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:54.070224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c91aa-01ac-4b30-8c98-5594481cfdfd admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:54.070637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c91aa-01ac-4b30-8c98-5594481cfdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:54.070990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c91aa-01ac-4b30-8c98-5594481cfdfd admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:54.071728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c91aa-01ac-4b30-8c98-5594481cfdfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:57:54.072188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c91aa-01ac-4b30-8c98-5594481cfdfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:54.074100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c91aa-01ac-4b30-8c98-5594481cfdfd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:54.092615 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1680/3438] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:57:54.093735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c91aa-01ac-4b30-8c98-5594481cfdfd admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:54.114522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c91aa-01ac-4b30-8c98-5594481cfdfd admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:54.128709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da50018-bed3-4100-862b-ae9c4ec0200d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:54.129805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da50018-bed3-4100-862b-ae9c4ec0200d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:54.130091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da50018-bed3-4100-862b-ae9c4ec0200d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:54.130367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da50018-bed3-4100-862b-ae9c4ec0200d admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:54.132191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da50018-bed3-4100-862b-ae9c4ec0200d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:54.136878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da50018-bed3-4100-862b-ae9c4ec0200d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:54.217311 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1681/3439] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:57:54 2022] POST /identity/v3/projects => generated 389 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:54.224269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da50018-bed3-4100-862b-ae9c4ec0200d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:54.232092 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1759/3440] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:53 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:54.236399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e060a2-a127-4601-b77f-8456c719e476 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:54.276816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e060a2-a127-4601-b77f-8456c719e476 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:54.468051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da50018-bed3-4100-862b-ae9c4ec0200d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:54.473702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f3f26-2f08-4d75-8b76-7538ed2a9c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:54.474371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f3f26-2f08-4d75-8b76-7538ed2a9c20 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:54.474736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f3f26-2f08-4d75-8b76-7538ed2a9c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:54.475071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f3f26-2f08-4d75-8b76-7538ed2a9c20 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:54.475709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f3f26-2f08-4d75-8b76-7538ed2a9c20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:54.477857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f3f26-2f08-4d75-8b76-7538ed2a9c20 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:54.510905 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1682/3441] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:57:54 2022] POST /identity/v3/users => generated 295 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:54.515553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f3f26-2f08-4d75-8b76-7538ed2a9c20 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:54.534307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14516be-6d1d-4d05-b877-ba45dba50db1 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:54.534966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14516be-6d1d-4d05-b877-ba45dba50db1 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:54.535313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14516be-6d1d-4d05-b877-ba45dba50db1 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:54.535592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14516be-6d1d-4d05-b877-ba45dba50db1 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:54.535887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f3f26-2f08-4d75-8b76-7538ed2a9c20 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:54.536251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14516be-6d1d-4d05-b877-ba45dba50db1 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:54.537470 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-a14516be-6d1d-4d05-b877-ba45dba50db1 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:57:54.537470 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 20:57:54.537470 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:57:54.540399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14516be-6d1d-4d05-b877-ba45dba50db1 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:54.547308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fffdb69-9e65-412c-bd31-901d73756d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:54.549379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffdb69-9e65-412c-bd31-901d73756d52 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:54.551013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffdb69-9e65-412c-bd31-901d73756d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:54.551442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffdb69-9e65-412c-bd31-901d73756d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:54.554189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fffdb69-9e65-412c-bd31-901d73756d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:54.557965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fffdb69-9e65-412c-bd31-901d73756d52 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:54.576620 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1760/3442] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:57:54 2022] POST /identity/v3/services => generated 288 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:54.578038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14516be-6d1d-4d05-b877-ba45dba50db1 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:54.592449 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1683/3443] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:54.596827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fffdb69-9e65-412c-bd31-901d73756d52 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:54.620609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fffdb69-9e65-412c-bd31-901d73756d52 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:54.630694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14516be-6d1d-4d05-b877-ba45dba50db1 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:54.635682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b97e63-b230-4c99-8b03-59897836c9e9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:54.637087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b97e63-b230-4c99-8b03-59897836c9e9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:54.637432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b97e63-b230-4c99-8b03-59897836c9e9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:54.637927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b97e63-b230-4c99-8b03-59897836c9e9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:54.639706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b97e63-b230-4c99-8b03-59897836c9e9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:54.640897 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-b7b97e63-b230-4c99-8b03-59897836c9e9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:57:54.640897 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 20:57:54.640897 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:57:54.652755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b97e63-b230-4c99-8b03-59897836c9e9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:54.705088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce455be-ee99-454f-b85b-8b8b4164ca67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:54.707712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce455be-ee99-454f-b85b-8b8b4164ca67 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:54.708154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce455be-ee99-454f-b85b-8b8b4164ca67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:54.710515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce455be-ee99-454f-b85b-8b8b4164ca67 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:54.712592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce455be-ee99-454f-b85b-8b8b4164ca67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:54.722280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce455be-ee99-454f-b85b-8b8b4164ca67 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:54.775984 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1684/3444] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:54.776740 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-b7b97e63-b230-4c99-8b03-59897836c9e9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-2139123418. To ensure interoperability, identifiers should be a valid uuid. Oct 01 20:57:54.776740 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 01 20:57:54.776740 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:57:54.777383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce455be-ee99-454f-b85b-8b8b4164ca67 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:54.799016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce455be-ee99-454f-b85b-8b8b4164ca67 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:54.850124 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1761/3445] 10.222.0.17 () {60 vars in 1156 bytes} [Sat Oct 1 20:57:54 2022] POST /identity/v3/endpoints => generated 361 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:54.851082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245e4f4-f1a6-4c47-bb0f-b49b808872c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:54.852742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b97e63-b230-4c99-8b03-59897836c9e9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:54.852925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245e4f4-f1a6-4c47-bb0f-b49b808872c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:54.853220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245e4f4-f1a6-4c47-bb0f-b49b808872c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:54.853497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245e4f4-f1a6-4c47-bb0f-b49b808872c9 admin admin] PATH_INFO: `/v3/projects/7f84f3e4972f4a9c968100efa5775214/users/f407ee9fccbf4fd6a9286642f8a015ad/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:54.885798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b97e63-b230-4c99-8b03-59897836c9e9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:54.891779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bb410-5264-4e9f-bfe6-4e63bc8c9e22 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:54.892901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bb410-5264-4e9f-bfe6-4e63bc8c9e22 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:54.893180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bb410-5264-4e9f-bfe6-4e63bc8c9e22 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:54.893464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bb410-5264-4e9f-bfe6-4e63bc8c9e22 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/regions/tempest-region-2139123418` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:54.895413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bb410-5264-4e9f-bfe6-4e63bc8c9e22 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-2139123418)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:54.896517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bb410-5264-4e9f-bfe6-4e63bc8c9e22 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:54.901862 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1762/3446] 10.222.0.17 () {58 vars in 1206 bytes} [Sat Oct 1 20:57:54 2022] GET /identity/v3/regions/tempest-region-2139123418 => generated 180 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:57:54.903078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245e4f4-f1a6-4c47-bb0f-b49b808872c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f84f3e4972f4a9c968100efa5775214, user_id=f407ee9fccbf4fd6a9286642f8a015ad, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:54.903557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bb410-5264-4e9f-bfe6-4e63bc8c9e22 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:54.906387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245e4f4-f1a6-4c47-bb0f-b49b808872c9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:54.917384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bb410-5264-4e9f-bfe6-4e63bc8c9e22 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:54.923503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae107c-b936-4c78-8590-fa3e8d5471e4 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:54.924256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae107c-b936-4c78-8590-fa3e8d5471e4 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:54.924606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae107c-b936-4c78-8590-fa3e8d5471e4 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:54.925499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae107c-b936-4c78-8590-fa3e8d5471e4 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:54.926209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae107c-b936-4c78-8590-fa3e8d5471e4 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:54.928429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae107c-b936-4c78-8590-fa3e8d5471e4 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:54.940252 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1685/3447] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:57:54 2022] PUT /identity/v3/projects/7f84f3e4972f4a9c968100efa5775214/users/f407ee9fccbf4fd6a9286642f8a015ad/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:54.946137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb5677-40b5-4627-a381-a2bb10d79ee5 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:54.946985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb5677-40b5-4627-a381-a2bb10d79ee5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:54.946985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb5677-40b5-4627-a381-a2bb10d79ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:54.955925 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1763/3448] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:57:54 2022] POST /identity/v3/services => generated 289 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:54.960230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae107c-b936-4c78-8590-fa3e8d5471e4 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:54.972115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:54.994461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae107c-b936-4c78-8590-fa3e8d5471e4 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:54.995040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b3eb5677-40b5-4627-a381-a2bb10d79ee5 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:54.996722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b3eb5677-40b5-4627-a381-a2bb10d79ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:55.031954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:55.137242 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1686/3449] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:54 2022] POST /identity/v3/auth/tokens => generated 3688 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:57:55.152741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231e027-ccd8-4647-a0bb-3f12c6394cb2 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:55.154053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231e027-ccd8-4647-a0bb-3f12c6394cb2 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:55.154053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231e027-ccd8-4647-a0bb-3f12c6394cb2 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:55.154053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231e027-ccd8-4647-a0bb-3f12c6394cb2 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:55.155666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231e027-ccd8-4647-a0bb-3f12c6394cb2 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:55.158195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231e027-ccd8-4647-a0bb-3f12c6394cb2 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:55.166818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb5677-40b5-4627-a381-a2bb10d79ee5 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:55.170584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b3eb5677-40b5-4627-a381-a2bb10d79ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:55.183377 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-a231e027-ccd8-4647-a0bb-3f12c6394cb2 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1071515215. To ensure interoperability, identifiers should be a valid uuid. Oct 01 20:57:55.183377 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 01 20:57:55.183377 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:57:55.278579 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1764/3450] 10.222.0.17 () {60 vars in 1156 bytes} [Sat Oct 1 20:57:54 2022] POST /identity/v3/endpoints => generated 363 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:55.283482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231e027-ccd8-4647-a0bb-3f12c6394cb2 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:55.324886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb5677-40b5-4627-a381-a2bb10d79ee5 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:55.351812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231e027-ccd8-4647-a0bb-3f12c6394cb2 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:55.359563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ca922-1e02-45bf-befc-8cbda910ab4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:55.360201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ca922-1e02-45bf-befc-8cbda910ab4f service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:55.360481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ca922-1e02-45bf-befc-8cbda910ab4f service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:55.360757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ca922-1e02-45bf-befc-8cbda910ab4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:55.366708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08d011-69b0-46e5-9c17-c308d98c80f4 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:55.367362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08d011-69b0-46e5-9c17-c308d98c80f4 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:55.368188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08d011-69b0-46e5-9c17-c308d98c80f4 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:55.368472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08d011-69b0-46e5-9c17-c308d98c80f4 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/regions/tempest-region-1071515215` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:55.369075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08d011-69b0-46e5-9c17-c308d98c80f4 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1071515215)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:55.370152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08d011-69b0-46e5-9c17-c308d98c80f4 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:55.374675 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1765/3451] 10.222.0.17 () {58 vars in 1206 bytes} [Sat Oct 1 20:57:55 2022] GET /identity/v3/regions/tempest-region-1071515215 => generated 180 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:57:55.395110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ca922-1e02-45bf-befc-8cbda910ab4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:55.397597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ca922-1e02-45bf-befc-8cbda910ab4f service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:55.398729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08d011-69b0-46e5-9c17-c308d98c80f4 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:55.426535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08d011-69b0-46e5-9c17-c308d98c80f4 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:55.433477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65e3e6-6c7e-48b7-9919-6ec35693fb66 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:55.434008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65e3e6-6c7e-48b7-9919-6ec35693fb66 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:55.434290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65e3e6-6c7e-48b7-9919-6ec35693fb66 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:55.434655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65e3e6-6c7e-48b7-9919-6ec35693fb66 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:55.436047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65e3e6-6c7e-48b7-9919-6ec35693fb66 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:55.439974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65e3e6-6c7e-48b7-9919-6ec35693fb66 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:55.465045 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-4a65e3e6-6c7e-48b7-9919-6ec35693fb66 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-692363818. To ensure interoperability, identifiers should be a valid uuid. Oct 01 20:57:55.465045 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 01 20:57:55.465045 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:57:55.472100 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1687/3452] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:55 2022] GET /identity/v3/auth/tokens => generated 3873 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:57:55.506148 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1766/3453] 10.222.0.17 () {60 vars in 1156 bytes} [Sat Oct 1 20:57:55 2022] POST /identity/v3/endpoints => generated 359 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:55.509682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ca922-1e02-45bf-befc-8cbda910ab4f service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:55.520937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ca922-1e02-45bf-befc-8cbda910ab4f service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:55.536955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a3a03-c5b2-4193-901e-5923556b1406 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:55.537982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a3a03-c5b2-4193-901e-5923556b1406 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:55.538519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a3a03-c5b2-4193-901e-5923556b1406 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:55.538892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a3a03-c5b2-4193-901e-5923556b1406 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/regions/tempest-region-692363818` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:55.539612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a3a03-c5b2-4193-901e-5923556b1406 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-692363818)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:55.540749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a3a03-c5b2-4193-901e-5923556b1406 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:55.543602 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1688/3454] 10.222.0.17 () {58 vars in 1203 bytes} [Sat Oct 1 20:57:55 2022] GET /identity/v3/regions/tempest-region-692363818 => generated 178 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:57:55.548920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65e3e6-6c7e-48b7-9919-6ec35693fb66 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:55.580860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65e3e6-6c7e-48b7-9919-6ec35693fb66 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:55.594513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d474f-d63a-4f2f-9d73-c496bfdbc524 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:55.595932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d474f-d63a-4f2f-9d73-c496bfdbc524 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:55.598772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d474f-d63a-4f2f-9d73-c496bfdbc524 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:55.599656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d474f-d63a-4f2f-9d73-c496bfdbc524 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:55.602438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d474f-d63a-4f2f-9d73-c496bfdbc524 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:55.605075 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-7c5d474f-d63a-4f2f-9d73-c496bfdbc524 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoints": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:57:55.605075 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 20:57:55.605075 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:57:55.608652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d474f-d63a-4f2f-9d73-c496bfdbc524 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:55.626615 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1767/3455] 10.222.0.17 () {58 vars in 1134 bytes} [Sat Oct 1 20:57:55 2022] GET /identity/v3/endpoints => generated 4397 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:55.630778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a3a03-c5b2-4193-901e-5923556b1406 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:55.641869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a3a03-c5b2-4193-901e-5923556b1406 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:55.647027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb0e1c-f123-467a-a1d2-7d1094a8ac61 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:55.647676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb0e1c-f123-467a-a1d2-7d1094a8ac61 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:55.647958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb0e1c-f123-467a-a1d2-7d1094a8ac61 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:55.648235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb0e1c-f123-467a-a1d2-7d1094a8ac61 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/endpoints/5923c39ae04e46f99e713a13dcb72bfa` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:55.657812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb0e1c-f123-467a-a1d2-7d1094a8ac61 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:get_endpoint(endpoint_id=5923c39ae04e46f99e713a13dcb72bfa)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:55.659050 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-2ceb0e1c-f123-467a-a1d2-7d1094a8ac61 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:57:55.659050 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:57:55.659050 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:57:55.660162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb0e1c-f123-467a-a1d2-7d1094a8ac61 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:55.662112 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1689/3456] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:57:55 2022] GET /identity/v3/endpoints/5923c39ae04e46f99e713a13dcb72bfa => generated 359 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:57:55.665855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d474f-d63a-4f2f-9d73-c496bfdbc524 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:55.677654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d474f-d63a-4f2f-9d73-c496bfdbc524 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:55.685681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a26eb-ab59-4b3e-8cfc-391f0a027349 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:55.686340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a26eb-ab59-4b3e-8cfc-391f0a027349 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:55.686694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a26eb-ab59-4b3e-8cfc-391f0a027349 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:55.686971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a26eb-ab59-4b3e-8cfc-391f0a027349 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/endpoints/5923c39ae04e46f99e713a13dcb72bfa` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:55.688596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a26eb-ab59-4b3e-8cfc-391f0a027349 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=5923c39ae04e46f99e713a13dcb72bfa)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:55.689680 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-c42a26eb-ab59-4b3e-8cfc-391f0a027349 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:57:55.689680 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 20:57:55.689680 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:57:55.691400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a26eb-ab59-4b3e-8cfc-391f0a027349 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:55.708047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c42a26eb-ab59-4b3e-8cfc-391f0a027349 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '5923c39ae04e46f99e713a13dcb72bfa', 'request_id': 'req-c42a26eb-ab59-4b3e-8cfc-391f0a027349'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:55.729669 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1768/3457] 10.222.0.17 () {58 vars in 1236 bytes} [Sat Oct 1 20:57:55 2022] DELETE /identity/v3/endpoints/5923c39ae04e46f99e713a13dcb72bfa => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:55.733689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb0e1c-f123-467a-a1d2-7d1094a8ac61 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:55.745398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb0e1c-f123-467a-a1d2-7d1094a8ac61 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:55.774134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e74868-561f-44b2-9152-eacd1b615eb8 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:55.775817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e74868-561f-44b2-9152-eacd1b615eb8 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:55.776333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e74868-561f-44b2-9152-eacd1b615eb8 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:55.777626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e74868-561f-44b2-9152-eacd1b615eb8 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:55.782546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e74868-561f-44b2-9152-eacd1b615eb8 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:55.783665 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-00e74868-561f-44b2-9152-eacd1b615eb8 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoints": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:57:55.783665 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:57:55.783665 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:57:55.784620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e74868-561f-44b2-9152-eacd1b615eb8 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:55.799944 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1690/3458] 10.222.0.17 () {58 vars in 1134 bytes} [Sat Oct 1 20:57:55 2022] GET /identity/v3/endpoints => generated 4051 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:55.804125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a26eb-ab59-4b3e-8cfc-391f0a027349 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:55.820683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a26eb-ab59-4b3e-8cfc-391f0a027349 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:55.825897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f9fa6-60b8-460a-b3b2-581367478f3c tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:55.826570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f9fa6-60b8-460a-b3b2-581367478f3c tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:55.826858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f9fa6-60b8-460a-b3b2-581367478f3c tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:55.827155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f9fa6-60b8-460a-b3b2-581367478f3c tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/endpoints/5923c39ae04e46f99e713a13dcb72bfa` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:55.833254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f9fa6-60b8-460a-b3b2-581367478f3c tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 20:57:55.852193 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-4b7f9fa6-60b8-460a-b3b2-581367478f3c tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Could not find endpoint: 5923c39ae04e46f99e713a13dcb72bfa.: keystone.exception.EndpointNotFound: Could not find endpoint: 5923c39ae04e46f99e713a13dcb72bfa. Oct 01 20:57:55.852193 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:57:55.852193 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 236, in _get_endpoint Oct 01 20:57:55.852193 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return session.query(Endpoint).filter_by(id=endpoint_id).one() Oct 01 20:57:55.852193 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Oct 01 20:57:55.852193 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._iter().one() Oct 01 20:57:55.852193 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Oct 01 20:57:55.852193 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._only_one_row( Oct 01 20:57:55.852193 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Oct 01 20:57:55.852193 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exc.NoResultFound( Oct 01 20:57:55.852193 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Oct 01 20:57:55.852193 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:57:55.852193 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:57:55.852193 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:57:55.852193 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:57:55.852193 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 228, in get_endpoint Oct 01 20:57:55.855131 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_endpoint(endpoint_id) Oct 01 20:57:55.855131 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 242, in get_endpoint Oct 01 20:57:55.855131 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_endpoint(session, endpoint_id).to_dict() Oct 01 20:57:55.855131 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 238, in _get_endpoint Oct 01 20:57:55.855131 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.EndpointNotFound(endpoint_id=endpoint_id) Oct 01 20:57:55.855131 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.EndpointNotFound: Could not find endpoint: 5923c39ae04e46f99e713a13dcb72bfa. Oct 01 20:57:55.855131 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:57:55.855131 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:57:55.855131 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:57:55.855131 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:57:55.855131 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:57:55.855131 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:57:55.855131 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:57:55.855131 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:57:55.855131 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:57:55.855131 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:57:55.855131 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:57:55.856068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:57:55.856068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:57:55.856068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:57:55.856068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 113, in delete Oct 01 20:57:55.856068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_endpoint') Oct 01 20:57:55.856068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 20:57:55.856068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 20:57:55.856068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 20:57:55.856068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 20:57:55.856068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:57:55.856068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:57:55.856068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 20:57:55.856068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 20:57:55.856068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 20:57:55.856068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 20:57:55.856068 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 20:57:55.856900 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 20:57:55.856900 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 20:57:55.856900 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 20:57:55.856900 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 20:57:55.856900 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 20:57:55.856900 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 20:57:55.856900 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 20:57:55.856900 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 20:57:55.856900 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 20:57:55.856900 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 230, in get_endpoint Oct 01 20:57:55.856900 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.EndpointNotFound(endpoint_id=endpoint_id) Oct 01 20:57:55.856900 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.EndpointNotFound: Could not find endpoint: 5923c39ae04e46f99e713a13dcb72bfa. Oct 01 20:57:55.856900 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:57:55.861178 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1769/3459] 10.222.0.17 () {58 vars in 1236 bytes} [Sat Oct 1 20:57:55 2022] DELETE /identity/v3/endpoints/5923c39ae04e46f99e713a13dcb72bfa => generated 114 bytes in 57 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 20:57:55.865647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e74868-561f-44b2-9152-eacd1b615eb8 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:55.884218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e74868-561f-44b2-9152-eacd1b615eb8 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:55.892516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d314b1-a03d-4141-be50-ab741cd606aa tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:55.893168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d314b1-a03d-4141-be50-ab741cd606aa tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:55.893453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d314b1-a03d-4141-be50-ab741cd606aa tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:55.895529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d314b1-a03d-4141-be50-ab741cd606aa tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/regions/tempest-region-692363818` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:55.896147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d314b1-a03d-4141-be50-ab741cd606aa tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-692363818)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:55.897636 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-d7d314b1-a03d-4141-be50-ab741cd606aa tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:57:55.897636 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:57:55.897636 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:57:55.898641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d314b1-a03d-4141-be50-ab741cd606aa tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:55.934420 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-d7d314b1-a03d-4141-be50-ab741cd606aa tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-692363818. To ensure interoperability, identifiers should be a valid uuid. Oct 01 20:57:55.934420 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 01 20:57:55.934420 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:57:55.950115 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1691/3460] 10.222.0.17 () {58 vars in 1206 bytes} [Sat Oct 1 20:57:55 2022] DELETE /identity/v3/regions/tempest-region-692363818 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:55.951441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f9fa6-60b8-460a-b3b2-581367478f3c tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:55.966085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f9fa6-60b8-460a-b3b2-581367478f3c tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:55.996890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3d984-5ae1-4e49-97bb-208131e3fdb9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:55.998037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3d984-5ae1-4e49-97bb-208131e3fdb9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:55.998688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3d984-5ae1-4e49-97bb-208131e3fdb9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:55.999222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3d984-5ae1-4e49-97bb-208131e3fdb9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:56.000210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db3d984-5ae1-4e49-97bb-208131e3fdb9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:56.006507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db3d984-5ae1-4e49-97bb-208131e3fdb9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:56.031758 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1770/3461] 10.222.0.17 () {58 vars in 1134 bytes} [Sat Oct 1 20:57:55 2022] GET /identity/v3/endpoints => generated 4051 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:56.033777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d314b1-a03d-4141-be50-ab741cd606aa tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:56.051220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d314b1-a03d-4141-be50-ab741cd606aa tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:56.057530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5eb41b-54c3-4a04-b4cd-4a3d377299a6 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:56.058438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eb41b-54c3-4a04-b4cd-4a3d377299a6 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:56.058976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eb41b-54c3-4a04-b4cd-4a3d377299a6 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:56.060693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eb41b-54c3-4a04-b4cd-4a3d377299a6 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:56.061568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5eb41b-54c3-4a04-b4cd-4a3d377299a6 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Adding query filter params (service_id=ef447934e7f5441fb438526011065c2e) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:57:56.064658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5eb41b-54c3-4a04-b4cd-4a3d377299a6 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:56.067737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5eb41b-54c3-4a04-b4cd-4a3d377299a6 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:56.087292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3d984-5ae1-4e49-97bb-208131e3fdb9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:56.093933 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1692/3462] 10.222.0.17 () {58 vars in 1221 bytes} [Sat Oct 1 20:57:56 2022] GET /identity/v3/endpoints?service_id=ef447934e7f5441fb438526011065c2e => generated 503 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:57:56.101933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3d984-5ae1-4e49-97bb-208131e3fdb9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:56.109789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed80c3-192c-49d8-b315-195d3dd349f8 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:56.111090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed80c3-192c-49d8-b315-195d3dd349f8 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:56.111681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed80c3-192c-49d8-b315-195d3dd349f8 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:56.113472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed80c3-192c-49d8-b315-195d3dd349f8 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:56.114454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed80c3-192c-49d8-b315-195d3dd349f8 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Adding query filter params (service_id=438675c126634697affaf0a0f465e1ec) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:57:56.115034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed80c3-192c-49d8-b315-195d3dd349f8 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:56.118534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed80c3-192c-49d8-b315-195d3dd349f8 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:56.142467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5eb41b-54c3-4a04-b4cd-4a3d377299a6 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:56.148888 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1771/3463] 10.222.0.17 () {58 vars in 1221 bytes} [Sat Oct 1 20:57:56 2022] GET /identity/v3/endpoints?service_id=438675c126634697affaf0a0f465e1ec => generated 505 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:57:56.176152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5eb41b-54c3-4a04-b4cd-4a3d377299a6 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:56.191106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad93ed-3e51-4e2f-919d-637aea647d60 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:56.191745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad93ed-3e51-4e2f-919d-637aea647d60 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:56.192172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad93ed-3e51-4e2f-919d-637aea647d60 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:56.192801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad93ed-3e51-4e2f-919d-637aea647d60 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:56.193474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad93ed-3e51-4e2f-919d-637aea647d60 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Adding query filter params (interface=public) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:57:56.193787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad93ed-3e51-4e2f-919d-637aea647d60 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:56.196639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad93ed-3e51-4e2f-919d-637aea647d60 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:56.219946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed80c3-192c-49d8-b315-195d3dd349f8 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:56.235254 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1693/3464] 10.222.0.17 () {58 vars in 1167 bytes} [Sat Oct 1 20:57:56 2022] GET /identity/v3/endpoints?interface=public => generated 3406 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:57:56.248075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed80c3-192c-49d8-b315-195d3dd349f8 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:56.271325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6db1d-6534-454a-931a-43f80b248b87 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:56.272163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6db1d-6534-454a-931a-43f80b248b87 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:56.272370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6db1d-6534-454a-931a-43f80b248b87 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:56.272644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6db1d-6534-454a-931a-43f80b248b87 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:56.273356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6db1d-6534-454a-931a-43f80b248b87 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Adding query filter params (interface=internal) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:57:56.273668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6db1d-6534-454a-931a-43f80b248b87 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:56.277202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6db1d-6534-454a-931a-43f80b248b87 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:56.290954 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1772/3465] 10.222.0.17 () {58 vars in 1171 bytes} [Sat Oct 1 20:57:56 2022] GET /identity/v3/endpoints?interface=internal => generated 480 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:57:56.296065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad93ed-3e51-4e2f-919d-637aea647d60 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:56.312009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad93ed-3e51-4e2f-919d-637aea647d60 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:56.325203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1342cd-b107-4602-a2df-8c4587324efe tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:56.326887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1342cd-b107-4602-a2df-8c4587324efe tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:56.327627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1342cd-b107-4602-a2df-8c4587324efe tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:56.327627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1342cd-b107-4602-a2df-8c4587324efe tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:56.329315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1342cd-b107-4602-a2df-8c4587324efe tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:56.333285 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-cb1342cd-b107-4602-a2df-8c4587324efe tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:57:56.333285 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:57:56.333285 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:57:56.334742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1342cd-b107-4602-a2df-8c4587324efe tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:56.371809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6db1d-6534-454a-931a-43f80b248b87 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:56.377540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:56.382235 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1694/3466] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:57:56 2022] POST /identity/v3/services => generated 288 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:56.391900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6db1d-6534-454a-931a-43f80b248b87 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:56.439576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3376f-c05d-439f-82c8-81a274405c3a tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:56.440412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3376f-c05d-439f-82c8-81a274405c3a tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:56.440705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3376f-c05d-439f-82c8-81a274405c3a tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:56.440983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3376f-c05d-439f-82c8-81a274405c3a tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:56.441672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb3376f-c05d-439f-82c8-81a274405c3a tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:56.447613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb3376f-c05d-439f-82c8-81a274405c3a tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:56.487656 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-acb3376f-c05d-439f-82c8-81a274405c3a tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-620038261. To ensure interoperability, identifiers should be a valid uuid. Oct 01 20:57:56.487656 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 01 20:57:56.487656 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:57:56.547019 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1773/3467] 10.222.0.17 () {60 vars in 1156 bytes} [Sat Oct 1 20:57:56 2022] POST /identity/v3/endpoints => generated 359 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:57:56.551367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1342cd-b107-4602-a2df-8c4587324efe tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:56.557244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:56.567871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1342cd-b107-4602-a2df-8c4587324efe tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:56.622448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59bdb41-2b0e-4323-b15a-f4f6278fbc5f tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:56.623317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bdb41-2b0e-4323-b15a-f4f6278fbc5f tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:56.623679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bdb41-2b0e-4323-b15a-f4f6278fbc5f tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:56.624225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bdb41-2b0e-4323-b15a-f4f6278fbc5f tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/regions/tempest-region-620038261` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:56.626850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59bdb41-2b0e-4323-b15a-f4f6278fbc5f tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-620038261)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:56.634623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59bdb41-2b0e-4323-b15a-f4f6278fbc5f tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:56.640251 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1695/3468] 10.222.0.17 () {58 vars in 1203 bytes} [Sat Oct 1 20:57:56 2022] GET /identity/v3/regions/tempest-region-620038261 => generated 178 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:57:56.641932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3376f-c05d-439f-82c8-81a274405c3a tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:56.666071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3376f-c05d-439f-82c8-81a274405c3a tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:56.685563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b85e7-0287-4107-ae58-876b391ccf0f tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:56.686623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b85e7-0287-4107-ae58-876b391ccf0f tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:56.686987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b85e7-0287-4107-ae58-876b391ccf0f tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:56.687697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b85e7-0287-4107-ae58-876b391ccf0f tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/endpoints/407478f7002f4948a09b255316cdc1be` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:56.707726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b85e7-0287-4107-ae58-876b391ccf0f tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=407478f7002f4948a09b255316cdc1be)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:56.710257 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-bd0b85e7-0287-4107-ae58-876b391ccf0f tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:57:56.710257 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 20:57:56.710257 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:57:56.714076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b85e7-0287-4107-ae58-876b391ccf0f tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:56.785434 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-bd0b85e7-0287-4107-ae58-876b391ccf0f tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1205545534. To ensure interoperability, identifiers should be a valid uuid. Oct 01 20:57:56.785434 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 01 20:57:56.785434 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:57:56.903875 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1774/3469] 10.222.0.17 () {60 vars in 1256 bytes} [Sat Oct 1 20:57:56 2022] PATCH /identity/v3/endpoints/407478f7002f4948a09b255316cdc1be => generated 363 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:57:56.904818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59bdb41-2b0e-4323-b15a-f4f6278fbc5f tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:56.919265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59bdb41-2b0e-4323-b15a-f4f6278fbc5f tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:56.956988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9fe44-9d64-40d7-8254-f498522569c0 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:56.957794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9fe44-9d64-40d7-8254-f498522569c0 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:56.958178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9fe44-9d64-40d7-8254-f498522569c0 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:56.958596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9fe44-9d64-40d7-8254-f498522569c0 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/regions/tempest-region-1205545534` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:56.959314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9fe44-9d64-40d7-8254-f498522569c0 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1205545534)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:56.960684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9fe44-9d64-40d7-8254-f498522569c0 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:56.963669 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1696/3470] 10.222.0.17 () {58 vars in 1206 bytes} [Sat Oct 1 20:57:56 2022] GET /identity/v3/regions/tempest-region-1205545534 => generated 180 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:57:56.967440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b85e7-0287-4107-ae58-876b391ccf0f tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:56.984172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b85e7-0287-4107-ae58-876b391ccf0f tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:56.998729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f2951-1cd8-43b1-90fc-6c5589fc39f2 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:56.999588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f2951-1cd8-43b1-90fc-6c5589fc39f2 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:57.000029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f2951-1cd8-43b1-90fc-6c5589fc39f2 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:57.000498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f2951-1cd8-43b1-90fc-6c5589fc39f2 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/endpoints/407478f7002f4948a09b255316cdc1be` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:57.020729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f2951-1cd8-43b1-90fc-6c5589fc39f2 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=407478f7002f4948a09b255316cdc1be)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:57.023623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f2951-1cd8-43b1-90fc-6c5589fc39f2 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:57.051769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-994f2951-1cd8-43b1-90fc-6c5589fc39f2 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '407478f7002f4948a09b255316cdc1be', 'request_id': 'req-994f2951-1cd8-43b1-90fc-6c5589fc39f2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:57.067470 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1775/3471] 10.222.0.17 () {58 vars in 1236 bytes} [Sat Oct 1 20:57:56 2022] DELETE /identity/v3/endpoints/407478f7002f4948a09b255316cdc1be => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:57.072947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9fe44-9d64-40d7-8254-f498522569c0 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:57.102620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9fe44-9d64-40d7-8254-f498522569c0 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:57.137337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6dcd1a-5f3f-4ed1-a671-6999ab576fbb tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:57.138012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dcd1a-5f3f-4ed1-a671-6999ab576fbb tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:57.138663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dcd1a-5f3f-4ed1-a671-6999ab576fbb tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:57.138947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dcd1a-5f3f-4ed1-a671-6999ab576fbb tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/regions/tempest-region-1205545534` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:57.144077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6dcd1a-5f3f-4ed1-a671-6999ab576fbb tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1205545534)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:57.156753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6dcd1a-5f3f-4ed1-a671-6999ab576fbb tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:57.189447 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-2c6dcd1a-5f3f-4ed1-a671-6999ab576fbb tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1205545534. To ensure interoperability, identifiers should be a valid uuid. Oct 01 20:57:57.189447 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 01 20:57:57.189447 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:57:57.218552 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1697/3472] 10.222.0.17 () {58 vars in 1209 bytes} [Sat Oct 1 20:57:57 2022] DELETE /identity/v3/regions/tempest-region-1205545534 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:57.223136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f2951-1cd8-43b1-90fc-6c5589fc39f2 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:57.241180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f2951-1cd8-43b1-90fc-6c5589fc39f2 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:57.296851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d3628-c279-47f3-910d-b4578b013246 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:57.297652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d3628-c279-47f3-910d-b4578b013246 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:57.298000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d3628-c279-47f3-910d-b4578b013246 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:57.298345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d3628-c279-47f3-910d-b4578b013246 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/regions/tempest-region-620038261` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:57.299020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d3628-c279-47f3-910d-b4578b013246 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-620038261)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:57.300167 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-310d3628-c279-47f3-910d-b4578b013246 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:57:57.300167 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 20:57:57.300167 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:57:57.301153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d3628-c279-47f3-910d-b4578b013246 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:57.356387 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1776/3473] 10.222.0.17 () {58 vars in 1206 bytes} [Sat Oct 1 20:57:57 2022] DELETE /identity/v3/regions/tempest-region-620038261 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:57.360702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6dcd1a-5f3f-4ed1-a671-6999ab576fbb tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:57.395273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6dcd1a-5f3f-4ed1-a671-6999ab576fbb tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:57.438141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5fda5-6a36-430f-a360-cc0056f36b96 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:57.438897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5fda5-6a36-430f-a360-cc0056f36b96 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:57.439182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5fda5-6a36-430f-a360-cc0056f36b96 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:57.439460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5fda5-6a36-430f-a360-cc0056f36b96 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/services/ca0355aa2b8c4fe59f87b1f3c5c17108` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:57.440047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5fda5-6a36-430f-a360-cc0056f36b96 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:delete_service(service_id=ca0355aa2b8c4fe59f87b1f3c5c17108)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:57.444053 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-d4d5fda5-6a36-430f-a360-cc0056f36b96 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:57:57.444053 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:57:57.444053 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:57:57.447256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5fda5-6a36-430f-a360-cc0056f36b96 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:57.483372 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1698/3474] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:57:57 2022] DELETE /identity/v3/services/ca0355aa2b8c4fe59f87b1f3c5c17108 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:57.488705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d3628-c279-47f3-910d-b4578b013246 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:57.526719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d3628-c279-47f3-910d-b4578b013246 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:57.552334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b5fc7-d39a-45e7-b4dd-cc38e7a52eee tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:57.552976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b5fc7-d39a-45e7-b4dd-cc38e7a52eee tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:57.553259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b5fc7-d39a-45e7-b4dd-cc38e7a52eee tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:57.553537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b5fc7-d39a-45e7-b4dd-cc38e7a52eee tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/endpoints/1156e976f47f47cbbee982683bea56a4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:57.560978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b5fc7-d39a-45e7-b4dd-cc38e7a52eee tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=1156e976f47f47cbbee982683bea56a4)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:57.563295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b5fc7-d39a-45e7-b4dd-cc38e7a52eee tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:57.576280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7a1b5fc7-d39a-45e7-b4dd-cc38e7a52eee tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '1156e976f47f47cbbee982683bea56a4', 'request_id': 'req-7a1b5fc7-d39a-45e7-b4dd-cc38e7a52eee'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:57.599296 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1777/3475] 10.222.0.17 () {58 vars in 1236 bytes} [Sat Oct 1 20:57:57 2022] DELETE /identity/v3/endpoints/1156e976f47f47cbbee982683bea56a4 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:57.602179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5fda5-6a36-430f-a360-cc0056f36b96 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:57.647042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5fda5-6a36-430f-a360-cc0056f36b96 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:57.681107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fd8a5-1dd4-4011-b96b-7113b4d64c7a tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:57.682347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fd8a5-1dd4-4011-b96b-7113b4d64c7a tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:57.682975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fd8a5-1dd4-4011-b96b-7113b4d64c7a tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:57.683612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fd8a5-1dd4-4011-b96b-7113b4d64c7a tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/regions/tempest-region-1071515215` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:57.684307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502fd8a5-1dd4-4011-b96b-7113b4d64c7a tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1071515215)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:57.686744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502fd8a5-1dd4-4011-b96b-7113b4d64c7a tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:57.713730 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-502fd8a5-1dd4-4011-b96b-7113b4d64c7a tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1071515215. To ensure interoperability, identifiers should be a valid uuid. Oct 01 20:57:57.713730 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 01 20:57:57.713730 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:57:57.724148 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1699/3476] 10.222.0.17 () {58 vars in 1209 bytes} [Sat Oct 1 20:57:57 2022] DELETE /identity/v3/regions/tempest-region-1071515215 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:57:57.729363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b5fc7-d39a-45e7-b4dd-cc38e7a52eee tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:57.760648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b5fc7-d39a-45e7-b4dd-cc38e7a52eee tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:57.801891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d63c5-be9b-416d-852a-8235125f5e7b tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:57.802718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d63c5-be9b-416d-852a-8235125f5e7b tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:57.802994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d63c5-be9b-416d-852a-8235125f5e7b tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:57.803422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d63c5-be9b-416d-852a-8235125f5e7b tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/services/438675c126634697affaf0a0f465e1ec` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:57.804683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d63c5-be9b-416d-852a-8235125f5e7b tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:delete_service(service_id=438675c126634697affaf0a0f465e1ec)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:57.805682 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-7f5d63c5-be9b-416d-852a-8235125f5e7b tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:57:57.805682 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 20:57:57.805682 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:57:57.806612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d63c5-be9b-416d-852a-8235125f5e7b tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:57.861600 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1778/3477] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:57:57 2022] DELETE /identity/v3/services/438675c126634697affaf0a0f465e1ec => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:57.866720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fd8a5-1dd4-4011-b96b-7113b4d64c7a tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:57.892707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fd8a5-1dd4-4011-b96b-7113b4d64c7a tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:57.931335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75dac0-5271-4326-b5b7-656c85ce1b6c tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:57.932006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75dac0-5271-4326-b5b7-656c85ce1b6c tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:57.932277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75dac0-5271-4326-b5b7-656c85ce1b6c tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:57.932540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75dac0-5271-4326-b5b7-656c85ce1b6c tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/endpoints/bbbc0d219d454d4b90858fc4190f26d5` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:57.942001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75dac0-5271-4326-b5b7-656c85ce1b6c tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=bbbc0d219d454d4b90858fc4190f26d5)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:57.943219 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-cb75dac0-5271-4326-b5b7-656c85ce1b6c tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:57:57.943219 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:57:57.943219 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:57:57.949375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75dac0-5271-4326-b5b7-656c85ce1b6c tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:57.968972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cb75dac0-5271-4326-b5b7-656c85ce1b6c tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': 'bbbc0d219d454d4b90858fc4190f26d5', 'request_id': 'req-cb75dac0-5271-4326-b5b7-656c85ce1b6c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:58.020883 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1700/3478] 10.222.0.17 () {58 vars in 1236 bytes} [Sat Oct 1 20:57:57 2022] DELETE /identity/v3/endpoints/bbbc0d219d454d4b90858fc4190f26d5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:57:58.025590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d63c5-be9b-416d-852a-8235125f5e7b tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:58.043262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d63c5-be9b-416d-852a-8235125f5e7b tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:58.076032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:UPDATE=1,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:58.112200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a4a58-3afe-4c22-bea1-5b46089eeeb9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:58.112784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a4a58-3afe-4c22-bea1-5b46089eeeb9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:58.113064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a4a58-3afe-4c22-bea1-5b46089eeeb9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:58.113339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a4a58-3afe-4c22-bea1-5b46089eeeb9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/regions/tempest-region-2139123418` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:58.113935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a4a58-3afe-4c22-bea1-5b46089eeeb9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-2139123418)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:58.115741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a4a58-3afe-4c22-bea1-5b46089eeeb9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:58.161977 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1779/3479] 10.222.0.17 () {58 vars in 1209 bytes} [Sat Oct 1 20:57:58 2022] DELETE /identity/v3/regions/tempest-region-2139123418 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:57:58.162354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75dac0-5271-4326-b5b7-656c85ce1b6c tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:58.220494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75dac0-5271-4326-b5b7-656c85ce1b6c tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:58.262694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe462ff7-3f2c-4915-b3f9-9c78ebe1da19 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33704d3fe1c14d8ea89aa999b2cb5d0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1557c02911274e9ab1b6a5e95b045f91', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:58.263351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe462ff7-3f2c-4915-b3f9-9c78ebe1da19 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:58.263657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe462ff7-3f2c-4915-b3f9-9c78ebe1da19 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:58.264061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe462ff7-3f2c-4915-b3f9-9c78ebe1da19 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] PATH_INFO: `/v3/services/ef447934e7f5441fb438526011065c2e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:58.264664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe462ff7-3f2c-4915-b3f9-9c78ebe1da19 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorizing `identity:delete_service(service_id=ef447934e7f5441fb438526011065c2e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:58.267079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe462ff7-3f2c-4915-b3f9-9c78ebe1da19 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:58.332679 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1701/3480] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:57:58 2022] DELETE /identity/v3/services/ef447934e7f5441fb438526011065c2e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:58.336428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a4a58-3afe-4c22-bea1-5b46089eeeb9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:58.379945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a4a58-3afe-4c22-bea1-5b46089eeeb9 tempest-EndPointsTestJSON-1314567943 tempest-EndPointsTestJSON-1314567943-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:58.430057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f0ef3-411a-4bad-8d67-5be600b6464b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:58.430812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f0ef3-411a-4bad-8d67-5be600b6464b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:58.431127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f0ef3-411a-4bad-8d67-5be600b6464b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:58.431402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f0ef3-411a-4bad-8d67-5be600b6464b admin admin] PATH_INFO: `/v3/users/46af47ed34344f09ad9c54ed16e392ee` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:58.433781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f0ef3-411a-4bad-8d67-5be600b6464b admin admin] RBAC: Authorizing `identity:delete_user(user_id=46af47ed34344f09ad9c54ed16e392ee)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:58.436339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f0ef3-411a-4bad-8d67-5be600b6464b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:58.547284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fc8f0ef3-411a-4bad-8d67-5be600b6464b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46af47ed34344f09ad9c54ed16e392ee'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:58.564331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fc8f0ef3-411a-4bad-8d67-5be600b6464b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46af47ed34344f09ad9c54ed16e392ee'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:58.594892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fc8f0ef3-411a-4bad-8d67-5be600b6464b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46af47ed34344f09ad9c54ed16e392ee'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:58.595666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fc8f0ef3-411a-4bad-8d67-5be600b6464b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46af47ed34344f09ad9c54ed16e392ee'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:58.617598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fc8f0ef3-411a-4bad-8d67-5be600b6464b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46af47ed34344f09ad9c54ed16e392ee'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:57:58.633001 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1780/3481] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:57:58 2022] DELETE /identity/v3/users/46af47ed34344f09ad9c54ed16e392ee => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:57:58.634941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4655bf2-8c00-48cd-99aa-c28db70df181 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:58.636549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4655bf2-8c00-48cd-99aa-c28db70df181 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:58.637026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4655bf2-8c00-48cd-99aa-c28db70df181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:58.671380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f4655bf2-8c00-48cd-99aa-c28db70df181 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:57:58.673521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f4655bf2-8c00-48cd-99aa-c28db70df181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:58.727877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:58.772545 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1702/3482] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:57:58.789900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f0ef3-411a-4bad-8d67-5be600b6464b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:57:58.791882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fc8f0ef3-411a-4bad-8d67-5be600b6464b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:57:58.926964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f0ef3-411a-4bad-8d67-5be600b6464b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:57:58.937928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8966db-365a-497d-8173-2a78c3a5ce67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:57:58.938585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8966db-365a-497d-8173-2a78c3a5ce67 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:57:58.938862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8966db-365a-497d-8173-2a78c3a5ce67 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:57:58.939140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8966db-365a-497d-8173-2a78c3a5ce67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:57:58.966567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8966db-365a-497d-8173-2a78c3a5ce67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:57:58.968046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8966db-365a-497d-8173-2a78c3a5ce67 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:57:58.980445 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1781/3483] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:57:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:58:00.197435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4655bf2-8c00-48cd-99aa-c28db70df181 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:00.199585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f4655bf2-8c00-48cd-99aa-c28db70df181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:00.244531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4655bf2-8c00-48cd-99aa-c28db70df181 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:00.262245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7103c7-31c2-407d-99e5-61711be08455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:00.265573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7103c7-31c2-407d-99e5-61711be08455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:00.265573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7103c7-31c2-407d-99e5-61711be08455 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:00.265573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7103c7-31c2-407d-99e5-61711be08455 admin admin] PATH_INFO: `/v3/projects/3e6f03ca2bb44f89b6f0b3c3adce1558` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:00.269586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7103c7-31c2-407d-99e5-61711be08455 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e6f03ca2bb44f89b6f0b3c3adce1558)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:00.275735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7103c7-31c2-407d-99e5-61711be08455 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:00.386076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9b7103c7-31c2-407d-99e5-61711be08455 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e6f03ca2bb44f89b6f0b3c3adce1558', 'request_id': 'req-9b7103c7-31c2-407d-99e5-61711be08455'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:00.401396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9b7103c7-31c2-407d-99e5-61711be08455 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e6f03ca2bb44f89b6f0b3c3adce1558', 'request_id': 'req-9b7103c7-31c2-407d-99e5-61711be08455'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:00.410350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9b7103c7-31c2-407d-99e5-61711be08455 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e6f03ca2bb44f89b6f0b3c3adce1558', 'request_id': 'req-9b7103c7-31c2-407d-99e5-61711be08455'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:00.437733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9b7103c7-31c2-407d-99e5-61711be08455 admin admin] The token cache is being invalidate because project 3e6f03ca2bb44f89b6f0b3c3adce1558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:00.439086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9b7103c7-31c2-407d-99e5-61711be08455 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:00.439965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9b7103c7-31c2-407d-99e5-61711be08455 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:00.442709 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1703/3484] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:00 2022] DELETE /identity/v3/projects/3e6f03ca2bb44f89b6f0b3c3adce1558 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:00.446448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8966db-365a-497d-8173-2a78c3a5ce67 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:00.468527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8966db-365a-497d-8173-2a78c3a5ce67 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:00.517055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:00.590594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c6705-b8e7-4c9d-8fa6-697d5aff72ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:00.592128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c6705-b8e7-4c9d-8fa6-697d5aff72ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:00.592705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c6705-b8e7-4c9d-8fa6-697d5aff72ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:00.594258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c6705-b8e7-4c9d-8fa6-697d5aff72ac admin admin] PATH_INFO: `/v3/users/33704d3fe1c14d8ea89aa999b2cb5d0c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:00.597038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c6705-b8e7-4c9d-8fa6-697d5aff72ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=33704d3fe1c14d8ea89aa999b2cb5d0c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:00.600199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c6705-b8e7-4c9d-8fa6-697d5aff72ac admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:00.731893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-921c6705-b8e7-4c9d-8fa6-697d5aff72ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33704d3fe1c14d8ea89aa999b2cb5d0c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:00.761875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-921c6705-b8e7-4c9d-8fa6-697d5aff72ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33704d3fe1c14d8ea89aa999b2cb5d0c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:00.804274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-921c6705-b8e7-4c9d-8fa6-697d5aff72ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33704d3fe1c14d8ea89aa999b2cb5d0c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:00.806286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-921c6705-b8e7-4c9d-8fa6-697d5aff72ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33704d3fe1c14d8ea89aa999b2cb5d0c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:00.834412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-921c6705-b8e7-4c9d-8fa6-697d5aff72ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33704d3fe1c14d8ea89aa999b2cb5d0c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:00.851903 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1782/3485] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:58:00 2022] DELETE /identity/v3/users/33704d3fe1c14d8ea89aa999b2cb5d0c => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:58:00.944154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89977c45-d506-4854-a634-55a848956ed2 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:00.944531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89977c45-d506-4854-a634-55a848956ed2 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:00.944894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89977c45-d506-4854-a634-55a848956ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:00.978399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-89977c45-d506-4854-a634-55a848956ed2 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:00.979707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-89977c45-d506-4854-a634-55a848956ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:01.098923 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1704/3486] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:58:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:01.143087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c6705-b8e7-4c9d-8fa6-697d5aff72ac admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:01.145338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-921c6705-b8e7-4c9d-8fa6-697d5aff72ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:01.318612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c6705-b8e7-4c9d-8fa6-697d5aff72ac admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:01.324278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312af6a-3563-402b-8032-660eefa18ee8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:01.324911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312af6a-3563-402b-8032-660eefa18ee8 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:01.325309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312af6a-3563-402b-8032-660eefa18ee8 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:01.325488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312af6a-3563-402b-8032-660eefa18ee8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:01.336235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312af6a-3563-402b-8032-660eefa18ee8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:01.337974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312af6a-3563-402b-8032-660eefa18ee8 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:01.349894 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1783/3487] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:58:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 20:58:03.304690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89977c45-d506-4854-a634-55a848956ed2 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:03.306276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-89977c45-d506-4854-a634-55a848956ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:03.432083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:03.439770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89977c45-d506-4854-a634-55a848956ed2 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:03.460148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bdb1a0-b4c9-4621-ab19-eb2bfa4fe72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:03.461749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdb1a0-b4c9-4621-ab19-eb2bfa4fe72a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:03.462220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdb1a0-b4c9-4621-ab19-eb2bfa4fe72a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:03.462675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdb1a0-b4c9-4621-ab19-eb2bfa4fe72a admin admin] PATH_INFO: `/v3/projects/1557c02911274e9ab1b6a5e95b045f91` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:03.464067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bdb1a0-b4c9-4621-ab19-eb2bfa4fe72a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1557c02911274e9ab1b6a5e95b045f91)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:03.468915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bdb1a0-b4c9-4621-ab19-eb2bfa4fe72a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:03.595369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-84bdb1a0-b4c9-4621-ab19-eb2bfa4fe72a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1557c02911274e9ab1b6a5e95b045f91', 'request_id': 'req-84bdb1a0-b4c9-4621-ab19-eb2bfa4fe72a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:03.622795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-84bdb1a0-b4c9-4621-ab19-eb2bfa4fe72a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1557c02911274e9ab1b6a5e95b045f91', 'request_id': 'req-84bdb1a0-b4c9-4621-ab19-eb2bfa4fe72a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:03.630758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-84bdb1a0-b4c9-4621-ab19-eb2bfa4fe72a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1557c02911274e9ab1b6a5e95b045f91', 'request_id': 'req-84bdb1a0-b4c9-4621-ab19-eb2bfa4fe72a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:03.663928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-84bdb1a0-b4c9-4621-ab19-eb2bfa4fe72a admin admin] The token cache is being invalidate because project 1557c02911274e9ab1b6a5e95b045f91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:03.664225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-84bdb1a0-b4c9-4621-ab19-eb2bfa4fe72a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:03.669344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-84bdb1a0-b4c9-4621-ab19-eb2bfa4fe72a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:03.671030 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1705/3488] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:03 2022] DELETE /identity/v3/projects/1557c02911274e9ab1b6a5e95b045f91 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:03.677463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e14a47-405c-4d18-91f4-7fd240a54ee9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:03.677947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e14a47-405c-4d18-91f4-7fd240a54ee9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:03.678334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e14a47-405c-4d18-91f4-7fd240a54ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:03.717390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-84e14a47-405c-4d18-91f4-7fd240a54ee9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:03.718846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-84e14a47-405c-4d18-91f4-7fd240a54ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:03.775624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:03.859797 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1784/3489] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:03.868506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862ba70-5409-466d-8b3c-6acacede8cae None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:03.868962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862ba70-5409-466d-8b3c-6acacede8cae None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:03.869529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862ba70-5409-466d-8b3c-6acacede8cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:03.991049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a862ba70-5409-466d-8b3c-6acacede8cae None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:03.993176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a862ba70-5409-466d-8b3c-6acacede8cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:04.021187 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1706/3490] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:04.024676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e14a47-405c-4d18-91f4-7fd240a54ee9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:04.051446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e14a47-405c-4d18-91f4-7fd240a54ee9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:04.073511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862ba70-5409-466d-8b3c-6acacede8cae None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:04.075105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a862ba70-5409-466d-8b3c-6acacede8cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:04.109214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411aeb15-6827-480d-af83-d280315a33a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:04.109875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411aeb15-6827-480d-af83-d280315a33a3 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:04.110164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411aeb15-6827-480d-af83-d280315a33a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:04.110440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411aeb15-6827-480d-af83-d280315a33a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:04.121020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411aeb15-6827-480d-af83-d280315a33a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:58:04.126719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411aeb15-6827-480d-af83-d280315a33a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:04.134994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411aeb15-6827-480d-af83-d280315a33a3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:04.153702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862ba70-5409-466d-8b3c-6acacede8cae None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:04.195401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f99fc-ffde-4a47-b112-ff4bc2f03f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:04.200910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f99fc-ffde-4a47-b112-ff4bc2f03f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:04.201211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f99fc-ffde-4a47-b112-ff4bc2f03f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:04.201492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f99fc-ffde-4a47-b112-ff4bc2f03f11 admin admin] PATH_INFO: `/v3/users/60d08233105349009c9dcf5f25b82229` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:04.214681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f99fc-ffde-4a47-b112-ff4bc2f03f11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60d08233105349009c9dcf5f25b82229)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:04.217111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f99fc-ffde-4a47-b112-ff4bc2f03f11 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:04.224338 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1785/3491] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:04.229379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411aeb15-6827-480d-af83-d280315a33a3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:04.249534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411aeb15-6827-480d-af83-d280315a33a3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:04.277589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4c001-b1ff-484c-969a-0641677317c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:04.278253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4c001-b1ff-484c-969a-0641677317c3 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:04.278562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4c001-b1ff-484c-969a-0641677317c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:04.278849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4c001-b1ff-484c-969a-0641677317c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:04.279542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b4c001-b1ff-484c-969a-0641677317c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:04.286818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b4c001-b1ff-484c-969a-0641677317c3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:04.370868 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1786/3492] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:58:04 2022] POST /identity/v3/projects => generated 389 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:58:04.371950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4c001-b1ff-484c-969a-0641677317c3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:04.399075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cc0f99fc-ffde-4a47-b112-ff4bc2f03f11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60d08233105349009c9dcf5f25b82229'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:04.400467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cc0f99fc-ffde-4a47-b112-ff4bc2f03f11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60d08233105349009c9dcf5f25b82229'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:04.401293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cc0f99fc-ffde-4a47-b112-ff4bc2f03f11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60d08233105349009c9dcf5f25b82229'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:04.420253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cc0f99fc-ffde-4a47-b112-ff4bc2f03f11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60d08233105349009c9dcf5f25b82229'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:04.472580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cc0f99fc-ffde-4a47-b112-ff4bc2f03f11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60d08233105349009c9dcf5f25b82229'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:04.501539 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1707/3493] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:58:04 2022] DELETE /identity/v3/users/60d08233105349009c9dcf5f25b82229 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:04.571645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4c001-b1ff-484c-969a-0641677317c3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:04.772275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b829006-863d-4690-81a6-4bea8aa00215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:04.773621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b829006-863d-4690-81a6-4bea8aa00215 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:04.774061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b829006-863d-4690-81a6-4bea8aa00215 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:04.774517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b829006-863d-4690-81a6-4bea8aa00215 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:04.775327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b829006-863d-4690-81a6-4bea8aa00215 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:04.777442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b829006-863d-4690-81a6-4bea8aa00215 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:04.849891 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1787/3494] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:58:04 2022] POST /identity/v3/users => generated 295 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:04.851316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f99fc-ffde-4a47-b112-ff4bc2f03f11 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:04.866461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cc0f99fc-ffde-4a47-b112-ff4bc2f03f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:04.905117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f99fc-ffde-4a47-b112-ff4bc2f03f11 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:04.927291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8d91f-e6e4-4e37-8a78-ff7c68689f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:04.927963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8d91f-e6e4-4e37-8a78-ff7c68689f6e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:04.928248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8d91f-e6e4-4e37-8a78-ff7c68689f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:04.928612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8d91f-e6e4-4e37-8a78-ff7c68689f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:04.929204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8d91f-e6e4-4e37-8a78-ff7c68689f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:04.930874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8d91f-e6e4-4e37-8a78-ff7c68689f6e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:04.961626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b829006-863d-4690-81a6-4bea8aa00215 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:04.973664 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1708/3495] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:04.977193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b829006-863d-4690-81a6-4bea8aa00215 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:04.995619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34507c1-c4f6-42fa-b882-806005e80ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:04.996527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34507c1-c4f6-42fa-b882-806005e80ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:04.996527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34507c1-c4f6-42fa-b882-806005e80ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:04.996805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34507c1-c4f6-42fa-b882-806005e80ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:04.998390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34507c1-c4f6-42fa-b882-806005e80ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:05.000223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34507c1-c4f6-42fa-b882-806005e80ce0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:05.022036 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1788/3496] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:05.026034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8d91f-e6e4-4e37-8a78-ff7c68689f6e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:05.048845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8d91f-e6e4-4e37-8a78-ff7c68689f6e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:05.072215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27952fff-c0cb-4650-ae88-98d9cf28329f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:05.073631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27952fff-c0cb-4650-ae88-98d9cf28329f admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:05.073921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27952fff-c0cb-4650-ae88-98d9cf28329f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:05.074200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27952fff-c0cb-4650-ae88-98d9cf28329f admin admin] PATH_INFO: `/v3/projects/32ff9960a594470e8df6a8abff16ba62/users/716bfeccd56f4444aef8fea0bba13edc/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:05.117260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27952fff-c0cb-4650-ae88-98d9cf28329f admin admin] RBAC: Authorizing `identity:create_grant(project_id=32ff9960a594470e8df6a8abff16ba62, user_id=716bfeccd56f4444aef8fea0bba13edc, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:05.123411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27952fff-c0cb-4650-ae88-98d9cf28329f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:05.195856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4c427-fc84-481d-957f-8a0677972027 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:05.196148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4c427-fc84-481d-957f-8a0677972027 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:05.196435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4c427-fc84-481d-957f-8a0677972027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:05.203159 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1709/3497] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:58:05 2022] PUT /identity/v3/projects/32ff9960a594470e8df6a8abff16ba62/users/716bfeccd56f4444aef8fea0bba13edc/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:05.278456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e1a4c427-fc84-481d-957f-8a0677972027 None None] MFA Rules not processed for user `716bfeccd56f4444aef8fea0bba13edc`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:05.279735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e1a4c427-fc84-481d-957f-8a0677972027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:05.311989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:05.481605 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1789/3498] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:05.488322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27952fff-c0cb-4650-ae88-98d9cf28329f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:05.533565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:05.677061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27952fff-c0cb-4650-ae88-98d9cf28329f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:05.692618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce990e-51eb-4835-82be-8330e3276ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:05.693495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce990e-51eb-4835-82be-8330e3276ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:05.694114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce990e-51eb-4835-82be-8330e3276ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:05.694699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce990e-51eb-4835-82be-8330e3276ee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:05.695501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce990e-51eb-4835-82be-8330e3276ee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:05.700806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce990e-51eb-4835-82be-8330e3276ee7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:05.785228 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1710/3499] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:58:05 2022] POST /identity/v3/projects => generated 389 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:05.789527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4c427-fc84-481d-957f-8a0677972027 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:05.815657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4617df-c9bf-48ed-afd7-e26f6fe8ea4e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:05.815997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4617df-c9bf-48ed-afd7-e26f6fe8ea4e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:05.816479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4617df-c9bf-48ed-afd7-e26f6fe8ea4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:05.819640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4c427-fc84-481d-957f-8a0677972027 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:05.914208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bb4617df-c9bf-48ed-afd7-e26f6fe8ea4e None None] MFA Rules not processed for user `0284a8ec6c4e41f0a1a88e2f11a4faa8`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:05.920342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bb4617df-c9bf-48ed-afd7-e26f6fe8ea4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:06.089318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc0293-fcce-49e1-9938-69c3bb6d9b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:06.091930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc0293-fcce-49e1-9938-69c3bb6d9b59 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:06.093859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc0293-fcce-49e1-9938-69c3bb6d9b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:06.094321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc0293-fcce-49e1-9938-69c3bb6d9b59 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:06.095149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc0293-fcce-49e1-9938-69c3bb6d9b59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:06.099669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc0293-fcce-49e1-9938-69c3bb6d9b59 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:06.102909 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1711/3500] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:05 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:06.104432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4617df-c9bf-48ed-afd7-e26f6fe8ea4e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:06.106913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bb4617df-c9bf-48ed-afd7-e26f6fe8ea4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:06.128105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4617df-c9bf-48ed-afd7-e26f6fe8ea4e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:06.136088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d53b4d-377a-4da7-9cd8-b8c98062f542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:06.137168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d53b4d-377a-4da7-9cd8-b8c98062f542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:06.137467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d53b4d-377a-4da7-9cd8-b8c98062f542 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:06.137935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d53b4d-377a-4da7-9cd8-b8c98062f542 admin admin] PATH_INFO: `/v3/projects/c007c403f0b8414eb5bb14d11897c71a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:06.142137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d53b4d-377a-4da7-9cd8-b8c98062f542 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c007c403f0b8414eb5bb14d11897c71a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:06.150478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d53b4d-377a-4da7-9cd8-b8c98062f542 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:06.151265 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1790/3501] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:58:05 2022] POST /identity/v3/users => generated 301 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:58:06.154832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc0293-fcce-49e1-9938-69c3bb6d9b59 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:06.160568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f6dc0293-fcce-49e1-9938-69c3bb6d9b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:06.288461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-06d53b4d-377a-4da7-9cd8-b8c98062f542 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c007c403f0b8414eb5bb14d11897c71a', 'request_id': 'req-06d53b4d-377a-4da7-9cd8-b8c98062f542'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:06.316468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-06d53b4d-377a-4da7-9cd8-b8c98062f542 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c007c403f0b8414eb5bb14d11897c71a', 'request_id': 'req-06d53b4d-377a-4da7-9cd8-b8c98062f542'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:06.325565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-06d53b4d-377a-4da7-9cd8-b8c98062f542 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c007c403f0b8414eb5bb14d11897c71a', 'request_id': 'req-06d53b4d-377a-4da7-9cd8-b8c98062f542'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:06.353145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-06d53b4d-377a-4da7-9cd8-b8c98062f542 admin admin] The token cache is being invalidate because project c007c403f0b8414eb5bb14d11897c71a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:06.353708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-06d53b4d-377a-4da7-9cd8-b8c98062f542 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:06.355652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-06d53b4d-377a-4da7-9cd8-b8c98062f542 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:06.358109 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1712/3502] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:06 2022] DELETE /identity/v3/projects/c007c403f0b8414eb5bb14d11897c71a => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:06.359235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d53b4d-377a-4da7-9cd8-b8c98062f542 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:06.363768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-06d53b4d-377a-4da7-9cd8-b8c98062f542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:06.392182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc0293-fcce-49e1-9938-69c3bb6d9b59 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:06.396033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:06.505894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:06.521679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b6001-6cc4-496c-9c80-bd15b722ea1e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:06.522681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b6001-6cc4-496c-9c80-bd15b722ea1e service glance] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:06.523252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b6001-6cc4-496c-9c80-bd15b722ea1e service glance] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:06.523868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b6001-6cc4-496c-9c80-bd15b722ea1e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:06.525907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e09b6001-6cc4-496c-9c80-bd15b722ea1e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:06.618362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d53b4d-377a-4da7-9cd8-b8c98062f542 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:06.639078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcddb0-2c72-424b-a010-686bdad6bc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:06.639078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcddb0-2c72-424b-a010-686bdad6bc34 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:06.639078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcddb0-2c72-424b-a010-686bdad6bc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:06.639078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcddb0-2c72-424b-a010-686bdad6bc34 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:06.639497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcddb0-2c72-424b-a010-686bdad6bc34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:06.645149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcddb0-2c72-424b-a010-686bdad6bc34 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:06.693825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b6001-6cc4-496c-9c80-bd15b722ea1e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:06.698051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b6001-6cc4-496c-9c80-bd15b722ea1e service glance] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:06.705485 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1713/3503] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:06.710872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56faa32-a09b-4df7-85f7-fe992423c3c1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:06.711168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56faa32-a09b-4df7-85f7-fe992423c3c1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:06.711449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56faa32-a09b-4df7-85f7-fe992423c3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:06.734623 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1791/3504] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:58:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:58:06.735580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b6001-6cc4-496c-9c80-bd15b722ea1e service glance] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:06.773894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b6001-6cc4-496c-9c80-bd15b722ea1e service glance] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:06.776110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b56faa32-a09b-4df7-85f7-fe992423c3c1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:06.777362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b56faa32-a09b-4df7-85f7-fe992423c3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:06.787587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10847dc9-0e01-451a-ab82-867674c88732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:06.788278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10847dc9-0e01-451a-ab82-867674c88732 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:06.788565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10847dc9-0e01-451a-ab82-867674c88732 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:06.788842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10847dc9-0e01-451a-ab82-867674c88732 admin admin] PATH_INFO: `/v3/projects/9f2ac590b6be415da95f7932d3249798/users/73ca2a04a7d84902912cbce8ccd82525/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:06.811369 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1714/3505] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:06.813501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e06a66-3a05-46a3-8d68-a8517a8b711d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:06.813798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e06a66-3a05-46a3-8d68-a8517a8b711d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:06.815220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e06a66-3a05-46a3-8d68-a8517a8b711d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:06.857255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10847dc9-0e01-451a-ab82-867674c88732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f2ac590b6be415da95f7932d3249798, user_id=73ca2a04a7d84902912cbce8ccd82525, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:06.864933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10847dc9-0e01-451a-ab82-867674c88732 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:06.882911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-21e06a66-3a05-46a3-8d68-a8517a8b711d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:06.885495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-21e06a66-3a05-46a3-8d68-a8517a8b711d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:06.894207 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1792/3506] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:58:06 2022] PUT /identity/v3/projects/9f2ac590b6be415da95f7932d3249798/users/73ca2a04a7d84902912cbce8ccd82525/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:06.899051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10847dc9-0e01-451a-ab82-867674c88732 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:07.061885 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1715/3507] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:07.067185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e06a66-3a05-46a3-8d68-a8517a8b711d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:07.088534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10847dc9-0e01-451a-ab82-867674c88732 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:07.112458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e06a66-3a05-46a3-8d68-a8517a8b711d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:07.121563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645ef3b-a70b-41d3-9397-fdbcf39f1beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:07.123067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645ef3b-a70b-41d3-9397-fdbcf39f1beb admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:07.124553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645ef3b-a70b-41d3-9397-fdbcf39f1beb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:07.124979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645ef3b-a70b-41d3-9397-fdbcf39f1beb admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:07.126229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645ef3b-a70b-41d3-9397-fdbcf39f1beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:07.128721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645ef3b-a70b-41d3-9397-fdbcf39f1beb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:07.132005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4bf0a-4ae2-4790-baef-b990c21043e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:07.132811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4bf0a-4ae2-4790-baef-b990c21043e4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:07.134827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4bf0a-4ae2-4790-baef-b990c21043e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:07.137871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4bf0a-4ae2-4790-baef-b990c21043e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:07.138751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4bf0a-4ae2-4790-baef-b990c21043e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:58:07.142413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4bf0a-4ae2-4790-baef-b990c21043e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:07.148553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4bf0a-4ae2-4790-baef-b990c21043e4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:07.192894 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1793/3508] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:07.196770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645ef3b-a70b-41d3-9397-fdbcf39f1beb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:07.215538 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1716/3509] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:07.216724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4bf0a-4ae2-4790-baef-b990c21043e4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:07.236129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645ef3b-a70b-41d3-9397-fdbcf39f1beb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:07.256509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4bf0a-4ae2-4790-baef-b990c21043e4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:07.269746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57c193-aeec-4a76-96e1-733656cf66a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:07.270771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57c193-aeec-4a76-96e1-733656cf66a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:07.272048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57c193-aeec-4a76-96e1-733656cf66a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:07.272965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57c193-aeec-4a76-96e1-733656cf66a2 admin admin] PATH_INFO: `/v3/projects/9f2ac590b6be415da95f7932d3249798/users/73ca2a04a7d84902912cbce8ccd82525/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:07.279613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67f136-f47e-49e8-a3ed-d121ef12ae39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:07.285742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67f136-f47e-49e8-a3ed-d121ef12ae39 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:07.286023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57c193-aeec-4a76-96e1-733656cf66a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f2ac590b6be415da95f7932d3249798, user_id=73ca2a04a7d84902912cbce8ccd82525, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:07.289532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67f136-f47e-49e8-a3ed-d121ef12ae39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:07.293475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67f136-f47e-49e8-a3ed-d121ef12ae39 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:07.294608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a67f136-f47e-49e8-a3ed-d121ef12ae39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:07.295709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57c193-aeec-4a76-96e1-733656cf66a2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:07.301207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a67f136-f47e-49e8-a3ed-d121ef12ae39 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:07.346930 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1794/3510] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:58:07 2022] PUT /identity/v3/projects/9f2ac590b6be415da95f7932d3249798/users/73ca2a04a7d84902912cbce8ccd82525/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:07.359897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c28878-38fc-4d4e-bf3b-3e7e82f5dccc None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:07.375033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c28878-38fc-4d4e-bf3b-3e7e82f5dccc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:07.375348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c28878-38fc-4d4e-bf3b-3e7e82f5dccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:07.464770 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1717/3511] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:58:07 2022] POST /identity/v3/projects => generated 365 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:07.465758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67f136-f47e-49e8-a3ed-d121ef12ae39 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:07.527399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-29c28878-38fc-4d4e-bf3b-3e7e82f5dccc None None] MFA Rules not processed for user `73ca2a04a7d84902912cbce8ccd82525`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:07.529469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-29c28878-38fc-4d4e-bf3b-3e7e82f5dccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:07.747417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:07.757216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67f136-f47e-49e8-a3ed-d121ef12ae39 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:07.772874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9224995-3ca1-4b57-b25e-2d12eb9e4192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:07.773288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9224995-3ca1-4b57-b25e-2d12eb9e4192 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:07.775464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9224995-3ca1-4b57-b25e-2d12eb9e4192 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:07.775688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9224995-3ca1-4b57-b25e-2d12eb9e4192 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:07.776343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9224995-3ca1-4b57-b25e-2d12eb9e4192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:07.780147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9224995-3ca1-4b57-b25e-2d12eb9e4192 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:07.816709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:07.859197 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1718/3512] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:58:07 2022] POST /identity/v3/users => generated 283 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:07.861454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9224995-3ca1-4b57-b25e-2d12eb9e4192 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:07.900138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9224995-3ca1-4b57-b25e-2d12eb9e4192 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:07.923884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9bcb9-56a0-4cbb-8fba-77d0c7c7a638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:07.927973 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1795/3513] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:07 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 572 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:58:07.929044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c28878-38fc-4d4e-bf3b-3e7e82f5dccc None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:07.929770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9bcb9-56a0-4cbb-8fba-77d0c7c7a638 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:07.930326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9bcb9-56a0-4cbb-8fba-77d0c7c7a638 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:07.930797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9bcb9-56a0-4cbb-8fba-77d0c7c7a638 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:07.931794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9bcb9-56a0-4cbb-8fba-77d0c7c7a638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:07.935257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9bcb9-56a0-4cbb-8fba-77d0c7c7a638 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:07.968710 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1719/3514] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:07.973610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9bcb9-56a0-4cbb-8fba-77d0c7c7a638 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:07.986092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9bcb9-56a0-4cbb-8fba-77d0c7c7a638 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:07.993958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c28878-38fc-4d4e-bf3b-3e7e82f5dccc None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:07.995890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29600895-b5c0-4d81-80fa-e00d522faa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:07.996712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29600895-b5c0-4d81-80fa-e00d522faa62 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:07.997137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29600895-b5c0-4d81-80fa-e00d522faa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:07.997578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29600895-b5c0-4d81-80fa-e00d522faa62 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:07.998364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29600895-b5c0-4d81-80fa-e00d522faa62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:08.000284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29600895-b5c0-4d81-80fa-e00d522faa62 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:08.023587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9331c-d490-4bcc-beee-602c7013c78b tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ca2a04a7d84902912cbce8ccd82525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f2ac590b6be415da95f7932d3249798', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:08.024242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9331c-d490-4bcc-beee-602c7013c78b tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:08.024522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9331c-d490-4bcc-beee-602c7013c78b tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:08.024806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9331c-d490-4bcc-beee-602c7013c78b tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] PATH_INFO: `/v3/projects/32ff9960a594470e8df6a8abff16ba62` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:08.042414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9331c-d490-4bcc-beee-602c7013c78b tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorizing `identity:get_project(project_id=32ff9960a594470e8df6a8abff16ba62)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:08.044521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9331c-d490-4bcc-beee-602c7013c78b tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:08.045049 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1720/3515] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:08.058036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29600895-b5c0-4d81-80fa-e00d522faa62 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:08.068163 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1796/3516] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 20:58:07 2022] GET /identity/v3/projects/32ff9960a594470e8df6a8abff16ba62 => generated 389 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:08.069059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9331c-d490-4bcc-beee-602c7013c78b tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:08.084871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29600895-b5c0-4d81-80fa-e00d522faa62 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:08.092154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c107f3-247e-4507-8f12-8487b1ef6cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:08.093173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c107f3-247e-4507-8f12-8487b1ef6cd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:08.093626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c107f3-247e-4507-8f12-8487b1ef6cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:08.094064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c107f3-247e-4507-8f12-8487b1ef6cd3 admin admin] PATH_INFO: `/v3/projects/bea57d3c53f74e3ea78e9e4bc790c434/users/775ff9091a794faaacc05a37550c8a72/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:08.131818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c107f3-247e-4507-8f12-8487b1ef6cd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bea57d3c53f74e3ea78e9e4bc790c434, user_id=775ff9091a794faaacc05a37550c8a72, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:08.133770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9331c-d490-4bcc-beee-602c7013c78b tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:08.135415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c107f3-247e-4507-8f12-8487b1ef6cd3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:08.166453 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1721/3517] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:58:08 2022] PUT /identity/v3/projects/bea57d3c53f74e3ea78e9e4bc790c434/users/775ff9091a794faaacc05a37550c8a72/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:08.168284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d8997-4cda-426f-9f02-319bbc1cdb48 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:08.168738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d8997-4cda-426f-9f02-319bbc1cdb48 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:08.169155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d8997-4cda-426f-9f02-319bbc1cdb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:08.176979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2749e4-9131-4227-ac73-34adb65b4ddb tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ca2a04a7d84902912cbce8ccd82525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f2ac590b6be415da95f7932d3249798', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:08.183360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2749e4-9131-4227-ac73-34adb65b4ddb tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:08.183678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2749e4-9131-4227-ac73-34adb65b4ddb tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:08.186584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2749e4-9131-4227-ac73-34adb65b4ddb tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:08.187294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2749e4-9131-4227-ac73-34adb65b4ddb tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:08.194088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2749e4-9131-4227-ac73-34adb65b4ddb tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:08.229522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c56d8997-4cda-426f-9f02-319bbc1cdb48 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:08.235988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c56d8997-4cda-426f-9f02-319bbc1cdb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:08.307688 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1797/3518] 10.222.0.17 () {60 vars in 1152 bytes} [Sat Oct 1 20:58:08 2022] POST /identity/v3/projects => generated 339 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:08.308690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2749e4-9131-4227-ac73-34adb65b4ddb tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:08.562772 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1722/3519] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:08.616032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d8997-4cda-426f-9f02-319bbc1cdb48 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:08.617821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c56d8997-4cda-426f-9f02-319bbc1cdb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:08.698302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2749e4-9131-4227-ac73-34adb65b4ddb tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:08.736084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4dd49-a4a5-46a7-9de4-19736e38d2c9 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ca2a04a7d84902912cbce8ccd82525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f2ac590b6be415da95f7932d3249798', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:08.736860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4dd49-a4a5-46a7-9de4-19736e38d2c9 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:08.737145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4dd49-a4a5-46a7-9de4-19736e38d2c9 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:08.740543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4dd49-a4a5-46a7-9de4-19736e38d2c9 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:08.741339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4dd49-a4a5-46a7-9de4-19736e38d2c9 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:08.748825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4dd49-a4a5-46a7-9de4-19736e38d2c9 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:08.855337 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1798/3520] 10.222.0.17 () {58 vars in 1131 bytes} [Sat Oct 1 20:58:08 2022] GET /identity/v3/projects => generated 6923 bytes in 544 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:08.857294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4dd49-a4a5-46a7-9de4-19736e38d2c9 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:08.875694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4dd49-a4a5-46a7-9de4-19736e38d2c9 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:08.881531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf5f56-28ac-4164-82f2-e2af3d767510 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ca2a04a7d84902912cbce8ccd82525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f2ac590b6be415da95f7932d3249798', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:08.883233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf5f56-28ac-4164-82f2-e2af3d767510 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:08.884545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf5f56-28ac-4164-82f2-e2af3d767510 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:08.887416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf5f56-28ac-4164-82f2-e2af3d767510 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] PATH_INFO: `/v3/projects/32ff9960a594470e8df6a8abff16ba62` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:08.891764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf5f56-28ac-4164-82f2-e2af3d767510 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorizing `identity:get_project(project_id=32ff9960a594470e8df6a8abff16ba62)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:08.895254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf5f56-28ac-4164-82f2-e2af3d767510 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:08.903829 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1799/3521] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 20:58:08 2022] GET /identity/v3/projects/32ff9960a594470e8df6a8abff16ba62 => generated 389 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:08.904984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf5f56-28ac-4164-82f2-e2af3d767510 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:08.925945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf5f56-28ac-4164-82f2-e2af3d767510 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:08.933849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d8997-4cda-426f-9f02-319bbc1cdb48 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:08.938076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6721532a-f224-45c6-ab24-7853155054b5 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ca2a04a7d84902912cbce8ccd82525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f2ac590b6be415da95f7932d3249798', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:08.939618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721532a-f224-45c6-ab24-7853155054b5 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:08.941367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721532a-f224-45c6-ab24-7853155054b5 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:08.941808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721532a-f224-45c6-ab24-7853155054b5 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] PATH_INFO: `/v3/projects/00151992bd2e4a4b948cd6e832cedbff` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:08.945883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6721532a-f224-45c6-ab24-7853155054b5 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorizing `identity:get_project(project_id=00151992bd2e4a4b948cd6e832cedbff)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:08.963113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6721532a-f224-45c6-ab24-7853155054b5 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:08.979815 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1800/3522] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 20:58:08 2022] GET /identity/v3/projects/00151992bd2e4a4b948cd6e832cedbff => generated 339 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:08.981001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6721532a-f224-45c6-ab24-7853155054b5 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:09.012727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6721532a-f224-45c6-ab24-7853155054b5 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:09.017705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55adf27c-defd-45b4-9e3e-14cc67ef3abc tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ca2a04a7d84902912cbce8ccd82525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f2ac590b6be415da95f7932d3249798', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:09.019394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adf27c-defd-45b4-9e3e-14cc67ef3abc tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:09.019826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adf27c-defd-45b4-9e3e-14cc67ef3abc tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:09.020239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adf27c-defd-45b4-9e3e-14cc67ef3abc tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:09.021466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55adf27c-defd-45b4-9e3e-14cc67ef3abc tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:09.023508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55adf27c-defd-45b4-9e3e-14cc67ef3abc tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:09.074723 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1801/3523] 10.222.0.17 () {58 vars in 1131 bytes} [Sat Oct 1 20:58:08 2022] GET /identity/v3/projects => generated 6923 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:09.079454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55adf27c-defd-45b4-9e3e-14cc67ef3abc tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:09.117307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55adf27c-defd-45b4-9e3e-14cc67ef3abc tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:09.137291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65ab8d-291f-4ff1-95d6-2c498fbe6623 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ca2a04a7d84902912cbce8ccd82525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f2ac590b6be415da95f7932d3249798', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:09.138418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65ab8d-291f-4ff1-95d6-2c498fbe6623 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:09.138937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65ab8d-291f-4ff1-95d6-2c498fbe6623 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:09.139548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65ab8d-291f-4ff1-95d6-2c498fbe6623 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:09.140398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65ab8d-291f-4ff1-95d6-2c498fbe6623 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:58:09.140837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65ab8d-291f-4ff1-95d6-2c498fbe6623 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:09.156143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65ab8d-291f-4ff1-95d6-2c498fbe6623 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:09.234182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1cbe3-1a16-459b-800d-364b02828acc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:09.235190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1cbe3-1a16-459b-800d-364b02828acc service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:09.235604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1cbe3-1a16-459b-800d-364b02828acc service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:09.236013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1cbe3-1a16-459b-800d-364b02828acc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:09.278768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:09.307305 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1802/3524] 10.222.0.17 () {58 vars in 1166 bytes} [Sat Oct 1 20:58:09 2022] GET /identity/v3/projects?domain_id=default => generated 6592 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:09.308349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65ab8d-291f-4ff1-95d6-2c498fbe6623 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:09.345790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65ab8d-291f-4ff1-95d6-2c498fbe6623 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:09.377113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d41401-8065-44fe-bbd2-5e101dd049c8 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ca2a04a7d84902912cbce8ccd82525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f2ac590b6be415da95f7932d3249798', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:09.377782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d41401-8065-44fe-bbd2-5e101dd049c8 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:09.378066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d41401-8065-44fe-bbd2-5e101dd049c8 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:09.378344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d41401-8065-44fe-bbd2-5e101dd049c8 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:09.387195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d41401-8065-44fe-bbd2-5e101dd049c8 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:09.395993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d41401-8065-44fe-bbd2-5e101dd049c8 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:09.479108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1cbe3-1a16-459b-800d-364b02828acc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:09.480906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1cbe3-1a16-459b-800d-364b02828acc service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:09.514681 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1723/3525] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:58:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 899 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:58:09.530718 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1803/3526] 10.222.0.17 () {58 vars in 1131 bytes} [Sat Oct 1 20:58:09 2022] GET /identity/v3/projects => generated 6923 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:09.540077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1cbe3-1a16-459b-800d-364b02828acc service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:09.566903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1cbe3-1a16-459b-800d-364b02828acc service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:09.580823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10fb12-1974-491e-9a67-149d2397e803 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ca2a04a7d84902912cbce8ccd82525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f2ac590b6be415da95f7932d3249798', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:09.581742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10fb12-1974-491e-9a67-149d2397e803 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:09.582173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10fb12-1974-491e-9a67-149d2397e803 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:09.582619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10fb12-1974-491e-9a67-149d2397e803 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:09.586340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10fb12-1974-491e-9a67-149d2397e803 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:58:09.586955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10fb12-1974-491e-9a67-149d2397e803 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:09.588135 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-ec10fb12-1974-491e-9a67-149d2397e803 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:58:09.588135 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:58:09.588135 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:58:09.591784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10fb12-1974-491e-9a67-149d2397e803 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:09.668315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d41401-8065-44fe-bbd2-5e101dd049c8 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:09.674944 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1724/3527] 10.222.0.17 () {58 vars in 1166 bytes} [Sat Oct 1 20:58:09 2022] GET /identity/v3/projects?domain_id=default => generated 6592 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:09.684845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d41401-8065-44fe-bbd2-5e101dd049c8 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:09.724261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31c296-ffb0-4e1a-8384-dad91f9200c1 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ca2a04a7d84902912cbce8ccd82525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f2ac590b6be415da95f7932d3249798', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:09.728662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31c296-ffb0-4e1a-8384-dad91f9200c1 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:09.728972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31c296-ffb0-4e1a-8384-dad91f9200c1 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:09.729246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31c296-ffb0-4e1a-8384-dad91f9200c1 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:09.732918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac31c296-ffb0-4e1a-8384-dad91f9200c1 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:09.740444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac31c296-ffb0-4e1a-8384-dad91f9200c1 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:09.846911 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1804/3528] 10.222.0.17 () {58 vars in 1131 bytes} [Sat Oct 1 20:58:09 2022] GET /identity/v3/projects => generated 6923 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:58:09.848159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10fb12-1974-491e-9a67-149d2397e803 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:09.910999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10fb12-1974-491e-9a67-149d2397e803 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:09.962539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8686d64-b18f-42ce-9033-a5db78931d14 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ca2a04a7d84902912cbce8ccd82525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f2ac590b6be415da95f7932d3249798', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:09.964118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8686d64-b18f-42ce-9033-a5db78931d14 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:09.964600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8686d64-b18f-42ce-9033-a5db78931d14 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:09.965144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8686d64-b18f-42ce-9033-a5db78931d14 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:09.969390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8686d64-b18f-42ce-9033-a5db78931d14 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Adding query filter params (name=tempest-ListProjectsStaticTestJSON-1251991043) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:58:09.969919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8686d64-b18f-42ce-9033-a5db78931d14 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:10.006009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8686d64-b18f-42ce-9033-a5db78931d14 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:10.051190 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1725/3529] 10.222.0.17 () {58 vars in 1232 bytes} [Sat Oct 1 20:58:09 2022] GET /identity/v3/projects?name=tempest-ListProjectsStaticTestJSON-1251991043 => generated 537 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:10.057161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31c296-ffb0-4e1a-8384-dad91f9200c1 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:10.081230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31c296-ffb0-4e1a-8384-dad91f9200c1 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:10.110873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbaf12-5b62-41f6-a724-8e52bb8a21bf tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ca2a04a7d84902912cbce8ccd82525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f2ac590b6be415da95f7932d3249798', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:10.114121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbaf12-5b62-41f6-a724-8e52bb8a21bf tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:10.117748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbaf12-5b62-41f6-a724-8e52bb8a21bf tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:10.117748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbaf12-5b62-41f6-a724-8e52bb8a21bf tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] PATH_INFO: `/v3/projects/00151992bd2e4a4b948cd6e832cedbff` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:10.126537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedbaf12-5b62-41f6-a724-8e52bb8a21bf tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorizing `identity:delete_project(project_id=00151992bd2e4a4b948cd6e832cedbff)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:10.126537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedbaf12-5b62-41f6-a724-8e52bb8a21bf tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:10.209655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8686d64-b18f-42ce-9033-a5db78931d14 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:10.212382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b8686d64-b18f-42ce-9033-a5db78931d14 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:10.251065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8686d64-b18f-42ce-9033-a5db78931d14 tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:10.267656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775742b-d79a-4b5b-91c0-7129fe4a6951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:10.270843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775742b-d79a-4b5b-91c0-7129fe4a6951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:10.271006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775742b-d79a-4b5b-91c0-7129fe4a6951 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:10.272627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775742b-d79a-4b5b-91c0-7129fe4a6951 admin admin] PATH_INFO: `/v3/users/1d180849a694494e89843c0c315acd6d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:10.284652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9775742b-d79a-4b5b-91c0-7129fe4a6951 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d180849a694494e89843c0c315acd6d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:10.300717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9775742b-d79a-4b5b-91c0-7129fe4a6951 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:10.332731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:10.349180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fedbaf12-5b62-41f6-a724-8e52bb8a21bf tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00151992bd2e4a4b948cd6e832cedbff', 'request_id': 'req-fedbaf12-5b62-41f6-a724-8e52bb8a21bf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:10.358130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fedbaf12-5b62-41f6-a724-8e52bb8a21bf tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00151992bd2e4a4b948cd6e832cedbff', 'request_id': 'req-fedbaf12-5b62-41f6-a724-8e52bb8a21bf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:10.369460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fedbaf12-5b62-41f6-a724-8e52bb8a21bf tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00151992bd2e4a4b948cd6e832cedbff', 'request_id': 'req-fedbaf12-5b62-41f6-a724-8e52bb8a21bf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:10.397521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fedbaf12-5b62-41f6-a724-8e52bb8a21bf tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] The token cache is being invalidate because project 00151992bd2e4a4b948cd6e832cedbff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:10.398096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fedbaf12-5b62-41f6-a724-8e52bb8a21bf tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:10.398986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fedbaf12-5b62-41f6-a724-8e52bb8a21bf tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:10.401051 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1805/3530] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:10 2022] DELETE /identity/v3/projects/00151992bd2e4a4b948cd6e832cedbff => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:10.405132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbaf12-5b62-41f6-a724-8e52bb8a21bf tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:10.407152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fedbaf12-5b62-41f6-a724-8e52bb8a21bf tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:10.465312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9775742b-d79a-4b5b-91c0-7129fe4a6951 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d180849a694494e89843c0c315acd6d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:10.468188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9775742b-d79a-4b5b-91c0-7129fe4a6951 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d180849a694494e89843c0c315acd6d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:10.470957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9775742b-d79a-4b5b-91c0-7129fe4a6951 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d180849a694494e89843c0c315acd6d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:10.492615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9775742b-d79a-4b5b-91c0-7129fe4a6951 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d180849a694494e89843c0c315acd6d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:10.524266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9775742b-d79a-4b5b-91c0-7129fe4a6951 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d180849a694494e89843c0c315acd6d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:10.563184 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1726/3531] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:58:10 2022] DELETE /identity/v3/users/1d180849a694494e89843c0c315acd6d => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:58:10.629483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbaf12-5b62-41f6-a724-8e52bb8a21bf tempest-ListProjectsStaticTestJSON-1992978680 tempest-ListProjectsStaticTestJSON-1992978680-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:10.873048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8c01e-7416-46d1-929d-8b712ec016dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:10.873853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8c01e-7416-46d1-929d-8b712ec016dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:10.874193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8c01e-7416-46d1-929d-8b712ec016dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:10.875683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8c01e-7416-46d1-929d-8b712ec016dc admin admin] PATH_INFO: `/v3/users/716bfeccd56f4444aef8fea0bba13edc` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:10.878198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a8c01e-7416-46d1-929d-8b712ec016dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=716bfeccd56f4444aef8fea0bba13edc)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:10.884425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a8c01e-7416-46d1-929d-8b712ec016dc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:11.068478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c4a8c01e-7416-46d1-929d-8b712ec016dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '716bfeccd56f4444aef8fea0bba13edc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:11.079882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c4a8c01e-7416-46d1-929d-8b712ec016dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '716bfeccd56f4444aef8fea0bba13edc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:11.109566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c4a8c01e-7416-46d1-929d-8b712ec016dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '716bfeccd56f4444aef8fea0bba13edc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:11.110329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c4a8c01e-7416-46d1-929d-8b712ec016dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '716bfeccd56f4444aef8fea0bba13edc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:11.130374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:11.137658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c4a8c01e-7416-46d1-929d-8b712ec016dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '716bfeccd56f4444aef8fea0bba13edc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:11.152042 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1806/3532] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:58:10 2022] DELETE /identity/v3/users/716bfeccd56f4444aef8fea0bba13edc => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:11.154190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b103784-3ac7-42a2-9eb5-1113cd066f2f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:11.154762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b103784-3ac7-42a2-9eb5-1113cd066f2f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:11.154762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b103784-3ac7-42a2-9eb5-1113cd066f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:11.193892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7b103784-3ac7-42a2-9eb5-1113cd066f2f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:11.198798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7b103784-3ac7-42a2-9eb5-1113cd066f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:11.239038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8c01e-7416-46d1-929d-8b712ec016dc admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:11.240500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c4a8c01e-7416-46d1-929d-8b712ec016dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:11.427041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:11.450381 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1727/3533] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:11.460635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b103784-3ac7-42a2-9eb5-1113cd066f2f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:11.462414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7b103784-3ac7-42a2-9eb5-1113cd066f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:11.609300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8c01e-7416-46d1-929d-8b712ec016dc admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:11.651401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d3f33-6c8a-480b-8238-5bc9f89da1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:11.651984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d3f33-6c8a-480b-8238-5bc9f89da1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:11.652270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d3f33-6c8a-480b-8238-5bc9f89da1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:11.652712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d3f33-6c8a-480b-8238-5bc9f89da1e4 admin admin] PATH_INFO: `/v3/projects/9c0b38cea2c345c68c51c4d8cb9c07e7` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:11.653706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b103784-3ac7-42a2-9eb5-1113cd066f2f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:11.654255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d3f33-6c8a-480b-8238-5bc9f89da1e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c0b38cea2c345c68c51c4d8cb9c07e7)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:11.657079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d3f33-6c8a-480b-8238-5bc9f89da1e4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:11.666568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df874378-78ce-4354-81fa-e67c16c3f762 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:11.667444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df874378-78ce-4354-81fa-e67c16c3f762 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:11.667865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df874378-78ce-4354-81fa-e67c16c3f762 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:11.670165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df874378-78ce-4354-81fa-e67c16c3f762 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:11.700215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df874378-78ce-4354-81fa-e67c16c3f762 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:11.704054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df874378-78ce-4354-81fa-e67c16c3f762 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:11.739970 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1728/3534] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:58:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:58:11.817013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9d5d3f33-6c8a-480b-8238-5bc9f89da1e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c0b38cea2c345c68c51c4d8cb9c07e7', 'request_id': 'req-9d5d3f33-6c8a-480b-8238-5bc9f89da1e4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:11.891284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9d5d3f33-6c8a-480b-8238-5bc9f89da1e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c0b38cea2c345c68c51c4d8cb9c07e7', 'request_id': 'req-9d5d3f33-6c8a-480b-8238-5bc9f89da1e4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:11.912212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9d5d3f33-6c8a-480b-8238-5bc9f89da1e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c0b38cea2c345c68c51c4d8cb9c07e7', 'request_id': 'req-9d5d3f33-6c8a-480b-8238-5bc9f89da1e4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:11.948426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9d5d3f33-6c8a-480b-8238-5bc9f89da1e4 admin admin] The token cache is being invalidate because project 9c0b38cea2c345c68c51c4d8cb9c07e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:11.948716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9d5d3f33-6c8a-480b-8238-5bc9f89da1e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:11.949474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9d5d3f33-6c8a-480b-8238-5bc9f89da1e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:11.959353 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1807/3535] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:11 2022] DELETE /identity/v3/projects/9c0b38cea2c345c68c51c4d8cb9c07e7 => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:58:11.965528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35f693-5891-4e13-8b77-f8c1785ed099 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:11.966120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35f693-5891-4e13-8b77-f8c1785ed099 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:11.966802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35f693-5891-4e13-8b77-f8c1785ed099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:12.048421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0b35f693-5891-4e13-8b77-f8c1785ed099 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:12.049706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0b35f693-5891-4e13-8b77-f8c1785ed099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:12.260599 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1729/3536] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:58:12.270541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f84a2-3130-417f-a374-a0aef51cc75d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:12.271184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f84a2-3130-417f-a374-a0aef51cc75d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:12.271184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f84a2-3130-417f-a374-a0aef51cc75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:12.394018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-428f84a2-3130-417f-a374-a0aef51cc75d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:12.395555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-428f84a2-3130-417f-a374-a0aef51cc75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:12.419872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35f693-5891-4e13-8b77-f8c1785ed099 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:12.420378 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1808/3537] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:12.434415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35f693-5891-4e13-8b77-f8c1785ed099 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:12.461607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae6d47-2e9f-4df4-ac83-e41721796038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:12.462244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae6d47-2e9f-4df4-ac83-e41721796038 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:12.462549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae6d47-2e9f-4df4-ac83-e41721796038 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:12.462831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae6d47-2e9f-4df4-ac83-e41721796038 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:12.463491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae6d47-2e9f-4df4-ac83-e41721796038 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:58:12.471858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae6d47-2e9f-4df4-ac83-e41721796038 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:12.473739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae6d47-2e9f-4df4-ac83-e41721796038 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:12.526929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f84a2-3130-417f-a374-a0aef51cc75d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:12.534635 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1730/3538] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:58:12.539419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f84a2-3130-417f-a374-a0aef51cc75d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:12.584837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28893342-0be8-4975-b045-96b490366929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:12.587165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28893342-0be8-4975-b045-96b490366929 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:12.591610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28893342-0be8-4975-b045-96b490366929 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:12.593870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28893342-0be8-4975-b045-96b490366929 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:12.600270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28893342-0be8-4975-b045-96b490366929 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:12.610403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28893342-0be8-4975-b045-96b490366929 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:12.701825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae6d47-2e9f-4df4-ac83-e41721796038 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:12.702613 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1809/3539] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:58:12 2022] POST /identity/v3/projects => generated 379 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:12.713087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae6d47-2e9f-4df4-ac83-e41721796038 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:12.759606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28893342-0be8-4975-b045-96b490366929 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:12.761392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-28893342-0be8-4975-b045-96b490366929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:12.924015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8ea27-fc35-4bb5-8863-02a99f5eaca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:12.924649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8ea27-fc35-4bb5-8863-02a99f5eaca9 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:12.924931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8ea27-fc35-4bb5-8863-02a99f5eaca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:12.925226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8ea27-fc35-4bb5-8863-02a99f5eaca9 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:12.925840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d8ea27-fc35-4bb5-8863-02a99f5eaca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:12.937099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d8ea27-fc35-4bb5-8863-02a99f5eaca9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:13.020764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:13.062267 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1731/3540] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:58:12 2022] POST /identity/v3/users => generated 290 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:13.068657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8ea27-fc35-4bb5-8863-02a99f5eaca9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:13.069668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28893342-0be8-4975-b045-96b490366929 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:13.094708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506726c-c76a-4e94-88f3-60bbf9eb5853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:13.095636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506726c-c76a-4e94-88f3-60bbf9eb5853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:13.111519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506726c-c76a-4e94-88f3-60bbf9eb5853 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:13.113677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506726c-c76a-4e94-88f3-60bbf9eb5853 admin admin] PATH_INFO: `/v3/projects/32ff9960a594470e8df6a8abff16ba62` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:13.120479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c506726c-c76a-4e94-88f3-60bbf9eb5853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32ff9960a594470e8df6a8abff16ba62)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:13.125677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c506726c-c76a-4e94-88f3-60bbf9eb5853 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:13.129251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8ea27-fc35-4bb5-8863-02a99f5eaca9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:13.148547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8882cfc-da9b-49c4-ab92-d9e675d73d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:13.150578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8882cfc-da9b-49c4-ab92-d9e675d73d20 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:13.152776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8882cfc-da9b-49c4-ab92-d9e675d73d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:13.153308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8882cfc-da9b-49c4-ab92-d9e675d73d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:13.154754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8882cfc-da9b-49c4-ab92-d9e675d73d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:13.159714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8882cfc-da9b-49c4-ab92-d9e675d73d20 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:13.235397 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1732/3541] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:13.239566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8882cfc-da9b-49c4-ab92-d9e675d73d20 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:13.297155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8882cfc-da9b-49c4-ab92-d9e675d73d20 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:13.300466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c506726c-c76a-4e94-88f3-60bbf9eb5853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32ff9960a594470e8df6a8abff16ba62', 'request_id': 'req-c506726c-c76a-4e94-88f3-60bbf9eb5853'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:13.309771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c506726c-c76a-4e94-88f3-60bbf9eb5853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32ff9960a594470e8df6a8abff16ba62', 'request_id': 'req-c506726c-c76a-4e94-88f3-60bbf9eb5853'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:13.319721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c506726c-c76a-4e94-88f3-60bbf9eb5853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32ff9960a594470e8df6a8abff16ba62', 'request_id': 'req-c506726c-c76a-4e94-88f3-60bbf9eb5853'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:13.339436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:13.376947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c506726c-c76a-4e94-88f3-60bbf9eb5853 admin admin] The token cache is being invalidate because project 32ff9960a594470e8df6a8abff16ba62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:13.377291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c506726c-c76a-4e94-88f3-60bbf9eb5853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:13.378144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c506726c-c76a-4e94-88f3-60bbf9eb5853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:13.385811 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1810/3542] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:12 2022] DELETE /identity/v3/projects/32ff9960a594470e8df6a8abff16ba62 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:13.391358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506726c-c76a-4e94-88f3-60bbf9eb5853 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:13.404773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c506726c-c76a-4e94-88f3-60bbf9eb5853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:13.621918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b8020-0662-42b2-b98d-a790c23107d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:13.622949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b8020-0662-42b2-b98d-a790c23107d6 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:13.626154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b8020-0662-42b2-b98d-a790c23107d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:13.629272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b8020-0662-42b2-b98d-a790c23107d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:13.634157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b8020-0662-42b2-b98d-a790c23107d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:13.642831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b8020-0662-42b2-b98d-a790c23107d6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:13.651964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506726c-c76a-4e94-88f3-60bbf9eb5853 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:13.670851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a89fd-8c08-4cec-a14a-6461982e5f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:13.671566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a89fd-8c08-4cec-a14a-6461982e5f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:13.672551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a89fd-8c08-4cec-a14a-6461982e5f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:13.672551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a89fd-8c08-4cec-a14a-6461982e5f3d admin admin] PATH_INFO: `/v3/users/73ca2a04a7d84902912cbce8ccd82525` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:13.676061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a89fd-8c08-4cec-a14a-6461982e5f3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=73ca2a04a7d84902912cbce8ccd82525)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:13.682552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a89fd-8c08-4cec-a14a-6461982e5f3d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:13.682725 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1733/3543] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:13.693941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b8020-0662-42b2-b98d-a790c23107d6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:13.701336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-638b8020-0662-42b2-b98d-a790c23107d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:13.767677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b8020-0662-42b2-b98d-a790c23107d6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:13.792614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a2951-9a86-4ba1-8663-1bd8adbd4133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:13.796892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a2951-9a86-4ba1-8663-1bd8adbd4133 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:13.798359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a2951-9a86-4ba1-8663-1bd8adbd4133 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:13.798704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a2951-9a86-4ba1-8663-1bd8adbd4133 admin admin] PATH_INFO: `/v3/projects/35315d2dc0e749a9910aa04a2ed06b42/users/1de7837875cf4298b7ac7f328a74694c/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:13.810934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-694a89fd-8c08-4cec-a14a-6461982e5f3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73ca2a04a7d84902912cbce8ccd82525'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:13.821650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-694a89fd-8c08-4cec-a14a-6461982e5f3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73ca2a04a7d84902912cbce8ccd82525'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:13.849775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-694a89fd-8c08-4cec-a14a-6461982e5f3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73ca2a04a7d84902912cbce8ccd82525'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:13.850688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-694a89fd-8c08-4cec-a14a-6461982e5f3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73ca2a04a7d84902912cbce8ccd82525'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:13.861147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-694a89fd-8c08-4cec-a14a-6461982e5f3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73ca2a04a7d84902912cbce8ccd82525'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:13.862289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a2951-9a86-4ba1-8663-1bd8adbd4133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35315d2dc0e749a9910aa04a2ed06b42, user_id=1de7837875cf4298b7ac7f328a74694c, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:13.867707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a2951-9a86-4ba1-8663-1bd8adbd4133 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:13.873223 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1811/3544] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:58:13 2022] DELETE /identity/v3/users/73ca2a04a7d84902912cbce8ccd82525 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:13.922003 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1734/3545] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:58:13 2022] PUT /identity/v3/projects/35315d2dc0e749a9910aa04a2ed06b42/users/1de7837875cf4298b7ac7f328a74694c/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:13.927531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff79aa-f5fd-412f-ba36-7a367c41ad4b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:13.927826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff79aa-f5fd-412f-ba36-7a367c41ad4b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:13.928175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff79aa-f5fd-412f-ba36-7a367c41ad4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:13.966292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:13.989688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5bff79aa-f5fd-412f-ba36-7a367c41ad4b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:13.990971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5bff79aa-f5fd-412f-ba36-7a367c41ad4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:14.209357 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1812/3546] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:14.226130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a2951-9a86-4ba1-8663-1bd8adbd4133 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:14.230558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d47a2951-9a86-4ba1-8663-1bd8adbd4133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:14.525815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a2951-9a86-4ba1-8663-1bd8adbd4133 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:14.544467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd87d8e-b5b1-463f-96ca-84944cd0731a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:14.545098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd87d8e-b5b1-463f-96ca-84944cd0731a service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:14.545378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd87d8e-b5b1-463f-96ca-84944cd0731a service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:14.545650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd87d8e-b5b1-463f-96ca-84944cd0731a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:14.590066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd87d8e-b5b1-463f-96ca-84944cd0731a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:14.591595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd87d8e-b5b1-463f-96ca-84944cd0731a service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:14.635378 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1735/3547] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:58:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:58:14.744455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff79aa-f5fd-412f-ba36-7a367c41ad4b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:14.746787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5bff79aa-f5fd-412f-ba36-7a367c41ad4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:14.780091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff79aa-f5fd-412f-ba36-7a367c41ad4b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:14.801585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a551094-47be-45c4-8197-23e6469f5ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:14.811786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a551094-47be-45c4-8197-23e6469f5ec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:14.812359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a551094-47be-45c4-8197-23e6469f5ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:14.814032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a551094-47be-45c4-8197-23e6469f5ec6 admin admin] PATH_INFO: `/v3/projects/9f2ac590b6be415da95f7932d3249798` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:14.817533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a551094-47be-45c4-8197-23e6469f5ec6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f2ac590b6be415da95f7932d3249798)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:14.823889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a551094-47be-45c4-8197-23e6469f5ec6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:14.985183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5a551094-47be-45c4-8197-23e6469f5ec6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f2ac590b6be415da95f7932d3249798', 'request_id': 'req-5a551094-47be-45c4-8197-23e6469f5ec6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:15.011892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5a551094-47be-45c4-8197-23e6469f5ec6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f2ac590b6be415da95f7932d3249798', 'request_id': 'req-5a551094-47be-45c4-8197-23e6469f5ec6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:15.036458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5a551094-47be-45c4-8197-23e6469f5ec6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f2ac590b6be415da95f7932d3249798', 'request_id': 'req-5a551094-47be-45c4-8197-23e6469f5ec6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:15.084033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5a551094-47be-45c4-8197-23e6469f5ec6 admin admin] The token cache is being invalidate because project 9f2ac590b6be415da95f7932d3249798 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:15.084033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5a551094-47be-45c4-8197-23e6469f5ec6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:15.086915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5a551094-47be-45c4-8197-23e6469f5ec6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:15.099808 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1813/3548] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:14 2022] DELETE /identity/v3/projects/9f2ac590b6be415da95f7932d3249798 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:15.101751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8294ce-97d4-4d3d-87e1-46880406919b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:15.103122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8294ce-97d4-4d3d-87e1-46880406919b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:15.103778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8294ce-97d4-4d3d-87e1-46880406919b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:15.175897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1b8294ce-97d4-4d3d-87e1-46880406919b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:15.180526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1b8294ce-97d4-4d3d-87e1-46880406919b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:15.391372 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1736/3549] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:15.399748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5030751-dd26-4c4e-8f10-667a53db1674 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:15.400068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5030751-dd26-4c4e-8f10-667a53db1674 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:15.400368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5030751-dd26-4c4e-8f10-667a53db1674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:15.436957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c5030751-dd26-4c4e-8f10-667a53db1674 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:15.438210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c5030751-dd26-4c4e-8f10-667a53db1674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:15.486075 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1814/3550] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:15.487413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8294ce-97d4-4d3d-87e1-46880406919b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:15.510304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8294ce-97d4-4d3d-87e1-46880406919b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:15.535521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d35ac-b491-41b2-85ac-43632607c297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:15.545112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d35ac-b491-41b2-85ac-43632607c297 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:15.545731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d35ac-b491-41b2-85ac-43632607c297 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:15.546409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d35ac-b491-41b2-85ac-43632607c297 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:15.550129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d35ac-b491-41b2-85ac-43632607c297 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:58:15.550445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d35ac-b491-41b2-85ac-43632607c297 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:15.552123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d35ac-b491-41b2-85ac-43632607c297 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:15.631912 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1737/3551] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:15.632925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5030751-dd26-4c4e-8f10-667a53db1674 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:15.648074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5030751-dd26-4c4e-8f10-667a53db1674 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:15.676607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc82245-6631-45c0-8768-bb8d10d21cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:15.688528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc82245-6631-45c0-8768-bb8d10d21cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:15.690231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc82245-6631-45c0-8768-bb8d10d21cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:15.691662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc82245-6631-45c0-8768-bb8d10d21cc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:15.693941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc82245-6631-45c0-8768-bb8d10d21cc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:15.702236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc82245-6631-45c0-8768-bb8d10d21cc5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:15.823389 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1815/3552] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:58:15 2022] POST /identity/v3/projects => generated 367 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:58:15.827511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d35ac-b491-41b2-85ac-43632607c297 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:15.846648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d35ac-b491-41b2-85ac-43632607c297 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:15.886793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:16.197594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa69466-1557-41b3-9848-7d71bfdab088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:16.198227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa69466-1557-41b3-9848-7d71bfdab088 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:16.198527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa69466-1557-41b3-9848-7d71bfdab088 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:16.198796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa69466-1557-41b3-9848-7d71bfdab088 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:16.215540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa69466-1557-41b3-9848-7d71bfdab088 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:16.220592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa69466-1557-41b3-9848-7d71bfdab088 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:16.386292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc82245-6631-45c0-8768-bb8d10d21cc5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:16.398643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc82245-6631-45c0-8768-bb8d10d21cc5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:16.402939 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1738/3553] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:58:15 2022] POST /identity/v3/users => generated 284 bytes in 576 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:16.428537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb41c9d-9469-4515-b28f-3fbd853d6724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:16.433720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb41c9d-9469-4515-b28f-3fbd853d6724 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:16.434119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb41c9d-9469-4515-b28f-3fbd853d6724 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:16.434776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb41c9d-9469-4515-b28f-3fbd853d6724 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:16.437827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb41c9d-9469-4515-b28f-3fbd853d6724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:16.445757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb41c9d-9469-4515-b28f-3fbd853d6724 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:16.516860 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1816/3554] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:58:16.521693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa69466-1557-41b3-9848-7d71bfdab088 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:16.538754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa69466-1557-41b3-9848-7d71bfdab088 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:16.568371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a24ef-a054-4579-849d-327e69726060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:16.569405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a24ef-a054-4579-849d-327e69726060 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:16.570014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a24ef-a054-4579-849d-327e69726060 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:16.570637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a24ef-a054-4579-849d-327e69726060 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:16.571541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a24ef-a054-4579-849d-327e69726060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:16.573864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a24ef-a054-4579-849d-327e69726060 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:16.650405 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1739/3555] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:16.655334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb41c9d-9469-4515-b28f-3fbd853d6724 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:16.670616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb41c9d-9469-4515-b28f-3fbd853d6724 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:16.706034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f7999-f2f7-4ff8-b85d-e4a3837f0018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:16.711468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f7999-f2f7-4ff8-b85d-e4a3837f0018 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:16.712044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f7999-f2f7-4ff8-b85d-e4a3837f0018 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:16.713267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f7999-f2f7-4ff8-b85d-e4a3837f0018 admin admin] PATH_INFO: `/v3/projects/2902e6fc58c647109835a584e7b17c82/users/c25b1a58d5c347d5b153df51a2ab2658/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:16.775127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f7999-f2f7-4ff8-b85d-e4a3837f0018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2902e6fc58c647109835a584e7b17c82, user_id=c25b1a58d5c347d5b153df51a2ab2658, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:16.785039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f7999-f2f7-4ff8-b85d-e4a3837f0018 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:16.822533 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1817/3556] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:58:16 2022] PUT /identity/v3/projects/2902e6fc58c647109835a584e7b17c82/users/c25b1a58d5c347d5b153df51a2ab2658/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:16.829900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ba5d4-537f-4f97-8cb9-32378d2c3b44 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:16.830942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ba5d4-537f-4f97-8cb9-32378d2c3b44 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:16.831326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ba5d4-537f-4f97-8cb9-32378d2c3b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:16.916639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e19ba5d4-537f-4f97-8cb9-32378d2c3b44 None None] MFA Rules not processed for user `c25b1a58d5c347d5b153df51a2ab2658`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:16.918011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e19ba5d4-537f-4f97-8cb9-32378d2c3b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:17.169880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f7999-f2f7-4ff8-b85d-e4a3837f0018 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:17.187667 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1740/3557] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:17.267488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:17.285991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1716b4-2fc6-4ee7-b8ad-16e0f515cbcf None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:17.286451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1716b4-2fc6-4ee7-b8ad-16e0f515cbcf None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:17.287036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1716b4-2fc6-4ee7-b8ad-16e0f515cbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:17.393627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-dc1716b4-2fc6-4ee7-b8ad-16e0f515cbcf None None] MFA Rules not processed for user `f407ee9fccbf4fd6a9286642f8a015ad`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:17.395028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-dc1716b4-2fc6-4ee7-b8ad-16e0f515cbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:17.404955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f7999-f2f7-4ff8-b85d-e4a3837f0018 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:17.421662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0a677-f9de-4e48-a838-5ed78be2c822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:17.422475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0a677-f9de-4e48-a838-5ed78be2c822 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:17.422797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0a677-f9de-4e48-a838-5ed78be2c822 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:17.423067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0a677-f9de-4e48-a838-5ed78be2c822 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:17.423711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0a677-f9de-4e48-a838-5ed78be2c822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:17.425670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0a677-f9de-4e48-a838-5ed78be2c822 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:17.517017 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1818/3558] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:58:17 2022] POST /identity/v3/projects => generated 365 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:17.522408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0a677-f9de-4e48-a838-5ed78be2c822 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:17.713564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0a677-f9de-4e48-a838-5ed78be2c822 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:17.728622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc80d6b-7195-4f19-a375-ac932e307901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:17.729254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc80d6b-7195-4f19-a375-ac932e307901 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:17.729533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc80d6b-7195-4f19-a375-ac932e307901 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:17.729808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc80d6b-7195-4f19-a375-ac932e307901 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:17.736126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc80d6b-7195-4f19-a375-ac932e307901 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:17.738212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc80d6b-7195-4f19-a375-ac932e307901 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:17.787327 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1741/3559] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 499 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:58:17.787919 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1819/3560] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:58:17 2022] POST /identity/v3/users => generated 289 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:17.795059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1716b4-2fc6-4ee7-b8ad-16e0f515cbcf None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:17.801923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b9982-1894-4f0f-9499-dfc58dd857ce None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:17.802561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b9982-1894-4f0f-9499-dfc58dd857ce None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:17.803796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b9982-1894-4f0f-9499-dfc58dd857ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:17.891592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1716b4-2fc6-4ee7-b8ad-16e0f515cbcf None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:17.907374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-df1b9982-1894-4f0f-9499-dfc58dd857ce None None] MFA Rules not processed for user `f407ee9fccbf4fd6a9286642f8a015ad`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:17.913695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-df1b9982-1894-4f0f-9499-dfc58dd857ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:17.917625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5048b-03af-4721-bf4c-381d864f5921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:17.918268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5048b-03af-4721-bf4c-381d864f5921 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:17.918577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5048b-03af-4721-bf4c-381d864f5921 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:17.918856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5048b-03af-4721-bf4c-381d864f5921 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:17.919508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5048b-03af-4721-bf4c-381d864f5921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:17.935430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5048b-03af-4721-bf4c-381d864f5921 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:17.963924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:18.024372 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1742/3561] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:18.035562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5048b-03af-4721-bf4c-381d864f5921 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:18.111521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5048b-03af-4721-bf4c-381d864f5921 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:18.149793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788c606-05a0-4cff-9183-2a26f8762a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:18.150407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788c606-05a0-4cff-9183-2a26f8762a12 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:18.150719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788c606-05a0-4cff-9183-2a26f8762a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:18.150998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788c606-05a0-4cff-9183-2a26f8762a12 admin admin] PATH_INFO: `/v3/projects/f810821a8de44447b66edec5c7b79a46/users/8c44433147e445008533efde0f04a41f/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:18.255055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788c606-05a0-4cff-9183-2a26f8762a12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f810821a8de44447b66edec5c7b79a46, user_id=8c44433147e445008533efde0f04a41f, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:18.257759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788c606-05a0-4cff-9183-2a26f8762a12 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:18.261365 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1820/3562] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:58:18.338872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b9982-1894-4f0f-9499-dfc58dd857ce None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:18.342801 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1743/3563] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:58:18 2022] PUT /identity/v3/projects/f810821a8de44447b66edec5c7b79a46/users/8c44433147e445008533efde0f04a41f/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:18.362158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b9982-1894-4f0f-9499-dfc58dd857ce None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:18.594509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:18.661051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93abc67-cfbf-48d3-8ffe-3769be74cff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:18.661731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93abc67-cfbf-48d3-8ffe-3769be74cff8 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:18.662022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93abc67-cfbf-48d3-8ffe-3769be74cff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:18.662291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93abc67-cfbf-48d3-8ffe-3769be74cff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:18.662975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93abc67-cfbf-48d3-8ffe-3769be74cff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:18.670114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93abc67-cfbf-48d3-8ffe-3769be74cff8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:18.710032 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1821/3564] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:18.714736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788c606-05a0-4cff-9183-2a26f8762a12 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:18.736712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788c606-05a0-4cff-9183-2a26f8762a12 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:18.758614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ccf87-972c-4bf5-909a-15e751164061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:18.759299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ccf87-972c-4bf5-909a-15e751164061 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:18.759577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ccf87-972c-4bf5-909a-15e751164061 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:18.759850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ccf87-972c-4bf5-909a-15e751164061 admin admin] PATH_INFO: `/v3/projects/f810821a8de44447b66edec5c7b79a46/users/8c44433147e445008533efde0f04a41f/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:18.771573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ccf87-972c-4bf5-909a-15e751164061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f810821a8de44447b66edec5c7b79a46, user_id=8c44433147e445008533efde0f04a41f, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:18.805510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ccf87-972c-4bf5-909a-15e751164061 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:18.857209 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1744/3565] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:58:18 2022] PUT /identity/v3/projects/f810821a8de44447b66edec5c7b79a46/users/8c44433147e445008533efde0f04a41f/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:18.873049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ec91c-056f-4932-a9e1-eb3eff00471b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:18.873921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ec91c-056f-4932-a9e1-eb3eff00471b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:18.873921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ec91c-056f-4932-a9e1-eb3eff00471b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:18.980894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-185ec91c-056f-4932-a9e1-eb3eff00471b None None] MFA Rules not processed for user `8c44433147e445008533efde0f04a41f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:18.982152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-185ec91c-056f-4932-a9e1-eb3eff00471b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:19.301114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ccf87-972c-4bf5-909a-15e751164061 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:19.301387 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1822/3566] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:19.317627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ccf87-972c-4bf5-909a-15e751164061 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:19.365393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61e42f-4aa3-468c-9e54-98ae87390b02 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c44433147e445008533efde0f04a41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f810821a8de44447b66edec5c7b79a46', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:19.366060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61e42f-4aa3-468c-9e54-98ae87390b02 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:19.366347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61e42f-4aa3-468c-9e54-98ae87390b02 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:19.366667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61e42f-4aa3-468c-9e54-98ae87390b02 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] PATH_INFO: `/v3/policies` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:19.367805 ncdv-master-1 devstack@keystone.service[99140]: WARNING oslo_log.versionutils [None req-6e61e42f-4aa3-468c-9e54-98ae87390b02 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Deprecated: identity:create_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Oct 01 20:58:19.368306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61e42f-4aa3-468c-9e54-98ae87390b02 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:19.369548 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-6e61e42f-4aa3-468c-9e54-98ae87390b02 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:58:19.369548 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:58:19.369548 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:58:19.370470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61e42f-4aa3-468c-9e54-98ae87390b02 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:19.420536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ec91c-056f-4932-a9e1-eb3eff00471b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:19.422158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-185ec91c-056f-4932-a9e1-eb3eff00471b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:19.479257 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1745/3567] 10.222.0.17 () {60 vars in 1152 bytes} [Sat Oct 1 20:58:19 2022] POST /identity/v3/policies => generated 230 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:19.507584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61e42f-4aa3-468c-9e54-98ae87390b02 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:19.570518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61e42f-4aa3-468c-9e54-98ae87390b02 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:19.606225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110827f-30ef-482b-8b75-cf76ef33d49f tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c44433147e445008533efde0f04a41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f810821a8de44447b66edec5c7b79a46', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:19.606708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110827f-30ef-482b-8b75-cf76ef33d49f tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:19.606995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110827f-30ef-482b-8b75-cf76ef33d49f tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:19.607321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110827f-30ef-482b-8b75-cf76ef33d49f tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] PATH_INFO: `/v3/policies/870003a49ae3426282d663fa39be8319` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:19.621220 ncdv-master-1 devstack@keystone.service[99140]: WARNING oslo_log.versionutils [None req-8110827f-30ef-482b-8b75-cf76ef33d49f tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Deprecated: identity:update_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Oct 01 20:58:19.621750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110827f-30ef-482b-8b75-cf76ef33d49f tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorizing `identity:update_policy(policy_id=870003a49ae3426282d663fa39be8319)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:19.623066 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-8110827f-30ef-482b-8b75-cf76ef33d49f tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:58:19.623066 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:58:19.623066 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:58:19.623978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110827f-30ef-482b-8b75-cf76ef33d49f tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:19.773381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ec91c-056f-4932-a9e1-eb3eff00471b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:19.794599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f8bee-98ca-4673-8fc0-e164dfc40581 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:19.796103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f8bee-98ca-4673-8fc0-e164dfc40581 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:19.799818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f8bee-98ca-4673-8fc0-e164dfc40581 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:19.800218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f8bee-98ca-4673-8fc0-e164dfc40581 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:19.889357 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1746/3568] 10.222.0.17 () {60 vars in 1252 bytes} [Sat Oct 1 20:58:19 2022] PATCH /identity/v3/policies/870003a49ae3426282d663fa39be8319 => generated 236 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:19.891882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110827f-30ef-482b-8b75-cf76ef33d49f tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:19.950372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110827f-30ef-482b-8b75-cf76ef33d49f tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:19.987516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe705a-e082-430f-9857-52fdbc6bb5ba tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c44433147e445008533efde0f04a41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f810821a8de44447b66edec5c7b79a46', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:19.995339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe705a-e082-430f-9857-52fdbc6bb5ba tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:19.998199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe705a-e082-430f-9857-52fdbc6bb5ba tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:19.999180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe705a-e082-430f-9857-52fdbc6bb5ba tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] PATH_INFO: `/v3/policies/870003a49ae3426282d663fa39be8319` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:20.000155 ncdv-master-1 devstack@keystone.service[99140]: WARNING oslo_log.versionutils [None req-6ffe705a-e082-430f-9857-52fdbc6bb5ba tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Deprecated: identity:get_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Oct 01 20:58:20.003196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe705a-e082-430f-9857-52fdbc6bb5ba tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorizing `identity:get_policy(policy_id=870003a49ae3426282d663fa39be8319)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:20.007353 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-6ffe705a-e082-430f-9857-52fdbc6bb5ba tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_policy": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:58:20.007353 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:58:20.007353 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:58:20.022555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe705a-e082-430f-9857-52fdbc6bb5ba tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:20.079240 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1747/3569] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 20:58:19 2022] GET /identity/v3/policies/870003a49ae3426282d663fa39be8319 => generated 236 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:58:20.081234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe705a-e082-430f-9857-52fdbc6bb5ba tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:20.136345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe705a-e082-430f-9857-52fdbc6bb5ba tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:20.145494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:20.149377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f8bee-98ca-4673-8fc0-e164dfc40581 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:20.155692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48996013-8117-452a-bb7d-aa01f9d59d63 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c44433147e445008533efde0f04a41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f810821a8de44447b66edec5c7b79a46', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:20.159310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48996013-8117-452a-bb7d-aa01f9d59d63 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:20.159903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48996013-8117-452a-bb7d-aa01f9d59d63 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:20.163684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48996013-8117-452a-bb7d-aa01f9d59d63 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] PATH_INFO: `/v3/policies/870003a49ae3426282d663fa39be8319` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:20.166536 ncdv-master-1 devstack@keystone.service[99140]: WARNING oslo_log.versionutils [None req-48996013-8117-452a-bb7d-aa01f9d59d63 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Deprecated: identity:delete_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Oct 01 20:58:20.166845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f8bee-98ca-4673-8fc0-e164dfc40581 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:20.167839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48996013-8117-452a-bb7d-aa01f9d59d63 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=870003a49ae3426282d663fa39be8319)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:20.179980 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-48996013-8117-452a-bb7d-aa01f9d59d63 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:58:20.179980 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:58:20.179980 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:58:20.184441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48996013-8117-452a-bb7d-aa01f9d59d63 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:20.222290 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1823/3570] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:58:19 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 797 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:58:20.271601 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1748/3571] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:20 2022] DELETE /identity/v3/policies/870003a49ae3426282d663fa39be8319 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:20.272701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f8bee-98ca-4673-8fc0-e164dfc40581 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:20.382428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f8bee-98ca-4673-8fc0-e164dfc40581 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:20.437193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23451895-0246-46cd-a8e4-78187db36c0b tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c44433147e445008533efde0f04a41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f810821a8de44447b66edec5c7b79a46', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:20.446547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23451895-0246-46cd-a8e4-78187db36c0b tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:20.447902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23451895-0246-46cd-a8e4-78187db36c0b tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:20.451750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23451895-0246-46cd-a8e4-78187db36c0b tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] PATH_INFO: `/v3/policies` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:20.453025 ncdv-master-1 devstack@keystone.service[99139]: WARNING oslo_log.versionutils [None req-23451895-0246-46cd-a8e4-78187db36c0b tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Deprecated: identity:create_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Oct 01 20:58:20.454186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23451895-0246-46cd-a8e4-78187db36c0b tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:20.456989 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-23451895-0246-46cd-a8e4-78187db36c0b tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:58:20.456989 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 20:58:20.456989 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:58:20.462897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23451895-0246-46cd-a8e4-78187db36c0b tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:20.524263 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1824/3572] 10.222.0.17 () {60 vars in 1152 bytes} [Sat Oct 1 20:58:20 2022] POST /identity/v3/policies => generated 228 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:58:20.528330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48996013-8117-452a-bb7d-aa01f9d59d63 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:20.565921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48996013-8117-452a-bb7d-aa01f9d59d63 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:20.589765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b8e2d5-93b1-4199-88c7-b3c985883abc tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c44433147e445008533efde0f04a41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f810821a8de44447b66edec5c7b79a46', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:20.590987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8e2d5-93b1-4199-88c7-b3c985883abc tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:20.591580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8e2d5-93b1-4199-88c7-b3c985883abc tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:20.592008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8e2d5-93b1-4199-88c7-b3c985883abc tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] PATH_INFO: `/v3/policies` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:20.593359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b8e2d5-93b1-4199-88c7-b3c985883abc tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:20.596395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b8e2d5-93b1-4199-88c7-b3c985883abc tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:20.646746 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1749/3573] 10.222.0.17 () {60 vars in 1152 bytes} [Sat Oct 1 20:58:20 2022] POST /identity/v3/policies => generated 230 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:20.651593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23451895-0246-46cd-a8e4-78187db36c0b tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:20.678787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23451895-0246-46cd-a8e4-78187db36c0b tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:20.690093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac29f7-e9ed-4f44-9eb5-691da183601e tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c44433147e445008533efde0f04a41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f810821a8de44447b66edec5c7b79a46', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:20.690973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac29f7-e9ed-4f44-9eb5-691da183601e tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:20.691605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac29f7-e9ed-4f44-9eb5-691da183601e tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:20.693143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac29f7-e9ed-4f44-9eb5-691da183601e tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] PATH_INFO: `/v3/policies` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:20.700271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac29f7-e9ed-4f44-9eb5-691da183601e tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:20.715385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac29f7-e9ed-4f44-9eb5-691da183601e tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:20.766370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b8e2d5-93b1-4199-88c7-b3c985883abc tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:20.780894 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1825/3574] 10.222.0.17 () {60 vars in 1152 bytes} [Sat Oct 1 20:58:20 2022] POST /identity/v3/policies => generated 230 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:20.790752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b8e2d5-93b1-4199-88c7-b3c985883abc tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:20.815099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2e2f8-5275-49ca-89fe-4b072bd0c029 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c44433147e445008533efde0f04a41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f810821a8de44447b66edec5c7b79a46', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:20.817252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2e2f8-5275-49ca-89fe-4b072bd0c029 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:20.818783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2e2f8-5275-49ca-89fe-4b072bd0c029 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:20.819309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2e2f8-5275-49ca-89fe-4b072bd0c029 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] PATH_INFO: `/v3/policies` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:20.821580 ncdv-master-1 devstack@keystone.service[99140]: WARNING oslo_log.versionutils [None req-f4e2e2f8-5275-49ca-89fe-4b072bd0c029 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Deprecated: identity:list_policies of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Oct 01 20:58:20.823429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2e2f8-5275-49ca-89fe-4b072bd0c029 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorizing `identity:list_policies()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:20.826739 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-f4e2e2f8-5275-49ca-89fe-4b072bd0c029 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_policies": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:58:20.826739 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:58:20.826739 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:58:20.829207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2e2f8-5275-49ca-89fe-4b072bd0c029 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:20.874003 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1750/3575] 10.222.0.17 () {58 vars in 1131 bytes} [Sat Oct 1 20:58:20 2022] GET /identity/v3/policies => generated 764 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:20.878803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac29f7-e9ed-4f44-9eb5-691da183601e tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:20.907571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac29f7-e9ed-4f44-9eb5-691da183601e tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:20.932946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4c27e-d745-4997-b26a-b52b8b4892a3 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c44433147e445008533efde0f04a41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f810821a8de44447b66edec5c7b79a46', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:20.933620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4c27e-d745-4997-b26a-b52b8b4892a3 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:20.933905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4c27e-d745-4997-b26a-b52b8b4892a3 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:20.934183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4c27e-d745-4997-b26a-b52b8b4892a3 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] PATH_INFO: `/v3/policies/a7c7dc4549df4cc5afd168ac4bd0e9f6` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:20.935268 ncdv-master-1 devstack@keystone.service[99139]: WARNING oslo_log.versionutils [None req-a4e4c27e-d745-4997-b26a-b52b8b4892a3 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Deprecated: identity:delete_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Oct 01 20:58:20.935837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4c27e-d745-4997-b26a-b52b8b4892a3 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=a7c7dc4549df4cc5afd168ac4bd0e9f6)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:20.953041 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-a4e4c27e-d745-4997-b26a-b52b8b4892a3 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:58:20.953041 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 20:58:20.953041 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:58:20.953041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4c27e-d745-4997-b26a-b52b8b4892a3 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:21.001685 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1826/3576] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:20 2022] DELETE /identity/v3/policies/a7c7dc4549df4cc5afd168ac4bd0e9f6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:21.006029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2e2f8-5275-49ca-89fe-4b072bd0c029 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:21.023232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2e2f8-5275-49ca-89fe-4b072bd0c029 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:21.099066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea6155-ac5a-4a77-8f01-e021873cefd0 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c44433147e445008533efde0f04a41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f810821a8de44447b66edec5c7b79a46', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:21.104538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea6155-ac5a-4a77-8f01-e021873cefd0 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:21.104843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea6155-ac5a-4a77-8f01-e021873cefd0 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:21.105891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea6155-ac5a-4a77-8f01-e021873cefd0 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] PATH_INFO: `/v3/policies/f2da134705b84bcbb916544c682c3b37` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:21.111046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea6155-ac5a-4a77-8f01-e021873cefd0 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=f2da134705b84bcbb916544c682c3b37)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:21.115363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea6155-ac5a-4a77-8f01-e021873cefd0 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:21.194532 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1751/3577] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:21 2022] DELETE /identity/v3/policies/f2da134705b84bcbb916544c682c3b37 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:58:21.196435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4c27e-d745-4997-b26a-b52b8b4892a3 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:21.281345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4c27e-d745-4997-b26a-b52b8b4892a3 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:21.314970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f7250-dceb-4da6-affe-6b48809a84c1 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c44433147e445008533efde0f04a41f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f810821a8de44447b66edec5c7b79a46', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:21.314970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f7250-dceb-4da6-affe-6b48809a84c1 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:21.314970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f7250-dceb-4da6-affe-6b48809a84c1 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:21.315377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f7250-dceb-4da6-affe-6b48809a84c1 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] PATH_INFO: `/v3/policies/799313824104474b82dc3b1108ab1af5` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:21.329962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f7250-dceb-4da6-affe-6b48809a84c1 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=799313824104474b82dc3b1108ab1af5)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:21.337986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f7250-dceb-4da6-affe-6b48809a84c1 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:21.397026 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1827/3578] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:21 2022] DELETE /identity/v3/policies/799313824104474b82dc3b1108ab1af5 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:21.401865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea6155-ac5a-4a77-8f01-e021873cefd0 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:21.627765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea6155-ac5a-4a77-8f01-e021873cefd0 tempest-PoliciesTestJSON-24802574 tempest-PoliciesTestJSON-24802574-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:21.650270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfa2b4-c420-41f4-8469-257d9c920c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:21.652220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfa2b4-c420-41f4-8469-257d9c920c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:21.652220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfa2b4-c420-41f4-8469-257d9c920c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:21.652220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfa2b4-c420-41f4-8469-257d9c920c26 admin admin] PATH_INFO: `/v3/users/c25b1a58d5c347d5b153df51a2ab2658` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:21.655020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfa2b4-c420-41f4-8469-257d9c920c26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c25b1a58d5c347d5b153df51a2ab2658)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:21.660648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfa2b4-c420-41f4-8469-257d9c920c26 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:21.747550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:UPDATE=1,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:21.788865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf83b8-a505-40c5-b684-70777ad94e92 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:21.789333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf83b8-a505-40c5-b684-70777ad94e92 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:21.789760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf83b8-a505-40c5-b684-70777ad94e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:21.799025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f5cfa2b4-c420-41f4-8469-257d9c920c26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c25b1a58d5c347d5b153df51a2ab2658'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:21.800163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f5cfa2b4-c420-41f4-8469-257d9c920c26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c25b1a58d5c347d5b153df51a2ab2658'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:21.801914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f5cfa2b4-c420-41f4-8469-257d9c920c26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c25b1a58d5c347d5b153df51a2ab2658'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:21.823135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f5cfa2b4-c420-41f4-8469-257d9c920c26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c25b1a58d5c347d5b153df51a2ab2658'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:21.835192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-abaf83b8-a505-40c5-b684-70777ad94e92 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:21.837153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-abaf83b8-a505-40c5-b684-70777ad94e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:21.850597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f5cfa2b4-c420-41f4-8469-257d9c920c26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c25b1a58d5c347d5b153df51a2ab2658'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:21.879912 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1752/3579] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:58:21 2022] DELETE /identity/v3/users/c25b1a58d5c347d5b153df51a2ab2658 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:21.886456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbac345-fb4b-40e1-8ee8-99dcb175e4ad None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:21.887003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbac345-fb4b-40e1-8ee8-99dcb175e4ad None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:21.887454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbac345-fb4b-40e1-8ee8-99dcb175e4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:21.931718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4dbac345-fb4b-40e1-8ee8-99dcb175e4ad None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:21.933922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4dbac345-fb4b-40e1-8ee8-99dcb175e4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:21.979406 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1828/3580] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:58:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:22.047275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf83b8-a505-40c5-b684-70777ad94e92 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:22.049995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-abaf83b8-a505-40c5-b684-70777ad94e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:22.129621 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1753/3581] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:58:22.151404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbac345-fb4b-40e1-8ee8-99dcb175e4ad None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:22.153590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4dbac345-fb4b-40e1-8ee8-99dcb175e4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:22.196402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf83b8-a505-40c5-b684-70777ad94e92 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:22.286314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbac345-fb4b-40e1-8ee8-99dcb175e4ad None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:22.304310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516043d-4127-46a0-a483-143d662e2119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:22.306282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516043d-4127-46a0-a483-143d662e2119 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:22.307054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516043d-4127-46a0-a483-143d662e2119 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:22.307566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516043d-4127-46a0-a483-143d662e2119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:22.313033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:22.344796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516043d-4127-46a0-a483-143d662e2119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:22.348871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516043d-4127-46a0-a483-143d662e2119 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:22.357105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6112c32-62c8-4749-bdb6-537f8a41e4ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:22.358238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6112c32-62c8-4749-bdb6-537f8a41e4ab service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:22.358837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6112c32-62c8-4749-bdb6-537f8a41e4ab service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:22.359266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6112c32-62c8-4749-bdb6-537f8a41e4ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:22.404109 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1754/3582] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:58:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:58:22.472333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6112c32-62c8-4749-bdb6-537f8a41e4ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:22.474116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6112c32-62c8-4749-bdb6-537f8a41e4ab service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:22.502794 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1829/3583] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:58:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 20:58:24.254854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516043d-4127-46a0-a483-143d662e2119 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:24.260000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1516043d-4127-46a0-a483-143d662e2119 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:24.328279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516043d-4127-46a0-a483-143d662e2119 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:24.343525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32c103-2f8c-49c2-8d87-2b3f4ef82235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:24.344350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32c103-2f8c-49c2-8d87-2b3f4ef82235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:24.344350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32c103-2f8c-49c2-8d87-2b3f4ef82235 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:24.344520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32c103-2f8c-49c2-8d87-2b3f4ef82235 admin admin] PATH_INFO: `/v3/projects/2902e6fc58c647109835a584e7b17c82` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:24.346846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32c103-2f8c-49c2-8d87-2b3f4ef82235 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2902e6fc58c647109835a584e7b17c82)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:24.348812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32c103-2f8c-49c2-8d87-2b3f4ef82235 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:24.609629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:24.615718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7e32c103-2f8c-49c2-8d87-2b3f4ef82235 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2902e6fc58c647109835a584e7b17c82', 'request_id': 'req-7e32c103-2f8c-49c2-8d87-2b3f4ef82235'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:24.671929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7e32c103-2f8c-49c2-8d87-2b3f4ef82235 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2902e6fc58c647109835a584e7b17c82', 'request_id': 'req-7e32c103-2f8c-49c2-8d87-2b3f4ef82235'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:24.696397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7e32c103-2f8c-49c2-8d87-2b3f4ef82235 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2902e6fc58c647109835a584e7b17c82', 'request_id': 'req-7e32c103-2f8c-49c2-8d87-2b3f4ef82235'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:24.759682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7e32c103-2f8c-49c2-8d87-2b3f4ef82235 admin admin] The token cache is being invalidate because project 2902e6fc58c647109835a584e7b17c82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:24.759969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7e32c103-2f8c-49c2-8d87-2b3f4ef82235 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:24.781081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7e32c103-2f8c-49c2-8d87-2b3f4ef82235 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:24.785336 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1755/3584] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:24 2022] DELETE /identity/v3/projects/2902e6fc58c647109835a584e7b17c82 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:24.789148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6112c32-62c8-4749-bdb6-537f8a41e4ab service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:24.805775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6112c32-62c8-4749-bdb6-537f8a41e4ab service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:25.138587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec2c9d-b766-4962-83e7-9396d4b91e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:25.139555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec2c9d-b766-4962-83e7-9396d4b91e89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:25.142580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec2c9d-b766-4962-83e7-9396d4b91e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:25.143982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec2c9d-b766-4962-83e7-9396d4b91e89 admin admin] PATH_INFO: `/v3/users/8c44433147e445008533efde0f04a41f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:25.154425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec2c9d-b766-4962-83e7-9396d4b91e89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c44433147e445008533efde0f04a41f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:25.169827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec2c9d-b766-4962-83e7-9396d4b91e89 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:25.403960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6fec2c9d-b766-4962-83e7-9396d4b91e89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c44433147e445008533efde0f04a41f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:25.422381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6fec2c9d-b766-4962-83e7-9396d4b91e89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c44433147e445008533efde0f04a41f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:25.454583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6fec2c9d-b766-4962-83e7-9396d4b91e89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c44433147e445008533efde0f04a41f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:25.456953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6fec2c9d-b766-4962-83e7-9396d4b91e89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c44433147e445008533efde0f04a41f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:25.476726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6fec2c9d-b766-4962-83e7-9396d4b91e89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c44433147e445008533efde0f04a41f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:25.490396 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1830/3585] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:58:24 2022] DELETE /identity/v3/users/8c44433147e445008533efde0f04a41f => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:30.068032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c907b3-17e8-49d7-9726-eee3b7103a4f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:30.068509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c907b3-17e8-49d7-9726-eee3b7103a4f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:30.068823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c907b3-17e8-49d7-9726-eee3b7103a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:30.150738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-79c907b3-17e8-49d7-9726-eee3b7103a4f None None] MFA Rules not processed for user `1de7837875cf4298b7ac7f328a74694c`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:30.152592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-79c907b3-17e8-49d7-9726-eee3b7103a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:30.512830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e109a-42a1-48c3-b265-4ea6af9b1697 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:30.513413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e109a-42a1-48c3-b265-4ea6af9b1697 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:30.513413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e109a-42a1-48c3-b265-4ea6af9b1697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:30.522598 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1756/3586] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:30.532051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:30.540356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-885e109a-42a1-48c3-b265-4ea6af9b1697 None None] MFA Rules not processed for user `1de7837875cf4298b7ac7f328a74694c`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:30.542176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-885e109a-42a1-48c3-b265-4ea6af9b1697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:30.556685 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1831/3587] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:31.369206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c907b3-17e8-49d7-9726-eee3b7103a4f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:31.370925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-79c907b3-17e8-49d7-9726-eee3b7103a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:31.598936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c907b3-17e8-49d7-9726-eee3b7103a4f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:31.612948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2f796-2ed0-4330-b818-93cca4d4f682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:31.620276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2f796-2ed0-4330-b818-93cca4d4f682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:31.620934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2f796-2ed0-4330-b818-93cca4d4f682 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:31.621591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2f796-2ed0-4330-b818-93cca4d4f682 admin admin] PATH_INFO: `/v3/projects/f810821a8de44447b66edec5c7b79a46` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:31.623338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2f796-2ed0-4330-b818-93cca4d4f682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f810821a8de44447b66edec5c7b79a46)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:31.626364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2f796-2ed0-4330-b818-93cca4d4f682 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:31.775497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34c2f796-2ed0-4330-b818-93cca4d4f682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f810821a8de44447b66edec5c7b79a46', 'request_id': 'req-34c2f796-2ed0-4330-b818-93cca4d4f682'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:31.795239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34c2f796-2ed0-4330-b818-93cca4d4f682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f810821a8de44447b66edec5c7b79a46', 'request_id': 'req-34c2f796-2ed0-4330-b818-93cca4d4f682'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:31.803471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34c2f796-2ed0-4330-b818-93cca4d4f682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f810821a8de44447b66edec5c7b79a46', 'request_id': 'req-34c2f796-2ed0-4330-b818-93cca4d4f682'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:31.810624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:31.829210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34c2f796-2ed0-4330-b818-93cca4d4f682 admin admin] The token cache is being invalidate because project f810821a8de44447b66edec5c7b79a46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:31.831260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34c2f796-2ed0-4330-b818-93cca4d4f682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:31.835914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34c2f796-2ed0-4330-b818-93cca4d4f682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:31.839556 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1757/3588] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:31 2022] DELETE /identity/v3/projects/f810821a8de44447b66edec5c7b79a46 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:31.844083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb8078-9beb-4b9d-b18e-c1267039006b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:31.844954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb8078-9beb-4b9d-b18e-c1267039006b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:31.846030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb8078-9beb-4b9d-b18e-c1267039006b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:31.881290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-07bb8078-9beb-4b9d-b18e-c1267039006b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:31.884115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-07bb8078-9beb-4b9d-b18e-c1267039006b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:31.950256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2f796-2ed0-4330-b818-93cca4d4f682 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:31.953403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-34c2f796-2ed0-4330-b818-93cca4d4f682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:32.263044 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1832/3589] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:58:32.263044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323b8c7-981f-4c1b-b48a-3e37f86daed4 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:32.263044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323b8c7-981f-4c1b-b48a-3e37f86daed4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:32.263044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323b8c7-981f-4c1b-b48a-3e37f86daed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:32.285438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e323b8c7-981f-4c1b-b48a-3e37f86daed4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:32.286908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e323b8c7-981f-4c1b-b48a-3e37f86daed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:32.309562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2f796-2ed0-4330-b818-93cca4d4f682 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:32.319681 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1833/3590] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:32.321879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323b8c7-981f-4c1b-b48a-3e37f86daed4 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:32.349219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ca425-d846-4021-8ff3-ed565b516975 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:32.353313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ca425-d846-4021-8ff3-ed565b516975 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:32.355668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323b8c7-981f-4c1b-b48a-3e37f86daed4 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:32.356063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ca425-d846-4021-8ff3-ed565b516975 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:32.356617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ca425-d846-4021-8ff3-ed565b516975 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:32.363984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b51ca425-d846-4021-8ff3-ed565b516975 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:32.378972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea6fa8-20b2-45ae-b752-5bd5a4442214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:32.380072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea6fa8-20b2-45ae-b752-5bd5a4442214 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:32.380895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea6fa8-20b2-45ae-b752-5bd5a4442214 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:32.381548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea6fa8-20b2-45ae-b752-5bd5a4442214 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:32.382691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea6fa8-20b2-45ae-b752-5bd5a4442214 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:58:32.384007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea6fa8-20b2-45ae-b752-5bd5a4442214 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:32.386071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea6fa8-20b2-45ae-b752-5bd5a4442214 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:32.427345 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1834/3591] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:32.428389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea6fa8-20b2-45ae-b752-5bd5a4442214 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:32.457441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea6fa8-20b2-45ae-b752-5bd5a4442214 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:32.495452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d4d26-a390-4a62-ad2d-255ab794904d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:32.496020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d4d26-a390-4a62-ad2d-255ab794904d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:32.496217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d4d26-a390-4a62-ad2d-255ab794904d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:32.496944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d4d26-a390-4a62-ad2d-255ab794904d admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:32.497593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d4d26-a390-4a62-ad2d-255ab794904d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:32.508075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d4d26-a390-4a62-ad2d-255ab794904d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:32.537873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ca425-d846-4021-8ff3-ed565b516975 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:32.539680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ca425-d846-4021-8ff3-ed565b516975 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:32.578994 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1758/3592] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:58:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:58:32.632641 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1835/3593] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:58:32 2022] POST /identity/v3/projects => generated 395 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:32.637452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ca425-d846-4021-8ff3-ed565b516975 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:32.659806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ca425-d846-4021-8ff3-ed565b516975 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:32.796165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6249af-79a3-4e3c-b16b-33086530abf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:32.800349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6249af-79a3-4e3c-b16b-33086530abf0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:32.800349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6249af-79a3-4e3c-b16b-33086530abf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:32.800349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6249af-79a3-4e3c-b16b-33086530abf0 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:32.812130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6249af-79a3-4e3c-b16b-33086530abf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:32.814647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6249af-79a3-4e3c-b16b-33086530abf0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:32.885468 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1759/3594] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:58:32 2022] POST /identity/v3/users => generated 298 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:32.890098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d4d26-a390-4a62-ad2d-255ab794904d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:32.921520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d4d26-a390-4a62-ad2d-255ab794904d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:32.939600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46486877-0e7d-428e-8247-3a3388734e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:32.944505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46486877-0e7d-428e-8247-3a3388734e29 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:32.945000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46486877-0e7d-428e-8247-3a3388734e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:32.945592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46486877-0e7d-428e-8247-3a3388734e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:32.948700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46486877-0e7d-428e-8247-3a3388734e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:32.956075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46486877-0e7d-428e-8247-3a3388734e29 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:33.053076 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1836/3595] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:33.057977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6249af-79a3-4e3c-b16b-33086530abf0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:33.072130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6249af-79a3-4e3c-b16b-33086530abf0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:33.082084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a09aa-1204-4615-9beb-52b857e16f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:33.083988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a09aa-1204-4615-9beb-52b857e16f72 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:33.086229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a09aa-1204-4615-9beb-52b857e16f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:33.086718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a09aa-1204-4615-9beb-52b857e16f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:33.088894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a09aa-1204-4615-9beb-52b857e16f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:33.091881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a09aa-1204-4615-9beb-52b857e16f72 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:33.120457 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1760/3596] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:33.125301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46486877-0e7d-428e-8247-3a3388734e29 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:33.141153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46486877-0e7d-428e-8247-3a3388734e29 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:33.159565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367e21f-12cf-4061-86d3-3b2cc3a38e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:33.160274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367e21f-12cf-4061-86d3-3b2cc3a38e0a admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:33.160554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367e21f-12cf-4061-86d3-3b2cc3a38e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:33.160831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367e21f-12cf-4061-86d3-3b2cc3a38e0a admin admin] PATH_INFO: `/v3/projects/defa530257d048d6827bc009758831fb/users/0dbcefce07614bf7b7f1c23d5d6fc9d6/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:33.221596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8367e21f-12cf-4061-86d3-3b2cc3a38e0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=defa530257d048d6827bc009758831fb, user_id=0dbcefce07614bf7b7f1c23d5d6fc9d6, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:33.231715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8367e21f-12cf-4061-86d3-3b2cc3a38e0a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:33.291438 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1837/3597] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:58:33 2022] PUT /identity/v3/projects/defa530257d048d6827bc009758831fb/users/0dbcefce07614bf7b7f1c23d5d6fc9d6/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:33.302912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84135c-dada-434d-b862-3898c923c5aa None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:33.303240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84135c-dada-434d-b862-3898c923c5aa None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:33.303624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84135c-dada-434d-b862-3898c923c5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:33.344377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:33.370525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7c84135c-dada-434d-b862-3898c923c5aa None None] MFA Rules not processed for user `0dbcefce07614bf7b7f1c23d5d6fc9d6`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:33.377659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7c84135c-dada-434d-b862-3898c923c5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:33.664183 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1761/3598] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:33 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:33.668906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367e21f-12cf-4061-86d3-3b2cc3a38e0a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:33.893158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367e21f-12cf-4061-86d3-3b2cc3a38e0a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:33.960362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c0e3a-d780-422d-8247-17d562196d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:33.963187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c0e3a-d780-422d-8247-17d562196d8b admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:33.963711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c0e3a-d780-422d-8247-17d562196d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:33.964144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c0e3a-d780-422d-8247-17d562196d8b admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:33.965299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c0e3a-d780-422d-8247-17d562196d8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:33.970411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c0e3a-d780-422d-8247-17d562196d8b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:34.054467 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1838/3599] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:58:33 2022] POST /identity/v3/projects => generated 397 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:34.057501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84135c-dada-434d-b862-3898c923c5aa None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:34.077251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84135c-dada-434d-b862-3898c923c5aa None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:34.187814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7894fbe-8e9b-4267-9731-db3a7d762a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:34.188594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7894fbe-8e9b-4267-9731-db3a7d762a03 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:34.189267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7894fbe-8e9b-4267-9731-db3a7d762a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:34.189616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7894fbe-8e9b-4267-9731-db3a7d762a03 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:34.190948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7894fbe-8e9b-4267-9731-db3a7d762a03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:34.194557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7894fbe-8e9b-4267-9731-db3a7d762a03 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:34.236625 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1762/3600] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:58:34 2022] POST /identity/v3/users => generated 305 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:34.238518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c0e3a-d780-422d-8247-17d562196d8b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:34.258143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c0e3a-d780-422d-8247-17d562196d8b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:34.274658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de1cd3-b760-4f2e-b75a-8aeac24c1d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:34.275417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de1cd3-b760-4f2e-b75a-8aeac24c1d5d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:34.275703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de1cd3-b760-4f2e-b75a-8aeac24c1d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:34.276106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de1cd3-b760-4f2e-b75a-8aeac24c1d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:34.276618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de1cd3-b760-4f2e-b75a-8aeac24c1d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:34.283596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de1cd3-b760-4f2e-b75a-8aeac24c1d5d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:34.291578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:34.312354 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1839/3601] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:34.315565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7894fbe-8e9b-4267-9731-db3a7d762a03 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:34.344859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7894fbe-8e9b-4267-9731-db3a7d762a03 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:34.361948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdab7b8-1c51-4476-88e3-5fde5866283c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:34.362524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdab7b8-1c51-4476-88e3-5fde5866283c admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:34.362861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdab7b8-1c51-4476-88e3-5fde5866283c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:34.363092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdab7b8-1c51-4476-88e3-5fde5866283c admin admin] PATH_INFO: `/v3/projects/c0374c63d6a64f7b9c35395fab297d51/users/0a39cbfd5565472b884bb47cb6ed92a4/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:34.420752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdab7b8-1c51-4476-88e3-5fde5866283c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0374c63d6a64f7b9c35395fab297d51, user_id=0a39cbfd5565472b884bb47cb6ed92a4, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:34.424697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdab7b8-1c51-4476-88e3-5fde5866283c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:34.457109 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1763/3602] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:58:34 2022] PUT /identity/v3/projects/c0374c63d6a64f7b9c35395fab297d51/users/0a39cbfd5565472b884bb47cb6ed92a4/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:34.461098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de1cd3-b760-4f2e-b75a-8aeac24c1d5d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:34.478773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de1cd3-b760-4f2e-b75a-8aeac24c1d5d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:34.681373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5486e-09f4-4e62-bb6b-742b150ab002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:34.682296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5486e-09f4-4e62-bb6b-742b150ab002 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:34.682771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5486e-09f4-4e62-bb6b-742b150ab002 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:34.687950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5486e-09f4-4e62-bb6b-742b150ab002 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:34.689180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf5486e-09f4-4e62-bb6b-742b150ab002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:34.691280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf5486e-09f4-4e62-bb6b-742b150ab002 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:34.715896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdab7b8-1c51-4476-88e3-5fde5866283c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:34.724769 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1840/3603] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:58:34.731892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdab7b8-1c51-4476-88e3-5fde5866283c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:34.745004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027c942-d1c0-4aca-bcb3-9ec8925141d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:34.745972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027c942-d1c0-4aca-bcb3-9ec8925141d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:34.746561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027c942-d1c0-4aca-bcb3-9ec8925141d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:34.747281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027c942-d1c0-4aca-bcb3-9ec8925141d4 admin admin] PATH_INFO: `/v3/projects/c0374c63d6a64f7b9c35395fab297d51/users/0a39cbfd5565472b884bb47cb6ed92a4/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:34.763093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027c942-d1c0-4aca-bcb3-9ec8925141d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0374c63d6a64f7b9c35395fab297d51, user_id=0a39cbfd5565472b884bb47cb6ed92a4, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:34.772094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027c942-d1c0-4aca-bcb3-9ec8925141d4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:34.797861 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1764/3604] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:58:34 2022] PUT /identity/v3/projects/c0374c63d6a64f7b9c35395fab297d51/users/0a39cbfd5565472b884bb47cb6ed92a4/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:58:34.812436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31cf53-da36-43b5-9998-5ed3e72fc280 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:34.812835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31cf53-da36-43b5-9998-5ed3e72fc280 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:34.813193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31cf53-da36-43b5-9998-5ed3e72fc280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:34.916472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fc31cf53-da36-43b5-9998-5ed3e72fc280 None None] MFA Rules not processed for user `0a39cbfd5565472b884bb47cb6ed92a4`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:34.919222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fc31cf53-da36-43b5-9998-5ed3e72fc280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:35.165580 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1841/3605] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:34 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:58:35.170918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027c942-d1c0-4aca-bcb3-9ec8925141d4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:35.391255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:35.405502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027c942-d1c0-4aca-bcb3-9ec8925141d4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:35.415459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6a2c3-167a-49ff-a0a0-d73565c0cf9c tempest-ProjectsNegativeStaticTestJSON-348920402 tempest-ProjectsNegativeStaticTestJSON-348920402-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0dbcefce07614bf7b7f1c23d5d6fc9d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defa530257d048d6827bc009758831fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:35.416348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6a2c3-167a-49ff-a0a0-d73565c0cf9c tempest-ProjectsNegativeStaticTestJSON-348920402 tempest-ProjectsNegativeStaticTestJSON-348920402-project] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:35.419349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6a2c3-167a-49ff-a0a0-d73565c0cf9c tempest-ProjectsNegativeStaticTestJSON-348920402 tempest-ProjectsNegativeStaticTestJSON-348920402-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:35.419796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6a2c3-167a-49ff-a0a0-d73565c0cf9c tempest-ProjectsNegativeStaticTestJSON-348920402 tempest-ProjectsNegativeStaticTestJSON-348920402-project] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:35.420618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6a2c3-167a-49ff-a0a0-d73565c0cf9c tempest-ProjectsNegativeStaticTestJSON-348920402 tempest-ProjectsNegativeStaticTestJSON-348920402-project] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:35.449920 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-fea6a2c3-167a-49ff-a0a0-d73565c0cf9c tempest-ProjectsNegativeStaticTestJSON-348920402 tempest-ProjectsNegativeStaticTestJSON-348920402-project] You are not authorized to perform the requested action: identity:create_project.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:create_project. Oct 01 20:58:35.449920 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:58:35.449920 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:58:35.449920 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:58:35.449920 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:58:35.449920 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:58:35.449920 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:58:35.449920 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:58:35.449920 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:58:35.449920 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:58:35.449920 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:58:35.449920 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:58:35.449920 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 162, in post Oct 01 20:58:35.449920 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Oct 01 20:58:35.451088 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Oct 01 20:58:35.451088 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application enforcer_obj._enforce( Oct 01 20:58:35.451088 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Oct 01 20:58:35.451088 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Oct 01 20:58:35.451088 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 01 20:58:35.451088 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Oct 01 20:58:35.451088 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:create_project. Oct 01 20:58:35.451088 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:58:35.460174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31cf53-da36-43b5-9998-5ed3e72fc280 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:35.460656 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1765/3606] 10.222.0.17 () {60 vars in 1152 bytes} [Sat Oct 1 20:58:35 2022] POST /identity/v3/projects => generated 136 bytes in 290 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Oct 01 20:58:35.478273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31cf53-da36-43b5-9998-5ed3e72fc280 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:35.495910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a001e1-8597-4a14-b228-4a82ff9d15d9 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a39cbfd5565472b884bb47cb6ed92a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0374c63d6a64f7b9c35395fab297d51', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:35.501709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a001e1-8597-4a14-b228-4a82ff9d15d9 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:35.503040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a001e1-8597-4a14-b228-4a82ff9d15d9 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:35.504559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a001e1-8597-4a14-b228-4a82ff9d15d9 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:35.506609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a001e1-8597-4a14-b228-4a82ff9d15d9 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:35.515273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a001e1-8597-4a14-b228-4a82ff9d15d9 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-38a001e1-8597-4a14-b228-4a82ff9d15d9 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Invalid input for field 'name': '' is too short Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: Failed validating 'minLength' in schema['properties']['name']: Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: On instance['name']: Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: '': keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: Failed validating 'minLength' in schema['properties']['name']: Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: On instance['name']: Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: '' Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise error Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: '' is too short Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application On instance['name']: Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application '' Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:58:35.556335 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:58:35.557941 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:58:35.557941 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:58:35.557941 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:58:35.557941 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:58:35.557941 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:58:35.557941 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:58:35.557941 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:58:35.557941 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:58:35.557941 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:58:35.557941 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:58:35.557941 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:58:35.557941 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:58:35.557941 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 165, in post Oct 01 20:58:35.557941 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application validation.lazy_validate(schema.project_create, project) Oct 01 20:58:35.557941 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Oct 01 20:58:35.557941 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Oct 01 20:58:35.557941 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Oct 01 20:58:35.559060 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Oct 01 20:58:35.559060 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Oct 01 20:58:35.559060 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:58:35.559060 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Oct 01 20:58:35.559060 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 01 20:58:35.559060 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:58:35.559060 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application On instance['name']: Oct 01 20:58:35.559060 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application '' Oct 01 20:58:35.559060 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:58:35.564972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6a2c3-167a-49ff-a0a0-d73565c0cf9c tempest-ProjectsNegativeStaticTestJSON-348920402 tempest-ProjectsNegativeStaticTestJSON-348920402-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:35.568532 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1842/3607] 10.222.0.17 () {60 vars in 1152 bytes} [Sat Oct 1 20:58:35 2022] POST /identity/v3/projects => generated 283 bytes in 109 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Oct 01 20:58:35.588186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6a2c3-167a-49ff-a0a0-d73565c0cf9c tempest-ProjectsNegativeStaticTestJSON-348920402 tempest-ProjectsNegativeStaticTestJSON-348920402-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:35.606740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fa8ed-5985-41b2-918b-1e7a82c295a2 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a39cbfd5565472b884bb47cb6ed92a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0374c63d6a64f7b9c35395fab297d51', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:35.606997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fa8ed-5985-41b2-918b-1e7a82c295a2 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:35.611707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fa8ed-5985-41b2-918b-1e7a82c295a2 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:35.611707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fa8ed-5985-41b2-918b-1e7a82c295a2 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:35.612344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238fa8ed-5985-41b2-918b-1e7a82c295a2 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:35.621030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238fa8ed-5985-41b2-918b-1e7a82c295a2 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:35.650123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-238fa8ed-5985-41b2-918b-1e7a82c295a2 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 01 20:58:35.650123 ncdv-master-1 devstack@keystone.service[99140]: Failed validating 'maxLength' in schema['properties']['name']: Oct 01 20:58:35.650123 ncdv-master-1 devstack@keystone.service[99140]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 01 20:58:35.650123 ncdv-master-1 devstack@keystone.service[99140]: On instance['name']: Oct 01 20:58:35.650123 ncdv-master-1 devstack@keystone.service[99140]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 01 20:58:35.650123 ncdv-master-1 devstack@keystone.service[99140]: Failed validating 'maxLength' in schema['properties']['name']: Oct 01 20:58:35.650123 ncdv-master-1 devstack@keystone.service[99140]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 01 20:58:35.650123 ncdv-master-1 devstack@keystone.service[99140]: On instance['name']: Oct 01 20:58:35.650123 ncdv-master-1 devstack@keystone.service[99140]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Oct 01 20:58:35.650123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:58:35.650123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Oct 01 20:58:35.650123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Oct 01 20:58:35.650123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 01 20:58:35.650123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise error Oct 01 20:58:35.650123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 01 20:58:35.650123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:58:35.650123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Oct 01 20:58:35.650123 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 01 20:58:35.651498 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:58:35.651498 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application On instance['name']: Oct 01 20:58:35.651498 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Oct 01 20:58:35.651498 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:58:35.651498 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:58:35.651498 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:58:35.651498 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:58:35.651498 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:58:35.651498 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:58:35.651498 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:58:35.651498 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:58:35.651498 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:58:35.651498 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:58:35.651498 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:58:35.651498 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:58:35.651498 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:58:35.651498 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:58:35.651498 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 165, in post Oct 01 20:58:35.652661 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application validation.lazy_validate(schema.project_create, project) Oct 01 20:58:35.652661 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Oct 01 20:58:35.652661 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Oct 01 20:58:35.652661 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Oct 01 20:58:35.652661 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Oct 01 20:58:35.652661 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 01 20:58:35.652661 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:58:35.652661 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Oct 01 20:58:35.652661 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 01 20:58:35.652661 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:58:35.652661 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application On instance['name']: Oct 01 20:58:35.652661 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Oct 01 20:58:35.652661 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:58:35.659698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a001e1-8597-4a14-b228-4a82ff9d15d9 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:35.668179 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1766/3608] 10.222.0.17 () {60 vars in 1152 bytes} [Sat Oct 1 20:58:35 2022] POST /identity/v3/projects => generated 412 bytes in 104 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 01 20:58:35.680214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a001e1-8597-4a14-b228-4a82ff9d15d9 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:35.702196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693e2ef-b7f2-4d50-b24b-155c645ddb2a tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a39cbfd5565472b884bb47cb6ed92a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0374c63d6a64f7b9c35395fab297d51', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:35.702695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693e2ef-b7f2-4d50-b24b-155c645ddb2a tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:35.702976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693e2ef-b7f2-4d50-b24b-155c645ddb2a tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:35.703245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693e2ef-b7f2-4d50-b24b-155c645ddb2a tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] PATH_INFO: `/v3/projects/b12e7e3280b64032bd9a63428c2d1cf8` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:35.739153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693e2ef-b7f2-4d50-b24b-155c645ddb2a tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b12e7e3280b64032bd9a63428c2d1cf8)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:35.749407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693e2ef-b7f2-4d50-b24b-155c645ddb2a tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:35.820301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-c693e2ef-b7f2-4d50-b24b-155c645ddb2a tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Could not find project: b12e7e3280b64032bd9a63428c2d1cf8.: keystone.exception.ProjectNotFound: Could not find project: b12e7e3280b64032bd9a63428c2d1cf8. Oct 01 20:58:35.820301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:58:35.820301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:58:35.820301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:58:35.820301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:58:35.820301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:58:35.820301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:58:35.820301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:58:35.820301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:58:35.820301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:58:35.820301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:58:35.820301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:58:35.820301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 211, in delete Oct 01 20:58:35.820301 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_project( Oct 01 20:58:35.821501 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:58:35.821501 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:58:35.821501 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 478, in delete_project Oct 01 20:58:35.821501 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = self.driver.get_project(project_id) Oct 01 20:58:35.821501 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 20:58:35.821501 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 20:58:35.821501 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 20:58:35.821501 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 20:58:35.821501 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: b12e7e3280b64032bd9a63428c2d1cf8. Oct 01 20:58:35.821501 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:58:35.829296 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1843/3609] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:35 2022] DELETE /identity/v3/projects/b12e7e3280b64032bd9a63428c2d1cf8 => generated 113 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 20:58:35.830907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fa8ed-5985-41b2-918b-1e7a82c295a2 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:35.845971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fa8ed-5985-41b2-918b-1e7a82c295a2 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:35.872424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12807ee-b72e-4afd-88b2-4f9ebfc153e3 tempest-ProjectsNegativeStaticTestJSON-348920402 tempest-ProjectsNegativeStaticTestJSON-348920402-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0dbcefce07614bf7b7f1c23d5d6fc9d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defa530257d048d6827bc009758831fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:35.873066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12807ee-b72e-4afd-88b2-4f9ebfc153e3 tempest-ProjectsNegativeStaticTestJSON-348920402 tempest-ProjectsNegativeStaticTestJSON-348920402-project] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:35.875852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12807ee-b72e-4afd-88b2-4f9ebfc153e3 tempest-ProjectsNegativeStaticTestJSON-348920402 tempest-ProjectsNegativeStaticTestJSON-348920402-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:35.876236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12807ee-b72e-4afd-88b2-4f9ebfc153e3 tempest-ProjectsNegativeStaticTestJSON-348920402 tempest-ProjectsNegativeStaticTestJSON-348920402-project] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:35.877418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12807ee-b72e-4afd-88b2-4f9ebfc153e3 tempest-ProjectsNegativeStaticTestJSON-348920402 tempest-ProjectsNegativeStaticTestJSON-348920402-project] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:35.899231 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-f12807ee-b72e-4afd-88b2-4f9ebfc153e3 tempest-ProjectsNegativeStaticTestJSON-348920402 tempest-ProjectsNegativeStaticTestJSON-348920402-project] You are not authorized to perform the requested action: identity:list_projects.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:list_projects. Oct 01 20:58:35.899231 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:58:35.899231 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:58:35.899231 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:58:35.899231 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:58:35.899231 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:58:35.899231 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:58:35.899231 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:58:35.899231 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:58:35.899231 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:58:35.899231 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:58:35.899231 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:58:35.899231 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 153, in get Oct 01 20:58:35.899231 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._list_projects() Oct 01 20:58:35.900443 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 120, in _list_projects Oct 01 20:58:35.900443 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:list_projects', Oct 01 20:58:35.900443 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Oct 01 20:58:35.900443 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application enforcer_obj._enforce( Oct 01 20:58:35.900443 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Oct 01 20:58:35.900443 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Oct 01 20:58:35.900443 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 01 20:58:35.900443 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Oct 01 20:58:35.900443 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:list_projects. Oct 01 20:58:35.900443 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:58:35.920680 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1767/3610] 10.222.0.17 () {58 vars in 1131 bytes} [Sat Oct 1 20:58:35 2022] GET /identity/v3/projects => generated 135 bytes in 90 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Oct 01 20:58:35.922650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693e2ef-b7f2-4d50-b24b-155c645ddb2a tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:35.951977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693e2ef-b7f2-4d50-b24b-155c645ddb2a tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:35.967632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e734cf-d971-4277-adfb-f33d38fe0546 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a39cbfd5565472b884bb47cb6ed92a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0374c63d6a64f7b9c35395fab297d51', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:35.969285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e734cf-d971-4277-adfb-f33d38fe0546 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:35.969765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e734cf-d971-4277-adfb-f33d38fe0546 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:35.970393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e734cf-d971-4277-adfb-f33d38fe0546 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:35.971525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e734cf-d971-4277-adfb-f33d38fe0546 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:35.976752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e734cf-d971-4277-adfb-f33d38fe0546 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:36.056133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12807ee-b72e-4afd-88b2-4f9ebfc153e3 tempest-ProjectsNegativeStaticTestJSON-348920402 tempest-ProjectsNegativeStaticTestJSON-348920402-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:36.056912 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1844/3611] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:58:35 2022] POST /identity/v3/projects => generated 357 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:36.087911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12807ee-b72e-4afd-88b2-4f9ebfc153e3 tempest-ProjectsNegativeStaticTestJSON-348920402 tempest-ProjectsNegativeStaticTestJSON-348920402-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:36.323256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749752f4-f7b1-47df-b79f-9756bffa8325 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a39cbfd5565472b884bb47cb6ed92a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0374c63d6a64f7b9c35395fab297d51', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:36.323896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749752f4-f7b1-47df-b79f-9756bffa8325 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:36.324182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749752f4-f7b1-47df-b79f-9756bffa8325 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:36.324851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749752f4-f7b1-47df-b79f-9756bffa8325 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:36.325270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749752f4-f7b1-47df-b79f-9756bffa8325 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:36.327201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749752f4-f7b1-47df-b79f-9756bffa8325 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:36.338936 ncdv-master-1 devstack@keystone.service[99140]: WARNING keystone.server.flask.common [None req-749752f4-f7b1-47df-b79f-9756bffa8325 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 01 20:58:36.380393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.sql.core [None req-749752f4-f7b1-47df-b79f-9756bffa8325 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-tempest-project-dup-637704329' for key 'project.ixu_project_name_domain_id'") Oct 01 20:58:36.380393 ncdv-master-1 devstack@keystone.service[99140]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 01 20:58:36.380393 ncdv-master-1 devstack@keystone.service[99140]: [parameters: {'id': '2fe0080b971e4956b75825e64d426f65', 'name': 'tempest-project-dup-637704329', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 01 20:58:36.380393 ncdv-master-1 devstack@keystone.service[99140]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99140) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 01 20:58:36.441837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-749752f4-f7b1-47df-b79f-9756bffa8325 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-637704329, project id 2fe0080b971e4956b75825e64d426f65.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-637704329, project id 2fe0080b971e4956b75825e64d426f65. Oct 01 20:58:36.441837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:58:36.441837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 01 20:58:36.441837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 01 20:58:36.441837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 01 20:58:36.441837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 01 20:58:36.441837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 01 20:58:36.441837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application result = self._query(query) Oct 01 20:58:36.441837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 01 20:58:36.441837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application conn.query(q) Oct 01 20:58:36.441837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 01 20:58:36.441837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 01 20:58:36.442742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 01 20:58:36.442742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application result.read() Oct 01 20:58:36.442742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 01 20:58:36.442742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 01 20:58:36.442742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 01 20:58:36.442742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 01 20:58:36.442742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 01 20:58:36.442742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 01 20:58:36.442742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 01 20:58:36.442742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 01 20:58:36.442742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-tempest-project-dup-637704329' for key 'project.ixu_project_name_domain_id'") Oct 01 20:58:36.442742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:58:36.442742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 01 20:58:36.442742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:58:36.442742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:58:36.442742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 01 20:58:36.442742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 01 20:58:36.443679 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 01 20:58:36.443679 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 01 20:58:36.443679 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 01 20:58:36.443679 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application next(self.gen) Oct 01 20:58:36.443679 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 01 20:58:36.443679 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application yield resource Oct 01 20:58:36.443679 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 01 20:58:36.443679 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application next(self.gen) Oct 01 20:58:36.443679 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 01 20:58:36.443679 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.session.rollback() Oct 01 20:58:36.443679 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 01 20:58:36.443679 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.force_reraise() Oct 01 20:58:36.443679 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 01 20:58:36.443679 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise self.value Oct 01 20:58:36.443679 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 01 20:58:36.443679 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 01 20:58:36.443679 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 01 20:58:36.443679 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application session.commit() Oct 01 20:58:36.444688 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 01 20:58:36.444688 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 01 20:58:36.444688 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 01 20:58:36.444688 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._prepare_impl() Oct 01 20:58:36.444688 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 01 20:58:36.444688 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.session.flush() Oct 01 20:58:36.444688 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 01 20:58:36.444688 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._flush(objects) Oct 01 20:58:36.444688 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 01 20:58:36.444688 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 01 20:58:36.444688 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 01 20:58:36.444688 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application compat.raise_( Oct 01 20:58:36.444688 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 01 20:58:36.444688 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception Oct 01 20:58:36.444688 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 01 20:58:36.444688 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application flush_context.execute() Oct 01 20:58:36.444688 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 01 20:58:36.445637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rec.execute(self) Oct 01 20:58:36.445637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 01 20:58:36.445637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 01 20:58:36.445637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 01 20:58:36.445637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 01 20:58:36.445637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 01 20:58:36.445637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 01 20:58:36.445637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 01 20:58:36.445637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 01 20:58:36.445637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 01 20:58:36.445637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 01 20:58:36.445637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 01 20:58:36.445637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 01 20:58:36.445637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 01 20:58:36.445637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 01 20:58:36.445637 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 01 20:58:36.449326 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 01 20:58:36.449326 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 01 20:58:36.449326 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception Oct 01 20:58:36.449326 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 01 20:58:36.449326 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 01 20:58:36.449326 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 01 20:58:36.449326 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 01 20:58:36.449326 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 01 20:58:36.449326 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application result = self._query(query) Oct 01 20:58:36.449326 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 01 20:58:36.449326 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application conn.query(q) Oct 01 20:58:36.449326 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 01 20:58:36.449326 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 01 20:58:36.449326 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 01 20:58:36.449326 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application result.read() Oct 01 20:58:36.449326 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 01 20:58:36.449326 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 01 20:58:36.450237 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 01 20:58:36.450237 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 01 20:58:36.450237 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 01 20:58:36.450237 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 01 20:58:36.450237 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 01 20:58:36.450237 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 01 20:58:36.450237 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-tempest-project-dup-637704329' for key 'project.ixu_project_name_domain_id'") Oct 01 20:58:36.450237 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 01 20:58:36.450237 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [parameters: {'id': '2fe0080b971e4956b75825e64d426f65', 'name': 'tempest-project-dup-637704329', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 01 20:58:36.450237 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 01 20:58:36.450237 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:58:36.450237 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:58:36.450237 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:58:36.450237 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:58:36.450237 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 01 20:58:36.451115 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 01 20:58:36.451115 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 01 20:58:36.451115 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 01 20:58:36.451115 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name tempest-project-dup-637704329 at domain ID default. Oct 01 20:58:36.451115 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:58:36.451115 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 20:58:36.451115 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:58:36.451115 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:58:36.451115 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:58:36.451115 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:58:36.451115 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:58:36.451115 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:58:36.451115 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:58:36.451115 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:58:36.451115 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:58:36.451115 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:58:36.451115 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:58:36.452102 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:58:36.452102 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 01 20:58:36.452102 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 01 20:58:36.452102 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:58:36.452102 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:58:36.452102 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 01 20:58:36.452102 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 01 20:58:36.452102 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-637704329, project id 2fe0080b971e4956b75825e64d426f65. Oct 01 20:58:36.452102 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:58:36.452102 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1768/3612] 10.222.0.17 () {60 vars in 1152 bytes} [Sat Oct 1 20:58:36 2022] POST /identity/v3/projects => generated 284 bytes in 387 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:58:36.452672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e734cf-d971-4277-adfb-f33d38fe0546 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:36.464551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e734cf-d971-4277-adfb-f33d38fe0546 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:36.483751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60887e7c-b2d9-417a-88c2-81922613aacf tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a39cbfd5565472b884bb47cb6ed92a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0374c63d6a64f7b9c35395fab297d51', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:36.488327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60887e7c-b2d9-417a-88c2-81922613aacf tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:36.488745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60887e7c-b2d9-417a-88c2-81922613aacf tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:36.489037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60887e7c-b2d9-417a-88c2-81922613aacf tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] PATH_INFO: `/v3/projects/8e6163649efd46d3b667df08e7267f9d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:36.500721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60887e7c-b2d9-417a-88c2-81922613aacf tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8e6163649efd46d3b667df08e7267f9d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:36.509235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60887e7c-b2d9-417a-88c2-81922613aacf tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:36.709638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:36.754181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-60887e7c-b2d9-417a-88c2-81922613aacf tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e6163649efd46d3b667df08e7267f9d', 'request_id': 'req-60887e7c-b2d9-417a-88c2-81922613aacf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:36.784719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-60887e7c-b2d9-417a-88c2-81922613aacf tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e6163649efd46d3b667df08e7267f9d', 'request_id': 'req-60887e7c-b2d9-417a-88c2-81922613aacf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:36.809288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-60887e7c-b2d9-417a-88c2-81922613aacf tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e6163649efd46d3b667df08e7267f9d', 'request_id': 'req-60887e7c-b2d9-417a-88c2-81922613aacf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:36.838997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-60887e7c-b2d9-417a-88c2-81922613aacf tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] The token cache is being invalidate because project 8e6163649efd46d3b667df08e7267f9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:36.840720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-60887e7c-b2d9-417a-88c2-81922613aacf tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:36.841979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-60887e7c-b2d9-417a-88c2-81922613aacf tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:36.845617 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1845/3613] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:36 2022] DELETE /identity/v3/projects/8e6163649efd46d3b667df08e7267f9d => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:36.850948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749752f4-f7b1-47df-b79f-9756bffa8325 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:36.853035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-749752f4-f7b1-47df-b79f-9756bffa8325 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:36.996087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749752f4-f7b1-47df-b79f-9756bffa8325 tempest-ProjectsNegativeStaticTestJSON-1661713922 tempest-ProjectsNegativeStaticTestJSON-1661713922-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:37.156093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b7c69-6dcd-4376-b5a2-658662eec9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:37.156763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b7c69-6dcd-4376-b5a2-658662eec9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:37.157047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b7c69-6dcd-4376-b5a2-658662eec9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:37.157561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b7c69-6dcd-4376-b5a2-658662eec9b3 admin admin] PATH_INFO: `/v3/users/0dbcefce07614bf7b7f1c23d5d6fc9d6` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:37.158626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b7c69-6dcd-4376-b5a2-658662eec9b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dbcefce07614bf7b7f1c23d5d6fc9d6)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:37.166586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b7c69-6dcd-4376-b5a2-658662eec9b3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:37.265038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:37.288984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8a0b7c69-6dcd-4376-b5a2-658662eec9b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dbcefce07614bf7b7f1c23d5d6fc9d6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:37.289778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8a0b7c69-6dcd-4376-b5a2-658662eec9b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dbcefce07614bf7b7f1c23d5d6fc9d6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:37.290481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8a0b7c69-6dcd-4376-b5a2-658662eec9b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dbcefce07614bf7b7f1c23d5d6fc9d6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:37.308598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8a0b7c69-6dcd-4376-b5a2-658662eec9b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dbcefce07614bf7b7f1c23d5d6fc9d6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:37.335177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8a0b7c69-6dcd-4376-b5a2-658662eec9b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dbcefce07614bf7b7f1c23d5d6fc9d6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:37.366699 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1769/3614] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:58:36 2022] DELETE /identity/v3/users/0dbcefce07614bf7b7f1c23d5d6fc9d6 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:37.368592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953ca13-8d85-44f3-949c-54147f14e31c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:37.370259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953ca13-8d85-44f3-949c-54147f14e31c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:37.370784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953ca13-8d85-44f3-949c-54147f14e31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:37.459863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8953ca13-8d85-44f3-949c-54147f14e31c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:37.465336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8953ca13-8d85-44f3-949c-54147f14e31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:37.742036 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1846/3615] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:37.777943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b7c69-6dcd-4376-b5a2-658662eec9b3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:37.779938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8a0b7c69-6dcd-4376-b5a2-658662eec9b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:37.939949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b7c69-6dcd-4376-b5a2-658662eec9b3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:37.950763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9ee68-8a9f-4772-b2c8-442e1e9a9339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:37.951443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9ee68-8a9f-4772-b2c8-442e1e9a9339 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:37.951679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9ee68-8a9f-4772-b2c8-442e1e9a9339 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:37.951950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9ee68-8a9f-4772-b2c8-442e1e9a9339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:37.998096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9ee68-8a9f-4772-b2c8-442e1e9a9339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:38.000620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9ee68-8a9f-4772-b2c8-442e1e9a9339 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:38.037007 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1770/3616] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:58:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:58:41.011128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5503dd-080b-40bb-8803-af4e44c53b55 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:41.011607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5503dd-080b-40bb-8803-af4e44c53b55 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:41.011721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5503dd-080b-40bb-8803-af4e44c53b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:41.098259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1e5503dd-080b-40bb-8803-af4e44c53b55 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:41.100002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1e5503dd-080b-40bb-8803-af4e44c53b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:41.325566 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1847/3617] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 20:58:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:41.333516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9ee68-8a9f-4772-b2c8-442e1e9a9339 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:41.335483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-73c9ee68-8a9f-4772-b2c8-442e1e9a9339 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:41.510703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9ee68-8a9f-4772-b2c8-442e1e9a9339 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:41.518388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee279d1c-7804-4b78-a7be-9ce52109d452 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:41.519433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee279d1c-7804-4b78-a7be-9ce52109d452 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:41.519866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee279d1c-7804-4b78-a7be-9ce52109d452 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:41.520288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee279d1c-7804-4b78-a7be-9ce52109d452 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:41.535375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee279d1c-7804-4b78-a7be-9ce52109d452 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:41.537332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee279d1c-7804-4b78-a7be-9ce52109d452 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:41.569830 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1771/3618] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:58:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 20:58:42.182572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5503dd-080b-40bb-8803-af4e44c53b55 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:42.184048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1e5503dd-080b-40bb-8803-af4e44c53b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:42.219023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5503dd-080b-40bb-8803-af4e44c53b55 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:42.238051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa9f0c-2f9a-40ec-b7df-eefa092e7cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:42.238738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa9f0c-2f9a-40ec-b7df-eefa092e7cc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:42.239023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa9f0c-2f9a-40ec-b7df-eefa092e7cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:42.239295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa9f0c-2f9a-40ec-b7df-eefa092e7cc4 admin admin] PATH_INFO: `/v3/projects/defa530257d048d6827bc009758831fb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:42.241574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa9f0c-2f9a-40ec-b7df-eefa092e7cc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=defa530257d048d6827bc009758831fb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:42.247223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa9f0c-2f9a-40ec-b7df-eefa092e7cc4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:42.377155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20fa9f0c-2f9a-40ec-b7df-eefa092e7cc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'defa530257d048d6827bc009758831fb', 'request_id': 'req-20fa9f0c-2f9a-40ec-b7df-eefa092e7cc4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:42.399174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20fa9f0c-2f9a-40ec-b7df-eefa092e7cc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'defa530257d048d6827bc009758831fb', 'request_id': 'req-20fa9f0c-2f9a-40ec-b7df-eefa092e7cc4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:42.411167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20fa9f0c-2f9a-40ec-b7df-eefa092e7cc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'defa530257d048d6827bc009758831fb', 'request_id': 'req-20fa9f0c-2f9a-40ec-b7df-eefa092e7cc4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:42.414195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:42.441967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20fa9f0c-2f9a-40ec-b7df-eefa092e7cc4 admin admin] The token cache is being invalidate because project defa530257d048d6827bc009758831fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:42.442286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20fa9f0c-2f9a-40ec-b7df-eefa092e7cc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:42.443302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20fa9f0c-2f9a-40ec-b7df-eefa092e7cc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:42.451536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee279d1c-7804-4b78-a7be-9ce52109d452 service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:42.452023 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1848/3619] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:42 2022] DELETE /identity/v3/projects/defa530257d048d6827bc009758831fb => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:42.475942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee279d1c-7804-4b78-a7be-9ce52109d452 service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:42.564308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:42.612618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552333b6-5291-4029-b433-cafa3a0ba147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:42.613735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552333b6-5291-4029-b433-cafa3a0ba147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:42.614246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552333b6-5291-4029-b433-cafa3a0ba147 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:42.614766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552333b6-5291-4029-b433-cafa3a0ba147 admin admin] PATH_INFO: `/v3/users/0a39cbfd5565472b884bb47cb6ed92a4` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:42.616205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552333b6-5291-4029-b433-cafa3a0ba147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a39cbfd5565472b884bb47cb6ed92a4)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:42.618377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552333b6-5291-4029-b433-cafa3a0ba147 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:42.755060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-552333b6-5291-4029-b433-cafa3a0ba147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a39cbfd5565472b884bb47cb6ed92a4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:42.758335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-552333b6-5291-4029-b433-cafa3a0ba147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a39cbfd5565472b884bb47cb6ed92a4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:42.762384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-552333b6-5291-4029-b433-cafa3a0ba147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a39cbfd5565472b884bb47cb6ed92a4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:42.775687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-552333b6-5291-4029-b433-cafa3a0ba147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a39cbfd5565472b884bb47cb6ed92a4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:42.800357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-552333b6-5291-4029-b433-cafa3a0ba147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a39cbfd5565472b884bb47cb6ed92a4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:42.817793 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1772/3620] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:58:42 2022] DELETE /identity/v3/users/0a39cbfd5565472b884bb47cb6ed92a4 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:44.388127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb87c0-942a-4c8a-97ea-638aaa2b847b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:44.388879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb87c0-942a-4c8a-97ea-638aaa2b847b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:44.388879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb87c0-942a-4c8a-97ea-638aaa2b847b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:44.479091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c6fb87c0-942a-4c8a-97ea-638aaa2b847b None None] MFA Rules not processed for user `775ff9091a794faaacc05a37550c8a72`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:44.480345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c6fb87c0-942a-4c8a-97ea-638aaa2b847b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:44.712665 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1849/3621] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:58:44.724081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c29e3-4049-4219-a8e6-248abdd05c31 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:44.725034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c29e3-4049-4219-a8e6-248abdd05c31 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:44.725518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c29e3-4049-4219-a8e6-248abdd05c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:44.753732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a38c29e3-4049-4219-a8e6-248abdd05c31 None None] MFA Rules not processed for user `775ff9091a794faaacc05a37550c8a72`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:44.755405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a38c29e3-4049-4219-a8e6-248abdd05c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:44.763319 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1773/3622] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:45.177744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb87c0-942a-4c8a-97ea-638aaa2b847b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:45.179451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c6fb87c0-942a-4c8a-97ea-638aaa2b847b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:45.182326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c29e3-4049-4219-a8e6-248abdd05c31 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:45.183822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a38c29e3-4049-4219-a8e6-248abdd05c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:45.304989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb87c0-942a-4c8a-97ea-638aaa2b847b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:45.310334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e2c99-a4ae-444d-8a78-5da526adb14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:45.311386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e2c99-a4ae-444d-8a78-5da526adb14c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:45.311880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e2c99-a4ae-444d-8a78-5da526adb14c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:45.312399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e2c99-a4ae-444d-8a78-5da526adb14c admin admin] PATH_INFO: `/v3/users/123a8a1d0b65407098b5a8e91091b9ca` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:45.315486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e2c99-a4ae-444d-8a78-5da526adb14c admin admin] RBAC: Authorizing `identity:delete_user(user_id=123a8a1d0b65407098b5a8e91091b9ca)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:45.319729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e2c99-a4ae-444d-8a78-5da526adb14c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:45.408294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c29e3-4049-4219-a8e6-248abdd05c31 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:45.425145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d4f6f-9842-4a1a-9287-e7a02eb577fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:45.426728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d4f6f-9842-4a1a-9287-e7a02eb577fc service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:45.430924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d4f6f-9842-4a1a-9287-e7a02eb577fc service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:45.433321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d4f6f-9842-4a1a-9287-e7a02eb577fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:45.458416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d4f6f-9842-4a1a-9287-e7a02eb577fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:45.461530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d4f6f-9842-4a1a-9287-e7a02eb577fc service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:45.487049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-082e2c99-a4ae-444d-8a78-5da526adb14c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '123a8a1d0b65407098b5a8e91091b9ca'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:45.498353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-082e2c99-a4ae-444d-8a78-5da526adb14c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '123a8a1d0b65407098b5a8e91091b9ca'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:45.501852 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1774/3623] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:58:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:58:45.523158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-082e2c99-a4ae-444d-8a78-5da526adb14c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '123a8a1d0b65407098b5a8e91091b9ca'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:45.524820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-082e2c99-a4ae-444d-8a78-5da526adb14c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '123a8a1d0b65407098b5a8e91091b9ca'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:45.556715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-082e2c99-a4ae-444d-8a78-5da526adb14c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '123a8a1d0b65407098b5a8e91091b9ca'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:45.567472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:45.587267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db446e76-56eb-456f-9f80-91434ae9ce04 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:45.587566 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1850/3624] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:58:45 2022] DELETE /identity/v3/users/123a8a1d0b65407098b5a8e91091b9ca => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:45.588106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db446e76-56eb-456f-9f80-91434ae9ce04 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:45.588565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db446e76-56eb-456f-9f80-91434ae9ce04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:45.645768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-db446e76-56eb-456f-9f80-91434ae9ce04 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:45.648732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-db446e76-56eb-456f-9f80-91434ae9ce04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:45.752839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:45.865494 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1775/3625] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:58:45.890399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e2c99-a4ae-444d-8a78-5da526adb14c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:45.892580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-082e2c99-a4ae-444d-8a78-5da526adb14c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:46.120264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e2c99-a4ae-444d-8a78-5da526adb14c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:46.131015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ed484-b080-49cd-bc59-41b416c1f4c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:46.142324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ed484-b080-49cd-bc59-41b416c1f4c1 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:46.146968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ed484-b080-49cd-bc59-41b416c1f4c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:46.148863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ed484-b080-49cd-bc59-41b416c1f4c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:46.175090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ed484-b080-49cd-bc59-41b416c1f4c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:46.182601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ed484-b080-49cd-bc59-41b416c1f4c1 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:46.213707 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1851/3626] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:58:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:58:46.746069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db446e76-56eb-456f-9f80-91434ae9ce04 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:46.748139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-db446e76-56eb-456f-9f80-91434ae9ce04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:46.775106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db446e76-56eb-456f-9f80-91434ae9ce04 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:46.780825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede843e-7cc5-47dc-affb-6f3632a903d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:46.781480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede843e-7cc5-47dc-affb-6f3632a903d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:46.781761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede843e-7cc5-47dc-affb-6f3632a903d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:46.782643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede843e-7cc5-47dc-affb-6f3632a903d6 admin admin] PATH_INFO: `/v3/projects/c0374c63d6a64f7b9c35395fab297d51` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:46.783812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede843e-7cc5-47dc-affb-6f3632a903d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0374c63d6a64f7b9c35395fab297d51)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:46.785755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede843e-7cc5-47dc-affb-6f3632a903d6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:46.915902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bede843e-7cc5-47dc-affb-6f3632a903d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0374c63d6a64f7b9c35395fab297d51', 'request_id': 'req-bede843e-7cc5-47dc-affb-6f3632a903d6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:46.936542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bede843e-7cc5-47dc-affb-6f3632a903d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0374c63d6a64f7b9c35395fab297d51', 'request_id': 'req-bede843e-7cc5-47dc-affb-6f3632a903d6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:46.943708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bede843e-7cc5-47dc-affb-6f3632a903d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0374c63d6a64f7b9c35395fab297d51', 'request_id': 'req-bede843e-7cc5-47dc-affb-6f3632a903d6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:46.970099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bede843e-7cc5-47dc-affb-6f3632a903d6 admin admin] The token cache is being invalidate because project c0374c63d6a64f7b9c35395fab297d51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:46.970099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bede843e-7cc5-47dc-affb-6f3632a903d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:46.979335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bede843e-7cc5-47dc-affb-6f3632a903d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:46.983311 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1776/3627] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:46 2022] DELETE /identity/v3/projects/c0374c63d6a64f7b9c35395fab297d51 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:46.989329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43dede-4780-4f46-848d-e118738043fb None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:46.990231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43dede-4780-4f46-848d-e118738043fb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:46.990717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43dede-4780-4f46-848d-e118738043fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:47.073520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ea43dede-4780-4f46-848d-e118738043fb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:47.074840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ea43dede-4780-4f46-848d-e118738043fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:47.262641 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1852/3628] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:47.271048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc8ca9-f6d4-45fb-a373-9bfdc6631197 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:47.271568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc8ca9-f6d4-45fb-a373-9bfdc6631197 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:47.272048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc8ca9-f6d4-45fb-a373-9bfdc6631197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:47.311781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2ebc8ca9-f6d4-45fb-a373-9bfdc6631197 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:47.316204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2ebc8ca9-f6d4-45fb-a373-9bfdc6631197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:47.380702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43dede-4780-4f46-848d-e118738043fb None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:47.381235 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1777/3629] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:47.392389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43dede-4780-4f46-848d-e118738043fb None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:47.410732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbc1cd-11c6-4763-9aea-c29d371edd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:47.411597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbc1cd-11c6-4763-9aea-c29d371edd68 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:47.412064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbc1cd-11c6-4763-9aea-c29d371edd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:47.412544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbc1cd-11c6-4763-9aea-c29d371edd68 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:47.415689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dbc1cd-11c6-4763-9aea-c29d371edd68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:58:47.416251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dbc1cd-11c6-4763-9aea-c29d371edd68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:47.423595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dbc1cd-11c6-4763-9aea-c29d371edd68 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:47.471698 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1853/3630] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:47.475883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc8ca9-f6d4-45fb-a373-9bfdc6631197 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:47.502147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc8ca9-f6d4-45fb-a373-9bfdc6631197 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:47.511884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898314ef-1708-43ad-9da6-701667d46498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:47.513323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898314ef-1708-43ad-9da6-701667d46498 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:47.513798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898314ef-1708-43ad-9da6-701667d46498 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:47.514282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898314ef-1708-43ad-9da6-701667d46498 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:47.515217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898314ef-1708-43ad-9da6-701667d46498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:47.518076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898314ef-1708-43ad-9da6-701667d46498 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:47.538322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbc1cd-11c6-4763-9aea-c29d371edd68 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:47.540503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-75dbc1cd-11c6-4763-9aea-c29d371edd68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:47.569941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbc1cd-11c6-4763-9aea-c29d371edd68 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:47.609581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1377b-fe73-4e59-9555-24bf2cf3e852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:47.610294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1377b-fe73-4e59-9555-24bf2cf3e852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:47.610975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1377b-fe73-4e59-9555-24bf2cf3e852 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:47.611564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1377b-fe73-4e59-9555-24bf2cf3e852 admin admin] PATH_INFO: `/v3/projects/a6bdd72ba854403e8fef7f651a3ef106` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:47.613363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1377b-fe73-4e59-9555-24bf2cf3e852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6bdd72ba854403e8fef7f651a3ef106)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:47.615483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1377b-fe73-4e59-9555-24bf2cf3e852 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:47.655794 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1778/3631] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:58:47 2022] POST /identity/v3/projects => generated 367 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:47.657782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898314ef-1708-43ad-9da6-701667d46498 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:47.805208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-21e1377b-fe73-4e59-9555-24bf2cf3e852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6bdd72ba854403e8fef7f651a3ef106', 'request_id': 'req-21e1377b-fe73-4e59-9555-24bf2cf3e852'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:47.837876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-21e1377b-fe73-4e59-9555-24bf2cf3e852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6bdd72ba854403e8fef7f651a3ef106', 'request_id': 'req-21e1377b-fe73-4e59-9555-24bf2cf3e852'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:47.844693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-21e1377b-fe73-4e59-9555-24bf2cf3e852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6bdd72ba854403e8fef7f651a3ef106', 'request_id': 'req-21e1377b-fe73-4e59-9555-24bf2cf3e852'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:47.845145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898314ef-1708-43ad-9da6-701667d46498 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:47.872486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-21e1377b-fe73-4e59-9555-24bf2cf3e852 admin admin] The token cache is being invalidate because project a6bdd72ba854403e8fef7f651a3ef106 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:47.872803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-21e1377b-fe73-4e59-9555-24bf2cf3e852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:47.873805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-21e1377b-fe73-4e59-9555-24bf2cf3e852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:47.879121 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1854/3632] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:47 2022] DELETE /identity/v3/projects/a6bdd72ba854403e8fef7f651a3ef106 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:47.881145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1377b-fe73-4e59-9555-24bf2cf3e852 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:47.885105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-21e1377b-fe73-4e59-9555-24bf2cf3e852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:47.925525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:47.956213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:48.076858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfdb5e3-7c51-4e3c-81f9-170a76a35c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:48.081814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdb5e3-7c51-4e3c-81f9-170a76a35c4c admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:48.082106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdb5e3-7c51-4e3c-81f9-170a76a35c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:48.082385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdb5e3-7c51-4e3c-81f9-170a76a35c4c admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:48.083077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfdb5e3-7c51-4e3c-81f9-170a76a35c4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:48.098620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfdb5e3-7c51-4e3c-81f9-170a76a35c4c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:48.148703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1377b-fe73-4e59-9555-24bf2cf3e852 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:48.159250 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1779/3633] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:58:47 2022] POST /identity/v3/users => generated 284 bytes in 502 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:48.162086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5aecae-0677-485e-95b6-7286f97aa103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:48.163634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfdb5e3-7c51-4e3c-81f9-170a76a35c4c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:48.165614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bdfdb5e3-7c51-4e3c-81f9-170a76a35c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:48.167152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5aecae-0677-485e-95b6-7286f97aa103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:48.167301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5aecae-0677-485e-95b6-7286f97aa103 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:48.168470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5aecae-0677-485e-95b6-7286f97aa103 admin admin] PATH_INFO: `/v3/users/a4be5d569e63435382d1bf274799a23b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:48.173254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5aecae-0677-485e-95b6-7286f97aa103 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4be5d569e63435382d1bf274799a23b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:48.177193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5aecae-0677-485e-95b6-7286f97aa103 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:48.199963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfdb5e3-7c51-4e3c-81f9-170a76a35c4c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:48.211345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967013cb-333a-4116-8167-a4307a0fa1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:48.213535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967013cb-333a-4116-8167-a4307a0fa1ef admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:48.213818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967013cb-333a-4116-8167-a4307a0fa1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:48.214096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967013cb-333a-4116-8167-a4307a0fa1ef admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:48.215085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967013cb-333a-4116-8167-a4307a0fa1ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:48.220632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967013cb-333a-4116-8167-a4307a0fa1ef admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:48.256874 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1780/3634] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:48.260405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967013cb-333a-4116-8167-a4307a0fa1ef admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:48.292858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967013cb-333a-4116-8167-a4307a0fa1ef admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:48.303370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099eb27-f512-45cb-b51b-1520a05d87b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:48.304945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ce5aecae-0677-485e-95b6-7286f97aa103 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4be5d569e63435382d1bf274799a23b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:48.306790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099eb27-f512-45cb-b51b-1520a05d87b4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:48.306940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099eb27-f512-45cb-b51b-1520a05d87b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:48.307184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099eb27-f512-45cb-b51b-1520a05d87b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:48.308186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099eb27-f512-45cb-b51b-1520a05d87b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:48.311662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099eb27-f512-45cb-b51b-1520a05d87b4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:48.322323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ce5aecae-0677-485e-95b6-7286f97aa103 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4be5d569e63435382d1bf274799a23b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:48.343043 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1781/3635] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:48.343885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099eb27-f512-45cb-b51b-1520a05d87b4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:48.379119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ce5aecae-0677-485e-95b6-7286f97aa103 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4be5d569e63435382d1bf274799a23b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:48.379890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ce5aecae-0677-485e-95b6-7286f97aa103 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4be5d569e63435382d1bf274799a23b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:48.394136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ce5aecae-0677-485e-95b6-7286f97aa103 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4be5d569e63435382d1bf274799a23b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:48.397961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099eb27-f512-45cb-b51b-1520a05d87b4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:48.402457 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1855/3636] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:58:47 2022] DELETE /identity/v3/users/a4be5d569e63435382d1bf274799a23b => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:48.436220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fc578-9270-4cce-aa8b-e10091bc3959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:48.436907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fc578-9270-4cce-aa8b-e10091bc3959 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:48.437190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fc578-9270-4cce-aa8b-e10091bc3959 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:48.444043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fc578-9270-4cce-aa8b-e10091bc3959 admin admin] PATH_INFO: `/v3/projects/897da389f8994ffc9444b347d75ea9ae/users/ddddd157af49408bb600c946165a13aa/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:48.521459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9fc578-9270-4cce-aa8b-e10091bc3959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=897da389f8994ffc9444b347d75ea9ae, user_id=ddddd157af49408bb600c946165a13aa, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:48.525607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9fc578-9270-4cce-aa8b-e10091bc3959 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:48.569903 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1782/3637] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:58:48 2022] PUT /identity/v3/projects/897da389f8994ffc9444b347d75ea9ae/users/ddddd157af49408bb600c946165a13aa/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:48.675886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732f296-6592-45df-987e-f98887f4ed56 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:48.676212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732f296-6592-45df-987e-f98887f4ed56 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:48.676455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732f296-6592-45df-987e-f98887f4ed56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:48.811722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b732f296-6592-45df-987e-f98887f4ed56 None None] MFA Rules not processed for user `ddddd157af49408bb600c946165a13aa`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:48.812970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b732f296-6592-45df-987e-f98887f4ed56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:49.003072 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1856/3638] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:58:49.007243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fc578-9270-4cce-aa8b-e10091bc3959 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:49.009165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3b9fc578-9270-4cce-aa8b-e10091bc3959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:49.117697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e14eec-330d-458c-b512-73dcfb039680 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:49.118016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e14eec-330d-458c-b512-73dcfb039680 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:49.118266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e14eec-330d-458c-b512-73dcfb039680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:49.159875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-99e14eec-330d-458c-b512-73dcfb039680 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:49.161462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-99e14eec-330d-458c-b512-73dcfb039680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:49.196811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fc578-9270-4cce-aa8b-e10091bc3959 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:49.220154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:49.220262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba7f9f-d722-4a13-aa08-f95ae7d3dfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:49.220262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba7f9f-d722-4a13-aa08-f95ae7d3dfa3 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:49.220535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba7f9f-d722-4a13-aa08-f95ae7d3dfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:49.220973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba7f9f-d722-4a13-aa08-f95ae7d3dfa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:49.222564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba7f9f-d722-4a13-aa08-f95ae7d3dfa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:49.224749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba7f9f-d722-4a13-aa08-f95ae7d3dfa3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:49.307745 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1857/3639] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:58:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:58:49.323293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e14eec-330d-458c-b512-73dcfb039680 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:49.324063 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1783/3640] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:58:49 2022] POST /identity/v3/projects => generated 367 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:49.324952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-99e14eec-330d-458c-b512-73dcfb039680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:49.327957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba7f9f-d722-4a13-aa08-f95ae7d3dfa3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:49.507091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e14eec-330d-458c-b512-73dcfb039680 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:49.530078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba7f9f-d722-4a13-aa08-f95ae7d3dfa3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:49.541179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5676b62-b868-410c-bf2f-595f904aba64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:49.544800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5676b62-b868-410c-bf2f-595f904aba64 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:49.546818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5676b62-b868-410c-bf2f-595f904aba64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:49.548416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5676b62-b868-410c-bf2f-595f904aba64 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:49.549186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5676b62-b868-410c-bf2f-595f904aba64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:49.560890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5676b62-b868-410c-bf2f-595f904aba64 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:49.605839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:49.646676 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1784/3641] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:58:49 2022] POST /identity/v3/users => generated 290 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:49.651046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5676b62-b868-410c-bf2f-595f904aba64 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:49.684859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5676b62-b868-410c-bf2f-595f904aba64 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:49.710081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b8ccd-d2c6-4b0a-a42b-fa5da8e47aa8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:49.711962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b8ccd-d2c6-4b0a-a42b-fa5da8e47aa8 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:49.712267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b8ccd-d2c6-4b0a-a42b-fa5da8e47aa8 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:49.713508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b8ccd-d2c6-4b0a-a42b-fa5da8e47aa8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:49.718459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903de61-3211-44b8-b527-e498c9a110b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:49.719166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903de61-3211-44b8-b527-e498c9a110b7 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:49.719967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903de61-3211-44b8-b527-e498c9a110b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:49.720230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903de61-3211-44b8-b527-e498c9a110b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:49.720889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903de61-3211-44b8-b527-e498c9a110b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:49.722525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903de61-3211-44b8-b527-e498c9a110b7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:49.792739 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1785/3642] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:49.805303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903de61-3211-44b8-b527-e498c9a110b7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:49.816289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d903de61-3211-44b8-b527-e498c9a110b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:49.882052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903de61-3211-44b8-b527-e498c9a110b7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:49.901996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b8ccd-d2c6-4b0a-a42b-fa5da8e47aa8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:49.904398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b8ccd-d2c6-4b0a-a42b-fa5da8e47aa8 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:49.908681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176dfa8-ce47-46f0-ad15-27e68c4b678f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:49.909004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176dfa8-ce47-46f0-ad15-27e68c4b678f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:49.909281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176dfa8-ce47-46f0-ad15-27e68c4b678f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:49.909556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176dfa8-ce47-46f0-ad15-27e68c4b678f admin admin] PATH_INFO: `/v3/users/56b4d78c7c6c450fb5fc1d3ac587fb26` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:49.917477 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1858/3643] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:58:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 20:58:49.919247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b8ccd-d2c6-4b0a-a42b-fa5da8e47aa8 service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:49.923881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176dfa8-ce47-46f0-ad15-27e68c4b678f admin admin] RBAC: Authorizing `identity:delete_user(user_id=56b4d78c7c6c450fb5fc1d3ac587fb26)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:49.925846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176dfa8-ce47-46f0-ad15-27e68c4b678f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:49.935507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b8ccd-d2c6-4b0a-a42b-fa5da8e47aa8 service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:49.951212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a092e0-db10-4ddf-a3fd-8930de33d86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:49.952634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a092e0-db10-4ddf-a3fd-8930de33d86c admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:49.953059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a092e0-db10-4ddf-a3fd-8930de33d86c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:49.953477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a092e0-db10-4ddf-a3fd-8930de33d86c admin admin] PATH_INFO: `/v3/projects/cab5a7d3943c4d148dd3e0e5855abe4f/users/38579d5d4d3641d7a706682ebe1f998f/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:49.967226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:49.987777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a092e0-db10-4ddf-a3fd-8930de33d86c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab5a7d3943c4d148dd3e0e5855abe4f, user_id=38579d5d4d3641d7a706682ebe1f998f, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:49.994133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a092e0-db10-4ddf-a3fd-8930de33d86c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:50.025540 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1859/3644] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:58:49 2022] PUT /identity/v3/projects/cab5a7d3943c4d148dd3e0e5855abe4f/users/38579d5d4d3641d7a706682ebe1f998f/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:50.030034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a092e0-db10-4ddf-a3fd-8930de33d86c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:50.097117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9176dfa8-ce47-46f0-ad15-27e68c4b678f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56b4d78c7c6c450fb5fc1d3ac587fb26'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:50.100195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9176dfa8-ce47-46f0-ad15-27e68c4b678f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56b4d78c7c6c450fb5fc1d3ac587fb26'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:50.101296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9176dfa8-ce47-46f0-ad15-27e68c4b678f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56b4d78c7c6c450fb5fc1d3ac587fb26'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:50.124843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9176dfa8-ce47-46f0-ad15-27e68c4b678f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56b4d78c7c6c450fb5fc1d3ac587fb26'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:50.181252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9176dfa8-ce47-46f0-ad15-27e68c4b678f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56b4d78c7c6c450fb5fc1d3ac587fb26'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:50.186601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a092e0-db10-4ddf-a3fd-8930de33d86c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:50.199130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cdd0b-a890-4587-b3a9-705455239c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:50.200550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cdd0b-a890-4587-b3a9-705455239c75 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:50.200929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cdd0b-a890-4587-b3a9-705455239c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:50.201984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cdd0b-a890-4587-b3a9-705455239c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:50.203204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7cdd0b-a890-4587-b3a9-705455239c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:50.207529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7cdd0b-a890-4587-b3a9-705455239c75 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:50.232899 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1860/3645] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:50.237170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cdd0b-a890-4587-b3a9-705455239c75 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:50.238936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6e7cdd0b-a890-4587-b3a9-705455239c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:50.252943 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1786/3646] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:58:49 2022] DELETE /identity/v3/users/56b4d78c7c6c450fb5fc1d3ac587fb26 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:50.256238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176dfa8-ce47-46f0-ad15-27e68c4b678f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:50.263169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9176dfa8-ce47-46f0-ad15-27e68c4b678f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:50.275573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cdd0b-a890-4587-b3a9-705455239c75 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:50.468122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a533f63-c9b3-4aef-8e74-0a774cc18985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:50.469933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a533f63-c9b3-4aef-8e74-0a774cc18985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:50.470478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a533f63-c9b3-4aef-8e74-0a774cc18985 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:50.472891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a533f63-c9b3-4aef-8e74-0a774cc18985 admin admin] PATH_INFO: `/v3/projects/6e41d13569c94212af2211abe1d8ba99` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:50.474319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a533f63-c9b3-4aef-8e74-0a774cc18985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e41d13569c94212af2211abe1d8ba99)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:50.485418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a533f63-c9b3-4aef-8e74-0a774cc18985 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:50.526924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176dfa8-ce47-46f0-ad15-27e68c4b678f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:50.554214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82fc87-2eca-4b06-a3db-0193d7c3a63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:50.554940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82fc87-2eca-4b06-a3db-0193d7c3a63a admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:50.555216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82fc87-2eca-4b06-a3db-0193d7c3a63a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:50.555489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82fc87-2eca-4b06-a3db-0193d7c3a63a admin admin] PATH_INFO: `/v3/projects/cab5a7d3943c4d148dd3e0e5855abe4f/users/38579d5d4d3641d7a706682ebe1f998f/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:50.567365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af82fc87-2eca-4b06-a3db-0193d7c3a63a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab5a7d3943c4d148dd3e0e5855abe4f, user_id=38579d5d4d3641d7a706682ebe1f998f, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:50.570089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af82fc87-2eca-4b06-a3db-0193d7c3a63a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:50.647223 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1787/3647] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:58:50 2022] PUT /identity/v3/projects/cab5a7d3943c4d148dd3e0e5855abe4f/users/38579d5d4d3641d7a706682ebe1f998f/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:50.660658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a533f63-c9b3-4aef-8e74-0a774cc18985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e41d13569c94212af2211abe1d8ba99', 'request_id': 'req-8a533f63-c9b3-4aef-8e74-0a774cc18985'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:50.665980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fea24-ab40-44f5-88d1-f87c899b9578 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:50.666581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fea24-ab40-44f5-88d1-f87c899b9578 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:50.666704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fea24-ab40-44f5-88d1-f87c899b9578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:50.683781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a533f63-c9b3-4aef-8e74-0a774cc18985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e41d13569c94212af2211abe1d8ba99', 'request_id': 'req-8a533f63-c9b3-4aef-8e74-0a774cc18985'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:50.705786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a533f63-c9b3-4aef-8e74-0a774cc18985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e41d13569c94212af2211abe1d8ba99', 'request_id': 'req-8a533f63-c9b3-4aef-8e74-0a774cc18985'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:50.744192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a533f63-c9b3-4aef-8e74-0a774cc18985 admin admin] The token cache is being invalidate because project 6e41d13569c94212af2211abe1d8ba99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:50.745996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a533f63-c9b3-4aef-8e74-0a774cc18985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:50.748973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a533f63-c9b3-4aef-8e74-0a774cc18985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:50.757022 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1861/3648] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:50 2022] DELETE /identity/v3/projects/6e41d13569c94212af2211abe1d8ba99 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:50.760682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a533f63-c9b3-4aef-8e74-0a774cc18985 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:50.763923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8a533f63-c9b3-4aef-8e74-0a774cc18985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:50.793450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ac0fea24-ab40-44f5-88d1-f87c899b9578 None None] MFA Rules not processed for user `38579d5d4d3641d7a706682ebe1f998f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:50.813961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ac0fea24-ab40-44f5-88d1-f87c899b9578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:50.867523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:50.927516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:INSERT=5,keystone:SELECT=85,keystone:DELETE=10 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:51.076321 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1788/3649] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:51.103551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744ce0a-2b19-4a10-9fb3-b4a35b22730b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:51.104668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744ce0a-2b19-4a10-9fb3-b4a35b22730b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:51.105042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744ce0a-2b19-4a10-9fb3-b4a35b22730b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:51.151979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1744ce0a-2b19-4a10-9fb3-b4a35b22730b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:51.153574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a533f63-c9b3-4aef-8e74-0a774cc18985 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:51.154303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1744ce0a-2b19-4a10-9fb3-b4a35b22730b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:51.316485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dde3bb-4921-4071-8438-ce6e97847336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:51.317943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dde3bb-4921-4071-8438-ce6e97847336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:51.318517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dde3bb-4921-4071-8438-ce6e97847336 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:51.319038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dde3bb-4921-4071-8438-ce6e97847336 admin admin] PATH_INFO: `/v3/projects/a4961525068042aba372c046eb589a7a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:51.321466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dde3bb-4921-4071-8438-ce6e97847336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4961525068042aba372c046eb589a7a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:51.323650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dde3bb-4921-4071-8438-ce6e97847336 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:51.371278 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1789/3650] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:51.379471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744ce0a-2b19-4a10-9fb3-b4a35b22730b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:51.454389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-37dde3bb-4921-4071-8438-ce6e97847336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4961525068042aba372c046eb589a7a', 'request_id': 'req-37dde3bb-4921-4071-8438-ce6e97847336'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:51.460264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744ce0a-2b19-4a10-9fb3-b4a35b22730b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:51.481521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-37dde3bb-4921-4071-8438-ce6e97847336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4961525068042aba372c046eb589a7a', 'request_id': 'req-37dde3bb-4921-4071-8438-ce6e97847336'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:51.505966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-37dde3bb-4921-4071-8438-ce6e97847336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4961525068042aba372c046eb589a7a', 'request_id': 'req-37dde3bb-4921-4071-8438-ce6e97847336'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:51.546257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-37dde3bb-4921-4071-8438-ce6e97847336 admin admin] The token cache is being invalidate because project a4961525068042aba372c046eb589a7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:51.546588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-37dde3bb-4921-4071-8438-ce6e97847336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:51.551267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-37dde3bb-4921-4071-8438-ce6e97847336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:51.558668 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1862/3651] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:50 2022] DELETE /identity/v3/projects/a4961525068042aba372c046eb589a7a => generated 0 bytes in 801 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:51.561253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be3447-0f66-45e0-9ee9-2d0b84e10406 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:51.561540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be3447-0f66-45e0-9ee9-2d0b84e10406 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:51.561826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be3447-0f66-45e0-9ee9-2d0b84e10406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:51.643799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-09be3447-0f66-45e0-9ee9-2d0b84e10406 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:51.648693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-09be3447-0f66-45e0-9ee9-2d0b84e10406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:51.726920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae47f7-5153-4f16-9b47-a283e456f77b tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:51.735621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae47f7-5153-4f16-9b47-a283e456f77b tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:51.735918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae47f7-5153-4f16-9b47-a283e456f77b tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:51.744974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae47f7-5153-4f16-9b47-a283e456f77b tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:51.744974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae47f7-5153-4f16-9b47-a283e456f77b tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:51.749608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae47f7-5153-4f16-9b47-a283e456f77b tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:51.771834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:51.795440 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1863/3652] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:51.797230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2987d-813d-4818-ae9f-8123ec29554b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:51.797704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2987d-813d-4818-ae9f-8123ec29554b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:51.798369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2987d-813d-4818-ae9f-8123ec29554b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:51.886801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-83a2987d-813d-4818-ae9f-8123ec29554b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:51.892965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-83a2987d-813d-4818-ae9f-8123ec29554b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:51.916199 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1790/3653] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:58:51 2022] POST /identity/v3/projects => generated 359 bytes in 544 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:51.916877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae47f7-5153-4f16-9b47-a283e456f77b tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:52.053401 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1864/3654] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:52.059805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2987d-813d-4818-ae9f-8123ec29554b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:52.063232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-83a2987d-813d-4818-ae9f-8123ec29554b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:52.106326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:52.134197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae47f7-5153-4f16-9b47-a283e456f77b tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:52.142289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c04aff-5494-471f-9cd3-0ad108555caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:52.143058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c04aff-5494-471f-9cd3-0ad108555caa admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:52.143409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c04aff-5494-471f-9cd3-0ad108555caa admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:52.143684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c04aff-5494-471f-9cd3-0ad108555caa admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:52.144495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c04aff-5494-471f-9cd3-0ad108555caa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:58:52.144819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c04aff-5494-471f-9cd3-0ad108555caa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:52.148832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c04aff-5494-471f-9cd3-0ad108555caa admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:52.176246 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1791/3655] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:52.178078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca3221-a279-4bb4-9dc5-df41e60c23d2 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:52.179373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca3221-a279-4bb4-9dc5-df41e60c23d2 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:52.179661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca3221-a279-4bb4-9dc5-df41e60c23d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:52.230914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-53ca3221-a279-4bb4-9dc5-df41e60c23d2 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:52.233004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-53ca3221-a279-4bb4-9dc5-df41e60c23d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:52.260344 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1792/3656] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:52.261309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca3221-a279-4bb4-9dc5-df41e60c23d2 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:52.288848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca3221-a279-4bb4-9dc5-df41e60c23d2 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:52.306324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621c09c-6bfe-47d5-8a06-c8bd4f6a3552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:52.307499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621c09c-6bfe-47d5-8a06-c8bd4f6a3552 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:52.307782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621c09c-6bfe-47d5-8a06-c8bd4f6a3552 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:52.308045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621c09c-6bfe-47d5-8a06-c8bd4f6a3552 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:52.308679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621c09c-6bfe-47d5-8a06-c8bd4f6a3552 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:52.311252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621c09c-6bfe-47d5-8a06-c8bd4f6a3552 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:52.399050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2987d-813d-4818-ae9f-8123ec29554b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:52.420814 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1793/3657] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:58:52 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:52.425311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621c09c-6bfe-47d5-8a06-c8bd4f6a3552 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:52.428833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ff3bf-d72a-4c83-9afc-e39fa7fd8243 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:52.430683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ff3bf-d72a-4c83-9afc-e39fa7fd8243 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:52.430978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ff3bf-d72a-4c83-9afc-e39fa7fd8243 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:52.442540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ff3bf-d72a-4c83-9afc-e39fa7fd8243 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:52.443811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ff3bf-d72a-4c83-9afc-e39fa7fd8243 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:52.452270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ff3bf-d72a-4c83-9afc-e39fa7fd8243 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:52.558107 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1865/3658] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:58:52 2022] POST /identity/v3/users => generated 371 bytes in 504 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:52.560451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ff3bf-d72a-4c83-9afc-e39fa7fd8243 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:52.591697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ff3bf-d72a-4c83-9afc-e39fa7fd8243 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:52.732126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621c09c-6bfe-47d5-8a06-c8bd4f6a3552 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:52.764719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89566a7f-1f34-4ef7-ad15-44f561135f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:52.768949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b5193-6824-41d3-ac86-760f787402b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:52.770953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b5193-6824-41d3-ac86-760f787402b3 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:52.772184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b5193-6824-41d3-ac86-760f787402b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:52.772677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b5193-6824-41d3-ac86-760f787402b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:52.773711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b5193-6824-41d3-ac86-760f787402b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:58:52.774169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b5193-6824-41d3-ac86-760f787402b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:52.778293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89566a7f-1f34-4ef7-ad15-44f561135f6b admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:52.779113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89566a7f-1f34-4ef7-ad15-44f561135f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:52.780268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b5193-6824-41d3-ac86-760f787402b3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:52.781183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89566a7f-1f34-4ef7-ad15-44f561135f6b admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:52.785481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89566a7f-1f34-4ef7-ad15-44f561135f6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:52.805343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89566a7f-1f34-4ef7-ad15-44f561135f6b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:52.838887 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1794/3659] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:52.844787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b5193-6824-41d3-ac86-760f787402b3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:52.861728 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1866/3660] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:58:52 2022] POST /identity/v3/users => generated 293 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:52.862890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89566a7f-1f34-4ef7-ad15-44f561135f6b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:52.866155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-89566a7f-1f34-4ef7-ad15-44f561135f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:52.906663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:53.061331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89566a7f-1f34-4ef7-ad15-44f561135f6b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:53.066678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79d34c-d978-4c37-aee6-6ff6c35b22de service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:53.067506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79d34c-d978-4c37-aee6-6ff6c35b22de service glance] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:53.067782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79d34c-d978-4c37-aee6-6ff6c35b22de service glance] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:53.068059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79d34c-d978-4c37-aee6-6ff6c35b22de service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:53.082973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b5193-6824-41d3-ac86-760f787402b3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:53.091832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4e79d34c-d978-4c37-aee6-6ff6c35b22de service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:53.095421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1541e33-8ad1-4a6b-9c22-0b698b2fba2f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:53.096254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1541e33-8ad1-4a6b-9c22-0b698b2fba2f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:53.097194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1541e33-8ad1-4a6b-9c22-0b698b2fba2f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:53.098603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1541e33-8ad1-4a6b-9c22-0b698b2fba2f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/users/2158df0bb4144f3eb39cc0aaf83e865e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:53.133168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1541e33-8ad1-4a6b-9c22-0b698b2fba2f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:get_user(user_id=2158df0bb4144f3eb39cc0aaf83e865e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:53.136321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1541e33-8ad1-4a6b-9c22-0b698b2fba2f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:53.140276 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1795/3661] 10.222.0.17 () {58 vars in 1221 bytes} [Sat Oct 1 20:58:52 2022] GET /identity/v3/users/2158df0bb4144f3eb39cc0aaf83e865e => generated 371 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:53.143945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1541e33-8ad1-4a6b-9c22-0b698b2fba2f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:53.181597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1541e33-8ad1-4a6b-9c22-0b698b2fba2f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:53.197785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24959e3f-0b4e-4f44-be0d-404d412b63e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:53.198571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24959e3f-0b4e-4f44-be0d-404d412b63e0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:53.198899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24959e3f-0b4e-4f44-be0d-404d412b63e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:53.199725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24959e3f-0b4e-4f44-be0d-404d412b63e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:53.200365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24959e3f-0b4e-4f44-be0d-404d412b63e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:53.204456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24959e3f-0b4e-4f44-be0d-404d412b63e0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:53.254653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79d34c-d978-4c37-aee6-6ff6c35b22de service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:53.256596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79d34c-d978-4c37-aee6-6ff6c35b22de service glance] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:53.269954 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1796/3662] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:58:53 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:53.272669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24959e3f-0b4e-4f44-be0d-404d412b63e0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:53.329128 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1867/3663] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:58:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:58:53.338772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79d34c-d978-4c37-aee6-6ff6c35b22de service glance] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:53.379897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79d34c-d978-4c37-aee6-6ff6c35b22de service glance] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:53.403733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:53.493442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24959e3f-0b4e-4f44-be0d-404d412b63e0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:53.507088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7717b-d370-440b-9757-fb181a999a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:53.509013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7717b-d370-440b-9757-fb181a999a5c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:53.509303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7717b-d370-440b-9757-fb181a999a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:53.509580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7717b-d370-440b-9757-fb181a999a5c admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:53.510240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba7717b-d370-440b-9757-fb181a999a5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:53.516593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba7717b-d370-440b-9757-fb181a999a5c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:53.546211 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1797/3664] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:53.551620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7717b-d370-440b-9757-fb181a999a5c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:53.571770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7717b-d370-440b-9757-fb181a999a5c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:53.586648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d64a7-4878-4b2d-836d-bff087af2e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:53.587349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d64a7-4878-4b2d-836d-bff087af2e24 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:53.587634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d64a7-4878-4b2d-836d-bff087af2e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:53.587913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d64a7-4878-4b2d-836d-bff087af2e24 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:53.588663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d64a7-4878-4b2d-836d-bff087af2e24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:53.590541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d64a7-4878-4b2d-836d-bff087af2e24 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:53.602197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5972c5-ebe6-48ac-bed2-70feec3542ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:53.603035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5972c5-ebe6-48ac-bed2-70feec3542ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:53.603266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5972c5-ebe6-48ac-bed2-70feec3542ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:53.603553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5972c5-ebe6-48ac-bed2-70feec3542ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/users/2158df0bb4144f3eb39cc0aaf83e865e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:53.604970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5972c5-ebe6-48ac-bed2-70feec3542ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:delete_user(user_id=2158df0bb4144f3eb39cc0aaf83e865e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:53.608572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5972c5-ebe6-48ac-bed2-70feec3542ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:53.650674 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1798/3665] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:58:53 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:53.650674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d64a7-4878-4b2d-836d-bff087af2e24 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:53.672290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d64a7-4878-4b2d-836d-bff087af2e24 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:53.691860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fe755-1956-49a6-a6af-466c62bc8efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:53.692323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fe755-1956-49a6-a6af-466c62bc8efe admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:53.692599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fe755-1956-49a6-a6af-466c62bc8efe admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:53.692875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fe755-1956-49a6-a6af-466c62bc8efe admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:53.693518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fe755-1956-49a6-a6af-466c62bc8efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:53.695360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fe755-1956-49a6-a6af-466c62bc8efe admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:53.706115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:53.709298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a5972c5-ebe6-48ac-bed2-70feec3542ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2158df0bb4144f3eb39cc0aaf83e865e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:53.724369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a5972c5-ebe6-48ac-bed2-70feec3542ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2158df0bb4144f3eb39cc0aaf83e865e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:53.728166 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1799/3666] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:53.729447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fe755-1956-49a6-a6af-466c62bc8efe admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:53.731648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b78fe755-1956-49a6-a6af-466c62bc8efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:53.757348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a5972c5-ebe6-48ac-bed2-70feec3542ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2158df0bb4144f3eb39cc0aaf83e865e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:53.758258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a5972c5-ebe6-48ac-bed2-70feec3542ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2158df0bb4144f3eb39cc0aaf83e865e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:53.775148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a5972c5-ebe6-48ac-bed2-70feec3542ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2158df0bb4144f3eb39cc0aaf83e865e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:53.783124 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1868/3667] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:58:53 2022] DELETE /identity/v3/users/2158df0bb4144f3eb39cc0aaf83e865e => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:53.783996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5972c5-ebe6-48ac-bed2-70feec3542ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:53.788042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9a5972c5-ebe6-48ac-bed2-70feec3542ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:53.949924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fe755-1956-49a6-a6af-466c62bc8efe admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:54.007379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5972c5-ebe6-48ac-bed2-70feec3542ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:54.018082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610ea1c-f0e8-4ce3-a62a-046ed7d2cf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:54.019044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610ea1c-f0e8-4ce3-a62a-046ed7d2cf77 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:54.019892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610ea1c-f0e8-4ce3-a62a-046ed7d2cf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:54.019892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610ea1c-f0e8-4ce3-a62a-046ed7d2cf77 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:54.020239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610ea1c-f0e8-4ce3-a62a-046ed7d2cf77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:54.024062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610ea1c-f0e8-4ce3-a62a-046ed7d2cf77 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:54.066793 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1869/3668] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:54.069807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610ea1c-f0e8-4ce3-a62a-046ed7d2cf77 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:54.073094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a610ea1c-f0e8-4ce3-a62a-046ed7d2cf77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:54.107222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610ea1c-f0e8-4ce3-a62a-046ed7d2cf77 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:54.120288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf45b88-94ca-4341-a1c7-af871dc15759 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:54.121190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf45b88-94ca-4341-a1c7-af871dc15759 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:54.121543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62ca72-ace2-4378-bd3c-8f9971beb42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:54.121803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf45b88-94ca-4341-a1c7-af871dc15759 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:54.122224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62ca72-ace2-4378-bd3c-8f9971beb42f admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:54.122411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf45b88-94ca-4341-a1c7-af871dc15759 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:54.122537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62ca72-ace2-4378-bd3c-8f9971beb42f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:54.122859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62ca72-ace2-4378-bd3c-8f9971beb42f admin admin] PATH_INFO: `/v3/projects/d27e9cda0a6d47d4acdbadc890b996c9/users/69002d46d4e54eadadf911ec3589d95c/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:54.124681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2bf45b88-94ca-4341-a1c7-af871dc15759 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:54.159189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62ca72-ace2-4378-bd3c-8f9971beb42f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d27e9cda0a6d47d4acdbadc890b996c9, user_id=69002d46d4e54eadadf911ec3589d95c, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:54.161759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62ca72-ace2-4378-bd3c-8f9971beb42f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:54.182316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:54.221025 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1870/3669] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:58:54 2022] PUT /identity/v3/projects/d27e9cda0a6d47d4acdbadc890b996c9/users/69002d46d4e54eadadf911ec3589d95c/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:54.223193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62ca72-ace2-4378-bd3c-8f9971beb42f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:54.227250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6e62ca72-ace2-4378-bd3c-8f9971beb42f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:54.268551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf45b88-94ca-4341-a1c7-af871dc15759 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:54.270410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf45b88-94ca-4341-a1c7-af871dc15759 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:54.301997 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1800/3670] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:58:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:58:54.305461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf45b88-94ca-4341-a1c7-af871dc15759 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:54.330264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf45b88-94ca-4341-a1c7-af871dc15759 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:54.467265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62ca72-ace2-4378-bd3c-8f9971beb42f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:54.486527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433b11f-e9b9-4280-a01b-c42745fdbda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:54.487356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433b11f-e9b9-4280-a01b-c42745fdbda9 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:54.487356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433b11f-e9b9-4280-a01b-c42745fdbda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:54.487823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433b11f-e9b9-4280-a01b-c42745fdbda9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:54.488914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433b11f-e9b9-4280-a01b-c42745fdbda9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:54.492553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852db99-0578-40a3-9785-825b2328fda3 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:54.494437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433b11f-e9b9-4280-a01b-c42745fdbda9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:54.495876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852db99-0578-40a3-9785-825b2328fda3 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:54.496064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852db99-0578-40a3-9785-825b2328fda3 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:54.496354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852db99-0578-40a3-9785-825b2328fda3 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/projects/407f5dcfa019484f82b2741142248050` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:54.505591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852db99-0578-40a3-9785-825b2328fda3 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:delete_project(project_id=407f5dcfa019484f82b2741142248050)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:54.515284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852db99-0578-40a3-9785-825b2328fda3 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:54.534193 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1801/3671] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:58:54.535400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3d0b7-eb65-4f37-a6d9-0222f107bd49 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:54.537399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3d0b7-eb65-4f37-a6d9-0222f107bd49 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:54.537806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3d0b7-eb65-4f37-a6d9-0222f107bd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:54.597813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3fb3d0b7-eb65-4f37-a6d9-0222f107bd49 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:54.599083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3fb3d0b7-eb65-4f37-a6d9-0222f107bd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:54.623301 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1802/3672] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:54.624227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3d0b7-eb65-4f37-a6d9-0222f107bd49 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:54.644347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3d0b7-eb65-4f37-a6d9-0222f107bd49 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:54.648810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:54.673345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2852db99-0578-40a3-9785-825b2328fda3 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '407f5dcfa019484f82b2741142248050', 'request_id': 'req-2852db99-0578-40a3-9785-825b2328fda3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:54.695859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2852db99-0578-40a3-9785-825b2328fda3 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '407f5dcfa019484f82b2741142248050', 'request_id': 'req-2852db99-0578-40a3-9785-825b2328fda3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:54.702285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2852db99-0578-40a3-9785-825b2328fda3 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '407f5dcfa019484f82b2741142248050', 'request_id': 'req-2852db99-0578-40a3-9785-825b2328fda3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:54.725459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2852db99-0578-40a3-9785-825b2328fda3 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] The token cache is being invalidate because project 407f5dcfa019484f82b2741142248050 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:54.725845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2852db99-0578-40a3-9785-825b2328fda3 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:54.736513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2852db99-0578-40a3-9785-825b2328fda3 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:54.740930 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1871/3673] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:54 2022] DELETE /identity/v3/projects/407f5dcfa019484f82b2741142248050 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:54.742022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852db99-0578-40a3-9785-825b2328fda3 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:54.743486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2852db99-0578-40a3-9785-825b2328fda3 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:54.793942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678eda86-4e33-489e-9f21-8ded3bf1bca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:54.794671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678eda86-4e33-489e-9f21-8ded3bf1bca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:54.795042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678eda86-4e33-489e-9f21-8ded3bf1bca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:54.795327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678eda86-4e33-489e-9f21-8ded3bf1bca9 admin admin] PATH_INFO: `/v3/projects/9566fa9a44f94c4ea3081a1a2e658de0/users/e84a109e337d450698e630acfb49f9a5/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:54.842946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678eda86-4e33-489e-9f21-8ded3bf1bca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9566fa9a44f94c4ea3081a1a2e658de0, user_id=e84a109e337d450698e630acfb49f9a5, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:54.851063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678eda86-4e33-489e-9f21-8ded3bf1bca9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:54.897387 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1803/3674] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:58:54 2022] PUT /identity/v3/projects/9566fa9a44f94c4ea3081a1a2e658de0/users/e84a109e337d450698e630acfb49f9a5/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:54.898438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678eda86-4e33-489e-9f21-8ded3bf1bca9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:54.902125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-678eda86-4e33-489e-9f21-8ded3bf1bca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:54.960354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852db99-0578-40a3-9785-825b2328fda3 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:55.013094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:55.089883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678eda86-4e33-489e-9f21-8ded3bf1bca9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:55.095026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d47173-8c41-41c3-a371-f7631627e0e7 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:55.095884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d47173-8c41-41c3-a371-f7631627e0e7 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:55.096367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d47173-8c41-41c3-a371-f7631627e0e7 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:55.096978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d47173-8c41-41c3-a371-f7631627e0e7 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:55.097969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d47173-8c41-41c3-a371-f7631627e0e7 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:55.101828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d47173-8c41-41c3-a371-f7631627e0e7 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:55.108570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941283e-ec85-4fc3-8b82-c1e29c6a42f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:55.109480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941283e-ec85-4fc3-8b82-c1e29c6a42f7 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:55.110783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941283e-ec85-4fc3-8b82-c1e29c6a42f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:55.111506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941283e-ec85-4fc3-8b82-c1e29c6a42f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:55.113606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4941283e-ec85-4fc3-8b82-c1e29c6a42f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:55.187790 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1872/3675] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:58:54 2022] POST /identity/v3/projects => generated 358 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:55.189409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d0c63-ed8e-481b-8c64-489d24d37642 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:55.189840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d0c63-ed8e-481b-8c64-489d24d37642 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:55.190183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d0c63-ed8e-481b-8c64-489d24d37642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:55.253130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a89d0c63-ed8e-481b-8c64-489d24d37642 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:55.255430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a89d0c63-ed8e-481b-8c64-489d24d37642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:55.318689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941283e-ec85-4fc3-8b82-c1e29c6a42f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:55.320416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941283e-ec85-4fc3-8b82-c1e29c6a42f7 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:55.336198 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1804/3676] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:58:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:58:55.337350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941283e-ec85-4fc3-8b82-c1e29c6a42f7 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:55.354768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:55.358270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941283e-ec85-4fc3-8b82-c1e29c6a42f7 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:55.377473 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1873/3677] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:58:55.388780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d0c63-ed8e-481b-8c64-489d24d37642 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:55.394630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a89d0c63-ed8e-481b-8c64-489d24d37642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:55.467242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631a3e3-5330-4d2e-b1c7-be2202b3332f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:55.468599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631a3e3-5330-4d2e-b1c7-be2202b3332f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:55.468958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631a3e3-5330-4d2e-b1c7-be2202b3332f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:55.469865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631a3e3-5330-4d2e-b1c7-be2202b3332f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:55.470629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5631a3e3-5330-4d2e-b1c7-be2202b3332f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:55.472591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5631a3e3-5330-4d2e-b1c7-be2202b3332f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:55.483159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d0c63-ed8e-481b-8c64-489d24d37642 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:55.488372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e8c04b-3216-4bbe-8c6d-47e3c414d25f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:55.489427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8c04b-3216-4bbe-8c6d-47e3c414d25f service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:55.489912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8c04b-3216-4bbe-8c6d-47e3c414d25f service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:55.490904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8c04b-3216-4bbe-8c6d-47e3c414d25f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:55.506350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e8c04b-3216-4bbe-8c6d-47e3c414d25f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:55.510194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e8c04b-3216-4bbe-8c6d-47e3c414d25f service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:55.524023 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1805/3678] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:58:55 2022] POST /identity/v3/users => generated 414 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:55.532905 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1874/3679] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:58:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:58:55.545552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631a3e3-5330-4d2e-b1c7-be2202b3332f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:55.568577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631a3e3-5330-4d2e-b1c7-be2202b3332f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:55.597703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c8232-2433-45ac-a5f0-9649c5042aaf tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:55.599703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c8232-2433-45ac-a5f0-9649c5042aaf tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:55.600343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c8232-2433-45ac-a5f0-9649c5042aaf tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:55.600921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c8232-2433-45ac-a5f0-9649c5042aaf tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:55.602100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c8232-2433-45ac-a5f0-9649c5042aaf tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:55.604672 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-ad1c8232-2433-45ac-a5f0-9649c5042aaf tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:58:55.604672 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:58:55.604672 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:58:55.607057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c8232-2433-45ac-a5f0-9649c5042aaf tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:55.643151 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1806/3680] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 20:58:55 2022] POST /identity/v3/roles => generated 240 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:55.646895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e8c04b-3216-4bbe-8c6d-47e3c414d25f service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:55.664205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e8c04b-3216-4bbe-8c6d-47e3c414d25f service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:55.675703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c82c6-b56c-49ce-96ab-5272a19a61a1 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:55.676378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c82c6-b56c-49ce-96ab-5272a19a61a1 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:55.676668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c82c6-b56c-49ce-96ab-5272a19a61a1 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:55.676960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c82c6-b56c-49ce-96ab-5272a19a61a1 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/users/c38321f1403f4772ad455b42c9f40115` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:55.731936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c82c6-b56c-49ce-96ab-5272a19a61a1 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:get_user(user_id=c38321f1403f4772ad455b42c9f40115)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:55.734782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c82c6-b56c-49ce-96ab-5272a19a61a1 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:55.740523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c8232-2433-45ac-a5f0-9649c5042aaf tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:55.744382 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1875/3681] 10.222.0.17 () {58 vars in 1221 bytes} [Sat Oct 1 20:58:55 2022] GET /identity/v3/users/c38321f1403f4772ad455b42c9f40115 => generated 414 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:55.753276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c8232-2433-45ac-a5f0-9649c5042aaf tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:55.768359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d9542-032d-4b28-b2ec-1bc88ee70797 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:55.769102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d9542-032d-4b28-b2ec-1bc88ee70797 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:55.769394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d9542-032d-4b28-b2ec-1bc88ee70797 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:55.769675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d9542-032d-4b28-b2ec-1bc88ee70797 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/roles/0b49578908a04b8aa5655bf2b9258139` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:55.783839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d9542-032d-4b28-b2ec-1bc88ee70797 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:get_role(role_id=0b49578908a04b8aa5655bf2b9258139)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:55.784880 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-bd8d9542-032d-4b28-b2ec-1bc88ee70797 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:58:55.784880 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:58:55.784880 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:58:55.785792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d9542-032d-4b28-b2ec-1bc88ee70797 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:55.790803 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1807/3682] 10.222.0.17 () {58 vars in 1221 bytes} [Sat Oct 1 20:58:55 2022] GET /identity/v3/roles/0b49578908a04b8aa5655bf2b9258139 => generated 240 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:55.791801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c82c6-b56c-49ce-96ab-5272a19a61a1 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:55.807511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c82c6-b56c-49ce-96ab-5272a19a61a1 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:55.823974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152b9d3-ba34-463b-bf7f-a03f6ef2e776 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:55.825307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152b9d3-ba34-463b-bf7f-a03f6ef2e776 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:55.827096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152b9d3-ba34-463b-bf7f-a03f6ef2e776 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:55.827641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152b9d3-ba34-463b-bf7f-a03f6ef2e776 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:55.829314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152b9d3-ba34-463b-bf7f-a03f6ef2e776 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:55.837599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152b9d3-ba34-463b-bf7f-a03f6ef2e776 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:55.933149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d9542-032d-4b28-b2ec-1bc88ee70797 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:55.933443 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1876/3683] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:58:55 2022] POST /identity/v3/projects => generated 358 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:58:55.951017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d9542-032d-4b28-b2ec-1bc88ee70797 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:56.098062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:56.111394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8dcc69-4faf-47eb-87c2-c2bf7969d729 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:56.112487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8dcc69-4faf-47eb-87c2-c2bf7969d729 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:56.112973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8dcc69-4faf-47eb-87c2-c2bf7969d729 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:56.113446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8dcc69-4faf-47eb-87c2-c2bf7969d729 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/projects/2a55907b23bb406d8a4610d9bea7359c` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:56.116522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8dcc69-4faf-47eb-87c2-c2bf7969d729 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:get_project(project_id=2a55907b23bb406d8a4610d9bea7359c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:56.118874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8dcc69-4faf-47eb-87c2-c2bf7969d729 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:56.128832 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1808/3684] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 20:58:55 2022] GET /identity/v3/projects/2a55907b23bb406d8a4610d9bea7359c => generated 358 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:56.129958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152b9d3-ba34-463b-bf7f-a03f6ef2e776 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:56.149332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152b9d3-ba34-463b-bf7f-a03f6ef2e776 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:56.159158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101995f-2891-4881-ae65-b0bd02b73a51 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:56.159849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101995f-2891-4881-ae65-b0bd02b73a51 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:56.160134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101995f-2891-4881-ae65-b0bd02b73a51 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:56.160412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101995f-2891-4881-ae65-b0bd02b73a51 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/projects/2a55907b23bb406d8a4610d9bea7359c/users/c38321f1403f4772ad455b42c9f40115/roles/0b49578908a04b8aa5655bf2b9258139` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:56.168871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101995f-2891-4881-ae65-b0bd02b73a51 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:create_grant(project_id=2a55907b23bb406d8a4610d9bea7359c, user_id=c38321f1403f4772ad455b42c9f40115, role_id=0b49578908a04b8aa5655bf2b9258139)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:56.171794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101995f-2891-4881-ae65-b0bd02b73a51 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:56.210013 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1877/3685] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:58:56 2022] PUT /identity/v3/projects/2a55907b23bb406d8a4610d9bea7359c/users/c38321f1403f4772ad455b42c9f40115/roles/0b49578908a04b8aa5655bf2b9258139 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:56.212177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8dcc69-4faf-47eb-87c2-c2bf7969d729 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:56.231475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8dcc69-4faf-47eb-87c2-c2bf7969d729 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:56.407847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c19953-be14-4869-a63d-ef48218412f5 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:56.408605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c19953-be14-4869-a63d-ef48218412f5 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:56.408978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c19953-be14-4869-a63d-ef48218412f5 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:56.409348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c19953-be14-4869-a63d-ef48218412f5 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:56.410043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c19953-be14-4869-a63d-ef48218412f5 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:56.414196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c19953-be14-4869-a63d-ef48218412f5 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:56.496953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101995f-2891-4881-ae65-b0bd02b73a51 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:56.498032 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1809/3686] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:58:56 2022] POST /identity/v3/projects => generated 358 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:58:56.513552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101995f-2891-4881-ae65-b0bd02b73a51 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:56.697250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8d6b2-af6e-466e-ba6c-75d80aad69e9 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:56.698253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8d6b2-af6e-466e-ba6c-75d80aad69e9 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:56.702654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8d6b2-af6e-466e-ba6c-75d80aad69e9 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:56.704181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8d6b2-af6e-466e-ba6c-75d80aad69e9 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/projects/1f5499785f4c4462b140fa65808ad688` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:56.705858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac8d6b2-af6e-466e-ba6c-75d80aad69e9 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:get_project(project_id=1f5499785f4c4462b140fa65808ad688)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:56.711960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac8d6b2-af6e-466e-ba6c-75d80aad69e9 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:56.718643 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1878/3687] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 20:58:56 2022] GET /identity/v3/projects/1f5499785f4c4462b140fa65808ad688 => generated 358 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:58:56.720266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c19953-be14-4869-a63d-ef48218412f5 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:56.731445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c19953-be14-4869-a63d-ef48218412f5 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:56.754530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5500c-d2c9-4aac-9570-8a896d083ae8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:56.757338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5500c-d2c9-4aac-9570-8a896d083ae8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:56.757629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5500c-d2c9-4aac-9570-8a896d083ae8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:56.758306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5500c-d2c9-4aac-9570-8a896d083ae8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/projects/1f5499785f4c4462b140fa65808ad688/users/c38321f1403f4772ad455b42c9f40115/roles/0b49578908a04b8aa5655bf2b9258139` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:56.767518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5500c-d2c9-4aac-9570-8a896d083ae8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:create_grant(project_id=1f5499785f4c4462b140fa65808ad688, user_id=c38321f1403f4772ad455b42c9f40115, role_id=0b49578908a04b8aa5655bf2b9258139)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:56.774892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5500c-d2c9-4aac-9570-8a896d083ae8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:56.815537 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1810/3688] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:58:56 2022] PUT /identity/v3/projects/1f5499785f4c4462b140fa65808ad688/users/c38321f1403f4772ad455b42c9f40115/roles/0b49578908a04b8aa5655bf2b9258139 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:56.818327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8d6b2-af6e-466e-ba6c-75d80aad69e9 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:56.829555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8d6b2-af6e-466e-ba6c-75d80aad69e9 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:56.970119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:56.993820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7f655-7b7d-49f4-9829-f76aba03fcda tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:56.994554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7f655-7b7d-49f4-9829-f76aba03fcda tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:56.995309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7f655-7b7d-49f4-9829-f76aba03fcda tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:56.995309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7f655-7b7d-49f4-9829-f76aba03fcda tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/users/c38321f1403f4772ad455b42c9f40115/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:56.997554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7f655-7b7d-49f4-9829-f76aba03fcda tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:list_user_projects(user_id=c38321f1403f4772ad455b42c9f40115)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:56.999772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7f655-7b7d-49f4-9829-f76aba03fcda tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:57.037514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5500c-d2c9-4aac-9570-8a896d083ae8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:57.115409 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1879/3689] 10.222.0.17 () {58 vars in 1248 bytes} [Sat Oct 1 20:58:56 2022] GET /identity/v3/users/c38321f1403f4772ad455b42c9f40115/projects => generated 840 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:57.116400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7f655-7b7d-49f4-9829-f76aba03fcda tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:57.136177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7f655-7b7d-49f4-9829-f76aba03fcda tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:57.146904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f68fe-77e9-442e-a04b-fa97a6e3aebb tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:57.147671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f68fe-77e9-442e-a04b-fa97a6e3aebb tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:57.148029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f68fe-77e9-442e-a04b-fa97a6e3aebb tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:57.148371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f68fe-77e9-442e-a04b-fa97a6e3aebb tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/projects/1f5499785f4c4462b140fa65808ad688` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:57.159561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f68fe-77e9-442e-a04b-fa97a6e3aebb tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1f5499785f4c4462b140fa65808ad688)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:57.161661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f68fe-77e9-442e-a04b-fa97a6e3aebb tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:57.225450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5500c-d2c9-4aac-9570-8a896d083ae8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:57.229445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773607f1-7026-4ce5-8e24-fb3655a4e781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:57.230800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773607f1-7026-4ce5-8e24-fb3655a4e781 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:57.231100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773607f1-7026-4ce5-8e24-fb3655a4e781 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:57.231368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773607f1-7026-4ce5-8e24-fb3655a4e781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:57.233809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-773607f1-7026-4ce5-8e24-fb3655a4e781 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:57.263406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f38f68fe-77e9-442e-a04b-fa97a6e3aebb tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f5499785f4c4462b140fa65808ad688', 'request_id': 'req-f38f68fe-77e9-442e-a04b-fa97a6e3aebb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:57.279825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f38f68fe-77e9-442e-a04b-fa97a6e3aebb tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f5499785f4c4462b140fa65808ad688', 'request_id': 'req-f38f68fe-77e9-442e-a04b-fa97a6e3aebb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:57.290649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f38f68fe-77e9-442e-a04b-fa97a6e3aebb tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f5499785f4c4462b140fa65808ad688', 'request_id': 'req-f38f68fe-77e9-442e-a04b-fa97a6e3aebb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:57.322546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f38f68fe-77e9-442e-a04b-fa97a6e3aebb tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] The token cache is being invalidate because project 1f5499785f4c4462b140fa65808ad688 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:57.323551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f38f68fe-77e9-442e-a04b-fa97a6e3aebb tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:57.329509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f38f68fe-77e9-442e-a04b-fa97a6e3aebb tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:57.334205 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1880/3690] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:57 2022] DELETE /identity/v3/projects/1f5499785f4c4462b140fa65808ad688 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:57.339519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f68fe-77e9-442e-a04b-fa97a6e3aebb tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:57.342696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f38f68fe-77e9-442e-a04b-fa97a6e3aebb tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:57.386048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773607f1-7026-4ce5-8e24-fb3655a4e781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:57.389483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773607f1-7026-4ce5-8e24-fb3655a4e781 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:57.423803 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1811/3691] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:58:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:58:57.507605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f68fe-77e9-442e-a04b-fa97a6e3aebb tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:57.522337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab642c7-2d54-4dd3-b407-c9267bc0c4ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:57.523204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab642c7-2d54-4dd3-b407-c9267bc0c4ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:57.524568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab642c7-2d54-4dd3-b407-c9267bc0c4ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:57.524928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab642c7-2d54-4dd3-b407-c9267bc0c4ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/projects/2a55907b23bb406d8a4610d9bea7359c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:57.527051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab642c7-2d54-4dd3-b407-c9267bc0c4ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2a55907b23bb406d8a4610d9bea7359c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:57.532024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab642c7-2d54-4dd3-b407-c9267bc0c4ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:57.666742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:57.676491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fab642c7-2d54-4dd3-b407-c9267bc0c4ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a55907b23bb406d8a4610d9bea7359c', 'request_id': 'req-fab642c7-2d54-4dd3-b407-c9267bc0c4ec'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:57.697027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fab642c7-2d54-4dd3-b407-c9267bc0c4ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a55907b23bb406d8a4610d9bea7359c', 'request_id': 'req-fab642c7-2d54-4dd3-b407-c9267bc0c4ec'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:57.714691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fab642c7-2d54-4dd3-b407-c9267bc0c4ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a55907b23bb406d8a4610d9bea7359c', 'request_id': 'req-fab642c7-2d54-4dd3-b407-c9267bc0c4ec'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:57.737285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fab642c7-2d54-4dd3-b407-c9267bc0c4ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] The token cache is being invalidate because project 2a55907b23bb406d8a4610d9bea7359c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:57.738069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fab642c7-2d54-4dd3-b407-c9267bc0c4ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:57.740137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fab642c7-2d54-4dd3-b407-c9267bc0c4ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:57.742522 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1881/3692] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:57 2022] DELETE /identity/v3/projects/2a55907b23bb406d8a4610d9bea7359c => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:57.746300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773607f1-7026-4ce5-8e24-fb3655a4e781 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:57.762751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773607f1-7026-4ce5-8e24-fb3655a4e781 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:57.773970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:57.896563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073878b9-bd65-44ef-9687-0eb3be61d1a8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:57.897934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073878b9-bd65-44ef-9687-0eb3be61d1a8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:57.899046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073878b9-bd65-44ef-9687-0eb3be61d1a8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:57.899797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073878b9-bd65-44ef-9687-0eb3be61d1a8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/roles/0b49578908a04b8aa5655bf2b9258139` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:57.902793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073878b9-bd65-44ef-9687-0eb3be61d1a8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:delete_role(role_id=0b49578908a04b8aa5655bf2b9258139)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:57.904803 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-073878b9-bd65-44ef-9687-0eb3be61d1a8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:58:57.904803 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:58:57.904803 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:58:57.906037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073878b9-bd65-44ef-9687-0eb3be61d1a8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:58.001836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-073878b9-bd65-44ef-9687-0eb3be61d1a8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invalidating the token cache because role 0b49578908a04b8aa5655bf2b9258139 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:58.002130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-073878b9-bd65-44ef-9687-0eb3be61d1a8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:58.003178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-073878b9-bd65-44ef-9687-0eb3be61d1a8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:58.007188 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1812/3693] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:58:57 2022] DELETE /identity/v3/roles/0b49578908a04b8aa5655bf2b9258139 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:58.012183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab642c7-2d54-4dd3-b407-c9267bc0c4ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:58.014834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fab642c7-2d54-4dd3-b407-c9267bc0c4ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:58.065992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab642c7-2d54-4dd3-b407-c9267bc0c4ec tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:58.165826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d2569-37c9-40f5-aae2-31b7d0e0a66d tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:58.169284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d2569-37c9-40f5-aae2-31b7d0e0a66d tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:58.169959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d2569-37c9-40f5-aae2-31b7d0e0a66d tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:58.170853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d2569-37c9-40f5-aae2-31b7d0e0a66d tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/users/c38321f1403f4772ad455b42c9f40115` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:58.173938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d2569-37c9-40f5-aae2-31b7d0e0a66d tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:delete_user(user_id=c38321f1403f4772ad455b42c9f40115)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:58.177817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d2569-37c9-40f5-aae2-31b7d0e0a66d tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:58.278511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-543d2569-37c9-40f5-aae2-31b7d0e0a66d tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c38321f1403f4772ad455b42c9f40115'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:58.287306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-543d2569-37c9-40f5-aae2-31b7d0e0a66d tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c38321f1403f4772ad455b42c9f40115'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:58.314887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-543d2569-37c9-40f5-aae2-31b7d0e0a66d tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c38321f1403f4772ad455b42c9f40115'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:58.317594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-543d2569-37c9-40f5-aae2-31b7d0e0a66d tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c38321f1403f4772ad455b42c9f40115'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:58.355265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-543d2569-37c9-40f5-aae2-31b7d0e0a66d tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c38321f1403f4772ad455b42c9f40115'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:58.370332 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1882/3694] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:58:58 2022] DELETE /identity/v3/users/c38321f1403f4772ad455b42c9f40115 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:58.379611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073878b9-bd65-44ef-9687-0eb3be61d1a8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:58.381124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-073878b9-bd65-44ef-9687-0eb3be61d1a8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:58.409123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073878b9-bd65-44ef-9687-0eb3be61d1a8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:58.586041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad63dec-0126-4ecb-bf69-282eac76f49c tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:58.586041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad63dec-0126-4ecb-bf69-282eac76f49c tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:58.586041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad63dec-0126-4ecb-bf69-282eac76f49c tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:58.588537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad63dec-0126-4ecb-bf69-282eac76f49c tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/projects/dc8912d28135412f9a2e5b893e370708` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:58.589961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad63dec-0126-4ecb-bf69-282eac76f49c tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:delete_project(project_id=dc8912d28135412f9a2e5b893e370708)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:58.603499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad63dec-0126-4ecb-bf69-282eac76f49c tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:58.805688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5ad63dec-0126-4ecb-bf69-282eac76f49c tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc8912d28135412f9a2e5b893e370708', 'request_id': 'req-5ad63dec-0126-4ecb-bf69-282eac76f49c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:58.822636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5ad63dec-0126-4ecb-bf69-282eac76f49c tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc8912d28135412f9a2e5b893e370708', 'request_id': 'req-5ad63dec-0126-4ecb-bf69-282eac76f49c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:58.854204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5ad63dec-0126-4ecb-bf69-282eac76f49c tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc8912d28135412f9a2e5b893e370708', 'request_id': 'req-5ad63dec-0126-4ecb-bf69-282eac76f49c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:58.954947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5ad63dec-0126-4ecb-bf69-282eac76f49c tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] The token cache is being invalidate because project dc8912d28135412f9a2e5b893e370708 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:58.955730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5ad63dec-0126-4ecb-bf69-282eac76f49c tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:58.960232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5ad63dec-0126-4ecb-bf69-282eac76f49c tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:58.970673 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1813/3695] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:58:58 2022] DELETE /identity/v3/projects/dc8912d28135412f9a2e5b893e370708 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:58:58.976365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d2569-37c9-40f5-aae2-31b7d0e0a66d tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:58.978174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-543d2569-37c9-40f5-aae2-31b7d0e0a66d tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:59.007286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d2569-37c9-40f5-aae2-31b7d0e0a66d tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:59.128895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:59.149286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31d47b-944d-4711-b56f-e34a192e339a tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:59.150200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31d47b-944d-4711-b56f-e34a192e339a tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:59.151293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31d47b-944d-4711-b56f-e34a192e339a tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:59.153966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31d47b-944d-4711-b56f-e34a192e339a tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/users/ddddd157af49408bb600c946165a13aa` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:59.157938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31d47b-944d-4711-b56f-e34a192e339a tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:update_user(user_id=ddddd157af49408bb600c946165a13aa)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:59.159370 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-bf31d47b-944d-4711-b56f-e34a192e339a tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:58:59.159370 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 20:58:59.159370 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 20:58:59.161096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31d47b-944d-4711-b56f-e34a192e339a tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:59.252519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bf31d47b-944d-4711-b56f-e34a192e339a tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'ddddd157af49408bb600c946165a13aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:59.277090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bf31d47b-944d-4711-b56f-e34a192e339a tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invalidating the token cache because user ddddd157af49408bb600c946165a13aa was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:59.277594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bf31d47b-944d-4711-b56f-e34a192e339a tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:59.279041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bf31d47b-944d-4711-b56f-e34a192e339a tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:59.293369 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1883/3696] 10.222.0.17 () {60 vars in 1243 bytes} [Sat Oct 1 20:58:58 2022] PATCH /identity/v3/users/ddddd157af49408bb600c946165a13aa => generated 297 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:59.297718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0c816-d472-4e97-a858-9946b407b118 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:59.298276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0c816-d472-4e97-a858-9946b407b118 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:59.298808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0c816-d472-4e97-a858-9946b407b118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:59.309021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:59.387006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4bb0c816-d472-4e97-a858-9946b407b118 None None] MFA Rules not processed for user `ddddd157af49408bb600c946165a13aa`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:59.392505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4bb0c816-d472-4e97-a858-9946b407b118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:59.402657 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1814/3697] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:59 2022] POST /identity/v3/auth/tokens => generated 349 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:58:59.407124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31d47b-944d-4711-b56f-e34a192e339a tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:59.412490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bf31d47b-944d-4711-b56f-e34a192e339a tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:58:59.438351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31d47b-944d-4711-b56f-e34a192e339a tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:59.455641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf9727-999a-435b-826b-26ed9f182d0b tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:59.456776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf9727-999a-435b-826b-26ed9f182d0b tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:59.457172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf9727-999a-435b-826b-26ed9f182d0b tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:59.457636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf9727-999a-435b-826b-26ed9f182d0b tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/users/ddddd157af49408bb600c946165a13aa` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:59.459553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf9727-999a-435b-826b-26ed9f182d0b tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:update_user(user_id=ddddd157af49408bb600c946165a13aa)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:59.461694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf9727-999a-435b-826b-26ed9f182d0b tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:59.571156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cbdf9727-999a-435b-826b-26ed9f182d0b tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'ddddd157af49408bb600c946165a13aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:59.590316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cbdf9727-999a-435b-826b-26ed9f182d0b tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invalidating the token cache because user ddddd157af49408bb600c946165a13aa was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:59.590704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cbdf9727-999a-435b-826b-26ed9f182d0b tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:59.591410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cbdf9727-999a-435b-826b-26ed9f182d0b tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:59.593893 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1884/3698] 10.222.0.17 () {60 vars in 1243 bytes} [Sat Oct 1 20:58:59 2022] PATCH /identity/v3/users/ddddd157af49408bb600c946165a13aa => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:59.597633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0c816-d472-4e97-a858-9946b407b118 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:58:59.627097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0c816-d472-4e97-a858-9946b407b118 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:58:59.640193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0cf25-ff1b-4fbe-bc42-e573d6804231 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:58:59.642042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0cf25-ff1b-4fbe-bc42-e573d6804231 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:59.643632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0cf25-ff1b-4fbe-bc42-e573d6804231 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:59.647605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0cf25-ff1b-4fbe-bc42-e573d6804231 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/users/ddddd157af49408bb600c946165a13aa` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:59.729078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0cf25-ff1b-4fbe-bc42-e573d6804231 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:update_user(user_id=ddddd157af49408bb600c946165a13aa)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:58:59.730365 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-60c0cf25-ff1b-4fbe-bc42-e573d6804231 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 20:58:59.730365 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 20:58:59.730365 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 20:58:59.731871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0cf25-ff1b-4fbe-bc42-e573d6804231 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:58:59.849128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-60c0cf25-ff1b-4fbe-bc42-e573d6804231 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'ddddd157af49408bb600c946165a13aa'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:59.865095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-60c0cf25-ff1b-4fbe-bc42-e573d6804231 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invalidating the token cache because user ddddd157af49408bb600c946165a13aa was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:58:59.865649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-60c0cf25-ff1b-4fbe-bc42-e573d6804231 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:59.867437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-60c0cf25-ff1b-4fbe-bc42-e573d6804231 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:58:59.871393 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1815/3699] 10.222.0.17 () {60 vars in 1243 bytes} [Sat Oct 1 20:58:59 2022] PATCH /identity/v3/users/ddddd157af49408bb600c946165a13aa => generated 297 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:58:59.876673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721b859-ef3c-4b60-9501-44590559ad9a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:58:59.877331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721b859-ef3c-4b60-9501-44590559ad9a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:58:59.877331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721b859-ef3c-4b60-9501-44590559ad9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:58:59.996830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d721b859-ef3c-4b60-9501-44590559ad9a None None] MFA Rules not processed for user `ddddd157af49408bb600c946165a13aa`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:58:59.998243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d721b859-ef3c-4b60-9501-44590559ad9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:00.010904 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1885/3700] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:58:59 2022] POST /identity/v3/auth/tokens => generated 349 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:59:00.015398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0cf25-ff1b-4fbe-bc42-e573d6804231 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:00.018293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-60c0cf25-ff1b-4fbe-bc42-e573d6804231 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:00.066786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0cf25-ff1b-4fbe-bc42-e573d6804231 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:00.095246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa8911-15b8-43ea-aee8-e52daaf5dcba tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:00.095911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa8911-15b8-43ea-aee8-e52daaf5dcba tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:00.096195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa8911-15b8-43ea-aee8-e52daaf5dcba tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:00.096468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa8911-15b8-43ea-aee8-e52daaf5dcba tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/users/ddddd157af49408bb600c946165a13aa` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:00.100299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa8911-15b8-43ea-aee8-e52daaf5dcba tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:update_user(user_id=ddddd157af49408bb600c946165a13aa)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:00.106430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa8911-15b8-43ea-aee8-e52daaf5dcba tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:00.235239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-93fa8911-15b8-43ea-aee8-e52daaf5dcba tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'ddddd157af49408bb600c946165a13aa'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:00.257604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-93fa8911-15b8-43ea-aee8-e52daaf5dcba tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invalidating the token cache because user ddddd157af49408bb600c946165a13aa was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:59:00.258583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-93fa8911-15b8-43ea-aee8-e52daaf5dcba tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:00.259528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-93fa8911-15b8-43ea-aee8-e52daaf5dcba tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:00.267583 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1816/3701] 10.222.0.17 () {60 vars in 1243 bytes} [Sat Oct 1 20:59:00 2022] PATCH /identity/v3/users/ddddd157af49408bb600c946165a13aa => generated 297 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:59:00.267770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721b859-ef3c-4b60-9501-44590559ad9a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:00.283259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721b859-ef3c-4b60-9501-44590559ad9a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:00.321426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e231bd-a04d-45a0-9747-f363ee048e1f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:00.322233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e231bd-a04d-45a0-9747-f363ee048e1f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:00.323949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e231bd-a04d-45a0-9747-f363ee048e1f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:00.324416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e231bd-a04d-45a0-9747-f363ee048e1f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:00.327220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e231bd-a04d-45a0-9747-f363ee048e1f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:00.330311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e231bd-a04d-45a0-9747-f363ee048e1f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:00.435846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa8911-15b8-43ea-aee8-e52daaf5dcba tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:00.437536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-93fa8911-15b8-43ea-aee8-e52daaf5dcba tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:00.442034 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1886/3702] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 20:59:00 2022] POST /identity/v3/users => generated 264 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:59:00.462530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa8911-15b8-43ea-aee8-e52daaf5dcba tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:00.479828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac3b1c-a524-47f7-b449-95b68fa6ccf5 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:00.481829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac3b1c-a524-47f7-b449-95b68fa6ccf5 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:00.481947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac3b1c-a524-47f7-b449-95b68fa6ccf5 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:00.484267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac3b1c-a524-47f7-b449-95b68fa6ccf5 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/users/aeb5bc89478d40bab818e8ee90619feb/password` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:00.610576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fdac3b1c-a524-47f7-b449-95b68fa6ccf5 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'aeb5bc89478d40bab818e8ee90619feb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:00.635210 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1817/3703] 10.222.0.17 () {60 vars in 1269 bytes} [Sat Oct 1 20:59:00 2022] POST /identity/v3/users/aeb5bc89478d40bab818e8ee90619feb/password => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:01.226796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e231bd-a04d-45a0-9747-f363ee048e1f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:01.230833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-30e231bd-a04d-45a0-9747-f363ee048e1f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:01.432269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af3b9b-199c-4321-b424-0cc7de23d3af None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:01.432552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af3b9b-199c-4321-b424-0cc7de23d3af None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:01.432874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af3b9b-199c-4321-b424-0cc7de23d3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:01.458353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e231bd-a04d-45a0-9747-f363ee048e1f tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:01.462286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35623f1-a2b9-407b-a8d0-c0f564a2f951 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:01.462940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35623f1-a2b9-407b-a8d0-c0f564a2f951 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:01.463224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35623f1-a2b9-407b-a8d0-c0f564a2f951 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:01.463497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35623f1-a2b9-407b-a8d0-c0f564a2f951 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:01.472071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b35623f1-a2b9-407b-a8d0-c0f564a2f951 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:01.487400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35623f1-a2b9-407b-a8d0-c0f564a2f951 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:01.489029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35623f1-a2b9-407b-a8d0-c0f564a2f951 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:01.496936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b1af3b9b-199c-4321-b424-0cc7de23d3af None None] MFA Rules not processed for user `ccbf2c814364466baca8dfcabd2c13f9`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:01.497749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b1af3b9b-199c-4321-b424-0cc7de23d3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:01.506389 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1887/3704] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:59:01.638856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31320eb-4d5e-4d16-91eb-65c9056ecb83 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:01.639238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31320eb-4d5e-4d16-91eb-65c9056ecb83 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:01.639427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31320eb-4d5e-4d16-91eb-65c9056ecb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:01.689706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e31320eb-4d5e-4d16-91eb-65c9056ecb83 None None] MFA Rules not processed for user `aeb5bc89478d40bab818e8ee90619feb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:01.691088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e31320eb-4d5e-4d16-91eb-65c9056ecb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:01.712197 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1888/3705] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:01 2022] POST /identity/v3/auth/tokens => generated 329 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:59:01.716290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31320eb-4d5e-4d16-91eb-65c9056ecb83 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:01.721070 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1818/3706] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:59:01.767130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31320eb-4d5e-4d16-91eb-65c9056ecb83 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:01.787394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ddf1ed-cd84-4c59-9053-cc11b141ba45 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:01.788439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ddf1ed-cd84-4c59-9053-cc11b141ba45 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:01.788439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ddf1ed-cd84-4c59-9053-cc11b141ba45 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:01.788774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ddf1ed-cd84-4c59-9053-cc11b141ba45 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:01.816717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ddf1ed-cd84-4c59-9053-cc11b141ba45 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:01.823212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ddf1ed-cd84-4c59-9053-cc11b141ba45 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:01.840490 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1889/3707] 10.222.0.17 () {56 vars in 1263 bytes} [Sat Oct 1 20:59:01 2022] GET /identity/v3/auth/tokens => generated 329 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 20:59:01.845638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af3b9b-199c-4321-b424-0cc7de23d3af None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:01.864031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af3b9b-199c-4321-b424-0cc7de23d3af None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:01.871348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e7eea-2b0e-481d-b116-d176bc7d9933 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:01.872052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e7eea-2b0e-481d-b116-d176bc7d9933 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:01.872335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e7eea-2b0e-481d-b116-d176bc7d9933 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:01.872616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e7eea-2b0e-481d-b116-d176bc7d9933 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/users/aeb5bc89478d40bab818e8ee90619feb` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:01.874066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e7eea-2b0e-481d-b116-d176bc7d9933 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:delete_user(user_id=aeb5bc89478d40bab818e8ee90619feb)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:01.876257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e7eea-2b0e-481d-b116-d176bc7d9933 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:01.941745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=84,keystone:UPDATE=3,keystone:INSERT=6,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:01.984196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-345e7eea-2b0e-481d-b116-d176bc7d9933 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeb5bc89478d40bab818e8ee90619feb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:01.985342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-345e7eea-2b0e-481d-b116-d176bc7d9933 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeb5bc89478d40bab818e8ee90619feb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:01.986012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-345e7eea-2b0e-481d-b116-d176bc7d9933 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeb5bc89478d40bab818e8ee90619feb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:02.038128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-345e7eea-2b0e-481d-b116-d176bc7d9933 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeb5bc89478d40bab818e8ee90619feb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:02.072430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-345e7eea-2b0e-481d-b116-d176bc7d9933 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeb5bc89478d40bab818e8ee90619feb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:02.108964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ddf1ed-cd84-4c59-9053-cc11b141ba45 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:02.109176 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1819/3708] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:59:01 2022] DELETE /identity/v3/users/aeb5bc89478d40bab818e8ee90619feb => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:02.123512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ddf1ed-cd84-4c59-9053-cc11b141ba45 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:02.245054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:UPDATE=2,keystone:INSERT=7,keystone:DELETE=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:02.271983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b55480-573f-4b8b-8a72-85e82ce3c01e tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:02.272626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b55480-573f-4b8b-8a72-85e82ce3c01e tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:02.272911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b55480-573f-4b8b-8a72-85e82ce3c01e tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:02.273191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b55480-573f-4b8b-8a72-85e82ce3c01e tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:02.273922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b55480-573f-4b8b-8a72-85e82ce3c01e tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:02.276741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b55480-573f-4b8b-8a72-85e82ce3c01e tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:02.333750 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1890/3709] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:59:02 2022] POST /identity/v3/users => generated 364 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:02.338199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e7eea-2b0e-481d-b116-d176bc7d9933 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:02.339696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-345e7eea-2b0e-481d-b116-d176bc7d9933 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:02.364455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e7eea-2b0e-481d-b116-d176bc7d9933 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:02.377016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aedd62-0846-4136-9de5-d73c24a23720 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:02.379358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aedd62-0846-4136-9de5-d73c24a23720 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:02.379685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aedd62-0846-4136-9de5-d73c24a23720 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:02.379969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aedd62-0846-4136-9de5-d73c24a23720 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:02.380785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aedd62-0846-4136-9de5-d73c24a23720 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:02.382741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aedd62-0846-4136-9de5-d73c24a23720 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:02.449722 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1820/3710] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:59:02 2022] POST /identity/v3/projects => generated 359 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:02.451219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b55480-573f-4b8b-8a72-85e82ce3c01e tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:02.466503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b55480-573f-4b8b-8a72-85e82ce3c01e tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:02.775073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e176a1-8080-4702-bbcc-29991ce3ee12 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:02.776227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e176a1-8080-4702-bbcc-29991ce3ee12 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:02.776759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e176a1-8080-4702-bbcc-29991ce3ee12 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:02.777271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e176a1-8080-4702-bbcc-29991ce3ee12 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/users/90006ef1bc37485db8539e3956d91053` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:02.800612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e176a1-8080-4702-bbcc-29991ce3ee12 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:update_user(user_id=90006ef1bc37485db8539e3956d91053)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:02.803770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e176a1-8080-4702-bbcc-29991ce3ee12 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:02.861724 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1891/3711] 10.222.0.17 () {60 vars in 1244 bytes} [Sat Oct 1 20:59:02 2022] PATCH /identity/v3/users/90006ef1bc37485db8539e3956d91053 => generated 521 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:59:02.865709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aedd62-0846-4136-9de5-d73c24a23720 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:02.900499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aedd62-0846-4136-9de5-d73c24a23720 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:02.912710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77847f6f-b373-4d5a-a369-f7365cdf7297 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:02.914337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77847f6f-b373-4d5a-a369-f7365cdf7297 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:02.914947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77847f6f-b373-4d5a-a369-f7365cdf7297 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:02.916004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77847f6f-b373-4d5a-a369-f7365cdf7297 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/users/90006ef1bc37485db8539e3956d91053` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:02.963527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77847f6f-b373-4d5a-a369-f7365cdf7297 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:get_user(user_id=90006ef1bc37485db8539e3956d91053)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:02.971130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77847f6f-b373-4d5a-a369-f7365cdf7297 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:02.975308 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1821/3712] 10.222.0.17 () {58 vars in 1221 bytes} [Sat Oct 1 20:59:02 2022] GET /identity/v3/users/90006ef1bc37485db8539e3956d91053 => generated 388 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:59:02.979973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e176a1-8080-4702-bbcc-29991ce3ee12 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:02.998073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e176a1-8080-4702-bbcc-29991ce3ee12 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:03.006321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce033fb-3160-4d9a-90e0-0fd5da509da6 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:03.007463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce033fb-3160-4d9a-90e0-0fd5da509da6 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:03.008027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce033fb-3160-4d9a-90e0-0fd5da509da6 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:03.008520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce033fb-3160-4d9a-90e0-0fd5da509da6 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/projects/5616fa324825485381e19823b5a90cfe` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:03.010329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce033fb-3160-4d9a-90e0-0fd5da509da6 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5616fa324825485381e19823b5a90cfe)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:03.014891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce033fb-3160-4d9a-90e0-0fd5da509da6 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:03.119106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0ce033fb-3160-4d9a-90e0-0fd5da509da6 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5616fa324825485381e19823b5a90cfe', 'request_id': 'req-0ce033fb-3160-4d9a-90e0-0fd5da509da6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:03.128471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0ce033fb-3160-4d9a-90e0-0fd5da509da6 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5616fa324825485381e19823b5a90cfe', 'request_id': 'req-0ce033fb-3160-4d9a-90e0-0fd5da509da6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:03.135011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0ce033fb-3160-4d9a-90e0-0fd5da509da6 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5616fa324825485381e19823b5a90cfe', 'request_id': 'req-0ce033fb-3160-4d9a-90e0-0fd5da509da6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:03.156922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0ce033fb-3160-4d9a-90e0-0fd5da509da6 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] The token cache is being invalidate because project 5616fa324825485381e19823b5a90cfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:59:03.157257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0ce033fb-3160-4d9a-90e0-0fd5da509da6 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:03.160190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0ce033fb-3160-4d9a-90e0-0fd5da509da6 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:03.167483 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1892/3713] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:59:02 2022] DELETE /identity/v3/projects/5616fa324825485381e19823b5a90cfe => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:03.169232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77847f6f-b373-4d5a-a369-f7365cdf7297 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:03.174833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-77847f6f-b373-4d5a-a369-f7365cdf7297 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:03.237885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77847f6f-b373-4d5a-a369-f7365cdf7297 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:03.407587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9387b25-aa15-48d1-a815-21eac65047f8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38579d5d4d3641d7a706682ebe1f998f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:03.408506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9387b25-aa15-48d1-a815-21eac65047f8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:03.408915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9387b25-aa15-48d1-a815-21eac65047f8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:03.409387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9387b25-aa15-48d1-a815-21eac65047f8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] PATH_INFO: `/v3/users/90006ef1bc37485db8539e3956d91053` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:03.410971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9387b25-aa15-48d1-a815-21eac65047f8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorizing `identity:delete_user(user_id=90006ef1bc37485db8539e3956d91053)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:03.414074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9387b25-aa15-48d1-a815-21eac65047f8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:03.574208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c9387b25-aa15-48d1-a815-21eac65047f8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90006ef1bc37485db8539e3956d91053'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:03.575882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c9387b25-aa15-48d1-a815-21eac65047f8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90006ef1bc37485db8539e3956d91053'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:03.578050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c9387b25-aa15-48d1-a815-21eac65047f8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90006ef1bc37485db8539e3956d91053'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:03.621764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c9387b25-aa15-48d1-a815-21eac65047f8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90006ef1bc37485db8539e3956d91053'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:03.636686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=17,keystone:SELECT=80,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:03.673564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c9387b25-aa15-48d1-a815-21eac65047f8 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90006ef1bc37485db8539e3956d91053'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:03.713160 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1822/3714] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:59:03 2022] DELETE /identity/v3/users/90006ef1bc37485db8539e3956d91053 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:03.717777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce033fb-3160-4d9a-90e0-0fd5da509da6 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:03.720201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0ce033fb-3160-4d9a-90e0-0fd5da509da6 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:03.801149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:UPDATE=2,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:03.873088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce033fb-3160-4d9a-90e0-0fd5da509da6 tempest-UsersV3TestJSON-1994630758 tempest-UsersV3TestJSON-1994630758-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:03.991787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e1e21-5599-4d60-9c6f-2020c0d5d3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:03.992442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e1e21-5599-4d60-9c6f-2020c0d5d3ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:03.992852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e1e21-5599-4d60-9c6f-2020c0d5d3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:03.993138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e1e21-5599-4d60-9c6f-2020c0d5d3ba admin admin] PATH_INFO: `/v3/users/ddddd157af49408bb600c946165a13aa` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:04.013695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e1e21-5599-4d60-9c6f-2020c0d5d3ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddddd157af49408bb600c946165a13aa)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:04.021543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e1e21-5599-4d60-9c6f-2020c0d5d3ba admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:04.148137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f05e1e21-5599-4d60-9c6f-2020c0d5d3ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddddd157af49408bb600c946165a13aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:04.167763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f05e1e21-5599-4d60-9c6f-2020c0d5d3ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddddd157af49408bb600c946165a13aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:04.190887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f05e1e21-5599-4d60-9c6f-2020c0d5d3ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddddd157af49408bb600c946165a13aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:04.191649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f05e1e21-5599-4d60-9c6f-2020c0d5d3ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddddd157af49408bb600c946165a13aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:04.206178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f05e1e21-5599-4d60-9c6f-2020c0d5d3ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddddd157af49408bb600c946165a13aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:04.215632 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1893/3715] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:59:03 2022] DELETE /identity/v3/users/ddddd157af49408bb600c946165a13aa => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:04.220772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e602943-af9a-4496-9f41-ba40f749487e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:04.221206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e602943-af9a-4496-9f41-ba40f749487e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:04.221453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e602943-af9a-4496-9f41-ba40f749487e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:04.268428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3e602943-af9a-4496-9f41-ba40f749487e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:04.273461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3e602943-af9a-4496-9f41-ba40f749487e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:04.461109 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1823/3716] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:59:04.541248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e1e21-5599-4d60-9c6f-2020c0d5d3ba admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:04.542760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f05e1e21-5599-4d60-9c6f-2020c0d5d3ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:04.674792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:04.691756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e1e21-5599-4d60-9c6f-2020c0d5d3ba admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:04.711225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b110e2-7510-40ae-afe8-4f6ef77bf0cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:04.712106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b110e2-7510-40ae-afe8-4f6ef77bf0cf service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:04.719529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b110e2-7510-40ae-afe8-4f6ef77bf0cf service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:04.720448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b110e2-7510-40ae-afe8-4f6ef77bf0cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:04.768373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b110e2-7510-40ae-afe8-4f6ef77bf0cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:04.770917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b110e2-7510-40ae-afe8-4f6ef77bf0cf service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:04.813251 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1894/3717] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:59:04.953933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e602943-af9a-4496-9f41-ba40f749487e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:04.955737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3e602943-af9a-4496-9f41-ba40f749487e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:05.255055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e602943-af9a-4496-9f41-ba40f749487e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:05.269506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e8f14-d013-4e59-ae50-d9603e8d55a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:05.270521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e8f14-d013-4e59-ae50-d9603e8d55a3 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:05.275506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e8f14-d013-4e59-ae50-d9603e8d55a3 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:05.276245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e8f14-d013-4e59-ae50-d9603e8d55a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:05.280347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f87e8f14-d013-4e59-ae50-d9603e8d55a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:05.317407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e8f14-d013-4e59-ae50-d9603e8d55a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:05.319258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e8f14-d013-4e59-ae50-d9603e8d55a3 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:05.359843 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1824/3718] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:59:07.188124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b110e2-7510-40ae-afe8-4f6ef77bf0cf service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:07.190442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-03b110e2-7510-40ae-afe8-4f6ef77bf0cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:07.221576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b110e2-7510-40ae-afe8-4f6ef77bf0cf service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:07.251322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f1f30-80fd-4385-a27b-9fad6606a51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:07.252541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f1f30-80fd-4385-a27b-9fad6606a51f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:07.256335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f1f30-80fd-4385-a27b-9fad6606a51f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:07.257525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f1f30-80fd-4385-a27b-9fad6606a51f admin admin] PATH_INFO: `/v3/projects/897da389f8994ffc9444b347d75ea9ae` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:07.265128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f1f30-80fd-4385-a27b-9fad6606a51f admin admin] RBAC: Authorizing `identity:delete_project(project_id=897da389f8994ffc9444b347d75ea9ae)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:07.272446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f1f30-80fd-4385-a27b-9fad6606a51f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:07.478841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-437f1f30-80fd-4385-a27b-9fad6606a51f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '897da389f8994ffc9444b347d75ea9ae', 'request_id': 'req-437f1f30-80fd-4385-a27b-9fad6606a51f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:07.503566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-437f1f30-80fd-4385-a27b-9fad6606a51f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '897da389f8994ffc9444b347d75ea9ae', 'request_id': 'req-437f1f30-80fd-4385-a27b-9fad6606a51f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:07.539306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-437f1f30-80fd-4385-a27b-9fad6606a51f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '897da389f8994ffc9444b347d75ea9ae', 'request_id': 'req-437f1f30-80fd-4385-a27b-9fad6606a51f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:07.604065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-437f1f30-80fd-4385-a27b-9fad6606a51f admin admin] The token cache is being invalidate because project 897da389f8994ffc9444b347d75ea9ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:59:07.604363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-437f1f30-80fd-4385-a27b-9fad6606a51f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:07.605420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-437f1f30-80fd-4385-a27b-9fad6606a51f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:07.611412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e8f14-d013-4e59-ae50-d9603e8d55a3 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:07.612716 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1895/3719] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:59:07 2022] DELETE /identity/v3/projects/897da389f8994ffc9444b347d75ea9ae => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:07.630650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e8f14-d013-4e59-ae50-d9603e8d55a3 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:07.793625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a7265-192c-4696-98fc-8c6760c2cf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:07.794746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a7265-192c-4696-98fc-8c6760c2cf92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:07.795320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a7265-192c-4696-98fc-8c6760c2cf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:07.795801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a7265-192c-4696-98fc-8c6760c2cf92 admin admin] PATH_INFO: `/v3/users/38579d5d4d3641d7a706682ebe1f998f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:07.797990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a7265-192c-4696-98fc-8c6760c2cf92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38579d5d4d3641d7a706682ebe1f998f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:07.815532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a7265-192c-4696-98fc-8c6760c2cf92 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:07.817818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:08.022018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb7b3c-4568-47a3-a5c9-6ce8315c4f20 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:08.022483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb7b3c-4568-47a3-a5c9-6ce8315c4f20 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:08.022942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb7b3c-4568-47a3-a5c9-6ce8315c4f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:08.057715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-126a7265-192c-4696-98fc-8c6760c2cf92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38579d5d4d3641d7a706682ebe1f998f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:08.058717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-126a7265-192c-4696-98fc-8c6760c2cf92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38579d5d4d3641d7a706682ebe1f998f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:08.063104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-126a7265-192c-4696-98fc-8c6760c2cf92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38579d5d4d3641d7a706682ebe1f998f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:08.144904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-126a7265-192c-4696-98fc-8c6760c2cf92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38579d5d4d3641d7a706682ebe1f998f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:08.225902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-126a7265-192c-4696-98fc-8c6760c2cf92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38579d5d4d3641d7a706682ebe1f998f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:08.263663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-dbcb7b3c-4568-47a3-a5c9-6ce8315c4f20 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:08.266143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-dbcb7b3c-4568-47a3-a5c9-6ce8315c4f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:08.282526 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1825/3720] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:59:07 2022] DELETE /identity/v3/users/38579d5d4d3641d7a706682ebe1f998f => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:08.289586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4e22a-fa71-466b-87f1-d3b7de94975a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:08.289872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4e22a-fa71-466b-87f1-d3b7de94975a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:08.293522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4e22a-fa71-466b-87f1-d3b7de94975a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:08.436392 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1896/3721] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 20:59:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:08.449748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb7b3c-4568-47a3-a5c9-6ce8315c4f20 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:08.451478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-dbcb7b3c-4568-47a3-a5c9-6ce8315c4f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:08.488019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-95a4e22a-fa71-466b-87f1-d3b7de94975a None None] MFA Rules not processed for user `e84a109e337d450698e630acfb49f9a5`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:08.489275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-95a4e22a-fa71-466b-87f1-d3b7de94975a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:08.599184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:08.663702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb7b3c-4568-47a3-a5c9-6ce8315c4f20 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:08.774022 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1826/3722] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:08.778027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d251bef-1cb2-4c83-bfc7-6857a60c6dc2 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:08.778452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d251bef-1cb2-4c83-bfc7-6857a60c6dc2 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:08.778919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d251bef-1cb2-4c83-bfc7-6857a60c6dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:08.814248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0d251bef-1cb2-4c83-bfc7-6857a60c6dc2 None None] MFA Rules not processed for user `e84a109e337d450698e630acfb49f9a5`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:08.817657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0d251bef-1cb2-4c83-bfc7-6857a60c6dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:08.830184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3719eb-f478-42d4-983d-61447d1a8d58 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:08.830832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3719eb-f478-42d4-983d-61447d1a8d58 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:08.831134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3719eb-f478-42d4-983d-61447d1a8d58 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:08.831412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3719eb-f478-42d4-983d-61447d1a8d58 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:08.842437 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1827/3723] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:08.964854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3719eb-f478-42d4-983d-61447d1a8d58 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:08.966540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3719eb-f478-42d4-983d-61447d1a8d58 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:09.001907 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1897/3724] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:59:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 20:59:10.114791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d251bef-1cb2-4c83-bfc7-6857a60c6dc2 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:10.117001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0d251bef-1cb2-4c83-bfc7-6857a60c6dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:10.270788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:10.317735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d251bef-1cb2-4c83-bfc7-6857a60c6dc2 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:10.336361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdd7bd-0554-45ff-9f51-6e3d8588a47a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:10.337189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdd7bd-0554-45ff-9f51-6e3d8588a47a service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:10.337475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdd7bd-0554-45ff-9f51-6e3d8588a47a service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:10.337752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdd7bd-0554-45ff-9f51-6e3d8588a47a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:10.363508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bdd7bd-0554-45ff-9f51-6e3d8588a47a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:10.365810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bdd7bd-0554-45ff-9f51-6e3d8588a47a service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:10.394282 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1828/3725] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:59:10.513351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3719eb-f478-42d4-983d-61447d1a8d58 service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:10.514785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ee3719eb-f478-42d4-983d-61447d1a8d58 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:10.838986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3719eb-f478-42d4-983d-61447d1a8d58 service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:10.867589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d256832-06f4-47cb-8f65-435096cfa353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:10.885364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d256832-06f4-47cb-8f65-435096cfa353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:10.887835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d256832-06f4-47cb-8f65-435096cfa353 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:10.888259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d256832-06f4-47cb-8f65-435096cfa353 admin admin] PATH_INFO: `/v3/projects/cab5a7d3943c4d148dd3e0e5855abe4f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:10.893868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d256832-06f4-47cb-8f65-435096cfa353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cab5a7d3943c4d148dd3e0e5855abe4f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:10.927045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d256832-06f4-47cb-8f65-435096cfa353 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:11.138569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:11.225106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6d256832-06f4-47cb-8f65-435096cfa353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'request_id': 'req-6d256832-06f4-47cb-8f65-435096cfa353'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:11.255863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6d256832-06f4-47cb-8f65-435096cfa353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'request_id': 'req-6d256832-06f4-47cb-8f65-435096cfa353'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:11.287020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6d256832-06f4-47cb-8f65-435096cfa353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cab5a7d3943c4d148dd3e0e5855abe4f', 'request_id': 'req-6d256832-06f4-47cb-8f65-435096cfa353'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:11.354427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6d256832-06f4-47cb-8f65-435096cfa353 admin admin] The token cache is being invalidate because project cab5a7d3943c4d148dd3e0e5855abe4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:59:11.354872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6d256832-06f4-47cb-8f65-435096cfa353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:11.355700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6d256832-06f4-47cb-8f65-435096cfa353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:11.370061 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1898/3726] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:59:10 2022] DELETE /identity/v3/projects/cab5a7d3943c4d148dd3e0e5855abe4f => generated 0 bytes in 853 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:11.371877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5eec9e-e9d3-4886-b4a3-4548b75f39c5 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:11.372343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5eec9e-e9d3-4886-b4a3-4548b75f39c5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:11.372423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5eec9e-e9d3-4886-b4a3-4548b75f39c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:11.426470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ae5eec9e-e9d3-4886-b4a3-4548b75f39c5 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:11.431960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ae5eec9e-e9d3-4886-b4a3-4548b75f39c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:11.646074 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1829/3727] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:11.647030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134fa85-e173-46be-8fa1-9cd0e797d3fb None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:11.647618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134fa85-e173-46be-8fa1-9cd0e797d3fb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:11.647726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134fa85-e173-46be-8fa1-9cd0e797d3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:11.748593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6134fa85-e173-46be-8fa1-9cd0e797d3fb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:11.750381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6134fa85-e173-46be-8fa1-9cd0e797d3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:11.827474 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1899/3728] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:59:11.829605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5eec9e-e9d3-4886-b4a3-4548b75f39c5 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:11.848366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5eec9e-e9d3-4886-b4a3-4548b75f39c5 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:11.879159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffd0c3-10d9-4492-9462-7c970de68d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:11.880799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffd0c3-10d9-4492-9462-7c970de68d9c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:11.881242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffd0c3-10d9-4492-9462-7c970de68d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:11.881693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffd0c3-10d9-4492-9462-7c970de68d9c admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:11.882828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ffd0c3-10d9-4492-9462-7c970de68d9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:59:11.883289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ffd0c3-10d9-4492-9462-7c970de68d9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:11.891296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ffd0c3-10d9-4492-9462-7c970de68d9c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:11.948652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134fa85-e173-46be-8fa1-9cd0e797d3fb None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:11.948945 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1830/3729] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:59:11.971694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134fa85-e173-46be-8fa1-9cd0e797d3fb None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:11.986076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e0cb8-8381-4beb-af65-cdebcab940f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:11.989356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e0cb8-8381-4beb-af65-cdebcab940f3 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:11.990414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e0cb8-8381-4beb-af65-cdebcab940f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:11.991376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e0cb8-8381-4beb-af65-cdebcab940f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:11.994378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e0cb8-8381-4beb-af65-cdebcab940f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:11.998987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e0cb8-8381-4beb-af65-cdebcab940f3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:12.146379 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1900/3730] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:59:11 2022] POST /identity/v3/projects => generated 379 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:59:12.150603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffd0c3-10d9-4492-9462-7c970de68d9c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:12.167557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffd0c3-10d9-4492-9462-7c970de68d9c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:12.465920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffe7fa-4650-43f9-98d5-2f9e21feae94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:12.466600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffe7fa-4650-43f9-98d5-2f9e21feae94 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:12.469777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffe7fa-4650-43f9-98d5-2f9e21feae94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:12.469777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffe7fa-4650-43f9-98d5-2f9e21feae94 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:12.469777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ffe7fa-4650-43f9-98d5-2f9e21feae94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:12.485186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ffe7fa-4650-43f9-98d5-2f9e21feae94 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:12.545210 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1831/3731] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:59:12 2022] POST /identity/v3/users => generated 290 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:12.549290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e0cb8-8381-4beb-af65-cdebcab940f3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:12.582946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e0cb8-8381-4beb-af65-cdebcab940f3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:12.608610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb411b1f-2fe9-494a-be32-1db6df04a9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:12.610179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb411b1f-2fe9-494a-be32-1db6df04a9cf admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:12.610704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb411b1f-2fe9-494a-be32-1db6df04a9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:12.611175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb411b1f-2fe9-494a-be32-1db6df04a9cf admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:12.612150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb411b1f-2fe9-494a-be32-1db6df04a9cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:12.613975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb411b1f-2fe9-494a-be32-1db6df04a9cf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:12.638848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffe7fa-4650-43f9-98d5-2f9e21feae94 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:12.639089 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1901/3732] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:59:12.653923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffe7fa-4650-43f9-98d5-2f9e21feae94 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:12.666547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d915e-233f-449c-b6f8-d556c98084b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:12.667196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d915e-233f-449c-b6f8-d556c98084b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:12.667709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d915e-233f-449c-b6f8-d556c98084b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:12.668019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d915e-233f-449c-b6f8-d556c98084b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:12.670919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d915e-233f-449c-b6f8-d556c98084b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:12.676481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d915e-233f-449c-b6f8-d556c98084b6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:12.722852 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1832/3733] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:59:12.725573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb411b1f-2fe9-494a-be32-1db6df04a9cf admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:12.746463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb411b1f-2fe9-494a-be32-1db6df04a9cf admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:12.765508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efdbb17-ba25-4156-be8a-7f39f8732716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:12.766761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdbb17-ba25-4156-be8a-7f39f8732716 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:12.767206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdbb17-ba25-4156-be8a-7f39f8732716 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:12.772942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdbb17-ba25-4156-be8a-7f39f8732716 admin admin] PATH_INFO: `/v3/projects/2889ffe1cedd48f7b88f68b121a47d3b/users/6ec11bb5a1e147e6aa4e49d69b59a1c0/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:12.863291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efdbb17-ba25-4156-be8a-7f39f8732716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2889ffe1cedd48f7b88f68b121a47d3b, user_id=6ec11bb5a1e147e6aa4e49d69b59a1c0, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:12.872318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efdbb17-ba25-4156-be8a-7f39f8732716 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:12.953637 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1902/3734] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:59:12 2022] PUT /identity/v3/projects/2889ffe1cedd48f7b88f68b121a47d3b/users/6ec11bb5a1e147e6aa4e49d69b59a1c0/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:12.966373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b334a4-e9ec-4363-ab15-f9631c566fab None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:12.966681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b334a4-e9ec-4363-ab15-f9631c566fab None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:12.966949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b334a4-e9ec-4363-ab15-f9631c566fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:13.054347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c9b334a4-e9ec-4363-ab15-f9631c566fab None None] MFA Rules not processed for user `6ec11bb5a1e147e6aa4e49d69b59a1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:13.055613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c9b334a4-e9ec-4363-ab15-f9631c566fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:13.169712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:13.236740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efdbb17-ba25-4156-be8a-7f39f8732716 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:13.242662 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1833/3735] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:13.391871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efdbb17-ba25-4156-be8a-7f39f8732716 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:13.404463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10ea52-9325-42b7-8209-4c6f44424077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:13.405405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10ea52-9325-42b7-8209-4c6f44424077 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:13.405887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10ea52-9325-42b7-8209-4c6f44424077 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:13.406990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10ea52-9325-42b7-8209-4c6f44424077 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:13.408253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10ea52-9325-42b7-8209-4c6f44424077 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:13.419591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10ea52-9325-42b7-8209-4c6f44424077 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:13.487166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b334a4-e9ec-4363-ab15-f9631c566fab None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:13.495386 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1903/3736] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:59:13 2022] POST /identity/v3/projects => generated 379 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:13.505150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b334a4-e9ec-4363-ab15-f9631c566fab None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:13.650825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc384984-58eb-48ad-b1ce-fc2c09ac3507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:13.651503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc384984-58eb-48ad-b1ce-fc2c09ac3507 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:13.651869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc384984-58eb-48ad-b1ce-fc2c09ac3507 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:13.652147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc384984-58eb-48ad-b1ce-fc2c09ac3507 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:13.652830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc384984-58eb-48ad-b1ce-fc2c09ac3507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:13.658794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc384984-58eb-48ad-b1ce-fc2c09ac3507 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:13.715693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10ea52-9325-42b7-8209-4c6f44424077 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:13.723597 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1834/3737] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:59:13 2022] POST /identity/v3/users => generated 290 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:13.737672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10ea52-9325-42b7-8209-4c6f44424077 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:13.744954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635aeb2-1afc-4a90-b48e-144dd6fec7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:13.745566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635aeb2-1afc-4a90-b48e-144dd6fec7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:13.745846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635aeb2-1afc-4a90-b48e-144dd6fec7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:13.746117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635aeb2-1afc-4a90-b48e-144dd6fec7f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:13.746806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635aeb2-1afc-4a90-b48e-144dd6fec7f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:13.748522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635aeb2-1afc-4a90-b48e-144dd6fec7f6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:13.781228 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1904/3738] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:59:13.785268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc384984-58eb-48ad-b1ce-fc2c09ac3507 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:13.797470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc384984-58eb-48ad-b1ce-fc2c09ac3507 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:13.811256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d91c86-0539-4fe2-b8a2-c21c4461a746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:13.811888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d91c86-0539-4fe2-b8a2-c21c4461a746 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:13.812701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d91c86-0539-4fe2-b8a2-c21c4461a746 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:13.813141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d91c86-0539-4fe2-b8a2-c21c4461a746 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:13.813994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d91c86-0539-4fe2-b8a2-c21c4461a746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:13.818935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d91c86-0539-4fe2-b8a2-c21c4461a746 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:13.850866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635aeb2-1afc-4a90-b48e-144dd6fec7f6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:13.859682 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1835/3739] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:59:13.866853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635aeb2-1afc-4a90-b48e-144dd6fec7f6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:13.883317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1879d-3250-484c-b056-49f489479ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:13.884279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1879d-3250-484c-b056-49f489479ed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:13.884802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1879d-3250-484c-b056-49f489479ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:13.885248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1879d-3250-484c-b056-49f489479ed8 admin admin] PATH_INFO: `/v3/projects/4d3d2fd9eb3c4263aa9d50d4a574cfab/users/07a9899351f84707aa4c7f9c1e1fe6d5/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:13.929295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1879d-3250-484c-b056-49f489479ed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d3d2fd9eb3c4263aa9d50d4a574cfab, user_id=07a9899351f84707aa4c7f9c1e1fe6d5, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:13.934520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1879d-3250-484c-b056-49f489479ed8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:13.980925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e045a9d-6d28-483c-a139-bc27a9aaee28 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:13.981200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e045a9d-6d28-483c-a139-bc27a9aaee28 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:13.981513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e045a9d-6d28-483c-a139-bc27a9aaee28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:13.989429 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1905/3740] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:59:13 2022] PUT /identity/v3/projects/4d3d2fd9eb3c4263aa9d50d4a574cfab/users/07a9899351f84707aa4c7f9c1e1fe6d5/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:14.048276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4e045a9d-6d28-483c-a139-bc27a9aaee28 None None] MFA Rules not processed for user `07a9899351f84707aa4c7f9c1e1fe6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:14.049727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4e045a9d-6d28-483c-a139-bc27a9aaee28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:14.221766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1879d-3250-484c-b056-49f489479ed8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:14.227302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:14.228180 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1836/3741] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:14.345888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1879d-3250-484c-b056-49f489479ed8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:14.353807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71efce9-9d93-476c-bf70-346d0f3ca330 tempest-IdentityV3ProjectsTest-128990240 tempest-IdentityV3ProjectsTest-128990240-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ec11bb5a1e147e6aa4e49d69b59a1c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2889ffe1cedd48f7b88f68b121a47d3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:14.354579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71efce9-9d93-476c-bf70-346d0f3ca330 tempest-IdentityV3ProjectsTest-128990240 tempest-IdentityV3ProjectsTest-128990240-project] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:14.354870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71efce9-9d93-476c-bf70-346d0f3ca330 tempest-IdentityV3ProjectsTest-128990240 tempest-IdentityV3ProjectsTest-128990240-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:14.355202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71efce9-9d93-476c-bf70-346d0f3ca330 tempest-IdentityV3ProjectsTest-128990240 tempest-IdentityV3ProjectsTest-128990240-project] PATH_INFO: `/v3/users/6ec11bb5a1e147e6aa4e49d69b59a1c0/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:14.363653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71efce9-9d93-476c-bf70-346d0f3ca330 tempest-IdentityV3ProjectsTest-128990240 tempest-IdentityV3ProjectsTest-128990240-project] RBAC: Authorizing `identity:list_user_projects(user_id=6ec11bb5a1e147e6aa4e49d69b59a1c0)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:14.366203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71efce9-9d93-476c-bf70-346d0f3ca330 tempest-IdentityV3ProjectsTest-128990240 tempest-IdentityV3ProjectsTest-128990240-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:14.432511 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1906/3742] 10.222.0.17 () {58 vars in 1248 bytes} [Sat Oct 1 20:59:14 2022] GET /identity/v3/users/6ec11bb5a1e147e6aa4e49d69b59a1c0/projects => generated 515 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:59:14.434875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4a1ef-5f5f-4c7e-9b84-69c2dc8aa02a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:14.435443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4a1ef-5f5f-4c7e-9b84-69c2dc8aa02a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:14.436030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4a1ef-5f5f-4c7e-9b84-69c2dc8aa02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:14.499273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1de4a1ef-5f5f-4c7e-9b84-69c2dc8aa02a None None] MFA Rules not processed for user `6ec11bb5a1e147e6aa4e49d69b59a1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:14.507376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1de4a1ef-5f5f-4c7e-9b84-69c2dc8aa02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:14.531572 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1837/3743] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:14.533791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e1822-e44d-4c21-aa15-a67eed49fd5c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:14.534085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e1822-e44d-4c21-aa15-a67eed49fd5c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:14.534368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e1822-e44d-4c21-aa15-a67eed49fd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:14.605860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c25e1822-e44d-4c21-aa15-a67eed49fd5c None None] MFA Rules not processed for user `6ec11bb5a1e147e6aa4e49d69b59a1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:14.607676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c25e1822-e44d-4c21-aa15-a67eed49fd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:14.800163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.models.token_model [None req-c25e1822-e44d-4c21-aa15-a67eed49fd5c None None] User 6ec11bb5a1e147e6aa4e49d69b59a1c0 has no access to project 4d3d2fd9eb3c4263aa9d50d4a574cfab {{(pid=99139) _validate_project_scope /opt/stack/keystone/keystone/models/token_model.py:539}} Oct 01 20:59:14.821179 ncdv-master-1 devstack@keystone.service[99139]: WARNING keystone.server.flask.application [None req-c25e1822-e44d-4c21-aa15-a67eed49fd5c None None] Authorization failed. The request you have made requires authentication. from 10.222.0.17: keystone.exception.Unauthorized: The request you have made requires authentication. Oct 01 20:59:14.826146 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1907/3744] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:14 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 293 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Oct 01 20:59:14.830908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4a1ef-5f5f-4c7e-9b84-69c2dc8aa02a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:15.037147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:15.125386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4a1ef-5f5f-4c7e-9b84-69c2dc8aa02a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:15.138057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e44dae-b934-4843-8f8a-d2a332d32002 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:15.139176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e44dae-b934-4843-8f8a-d2a332d32002 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:15.139467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e44dae-b934-4843-8f8a-d2a332d32002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:15.146549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ee5c3-65bf-4911-8a14-1bc816788e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:15.147335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ee5c3-65bf-4911-8a14-1bc816788e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:15.147625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ee5c3-65bf-4911-8a14-1bc816788e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:15.147994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ee5c3-65bf-4911-8a14-1bc816788e4a admin admin] PATH_INFO: `/v3/users/6ec11bb5a1e147e6aa4e49d69b59a1c0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:15.160903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ee5c3-65bf-4911-8a14-1bc816788e4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ec11bb5a1e147e6aa4e49d69b59a1c0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:15.163114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ee5c3-65bf-4911-8a14-1bc816788e4a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:15.295335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-30e44dae-b934-4843-8f8a-d2a332d32002 None None] MFA Rules not processed for user `69002d46d4e54eadadf911ec3589d95c`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:15.297857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-30e44dae-b934-4843-8f8a-d2a332d32002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:15.329142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2a6ee5c3-65bf-4911-8a14-1bc816788e4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ec11bb5a1e147e6aa4e49d69b59a1c0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:15.331368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2a6ee5c3-65bf-4911-8a14-1bc816788e4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ec11bb5a1e147e6aa4e49d69b59a1c0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:15.332043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2a6ee5c3-65bf-4911-8a14-1bc816788e4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ec11bb5a1e147e6aa4e49d69b59a1c0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:15.352455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2a6ee5c3-65bf-4911-8a14-1bc816788e4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ec11bb5a1e147e6aa4e49d69b59a1c0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:15.404619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2a6ee5c3-65bf-4911-8a14-1bc816788e4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ec11bb5a1e147e6aa4e49d69b59a1c0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:15.436905 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1838/3745] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:59:14 2022] DELETE /identity/v3/users/6ec11bb5a1e147e6aa4e49d69b59a1c0 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:15.446653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c3d53-ecff-4242-8c05-f2cc138eeda4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:15.446978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c3d53-ecff-4242-8c05-f2cc138eeda4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:15.447249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c3d53-ecff-4242-8c05-f2cc138eeda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:15.490786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-932c3d53-ecff-4242-8c05-f2cc138eeda4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:15.497906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-932c3d53-ecff-4242-8c05-f2cc138eeda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:15.543317 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1908/3746] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:15.550601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e44dae-b934-4843-8f8a-d2a332d32002 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:15.552767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-30e44dae-b934-4843-8f8a-d2a332d32002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:15.588511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e44dae-b934-4843-8f8a-d2a332d32002 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:15.617901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:15.675970 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1839/3747] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:15.709998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c3d53-ecff-4242-8c05-f2cc138eeda4 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:15.711556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-932c3d53-ecff-4242-8c05-f2cc138eeda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:15.764353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103a7f5-4472-4c06-8d95-4b2f47f5023b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:15.765842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103a7f5-4472-4c06-8d95-4b2f47f5023b admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:15.767459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103a7f5-4472-4c06-8d95-4b2f47f5023b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:15.768561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103a7f5-4472-4c06-8d95-4b2f47f5023b admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:15.770642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103a7f5-4472-4c06-8d95-4b2f47f5023b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:15.774462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103a7f5-4472-4c06-8d95-4b2f47f5023b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:15.853334 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1909/3748] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:59:15 2022] POST /identity/v3/projects => generated 383 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:15.857657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103a7f5-4472-4c06-8d95-4b2f47f5023b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:15.950185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c3d53-ecff-4242-8c05-f2cc138eeda4 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:15.976235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:16.032251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103a7f5-4472-4c06-8d95-4b2f47f5023b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:16.055079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400cbe8-e72a-477e-a8a7-057caf362376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:16.056170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400cbe8-e72a-477e-a8a7-057caf362376 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:16.056529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400cbe8-e72a-477e-a8a7-057caf362376 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:16.057263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400cbe8-e72a-477e-a8a7-057caf362376 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:16.058009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400cbe8-e72a-477e-a8a7-057caf362376 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:16.064165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400cbe8-e72a-477e-a8a7-057caf362376 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:16.079055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48587922-1162-49b1-8dbc-59724ed6ba02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:16.080259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48587922-1162-49b1-8dbc-59724ed6ba02 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:16.082019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48587922-1162-49b1-8dbc-59724ed6ba02 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:16.082019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48587922-1162-49b1-8dbc-59724ed6ba02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:16.111284 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1910/3749] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:59:15 2022] POST /identity/v3/users => generated 298 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:16.117426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400cbe8-e72a-477e-a8a7-057caf362376 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:16.121639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48587922-1162-49b1-8dbc-59724ed6ba02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:16.130115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48587922-1162-49b1-8dbc-59724ed6ba02 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:16.147630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400cbe8-e72a-477e-a8a7-057caf362376 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:16.149628 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1840/3750] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:59:16.173835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc2aa0-e1f5-4245-b67f-205bd9eb8961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:16.178389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc2aa0-e1f5-4245-b67f-205bd9eb8961 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:16.179888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc2aa0-e1f5-4245-b67f-205bd9eb8961 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:16.180179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc2aa0-e1f5-4245-b67f-205bd9eb8961 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:16.180883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc2aa0-e1f5-4245-b67f-205bd9eb8961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:16.182634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc2aa0-e1f5-4245-b67f-205bd9eb8961 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:16.225922 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1911/3751] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:59:16.227640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48587922-1162-49b1-8dbc-59724ed6ba02 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:16.251347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48587922-1162-49b1-8dbc-59724ed6ba02 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:16.278016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc25ac7-b0f7-4fe9-9238-f3459595547e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:16.281724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc25ac7-b0f7-4fe9-9238-f3459595547e admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:16.282098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc25ac7-b0f7-4fe9-9238-f3459595547e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:16.285969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc25ac7-b0f7-4fe9-9238-f3459595547e admin admin] PATH_INFO: `/v3/projects/29689e74f89d4e7b9b48d5872770ccb9/users/b7334bfa53c0483e8f5d15c550e0db5a/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:16.355186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc25ac7-b0f7-4fe9-9238-f3459595547e admin admin] RBAC: Authorizing `identity:create_grant(project_id=29689e74f89d4e7b9b48d5872770ccb9, user_id=b7334bfa53c0483e8f5d15c550e0db5a, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:16.368877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc25ac7-b0f7-4fe9-9238-f3459595547e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:16.415458 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1841/3752] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:59:16 2022] PUT /identity/v3/projects/29689e74f89d4e7b9b48d5872770ccb9/users/b7334bfa53c0483e8f5d15c550e0db5a/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:16.416681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc2aa0-e1f5-4245-b67f-205bd9eb8961 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:16.435618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc2aa0-e1f5-4245-b67f-205bd9eb8961 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:16.747773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8195fb-1434-438c-a80a-2a45eea12d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:16.748505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8195fb-1434-438c-a80a-2a45eea12d97 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:16.748872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8195fb-1434-438c-a80a-2a45eea12d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:16.749218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8195fb-1434-438c-a80a-2a45eea12d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:16.750020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8195fb-1434-438c-a80a-2a45eea12d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:16.751760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8195fb-1434-438c-a80a-2a45eea12d97 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:16.782717 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1912/3753] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:59:16.787948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc25ac7-b0f7-4fe9-9238-f3459595547e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:16.819339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc25ac7-b0f7-4fe9-9238-f3459595547e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:16.867668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8195fb-1434-438c-a80a-2a45eea12d97 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:16.870430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2c8195fb-1434-438c-a80a-2a45eea12d97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:16.888025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897175d1-5230-42b0-8c60-2dc9f83b5276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:16.888723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897175d1-5230-42b0-8c60-2dc9f83b5276 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:16.889006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897175d1-5230-42b0-8c60-2dc9f83b5276 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:16.889285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897175d1-5230-42b0-8c60-2dc9f83b5276 admin admin] PATH_INFO: `/v3/projects/29689e74f89d4e7b9b48d5872770ccb9/users/b7334bfa53c0483e8f5d15c550e0db5a/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:16.906403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897175d1-5230-42b0-8c60-2dc9f83b5276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29689e74f89d4e7b9b48d5872770ccb9, user_id=b7334bfa53c0483e8f5d15c550e0db5a, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:16.909145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897175d1-5230-42b0-8c60-2dc9f83b5276 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:16.909276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8195fb-1434-438c-a80a-2a45eea12d97 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:16.924278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab0f42-5274-474a-a6eb-40ca0549a9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:16.924925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab0f42-5274-474a-a6eb-40ca0549a9d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:16.925205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab0f42-5274-474a-a6eb-40ca0549a9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:16.925679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab0f42-5274-474a-a6eb-40ca0549a9d0 admin admin] PATH_INFO: `/v3/projects/2889ffe1cedd48f7b88f68b121a47d3b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:16.926857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab0f42-5274-474a-a6eb-40ca0549a9d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2889ffe1cedd48f7b88f68b121a47d3b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:16.928764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab0f42-5274-474a-a6eb-40ca0549a9d0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:16.932512 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1842/3754] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:59:16 2022] PUT /identity/v3/projects/29689e74f89d4e7b9b48d5872770ccb9/users/b7334bfa53c0483e8f5d15c550e0db5a/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:59:16.976021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:17.059590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b6ab0f42-5274-474a-a6eb-40ca0549a9d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2889ffe1cedd48f7b88f68b121a47d3b', 'request_id': 'req-b6ab0f42-5274-474a-a6eb-40ca0549a9d0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:17.076043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b6ab0f42-5274-474a-a6eb-40ca0549a9d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2889ffe1cedd48f7b88f68b121a47d3b', 'request_id': 'req-b6ab0f42-5274-474a-a6eb-40ca0549a9d0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:17.092924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b6ab0f42-5274-474a-a6eb-40ca0549a9d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2889ffe1cedd48f7b88f68b121a47d3b', 'request_id': 'req-b6ab0f42-5274-474a-a6eb-40ca0549a9d0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:17.119742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b6ab0f42-5274-474a-a6eb-40ca0549a9d0 admin admin] The token cache is being invalidate because project 2889ffe1cedd48f7b88f68b121a47d3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:59:17.119974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b6ab0f42-5274-474a-a6eb-40ca0549a9d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:17.121540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b6ab0f42-5274-474a-a6eb-40ca0549a9d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:17.130222 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1913/3755] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:59:16 2022] DELETE /identity/v3/projects/2889ffe1cedd48f7b88f68b121a47d3b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:17.131998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897175d1-5230-42b0-8c60-2dc9f83b5276 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:17.134345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-897175d1-5230-42b0-8c60-2dc9f83b5276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:17.420753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897175d1-5230-42b0-8c60-2dc9f83b5276 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:17.729380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f791bb9-1317-4ea7-b37c-9d55e297e880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:17.730055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f791bb9-1317-4ea7-b37c-9d55e297e880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:17.730340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f791bb9-1317-4ea7-b37c-9d55e297e880 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:17.730651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f791bb9-1317-4ea7-b37c-9d55e297e880 admin admin] PATH_INFO: `/v3/users/07a9899351f84707aa4c7f9c1e1fe6d5` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:17.733571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f791bb9-1317-4ea7-b37c-9d55e297e880 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07a9899351f84707aa4c7f9c1e1fe6d5)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:17.735584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f791bb9-1317-4ea7-b37c-9d55e297e880 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:17.799107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:17.834163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2f791bb9-1317-4ea7-b37c-9d55e297e880 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07a9899351f84707aa4c7f9c1e1fe6d5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:17.838965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2f791bb9-1317-4ea7-b37c-9d55e297e880 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07a9899351f84707aa4c7f9c1e1fe6d5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:17.839768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2f791bb9-1317-4ea7-b37c-9d55e297e880 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07a9899351f84707aa4c7f9c1e1fe6d5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:17.854154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2f791bb9-1317-4ea7-b37c-9d55e297e880 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07a9899351f84707aa4c7f9c1e1fe6d5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:17.887510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2f791bb9-1317-4ea7-b37c-9d55e297e880 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07a9899351f84707aa4c7f9c1e1fe6d5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:17.921908 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1843/3756] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:59:17 2022] DELETE /identity/v3/users/07a9899351f84707aa4c7f9c1e1fe6d5 => generated 0 bytes in 790 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:19.105753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab0f42-5274-474a-a6eb-40ca0549a9d0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:19.121054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab0f42-5274-474a-a6eb-40ca0549a9d0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:19.126328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f791bb9-1317-4ea7-b37c-9d55e297e880 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:19.128584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2f791bb9-1317-4ea7-b37c-9d55e297e880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:19.208112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7b1d0-ad18-4b06-b7ae-bb6de015e160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:19.209487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7b1d0-ad18-4b06-b7ae-bb6de015e160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:19.209950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7b1d0-ad18-4b06-b7ae-bb6de015e160 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:19.210229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7b1d0-ad18-4b06-b7ae-bb6de015e160 admin admin] PATH_INFO: `/v3/projects/4d3d2fd9eb3c4263aa9d50d4a574cfab` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:19.212662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7b1d0-ad18-4b06-b7ae-bb6de015e160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d3d2fd9eb3c4263aa9d50d4a574cfab)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:19.220148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7b1d0-ad18-4b06-b7ae-bb6de015e160 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:19.327971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f791bb9-1317-4ea7-b37c-9d55e297e880 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:19.333579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-47a7b1d0-ad18-4b06-b7ae-bb6de015e160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d3d2fd9eb3c4263aa9d50d4a574cfab', 'request_id': 'req-47a7b1d0-ad18-4b06-b7ae-bb6de015e160'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:19.350058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-47a7b1d0-ad18-4b06-b7ae-bb6de015e160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d3d2fd9eb3c4263aa9d50d4a574cfab', 'request_id': 'req-47a7b1d0-ad18-4b06-b7ae-bb6de015e160'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:19.366989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-47a7b1d0-ad18-4b06-b7ae-bb6de015e160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d3d2fd9eb3c4263aa9d50d4a574cfab', 'request_id': 'req-47a7b1d0-ad18-4b06-b7ae-bb6de015e160'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:19.397868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-47a7b1d0-ad18-4b06-b7ae-bb6de015e160 admin admin] The token cache is being invalidate because project 4d3d2fd9eb3c4263aa9d50d4a574cfab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:59:19.399774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-47a7b1d0-ad18-4b06-b7ae-bb6de015e160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:19.401288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-47a7b1d0-ad18-4b06-b7ae-bb6de015e160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:19.408443 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1914/3757] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:59:19 2022] DELETE /identity/v3/projects/4d3d2fd9eb3c4263aa9d50d4a574cfab => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:19.412626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e5eb0-d84e-45ce-9ed1-57bf2620da04 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:19.412919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e5eb0-d84e-45ce-9ed1-57bf2620da04 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:19.415608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e5eb0-d84e-45ce-9ed1-57bf2620da04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:19.472806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a58e5eb0-d84e-45ce-9ed1-57bf2620da04 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:19.476180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a58e5eb0-d84e-45ce-9ed1-57bf2620da04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:19.503372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaed821-9e8d-48c0-ab90-d5f3a6962be1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:19.505262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaed821-9e8d-48c0-ab90-d5f3a6962be1 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:19.506660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaed821-9e8d-48c0-ab90-d5f3a6962be1 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:19.508003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaed821-9e8d-48c0-ab90-d5f3a6962be1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:19.531112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9eaed821-9e8d-48c0-ab90-d5f3a6962be1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:19.550644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:19.580084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaed821-9e8d-48c0-ab90-d5f3a6962be1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:19.582029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaed821-9e8d-48c0-ab90-d5f3a6962be1 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:19.638238 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1844/3758] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:59:19.640691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaed821-9e8d-48c0-ab90-d5f3a6962be1 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:19.647030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9eaed821-9e8d-48c0-ab90-d5f3a6962be1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:19.647240 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1915/3759] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:19.659225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f4b8f-0f43-459e-8ee5-c6c51abcf5cb None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:19.659519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f4b8f-0f43-459e-8ee5-c6c51abcf5cb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:19.659802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f4b8f-0f43-459e-8ee5-c6c51abcf5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:19.727504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-de5f4b8f-0f43-459e-8ee5-c6c51abcf5cb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:19.731092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-de5f4b8f-0f43-459e-8ee5-c6c51abcf5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:19.753464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaed821-9e8d-48c0-ab90-d5f3a6962be1 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:19.774200 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1916/3760] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:19.775805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f4b8f-0f43-459e-8ee5-c6c51abcf5cb None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:19.780155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-de5f4b8f-0f43-459e-8ee5-c6c51abcf5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:19.812420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644c354-edab-47bc-8cc5-d7c72dd0f009 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:19.813053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644c354-edab-47bc-8cc5-d7c72dd0f009 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:19.813472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644c354-edab-47bc-8cc5-d7c72dd0f009 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:19.813756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644c354-edab-47bc-8cc5-d7c72dd0f009 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:19.820430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7644c354-edab-47bc-8cc5-d7c72dd0f009 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:19.840306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f4b8f-0f43-459e-8ee5-c6c51abcf5cb None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:19.857246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644c354-edab-47bc-8cc5-d7c72dd0f009 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:19.863039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff98e2-e66a-4b94-b3d8-6acd7d765902 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:19.863424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644c354-edab-47bc-8cc5-d7c72dd0f009 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:19.863666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff98e2-e66a-4b94-b3d8-6acd7d765902 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:19.863963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff98e2-e66a-4b94-b3d8-6acd7d765902 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:19.864238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff98e2-e66a-4b94-b3d8-6acd7d765902 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:19.867736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4aff98e2-e66a-4b94-b3d8-6acd7d765902 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:19.904330 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1845/3761] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:59:19.906248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644c354-edab-47bc-8cc5-d7c72dd0f009 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:19.918372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff98e2-e66a-4b94-b3d8-6acd7d765902 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:19.925641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff98e2-e66a-4b94-b3d8-6acd7d765902 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:19.948119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644c354-edab-47bc-8cc5-d7c72dd0f009 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:19.969255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534782ab-44b6-4d2a-b3bd-29865b3b4f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:19.969883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534782ab-44b6-4d2a-b3bd-29865b3b4f5a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:19.970157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534782ab-44b6-4d2a-b3bd-29865b3b4f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:19.970445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534782ab-44b6-4d2a-b3bd-29865b3b4f5a admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:19.972155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534782ab-44b6-4d2a-b3bd-29865b3b4f5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:59:19.972455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534782ab-44b6-4d2a-b3bd-29865b3b4f5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:19.983155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534782ab-44b6-4d2a-b3bd-29865b3b4f5a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:19.984868 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1917/3762] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:59:20.080751 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1846/3763] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:59:20.084139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff98e2-e66a-4b94-b3d8-6acd7d765902 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:20.119309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff98e2-e66a-4b94-b3d8-6acd7d765902 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:20.151974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dea47a-27e6-4495-8b00-2cb1e5238f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:20.152600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dea47a-27e6-4495-8b00-2cb1e5238f62 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:20.152881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dea47a-27e6-4495-8b00-2cb1e5238f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:20.153153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dea47a-27e6-4495-8b00-2cb1e5238f62 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:20.153786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dea47a-27e6-4495-8b00-2cb1e5238f62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:20.156283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dea47a-27e6-4495-8b00-2cb1e5238f62 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:20.189376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534782ab-44b6-4d2a-b3bd-29865b3b4f5a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:20.191970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-534782ab-44b6-4d2a-b3bd-29865b3b4f5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:20.207934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:20.324763 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1918/3764] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:59:20 2022] POST /identity/v3/projects => generated 371 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:20.324763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dea47a-27e6-4495-8b00-2cb1e5238f62 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:20.434951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534782ab-44b6-4d2a-b3bd-29865b3b4f5a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:20.547069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dea47a-27e6-4495-8b00-2cb1e5238f62 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:20.564082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fe805-665c-4068-ae56-cf842784d5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:20.564725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fe805-665c-4068-ae56-cf842784d5a7 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:20.565005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fe805-665c-4068-ae56-cf842784d5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:20.565280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fe805-665c-4068-ae56-cf842784d5a7 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:20.565919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742fe805-665c-4068-ae56-cf842784d5a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:20.568032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742fe805-665c-4068-ae56-cf842784d5a7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:20.631359 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1919/3765] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:59:20 2022] POST /identity/v3/users => generated 286 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:20.632395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fe805-665c-4068-ae56-cf842784d5a7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:20.659538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fe805-665c-4068-ae56-cf842784d5a7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:20.674117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229074a-bcd7-4a44-b4fc-2457a8e5aaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:20.676232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229074a-bcd7-4a44-b4fc-2457a8e5aaf3 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:20.676970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229074a-bcd7-4a44-b4fc-2457a8e5aaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:20.678337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31209a39-4c4b-46e1-83f6-ac209a12b513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:20.678910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229074a-bcd7-4a44-b4fc-2457a8e5aaf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:20.679722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a229074a-bcd7-4a44-b4fc-2457a8e5aaf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:20.681766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a229074a-bcd7-4a44-b4fc-2457a8e5aaf3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:20.685668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31209a39-4c4b-46e1-83f6-ac209a12b513 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:20.685950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31209a39-4c4b-46e1-83f6-ac209a12b513 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:20.686222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31209a39-4c4b-46e1-83f6-ac209a12b513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:20.695777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-31209a39-4c4b-46e1-83f6-ac209a12b513 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:20.718556 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1920/3766] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:59:20.727606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31209a39-4c4b-46e1-83f6-ac209a12b513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:20.729243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31209a39-4c4b-46e1-83f6-ac209a12b513 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:20.731910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229074a-bcd7-4a44-b4fc-2457a8e5aaf3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:20.760540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229074a-bcd7-4a44-b4fc-2457a8e5aaf3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:20.764055 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1847/3767] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:59:20.773571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4f027-5578-4bc6-9d40-293a9fb83a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:20.774379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4f027-5578-4bc6-9d40-293a9fb83a0c admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:20.775430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4f027-5578-4bc6-9d40-293a9fb83a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:20.776956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4f027-5578-4bc6-9d40-293a9fb83a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:20.781304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4f027-5578-4bc6-9d40-293a9fb83a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:20.785556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4f027-5578-4bc6-9d40-293a9fb83a0c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:20.829317 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1921/3768] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:59:20.830338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31209a39-4c4b-46e1-83f6-ac209a12b513 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:20.857629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31209a39-4c4b-46e1-83f6-ac209a12b513 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:20.873804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8e687-5951-478a-84c8-d24a0fa83d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:20.875374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8e687-5951-478a-84c8-d24a0fa83d9b admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:20.875882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8e687-5951-478a-84c8-d24a0fa83d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:20.876363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8e687-5951-478a-84c8-d24a0fa83d9b admin admin] PATH_INFO: `/v3/projects/9c9cfa2e7f5348b4a525294ba662e4e6/users/a5c22b659fed42d1b5bc3477d267f76d/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:20.954256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8e687-5951-478a-84c8-d24a0fa83d9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c9cfa2e7f5348b4a525294ba662e4e6, user_id=a5c22b659fed42d1b5bc3477d267f76d, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:20.963675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8e687-5951-478a-84c8-d24a0fa83d9b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:20.997839 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1848/3769] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:59:20 2022] PUT /identity/v3/projects/9c9cfa2e7f5348b4a525294ba662e4e6/users/a5c22b659fed42d1b5bc3477d267f76d/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:59:21.009326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e19758-1bb1-475c-bf30-8924bdf2196c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:21.009613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e19758-1bb1-475c-bf30-8924bdf2196c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:21.009887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e19758-1bb1-475c-bf30-8924bdf2196c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:21.105101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-70e19758-1bb1-475c-bf30-8924bdf2196c None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:21.109328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-70e19758-1bb1-475c-bf30-8924bdf2196c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:21.258770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:21.337054 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1922/3770] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:21.340832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8e687-5951-478a-84c8-d24a0fa83d9b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:21.363619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8e687-5951-478a-84c8-d24a0fa83d9b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:21.378179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a2c695-40a7-496e-862d-416c9bfc3f3d tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c22b659fed42d1b5bc3477d267f76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c9cfa2e7f5348b4a525294ba662e4e6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:21.381887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a2c695-40a7-496e-862d-416c9bfc3f3d tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:21.389464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a2c695-40a7-496e-862d-416c9bfc3f3d tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:21.390002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a2c695-40a7-496e-862d-416c9bfc3f3d tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] PATH_INFO: `/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:21.527205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-85a2c695-40a7-496e-862d-416c9bfc3f3d tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'a5c22b659fed42d1b5bc3477d267f76d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:21.576046 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1849/3771] 10.222.0.17 () {60 vars in 1269 bytes} [Sat Oct 1 20:59:21 2022] POST /identity/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:22.582144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b63eb-d583-480f-a9f6-a1350c3f65fb None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:22.582751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b63eb-d583-480f-a9f6-a1350c3f65fb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:22.583138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b63eb-d583-480f-a9f6-a1350c3f65fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:22.639483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d975f1c-7c9a-40c4-abf9-4c0a6115cf6b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:22.640126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d975f1c-7c9a-40c4-abf9-4c0a6115cf6b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:22.640293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d975f1c-7c9a-40c4-abf9-4c0a6115cf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:22.660385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ff6b63eb-d583-480f-a9f6-a1350c3f65fb None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:22.667287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ff6b63eb-d583-480f-a9f6-a1350c3f65fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:22.681348 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1923/3772] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:22 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:59:22.687906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c98d2-3fcb-470f-8757-005fff3729d1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:22.688651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c98d2-3fcb-470f-8757-005fff3729d1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:22.689153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c98d2-3fcb-470f-8757-005fff3729d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:22.726810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7d975f1c-7c9a-40c4-abf9-4c0a6115cf6b None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:22.731951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7d975f1c-7c9a-40c4-abf9-4c0a6115cf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:22.799117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9b6c98d2-3fcb-470f-8757-005fff3729d1 None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:22.804734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9b6c98d2-3fcb-470f-8757-005fff3729d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:22.830940 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1924/3773] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:22.833429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c98d2-3fcb-470f-8757-005fff3729d1 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:22.865199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c98d2-3fcb-470f-8757-005fff3729d1 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:22.885688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf40569-e3d3-4113-ba5b-3b1c78e3b4a3 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c22b659fed42d1b5bc3477d267f76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c9cfa2e7f5348b4a525294ba662e4e6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:22.889370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf40569-e3d3-4113-ba5b-3b1c78e3b4a3 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:22.891829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf40569-e3d3-4113-ba5b-3b1c78e3b4a3 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:22.900532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf40569-e3d3-4113-ba5b-3b1c78e3b4a3 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] PATH_INFO: `/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:23.026023 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1850/3774] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 20:59:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:23.110446 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-9bf40569-e3d3-4113-ba5b-3b1c78e3b4a3 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2.: keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Oct 01 20:59:23.110446 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:59:23.110446 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:59:23.110446 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:59:23.110446 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:59:23.110446 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:59:23.110446 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:59:23.110446 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:59:23.110446 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:59:23.110446 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:59:23.110446 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:59:23.110446 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:59:23.110446 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 01 20:59:23.111977 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 01 20:59:23.111977 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 291, in post Oct 01 20:59:23.111977 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application PROVIDERS.identity_api.change_password( Oct 01 20:59:23.111977 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:59:23.111977 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:59:23.111977 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:59:23.111977 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:59:23.111977 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1468, in change_password Oct 01 20:59:23.111977 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application driver.change_password(entity_id, new_password) Oct 01 20:59:23.111977 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 268, in change_password Oct 01 20:59:23.111977 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._validate_password_history(new_password, user_ref) Oct 01 20:59:23.111977 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 256, in _validate_password_history Oct 01 20:59:23.111977 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.PasswordHistoryValidationError( Oct 01 20:59:23.111977 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Oct 01 20:59:23.111977 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:59:23.114039 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1925/3775] 10.222.0.17 () {60 vars in 1269 bytes} [Sat Oct 1 20:59:22 2022] POST /identity/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password => generated 188 bytes in 281 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 01 20:59:23.116385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d975f1c-7c9a-40c4-abf9-4c0a6115cf6b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:23.133319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d975f1c-7c9a-40c4-abf9-4c0a6115cf6b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:23.144884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8a272-e40a-44c1-9349-431814dd10cb tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c22b659fed42d1b5bc3477d267f76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c9cfa2e7f5348b4a525294ba662e4e6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:23.145554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8a272-e40a-44c1-9349-431814dd10cb tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:23.146033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8a272-e40a-44c1-9349-431814dd10cb tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:23.146321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8a272-e40a-44c1-9349-431814dd10cb tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] PATH_INFO: `/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:23.170370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:23.176067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf40569-e3d3-4113-ba5b-3b1c78e3b4a3 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:23.178337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9bf40569-e3d3-4113-ba5b-3b1c78e3b4a3 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:23.251787 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-78e8a272-e40a-44c1-9349-431814dd10cb tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2.: keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Oct 01 20:59:23.251787 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:59:23.251787 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:59:23.251787 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:59:23.251787 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:59:23.251787 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:59:23.251787 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:59:23.251787 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:59:23.251787 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:59:23.251787 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:59:23.251787 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:59:23.251787 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:59:23.251787 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 01 20:59:23.252832 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 01 20:59:23.252832 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 291, in post Oct 01 20:59:23.252832 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application PROVIDERS.identity_api.change_password( Oct 01 20:59:23.252832 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:59:23.252832 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:59:23.252832 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 20:59:23.252832 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 20:59:23.252832 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1468, in change_password Oct 01 20:59:23.252832 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application driver.change_password(entity_id, new_password) Oct 01 20:59:23.252832 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 268, in change_password Oct 01 20:59:23.252832 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._validate_password_history(new_password, user_ref) Oct 01 20:59:23.252832 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 256, in _validate_password_history Oct 01 20:59:23.252832 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.PasswordHistoryValidationError( Oct 01 20:59:23.252832 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Oct 01 20:59:23.252832 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 20:59:23.256255 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1851/3776] 10.222.0.17 () {60 vars in 1269 bytes} [Sat Oct 1 20:59:23 2022] POST /identity/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password => generated 188 bytes in 140 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 01 20:59:23.261049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8a272-e40a-44c1-9349-431814dd10cb tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:23.291415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8a272-e40a-44c1-9349-431814dd10cb tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:23.318126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed31f46-fc5d-4223-94ec-ff2c1d5788d7 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c22b659fed42d1b5bc3477d267f76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c9cfa2e7f5348b4a525294ba662e4e6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:23.321236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed31f46-fc5d-4223-94ec-ff2c1d5788d7 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:23.321720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed31f46-fc5d-4223-94ec-ff2c1d5788d7 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:23.322173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed31f46-fc5d-4223-94ec-ff2c1d5788d7 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] PATH_INFO: `/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:23.447534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf40569-e3d3-4113-ba5b-3b1c78e3b4a3 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:23.456480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda1bf2-dd4b-4e06-b9fb-59694f7a171e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:23.457317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda1bf2-dd4b-4e06-b9fb-59694f7a171e service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:23.457650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda1bf2-dd4b-4e06-b9fb-59694f7a171e service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:23.457965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda1bf2-dd4b-4e06-b9fb-59694f7a171e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:23.473940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda1bf2-dd4b-4e06-b9fb-59694f7a171e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:23.475987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda1bf2-dd4b-4e06-b9fb-59694f7a171e service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:23.498954 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1926/3777] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 20:59:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 20:59:23.517389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7ed31f46-fc5d-4223-94ec-ff2c1d5788d7 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'a5c22b659fed42d1b5bc3477d267f76d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:23.576794 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1852/3778] 10.222.0.17 () {60 vars in 1269 bytes} [Sat Oct 1 20:59:23 2022] POST /identity/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:24.343394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543809b-b4c1-4ca2-99fa-7b0b2bf397ae None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:24.343963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543809b-b4c1-4ca2-99fa-7b0b2bf397ae None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:24.344660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543809b-b4c1-4ca2-99fa-7b0b2bf397ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:24.422444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f543809b-b4c1-4ca2-99fa-7b0b2bf397ae None None] MFA Rules not processed for user `775ff9091a794faaacc05a37550c8a72`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:24.432905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f543809b-b4c1-4ca2-99fa-7b0b2bf397ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:24.580783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db3048-3df9-4954-9506-4cad51cda1cd None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:24.581757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db3048-3df9-4954-9506-4cad51cda1cd None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:24.582556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db3048-3df9-4954-9506-4cad51cda1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:24.637862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f1db3048-3df9-4954-9506-4cad51cda1cd None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:24.644556 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1927/3779] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:24.646077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f1db3048-3df9-4954-9506-4cad51cda1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:24.658210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7d4f4-ae7e-42fc-827d-67f96a280a4c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:24.658546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7d4f4-ae7e-42fc-827d-67f96a280a4c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:24.658846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7d4f4-ae7e-42fc-827d-67f96a280a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:24.667416 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1853/3780] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:24 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:59:24.735481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c5e7d4f4-ae7e-42fc-827d-67f96a280a4c None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:24.745287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c5e7d4f4-ae7e-42fc-827d-67f96a280a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:24.757814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db3048-3df9-4954-9506-4cad51cda1cd None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:24.759284 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1928/3781] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:24.777882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db3048-3df9-4954-9506-4cad51cda1cd None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:24.786044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487d27c-7872-4737-94fc-066e262b615a tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c22b659fed42d1b5bc3477d267f76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c9cfa2e7f5348b4a525294ba662e4e6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:24.787072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487d27c-7872-4737-94fc-066e262b615a tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:24.787563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487d27c-7872-4737-94fc-066e262b615a tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:24.788019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487d27c-7872-4737-94fc-066e262b615a tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] PATH_INFO: `/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:24.918816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4487d27c-7872-4737-94fc-066e262b615a tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'a5c22b659fed42d1b5bc3477d267f76d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:24.946142 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1854/3782] 10.222.0.17 () {60 vars in 1269 bytes} [Sat Oct 1 20:59:24 2022] POST /identity/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:25.950020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014212f6-1d1f-41c3-b699-f95eff7e33a9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:25.950976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014212f6-1d1f-41c3-b699-f95eff7e33a9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:25.951431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014212f6-1d1f-41c3-b699-f95eff7e33a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:25.985295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-014212f6-1d1f-41c3-b699-f95eff7e33a9 None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:25.987475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-014212f6-1d1f-41c3-b699-f95eff7e33a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:26.001655 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1929/3783] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:25 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:59:26.003508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f7b64-cac1-43e6-b19e-e5f6ca016a95 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:26.004022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f7b64-cac1-43e6-b19e-e5f6ca016a95 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:26.004310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f7b64-cac1-43e6-b19e-e5f6ca016a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:26.036418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0c0f7b64-cac1-43e6-b19e-e5f6ca016a95 None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:26.039817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0c0f7b64-cac1-43e6-b19e-e5f6ca016a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:26.051235 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1855/3784] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:26.055268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014212f6-1d1f-41c3-b699-f95eff7e33a9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:26.071210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014212f6-1d1f-41c3-b699-f95eff7e33a9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:26.077234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c6b87-16eb-41fe-9798-acc49fde20ad tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c22b659fed42d1b5bc3477d267f76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c9cfa2e7f5348b4a525294ba662e4e6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:26.078317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c6b87-16eb-41fe-9798-acc49fde20ad tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:26.078875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c6b87-16eb-41fe-9798-acc49fde20ad tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:26.079378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c6b87-16eb-41fe-9798-acc49fde20ad tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] PATH_INFO: `/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:26.203535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-aa5c6b87-16eb-41fe-9798-acc49fde20ad tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'a5c22b659fed42d1b5bc3477d267f76d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:26.206839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:26.222309 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1930/3785] 10.222.0.17 () {60 vars in 1269 bytes} [Sat Oct 1 20:59:26 2022] POST /identity/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:59:27.229958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bd350-059e-423e-963d-004c5162373c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:27.230548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bd350-059e-423e-963d-004c5162373c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:27.230548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bd350-059e-423e-963d-004c5162373c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:27.267814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6b4bd350-059e-423e-963d-004c5162373c None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:27.270082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6b4bd350-059e-423e-963d-004c5162373c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:27.278385 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1856/3786] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:27 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:59:27.285028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49df770-9587-4c20-bdc1-830a3bde0d8e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:27.286238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49df770-9587-4c20-bdc1-830a3bde0d8e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:27.286781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49df770-9587-4c20-bdc1-830a3bde0d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:27.324337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a49df770-9587-4c20-bdc1-830a3bde0d8e None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:27.327246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a49df770-9587-4c20-bdc1-830a3bde0d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:27.335735 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1931/3787] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:27.339520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bd350-059e-423e-963d-004c5162373c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:27.354938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bd350-059e-423e-963d-004c5162373c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:27.362106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff0d13-7fcc-49ae-9df1-b6ae0d978fb9 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c22b659fed42d1b5bc3477d267f76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c9cfa2e7f5348b4a525294ba662e4e6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:27.362982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff0d13-7fcc-49ae-9df1-b6ae0d978fb9 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:27.372217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff0d13-7fcc-49ae-9df1-b6ae0d978fb9 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:27.372800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff0d13-7fcc-49ae-9df1-b6ae0d978fb9 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] PATH_INFO: `/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:27.454407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2eff0d13-7fcc-49ae-9df1-b6ae0d978fb9 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'a5c22b659fed42d1b5bc3477d267f76d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:27.468998 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1857/3788] 10.222.0.17 () {60 vars in 1269 bytes} [Sat Oct 1 20:59:27 2022] POST /identity/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:27.891603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4157db-641e-4312-8339-a2c209e1e964 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:27.892143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4157db-641e-4312-8339-a2c209e1e964 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:27.892596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4157db-641e-4312-8339-a2c209e1e964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:28.011397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ac4157db-641e-4312-8339-a2c209e1e964 None None] MFA Rules not processed for user `b7334bfa53c0483e8f5d15c550e0db5a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:28.026171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ac4157db-641e-4312-8339-a2c209e1e964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:28.246776 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1932/3789] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:28.473659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3a9f9-1e4c-4941-aa10-079b6e216de0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:28.474836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3a9f9-1e4c-4941-aa10-079b6e216de0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:28.475466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3a9f9-1e4c-4941-aa10-079b6e216de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:28.515442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-28a3a9f9-1e4c-4941-aa10-079b6e216de0 None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:28.517263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-28a3a9f9-1e4c-4941-aa10-079b6e216de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:28.521656 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1858/3790] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:28 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:59:29.330848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4157db-641e-4312-8339-a2c209e1e964 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:29.421203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4157db-641e-4312-8339-a2c209e1e964 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:29.427148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a20a3-f0b9-407d-8748-55351c941773 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:29.427944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a20a3-f0b9-407d-8748-55351c941773 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:29.428223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a20a3-f0b9-407d-8748-55351c941773 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:29.428490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a20a3-f0b9-407d-8748-55351c941773 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:29.430285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-480a20a3-f0b9-407d-8748-55351c941773 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:29.494763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a20a3-f0b9-407d-8748-55351c941773 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:29.496378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a20a3-f0b9-407d-8748-55351c941773 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:29.506290 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1933/3791] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:59:29.527391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f5671-b924-4502-992d-28db827a40f5 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:29.528300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f5671-b924-4502-992d-28db827a40f5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:29.528988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f5671-b924-4502-992d-28db827a40f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:29.552796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f54f5671-b924-4502-992d-28db827a40f5 None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:29.554191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f54f5671-b924-4502-992d-28db827a40f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:29.561943 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1859/3792] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:29.568274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dcb66-8fa5-41fd-8480-f2b75bd7210a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:29.578239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dcb66-8fa5-41fd-8480-f2b75bd7210a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:29.578239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dcb66-8fa5-41fd-8480-f2b75bd7210a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:29.610584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e31dcb66-8fa5-41fd-8480-f2b75bd7210a None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:29.611815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e31dcb66-8fa5-41fd-8480-f2b75bd7210a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:29.615397 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1934/3793] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:29 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:59:29.619543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c9f6c-6abb-4647-8b48-c007bfb0d06f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:29.619839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c9f6c-6abb-4647-8b48-c007bfb0d06f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:29.620120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c9f6c-6abb-4647-8b48-c007bfb0d06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:29.706566 ncdv-master-1 devstack@keystone.service[99140]: WARNING keystone.server.flask.application [None req-e83c9f6c-6abb-4647-8b48-c007bfb0d06f None None] Authorization failed. The request you have made requires authentication. from 10.222.0.17: keystone.exception.Unauthorized: The request you have made requires authentication. Oct 01 20:59:29.708793 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1860/3794] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:29 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 90 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Oct 01 20:59:29.712847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a598b4-9f24-42e0-b9cc-8b8176a1a651 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:29.713127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a598b4-9f24-42e0-b9cc-8b8176a1a651 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:29.713422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a598b4-9f24-42e0-b9cc-8b8176a1a651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:29.724236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:29.810754 ncdv-master-1 devstack@keystone.service[99139]: WARNING keystone.server.flask.application [None req-71a598b4-9f24-42e0-b9cc-8b8176a1a651 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.17: keystone.exception.Unauthorized: The request you have made requires authentication. Oct 01 20:59:29.824742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d8eb1-63bc-4f8c-bb86-444901dc9c0b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:29.825663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d8eb1-63bc-4f8c-bb86-444901dc9c0b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:29.826430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d8eb1-63bc-4f8c-bb86-444901dc9c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:29.835399 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1935/3795] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:29 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 119 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Oct 01 20:59:29.861494 ncdv-master-1 devstack@keystone.service[99140]: WARNING keystone.server.flask.application [None req-e01d8eb1-63bc-4f8c-bb86-444901dc9c0b None None] Authorization failed. The request you have made requires authentication. from 10.222.0.17: keystone.exception.Unauthorized: The request you have made requires authentication. Oct 01 20:59:29.866261 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1861/3796] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:29 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 42 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Oct 01 20:59:32.011512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a598b4-9f24-42e0-b9cc-8b8176a1a651 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:32.013516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-71a598b4-9f24-42e0-b9cc-8b8176a1a651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:32.178220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a598b4-9f24-42e0-b9cc-8b8176a1a651 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:32.184287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f6da9-63d8-4b9e-b9bb-d4b26781fbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:32.185955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f6da9-63d8-4b9e-b9bb-d4b26781fbbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:32.187279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f6da9-63d8-4b9e-b9bb-d4b26781fbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:32.188110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f6da9-63d8-4b9e-b9bb-d4b26781fbbf admin admin] PATH_INFO: `/v3/users/4e76b153982141d8964ecbb118d19eab` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:32.189696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f6da9-63d8-4b9e-b9bb-d4b26781fbbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e76b153982141d8964ecbb118d19eab)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:32.194157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f6da9-63d8-4b9e-b9bb-d4b26781fbbf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:32.273771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ee4f6da9-63d8-4b9e-b9bb-d4b26781fbbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e76b153982141d8964ecbb118d19eab'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:32.288860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ee4f6da9-63d8-4b9e-b9bb-d4b26781fbbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e76b153982141d8964ecbb118d19eab'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:32.313311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ee4f6da9-63d8-4b9e-b9bb-d4b26781fbbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e76b153982141d8964ecbb118d19eab'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:32.317488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ee4f6da9-63d8-4b9e-b9bb-d4b26781fbbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e76b153982141d8964ecbb118d19eab'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:32.331364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ee4f6da9-63d8-4b9e-b9bb-d4b26781fbbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e76b153982141d8964ecbb118d19eab'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:32.342700 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1936/3797] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:59:32 2022] DELETE /identity/v3/users/4e76b153982141d8964ecbb118d19eab => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:32.346589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de79364-e12a-4909-aa51-66bfbf4284cd None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:32.347053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de79364-e12a-4909-aa51-66bfbf4284cd None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:32.347134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de79364-e12a-4909-aa51-66bfbf4284cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:32.382362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7de79364-e12a-4909-aa51-66bfbf4284cd None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:32.385211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7de79364-e12a-4909-aa51-66bfbf4284cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:32.442315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=85,keystone:UPDATE=4,keystone:INSERT=6,keystone:DELETE=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:32.536945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2656f9-c55d-45d1-ba9e-74b5745d2a55 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:32.537243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2656f9-c55d-45d1-ba9e-74b5745d2a55 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:32.537524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2656f9-c55d-45d1-ba9e-74b5745d2a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:32.540014 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1862/3798] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:32.548852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de79364-e12a-4909-aa51-66bfbf4284cd None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:32.550781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7de79364-e12a-4909-aa51-66bfbf4284cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:32.569475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9f2656f9-c55d-45d1-ba9e-74b5745d2a55 None None] MFA Rules not processed for user `69002d46d4e54eadadf911ec3589d95c`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:32.571128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9f2656f9-c55d-45d1-ba9e-74b5745d2a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:32.667169 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1937/3799] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:59:32.683710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de79364-e12a-4909-aa51-66bfbf4284cd None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:32.689153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b52c7-afd5-42fe-956d-48f5287fb5c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:32.689944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b52c7-afd5-42fe-956d-48f5287fb5c3 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:32.690227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b52c7-afd5-42fe-956d-48f5287fb5c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:32.690723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b52c7-afd5-42fe-956d-48f5287fb5c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:32.706663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b52c7-afd5-42fe-956d-48f5287fb5c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:32.709086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b52c7-afd5-42fe-956d-48f5287fb5c3 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:32.721681 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1863/3800] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:59:32.925770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2656f9-c55d-45d1-ba9e-74b5745d2a55 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:32.927829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9f2656f9-c55d-45d1-ba9e-74b5745d2a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:32.950165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2656f9-c55d-45d1-ba9e-74b5745d2a55 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:32.955597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bed2ef-39e7-4143-92be-e9d6614e87f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:32.957042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bed2ef-39e7-4143-92be-e9d6614e87f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:32.957502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bed2ef-39e7-4143-92be-e9d6614e87f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:32.957772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bed2ef-39e7-4143-92be-e9d6614e87f1 admin admin] PATH_INFO: `/v3/projects/3cf38b1a4bc4458199fea3373632117d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:32.959215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bed2ef-39e7-4143-92be-e9d6614e87f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cf38b1a4bc4458199fea3373632117d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:32.961175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bed2ef-39e7-4143-92be-e9d6614e87f1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:32.964059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:UPDATE=1,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:33.034036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-53bed2ef-39e7-4143-92be-e9d6614e87f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cf38b1a4bc4458199fea3373632117d', 'request_id': 'req-53bed2ef-39e7-4143-92be-e9d6614e87f1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:33.043155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-53bed2ef-39e7-4143-92be-e9d6614e87f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cf38b1a4bc4458199fea3373632117d', 'request_id': 'req-53bed2ef-39e7-4143-92be-e9d6614e87f1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:33.053323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-53bed2ef-39e7-4143-92be-e9d6614e87f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cf38b1a4bc4458199fea3373632117d', 'request_id': 'req-53bed2ef-39e7-4143-92be-e9d6614e87f1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:33.096390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-53bed2ef-39e7-4143-92be-e9d6614e87f1 admin admin] The token cache is being invalidate because project 3cf38b1a4bc4458199fea3373632117d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:59:33.096714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-53bed2ef-39e7-4143-92be-e9d6614e87f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:33.099076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-53bed2ef-39e7-4143-92be-e9d6614e87f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:33.107682 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1938/3801] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:59:32 2022] DELETE /identity/v3/projects/3cf38b1a4bc4458199fea3373632117d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:33.108258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca43c9f-e0b6-4d30-be71-142caa3fb427 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:33.108555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca43c9f-e0b6-4d30-be71-142caa3fb427 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:33.109094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca43c9f-e0b6-4d30-be71-142caa3fb427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:33.145308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-cca43c9f-e0b6-4d30-be71-142caa3fb427 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:33.149036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cca43c9f-e0b6-4d30-be71-142caa3fb427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:33.272086 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1864/3802] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:33.280608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39515880-6dc1-4453-9a01-570694aabe1d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:33.280903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39515880-6dc1-4453-9a01-570694aabe1d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:33.281173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39515880-6dc1-4453-9a01-570694aabe1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:33.311660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-39515880-6dc1-4453-9a01-570694aabe1d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:33.313151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-39515880-6dc1-4453-9a01-570694aabe1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:33.322520 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1939/3803] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:33.327604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca43c9f-e0b6-4d30-be71-142caa3fb427 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:33.338629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39515880-6dc1-4453-9a01-570694aabe1d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:33.340039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-39515880-6dc1-4453-9a01-570694aabe1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:33.342964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca43c9f-e0b6-4d30-be71-142caa3fb427 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:33.350141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e661472c-f695-4356-9bfc-6844c59ba3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:33.352152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661472c-f695-4356-9bfc-6844c59ba3c6 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:33.353882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39515880-6dc1-4453-9a01-570694aabe1d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:33.356613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661472c-f695-4356-9bfc-6844c59ba3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:33.357623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661472c-f695-4356-9bfc-6844c59ba3c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:33.358678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e661472c-f695-4356-9bfc-6844c59ba3c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:59:33.359199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e661472c-f695-4356-9bfc-6844c59ba3c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:33.360632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac4eae-9b5d-44f9-933b-508f168a3fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:33.361423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac4eae-9b5d-44f9-933b-508f168a3fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:33.361700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac4eae-9b5d-44f9-933b-508f168a3fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:33.361964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac4eae-9b5d-44f9-933b-508f168a3fb1 admin admin] PATH_INFO: `/v3/users/1de7837875cf4298b7ac7f328a74694c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:33.362220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e661472c-f695-4356-9bfc-6844c59ba3c6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:33.364110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac4eae-9b5d-44f9-933b-508f168a3fb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1de7837875cf4298b7ac7f328a74694c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:33.367066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac4eae-9b5d-44f9-933b-508f168a3fb1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:33.395662 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1865/3804] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:59:33.396662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e661472c-f695-4356-9bfc-6844c59ba3c6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:33.414604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e661472c-f695-4356-9bfc-6844c59ba3c6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:33.420807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4adc7db-8be7-425a-8f06-d86bc3566a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:33.421669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adc7db-8be7-425a-8f06-d86bc3566a58 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:33.421951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adc7db-8be7-425a-8f06-d86bc3566a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:33.422295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adc7db-8be7-425a-8f06-d86bc3566a58 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:33.422966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4adc7db-8be7-425a-8f06-d86bc3566a58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:33.425729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4adc7db-8be7-425a-8f06-d86bc3566a58 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:33.468082 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1866/3805] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:59:33 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:33.472418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4adc7db-8be7-425a-8f06-d86bc3566a58 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:33.472696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-69ac4eae-9b5d-44f9-933b-508f168a3fb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1de7837875cf4298b7ac7f328a74694c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:33.484458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-69ac4eae-9b5d-44f9-933b-508f168a3fb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1de7837875cf4298b7ac7f328a74694c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:33.500891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-69ac4eae-9b5d-44f9-933b-508f168a3fb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1de7837875cf4298b7ac7f328a74694c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:33.502955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-69ac4eae-9b5d-44f9-933b-508f168a3fb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1de7837875cf4298b7ac7f328a74694c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:33.514564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-69ac4eae-9b5d-44f9-933b-508f168a3fb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1de7837875cf4298b7ac7f328a74694c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:33.529514 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1940/3806] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:59:33 2022] DELETE /identity/v3/users/1de7837875cf4298b7ac7f328a74694c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:33.574909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:33.603956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4adc7db-8be7-425a-8f06-d86bc3566a58 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:33.716002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa876ee6-111b-4d6d-91c9-d21e465b4fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:33.719159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac4eae-9b5d-44f9-933b-508f168a3fb1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:33.720949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-69ac4eae-9b5d-44f9-933b-508f168a3fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:33.722619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa876ee6-111b-4d6d-91c9-d21e465b4fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:33.722912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa876ee6-111b-4d6d-91c9-d21e465b4fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:33.723190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa876ee6-111b-4d6d-91c9-d21e465b4fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:33.723870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa876ee6-111b-4d6d-91c9-d21e465b4fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:33.726202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa876ee6-111b-4d6d-91c9-d21e465b4fc1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:33.784119 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1867/3807] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:59:33 2022] POST /identity/v3/users => generated 288 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:33.784119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa876ee6-111b-4d6d-91c9-d21e465b4fc1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:33.785238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fa876ee6-111b-4d6d-91c9-d21e465b4fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:33.797754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa876ee6-111b-4d6d-91c9-d21e465b4fc1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:33.802229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac4eae-9b5d-44f9-933b-508f168a3fb1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:33.803145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c017759-8cf9-4abf-a29d-26ee8d4f9266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:33.803899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c017759-8cf9-4abf-a29d-26ee8d4f9266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:33.804319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c017759-8cf9-4abf-a29d-26ee8d4f9266 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:33.804685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c017759-8cf9-4abf-a29d-26ee8d4f9266 admin admin] PATH_INFO: `/v3/projects/35315d2dc0e749a9910aa04a2ed06b42` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:33.805920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c017759-8cf9-4abf-a29d-26ee8d4f9266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35315d2dc0e749a9910aa04a2ed06b42)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:33.808109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c017759-8cf9-4abf-a29d-26ee8d4f9266 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:33.816672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27353d61-845a-4d9c-817d-0306b8fc8c6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:33.817352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27353d61-845a-4d9c-817d-0306b8fc8c6c service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:33.817787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27353d61-845a-4d9c-817d-0306b8fc8c6c service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:33.818802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27353d61-845a-4d9c-817d-0306b8fc8c6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:33.820854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-27353d61-845a-4d9c-817d-0306b8fc8c6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:33.835902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27353d61-845a-4d9c-817d-0306b8fc8c6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:33.837576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27353d61-845a-4d9c-817d-0306b8fc8c6c service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:33.844327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:33.856291 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1941/3808] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:59:33.857332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27353d61-845a-4d9c-817d-0306b8fc8c6c service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:33.860527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-27353d61-845a-4d9c-817d-0306b8fc8c6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:33.876091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27353d61-845a-4d9c-817d-0306b8fc8c6c service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:33.878202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0c017759-8cf9-4abf-a29d-26ee8d4f9266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35315d2dc0e749a9910aa04a2ed06b42', 'request_id': 'req-0c017759-8cf9-4abf-a29d-26ee8d4f9266'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:33.890637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0c017759-8cf9-4abf-a29d-26ee8d4f9266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35315d2dc0e749a9910aa04a2ed06b42', 'request_id': 'req-0c017759-8cf9-4abf-a29d-26ee8d4f9266'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:33.902667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0c017759-8cf9-4abf-a29d-26ee8d4f9266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35315d2dc0e749a9910aa04a2ed06b42', 'request_id': 'req-0c017759-8cf9-4abf-a29d-26ee8d4f9266'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:33.925046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0c017759-8cf9-4abf-a29d-26ee8d4f9266 admin admin] The token cache is being invalidate because project 35315d2dc0e749a9910aa04a2ed06b42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:59:33.925344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0c017759-8cf9-4abf-a29d-26ee8d4f9266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:33.926808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0c017759-8cf9-4abf-a29d-26ee8d4f9266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:33.929797 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1868/3809] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:59:33 2022] DELETE /identity/v3/projects/35315d2dc0e749a9910aa04a2ed06b42 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:33.935297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e864f-845a-4fd0-a091-c2ebdbdbc828 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:33.935297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e864f-845a-4fd0-a091-c2ebdbdbc828 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:33.935721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e864f-845a-4fd0-a091-c2ebdbdbc828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:33.949656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61faba2b-c743-46d5-af21-0b95c18c8696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:33.950342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61faba2b-c743-46d5-af21-0b95c18c8696 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:33.951142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61faba2b-c743-46d5-af21-0b95c18c8696 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:33.951423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61faba2b-c743-46d5-af21-0b95c18c8696 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:33.955455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61faba2b-c743-46d5-af21-0b95c18c8696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:33.957257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61faba2b-c743-46d5-af21-0b95c18c8696 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:33.968657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-cd5e864f-845a-4fd0-a091-c2ebdbdbc828 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:33.969898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cd5e864f-845a-4fd0-a091-c2ebdbdbc828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:33.974532 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1942/3810] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:59:33.977160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61faba2b-c743-46d5-af21-0b95c18c8696 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:33.979935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-61faba2b-c743-46d5-af21-0b95c18c8696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:33.980958 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1869/3811] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:59:33.989345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482ad60-ac2e-4919-85bd-a514e301168d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:33.989631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482ad60-ac2e-4919-85bd-a514e301168d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:33.990740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482ad60-ac2e-4919-85bd-a514e301168d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:33.994696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61faba2b-c743-46d5-af21-0b95c18c8696 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:34.003640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe4f69-f115-4b97-b3a4-7fb1e4a0403d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:34.004659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe4f69-f115-4b97-b3a4-7fb1e4a0403d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:34.005171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe4f69-f115-4b97-b3a4-7fb1e4a0403d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:34.005680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe4f69-f115-4b97-b3a4-7fb1e4a0403d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:34.006728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe4f69-f115-4b97-b3a4-7fb1e4a0403d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:34.009122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe4f69-f115-4b97-b3a4-7fb1e4a0403d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:34.017530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e482ad60-ac2e-4919-85bd-a514e301168d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:34.018989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e482ad60-ac2e-4919-85bd-a514e301168d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:34.020907 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1943/3812] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:59:34.025076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe4f69-f115-4b97-b3a4-7fb1e4a0403d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:34.036947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe4f69-f115-4b97-b3a4-7fb1e4a0403d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:34.041447 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1870/3813] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:34.042592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482ad60-ac2e-4919-85bd-a514e301168d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:34.047245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b2039-6375-45b0-b9f0-9d029731de57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:34.048288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b2039-6375-45b0-b9f0-9d029731de57 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:34.048761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b2039-6375-45b0-b9f0-9d029731de57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:34.049235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b2039-6375-45b0-b9f0-9d029731de57 admin admin] PATH_INFO: `/v3/projects/e041ae5d1fee42ebb7179ed70ff8bc28/users/50d792f641674214bd6f1b5d6900c32c/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:34.058106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482ad60-ac2e-4919-85bd-a514e301168d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:34.065752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ad13f-c74d-4edf-8657-b54225e464c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:34.067552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ad13f-c74d-4edf-8657-b54225e464c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:34.067847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ad13f-c74d-4edf-8657-b54225e464c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:34.068123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ad13f-c74d-4edf-8657-b54225e464c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:34.069525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ad13f-c74d-4edf-8657-b54225e464c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:59:34.069933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ad13f-c74d-4edf-8657-b54225e464c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:34.071456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b2039-6375-45b0-b9f0-9d029731de57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e041ae5d1fee42ebb7179ed70ff8bc28, user_id=50d792f641674214bd6f1b5d6900c32c, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:34.072184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ad13f-c74d-4edf-8657-b54225e464c7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:34.075479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b2039-6375-45b0-b9f0-9d029731de57 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:34.090530 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1871/3814] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 20:59:34.096676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ad13f-c74d-4edf-8657-b54225e464c7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:34.109849 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1944/3815] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:59:34 2022] PUT /identity/v3/projects/e041ae5d1fee42ebb7179ed70ff8bc28/users/50d792f641674214bd6f1b5d6900c32c/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:59:34.110063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ad13f-c74d-4edf-8657-b54225e464c7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:34.111785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d498bf-16e9-4cf2-9ca5-36b65bb98b87 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:34.112226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d498bf-16e9-4cf2-9ca5-36b65bb98b87 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:34.112512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d498bf-16e9-4cf2-9ca5-36b65bb98b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:34.154749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-83d498bf-16e9-4cf2-9ca5-36b65bb98b87 None None] MFA Rules not processed for user `50d792f641674214bd6f1b5d6900c32c`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:34.157067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-83d498bf-16e9-4cf2-9ca5-36b65bb98b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:34.186171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:34.234083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82146a72-6c97-4682-bab2-014e73383679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:34.234753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82146a72-6c97-4682-bab2-014e73383679 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:34.235034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82146a72-6c97-4682-bab2-014e73383679 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:34.235310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82146a72-6c97-4682-bab2-014e73383679 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:34.235941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82146a72-6c97-4682-bab2-014e73383679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:34.238635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82146a72-6c97-4682-bab2-014e73383679 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:34.274538 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1945/3816] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:34.335374 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1872/3817] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:59:34 2022] POST /identity/v3/projects => generated 375 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:34.338945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d498bf-16e9-4cf2-9ca5-36b65bb98b87 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:34.356152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d498bf-16e9-4cf2-9ca5-36b65bb98b87 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:34.443265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:34.528775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3d8e2-b6e8-4081-9c40-e8925d1da8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:34.532227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3d8e2-b6e8-4081-9c40-e8925d1da8b8 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:34.532626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3d8e2-b6e8-4081-9c40-e8925d1da8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:34.533582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3d8e2-b6e8-4081-9c40-e8925d1da8b8 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:34.534867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3d8e2-b6e8-4081-9c40-e8925d1da8b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:34.537672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3d8e2-b6e8-4081-9c40-e8925d1da8b8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:34.603206 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1946/3818] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:59:34 2022] POST /identity/v3/users => generated 288 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:34.607147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82146a72-6c97-4682-bab2-014e73383679 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:34.624471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82146a72-6c97-4682-bab2-014e73383679 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:34.629466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44693e6a-b43f-457d-bcd1-8904c0dd90b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:34.630067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44693e6a-b43f-457d-bcd1-8904c0dd90b4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:34.630350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44693e6a-b43f-457d-bcd1-8904c0dd90b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:34.630662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44693e6a-b43f-457d-bcd1-8904c0dd90b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:34.631303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44693e6a-b43f-457d-bcd1-8904c0dd90b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:34.633140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44693e6a-b43f-457d-bcd1-8904c0dd90b4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:34.648733 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1873/3819] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:59:34.650066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3d8e2-b6e8-4081-9c40-e8925d1da8b8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:34.668472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3d8e2-b6e8-4081-9c40-e8925d1da8b8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:34.674730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab499f7-ab05-4eb3-b9f3-d66bea5eba0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:34.675614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab499f7-ab05-4eb3-b9f3-d66bea5eba0b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:34.675898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab499f7-ab05-4eb3-b9f3-d66bea5eba0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:34.676183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab499f7-ab05-4eb3-b9f3-d66bea5eba0b admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:34.676826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab499f7-ab05-4eb3-b9f3-d66bea5eba0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:34.678433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab499f7-ab05-4eb3-b9f3-d66bea5eba0b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:34.698661 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1947/3820] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:59:34.700707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44693e6a-b43f-457d-bcd1-8904c0dd90b4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:34.713080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44693e6a-b43f-457d-bcd1-8904c0dd90b4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:34.719051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206a5fa-be21-4967-9f60-8625d7ebe0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:34.719785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206a5fa-be21-4967-9f60-8625d7ebe0c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:34.720082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206a5fa-be21-4967-9f60-8625d7ebe0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:34.720337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206a5fa-be21-4967-9f60-8625d7ebe0c1 admin admin] PATH_INFO: `/v3/projects/04cc84345d874fc69776490bea69d81a/users/6ade3ba047d348a387e8e4077072b90a/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:34.741818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206a5fa-be21-4967-9f60-8625d7ebe0c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04cc84345d874fc69776490bea69d81a, user_id=6ade3ba047d348a387e8e4077072b90a, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:34.745238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206a5fa-be21-4967-9f60-8625d7ebe0c1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:34.781621 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1874/3821] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:59:34 2022] PUT /identity/v3/projects/04cc84345d874fc69776490bea69d81a/users/6ade3ba047d348a387e8e4077072b90a/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:34.786412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3c8d3-7f65-4263-a2a1-6623d4129c78 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:34.786797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3c8d3-7f65-4263-a2a1-6623d4129c78 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:34.786990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3c8d3-7f65-4263-a2a1-6623d4129c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:34.820186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-64a3c8d3-7f65-4263-a2a1-6623d4129c78 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:34.825920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-64a3c8d3-7f65-4263-a2a1-6623d4129c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:34.945681 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1948/3822] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:34.956784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206a5fa-be21-4967-9f60-8625d7ebe0c1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:34.959413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1206a5fa-be21-4967-9f60-8625d7ebe0c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:35.059548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206a5fa-be21-4967-9f60-8625d7ebe0c1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:35.063915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31712fea-ad72-42c0-b8b9-f466d21698a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:35.064596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31712fea-ad72-42c0-b8b9-f466d21698a8 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:35.064888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31712fea-ad72-42c0-b8b9-f466d21698a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:35.065166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31712fea-ad72-42c0-b8b9-f466d21698a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:35.084136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31712fea-ad72-42c0-b8b9-f466d21698a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:35.085180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31712fea-ad72-42c0-b8b9-f466d21698a8 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:35.097971 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1875/3823] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:59:35.144541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3c8d3-7f65-4263-a2a1-6623d4129c78 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:35.146188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-64a3c8d3-7f65-4263-a2a1-6623d4129c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:35.241164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3c8d3-7f65-4263-a2a1-6623d4129c78 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:35.249484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbfae3-bfa4-4c18-90fe-0c732c122d99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:35.250023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbfae3-bfa4-4c18-90fe-0c732c122d99 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:35.250365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbfae3-bfa4-4c18-90fe-0c732c122d99 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:35.250680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbfae3-bfa4-4c18-90fe-0c732c122d99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:35.319404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effbfae3-bfa4-4c18-90fe-0c732c122d99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:35.321245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effbfae3-bfa4-4c18-90fe-0c732c122d99 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:35.331332 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1949/3824] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:59:40.877864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e057f7e-3345-4026-ad57-5eaf19e4c405 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:40.878310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e057f7e-3345-4026-ad57-5eaf19e4c405 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:40.878310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e057f7e-3345-4026-ad57-5eaf19e4c405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:40.987424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7e057f7e-3345-4026-ad57-5eaf19e4c405 None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:40.988949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7e057f7e-3345-4026-ad57-5eaf19e4c405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:40.998790 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1876/3825] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:40 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:59:40.999775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbfae3-bfa4-4c18-90fe-0c732c122d99 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:41.001251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-effbfae3-bfa4-4c18-90fe-0c732c122d99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:41.025007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:41.111469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbfae3-bfa4-4c18-90fe-0c732c122d99 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:41.116653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f43dbf-1fd8-41cb-b36b-d4c203475f60 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c22b659fed42d1b5bc3477d267f76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c9cfa2e7f5348b4a525294ba662e4e6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:41.117529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f43dbf-1fd8-41cb-b36b-d4c203475f60 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:41.117993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f43dbf-1fd8-41cb-b36b-d4c203475f60 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:41.118415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f43dbf-1fd8-41cb-b36b-d4c203475f60 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] PATH_INFO: `/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:41.184526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d0f43dbf-1fd8-41cb-b36b-d4c203475f60 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'a5c22b659fed42d1b5bc3477d267f76d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:41.201125 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1950/3826] 10.222.0.17 () {60 vars in 1269 bytes} [Sat Oct 1 20:59:40 2022] POST /identity/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:42.206944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa508d-1184-42f1-b88c-17f4c59fcbd7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:42.207366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa508d-1184-42f1-b88c-17f4c59fcbd7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:42.207496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa508d-1184-42f1-b88c-17f4c59fcbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:42.245289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0faa508d-1184-42f1-b88c-17f4c59fcbd7 None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:42.246536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0faa508d-1184-42f1-b88c-17f4c59fcbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:42.253068 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1877/3827] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:42 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:59:42.256923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f8e0e-034d-43bd-9103-a29646fb58f6 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:42.257713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f8e0e-034d-43bd-9103-a29646fb58f6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:42.258144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f8e0e-034d-43bd-9103-a29646fb58f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:42.346594 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-116f8e0e-034d-43bd-9103-a29646fb58f6 None None] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Oct 01 20:59:42.346594 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 20:59:42.346594 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 20:59:42.346594 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 20:59:42.346594 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 20:59:42.346594 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 20:59:42.346594 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 20:59:42.346594 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 20:59:42.346594 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 20:59:42.346594 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 20:59:42.346594 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 20:59:42.346594 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 20:59:42.346594 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 01 20:59:42.346594 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 01 20:59:42.346594 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Oct 01 20:59:42.346594 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Oct 01 20:59:42.347922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 185, in authenticate_for_token Oct 01 20:59:42.347922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application authenticate(auth_info, auth_context) Oct 01 20:59:42.347922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 152, in authenticate Oct 01 20:59:42.347922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = method.authenticate(auth_info.get_method_data(method_name)) Oct 01 20:59:42.347922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/plugins/token.py", line 42, in authenticate Oct 01 20:59:42.347922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application token = self._get_token_ref(auth_payload) Oct 01 20:59:42.347922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/plugins/token.py", line 36, in _get_token_ref Oct 01 20:59:42.347922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return PROVIDERS.token_provider_api.validate_token(token_id) Oct 01 20:59:42.347922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:59:42.347922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:59:42.347922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Oct 01 20:59:42.347922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Oct 01 20:59:42.347922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Oct 01 20:59:42.347922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.check_revocation(token) Oct 01 20:59:42.347922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:59:42.347922 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:59:42.349049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Oct 01 20:59:42.349049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Oct 01 20:59:42.349049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:59:42.349049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:59:42.349049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Oct 01 20:59:42.349049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Oct 01 20:59:42.349049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 20:59:42.349049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 20:59:42.349049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Oct 01 20:59:42.349049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Oct 01 20:59:42.349049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Oct 01 20:59:42.349049 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 20:59:42.356580 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1951/3828] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:42 2022] POST /identity/v3/auth/tokens => generated 80 bytes in 100 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 20:59:42.356779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af27d64-c014-4211-ba92-c45910c8933f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:42.356914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af27d64-c014-4211-ba92-c45910c8933f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:42.357260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af27d64-c014-4211-ba92-c45910c8933f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:42.424803 ncdv-master-1 devstack@keystone.service[99140]: WARNING keystone.server.flask.application [None req-8af27d64-c014-4211-ba92-c45910c8933f None None] Authorization failed. The request you have made requires authentication. from 10.222.0.17: keystone.exception.Unauthorized: The request you have made requires authentication. Oct 01 20:59:42.431804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab53a9a-ec79-4d3f-bf42-3e40cc7ac28d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:42.432305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab53a9a-ec79-4d3f-bf42-3e40cc7ac28d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:42.432912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab53a9a-ec79-4d3f-bf42-3e40cc7ac28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:42.438420 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1878/3829] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:42 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 82 msecs (HTTP/1.1 401) 6 headers in 252 bytes (2 switches on core 0) Oct 01 20:59:42.482673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fab53a9a-ec79-4d3f-bf42-3e40cc7ac28d None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:42.484646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fab53a9a-ec79-4d3f-bf42-3e40cc7ac28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:42.508965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af27d64-c014-4211-ba92-c45910c8933f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:42.512254 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1952/3830] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:42.532740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af27d64-c014-4211-ba92-c45910c8933f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:42.548698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9f043-284f-4f9e-b4b2-a37e8199f5d0 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c22b659fed42d1b5bc3477d267f76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c9cfa2e7f5348b4a525294ba662e4e6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:42.550580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9f043-284f-4f9e-b4b2-a37e8199f5d0 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:42.550580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9f043-284f-4f9e-b4b2-a37e8199f5d0 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:42.550580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9f043-284f-4f9e-b4b2-a37e8199f5d0 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] PATH_INFO: `/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:42.656671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e0f9f043-284f-4f9e-b4b2-a37e8199f5d0 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'a5c22b659fed42d1b5bc3477d267f76d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:42.679399 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1879/3831] 10.222.0.17 () {60 vars in 1269 bytes} [Sat Oct 1 20:59:42 2022] POST /identity/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:43.303162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab53a9a-ec79-4d3f-bf42-3e40cc7ac28d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:43.319088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab53a9a-ec79-4d3f-bf42-3e40cc7ac28d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:43.331082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90fd28-abec-4368-9124-a00169d6a0f0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:43.332304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90fd28-abec-4368-9124-a00169d6a0f0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:43.332744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90fd28-abec-4368-9124-a00169d6a0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:43.339591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9475a8-ad39-4e7c-8302-55a3ee2873f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:43.341111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9475a8-ad39-4e7c-8302-55a3ee2873f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:43.344986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9475a8-ad39-4e7c-8302-55a3ee2873f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:43.345611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9475a8-ad39-4e7c-8302-55a3ee2873f9 admin admin] PATH_INFO: `/v3/users/50d792f641674214bd6f1b5d6900c32c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:43.347239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9475a8-ad39-4e7c-8302-55a3ee2873f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50d792f641674214bd6f1b5d6900c32c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:43.352533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9475a8-ad39-4e7c-8302-55a3ee2873f9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:43.360652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:UPDATE=3,keystone:DELETE=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:43.371605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ce90fd28-abec-4368-9124-a00169d6a0f0 None None] MFA Rules not processed for user `6ade3ba047d348a387e8e4077072b90a`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:43.374806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ce90fd28-abec-4368-9124-a00169d6a0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:43.460493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5a9475a8-ad39-4e7c-8302-55a3ee2873f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50d792f641674214bd6f1b5d6900c32c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:43.475868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5a9475a8-ad39-4e7c-8302-55a3ee2873f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50d792f641674214bd6f1b5d6900c32c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:43.507426 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1880/3832] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:43.532093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90fd28-abec-4368-9124-a00169d6a0f0 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:43.535618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5a9475a8-ad39-4e7c-8302-55a3ee2873f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50d792f641674214bd6f1b5d6900c32c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:43.542306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5a9475a8-ad39-4e7c-8302-55a3ee2873f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50d792f641674214bd6f1b5d6900c32c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:43.561696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90fd28-abec-4368-9124-a00169d6a0f0 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:43.563993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5a9475a8-ad39-4e7c-8302-55a3ee2873f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50d792f641674214bd6f1b5d6900c32c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:43.581719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82bcbb-38b0-49ed-b059-0bf70779636b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:43.585078 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1953/3833] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:59:43 2022] DELETE /identity/v3/users/50d792f641674214bd6f1b5d6900c32c => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:43.585233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82bcbb-38b0-49ed-b059-0bf70779636b service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:43.585233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82bcbb-38b0-49ed-b059-0bf70779636b service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:43.585233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82bcbb-38b0-49ed-b059-0bf70779636b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:43.585501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4b82bcbb-38b0-49ed-b059-0bf70779636b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:43.588845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9af68-48ad-4b1d-b033-6e864486e48e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:43.589532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9af68-48ad-4b1d-b033-6e864486e48e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:43.589975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9af68-48ad-4b1d-b033-6e864486e48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:43.621607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82bcbb-38b0-49ed-b059-0bf70779636b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:43.623257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82bcbb-38b0-49ed-b059-0bf70779636b service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:43.635547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b0f9af68-48ad-4b1d-b033-6e864486e48e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:43.639313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b0f9af68-48ad-4b1d-b033-6e864486e48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:43.649333 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1881/3834] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:59:43.679516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=83,keystone:UPDATE=2,keystone:INSERT=3,keystone:DELETE=12 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:43.685392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa55f2-aa44-4112-abb6-84ef678026c0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:43.685921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa55f2-aa44-4112-abb6-84ef678026c0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:43.686211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa55f2-aa44-4112-abb6-84ef678026c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:43.721289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-58fa55f2-aa44-4112-abb6-84ef678026c0 None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:43.726868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-58fa55f2-aa44-4112-abb6-84ef678026c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:43.746418 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1882/3835] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:43 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:59:43.752262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8b4af-8853-4c50-a452-5ea4fdac47b7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:43.752556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8b4af-8853-4c50-a452-5ea4fdac47b7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:43.752846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8b4af-8853-4c50-a452-5ea4fdac47b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:43.764407 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1954/3836] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:43.777439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9af68-48ad-4b1d-b033-6e864486e48e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:43.777962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b0f9af68-48ad-4b1d-b033-6e864486e48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:43.814590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e9c8b4af-8853-4c50-a452-5ea4fdac47b7 None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:43.824169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e9c8b4af-8853-4c50-a452-5ea4fdac47b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:43.874229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9af68-48ad-4b1d-b033-6e864486e48e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:43.883511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824fa21-bff0-4a28-959a-6afce6bc1653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:43.884146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824fa21-bff0-4a28-959a-6afce6bc1653 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:43.884434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824fa21-bff0-4a28-959a-6afce6bc1653 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:43.884717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824fa21-bff0-4a28-959a-6afce6bc1653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:43.915482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824fa21-bff0-4a28-959a-6afce6bc1653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:43.917040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824fa21-bff0-4a28-959a-6afce6bc1653 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:43.943938 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1883/3837] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:43.951075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8b4af-8853-4c50-a452-5ea4fdac47b7 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:43.951894 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1955/3838] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:59:43.968731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8b4af-8853-4c50-a452-5ea4fdac47b7 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:43.981123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78fc40f-56c8-49b3-87bb-b0421ef89e32 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c22b659fed42d1b5bc3477d267f76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c9cfa2e7f5348b4a525294ba662e4e6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:43.982299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fc40f-56c8-49b3-87bb-b0421ef89e32 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:43.983907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fc40f-56c8-49b3-87bb-b0421ef89e32 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:43.984306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fc40f-56c8-49b3-87bb-b0421ef89e32 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] PATH_INFO: `/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:44.090413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e78fc40f-56c8-49b3-87bb-b0421ef89e32 tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'a5c22b659fed42d1b5bc3477d267f76d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:44.104500 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1884/3839] 10.222.0.17 () {60 vars in 1269 bytes} [Sat Oct 1 20:59:43 2022] POST /identity/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:44.654170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824fa21-bff0-4a28-959a-6afce6bc1653 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:44.656024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9824fa21-bff0-4a28-959a-6afce6bc1653 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:44.675319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824fa21-bff0-4a28-959a-6afce6bc1653 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:44.681575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0559c-b933-42ff-8364-4ad00f3f2f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:44.682639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0559c-b933-42ff-8364-4ad00f3f2f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:44.683187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0559c-b933-42ff-8364-4ad00f3f2f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:44.684056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0559c-b933-42ff-8364-4ad00f3f2f93 admin admin] PATH_INFO: `/v3/projects/e041ae5d1fee42ebb7179ed70ff8bc28` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:44.688376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b0559c-b933-42ff-8364-4ad00f3f2f93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e041ae5d1fee42ebb7179ed70ff8bc28)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:44.695507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b0559c-b933-42ff-8364-4ad00f3f2f93 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:44.829022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-11b0559c-b933-42ff-8364-4ad00f3f2f93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e041ae5d1fee42ebb7179ed70ff8bc28', 'request_id': 'req-11b0559c-b933-42ff-8364-4ad00f3f2f93'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:44.846964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-11b0559c-b933-42ff-8364-4ad00f3f2f93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e041ae5d1fee42ebb7179ed70ff8bc28', 'request_id': 'req-11b0559c-b933-42ff-8364-4ad00f3f2f93'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:44.858931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-11b0559c-b933-42ff-8364-4ad00f3f2f93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e041ae5d1fee42ebb7179ed70ff8bc28', 'request_id': 'req-11b0559c-b933-42ff-8364-4ad00f3f2f93'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:44.882944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-11b0559c-b933-42ff-8364-4ad00f3f2f93 admin admin] The token cache is being invalidate because project e041ae5d1fee42ebb7179ed70ff8bc28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:59:44.883149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-11b0559c-b933-42ff-8364-4ad00f3f2f93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:44.884018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-11b0559c-b933-42ff-8364-4ad00f3f2f93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:44.890157 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1956/3840] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:59:44 2022] DELETE /identity/v3/projects/e041ae5d1fee42ebb7179ed70ff8bc28 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:44.892085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a26e0-ea9e-48bc-9285-4a055b80deec None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:44.892550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a26e0-ea9e-48bc-9285-4a055b80deec None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:44.892691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a26e0-ea9e-48bc-9285-4a055b80deec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:44.931741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-846a26e0-ea9e-48bc-9285-4a055b80deec None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:44.932992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-846a26e0-ea9e-48bc-9285-4a055b80deec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:45.053071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:UPDATE=1,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:45.086441 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1885/3841] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:59:45.095789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621afeb-1650-4776-8f9e-56c86997815e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:45.096214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621afeb-1650-4776-8f9e-56c86997815e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:45.096627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621afeb-1650-4776-8f9e-56c86997815e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:45.109632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519a0d1-7890-4a4d-9998-a077234a6375 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:45.109926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519a0d1-7890-4a4d-9998-a077234a6375 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:45.110212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519a0d1-7890-4a4d-9998-a077234a6375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:45.120094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5621afeb-1650-4776-8f9e-56c86997815e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:45.121517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5621afeb-1650-4776-8f9e-56c86997815e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:45.132646 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1957/3842] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:45.137865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621afeb-1650-4776-8f9e-56c86997815e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:45.142948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5519a0d1-7890-4a4d-9998-a077234a6375 None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:45.144529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5519a0d1-7890-4a4d-9998-a077234a6375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:45.149835 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1886/3843] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:45 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 20:59:45.153737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621afeb-1650-4776-8f9e-56c86997815e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:45.155390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d720a-6414-4883-b4e7-08ff791fafa5 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:45.155925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d720a-6414-4883-b4e7-08ff791fafa5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:45.156863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d720a-6414-4883-b4e7-08ff791fafa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:45.159926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc5114-342d-4f08-ba5b-5bf4c82e2576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:45.160684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc5114-342d-4f08-ba5b-5bf4c82e2576 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:45.161070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc5114-342d-4f08-ba5b-5bf4c82e2576 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:45.161446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc5114-342d-4f08-ba5b-5bf4c82e2576 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:45.162832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc5114-342d-4f08-ba5b-5bf4c82e2576 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:59:45.163292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc5114-342d-4f08-ba5b-5bf4c82e2576 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:45.165762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc5114-342d-4f08-ba5b-5bf4c82e2576 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:45.204420 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1958/3844] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:59:45.206146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc5114-342d-4f08-ba5b-5bf4c82e2576 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:45.215147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8e5d720a-6414-4883-b4e7-08ff791fafa5 None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:45.220851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8e5d720a-6414-4883-b4e7-08ff791fafa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:45.227308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc5114-342d-4f08-ba5b-5bf4c82e2576 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:45.237293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c7e6d-efc9-493f-bb85-791920df61ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:45.240019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c7e6d-efc9-493f-bb85-791920df61ee admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:45.240422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c7e6d-efc9-493f-bb85-791920df61ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:45.241333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c7e6d-efc9-493f-bb85-791920df61ee admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:45.242071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c7e6d-efc9-493f-bb85-791920df61ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:45.251274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c7e6d-efc9-493f-bb85-791920df61ee admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:45.319940 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1959/3845] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:59:45 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:45.322469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c7e6d-efc9-493f-bb85-791920df61ee admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:45.332369 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1887/3846] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:45.336280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d720a-6414-4883-b4e7-08ff791fafa5 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:45.348215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d720a-6414-4883-b4e7-08ff791fafa5 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:45.381123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:45.457110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c7e6d-efc9-493f-bb85-791920df61ee admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:45.462800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03f5d1-df2d-4768-9caa-bb48bbc0f47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:45.463930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03f5d1-df2d-4768-9caa-bb48bbc0f47a admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:45.464209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03f5d1-df2d-4768-9caa-bb48bbc0f47a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:45.464868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03f5d1-df2d-4768-9caa-bb48bbc0f47a admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:45.465680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03f5d1-df2d-4768-9caa-bb48bbc0f47a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:45.468719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03f5d1-df2d-4768-9caa-bb48bbc0f47a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:45.473726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52d987-bdcf-4df0-83ca-3013d1d5a03a tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5c22b659fed42d1b5bc3477d267f76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c9cfa2e7f5348b4a525294ba662e4e6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:45.474406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52d987-bdcf-4df0-83ca-3013d1d5a03a tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:45.475052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52d987-bdcf-4df0-83ca-3013d1d5a03a tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:45.475338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52d987-bdcf-4df0-83ca-3013d1d5a03a tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] PATH_INFO: `/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:45.511646 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1960/3847] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:59:45 2022] POST /identity/v3/users => generated 291 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:45.514424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03f5d1-df2d-4768-9caa-bb48bbc0f47a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:45.534885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03f5d1-df2d-4768-9caa-bb48bbc0f47a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:45.540028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866ed03-ab04-4e3a-b809-d56e9ac8fa7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:45.540736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866ed03-ab04-4e3a-b809-d56e9ac8fa7e admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:45.543214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866ed03-ab04-4e3a-b809-d56e9ac8fa7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:45.544031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866ed03-ab04-4e3a-b809-d56e9ac8fa7e admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:45.547046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866ed03-ab04-4e3a-b809-d56e9ac8fa7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:45.548132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866ed03-ab04-4e3a-b809-d56e9ac8fa7e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:45.569313 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1961/3848] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:59:45.571573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dd52d987-bdcf-4df0-83ca-3013d1d5a03a tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'a5c22b659fed42d1b5bc3477d267f76d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:45.574287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866ed03-ab04-4e3a-b809-d56e9ac8fa7e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:45.589075 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1888/3849] 10.222.0.17 () {60 vars in 1269 bytes} [Sat Oct 1 20:59:45 2022] POST /identity/v3/users/a5c22b659fed42d1b5bc3477d267f76d/password => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:59:45.593927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866ed03-ab04-4e3a-b809-d56e9ac8fa7e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:45.600387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c66c5-e1ed-4635-8479-9599f9549cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:45.601089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c66c5-e1ed-4635-8479-9599f9549cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:45.601443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c66c5-e1ed-4635-8479-9599f9549cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:45.601790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c66c5-e1ed-4635-8479-9599f9549cf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:45.602610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c66c5-e1ed-4635-8479-9599f9549cf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:45.604622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c66c5-e1ed-4635-8479-9599f9549cf3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:45.623874 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1962/3850] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:59:45.624818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52d987-bdcf-4df0-83ca-3013d1d5a03a tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:45.655037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52d987-bdcf-4df0-83ca-3013d1d5a03a tempest-IdentityV3UsersTest-54837439 tempest-IdentityV3UsersTest-54837439-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:45.664555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3ea25-69d5-41bd-a23a-b537f6a7e361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:45.668651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3ea25-69d5-41bd-a23a-b537f6a7e361 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:45.668943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3ea25-69d5-41bd-a23a-b537f6a7e361 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:45.669224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3ea25-69d5-41bd-a23a-b537f6a7e361 admin admin] PATH_INFO: `/v3/projects/509d035fd7f44378afdd3bfe6422c5ad/users/d91992987c1347eab04016bd79c263f6/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:45.704354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3ea25-69d5-41bd-a23a-b537f6a7e361 admin admin] RBAC: Authorizing `identity:create_grant(project_id=509d035fd7f44378afdd3bfe6422c5ad, user_id=d91992987c1347eab04016bd79c263f6, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:45.707780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3ea25-69d5-41bd-a23a-b537f6a7e361 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:45.740889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e8f1c-4f0c-455a-80e6-6f8ec6e20d02 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:45.741596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e8f1c-4f0c-455a-80e6-6f8ec6e20d02 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:45.741596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e8f1c-4f0c-455a-80e6-6f8ec6e20d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:45.744771 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1889/3851] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:59:45 2022] PUT /identity/v3/projects/509d035fd7f44378afdd3bfe6422c5ad/users/d91992987c1347eab04016bd79c263f6/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:59:45.827765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3f4e8f1c-4f0c-455a-80e6-6f8ec6e20d02 None None] MFA Rules not processed for user `d91992987c1347eab04016bd79c263f6`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:45.836669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3f4e8f1c-4f0c-455a-80e6-6f8ec6e20d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:45.934149 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1963/3852] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:45.946359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3ea25-69d5-41bd-a23a-b537f6a7e361 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:45.948891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f8a3ea25-69d5-41bd-a23a-b537f6a7e361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:46.023992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:UPDATE=1,keystone:INSERT=3,keystone:DELETE=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:46.087694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3ea25-69d5-41bd-a23a-b537f6a7e361 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:46.097211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b5120-bcfe-4c91-ba40-24b146f8e77a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:46.098268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b5120-bcfe-4c91-ba40-24b146f8e77a service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:46.099225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b5120-bcfe-4c91-ba40-24b146f8e77a service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:46.101550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b5120-bcfe-4c91-ba40-24b146f8e77a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:46.115817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b5120-bcfe-4c91-ba40-24b146f8e77a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:46.117861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b5120-bcfe-4c91-ba40-24b146f8e77a service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:46.146040 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1890/3853] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:59:46.216289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e8f1c-4f0c-455a-80e6-6f8ec6e20d02 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:46.377460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e8f1c-4f0c-455a-80e6-6f8ec6e20d02 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:46.384817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba2fa8-a49d-491f-bcfd-69e5a6c23264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:46.385783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba2fa8-a49d-491f-bcfd-69e5a6c23264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:46.386392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba2fa8-a49d-491f-bcfd-69e5a6c23264 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:46.386906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba2fa8-a49d-491f-bcfd-69e5a6c23264 admin admin] PATH_INFO: `/v3/users/d91992987c1347eab04016bd79c263f6` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:46.388911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba2fa8-a49d-491f-bcfd-69e5a6c23264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d91992987c1347eab04016bd79c263f6)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:46.391776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba2fa8-a49d-491f-bcfd-69e5a6c23264 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:46.421323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:46.467580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-aeba2fa8-a49d-491f-bcfd-69e5a6c23264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd91992987c1347eab04016bd79c263f6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:46.476790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-aeba2fa8-a49d-491f-bcfd-69e5a6c23264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd91992987c1347eab04016bd79c263f6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:46.492950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-aeba2fa8-a49d-491f-bcfd-69e5a6c23264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd91992987c1347eab04016bd79c263f6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:46.493868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-aeba2fa8-a49d-491f-bcfd-69e5a6c23264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd91992987c1347eab04016bd79c263f6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:46.506103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-aeba2fa8-a49d-491f-bcfd-69e5a6c23264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd91992987c1347eab04016bd79c263f6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:46.522872 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1964/3854] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:59:46 2022] DELETE /identity/v3/users/d91992987c1347eab04016bd79c263f6 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:46.524633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8d4e7-5238-4dae-bc5c-4fb6b6d23c3f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:46.524940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8d4e7-5238-4dae-bc5c-4fb6b6d23c3f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:46.525211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8d4e7-5238-4dae-bc5c-4fb6b6d23c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:46.546208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-62b8d4e7-5238-4dae-bc5c-4fb6b6d23c3f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:46.547845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-62b8d4e7-5238-4dae-bc5c-4fb6b6d23c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:46.592587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabc07c-b5e0-45b7-bc57-5e1a20b100b3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:46.592774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabc07c-b5e0-45b7-bc57-5e1a20b100b3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:46.593058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabc07c-b5e0-45b7-bc57-5e1a20b100b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:46.614309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4fabc07c-b5e0-45b7-bc57-5e1a20b100b3 None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:46.615582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4fabc07c-b5e0-45b7-bc57-5e1a20b100b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:46.621174 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1965/3855] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:46 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 01 20:59:46.641095 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1891/3856] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:46.652513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabc07c-b5e0-45b7-bc57-5e1a20b100b3 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:46.656054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4fabc07c-b5e0-45b7-bc57-5e1a20b100b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:46.734954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabc07c-b5e0-45b7-bc57-5e1a20b100b3 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:46.739079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7465de-e267-4ce8-aec5-9cf58714f4a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:46.739778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7465de-e267-4ce8-aec5-9cf58714f4a9 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:46.741640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7465de-e267-4ce8-aec5-9cf58714f4a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:46.743164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7465de-e267-4ce8-aec5-9cf58714f4a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:46.758040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7465de-e267-4ce8-aec5-9cf58714f4a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:46.760828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7465de-e267-4ce8-aec5-9cf58714f4a9 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:46.796150 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1966/3857] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 20:59:47.247346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8d4e7-5238-4dae-bc5c-4fb6b6d23c3f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:47.249191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-62b8d4e7-5238-4dae-bc5c-4fb6b6d23c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:47.270868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8d4e7-5238-4dae-bc5c-4fb6b6d23c3f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:47.276621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f32223-6711-433d-b2f9-32d1b53eeb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:47.277279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f32223-6711-433d-b2f9-32d1b53eeb54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:47.277561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f32223-6711-433d-b2f9-32d1b53eeb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:47.277837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f32223-6711-433d-b2f9-32d1b53eeb54 admin admin] PATH_INFO: `/v3/projects/509d035fd7f44378afdd3bfe6422c5ad` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:47.281845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f32223-6711-433d-b2f9-32d1b53eeb54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=509d035fd7f44378afdd3bfe6422c5ad)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:47.283499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f32223-6711-433d-b2f9-32d1b53eeb54 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:47.357858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89f32223-6711-433d-b2f9-32d1b53eeb54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '509d035fd7f44378afdd3bfe6422c5ad', 'request_id': 'req-89f32223-6711-433d-b2f9-32d1b53eeb54'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:47.370157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89f32223-6711-433d-b2f9-32d1b53eeb54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '509d035fd7f44378afdd3bfe6422c5ad', 'request_id': 'req-89f32223-6711-433d-b2f9-32d1b53eeb54'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:47.397884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89f32223-6711-433d-b2f9-32d1b53eeb54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '509d035fd7f44378afdd3bfe6422c5ad', 'request_id': 'req-89f32223-6711-433d-b2f9-32d1b53eeb54'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:47.449294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89f32223-6711-433d-b2f9-32d1b53eeb54 admin admin] The token cache is being invalidate because project 509d035fd7f44378afdd3bfe6422c5ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:59:47.449884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89f32223-6711-433d-b2f9-32d1b53eeb54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:47.450922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89f32223-6711-433d-b2f9-32d1b53eeb54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:47.453762 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1892/3858] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:59:47 2022] DELETE /identity/v3/projects/509d035fd7f44378afdd3bfe6422c5ad => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:47.461827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22887978-5986-495d-88fc-2416d4d61d74 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:47.463148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22887978-5986-495d-88fc-2416d4d61d74 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:47.463610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22887978-5986-495d-88fc-2416d4d61d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:47.500252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-22887978-5986-495d-88fc-2416d4d61d74 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:47.501487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-22887978-5986-495d-88fc-2416d4d61d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:47.625177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd49744-f7aa-492b-9395-99a214e519fa None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:47.625510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd49744-f7aa-492b-9395-99a214e519fa None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:47.626791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd49744-f7aa-492b-9395-99a214e519fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:47.681825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6cd49744-f7aa-492b-9395-99a214e519fa None None] MFA Rules not processed for user `a5c22b659fed42d1b5bc3477d267f76d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:47.683555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6cd49744-f7aa-492b-9395-99a214e519fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:47.697759 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1967/3859] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:47.699555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2d60d-49bd-4b1d-8020-d03d36d3395e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:47.699555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2d60d-49bd-4b1d-8020-d03d36d3395e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:47.699756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2d60d-49bd-4b1d-8020-d03d36d3395e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:47.734557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e1c2d60d-49bd-4b1d-8020-d03d36d3395e None None] MFA Rules not processed for user `6ade3ba047d348a387e8e4077072b90a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:47.735805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e1c2d60d-49bd-4b1d-8020-d03d36d3395e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:47.788007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:47.815424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:47.843276 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1893/3860] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:59:47.843276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1fc34-8349-413c-b68f-e32d08d67795 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:47.843276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1fc34-8349-413c-b68f-e32d08d67795 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:47.843276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1fc34-8349-413c-b68f-e32d08d67795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:47.867122 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1968/3861] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:47.868180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2d60d-49bd-4b1d-8020-d03d36d3395e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:47.869625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e1c2d60d-49bd-4b1d-8020-d03d36d3395e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:47.890913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2d60d-49bd-4b1d-8020-d03d36d3395e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:47.893127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-21e1fc34-8349-413c-b68f-e32d08d67795 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:47.900431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c657f-d5a2-4555-b21d-96bc766f8fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:47.901734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-21e1fc34-8349-413c-b68f-e32d08d67795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:47.902158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c657f-d5a2-4555-b21d-96bc766f8fd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:47.902661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c657f-d5a2-4555-b21d-96bc766f8fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:47.903125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c657f-d5a2-4555-b21d-96bc766f8fd8 admin admin] PATH_INFO: `/v3/users/a5c22b659fed42d1b5bc3477d267f76d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:47.906965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c657f-d5a2-4555-b21d-96bc766f8fd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5c22b659fed42d1b5bc3477d267f76d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:47.909228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c657f-d5a2-4555-b21d-96bc766f8fd8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:47.939802 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1894/3862] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:47.943825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1fc34-8349-413c-b68f-e32d08d67795 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:47.968311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1fc34-8349-413c-b68f-e32d08d67795 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:47.980375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bf343-5875-45a0-afaa-196938c9616f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:47.981095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bf343-5875-45a0-afaa-196938c9616f admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:47.981452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bf343-5875-45a0-afaa-196938c9616f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:47.981804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bf343-5875-45a0-afaa-196938c9616f admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:47.982855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5bf343-5875-45a0-afaa-196938c9616f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:59:47.984087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5bf343-5875-45a0-afaa-196938c9616f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:47.987020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5bf343-5875-45a0-afaa-196938c9616f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:48.001427 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1895/3863] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:59:48.005790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bf343-5875-45a0-afaa-196938c9616f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:48.030001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bf343-5875-45a0-afaa-196938c9616f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:48.047022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abe14d-68fd-42aa-bbcd-de2dc567dbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:48.047922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abe14d-68fd-42aa-bbcd-de2dc567dbdb admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:48.048284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abe14d-68fd-42aa-bbcd-de2dc567dbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:48.048636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abe14d-68fd-42aa-bbcd-de2dc567dbdb admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:48.049689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abe14d-68fd-42aa-bbcd-de2dc567dbdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:48.055500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abe14d-68fd-42aa-bbcd-de2dc567dbdb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:48.134316 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1896/3864] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:59:48 2022] POST /identity/v3/projects => generated 397 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:59:48.139064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abe14d-68fd-42aa-bbcd-de2dc567dbdb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:48.357290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abe14d-68fd-42aa-bbcd-de2dc567dbdb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:48.374737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f343d-5902-4eb7-9f7a-5185ce22ae3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:48.377024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f343d-5902-4eb7-9f7a-5185ce22ae3d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:48.377551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f343d-5902-4eb7-9f7a-5185ce22ae3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:48.378098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f343d-5902-4eb7-9f7a-5185ce22ae3d admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:48.379567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f343d-5902-4eb7-9f7a-5185ce22ae3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:48.389355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f343d-5902-4eb7-9f7a-5185ce22ae3d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:48.403425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-867c657f-d5a2-4555-b21d-96bc766f8fd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5c22b659fed42d1b5bc3477d267f76d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:48.435166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-867c657f-d5a2-4555-b21d-96bc766f8fd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5c22b659fed42d1b5bc3477d267f76d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:48.451815 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1897/3865] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:59:48 2022] POST /identity/v3/users => generated 299 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:48.453396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f343d-5902-4eb7-9f7a-5185ce22ae3d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:48.458597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-867c657f-d5a2-4555-b21d-96bc766f8fd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5c22b659fed42d1b5bc3477d267f76d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:48.459445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-867c657f-d5a2-4555-b21d-96bc766f8fd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5c22b659fed42d1b5bc3477d267f76d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:48.470197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-867c657f-d5a2-4555-b21d-96bc766f8fd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5c22b659fed42d1b5bc3477d267f76d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:48.470668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f343d-5902-4eb7-9f7a-5185ce22ae3d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:48.476957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a004c-a0cf-454e-b581-0f5a2c37eccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:48.477863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a004c-a0cf-454e-b581-0f5a2c37eccb admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:48.478427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a004c-a0cf-454e-b581-0f5a2c37eccb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:48.479364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a004c-a0cf-454e-b581-0f5a2c37eccb admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:48.480198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a004c-a0cf-454e-b581-0f5a2c37eccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:48.482010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a004c-a0cf-454e-b581-0f5a2c37eccb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:48.482146 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1969/3866] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:59:47 2022] DELETE /identity/v3/users/a5c22b659fed42d1b5bc3477d267f76d => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:48.487353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4da12-f862-442d-9fef-e312c7e912b1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:48.487762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4da12-f862-442d-9fef-e312c7e912b1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:48.488137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4da12-f862-442d-9fef-e312c7e912b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:48.508748 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1898/3867] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:59:48.512217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a004c-a0cf-454e-b581-0f5a2c37eccb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:48.522674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5c5a004c-a0cf-454e-b581-0f5a2c37eccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:48.533762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a4c4da12-f862-442d-9fef-e312c7e912b1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:48.535170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a4c4da12-f862-442d-9fef-e312c7e912b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:48.537803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a004c-a0cf-454e-b581-0f5a2c37eccb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:48.632042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6707bd-efb1-4950-bc71-930c657e02c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:48.633639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6707bd-efb1-4950-bc71-930c657e02c3 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:48.634084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6707bd-efb1-4950-bc71-930c657e02c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:48.634583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6707bd-efb1-4950-bc71-930c657e02c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:48.635476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6707bd-efb1-4950-bc71-930c657e02c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:48.638242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6707bd-efb1-4950-bc71-930c657e02c3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:48.656592 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1970/3868] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:59:48.657651 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1899/3869] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:59:48.664184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4da12-f862-442d-9fef-e312c7e912b1 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:48.667756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6707bd-efb1-4950-bc71-930c657e02c3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:48.670068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5c6707bd-efb1-4950-bc71-930c657e02c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:48.677634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4da12-f862-442d-9fef-e312c7e912b1 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:48.692433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff2289-53a3-4d7f-9677-77e2cdb32d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:48.694914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff2289-53a3-4d7f-9677-77e2cdb32d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:48.696184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff2289-53a3-4d7f-9677-77e2cdb32d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:48.700352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff2289-53a3-4d7f-9677-77e2cdb32d53 admin admin] PATH_INFO: `/v3/projects/840cdc42d8e1415da91572fac902d284/users/0779a741ea8946b28756ddd87fd7a792/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:48.725405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:48.746802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff2289-53a3-4d7f-9677-77e2cdb32d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=840cdc42d8e1415da91572fac902d284, user_id=0779a741ea8946b28756ddd87fd7a792, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:48.754435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff2289-53a3-4d7f-9677-77e2cdb32d53 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:48.798296 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1971/3870] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:59:48 2022] PUT /identity/v3/projects/840cdc42d8e1415da91572fac902d284/users/0779a741ea8946b28756ddd87fd7a792/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:48.799707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9ff79-b051-4db9-bb58-6082351adfb6 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:48.800664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9ff79-b051-4db9-bb58-6082351adfb6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:48.800960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9ff79-b051-4db9-bb58-6082351adfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:48.816394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6707bd-efb1-4950-bc71-930c657e02c3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:48.846330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6ca9ff79-b051-4db9-bb58-6082351adfb6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:48.847592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6ca9ff79-b051-4db9-bb58-6082351adfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:48.915956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:48.922969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a2a64-83ba-4848-8fff-77c7fdeeb682 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:48.923663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a2a64-83ba-4848-8fff-77c7fdeeb682 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:48.923992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a2a64-83ba-4848-8fff-77c7fdeeb682 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:48.924264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a2a64-83ba-4848-8fff-77c7fdeeb682 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:49.047585 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1972/3871] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:49.059420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a2a64-83ba-4848-8fff-77c7fdeeb682 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:49.062813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a2a64-83ba-4848-8fff-77c7fdeeb682 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:49.070078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9ff79-b051-4db9-bb58-6082351adfb6 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:49.071626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6ca9ff79-b051-4db9-bb58-6082351adfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:49.094920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9ff79-b051-4db9-bb58-6082351adfb6 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:49.099453 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1900/3872] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:59:49.102027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ea905-8ce5-4edd-8cd1-cabcabb4f3b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:49.102712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ea905-8ce5-4edd-8cd1-cabcabb4f3b5 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:49.103005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ea905-8ce5-4edd-8cd1-cabcabb4f3b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:49.103297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ea905-8ce5-4edd-8cd1-cabcabb4f3b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:49.120726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ea905-8ce5-4edd-8cd1-cabcabb4f3b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:49.122193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ea905-8ce5-4edd-8cd1-cabcabb4f3b5 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:49.142246 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1973/3873] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:59:49.988557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a2a64-83ba-4848-8fff-77c7fdeeb682 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:49.989997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3d8a2a64-83ba-4848-8fff-77c7fdeeb682 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:50.021275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a2a64-83ba-4848-8fff-77c7fdeeb682 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:50.035845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6339c7-1761-40a8-8e79-113f046fa4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:50.036604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6339c7-1761-40a8-8e79-113f046fa4b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:50.036992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6339c7-1761-40a8-8e79-113f046fa4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:50.037369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6339c7-1761-40a8-8e79-113f046fa4b0 admin admin] PATH_INFO: `/v3/projects/9c9cfa2e7f5348b4a525294ba662e4e6` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:50.039180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6339c7-1761-40a8-8e79-113f046fa4b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c9cfa2e7f5348b4a525294ba662e4e6)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:50.042981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6339c7-1761-40a8-8e79-113f046fa4b0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:50.082697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ea905-8ce5-4edd-8cd1-cabcabb4f3b5 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:50.084711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-849ea905-8ce5-4edd-8cd1-cabcabb4f3b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:50.109092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ea905-8ce5-4edd-8cd1-cabcabb4f3b5 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:50.126980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c82620-0554-4b6f-9fd6-1e5799c7f6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:50.127665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c82620-0554-4b6f-9fd6-1e5799c7f6a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:50.127998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c82620-0554-4b6f-9fd6-1e5799c7f6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:50.128241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c82620-0554-4b6f-9fd6-1e5799c7f6a5 admin admin] PATH_INFO: `/v3/users/f407ee9fccbf4fd6a9286642f8a015ad` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:50.130052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c82620-0554-4b6f-9fd6-1e5799c7f6a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f407ee9fccbf4fd6a9286642f8a015ad)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:50.132618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c82620-0554-4b6f-9fd6-1e5799c7f6a5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:50.141407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ad6339c7-1761-40a8-8e79-113f046fa4b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c9cfa2e7f5348b4a525294ba662e4e6', 'request_id': 'req-ad6339c7-1761-40a8-8e79-113f046fa4b0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:50.151760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ad6339c7-1761-40a8-8e79-113f046fa4b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c9cfa2e7f5348b4a525294ba662e4e6', 'request_id': 'req-ad6339c7-1761-40a8-8e79-113f046fa4b0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:50.161586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ad6339c7-1761-40a8-8e79-113f046fa4b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c9cfa2e7f5348b4a525294ba662e4e6', 'request_id': 'req-ad6339c7-1761-40a8-8e79-113f046fa4b0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:50.192972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ad6339c7-1761-40a8-8e79-113f046fa4b0 admin admin] The token cache is being invalidate because project 9c9cfa2e7f5348b4a525294ba662e4e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:59:50.195063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ad6339c7-1761-40a8-8e79-113f046fa4b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:50.196670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ad6339c7-1761-40a8-8e79-113f046fa4b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:50.203155 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1901/3874] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:59:49 2022] DELETE /identity/v3/projects/9c9cfa2e7f5348b4a525294ba662e4e6 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:50.213746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88739dad-5134-46d9-b15f-51f8300ab1c5 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:50.214040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88739dad-5134-46d9-b15f-51f8300ab1c5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:50.214303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88739dad-5134-46d9-b15f-51f8300ab1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:50.252120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b5c82620-0554-4b6f-9fd6-1e5799c7f6a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f407ee9fccbf4fd6a9286642f8a015ad'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:50.270345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b5c82620-0554-4b6f-9fd6-1e5799c7f6a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f407ee9fccbf4fd6a9286642f8a015ad'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:50.277465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-88739dad-5134-46d9-b15f-51f8300ab1c5 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:50.280792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-88739dad-5134-46d9-b15f-51f8300ab1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:50.298440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b5c82620-0554-4b6f-9fd6-1e5799c7f6a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f407ee9fccbf4fd6a9286642f8a015ad'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:50.301462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b5c82620-0554-4b6f-9fd6-1e5799c7f6a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f407ee9fccbf4fd6a9286642f8a015ad'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:50.310999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:50.319981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b5c82620-0554-4b6f-9fd6-1e5799c7f6a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f407ee9fccbf4fd6a9286642f8a015ad'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:50.334698 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1974/3875] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 20:59:50 2022] DELETE /identity/v3/users/f407ee9fccbf4fd6a9286642f8a015ad => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:59:50.409861 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1902/3876] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:50.417982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007f121-785a-4036-bc85-86b308f065fa None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:50.418373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007f121-785a-4036-bc85-86b308f065fa None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:50.418685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007f121-785a-4036-bc85-86b308f065fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:50.444537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0007f121-785a-4036-bc85-86b308f065fa None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:50.445912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0007f121-785a-4036-bc85-86b308f065fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:50.537593 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1975/3877] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:50.538334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88739dad-5134-46d9-b15f-51f8300ab1c5 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:50.550033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88739dad-5134-46d9-b15f-51f8300ab1c5 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:50.555231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1c042-3e11-4a05-b16c-1a7bb2486e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:50.556142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1c042-3e11-4a05-b16c-1a7bb2486e93 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:50.556598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1c042-3e11-4a05-b16c-1a7bb2486e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:50.557026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1c042-3e11-4a05-b16c-1a7bb2486e93 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:50.557962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1c042-3e11-4a05-b16c-1a7bb2486e93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:59:50.558836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1c042-3e11-4a05-b16c-1a7bb2486e93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:50.560911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1c042-3e11-4a05-b16c-1a7bb2486e93 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:50.585303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007f121-785a-4036-bc85-86b308f065fa None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:50.590706 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1903/3878] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:59:50.597821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007f121-785a-4036-bc85-86b308f065fa None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:50.604434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdc765-d0ae-4474-aa42-4375ea7cdc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:50.605055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc765-d0ae-4474-aa42-4375ea7cdc23 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:50.605333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc765-d0ae-4474-aa42-4375ea7cdc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:50.605598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc765-d0ae-4474-aa42-4375ea7cdc23 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:50.606467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fdc765-d0ae-4474-aa42-4375ea7cdc23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:50.609033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fdc765-d0ae-4474-aa42-4375ea7cdc23 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:50.651073 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1976/3879] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:59:50 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:50.658356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1c042-3e11-4a05-b16c-1a7bb2486e93 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:50.675827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1c042-3e11-4a05-b16c-1a7bb2486e93 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:50.814946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec1229-0721-456c-a872-9f0ae2c55a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:50.815692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec1229-0721-456c-a872-9f0ae2c55a6b admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:50.816007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec1229-0721-456c-a872-9f0ae2c55a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:50.816338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec1229-0721-456c-a872-9f0ae2c55a6b admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:50.817094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec1229-0721-456c-a872-9f0ae2c55a6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:50.820501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec1229-0721-456c-a872-9f0ae2c55a6b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:50.867991 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1904/3880] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:59:50 2022] POST /identity/v3/users => generated 291 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:50.871886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdc765-d0ae-4474-aa42-4375ea7cdc23 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:50.883437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:50.888086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdc765-d0ae-4474-aa42-4375ea7cdc23 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:50.901362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9e1e3-d702-48b0-9645-8f560e43d16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:50.902011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9e1e3-d702-48b0-9645-8f560e43d16d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:50.902477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9e1e3-d702-48b0-9645-8f560e43d16d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:50.902717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9e1e3-d702-48b0-9645-8f560e43d16d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:50.903898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9e1e3-d702-48b0-9645-8f560e43d16d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:50.927067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9e1e3-d702-48b0-9645-8f560e43d16d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:50.933885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec1229-0721-456c-a872-9f0ae2c55a6b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:50.936485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c9ec1229-0721-456c-a872-9f0ae2c55a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:50.976427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec1229-0721-456c-a872-9f0ae2c55a6b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:50.992299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67bba5-5932-4b08-b475-203f6148965c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:50.993251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67bba5-5932-4b08-b475-203f6148965c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:50.993251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67bba5-5932-4b08-b475-203f6148965c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:50.993488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67bba5-5932-4b08-b475-203f6148965c admin admin] PATH_INFO: `/v3/projects/7f84f3e4972f4a9c968100efa5775214` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:51.001034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67bba5-5932-4b08-b475-203f6148965c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f84f3e4972f4a9c968100efa5775214)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:51.003255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67bba5-5932-4b08-b475-203f6148965c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:51.004260 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1977/3881] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:59:51.008131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9e1e3-d702-48b0-9645-8f560e43d16d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:51.020020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9e1e3-d702-48b0-9645-8f560e43d16d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:51.046679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adce474-a177-454f-a146-9f6c56cf615f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:51.050791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adce474-a177-454f-a146-9f6c56cf615f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:51.051285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adce474-a177-454f-a146-9f6c56cf615f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:51.051624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adce474-a177-454f-a146-9f6c56cf615f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:51.052704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adce474-a177-454f-a146-9f6c56cf615f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:51.054109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adce474-a177-454f-a146-9f6c56cf615f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:51.080337 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1978/3882] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:59:51.084715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adce474-a177-454f-a146-9f6c56cf615f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:51.117866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adce474-a177-454f-a146-9f6c56cf615f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:51.130443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4d67bba5-5932-4b08-b475-203f6148965c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f84f3e4972f4a9c968100efa5775214', 'request_id': 'req-4d67bba5-5932-4b08-b475-203f6148965c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:51.144263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4d67bba5-5932-4b08-b475-203f6148965c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f84f3e4972f4a9c968100efa5775214', 'request_id': 'req-4d67bba5-5932-4b08-b475-203f6148965c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:51.166417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4d67bba5-5932-4b08-b475-203f6148965c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f84f3e4972f4a9c968100efa5775214', 'request_id': 'req-4d67bba5-5932-4b08-b475-203f6148965c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:51.207487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4d67bba5-5932-4b08-b475-203f6148965c admin admin] The token cache is being invalidate because project 7f84f3e4972f4a9c968100efa5775214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 20:59:51.208588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4d67bba5-5932-4b08-b475-203f6148965c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:51.212526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4d67bba5-5932-4b08-b475-203f6148965c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 20:59:51.215136 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1905/3883] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 20:59:50 2022] DELETE /identity/v3/projects/7f84f3e4972f4a9c968100efa5775214 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:59:51.220721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000a054-8c4b-43dc-8051-81e37320e7c6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:51.221533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000a054-8c4b-43dc-8051-81e37320e7c6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:51.222184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000a054-8c4b-43dc-8051-81e37320e7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:51.251268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0000a054-8c4b-43dc-8051-81e37320e7c6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:51.252506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0000a054-8c4b-43dc-8051-81e37320e7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:51.292765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:51.371456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40d573-0da5-4097-a462-4372e74cb380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:51.372211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40d573-0da5-4097-a462-4372e74cb380 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:51.372504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40d573-0da5-4097-a462-4372e74cb380 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:51.372660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40d573-0da5-4097-a462-4372e74cb380 admin admin] PATH_INFO: `/v3/projects/1e3f0598f3eb4c24822f8873832f343b/users/dbfb4cf3eb1a401aa2570f925c74a7e6/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:51.393021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40d573-0da5-4097-a462-4372e74cb380 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e3f0598f3eb4c24822f8873832f343b, user_id=dbfb4cf3eb1a401aa2570f925c74a7e6, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:51.396404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40d573-0da5-4097-a462-4372e74cb380 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:51.421230 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1906/3884] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:51.421459 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1979/3885] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:59:51 2022] PUT /identity/v3/projects/1e3f0598f3eb4c24822f8873832f343b/users/dbfb4cf3eb1a401aa2570f925c74a7e6/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 20:59:51.423032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da97da-cbe2-4b3a-bdb1-f9de4d97e789 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:51.423328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da97da-cbe2-4b3a-bdb1-f9de4d97e789 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:51.423673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da97da-cbe2-4b3a-bdb1-f9de4d97e789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:51.431797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69742f5d-3ba8-4284-9812-743dd9afcf90 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:51.432106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69742f5d-3ba8-4284-9812-743dd9afcf90 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:51.432319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69742f5d-3ba8-4284-9812-743dd9afcf90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:51.472970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e1da97da-cbe2-4b3a-bdb1-f9de4d97e789 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:51.474650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e1da97da-cbe2-4b3a-bdb1-f9de4d97e789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:51.484003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-69742f5d-3ba8-4284-9812-743dd9afcf90 None None] MFA Rules not processed for user `dbfb4cf3eb1a401aa2570f925c74a7e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:51.485802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-69742f5d-3ba8-4284-9812-743dd9afcf90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:51.601197 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1980/3886] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:51.606149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69742f5d-3ba8-4284-9812-743dd9afcf90 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:51.608236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-69742f5d-3ba8-4284-9812-743dd9afcf90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:51.625222 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1907/3887] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:51.629952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da97da-cbe2-4b3a-bdb1-f9de4d97e789 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:51.642875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da97da-cbe2-4b3a-bdb1-f9de4d97e789 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:51.648468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f374f5-557b-4dd5-9716-dda72fe81f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:51.649182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f374f5-557b-4dd5-9716-dda72fe81f56 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:51.649457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f374f5-557b-4dd5-9716-dda72fe81f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:51.649790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f374f5-557b-4dd5-9716-dda72fe81f56 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:51.650514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f374f5-557b-4dd5-9716-dda72fe81f56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 20:59:51.650902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f374f5-557b-4dd5-9716-dda72fe81f56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:51.652823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f374f5-557b-4dd5-9716-dda72fe81f56 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:51.673087 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1908/3888] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 20:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 20:59:51.675192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f374f5-557b-4dd5-9716-dda72fe81f56 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:51.690684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f374f5-557b-4dd5-9716-dda72fe81f56 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:51.704092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35017b19-9e81-4803-a636-92ba2198640f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:51.704756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35017b19-9e81-4803-a636-92ba2198640f admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:51.705002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35017b19-9e81-4803-a636-92ba2198640f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:51.705276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35017b19-9e81-4803-a636-92ba2198640f admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:51.706220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35017b19-9e81-4803-a636-92ba2198640f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:51.713970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35017b19-9e81-4803-a636-92ba2198640f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:51.718744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:51.728221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69742f5d-3ba8-4284-9812-743dd9afcf90 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:51.737576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51668d-da7e-4088-900d-912de8106c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:51.738819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51668d-da7e-4088-900d-912de8106c27 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:51.739383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51668d-da7e-4088-900d-912de8106c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:51.739878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51668d-da7e-4088-900d-912de8106c27 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:51.740939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51668d-da7e-4088-900d-912de8106c27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:51.743649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51668d-da7e-4088-900d-912de8106c27 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:51.782858 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1909/3889] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:59:51 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:51.784438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35017b19-9e81-4803-a636-92ba2198640f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:51.813108 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1981/3890] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:59:51 2022] POST /identity/v3/projects => generated 381 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:59:51.814795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51668d-da7e-4088-900d-912de8106c27 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:51.978943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35017b19-9e81-4803-a636-92ba2198640f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:51.997713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51668d-da7e-4088-900d-912de8106c27 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:51.999146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007bd1d-e035-4693-86b1-f81eed1fb5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:51.999777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007bd1d-e035-4693-86b1-f81eed1fb5f8 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:52.000056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007bd1d-e035-4693-86b1-f81eed1fb5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:52.000328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007bd1d-e035-4693-86b1-f81eed1fb5f8 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:52.000959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007bd1d-e035-4693-86b1-f81eed1fb5f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:52.002929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007bd1d-e035-4693-86b1-f81eed1fb5f8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:52.005198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6aa143-de4a-44b8-960a-3e9706010044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:52.005823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6aa143-de4a-44b8-960a-3e9706010044 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:52.006103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6aa143-de4a-44b8-960a-3e9706010044 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:52.006377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6aa143-de4a-44b8-960a-3e9706010044 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:52.007277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6aa143-de4a-44b8-960a-3e9706010044 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:52.009180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6aa143-de4a-44b8-960a-3e9706010044 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:52.067398 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1982/3891] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:59:51 2022] POST /identity/v3/users => generated 291 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:52.069307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6aa143-de4a-44b8-960a-3e9706010044 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:52.086046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6aa143-de4a-44b8-960a-3e9706010044 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:52.093442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42efee98-98ed-4acb-a486-4d650c938bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:52.094403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efee98-98ed-4acb-a486-4d650c938bee admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:52.094952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efee98-98ed-4acb-a486-4d650c938bee admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:52.095475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efee98-98ed-4acb-a486-4d650c938bee admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:52.098717 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1910/3892] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:59:51 2022] POST /identity/v3/users => generated 290 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 20:59:52.100105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007bd1d-e035-4693-86b1-f81eed1fb5f8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:52.100674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42efee98-98ed-4acb-a486-4d650c938bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:52.104919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42efee98-98ed-4acb-a486-4d650c938bee admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:52.116845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007bd1d-e035-4693-86b1-f81eed1fb5f8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:52.126797 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1983/3893] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:59:52.130173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42efee98-98ed-4acb-a486-4d650c938bee admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:52.131899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11226720-08dc-4ae3-94e8-1ad124289b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:52.132536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11226720-08dc-4ae3-94e8-1ad124289b19 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:52.132818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11226720-08dc-4ae3-94e8-1ad124289b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:52.133090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11226720-08dc-4ae3-94e8-1ad124289b19 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:52.135409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11226720-08dc-4ae3-94e8-1ad124289b19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:52.141564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11226720-08dc-4ae3-94e8-1ad124289b19 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:52.157253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42efee98-98ed-4acb-a486-4d650c938bee admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:52.167695 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1911/3894] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:59:52.168225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21af7b6-1051-49f9-8490-247445d5a236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:52.168853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21af7b6-1051-49f9-8490-247445d5a236 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:52.171662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21af7b6-1051-49f9-8490-247445d5a236 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:52.171978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21af7b6-1051-49f9-8490-247445d5a236 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:52.173126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11226720-08dc-4ae3-94e8-1ad124289b19 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:52.174025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21af7b6-1051-49f9-8490-247445d5a236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:52.185099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21af7b6-1051-49f9-8490-247445d5a236 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:52.187810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:52.196883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11226720-08dc-4ae3-94e8-1ad124289b19 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:52.206267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa060c58-034a-4e8a-9842-d0bafc902f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:52.207250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa060c58-034a-4e8a-9842-d0bafc902f93 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:52.207637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa060c58-034a-4e8a-9842-d0bafc902f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:52.208013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa060c58-034a-4e8a-9842-d0bafc902f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:52.208790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa060c58-034a-4e8a-9842-d0bafc902f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:52.210584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa060c58-034a-4e8a-9842-d0bafc902f93 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:52.221307 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1984/3895] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:59:52.225596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21af7b6-1051-49f9-8490-247445d5a236 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:52.229380 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1912/3896] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:59:52.234196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa060c58-034a-4e8a-9842-d0bafc902f93 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:52.244120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21af7b6-1051-49f9-8490-247445d5a236 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:52.245791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa060c58-034a-4e8a-9842-d0bafc902f93 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:52.252547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7bf1e-0a23-49b0-8add-26c49205cbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:52.252945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a9c5b4-fe3c-4faa-9aeb-bbd959740892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:52.253583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7bf1e-0a23-49b0-8add-26c49205cbe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:52.253996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9c5b4-fe3c-4faa-9aeb-bbd959740892 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:52.254268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7bf1e-0a23-49b0-8add-26c49205cbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:52.254376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9c5b4-fe3c-4faa-9aeb-bbd959740892 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:52.254553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9c5b4-fe3c-4faa-9aeb-bbd959740892 admin admin] PATH_INFO: `/v3/projects/4ad79e5d6445413588cd919bd2d8cd3c/users/bfca41094f1a4d45b18c099226150268/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:52.254654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7bf1e-0a23-49b0-8add-26c49205cbe1 admin admin] PATH_INFO: `/v3/projects/941a3ca19b4c4c5ea613c664ba054eda/users/6ba7625dbe4b46a2826f379ba6852b58/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:52.299352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a9c5b4-fe3c-4faa-9aeb-bbd959740892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ad79e5d6445413588cd919bd2d8cd3c, user_id=bfca41094f1a4d45b18c099226150268, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:52.304744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a9c5b4-fe3c-4faa-9aeb-bbd959740892 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:52.314105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7bf1e-0a23-49b0-8add-26c49205cbe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=941a3ca19b4c4c5ea613c664ba054eda, user_id=6ba7625dbe4b46a2826f379ba6852b58, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:52.322111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7bf1e-0a23-49b0-8add-26c49205cbe1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:52.337361 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1913/3897] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:59:52 2022] PUT /identity/v3/projects/4ad79e5d6445413588cd919bd2d8cd3c/users/bfca41094f1a4d45b18c099226150268/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:52.339099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8f7a2-324f-4d66-8d46-951e038902be None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:52.339486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8f7a2-324f-4d66-8d46-951e038902be None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:52.339769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8f7a2-324f-4d66-8d46-951e038902be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:52.346562 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1985/3898] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:59:52 2022] PUT /identity/v3/projects/941a3ca19b4c4c5ea613c664ba054eda/users/6ba7625dbe4b46a2826f379ba6852b58/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:52.355205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e36392-36f4-4f3d-a8a0-0423c11cf903 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:52.355745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e36392-36f4-4f3d-a8a0-0423c11cf903 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:52.356164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e36392-36f4-4f3d-a8a0-0423c11cf903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:52.394302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-28d8f7a2-324f-4d66-8d46-951e038902be None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:52.395622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-28d8f7a2-324f-4d66-8d46-951e038902be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:52.431272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-87e36392-36f4-4f3d-a8a0-0423c11cf903 None None] MFA Rules not processed for user `6ba7625dbe4b46a2826f379ba6852b58`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:52.436679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-87e36392-36f4-4f3d-a8a0-0423c11cf903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:52.532420 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1914/3899] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:52.572365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8f7a2-324f-4d66-8d46-951e038902be None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:52.578232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-28d8f7a2-324f-4d66-8d46-951e038902be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:52.588066 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1986/3900] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 20:59:52.594857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e36392-36f4-4f3d-a8a0-0423c11cf903 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:52.625997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e36392-36f4-4f3d-a8a0-0423c11cf903 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:52.639247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d6e73-7b04-4bcf-9c9f-36a7922e47fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:52.639966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d6e73-7b04-4bcf-9c9f-36a7922e47fd admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:52.640333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d6e73-7b04-4bcf-9c9f-36a7922e47fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:52.641999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d6e73-7b04-4bcf-9c9f-36a7922e47fd admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:52.642159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4d6e73-7b04-4bcf-9c9f-36a7922e47fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:52.646783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4d6e73-7b04-4bcf-9c9f-36a7922e47fd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:52.660323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:52.691413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8f7a2-324f-4d66-8d46-951e038902be None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:52.694923 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1987/3901] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 20:59:52 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:52.696645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4bcde-61fb-417c-aeb4-708fa7545832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:52.697275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4bcde-61fb-417c-aeb4-708fa7545832 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:52.697601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4bcde-61fb-417c-aeb4-708fa7545832 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:52.698401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4bcde-61fb-417c-aeb4-708fa7545832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:52.709765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d6e73-7b04-4bcf-9c9f-36a7922e47fd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:52.711533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4bcde-61fb-417c-aeb4-708fa7545832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:52.717427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4bcde-61fb-417c-aeb4-708fa7545832 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:52.741709 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1915/3902] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 20:59:52.867049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d6e73-7b04-4bcf-9c9f-36a7922e47fd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:52.873883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd0470-492a-4f1e-a212-32ca5262d7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:52.874771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd0470-492a-4f1e-a212-32ca5262d7b8 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:52.881171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd0470-492a-4f1e-a212-32ca5262d7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:52.886717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd0470-492a-4f1e-a212-32ca5262d7b8 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:52.890750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd0470-492a-4f1e-a212-32ca5262d7b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:52.893069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd0470-492a-4f1e-a212-32ca5262d7b8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:52.986956 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1988/3903] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 20:59:52 2022] POST /identity/v3/users => generated 297 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 20:59:52.987325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4bcde-61fb-417c-aeb4-708fa7545832 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:53.046257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4bcde-61fb-417c-aeb4-708fa7545832 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:53.055467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f3855-a9e7-4348-8208-56a2e17bc972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:53.056343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f3855-a9e7-4348-8208-56a2e17bc972 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:53.056895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f3855-a9e7-4348-8208-56a2e17bc972 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:53.057393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f3855-a9e7-4348-8208-56a2e17bc972 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:53.060466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f3855-a9e7-4348-8208-56a2e17bc972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:53.068653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f3855-a9e7-4348-8208-56a2e17bc972 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:53.096202 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1916/3904] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 20:59:53.097816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd0470-492a-4f1e-a212-32ca5262d7b8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:53.112059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd0470-492a-4f1e-a212-32ca5262d7b8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:53.117582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8767de5-8093-4154-8631-988119a7f91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:53.118239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8767de5-8093-4154-8631-988119a7f91b admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:53.118551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8767de5-8093-4154-8631-988119a7f91b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:53.118977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8767de5-8093-4154-8631-988119a7f91b admin admin] PATH_INFO: `/v3/projects/6f3c12a9ae894a9cbd5ce16f0accf355/users/c49cc11711e3465892b0a46b5b9c055a/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:53.141264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8767de5-8093-4154-8631-988119a7f91b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f3c12a9ae894a9cbd5ce16f0accf355, user_id=c49cc11711e3465892b0a46b5b9c055a, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:53.144643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8767de5-8093-4154-8631-988119a7f91b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:53.168115 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1989/3905] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:59:53 2022] PUT /identity/v3/projects/6f3c12a9ae894a9cbd5ce16f0accf355/users/c49cc11711e3465892b0a46b5b9c055a/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:53.172412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f3855-a9e7-4348-8208-56a2e17bc972 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:53.185462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f3855-a9e7-4348-8208-56a2e17bc972 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:53.331696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef39567-1463-4853-978a-27efd8dc0158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:53.332508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef39567-1463-4853-978a-27efd8dc0158 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:53.333349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef39567-1463-4853-978a-27efd8dc0158 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:53.333349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef39567-1463-4853-978a-27efd8dc0158 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:53.333878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef39567-1463-4853-978a-27efd8dc0158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:53.336210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef39567-1463-4853-978a-27efd8dc0158 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:53.337747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:53.365826 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1917/3906] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 20:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 20:59:53.369847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8767de5-8093-4154-8631-988119a7f91b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:53.385960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8767de5-8093-4154-8631-988119a7f91b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:53.398217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67072b48-8a2b-4a04-839c-bfd7387a8cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:53.399506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67072b48-8a2b-4a04-839c-bfd7387a8cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:53.399896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67072b48-8a2b-4a04-839c-bfd7387a8cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:53.400258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67072b48-8a2b-4a04-839c-bfd7387a8cdd admin admin] PATH_INFO: `/v3/projects/6f3c12a9ae894a9cbd5ce16f0accf355/users/c49cc11711e3465892b0a46b5b9c055a/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:53.407091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67072b48-8a2b-4a04-839c-bfd7387a8cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f3c12a9ae894a9cbd5ce16f0accf355, user_id=c49cc11711e3465892b0a46b5b9c055a, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:53.414455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67072b48-8a2b-4a04-839c-bfd7387a8cdd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:53.440559 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1990/3907] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 20:59:53 2022] PUT /identity/v3/projects/6f3c12a9ae894a9cbd5ce16f0accf355/users/c49cc11711e3465892b0a46b5b9c055a/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 20:59:53.452150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7f146-a52a-4d5c-a1a2-71cb7c58aaea None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:53.452302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7f146-a52a-4d5c-a1a2-71cb7c58aaea None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:53.452576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7f146-a52a-4d5c-a1a2-71cb7c58aaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:53.528275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-07d7f146-a52a-4d5c-a1a2-71cb7c58aaea None None] MFA Rules not processed for user `c49cc11711e3465892b0a46b5b9c055a`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 20:59:53.543567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-07d7f146-a52a-4d5c-a1a2-71cb7c58aaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:53.757940 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1918/3908] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 20:59:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 20:59:53.782733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67072b48-8a2b-4a04-839c-bfd7387a8cdd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 20:59:53.787356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-67072b48-8a2b-4a04-839c-bfd7387a8cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 20:59:53.904332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67072b48-8a2b-4a04-839c-bfd7387a8cdd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 20:59:53.909574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ead898-0780-4c51-8958-5e3f57fdee7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 20:59:53.910808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ead898-0780-4c51-8958-5e3f57fdee7b service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 20:59:53.911195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ead898-0780-4c51-8958-5e3f57fdee7b service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 20:59:53.911660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ead898-0780-4c51-8958-5e3f57fdee7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 20:59:54.018563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ead898-0780-4c51-8958-5e3f57fdee7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 20:59:54.020478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ead898-0780-4c51-8958-5e3f57fdee7b service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 20:59:54.027130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:54.046222 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1991/3909] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 20:59:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:00:00.360585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b227788-a787-440b-9c75-40964f7452af None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:00.361130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b227788-a787-440b-9c75-40964f7452af None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:00.361236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b227788-a787-440b-9c75-40964f7452af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:00.436172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4b227788-a787-440b-9c75-40964f7452af None None] MFA Rules not processed for user `0779a741ea8946b28756ddd87fd7a792`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:00.437777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4b227788-a787-440b-9c75-40964f7452af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:00.577726 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1919/3910] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:00 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:00.949727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ead898-0780-4c51-8958-5e3f57fdee7b service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:00.952249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b4ead898-0780-4c51-8958-5e3f57fdee7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:01.141349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ead898-0780-4c51-8958-5e3f57fdee7b service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:01.169460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953afb1d-0d27-4563-b21e-24439b0175e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:01.170614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953afb1d-0d27-4563-b21e-24439b0175e8 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:01.171124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953afb1d-0d27-4563-b21e-24439b0175e8 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:01.171602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953afb1d-0d27-4563-b21e-24439b0175e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:01.186472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953afb1d-0d27-4563-b21e-24439b0175e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:01.188801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953afb1d-0d27-4563-b21e-24439b0175e8 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:01.219005 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1992/3911] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:00:00 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:00:01.957510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3daec73-3871-44e4-9d76-636134e2328b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:01.958527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3daec73-3871-44e4-9d76-636134e2328b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:01.959134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3daec73-3871-44e4-9d76-636134e2328b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:02.008464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b3daec73-3871-44e4-9d76-636134e2328b None None] MFA Rules not processed for user `bfca41094f1a4d45b18c099226150268`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:02.010625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b3daec73-3871-44e4-9d76-636134e2328b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:02.105512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:02.111062 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1920/3912] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:03.638647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953afb1d-0d27-4563-b21e-24439b0175e8 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:03.656322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953afb1d-0d27-4563-b21e-24439b0175e8 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:03.666148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951e5bf-fb74-4147-b7ff-da30e4864846 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:03.666881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951e5bf-fb74-4147-b7ff-da30e4864846 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:03.667384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951e5bf-fb74-4147-b7ff-da30e4864846 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:03.667726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951e5bf-fb74-4147-b7ff-da30e4864846 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:03.683798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951e5bf-fb74-4147-b7ff-da30e4864846 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:03.685718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951e5bf-fb74-4147-b7ff-da30e4864846 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:03.725784 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1993/3913] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:00:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:00:12.173087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3daec73-3871-44e4-9d76-636134e2328b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:12.174579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b3daec73-3871-44e4-9d76-636134e2328b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:12.327276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3daec73-3871-44e4-9d76-636134e2328b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:12.333155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e0536-29cf-4e1d-bc30-91f508b1b4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:12.334135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e0536-29cf-4e1d-bc30-91f508b1b4af admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:12.334977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e0536-29cf-4e1d-bc30-91f508b1b4af admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:12.335613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e0536-29cf-4e1d-bc30-91f508b1b4af admin admin] PATH_INFO: `/v3/users/0779a741ea8946b28756ddd87fd7a792` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:12.337110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e0536-29cf-4e1d-bc30-91f508b1b4af admin admin] RBAC: Authorizing `identity:delete_user(user_id=0779a741ea8946b28756ddd87fd7a792)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:12.361540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e0536-29cf-4e1d-bc30-91f508b1b4af admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:12.452108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3b5e0536-29cf-4e1d-bc30-91f508b1b4af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0779a741ea8946b28756ddd87fd7a792'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:12.452774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3b5e0536-29cf-4e1d-bc30-91f508b1b4af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0779a741ea8946b28756ddd87fd7a792'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:12.453954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3b5e0536-29cf-4e1d-bc30-91f508b1b4af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0779a741ea8946b28756ddd87fd7a792'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:12.466715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3b5e0536-29cf-4e1d-bc30-91f508b1b4af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0779a741ea8946b28756ddd87fd7a792'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:12.491974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3b5e0536-29cf-4e1d-bc30-91f508b1b4af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0779a741ea8946b28756ddd87fd7a792'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:12.516768 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1921/3914] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:00:12 2022] DELETE /identity/v3/users/0779a741ea8946b28756ddd87fd7a792 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:00:12.573582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951e5bf-fb74-4147-b7ff-da30e4864846 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:12.597347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951e5bf-fb74-4147-b7ff-da30e4864846 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:12.722764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dadced7-140d-42a7-bb76-78be5d1de9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:12.723737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dadced7-140d-42a7-bb76-78be5d1de9ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:12.724208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dadced7-140d-42a7-bb76-78be5d1de9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:12.729725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dadced7-140d-42a7-bb76-78be5d1de9ed admin admin] PATH_INFO: `/v3/users/bfca41094f1a4d45b18c099226150268` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:12.734251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dadced7-140d-42a7-bb76-78be5d1de9ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfca41094f1a4d45b18c099226150268)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:12.736762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dadced7-140d-42a7-bb76-78be5d1de9ed admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:12.847343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2dadced7-140d-42a7-bb76-78be5d1de9ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfca41094f1a4d45b18c099226150268'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:12.866949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2dadced7-140d-42a7-bb76-78be5d1de9ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfca41094f1a4d45b18c099226150268'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:12.911554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2dadced7-140d-42a7-bb76-78be5d1de9ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfca41094f1a4d45b18c099226150268'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:12.913541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2dadced7-140d-42a7-bb76-78be5d1de9ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfca41094f1a4d45b18c099226150268'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:12.915841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:12.928213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2dadced7-140d-42a7-bb76-78be5d1de9ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfca41094f1a4d45b18c099226150268'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:12.938769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e0536-29cf-4e1d-bc30-91f508b1b4af admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:12.940461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3b5e0536-29cf-4e1d-bc30-91f508b1b4af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:12.974830 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1994/3915] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:00:12 2022] DELETE /identity/v3/users/bfca41094f1a4d45b18c099226150268 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:00:12.977512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e0536-29cf-4e1d-bc30-91f508b1b4af admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:13.102431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dadced7-140d-42a7-bb76-78be5d1de9ed admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:13.104141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2dadced7-140d-42a7-bb76-78be5d1de9ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:13.144194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8ad71-1fee-4640-a9c2-d0669f6326fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:13.145195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8ad71-1fee-4640-a9c2-d0669f6326fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:13.145582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8ad71-1fee-4640-a9c2-d0669f6326fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:13.146396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8ad71-1fee-4640-a9c2-d0669f6326fe admin admin] PATH_INFO: `/v3/projects/840cdc42d8e1415da91572fac902d284` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:13.147743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8ad71-1fee-4640-a9c2-d0669f6326fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=840cdc42d8e1415da91572fac902d284)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:13.150001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8ad71-1fee-4640-a9c2-d0669f6326fe admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:13.216550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:13.255906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dadced7-140d-42a7-bb76-78be5d1de9ed admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:13.283962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-55f8ad71-1fee-4640-a9c2-d0669f6326fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '840cdc42d8e1415da91572fac902d284', 'request_id': 'req-55f8ad71-1fee-4640-a9c2-d0669f6326fe'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:13.298590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-55f8ad71-1fee-4640-a9c2-d0669f6326fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '840cdc42d8e1415da91572fac902d284', 'request_id': 'req-55f8ad71-1fee-4640-a9c2-d0669f6326fe'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:13.307651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-55f8ad71-1fee-4640-a9c2-d0669f6326fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '840cdc42d8e1415da91572fac902d284', 'request_id': 'req-55f8ad71-1fee-4640-a9c2-d0669f6326fe'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:13.333300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-55f8ad71-1fee-4640-a9c2-d0669f6326fe admin admin] The token cache is being invalidate because project 840cdc42d8e1415da91572fac902d284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:00:13.333516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-55f8ad71-1fee-4640-a9c2-d0669f6326fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:13.337471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-55f8ad71-1fee-4640-a9c2-d0669f6326fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:13.339536 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1922/3916] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:00:12 2022] DELETE /identity/v3/projects/840cdc42d8e1415da91572fac902d284 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:00:13.344884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f4124-3ef9-43f7-9c0e-481e56ed9032 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:13.345179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f4124-3ef9-43f7-9c0e-481e56ed9032 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:13.345461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f4124-3ef9-43f7-9c0e-481e56ed9032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:13.370002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4d649-25fd-4076-89ad-015fe6a4c20d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:13.370611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4d649-25fd-4076-89ad-015fe6a4c20d service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:13.370899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4d649-25fd-4076-89ad-015fe6a4c20d service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:13.371174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4d649-25fd-4076-89ad-015fe6a4c20d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:13.383292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f41f4124-3ef9-43f7-9c0e-481e56ed9032 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:13.387517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-07d4d649-25fd-4076-89ad-015fe6a4c20d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:13.394345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f41f4124-3ef9-43f7-9c0e-481e56ed9032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:13.534607 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1923/3917] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:13.536076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f4124-3ef9-43f7-9c0e-481e56ed9032 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:13.537786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f41f4124-3ef9-43f7-9c0e-481e56ed9032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:13.546997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4d649-25fd-4076-89ad-015fe6a4c20d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:13.550377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4d649-25fd-4076-89ad-015fe6a4c20d service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:13.581871 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1995/3918] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:00:13 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:00:13.583447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed6762-920e-4331-9646-28e716155f27 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:13.583725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed6762-920e-4331-9646-28e716155f27 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:13.588443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed6762-920e-4331-9646-28e716155f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:13.595997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f4124-3ef9-43f7-9c0e-481e56ed9032 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:13.607199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa3717-e31b-4e7f-8aa9-d3f091cb4aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:13.618115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa3717-e31b-4e7f-8aa9-d3f091cb4aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:13.618547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa3717-e31b-4e7f-8aa9-d3f091cb4aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:13.618840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa3717-e31b-4e7f-8aa9-d3f091cb4aa0 admin admin] PATH_INFO: `/v3/projects/4ad79e5d6445413588cd919bd2d8cd3c` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:13.624164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa3717-e31b-4e7f-8aa9-d3f091cb4aa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ad79e5d6445413588cd919bd2d8cd3c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:13.635467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa3717-e31b-4e7f-8aa9-d3f091cb4aa0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:13.661111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a8ed6762-920e-4331-9646-28e716155f27 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:13.668384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a8ed6762-920e-4331-9646-28e716155f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:13.692627 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1996/3919] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:13.696505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed6762-920e-4331-9646-28e716155f27 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:13.704155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-87fa3717-e31b-4e7f-8aa9-d3f091cb4aa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ad79e5d6445413588cd919bd2d8cd3c', 'request_id': 'req-87fa3717-e31b-4e7f-8aa9-d3f091cb4aa0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:13.713542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed6762-920e-4331-9646-28e716155f27 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:13.719859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-87fa3717-e31b-4e7f-8aa9-d3f091cb4aa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ad79e5d6445413588cd919bd2d8cd3c', 'request_id': 'req-87fa3717-e31b-4e7f-8aa9-d3f091cb4aa0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:13.734343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-87fa3717-e31b-4e7f-8aa9-d3f091cb4aa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ad79e5d6445413588cd919bd2d8cd3c', 'request_id': 'req-87fa3717-e31b-4e7f-8aa9-d3f091cb4aa0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:13.763769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-87fa3717-e31b-4e7f-8aa9-d3f091cb4aa0 admin admin] The token cache is being invalidate because project 4ad79e5d6445413588cd919bd2d8cd3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:00:13.764151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-87fa3717-e31b-4e7f-8aa9-d3f091cb4aa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:13.764929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-87fa3717-e31b-4e7f-8aa9-d3f091cb4aa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:13.768204 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1924/3920] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:00:13 2022] DELETE /identity/v3/projects/4ad79e5d6445413588cd919bd2d8cd3c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:00:13.775296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5347f4-0949-426b-8874-daca025667b7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:13.775625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5347f4-0949-426b-8874-daca025667b7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:13.776181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5347f4-0949-426b-8874-daca025667b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:13.793612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:13.812092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841228a0-70a6-4807-a38b-8b5b8185bc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:13.812807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841228a0-70a6-4807-a38b-8b5b8185bc49 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:13.813366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841228a0-70a6-4807-a38b-8b5b8185bc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:13.813853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841228a0-70a6-4807-a38b-8b5b8185bc49 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:13.814675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841228a0-70a6-4807-a38b-8b5b8185bc49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:00:13.815657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841228a0-70a6-4807-a38b-8b5b8185bc49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:13.817283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841228a0-70a6-4807-a38b-8b5b8185bc49 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:13.831421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8f5347f4-0949-426b-8874-daca025667b7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:13.833047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8f5347f4-0949-426b-8874-daca025667b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:13.835885 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1997/3921] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:00:13.836809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841228a0-70a6-4807-a38b-8b5b8185bc49 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:13.838771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-841228a0-70a6-4807-a38b-8b5b8185bc49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:13.854554 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1925/3922] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:13.859084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3d070-e031-4d76-88be-f58ce8533750 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:13.859678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3d070-e031-4d76-88be-f58ce8533750 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:13.860867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3d070-e031-4d76-88be-f58ce8533750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:13.865641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841228a0-70a6-4807-a38b-8b5b8185bc49 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:13.873960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f89d15-6b8b-43bb-a9ca-780ad7d796f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:13.876216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f89d15-6b8b-43bb-a9ca-780ad7d796f8 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:13.876674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f89d15-6b8b-43bb-a9ca-780ad7d796f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:13.877105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f89d15-6b8b-43bb-a9ca-780ad7d796f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:13.877902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f89d15-6b8b-43bb-a9ca-780ad7d796f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:13.880825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f89d15-6b8b-43bb-a9ca-780ad7d796f8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:13.889030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-23d3d070-e031-4d76-88be-f58ce8533750 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:13.890751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-23d3d070-e031-4d76-88be-f58ce8533750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:13.901798 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1926/3923] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:00:13.906809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3d070-e031-4d76-88be-f58ce8533750 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:13.918113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3d070-e031-4d76-88be-f58ce8533750 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:13.938353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64acf78-b289-4fa7-9c0c-9bc9266446fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:13.939223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64acf78-b289-4fa7-9c0c-9bc9266446fe admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:13.939383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64acf78-b289-4fa7-9c0c-9bc9266446fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:13.939742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64acf78-b289-4fa7-9c0c-9bc9266446fe admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:13.940399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64acf78-b289-4fa7-9c0c-9bc9266446fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:00:13.940716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64acf78-b289-4fa7-9c0c-9bc9266446fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:13.942341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64acf78-b289-4fa7-9c0c-9bc9266446fe admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:13.947469 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1998/3924] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:00:13 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:00:13.952165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f89d15-6b8b-43bb-a9ca-780ad7d796f8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:13.957406 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1927/3925] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:00:13.961896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64acf78-b289-4fa7-9c0c-9bc9266446fe admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:13.974814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64acf78-b289-4fa7-9c0c-9bc9266446fe admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:13.984419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:14.070195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f89d15-6b8b-43bb-a9ca-780ad7d796f8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:14.078803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9244a-2b55-4b67-9c71-ae61a6ee3b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:14.082264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9244a-2b55-4b67-9c71-ae61a6ee3b87 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:14.082717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9244a-2b55-4b67-9c71-ae61a6ee3b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:14.083071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9244a-2b55-4b67-9c71-ae61a6ee3b87 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:14.083756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d9244a-2b55-4b67-9c71-ae61a6ee3b87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:14.085993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d9244a-2b55-4b67-9c71-ae61a6ee3b87 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:14.086478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0e3de-ca62-48cb-a147-a1727198785d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:14.087465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0e3de-ca62-48cb-a147-a1727198785d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:14.088035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0e3de-ca62-48cb-a147-a1727198785d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:14.088455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0e3de-ca62-48cb-a147-a1727198785d admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:14.089224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d0e3de-ca62-48cb-a147-a1727198785d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:14.091928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d0e3de-ca62-48cb-a147-a1727198785d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:14.122939 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 1999/3926] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:00:13 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:00:14.124033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9244a-2b55-4b67-9c71-ae61a6ee3b87 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:14.134540 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1928/3927] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:00:13 2022] POST /identity/v3/projects => generated 397 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:00:14.139771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0e3de-ca62-48cb-a147-a1727198785d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:14.141805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9244a-2b55-4b67-9c71-ae61a6ee3b87 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:14.248885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca70e6-afd6-41da-b618-25c4f5d1fe65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:14.249506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca70e6-afd6-41da-b618-25c4f5d1fe65 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:14.249862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca70e6-afd6-41da-b618-25c4f5d1fe65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:14.250121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca70e6-afd6-41da-b618-25c4f5d1fe65 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:14.250850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca70e6-afd6-41da-b618-25c4f5d1fe65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:14.252607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca70e6-afd6-41da-b618-25c4f5d1fe65 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:14.264090 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2000/3928] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:00:14.268079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca70e6-afd6-41da-b618-25c4f5d1fe65 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:14.278516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca70e6-afd6-41da-b618-25c4f5d1fe65 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:14.280164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0e3de-ca62-48cb-a147-a1727198785d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:14.287549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ddf00-e896-4062-b64c-f843c26e2814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:14.288286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ddf00-e896-4062-b64c-f843c26e2814 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:14.289278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ddf00-e896-4062-b64c-f843c26e2814 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:14.289730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ddf00-e896-4062-b64c-f843c26e2814 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:14.290626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ddf00-e896-4062-b64c-f843c26e2814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:14.290751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1191da-d2cb-4bfc-b5c7-bb288ced2d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:14.292766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ddf00-e896-4062-b64c-f843c26e2814 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:14.294578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1191da-d2cb-4bfc-b5c7-bb288ced2d69 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:14.294872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1191da-d2cb-4bfc-b5c7-bb288ced2d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:14.295150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1191da-d2cb-4bfc-b5c7-bb288ced2d69 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:14.295937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1191da-d2cb-4bfc-b5c7-bb288ced2d69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:14.298010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1191da-d2cb-4bfc-b5c7-bb288ced2d69 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:14.320028 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2001/3929] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:00:14.322289 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1929/3930] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:00:14 2022] POST /identity/v3/users => generated 299 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:00:14.323997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1191da-d2cb-4bfc-b5c7-bb288ced2d69 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:14.329108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ddf00-e896-4062-b64c-f843c26e2814 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:14.335744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1191da-d2cb-4bfc-b5c7-bb288ced2d69 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:14.341900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d673d-58ee-45f6-b0cb-6f8f8ca61d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:14.342521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d673d-58ee-45f6-b0cb-6f8f8ca61d8a admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:14.342803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d673d-58ee-45f6-b0cb-6f8f8ca61d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:14.343076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d673d-58ee-45f6-b0cb-6f8f8ca61d8a admin admin] PATH_INFO: `/v3/projects/91d32a0fa04648de8ec34b7937b36ef3/users/ce758fa3950048549fbc375a80deb1a7/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:14.343641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ddf00-e896-4062-b64c-f843c26e2814 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:14.348857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeabbbc-dd27-42ea-a56d-23d8de25e22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:14.349769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeabbbc-dd27-42ea-a56d-23d8de25e22b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:14.350140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeabbbc-dd27-42ea-a56d-23d8de25e22b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:14.350556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeabbbc-dd27-42ea-a56d-23d8de25e22b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:14.351452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeabbbc-dd27-42ea-a56d-23d8de25e22b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:14.354328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeabbbc-dd27-42ea-a56d-23d8de25e22b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:14.369245 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1930/3931] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:00:14.371020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d673d-58ee-45f6-b0cb-6f8f8ca61d8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=91d32a0fa04648de8ec34b7937b36ef3, user_id=ce758fa3950048549fbc375a80deb1a7, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:14.374333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d673d-58ee-45f6-b0cb-6f8f8ca61d8a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:14.375644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeabbbc-dd27-42ea-a56d-23d8de25e22b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:14.394642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeabbbc-dd27-42ea-a56d-23d8de25e22b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:14.401016 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2002/3932] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:00:14 2022] PUT /identity/v3/projects/91d32a0fa04648de8ec34b7937b36ef3/users/ce758fa3950048549fbc375a80deb1a7/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:00:14.407909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e8dbd-2c93-48db-b45d-d8cc5cd09612 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:14.408322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e8dbd-2c93-48db-b45d-d8cc5cd09612 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:14.408599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e8dbd-2c93-48db-b45d-d8cc5cd09612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:14.439264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-943e8dbd-2c93-48db-b45d-d8cc5cd09612 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:14.440474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-943e8dbd-2c93-48db-b45d-d8cc5cd09612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:14.446457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:14.487989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7286c6-2e83-46ce-91d9-4fde8af8d98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:14.488953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7286c6-2e83-46ce-91d9-4fde8af8d98a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:14.491212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7286c6-2e83-46ce-91d9-4fde8af8d98a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:14.491648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7286c6-2e83-46ce-91d9-4fde8af8d98a admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:14.492448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7286c6-2e83-46ce-91d9-4fde8af8d98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:14.495327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7286c6-2e83-46ce-91d9-4fde8af8d98a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:14.519272 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1931/3933] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:00:14.522922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7286c6-2e83-46ce-91d9-4fde8af8d98a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:14.539265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7286c6-2e83-46ce-91d9-4fde8af8d98a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:14.544515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df840ca7-7e1a-4b3c-bb20-43ed6a56e7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:14.544704 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2003/3934] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:14.545499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df840ca7-7e1a-4b3c-bb20-43ed6a56e7be admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:14.545930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df840ca7-7e1a-4b3c-bb20-43ed6a56e7be admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:14.546350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df840ca7-7e1a-4b3c-bb20-43ed6a56e7be admin admin] PATH_INFO: `/v3/projects/b9e916618c504ec2b23064c883ba7d56/users/7cd23af965b147248c5f8c7b9098fc3d/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:14.549327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:14.554793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e8dbd-2c93-48db-b45d-d8cc5cd09612 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:14.556223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-943e8dbd-2c93-48db-b45d-d8cc5cd09612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:14.568574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df840ca7-7e1a-4b3c-bb20-43ed6a56e7be admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9e916618c504ec2b23064c883ba7d56, user_id=7cd23af965b147248c5f8c7b9098fc3d, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:14.572804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df840ca7-7e1a-4b3c-bb20-43ed6a56e7be admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:14.595330 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1932/3935] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:00:14 2022] PUT /identity/v3/projects/b9e916618c504ec2b23064c883ba7d56/users/7cd23af965b147248c5f8c7b9098fc3d/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:00:14.596594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebe666-a622-4125-aaf1-fec53b75f92f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:14.597062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebe666-a622-4125-aaf1-fec53b75f92f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:14.597346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebe666-a622-4125-aaf1-fec53b75f92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:14.633438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-99ebe666-a622-4125-aaf1-fec53b75f92f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:14.636536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-99ebe666-a622-4125-aaf1-fec53b75f92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:14.702832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e8dbd-2c93-48db-b45d-d8cc5cd09612 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:14.750891 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1933/3936] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:14.761346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebe666-a622-4125-aaf1-fec53b75f92f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:14.820404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e252bb-8feb-4355-bd37-ac3fc97aa646 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:14.821381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e252bb-8feb-4355-bd37-ac3fc97aa646 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:14.821838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e252bb-8feb-4355-bd37-ac3fc97aa646 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:14.822275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e252bb-8feb-4355-bd37-ac3fc97aa646 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:14.834821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e252bb-8feb-4355-bd37-ac3fc97aa646 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:14.837031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e252bb-8feb-4355-bd37-ac3fc97aa646 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:14.837643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebe666-a622-4125-aaf1-fec53b75f92f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:14.841485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade4c9d-fcbd-4146-9af3-c800253557f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:14.842096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade4c9d-fcbd-4146-9af3-c800253557f3 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:14.842376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade4c9d-fcbd-4146-9af3-c800253557f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:14.842685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade4c9d-fcbd-4146-9af3-c800253557f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:14.852394 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2004/3937] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:00:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:00:14.853249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade4c9d-fcbd-4146-9af3-c800253557f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:14.854821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade4c9d-fcbd-4146-9af3-c800253557f3 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:14.866197 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1934/3938] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:00:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:00:18.236218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094e14d-87b5-4d54-b448-0781d6c74db9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:18.236766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094e14d-87b5-4d54-b448-0781d6c74db9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:18.236984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094e14d-87b5-4d54-b448-0781d6c74db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:18.285077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9094e14d-87b5-4d54-b448-0781d6c74db9 None None] MFA Rules not processed for user `e84a109e337d450698e630acfb49f9a5`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:18.288075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9094e14d-87b5-4d54-b448-0781d6c74db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:18.431780 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2005/3939] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:24.295297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49139f-e21f-40ad-a9f7-4a9bbc98bafa None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:24.296320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49139f-e21f-40ad-a9f7-4a9bbc98bafa None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:24.296856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49139f-e21f-40ad-a9f7-4a9bbc98bafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:24.405326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fb49139f-e21f-40ad-a9f7-4a9bbc98bafa None None] MFA Rules not processed for user `7cd23af965b147248c5f8c7b9098fc3d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:24.407264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fb49139f-e21f-40ad-a9f7-4a9bbc98bafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:24.524988 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1935/3940] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:24 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:24.539156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f026aca-f542-4986-8ce0-5f86bc0ddbf7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:24.539747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f026aca-f542-4986-8ce0-5f86bc0ddbf7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:24.539747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f026aca-f542-4986-8ce0-5f86bc0ddbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:24.571171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2f026aca-f542-4986-8ce0-5f86bc0ddbf7 None None] MFA Rules not processed for user `7cd23af965b147248c5f8c7b9098fc3d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:24.572640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2f026aca-f542-4986-8ce0-5f86bc0ddbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:24.599980 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2006/3941] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:24 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:24.861984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff22e4a-da7d-45a8-ad63-4785badfdd0c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:24.862275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff22e4a-da7d-45a8-ad63-4785badfdd0c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:24.862590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff22e4a-da7d-45a8-ad63-4785badfdd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:24.916123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4ff22e4a-da7d-45a8-ad63-4785badfdd0c None None] MFA Rules not processed for user `775ff9091a794faaacc05a37550c8a72`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:24.917410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4ff22e4a-da7d-45a8-ad63-4785badfdd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:24.940993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:25.073923 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1936/3942] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:25.213361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f026aca-f542-4986-8ce0-5f86bc0ddbf7 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:25.242637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f026aca-f542-4986-8ce0-5f86bc0ddbf7 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:25.254573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0356e3-3f44-45e5-95f6-0149f5ad36ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:25.255226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0356e3-3f44-45e5-95f6-0149f5ad36ed service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:25.255742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0356e3-3f44-45e5-95f6-0149f5ad36ed service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:25.256221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0356e3-3f44-45e5-95f6-0149f5ad36ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:25.258876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0f0356e3-3f44-45e5-95f6-0149f5ad36ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:25.285102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:25.322114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff22e4a-da7d-45a8-ad63-4785badfdd0c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:25.323816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4ff22e4a-da7d-45a8-ad63-4785badfdd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:25.374760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0356e3-3f44-45e5-95f6-0149f5ad36ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:25.376207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0356e3-3f44-45e5-95f6-0149f5ad36ed service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:25.387652 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2007/3943] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:00:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:00:25.464989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff22e4a-da7d-45a8-ad63-4785badfdd0c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:25.471243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ad0f7e-ac47-418f-b6dc-ca103ea458bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:25.471837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad0f7e-ac47-418f-b6dc-ca103ea458bd service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:25.472128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad0f7e-ac47-418f-b6dc-ca103ea458bd service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:25.472408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad0f7e-ac47-418f-b6dc-ca103ea458bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:25.481939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ad0f7e-ac47-418f-b6dc-ca103ea458bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:25.483809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ad0f7e-ac47-418f-b6dc-ca103ea458bd service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:25.500906 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1937/3944] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:00:25 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:00:26.955382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a247ba-4af1-4be6-87f8-047a65b84290 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:26.956458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a247ba-4af1-4be6-87f8-047a65b84290 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:26.957154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a247ba-4af1-4be6-87f8-047a65b84290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:27.019274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d9a247ba-4af1-4be6-87f8-047a65b84290 None None] MFA Rules not processed for user `ce758fa3950048549fbc375a80deb1a7`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:27.021742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d9a247ba-4af1-4be6-87f8-047a65b84290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:27.180164 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2008/3945] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:29.874880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ad0f7e-ac47-418f-b6dc-ca103ea458bd service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:29.890367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ad0f7e-ac47-418f-b6dc-ca103ea458bd service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:29.899606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81868d6-7233-4a54-b55e-bfb919ae11ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:29.901233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81868d6-7233-4a54-b55e-bfb919ae11ef service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:29.901787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81868d6-7233-4a54-b55e-bfb919ae11ef service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:29.902278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81868d6-7233-4a54-b55e-bfb919ae11ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:29.912301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81868d6-7233-4a54-b55e-bfb919ae11ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:29.914619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81868d6-7233-4a54-b55e-bfb919ae11ef service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:29.926193 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1938/3946] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:00:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:00:34.263186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c5199-c383-4b1f-a15a-d148a06c9eaf None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:34.264184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c5199-c383-4b1f-a15a-d148a06c9eaf None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:34.264665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c5199-c383-4b1f-a15a-d148a06c9eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:34.303343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-351c5199-c383-4b1f-a15a-d148a06c9eaf None None] MFA Rules not processed for user `ce758fa3950048549fbc375a80deb1a7`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:34.307255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-351c5199-c383-4b1f-a15a-d148a06c9eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:34.329304 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2009/3947] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:39.921301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=58 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:40.102704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81868d6-7233-4a54-b55e-bfb919ae11ef service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:40.112732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81868d6-7233-4a54-b55e-bfb919ae11ef service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:40.119726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716dd536-1ec1-4084-826e-a7ea667d41bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:40.120601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716dd536-1ec1-4084-826e-a7ea667d41bc service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:40.121665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716dd536-1ec1-4084-826e-a7ea667d41bc service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:40.122148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716dd536-1ec1-4084-826e-a7ea667d41bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:40.124262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-716dd536-1ec1-4084-826e-a7ea667d41bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:40.285317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716dd536-1ec1-4084-826e-a7ea667d41bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:40.287097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716dd536-1ec1-4084-826e-a7ea667d41bc service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:40.300025 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1939/3948] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:00:40 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:00:41.397920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c5199-c383-4b1f-a15a-d148a06c9eaf None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:41.400079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-351c5199-c383-4b1f-a15a-d148a06c9eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:41.416043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c5199-c383-4b1f-a15a-d148a06c9eaf None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:41.422972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf4ceb-c66e-4c79-8d2b-397f0165c800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:41.424353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf4ceb-c66e-4c79-8d2b-397f0165c800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:41.424851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf4ceb-c66e-4c79-8d2b-397f0165c800 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:41.425321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf4ceb-c66e-4c79-8d2b-397f0165c800 admin admin] PATH_INFO: `/v3/users/dbfb4cf3eb1a401aa2570f925c74a7e6` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:41.426834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf4ceb-c66e-4c79-8d2b-397f0165c800 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbfb4cf3eb1a401aa2570f925c74a7e6)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:41.429022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf4ceb-c66e-4c79-8d2b-397f0165c800 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:41.509840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-99bf4ceb-c66e-4c79-8d2b-397f0165c800 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbfb4cf3eb1a401aa2570f925c74a7e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:41.533354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-99bf4ceb-c66e-4c79-8d2b-397f0165c800 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbfb4cf3eb1a401aa2570f925c74a7e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:41.575266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-99bf4ceb-c66e-4c79-8d2b-397f0165c800 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbfb4cf3eb1a401aa2570f925c74a7e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:41.577175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-99bf4ceb-c66e-4c79-8d2b-397f0165c800 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbfb4cf3eb1a401aa2570f925c74a7e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:41.595527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-99bf4ceb-c66e-4c79-8d2b-397f0165c800 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbfb4cf3eb1a401aa2570f925c74a7e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:41.612885 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2010/3949] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:00:41 2022] DELETE /identity/v3/users/dbfb4cf3eb1a401aa2570f925c74a7e6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:00:41.613719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740bbcd-4f9a-45ee-9ea8-98bd318b2acf None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:41.614202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740bbcd-4f9a-45ee-9ea8-98bd318b2acf None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:41.614287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740bbcd-4f9a-45ee-9ea8-98bd318b2acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:41.640576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1740bbcd-4f9a-45ee-9ea8-98bd318b2acf None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:41.642853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1740bbcd-4f9a-45ee-9ea8-98bd318b2acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:41.732988 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1940/3950] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:00:41.754144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf4ceb-c66e-4c79-8d2b-397f0165c800 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:41.755668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-99bf4ceb-c66e-4c79-8d2b-397f0165c800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:41.762302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:41.851350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf4ceb-c66e-4c79-8d2b-397f0165c800 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:41.856135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f99f21-951e-4da7-b7be-549c315187f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:41.856754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f99f21-951e-4da7-b7be-549c315187f7 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:41.857034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f99f21-951e-4da7-b7be-549c315187f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:41.857308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f99f21-951e-4da7-b7be-549c315187f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:41.868514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f99f21-951e-4da7-b7be-549c315187f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:41.869959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f99f21-951e-4da7-b7be-549c315187f7 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:41.881590 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2011/3951] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:00:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:00:42.281458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740bbcd-4f9a-45ee-9ea8-98bd318b2acf None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:42.283342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1740bbcd-4f9a-45ee-9ea8-98bd318b2acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:42.332454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740bbcd-4f9a-45ee-9ea8-98bd318b2acf None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:42.344288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13944e0e-9e8d-47cc-ae4c-402f46dc7c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:42.346105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13944e0e-9e8d-47cc-ae4c-402f46dc7c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:42.347612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13944e0e-9e8d-47cc-ae4c-402f46dc7c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:42.348191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13944e0e-9e8d-47cc-ae4c-402f46dc7c6f admin admin] PATH_INFO: `/v3/projects/1e3f0598f3eb4c24822f8873832f343b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:42.353145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13944e0e-9e8d-47cc-ae4c-402f46dc7c6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e3f0598f3eb4c24822f8873832f343b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:42.358434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13944e0e-9e8d-47cc-ae4c-402f46dc7c6f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:42.373233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f9b04-af51-404d-8b67-ad3b16c22450 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:42.373572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f9b04-af51-404d-8b67-ad3b16c22450 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:42.373784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f9b04-af51-404d-8b67-ad3b16c22450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:42.404936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9a7f9b04-af51-404d-8b67-ad3b16c22450 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:42.406684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9a7f9b04-af51-404d-8b67-ad3b16c22450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:42.450771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-13944e0e-9e8d-47cc-ae4c-402f46dc7c6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e3f0598f3eb4c24822f8873832f343b', 'request_id': 'req-13944e0e-9e8d-47cc-ae4c-402f46dc7c6f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:42.462230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-13944e0e-9e8d-47cc-ae4c-402f46dc7c6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e3f0598f3eb4c24822f8873832f343b', 'request_id': 'req-13944e0e-9e8d-47cc-ae4c-402f46dc7c6f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:42.471259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-13944e0e-9e8d-47cc-ae4c-402f46dc7c6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e3f0598f3eb4c24822f8873832f343b', 'request_id': 'req-13944e0e-9e8d-47cc-ae4c-402f46dc7c6f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:42.487022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-13944e0e-9e8d-47cc-ae4c-402f46dc7c6f admin admin] The token cache is being invalidate because project 1e3f0598f3eb4c24822f8873832f343b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:00:42.487601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-13944e0e-9e8d-47cc-ae4c-402f46dc7c6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:42.488523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-13944e0e-9e8d-47cc-ae4c-402f46dc7c6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:42.489715 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2012/3952] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:00:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:00:42.490937 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1941/3953] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:00:42 2022] DELETE /identity/v3/projects/1e3f0598f3eb4c24822f8873832f343b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:00:42.494776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f9b04-af51-404d-8b67-ad3b16c22450 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:42.517686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f9b04-af51-404d-8b67-ad3b16c22450 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:42.545318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13944e0e-9e8d-47cc-ae4c-402f46dc7c6f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:42.546991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-13944e0e-9e8d-47cc-ae4c-402f46dc7c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:42.576771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:42.640741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3149eb-f3af-472f-9fa2-011d74162f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:42.641593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3149eb-f3af-472f-9fa2-011d74162f0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:42.641883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3149eb-f3af-472f-9fa2-011d74162f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:42.642250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3149eb-f3af-472f-9fa2-011d74162f0a admin admin] PATH_INFO: `/v3/users/6ba7625dbe4b46a2826f379ba6852b58` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:42.643905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3149eb-f3af-472f-9fa2-011d74162f0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ba7625dbe4b46a2826f379ba6852b58)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:42.646966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3149eb-f3af-472f-9fa2-011d74162f0a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:42.681054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13944e0e-9e8d-47cc-ae4c-402f46dc7c6f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:42.689051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f884d3-f141-4a22-9308-8a296f8c3b7d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:42.690076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f884d3-f141-4a22-9308-8a296f8c3b7d service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:42.690663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f884d3-f141-4a22-9308-8a296f8c3b7d service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:42.691429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f884d3-f141-4a22-9308-8a296f8c3b7d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:42.699834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-78f884d3-f141-4a22-9308-8a296f8c3b7d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:42.724926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ef3149eb-f3af-472f-9fa2-011d74162f0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ba7625dbe4b46a2826f379ba6852b58'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:42.744600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:42.757034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ef3149eb-f3af-472f-9fa2-011d74162f0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ba7625dbe4b46a2826f379ba6852b58'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:42.775251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ef3149eb-f3af-472f-9fa2-011d74162f0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ba7625dbe4b46a2826f379ba6852b58'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:42.776173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ef3149eb-f3af-472f-9fa2-011d74162f0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ba7625dbe4b46a2826f379ba6852b58'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:42.791624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ef3149eb-f3af-472f-9fa2-011d74162f0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ba7625dbe4b46a2826f379ba6852b58'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:42.805522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f884d3-f141-4a22-9308-8a296f8c3b7d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:42.809106 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2013/3954] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:00:42 2022] DELETE /identity/v3/users/6ba7625dbe4b46a2826f379ba6852b58 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:00:42.809281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f884d3-f141-4a22-9308-8a296f8c3b7d service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:42.822151 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1942/3955] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:00:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 21:00:43.151230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa316972-a1b7-41d2-965f-c3bd247f417c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:43.151519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa316972-a1b7-41d2-965f-c3bd247f417c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:43.152473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa316972-a1b7-41d2-965f-c3bd247f417c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:43.197141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fa316972-a1b7-41d2-965f-c3bd247f417c None None] MFA Rules not processed for user `69002d46d4e54eadadf911ec3589d95c`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:43.201423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fa316972-a1b7-41d2-965f-c3bd247f417c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:43.335177 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2014/3956] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:43.723998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f884d3-f141-4a22-9308-8a296f8c3b7d service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:43.725836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-78f884d3-f141-4a22-9308-8a296f8c3b7d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:43.752733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f9914-b95e-4771-9d19-f35afc34ad45 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:43.754868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f9914-b95e-4771-9d19-f35afc34ad45 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:43.756887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f9914-b95e-4771-9d19-f35afc34ad45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:43.775106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f884d3-f141-4a22-9308-8a296f8c3b7d service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:43.819428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-588f9914-b95e-4771-9d19-f35afc34ad45 None None] MFA Rules not processed for user `7cd23af965b147248c5f8c7b9098fc3d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:43.821990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-588f9914-b95e-4771-9d19-f35afc34ad45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:43.925484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3dd6e-0e8f-460c-8779-3f0237202760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:43.927037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3dd6e-0e8f-460c-8779-3f0237202760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:43.927345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3dd6e-0e8f-460c-8779-3f0237202760 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:43.928610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3dd6e-0e8f-460c-8779-3f0237202760 admin admin] PATH_INFO: `/v3/projects/941a3ca19b4c4c5ea613c664ba054eda` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:43.931987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c3dd6e-0e8f-460c-8779-3f0237202760 admin admin] RBAC: Authorizing `identity:delete_project(project_id=941a3ca19b4c4c5ea613c664ba054eda)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:43.938539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c3dd6e-0e8f-460c-8779-3f0237202760 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:43.947241 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2015/3957] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:43 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:44.048603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-35c3dd6e-0e8f-460c-8779-3f0237202760 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '941a3ca19b4c4c5ea613c664ba054eda', 'request_id': 'req-35c3dd6e-0e8f-460c-8779-3f0237202760'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:44.063338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-35c3dd6e-0e8f-460c-8779-3f0237202760 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '941a3ca19b4c4c5ea613c664ba054eda', 'request_id': 'req-35c3dd6e-0e8f-460c-8779-3f0237202760'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:44.070619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:44.075444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-35c3dd6e-0e8f-460c-8779-3f0237202760 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '941a3ca19b4c4c5ea613c664ba054eda', 'request_id': 'req-35c3dd6e-0e8f-460c-8779-3f0237202760'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:44.093112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-35c3dd6e-0e8f-460c-8779-3f0237202760 admin admin] The token cache is being invalidate because project 941a3ca19b4c4c5ea613c664ba054eda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:00:44.093328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-35c3dd6e-0e8f-460c-8779-3f0237202760 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:44.094304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-35c3dd6e-0e8f-460c-8779-3f0237202760 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:44.097596 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1943/3958] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:00:43 2022] DELETE /identity/v3/projects/941a3ca19b4c4c5ea613c664ba054eda => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:00:44.102581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f9914-b95e-4771-9d19-f35afc34ad45 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:44.107798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-588f9914-b95e-4771-9d19-f35afc34ad45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:44.118714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3dd6e-0e8f-460c-8779-3f0237202760 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:44.122448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-35c3dd6e-0e8f-460c-8779-3f0237202760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:44.132089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f9914-b95e-4771-9d19-f35afc34ad45 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:44.269020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3dd6e-0e8f-460c-8779-3f0237202760 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:44.275813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060bdb6-87b8-4b83-9963-263caec8d526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:44.276562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060bdb6-87b8-4b83-9963-263caec8d526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:44.277429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060bdb6-87b8-4b83-9963-263caec8d526 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:44.277904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060bdb6-87b8-4b83-9963-263caec8d526 admin admin] PATH_INFO: `/v3/users/c49cc11711e3465892b0a46b5b9c055a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:44.280124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060bdb6-87b8-4b83-9963-263caec8d526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c49cc11711e3465892b0a46b5b9c055a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:44.280590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565671ea-535a-42db-b2ba-cb26edfd1330 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:44.281551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565671ea-535a-42db-b2ba-cb26edfd1330 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:44.282039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565671ea-535a-42db-b2ba-cb26edfd1330 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:44.282521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565671ea-535a-42db-b2ba-cb26edfd1330 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:44.285138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060bdb6-87b8-4b83-9963-263caec8d526 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:44.286167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-565671ea-535a-42db-b2ba-cb26edfd1330 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:44.318306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:44.319869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565671ea-535a-42db-b2ba-cb26edfd1330 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:44.323851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565671ea-535a-42db-b2ba-cb26edfd1330 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:44.338139 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1944/3959] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:00:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:00:44.365320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9060bdb6-87b8-4b83-9963-263caec8d526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c49cc11711e3465892b0a46b5b9c055a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:44.375473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9060bdb6-87b8-4b83-9963-263caec8d526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c49cc11711e3465892b0a46b5b9c055a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:44.392675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9060bdb6-87b8-4b83-9963-263caec8d526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c49cc11711e3465892b0a46b5b9c055a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:44.393436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9060bdb6-87b8-4b83-9963-263caec8d526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c49cc11711e3465892b0a46b5b9c055a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:44.403547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9060bdb6-87b8-4b83-9963-263caec8d526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c49cc11711e3465892b0a46b5b9c055a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:44.413488 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2016/3960] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:00:44 2022] DELETE /identity/v3/users/c49cc11711e3465892b0a46b5b9c055a => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:00:44.882769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565671ea-535a-42db-b2ba-cb26edfd1330 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:44.885262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-565671ea-535a-42db-b2ba-cb26edfd1330 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:44.903608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565671ea-535a-42db-b2ba-cb26edfd1330 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:45.018883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ce75c-3071-42d4-9d81-a192a1256463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:45.019721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ce75c-3071-42d4-9d81-a192a1256463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:45.020065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ce75c-3071-42d4-9d81-a192a1256463 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:45.021448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ce75c-3071-42d4-9d81-a192a1256463 admin admin] PATH_INFO: `/v3/projects/6f3c12a9ae894a9cbd5ce16f0accf355` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:45.022736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ce75c-3071-42d4-9d81-a192a1256463 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f3c12a9ae894a9cbd5ce16f0accf355)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:45.035276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ce75c-3071-42d4-9d81-a192a1256463 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:45.118763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1f0ce75c-3071-42d4-9d81-a192a1256463 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f3c12a9ae894a9cbd5ce16f0accf355', 'request_id': 'req-1f0ce75c-3071-42d4-9d81-a192a1256463'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:45.133930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1f0ce75c-3071-42d4-9d81-a192a1256463 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f3c12a9ae894a9cbd5ce16f0accf355', 'request_id': 'req-1f0ce75c-3071-42d4-9d81-a192a1256463'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:45.151060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1f0ce75c-3071-42d4-9d81-a192a1256463 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f3c12a9ae894a9cbd5ce16f0accf355', 'request_id': 'req-1f0ce75c-3071-42d4-9d81-a192a1256463'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:45.176636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1f0ce75c-3071-42d4-9d81-a192a1256463 admin admin] The token cache is being invalidate because project 6f3c12a9ae894a9cbd5ce16f0accf355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:00:45.177239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1f0ce75c-3071-42d4-9d81-a192a1256463 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:45.181209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1f0ce75c-3071-42d4-9d81-a192a1256463 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:45.192044 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1945/3961] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:00:44 2022] DELETE /identity/v3/projects/6f3c12a9ae894a9cbd5ce16f0accf355 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:00:45.198095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bacfbb9-8b06-4420-91b8-b2e5e1c2ee7a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:45.199686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bacfbb9-8b06-4420-91b8-b2e5e1c2ee7a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:45.200153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bacfbb9-8b06-4420-91b8-b2e5e1c2ee7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:45.239468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9bacfbb9-8b06-4420-91b8-b2e5e1c2ee7a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:45.241162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9bacfbb9-8b06-4420-91b8-b2e5e1c2ee7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:45.342091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea633dcd-b283-4f0d-a421-87de4239b7b9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:45.342328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea633dcd-b283-4f0d-a421-87de4239b7b9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:45.342611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea633dcd-b283-4f0d-a421-87de4239b7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:45.358716 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2017/3962] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:45.384249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ea633dcd-b283-4f0d-a421-87de4239b7b9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:45.385666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ea633dcd-b283-4f0d-a421-87de4239b7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:45.404651 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1946/3963] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:45.405516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bacfbb9-8b06-4420-91b8-b2e5e1c2ee7a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:45.442522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bacfbb9-8b06-4420-91b8-b2e5e1c2ee7a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:45.464577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6aab4-8eba-4873-a7d8-2da99a362b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:45.465540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6aab4-8eba-4873-a7d8-2da99a362b4e admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:45.466006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6aab4-8eba-4873-a7d8-2da99a362b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:45.466379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6aab4-8eba-4873-a7d8-2da99a362b4e admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:45.467174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6aab4-8eba-4873-a7d8-2da99a362b4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:00:45.467578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6aab4-8eba-4873-a7d8-2da99a362b4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:45.469490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6aab4-8eba-4873-a7d8-2da99a362b4e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:45.502780 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2018/3964] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:00:45.507050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea633dcd-b283-4f0d-a421-87de4239b7b9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:45.525052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:45.531748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea633dcd-b283-4f0d-a421-87de4239b7b9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:45.545805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2cc10c-a31a-4bcb-b879-c18e20837ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:45.546866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2cc10c-a31a-4bcb-b879-c18e20837ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:45.547002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2cc10c-a31a-4bcb-b879-c18e20837ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:45.547279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2cc10c-a31a-4bcb-b879-c18e20837ff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:45.547911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2cc10c-a31a-4bcb-b879-c18e20837ff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:45.553506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2cc10c-a31a-4bcb-b879-c18e20837ff0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:45.612462 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1947/3965] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:00:45 2022] POST /identity/v3/projects => generated 367 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:00:45.616365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6aab4-8eba-4873-a7d8-2da99a362b4e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:45.630721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6aab4-8eba-4873-a7d8-2da99a362b4e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:45.740047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45882606-6b18-452e-8002-3968fc7d78a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:45.740981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45882606-6b18-452e-8002-3968fc7d78a0 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:45.741416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45882606-6b18-452e-8002-3968fc7d78a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:45.741841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45882606-6b18-452e-8002-3968fc7d78a0 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:45.742659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45882606-6b18-452e-8002-3968fc7d78a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:45.744725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45882606-6b18-452e-8002-3968fc7d78a0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:45.777504 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2019/3966] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:00:45 2022] POST /identity/v3/users => generated 284 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:00:45.778580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2cc10c-a31a-4bcb-b879-c18e20837ff0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:45.792103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2cc10c-a31a-4bcb-b879-c18e20837ff0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:45.805351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc120c-0d74-43cb-9c2b-8ad0e8812752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:45.806209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc120c-0d74-43cb-9c2b-8ad0e8812752 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:45.806717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc120c-0d74-43cb-9c2b-8ad0e8812752 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:45.807218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc120c-0d74-43cb-9c2b-8ad0e8812752 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:45.808898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc120c-0d74-43cb-9c2b-8ad0e8812752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:45.811132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc120c-0d74-43cb-9c2b-8ad0e8812752 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:45.826914 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1948/3967] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:00:45.828353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45882606-6b18-452e-8002-3968fc7d78a0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:45.843172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45882606-6b18-452e-8002-3968fc7d78a0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:45.849578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50176d7f-37d7-4f53-97fc-d4e39b113351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:45.850204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50176d7f-37d7-4f53-97fc-d4e39b113351 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:45.850610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50176d7f-37d7-4f53-97fc-d4e39b113351 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:45.850959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50176d7f-37d7-4f53-97fc-d4e39b113351 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:45.851766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50176d7f-37d7-4f53-97fc-d4e39b113351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:45.853597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50176d7f-37d7-4f53-97fc-d4e39b113351 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:45.867991 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2020/3968] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:00:45.872040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc120c-0d74-43cb-9c2b-8ad0e8812752 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:45.895574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc120c-0d74-43cb-9c2b-8ad0e8812752 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:45.926945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0095d27-89dc-4eaf-926c-89b0eb705072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:45.935781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0095d27-89dc-4eaf-926c-89b0eb705072 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:45.936078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0095d27-89dc-4eaf-926c-89b0eb705072 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:45.936364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0095d27-89dc-4eaf-926c-89b0eb705072 admin admin] PATH_INFO: `/v3/projects/bf470fc5619243f0a15b79ad010fafcc/users/b3f5a85ec6eb414ea20b8be9c2b9d3b1/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:45.980588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0095d27-89dc-4eaf-926c-89b0eb705072 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf470fc5619243f0a15b79ad010fafcc, user_id=b3f5a85ec6eb414ea20b8be9c2b9d3b1, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:45.985377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0095d27-89dc-4eaf-926c-89b0eb705072 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:46.007823 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1949/3969] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:00:45 2022] PUT /identity/v3/projects/bf470fc5619243f0a15b79ad010fafcc/users/b3f5a85ec6eb414ea20b8be9c2b9d3b1/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:00:46.020290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39885f7d-56eb-4c75-9d64-58ff18111444 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:46.020780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39885f7d-56eb-4c75-9d64-58ff18111444 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:46.021196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39885f7d-56eb-4c75-9d64-58ff18111444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:46.066985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:46.099154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-39885f7d-56eb-4c75-9d64-58ff18111444 None None] MFA Rules not processed for user `b3f5a85ec6eb414ea20b8be9c2b9d3b1`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:46.100613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-39885f7d-56eb-4c75-9d64-58ff18111444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:46.178532 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2021/3970] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:46.183440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0095d27-89dc-4eaf-926c-89b0eb705072 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:46.289441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0095d27-89dc-4eaf-926c-89b0eb705072 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:46.298333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff9dd5-5a97-4dec-b022-0824e8c15dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:46.299764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff9dd5-5a97-4dec-b022-0824e8c15dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:46.300261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff9dd5-5a97-4dec-b022-0824e8c15dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:46.301016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff9dd5-5a97-4dec-b022-0824e8c15dc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:46.302029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff9dd5-5a97-4dec-b022-0824e8c15dc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:46.304731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff9dd5-5a97-4dec-b022-0824e8c15dc7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:46.350530 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1950/3971] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:00:46 2022] POST /identity/v3/projects => generated 369 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:00:46.357300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39885f7d-56eb-4c75-9d64-58ff18111444 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:46.359567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166e63d-f300-4166-b125-4645f3fa92e8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:46.360093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166e63d-f300-4166-b125-4645f3fa92e8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:46.361228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166e63d-f300-4166-b125-4645f3fa92e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:46.379305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39885f7d-56eb-4c75-9d64-58ff18111444 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:46.397004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2166e63d-f300-4166-b125-4645f3fa92e8 None None] MFA Rules not processed for user `7cd23af965b147248c5f8c7b9098fc3d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:46.399017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2166e63d-f300-4166-b125-4645f3fa92e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:46.497618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a5129-d368-4773-808e-3bccafe7bf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:46.498236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a5129-d368-4773-808e-3bccafe7bf0d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:46.498534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a5129-d368-4773-808e-3bccafe7bf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:46.498810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a5129-d368-4773-808e-3bccafe7bf0d admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:46.499626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3a5129-d368-4773-808e-3bccafe7bf0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:46.501527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3a5129-d368-4773-808e-3bccafe7bf0d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:46.546416 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1951/3972] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:46 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:46.552282 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2022/3973] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:00:46 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:00:46.553646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166e63d-f300-4166-b125-4645f3fa92e8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:46.567649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166e63d-f300-4166-b125-4645f3fa92e8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:46.574004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e6edb-b91e-4694-b6b6-36be765bcf84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:46.574681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e6edb-b91e-4694-b6b6-36be765bcf84 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:46.574965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e6edb-b91e-4694-b6b6-36be765bcf84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:46.575244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e6edb-b91e-4694-b6b6-36be765bcf84 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:46.575897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039e6edb-b91e-4694-b6b6-36be765bcf84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:46.577893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039e6edb-b91e-4694-b6b6-36be765bcf84 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:46.596750 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1952/3974] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:00:46.600878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a5129-d368-4773-808e-3bccafe7bf0d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:46.616006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a5129-d368-4773-808e-3bccafe7bf0d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:46.631647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c2731-04a4-4f4e-9718-0ee8767249f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:46.634023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c2731-04a4-4f4e-9718-0ee8767249f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:46.635642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c2731-04a4-4f4e-9718-0ee8767249f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:46.636236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c2731-04a4-4f4e-9718-0ee8767249f8 admin admin] PATH_INFO: `/v3/projects/ab00d1a09a824f9aa55939cc08cc524c/users/a67229cfd2524cd9a438f72acbf476aa/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:46.684325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c2731-04a4-4f4e-9718-0ee8767249f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab00d1a09a824f9aa55939cc08cc524c, user_id=a67229cfd2524cd9a438f72acbf476aa, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:46.687283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c2731-04a4-4f4e-9718-0ee8767249f8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:46.717574 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2023/3975] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:00:46 2022] PUT /identity/v3/projects/ab00d1a09a824f9aa55939cc08cc524c/users/a67229cfd2524cd9a438f72acbf476aa/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:00:46.721692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e6edb-b91e-4694-b6b6-36be765bcf84 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:46.738717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e6edb-b91e-4694-b6b6-36be765bcf84 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:46.782147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:46.870151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92df954-b06c-41f6-8e28-8294bf3d1f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:46.872238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92df954-b06c-41f6-8e28-8294bf3d1f45 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:46.872594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92df954-b06c-41f6-8e28-8294bf3d1f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:46.873005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92df954-b06c-41f6-8e28-8294bf3d1f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:46.873742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92df954-b06c-41f6-8e28-8294bf3d1f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:46.878011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92df954-b06c-41f6-8e28-8294bf3d1f45 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:46.889738 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1953/3976] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:00:46.893641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c2731-04a4-4f4e-9718-0ee8767249f8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:46.911548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c2731-04a4-4f4e-9718-0ee8767249f8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:46.923224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8f9ef-7c96-483c-ab3f-d92d3a85d38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:46.925453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8f9ef-7c96-483c-ab3f-d92d3a85d38d admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:46.925965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8f9ef-7c96-483c-ab3f-d92d3a85d38d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:46.926728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8f9ef-7c96-483c-ab3f-d92d3a85d38d admin admin] PATH_INFO: `/v3/projects/ab00d1a09a824f9aa55939cc08cc524c/users/a67229cfd2524cd9a438f72acbf476aa/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:46.929775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8f9ef-7c96-483c-ab3f-d92d3a85d38d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab00d1a09a824f9aa55939cc08cc524c, user_id=a67229cfd2524cd9a438f72acbf476aa, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:46.934474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8f9ef-7c96-483c-ab3f-d92d3a85d38d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:46.960651 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2024/3977] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:00:46 2022] PUT /identity/v3/projects/ab00d1a09a824f9aa55939cc08cc524c/users/a67229cfd2524cd9a438f72acbf476aa/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:00:46.963104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31c90f-1166-4a07-882a-22ed579d4676 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:46.963593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31c90f-1166-4a07-882a-22ed579d4676 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:46.963786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31c90f-1166-4a07-882a-22ed579d4676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:47.002800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7a31c90f-1166-4a07-882a-22ed579d4676 None None] MFA Rules not processed for user `a67229cfd2524cd9a438f72acbf476aa`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:47.004130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7a31c90f-1166-4a07-882a-22ed579d4676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:47.155996 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1954/3978] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:47.173187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8f9ef-7c96-483c-ab3f-d92d3a85d38d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:47.175269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c1b8f9ef-7c96-483c-ab3f-d92d3a85d38d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:47.242678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd96ce-201f-4c08-9f02-2450dfd44149 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:47.243079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd96ce-201f-4c08-9f02-2450dfd44149 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:47.243291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd96ce-201f-4c08-9f02-2450dfd44149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:47.275460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a0fd96ce-201f-4c08-9f02-2450dfd44149 None None] MFA Rules not processed for user `7cd23af965b147248c5f8c7b9098fc3d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:47.278848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:47.281235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a0fd96ce-201f-4c08-9f02-2450dfd44149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:47.293248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8f9ef-7c96-483c-ab3f-d92d3a85d38d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:47.304645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8d0b2-78da-4320-b9b3-a8c5664c510f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:47.305415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8d0b2-78da-4320-b9b3-a8c5664c510f service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:47.305801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8d0b2-78da-4320-b9b3-a8c5664c510f service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:47.306101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8d0b2-78da-4320-b9b3-a8c5664c510f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:47.334279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8d0b2-78da-4320-b9b3-a8c5664c510f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:47.335946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8d0b2-78da-4320-b9b3-a8c5664c510f service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:47.372502 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2025/3979] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:00:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:00:47.420192 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1955/3980] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:47 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:00:48.708609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981a53e-60a2-430f-b709-002a413a0237 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:48.709109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981a53e-60a2-430f-b709-002a413a0237 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:48.709109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981a53e-60a2-430f-b709-002a413a0237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:48.750749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b981a53e-60a2-430f-b709-002a413a0237 None None] MFA Rules not processed for user `7cd23af965b147248c5f8c7b9098fc3d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:48.755219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b981a53e-60a2-430f-b709-002a413a0237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:48.773954 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2026/3981] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:48 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:00:48.825725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39c162-65f6-4c82-bfbc-402806524f94 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:48.826215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39c162-65f6-4c82-bfbc-402806524f94 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:48.826306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39c162-65f6-4c82-bfbc-402806524f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:48.852549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-eb39c162-65f6-4c82-bfbc-402806524f94 None None] MFA Rules not processed for user `6ade3ba047d348a387e8e4077072b90a`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:48.854264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-eb39c162-65f6-4c82-bfbc-402806524f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:48.952196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:48.981251 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1956/3982] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:49.487337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbebd7-bacf-47f6-889e-559ddb867b1a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:49.487785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbebd7-bacf-47f6-889e-559ddb867b1a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:49.488167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbebd7-bacf-47f6-889e-559ddb867b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:49.522090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-72cbebd7-bacf-47f6-889e-559ddb867b1a None None] MFA Rules not processed for user `7cd23af965b147248c5f8c7b9098fc3d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:49.530454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-72cbebd7-bacf-47f6-889e-559ddb867b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:49.549191 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2027/3983] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:49 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:50.141223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39c162-65f6-4c82-bfbc-402806524f94 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:50.323268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39c162-65f6-4c82-bfbc-402806524f94 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:50.342899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c560cf3-686e-4ba8-ac86-239e093818e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:50.343499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c560cf3-686e-4ba8-ac86-239e093818e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:50.343783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c560cf3-686e-4ba8-ac86-239e093818e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:50.344150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c560cf3-686e-4ba8-ac86-239e093818e4 admin admin] PATH_INFO: `/v3/users/b3f5a85ec6eb414ea20b8be9c2b9d3b1` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:50.345817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c560cf3-686e-4ba8-ac86-239e093818e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3f5a85ec6eb414ea20b8be9c2b9d3b1)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:50.348045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c560cf3-686e-4ba8-ac86-239e093818e4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:50.418340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9c560cf3-686e-4ba8-ac86-239e093818e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3f5a85ec6eb414ea20b8be9c2b9d3b1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:50.420450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9c560cf3-686e-4ba8-ac86-239e093818e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3f5a85ec6eb414ea20b8be9c2b9d3b1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:50.422582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9c560cf3-686e-4ba8-ac86-239e093818e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3f5a85ec6eb414ea20b8be9c2b9d3b1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:50.437130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9c560cf3-686e-4ba8-ac86-239e093818e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3f5a85ec6eb414ea20b8be9c2b9d3b1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:50.452218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9c560cf3-686e-4ba8-ac86-239e093818e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3f5a85ec6eb414ea20b8be9c2b9d3b1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:50.466677 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1957/3984] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:00:50 2022] DELETE /identity/v3/users/b3f5a85ec6eb414ea20b8be9c2b9d3b1 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:00:50.471022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbfd143-ca27-4867-9ad0-c156c199a6b7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:50.471855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbfd143-ca27-4867-9ad0-c156c199a6b7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:50.472310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbfd143-ca27-4867-9ad0-c156c199a6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:50.508150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6dbfd143-ca27-4867-9ad0-c156c199a6b7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:50.511795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6dbfd143-ca27-4867-9ad0-c156c199a6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:50.588297 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2028/3985] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:50.600126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c560cf3-686e-4ba8-ac86-239e093818e4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:50.603144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9c560cf3-686e-4ba8-ac86-239e093818e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:50.683644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c560cf3-686e-4ba8-ac86-239e093818e4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:50.690530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40fbb1-e49b-42c7-a4e0-e89113fc9bc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:50.691248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40fbb1-e49b-42c7-a4e0-e89113fc9bc2 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:50.691605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40fbb1-e49b-42c7-a4e0-e89113fc9bc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:50.691954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40fbb1-e49b-42c7-a4e0-e89113fc9bc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:50.703409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d40fbb1-e49b-42c7-a4e0-e89113fc9bc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:50.705123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d40fbb1-e49b-42c7-a4e0-e89113fc9bc2 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:50.717196 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1958/3986] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:00:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:00:51.362574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbfd143-ca27-4867-9ad0-c156c199a6b7 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:51.364118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6dbfd143-ca27-4867-9ad0-c156c199a6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:51.385596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbfd143-ca27-4867-9ad0-c156c199a6b7 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:51.391035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd8dc4-d69a-486b-818a-600f1c13b53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:51.391903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd8dc4-d69a-486b-818a-600f1c13b53a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:51.392336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd8dc4-d69a-486b-818a-600f1c13b53a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:51.392795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd8dc4-d69a-486b-818a-600f1c13b53a admin admin] PATH_INFO: `/v3/projects/bf470fc5619243f0a15b79ad010fafcc` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:51.394127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd8dc4-d69a-486b-818a-600f1c13b53a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf470fc5619243f0a15b79ad010fafcc)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:51.396206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd8dc4-d69a-486b-818a-600f1c13b53a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:51.472379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-73dd8dc4-d69a-486b-818a-600f1c13b53a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf470fc5619243f0a15b79ad010fafcc', 'request_id': 'req-73dd8dc4-d69a-486b-818a-600f1c13b53a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:51.480817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-73dd8dc4-d69a-486b-818a-600f1c13b53a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf470fc5619243f0a15b79ad010fafcc', 'request_id': 'req-73dd8dc4-d69a-486b-818a-600f1c13b53a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:51.493177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-73dd8dc4-d69a-486b-818a-600f1c13b53a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf470fc5619243f0a15b79ad010fafcc', 'request_id': 'req-73dd8dc4-d69a-486b-818a-600f1c13b53a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:51.510843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-73dd8dc4-d69a-486b-818a-600f1c13b53a admin admin] The token cache is being invalidate because project bf470fc5619243f0a15b79ad010fafcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:00:51.511137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-73dd8dc4-d69a-486b-818a-600f1c13b53a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:51.511772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-73dd8dc4-d69a-486b-818a-600f1c13b53a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:51.513543 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2029/3987] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:00:51 2022] DELETE /identity/v3/projects/bf470fc5619243f0a15b79ad010fafcc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:00:51.517793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40fbb1-e49b-42c7-a4e0-e89113fc9bc2 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:51.528992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40fbb1-e49b-42c7-a4e0-e89113fc9bc2 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:51.549918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:51.608957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1428d51-ccaa-495a-8ef8-05b14a23f4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:51.609948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1428d51-ccaa-495a-8ef8-05b14a23f4b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:51.610242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1428d51-ccaa-495a-8ef8-05b14a23f4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:51.610543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1428d51-ccaa-495a-8ef8-05b14a23f4b3 admin admin] PATH_INFO: `/v3/users/a67229cfd2524cd9a438f72acbf476aa` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:51.612149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1428d51-ccaa-495a-8ef8-05b14a23f4b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a67229cfd2524cd9a438f72acbf476aa)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:51.616201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1428d51-ccaa-495a-8ef8-05b14a23f4b3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:51.670328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a1428d51-ccaa-495a-8ef8-05b14a23f4b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a67229cfd2524cd9a438f72acbf476aa'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:51.671149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a1428d51-ccaa-495a-8ef8-05b14a23f4b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a67229cfd2524cd9a438f72acbf476aa'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:51.671760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a1428d51-ccaa-495a-8ef8-05b14a23f4b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a67229cfd2524cd9a438f72acbf476aa'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:51.681589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a1428d51-ccaa-495a-8ef8-05b14a23f4b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a67229cfd2524cd9a438f72acbf476aa'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:51.694838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a1428d51-ccaa-495a-8ef8-05b14a23f4b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a67229cfd2524cd9a438f72acbf476aa'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:51.707084 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1959/3988] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:00:51 2022] DELETE /identity/v3/users/a67229cfd2524cd9a438f72acbf476aa => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:00:52.035962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd8dc4-d69a-486b-818a-600f1c13b53a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:52.037619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-73dd8dc4-d69a-486b-818a-600f1c13b53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:52.063304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd8dc4-d69a-486b-818a-600f1c13b53a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:52.150645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed94284-9a15-4894-af24-79a376751625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:52.152106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed94284-9a15-4894-af24-79a376751625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:52.152455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed94284-9a15-4894-af24-79a376751625 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:52.152828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed94284-9a15-4894-af24-79a376751625 admin admin] PATH_INFO: `/v3/projects/ab00d1a09a824f9aa55939cc08cc524c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:52.154185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed94284-9a15-4894-af24-79a376751625 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab00d1a09a824f9aa55939cc08cc524c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:52.156725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed94284-9a15-4894-af24-79a376751625 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:52.158106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:52.237135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ed94284-9a15-4894-af24-79a376751625 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab00d1a09a824f9aa55939cc08cc524c', 'request_id': 'req-2ed94284-9a15-4894-af24-79a376751625'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:52.247209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ed94284-9a15-4894-af24-79a376751625 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab00d1a09a824f9aa55939cc08cc524c', 'request_id': 'req-2ed94284-9a15-4894-af24-79a376751625'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:52.254463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ed94284-9a15-4894-af24-79a376751625 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab00d1a09a824f9aa55939cc08cc524c', 'request_id': 'req-2ed94284-9a15-4894-af24-79a376751625'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:52.270775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ed94284-9a15-4894-af24-79a376751625 admin admin] The token cache is being invalidate because project ab00d1a09a824f9aa55939cc08cc524c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:00:52.271317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ed94284-9a15-4894-af24-79a376751625 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:52.272119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ed94284-9a15-4894-af24-79a376751625 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:00:52.277695 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2030/3989] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:00:52 2022] DELETE /identity/v3/projects/ab00d1a09a824f9aa55939cc08cc524c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:00:52.279378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbd477-9862-4da4-bf3b-9bb38b1d3b26 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:52.279736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbd477-9862-4da4-bf3b-9bb38b1d3b26 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:52.279952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbd477-9862-4da4-bf3b-9bb38b1d3b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:52.321229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4fcbd477-9862-4da4-bf3b-9bb38b1d3b26 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:52.322875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4fcbd477-9862-4da4-bf3b-9bb38b1d3b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:52.398049 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1960/3990] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:00:52.407754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36078cf0-8284-4b05-af4b-7c592c0055a3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:52.408147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36078cf0-8284-4b05-af4b-7c592c0055a3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:52.408532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36078cf0-8284-4b05-af4b-7c592c0055a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:52.438137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-36078cf0-8284-4b05-af4b-7c592c0055a3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:52.440116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-36078cf0-8284-4b05-af4b-7c592c0055a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:52.452925 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2031/3991] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:52.456800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbd477-9862-4da4-bf3b-9bb38b1d3b26 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:52.468322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbd477-9862-4da4-bf3b-9bb38b1d3b26 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:52.474236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5e74e-81b6-475c-9da9-5a4d5589a130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:52.474969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5e74e-81b6-475c-9da9-5a4d5589a130 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:52.475857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5e74e-81b6-475c-9da9-5a4d5589a130 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:52.476143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5e74e-81b6-475c-9da9-5a4d5589a130 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:52.476794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5e74e-81b6-475c-9da9-5a4d5589a130 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:00:52.477198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5e74e-81b6-475c-9da9-5a4d5589a130 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:52.479530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5e74e-81b6-475c-9da9-5a4d5589a130 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:52.499618 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1961/3992] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:00:52.500301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36078cf0-8284-4b05-af4b-7c592c0055a3 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:52.511658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36078cf0-8284-4b05-af4b-7c592c0055a3 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:52.517583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017d05f-504c-47bd-8a4b-1f519f93d7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:52.518418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017d05f-504c-47bd-8a4b-1f519f93d7b2 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:52.518895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017d05f-504c-47bd-8a4b-1f519f93d7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:52.519334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017d05f-504c-47bd-8a4b-1f519f93d7b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:52.520749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f017d05f-504c-47bd-8a4b-1f519f93d7b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:52.524122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f017d05f-504c-47bd-8a4b-1f519f93d7b2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:52.590104 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2032/3993] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:00:52 2022] POST /identity/v3/projects => generated 407 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:00:52.592868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5e74e-81b6-475c-9da9-5a4d5589a130 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:52.618563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5e74e-81b6-475c-9da9-5a4d5589a130 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:52.671634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:52.741625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7a80c-56de-4982-9f0f-1f87a3798152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:52.742623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7a80c-56de-4982-9f0f-1f87a3798152 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:52.743016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7a80c-56de-4982-9f0f-1f87a3798152 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:52.743400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7a80c-56de-4982-9f0f-1f87a3798152 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:52.744151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7a80c-56de-4982-9f0f-1f87a3798152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:52.746283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7a80c-56de-4982-9f0f-1f87a3798152 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:52.771803 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1962/3994] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:00:52 2022] POST /identity/v3/users => generated 304 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:00:52.782353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017d05f-504c-47bd-8a4b-1f519f93d7b2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:52.805319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017d05f-504c-47bd-8a4b-1f519f93d7b2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:52.810566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c4500-6250-4918-af81-c284d59bf515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:52.811204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c4500-6250-4918-af81-c284d59bf515 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:52.811484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c4500-6250-4918-af81-c284d59bf515 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:52.811941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c4500-6250-4918-af81-c284d59bf515 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:52.812767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c4500-6250-4918-af81-c284d59bf515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:52.814552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c4500-6250-4918-af81-c284d59bf515 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:52.829350 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2033/3995] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:00:52 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:00:52.833189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7a80c-56de-4982-9f0f-1f87a3798152 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:52.847461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7a80c-56de-4982-9f0f-1f87a3798152 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:52.852466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c6f7b-b242-455a-9432-2af2b430b9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:52.853166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c6f7b-b242-455a-9432-2af2b430b9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:52.853545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c6f7b-b242-455a-9432-2af2b430b9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:52.853892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c6f7b-b242-455a-9432-2af2b430b9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:52.855176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c6f7b-b242-455a-9432-2af2b430b9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:52.857073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c6f7b-b242-455a-9432-2af2b430b9b8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:52.870128 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1963/3996] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:00:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:00:52.874904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c4500-6250-4918-af81-c284d59bf515 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:52.885650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c4500-6250-4918-af81-c284d59bf515 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:52.891653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73f258-6469-45f6-b685-127175a2a713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:52.892526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73f258-6469-45f6-b685-127175a2a713 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:52.892815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73f258-6469-45f6-b685-127175a2a713 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:52.893096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73f258-6469-45f6-b685-127175a2a713 admin admin] PATH_INFO: `/v3/projects/944af14831394cc988236387e13e9f7a/users/7da28ff572834606911bf747228f55b5/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:52.915438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73f258-6469-45f6-b685-127175a2a713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=944af14831394cc988236387e13e9f7a, user_id=7da28ff572834606911bf747228f55b5, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:52.922662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73f258-6469-45f6-b685-127175a2a713 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:52.944423 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2034/3997] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:00:52 2022] PUT /identity/v3/projects/944af14831394cc988236387e13e9f7a/users/7da28ff572834606911bf747228f55b5/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:00:52.954465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c879b-3678-47eb-ae1d-98b6342e2621 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:52.954789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c879b-3678-47eb-ae1d-98b6342e2621 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:52.955104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c879b-3678-47eb-ae1d-98b6342e2621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:52.986317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0b4c879b-3678-47eb-ae1d-98b6342e2621 None None] MFA Rules not processed for user `7da28ff572834606911bf747228f55b5`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:52.987567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0b4c879b-3678-47eb-ae1d-98b6342e2621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:53.077205 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1964/3998] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:52 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:53.078560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73f258-6469-45f6-b685-127175a2a713 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:53.168824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73f258-6469-45f6-b685-127175a2a713 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:53.175378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb80ae-7bcc-4043-aef2-11661c7abd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:53.176280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb80ae-7bcc-4043-aef2-11661c7abd7b admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:53.176432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb80ae-7bcc-4043-aef2-11661c7abd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:53.176656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb80ae-7bcc-4043-aef2-11661c7abd7b admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:53.177303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb80ae-7bcc-4043-aef2-11661c7abd7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:53.179383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb80ae-7bcc-4043-aef2-11661c7abd7b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:53.220586 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2035/3999] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:00:53 2022] POST /identity/v3/projects => generated 407 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:00:53.221330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c879b-3678-47eb-ae1d-98b6342e2621 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:53.235759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c879b-3678-47eb-ae1d-98b6342e2621 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:53.324623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40779225-7729-4556-8688-cd7db88c8870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:53.325429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40779225-7729-4556-8688-cd7db88c8870 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:53.325802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40779225-7729-4556-8688-cd7db88c8870 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:53.326722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40779225-7729-4556-8688-cd7db88c8870 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:53.327596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40779225-7729-4556-8688-cd7db88c8870 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:53.329660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40779225-7729-4556-8688-cd7db88c8870 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:53.360648 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1965/4000] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:00:53 2022] POST /identity/v3/users => generated 310 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:00:53.365668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb80ae-7bcc-4043-aef2-11661c7abd7b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:53.377343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb80ae-7bcc-4043-aef2-11661c7abd7b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:53.383282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab3252-2267-4d9e-8671-b6e339b4ab48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:53.384066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab3252-2267-4d9e-8671-b6e339b4ab48 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:53.384417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab3252-2267-4d9e-8671-b6e339b4ab48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:53.384760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab3252-2267-4d9e-8671-b6e339b4ab48 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:53.385582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab3252-2267-4d9e-8671-b6e339b4ab48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:53.387436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab3252-2267-4d9e-8671-b6e339b4ab48 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:53.394062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:53.416093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40779225-7729-4556-8688-cd7db88c8870 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:53.416203 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2036/4001] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:00:53.419155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40779225-7729-4556-8688-cd7db88c8870 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:53.427647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679f412-1db7-491e-b15c-29aee46aa9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:53.437975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679f412-1db7-491e-b15c-29aee46aa9ef admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:53.438613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679f412-1db7-491e-b15c-29aee46aa9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:53.439087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679f412-1db7-491e-b15c-29aee46aa9ef admin admin] PATH_INFO: `/v3/projects/ec2b810463fc4e548240c81e353ae9fd/users/f9fce09a093d40b9ace10301ba512023/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:53.458464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679f412-1db7-491e-b15c-29aee46aa9ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2b810463fc4e548240c81e353ae9fd, user_id=f9fce09a093d40b9ace10301ba512023, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:53.461388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679f412-1db7-491e-b15c-29aee46aa9ef admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:53.473682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:53.484143 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1966/4002] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:00:53 2022] PUT /identity/v3/projects/ec2b810463fc4e548240c81e353ae9fd/users/f9fce09a093d40b9ace10301ba512023/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:00:53.488504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab3252-2267-4d9e-8671-b6e339b4ab48 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:53.498822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab3252-2267-4d9e-8671-b6e339b4ab48 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:53.571791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7c1014-b92d-40c3-a600-2de383fa4122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:53.572646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c1014-b92d-40c3-a600-2de383fa4122 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:53.573052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c1014-b92d-40c3-a600-2de383fa4122 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:53.573990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c1014-b92d-40c3-a600-2de383fa4122 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:53.574811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7c1014-b92d-40c3-a600-2de383fa4122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:53.576553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7c1014-b92d-40c3-a600-2de383fa4122 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:53.602991 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2037/4003] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:00:53.604529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679f412-1db7-491e-b15c-29aee46aa9ef admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:53.624736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679f412-1db7-491e-b15c-29aee46aa9ef admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:53.635016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c767b9d-7409-4b4d-98f0-4742c5983bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:53.636320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c767b9d-7409-4b4d-98f0-4742c5983bab admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:53.636679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c767b9d-7409-4b4d-98f0-4742c5983bab admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:53.637021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c767b9d-7409-4b4d-98f0-4742c5983bab admin admin] PATH_INFO: `/v3/projects/ec2b810463fc4e548240c81e353ae9fd/users/f9fce09a093d40b9ace10301ba512023/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:53.640145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c767b9d-7409-4b4d-98f0-4742c5983bab admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2b810463fc4e548240c81e353ae9fd, user_id=f9fce09a093d40b9ace10301ba512023, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:53.643156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c767b9d-7409-4b4d-98f0-4742c5983bab admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:53.665017 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1967/4004] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:00:53 2022] PUT /identity/v3/projects/ec2b810463fc4e548240c81e353ae9fd/users/f9fce09a093d40b9ace10301ba512023/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:00:53.676937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad472a-0e00-4377-8dc5-2feaec53c9be None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:53.678337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad472a-0e00-4377-8dc5-2feaec53c9be None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:53.678646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad472a-0e00-4377-8dc5-2feaec53c9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:53.710966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-63ad472a-0e00-4377-8dc5-2feaec53c9be None None] MFA Rules not processed for user `f9fce09a093d40b9ace10301ba512023`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:00:53.712591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-63ad472a-0e00-4377-8dc5-2feaec53c9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:53.800109 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2038/4005] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:00:53 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:00:53.813670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c767b9d-7409-4b4d-98f0-4742c5983bab admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:53.816129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6c767b9d-7409-4b4d-98f0-4742c5983bab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:53.884354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c767b9d-7409-4b4d-98f0-4742c5983bab admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:53.889014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ba7d9-a867-4a74-bb64-5faf706d58e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:53.889630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ba7d9-a867-4a74-bb64-5faf706d58e1 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:53.889913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ba7d9-a867-4a74-bb64-5faf706d58e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:53.890191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ba7d9-a867-4a74-bb64-5faf706d58e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:53.953364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ba7d9-a867-4a74-bb64-5faf706d58e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:53.954849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ba7d9-a867-4a74-bb64-5faf706d58e1 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:53.965012 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1968/4006] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:00:53 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:00:55.226375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad472a-0e00-4377-8dc5-2feaec53c9be None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:00:55.228914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-63ad472a-0e00-4377-8dc5-2feaec53c9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:00:55.353842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad472a-0e00-4377-8dc5-2feaec53c9be None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:00:55.363627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e010e-9fb8-48f0-bae4-79df7a013a2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:00:55.367218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e010e-9fb8-48f0-bae4-79df7a013a2e service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:00:55.367218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e010e-9fb8-48f0-bae4-79df7a013a2e service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:00:55.367421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e010e-9fb8-48f0-bae4-79df7a013a2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:00:55.397394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e010e-9fb8-48f0-bae4-79df7a013a2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:00:55.399844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e010e-9fb8-48f0-bae4-79df7a013a2e service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:00:55.431430 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2039/4007] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:00:55 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:01:01.006782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f77a9-0870-476f-8ad9-ed44c3100fc6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:01.007273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f77a9-0870-476f-8ad9-ed44c3100fc6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:01.007579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f77a9-0870-476f-8ad9-ed44c3100fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:01.082726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-017f77a9-0870-476f-8ad9-ed44c3100fc6 None None] MFA Rules not processed for user `775ff9091a794faaacc05a37550c8a72`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:01.085789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-017f77a9-0870-476f-8ad9-ed44c3100fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:01.195107 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1969/4008] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:02.726980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e010e-9fb8-48f0-bae4-79df7a013a2e service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:02.729045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b90e010e-9fb8-48f0-bae4-79df7a013a2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:02.742140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:02.880409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e010e-9fb8-48f0-bae4-79df7a013a2e service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:02.888069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e24bb-0688-41fd-905b-c5fe226d7838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:02.889315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e24bb-0688-41fd-905b-c5fe226d7838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:02.889838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e24bb-0688-41fd-905b-c5fe226d7838 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:02.890340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e24bb-0688-41fd-905b-c5fe226d7838 admin admin] PATH_INFO: `/v3/users/ccbf2c814364466baca8dfcabd2c13f9` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:02.892216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e24bb-0688-41fd-905b-c5fe226d7838 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccbf2c814364466baca8dfcabd2c13f9)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:02.896019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e24bb-0688-41fd-905b-c5fe226d7838 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:03.000195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-793e24bb-0688-41fd-905b-c5fe226d7838 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccbf2c814364466baca8dfcabd2c13f9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:03.016512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-793e24bb-0688-41fd-905b-c5fe226d7838 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccbf2c814364466baca8dfcabd2c13f9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:03.054035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-793e24bb-0688-41fd-905b-c5fe226d7838 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccbf2c814364466baca8dfcabd2c13f9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:03.054912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-793e24bb-0688-41fd-905b-c5fe226d7838 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccbf2c814364466baca8dfcabd2c13f9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:03.068596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-793e24bb-0688-41fd-905b-c5fe226d7838 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccbf2c814364466baca8dfcabd2c13f9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:03.083182 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2040/4009] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:01:02 2022] DELETE /identity/v3/users/ccbf2c814364466baca8dfcabd2c13f9 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:04.091102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f77a9-0870-476f-8ad9-ed44c3100fc6 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:04.109195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f77a9-0870-476f-8ad9-ed44c3100fc6 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:04.190229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:04.226092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2cf4c-5b9d-4d62-9616-24348fd76edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:04.227278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2cf4c-5b9d-4d62-9616-24348fd76edd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:04.227783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2cf4c-5b9d-4d62-9616-24348fd76edd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:04.228325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2cf4c-5b9d-4d62-9616-24348fd76edd admin admin] PATH_INFO: `/v3/projects/bcdb77df347145639f2de60eb820a3ca` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:04.230683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2cf4c-5b9d-4d62-9616-24348fd76edd admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcdb77df347145639f2de60eb820a3ca)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:04.234298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2cf4c-5b9d-4d62-9616-24348fd76edd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:04.325132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2eb2cf4c-5b9d-4d62-9616-24348fd76edd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcdb77df347145639f2de60eb820a3ca', 'request_id': 'req-2eb2cf4c-5b9d-4d62-9616-24348fd76edd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:04.344823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2eb2cf4c-5b9d-4d62-9616-24348fd76edd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcdb77df347145639f2de60eb820a3ca', 'request_id': 'req-2eb2cf4c-5b9d-4d62-9616-24348fd76edd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:04.355885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2eb2cf4c-5b9d-4d62-9616-24348fd76edd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcdb77df347145639f2de60eb820a3ca', 'request_id': 'req-2eb2cf4c-5b9d-4d62-9616-24348fd76edd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:04.381882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2eb2cf4c-5b9d-4d62-9616-24348fd76edd admin admin] The token cache is being invalidate because project bcdb77df347145639f2de60eb820a3ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:01:04.382181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2eb2cf4c-5b9d-4d62-9616-24348fd76edd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:04.385699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2eb2cf4c-5b9d-4d62-9616-24348fd76edd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:04.391894 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1970/4010] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:01:04 2022] DELETE /identity/v3/projects/bcdb77df347145639f2de60eb820a3ca => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:04.395779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e24bb-0688-41fd-905b-c5fe226d7838 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:04.397822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-793e24bb-0688-41fd-905b-c5fe226d7838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:04.421550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e24bb-0688-41fd-905b-c5fe226d7838 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:04.587720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055a491-fd45-429c-9b59-d78b2af51f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:04.589602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055a491-fd45-429c-9b59-d78b2af51f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:04.591369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055a491-fd45-429c-9b59-d78b2af51f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:04.591818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055a491-fd45-429c-9b59-d78b2af51f67 admin admin] PATH_INFO: `/v3/users/0284a8ec6c4e41f0a1a88e2f11a4faa8` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:04.595669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055a491-fd45-429c-9b59-d78b2af51f67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0284a8ec6c4e41f0a1a88e2f11a4faa8)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:04.598642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055a491-fd45-429c-9b59-d78b2af51f67 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:04.663767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:04.695241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f055a491-fd45-429c-9b59-d78b2af51f67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0284a8ec6c4e41f0a1a88e2f11a4faa8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:04.705179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f055a491-fd45-429c-9b59-d78b2af51f67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0284a8ec6c4e41f0a1a88e2f11a4faa8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:04.720991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f055a491-fd45-429c-9b59-d78b2af51f67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0284a8ec6c4e41f0a1a88e2f11a4faa8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:04.721985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f055a491-fd45-429c-9b59-d78b2af51f67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0284a8ec6c4e41f0a1a88e2f11a4faa8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:04.746724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f055a491-fd45-429c-9b59-d78b2af51f67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0284a8ec6c4e41f0a1a88e2f11a4faa8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:04.756066 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2041/4011] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:01:04 2022] DELETE /identity/v3/users/0284a8ec6c4e41f0a1a88e2f11a4faa8 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:05.713372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2cf4c-5b9d-4d62-9616-24348fd76edd admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:05.715517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2eb2cf4c-5b9d-4d62-9616-24348fd76edd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:05.750230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2cf4c-5b9d-4d62-9616-24348fd76edd admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:05.855916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8216c7-aac4-43e1-ad64-57b8195e2b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:05.857604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8216c7-aac4-43e1-ad64-57b8195e2b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:05.858112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8216c7-aac4-43e1-ad64-57b8195e2b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:05.859090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8216c7-aac4-43e1-ad64-57b8195e2b89 admin admin] PATH_INFO: `/v3/projects/24a9d88d60164b2785bacc9f8bfa3c05` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:05.861009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8216c7-aac4-43e1-ad64-57b8195e2b89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24a9d88d60164b2785bacc9f8bfa3c05)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:05.863342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8216c7-aac4-43e1-ad64-57b8195e2b89 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:05.938341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bd8216c7-aac4-43e1-ad64-57b8195e2b89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24a9d88d60164b2785bacc9f8bfa3c05', 'request_id': 'req-bd8216c7-aac4-43e1-ad64-57b8195e2b89'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:05.948282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bd8216c7-aac4-43e1-ad64-57b8195e2b89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24a9d88d60164b2785bacc9f8bfa3c05', 'request_id': 'req-bd8216c7-aac4-43e1-ad64-57b8195e2b89'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:05.957375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bd8216c7-aac4-43e1-ad64-57b8195e2b89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24a9d88d60164b2785bacc9f8bfa3c05', 'request_id': 'req-bd8216c7-aac4-43e1-ad64-57b8195e2b89'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:05.974446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bd8216c7-aac4-43e1-ad64-57b8195e2b89 admin admin] The token cache is being invalidate because project 24a9d88d60164b2785bacc9f8bfa3c05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:01:05.975109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bd8216c7-aac4-43e1-ad64-57b8195e2b89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:05.976319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bd8216c7-aac4-43e1-ad64-57b8195e2b89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:05.978805 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1971/4012] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:01:05 2022] DELETE /identity/v3/projects/24a9d88d60164b2785bacc9f8bfa3c05 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:05.983757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ec27f-f86b-413f-b973-7c082072d207 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:05.984300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ec27f-f86b-413f-b973-7c082072d207 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:05.984501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ec27f-f86b-413f-b973-7c082072d207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:06.056529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fb4ec27f-f86b-413f-b973-7c082072d207 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:06.058671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fb4ec27f-f86b-413f-b973-7c082072d207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:06.184888 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2042/4013] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:06.193863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4247f5-4c79-41b4-95a0-d2e01c9cfe10 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:06.194232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4247f5-4c79-41b4-95a0-d2e01c9cfe10 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:06.194423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4247f5-4c79-41b4-95a0-d2e01c9cfe10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:06.211645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:06.255991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-dc4247f5-4c79-41b4-95a0-d2e01c9cfe10 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:06.257256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-dc4247f5-4c79-41b4-95a0-d2e01c9cfe10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:06.281328 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1972/4014] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:06.285398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ec27f-f86b-413f-b973-7c082072d207 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:06.299986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ec27f-f86b-413f-b973-7c082072d207 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:06.311128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01004162-a164-469d-b285-6e0db5813e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:06.311812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01004162-a164-469d-b285-6e0db5813e44 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:06.312092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01004162-a164-469d-b285-6e0db5813e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:06.312368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01004162-a164-469d-b285-6e0db5813e44 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:06.313039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01004162-a164-469d-b285-6e0db5813e44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:01:06.313394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01004162-a164-469d-b285-6e0db5813e44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:06.315280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01004162-a164-469d-b285-6e0db5813e44 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:06.334089 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2043/4015] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:01:06.335406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4247f5-4c79-41b4-95a0-d2e01c9cfe10 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:06.350332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4247f5-4c79-41b4-95a0-d2e01c9cfe10 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:06.357171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764237f-6b0d-4ff5-90c0-27f03bd836cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:06.357898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764237f-6b0d-4ff5-90c0-27f03bd836cf admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:06.358188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764237f-6b0d-4ff5-90c0-27f03bd836cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:06.358547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764237f-6b0d-4ff5-90c0-27f03bd836cf admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:06.359204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764237f-6b0d-4ff5-90c0-27f03bd836cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:06.361320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764237f-6b0d-4ff5-90c0-27f03bd836cf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:06.400296 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1973/4016] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:01:06 2022] POST /identity/v3/projects => generated 391 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:01:06.405120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01004162-a164-469d-b285-6e0db5813e44 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:06.424897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01004162-a164-469d-b285-6e0db5813e44 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:06.501270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0b871-62fe-42c5-8eaf-433bd12a0d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:06.502829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0b871-62fe-42c5-8eaf-433bd12a0d3d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:06.503329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0b871-62fe-42c5-8eaf-433bd12a0d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:06.503920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0b871-62fe-42c5-8eaf-433bd12a0d3d admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:06.507327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0b871-62fe-42c5-8eaf-433bd12a0d3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:06.509369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0b871-62fe-42c5-8eaf-433bd12a0d3d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:06.548803 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2044/4017] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:01:06 2022] POST /identity/v3/users => generated 296 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:01:06.553077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764237f-6b0d-4ff5-90c0-27f03bd836cf admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:06.564035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764237f-6b0d-4ff5-90c0-27f03bd836cf admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:06.570179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f43811-189f-4d2f-8824-275ae209a435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:06.570831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f43811-189f-4d2f-8824-275ae209a435 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:06.571117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f43811-189f-4d2f-8824-275ae209a435 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:06.571566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f43811-189f-4d2f-8824-275ae209a435 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:06.574927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f43811-189f-4d2f-8824-275ae209a435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:06.574927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f43811-189f-4d2f-8824-275ae209a435 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:06.588015 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1974/4018] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:01:06.592647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0b871-62fe-42c5-8eaf-433bd12a0d3d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:06.605323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0b871-62fe-42c5-8eaf-433bd12a0d3d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:06.610527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231eebde-5dbc-4e48-857f-be372b13a040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:06.612712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231eebde-5dbc-4e48-857f-be372b13a040 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:06.612712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231eebde-5dbc-4e48-857f-be372b13a040 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:06.612712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231eebde-5dbc-4e48-857f-be372b13a040 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:06.612712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231eebde-5dbc-4e48-857f-be372b13a040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:06.614752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231eebde-5dbc-4e48-857f-be372b13a040 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:06.627480 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2045/4019] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:01:06.631567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f43811-189f-4d2f-8824-275ae209a435 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:06.649585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f43811-189f-4d2f-8824-275ae209a435 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:06.655871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbd4e2-3d51-4f91-87ac-ca591385f4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:06.656751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbd4e2-3d51-4f91-87ac-ca591385f4e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:06.657034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbd4e2-3d51-4f91-87ac-ca591385f4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:06.657310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbd4e2-3d51-4f91-87ac-ca591385f4e3 admin admin] PATH_INFO: `/v3/projects/497873f728f34c20aaa348bec348ed4f/users/0bf5b69507cd4ca884e355454f49d5e1/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:06.681606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bbd4e2-3d51-4f91-87ac-ca591385f4e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=497873f728f34c20aaa348bec348ed4f, user_id=0bf5b69507cd4ca884e355454f49d5e1, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:06.685625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bbd4e2-3d51-4f91-87ac-ca591385f4e3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:06.707072 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1975/4020] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:01:06 2022] PUT /identity/v3/projects/497873f728f34c20aaa348bec348ed4f/users/0bf5b69507cd4ca884e355454f49d5e1/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:01:06.717708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e06ed-a0cf-432e-8c18-94340a5f27aa None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:06.718053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e06ed-a0cf-432e-8c18-94340a5f27aa None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:06.718272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e06ed-a0cf-432e-8c18-94340a5f27aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:06.747611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:06.758122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-595e06ed-a0cf-432e-8c18-94340a5f27aa None None] MFA Rules not processed for user `0bf5b69507cd4ca884e355454f49d5e1`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:06.759765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-595e06ed-a0cf-432e-8c18-94340a5f27aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:06.879513 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2046/4021] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:07.003704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbd4e2-3d51-4f91-87ac-ca591385f4e3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:07.006139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-10bbd4e2-3d51-4f91-87ac-ca591385f4e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:07.132763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbd4e2-3d51-4f91-87ac-ca591385f4e3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:07.142665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3602c1-a027-4220-827b-f443cbdc5cf9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:07.144077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3602c1-a027-4220-827b-f443cbdc5cf9 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:07.144077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3602c1-a027-4220-827b-f443cbdc5cf9 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:07.144566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3602c1-a027-4220-827b-f443cbdc5cf9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:07.156249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3602c1-a027-4220-827b-f443cbdc5cf9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:07.158385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3602c1-a027-4220-827b-f443cbdc5cf9 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:07.179533 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1976/4022] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:01:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:01:09.002028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e06ed-a0cf-432e-8c18-94340a5f27aa None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:09.004033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-595e06ed-a0cf-432e-8c18-94340a5f27aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:09.202973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e06ed-a0cf-432e-8c18-94340a5f27aa None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:09.224757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92028b4-86aa-4591-82a0-5e7547670538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:09.226729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92028b4-86aa-4591-82a0-5e7547670538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:09.227807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92028b4-86aa-4591-82a0-5e7547670538 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:09.229748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92028b4-86aa-4591-82a0-5e7547670538 admin admin] PATH_INFO: `/v3/users/7cd23af965b147248c5f8c7b9098fc3d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:09.231183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92028b4-86aa-4591-82a0-5e7547670538 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cd23af965b147248c5f8c7b9098fc3d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:09.234141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92028b4-86aa-4591-82a0-5e7547670538 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:09.347741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d92028b4-86aa-4591-82a0-5e7547670538 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cd23af965b147248c5f8c7b9098fc3d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:09.359542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d92028b4-86aa-4591-82a0-5e7547670538 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cd23af965b147248c5f8c7b9098fc3d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:09.382009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d92028b4-86aa-4591-82a0-5e7547670538 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cd23af965b147248c5f8c7b9098fc3d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:09.384338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d92028b4-86aa-4591-82a0-5e7547670538 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cd23af965b147248c5f8c7b9098fc3d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:09.395448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d92028b4-86aa-4591-82a0-5e7547670538 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cd23af965b147248c5f8c7b9098fc3d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:09.408815 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2047/4023] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:01:09 2022] DELETE /identity/v3/users/7cd23af965b147248c5f8c7b9098fc3d => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:09.863863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3602c1-a027-4220-827b-f443cbdc5cf9 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:09.871365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92028b4-86aa-4591-82a0-5e7547670538 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:09.873859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d92028b4-86aa-4591-82a0-5e7547670538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:09.877958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3602c1-a027-4220-827b-f443cbdc5cf9 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:09.923609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:10.012349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f47b6-b8ed-43e5-855e-388c881aca4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:10.012349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f47b6-b8ed-43e5-855e-388c881aca4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:10.012349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f47b6-b8ed-43e5-855e-388c881aca4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:10.012349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f47b6-b8ed-43e5-855e-388c881aca4d admin admin] PATH_INFO: `/v3/users/0bf5b69507cd4ca884e355454f49d5e1` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:10.014323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92028b4-86aa-4591-82a0-5e7547670538 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:10.019158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f47b6-b8ed-43e5-855e-388c881aca4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bf5b69507cd4ca884e355454f49d5e1)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:10.021824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1783f2-dd70-4af2-96f0-9c264c37b2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:10.023603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f47b6-b8ed-43e5-855e-388c881aca4d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:10.024047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1783f2-dd70-4af2-96f0-9c264c37b2b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:10.024339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1783f2-dd70-4af2-96f0-9c264c37b2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:10.024621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1783f2-dd70-4af2-96f0-9c264c37b2b1 admin admin] PATH_INFO: `/v3/projects/b9e916618c504ec2b23064c883ba7d56` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:10.033128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1783f2-dd70-4af2-96f0-9c264c37b2b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9e916618c504ec2b23064c883ba7d56)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:10.039360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1783f2-dd70-4af2-96f0-9c264c37b2b1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:10.109928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:10.128515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e1783f2-dd70-4af2-96f0-9c264c37b2b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9e916618c504ec2b23064c883ba7d56', 'request_id': 'req-9e1783f2-dd70-4af2-96f0-9c264c37b2b1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:10.135563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e1783f2-dd70-4af2-96f0-9c264c37b2b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9e916618c504ec2b23064c883ba7d56', 'request_id': 'req-9e1783f2-dd70-4af2-96f0-9c264c37b2b1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:10.138671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2a4f47b6-b8ed-43e5-855e-388c881aca4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bf5b69507cd4ca884e355454f49d5e1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:10.139362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2a4f47b6-b8ed-43e5-855e-388c881aca4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bf5b69507cd4ca884e355454f49d5e1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:10.139990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2a4f47b6-b8ed-43e5-855e-388c881aca4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bf5b69507cd4ca884e355454f49d5e1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:10.143278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e1783f2-dd70-4af2-96f0-9c264c37b2b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9e916618c504ec2b23064c883ba7d56', 'request_id': 'req-9e1783f2-dd70-4af2-96f0-9c264c37b2b1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:10.147939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2a4f47b6-b8ed-43e5-855e-388c881aca4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bf5b69507cd4ca884e355454f49d5e1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:10.157444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e1783f2-dd70-4af2-96f0-9c264c37b2b1 admin admin] The token cache is being invalidate because project b9e916618c504ec2b23064c883ba7d56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:01:10.158340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e1783f2-dd70-4af2-96f0-9c264c37b2b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:10.158969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e1783f2-dd70-4af2-96f0-9c264c37b2b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:10.160840 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2048/4024] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:01:09 2022] DELETE /identity/v3/projects/b9e916618c504ec2b23064c883ba7d56 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:10.164913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2a4f47b6-b8ed-43e5-855e-388c881aca4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bf5b69507cd4ca884e355454f49d5e1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:10.168142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b336da-c9a9-4677-a0dd-4cd8379677c1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:10.168415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b336da-c9a9-4677-a0dd-4cd8379677c1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:10.168688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b336da-c9a9-4677-a0dd-4cd8379677c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:10.181721 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1977/4025] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:01:09 2022] DELETE /identity/v3/users/0bf5b69507cd4ca884e355454f49d5e1 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:10.186800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2f9b4-6a2d-435c-8660-d5c1e3aa7011 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:10.187354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2f9b4-6a2d-435c-8660-d5c1e3aa7011 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:10.188125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2f9b4-6a2d-435c-8660-d5c1e3aa7011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:10.192570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d6b336da-c9a9-4677-a0dd-4cd8379677c1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:10.194183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d6b336da-c9a9-4677-a0dd-4cd8379677c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:10.209260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-15e2f9b4-6a2d-435c-8660-d5c1e3aa7011 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:10.210435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-15e2f9b4-6a2d-435c-8660-d5c1e3aa7011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:10.266919 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2049/4026] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:01:10.275290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d88b27-2ac8-4113-96c4-1622a45fde7d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:10.275681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d88b27-2ac8-4113-96c4-1622a45fde7d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:10.275917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d88b27-2ac8-4113-96c4-1622a45fde7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:10.283748 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1978/4027] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:01:10.290815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2f9b4-6a2d-435c-8660-d5c1e3aa7011 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:10.293070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-15e2f9b4-6a2d-435c-8660-d5c1e3aa7011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:10.301437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-03d88b27-2ac8-4113-96c4-1622a45fde7d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:10.302918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-03d88b27-2ac8-4113-96c4-1622a45fde7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:10.312158 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2050/4028] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:10.316390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d88b27-2ac8-4113-96c4-1622a45fde7d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:10.326265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d88b27-2ac8-4113-96c4-1622a45fde7d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:10.331077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb5802-aa7d-410d-a9bb-5c199dacdc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:10.331702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb5802-aa7d-410d-a9bb-5c199dacdc9a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:10.331992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb5802-aa7d-410d-a9bb-5c199dacdc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:10.332270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb5802-aa7d-410d-a9bb-5c199dacdc9a admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:10.332936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb5802-aa7d-410d-a9bb-5c199dacdc9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:01:10.333373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb5802-aa7d-410d-a9bb-5c199dacdc9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:10.335107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb5802-aa7d-410d-a9bb-5c199dacdc9a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:10.349538 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2051/4029] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:01:10.354733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb5802-aa7d-410d-a9bb-5c199dacdc9a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:10.365815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb5802-aa7d-410d-a9bb-5c199dacdc9a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:10.371763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda77a5-34f6-4a39-bf81-1e135bc478be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:10.372476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda77a5-34f6-4a39-bf81-1e135bc478be admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:10.372808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2f9b4-6a2d-435c-8660-d5c1e3aa7011 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:10.372920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda77a5-34f6-4a39-bf81-1e135bc478be admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:10.373193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda77a5-34f6-4a39-bf81-1e135bc478be admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:10.373905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda77a5-34f6-4a39-bf81-1e135bc478be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:10.375938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda77a5-34f6-4a39-bf81-1e135bc478be admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:10.376847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd9984a-7564-483d-a887-d4e595041299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:10.377501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9984a-7564-483d-a887-d4e595041299 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:10.377769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9984a-7564-483d-a887-d4e595041299 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:10.378048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9984a-7564-483d-a887-d4e595041299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:10.388676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd9984a-7564-483d-a887-d4e595041299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:10.390173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd9984a-7564-483d-a887-d4e595041299 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:10.401772 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1979/4030] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:01:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:01:10.407594 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2052/4031] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:01:10 2022] POST /identity/v3/projects => generated 391 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:01:10.411734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd9984a-7564-483d-a887-d4e595041299 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:10.422855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd9984a-7564-483d-a887-d4e595041299 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:10.483179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:10.498298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90eb74-0b7e-445c-b49c-2c2bae62179f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:10.498962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90eb74-0b7e-445c-b49c-2c2bae62179f admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:10.499248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90eb74-0b7e-445c-b49c-2c2bae62179f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:10.499518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90eb74-0b7e-445c-b49c-2c2bae62179f admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:10.500242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90eb74-0b7e-445c-b49c-2c2bae62179f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:10.502199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90eb74-0b7e-445c-b49c-2c2bae62179f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:10.531641 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1980/4032] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:01:10 2022] POST /identity/v3/users => generated 296 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:01:10.536880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda77a5-34f6-4a39-bf81-1e135bc478be admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:10.551153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda77a5-34f6-4a39-bf81-1e135bc478be admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:10.556624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30544a2-7d51-4897-a8eb-63a15eef4f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:10.557201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30544a2-7d51-4897-a8eb-63a15eef4f07 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:10.557947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30544a2-7d51-4897-a8eb-63a15eef4f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:10.558602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30544a2-7d51-4897-a8eb-63a15eef4f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:10.558874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30544a2-7d51-4897-a8eb-63a15eef4f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:10.566554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30544a2-7d51-4897-a8eb-63a15eef4f07 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:10.575563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:10.577169 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2053/4033] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:01:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:01:10.578316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90eb74-0b7e-445c-b49c-2c2bae62179f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:10.595563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90eb74-0b7e-445c-b49c-2c2bae62179f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:10.601322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bcf47-8747-4da7-86e0-a0f5cebf3e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:10.602007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bcf47-8747-4da7-86e0-a0f5cebf3e95 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:10.602291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bcf47-8747-4da7-86e0-a0f5cebf3e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:10.602662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bcf47-8747-4da7-86e0-a0f5cebf3e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:10.603318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648bcf47-8747-4da7-86e0-a0f5cebf3e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:10.607967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648bcf47-8747-4da7-86e0-a0f5cebf3e95 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:10.621654 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1981/4034] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:01:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:01:10.622921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30544a2-7d51-4897-a8eb-63a15eef4f07 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:10.636635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30544a2-7d51-4897-a8eb-63a15eef4f07 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:10.643025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c82fc-e19a-42c5-a5fc-3f9e38409a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:10.643891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c82fc-e19a-42c5-a5fc-3f9e38409a4d admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:10.644182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c82fc-e19a-42c5-a5fc-3f9e38409a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:10.644462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c82fc-e19a-42c5-a5fc-3f9e38409a4d admin admin] PATH_INFO: `/v3/projects/10fefee3d6a6430682316cd94b859f1e/users/6bda28994cb14f92b014a147a7e74476/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:10.663280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c82fc-e19a-42c5-a5fc-3f9e38409a4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=10fefee3d6a6430682316cd94b859f1e, user_id=6bda28994cb14f92b014a147a7e74476, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:10.666012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c82fc-e19a-42c5-a5fc-3f9e38409a4d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:10.679686 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2054/4035] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:01:10 2022] PUT /identity/v3/projects/10fefee3d6a6430682316cd94b859f1e/users/6bda28994cb14f92b014a147a7e74476/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:10.685086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298123f0-fd92-4b18-a408-81b499c573ce None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:10.685468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298123f0-fd92-4b18-a408-81b499c573ce None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:10.685834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298123f0-fd92-4b18-a408-81b499c573ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:10.705785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-298123f0-fd92-4b18-a408-81b499c573ce None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:10.707052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-298123f0-fd92-4b18-a408-81b499c573ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:10.773822 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1982/4036] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:10.796076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c82fc-e19a-42c5-a5fc-3f9e38409a4d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:10.798819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-884c82fc-e19a-42c5-a5fc-3f9e38409a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:10.868182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c82fc-e19a-42c5-a5fc-3f9e38409a4d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:10.874127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5472c37-277b-47b6-913c-cd158227f849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:10.875197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5472c37-277b-47b6-913c-cd158227f849 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:10.875587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5472c37-277b-47b6-913c-cd158227f849 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:10.875958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5472c37-277b-47b6-913c-cd158227f849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:10.889760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5472c37-277b-47b6-913c-cd158227f849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:10.890708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298123f0-fd92-4b18-a408-81b499c573ce None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:10.891661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5472c37-277b-47b6-913c-cd158227f849 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:10.892563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-298123f0-fd92-4b18-a408-81b499c573ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:10.926024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298123f0-fd92-4b18-a408-81b499c573ce None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:10.933122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3298407-7be1-44c6-9e0c-d189e3915aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:10.933983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3298407-7be1-44c6-9e0c-d189e3915aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:10.934422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3298407-7be1-44c6-9e0c-d189e3915aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:10.934882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3298407-7be1-44c6-9e0c-d189e3915aaa admin admin] PATH_INFO: `/v3/projects/497873f728f34c20aaa348bec348ed4f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:10.937780 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2055/4037] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:01:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:01:10.939349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3298407-7be1-44c6-9e0c-d189e3915aaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=497873f728f34c20aaa348bec348ed4f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:10.942762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3298407-7be1-44c6-9e0c-d189e3915aaa admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:11.018261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f3298407-7be1-44c6-9e0c-d189e3915aaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '497873f728f34c20aaa348bec348ed4f', 'request_id': 'req-f3298407-7be1-44c6-9e0c-d189e3915aaa'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:11.029784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f3298407-7be1-44c6-9e0c-d189e3915aaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '497873f728f34c20aaa348bec348ed4f', 'request_id': 'req-f3298407-7be1-44c6-9e0c-d189e3915aaa'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:11.039281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f3298407-7be1-44c6-9e0c-d189e3915aaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '497873f728f34c20aaa348bec348ed4f', 'request_id': 'req-f3298407-7be1-44c6-9e0c-d189e3915aaa'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:11.065100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f3298407-7be1-44c6-9e0c-d189e3915aaa admin admin] The token cache is being invalidate because project 497873f728f34c20aaa348bec348ed4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:01:11.065219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f3298407-7be1-44c6-9e0c-d189e3915aaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:11.065851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f3298407-7be1-44c6-9e0c-d189e3915aaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:11.070846 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1983/4038] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:01:10 2022] DELETE /identity/v3/projects/497873f728f34c20aaa348bec348ed4f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:11.072342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352c809-02d7-4e47-b2a3-067c116f449c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:11.073336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352c809-02d7-4e47-b2a3-067c116f449c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:11.073632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352c809-02d7-4e47-b2a3-067c116f449c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:11.109948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4352c809-02d7-4e47-b2a3-067c116f449c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:11.111270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4352c809-02d7-4e47-b2a3-067c116f449c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:11.227248 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2056/4039] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:11.230710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad55841-b9ee-48e6-a714-f683a66e23ae None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:11.231010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad55841-b9ee-48e6-a714-f683a66e23ae None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:11.231347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad55841-b9ee-48e6-a714-f683a66e23ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:11.276989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6ad55841-b9ee-48e6-a714-f683a66e23ae None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:11.278438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6ad55841-b9ee-48e6-a714-f683a66e23ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:11.295484 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1984/4040] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:11.299491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352c809-02d7-4e47-b2a3-067c116f449c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:11.315234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352c809-02d7-4e47-b2a3-067c116f449c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:11.331267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d3941-97d8-44fe-a304-84d03747ee12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:11.331905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d3941-97d8-44fe-a304-84d03747ee12 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:11.332189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d3941-97d8-44fe-a304-84d03747ee12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:11.332460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d3941-97d8-44fe-a304-84d03747ee12 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:11.333121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d3941-97d8-44fe-a304-84d03747ee12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:01:11.333421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d3941-97d8-44fe-a304-84d03747ee12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:11.335046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d3941-97d8-44fe-a304-84d03747ee12 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:11.347293 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2057/4041] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:01:11.351994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad55841-b9ee-48e6-a714-f683a66e23ae None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:11.362936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad55841-b9ee-48e6-a714-f683a66e23ae None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:11.368427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdb9af-99d1-4827-aa5c-fef287716856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:11.369048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdb9af-99d1-4827-aa5c-fef287716856 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:11.369847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdb9af-99d1-4827-aa5c-fef287716856 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:11.370181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdb9af-99d1-4827-aa5c-fef287716856 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:11.370944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdb9af-99d1-4827-aa5c-fef287716856 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:11.373092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdb9af-99d1-4827-aa5c-fef287716856 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:11.394761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d3941-97d8-44fe-a304-84d03747ee12 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:11.396689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b92d3941-97d8-44fe-a304-84d03747ee12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:11.409566 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1985/4042] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:01:11 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:01:11.410963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d3941-97d8-44fe-a304-84d03747ee12 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:11.413318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdb9af-99d1-4827-aa5c-fef287716856 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:11.455028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:11.484384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883e4dd-70ce-44be-98e5-6e794346665a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:11.485232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883e4dd-70ce-44be-98e5-6e794346665a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:11.485586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883e4dd-70ce-44be-98e5-6e794346665a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:11.485929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883e4dd-70ce-44be-98e5-6e794346665a admin admin] PATH_INFO: `/v3/users/7da28ff572834606911bf747228f55b5` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:11.488400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883e4dd-70ce-44be-98e5-6e794346665a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7da28ff572834606911bf747228f55b5)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:11.491402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883e4dd-70ce-44be-98e5-6e794346665a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:11.498619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:11.535437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdb9af-99d1-4827-aa5c-fef287716856 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:11.540860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8b75f-d1be-4dc9-842e-bc1fd6216268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:11.542230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8b75f-d1be-4dc9-842e-bc1fd6216268 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:11.542642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8b75f-d1be-4dc9-842e-bc1fd6216268 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:11.543192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8b75f-d1be-4dc9-842e-bc1fd6216268 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:11.543932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8b75f-d1be-4dc9-842e-bc1fd6216268 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:11.546685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8b75f-d1be-4dc9-842e-bc1fd6216268 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:11.580889 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1986/4043] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:01:11 2022] POST /identity/v3/users => generated 292 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:01:11.585179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8b75f-d1be-4dc9-842e-bc1fd6216268 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:11.602431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8b75f-d1be-4dc9-842e-bc1fd6216268 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:11.608812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc34e93-f6b3-46f0-93b9-29c0433eed0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:11.609430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc34e93-f6b3-46f0-93b9-29c0433eed0a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:11.609728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc34e93-f6b3-46f0-93b9-29c0433eed0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:11.609986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc34e93-f6b3-46f0-93b9-29c0433eed0a admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:11.610697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc34e93-f6b3-46f0-93b9-29c0433eed0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:11.612306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc34e93-f6b3-46f0-93b9-29c0433eed0a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:11.621067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6883e4dd-70ce-44be-98e5-6e794346665a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7da28ff572834606911bf747228f55b5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:11.637503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6883e4dd-70ce-44be-98e5-6e794346665a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7da28ff572834606911bf747228f55b5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:11.639084 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1987/4044] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:01:11.642444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc34e93-f6b3-46f0-93b9-29c0433eed0a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:11.661644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc34e93-f6b3-46f0-93b9-29c0433eed0a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:11.668282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba816f0-2776-4139-9f49-5953a5cf1613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:11.669053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba816f0-2776-4139-9f49-5953a5cf1613 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:11.669477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba816f0-2776-4139-9f49-5953a5cf1613 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:11.669824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba816f0-2776-4139-9f49-5953a5cf1613 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:11.670781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba816f0-2776-4139-9f49-5953a5cf1613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:11.671657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6883e4dd-70ce-44be-98e5-6e794346665a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7da28ff572834606911bf747228f55b5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:11.673319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba816f0-2776-4139-9f49-5953a5cf1613 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:11.689019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6883e4dd-70ce-44be-98e5-6e794346665a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7da28ff572834606911bf747228f55b5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:11.710613 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1988/4045] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:01:11.712139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba816f0-2776-4139-9f49-5953a5cf1613 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:11.717658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fba816f0-2776-4139-9f49-5953a5cf1613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:11.719099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6883e4dd-70ce-44be-98e5-6e794346665a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7da28ff572834606911bf747228f55b5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:11.733342 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2058/4046] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:01:11 2022] DELETE /identity/v3/users/7da28ff572834606911bf747228f55b5 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:01:11.738411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4884bbc-976b-43da-8fb1-d81f94ae32cd None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:11.740131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4884bbc-976b-43da-8fb1-d81f94ae32cd None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:11.740673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4884bbc-976b-43da-8fb1-d81f94ae32cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:11.752524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba816f0-2776-4139-9f49-5953a5cf1613 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:11.769124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c4884bbc-976b-43da-8fb1-d81f94ae32cd None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:11.770391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c4884bbc-976b-43da-8fb1-d81f94ae32cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:11.858578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a5f86-977d-4a47-beac-9570d19e9b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:11.859233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a5f86-977d-4a47-beac-9570d19e9b8e admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:11.859517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a5f86-977d-4a47-beac-9570d19e9b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:11.859795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a5f86-977d-4a47-beac-9570d19e9b8e admin admin] PATH_INFO: `/v3/projects/600066b2cda643488c4a2965d5b20661/users/779cdf8245f7483881c218306e3a76cb/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:11.883709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a5f86-977d-4a47-beac-9570d19e9b8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=600066b2cda643488c4a2965d5b20661, user_id=779cdf8245f7483881c218306e3a76cb, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:11.887500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a5f86-977d-4a47-beac-9570d19e9b8e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:11.894720 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2059/4047] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:11.896477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7d88a-c2cd-4381-8e4d-7ad912b44cc9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:11.896781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7d88a-c2cd-4381-8e4d-7ad912b44cc9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:11.897068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7d88a-c2cd-4381-8e4d-7ad912b44cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:11.926584 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1989/4048] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:01:11 2022] PUT /identity/v3/projects/600066b2cda643488c4a2965d5b20661/users/779cdf8245f7483881c218306e3a76cb/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:01:11.926584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a5f86-977d-4a47-beac-9570d19e9b8e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:11.937823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-488a5f86-977d-4a47-beac-9570d19e9b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:11.955630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a5c7d88a-c2cd-4381-8e4d-7ad912b44cc9 None None] MFA Rules not processed for user `e84a109e337d450698e630acfb49f9a5`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:11.958739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a5c7d88a-c2cd-4381-8e4d-7ad912b44cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:12.073062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a5f86-977d-4a47-beac-9570d19e9b8e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:12.080231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a04c63-1434-4681-84b8-4acc92cca60a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:12.081343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a04c63-1434-4681-84b8-4acc92cca60a service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:12.081961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a04c63-1434-4681-84b8-4acc92cca60a service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:12.082444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a04c63-1434-4681-84b8-4acc92cca60a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:12.133859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:12.138747 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2060/4049] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:12.141231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56108451-b27e-4fcb-989d-0fa976919824 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:12.144540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56108451-b27e-4fcb-989d-0fa976919824 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:12.145115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56108451-b27e-4fcb-989d-0fa976919824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:12.199623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-56108451-b27e-4fcb-989d-0fa976919824 None None] MFA Rules not processed for user `779cdf8245f7483881c218306e3a76cb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:12.203691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-56108451-b27e-4fcb-989d-0fa976919824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:12.227352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a04c63-1434-4681-84b8-4acc92cca60a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:12.229389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a04c63-1434-4681-84b8-4acc92cca60a service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:12.240725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:12.256132 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1990/4050] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:01:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:01:12.377427 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2061/4051] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:12.388962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a04c63-1434-4681-84b8-4acc92cca60a service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:12.390345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f3a04c63-1434-4681-84b8-4acc92cca60a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:12.543445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a04c63-1434-4681-84b8-4acc92cca60a service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:12.550186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06bc68-eab1-4d0e-be0c-c471a4fefe02 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:12.551473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06bc68-eab1-4d0e-be0c-c471a4fefe02 service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:12.552157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06bc68-eab1-4d0e-be0c-c471a4fefe02 service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:12.552599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06bc68-eab1-4d0e-be0c-c471a4fefe02 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:12.564259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06bc68-eab1-4d0e-be0c-c471a4fefe02 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:12.565841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06bc68-eab1-4d0e-be0c-c471a4fefe02 service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:12.580612 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1991/4052] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:01:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:01:13.181852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56108451-b27e-4fcb-989d-0fa976919824 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:13.184170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-56108451-b27e-4fcb-989d-0fa976919824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:13.204604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56108451-b27e-4fcb-989d-0fa976919824 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:13.211951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffae781-45fa-4e16-a036-8541f53b7cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:13.214449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffae781-45fa-4e16-a036-8541f53b7cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:13.214898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffae781-45fa-4e16-a036-8541f53b7cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:13.215288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffae781-45fa-4e16-a036-8541f53b7cd3 admin admin] PATH_INFO: `/v3/projects/944af14831394cc988236387e13e9f7a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:13.216780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffae781-45fa-4e16-a036-8541f53b7cd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=944af14831394cc988236387e13e9f7a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:13.219130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffae781-45fa-4e16-a036-8541f53b7cd3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:13.335659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6ffae781-45fa-4e16-a036-8541f53b7cd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '944af14831394cc988236387e13e9f7a', 'request_id': 'req-6ffae781-45fa-4e16-a036-8541f53b7cd3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:13.344187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6ffae781-45fa-4e16-a036-8541f53b7cd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '944af14831394cc988236387e13e9f7a', 'request_id': 'req-6ffae781-45fa-4e16-a036-8541f53b7cd3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:13.382730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6ffae781-45fa-4e16-a036-8541f53b7cd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '944af14831394cc988236387e13e9f7a', 'request_id': 'req-6ffae781-45fa-4e16-a036-8541f53b7cd3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:13.404178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6ffae781-45fa-4e16-a036-8541f53b7cd3 admin admin] The token cache is being invalidate because project 944af14831394cc988236387e13e9f7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:01:13.404818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6ffae781-45fa-4e16-a036-8541f53b7cd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:13.406219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6ffae781-45fa-4e16-a036-8541f53b7cd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:13.417694 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2062/4053] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:01:13 2022] DELETE /identity/v3/projects/944af14831394cc988236387e13e9f7a => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:13.420944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06bc68-eab1-4d0e-be0c-c471a4fefe02 service glance] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:13.448720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06bc68-eab1-4d0e-be0c-c471a4fefe02 service glance] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:13.566999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9a29d-8f59-4e06-93b4-8f2c9eca1cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:13.571368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9a29d-8f59-4e06-93b4-8f2c9eca1cdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:13.580514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9a29d-8f59-4e06-93b4-8f2c9eca1cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:13.580827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9a29d-8f59-4e06-93b4-8f2c9eca1cdb admin admin] PATH_INFO: `/v3/users/f9fce09a093d40b9ace10301ba512023` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:13.582834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9a29d-8f59-4e06-93b4-8f2c9eca1cdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9fce09a093d40b9ace10301ba512023)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:13.585479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9a29d-8f59-4e06-93b4-8f2c9eca1cdb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:13.647291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a7c9a29d-8f59-4e06-93b4-8f2c9eca1cdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9fce09a093d40b9ace10301ba512023'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:13.648485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a7c9a29d-8f59-4e06-93b4-8f2c9eca1cdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9fce09a093d40b9ace10301ba512023'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:13.649555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a7c9a29d-8f59-4e06-93b4-8f2c9eca1cdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9fce09a093d40b9ace10301ba512023'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:13.658650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a7c9a29d-8f59-4e06-93b4-8f2c9eca1cdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9fce09a093d40b9ace10301ba512023'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:13.701423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:13.723084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a7c9a29d-8f59-4e06-93b4-8f2c9eca1cdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9fce09a093d40b9ace10301ba512023'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:13.786627 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1992/4054] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:01:13 2022] DELETE /identity/v3/users/f9fce09a093d40b9ace10301ba512023 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:13.815555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe54017-260b-484e-9bf1-7c61a7dedece None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:13.815952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe54017-260b-484e-9bf1-7c61a7dedece None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:13.816357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe54017-260b-484e-9bf1-7c61a7dedece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:13.848438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2fe54017-260b-484e-9bf1-7c61a7dedece None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:13.849670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2fe54017-260b-484e-9bf1-7c61a7dedece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:13.984688 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2063/4055] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:01:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:13.999011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9a29d-8f59-4e06-93b4-8f2c9eca1cdb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:14.000623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a7c9a29d-8f59-4e06-93b4-8f2c9eca1cdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:14.223835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe54017-260b-484e-9bf1-7c61a7dedece None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:14.226606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2fe54017-260b-484e-9bf1-7c61a7dedece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:14.253316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9a29d-8f59-4e06-93b4-8f2c9eca1cdb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:14.271441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b27e9-5135-40fe-83fb-cf9bb79c86a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:14.273394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b27e9-5135-40fe-83fb-cf9bb79c86a2 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:14.273682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b27e9-5135-40fe-83fb-cf9bb79c86a2 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:14.275177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b27e9-5135-40fe-83fb-cf9bb79c86a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:14.291825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b27e9-5135-40fe-83fb-cf9bb79c86a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:14.293842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b27e9-5135-40fe-83fb-cf9bb79c86a2 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:14.304342 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1993/4056] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:01:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:01:14.358069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:14.402438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe54017-260b-484e-9bf1-7c61a7dedece None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:14.416362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f9dc6-31c1-45b5-bf57-3e83e79d9df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:14.417028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f9dc6-31c1-45b5-bf57-3e83e79d9df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:14.417313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f9dc6-31c1-45b5-bf57-3e83e79d9df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:14.417589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f9dc6-31c1-45b5-bf57-3e83e79d9df4 admin admin] PATH_INFO: `/v3/projects/ec2b810463fc4e548240c81e353ae9fd` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:14.419245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f9dc6-31c1-45b5-bf57-3e83e79d9df4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec2b810463fc4e548240c81e353ae9fd)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:14.421162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f9dc6-31c1-45b5-bf57-3e83e79d9df4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:14.532889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-670f9dc6-31c1-45b5-bf57-3e83e79d9df4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec2b810463fc4e548240c81e353ae9fd', 'request_id': 'req-670f9dc6-31c1-45b5-bf57-3e83e79d9df4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:14.541776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-670f9dc6-31c1-45b5-bf57-3e83e79d9df4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec2b810463fc4e548240c81e353ae9fd', 'request_id': 'req-670f9dc6-31c1-45b5-bf57-3e83e79d9df4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:14.555611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-670f9dc6-31c1-45b5-bf57-3e83e79d9df4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec2b810463fc4e548240c81e353ae9fd', 'request_id': 'req-670f9dc6-31c1-45b5-bf57-3e83e79d9df4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:14.582947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-670f9dc6-31c1-45b5-bf57-3e83e79d9df4 admin admin] The token cache is being invalidate because project ec2b810463fc4e548240c81e353ae9fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:01:14.583244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-670f9dc6-31c1-45b5-bf57-3e83e79d9df4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:14.584881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-670f9dc6-31c1-45b5-bf57-3e83e79d9df4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:14.591604 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2064/4057] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:01:14 2022] DELETE /identity/v3/projects/ec2b810463fc4e548240c81e353ae9fd => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:01:14.592927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c5c18-659c-4372-8387-ed8468134a36 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:14.592927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c5c18-659c-4372-8387-ed8468134a36 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:14.593465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c5c18-659c-4372-8387-ed8468134a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:14.638575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-224c5c18-659c-4372-8387-ed8468134a36 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:14.640798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-224c5c18-659c-4372-8387-ed8468134a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:14.743663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc344883-e367-426b-9891-cdddf60a9dfe None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:14.745936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc344883-e367-426b-9891-cdddf60a9dfe None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:14.745936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc344883-e367-426b-9891-cdddf60a9dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:14.750002 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1994/4058] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:01:14.770914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-dc344883-e367-426b-9891-cdddf60a9dfe None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:14.772356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-dc344883-e367-426b-9891-cdddf60a9dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:14.788263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c5c18-659c-4372-8387-ed8468134a36 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:14.798703 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2065/4059] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:14.801423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c5c18-659c-4372-8387-ed8468134a36 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:14.832325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2addf6b-ccf3-4819-81bd-3c67f521df05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:14.833075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2addf6b-ccf3-4819-81bd-3c67f521df05 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:14.833469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2addf6b-ccf3-4819-81bd-3c67f521df05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:14.833984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2addf6b-ccf3-4819-81bd-3c67f521df05 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:14.840430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2addf6b-ccf3-4819-81bd-3c67f521df05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:01:14.840819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2addf6b-ccf3-4819-81bd-3c67f521df05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:14.842622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2addf6b-ccf3-4819-81bd-3c67f521df05 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:14.873963 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1995/4060] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:01:14.878558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc344883-e367-426b-9891-cdddf60a9dfe None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:14.889317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc344883-e367-426b-9891-cdddf60a9dfe None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:14.914967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c7ff9-426c-4c82-9c16-1a5b4e9eb656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:14.915751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c7ff9-426c-4c82-9c16-1a5b4e9eb656 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:14.918328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c7ff9-426c-4c82-9c16-1a5b4e9eb656 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:14.920490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c7ff9-426c-4c82-9c16-1a5b4e9eb656 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:14.921774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c7ff9-426c-4c82-9c16-1a5b4e9eb656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:14.927167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c7ff9-426c-4c82-9c16-1a5b4e9eb656 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:15.000962 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2066/4061] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:01:14 2022] POST /identity/v3/projects => generated 387 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:01:15.004583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2addf6b-ccf3-4819-81bd-3c67f521df05 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:15.021199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2addf6b-ccf3-4819-81bd-3c67f521df05 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:15.190946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940305ab-0b6c-4862-a6d5-cd15eea3adae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:15.194120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940305ab-0b6c-4862-a6d5-cd15eea3adae admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:15.194738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940305ab-0b6c-4862-a6d5-cd15eea3adae admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:15.195026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940305ab-0b6c-4862-a6d5-cd15eea3adae admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:15.195691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940305ab-0b6c-4862-a6d5-cd15eea3adae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:15.203206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940305ab-0b6c-4862-a6d5-cd15eea3adae admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:15.228603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:15.254614 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1996/4062] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:01:15 2022] POST /identity/v3/users => generated 294 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:01:15.259229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c7ff9-426c-4c82-9c16-1a5b4e9eb656 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:15.282789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c7ff9-426c-4c82-9c16-1a5b4e9eb656 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:15.295297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a691455-8c5f-44ec-a890-fdebe7558c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:15.297997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a691455-8c5f-44ec-a890-fdebe7558c89 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:15.298614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a691455-8c5f-44ec-a890-fdebe7558c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:15.299268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a691455-8c5f-44ec-a890-fdebe7558c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:15.300676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a691455-8c5f-44ec-a890-fdebe7558c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:15.305043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a691455-8c5f-44ec-a890-fdebe7558c89 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:15.320035 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2067/4063] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:01:15.323888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940305ab-0b6c-4862-a6d5-cd15eea3adae admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:15.338727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940305ab-0b6c-4862-a6d5-cd15eea3adae admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:15.363985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ef029-2f31-44f7-8c99-d0e680c7873a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:15.364625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ef029-2f31-44f7-8c99-d0e680c7873a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:15.366242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ef029-2f31-44f7-8c99-d0e680c7873a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:15.366427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ef029-2f31-44f7-8c99-d0e680c7873a admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:15.368935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0ef029-2f31-44f7-8c99-d0e680c7873a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:15.374663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0ef029-2f31-44f7-8c99-d0e680c7873a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:15.396728 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1997/4064] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:01:15.401041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a691455-8c5f-44ec-a890-fdebe7558c89 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:15.417312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a691455-8c5f-44ec-a890-fdebe7558c89 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:15.423682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b851c9b-bcec-4584-8921-17cc5e9c7674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:15.424587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b851c9b-bcec-4584-8921-17cc5e9c7674 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:15.425056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b851c9b-bcec-4584-8921-17cc5e9c7674 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:15.426035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b851c9b-bcec-4584-8921-17cc5e9c7674 admin admin] PATH_INFO: `/v3/projects/d730261bad5b42e889aa8c8968faea0b/users/3c710e13874046689fad9461a82f16a0/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:15.455295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b851c9b-bcec-4584-8921-17cc5e9c7674 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d730261bad5b42e889aa8c8968faea0b, user_id=3c710e13874046689fad9461a82f16a0, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:15.460320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b851c9b-bcec-4584-8921-17cc5e9c7674 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:15.487870 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2068/4065] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:01:15 2022] PUT /identity/v3/projects/d730261bad5b42e889aa8c8968faea0b/users/3c710e13874046689fad9461a82f16a0/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:15.499571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcaefc-fa3f-4071-9cf1-c21aa0dc2aad None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:15.499783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcaefc-fa3f-4071-9cf1-c21aa0dc2aad None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:15.500091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcaefc-fa3f-4071-9cf1-c21aa0dc2aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:15.532902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e5dcaefc-fa3f-4071-9cf1-c21aa0dc2aad None None] MFA Rules not processed for user `3c710e13874046689fad9461a82f16a0`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:15.534336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e5dcaefc-fa3f-4071-9cf1-c21aa0dc2aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:15.665723 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1998/4066] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:15.688504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b851c9b-bcec-4584-8921-17cc5e9c7674 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:15.690541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0b851c9b-bcec-4584-8921-17cc5e9c7674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:15.801944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b851c9b-bcec-4584-8921-17cc5e9c7674 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:15.821332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa7cc2-d380-4979-9df5-ddf1efc2e0dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:15.822050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa7cc2-d380-4979-9df5-ddf1efc2e0dc service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:15.825379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa7cc2-d380-4979-9df5-ddf1efc2e0dc service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:15.825379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa7cc2-d380-4979-9df5-ddf1efc2e0dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:15.862815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa7cc2-d380-4979-9df5-ddf1efc2e0dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:15.864806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa7cc2-d380-4979-9df5-ddf1efc2e0dc service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:15.890385 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2069/4067] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:01:15 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:01:17.141146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ca33e-5a07-4558-be54-547d53ef01a1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:17.141650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ca33e-5a07-4558-be54-547d53ef01a1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:17.141650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ca33e-5a07-4558-be54-547d53ef01a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:17.176065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e01ca33e-5a07-4558-be54-547d53ef01a1 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:17.178297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e01ca33e-5a07-4558-be54-547d53ef01a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:17.266161 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 1999/4068] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:01:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:17.266377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa7cc2-d380-4979-9df5-ddf1efc2e0dc service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:17.268079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-69aa7cc2-d380-4979-9df5-ddf1efc2e0dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:17.325984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:17.420946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa7cc2-d380-4979-9df5-ddf1efc2e0dc service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:17.431263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce93be-1c75-49dc-99c7-a2acd3f553fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:17.432827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce93be-1c75-49dc-99c7-a2acd3f553fb service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:17.433275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce93be-1c75-49dc-99c7-a2acd3f553fb service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:17.433671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce93be-1c75-49dc-99c7-a2acd3f553fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:17.444084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce93be-1c75-49dc-99c7-a2acd3f553fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:17.445929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce93be-1c75-49dc-99c7-a2acd3f553fb service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:17.457750 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2070/4069] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:01:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 21:01:20.941167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdc07b-5fb7-47ab-bc54-ab760846bc3a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:20.941701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdc07b-5fb7-47ab-bc54-ab760846bc3a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:20.941701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdc07b-5fb7-47ab-bc54-ab760846bc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:20.968846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-47bdc07b-5fb7-47ab-bc54-ab760846bc3a None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:20.970606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-47bdc07b-5fb7-47ab-bc54-ab760846bc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:20.990989 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2000/4070] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:01:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:20.998803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce93be-1c75-49dc-99c7-a2acd3f553fb service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:21.012654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce93be-1c75-49dc-99c7-a2acd3f553fb service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:21.020627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a3a84-47f1-44df-ad26-454c9b18b112 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:21.021346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a3a84-47f1-44df-ad26-454c9b18b112 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:21.021704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a3a84-47f1-44df-ad26-454c9b18b112 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:21.022099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a3a84-47f1-44df-ad26-454c9b18b112 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:21.048819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a3a84-47f1-44df-ad26-454c9b18b112 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:21.050513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a3a84-47f1-44df-ad26-454c9b18b112 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:21.071467 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2071/4071] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:01:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:01:24.193482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef73614-18a3-464a-8ee2-c6c339af696a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:24.195009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef73614-18a3-464a-8ee2-c6c339af696a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:24.196106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef73614-18a3-464a-8ee2-c6c339af696a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:24.255593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0ef73614-18a3-464a-8ee2-c6c339af696a None None] MFA Rules not processed for user `6bda28994cb14f92b014a147a7e74476`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:24.256826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0ef73614-18a3-464a-8ee2-c6c339af696a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:24.406873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:24.411986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50037123-d926-49d3-bfa1-8fba1cce7a99 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:24.412806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50037123-d926-49d3-bfa1-8fba1cce7a99 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:24.413271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50037123-d926-49d3-bfa1-8fba1cce7a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:24.416152 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2001/4072] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:24.426304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5d9c1-70db-4e8c-a6f1-862125d37d23 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:24.426897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5d9c1-70db-4e8c-a6f1-862125d37d23 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:24.427544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5d9c1-70db-4e8c-a6f1-862125d37d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:24.459977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-50037123-d926-49d3-bfa1-8fba1cce7a99 None None] MFA Rules not processed for user `779cdf8245f7483881c218306e3a76cb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:24.465263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-50037123-d926-49d3-bfa1-8fba1cce7a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:24.480845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-eaf5d9c1-70db-4e8c-a6f1-862125d37d23 None None] MFA Rules not processed for user `6bda28994cb14f92b014a147a7e74476`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:24.483349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-eaf5d9c1-70db-4e8c-a6f1-862125d37d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:24.521269 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2002/4073] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:01:24.553470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbc45e-9f7a-4333-873f-f4d5e9962125 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:24.553985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbc45e-9f7a-4333-873f-f4d5e9962125 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:24.554459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbc45e-9f7a-4333-873f-f4d5e9962125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:24.596549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-96bbc45e-9f7a-4333-873f-f4d5e9962125 None None] MFA Rules not processed for user `775ff9091a794faaacc05a37550c8a72`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:24.601113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-96bbc45e-9f7a-4333-873f-f4d5e9962125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:24.601862 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2072/4074] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:01:24.605090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50037123-d926-49d3-bfa1-8fba1cce7a99 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:24.609381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-50037123-d926-49d3-bfa1-8fba1cce7a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:24.724630 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2003/4075] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:01:24.798473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50037123-d926-49d3-bfa1-8fba1cce7a99 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:24.808474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fba301-3323-465a-bbe0-c6c24e49a381 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:24.809236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fba301-3323-465a-bbe0-c6c24e49a381 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:24.809770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fba301-3323-465a-bbe0-c6c24e49a381 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:24.810249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fba301-3323-465a-bbe0-c6c24e49a381 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:24.822932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fba301-3323-465a-bbe0-c6c24e49a381 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:24.825060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fba301-3323-465a-bbe0-c6c24e49a381 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:24.862638 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2073/4076] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:01:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:01:27.730126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456dc58-b7c1-4f59-a7ee-6010a8ee89f9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:27.731005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456dc58-b7c1-4f59-a7ee-6010a8ee89f9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:27.731434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456dc58-b7c1-4f59-a7ee-6010a8ee89f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:27.780785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1456dc58-b7c1-4f59-a7ee-6010a8ee89f9 None None] MFA Rules not processed for user `779cdf8245f7483881c218306e3a76cb`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:27.784773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1456dc58-b7c1-4f59-a7ee-6010a8ee89f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:27.796397 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2004/4077] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:31.071166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b393958-dffb-44d2-be9d-7a7f15dadfa5 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:31.071726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b393958-dffb-44d2-be9d-7a7f15dadfa5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:31.071726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b393958-dffb-44d2-be9d-7a7f15dadfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:31.123269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6b393958-dffb-44d2-be9d-7a7f15dadfa5 None None] MFA Rules not processed for user `6bda28994cb14f92b014a147a7e74476`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:31.125795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6b393958-dffb-44d2-be9d-7a7f15dadfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:31.145995 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2074/4078] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:01:32.083956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456dc58-b7c1-4f59-a7ee-6010a8ee89f9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:32.175720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456dc58-b7c1-4f59-a7ee-6010a8ee89f9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:32.181440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b2e18-6aad-4c8e-bd4a-069eb090dc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:32.182095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b2e18-6aad-4c8e-bd4a-069eb090dc01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:32.182673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b2e18-6aad-4c8e-bd4a-069eb090dc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:32.182955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b2e18-6aad-4c8e-bd4a-069eb090dc01 admin admin] PATH_INFO: `/v3/users/3c710e13874046689fad9461a82f16a0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:32.184116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b2e18-6aad-4c8e-bd4a-069eb090dc01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c710e13874046689fad9461a82f16a0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:32.187733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b2e18-6aad-4c8e-bd4a-069eb090dc01 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:32.287373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-869b2e18-6aad-4c8e-bd4a-069eb090dc01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c710e13874046689fad9461a82f16a0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:32.288893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-869b2e18-6aad-4c8e-bd4a-069eb090dc01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c710e13874046689fad9461a82f16a0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:32.291066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-869b2e18-6aad-4c8e-bd4a-069eb090dc01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c710e13874046689fad9461a82f16a0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:32.325552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-869b2e18-6aad-4c8e-bd4a-069eb090dc01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c710e13874046689fad9461a82f16a0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:32.362241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-869b2e18-6aad-4c8e-bd4a-069eb090dc01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c710e13874046689fad9461a82f16a0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:32.388258 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2005/4079] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:01:32 2022] DELETE /identity/v3/users/3c710e13874046689fad9461a82f16a0 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:32.392727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa2a03-2d88-40b2-93dc-cb88cf1b9301 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:32.394978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa2a03-2d88-40b2-93dc-cb88cf1b9301 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:32.395454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa2a03-2d88-40b2-93dc-cb88cf1b9301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:32.435887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-91aa2a03-2d88-40b2-93dc-cb88cf1b9301 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:32.437318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-91aa2a03-2d88-40b2-93dc-cb88cf1b9301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:32.473598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:32.629979 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2075/4080] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:01:32.679769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b2e18-6aad-4c8e-bd4a-069eb090dc01 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:32.681835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-869b2e18-6aad-4c8e-bd4a-069eb090dc01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:32.718154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:32.912914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b2e18-6aad-4c8e-bd4a-069eb090dc01 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:32.919340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493182c-6a9a-4d8d-aade-b8b07c95cb50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:32.920021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493182c-6a9a-4d8d-aade-b8b07c95cb50 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:32.920367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493182c-6a9a-4d8d-aade-b8b07c95cb50 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:32.920775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493182c-6a9a-4d8d-aade-b8b07c95cb50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:32.947634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493182c-6a9a-4d8d-aade-b8b07c95cb50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:32.953450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493182c-6a9a-4d8d-aade-b8b07c95cb50 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:33.012280 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2006/4081] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:01:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:01:34.091354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa2a03-2d88-40b2-93dc-cb88cf1b9301 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:34.094074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-91aa2a03-2d88-40b2-93dc-cb88cf1b9301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:34.111412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa2a03-2d88-40b2-93dc-cb88cf1b9301 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:34.125803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b0a6e8-cb7a-4e1f-8e2c-513833cf89a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:34.127968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0a6e8-cb7a-4e1f-8e2c-513833cf89a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:34.141377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0a6e8-cb7a-4e1f-8e2c-513833cf89a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:34.142633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0a6e8-cb7a-4e1f-8e2c-513833cf89a4 admin admin] PATH_INFO: `/v3/projects/d730261bad5b42e889aa8c8968faea0b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:34.146717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0a6e8-cb7a-4e1f-8e2c-513833cf89a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d730261bad5b42e889aa8c8968faea0b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:34.152310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0a6e8-cb7a-4e1f-8e2c-513833cf89a4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:34.218680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-56b0a6e8-cb7a-4e1f-8e2c-513833cf89a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd730261bad5b42e889aa8c8968faea0b', 'request_id': 'req-56b0a6e8-cb7a-4e1f-8e2c-513833cf89a4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:34.227282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-56b0a6e8-cb7a-4e1f-8e2c-513833cf89a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd730261bad5b42e889aa8c8968faea0b', 'request_id': 'req-56b0a6e8-cb7a-4e1f-8e2c-513833cf89a4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:34.242403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-56b0a6e8-cb7a-4e1f-8e2c-513833cf89a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd730261bad5b42e889aa8c8968faea0b', 'request_id': 'req-56b0a6e8-cb7a-4e1f-8e2c-513833cf89a4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:34.291189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-56b0a6e8-cb7a-4e1f-8e2c-513833cf89a4 admin admin] The token cache is being invalidate because project d730261bad5b42e889aa8c8968faea0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:01:34.291877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-56b0a6e8-cb7a-4e1f-8e2c-513833cf89a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:34.293826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-56b0a6e8-cb7a-4e1f-8e2c-513833cf89a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:34.300145 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2076/4082] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:01:34 2022] DELETE /identity/v3/projects/d730261bad5b42e889aa8c8968faea0b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:34.301414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70ad15-9e7f-4b1d-a141-09f7be178a49 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:34.301869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70ad15-9e7f-4b1d-a141-09f7be178a49 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:34.302099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70ad15-9e7f-4b1d-a141-09f7be178a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:34.340267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0d70ad15-9e7f-4b1d-a141-09f7be178a49 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:34.343784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0d70ad15-9e7f-4b1d-a141-09f7be178a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:34.419627 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2007/4083] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:34.428688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f041a-dc42-4e26-8b70-a26cd6870ccd None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:34.429204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f041a-dc42-4e26-8b70-a26cd6870ccd None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:34.429551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f041a-dc42-4e26-8b70-a26cd6870ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:34.451460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-418f041a-dc42-4e26-8b70-a26cd6870ccd None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:34.452996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-418f041a-dc42-4e26-8b70-a26cd6870ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:34.461625 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2077/4084] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:34.466527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70ad15-9e7f-4b1d-a141-09f7be178a49 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:34.477608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70ad15-9e7f-4b1d-a141-09f7be178a49 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:34.483398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65a206-b3d8-471a-acc1-aec940e8d940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:34.484064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65a206-b3d8-471a-acc1-aec940e8d940 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:34.484349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65a206-b3d8-471a-acc1-aec940e8d940 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:34.484622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65a206-b3d8-471a-acc1-aec940e8d940 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:34.485267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65a206-b3d8-471a-acc1-aec940e8d940 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:01:34.485566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65a206-b3d8-471a-acc1-aec940e8d940 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:34.487227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65a206-b3d8-471a-acc1-aec940e8d940 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:34.501350 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2008/4085] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:01:34.505728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f041a-dc42-4e26-8b70-a26cd6870ccd None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:34.519086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f041a-dc42-4e26-8b70-a26cd6870ccd None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:34.525038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337c1c7-78ed-4440-8f71-220397d1984e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:34.525663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337c1c7-78ed-4440-8f71-220397d1984e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:34.525940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337c1c7-78ed-4440-8f71-220397d1984e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:34.526256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337c1c7-78ed-4440-8f71-220397d1984e admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:34.526934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337c1c7-78ed-4440-8f71-220397d1984e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:34.528826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337c1c7-78ed-4440-8f71-220397d1984e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:34.557863 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2078/4086] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:01:34 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:01:34.562161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65a206-b3d8-471a-acc1-aec940e8d940 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:34.574969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65a206-b3d8-471a-acc1-aec940e8d940 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:34.699597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bfdcf-3bda-4118-85cd-7ee07fd79a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:34.700437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bfdcf-3bda-4118-85cd-7ee07fd79a9d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:34.700898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bfdcf-3bda-4118-85cd-7ee07fd79a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:34.701308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bfdcf-3bda-4118-85cd-7ee07fd79a9d admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:34.702235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bfdcf-3bda-4118-85cd-7ee07fd79a9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:34.704937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bfdcf-3bda-4118-85cd-7ee07fd79a9d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:34.749070 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2009/4087] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:01:34 2022] POST /identity/v3/users => generated 285 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:01:34.753663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337c1c7-78ed-4440-8f71-220397d1984e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:34.764977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337c1c7-78ed-4440-8f71-220397d1984e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:34.770128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e692ad-831f-4d8d-a9ea-b9af7c4421a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:34.771062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e692ad-831f-4d8d-a9ea-b9af7c4421a6 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:34.771415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e692ad-831f-4d8d-a9ea-b9af7c4421a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:34.771696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e692ad-831f-4d8d-a9ea-b9af7c4421a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:34.773288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e692ad-831f-4d8d-a9ea-b9af7c4421a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:34.774803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e692ad-831f-4d8d-a9ea-b9af7c4421a6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:34.790817 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2079/4088] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:01:34.791721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bfdcf-3bda-4118-85cd-7ee07fd79a9d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:34.802309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bfdcf-3bda-4118-85cd-7ee07fd79a9d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:34.808572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a413ec-2f17-496e-b373-5355785acdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:34.808769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a413ec-2f17-496e-b373-5355785acdf1 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:34.809033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a413ec-2f17-496e-b373-5355785acdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:34.809400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a413ec-2f17-496e-b373-5355785acdf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:34.809936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a413ec-2f17-496e-b373-5355785acdf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:34.811812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a413ec-2f17-496e-b373-5355785acdf1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:34.824259 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2010/4089] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:01:34.828543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e692ad-831f-4d8d-a9ea-b9af7c4421a6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:34.840189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e692ad-831f-4d8d-a9ea-b9af7c4421a6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:34.845134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe313be-b9e4-4c27-87c8-3ca7832519a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:34.846034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe313be-b9e4-4c27-87c8-3ca7832519a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:34.846312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe313be-b9e4-4c27-87c8-3ca7832519a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:34.846687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe313be-b9e4-4c27-87c8-3ca7832519a6 admin admin] PATH_INFO: `/v3/projects/dbbe49ca8a0b43c2b7bb4b886c4393e8/users/b4098dddfc2f442895545cc758ee430a/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:34.867015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe313be-b9e4-4c27-87c8-3ca7832519a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbbe49ca8a0b43c2b7bb4b886c4393e8, user_id=b4098dddfc2f442895545cc758ee430a, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:34.870252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe313be-b9e4-4c27-87c8-3ca7832519a6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:34.888933 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2080/4090] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:01:34 2022] PUT /identity/v3/projects/dbbe49ca8a0b43c2b7bb4b886c4393e8/users/b4098dddfc2f442895545cc758ee430a/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:34.900826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1b9d3-ff01-47cd-b17b-13c23fef7fa1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:34.901109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1b9d3-ff01-47cd-b17b-13c23fef7fa1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:34.901301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1b9d3-ff01-47cd-b17b-13c23fef7fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:34.944477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4fe1b9d3-ff01-47cd-b17b-13c23fef7fa1 None None] MFA Rules not processed for user `b4098dddfc2f442895545cc758ee430a`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:34.945665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4fe1b9d3-ff01-47cd-b17b-13c23fef7fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:34.952418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:35.071909 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2011/4091] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:35.085815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe313be-b9e4-4c27-87c8-3ca7832519a6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:35.088582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-efe313be-b9e4-4c27-87c8-3ca7832519a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:35.155085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:35.206879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe313be-b9e4-4c27-87c8-3ca7832519a6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:35.215452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c44ca3-dc7c-47a5-88db-62822371f501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:35.217185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c44ca3-dc7c-47a5-88db-62822371f501 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:35.218171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c44ca3-dc7c-47a5-88db-62822371f501 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:35.218805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c44ca3-dc7c-47a5-88db-62822371f501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:35.254242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c44ca3-dc7c-47a5-88db-62822371f501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:35.257936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c44ca3-dc7c-47a5-88db-62822371f501 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:35.284606 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2081/4092] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:01:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:01:36.910547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1b9d3-ff01-47cd-b17b-13c23fef7fa1 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:36.912840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4fe1b9d3-ff01-47cd-b17b-13c23fef7fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:37.146523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1b9d3-ff01-47cd-b17b-13c23fef7fa1 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:37.164464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720884c-929a-478c-9aa7-8f77b7ebfdfd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:37.165424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720884c-929a-478c-9aa7-8f77b7ebfdfd service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:37.165868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720884c-929a-478c-9aa7-8f77b7ebfdfd service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:37.167094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720884c-929a-478c-9aa7-8f77b7ebfdfd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:37.170321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1720884c-929a-478c-9aa7-8f77b7ebfdfd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:37.202457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720884c-929a-478c-9aa7-8f77b7ebfdfd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:37.207350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720884c-929a-478c-9aa7-8f77b7ebfdfd service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:37.247323 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2012/4093] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:01:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:01:42.079510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c44ca3-dc7c-47a5-88db-62822371f501 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:42.095200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c44ca3-dc7c-47a5-88db-62822371f501 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:42.102200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010cac5-dfc5-4ca3-9777-3590edde6469 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:42.103380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010cac5-dfc5-4ca3-9777-3590edde6469 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:42.103882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010cac5-dfc5-4ca3-9777-3590edde6469 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:42.104359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010cac5-dfc5-4ca3-9777-3590edde6469 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:42.106826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b010cac5-dfc5-4ca3-9777-3590edde6469 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:42.122281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010cac5-dfc5-4ca3-9777-3590edde6469 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:42.125124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010cac5-dfc5-4ca3-9777-3590edde6469 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:42.150426 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2082/4094] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:01:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:01:47.230006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=56 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:47.876118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7e55e-b45f-4583-8590-ac09bbe459ef None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:47.876705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7e55e-b45f-4583-8590-ac09bbe459ef None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:47.877205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7e55e-b45f-4583-8590-ac09bbe459ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:47.911188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-51e7e55e-b45f-4583-8590-ac09bbe459ef None None] MFA Rules not processed for user `775ff9091a794faaacc05a37550c8a72`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:47.912548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-51e7e55e-b45f-4583-8590-ac09bbe459ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:47.988707 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2013/4095] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:49.070922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010cac5-dfc5-4ca3-9777-3590edde6469 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:49.073029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b010cac5-dfc5-4ca3-9777-3590edde6469 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:49.196550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010cac5-dfc5-4ca3-9777-3590edde6469 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:49.202519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3472b8-56a0-4e61-bfa5-e71c1878b92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:49.203803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3472b8-56a0-4e61-bfa5-e71c1878b92c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:49.204137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3472b8-56a0-4e61-bfa5-e71c1878b92c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:49.204425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3472b8-56a0-4e61-bfa5-e71c1878b92c admin admin] PATH_INFO: `/v3/users/69002d46d4e54eadadf911ec3589d95c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:49.205610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3472b8-56a0-4e61-bfa5-e71c1878b92c admin admin] RBAC: Authorizing `identity:delete_user(user_id=69002d46d4e54eadadf911ec3589d95c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:49.208978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3472b8-56a0-4e61-bfa5-e71c1878b92c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:49.313366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0e3472b8-56a0-4e61-bfa5-e71c1878b92c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69002d46d4e54eadadf911ec3589d95c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:49.323842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0e3472b8-56a0-4e61-bfa5-e71c1878b92c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69002d46d4e54eadadf911ec3589d95c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:49.349258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0e3472b8-56a0-4e61-bfa5-e71c1878b92c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69002d46d4e54eadadf911ec3589d95c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:49.350765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0e3472b8-56a0-4e61-bfa5-e71c1878b92c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69002d46d4e54eadadf911ec3589d95c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:49.372652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0e3472b8-56a0-4e61-bfa5-e71c1878b92c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69002d46d4e54eadadf911ec3589d95c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:49.398369 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2083/4096] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:01:49 2022] DELETE /identity/v3/users/69002d46d4e54eadadf911ec3589d95c => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:50.174876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7e55e-b45f-4583-8590-ac09bbe459ef None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:50.191462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7e55e-b45f-4583-8590-ac09bbe459ef None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:50.282445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f760f4-e584-4d45-8c8a-324c3d7e0e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:50.283204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f760f4-e584-4d45-8c8a-324c3d7e0e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:50.283552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f760f4-e584-4d45-8c8a-324c3d7e0e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:50.284491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f760f4-e584-4d45-8c8a-324c3d7e0e44 admin admin] PATH_INFO: `/v3/projects/d27e9cda0a6d47d4acdbadc890b996c9` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:50.286409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f760f4-e584-4d45-8c8a-324c3d7e0e44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d27e9cda0a6d47d4acdbadc890b996c9)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:50.288512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f760f4-e584-4d45-8c8a-324c3d7e0e44 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:50.375793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-23f760f4-e584-4d45-8c8a-324c3d7e0e44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd27e9cda0a6d47d4acdbadc890b996c9', 'request_id': 'req-23f760f4-e584-4d45-8c8a-324c3d7e0e44'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:50.401285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-23f760f4-e584-4d45-8c8a-324c3d7e0e44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd27e9cda0a6d47d4acdbadc890b996c9', 'request_id': 'req-23f760f4-e584-4d45-8c8a-324c3d7e0e44'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:50.440450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-23f760f4-e584-4d45-8c8a-324c3d7e0e44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd27e9cda0a6d47d4acdbadc890b996c9', 'request_id': 'req-23f760f4-e584-4d45-8c8a-324c3d7e0e44'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:50.476970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-23f760f4-e584-4d45-8c8a-324c3d7e0e44 admin admin] The token cache is being invalidate because project d27e9cda0a6d47d4acdbadc890b996c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:01:50.477384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-23f760f4-e584-4d45-8c8a-324c3d7e0e44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:50.478692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-23f760f4-e584-4d45-8c8a-324c3d7e0e44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:50.481966 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2014/4097] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:01:50 2022] DELETE /identity/v3/projects/d27e9cda0a6d47d4acdbadc890b996c9 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:50.485755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3472b8-56a0-4e61-bfa5-e71c1878b92c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:50.487302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0e3472b8-56a0-4e61-bfa5-e71c1878b92c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:50.510535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3472b8-56a0-4e61-bfa5-e71c1878b92c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:50.615270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:50.663813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac7c26-d017-459a-994f-9b7c823d1d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:50.666321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac7c26-d017-459a-994f-9b7c823d1d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:50.667232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac7c26-d017-459a-994f-9b7c823d1d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:50.671698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac7c26-d017-459a-994f-9b7c823d1d55 admin admin] PATH_INFO: `/v3/users/b7334bfa53c0483e8f5d15c550e0db5a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:50.675639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac7c26-d017-459a-994f-9b7c823d1d55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7334bfa53c0483e8f5d15c550e0db5a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:50.681507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac7c26-d017-459a-994f-9b7c823d1d55 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:50.833318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1bac7c26-d017-459a-994f-9b7c823d1d55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7334bfa53c0483e8f5d15c550e0db5a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:50.860034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1bac7c26-d017-459a-994f-9b7c823d1d55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7334bfa53c0483e8f5d15c550e0db5a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:50.891248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1bac7c26-d017-459a-994f-9b7c823d1d55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7334bfa53c0483e8f5d15c550e0db5a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:50.894703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1bac7c26-d017-459a-994f-9b7c823d1d55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7334bfa53c0483e8f5d15c550e0db5a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:50.940428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1bac7c26-d017-459a-994f-9b7c823d1d55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7334bfa53c0483e8f5d15c550e0db5a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:50.970161 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2084/4098] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:01:50 2022] DELETE /identity/v3/users/b7334bfa53c0483e8f5d15c550e0db5a => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:51.379733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f760f4-e584-4d45-8c8a-324c3d7e0e44 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:51.382367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-23f760f4-e584-4d45-8c8a-324c3d7e0e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:51.426223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f760f4-e584-4d45-8c8a-324c3d7e0e44 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:51.493482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:51.540316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16152eb-be93-472c-bda5-6375189fbe93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:51.541173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16152eb-be93-472c-bda5-6375189fbe93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:51.541461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16152eb-be93-472c-bda5-6375189fbe93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:51.541982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16152eb-be93-472c-bda5-6375189fbe93 admin admin] PATH_INFO: `/v3/projects/29689e74f89d4e7b9b48d5872770ccb9` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:51.543655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16152eb-be93-472c-bda5-6375189fbe93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29689e74f89d4e7b9b48d5872770ccb9)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:51.546166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16152eb-be93-472c-bda5-6375189fbe93 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:51.637674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac7c26-d017-459a-994f-9b7c823d1d55 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:51.639424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1bac7c26-d017-459a-994f-9b7c823d1d55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:51.655456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c16152eb-be93-472c-bda5-6375189fbe93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29689e74f89d4e7b9b48d5872770ccb9', 'request_id': 'req-c16152eb-be93-472c-bda5-6375189fbe93'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:51.676484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c16152eb-be93-472c-bda5-6375189fbe93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29689e74f89d4e7b9b48d5872770ccb9', 'request_id': 'req-c16152eb-be93-472c-bda5-6375189fbe93'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:51.678154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac7c26-d017-459a-994f-9b7c823d1d55 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:51.686093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c16152eb-be93-472c-bda5-6375189fbe93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29689e74f89d4e7b9b48d5872770ccb9', 'request_id': 'req-c16152eb-be93-472c-bda5-6375189fbe93'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:51.709915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c16152eb-be93-472c-bda5-6375189fbe93 admin admin] The token cache is being invalidate because project 29689e74f89d4e7b9b48d5872770ccb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:01:51.710209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c16152eb-be93-472c-bda5-6375189fbe93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:51.713033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c16152eb-be93-472c-bda5-6375189fbe93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:51.721156 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2015/4099] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:01:51 2022] DELETE /identity/v3/projects/29689e74f89d4e7b9b48d5872770ccb9 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:51.725835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf19638-9f16-4a3f-ac90-947ea0a423ea None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:51.726356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf19638-9f16-4a3f-ac90-947ea0a423ea None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:51.728080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf19638-9f16-4a3f-ac90-947ea0a423ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:51.770868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6cf19638-9f16-4a3f-ac90-947ea0a423ea None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:51.772731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6cf19638-9f16-4a3f-ac90-947ea0a423ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:51.809804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1efc2-38ee-43c2-b1cb-fd2e45f69e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:51.810446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1efc2-38ee-43c2-b1cb-fd2e45f69e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:51.810939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1efc2-38ee-43c2-b1cb-fd2e45f69e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:51.811404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1efc2-38ee-43c2-b1cb-fd2e45f69e1b admin admin] PATH_INFO: `/v3/users/6ade3ba047d348a387e8e4077072b90a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:51.812749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1efc2-38ee-43c2-b1cb-fd2e45f69e1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ade3ba047d348a387e8e4077072b90a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:51.814968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1efc2-38ee-43c2-b1cb-fd2e45f69e1b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:51.871254 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2016/4100] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:51.874410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393864cd-56de-436b-a15f-aa56364e7867 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:51.876664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393864cd-56de-436b-a15f-aa56364e7867 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:51.877487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393864cd-56de-436b-a15f-aa56364e7867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:51.900030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dba1efc2-38ee-43c2-b1cb-fd2e45f69e1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ade3ba047d348a387e8e4077072b90a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:51.909896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dba1efc2-38ee-43c2-b1cb-fd2e45f69e1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ade3ba047d348a387e8e4077072b90a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:51.911311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-393864cd-56de-436b-a15f-aa56364e7867 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:51.913075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-393864cd-56de-436b-a15f-aa56364e7867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:51.922213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:51.926763 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2017/4101] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:51.933273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393864cd-56de-436b-a15f-aa56364e7867 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:51.940230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dba1efc2-38ee-43c2-b1cb-fd2e45f69e1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ade3ba047d348a387e8e4077072b90a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:51.941977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dba1efc2-38ee-43c2-b1cb-fd2e45f69e1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ade3ba047d348a387e8e4077072b90a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:51.955012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393864cd-56de-436b-a15f-aa56364e7867 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:51.964510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dba1efc2-38ee-43c2-b1cb-fd2e45f69e1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ade3ba047d348a387e8e4077072b90a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:51.972008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d911e3a-ef97-4350-9fc5-7d0613f84003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:51.974647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d911e3a-ef97-4350-9fc5-7d0613f84003 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:51.977872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d911e3a-ef97-4350-9fc5-7d0613f84003 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:51.977872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d911e3a-ef97-4350-9fc5-7d0613f84003 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:51.979473 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2085/4102] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:01:51 2022] DELETE /identity/v3/users/6ade3ba047d348a387e8e4077072b90a => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:51.980160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d911e3a-ef97-4350-9fc5-7d0613f84003 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:01:51.980733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d911e3a-ef97-4350-9fc5-7d0613f84003 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:51.983058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d911e3a-ef97-4350-9fc5-7d0613f84003 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:52.002863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1efc2-38ee-43c2-b1cb-fd2e45f69e1b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:52.004559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-dba1efc2-38ee-43c2-b1cb-fd2e45f69e1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:52.013751 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2018/4103] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:01:52.115856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1efc2-38ee-43c2-b1cb-fd2e45f69e1b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:52.124355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559f333-559b-49ee-8ed6-6b353542ff07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:52.125508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559f333-559b-49ee-8ed6-6b353542ff07 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:52.125980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559f333-559b-49ee-8ed6-6b353542ff07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:52.126512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559f333-559b-49ee-8ed6-6b353542ff07 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:52.127467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559f333-559b-49ee-8ed6-6b353542ff07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:52.129895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559f333-559b-49ee-8ed6-6b353542ff07 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:52.188362 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2086/4104] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:01:52 2022] POST /identity/v3/projects => generated 403 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:01:52.189751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d911e3a-ef97-4350-9fc5-7d0613f84003 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:52.204491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d911e3a-ef97-4350-9fc5-7d0613f84003 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:52.301183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:52.322148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08729f-ff8b-4a55-bb2e-84dd4b09c9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:52.323229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08729f-ff8b-4a55-bb2e-84dd4b09c9a6 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:52.323734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08729f-ff8b-4a55-bb2e-84dd4b09c9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:52.324362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08729f-ff8b-4a55-bb2e-84dd4b09c9a6 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:52.325230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08729f-ff8b-4a55-bb2e-84dd4b09c9a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:52.327502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08729f-ff8b-4a55-bb2e-84dd4b09c9a6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:52.362912 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2019/4105] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:01:52 2022] POST /identity/v3/users => generated 302 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:01:52.363668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559f333-559b-49ee-8ed6-6b353542ff07 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:52.381295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559f333-559b-49ee-8ed6-6b353542ff07 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:52.389488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cc17b-d3c7-4159-99e9-d72420e1a415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:52.390123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cc17b-d3c7-4159-99e9-d72420e1a415 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:52.390573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cc17b-d3c7-4159-99e9-d72420e1a415 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:52.390915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cc17b-d3c7-4159-99e9-d72420e1a415 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:52.392216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cc17b-d3c7-4159-99e9-d72420e1a415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:52.394011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cc17b-d3c7-4159-99e9-d72420e1a415 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:52.411761 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2087/4106] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:01:52.412188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08729f-ff8b-4a55-bb2e-84dd4b09c9a6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:52.424615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08729f-ff8b-4a55-bb2e-84dd4b09c9a6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:52.431223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818cea74-db04-4915-a6bc-173eb8099a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:52.432631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cea74-db04-4915-a6bc-173eb8099a26 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:52.433071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cea74-db04-4915-a6bc-173eb8099a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:52.433502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cea74-db04-4915-a6bc-173eb8099a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:52.435065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818cea74-db04-4915-a6bc-173eb8099a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:52.437045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818cea74-db04-4915-a6bc-173eb8099a26 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:52.450843 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2020/4107] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:01:52.455168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cc17b-d3c7-4159-99e9-d72420e1a415 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:52.470831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cc17b-d3c7-4159-99e9-d72420e1a415 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:52.479806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f821d-cd0b-444f-87ba-c5a28b14f3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:52.480146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f821d-cd0b-444f-87ba-c5a28b14f3ce admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:52.480588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f821d-cd0b-444f-87ba-c5a28b14f3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:52.480870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f821d-cd0b-444f-87ba-c5a28b14f3ce admin admin] PATH_INFO: `/v3/projects/4211ad493bf14dbd840457299d531cab/users/fcbbafa8ad2149cf8d9778badd73237c/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:52.500701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f821d-cd0b-444f-87ba-c5a28b14f3ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=4211ad493bf14dbd840457299d531cab, user_id=fcbbafa8ad2149cf8d9778badd73237c, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:52.503445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f821d-cd0b-444f-87ba-c5a28b14f3ce admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:52.529956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db120c-17c0-4243-806b-ef05fc1faf8d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:52.530522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db120c-17c0-4243-806b-ef05fc1faf8d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:52.530970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db120c-17c0-4243-806b-ef05fc1faf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:52.534963 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2088/4108] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:01:52 2022] PUT /identity/v3/projects/4211ad493bf14dbd840457299d531cab/users/fcbbafa8ad2149cf8d9778badd73237c/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:52.560945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-42db120c-17c0-4243-806b-ef05fc1faf8d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:52.562893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-42db120c-17c0-4243-806b-ef05fc1faf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:52.701301 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2021/4109] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:52.724886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f821d-cd0b-444f-87ba-c5a28b14f3ce admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:52.729636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7f3f821d-cd0b-444f-87ba-c5a28b14f3ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:52.850058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f821d-cd0b-444f-87ba-c5a28b14f3ce admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:52.859813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566cce0-060e-4020-a879-a781a561b46d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:52.874148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566cce0-060e-4020-a879-a781a561b46d service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:52.875559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566cce0-060e-4020-a879-a781a561b46d service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:52.875948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566cce0-060e-4020-a879-a781a561b46d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:52.905977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566cce0-060e-4020-a879-a781a561b46d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:52.912567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566cce0-060e-4020-a879-a781a561b46d service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:52.941989 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2089/4110] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:01:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:01:53.097565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db120c-17c0-4243-806b-ef05fc1faf8d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:53.099044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-42db120c-17c0-4243-806b-ef05fc1faf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:53.122952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db120c-17c0-4243-806b-ef05fc1faf8d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:53.142796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3a916-134f-46b2-9678-250b292a1e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:53.144741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3a916-134f-46b2-9678-250b292a1e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:53.145335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3a916-134f-46b2-9678-250b292a1e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:53.146096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3a916-134f-46b2-9678-250b292a1e96 admin admin] PATH_INFO: `/v3/projects/04cc84345d874fc69776490bea69d81a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:53.148355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd3a916-134f-46b2-9678-250b292a1e96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04cc84345d874fc69776490bea69d81a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:53.153414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd3a916-134f-46b2-9678-250b292a1e96 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:53.235446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4dd3a916-134f-46b2-9678-250b292a1e96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04cc84345d874fc69776490bea69d81a', 'request_id': 'req-4dd3a916-134f-46b2-9678-250b292a1e96'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:53.259062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4dd3a916-134f-46b2-9678-250b292a1e96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04cc84345d874fc69776490bea69d81a', 'request_id': 'req-4dd3a916-134f-46b2-9678-250b292a1e96'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:53.267197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4dd3a916-134f-46b2-9678-250b292a1e96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04cc84345d874fc69776490bea69d81a', 'request_id': 'req-4dd3a916-134f-46b2-9678-250b292a1e96'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:53.313901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4dd3a916-134f-46b2-9678-250b292a1e96 admin admin] The token cache is being invalidate because project 04cc84345d874fc69776490bea69d81a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:01:53.316129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4dd3a916-134f-46b2-9678-250b292a1e96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:53.317344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4dd3a916-134f-46b2-9678-250b292a1e96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:53.327638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0e6b1-ec1a-44f2-9f0c-c5a3418c641b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:53.328101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0e6b1-ec1a-44f2-9f0c-c5a3418c641b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:53.328471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0e6b1-ec1a-44f2-9f0c-c5a3418c641b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:53.331312 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2022/4111] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:01:53 2022] DELETE /identity/v3/projects/04cc84345d874fc69776490bea69d81a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:53.372180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e1e0e6b1-ec1a-44f2-9f0c-c5a3418c641b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:53.373648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e1e0e6b1-ec1a-44f2-9f0c-c5a3418c641b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:53.415963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:53.472226 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2090/4112] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:53.480331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e9fef-ef46-41ff-8ef6-0a170a23d9de None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:53.481325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e9fef-ef46-41ff-8ef6-0a170a23d9de None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:53.481559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e9fef-ef46-41ff-8ef6-0a170a23d9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:53.509551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b07e9fef-ef46-41ff-8ef6-0a170a23d9de None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:53.511128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b07e9fef-ef46-41ff-8ef6-0a170a23d9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:53.522219 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2023/4113] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:01:53.526369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0e6b1-ec1a-44f2-9f0c-c5a3418c641b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:53.546251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0e6b1-ec1a-44f2-9f0c-c5a3418c641b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:53.575829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb2ee8-d72e-43e0-82c5-5a4dea54979e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:53.578592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb2ee8-d72e-43e0-82c5-5a4dea54979e admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:53.579598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb2ee8-d72e-43e0-82c5-5a4dea54979e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:53.580361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb2ee8-d72e-43e0-82c5-5a4dea54979e admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:53.583818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb2ee8-d72e-43e0-82c5-5a4dea54979e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:01:53.585295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb2ee8-d72e-43e0-82c5-5a4dea54979e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:53.591272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb2ee8-d72e-43e0-82c5-5a4dea54979e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:53.623236 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2091/4114] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:01:53.626995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e9fef-ef46-41ff-8ef6-0a170a23d9de None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:53.643747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e9fef-ef46-41ff-8ef6-0a170a23d9de None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:53.659297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614e9ee-6f61-423a-a8e9-94ad68157c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:53.660486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614e9ee-6f61-423a-a8e9-94ad68157c40 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:53.660486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614e9ee-6f61-423a-a8e9-94ad68157c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:53.660776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614e9ee-6f61-423a-a8e9-94ad68157c40 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:53.661848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614e9ee-6f61-423a-a8e9-94ad68157c40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:53.663809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614e9ee-6f61-423a-a8e9-94ad68157c40 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:53.739488 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2024/4115] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:01:53 2022] POST /identity/v3/projects => generated 393 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:01:53.744601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb2ee8-d72e-43e0-82c5-5a4dea54979e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:53.757599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb2ee8-d72e-43e0-82c5-5a4dea54979e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:53.915267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1674a7-67fe-477e-9f70-44da5d3daa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:53.915787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1674a7-67fe-477e-9f70-44da5d3daa98 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:53.916133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1674a7-67fe-477e-9f70-44da5d3daa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:53.916534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1674a7-67fe-477e-9f70-44da5d3daa98 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:53.917406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1674a7-67fe-477e-9f70-44da5d3daa98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:53.919667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1674a7-67fe-477e-9f70-44da5d3daa98 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:53.959700 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2092/4116] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:01:53 2022] POST /identity/v3/users => generated 297 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:01:53.965515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614e9ee-6f61-423a-a8e9-94ad68157c40 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:53.987138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614e9ee-6f61-423a-a8e9-94ad68157c40 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:53.993571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485fbc9-02b7-4eed-9078-e07642dcf4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:53.994317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485fbc9-02b7-4eed-9078-e07642dcf4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:53.994700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485fbc9-02b7-4eed-9078-e07642dcf4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:53.995060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485fbc9-02b7-4eed-9078-e07642dcf4e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:53.995811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485fbc9-02b7-4eed-9078-e07642dcf4e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:53.998831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485fbc9-02b7-4eed-9078-e07642dcf4e3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:54.029797 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2025/4117] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:01:54.046198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1674a7-67fe-477e-9f70-44da5d3daa98 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:54.062686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1674a7-67fe-477e-9f70-44da5d3daa98 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:54.075795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39193c1a-88ab-4368-8b16-01023c2ff502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:54.076437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39193c1a-88ab-4368-8b16-01023c2ff502 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:54.077049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39193c1a-88ab-4368-8b16-01023c2ff502 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:54.077397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39193c1a-88ab-4368-8b16-01023c2ff502 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:54.078195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39193c1a-88ab-4368-8b16-01023c2ff502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:54.080195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39193c1a-88ab-4368-8b16-01023c2ff502 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:54.099036 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2093/4118] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:01:54.100357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485fbc9-02b7-4eed-9078-e07642dcf4e3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:54.117726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485fbc9-02b7-4eed-9078-e07642dcf4e3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:54.139430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7097f3c-22cc-4842-9247-19da4ba3487a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:54.140108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7097f3c-22cc-4842-9247-19da4ba3487a admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:54.141530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7097f3c-22cc-4842-9247-19da4ba3487a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:54.141530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7097f3c-22cc-4842-9247-19da4ba3487a admin admin] PATH_INFO: `/v3/projects/2c8d44c4bbfc44a297c82cda39868ddd/users/5a92405f06244dfeba65a9cc22bdaa2e/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:54.156634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:54.177762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7097f3c-22cc-4842-9247-19da4ba3487a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c8d44c4bbfc44a297c82cda39868ddd, user_id=5a92405f06244dfeba65a9cc22bdaa2e, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:54.180852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7097f3c-22cc-4842-9247-19da4ba3487a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:54.212462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b067e-1cfb-473d-b23f-6bec6602e69d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:54.212807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b067e-1cfb-473d-b23f-6bec6602e69d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:54.213006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b067e-1cfb-473d-b23f-6bec6602e69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:54.213209 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2026/4119] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:01:54 2022] PUT /identity/v3/projects/2c8d44c4bbfc44a297c82cda39868ddd/users/5a92405f06244dfeba65a9cc22bdaa2e/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:01:54.248818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-474b067e-1cfb-473d-b23f-6bec6602e69d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:01:54.250617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-474b067e-1cfb-473d-b23f-6bec6602e69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:54.368105 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2094/4120] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:01:54.390327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7097f3c-22cc-4842-9247-19da4ba3487a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:54.392543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f7097f3c-22cc-4842-9247-19da4ba3487a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:54.518981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7097f3c-22cc-4842-9247-19da4ba3487a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:54.530242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00b324-18f7-4f17-9116-d283584402cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:54.537718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00b324-18f7-4f17-9116-d283584402cd service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:54.538353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00b324-18f7-4f17-9116-d283584402cd service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:54.538875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00b324-18f7-4f17-9116-d283584402cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:54.556447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00b324-18f7-4f17-9116-d283584402cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:54.558206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00b324-18f7-4f17-9116-d283584402cd service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:54.586181 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2027/4121] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:01:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:01:59.220235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b067e-1cfb-473d-b23f-6bec6602e69d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:01:59.221226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-474b067e-1cfb-473d-b23f-6bec6602e69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:01:59.243638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b067e-1cfb-473d-b23f-6bec6602e69d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:01:59.249943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2560a-a5cd-4def-9f24-acb5529cb0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:01:59.252011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2560a-a5cd-4def-9f24-acb5529cb0e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:01:59.252429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2560a-a5cd-4def-9f24-acb5529cb0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:01:59.252819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2560a-a5cd-4def-9f24-acb5529cb0e9 admin admin] PATH_INFO: `/v3/users/ce758fa3950048549fbc375a80deb1a7` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:01:59.254310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2560a-a5cd-4def-9f24-acb5529cb0e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce758fa3950048549fbc375a80deb1a7)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:01:59.256546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2560a-a5cd-4def-9f24-acb5529cb0e9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:01:59.353081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:59.379973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-95f2560a-a5cd-4def-9f24-acb5529cb0e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce758fa3950048549fbc375a80deb1a7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:59.406452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-95f2560a-a5cd-4def-9f24-acb5529cb0e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce758fa3950048549fbc375a80deb1a7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:59.429177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-95f2560a-a5cd-4def-9f24-acb5529cb0e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce758fa3950048549fbc375a80deb1a7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:59.430458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-95f2560a-a5cd-4def-9f24-acb5529cb0e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce758fa3950048549fbc375a80deb1a7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:59.447023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-95f2560a-a5cd-4def-9f24-acb5529cb0e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce758fa3950048549fbc375a80deb1a7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:01:59.461073 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2095/4122] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:01:59 2022] DELETE /identity/v3/users/ce758fa3950048549fbc375a80deb1a7 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:00.246983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00b324-18f7-4f17-9116-d283584402cd service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:00.260965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00b324-18f7-4f17-9116-d283584402cd service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:00.413422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dde0eb-2f69-4d71-a711-f8fc6e66adf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:00.414310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dde0eb-2f69-4d71-a711-f8fc6e66adf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:00.415201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dde0eb-2f69-4d71-a711-f8fc6e66adf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:00.416328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dde0eb-2f69-4d71-a711-f8fc6e66adf5 admin admin] PATH_INFO: `/v3/projects/91d32a0fa04648de8ec34b7937b36ef3` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:00.417827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dde0eb-2f69-4d71-a711-f8fc6e66adf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91d32a0fa04648de8ec34b7937b36ef3)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:00.430640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dde0eb-2f69-4d71-a711-f8fc6e66adf5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:00.529147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-65dde0eb-2f69-4d71-a711-f8fc6e66adf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91d32a0fa04648de8ec34b7937b36ef3', 'request_id': 'req-65dde0eb-2f69-4d71-a711-f8fc6e66adf5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:00.551234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-65dde0eb-2f69-4d71-a711-f8fc6e66adf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91d32a0fa04648de8ec34b7937b36ef3', 'request_id': 'req-65dde0eb-2f69-4d71-a711-f8fc6e66adf5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:00.560748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-65dde0eb-2f69-4d71-a711-f8fc6e66adf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91d32a0fa04648de8ec34b7937b36ef3', 'request_id': 'req-65dde0eb-2f69-4d71-a711-f8fc6e66adf5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:00.604894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-65dde0eb-2f69-4d71-a711-f8fc6e66adf5 admin admin] The token cache is being invalidate because project 91d32a0fa04648de8ec34b7937b36ef3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:02:00.605190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-65dde0eb-2f69-4d71-a711-f8fc6e66adf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:00.605875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-65dde0eb-2f69-4d71-a711-f8fc6e66adf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:00.607703 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2028/4123] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:02:00 2022] DELETE /identity/v3/projects/91d32a0fa04648de8ec34b7937b36ef3 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:00.612409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785236bf-4cc4-48c9-bee0-8f4dbef041ba None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:00.612942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785236bf-4cc4-48c9-bee0-8f4dbef041ba None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:00.613044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785236bf-4cc4-48c9-bee0-8f4dbef041ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:00.641119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-785236bf-4cc4-48c9-bee0-8f4dbef041ba None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:00.642520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-785236bf-4cc4-48c9-bee0-8f4dbef041ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:00.758311 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2096/4124] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:00.766809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7184040-9138-4936-83b6-717b1829fc7f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:00.767694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7184040-9138-4936-83b6-717b1829fc7f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:00.768425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7184040-9138-4936-83b6-717b1829fc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:00.802468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b7184040-9138-4936-83b6-717b1829fc7f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:00.803712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b7184040-9138-4936-83b6-717b1829fc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:00.826727 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2029/4125] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:00.828379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785236bf-4cc4-48c9-bee0-8f4dbef041ba None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:00.848428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785236bf-4cc4-48c9-bee0-8f4dbef041ba None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:00.854660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666be16-f611-4def-8737-c34a06f7aea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:00.855593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666be16-f611-4def-8737-c34a06f7aea3 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:00.856080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666be16-f611-4def-8737-c34a06f7aea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:00.856550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666be16-f611-4def-8737-c34a06f7aea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:00.857405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666be16-f611-4def-8737-c34a06f7aea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:02:00.857905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666be16-f611-4def-8737-c34a06f7aea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:00.859764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666be16-f611-4def-8737-c34a06f7aea3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:00.873433 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2097/4126] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:02:00.878150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7184040-9138-4936-83b6-717b1829fc7f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:00.888409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7184040-9138-4936-83b6-717b1829fc7f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:00.894546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc81fd-ef8c-4a89-92fd-a04148b926a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:00.895221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc81fd-ef8c-4a89-92fd-a04148b926a3 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:00.895508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc81fd-ef8c-4a89-92fd-a04148b926a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:00.895776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc81fd-ef8c-4a89-92fd-a04148b926a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:00.896828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc81fd-ef8c-4a89-92fd-a04148b926a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:00.899329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc81fd-ef8c-4a89-92fd-a04148b926a3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:00.911083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:00.955274 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2030/4127] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:02:00 2022] POST /identity/v3/projects => generated 367 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:02:00.956461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666be16-f611-4def-8737-c34a06f7aea3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:00.977577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666be16-f611-4def-8737-c34a06f7aea3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:01.075381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0075d-9c3b-44a2-993b-9673996d0404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:01.076095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0075d-9c3b-44a2-993b-9673996d0404 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:01.076441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0075d-9c3b-44a2-993b-9673996d0404 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:01.076809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0075d-9c3b-44a2-993b-9673996d0404 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:01.077528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0075d-9c3b-44a2-993b-9673996d0404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:01.079900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0075d-9c3b-44a2-993b-9673996d0404 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:01.115928 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2098/4128] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:02:00 2022] POST /identity/v3/users => generated 284 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:02:01.116748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc81fd-ef8c-4a89-92fd-a04148b926a3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:01.127804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc81fd-ef8c-4a89-92fd-a04148b926a3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:01.136126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831c7cf-df91-43e2-af49-510d52f30f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:01.136975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831c7cf-df91-43e2-af49-510d52f30f78 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:01.137474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831c7cf-df91-43e2-af49-510d52f30f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:01.137944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831c7cf-df91-43e2-af49-510d52f30f78 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:01.138800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831c7cf-df91-43e2-af49-510d52f30f78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:01.141295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831c7cf-df91-43e2-af49-510d52f30f78 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:01.157205 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2031/4129] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:02:01.158455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0075d-9c3b-44a2-993b-9673996d0404 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:01.174145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0075d-9c3b-44a2-993b-9673996d0404 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:01.182561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c21aa6-6b46-43a8-a47d-a110a721ade3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:01.183603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c21aa6-6b46-43a8-a47d-a110a721ade3 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:01.184116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c21aa6-6b46-43a8-a47d-a110a721ade3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:01.184779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c21aa6-6b46-43a8-a47d-a110a721ade3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:01.185728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c21aa6-6b46-43a8-a47d-a110a721ade3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:01.188103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c21aa6-6b46-43a8-a47d-a110a721ade3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:01.202552 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2099/4130] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:02:01.206535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831c7cf-df91-43e2-af49-510d52f30f78 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:01.217428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831c7cf-df91-43e2-af49-510d52f30f78 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:01.223079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8f45c-b99b-4914-a7ea-3b5661eb6ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:01.224062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8f45c-b99b-4914-a7ea-3b5661eb6ad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:01.224808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8f45c-b99b-4914-a7ea-3b5661eb6ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:01.225336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8f45c-b99b-4914-a7ea-3b5661eb6ad5 admin admin] PATH_INFO: `/v3/projects/4ae7982f0b4349c6ab1c445988ea4f9f/users/221b665fc7cb4758a6e35cc81c6a4d43/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:01.249385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8f45c-b99b-4914-a7ea-3b5661eb6ad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ae7982f0b4349c6ab1c445988ea4f9f, user_id=221b665fc7cb4758a6e35cc81c6a4d43, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:01.253197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8f45c-b99b-4914-a7ea-3b5661eb6ad5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:01.273247 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2032/4131] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:02:01 2022] PUT /identity/v3/projects/4ae7982f0b4349c6ab1c445988ea4f9f/users/221b665fc7cb4758a6e35cc81c6a4d43/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:01.278726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e27b1-61db-4461-9016-08d5420adcd1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:01.279408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e27b1-61db-4461-9016-08d5420adcd1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:01.279968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e27b1-61db-4461-9016-08d5420adcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:01.324481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6f2e27b1-61db-4461-9016-08d5420adcd1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:01.328489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6f2e27b1-61db-4461-9016-08d5420adcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:01.345252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:01.430865 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2100/4132] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:01.496248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8f45c-b99b-4914-a7ea-3b5661eb6ad5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:01.498525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3cb8f45c-b99b-4914-a7ea-3b5661eb6ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:01.571850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8f45c-b99b-4914-a7ea-3b5661eb6ad5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:01.577684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16181e34-0611-4c89-aa63-14d8e738f180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:01.578933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16181e34-0611-4c89-aa63-14d8e738f180 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:01.580407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16181e34-0611-4c89-aa63-14d8e738f180 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:01.581696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16181e34-0611-4c89-aa63-14d8e738f180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:01.597104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16181e34-0611-4c89-aa63-14d8e738f180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:01.599809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16181e34-0611-4c89-aa63-14d8e738f180 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:01.615136 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2033/4133] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:02:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:02:02.571938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503b350-1764-417b-8272-837cf854ccb5 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:02.572808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503b350-1764-417b-8272-837cf854ccb5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:02.573249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503b350-1764-417b-8272-837cf854ccb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:02.606708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a503b350-1764-417b-8272-837cf854ccb5 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:02.608015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a503b350-1764-417b-8272-837cf854ccb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:02.794387 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2101/4134] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:02:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:02.820082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16181e34-0611-4c89-aa63-14d8e738f180 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:02.821579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-16181e34-0611-4c89-aa63-14d8e738f180 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:03.059567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16181e34-0611-4c89-aa63-14d8e738f180 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:03.073487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f383ce-5c36-4dda-8093-73fac375b22e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:03.074152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f383ce-5c36-4dda-8093-73fac375b22e service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:03.074436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f383ce-5c36-4dda-8093-73fac375b22e service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:03.074754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f383ce-5c36-4dda-8093-73fac375b22e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:03.092325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f383ce-5c36-4dda-8093-73fac375b22e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:03.094232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f383ce-5c36-4dda-8093-73fac375b22e service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:03.112423 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2034/4135] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:02:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:02:05.080057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a715bfe-ca60-4a52-a238-c489125f48a2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:05.080577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a715bfe-ca60-4a52-a238-c489125f48a2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:05.080762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a715bfe-ca60-4a52-a238-c489125f48a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:05.156128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5a715bfe-ca60-4a52-a238-c489125f48a2 None None] MFA Rules not processed for user `fcbbafa8ad2149cf8d9778badd73237c`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:05.157291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5a715bfe-ca60-4a52-a238-c489125f48a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:05.323953 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2102/4136] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:05 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:05.935160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f383ce-5c36-4dda-8093-73fac375b22e service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:05.936505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-45f383ce-5c36-4dda-8093-73fac375b22e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:06.022946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:06.090718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f383ce-5c36-4dda-8093-73fac375b22e service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:06.104295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8ad21-6d23-4a7f-9fa0-23fc4df80845 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:06.104961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8ad21-6d23-4a7f-9fa0-23fc4df80845 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:06.105235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8ad21-6d23-4a7f-9fa0-23fc4df80845 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:06.105508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8ad21-6d23-4a7f-9fa0-23fc4df80845 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:06.126082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8ad21-6d23-4a7f-9fa0-23fc4df80845 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:06.127932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8ad21-6d23-4a7f-9fa0-23fc4df80845 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:06.156166 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2035/4137] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:02:05 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:02:15.311466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=75 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:15.332718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a715bfe-ca60-4a52-a238-c489125f48a2 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:15.334419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5a715bfe-ca60-4a52-a238-c489125f48a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:15.356680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a715bfe-ca60-4a52-a238-c489125f48a2 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:15.370377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b32eb3-7363-4ea5-acdd-738f76aa0a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:15.371087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b32eb3-7363-4ea5-acdd-738f76aa0a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:15.371363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b32eb3-7363-4ea5-acdd-738f76aa0a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:15.371650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b32eb3-7363-4ea5-acdd-738f76aa0a5a admin admin] PATH_INFO: `/v3/users/e84a109e337d450698e630acfb49f9a5` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:15.373901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b32eb3-7363-4ea5-acdd-738f76aa0a5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e84a109e337d450698e630acfb49f9a5)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:15.378711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b32eb3-7363-4ea5-acdd-738f76aa0a5a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:15.493579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-13b32eb3-7363-4ea5-acdd-738f76aa0a5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e84a109e337d450698e630acfb49f9a5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:15.521906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-13b32eb3-7363-4ea5-acdd-738f76aa0a5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e84a109e337d450698e630acfb49f9a5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:15.560235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-13b32eb3-7363-4ea5-acdd-738f76aa0a5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e84a109e337d450698e630acfb49f9a5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:15.560997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-13b32eb3-7363-4ea5-acdd-738f76aa0a5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e84a109e337d450698e630acfb49f9a5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:15.575264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-13b32eb3-7363-4ea5-acdd-738f76aa0a5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e84a109e337d450698e630acfb49f9a5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:15.595707 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2103/4138] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:02:15 2022] DELETE /identity/v3/users/e84a109e337d450698e630acfb49f9a5 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:16.110476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8ad21-6d23-4a7f-9fa0-23fc4df80845 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:16.153300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8ad21-6d23-4a7f-9fa0-23fc4df80845 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:16.317074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99542e5b-bcee-4a1d-b8ad-2b743b62367c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:16.318130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99542e5b-bcee-4a1d-b8ad-2b743b62367c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:16.318887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99542e5b-bcee-4a1d-b8ad-2b743b62367c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:16.319809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99542e5b-bcee-4a1d-b8ad-2b743b62367c admin admin] PATH_INFO: `/v3/projects/9566fa9a44f94c4ea3081a1a2e658de0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:16.324403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99542e5b-bcee-4a1d-b8ad-2b743b62367c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9566fa9a44f94c4ea3081a1a2e658de0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:16.326679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99542e5b-bcee-4a1d-b8ad-2b743b62367c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:16.418108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-99542e5b-bcee-4a1d-b8ad-2b743b62367c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9566fa9a44f94c4ea3081a1a2e658de0', 'request_id': 'req-99542e5b-bcee-4a1d-b8ad-2b743b62367c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:16.443143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-99542e5b-bcee-4a1d-b8ad-2b743b62367c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9566fa9a44f94c4ea3081a1a2e658de0', 'request_id': 'req-99542e5b-bcee-4a1d-b8ad-2b743b62367c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:16.453105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-99542e5b-bcee-4a1d-b8ad-2b743b62367c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9566fa9a44f94c4ea3081a1a2e658de0', 'request_id': 'req-99542e5b-bcee-4a1d-b8ad-2b743b62367c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:16.482136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-99542e5b-bcee-4a1d-b8ad-2b743b62367c admin admin] The token cache is being invalidate because project 9566fa9a44f94c4ea3081a1a2e658de0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:02:16.482884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-99542e5b-bcee-4a1d-b8ad-2b743b62367c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:16.484758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-99542e5b-bcee-4a1d-b8ad-2b743b62367c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:16.488146 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2036/4139] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:02:16 2022] DELETE /identity/v3/projects/9566fa9a44f94c4ea3081a1a2e658de0 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:16.494002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fd4b8-15a2-4476-8f46-36996a4bfc37 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:16.494986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fd4b8-15a2-4476-8f46-36996a4bfc37 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:16.495427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fd4b8-15a2-4476-8f46-36996a4bfc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:16.529143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2c7fd4b8-15a2-4476-8f46-36996a4bfc37 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:16.530831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2c7fd4b8-15a2-4476-8f46-36996a4bfc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:16.587808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e66f87-ff4e-49bc-80bc-e81514099896 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:16.588096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e66f87-ff4e-49bc-80bc-e81514099896 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:16.588376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e66f87-ff4e-49bc-80bc-e81514099896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:16.677593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b5e66f87-ff4e-49bc-80bc-e81514099896 None None] MFA Rules not processed for user `5a92405f06244dfeba65a9cc22bdaa2e`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:16.678881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b5e66f87-ff4e-49bc-80bc-e81514099896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:16.687783 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2104/4140] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:16.692566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ce791-9755-4c61-a56f-94ecf3cec722 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:16.692908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ce791-9755-4c61-a56f-94ecf3cec722 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:16.693182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ce791-9755-4c61-a56f-94ecf3cec722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:16.743044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-098ce791-9755-4c61-a56f-94ecf3cec722 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:16.744274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-098ce791-9755-4c61-a56f-94ecf3cec722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:16.767471 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2105/4141] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:16.769094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ce791-9755-4c61-a56f-94ecf3cec722 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:16.792552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ce791-9755-4c61-a56f-94ecf3cec722 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:16.804861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52cd7e-4882-4fae-a77b-9d19012fc7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:16.805931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52cd7e-4882-4fae-a77b-9d19012fc7cd admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:16.808567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52cd7e-4882-4fae-a77b-9d19012fc7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:16.809061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52cd7e-4882-4fae-a77b-9d19012fc7cd admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:16.810942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52cd7e-4882-4fae-a77b-9d19012fc7cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:02:16.811435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52cd7e-4882-4fae-a77b-9d19012fc7cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:16.813327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52cd7e-4882-4fae-a77b-9d19012fc7cd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:16.816873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:16.854846 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2106/4142] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:02:16.856039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52cd7e-4882-4fae-a77b-9d19012fc7cd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:16.879099 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2037/4143] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:02:16.880405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd29995-ef09-434e-88c4-971b6e5f6ecd None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:16.880775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd29995-ef09-434e-88c4-971b6e5f6ecd None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:16.881187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd29995-ef09-434e-88c4-971b6e5f6ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:16.881999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52cd7e-4882-4fae-a77b-9d19012fc7cd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:16.895437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64466ff9-bc67-4d6b-b0b5-11dd1e0031d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:16.896067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64466ff9-bc67-4d6b-b0b5-11dd1e0031d7 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:16.896341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64466ff9-bc67-4d6b-b0b5-11dd1e0031d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:16.896806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64466ff9-bc67-4d6b-b0b5-11dd1e0031d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:16.897837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64466ff9-bc67-4d6b-b0b5-11dd1e0031d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:16.904449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64466ff9-bc67-4d6b-b0b5-11dd1e0031d7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:16.925880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fbd29995-ef09-434e-88c4-971b6e5f6ecd None None] MFA Rules not processed for user `5a92405f06244dfeba65a9cc22bdaa2e`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:16.929928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fbd29995-ef09-434e-88c4-971b6e5f6ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:16.957968 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2038/4144] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:02:16.980947 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2107/4145] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:02:16 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:02:16.981508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd29995-ef09-434e-88c4-971b6e5f6ecd None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:16.997533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd29995-ef09-434e-88c4-971b6e5f6ecd None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:17.151296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486f4d7-9483-469e-afe5-643d52c84743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:17.151934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486f4d7-9483-469e-afe5-643d52c84743 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:17.152250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486f4d7-9483-469e-afe5-643d52c84743 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:17.152487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486f4d7-9483-469e-afe5-643d52c84743 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:17.153124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486f4d7-9483-469e-afe5-643d52c84743 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:17.161560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486f4d7-9483-469e-afe5-643d52c84743 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:17.215755 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2039/4146] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:02:16 2022] POST /identity/v3/users => generated 294 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:02:17.219611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64466ff9-bc67-4d6b-b0b5-11dd1e0031d7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:17.238886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64466ff9-bc67-4d6b-b0b5-11dd1e0031d7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:17.256860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd04153-985a-4971-98ab-32ad9def9a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:17.258687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd04153-985a-4971-98ab-32ad9def9a6b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:17.258995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd04153-985a-4971-98ab-32ad9def9a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:17.259277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd04153-985a-4971-98ab-32ad9def9a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:17.259970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd04153-985a-4971-98ab-32ad9def9a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:17.267966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd04153-985a-4971-98ab-32ad9def9a6b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:17.295930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486f4d7-9483-469e-afe5-643d52c84743 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:17.299131 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2108/4147] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:02:17.354612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486f4d7-9483-469e-afe5-643d52c84743 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:17.371590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08dde91-4616-4c73-9daf-c2d2d5454a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:17.372234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08dde91-4616-4c73-9daf-c2d2d5454a81 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:17.373089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08dde91-4616-4c73-9daf-c2d2d5454a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:17.373089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08dde91-4616-4c73-9daf-c2d2d5454a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:17.373445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08dde91-4616-4c73-9daf-c2d2d5454a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:17.375166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08dde91-4616-4c73-9daf-c2d2d5454a81 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:17.399479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd04153-985a-4971-98ab-32ad9def9a6b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:17.411153 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2040/4148] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:02:17.412398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd04153-985a-4971-98ab-32ad9def9a6b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:17.447102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f60b4-017f-442c-9325-b1fbf5d6c655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:17.448069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f60b4-017f-442c-9325-b1fbf5d6c655 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:17.451463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f60b4-017f-442c-9325-b1fbf5d6c655 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:17.452023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f60b4-017f-442c-9325-b1fbf5d6c655 admin admin] PATH_INFO: `/v3/projects/c59393807d9d41ed9a7903a6e0b55755/users/d4cb0ffdb0ef42c18656309683ab0553/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:17.486385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f60b4-017f-442c-9325-b1fbf5d6c655 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c59393807d9d41ed9a7903a6e0b55755, user_id=d4cb0ffdb0ef42c18656309683ab0553, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:17.489681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f60b4-017f-442c-9325-b1fbf5d6c655 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:17.493682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:17.521300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca018c-6a69-44a8-9b6d-650ab1b2dd64 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:17.521781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca018c-6a69-44a8-9b6d-650ab1b2dd64 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:17.522325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca018c-6a69-44a8-9b6d-650ab1b2dd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:17.530586 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2109/4149] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:02:17 2022] PUT /identity/v3/projects/c59393807d9d41ed9a7903a6e0b55755/users/d4cb0ffdb0ef42c18656309683ab0553/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:17.551389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-11ca018c-6a69-44a8-9b6d-650ab1b2dd64 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:17.553027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-11ca018c-6a69-44a8-9b6d-650ab1b2dd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:17.703471 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2041/4150] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:17.716465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f60b4-017f-442c-9325-b1fbf5d6c655 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:17.719178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-509f60b4-017f-442c-9325-b1fbf5d6c655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:17.854792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f60b4-017f-442c-9325-b1fbf5d6c655 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:17.859110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce518e-47d5-43a4-9f3d-1473aeb09692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:17.860019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce518e-47d5-43a4-9f3d-1473aeb09692 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:17.860574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce518e-47d5-43a4-9f3d-1473aeb09692 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:17.861033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce518e-47d5-43a4-9f3d-1473aeb09692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:17.885631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce518e-47d5-43a4-9f3d-1473aeb09692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:17.887291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce518e-47d5-43a4-9f3d-1473aeb09692 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:17.913101 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2110/4151] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:02:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:02:18.993646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca018c-6a69-44a8-9b6d-650ab1b2dd64 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:18.995850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-11ca018c-6a69-44a8-9b6d-650ab1b2dd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:19.221226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:19.248632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca018c-6a69-44a8-9b6d-650ab1b2dd64 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:19.269432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39b113-dfe9-4d8b-9df1-70fafdd2c245 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:19.270207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39b113-dfe9-4d8b-9df1-70fafdd2c245 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:19.273335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39b113-dfe9-4d8b-9df1-70fafdd2c245 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:19.273702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39b113-dfe9-4d8b-9df1-70fafdd2c245 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:19.470410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39b113-dfe9-4d8b-9df1-70fafdd2c245 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:19.472375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39b113-dfe9-4d8b-9df1-70fafdd2c245 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:19.505132 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2042/4152] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:02:18 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:02:22.093497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6427d-ea57-4da6-8919-b1841c461a84 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:22.093972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6427d-ea57-4da6-8919-b1841c461a84 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:22.094191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6427d-ea57-4da6-8919-b1841c461a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:22.174109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-90e6427d-ea57-4da6-8919-b1841c461a84 None None] MFA Rules not processed for user `221b665fc7cb4758a6e35cc81c6a4d43`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:22.186525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-90e6427d-ea57-4da6-8919-b1841c461a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:22.494736 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2111/4153] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:25.201007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39b113-dfe9-4d8b-9df1-70fafdd2c245 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:25.227080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39b113-dfe9-4d8b-9df1-70fafdd2c245 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:25.248334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ef5f9-0bba-4e8b-8daa-2b5bac790c07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:25.248977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ef5f9-0bba-4e8b-8daa-2b5bac790c07 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:25.249256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ef5f9-0bba-4e8b-8daa-2b5bac790c07 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:25.249530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ef5f9-0bba-4e8b-8daa-2b5bac790c07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:25.279077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ef5f9-0bba-4e8b-8daa-2b5bac790c07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:25.281605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ef5f9-0bba-4e8b-8daa-2b5bac790c07 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:25.324874 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2043/4154] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:02:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:02:27.305143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6427d-ea57-4da6-8919-b1841c461a84 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:27.308229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-90e6427d-ea57-4da6-8919-b1841c461a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:27.348113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6427d-ea57-4da6-8919-b1841c461a84 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:27.364586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccad40c-325d-4748-90db-90b0fa7fb4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:27.368275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccad40c-325d-4748-90db-90b0fa7fb4ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:27.368778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccad40c-325d-4748-90db-90b0fa7fb4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:27.374381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccad40c-325d-4748-90db-90b0fa7fb4ec admin admin] PATH_INFO: `/v3/users/fcbbafa8ad2149cf8d9778badd73237c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:27.378321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccad40c-325d-4748-90db-90b0fa7fb4ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcbbafa8ad2149cf8d9778badd73237c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:27.384876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccad40c-325d-4748-90db-90b0fa7fb4ec admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:27.539986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eccad40c-325d-4748-90db-90b0fa7fb4ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcbbafa8ad2149cf8d9778badd73237c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:27.577021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eccad40c-325d-4748-90db-90b0fa7fb4ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcbbafa8ad2149cf8d9778badd73237c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:27.611544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eccad40c-325d-4748-90db-90b0fa7fb4ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcbbafa8ad2149cf8d9778badd73237c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:27.615085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eccad40c-325d-4748-90db-90b0fa7fb4ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcbbafa8ad2149cf8d9778badd73237c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:27.651753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eccad40c-325d-4748-90db-90b0fa7fb4ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcbbafa8ad2149cf8d9778badd73237c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:27.660221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:27.682412 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2112/4155] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:02:27 2022] DELETE /identity/v3/users/fcbbafa8ad2149cf8d9778badd73237c => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:28.444726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ef5f9-0bba-4e8b-8daa-2b5bac790c07 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:28.465145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ef5f9-0bba-4e8b-8daa-2b5bac790c07 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:28.743242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48801db5-201a-4faf-854b-86250db95415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:28.743988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48801db5-201a-4faf-854b-86250db95415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:28.744456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48801db5-201a-4faf-854b-86250db95415 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:28.759296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48801db5-201a-4faf-854b-86250db95415 admin admin] PATH_INFO: `/v3/projects/4211ad493bf14dbd840457299d531cab` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:28.765518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48801db5-201a-4faf-854b-86250db95415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4211ad493bf14dbd840457299d531cab)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:28.768010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48801db5-201a-4faf-854b-86250db95415 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:28.917084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-48801db5-201a-4faf-854b-86250db95415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4211ad493bf14dbd840457299d531cab', 'request_id': 'req-48801db5-201a-4faf-854b-86250db95415'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:28.933633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-48801db5-201a-4faf-854b-86250db95415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4211ad493bf14dbd840457299d531cab', 'request_id': 'req-48801db5-201a-4faf-854b-86250db95415'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:28.950365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-48801db5-201a-4faf-854b-86250db95415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4211ad493bf14dbd840457299d531cab', 'request_id': 'req-48801db5-201a-4faf-854b-86250db95415'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:28.988127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-48801db5-201a-4faf-854b-86250db95415 admin admin] The token cache is being invalidate because project 4211ad493bf14dbd840457299d531cab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:02:28.988668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-48801db5-201a-4faf-854b-86250db95415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:28.989481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-48801db5-201a-4faf-854b-86250db95415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:28.991672 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2044/4156] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:02:28 2022] DELETE /identity/v3/projects/4211ad493bf14dbd840457299d531cab => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:28.996612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7224e3e-8a3e-436f-8ee1-289bf83c5d60 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:28.997098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7224e3e-8a3e-436f-8ee1-289bf83c5d60 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:28.997182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7224e3e-8a3e-436f-8ee1-289bf83c5d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:29.047975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b7224e3e-8a3e-436f-8ee1-289bf83c5d60 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:29.055749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b7224e3e-8a3e-436f-8ee1-289bf83c5d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:29.223640 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2113/4157] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:29.232685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68c95c-27fc-4621-af97-90505f97c136 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:29.233082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68c95c-27fc-4621-af97-90505f97c136 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:29.233963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68c95c-27fc-4621-af97-90505f97c136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:29.296048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fe68c95c-27fc-4621-af97-90505f97c136 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:29.309395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fe68c95c-27fc-4621-af97-90505f97c136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:29.323622 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2045/4158] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:29.328088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7224e3e-8a3e-436f-8ee1-289bf83c5d60 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:29.344020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7224e3e-8a3e-436f-8ee1-289bf83c5d60 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:29.362755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483117f-0376-4e64-94e3-01eef98ca251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:29.363406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483117f-0376-4e64-94e3-01eef98ca251 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:29.364473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483117f-0376-4e64-94e3-01eef98ca251 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:29.364767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483117f-0376-4e64-94e3-01eef98ca251 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:29.365462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483117f-0376-4e64-94e3-01eef98ca251 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:02:29.371039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483117f-0376-4e64-94e3-01eef98ca251 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:29.373492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483117f-0376-4e64-94e3-01eef98ca251 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:29.406753 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2114/4159] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:02:29.412207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68c95c-27fc-4621-af97-90505f97c136 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:29.453627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68c95c-27fc-4621-af97-90505f97c136 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:29.486817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7ea16-47a7-472b-943b-dfcdd2c2136c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:29.490114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7ea16-47a7-472b-943b-dfcdd2c2136c admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:29.490401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7ea16-47a7-472b-943b-dfcdd2c2136c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:29.492314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7ea16-47a7-472b-943b-dfcdd2c2136c admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:29.495052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7ea16-47a7-472b-943b-dfcdd2c2136c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:29.499838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7ea16-47a7-472b-943b-dfcdd2c2136c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:29.549474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:29.655821 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2046/4160] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:02:29 2022] POST /identity/v3/projects => generated 381 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:02:29.659844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483117f-0376-4e64-94e3-01eef98ca251 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:29.679673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483117f-0376-4e64-94e3-01eef98ca251 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:29.877891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c91f6-aa61-4248-81ef-2b86c09ce888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:29.878683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c91f6-aa61-4248-81ef-2b86c09ce888 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:29.879056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c91f6-aa61-4248-81ef-2b86c09ce888 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:29.879338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c91f6-aa61-4248-81ef-2b86c09ce888 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:29.880792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c91f6-aa61-4248-81ef-2b86c09ce888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:29.885102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c91f6-aa61-4248-81ef-2b86c09ce888 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:29.954185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7ea16-47a7-472b-943b-dfcdd2c2136c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:29.954471 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2115/4161] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:02:29 2022] POST /identity/v3/users => generated 291 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:02:29.969677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7ea16-47a7-472b-943b-dfcdd2c2136c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:29.982651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9f949-a8c2-4887-a830-d318b32d28f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:29.983642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9f949-a8c2-4887-a830-d318b32d28f1 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:29.986291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9f949-a8c2-4887-a830-d318b32d28f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:29.986793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9f949-a8c2-4887-a830-d318b32d28f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:29.987901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9f949-a8c2-4887-a830-d318b32d28f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:30.011861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9f949-a8c2-4887-a830-d318b32d28f1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:30.058234 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2047/4162] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:02:30.063808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c91f6-aa61-4248-81ef-2b86c09ce888 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:30.107676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c91f6-aa61-4248-81ef-2b86c09ce888 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:30.142350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87115f-18e4-4d38-b545-9488d86daa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:30.143022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87115f-18e4-4d38-b545-9488d86daa9a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:30.143306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87115f-18e4-4d38-b545-9488d86daa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:30.143582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87115f-18e4-4d38-b545-9488d86daa9a admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:30.144244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87115f-18e4-4d38-b545-9488d86daa9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:30.145930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87115f-18e4-4d38-b545-9488d86daa9a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:30.164629 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2116/4163] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:02:30.169032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9f949-a8c2-4887-a830-d318b32d28f1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:30.183459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9f949-a8c2-4887-a830-d318b32d28f1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:30.189822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df41002-dfe3-47d1-94ff-e878201d961d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:30.220631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df41002-dfe3-47d1-94ff-e878201d961d admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:30.221105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df41002-dfe3-47d1-94ff-e878201d961d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:30.221498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df41002-dfe3-47d1-94ff-e878201d961d admin admin] PATH_INFO: `/v3/projects/1c3bf48f17ec49c8abbaa9eec9a4f93f/users/6246e90f7e1b4fb39f6ed6a90cb67d61/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:30.251283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df41002-dfe3-47d1-94ff-e878201d961d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c3bf48f17ec49c8abbaa9eec9a4f93f, user_id=6246e90f7e1b4fb39f6ed6a90cb67d61, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:30.259282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df41002-dfe3-47d1-94ff-e878201d961d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:30.291984 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2048/4164] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:02:30 2022] PUT /identity/v3/projects/1c3bf48f17ec49c8abbaa9eec9a4f93f/users/6246e90f7e1b4fb39f6ed6a90cb67d61/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:30.292926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976c930-1c42-4813-8ffe-f66faf29eb9a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:30.292926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976c930-1c42-4813-8ffe-f66faf29eb9a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:30.293143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976c930-1c42-4813-8ffe-f66faf29eb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:30.356007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2976c930-1c42-4813-8ffe-f66faf29eb9a None None] MFA Rules not processed for user `6246e90f7e1b4fb39f6ed6a90cb67d61`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:30.357531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2976c930-1c42-4813-8ffe-f66faf29eb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:30.487158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:30.519018 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2117/4165] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:30.531829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df41002-dfe3-47d1-94ff-e878201d961d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:30.534231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6df41002-dfe3-47d1-94ff-e878201d961d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:30.680239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df41002-dfe3-47d1-94ff-e878201d961d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:30.685481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ea685-193d-43e5-a8a5-5ad9e27de846 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:30.686117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ea685-193d-43e5-a8a5-5ad9e27de846 service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:30.686449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ea685-193d-43e5-a8a5-5ad9e27de846 service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:30.686752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ea685-193d-43e5-a8a5-5ad9e27de846 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:30.709294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ea685-193d-43e5-a8a5-5ad9e27de846 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:30.715988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ea685-193d-43e5-a8a5-5ad9e27de846 service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:30.738978 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2049/4166] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:02:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:02:30.981696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4f961-3f49-485b-9cc0-a1b1bb85b324 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:30.981983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4f961-3f49-485b-9cc0-a1b1bb85b324 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:30.982256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4f961-3f49-485b-9cc0-a1b1bb85b324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:31.005670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-64a4f961-3f49-485b-9cc0-a1b1bb85b324 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:31.007797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-64a4f961-3f49-485b-9cc0-a1b1bb85b324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:31.137739 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2118/4167] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:02:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:31.151238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ea685-193d-43e5-a8a5-5ad9e27de846 service glance] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:31.152940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f39ea685-193d-43e5-a8a5-5ad9e27de846 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:31.362381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ea685-193d-43e5-a8a5-5ad9e27de846 service glance] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:31.369545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eda4b3-9579-4728-93db-7f510911f11b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:31.370183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eda4b3-9579-4728-93db-7f510911f11b service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:31.370467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eda4b3-9579-4728-93db-7f510911f11b service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:31.370792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eda4b3-9579-4728-93db-7f510911f11b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:31.396520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eda4b3-9579-4728-93db-7f510911f11b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:31.399344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eda4b3-9579-4728-93db-7f510911f11b service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:31.433533 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2050/4168] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:02:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:02:31.600609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89476011-5902-46a4-baed-ba25c2127c6d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:31.600892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89476011-5902-46a4-baed-ba25c2127c6d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:31.601396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89476011-5902-46a4-baed-ba25c2127c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:31.675009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-89476011-5902-46a4-baed-ba25c2127c6d None None] MFA Rules not processed for user `d4cb0ffdb0ef42c18656309683ab0553`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:31.676612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-89476011-5902-46a4-baed-ba25c2127c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:31.806536 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2119/4169] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:31.965419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2bc50-9ca5-44c6-8793-5e5f2e7cfd2b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:31.966292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2bc50-9ca5-44c6-8793-5e5f2e7cfd2b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:31.966761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2bc50-9ca5-44c6-8793-5e5f2e7cfd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:31.993958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a9d2bc50-9ca5-44c6-8793-5e5f2e7cfd2b None None] MFA Rules not processed for user `221b665fc7cb4758a6e35cc81c6a4d43`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:31.995480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a9d2bc50-9ca5-44c6-8793-5e5f2e7cfd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:32.059375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:32.090107 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2051/4170] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:38.046144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89476011-5902-46a4-baed-ba25c2127c6d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:38.049792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-89476011-5902-46a4-baed-ba25c2127c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:38.257710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89476011-5902-46a4-baed-ba25c2127c6d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:38.270215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a80388-ba21-4a9c-89f3-f2b5ba6bfa02 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:38.270796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a80388-ba21-4a9c-89f3-f2b5ba6bfa02 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:38.271083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a80388-ba21-4a9c-89f3-f2b5ba6bfa02 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:38.271614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a80388-ba21-4a9c-89f3-f2b5ba6bfa02 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:38.274254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b5a80388-ba21-4a9c-89f3-f2b5ba6bfa02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:38.302438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a80388-ba21-4a9c-89f3-f2b5ba6bfa02 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:38.305651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a80388-ba21-4a9c-89f3-f2b5ba6bfa02 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:38.323763 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2120/4171] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:02:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:02:39.238821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c0902-b445-4a6e-ab03-b6b6ed671f59 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:39.239771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c0902-b445-4a6e-ab03-b6b6ed671f59 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:39.240273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c0902-b445-4a6e-ab03-b6b6ed671f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:39.271403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-858c0902-b445-4a6e-ab03-b6b6ed671f59 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:39.274030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-858c0902-b445-4a6e-ab03-b6b6ed671f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:39.284319 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2052/4172] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:02:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:39.307640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a80388-ba21-4a9c-89f3-f2b5ba6bfa02 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:39.322858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a80388-ba21-4a9c-89f3-f2b5ba6bfa02 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:39.339097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf942f0-4dc4-4992-baae-7a28a8d1d25b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:39.340028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf942f0-4dc4-4992-baae-7a28a8d1d25b service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:39.340577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf942f0-4dc4-4992-baae-7a28a8d1d25b service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:39.341209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf942f0-4dc4-4992-baae-7a28a8d1d25b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:39.352460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf942f0-4dc4-4992-baae-7a28a8d1d25b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:39.354411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf942f0-4dc4-4992-baae-7a28a8d1d25b service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:39.371821 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2121/4173] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:02:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:02:44.100751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c0902-b445-4a6e-ab03-b6b6ed671f59 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:44.103428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-858c0902-b445-4a6e-ab03-b6b6ed671f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:44.344998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c0902-b445-4a6e-ab03-b6b6ed671f59 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:44.359555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc63e23-0ba5-49f2-b98e-d84b475b7905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:44.360071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc63e23-0ba5-49f2-b98e-d84b475b7905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:44.360341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc63e23-0ba5-49f2-b98e-d84b475b7905 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:44.360843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc63e23-0ba5-49f2-b98e-d84b475b7905 admin admin] PATH_INFO: `/v3/users/d4cb0ffdb0ef42c18656309683ab0553` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:44.363496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc63e23-0ba5-49f2-b98e-d84b475b7905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4cb0ffdb0ef42c18656309683ab0553)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:44.367334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc63e23-0ba5-49f2-b98e-d84b475b7905 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:44.547577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9fc63e23-0ba5-49f2-b98e-d84b475b7905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4cb0ffdb0ef42c18656309683ab0553'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:44.548285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9fc63e23-0ba5-49f2-b98e-d84b475b7905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4cb0ffdb0ef42c18656309683ab0553'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:44.548909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9fc63e23-0ba5-49f2-b98e-d84b475b7905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4cb0ffdb0ef42c18656309683ab0553'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:44.571019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9fc63e23-0ba5-49f2-b98e-d84b475b7905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4cb0ffdb0ef42c18656309683ab0553'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:44.591037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9fc63e23-0ba5-49f2-b98e-d84b475b7905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4cb0ffdb0ef42c18656309683ab0553'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:44.625838 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2053/4174] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:02:44 2022] DELETE /identity/v3/users/d4cb0ffdb0ef42c18656309683ab0553 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:45.010346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf942f0-4dc4-4992-baae-7a28a8d1d25b service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:45.044758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf942f0-4dc4-4992-baae-7a28a8d1d25b service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:45.051476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:45.105069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24790260-94a4-4668-b9d5-a83d18a4549b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:45.105533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24790260-94a4-4668-b9d5-a83d18a4549b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:45.105533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24790260-94a4-4668-b9d5-a83d18a4549b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:45.157337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-24790260-94a4-4668-b9d5-a83d18a4549b None None] MFA Rules not processed for user `775ff9091a794faaacc05a37550c8a72`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:45.159572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-24790260-94a4-4668-b9d5-a83d18a4549b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:45.214940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc4cc3-2951-4c13-815a-5eb7f7a55b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:45.218549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc4cc3-2951-4c13-815a-5eb7f7a55b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:45.219854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc4cc3-2951-4c13-815a-5eb7f7a55b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:45.222081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc4cc3-2951-4c13-815a-5eb7f7a55b27 admin admin] PATH_INFO: `/v3/projects/c59393807d9d41ed9a7903a6e0b55755` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:45.229476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc4cc3-2951-4c13-815a-5eb7f7a55b27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c59393807d9d41ed9a7903a6e0b55755)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:45.237713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc4cc3-2951-4c13-815a-5eb7f7a55b27 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:45.314086 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2054/4175] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:45.375359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fccc4cc3-2951-4c13-815a-5eb7f7a55b27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c59393807d9d41ed9a7903a6e0b55755', 'request_id': 'req-fccc4cc3-2951-4c13-815a-5eb7f7a55b27'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:45.394185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fccc4cc3-2951-4c13-815a-5eb7f7a55b27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c59393807d9d41ed9a7903a6e0b55755', 'request_id': 'req-fccc4cc3-2951-4c13-815a-5eb7f7a55b27'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:45.411453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fccc4cc3-2951-4c13-815a-5eb7f7a55b27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c59393807d9d41ed9a7903a6e0b55755', 'request_id': 'req-fccc4cc3-2951-4c13-815a-5eb7f7a55b27'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:45.435187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fccc4cc3-2951-4c13-815a-5eb7f7a55b27 admin admin] The token cache is being invalidate because project c59393807d9d41ed9a7903a6e0b55755 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:02:45.435771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fccc4cc3-2951-4c13-815a-5eb7f7a55b27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:45.437708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fccc4cc3-2951-4c13-815a-5eb7f7a55b27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:45.440719 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2122/4176] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:02:45 2022] DELETE /identity/v3/projects/c59393807d9d41ed9a7903a6e0b55755 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:45.448576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc25a41-b0fe-4053-8bea-f452788982cf None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:45.449437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc25a41-b0fe-4053-8bea-f452788982cf None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:45.449875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc25a41-b0fe-4053-8bea-f452788982cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:45.485768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3dc25a41-b0fe-4053-8bea-f452788982cf None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:45.487571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3dc25a41-b0fe-4053-8bea-f452788982cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:45.508683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:45.651284 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2055/4177] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:45.659193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c9200-9cbc-45b4-a199-5e9fef9c98f4 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:45.659714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c9200-9cbc-45b4-a199-5e9fef9c98f4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:45.660177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c9200-9cbc-45b4-a199-5e9fef9c98f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:45.700367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-893c9200-9cbc-45b4-a199-5e9fef9c98f4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:45.703260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-893c9200-9cbc-45b4-a199-5e9fef9c98f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:45.718561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc25a41-b0fe-4053-8bea-f452788982cf None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:45.719800 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2123/4178] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:45.733765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc25a41-b0fe-4053-8bea-f452788982cf None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:45.751326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328134db-d167-419c-ad9e-f3a170fd7a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:45.751967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328134db-d167-419c-ad9e-f3a170fd7a61 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:45.752252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328134db-d167-419c-ad9e-f3a170fd7a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:45.752535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328134db-d167-419c-ad9e-f3a170fd7a61 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:45.753209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328134db-d167-419c-ad9e-f3a170fd7a61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:02:45.753515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328134db-d167-419c-ad9e-f3a170fd7a61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:45.763891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328134db-d167-419c-ad9e-f3a170fd7a61 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:45.811505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c9200-9cbc-45b4-a199-5e9fef9c98f4 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:45.814978 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2056/4179] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:02:45.851001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c9200-9cbc-45b4-a199-5e9fef9c98f4 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:45.875838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f561d2-e146-402a-9070-bd6cca668951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:45.876520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f561d2-e146-402a-9070-bd6cca668951 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:45.878732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f561d2-e146-402a-9070-bd6cca668951 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:45.879030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f561d2-e146-402a-9070-bd6cca668951 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:45.879686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f561d2-e146-402a-9070-bd6cca668951 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:45.892510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f561d2-e146-402a-9070-bd6cca668951 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:45.962601 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2124/4180] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:02:45 2022] POST /identity/v3/projects => generated 403 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:02:45.966829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328134db-d167-419c-ad9e-f3a170fd7a61 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:45.978291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328134db-d167-419c-ad9e-f3a170fd7a61 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:46.127542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b8d5e-3d8e-49ce-8f45-e4860682b2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:46.128246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b8d5e-3d8e-49ce-8f45-e4860682b2a5 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:46.128441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b8d5e-3d8e-49ce-8f45-e4860682b2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:46.128699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b8d5e-3d8e-49ce-8f45-e4860682b2a5 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:46.132244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b8d5e-3d8e-49ce-8f45-e4860682b2a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:46.137629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b8d5e-3d8e-49ce-8f45-e4860682b2a5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:46.190286 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2057/4181] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:02:45 2022] POST /identity/v3/users => generated 302 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:02:46.194697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f561d2-e146-402a-9070-bd6cca668951 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:46.210007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f561d2-e146-402a-9070-bd6cca668951 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:46.221717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000eae2b-50e8-4fd7-a568-392d584e7a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:46.222341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000eae2b-50e8-4fd7-a568-392d584e7a47 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:46.222829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000eae2b-50e8-4fd7-a568-392d584e7a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:46.223196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000eae2b-50e8-4fd7-a568-392d584e7a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:46.223863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000eae2b-50e8-4fd7-a568-392d584e7a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:46.227434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000eae2b-50e8-4fd7-a568-392d584e7a47 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:46.254464 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2125/4182] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:02:46.259146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b8d5e-3d8e-49ce-8f45-e4860682b2a5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:46.277904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b8d5e-3d8e-49ce-8f45-e4860682b2a5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:46.283484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a7bbd-d577-4a51-b241-0303ae49f875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:46.284393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a7bbd-d577-4a51-b241-0303ae49f875 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:46.284676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a7bbd-d577-4a51-b241-0303ae49f875 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:46.285120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a7bbd-d577-4a51-b241-0303ae49f875 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:46.285775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a7bbd-d577-4a51-b241-0303ae49f875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:46.287680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a7bbd-d577-4a51-b241-0303ae49f875 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:46.318192 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2058/4183] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:02:46.322782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000eae2b-50e8-4fd7-a568-392d584e7a47 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:46.347432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000eae2b-50e8-4fd7-a568-392d584e7a47 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:46.356370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21db300-2a23-4f3d-9a95-f1204cd46f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:46.357041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21db300-2a23-4f3d-9a95-f1204cd46f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:46.357500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21db300-2a23-4f3d-9a95-f1204cd46f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:46.357779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21db300-2a23-4f3d-9a95-f1204cd46f1a admin admin] PATH_INFO: `/v3/projects/eb944528d0c74f56a4ea375c15d2300d/users/bc3672f435784f1597e736e55fd313ca/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:46.386291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21db300-2a23-4f3d-9a95-f1204cd46f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb944528d0c74f56a4ea375c15d2300d, user_id=bc3672f435784f1597e736e55fd313ca, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:46.392690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21db300-2a23-4f3d-9a95-f1204cd46f1a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:46.411836 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2126/4184] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:02:46 2022] PUT /identity/v3/projects/eb944528d0c74f56a4ea375c15d2300d/users/bc3672f435784f1597e736e55fd313ca/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:02:46.421922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db08f9-98b5-4b4d-a40f-68c4c24e825c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:46.422253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db08f9-98b5-4b4d-a40f-68c4c24e825c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:46.422514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db08f9-98b5-4b4d-a40f-68c4c24e825c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:46.476337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b7db08f9-98b5-4b4d-a40f-68c4c24e825c None None] MFA Rules not processed for user `bc3672f435784f1597e736e55fd313ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:46.477467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b7db08f9-98b5-4b4d-a40f-68c4c24e825c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:46.587532 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2059/4185] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:46 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:48.600726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21db300-2a23-4f3d-9a95-f1204cd46f1a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:48.664869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:48.718781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21db300-2a23-4f3d-9a95-f1204cd46f1a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:48.726124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61cb6ba-dc6e-429f-8c53-a35b6562b113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:48.726835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61cb6ba-dc6e-429f-8c53-a35b6562b113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:48.727132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61cb6ba-dc6e-429f-8c53-a35b6562b113 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:48.727475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61cb6ba-dc6e-429f-8c53-a35b6562b113 admin admin] PATH_INFO: `/v3/users/bc3672f435784f1597e736e55fd313ca` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:48.730060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61cb6ba-dc6e-429f-8c53-a35b6562b113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc3672f435784f1597e736e55fd313ca)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:48.733195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61cb6ba-dc6e-429f-8c53-a35b6562b113 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:48.842562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d61cb6ba-dc6e-429f-8c53-a35b6562b113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc3672f435784f1597e736e55fd313ca'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:48.869175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d61cb6ba-dc6e-429f-8c53-a35b6562b113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc3672f435784f1597e736e55fd313ca'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:48.898675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d61cb6ba-dc6e-429f-8c53-a35b6562b113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc3672f435784f1597e736e55fd313ca'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:48.900447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d61cb6ba-dc6e-429f-8c53-a35b6562b113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc3672f435784f1597e736e55fd313ca'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:48.915113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d61cb6ba-dc6e-429f-8c53-a35b6562b113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc3672f435784f1597e736e55fd313ca'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:48.939806 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2127/4186] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:02:48 2022] DELETE /identity/v3/users/bc3672f435784f1597e736e55fd313ca => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:48.944478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b964efc-c0c3-46f5-b051-2ee1e8b37fa2 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:48.945287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b964efc-c0c3-46f5-b051-2ee1e8b37fa2 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:48.945724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b964efc-c0c3-46f5-b051-2ee1e8b37fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:48.963660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:49.003760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0b964efc-c0c3-46f5-b051-2ee1e8b37fa2 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:49.004995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0b964efc-c0c3-46f5-b051-2ee1e8b37fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:49.145796 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2060/4187] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:49.156852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61cb6ba-dc6e-429f-8c53-a35b6562b113 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:49.158344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d61cb6ba-dc6e-429f-8c53-a35b6562b113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:49.303671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61cb6ba-dc6e-429f-8c53-a35b6562b113 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:49.317235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dab62a-4192-4939-b3cd-aa984821cd0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:49.319546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dab62a-4192-4939-b3cd-aa984821cd0e service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:49.320000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dab62a-4192-4939-b3cd-aa984821cd0e service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:49.324965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dab62a-4192-4939-b3cd-aa984821cd0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:49.346214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dab62a-4192-4939-b3cd-aa984821cd0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:49.349015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dab62a-4192-4939-b3cd-aa984821cd0e service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:49.407931 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2128/4188] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:02:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:02:50.401650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b964efc-c0c3-46f5-b051-2ee1e8b37fa2 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:50.403507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0b964efc-c0c3-46f5-b051-2ee1e8b37fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:50.424811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b964efc-c0c3-46f5-b051-2ee1e8b37fa2 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:50.439059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592bf3a-4182-442f-b50e-ae18ce0636cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:50.440271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592bf3a-4182-442f-b50e-ae18ce0636cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:50.440764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592bf3a-4182-442f-b50e-ae18ce0636cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:50.441364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592bf3a-4182-442f-b50e-ae18ce0636cb admin admin] PATH_INFO: `/v3/projects/eb944528d0c74f56a4ea375c15d2300d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:50.445120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4592bf3a-4182-442f-b50e-ae18ce0636cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb944528d0c74f56a4ea375c15d2300d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:50.448914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4592bf3a-4182-442f-b50e-ae18ce0636cb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:50.604121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4592bf3a-4182-442f-b50e-ae18ce0636cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb944528d0c74f56a4ea375c15d2300d', 'request_id': 'req-4592bf3a-4182-442f-b50e-ae18ce0636cb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:50.620968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4592bf3a-4182-442f-b50e-ae18ce0636cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb944528d0c74f56a4ea375c15d2300d', 'request_id': 'req-4592bf3a-4182-442f-b50e-ae18ce0636cb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:50.635701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4592bf3a-4182-442f-b50e-ae18ce0636cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb944528d0c74f56a4ea375c15d2300d', 'request_id': 'req-4592bf3a-4182-442f-b50e-ae18ce0636cb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:50.664643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4592bf3a-4182-442f-b50e-ae18ce0636cb admin admin] The token cache is being invalidate because project eb944528d0c74f56a4ea375c15d2300d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:02:50.664940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4592bf3a-4182-442f-b50e-ae18ce0636cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:50.668857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4592bf3a-4182-442f-b50e-ae18ce0636cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:50.678293 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2061/4189] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:02:50 2022] DELETE /identity/v3/projects/eb944528d0c74f56a4ea375c15d2300d => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:50.682343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba5be9-2aef-4899-bbe1-9ee811049646 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:50.684318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba5be9-2aef-4899-bbe1-9ee811049646 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:50.684762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba5be9-2aef-4899-bbe1-9ee811049646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:50.715341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-45ba5be9-2aef-4899-bbe1-9ee811049646 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:50.717054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-45ba5be9-2aef-4899-bbe1-9ee811049646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:50.788829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:50.817114 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2129/4190] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:50.826523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b2cd6-4f09-45e6-bc39-dcaffc20f24e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:50.827080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b2cd6-4f09-45e6-bc39-dcaffc20f24e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:50.827686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b2cd6-4f09-45e6-bc39-dcaffc20f24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:50.866294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-454b2cd6-4f09-45e6-bc39-dcaffc20f24e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:50.867559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-454b2cd6-4f09-45e6-bc39-dcaffc20f24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:50.882592 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2062/4191] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:50.885737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba5be9-2aef-4899-bbe1-9ee811049646 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:50.897452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba5be9-2aef-4899-bbe1-9ee811049646 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:50.913225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a430c-68af-4f2b-beac-eaac7809d445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:50.914048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a430c-68af-4f2b-beac-eaac7809d445 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:50.915077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a430c-68af-4f2b-beac-eaac7809d445 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:50.915508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a430c-68af-4f2b-beac-eaac7809d445 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:50.916784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a430c-68af-4f2b-beac-eaac7809d445 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:02:50.917364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a430c-68af-4f2b-beac-eaac7809d445 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:50.920962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a430c-68af-4f2b-beac-eaac7809d445 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:50.938460 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2130/4192] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:02:50.944582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b2cd6-4f09-45e6-bc39-dcaffc20f24e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:50.967647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b2cd6-4f09-45e6-bc39-dcaffc20f24e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:51.002416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a33695-3d76-4bbc-b4d3-430df8e06a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:51.003302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a33695-3d76-4bbc-b4d3-430df8e06a0c admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:51.004230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a33695-3d76-4bbc-b4d3-430df8e06a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:51.004522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a33695-3d76-4bbc-b4d3-430df8e06a0c admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:51.005267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a33695-3d76-4bbc-b4d3-430df8e06a0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:51.011429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a33695-3d76-4bbc-b4d3-430df8e06a0c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:51.106178 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2063/4193] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:02:50 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:02:51.106851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a430c-68af-4f2b-beac-eaac7809d445 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:51.121835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a430c-68af-4f2b-beac-eaac7809d445 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:51.275037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76bf88-0055-4ec2-9232-388277d184a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:51.275660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76bf88-0055-4ec2-9232-388277d184a8 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:51.275939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76bf88-0055-4ec2-9232-388277d184a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:51.276215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76bf88-0055-4ec2-9232-388277d184a8 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:51.276974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76bf88-0055-4ec2-9232-388277d184a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:51.282424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76bf88-0055-4ec2-9232-388277d184a8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:51.340612 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2131/4194] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:02:51 2022] POST /identity/v3/users => generated 294 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:02:51.341715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a33695-3d76-4bbc-b4d3-430df8e06a0c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:51.357041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a33695-3d76-4bbc-b4d3-430df8e06a0c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:51.379922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf6bb6-2ec2-423f-b99a-a0e92f7b58b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:51.379922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf6bb6-2ec2-423f-b99a-a0e92f7b58b7 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:51.379922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf6bb6-2ec2-423f-b99a-a0e92f7b58b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:51.379922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf6bb6-2ec2-423f-b99a-a0e92f7b58b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:51.380371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf6bb6-2ec2-423f-b99a-a0e92f7b58b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:51.384643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf6bb6-2ec2-423f-b99a-a0e92f7b58b7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:51.424743 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2064/4195] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:02:51.426078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76bf88-0055-4ec2-9232-388277d184a8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:51.437762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76bf88-0055-4ec2-9232-388277d184a8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:51.456424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c88f78-39b7-4af7-b05c-f88e01063eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:51.457130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c88f78-39b7-4af7-b05c-f88e01063eee admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:51.457522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c88f78-39b7-4af7-b05c-f88e01063eee admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:51.457903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c88f78-39b7-4af7-b05c-f88e01063eee admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:51.459582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c88f78-39b7-4af7-b05c-f88e01063eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:51.461725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c88f78-39b7-4af7-b05c-f88e01063eee admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:51.489372 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2132/4196] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:02:51.490827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf6bb6-2ec2-423f-b99a-a0e92f7b58b7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:51.528242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf6bb6-2ec2-423f-b99a-a0e92f7b58b7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:51.554333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f524004-71c8-453b-a126-bb4efffa377a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:51.556955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f524004-71c8-453b-a126-bb4efffa377a admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:51.560686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f524004-71c8-453b-a126-bb4efffa377a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:51.561522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f524004-71c8-453b-a126-bb4efffa377a admin admin] PATH_INFO: `/v3/projects/a6f2c8a6f73749a49068fe4f08ea057b/users/a75b672d0f4b458fbd4e82289c4d0eb2/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:51.604648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f524004-71c8-453b-a126-bb4efffa377a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6f2c8a6f73749a49068fe4f08ea057b, user_id=a75b672d0f4b458fbd4e82289c4d0eb2, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:51.611834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f524004-71c8-453b-a126-bb4efffa377a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:51.644658 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2065/4197] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:02:51 2022] PUT /identity/v3/projects/a6f2c8a6f73749a49068fe4f08ea057b/users/a75b672d0f4b458fbd4e82289c4d0eb2/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:51.729220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9116dd-527a-4d93-ad89-b02e0706830a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:51.729774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9116dd-527a-4d93-ad89-b02e0706830a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:51.729774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9116dd-527a-4d93-ad89-b02e0706830a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:51.815048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2b9116dd-527a-4d93-ad89-b02e0706830a None None] MFA Rules not processed for user `a75b672d0f4b458fbd4e82289c4d0eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:51.816173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2b9116dd-527a-4d93-ad89-b02e0706830a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:51.950752 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2133/4198] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:51.955438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f524004-71c8-453b-a126-bb4efffa377a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:52.012882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:52.108377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f524004-71c8-453b-a126-bb4efffa377a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:52.119677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab2cab-293e-4b18-8172-cfb3f8890159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:52.120673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab2cab-293e-4b18-8172-cfb3f8890159 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:52.121595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab2cab-293e-4b18-8172-cfb3f8890159 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:52.122046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab2cab-293e-4b18-8172-cfb3f8890159 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:52.124701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab2cab-293e-4b18-8172-cfb3f8890159 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:52.126875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab2cab-293e-4b18-8172-cfb3f8890159 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:52.191155 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2066/4199] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:02:51 2022] POST /identity/v3/projects => generated 385 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:02:52.195704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9116dd-527a-4d93-ad89-b02e0706830a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:52.214358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9116dd-527a-4d93-ad89-b02e0706830a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:52.280375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:52.335263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3db083-2ed0-4042-a742-60a4658a5dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:52.335817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3db083-2ed0-4042-a742-60a4658a5dd3 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:52.336101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3db083-2ed0-4042-a742-60a4658a5dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:52.336378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3db083-2ed0-4042-a742-60a4658a5dd3 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:52.337030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3db083-2ed0-4042-a742-60a4658a5dd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:52.339405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3db083-2ed0-4042-a742-60a4658a5dd3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:52.385645 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2134/4200] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:02:52 2022] POST /identity/v3/users => generated 299 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:02:52.388970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab2cab-293e-4b18-8172-cfb3f8890159 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:52.402141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab2cab-293e-4b18-8172-cfb3f8890159 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:52.409656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8af6d2-c4f8-47eb-8b5b-2baba12791c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:52.410575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8af6d2-c4f8-47eb-8b5b-2baba12791c8 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:52.411610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8af6d2-c4f8-47eb-8b5b-2baba12791c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:52.414663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8af6d2-c4f8-47eb-8b5b-2baba12791c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:52.416342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8af6d2-c4f8-47eb-8b5b-2baba12791c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:52.418775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8af6d2-c4f8-47eb-8b5b-2baba12791c8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:52.454043 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2067/4201] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:02:52.457982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3db083-2ed0-4042-a742-60a4658a5dd3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:52.504541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3db083-2ed0-4042-a742-60a4658a5dd3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:52.522808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e3134-58f0-478a-bfa2-c66f0c1d6e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:52.522808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e3134-58f0-478a-bfa2-c66f0c1d6e07 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:52.522808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e3134-58f0-478a-bfa2-c66f0c1d6e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:52.522808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e3134-58f0-478a-bfa2-c66f0c1d6e07 admin admin] PATH_INFO: `/v3/projects/cebf7dcaa23d49ec8f0a0b39bfa48400/users/384e46aa04a9460f997765e9e5bbe0a0/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:52.579101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e3134-58f0-478a-bfa2-c66f0c1d6e07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cebf7dcaa23d49ec8f0a0b39bfa48400, user_id=384e46aa04a9460f997765e9e5bbe0a0, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:52.581921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e3134-58f0-478a-bfa2-c66f0c1d6e07 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:52.610706 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2135/4202] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:02:52 2022] PUT /identity/v3/projects/cebf7dcaa23d49ec8f0a0b39bfa48400/users/384e46aa04a9460f997765e9e5bbe0a0/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:52.616361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8af6d2-c4f8-47eb-8b5b-2baba12791c8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:52.633757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8af6d2-c4f8-47eb-8b5b-2baba12791c8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:52.784779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c5a9be-b6b4-49bf-8cdc-43a94965d072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:52.785662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5a9be-b6b4-49bf-8cdc-43a94965d072 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:52.785955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5a9be-b6b4-49bf-8cdc-43a94965d072 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:52.786412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5a9be-b6b4-49bf-8cdc-43a94965d072 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:52.787838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c5a9be-b6b4-49bf-8cdc-43a94965d072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:52.790050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c5a9be-b6b4-49bf-8cdc-43a94965d072 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:52.807385 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2068/4203] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:02:52.809171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e3134-58f0-478a-bfa2-c66f0c1d6e07 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:52.846014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e3134-58f0-478a-bfa2-c66f0c1d6e07 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:52.866240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47740fb4-1ded-43c2-a7b7-c30c287a90f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:52.867260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47740fb4-1ded-43c2-a7b7-c30c287a90f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:52.867694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47740fb4-1ded-43c2-a7b7-c30c287a90f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:52.868137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47740fb4-1ded-43c2-a7b7-c30c287a90f4 admin admin] PATH_INFO: `/v3/projects/cebf7dcaa23d49ec8f0a0b39bfa48400/users/384e46aa04a9460f997765e9e5bbe0a0/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:52.870242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47740fb4-1ded-43c2-a7b7-c30c287a90f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cebf7dcaa23d49ec8f0a0b39bfa48400, user_id=384e46aa04a9460f997765e9e5bbe0a0, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:52.874855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47740fb4-1ded-43c2-a7b7-c30c287a90f4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:52.900947 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2136/4204] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:02:52 2022] PUT /identity/v3/projects/cebf7dcaa23d49ec8f0a0b39bfa48400/users/384e46aa04a9460f997765e9e5bbe0a0/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:52.910841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec3c5c-5aa4-4d5a-af30-10628be8c7fd None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:52.911059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec3c5c-5aa4-4d5a-af30-10628be8c7fd None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:52.911825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec3c5c-5aa4-4d5a-af30-10628be8c7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:52.979768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d6ec3c5c-5aa4-4d5a-af30-10628be8c7fd None None] MFA Rules not processed for user `384e46aa04a9460f997765e9e5bbe0a0`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:52.982292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d6ec3c5c-5aa4-4d5a-af30-10628be8c7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:53.142851 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2069/4205] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:53.147332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47740fb4-1ded-43c2-a7b7-c30c287a90f4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:53.161133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47740fb4-1ded-43c2-a7b7-c30c287a90f4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:53.182766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6f788-e1a0-4209-9cf0-a2a66c6021b7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '384e46aa04a9460f997765e9e5bbe0a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cebf7dcaa23d49ec8f0a0b39bfa48400', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:53.184775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6f788-e1a0-4209-9cf0-a2a66c6021b7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:53.185530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6f788-e1a0-4209-9cf0-a2a66c6021b7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:53.186080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6f788-e1a0-4209-9cf0-a2a66c6021b7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:53.187466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6f788-e1a0-4209-9cf0-a2a66c6021b7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:53.189515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6f788-e1a0-4209-9cf0-a2a66c6021b7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:53.226096 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2137/4206] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:02:53 2022] POST /identity/v3/services => generated 287 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:02:53.231198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec3c5c-5aa4-4d5a-af30-10628be8c7fd None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:53.249946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec3c5c-5aa4-4d5a-af30-10628be8c7fd None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:53.323518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6a10a-c837-41f7-9e52-d0c037121e70 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '384e46aa04a9460f997765e9e5bbe0a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cebf7dcaa23d49ec8f0a0b39bfa48400', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:53.325704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6a10a-c837-41f7-9e52-d0c037121e70 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:53.329091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6a10a-c837-41f7-9e52-d0c037121e70 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:53.329581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6a10a-c837-41f7-9e52-d0c037121e70 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:53.335352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c6a10a-c837-41f7-9e52-d0c037121e70 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:53.337443 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-64c6a10a-c837-41f7-9e52-d0c037121e70 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:02:53.337443 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:02:53.337443 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:02:53.338942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c6a10a-c837-41f7-9e52-d0c037121e70 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-64c6a10a-c837-41f7-9e52-d0c037121e70 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Invalid input for field 'enabled': 'False' is not of type 'boolean' Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: Failed validating 'type' in schema['properties']['enabled']: Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: {'enum': [True, False, None], 'type': 'boolean'} Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: On instance['enabled']: Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: 'False': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: Failed validating 'type' in schema['properties']['enabled']: Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: {'enum': [True, False, None], 'type': 'boolean'} Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: On instance['enabled']: Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: 'False' Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise error Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'False' is not of type 'boolean' Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application On instance['enabled']: Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application 'False' Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:02:53.358811 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:02:53.360862 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:02:53.360862 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:02:53.360862 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:02:53.360862 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:02:53.360862 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:02:53.360862 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:02:53.360862 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:02:53.360862 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:02:53.360862 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:02:53.360862 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:02:53.360862 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:02:53.360862 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:02:53.360862 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 94, in post Oct 01 21:02:53.360862 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_create, endpoint) Oct 01 21:02:53.360862 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Oct 01 21:02:53.360862 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Oct 01 21:02:53.360862 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Oct 01 21:02:53.362146 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Oct 01 21:02:53.362146 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Oct 01 21:02:53.362146 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:02:53.362146 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Oct 01 21:02:53.362146 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Oct 01 21:02:53.362146 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:02:53.362146 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application On instance['enabled']: Oct 01 21:02:53.362146 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application 'False' Oct 01 21:02:53.362146 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:02:53.366928 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2070/4207] 10.222.0.17 () {60 vars in 1156 bytes} [Sat Oct 1 21:02:53 2022] POST /identity/v3/endpoints => generated 283 bytes in 137 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 01 21:02:53.372557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6f788-e1a0-4209-9cf0-a2a66c6021b7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:53.394422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6f788-e1a0-4209-9cf0-a2a66c6021b7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:53.407292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9750d-4aee-4030-be60-167da08736ea tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '384e46aa04a9460f997765e9e5bbe0a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cebf7dcaa23d49ec8f0a0b39bfa48400', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:53.407734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9750d-4aee-4030-be60-167da08736ea tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:53.409892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9750d-4aee-4030-be60-167da08736ea tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:53.410200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9750d-4aee-4030-be60-167da08736ea tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:53.410969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed9750d-4aee-4030-be60-167da08736ea tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:53.413194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed9750d-4aee-4030-be60-167da08736ea tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-2ed9750d-4aee-4030-be60-167da08736ea tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Invalid input for field 'enabled': 'True' is not of type 'boolean' Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: Failed validating 'type' in schema['properties']['enabled']: Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: {'enum': [True, False, None], 'type': 'boolean'} Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: On instance['enabled']: Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: 'True': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: Failed validating 'type' in schema['properties']['enabled']: Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: {'enum': [True, False, None], 'type': 'boolean'} Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: On instance['enabled']: Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: 'True' Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise error Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'True' is not of type 'boolean' Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application On instance['enabled']: Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application 'True' Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:02:53.433112 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:02:53.435781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:02:53.435781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:02:53.435781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:02:53.435781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:02:53.435781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:02:53.435781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:02:53.435781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:02:53.435781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:02:53.435781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:02:53.435781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:02:53.435781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:02:53.435781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:02:53.435781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 94, in post Oct 01 21:02:53.435781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_create, endpoint) Oct 01 21:02:53.435781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Oct 01 21:02:53.435781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Oct 01 21:02:53.435781 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Oct 01 21:02:53.436845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Oct 01 21:02:53.436845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Oct 01 21:02:53.436845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:02:53.436845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Oct 01 21:02:53.436845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Oct 01 21:02:53.436845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:02:53.436845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application On instance['enabled']: Oct 01 21:02:53.436845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application 'True' Oct 01 21:02:53.436845 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:02:53.436845 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2138/4208] 10.222.0.17 () {60 vars in 1156 bytes} [Sat Oct 1 21:02:53 2022] POST /identity/v3/endpoints => generated 281 bytes in 63 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 01 21:02:53.439590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6a10a-c837-41f7-9e52-d0c037121e70 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:53.452047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6a10a-c837-41f7-9e52-d0c037121e70 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:53.463167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8667e-c089-4209-9a22-6792fe0ab150 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '384e46aa04a9460f997765e9e5bbe0a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cebf7dcaa23d49ec8f0a0b39bfa48400', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:53.463894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8667e-c089-4209-9a22-6792fe0ab150 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:53.464199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8667e-c089-4209-9a22-6792fe0ab150 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:53.464465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8667e-c089-4209-9a22-6792fe0ab150 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:53.465287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc8667e-c089-4209-9a22-6792fe0ab150 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:53.472777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc8667e-c089-4209-9a22-6792fe0ab150 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:53.479977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:53.527135 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-bbc8667e-c089-4209-9a22-6792fe0ab150 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1592625759. To ensure interoperability, identifiers should be a valid uuid. Oct 01 21:02:53.527135 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 01 21:02:53.527135 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:02:53.591923 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2071/4209] 10.222.0.17 () {60 vars in 1156 bytes} [Sat Oct 1 21:02:53 2022] POST /identity/v3/endpoints => generated 360 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:02:53.595822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9750d-4aee-4030-be60-167da08736ea tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:53.616615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9750d-4aee-4030-be60-167da08736ea tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:53.665823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ec4ca-631d-44b0-8209-59dbcd0f4e24 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '384e46aa04a9460f997765e9e5bbe0a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cebf7dcaa23d49ec8f0a0b39bfa48400', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:53.666759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ec4ca-631d-44b0-8209-59dbcd0f4e24 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:53.667232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ec4ca-631d-44b0-8209-59dbcd0f4e24 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:53.667680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ec4ca-631d-44b0-8209-59dbcd0f4e24 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] PATH_INFO: `/v3/regions/tempest-region-1592625759` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:53.668431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7ec4ca-631d-44b0-8209-59dbcd0f4e24 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1592625759)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:53.669682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7ec4ca-631d-44b0-8209-59dbcd0f4e24 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:53.678721 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2139/4210] 10.222.0.17 () {58 vars in 1206 bytes} [Sat Oct 1 21:02:53 2022] GET /identity/v3/regions/tempest-region-1592625759 => generated 180 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:02:53.679729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8667e-c089-4209-9a22-6792fe0ab150 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:53.697732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8667e-c089-4209-9a22-6792fe0ab150 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:53.710120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7f2f2-0b52-4d9b-a7d7-bcc827f6408d tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '384e46aa04a9460f997765e9e5bbe0a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cebf7dcaa23d49ec8f0a0b39bfa48400', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:53.710818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7f2f2-0b52-4d9b-a7d7-bcc827f6408d tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:53.711099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7f2f2-0b52-4d9b-a7d7-bcc827f6408d tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:53.711377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7f2f2-0b52-4d9b-a7d7-bcc827f6408d tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] PATH_INFO: `/v3/endpoints/47e3cc1ec12c4cfcbfab051f62e5f204` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:53.727204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7f2f2-0b52-4d9b-a7d7-bcc827f6408d tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=47e3cc1ec12c4cfcbfab051f62e5f204)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:53.729839 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-62a7f2f2-0b52-4d9b-a7d7-bcc827f6408d tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:02:53.729839 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:02:53.729839 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:02:53.733367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7f2f2-0b52-4d9b-a7d7-bcc827f6408d tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-62a7f2f2-0b52-4d9b-a7d7-bcc827f6408d tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Invalid input for field 'enabled': 'False' is not of type 'boolean' Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: Failed validating 'type' in schema['properties']['enabled']: Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: {'enum': [True, False, None], 'type': 'boolean'} Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: On instance['enabled']: Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: 'False': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: Failed validating 'type' in schema['properties']['enabled']: Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: {'enum': [True, False, None], 'type': 'boolean'} Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: On instance['enabled']: Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: 'False' Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise error Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'False' is not of type 'boolean' Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application On instance['enabled']: Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application 'False' Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:02:53.744579 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:02:53.746389 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:02:53.746389 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:02:53.746389 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:02:53.746389 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:02:53.746389 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:02:53.746389 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:02:53.746389 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:02:53.746389 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:02:53.746389 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:02:53.746389 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:02:53.746389 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:02:53.746389 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:02:53.746389 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 105, in patch Oct 01 21:02:53.746389 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_update, endpoint) Oct 01 21:02:53.746389 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Oct 01 21:02:53.746389 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Oct 01 21:02:53.746389 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Oct 01 21:02:53.747670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Oct 01 21:02:53.747670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Oct 01 21:02:53.747670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:02:53.747670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Oct 01 21:02:53.747670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Oct 01 21:02:53.747670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:02:53.747670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application On instance['enabled']: Oct 01 21:02:53.747670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application 'False' Oct 01 21:02:53.747670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:02:53.757997 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2072/4211] 10.222.0.17 () {60 vars in 1255 bytes} [Sat Oct 1 21:02:53 2022] PATCH /identity/v3/endpoints/47e3cc1ec12c4cfcbfab051f62e5f204 => generated 283 bytes in 76 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 01 21:02:53.759244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ec4ca-631d-44b0-8209-59dbcd0f4e24 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:53.780051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ec4ca-631d-44b0-8209-59dbcd0f4e24 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:53.789792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea296e-2c7e-4582-a50c-291e2636f38f tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '384e46aa04a9460f997765e9e5bbe0a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cebf7dcaa23d49ec8f0a0b39bfa48400', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:53.791706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea296e-2c7e-4582-a50c-291e2636f38f tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:53.792994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea296e-2c7e-4582-a50c-291e2636f38f tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:53.793424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea296e-2c7e-4582-a50c-291e2636f38f tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] PATH_INFO: `/v3/endpoints/47e3cc1ec12c4cfcbfab051f62e5f204` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:53.796015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea296e-2c7e-4582-a50c-291e2636f38f tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=47e3cc1ec12c4cfcbfab051f62e5f204)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:53.799832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea296e-2c7e-4582-a50c-291e2636f38f tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:53.821142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-47ea296e-2c7e-4582-a50c-291e2636f38f tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '47e3cc1ec12c4cfcbfab051f62e5f204', 'request_id': 'req-47ea296e-2c7e-4582-a50c-291e2636f38f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:53.844217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7f2f2-0b52-4d9b-a7d7-bcc827f6408d tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:53.844928 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2140/4212] 10.222.0.17 () {58 vars in 1236 bytes} [Sat Oct 1 21:02:53 2022] DELETE /identity/v3/endpoints/47e3cc1ec12c4cfcbfab051f62e5f204 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:53.857182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7f2f2-0b52-4d9b-a7d7-bcc827f6408d tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:53.898127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0bc58-5414-4e5f-a130-a3acddc2a6e7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '384e46aa04a9460f997765e9e5bbe0a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cebf7dcaa23d49ec8f0a0b39bfa48400', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:53.899003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0bc58-5414-4e5f-a130-a3acddc2a6e7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:53.899913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0bc58-5414-4e5f-a130-a3acddc2a6e7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:53.899913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0bc58-5414-4e5f-a130-a3acddc2a6e7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] PATH_INFO: `/v3/regions/tempest-region-1592625759` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:53.902385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0bc58-5414-4e5f-a130-a3acddc2a6e7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1592625759)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:53.904691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0bc58-5414-4e5f-a130-a3acddc2a6e7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:53.952621 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2073/4213] 10.222.0.17 () {58 vars in 1209 bytes} [Sat Oct 1 21:02:53 2022] DELETE /identity/v3/regions/tempest-region-1592625759 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:53.959617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea296e-2c7e-4582-a50c-291e2636f38f tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:53.979912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea296e-2c7e-4582-a50c-291e2636f38f tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:54.011675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9064e2a-6768-4076-a849-96c52d51dd66 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '384e46aa04a9460f997765e9e5bbe0a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cebf7dcaa23d49ec8f0a0b39bfa48400', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:54.014917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9064e2a-6768-4076-a849-96c52d51dd66 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:54.015506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9064e2a-6768-4076-a849-96c52d51dd66 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:54.016024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9064e2a-6768-4076-a849-96c52d51dd66 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:54.017105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9064e2a-6768-4076-a849-96c52d51dd66 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:54.020320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9064e2a-6768-4076-a849-96c52d51dd66 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:54.081521 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-d9064e2a-6768-4076-a849-96c52d51dd66 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-258293085. To ensure interoperability, identifiers should be a valid uuid. Oct 01 21:02:54.081521 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 01 21:02:54.081521 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:02:54.132864 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2141/4214] 10.222.0.17 () {60 vars in 1156 bytes} [Sat Oct 1 21:02:53 2022] POST /identity/v3/endpoints => generated 359 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:02:54.136661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0bc58-5414-4e5f-a130-a3acddc2a6e7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:54.156568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0bc58-5414-4e5f-a130-a3acddc2a6e7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:54.207930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529c925-de6f-4527-96c3-1837a626c7c7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '384e46aa04a9460f997765e9e5bbe0a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cebf7dcaa23d49ec8f0a0b39bfa48400', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:54.209549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529c925-de6f-4527-96c3-1837a626c7c7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:54.209549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529c925-de6f-4527-96c3-1837a626c7c7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:54.209774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529c925-de6f-4527-96c3-1837a626c7c7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] PATH_INFO: `/v3/regions/tempest-region-258293085` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:54.217621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b529c925-de6f-4527-96c3-1837a626c7c7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-258293085)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:54.218771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b529c925-de6f-4527-96c3-1837a626c7c7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:54.232131 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2074/4215] 10.222.0.17 () {58 vars in 1203 bytes} [Sat Oct 1 21:02:54 2022] GET /identity/v3/regions/tempest-region-258293085 => generated 178 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:02:54.236475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9064e2a-6768-4076-a849-96c52d51dd66 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:54.257047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9064e2a-6768-4076-a849-96c52d51dd66 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:54.269586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2844642-51e2-4c66-922b-8a9056107efa tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '384e46aa04a9460f997765e9e5bbe0a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cebf7dcaa23d49ec8f0a0b39bfa48400', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:54.270243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2844642-51e2-4c66-922b-8a9056107efa tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:54.270758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2844642-51e2-4c66-922b-8a9056107efa tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:54.271997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2844642-51e2-4c66-922b-8a9056107efa tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] PATH_INFO: `/v3/endpoints/c28c9f5d13654d4b84d393efdd9319d2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:54.289527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2844642-51e2-4c66-922b-8a9056107efa tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=c28c9f5d13654d4b84d393efdd9319d2)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:54.291627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2844642-51e2-4c66-922b-8a9056107efa tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-d2844642-51e2-4c66-922b-8a9056107efa tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Invalid input for field 'enabled': 'True' is not of type 'boolean' Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: Failed validating 'type' in schema['properties']['enabled']: Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: {'enum': [True, False, None], 'type': 'boolean'} Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: On instance['enabled']: Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: 'True': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: Failed validating 'type' in schema['properties']['enabled']: Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: {'enum': [True, False, None], 'type': 'boolean'} Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: On instance['enabled']: Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: 'True' Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise error Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'True' is not of type 'boolean' Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application On instance['enabled']: Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application 'True' Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:02:54.303949 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:02:54.305955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:02:54.305955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:02:54.305955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:02:54.305955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:02:54.305955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:02:54.305955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:02:54.305955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:02:54.305955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:02:54.305955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:02:54.305955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:02:54.305955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:02:54.305955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:02:54.305955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 105, in patch Oct 01 21:02:54.305955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_update, endpoint) Oct 01 21:02:54.305955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Oct 01 21:02:54.305955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Oct 01 21:02:54.305955 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Oct 01 21:02:54.307477 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Oct 01 21:02:54.307477 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Oct 01 21:02:54.307477 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:02:54.307477 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Oct 01 21:02:54.307477 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Oct 01 21:02:54.307477 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:02:54.307477 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application On instance['enabled']: Oct 01 21:02:54.307477 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application 'True' Oct 01 21:02:54.307477 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:02:54.310910 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2142/4216] 10.222.0.17 () {60 vars in 1255 bytes} [Sat Oct 1 21:02:54 2022] PATCH /identity/v3/endpoints/c28c9f5d13654d4b84d393efdd9319d2 => generated 281 bytes in 75 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Oct 01 21:02:54.315864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529c925-de6f-4527-96c3-1837a626c7c7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:54.360341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529c925-de6f-4527-96c3-1837a626c7c7 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:54.371734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b9d10-dc47-49a4-8808-96a83a68f7a8 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '384e46aa04a9460f997765e9e5bbe0a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cebf7dcaa23d49ec8f0a0b39bfa48400', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:54.372959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b9d10-dc47-49a4-8808-96a83a68f7a8 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:54.373681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b9d10-dc47-49a4-8808-96a83a68f7a8 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:54.374160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b9d10-dc47-49a4-8808-96a83a68f7a8 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] PATH_INFO: `/v3/endpoints/c28c9f5d13654d4b84d393efdd9319d2` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:54.376143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b9d10-dc47-49a4-8808-96a83a68f7a8 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=c28c9f5d13654d4b84d393efdd9319d2)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:54.390339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b9d10-dc47-49a4-8808-96a83a68f7a8 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:54.407895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-063b9d10-dc47-49a4-8808-96a83a68f7a8 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': 'c28c9f5d13654d4b84d393efdd9319d2', 'request_id': 'req-063b9d10-dc47-49a4-8808-96a83a68f7a8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:54.427038 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2075/4217] 10.222.0.17 () {58 vars in 1236 bytes} [Sat Oct 1 21:02:54 2022] DELETE /identity/v3/endpoints/c28c9f5d13654d4b84d393efdd9319d2 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:54.433956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2844642-51e2-4c66-922b-8a9056107efa tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:54.447717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2844642-51e2-4c66-922b-8a9056107efa tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:54.470995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b51e8-3e26-4d31-9a8c-3776810d874a tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '384e46aa04a9460f997765e9e5bbe0a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cebf7dcaa23d49ec8f0a0b39bfa48400', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:54.471847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b51e8-3e26-4d31-9a8c-3776810d874a tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:54.472364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b51e8-3e26-4d31-9a8c-3776810d874a tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:54.472741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b51e8-3e26-4d31-9a8c-3776810d874a tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] PATH_INFO: `/v3/regions/tempest-region-258293085` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:54.473840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b51e8-3e26-4d31-9a8c-3776810d874a tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-258293085)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:54.476834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b51e8-3e26-4d31-9a8c-3776810d874a tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:54.542847 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2143/4218] 10.222.0.17 () {58 vars in 1206 bytes} [Sat Oct 1 21:02:54 2022] DELETE /identity/v3/regions/tempest-region-258293085 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:54.548014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b9d10-dc47-49a4-8808-96a83a68f7a8 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:54.563199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b9d10-dc47-49a4-8808-96a83a68f7a8 tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:54.600047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3dec45-69f2-4227-b382-54bbd3a849de tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '384e46aa04a9460f997765e9e5bbe0a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cebf7dcaa23d49ec8f0a0b39bfa48400', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:54.602112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dec45-69f2-4227-b382-54bbd3a849de tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:54.604045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dec45-69f2-4227-b382-54bbd3a849de tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:54.604566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dec45-69f2-4227-b382-54bbd3a849de tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] PATH_INFO: `/v3/services/91f804d7056540d297470749f94c2ddc` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:54.605263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3dec45-69f2-4227-b382-54bbd3a849de tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorizing `identity:delete_service(service_id=91f804d7056540d297470749f94c2ddc)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:54.610150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3dec45-69f2-4227-b382-54bbd3a849de tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:54.668035 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2076/4219] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:02:54 2022] DELETE /identity/v3/services/91f804d7056540d297470749f94c2ddc => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:54.672030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b51e8-3e26-4d31-9a8c-3776810d874a tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:54.732550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:INSERT=8,keystone:DELETE=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:54.824221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b51e8-3e26-4d31-9a8c-3776810d874a tempest-EndpointsNegativeTestJSON-431432730 tempest-EndpointsNegativeTestJSON-431432730-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:54.861115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbaa6f3-3131-44d6-a9b7-ce1684800373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:54.861795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbaa6f3-3131-44d6-a9b7-ce1684800373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:54.861940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbaa6f3-3131-44d6-a9b7-ce1684800373 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:54.862219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbaa6f3-3131-44d6-a9b7-ce1684800373 admin admin] PATH_INFO: `/v3/users/a75b672d0f4b458fbd4e82289c4d0eb2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:54.864002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbaa6f3-3131-44d6-a9b7-ce1684800373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a75b672d0f4b458fbd4e82289c4d0eb2)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:54.868344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbaa6f3-3131-44d6-a9b7-ce1684800373 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:54.975752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-efbaa6f3-3131-44d6-a9b7-ce1684800373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a75b672d0f4b458fbd4e82289c4d0eb2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:54.995973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-efbaa6f3-3131-44d6-a9b7-ce1684800373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a75b672d0f4b458fbd4e82289c4d0eb2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:55.031914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-efbaa6f3-3131-44d6-a9b7-ce1684800373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a75b672d0f4b458fbd4e82289c4d0eb2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:55.032690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-efbaa6f3-3131-44d6-a9b7-ce1684800373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a75b672d0f4b458fbd4e82289c4d0eb2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:55.059517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-efbaa6f3-3131-44d6-a9b7-ce1684800373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a75b672d0f4b458fbd4e82289c4d0eb2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:55.080164 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2144/4220] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:02:54 2022] DELETE /identity/v3/users/a75b672d0f4b458fbd4e82289c4d0eb2 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:55.085369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f944b-b94c-4247-a9d4-ee0a64cb150c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:55.085652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f944b-b94c-4247-a9d4-ee0a64cb150c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:55.085929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f944b-b94c-4247-a9d4-ee0a64cb150c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:55.116385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9c9f944b-b94c-4247-a9d4-ee0a64cb150c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:55.118194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9c9f944b-b94c-4247-a9d4-ee0a64cb150c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:55.223603 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2077/4221] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:55.238687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbaa6f3-3131-44d6-a9b7-ce1684800373 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:55.240870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-efbaa6f3-3131-44d6-a9b7-ce1684800373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:55.456751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbaa6f3-3131-44d6-a9b7-ce1684800373 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:55.468319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5617ad-4259-40e3-8432-d1c019d38859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:55.469011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5617ad-4259-40e3-8432-d1c019d38859 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:55.469393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5617ad-4259-40e3-8432-d1c019d38859 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:55.469764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5617ad-4259-40e3-8432-d1c019d38859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:55.504302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5617ad-4259-40e3-8432-d1c019d38859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:55.511375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5617ad-4259-40e3-8432-d1c019d38859 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:55.553527 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2145/4222] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:02:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:02:56.546409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f944b-b94c-4247-a9d4-ee0a64cb150c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:56.547910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9c9f944b-b94c-4247-a9d4-ee0a64cb150c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:56.579677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f944b-b94c-4247-a9d4-ee0a64cb150c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:56.588522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abaeeda-2e24-44c0-8f9f-6250d06b26d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:56.589551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abaeeda-2e24-44c0-8f9f-6250d06b26d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:56.589989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abaeeda-2e24-44c0-8f9f-6250d06b26d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:56.590915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abaeeda-2e24-44c0-8f9f-6250d06b26d8 admin admin] PATH_INFO: `/v3/projects/a6f2c8a6f73749a49068fe4f08ea057b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:56.593075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abaeeda-2e24-44c0-8f9f-6250d06b26d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6f2c8a6f73749a49068fe4f08ea057b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:56.595046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abaeeda-2e24-44c0-8f9f-6250d06b26d8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:56.615862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:56.665071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0abaeeda-2e24-44c0-8f9f-6250d06b26d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6f2c8a6f73749a49068fe4f08ea057b', 'request_id': 'req-0abaeeda-2e24-44c0-8f9f-6250d06b26d8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:56.676659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0abaeeda-2e24-44c0-8f9f-6250d06b26d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6f2c8a6f73749a49068fe4f08ea057b', 'request_id': 'req-0abaeeda-2e24-44c0-8f9f-6250d06b26d8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:56.685442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0abaeeda-2e24-44c0-8f9f-6250d06b26d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6f2c8a6f73749a49068fe4f08ea057b', 'request_id': 'req-0abaeeda-2e24-44c0-8f9f-6250d06b26d8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:56.702267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0abaeeda-2e24-44c0-8f9f-6250d06b26d8 admin admin] The token cache is being invalidate because project a6f2c8a6f73749a49068fe4f08ea057b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:02:56.702659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0abaeeda-2e24-44c0-8f9f-6250d06b26d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:56.703244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0abaeeda-2e24-44c0-8f9f-6250d06b26d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:56.704731 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2078/4223] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:02:56 2022] DELETE /identity/v3/projects/a6f2c8a6f73749a49068fe4f08ea057b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:56.709421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5617ad-4259-40e3-8432-d1c019d38859 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:56.720389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5617ad-4259-40e3-8432-d1c019d38859 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:56.750105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:56.822002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428fbcd-62e7-45fd-b246-35ec3732aacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:56.823154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428fbcd-62e7-45fd-b246-35ec3732aacb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:56.823843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428fbcd-62e7-45fd-b246-35ec3732aacb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:56.824233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428fbcd-62e7-45fd-b246-35ec3732aacb admin admin] PATH_INFO: `/v3/users/384e46aa04a9460f997765e9e5bbe0a0` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:56.828632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428fbcd-62e7-45fd-b246-35ec3732aacb admin admin] RBAC: Authorizing `identity:delete_user(user_id=384e46aa04a9460f997765e9e5bbe0a0)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:56.831939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428fbcd-62e7-45fd-b246-35ec3732aacb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:56.954205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1428fbcd-62e7-45fd-b246-35ec3732aacb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '384e46aa04a9460f997765e9e5bbe0a0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:56.975782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1428fbcd-62e7-45fd-b246-35ec3732aacb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '384e46aa04a9460f997765e9e5bbe0a0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:57.001253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1428fbcd-62e7-45fd-b246-35ec3732aacb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '384e46aa04a9460f997765e9e5bbe0a0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:57.002589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1428fbcd-62e7-45fd-b246-35ec3732aacb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '384e46aa04a9460f997765e9e5bbe0a0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:57.019926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1428fbcd-62e7-45fd-b246-35ec3732aacb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '384e46aa04a9460f997765e9e5bbe0a0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:57.027606 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2146/4224] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:02:56 2022] DELETE /identity/v3/users/384e46aa04a9460f997765e9e5bbe0a0 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:57.965437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abaeeda-2e24-44c0-8f9f-6250d06b26d8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:57.967811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0abaeeda-2e24-44c0-8f9f-6250d06b26d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:58.004686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abaeeda-2e24-44c0-8f9f-6250d06b26d8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:58.220364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb358e-6388-49d7-a9ae-6f2712d49ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:58.221028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb358e-6388-49d7-a9ae-6f2712d49ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:58.221309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb358e-6388-49d7-a9ae-6f2712d49ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:58.221582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb358e-6388-49d7-a9ae-6f2712d49ddd admin admin] PATH_INFO: `/v3/projects/cebf7dcaa23d49ec8f0a0b39bfa48400` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:58.224997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb358e-6388-49d7-a9ae-6f2712d49ddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cebf7dcaa23d49ec8f0a0b39bfa48400)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:58.228779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb358e-6388-49d7-a9ae-6f2712d49ddd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:58.362441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d9bb358e-6388-49d7-a9ae-6f2712d49ddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cebf7dcaa23d49ec8f0a0b39bfa48400', 'request_id': 'req-d9bb358e-6388-49d7-a9ae-6f2712d49ddd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:58.380312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d9bb358e-6388-49d7-a9ae-6f2712d49ddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cebf7dcaa23d49ec8f0a0b39bfa48400', 'request_id': 'req-d9bb358e-6388-49d7-a9ae-6f2712d49ddd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:58.390702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d9bb358e-6388-49d7-a9ae-6f2712d49ddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cebf7dcaa23d49ec8f0a0b39bfa48400', 'request_id': 'req-d9bb358e-6388-49d7-a9ae-6f2712d49ddd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:58.416511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d9bb358e-6388-49d7-a9ae-6f2712d49ddd admin admin] The token cache is being invalidate because project cebf7dcaa23d49ec8f0a0b39bfa48400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:02:58.416889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d9bb358e-6388-49d7-a9ae-6f2712d49ddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:58.417646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d9bb358e-6388-49d7-a9ae-6f2712d49ddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:02:58.420041 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2079/4225] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:02:57 2022] DELETE /identity/v3/projects/cebf7dcaa23d49ec8f0a0b39bfa48400 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:58.424496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c57f3-3d1d-4f99-be65-7e245bb7bde2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:58.424969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c57f3-3d1d-4f99-be65-7e245bb7bde2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:58.425066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c57f3-3d1d-4f99-be65-7e245bb7bde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:58.465740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-aa6c57f3-3d1d-4f99-be65-7e245bb7bde2 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:58.468826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-aa6c57f3-3d1d-4f99-be65-7e245bb7bde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:58.538754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f755152-fcb0-425a-9a8a-c115cc761d6e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:58.539192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f755152-fcb0-425a-9a8a-c115cc761d6e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:58.539570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f755152-fcb0-425a-9a8a-c115cc761d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:58.600157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5f755152-fcb0-425a-9a8a-c115cc761d6e None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:58.602084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5f755152-fcb0-425a-9a8a-c115cc761d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:58.698960 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2147/4226] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:58.701398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09622e42-eab6-4158-8c90-9b3f65082541 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:58.701555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09622e42-eab6-4158-8c90-9b3f65082541 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:58.702163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09622e42-eab6-4158-8c90-9b3f65082541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:58.706917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:58.732505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-09622e42-eab6-4158-8c90-9b3f65082541 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:58.733880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-09622e42-eab6-4158-8c90-9b3f65082541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:58.765279 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2148/4227] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:58.766220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09622e42-eab6-4158-8c90-9b3f65082541 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:58.792833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09622e42-eab6-4158-8c90-9b3f65082541 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:58.810976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ddc30-da2a-4207-b768-2c4b4ef0a079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:58.811864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ddc30-da2a-4207-b768-2c4b4ef0a079 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:58.812496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ddc30-da2a-4207-b768-2c4b4ef0a079 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:58.812838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ddc30-da2a-4207-b768-2c4b4ef0a079 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:58.813919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ddc30-da2a-4207-b768-2c4b4ef0a079 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:02:58.816989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ddc30-da2a-4207-b768-2c4b4ef0a079 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:58.818841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ddc30-da2a-4207-b768-2c4b4ef0a079 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:58.827384 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2080/4228] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:02:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:02:58.859998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f755152-fcb0-425a-9a8a-c115cc761d6e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:58.863117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5f755152-fcb0-425a-9a8a-c115cc761d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:58.867596 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2149/4229] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:02:58.868533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ddc30-da2a-4207-b768-2c4b4ef0a079 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:58.884554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ddc30-da2a-4207-b768-2c4b4ef0a079 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:58.891738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fb632-00cb-42f1-8a92-490d2603824f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:58.892469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fb632-00cb-42f1-8a92-490d2603824f admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:58.892928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fb632-00cb-42f1-8a92-490d2603824f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:58.893290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fb632-00cb-42f1-8a92-490d2603824f admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:58.894759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fb632-00cb-42f1-8a92-490d2603824f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:58.897591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fb632-00cb-42f1-8a92-490d2603824f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:58.971923 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2150/4230] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:02:58 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:02:58.982324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fb632-00cb-42f1-8a92-490d2603824f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:59.005808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:59.098433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f755152-fcb0-425a-9a8a-c115cc761d6e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:59.196928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fb632-00cb-42f1-8a92-490d2603824f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:59.210307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd2394-40ad-482e-b878-aea76b328677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:59.211098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd2394-40ad-482e-b878-aea76b328677 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:59.211742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd2394-40ad-482e-b878-aea76b328677 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:59.213739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd2394-40ad-482e-b878-aea76b328677 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:59.214520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd2394-40ad-482e-b878-aea76b328677 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:59.216934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd2394-40ad-482e-b878-aea76b328677 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:59.253050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306640fb-3ba4-4ccd-a0bc-c22d676ae660 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:59.256869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306640fb-3ba4-4ccd-a0bc-c22d676ae660 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:59.259001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306640fb-3ba4-4ccd-a0bc-c22d676ae660 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:59.259550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306640fb-3ba4-4ccd-a0bc-c22d676ae660 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:59.264140 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2151/4231] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:02:58 2022] POST /identity/v3/users => generated 289 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:02:59.281363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd2394-40ad-482e-b878-aea76b328677 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:59.297331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd2394-40ad-482e-b878-aea76b328677 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:59.348747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a345d2-f15a-4f1d-a030-989c2c3eb837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:59.349889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a345d2-f15a-4f1d-a030-989c2c3eb837 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:59.350165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a345d2-f15a-4f1d-a030-989c2c3eb837 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:59.350432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a345d2-f15a-4f1d-a030-989c2c3eb837 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:59.362267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a345d2-f15a-4f1d-a030-989c2c3eb837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:59.362267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a345d2-f15a-4f1d-a030-989c2c3eb837 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:59.423548 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2152/4232] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:02:59.426906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a345d2-f15a-4f1d-a030-989c2c3eb837 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:59.453436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a345d2-f15a-4f1d-a030-989c2c3eb837 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:59.485074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bc3ef-11ff-4ad4-a987-c03a153870f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:59.487723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bc3ef-11ff-4ad4-a987-c03a153870f8 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:59.488012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bc3ef-11ff-4ad4-a987-c03a153870f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:59.488285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bc3ef-11ff-4ad4-a987-c03a153870f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:59.488945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407bc3ef-11ff-4ad4-a987-c03a153870f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:59.492997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407bc3ef-11ff-4ad4-a987-c03a153870f8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:59.500298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306640fb-3ba4-4ccd-a0bc-c22d676ae660 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:59.501950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306640fb-3ba4-4ccd-a0bc-c22d676ae660 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:59.529880 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2081/4234] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:02:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 670 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:02:59.532102 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2153/4234] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:02:59.534270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306640fb-3ba4-4ccd-a0bc-c22d676ae660 service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:59.558825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306640fb-3ba4-4ccd-a0bc-c22d676ae660 service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:02:59.570025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca21ebe-f685-4080-b660-423be691c550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:02:59.570751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca21ebe-f685-4080-b660-423be691c550 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:59.571032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca21ebe-f685-4080-b660-423be691c550 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:59.571308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca21ebe-f685-4080-b660-423be691c550 admin admin] PATH_INFO: `/v3/projects/083f87d7fcb54aed94ba016ec32d6423/users/fb0be548096544d1a1ba68b41e8eb977/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:59.576969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:59.612503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca21ebe-f685-4080-b660-423be691c550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=083f87d7fcb54aed94ba016ec32d6423, user_id=fb0be548096544d1a1ba68b41e8eb977, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:02:59.631431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca21ebe-f685-4080-b660-423be691c550 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:02:59.670425 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2082/4235] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:02:59 2022] PUT /identity/v3/projects/083f87d7fcb54aed94ba016ec32d6423/users/fb0be548096544d1a1ba68b41e8eb977/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:02:59.681596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5cf7d-57b1-450e-b760-722a0d09df53 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:02:59.682142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5cf7d-57b1-450e-b760-722a0d09df53 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:02:59.683408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5cf7d-57b1-450e-b760-722a0d09df53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:02:59.796069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca21ebe-f685-4080-b660-423be691c550 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:02:59.799697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-aca21ebe-f685-4080-b660-423be691c550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:02:59.813298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c3f5cf7d-57b1-450e-b760-722a0d09df53 None None] MFA Rules not processed for user `fb0be548096544d1a1ba68b41e8eb977`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:02:59.814583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c3f5cf7d-57b1-450e-b760-722a0d09df53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:00.038139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca21ebe-f685-4080-b660-423be691c550 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:00.048000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6e4b7-4e9a-4c5b-a5b8-8a89ae3eb736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:00.048660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6e4b7-4e9a-4c5b-a5b8-8a89ae3eb736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:00.048940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6e4b7-4e9a-4c5b-a5b8-8a89ae3eb736 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:00.051429 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2154/4236] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:02:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:00.051571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6e4b7-4e9a-4c5b-a5b8-8a89ae3eb736 admin admin] PATH_INFO: `/v3/users/6246e90f7e1b4fb39f6ed6a90cb67d61` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:00.053219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6e4b7-4e9a-4c5b-a5b8-8a89ae3eb736 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6246e90f7e1b4fb39f6ed6a90cb67d61)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:00.056056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5cf7d-57b1-450e-b760-722a0d09df53 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:00.060736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6e4b7-4e9a-4c5b-a5b8-8a89ae3eb736 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:00.077053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5cf7d-57b1-450e-b760-722a0d09df53 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:00.084863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8ec75-1656-46e1-a1ee-ebbc2ed401ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:00.086217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8ec75-1656-46e1-a1ee-ebbc2ed401ca admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:00.086518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8ec75-1656-46e1-a1ee-ebbc2ed401ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:00.086828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8ec75-1656-46e1-a1ee-ebbc2ed401ca admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:00.087473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8ec75-1656-46e1-a1ee-ebbc2ed401ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:00.089406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8ec75-1656-46e1-a1ee-ebbc2ed401ca admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:00.131961 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2155/4237] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:00 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:00.135836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8ec75-1656-46e1-a1ee-ebbc2ed401ca admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:00.212856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:00.231350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-90f6e4b7-4e9a-4c5b-a5b8-8a89ae3eb736 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6246e90f7e1b4fb39f6ed6a90cb67d61'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:00.234101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-90f6e4b7-4e9a-4c5b-a5b8-8a89ae3eb736 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6246e90f7e1b4fb39f6ed6a90cb67d61'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:00.234774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-90f6e4b7-4e9a-4c5b-a5b8-8a89ae3eb736 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6246e90f7e1b4fb39f6ed6a90cb67d61'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:00.263768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-90f6e4b7-4e9a-4c5b-a5b8-8a89ae3eb736 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6246e90f7e1b4fb39f6ed6a90cb67d61'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:00.274642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8ec75-1656-46e1-a1ee-ebbc2ed401ca admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:00.290228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94369cca-53d4-432d-a6ec-0f564f3c8840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:00.292914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94369cca-53d4-432d-a6ec-0f564f3c8840 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:00.293222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94369cca-53d4-432d-a6ec-0f564f3c8840 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:00.293503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94369cca-53d4-432d-a6ec-0f564f3c8840 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:00.294178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94369cca-53d4-432d-a6ec-0f564f3c8840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:00.298889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-90f6e4b7-4e9a-4c5b-a5b8-8a89ae3eb736 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6246e90f7e1b4fb39f6ed6a90cb67d61'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:00.300602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94369cca-53d4-432d-a6ec-0f564f3c8840 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:00.327095 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2083/4238] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:02:59 2022] DELETE /identity/v3/users/6246e90f7e1b4fb39f6ed6a90cb67d61 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:00.332401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e211db1-a48f-482a-867d-7b7c3f094cc6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:00.332744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e211db1-a48f-482a-867d-7b7c3f094cc6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:00.333028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e211db1-a48f-482a-867d-7b7c3f094cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:00.349529 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2156/4239] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:03:00 2022] POST /identity/v3/users => generated 295 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:00.352049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94369cca-53d4-432d-a6ec-0f564f3c8840 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:00.361018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-94369cca-53d4-432d-a6ec-0f564f3c8840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:00.376860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4e211db1-a48f-482a-867d-7b7c3f094cc6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:00.378090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4e211db1-a48f-482a-867d-7b7c3f094cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:00.390785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94369cca-53d4-432d-a6ec-0f564f3c8840 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:00.518905 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2084/4240] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:03:00.526274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d519b-5d7e-4a30-9069-b20c39e09278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:00.527558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d519b-5d7e-4a30-9069-b20c39e09278 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:00.528365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d519b-5d7e-4a30-9069-b20c39e09278 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:00.528855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d519b-5d7e-4a30-9069-b20c39e09278 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:00.529272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e211db1-a48f-482a-867d-7b7c3f094cc6 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:00.529967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d519b-5d7e-4a30-9069-b20c39e09278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:00.530707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4e211db1-a48f-482a-867d-7b7c3f094cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:00.532052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d519b-5d7e-4a30-9069-b20c39e09278 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:00.536681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:00.549212 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2157/4241] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:00.553561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d519b-5d7e-4a30-9069-b20c39e09278 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:00.588438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d519b-5d7e-4a30-9069-b20c39e09278 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:00.606425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3141000-6d6a-4b0d-a6c5-2611df9bd015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:00.607638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3141000-6d6a-4b0d-a6c5-2611df9bd015 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:00.607929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3141000-6d6a-4b0d-a6c5-2611df9bd015 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:00.608226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3141000-6d6a-4b0d-a6c5-2611df9bd015 admin admin] PATH_INFO: `/v3/projects/26ae38a425c248cca0f31e8a31405c1e/users/354c4495648c4c6a9630b2f600d2c17e/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:00.664749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3141000-6d6a-4b0d-a6c5-2611df9bd015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26ae38a425c248cca0f31e8a31405c1e, user_id=354c4495648c4c6a9630b2f600d2c17e, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:00.667625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3141000-6d6a-4b0d-a6c5-2611df9bd015 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:00.689944 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2158/4242] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:00 2022] PUT /identity/v3/projects/26ae38a425c248cca0f31e8a31405c1e/users/354c4495648c4c6a9630b2f600d2c17e/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:00.693466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3141000-6d6a-4b0d-a6c5-2611df9bd015 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:00.702560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e211db1-a48f-482a-867d-7b7c3f094cc6 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:00.780270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef69bc-2ddf-4ff2-b41f-8c7f552f0c57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:00.781041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef69bc-2ddf-4ff2-b41f-8c7f552f0c57 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:00.781391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef69bc-2ddf-4ff2-b41f-8c7f552f0c57 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:00.781979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef69bc-2ddf-4ff2-b41f-8c7f552f0c57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:00.790351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3141000-6d6a-4b0d-a6c5-2611df9bd015 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:00.793348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef69bc-2ddf-4ff2-b41f-8c7f552f0c57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:00.795248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef69bc-2ddf-4ff2-b41f-8c7f552f0c57 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:00.798670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281059b5-7395-4388-9c04-d6f4e84ab88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:00.799369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281059b5-7395-4388-9c04-d6f4e84ab88e admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:00.799752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281059b5-7395-4388-9c04-d6f4e84ab88e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:00.800094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281059b5-7395-4388-9c04-d6f4e84ab88e admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:00.800808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281059b5-7395-4388-9c04-d6f4e84ab88e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:00.802692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281059b5-7395-4388-9c04-d6f4e84ab88e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:00.808083 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2085/4243] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:03:00.809259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef69bc-2ddf-4ff2-b41f-8c7f552f0c57 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:00.810982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bfef69bc-2ddf-4ff2-b41f-8c7f552f0c57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:00.816601 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2159/4244] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:00.820858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281059b5-7395-4388-9c04-d6f4e84ab88e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:00.837713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281059b5-7395-4388-9c04-d6f4e84ab88e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:00.840301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef69bc-2ddf-4ff2-b41f-8c7f552f0c57 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:00.847242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1419b6a-adf2-4b4c-a94f-28afd02c78fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:00.848766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1419b6a-adf2-4b4c-a94f-28afd02c78fb admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:00.848766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1419b6a-adf2-4b4c-a94f-28afd02c78fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:00.848766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1419b6a-adf2-4b4c-a94f-28afd02c78fb admin admin] PATH_INFO: `/v3/projects/26ae38a425c248cca0f31e8a31405c1e/users/354c4495648c4c6a9630b2f600d2c17e/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:00.851069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39486f9a-1144-4540-9f73-c90e8bb294e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:00.851329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1419b6a-adf2-4b4c-a94f-28afd02c78fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=26ae38a425c248cca0f31e8a31405c1e, user_id=354c4495648c4c6a9630b2f600d2c17e, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:00.854118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1419b6a-adf2-4b4c-a94f-28afd02c78fb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:00.854826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39486f9a-1144-4540-9f73-c90e8bb294e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:00.855346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39486f9a-1144-4540-9f73-c90e8bb294e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:00.855780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39486f9a-1144-4540-9f73-c90e8bb294e1 admin admin] PATH_INFO: `/v3/users/b4098dddfc2f442895545cc758ee430a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:00.858952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39486f9a-1144-4540-9f73-c90e8bb294e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4098dddfc2f442895545cc758ee430a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:00.862720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39486f9a-1144-4540-9f73-c90e8bb294e1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:00.874008 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2160/4245] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:00 2022] PUT /identity/v3/projects/26ae38a425c248cca0f31e8a31405c1e/users/354c4495648c4c6a9630b2f600d2c17e/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:00.884677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fb646-d888-47cf-8f1a-695dd9451670 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:00.885109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fb646-d888-47cf-8f1a-695dd9451670 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:00.885462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fb646-d888-47cf-8f1a-695dd9451670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:00.942121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-459fb646-d888-47cf-8f1a-695dd9451670 None None] MFA Rules not processed for user `354c4495648c4c6a9630b2f600d2c17e`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:00.944191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-459fb646-d888-47cf-8f1a-695dd9451670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:00.965246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:00.974873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-39486f9a-1144-4540-9f73-c90e8bb294e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4098dddfc2f442895545cc758ee430a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:00.981751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-39486f9a-1144-4540-9f73-c90e8bb294e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4098dddfc2f442895545cc758ee430a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:00.982808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-39486f9a-1144-4540-9f73-c90e8bb294e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4098dddfc2f442895545cc758ee430a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:00.995262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-39486f9a-1144-4540-9f73-c90e8bb294e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4098dddfc2f442895545cc758ee430a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:01.015676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-39486f9a-1144-4540-9f73-c90e8bb294e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4098dddfc2f442895545cc758ee430a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:01.036327 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2086/4246] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:00 2022] DELETE /identity/v3/users/b4098dddfc2f442895545cc758ee430a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:01.042524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb983f-8239-4bbf-89ce-bf895213aa19 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:01.043526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb983f-8239-4bbf-89ce-bf895213aa19 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:01.043756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb983f-8239-4bbf-89ce-bf895213aa19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:01.080947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6acb983f-8239-4bbf-89ce-bf895213aa19 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:01.082335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6acb983f-8239-4bbf-89ce-bf895213aa19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:01.128554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:01.199187 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2161/4247] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:01.204833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459fb646-d888-47cf-8f1a-695dd9451670 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:01.211308 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2087/4248] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:01.218216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459fb646-d888-47cf-8f1a-695dd9451670 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:01.250755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb983f-8239-4bbf-89ce-bf895213aa19 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:01.252291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6acb983f-8239-4bbf-89ce-bf895213aa19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:01.308464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20d4ef-ffc8-4a1c-a701-1e1e59e14a53 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '354c4495648c4c6a9630b2f600d2c17e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ae38a425c248cca0f31e8a31405c1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:01.309297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20d4ef-ffc8-4a1c-a701-1e1e59e14a53 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:01.319984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20d4ef-ffc8-4a1c-a701-1e1e59e14a53 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:01.322292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20d4ef-ffc8-4a1c-a701-1e1e59e14a53 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:01.323169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e20d4ef-ffc8-4a1c-a701-1e1e59e14a53 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:01.324459 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-2e20d4ef-ffc8-4a1c-a701-1e1e59e14a53 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:01.324459 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:01.324459 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:01.328006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e20d4ef-ffc8-4a1c-a701-1e1e59e14a53 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:01.353334 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2162/4249] 10.222.0.17 () {60 vars in 1185 bytes} [Sat Oct 1 21:03:01 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:01.354290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20d4ef-ffc8-4a1c-a701-1e1e59e14a53 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:01.356641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2e20d4ef-ffc8-4a1c-a701-1e1e59e14a53 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:01.373925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20d4ef-ffc8-4a1c-a701-1e1e59e14a53 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:01.379719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17890ef-ec45-495d-9720-59bba78ade43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:01.383346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17890ef-ec45-495d-9720-59bba78ade43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:01.383639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17890ef-ec45-495d-9720-59bba78ade43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:01.383924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17890ef-ec45-495d-9720-59bba78ade43 admin admin] PATH_INFO: `/v3/projects/1c3bf48f17ec49c8abbaa9eec9a4f93f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:01.385543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17890ef-ec45-495d-9720-59bba78ade43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c3bf48f17ec49c8abbaa9eec9a4f93f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:01.387283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17890ef-ec45-495d-9720-59bba78ade43 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:01.387431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb983f-8239-4bbf-89ce-bf895213aa19 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:01.391615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc189934-bbf8-46b5-8f0b-924791eb03d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:01.392230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc189934-bbf8-46b5-8f0b-924791eb03d5 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:01.392512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc189934-bbf8-46b5-8f0b-924791eb03d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:01.392789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc189934-bbf8-46b5-8f0b-924791eb03d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:01.411617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc189934-bbf8-46b5-8f0b-924791eb03d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:01.413022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc189934-bbf8-46b5-8f0b-924791eb03d5 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:01.428659 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2088/4250] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:03:01.429835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc189934-bbf8-46b5-8f0b-924791eb03d5 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:01.442000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc189934-bbf8-46b5-8f0b-924791eb03d5 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:01.459355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c17890ef-ec45-495d-9720-59bba78ade43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c3bf48f17ec49c8abbaa9eec9a4f93f', 'request_id': 'req-c17890ef-ec45-495d-9720-59bba78ade43'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:01.468329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c17890ef-ec45-495d-9720-59bba78ade43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c3bf48f17ec49c8abbaa9eec9a4f93f', 'request_id': 'req-c17890ef-ec45-495d-9720-59bba78ade43'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:01.477147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c17890ef-ec45-495d-9720-59bba78ade43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c3bf48f17ec49c8abbaa9eec9a4f93f', 'request_id': 'req-c17890ef-ec45-495d-9720-59bba78ade43'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:01.496088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c17890ef-ec45-495d-9720-59bba78ade43 admin admin] The token cache is being invalidate because project 1c3bf48f17ec49c8abbaa9eec9a4f93f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:01.496429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c17890ef-ec45-495d-9720-59bba78ade43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:01.497212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c17890ef-ec45-495d-9720-59bba78ade43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:01.499022 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2163/4251] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:01 2022] DELETE /identity/v3/projects/1c3bf48f17ec49c8abbaa9eec9a4f93f => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:01.506086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f5224-6c86-46d3-ac58-6cbd5882b791 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:01.507032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f5224-6c86-46d3-ac58-6cbd5882b791 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:01.507519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f5224-6c86-46d3-ac58-6cbd5882b791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:01.527076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd9ade-b6de-4759-89b5-362680356568 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '354c4495648c4c6a9630b2f600d2c17e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ae38a425c248cca0f31e8a31405c1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:01.527732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd9ade-b6de-4759-89b5-362680356568 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:01.528014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd9ade-b6de-4759-89b5-362680356568 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:01.528293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd9ade-b6de-4759-89b5-362680356568 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/31010811798f4fdc99131f8ecae7b590` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:01.528878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abd9ade-b6de-4759-89b5-362680356568 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=31010811798f4fdc99131f8ecae7b590)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:01.529938 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-7abd9ade-b6de-4759-89b5-362680356568 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_consumer": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:01.529938 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:01.529938 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:01.532664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8a3f5224-6c86-46d3-ac58-6cbd5882b791 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:01.534050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8a3f5224-6c86-46d3-ac58-6cbd5882b791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:01.535425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abd9ade-b6de-4759-89b5-362680356568 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:01.551131 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2089/4252] 10.222.0.17 () {58 vars in 1263 bytes} [Sat Oct 1 21:03:01 2022] GET /identity/v3/OS-OAUTH1/consumers/31010811798f4fdc99131f8ecae7b590 => generated 220 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:01.556793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd9ade-b6de-4759-89b5-362680356568 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:01.558784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7abd9ade-b6de-4759-89b5-362680356568 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:01.562311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:01.580282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd9ade-b6de-4759-89b5-362680356568 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:01.588449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc930128-da7b-45ce-aa82-d8589c71932e tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '354c4495648c4c6a9630b2f600d2c17e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ae38a425c248cca0f31e8a31405c1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:01.589086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc930128-da7b-45ce-aa82-d8589c71932e tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:01.589367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc930128-da7b-45ce-aa82-d8589c71932e tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:01.590130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc930128-da7b-45ce-aa82-d8589c71932e tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/31010811798f4fdc99131f8ecae7b590` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:01.591279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc930128-da7b-45ce-aa82-d8589c71932e tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=31010811798f4fdc99131f8ecae7b590)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:01.592420 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-dc930128-da7b-45ce-aa82-d8589c71932e tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:01.592420 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:01.592420 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:01.593472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc930128-da7b-45ce-aa82-d8589c71932e tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:01.593743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dc930128-da7b-45ce-aa82-d8589c71932e tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invalidating token cache because consumer 31010811798f4fdc99131f8ecae7b590 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:01.594025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dc930128-da7b-45ce-aa82-d8589c71932e tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:01.594766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dc930128-da7b-45ce-aa82-d8589c71932e tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:01.617068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dc930128-da7b-45ce-aa82-d8589c71932e tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '31010811798f4fdc99131f8ecae7b590', 'request_id': 'req-dc930128-da7b-45ce-aa82-d8589c71932e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:01.626871 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2164/4253] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:01.635688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7536b-2b52-4550-815d-a61caf2aa89c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:01.635989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7536b-2b52-4550-815d-a61caf2aa89c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:01.636319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7536b-2b52-4550-815d-a61caf2aa89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:01.643606 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2090/4254] 10.222.0.17 () {58 vars in 1266 bytes} [Sat Oct 1 21:03:01 2022] DELETE /identity/v3/OS-OAUTH1/consumers/31010811798f4fdc99131f8ecae7b590 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:01.646578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc930128-da7b-45ce-aa82-d8589c71932e tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:01.651337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-dc930128-da7b-45ce-aa82-d8589c71932e tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:01.663693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3bf7536b-2b52-4550-815d-a61caf2aa89c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:01.664993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3bf7536b-2b52-4550-815d-a61caf2aa89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:01.679778 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2165/4255] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:01.680624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7536b-2b52-4550-815d-a61caf2aa89c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:01.685788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3bf7536b-2b52-4550-815d-a61caf2aa89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:01.686352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc930128-da7b-45ce-aa82-d8589c71932e tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:01.691308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79d3e8-2106-47e3-a105-07350182f40f tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '354c4495648c4c6a9630b2f600d2c17e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ae38a425c248cca0f31e8a31405c1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:01.691936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79d3e8-2106-47e3-a105-07350182f40f tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:01.692210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79d3e8-2106-47e3-a105-07350182f40f tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:01.692489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79d3e8-2106-47e3-a105-07350182f40f tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:01.693696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79d3e8-2106-47e3-a105-07350182f40f tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:01.694811 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-de79d3e8-2106-47e3-a105-07350182f40f tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:01.694811 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:01.694811 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:01.695628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79d3e8-2106-47e3-a105-07350182f40f tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:01.711053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7536b-2b52-4550-815d-a61caf2aa89c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:01.719630 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2091/4256] 10.222.0.17 () {60 vars in 1185 bytes} [Sat Oct 1 21:03:01 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 267 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:01.719863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a9735-91dc-4b83-a215-d9b4298775d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:01.720412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a9735-91dc-4b83-a215-d9b4298775d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:01.720530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79d3e8-2106-47e3-a105-07350182f40f tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:01.720697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a9735-91dc-4b83-a215-d9b4298775d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:01.720963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a9735-91dc-4b83-a215-d9b4298775d6 admin admin] PATH_INFO: `/v3/projects/dbbe49ca8a0b43c2b7bb4b886c4393e8` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:01.723263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a9735-91dc-4b83-a215-d9b4298775d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbbe49ca8a0b43c2b7bb4b886c4393e8)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:01.725912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a9735-91dc-4b83-a215-d9b4298775d6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:01.735505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79d3e8-2106-47e3-a105-07350182f40f tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:01.740943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b77f4-dcff-423c-8044-9bec98417e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:01.741681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b77f4-dcff-423c-8044-9bec98417e24 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:01.741988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b77f4-dcff-423c-8044-9bec98417e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:01.742259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b77f4-dcff-423c-8044-9bec98417e24 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:01.742972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b77f4-dcff-423c-8044-9bec98417e24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:03:01.743275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b77f4-dcff-423c-8044-9bec98417e24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:01.750542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b77f4-dcff-423c-8044-9bec98417e24 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:01.764489 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2092/4257] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:01.764489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b77f4-dcff-423c-8044-9bec98417e24 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:01.776987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b77f4-dcff-423c-8044-9bec98417e24 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:01.787050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b56a9735-91dc-4b83-a215-d9b4298775d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbbe49ca8a0b43c2b7bb4b886c4393e8', 'request_id': 'req-b56a9735-91dc-4b83-a215-d9b4298775d6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:01.793760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b56a9735-91dc-4b83-a215-d9b4298775d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbbe49ca8a0b43c2b7bb4b886c4393e8', 'request_id': 'req-b56a9735-91dc-4b83-a215-d9b4298775d6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:01.800054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b56a9735-91dc-4b83-a215-d9b4298775d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbbe49ca8a0b43c2b7bb4b886c4393e8', 'request_id': 'req-b56a9735-91dc-4b83-a215-d9b4298775d6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:01.811697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:01.818018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b56a9735-91dc-4b83-a215-d9b4298775d6 admin admin] The token cache is being invalidate because project dbbe49ca8a0b43c2b7bb4b886c4393e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:01.818370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b56a9735-91dc-4b83-a215-d9b4298775d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:01.819085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b56a9735-91dc-4b83-a215-d9b4298775d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:01.821176 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2166/4258] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:01 2022] DELETE /identity/v3/projects/dbbe49ca8a0b43c2b7bb4b886c4393e8 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:01.821979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a9735-91dc-4b83-a215-d9b4298775d6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:01.823364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b56a9735-91dc-4b83-a215-d9b4298775d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:01.868307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1884f6-638d-47b0-b64e-c956a23b9576 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '354c4495648c4c6a9630b2f600d2c17e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ae38a425c248cca0f31e8a31405c1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:01.868955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1884f6-638d-47b0-b64e-c956a23b9576 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:01.869239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1884f6-638d-47b0-b64e-c956a23b9576 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:01.869516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1884f6-638d-47b0-b64e-c956a23b9576 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/513b0b664a714485b55182bf6b7bd5fb` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:01.870110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1884f6-638d-47b0-b64e-c956a23b9576 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=513b0b664a714485b55182bf6b7bd5fb)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:01.871758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1884f6-638d-47b0-b64e-c956a23b9576 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:01.877368 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2093/4259] 10.222.0.17 () {58 vars in 1263 bytes} [Sat Oct 1 21:03:01 2022] GET /identity/v3/OS-OAUTH1/consumers/513b0b664a714485b55182bf6b7bd5fb => generated 221 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:01.878742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206930a-d940-46a7-bcb1-6ec48fc961e0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:01.879095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206930a-d940-46a7-bcb1-6ec48fc961e0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:01.879442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206930a-d940-46a7-bcb1-6ec48fc961e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:01.899637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a9735-91dc-4b83-a215-d9b4298775d6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:01.902073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9206930a-d940-46a7-bcb1-6ec48fc961e0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:01.903326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9206930a-d940-46a7-bcb1-6ec48fc961e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:01.905878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a83b9-d762-4d47-91df-f910c0df81ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:01.906518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a83b9-d762-4d47-91df-f910c0df81ad admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:01.906882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a83b9-d762-4d47-91df-f910c0df81ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:01.907172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a83b9-d762-4d47-91df-f910c0df81ad admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:01.907787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a83b9-d762-4d47-91df-f910c0df81ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:01.909695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a83b9-d762-4d47-91df-f910c0df81ad admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:01.912767 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2094/4260] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:01.913559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206930a-d940-46a7-bcb1-6ec48fc961e0 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:01.914936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9206930a-d940-46a7-bcb1-6ec48fc961e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:01.938595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206930a-d940-46a7-bcb1-6ec48fc961e0 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:01.938595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d4730-52c2-416e-b637-512221e7cdc0 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '354c4495648c4c6a9630b2f600d2c17e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ae38a425c248cca0f31e8a31405c1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:01.938595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d4730-52c2-416e-b637-512221e7cdc0 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:01.938932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d4730-52c2-416e-b637-512221e7cdc0 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:01.938932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d4730-52c2-416e-b637-512221e7cdc0 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/513b0b664a714485b55182bf6b7bd5fb` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:01.938932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d4730-52c2-416e-b637-512221e7cdc0 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=513b0b664a714485b55182bf6b7bd5fb)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:01.939529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d4730-52c2-416e-b637-512221e7cdc0 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:01.939804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-995d4730-52c2-416e-b637-512221e7cdc0 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invalidating token cache because consumer 513b0b664a714485b55182bf6b7bd5fb has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:01.940090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-995d4730-52c2-416e-b637-512221e7cdc0 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:01.940905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-995d4730-52c2-416e-b637-512221e7cdc0 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:01.949555 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2167/4261] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:01 2022] POST /identity/v3/projects => generated 399 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:01.951104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de17217-b455-4e70-a676-db6099e993fa None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:01.951488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de17217-b455-4e70-a676-db6099e993fa None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:01.951725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de17217-b455-4e70-a676-db6099e993fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:01.957416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-995d4730-52c2-416e-b637-512221e7cdc0 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '513b0b664a714485b55182bf6b7bd5fb', 'request_id': 'req-995d4730-52c2-416e-b637-512221e7cdc0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:01.973538 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2095/4262] 10.222.0.17 () {58 vars in 1266 bytes} [Sat Oct 1 21:03:01 2022] DELETE /identity/v3/OS-OAUTH1/consumers/513b0b664a714485b55182bf6b7bd5fb => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:01.975308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d4730-52c2-416e-b637-512221e7cdc0 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:01.976127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0de17217-b455-4e70-a676-db6099e993fa None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:01.977039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-995d4730-52c2-416e-b637-512221e7cdc0 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:01.977596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0de17217-b455-4e70-a676-db6099e993fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:01.993957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d4730-52c2-416e-b637-512221e7cdc0 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:01.999841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:02.055622 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2168/4263] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:02.057030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de17217-b455-4e70-a676-db6099e993fa None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:02.058351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0de17217-b455-4e70-a676-db6099e993fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:02.074153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e8bb8-1203-4e90-ac4c-ed1b0334f127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:02.074803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e8bb8-1203-4e90-ac4c-ed1b0334f127 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:02.075084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e8bb8-1203-4e90-ac4c-ed1b0334f127 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:02.075356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e8bb8-1203-4e90-ac4c-ed1b0334f127 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:02.076026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e8bb8-1203-4e90-ac4c-ed1b0334f127 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:02.077910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e8bb8-1203-4e90-ac4c-ed1b0334f127 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:02.107209 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2096/4264] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:03:01 2022] POST /identity/v3/users => generated 300 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:02.107209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e8bb8-1203-4e90-ac4c-ed1b0334f127 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:02.119781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e8bb8-1203-4e90-ac4c-ed1b0334f127 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:02.135551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a2d67-6a87-4722-a323-aeabe4af1289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:02.136506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a2d67-6a87-4722-a323-aeabe4af1289 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:02.136847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a2d67-6a87-4722-a323-aeabe4af1289 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:02.137109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a2d67-6a87-4722-a323-aeabe4af1289 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:02.137765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a2d67-6a87-4722-a323-aeabe4af1289 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:03:02.138074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a2d67-6a87-4722-a323-aeabe4af1289 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:02.139768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a2d67-6a87-4722-a323-aeabe4af1289 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:02.142483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de17217-b455-4e70-a676-db6099e993fa None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:02.147989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c658e3c-d218-4ca8-8d49-2bded24e6745 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '354c4495648c4c6a9630b2f600d2c17e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ae38a425c248cca0f31e8a31405c1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:02.148655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c658e3c-d218-4ca8-8d49-2bded24e6745 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:02.148903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c658e3c-d218-4ca8-8d49-2bded24e6745 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:02.149178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c658e3c-d218-4ca8-8d49-2bded24e6745 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/513b0b664a714485b55182bf6b7bd5fb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:02.149767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c658e3c-d218-4ca8-8d49-2bded24e6745 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=513b0b664a714485b55182bf6b7bd5fb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:02.151132 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-5c658e3c-d218-4ca8-8d49-2bded24e6745 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_consumer": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:02.151132 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:02.151132 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:02.151670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c658e3c-d218-4ca8-8d49-2bded24e6745 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:02.157753 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2097/4265] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:02.157753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a2d67-6a87-4722-a323-aeabe4af1289 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:02.168264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a2d67-6a87-4722-a323-aeabe4af1289 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:02.173174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26491a76-8a1d-47db-9b32-a629d6cf21dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:02.173777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26491a76-8a1d-47db-9b32-a629d6cf21dd admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:02.174057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26491a76-8a1d-47db-9b32-a629d6cf21dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:02.174356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26491a76-8a1d-47db-9b32-a629d6cf21dd admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:02.175067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26491a76-8a1d-47db-9b32-a629d6cf21dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:02.176677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26491a76-8a1d-47db-9b32-a629d6cf21dd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:02.178960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-5c658e3c-d218-4ca8-8d49-2bded24e6745 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Consumer not found: keystone.exception.NotFound: Consumer not found Oct 01 21:03:02.178960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:03:02.178960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:03:02.178960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:03:02.178960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:03:02.178960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:03:02.178960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:03:02.178960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:03:02.178960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:03:02.178960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:03:02.178960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:03:02.178960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:03:02.178960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 100, in get Oct 01 21:03:02.178960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_consumer(consumer_id) Oct 01 21:03:02.178960 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 95, in _get_consumer Oct 01 21:03:02.180041 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.oauth_api.get_consumer(consumer_id)) Oct 01 21:03:02.180041 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 101, in get_consumer Oct 01 21:03:02.180041 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.get_consumer_with_secret(consumer_id)) Oct 01 21:03:02.180041 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 96, in get_consumer_with_secret Oct 01 21:03:02.180041 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application consumer_ref = self._get_consumer(session, consumer_id) Oct 01 21:03:02.180041 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 91, in _get_consumer Oct 01 21:03:02.180041 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.NotFound(_('Consumer not found')) Oct 01 21:03:02.180041 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.NotFound: Consumer not found Oct 01 21:03:02.180041 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:03:02.181025 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2169/4266] 10.222.0.17 () {58 vars in 1263 bytes} [Sat Oct 1 21:03:02 2022] GET /identity/v3/OS-OAUTH1/consumers/513b0b664a714485b55182bf6b7bd5fb => generated 74 bytes in 125 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 21:03:02.185813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c658e3c-d218-4ca8-8d49-2bded24e6745 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:02.187742 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2098/4267] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:02.191323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26491a76-8a1d-47db-9b32-a629d6cf21dd admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:02.200368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c658e3c-d218-4ca8-8d49-2bded24e6745 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:02.203227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26491a76-8a1d-47db-9b32-a629d6cf21dd admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:02.207973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5cf13c-6731-4bc6-9d2d-bfd6a6994ee6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '354c4495648c4c6a9630b2f600d2c17e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ae38a425c248cca0f31e8a31405c1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:02.208142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad8812-2e67-470a-b143-831b25143568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:02.209134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cf13c-6731-4bc6-9d2d-bfd6a6994ee6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:02.209256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad8812-2e67-470a-b143-831b25143568 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:02.209452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cf13c-6731-4bc6-9d2d-bfd6a6994ee6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:02.209578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad8812-2e67-470a-b143-831b25143568 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:02.209663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad8812-2e67-470a-b143-831b25143568 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:02.209740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cf13c-6731-4bc6-9d2d-bfd6a6994ee6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/513b0b664a714485b55182bf6b7bd5fb` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:02.210249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5cf13c-6731-4bc6-9d2d-bfd6a6994ee6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=513b0b664a714485b55182bf6b7bd5fb)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:02.210362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad8812-2e67-470a-b143-831b25143568 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:02.211916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5cf13c-6731-4bc6-9d2d-bfd6a6994ee6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:02.212174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad8812-2e67-470a-b143-831b25143568 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:02.212292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cd5cf13c-6731-4bc6-9d2d-bfd6a6994ee6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invalidating token cache because consumer 513b0b664a714485b55182bf6b7bd5fb has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:02.212515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cd5cf13c-6731-4bc6-9d2d-bfd6a6994ee6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:02.213206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cd5cf13c-6731-4bc6-9d2d-bfd6a6994ee6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:02.230440 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-cd5cf13c-6731-4bc6-9d2d-bfd6a6994ee6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Consumer not found: keystone.exception.NotFound: Consumer not found Oct 01 21:03:02.230440 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:03:02.230440 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:03:02.230440 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:03:02.230440 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:03:02.230440 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:03:02.230440 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:03:02.230440 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:03:02.230440 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:03:02.230440 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:03:02.230440 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:03:02.230440 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:03:02.230440 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 123, in delete Oct 01 21:03:02.230440 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application PROVIDERS.oauth_api.delete_consumer( Oct 01 21:03:02.230440 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:03:02.231671 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:03:02.231671 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/core.py", line 139, in delete_consumer Oct 01 21:03:02.231671 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ret = self.driver.delete_consumer(consumer_id) Oct 01 21:03:02.231671 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 133, in delete_consumer Oct 01 21:03:02.231671 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._delete_consumer(session, consumer_id) Oct 01 21:03:02.231671 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 110, in _delete_consumer Oct 01 21:03:02.231671 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application consumer_ref = self._get_consumer(session, consumer_id) Oct 01 21:03:02.231671 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 91, in _get_consumer Oct 01 21:03:02.231671 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.NotFound(_('Consumer not found')) Oct 01 21:03:02.231671 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.NotFound: Consumer not found Oct 01 21:03:02.231671 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:03:02.231671 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2099/4268] 10.222.0.17 () {58 vars in 1266 bytes} [Sat Oct 1 21:03:02 2022] DELETE /identity/v3/OS-OAUTH1/consumers/513b0b664a714485b55182bf6b7bd5fb => generated 74 bytes in 43 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 21:03:02.232543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5cf13c-6731-4bc6-9d2d-bfd6a6994ee6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:02.233800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cd5cf13c-6731-4bc6-9d2d-bfd6a6994ee6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:02.244121 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2170/4269] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:02 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:02.245158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad8812-2e67-470a-b143-831b25143568 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:02.248354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b4ad8812-2e67-470a-b143-831b25143568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:02.251634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5cf13c-6731-4bc6-9d2d-bfd6a6994ee6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:02.299542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:02.324772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d657197-f3be-41a0-b87f-942a7633198e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:02.325512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d657197-f3be-41a0-b87f-942a7633198e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:02.325795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d657197-f3be-41a0-b87f-942a7633198e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:02.326084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d657197-f3be-41a0-b87f-942a7633198e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:02.326726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d657197-f3be-41a0-b87f-942a7633198e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:02.328802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d657197-f3be-41a0-b87f-942a7633198e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:02.333942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad8812-2e67-470a-b143-831b25143568 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:02.338924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf1a44-e153-4d3a-93a2-f27a2e8107e1 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '354c4495648c4c6a9630b2f600d2c17e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ae38a425c248cca0f31e8a31405c1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:02.342543 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2100/4270] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:02.342726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf1a44-e153-4d3a-93a2-f27a2e8107e1 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:02.342726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf1a44-e153-4d3a-93a2-f27a2e8107e1 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:02.342726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf1a44-e153-4d3a-93a2-f27a2e8107e1 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:02.342981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d657197-f3be-41a0-b87f-942a7633198e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:02.343070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbf1a44-e153-4d3a-93a2-f27a2e8107e1 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:02.343070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbf1a44-e153-4d3a-93a2-f27a2e8107e1 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:02.344354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1d657197-f3be-41a0-b87f-942a7633198e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:02.358618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d657197-f3be-41a0-b87f-942a7633198e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:02.359688 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2171/4271] 10.222.0.17 () {60 vars in 1185 bytes} [Sat Oct 1 21:03:02 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:02.360462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf1a44-e153-4d3a-93a2-f27a2e8107e1 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:02.368378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2c6da-1528-49e0-a2ee-f0b2005d4588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:02.369040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2c6da-1528-49e0-a2ee-f0b2005d4588 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:02.369317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2c6da-1528-49e0-a2ee-f0b2005d4588 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:02.369589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2c6da-1528-49e0-a2ee-f0b2005d4588 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:02.370465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2c6da-1528-49e0-a2ee-f0b2005d4588 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:02.372219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2c6da-1528-49e0-a2ee-f0b2005d4588 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:02.377230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf1a44-e153-4d3a-93a2-f27a2e8107e1 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:02.382991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ee060-f8ac-460b-98c3-00c060e3d14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:02.383611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ee060-f8ac-460b-98c3-00c060e3d14f admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:02.384257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ee060-f8ac-460b-98c3-00c060e3d14f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:02.384402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ee060-f8ac-460b-98c3-00c060e3d14f admin admin] PATH_INFO: `/v3/projects/99e8daf9de4d4fa395b7eee86d33a5f5/users/aa4fbb28c9de4089870d91cf52dac8ea/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:02.404741 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2101/4272] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:03:02 2022] POST /identity/v3/users => generated 288 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:02.405576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2c6da-1528-49e0-a2ee-f0b2005d4588 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:02.417891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ee060-f8ac-460b-98c3-00c060e3d14f admin admin] RBAC: Authorizing `identity:create_grant(project_id=99e8daf9de4d4fa395b7eee86d33a5f5, user_id=aa4fbb28c9de4089870d91cf52dac8ea, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:02.419309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2c6da-1528-49e0-a2ee-f0b2005d4588 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:02.420628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ee060-f8ac-460b-98c3-00c060e3d14f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:02.425254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7373969-d5b4-443d-be2f-01bbb7cea1c3 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '354c4495648c4c6a9630b2f600d2c17e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ae38a425c248cca0f31e8a31405c1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:02.425757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7373969-d5b4-443d-be2f-01bbb7cea1c3 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:02.426115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7373969-d5b4-443d-be2f-01bbb7cea1c3 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:02.426395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7373969-d5b4-443d-be2f-01bbb7cea1c3 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:02.427307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7373969-d5b4-443d-be2f-01bbb7cea1c3 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:02.430440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7373969-d5b4-443d-be2f-01bbb7cea1c3 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:02.439751 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2172/4273] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:02 2022] PUT /identity/v3/projects/99e8daf9de4d4fa395b7eee86d33a5f5/users/aa4fbb28c9de4089870d91cf52dac8ea/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:02.440923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ee060-f8ac-460b-98c3-00c060e3d14f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:02.449936 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2102/4274] 10.222.0.17 () {60 vars in 1185 bytes} [Sat Oct 1 21:03:02 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:02.451370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0244f9d-141b-41b6-acde-759246ef75a4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:02.451656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0244f9d-141b-41b6-acde-759246ef75a4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:02.451980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0244f9d-141b-41b6-acde-759246ef75a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:02.463144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:02.480674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f0244f9d-141b-41b6-acde-759246ef75a4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:02.481582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f0244f9d-141b-41b6-acde-759246ef75a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:02.525461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ee060-f8ac-460b-98c3-00c060e3d14f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:02.531819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310959bd-4f6a-4abb-aff9-ad8b142aeb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:02.532770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310959bd-4f6a-4abb-aff9-ad8b142aeb95 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:02.533788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310959bd-4f6a-4abb-aff9-ad8b142aeb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:02.534265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310959bd-4f6a-4abb-aff9-ad8b142aeb95 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:02.535043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310959bd-4f6a-4abb-aff9-ad8b142aeb95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:02.537969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310959bd-4f6a-4abb-aff9-ad8b142aeb95 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:02.548736 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2103/4275] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:02.549579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0244f9d-141b-41b6-acde-759246ef75a4 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:02.553367 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2173/4276] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:02.556792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310959bd-4f6a-4abb-aff9-ad8b142aeb95 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:02.568563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310959bd-4f6a-4abb-aff9-ad8b142aeb95 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:02.574247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ce388-0813-4d21-95c9-d9a84672f02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:02.574883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ce388-0813-4d21-95c9-d9a84672f02a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:02.575162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ce388-0813-4d21-95c9-d9a84672f02a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:02.575430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ce388-0813-4d21-95c9-d9a84672f02a admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:02.576066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ce388-0813-4d21-95c9-d9a84672f02a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:02.577665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ce388-0813-4d21-95c9-d9a84672f02a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:02.592543 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2174/4277] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:02.592543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ce388-0813-4d21-95c9-d9a84672f02a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:02.594321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ef6ce388-0813-4d21-95c9-d9a84672f02a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:02.641750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0244f9d-141b-41b6-acde-759246ef75a4 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:02.647550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b930a-abd5-4780-b1b4-539ae9026ba6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '354c4495648c4c6a9630b2f600d2c17e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ae38a425c248cca0f31e8a31405c1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:02.648175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b930a-abd5-4780-b1b4-539ae9026ba6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:02.648461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b930a-abd5-4780-b1b4-539ae9026ba6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:02.648756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b930a-abd5-4780-b1b4-539ae9026ba6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:02.649464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b930a-abd5-4780-b1b4-539ae9026ba6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorizing `identity:list_consumers()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:02.650511 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-9c9b930a-abd5-4780-b1b4-539ae9026ba6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_consumers": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:02.650511 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:02.650511 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:02.651417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b930a-abd5-4780-b1b4-539ae9026ba6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:02.658672 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2104/4278] 10.222.0.17 () {58 vars in 1164 bytes} [Sat Oct 1 21:03:02 2022] GET /identity/v3/OS-OAUTH1/consumers => generated 535 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:02.659536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b930a-abd5-4780-b1b4-539ae9026ba6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:02.672383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ce388-0813-4d21-95c9-d9a84672f02a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:02.672579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b930a-abd5-4780-b1b4-539ae9026ba6 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:02.676914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6023650-5de7-4fac-9f15-ec30b61dd7a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:02.677380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1cc60-a5e4-4bb6-99d0-5d03afd30741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:02.677576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6023650-5de7-4fac-9f15-ec30b61dd7a3 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:02.677900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6023650-5de7-4fac-9f15-ec30b61dd7a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:02.678083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1cc60-a5e4-4bb6-99d0-5d03afd30741 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:02.678187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6023650-5de7-4fac-9f15-ec30b61dd7a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:02.678359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1cc60-a5e4-4bb6-99d0-5d03afd30741 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:02.678676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1cc60-a5e4-4bb6-99d0-5d03afd30741 admin admin] PATH_INFO: `/v3/projects/02dfb544d3a54a6db8114885059c4dec/users/6adf8eb2f77344119b68fe8a71f2f49e/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:02.688650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6023650-5de7-4fac-9f15-ec30b61dd7a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:02.690253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6023650-5de7-4fac-9f15-ec30b61dd7a3 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:02.694698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1cc60-a5e4-4bb6-99d0-5d03afd30741 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02dfb544d3a54a6db8114885059c4dec, user_id=6adf8eb2f77344119b68fe8a71f2f49e, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:02.697433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1cc60-a5e4-4bb6-99d0-5d03afd30741 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:02.705001 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2175/4279] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:03:02.706767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6023650-5de7-4fac-9f15-ec30b61dd7a3 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:02.720942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6023650-5de7-4fac-9f15-ec30b61dd7a3 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:02.727000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a89f3a-1e2c-4475-8158-4e1977897806 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '354c4495648c4c6a9630b2f600d2c17e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ae38a425c248cca0f31e8a31405c1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:02.727887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a89f3a-1e2c-4475-8158-4e1977897806 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:02.728368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a89f3a-1e2c-4475-8158-4e1977897806 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:02.728871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a89f3a-1e2c-4475-8158-4e1977897806 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/42acbdfe16a5424fa32530f740ad05eb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:02.729656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a89f3a-1e2c-4475-8158-4e1977897806 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=42acbdfe16a5424fa32530f740ad05eb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:02.730904 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-e2a89f3a-1e2c-4475-8158-4e1977897806 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:02.730904 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:02.730904 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:02.731700 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2105/4280] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:02 2022] PUT /identity/v3/projects/02dfb544d3a54a6db8114885059c4dec/users/6adf8eb2f77344119b68fe8a71f2f49e/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:02.732562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a89f3a-1e2c-4475-8158-4e1977897806 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:02.733078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e2a89f3a-1e2c-4475-8158-4e1977897806 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invalidating token cache because consumer 42acbdfe16a5424fa32530f740ad05eb has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:02.733548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e2a89f3a-1e2c-4475-8158-4e1977897806 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:02.735058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e2a89f3a-1e2c-4475-8158-4e1977897806 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:02.743614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcae5e-b0fa-445b-92d4-8a23cf78b3bc None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:02.744019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcae5e-b0fa-445b-92d4-8a23cf78b3bc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:02.744370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcae5e-b0fa-445b-92d4-8a23cf78b3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:02.769142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e2a89f3a-1e2c-4475-8158-4e1977897806 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '42acbdfe16a5424fa32530f740ad05eb', 'request_id': 'req-e2a89f3a-1e2c-4475-8158-4e1977897806'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:02.796945 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2176/4281] 10.222.0.17 () {58 vars in 1266 bytes} [Sat Oct 1 21:03:02 2022] DELETE /identity/v3/OS-OAUTH1/consumers/42acbdfe16a5424fa32530f740ad05eb => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:02.798028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a89f3a-1e2c-4475-8158-4e1977897806 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:02.800115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e2a89f3a-1e2c-4475-8158-4e1977897806 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:02.802144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e2dcae5e-b0fa-445b-92d4-8a23cf78b3bc None None] MFA Rules not processed for user `6adf8eb2f77344119b68fe8a71f2f49e`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:02.803726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e2dcae5e-b0fa-445b-92d4-8a23cf78b3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:02.815013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:02.825301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a89f3a-1e2c-4475-8158-4e1977897806 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:02.892254 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2106/4282] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:03:02.923960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f2891-d182-4323-9086-8d0ed9c34007 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '354c4495648c4c6a9630b2f600d2c17e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ae38a425c248cca0f31e8a31405c1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:02.925267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f2891-d182-4323-9086-8d0ed9c34007 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:02.926388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f2891-d182-4323-9086-8d0ed9c34007 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:02.926816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f2891-d182-4323-9086-8d0ed9c34007 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/fa48bc46ee1a42aba4688fc25b952b10` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:02.928840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f2891-d182-4323-9086-8d0ed9c34007 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=fa48bc46ee1a42aba4688fc25b952b10)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:02.932160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f2891-d182-4323-9086-8d0ed9c34007 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:02.932497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-844f2891-d182-4323-9086-8d0ed9c34007 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invalidating token cache because consumer fa48bc46ee1a42aba4688fc25b952b10 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:02.932784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-844f2891-d182-4323-9086-8d0ed9c34007 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:02.933669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-844f2891-d182-4323-9086-8d0ed9c34007 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:02.940675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcae5e-b0fa-445b-92d4-8a23cf78b3bc None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:02.942124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e2dcae5e-b0fa-445b-92d4-8a23cf78b3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:02.956377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-844f2891-d182-4323-9086-8d0ed9c34007 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': 'fa48bc46ee1a42aba4688fc25b952b10', 'request_id': 'req-844f2891-d182-4323-9086-8d0ed9c34007'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:02.962590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:02.982067 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2177/4283] 10.222.0.17 () {58 vars in 1266 bytes} [Sat Oct 1 21:03:02 2022] DELETE /identity/v3/OS-OAUTH1/consumers/fa48bc46ee1a42aba4688fc25b952b10 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:02.988021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f2891-d182-4323-9086-8d0ed9c34007 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:02.990168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-844f2891-d182-4323-9086-8d0ed9c34007 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:03.003965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f2891-d182-4323-9086-8d0ed9c34007 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:03.017791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009c8eb-addb-4e2d-b735-c3ac12296bbc tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '354c4495648c4c6a9630b2f600d2c17e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ae38a425c248cca0f31e8a31405c1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:03.019606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009c8eb-addb-4e2d-b735-c3ac12296bbc tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:03.019857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009c8eb-addb-4e2d-b735-c3ac12296bbc tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:03.020103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009c8eb-addb-4e2d-b735-c3ac12296bbc tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:03.022646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009c8eb-addb-4e2d-b735-c3ac12296bbc tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:03.024578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009c8eb-addb-4e2d-b735-c3ac12296bbc tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:03.058537 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2178/4284] 10.222.0.17 () {60 vars in 1185 bytes} [Sat Oct 1 21:03:02 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:03.059792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009c8eb-addb-4e2d-b735-c3ac12296bbc tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:03.064295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcae5e-b0fa-445b-92d4-8a23cf78b3bc None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:03.071298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d81bc-5c5b-4e17-a32f-dc9eadab395a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:03.072274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d81bc-5c5b-4e17-a32f-dc9eadab395a service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:03.072705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d81bc-5c5b-4e17-a32f-dc9eadab395a service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:03.073113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d81bc-5c5b-4e17-a32f-dc9eadab395a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:03.074215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009c8eb-addb-4e2d-b735-c3ac12296bbc tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:03.075101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-035d81bc-5c5b-4e17-a32f-dc9eadab395a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:03.079864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a100f-91e6-4234-8350-89807100b0e5 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '354c4495648c4c6a9630b2f600d2c17e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ae38a425c248cca0f31e8a31405c1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:03.080501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a100f-91e6-4234-8350-89807100b0e5 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:03.080839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a100f-91e6-4234-8350-89807100b0e5 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:03.081170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a100f-91e6-4234-8350-89807100b0e5 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/c96bfe45bf0945cab5b7888a28afeeee` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:03.081873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a100f-91e6-4234-8350-89807100b0e5 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorizing `identity:update_consumer(consumer_id=c96bfe45bf0945cab5b7888a28afeeee)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:03.082927 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-c90a100f-91e6-4234-8350-89807100b0e5 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:03.082927 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:03.082927 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:03.083909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a100f-91e6-4234-8350-89807100b0e5 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:03.089601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d81bc-5c5b-4e17-a32f-dc9eadab395a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:03.091229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d81bc-5c5b-4e17-a32f-dc9eadab395a service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:03.108340 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2107/4285] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:03:03.116770 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2179/4286] 10.222.0.17 () {60 vars in 1285 bytes} [Sat Oct 1 21:03:03 2022] PATCH /identity/v3/OS-OAUTH1/consumers/c96bfe45bf0945cab5b7888a28afeeee => generated 220 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:03.118206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d81bc-5c5b-4e17-a32f-dc9eadab395a service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:03.144198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d81bc-5c5b-4e17-a32f-dc9eadab395a service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:03.149804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07917090-6b96-44ae-bbe7-aeaf15cb523c tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '354c4495648c4c6a9630b2f600d2c17e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ae38a425c248cca0f31e8a31405c1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:03.150718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07917090-6b96-44ae-bbe7-aeaf15cb523c tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:03.151009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07917090-6b96-44ae-bbe7-aeaf15cb523c tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:03.151293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07917090-6b96-44ae-bbe7-aeaf15cb523c tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/c96bfe45bf0945cab5b7888a28afeeee` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:03.151892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07917090-6b96-44ae-bbe7-aeaf15cb523c tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=c96bfe45bf0945cab5b7888a28afeeee)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:03.153949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07917090-6b96-44ae-bbe7-aeaf15cb523c tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:03.159904 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2108/4287] 10.222.0.17 () {58 vars in 1263 bytes} [Sat Oct 1 21:03:03 2022] GET /identity/v3/OS-OAUTH1/consumers/c96bfe45bf0945cab5b7888a28afeeee => generated 220 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:03:03.164796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a100f-91e6-4234-8350-89807100b0e5 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:03.179164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a100f-91e6-4234-8350-89807100b0e5 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:03.185216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092112e6-dc2e-42a2-9220-0edcee257755 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '354c4495648c4c6a9630b2f600d2c17e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ae38a425c248cca0f31e8a31405c1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:03.185866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092112e6-dc2e-42a2-9220-0edcee257755 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:03.186271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092112e6-dc2e-42a2-9220-0edcee257755 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:03.186616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092112e6-dc2e-42a2-9220-0edcee257755 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/c96bfe45bf0945cab5b7888a28afeeee` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:03.188274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092112e6-dc2e-42a2-9220-0edcee257755 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=c96bfe45bf0945cab5b7888a28afeeee)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:03.189085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092112e6-dc2e-42a2-9220-0edcee257755 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:03.189417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-092112e6-dc2e-42a2-9220-0edcee257755 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invalidating token cache because consumer c96bfe45bf0945cab5b7888a28afeeee has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:03.189706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-092112e6-dc2e-42a2-9220-0edcee257755 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:03.192060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-092112e6-dc2e-42a2-9220-0edcee257755 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:03.210924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-092112e6-dc2e-42a2-9220-0edcee257755 tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': 'c96bfe45bf0945cab5b7888a28afeeee', 'request_id': 'req-092112e6-dc2e-42a2-9220-0edcee257755'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:03.236059 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2180/4288] 10.222.0.17 () {58 vars in 1266 bytes} [Sat Oct 1 21:03:03 2022] DELETE /identity/v3/OS-OAUTH1/consumers/c96bfe45bf0945cab5b7888a28afeeee => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:03.241466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07917090-6b96-44ae-bbe7-aeaf15cb523c tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:03.243490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-07917090-6b96-44ae-bbe7-aeaf15cb523c tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:03.337396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07917090-6b96-44ae-bbe7-aeaf15cb523c tempest-OAUTHConsumersV3Test-1180698554 tempest-OAUTHConsumersV3Test-1180698554-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:03.344984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640c5e5-f732-4c36-a007-37a7ddfb7653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:03.344984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640c5e5-f732-4c36-a007-37a7ddfb7653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:03.344984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640c5e5-f732-4c36-a007-37a7ddfb7653 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:03.344984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640c5e5-f732-4c36-a007-37a7ddfb7653 admin admin] PATH_INFO: `/v3/users/fb0be548096544d1a1ba68b41e8eb977` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:03.348911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640c5e5-f732-4c36-a007-37a7ddfb7653 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb0be548096544d1a1ba68b41e8eb977)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:03.348911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640c5e5-f732-4c36-a007-37a7ddfb7653 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:03.406984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:03.410025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6640c5e5-f732-4c36-a007-37a7ddfb7653 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb0be548096544d1a1ba68b41e8eb977'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:03.411510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6640c5e5-f732-4c36-a007-37a7ddfb7653 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb0be548096544d1a1ba68b41e8eb977'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:03.412060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6640c5e5-f732-4c36-a007-37a7ddfb7653 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb0be548096544d1a1ba68b41e8eb977'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:03.426853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6640c5e5-f732-4c36-a007-37a7ddfb7653 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb0be548096544d1a1ba68b41e8eb977'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:03.443122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6640c5e5-f732-4c36-a007-37a7ddfb7653 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb0be548096544d1a1ba68b41e8eb977'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:03.461171 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2109/4289] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:03 2022] DELETE /identity/v3/users/fb0be548096544d1a1ba68b41e8eb977 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:03.462950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0746a88-d18a-4f30-8e93-b2ba61733401 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:03.463244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0746a88-d18a-4f30-8e93-b2ba61733401 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:03.463539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0746a88-d18a-4f30-8e93-b2ba61733401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:03.490416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c0746a88-d18a-4f30-8e93-b2ba61733401 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:03.491841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c0746a88-d18a-4f30-8e93-b2ba61733401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:03.572576 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2181/4290] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:03:03.584501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640c5e5-f732-4c36-a007-37a7ddfb7653 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:03.587937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6640c5e5-f732-4c36-a007-37a7ddfb7653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:03.693771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640c5e5-f732-4c36-a007-37a7ddfb7653 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:03.710943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c22a57-e733-4767-9063-c9c9aa7ee617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:03.711585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c22a57-e733-4767-9063-c9c9aa7ee617 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:03.711871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c22a57-e733-4767-9063-c9c9aa7ee617 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:03.712150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c22a57-e733-4767-9063-c9c9aa7ee617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:03.741962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c22a57-e733-4767-9063-c9c9aa7ee617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:03.743470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c22a57-e733-4767-9063-c9c9aa7ee617 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:03.768284 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2110/4291] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:03:04.832551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0746a88-d18a-4f30-8e93-b2ba61733401 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:04.834939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c0746a88-d18a-4f30-8e93-b2ba61733401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:04.862136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0746a88-d18a-4f30-8e93-b2ba61733401 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:04.871879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b45c631-63e1-4a09-bef1-58253d4bdef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:04.872779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45c631-63e1-4a09-bef1-58253d4bdef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:04.873257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45c631-63e1-4a09-bef1-58253d4bdef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:04.873707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45c631-63e1-4a09-bef1-58253d4bdef6 admin admin] PATH_INFO: `/v3/projects/083f87d7fcb54aed94ba016ec32d6423` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:04.879424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b45c631-63e1-4a09-bef1-58253d4bdef6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=083f87d7fcb54aed94ba016ec32d6423)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:04.882112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b45c631-63e1-4a09-bef1-58253d4bdef6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:05.079801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2b45c631-63e1-4a09-bef1-58253d4bdef6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '083f87d7fcb54aed94ba016ec32d6423', 'request_id': 'req-2b45c631-63e1-4a09-bef1-58253d4bdef6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:05.090535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2b45c631-63e1-4a09-bef1-58253d4bdef6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '083f87d7fcb54aed94ba016ec32d6423', 'request_id': 'req-2b45c631-63e1-4a09-bef1-58253d4bdef6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:05.099555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2b45c631-63e1-4a09-bef1-58253d4bdef6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '083f87d7fcb54aed94ba016ec32d6423', 'request_id': 'req-2b45c631-63e1-4a09-bef1-58253d4bdef6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:05.119766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2b45c631-63e1-4a09-bef1-58253d4bdef6 admin admin] The token cache is being invalidate because project 083f87d7fcb54aed94ba016ec32d6423 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:05.120240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2b45c631-63e1-4a09-bef1-58253d4bdef6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:05.120941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2b45c631-63e1-4a09-bef1-58253d4bdef6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:05.122554 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2182/4292] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:04 2022] DELETE /identity/v3/projects/083f87d7fcb54aed94ba016ec32d6423 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:05.127363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c22a57-e733-4767-9063-c9c9aa7ee617 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:05.167206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c22a57-e733-4767-9063-c9c9aa7ee617 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:05.274467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6edb3-9196-4518-b64f-d4b07c41e33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:05.274902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6edb3-9196-4518-b64f-d4b07c41e33d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:05.275289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6edb3-9196-4518-b64f-d4b07c41e33d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:05.275672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6edb3-9196-4518-b64f-d4b07c41e33d admin admin] PATH_INFO: `/v3/users/354c4495648c4c6a9630b2f600d2c17e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:05.276965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6edb3-9196-4518-b64f-d4b07c41e33d admin admin] RBAC: Authorizing `identity:delete_user(user_id=354c4495648c4c6a9630b2f600d2c17e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:05.279212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6edb3-9196-4518-b64f-d4b07c41e33d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:05.363156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c6d6edb3-9196-4518-b64f-d4b07c41e33d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '354c4495648c4c6a9630b2f600d2c17e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:05.364176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c6d6edb3-9196-4518-b64f-d4b07c41e33d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '354c4495648c4c6a9630b2f600d2c17e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:05.366793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c6d6edb3-9196-4518-b64f-d4b07c41e33d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '354c4495648c4c6a9630b2f600d2c17e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:05.383498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c6d6edb3-9196-4518-b64f-d4b07c41e33d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '354c4495648c4c6a9630b2f600d2c17e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:05.396267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:05.398830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c6d6edb3-9196-4518-b64f-d4b07c41e33d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '354c4495648c4c6a9630b2f600d2c17e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:05.419506 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2111/4293] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:05 2022] DELETE /identity/v3/users/354c4495648c4c6a9630b2f600d2c17e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:07.028650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd138c-f344-464f-aaa4-b5a3caf5019d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:07.028650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd138c-f344-464f-aaa4-b5a3caf5019d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:07.028650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd138c-f344-464f-aaa4-b5a3caf5019d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:07.029951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6edb3-9196-4518-b64f-d4b07c41e33d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:07.031523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c6d6edb3-9196-4518-b64f-d4b07c41e33d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:07.063553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8cdd138c-f344-464f-aaa4-b5a3caf5019d None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:07.068855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8cdd138c-f344-464f-aaa4-b5a3caf5019d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:07.113652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:UPDATE=1,keystone:DELETE=9 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:07.179990 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2183/4294] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:03:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:03:07.214935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd138c-f344-464f-aaa4-b5a3caf5019d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:07.217027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8cdd138c-f344-464f-aaa4-b5a3caf5019d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:07.247109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6edb3-9196-4518-b64f-d4b07c41e33d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:07.261950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a80460-5505-4f7b-9765-465fde626117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:07.263056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a80460-5505-4f7b-9765-465fde626117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:07.263297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a80460-5505-4f7b-9765-465fde626117 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:07.263654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a80460-5505-4f7b-9765-465fde626117 admin admin] PATH_INFO: `/v3/projects/26ae38a425c248cca0f31e8a31405c1e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:07.265175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a80460-5505-4f7b-9765-465fde626117 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26ae38a425c248cca0f31e8a31405c1e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:07.270400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a80460-5505-4f7b-9765-465fde626117 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:07.399059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-91a80460-5505-4f7b-9765-465fde626117 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26ae38a425c248cca0f31e8a31405c1e', 'request_id': 'req-91a80460-5505-4f7b-9765-465fde626117'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:07.412869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-91a80460-5505-4f7b-9765-465fde626117 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26ae38a425c248cca0f31e8a31405c1e', 'request_id': 'req-91a80460-5505-4f7b-9765-465fde626117'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:07.423437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-91a80460-5505-4f7b-9765-465fde626117 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26ae38a425c248cca0f31e8a31405c1e', 'request_id': 'req-91a80460-5505-4f7b-9765-465fde626117'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:07.434179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd138c-f344-464f-aaa4-b5a3caf5019d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:07.441063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-91a80460-5505-4f7b-9765-465fde626117 admin admin] The token cache is being invalidate because project 26ae38a425c248cca0f31e8a31405c1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:07.441361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-91a80460-5505-4f7b-9765-465fde626117 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:07.442280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-91a80460-5505-4f7b-9765-465fde626117 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:07.445449 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2112/4295] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:07 2022] DELETE /identity/v3/projects/26ae38a425c248cca0f31e8a31405c1e => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:07.452305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48f77e-2816-4c38-a15d-20d274a31824 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:07.452730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48f77e-2816-4c38-a15d-20d274a31824 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:07.453443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48f77e-2816-4c38-a15d-20d274a31824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:07.489796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3f48f77e-2816-4c38-a15d-20d274a31824 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:07.491144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3f48f77e-2816-4c38-a15d-20d274a31824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:07.609887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c87c37-1b35-4275-bd7e-8a535e09171f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:07.611008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c87c37-1b35-4275-bd7e-8a535e09171f service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:07.611500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c87c37-1b35-4275-bd7e-8a535e09171f service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:07.611928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c87c37-1b35-4275-bd7e-8a535e09171f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:07.615623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-29c87c37-1b35-4275-bd7e-8a535e09171f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:07.642035 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2113/4296] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:07.654583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3f152-cc22-4653-9441-142cdc5d3d67 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:07.654876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3f152-cc22-4653-9441-142cdc5d3d67 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:07.655153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3f152-cc22-4653-9441-142cdc5d3d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:07.697314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-77a3f152-cc22-4653-9441-142cdc5d3d67 None None] MFA Rules not processed for user `775ff9091a794faaacc05a37550c8a72`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:07.698582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-77a3f152-cc22-4653-9441-142cdc5d3d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:07.722204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:07.779125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c87c37-1b35-4275-bd7e-8a535e09171f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:07.780888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c87c37-1b35-4275-bd7e-8a535e09171f service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:07.804644 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2184/4297] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:03:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:03:07.808962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3dab0-421f-488b-be63-30ab0eb8ae91 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:07.809183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3dab0-421f-488b-be63-30ab0eb8ae91 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:07.810424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3dab0-421f-488b-be63-30ab0eb8ae91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:07.872976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-34b3dab0-421f-488b-be63-30ab0eb8ae91 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:07.878245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-34b3dab0-421f-488b-be63-30ab0eb8ae91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:07.878593 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2114/4298] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:07.904179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3f152-cc22-4653-9441-142cdc5d3d67 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:07.910292 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2185/4299] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:07.919742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3f152-cc22-4653-9441-142cdc5d3d67 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:07.934385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f4273-4726-4352-bd9d-b60576b7a6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:07.935161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f4273-4726-4352-bd9d-b60576b7a6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:07.935448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f4273-4726-4352-bd9d-b60576b7a6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:07.935720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f4273-4726-4352-bd9d-b60576b7a6f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:07.938397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f4273-4726-4352-bd9d-b60576b7a6f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:03:07.941162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f4273-4726-4352-bd9d-b60576b7a6f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:07.944566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f4273-4726-4352-bd9d-b60576b7a6f8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:07.984749 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2115/4300] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:07.990584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3dab0-421f-488b-be63-30ab0eb8ae91 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:08.003472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3dab0-421f-488b-be63-30ab0eb8ae91 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:08.020318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adfce2-52e9-49f0-bf4d-2739579fb660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:08.026654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adfce2-52e9-49f0-bf4d-2739579fb660 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:08.026654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adfce2-52e9-49f0-bf4d-2739579fb660 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:08.026654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adfce2-52e9-49f0-bf4d-2739579fb660 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:08.026654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42adfce2-52e9-49f0-bf4d-2739579fb660 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:08.028927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42adfce2-52e9-49f0-bf4d-2739579fb660 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:08.042024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:08.100151 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2186/4301] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:07 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:08.125473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f4273-4726-4352-bd9d-b60576b7a6f8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:08.153271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f4273-4726-4352-bd9d-b60576b7a6f8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:08.346989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b5685-2f7a-4a38-8605-5e00f0926478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:08.347620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b5685-2f7a-4a38-8605-5e00f0926478 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:08.347907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b5685-2f7a-4a38-8605-5e00f0926478 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:08.348183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b5685-2f7a-4a38-8605-5e00f0926478 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:08.349395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b5685-2f7a-4a38-8605-5e00f0926478 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:08.355447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b5685-2f7a-4a38-8605-5e00f0926478 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:08.408615 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2116/4302] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:03:08 2022] POST /identity/v3/users => generated 283 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:08.410294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adfce2-52e9-49f0-bf4d-2739579fb660 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:08.425613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adfce2-52e9-49f0-bf4d-2739579fb660 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:08.448820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43eecef-8151-4d27-b28c-e59b4dde459a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:08.449078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43eecef-8151-4d27-b28c-e59b4dde459a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:08.449283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43eecef-8151-4d27-b28c-e59b4dde459a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:08.449681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43eecef-8151-4d27-b28c-e59b4dde459a admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:08.450698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43eecef-8151-4d27-b28c-e59b4dde459a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:08.452064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43eecef-8151-4d27-b28c-e59b4dde459a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:08.494072 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2187/4303] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:08.496424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b5685-2f7a-4a38-8605-5e00f0926478 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:08.509654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b5685-2f7a-4a38-8605-5e00f0926478 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:08.522980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e2932-3dcd-44a3-975d-9272b08c8acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:08.523615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e2932-3dcd-44a3-975d-9272b08c8acf admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:08.523901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e2932-3dcd-44a3-975d-9272b08c8acf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:08.524434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e2932-3dcd-44a3-975d-9272b08c8acf admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:08.525091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e2932-3dcd-44a3-975d-9272b08c8acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:08.526784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e2932-3dcd-44a3-975d-9272b08c8acf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:08.555570 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2117/4304] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:08.557050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43eecef-8151-4d27-b28c-e59b4dde459a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:08.576145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43eecef-8151-4d27-b28c-e59b4dde459a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:08.601415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48487678-c427-4578-b34d-23f4c836e60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:08.602018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48487678-c427-4578-b34d-23f4c836e60d admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:08.603661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48487678-c427-4578-b34d-23f4c836e60d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:08.603959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48487678-c427-4578-b34d-23f4c836e60d admin admin] PATH_INFO: `/v3/projects/3c17730b45e3489f89165499dc67eb21/users/1394694d13444278a0e18fd66376fa0c/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:08.651282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48487678-c427-4578-b34d-23f4c836e60d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c17730b45e3489f89165499dc67eb21, user_id=1394694d13444278a0e18fd66376fa0c, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:08.665053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48487678-c427-4578-b34d-23f4c836e60d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:08.722298 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2188/4305] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:08 2022] PUT /identity/v3/projects/3c17730b45e3489f89165499dc67eb21/users/1394694d13444278a0e18fd66376fa0c/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:08.726117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d35558-9b28-46a3-aafe-bc56371007c2 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:08.726117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d35558-9b28-46a3-aafe-bc56371007c2 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:08.726117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d35558-9b28-46a3-aafe-bc56371007c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:08.844292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d5d35558-9b28-46a3-aafe-bc56371007c2 None None] MFA Rules not processed for user `1394694d13444278a0e18fd66376fa0c`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:08.845548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d5d35558-9b28-46a3-aafe-bc56371007c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:08.983522 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2118/4306] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:08.984746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48487678-c427-4578-b34d-23f4c836e60d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:09.140975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48487678-c427-4578-b34d-23f4c836e60d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:09.158332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a587554-3bdd-48fa-8418-5b1ac67c3afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:09.159007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a587554-3bdd-48fa-8418-5b1ac67c3afe admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:09.159295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a587554-3bdd-48fa-8418-5b1ac67c3afe admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:09.159572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a587554-3bdd-48fa-8418-5b1ac67c3afe admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:09.160216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a587554-3bdd-48fa-8418-5b1ac67c3afe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:09.162754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a587554-3bdd-48fa-8418-5b1ac67c3afe admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:09.222203 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2189/4307] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:08 2022] POST /identity/v3/projects => generated 367 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:09.226765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d35558-9b28-46a3-aafe-bc56371007c2 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:09.248037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d35558-9b28-46a3-aafe-bc56371007c2 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:09.263004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:09.396162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff55ec-478b-466c-9636-4ce05fd73f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:09.398254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff55ec-478b-466c-9636-4ce05fd73f5a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:09.398809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff55ec-478b-466c-9636-4ce05fd73f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:09.399315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff55ec-478b-466c-9636-4ce05fd73f5a admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:09.400152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff55ec-478b-466c-9636-4ce05fd73f5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:09.402300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff55ec-478b-466c-9636-4ce05fd73f5a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:09.452071 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2119/4308] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:03:09 2022] POST /identity/v3/users => generated 290 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:09.456520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a587554-3bdd-48fa-8418-5b1ac67c3afe admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:09.476351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a587554-3bdd-48fa-8418-5b1ac67c3afe admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:09.498934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f42fc-ed71-490d-aa13-10c3d0895957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:09.500169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f42fc-ed71-490d-aa13-10c3d0895957 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:09.500537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f42fc-ed71-490d-aa13-10c3d0895957 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:09.500883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f42fc-ed71-490d-aa13-10c3d0895957 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:09.502121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f42fc-ed71-490d-aa13-10c3d0895957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:09.508078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f42fc-ed71-490d-aa13-10c3d0895957 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:09.546118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff55ec-478b-466c-9636-4ce05fd73f5a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:09.548182 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2190/4309] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:09.566299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff55ec-478b-466c-9636-4ce05fd73f5a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:09.578766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f3313-a627-4ed1-84ec-bc3c2808169c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:09.580089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f3313-a627-4ed1-84ec-bc3c2808169c admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:09.580586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f3313-a627-4ed1-84ec-bc3c2808169c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:09.581073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f3313-a627-4ed1-84ec-bc3c2808169c admin admin] PATH_INFO: `/v3/projects/a1653d0e18814d7e8e6e0489a5b27fc4/users/cd8a17c6e9d3437295316e2018e7f6e2/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:09.621829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f3313-a627-4ed1-84ec-bc3c2808169c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1653d0e18814d7e8e6e0489a5b27fc4, user_id=cd8a17c6e9d3437295316e2018e7f6e2, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:09.630040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f3313-a627-4ed1-84ec-bc3c2808169c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:09.662128 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2120/4310] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:09 2022] PUT /identity/v3/projects/a1653d0e18814d7e8e6e0489a5b27fc4/users/cd8a17c6e9d3437295316e2018e7f6e2/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:09.666384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f42fc-ed71-490d-aa13-10c3d0895957 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:09.686958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f42fc-ed71-490d-aa13-10c3d0895957 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:09.867589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d2694-4211-46b6-9896-b88cfbfb756d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:09.869793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d2694-4211-46b6-9896-b88cfbfb756d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:09.870096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d2694-4211-46b6-9896-b88cfbfb756d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:09.872590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d2694-4211-46b6-9896-b88cfbfb756d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:09.873276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d2694-4211-46b6-9896-b88cfbfb756d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:09.876093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d2694-4211-46b6-9896-b88cfbfb756d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:09.908387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f3313-a627-4ed1-84ec-bc3c2808169c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:09.908678 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2191/4311] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:09.927145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f3313-a627-4ed1-84ec-bc3c2808169c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:09.945086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5976e-3e4b-439f-b74c-0f15ebadc63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:09.950181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5976e-3e4b-439f-b74c-0f15ebadc63a admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:09.951828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5976e-3e4b-439f-b74c-0f15ebadc63a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:09.952409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5976e-3e4b-439f-b74c-0f15ebadc63a admin admin] PATH_INFO: `/v3/projects/a1653d0e18814d7e8e6e0489a5b27fc4/users/cd8a17c6e9d3437295316e2018e7f6e2/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:09.962039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5976e-3e4b-439f-b74c-0f15ebadc63a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1653d0e18814d7e8e6e0489a5b27fc4, user_id=cd8a17c6e9d3437295316e2018e7f6e2, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:09.966798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5976e-3e4b-439f-b74c-0f15ebadc63a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:10.004166 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2121/4312] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:09 2022] PUT /identity/v3/projects/a1653d0e18814d7e8e6e0489a5b27fc4/users/cd8a17c6e9d3437295316e2018e7f6e2/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:10.008131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0cb01-3061-49e6-ba57-f46535a3fa54 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:10.009074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0cb01-3061-49e6-ba57-f46535a3fa54 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:10.014680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0cb01-3061-49e6-ba57-f46535a3fa54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:10.085236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c9b0cb01-3061-49e6-ba57-f46535a3fa54 None None] MFA Rules not processed for user `cd8a17c6e9d3437295316e2018e7f6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:10.087119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c9b0cb01-3061-49e6-ba57-f46535a3fa54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:10.099124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:10.296022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5976e-3e4b-439f-b74c-0f15ebadc63a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:10.311114 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2192/4313] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:10 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:10.316110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5976e-3e4b-439f-b74c-0f15ebadc63a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:10.324161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669a63a-0b30-42d1-86e0-8d891e5f30b0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:10.324899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669a63a-0b30-42d1-86e0-8d891e5f30b0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:10.325116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669a63a-0b30-42d1-86e0-8d891e5f30b0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:10.325445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669a63a-0b30-42d1-86e0-8d891e5f30b0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:10.326291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669a63a-0b30-42d1-86e0-8d891e5f30b0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:10.328501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669a63a-0b30-42d1-86e0-8d891e5f30b0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:10.418727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0cb01-3061-49e6-ba57-f46535a3fa54 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:10.429183 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2122/4314] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:03:10 2022] POST /identity/v3/roles => generated 235 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:10.435061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0cb01-3061-49e6-ba57-f46535a3fa54 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:10.446690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70ca9a-6a90-4595-9da1-770138ced175 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:10.447322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70ca9a-6a90-4595-9da1-770138ced175 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:10.447606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70ca9a-6a90-4595-9da1-770138ced175 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:10.447878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70ca9a-6a90-4595-9da1-770138ced175 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:10.449185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70ca9a-6a90-4595-9da1-770138ced175 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:10.450324 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-3d70ca9a-6a90-4595-9da1-770138ced175 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:10.450324 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:10.450324 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:10.453294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70ca9a-6a90-4595-9da1-770138ced175 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:10.510541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669a63a-0b30-42d1-86e0-8d891e5f30b0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:10.510769 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2193/4315] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:03:10 2022] POST /identity/v3/roles => generated 235 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:03:10.515097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669a63a-0b30-42d1-86e0-8d891e5f30b0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:10.522570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801f91f-9669-44c0-a6d1-1dbb190b1686 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:10.523500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801f91f-9669-44c0-a6d1-1dbb190b1686 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:10.525500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801f91f-9669-44c0-a6d1-1dbb190b1686 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:10.526452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801f91f-9669-44c0-a6d1-1dbb190b1686 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:10.527390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801f91f-9669-44c0-a6d1-1dbb190b1686 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:10.529587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801f91f-9669-44c0-a6d1-1dbb190b1686 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:10.553847 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2123/4316] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:03:10 2022] POST /identity/v3/roles => generated 234 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:10.559513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70ca9a-6a90-4595-9da1-770138ced175 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:10.576435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70ca9a-6a90-4595-9da1-770138ced175 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:10.605526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827411b-e1bc-440e-b566-30fbfae16e86 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:10.606160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827411b-e1bc-440e-b566-30fbfae16e86 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:10.606443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827411b-e1bc-440e-b566-30fbfae16e86 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:10.608054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827411b-e1bc-440e-b566-30fbfae16e86 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:10.608908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3827411b-e1bc-440e-b566-30fbfae16e86 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:10.613011 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-3827411b-e1bc-440e-b566-30fbfae16e86 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:10.613011 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:10.613011 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:10.613931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3827411b-e1bc-440e-b566-30fbfae16e86 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:10.699769 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2194/4317] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:03:10 2022] POST /identity/v3/domains => generated 275 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:10.704644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801f91f-9669-44c0-a6d1-1dbb190b1686 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:10.719788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801f91f-9669-44c0-a6d1-1dbb190b1686 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:11.167558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab4255-b91d-4aee-b24e-4b33d702e330 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:11.168202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab4255-b91d-4aee-b24e-4b33d702e330 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:11.172949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab4255-b91d-4aee-b24e-4b33d702e330 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:11.173251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab4255-b91d-4aee-b24e-4b33d702e330 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:11.174227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab4255-b91d-4aee-b24e-4b33d702e330 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:11.182879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab4255-b91d-4aee-b24e-4b33d702e330 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:11.359083 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2124/4318] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:10 2022] POST /identity/v3/projects => generated 400 bytes in 655 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:11.365668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827411b-e1bc-440e-b566-30fbfae16e86 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:11.386902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827411b-e1bc-440e-b566-30fbfae16e86 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:11.553223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b620468-5b38-484c-828f-599dd754b36c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:11.554646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b620468-5b38-484c-828f-599dd754b36c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:11.554928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b620468-5b38-484c-828f-599dd754b36c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:11.558896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b620468-5b38-484c-828f-599dd754b36c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/groups` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:11.559627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b620468-5b38-484c-828f-599dd754b36c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:11.569773 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-3b620468-5b38-484c-828f-599dd754b36c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:11.569773 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:11.569773 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:11.571143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b620468-5b38-484c-828f-599dd754b36c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:11.628892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab4255-b91d-4aee-b24e-4b33d702e330 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:11.631753 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2195/4319] 10.222.0.17 () {60 vars in 1147 bytes} [Sat Oct 1 21:03:11 2022] POST /identity/v3/groups => generated 301 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:11.646025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab4255-b91d-4aee-b24e-4b33d702e330 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:11.674213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3102d6-9e7d-4d21-8795-4e5ef8a78b17 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:11.675343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3102d6-9e7d-4d21-8795-4e5ef8a78b17 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:11.675629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3102d6-9e7d-4d21-8795-4e5ef8a78b17 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:11.675905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3102d6-9e7d-4d21-8795-4e5ef8a78b17 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:11.677181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3102d6-9e7d-4d21-8795-4e5ef8a78b17 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:11.678975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3102d6-9e7d-4d21-8795-4e5ef8a78b17 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:11.692429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:11.756498 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2125/4320] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:03:11 2022] POST /identity/v3/roles => generated 234 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:11.756735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b620468-5b38-484c-828f-599dd754b36c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:11.769401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b620468-5b38-484c-828f-599dd754b36c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:11.795972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eddf0c-823c-4c12-a32d-e40371c5a69b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:11.796769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eddf0c-823c-4c12-a32d-e40371c5a69b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:11.796915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eddf0c-823c-4c12-a32d-e40371c5a69b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:11.803404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eddf0c-823c-4c12-a32d-e40371c5a69b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:11.804164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eddf0c-823c-4c12-a32d-e40371c5a69b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:11.815289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eddf0c-823c-4c12-a32d-e40371c5a69b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:11.897526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3102d6-9e7d-4d21-8795-4e5ef8a78b17 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:11.908771 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2196/4321] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:03:11 2022] POST /identity/v3/users => generated 439 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:03:11.916785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3102d6-9e7d-4d21-8795-4e5ef8a78b17 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:11.922706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e8469-76a0-407b-9446-f3704c30bfa2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:11.923413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e8469-76a0-407b-9446-f3704c30bfa2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:11.923770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e8469-76a0-407b-9446-f3704c30bfa2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:11.923943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e8469-76a0-407b-9446-f3704c30bfa2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:11.924931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e8469-76a0-407b-9446-f3704c30bfa2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:11.926043 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-f51e8469-76a0-407b-9446-f3704c30bfa2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:11.926043 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:11.926043 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:11.927128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e8469-76a0-407b-9446-f3704c30bfa2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:11.991387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eddf0c-823c-4c12-a32d-e40371c5a69b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:12.003050 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2126/4322] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:03:11 2022] POST /identity/v3/roles => generated 270 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:03:12.011023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eddf0c-823c-4c12-a32d-e40371c5a69b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:12.027450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92ed03-7109-4f38-b596-1294d7b93b02 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:12.028500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92ed03-7109-4f38-b596-1294d7b93b02 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:12.028969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92ed03-7109-4f38-b596-1294d7b93b02 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:12.029722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92ed03-7109-4f38-b596-1294d7b93b02 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/8e8bb5b4e316492ba51a83b109e89ec1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:12.059214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92ed03-7109-4f38-b596-1294d7b93b02 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_grant(project_id=f69494f2b2fc40f29be4d38fe0a42b87, user_id=1ec430a8ba524da0af3eef8e1e8d9d33, role_id=8e8bb5b4e316492ba51a83b109e89ec1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:12.061939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92ed03-7109-4f38-b596-1294d7b93b02 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:12.096394 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2197/4323] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:11 2022] PUT /identity/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/8e8bb5b4e316492ba51a83b109e89ec1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:12.101474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e8469-76a0-407b-9446-f3704c30bfa2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:12.113139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e8469-76a0-407b-9446-f3704c30bfa2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:12.296907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5517f32-19ea-4d19-9716-ac29ff7553fd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:12.298969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5517f32-19ea-4d19-9716-ac29ff7553fd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:12.299290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5517f32-19ea-4d19-9716-ac29ff7553fd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:12.300733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5517f32-19ea-4d19-9716-ac29ff7553fd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:12.304208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5517f32-19ea-4d19-9716-ac29ff7553fd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Adding query filter params (scope.project.id=f69494f2b2fc40f29be4d38fe0a42b87, user.id=1ec430a8ba524da0af3eef8e1e8d9d33) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:03:12.304661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5517f32-19ea-4d19-9716-ac29ff7553fd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:12.306762 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-b5517f32-19ea-4d19-9716-ac29ff7553fd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_assignments": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:12.306762 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:12.306762 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:12.309281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5517f32-19ea-4d19-9716-ac29ff7553fd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:12.445090 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2127/4324] 10.222.0.17 () {58 vars in 1356 bytes} [Sat Oct 1 21:03:12 2022] GET /identity/v3/role_assignments?scope.project.id=f69494f2b2fc40f29be4d38fe0a42b87&user.id=1ec430a8ba524da0af3eef8e1e8d9d33&effective => generated 228 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:12.447410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92ed03-7109-4f38-b596-1294d7b93b02 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:12.471489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92ed03-7109-4f38-b596-1294d7b93b02 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:12.488296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1094be-99fe-43e4-8311-85331e58e60e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:12.489331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1094be-99fe-43e4-8311-85331e58e60e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:12.489790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1094be-99fe-43e4-8311-85331e58e60e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:12.490229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1094be-99fe-43e4-8311-85331e58e60e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/8e8bb5b4e316492ba51a83b109e89ec1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:12.496223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1094be-99fe-43e4-8311-85331e58e60e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=f69494f2b2fc40f29be4d38fe0a42b87, user_id=1ec430a8ba524da0af3eef8e1e8d9d33, role_id=8e8bb5b4e316492ba51a83b109e89ec1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:12.497685 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-1a1094be-99fe-43e4-8311-85331e58e60e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:12.497685 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:12.497685 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:12.500072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1094be-99fe-43e4-8311-85331e58e60e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:12.522109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1a1094be-99fe-43e4-8311-85331e58e60e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invalidating the token cache because role 8e8bb5b4e316492ba51a83b109e89ec1 was removed from user 1ec430a8ba524da0af3eef8e1e8d9d33 on project f69494f2b2fc40f29be4d38fe0a42b87. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:12.522796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1a1094be-99fe-43e4-8311-85331e58e60e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:12.524034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1a1094be-99fe-43e4-8311-85331e58e60e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:12.555539 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2198/4325] 10.222.0.17 () {58 vars in 1467 bytes} [Sat Oct 1 21:03:12 2022] DELETE /identity/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/8e8bb5b4e316492ba51a83b109e89ec1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:12.560260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5517f32-19ea-4d19-9716-ac29ff7553fd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:12.563035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b5517f32-19ea-4d19-9716-ac29ff7553fd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:12.591000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5517f32-19ea-4d19-9716-ac29ff7553fd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:12.912038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ff462-0a24-41ec-aab0-41a5e8df0db9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:12.915349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ff462-0a24-41ec-aab0-41a5e8df0db9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:12.919613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ff462-0a24-41ec-aab0-41a5e8df0db9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:12.920105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ff462-0a24-41ec-aab0-41a5e8df0db9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/8e8bb5b4e316492ba51a83b109e89ec1` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:12.926013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ff462-0a24-41ec-aab0-41a5e8df0db9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=8e8bb5b4e316492ba51a83b109e89ec1)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:12.932575 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-3a1ff462-0a24-41ec-aab0-41a5e8df0db9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:12.932575 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:12.932575 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:12.936842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ff462-0a24-41ec-aab0-41a5e8df0db9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:13.163091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3a1ff462-0a24-41ec-aab0-41a5e8df0db9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invalidating the token cache because role 8e8bb5b4e316492ba51a83b109e89ec1 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:13.163951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3a1ff462-0a24-41ec-aab0-41a5e8df0db9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:13.165054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3a1ff462-0a24-41ec-aab0-41a5e8df0db9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:13.168118 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2128/4326] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:12 2022] DELETE /identity/v3/roles/8e8bb5b4e316492ba51a83b109e89ec1 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:13.173065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1094be-99fe-43e4-8311-85331e58e60e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:13.205874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1094be-99fe-43e4-8311-85331e58e60e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:13.234091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:INSERT=7,keystone:DELETE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:13.450309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21444f14-8999-4960-bd5d-8c0c7705fc22 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:13.451794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21444f14-8999-4960-bd5d-8c0c7705fc22 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:13.452805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21444f14-8999-4960-bd5d-8c0c7705fc22 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:13.453260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21444f14-8999-4960-bd5d-8c0c7705fc22 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/b4075ffb7dc54af7bc5c170586efbec1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:13.464432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21444f14-8999-4960-bd5d-8c0c7705fc22 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_grant(project_id=f69494f2b2fc40f29be4d38fe0a42b87, user_id=1ec430a8ba524da0af3eef8e1e8d9d33, role_id=b4075ffb7dc54af7bc5c170586efbec1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:13.468075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21444f14-8999-4960-bd5d-8c0c7705fc22 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:13.532162 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2199/4327] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:13 2022] PUT /identity/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/b4075ffb7dc54af7bc5c170586efbec1 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:13.536574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ff462-0a24-41ec-aab0-41a5e8df0db9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:13.539347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3a1ff462-0a24-41ec-aab0-41a5e8df0db9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:13.563844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ff462-0a24-41ec-aab0-41a5e8df0db9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:13.581075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:13.834928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0cf04a-6295-4e68-8af4-bc2bed917b7e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:13.836244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cf04a-6295-4e68-8af4-bc2bed917b7e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:13.836636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cf04a-6295-4e68-8af4-bc2bed917b7e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:13.837513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cf04a-6295-4e68-8af4-bc2bed917b7e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:13.840831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0cf04a-6295-4e68-8af4-bc2bed917b7e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=b4075ffb7dc54af7bc5c170586efbec1, implied_role_id=a961c41374514b48ad472fddaade0d27)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:13.842036 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-1f0cf04a-6295-4e68-8af4-bc2bed917b7e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:13.842036 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:13.842036 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:13.843270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0cf04a-6295-4e68-8af4-bc2bed917b7e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:13.909402 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2129/4328] 10.222.0.17 () {60 vars in 1363 bytes} [Sat Oct 1 21:03:13 2022] PUT /identity/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27 => generated 521 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:03:13.911358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21444f14-8999-4960-bd5d-8c0c7705fc22 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:13.928495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21444f14-8999-4960-bd5d-8c0c7705fc22 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:14.171636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d077a5-add1-4fd9-a784-22dd7d464730 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:14.172437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d077a5-add1-4fd9-a784-22dd7d464730 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:14.172825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d077a5-add1-4fd9-a784-22dd7d464730 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:14.173207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d077a5-add1-4fd9-a784-22dd7d464730 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:14.174064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d077a5-add1-4fd9-a784-22dd7d464730 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Adding query filter params (scope.project.id=f69494f2b2fc40f29be4d38fe0a42b87, user.id=1ec430a8ba524da0af3eef8e1e8d9d33) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:03:14.174484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d077a5-add1-4fd9-a784-22dd7d464730 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:14.175825 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-81d077a5-add1-4fd9-a784-22dd7d464730 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_assignments": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:14.175825 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:14.175825 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:14.177324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d077a5-add1-4fd9-a784-22dd7d464730 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:14.248706 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2200/4329] 10.222.0.17 () {58 vars in 1356 bytes} [Sat Oct 1 21:03:13 2022] GET /identity/v3/role_assignments?scope.project.id=f69494f2b2fc40f29be4d38fe0a42b87&user.id=1ec430a8ba524da0af3eef8e1e8d9d33&effective => generated 1061 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:14.252939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0cf04a-6295-4e68-8af4-bc2bed917b7e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:14.273908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0cf04a-6295-4e68-8af4-bc2bed917b7e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:14.289897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca97c5-672a-4588-8d03-bcca21fe3fe9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:14.298582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca97c5-672a-4588-8d03-bcca21fe3fe9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:14.300178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca97c5-672a-4588-8d03-bcca21fe3fe9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:14.300476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca97c5-672a-4588-8d03-bcca21fe3fe9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:14.304344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca97c5-672a-4588-8d03-bcca21fe3fe9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=b4075ffb7dc54af7bc5c170586efbec1, implied_role_id=a961c41374514b48ad472fddaade0d27)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:14.307771 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-eaca97c5-672a-4588-8d03-bcca21fe3fe9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:14.307771 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:14.307771 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:14.309216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca97c5-672a-4588-8d03-bcca21fe3fe9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:14.354706 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2130/4330] 10.222.0.17 () {58 vars in 1347 bytes} [Sat Oct 1 21:03:14 2022] DELETE /identity/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:14.359113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d077a5-add1-4fd9-a784-22dd7d464730 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:14.377991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d077a5-add1-4fd9-a784-22dd7d464730 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:14.547692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabe2ae-ce7c-406b-b785-561789c22782 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:14.548418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabe2ae-ce7c-406b-b785-561789c22782 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:14.548906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabe2ae-ce7c-406b-b785-561789c22782 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:14.549452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabe2ae-ce7c-406b-b785-561789c22782 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:14.551312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aabe2ae-ce7c-406b-b785-561789c22782 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Adding query filter params (scope.project.id=f69494f2b2fc40f29be4d38fe0a42b87, user.id=1ec430a8ba524da0af3eef8e1e8d9d33) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:03:14.552381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aabe2ae-ce7c-406b-b785-561789c22782 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:14.554612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aabe2ae-ce7c-406b-b785-561789c22782 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:14.565283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:14.664867 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2201/4331] 10.222.0.17 () {58 vars in 1356 bytes} [Sat Oct 1 21:03:14 2022] GET /identity/v3/role_assignments?scope.project.id=f69494f2b2fc40f29be4d38fe0a42b87&user.id=1ec430a8ba524da0af3eef8e1e8d9d33&effective => generated 577 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:14.669042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca97c5-672a-4588-8d03-bcca21fe3fe9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:14.686169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca97c5-672a-4588-8d03-bcca21fe3fe9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:14.694010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380d58f-db0c-474e-b810-321dee36ca47 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:14.695433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380d58f-db0c-474e-b810-321dee36ca47 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:14.695740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380d58f-db0c-474e-b810-321dee36ca47 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:14.696024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380d58f-db0c-474e-b810-321dee36ca47 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:14.698045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380d58f-db0c-474e-b810-321dee36ca47 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=b4075ffb7dc54af7bc5c170586efbec1, implied_role_id=a961c41374514b48ad472fddaade0d27)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:14.699864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380d58f-db0c-474e-b810-321dee36ca47 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:14.763602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-5380d58f-db0c-474e-b810-321dee36ca47 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] b4075ffb7dc54af7bc5c170586efbec1 does not imply a961c41374514b48ad472fddaade0d27.: keystone.exception.ImpliedRoleNotFound: b4075ffb7dc54af7bc5c170586efbec1 does not imply a961c41374514b48ad472fddaade0d27. Oct 01 21:03:14.763602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:03:14.763602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Oct 01 21:03:14.763602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = query.one() Oct 01 21:03:14.763602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Oct 01 21:03:14.763602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._iter().one() Oct 01 21:03:14.763602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Oct 01 21:03:14.763602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._only_one_row( Oct 01 21:03:14.763602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Oct 01 21:03:14.763602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exc.NoResultFound( Oct 01 21:03:14.763602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Oct 01 21:03:14.763602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:03:14.763602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:03:14.763602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:03:14.763602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:03:14.763602 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:03:14.764876 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:03:14.764876 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:03:14.764876 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:03:14.764876 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:03:14.764876 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:03:14.764876 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:03:14.764876 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:03:14.764876 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:03:14.764876 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:03:14.764876 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 270, in delete Oct 01 21:03:14.764876 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application PROVIDERS.role_api.delete_implied_role(prior_role_id, implied_role_id) Oct 01 21:03:14.764876 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:03:14.764876 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:03:14.764876 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1369, in delete_implied_role Oct 01 21:03:14.764876 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.driver.delete_implied_role(prior_role_id, implied_role_id) Oct 01 21:03:14.764876 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 129, in delete_implied_role Oct 01 21:03:14.765974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Oct 01 21:03:14.765974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Oct 01 21:03:14.765974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Oct 01 21:03:14.765974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: b4075ffb7dc54af7bc5c170586efbec1 does not imply a961c41374514b48ad472fddaade0d27. Oct 01 21:03:14.765974 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:03:14.765974 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2131/4332] 10.222.0.17 () {58 vars in 1347 bytes} [Sat Oct 1 21:03:14 2022] DELETE /identity/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27 => generated 137 bytes in 97 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:03:14.769312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabe2ae-ce7c-406b-b785-561789c22782 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:14.780765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabe2ae-ce7c-406b-b785-561789c22782 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:14.787230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce6545-2c6b-49a0-9ec6-ec336fbabde4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:14.788434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce6545-2c6b-49a0-9ec6-ec336fbabde4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:14.788937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce6545-2c6b-49a0-9ec6-ec336fbabde4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:14.789430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce6545-2c6b-49a0-9ec6-ec336fbabde4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/b4075ffb7dc54af7bc5c170586efbec1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:14.795246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce6545-2c6b-49a0-9ec6-ec336fbabde4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=f69494f2b2fc40f29be4d38fe0a42b87, user_id=1ec430a8ba524da0af3eef8e1e8d9d33, role_id=b4075ffb7dc54af7bc5c170586efbec1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:14.800014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce6545-2c6b-49a0-9ec6-ec336fbabde4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:14.807973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e5ce6545-2c6b-49a0-9ec6-ec336fbabde4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invalidating the token cache because role b4075ffb7dc54af7bc5c170586efbec1 was removed from user 1ec430a8ba524da0af3eef8e1e8d9d33 on project f69494f2b2fc40f29be4d38fe0a42b87. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:14.809015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e5ce6545-2c6b-49a0-9ec6-ec336fbabde4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:14.810021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e5ce6545-2c6b-49a0-9ec6-ec336fbabde4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:14.847928 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2202/4333] 10.222.0.17 () {58 vars in 1467 bytes} [Sat Oct 1 21:03:14 2022] DELETE /identity/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/b4075ffb7dc54af7bc5c170586efbec1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:14.853771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380d58f-db0c-474e-b810-321dee36ca47 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:14.855941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5380d58f-db0c-474e-b810-321dee36ca47 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:14.883250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380d58f-db0c-474e-b810-321dee36ca47 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:15.051201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5b929-217f-48cf-b7a3-40b3fcfe1723 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:15.055316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5b929-217f-48cf-b7a3-40b3fcfe1723 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:15.056597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5b929-217f-48cf-b7a3-40b3fcfe1723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:15.153377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf664d69-3209-4b05-b875-e47ae83e3e45 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:15.155501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf664d69-3209-4b05-b875-e47ae83e3e45 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:15.156170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf664d69-3209-4b05-b875-e47ae83e3e45 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:15.157075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf664d69-3209-4b05-b875-e47ae83e3e45 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:15.160575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf664d69-3209-4b05-b875-e47ae83e3e45 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:15.167164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf664d69-3209-4b05-b875-e47ae83e3e45 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:15.186949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-05e5b929-217f-48cf-b7a3-40b3fcfe1723 None None] MFA Rules not processed for user `aa4fbb28c9de4089870d91cf52dac8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:15.188574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-05e5b929-217f-48cf-b7a3-40b3fcfe1723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:15.208846 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2132/4334] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:03:14 2022] POST /identity/v3/roles => generated 272 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:15.210688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf664d69-3209-4b05-b875-e47ae83e3e45 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:15.258465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf664d69-3209-4b05-b875-e47ae83e3e45 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:15.280363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def66f7-51bb-4848-b9a3-a54c119d3a14 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:15.281228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def66f7-51bb-4848-b9a3-a54c119d3a14 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:15.281623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def66f7-51bb-4848-b9a3-a54c119d3a14 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:15.282009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def66f7-51bb-4848-b9a3-a54c119d3a14 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:15.285215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def66f7-51bb-4848-b9a3-a54c119d3a14 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Adding query filter params (domain_id=c2c3d42d4aa4486aa40ca857514b1d86) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:03:15.285654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def66f7-51bb-4848-b9a3-a54c119d3a14 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:list_domain_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:15.288668 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-8def66f7-51bb-4848-b9a3-a54c119d3a14 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domain_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:15.288668 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:15.288668 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:15.294106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def66f7-51bb-4848-b9a3-a54c119d3a14 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:15.356183 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2133/4335] 10.222.0.17 () {58 vars in 1207 bytes} [Sat Oct 1 21:03:15 2022] GET /identity/v3/roles?domain_id=c2c3d42d4aa4486aa40ca857514b1d86 => generated 409 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:15.357312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def66f7-51bb-4848-b9a3-a54c119d3a14 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:15.418205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def66f7-51bb-4848-b9a3-a54c119d3a14 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:15.444974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ad993-589b-46be-a594-439c93dedcc5 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:15.445630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ad993-589b-46be-a594-439c93dedcc5 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:15.445915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ad993-589b-46be-a594-439c93dedcc5 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:15.446193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ad993-589b-46be-a594-439c93dedcc5 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/d289cb87560547d5a57cbf5442eda975` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:15.455468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ad993-589b-46be-a594-439c93dedcc5 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=d289cb87560547d5a57cbf5442eda975)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:15.457121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ad993-589b-46be-a594-439c93dedcc5 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:15.524932 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2203/4336] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:15 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:15.530647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689ac5f-b45c-4e77-a980-9053b6f09685 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:15.532176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689ac5f-b45c-4e77-a980-9053b6f09685 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:15.533230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689ac5f-b45c-4e77-a980-9053b6f09685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:15.565838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bc8ad993-589b-46be-a594-439c93dedcc5 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invalidating the token cache because role d289cb87560547d5a57cbf5442eda975 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:15.566116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bc8ad993-589b-46be-a594-439c93dedcc5 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:15.568056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bc8ad993-589b-46be-a594-439c93dedcc5 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:15.571676 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2134/4337] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:15 2022] DELETE /identity/v3/roles/d289cb87560547d5a57cbf5442eda975 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:15.573576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5689ac5f-b45c-4e77-a980-9053b6f09685 None None] MFA Rules not processed for user `aa4fbb28c9de4089870d91cf52dac8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:15.575208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5689ac5f-b45c-4e77-a980-9053b6f09685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:15.575766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ad993-589b-46be-a594-439c93dedcc5 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:15.578764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bc8ad993-589b-46be-a594-439c93dedcc5 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:15.618725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:15.696011 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2204/4338] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:15 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:15.742940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ad993-589b-46be-a594-439c93dedcc5 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:15.749539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e54345-93c6-424c-a1fe-1bdc0af81304 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:15.750305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e54345-93c6-424c-a1fe-1bdc0af81304 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:15.750745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e54345-93c6-424c-a1fe-1bdc0af81304 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:15.751133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e54345-93c6-424c-a1fe-1bdc0af81304 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:15.751961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e54345-93c6-424c-a1fe-1bdc0af81304 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Adding query filter params (domain_id=c2c3d42d4aa4486aa40ca857514b1d86) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:03:15.759149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e54345-93c6-424c-a1fe-1bdc0af81304 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:list_domain_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:15.761002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e54345-93c6-424c-a1fe-1bdc0af81304 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:15.769650 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2135/4339] 10.222.0.17 () {58 vars in 1207 bytes} [Sat Oct 1 21:03:15 2022] GET /identity/v3/roles?domain_id=c2c3d42d4aa4486aa40ca857514b1d86 => generated 148 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:15.774099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689ac5f-b45c-4e77-a980-9053b6f09685 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:15.796543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689ac5f-b45c-4e77-a980-9053b6f09685 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:15.821423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010f834-8392-474e-90b4-7b58615317cb tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:15.822079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010f834-8392-474e-90b4-7b58615317cb tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:15.822359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010f834-8392-474e-90b4-7b58615317cb tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:15.822655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010f834-8392-474e-90b4-7b58615317cb tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/d289cb87560547d5a57cbf5442eda975` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:15.859801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010f834-8392-474e-90b4-7b58615317cb tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 21:03:15.874313 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-5010f834-8392-474e-90b4-7b58615317cb tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Could not find role: d289cb87560547d5a57cbf5442eda975.: keystone.exception.RoleNotFound: Could not find role: d289cb87560547d5a57cbf5442eda975. Oct 01 21:03:15.874313 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:03:15.874313 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:03:15.874313 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:03:15.874313 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:03:15.874313 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:03:15.874313 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:03:15.874313 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:03:15.874313 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:03:15.874313 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:03:15.874313 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:03:15.874313 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:03:15.874313 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 165, in delete Oct 01 21:03:15.874313 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_role') Oct 01 21:03:15.875585 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 21:03:15.875585 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 21:03:15.875585 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 21:03:15.875585 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 21:03:15.875585 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:03:15.875585 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:03:15.875585 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:03:15.875585 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:03:15.875585 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:03:15.875585 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:03:15.875585 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:03:15.875585 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 21:03:15.875585 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:03:15.875585 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:03:15.875585 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:03:15.875585 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:03:15.876791 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:03:15.876791 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:03:15.876791 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:03:15.876791 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:03:15.876791 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 01 21:03:15.876791 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 01 21:03:15.876791 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 01 21:03:15.876791 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 01 21:03:15.876791 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 01 21:03:15.876791 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 21:03:15.876791 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: d289cb87560547d5a57cbf5442eda975. Oct 01 21:03:15.876791 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:03:15.882415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e54345-93c6-424c-a1fe-1bdc0af81304 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:15.882779 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2205/4340] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:15 2022] DELETE /identity/v3/roles/d289cb87560547d5a57cbf5442eda975 => generated 110 bytes in 109 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Oct 01 21:03:15.905339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e54345-93c6-424c-a1fe-1bdc0af81304 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:15.925043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5c1e6-8790-42d4-9fad-44e55ab9bfc0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:15.925770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5c1e6-8790-42d4-9fad-44e55ab9bfc0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:15.926053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5c1e6-8790-42d4-9fad-44e55ab9bfc0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:15.930234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5c1e6-8790-42d4-9fad-44e55ab9bfc0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/domains/c2c3d42d4aa4486aa40ca857514b1d86/groups/2d025e446f854f71a7d59440f98f3eaf/roles/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:15.947900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5c1e6-8790-42d4-9fad-44e55ab9bfc0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=c2c3d42d4aa4486aa40ca857514b1d86, group_id=2d025e446f854f71a7d59440f98f3eaf, role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:15.951952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5c1e6-8790-42d4-9fad-44e55ab9bfc0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:15.994649 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2136/4341] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:15 2022] PUT /identity/v3/domains/c2c3d42d4aa4486aa40ca857514b1d86/groups/2d025e446f854f71a7d59440f98f3eaf/roles/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:15.998579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010f834-8392-474e-90b4-7b58615317cb tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:16.015641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010f834-8392-474e-90b4-7b58615317cb tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:16.082278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:16.193620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c3bf0-c945-4970-8f6c-44192204ad7a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:16.194831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c3bf0-c945-4970-8f6c-44192204ad7a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:16.196038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c3bf0-c945-4970-8f6c-44192204ad7a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:16.196488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c3bf0-c945-4970-8f6c-44192204ad7a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/domains/c2c3d42d4aa4486aa40ca857514b1d86/groups/2d025e446f854f71a7d59440f98f3eaf/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:16.204895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c3bf0-c945-4970-8f6c-44192204ad7a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=c2c3d42d4aa4486aa40ca857514b1d86, group_id=2d025e446f854f71a7d59440f98f3eaf)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:16.206303 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-0c4c3bf0-c945-4970-8f6c-44192204ad7a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_grants": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:16.206303 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:16.206303 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:16.206641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5c1e6-8790-42d4-9fad-44e55ab9bfc0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:16.208690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c3bf0-c945-4970-8f6c-44192204ad7a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:16.209170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-16c5c1e6-8790-42d4-9fad-44e55ab9bfc0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:16.246681 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2206/4342] 10.222.0.17 () {58 vars in 1365 bytes} [Sat Oct 1 21:03:15 2022] GET /identity/v3/domains/c2c3d42d4aa4486aa40ca857514b1d86/groups/2d025e446f854f71a7d59440f98f3eaf/roles => generated 409 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:16.255488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c3bf0-c945-4970-8f6c-44192204ad7a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:16.299810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c3bf0-c945-4970-8f6c-44192204ad7a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:16.331233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bce4d-8f8f-4706-992a-5c92b3973593 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:16.332056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bce4d-8f8f-4706-992a-5c92b3973593 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:16.332470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bce4d-8f8f-4706-992a-5c92b3973593 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:16.333802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bce4d-8f8f-4706-992a-5c92b3973593 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/domains/c2c3d42d4aa4486aa40ca857514b1d86/groups/2d025e446f854f71a7d59440f98f3eaf/roles/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:16.343893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bce4d-8f8f-4706-992a-5c92b3973593 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=c2c3d42d4aa4486aa40ca857514b1d86, group_id=2d025e446f854f71a7d59440f98f3eaf, role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:16.345116 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-e76bce4d-8f8f-4706-992a-5c92b3973593 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_grant": "(role:reader and system_scope:all) or ((role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:16.345116 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:16.345116 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:16.351534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bce4d-8f8f-4706-992a-5c92b3973593 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:16.368826 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2207/4343] 10.222.0.17 () {58 vars in 1465 bytes} [Sat Oct 1 21:03:16 2022] HEAD /identity/v3/domains/c2c3d42d4aa4486aa40ca857514b1d86/groups/2d025e446f854f71a7d59440f98f3eaf/roles/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:16.372952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bce4d-8f8f-4706-992a-5c92b3973593 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:16.391521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bce4d-8f8f-4706-992a-5c92b3973593 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:16.402796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bcf674-0f75-4dd0-ac33-d7a40d3e3896 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:16.405745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcf674-0f75-4dd0-ac33-d7a40d3e3896 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:16.406199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcf674-0f75-4dd0-ac33-d7a40d3e3896 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:16.406633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcf674-0f75-4dd0-ac33-d7a40d3e3896 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/domains/c2c3d42d4aa4486aa40ca857514b1d86/groups/2d025e446f854f71a7d59440f98f3eaf/roles/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:16.419222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bcf674-0f75-4dd0-ac33-d7a40d3e3896 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=c2c3d42d4aa4486aa40ca857514b1d86, group_id=2d025e446f854f71a7d59440f98f3eaf, role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:16.422473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bcf674-0f75-4dd0-ac33-d7a40d3e3896 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:16.425222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5c1e6-8790-42d4-9fad-44e55ab9bfc0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:16.433542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28e6c9-130a-4d69-aa2b-8c9f89c93237 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:16.434802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28e6c9-130a-4d69-aa2b-8c9f89c93237 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:16.434949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-31bcf674-0f75-4dd0-ac33-d7a40d3e3896 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invalidating the token cache because role 4dccb2174d5342a2ae2e058c75f23e10 was removed from group 2d025e446f854f71a7d59440f98f3eaf on domain c2c3d42d4aa4486aa40ca857514b1d86. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:16.435102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28e6c9-130a-4d69-aa2b-8c9f89c93237 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:16.435377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28e6c9-130a-4d69-aa2b-8c9f89c93237 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:16.435510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-31bcf674-0f75-4dd0-ac33-d7a40d3e3896 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:16.436341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-31bcf674-0f75-4dd0-ac33-d7a40d3e3896 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:16.440984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2f28e6c9-130a-4d69-aa2b-8c9f89c93237 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:16.471365 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2208/4344] 10.222.0.17 () {58 vars in 1467 bytes} [Sat Oct 1 21:03:16 2022] DELETE /identity/v3/domains/c2c3d42d4aa4486aa40ca857514b1d86/groups/2d025e446f854f71a7d59440f98f3eaf/roles/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:16.477256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bcf674-0f75-4dd0-ac33-d7a40d3e3896 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:16.479919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-31bcf674-0f75-4dd0-ac33-d7a40d3e3896 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:16.585268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f28e6c9-130a-4d69-aa2b-8c9f89c93237 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:16.587496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f28e6c9-130a-4d69-aa2b-8c9f89c93237 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:16.619491 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2137/4345] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:16 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:03:16.624022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bcf674-0f75-4dd0-ac33-d7a40d3e3896 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:16.635604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83750873-f9a7-43be-a2e6-816440b7454a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:16.636099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83750873-f9a7-43be-a2e6-816440b7454a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:16.636501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83750873-f9a7-43be-a2e6-816440b7454a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:16.636894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83750873-f9a7-43be-a2e6-816440b7454a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/groups/2d025e446f854f71a7d59440f98f3eaf/roles/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:16.646832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83750873-f9a7-43be-a2e6-816440b7454a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_grant(project_id=f69494f2b2fc40f29be4d38fe0a42b87, group_id=2d025e446f854f71a7d59440f98f3eaf, role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:16.659220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83750873-f9a7-43be-a2e6-816440b7454a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:16.742386 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2209/4346] 10.222.0.17 () {60 vars in 1486 bytes} [Sat Oct 1 21:03:16 2022] PUT /identity/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/groups/2d025e446f854f71a7d59440f98f3eaf/roles/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:16.744150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28e6c9-130a-4d69-aa2b-8c9f89c93237 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:16.772148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28e6c9-130a-4d69-aa2b-8c9f89c93237 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:16.795751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:16.883397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869da4b-a492-4c26-b1b0-2faf09d37598 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:16.885874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869da4b-a492-4c26-b1b0-2faf09d37598 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:16.886281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869da4b-a492-4c26-b1b0-2faf09d37598 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:16.886704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869da4b-a492-4c26-b1b0-2faf09d37598 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/groups/2d025e446f854f71a7d59440f98f3eaf/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:16.888596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869da4b-a492-4c26-b1b0-2faf09d37598 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:list_grants(project_id=f69494f2b2fc40f29be4d38fe0a42b87, group_id=2d025e446f854f71a7d59440f98f3eaf)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:16.889745 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-e869da4b-a492-4c26-b1b0-2faf09d37598 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_grants": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:16.889745 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:16.889745 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:16.893455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869da4b-a492-4c26-b1b0-2faf09d37598 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:16.940781 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2138/4347] 10.222.0.17 () {58 vars in 1368 bytes} [Sat Oct 1 21:03:16 2022] GET /identity/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/groups/2d025e446f854f71a7d59440f98f3eaf/roles => generated 410 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:03:16.945176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83750873-f9a7-43be-a2e6-816440b7454a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:16.967830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83750873-f9a7-43be-a2e6-816440b7454a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:16.989060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fda111-2443-46e5-8a05-276395bffda6 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:16.989747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fda111-2443-46e5-8a05-276395bffda6 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:16.990039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fda111-2443-46e5-8a05-276395bffda6 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:16.991601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fda111-2443-46e5-8a05-276395bffda6 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/groups/2d025e446f854f71a7d59440f98f3eaf/users/1ec430a8ba524da0af3eef8e1e8d9d33` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:16.998744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fda111-2443-46e5-8a05-276395bffda6 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=2d025e446f854f71a7d59440f98f3eaf, user_id=1ec430a8ba524da0af3eef8e1e8d9d33)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:17.001962 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-b6fda111-2443-46e5-8a05-276395bffda6 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:add_user_to_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:17.001962 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:17.001962 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:17.005010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fda111-2443-46e5-8a05-276395bffda6 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:17.066895 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2210/4348] 10.222.0.17 () {60 vars in 1360 bytes} [Sat Oct 1 21:03:16 2022] PUT /identity/v3/groups/2d025e446f854f71a7d59440f98f3eaf/users/1ec430a8ba524da0af3eef8e1e8d9d33 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:17.073960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad20b5-c99b-4677-a77a-ad272a21093e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:17.074427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad20b5-c99b-4677-a77a-ad272a21093e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:17.074845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad20b5-c99b-4677-a77a-ad272a21093e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:17.151475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b6ad20b5-c99b-4677-a77a-ad272a21093e None None] MFA Rules not processed for user `1ec430a8ba524da0af3eef8e1e8d9d33`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:17.153512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b6ad20b5-c99b-4677-a77a-ad272a21093e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:17.355826 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2139/4349] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:17 2022] POST /identity/v3/auth/tokens => generated 3263 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:17.359877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fda111-2443-46e5-8a05-276395bffda6 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:17.523217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fda111-2443-46e5-8a05-276395bffda6 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:17.542125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9a9d1-7ff1-4d82-819c-809ce01074e7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:17.542955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9a9d1-7ff1-4d82-819c-809ce01074e7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:17.543310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9a9d1-7ff1-4d82-819c-809ce01074e7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:17.543747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9a9d1-7ff1-4d82-819c-809ce01074e7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/groups/2d025e446f854f71a7d59440f98f3eaf/roles/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:17.546241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e9a9d1-7ff1-4d82-819c-809ce01074e7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:check_grant(project_id=f69494f2b2fc40f29be4d38fe0a42b87, group_id=2d025e446f854f71a7d59440f98f3eaf, role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:17.551070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e9a9d1-7ff1-4d82-819c-809ce01074e7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:17.563913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:17.579176 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2211/4350] 10.222.0.17 () {58 vars in 1468 bytes} [Sat Oct 1 21:03:17 2022] HEAD /identity/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/groups/2d025e446f854f71a7d59440f98f3eaf/roles/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:17.580681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad20b5-c99b-4677-a77a-ad272a21093e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:17.599430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad20b5-c99b-4677-a77a-ad272a21093e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:17.614597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62957426-be09-40d6-8e08-0135daf79a4b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:17.615914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62957426-be09-40d6-8e08-0135daf79a4b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:17.616231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62957426-be09-40d6-8e08-0135daf79a4b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:17.616501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62957426-be09-40d6-8e08-0135daf79a4b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/groups/2d025e446f854f71a7d59440f98f3eaf/roles/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:17.628068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62957426-be09-40d6-8e08-0135daf79a4b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=f69494f2b2fc40f29be4d38fe0a42b87, group_id=2d025e446f854f71a7d59440f98f3eaf, role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:17.629116 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-62957426-be09-40d6-8e08-0135daf79a4b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:17.629116 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:17.629116 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:17.631150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62957426-be09-40d6-8e08-0135daf79a4b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:17.648720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-62957426-be09-40d6-8e08-0135daf79a4b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invalidating the token cache because role 4dccb2174d5342a2ae2e058c75f23e10 was removed from group 2d025e446f854f71a7d59440f98f3eaf on project f69494f2b2fc40f29be4d38fe0a42b87. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:17.649025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-62957426-be09-40d6-8e08-0135daf79a4b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:17.649770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-62957426-be09-40d6-8e08-0135daf79a4b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:17.683111 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2140/4351] 10.222.0.17 () {58 vars in 1470 bytes} [Sat Oct 1 21:03:17 2022] DELETE /identity/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/groups/2d025e446f854f71a7d59440f98f3eaf/roles/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:17.684112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9a9d1-7ff1-4d82-819c-809ce01074e7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:17.685900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f2e9a9d1-7ff1-4d82-819c-809ce01074e7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:17.710992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9a9d1-7ff1-4d82-819c-809ce01074e7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:17.871851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfddd501-5520-411a-9726-434019c0f6b2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:17.872840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddd501-5520-411a-9726-434019c0f6b2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:17.873329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddd501-5520-411a-9726-434019c0f6b2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:17.873717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddd501-5520-411a-9726-434019c0f6b2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/groups/2d025e446f854f71a7d59440f98f3eaf/users/1ec430a8ba524da0af3eef8e1e8d9d33` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:17.880669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfddd501-5520-411a-9726-434019c0f6b2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=2d025e446f854f71a7d59440f98f3eaf, user_id=1ec430a8ba524da0af3eef8e1e8d9d33)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:17.882331 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-cfddd501-5520-411a-9726-434019c0f6b2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:remove_user_from_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:17.882331 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:17.882331 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:17.886200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfddd501-5520-411a-9726-434019c0f6b2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:17.920077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cfddd501-5520-411a-9726-434019c0f6b2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '1ec430a8ba524da0af3eef8e1e8d9d33'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:17.979571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62957426-be09-40d6-8e08-0135daf79a4b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:17.981371 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2212/4352] 10.222.0.17 () {58 vars in 1344 bytes} [Sat Oct 1 21:03:17 2022] DELETE /identity/v3/groups/2d025e446f854f71a7d59440f98f3eaf/users/1ec430a8ba524da0af3eef8e1e8d9d33 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:17.995682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62957426-be09-40d6-8e08-0135daf79a4b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:18.179806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:18.227454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e074723-b0ec-460d-9789-7c6d4e80f21b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:18.228232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e074723-b0ec-460d-9789-7c6d4e80f21b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:18.228366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e074723-b0ec-460d-9789-7c6d4e80f21b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:18.228642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e074723-b0ec-460d-9789-7c6d4e80f21b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/system/groups/2d025e446f854f71a7d59440f98f3eaf/roles/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:18.236249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e074723-b0ec-460d-9789-7c6d4e80f21b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_system_grant_for_group(group_id=2d025e446f854f71a7d59440f98f3eaf, role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:18.242724 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-3e074723-b0ec-460d-9789-7c6d4e80f21b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:18.242724 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:18.242724 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:18.243887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e074723-b0ec-460d-9789-7c6d4e80f21b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:18.272786 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2141/4353] 10.222.0.17 () {60 vars in 1381 bytes} [Sat Oct 1 21:03:17 2022] PUT /identity/v3/system/groups/2d025e446f854f71a7d59440f98f3eaf/roles/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:18.277141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfddd501-5520-411a-9726-434019c0f6b2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:18.293911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfddd501-5520-411a-9726-434019c0f6b2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:18.310997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fbfbd-ac22-4282-921d-53a13bf8051a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:18.312520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fbfbd-ac22-4282-921d-53a13bf8051a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:18.312927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fbfbd-ac22-4282-921d-53a13bf8051a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:18.313645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fbfbd-ac22-4282-921d-53a13bf8051a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/system/groups/2d025e446f854f71a7d59440f98f3eaf/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:18.317937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fbfbd-ac22-4282-921d-53a13bf8051a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:list_system_grants_for_group(group_id=2d025e446f854f71a7d59440f98f3eaf)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:18.323673 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-3e9fbfbd-ac22-4282-921d-53a13bf8051a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_system_grants_for_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:18.323673 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:18.323673 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:18.324887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fbfbd-ac22-4282-921d-53a13bf8051a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:18.351611 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2213/4354] 10.222.0.17 () {58 vars in 1263 bytes} [Sat Oct 1 21:03:18 2022] GET /identity/v3/system/groups/2d025e446f854f71a7d59440f98f3eaf/roles => generated 375 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:18.356381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e074723-b0ec-460d-9789-7c6d4e80f21b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:18.371100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e074723-b0ec-460d-9789-7c6d4e80f21b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:18.386596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a23720-7e70-4d63-b6aa-7c7c2488e5f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:18.387512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a23720-7e70-4d63-b6aa-7c7c2488e5f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:18.387971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a23720-7e70-4d63-b6aa-7c7c2488e5f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:18.388357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a23720-7e70-4d63-b6aa-7c7c2488e5f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/system/groups/2d025e446f854f71a7d59440f98f3eaf/roles/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:18.391424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a23720-7e70-4d63-b6aa-7c7c2488e5f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:check_system_grant_for_group(group_id=2d025e446f854f71a7d59440f98f3eaf, role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:18.396617 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-f0a23720-7e70-4d63-b6aa-7c7c2488e5f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_system_grant_for_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:18.396617 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:18.396617 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:18.397810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a23720-7e70-4d63-b6aa-7c7c2488e5f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:18.429027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fbfbd-ac22-4282-921d-53a13bf8051a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:18.438678 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2142/4355] 10.222.0.17 () {58 vars in 1363 bytes} [Sat Oct 1 21:03:18 2022] HEAD /identity/v3/system/groups/2d025e446f854f71a7d59440f98f3eaf/roles/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:18.448543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fbfbd-ac22-4282-921d-53a13bf8051a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:18.458966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdd07d-9788-4d21-9d5b-e8a6d84c6e1d tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:18.459642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdd07d-9788-4d21-9d5b-e8a6d84c6e1d tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:18.459935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdd07d-9788-4d21-9d5b-e8a6d84c6e1d tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:18.460205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdd07d-9788-4d21-9d5b-e8a6d84c6e1d tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/system/groups/2d025e446f854f71a7d59440f98f3eaf/roles/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:18.467929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfdd07d-9788-4d21-9d5b-e8a6d84c6e1d tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:revoke_system_grant_for_group(group_id=2d025e446f854f71a7d59440f98f3eaf, role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:18.469109 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-7dfdd07d-9788-4d21-9d5b-e8a6d84c6e1d tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_system_grant_for_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:18.469109 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:18.469109 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:18.470025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfdd07d-9788-4d21-9d5b-e8a6d84c6e1d tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:18.499730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a23720-7e70-4d63-b6aa-7c7c2488e5f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:18.500894 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2214/4356] 10.222.0.17 () {58 vars in 1365 bytes} [Sat Oct 1 21:03:18 2022] DELETE /identity/v3/system/groups/2d025e446f854f71a7d59440f98f3eaf/roles/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:18.513474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a23720-7e70-4d63-b6aa-7c7c2488e5f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:18.531119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2dfcf-c31e-451e-ad7e-75766b87a8e8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:18.531716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2dfcf-c31e-451e-ad7e-75766b87a8e8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:18.532005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2dfcf-c31e-451e-ad7e-75766b87a8e8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:18.532286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2dfcf-c31e-451e-ad7e-75766b87a8e8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/domains/c2c3d42d4aa4486aa40ca857514b1d86/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:18.541487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2dfcf-c31e-451e-ad7e-75766b87a8e8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=c2c3d42d4aa4486aa40ca857514b1d86, user_id=1ec430a8ba524da0af3eef8e1e8d9d33, role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:18.544931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2dfcf-c31e-451e-ad7e-75766b87a8e8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:18.575732 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2143/4357] 10.222.0.17 () {60 vars in 1480 bytes} [Sat Oct 1 21:03:18 2022] PUT /identity/v3/domains/c2c3d42d4aa4486aa40ca857514b1d86/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:18.576883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdd07d-9788-4d21-9d5b-e8a6d84c6e1d tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:18.593239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdd07d-9788-4d21-9d5b-e8a6d84c6e1d tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:18.742151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fd0b0-831c-4d0b-8f56-5ca2b10c4f09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:18.743596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fd0b0-831c-4d0b-8f56-5ca2b10c4f09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:18.744158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fd0b0-831c-4d0b-8f56-5ca2b10c4f09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:18.744916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fd0b0-831c-4d0b-8f56-5ca2b10c4f09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/domains/c2c3d42d4aa4486aa40ca857514b1d86/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:18.746949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30fd0b0-831c-4d0b-8f56-5ca2b10c4f09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=c2c3d42d4aa4486aa40ca857514b1d86, user_id=1ec430a8ba524da0af3eef8e1e8d9d33)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:18.749714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30fd0b0-831c-4d0b-8f56-5ca2b10c4f09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:18.780386 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2215/4358] 10.222.0.17 () {58 vars in 1362 bytes} [Sat Oct 1 21:03:18 2022] GET /identity/v3/domains/c2c3d42d4aa4486aa40ca857514b1d86/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles => generated 408 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:03:18.787458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2dfcf-c31e-451e-ad7e-75766b87a8e8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:18.851118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2dfcf-c31e-451e-ad7e-75766b87a8e8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:18.858309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42e345-6da8-402d-a407-88e04de5d73a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:18.858701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42e345-6da8-402d-a407-88e04de5d73a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:18.861804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42e345-6da8-402d-a407-88e04de5d73a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:18.862053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42e345-6da8-402d-a407-88e04de5d73a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/domains/c2c3d42d4aa4486aa40ca857514b1d86/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:18.868888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e42e345-6da8-402d-a407-88e04de5d73a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=c2c3d42d4aa4486aa40ca857514b1d86, user_id=1ec430a8ba524da0af3eef8e1e8d9d33, role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:18.871153 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-9e42e345-6da8-402d-a407-88e04de5d73a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_grant": "(role:reader and system_scope:all) or ((role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:18.871153 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:18.871153 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:18.876108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e42e345-6da8-402d-a407-88e04de5d73a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:18.898227 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2144/4359] 10.222.0.17 () {58 vars in 1462 bytes} [Sat Oct 1 21:03:18 2022] HEAD /identity/v3/domains/c2c3d42d4aa4486aa40ca857514b1d86/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:18.899343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fd0b0-831c-4d0b-8f56-5ca2b10c4f09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:18.919006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fd0b0-831c-4d0b-8f56-5ca2b10c4f09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:18.936953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992cbf6-3a69-4c66-8528-c5cc8bd0f500 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:18.937795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992cbf6-3a69-4c66-8528-c5cc8bd0f500 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:18.938121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992cbf6-3a69-4c66-8528-c5cc8bd0f500 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:18.938343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992cbf6-3a69-4c66-8528-c5cc8bd0f500 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/domains/c2c3d42d4aa4486aa40ca857514b1d86/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:18.951114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992cbf6-3a69-4c66-8528-c5cc8bd0f500 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=c2c3d42d4aa4486aa40ca857514b1d86, user_id=1ec430a8ba524da0af3eef8e1e8d9d33, role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:18.953850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992cbf6-3a69-4c66-8528-c5cc8bd0f500 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:18.980627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9992cbf6-3a69-4c66-8528-c5cc8bd0f500 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invalidating the token cache because role 4dccb2174d5342a2ae2e058c75f23e10 was removed from user 1ec430a8ba524da0af3eef8e1e8d9d33 on domain c2c3d42d4aa4486aa40ca857514b1d86. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:18.981556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9992cbf6-3a69-4c66-8528-c5cc8bd0f500 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:18.984085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9992cbf6-3a69-4c66-8528-c5cc8bd0f500 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:19.013751 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2216/4360] 10.222.0.17 () {58 vars in 1464 bytes} [Sat Oct 1 21:03:18 2022] DELETE /identity/v3/domains/c2c3d42d4aa4486aa40ca857514b1d86/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:19.021251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42e345-6da8-402d-a407-88e04de5d73a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:19.024229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9e42e345-6da8-402d-a407-88e04de5d73a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:19.047238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42e345-6da8-402d-a407-88e04de5d73a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:19.155372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7e9a9-3ff8-427a-84f7-b65c5e6efa9f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:19.156063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7e9a9-3ff8-427a-84f7-b65c5e6efa9f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:19.158154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7e9a9-3ff8-427a-84f7-b65c5e6efa9f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:19.159204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7e9a9-3ff8-427a-84f7-b65c5e6efa9f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:19.165294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7e9a9-3ff8-427a-84f7-b65c5e6efa9f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_grant(project_id=f69494f2b2fc40f29be4d38fe0a42b87, user_id=1ec430a8ba524da0af3eef8e1e8d9d33, role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:19.171871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7e9a9-3ff8-427a-84f7-b65c5e6efa9f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:19.210457 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2145/4361] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:19 2022] PUT /identity/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:19.211325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992cbf6-3a69-4c66-8528-c5cc8bd0f500 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:19.234239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992cbf6-3a69-4c66-8528-c5cc8bd0f500 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:19.379410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:19.447223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc53c8-1dc8-4afe-84b6-f8aa3cdf8d9f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:19.447908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc53c8-1dc8-4afe-84b6-f8aa3cdf8d9f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:19.448306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc53c8-1dc8-4afe-84b6-f8aa3cdf8d9f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:19.449090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc53c8-1dc8-4afe-84b6-f8aa3cdf8d9f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:19.456688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc53c8-1dc8-4afe-84b6-f8aa3cdf8d9f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:list_grants(project_id=f69494f2b2fc40f29be4d38fe0a42b87, user_id=1ec430a8ba524da0af3eef8e1e8d9d33)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:19.475531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc53c8-1dc8-4afe-84b6-f8aa3cdf8d9f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:19.521930 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2217/4362] 10.222.0.17 () {58 vars in 1365 bytes} [Sat Oct 1 21:03:19 2022] GET /identity/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles => generated 409 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:19.526305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7e9a9-3ff8-427a-84f7-b65c5e6efa9f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:19.551631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7e9a9-3ff8-427a-84f7-b65c5e6efa9f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:19.575884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4452f-81e7-43d6-baa4-cdc7b7b84abe tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:19.576573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4452f-81e7-43d6-baa4-cdc7b7b84abe tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:19.583933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4452f-81e7-43d6-baa4-cdc7b7b84abe tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:19.584650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4452f-81e7-43d6-baa4-cdc7b7b84abe tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:19.602966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4452f-81e7-43d6-baa4-cdc7b7b84abe tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:check_grant(project_id=f69494f2b2fc40f29be4d38fe0a42b87, user_id=1ec430a8ba524da0af3eef8e1e8d9d33, role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:19.622729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4452f-81e7-43d6-baa4-cdc7b7b84abe tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:19.653219 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2146/4363] 10.222.0.17 () {58 vars in 1465 bytes} [Sat Oct 1 21:03:19 2022] HEAD /identity/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:19.657237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc53c8-1dc8-4afe-84b6-f8aa3cdf8d9f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:19.675009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc53c8-1dc8-4afe-84b6-f8aa3cdf8d9f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:19.680538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711fbb6-e78e-466b-a72f-dd02e8ae79e4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:19.681457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711fbb6-e78e-466b-a72f-dd02e8ae79e4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:19.681932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711fbb6-e78e-466b-a72f-dd02e8ae79e4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:19.682520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711fbb6-e78e-466b-a72f-dd02e8ae79e4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:19.685679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711fbb6-e78e-466b-a72f-dd02e8ae79e4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=f69494f2b2fc40f29be4d38fe0a42b87, user_id=1ec430a8ba524da0af3eef8e1e8d9d33, role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:19.688659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711fbb6-e78e-466b-a72f-dd02e8ae79e4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:19.699689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7711fbb6-e78e-466b-a72f-dd02e8ae79e4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invalidating the token cache because role 4dccb2174d5342a2ae2e058c75f23e10 was removed from user 1ec430a8ba524da0af3eef8e1e8d9d33 on project f69494f2b2fc40f29be4d38fe0a42b87. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:19.699996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7711fbb6-e78e-466b-a72f-dd02e8ae79e4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:19.701654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7711fbb6-e78e-466b-a72f-dd02e8ae79e4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:19.729774 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2218/4364] 10.222.0.17 () {58 vars in 1467 bytes} [Sat Oct 1 21:03:19 2022] DELETE /identity/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:19.735036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4452f-81e7-43d6-baa4-cdc7b7b84abe tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:19.737220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b1b4452f-81e7-43d6-baa4-cdc7b7b84abe tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:19.770817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4452f-81e7-43d6-baa4-cdc7b7b84abe tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:19.921204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25479d-8b42-46e8-b466-fdd45c3c90c9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:19.922112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25479d-8b42-46e8-b466-fdd45c3c90c9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:19.922595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25479d-8b42-46e8-b466-fdd45c3c90c9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:19.922980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25479d-8b42-46e8-b466-fdd45c3c90c9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/system/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:19.924911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25479d-8b42-46e8-b466-fdd45c3c90c9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1ec430a8ba524da0af3eef8e1e8d9d33, role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:19.926139 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-db25479d-8b42-46e8-b466-fdd45c3c90c9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:19.926139 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:19.926139 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:19.928020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25479d-8b42-46e8-b466-fdd45c3c90c9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:19.949011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711fbb6-e78e-466b-a72f-dd02e8ae79e4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:19.959317 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2147/4365] 10.222.0.17 () {60 vars in 1378 bytes} [Sat Oct 1 21:03:19 2022] PUT /identity/v3/system/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:19.962843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711fbb6-e78e-466b-a72f-dd02e8ae79e4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:19.969150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5a0d6-6d04-44d3-a7f9-d7d66cf9390a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:19.970053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5a0d6-6d04-44d3-a7f9-d7d66cf9390a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:19.970410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5a0d6-6d04-44d3-a7f9-d7d66cf9390a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:19.970822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5a0d6-6d04-44d3-a7f9-d7d66cf9390a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/system/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:19.972481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5a0d6-6d04-44d3-a7f9-d7d66cf9390a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:list_system_grants_for_user(user_id=1ec430a8ba524da0af3eef8e1e8d9d33)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:19.974653 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-a9b5a0d6-6d04-44d3-a7f9-d7d66cf9390a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_system_grants_for_user": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:19.974653 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:19.974653 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:19.979947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5a0d6-6d04-44d3-a7f9-d7d66cf9390a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:20.002340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25479d-8b42-46e8-b466-fdd45c3c90c9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:20.003644 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2219/4366] 10.222.0.17 () {58 vars in 1260 bytes} [Sat Oct 1 21:03:19 2022] GET /identity/v3/system/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles => generated 374 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:20.014759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25479d-8b42-46e8-b466-fdd45c3c90c9 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:20.052778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a9729-aa64-4b48-8418-ae21d101f839 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:20.053566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a9729-aa64-4b48-8418-ae21d101f839 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:20.053968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a9729-aa64-4b48-8418-ae21d101f839 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:20.054346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a9729-aa64-4b48-8418-ae21d101f839 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/system/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:20.058545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a9729-aa64-4b48-8418-ae21d101f839 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:check_system_grant_for_user(user_id=1ec430a8ba524da0af3eef8e1e8d9d33, role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:20.059902 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-b57a9729-aa64-4b48-8418-ae21d101f839 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_system_grant_for_user": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:20.059902 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:20.059902 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:20.065120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a9729-aa64-4b48-8418-ae21d101f839 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:20.085247 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2148/4367] 10.222.0.17 () {58 vars in 1360 bytes} [Sat Oct 1 21:03:20 2022] HEAD /identity/v3/system/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:20.086272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5a0d6-6d04-44d3-a7f9-d7d66cf9390a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:20.111693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5a0d6-6d04-44d3-a7f9-d7d66cf9390a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:20.139071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165584a6-0d4b-48e9-b6e9-1913c0924ce2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:20.139758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165584a6-0d4b-48e9-b6e9-1913c0924ce2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:20.140578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165584a6-0d4b-48e9-b6e9-1913c0924ce2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:20.140578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165584a6-0d4b-48e9-b6e9-1913c0924ce2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/system/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:20.143670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165584a6-0d4b-48e9-b6e9-1913c0924ce2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:revoke_system_grant_for_user(user_id=1ec430a8ba524da0af3eef8e1e8d9d33, role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:20.144267 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-165584a6-0d4b-48e9-b6e9-1913c0924ce2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:20.144267 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:20.144267 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:20.146049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165584a6-0d4b-48e9-b6e9-1913c0924ce2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:20.179106 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2220/4368] 10.222.0.17 () {58 vars in 1362 bytes} [Sat Oct 1 21:03:20 2022] DELETE /identity/v3/system/users/1ec430a8ba524da0af3eef8e1e8d9d33/roles/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:20.184142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a9729-aa64-4b48-8418-ae21d101f839 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:20.210697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a9729-aa64-4b48-8418-ae21d101f839 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:20.216992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e7ab7-1e06-4218-96f2-c9552e5609fd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:20.217947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e7ab7-1e06-4218-96f2-c9552e5609fd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:20.218396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e7ab7-1e06-4218-96f2-c9552e5609fd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:20.218871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e7ab7-1e06-4218-96f2-c9552e5609fd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:20.219742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e7ab7-1e06-4218-96f2-c9552e5609fd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:20.221624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e7ab7-1e06-4218-96f2-c9552e5609fd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:20.253843 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2149/4369] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:03:20 2022] POST /identity/v3/roles => generated 269 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:20.258586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165584a6-0d4b-48e9-b6e9-1913c0924ce2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:20.274197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165584a6-0d4b-48e9-b6e9-1913c0924ce2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:20.285918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c2d9d8-8f73-44e6-910b-183325adf1c2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:20.286576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2d9d8-8f73-44e6-910b-183325adf1c2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:20.286921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2d9d8-8f73-44e6-910b-183325adf1c2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:20.287146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2d9d8-8f73-44e6-910b-183325adf1c2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:20.287954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c2d9d8-8f73-44e6-910b-183325adf1c2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:20.289018 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-e5c2d9d8-8f73-44e6-910b-183325adf1c2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:20.289018 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:20.289018 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:20.289921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c2d9d8-8f73-44e6-910b-183325adf1c2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:20.326987 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2221/4370] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:03:20 2022] POST /identity/v3/roles => generated 269 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:20.330822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e7ab7-1e06-4218-96f2-c9552e5609fd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:20.342944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e7ab7-1e06-4218-96f2-c9552e5609fd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:20.349176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8eee1-6e64-4f8d-974f-77396c15ee09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:20.350113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8eee1-6e64-4f8d-974f-77396c15ee09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:20.350610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8eee1-6e64-4f8d-974f-77396c15ee09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:20.352193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8eee1-6e64-4f8d-974f-77396c15ee09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/dcb71bef142a4f7095863b543d1e1da2/implies/901d724659b247259fc6ccbbd8481aee` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:20.354611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e8eee1-6e64-4f8d-974f-77396c15ee09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=dcb71bef142a4f7095863b543d1e1da2, implied_role_id=901d724659b247259fc6ccbbd8481aee)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:20.356381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e8eee1-6e64-4f8d-974f-77396c15ee09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:20.384232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:20.409381 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2150/4371] 10.222.0.17 () {60 vars in 1363 bytes} [Sat Oct 1 21:03:20 2022] PUT /identity/v3/roles/dcb71bef142a4f7095863b543d1e1da2/implies/901d724659b247259fc6ccbbd8481aee => generated 529 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:20.413512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c2d9d8-8f73-44e6-910b-183325adf1c2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:20.440278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c2d9d8-8f73-44e6-910b-183325adf1c2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:20.586101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317f33e-c5cb-4e5d-b8dd-c6a617038a47 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:20.587465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317f33e-c5cb-4e5d-b8dd-c6a617038a47 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:20.588001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317f33e-c5cb-4e5d-b8dd-c6a617038a47 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:20.588506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317f33e-c5cb-4e5d-b8dd-c6a617038a47 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:20.589523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317f33e-c5cb-4e5d-b8dd-c6a617038a47 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:20.591712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317f33e-c5cb-4e5d-b8dd-c6a617038a47 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:20.632403 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2222/4372] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:03:20 2022] POST /identity/v3/domains => generated 275 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:20.636207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8eee1-6e64-4f8d-974f-77396c15ee09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:20.650399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8eee1-6e64-4f8d-974f-77396c15ee09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:20.810203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a6c27-12e7-4222-b71e-92481b8cedfd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:20.812154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a6c27-12e7-4222-b71e-92481b8cedfd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:20.812645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a6c27-12e7-4222-b71e-92481b8cedfd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:20.813118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a6c27-12e7-4222-b71e-92481b8cedfd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:20.814164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a6c27-12e7-4222-b71e-92481b8cedfd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:20.816035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a6c27-12e7-4222-b71e-92481b8cedfd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:20.841383 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2151/4373] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:03:20 2022] POST /identity/v3/roles => generated 270 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:03:20.845895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317f33e-c5cb-4e5d-b8dd-c6a617038a47 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:20.866127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317f33e-c5cb-4e5d-b8dd-c6a617038a47 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:20.882621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef05b0-02c5-4ab1-9f62-e40c25786f75 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:20.885454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef05b0-02c5-4ab1-9f62-e40c25786f75 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:20.886651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef05b0-02c5-4ab1-9f62-e40c25786f75 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:20.888483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef05b0-02c5-4ab1-9f62-e40c25786f75 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/dcb71bef142a4f7095863b543d1e1da2/implies/cc8527f2938344b996be09677adc14f9` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:20.899379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef05b0-02c5-4ab1-9f62-e40c25786f75 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=dcb71bef142a4f7095863b543d1e1da2, implied_role_id=cc8527f2938344b996be09677adc14f9)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:20.906895 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-45ef05b0-02c5-4ab1-9f62-e40c25786f75 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:20.906895 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:20.906895 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:20.907865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef05b0-02c5-4ab1-9f62-e40c25786f75 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:20.998069 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2223/4374] 10.222.0.17 () {60 vars in 1363 bytes} [Sat Oct 1 21:03:20 2022] PUT /identity/v3/roles/dcb71bef142a4f7095863b543d1e1da2/implies/cc8527f2938344b996be09677adc14f9 => generated 530 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:03:21.002185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a6c27-12e7-4222-b71e-92481b8cedfd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:21.029514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a6c27-12e7-4222-b71e-92481b8cedfd tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:21.162976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c786610-0f34-4053-8aea-d6a06342c2cb tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:21.164422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c786610-0f34-4053-8aea-d6a06342c2cb tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:21.166227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c786610-0f34-4053-8aea-d6a06342c2cb tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:21.168902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c786610-0f34-4053-8aea-d6a06342c2cb tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/dcb71bef142a4f7095863b543d1e1da2/implies/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:21.174269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c786610-0f34-4053-8aea-d6a06342c2cb tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=dcb71bef142a4f7095863b543d1e1da2, implied_role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:21.176058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c786610-0f34-4053-8aea-d6a06342c2cb tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:21.268753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef05b0-02c5-4ab1-9f62-e40c25786f75 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:21.269022 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2152/4375] 10.222.0.17 () {60 vars in 1363 bytes} [Sat Oct 1 21:03:21 2022] PUT /identity/v3/roles/dcb71bef142a4f7095863b543d1e1da2/implies/4dccb2174d5342a2ae2e058c75f23e10 => generated 524 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:03:21.300030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef05b0-02c5-4ab1-9f62-e40c25786f75 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:21.437681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:21.494325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221b658-3a50-4388-ab59-dd7be0e31e15 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:21.495362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221b658-3a50-4388-ab59-dd7be0e31e15 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:21.495921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221b658-3a50-4388-ab59-dd7be0e31e15 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:21.496383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221b658-3a50-4388-ab59-dd7be0e31e15 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/4dccb2174d5342a2ae2e058c75f23e10/implies/dcb71bef142a4f7095863b543d1e1da2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:21.498612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221b658-3a50-4388-ab59-dd7be0e31e15 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=4dccb2174d5342a2ae2e058c75f23e10, implied_role_id=dcb71bef142a4f7095863b543d1e1da2)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:21.501004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221b658-3a50-4388-ab59-dd7be0e31e15 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:21.533620 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-5221b658-3a50-4388-ab59-dd7be0e31e15 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] dcb71bef142a4f7095863b543d1e1da2 cannot be an implied roles.: keystone.exception.InvalidImpliedRole: dcb71bef142a4f7095863b543d1e1da2 cannot be an implied roles. Oct 01 21:03:21.533620 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:03:21.533620 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:03:21.533620 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:03:21.533620 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:03:21.533620 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:03:21.533620 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:03:21.533620 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:03:21.533620 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:03:21.533620 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:03:21.533620 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:03:21.533620 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:03:21.533620 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 259, in put Oct 01 21:03:21.533620 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application PROVIDERS.role_api.create_implied_role(prior_role_id, implied_role_id) Oct 01 21:03:21.534910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:03:21.534910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:03:21.534910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1361, in create_implied_role Oct 01 21:03:21.534910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.InvalidImpliedRole(msg, Oct 01 21:03:21.534910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.InvalidImpliedRole: dcb71bef142a4f7095863b543d1e1da2 cannot be an implied roles. Oct 01 21:03:21.534910 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:03:21.537496 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2224/4376] 10.222.0.17 () {60 vars in 1363 bytes} [Sat Oct 1 21:03:21 2022] PUT /identity/v3/roles/4dccb2174d5342a2ae2e058c75f23e10/implies/dcb71bef142a4f7095863b543d1e1da2 => generated 116 bytes in 269 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:03:21.541802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c786610-0f34-4053-8aea-d6a06342c2cb tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:21.557624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c786610-0f34-4053-8aea-d6a06342c2cb tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:21.570373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94863f4d-9825-451a-add4-d98be2ba5767 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:21.571119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94863f4d-9825-451a-add4-d98be2ba5767 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:21.571513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94863f4d-9825-451a-add4-d98be2ba5767 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:21.571791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94863f4d-9825-451a-add4-d98be2ba5767 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/dcb71bef142a4f7095863b543d1e1da2/implies/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:21.576887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94863f4d-9825-451a-add4-d98be2ba5767 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=dcb71bef142a4f7095863b543d1e1da2, implied_role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:21.581548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94863f4d-9825-451a-add4-d98be2ba5767 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:21.609216 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2153/4377] 10.222.0.17 () {58 vars in 1347 bytes} [Sat Oct 1 21:03:21 2022] DELETE /identity/v3/roles/dcb71bef142a4f7095863b543d1e1da2/implies/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:21.610107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221b658-3a50-4388-ab59-dd7be0e31e15 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:21.633310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221b658-3a50-4388-ab59-dd7be0e31e15 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:21.893847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411319a-6126-4016-beb8-96014ca7290e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:21.907034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411319a-6126-4016-beb8-96014ca7290e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:21.907426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411319a-6126-4016-beb8-96014ca7290e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:21.907632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411319a-6126-4016-beb8-96014ca7290e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/dcb71bef142a4f7095863b543d1e1da2/implies/cc8527f2938344b996be09677adc14f9` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:21.909507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411319a-6126-4016-beb8-96014ca7290e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=dcb71bef142a4f7095863b543d1e1da2, implied_role_id=cc8527f2938344b996be09677adc14f9)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:21.911118 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-7411319a-6126-4016-beb8-96014ca7290e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:21.911118 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:21.911118 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:21.912043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411319a-6126-4016-beb8-96014ca7290e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:21.930022 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2225/4378] 10.222.0.17 () {58 vars in 1347 bytes} [Sat Oct 1 21:03:21 2022] DELETE /identity/v3/roles/dcb71bef142a4f7095863b543d1e1da2/implies/cc8527f2938344b996be09677adc14f9 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:21.934471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94863f4d-9825-451a-add4-d98be2ba5767 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:21.956016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94863f4d-9825-451a-add4-d98be2ba5767 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:22.095590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:22.115153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27ec39-20e1-40e7-bf36-062897a5a60b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:22.116934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27ec39-20e1-40e7-bf36-062897a5a60b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:22.118854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27ec39-20e1-40e7-bf36-062897a5a60b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:22.119798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27ec39-20e1-40e7-bf36-062897a5a60b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/cc8527f2938344b996be09677adc14f9` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:22.122855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27ec39-20e1-40e7-bf36-062897a5a60b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=cc8527f2938344b996be09677adc14f9)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:22.127238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27ec39-20e1-40e7-bf36-062897a5a60b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:22.227010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7f27ec39-20e1-40e7-bf36-062897a5a60b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invalidating the token cache because role cc8527f2938344b996be09677adc14f9 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:22.227597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7f27ec39-20e1-40e7-bf36-062897a5a60b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:22.228042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7f27ec39-20e1-40e7-bf36-062897a5a60b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:22.230180 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2154/4379] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:21 2022] DELETE /identity/v3/roles/cc8527f2938344b996be09677adc14f9 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:22.234643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411319a-6126-4016-beb8-96014ca7290e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:22.237713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7411319a-6126-4016-beb8-96014ca7290e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:22.279781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411319a-6126-4016-beb8-96014ca7290e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:22.427356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b32a21-66f1-4600-85ee-050faef5116b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:22.428020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b32a21-66f1-4600-85ee-050faef5116b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:22.428306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b32a21-66f1-4600-85ee-050faef5116b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:22.428585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b32a21-66f1-4600-85ee-050faef5116b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/domains/4bc520374aba4df2a4e0d29ed4a8deea` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:22.467014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b32a21-66f1-4600-85ee-050faef5116b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=4bc520374aba4df2a4e0d29ed4a8deea)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:22.468066 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-05b32a21-66f1-4600-85ee-050faef5116b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:22.468066 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:22.468066 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:22.468980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b32a21-66f1-4600-85ee-050faef5116b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:22.531862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05b32a21-66f1-4600-85ee-050faef5116b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '4bc520374aba4df2a4e0d29ed4a8deea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:22.534939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05b32a21-66f1-4600-85ee-050faef5116b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '4bc520374aba4df2a4e0d29ed4a8deea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:22.606161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05b32a21-66f1-4600-85ee-050faef5116b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '4bc520374aba4df2a4e0d29ed4a8deea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:22.607016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05b32a21-66f1-4600-85ee-050faef5116b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '4bc520374aba4df2a4e0d29ed4a8deea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:22.610614 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2226/4380] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:03:22 2022] PATCH /identity/v3/domains/4bc520374aba4df2a4e0d29ed4a8deea => generated 276 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:22.614546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27ec39-20e1-40e7-bf36-062897a5a60b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:22.631208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27ec39-20e1-40e7-bf36-062897a5a60b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:22.803123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:22.804306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:22.804813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:22.805315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/domains/4bc520374aba4df2a4e0d29ed4a8deea` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:22.838758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=4bc520374aba4df2a4e0d29ed4a8deea)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:22.840397 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:22.840397 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:22.840397 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:22.841772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:22.911154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '4bc520374aba4df2a4e0d29ed4a8deea'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:22.974588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '4bc520374aba4df2a4e0d29ed4a8deea'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:23.133206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bc520374aba4df2a4e0d29ed4a8deea', 'request_id': 'req-32512454-0611-4e08-a172-b6a3afe4cec1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:23.162731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bc520374aba4df2a4e0d29ed4a8deea', 'request_id': 'req-32512454-0611-4e08-a172-b6a3afe4cec1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:23.186232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bc520374aba4df2a4e0d29ed4a8deea', 'request_id': 'req-32512454-0611-4e08-a172-b6a3afe4cec1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:23.223404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] The token cache is being invalidate because project 4bc520374aba4df2a4e0d29ed4a8deea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:23.223731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:23.226901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:23.264463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '4bc520374aba4df2a4e0d29ed4a8deea', 'request_id': 'req-32512454-0611-4e08-a172-b6a3afe4cec1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:23.265434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '4bc520374aba4df2a4e0d29ed4a8deea', 'request_id': 'req-32512454-0611-4e08-a172-b6a3afe4cec1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:23.272833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '4bc520374aba4df2a4e0d29ed4a8deea', 'request_id': 'req-32512454-0611-4e08-a172-b6a3afe4cec1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:23.275988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:23.310778 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2155/4381] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:03:22 2022] DELETE /identity/v3/domains/4bc520374aba4df2a4e0d29ed4a8deea => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:23.315365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b32a21-66f1-4600-85ee-050faef5116b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:23.317501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-05b32a21-66f1-4600-85ee-050faef5116b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:23.344581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b32a21-66f1-4600-85ee-050faef5116b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:23.401494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:UPDATE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:23.493533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d756fd-9a68-4839-897f-8729ff2e9c10 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:23.496728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d756fd-9a68-4839-897f-8729ff2e9c10 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:23.497441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d756fd-9a68-4839-897f-8729ff2e9c10 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:23.498337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d756fd-9a68-4839-897f-8729ff2e9c10 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/dcb71bef142a4f7095863b543d1e1da2/implies/901d724659b247259fc6ccbbd8481aee` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:23.502533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d756fd-9a68-4839-897f-8729ff2e9c10 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=dcb71bef142a4f7095863b543d1e1da2, implied_role_id=901d724659b247259fc6ccbbd8481aee)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:23.505672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d756fd-9a68-4839-897f-8729ff2e9c10 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:23.528281 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2227/4382] 10.222.0.17 () {58 vars in 1347 bytes} [Sat Oct 1 21:03:23 2022] DELETE /identity/v3/roles/dcb71bef142a4f7095863b543d1e1da2/implies/901d724659b247259fc6ccbbd8481aee => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:23.532890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:23.534957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:23.563431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32512454-0611-4e08-a172-b6a3afe4cec1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:23.725189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d0883-37eb-46d1-b499-e697f3798b29 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:23.726002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d0883-37eb-46d1-b499-e697f3798b29 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:23.726398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d0883-37eb-46d1-b499-e697f3798b29 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:23.726794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d0883-37eb-46d1-b499-e697f3798b29 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/901d724659b247259fc6ccbbd8481aee` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:23.728042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d0883-37eb-46d1-b499-e697f3798b29 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=901d724659b247259fc6ccbbd8481aee)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:23.729744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d0883-37eb-46d1-b499-e697f3798b29 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:23.835680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ae3d0883-37eb-46d1-b499-e697f3798b29 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invalidating the token cache because role 901d724659b247259fc6ccbbd8481aee has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:23.835981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ae3d0883-37eb-46d1-b499-e697f3798b29 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:23.840888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ae3d0883-37eb-46d1-b499-e697f3798b29 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:23.851612 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2156/4383] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:23 2022] DELETE /identity/v3/roles/901d724659b247259fc6ccbbd8481aee => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:23.853226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d756fd-9a68-4839-897f-8729ff2e9c10 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:23.855538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a6d756fd-9a68-4839-897f-8729ff2e9c10 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:23.884147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d756fd-9a68-4839-897f-8729ff2e9c10 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:24.047689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240f8ef-6b31-4eca-b256-d48a3aae2a80 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:24.048619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240f8ef-6b31-4eca-b256-d48a3aae2a80 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:24.048946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240f8ef-6b31-4eca-b256-d48a3aae2a80 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:24.049426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240f8ef-6b31-4eca-b256-d48a3aae2a80 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/dcb71bef142a4f7095863b543d1e1da2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:24.052124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240f8ef-6b31-4eca-b256-d48a3aae2a80 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=dcb71bef142a4f7095863b543d1e1da2)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:24.053254 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-c240f8ef-6b31-4eca-b256-d48a3aae2a80 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:24.053254 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:24.053254 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:24.055260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240f8ef-6b31-4eca-b256-d48a3aae2a80 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:24.167522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c240f8ef-6b31-4eca-b256-d48a3aae2a80 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invalidating the token cache because role dcb71bef142a4f7095863b543d1e1da2 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:24.168832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c240f8ef-6b31-4eca-b256-d48a3aae2a80 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:24.174015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c240f8ef-6b31-4eca-b256-d48a3aae2a80 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:24.190378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d0883-37eb-46d1-b499-e697f3798b29 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:24.200218 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2228/4384] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:23 2022] DELETE /identity/v3/roles/dcb71bef142a4f7095863b543d1e1da2 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:24.206999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d0883-37eb-46d1-b499-e697f3798b29 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:24.433628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797925e7-5dc7-44bf-ae0b-ea65ae2e8a2e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:24.434588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797925e7-5dc7-44bf-ae0b-ea65ae2e8a2e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:24.434885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797925e7-5dc7-44bf-ae0b-ea65ae2e8a2e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:24.435164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797925e7-5dc7-44bf-ae0b-ea65ae2e8a2e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:24.441705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797925e7-5dc7-44bf-ae0b-ea65ae2e8a2e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=b4075ffb7dc54af7bc5c170586efbec1, implied_role_id=a961c41374514b48ad472fddaade0d27)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:24.443415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797925e7-5dc7-44bf-ae0b-ea65ae2e8a2e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:24.499656 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2157/4385] 10.222.0.17 () {60 vars in 1363 bytes} [Sat Oct 1 21:03:24 2022] PUT /identity/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27 => generated 521 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:24.500075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240f8ef-6b31-4eca-b256-d48a3aae2a80 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:24.501552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c240f8ef-6b31-4eca-b256-d48a3aae2a80 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:24.521994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240f8ef-6b31-4eca-b256-d48a3aae2a80 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:24.667290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4a3a1-4ece-4d0a-a020-4a3334c2cf09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:24.668055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4a3a1-4ece-4d0a-a020-4a3334c2cf09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:24.668343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4a3a1-4ece-4d0a-a020-4a3334c2cf09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:24.669136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4a3a1-4ece-4d0a-a020-4a3334c2cf09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:24.670386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4a3a1-4ece-4d0a-a020-4a3334c2cf09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:check_implied_role(prior_role_id=b4075ffb7dc54af7bc5c170586efbec1, implied_role_id=a961c41374514b48ad472fddaade0d27)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:24.671501 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-f1c4a3a1-4ece-4d0a-a020-4a3334c2cf09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:24.671501 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:24.671501 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:24.673098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4a3a1-4ece-4d0a-a020-4a3334c2cf09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:24.674031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4a3a1-4ece-4d0a-a020-4a3334c2cf09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=b4075ffb7dc54af7bc5c170586efbec1, implied_role_id=a961c41374514b48ad472fddaade0d27)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:24.674944 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-f1c4a3a1-4ece-4d0a-a020-4a3334c2cf09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:24.674944 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:24.674944 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:24.675774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4a3a1-4ece-4d0a-a020-4a3334c2cf09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:24.692839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797925e7-5dc7-44bf-ae0b-ea65ae2e8a2e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:24.693710 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2229/4386] 10.222.0.17 () {58 vars in 1345 bytes} [Sat Oct 1 21:03:24 2022] HEAD /identity/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:24.706112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797925e7-5dc7-44bf-ae0b-ea65ae2e8a2e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:24.711971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da333de2-10c2-422a-ba0b-a6159fa1356d tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:24.712635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da333de2-10c2-422a-ba0b-a6159fa1356d tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:24.712917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da333de2-10c2-422a-ba0b-a6159fa1356d tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:24.713194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da333de2-10c2-422a-ba0b-a6159fa1356d tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:24.715261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da333de2-10c2-422a-ba0b-a6159fa1356d tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=b4075ffb7dc54af7bc5c170586efbec1, implied_role_id=a961c41374514b48ad472fddaade0d27)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:24.716517 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-da333de2-10c2-422a-ba0b-a6159fa1356d tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:24.716517 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:24.716517 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:24.717487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da333de2-10c2-422a-ba0b-a6159fa1356d tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:24.742952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4a3a1-4ece-4d0a-a020-4a3334c2cf09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:24.755948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4a3a1-4ece-4d0a-a020-4a3334c2cf09 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:24.756826 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2158/4387] 10.222.0.17 () {58 vars in 1344 bytes} [Sat Oct 1 21:03:24 2022] GET /identity/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27 => generated 521 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:24.761550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323f852-74a9-4762-acde-d1aa5a7b31f5 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:24.762343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323f852-74a9-4762-acde-d1aa5a7b31f5 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:24.762768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323f852-74a9-4762-acde-d1aa5a7b31f5 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:24.763045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323f852-74a9-4762-acde-d1aa5a7b31f5 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:24.764828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323f852-74a9-4762-acde-d1aa5a7b31f5 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=b4075ffb7dc54af7bc5c170586efbec1, implied_role_id=a961c41374514b48ad472fddaade0d27)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:24.766694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323f852-74a9-4762-acde-d1aa5a7b31f5 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:24.780887 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2230/4388] 10.222.0.17 () {58 vars in 1347 bytes} [Sat Oct 1 21:03:24 2022] DELETE /identity/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:24.785264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da333de2-10c2-422a-ba0b-a6159fa1356d tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:24.805698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da333de2-10c2-422a-ba0b-a6159fa1356d tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:24.880314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:24.935877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b5ffd-bd8d-4703-b2d8-7558ea08e611 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:24.936621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b5ffd-bd8d-4703-b2d8-7558ea08e611 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:24.936916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b5ffd-bd8d-4703-b2d8-7558ea08e611 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:24.937198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b5ffd-bd8d-4703-b2d8-7558ea08e611 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:24.939917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b5ffd-bd8d-4703-b2d8-7558ea08e611 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=b4075ffb7dc54af7bc5c170586efbec1, implied_role_id=a961c41374514b48ad472fddaade0d27)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:24.941538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b5ffd-bd8d-4703-b2d8-7558ea08e611 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:24.952493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-580b5ffd-bd8d-4703-b2d8-7558ea08e611 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] b4075ffb7dc54af7bc5c170586efbec1 does not imply a961c41374514b48ad472fddaade0d27.: keystone.exception.ImpliedRoleNotFound: b4075ffb7dc54af7bc5c170586efbec1 does not imply a961c41374514b48ad472fddaade0d27. Oct 01 21:03:24.952493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:03:24.952493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Oct 01 21:03:24.952493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = query.one() Oct 01 21:03:24.952493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Oct 01 21:03:24.952493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._iter().one() Oct 01 21:03:24.952493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Oct 01 21:03:24.952493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._only_one_row( Oct 01 21:03:24.952493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Oct 01 21:03:24.952493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exc.NoResultFound( Oct 01 21:03:24.952493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Oct 01 21:03:24.952493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:03:24.952493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:03:24.952493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:03:24.952493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:03:24.952493 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:03:24.953727 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:03:24.953727 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:03:24.953727 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:03:24.953727 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:03:24.953727 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:03:24.953727 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:03:24.953727 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:03:24.953727 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:03:24.953727 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:03:24.953727 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 235, in get Oct 01 21:03:24.953727 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_implied_role(prior_role_id, implied_role_id) Oct 01 21:03:24.953727 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 239, in _get_implied_role Oct 01 21:03:24.953727 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application PROVIDERS.role_api.get_implied_role( Oct 01 21:03:24.953727 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 149, in get_implied_role Oct 01 21:03:24.953727 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Oct 01 21:03:24.953727 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Oct 01 21:03:24.955282 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Oct 01 21:03:24.955282 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: b4075ffb7dc54af7bc5c170586efbec1 does not imply a961c41374514b48ad472fddaade0d27. Oct 01 21:03:24.955282 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:03:24.955282 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2159/4389] 10.222.0.17 () {58 vars in 1344 bytes} [Sat Oct 1 21:03:24 2022] GET /identity/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27 => generated 137 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Oct 01 21:03:24.958431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323f852-74a9-4762-acde-d1aa5a7b31f5 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:24.978804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323f852-74a9-4762-acde-d1aa5a7b31f5 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:24.990098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2db5a-ef84-4a2f-a44e-cd8144fe4d20 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:24.990792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2db5a-ef84-4a2f-a44e-cd8144fe4d20 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:24.991073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2db5a-ef84-4a2f-a44e-cd8144fe4d20 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:24.991348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2db5a-ef84-4a2f-a44e-cd8144fe4d20 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:24.993052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca2db5a-ef84-4a2f-a44e-cd8144fe4d20 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=b4075ffb7dc54af7bc5c170586efbec1, implied_role_id=a961c41374514b48ad472fddaade0d27)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:24.994682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca2db5a-ef84-4a2f-a44e-cd8144fe4d20 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:25.013136 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-2ca2db5a-ef84-4a2f-a44e-cd8144fe4d20 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] b4075ffb7dc54af7bc5c170586efbec1 does not imply a961c41374514b48ad472fddaade0d27.: keystone.exception.ImpliedRoleNotFound: b4075ffb7dc54af7bc5c170586efbec1 does not imply a961c41374514b48ad472fddaade0d27. Oct 01 21:03:25.013136 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:03:25.013136 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Oct 01 21:03:25.013136 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = query.one() Oct 01 21:03:25.013136 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Oct 01 21:03:25.013136 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._iter().one() Oct 01 21:03:25.013136 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Oct 01 21:03:25.013136 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._only_one_row( Oct 01 21:03:25.013136 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Oct 01 21:03:25.013136 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exc.NoResultFound( Oct 01 21:03:25.013136 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Oct 01 21:03:25.013136 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:03:25.013136 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:03:25.013136 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:03:25.013136 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:03:25.013136 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:03:25.014443 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:03:25.014443 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:03:25.014443 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:03:25.014443 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:03:25.014443 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:03:25.014443 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:03:25.014443 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:03:25.014443 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:03:25.014443 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:03:25.014443 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 270, in delete Oct 01 21:03:25.014443 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application PROVIDERS.role_api.delete_implied_role(prior_role_id, implied_role_id) Oct 01 21:03:25.014443 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:03:25.014443 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:03:25.014443 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1369, in delete_implied_role Oct 01 21:03:25.014443 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.driver.delete_implied_role(prior_role_id, implied_role_id) Oct 01 21:03:25.014443 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 129, in delete_implied_role Oct 01 21:03:25.015765 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Oct 01 21:03:25.015765 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Oct 01 21:03:25.015765 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Oct 01 21:03:25.015765 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: b4075ffb7dc54af7bc5c170586efbec1 does not imply a961c41374514b48ad472fddaade0d27. Oct 01 21:03:25.015765 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:03:25.018613 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2231/4390] 10.222.0.17 () {58 vars in 1347 bytes} [Sat Oct 1 21:03:24 2022] DELETE /identity/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27 => generated 137 bytes in 61 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:03:25.019429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b5ffd-bd8d-4703-b2d8-7558ea08e611 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:25.035886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b5ffd-bd8d-4703-b2d8-7558ea08e611 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:25.051665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401e45f-9f55-44f3-acb9-cc8de7c1c72f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:25.052472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401e45f-9f55-44f3-acb9-cc8de7c1c72f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:25.052878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401e45f-9f55-44f3-acb9-cc8de7c1c72f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:25.053781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401e45f-9f55-44f3-acb9-cc8de7c1c72f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:25.055538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401e45f-9f55-44f3-acb9-cc8de7c1c72f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=b4075ffb7dc54af7bc5c170586efbec1, implied_role_id=a961c41374514b48ad472fddaade0d27)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:25.057677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401e45f-9f55-44f3-acb9-cc8de7c1c72f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:25.135898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2db5a-ef84-4a2f-a44e-cd8144fe4d20 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:25.143908 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2160/4391] 10.222.0.17 () {60 vars in 1363 bytes} [Sat Oct 1 21:03:25 2022] PUT /identity/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27 => generated 521 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:25.144532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:25.208756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2db5a-ef84-4a2f-a44e-cd8144fe4d20 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:25.351263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5461eb-4b50-4539-8a26-191cb8525337 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:25.351994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5461eb-4b50-4539-8a26-191cb8525337 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:25.352288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5461eb-4b50-4539-8a26-191cb8525337 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:25.352567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5461eb-4b50-4539-8a26-191cb8525337 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/4f8031b801684cb4bba94222618410d9` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:25.357200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5461eb-4b50-4539-8a26-191cb8525337 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=b4075ffb7dc54af7bc5c170586efbec1, implied_role_id=4f8031b801684cb4bba94222618410d9)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:25.359035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5461eb-4b50-4539-8a26-191cb8525337 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:25.419471 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2232/4392] 10.222.0.17 () {60 vars in 1363 bytes} [Sat Oct 1 21:03:25 2022] PUT /identity/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/4f8031b801684cb4bba94222618410d9 => generated 520 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:25.423356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401e45f-9f55-44f3-acb9-cc8de7c1c72f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:25.476148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401e45f-9f55-44f3-acb9-cc8de7c1c72f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:25.585662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ad150-2d0b-4af4-8f92-1f50ac015845 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:25.586396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ad150-2d0b-4af4-8f92-1f50ac015845 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:25.586709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ad150-2d0b-4af4-8f92-1f50ac015845 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:25.586989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ad150-2d0b-4af4-8f92-1f50ac015845 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/4f8031b801684cb4bba94222618410d9/implies/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:25.588715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ad150-2d0b-4af4-8f92-1f50ac015845 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=4f8031b801684cb4bba94222618410d9, implied_role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:25.592850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ad150-2d0b-4af4-8f92-1f50ac015845 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:25.637017 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2161/4393] 10.222.0.17 () {60 vars in 1363 bytes} [Sat Oct 1 21:03:25 2022] PUT /identity/v3/roles/4f8031b801684cb4bba94222618410d9/implies/4dccb2174d5342a2ae2e058c75f23e10 => generated 519 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:25.640973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5461eb-4b50-4539-8a26-191cb8525337 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:25.656194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5461eb-4b50-4539-8a26-191cb8525337 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:25.763200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3f397-9780-42c8-88c9-1f4739f8f240 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:25.763937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3f397-9780-42c8-88c9-1f4739f8f240 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:25.764233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3f397-9780-42c8-88c9-1f4739f8f240 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:25.764528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3f397-9780-42c8-88c9-1f4739f8f240 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/role_inferences` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:25.771757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3f397-9780-42c8-88c9-1f4739f8f240 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:list_role_inference_rules()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:25.773249 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-b6c3f397-9780-42c8-88c9-1f4739f8f240 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_inference_rules": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:25.773249 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:25.773249 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:25.775557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3f397-9780-42c8-88c9-1f4739f8f240 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:25.842541 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2233/4394] 10.222.0.17 () {58 vars in 1152 bytes} [Sat Oct 1 21:03:25 2022] GET /identity/v3/role_inferences => generated 1605 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:25.844374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ad150-2d0b-4af4-8f92-1f50ac015845 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:25.863111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ad150-2d0b-4af4-8f92-1f50ac015845 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:25.876727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a76495-9453-4a2c-a7be-801fc2963869 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:25.877477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a76495-9453-4a2c-a7be-801fc2963869 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:25.878078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a76495-9453-4a2c-a7be-801fc2963869 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:25.878743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a76495-9453-4a2c-a7be-801fc2963869 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/4f8031b801684cb4bba94222618410d9/implies/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:25.882697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a76495-9453-4a2c-a7be-801fc2963869 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=4f8031b801684cb4bba94222618410d9, implied_role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:25.887106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a76495-9453-4a2c-a7be-801fc2963869 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:25.908228 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2162/4395] 10.222.0.17 () {58 vars in 1347 bytes} [Sat Oct 1 21:03:25 2022] DELETE /identity/v3/roles/4f8031b801684cb4bba94222618410d9/implies/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:25.912600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3f397-9780-42c8-88c9-1f4739f8f240 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:25.929010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3f397-9780-42c8-88c9-1f4739f8f240 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:26.172841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f5435-6b2e-4837-b571-a347e7be42f2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:26.173829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f5435-6b2e-4837-b571-a347e7be42f2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:26.174187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f5435-6b2e-4837-b571-a347e7be42f2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:26.174656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f5435-6b2e-4837-b571-a347e7be42f2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/4f8031b801684cb4bba94222618410d9` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:26.176840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f5435-6b2e-4837-b571-a347e7be42f2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=b4075ffb7dc54af7bc5c170586efbec1, implied_role_id=4f8031b801684cb4bba94222618410d9)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:26.179763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f5435-6b2e-4837-b571-a347e7be42f2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:26.194779 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2234/4396] 10.222.0.17 () {58 vars in 1347 bytes} [Sat Oct 1 21:03:25 2022] DELETE /identity/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/4f8031b801684cb4bba94222618410d9 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:26.196751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a76495-9453-4a2c-a7be-801fc2963869 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:26.213759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a76495-9453-4a2c-a7be-801fc2963869 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:26.300020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50f3d6-35b0-480f-acf3-72c1609bb560 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:26.300977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50f3d6-35b0-480f-acf3-72c1609bb560 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:26.301572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50f3d6-35b0-480f-acf3-72c1609bb560 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:26.302090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50f3d6-35b0-480f-acf3-72c1609bb560 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:26.304654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50f3d6-35b0-480f-acf3-72c1609bb560 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=b4075ffb7dc54af7bc5c170586efbec1, implied_role_id=a961c41374514b48ad472fddaade0d27)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:26.307426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50f3d6-35b0-480f-acf3-72c1609bb560 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:26.320748 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2163/4397] 10.222.0.17 () {58 vars in 1347 bytes} [Sat Oct 1 21:03:26 2022] DELETE /identity/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:26.327065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f5435-6b2e-4837-b571-a347e7be42f2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:26.349523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f5435-6b2e-4837-b571-a347e7be42f2 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:26.371105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:26.471477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41281294-aee0-4fc6-82d7-4a2c992cdbf0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:26.472200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41281294-aee0-4fc6-82d7-4a2c992cdbf0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:26.472548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41281294-aee0-4fc6-82d7-4a2c992cdbf0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:26.473892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41281294-aee0-4fc6-82d7-4a2c992cdbf0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:26.474735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41281294-aee0-4fc6-82d7-4a2c992cdbf0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:26.476666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41281294-aee0-4fc6-82d7-4a2c992cdbf0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:26.490420 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2235/4398] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:26 2022] GET /identity/v3/roles => generated 2322 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:26.495586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50f3d6-35b0-480f-acf3-72c1609bb560 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:26.520702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50f3d6-35b0-480f-acf3-72c1609bb560 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:26.529745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e906e-f87c-4b93-ae83-12d2436a4069 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:26.531324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e906e-f87c-4b93-ae83-12d2436a4069 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:26.531854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e906e-f87c-4b93-ae83-12d2436a4069 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:26.532213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e906e-f87c-4b93-ae83-12d2436a4069 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:26.532977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e906e-f87c-4b93-ae83-12d2436a4069 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:26.534954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e906e-f87c-4b93-ae83-12d2436a4069 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:26.554589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:26.575933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41281294-aee0-4fc6-82d7-4a2c992cdbf0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:26.579502 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2164/4399] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:03:26 2022] POST /identity/v3/roles => generated 234 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:26.591992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41281294-aee0-4fc6-82d7-4a2c992cdbf0 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:26.598074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15d07b-fa20-44c3-9c09-6d6a5a2da456 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:26.599077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15d07b-fa20-44c3-9c09-6d6a5a2da456 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:26.599366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15d07b-fa20-44c3-9c09-6d6a5a2da456 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:26.599638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15d07b-fa20-44c3-9c09-6d6a5a2da456 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/166d3a1d52f644b590467849cd44b857` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:26.601266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f15d07b-fa20-44c3-9c09-6d6a5a2da456 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:update_role(role_id=166d3a1d52f644b590467849cd44b857)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:26.602301 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-2f15d07b-fa20-44c3-9c09-6d6a5a2da456 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:26.602301 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:26.602301 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:26.603238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f15d07b-fa20-44c3-9c09-6d6a5a2da456 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:26.655117 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2236/4400] 10.222.0.17 () {60 vars in 1243 bytes} [Sat Oct 1 21:03:26 2022] PATCH /identity/v3/roles/166d3a1d52f644b590467849cd44b857 => generated 238 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:26.655892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e906e-f87c-4b93-ae83-12d2436a4069 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:26.667942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e906e-f87c-4b93-ae83-12d2436a4069 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:26.674880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b7b4a-418e-46f0-950d-116e613c3d86 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:26.675580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b7b4a-418e-46f0-950d-116e613c3d86 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:26.676003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b7b4a-418e-46f0-950d-116e613c3d86 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:26.676657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b7b4a-418e-46f0-950d-116e613c3d86 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/166d3a1d52f644b590467849cd44b857` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:26.689370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b7b4a-418e-46f0-950d-116e613c3d86 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:get_role(role_id=166d3a1d52f644b590467849cd44b857)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:26.692095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b7b4a-418e-46f0-950d-116e613c3d86 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:26.697824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15d07b-fa20-44c3-9c09-6d6a5a2da456 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:26.710771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15d07b-fa20-44c3-9c09-6d6a5a2da456 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:26.711180 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2165/4401] 10.222.0.17 () {58 vars in 1221 bytes} [Sat Oct 1 21:03:26 2022] GET /identity/v3/roles/166d3a1d52f644b590467849cd44b857 => generated 238 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:03:26.716600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec8bc5-36d3-49c3-a164-2ac611cf157b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:26.717447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec8bc5-36d3-49c3-a164-2ac611cf157b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:26.717734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec8bc5-36d3-49c3-a164-2ac611cf157b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:26.718282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec8bc5-36d3-49c3-a164-2ac611cf157b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:26.718981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec8bc5-36d3-49c3-a164-2ac611cf157b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:26.720601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec8bc5-36d3-49c3-a164-2ac611cf157b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:26.740202 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2237/4402] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:26 2022] GET /identity/v3/roles => generated 2551 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:26.744615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b7b4a-418e-46f0-950d-116e613c3d86 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:26.759949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b7b4a-418e-46f0-950d-116e613c3d86 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:26.769035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519336c9-f92b-49ac-99b9-78a37ebb484e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:26.772281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519336c9-f92b-49ac-99b9-78a37ebb484e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:26.772281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519336c9-f92b-49ac-99b9-78a37ebb484e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:26.772281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519336c9-f92b-49ac-99b9-78a37ebb484e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/166d3a1d52f644b590467849cd44b857` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:26.773241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519336c9-f92b-49ac-99b9-78a37ebb484e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_role(role_id=166d3a1d52f644b590467849cd44b857)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:26.777238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519336c9-f92b-49ac-99b9-78a37ebb484e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:26.862554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-519336c9-f92b-49ac-99b9-78a37ebb484e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invalidating the token cache because role 166d3a1d52f644b590467849cd44b857 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:26.863038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-519336c9-f92b-49ac-99b9-78a37ebb484e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:26.864383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-519336c9-f92b-49ac-99b9-78a37ebb484e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:26.873223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec8bc5-36d3-49c3-a164-2ac611cf157b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:26.874294 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2166/4403] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:26 2022] DELETE /identity/v3/roles/166d3a1d52f644b590467849cd44b857 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:26.875356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0dec8bc5-36d3-49c3-a164-2ac611cf157b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:26.893521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec8bc5-36d3-49c3-a164-2ac611cf157b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:26.998324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fb0fd-db33-4ff6-bf24-b84d58a565bf tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:26.999560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fb0fd-db33-4ff6-bf24-b84d58a565bf tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:27.000296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fb0fd-db33-4ff6-bf24-b84d58a565bf tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:27.000794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fb0fd-db33-4ff6-bf24-b84d58a565bf tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:27.002725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fb0fd-db33-4ff6-bf24-b84d58a565bf tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=b4075ffb7dc54af7bc5c170586efbec1, implied_role_id=a961c41374514b48ad472fddaade0d27)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:27.005246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fb0fd-db33-4ff6-bf24-b84d58a565bf tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:27.068759 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2238/4404] 10.222.0.17 () {60 vars in 1363 bytes} [Sat Oct 1 21:03:26 2022] PUT /identity/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27 => generated 521 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:27.070204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519336c9-f92b-49ac-99b9-78a37ebb484e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:27.098534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519336c9-f92b-49ac-99b9-78a37ebb484e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:27.192991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b40ff-0932-4174-a61d-f0f03e1b7b78 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:27.194044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b40ff-0932-4174-a61d-f0f03e1b7b78 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:27.194517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b40ff-0932-4174-a61d-f0f03e1b7b78 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:27.194983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b40ff-0932-4174-a61d-f0f03e1b7b78 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/4f8031b801684cb4bba94222618410d9` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:27.196667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b40ff-0932-4174-a61d-f0f03e1b7b78 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=b4075ffb7dc54af7bc5c170586efbec1, implied_role_id=4f8031b801684cb4bba94222618410d9)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:27.198281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b40ff-0932-4174-a61d-f0f03e1b7b78 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:27.239110 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2167/4405] 10.222.0.17 () {60 vars in 1363 bytes} [Sat Oct 1 21:03:27 2022] PUT /identity/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/4f8031b801684cb4bba94222618410d9 => generated 520 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:27.240273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fb0fd-db33-4ff6-bf24-b84d58a565bf tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:27.257331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fb0fd-db33-4ff6-bf24-b84d58a565bf tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:27.381866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:27.404403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d90bcb-00fe-46e6-a37b-32327c757544 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:27.405069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d90bcb-00fe-46e6-a37b-32327c757544 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:27.405351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d90bcb-00fe-46e6-a37b-32327c757544 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:27.405644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d90bcb-00fe-46e6-a37b-32327c757544 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/4f8031b801684cb4bba94222618410d9/implies/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:27.407318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d90bcb-00fe-46e6-a37b-32327c757544 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=4f8031b801684cb4bba94222618410d9, implied_role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:27.410875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d90bcb-00fe-46e6-a37b-32327c757544 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:27.475523 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2239/4406] 10.222.0.17 () {60 vars in 1363 bytes} [Sat Oct 1 21:03:27 2022] PUT /identity/v3/roles/4f8031b801684cb4bba94222618410d9/implies/4dccb2174d5342a2ae2e058c75f23e10 => generated 519 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:27.479573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b40ff-0932-4174-a61d-f0f03e1b7b78 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:27.522092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b40ff-0932-4174-a61d-f0f03e1b7b78 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:27.701600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4d0c5-d9f6-495d-b9bb-3ee867507f05 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:27.702796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4d0c5-d9f6-495d-b9bb-3ee867507f05 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:27.703325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4d0c5-d9f6-495d-b9bb-3ee867507f05 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:27.704808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4d0c5-d9f6-495d-b9bb-3ee867507f05 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/4f8031b801684cb4bba94222618410d9/implies` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:27.706815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4d0c5-d9f6-495d-b9bb-3ee867507f05 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:list_implied_roles(prior_role_id=4f8031b801684cb4bba94222618410d9)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:27.711584 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-e1a4d0c5-d9f6-495d-b9bb-3ee867507f05 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_implied_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:27.711584 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:27.711584 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:27.713287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4d0c5-d9f6-495d-b9bb-3ee867507f05 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:27.735621 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2168/4407] 10.222.0.17 () {58 vars in 1245 bytes} [Sat Oct 1 21:03:27 2022] GET /identity/v3/roles/4f8031b801684cb4bba94222618410d9/implies => generated 488 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:27.741436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d90bcb-00fe-46e6-a37b-32327c757544 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:27.765171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d90bcb-00fe-46e6-a37b-32327c757544 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:27.773371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401951da-ef78-4a6a-8682-43a2d45b1a8a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:27.774255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401951da-ef78-4a6a-8682-43a2d45b1a8a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:27.774548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401951da-ef78-4a6a-8682-43a2d45b1a8a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:27.774787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401951da-ef78-4a6a-8682-43a2d45b1a8a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:27.778124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401951da-ef78-4a6a-8682-43a2d45b1a8a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:list_implied_roles(prior_role_id=b4075ffb7dc54af7bc5c170586efbec1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:27.779199 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-401951da-ef78-4a6a-8682-43a2d45b1a8a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_implied_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:27.779199 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:27.779199 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:27.780540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401951da-ef78-4a6a-8682-43a2d45b1a8a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:27.798620 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2240/4408] 10.222.0.17 () {58 vars in 1245 bytes} [Sat Oct 1 21:03:27 2022] GET /identity/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies => generated 660 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:27.802259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4d0c5-d9f6-495d-b9bb-3ee867507f05 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:27.825194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4d0c5-d9f6-495d-b9bb-3ee867507f05 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:27.835665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d9339-4f0d-497d-bf7f-f7ea60a7fdb1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:27.841716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d9339-4f0d-497d-bf7f-f7ea60a7fdb1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:27.842747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d9339-4f0d-497d-bf7f-f7ea60a7fdb1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:27.844449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d9339-4f0d-497d-bf7f-f7ea60a7fdb1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/4f8031b801684cb4bba94222618410d9/implies/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:27.847129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d9339-4f0d-497d-bf7f-f7ea60a7fdb1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=4f8031b801684cb4bba94222618410d9, implied_role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:27.849151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d9339-4f0d-497d-bf7f-f7ea60a7fdb1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:27.886405 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2169/4409] 10.222.0.17 () {58 vars in 1347 bytes} [Sat Oct 1 21:03:27 2022] DELETE /identity/v3/roles/4f8031b801684cb4bba94222618410d9/implies/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:27.890582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401951da-ef78-4a6a-8682-43a2d45b1a8a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:27.903081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401951da-ef78-4a6a-8682-43a2d45b1a8a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:27.999410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fb5de-2424-4c99-ad42-a5f57e7faf67 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:28.000085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fb5de-2424-4c99-ad42-a5f57e7faf67 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:28.001037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fb5de-2424-4c99-ad42-a5f57e7faf67 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:28.001037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fb5de-2424-4c99-ad42-a5f57e7faf67 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/4f8031b801684cb4bba94222618410d9` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:28.003605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3fb5de-2424-4c99-ad42-a5f57e7faf67 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=b4075ffb7dc54af7bc5c170586efbec1, implied_role_id=4f8031b801684cb4bba94222618410d9)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:28.007319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3fb5de-2424-4c99-ad42-a5f57e7faf67 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:28.026530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d9339-4f0d-497d-bf7f-f7ea60a7fdb1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:28.027500 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2241/4410] 10.222.0.17 () {58 vars in 1347 bytes} [Sat Oct 1 21:03:27 2022] DELETE /identity/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/4f8031b801684cb4bba94222618410d9 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:28.050003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d9339-4f0d-497d-bf7f-f7ea60a7fdb1 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:28.073665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:28.160008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32eae41-804b-40f8-9a2e-48f20171563f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:28.161259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32eae41-804b-40f8-9a2e-48f20171563f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:28.161589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32eae41-804b-40f8-9a2e-48f20171563f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:28.161882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32eae41-804b-40f8-9a2e-48f20171563f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:28.164260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32eae41-804b-40f8-9a2e-48f20171563f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=b4075ffb7dc54af7bc5c170586efbec1, implied_role_id=a961c41374514b48ad472fddaade0d27)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:28.166526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32eae41-804b-40f8-9a2e-48f20171563f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:28.179300 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2170/4411] 10.222.0.17 () {58 vars in 1347 bytes} [Sat Oct 1 21:03:28 2022] DELETE /identity/v3/roles/b4075ffb7dc54af7bc5c170586efbec1/implies/a961c41374514b48ad472fddaade0d27 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:28.184401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fb5de-2424-4c99-ad42-a5f57e7faf67 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:28.224828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fb5de-2424-4c99-ad42-a5f57e7faf67 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:28.370437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d52ddb-fbf3-4e64-9966-3108af179d4e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:28.371148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d52ddb-fbf3-4e64-9966-3108af179d4e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:28.371431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d52ddb-fbf3-4e64-9966-3108af179d4e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:28.371706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d52ddb-fbf3-4e64-9966-3108af179d4e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/domains/4bc520374aba4df2a4e0d29ed4a8deea` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:28.399188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d52ddb-fbf3-4e64-9966-3108af179d4e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 21:03:28.410162 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-47d52ddb-fbf3-4e64-9966-3108af179d4e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Could not find domain: 4bc520374aba4df2a4e0d29ed4a8deea.: keystone.exception.DomainNotFound: Could not find domain: 4bc520374aba4df2a4e0d29ed4a8deea. Oct 01 21:03:28.410162 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:03:28.410162 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 01 21:03:28.410162 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 01 21:03:28.410162 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 21:03:28.410162 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 21:03:28.410162 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 21:03:28.410162 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 21:03:28.410162 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 4bc520374aba4df2a4e0d29ed4a8deea. Oct 01 21:03:28.410162 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:03:28.410162 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:03:28.410162 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:03:28.410162 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:03:28.410162 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:03:28.410162 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:03:28.410162 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:03:28.412252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:03:28.412252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:03:28.412252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:03:28.412252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:03:28.412252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:03:28.412252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:03:28.412252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:03:28.412252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 01 21:03:28.412252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 01 21:03:28.412252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 21:03:28.412252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 21:03:28.412252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 21:03:28.412252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 21:03:28.412252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:03:28.412252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:03:28.412252 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:03:28.413370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:03:28.413370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:03:28.413370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:03:28.413370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:03:28.413370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 21:03:28.413370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:03:28.413370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:03:28.413370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:03:28.413370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:03:28.413370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:03:28.413370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:03:28.413370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:03:28.413370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:03:28.413370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 01 21:03:28.413370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 21:03:28.413370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 4bc520374aba4df2a4e0d29ed4a8deea. Oct 01 21:03:28.413370 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:03:28.422087 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2242/4412] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:03:28 2022] PATCH /identity/v3/domains/4bc520374aba4df2a4e0d29ed4a8deea => generated 112 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:03:28.423732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32eae41-804b-40f8-9a2e-48f20171563f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:28.438041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32eae41-804b-40f8-9a2e-48f20171563f tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:28.449331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fffb69-ec7c-4d8e-8664-f41084b4fd7c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:28.450467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fffb69-ec7c-4d8e-8664-f41084b4fd7c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:28.451126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fffb69-ec7c-4d8e-8664-f41084b4fd7c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:28.451675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fffb69-ec7c-4d8e-8664-f41084b4fd7c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/users/1ec430a8ba524da0af3eef8e1e8d9d33` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:28.453180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fffb69-ec7c-4d8e-8664-f41084b4fd7c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_user(user_id=1ec430a8ba524da0af3eef8e1e8d9d33)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:28.455526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fffb69-ec7c-4d8e-8664-f41084b4fd7c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:28.560217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f3fffb69-ec7c-4d8e-8664-f41084b4fd7c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ec430a8ba524da0af3eef8e1e8d9d33'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:28.561219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f3fffb69-ec7c-4d8e-8664-f41084b4fd7c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ec430a8ba524da0af3eef8e1e8d9d33'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:28.561951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f3fffb69-ec7c-4d8e-8664-f41084b4fd7c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ec430a8ba524da0af3eef8e1e8d9d33'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:28.577236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f3fffb69-ec7c-4d8e-8664-f41084b4fd7c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ec430a8ba524da0af3eef8e1e8d9d33'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:28.600168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f3fffb69-ec7c-4d8e-8664-f41084b4fd7c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ec430a8ba524da0af3eef8e1e8d9d33'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:28.615052 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2171/4413] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:28 2022] DELETE /identity/v3/users/1ec430a8ba524da0af3eef8e1e8d9d33 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:28.619851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d52ddb-fbf3-4e64-9966-3108af179d4e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:28.621730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-47d52ddb-fbf3-4e64-9966-3108af179d4e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:28.656024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d52ddb-fbf3-4e64-9966-3108af179d4e tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:28.834243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6dc79d-4590-4a32-a58f-c1ed9a9fcbe7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:28.837206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dc79d-4590-4a32-a58f-c1ed9a9fcbe7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:28.837379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dc79d-4590-4a32-a58f-c1ed9a9fcbe7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:28.838328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dc79d-4590-4a32-a58f-c1ed9a9fcbe7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/4dccb2174d5342a2ae2e058c75f23e10` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:28.842325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6dc79d-4590-4a32-a58f-c1ed9a9fcbe7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_role(role_id=4dccb2174d5342a2ae2e058c75f23e10)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:28.847533 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-9c6dc79d-4590-4a32-a58f-c1ed9a9fcbe7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:28.847533 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:28.847533 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:28.850660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6dc79d-4590-4a32-a58f-c1ed9a9fcbe7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:28.853913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:28.968412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9c6dc79d-4590-4a32-a58f-c1ed9a9fcbe7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invalidating the token cache because role 4dccb2174d5342a2ae2e058c75f23e10 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:28.968843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9c6dc79d-4590-4a32-a58f-c1ed9a9fcbe7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:28.970002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9c6dc79d-4590-4a32-a58f-c1ed9a9fcbe7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:28.975904 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2243/4414] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:28 2022] DELETE /identity/v3/roles/4dccb2174d5342a2ae2e058c75f23e10 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:28.976960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fffb69-ec7c-4d8e-8664-f41084b4fd7c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:28.997806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fffb69-ec7c-4d8e-8664-f41084b4fd7c tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:29.130208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98733f9a-73d5-40fb-bb90-fd88c8ea759b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:29.132978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98733f9a-73d5-40fb-bb90-fd88c8ea759b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:29.133548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98733f9a-73d5-40fb-bb90-fd88c8ea759b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:29.138379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98733f9a-73d5-40fb-bb90-fd88c8ea759b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/groups/2d025e446f854f71a7d59440f98f3eaf` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:29.140159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98733f9a-73d5-40fb-bb90-fd88c8ea759b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_group(group_id=2d025e446f854f71a7d59440f98f3eaf)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:29.141457 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-98733f9a-73d5-40fb-bb90-fd88c8ea759b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:29.141457 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:29.141457 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:29.144009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98733f9a-73d5-40fb-bb90-fd88c8ea759b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:29.195852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:29.235872 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2172/4415] 10.222.0.17 () {58 vars in 1227 bytes} [Sat Oct 1 21:03:28 2022] DELETE /identity/v3/groups/2d025e446f854f71a7d59440f98f3eaf => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:29.239604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6dc79d-4590-4a32-a58f-c1ed9a9fcbe7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:29.241027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9c6dc79d-4590-4a32-a58f-c1ed9a9fcbe7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:29.260402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6dc79d-4590-4a32-a58f-c1ed9a9fcbe7 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:29.385654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5fe0f-8a79-4a4e-9d74-23a7d17ac8f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:29.386730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5fe0f-8a79-4a4e-9d74-23a7d17ac8f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:29.387123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5fe0f-8a79-4a4e-9d74-23a7d17ac8f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:29.387507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5fe0f-8a79-4a4e-9d74-23a7d17ac8f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:29.388971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5fe0f-8a79-4a4e-9d74-23a7d17ac8f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f69494f2b2fc40f29be4d38fe0a42b87)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:29.391216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5fe0f-8a79-4a4e-9d74-23a7d17ac8f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:29.520720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-73e5fe0f-8a79-4a4e-9d74-23a7d17ac8f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f69494f2b2fc40f29be4d38fe0a42b87', 'request_id': 'req-73e5fe0f-8a79-4a4e-9d74-23a7d17ac8f8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:29.543163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-73e5fe0f-8a79-4a4e-9d74-23a7d17ac8f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f69494f2b2fc40f29be4d38fe0a42b87', 'request_id': 'req-73e5fe0f-8a79-4a4e-9d74-23a7d17ac8f8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:29.553709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-73e5fe0f-8a79-4a4e-9d74-23a7d17ac8f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f69494f2b2fc40f29be4d38fe0a42b87', 'request_id': 'req-73e5fe0f-8a79-4a4e-9d74-23a7d17ac8f8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:29.575148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-73e5fe0f-8a79-4a4e-9d74-23a7d17ac8f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] The token cache is being invalidate because project f69494f2b2fc40f29be4d38fe0a42b87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:29.575465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-73e5fe0f-8a79-4a4e-9d74-23a7d17ac8f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:29.576593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-73e5fe0f-8a79-4a4e-9d74-23a7d17ac8f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:29.580356 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2244/4416] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:29 2022] DELETE /identity/v3/projects/f69494f2b2fc40f29be4d38fe0a42b87 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:29.584663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98733f9a-73d5-40fb-bb90-fd88c8ea759b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:29.586612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-98733f9a-73d5-40fb-bb90-fd88c8ea759b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:29.605556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98733f9a-73d5-40fb-bb90-fd88c8ea759b tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:29.714347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92697d48-f0ea-499c-b1bd-d86d621eb8ef tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:29.715502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92697d48-f0ea-499c-b1bd-d86d621eb8ef tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:29.715950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92697d48-f0ea-499c-b1bd-d86d621eb8ef tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:29.716386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92697d48-f0ea-499c-b1bd-d86d621eb8ef tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/domains/c2c3d42d4aa4486aa40ca857514b1d86` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:29.718216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92697d48-f0ea-499c-b1bd-d86d621eb8ef tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=c2c3d42d4aa4486aa40ca857514b1d86)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:29.719804 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-92697d48-f0ea-499c-b1bd-d86d621eb8ef tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:29.719804 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:29.719804 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:29.721219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92697d48-f0ea-499c-b1bd-d86d621eb8ef tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:29.756357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-92697d48-f0ea-499c-b1bd-d86d621eb8ef tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'c2c3d42d4aa4486aa40ca857514b1d86'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:29.757491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-92697d48-f0ea-499c-b1bd-d86d621eb8ef tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'c2c3d42d4aa4486aa40ca857514b1d86'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:29.803360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-92697d48-f0ea-499c-b1bd-d86d621eb8ef tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'c2c3d42d4aa4486aa40ca857514b1d86'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:29.805683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-92697d48-f0ea-499c-b1bd-d86d621eb8ef tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'c2c3d42d4aa4486aa40ca857514b1d86'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:29.811570 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2173/4417] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:03:29 2022] PATCH /identity/v3/domains/c2c3d42d4aa4486aa40ca857514b1d86 => generated 276 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:29.815365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5fe0f-8a79-4a4e-9d74-23a7d17ac8f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:29.831794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5fe0f-8a79-4a4e-9d74-23a7d17ac8f8 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:29.954947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:29.968175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:29.968806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:29.969099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:29.969402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/domains/c2c3d42d4aa4486aa40ca857514b1d86` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:29.989603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=c2c3d42d4aa4486aa40ca857514b1d86)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:29.990828 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:29.990828 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:29.990828 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:29.992265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:30.031932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'c2c3d42d4aa4486aa40ca857514b1d86'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:30.040208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'c2c3d42d4aa4486aa40ca857514b1d86'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:30.140044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2c3d42d4aa4486aa40ca857514b1d86', 'request_id': 'req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:30.154517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2c3d42d4aa4486aa40ca857514b1d86', 'request_id': 'req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:30.156588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2c3d42d4aa4486aa40ca857514b1d86', 'request_id': 'req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:30.187652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] The token cache is being invalidate because project c2c3d42d4aa4486aa40ca857514b1d86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:30.187956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:30.188829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:30.204052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'c2c3d42d4aa4486aa40ca857514b1d86', 'request_id': 'req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:30.206460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'c2c3d42d4aa4486aa40ca857514b1d86', 'request_id': 'req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:30.209080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'c2c3d42d4aa4486aa40ca857514b1d86', 'request_id': 'req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:30.229813 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2245/4418] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:03:29 2022] DELETE /identity/v3/domains/c2c3d42d4aa4486aa40ca857514b1d86 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:30.231726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92697d48-f0ea-499c-b1bd-d86d621eb8ef tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:30.233578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-92697d48-f0ea-499c-b1bd-d86d621eb8ef tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:30.257926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92697d48-f0ea-499c-b1bd-d86d621eb8ef tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:30.388435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b622fa3-bc65-46e5-941d-a9f3fa950e49 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:30.389042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b622fa3-bc65-46e5-941d-a9f3fa950e49 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:30.389329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b622fa3-bc65-46e5-941d-a9f3fa950e49 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:30.389602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b622fa3-bc65-46e5-941d-a9f3fa950e49 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/4f8031b801684cb4bba94222618410d9` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:30.391100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b622fa3-bc65-46e5-941d-a9f3fa950e49 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_role(role_id=4f8031b801684cb4bba94222618410d9)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:30.392706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b622fa3-bc65-46e5-941d-a9f3fa950e49 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:30.475560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8b622fa3-bc65-46e5-941d-a9f3fa950e49 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invalidating the token cache because role 4f8031b801684cb4bba94222618410d9 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:30.475843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8b622fa3-bc65-46e5-941d-a9f3fa950e49 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:30.476754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8b622fa3-bc65-46e5-941d-a9f3fa950e49 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:30.482050 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2174/4419] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:30 2022] DELETE /identity/v3/roles/4f8031b801684cb4bba94222618410d9 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:30.483224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:30.485488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:30.518576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37f080-6de5-4e2a-805a-1fc8a1c14bb3 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:30.593435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cd317-2cc1-447f-89cb-e4f4bec283b4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:30.594184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cd317-2cc1-447f-89cb-e4f4bec283b4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:30.594564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cd317-2cc1-447f-89cb-e4f4bec283b4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:30.594910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cd317-2cc1-447f-89cb-e4f4bec283b4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/a961c41374514b48ad472fddaade0d27` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:30.599101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2cd317-2cc1-447f-89cb-e4f4bec283b4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_role(role_id=a961c41374514b48ad472fddaade0d27)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:30.600833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2cd317-2cc1-447f-89cb-e4f4bec283b4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:30.663485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:30.671112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0a2cd317-2cc1-447f-89cb-e4f4bec283b4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invalidating the token cache because role a961c41374514b48ad472fddaade0d27 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:30.671532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0a2cd317-2cc1-447f-89cb-e4f4bec283b4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:30.679022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0a2cd317-2cc1-447f-89cb-e4f4bec283b4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:30.682682 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2246/4420] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:30 2022] DELETE /identity/v3/roles/a961c41374514b48ad472fddaade0d27 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:30.686753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b622fa3-bc65-46e5-941d-a9f3fa950e49 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:30.689089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8b622fa3-bc65-46e5-941d-a9f3fa950e49 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:30.728237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b622fa3-bc65-46e5-941d-a9f3fa950e49 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:30.771492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:UPDATE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:30.878043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ac606-984d-43c7-aa7e-5e15dbcc548a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd8a17c6e9d3437295316e2018e7f6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:30.880429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac606-984d-43c7-aa7e-5e15dbcc548a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:30.880838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac606-984d-43c7-aa7e-5e15dbcc548a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:30.881221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac606-984d-43c7-aa7e-5e15dbcc548a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] PATH_INFO: `/v3/roles/b4075ffb7dc54af7bc5c170586efbec1` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:30.885970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ac606-984d-43c7-aa7e-5e15dbcc548a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorizing `identity:delete_role(role_id=b4075ffb7dc54af7bc5c170586efbec1)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:30.891901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ac606-984d-43c7-aa7e-5e15dbcc548a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:30.964198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cd5ac606-984d-43c7-aa7e-5e15dbcc548a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invalidating the token cache because role b4075ffb7dc54af7bc5c170586efbec1 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:30.964811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cd5ac606-984d-43c7-aa7e-5e15dbcc548a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:30.965889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cd5ac606-984d-43c7-aa7e-5e15dbcc548a tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:30.969605 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2175/4421] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:30 2022] DELETE /identity/v3/roles/b4075ffb7dc54af7bc5c170586efbec1 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:30.974441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cd317-2cc1-447f-89cb-e4f4bec283b4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:30.976002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0a2cd317-2cc1-447f-89cb-e4f4bec283b4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:31.138658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cd317-2cc1-447f-89cb-e4f4bec283b4 tempest-RolesV3TestJSON-1093901009 tempest-RolesV3TestJSON-1093901009-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:31.249837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a8a42-d640-485a-9e03-834402714d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:31.251337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a8a42-d640-485a-9e03-834402714d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:31.251762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a8a42-d640-485a-9e03-834402714d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:31.252137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a8a42-d640-485a-9e03-834402714d25 admin admin] PATH_INFO: `/v3/users/1394694d13444278a0e18fd66376fa0c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:31.254595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a8a42-d640-485a-9e03-834402714d25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1394694d13444278a0e18fd66376fa0c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:31.257732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a8a42-d640-485a-9e03-834402714d25 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:31.357653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-870a8a42-d640-485a-9e03-834402714d25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1394694d13444278a0e18fd66376fa0c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:31.386524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-870a8a42-d640-485a-9e03-834402714d25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1394694d13444278a0e18fd66376fa0c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:31.414055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-870a8a42-d640-485a-9e03-834402714d25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1394694d13444278a0e18fd66376fa0c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:31.414055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-870a8a42-d640-485a-9e03-834402714d25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1394694d13444278a0e18fd66376fa0c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:31.426529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-870a8a42-d640-485a-9e03-834402714d25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1394694d13444278a0e18fd66376fa0c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:31.442128 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2247/4422] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:30 2022] DELETE /identity/v3/users/1394694d13444278a0e18fd66376fa0c => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:31.445997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385f399-e197-4611-9f4d-844eb7771dcc None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:31.446298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385f399-e197-4611-9f4d-844eb7771dcc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:31.446571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385f399-e197-4611-9f4d-844eb7771dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:31.488944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d385f399-e197-4611-9f4d-844eb7771dcc None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:31.490781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d385f399-e197-4611-9f4d-844eb7771dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:31.689820 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2176/4423] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:31.714605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a8a42-d640-485a-9e03-834402714d25 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:31.716270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-870a8a42-d640-485a-9e03-834402714d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:31.787219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:31.871591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a8a42-d640-485a-9e03-834402714d25 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:31.873337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69ef9a-cdc2-4c1b-a53e-fb415f950373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:31.874051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69ef9a-cdc2-4c1b-a53e-fb415f950373 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:31.874482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69ef9a-cdc2-4c1b-a53e-fb415f950373 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:31.874918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69ef9a-cdc2-4c1b-a53e-fb415f950373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:31.886832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69ef9a-cdc2-4c1b-a53e-fb415f950373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:31.890256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69ef9a-cdc2-4c1b-a53e-fb415f950373 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:31.907770 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2248/4424] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:03:32.799876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385f399-e197-4611-9f4d-844eb7771dcc None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:32.803714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d385f399-e197-4611-9f4d-844eb7771dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:32.825627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385f399-e197-4611-9f4d-844eb7771dcc None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:32.842170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d79a81-f937-4360-a366-4c590f0bcee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:32.842860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d79a81-f937-4360-a366-4c590f0bcee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:32.843232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d79a81-f937-4360-a366-4c590f0bcee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:32.843521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d79a81-f937-4360-a366-4c590f0bcee2 admin admin] PATH_INFO: `/v3/projects/3c17730b45e3489f89165499dc67eb21` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:32.844789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d79a81-f937-4360-a366-4c590f0bcee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c17730b45e3489f89165499dc67eb21)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:32.846796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d79a81-f937-4360-a366-4c590f0bcee2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:32.980080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e2d79a81-f937-4360-a366-4c590f0bcee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c17730b45e3489f89165499dc67eb21', 'request_id': 'req-e2d79a81-f937-4360-a366-4c590f0bcee2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:32.991515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e2d79a81-f937-4360-a366-4c590f0bcee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c17730b45e3489f89165499dc67eb21', 'request_id': 'req-e2d79a81-f937-4360-a366-4c590f0bcee2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:33.000642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e2d79a81-f937-4360-a366-4c590f0bcee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c17730b45e3489f89165499dc67eb21', 'request_id': 'req-e2d79a81-f937-4360-a366-4c590f0bcee2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:33.040639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e2d79a81-f937-4360-a366-4c590f0bcee2 admin admin] The token cache is being invalidate because project 3c17730b45e3489f89165499dc67eb21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:33.040959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e2d79a81-f937-4360-a366-4c590f0bcee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:33.041852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e2d79a81-f937-4360-a366-4c590f0bcee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:33.051705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69ef9a-cdc2-4c1b-a53e-fb415f950373 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:33.052246 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2177/4425] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:32 2022] DELETE /identity/v3/projects/3c17730b45e3489f89165499dc67eb21 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:33.069805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69ef9a-cdc2-4c1b-a53e-fb415f950373 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:33.218873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc6ac2-800b-4d03-abac-787b074bab38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:33.219540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc6ac2-800b-4d03-abac-787b074bab38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:33.219823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc6ac2-800b-4d03-abac-787b074bab38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:33.220106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc6ac2-800b-4d03-abac-787b074bab38 admin admin] PATH_INFO: `/v3/users/cd8a17c6e9d3437295316e2018e7f6e2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:33.224678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc6ac2-800b-4d03-abac-787b074bab38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd8a17c6e9d3437295316e2018e7f6e2)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:33.227652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc6ac2-800b-4d03-abac-787b074bab38 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:33.334255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3bfc6ac2-800b-4d03-abac-787b074bab38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd8a17c6e9d3437295316e2018e7f6e2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:33.350366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3bfc6ac2-800b-4d03-abac-787b074bab38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd8a17c6e9d3437295316e2018e7f6e2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:33.383168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3bfc6ac2-800b-4d03-abac-787b074bab38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd8a17c6e9d3437295316e2018e7f6e2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:33.384692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3bfc6ac2-800b-4d03-abac-787b074bab38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd8a17c6e9d3437295316e2018e7f6e2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:33.400935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3bfc6ac2-800b-4d03-abac-787b074bab38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd8a17c6e9d3437295316e2018e7f6e2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:33.421333 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2249/4426] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:33 2022] DELETE /identity/v3/users/cd8a17c6e9d3437295316e2018e7f6e2 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:34.188310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d79a81-f937-4360-a366-4c590f0bcee2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:34.189935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e2d79a81-f937-4360-a366-4c590f0bcee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:34.202922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:34.208728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d79a81-f937-4360-a366-4c590f0bcee2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:34.288088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f9938-caf2-48e6-8bdb-12772c89459a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:34.288732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f9938-caf2-48e6-8bdb-12772c89459a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:34.289224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f9938-caf2-48e6-8bdb-12772c89459a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:34.289505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f9938-caf2-48e6-8bdb-12772c89459a admin admin] PATH_INFO: `/v3/projects/a1653d0e18814d7e8e6e0489a5b27fc4` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:34.290732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f9938-caf2-48e6-8bdb-12772c89459a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1653d0e18814d7e8e6e0489a5b27fc4)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:34.292663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f9938-caf2-48e6-8bdb-12772c89459a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:34.355705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-607f9938-caf2-48e6-8bdb-12772c89459a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'request_id': 'req-607f9938-caf2-48e6-8bdb-12772c89459a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:34.366624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-607f9938-caf2-48e6-8bdb-12772c89459a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'request_id': 'req-607f9938-caf2-48e6-8bdb-12772c89459a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:34.377504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-607f9938-caf2-48e6-8bdb-12772c89459a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1653d0e18814d7e8e6e0489a5b27fc4', 'request_id': 'req-607f9938-caf2-48e6-8bdb-12772c89459a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:34.407870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-607f9938-caf2-48e6-8bdb-12772c89459a admin admin] The token cache is being invalidate because project a1653d0e18814d7e8e6e0489a5b27fc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:34.408545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-607f9938-caf2-48e6-8bdb-12772c89459a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:34.409226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-607f9938-caf2-48e6-8bdb-12772c89459a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:34.411514 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2178/4427] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:34 2022] DELETE /identity/v3/projects/a1653d0e18814d7e8e6e0489a5b27fc4 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:34.416830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c3325-3613-431f-92ef-41b4e7a37344 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:34.417308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c3325-3613-431f-92ef-41b4e7a37344 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:34.417522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c3325-3613-431f-92ef-41b4e7a37344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:34.452453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ce0c3325-3613-431f-92ef-41b4e7a37344 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:34.453788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ce0c3325-3613-431f-92ef-41b4e7a37344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:34.524495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:34.558916 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2250/4428] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:34.562568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2516f-b161-448e-b2a9-c8ee26db2f3f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:34.563633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2516f-b161-448e-b2a9-c8ee26db2f3f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:34.565160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2516f-b161-448e-b2a9-c8ee26db2f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:34.596802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e7c2516f-b161-448e-b2a9-c8ee26db2f3f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:34.598880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e7c2516f-b161-448e-b2a9-c8ee26db2f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:34.611299 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2179/4429] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:34.615520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c3325-3613-431f-92ef-41b4e7a37344 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:34.632852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c3325-3613-431f-92ef-41b4e7a37344 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:34.643705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa42986-26aa-41b9-874a-59ff4d3e2968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:34.645274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa42986-26aa-41b9-874a-59ff4d3e2968 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:34.646216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa42986-26aa-41b9-874a-59ff4d3e2968 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:34.646617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa42986-26aa-41b9-874a-59ff4d3e2968 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:34.647557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa42986-26aa-41b9-874a-59ff4d3e2968 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:03:34.647883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa42986-26aa-41b9-874a-59ff4d3e2968 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:34.649931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa42986-26aa-41b9-874a-59ff4d3e2968 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:34.667074 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2251/4430] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:34.671549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2516f-b161-448e-b2a9-c8ee26db2f3f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:34.684764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2516f-b161-448e-b2a9-c8ee26db2f3f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:34.691965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438741d-3b3a-427c-b789-6562ae812f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:34.692977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438741d-3b3a-427c-b789-6562ae812f12 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:34.693323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438741d-3b3a-427c-b789-6562ae812f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:34.693712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438741d-3b3a-427c-b789-6562ae812f12 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:34.694577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438741d-3b3a-427c-b789-6562ae812f12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:34.696696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438741d-3b3a-427c-b789-6562ae812f12 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:34.791420 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2180/4431] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:34 2022] POST /identity/v3/projects => generated 369 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:34.795674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa42986-26aa-41b9-874a-59ff4d3e2968 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:34.833264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa42986-26aa-41b9-874a-59ff4d3e2968 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:34.917381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593b052-b13c-4831-91c9-cd3625bec6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:34.918069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593b052-b13c-4831-91c9-cd3625bec6a0 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:34.918268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593b052-b13c-4831-91c9-cd3625bec6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:34.918576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593b052-b13c-4831-91c9-cd3625bec6a0 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:34.926871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593b052-b13c-4831-91c9-cd3625bec6a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:34.931589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593b052-b13c-4831-91c9-cd3625bec6a0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:34.963021 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2252/4432] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:03:34 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:34.966997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438741d-3b3a-427c-b789-6562ae812f12 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:34.993930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438741d-3b3a-427c-b789-6562ae812f12 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:35.001552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450141a-d097-4cdb-94b7-1fa64f1be91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:35.002262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450141a-d097-4cdb-94b7-1fa64f1be91e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:35.002581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450141a-d097-4cdb-94b7-1fa64f1be91e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:35.002861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450141a-d097-4cdb-94b7-1fa64f1be91e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:35.003520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9450141a-d097-4cdb-94b7-1fa64f1be91e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:35.005687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9450141a-d097-4cdb-94b7-1fa64f1be91e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:35.039149 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2181/4433] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:35.042777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593b052-b13c-4831-91c9-cd3625bec6a0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:35.082114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593b052-b13c-4831-91c9-cd3625bec6a0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:35.090405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36126843-3bb0-47fe-bd69-fb783770fefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:35.091255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36126843-3bb0-47fe-bd69-fb783770fefc admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:35.091530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36126843-3bb0-47fe-bd69-fb783770fefc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:35.093468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36126843-3bb0-47fe-bd69-fb783770fefc admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:35.093468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36126843-3bb0-47fe-bd69-fb783770fefc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:35.094376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36126843-3bb0-47fe-bd69-fb783770fefc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:35.139718 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2253/4434] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:35.141078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450141a-d097-4cdb-94b7-1fa64f1be91e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:35.161413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450141a-d097-4cdb-94b7-1fa64f1be91e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:35.199852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197db17-cfbd-40d9-b7d6-78a8a8535bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:35.204887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197db17-cfbd-40d9-b7d6-78a8a8535bac admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:35.204887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197db17-cfbd-40d9-b7d6-78a8a8535bac admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:35.204887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197db17-cfbd-40d9-b7d6-78a8a8535bac admin admin] PATH_INFO: `/v3/projects/4018320ded0c4162a623a5b5b8fd57d0/users/acdb7b0598b3497bbd8d197c7e7a9ee3/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:35.237334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197db17-cfbd-40d9-b7d6-78a8a8535bac admin admin] RBAC: Authorizing `identity:create_grant(project_id=4018320ded0c4162a623a5b5b8fd57d0, user_id=acdb7b0598b3497bbd8d197c7e7a9ee3, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:35.241194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197db17-cfbd-40d9-b7d6-78a8a8535bac admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:35.258702 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2182/4435] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:35 2022] PUT /identity/v3/projects/4018320ded0c4162a623a5b5b8fd57d0/users/acdb7b0598b3497bbd8d197c7e7a9ee3/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:35.269341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6705862-8e45-45ed-bb58-2d1d10323df1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:35.269619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6705862-8e45-45ed-bb58-2d1d10323df1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:35.269896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6705862-8e45-45ed-bb58-2d1d10323df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:35.304758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a6705862-8e45-45ed-bb58-2d1d10323df1 None None] MFA Rules not processed for user `acdb7b0598b3497bbd8d197c7e7a9ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:35.308410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a6705862-8e45-45ed-bb58-2d1d10323df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:35.385733 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2254/4436] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:35.390704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197db17-cfbd-40d9-b7d6-78a8a8535bac admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:35.429811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:35.470744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197db17-cfbd-40d9-b7d6-78a8a8535bac admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:35.477210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a23bae-b0c4-4374-9e3c-685e32caac2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:35.477893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a23bae-b0c4-4374-9e3c-685e32caac2a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:35.478232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a23bae-b0c4-4374-9e3c-685e32caac2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:35.478538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a23bae-b0c4-4374-9e3c-685e32caac2a admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:35.479445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a23bae-b0c4-4374-9e3c-685e32caac2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:35.481495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a23bae-b0c4-4374-9e3c-685e32caac2a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:35.521126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6705862-8e45-45ed-bb58-2d1d10323df1 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:35.529131 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2183/4437] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:35 2022] POST /identity/v3/projects => generated 369 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:35.534577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6705862-8e45-45ed-bb58-2d1d10323df1 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:35.592222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:35.626664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861d508-0641-48f6-a5cf-97d949e7217e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:35.627470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861d508-0641-48f6-a5cf-97d949e7217e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:35.627663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861d508-0641-48f6-a5cf-97d949e7217e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:35.628057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861d508-0641-48f6-a5cf-97d949e7217e admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:35.629455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861d508-0641-48f6-a5cf-97d949e7217e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:35.631642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861d508-0641-48f6-a5cf-97d949e7217e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:35.662191 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2255/4438] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:03:35 2022] POST /identity/v3/users => generated 291 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:35.666112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a23bae-b0c4-4374-9e3c-685e32caac2a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:35.688318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a23bae-b0c4-4374-9e3c-685e32caac2a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:35.701630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe4c2b-f693-4636-ac22-2180f3b0aaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:35.702291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe4c2b-f693-4636-ac22-2180f3b0aaa9 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:35.702747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe4c2b-f693-4636-ac22-2180f3b0aaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:35.703033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe4c2b-f693-4636-ac22-2180f3b0aaa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:35.703689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe4c2b-f693-4636-ac22-2180f3b0aaa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:35.705319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe4c2b-f693-4636-ac22-2180f3b0aaa9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:35.724278 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2184/4439] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:35.725801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861d508-0641-48f6-a5cf-97d949e7217e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:35.741210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861d508-0641-48f6-a5cf-97d949e7217e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:35.748081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e820cf-3b7e-49bd-8f24-82a7dc443220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:35.749038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e820cf-3b7e-49bd-8f24-82a7dc443220 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:35.749326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e820cf-3b7e-49bd-8f24-82a7dc443220 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:35.749603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e820cf-3b7e-49bd-8f24-82a7dc443220 admin admin] PATH_INFO: `/v3/projects/1045b9dc011e474086924aa42c3fa371/users/fabd1d6030774f91af00089422dc0806/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:35.785646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e820cf-3b7e-49bd-8f24-82a7dc443220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1045b9dc011e474086924aa42c3fa371, user_id=fabd1d6030774f91af00089422dc0806, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:35.788673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e820cf-3b7e-49bd-8f24-82a7dc443220 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:35.808703 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2256/4440] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:35 2022] PUT /identity/v3/projects/1045b9dc011e474086924aa42c3fa371/users/fabd1d6030774f91af00089422dc0806/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:35.812451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe4c2b-f693-4636-ac22-2180f3b0aaa9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:35.826665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe4c2b-f693-4636-ac22-2180f3b0aaa9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:35.913528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522834a-d31e-45b3-9d59-065e4dc6ccca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:35.914236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522834a-d31e-45b3-9d59-065e4dc6ccca admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:35.914605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522834a-d31e-45b3-9d59-065e4dc6ccca admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:35.914952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522834a-d31e-45b3-9d59-065e4dc6ccca admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:35.915863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522834a-d31e-45b3-9d59-065e4dc6ccca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:35.917618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522834a-d31e-45b3-9d59-065e4dc6ccca admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:35.934730 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2185/4441] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:35.937233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e820cf-3b7e-49bd-8f24-82a7dc443220 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:35.955000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e820cf-3b7e-49bd-8f24-82a7dc443220 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:35.981257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944485c9-8225-4cb8-850a-554f9a4c4ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:35.982318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944485c9-8225-4cb8-850a-554f9a4c4ce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:35.982640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944485c9-8225-4cb8-850a-554f9a4c4ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:35.983764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944485c9-8225-4cb8-850a-554f9a4c4ce3 admin admin] PATH_INFO: `/v3/projects/1045b9dc011e474086924aa42c3fa371/users/fabd1d6030774f91af00089422dc0806/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:35.988610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944485c9-8225-4cb8-850a-554f9a4c4ce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1045b9dc011e474086924aa42c3fa371, user_id=fabd1d6030774f91af00089422dc0806, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:35.997757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944485c9-8225-4cb8-850a-554f9a4c4ce3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:36.041365 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2257/4442] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:35 2022] PUT /identity/v3/projects/1045b9dc011e474086924aa42c3fa371/users/fabd1d6030774f91af00089422dc0806/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:36.056088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85916c6f-046f-4c42-89bd-fc05050a9fbe None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:36.056478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85916c6f-046f-4c42-89bd-fc05050a9fbe None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:36.056834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85916c6f-046f-4c42-89bd-fc05050a9fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:36.097923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-85916c6f-046f-4c42-89bd-fc05050a9fbe None None] MFA Rules not processed for user `fabd1d6030774f91af00089422dc0806`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:36.099252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-85916c6f-046f-4c42-89bd-fc05050a9fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:36.186533 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2186/4443] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:36 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:36.192473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944485c9-8225-4cb8-850a-554f9a4c4ce3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:36.205228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944485c9-8225-4cb8-850a-554f9a4c4ce3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:36.211343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c1d9b-7306-451f-9f45-3bea2e16f100 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fabd1d6030774f91af00089422dc0806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1045b9dc011e474086924aa42c3fa371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:36.211969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c1d9b-7306-451f-9f45-3bea2e16f100 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:36.212262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c1d9b-7306-451f-9f45-3bea2e16f100 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:36.212542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c1d9b-7306-451f-9f45-3bea2e16f100 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:36.213203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c1d9b-7306-451f-9f45-3bea2e16f100 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:36.214812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c1d9b-7306-451f-9f45-3bea2e16f100 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:36.231706 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2258/4444] 10.222.0.17 () {60 vars in 1152 bytes} [Sat Oct 1 21:03:36 2022] POST /identity/v3/services => generated 240 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:03:36.236102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85916c6f-046f-4c42-89bd-fc05050a9fbe None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:36.245988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85916c6f-046f-4c42-89bd-fc05050a9fbe None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:36.262029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8f9f0-bbd7-4b57-97dd-f0096e59c8fa tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fabd1d6030774f91af00089422dc0806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1045b9dc011e474086924aa42c3fa371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:36.262680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8f9f0-bbd7-4b57-97dd-f0096e59c8fa tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:36.263097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8f9f0-bbd7-4b57-97dd-f0096e59c8fa tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:36.263380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8f9f0-bbd7-4b57-97dd-f0096e59c8fa tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] PATH_INFO: `/v3/services/dcf1d974aef340bea93372bd4ba66987` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:36.263961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8f9f0-bbd7-4b57-97dd-f0096e59c8fa tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorizing `identity:delete_service(service_id=dcf1d974aef340bea93372bd4ba66987)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:36.265948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8f9f0-bbd7-4b57-97dd-f0096e59c8fa tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:36.275288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:36.290386 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2187/4445] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:36 2022] DELETE /identity/v3/services/dcf1d974aef340bea93372bd4ba66987 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:36.296083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c1d9b-7306-451f-9f45-3bea2e16f100 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:36.307242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c1d9b-7306-451f-9f45-3bea2e16f100 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:36.322990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945ac7f-d44b-411e-9565-b30f51740f1d tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fabd1d6030774f91af00089422dc0806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1045b9dc011e474086924aa42c3fa371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:36.323595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945ac7f-d44b-411e-9565-b30f51740f1d tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:36.323874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945ac7f-d44b-411e-9565-b30f51740f1d tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:36.324149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945ac7f-d44b-411e-9565-b30f51740f1d tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:36.324797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945ac7f-d44b-411e-9565-b30f51740f1d tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:36.326372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945ac7f-d44b-411e-9565-b30f51740f1d tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:36.347735 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2259/4446] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:36 2022] POST /identity/v3/services => generated 289 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:36.349049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8f9f0-bbd7-4b57-97dd-f0096e59c8fa tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:36.359688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8f9f0-bbd7-4b57-97dd-f0096e59c8fa tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:36.376451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065dd427-f5ab-4b99-894a-6e094117f4a0 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fabd1d6030774f91af00089422dc0806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1045b9dc011e474086924aa42c3fa371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:36.377168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065dd427-f5ab-4b99-894a-6e094117f4a0 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:36.377488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065dd427-f5ab-4b99-894a-6e094117f4a0 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:36.377743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065dd427-f5ab-4b99-894a-6e094117f4a0 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] PATH_INFO: `/v3/services/a194adb825034310916ad4e14af36c1f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:36.379680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065dd427-f5ab-4b99-894a-6e094117f4a0 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorizing `identity:update_service(service_id=a194adb825034310916ad4e14af36c1f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:36.380737 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-065dd427-f5ab-4b99-894a-6e094117f4a0 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:36.380737 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:36.380737 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:36.381826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065dd427-f5ab-4b99-894a-6e094117f4a0 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:36.403214 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2188/4447] 10.222.0.17 () {60 vars in 1252 bytes} [Sat Oct 1 21:03:36 2022] PATCH /identity/v3/services/a194adb825034310916ad4e14af36c1f => generated 283 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:36.407303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945ac7f-d44b-411e-9565-b30f51740f1d tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:36.418668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945ac7f-d44b-411e-9565-b30f51740f1d tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:36.432879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c581d-869f-43b6-b94a-97c1954b76ce tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fabd1d6030774f91af00089422dc0806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1045b9dc011e474086924aa42c3fa371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:36.433605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c581d-869f-43b6-b94a-97c1954b76ce tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:36.433962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c581d-869f-43b6-b94a-97c1954b76ce tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:36.434332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c581d-869f-43b6-b94a-97c1954b76ce tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] PATH_INFO: `/v3/services/a194adb825034310916ad4e14af36c1f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:36.435018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c581d-869f-43b6-b94a-97c1954b76ce tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorizing `identity:get_service(service_id=a194adb825034310916ad4e14af36c1f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:36.436140 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-dd3c581d-869f-43b6-b94a-97c1954b76ce tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_service": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:36.436140 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:36.436140 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:36.437204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c581d-869f-43b6-b94a-97c1954b76ce tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:36.443638 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2260/4448] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:03:36 2022] GET /identity/v3/services/a194adb825034310916ad4e14af36c1f => generated 283 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:36.447640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065dd427-f5ab-4b99-894a-6e094117f4a0 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:36.458869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065dd427-f5ab-4b99-894a-6e094117f4a0 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:36.463631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f449a-60b2-4fd1-a72b-7567a3b35e8e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fabd1d6030774f91af00089422dc0806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1045b9dc011e474086924aa42c3fa371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:36.464260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f449a-60b2-4fd1-a72b-7567a3b35e8e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:36.464544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f449a-60b2-4fd1-a72b-7567a3b35e8e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:36.464819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f449a-60b2-4fd1-a72b-7567a3b35e8e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] PATH_INFO: `/v3/services/a194adb825034310916ad4e14af36c1f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:36.465398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f449a-60b2-4fd1-a72b-7567a3b35e8e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorizing `identity:delete_service(service_id=a194adb825034310916ad4e14af36c1f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:36.467038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f449a-60b2-4fd1-a72b-7567a3b35e8e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:36.491190 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2189/4449] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:36 2022] DELETE /identity/v3/services/a194adb825034310916ad4e14af36c1f => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:36.495649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c581d-869f-43b6-b94a-97c1954b76ce tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:36.507456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c581d-869f-43b6-b94a-97c1954b76ce tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:36.536481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18523624-37f7-4e5a-818f-72d362155d12 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fabd1d6030774f91af00089422dc0806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1045b9dc011e474086924aa42c3fa371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:36.537901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18523624-37f7-4e5a-818f-72d362155d12 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:36.538420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18523624-37f7-4e5a-818f-72d362155d12 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:36.538838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18523624-37f7-4e5a-818f-72d362155d12 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] PATH_INFO: `/v3/services/a194adb825034310916ad4e14af36c1f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:36.539681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18523624-37f7-4e5a-818f-72d362155d12 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorizing `identity:get_service(service_id=a194adb825034310916ad4e14af36c1f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:36.541345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18523624-37f7-4e5a-818f-72d362155d12 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:36.558932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-18523624-37f7-4e5a-818f-72d362155d12 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Could not find service: a194adb825034310916ad4e14af36c1f.: keystone.exception.ServiceNotFound: Could not find service: a194adb825034310916ad4e14af36c1f. Oct 01 21:03:36.558932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:03:36.558932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 01 21:03:36.558932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 01 21:03:36.558932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 01 21:03:36.558932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 01 21:03:36.558932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 01 21:03:36.558932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 21:03:36.558932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: a194adb825034310916ad4e14af36c1f. Oct 01 21:03:36.558932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:03:36.558932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:03:36.558932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:03:36.558932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:03:36.558932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:03:36.558932 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:03:36.560278 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:03:36.560278 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:03:36.560278 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:03:36.560278 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:03:36.560278 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:03:36.560278 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:03:36.560278 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:03:36.560278 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:03:36.560278 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 01 21:03:36.560278 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 01 21:03:36.560278 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 01 21:03:36.560278 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 01 21:03:36.560278 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:03:36.560278 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:03:36.560278 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:03:36.560278 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:03:36.561543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:03:36.561543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:03:36.561543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:03:36.561543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 21:03:36.561543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:03:36.561543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:03:36.561543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:03:36.561543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:03:36.561543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:03:36.561543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:03:36.561543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:03:36.561543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:03:36.561543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 01 21:03:36.561543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 01 21:03:36.561543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: a194adb825034310916ad4e14af36c1f. Oct 01 21:03:36.561543 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:03:36.563073 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2261/4450] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:03:36 2022] GET /identity/v3/services/a194adb825034310916ad4e14af36c1f => generated 113 bytes in 68 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:03:36.568435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f449a-60b2-4fd1-a72b-7567a3b35e8e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:36.580110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f449a-60b2-4fd1-a72b-7567a3b35e8e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:36.591014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64850d0f-a33b-48d7-8bce-a90e19c8a6f9 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fabd1d6030774f91af00089422dc0806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1045b9dc011e474086924aa42c3fa371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:36.591827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64850d0f-a33b-48d7-8bce-a90e19c8a6f9 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:36.592215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64850d0f-a33b-48d7-8bce-a90e19c8a6f9 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:36.592970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64850d0f-a33b-48d7-8bce-a90e19c8a6f9 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:36.594110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64850d0f-a33b-48d7-8bce-a90e19c8a6f9 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:36.596043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64850d0f-a33b-48d7-8bce-a90e19c8a6f9 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:36.617408 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2190/4451] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:36 2022] POST /identity/v3/services => generated 275 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:03:36.618721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18523624-37f7-4e5a-818f-72d362155d12 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:36.630607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18523624-37f7-4e5a-818f-72d362155d12 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:36.651426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccc049-094b-49b2-b1c6-5c8f4efda8f4 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fabd1d6030774f91af00089422dc0806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1045b9dc011e474086924aa42c3fa371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:36.652412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccc049-094b-49b2-b1c6-5c8f4efda8f4 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:36.652934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccc049-094b-49b2-b1c6-5c8f4efda8f4 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:36.653370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccc049-094b-49b2-b1c6-5c8f4efda8f4 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:36.654325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eccc049-094b-49b2-b1c6-5c8f4efda8f4 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:36.656467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eccc049-094b-49b2-b1c6-5c8f4efda8f4 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:36.678407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64850d0f-a33b-48d7-8bce-a90e19c8a6f9 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:36.679508 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2262/4452] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:36 2022] POST /identity/v3/services => generated 272 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:36.690029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64850d0f-a33b-48d7-8bce-a90e19c8a6f9 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:36.706261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad0d3c-cadb-46b2-87a8-97a7a4d899ca tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fabd1d6030774f91af00089422dc0806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1045b9dc011e474086924aa42c3fa371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:36.707485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad0d3c-cadb-46b2-87a8-97a7a4d899ca tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:36.708028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad0d3c-cadb-46b2-87a8-97a7a4d899ca tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:36.708028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad0d3c-cadb-46b2-87a8-97a7a4d899ca tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:36.708731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad0d3c-cadb-46b2-87a8-97a7a4d899ca tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:36.710398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad0d3c-cadb-46b2-87a8-97a7a4d899ca tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:36.734336 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2191/4453] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:36 2022] POST /identity/v3/services => generated 275 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:03:36.735336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccc049-094b-49b2-b1c6-5c8f4efda8f4 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:36.747829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccc049-094b-49b2-b1c6-5c8f4efda8f4 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:36.777272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b100eb8-6c55-4124-8883-8a02041678c7 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fabd1d6030774f91af00089422dc0806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1045b9dc011e474086924aa42c3fa371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:36.778017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b100eb8-6c55-4124-8883-8a02041678c7 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:36.778304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b100eb8-6c55-4124-8883-8a02041678c7 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:36.778613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b100eb8-6c55-4124-8883-8a02041678c7 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:36.779227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b100eb8-6c55-4124-8883-8a02041678c7 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:36.780450 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-4b100eb8-6c55-4124-8883-8a02041678c7 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_services": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:36.780450 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:03:36.780450 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:03:36.781623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b100eb8-6c55-4124-8883-8a02041678c7 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:36.792379 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2263/4454] 10.222.0.17 () {58 vars in 1131 bytes} [Sat Oct 1 21:03:36 2022] GET /identity/v3/services => generated 2977 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:03:36.796332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad0d3c-cadb-46b2-87a8-97a7a4d899ca tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:36.807797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad0d3c-cadb-46b2-87a8-97a7a4d899ca tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:36.813619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4aa9ca-c8c8-49cf-8390-b1752a4a0466 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fabd1d6030774f91af00089422dc0806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1045b9dc011e474086924aa42c3fa371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:36.814346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4aa9ca-c8c8-49cf-8390-b1752a4a0466 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:36.814733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4aa9ca-c8c8-49cf-8390-b1752a4a0466 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:36.815077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4aa9ca-c8c8-49cf-8390-b1752a4a0466 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:36.815875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4aa9ca-c8c8-49cf-8390-b1752a4a0466 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-2102204008) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:03:36.816339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4aa9ca-c8c8-49cf-8390-b1752a4a0466 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:36.817477 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-fb4aa9ca-c8c8-49cf-8390-b1752a4a0466 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_services": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:03:36.817477 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:03:36.817477 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:03:36.818666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4aa9ca-c8c8-49cf-8390-b1752a4a0466 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:36.825464 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2192/4455] 10.222.0.17 () {58 vars in 1222 bytes} [Sat Oct 1 21:03:36 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-2102204008 => generated 418 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:36.829736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b100eb8-6c55-4124-8883-8a02041678c7 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:36.843080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b100eb8-6c55-4124-8883-8a02041678c7 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:36.849509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e3663-b497-486b-9bb9-d89e90707d0e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fabd1d6030774f91af00089422dc0806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1045b9dc011e474086924aa42c3fa371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:36.850139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e3663-b497-486b-9bb9-d89e90707d0e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:36.850429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e3663-b497-486b-9bb9-d89e90707d0e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:36.850900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e3663-b497-486b-9bb9-d89e90707d0e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:36.851598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e3663-b497-486b-9bb9-d89e90707d0e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-310721246) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:03:36.851910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e3663-b497-486b-9bb9-d89e90707d0e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:36.853545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e3663-b497-486b-9bb9-d89e90707d0e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:36.863107 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2264/4456] 10.222.0.17 () {58 vars in 1220 bytes} [Sat Oct 1 21:03:36 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-310721246 => generated 414 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:36.863997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4aa9ca-c8c8-49cf-8390-b1752a4a0466 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:36.875891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4aa9ca-c8c8-49cf-8390-b1752a4a0466 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:36.881902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba48ac-d37a-4fd4-bfd0-3ec36b558b37 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fabd1d6030774f91af00089422dc0806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1045b9dc011e474086924aa42c3fa371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:36.883123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba48ac-d37a-4fd4-bfd0-3ec36b558b37 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:36.883628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba48ac-d37a-4fd4-bfd0-3ec36b558b37 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:36.885335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba48ac-d37a-4fd4-bfd0-3ec36b558b37 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:36.886246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba48ac-d37a-4fd4-bfd0-3ec36b558b37 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-1403860858) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:03:36.886799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba48ac-d37a-4fd4-bfd0-3ec36b558b37 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:36.888990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba48ac-d37a-4fd4-bfd0-3ec36b558b37 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:36.901502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e3663-b497-486b-9bb9-d89e90707d0e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:36.906674 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2193/4457] 10.222.0.17 () {58 vars in 1222 bytes} [Sat Oct 1 21:03:36 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-1403860858 => generated 418 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:03:36.908037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba48ac-d37a-4fd4-bfd0-3ec36b558b37 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:36.912249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9eba48ac-d37a-4fd4-bfd0-3ec36b558b37 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:36.913823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e3663-b497-486b-9bb9-d89e90707d0e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:36.919166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b232e9-edb0-471b-b2c8-6b20b2cd15e8 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fabd1d6030774f91af00089422dc0806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1045b9dc011e474086924aa42c3fa371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:36.919922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b232e9-edb0-471b-b2c8-6b20b2cd15e8 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:36.920310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b232e9-edb0-471b-b2c8-6b20b2cd15e8 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:36.920725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b232e9-edb0-471b-b2c8-6b20b2cd15e8 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] PATH_INFO: `/v3/services/3769cf5fac394955b61891f7b9fa5dc1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:36.921763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b232e9-edb0-471b-b2c8-6b20b2cd15e8 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorizing `identity:delete_service(service_id=3769cf5fac394955b61891f7b9fa5dc1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:36.923291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b232e9-edb0-471b-b2c8-6b20b2cd15e8 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:36.959013 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2265/4458] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:36 2022] DELETE /identity/v3/services/3769cf5fac394955b61891f7b9fa5dc1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:36.960154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b232e9-edb0-471b-b2c8-6b20b2cd15e8 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:36.977958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:INSERT=8,keystone:DELETE=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:36.983442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b232e9-edb0-471b-b2c8-6b20b2cd15e8 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:36.993349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cec0d6-b551-4f4a-afef-91ca3f5a4b8e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fabd1d6030774f91af00089422dc0806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1045b9dc011e474086924aa42c3fa371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:36.994664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cec0d6-b551-4f4a-afef-91ca3f5a4b8e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:36.995565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cec0d6-b551-4f4a-afef-91ca3f5a4b8e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:36.996234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cec0d6-b551-4f4a-afef-91ca3f5a4b8e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] PATH_INFO: `/v3/services/f071f49453c64469a62b07d758fa18f1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:36.997881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cec0d6-b551-4f4a-afef-91ca3f5a4b8e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorizing `identity:delete_service(service_id=f071f49453c64469a62b07d758fa18f1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:37.000627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cec0d6-b551-4f4a-afef-91ca3f5a4b8e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:37.047602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba48ac-d37a-4fd4-bfd0-3ec36b558b37 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:37.057332 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2266/4459] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:36 2022] DELETE /identity/v3/services/f071f49453c64469a62b07d758fa18f1 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:37.062295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cec0d6-b551-4f4a-afef-91ca3f5a4b8e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:37.070146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181dece-1dcb-4026-ae84-047d05dbd098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:37.070994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181dece-1dcb-4026-ae84-047d05dbd098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:37.071288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181dece-1dcb-4026-ae84-047d05dbd098 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:37.071568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181dece-1dcb-4026-ae84-047d05dbd098 admin admin] PATH_INFO: `/v3/users/6adf8eb2f77344119b68fe8a71f2f49e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:37.072811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181dece-1dcb-4026-ae84-047d05dbd098 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6adf8eb2f77344119b68fe8a71f2f49e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:37.075217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181dece-1dcb-4026-ae84-047d05dbd098 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:37.119027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cec0d6-b551-4f4a-afef-91ca3f5a4b8e tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:37.126965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019c4b4-d560-49b1-b5e9-e54f870e7c17 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fabd1d6030774f91af00089422dc0806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1045b9dc011e474086924aa42c3fa371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:37.127676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019c4b4-d560-49b1-b5e9-e54f870e7c17 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:37.127878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019c4b4-d560-49b1-b5e9-e54f870e7c17 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:37.128319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019c4b4-d560-49b1-b5e9-e54f870e7c17 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] PATH_INFO: `/v3/services/6fe0a84cc24c4727874406dff4de111b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:37.129423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6019c4b4-d560-49b1-b5e9-e54f870e7c17 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorizing `identity:delete_service(service_id=6fe0a84cc24c4727874406dff4de111b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:37.131573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6019c4b4-d560-49b1-b5e9-e54f870e7c17 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:37.147563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:UPDATE=1,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:37.167500 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2267/4460] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:37 2022] DELETE /identity/v3/services/6fe0a84cc24c4727874406dff4de111b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:37.172819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019c4b4-d560-49b1-b5e9-e54f870e7c17 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:37.181105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b181dece-1dcb-4026-ae84-047d05dbd098 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6adf8eb2f77344119b68fe8a71f2f49e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:37.185785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b181dece-1dcb-4026-ae84-047d05dbd098 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6adf8eb2f77344119b68fe8a71f2f49e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:37.186468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b181dece-1dcb-4026-ae84-047d05dbd098 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6adf8eb2f77344119b68fe8a71f2f49e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:37.197149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b181dece-1dcb-4026-ae84-047d05dbd098 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6adf8eb2f77344119b68fe8a71f2f49e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:37.215518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019c4b4-d560-49b1-b5e9-e54f870e7c17 tempest-ServicesTestJSON-1026112433 tempest-ServicesTestJSON-1026112433-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:37.217318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b181dece-1dcb-4026-ae84-047d05dbd098 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6adf8eb2f77344119b68fe8a71f2f49e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:37.222848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab874150-57b1-4071-a603-5220e7bc8f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:37.224012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab874150-57b1-4071-a603-5220e7bc8f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:37.224012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab874150-57b1-4071-a603-5220e7bc8f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:37.224012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab874150-57b1-4071-a603-5220e7bc8f91 admin admin] PATH_INFO: `/v3/users/acdb7b0598b3497bbd8d197c7e7a9ee3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:37.225097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab874150-57b1-4071-a603-5220e7bc8f91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acdb7b0598b3497bbd8d197c7e7a9ee3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:37.227037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab874150-57b1-4071-a603-5220e7bc8f91 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:37.231879 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2194/4461] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:36 2022] DELETE /identity/v3/users/6adf8eb2f77344119b68fe8a71f2f49e => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:37.235885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c9ba6-0592-4523-8bf6-49e946d8adb8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:37.236256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c9ba6-0592-4523-8bf6-49e946d8adb8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:37.236882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c9ba6-0592-4523-8bf6-49e946d8adb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:37.265755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-039c9ba6-0592-4523-8bf6-49e946d8adb8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:37.267255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-039c9ba6-0592-4523-8bf6-49e946d8adb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:37.297892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ab874150-57b1-4071-a603-5220e7bc8f91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acdb7b0598b3497bbd8d197c7e7a9ee3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:37.311104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ab874150-57b1-4071-a603-5220e7bc8f91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acdb7b0598b3497bbd8d197c7e7a9ee3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:37.330101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ab874150-57b1-4071-a603-5220e7bc8f91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acdb7b0598b3497bbd8d197c7e7a9ee3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:37.331053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ab874150-57b1-4071-a603-5220e7bc8f91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acdb7b0598b3497bbd8d197c7e7a9ee3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:37.345872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ab874150-57b1-4071-a603-5220e7bc8f91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acdb7b0598b3497bbd8d197c7e7a9ee3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:37.355809 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2268/4462] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:37 2022] DELETE /identity/v3/users/acdb7b0598b3497bbd8d197c7e7a9ee3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:37.360001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da244a40-a2a3-4f9d-b77e-0a5c9e588baa None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:37.360491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da244a40-a2a3-4f9d-b77e-0a5c9e588baa None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:37.360770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da244a40-a2a3-4f9d-b77e-0a5c9e588baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:37.365129 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2195/4463] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:03:37.376560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c9ba6-0592-4523-8bf6-49e946d8adb8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:37.379683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-039c9ba6-0592-4523-8bf6-49e946d8adb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:37.384970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-da244a40-a2a3-4f9d-b77e-0a5c9e588baa None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:37.386519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-da244a40-a2a3-4f9d-b77e-0a5c9e588baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:37.465435 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2269/4464] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:37.475893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da244a40-a2a3-4f9d-b77e-0a5c9e588baa None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:37.479774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c9ba6-0592-4523-8bf6-49e946d8adb8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:37.488699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:37.545263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da244a40-a2a3-4f9d-b77e-0a5c9e588baa None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:37.550336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bf5db-0dc8-477d-80cf-28c94c343115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:37.551705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bf5db-0dc8-477d-80cf-28c94c343115 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:37.552146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bf5db-0dc8-477d-80cf-28c94c343115 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:37.552605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bf5db-0dc8-477d-80cf-28c94c343115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:37.552884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d776f-f4ae-430a-9b14-ce92fcaf2f15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:37.553831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d776f-f4ae-430a-9b14-ce92fcaf2f15 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:37.554013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d776f-f4ae-430a-9b14-ce92fcaf2f15 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:37.554287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d776f-f4ae-430a-9b14-ce92fcaf2f15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:37.572331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d776f-f4ae-430a-9b14-ce92fcaf2f15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:37.574340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d776f-f4ae-430a-9b14-ce92fcaf2f15 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:37.579841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7bf5db-0dc8-477d-80cf-28c94c343115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:37.581283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7bf5db-0dc8-477d-80cf-28c94c343115 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:37.592726 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2270/4465] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:03:37.593104 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2196/4466] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:03:37.978695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bf5db-0dc8-477d-80cf-28c94c343115 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:37.981421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4c7bf5db-0dc8-477d-80cf-28c94c343115 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:37.997646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bf5db-0dc8-477d-80cf-28c94c343115 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:38.002774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d9874-0409-4499-a401-7e4b4374437f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:38.003631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d9874-0409-4499-a401-7e4b4374437f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:38.003979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d9874-0409-4499-a401-7e4b4374437f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:38.004324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d9874-0409-4499-a401-7e4b4374437f admin admin] PATH_INFO: `/v3/projects/4018320ded0c4162a623a5b5b8fd57d0` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:38.005591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d9874-0409-4499-a401-7e4b4374437f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4018320ded0c4162a623a5b5b8fd57d0)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:38.007742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d9874-0409-4499-a401-7e4b4374437f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:38.073058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cd1d9874-0409-4499-a401-7e4b4374437f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4018320ded0c4162a623a5b5b8fd57d0', 'request_id': 'req-cd1d9874-0409-4499-a401-7e4b4374437f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:38.080433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cd1d9874-0409-4499-a401-7e4b4374437f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4018320ded0c4162a623a5b5b8fd57d0', 'request_id': 'req-cd1d9874-0409-4499-a401-7e4b4374437f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:38.088589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cd1d9874-0409-4499-a401-7e4b4374437f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4018320ded0c4162a623a5b5b8fd57d0', 'request_id': 'req-cd1d9874-0409-4499-a401-7e4b4374437f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:38.104390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cd1d9874-0409-4499-a401-7e4b4374437f admin admin] The token cache is being invalidate because project 4018320ded0c4162a623a5b5b8fd57d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:38.104776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cd1d9874-0409-4499-a401-7e4b4374437f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:38.105350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cd1d9874-0409-4499-a401-7e4b4374437f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:38.107243 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2271/4467] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:37 2022] DELETE /identity/v3/projects/4018320ded0c4162a623a5b5b8fd57d0 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:38.111504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d776f-f4ae-430a-9b14-ce92fcaf2f15 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:38.117258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:38.123933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d776f-f4ae-430a-9b14-ce92fcaf2f15 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:38.237488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89493da-5744-4154-94b1-46a441685496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:38.239096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89493da-5744-4154-94b1-46a441685496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:38.239307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89493da-5744-4154-94b1-46a441685496 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:38.239553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89493da-5744-4154-94b1-46a441685496 admin admin] PATH_INFO: `/v3/users/fabd1d6030774f91af00089422dc0806` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:38.243763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89493da-5744-4154-94b1-46a441685496 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fabd1d6030774f91af00089422dc0806)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:38.245656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89493da-5744-4154-94b1-46a441685496 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:38.257612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d9874-0409-4499-a401-7e4b4374437f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:38.261642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cd1d9874-0409-4499-a401-7e4b4374437f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:38.335878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b89493da-5744-4154-94b1-46a441685496 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fabd1d6030774f91af00089422dc0806'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:38.337140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b89493da-5744-4154-94b1-46a441685496 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fabd1d6030774f91af00089422dc0806'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:38.337930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b89493da-5744-4154-94b1-46a441685496 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fabd1d6030774f91af00089422dc0806'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:38.349273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d9874-0409-4499-a401-7e4b4374437f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:38.352339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b89493da-5744-4154-94b1-46a441685496 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fabd1d6030774f91af00089422dc0806'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:38.357139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666284e1-0dc7-46ec-aaf1-d552b67d5f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:38.357960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666284e1-0dc7-46ec-aaf1-d552b67d5f8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:38.358476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666284e1-0dc7-46ec-aaf1-d552b67d5f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:38.358966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666284e1-0dc7-46ec-aaf1-d552b67d5f8d admin admin] PATH_INFO: `/v3/projects/02dfb544d3a54a6db8114885059c4dec` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:38.360592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666284e1-0dc7-46ec-aaf1-d552b67d5f8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=02dfb544d3a54a6db8114885059c4dec)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:38.362753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666284e1-0dc7-46ec-aaf1-d552b67d5f8d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:38.375516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b89493da-5744-4154-94b1-46a441685496 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fabd1d6030774f91af00089422dc0806'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:38.400239 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2197/4468] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:38 2022] DELETE /identity/v3/users/fabd1d6030774f91af00089422dc0806 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:38.430189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-666284e1-0dc7-46ec-aaf1-d552b67d5f8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02dfb544d3a54a6db8114885059c4dec', 'request_id': 'req-666284e1-0dc7-46ec-aaf1-d552b67d5f8d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:38.439302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-666284e1-0dc7-46ec-aaf1-d552b67d5f8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02dfb544d3a54a6db8114885059c4dec', 'request_id': 'req-666284e1-0dc7-46ec-aaf1-d552b67d5f8d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:38.446169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-666284e1-0dc7-46ec-aaf1-d552b67d5f8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02dfb544d3a54a6db8114885059c4dec', 'request_id': 'req-666284e1-0dc7-46ec-aaf1-d552b67d5f8d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:38.449310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:38.467549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-666284e1-0dc7-46ec-aaf1-d552b67d5f8d admin admin] The token cache is being invalidate because project 02dfb544d3a54a6db8114885059c4dec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:38.467915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-666284e1-0dc7-46ec-aaf1-d552b67d5f8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:38.468578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-666284e1-0dc7-46ec-aaf1-d552b67d5f8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:38.470611 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2272/4469] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:38 2022] DELETE /identity/v3/projects/02dfb544d3a54a6db8114885059c4dec => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:38.477395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9de1f-f7f8-4ad9-941a-b6733d65456e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:38.477562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9de1f-f7f8-4ad9-941a-b6733d65456e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:38.477841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9de1f-f7f8-4ad9-941a-b6733d65456e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:38.516176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a6c9de1f-f7f8-4ad9-941a-b6733d65456e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:38.517350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a6c9de1f-f7f8-4ad9-941a-b6733d65456e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:38.593485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:38.618514 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2198/4470] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:38.626753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25da86-6d70-43bd-a0d7-99db983679c6 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:38.626753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25da86-6d70-43bd-a0d7-99db983679c6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:38.626753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25da86-6d70-43bd-a0d7-99db983679c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:38.649398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fc25da86-6d70-43bd-a0d7-99db983679c6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:38.650728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fc25da86-6d70-43bd-a0d7-99db983679c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:38.673667 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2273/4471] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:38.679827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9de1f-f7f8-4ad9-941a-b6733d65456e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:38.693891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9de1f-f7f8-4ad9-941a-b6733d65456e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:38.702126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5eb53-8546-419e-9209-2e4a60135015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:38.703106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5eb53-8546-419e-9209-2e4a60135015 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:38.703494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5eb53-8546-419e-9209-2e4a60135015 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:38.703926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5eb53-8546-419e-9209-2e4a60135015 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:38.704701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5eb53-8546-419e-9209-2e4a60135015 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:03:38.705717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5eb53-8546-419e-9209-2e4a60135015 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:38.707673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5eb53-8546-419e-9209-2e4a60135015 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:38.743848 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2199/4472] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:38.748106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc25da86-6d70-43bd-a0d7-99db983679c6 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:38.758766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc25da86-6d70-43bd-a0d7-99db983679c6 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:38.765699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a0e1c-35bd-4aa2-a642-ffcd0592b4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:38.766321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a0e1c-35bd-4aa2-a642-ffcd0592b4f6 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:38.766639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a0e1c-35bd-4aa2-a642-ffcd0592b4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:38.766917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a0e1c-35bd-4aa2-a642-ffcd0592b4f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:38.768194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a0e1c-35bd-4aa2-a642-ffcd0592b4f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:38.769844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a0e1c-35bd-4aa2-a642-ffcd0592b4f6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:38.827168 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2274/4473] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:38 2022] POST /identity/v3/projects => generated 359 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:03:38.831544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5eb53-8546-419e-9209-2e4a60135015 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:38.846436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5eb53-8546-419e-9209-2e4a60135015 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:38.940747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74dd754-18c8-430b-ad49-384c7ec32c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:38.941463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dd754-18c8-430b-ad49-384c7ec32c8f admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:38.941852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dd754-18c8-430b-ad49-384c7ec32c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:38.942005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dd754-18c8-430b-ad49-384c7ec32c8f admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:38.942794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74dd754-18c8-430b-ad49-384c7ec32c8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:38.944994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74dd754-18c8-430b-ad49-384c7ec32c8f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:38.952248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a0e1c-35bd-4aa2-a642-ffcd0592b4f6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:38.954796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-aa2a0e1c-35bd-4aa2-a642-ffcd0592b4f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:38.974421 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2200/4474] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:03:38 2022] POST /identity/v3/users => generated 280 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:38.978261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74dd754-18c8-430b-ad49-384c7ec32c8f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:38.979685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a0e1c-35bd-4aa2-a642-ffcd0592b4f6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:38.986559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2edf7f6-f3a6-4938-b76d-5e50f716c3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:38.987976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2edf7f6-f3a6-4938-b76d-5e50f716c3ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:38.988289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2edf7f6-f3a6-4938-b76d-5e50f716c3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:38.988558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2edf7f6-f3a6-4938-b76d-5e50f716c3ad admin admin] PATH_INFO: `/v3/projects/1045b9dc011e474086924aa42c3fa371` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:38.990930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2edf7f6-f3a6-4938-b76d-5e50f716c3ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=1045b9dc011e474086924aa42c3fa371)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:38.993178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2edf7f6-f3a6-4938-b76d-5e50f716c3ad admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:39.004118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74dd754-18c8-430b-ad49-384c7ec32c8f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:39.033896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8746266-4f14-431c-8cc4-a4854a2501b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:39.035466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8746266-4f14-431c-8cc4-a4854a2501b1 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:39.035725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8746266-4f14-431c-8cc4-a4854a2501b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:39.036113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8746266-4f14-431c-8cc4-a4854a2501b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:39.036871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8746266-4f14-431c-8cc4-a4854a2501b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:39.038869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8746266-4f14-431c-8cc4-a4854a2501b1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:39.067490 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2201/4475] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:03:39.070944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8746266-4f14-431c-8cc4-a4854a2501b1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:39.075682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e2edf7f6-f3a6-4938-b76d-5e50f716c3ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1045b9dc011e474086924aa42c3fa371', 'request_id': 'req-e2edf7f6-f3a6-4938-b76d-5e50f716c3ad'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:39.085485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8746266-4f14-431c-8cc4-a4854a2501b1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:39.086437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e2edf7f6-f3a6-4938-b76d-5e50f716c3ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1045b9dc011e474086924aa42c3fa371', 'request_id': 'req-e2edf7f6-f3a6-4938-b76d-5e50f716c3ad'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:39.094242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e2edf7f6-f3a6-4938-b76d-5e50f716c3ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1045b9dc011e474086924aa42c3fa371', 'request_id': 'req-e2edf7f6-f3a6-4938-b76d-5e50f716c3ad'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:39.117210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e2edf7f6-f3a6-4938-b76d-5e50f716c3ad admin admin] The token cache is being invalidate because project 1045b9dc011e474086924aa42c3fa371 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:39.117901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e2edf7f6-f3a6-4938-b76d-5e50f716c3ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:39.119725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e2edf7f6-f3a6-4938-b76d-5e50f716c3ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:39.122619 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2275/4476] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:38 2022] DELETE /identity/v3/projects/1045b9dc011e474086924aa42c3fa371 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:39.127393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076e2b3-80ea-45f6-892d-78b64be110fd None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:39.128063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076e2b3-80ea-45f6-892d-78b64be110fd None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:39.129236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076e2b3-80ea-45f6-892d-78b64be110fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:39.155000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1076e2b3-80ea-45f6-892d-78b64be110fd None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:39.156383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1076e2b3-80ea-45f6-892d-78b64be110fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:39.180376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d87d4-0bb3-434f-a65d-943e6ab4c489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:39.181900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d87d4-0bb3-434f-a65d-943e6ab4c489 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:39.182989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d87d4-0bb3-434f-a65d-943e6ab4c489 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:39.184662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d87d4-0bb3-434f-a65d-943e6ab4c489 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:39.185798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451d87d4-0bb3-434f-a65d-943e6ab4c489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:39.189409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451d87d4-0bb3-434f-a65d-943e6ab4c489 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:39.203948 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2202/4477] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:39.207979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d87d4-0bb3-434f-a65d-943e6ab4c489 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:39.209861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-451d87d4-0bb3-434f-a65d-943e6ab4c489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:39.224217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d87d4-0bb3-434f-a65d-943e6ab4c489 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:39.230192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b182203-4244-442e-8d6e-c23f225e71af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:39.231004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b182203-4244-442e-8d6e-c23f225e71af admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:39.231292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b182203-4244-442e-8d6e-c23f225e71af admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:39.231564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b182203-4244-442e-8d6e-c23f225e71af admin admin] PATH_INFO: `/v3/projects/e63f74bcb25e44e581212a829ae7da58/users/8eb0d60068ff4776a7d79d3b891bb9a1/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:39.243963 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2276/4478] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:03:39.251538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59102f46-729a-4c72-b8db-56f2542c831e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:39.252363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59102f46-729a-4c72-b8db-56f2542c831e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:39.253207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59102f46-729a-4c72-b8db-56f2542c831e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:39.259209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b182203-4244-442e-8d6e-c23f225e71af admin admin] RBAC: Authorizing `identity:create_grant(project_id=e63f74bcb25e44e581212a829ae7da58, user_id=8eb0d60068ff4776a7d79d3b891bb9a1, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:39.262464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b182203-4244-442e-8d6e-c23f225e71af admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:39.280266 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2203/4479] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:39 2022] PUT /identity/v3/projects/e63f74bcb25e44e581212a829ae7da58/users/8eb0d60068ff4776a7d79d3b891bb9a1/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:39.287580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-59102f46-729a-4c72-b8db-56f2542c831e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:39.289290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-59102f46-729a-4c72-b8db-56f2542c831e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:39.292071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663b735-1ed2-4c22-a813-4a1fd004138b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:39.292530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663b735-1ed2-4c22-a813-4a1fd004138b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:39.292958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663b735-1ed2-4c22-a813-4a1fd004138b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:39.340522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e663b735-1ed2-4c22-a813-4a1fd004138b None None] MFA Rules not processed for user `8eb0d60068ff4776a7d79d3b891bb9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:39.341763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e663b735-1ed2-4c22-a813-4a1fd004138b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:39.352667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:39.380725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:39.384569 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2277/4480] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:39.389602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59102f46-729a-4c72-b8db-56f2542c831e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:39.421330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59102f46-729a-4c72-b8db-56f2542c831e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:39.429778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3bc907-91e2-4a23-8f32-21217d6db417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:39.430723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bc907-91e2-4a23-8f32-21217d6db417 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:39.431540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bc907-91e2-4a23-8f32-21217d6db417 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:39.433551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bc907-91e2-4a23-8f32-21217d6db417 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:39.435380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3bc907-91e2-4a23-8f32-21217d6db417 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:03:39.436035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3bc907-91e2-4a23-8f32-21217d6db417 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:39.438471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3bc907-91e2-4a23-8f32-21217d6db417 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:39.459524 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2278/4481] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:39.464340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3bc907-91e2-4a23-8f32-21217d6db417 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:39.470516 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2204/4482] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:39 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:39.482452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3bc907-91e2-4a23-8f32-21217d6db417 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:39.487390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b28cc-a2db-4cf4-89c2-1865ba3c44a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:39.488100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b28cc-a2db-4cf4-89c2-1865ba3c44a5 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:39.488825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b28cc-a2db-4cf4-89c2-1865ba3c44a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:39.489203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b28cc-a2db-4cf4-89c2-1865ba3c44a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:39.490062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b28cc-a2db-4cf4-89c2-1865ba3c44a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:39.492165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b28cc-a2db-4cf4-89c2-1865ba3c44a5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:39.498607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663b735-1ed2-4c22-a813-4a1fd004138b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:39.500560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e663b735-1ed2-4c22-a813-4a1fd004138b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:39.550952 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2279/4483] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:39 2022] POST /identity/v3/projects => generated 375 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:39.552613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b28cc-a2db-4cf4-89c2-1865ba3c44a5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:39.605084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663b735-1ed2-4c22-a813-4a1fd004138b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:39.670004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b28cc-a2db-4cf4-89c2-1865ba3c44a5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:39.677555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c088f9-6bab-4e27-b209-7d06731f9db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:39.678579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c088f9-6bab-4e27-b209-7d06731f9db1 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:39.679030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c088f9-6bab-4e27-b209-7d06731f9db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:39.679494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c088f9-6bab-4e27-b209-7d06731f9db1 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:39.680569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c088f9-6bab-4e27-b209-7d06731f9db1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:39.683289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c088f9-6bab-4e27-b209-7d06731f9db1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:39.685760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc0cbf-565e-4a2d-9f3f-a6b31175f5b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:39.686478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc0cbf-565e-4a2d-9f3f-a6b31175f5b9 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:39.686898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc0cbf-565e-4a2d-9f3f-a6b31175f5b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:39.687162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc0cbf-565e-4a2d-9f3f-a6b31175f5b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:39.721409 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2280/4484] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:03:39 2022] POST /identity/v3/users => generated 288 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:39.728052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c088f9-6bab-4e27-b209-7d06731f9db1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:39.744672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c088f9-6bab-4e27-b209-7d06731f9db1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:39.750741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4faac-9e20-485f-a3a1-c694001542e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:39.752088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4faac-9e20-485f-a3a1-c694001542e2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:39.752542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4faac-9e20-485f-a3a1-c694001542e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:39.752981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4faac-9e20-485f-a3a1-c694001542e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:39.753969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4faac-9e20-485f-a3a1-c694001542e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:39.757240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4faac-9e20-485f-a3a1-c694001542e2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:39.782761 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2281/4485] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:39.787777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4faac-9e20-485f-a3a1-c694001542e2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:39.793640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc0cbf-565e-4a2d-9f3f-a6b31175f5b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:39.795629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc0cbf-565e-4a2d-9f3f-a6b31175f5b9 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:39.808669 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2205/4486] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:39 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:03:39.810714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4faac-9e20-485f-a3a1-c694001542e2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:39.818208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df1ef4-1eb4-4716-8b48-43781d47239d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:39.818971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df1ef4-1eb4-4716-8b48-43781d47239d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:39.819558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df1ef4-1eb4-4716-8b48-43781d47239d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:39.821349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df1ef4-1eb4-4716-8b48-43781d47239d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:39.823981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df1ef4-1eb4-4716-8b48-43781d47239d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:39.827780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df1ef4-1eb4-4716-8b48-43781d47239d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:39.850132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc0cbf-565e-4a2d-9f3f-a6b31175f5b9 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:39.850999 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2282/4487] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:39.863486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc0cbf-565e-4a2d-9f3f-a6b31175f5b9 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:39.868692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b048a-a095-4981-811a-43ecc4d8fb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:39.869484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b048a-a095-4981-811a-43ecc4d8fb48 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:39.869863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b048a-a095-4981-811a-43ecc4d8fb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:39.870235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b048a-a095-4981-811a-43ecc4d8fb48 admin admin] PATH_INFO: `/v3/projects/1a6d0077ff9a4b83b7489aeb8f1875b4/users/bc185b0859084c90905a8b109115e92f/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:39.889257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b048a-a095-4981-811a-43ecc4d8fb48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a6d0077ff9a4b83b7489aeb8f1875b4, user_id=bc185b0859084c90905a8b109115e92f, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:39.892434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b048a-a095-4981-811a-43ecc4d8fb48 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:39.923812 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2206/4488] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:39 2022] PUT /identity/v3/projects/1a6d0077ff9a4b83b7489aeb8f1875b4/users/bc185b0859084c90905a8b109115e92f/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:39.925055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454a42a-ad99-4647-947b-be7d5e2ad65d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:39.926048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454a42a-ad99-4647-947b-be7d5e2ad65d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:39.926479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454a42a-ad99-4647-947b-be7d5e2ad65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:39.971869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5454a42a-ad99-4647-947b-be7d5e2ad65d None None] MFA Rules not processed for user `bc185b0859084c90905a8b109115e92f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:39.973556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5454a42a-ad99-4647-947b-be7d5e2ad65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:40.070664 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2283/4489] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:40.071240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b048a-a095-4981-811a-43ecc4d8fb48 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:40.085211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b048a-a095-4981-811a-43ecc4d8fb48 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:40.090601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba270f6-1ab9-4dfe-b80b-948f15688342 tempest-IdentityCatalogTest-1314639703 tempest-IdentityCatalogTest-1314639703-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc185b0859084c90905a8b109115e92f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a6d0077ff9a4b83b7489aeb8f1875b4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:40.091288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba270f6-1ab9-4dfe-b80b-948f15688342 tempest-IdentityCatalogTest-1314639703 tempest-IdentityCatalogTest-1314639703-project] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:40.091642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba270f6-1ab9-4dfe-b80b-948f15688342 tempest-IdentityCatalogTest-1314639703 tempest-IdentityCatalogTest-1314639703-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:40.091979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba270f6-1ab9-4dfe-b80b-948f15688342 tempest-IdentityCatalogTest-1314639703 tempest-IdentityCatalogTest-1314639703-project] PATH_INFO: `/v3/auth/catalog` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:40.092615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba270f6-1ab9-4dfe-b80b-948f15688342 tempest-IdentityCatalogTest-1314639703 tempest-IdentityCatalogTest-1314639703-project] RBAC: Authorizing `identity:get_auth_catalog()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:40.093905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba270f6-1ab9-4dfe-b80b-948f15688342 tempest-IdentityCatalogTest-1314639703 tempest-IdentityCatalogTest-1314639703-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:40.100427 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2207/4490] 10.222.0.17 () {58 vars in 1143 bytes} [Sat Oct 1 21:03:40 2022] GET /identity/v3/auth/catalog => generated 2658 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:40.101615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454a42a-ad99-4647-947b-be7d5e2ad65d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:40.116307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:40.182572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454a42a-ad99-4647-947b-be7d5e2ad65d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:40.187735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b55cf-2ebe-4fd3-bf18-d1be5e1cf847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:40.188382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b55cf-2ebe-4fd3-bf18-d1be5e1cf847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:40.188893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b55cf-2ebe-4fd3-bf18-d1be5e1cf847 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:40.189049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b55cf-2ebe-4fd3-bf18-d1be5e1cf847 admin admin] PATH_INFO: `/v3/users/bc185b0859084c90905a8b109115e92f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:40.190473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b55cf-2ebe-4fd3-bf18-d1be5e1cf847 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc185b0859084c90905a8b109115e92f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:40.192997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b55cf-2ebe-4fd3-bf18-d1be5e1cf847 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:40.254026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-299b55cf-2ebe-4fd3-bf18-d1be5e1cf847 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc185b0859084c90905a8b109115e92f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:40.266279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-299b55cf-2ebe-4fd3-bf18-d1be5e1cf847 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc185b0859084c90905a8b109115e92f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:40.282527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-299b55cf-2ebe-4fd3-bf18-d1be5e1cf847 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc185b0859084c90905a8b109115e92f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:40.283465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-299b55cf-2ebe-4fd3-bf18-d1be5e1cf847 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc185b0859084c90905a8b109115e92f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:40.295029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-299b55cf-2ebe-4fd3-bf18-d1be5e1cf847 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc185b0859084c90905a8b109115e92f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:40.306525 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2284/4491] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:40 2022] DELETE /identity/v3/users/bc185b0859084c90905a8b109115e92f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:40.311098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa104918-7209-4d43-bd0a-92781748050c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:40.311616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa104918-7209-4d43-bd0a-92781748050c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:40.311616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa104918-7209-4d43-bd0a-92781748050c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:40.325616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:40.347040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fa104918-7209-4d43-bd0a-92781748050c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:40.348655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fa104918-7209-4d43-bd0a-92781748050c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:40.431403 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2208/4492] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:40.451899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b55cf-2ebe-4fd3-bf18-d1be5e1cf847 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:40.459738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-299b55cf-2ebe-4fd3-bf18-d1be5e1cf847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:40.533315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b55cf-2ebe-4fd3-bf18-d1be5e1cf847 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:40.538240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea183fe1-b021-43cd-8be9-e5da2b95e02d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:40.538988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea183fe1-b021-43cd-8be9-e5da2b95e02d service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:40.539351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea183fe1-b021-43cd-8be9-e5da2b95e02d service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:40.539702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea183fe1-b021-43cd-8be9-e5da2b95e02d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:40.551077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea183fe1-b021-43cd-8be9-e5da2b95e02d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:40.552550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea183fe1-b021-43cd-8be9-e5da2b95e02d service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:40.563965 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2285/4493] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:03:41.271558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa104918-7209-4d43-bd0a-92781748050c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:41.273879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fa104918-7209-4d43-bd0a-92781748050c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:41.302923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa104918-7209-4d43-bd0a-92781748050c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:41.311963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb7e83-b9e7-419c-88c4-3490f3683cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:41.312832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb7e83-b9e7-419c-88c4-3490f3683cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:41.313318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb7e83-b9e7-419c-88c4-3490f3683cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:41.313665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb7e83-b9e7-419c-88c4-3490f3683cb9 admin admin] PATH_INFO: `/v3/projects/1a6d0077ff9a4b83b7489aeb8f1875b4` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:41.315186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb7e83-b9e7-419c-88c4-3490f3683cb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a6d0077ff9a4b83b7489aeb8f1875b4)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:41.319145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb7e83-b9e7-419c-88c4-3490f3683cb9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:41.393647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89fb7e83-b9e7-419c-88c4-3490f3683cb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a6d0077ff9a4b83b7489aeb8f1875b4', 'request_id': 'req-89fb7e83-b9e7-419c-88c4-3490f3683cb9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:41.402277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89fb7e83-b9e7-419c-88c4-3490f3683cb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a6d0077ff9a4b83b7489aeb8f1875b4', 'request_id': 'req-89fb7e83-b9e7-419c-88c4-3490f3683cb9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:41.414850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89fb7e83-b9e7-419c-88c4-3490f3683cb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a6d0077ff9a4b83b7489aeb8f1875b4', 'request_id': 'req-89fb7e83-b9e7-419c-88c4-3490f3683cb9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:41.427209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89fb7e83-b9e7-419c-88c4-3490f3683cb9 admin admin] The token cache is being invalidate because project 1a6d0077ff9a4b83b7489aeb8f1875b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:41.427496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89fb7e83-b9e7-419c-88c4-3490f3683cb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:41.428377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89fb7e83-b9e7-419c-88c4-3490f3683cb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:41.429880 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2209/4494] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:41 2022] DELETE /identity/v3/projects/1a6d0077ff9a4b83b7489aeb8f1875b4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:41.435711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb8951-bcd7-4183-8d37-17743be1217a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:41.436561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb8951-bcd7-4183-8d37-17743be1217a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:41.436846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb8951-bcd7-4183-8d37-17743be1217a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:41.459041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1cfb8951-bcd7-4183-8d37-17743be1217a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:41.460687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1cfb8951-bcd7-4183-8d37-17743be1217a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:41.499857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:41.590538 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2286/4495] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:41.599269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557eed2-c3bf-4e4b-965b-e4529f7a2a75 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:41.599564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557eed2-c3bf-4e4b-965b-e4529f7a2a75 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:41.600051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557eed2-c3bf-4e4b-965b-e4529f7a2a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:41.626013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8557eed2-c3bf-4e4b-965b-e4529f7a2a75 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:41.627278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8557eed2-c3bf-4e4b-965b-e4529f7a2a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:41.635813 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2210/4496] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:41.642201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb8951-bcd7-4183-8d37-17743be1217a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:41.656987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb8951-bcd7-4183-8d37-17743be1217a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:41.662009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086dfb2a-4cef-42f4-9042-2ceafde59866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:41.663384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086dfb2a-4cef-42f4-9042-2ceafde59866 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:41.663704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086dfb2a-4cef-42f4-9042-2ceafde59866 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:41.663953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086dfb2a-4cef-42f4-9042-2ceafde59866 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:41.664620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086dfb2a-4cef-42f4-9042-2ceafde59866 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:03:41.664916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086dfb2a-4cef-42f4-9042-2ceafde59866 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:41.666605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086dfb2a-4cef-42f4-9042-2ceafde59866 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:41.683174 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2287/4497] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:03:41.687596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557eed2-c3bf-4e4b-965b-e4529f7a2a75 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:41.708106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557eed2-c3bf-4e4b-965b-e4529f7a2a75 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:41.718257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c3d22-ca59-4b3b-b984-d4f3d7ac3d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:41.719283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c3d22-ca59-4b3b-b984-d4f3d7ac3d03 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:41.720541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c3d22-ca59-4b3b-b984-d4f3d7ac3d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:41.721247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c3d22-ca59-4b3b-b984-d4f3d7ac3d03 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:41.722341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c3d22-ca59-4b3b-b984-d4f3d7ac3d03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:41.724954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c3d22-ca59-4b3b-b984-d4f3d7ac3d03 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:41.735721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d25c12-990e-407c-9cb1-7366cc1e9cf1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:41.736144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d25c12-990e-407c-9cb1-7366cc1e9cf1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:41.736521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d25c12-990e-407c-9cb1-7366cc1e9cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:41.758524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-78d25c12-990e-407c-9cb1-7366cc1e9cf1 None None] MFA Rules not processed for user `6bda28994cb14f92b014a147a7e74476`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:41.758909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-78d25c12-990e-407c-9cb1-7366cc1e9cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:41.761110 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2211/4498] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:41 2022] POST /identity/v3/projects => generated 361 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:41.765412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c3d22-ca59-4b3b-b984-d4f3d7ac3d03 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:41.836642 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2288/4499] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:03:41.847860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c3d22-ca59-4b3b-b984-d4f3d7ac3d03 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:41.852858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328405c-ae1d-4a07-9d0f-02b8caa33db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:41.853476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328405c-ae1d-4a07-9d0f-02b8caa33db5 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:41.853752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328405c-ae1d-4a07-9d0f-02b8caa33db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:41.854218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328405c-ae1d-4a07-9d0f-02b8caa33db5 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:41.854933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328405c-ae1d-4a07-9d0f-02b8caa33db5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:41.856870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328405c-ae1d-4a07-9d0f-02b8caa33db5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:41.866907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:41.881599 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2212/4500] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:03:41 2022] POST /identity/v3/users => generated 281 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:03:41.885565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d25c12-990e-407c-9cb1-7366cc1e9cf1 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:41.896828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d25c12-990e-407c-9cb1-7366cc1e9cf1 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:41.901985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79f550-7120-4828-8a6a-16a7a209ef43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:41.902656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79f550-7120-4828-8a6a-16a7a209ef43 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:41.902942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79f550-7120-4828-8a6a-16a7a209ef43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:41.903262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79f550-7120-4828-8a6a-16a7a209ef43 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:41.903860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79f550-7120-4828-8a6a-16a7a209ef43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:41.905445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79f550-7120-4828-8a6a-16a7a209ef43 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:41.926825 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2289/4501] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:41.926927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328405c-ae1d-4a07-9d0f-02b8caa33db5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:41.937298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328405c-ae1d-4a07-9d0f-02b8caa33db5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:41.945352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650680d5-e27d-4dca-be14-1946d9b8263e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:41.947179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650680d5-e27d-4dca-be14-1946d9b8263e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:41.947462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650680d5-e27d-4dca-be14-1946d9b8263e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:41.947802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650680d5-e27d-4dca-be14-1946d9b8263e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:41.950754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650680d5-e27d-4dca-be14-1946d9b8263e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:41.953052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650680d5-e27d-4dca-be14-1946d9b8263e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:41.968962 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2213/4502] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:41.973855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79f550-7120-4828-8a6a-16a7a209ef43 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:41.986856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79f550-7120-4828-8a6a-16a7a209ef43 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:41.994761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9cb7f-11ec-42fd-bcd4-99595b7cbdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:41.995448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9cb7f-11ec-42fd-bcd4-99595b7cbdc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:41.995731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9cb7f-11ec-42fd-bcd4-99595b7cbdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:41.996008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9cb7f-11ec-42fd-bcd4-99595b7cbdc7 admin admin] PATH_INFO: `/v3/projects/a54ef6ce1fee4fd8a9400c8ade186ad0/users/f445b5efc6574c2e93b3495a9d3bd874/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:42.012921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba9cb7f-11ec-42fd-bcd4-99595b7cbdc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a54ef6ce1fee4fd8a9400c8ade186ad0, user_id=f445b5efc6574c2e93b3495a9d3bd874, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:42.015617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba9cb7f-11ec-42fd-bcd4-99595b7cbdc7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:42.034245 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2290/4503] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:41 2022] PUT /identity/v3/projects/a54ef6ce1fee4fd8a9400c8ade186ad0/users/f445b5efc6574c2e93b3495a9d3bd874/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:42.047210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297828b-3780-4197-b2ea-50de3568306b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:42.047502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297828b-3780-4197-b2ea-50de3568306b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:42.048221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297828b-3780-4197-b2ea-50de3568306b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:42.084502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3297828b-3780-4197-b2ea-50de3568306b None None] MFA Rules not processed for user `f445b5efc6574c2e93b3495a9d3bd874`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:42.086246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3297828b-3780-4197-b2ea-50de3568306b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:42.163335 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2214/4504] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:42.168871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afe433-a3c7-4e43-8f8c-151de19c7e7c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:42.169159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afe433-a3c7-4e43-8f8c-151de19c7e7c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:42.169437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afe433-a3c7-4e43-8f8c-151de19c7e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:42.187910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-30afe433-a3c7-4e43-8f8c-151de19c7e7c None None] MFA Rules not processed for user `f445b5efc6574c2e93b3495a9d3bd874`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:42.189102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-30afe433-a3c7-4e43-8f8c-151de19c7e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:42.192744 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2291/4505] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:42 2022] POST /identity/v3/auth/tokens => generated 346 bytes in 25 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 21:03:42.198724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4442fb4-9544-4e99-86b3-a9ac9522a91f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:42.199019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4442fb4-9544-4e99-86b3-a9ac9522a91f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:42.199298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4442fb4-9544-4e99-86b3-a9ac9522a91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:42.218588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c4442fb4-9544-4e99-86b3-a9ac9522a91f None None] MFA Rules not processed for user `f445b5efc6574c2e93b3495a9d3bd874`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:42.219794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c4442fb4-9544-4e99-86b3-a9ac9522a91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:42.223517 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2215/4506] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:42 2022] POST /identity/v3/auth/tokens => generated 346 bytes in 26 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 21:03:42.227827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30afe433-a3c7-4e43-8f8c-151de19c7e7c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:42.240120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30afe433-a3c7-4e43-8f8c-151de19c7e7c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:42.244794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332beee-af05-42fa-b29f-e0dcdd072914 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f445b5efc6574c2e93b3495a9d3bd874', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a54ef6ce1fee4fd8a9400c8ade186ad0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:42.245413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332beee-af05-42fa-b29f-e0dcdd072914 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] REQUEST_METHOD: `HEAD` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:42.245721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332beee-af05-42fa-b29f-e0dcdd072914 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:42.245999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332beee-af05-42fa-b29f-e0dcdd072914 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:42.253039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c332beee-af05-42fa-b29f-e0dcdd072914 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] RBAC: Authorizing `identity:check_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:42.255366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c332beee-af05-42fa-b29f-e0dcdd072914 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:42.265342 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2292/4507] 10.222.0.17 () {56 vars in 1264 bytes} [Sat Oct 1 21:03:42 2022] HEAD /identity/v3/auth/tokens => generated 0 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 358 bytes (0 switches on core 0) Oct 01 21:03:42.266618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4442fb4-9544-4e99-86b3-a9ac9522a91f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:42.280858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4442fb4-9544-4e99-86b3-a9ac9522a91f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:42.288284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4472c-15c4-4491-bd7a-d64f980dcb93 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f445b5efc6574c2e93b3495a9d3bd874', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a54ef6ce1fee4fd8a9400c8ade186ad0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:42.289265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4472c-15c4-4491-bd7a-d64f980dcb93 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:42.289766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4472c-15c4-4491-bd7a-d64f980dcb93 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:42.290258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4472c-15c4-4491-bd7a-d64f980dcb93 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:42.298921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4472c-15c4-4491-bd7a-d64f980dcb93 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:42.301071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4472c-15c4-4491-bd7a-d64f980dcb93 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:42.319702 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2216/4508] 10.222.0.17 () {56 vars in 1263 bytes} [Sat Oct 1 21:03:42 2022] GET /identity/v3/auth/tokens => generated 346 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 21:03:42.322898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332beee-af05-42fa-b29f-e0dcdd072914 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:42.333499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332beee-af05-42fa-b29f-e0dcdd072914 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:42.337921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d282f7-9de4-4d4b-955a-de26459778b7 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f445b5efc6574c2e93b3495a9d3bd874', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a54ef6ce1fee4fd8a9400c8ade186ad0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:42.338563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d282f7-9de4-4d4b-955a-de26459778b7 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:42.338850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d282f7-9de4-4d4b-955a-de26459778b7 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:42.339127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d282f7-9de4-4d4b-955a-de26459778b7 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:42.347395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d282f7-9de4-4d4b-955a-de26459778b7 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] RBAC: Authorizing `identity:revoke_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:42.349015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d282f7-9de4-4d4b-955a-de26459778b7 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:42.367738 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2293/4509] 10.222.0.17 () {56 vars in 1266 bytes} [Sat Oct 1 21:03:42 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:42.371814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4472c-15c4-4491-bd7a-d64f980dcb93 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:42.383159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4472c-15c4-4491-bd7a-d64f980dcb93 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:42.387715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2f7c4-4111-4093-8a37-40195c46f5e4 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f445b5efc6574c2e93b3495a9d3bd874', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a54ef6ce1fee4fd8a9400c8ade186ad0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:42.388315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2f7c4-4111-4093-8a37-40195c46f5e4 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] REQUEST_METHOD: `HEAD` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:42.388594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2f7c4-4111-4093-8a37-40195c46f5e4 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:42.388868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2f7c4-4111-4093-8a37-40195c46f5e4 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:42.391225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-10f2f7c4-4111-4093-8a37-40195c46f5e4 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:42.405111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-10f2f7c4-4111-4093-8a37-40195c46f5e4 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Oct 01 21:03:42.405111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:03:42.405111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:03:42.405111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:03:42.405111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:03:42.405111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:03:42.405111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:03:42.405111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:03:42.405111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:03:42.405111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:03:42.405111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:03:42.405111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:03:42.405111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 283, in get Oct 01 21:03:42.405111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:check_token') Oct 01 21:03:42.405111 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Oct 01 21:03:42.406374 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Oct 01 21:03:42.406374 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Oct 01 21:03:42.406374 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Oct 01 21:03:42.406374 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:03:42.406374 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:03:42.406374 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Oct 01 21:03:42.406374 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Oct 01 21:03:42.406374 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Oct 01 21:03:42.406374 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.check_revocation(token) Oct 01 21:03:42.406374 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:03:42.406374 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:03:42.406374 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Oct 01 21:03:42.406374 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Oct 01 21:03:42.406374 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:03:42.406374 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:03:42.406374 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Oct 01 21:03:42.406374 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Oct 01 21:03:42.411829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:03:42.411829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:03:42.411829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Oct 01 21:03:42.411829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Oct 01 21:03:42.411829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Oct 01 21:03:42.411829 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:03:42.411829 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2217/4510] 10.222.0.17 () {56 vars in 1264 bytes} [Sat Oct 1 21:03:42 2022] HEAD /identity/v3/auth/tokens => generated 0 bytes in 35 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 21:03:42.414447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98af1b-630b-484f-92a0-ef7a604df75c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:42.414827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98af1b-630b-484f-92a0-ef7a604df75c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:42.415116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98af1b-630b-484f-92a0-ef7a604df75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:42.447662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8d98af1b-630b-484f-92a0-ef7a604df75c None None] MFA Rules not processed for user `f445b5efc6574c2e93b3495a9d3bd874`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:42.448899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8d98af1b-630b-484f-92a0-ef7a604df75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:42.452120 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2294/4511] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:42 2022] POST /identity/v3/auth/tokens => generated 346 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 21:03:42.464018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2f7c4-4111-4093-8a37-40195c46f5e4 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:42.475452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2f7c4-4111-4093-8a37-40195c46f5e4 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:42.480031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa5d32-54ad-4bf5-b7d6-904198cce2c2 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f445b5efc6574c2e93b3495a9d3bd874', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a54ef6ce1fee4fd8a9400c8ade186ad0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:42.480652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa5d32-54ad-4bf5-b7d6-904198cce2c2 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:42.480936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa5d32-54ad-4bf5-b7d6-904198cce2c2 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:42.481212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa5d32-54ad-4bf5-b7d6-904198cce2c2 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:42.492556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa5d32-54ad-4bf5-b7d6-904198cce2c2 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:42.495322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa5d32-54ad-4bf5-b7d6-904198cce2c2 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:42.505122 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2218/4512] 10.222.0.17 () {56 vars in 1263 bytes} [Sat Oct 1 21:03:42 2022] GET /identity/v3/auth/tokens => generated 346 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 01 21:03:42.509205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98af1b-630b-484f-92a0-ef7a604df75c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:42.521952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98af1b-630b-484f-92a0-ef7a604df75c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:42.536525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5745c-227b-4e65-a648-6e0f654ca3a3 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f445b5efc6574c2e93b3495a9d3bd874', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a54ef6ce1fee4fd8a9400c8ade186ad0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:42.537164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5745c-227b-4e65-a648-6e0f654ca3a3 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:42.537619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5745c-227b-4e65-a648-6e0f654ca3a3 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:42.537895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5745c-227b-4e65-a648-6e0f654ca3a3 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:42.546025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5745c-227b-4e65-a648-6e0f654ca3a3 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] RBAC: Authorizing `identity:revoke_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:42.547812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5745c-227b-4e65-a648-6e0f654ca3a3 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:42.568098 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2295/4513] 10.222.0.17 () {56 vars in 1266 bytes} [Sat Oct 1 21:03:42 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:42.572349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa5d32-54ad-4bf5-b7d6-904198cce2c2 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:42.583123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa5d32-54ad-4bf5-b7d6-904198cce2c2 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:42.590782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e5c55-3c3b-41c2-b558-f032c695bcc9 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f445b5efc6574c2e93b3495a9d3bd874', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a54ef6ce1fee4fd8a9400c8ade186ad0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:42.591493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e5c55-3c3b-41c2-b558-f032c695bcc9 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:42.591869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e5c55-3c3b-41c2-b558-f032c695bcc9 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:42.592380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e5c55-3c3b-41c2-b558-f032c695bcc9 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:42.594416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-156e5c55-3c3b-41c2-b558-f032c695bcc9 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:42.618709 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-156e5c55-3c3b-41c2-b558-f032c695bcc9 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Oct 01 21:03:42.618709 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:03:42.618709 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:03:42.618709 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:03:42.618709 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:03:42.618709 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:03:42.618709 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:03:42.618709 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:03:42.618709 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:03:42.618709 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:03:42.618709 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:03:42.618709 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:03:42.618709 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Oct 01 21:03:42.618709 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Oct 01 21:03:42.618709 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Oct 01 21:03:42.620105 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Oct 01 21:03:42.620105 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Oct 01 21:03:42.620105 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Oct 01 21:03:42.620105 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:03:42.620105 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:03:42.620105 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Oct 01 21:03:42.620105 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Oct 01 21:03:42.620105 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Oct 01 21:03:42.620105 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.check_revocation(token) Oct 01 21:03:42.620105 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:03:42.620105 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:03:42.620105 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Oct 01 21:03:42.620105 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Oct 01 21:03:42.620105 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:03:42.620105 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:03:42.620105 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Oct 01 21:03:42.620105 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Oct 01 21:03:42.621454 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:03:42.621454 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:03:42.621454 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Oct 01 21:03:42.621454 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Oct 01 21:03:42.621454 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Oct 01 21:03:42.621454 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:03:42.622762 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2219/4514] 10.222.0.17 () {56 vars in 1263 bytes} [Sat Oct 1 21:03:42 2022] GET /identity/v3/auth/tokens => generated 80 bytes in 51 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 01 21:03:42.627228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5745c-227b-4e65-a648-6e0f654ca3a3 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:42.648767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:42.710268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5745c-227b-4e65-a648-6e0f654ca3a3 tempest-TokensV3Test-1957947522 tempest-TokensV3Test-1957947522-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:42.715150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0c7cf-1397-4ea5-a90a-c8adfd5a015b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:42.715830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0c7cf-1397-4ea5-a90a-c8adfd5a015b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:42.716253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0c7cf-1397-4ea5-a90a-c8adfd5a015b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:42.716536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0c7cf-1397-4ea5-a90a-c8adfd5a015b admin admin] PATH_INFO: `/v3/users/f445b5efc6574c2e93b3495a9d3bd874` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:42.717674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0c7cf-1397-4ea5-a90a-c8adfd5a015b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f445b5efc6574c2e93b3495a9d3bd874)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:42.719621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0c7cf-1397-4ea5-a90a-c8adfd5a015b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:42.769991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bae0c7cf-1397-4ea5-a90a-c8adfd5a015b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f445b5efc6574c2e93b3495a9d3bd874'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:42.777518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bae0c7cf-1397-4ea5-a90a-c8adfd5a015b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f445b5efc6574c2e93b3495a9d3bd874'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:42.789897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bae0c7cf-1397-4ea5-a90a-c8adfd5a015b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f445b5efc6574c2e93b3495a9d3bd874'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:42.790672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bae0c7cf-1397-4ea5-a90a-c8adfd5a015b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f445b5efc6574c2e93b3495a9d3bd874'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:42.799814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bae0c7cf-1397-4ea5-a90a-c8adfd5a015b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f445b5efc6574c2e93b3495a9d3bd874'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:42.806817 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2296/4515] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:42 2022] DELETE /identity/v3/users/f445b5efc6574c2e93b3495a9d3bd874 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:42.811337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef854b-38cf-4a73-8992-5066cde42cda None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:42.811690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef854b-38cf-4a73-8992-5066cde42cda None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:42.812156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef854b-38cf-4a73-8992-5066cde42cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:42.831800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-34ef854b-38cf-4a73-8992-5066cde42cda None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:42.833103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-34ef854b-38cf-4a73-8992-5066cde42cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:42.893451 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2220/4516] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:03:42.903342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0c7cf-1397-4ea5-a90a-c8adfd5a015b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:42.905532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bae0c7cf-1397-4ea5-a90a-c8adfd5a015b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:42.972452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0c7cf-1397-4ea5-a90a-c8adfd5a015b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:42.976751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225f66f-7c8f-441b-9bb0-fb0c9754384c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:42.977369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225f66f-7c8f-441b-9bb0-fb0c9754384c service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:42.977650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225f66f-7c8f-441b-9bb0-fb0c9754384c service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:42.977925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225f66f-7c8f-441b-9bb0-fb0c9754384c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:42.987394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225f66f-7c8f-441b-9bb0-fb0c9754384c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:42.989491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225f66f-7c8f-441b-9bb0-fb0c9754384c service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:42.999328 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2297/4517] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:03:43.341680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef854b-38cf-4a73-8992-5066cde42cda None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:43.345107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-34ef854b-38cf-4a73-8992-5066cde42cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:43.361499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef854b-38cf-4a73-8992-5066cde42cda None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:43.368513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69a5bf-66f0-47de-8c38-8fc76a2db33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:43.369359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69a5bf-66f0-47de-8c38-8fc76a2db33c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:43.369751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69a5bf-66f0-47de-8c38-8fc76a2db33c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:43.370189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69a5bf-66f0-47de-8c38-8fc76a2db33c admin admin] PATH_INFO: `/v3/projects/a54ef6ce1fee4fd8a9400c8ade186ad0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:43.371789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69a5bf-66f0-47de-8c38-8fc76a2db33c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a54ef6ce1fee4fd8a9400c8ade186ad0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:43.374899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69a5bf-66f0-47de-8c38-8fc76a2db33c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:43.402853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:43.455835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1b69a5bf-66f0-47de-8c38-8fc76a2db33c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a54ef6ce1fee4fd8a9400c8ade186ad0', 'request_id': 'req-1b69a5bf-66f0-47de-8c38-8fc76a2db33c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:43.475378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1b69a5bf-66f0-47de-8c38-8fc76a2db33c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a54ef6ce1fee4fd8a9400c8ade186ad0', 'request_id': 'req-1b69a5bf-66f0-47de-8c38-8fc76a2db33c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:43.486481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1b69a5bf-66f0-47de-8c38-8fc76a2db33c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a54ef6ce1fee4fd8a9400c8ade186ad0', 'request_id': 'req-1b69a5bf-66f0-47de-8c38-8fc76a2db33c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:43.504045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1b69a5bf-66f0-47de-8c38-8fc76a2db33c admin admin] The token cache is being invalidate because project a54ef6ce1fee4fd8a9400c8ade186ad0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:43.504350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1b69a5bf-66f0-47de-8c38-8fc76a2db33c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:43.505022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1b69a5bf-66f0-47de-8c38-8fc76a2db33c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:43.506673 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2221/4518] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:43 2022] DELETE /identity/v3/projects/a54ef6ce1fee4fd8a9400c8ade186ad0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:43.512849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cf381-b8da-4a4d-973a-f65d8a36f06f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:43.513347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cf381-b8da-4a4d-973a-f65d8a36f06f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:43.513347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cf381-b8da-4a4d-973a-f65d8a36f06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:43.535372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2d4cf381-b8da-4a4d-973a-f65d8a36f06f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:43.536856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2d4cf381-b8da-4a4d-973a-f65d8a36f06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:43.579730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:43.634094 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2298/4519] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:03:43.642817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cb3b9-1e0e-4cae-8587-77e8637bbb2c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:43.643238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cb3b9-1e0e-4cae-8587-77e8637bbb2c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:43.643610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cb3b9-1e0e-4cae-8587-77e8637bbb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:43.673521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8e0cb3b9-1e0e-4cae-8587-77e8637bbb2c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:43.675301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8e0cb3b9-1e0e-4cae-8587-77e8637bbb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:43.709682 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2222/4520] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:03:43.714584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4cf381-b8da-4a4d-973a-f65d8a36f06f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:43.732300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4cf381-b8da-4a4d-973a-f65d8a36f06f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:43.741076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771ad13-93d0-49be-a97f-0336b9744568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:43.742025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771ad13-93d0-49be-a97f-0336b9744568 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:43.742706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771ad13-93d0-49be-a97f-0336b9744568 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:43.743974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771ad13-93d0-49be-a97f-0336b9744568 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:43.745057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0771ad13-93d0-49be-a97f-0336b9744568 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:03:43.745845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0771ad13-93d0-49be-a97f-0336b9744568 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:43.748790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0771ad13-93d0-49be-a97f-0336b9744568 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:43.768415 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2299/4521] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:03:43.773424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cb3b9-1e0e-4cae-8587-77e8637bbb2c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:43.793310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cb3b9-1e0e-4cae-8587-77e8637bbb2c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:43.808977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb3103-96ed-4408-8cac-3332192e61f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:43.809612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb3103-96ed-4408-8cac-3332192e61f3 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:43.809897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb3103-96ed-4408-8cac-3332192e61f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:43.810171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb3103-96ed-4408-8cac-3332192e61f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:43.810842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb3103-96ed-4408-8cac-3332192e61f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:43.816472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb3103-96ed-4408-8cac-3332192e61f3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:43.899152 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2223/4522] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:43 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:03:43.903115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771ad13-93d0-49be-a97f-0336b9744568 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:43.923447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771ad13-93d0-49be-a97f-0336b9744568 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:44.102363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844ad0e-4c30-4cde-abe9-21f70d295dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:44.103123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844ad0e-4c30-4cde-abe9-21f70d295dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:44.103470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844ad0e-4c30-4cde-abe9-21f70d295dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:44.103663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844ad0e-4c30-4cde-abe9-21f70d295dc4 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:44.104662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1844ad0e-4c30-4cde-abe9-21f70d295dc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:44.106762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1844ad0e-4c30-4cde-abe9-21f70d295dc4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:44.137713 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2300/4523] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:03:43 2022] POST /identity/v3/users => generated 289 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:44.143003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb3103-96ed-4408-8cac-3332192e61f3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:44.157030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb3103-96ed-4408-8cac-3332192e61f3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:44.162011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8033433-755a-44e5-bf89-11c41bd0a273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:44.162630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8033433-755a-44e5-bf89-11c41bd0a273 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:44.162916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8033433-755a-44e5-bf89-11c41bd0a273 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:44.163190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8033433-755a-44e5-bf89-11c41bd0a273 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:44.163832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8033433-755a-44e5-bf89-11c41bd0a273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:44.165448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8033433-755a-44e5-bf89-11c41bd0a273 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:44.194292 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2224/4524] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:03:44.195407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844ad0e-4c30-4cde-abe9-21f70d295dc4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:44.209263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844ad0e-4c30-4cde-abe9-21f70d295dc4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:44.217206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d915b-200d-4ab8-a89a-242593bdb2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:44.218239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d915b-200d-4ab8-a89a-242593bdb2ca admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:44.218871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d915b-200d-4ab8-a89a-242593bdb2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:44.219331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d915b-200d-4ab8-a89a-242593bdb2ca admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:44.220242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d915b-200d-4ab8-a89a-242593bdb2ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:44.222852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d915b-200d-4ab8-a89a-242593bdb2ca admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:44.246276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8033433-755a-44e5-bf89-11c41bd0a273 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:44.257253 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2301/4525] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:44.262861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8033433-755a-44e5-bf89-11c41bd0a273 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:44.268493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d7fc2-58c2-4d72-9c1a-0a6cb6976bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:44.269344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d7fc2-58c2-4d72-9c1a-0a6cb6976bd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:44.270298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d7fc2-58c2-4d72-9c1a-0a6cb6976bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:44.270725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d7fc2-58c2-4d72-9c1a-0a6cb6976bd1 admin admin] PATH_INFO: `/v3/projects/93dd543e8beb414f83af027874ffeed3/users/357a8417f9304887a66004322127cd42/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:44.309431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d7fc2-58c2-4d72-9c1a-0a6cb6976bd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93dd543e8beb414f83af027874ffeed3, user_id=357a8417f9304887a66004322127cd42, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:44.315367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d7fc2-58c2-4d72-9c1a-0a6cb6976bd1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:44.382204 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2225/4526] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:44 2022] PUT /identity/v3/projects/93dd543e8beb414f83af027874ffeed3/users/357a8417f9304887a66004322127cd42/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:44.398007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2c85d-df5e-480c-9ace-aa4141f3ec2f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:44.398880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2c85d-df5e-480c-9ace-aa4141f3ec2f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:44.398880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2c85d-df5e-480c-9ace-aa4141f3ec2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:44.483104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d6b2c85d-df5e-480c-9ace-aa4141f3ec2f None None] MFA Rules not processed for user `357a8417f9304887a66004322127cd42`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:44.484488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d6b2c85d-df5e-480c-9ace-aa4141f3ec2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:44.487242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d7fc2-58c2-4d72-9c1a-0a6cb6976bd1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:44.489990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-621d7fc2-58c2-4d72-9c1a-0a6cb6976bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:44.616212 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2302/4527] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:03:44.620771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2c85d-df5e-480c-9ace-aa4141f3ec2f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:44.674006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d7fc2-58c2-4d72-9c1a-0a6cb6976bd1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:44.683851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8fc6ae-eeac-4059-bf86-9414701f2372 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:44.685293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fc6ae-eeac-4059-bf86-9414701f2372 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:44.685753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fc6ae-eeac-4059-bf86-9414701f2372 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:44.686336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fc6ae-eeac-4059-bf86-9414701f2372 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:44.690696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0f8fc6ae-eeac-4059-bf86-9414701f2372 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:44.711882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:44.735729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:44.751582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2c85d-df5e-480c-9ace-aa4141f3ec2f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:44.757893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70002bd2-fce3-43b8-b25a-0a41ad918886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:44.758933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70002bd2-fce3-43b8-b25a-0a41ad918886 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:44.759282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70002bd2-fce3-43b8-b25a-0a41ad918886 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:44.759890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70002bd2-fce3-43b8-b25a-0a41ad918886 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:44.760620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70002bd2-fce3-43b8-b25a-0a41ad918886 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:44.762656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70002bd2-fce3-43b8-b25a-0a41ad918886 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:44.797373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8fc6ae-eeac-4059-bf86-9414701f2372 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:44.799543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8fc6ae-eeac-4059-bf86-9414701f2372 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:44.803370 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2303/4528] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:44 2022] POST /identity/v3/projects => generated 377 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:44.804961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70002bd2-fce3-43b8-b25a-0a41ad918886 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:44.848548 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2226/4529] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:03:44.910446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70002bd2-fce3-43b8-b25a-0a41ad918886 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:44.915339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806fd1c-e57e-48ef-8241-6992258fe894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:44.915958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806fd1c-e57e-48ef-8241-6992258fe894 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:44.916240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806fd1c-e57e-48ef-8241-6992258fe894 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:44.916513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806fd1c-e57e-48ef-8241-6992258fe894 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:44.917158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3806fd1c-e57e-48ef-8241-6992258fe894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:44.919269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3806fd1c-e57e-48ef-8241-6992258fe894 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:44.969230 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2304/4530] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:03:44 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:44.970727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8fc6ae-eeac-4059-bf86-9414701f2372 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:44.988375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8fc6ae-eeac-4059-bf86-9414701f2372 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:44.998660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2384d4-40dc-41fe-a53a-01eb2698421c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:44.999357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2384d4-40dc-41fe-a53a-01eb2698421c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:44.999703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2384d4-40dc-41fe-a53a-01eb2698421c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:45.000043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2384d4-40dc-41fe-a53a-01eb2698421c admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:45.000756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2384d4-40dc-41fe-a53a-01eb2698421c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:45.002462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2384d4-40dc-41fe-a53a-01eb2698421c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:45.017762 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2227/4531] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:45.019642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806fd1c-e57e-48ef-8241-6992258fe894 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:45.031457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806fd1c-e57e-48ef-8241-6992258fe894 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:45.049449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f1ea0-cd84-445c-8128-03a676d235ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:45.050368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f1ea0-cd84-445c-8128-03a676d235ff admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:45.051075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f1ea0-cd84-445c-8128-03a676d235ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:45.051565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f1ea0-cd84-445c-8128-03a676d235ff admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:45.052526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f1ea0-cd84-445c-8128-03a676d235ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:45.054421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f1ea0-cd84-445c-8128-03a676d235ff admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:45.071343 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2305/4532] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:45.074295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2384d4-40dc-41fe-a53a-01eb2698421c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:45.100146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2384d4-40dc-41fe-a53a-01eb2698421c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:45.104921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e08d1c-0f6d-47ae-9b72-76308231a6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:45.105645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e08d1c-0f6d-47ae-9b72-76308231a6d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:45.105910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e08d1c-0f6d-47ae-9b72-76308231a6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:45.106192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e08d1c-0f6d-47ae-9b72-76308231a6d6 admin admin] PATH_INFO: `/v3/projects/22d070a0e52e42d795187402fa0efac4/users/fec57dd0c70a40bfb1dde62b9b809cc1/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:45.138931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e08d1c-0f6d-47ae-9b72-76308231a6d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d070a0e52e42d795187402fa0efac4, user_id=fec57dd0c70a40bfb1dde62b9b809cc1, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:45.142641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e08d1c-0f6d-47ae-9b72-76308231a6d6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:45.159750 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2228/4533] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:45 2022] PUT /identity/v3/projects/22d070a0e52e42d795187402fa0efac4/users/fec57dd0c70a40bfb1dde62b9b809cc1/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:45.170335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5f5d5-d8fe-477b-9285-0c85b602389e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:45.170658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5f5d5-d8fe-477b-9285-0c85b602389e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:45.170939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5f5d5-d8fe-477b-9285-0c85b602389e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:45.225085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-09a5f5d5-d8fe-477b-9285-0c85b602389e None None] MFA Rules not processed for user `fec57dd0c70a40bfb1dde62b9b809cc1`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:45.226931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-09a5f5d5-d8fe-477b-9285-0c85b602389e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:45.341927 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2306/4534] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:45.349881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e08d1c-0f6d-47ae-9b72-76308231a6d6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:45.351901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d5e08d1c-0f6d-47ae-9b72-76308231a6d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:45.554451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e08d1c-0f6d-47ae-9b72-76308231a6d6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:45.571570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7183c-0455-4bc9-8e3f-5c177fa22b4d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:45.572602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7183c-0455-4bc9-8e3f-5c177fa22b4d service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:45.572959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7183c-0455-4bc9-8e3f-5c177fa22b4d service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:45.574836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7183c-0455-4bc9-8e3f-5c177fa22b4d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:45.661362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad7183c-0455-4bc9-8e3f-5c177fa22b4d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:45.662866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad7183c-0455-4bc9-8e3f-5c177fa22b4d service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:45.689054 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2229/4535] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:03:45.875253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553d7c2-cffc-4a35-a985-c630286f7d65 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:45.875789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553d7c2-cffc-4a35-a985-c630286f7d65 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:45.875789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553d7c2-cffc-4a35-a985-c630286f7d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:45.903050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5553d7c2-cffc-4a35-a985-c630286f7d65 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:45.904285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5553d7c2-cffc-4a35-a985-c630286f7d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:45.939822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:46.046900 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2307/4536] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:03:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:46.061846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7183c-0455-4bc9-8e3f-5c177fa22b4d service glance] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:46.063628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7ad7183c-0455-4bc9-8e3f-5c177fa22b4d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:46.165173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:46.282601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7183c-0455-4bc9-8e3f-5c177fa22b4d service glance] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:46.296499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7e6ec-25b8-4d82-8399-86235d1891b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:46.297130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7e6ec-25b8-4d82-8399-86235d1891b7 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:46.298026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7e6ec-25b8-4d82-8399-86235d1891b7 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:46.298302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7e6ec-25b8-4d82-8399-86235d1891b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:46.310109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7e6ec-25b8-4d82-8399-86235d1891b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:46.312105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7e6ec-25b8-4d82-8399-86235d1891b7 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:46.336630 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2230/4537] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:03:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:03:46.650311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553d7c2-cffc-4a35-a985-c630286f7d65 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:46.663347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553d7c2-cffc-4a35-a985-c630286f7d65 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:46.668612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e6c37-d9a3-4853-bb6c-958caa9b5b63 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:46.669515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e6c37-d9a3-4853-bb6c-958caa9b5b63 service glance] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:46.672158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e6c37-d9a3-4853-bb6c-958caa9b5b63 service glance] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:46.672455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e6c37-d9a3-4853-bb6c-958caa9b5b63 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:46.696981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e6c37-d9a3-4853-bb6c-958caa9b5b63 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:46.698552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e6c37-d9a3-4853-bb6c-958caa9b5b63 service glance] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:46.711969 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2308/4538] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:03:46.864805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f49bc9-8c2c-420e-97fd-bb4bfb1c2509 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:46.865351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f49bc9-8c2c-420e-97fd-bb4bfb1c2509 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:46.865351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f49bc9-8c2c-420e-97fd-bb4bfb1c2509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:46.891872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-69f49bc9-8c2c-420e-97fd-bb4bfb1c2509 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:46.893389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-69f49bc9-8c2c-420e-97fd-bb4bfb1c2509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:46.908203 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2231/4539] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:03:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:46.924730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e6c37-d9a3-4853-bb6c-958caa9b5b63 service glance] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:46.955824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e6c37-d9a3-4853-bb6c-958caa9b5b63 service glance] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:46.967194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e143c4-879b-4fe4-a9c4-d0111031e23d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:46.969077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e143c4-879b-4fe4-a9c4-d0111031e23d service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:46.969563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e143c4-879b-4fe4-a9c4-d0111031e23d service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:46.970092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e143c4-879b-4fe4-a9c4-d0111031e23d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:46.985268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e143c4-879b-4fe4-a9c4-d0111031e23d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:46.987356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e143c4-879b-4fe4-a9c4-d0111031e23d service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:47.008807 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2309/4540] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:03:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 21:03:47.181602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f49bc9-8c2c-420e-97fd-bb4bfb1c2509 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:47.300740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f49bc9-8c2c-420e-97fd-bb4bfb1c2509 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:47.306161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f371938-a1a6-4c91-84e8-c6e8b4bf6a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:47.307073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f371938-a1a6-4c91-84e8-c6e8b4bf6a91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:47.307359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f371938-a1a6-4c91-84e8-c6e8b4bf6a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:47.307631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f371938-a1a6-4c91-84e8-c6e8b4bf6a91 admin admin] PATH_INFO: `/v3/users/357a8417f9304887a66004322127cd42` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:47.308780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f371938-a1a6-4c91-84e8-c6e8b4bf6a91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=357a8417f9304887a66004322127cd42)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:47.310963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f371938-a1a6-4c91-84e8-c6e8b4bf6a91 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:47.374736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6f371938-a1a6-4c91-84e8-c6e8b4bf6a91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '357a8417f9304887a66004322127cd42'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:47.375699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6f371938-a1a6-4c91-84e8-c6e8b4bf6a91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '357a8417f9304887a66004322127cd42'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:47.376088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6f371938-a1a6-4c91-84e8-c6e8b4bf6a91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '357a8417f9304887a66004322127cd42'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:47.387469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6f371938-a1a6-4c91-84e8-c6e8b4bf6a91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '357a8417f9304887a66004322127cd42'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:47.418068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6f371938-a1a6-4c91-84e8-c6e8b4bf6a91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '357a8417f9304887a66004322127cd42'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:47.443913 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2232/4541] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:47 2022] DELETE /identity/v3/users/357a8417f9304887a66004322127cd42 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:47.448495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4c0b9-3d71-4213-b048-96e6e233ad0b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:47.448792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4c0b9-3d71-4213-b048-96e6e233ad0b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:47.449074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4c0b9-3d71-4213-b048-96e6e233ad0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:47.469095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7fb4c0b9-3d71-4213-b048-96e6e233ad0b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:47.470381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7fb4c0b9-3d71-4213-b048-96e6e233ad0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:47.545174 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2310/4542] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:47.562094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f371938-a1a6-4c91-84e8-c6e8b4bf6a91 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:47.563608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6f371938-a1a6-4c91-84e8-c6e8b4bf6a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:47.606348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:47.653776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f371938-a1a6-4c91-84e8-c6e8b4bf6a91 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:47.657680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faee7e4-8ab5-4545-b39f-373621d966cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:47.658779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faee7e4-8ab5-4545-b39f-373621d966cd service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:47.658779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faee7e4-8ab5-4545-b39f-373621d966cd service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:47.658969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faee7e4-8ab5-4545-b39f-373621d966cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:47.684199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faee7e4-8ab5-4545-b39f-373621d966cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:47.685663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faee7e4-8ab5-4545-b39f-373621d966cd service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:47.697088 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2233/4543] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:03:48.993928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4c0b9-3d71-4213-b048-96e6e233ad0b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:48.997549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7fb4c0b9-3d71-4213-b048-96e6e233ad0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:49.015939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4c0b9-3d71-4213-b048-96e6e233ad0b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:49.020903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd5361-4787-446b-855b-9481f35924c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:49.021544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd5361-4787-446b-855b-9481f35924c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:49.021917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd5361-4787-446b-855b-9481f35924c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:49.022220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd5361-4787-446b-855b-9481f35924c9 admin admin] PATH_INFO: `/v3/projects/93dd543e8beb414f83af027874ffeed3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:49.023621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd5361-4787-446b-855b-9481f35924c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93dd543e8beb414f83af027874ffeed3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:49.025881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd5361-4787-446b-855b-9481f35924c9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:49.128654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-52dd5361-4787-446b-855b-9481f35924c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93dd543e8beb414f83af027874ffeed3', 'request_id': 'req-52dd5361-4787-446b-855b-9481f35924c9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:49.141131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-52dd5361-4787-446b-855b-9481f35924c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93dd543e8beb414f83af027874ffeed3', 'request_id': 'req-52dd5361-4787-446b-855b-9481f35924c9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:49.149338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-52dd5361-4787-446b-855b-9481f35924c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93dd543e8beb414f83af027874ffeed3', 'request_id': 'req-52dd5361-4787-446b-855b-9481f35924c9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:49.168751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-52dd5361-4787-446b-855b-9481f35924c9 admin admin] The token cache is being invalidate because project 93dd543e8beb414f83af027874ffeed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:49.169292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-52dd5361-4787-446b-855b-9481f35924c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:49.172080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-52dd5361-4787-446b-855b-9481f35924c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:49.177492 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2311/4544] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:48 2022] DELETE /identity/v3/projects/93dd543e8beb414f83af027874ffeed3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:49.179214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faee7e4-8ab5-4545-b39f-373621d966cd service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:49.192355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faee7e4-8ab5-4545-b39f-373621d966cd service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:49.269165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c4aa7-b04e-4f8f-8064-357df01992fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:49.269452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c4aa7-b04e-4f8f-8064-357df01992fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:49.269814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c4aa7-b04e-4f8f-8064-357df01992fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:49.270176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c4aa7-b04e-4f8f-8064-357df01992fc admin admin] PATH_INFO: `/v3/users/fec57dd0c70a40bfb1dde62b9b809cc1` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:49.271519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c4aa7-b04e-4f8f-8064-357df01992fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fec57dd0c70a40bfb1dde62b9b809cc1)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:49.273704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c4aa7-b04e-4f8f-8064-357df01992fc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:49.344359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e26c4aa7-b04e-4f8f-8064-357df01992fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fec57dd0c70a40bfb1dde62b9b809cc1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:49.345407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e26c4aa7-b04e-4f8f-8064-357df01992fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fec57dd0c70a40bfb1dde62b9b809cc1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:49.345797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e26c4aa7-b04e-4f8f-8064-357df01992fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fec57dd0c70a40bfb1dde62b9b809cc1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:49.360177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e26c4aa7-b04e-4f8f-8064-357df01992fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fec57dd0c70a40bfb1dde62b9b809cc1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:49.379761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e26c4aa7-b04e-4f8f-8064-357df01992fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fec57dd0c70a40bfb1dde62b9b809cc1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:49.401179 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2234/4545] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:49 2022] DELETE /identity/v3/users/fec57dd0c70a40bfb1dde62b9b809cc1 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:49.897096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd5361-4787-446b-855b-9481f35924c9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:49.898895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-52dd5361-4787-446b-855b-9481f35924c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:49.943463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:49.961690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd5361-4787-446b-855b-9481f35924c9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:50.049225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2297e-c4c5-4608-a0c1-6b1147d530b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:50.051730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2297e-c4c5-4608-a0c1-6b1147d530b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:50.052180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2297e-c4c5-4608-a0c1-6b1147d530b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:50.052724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2297e-c4c5-4608-a0c1-6b1147d530b3 admin admin] PATH_INFO: `/v3/projects/22d070a0e52e42d795187402fa0efac4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:50.054065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2297e-c4c5-4608-a0c1-6b1147d530b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22d070a0e52e42d795187402fa0efac4)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:50.056550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2297e-c4c5-4608-a0c1-6b1147d530b3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:50.130709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e5a2297e-c4c5-4608-a0c1-6b1147d530b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22d070a0e52e42d795187402fa0efac4', 'request_id': 'req-e5a2297e-c4c5-4608-a0c1-6b1147d530b3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:50.139572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e5a2297e-c4c5-4608-a0c1-6b1147d530b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22d070a0e52e42d795187402fa0efac4', 'request_id': 'req-e5a2297e-c4c5-4608-a0c1-6b1147d530b3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:50.147379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e5a2297e-c4c5-4608-a0c1-6b1147d530b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22d070a0e52e42d795187402fa0efac4', 'request_id': 'req-e5a2297e-c4c5-4608-a0c1-6b1147d530b3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:50.167435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e5a2297e-c4c5-4608-a0c1-6b1147d530b3 admin admin] The token cache is being invalidate because project 22d070a0e52e42d795187402fa0efac4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:50.167823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e5a2297e-c4c5-4608-a0c1-6b1147d530b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:50.168415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e5a2297e-c4c5-4608-a0c1-6b1147d530b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:50.171202 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2312/4546] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:49 2022] DELETE /identity/v3/projects/22d070a0e52e42d795187402fa0efac4 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:50.177038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cb3b8-5135-411a-9dcd-a9af473bdb28 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:50.177335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cb3b8-5135-411a-9dcd-a9af473bdb28 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:50.177606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cb3b8-5135-411a-9dcd-a9af473bdb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:50.222520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-155cb3b8-5135-411a-9dcd-a9af473bdb28 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:50.224493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-155cb3b8-5135-411a-9dcd-a9af473bdb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:50.275189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:50.309654 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2235/4547] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:50.319345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc1513-95b0-4973-b7e6-c271fb82efb0 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:50.323267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc1513-95b0-4973-b7e6-c271fb82efb0 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:50.323977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc1513-95b0-4973-b7e6-c271fb82efb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:50.357864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7fbc1513-95b0-4973-b7e6-c271fb82efb0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:50.366550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7fbc1513-95b0-4973-b7e6-c271fb82efb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:50.381783 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2313/4548] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:03:50.382955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cb3b8-5135-411a-9dcd-a9af473bdb28 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:50.396452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cb3b8-5135-411a-9dcd-a9af473bdb28 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:50.407375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777aa6c-cfeb-4028-8dd4-9786a54d1a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:50.408882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777aa6c-cfeb-4028-8dd4-9786a54d1a31 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:50.410063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777aa6c-cfeb-4028-8dd4-9786a54d1a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:50.411071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777aa6c-cfeb-4028-8dd4-9786a54d1a31 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:50.414246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777aa6c-cfeb-4028-8dd4-9786a54d1a31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:03:50.414889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777aa6c-cfeb-4028-8dd4-9786a54d1a31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:50.416886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777aa6c-cfeb-4028-8dd4-9786a54d1a31 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:50.443315 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2236/4549] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:50.447834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc1513-95b0-4973-b7e6-c271fb82efb0 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:50.466364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc1513-95b0-4973-b7e6-c271fb82efb0 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:50.475065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce1785-da8a-4ee2-bae9-90382fecf01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:50.476088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce1785-da8a-4ee2-bae9-90382fecf01b admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:50.476642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce1785-da8a-4ee2-bae9-90382fecf01b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:50.477091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce1785-da8a-4ee2-bae9-90382fecf01b admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:50.478035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce1785-da8a-4ee2-bae9-90382fecf01b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:50.480122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce1785-da8a-4ee2-bae9-90382fecf01b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:50.529125 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2314/4550] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:50 2022] POST /identity/v3/projects => generated 367 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:03:50.530073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777aa6c-cfeb-4028-8dd4-9786a54d1a31 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:50.542300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777aa6c-cfeb-4028-8dd4-9786a54d1a31 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:50.634620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fd16c-eadd-4c1b-b70c-c4028d821e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:50.635389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fd16c-eadd-4c1b-b70c-c4028d821e5d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:50.635784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fd16c-eadd-4c1b-b70c-c4028d821e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:50.636129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fd16c-eadd-4c1b-b70c-c4028d821e5d admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:50.636760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169fd16c-eadd-4c1b-b70c-c4028d821e5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:50.639546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169fd16c-eadd-4c1b-b70c-c4028d821e5d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:50.669607 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2237/4551] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:03:50 2022] POST /identity/v3/users => generated 284 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:50.671481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce1785-da8a-4ee2-bae9-90382fecf01b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:50.687737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce1785-da8a-4ee2-bae9-90382fecf01b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:50.694281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175ab75-42e9-49d5-92ab-4d390f46be09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:50.695076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175ab75-42e9-49d5-92ab-4d390f46be09 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:50.695496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175ab75-42e9-49d5-92ab-4d390f46be09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:50.695820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175ab75-42e9-49d5-92ab-4d390f46be09 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:50.696421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175ab75-42e9-49d5-92ab-4d390f46be09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:50.698043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175ab75-42e9-49d5-92ab-4d390f46be09 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:50.710308 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2315/4552] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:50.714973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fd16c-eadd-4c1b-b70c-c4028d821e5d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:50.726939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fd16c-eadd-4c1b-b70c-c4028d821e5d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:50.733086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389272c-78c2-46ee-8185-a74c3774c7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:50.734612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389272c-78c2-46ee-8185-a74c3774c7dc admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:50.735090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389272c-78c2-46ee-8185-a74c3774c7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:50.735417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389272c-78c2-46ee-8185-a74c3774c7dc admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:50.736074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389272c-78c2-46ee-8185-a74c3774c7dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:50.738625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389272c-78c2-46ee-8185-a74c3774c7dc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:50.758817 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2238/4553] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:50.759579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175ab75-42e9-49d5-92ab-4d390f46be09 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:50.777431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175ab75-42e9-49d5-92ab-4d390f46be09 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:50.783283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25802a-9579-41b6-a3c5-cee4b4a37037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:50.783956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25802a-9579-41b6-a3c5-cee4b4a37037 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:50.784275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25802a-9579-41b6-a3c5-cee4b4a37037 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:50.785267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25802a-9579-41b6-a3c5-cee4b4a37037 admin admin] PATH_INFO: `/v3/projects/dfe3cfd637894853b34a1450f7bcf683/users/e7456555a04d4305a983ed5768a5cf7f/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:50.806923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25802a-9579-41b6-a3c5-cee4b4a37037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfe3cfd637894853b34a1450f7bcf683, user_id=e7456555a04d4305a983ed5768a5cf7f, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:50.809574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25802a-9579-41b6-a3c5-cee4b4a37037 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:50.843540 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2316/4554] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:50 2022] PUT /identity/v3/projects/dfe3cfd637894853b34a1450f7bcf683/users/e7456555a04d4305a983ed5768a5cf7f/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:50.845362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85933dcb-f51e-43a7-af23-36a01f89bf4f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:50.845648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85933dcb-f51e-43a7-af23-36a01f89bf4f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:50.845922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85933dcb-f51e-43a7-af23-36a01f89bf4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:50.902163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-85933dcb-f51e-43a7-af23-36a01f89bf4f None None] MFA Rules not processed for user `e7456555a04d4305a983ed5768a5cf7f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:50.903530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-85933dcb-f51e-43a7-af23-36a01f89bf4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:51.006003 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2239/4555] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:51.013259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25802a-9579-41b6-a3c5-cee4b4a37037 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:51.083913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:51.147514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25802a-9579-41b6-a3c5-cee4b4a37037 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:51.160950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914066c6-cd78-4d87-aec7-5cee7ffe64be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:51.163042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914066c6-cd78-4d87-aec7-5cee7ffe64be admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:51.163606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914066c6-cd78-4d87-aec7-5cee7ffe64be admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:51.164163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914066c6-cd78-4d87-aec7-5cee7ffe64be admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:51.165094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914066c6-cd78-4d87-aec7-5cee7ffe64be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:51.167503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914066c6-cd78-4d87-aec7-5cee7ffe64be admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:51.212434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85933dcb-f51e-43a7-af23-36a01f89bf4f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:51.222471 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2317/4556] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:51 2022] POST /identity/v3/projects => generated 367 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:51.242834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85933dcb-f51e-43a7-af23-36a01f89bf4f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:51.320133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:51.351821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816e84e-97b7-4b54-a9f6-218b97242a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:51.352762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816e84e-97b7-4b54-a9f6-218b97242a6b admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:51.353050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816e84e-97b7-4b54-a9f6-218b97242a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:51.353399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816e84e-97b7-4b54-a9f6-218b97242a6b admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:51.354066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4816e84e-97b7-4b54-a9f6-218b97242a6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:51.356085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4816e84e-97b7-4b54-a9f6-218b97242a6b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:51.403632 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2240/4557] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:03:51 2022] POST /identity/v3/users => generated 290 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:03:51.407468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914066c6-cd78-4d87-aec7-5cee7ffe64be admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:51.420001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914066c6-cd78-4d87-aec7-5cee7ffe64be admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:51.427185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f8560f-0d04-48be-acc1-145e6e99180a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:51.427921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8560f-0d04-48be-acc1-145e6e99180a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:51.428208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8560f-0d04-48be-acc1-145e6e99180a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:51.428479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8560f-0d04-48be-acc1-145e6e99180a admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:51.429483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f8560f-0d04-48be-acc1-145e6e99180a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:51.431618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f8560f-0d04-48be-acc1-145e6e99180a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:51.445459 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2318/4558] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:03:51.449354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816e84e-97b7-4b54-a9f6-218b97242a6b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:51.461357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816e84e-97b7-4b54-a9f6-218b97242a6b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:51.469836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab33d3-d25c-4d61-b13e-680ea8ffb32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:51.470385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab33d3-d25c-4d61-b13e-680ea8ffb32d admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:51.470766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab33d3-d25c-4d61-b13e-680ea8ffb32d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:51.471046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab33d3-d25c-4d61-b13e-680ea8ffb32d admin admin] PATH_INFO: `/v3/projects/edc972be74574a12b82fe88feef1ce76/users/b6abf93f4959468caf39ce44d472b7f8/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:51.490886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab33d3-d25c-4d61-b13e-680ea8ffb32d admin admin] RBAC: Authorizing `identity:create_grant(project_id=edc972be74574a12b82fe88feef1ce76, user_id=b6abf93f4959468caf39ce44d472b7f8, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:51.494746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab33d3-d25c-4d61-b13e-680ea8ffb32d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:51.518319 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2241/4559] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:51 2022] PUT /identity/v3/projects/edc972be74574a12b82fe88feef1ce76/users/b6abf93f4959468caf39ce44d472b7f8/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:51.522454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f8560f-0d04-48be-acc1-145e6e99180a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:51.538897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f8560f-0d04-48be-acc1-145e6e99180a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:51.618178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843a0bf-c9d0-461c-9aae-ee7cff2fcaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:51.618899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843a0bf-c9d0-461c-9aae-ee7cff2fcaeb admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:51.619186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843a0bf-c9d0-461c-9aae-ee7cff2fcaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:51.619463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843a0bf-c9d0-461c-9aae-ee7cff2fcaeb admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:51.620164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843a0bf-c9d0-461c-9aae-ee7cff2fcaeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:51.621869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843a0bf-c9d0-461c-9aae-ee7cff2fcaeb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:51.654140 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2319/4560] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:51.658294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab33d3-d25c-4d61-b13e-680ea8ffb32d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:51.673669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab33d3-d25c-4d61-b13e-680ea8ffb32d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:51.689002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1ef4d-6831-4df9-9d78-33aa0cab0e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:51.691848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1ef4d-6831-4df9-9d78-33aa0cab0e79 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:51.692211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1ef4d-6831-4df9-9d78-33aa0cab0e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:51.694706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1ef4d-6831-4df9-9d78-33aa0cab0e79 admin admin] PATH_INFO: `/v3/projects/edc972be74574a12b82fe88feef1ce76/users/b6abf93f4959468caf39ce44d472b7f8/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:51.700121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1ef4d-6831-4df9-9d78-33aa0cab0e79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edc972be74574a12b82fe88feef1ce76, user_id=b6abf93f4959468caf39ce44d472b7f8, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:51.704946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1ef4d-6831-4df9-9d78-33aa0cab0e79 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:51.726051 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2242/4561] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:51 2022] PUT /identity/v3/projects/edc972be74574a12b82fe88feef1ce76/users/b6abf93f4959468caf39ce44d472b7f8/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:51.729278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3457d5-0553-4d0a-8991-5f383cecec52 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:51.729960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3457d5-0553-4d0a-8991-5f383cecec52 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:51.730484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3457d5-0553-4d0a-8991-5f383cecec52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:51.792525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6c3457d5-0553-4d0a-8991-5f383cecec52 None None] MFA Rules not processed for user `b6abf93f4959468caf39ce44d472b7f8`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:51.795355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6c3457d5-0553-4d0a-8991-5f383cecec52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:51.953868 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2320/4562] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:51 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:51.984878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1ef4d-6831-4df9-9d78-33aa0cab0e79 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:51.986960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-06f1ef4d-6831-4df9-9d78-33aa0cab0e79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:52.157834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1ef4d-6831-4df9-9d78-33aa0cab0e79 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:52.178015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e27d72-16a3-4ea0-91e4-4fb6c631d90c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:52.178675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e27d72-16a3-4ea0-91e4-4fb6c631d90c service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:52.178954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e27d72-16a3-4ea0-91e4-4fb6c631d90c service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:52.179390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e27d72-16a3-4ea0-91e4-4fb6c631d90c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:52.203555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e27d72-16a3-4ea0-91e4-4fb6c631d90c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:52.207834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e27d72-16a3-4ea0-91e4-4fb6c631d90c service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:52.246153 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2243/4563] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:51 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:03:52.875983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378af04-198d-4fca-81f0-9917ed8ffa8c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:52.876530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378af04-198d-4fca-81f0-9917ed8ffa8c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:52.876530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378af04-198d-4fca-81f0-9917ed8ffa8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:52.907852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8378af04-198d-4fca-81f0-9917ed8ffa8c None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:52.910093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8378af04-198d-4fca-81f0-9917ed8ffa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:52.958643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:53.113482 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2321/4564] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:03:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:53.129163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e27d72-16a3-4ea0-91e4-4fb6c631d90c service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:53.130760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a0e27d72-16a3-4ea0-91e4-4fb6c631d90c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:53.330792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e27d72-16a3-4ea0-91e4-4fb6c631d90c service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:53.345999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6ae2d-cd53-4aca-bc2e-7a06b2ab15aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:53.347274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6ae2d-cd53-4aca-bc2e-7a06b2ab15aa service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:53.360671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6ae2d-cd53-4aca-bc2e-7a06b2ab15aa service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:53.361252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6ae2d-cd53-4aca-bc2e-7a06b2ab15aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:53.375327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6ae2d-cd53-4aca-bc2e-7a06b2ab15aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:53.377546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6ae2d-cd53-4aca-bc2e-7a06b2ab15aa service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:53.393008 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2244/4565] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:03:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:03:54.888249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378af04-198d-4fca-81f0-9917ed8ffa8c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:54.890900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8378af04-198d-4fca-81f0-9917ed8ffa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:55.008441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378af04-198d-4fca-81f0-9917ed8ffa8c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:55.013813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8915f-7797-458c-9581-9c840df7e2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:55.014621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8915f-7797-458c-9581-9c840df7e2f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:55.017163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8915f-7797-458c-9581-9c840df7e2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:55.017601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8915f-7797-458c-9581-9c840df7e2f4 admin admin] PATH_INFO: `/v3/users/5a92405f06244dfeba65a9cc22bdaa2e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:55.020756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8915f-7797-458c-9581-9c840df7e2f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a92405f06244dfeba65a9cc22bdaa2e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:55.024534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8915f-7797-458c-9581-9c840df7e2f4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:55.145126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cad8915f-7797-458c-9581-9c840df7e2f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a92405f06244dfeba65a9cc22bdaa2e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:55.163392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cad8915f-7797-458c-9581-9c840df7e2f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a92405f06244dfeba65a9cc22bdaa2e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:55.182562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cad8915f-7797-458c-9581-9c840df7e2f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a92405f06244dfeba65a9cc22bdaa2e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:55.183604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cad8915f-7797-458c-9581-9c840df7e2f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a92405f06244dfeba65a9cc22bdaa2e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:55.198127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cad8915f-7797-458c-9581-9c840df7e2f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a92405f06244dfeba65a9cc22bdaa2e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:55.211377 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2322/4566] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:54 2022] DELETE /identity/v3/users/5a92405f06244dfeba65a9cc22bdaa2e => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:55.361296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4cb14-7b3b-4be2-ab2e-a5347fa96f51 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:55.363340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4cb14-7b3b-4be2-ab2e-a5347fa96f51 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:55.363836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4cb14-7b3b-4be2-ab2e-a5347fa96f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:55.394668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d4c4cb14-7b3b-4be2-ab2e-a5347fa96f51 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:55.396172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d4c4cb14-7b3b-4be2-ab2e-a5347fa96f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:55.425757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:55.492982 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2245/4567] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:03:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:03:55.518540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8915f-7797-458c-9581-9c840df7e2f4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:55.521046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cad8915f-7797-458c-9581-9c840df7e2f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:55.583349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:55.609091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8915f-7797-458c-9581-9c840df7e2f4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:55.619267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fa47f-919e-48c0-8288-ce364c74c43a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:55.619899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fa47f-919e-48c0-8288-ce364c74c43a service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:55.621166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fa47f-919e-48c0-8288-ce364c74c43a service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:55.621530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fa47f-919e-48c0-8288-ce364c74c43a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:55.633017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950fa47f-919e-48c0-8288-ce364c74c43a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:55.634761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950fa47f-919e-48c0-8288-ce364c74c43a service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:55.643656 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2323/4568] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:03:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:03:55.859390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4cb14-7b3b-4be2-ab2e-a5347fa96f51 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:55.863476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d4c4cb14-7b3b-4be2-ab2e-a5347fa96f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:56.018212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4cb14-7b3b-4be2-ab2e-a5347fa96f51 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:56.027369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee1e34-c78f-4076-a9bc-2a49e2aa4edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:56.028035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee1e34-c78f-4076-a9bc-2a49e2aa4edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:56.028327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee1e34-c78f-4076-a9bc-2a49e2aa4edf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:56.028599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee1e34-c78f-4076-a9bc-2a49e2aa4edf admin admin] PATH_INFO: `/v3/projects/2c8d44c4bbfc44a297c82cda39868ddd` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:56.043020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee1e34-c78f-4076-a9bc-2a49e2aa4edf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c8d44c4bbfc44a297c82cda39868ddd)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:56.056258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee1e34-c78f-4076-a9bc-2a49e2aa4edf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:56.184237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f6ee1e34-c78f-4076-a9bc-2a49e2aa4edf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c8d44c4bbfc44a297c82cda39868ddd', 'request_id': 'req-f6ee1e34-c78f-4076-a9bc-2a49e2aa4edf'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:56.196986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f6ee1e34-c78f-4076-a9bc-2a49e2aa4edf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c8d44c4bbfc44a297c82cda39868ddd', 'request_id': 'req-f6ee1e34-c78f-4076-a9bc-2a49e2aa4edf'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:56.216412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f6ee1e34-c78f-4076-a9bc-2a49e2aa4edf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c8d44c4bbfc44a297c82cda39868ddd', 'request_id': 'req-f6ee1e34-c78f-4076-a9bc-2a49e2aa4edf'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:56.231423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f6ee1e34-c78f-4076-a9bc-2a49e2aa4edf admin admin] The token cache is being invalidate because project 2c8d44c4bbfc44a297c82cda39868ddd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:56.231807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f6ee1e34-c78f-4076-a9bc-2a49e2aa4edf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:56.232557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f6ee1e34-c78f-4076-a9bc-2a49e2aa4edf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:56.240213 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2246/4569] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:55 2022] DELETE /identity/v3/projects/2c8d44c4bbfc44a297c82cda39868ddd => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:56.241160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45ed81-1f15-494b-af0a-7bc6d986fe92 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:56.241684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45ed81-1f15-494b-af0a-7bc6d986fe92 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:56.241814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45ed81-1f15-494b-af0a-7bc6d986fe92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:56.275767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5d45ed81-1f15-494b-af0a-7bc6d986fe92 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:56.278780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5d45ed81-1f15-494b-af0a-7bc6d986fe92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:56.431750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72ac57-7d48-44d9-933e-4c766206763a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:56.432890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72ac57-7d48-44d9-933e-4c766206763a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:56.433478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72ac57-7d48-44d9-933e-4c766206763a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:56.454675 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2324/4570] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:03:56.493870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ab72ac57-7d48-44d9-933e-4c766206763a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:56.500314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ab72ac57-7d48-44d9-933e-4c766206763a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:56.526302 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2247/4571] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:56.530788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45ed81-1f15-494b-af0a-7bc6d986fe92 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:56.545271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45ed81-1f15-494b-af0a-7bc6d986fe92 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:56.565805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed6dc1-c8aa-420b-8850-1d817939a12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:56.566554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed6dc1-c8aa-420b-8850-1d817939a12e admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:56.566897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed6dc1-c8aa-420b-8850-1d817939a12e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:56.567227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed6dc1-c8aa-420b-8850-1d817939a12e admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:56.568068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed6dc1-c8aa-420b-8850-1d817939a12e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:03:56.568465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed6dc1-c8aa-420b-8850-1d817939a12e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:56.570850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed6dc1-c8aa-420b-8850-1d817939a12e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:56.597138 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2325/4572] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:56.601640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72ac57-7d48-44d9-933e-4c766206763a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:56.615553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72ac57-7d48-44d9-933e-4c766206763a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:56.625908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46dadc-3efa-430c-b151-ef05ed3763fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:56.628824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46dadc-3efa-430c-b151-ef05ed3763fd admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:56.629113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46dadc-3efa-430c-b151-ef05ed3763fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:56.629392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46dadc-3efa-430c-b151-ef05ed3763fd admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:56.630036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46dadc-3efa-430c-b151-ef05ed3763fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:56.637942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46dadc-3efa-430c-b151-ef05ed3763fd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:56.698855 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2248/4573] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:56 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:56.702473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed6dc1-c8aa-420b-8850-1d817939a12e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:56.714550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46dadc-3efa-430c-b151-ef05ed3763fd admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:56.719226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-db46dadc-3efa-430c-b151-ef05ed3763fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:56.720419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed6dc1-c8aa-420b-8850-1d817939a12e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:56.794005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:56.848778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46dadc-3efa-430c-b151-ef05ed3763fd admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:56.854819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3381782-f248-4cfe-9775-1149d568d7bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:56.855737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3381782-f248-4cfe-9775-1149d568d7bc service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:56.856198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3381782-f248-4cfe-9775-1149d568d7bc service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:56.856662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3381782-f248-4cfe-9775-1149d568d7bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:56.859133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b3381782-f248-4cfe-9775-1149d568d7bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:56.884536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c699473-c5ce-49b3-9ca9-1edb51073e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:56.885193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c699473-c5ce-49b3-9ca9-1edb51073e92 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:56.885473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c699473-c5ce-49b3-9ca9-1edb51073e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:56.885744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c699473-c5ce-49b3-9ca9-1edb51073e92 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:56.886377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c699473-c5ce-49b3-9ca9-1edb51073e92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:56.888416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c699473-c5ce-49b3-9ca9-1edb51073e92 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:56.932070 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2326/4574] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:03:56 2022] POST /identity/v3/users => generated 286 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:03:56.938101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c699473-c5ce-49b3-9ca9-1edb51073e92 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:56.959139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c699473-c5ce-49b3-9ca9-1edb51073e92 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:56.965294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3381782-f248-4cfe-9775-1149d568d7bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:56.967181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3381782-f248-4cfe-9775-1149d568d7bc service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:56.975188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce724449-18cf-4596-8ae1-fd74fbb97e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:56.975981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce724449-18cf-4596-8ae1-fd74fbb97e20 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:56.976270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce724449-18cf-4596-8ae1-fd74fbb97e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:56.976547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce724449-18cf-4596-8ae1-fd74fbb97e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:56.977537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce724449-18cf-4596-8ae1-fd74fbb97e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:56.979265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce724449-18cf-4596-8ae1-fd74fbb97e20 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:56.999718 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2327/4575] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:57.000647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce724449-18cf-4596-8ae1-fd74fbb97e20 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:57.001773 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2249/4576] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:03:57.013904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce724449-18cf-4596-8ae1-fd74fbb97e20 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:57.022438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07117c-32dc-4a82-970a-dd1e956bc451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:57.023103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07117c-32dc-4a82-970a-dd1e956bc451 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:57.023386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07117c-32dc-4a82-970a-dd1e956bc451 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:57.023659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07117c-32dc-4a82-970a-dd1e956bc451 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:57.024315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07117c-32dc-4a82-970a-dd1e956bc451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:57.038421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07117c-32dc-4a82-970a-dd1e956bc451 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:57.069248 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2328/4577] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:57.074068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3381782-f248-4cfe-9775-1149d568d7bc service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:57.093173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3381782-f248-4cfe-9775-1149d568d7bc service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:57.107713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9fb56-6af8-44c8-9602-9923acc5571d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:57.108458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9fb56-6af8-44c8-9602-9923acc5571d admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:57.109280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9fb56-6af8-44c8-9602-9923acc5571d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:57.109690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9fb56-6af8-44c8-9602-9923acc5571d admin admin] PATH_INFO: `/v3/projects/adf310503fa74abf83949a914e728693/users/6736b832e7034ce3b431b79af50095aa/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:57.140307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9fb56-6af8-44c8-9602-9923acc5571d admin admin] RBAC: Authorizing `identity:create_grant(project_id=adf310503fa74abf83949a914e728693, user_id=6736b832e7034ce3b431b79af50095aa, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:57.143103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9fb56-6af8-44c8-9602-9923acc5571d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:57.169732 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2250/4578] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:57 2022] PUT /identity/v3/projects/adf310503fa74abf83949a914e728693/users/6736b832e7034ce3b431b79af50095aa/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:57.181878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79de00-ce7b-4413-8ae1-32c7399a3925 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:57.182167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79de00-ce7b-4413-8ae1-32c7399a3925 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:57.182444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79de00-ce7b-4413-8ae1-32c7399a3925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:57.242368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-be79de00-ce7b-4413-8ae1-32c7399a3925 None None] MFA Rules not processed for user `6736b832e7034ce3b431b79af50095aa`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:57.245560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-be79de00-ce7b-4413-8ae1-32c7399a3925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:57.279092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:57.353262 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2329/4579] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:57.356838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f03e3-df99-4ddd-8ec5-058968041775 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:57.357595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f03e3-df99-4ddd-8ec5-058968041775 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:57.357595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f03e3-df99-4ddd-8ec5-058968041775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:57.384332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-264f03e3-df99-4ddd-8ec5-058968041775 None None] MFA Rules not processed for user `6736b832e7034ce3b431b79af50095aa`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:57.386254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-264f03e3-df99-4ddd-8ec5-058968041775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:57.400380 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2251/4580] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:57.415382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82618d-1888-415d-911f-5fca9d0c737a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:57.415660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82618d-1888-415d-911f-5fca9d0c737a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:57.415948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82618d-1888-415d-911f-5fca9d0c737a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:57.438668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-aa82618d-1888-415d-911f-5fca9d0c737a None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:57.439958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-aa82618d-1888-415d-911f-5fca9d0c737a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:57.505249 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2330/4581] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:03:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:57.523505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f03e3-df99-4ddd-8ec5-058968041775 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:57.525946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-264f03e3-df99-4ddd-8ec5-058968041775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:57.552933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82618d-1888-415d-911f-5fca9d0c737a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:57.554399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-aa82618d-1888-415d-911f-5fca9d0c737a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:57.629042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f03e3-df99-4ddd-8ec5-058968041775 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:57.634381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b479209-cfef-4998-8396-3fc5f3cb6cd3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:57.635120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b479209-cfef-4998-8396-3fc5f3cb6cd3 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:57.635472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b479209-cfef-4998-8396-3fc5f3cb6cd3 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:57.635853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b479209-cfef-4998-8396-3fc5f3cb6cd3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:57.645633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b479209-cfef-4998-8396-3fc5f3cb6cd3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:57.646667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82618d-1888-415d-911f-5fca9d0c737a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:57.647589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b479209-cfef-4998-8396-3fc5f3cb6cd3 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:57.652163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e93085-6888-4c9f-8ebd-87225f46a106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:57.652795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e93085-6888-4c9f-8ebd-87225f46a106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:57.653073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e93085-6888-4c9f-8ebd-87225f46a106 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:57.653348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e93085-6888-4c9f-8ebd-87225f46a106 admin admin] PATH_INFO: `/v3/users/221b665fc7cb4758a6e35cc81c6a4d43` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:57.654483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e93085-6888-4c9f-8ebd-87225f46a106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=221b665fc7cb4758a6e35cc81c6a4d43)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:57.656398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e93085-6888-4c9f-8ebd-87225f46a106 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:57.665961 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2252/4582] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:03:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:03:57.741025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05e93085-6888-4c9f-8ebd-87225f46a106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '221b665fc7cb4758a6e35cc81c6a4d43'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:57.753175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05e93085-6888-4c9f-8ebd-87225f46a106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '221b665fc7cb4758a6e35cc81c6a4d43'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:57.771822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:57.774980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05e93085-6888-4c9f-8ebd-87225f46a106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '221b665fc7cb4758a6e35cc81c6a4d43'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:57.777477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05e93085-6888-4c9f-8ebd-87225f46a106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '221b665fc7cb4758a6e35cc81c6a4d43'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:57.791422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05e93085-6888-4c9f-8ebd-87225f46a106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '221b665fc7cb4758a6e35cc81c6a4d43'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:57.806984 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2331/4583] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:57 2022] DELETE /identity/v3/users/221b665fc7cb4758a6e35cc81c6a4d43 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:57.818391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b479209-cfef-4998-8396-3fc5f3cb6cd3 service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:57.822250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7b479209-cfef-4998-8396-3fc5f3cb6cd3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:57.849345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b479209-cfef-4998-8396-3fc5f3cb6cd3 service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:57.898750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:57.938011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d1ec6-01e5-48ba-bc91-9f87653554e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:57.939131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d1ec6-01e5-48ba-bc91-9f87653554e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:57.939718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d1ec6-01e5-48ba-bc91-9f87653554e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:57.940275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d1ec6-01e5-48ba-bc91-9f87653554e2 admin admin] PATH_INFO: `/v3/users/8eb0d60068ff4776a7d79d3b891bb9a1` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:57.942547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d1ec6-01e5-48ba-bc91-9f87653554e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eb0d60068ff4776a7d79d3b891bb9a1)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:57.944771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d1ec6-01e5-48ba-bc91-9f87653554e2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:58.028184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2d8d1ec6-01e5-48ba-bc91-9f87653554e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eb0d60068ff4776a7d79d3b891bb9a1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:58.028917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2d8d1ec6-01e5-48ba-bc91-9f87653554e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eb0d60068ff4776a7d79d3b891bb9a1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:58.029593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2d8d1ec6-01e5-48ba-bc91-9f87653554e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eb0d60068ff4776a7d79d3b891bb9a1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:58.043347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2d8d1ec6-01e5-48ba-bc91-9f87653554e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eb0d60068ff4776a7d79d3b891bb9a1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:58.050040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e93085-6888-4c9f-8ebd-87225f46a106 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:58.051492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-05e93085-6888-4c9f-8ebd-87225f46a106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:58.059291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2d8d1ec6-01e5-48ba-bc91-9f87653554e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eb0d60068ff4776a7d79d3b891bb9a1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:58.085757 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2253/4584] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:03:57 2022] DELETE /identity/v3/users/8eb0d60068ff4776a7d79d3b891bb9a1 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:58.087238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2343f-e0ed-4837-9040-5fb377b24d5e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:58.087941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2343f-e0ed-4837-9040-5fb377b24d5e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:58.088232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2343f-e0ed-4837-9040-5fb377b24d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:58.088804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e93085-6888-4c9f-8ebd-87225f46a106 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:58.126849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-43d2343f-e0ed-4837-9040-5fb377b24d5e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:58.128302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-43d2343f-e0ed-4837-9040-5fb377b24d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:58.178193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efaf9f0-d678-42dd-8bf0-599d1796867c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:58.179198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efaf9f0-d678-42dd-8bf0-599d1796867c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:58.179482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efaf9f0-d678-42dd-8bf0-599d1796867c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:58.179975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efaf9f0-d678-42dd-8bf0-599d1796867c admin admin] PATH_INFO: `/v3/projects/4ae7982f0b4349c6ab1c445988ea4f9f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:58.181830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efaf9f0-d678-42dd-8bf0-599d1796867c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ae7982f0b4349c6ab1c445988ea4f9f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:58.184543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efaf9f0-d678-42dd-8bf0-599d1796867c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:58.256113 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2254/4585] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:58.287858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2343f-e0ed-4837-9040-5fb377b24d5e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:58.289389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-43d2343f-e0ed-4837-9040-5fb377b24d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:58.303563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4efaf9f0-d678-42dd-8bf0-599d1796867c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ae7982f0b4349c6ab1c445988ea4f9f', 'request_id': 'req-4efaf9f0-d678-42dd-8bf0-599d1796867c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:58.321904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4efaf9f0-d678-42dd-8bf0-599d1796867c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ae7982f0b4349c6ab1c445988ea4f9f', 'request_id': 'req-4efaf9f0-d678-42dd-8bf0-599d1796867c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:58.338044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4efaf9f0-d678-42dd-8bf0-599d1796867c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ae7982f0b4349c6ab1c445988ea4f9f', 'request_id': 'req-4efaf9f0-d678-42dd-8bf0-599d1796867c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:58.355536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4efaf9f0-d678-42dd-8bf0-599d1796867c admin admin] The token cache is being invalidate because project 4ae7982f0b4349c6ab1c445988ea4f9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:58.356025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4efaf9f0-d678-42dd-8bf0-599d1796867c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:58.356795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4efaf9f0-d678-42dd-8bf0-599d1796867c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:58.361008 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2332/4586] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:58 2022] DELETE /identity/v3/projects/4ae7982f0b4349c6ab1c445988ea4f9f => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:03:58.367320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818732ee-7a38-404f-9f11-6fbe0f4adeae None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:58.367703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818732ee-7a38-404f-9f11-6fbe0f4adeae None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:58.368054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818732ee-7a38-404f-9f11-6fbe0f4adeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:58.390657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2343f-e0ed-4837-9040-5fb377b24d5e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:58.395074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:58.399507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-818732ee-7a38-404f-9f11-6fbe0f4adeae None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:58.400998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-818732ee-7a38-404f-9f11-6fbe0f4adeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:58.478779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f3f25-58c5-45bc-9afa-0b04eaa66eb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:58.479399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f3f25-58c5-45bc-9afa-0b04eaa66eb5 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:58.479683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f3f25-58c5-45bc-9afa-0b04eaa66eb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:58.479960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f3f25-58c5-45bc-9afa-0b04eaa66eb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:58.482173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a42f3f25-58c5-45bc-9afa-0b04eaa66eb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:58.494816 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2333/4587] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:03:58.503602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb1a7f-0045-4329-a838-9704d0a32308 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:58.503971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb1a7f-0045-4329-a838-9704d0a32308 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:58.504262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb1a7f-0045-4329-a838-9704d0a32308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:58.528699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c2fb1a7f-0045-4329-a838-9704d0a32308 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:58.529944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c2fb1a7f-0045-4329-a838-9704d0a32308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:58.541807 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2334/4588] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:03:58.546866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb1a7f-0045-4329-a838-9704d0a32308 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:58.562960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb1a7f-0045-4329-a838-9704d0a32308 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:58.576141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9237a-d3d2-40a7-a08b-1aba313fb295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:58.576769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9237a-d3d2-40a7-a08b-1aba313fb295 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:58.577362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9237a-d3d2-40a7-a08b-1aba313fb295 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:58.577362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9237a-d3d2-40a7-a08b-1aba313fb295 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:58.577979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe9237a-d3d2-40a7-a08b-1aba313fb295 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:03:58.578284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe9237a-d3d2-40a7-a08b-1aba313fb295 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:58.584218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe9237a-d3d2-40a7-a08b-1aba313fb295 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:58.590282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:58.591966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f3f25-58c5-45bc-9afa-0b04eaa66eb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:58.596583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f3f25-58c5-45bc-9afa-0b04eaa66eb5 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:58.615360 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2335/4589] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:58.616395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9237a-d3d2-40a7-a08b-1aba313fb295 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:58.625884 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2255/4590] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:03:58.630579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9237a-d3d2-40a7-a08b-1aba313fb295 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:58.637204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f549abc-72b2-48f9-9d7f-9d782226556d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:58.637633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f549abc-72b2-48f9-9d7f-9d782226556d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:58.637834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f549abc-72b2-48f9-9d7f-9d782226556d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:58.638359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f549abc-72b2-48f9-9d7f-9d782226556d admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:58.639196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f549abc-72b2-48f9-9d7f-9d782226556d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:58.641115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f549abc-72b2-48f9-9d7f-9d782226556d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:58.686450 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2336/4591] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:58 2022] POST /identity/v3/projects => generated 399 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:03:58.687444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f3f25-58c5-45bc-9afa-0b04eaa66eb5 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:58.702115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f3f25-58c5-45bc-9afa-0b04eaa66eb5 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:58.790658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22991534-5189-4ae9-8fb9-9f0674fde962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:58.791517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22991534-5189-4ae9-8fb9-9f0674fde962 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:58.791799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22991534-5189-4ae9-8fb9-9f0674fde962 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:58.792067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22991534-5189-4ae9-8fb9-9f0674fde962 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:58.792696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22991534-5189-4ae9-8fb9-9f0674fde962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:58.794671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22991534-5189-4ae9-8fb9-9f0674fde962 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:58.840238 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2256/4592] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:03:58 2022] POST /identity/v3/users => generated 300 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:03:58.844410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f549abc-72b2-48f9-9d7f-9d782226556d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:58.856815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f549abc-72b2-48f9-9d7f-9d782226556d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:58.863272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541967d6-c279-473b-854b-ca75f63db8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:58.864107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541967d6-c279-473b-854b-ca75f63db8fe admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:58.864583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541967d6-c279-473b-854b-ca75f63db8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:58.865033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541967d6-c279-473b-854b-ca75f63db8fe admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:58.865863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541967d6-c279-473b-854b-ca75f63db8fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:58.867743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541967d6-c279-473b-854b-ca75f63db8fe admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:58.883890 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2337/4593] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:03:58.884341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22991534-5189-4ae9-8fb9-9f0674fde962 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:58.902234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22991534-5189-4ae9-8fb9-9f0674fde962 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:58.909502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e20f3a-1ce9-4f3b-b81d-4b582010a50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:58.910395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e20f3a-1ce9-4f3b-b81d-4b582010a50d admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:58.910712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e20f3a-1ce9-4f3b-b81d-4b582010a50d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:58.910988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e20f3a-1ce9-4f3b-b81d-4b582010a50d admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:58.911635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e20f3a-1ce9-4f3b-b81d-4b582010a50d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:58.913963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e20f3a-1ce9-4f3b-b81d-4b582010a50d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:58.927555 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2257/4594] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:03:58.931911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541967d6-c279-473b-854b-ca75f63db8fe admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:58.950014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541967d6-c279-473b-854b-ca75f63db8fe admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:58.957917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290e751-9d5a-45e9-add7-0134e38f5455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:58.958766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290e751-9d5a-45e9-add7-0134e38f5455 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:58.959665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290e751-9d5a-45e9-add7-0134e38f5455 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:58.960059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290e751-9d5a-45e9-add7-0134e38f5455 admin admin] PATH_INFO: `/v3/projects/99422a6e3b744bff964052dfddb678ef/users/aad36d5e04a7453aba1e6d0b15d2d06f/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:58.979471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290e751-9d5a-45e9-add7-0134e38f5455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99422a6e3b744bff964052dfddb678ef, user_id=aad36d5e04a7453aba1e6d0b15d2d06f, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:58.982961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290e751-9d5a-45e9-add7-0134e38f5455 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:59.022866 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2338/4595] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:03:58 2022] PUT /identity/v3/projects/99422a6e3b744bff964052dfddb678ef/users/aad36d5e04a7453aba1e6d0b15d2d06f/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:59.025889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fdabef-19ed-4df2-9c08-fb7786a1871a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:59.026181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fdabef-19ed-4df2-9c08-fb7786a1871a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:59.026460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fdabef-19ed-4df2-9c08-fb7786a1871a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:59.064787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a7fdabef-19ed-4df2-9c08-fb7786a1871a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:59.069852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a7fdabef-19ed-4df2-9c08-fb7786a1871a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:59.161676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:59.188577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290e751-9d5a-45e9-add7-0134e38f5455 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:59.193718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1290e751-9d5a-45e9-add7-0134e38f5455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:59.262341 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2258/4596] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:59.273674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fdabef-19ed-4df2-9c08-fb7786a1871a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:59.276425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a7fdabef-19ed-4df2-9c08-fb7786a1871a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:59.349931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290e751-9d5a-45e9-add7-0134e38f5455 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:59.363254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74002a85-96c9-4f86-a985-1c29846c2be2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:59.364377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74002a85-96c9-4f86-a985-1c29846c2be2 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:59.364900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74002a85-96c9-4f86-a985-1c29846c2be2 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:59.365546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74002a85-96c9-4f86-a985-1c29846c2be2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:59.368689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-74002a85-96c9-4f86-a985-1c29846c2be2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:59.395620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fdabef-19ed-4df2-9c08-fb7786a1871a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:59.404684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567f37d-1156-49ca-a222-bb52bedc85db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:59.405470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567f37d-1156-49ca-a222-bb52bedc85db service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:59.405927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567f37d-1156-49ca-a222-bb52bedc85db service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:59.407718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567f37d-1156-49ca-a222-bb52bedc85db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:59.428104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567f37d-1156-49ca-a222-bb52bedc85db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:59.432502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567f37d-1156-49ca-a222-bb52bedc85db service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:59.458479 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2259/4597] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:03:59.479931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74002a85-96c9-4f86-a985-1c29846c2be2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:59.481847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74002a85-96c9-4f86-a985-1c29846c2be2 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:59.502515 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2339/4598] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:03:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:03:59.513120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567f37d-1156-49ca-a222-bb52bedc85db service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:59.515485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f567f37d-1156-49ca-a222-bb52bedc85db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:59.534794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567f37d-1156-49ca-a222-bb52bedc85db service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:59.541606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c4b34-3b11-40d6-b222-691260f846d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:59.542639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c4b34-3b11-40d6-b222-691260f846d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:59.543137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c4b34-3b11-40d6-b222-691260f846d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:59.543978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c4b34-3b11-40d6-b222-691260f846d0 admin admin] PATH_INFO: `/v3/projects/e63f74bcb25e44e581212a829ae7da58` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:59.545755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c4b34-3b11-40d6-b222-691260f846d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e63f74bcb25e44e581212a829ae7da58)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:59.547996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c4b34-3b11-40d6-b222-691260f846d0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:59.641763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c43c4b34-3b11-40d6-b222-691260f846d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e63f74bcb25e44e581212a829ae7da58', 'request_id': 'req-c43c4b34-3b11-40d6-b222-691260f846d0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:59.651981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c43c4b34-3b11-40d6-b222-691260f846d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e63f74bcb25e44e581212a829ae7da58', 'request_id': 'req-c43c4b34-3b11-40d6-b222-691260f846d0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:59.659334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c43c4b34-3b11-40d6-b222-691260f846d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e63f74bcb25e44e581212a829ae7da58', 'request_id': 'req-c43c4b34-3b11-40d6-b222-691260f846d0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:59.672740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c43c4b34-3b11-40d6-b222-691260f846d0 admin admin] The token cache is being invalidate because project e63f74bcb25e44e581212a829ae7da58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:03:59.673061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c43c4b34-3b11-40d6-b222-691260f846d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:59.673740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c43c4b34-3b11-40d6-b222-691260f846d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:03:59.675455 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2260/4599] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:03:59 2022] DELETE /identity/v3/projects/e63f74bcb25e44e581212a829ae7da58 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:03:59.680296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161d7fb-66cf-490f-a799-f36a1a356f11 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:59.680943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161d7fb-66cf-490f-a799-f36a1a356f11 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:59.681271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161d7fb-66cf-490f-a799-f36a1a356f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:59.704264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d161d7fb-66cf-490f-a799-f36a1a356f11 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:59.705684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d161d7fb-66cf-490f-a799-f36a1a356f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:59.748446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:59.830012 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2340/4600] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:59.838114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380179a8-84e1-40c2-b419-88d9dbb8d3eb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:59.838317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380179a8-84e1-40c2-b419-88d9dbb8d3eb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:59.838781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380179a8-84e1-40c2-b419-88d9dbb8d3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:59.875633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-380179a8-84e1-40c2-b419-88d9dbb8d3eb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:03:59.887425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-380179a8-84e1-40c2-b419-88d9dbb8d3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:03:59.901972 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2261/4601] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:03:59.906557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161d7fb-66cf-490f-a799-f36a1a356f11 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:59.923327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161d7fb-66cf-490f-a799-f36a1a356f11 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:59.935090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf279c-487c-48de-b283-b2d05d32ed86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:59.935737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf279c-487c-48de-b283-b2d05d32ed86 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:59.936226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf279c-487c-48de-b283-b2d05d32ed86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:03:59.936506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf279c-487c-48de-b283-b2d05d32ed86 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:03:59.937251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf279c-487c-48de-b283-b2d05d32ed86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:03:59.937584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf279c-487c-48de-b283-b2d05d32ed86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:03:59.939822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf279c-487c-48de-b283-b2d05d32ed86 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:03:59.975261 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2341/4602] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:03:59.978831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380179a8-84e1-40c2-b419-88d9dbb8d3eb None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:03:59.990605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380179a8-84e1-40c2-b419-88d9dbb8d3eb None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:03:59.998699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985f6fb-9b98-457d-a6dc-8d6c5e3e3b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:03:59.999622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985f6fb-9b98-457d-a6dc-8d6c5e3e3b46 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:03:59.999974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985f6fb-9b98-457d-a6dc-8d6c5e3e3b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:00.000369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985f6fb-9b98-457d-a6dc-8d6c5e3e3b46 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:00.001095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985f6fb-9b98-457d-a6dc-8d6c5e3e3b46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:00.003361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985f6fb-9b98-457d-a6dc-8d6c5e3e3b46 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:00.058986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf279c-487c-48de-b283-b2d05d32ed86 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:00.059642 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2262/4603] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:03:59 2022] POST /identity/v3/projects => generated 377 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:00.079621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf279c-487c-48de-b283-b2d05d32ed86 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:00.216013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e8815-5f22-495c-960b-a105ea614c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:00.220694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e8815-5f22-495c-960b-a105ea614c71 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:00.224966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e8815-5f22-495c-960b-a105ea614c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:00.225506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e8815-5f22-495c-960b-a105ea614c71 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:00.227832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e8815-5f22-495c-960b-a105ea614c71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:00.242537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e8815-5f22-495c-960b-a105ea614c71 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:00.303740 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2342/4604] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:04:00 2022] POST /identity/v3/users => generated 289 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:04:00.308285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985f6fb-9b98-457d-a6dc-8d6c5e3e3b46 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:00.328128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985f6fb-9b98-457d-a6dc-8d6c5e3e3b46 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:00.338113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ee8f9-b147-4ad1-bd96-d48a81b9da7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:00.339301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ee8f9-b147-4ad1-bd96-d48a81b9da7b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:00.339787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ee8f9-b147-4ad1-bd96-d48a81b9da7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:00.340258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ee8f9-b147-4ad1-bd96-d48a81b9da7b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:00.341153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ee8f9-b147-4ad1-bd96-d48a81b9da7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:00.343220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ee8f9-b147-4ad1-bd96-d48a81b9da7b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:00.365522 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2263/4605] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:04:00.365745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e8815-5f22-495c-960b-a105ea614c71 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:00.383062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e8815-5f22-495c-960b-a105ea614c71 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:00.389036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b585a-01e8-400c-8d51-4cb8179fcd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:00.390019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b585a-01e8-400c-8d51-4cb8179fcd75 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:00.390557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b585a-01e8-400c-8d51-4cb8179fcd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:00.391047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b585a-01e8-400c-8d51-4cb8179fcd75 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:00.391962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b585a-01e8-400c-8d51-4cb8179fcd75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:00.394745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b585a-01e8-400c-8d51-4cb8179fcd75 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:00.407579 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2343/4606] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:04:00.412422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ee8f9-b147-4ad1-bd96-d48a81b9da7b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:00.429653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ee8f9-b147-4ad1-bd96-d48a81b9da7b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:00.435804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128be919-60d5-4ad8-816e-69fb28769fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:00.436655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128be919-60d5-4ad8-816e-69fb28769fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:00.436961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128be919-60d5-4ad8-816e-69fb28769fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:00.437210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128be919-60d5-4ad8-816e-69fb28769fe7 admin admin] PATH_INFO: `/v3/projects/e51ee11a2afc46139d7021758d753513/users/933da0e7f0404d0286aba2c12e10bd8d/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:00.456657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128be919-60d5-4ad8-816e-69fb28769fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e51ee11a2afc46139d7021758d753513, user_id=933da0e7f0404d0286aba2c12e10bd8d, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:00.459889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128be919-60d5-4ad8-816e-69fb28769fe7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:00.473706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:00.489595 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2264/4607] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:04:00 2022] PUT /identity/v3/projects/e51ee11a2afc46139d7021758d753513/users/933da0e7f0404d0286aba2c12e10bd8d/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:04:00.498164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293e791-1ae5-4212-a7bb-fbdd48cc5244 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:00.498450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293e791-1ae5-4212-a7bb-fbdd48cc5244 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:00.498743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293e791-1ae5-4212-a7bb-fbdd48cc5244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:00.554697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2293e791-1ae5-4212-a7bb-fbdd48cc5244 None None] MFA Rules not processed for user `933da0e7f0404d0286aba2c12e10bd8d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:00.557028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2293e791-1ae5-4212-a7bb-fbdd48cc5244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:00.666654 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2344/4608] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:00.675575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128be919-60d5-4ad8-816e-69fb28769fe7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:00.677744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-128be919-60d5-4ad8-816e-69fb28769fe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:00.763969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128be919-60d5-4ad8-816e-69fb28769fe7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:00.788465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ce1a5-9236-49ac-8a57-edbc96140cf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:00.789380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ce1a5-9236-49ac-8a57-edbc96140cf4 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:00.790474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ce1a5-9236-49ac-8a57-edbc96140cf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:00.793697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ce1a5-9236-49ac-8a57-edbc96140cf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:00.804457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ce1a5-9236-49ac-8a57-edbc96140cf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:00.806190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ce1a5-9236-49ac-8a57-edbc96140cf4 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:00.817638 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2265/4609] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:00.842629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293e791-1ae5-4212-a7bb-fbdd48cc5244 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:00.900794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:00.972217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293e791-1ae5-4212-a7bb-fbdd48cc5244 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:00.990868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991b64b-7752-4d5b-9278-6ca4b6e64eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:00.992056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991b64b-7752-4d5b-9278-6ca4b6e64eab admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:00.992337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991b64b-7752-4d5b-9278-6ca4b6e64eab admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:00.992610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991b64b-7752-4d5b-9278-6ca4b6e64eab admin admin] PATH_INFO: `/v3/users/933da0e7f0404d0286aba2c12e10bd8d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:00.995793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991b64b-7752-4d5b-9278-6ca4b6e64eab admin admin] RBAC: Authorizing `identity:delete_user(user_id=933da0e7f0404d0286aba2c12e10bd8d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:00.998211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991b64b-7752-4d5b-9278-6ca4b6e64eab admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:01.095014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1991b64b-7752-4d5b-9278-6ca4b6e64eab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '933da0e7f0404d0286aba2c12e10bd8d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:01.108801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1991b64b-7752-4d5b-9278-6ca4b6e64eab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '933da0e7f0404d0286aba2c12e10bd8d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:01.139350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1991b64b-7752-4d5b-9278-6ca4b6e64eab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '933da0e7f0404d0286aba2c12e10bd8d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:01.140916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1991b64b-7752-4d5b-9278-6ca4b6e64eab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '933da0e7f0404d0286aba2c12e10bd8d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:01.154671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1991b64b-7752-4d5b-9278-6ca4b6e64eab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '933da0e7f0404d0286aba2c12e10bd8d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:01.167619 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2345/4610] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:04:00 2022] DELETE /identity/v3/users/933da0e7f0404d0286aba2c12e10bd8d => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:01.167860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21632d94-a757-4722-ab46-bcece891fae9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:01.167860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21632d94-a757-4722-ab46-bcece891fae9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:01.168109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21632d94-a757-4722-ab46-bcece891fae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:01.169203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991b64b-7752-4d5b-9278-6ca4b6e64eab admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:01.170838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1991b64b-7752-4d5b-9278-6ca4b6e64eab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:01.192464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-21632d94-a757-4722-ab46-bcece891fae9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:01.193671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-21632d94-a757-4722-ab46-bcece891fae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:01.268236 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2266/4611] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:01.280702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21632d94-a757-4722-ab46-bcece891fae9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:01.282942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-21632d94-a757-4722-ab46-bcece891fae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:01.302552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991b64b-7752-4d5b-9278-6ca4b6e64eab admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:01.311203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477abda-da42-4568-8428-374e822cdf17 tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6abf93f4959468caf39ce44d472b7f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edc972be74574a12b82fe88feef1ce76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:01.311938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477abda-da42-4568-8428-374e822cdf17 tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:01.312294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477abda-da42-4568-8428-374e822cdf17 tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:01.312671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477abda-da42-4568-8428-374e822cdf17 tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:01.315377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2477abda-da42-4568-8428-374e822cdf17 tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:04:01.317901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2477abda-da42-4568-8428-374e822cdf17 tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:01.326778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2477abda-da42-4568-8428-374e822cdf17 tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:01.350393 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2346/4612] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:04:01.355067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477abda-da42-4568-8428-374e822cdf17 tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:01.372580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477abda-da42-4568-8428-374e822cdf17 tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:01.396385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b645f-d74c-43a4-8f62-3cdc2af61a4e tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6abf93f4959468caf39ce44d472b7f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edc972be74574a12b82fe88feef1ce76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:01.397208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b645f-d74c-43a4-8f62-3cdc2af61a4e tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:01.397599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b645f-d74c-43a4-8f62-3cdc2af61a4e tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:01.398201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b645f-d74c-43a4-8f62-3cdc2af61a4e tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:01.399116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b645f-d74c-43a4-8f62-3cdc2af61a4e tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:01.401371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b645f-d74c-43a4-8f62-3cdc2af61a4e tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:01.475046 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2347/4613] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:04:01 2022] POST /identity/v3/projects => generated 347 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:04:01.476333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b645f-d74c-43a4-8f62-3cdc2af61a4e tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:01.505061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21632d94-a757-4722-ab46-bcece891fae9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:01.540889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:01.616237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b645f-d74c-43a4-8f62-3cdc2af61a4e tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:01.628040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba2e3d-c0e0-41ce-beec-d0437b864930 tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6abf93f4959468caf39ce44d472b7f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edc972be74574a12b82fe88feef1ce76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:01.628675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba2e3d-c0e0-41ce-beec-d0437b864930 tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:01.628962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba2e3d-c0e0-41ce-beec-d0437b864930 tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:01.629337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba2e3d-c0e0-41ce-beec-d0437b864930 tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:01.630016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba2e3d-c0e0-41ce-beec-d0437b864930 tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:04:01.630325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba2e3d-c0e0-41ce-beec-d0437b864930 tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:01.633356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba2e3d-c0e0-41ce-beec-d0437b864930 tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:01.639002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6caa904-cd56-4712-8052-31a0b4ced3d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:01.641883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6caa904-cd56-4712-8052-31a0b4ced3d1 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:01.644085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6caa904-cd56-4712-8052-31a0b4ced3d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:01.644397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6caa904-cd56-4712-8052-31a0b4ced3d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:01.670036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:01.680882 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2348/4614] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:04:01.804403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6caa904-cd56-4712-8052-31a0b4ced3d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:01.806804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6caa904-cd56-4712-8052-31a0b4ced3d1 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:01.836459 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2267/4615] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:02.227007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba2e3d-c0e0-41ce-beec-d0437b864930 tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:02.228829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4eba2e3d-c0e0-41ce-beec-d0437b864930 tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:02.244726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba2e3d-c0e0-41ce-beec-d0437b864930 tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:02.250283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe147ff-31ea-47c7-af72-e3f00b20ec91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:02.251003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe147ff-31ea-47c7-af72-e3f00b20ec91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:02.251307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe147ff-31ea-47c7-af72-e3f00b20ec91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:02.251597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe147ff-31ea-47c7-af72-e3f00b20ec91 admin admin] PATH_INFO: `/v3/projects/e51ee11a2afc46139d7021758d753513` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:02.253858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe147ff-31ea-47c7-af72-e3f00b20ec91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e51ee11a2afc46139d7021758d753513)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:02.255925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe147ff-31ea-47c7-af72-e3f00b20ec91 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:02.330085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1fe147ff-31ea-47c7-af72-e3f00b20ec91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e51ee11a2afc46139d7021758d753513', 'request_id': 'req-1fe147ff-31ea-47c7-af72-e3f00b20ec91'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:02.339939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1fe147ff-31ea-47c7-af72-e3f00b20ec91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e51ee11a2afc46139d7021758d753513', 'request_id': 'req-1fe147ff-31ea-47c7-af72-e3f00b20ec91'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:02.347546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1fe147ff-31ea-47c7-af72-e3f00b20ec91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e51ee11a2afc46139d7021758d753513', 'request_id': 'req-1fe147ff-31ea-47c7-af72-e3f00b20ec91'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:02.361458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1fe147ff-31ea-47c7-af72-e3f00b20ec91 admin admin] The token cache is being invalidate because project e51ee11a2afc46139d7021758d753513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:04:02.361732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1fe147ff-31ea-47c7-af72-e3f00b20ec91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:02.363114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1fe147ff-31ea-47c7-af72-e3f00b20ec91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:02.364773 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2349/4616] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:04:02 2022] DELETE /identity/v3/projects/e51ee11a2afc46139d7021758d753513 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:04:02.369841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d472216-4671-491e-b48b-a0e04d56b443 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:02.370390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d472216-4671-491e-b48b-a0e04d56b443 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:02.370921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d472216-4671-491e-b48b-a0e04d56b443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:02.397794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6d472216-4671-491e-b48b-a0e04d56b443 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:02.399842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6d472216-4671-491e-b48b-a0e04d56b443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:02.473148 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2268/4617] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:02.480256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29009f8c-2bbd-4298-8676-acd3d3862786 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:02.480752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29009f8c-2bbd-4298-8676-acd3d3862786 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:02.481103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29009f8c-2bbd-4298-8676-acd3d3862786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:02.500990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-29009f8c-2bbd-4298-8676-acd3d3862786 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:02.502139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-29009f8c-2bbd-4298-8676-acd3d3862786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:02.510958 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2350/4618] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:02.514969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d472216-4671-491e-b48b-a0e04d56b443 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:02.525459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d472216-4671-491e-b48b-a0e04d56b443 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:02.530203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c7505-3f46-49fa-bd2b-12136c55683a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:02.530851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c7505-3f46-49fa-bd2b-12136c55683a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:02.531138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c7505-3f46-49fa-bd2b-12136c55683a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:02.531571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c7505-3f46-49fa-bd2b-12136c55683a admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:02.532117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c7505-3f46-49fa-bd2b-12136c55683a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:04:02.532447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c7505-3f46-49fa-bd2b-12136c55683a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:02.534145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c7505-3f46-49fa-bd2b-12136c55683a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:02.550426 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2269/4619] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:04:02.551513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29009f8c-2bbd-4298-8676-acd3d3862786 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:02.562255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29009f8c-2bbd-4298-8676-acd3d3862786 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:02.571243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385ce00-5bdb-4845-8f25-d70b71f63b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:02.571973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385ce00-5bdb-4845-8f25-d70b71f63b73 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:02.572317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385ce00-5bdb-4845-8f25-d70b71f63b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:02.572662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385ce00-5bdb-4845-8f25-d70b71f63b73 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:02.573446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385ce00-5bdb-4845-8f25-d70b71f63b73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:02.575433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385ce00-5bdb-4845-8f25-d70b71f63b73 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:02.608008 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2351/4620] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:04:02 2022] POST /identity/v3/projects => generated 403 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:04:02.616678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c7505-3f46-49fa-bd2b-12136c55683a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:02.631005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c7505-3f46-49fa-bd2b-12136c55683a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:02.703749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9c46d-cea2-4063-8c86-2a471e602820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:02.704590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9c46d-cea2-4063-8c86-2a471e602820 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:02.704939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9c46d-cea2-4063-8c86-2a471e602820 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:02.705288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9c46d-cea2-4063-8c86-2a471e602820 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:02.706519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9c46d-cea2-4063-8c86-2a471e602820 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:02.709213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9c46d-cea2-4063-8c86-2a471e602820 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:02.736011 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2270/4621] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:04:02 2022] POST /identity/v3/users => generated 302 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:02.740506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385ce00-5bdb-4845-8f25-d70b71f63b73 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:02.750767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9c46d-cea2-4063-8c86-2a471e602820 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:02.756341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385ce00-5bdb-4845-8f25-d70b71f63b73 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:02.758443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1fd9c46d-cea2-4063-8c86-2a471e602820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:02.767501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e364b-80c5-47a4-ab55-0ac7f2c94714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:02.768201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e364b-80c5-47a4-ab55-0ac7f2c94714 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:02.769141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e364b-80c5-47a4-ab55-0ac7f2c94714 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:02.769471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e364b-80c5-47a4-ab55-0ac7f2c94714 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:02.770130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e364b-80c5-47a4-ab55-0ac7f2c94714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:02.771765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e364b-80c5-47a4-ab55-0ac7f2c94714 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:02.803024 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2352/4622] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:02.804300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e364b-80c5-47a4-ab55-0ac7f2c94714 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:02.819744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e364b-80c5-47a4-ab55-0ac7f2c94714 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:02.832276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:02.833064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b8411-48b0-4b2c-a333-27585a4f5efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:02.833748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b8411-48b0-4b2c-a333-27585a4f5efa admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:02.834035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b8411-48b0-4b2c-a333-27585a4f5efa admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:02.834309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b8411-48b0-4b2c-a333-27585a4f5efa admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:02.835074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b8411-48b0-4b2c-a333-27585a4f5efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:02.837315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b8411-48b0-4b2c-a333-27585a4f5efa admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:02.852037 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2353/4623] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:02.856845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b8411-48b0-4b2c-a333-27585a4f5efa admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:02.871960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b8411-48b0-4b2c-a333-27585a4f5efa admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:02.879208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33350f27-e4f2-4140-9a02-4958e43b0cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:02.880613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33350f27-e4f2-4140-9a02-4958e43b0cc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:02.881116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33350f27-e4f2-4140-9a02-4958e43b0cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:02.881400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33350f27-e4f2-4140-9a02-4958e43b0cc8 admin admin] PATH_INFO: `/v3/projects/36f2db3af5844e7d8f8f124b8d61e382/users/e6633680445949409b03fb63b621018c/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:02.888134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9c46d-cea2-4063-8c86-2a471e602820 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:02.894227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c3e28-929b-4f12-83c0-5666259e4fdd tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6abf93f4959468caf39ce44d472b7f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'edc972be74574a12b82fe88feef1ce76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:02.895229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c3e28-929b-4f12-83c0-5666259e4fdd tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:02.895744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c3e28-929b-4f12-83c0-5666259e4fdd tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:02.896222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c3e28-929b-4f12-83c0-5666259e4fdd tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] PATH_INFO: `/v3/projects/68ca24fc2e954889872a9a9ae925d416` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:02.897605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c3e28-929b-4f12-83c0-5666259e4fdd tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] RBAC: Authorizing `identity:delete_project(project_id=68ca24fc2e954889872a9a9ae925d416)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:02.900033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c3e28-929b-4f12-83c0-5666259e4fdd tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:02.902653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33350f27-e4f2-4140-9a02-4958e43b0cc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36f2db3af5844e7d8f8f124b8d61e382, user_id=e6633680445949409b03fb63b621018c, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:02.905749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33350f27-e4f2-4140-9a02-4958e43b0cc8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:02.944638 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2354/4624] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:04:02 2022] PUT /identity/v3/projects/36f2db3af5844e7d8f8f124b8d61e382/users/e6633680445949409b03fb63b621018c/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:02.950794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107ad9a-2554-4cc6-8c3e-8a7dc806fa2c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:02.951097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107ad9a-2554-4cc6-8c3e-8a7dc806fa2c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:02.951963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107ad9a-2554-4cc6-8c3e-8a7dc806fa2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:02.972616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c107ad9a-2554-4cc6-8c3e-8a7dc806fa2c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:02.973896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c107ad9a-2554-4cc6-8c3e-8a7dc806fa2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:02.987851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ab9c3e28-929b-4f12-83c0-5666259e4fdd tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68ca24fc2e954889872a9a9ae925d416', 'request_id': 'req-ab9c3e28-929b-4f12-83c0-5666259e4fdd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:02.998940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:03.001201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ab9c3e28-929b-4f12-83c0-5666259e4fdd tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68ca24fc2e954889872a9a9ae925d416', 'request_id': 'req-ab9c3e28-929b-4f12-83c0-5666259e4fdd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:03.021257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ab9c3e28-929b-4f12-83c0-5666259e4fdd tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68ca24fc2e954889872a9a9ae925d416', 'request_id': 'req-ab9c3e28-929b-4f12-83c0-5666259e4fdd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:03.059001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ab9c3e28-929b-4f12-83c0-5666259e4fdd tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] The token cache is being invalidate because project 68ca24fc2e954889872a9a9ae925d416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:04:03.059455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ab9c3e28-929b-4f12-83c0-5666259e4fdd tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:03.060403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ab9c3e28-929b-4f12-83c0-5666259e4fdd tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:03.065628 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2271/4625] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:04:02 2022] DELETE /identity/v3/projects/68ca24fc2e954889872a9a9ae925d416 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:03.124190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c3e28-929b-4f12-83c0-5666259e4fdd tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:03.125817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ab9c3e28-929b-4f12-83c0-5666259e4fdd tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:03.134230 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2355/4626] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:03.158271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107ad9a-2554-4cc6-8c3e-8a7dc806fa2c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:03.161324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c107ad9a-2554-4cc6-8c3e-8a7dc806fa2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:03.278160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c3e28-929b-4f12-83c0-5666259e4fdd tempest-RoutersAdminTest-222371132 tempest-RoutersAdminTest-222371132-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:03.284296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511e24f-a228-4baf-b0a4-2c525e46433a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:03.287471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511e24f-a228-4baf-b0a4-2c525e46433a service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:03.287769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511e24f-a228-4baf-b0a4-2c525e46433a service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:03.288197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511e24f-a228-4baf-b0a4-2c525e46433a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:03.291838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6511e24f-a228-4baf-b0a4-2c525e46433a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:03.302522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107ad9a-2554-4cc6-8c3e-8a7dc806fa2c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:03.311154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3da74d-9af9-4095-8500-e00a80a943d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:03.311763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3da74d-9af9-4095-8500-e00a80a943d7 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:03.312042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3da74d-9af9-4095-8500-e00a80a943d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:03.312312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3da74d-9af9-4095-8500-e00a80a943d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:03.347637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3da74d-9af9-4095-8500-e00a80a943d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:03.349265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3da74d-9af9-4095-8500-e00a80a943d7 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:03.363324 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2356/4627] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:04:03.425085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3da74d-9af9-4095-8500-e00a80a943d7 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:03.447535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3da74d-9af9-4095-8500-e00a80a943d7 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:03.455635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fde1c5-d2fb-4836-887a-9e31e4eae001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:03.456396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511e24f-a228-4baf-b0a4-2c525e46433a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:03.457720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511e24f-a228-4baf-b0a4-2c525e46433a service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:03.458445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fde1c5-d2fb-4836-887a-9e31e4eae001 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:03.458909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fde1c5-d2fb-4836-887a-9e31e4eae001 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:03.461922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fde1c5-d2fb-4836-887a-9e31e4eae001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:03.465386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-02fde1c5-d2fb-4836-887a-9e31e4eae001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:03.480706 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2272/4628] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:03.495746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fde1c5-d2fb-4836-887a-9e31e4eae001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:03.497209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fde1c5-d2fb-4836-887a-9e31e4eae001 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:03.514035 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2357/4629] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:06.048397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53446ae5-a141-4372-bf8e-ea902c48c3c6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:06.049802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53446ae5-a141-4372-bf8e-ea902c48c3c6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:06.050283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53446ae5-a141-4372-bf8e-ea902c48c3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:06.102607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:06.123602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-53446ae5-a141-4372-bf8e-ea902c48c3c6 None None] MFA Rules not processed for user `e6633680445949409b03fb63b621018c`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:06.128119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-53446ae5-a141-4372-bf8e-ea902c48c3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:06.250072 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2273/4630] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:06 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:06.255076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fde1c5-d2fb-4836-887a-9e31e4eae001 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:06.257103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-02fde1c5-d2fb-4836-887a-9e31e4eae001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:06.272214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fde1c5-d2fb-4836-887a-9e31e4eae001 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:06.277332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ce526-3f4e-4c2c-a6ca-308722e26b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:06.278229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ce526-3f4e-4c2c-a6ca-308722e26b81 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:06.278746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ce526-3f4e-4c2c-a6ca-308722e26b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:06.279166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ce526-3f4e-4c2c-a6ca-308722e26b81 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:06.279960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049ce526-3f4e-4c2c-a6ca-308722e26b81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:06.282080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049ce526-3f4e-4c2c-a6ca-308722e26b81 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:06.320463 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2358/4631] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:04:06 2022] POST /identity/v3/projects => generated 401 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:06.321568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53446ae5-a141-4372-bf8e-ea902c48c3c6 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:06.339079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53446ae5-a141-4372-bf8e-ea902c48c3c6 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:06.466721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f3c34-7fa4-45e8-b015-398de79ecbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:06.467638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f3c34-7fa4-45e8-b015-398de79ecbc0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:06.468324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f3c34-7fa4-45e8-b015-398de79ecbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:06.469359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f3c34-7fa4-45e8-b015-398de79ecbc0 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:06.470134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f3c34-7fa4-45e8-b015-398de79ecbc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:06.475004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f3c34-7fa4-45e8-b015-398de79ecbc0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:06.507109 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2274/4632] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:04:06 2022] POST /identity/v3/users => generated 307 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:04:06.510839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ce526-3f4e-4c2c-a6ca-308722e26b81 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:06.528125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ce526-3f4e-4c2c-a6ca-308722e26b81 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:06.536275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9058de-0e2b-4c26-b83a-cb26807e0bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:06.539270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9058de-0e2b-4c26-b83a-cb26807e0bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:06.539777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9058de-0e2b-4c26-b83a-cb26807e0bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:06.540284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9058de-0e2b-4c26-b83a-cb26807e0bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:06.541165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9058de-0e2b-4c26-b83a-cb26807e0bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:06.544538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9058de-0e2b-4c26-b83a-cb26807e0bd6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:06.580386 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2359/4633] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:06.581403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f3c34-7fa4-45e8-b015-398de79ecbc0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:06.598080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f3c34-7fa4-45e8-b015-398de79ecbc0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:06.617255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086693f-7eaf-4116-bcad-b514fdb90819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:06.618099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086693f-7eaf-4116-bcad-b514fdb90819 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:06.618459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086693f-7eaf-4116-bcad-b514fdb90819 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:06.618829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086693f-7eaf-4116-bcad-b514fdb90819 admin admin] PATH_INFO: `/v3/projects/f960e7bf88c744468c248df8210a6576/users/80d74b345f16424dafee35c2c8b36c75/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:06.661671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086693f-7eaf-4116-bcad-b514fdb90819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f960e7bf88c744468c248df8210a6576, user_id=80d74b345f16424dafee35c2c8b36c75, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:06.670930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086693f-7eaf-4116-bcad-b514fdb90819 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:06.708030 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2275/4634] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:04:06 2022] PUT /identity/v3/projects/f960e7bf88c744468c248df8210a6576/users/80d74b345f16424dafee35c2c8b36c75/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:04:06.709473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9058de-0e2b-4c26-b83a-cb26807e0bd6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:06.726947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9058de-0e2b-4c26-b83a-cb26807e0bd6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:06.862066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9fa21-7e03-4c93-b917-097502d1e4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:06.863522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9fa21-7e03-4c93-b917-097502d1e4a5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:06.866817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9fa21-7e03-4c93-b917-097502d1e4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:06.867823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9fa21-7e03-4c93-b917-097502d1e4a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:06.868896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9fa21-7e03-4c93-b917-097502d1e4a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:06.871613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9fa21-7e03-4c93-b917-097502d1e4a5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:06.899158 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2360/4635] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:06.903765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086693f-7eaf-4116-bcad-b514fdb90819 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:06.920450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086693f-7eaf-4116-bcad-b514fdb90819 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:06.927149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a93547-e3b2-40f0-aa69-d93dbde514f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:06.929985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a93547-e3b2-40f0-aa69-d93dbde514f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:06.931722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a93547-e3b2-40f0-aa69-d93dbde514f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:06.931722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a93547-e3b2-40f0-aa69-d93dbde514f2 admin admin] PATH_INFO: `/v3/projects/f960e7bf88c744468c248df8210a6576/users/80d74b345f16424dafee35c2c8b36c75/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:06.937493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a93547-e3b2-40f0-aa69-d93dbde514f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f960e7bf88c744468c248df8210a6576, user_id=80d74b345f16424dafee35c2c8b36c75, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:06.945065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a93547-e3b2-40f0-aa69-d93dbde514f2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:06.978722 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2276/4636] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:04:06 2022] PUT /identity/v3/projects/f960e7bf88c744468c248df8210a6576/users/80d74b345f16424dafee35c2c8b36c75/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:08.399242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e93da9-90f5-4ae9-8e15-7d74f5fae145 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:08.399727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e93da9-90f5-4ae9-8e15-7d74f5fae145 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:08.399839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e93da9-90f5-4ae9-8e15-7d74f5fae145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:08.421356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b4e93da9-90f5-4ae9-8e15-7d74f5fae145 None None] MFA Rules not processed for user `aa4fbb28c9de4089870d91cf52dac8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:08.422633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b4e93da9-90f5-4ae9-8e15-7d74f5fae145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:08.444019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:08.543575 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2361/4637] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:09.488420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ba2e4-0586-4873-841a-556a127209cd None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:09.489398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ba2e4-0586-4873-841a-556a127209cd None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:09.489878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ba2e4-0586-4873-841a-556a127209cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:09.541375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bd7ba2e4-0586-4873-841a-556a127209cd None None] MFA Rules not processed for user `80d74b345f16424dafee35c2c8b36c75`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:09.543114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bd7ba2e4-0586-4873-841a-556a127209cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:09.600856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:09.649918 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2277/4638] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:09 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:10.200592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e93da9-90f5-4ae9-8e15-7d74f5fae145 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:10.202697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b4e93da9-90f5-4ae9-8e15-7d74f5fae145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:10.405364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e93da9-90f5-4ae9-8e15-7d74f5fae145 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:10.414612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ce911-5a26-4aec-9060-7ffa5bfe6074 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:10.415557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ce911-5a26-4aec-9060-7ffa5bfe6074 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:10.416020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ce911-5a26-4aec-9060-7ffa5bfe6074 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:10.416479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ce911-5a26-4aec-9060-7ffa5bfe6074 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:10.418741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-014ce911-5a26-4aec-9060-7ffa5bfe6074 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:10.469060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ce911-5a26-4aec-9060-7ffa5bfe6074 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:10.472898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ce911-5a26-4aec-9060-7ffa5bfe6074 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:10.524903 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2362/4639] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:11.623481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436a9ef-f01c-4056-89bd-8adb091a2d2d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:11.624429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436a9ef-f01c-4056-89bd-8adb091a2d2d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:11.624938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436a9ef-f01c-4056-89bd-8adb091a2d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:11.653239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6436a9ef-f01c-4056-89bd-8adb091a2d2d None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:11.655552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6436a9ef-f01c-4056-89bd-8adb091a2d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:11.766689 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2278/4640] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:04:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:11.796354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ce911-5a26-4aec-9060-7ffa5bfe6074 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:11.798353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-014ce911-5a26-4aec-9060-7ffa5bfe6074 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:11.876396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436a9ef-f01c-4056-89bd-8adb091a2d2d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:11.948511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ce911-5a26-4aec-9060-7ffa5bfe6074 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:11.957925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de58651-2c64-4765-a9d8-623f6ff98675 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:11.958588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de58651-2c64-4765-a9d8-623f6ff98675 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:11.959088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de58651-2c64-4765-a9d8-623f6ff98675 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:11.959516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de58651-2c64-4765-a9d8-623f6ff98675 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:11.970121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de58651-2c64-4765-a9d8-623f6ff98675 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:11.972223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de58651-2c64-4765-a9d8-623f6ff98675 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:11.976530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436a9ef-f01c-4056-89bd-8adb091a2d2d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:11.986194 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2363/4641] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:04:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:04:11.987678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1bf58-5670-41e3-b5fa-320a30349a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:11.990123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1bf58-5670-41e3-b5fa-320a30349a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:11.990626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1bf58-5670-41e3-b5fa-320a30349a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:11.990990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1bf58-5670-41e3-b5fa-320a30349a8e admin admin] PATH_INFO: `/v3/users/e6633680445949409b03fb63b621018c` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:11.994266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1bf58-5670-41e3-b5fa-320a30349a8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6633680445949409b03fb63b621018c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:11.994659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e74d25-4bf9-4823-977b-b29500e61390 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:11.995021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e74d25-4bf9-4823-977b-b29500e61390 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:11.995198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e74d25-4bf9-4823-977b-b29500e61390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:11.997285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1bf58-5670-41e3-b5fa-320a30349a8e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:12.070161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a3e74d25-4bf9-4823-977b-b29500e61390 None None] MFA Rules not processed for user `aad36d5e04a7453aba1e6d0b15d2d06f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:12.071827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a3e74d25-4bf9-4823-977b-b29500e61390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:12.116782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:12.120285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c6c1bf58-5670-41e3-b5fa-320a30349a8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6633680445949409b03fb63b621018c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:12.121256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c6c1bf58-5670-41e3-b5fa-320a30349a8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6633680445949409b03fb63b621018c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:12.122716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c6c1bf58-5670-41e3-b5fa-320a30349a8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6633680445949409b03fb63b621018c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:12.144641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c6c1bf58-5670-41e3-b5fa-320a30349a8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6633680445949409b03fb63b621018c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:12.174762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c6c1bf58-5670-41e3-b5fa-320a30349a8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6633680445949409b03fb63b621018c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:12.208240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:12.253267 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2279/4642] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:04:11 2022] DELETE /identity/v3/users/e6633680445949409b03fb63b621018c => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:12.288416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5c24f-6acc-42eb-a7dc-939ef24c7047 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:12.288718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5c24f-6acc-42eb-a7dc-939ef24c7047 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:12.288992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5c24f-6acc-42eb-a7dc-939ef24c7047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:12.292601 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2364/4643] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:11 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:12.328418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e7f5c24f-6acc-42eb-a7dc-939ef24c7047 None None] MFA Rules not processed for user `aad36d5e04a7453aba1e6d0b15d2d06f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:12.329603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e7f5c24f-6acc-42eb-a7dc-939ef24c7047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:12.458373 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2280/4644] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:12 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:13.126520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffb654-d508-43ef-9105-19ab353fa609 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:13.128698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffb654-d508-43ef-9105-19ab353fa609 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:13.129275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffb654-d508-43ef-9105-19ab353fa609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:13.157336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-54ffb654-d508-43ef-9105-19ab353fa609 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:13.159702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-54ffb654-d508-43ef-9105-19ab353fa609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:13.168014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5c24f-6acc-42eb-a7dc-939ef24c7047 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:13.169719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e7f5c24f-6acc-42eb-a7dc-939ef24c7047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:13.298648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5c24f-6acc-42eb-a7dc-939ef24c7047 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:13.306067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae346e92-45cf-4de6-a27b-5eee8174adf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:13.306377 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2365/4645] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:04:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:13.306732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae346e92-45cf-4de6-a27b-5eee8174adf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:13.307078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae346e92-45cf-4de6-a27b-5eee8174adf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:13.307532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae346e92-45cf-4de6-a27b-5eee8174adf1 admin admin] PATH_INFO: `/v3/projects/36f2db3af5844e7d8f8f124b8d61e382` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:13.310086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae346e92-45cf-4de6-a27b-5eee8174adf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36f2db3af5844e7d8f8f124b8d61e382)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:13.314000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae346e92-45cf-4de6-a27b-5eee8174adf1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:13.323916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffb654-d508-43ef-9105-19ab353fa609 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:13.326279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-54ffb654-d508-43ef-9105-19ab353fa609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:13.418246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ae346e92-45cf-4de6-a27b-5eee8174adf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36f2db3af5844e7d8f8f124b8d61e382', 'request_id': 'req-ae346e92-45cf-4de6-a27b-5eee8174adf1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:13.432832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffb654-d508-43ef-9105-19ab353fa609 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:13.436822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ae346e92-45cf-4de6-a27b-5eee8174adf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36f2db3af5844e7d8f8f124b8d61e382', 'request_id': 'req-ae346e92-45cf-4de6-a27b-5eee8174adf1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:13.448487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ae346e92-45cf-4de6-a27b-5eee8174adf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36f2db3af5844e7d8f8f124b8d61e382', 'request_id': 'req-ae346e92-45cf-4de6-a27b-5eee8174adf1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:13.475883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ae346e92-45cf-4de6-a27b-5eee8174adf1 admin admin] The token cache is being invalidate because project 36f2db3af5844e7d8f8f124b8d61e382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:04:13.476250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ae346e92-45cf-4de6-a27b-5eee8174adf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:13.477052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ae346e92-45cf-4de6-a27b-5eee8174adf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:13.478996 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2281/4646] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:04:13 2022] DELETE /identity/v3/projects/36f2db3af5844e7d8f8f124b8d61e382 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:13.482661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae346e92-45cf-4de6-a27b-5eee8174adf1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:13.484133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ae346e92-45cf-4de6-a27b-5eee8174adf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:13.537024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:13.546387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc33442-f76a-4023-bb49-be35ea1c1142 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:13.547833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc33442-f76a-4023-bb49-be35ea1c1142 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:13.548416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc33442-f76a-4023-bb49-be35ea1c1142 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:13.548806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc33442-f76a-4023-bb49-be35ea1c1142 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:13.552997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8bc33442-f76a-4023-bb49-be35ea1c1142 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:13.566744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:13.621701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae346e92-45cf-4de6-a27b-5eee8174adf1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:13.634864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39237018-2e85-48a8-bdc9-ce7e562a5dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:13.635924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39237018-2e85-48a8-bdc9-ce7e562a5dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:13.635924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39237018-2e85-48a8-bdc9-ce7e562a5dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:13.636132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39237018-2e85-48a8-bdc9-ce7e562a5dc6 admin admin] PATH_INFO: `/v3/users/80d74b345f16424dafee35c2c8b36c75` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:13.638635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39237018-2e85-48a8-bdc9-ce7e562a5dc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80d74b345f16424dafee35c2c8b36c75)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:13.640676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39237018-2e85-48a8-bdc9-ce7e562a5dc6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:13.723793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-39237018-2e85-48a8-bdc9-ce7e562a5dc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80d74b345f16424dafee35c2c8b36c75'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:13.725579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-39237018-2e85-48a8-bdc9-ce7e562a5dc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80d74b345f16424dafee35c2c8b36c75'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:13.726404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-39237018-2e85-48a8-bdc9-ce7e562a5dc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80d74b345f16424dafee35c2c8b36c75'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:13.726641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc33442-f76a-4023-bb49-be35ea1c1142 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:13.728613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc33442-f76a-4023-bb49-be35ea1c1142 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:13.736886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-39237018-2e85-48a8-bdc9-ce7e562a5dc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80d74b345f16424dafee35c2c8b36c75'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:13.760019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-39237018-2e85-48a8-bdc9-ce7e562a5dc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80d74b345f16424dafee35c2c8b36c75'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:13.766520 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2366/4647] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:04:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 21:04:13.778872 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2282/4648] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:04:13 2022] DELETE /identity/v3/users/80d74b345f16424dafee35c2c8b36c75 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:13.999726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc33442-f76a-4023-bb49-be35ea1c1142 service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:14.001500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8bc33442-f76a-4023-bb49-be35ea1c1142 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:14.132530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc33442-f76a-4023-bb49-be35ea1c1142 service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:14.139402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39237018-2e85-48a8-bdc9-ce7e562a5dc6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:14.141322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-39237018-2e85-48a8-bdc9-ce7e562a5dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:14.248436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f886fcc-ad40-4020-b8dd-8ff52f32e16f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:14.249367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f886fcc-ad40-4020-b8dd-8ff52f32e16f service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:14.249720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f886fcc-ad40-4020-b8dd-8ff52f32e16f service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:14.250292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f886fcc-ad40-4020-b8dd-8ff52f32e16f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:14.254304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7f886fcc-ad40-4020-b8dd-8ff52f32e16f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:14.304934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39237018-2e85-48a8-bdc9-ce7e562a5dc6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:14.310696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d69d43-fa6f-4c72-9672-9106f2aee1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:14.311001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d69d43-fa6f-4c72-9672-9106f2aee1eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:14.311226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d69d43-fa6f-4c72-9672-9106f2aee1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:14.311590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d69d43-fa6f-4c72-9672-9106f2aee1eb admin admin] PATH_INFO: `/v3/projects/f960e7bf88c744468c248df8210a6576` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:14.312762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d69d43-fa6f-4c72-9672-9106f2aee1eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f960e7bf88c744468c248df8210a6576)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:14.314769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d69d43-fa6f-4c72-9672-9106f2aee1eb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:14.319846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:14.356070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f886fcc-ad40-4020-b8dd-8ff52f32e16f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:14.358959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f886fcc-ad40-4020-b8dd-8ff52f32e16f service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:14.372361 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2367/4649] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:13 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:14.436228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-19d69d43-fa6f-4c72-9672-9106f2aee1eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f960e7bf88c744468c248df8210a6576', 'request_id': 'req-19d69d43-fa6f-4c72-9672-9106f2aee1eb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:14.448989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-19d69d43-fa6f-4c72-9672-9106f2aee1eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f960e7bf88c744468c248df8210a6576', 'request_id': 'req-19d69d43-fa6f-4c72-9672-9106f2aee1eb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:14.451112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:14.458860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-19d69d43-fa6f-4c72-9672-9106f2aee1eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f960e7bf88c744468c248df8210a6576', 'request_id': 'req-19d69d43-fa6f-4c72-9672-9106f2aee1eb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:14.491610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-19d69d43-fa6f-4c72-9672-9106f2aee1eb admin admin] The token cache is being invalidate because project f960e7bf88c744468c248df8210a6576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:04:14.492005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-19d69d43-fa6f-4c72-9672-9106f2aee1eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:14.492823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-19d69d43-fa6f-4c72-9672-9106f2aee1eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:14.494988 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2283/4650] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:04:14 2022] DELETE /identity/v3/projects/f960e7bf88c744468c248df8210a6576 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:14.500176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff43a46-4970-44b2-be10-6af3ff03a82d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:14.500474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff43a46-4970-44b2-be10-6af3ff03a82d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:14.500736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff43a46-4970-44b2-be10-6af3ff03a82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:14.531476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-aff43a46-4970-44b2-be10-6af3ff03a82d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:14.532830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-aff43a46-4970-44b2-be10-6af3ff03a82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:14.612045 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2368/4651] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:14.613259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abb2d9-4a12-41ac-9f0e-2b90a9ef5b64 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:14.613561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abb2d9-4a12-41ac-9f0e-2b90a9ef5b64 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:14.613840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abb2d9-4a12-41ac-9f0e-2b90a9ef5b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:14.643705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-00abb2d9-4a12-41ac-9f0e-2b90a9ef5b64 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:14.644698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-00abb2d9-4a12-41ac-9f0e-2b90a9ef5b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:14.654830 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2284/4652] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:14.658994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff43a46-4970-44b2-be10-6af3ff03a82d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:14.671430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff43a46-4970-44b2-be10-6af3ff03a82d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:14.680872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f8665-49d3-4011-8f5f-617c6b3f0853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:14.681854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f8665-49d3-4011-8f5f-617c6b3f0853 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:14.682361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f8665-49d3-4011-8f5f-617c6b3f0853 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:14.682885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f8665-49d3-4011-8f5f-617c6b3f0853 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:14.683806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f8665-49d3-4011-8f5f-617c6b3f0853 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:04:14.684373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f8665-49d3-4011-8f5f-617c6b3f0853 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:14.687226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f8665-49d3-4011-8f5f-617c6b3f0853 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:14.712673 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2369/4653] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:04:14.713216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abb2d9-4a12-41ac-9f0e-2b90a9ef5b64 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:14.724568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abb2d9-4a12-41ac-9f0e-2b90a9ef5b64 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:14.734164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a8552-6310-475c-b58b-d2efc9ab9fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:14.734843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a8552-6310-475c-b58b-d2efc9ab9fca admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:14.735121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a8552-6310-475c-b58b-d2efc9ab9fca admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:14.735392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a8552-6310-475c-b58b-d2efc9ab9fca admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:14.736065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a8552-6310-475c-b58b-d2efc9ab9fca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:14.738673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a8552-6310-475c-b58b-d2efc9ab9fca admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:14.787456 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2285/4654] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:04:14 2022] POST /identity/v3/projects => generated 363 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:14.789041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f8665-49d3-4011-8f5f-617c6b3f0853 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:14.803332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f8665-49d3-4011-8f5f-617c6b3f0853 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:14.942807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581318ee-9c5e-491d-9f77-cefbb0c0fb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:14.943678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581318ee-9c5e-491d-9f77-cefbb0c0fb64 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:14.943999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581318ee-9c5e-491d-9f77-cefbb0c0fb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:14.944308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581318ee-9c5e-491d-9f77-cefbb0c0fb64 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:14.945140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581318ee-9c5e-491d-9f77-cefbb0c0fb64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:14.948900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581318ee-9c5e-491d-9f77-cefbb0c0fb64 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:14.978349 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2370/4655] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:04:14 2022] POST /identity/v3/users => generated 282 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:14.983407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a8552-6310-475c-b58b-d2efc9ab9fca admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:14.999637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a8552-6310-475c-b58b-d2efc9ab9fca admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:15.021384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18f7d5-9a79-43f0-af02-90748c6cd077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:15.022442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18f7d5-9a79-43f0-af02-90748c6cd077 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:15.022864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18f7d5-9a79-43f0-af02-90748c6cd077 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:15.023617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18f7d5-9a79-43f0-af02-90748c6cd077 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:15.024377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18f7d5-9a79-43f0-af02-90748c6cd077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:15.026144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18f7d5-9a79-43f0-af02-90748c6cd077 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:15.048778 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2286/4656] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:15.049851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581318ee-9c5e-491d-9f77-cefbb0c0fb64 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:15.066565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581318ee-9c5e-491d-9f77-cefbb0c0fb64 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:15.080722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac4633-b3c2-4b85-baa4-54f602926cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:15.081494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac4633-b3c2-4b85-baa4-54f602926cea admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:15.081628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac4633-b3c2-4b85-baa4-54f602926cea admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:15.081891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac4633-b3c2-4b85-baa4-54f602926cea admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:15.082573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac4633-b3c2-4b85-baa4-54f602926cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:15.084338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac4633-b3c2-4b85-baa4-54f602926cea admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:15.115685 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2371/4657] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:15.119472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18f7d5-9a79-43f0-af02-90748c6cd077 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:15.137493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18f7d5-9a79-43f0-af02-90748c6cd077 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:15.153066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a05f7-cf0a-436f-9336-8ea5229c560c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:15.153764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a05f7-cf0a-436f-9336-8ea5229c560c admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:15.154049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a05f7-cf0a-436f-9336-8ea5229c560c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:15.154325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a05f7-cf0a-436f-9336-8ea5229c560c admin admin] PATH_INFO: `/v3/projects/0b0c3e69cf594f66aeb0f66e87aa9204/users/4951f80bc94a4700bea2b8f01544495c/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:15.206976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a05f7-cf0a-436f-9336-8ea5229c560c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b0c3e69cf594f66aeb0f66e87aa9204, user_id=4951f80bc94a4700bea2b8f01544495c, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:15.209705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a05f7-cf0a-436f-9336-8ea5229c560c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:15.247394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bc455-de1d-4fd1-aacc-40266e285951 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:15.247981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bc455-de1d-4fd1-aacc-40266e285951 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:15.247981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bc455-de1d-4fd1-aacc-40266e285951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:15.251675 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2287/4658] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:04:15 2022] PUT /identity/v3/projects/0b0c3e69cf594f66aeb0f66e87aa9204/users/4951f80bc94a4700bea2b8f01544495c/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:15.298373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-848bc455-de1d-4fd1-aacc-40266e285951 None None] MFA Rules not processed for user `4951f80bc94a4700bea2b8f01544495c`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:15.299792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-848bc455-de1d-4fd1-aacc-40266e285951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:15.379483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:15.425644 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2372/4659] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:15.430075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a05f7-cf0a-436f-9336-8ea5229c560c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:15.516430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a05f7-cf0a-436f-9336-8ea5229c560c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:15.536337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4abedf-ca66-4cec-95cf-2a9df7a8edb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:15.536337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4abedf-ca66-4cec-95cf-2a9df7a8edb0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:15.536337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4abedf-ca66-4cec-95cf-2a9df7a8edb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:15.536337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4abedf-ca66-4cec-95cf-2a9df7a8edb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:15.537555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4abedf-ca66-4cec-95cf-2a9df7a8edb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:15.540009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4abedf-ca66-4cec-95cf-2a9df7a8edb0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:15.607652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bc455-de1d-4fd1-aacc-40266e285951 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:15.607889 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2288/4660] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:04:15 2022] POST /identity/v3/projects => generated 363 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:15.625671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bc455-de1d-4fd1-aacc-40266e285951 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:15.821646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e65265-2723-416b-a1ef-0aa514243b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:15.822274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e65265-2723-416b-a1ef-0aa514243b40 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:15.822575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e65265-2723-416b-a1ef-0aa514243b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:15.822850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e65265-2723-416b-a1ef-0aa514243b40 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:15.823499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e65265-2723-416b-a1ef-0aa514243b40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:15.831404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e65265-2723-416b-a1ef-0aa514243b40 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:15.874747 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2373/4661] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:04:15 2022] POST /identity/v3/users => generated 288 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:04:15.876475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4abedf-ca66-4cec-95cf-2a9df7a8edb0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:15.895899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4abedf-ca66-4cec-95cf-2a9df7a8edb0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:15.912378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066273ca-5c33-4a55-99cf-f99edd801805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:15.913272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066273ca-5c33-4a55-99cf-f99edd801805 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:15.913764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066273ca-5c33-4a55-99cf-f99edd801805 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:15.914043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066273ca-5c33-4a55-99cf-f99edd801805 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:15.914822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066273ca-5c33-4a55-99cf-f99edd801805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:15.917086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066273ca-5c33-4a55-99cf-f99edd801805 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:15.948310 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2289/4662] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:15.948770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e65265-2723-416b-a1ef-0aa514243b40 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:15.961513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e65265-2723-416b-a1ef-0aa514243b40 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:15.977500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c6126-4707-4265-8c29-25e52e741c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:15.978377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c6126-4707-4265-8c29-25e52e741c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:15.981733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c6126-4707-4265-8c29-25e52e741c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:15.982144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c6126-4707-4265-8c29-25e52e741c84 admin admin] PATH_INFO: `/v3/projects/742bd8880fda487caab443468919f5f1/users/71e592e4c0d8411797d5d5ad53906385/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:16.025091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c6126-4707-4265-8c29-25e52e741c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=742bd8880fda487caab443468919f5f1, user_id=71e592e4c0d8411797d5d5ad53906385, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:16.032179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c6126-4707-4265-8c29-25e52e741c84 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:16.105052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066273ca-5c33-4a55-99cf-f99edd801805 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:16.118353 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2374/4663] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:04:15 2022] PUT /identity/v3/projects/742bd8880fda487caab443468919f5f1/users/71e592e4c0d8411797d5d5ad53906385/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:16.129463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066273ca-5c33-4a55-99cf-f99edd801805 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:16.167348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb6490-7fc8-4c15-8550-63743747d4e2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:16.167633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb6490-7fc8-4c15-8550-63743747d4e2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:16.167910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb6490-7fc8-4c15-8550-63743747d4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:16.224283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6feb6490-7fc8-4c15-8550-63743747d4e2 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:16.228194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6feb6490-7fc8-4c15-8550-63743747d4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:16.305907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:16.326894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961581fc-42b0-4a45-946a-391c7c25d894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:16.328477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961581fc-42b0-4a45-946a-391c7c25d894 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:16.328764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961581fc-42b0-4a45-946a-391c7c25d894 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:16.329140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961581fc-42b0-4a45-946a-391c7c25d894 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:16.329858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961581fc-42b0-4a45-946a-391c7c25d894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:16.337062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961581fc-42b0-4a45-946a-391c7c25d894 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:16.366805 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2290/4664] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:16.373548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961581fc-42b0-4a45-946a-391c7c25d894 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:16.397395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961581fc-42b0-4a45-946a-391c7c25d894 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:16.400662 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2375/4665] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:04:16.412515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb6490-7fc8-4c15-8550-63743747d4e2 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:16.414007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6feb6490-7fc8-4c15-8550-63743747d4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:16.423631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde5e5c-ad67-4281-8300-8a9ae52e3e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:16.424376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde5e5c-ad67-4281-8300-8a9ae52e3e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:16.424703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde5e5c-ad67-4281-8300-8a9ae52e3e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:16.425023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde5e5c-ad67-4281-8300-8a9ae52e3e63 admin admin] PATH_INFO: `/v3/projects/742bd8880fda487caab443468919f5f1/users/71e592e4c0d8411797d5d5ad53906385/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:16.427157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde5e5c-ad67-4281-8300-8a9ae52e3e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=742bd8880fda487caab443468919f5f1, user_id=71e592e4c0d8411797d5d5ad53906385, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:16.429871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde5e5c-ad67-4281-8300-8a9ae52e3e63 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:16.449049 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2291/4666] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:04:16 2022] PUT /identity/v3/projects/742bd8880fda487caab443468919f5f1/users/71e592e4c0d8411797d5d5ad53906385/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:16.460364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53325a37-b7fc-4976-804f-3892bd0f5187 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:16.461169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53325a37-b7fc-4976-804f-3892bd0f5187 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:16.461169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53325a37-b7fc-4976-804f-3892bd0f5187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:16.550586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-53325a37-b7fc-4976-804f-3892bd0f5187 None None] MFA Rules not processed for user `71e592e4c0d8411797d5d5ad53906385`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:16.552049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-53325a37-b7fc-4976-804f-3892bd0f5187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:16.553065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:16.604836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb6490-7fc8-4c15-8550-63743747d4e2 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:16.707092 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2292/4667] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:04:16.718998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53325a37-b7fc-4976-804f-3892bd0f5187 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:16.720468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-53325a37-b7fc-4976-804f-3892bd0f5187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:16.747577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2d5c7-9e66-40fb-ae46-932550ea5052 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:16.748169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2d5c7-9e66-40fb-ae46-932550ea5052 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:16.748463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2d5c7-9e66-40fb-ae46-932550ea5052 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:16.748757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2d5c7-9e66-40fb-ae46-932550ea5052 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:16.845190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2d5c7-9e66-40fb-ae46-932550ea5052 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:16.845640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2d5c7-9e66-40fb-ae46-932550ea5052 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:16.858823 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2376/4668] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:04:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 21:04:16.876473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53325a37-b7fc-4976-804f-3892bd0f5187 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:16.881413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527fb5c-554c-4ec3-bf6e-239b9c65f3b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:16.882105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527fb5c-554c-4ec3-bf6e-239b9c65f3b2 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:16.882638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527fb5c-554c-4ec3-bf6e-239b9c65f3b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:16.882955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527fb5c-554c-4ec3-bf6e-239b9c65f3b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:16.897865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527fb5c-554c-4ec3-bf6e-239b9c65f3b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:16.899358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527fb5c-554c-4ec3-bf6e-239b9c65f3b2 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:16.928471 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2293/4669] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:04:17.019680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2d5c7-9e66-40fb-ae46-932550ea5052 service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:17.022389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2ad2d5c7-9e66-40fb-ae46-932550ea5052 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:17.114771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527fb5c-554c-4ec3-bf6e-239b9c65f3b2 service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:17.127260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527fb5c-554c-4ec3-bf6e-239b9c65f3b2 service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:17.131802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6936c37-7ee4-4f71-a1f2-214f7487b77c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:17.132508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6936c37-7ee4-4f71-a1f2-214f7487b77c service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:17.132864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6936c37-7ee4-4f71-a1f2-214f7487b77c service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:17.133272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6936c37-7ee4-4f71-a1f2-214f7487b77c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:17.207016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:17.256601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6936c37-7ee4-4f71-a1f2-214f7487b77c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:17.258754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6936c37-7ee4-4f71-a1f2-214f7487b77c service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:17.276005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2d5c7-9e66-40fb-ae46-932550ea5052 service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:17.282027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489f8cf-ada4-43ef-8624-9a6e7fb9f1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:17.282913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489f8cf-ada4-43ef-8624-9a6e7fb9f1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:17.283502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489f8cf-ada4-43ef-8624-9a6e7fb9f1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:17.283882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489f8cf-ada4-43ef-8624-9a6e7fb9f1e4 admin admin] PATH_INFO: `/v3/users/779cdf8245f7483881c218306e3a76cb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:17.291915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0489f8cf-ada4-43ef-8624-9a6e7fb9f1e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=779cdf8245f7483881c218306e3a76cb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:17.294890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0489f8cf-ada4-43ef-8624-9a6e7fb9f1e4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:17.295170 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2294/4670] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:04:17.368908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:17.396483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0489f8cf-ada4-43ef-8624-9a6e7fb9f1e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '779cdf8245f7483881c218306e3a76cb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:17.428800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0489f8cf-ada4-43ef-8624-9a6e7fb9f1e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '779cdf8245f7483881c218306e3a76cb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:17.453168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0489f8cf-ada4-43ef-8624-9a6e7fb9f1e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '779cdf8245f7483881c218306e3a76cb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:17.454676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0489f8cf-ada4-43ef-8624-9a6e7fb9f1e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '779cdf8245f7483881c218306e3a76cb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:17.482839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0489f8cf-ada4-43ef-8624-9a6e7fb9f1e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '779cdf8245f7483881c218306e3a76cb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:17.503706 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2377/4671] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:04:17 2022] DELETE /identity/v3/users/779cdf8245f7483881c218306e3a76cb => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:17.505914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaee658-0c28-41d9-84be-b805346a763f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:17.505914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaee658-0c28-41d9-84be-b805346a763f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:17.506356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaee658-0c28-41d9-84be-b805346a763f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:17.562981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ecaee658-0c28-41d9-84be-b805346a763f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:17.566962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ecaee658-0c28-41d9-84be-b805346a763f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:17.715815 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2295/4672] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:17.722561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489f8cf-ada4-43ef-8624-9a6e7fb9f1e4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:17.724385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0489f8cf-ada4-43ef-8624-9a6e7fb9f1e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:17.881831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489f8cf-ada4-43ef-8624-9a6e7fb9f1e4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:17.887107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d641e7b-f6b8-4092-870c-ca328a6f02df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:17.888841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d641e7b-f6b8-4092-870c-ca328a6f02df service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:17.889262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d641e7b-f6b8-4092-870c-ca328a6f02df service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:17.889550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d641e7b-f6b8-4092-870c-ca328a6f02df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:17.901147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d641e7b-f6b8-4092-870c-ca328a6f02df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:17.902684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d641e7b-f6b8-4092-870c-ca328a6f02df service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:17.925943 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2378/4673] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:18.983071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaee658-0c28-41d9-84be-b805346a763f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:18.984601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ecaee658-0c28-41d9-84be-b805346a763f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:19.008848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaee658-0c28-41d9-84be-b805346a763f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:19.017181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710afc9d-4f41-4ee4-9042-23b70f1f8294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:19.018561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710afc9d-4f41-4ee4-9042-23b70f1f8294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:19.019798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710afc9d-4f41-4ee4-9042-23b70f1f8294 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:19.020302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710afc9d-4f41-4ee4-9042-23b70f1f8294 admin admin] PATH_INFO: `/v3/projects/600066b2cda643488c4a2965d5b20661` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:19.021782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710afc9d-4f41-4ee4-9042-23b70f1f8294 admin admin] RBAC: Authorizing `identity:delete_project(project_id=600066b2cda643488c4a2965d5b20661)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:19.024423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710afc9d-4f41-4ee4-9042-23b70f1f8294 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:19.184248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-710afc9d-4f41-4ee4-9042-23b70f1f8294 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '600066b2cda643488c4a2965d5b20661', 'request_id': 'req-710afc9d-4f41-4ee4-9042-23b70f1f8294'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:19.204769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-710afc9d-4f41-4ee4-9042-23b70f1f8294 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '600066b2cda643488c4a2965d5b20661', 'request_id': 'req-710afc9d-4f41-4ee4-9042-23b70f1f8294'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:19.212465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-710afc9d-4f41-4ee4-9042-23b70f1f8294 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '600066b2cda643488c4a2965d5b20661', 'request_id': 'req-710afc9d-4f41-4ee4-9042-23b70f1f8294'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:19.249550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-710afc9d-4f41-4ee4-9042-23b70f1f8294 admin admin] The token cache is being invalidate because project 600066b2cda643488c4a2965d5b20661 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:04:19.249882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-710afc9d-4f41-4ee4-9042-23b70f1f8294 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:19.254949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-710afc9d-4f41-4ee4-9042-23b70f1f8294 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:19.256648 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2296/4674] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:04:18 2022] DELETE /identity/v3/projects/600066b2cda643488c4a2965d5b20661 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:19.262616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56381a-ae4f-43b0-918a-5eb484d22da0 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:19.263852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56381a-ae4f-43b0-918a-5eb484d22da0 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:19.264402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56381a-ae4f-43b0-918a-5eb484d22da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:19.305802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8f56381a-ae4f-43b0-918a-5eb484d22da0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:19.307055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8f56381a-ae4f-43b0-918a-5eb484d22da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:19.500529 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2379/4675] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:19.502892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f06bb6-3fa2-49fa-8d79-15acf7d35011 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:19.503356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f06bb6-3fa2-49fa-8d79-15acf7d35011 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:19.503478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f06bb6-3fa2-49fa-8d79-15acf7d35011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:19.536664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-61f06bb6-3fa2-49fa-8d79-15acf7d35011 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:19.538231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-61f06bb6-3fa2-49fa-8d79-15acf7d35011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:19.550152 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2297/4676] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:19.554780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56381a-ae4f-43b0-918a-5eb484d22da0 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:19.569742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56381a-ae4f-43b0-918a-5eb484d22da0 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:19.581961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d44a8-03b6-4e49-bba0-3e43ffbe9d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:19.583468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d44a8-03b6-4e49-bba0-3e43ffbe9d1b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:19.583957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d44a8-03b6-4e49-bba0-3e43ffbe9d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:19.585179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d44a8-03b6-4e49-bba0-3e43ffbe9d1b admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:19.585510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d44a8-03b6-4e49-bba0-3e43ffbe9d1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:04:19.585919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d44a8-03b6-4e49-bba0-3e43ffbe9d1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:19.588023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d44a8-03b6-4e49-bba0-3e43ffbe9d1b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:19.613827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f06bb6-3fa2-49fa-8d79-15acf7d35011 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:19.616416 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2380/4677] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:04:19.624063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f06bb6-3fa2-49fa-8d79-15acf7d35011 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:19.631146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0bbf0-c7f8-46c2-8c9f-8f299f7b8ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:19.632205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0bbf0-c7f8-46c2-8c9f-8f299f7b8ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:19.632205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0bbf0-c7f8-46c2-8c9f-8f299f7b8ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:19.632434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0bbf0-c7f8-46c2-8c9f-8f299f7b8ff6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:19.632979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0bbf0-c7f8-46c2-8c9f-8f299f7b8ff6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:19.635005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0bbf0-c7f8-46c2-8c9f-8f299f7b8ff6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:19.689893 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2298/4678] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:04:19 2022] POST /identity/v3/projects => generated 359 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:19.690791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d44a8-03b6-4e49-bba0-3e43ffbe9d1b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:19.706644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d44a8-03b6-4e49-bba0-3e43ffbe9d1b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:19.860973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:19.930254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fe452-53ca-4cc6-a493-c3fd96d23c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:19.930931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fe452-53ca-4cc6-a493-c3fd96d23c61 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:19.931192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fe452-53ca-4cc6-a493-c3fd96d23c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:19.931423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fe452-53ca-4cc6-a493-c3fd96d23c61 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:19.932362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fe452-53ca-4cc6-a493-c3fd96d23c61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:19.936793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fe452-53ca-4cc6-a493-c3fd96d23c61 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:19.991961 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2381/4679] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:04:19 2022] POST /identity/v3/users => generated 280 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:19.997051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0bbf0-c7f8-46c2-8c9f-8f299f7b8ff6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:20.010156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0bbf0-c7f8-46c2-8c9f-8f299f7b8ff6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:20.023944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff14b9-33c9-4cd0-a0cb-8389239df304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:20.036156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff14b9-33c9-4cd0-a0cb-8389239df304 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:20.036721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff14b9-33c9-4cd0-a0cb-8389239df304 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:20.037243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff14b9-33c9-4cd0-a0cb-8389239df304 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:20.038163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff14b9-33c9-4cd0-a0cb-8389239df304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:20.049813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff14b9-33c9-4cd0-a0cb-8389239df304 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:20.078977 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2299/4680] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:20.079698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fe452-53ca-4cc6-a493-c3fd96d23c61 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:20.102946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fe452-53ca-4cc6-a493-c3fd96d23c61 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:20.110282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56317a07-17ee-43b9-a359-d4c48670f947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:20.112181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56317a07-17ee-43b9-a359-d4c48670f947 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:20.112463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56317a07-17ee-43b9-a359-d4c48670f947 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:20.112765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56317a07-17ee-43b9-a359-d4c48670f947 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:20.113611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56317a07-17ee-43b9-a359-d4c48670f947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:20.117700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56317a07-17ee-43b9-a359-d4c48670f947 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:20.155176 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2382/4681] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:04:20.157288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff14b9-33c9-4cd0-a0cb-8389239df304 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:20.171034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff14b9-33c9-4cd0-a0cb-8389239df304 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:20.187539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68221d-7bcf-4658-b72a-0ac6500cf2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:20.188509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68221d-7bcf-4658-b72a-0ac6500cf2c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:20.189055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68221d-7bcf-4658-b72a-0ac6500cf2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:20.190079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68221d-7bcf-4658-b72a-0ac6500cf2c7 admin admin] PATH_INFO: `/v3/projects/fde22a54f2b34f6a895047b998263d6c/users/8092a093112d413189ce69141fe72920/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:20.214890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68221d-7bcf-4658-b72a-0ac6500cf2c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fde22a54f2b34f6a895047b998263d6c, user_id=8092a093112d413189ce69141fe72920, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:20.218627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68221d-7bcf-4658-b72a-0ac6500cf2c7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:20.242126 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2300/4682] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:04:20 2022] PUT /identity/v3/projects/fde22a54f2b34f6a895047b998263d6c/users/8092a093112d413189ce69141fe72920/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:20.259646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a7f52-b134-411a-a10f-341b0eceb0fb None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:20.260340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a7f52-b134-411a-a10f-341b0eceb0fb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:20.260990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a7f52-b134-411a-a10f-341b0eceb0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:20.308500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2f9a7f52-b134-411a-a10f-341b0eceb0fb None None] MFA Rules not processed for user `8092a093112d413189ce69141fe72920`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:20.310238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2f9a7f52-b134-411a-a10f-341b0eceb0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:20.547169 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2383/4683] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:20 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:20.756263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68221d-7bcf-4658-b72a-0ac6500cf2c7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:20.762862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5e68221d-7bcf-4658-b72a-0ac6500cf2c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:20.773456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:20.892888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68221d-7bcf-4658-b72a-0ac6500cf2c7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:20.901490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2fd87-e104-4ec8-9bb4-fd68d0804df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:20.902152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2fd87-e104-4ec8-9bb4-fd68d0804df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:20.902427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2fd87-e104-4ec8-9bb4-fd68d0804df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:20.902724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2fd87-e104-4ec8-9bb4-fd68d0804df7 admin admin] PATH_INFO: `/v3/users/4951f80bc94a4700bea2b8f01544495c` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:20.908954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2fd87-e104-4ec8-9bb4-fd68d0804df7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4951f80bc94a4700bea2b8f01544495c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:20.912090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2fd87-e104-4ec8-9bb4-fd68d0804df7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:20.998734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dbd2fd87-e104-4ec8-9bb4-fd68d0804df7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4951f80bc94a4700bea2b8f01544495c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:20.999371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dbd2fd87-e104-4ec8-9bb4-fd68d0804df7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4951f80bc94a4700bea2b8f01544495c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:21.000020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dbd2fd87-e104-4ec8-9bb4-fd68d0804df7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4951f80bc94a4700bea2b8f01544495c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:21.014825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dbd2fd87-e104-4ec8-9bb4-fd68d0804df7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4951f80bc94a4700bea2b8f01544495c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:21.031614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dbd2fd87-e104-4ec8-9bb4-fd68d0804df7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4951f80bc94a4700bea2b8f01544495c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:21.078929 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2301/4684] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:04:20 2022] DELETE /identity/v3/users/4951f80bc94a4700bea2b8f01544495c => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:21.083894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d526c-f8be-4442-847b-3e6fb8f3b47b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:21.084605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d526c-f8be-4442-847b-3e6fb8f3b47b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:21.084605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d526c-f8be-4442-847b-3e6fb8f3b47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:21.179775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ad4d526c-f8be-4442-847b-3e6fb8f3b47b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:21.184365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ad4d526c-f8be-4442-847b-3e6fb8f3b47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:21.311009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2fd87-e104-4ec8-9bb4-fd68d0804df7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:21.313648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-dbd2fd87-e104-4ec8-9bb4-fd68d0804df7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:21.383829 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2384/4685] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:21.395492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d526c-f8be-4442-847b-3e6fb8f3b47b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:21.397381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ad4d526c-f8be-4442-847b-3e6fb8f3b47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:21.428683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2fd87-e104-4ec8-9bb4-fd68d0804df7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:21.438834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ad134-8307-4691-bb21-3ad7d70e364e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:21.440561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ad134-8307-4691-bb21-3ad7d70e364e service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:21.440848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ad134-8307-4691-bb21-3ad7d70e364e service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:21.441208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ad134-8307-4691-bb21-3ad7d70e364e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:21.443790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5b3ad134-8307-4691-bb21-3ad7d70e364e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:21.472082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ad134-8307-4691-bb21-3ad7d70e364e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:21.475247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ad134-8307-4691-bb21-3ad7d70e364e service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:21.494167 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2302/4686] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:04:21.515994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:21.526759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d526c-f8be-4442-847b-3e6fb8f3b47b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:21.537739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c9c71-18ae-4af8-9e63-49705ffe9f8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:21.538731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c9c71-18ae-4af8-9e63-49705ffe9f8f service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:21.539192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c9c71-18ae-4af8-9e63-49705ffe9f8f service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:21.539638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c9c71-18ae-4af8-9e63-49705ffe9f8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:21.553665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ad134-8307-4691-bb21-3ad7d70e364e service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:21.555163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c9c71-18ae-4af8-9e63-49705ffe9f8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:21.556571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c9c71-18ae-4af8-9e63-49705ffe9f8f service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:21.571003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ad134-8307-4691-bb21-3ad7d70e364e service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:21.572325 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2385/4687] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:21.578425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e89ca-78e3-4f91-ab75-dc6a76c15303 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:21.579499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e89ca-78e3-4f91-ab75-dc6a76c15303 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:21.579499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e89ca-78e3-4f91-ab75-dc6a76c15303 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:21.579897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e89ca-78e3-4f91-ab75-dc6a76c15303 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:21.582357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-620e89ca-78e3-4f91-ab75-dc6a76c15303 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:21.588966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:21.655210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e89ca-78e3-4f91-ab75-dc6a76c15303 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:21.656985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e89ca-78e3-4f91-ab75-dc6a76c15303 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:21.668675 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2303/4688] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:21 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:04:21.723539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c9c71-18ae-4af8-9e63-49705ffe9f8f service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:21.726036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cb7c9c71-18ae-4af8-9e63-49705ffe9f8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:21.741847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c9c71-18ae-4af8-9e63-49705ffe9f8f service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:21.748375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab53aef-d788-495e-94a3-fb2b662aa0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:21.749013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab53aef-d788-495e-94a3-fb2b662aa0b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:21.749300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab53aef-d788-495e-94a3-fb2b662aa0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:21.749564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab53aef-d788-495e-94a3-fb2b662aa0b2 admin admin] PATH_INFO: `/v3/users/8092a093112d413189ce69141fe72920` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:21.750789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab53aef-d788-495e-94a3-fb2b662aa0b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8092a093112d413189ce69141fe72920)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:21.752931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab53aef-d788-495e-94a3-fb2b662aa0b2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:21.843320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0ab53aef-d788-495e-94a3-fb2b662aa0b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8092a093112d413189ce69141fe72920'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:21.854092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0ab53aef-d788-495e-94a3-fb2b662aa0b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8092a093112d413189ce69141fe72920'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:21.874419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0ab53aef-d788-495e-94a3-fb2b662aa0b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8092a093112d413189ce69141fe72920'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:21.875389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0ab53aef-d788-495e-94a3-fb2b662aa0b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8092a093112d413189ce69141fe72920'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:21.889285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0ab53aef-d788-495e-94a3-fb2b662aa0b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8092a093112d413189ce69141fe72920'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:21.899869 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2386/4689] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:04:21 2022] DELETE /identity/v3/users/8092a093112d413189ce69141fe72920 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:21.904168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ae8c6-dfd1-4a9b-8677-335ce46f8a1f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:21.904452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ae8c6-dfd1-4a9b-8677-335ce46f8a1f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:21.904729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ae8c6-dfd1-4a9b-8677-335ce46f8a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:21.943692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bc5ae8c6-dfd1-4a9b-8677-335ce46f8a1f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:21.945057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bc5ae8c6-dfd1-4a9b-8677-335ce46f8a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:21.999151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab53aef-d788-495e-94a3-fb2b662aa0b2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:22.000415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0ab53aef-d788-495e-94a3-fb2b662aa0b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:22.089977 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2304/4690] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:22.091588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ae8c6-dfd1-4a9b-8677-335ce46f8a1f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:22.093577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bc5ae8c6-dfd1-4a9b-8677-335ce46f8a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:22.112847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab53aef-d788-495e-94a3-fb2b662aa0b2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:22.119422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f44af9-0720-4aaf-971b-73a665ff37c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:22.120092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f44af9-0720-4aaf-971b-73a665ff37c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:22.120590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f44af9-0720-4aaf-971b-73a665ff37c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:22.120908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f44af9-0720-4aaf-971b-73a665ff37c5 admin admin] PATH_INFO: `/v3/projects/0b0c3e69cf594f66aeb0f66e87aa9204` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:22.122827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f44af9-0720-4aaf-971b-73a665ff37c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b0c3e69cf594f66aeb0f66e87aa9204)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:22.125081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f44af9-0720-4aaf-971b-73a665ff37c5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:22.187803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ae8c6-dfd1-4a9b-8677-335ce46f8a1f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:22.193785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c2f44af9-0720-4aaf-971b-73a665ff37c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b0c3e69cf594f66aeb0f66e87aa9204', 'request_id': 'req-c2f44af9-0720-4aaf-971b-73a665ff37c5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:22.201589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c2f44af9-0720-4aaf-971b-73a665ff37c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b0c3e69cf594f66aeb0f66e87aa9204', 'request_id': 'req-c2f44af9-0720-4aaf-971b-73a665ff37c5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:22.207682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c2f44af9-0720-4aaf-971b-73a665ff37c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b0c3e69cf594f66aeb0f66e87aa9204', 'request_id': 'req-c2f44af9-0720-4aaf-971b-73a665ff37c5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:22.209678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:22.237096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c2f44af9-0720-4aaf-971b-73a665ff37c5 admin admin] The token cache is being invalidate because project 0b0c3e69cf594f66aeb0f66e87aa9204 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:04:22.237803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c2f44af9-0720-4aaf-971b-73a665ff37c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:22.239773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c2f44af9-0720-4aaf-971b-73a665ff37c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:22.242652 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2387/4691] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:04:21 2022] DELETE /identity/v3/projects/0b0c3e69cf594f66aeb0f66e87aa9204 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:04:22.244407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f44af9-0720-4aaf-971b-73a665ff37c5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:22.246141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c2f44af9-0720-4aaf-971b-73a665ff37c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:22.255034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:22.285590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897a7c4-d2db-434e-8fb7-ce0dbfb84124 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:22.286278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897a7c4-d2db-434e-8fb7-ce0dbfb84124 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:22.286772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897a7c4-d2db-434e-8fb7-ce0dbfb84124 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:22.287694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897a7c4-d2db-434e-8fb7-ce0dbfb84124 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:22.295443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2897a7c4-d2db-434e-8fb7-ce0dbfb84124 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:22.312684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897a7c4-d2db-434e-8fb7-ce0dbfb84124 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:22.314906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897a7c4-d2db-434e-8fb7-ce0dbfb84124 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:22.328941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f44af9-0720-4aaf-971b-73a665ff37c5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:22.332663 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2305/4692] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:22.333681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8b3ca-b97a-4b6c-86f8-5c89971de700 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:22.334044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897a7c4-d2db-434e-8fb7-ce0dbfb84124 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:22.334523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8b3ca-b97a-4b6c-86f8-5c89971de700 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:22.334878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8b3ca-b97a-4b6c-86f8-5c89971de700 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:22.335340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8b3ca-b97a-4b6c-86f8-5c89971de700 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:22.337194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0ba8b3ca-b97a-4b6c-86f8-5c89971de700 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:22.352180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897a7c4-d2db-434e-8fb7-ce0dbfb84124 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:22.368344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f685053-78d4-4ed1-a0e0-ddef748a1aee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:22.369159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f685053-78d4-4ed1-a0e0-ddef748a1aee service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:22.369477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f685053-78d4-4ed1-a0e0-ddef748a1aee service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:22.370150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f685053-78d4-4ed1-a0e0-ddef748a1aee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:22.373763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6f685053-78d4-4ed1-a0e0-ddef748a1aee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:22.400471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f685053-78d4-4ed1-a0e0-ddef748a1aee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:22.401934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f685053-78d4-4ed1-a0e0-ddef748a1aee service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:22.430886 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2306/4693] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:22.432245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f685053-78d4-4ed1-a0e0-ddef748a1aee service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:22.434806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6f685053-78d4-4ed1-a0e0-ddef748a1aee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:22.441610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8b3ca-b97a-4b6c-86f8-5c89971de700 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:22.443919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8b3ca-b97a-4b6c-86f8-5c89971de700 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:22.465829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f685053-78d4-4ed1-a0e0-ddef748a1aee service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:22.471227 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2388/4694] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:22.473065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02ad96-f773-469c-adf9-e6329e2667cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:22.473709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02ad96-f773-469c-adf9-e6329e2667cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:22.473986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02ad96-f773-469c-adf9-e6329e2667cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:22.474257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02ad96-f773-469c-adf9-e6329e2667cf admin admin] PATH_INFO: `/v3/users/71e592e4c0d8411797d5d5ad53906385` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:22.475440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02ad96-f773-469c-adf9-e6329e2667cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=71e592e4c0d8411797d5d5ad53906385)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:22.478774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02ad96-f773-469c-adf9-e6329e2667cf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:22.551727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8f02ad96-f773-469c-adf9-e6329e2667cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71e592e4c0d8411797d5d5ad53906385'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:22.552543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8f02ad96-f773-469c-adf9-e6329e2667cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71e592e4c0d8411797d5d5ad53906385'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:22.553285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8f02ad96-f773-469c-adf9-e6329e2667cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71e592e4c0d8411797d5d5ad53906385'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:22.567015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8f02ad96-f773-469c-adf9-e6329e2667cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71e592e4c0d8411797d5d5ad53906385'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:22.587714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8f02ad96-f773-469c-adf9-e6329e2667cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71e592e4c0d8411797d5d5ad53906385'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:22.618997 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2307/4695] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:04:22 2022] DELETE /identity/v3/users/71e592e4c0d8411797d5d5ad53906385 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:23.051773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8b3ca-b97a-4b6c-86f8-5c89971de700 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:23.064146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8b3ca-b97a-4b6c-86f8-5c89971de700 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:23.174258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8441541-f697-48d6-ae12-5a6da7699b6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:23.175530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8441541-f697-48d6-ae12-5a6da7699b6b service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:23.176932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8441541-f697-48d6-ae12-5a6da7699b6b service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:23.178265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8441541-f697-48d6-ae12-5a6da7699b6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:23.181661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d8441541-f697-48d6-ae12-5a6da7699b6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:23.295742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:23.313998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8441541-f697-48d6-ae12-5a6da7699b6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:23.316157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8441541-f697-48d6-ae12-5a6da7699b6b service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:23.330717 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2389/4696] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:23.482398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02ad96-f773-469c-adf9-e6329e2667cf admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:23.484557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8f02ad96-f773-469c-adf9-e6329e2667cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:23.584037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02ad96-f773-469c-adf9-e6329e2667cf admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:23.592068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0918007-a436-4c4f-bde6-d39d77652f96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:23.593007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0918007-a436-4c4f-bde6-d39d77652f96 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:23.593007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0918007-a436-4c4f-bde6-d39d77652f96 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:23.593298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0918007-a436-4c4f-bde6-d39d77652f96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:23.596995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b0918007-a436-4c4f-bde6-d39d77652f96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:23.620565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0918007-a436-4c4f-bde6-d39d77652f96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:23.622240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0918007-a436-4c4f-bde6-d39d77652f96 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:23.629255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8441541-f697-48d6-ae12-5a6da7699b6b service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:23.631926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d8441541-f697-48d6-ae12-5a6da7699b6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:23.639736 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2308/4697] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:23.648346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8441541-f697-48d6-ae12-5a6da7699b6b service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:23.654046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3aa7e-ee69-4379-8c68-fb814a47f409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:23.654912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3aa7e-ee69-4379-8c68-fb814a47f409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:23.655266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3aa7e-ee69-4379-8c68-fb814a47f409 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:23.655541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3aa7e-ee69-4379-8c68-fb814a47f409 admin admin] PATH_INFO: `/v3/projects/fde22a54f2b34f6a895047b998263d6c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:23.657334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3aa7e-ee69-4379-8c68-fb814a47f409 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fde22a54f2b34f6a895047b998263d6c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:23.666196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3aa7e-ee69-4379-8c68-fb814a47f409 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:23.768180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3ca3aa7e-ee69-4379-8c68-fb814a47f409 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fde22a54f2b34f6a895047b998263d6c', 'request_id': 'req-3ca3aa7e-ee69-4379-8c68-fb814a47f409'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:23.786004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3ca3aa7e-ee69-4379-8c68-fb814a47f409 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fde22a54f2b34f6a895047b998263d6c', 'request_id': 'req-3ca3aa7e-ee69-4379-8c68-fb814a47f409'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:23.794504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3ca3aa7e-ee69-4379-8c68-fb814a47f409 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fde22a54f2b34f6a895047b998263d6c', 'request_id': 'req-3ca3aa7e-ee69-4379-8c68-fb814a47f409'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:23.835954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3ca3aa7e-ee69-4379-8c68-fb814a47f409 admin admin] The token cache is being invalidate because project fde22a54f2b34f6a895047b998263d6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:04:23.836232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3ca3aa7e-ee69-4379-8c68-fb814a47f409 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:23.837058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3ca3aa7e-ee69-4379-8c68-fb814a47f409 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:23.839902 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2390/4698] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:04:23 2022] DELETE /identity/v3/projects/fde22a54f2b34f6a895047b998263d6c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:04:23.843988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc88b2-5897-41ab-8f05-22e2548c8b2d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:23.844284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc88b2-5897-41ab-8f05-22e2548c8b2d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:23.844563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc88b2-5897-41ab-8f05-22e2548c8b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:23.867951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9acc88b2-5897-41ab-8f05-22e2548c8b2d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:23.869347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9acc88b2-5897-41ab-8f05-22e2548c8b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:23.883614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:23.913117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3aa7e-ee69-4379-8c68-fb814a47f409 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:23.915859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3ca3aa7e-ee69-4379-8c68-fb814a47f409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:23.989504 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2309/4699] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:23.997158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf83278-2967-4deb-89f3-c5c8088da7ef None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:23.998843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf83278-2967-4deb-89f3-c5c8088da7ef None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:23.999353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf83278-2967-4deb-89f3-c5c8088da7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:24.028621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1cf83278-2967-4deb-89f3-c5c8088da7ef None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:24.030237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1cf83278-2967-4deb-89f3-c5c8088da7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:24.050244 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2310/4700] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:24.061265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf83278-2967-4deb-89f3-c5c8088da7ef None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:24.071210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf83278-2967-4deb-89f3-c5c8088da7ef None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:24.077573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06146428-66c4-461b-a8dc-100c9ce48b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:24.078865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06146428-66c4-461b-a8dc-100c9ce48b03 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:24.079147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06146428-66c4-461b-a8dc-100c9ce48b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:24.079637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06146428-66c4-461b-a8dc-100c9ce48b03 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:24.080971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06146428-66c4-461b-a8dc-100c9ce48b03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:04:24.081282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06146428-66c4-461b-a8dc-100c9ce48b03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:24.083666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06146428-66c4-461b-a8dc-100c9ce48b03 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:24.086924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3aa7e-ee69-4379-8c68-fb814a47f409 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:24.095075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bafe854-abf4-4d75-a985-84c625920f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:24.096632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bafe854-abf4-4d75-a985-84c625920f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:24.097043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bafe854-abf4-4d75-a985-84c625920f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:24.098008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bafe854-abf4-4d75-a985-84c625920f8b admin admin] PATH_INFO: `/v3/projects/742bd8880fda487caab443468919f5f1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:24.098659 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2311/4701] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:04:24.102883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06146428-66c4-461b-a8dc-100c9ce48b03 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:24.104952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bafe854-abf4-4d75-a985-84c625920f8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=742bd8880fda487caab443468919f5f1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:24.108044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bafe854-abf4-4d75-a985-84c625920f8b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:24.122414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06146428-66c4-461b-a8dc-100c9ce48b03 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:24.132992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23176b5-8eab-48ed-ab0c-fa4e0156a26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:24.133768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23176b5-8eab-48ed-ab0c-fa4e0156a26e admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:24.134108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23176b5-8eab-48ed-ab0c-fa4e0156a26e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:24.134507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23176b5-8eab-48ed-ab0c-fa4e0156a26e admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:24.135152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23176b5-8eab-48ed-ab0c-fa4e0156a26e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:24.137182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23176b5-8eab-48ed-ab0c-fa4e0156a26e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:24.176793 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2312/4702] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:04:24 2022] POST /identity/v3/projects => generated 377 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:04:24.186017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3bafe854-abf4-4d75-a985-84c625920f8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '742bd8880fda487caab443468919f5f1', 'request_id': 'req-3bafe854-abf4-4d75-a985-84c625920f8b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:24.190459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23176b5-8eab-48ed-ab0c-fa4e0156a26e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:24.194462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3bafe854-abf4-4d75-a985-84c625920f8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '742bd8880fda487caab443468919f5f1', 'request_id': 'req-3bafe854-abf4-4d75-a985-84c625920f8b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:24.202533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3bafe854-abf4-4d75-a985-84c625920f8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '742bd8880fda487caab443468919f5f1', 'request_id': 'req-3bafe854-abf4-4d75-a985-84c625920f8b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:24.240686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3bafe854-abf4-4d75-a985-84c625920f8b admin admin] The token cache is being invalidate because project 742bd8880fda487caab443468919f5f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:04:24.241817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3bafe854-abf4-4d75-a985-84c625920f8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:24.243427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3bafe854-abf4-4d75-a985-84c625920f8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:24.245812 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2391/4703] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:04:23 2022] DELETE /identity/v3/projects/742bd8880fda487caab443468919f5f1 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:24.251504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5983fd-4104-4ef8-a03b-e5b1bce49817 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:24.253390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5983fd-4104-4ef8-a03b-e5b1bce49817 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:24.253889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5983fd-4104-4ef8-a03b-e5b1bce49817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:24.267494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:24.274649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23176b5-8eab-48ed-ab0c-fa4e0156a26e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:24.280493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312954b4-3927-48f9-a6c3-8f7fa0701f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:24.281332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312954b4-3927-48f9-a6c3-8f7fa0701f8a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:24.281768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312954b4-3927-48f9-a6c3-8f7fa0701f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:24.282135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312954b4-3927-48f9-a6c3-8f7fa0701f8a admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:24.283910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312954b4-3927-48f9-a6c3-8f7fa0701f8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:24.286744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312954b4-3927-48f9-a6c3-8f7fa0701f8a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:24.286933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0b5983fd-4104-4ef8-a03b-e5b1bce49817 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:24.288698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0b5983fd-4104-4ef8-a03b-e5b1bce49817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:24.313438 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2392/4704] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:04:24.313665 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2313/4705] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:04:24 2022] POST /identity/v3/users => generated 289 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:24.319431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d013f64-e1c3-4a94-bdc6-8ce79e1b1674 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:24.319738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d013f64-e1c3-4a94-bdc6-8ce79e1b1674 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:24.320074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312954b4-3927-48f9-a6c3-8f7fa0701f8a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:24.320433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d013f64-e1c3-4a94-bdc6-8ce79e1b1674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:24.322706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-312954b4-3927-48f9-a6c3-8f7fa0701f8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:24.338481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312954b4-3927-48f9-a6c3-8f7fa0701f8a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:24.351525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8533745-cfb7-4423-9924-fd53b4ef4680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:24.352155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8533745-cfb7-4423-9924-fd53b4ef4680 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:24.352439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8533745-cfb7-4423-9924-fd53b4ef4680 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:24.352715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8533745-cfb7-4423-9924-fd53b4ef4680 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:24.353425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8533745-cfb7-4423-9924-fd53b4ef4680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:24.355306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8533745-cfb7-4423-9924-fd53b4ef4680 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:24.361731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2d013f64-e1c3-4a94-bdc6-8ce79e1b1674 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:24.363830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2d013f64-e1c3-4a94-bdc6-8ce79e1b1674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:24.372194 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2314/4706] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:24.377246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8533745-cfb7-4423-9924-fd53b4ef4680 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:24.379666 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2393/4707] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:24.385972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d013f64-e1c3-4a94-bdc6-8ce79e1b1674 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:24.397610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8533745-cfb7-4423-9924-fd53b4ef4680 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:24.398769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d013f64-e1c3-4a94-bdc6-8ce79e1b1674 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:24.403044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d811fd-8cb2-4830-8c75-6336f8037421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:24.403651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d811fd-8cb2-4830-8c75-6336f8037421 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:24.403931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d811fd-8cb2-4830-8c75-6336f8037421 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:24.404496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d811fd-8cb2-4830-8c75-6336f8037421 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:24.405415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d811fd-8cb2-4830-8c75-6336f8037421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:24.406087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be59cf-f283-4185-a4ee-65686d075d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:24.407326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d811fd-8cb2-4830-8c75-6336f8037421 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:24.408368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be59cf-f283-4185-a4ee-65686d075d3d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:24.408869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be59cf-f283-4185-a4ee-65686d075d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:24.409537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be59cf-f283-4185-a4ee-65686d075d3d admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:24.410874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be59cf-f283-4185-a4ee-65686d075d3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:04:24.411565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be59cf-f283-4185-a4ee-65686d075d3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:24.413380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be59cf-f283-4185-a4ee-65686d075d3d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:24.425779 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2315/4708] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:24.427284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d811fd-8cb2-4830-8c75-6336f8037421 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:24.429145 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2394/4709] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:04:24.433461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be59cf-f283-4185-a4ee-65686d075d3d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:24.438413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d811fd-8cb2-4830-8c75-6336f8037421 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:24.443630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09dad2-335f-4875-b6bd-8ec140afe978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:24.444347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09dad2-335f-4875-b6bd-8ec140afe978 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:24.444631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09dad2-335f-4875-b6bd-8ec140afe978 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:24.444908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09dad2-335f-4875-b6bd-8ec140afe978 admin admin] PATH_INFO: `/v3/projects/0bf01814edfe46d5b1dd62f42c2fa45d/users/9b5f90f4ec8b4b5cb4b64a2b99ca9beb/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:24.445027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be59cf-f283-4185-a4ee-65686d075d3d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:24.450186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4cdaac-6fed-47e4-b77e-ef518189e1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:24.450821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4cdaac-6fed-47e4-b77e-ef518189e1e9 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:24.451100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4cdaac-6fed-47e4-b77e-ef518189e1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:24.451373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4cdaac-6fed-47e4-b77e-ef518189e1e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:24.452062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4cdaac-6fed-47e4-b77e-ef518189e1e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:24.453952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4cdaac-6fed-47e4-b77e-ef518189e1e9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:24.464626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09dad2-335f-4875-b6bd-8ec140afe978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bf01814edfe46d5b1dd62f42c2fa45d, user_id=9b5f90f4ec8b4b5cb4b64a2b99ca9beb, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:24.467953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09dad2-335f-4875-b6bd-8ec140afe978 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:24.499249 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2316/4710] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:04:24 2022] PUT /identity/v3/projects/0bf01814edfe46d5b1dd62f42c2fa45d/users/9b5f90f4ec8b4b5cb4b64a2b99ca9beb/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:04:24.500681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7cf03-7130-43fa-9863-5c83033ecc2e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:24.500990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7cf03-7130-43fa-9863-5c83033ecc2e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:24.501232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7cf03-7130-43fa-9863-5c83033ecc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:24.510813 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2395/4711] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:04:24 2022] POST /identity/v3/projects => generated 407 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:24.511864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4cdaac-6fed-47e4-b77e-ef518189e1e9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:24.522056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-91a7cf03-7130-43fa-9863-5c83033ecc2e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:24.523324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-91a7cf03-7130-43fa-9863-5c83033ecc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:24.545507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:24.601779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4cdaac-6fed-47e4-b77e-ef518189e1e9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:24.619757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde70ef-f347-4989-8eec-0cd5f841f634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:24.620549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde70ef-f347-4989-8eec-0cd5f841f634 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:24.622108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde70ef-f347-4989-8eec-0cd5f841f634 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:24.622475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde70ef-f347-4989-8eec-0cd5f841f634 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:24.623400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde70ef-f347-4989-8eec-0cd5f841f634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:24.625840 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2317/4712] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:24.626002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde70ef-f347-4989-8eec-0cd5f841f634 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:24.644461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7cf03-7130-43fa-9863-5c83033ecc2e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:24.646115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-91a7cf03-7130-43fa-9863-5c83033ecc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:24.658667 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2396/4713] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:04:24 2022] POST /identity/v3/users => generated 304 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:24.663364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde70ef-f347-4989-8eec-0cd5f841f634 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:24.680528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde70ef-f347-4989-8eec-0cd5f841f634 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:24.693676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a18af1-12f5-486e-a03d-090d083f3649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:24.693676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a18af1-12f5-486e-a03d-090d083f3649 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:24.693676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a18af1-12f5-486e-a03d-090d083f3649 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:24.693676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a18af1-12f5-486e-a03d-090d083f3649 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:24.694217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a18af1-12f5-486e-a03d-090d083f3649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:24.696621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a18af1-12f5-486e-a03d-090d083f3649 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:24.715493 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2397/4714] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:24.716832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7cf03-7130-43fa-9863-5c83033ecc2e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:24.718041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a18af1-12f5-486e-a03d-090d083f3649 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:24.722577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f03ec9-aa7d-4331-a055-44184be594d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:24.723330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f03ec9-aa7d-4331-a055-44184be594d2 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:24.723584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f03ec9-aa7d-4331-a055-44184be594d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:24.723935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f03ec9-aa7d-4331-a055-44184be594d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:24.738031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f03ec9-aa7d-4331-a055-44184be594d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:24.739125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a18af1-12f5-486e-a03d-090d083f3649 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:24.739706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f03ec9-aa7d-4331-a055-44184be594d2 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:24.746684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8705b3-5bdd-42d4-9293-8f0e609d9401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:24.747394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8705b3-5bdd-42d4-9293-8f0e609d9401 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:24.747749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8705b3-5bdd-42d4-9293-8f0e609d9401 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:24.748133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8705b3-5bdd-42d4-9293-8f0e609d9401 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:24.749077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8705b3-5bdd-42d4-9293-8f0e609d9401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:24.750851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8705b3-5bdd-42d4-9293-8f0e609d9401 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:24.751661 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2318/4715] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:24.764044 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2398/4716] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:24.768970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f03ec9-aa7d-4331-a055-44184be594d2 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:24.780075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f03ec9-aa7d-4331-a055-44184be594d2 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:24.787835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1ef52-487b-4a7b-a021-69117a30c296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:24.788333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1ef52-487b-4a7b-a021-69117a30c296 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:24.788615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1ef52-487b-4a7b-a021-69117a30c296 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:24.788943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1ef52-487b-4a7b-a021-69117a30c296 admin admin] PATH_INFO: `/v3/projects/27cbdc934b2945f3b1765acee67fc104/users/6e66430017414257a4626b418ee714eb/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:24.812617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd1ef52-487b-4a7b-a021-69117a30c296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27cbdc934b2945f3b1765acee67fc104, user_id=6e66430017414257a4626b418ee714eb, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:24.815432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd1ef52-487b-4a7b-a021-69117a30c296 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:24.833207 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2319/4717] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:04:24 2022] PUT /identity/v3/projects/27cbdc934b2945f3b1765acee67fc104/users/6e66430017414257a4626b418ee714eb/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:24.843091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e5934-4e1b-44ce-8896-e3c55e2291e2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:24.844460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e5934-4e1b-44ce-8896-e3c55e2291e2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:24.844460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e5934-4e1b-44ce-8896-e3c55e2291e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:24.876774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-367e5934-4e1b-44ce-8896-e3c55e2291e2 None None] MFA Rules not processed for user `6e66430017414257a4626b418ee714eb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:24.877856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-367e5934-4e1b-44ce-8896-e3c55e2291e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:24.980576 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2399/4718] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:24 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:24.986345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1ef52-487b-4a7b-a021-69117a30c296 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:25.170985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1ef52-487b-4a7b-a021-69117a30c296 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:25.178016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f9edd-d6fe-452e-93ad-9d0804b83a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:25.179799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f9edd-d6fe-452e-93ad-9d0804b83a85 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:25.180485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f9edd-d6fe-452e-93ad-9d0804b83a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:25.181018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f9edd-d6fe-452e-93ad-9d0804b83a85 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:25.181872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f9edd-d6fe-452e-93ad-9d0804b83a85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:25.184358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f9edd-d6fe-452e-93ad-9d0804b83a85 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:25.273733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e5934-4e1b-44ce-8896-e3c55e2291e2 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:25.286133 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2320/4719] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:04:24 2022] POST /identity/v3/projects => generated 407 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:25.297760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e5934-4e1b-44ce-8896-e3c55e2291e2 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:25.309722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:25.408543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a44eb-ed1d-4403-816b-121911c0760a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:25.409737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a44eb-ed1d-4403-816b-121911c0760a admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:25.410422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a44eb-ed1d-4403-816b-121911c0760a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:25.410607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a44eb-ed1d-4403-816b-121911c0760a admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:25.411317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a44eb-ed1d-4403-816b-121911c0760a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:25.413348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a44eb-ed1d-4403-816b-121911c0760a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:25.446618 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2400/4720] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:04:25 2022] POST /identity/v3/users => generated 310 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:25.448459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f9edd-d6fe-452e-93ad-9d0804b83a85 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:25.459507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f9edd-d6fe-452e-93ad-9d0804b83a85 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:25.464709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce97166-46ea-4840-9cc8-eff71ee7a761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:25.465412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce97166-46ea-4840-9cc8-eff71ee7a761 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:25.465769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce97166-46ea-4840-9cc8-eff71ee7a761 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:25.466117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce97166-46ea-4840-9cc8-eff71ee7a761 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:25.466882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce97166-46ea-4840-9cc8-eff71ee7a761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:25.468580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce97166-46ea-4840-9cc8-eff71ee7a761 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:25.488324 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2321/4721] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:04:25.490196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a44eb-ed1d-4403-816b-121911c0760a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:25.504068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a44eb-ed1d-4403-816b-121911c0760a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:25.509798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b275f6f-9af0-4b2b-80dc-3bb366f660fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:25.510596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b275f6f-9af0-4b2b-80dc-3bb366f660fd admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:25.510949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b275f6f-9af0-4b2b-80dc-3bb366f660fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:25.511294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b275f6f-9af0-4b2b-80dc-3bb366f660fd admin admin] PATH_INFO: `/v3/projects/b7c0fe63fa204ba4898b1a7fa8a7d189/users/52768a09c5554d55b6b470e7508be1e9/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:25.538824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b275f6f-9af0-4b2b-80dc-3bb366f660fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7c0fe63fa204ba4898b1a7fa8a7d189, user_id=52768a09c5554d55b6b470e7508be1e9, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:25.538824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b275f6f-9af0-4b2b-80dc-3bb366f660fd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:25.559815 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2401/4722] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:04:25 2022] PUT /identity/v3/projects/b7c0fe63fa204ba4898b1a7fa8a7d189/users/52768a09c5554d55b6b470e7508be1e9/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:25.564662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce97166-46ea-4840-9cc8-eff71ee7a761 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:25.580389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce97166-46ea-4840-9cc8-eff71ee7a761 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:25.593038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:25.664476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ca225-a3f8-4bce-86eb-fe3badd4f1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:25.665099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ca225-a3f8-4bce-86eb-fe3badd4f1a3 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:25.665381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ca225-a3f8-4bce-86eb-fe3badd4f1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:25.665727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ca225-a3f8-4bce-86eb-fe3badd4f1a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:25.666379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ca225-a3f8-4bce-86eb-fe3badd4f1a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:25.668053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ca225-a3f8-4bce-86eb-fe3badd4f1a3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:25.682642 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2322/4723] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:04:25.683846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b275f6f-9af0-4b2b-80dc-3bb366f660fd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:25.697418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b275f6f-9af0-4b2b-80dc-3bb366f660fd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:25.708488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d66c33c-5914-47d1-b376-b01991a046e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:25.710467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66c33c-5914-47d1-b376-b01991a046e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:25.714036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66c33c-5914-47d1-b376-b01991a046e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:25.715339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66c33c-5914-47d1-b376-b01991a046e6 admin admin] PATH_INFO: `/v3/projects/b7c0fe63fa204ba4898b1a7fa8a7d189/users/52768a09c5554d55b6b470e7508be1e9/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:25.717577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d66c33c-5914-47d1-b376-b01991a046e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7c0fe63fa204ba4898b1a7fa8a7d189, user_id=52768a09c5554d55b6b470e7508be1e9, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:25.723664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d66c33c-5914-47d1-b376-b01991a046e6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:25.744488 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2402/4724] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:04:25 2022] PUT /identity/v3/projects/b7c0fe63fa204ba4898b1a7fa8a7d189/users/52768a09c5554d55b6b470e7508be1e9/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:25.752415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d7a39-e9fa-4d82-aee3-11c962889916 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:25.752706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d7a39-e9fa-4d82-aee3-11c962889916 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:25.752991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d7a39-e9fa-4d82-aee3-11c962889916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:25.812848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-117d7a39-e9fa-4d82-aee3-11c962889916 None None] MFA Rules not processed for user `52768a09c5554d55b6b470e7508be1e9`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:25.816495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-117d7a39-e9fa-4d82-aee3-11c962889916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:25.942744 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2323/4725] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:25 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:25.949714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d66c33c-5914-47d1-b376-b01991a046e6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:25.952836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4d66c33c-5914-47d1-b376-b01991a046e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:26.073449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d66c33c-5914-47d1-b376-b01991a046e6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:26.077788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc07d70-4449-4e37-94c7-cd0a925f9dea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:26.078841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc07d70-4449-4e37-94c7-cd0a925f9dea service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:26.079130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc07d70-4449-4e37-94c7-cd0a925f9dea service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:26.079468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc07d70-4449-4e37-94c7-cd0a925f9dea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:26.098998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc07d70-4449-4e37-94c7-cd0a925f9dea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:26.102291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc07d70-4449-4e37-94c7-cd0a925f9dea service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:26.115266 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2403/4726] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:25 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:26.423100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d7a39-e9fa-4d82-aee3-11c962889916 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:26.438206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d7a39-e9fa-4d82-aee3-11c962889916 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:26.448080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bec26-54a8-48d6-a38b-2ffde53dc2a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:26.448504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bec26-54a8-48d6-a38b-2ffde53dc2a4 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:26.448663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bec26-54a8-48d6-a38b-2ffde53dc2a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:26.448943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bec26-54a8-48d6-a38b-2ffde53dc2a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:26.581074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25bec26-54a8-48d6-a38b-2ffde53dc2a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:26.582213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25bec26-54a8-48d6-a38b-2ffde53dc2a4 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:26.593156 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2324/4727] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:26 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:31.078220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc07d70-4449-4e37-94c7-cd0a925f9dea service cinder] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:31.080391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fdc07d70-4449-4e37-94c7-cd0a925f9dea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:31.294892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc07d70-4449-4e37-94c7-cd0a925f9dea service cinder] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:31.301612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7177d4-bb36-4254-b6bd-c122bf2d633c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:31.305643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7177d4-bb36-4254-b6bd-c122bf2d633c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:31.306026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7177d4-bb36-4254-b6bd-c122bf2d633c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:31.306400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7177d4-bb36-4254-b6bd-c122bf2d633c admin admin] PATH_INFO: `/v3/users/775ff9091a794faaacc05a37550c8a72` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:31.308192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7177d4-bb36-4254-b6bd-c122bf2d633c admin admin] RBAC: Authorizing `identity:delete_user(user_id=775ff9091a794faaacc05a37550c8a72)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:31.310687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7177d4-bb36-4254-b6bd-c122bf2d633c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:31.332691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:31.425736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fb7177d4-bb36-4254-b6bd-c122bf2d633c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '775ff9091a794faaacc05a37550c8a72'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:31.441292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fb7177d4-bb36-4254-b6bd-c122bf2d633c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '775ff9091a794faaacc05a37550c8a72'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:31.475412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fb7177d4-bb36-4254-b6bd-c122bf2d633c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '775ff9091a794faaacc05a37550c8a72'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:31.477867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fb7177d4-bb36-4254-b6bd-c122bf2d633c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '775ff9091a794faaacc05a37550c8a72'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:31.502378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fb7177d4-bb36-4254-b6bd-c122bf2d633c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '775ff9091a794faaacc05a37550c8a72'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:31.533904 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2404/4728] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:04:31 2022] DELETE /identity/v3/users/775ff9091a794faaacc05a37550c8a72 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:31.829619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bec26-54a8-48d6-a38b-2ffde53dc2a4 service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:31.852406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bec26-54a8-48d6-a38b-2ffde53dc2a4 service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:31.943055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:31.979505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417a205-d734-45a2-a043-2aa1764d3fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:31.980544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417a205-d734-45a2-a043-2aa1764d3fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:31.980912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417a205-d734-45a2-a043-2aa1764d3fec admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:31.981292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417a205-d734-45a2-a043-2aa1764d3fec admin admin] PATH_INFO: `/v3/projects/bea57d3c53f74e3ea78e9e4bc790c434` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:31.982610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8417a205-d734-45a2-a043-2aa1764d3fec admin admin] RBAC: Authorizing `identity:delete_project(project_id=bea57d3c53f74e3ea78e9e4bc790c434)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:31.984684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8417a205-d734-45a2-a043-2aa1764d3fec admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:32.050154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8417a205-d734-45a2-a043-2aa1764d3fec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bea57d3c53f74e3ea78e9e4bc790c434', 'request_id': 'req-8417a205-d734-45a2-a043-2aa1764d3fec'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:32.066261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8417a205-d734-45a2-a043-2aa1764d3fec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bea57d3c53f74e3ea78e9e4bc790c434', 'request_id': 'req-8417a205-d734-45a2-a043-2aa1764d3fec'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:32.080290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8417a205-d734-45a2-a043-2aa1764d3fec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bea57d3c53f74e3ea78e9e4bc790c434', 'request_id': 'req-8417a205-d734-45a2-a043-2aa1764d3fec'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:32.094016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8417a205-d734-45a2-a043-2aa1764d3fec admin admin] The token cache is being invalidate because project bea57d3c53f74e3ea78e9e4bc790c434 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:04:32.094298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8417a205-d734-45a2-a043-2aa1764d3fec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:32.095541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8417a205-d734-45a2-a043-2aa1764d3fec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:32.101049 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2325/4729] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:04:31 2022] DELETE /identity/v3/projects/bea57d3c53f74e3ea78e9e4bc790c434 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:32.102517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef2528-d96e-4c7f-867e-e25d255cdfcb None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:32.103009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef2528-d96e-4c7f-867e-e25d255cdfcb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:32.103102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef2528-d96e-4c7f-867e-e25d255cdfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:32.138809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ebef2528-d96e-4c7f-867e-e25d255cdfcb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:32.140365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ebef2528-d96e-4c7f-867e-e25d255cdfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:32.269025 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2405/4730] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:32.277532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2db63-cbb5-4eee-ad8c-e5ae7675614c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:32.277823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2db63-cbb5-4eee-ad8c-e5ae7675614c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:32.278102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2db63-cbb5-4eee-ad8c-e5ae7675614c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:32.305108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-42d2db63-cbb5-4eee-ad8c-e5ae7675614c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:32.307247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-42d2db63-cbb5-4eee-ad8c-e5ae7675614c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:32.317410 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2326/4731] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:32.321493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef2528-d96e-4c7f-867e-e25d255cdfcb None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:32.333918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef2528-d96e-4c7f-867e-e25d255cdfcb None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:32.339278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d431b8-0f15-4ba4-bf15-5246d86e44a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:32.339973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d431b8-0f15-4ba4-bf15-5246d86e44a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:32.340263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d431b8-0f15-4ba4-bf15-5246d86e44a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:32.340536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d431b8-0f15-4ba4-bf15-5246d86e44a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:32.341373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d431b8-0f15-4ba4-bf15-5246d86e44a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:04:32.341683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d431b8-0f15-4ba4-bf15-5246d86e44a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:32.343482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d431b8-0f15-4ba4-bf15-5246d86e44a7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:32.363099 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2406/4732] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:04:32.364434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2db63-cbb5-4eee-ad8c-e5ae7675614c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:32.381707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2db63-cbb5-4eee-ad8c-e5ae7675614c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:32.388378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb353a87-a48d-4d28-8f2c-ba1e154626e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:32.389423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb353a87-a48d-4d28-8f2c-ba1e154626e7 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:32.390084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb353a87-a48d-4d28-8f2c-ba1e154626e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:32.390565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb353a87-a48d-4d28-8f2c-ba1e154626e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:32.391770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb353a87-a48d-4d28-8f2c-ba1e154626e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:32.394385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb353a87-a48d-4d28-8f2c-ba1e154626e7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:32.438611 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2327/4733] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:04:32 2022] POST /identity/v3/projects => generated 383 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:32.444896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d431b8-0f15-4ba4-bf15-5246d86e44a7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:32.489341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d431b8-0f15-4ba4-bf15-5246d86e44a7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:32.616554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffb26f-e8cb-4272-80a7-30e256907f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:32.617506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffb26f-e8cb-4272-80a7-30e256907f41 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:32.622615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffb26f-e8cb-4272-80a7-30e256907f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:32.623170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffb26f-e8cb-4272-80a7-30e256907f41 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:32.624095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ffb26f-e8cb-4272-80a7-30e256907f41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:32.626667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ffb26f-e8cb-4272-80a7-30e256907f41 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:32.697592 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2407/4734] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:04:32 2022] POST /identity/v3/users => generated 292 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:32.701792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb353a87-a48d-4d28-8f2c-ba1e154626e7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:32.716601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb353a87-a48d-4d28-8f2c-ba1e154626e7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:32.749673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14f688-dcf3-47f6-b1ea-60f38b039b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:32.750608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14f688-dcf3-47f6-b1ea-60f38b039b00 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:32.751081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14f688-dcf3-47f6-b1ea-60f38b039b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:32.752068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14f688-dcf3-47f6-b1ea-60f38b039b00 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:32.752892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14f688-dcf3-47f6-b1ea-60f38b039b00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:32.754767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14f688-dcf3-47f6-b1ea-60f38b039b00 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:32.788672 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2328/4735] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:32.793180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffb26f-e8cb-4272-80a7-30e256907f41 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:32.802813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:32.811186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffb26f-e8cb-4272-80a7-30e256907f41 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:32.822362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082a125-71c9-4db4-b178-c1417404e356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:32.823132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082a125-71c9-4db4-b178-c1417404e356 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:32.823671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082a125-71c9-4db4-b178-c1417404e356 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:32.824064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082a125-71c9-4db4-b178-c1417404e356 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:32.824845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9082a125-71c9-4db4-b178-c1417404e356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:32.831275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9082a125-71c9-4db4-b178-c1417404e356 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:32.858060 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2408/4736] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:32.862640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14f688-dcf3-47f6-b1ea-60f38b039b00 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:32.875052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14f688-dcf3-47f6-b1ea-60f38b039b00 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:32.882106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28fec9-26ef-4d26-9c9b-71d21614e1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:32.883127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28fec9-26ef-4d26-9c9b-71d21614e1e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:32.884165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28fec9-26ef-4d26-9c9b-71d21614e1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:32.884741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28fec9-26ef-4d26-9c9b-71d21614e1e2 admin admin] PATH_INFO: `/v3/projects/e8be683460bb4157824a7f999d0dd106/users/644bb91fc7d14468a59c399eb3df8473/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:32.906883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28fec9-26ef-4d26-9c9b-71d21614e1e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8be683460bb4157824a7f999d0dd106, user_id=644bb91fc7d14468a59c399eb3df8473, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:32.911219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28fec9-26ef-4d26-9c9b-71d21614e1e2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:32.935694 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2329/4737] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:04:32 2022] PUT /identity/v3/projects/e8be683460bb4157824a7f999d0dd106/users/644bb91fc7d14468a59c399eb3df8473/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:32.946752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53d91c-0937-4a83-9885-a9083456622b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:32.947452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53d91c-0937-4a83-9885-a9083456622b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:32.947947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53d91c-0937-4a83-9885-a9083456622b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:33.017166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5e53d91c-0937-4a83-9885-a9083456622b None None] MFA Rules not processed for user `644bb91fc7d14468a59c399eb3df8473`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:33.021561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5e53d91c-0937-4a83-9885-a9083456622b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:33.026412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28fec9-26ef-4d26-9c9b-71d21614e1e2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:33.028553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cd28fec9-26ef-4d26-9c9b-71d21614e1e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:33.124941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:33.136902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28fec9-26ef-4d26-9c9b-71d21614e1e2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:33.151413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c805b-ce87-4613-8cd3-32ed53290755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:33.152502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c805b-ce87-4613-8cd3-32ed53290755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:33.152961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c805b-ce87-4613-8cd3-32ed53290755 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:33.153617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c805b-ce87-4613-8cd3-32ed53290755 admin admin] PATH_INFO: `/v3/users/6e66430017414257a4626b418ee714eb` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:33.155336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c805b-ce87-4613-8cd3-32ed53290755 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e66430017414257a4626b418ee714eb)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:33.157754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c805b-ce87-4613-8cd3-32ed53290755 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:33.186252 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2409/4738] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:33.224102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9e0c805b-ce87-4613-8cd3-32ed53290755 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e66430017414257a4626b418ee714eb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:33.225249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9e0c805b-ce87-4613-8cd3-32ed53290755 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e66430017414257a4626b418ee714eb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:33.226386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9e0c805b-ce87-4613-8cd3-32ed53290755 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e66430017414257a4626b418ee714eb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:33.240031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9e0c805b-ce87-4613-8cd3-32ed53290755 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e66430017414257a4626b418ee714eb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:33.265588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9e0c805b-ce87-4613-8cd3-32ed53290755 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e66430017414257a4626b418ee714eb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:33.281941 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2330/4739] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:04:33 2022] DELETE /identity/v3/users/6e66430017414257a4626b418ee714eb => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:33.287343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65701af8-2255-4113-aeaa-61d10a7c4222 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:33.287852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65701af8-2255-4113-aeaa-61d10a7c4222 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:33.288138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65701af8-2255-4113-aeaa-61d10a7c4222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:33.328850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-65701af8-2255-4113-aeaa-61d10a7c4222 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:33.331909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-65701af8-2255-4113-aeaa-61d10a7c4222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:33.439536 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2410/4740] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:04:33.484021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c805b-ce87-4613-8cd3-32ed53290755 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:33.488688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9e0c805b-ce87-4613-8cd3-32ed53290755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:33.580374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c805b-ce87-4613-8cd3-32ed53290755 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:33.596606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65701af8-2255-4113-aeaa-61d10a7c4222 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:33.600032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-65701af8-2255-4113-aeaa-61d10a7c4222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:33.600759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8444815-1f96-454d-a73f-f45fa110183c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:33.601823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8444815-1f96-454d-a73f-f45fa110183c service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:33.602134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8444815-1f96-454d-a73f-f45fa110183c service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:33.602576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8444815-1f96-454d-a73f-f45fa110183c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:33.625333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8444815-1f96-454d-a73f-f45fa110183c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:33.626998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8444815-1f96-454d-a73f-f45fa110183c service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:33.653709 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2331/4741] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:33.711519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65701af8-2255-4113-aeaa-61d10a7c4222 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:33.721964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc71fe-cae8-47a5-9d14-85b2294f4b5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:33.723719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc71fe-cae8-47a5-9d14-85b2294f4b5b service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:33.724578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc71fe-cae8-47a5-9d14-85b2294f4b5b service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:33.725282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc71fe-cae8-47a5-9d14-85b2294f4b5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:33.727692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-17fc71fe-cae8-47a5-9d14-85b2294f4b5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:33.761712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:33.846902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc71fe-cae8-47a5-9d14-85b2294f4b5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:33.849646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc71fe-cae8-47a5-9d14-85b2294f4b5b service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:33.880381 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2411/4742] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:34.245972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8444815-1f96-454d-a73f-f45fa110183c service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:34.247854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d8444815-1f96-454d-a73f-f45fa110183c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:34.275786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8444815-1f96-454d-a73f-f45fa110183c service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:34.300354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa176fe-a707-4994-9918-d0a6643715a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:34.306806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa176fe-a707-4994-9918-d0a6643715a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:34.307431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa176fe-a707-4994-9918-d0a6643715a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:34.308415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa176fe-a707-4994-9918-d0a6643715a9 admin admin] PATH_INFO: `/v3/projects/27cbdc934b2945f3b1765acee67fc104` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:34.327247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa176fe-a707-4994-9918-d0a6643715a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27cbdc934b2945f3b1765acee67fc104)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:34.337638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa176fe-a707-4994-9918-d0a6643715a9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:34.498278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-afa176fe-a707-4994-9918-d0a6643715a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27cbdc934b2945f3b1765acee67fc104', 'request_id': 'req-afa176fe-a707-4994-9918-d0a6643715a9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:34.517057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-afa176fe-a707-4994-9918-d0a6643715a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27cbdc934b2945f3b1765acee67fc104', 'request_id': 'req-afa176fe-a707-4994-9918-d0a6643715a9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:34.525704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-afa176fe-a707-4994-9918-d0a6643715a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27cbdc934b2945f3b1765acee67fc104', 'request_id': 'req-afa176fe-a707-4994-9918-d0a6643715a9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:34.557549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-afa176fe-a707-4994-9918-d0a6643715a9 admin admin] The token cache is being invalidate because project 27cbdc934b2945f3b1765acee67fc104 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:04:34.557915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-afa176fe-a707-4994-9918-d0a6643715a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:34.560036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-afa176fe-a707-4994-9918-d0a6643715a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:34.566835 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2332/4743] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:04:34 2022] DELETE /identity/v3/projects/27cbdc934b2945f3b1765acee67fc104 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:34.567950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc71fe-cae8-47a5-9d14-85b2294f4b5b service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:34.581640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc71fe-cae8-47a5-9d14-85b2294f4b5b service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:34.718656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a8004-f32c-4681-afa9-2854fa9ba7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:34.719328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a8004-f32c-4681-afa9-2854fa9ba7c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:34.719616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a8004-f32c-4681-afa9-2854fa9ba7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:34.719929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a8004-f32c-4681-afa9-2854fa9ba7c3 admin admin] PATH_INFO: `/v3/users/52768a09c5554d55b6b470e7508be1e9` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:34.721183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a8004-f32c-4681-afa9-2854fa9ba7c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52768a09c5554d55b6b470e7508be1e9)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:34.723520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a8004-f32c-4681-afa9-2854fa9ba7c3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:34.827175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3f3a8004-f32c-4681-afa9-2854fa9ba7c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52768a09c5554d55b6b470e7508be1e9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:34.839145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3f3a8004-f32c-4681-afa9-2854fa9ba7c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52768a09c5554d55b6b470e7508be1e9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:34.853260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3f3a8004-f32c-4681-afa9-2854fa9ba7c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52768a09c5554d55b6b470e7508be1e9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:34.854401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3f3a8004-f32c-4681-afa9-2854fa9ba7c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52768a09c5554d55b6b470e7508be1e9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:34.864281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3f3a8004-f32c-4681-afa9-2854fa9ba7c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52768a09c5554d55b6b470e7508be1e9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:34.882181 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2412/4744] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:04:34 2022] DELETE /identity/v3/users/52768a09c5554d55b6b470e7508be1e9 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:36.172695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa176fe-a707-4994-9918-d0a6643715a9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:36.175338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-afa176fe-a707-4994-9918-d0a6643715a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:36.186386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:36.197742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa176fe-a707-4994-9918-d0a6643715a9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:36.223152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10550af1-3019-4cff-986f-1642543e6e3c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:36.225577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10550af1-3019-4cff-986f-1642543e6e3c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:36.226061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10550af1-3019-4cff-986f-1642543e6e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:36.320794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-10550af1-3019-4cff-986f-1642543e6e3c None None] MFA Rules not processed for user `9b5f90f4ec8b4b5cb4b64a2b99ca9beb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:36.325982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-10550af1-3019-4cff-986f-1642543e6e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:36.403250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400602b5-316e-48a6-bf7a-8912100b8c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:36.404217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400602b5-316e-48a6-bf7a-8912100b8c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:36.404667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400602b5-316e-48a6-bf7a-8912100b8c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:36.405104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400602b5-316e-48a6-bf7a-8912100b8c63 admin admin] PATH_INFO: `/v3/projects/b7c0fe63fa204ba4898b1a7fa8a7d189` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:36.406537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:36.409334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400602b5-316e-48a6-bf7a-8912100b8c63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7c0fe63fa204ba4898b1a7fa8a7d189)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:36.417123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400602b5-316e-48a6-bf7a-8912100b8c63 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:36.493681 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2413/4745] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:36.494825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10550af1-3019-4cff-986f-1642543e6e3c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:36.496443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-10550af1-3019-4cff-986f-1642543e6e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:36.515610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-400602b5-316e-48a6-bf7a-8912100b8c63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7c0fe63fa204ba4898b1a7fa8a7d189', 'request_id': 'req-400602b5-316e-48a6-bf7a-8912100b8c63'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:36.523045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10550af1-3019-4cff-986f-1642543e6e3c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:36.528867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-400602b5-316e-48a6-bf7a-8912100b8c63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7c0fe63fa204ba4898b1a7fa8a7d189', 'request_id': 'req-400602b5-316e-48a6-bf7a-8912100b8c63'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:36.542141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-400602b5-316e-48a6-bf7a-8912100b8c63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7c0fe63fa204ba4898b1a7fa8a7d189', 'request_id': 'req-400602b5-316e-48a6-bf7a-8912100b8c63'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:36.560748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-400602b5-316e-48a6-bf7a-8912100b8c63 admin admin] The token cache is being invalidate because project b7c0fe63fa204ba4898b1a7fa8a7d189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:04:36.561046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-400602b5-316e-48a6-bf7a-8912100b8c63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:36.561947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-400602b5-316e-48a6-bf7a-8912100b8c63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:36.563764 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2333/4746] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:04:36 2022] DELETE /identity/v3/projects/b7c0fe63fa204ba4898b1a7fa8a7d189 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:36.571402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48f727-6508-4383-b0c6-638dc0a13371 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:36.572531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48f727-6508-4383-b0c6-638dc0a13371 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:36.574391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48f727-6508-4383-b0c6-638dc0a13371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:36.610083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3f48f727-6508-4383-b0c6-638dc0a13371 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:36.612048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3f48f727-6508-4383-b0c6-638dc0a13371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:36.613084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02aa7be-96c7-4246-b4e4-d453581a4ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:36.614057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aa7be-96c7-4246-b4e4-d453581a4ae8 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:36.614524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aa7be-96c7-4246-b4e4-d453581a4ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:36.615081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aa7be-96c7-4246-b4e4-d453581a4ae8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:36.616081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02aa7be-96c7-4246-b4e4-d453581a4ae8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:36.618381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02aa7be-96c7-4246-b4e4-d453581a4ae8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:36.620828 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2334/4747] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:36.630792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ab632-57bb-4da5-8b9a-7d1e3758bc17 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:36.632547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ab632-57bb-4da5-8b9a-7d1e3758bc17 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:36.632547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ab632-57bb-4da5-8b9a-7d1e3758bc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:36.659340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f48ab632-57bb-4da5-8b9a-7d1e3758bc17 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:36.660400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f48ab632-57bb-4da5-8b9a-7d1e3758bc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:36.674543 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2414/4748] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:04:36 2022] POST /identity/v3/projects => generated 377 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:36.677345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02aa7be-96c7-4246-b4e4-d453581a4ae8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:36.678961 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2335/4749] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:36.680785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b02aa7be-96c7-4246-b4e4-d453581a4ae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:36.683223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ab632-57bb-4da5-8b9a-7d1e3758bc17 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:36.695821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ab632-57bb-4da5-8b9a-7d1e3758bc17 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:36.793211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02aa7be-96c7-4246-b4e4-d453581a4ae8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:36.812651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cc27d-151f-4690-92d7-3d3cbc9ad41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:36.814019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cc27d-151f-4690-92d7-3d3cbc9ad41b admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:36.814587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cc27d-151f-4690-92d7-3d3cbc9ad41b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:36.815058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cc27d-151f-4690-92d7-3d3cbc9ad41b admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:36.816049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31cc27d-151f-4690-92d7-3d3cbc9ad41b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:36.819597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31cc27d-151f-4690-92d7-3d3cbc9ad41b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:36.855884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd4fa3-05b1-4fac-a55e-4664269709d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:36.856452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd4fa3-05b1-4fac-a55e-4664269709d6 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:36.856730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd4fa3-05b1-4fac-a55e-4664269709d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:36.856999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd4fa3-05b1-4fac-a55e-4664269709d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:36.857805 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2415/4750] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:04:36 2022] POST /identity/v3/users => generated 295 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:36.861353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cc27d-151f-4690-92d7-3d3cbc9ad41b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:36.861532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd4fa3-05b1-4fac-a55e-4664269709d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:04:36.861888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd4fa3-05b1-4fac-a55e-4664269709d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:36.863490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd4fa3-05b1-4fac-a55e-4664269709d6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:36.878112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cc27d-151f-4690-92d7-3d3cbc9ad41b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:36.890013 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2336/4751] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:04:36.898184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9b605-a400-447d-9c44-52cb9d84d0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:36.898784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9b605-a400-447d-9c44-52cb9d84d0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:36.899074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9b605-a400-447d-9c44-52cb9d84d0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:36.899338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9b605-a400-447d-9c44-52cb9d84d0b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:36.899992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d9b605-a400-447d-9c44-52cb9d84d0b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:36.901777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd4fa3-05b1-4fac-a55e-4664269709d6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:36.910138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d9b605-a400-447d-9c44-52cb9d84d0b4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:36.925284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd4fa3-05b1-4fac-a55e-4664269709d6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:36.936572 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2416/4752] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:36.939561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9b605-a400-447d-9c44-52cb9d84d0b4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:36.944826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84495f5-dec4-4a11-8faf-59604a8b9ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:36.945490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84495f5-dec4-4a11-8faf-59604a8b9ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:36.945798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84495f5-dec4-4a11-8faf-59604a8b9ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:36.946124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84495f5-dec4-4a11-8faf-59604a8b9ff4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:36.946966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84495f5-dec4-4a11-8faf-59604a8b9ff4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:36.950310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84495f5-dec4-4a11-8faf-59604a8b9ff4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:36.965202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9b605-a400-447d-9c44-52cb9d84d0b4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:36.975756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:36.982853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128b4c4-fd21-4dee-8a6b-fdf590177a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:36.983510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128b4c4-fd21-4dee-8a6b-fdf590177a50 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:36.983792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128b4c4-fd21-4dee-8a6b-fdf590177a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:36.984073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128b4c4-fd21-4dee-8a6b-fdf590177a50 admin admin] PATH_INFO: `/v3/projects/921b1307c8974dcdbef3150b7712ec41/users/f6f982c714554445b6510ca5f726d126/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:37.007029 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2337/4753] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:04:36 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:37.008178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128b4c4-fd21-4dee-8a6b-fdf590177a50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=921b1307c8974dcdbef3150b7712ec41, user_id=f6f982c714554445b6510ca5f726d126, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:37.011226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84495f5-dec4-4a11-8faf-59604a8b9ff4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:37.012749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128b4c4-fd21-4dee-8a6b-fdf590177a50 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:37.064857 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2417/4754] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:04:36 2022] PUT /identity/v3/projects/921b1307c8974dcdbef3150b7712ec41/users/f6f982c714554445b6510ca5f726d126/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:37.071875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128b4c4-fd21-4dee-8a6b-fdf590177a50 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:37.109671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:37.271621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84495f5-dec4-4a11-8faf-59604a8b9ff4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:37.307714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128b4c4-fd21-4dee-8a6b-fdf590177a50 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:37.334001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3c13b-7a58-494f-a1c3-1818b7d038d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:37.334664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3c13b-7a58-494f-a1c3-1818b7d038d0 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:37.334944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3c13b-7a58-494f-a1c3-1818b7d038d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:37.335396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3c13b-7a58-494f-a1c3-1818b7d038d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:37.336235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3c13b-7a58-494f-a1c3-1818b7d038d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:37.337876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3c13b-7a58-494f-a1c3-1818b7d038d0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:37.385015 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2418/4755] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:37.387431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3c13b-7a58-494f-a1c3-1818b7d038d0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:37.404497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3c13b-7a58-494f-a1c3-1818b7d038d0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:37.411690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83249227-3aa7-4083-87f0-e5cf33fc79e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:37.412657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83249227-3aa7-4083-87f0-e5cf33fc79e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:37.413001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83249227-3aa7-4083-87f0-e5cf33fc79e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:37.413413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83249227-3aa7-4083-87f0-e5cf33fc79e6 admin admin] PATH_INFO: `/v3/projects/921b1307c8974dcdbef3150b7712ec41/users/f6f982c714554445b6510ca5f726d126/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:37.416972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83249227-3aa7-4083-87f0-e5cf33fc79e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=921b1307c8974dcdbef3150b7712ec41, user_id=f6f982c714554445b6510ca5f726d126, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:37.420851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83249227-3aa7-4083-87f0-e5cf33fc79e6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:37.444621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03289956-a5f7-4bca-b091-6b7008c38fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:37.445307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03289956-a5f7-4bca-b091-6b7008c38fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:37.445589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03289956-a5f7-4bca-b091-6b7008c38fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:37.445865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03289956-a5f7-4bca-b091-6b7008c38fa3 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:37.446619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03289956-a5f7-4bca-b091-6b7008c38fa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:37.447885 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2419/4756] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:04:37 2022] PUT /identity/v3/projects/921b1307c8974dcdbef3150b7712ec41/users/f6f982c714554445b6510ca5f726d126/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:37.449912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03289956-a5f7-4bca-b091-6b7008c38fa3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:37.497458 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2338/4757] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:04:37 2022] POST /identity/v3/users => generated 289 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:37.498742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83249227-3aa7-4083-87f0-e5cf33fc79e6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:37.625757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83249227-3aa7-4083-87f0-e5cf33fc79e6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:37.636013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab90bb-103f-42b0-965b-70bd9d236511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:37.640434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab90bb-103f-42b0-965b-70bd9d236511 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:37.640718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab90bb-103f-42b0-965b-70bd9d236511 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:37.640997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab90bb-103f-42b0-965b-70bd9d236511 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:37.642190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab90bb-103f-42b0-965b-70bd9d236511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:37.644264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab90bb-103f-42b0-965b-70bd9d236511 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:37.662259 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2420/4758] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:37.664058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03289956-a5f7-4bca-b091-6b7008c38fa3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:37.678969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03289956-a5f7-4bca-b091-6b7008c38fa3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:37.687538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e7f3b-11b1-4c45-96d9-c1edfacd0ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:37.689928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e7f3b-11b1-4c45-96d9-c1edfacd0ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:37.690275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e7f3b-11b1-4c45-96d9-c1edfacd0ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:37.690689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e7f3b-11b1-4c45-96d9-c1edfacd0ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:37.692347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e7f3b-11b1-4c45-96d9-c1edfacd0ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:37.694155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e7f3b-11b1-4c45-96d9-c1edfacd0ba1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:37.721215 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2339/4759] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:04:37.725446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab90bb-103f-42b0-965b-70bd9d236511 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:37.739322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab90bb-103f-42b0-965b-70bd9d236511 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:37.749301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d85ac-0848-480b-9549-a3b043d76a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:37.750004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d85ac-0848-480b-9549-a3b043d76a7d admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:37.750314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d85ac-0848-480b-9549-a3b043d76a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:37.750583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d85ac-0848-480b-9549-a3b043d76a7d admin admin] PATH_INFO: `/v3/projects/cd3526ab00f3485e98db7f02e57bea4d/users/477eb33574da473284d09e765825bcc6/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:37.788567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d85ac-0848-480b-9549-a3b043d76a7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd3526ab00f3485e98db7f02e57bea4d, user_id=477eb33574da473284d09e765825bcc6, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:37.791320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d85ac-0848-480b-9549-a3b043d76a7d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:37.816852 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2421/4760] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:04:37 2022] PUT /identity/v3/projects/cd3526ab00f3485e98db7f02e57bea4d/users/477eb33574da473284d09e765825bcc6/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:04:37.827539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d72c1-3889-472c-a33c-3c6dd08a0dce None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:37.828036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d72c1-3889-472c-a33c-3c6dd08a0dce None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:37.828457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d72c1-3889-472c-a33c-3c6dd08a0dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:37.861982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7f1d72c1-3889-472c-a33c-3c6dd08a0dce None None] MFA Rules not processed for user `477eb33574da473284d09e765825bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:37.864139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7f1d72c1-3889-472c-a33c-3c6dd08a0dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:37.965567 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2340/4761] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:37.977872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d85ac-0848-480b-9549-a3b043d76a7d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:37.980137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8f9d85ac-0848-480b-9549-a3b043d76a7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:38.054990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d85ac-0848-480b-9549-a3b043d76a7d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:38.066464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed339104-e0b2-47a7-b74e-6529dcba2f5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:38.066464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed339104-e0b2-47a7-b74e-6529dcba2f5b service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:38.066464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed339104-e0b2-47a7-b74e-6529dcba2f5b service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:38.066464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed339104-e0b2-47a7-b74e-6529dcba2f5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:38.089353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed339104-e0b2-47a7-b74e-6529dcba2f5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:38.090996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed339104-e0b2-47a7-b74e-6529dcba2f5b service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:38.102660 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2422/4762] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:42.148998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebafa41-8a0b-4b83-8af2-9924cc05181d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:42.149507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebafa41-8a0b-4b83-8af2-9924cc05181d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:42.149621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebafa41-8a0b-4b83-8af2-9924cc05181d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:42.181930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:42.201772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2ebafa41-8a0b-4b83-8af2-9924cc05181d None None] MFA Rules not processed for user `aa4fbb28c9de4089870d91cf52dac8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:42.205109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2ebafa41-8a0b-4b83-8af2-9924cc05181d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:42.434704 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2341/4763] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:42 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:43.446257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed339104-e0b2-47a7-b74e-6529dcba2f5b service cinder] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:43.447803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ed339104-e0b2-47a7-b74e-6529dcba2f5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:43.471274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:43.582021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed339104-e0b2-47a7-b74e-6529dcba2f5b service cinder] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:43.591705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e86cf9-fda8-4287-a9df-9a2f29b18a52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:43.592341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e86cf9-fda8-4287-a9df-9a2f29b18a52 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:43.592616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e86cf9-fda8-4287-a9df-9a2f29b18a52 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:43.592883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e86cf9-fda8-4287-a9df-9a2f29b18a52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:43.595961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-02e86cf9-fda8-4287-a9df-9a2f29b18a52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:43.653407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e86cf9-fda8-4287-a9df-9a2f29b18a52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:43.654928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e86cf9-fda8-4287-a9df-9a2f29b18a52 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:43.686416 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2423/4764] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:44.813035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebafa41-8a0b-4b83-8af2-9924cc05181d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:44.815189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2ebafa41-8a0b-4b83-8af2-9924cc05181d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:44.978003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebafa41-8a0b-4b83-8af2-9924cc05181d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:44.996285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e58c69-bb63-4322-8788-d8343086ac5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:44.999147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e58c69-bb63-4322-8788-d8343086ac5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:45.000060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e58c69-bb63-4322-8788-d8343086ac5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:45.002322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e58c69-bb63-4322-8788-d8343086ac5a admin admin] PATH_INFO: `/v3/users/aad36d5e04a7453aba1e6d0b15d2d06f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:45.009052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e58c69-bb63-4322-8788-d8343086ac5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=aad36d5e04a7453aba1e6d0b15d2d06f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:45.012016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e58c69-bb63-4322-8788-d8343086ac5a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:45.135774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-52e58c69-bb63-4322-8788-d8343086ac5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aad36d5e04a7453aba1e6d0b15d2d06f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:45.136856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-52e58c69-bb63-4322-8788-d8343086ac5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aad36d5e04a7453aba1e6d0b15d2d06f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:45.138775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-52e58c69-bb63-4322-8788-d8343086ac5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aad36d5e04a7453aba1e6d0b15d2d06f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:45.152216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-52e58c69-bb63-4322-8788-d8343086ac5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aad36d5e04a7453aba1e6d0b15d2d06f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:45.182921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-52e58c69-bb63-4322-8788-d8343086ac5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aad36d5e04a7453aba1e6d0b15d2d06f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:45.202142 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2342/4765] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:04:44 2022] DELETE /identity/v3/users/aad36d5e04a7453aba1e6d0b15d2d06f => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:45.505425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e86cf9-fda8-4287-a9df-9a2f29b18a52 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:45.509186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-02e86cf9-fda8-4287-a9df-9a2f29b18a52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:45.539836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e86cf9-fda8-4287-a9df-9a2f29b18a52 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:45.682814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c6ab6-6a7b-43f9-b919-946f3799f2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:45.684286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c6ab6-6a7b-43f9-b919-946f3799f2db admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:45.684543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c6ab6-6a7b-43f9-b919-946f3799f2db admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:45.684822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c6ab6-6a7b-43f9-b919-946f3799f2db admin admin] PATH_INFO: `/v3/users/6736b832e7034ce3b431b79af50095aa` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:45.686891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c6ab6-6a7b-43f9-b919-946f3799f2db admin admin] RBAC: Authorizing `identity:delete_user(user_id=6736b832e7034ce3b431b79af50095aa)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:45.689727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c6ab6-6a7b-43f9-b919-946f3799f2db admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:45.774157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e41c6ab6-6a7b-43f9-b919-946f3799f2db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6736b832e7034ce3b431b79af50095aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:45.795061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e41c6ab6-6a7b-43f9-b919-946f3799f2db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6736b832e7034ce3b431b79af50095aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:45.830642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e41c6ab6-6a7b-43f9-b919-946f3799f2db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6736b832e7034ce3b431b79af50095aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:45.831825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e41c6ab6-6a7b-43f9-b919-946f3799f2db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6736b832e7034ce3b431b79af50095aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:45.845445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e41c6ab6-6a7b-43f9-b919-946f3799f2db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6736b832e7034ce3b431b79af50095aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:45.855286 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2424/4766] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:04:45 2022] DELETE /identity/v3/users/6736b832e7034ce3b431b79af50095aa => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:45.859004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e254f3-53a8-440d-a1e3-bbd2cad1c4df None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:45.860883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e254f3-53a8-440d-a1e3-bbd2cad1c4df None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:45.861387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e254f3-53a8-440d-a1e3-bbd2cad1c4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:45.890995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c2e254f3-53a8-440d-a1e3-bbd2cad1c4df None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:45.893086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c2e254f3-53a8-440d-a1e3-bbd2cad1c4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:45.942645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:46.060909 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2343/4767] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:04:46.073396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c6ab6-6a7b-43f9-b919-946f3799f2db admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:46.074948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e41c6ab6-6a7b-43f9-b919-946f3799f2db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:46.176786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:46.240759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c6ab6-6a7b-43f9-b919-946f3799f2db admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:46.245506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e37fd5-a827-421a-8792-3fddf4414085 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:46.246571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e37fd5-a827-421a-8792-3fddf4414085 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:46.247651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e37fd5-a827-421a-8792-3fddf4414085 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:46.248216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e37fd5-a827-421a-8792-3fddf4414085 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:46.263637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e37fd5-a827-421a-8792-3fddf4414085 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:46.265839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e37fd5-a827-421a-8792-3fddf4414085 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:46.279722 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2425/4768] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:46.365334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e254f3-53a8-440d-a1e3-bbd2cad1c4df None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:46.367391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c2e254f3-53a8-440d-a1e3-bbd2cad1c4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:46.402444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e254f3-53a8-440d-a1e3-bbd2cad1c4df None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:46.419952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156ecf5-0722-45a0-a38d-383668d87e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:46.423173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156ecf5-0722-45a0-a38d-383668d87e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:46.424044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156ecf5-0722-45a0-a38d-383668d87e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:46.424884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156ecf5-0722-45a0-a38d-383668d87e52 admin admin] PATH_INFO: `/v3/projects/99422a6e3b744bff964052dfddb678ef` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:46.428196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156ecf5-0722-45a0-a38d-383668d87e52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99422a6e3b744bff964052dfddb678ef)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:46.433322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156ecf5-0722-45a0-a38d-383668d87e52 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:46.541050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8156ecf5-0722-45a0-a38d-383668d87e52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99422a6e3b744bff964052dfddb678ef', 'request_id': 'req-8156ecf5-0722-45a0-a38d-383668d87e52'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:46.556137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8156ecf5-0722-45a0-a38d-383668d87e52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99422a6e3b744bff964052dfddb678ef', 'request_id': 'req-8156ecf5-0722-45a0-a38d-383668d87e52'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:46.582561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8156ecf5-0722-45a0-a38d-383668d87e52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99422a6e3b744bff964052dfddb678ef', 'request_id': 'req-8156ecf5-0722-45a0-a38d-383668d87e52'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:46.611272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8156ecf5-0722-45a0-a38d-383668d87e52 admin admin] The token cache is being invalidate because project 99422a6e3b744bff964052dfddb678ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:04:46.612970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8156ecf5-0722-45a0-a38d-383668d87e52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:46.615386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8156ecf5-0722-45a0-a38d-383668d87e52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:46.625694 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2344/4769] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:04:46 2022] DELETE /identity/v3/projects/99422a6e3b744bff964052dfddb678ef => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:46.626741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a05fa-867a-4656-8ce9-ea54a1fe4826 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:46.627226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a05fa-867a-4656-8ce9-ea54a1fe4826 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:46.627226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a05fa-867a-4656-8ce9-ea54a1fe4826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:46.665409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-706a05fa-867a-4656-8ce9-ea54a1fe4826 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:46.666901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-706a05fa-867a-4656-8ce9-ea54a1fe4826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:46.770202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156ecf5-0722-45a0-a38d-383668d87e52 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:46.770202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8156ecf5-0722-45a0-a38d-383668d87e52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:46.863904 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2426/4770] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:46.869757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da15cd6-132f-434c-8390-45488a2d4b80 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:46.872524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da15cd6-132f-434c-8390-45488a2d4b80 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:46.872829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da15cd6-132f-434c-8390-45488a2d4b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:46.936251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7da15cd6-132f-434c-8390-45488a2d4b80 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:46.939407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7da15cd6-132f-434c-8390-45488a2d4b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:46.964425 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2427/4771] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:04:46.968322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da15cd6-132f-434c-8390-45488a2d4b80 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:46.973571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156ecf5-0722-45a0-a38d-383668d87e52 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:46.982776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da15cd6-132f-434c-8390-45488a2d4b80 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:46.985803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98d94e-868e-414b-a2b4-9dbc638817b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:46.987120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98d94e-868e-414b-a2b4-9dbc638817b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:46.987775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98d94e-868e-414b-a2b4-9dbc638817b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:46.988590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98d94e-868e-414b-a2b4-9dbc638817b4 admin admin] PATH_INFO: `/v3/users/644bb91fc7d14468a59c399eb3df8473` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:46.990734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de98d94e-868e-414b-a2b4-9dbc638817b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=644bb91fc7d14468a59c399eb3df8473)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:46.993331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de98d94e-868e-414b-a2b4-9dbc638817b4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:46.996153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca588e7-02d8-454c-801d-9781102f4365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:46.996978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca588e7-02d8-454c-801d-9781102f4365 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:46.997916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca588e7-02d8-454c-801d-9781102f4365 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:46.999129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca588e7-02d8-454c-801d-9781102f4365 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:46.999967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca588e7-02d8-454c-801d-9781102f4365 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:04:47.001328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca588e7-02d8-454c-801d-9781102f4365 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:47.003141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca588e7-02d8-454c-801d-9781102f4365 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:47.045630 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2428/4772] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:04:47.046936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca588e7-02d8-454c-801d-9781102f4365 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:47.062946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca588e7-02d8-454c-801d-9781102f4365 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:47.073941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d8bd8-dde0-49df-93ef-e5c3b4d932cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:47.074942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d8bd8-dde0-49df-93ef-e5c3b4d932cb admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:47.076305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d8bd8-dde0-49df-93ef-e5c3b4d932cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:47.077691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d8bd8-dde0-49df-93ef-e5c3b4d932cb admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:47.079808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d8bd8-dde0-49df-93ef-e5c3b4d932cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:47.083602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d8bd8-dde0-49df-93ef-e5c3b4d932cb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:47.158223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-de98d94e-868e-414b-a2b4-9dbc638817b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '644bb91fc7d14468a59c399eb3df8473'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:47.158935 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2429/4773] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:04:47 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:47.159789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-de98d94e-868e-414b-a2b4-9dbc638817b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '644bb91fc7d14468a59c399eb3df8473'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:47.161212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d8bd8-dde0-49df-93ef-e5c3b4d932cb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:47.162964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-de98d94e-868e-414b-a2b4-9dbc638817b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '644bb91fc7d14468a59c399eb3df8473'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:47.166520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-445d8bd8-dde0-49df-93ef-e5c3b4d932cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:47.204628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-de98d94e-868e-414b-a2b4-9dbc638817b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '644bb91fc7d14468a59c399eb3df8473'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:47.208736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:47.231039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-de98d94e-868e-414b-a2b4-9dbc638817b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '644bb91fc7d14468a59c399eb3df8473'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:47.260844 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2345/4774] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:04:46 2022] DELETE /identity/v3/users/644bb91fc7d14468a59c399eb3df8473 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:47.265908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac21a7-f78e-445c-988b-981d90e31058 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:47.267083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac21a7-f78e-445c-988b-981d90e31058 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:47.267608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac21a7-f78e-445c-988b-981d90e31058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:47.302707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d1ac21a7-f78e-445c-988b-981d90e31058 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:47.304318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d1ac21a7-f78e-445c-988b-981d90e31058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:47.342191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d8bd8-dde0-49df-93ef-e5c3b4d932cb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:47.408892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:47.466097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b26a2-17ca-492a-8cce-21526525afa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:47.467995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b26a2-17ca-492a-8cce-21526525afa5 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:47.468754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b26a2-17ca-492a-8cce-21526525afa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:47.469401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b26a2-17ca-492a-8cce-21526525afa5 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:47.470809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b26a2-17ca-492a-8cce-21526525afa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:47.474522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b26a2-17ca-492a-8cce-21526525afa5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:47.498826 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2346/4775] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:47.499841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac21a7-f78e-445c-988b-981d90e31058 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:47.503512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d1ac21a7-f78e-445c-988b-981d90e31058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:47.528372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac21a7-f78e-445c-988b-981d90e31058 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:47.534380 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2430/4776] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:04:47 2022] POST /identity/v3/users => generated 287 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:47.534380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b26a2-17ca-492a-8cce-21526525afa5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:47.536206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b57b26a2-17ca-492a-8cce-21526525afa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:47.542361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43fce31-a79d-4a74-8bff-c64b85821a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:47.543040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fce31-a79d-4a74-8bff-c64b85821a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:47.545456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fce31-a79d-4a74-8bff-c64b85821a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:47.545772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fce31-a79d-4a74-8bff-c64b85821a94 admin admin] PATH_INFO: `/v3/projects/adf310503fa74abf83949a914e728693` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:47.549969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43fce31-a79d-4a74-8bff-c64b85821a94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adf310503fa74abf83949a914e728693)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:47.553093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43fce31-a79d-4a74-8bff-c64b85821a94 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:47.631396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d43fce31-a79d-4a74-8bff-c64b85821a94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adf310503fa74abf83949a914e728693', 'request_id': 'req-d43fce31-a79d-4a74-8bff-c64b85821a94'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:47.645522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b26a2-17ca-492a-8cce-21526525afa5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:47.646684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d43fce31-a79d-4a74-8bff-c64b85821a94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adf310503fa74abf83949a914e728693', 'request_id': 'req-d43fce31-a79d-4a74-8bff-c64b85821a94'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:47.657535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d43fce31-a79d-4a74-8bff-c64b85821a94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adf310503fa74abf83949a914e728693', 'request_id': 'req-d43fce31-a79d-4a74-8bff-c64b85821a94'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:47.679708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d43fce31-a79d-4a74-8bff-c64b85821a94 admin admin] The token cache is being invalidate because project adf310503fa74abf83949a914e728693 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:04:47.683445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d43fce31-a79d-4a74-8bff-c64b85821a94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:47.686314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d43fce31-a79d-4a74-8bff-c64b85821a94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:47.692641 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2347/4777] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:04:47 2022] DELETE /identity/v3/projects/adf310503fa74abf83949a914e728693 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:47.692641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43fce31-a79d-4a74-8bff-c64b85821a94 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:47.698580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d43fce31-a79d-4a74-8bff-c64b85821a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:47.764676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0cff01-87e8-46fc-bdda-38ae084ff2e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:47.765466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0cff01-87e8-46fc-bdda-38ae084ff2e5 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:47.766061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0cff01-87e8-46fc-bdda-38ae084ff2e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:47.767096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0cff01-87e8-46fc-bdda-38ae084ff2e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:47.770173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ab0cff01-87e8-46fc-bdda-38ae084ff2e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:47.796405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:47.805727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43fce31-a79d-4a74-8bff-c64b85821a94 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:47.819477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5367a6-4288-43c4-8218-4f11458f5221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:47.820806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5367a6-4288-43c4-8218-4f11458f5221 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:47.822837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5367a6-4288-43c4-8218-4f11458f5221 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:47.823164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5367a6-4288-43c4-8218-4f11458f5221 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:47.824812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5367a6-4288-43c4-8218-4f11458f5221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:47.829372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5367a6-4288-43c4-8218-4f11458f5221 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:47.855774 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2348/4778] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:47.857124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4805b-4a04-4081-90de-cff6807ac105 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:47.857598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4805b-4a04-4081-90de-cff6807ac105 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:47.858465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4805b-4a04-4081-90de-cff6807ac105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:47.893239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0cff01-87e8-46fc-bdda-38ae084ff2e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:47.895304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0cff01-87e8-46fc-bdda-38ae084ff2e5 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:47.918633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-23b4805b-4a04-4081-90de-cff6807ac105 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:47.922092 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2431/4779] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:47.923841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0cff01-87e8-46fc-bdda-38ae084ff2e5 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:47.926376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-23b4805b-4a04-4081-90de-cff6807ac105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:47.940276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0cff01-87e8-46fc-bdda-38ae084ff2e5 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:47.948390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f5c0f-0e58-4eec-b8be-4a82623baec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:47.949771 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2349/4780] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:47.949996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f5c0f-0e58-4eec-b8be-4a82623baec7 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:47.950513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f5c0f-0e58-4eec-b8be-4a82623baec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:47.950891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f5c0f-0e58-4eec-b8be-4a82623baec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:47.951818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f5c0f-0e58-4eec-b8be-4a82623baec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:47.953487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f5c0f-0e58-4eec-b8be-4a82623baec7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:47.957768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b2d07-3493-4368-80e7-89af2eb624ca None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:47.958631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b2d07-3493-4368-80e7-89af2eb624ca None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:47.959126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b2d07-3493-4368-80e7-89af2eb624ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:47.969722 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2432/4781] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:47.981460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f5c0f-0e58-4eec-b8be-4a82623baec7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:48.002274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8e2b2d07-3493-4368-80e7-89af2eb624ca None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:48.005700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8e2b2d07-3493-4368-80e7-89af2eb624ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:48.010595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f5c0f-0e58-4eec-b8be-4a82623baec7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:48.021273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e6b5e-5fe4-4b20-bbf8-1ca7a5a83bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:48.026376 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2350/4782] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:04:48.027641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b2d07-3493-4368-80e7-89af2eb624ca None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:48.028413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e6b5e-5fe4-4b20-bbf8-1ca7a5a83bc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:48.029374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e6b5e-5fe4-4b20-bbf8-1ca7a5a83bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:48.029796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e6b5e-5fe4-4b20-bbf8-1ca7a5a83bc9 admin admin] PATH_INFO: `/v3/projects/8ea6424dd45e48ad93390e398a6fc926/users/46c6f0cba68e41eab2d90bac5fba9442/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:48.047995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b2d07-3493-4368-80e7-89af2eb624ca None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:48.056645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e6b5e-5fe4-4b20-bbf8-1ca7a5a83bc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ea6424dd45e48ad93390e398a6fc926, user_id=46c6f0cba68e41eab2d90bac5fba9442, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:48.057980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756a957-1745-4708-bb9f-9a5f12cb7354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:48.060237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756a957-1745-4708-bb9f-9a5f12cb7354 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:48.060523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756a957-1745-4708-bb9f-9a5f12cb7354 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:48.060801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756a957-1745-4708-bb9f-9a5f12cb7354 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:48.061459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756a957-1745-4708-bb9f-9a5f12cb7354 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:04:48.061781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756a957-1745-4708-bb9f-9a5f12cb7354 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:48.063846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e6b5e-5fe4-4b20-bbf8-1ca7a5a83bc9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:48.066602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756a957-1745-4708-bb9f-9a5f12cb7354 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:48.085761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:48.087595 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2351/4783] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:04:48.095363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756a957-1745-4708-bb9f-9a5f12cb7354 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:48.106995 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2433/4784] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:04:47 2022] PUT /identity/v3/projects/8ea6424dd45e48ad93390e398a6fc926/users/46c6f0cba68e41eab2d90bac5fba9442/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:48.109870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce55cf-ad3f-4d94-ad4b-3038fab4334f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:48.110816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce55cf-ad3f-4d94-ad4b-3038fab4334f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:48.111145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce55cf-ad3f-4d94-ad4b-3038fab4334f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:48.132206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756a957-1745-4708-bb9f-9a5f12cb7354 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:48.191969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-45ce55cf-ad3f-4d94-ad4b-3038fab4334f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:48.195750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-45ce55cf-ad3f-4d94-ad4b-3038fab4334f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:48.298327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66185bac-c470-404a-95ea-fc3d6f0a3345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:48.305226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66185bac-c470-404a-95ea-fc3d6f0a3345 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:48.307408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66185bac-c470-404a-95ea-fc3d6f0a3345 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:48.308326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66185bac-c470-404a-95ea-fc3d6f0a3345 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:48.308972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66185bac-c470-404a-95ea-fc3d6f0a3345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:48.311307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66185bac-c470-404a-95ea-fc3d6f0a3345 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:48.371562 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2434/4785] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:48.397976 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2352/4786] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:04:48 2022] POST /identity/v3/projects => generated 383 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:48.398395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce55cf-ad3f-4d94-ad4b-3038fab4334f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:48.403396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66185bac-c470-404a-95ea-fc3d6f0a3345 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:48.408795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-66185bac-c470-404a-95ea-fc3d6f0a3345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:48.412236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce55cf-ad3f-4d94-ad4b-3038fab4334f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:48.530101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32020a-d74e-413a-ae86-75cd9ace8ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:48.531414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32020a-d74e-413a-ae86-75cd9ace8ca0 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:48.532580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32020a-d74e-413a-ae86-75cd9ace8ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:48.533086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32020a-d74e-413a-ae86-75cd9ace8ca0 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:48.535365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c32020a-d74e-413a-ae86-75cd9ace8ca0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:48.537893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c32020a-d74e-413a-ae86-75cd9ace8ca0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:48.544901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66185bac-c470-404a-95ea-fc3d6f0a3345 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:48.555686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983f636-ebfa-4a22-b120-d96753334f0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:48.557043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983f636-ebfa-4a22-b120-d96753334f0c service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:48.557830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983f636-ebfa-4a22-b120-d96753334f0c service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:48.558127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983f636-ebfa-4a22-b120-d96753334f0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:48.584779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a983f636-ebfa-4a22-b120-d96753334f0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:48.589198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a983f636-ebfa-4a22-b120-d96753334f0c service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:48.594993 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2435/4787] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:04:48 2022] POST /identity/v3/users => generated 292 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:04:48.596461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32020a-d74e-413a-ae86-75cd9ace8ca0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:48.603314 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2353/4788] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:48.610815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32020a-d74e-413a-ae86-75cd9ace8ca0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:48.616531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a6c22-7df2-4457-8991-3fbcd56daf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:48.617598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a6c22-7df2-4457-8991-3fbcd56daf18 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:48.618251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a6c22-7df2-4457-8991-3fbcd56daf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:48.618733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a6c22-7df2-4457-8991-3fbcd56daf18 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:48.619511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a6c22-7df2-4457-8991-3fbcd56daf18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:48.621318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a6c22-7df2-4457-8991-3fbcd56daf18 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:48.634575 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2436/4789] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:48.639570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983f636-ebfa-4a22-b120-d96753334f0c service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:48.659060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983f636-ebfa-4a22-b120-d96753334f0c service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:48.665498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621cfeed-27d5-449b-a05d-dff4e1423d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:48.667347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cfeed-27d5-449b-a05d-dff4e1423d88 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:48.667636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cfeed-27d5-449b-a05d-dff4e1423d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:48.667911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cfeed-27d5-449b-a05d-dff4e1423d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:48.668745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621cfeed-27d5-449b-a05d-dff4e1423d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:48.671430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621cfeed-27d5-449b-a05d-dff4e1423d88 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:48.691343 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2354/4790] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:48.693159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a6c22-7df2-4457-8991-3fbcd56daf18 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:48.705959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a6c22-7df2-4457-8991-3fbcd56daf18 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:48.711639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef17507-35ba-49ef-ac59-ab90ea8d98db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:48.712715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef17507-35ba-49ef-ac59-ab90ea8d98db admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:48.713093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef17507-35ba-49ef-ac59-ab90ea8d98db admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:48.713467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef17507-35ba-49ef-ac59-ab90ea8d98db admin admin] PATH_INFO: `/v3/projects/6c0ad82595344eb4a1122d294faf15b2/users/6d63c633112e49d091421c84cea3422b/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:48.733928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef17507-35ba-49ef-ac59-ab90ea8d98db admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c0ad82595344eb4a1122d294faf15b2, user_id=6d63c633112e49d091421c84cea3422b, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:48.740954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef17507-35ba-49ef-ac59-ab90ea8d98db admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:48.776439 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2437/4791] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:04:48 2022] PUT /identity/v3/projects/6c0ad82595344eb4a1122d294faf15b2/users/6d63c633112e49d091421c84cea3422b/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:48.778743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34000545-ec69-484a-9834-f9f6f7f7602e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:48.779552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34000545-ec69-484a-9834-f9f6f7f7602e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:48.780189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34000545-ec69-484a-9834-f9f6f7f7602e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:48.824307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-34000545-ec69-484a-9834-f9f6f7f7602e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:48.825607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-34000545-ec69-484a-9834-f9f6f7f7602e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:48.891094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:48.942225 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2355/4792] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:49.001186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef17507-35ba-49ef-ac59-ab90ea8d98db admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:49.003269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4ef17507-35ba-49ef-ac59-ab90ea8d98db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:49.135745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef17507-35ba-49ef-ac59-ab90ea8d98db admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:49.147354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32aef0-8568-47e7-a353-45747bc55a46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:49.148258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32aef0-8568-47e7-a353-45747bc55a46 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:49.148647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32aef0-8568-47e7-a353-45747bc55a46 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:49.149013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32aef0-8568-47e7-a353-45747bc55a46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:49.167989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:49.179521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b32aef0-8568-47e7-a353-45747bc55a46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:49.181954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b32aef0-8568-47e7-a353-45747bc55a46 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:49.204251 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2438/4793] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:49.341896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34000545-ec69-484a-9834-f9f6f7f7602e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:49.343783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-34000545-ec69-484a-9834-f9f6f7f7602e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:49.390880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34000545-ec69-484a-9834-f9f6f7f7602e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:49.432951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561984e-5568-4149-afec-b1fa77cdd392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:49.436091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561984e-5568-4149-afec-b1fa77cdd392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:49.436618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561984e-5568-4149-afec-b1fa77cdd392 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:49.437066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561984e-5568-4149-afec-b1fa77cdd392 admin admin] PATH_INFO: `/v3/projects/e8be683460bb4157824a7f999d0dd106` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:49.439861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561984e-5568-4149-afec-b1fa77cdd392 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8be683460bb4157824a7f999d0dd106)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:49.448229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561984e-5568-4149-afec-b1fa77cdd392 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:49.532779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f561984e-5568-4149-afec-b1fa77cdd392 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8be683460bb4157824a7f999d0dd106', 'request_id': 'req-f561984e-5568-4149-afec-b1fa77cdd392'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:49.548925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f561984e-5568-4149-afec-b1fa77cdd392 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8be683460bb4157824a7f999d0dd106', 'request_id': 'req-f561984e-5568-4149-afec-b1fa77cdd392'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:49.561225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f561984e-5568-4149-afec-b1fa77cdd392 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8be683460bb4157824a7f999d0dd106', 'request_id': 'req-f561984e-5568-4149-afec-b1fa77cdd392'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:49.581320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f561984e-5568-4149-afec-b1fa77cdd392 admin admin] The token cache is being invalidate because project e8be683460bb4157824a7f999d0dd106 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:04:49.581919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f561984e-5568-4149-afec-b1fa77cdd392 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:49.582948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f561984e-5568-4149-afec-b1fa77cdd392 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:49.587383 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2356/4794] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:04:49 2022] DELETE /identity/v3/projects/e8be683460bb4157824a7f999d0dd106 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:49.592569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718dc061-13cd-45a6-8afa-406700b1573b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:49.593098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718dc061-13cd-45a6-8afa-406700b1573b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:49.593623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718dc061-13cd-45a6-8afa-406700b1573b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:49.644818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-718dc061-13cd-45a6-8afa-406700b1573b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:49.648817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-718dc061-13cd-45a6-8afa-406700b1573b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:49.770667 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2439/4795] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:04:49.774612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e0251-4a23-49b7-bfb2-e9db62f880c3 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:49.775498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e0251-4a23-49b7-bfb2-e9db62f880c3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:49.776104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e0251-4a23-49b7-bfb2-e9db62f880c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:49.820122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1d8e0251-4a23-49b7-bfb2-e9db62f880c3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:49.821362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1d8e0251-4a23-49b7-bfb2-e9db62f880c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:49.849810 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2357/4796] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:04:49.852465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718dc061-13cd-45a6-8afa-406700b1573b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:49.864367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718dc061-13cd-45a6-8afa-406700b1573b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:49.872482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4f927-78d0-45c0-9ee7-b805e2a24fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:49.873358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4f927-78d0-45c0-9ee7-b805e2a24fc2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:49.873856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4f927-78d0-45c0-9ee7-b805e2a24fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:49.874341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4f927-78d0-45c0-9ee7-b805e2a24fc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:49.875313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4f927-78d0-45c0-9ee7-b805e2a24fc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:04:49.876438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4f927-78d0-45c0-9ee7-b805e2a24fc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:49.879830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4f927-78d0-45c0-9ee7-b805e2a24fc2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:49.934516 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2440/4797] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:04:49.936002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e0251-4a23-49b7-bfb2-e9db62f880c3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:49.959541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e0251-4a23-49b7-bfb2-e9db62f880c3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:49.980906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a4999-4e50-4fe1-9a81-9f4ffd6c11dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:49.980906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a4999-4e50-4fe1-9a81-9f4ffd6c11dd admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:49.980906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a4999-4e50-4fe1-9a81-9f4ffd6c11dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:49.981621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a4999-4e50-4fe1-9a81-9f4ffd6c11dd admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:49.983569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a4999-4e50-4fe1-9a81-9f4ffd6c11dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:49.986683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a4999-4e50-4fe1-9a81-9f4ffd6c11dd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:50.031669 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2358/4798] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:04:49 2022] POST /identity/v3/projects => generated 373 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:50.034309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4f927-78d0-45c0-9ee7-b805e2a24fc2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:50.067246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4f927-78d0-45c0-9ee7-b805e2a24fc2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:50.262681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfdfe1-b88c-484c-af24-3967a5058028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:50.263327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfdfe1-b88c-484c-af24-3967a5058028 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:50.263588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfdfe1-b88c-484c-af24-3967a5058028 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:50.263852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfdfe1-b88c-484c-af24-3967a5058028 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:50.264541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfdfe1-b88c-484c-af24-3967a5058028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:50.267057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfdfe1-b88c-484c-af24-3967a5058028 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:50.316748 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2441/4799] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:04:50 2022] POST /identity/v3/users => generated 287 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:50.320664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a4999-4e50-4fe1-9a81-9f4ffd6c11dd admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:50.341289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a4999-4e50-4fe1-9a81-9f4ffd6c11dd admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:50.359537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47660e-8ab1-4488-9f8a-d5ba7deecd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:50.361484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47660e-8ab1-4488-9f8a-d5ba7deecd5f admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:50.362367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47660e-8ab1-4488-9f8a-d5ba7deecd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:50.362746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47660e-8ab1-4488-9f8a-d5ba7deecd5f admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:50.363628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e47660e-8ab1-4488-9f8a-d5ba7deecd5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:50.366871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e47660e-8ab1-4488-9f8a-d5ba7deecd5f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:50.410680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfdfe1-b88c-484c-af24-3967a5058028 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:50.425015 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2359/4800] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:50.433253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfdfe1-b88c-484c-af24-3967a5058028 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:50.441414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef3d40-ee51-40b1-8c3d-19841ccf1c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:50.442454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef3d40-ee51-40b1-8c3d-19841ccf1c18 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:50.443423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef3d40-ee51-40b1-8c3d-19841ccf1c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:50.443841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef3d40-ee51-40b1-8c3d-19841ccf1c18 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:50.444832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef3d40-ee51-40b1-8c3d-19841ccf1c18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:50.448182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef3d40-ee51-40b1-8c3d-19841ccf1c18 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:50.463036 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2442/4801] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:50.467300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47660e-8ab1-4488-9f8a-d5ba7deecd5f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:50.493364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47660e-8ab1-4488-9f8a-d5ba7deecd5f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:50.509471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae84070-e157-4d7f-a81c-42462fa09201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:50.510804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae84070-e157-4d7f-a81c-42462fa09201 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:50.511557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae84070-e157-4d7f-a81c-42462fa09201 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:50.518152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae84070-e157-4d7f-a81c-42462fa09201 admin admin] PATH_INFO: `/v3/projects/3196d4aa5c2040a2b114b8b21352e4c2/users/d042705af4284cca8741304d690ae343/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:50.583096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae84070-e157-4d7f-a81c-42462fa09201 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3196d4aa5c2040a2b114b8b21352e4c2, user_id=d042705af4284cca8741304d690ae343, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:50.585421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae84070-e157-4d7f-a81c-42462fa09201 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:50.613738 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2360/4802] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:04:50 2022] PUT /identity/v3/projects/3196d4aa5c2040a2b114b8b21352e4c2/users/d042705af4284cca8741304d690ae343/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:50.627773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f07af3-5cfd-4451-9dca-aa90f5223192 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:50.628067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f07af3-5cfd-4451-9dca-aa90f5223192 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:50.628439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f07af3-5cfd-4451-9dca-aa90f5223192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:50.697056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-18f07af3-5cfd-4451-9dca-aa90f5223192 None None] MFA Rules not processed for user `d042705af4284cca8741304d690ae343`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:50.698372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-18f07af3-5cfd-4451-9dca-aa90f5223192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:50.774369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:50.811507 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2443/4803] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:51.228777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae84070-e157-4d7f-a81c-42462fa09201 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:51.231072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0ae84070-e157-4d7f-a81c-42462fa09201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:51.407220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:51.423759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae84070-e157-4d7f-a81c-42462fa09201 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:51.440232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feac06a-8746-44ea-8189-586fc032bb5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:51.441297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feac06a-8746-44ea-8189-586fc032bb5f service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:51.441791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feac06a-8746-44ea-8189-586fc032bb5f service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:51.442593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feac06a-8746-44ea-8189-586fc032bb5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:51.456235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feac06a-8746-44ea-8189-586fc032bb5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:51.458167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feac06a-8746-44ea-8189-586fc032bb5f service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:51.475019 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2361/4804] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:54.468889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db42c7-18bf-4f48-b40c-72c2eafd0054 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:54.470033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db42c7-18bf-4f48-b40c-72c2eafd0054 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:54.470033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db42c7-18bf-4f48-b40c-72c2eafd0054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:54.522282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-97db42c7-18bf-4f48-b40c-72c2eafd0054 None None] MFA Rules not processed for user `f6f982c714554445b6510ca5f726d126`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:54.526632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-97db42c7-18bf-4f48-b40c-72c2eafd0054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:54.631889 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2444/4805] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:54.644378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feac06a-8746-44ea-8189-586fc032bb5f service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:54.646506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0feac06a-8746-44ea-8189-586fc032bb5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:54.748497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db42c7-18bf-4f48-b40c-72c2eafd0054 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:54.789401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feac06a-8746-44ea-8189-586fc032bb5f service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:54.800311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0409d-a12a-44c4-a5f3-e9ffdc008277 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:54.800939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0409d-a12a-44c4-a5f3-e9ffdc008277 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:54.801221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0409d-a12a-44c4-a5f3-e9ffdc008277 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:54.801494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0409d-a12a-44c4-a5f3-e9ffdc008277 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:54.806414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bbc0409d-a12a-44c4-a5f3-e9ffdc008277 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:54.893546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db42c7-18bf-4f48-b40c-72c2eafd0054 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:54.915422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e34b6b-5342-488b-8c49-afc07dc17e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:54.916380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e34b6b-5342-488b-8c49-afc07dc17e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:54.916825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e34b6b-5342-488b-8c49-afc07dc17e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:54.918073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e34b6b-5342-488b-8c49-afc07dc17e39 admin admin] PATH_INFO: `/v3/users/d042705af4284cca8741304d690ae343` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:54.921294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e34b6b-5342-488b-8c49-afc07dc17e39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d042705af4284cca8741304d690ae343)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:54.924038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e34b6b-5342-488b-8c49-afc07dc17e39 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:54.933445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0409d-a12a-44c4-a5f3-e9ffdc008277 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:54.934982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0409d-a12a-44c4-a5f3-e9ffdc008277 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:54.947414 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2362/4806] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:55.095786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-29e34b6b-5342-488b-8c49-afc07dc17e39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd042705af4284cca8741304d690ae343'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:55.114035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-29e34b6b-5342-488b-8c49-afc07dc17e39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd042705af4284cca8741304d690ae343'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:55.152812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-29e34b6b-5342-488b-8c49-afc07dc17e39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd042705af4284cca8741304d690ae343'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:55.153663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-29e34b6b-5342-488b-8c49-afc07dc17e39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd042705af4284cca8741304d690ae343'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:55.163158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:55.171277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-29e34b6b-5342-488b-8c49-afc07dc17e39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd042705af4284cca8741304d690ae343'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:55.185499 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2445/4807] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:04:54 2022] DELETE /identity/v3/users/d042705af4284cca8741304d690ae343 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:04:55.191288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ec622-44db-4a7b-b65f-4c3580d83625 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:55.191288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ec622-44db-4a7b-b65f-4c3580d83625 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:55.191288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ec622-44db-4a7b-b65f-4c3580d83625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:55.225622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bf9ec622-44db-4a7b-b65f-4c3580d83625 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:55.226863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bf9ec622-44db-4a7b-b65f-4c3580d83625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:55.425126 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2363/4808] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:55.921341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e34b6b-5342-488b-8c49-afc07dc17e39 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:55.924771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-29e34b6b-5342-488b-8c49-afc07dc17e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:56.102416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e34b6b-5342-488b-8c49-afc07dc17e39 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:56.106636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e7f42-1dbc-48b6-9a98-2c0458f38848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:56.107782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e7f42-1dbc-48b6-9a98-2c0458f38848 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:56.108261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e7f42-1dbc-48b6-9a98-2c0458f38848 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:56.108725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e7f42-1dbc-48b6-9a98-2c0458f38848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:56.123546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e7f42-1dbc-48b6-9a98-2c0458f38848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:56.125926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e7f42-1dbc-48b6-9a98-2c0458f38848 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:56.144253 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2446/4809] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:04:57.574881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ec622-44db-4a7b-b65f-4c3580d83625 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:57.579239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bf9ec622-44db-4a7b-b65f-4c3580d83625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:57.597972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ec622-44db-4a7b-b65f-4c3580d83625 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:57.604084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9eeb2e-e29f-47bb-b4db-5548057d37e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:57.604996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9eeb2e-e29f-47bb-b4db-5548057d37e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:57.605454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9eeb2e-e29f-47bb-b4db-5548057d37e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:57.605897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9eeb2e-e29f-47bb-b4db-5548057d37e3 admin admin] PATH_INFO: `/v3/projects/3196d4aa5c2040a2b114b8b21352e4c2` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:57.607264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9eeb2e-e29f-47bb-b4db-5548057d37e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3196d4aa5c2040a2b114b8b21352e4c2)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:57.610321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9eeb2e-e29f-47bb-b4db-5548057d37e3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:57.707578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:57.759909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3a9eeb2e-e29f-47bb-b4db-5548057d37e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3196d4aa5c2040a2b114b8b21352e4c2', 'request_id': 'req-3a9eeb2e-e29f-47bb-b4db-5548057d37e3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:57.775547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3a9eeb2e-e29f-47bb-b4db-5548057d37e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3196d4aa5c2040a2b114b8b21352e4c2', 'request_id': 'req-3a9eeb2e-e29f-47bb-b4db-5548057d37e3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:57.789140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3a9eeb2e-e29f-47bb-b4db-5548057d37e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3196d4aa5c2040a2b114b8b21352e4c2', 'request_id': 'req-3a9eeb2e-e29f-47bb-b4db-5548057d37e3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:57.819224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3a9eeb2e-e29f-47bb-b4db-5548057d37e3 admin admin] The token cache is being invalidate because project 3196d4aa5c2040a2b114b8b21352e4c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:04:57.819936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3a9eeb2e-e29f-47bb-b4db-5548057d37e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:57.821554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3a9eeb2e-e29f-47bb-b4db-5548057d37e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:04:57.829090 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2364/4810] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:04:57 2022] DELETE /identity/v3/projects/3196d4aa5c2040a2b114b8b21352e4c2 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:04:57.830085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb693864-3f17-4c10-be5a-a17b1e362331 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:57.830906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb693864-3f17-4c10-be5a-a17b1e362331 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:57.830906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb693864-3f17-4c10-be5a-a17b1e362331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:57.895684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-cb693864-3f17-4c10-be5a-a17b1e362331 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:57.896922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cb693864-3f17-4c10-be5a-a17b1e362331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:58.234576 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2447/4811] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:58.236160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6054c-af81-4139-9692-4dee6cf6eb97 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:58.237358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6054c-af81-4139-9692-4dee6cf6eb97 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:58.237821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6054c-af81-4139-9692-4dee6cf6eb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:58.279751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-40d6054c-af81-4139-9692-4dee6cf6eb97 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:58.281208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-40d6054c-af81-4139-9692-4dee6cf6eb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:58.299364 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2365/4812] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:58.306330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb693864-3f17-4c10-be5a-a17b1e362331 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:58.324757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb693864-3f17-4c10-be5a-a17b1e362331 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:58.345365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730d104-f8eb-45b3-9c05-a0d6f98205f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:58.346903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730d104-f8eb-45b3-9c05-a0d6f98205f5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:58.347188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730d104-f8eb-45b3-9c05-a0d6f98205f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:58.347463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730d104-f8eb-45b3-9c05-a0d6f98205f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:58.348877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730d104-f8eb-45b3-9c05-a0d6f98205f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:04:58.349371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730d104-f8eb-45b3-9c05-a0d6f98205f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:58.352664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730d104-f8eb-45b3-9c05-a0d6f98205f5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:58.392302 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2448/4813] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:04:58.400976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6054c-af81-4139-9692-4dee6cf6eb97 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:58.418216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6054c-af81-4139-9692-4dee6cf6eb97 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:58.430956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ed7d5-77bc-42d1-bbcf-b70a1de3b044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:58.431975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ed7d5-77bc-42d1-bbcf-b70a1de3b044 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:58.432347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ed7d5-77bc-42d1-bbcf-b70a1de3b044 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:58.432726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ed7d5-77bc-42d1-bbcf-b70a1de3b044 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:58.433449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ed7d5-77bc-42d1-bbcf-b70a1de3b044 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:58.435747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ed7d5-77bc-42d1-bbcf-b70a1de3b044 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:58.479308 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2366/4814] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:04:58 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:58.483381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730d104-f8eb-45b3-9c05-a0d6f98205f5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:58.499470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730d104-f8eb-45b3-9c05-a0d6f98205f5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:58.654044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55201b-4c62-4141-9ee5-27805d7f746c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:58.655177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55201b-4c62-4141-9ee5-27805d7f746c admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:58.655705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55201b-4c62-4141-9ee5-27805d7f746c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:58.656272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55201b-4c62-4141-9ee5-27805d7f746c admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:58.657886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55201b-4c62-4141-9ee5-27805d7f746c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:58.660114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55201b-4c62-4141-9ee5-27805d7f746c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:58.705990 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2449/4815] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:04:58 2022] POST /identity/v3/users => generated 287 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:04:58.710131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ed7d5-77bc-42d1-bbcf-b70a1de3b044 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:58.740947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ed7d5-77bc-42d1-bbcf-b70a1de3b044 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:58.767879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ee174-ed7c-4dd3-9ffb-6ebc1bd0fc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:58.768964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ee174-ed7c-4dd3-9ffb-6ebc1bd0fc4b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:58.769419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ee174-ed7c-4dd3-9ffb-6ebc1bd0fc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:58.769883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ee174-ed7c-4dd3-9ffb-6ebc1bd0fc4b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:58.770725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ee174-ed7c-4dd3-9ffb-6ebc1bd0fc4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:58.775603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ee174-ed7c-4dd3-9ffb-6ebc1bd0fc4b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:58.812466 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2367/4816] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:58.818279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55201b-4c62-4141-9ee5-27805d7f746c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:58.832790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55201b-4c62-4141-9ee5-27805d7f746c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:58.850780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0aefc7-4c7f-4270-8ee7-1fb6e9ce4390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:58.852088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aefc7-4c7f-4270-8ee7-1fb6e9ce4390 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:58.852671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aefc7-4c7f-4270-8ee7-1fb6e9ce4390 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:58.853149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aefc7-4c7f-4270-8ee7-1fb6e9ce4390 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:58.853820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0aefc7-4c7f-4270-8ee7-1fb6e9ce4390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:58.855491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0aefc7-4c7f-4270-8ee7-1fb6e9ce4390 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:58.916294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ee174-ed7c-4dd3-9ffb-6ebc1bd0fc4b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:58.954484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ee174-ed7c-4dd3-9ffb-6ebc1bd0fc4b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:58.961979 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2450/4817] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:04:58.991753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be5efd-e47e-4635-af62-749ced2f0825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:58.994885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be5efd-e47e-4635-af62-749ced2f0825 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:58.996410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be5efd-e47e-4635-af62-749ced2f0825 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:58.996957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be5efd-e47e-4635-af62-749ced2f0825 admin admin] PATH_INFO: `/v3/projects/a797000ad5944ca096f1bb8db8051537/users/334a9101cc534dbc91272b95ab8b70f0/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:59.057012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be5efd-e47e-4635-af62-749ced2f0825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a797000ad5944ca096f1bb8db8051537, user_id=334a9101cc534dbc91272b95ab8b70f0, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:59.080681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be5efd-e47e-4635-af62-749ced2f0825 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:59.134791 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2368/4818] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:04:58 2022] PUT /identity/v3/projects/a797000ad5944ca096f1bb8db8051537/users/334a9101cc534dbc91272b95ab8b70f0/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:04:59.149393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b26b92-730e-481f-8560-6f40a4441bae None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:59.149884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b26b92-730e-481f-8560-6f40a4441bae None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:59.150229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b26b92-730e-481f-8560-6f40a4441bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:59.164308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:59.209566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b6b26b92-730e-481f-8560-6f40a4441bae None None] MFA Rules not processed for user `334a9101cc534dbc91272b95ab8b70f0`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:04:59.211519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b6b26b92-730e-481f-8560-6f40a4441bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:59.308451 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2451/4819] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:04:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:04:59.614897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be5efd-e47e-4635-af62-749ced2f0825 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:04:59.617297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d0be5efd-e47e-4635-af62-749ced2f0825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:04:59.729695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be5efd-e47e-4635-af62-749ced2f0825 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:04:59.738693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e717c3-58bb-42c2-98b8-6ea119949ab6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:04:59.739363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e717c3-58bb-42c2-98b8-6ea119949ab6 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:04:59.739742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e717c3-58bb-42c2-98b8-6ea119949ab6 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:04:59.740315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e717c3-58bb-42c2-98b8-6ea119949ab6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:04:59.753193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e717c3-58bb-42c2-98b8-6ea119949ab6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:04:59.755265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e717c3-58bb-42c2-98b8-6ea119949ab6 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:04:59.773359 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2369/4820] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:04:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:05:01.202915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b26b92-730e-481f-8560-6f40a4441bae None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:01.206142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b6b26b92-730e-481f-8560-6f40a4441bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:01.452141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b26b92-730e-481f-8560-6f40a4441bae None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:01.482315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e717c3-58bb-42c2-98b8-6ea119949ab6 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:01.493831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe09d58-f5d2-483d-a071-cf57e1fc33a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:01.494563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe09d58-f5d2-483d-a071-cf57e1fc33a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:01.494967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe09d58-f5d2-483d-a071-cf57e1fc33a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:01.495588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe09d58-f5d2-483d-a071-cf57e1fc33a6 admin admin] PATH_INFO: `/v3/users/477eb33574da473284d09e765825bcc6` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:01.497206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe09d58-f5d2-483d-a071-cf57e1fc33a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=477eb33574da473284d09e765825bcc6)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:01.501951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe09d58-f5d2-483d-a071-cf57e1fc33a6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:01.511300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e717c3-58bb-42c2-98b8-6ea119949ab6 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:01.524532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d8008-6e55-4377-ba40-762c082feecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:01.525368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d8008-6e55-4377-ba40-762c082feecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:01.526343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d8008-6e55-4377-ba40-762c082feecb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:01.526343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d8008-6e55-4377-ba40-762c082feecb admin admin] PATH_INFO: `/v3/users/334a9101cc534dbc91272b95ab8b70f0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:01.533867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d8008-6e55-4377-ba40-762c082feecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=334a9101cc534dbc91272b95ab8b70f0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:01.540344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d8008-6e55-4377-ba40-762c082feecb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:01.621271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8fe09d58-f5d2-483d-a071-cf57e1fc33a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '477eb33574da473284d09e765825bcc6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:01.644160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8fe09d58-f5d2-483d-a071-cf57e1fc33a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '477eb33574da473284d09e765825bcc6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:01.683528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8fe09d58-f5d2-483d-a071-cf57e1fc33a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '477eb33574da473284d09e765825bcc6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:01.686635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8fe09d58-f5d2-483d-a071-cf57e1fc33a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '477eb33574da473284d09e765825bcc6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:01.739705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:01.744991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8fe09d58-f5d2-483d-a071-cf57e1fc33a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '477eb33574da473284d09e765825bcc6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:01.772637 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2452/4821] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:01 2022] DELETE /identity/v3/users/477eb33574da473284d09e765825bcc6 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:01.790589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42276090-302c-4de4-8a68-9f2e068277c1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:01.790589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42276090-302c-4de4-8a68-9f2e068277c1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:01.790589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42276090-302c-4de4-8a68-9f2e068277c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:01.794039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1e2d8008-6e55-4377-ba40-762c082feecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '334a9101cc534dbc91272b95ab8b70f0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:01.797510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1e2d8008-6e55-4377-ba40-762c082feecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '334a9101cc534dbc91272b95ab8b70f0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:01.800709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1e2d8008-6e55-4377-ba40-762c082feecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '334a9101cc534dbc91272b95ab8b70f0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:01.820604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1e2d8008-6e55-4377-ba40-762c082feecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '334a9101cc534dbc91272b95ab8b70f0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:01.852875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1e2d8008-6e55-4377-ba40-762c082feecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '334a9101cc534dbc91272b95ab8b70f0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:01.860666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-42276090-302c-4de4-8a68-9f2e068277c1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:01.871616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-42276090-302c-4de4-8a68-9f2e068277c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:01.895538 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2370/4822] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:01 2022] DELETE /identity/v3/users/334a9101cc534dbc91272b95ab8b70f0 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:05:01.898218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b389500-6a3d-4a04-b442-0b6a138d9a48 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:01.898528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b389500-6a3d-4a04-b442-0b6a138d9a48 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:01.898806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b389500-6a3d-4a04-b442-0b6a138d9a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:01.918007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:01.942834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3b389500-6a3d-4a04-b442-0b6a138d9a48 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:01.944140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3b389500-6a3d-4a04-b442-0b6a138d9a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:02.089671 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2371/4823] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:02.106234 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2453/4824] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:05:02.116267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b389500-6a3d-4a04-b442-0b6a138d9a48 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:02.117759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3b389500-6a3d-4a04-b442-0b6a138d9a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:02.271735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b389500-6a3d-4a04-b442-0b6a138d9a48 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:02.279570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba9909-d0d2-4d97-af16-3fb162b571b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:02.280534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba9909-d0d2-4d97-af16-3fb162b571b9 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:02.281037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba9909-d0d2-4d97-af16-3fb162b571b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:02.282209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba9909-d0d2-4d97-af16-3fb162b571b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:02.302679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba9909-d0d2-4d97-af16-3fb162b571b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:02.305559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba9909-d0d2-4d97-af16-3fb162b571b9 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:02.358121 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2372/4825] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:05:02.818411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42276090-302c-4de4-8a68-9f2e068277c1 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:02.973555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee330c-c418-4b1e-97e5-dda051f93693 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:02.974123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee330c-c418-4b1e-97e5-dda051f93693 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:02.974123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee330c-c418-4b1e-97e5-dda051f93693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:03.045175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42276090-302c-4de4-8a68-9f2e068277c1 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:03.058410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ec0bc-4654-4c03-9260-362b94e91abf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:03.061246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ec0bc-4654-4c03-9260-362b94e91abf service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:03.061537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ec0bc-4654-4c03-9260-362b94e91abf service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:03.061817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ec0bc-4654-4c03-9260-362b94e91abf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:03.083109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ffee330c-c418-4b1e-97e5-dda051f93693 None None] MFA Rules not processed for user `6d63c633112e49d091421c84cea3422b`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:03.090203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ffee330c-c418-4b1e-97e5-dda051f93693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:03.166714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ec0bc-4654-4c03-9260-362b94e91abf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:03.169390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ec0bc-4654-4c03-9260-362b94e91abf service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:03.206191 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2454/4826] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:05:03.271757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:03.290310 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2373/4827] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:03.301367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1592aee-f657-4eff-87c3-a30e14eedc66 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:03.302014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1592aee-f657-4eff-87c3-a30e14eedc66 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:03.302597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1592aee-f657-4eff-87c3-a30e14eedc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:03.334967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f1592aee-f657-4eff-87c3-a30e14eedc66 None None] MFA Rules not processed for user `6d63c633112e49d091421c84cea3422b`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:03.336622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f1592aee-f657-4eff-87c3-a30e14eedc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:03.348197 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2455/4828] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:03.664771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee330c-c418-4b1e-97e5-dda051f93693 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:03.667049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ffee330c-c418-4b1e-97e5-dda051f93693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:03.808668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee330c-c418-4b1e-97e5-dda051f93693 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:03.817376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661dae14-a3d2-449a-939e-435b41008294 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:03.818068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dae14-a3d2-449a-939e-435b41008294 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:03.818354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dae14-a3d2-449a-939e-435b41008294 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:03.818908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dae14-a3d2-449a-939e-435b41008294 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:03.836037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661dae14-a3d2-449a-939e-435b41008294 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:03.837899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661dae14-a3d2-449a-939e-435b41008294 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:03.874576 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2374/4829] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:05:03.964848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1592aee-f657-4eff-87c3-a30e14eedc66 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:03.971415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f1592aee-f657-4eff-87c3-a30e14eedc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:04.007641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1592aee-f657-4eff-87c3-a30e14eedc66 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:04.022246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e8ca9-78ee-49f8-a776-3ee08767670d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:04.023273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e8ca9-78ee-49f8-a776-3ee08767670d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:04.023843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e8ca9-78ee-49f8-a776-3ee08767670d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:04.024283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e8ca9-78ee-49f8-a776-3ee08767670d admin admin] PATH_INFO: `/v3/projects/a797000ad5944ca096f1bb8db8051537` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:04.036110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e8ca9-78ee-49f8-a776-3ee08767670d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a797000ad5944ca096f1bb8db8051537)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:04.038665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e8ca9-78ee-49f8-a776-3ee08767670d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:04.144247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1a7e8ca9-78ee-49f8-a776-3ee08767670d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a797000ad5944ca096f1bb8db8051537', 'request_id': 'req-1a7e8ca9-78ee-49f8-a776-3ee08767670d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:04.169433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1a7e8ca9-78ee-49f8-a776-3ee08767670d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a797000ad5944ca096f1bb8db8051537', 'request_id': 'req-1a7e8ca9-78ee-49f8-a776-3ee08767670d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:04.182485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1a7e8ca9-78ee-49f8-a776-3ee08767670d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a797000ad5944ca096f1bb8db8051537', 'request_id': 'req-1a7e8ca9-78ee-49f8-a776-3ee08767670d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:04.222853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1a7e8ca9-78ee-49f8-a776-3ee08767670d admin admin] The token cache is being invalidate because project a797000ad5944ca096f1bb8db8051537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:04.223476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1a7e8ca9-78ee-49f8-a776-3ee08767670d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:04.224452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1a7e8ca9-78ee-49f8-a776-3ee08767670d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:04.226550 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2456/4830] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:03 2022] DELETE /identity/v3/projects/a797000ad5944ca096f1bb8db8051537 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:04.233384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78252a39-2ad5-4eec-b6bf-d9e6070892c1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:04.235029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78252a39-2ad5-4eec-b6bf-d9e6070892c1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:04.235600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78252a39-2ad5-4eec-b6bf-d9e6070892c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:04.247618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e8ca9-78ee-49f8-a776-3ee08767670d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:04.249811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1a7e8ca9-78ee-49f8-a776-3ee08767670d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:04.305671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-78252a39-2ad5-4eec-b6bf-d9e6070892c1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:04.311346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-78252a39-2ad5-4eec-b6bf-d9e6070892c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:04.321223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:04.393892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e8ca9-78ee-49f8-a776-3ee08767670d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:04.411523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adeebf-cd63-4317-b244-f6d72f899006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:04.412196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adeebf-cd63-4317-b244-f6d72f899006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:04.412476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adeebf-cd63-4317-b244-f6d72f899006 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:04.412807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adeebf-cd63-4317-b244-f6d72f899006 admin admin] PATH_INFO: `/v3/projects/cd3526ab00f3485e98db7f02e57bea4d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:04.417354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adeebf-cd63-4317-b244-f6d72f899006 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd3526ab00f3485e98db7f02e57bea4d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:04.419416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adeebf-cd63-4317-b244-f6d72f899006 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:04.469424 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2375/4831] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:04.555748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564069d2-d655-4ef3-84f8-2706126056c8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:04.556211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564069d2-d655-4ef3-84f8-2706126056c8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:04.557172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564069d2-d655-4ef3-84f8-2706126056c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:04.557373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e2adeebf-cd63-4317-b244-f6d72f899006 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd3526ab00f3485e98db7f02e57bea4d', 'request_id': 'req-e2adeebf-cd63-4317-b244-f6d72f899006'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:04.568222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e2adeebf-cd63-4317-b244-f6d72f899006 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd3526ab00f3485e98db7f02e57bea4d', 'request_id': 'req-e2adeebf-cd63-4317-b244-f6d72f899006'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:04.583526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e2adeebf-cd63-4317-b244-f6d72f899006 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd3526ab00f3485e98db7f02e57bea4d', 'request_id': 'req-e2adeebf-cd63-4317-b244-f6d72f899006'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:04.600967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-564069d2-d655-4ef3-84f8-2706126056c8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:04.602693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-564069d2-d655-4ef3-84f8-2706126056c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:04.614220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e2adeebf-cd63-4317-b244-f6d72f899006 admin admin] The token cache is being invalidate because project cd3526ab00f3485e98db7f02e57bea4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:04.615143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e2adeebf-cd63-4317-b244-f6d72f899006 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:04.618909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e2adeebf-cd63-4317-b244-f6d72f899006 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:04.634286 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2457/4832] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:04 2022] DELETE /identity/v3/projects/cd3526ab00f3485e98db7f02e57bea4d => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:04.635363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18468c36-0b61-4385-a9b0-ce05457119de None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:04.635707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18468c36-0b61-4385-a9b0-ce05457119de None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:04.635984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18468c36-0b61-4385-a9b0-ce05457119de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:04.678364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-18468c36-0b61-4385-a9b0-ce05457119de None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:04.679621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-18468c36-0b61-4385-a9b0-ce05457119de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:04.839405 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2458/4833] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:04.840446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18468c36-0b61-4385-a9b0-ce05457119de None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:04.840599 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2376/4834] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:04.844386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4a0f5-9bb1-4d47-8043-6e4ce34648b3 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:04.844897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4a0f5-9bb1-4d47-8043-6e4ce34648b3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:04.845999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4a0f5-9bb1-4d47-8043-6e4ce34648b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:04.861869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18468c36-0b61-4385-a9b0-ce05457119de None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:04.881786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe6cbf-2f2d-4ad7-8f64-cf7a736f62af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:04.882422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe6cbf-2f2d-4ad7-8f64-cf7a736f62af admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:04.882802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe6cbf-2f2d-4ad7-8f64-cf7a736f62af admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:04.883075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe6cbf-2f2d-4ad7-8f64-cf7a736f62af admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:04.883738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe6cbf-2f2d-4ad7-8f64-cf7a736f62af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:05:04.884046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe6cbf-2f2d-4ad7-8f64-cf7a736f62af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:04.885914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe6cbf-2f2d-4ad7-8f64-cf7a736f62af admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:04.911809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-afe4a0f5-9bb1-4d47-8043-6e4ce34648b3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:04.913074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-afe4a0f5-9bb1-4d47-8043-6e4ce34648b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:04.918654 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2459/4835] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:04.923652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe6cbf-2f2d-4ad7-8f64-cf7a736f62af admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:04.943782 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2377/4836] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:04.948454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4a0f5-9bb1-4d47-8043-6e4ce34648b3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:04.957952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe6cbf-2f2d-4ad7-8f64-cf7a736f62af admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:04.964539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4a0f5-9bb1-4d47-8043-6e4ce34648b3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:04.977624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe540d-2fd7-4f8a-b9c5-cd3b52cbecd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:04.978426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe540d-2fd7-4f8a-b9c5-cd3b52cbecd6 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:04.978742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe540d-2fd7-4f8a-b9c5-cd3b52cbecd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:04.980295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe540d-2fd7-4f8a-b9c5-cd3b52cbecd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:04.980426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ed33c-9660-46e1-bfdc-ecd67ec9e710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:04.980537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe540d-2fd7-4f8a-b9c5-cd3b52cbecd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:04.980627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ed33c-9660-46e1-bfdc-ecd67ec9e710 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:04.980627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ed33c-9660-46e1-bfdc-ecd67ec9e710 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:04.980627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ed33c-9660-46e1-bfdc-ecd67ec9e710 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:04.981010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ed33c-9660-46e1-bfdc-ecd67ec9e710 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:05:04.981740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ed33c-9660-46e1-bfdc-ecd67ec9e710 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:04.983303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ed33c-9660-46e1-bfdc-ecd67ec9e710 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:04.993313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe540d-2fd7-4f8a-b9c5-cd3b52cbecd6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:05.006319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:05.016137 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2378/4837] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:05.020279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ed33c-9660-46e1-bfdc-ecd67ec9e710 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:05.045164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ed33c-9660-46e1-bfdc-ecd67ec9e710 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:05.063582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf2955-b9b4-46d9-97c5-917b32896c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:05.064101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf2955-b9b4-46d9-97c5-917b32896c58 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:05.065027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf2955-b9b4-46d9-97c5-917b32896c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:05.065464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf2955-b9b4-46d9-97c5-917b32896c58 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:05.066110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf2955-b9b4-46d9-97c5-917b32896c58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:05.068109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf2955-b9b4-46d9-97c5-917b32896c58 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:05.094467 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2460/4838] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:04 2022] POST /identity/v3/projects => generated 373 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:05:05.095618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe540d-2fd7-4f8a-b9c5-cd3b52cbecd6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:05.164117 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2379/4839] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:05 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:05:05.168169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf2955-b9b4-46d9-97c5-917b32896c58 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:05.207983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:05.290183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe540d-2fd7-4f8a-b9c5-cd3b52cbecd6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:05.349084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf2955-b9b4-46d9-97c5-917b32896c58 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:05.364754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df1955-23c0-49fa-9d0e-c711dbb4cc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:05.365389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df1955-23c0-49fa-9d0e-c711dbb4cc26 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:05.365702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df1955-23c0-49fa-9d0e-c711dbb4cc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:05.365947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df1955-23c0-49fa-9d0e-c711dbb4cc26 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:05.366818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df1955-23c0-49fa-9d0e-c711dbb4cc26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:05.369900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df1955-23c0-49fa-9d0e-c711dbb4cc26 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:05.420863 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2380/4840] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:05 2022] POST /identity/v3/users => generated 283 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:05:05.425820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df1955-23c0-49fa-9d0e-c711dbb4cc26 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:05.448912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee57a01e-1e90-465d-b954-5dc1f12cc504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:05.449853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57a01e-1e90-465d-b954-5dc1f12cc504 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:05.450332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df1955-23c0-49fa-9d0e-c711dbb4cc26 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:05.451546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57a01e-1e90-465d-b954-5dc1f12cc504 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:05.452010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57a01e-1e90-465d-b954-5dc1f12cc504 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:05.452811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee57a01e-1e90-465d-b954-5dc1f12cc504 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:05.456610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee57a01e-1e90-465d-b954-5dc1f12cc504 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:05.463353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abc681-e3fa-49de-8e44-b3f013bdd1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:05.464246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abc681-e3fa-49de-8e44-b3f013bdd1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:05.464787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abc681-e3fa-49de-8e44-b3f013bdd1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:05.465361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abc681-e3fa-49de-8e44-b3f013bdd1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:05.466988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66abc681-e3fa-49de-8e44-b3f013bdd1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:05.470280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66abc681-e3fa-49de-8e44-b3f013bdd1d8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:05.491751 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2381/4841] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:05:05.496671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abc681-e3fa-49de-8e44-b3f013bdd1d8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:05.519855 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2461/4842] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:05 2022] POST /identity/v3/users => generated 287 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:05.521916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee57a01e-1e90-465d-b954-5dc1f12cc504 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:05.538770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abc681-e3fa-49de-8e44-b3f013bdd1d8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:05.551925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee57a01e-1e90-465d-b954-5dc1f12cc504 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:05.566545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad5c63-2d03-4083-9ca6-5c9347336ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:05.567629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad5c63-2d03-4083-9ca6-5c9347336ddc admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:05.568635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad5c63-2d03-4083-9ca6-5c9347336ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:05.568981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad5c63-2d03-4083-9ca6-5c9347336ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:05.569753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad5c63-2d03-4083-9ca6-5c9347336ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:05.571460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad5c63-2d03-4083-9ca6-5c9347336ddc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:05.577724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c70e7-e5f7-4a06-a7e4-faa02ceede5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:05.578635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c70e7-e5f7-4a06-a7e4-faa02ceede5f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:05.578814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c70e7-e5f7-4a06-a7e4-faa02ceede5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:05.579120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c70e7-e5f7-4a06-a7e4-faa02ceede5f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:05.579879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c70e7-e5f7-4a06-a7e4-faa02ceede5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:05.581504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c70e7-e5f7-4a06-a7e4-faa02ceede5f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:05.642139 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2382/4843] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:05.646762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b046a-eb8f-492e-87fa-49f1c8bd33f1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:05.646762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b046a-eb8f-492e-87fa-49f1c8bd33f1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:05.646762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b046a-eb8f-492e-87fa-49f1c8bd33f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:05.647680 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2462/4844] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:05.648574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c70e7-e5f7-4a06-a7e4-faa02ceede5f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:05.683498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c70e7-e5f7-4a06-a7e4-faa02ceede5f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:05.701411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa53a0-27d0-40d7-ab1e-69e7a8ff82f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:05.702800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa53a0-27d0-40d7-ab1e-69e7a8ff82f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:05.703156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa53a0-27d0-40d7-ab1e-69e7a8ff82f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:05.703466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa53a0-27d0-40d7-ab1e-69e7a8ff82f4 admin admin] PATH_INFO: `/v3/projects/d591215350fc4c34939709505679a996/users/4b540aab3baf4bf7b28eb8a7e9c37eb0/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:05.737220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2c1b046a-eb8f-492e-87fa-49f1c8bd33f1 None None] MFA Rules not processed for user `46c6f0cba68e41eab2d90bac5fba9442`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:05.738513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2c1b046a-eb8f-492e-87fa-49f1c8bd33f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:05.754637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa53a0-27d0-40d7-ab1e-69e7a8ff82f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d591215350fc4c34939709505679a996, user_id=4b540aab3baf4bf7b28eb8a7e9c37eb0, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:05.763967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa53a0-27d0-40d7-ab1e-69e7a8ff82f4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:05.800691 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2463/4845] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:05 2022] PUT /identity/v3/projects/d591215350fc4c34939709505679a996/users/4b540aab3baf4bf7b28eb8a7e9c37eb0/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:05.802126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa53a0-27d0-40d7-ab1e-69e7a8ff82f4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:05.901425 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2383/4846] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:05.902610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa53a0-27d0-40d7-ab1e-69e7a8ff82f4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:05.904855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc0a46-ac63-4424-a553-7ccf82659765 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:05.905216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc0a46-ac63-4424-a553-7ccf82659765 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:05.907612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc0a46-ac63-4424-a553-7ccf82659765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:05.909228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546242e9-794e-419c-9345-71f9f0cb15b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:05.910233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546242e9-794e-419c-9345-71f9f0cb15b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:05.910642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546242e9-794e-419c-9345-71f9f0cb15b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:05.911018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546242e9-794e-419c-9345-71f9f0cb15b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:05.911770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546242e9-794e-419c-9345-71f9f0cb15b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:05.913669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546242e9-794e-419c-9345-71f9f0cb15b6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:05.927924 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2464/4847] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:05.930020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347687c-80e8-4c88-8325-715fbceb37db None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:05.930418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347687c-80e8-4c88-8325-715fbceb37db None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:05.931133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347687c-80e8-4c88-8325-715fbceb37db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:05.971650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-14fc0a46-ac63-4424-a553-7ccf82659765 None None] MFA Rules not processed for user `4b540aab3baf4bf7b28eb8a7e9c37eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:05.972733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-14fc0a46-ac63-4424-a553-7ccf82659765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:05.976524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4347687c-80e8-4c88-8325-715fbceb37db None None] MFA Rules not processed for user `46c6f0cba68e41eab2d90bac5fba9442`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:05.977996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4347687c-80e8-4c88-8325-715fbceb37db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:06.035796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:06.036040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:06.136757 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2384/4848] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:06.137680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc0a46-ac63-4424-a553-7ccf82659765 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:06.172727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc0a46-ac63-4424-a553-7ccf82659765 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:06.177784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713273e1-3a31-4583-942b-17fc34e69bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:06.178681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713273e1-3a31-4583-942b-17fc34e69bf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:06.179185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713273e1-3a31-4583-942b-17fc34e69bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:06.179558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713273e1-3a31-4583-942b-17fc34e69bf3 admin admin] PATH_INFO: `/v3/projects/7129c6a83d9b442aa04d8c06e0cd8676/users/7522282fa3b74ecf88dded804af360c2/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:06.183976 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2465/4849] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:06.185843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347687c-80e8-4c88-8325-715fbceb37db None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:06.189845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4347687c-80e8-4c88-8325-715fbceb37db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:06.204854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713273e1-3a31-4583-942b-17fc34e69bf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7129c6a83d9b442aa04d8c06e0cd8676, user_id=7522282fa3b74ecf88dded804af360c2, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:06.209191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713273e1-3a31-4583-942b-17fc34e69bf3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:06.235678 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2385/4850] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:06 2022] PUT /identity/v3/projects/7129c6a83d9b442aa04d8c06e0cd8676/users/7522282fa3b74ecf88dded804af360c2/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:06.256419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6826c7c-3a9a-4f83-bec7-c4c3e7893de0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:06.256731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6826c7c-3a9a-4f83-bec7-c4c3e7893de0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:06.257114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6826c7c-3a9a-4f83-bec7-c4c3e7893de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:06.352663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b6826c7c-3a9a-4f83-bec7-c4c3e7893de0 None None] MFA Rules not processed for user `7522282fa3b74ecf88dded804af360c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:06.356779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b6826c7c-3a9a-4f83-bec7-c4c3e7893de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:06.368974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347687c-80e8-4c88-8325-715fbceb37db None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:06.540179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1426579-d2d9-4228-9b12-4b3714d6ba0e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:06.543237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1426579-d2d9-4228-9b12-4b3714d6ba0e service glance] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:06.543636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1426579-d2d9-4228-9b12-4b3714d6ba0e service glance] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:06.544963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1426579-d2d9-4228-9b12-4b3714d6ba0e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:06.546376 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2386/4851] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:06.550334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9b9db-541d-418d-82b0-b3ea28ea9782 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:06.551900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9b9db-541d-418d-82b0-b3ea28ea9782 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:06.553739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9b9db-541d-418d-82b0-b3ea28ea9782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:06.595739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4de9b9db-541d-418d-82b0-b3ea28ea9782 None None] MFA Rules not processed for user `6bda28994cb14f92b014a147a7e74476`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:06.598253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4de9b9db-541d-418d-82b0-b3ea28ea9782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:06.717025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1426579-d2d9-4228-9b12-4b3714d6ba0e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:06.721382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1426579-d2d9-4228-9b12-4b3714d6ba0e service glance] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:06.751272 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2387/4852] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:06.753369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9b9db-541d-418d-82b0-b3ea28ea9782 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:06.754810 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2466/4853] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:05:06.835082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:06.924563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9b9db-541d-418d-82b0-b3ea28ea9782 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:06.934617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433d039-7cfa-4f6a-a0a9-a2147a55a014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:06.949168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433d039-7cfa-4f6a-a0a9-a2147a55a014 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:06.952231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433d039-7cfa-4f6a-a0a9-a2147a55a014 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:06.952524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433d039-7cfa-4f6a-a0a9-a2147a55a014 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:06.955389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433d039-7cfa-4f6a-a0a9-a2147a55a014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:06.969403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433d039-7cfa-4f6a-a0a9-a2147a55a014 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:07.049972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1426579-d2d9-4228-9b12-4b3714d6ba0e service glance] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:07.063517 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2388/4854] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:06 2022] POST /identity/v3/projects => generated 373 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:07.068703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1426579-d2d9-4228-9b12-4b3714d6ba0e service glance] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:07.156948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:07.207784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6d1a2-58d0-46f3-a9bc-cc5a7b0e032a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:07.208507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6d1a2-58d0-46f3-a9bc-cc5a7b0e032a admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:07.208791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6d1a2-58d0-46f3-a9bc-cc5a7b0e032a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:07.209068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6d1a2-58d0-46f3-a9bc-cc5a7b0e032a admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:07.212668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6d1a2-58d0-46f3-a9bc-cc5a7b0e032a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:07.218024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6d1a2-58d0-46f3-a9bc-cc5a7b0e032a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:07.281698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433d039-7cfa-4f6a-a0a9-a2147a55a014 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:07.291829 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2467/4855] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:07 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:07.297407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433d039-7cfa-4f6a-a0a9-a2147a55a014 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:07.324005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c145e9-ef3f-4161-a0b3-fca51d04e04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:07.324655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c145e9-ef3f-4161-a0b3-fca51d04e04e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:07.324935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c145e9-ef3f-4161-a0b3-fca51d04e04e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:07.325204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c145e9-ef3f-4161-a0b3-fca51d04e04e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:07.331884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c145e9-ef3f-4161-a0b3-fca51d04e04e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:07.336302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c145e9-ef3f-4161-a0b3-fca51d04e04e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:07.364330 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2389/4856] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:05:07.370832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6d1a2-58d0-46f3-a9bc-cc5a7b0e032a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:07.390891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6d1a2-58d0-46f3-a9bc-cc5a7b0e032a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:07.402153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45083633-885d-4d7d-a9ff-5964be8f9325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:07.402997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45083633-885d-4d7d-a9ff-5964be8f9325 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:07.403130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45083633-885d-4d7d-a9ff-5964be8f9325 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:07.403449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45083633-885d-4d7d-a9ff-5964be8f9325 admin admin] PATH_INFO: `/v3/projects/d3e02e12da3343258656e27e11718ab6/users/fc2c08b8e1d64be796cda8b3ff6f0e31/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:07.446609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45083633-885d-4d7d-a9ff-5964be8f9325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3e02e12da3343258656e27e11718ab6, user_id=fc2c08b8e1d64be796cda8b3ff6f0e31, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:07.449280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45083633-885d-4d7d-a9ff-5964be8f9325 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:07.468407 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2468/4857] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:07 2022] PUT /identity/v3/projects/d3e02e12da3343258656e27e11718ab6/users/fc2c08b8e1d64be796cda8b3ff6f0e31/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:07.469139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c145e9-ef3f-4161-a0b3-fca51d04e04e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:07.485642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c145e9-ef3f-4161-a0b3-fca51d04e04e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:07.609798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45083633-885d-4d7d-a9ff-5964be8f9325 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:07.609798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-45083633-885d-4d7d-a9ff-5964be8f9325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:07.614053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079af13b-b6c0-467f-930b-777257e096a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:07.615677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079af13b-b6c0-467f-930b-777257e096a2 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:07.616176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079af13b-b6c0-467f-930b-777257e096a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:07.616691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079af13b-b6c0-467f-930b-777257e096a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:07.617604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079af13b-b6c0-467f-930b-777257e096a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:07.620316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079af13b-b6c0-467f-930b-777257e096a2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:07.634172 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2390/4858] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:07.640606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079af13b-b6c0-467f-930b-777257e096a2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:07.653871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079af13b-b6c0-467f-930b-777257e096a2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:07.659704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca037c-5356-43a2-8e5a-4e0d8cd842fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:07.660955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca037c-5356-43a2-8e5a-4e0d8cd842fd admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:07.661225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca037c-5356-43a2-8e5a-4e0d8cd842fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:07.661500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca037c-5356-43a2-8e5a-4e0d8cd842fd admin admin] PATH_INFO: `/v3/projects/d3e02e12da3343258656e27e11718ab6/users/fc2c08b8e1d64be796cda8b3ff6f0e31/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:07.663588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca037c-5356-43a2-8e5a-4e0d8cd842fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3e02e12da3343258656e27e11718ab6, user_id=fc2c08b8e1d64be796cda8b3ff6f0e31, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:07.667156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca037c-5356-43a2-8e5a-4e0d8cd842fd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:07.681575 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2391/4859] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:07 2022] PUT /identity/v3/projects/d3e02e12da3343258656e27e11718ab6/users/fc2c08b8e1d64be796cda8b3ff6f0e31/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:07.691766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c738d-322f-4a9c-b183-857337750566 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:07.692121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c738d-322f-4a9c-b183-857337750566 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:07.692606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c738d-322f-4a9c-b183-857337750566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:07.743652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0c3c738d-322f-4a9c-b183-857337750566 None None] MFA Rules not processed for user `fc2c08b8e1d64be796cda8b3ff6f0e31`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:07.744939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0c3c738d-322f-4a9c-b183-857337750566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:07.889033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45083633-885d-4d7d-a9ff-5964be8f9325 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:07.911991 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2392/4860] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:07.913774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c738d-322f-4a9c-b183-857337750566 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:07.925191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c738d-322f-4a9c-b183-857337750566 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:07.935316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c3bec-b19e-4030-b437-73be8e060844 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2c08b8e1d64be796cda8b3ff6f0e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3e02e12da3343258656e27e11718ab6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:07.935943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c3bec-b19e-4030-b437-73be8e060844 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:07.936229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c3bec-b19e-4030-b437-73be8e060844 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:07.936501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c3bec-b19e-4030-b437-73be8e060844 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:07.937942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c3bec-b19e-4030-b437-73be8e060844 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:07.939406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c3bec-b19e-4030-b437-73be8e060844 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:07.970225 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2393/4861] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:07 2022] POST /identity/v3/services => generated 288 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:05:07.974094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c3bec-b19e-4030-b437-73be8e060844 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:07.989053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:08.013793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c3bec-b19e-4030-b437-73be8e060844 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:08.021905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b38c1f-1c97-42fd-8193-f5cd001f560c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2c08b8e1d64be796cda8b3ff6f0e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3e02e12da3343258656e27e11718ab6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:08.023205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b38c1f-1c97-42fd-8193-f5cd001f560c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:08.023620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b38c1f-1c97-42fd-8193-f5cd001f560c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:08.024227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b38c1f-1c97-42fd-8193-f5cd001f560c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:08.025056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b38c1f-1c97-42fd-8193-f5cd001f560c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:08.033895 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-d7b38c1f-1c97-42fd-8193-f5cd001f560c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:05:08.033895 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:05:08.033895 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:05:08.035171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b38c1f-1c97-42fd-8193-f5cd001f560c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:08.071486 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2394/4862] 10.222.0.17 () {60 vars in 1213 bytes} [Sat Oct 1 21:05:07 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 333 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:05:08.072904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b38c1f-1c97-42fd-8193-f5cd001f560c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:08.073596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48d8b8-f9bc-4a92-91c4-abe0a4e9f8aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:08.074867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48d8b8-f9bc-4a92-91c4-abe0a4e9f8aa service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:08.076599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48d8b8-f9bc-4a92-91c4-abe0a4e9f8aa service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:08.077199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48d8b8-f9bc-4a92-91c4-abe0a4e9f8aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:08.096199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b38c1f-1c97-42fd-8193-f5cd001f560c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:08.104753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54a989-2fb6-42ce-b5e8-23929cc2887c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2c08b8e1d64be796cda8b3ff6f0e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3e02e12da3343258656e27e11718ab6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:08.105485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54a989-2fb6-42ce-b5e8-23929cc2887c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:08.106080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54a989-2fb6-42ce-b5e8-23929cc2887c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:08.107160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54a989-2fb6-42ce-b5e8-23929cc2887c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] PATH_INFO: `/v3/services` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:08.110085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca54a989-2fb6-42ce-b5e8-23929cc2887c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:08.118729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca54a989-2fb6-42ce-b5e8-23929cc2887c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:08.175253 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2395/4863] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:08 2022] POST /identity/v3/services => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:08.176746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54a989-2fb6-42ce-b5e8-23929cc2887c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:08.182076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:08.221040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54a989-2fb6-42ce-b5e8-23929cc2887c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:08.244261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba267093-5010-4601-8441-39bd1cc83aa3 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2c08b8e1d64be796cda8b3ff6f0e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3e02e12da3343258656e27e11718ab6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:08.246570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba267093-5010-4601-8441-39bd1cc83aa3 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:08.246866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba267093-5010-4601-8441-39bd1cc83aa3 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:08.247139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba267093-5010-4601-8441-39bd1cc83aa3 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:08.250872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba267093-5010-4601-8441-39bd1cc83aa3 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:08.262410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48d8b8-f9bc-4a92-91c4-abe0a4e9f8aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:08.264253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba267093-5010-4601-8441-39bd1cc83aa3 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:08.273951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48d8b8-f9bc-4a92-91c4-abe0a4e9f8aa service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:08.287166 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2396/4864] 10.222.0.17 () {60 vars in 1213 bytes} [Sat Oct 1 21:05:08 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 334 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:08.291433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba267093-5010-4601-8441-39bd1cc83aa3 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:08.318261 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2469/4865] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:07 2022] GET /identity/v3/auth/tokens => generated 3515 bytes in 722 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:05:08.320729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba267093-5010-4601-8441-39bd1cc83aa3 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:08.334031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b73185-8621-4168-a89d-2568afccf5de tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2c08b8e1d64be796cda8b3ff6f0e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3e02e12da3343258656e27e11718ab6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:08.335982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b73185-8621-4168-a89d-2568afccf5de tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:08.338395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b73185-8621-4168-a89d-2568afccf5de tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:08.339437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b73185-8621-4168-a89d-2568afccf5de tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:08.340296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b73185-8621-4168-a89d-2568afccf5de tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorizing `identity:list_endpoint_groups()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:08.345152 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-21b73185-8621-4168-a89d-2568afccf5de tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoint_groups": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:05:08.345152 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:05:08.345152 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:05:08.346106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b73185-8621-4168-a89d-2568afccf5de tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:08.377452 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2397/4866] 10.222.0.17 () {58 vars in 1191 bytes} [Sat Oct 1 21:05:08 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups => generated 765 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:08.379826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48d8b8-f9bc-4a92-91c4-abe0a4e9f8aa service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:08.408451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48d8b8-f9bc-4a92-91c4-abe0a4e9f8aa service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:08.427823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0458e-39a8-444a-91f6-5528d0967cc6 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2c08b8e1d64be796cda8b3ff6f0e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3e02e12da3343258656e27e11718ab6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:08.428499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0458e-39a8-444a-91f6-5528d0967cc6 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:08.428944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0458e-39a8-444a-91f6-5528d0967cc6 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:08.429090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0458e-39a8-444a-91f6-5528d0967cc6 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/b97da99729084f71b6684d3d64dd4303` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:08.430439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0458e-39a8-444a-91f6-5528d0967cc6 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorizing `identity:get_endpoint_group(endpoint_group_id=b97da99729084f71b6684d3d64dd4303)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:08.430948 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-4dd0458e-39a8-444a-91f6-5528d0967cc6 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:05:08.430948 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:05:08.430948 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:05:08.431856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0458e-39a8-444a-91f6-5528d0967cc6 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:08.447461 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2470/4867] 10.222.0.17 () {58 vars in 1290 bytes} [Sat Oct 1 21:05:08 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups/b97da99729084f71b6684d3d64dd4303 => generated 334 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:08.448334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b73185-8621-4168-a89d-2568afccf5de tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:08.466779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b73185-8621-4168-a89d-2568afccf5de tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:08.475440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d11c3-38cb-4b19-a201-31927c5345d2 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2c08b8e1d64be796cda8b3ff6f0e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3e02e12da3343258656e27e11718ab6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:08.476613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d11c3-38cb-4b19-a201-31927c5345d2 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:08.479311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d11c3-38cb-4b19-a201-31927c5345d2 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:08.480328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d11c3-38cb-4b19-a201-31927c5345d2 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/b97da99729084f71b6684d3d64dd4303` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:08.481184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d11c3-38cb-4b19-a201-31927c5345d2 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorizing `identity:get_endpoint_group(endpoint_group_id=b97da99729084f71b6684d3d64dd4303)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:08.482872 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-2d1d11c3-38cb-4b19-a201-31927c5345d2 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:05:08.482872 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:05:08.482872 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:05:08.486148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d11c3-38cb-4b19-a201-31927c5345d2 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:08.508874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0458e-39a8-444a-91f6-5528d0967cc6 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:08.509719 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2398/4868] 10.222.0.17 () {58 vars in 1291 bytes} [Sat Oct 1 21:05:08 2022] HEAD /identity/v3/OS-EP-FILTER/endpoint_groups/b97da99729084f71b6684d3d64dd4303 => generated 0 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Oct 01 21:05:08.525190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0458e-39a8-444a-91f6-5528d0967cc6 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:08.533213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581e9b9-70e5-4b4a-b21e-28c15264212f tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2c08b8e1d64be796cda8b3ff6f0e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3e02e12da3343258656e27e11718ab6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:08.534086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581e9b9-70e5-4b4a-b21e-28c15264212f tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:08.534570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581e9b9-70e5-4b4a-b21e-28c15264212f tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:08.535068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581e9b9-70e5-4b4a-b21e-28c15264212f tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/b97da99729084f71b6684d3d64dd4303` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:08.536058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581e9b9-70e5-4b4a-b21e-28c15264212f tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=b97da99729084f71b6684d3d64dd4303)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:08.537271 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-f581e9b9-70e5-4b4a-b21e-28c15264212f tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:05:08.537271 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:05:08.537271 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:05:08.538340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581e9b9-70e5-4b4a-b21e-28c15264212f tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:08.569851 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2471/4869] 10.222.0.17 () {58 vars in 1293 bytes} [Sat Oct 1 21:05:08 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/b97da99729084f71b6684d3d64dd4303 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:08.575833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d11c3-38cb-4b19-a201-31927c5345d2 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:08.591537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d11c3-38cb-4b19-a201-31927c5345d2 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:08.600037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898146ed-9615-4912-a44d-6df32cdcc1b7 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2c08b8e1d64be796cda8b3ff6f0e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3e02e12da3343258656e27e11718ab6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:08.600678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898146ed-9615-4912-a44d-6df32cdcc1b7 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:08.600959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898146ed-9615-4912-a44d-6df32cdcc1b7 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:08.601231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898146ed-9615-4912-a44d-6df32cdcc1b7 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:08.601854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898146ed-9615-4912-a44d-6df32cdcc1b7 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorizing `identity:list_endpoint_groups()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:08.603662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898146ed-9615-4912-a44d-6df32cdcc1b7 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:08.616425 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2399/4870] 10.222.0.17 () {58 vars in 1191 bytes} [Sat Oct 1 21:05:08 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups => generated 450 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:08.620010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581e9b9-70e5-4b4a-b21e-28c15264212f tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:08.634394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581e9b9-70e5-4b4a-b21e-28c15264212f tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:08.641964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ce03a-57f5-4162-82cb-871cf91fbbb2 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2c08b8e1d64be796cda8b3ff6f0e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3e02e12da3343258656e27e11718ab6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:08.642665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ce03a-57f5-4162-82cb-871cf91fbbb2 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:08.642954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ce03a-57f5-4162-82cb-871cf91fbbb2 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:08.643342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ce03a-57f5-4162-82cb-871cf91fbbb2 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/b97da99729084f71b6684d3d64dd4303` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:08.643940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ce03a-57f5-4162-82cb-871cf91fbbb2 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=b97da99729084f71b6684d3d64dd4303)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:08.645662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ce03a-57f5-4162-82cb-871cf91fbbb2 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:08.673439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-705ce03a-57f5-4162-82cb-871cf91fbbb2 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Could not find Endpoint Group: b97da99729084f71b6684d3d64dd4303.: keystone.exception.EndpointGroupNotFound: Could not find Endpoint Group: b97da99729084f71b6684d3d64dd4303. Oct 01 21:05:08.673439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:05:08.673439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:05:08.673439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:05:08.673439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:05:08.673439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:05:08.673439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:05:08.673439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:05:08.673439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:05:08.673439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:05:08.673439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:05:08.673439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:05:08.673439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_ep_filter.py", line 107, in delete Oct 01 21:05:08.673439 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return (PROVIDERS.catalog_api.delete_endpoint_group(endpoint_group_id), Oct 01 21:05:08.674636 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 519, in delete_endpoint_group Oct 01 21:05:08.674636 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application endpoint_group_ref = self._get_endpoint_group(session, Oct 01 21:05:08.674636 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 495, in _get_endpoint_group Oct 01 21:05:08.674636 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.EndpointGroupNotFound( Oct 01 21:05:08.674636 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.EndpointGroupNotFound: Could not find Endpoint Group: b97da99729084f71b6684d3d64dd4303. Oct 01 21:05:08.674636 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:05:08.679473 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2472/4871] 10.222.0.17 () {58 vars in 1293 bytes} [Sat Oct 1 21:05:08 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/b97da99729084f71b6684d3d64dd4303 => generated 120 bytes in 59 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:05:08.682405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898146ed-9615-4912-a44d-6df32cdcc1b7 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:08.695705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898146ed-9615-4912-a44d-6df32cdcc1b7 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:08.702526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e2ac1-db49-4e7e-b2f7-a9293e52b45a tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2c08b8e1d64be796cda8b3ff6f0e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3e02e12da3343258656e27e11718ab6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:08.703632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e2ac1-db49-4e7e-b2f7-a9293e52b45a tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:08.704096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e2ac1-db49-4e7e-b2f7-a9293e52b45a tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:08.704583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e2ac1-db49-4e7e-b2f7-a9293e52b45a tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] PATH_INFO: `/v3/services/1eb4920f72ae4eada633f6395c62712d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:08.705316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e2ac1-db49-4e7e-b2f7-a9293e52b45a tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorizing `identity:delete_service(service_id=1eb4920f72ae4eada633f6395c62712d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:08.707311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e2ac1-db49-4e7e-b2f7-a9293e52b45a tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:08.756638 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2400/4872] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:08 2022] DELETE /identity/v3/services/1eb4920f72ae4eada633f6395c62712d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:08.761490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ce03a-57f5-4162-82cb-871cf91fbbb2 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:08.775104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ce03a-57f5-4162-82cb-871cf91fbbb2 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:08.819886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00bc09-f836-451c-a24d-0328ff45a05f tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2c08b8e1d64be796cda8b3ff6f0e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3e02e12da3343258656e27e11718ab6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:08.820522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00bc09-f836-451c-a24d-0328ff45a05f tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:08.821331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00bc09-f836-451c-a24d-0328ff45a05f tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:08.821331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00bc09-f836-451c-a24d-0328ff45a05f tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:08.821783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00bc09-f836-451c-a24d-0328ff45a05f tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:08.823877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00bc09-f836-451c-a24d-0328ff45a05f tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:08.857835 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2473/4873] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:08 2022] POST /identity/v3/services => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:08.862804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e2ac1-db49-4e7e-b2f7-a9293e52b45a tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:08.879786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e2ac1-db49-4e7e-b2f7-a9293e52b45a tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:08.918273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ac568-a797-4e38-9815-dc48db3c6e04 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2c08b8e1d64be796cda8b3ff6f0e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3e02e12da3343258656e27e11718ab6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:08.918947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ac568-a797-4e38-9815-dc48db3c6e04 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:08.919233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ac568-a797-4e38-9815-dc48db3c6e04 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:08.919511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ac568-a797-4e38-9815-dc48db3c6e04 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:08.920177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ac568-a797-4e38-9815-dc48db3c6e04 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:08.921896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ac568-a797-4e38-9815-dc48db3c6e04 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:08.947547 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2401/4874] 10.222.0.17 () {60 vars in 1213 bytes} [Sat Oct 1 21:05:08 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 332 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:08.951642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00bc09-f836-451c-a24d-0328ff45a05f tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:08.967497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00bc09-f836-451c-a24d-0328ff45a05f tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:08.974627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f1ff3-cc7f-45d6-b32d-4da9100806e4 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2c08b8e1d64be796cda8b3ff6f0e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3e02e12da3343258656e27e11718ab6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:08.975253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f1ff3-cc7f-45d6-b32d-4da9100806e4 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:08.975538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f1ff3-cc7f-45d6-b32d-4da9100806e4 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:08.975843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f1ff3-cc7f-45d6-b32d-4da9100806e4 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:08.976450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f1ff3-cc7f-45d6-b32d-4da9100806e4 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:08.978784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f1ff3-cc7f-45d6-b32d-4da9100806e4 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:09.010467 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2474/4875] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:08 2022] POST /identity/v3/services => generated 287 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:09.011844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ac568-a797-4e38-9815-dc48db3c6e04 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:09.033540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ac568-a797-4e38-9815-dc48db3c6e04 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:09.065357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed3546-4671-4383-a135-7532aebe5fb3 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2c08b8e1d64be796cda8b3ff6f0e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3e02e12da3343258656e27e11718ab6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:09.066832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed3546-4671-4383-a135-7532aebe5fb3 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:09.067119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed3546-4671-4383-a135-7532aebe5fb3 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:09.067401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed3546-4671-4383-a135-7532aebe5fb3 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/e7aa4ff80e134be18554fde57a6e996d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:09.068420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed3546-4671-4383-a135-7532aebe5fb3 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorizing `identity:update_endpoint_group(endpoint_group_id=e7aa4ff80e134be18554fde57a6e996d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:09.069268 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-84ed3546-4671-4383-a135-7532aebe5fb3 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:05:09.069268 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:05:09.069268 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:05:09.071688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed3546-4671-4383-a135-7532aebe5fb3 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:09.104806 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2402/4876] 10.222.0.17 () {60 vars in 1313 bytes} [Sat Oct 1 21:05:09 2022] PATCH /identity/v3/OS-EP-FILTER/endpoint_groups/e7aa4ff80e134be18554fde57a6e996d => generated 335 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:09.111352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f1ff3-cc7f-45d6-b32d-4da9100806e4 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:09.128007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f1ff3-cc7f-45d6-b32d-4da9100806e4 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:09.135558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2e9bc-b760-45e4-a824-ed13431abf9c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2c08b8e1d64be796cda8b3ff6f0e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3e02e12da3343258656e27e11718ab6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:09.136299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2e9bc-b760-45e4-a824-ed13431abf9c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:09.136588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2e9bc-b760-45e4-a824-ed13431abf9c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:09.136860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2e9bc-b760-45e4-a824-ed13431abf9c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] PATH_INFO: `/v3/services/10262a04c6434bebb303b05dcfe14eb1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:09.137450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2e9bc-b760-45e4-a824-ed13431abf9c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorizing `identity:delete_service(service_id=10262a04c6434bebb303b05dcfe14eb1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:09.139442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2e9bc-b760-45e4-a824-ed13431abf9c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:09.183482 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2475/4877] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:09 2022] DELETE /identity/v3/services/10262a04c6434bebb303b05dcfe14eb1 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:09.187983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed3546-4671-4383-a135-7532aebe5fb3 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:09.208436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed3546-4671-4383-a135-7532aebe5fb3 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:09.262179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5a875-8821-42fd-8d73-7b9caa226574 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2c08b8e1d64be796cda8b3ff6f0e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3e02e12da3343258656e27e11718ab6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:09.262880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5a875-8821-42fd-8d73-7b9caa226574 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:09.263158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5a875-8821-42fd-8d73-7b9caa226574 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:09.263430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5a875-8821-42fd-8d73-7b9caa226574 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/e7aa4ff80e134be18554fde57a6e996d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:09.264017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5a875-8821-42fd-8d73-7b9caa226574 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=e7aa4ff80e134be18554fde57a6e996d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:09.267046 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-aed5a875-8821-42fd-8d73-7b9caa226574 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:05:09.267046 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:05:09.267046 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:05:09.268040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5a875-8821-42fd-8d73-7b9caa226574 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:09.289884 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2403/4878] 10.222.0.17 () {58 vars in 1293 bytes} [Sat Oct 1 21:05:09 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/e7aa4ff80e134be18554fde57a6e996d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:09.302825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2e9bc-b760-45e4-a824-ed13431abf9c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:09.329784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2e9bc-b760-45e4-a824-ed13431abf9c tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:09.339930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdba427-b0e7-41eb-b67e-fe948fb3f834 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2c08b8e1d64be796cda8b3ff6f0e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3e02e12da3343258656e27e11718ab6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:09.340826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdba427-b0e7-41eb-b67e-fe948fb3f834 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:09.341271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdba427-b0e7-41eb-b67e-fe948fb3f834 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:09.341892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdba427-b0e7-41eb-b67e-fe948fb3f834 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] PATH_INFO: `/v3/services/173dc7d384a6427b99801bdec0b01dca` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:09.343826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdba427-b0e7-41eb-b67e-fe948fb3f834 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorizing `identity:delete_service(service_id=173dc7d384a6427b99801bdec0b01dca)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:09.349007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdba427-b0e7-41eb-b67e-fe948fb3f834 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:09.396696 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2476/4879] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:09 2022] DELETE /identity/v3/services/173dc7d384a6427b99801bdec0b01dca => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:09.398354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5a875-8821-42fd-8d73-7b9caa226574 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:09.463779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5a875-8821-42fd-8d73-7b9caa226574 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:09.534659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b90bc-2a61-4e57-9ee6-b3dae4b089c9 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2c08b8e1d64be796cda8b3ff6f0e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3e02e12da3343258656e27e11718ab6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:09.535382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b90bc-2a61-4e57-9ee6-b3dae4b089c9 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:09.536333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b90bc-2a61-4e57-9ee6-b3dae4b089c9 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:09.536615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b90bc-2a61-4e57-9ee6-b3dae4b089c9 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/71ee6e91832e413d8c54059c1cd09cda` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:09.537248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b90bc-2a61-4e57-9ee6-b3dae4b089c9 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=71ee6e91832e413d8c54059c1cd09cda)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:09.538913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b90bc-2a61-4e57-9ee6-b3dae4b089c9 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:09.570327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdba427-b0e7-41eb-b67e-fe948fb3f834 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:09.571773 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2404/4880] 10.222.0.17 () {58 vars in 1293 bytes} [Sat Oct 1 21:05:09 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/71ee6e91832e413d8c54059c1cd09cda => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:09.591026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdba427-b0e7-41eb-b67e-fe948fb3f834 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:09.602084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f59fe-c3ff-4941-8965-dbc74c8f6b78 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2c08b8e1d64be796cda8b3ff6f0e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3e02e12da3343258656e27e11718ab6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:09.604475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f59fe-c3ff-4941-8965-dbc74c8f6b78 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:09.604803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f59fe-c3ff-4941-8965-dbc74c8f6b78 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:09.605731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f59fe-c3ff-4941-8965-dbc74c8f6b78 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] PATH_INFO: `/v3/services/dd01562c93404a758ad5808e3663ca8c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:09.606539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f59fe-c3ff-4941-8965-dbc74c8f6b78 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorizing `identity:delete_service(service_id=dd01562c93404a758ad5808e3663ca8c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:09.608360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f59fe-c3ff-4941-8965-dbc74c8f6b78 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:09.669062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b90bc-2a61-4e57-9ee6-b3dae4b089c9 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:09.669518 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2477/4881] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:09 2022] DELETE /identity/v3/services/dd01562c93404a758ad5808e3663ca8c => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:09.745186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=6,keystone:UPDATE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:09.817513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b90bc-2a61-4e57-9ee6-b3dae4b089c9 tempest-EndPointGroupsTest-1923887517 tempest-EndPointGroupsTest-1923887517-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:09.847044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060b8d2-cd25-4c19-92ff-4e6c700e9de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:09.848191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060b8d2-cd25-4c19-92ff-4e6c700e9de9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:09.848738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060b8d2-cd25-4c19-92ff-4e6c700e9de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:09.849260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060b8d2-cd25-4c19-92ff-4e6c700e9de9 admin admin] PATH_INFO: `/v3/users/7522282fa3b74ecf88dded804af360c2` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:09.850956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060b8d2-cd25-4c19-92ff-4e6c700e9de9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7522282fa3b74ecf88dded804af360c2)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:09.853518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060b8d2-cd25-4c19-92ff-4e6c700e9de9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:09.958908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7060b8d2-cd25-4c19-92ff-4e6c700e9de9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7522282fa3b74ecf88dded804af360c2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:09.962729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7060b8d2-cd25-4c19-92ff-4e6c700e9de9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7522282fa3b74ecf88dded804af360c2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:09.964226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7060b8d2-cd25-4c19-92ff-4e6c700e9de9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7522282fa3b74ecf88dded804af360c2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:09.986999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7060b8d2-cd25-4c19-92ff-4e6c700e9de9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7522282fa3b74ecf88dded804af360c2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:10.008709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7060b8d2-cd25-4c19-92ff-4e6c700e9de9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7522282fa3b74ecf88dded804af360c2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:10.062241 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2405/4882] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:09 2022] DELETE /identity/v3/users/7522282fa3b74ecf88dded804af360c2 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:10.063956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754fe2f-ba38-4cf4-9201-99ef91d673b9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:10.064245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754fe2f-ba38-4cf4-9201-99ef91d673b9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:10.064514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754fe2f-ba38-4cf4-9201-99ef91d673b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:10.108605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9754fe2f-ba38-4cf4-9201-99ef91d673b9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:10.111540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9754fe2f-ba38-4cf4-9201-99ef91d673b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:10.262142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:10.318409 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2478/4883] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:10.329829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060b8d2-cd25-4c19-92ff-4e6c700e9de9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:10.333325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7060b8d2-cd25-4c19-92ff-4e6c700e9de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:10.537355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060b8d2-cd25-4c19-92ff-4e6c700e9de9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:10.543020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c93a82-65a8-4bd6-a36b-6db5c0548f94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:10.546111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c93a82-65a8-4bd6-a36b-6db5c0548f94 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:10.546778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c93a82-65a8-4bd6-a36b-6db5c0548f94 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:10.548591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c93a82-65a8-4bd6-a36b-6db5c0548f94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:10.589225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c93a82-65a8-4bd6-a36b-6db5c0548f94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:10.590815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c93a82-65a8-4bd6-a36b-6db5c0548f94 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:10.619951 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2406/4884] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:05:12.393679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754fe2f-ba38-4cf4-9201-99ef91d673b9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:12.395911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9754fe2f-ba38-4cf4-9201-99ef91d673b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:12.422057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754fe2f-ba38-4cf4-9201-99ef91d673b9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:12.441632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd4fb8-993c-47ff-a412-f6612bc1ceb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:12.442472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd4fb8-993c-47ff-a412-f6612bc1ceb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:12.442784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd4fb8-993c-47ff-a412-f6612bc1ceb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:12.443057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd4fb8-993c-47ff-a412-f6612bc1ceb0 admin admin] PATH_INFO: `/v3/projects/7129c6a83d9b442aa04d8c06e0cd8676` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:12.445690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd4fb8-993c-47ff-a412-f6612bc1ceb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7129c6a83d9b442aa04d8c06e0cd8676)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:12.454518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd4fb8-993c-47ff-a412-f6612bc1ceb0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:12.584279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5bbd4fb8-993c-47ff-a412-f6612bc1ceb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7129c6a83d9b442aa04d8c06e0cd8676', 'request_id': 'req-5bbd4fb8-993c-47ff-a412-f6612bc1ceb0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:12.600658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5bbd4fb8-993c-47ff-a412-f6612bc1ceb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7129c6a83d9b442aa04d8c06e0cd8676', 'request_id': 'req-5bbd4fb8-993c-47ff-a412-f6612bc1ceb0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:12.619270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5bbd4fb8-993c-47ff-a412-f6612bc1ceb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7129c6a83d9b442aa04d8c06e0cd8676', 'request_id': 'req-5bbd4fb8-993c-47ff-a412-f6612bc1ceb0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:12.646122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5bbd4fb8-993c-47ff-a412-f6612bc1ceb0 admin admin] The token cache is being invalidate because project 7129c6a83d9b442aa04d8c06e0cd8676 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:12.646855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5bbd4fb8-993c-47ff-a412-f6612bc1ceb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:12.650796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5bbd4fb8-993c-47ff-a412-f6612bc1ceb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:12.653094 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2479/4885] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:12 2022] DELETE /identity/v3/projects/7129c6a83d9b442aa04d8c06e0cd8676 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:12.657838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c93a82-65a8-4bd6-a36b-6db5c0548f94 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:12.670298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c93a82-65a8-4bd6-a36b-6db5c0548f94 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:12.808676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:12.869006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a87769-6bb1-4bc9-b601-cfb18f0312e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:12.872754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a87769-6bb1-4bc9-b601-cfb18f0312e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:12.873336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a87769-6bb1-4bc9-b601-cfb18f0312e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:12.873839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a87769-6bb1-4bc9-b601-cfb18f0312e5 admin admin] PATH_INFO: `/v3/users/fc2c08b8e1d64be796cda8b3ff6f0e31` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:12.880684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a87769-6bb1-4bc9-b601-cfb18f0312e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc2c08b8e1d64be796cda8b3ff6f0e31)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:12.883447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a87769-6bb1-4bc9-b601-cfb18f0312e5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:12.967580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-27a87769-6bb1-4bc9-b601-cfb18f0312e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc2c08b8e1d64be796cda8b3ff6f0e31'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:12.969259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-27a87769-6bb1-4bc9-b601-cfb18f0312e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc2c08b8e1d64be796cda8b3ff6f0e31'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:12.969964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-27a87769-6bb1-4bc9-b601-cfb18f0312e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc2c08b8e1d64be796cda8b3ff6f0e31'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:12.990602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-27a87769-6bb1-4bc9-b601-cfb18f0312e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc2c08b8e1d64be796cda8b3ff6f0e31'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:13.011758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-27a87769-6bb1-4bc9-b601-cfb18f0312e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc2c08b8e1d64be796cda8b3ff6f0e31'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:13.034203 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2407/4886] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:12 2022] DELETE /identity/v3/users/fc2c08b8e1d64be796cda8b3ff6f0e31 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:15.835042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd4fb8-993c-47ff-a412-f6612bc1ceb0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:15.836522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5bbd4fb8-993c-47ff-a412-f6612bc1ceb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:15.856567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd4fb8-993c-47ff-a412-f6612bc1ceb0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:15.949830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b642c-cd53-4705-9b67-17b5410d7ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:15.950959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b642c-cd53-4705-9b67-17b5410d7ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:15.951441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b642c-cd53-4705-9b67-17b5410d7ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:15.951893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b642c-cd53-4705-9b67-17b5410d7ef8 admin admin] PATH_INFO: `/v3/projects/d3e02e12da3343258656e27e11718ab6` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:15.954022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b642c-cd53-4705-9b67-17b5410d7ef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3e02e12da3343258656e27e11718ab6)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:15.957357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b642c-cd53-4705-9b67-17b5410d7ef8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:16.044799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-545b642c-cd53-4705-9b67-17b5410d7ef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3e02e12da3343258656e27e11718ab6', 'request_id': 'req-545b642c-cd53-4705-9b67-17b5410d7ef8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:16.067473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-545b642c-cd53-4705-9b67-17b5410d7ef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3e02e12da3343258656e27e11718ab6', 'request_id': 'req-545b642c-cd53-4705-9b67-17b5410d7ef8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:16.088460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-545b642c-cd53-4705-9b67-17b5410d7ef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3e02e12da3343258656e27e11718ab6', 'request_id': 'req-545b642c-cd53-4705-9b67-17b5410d7ef8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:16.110299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:16.117933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-545b642c-cd53-4705-9b67-17b5410d7ef8 admin admin] The token cache is being invalidate because project d3e02e12da3343258656e27e11718ab6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:16.119593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-545b642c-cd53-4705-9b67-17b5410d7ef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:16.122333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-545b642c-cd53-4705-9b67-17b5410d7ef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:16.128038 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2480/4887] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:15 2022] DELETE /identity/v3/projects/d3e02e12da3343258656e27e11718ab6 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:16.136218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579412ef-9cac-4dc3-bd6b-4edd09973a72 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:16.136759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579412ef-9cac-4dc3-bd6b-4edd09973a72 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:16.137078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579412ef-9cac-4dc3-bd6b-4edd09973a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:16.166133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b642c-cd53-4705-9b67-17b5410d7ef8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:16.172622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-545b642c-cd53-4705-9b67-17b5410d7ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:16.198529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-579412ef-9cac-4dc3-bd6b-4edd09973a72 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:16.200009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-579412ef-9cac-4dc3-bd6b-4edd09973a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:16.345479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b642c-cd53-4705-9b67-17b5410d7ef8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:16.353860 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2408/4888] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:16.355590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c348ba-e1ba-43da-b031-69f56b70edd7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:16.362060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c348ba-e1ba-43da-b031-69f56b70edd7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:16.362620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c348ba-e1ba-43da-b031-69f56b70edd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:16.370146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf90c7-211e-440f-9695-38710259fb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:16.370819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf90c7-211e-440f-9695-38710259fb9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:16.371975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf90c7-211e-440f-9695-38710259fb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:16.372870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf90c7-211e-440f-9695-38710259fb9a admin admin] PATH_INFO: `/v3/users/e7456555a04d4305a983ed5768a5cf7f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:16.374871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf90c7-211e-440f-9695-38710259fb9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7456555a04d4305a983ed5768a5cf7f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:16.380532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf90c7-211e-440f-9695-38710259fb9a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:16.415260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-83c348ba-e1ba-43da-b031-69f56b70edd7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:16.419114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-83c348ba-e1ba-43da-b031-69f56b70edd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:16.431292 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2409/4889] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:16.433990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c348ba-e1ba-43da-b031-69f56b70edd7 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:16.455435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c348ba-e1ba-43da-b031-69f56b70edd7 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:16.496195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce364ae9-5ef2-4eca-ad46-69100ba87117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:16.500029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce364ae9-5ef2-4eca-ad46-69100ba87117 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:16.500339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce364ae9-5ef2-4eca-ad46-69100ba87117 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:16.502988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce364ae9-5ef2-4eca-ad46-69100ba87117 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:16.509331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce364ae9-5ef2-4eca-ad46-69100ba87117 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:05:16.511997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce364ae9-5ef2-4eca-ad46-69100ba87117 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:16.513803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce364ae9-5ef2-4eca-ad46-69100ba87117 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:16.514204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-41cf90c7-211e-440f-9695-38710259fb9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7456555a04d4305a983ed5768a5cf7f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:16.524692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-41cf90c7-211e-440f-9695-38710259fb9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7456555a04d4305a983ed5768a5cf7f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:16.543939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-41cf90c7-211e-440f-9695-38710259fb9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7456555a04d4305a983ed5768a5cf7f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:16.545190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-41cf90c7-211e-440f-9695-38710259fb9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7456555a04d4305a983ed5768a5cf7f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:16.551847 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2410/4890] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:05:16.561003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce364ae9-5ef2-4eca-ad46-69100ba87117 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:16.561814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-41cf90c7-211e-440f-9695-38710259fb9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7456555a04d4305a983ed5768a5cf7f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:16.562873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ce364ae9-5ef2-4eca-ad46-69100ba87117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:16.572378 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2481/4891] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:16 2022] DELETE /identity/v3/users/e7456555a04d4305a983ed5768a5cf7f => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:16.580213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5654b7-f1a5-4752-a834-b3d8e18d76ce None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:16.580506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5654b7-f1a5-4752-a834-b3d8e18d76ce None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:16.580788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5654b7-f1a5-4752-a834-b3d8e18d76ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:16.590329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce364ae9-5ef2-4eca-ad46-69100ba87117 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:16.616522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-bf5654b7-f1a5-4752-a834-b3d8e18d76ce None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:16.620101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bf5654b7-f1a5-4752-a834-b3d8e18d76ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:16.677119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:16.747574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d76866-9eb5-4d17-8531-6e4b9aff89e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:16.748220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d76866-9eb5-4d17-8531-6e4b9aff89e4 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:16.748504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d76866-9eb5-4d17-8531-6e4b9aff89e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:16.748855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d76866-9eb5-4d17-8531-6e4b9aff89e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:16.750158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d76866-9eb5-4d17-8531-6e4b9aff89e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:16.753850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d76866-9eb5-4d17-8531-6e4b9aff89e4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:16.806861 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2482/4892] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:16.835970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5654b7-f1a5-4752-a834-b3d8e18d76ce None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:16.836680 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2411/4893] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:16 2022] POST /identity/v3/projects => generated 367 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:16.839388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d76866-9eb5-4d17-8531-6e4b9aff89e4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:16.845492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b4d76866-9eb5-4d17-8531-6e4b9aff89e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:16.850798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5654b7-f1a5-4752-a834-b3d8e18d76ce None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:16.896889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:17.009426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d76866-9eb5-4d17-8531-6e4b9aff89e4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:17.016253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de542c-8bda-4f76-9a67-2902c9a8d2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:17.016897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de542c-8bda-4f76-9a67-2902c9a8d2e9 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:17.017183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de542c-8bda-4f76-9a67-2902c9a8d2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:17.017461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de542c-8bda-4f76-9a67-2902c9a8d2e9 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:17.018102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de542c-8bda-4f76-9a67-2902c9a8d2e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:17.020987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de542c-8bda-4f76-9a67-2902c9a8d2e9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:17.022773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a8672-a3d8-4128-9ed5-8a5d3151f662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:17.023549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a8672-a3d8-4128-9ed5-8a5d3151f662 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:17.023898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a8672-a3d8-4128-9ed5-8a5d3151f662 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:17.026412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a8672-a3d8-4128-9ed5-8a5d3151f662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:17.050072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a8672-a3d8-4128-9ed5-8a5d3151f662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:17.053923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a8672-a3d8-4128-9ed5-8a5d3151f662 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:17.061977 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2483/4894] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:16 2022] POST /identity/v3/users => generated 284 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:17.066259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de542c-8bda-4f76-9a67-2902c9a8d2e9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:17.080806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de542c-8bda-4f76-9a67-2902c9a8d2e9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:17.082924 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2412/4895] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:05:17.097853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606894b-96ca-4b8e-b27a-3ee2662a72af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:17.099020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606894b-96ca-4b8e-b27a-3ee2662a72af admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:17.099503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606894b-96ca-4b8e-b27a-3ee2662a72af admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:17.100032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606894b-96ca-4b8e-b27a-3ee2662a72af admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:17.100875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5606894b-96ca-4b8e-b27a-3ee2662a72af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:17.103502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5606894b-96ca-4b8e-b27a-3ee2662a72af admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:17.131318 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2484/4896] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:17.135721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a8672-a3d8-4128-9ed5-8a5d3151f662 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:17.146918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a8672-a3d8-4128-9ed5-8a5d3151f662 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:17.164110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c160f4-5dbd-4c91-a1a4-0b818b218e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:17.164988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c160f4-5dbd-4c91-a1a4-0b818b218e85 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:17.165369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c160f4-5dbd-4c91-a1a4-0b818b218e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:17.165765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c160f4-5dbd-4c91-a1a4-0b818b218e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:17.166567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c160f4-5dbd-4c91-a1a4-0b818b218e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:17.168312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c160f4-5dbd-4c91-a1a4-0b818b218e85 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:17.197136 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2413/4897] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:17.198595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606894b-96ca-4b8e-b27a-3ee2662a72af admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:17.216703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606894b-96ca-4b8e-b27a-3ee2662a72af admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:17.227440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54293da-88cd-4f0c-bc02-e37ccf0fe63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:17.228691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54293da-88cd-4f0c-bc02-e37ccf0fe63f admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:17.228985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54293da-88cd-4f0c-bc02-e37ccf0fe63f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:17.229499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54293da-88cd-4f0c-bc02-e37ccf0fe63f admin admin] PATH_INFO: `/v3/projects/683d68d5332246c1b9d4415348a85200/users/97e2521ab50645fa9a6fbc7d76751327/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:17.257063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54293da-88cd-4f0c-bc02-e37ccf0fe63f admin admin] RBAC: Authorizing `identity:create_grant(project_id=683d68d5332246c1b9d4415348a85200, user_id=97e2521ab50645fa9a6fbc7d76751327, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:17.271211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54293da-88cd-4f0c-bc02-e37ccf0fe63f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:17.316432 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2485/4898] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:17 2022] PUT /identity/v3/projects/683d68d5332246c1b9d4415348a85200/users/97e2521ab50645fa9a6fbc7d76751327/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:17.319382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4c9b0-9b08-4040-a29d-182a9bb90388 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:17.320303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4c9b0-9b08-4040-a29d-182a9bb90388 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:17.320597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4c9b0-9b08-4040-a29d-182a9bb90388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:17.448145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9bb4c9b0-9b08-4040-a29d-182a9bb90388 None None] MFA Rules not processed for user `97e2521ab50645fa9a6fbc7d76751327`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:17.453190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9bb4c9b0-9b08-4040-a29d-182a9bb90388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:17.653837 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2414/4899] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:17.659621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54293da-88cd-4f0c-bc02-e37ccf0fe63f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:17.804030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54293da-88cd-4f0c-bc02-e37ccf0fe63f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:17.830603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773d0b8-2b81-4d41-994f-6c7efea56ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:17.831789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773d0b8-2b81-4d41-994f-6c7efea56ff1 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:17.832589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773d0b8-2b81-4d41-994f-6c7efea56ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:17.833123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773d0b8-2b81-4d41-994f-6c7efea56ff1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:17.836460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773d0b8-2b81-4d41-994f-6c7efea56ff1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:17.841350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773d0b8-2b81-4d41-994f-6c7efea56ff1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:17.912502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4c9b0-9b08-4040-a29d-182a9bb90388 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:17.913013 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2486/4900] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:17 2022] POST /identity/v3/projects => generated 369 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:05:17.969888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4c9b0-9b08-4040-a29d-182a9bb90388 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:18.091727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:18.122381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d188e-e8b6-4b3e-b461-feebfbb65eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:18.123392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d188e-e8b6-4b3e-b461-feebfbb65eba admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:18.123675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d188e-e8b6-4b3e-b461-feebfbb65eba admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:18.124013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d188e-e8b6-4b3e-b461-feebfbb65eba admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:18.125492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d188e-e8b6-4b3e-b461-feebfbb65eba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:18.160718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d188e-e8b6-4b3e-b461-feebfbb65eba admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:18.195124 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2415/4901] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:17 2022] POST /identity/v3/users => generated 291 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:05:18.198932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773d0b8-2b81-4d41-994f-6c7efea56ff1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:18.230166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773d0b8-2b81-4d41-994f-6c7efea56ff1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:18.245332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77df1a4-4c37-4bdd-bd02-2737d9ce1ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:18.246285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77df1a4-4c37-4bdd-bd02-2737d9ce1ffa admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:18.246833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77df1a4-4c37-4bdd-bd02-2737d9ce1ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:18.247358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77df1a4-4c37-4bdd-bd02-2737d9ce1ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:18.248240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77df1a4-4c37-4bdd-bd02-2737d9ce1ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:18.253918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77df1a4-4c37-4bdd-bd02-2737d9ce1ffa admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:18.290996 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2487/4902] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:05:18.293637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d188e-e8b6-4b3e-b461-feebfbb65eba admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:18.315160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d188e-e8b6-4b3e-b461-feebfbb65eba admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:18.331999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf00f54-e54f-4519-a265-1091210fc535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:18.335580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf00f54-e54f-4519-a265-1091210fc535 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:18.338461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf00f54-e54f-4519-a265-1091210fc535 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:18.338917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf00f54-e54f-4519-a265-1091210fc535 admin admin] PATH_INFO: `/v3/projects/a49a297357e94b7a8ba794d60798b48e/users/3b356c625610471da308bb17d17b940e/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:18.424840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf00f54-e54f-4519-a265-1091210fc535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49a297357e94b7a8ba794d60798b48e, user_id=3b356c625610471da308bb17d17b940e, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:18.428122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf00f54-e54f-4519-a265-1091210fc535 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:18.457249 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2416/4903] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:18 2022] PUT /identity/v3/projects/a49a297357e94b7a8ba794d60798b48e/users/3b356c625610471da308bb17d17b940e/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:18.461461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77df1a4-4c37-4bdd-bd02-2737d9ce1ffa admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:18.480770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77df1a4-4c37-4bdd-bd02-2737d9ce1ffa admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:18.604208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:18.635918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170e74a-f58e-4b71-bf89-a27393b28c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:18.642057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170e74a-f58e-4b71-bf89-a27393b28c38 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:18.642544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170e74a-f58e-4b71-bf89-a27393b28c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:18.643311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170e74a-f58e-4b71-bf89-a27393b28c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:18.644131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170e74a-f58e-4b71-bf89-a27393b28c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:18.645883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170e74a-f58e-4b71-bf89-a27393b28c38 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:18.681442 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2488/4904] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:05:18.685257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf00f54-e54f-4519-a265-1091210fc535 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:18.697790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf00f54-e54f-4519-a265-1091210fc535 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:18.721385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9763d-230b-473d-ba13-772be60c59b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:18.722149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9763d-230b-473d-ba13-772be60c59b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:18.722674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9763d-230b-473d-ba13-772be60c59b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:18.723112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9763d-230b-473d-ba13-772be60c59b6 admin admin] PATH_INFO: `/v3/projects/a49a297357e94b7a8ba794d60798b48e/users/3b356c625610471da308bb17d17b940e/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:18.731112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9763d-230b-473d-ba13-772be60c59b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49a297357e94b7a8ba794d60798b48e, user_id=3b356c625610471da308bb17d17b940e, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:18.734670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9763d-230b-473d-ba13-772be60c59b6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:18.763402 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2417/4905] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:18 2022] PUT /identity/v3/projects/a49a297357e94b7a8ba794d60798b48e/users/3b356c625610471da308bb17d17b940e/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:18.774335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d112d-6c0c-4f59-a565-db89fa83da2f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:18.775004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d112d-6c0c-4f59-a565-db89fa83da2f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:18.775446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d112d-6c0c-4f59-a565-db89fa83da2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:18.872506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f61d112d-6c0c-4f59-a565-db89fa83da2f None None] MFA Rules not processed for user `3b356c625610471da308bb17d17b940e`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:18.874270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f61d112d-6c0c-4f59-a565-db89fa83da2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:19.015747 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2489/4906] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:05:19.020734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9763d-230b-473d-ba13-772be60c59b6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:19.039453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9763d-230b-473d-ba13-772be60c59b6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:19.051468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc2bdf-0e9b-4644-b8c6-0ab5b3caa0c6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:19.052341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc2bdf-0e9b-4644-b8c6-0ab5b3caa0c6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:19.052730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc2bdf-0e9b-4644-b8c6-0ab5b3caa0c6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:19.053108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc2bdf-0e9b-4644-b8c6-0ab5b3caa0c6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:19.054410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc2bdf-0e9b-4644-b8c6-0ab5b3caa0c6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:19.056088 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-c2dc2bdf-0e9b-4644-b8c6-0ab5b3caa0c6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:05:19.056088 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:05:19.056088 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:05:19.057601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc2bdf-0e9b-4644-b8c6-0ab5b3caa0c6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:19.094392 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2418/4907] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:05:19 2022] POST /identity/v3/domains => generated 276 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:19.098618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d112d-6c0c-4f59-a565-db89fa83da2f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:19.109732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d112d-6c0c-4f59-a565-db89fa83da2f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:19.192603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735b6aa-d0aa-45bf-93f1-56013acf89c6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:19.194520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735b6aa-d0aa-45bf-93f1-56013acf89c6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:19.194520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735b6aa-d0aa-45bf-93f1-56013acf89c6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:19.194520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735b6aa-d0aa-45bf-93f1-56013acf89c6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:19.194520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735b6aa-d0aa-45bf-93f1-56013acf89c6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:19.197461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735b6aa-d0aa-45bf-93f1-56013acf89c6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:19.233718 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2490/4908] 10.222.0.17 () {60 vars in 1147 bytes} [Sat Oct 1 21:05:19 2022] POST /identity/v3/groups => generated 281 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:19.239034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc2bdf-0e9b-4644-b8c6-0ab5b3caa0c6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:19.253619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc2bdf-0e9b-4644-b8c6-0ab5b3caa0c6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:19.255236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735b6aa-d0aa-45bf-93f1-56013acf89c6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:19.257623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4735b6aa-d0aa-45bf-93f1-56013acf89c6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:19.265036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7750dc-3710-403d-be44-6308d1d3f906 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:19.268877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7750dc-3710-403d-be44-6308d1d3f906 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:19.269236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7750dc-3710-403d-be44-6308d1d3f906 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:19.271095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7750dc-3710-403d-be44-6308d1d3f906 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/932ca5d551d648929d661c726a9b2814` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:19.278969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7750dc-3710-403d-be44-6308d1d3f906 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:update_group(group_id=932ca5d551d648929d661c726a9b2814)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:19.280288 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-6c7750dc-3710-403d-be44-6308d1d3f906 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:05:19.280288 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:05:19.280288 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:05:19.282664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7750dc-3710-403d-be44-6308d1d3f906 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:19.320384 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2419/4909] 10.222.0.17 () {60 vars in 1247 bytes} [Sat Oct 1 21:05:19 2022] PATCH /identity/v3/groups/932ca5d551d648929d661c726a9b2814 => generated 292 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:05:19.325062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7750dc-3710-403d-be44-6308d1d3f906 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:19.350402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7750dc-3710-403d-be44-6308d1d3f906 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:19.363002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e9945-01ad-4d0c-ba2b-84d1f909541d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:19.364450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e9945-01ad-4d0c-ba2b-84d1f909541d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:19.367320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e9945-01ad-4d0c-ba2b-84d1f909541d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:19.367812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e9945-01ad-4d0c-ba2b-84d1f909541d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/932ca5d551d648929d661c726a9b2814` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:19.382182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:19.382564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e9945-01ad-4d0c-ba2b-84d1f909541d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:get_group(group_id=932ca5d551d648929d661c726a9b2814)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:19.389437 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-355e9945-01ad-4d0c-ba2b-84d1f909541d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:05:19.389437 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:05:19.389437 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:05:19.393103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e9945-01ad-4d0c-ba2b-84d1f909541d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:19.399140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735b6aa-d0aa-45bf-93f1-56013acf89c6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:19.399855 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2420/4910] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:19 2022] GET /identity/v3/groups/932ca5d551d648929d661c726a9b2814 => generated 292 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:19.404978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e9945-01ad-4d0c-ba2b-84d1f909541d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:19.419321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345da91a-b41f-42e8-99cb-423e6d6867f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:19.420202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345da91a-b41f-42e8-99cb-423e6d6867f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:19.420666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345da91a-b41f-42e8-99cb-423e6d6867f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:19.421191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345da91a-b41f-42e8-99cb-423e6d6867f6 admin admin] PATH_INFO: `/v3/projects/dfe3cfd637894853b34a1450f7bcf683` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:19.423720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345da91a-b41f-42e8-99cb-423e6d6867f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfe3cfd637894853b34a1450f7bcf683)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:19.429251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345da91a-b41f-42e8-99cb-423e6d6867f6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:19.433322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e9945-01ad-4d0c-ba2b-84d1f909541d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:19.441156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156bf32-8fd6-400e-82e3-9abb871bf5a4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:19.443073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156bf32-8fd6-400e-82e3-9abb871bf5a4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:19.443619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156bf32-8fd6-400e-82e3-9abb871bf5a4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:19.444229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156bf32-8fd6-400e-82e3-9abb871bf5a4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/932ca5d551d648929d661c726a9b2814` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:19.446211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156bf32-8fd6-400e-82e3-9abb871bf5a4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:update_group(group_id=932ca5d551d648929d661c726a9b2814)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:19.453532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156bf32-8fd6-400e-82e3-9abb871bf5a4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:19.503970 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2421/4911] 10.222.0.17 () {60 vars in 1246 bytes} [Sat Oct 1 21:05:19 2022] PATCH /identity/v3/groups/932ca5d551d648929d661c726a9b2814 => generated 307 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:19.507803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156bf32-8fd6-400e-82e3-9abb871bf5a4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:19.523960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156bf32-8fd6-400e-82e3-9abb871bf5a4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:19.535169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a4da0-ad41-4267-87ed-ebfc1ee460a6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:19.536370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a4da0-ad41-4267-87ed-ebfc1ee460a6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:19.537020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a4da0-ad41-4267-87ed-ebfc1ee460a6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:19.537541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a4da0-ad41-4267-87ed-ebfc1ee460a6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/932ca5d551d648929d661c726a9b2814` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:19.549199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a4da0-ad41-4267-87ed-ebfc1ee460a6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:delete_group(group_id=932ca5d551d648929d661c726a9b2814)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:19.555745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a4da0-ad41-4267-87ed-ebfc1ee460a6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:19.570982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-345da91a-b41f-42e8-99cb-423e6d6867f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfe3cfd637894853b34a1450f7bcf683', 'request_id': 'req-345da91a-b41f-42e8-99cb-423e6d6867f6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:19.589064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-345da91a-b41f-42e8-99cb-423e6d6867f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfe3cfd637894853b34a1450f7bcf683', 'request_id': 'req-345da91a-b41f-42e8-99cb-423e6d6867f6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:19.610411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-345da91a-b41f-42e8-99cb-423e6d6867f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfe3cfd637894853b34a1450f7bcf683', 'request_id': 'req-345da91a-b41f-42e8-99cb-423e6d6867f6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:19.646069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-345da91a-b41f-42e8-99cb-423e6d6867f6 admin admin] The token cache is being invalidate because project dfe3cfd637894853b34a1450f7bcf683 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:19.646472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-345da91a-b41f-42e8-99cb-423e6d6867f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:19.649172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-345da91a-b41f-42e8-99cb-423e6d6867f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:19.653763 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2491/4912] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:19 2022] DELETE /identity/v3/projects/dfe3cfd637894853b34a1450f7bcf683 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:19.657668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345da91a-b41f-42e8-99cb-423e6d6867f6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:19.660774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-345da91a-b41f-42e8-99cb-423e6d6867f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:19.679987 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2422/4913] 10.222.0.17 () {58 vars in 1227 bytes} [Sat Oct 1 21:05:19 2022] DELETE /identity/v3/groups/932ca5d551d648929d661c726a9b2814 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:19.684781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a4da0-ad41-4267-87ed-ebfc1ee460a6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:19.687660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-869a4da0-ad41-4267-87ed-ebfc1ee460a6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:19.768818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:INSERT=6,keystone:UPDATE=2,keystone:DELETE=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:19.823469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345da91a-b41f-42e8-99cb-423e6d6867f6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:19.828845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a4da0-ad41-4267-87ed-ebfc1ee460a6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:19.846582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9734bb8-1e49-4ebb-a973-229cf28e55e9 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:19.848564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9734bb8-1e49-4ebb-a973-229cf28e55e9 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:19.849015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9734bb8-1e49-4ebb-a973-229cf28e55e9 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:19.851966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9734bb8-1e49-4ebb-a973-229cf28e55e9 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:19.853120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9734bb8-1e49-4ebb-a973-229cf28e55e9 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:19.856004 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-f9734bb8-1e49-4ebb-a973-229cf28e55e9 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:05:19.856004 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:05:19.856004 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:05:19.861863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9734bb8-1e49-4ebb-a973-229cf28e55e9 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:19.904326 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2423/4914] 10.222.0.17 () {60 vars in 1147 bytes} [Sat Oct 1 21:05:19 2022] POST /identity/v3/groups => generated 327 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:19.906664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9734bb8-1e49-4ebb-a973-229cf28e55e9 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:19.917708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9734bb8-1e49-4ebb-a973-229cf28e55e9 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:19.924094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ffd61-3b51-49b2-80f1-04028871c405 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:19.925034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ffd61-3b51-49b2-80f1-04028871c405 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:19.925667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ffd61-3b51-49b2-80f1-04028871c405 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:19.926180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ffd61-3b51-49b2-80f1-04028871c405 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:19.928059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ffd61-3b51-49b2-80f1-04028871c405 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:19.930737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ffd61-3b51-49b2-80f1-04028871c405 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:19.955488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbb4a5-c7e4-4535-a4aa-66842d7ab24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:19.956144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbb4a5-c7e4-4535-a4aa-66842d7ab24b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:19.956405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbb4a5-c7e4-4535-a4aa-66842d7ab24b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:19.956703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbb4a5-c7e4-4535-a4aa-66842d7ab24b admin admin] PATH_INFO: `/v3/users/b6abf93f4959468caf39ce44d472b7f8` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:19.959706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbb4a5-c7e4-4535-a4aa-66842d7ab24b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6abf93f4959468caf39ce44d472b7f8)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:19.962827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbb4a5-c7e4-4535-a4aa-66842d7ab24b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:19.995470 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2424/4915] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:19 2022] POST /identity/v3/users => generated 323 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:19.997021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ffd61-3b51-49b2-80f1-04028871c405 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:20.017839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ffd61-3b51-49b2-80f1-04028871c405 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:20.038719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:20.048174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1a0bd-80a9-449a-a8f5-9c663fbc1897 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:20.049013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1a0bd-80a9-449a-a8f5-9c663fbc1897 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:20.049301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1a0bd-80a9-449a-a8f5-9c663fbc1897 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:20.049478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1a0bd-80a9-449a-a8f5-9c663fbc1897 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/85258326386c476e8811a1673c5c99bc/users/902022469b5b49ae8ee86d24eed9da49` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:20.055371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2cbbb4a5-c7e4-4535-a4aa-66842d7ab24b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6abf93f4959468caf39ce44d472b7f8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:20.078053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2cbbb4a5-c7e4-4535-a4aa-66842d7ab24b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6abf93f4959468caf39ce44d472b7f8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:20.078315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1a0bd-80a9-449a-a8f5-9c663fbc1897 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=85258326386c476e8811a1673c5c99bc, user_id=902022469b5b49ae8ee86d24eed9da49)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:20.079624 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-01c1a0bd-80a9-449a-a8f5-9c663fbc1897 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:add_user_to_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:05:20.079624 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:05:20.079624 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:05:20.080843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1a0bd-80a9-449a-a8f5-9c663fbc1897 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:20.113337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2cbbb4a5-c7e4-4535-a4aa-66842d7ab24b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6abf93f4959468caf39ce44d472b7f8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:20.115263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2cbbb4a5-c7e4-4535-a4aa-66842d7ab24b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6abf93f4959468caf39ce44d472b7f8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:20.130967 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2425/4916] 10.222.0.17 () {60 vars in 1360 bytes} [Sat Oct 1 21:05:19 2022] PUT /identity/v3/groups/85258326386c476e8811a1673c5c99bc/users/902022469b5b49ae8ee86d24eed9da49 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:20.146276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2cbbb4a5-c7e4-4535-a4aa-66842d7ab24b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6abf93f4959468caf39ce44d472b7f8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:20.148309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1a0bd-80a9-449a-a8f5-9c663fbc1897 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:20.154340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-01c1a0bd-80a9-449a-a8f5-9c663fbc1897 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:20.168062 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2492/4917] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:19 2022] DELETE /identity/v3/users/b6abf93f4959468caf39ce44d472b7f8 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:20.286200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1a0bd-80a9-449a-a8f5-9c663fbc1897 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:20.408334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411855e7-5583-4416-b2dc-f010f6cc71e3 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:20.409094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411855e7-5583-4416-b2dc-f010f6cc71e3 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:20.409481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411855e7-5583-4416-b2dc-f010f6cc71e3 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:20.409858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411855e7-5583-4416-b2dc-f010f6cc71e3 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:20.410698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411855e7-5583-4416-b2dc-f010f6cc71e3 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:20.413342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411855e7-5583-4416-b2dc-f010f6cc71e3 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:20.458678 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2426/4918] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:20 2022] POST /identity/v3/users => generated 321 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:20.463029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbb4a5-c7e4-4535-a4aa-66842d7ab24b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:20.484761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbb4a5-c7e4-4535-a4aa-66842d7ab24b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:20.496816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f3b3a-5aee-4ca3-8869-c3d3da3c8460 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:20.500559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f3b3a-5aee-4ca3-8869-c3d3da3c8460 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:20.500905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f3b3a-5aee-4ca3-8869-c3d3da3c8460 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:20.504929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f3b3a-5aee-4ca3-8869-c3d3da3c8460 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/85258326386c476e8811a1673c5c99bc/users/00af2cfb51404f3b85333dcdcb8e7783` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:20.547741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f3b3a-5aee-4ca3-8869-c3d3da3c8460 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=85258326386c476e8811a1673c5c99bc, user_id=00af2cfb51404f3b85333dcdcb8e7783)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:20.549690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f3b3a-5aee-4ca3-8869-c3d3da3c8460 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:20.594599 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2493/4919] 10.222.0.17 () {60 vars in 1360 bytes} [Sat Oct 1 21:05:20 2022] PUT /identity/v3/groups/85258326386c476e8811a1673c5c99bc/users/00af2cfb51404f3b85333dcdcb8e7783 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:05:20.597896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411855e7-5583-4416-b2dc-f010f6cc71e3 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:20.626431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411855e7-5583-4416-b2dc-f010f6cc71e3 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:20.663912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:20.744319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b43c2-b5dd-47f9-8102-eed7e5f29825 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:20.745453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b43c2-b5dd-47f9-8102-eed7e5f29825 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:20.746007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b43c2-b5dd-47f9-8102-eed7e5f29825 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:20.746417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b43c2-b5dd-47f9-8102-eed7e5f29825 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:20.747267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b43c2-b5dd-47f9-8102-eed7e5f29825 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:20.762661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b43c2-b5dd-47f9-8102-eed7e5f29825 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:20.778811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f3b3a-5aee-4ca3-8869-c3d3da3c8460 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:20.781872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-828f3b3a-5aee-4ca3-8869-c3d3da3c8460 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:20.962833 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2427/4920] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:20 2022] POST /identity/v3/users => generated 321 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:20.964008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b43c2-b5dd-47f9-8102-eed7e5f29825 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:20.985825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b43c2-b5dd-47f9-8102-eed7e5f29825 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:20.994810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996dca1-954f-4aab-b69a-796021c00986 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:20.995549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996dca1-954f-4aab-b69a-796021c00986 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:20.995837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996dca1-954f-4aab-b69a-796021c00986 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:20.996174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996dca1-954f-4aab-b69a-796021c00986 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/85258326386c476e8811a1673c5c99bc/users/8bda20c8d0f942629487614c29b07ea4` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:21.031760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f3b3a-5aee-4ca3-8869-c3d3da3c8460 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:21.040205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7996dca1-954f-4aab-b69a-796021c00986 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=85258326386c476e8811a1673c5c99bc, user_id=8bda20c8d0f942629487614c29b07ea4)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:21.040571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baefe78-d0b0-45a0-831b-89b4d168f460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:21.041712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baefe78-d0b0-45a0-831b-89b4d168f460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:21.042208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baefe78-d0b0-45a0-831b-89b4d168f460 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:21.042762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7996dca1-954f-4aab-b69a-796021c00986 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:21.043104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baefe78-d0b0-45a0-831b-89b4d168f460 admin admin] PATH_INFO: `/v3/projects/edc972be74574a12b82fe88feef1ce76` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:21.044588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baefe78-d0b0-45a0-831b-89b4d168f460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edc972be74574a12b82fe88feef1ce76)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:21.051221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baefe78-d0b0-45a0-831b-89b4d168f460 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:21.094348 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2428/4921] 10.222.0.17 () {60 vars in 1360 bytes} [Sat Oct 1 21:05:20 2022] PUT /identity/v3/groups/85258326386c476e8811a1673c5c99bc/users/8bda20c8d0f942629487614c29b07ea4 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:21.099675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996dca1-954f-4aab-b69a-796021c00986 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:21.180636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6baefe78-d0b0-45a0-831b-89b4d168f460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edc972be74574a12b82fe88feef1ce76', 'request_id': 'req-6baefe78-d0b0-45a0-831b-89b4d168f460'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:21.196365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6baefe78-d0b0-45a0-831b-89b4d168f460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edc972be74574a12b82fe88feef1ce76', 'request_id': 'req-6baefe78-d0b0-45a0-831b-89b4d168f460'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:21.210198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6baefe78-d0b0-45a0-831b-89b4d168f460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edc972be74574a12b82fe88feef1ce76', 'request_id': 'req-6baefe78-d0b0-45a0-831b-89b4d168f460'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:21.228127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996dca1-954f-4aab-b69a-796021c00986 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:21.233398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6baefe78-d0b0-45a0-831b-89b4d168f460 admin admin] The token cache is being invalidate because project edc972be74574a12b82fe88feef1ce76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:21.236174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6baefe78-d0b0-45a0-831b-89b4d168f460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:21.237274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6baefe78-d0b0-45a0-831b-89b4d168f460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:21.241530 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2494/4922] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:20 2022] DELETE /identity/v3/projects/edc972be74574a12b82fe88feef1ce76 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:21.247233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5bd7e-824e-4f7b-896e-88a9888703ad None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:21.247583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5bd7e-824e-4f7b-896e-88a9888703ad None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:21.248092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5bd7e-824e-4f7b-896e-88a9888703ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:21.295431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-54a5bd7e-824e-4f7b-896e-88a9888703ad None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:21.302244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-54a5bd7e-824e-4f7b-896e-88a9888703ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:21.319056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:21.376239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30e67c-8bc1-4ee8-b418-8a3072b0d08c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:21.376981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30e67c-8bc1-4ee8-b418-8a3072b0d08c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:21.377329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30e67c-8bc1-4ee8-b418-8a3072b0d08c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:21.377672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30e67c-8bc1-4ee8-b418-8a3072b0d08c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/85258326386c476e8811a1673c5c99bc/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:21.379293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca30e67c-8bc1-4ee8-b418-8a3072b0d08c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:list_users_in_group(group_id=85258326386c476e8811a1673c5c99bc)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:21.380847 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-ca30e67c-8bc1-4ee8-b418-8a3072b0d08c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:05:21.380847 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:05:21.380847 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:05:21.382649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca30e67c-8bc1-4ee8-b418-8a3072b0d08c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:21.430656 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2429/4923] 10.222.0.17 () {58 vars in 1242 bytes} [Sat Oct 1 21:05:21 2022] GET /identity/v3/groups/85258326386c476e8811a1673c5c99bc/users => generated 1081 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:21.433327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30e67c-8bc1-4ee8-b418-8a3072b0d08c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:21.440249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ca30e67c-8bc1-4ee8-b418-8a3072b0d08c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:21.486064 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2495/4924] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:21.493842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e070b6a-e3ac-495b-910d-25b906d0bab9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:21.495041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e070b6a-e3ac-495b-910d-25b906d0bab9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:21.495417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e070b6a-e3ac-495b-910d-25b906d0bab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:21.500947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30e67c-8bc1-4ee8-b418-8a3072b0d08c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:21.515523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc77b5a-23cf-4aaa-9311-64c3aa5f81c4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:21.517605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc77b5a-23cf-4aaa-9311-64c3aa5f81c4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:21.519487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc77b5a-23cf-4aaa-9311-64c3aa5f81c4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:21.520040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc77b5a-23cf-4aaa-9311-64c3aa5f81c4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/85258326386c476e8811a1673c5c99bc/users/902022469b5b49ae8ee86d24eed9da49` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:21.526024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc77b5a-23cf-4aaa-9311-64c3aa5f81c4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=85258326386c476e8811a1673c5c99bc, user_id=902022469b5b49ae8ee86d24eed9da49)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:21.529225 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-bfc77b5a-23cf-4aaa-9311-64c3aa5f81c4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_user_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:05:21.529225 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:05:21.529225 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:05:21.531898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc77b5a-23cf-4aaa-9311-64c3aa5f81c4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:21.552625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1e070b6a-e3ac-495b-910d-25b906d0bab9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:21.553998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1e070b6a-e3ac-495b-910d-25b906d0bab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:21.572057 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2430/4925] 10.222.0.17 () {58 vars in 1342 bytes} [Sat Oct 1 21:05:21 2022] HEAD /identity/v3/groups/85258326386c476e8811a1673c5c99bc/users/902022469b5b49ae8ee86d24eed9da49 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:21.572831 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2496/4926] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:21.575172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc77b5a-23cf-4aaa-9311-64c3aa5f81c4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:21.577338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e070b6a-e3ac-495b-910d-25b906d0bab9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:21.586733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc77b5a-23cf-4aaa-9311-64c3aa5f81c4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:21.597672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36343ae2-8085-4954-bc33-1571e96be995 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:21.598528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36343ae2-8085-4954-bc33-1571e96be995 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:21.599117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36343ae2-8085-4954-bc33-1571e96be995 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:21.600284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36343ae2-8085-4954-bc33-1571e96be995 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/85258326386c476e8811a1673c5c99bc/users/902022469b5b49ae8ee86d24eed9da49` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:21.601377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e070b6a-e3ac-495b-910d-25b906d0bab9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:21.603242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36343ae2-8085-4954-bc33-1571e96be995 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=85258326386c476e8811a1673c5c99bc, user_id=902022469b5b49ae8ee86d24eed9da49)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:21.605151 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-36343ae2-8085-4954-bc33-1571e96be995 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:remove_user_from_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:05:21.605151 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:05:21.605151 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:05:21.607472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36343ae2-8085-4954-bc33-1571e96be995 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:21.612124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:21.614607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e3b81-9556-432f-aa5c-3c52b0f0391d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:21.615410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e3b81-9556-432f-aa5c-3c52b0f0391d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:21.615824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e3b81-9556-432f-aa5c-3c52b0f0391d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:21.616224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e3b81-9556-432f-aa5c-3c52b0f0391d admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:21.617599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1e3b81-9556-432f-aa5c-3c52b0f0391d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:05:21.618626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1e3b81-9556-432f-aa5c-3c52b0f0391d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:21.621329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1e3b81-9556-432f-aa5c-3c52b0f0391d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:21.631651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-36343ae2-8085-4954-bc33-1571e96be995 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '902022469b5b49ae8ee86d24eed9da49'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:21.678550 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2497/4928] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:21.678768 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2431/4928] 10.222.0.17 () {58 vars in 1344 bytes} [Sat Oct 1 21:05:21 2022] DELETE /identity/v3/groups/85258326386c476e8811a1673c5c99bc/users/902022469b5b49ae8ee86d24eed9da49 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:21.683020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36343ae2-8085-4954-bc33-1571e96be995 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:21.683181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e3b81-9556-432f-aa5c-3c52b0f0391d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:21.707147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e3b81-9556-432f-aa5c-3c52b0f0391d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:21.804799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36343ae2-8085-4954-bc33-1571e96be995 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:21.814507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf2276-3b54-471c-96ab-f4ea57f8fbf2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:21.815264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf2276-3b54-471c-96ab-f4ea57f8fbf2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:21.815637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf2276-3b54-471c-96ab-f4ea57f8fbf2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:21.816070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf2276-3b54-471c-96ab-f4ea57f8fbf2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/85258326386c476e8811a1673c5c99bc/users/00af2cfb51404f3b85333dcdcb8e7783` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:21.818942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf2276-3b54-471c-96ab-f4ea57f8fbf2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=85258326386c476e8811a1673c5c99bc, user_id=00af2cfb51404f3b85333dcdcb8e7783)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:21.821219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf2276-3b54-471c-96ab-f4ea57f8fbf2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:21.830181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac4c5f-8539-48c2-9c26-d397bd3b1dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:21.830995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac4c5f-8539-48c2-9c26-d397bd3b1dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:21.832097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac4c5f-8539-48c2-9c26-d397bd3b1dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:21.832097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac4c5f-8539-48c2-9c26-d397bd3b1dd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:21.832443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac4c5f-8539-48c2-9c26-d397bd3b1dd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:21.834796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac4c5f-8539-48c2-9c26-d397bd3b1dd6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:21.849496 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2432/4929] 10.222.0.17 () {58 vars in 1342 bytes} [Sat Oct 1 21:05:21 2022] HEAD /identity/v3/groups/85258326386c476e8811a1673c5c99bc/users/00af2cfb51404f3b85333dcdcb8e7783 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:21.850804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf2276-3b54-471c-96ab-f4ea57f8fbf2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:21.862362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf2276-3b54-471c-96ab-f4ea57f8fbf2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:21.868630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ca004-adf9-43d9-ab42-13aaefce256e tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:21.869550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ca004-adf9-43d9-ab42-13aaefce256e tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:21.870919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ca004-adf9-43d9-ab42-13aaefce256e tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:21.871497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ca004-adf9-43d9-ab42-13aaefce256e tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/85258326386c476e8811a1673c5c99bc/users/00af2cfb51404f3b85333dcdcb8e7783` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:21.876330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ca004-adf9-43d9-ab42-13aaefce256e tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=85258326386c476e8811a1673c5c99bc, user_id=00af2cfb51404f3b85333dcdcb8e7783)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:21.878288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ca004-adf9-43d9-ab42-13aaefce256e tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:21.892266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-130ca004-adf9-43d9-ab42-13aaefce256e tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '00af2cfb51404f3b85333dcdcb8e7783'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:21.897526 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2498/4930] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:21 2022] POST /identity/v3/projects => generated 383 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:21.899129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27198931-8c42-4098-90a4-87115d977ab4 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:21.899638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27198931-8c42-4098-90a4-87115d977ab4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:21.900117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27198931-8c42-4098-90a4-87115d977ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:21.916018 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2433/4931] 10.222.0.17 () {58 vars in 1344 bytes} [Sat Oct 1 21:05:21 2022] DELETE /identity/v3/groups/85258326386c476e8811a1673c5c99bc/users/00af2cfb51404f3b85333dcdcb8e7783 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:21.919669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ca004-adf9-43d9-ab42-13aaefce256e tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:21.922544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-27198931-8c42-4098-90a4-87115d977ab4 None None] MFA Rules not processed for user `9b5f90f4ec8b4b5cb4b64a2b99ca9beb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:21.923984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-27198931-8c42-4098-90a4-87115d977ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:21.994288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ca004-adf9-43d9-ab42-13aaefce256e tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:22.001844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841ca1c-853a-4549-b773-15a9d987285a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:22.002657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841ca1c-853a-4549-b773-15a9d987285a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:22.004850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841ca1c-853a-4549-b773-15a9d987285a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:22.005336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841ca1c-853a-4549-b773-15a9d987285a admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:22.006098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841ca1c-853a-4549-b773-15a9d987285a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:22.008362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841ca1c-853a-4549-b773-15a9d987285a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:22.014762 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2499/4932] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:22.016447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27198931-8c42-4098-90a4-87115d977ab4 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:22.040932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:22.047664 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2434/4933] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:21 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:22.049405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a213e-ee04-4f7e-8162-5074752d896f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:22.049913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a213e-ee04-4f7e-8162-5074752d896f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:22.050759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a213e-ee04-4f7e-8162-5074752d896f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:22.085324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bf5a213e-ee04-4f7e-8162-5074752d896f None None] MFA Rules not processed for user `9b5f90f4ec8b4b5cb4b64a2b99ca9beb`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:22.087220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bf5a213e-ee04-4f7e-8162-5074752d896f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:22.098630 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2435/4934] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:22.098630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a213e-ee04-4f7e-8162-5074752d896f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:22.121346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a213e-ee04-4f7e-8162-5074752d896f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:22.128376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f5136-5289-4ea8-be21-7a28e99643c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:22.129307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f5136-5289-4ea8-be21-7a28e99643c8 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:22.129597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f5136-5289-4ea8-be21-7a28e99643c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:22.129987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f5136-5289-4ea8-be21-7a28e99643c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:22.130799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873f5136-5289-4ea8-be21-7a28e99643c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:22.139405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873f5136-5289-4ea8-be21-7a28e99643c8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:22.177445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27198931-8c42-4098-90a4-87115d977ab4 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:22.179948 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2436/4935] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:22.182659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f5136-5289-4ea8-be21-7a28e99643c8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:22.188912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a448ba0-68d2-47c1-840b-a46b6f6079b0 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:22.189702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a448ba0-68d2-47c1-840b-a46b6f6079b0 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:22.189995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a448ba0-68d2-47c1-840b-a46b6f6079b0 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:22.190372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a448ba0-68d2-47c1-840b-a46b6f6079b0 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/85258326386c476e8811a1673c5c99bc/users/8bda20c8d0f942629487614c29b07ea4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:22.195287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a448ba0-68d2-47c1-840b-a46b6f6079b0 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=85258326386c476e8811a1673c5c99bc, user_id=8bda20c8d0f942629487614c29b07ea4)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:22.196404 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-9a448ba0-68d2-47c1-840b-a46b6f6079b0 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_user_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:05:22.196404 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:05:22.196404 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:05:22.199504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a448ba0-68d2-47c1-840b-a46b6f6079b0 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:22.211387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f5136-5289-4ea8-be21-7a28e99643c8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:22.229080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81469e0e-f551-40c3-9b81-13594280ddc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:22.230095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81469e0e-f551-40c3-9b81-13594280ddc6 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:22.230791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81469e0e-f551-40c3-9b81-13594280ddc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:22.231222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81469e0e-f551-40c3-9b81-13594280ddc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:22.232264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81469e0e-f551-40c3-9b81-13594280ddc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:22.235695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81469e0e-f551-40c3-9b81-13594280ddc6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:22.240914 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2500/4936] 10.222.0.17 () {58 vars in 1342 bytes} [Sat Oct 1 21:05:22 2022] HEAD /identity/v3/groups/85258326386c476e8811a1673c5c99bc/users/8bda20c8d0f942629487614c29b07ea4 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:22.242306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a448ba0-68d2-47c1-840b-a46b6f6079b0 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:22.264663 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2437/4937] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:22.269841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81469e0e-f551-40c3-9b81-13594280ddc6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:22.273304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a448ba0-68d2-47c1-840b-a46b6f6079b0 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:22.281788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad76e4-70e7-4ebb-8825-7355eb266ad6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:22.282909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad76e4-70e7-4ebb-8825-7355eb266ad6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:22.283949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad76e4-70e7-4ebb-8825-7355eb266ad6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:22.284429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad76e4-70e7-4ebb-8825-7355eb266ad6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/85258326386c476e8811a1673c5c99bc/users/8bda20c8d0f942629487614c29b07ea4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:22.287157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad76e4-70e7-4ebb-8825-7355eb266ad6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=85258326386c476e8811a1673c5c99bc, user_id=8bda20c8d0f942629487614c29b07ea4)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:22.291290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81469e0e-f551-40c3-9b81-13594280ddc6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:22.292680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad76e4-70e7-4ebb-8825-7355eb266ad6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:22.303366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce54e81-4666-4f41-b159-01e9cb68d8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:22.304114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce54e81-4666-4f41-b159-01e9cb68d8b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:22.304527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce54e81-4666-4f41-b159-01e9cb68d8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:22.305087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce54e81-4666-4f41-b159-01e9cb68d8b6 admin admin] PATH_INFO: `/v3/projects/b6cce17fe4f34afa9a894e632aed5126/users/b0cbab34b69f481989e118bd0508deb4/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:22.310074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-35ad76e4-70e7-4ebb-8825-7355eb266ad6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '8bda20c8d0f942629487614c29b07ea4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:22.333592 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2501/4938] 10.222.0.17 () {58 vars in 1344 bytes} [Sat Oct 1 21:05:22 2022] DELETE /identity/v3/groups/85258326386c476e8811a1673c5c99bc/users/8bda20c8d0f942629487614c29b07ea4 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:22.338041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad76e4-70e7-4ebb-8825-7355eb266ad6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:22.345442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:22.350408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce54e81-4666-4f41-b159-01e9cb68d8b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6cce17fe4f34afa9a894e632aed5126, user_id=b0cbab34b69f481989e118bd0508deb4, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:22.355255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce54e81-4666-4f41-b159-01e9cb68d8b6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:22.392286 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2438/4939] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:22 2022] PUT /identity/v3/projects/b6cce17fe4f34afa9a894e632aed5126/users/b0cbab34b69f481989e118bd0508deb4/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:22.406395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28729098-dc78-4905-9dca-49043abbbcfe None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:22.408960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28729098-dc78-4905-9dca-49043abbbcfe None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:22.410950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28729098-dc78-4905-9dca-49043abbbcfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:22.477952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-28729098-dc78-4905-9dca-49043abbbcfe None None] MFA Rules not processed for user `b0cbab34b69f481989e118bd0508deb4`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:22.479757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-28729098-dc78-4905-9dca-49043abbbcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:22.483113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad76e4-70e7-4ebb-8825-7355eb266ad6 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:22.579218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2d890-26d9-4f86-b561-90c45d1c37b1 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:22.580046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2d890-26d9-4f86-b561-90c45d1c37b1 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:22.580713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2d890-26d9-4f86-b561-90c45d1c37b1 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:22.580857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2d890-26d9-4f86-b561-90c45d1c37b1 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/85258326386c476e8811a1673c5c99bc/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:22.583460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2d890-26d9-4f86-b561-90c45d1c37b1 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:list_users_in_group(group_id=85258326386c476e8811a1673c5c99bc)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:22.584890 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-cdc2d890-26d9-4f86-b561-90c45d1c37b1 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:05:22.584890 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:05:22.584890 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:05:22.586696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2d890-26d9-4f86-b561-90c45d1c37b1 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:22.621203 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2502/4940] 10.222.0.17 () {58 vars in 1242 bytes} [Sat Oct 1 21:05:22 2022] GET /identity/v3/groups/85258326386c476e8811a1673c5c99bc/users => generated 145 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:22.626076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2d890-26d9-4f86-b561-90c45d1c37b1 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:22.651057 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2439/4941] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:22.655680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28729098-dc78-4905-9dca-49043abbbcfe None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:22.661979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2d890-26d9-4f86-b561-90c45d1c37b1 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:22.679432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942044c-db4e-4c96-a898-7a6221972fa5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:22.680620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942044c-db4e-4c96-a898-7a6221972fa5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:22.680984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942044c-db4e-4c96-a898-7a6221972fa5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:22.681361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942044c-db4e-4c96-a898-7a6221972fa5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/users/8bda20c8d0f942629487614c29b07ea4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:22.683264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942044c-db4e-4c96-a898-7a6221972fa5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:delete_user(user_id=8bda20c8d0f942629487614c29b07ea4)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:22.686082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942044c-db4e-4c96-a898-7a6221972fa5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:22.736233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:22.745364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28729098-dc78-4905-9dca-49043abbbcfe None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:22.752982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a24057-71a9-4b11-ac32-92c924498818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:22.754591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a24057-71a9-4b11-ac32-92c924498818 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:22.754880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a24057-71a9-4b11-ac32-92c924498818 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:22.755162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a24057-71a9-4b11-ac32-92c924498818 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:22.755804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a24057-71a9-4b11-ac32-92c924498818 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:22.757906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a24057-71a9-4b11-ac32-92c924498818 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:22.790034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b942044c-db4e-4c96-a898-7a6221972fa5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bda20c8d0f942629487614c29b07ea4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:22.813570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b942044c-db4e-4c96-a898-7a6221972fa5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bda20c8d0f942629487614c29b07ea4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:22.837118 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2440/4942] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:22 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:05:22.841341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a24057-71a9-4b11-ac32-92c924498818 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:22.852266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b942044c-db4e-4c96-a898-7a6221972fa5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bda20c8d0f942629487614c29b07ea4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:22.855361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b942044c-db4e-4c96-a898-7a6221972fa5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bda20c8d0f942629487614c29b07ea4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:22.876131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b942044c-db4e-4c96-a898-7a6221972fa5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bda20c8d0f942629487614c29b07ea4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:22.889484 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2503/4943] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:22 2022] DELETE /identity/v3/users/8bda20c8d0f942629487614c29b07ea4 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:22.894951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942044c-db4e-4c96-a898-7a6221972fa5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:22.897100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b942044c-db4e-4c96-a898-7a6221972fa5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:23.012338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a24057-71a9-4b11-ac32-92c924498818 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:23.015280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942044c-db4e-4c96-a898-7a6221972fa5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:23.021436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf981eb-9b5b-4c6c-90c9-31fa7864016d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:23.022729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf981eb-9b5b-4c6c-90c9-31fa7864016d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:23.023019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf981eb-9b5b-4c6c-90c9-31fa7864016d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:23.023302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf981eb-9b5b-4c6c-90c9-31fa7864016d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/users/00af2cfb51404f3b85333dcdcb8e7783` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:23.034406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:23.034685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf981eb-9b5b-4c6c-90c9-31fa7864016d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:delete_user(user_id=00af2cfb51404f3b85333dcdcb8e7783)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:23.037763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf981eb-9b5b-4c6c-90c9-31fa7864016d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:23.129710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bdf981eb-9b5b-4c6c-90c9-31fa7864016d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00af2cfb51404f3b85333dcdcb8e7783'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:23.155915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bdf981eb-9b5b-4c6c-90c9-31fa7864016d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00af2cfb51404f3b85333dcdcb8e7783'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:23.161447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88e4e8-239a-4928-9ce7-93371fb5153c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:23.162900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88e4e8-239a-4928-9ce7-93371fb5153c admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:23.164301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88e4e8-239a-4928-9ce7-93371fb5153c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:23.164912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88e4e8-239a-4928-9ce7-93371fb5153c admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:23.167050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88e4e8-239a-4928-9ce7-93371fb5153c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:23.169184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88e4e8-239a-4928-9ce7-93371fb5153c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:23.187334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bdf981eb-9b5b-4c6c-90c9-31fa7864016d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00af2cfb51404f3b85333dcdcb8e7783'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:23.188218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bdf981eb-9b5b-4c6c-90c9-31fa7864016d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00af2cfb51404f3b85333dcdcb8e7783'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:23.202929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bdf981eb-9b5b-4c6c-90c9-31fa7864016d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00af2cfb51404f3b85333dcdcb8e7783'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:23.213316 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2504/4944] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:22 2022] DELETE /identity/v3/users/00af2cfb51404f3b85333dcdcb8e7783 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:23.215951 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2441/4945] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:22 2022] POST /identity/v3/users => generated 299 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:23.221530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88e4e8-239a-4928-9ce7-93371fb5153c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:23.221747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf981eb-9b5b-4c6c-90c9-31fa7864016d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:23.223803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bdf981eb-9b5b-4c6c-90c9-31fa7864016d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:23.224549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3d88e4e8-239a-4928-9ce7-93371fb5153c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:23.253728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88e4e8-239a-4928-9ce7-93371fb5153c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:23.284649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:23.395438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd849512-19d7-44c7-8258-b124302ea126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:23.396243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd849512-19d7-44c7-8258-b124302ea126 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:23.396529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd849512-19d7-44c7-8258-b124302ea126 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:23.396801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd849512-19d7-44c7-8258-b124302ea126 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:23.398109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd849512-19d7-44c7-8258-b124302ea126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:23.400323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd849512-19d7-44c7-8258-b124302ea126 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:23.412284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf981eb-9b5b-4c6c-90c9-31fa7864016d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:23.423198 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2442/4946] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:23.426332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9ee23-4938-4850-ab03-1f60bce81175 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:23.427359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd849512-19d7-44c7-8258-b124302ea126 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:23.428712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9ee23-4938-4850-ab03-1f60bce81175 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:23.429071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9ee23-4938-4850-ab03-1f60bce81175 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:23.431108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9ee23-4938-4850-ab03-1f60bce81175 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/users/902022469b5b49ae8ee86d24eed9da49` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:23.433931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9ee23-4938-4850-ab03-1f60bce81175 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:delete_user(user_id=902022469b5b49ae8ee86d24eed9da49)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:23.438778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9ee23-4938-4850-ab03-1f60bce81175 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:23.460944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd849512-19d7-44c7-8258-b124302ea126 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:23.473996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec78fc0-368c-4ce1-8d30-91b0a630f053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:23.474872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec78fc0-368c-4ce1-8d30-91b0a630f053 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:23.475153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec78fc0-368c-4ce1-8d30-91b0a630f053 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:23.475429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec78fc0-368c-4ce1-8d30-91b0a630f053 admin admin] PATH_INFO: `/v3/projects/3fef666fa68c4e37be220fb259e6bb42/users/c2aec45b11bd45b681d1da18202318c1/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:23.507298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec78fc0-368c-4ce1-8d30-91b0a630f053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fef666fa68c4e37be220fb259e6bb42, user_id=c2aec45b11bd45b681d1da18202318c1, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:23.513119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec78fc0-368c-4ce1-8d30-91b0a630f053 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:23.543315 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2443/4947] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:23 2022] PUT /identity/v3/projects/3fef666fa68c4e37be220fb259e6bb42/users/c2aec45b11bd45b681d1da18202318c1/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:05:23.543577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c0e9ee23-4938-4850-ab03-1f60bce81175 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '902022469b5b49ae8ee86d24eed9da49'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:23.543695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec78fc0-368c-4ce1-8d30-91b0a630f053 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:23.560389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c0e9ee23-4938-4850-ab03-1f60bce81175 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '902022469b5b49ae8ee86d24eed9da49'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:23.602444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c0e9ee23-4938-4850-ab03-1f60bce81175 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '902022469b5b49ae8ee86d24eed9da49'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:23.603941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c0e9ee23-4938-4850-ab03-1f60bce81175 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '902022469b5b49ae8ee86d24eed9da49'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:23.623585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c0e9ee23-4938-4850-ab03-1f60bce81175 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '902022469b5b49ae8ee86d24eed9da49'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:23.636913 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2505/4948] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:23 2022] DELETE /identity/v3/users/902022469b5b49ae8ee86d24eed9da49 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:23.642342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9ee23-4938-4850-ab03-1f60bce81175 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:23.645464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c0e9ee23-4938-4850-ab03-1f60bce81175 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:23.694379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec78fc0-368c-4ce1-8d30-91b0a630f053 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:23.748863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:23.820610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9ee23-4938-4850-ab03-1f60bce81175 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:23.833660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f54214-b7f4-43a4-ac5c-997f0f313400 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:23.834519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f54214-b7f4-43a4-ac5c-997f0f313400 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:23.835731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f54214-b7f4-43a4-ac5c-997f0f313400 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:23.837186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f54214-b7f4-43a4-ac5c-997f0f313400 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/85258326386c476e8811a1673c5c99bc` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:23.838307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f7fb2-2999-491c-80af-c9cdbc6cbaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:23.838944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f7fb2-2999-491c-80af-c9cdbc6cbaf1 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:23.839218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f7fb2-2999-491c-80af-c9cdbc6cbaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:23.839483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f7fb2-2999-491c-80af-c9cdbc6cbaf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:23.841835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f54214-b7f4-43a4-ac5c-997f0f313400 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:delete_group(group_id=85258326386c476e8811a1673c5c99bc)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:23.842460 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-28f54214-b7f4-43a4-ac5c-997f0f313400 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:05:23.842460 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:05:23.842460 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:05:23.843852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f7fb2-2999-491c-80af-c9cdbc6cbaf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:23.846541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f54214-b7f4-43a4-ac5c-997f0f313400 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:23.848528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f7fb2-2999-491c-80af-c9cdbc6cbaf1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:23.862547 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2444/4949] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:05:23.866031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f7fb2-2999-491c-80af-c9cdbc6cbaf1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:23.868409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-151f7fb2-2999-491c-80af-c9cdbc6cbaf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:23.897028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f7fb2-2999-491c-80af-c9cdbc6cbaf1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:23.903194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd072cc8-c574-447d-9dd3-1df4fb2da6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:23.904092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd072cc8-c574-447d-9dd3-1df4fb2da6db admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:23.904523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd072cc8-c574-447d-9dd3-1df4fb2da6db admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:23.905013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd072cc8-c574-447d-9dd3-1df4fb2da6db admin admin] PATH_INFO: `/v3/projects/3fef666fa68c4e37be220fb259e6bb42/users/c2aec45b11bd45b681d1da18202318c1/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:23.908379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd072cc8-c574-447d-9dd3-1df4fb2da6db admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fef666fa68c4e37be220fb259e6bb42, user_id=c2aec45b11bd45b681d1da18202318c1, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:23.913535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd072cc8-c574-447d-9dd3-1df4fb2da6db admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:23.916295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:23.930753 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2445/4950] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:23 2022] PUT /identity/v3/projects/3fef666fa68c4e37be220fb259e6bb42/users/c2aec45b11bd45b681d1da18202318c1/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:05:23.935950 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2506/4951] 10.222.0.17 () {58 vars in 1227 bytes} [Sat Oct 1 21:05:23 2022] DELETE /identity/v3/groups/85258326386c476e8811a1673c5c99bc => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:23.939995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd072cc8-c574-447d-9dd3-1df4fb2da6db admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:23.942566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf072bd-0a07-4312-88ea-c276293874c7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:23.942858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf072bd-0a07-4312-88ea-c276293874c7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:23.943425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf072bd-0a07-4312-88ea-c276293874c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:23.986148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-bdf072bd-0a07-4312-88ea-c276293874c7 None None] MFA Rules not processed for user `c2aec45b11bd45b681d1da18202318c1`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:23.986786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bdf072bd-0a07-4312-88ea-c276293874c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:24.025758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd072cc8-c574-447d-9dd3-1df4fb2da6db admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:24.090638 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2507/4952] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:23 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:24.103106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf072bd-0a07-4312-88ea-c276293874c7 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:24.105754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bdf072bd-0a07-4312-88ea-c276293874c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:24.105882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5064f5f-c541-4c83-9cf6-ec6b1f1db76f tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:24.106392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5064f5f-c541-4c83-9cf6-ec6b1f1db76f tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:24.106729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5064f5f-c541-4c83-9cf6-ec6b1f1db76f tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:24.107007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5064f5f-c541-4c83-9cf6-ec6b1f1db76f tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:24.107642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5064f5f-c541-4c83-9cf6-ec6b1f1db76f tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:24.109853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5064f5f-c541-4c83-9cf6-ec6b1f1db76f tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:24.131032 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2446/4953] 10.222.0.17 () {60 vars in 1147 bytes} [Sat Oct 1 21:05:23 2022] POST /identity/v3/groups => generated 325 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:05:24.134811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5064f5f-c541-4c83-9cf6-ec6b1f1db76f tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:24.149142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5064f5f-c541-4c83-9cf6-ec6b1f1db76f tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:24.155228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385ff6b-3e65-4ba8-9107-ab793a0991c7 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:24.156450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385ff6b-3e65-4ba8-9107-ab793a0991c7 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:24.157305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385ff6b-3e65-4ba8-9107-ab793a0991c7 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:24.157305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385ff6b-3e65-4ba8-9107-ab793a0991c7 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:24.157617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385ff6b-3e65-4ba8-9107-ab793a0991c7 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:24.159740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385ff6b-3e65-4ba8-9107-ab793a0991c7 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:24.184575 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2447/4954] 10.222.0.17 () {60 vars in 1147 bytes} [Sat Oct 1 21:05:24 2022] POST /identity/v3/groups => generated 327 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:24.185447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385ff6b-3e65-4ba8-9107-ab793a0991c7 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:24.196296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385ff6b-3e65-4ba8-9107-ab793a0991c7 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:24.198271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf072bd-0a07-4312-88ea-c276293874c7 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:24.201065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf75a34-7694-4b6e-9c33-3114eaf6f5c4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:24.201665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf75a34-7694-4b6e-9c33-3114eaf6f5c4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:24.201942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf75a34-7694-4b6e-9c33-3114eaf6f5c4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:24.202798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf75a34-7694-4b6e-9c33-3114eaf6f5c4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:24.202935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba721748-b05e-4f03-aaba-8d4fc82bdeb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:24.203478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf75a34-7694-4b6e-9c33-3114eaf6f5c4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:24.203603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba721748-b05e-4f03-aaba-8d4fc82bdeb2 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:24.203782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba721748-b05e-4f03-aaba-8d4fc82bdeb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:24.204060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba721748-b05e-4f03-aaba-8d4fc82bdeb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:24.206020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf75a34-7694-4b6e-9c33-3114eaf6f5c4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:24.214172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba721748-b05e-4f03-aaba-8d4fc82bdeb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:24.215642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba721748-b05e-4f03-aaba-8d4fc82bdeb2 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:24.230225 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2448/4955] 10.222.0.17 () {60 vars in 1147 bytes} [Sat Oct 1 21:05:24 2022] POST /identity/v3/groups => generated 326 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:05:24.230454 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2508/4956] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:05:24.231436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf75a34-7694-4b6e-9c33-3114eaf6f5c4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:24.241645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf75a34-7694-4b6e-9c33-3114eaf6f5c4 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:24.247270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5a6d3-92fa-4a84-b299-55143eb60987 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:24.247894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5a6d3-92fa-4a84-b299-55143eb60987 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:24.248183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5a6d3-92fa-4a84-b299-55143eb60987 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:24.248475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5a6d3-92fa-4a84-b299-55143eb60987 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:24.249635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5a6d3-92fa-4a84-b299-55143eb60987 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:list_groups()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:24.250828 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-ade5a6d3-92fa-4a84-b299-55143eb60987 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_groups": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:05:24.250828 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:05:24.250828 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:05:24.252032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5a6d3-92fa-4a84-b299-55143eb60987 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:24.260855 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2449/4957] 10.222.0.17 () {58 vars in 1125 bytes} [Sat Oct 1 21:05:24 2022] GET /identity/v3/groups => generated 1488 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:24.263802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba721748-b05e-4f03-aaba-8d4fc82bdeb2 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:24.281530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba721748-b05e-4f03-aaba-8d4fc82bdeb2 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:24.288488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa55660-dade-47b7-b9b2-3a7a6b808c91 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:24.289131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa55660-dade-47b7-b9b2-3a7a6b808c91 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:24.289415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa55660-dade-47b7-b9b2-3a7a6b808c91 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:24.289694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa55660-dade-47b7-b9b2-3a7a6b808c91 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/d0a9328838b2452887ee9075a375604a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:24.295691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa55660-dade-47b7-b9b2-3a7a6b808c91 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:delete_group(group_id=d0a9328838b2452887ee9075a375604a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:24.298621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa55660-dade-47b7-b9b2-3a7a6b808c91 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:24.370641 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2509/4958] 10.222.0.17 () {58 vars in 1227 bytes} [Sat Oct 1 21:05:24 2022] DELETE /identity/v3/groups/d0a9328838b2452887ee9075a375604a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:24.375563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5a6d3-92fa-4a84-b299-55143eb60987 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:24.394615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5a6d3-92fa-4a84-b299-55143eb60987 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:24.416607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:24.536153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d82c9c-cbee-4eae-bdfe-dd3013b4e82d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:24.536983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d82c9c-cbee-4eae-bdfe-dd3013b4e82d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:24.537870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d82c9c-cbee-4eae-bdfe-dd3013b4e82d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:24.537870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d82c9c-cbee-4eae-bdfe-dd3013b4e82d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/e042b8f8429a4f188266f90b6d7b5891` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:24.564650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d82c9c-cbee-4eae-bdfe-dd3013b4e82d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:delete_group(group_id=e042b8f8429a4f188266f90b6d7b5891)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:24.566890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d82c9c-cbee-4eae-bdfe-dd3013b4e82d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:24.652613 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2450/4959] 10.222.0.17 () {58 vars in 1227 bytes} [Sat Oct 1 21:05:24 2022] DELETE /identity/v3/groups/e042b8f8429a4f188266f90b6d7b5891 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:24.654549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa55660-dade-47b7-b9b2-3a7a6b808c91 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:24.671779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa55660-dade-47b7-b9b2-3a7a6b808c91 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:24.718601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:24.771461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bb5b1-aacd-4e88-832d-d5966ff7288b tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:24.772470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bb5b1-aacd-4e88-832d-d5966ff7288b tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:24.772797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bb5b1-aacd-4e88-832d-d5966ff7288b tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:24.773095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bb5b1-aacd-4e88-832d-d5966ff7288b tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/67ad0e51e2774ab789f80785e4e45caf` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:24.782442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33bb5b1-aacd-4e88-832d-d5966ff7288b tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:delete_group(group_id=67ad0e51e2774ab789f80785e4e45caf)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:24.784770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33bb5b1-aacd-4e88-832d-d5966ff7288b tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:24.851073 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2510/4960] 10.222.0.17 () {58 vars in 1227 bytes} [Sat Oct 1 21:05:24 2022] DELETE /identity/v3/groups/67ad0e51e2774ab789f80785e4e45caf => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:05:24.856801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d82c9c-cbee-4eae-bdfe-dd3013b4e82d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:24.870184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d82c9c-cbee-4eae-bdfe-dd3013b4e82d tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:24.961217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ba5c7-f69e-4799-9a88-a5333c3d0430 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:24.961844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ba5c7-f69e-4799-9a88-a5333c3d0430 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:24.962129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ba5c7-f69e-4799-9a88-a5333c3d0430 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:24.962410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ba5c7-f69e-4799-9a88-a5333c3d0430 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:24.963090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ba5c7-f69e-4799-9a88-a5333c3d0430 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:24.964980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ba5c7-f69e-4799-9a88-a5333c3d0430 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:24.995408 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2451/4961] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:24 2022] POST /identity/v3/users => generated 321 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:25.000045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bb5b1-aacd-4e88-832d-d5966ff7288b tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:25.011920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bb5b1-aacd-4e88-832d-d5966ff7288b tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:25.017301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596cc8d-6b1f-4bd2-961c-31b87380853c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:25.018082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596cc8d-6b1f-4bd2-961c-31b87380853c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:25.018432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596cc8d-6b1f-4bd2-961c-31b87380853c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:25.018810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596cc8d-6b1f-4bd2-961c-31b87380853c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:25.019508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596cc8d-6b1f-4bd2-961c-31b87380853c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:25.021535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596cc8d-6b1f-4bd2-961c-31b87380853c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:25.055256 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2511/4962] 10.222.0.17 () {60 vars in 1147 bytes} [Sat Oct 1 21:05:24 2022] POST /identity/v3/groups => generated 326 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:25.059737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ba5c7-f69e-4799-9a88-a5333c3d0430 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:25.073342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ba5c7-f69e-4799-9a88-a5333c3d0430 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:25.078654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09da243-f78d-465e-a8d6-c93ab9908e6c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:25.079424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09da243-f78d-465e-a8d6-c93ab9908e6c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:25.079732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09da243-f78d-465e-a8d6-c93ab9908e6c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:25.080001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09da243-f78d-465e-a8d6-c93ab9908e6c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/cc3e10dc3acd4d57a58e15510363b650/users/03ef55d649bf41a2a6e54f5da0930b33` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:25.106247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09da243-f78d-465e-a8d6-c93ab9908e6c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=cc3e10dc3acd4d57a58e15510363b650, user_id=03ef55d649bf41a2a6e54f5da0930b33)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:25.109139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09da243-f78d-465e-a8d6-c93ab9908e6c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:25.143107 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2452/4963] 10.222.0.17 () {60 vars in 1360 bytes} [Sat Oct 1 21:05:25 2022] PUT /identity/v3/groups/cc3e10dc3acd4d57a58e15510363b650/users/03ef55d649bf41a2a6e54f5da0930b33 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:25.144352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596cc8d-6b1f-4bd2-961c-31b87380853c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:25.164887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596cc8d-6b1f-4bd2-961c-31b87380853c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:25.276767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13304a87-b729-4d0f-8d1b-d89799e77187 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:25.277472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13304a87-b729-4d0f-8d1b-d89799e77187 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:25.277472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13304a87-b729-4d0f-8d1b-d89799e77187 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:25.278068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13304a87-b729-4d0f-8d1b-d89799e77187 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:25.278977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13304a87-b729-4d0f-8d1b-d89799e77187 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:25.282871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13304a87-b729-4d0f-8d1b-d89799e77187 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:25.311067 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2512/4964] 10.222.0.17 () {60 vars in 1147 bytes} [Sat Oct 1 21:05:25 2022] POST /identity/v3/groups => generated 326 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:25.315241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09da243-f78d-465e-a8d6-c93ab9908e6c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:25.331165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09da243-f78d-465e-a8d6-c93ab9908e6c tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:25.340966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18df627-02cb-4ec4-98d4-82b02077525b tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:25.342115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18df627-02cb-4ec4-98d4-82b02077525b tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:25.342535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18df627-02cb-4ec4-98d4-82b02077525b tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:25.343906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18df627-02cb-4ec4-98d4-82b02077525b tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/e43e63c20a4b446b81b5ce8148f523e5/users/03ef55d649bf41a2a6e54f5da0930b33` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:25.356899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18df627-02cb-4ec4-98d4-82b02077525b tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=e43e63c20a4b446b81b5ce8148f523e5, user_id=03ef55d649bf41a2a6e54f5da0930b33)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:25.365037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18df627-02cb-4ec4-98d4-82b02077525b tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:25.409336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:25.436017 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2453/4965] 10.222.0.17 () {60 vars in 1360 bytes} [Sat Oct 1 21:05:25 2022] PUT /identity/v3/groups/e43e63c20a4b446b81b5ce8148f523e5/users/03ef55d649bf41a2a6e54f5da0930b33 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:25.440698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13304a87-b729-4d0f-8d1b-d89799e77187 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:25.469911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13304a87-b729-4d0f-8d1b-d89799e77187 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:25.555363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18df627-02cb-4ec4-98d4-82b02077525b tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:25.558039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f18df627-02cb-4ec4-98d4-82b02077525b tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:25.565738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73343b-4a25-43e2-b020-6aba3992a0ba tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:25.567007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73343b-4a25-43e2-b020-6aba3992a0ba tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:25.567269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73343b-4a25-43e2-b020-6aba3992a0ba tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:25.567555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73343b-4a25-43e2-b020-6aba3992a0ba tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/users/03ef55d649bf41a2a6e54f5da0930b33/groups` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:25.568804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73343b-4a25-43e2-b020-6aba3992a0ba tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:list_groups_for_user(user_id=03ef55d649bf41a2a6e54f5da0930b33)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:25.571813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73343b-4a25-43e2-b020-6aba3992a0ba tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:25.582734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:25.597706 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2513/4966] 10.222.0.17 () {58 vars in 1242 bytes} [Sat Oct 1 21:05:25 2022] GET /identity/v3/users/03ef55d649bf41a2a6e54f5da0930b33/groups => generated 838 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:25.603703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73343b-4a25-43e2-b020-6aba3992a0ba tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:25.615683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73343b-4a25-43e2-b020-6aba3992a0ba tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:25.622856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec330c11-afb6-483e-a1e7-e1631df83854 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:25.623655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec330c11-afb6-483e-a1e7-e1631df83854 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:25.624818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec330c11-afb6-483e-a1e7-e1631df83854 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:25.625141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec330c11-afb6-483e-a1e7-e1631df83854 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/e43e63c20a4b446b81b5ce8148f523e5` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:25.626613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec330c11-afb6-483e-a1e7-e1631df83854 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:delete_group(group_id=e43e63c20a4b446b81b5ce8148f523e5)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:25.628971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec330c11-afb6-483e-a1e7-e1631df83854 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:25.656297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18df627-02cb-4ec4-98d4-82b02077525b tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:25.662659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182359e4-fec3-417c-b9c2-9d1db10b63bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:25.663102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182359e4-fec3-417c-b9c2-9d1db10b63bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:25.663494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182359e4-fec3-417c-b9c2-9d1db10b63bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:25.663877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182359e4-fec3-417c-b9c2-9d1db10b63bc admin admin] PATH_INFO: `/v3/users/4b540aab3baf4bf7b28eb8a7e9c37eb0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:25.665708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182359e4-fec3-417c-b9c2-9d1db10b63bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b540aab3baf4bf7b28eb8a7e9c37eb0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:25.667854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182359e4-fec3-417c-b9c2-9d1db10b63bc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:25.712006 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2514/4967] 10.222.0.17 () {58 vars in 1227 bytes} [Sat Oct 1 21:05:25 2022] DELETE /identity/v3/groups/e43e63c20a4b446b81b5ce8148f523e5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:25.713469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec330c11-afb6-483e-a1e7-e1631df83854 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:25.741234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-182359e4-fec3-417c-b9c2-9d1db10b63bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b540aab3baf4bf7b28eb8a7e9c37eb0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:25.742207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-182359e4-fec3-417c-b9c2-9d1db10b63bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b540aab3baf4bf7b28eb8a7e9c37eb0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:25.743065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-182359e4-fec3-417c-b9c2-9d1db10b63bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b540aab3baf4bf7b28eb8a7e9c37eb0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:25.756076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-182359e4-fec3-417c-b9c2-9d1db10b63bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b540aab3baf4bf7b28eb8a7e9c37eb0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:25.777427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-182359e4-fec3-417c-b9c2-9d1db10b63bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b540aab3baf4bf7b28eb8a7e9c37eb0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:25.791823 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2454/4968] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:25 2022] DELETE /identity/v3/users/4b540aab3baf4bf7b28eb8a7e9c37eb0 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:25.797782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111156c-7fbd-4a02-b696-34c96b6d3526 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:25.797782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111156c-7fbd-4a02-b696-34c96b6d3526 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:25.798056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111156c-7fbd-4a02-b696-34c96b6d3526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:25.819605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1111156c-7fbd-4a02-b696-34c96b6d3526 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:25.821031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1111156c-7fbd-4a02-b696-34c96b6d3526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:25.824691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec330c11-afb6-483e-a1e7-e1631df83854 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:25.908654 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2455/4969] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:05:25.919536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111156c-7fbd-4a02-b696-34c96b6d3526 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:25.920568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a76ef-e44e-4087-9153-011d0ae901af tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:25.921238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a76ef-e44e-4087-9153-011d0ae901af tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:25.921518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a76ef-e44e-4087-9153-011d0ae901af tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:25.921792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a76ef-e44e-4087-9153-011d0ae901af tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/groups/cc3e10dc3acd4d57a58e15510363b650` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:25.923220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1111156c-7fbd-4a02-b696-34c96b6d3526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:25.924893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a76ef-e44e-4087-9153-011d0ae901af tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:delete_group(group_id=cc3e10dc3acd4d57a58e15510363b650)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:25.928457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a76ef-e44e-4087-9153-011d0ae901af tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:25.971644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:26.014338 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2515/4970] 10.222.0.17 () {58 vars in 1227 bytes} [Sat Oct 1 21:05:25 2022] DELETE /identity/v3/groups/cc3e10dc3acd4d57a58e15510363b650 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:05:26.019318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a76ef-e44e-4087-9153-011d0ae901af tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:26.021900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1e4a76ef-e44e-4087-9153-011d0ae901af tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:26.048510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:26.080855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111156c-7fbd-4a02-b696-34c96b6d3526 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:26.132953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a76ef-e44e-4087-9153-011d0ae901af tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:26.149258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d153c-bb43-4322-87f1-c0d986c15ff5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:26.151547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d153c-bb43-4322-87f1-c0d986c15ff5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:26.151937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d153c-bb43-4322-87f1-c0d986c15ff5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:26.153461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d153c-bb43-4322-87f1-c0d986c15ff5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/users/03ef55d649bf41a2a6e54f5da0930b33` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:26.155121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d153c-bb43-4322-87f1-c0d986c15ff5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:delete_user(user_id=03ef55d649bf41a2a6e54f5da0930b33)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:26.160279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d153c-bb43-4322-87f1-c0d986c15ff5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:26.185873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07802384-ce78-46b0-8a79-44a2297df8af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:26.187389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07802384-ce78-46b0-8a79-44a2297df8af service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:26.187846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07802384-ce78-46b0-8a79-44a2297df8af service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:26.188289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07802384-ce78-46b0-8a79-44a2297df8af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:26.237595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c78d153c-bb43-4322-87f1-c0d986c15ff5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03ef55d649bf41a2a6e54f5da0930b33'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:26.247431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c78d153c-bb43-4322-87f1-c0d986c15ff5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03ef55d649bf41a2a6e54f5da0930b33'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:26.266654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c78d153c-bb43-4322-87f1-c0d986c15ff5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03ef55d649bf41a2a6e54f5da0930b33'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:26.267879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c78d153c-bb43-4322-87f1-c0d986c15ff5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03ef55d649bf41a2a6e54f5da0930b33'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:26.279958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07802384-ce78-46b0-8a79-44a2297df8af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:26.281585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07802384-ce78-46b0-8a79-44a2297df8af service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:26.283169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c78d153c-bb43-4322-87f1-c0d986c15ff5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03ef55d649bf41a2a6e54f5da0930b33'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:26.296602 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2456/4971] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:05:26.297660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07802384-ce78-46b0-8a79-44a2297df8af service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:26.299685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-07802384-ce78-46b0-8a79-44a2297df8af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:26.308013 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2516/4972] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:26 2022] DELETE /identity/v3/users/03ef55d649bf41a2a6e54f5da0930b33 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:26.316281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d153c-bb43-4322-87f1-c0d986c15ff5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:26.318512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c78d153c-bb43-4322-87f1-c0d986c15ff5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:26.318647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07802384-ce78-46b0-8a79-44a2297df8af service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:26.408191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3bea1-9208-4d23-8e4e-f1ec56d5d091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:26.409291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3bea1-9208-4d23-8e4e-f1ec56d5d091 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:26.409291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3bea1-9208-4d23-8e4e-f1ec56d5d091 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:26.409732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3bea1-9208-4d23-8e4e-f1ec56d5d091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:26.412495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-27b3bea1-9208-4d23-8e4e-f1ec56d5d091 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:26.439416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d153c-bb43-4322-87f1-c0d986c15ff5 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:26.450453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716fa25-4dfe-4393-ad8f-a82e7981e8e2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:26.452520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716fa25-4dfe-4393-ad8f-a82e7981e8e2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:26.453278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716fa25-4dfe-4393-ad8f-a82e7981e8e2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:26.453764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716fa25-4dfe-4393-ad8f-a82e7981e8e2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/domains/4e80978a4e0746238c9b07dcefb5fd41` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:26.455971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716fa25-4dfe-4393-ad8f-a82e7981e8e2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=4e80978a4e0746238c9b07dcefb5fd41)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:26.457597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:26.461612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716fa25-4dfe-4393-ad8f-a82e7981e8e2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:26.500122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e716fa25-4dfe-4393-ad8f-a82e7981e8e2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '4e80978a4e0746238c9b07dcefb5fd41'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:26.501019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e716fa25-4dfe-4393-ad8f-a82e7981e8e2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '4e80978a4e0746238c9b07dcefb5fd41'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:26.516602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3bea1-9208-4d23-8e4e-f1ec56d5d091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:26.518252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3bea1-9208-4d23-8e4e-f1ec56d5d091 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:26.525179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:26.555040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e716fa25-4dfe-4393-ad8f-a82e7981e8e2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '4e80978a4e0746238c9b07dcefb5fd41'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:26.555723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e716fa25-4dfe-4393-ad8f-a82e7981e8e2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '4e80978a4e0746238c9b07dcefb5fd41'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:26.561028 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2457/4974] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:05:26.561254 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2517/4974] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:05:26 2022] PATCH /identity/v3/domains/4e80978a4e0746238c9b07dcefb5fd41 => generated 277 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:26.565584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3bea1-9208-4d23-8e4e-f1ec56d5d091 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:26.577825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3bea1-9208-4d23-8e4e-f1ec56d5d091 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:26.648711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954c841-dbf2-48d7-bca7-836bf8915844 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b356c625610471da308bb17d17b940e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a49a297357e94b7a8ba794d60798b48e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:26.649460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954c841-dbf2-48d7-bca7-836bf8915844 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:26.649815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954c841-dbf2-48d7-bca7-836bf8915844 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:26.650161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954c841-dbf2-48d7-bca7-836bf8915844 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] PATH_INFO: `/v3/domains/4e80978a4e0746238c9b07dcefb5fd41` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:26.663186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954c841-dbf2-48d7-bca7-836bf8915844 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=4e80978a4e0746238c9b07dcefb5fd41)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:26.664925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954c841-dbf2-48d7-bca7-836bf8915844 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:26.694327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d954c841-dbf2-48d7-bca7-836bf8915844 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '4e80978a4e0746238c9b07dcefb5fd41'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:26.705807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d954c841-dbf2-48d7-bca7-836bf8915844 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '4e80978a4e0746238c9b07dcefb5fd41'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:26.753672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d954c841-dbf2-48d7-bca7-836bf8915844 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e80978a4e0746238c9b07dcefb5fd41', 'request_id': 'req-d954c841-dbf2-48d7-bca7-836bf8915844'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:26.767762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d954c841-dbf2-48d7-bca7-836bf8915844 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e80978a4e0746238c9b07dcefb5fd41', 'request_id': 'req-d954c841-dbf2-48d7-bca7-836bf8915844'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:26.780835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d954c841-dbf2-48d7-bca7-836bf8915844 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e80978a4e0746238c9b07dcefb5fd41', 'request_id': 'req-d954c841-dbf2-48d7-bca7-836bf8915844'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:26.795023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d954c841-dbf2-48d7-bca7-836bf8915844 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] The token cache is being invalidate because project 4e80978a4e0746238c9b07dcefb5fd41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:26.796506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d954c841-dbf2-48d7-bca7-836bf8915844 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:26.797413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d954c841-dbf2-48d7-bca7-836bf8915844 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:26.802222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:26.812426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d954c841-dbf2-48d7-bca7-836bf8915844 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '4e80978a4e0746238c9b07dcefb5fd41', 'request_id': 'req-d954c841-dbf2-48d7-bca7-836bf8915844'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:26.814316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d954c841-dbf2-48d7-bca7-836bf8915844 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '4e80978a4e0746238c9b07dcefb5fd41', 'request_id': 'req-d954c841-dbf2-48d7-bca7-836bf8915844'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:26.825403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d954c841-dbf2-48d7-bca7-836bf8915844 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '4e80978a4e0746238c9b07dcefb5fd41', 'request_id': 'req-d954c841-dbf2-48d7-bca7-836bf8915844'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:26.838679 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2458/4975] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:05:26 2022] DELETE /identity/v3/domains/4e80978a4e0746238c9b07dcefb5fd41 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:26.842562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716fa25-4dfe-4393-ad8f-a82e7981e8e2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:26.859412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e716fa25-4dfe-4393-ad8f-a82e7981e8e2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:26.972392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716fa25-4dfe-4393-ad8f-a82e7981e8e2 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:26.996487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954c841-dbf2-48d7-bca7-836bf8915844 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:26.998883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d954c841-dbf2-48d7-bca7-836bf8915844 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:27.078811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e900b-2caa-4cad-9de5-d37c2968c149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:27.079402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e900b-2caa-4cad-9de5-d37c2968c149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:27.079766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e900b-2caa-4cad-9de5-d37c2968c149 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:27.080146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e900b-2caa-4cad-9de5-d37c2968c149 admin admin] PATH_INFO: `/v3/users/97e2521ab50645fa9a6fbc7d76751327` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:27.082286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e900b-2caa-4cad-9de5-d37c2968c149 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97e2521ab50645fa9a6fbc7d76751327)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:27.084545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e900b-2caa-4cad-9de5-d37c2968c149 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:27.144693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954c841-dbf2-48d7-bca7-836bf8915844 tempest-GroupsV3TestJSON-1449689585 tempest-GroupsV3TestJSON-1449689585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:27.149136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a4e900b-2caa-4cad-9de5-d37c2968c149 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97e2521ab50645fa9a6fbc7d76751327'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:27.151835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f45d2-1559-4569-8a83-d77dcf26b742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:27.152958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f45d2-1559-4569-8a83-d77dcf26b742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:27.153489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f45d2-1559-4569-8a83-d77dcf26b742 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:27.154149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f45d2-1559-4569-8a83-d77dcf26b742 admin admin] PATH_INFO: `/v3/users/b0cbab34b69f481989e118bd0508deb4` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:27.156288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f45d2-1559-4569-8a83-d77dcf26b742 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0cbab34b69f481989e118bd0508deb4)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:27.159173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f45d2-1559-4569-8a83-d77dcf26b742 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:27.169722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a4e900b-2caa-4cad-9de5-d37c2968c149 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97e2521ab50645fa9a6fbc7d76751327'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:27.189660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a4e900b-2caa-4cad-9de5-d37c2968c149 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97e2521ab50645fa9a6fbc7d76751327'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:27.190552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a4e900b-2caa-4cad-9de5-d37c2968c149 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97e2521ab50645fa9a6fbc7d76751327'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:27.203117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a4e900b-2caa-4cad-9de5-d37c2968c149 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97e2521ab50645fa9a6fbc7d76751327'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:27.218569 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2518/4976] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:26 2022] DELETE /identity/v3/users/97e2521ab50645fa9a6fbc7d76751327 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:27.222010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54685f39-2fe3-48c3-b166-88b591ce3ca9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:27.222304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54685f39-2fe3-48c3-b166-88b591ce3ca9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:27.224886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54685f39-2fe3-48c3-b166-88b591ce3ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:27.253586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-54685f39-2fe3-48c3-b166-88b591ce3ca9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:27.254182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cc7f45d2-1559-4569-8a83-d77dcf26b742 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0cbab34b69f481989e118bd0508deb4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:27.255088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-54685f39-2fe3-48c3-b166-88b591ce3ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:27.255553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cc7f45d2-1559-4569-8a83-d77dcf26b742 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0cbab34b69f481989e118bd0508deb4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:27.256510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cc7f45d2-1559-4569-8a83-d77dcf26b742 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0cbab34b69f481989e118bd0508deb4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:27.269941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cc7f45d2-1559-4569-8a83-d77dcf26b742 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0cbab34b69f481989e118bd0508deb4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:27.289447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cc7f45d2-1559-4569-8a83-d77dcf26b742 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0cbab34b69f481989e118bd0508deb4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:27.298230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:27.341685 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2459/4977] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:26 2022] DELETE /identity/v3/users/b0cbab34b69f481989e118bd0508deb4 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:27.343352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30e965-3baf-4938-ae0a-74faeb599ad4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:27.344333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30e965-3baf-4938-ae0a-74faeb599ad4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:27.344333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30e965-3baf-4938-ae0a-74faeb599ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:27.411946 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2519/4978] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:05:27.418523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3b30e965-3baf-4938-ae0a-74faeb599ad4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:27.420828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3b30e965-3baf-4938-ae0a-74faeb599ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:27.424662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54685f39-2fe3-48c3-b166-88b591ce3ca9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:27.428172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-54685f39-2fe3-48c3-b166-88b591ce3ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:27.537319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54685f39-2fe3-48c3-b166-88b591ce3ca9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:27.616691 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2460/4979] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:05:27.617687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30e965-3baf-4938-ae0a-74faeb599ad4 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:27.619129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3b30e965-3baf-4938-ae0a-74faeb599ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:27.629405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745da07-bccd-4c75-96ee-709c87ff18a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:27.630136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745da07-bccd-4c75-96ee-709c87ff18a8 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:27.630423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745da07-bccd-4c75-96ee-709c87ff18a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:27.630765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745da07-bccd-4c75-96ee-709c87ff18a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:27.636576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30e965-3baf-4938-ae0a-74faeb599ad4 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:27.642238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a470b3-f182-477c-af76-92fa4278525f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:27.643792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a470b3-f182-477c-af76-92fa4278525f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:27.644036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a470b3-f182-477c-af76-92fa4278525f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:27.644305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a470b3-f182-477c-af76-92fa4278525f admin admin] PATH_INFO: `/v3/projects/d591215350fc4c34939709505679a996` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:27.645469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a470b3-f182-477c-af76-92fa4278525f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d591215350fc4c34939709505679a996)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:27.647495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a470b3-f182-477c-af76-92fa4278525f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:27.649204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:27.660039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745da07-bccd-4c75-96ee-709c87ff18a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:27.662052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745da07-bccd-4c75-96ee-709c87ff18a8 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:27.690837 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2520/4980] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:05:27.691767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745da07-bccd-4c75-96ee-709c87ff18a8 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:27.698134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e745da07-bccd-4c75-96ee-709c87ff18a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:27.716632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c9a470b3-f182-477c-af76-92fa4278525f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd591215350fc4c34939709505679a996', 'request_id': 'req-c9a470b3-f182-477c-af76-92fa4278525f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:27.721468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745da07-bccd-4c75-96ee-709c87ff18a8 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:27.731811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c9a470b3-f182-477c-af76-92fa4278525f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd591215350fc4c34939709505679a996', 'request_id': 'req-c9a470b3-f182-477c-af76-92fa4278525f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:27.756195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c9a470b3-f182-477c-af76-92fa4278525f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd591215350fc4c34939709505679a996', 'request_id': 'req-c9a470b3-f182-477c-af76-92fa4278525f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:27.776076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c9a470b3-f182-477c-af76-92fa4278525f admin admin] The token cache is being invalidate because project d591215350fc4c34939709505679a996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:27.776652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c9a470b3-f182-477c-af76-92fa4278525f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:27.777341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c9a470b3-f182-477c-af76-92fa4278525f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:27.784456 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2461/4981] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:27 2022] DELETE /identity/v3/projects/d591215350fc4c34939709505679a996 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:27.787650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac20802-f0e1-4bd4-be9c-44da5c4fa381 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:27.788636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac20802-f0e1-4bd4-be9c-44da5c4fa381 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:27.788925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac20802-f0e1-4bd4-be9c-44da5c4fa381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:27.813287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a45d4-a568-452b-a728-c05c23cc066b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:27.813667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fac20802-f0e1-4bd4-be9c-44da5c4fa381 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:27.814815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a45d4-a568-452b-a728-c05c23cc066b service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:27.815334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fac20802-f0e1-4bd4-be9c-44da5c4fa381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:27.815947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a45d4-a568-452b-a728-c05c23cc066b service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:27.817914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a45d4-a568-452b-a728-c05c23cc066b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:27.825105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a06a45d4-a568-452b-a728-c05c23cc066b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:27.910363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:27.938622 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2462/4982] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:27.940533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0837ab-45fc-448e-8de8-7a89c8affbba None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:27.940799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0837ab-45fc-448e-8de8-7a89c8affbba None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:27.942108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0837ab-45fc-448e-8de8-7a89c8affbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:27.975790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a45d4-a568-452b-a728-c05c23cc066b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:27.976878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a45d4-a568-452b-a728-c05c23cc066b service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:27.977252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ca0837ab-45fc-448e-8de8-7a89c8affbba None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:27.978993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ca0837ab-45fc-448e-8de8-7a89c8affbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:27.990953 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2521/4983] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:05:28.005440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a45d4-a568-452b-a728-c05c23cc066b service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:28.007564 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2463/4984] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:28.045133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a45d4-a568-452b-a728-c05c23cc066b service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:28.064776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa009bb-e92b-43c6-be86-a298bd9394ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:28.065408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa009bb-e92b-43c6-be86-a298bd9394ad admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:28.065690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa009bb-e92b-43c6-be86-a298bd9394ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:28.066388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa009bb-e92b-43c6-be86-a298bd9394ad admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:28.066674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa009bb-e92b-43c6-be86-a298bd9394ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:05:28.067025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa009bb-e92b-43c6-be86-a298bd9394ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:28.069543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa009bb-e92b-43c6-be86-a298bd9394ad admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:28.086690 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2522/4985] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:28.092775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0837ab-45fc-448e-8de8-7a89c8affbba None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:28.105356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0837ab-45fc-448e-8de8-7a89c8affbba None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:28.110973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a3794-66ef-4b6c-ace9-fd1f3d81b728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:28.111848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a3794-66ef-4b6c-ace9-fd1f3d81b728 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:28.111848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a3794-66ef-4b6c-ace9-fd1f3d81b728 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:28.112336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a3794-66ef-4b6c-ace9-fd1f3d81b728 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:28.112970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a3794-66ef-4b6c-ace9-fd1f3d81b728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:28.116415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a3794-66ef-4b6c-ace9-fd1f3d81b728 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:28.182882 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2464/4986] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:28 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:28.187075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa009bb-e92b-43c6-be86-a298bd9394ad admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:28.202606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa009bb-e92b-43c6-be86-a298bd9394ad admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:28.280590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941b157-7b27-46bf-8c6f-445667291aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:28.281289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941b157-7b27-46bf-8c6f-445667291aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:28.281569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941b157-7b27-46bf-8c6f-445667291aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:28.281841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941b157-7b27-46bf-8c6f-445667291aa1 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:28.282638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0941b157-7b27-46bf-8c6f-445667291aa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:28.284589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0941b157-7b27-46bf-8c6f-445667291aa1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:28.316769 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2523/4987] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:28 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:28.321600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a3794-66ef-4b6c-ace9-fd1f3d81b728 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:28.337526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a3794-66ef-4b6c-ace9-fd1f3d81b728 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:28.344902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2001aa7-e07f-4268-a914-1dd53e135a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:28.345145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2001aa7-e07f-4268-a914-1dd53e135a09 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:28.345378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2001aa7-e07f-4268-a914-1dd53e135a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:28.345807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2001aa7-e07f-4268-a914-1dd53e135a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:28.346980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2001aa7-e07f-4268-a914-1dd53e135a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:28.349191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2001aa7-e07f-4268-a914-1dd53e135a09 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:28.365460 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2465/4988] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:28.369201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941b157-7b27-46bf-8c6f-445667291aa1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:28.382175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941b157-7b27-46bf-8c6f-445667291aa1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:28.393051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f78746-f18d-4309-afb3-391aaceae602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:28.393833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f78746-f18d-4309-afb3-391aaceae602 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:28.394352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f78746-f18d-4309-afb3-391aaceae602 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:28.394753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f78746-f18d-4309-afb3-391aaceae602 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:28.396017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f78746-f18d-4309-afb3-391aaceae602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:28.400575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f78746-f18d-4309-afb3-391aaceae602 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:28.412929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2001aa7-e07f-4268-a914-1dd53e135a09 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:28.415034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a2001aa7-e07f-4268-a914-1dd53e135a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:28.430428 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2524/4989] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:05:28.438745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2001aa7-e07f-4268-a914-1dd53e135a09 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:28.439758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f78746-f18d-4309-afb3-391aaceae602 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:28.453576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38c7e0-9421-40b3-8f71-3def5c7e0f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:28.454334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38c7e0-9421-40b3-8f71-3def5c7e0f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:28.454736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38c7e0-9421-40b3-8f71-3def5c7e0f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:28.455800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38c7e0-9421-40b3-8f71-3def5c7e0f08 admin admin] PATH_INFO: `/v3/projects/683d68d5332246c1b9d4415348a85200` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:28.456821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b38c7e0-9421-40b3-8f71-3def5c7e0f08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=683d68d5332246c1b9d4415348a85200)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:28.459844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f78746-f18d-4309-afb3-391aaceae602 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:28.460824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b38c7e0-9421-40b3-8f71-3def5c7e0f08 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:28.465371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae3d2ad-53e9-4530-a5f0-be8bbad9af53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:28.466086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3d2ad-53e9-4530-a5f0-be8bbad9af53 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:28.466366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3d2ad-53e9-4530-a5f0-be8bbad9af53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:28.467355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3d2ad-53e9-4530-a5f0-be8bbad9af53 admin admin] PATH_INFO: `/v3/projects/e3e9e56444d448cb9219affd1e0e41bc/users/40a38d38fee342fb84f77758525b591b/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:28.488525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae3d2ad-53e9-4530-a5f0-be8bbad9af53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e9e56444d448cb9219affd1e0e41bc, user_id=40a38d38fee342fb84f77758525b591b, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:28.491637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae3d2ad-53e9-4530-a5f0-be8bbad9af53 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:28.526549 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2525/4990] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:28 2022] PUT /identity/v3/projects/e3e9e56444d448cb9219affd1e0e41bc/users/40a38d38fee342fb84f77758525b591b/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:28.532996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536b38d-d0c9-4517-b3eb-1b69be1fb2ba None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:28.533504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536b38d-d0c9-4517-b3eb-1b69be1fb2ba None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:28.533798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536b38d-d0c9-4517-b3eb-1b69be1fb2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:28.569878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8536b38d-d0c9-4517-b3eb-1b69be1fb2ba None None] MFA Rules not processed for user `40a38d38fee342fb84f77758525b591b`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:28.571620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8536b38d-d0c9-4517-b3eb-1b69be1fb2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:28.602909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:28.614458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2b38c7e0-9421-40b3-8f71-3def5c7e0f08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '683d68d5332246c1b9d4415348a85200', 'request_id': 'req-2b38c7e0-9421-40b3-8f71-3def5c7e0f08'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:28.635685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2b38c7e0-9421-40b3-8f71-3def5c7e0f08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '683d68d5332246c1b9d4415348a85200', 'request_id': 'req-2b38c7e0-9421-40b3-8f71-3def5c7e0f08'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:28.647920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2b38c7e0-9421-40b3-8f71-3def5c7e0f08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '683d68d5332246c1b9d4415348a85200', 'request_id': 'req-2b38c7e0-9421-40b3-8f71-3def5c7e0f08'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:28.665037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2b38c7e0-9421-40b3-8f71-3def5c7e0f08 admin admin] The token cache is being invalidate because project 683d68d5332246c1b9d4415348a85200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:28.665638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2b38c7e0-9421-40b3-8f71-3def5c7e0f08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:28.666319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2b38c7e0-9421-40b3-8f71-3def5c7e0f08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:28.673312 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2466/4991] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:28 2022] DELETE /identity/v3/projects/683d68d5332246c1b9d4415348a85200 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:28.674583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38c7e0-9421-40b3-8f71-3def5c7e0f08 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:28.677055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2b38c7e0-9421-40b3-8f71-3def5c7e0f08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:28.678877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:28.686167 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2526/4992] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:05:28.691522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536b38d-d0c9-4517-b3eb-1b69be1fb2ba None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:28.788586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38c7e0-9421-40b3-8f71-3def5c7e0f08 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:28.795768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c04932-90de-48e3-9353-93116200d859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:28.797417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c04932-90de-48e3-9353-93116200d859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:28.798019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c04932-90de-48e3-9353-93116200d859 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:28.798691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c04932-90de-48e3-9353-93116200d859 admin admin] PATH_INFO: `/v3/users/3b356c625610471da308bb17d17b940e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:28.801273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c04932-90de-48e3-9353-93116200d859 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b356c625610471da308bb17d17b940e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:28.804217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c04932-90de-48e3-9353-93116200d859 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:28.835838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536b38d-d0c9-4517-b3eb-1b69be1fb2ba None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:28.851472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecf50e-b9fd-46f6-9475-72ba8a5c13e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:28.852171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecf50e-b9fd-46f6-9475-72ba8a5c13e0 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:28.852517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecf50e-b9fd-46f6-9475-72ba8a5c13e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:28.852841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecf50e-b9fd-46f6-9475-72ba8a5c13e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:28.853506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecf50e-b9fd-46f6-9475-72ba8a5c13e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:28.855823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecf50e-b9fd-46f6-9475-72ba8a5c13e0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:28.881206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34c04932-90de-48e3-9353-93116200d859 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b356c625610471da308bb17d17b940e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:28.882122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34c04932-90de-48e3-9353-93116200d859 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b356c625610471da308bb17d17b940e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:28.882956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34c04932-90de-48e3-9353-93116200d859 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b356c625610471da308bb17d17b940e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:28.914432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34c04932-90de-48e3-9353-93116200d859 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b356c625610471da308bb17d17b940e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:28.929105 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2527/4993] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:28 2022] POST /identity/v3/projects => generated 387 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:28.934646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecf50e-b9fd-46f6-9475-72ba8a5c13e0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:28.938110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1aecf50e-b9fd-46f6-9475-72ba8a5c13e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:28.952890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34c04932-90de-48e3-9353-93116200d859 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b356c625610471da308bb17d17b940e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:29.023085 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2467/4994] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:28 2022] DELETE /identity/v3/users/3b356c625610471da308bb17d17b940e => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:29.024627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c04932-90de-48e3-9353-93116200d859 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:29.027466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-34c04932-90de-48e3-9353-93116200d859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:29.066522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecf50e-b9fd-46f6-9475-72ba8a5c13e0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:29.182039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:29.214517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8532f-3a32-4381-8d55-e207aeb0d157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:29.214517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8532f-3a32-4381-8d55-e207aeb0d157 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:29.214517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8532f-3a32-4381-8d55-e207aeb0d157 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:29.214517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8532f-3a32-4381-8d55-e207aeb0d157 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:29.215092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8532f-3a32-4381-8d55-e207aeb0d157 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:29.215215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c04932-90de-48e3-9353-93116200d859 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:29.215322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8532f-3a32-4381-8d55-e207aeb0d157 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:29.224440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253077e7-d9f9-4645-9766-678ece9b2e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:29.225114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253077e7-d9f9-4645-9766-678ece9b2e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:29.225395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253077e7-d9f9-4645-9766-678ece9b2e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:29.225872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253077e7-d9f9-4645-9766-678ece9b2e81 admin admin] PATH_INFO: `/v3/projects/b6cce17fe4f34afa9a894e632aed5126` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:29.227563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253077e7-d9f9-4645-9766-678ece9b2e81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6cce17fe4f34afa9a894e632aed5126)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:29.230470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253077e7-d9f9-4645-9766-678ece9b2e81 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:29.261817 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2528/4995] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:28 2022] POST /identity/v3/users => generated 301 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:05:29.267792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8532f-3a32-4381-8d55-e207aeb0d157 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:29.287881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8532f-3a32-4381-8d55-e207aeb0d157 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:29.295204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc1320-5f17-4282-8b74-a55fe3a9eaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:29.297858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc1320-5f17-4282-8b74-a55fe3a9eaac admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:29.298135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc1320-5f17-4282-8b74-a55fe3a9eaac admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:29.298463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc1320-5f17-4282-8b74-a55fe3a9eaac admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:29.299861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc1320-5f17-4282-8b74-a55fe3a9eaac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:29.301493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc1320-5f17-4282-8b74-a55fe3a9eaac admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:29.327764 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2529/4996] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:29.329388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc1320-5f17-4282-8b74-a55fe3a9eaac admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:29.343559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc1320-5f17-4282-8b74-a55fe3a9eaac admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:29.349345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-253077e7-d9f9-4645-9766-678ece9b2e81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6cce17fe4f34afa9a894e632aed5126', 'request_id': 'req-253077e7-d9f9-4645-9766-678ece9b2e81'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:29.354768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:29.360434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-253077e7-d9f9-4645-9766-678ece9b2e81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6cce17fe4f34afa9a894e632aed5126', 'request_id': 'req-253077e7-d9f9-4645-9766-678ece9b2e81'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:29.366735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-253077e7-d9f9-4645-9766-678ece9b2e81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6cce17fe4f34afa9a894e632aed5126', 'request_id': 'req-253077e7-d9f9-4645-9766-678ece9b2e81'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:29.400433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-253077e7-d9f9-4645-9766-678ece9b2e81 admin admin] The token cache is being invalidate because project b6cce17fe4f34afa9a894e632aed5126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:29.400433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-253077e7-d9f9-4645-9766-678ece9b2e81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:29.406248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-253077e7-d9f9-4645-9766-678ece9b2e81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:29.407864 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2468/4997] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:29 2022] DELETE /identity/v3/projects/b6cce17fe4f34afa9a894e632aed5126 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:29.412736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253077e7-d9f9-4645-9766-678ece9b2e81 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:29.415017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-253077e7-d9f9-4645-9766-678ece9b2e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:29.490308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192401d-8409-4750-988f-28c9aa3f6fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:29.491744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192401d-8409-4750-988f-28c9aa3f6fbf admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:29.492034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192401d-8409-4750-988f-28c9aa3f6fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:29.492362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192401d-8409-4750-988f-28c9aa3f6fbf admin admin] PATH_INFO: `/v3/projects/a0a8e2e9f2bd44dca5c729103e29a882/users/d7d71fbab5fa443e843b0c6573decf61/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:29.515092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192401d-8409-4750-988f-28c9aa3f6fbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a8e2e9f2bd44dca5c729103e29a882, user_id=d7d71fbab5fa443e843b0c6573decf61, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:29.518896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192401d-8409-4750-988f-28c9aa3f6fbf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:29.551153 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2530/4998] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:29 2022] PUT /identity/v3/projects/a0a8e2e9f2bd44dca5c729103e29a882/users/d7d71fbab5fa443e843b0c6573decf61/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:29.555345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cd34f-cf38-4c48-8876-bcd7cd24bdcb None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:29.555693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cd34f-cf38-4c48-8876-bcd7cd24bdcb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:29.556037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cd34f-cf38-4c48-8876-bcd7cd24bdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:29.577649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253077e7-d9f9-4645-9766-678ece9b2e81 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:29.591493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-694cd34f-cf38-4c48-8876-bcd7cd24bdcb None None] MFA Rules not processed for user `d7d71fbab5fa443e843b0c6573decf61`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:29.593520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-694cd34f-cf38-4c48-8876-bcd7cd24bdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:29.660202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd62aea-8eee-4185-98a5-679626d87f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:29.662080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd62aea-8eee-4185-98a5-679626d87f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:29.662816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd62aea-8eee-4185-98a5-679626d87f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:29.663578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd62aea-8eee-4185-98a5-679626d87f7f admin admin] PATH_INFO: `/v3/users/c2aec45b11bd45b681d1da18202318c1` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:29.665335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd62aea-8eee-4185-98a5-679626d87f7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2aec45b11bd45b681d1da18202318c1)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:29.668689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd62aea-8eee-4185-98a5-679626d87f7f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:29.696980 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2531/4999] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:29 2022] POST /identity/v3/auth/tokens => generated 3271 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:29.737743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:29.744197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cd34f-cf38-4c48-8876-bcd7cd24bdcb None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:29.745846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-694cd34f-cf38-4c48-8876-bcd7cd24bdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:29.765020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-efd62aea-8eee-4185-98a5-679626d87f7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2aec45b11bd45b681d1da18202318c1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:29.766404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-efd62aea-8eee-4185-98a5-679626d87f7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2aec45b11bd45b681d1da18202318c1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:29.770997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-efd62aea-8eee-4185-98a5-679626d87f7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2aec45b11bd45b681d1da18202318c1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:29.783894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-efd62aea-8eee-4185-98a5-679626d87f7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2aec45b11bd45b681d1da18202318c1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:29.833425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-efd62aea-8eee-4185-98a5-679626d87f7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2aec45b11bd45b681d1da18202318c1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:29.859114 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2469/5000] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:29 2022] DELETE /identity/v3/users/c2aec45b11bd45b681d1da18202318c1 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:29.862243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cd34f-cf38-4c48-8876-bcd7cd24bdcb None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:29.883416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:29.971130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84f99d-2eb4-473d-b967-dc5664e4f2c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:29.971910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84f99d-2eb4-473d-b967-dc5664e4f2c4 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:29.972197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84f99d-2eb4-473d-b967-dc5664e4f2c4 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:29.972469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84f99d-2eb4-473d-b967-dc5664e4f2c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:29.976206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cd84f99d-2eb4-473d-b967-dc5664e4f2c4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:30.089438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84f99d-2eb4-473d-b967-dc5664e4f2c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:30.091797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84f99d-2eb4-473d-b967-dc5664e4f2c4 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:30.106693 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2532/5001] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:05:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 680 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:05:30.446426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd62aea-8eee-4185-98a5-679626d87f7f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:30.446426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-efd62aea-8eee-4185-98a5-679626d87f7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:30.458203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84f99d-2eb4-473d-b967-dc5664e4f2c4 service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:30.463167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cd84f99d-2eb4-473d-b967-dc5664e4f2c4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:30.645644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd62aea-8eee-4185-98a5-679626d87f7f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:30.660979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4223d3-3a05-445c-a4f6-05f40a22ff48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:30.661634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4223d3-3a05-445c-a4f6-05f40a22ff48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:30.662298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4223d3-3a05-445c-a4f6-05f40a22ff48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:30.663848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4223d3-3a05-445c-a4f6-05f40a22ff48 admin admin] PATH_INFO: `/v3/projects/a49a297357e94b7a8ba794d60798b48e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:30.681909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4223d3-3a05-445c-a4f6-05f40a22ff48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a49a297357e94b7a8ba794d60798b48e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:30.686642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4223d3-3a05-445c-a4f6-05f40a22ff48 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:30.693902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84f99d-2eb4-473d-b967-dc5664e4f2c4 service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:30.711245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de6eed5-1349-4b82-bb2c-2923bf448a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:30.711818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6eed5-1349-4b82-bb2c-2923bf448a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:30.712091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6eed5-1349-4b82-bb2c-2923bf448a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:30.712352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6eed5-1349-4b82-bb2c-2923bf448a54 admin admin] PATH_INFO: `/v3/projects/3fef666fa68c4e37be220fb259e6bb42` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:30.716913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de6eed5-1349-4b82-bb2c-2923bf448a54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fef666fa68c4e37be220fb259e6bb42)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:30.720808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de6eed5-1349-4b82-bb2c-2923bf448a54 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:30.807251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2f4223d3-3a05-445c-a4f6-05f40a22ff48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a49a297357e94b7a8ba794d60798b48e', 'request_id': 'req-2f4223d3-3a05-445c-a4f6-05f40a22ff48'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:30.821705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2f4223d3-3a05-445c-a4f6-05f40a22ff48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a49a297357e94b7a8ba794d60798b48e', 'request_id': 'req-2f4223d3-3a05-445c-a4f6-05f40a22ff48'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:30.833534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2f4223d3-3a05-445c-a4f6-05f40a22ff48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a49a297357e94b7a8ba794d60798b48e', 'request_id': 'req-2f4223d3-3a05-445c-a4f6-05f40a22ff48'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:30.852575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2f4223d3-3a05-445c-a4f6-05f40a22ff48 admin admin] The token cache is being invalidate because project a49a297357e94b7a8ba794d60798b48e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:30.853038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2f4223d3-3a05-445c-a4f6-05f40a22ff48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:30.854118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2f4223d3-3a05-445c-a4f6-05f40a22ff48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:30.859432 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2470/5002] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:30 2022] DELETE /identity/v3/projects/a49a297357e94b7a8ba794d60798b48e => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:30.861228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecd2b0-c063-4452-913e-86badcef7268 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:30.861708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecd2b0-c063-4452-913e-86badcef7268 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:30.862148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecd2b0-c063-4452-913e-86badcef7268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:30.870560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2de6eed5-1349-4b82-bb2c-2923bf448a54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fef666fa68c4e37be220fb259e6bb42', 'request_id': 'req-2de6eed5-1349-4b82-bb2c-2923bf448a54'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:30.886053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2de6eed5-1349-4b82-bb2c-2923bf448a54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fef666fa68c4e37be220fb259e6bb42', 'request_id': 'req-2de6eed5-1349-4b82-bb2c-2923bf448a54'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:30.895948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:30.900291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2de6eed5-1349-4b82-bb2c-2923bf448a54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fef666fa68c4e37be220fb259e6bb42', 'request_id': 'req-2de6eed5-1349-4b82-bb2c-2923bf448a54'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:30.905710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c5ecd2b0-c063-4452-913e-86badcef7268 None None] MFA Rules not processed for user `6d63c633112e49d091421c84cea3422b`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:30.907888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c5ecd2b0-c063-4452-913e-86badcef7268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:30.927248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2de6eed5-1349-4b82-bb2c-2923bf448a54 admin admin] The token cache is being invalidate because project 3fef666fa68c4e37be220fb259e6bb42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:30.928079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2de6eed5-1349-4b82-bb2c-2923bf448a54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:30.928839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2de6eed5-1349-4b82-bb2c-2923bf448a54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:30.931236 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2533/5003] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:30 2022] DELETE /identity/v3/projects/3fef666fa68c4e37be220fb259e6bb42 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:30.932777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbb48f-e0bd-4626-a63b-50fef2e2a74c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:30.933058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbb48f-e0bd-4626-a63b-50fef2e2a74c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:30.933331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbb48f-e0bd-4626-a63b-50fef2e2a74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:30.971355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3afbb48f-e0bd-4626-a63b-50fef2e2a74c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:30.977312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3afbb48f-e0bd-4626-a63b-50fef2e2a74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:31.017773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:31.141498 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2471/5004] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:31.143188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a7e1a-d866-4222-ad3b-2876262dbfa3 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:31.143396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a7e1a-d866-4222-ad3b-2876262dbfa3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:31.143738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a7e1a-d866-4222-ad3b-2876262dbfa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:31.190109 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2534/5005] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:31.195048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-303a7e1a-d866-4222-ad3b-2876262dbfa3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:31.197421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab6165-a99a-46ef-b570-27e86502d607 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:31.197757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab6165-a99a-46ef-b570-27e86502d607 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:31.198048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab6165-a99a-46ef-b570-27e86502d607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:31.202219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-303a7e1a-d866-4222-ad3b-2876262dbfa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:31.235419 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2472/5006] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:31.239046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a7e1a-d866-4222-ad3b-2876262dbfa3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:31.241348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-303a7e1a-d866-4222-ad3b-2876262dbfa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:31.243596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2eab6165-a99a-46ef-b570-27e86502d607 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:31.249640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2eab6165-a99a-46ef-b570-27e86502d607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:31.258252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a7e1a-d866-4222-ad3b-2876262dbfa3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:31.264603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff27b9c-8236-4db2-9590-e636cffe79bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:31.265331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff27b9c-8236-4db2-9590-e636cffe79bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:31.265683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff27b9c-8236-4db2-9590-e636cffe79bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:31.266118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff27b9c-8236-4db2-9590-e636cffe79bc admin admin] PATH_INFO: `/v3/users/40a38d38fee342fb84f77758525b591b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:31.268294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff27b9c-8236-4db2-9590-e636cffe79bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=40a38d38fee342fb84f77758525b591b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:31.271476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff27b9c-8236-4db2-9590-e636cffe79bc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:31.279556 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2535/5007] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:31.284274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcaa98-865c-47a7-b6d0-62b1cb82688f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:31.286836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcaa98-865c-47a7-b6d0-62b1cb82688f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:31.287414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcaa98-865c-47a7-b6d0-62b1cb82688f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:31.330909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5bbcaa98-865c-47a7-b6d0-62b1cb82688f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:31.333741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5bbcaa98-865c-47a7-b6d0-62b1cb82688f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:31.348138 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2536/5008] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:31.349122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcaa98-865c-47a7-b6d0-62b1cb82688f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:31.363731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcaa98-865c-47a7-b6d0-62b1cb82688f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:31.379510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12976b4e-2dc1-4d85-8220-eb4cacf9fec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:31.381058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12976b4e-2dc1-4d85-8220-eb4cacf9fec4 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:31.381353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12976b4e-2dc1-4d85-8220-eb4cacf9fec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:31.381778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12976b4e-2dc1-4d85-8220-eb4cacf9fec4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:31.382456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12976b4e-2dc1-4d85-8220-eb4cacf9fec4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:05:31.383320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12976b4e-2dc1-4d85-8220-eb4cacf9fec4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:31.384430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12976b4e-2dc1-4d85-8220-eb4cacf9fec4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:31.422764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5ff27b9c-8236-4db2-9590-e636cffe79bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40a38d38fee342fb84f77758525b591b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:31.424215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5ff27b9c-8236-4db2-9590-e636cffe79bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40a38d38fee342fb84f77758525b591b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:31.427500 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2537/5009] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:31.431365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12976b4e-2dc1-4d85-8220-eb4cacf9fec4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:31.432077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5ff27b9c-8236-4db2-9590-e636cffe79bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40a38d38fee342fb84f77758525b591b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:31.434787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-12976b4e-2dc1-4d85-8220-eb4cacf9fec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:31.454291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5ff27b9c-8236-4db2-9590-e636cffe79bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40a38d38fee342fb84f77758525b591b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:31.464335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12976b4e-2dc1-4d85-8220-eb4cacf9fec4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:31.477401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c4393-bed0-4afc-8ac6-61aada563dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:31.478115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c4393-bed0-4afc-8ac6-61aada563dff admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:31.478347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5ff27b9c-8236-4db2-9590-e636cffe79bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40a38d38fee342fb84f77758525b591b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:31.478451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c4393-bed0-4afc-8ac6-61aada563dff admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:31.478787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c4393-bed0-4afc-8ac6-61aada563dff admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:31.479454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c4393-bed0-4afc-8ac6-61aada563dff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:05:31.479889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c4393-bed0-4afc-8ac6-61aada563dff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:31.484444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c4393-bed0-4afc-8ac6-61aada563dff admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:31.495297 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2473/5010] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:31 2022] DELETE /identity/v3/users/40a38d38fee342fb84f77758525b591b => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:31.496293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff27b9c-8236-4db2-9590-e636cffe79bc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:31.498453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5ff27b9c-8236-4db2-9590-e636cffe79bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:31.515787 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2538/5011] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:31.519677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6ac60-178a-49cd-88b5-522c20e663c3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:31.523083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6ac60-178a-49cd-88b5-522c20e663c3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:31.523603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6ac60-178a-49cd-88b5-522c20e663c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:31.567222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-75f6ac60-178a-49cd-88b5-522c20e663c3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:31.568473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-75f6ac60-178a-49cd-88b5-522c20e663c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:31.656301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff27b9c-8236-4db2-9590-e636cffe79bc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:31.673638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1b9c8-62b1-4d6e-95a2-73216f134b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:31.681463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1b9c8-62b1-4d6e-95a2-73216f134b66 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:31.681842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1b9c8-62b1-4d6e-95a2-73216f134b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:31.682083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1b9c8-62b1-4d6e-95a2-73216f134b66 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:31.682861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa1b9c8-62b1-4d6e-95a2-73216f134b66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:31.685204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa1b9c8-62b1-4d6e-95a2-73216f134b66 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:31.791798 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2539/5013] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:31.792963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6ac60-178a-49cd-88b5-522c20e663c3 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:31.794011 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2474/5012] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:31 2022] POST /identity/v3/projects => generated 371 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:31.794953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1b9c8-62b1-4d6e-95a2-73216f134b66 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:31.818982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6ac60-178a-49cd-88b5-522c20e663c3 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:31.893799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:31.934475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:31.969508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834d347-4c6e-44ad-995d-8bb6ca07be2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:31.970289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834d347-4c6e-44ad-995d-8bb6ca07be2c admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:31.970739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834d347-4c6e-44ad-995d-8bb6ca07be2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:31.971134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834d347-4c6e-44ad-995d-8bb6ca07be2c admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:31.971852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834d347-4c6e-44ad-995d-8bb6ca07be2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:31.974168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834d347-4c6e-44ad-995d-8bb6ca07be2c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:31.982076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1b9c8-62b1-4d6e-95a2-73216f134b66 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:31.989038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f073f9e-369c-46f8-a5e5-bcec9c5316fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:31.989664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f073f9e-369c-46f8-a5e5-bcec9c5316fb admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:31.989949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f073f9e-369c-46f8-a5e5-bcec9c5316fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:31.990227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f073f9e-369c-46f8-a5e5-bcec9c5316fb admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:31.991363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f073f9e-369c-46f8-a5e5-bcec9c5316fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:31.994482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f073f9e-369c-46f8-a5e5-bcec9c5316fb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:32.038466 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2540/5014] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:31 2022] POST /identity/v3/projects => generated 369 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:32.039965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834d347-4c6e-44ad-995d-8bb6ca07be2c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:32.045232 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2475/5015] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:31 2022] POST /identity/v3/users => generated 286 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:32.047959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9834d347-4c6e-44ad-995d-8bb6ca07be2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:32.048209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f073f9e-369c-46f8-a5e5-bcec9c5316fb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:32.073431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f073f9e-369c-46f8-a5e5-bcec9c5316fb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:32.157068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834d347-4c6e-44ad-995d-8bb6ca07be2c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:32.163319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68993d4-b454-4466-b9a1-d0b48ce7afad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:32.164478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68993d4-b454-4466-b9a1-d0b48ce7afad service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:32.164758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68993d4-b454-4466-b9a1-d0b48ce7afad service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:32.165030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68993d4-b454-4466-b9a1-d0b48ce7afad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:32.171191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35e295-1ac6-49cc-8640-9a0eb2606cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:32.172094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35e295-1ac6-49cc-8640-9a0eb2606cac admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:32.172303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35e295-1ac6-49cc-8640-9a0eb2606cac admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:32.173366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35e295-1ac6-49cc-8640-9a0eb2606cac admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:32.174929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35e295-1ac6-49cc-8640-9a0eb2606cac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:32.179413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35e295-1ac6-49cc-8640-9a0eb2606cac admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:32.191184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68993d4-b454-4466-b9a1-d0b48ce7afad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:32.192912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68993d4-b454-4466-b9a1-d0b48ce7afad service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:32.209535 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2541/5016] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:05:32.212163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68993d4-b454-4466-b9a1-d0b48ce7afad service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:32.224715 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2476/5017] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:32 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:32.229758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35e295-1ac6-49cc-8640-9a0eb2606cac admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:32.246364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68993d4-b454-4466-b9a1-d0b48ce7afad service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:32.255372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35e295-1ac6-49cc-8640-9a0eb2606cac admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:32.262256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57df09-f718-4642-87c3-8a150750a1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:32.263300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57df09-f718-4642-87c3-8a150750a1ba admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:32.263935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57df09-f718-4642-87c3-8a150750a1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:32.264918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57df09-f718-4642-87c3-8a150750a1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:32.265695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47f79b-a850-429a-98de-bafde8caac94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:32.265869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47f79b-a850-429a-98de-bafde8caac94 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:32.266164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47f79b-a850-429a-98de-bafde8caac94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:32.266446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47f79b-a850-429a-98de-bafde8caac94 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:32.267423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47f79b-a850-429a-98de-bafde8caac94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:32.267696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce57df09-f718-4642-87c3-8a150750a1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:32.270006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce57df09-f718-4642-87c3-8a150750a1ba admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:32.273160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47f79b-a850-429a-98de-bafde8caac94 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:32.287683 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2477/5018] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:32.292874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57df09-f718-4642-87c3-8a150750a1ba admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:32.298177 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2542/5019] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:32.306150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47f79b-a850-429a-98de-bafde8caac94 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:32.318790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57df09-f718-4642-87c3-8a150750a1ba admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:32.323934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47f79b-a850-429a-98de-bafde8caac94 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:32.327877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b997f-a652-4f34-9fbb-9ccff7c0327f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:32.328565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b997f-a652-4f34-9fbb-9ccff7c0327f admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:32.328846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b997f-a652-4f34-9fbb-9ccff7c0327f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:32.329119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b997f-a652-4f34-9fbb-9ccff7c0327f admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:32.329884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b997f-a652-4f34-9fbb-9ccff7c0327f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:32.331751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b997f-a652-4f34-9fbb-9ccff7c0327f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:32.338151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f727dbb-3753-42f4-b471-d16aa230f87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:32.339017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f727dbb-3753-42f4-b471-d16aa230f87d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:32.342276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f727dbb-3753-42f4-b471-d16aa230f87d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:32.342677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f727dbb-3753-42f4-b471-d16aa230f87d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:32.343359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f727dbb-3753-42f4-b471-d16aa230f87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:32.346656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f727dbb-3753-42f4-b471-d16aa230f87d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:32.360009 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2478/5020] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:32.363875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b997f-a652-4f34-9fbb-9ccff7c0327f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:32.370653 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2543/5021] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:05:32.374832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f727dbb-3753-42f4-b471-d16aa230f87d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:32.402723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b997f-a652-4f34-9fbb-9ccff7c0327f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:32.405897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f727dbb-3753-42f4-b471-d16aa230f87d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:32.414688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801078f8-aebe-404c-ba33-ea273889095c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:32.415491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801078f8-aebe-404c-ba33-ea273889095c admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:32.415945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801078f8-aebe-404c-ba33-ea273889095c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:32.416225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801078f8-aebe-404c-ba33-ea273889095c admin admin] PATH_INFO: `/v3/projects/fee45202b6504d398abe0081a92ab8dd/users/33e1635d451448fc9ca07f5ab9b93487/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:32.429735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1457a-cb31-496e-bb7f-485edc66518b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:32.430796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1457a-cb31-496e-bb7f-485edc66518b admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:32.431236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1457a-cb31-496e-bb7f-485edc66518b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:32.431521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1457a-cb31-496e-bb7f-485edc66518b admin admin] PATH_INFO: `/v3/projects/5a2f68dc593c49249a1e86a2c0678b5a/users/dbb018c0c0a54b39b879f959a370a63a/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:32.451113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801078f8-aebe-404c-ba33-ea273889095c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee45202b6504d398abe0081a92ab8dd, user_id=33e1635d451448fc9ca07f5ab9b93487, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:32.455011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801078f8-aebe-404c-ba33-ea273889095c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:32.477366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1457a-cb31-496e-bb7f-485edc66518b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a2f68dc593c49249a1e86a2c0678b5a, user_id=dbb018c0c0a54b39b879f959a370a63a, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:32.480206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1457a-cb31-496e-bb7f-485edc66518b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:32.486104 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2479/5022] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:32 2022] PUT /identity/v3/projects/fee45202b6504d398abe0081a92ab8dd/users/33e1635d451448fc9ca07f5ab9b93487/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:32.496342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045b621-fcd3-4d2f-9937-a821557e7d08 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:32.496716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045b621-fcd3-4d2f-9937-a821557e7d08 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:32.497018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045b621-fcd3-4d2f-9937-a821557e7d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:32.527964 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2544/5023] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:32 2022] PUT /identity/v3/projects/5a2f68dc593c49249a1e86a2c0678b5a/users/dbb018c0c0a54b39b879f959a370a63a/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:32.532516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80083085-574e-4525-b18f-dea5d29ecd21 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:32.532808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80083085-574e-4525-b18f-dea5d29ecd21 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:32.533107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80083085-574e-4525-b18f-dea5d29ecd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:32.548215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f045b621-fcd3-4d2f-9937-a821557e7d08 None None] MFA Rules not processed for user `33e1635d451448fc9ca07f5ab9b93487`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:32.549473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f045b621-fcd3-4d2f-9937-a821557e7d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:32.595531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-80083085-574e-4525-b18f-dea5d29ecd21 None None] MFA Rules not processed for user `dbb018c0c0a54b39b879f959a370a63a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:32.597660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-80083085-574e-4525-b18f-dea5d29ecd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:32.644586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:32.687033 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2480/5024] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:05:32.710976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045b621-fcd3-4d2f-9937-a821557e7d08 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:32.742098 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2545/5025] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:32.743309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80083085-574e-4525-b18f-dea5d29ecd21 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:32.830764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:32.831195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045b621-fcd3-4d2f-9937-a821557e7d08 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:32.839398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b81b8b-37de-457d-bee4-7d3b5d5f4701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:32.840540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b81b8b-37de-457d-bee4-7d3b5d5f4701 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:32.853945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b81b8b-37de-457d-bee4-7d3b5d5f4701 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:32.854606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b81b8b-37de-457d-bee4-7d3b5d5f4701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:32.868871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b81b8b-37de-457d-bee4-7d3b5d5f4701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:32.870731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b81b8b-37de-457d-bee4-7d3b5d5f4701 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:32.888824 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2481/5026] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:05:32.936247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80083085-574e-4525-b18f-dea5d29ecd21 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:32.945157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59cfa1-1c24-4c41-b30a-cda6c42fd9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:32.947172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59cfa1-1c24-4c41-b30a-cda6c42fd9b2 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:32.948829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59cfa1-1c24-4c41-b30a-cda6c42fd9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:32.949126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59cfa1-1c24-4c41-b30a-cda6c42fd9b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:32.951104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59cfa1-1c24-4c41-b30a-cda6c42fd9b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:32.956915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59cfa1-1c24-4c41-b30a-cda6c42fd9b2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:33.007501 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2546/5027] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:32 2022] POST /identity/v3/projects => generated 373 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:33.013321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b81b8b-37de-457d-bee4-7d3b5d5f4701 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:33.052837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b81b8b-37de-457d-bee4-7d3b5d5f4701 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:33.246141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63faeb-6f86-47fb-8d5a-ce8761ed8d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:33.247075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63faeb-6f86-47fb-8d5a-ce8761ed8d75 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:33.247351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63faeb-6f86-47fb-8d5a-ce8761ed8d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:33.248516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63faeb-6f86-47fb-8d5a-ce8761ed8d75 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:33.250386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63faeb-6f86-47fb-8d5a-ce8761ed8d75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:33.252627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63faeb-6f86-47fb-8d5a-ce8761ed8d75 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:33.286771 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2482/5028] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:33 2022] POST /identity/v3/users => generated 293 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:33.288355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59cfa1-1c24-4c41-b30a-cda6c42fd9b2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:33.302359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59cfa1-1c24-4c41-b30a-cda6c42fd9b2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:33.313278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ab9bd-0c0e-4639-9a0e-8a2412dcd166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:33.314095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ab9bd-0c0e-4639-9a0e-8a2412dcd166 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:33.314878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ab9bd-0c0e-4639-9a0e-8a2412dcd166 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:33.314878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ab9bd-0c0e-4639-9a0e-8a2412dcd166 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:33.325467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503ab9bd-0c0e-4639-9a0e-8a2412dcd166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:33.325467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503ab9bd-0c0e-4639-9a0e-8a2412dcd166 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:33.344607 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2547/5029] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:33.349812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63faeb-6f86-47fb-8d5a-ce8761ed8d75 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:33.364575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63faeb-6f86-47fb-8d5a-ce8761ed8d75 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:33.372223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71aaf81-add1-4ee1-b6f2-af41751b98a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:33.373449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71aaf81-add1-4ee1-b6f2-af41751b98a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:33.376175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71aaf81-add1-4ee1-b6f2-af41751b98a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:33.376884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71aaf81-add1-4ee1-b6f2-af41751b98a2 admin admin] PATH_INFO: `/v3/projects/10aee3c37c164abea2ace622dd851605/users/b52786be44f04aaebae93c409a8debeb/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:33.399870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ab9bd-0c0e-4639-9a0e-8a2412dcd166 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:33.403694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71aaf81-add1-4ee1-b6f2-af41751b98a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10aee3c37c164abea2ace622dd851605, user_id=b52786be44f04aaebae93c409a8debeb, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:33.404565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-503ab9bd-0c0e-4639-9a0e-8a2412dcd166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:33.407914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71aaf81-add1-4ee1-b6f2-af41751b98a2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:33.426392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ab9bd-0c0e-4639-9a0e-8a2412dcd166 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:33.443932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350854a-af2f-4da5-b56d-1733b9a1ce76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:33.445953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350854a-af2f-4da5-b56d-1733b9a1ce76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:33.447376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350854a-af2f-4da5-b56d-1733b9a1ce76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:33.447776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350854a-af2f-4da5-b56d-1733b9a1ce76 admin admin] PATH_INFO: `/v3/users/aa4fbb28c9de4089870d91cf52dac8ea` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:33.451085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2350854a-af2f-4da5-b56d-1733b9a1ce76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa4fbb28c9de4089870d91cf52dac8ea)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:33.455544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2350854a-af2f-4da5-b56d-1733b9a1ce76 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:33.461466 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2483/5030] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:33 2022] PUT /identity/v3/projects/10aee3c37c164abea2ace622dd851605/users/b52786be44f04aaebae93c409a8debeb/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:33.466806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71aaf81-add1-4ee1-b6f2-af41751b98a2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:33.560910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2350854a-af2f-4da5-b56d-1733b9a1ce76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa4fbb28c9de4089870d91cf52dac8ea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:33.581169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2350854a-af2f-4da5-b56d-1733b9a1ce76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa4fbb28c9de4089870d91cf52dac8ea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:33.610978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:33.615752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71aaf81-add1-4ee1-b6f2-af41751b98a2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:33.617504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2350854a-af2f-4da5-b56d-1733b9a1ce76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa4fbb28c9de4089870d91cf52dac8ea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:33.618987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2350854a-af2f-4da5-b56d-1733b9a1ce76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa4fbb28c9de4089870d91cf52dac8ea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:33.632025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ba128-a30c-4e04-b6b3-8003b6d2d9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:33.632753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ba128-a30c-4e04-b6b3-8003b6d2d9be admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:33.633233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ba128-a30c-4e04-b6b3-8003b6d2d9be admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:33.634774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ba128-a30c-4e04-b6b3-8003b6d2d9be admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:33.635236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2350854a-af2f-4da5-b56d-1733b9a1ce76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa4fbb28c9de4089870d91cf52dac8ea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:33.639057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ba128-a30c-4e04-b6b3-8003b6d2d9be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:33.641263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ba128-a30c-4e04-b6b3-8003b6d2d9be admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:33.649137 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2548/5031] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:33 2022] DELETE /identity/v3/users/aa4fbb28c9de4089870d91cf52dac8ea => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:33.659215 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2484/5032] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:33.663467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350854a-af2f-4da5-b56d-1733b9a1ce76 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:33.669899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2350854a-af2f-4da5-b56d-1733b9a1ce76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:33.817106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350854a-af2f-4da5-b56d-1733b9a1ce76 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:33.825450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f697e5-a4f0-4675-b394-9f33dc91ac82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:33.826305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f697e5-a4f0-4675-b394-9f33dc91ac82 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:33.827450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f697e5-a4f0-4675-b394-9f33dc91ac82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:33.827959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f697e5-a4f0-4675-b394-9f33dc91ac82 admin admin] PATH_INFO: `/v3/projects/10aee3c37c164abea2ace622dd851605/users/b52786be44f04aaebae93c409a8debeb/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:33.830292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f697e5-a4f0-4675-b394-9f33dc91ac82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10aee3c37c164abea2ace622dd851605, user_id=b52786be44f04aaebae93c409a8debeb, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:33.833659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f697e5-a4f0-4675-b394-9f33dc91ac82 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:33.847990 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2549/5033] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:33 2022] PUT /identity/v3/projects/10aee3c37c164abea2ace622dd851605/users/b52786be44f04aaebae93c409a8debeb/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:33.859952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5e51c-44bb-4e61-9ea5-5519b392aaa6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:33.860443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5e51c-44bb-4e61-9ea5-5519b392aaa6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:33.860710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5e51c-44bb-4e61-9ea5-5519b392aaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:33.903843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4ad5e51c-44bb-4e61-9ea5-5519b392aaa6 None None] MFA Rules not processed for user `b52786be44f04aaebae93c409a8debeb`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:33.927377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4ad5e51c-44bb-4e61-9ea5-5519b392aaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:34.064822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f697e5-a4f0-4675-b394-9f33dc91ac82 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:34.066618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-66f697e5-a4f0-4675-b394-9f33dc91ac82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:34.072368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:34.085292 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2485/5034] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:34.090054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5e51c-44bb-4e61-9ea5-5519b392aaa6 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:34.119963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5e51c-44bb-4e61-9ea5-5519b392aaa6 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:34.139557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13bac0-e54e-44dd-b8f2-fd60599df7af tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:34.148124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13bac0-e54e-44dd-b8f2-fd60599df7af tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:34.148429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13bac0-e54e-44dd-b8f2-fd60599df7af tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:34.148694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13bac0-e54e-44dd-b8f2-fd60599df7af tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:34.149875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13bac0-e54e-44dd-b8f2-fd60599df7af tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:34.151534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13bac0-e54e-44dd-b8f2-fd60599df7af tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:34.187371 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2486/5035] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:05:34 2022] POST /identity/v3/domains => generated 277 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:34.191101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13bac0-e54e-44dd-b8f2-fd60599df7af tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:34.193816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f697e5-a4f0-4675-b394-9f33dc91ac82 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:34.280949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50007a49-dc99-463f-93a3-949f13a8d796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:34.282094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50007a49-dc99-463f-93a3-949f13a8d796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:34.282631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50007a49-dc99-463f-93a3-949f13a8d796 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:34.283175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50007a49-dc99-463f-93a3-949f13a8d796 admin admin] PATH_INFO: `/v3/projects/99e8daf9de4d4fa395b7eee86d33a5f5` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:34.284533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50007a49-dc99-463f-93a3-949f13a8d796 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99e8daf9de4d4fa395b7eee86d33a5f5)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:34.290183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50007a49-dc99-463f-93a3-949f13a8d796 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:34.291089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13bac0-e54e-44dd-b8f2-fd60599df7af tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:34.298031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786a63f-32c0-4ae2-a098-ea32acdb49cc tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:34.298766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786a63f-32c0-4ae2-a098-ea32acdb49cc tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:34.299290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786a63f-32c0-4ae2-a098-ea32acdb49cc tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:34.299790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786a63f-32c0-4ae2-a098-ea32acdb49cc tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:34.300653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786a63f-32c0-4ae2-a098-ea32acdb49cc tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:34.303457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786a63f-32c0-4ae2-a098-ea32acdb49cc tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:34.391489 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2487/5036] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:34 2022] POST /identity/v3/projects => generated 403 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:34.399946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786a63f-32c0-4ae2-a098-ea32acdb49cc tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:34.450955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-50007a49-dc99-463f-93a3-949f13a8d796 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99e8daf9de4d4fa395b7eee86d33a5f5', 'request_id': 'req-50007a49-dc99-463f-93a3-949f13a8d796'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:34.471945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-50007a49-dc99-463f-93a3-949f13a8d796 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99e8daf9de4d4fa395b7eee86d33a5f5', 'request_id': 'req-50007a49-dc99-463f-93a3-949f13a8d796'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:34.484890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-50007a49-dc99-463f-93a3-949f13a8d796 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99e8daf9de4d4fa395b7eee86d33a5f5', 'request_id': 'req-50007a49-dc99-463f-93a3-949f13a8d796'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:34.504836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:34.516050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-50007a49-dc99-463f-93a3-949f13a8d796 admin admin] The token cache is being invalidate because project 99e8daf9de4d4fa395b7eee86d33a5f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:34.527498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-50007a49-dc99-463f-93a3-949f13a8d796 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:34.529286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-50007a49-dc99-463f-93a3-949f13a8d796 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:34.532223 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2550/5037] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:34 2022] DELETE /identity/v3/projects/99e8daf9de4d4fa395b7eee86d33a5f5 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:05:34.533653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50007a49-dc99-463f-93a3-949f13a8d796 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:34.536254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-50007a49-dc99-463f-93a3-949f13a8d796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:34.550411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786a63f-32c0-4ae2-a098-ea32acdb49cc tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:34.667903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3dcbb-23d9-4ebc-a3e0-94fbe44114c2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:34.668535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3dcbb-23d9-4ebc-a3e0-94fbe44114c2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:34.668821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3dcbb-23d9-4ebc-a3e0-94fbe44114c2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:34.669098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3dcbb-23d9-4ebc-a3e0-94fbe44114c2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/groups` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:34.669737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3dcbb-23d9-4ebc-a3e0-94fbe44114c2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:34.672243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3dcbb-23d9-4ebc-a3e0-94fbe44114c2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:34.707385 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2488/5038] 10.222.0.17 () {60 vars in 1147 bytes} [Sat Oct 1 21:05:34 2022] POST /identity/v3/groups => generated 301 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:34.707385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70274f4c-3502-4113-a6b4-255953292ce5 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:34.707385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70274f4c-3502-4113-a6b4-255953292ce5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:34.707385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70274f4c-3502-4113-a6b4-255953292ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:34.715134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:34.742152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-70274f4c-3502-4113-a6b4-255953292ce5 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:34.743538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-70274f4c-3502-4113-a6b4-255953292ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:34.763457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50007a49-dc99-463f-93a3-949f13a8d796 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:34.769435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ccda7-06c9-4c7d-9c4b-94078cc9cd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:34.771524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ccda7-06c9-4c7d-9c4b-94078cc9cd2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:34.771949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ccda7-06c9-4c7d-9c4b-94078cc9cd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:34.773322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ccda7-06c9-4c7d-9c4b-94078cc9cd2f admin admin] PATH_INFO: `/v3/projects/e3e9e56444d448cb9219affd1e0e41bc` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:34.777146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ccda7-06c9-4c7d-9c4b-94078cc9cd2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3e9e56444d448cb9219affd1e0e41bc)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:34.778436 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2489/5039] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:34.779622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70274f4c-3502-4113-a6b4-255953292ce5 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:34.782540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-70274f4c-3502-4113-a6b4-255953292ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:34.783405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ccda7-06c9-4c7d-9c4b-94078cc9cd2f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:34.827914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70274f4c-3502-4113-a6b4-255953292ce5 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:34.840399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47380206-96ee-4354-bb77-e1789a974bbf tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:34.842954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47380206-96ee-4354-bb77-e1789a974bbf tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:34.843693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47380206-96ee-4354-bb77-e1789a974bbf tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:34.844578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47380206-96ee-4354-bb77-e1789a974bbf tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:34.846326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47380206-96ee-4354-bb77-e1789a974bbf tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:34.850297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47380206-96ee-4354-bb77-e1789a974bbf tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:34.904033 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2490/5040] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:34 2022] POST /identity/v3/users => generated 445 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:34.904603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e55ccda7-06c9-4c7d-9c4b-94078cc9cd2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3e9e56444d448cb9219affd1e0e41bc', 'request_id': 'req-e55ccda7-06c9-4c7d-9c4b-94078cc9cd2f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:34.911038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ccc13-ee0b-4ee3-993f-a5c721471439 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:34.911395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ccc13-ee0b-4ee3-993f-a5c721471439 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:34.911747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ccc13-ee0b-4ee3-993f-a5c721471439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:34.913972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e55ccda7-06c9-4c7d-9c4b-94078cc9cd2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3e9e56444d448cb9219affd1e0e41bc', 'request_id': 'req-e55ccda7-06c9-4c7d-9c4b-94078cc9cd2f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:34.937496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-714ccc13-ee0b-4ee3-993f-a5c721471439 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:34.939005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-714ccc13-ee0b-4ee3-993f-a5c721471439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:34.941575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e55ccda7-06c9-4c7d-9c4b-94078cc9cd2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3e9e56444d448cb9219affd1e0e41bc', 'request_id': 'req-e55ccda7-06c9-4c7d-9c4b-94078cc9cd2f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:34.973821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e55ccda7-06c9-4c7d-9c4b-94078cc9cd2f admin admin] The token cache is being invalidate because project e3e9e56444d448cb9219affd1e0e41bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:34.974121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e55ccda7-06c9-4c7d-9c4b-94078cc9cd2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:34.978508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e55ccda7-06c9-4c7d-9c4b-94078cc9cd2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:34.982054 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2551/5041] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:34 2022] DELETE /identity/v3/projects/e3e9e56444d448cb9219affd1e0e41bc => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:34.982564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ccda7-06c9-4c7d-9c4b-94078cc9cd2f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:34.986563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e55ccda7-06c9-4c7d-9c4b-94078cc9cd2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:35.094125 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2491/5042] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:35.095104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ccc13-ee0b-4ee3-993f-a5c721471439 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:35.102217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-714ccc13-ee0b-4ee3-993f-a5c721471439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:35.122030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ccc13-ee0b-4ee3-993f-a5c721471439 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:35.128069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac04db8-0e3b-4d37-ad9a-a932a8f73334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:35.128771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac04db8-0e3b-4d37-ad9a-a932a8f73334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:35.129196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac04db8-0e3b-4d37-ad9a-a932a8f73334 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:35.129525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac04db8-0e3b-4d37-ad9a-a932a8f73334 admin admin] PATH_INFO: `/v3/users/d7d71fbab5fa443e843b0c6573decf61` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:35.131860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac04db8-0e3b-4d37-ad9a-a932a8f73334 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7d71fbab5fa443e843b0c6573decf61)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:35.135212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac04db8-0e3b-4d37-ad9a-a932a8f73334 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:35.164067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ccda7-06c9-4c7d-9c4b-94078cc9cd2f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:35.174882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4e1bf-56f8-45b4-99f0-f2080445e0b1 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:35.175520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4e1bf-56f8-45b4-99f0-f2080445e0b1 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:35.175810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4e1bf-56f8-45b4-99f0-f2080445e0b1 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:35.176081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4e1bf-56f8-45b4-99f0-f2080445e0b1 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:35.176833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc4e1bf-56f8-45b4-99f0-f2080445e0b1 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:35.178344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc4e1bf-56f8-45b4-99f0-f2080445e0b1 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:35.230245 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2552/5043] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:05:34 2022] POST /identity/v3/roles => generated 240 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:35.231076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4e1bf-56f8-45b4-99f0-f2080445e0b1 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:35.249487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4e1bf-56f8-45b4-99f0-f2080445e0b1 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:35.250980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4ac04db8-0e3b-4d37-ad9a-a932a8f73334 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7d71fbab5fa443e843b0c6573decf61'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:35.252140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4ac04db8-0e3b-4d37-ad9a-a932a8f73334 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7d71fbab5fa443e843b0c6573decf61'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:35.260152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4ac04db8-0e3b-4d37-ad9a-a932a8f73334 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7d71fbab5fa443e843b0c6573decf61'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:35.268632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f4486-a2fe-411a-8f5c-994e6ca55161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:35.269748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f4486-a2fe-411a-8f5c-994e6ca55161 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:35.272016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f4486-a2fe-411a-8f5c-994e6ca55161 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:35.272333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f4486-a2fe-411a-8f5c-994e6ca55161 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:35.274400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f4486-a2fe-411a-8f5c-994e6ca55161 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:05:35.274823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f4486-a2fe-411a-8f5c-994e6ca55161 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:35.280643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4ac04db8-0e3b-4d37-ad9a-a932a8f73334 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7d71fbab5fa443e843b0c6573decf61'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:35.281862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f4486-a2fe-411a-8f5c-994e6ca55161 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:35.283823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:35.308222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4ac04db8-0e3b-4d37-ad9a-a932a8f73334 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7d71fbab5fa443e843b0c6573decf61'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:35.325295 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2553/5044] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:35.326138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f4486-a2fe-411a-8f5c-994e6ca55161 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:35.329535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-319f4486-a2fe-411a-8f5c-994e6ca55161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:35.347919 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2492/5045] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:35 2022] DELETE /identity/v3/users/d7d71fbab5fa443e843b0c6573decf61 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:35.354871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac04db8-0e3b-4d37-ad9a-a932a8f73334 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:35.361137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4ac04db8-0e3b-4d37-ad9a-a932a8f73334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:35.368351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f4486-a2fe-411a-8f5c-994e6ca55161 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:35.449129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:35.476311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be41794-2655-487a-9766-df02272fa722 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:35.477380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be41794-2655-487a-9766-df02272fa722 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:35.477519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be41794-2655-487a-9766-df02272fa722 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:35.477828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be41794-2655-487a-9766-df02272fa722 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/c18eaa6e4f1a46b782e3c975baede48d/groups/44d87a83a5a3478e8bd9fddf037e80c9/roles/9ca06eab8ddc4614a4863c773095bb95/inherited_to_projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:35.496178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be41794-2655-487a-9766-df02272fa722 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:create_grant(project_id=c18eaa6e4f1a46b782e3c975baede48d, group_id=44d87a83a5a3478e8bd9fddf037e80c9, role_id=9ca06eab8ddc4614a4863c773095bb95)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:35.502989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be41794-2655-487a-9766-df02272fa722 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:35.528562 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2554/5046] 10.222.0.17 () {60 vars in 1585 bytes} [Sat Oct 1 21:05:35 2022] PUT /identity/v3/OS-INHERIT/projects/c18eaa6e4f1a46b782e3c975baede48d/groups/44d87a83a5a3478e8bd9fddf037e80c9/roles/9ca06eab8ddc4614a4863c773095bb95/inherited_to_projects => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:35.539130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be41794-2655-487a-9766-df02272fa722 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:35.576019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac04db8-0e3b-4d37-ad9a-a932a8f73334 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:35.666662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be41794-2655-487a-9766-df02272fa722 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:35.679784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51102cad-e24c-47b5-a77c-96cdd03ed0e7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:35.681375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51102cad-e24c-47b5-a77c-96cdd03ed0e7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:35.681750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51102cad-e24c-47b5-a77c-96cdd03ed0e7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:35.683385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51102cad-e24c-47b5-a77c-96cdd03ed0e7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/c18eaa6e4f1a46b782e3c975baede48d/groups/44d87a83a5a3478e8bd9fddf037e80c9/roles/9ca06eab8ddc4614a4863c773095bb95/inherited_to_projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:35.689412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51102cad-e24c-47b5-a77c-96cdd03ed0e7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:check_grant(project_id=c18eaa6e4f1a46b782e3c975baede48d, group_id=44d87a83a5a3478e8bd9fddf037e80c9, role_id=9ca06eab8ddc4614a4863c773095bb95)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:35.690724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f47841-73a2-4f16-ad03-507a8e0aabc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:35.693272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f47841-73a2-4f16-ad03-507a8e0aabc9 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:35.693272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f47841-73a2-4f16-ad03-507a8e0aabc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:35.694074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f47841-73a2-4f16-ad03-507a8e0aabc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:35.694763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f47841-73a2-4f16-ad03-507a8e0aabc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:35.697152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51102cad-e24c-47b5-a77c-96cdd03ed0e7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:35.699741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f47841-73a2-4f16-ad03-507a8e0aabc9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:35.722562 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2555/5047] 10.222.0.17 () {58 vars in 1567 bytes} [Sat Oct 1 21:05:35 2022] HEAD /identity/v3/OS-INHERIT/projects/c18eaa6e4f1a46b782e3c975baede48d/groups/44d87a83a5a3478e8bd9fddf037e80c9/roles/9ca06eab8ddc4614a4863c773095bb95/inherited_to_projects => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:35.731630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51102cad-e24c-47b5-a77c-96cdd03ed0e7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:35.742360 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2493/5048] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:35 2022] POST /identity/v3/projects => generated 401 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:35.747157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f47841-73a2-4f16-ad03-507a8e0aabc9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:35.773464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51102cad-e24c-47b5-a77c-96cdd03ed0e7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:35.926323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f47841-73a2-4f16-ad03-507a8e0aabc9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:35.929085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a9fa1-f788-42d8-a002-803ae003612b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:35.930030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a9fa1-f788-42d8-a002-803ae003612b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:35.931254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a9fa1-f788-42d8-a002-803ae003612b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:35.932787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a9fa1-f788-42d8-a002-803ae003612b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/c18eaa6e4f1a46b782e3c975baede48d/groups/44d87a83a5a3478e8bd9fddf037e80c9/roles/9ca06eab8ddc4614a4863c773095bb95/inherited_to_projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:35.948523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adaebc8-4e6c-4b29-9a7b-1d1180a5ff59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:35.949261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adaebc8-4e6c-4b29-9a7b-1d1180a5ff59 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:35.949554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adaebc8-4e6c-4b29-9a7b-1d1180a5ff59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:35.949828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adaebc8-4e6c-4b29-9a7b-1d1180a5ff59 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:35.955247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a9fa1-f788-42d8-a002-803ae003612b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=c18eaa6e4f1a46b782e3c975baede48d, group_id=44d87a83a5a3478e8bd9fddf037e80c9, role_id=9ca06eab8ddc4614a4863c773095bb95)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:35.957340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adaebc8-4e6c-4b29-9a7b-1d1180a5ff59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:35.962170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a9fa1-f788-42d8-a002-803ae003612b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:35.964118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adaebc8-4e6c-4b29-9a7b-1d1180a5ff59 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:35.986262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0d4a9fa1-f788-42d8-a002-803ae003612b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invalidating the token cache because role 9ca06eab8ddc4614a4863c773095bb95 was removed from group 44d87a83a5a3478e8bd9fddf037e80c9 on project c18eaa6e4f1a46b782e3c975baede48d. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:35.986476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0d4a9fa1-f788-42d8-a002-803ae003612b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:35.988859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0d4a9fa1-f788-42d8-a002-803ae003612b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:36.010581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:36.046513 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2556/5049] 10.222.0.17 () {58 vars in 1569 bytes} [Sat Oct 1 21:05:35 2022] DELETE /identity/v3/OS-INHERIT/projects/c18eaa6e4f1a46b782e3c975baede48d/groups/44d87a83a5a3478e8bd9fddf037e80c9/roles/9ca06eab8ddc4614a4863c773095bb95/inherited_to_projects => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:36.051737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a9fa1-f788-42d8-a002-803ae003612b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:36.055507 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2494/5050] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:35 2022] POST /identity/v3/users => generated 301 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:36.064979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0d4a9fa1-f788-42d8-a002-803ae003612b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:36.066333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adaebc8-4e6c-4b29-9a7b-1d1180a5ff59 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:36.075415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8adaebc8-4e6c-4b29-9a7b-1d1180a5ff59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:36.121200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adaebc8-4e6c-4b29-9a7b-1d1180a5ff59 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:36.284377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a9fa1-f788-42d8-a002-803ae003612b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:36.303304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc56abd-c32f-46ed-8daf-b89ca418cb0e tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:36.304073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc56abd-c32f-46ed-8daf-b89ca418cb0e tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:36.304364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc56abd-c32f-46ed-8daf-b89ca418cb0e tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:36.304900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc56abd-c32f-46ed-8daf-b89ca418cb0e tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/roles/9ca06eab8ddc4614a4863c773095bb95` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:36.307169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc56abd-c32f-46ed-8daf-b89ca418cb0e tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:delete_role(role_id=9ca06eab8ddc4614a4863c773095bb95)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:36.308937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc56abd-c32f-46ed-8daf-b89ca418cb0e tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:36.310726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745cc4a-dbde-4727-b937-c47a410a5982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:36.311656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745cc4a-dbde-4727-b937-c47a410a5982 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:36.314280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745cc4a-dbde-4727-b937-c47a410a5982 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:36.314620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745cc4a-dbde-4727-b937-c47a410a5982 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:36.315294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745cc4a-dbde-4727-b937-c47a410a5982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:36.320016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745cc4a-dbde-4727-b937-c47a410a5982 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:36.325880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:36.363130 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2495/5051] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:36 2022] GET /identity/v3/roles => generated 1651 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:36.364787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745cc4a-dbde-4727-b937-c47a410a5982 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:36.381285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745cc4a-dbde-4727-b937-c47a410a5982 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:36.389188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c186a5-c047-459c-a436-7912816822a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:36.389967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c186a5-c047-459c-a436-7912816822a9 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:36.391188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c186a5-c047-459c-a436-7912816822a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:36.391188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c186a5-c047-459c-a436-7912816822a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:36.391358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5dc56abd-c32f-46ed-8daf-b89ca418cb0e tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invalidating the token cache because role 9ca06eab8ddc4614a4863c773095bb95 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:36.391358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5dc56abd-c32f-46ed-8daf-b89ca418cb0e tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:36.391644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c186a5-c047-459c-a436-7912816822a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:36.393890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c186a5-c047-459c-a436-7912816822a9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:36.396472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5dc56abd-c32f-46ed-8daf-b89ca418cb0e tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:36.398944 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2557/5052] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:36 2022] DELETE /identity/v3/roles/9ca06eab8ddc4614a4863c773095bb95 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:36.403957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc56abd-c32f-46ed-8daf-b89ca418cb0e tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:36.405912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5dc56abd-c32f-46ed-8daf-b89ca418cb0e tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:36.414358 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2496/5053] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:36.416325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c186a5-c047-459c-a436-7912816822a9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:36.421127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-69c186a5-c047-459c-a436-7912816822a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:36.439996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c186a5-c047-459c-a436-7912816822a9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:36.543607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc56abd-c32f-46ed-8daf-b89ca418cb0e tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:36.561923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f997e-75e5-4ee8-b95a-64d89f7fb65a tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:36.562828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f997e-75e5-4ee8-b95a-64d89f7fb65a tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:36.563119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f997e-75e5-4ee8-b95a-64d89f7fb65a tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:36.565639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f997e-75e5-4ee8-b95a-64d89f7fb65a tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:36.567002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f997e-75e5-4ee8-b95a-64d89f7fb65a tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:36.573718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f997e-75e5-4ee8-b95a-64d89f7fb65a tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:36.640957 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2558/5054] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:05:36 2022] POST /identity/v3/roles => generated 239 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:05:36.643309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f997e-75e5-4ee8-b95a-64d89f7fb65a tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:36.655749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea7c81-26ff-43f1-8941-6f387471dffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:36.666232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea7c81-26ff-43f1-8941-6f387471dffb admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:36.666545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea7c81-26ff-43f1-8941-6f387471dffb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:36.666829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea7c81-26ff-43f1-8941-6f387471dffb admin admin] PATH_INFO: `/v3/projects/79942f157bd4443bb422b629eea5b30f/users/a4e3cd820f0e49cc969a94cb48422e42/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:36.675326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f997e-75e5-4ee8-b95a-64d89f7fb65a tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:36.689161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3dc8f-ff6b-47cb-8ce0-a86471069bf8 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:36.689772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3dc8f-ff6b-47cb-8ce0-a86471069bf8 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:36.690062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3dc8f-ff6b-47cb-8ce0-a86471069bf8 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:36.690345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3dc8f-ff6b-47cb-8ce0-a86471069bf8 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/c18eaa6e4f1a46b782e3c975baede48d/users/9c9383ae72e647278c4617af86a60544/roles/622a5c47ef5447bdbbabb0307f6e8f4a/inherited_to_projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:36.735860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea7c81-26ff-43f1-8941-6f387471dffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=79942f157bd4443bb422b629eea5b30f, user_id=a4e3cd820f0e49cc969a94cb48422e42, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:36.747603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea7c81-26ff-43f1-8941-6f387471dffb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:36.752437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3dc8f-ff6b-47cb-8ce0-a86471069bf8 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:create_grant(project_id=c18eaa6e4f1a46b782e3c975baede48d, user_id=9c9383ae72e647278c4617af86a60544, role_id=622a5c47ef5447bdbbabb0307f6e8f4a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:36.755749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3dc8f-ff6b-47cb-8ce0-a86471069bf8 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:36.802580 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2497/5055] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:36 2022] PUT /identity/v3/projects/79942f157bd4443bb422b629eea5b30f/users/a4e3cd820f0e49cc969a94cb48422e42/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:36.802580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea7c81-26ff-43f1-8941-6f387471dffb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:36.802580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b9ea7c81-26ff-43f1-8941-6f387471dffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:36.803139 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2559/5056] 10.222.0.17 () {60 vars in 1582 bytes} [Sat Oct 1 21:05:36 2022] PUT /identity/v3/OS-INHERIT/projects/c18eaa6e4f1a46b782e3c975baede48d/users/9c9383ae72e647278c4617af86a60544/roles/622a5c47ef5447bdbbabb0307f6e8f4a/inherited_to_projects => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:36.803139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3dc8f-ff6b-47cb-8ce0-a86471069bf8 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:36.970984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea7c81-26ff-43f1-8941-6f387471dffb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:36.976755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3dc8f-ff6b-47cb-8ce0-a86471069bf8 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:36.981619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:36.990022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531c3b0-059a-4649-846f-d17d2234f809 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:36.990373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531c3b0-059a-4649-846f-d17d2234f809 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:36.990840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531c3b0-059a-4649-846f-d17d2234f809 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:36.991347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531c3b0-059a-4649-846f-d17d2234f809 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/c18eaa6e4f1a46b782e3c975baede48d/users/9c9383ae72e647278c4617af86a60544/roles/622a5c47ef5447bdbbabb0307f6e8f4a/inherited_to_projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:36.994685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531c3b0-059a-4649-846f-d17d2234f809 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:check_grant(project_id=c18eaa6e4f1a46b782e3c975baede48d, user_id=9c9383ae72e647278c4617af86a60544, role_id=622a5c47ef5447bdbbabb0307f6e8f4a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:36.999691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531c3b0-059a-4649-846f-d17d2234f809 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:37.011813 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2560/5057] 10.222.0.17 () {58 vars in 1564 bytes} [Sat Oct 1 21:05:36 2022] HEAD /identity/v3/OS-INHERIT/projects/c18eaa6e4f1a46b782e3c975baede48d/users/9c9383ae72e647278c4617af86a60544/roles/622a5c47ef5447bdbbabb0307f6e8f4a/inherited_to_projects => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:37.012971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e3b34-4b47-4074-9e1f-46d181983d9c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:37.013363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e3b34-4b47-4074-9e1f-46d181983d9c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:37.013680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e3b34-4b47-4074-9e1f-46d181983d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:37.068483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ed4e3b34-4b47-4074-9e1f-46d181983d9c None None] MFA Rules not processed for user `a4e3cd820f0e49cc969a94cb48422e42`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:37.070921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ed4e3b34-4b47-4074-9e1f-46d181983d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:37.087052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a9d89-6742-4751-a5c7-d0c9d6916d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:37.087701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a9d89-6742-4751-a5c7-d0c9d6916d57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:37.087981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a9d89-6742-4751-a5c7-d0c9d6916d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:37.088644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a9d89-6742-4751-a5c7-d0c9d6916d57 admin admin] PATH_INFO: `/v3/projects/a0a8e2e9f2bd44dca5c729103e29a882` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:37.089835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a9d89-6742-4751-a5c7-d0c9d6916d57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0a8e2e9f2bd44dca5c729103e29a882)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:37.094894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a9d89-6742-4751-a5c7-d0c9d6916d57 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:37.138406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:37.187534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-871a9d89-6742-4751-a5c7-d0c9d6916d57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0a8e2e9f2bd44dca5c729103e29a882', 'request_id': 'req-871a9d89-6742-4751-a5c7-d0c9d6916d57'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:37.204893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-871a9d89-6742-4751-a5c7-d0c9d6916d57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0a8e2e9f2bd44dca5c729103e29a882', 'request_id': 'req-871a9d89-6742-4751-a5c7-d0c9d6916d57'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:37.215275 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2561/5058] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:37 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:37.215492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-871a9d89-6742-4751-a5c7-d0c9d6916d57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0a8e2e9f2bd44dca5c729103e29a882', 'request_id': 'req-871a9d89-6742-4751-a5c7-d0c9d6916d57'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:37.217739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e3b34-4b47-4074-9e1f-46d181983d9c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:37.242962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-871a9d89-6742-4751-a5c7-d0c9d6916d57 admin admin] The token cache is being invalidate because project a0a8e2e9f2bd44dca5c729103e29a882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:37.243335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e3b34-4b47-4074-9e1f-46d181983d9c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:37.243789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-871a9d89-6742-4751-a5c7-d0c9d6916d57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:37.244723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-871a9d89-6742-4751-a5c7-d0c9d6916d57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:37.246526 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2498/5059] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:36 2022] DELETE /identity/v3/projects/a0a8e2e9f2bd44dca5c729103e29a882 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:37.252886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc273876-f838-4c77-ad9d-f5bf691cf235 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:37.253394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc273876-f838-4c77-ad9d-f5bf691cf235 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:37.253854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc273876-f838-4c77-ad9d-f5bf691cf235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:37.295152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bc273876-f838-4c77-ad9d-f5bf691cf235 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:37.296823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bc273876-f838-4c77-ad9d-f5bf691cf235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:37.360943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83505b67-0a7b-4203-ae0a-8f98d7dd06c2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:37.362222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83505b67-0a7b-4203-ae0a-8f98d7dd06c2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:37.362812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83505b67-0a7b-4203-ae0a-8f98d7dd06c2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:37.363407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83505b67-0a7b-4203-ae0a-8f98d7dd06c2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/c18eaa6e4f1a46b782e3c975baede48d/users/9c9383ae72e647278c4617af86a60544/roles/622a5c47ef5447bdbbabb0307f6e8f4a/inherited_to_projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:37.370579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83505b67-0a7b-4203-ae0a-8f98d7dd06c2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=c18eaa6e4f1a46b782e3c975baede48d, user_id=9c9383ae72e647278c4617af86a60544, role_id=622a5c47ef5447bdbbabb0307f6e8f4a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:37.377363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83505b67-0a7b-4203-ae0a-8f98d7dd06c2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:37.400407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-83505b67-0a7b-4203-ae0a-8f98d7dd06c2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invalidating the token cache because role 622a5c47ef5447bdbbabb0307f6e8f4a was removed from user 9c9383ae72e647278c4617af86a60544 on project c18eaa6e4f1a46b782e3c975baede48d. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:37.400713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-83505b67-0a7b-4203-ae0a-8f98d7dd06c2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:37.401518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-83505b67-0a7b-4203-ae0a-8f98d7dd06c2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:37.438916 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2562/5060] 10.222.0.17 () {58 vars in 1566 bytes} [Sat Oct 1 21:05:37 2022] DELETE /identity/v3/OS-INHERIT/projects/c18eaa6e4f1a46b782e3c975baede48d/users/9c9383ae72e647278c4617af86a60544/roles/622a5c47ef5447bdbbabb0307f6e8f4a/inherited_to_projects => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:37.444003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83505b67-0a7b-4203-ae0a-8f98d7dd06c2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:37.446605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-83505b67-0a7b-4203-ae0a-8f98d7dd06c2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:37.473744 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2499/5061] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:05:37.475500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfee44f-94f8-48d6-9e5b-4d472a5f0fd9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:37.475754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfee44f-94f8-48d6-9e5b-4d472a5f0fd9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:37.476183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfee44f-94f8-48d6-9e5b-4d472a5f0fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:37.532228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0bfee44f-94f8-48d6-9e5b-4d472a5f0fd9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:37.533274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0bfee44f-94f8-48d6-9e5b-4d472a5f0fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:37.559104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83505b67-0a7b-4203-ae0a-8f98d7dd06c2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:37.566753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1feb8-3291-490e-bbb1-87925b26bdf2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:37.568278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1feb8-3291-490e-bbb1-87925b26bdf2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:37.568575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1feb8-3291-490e-bbb1-87925b26bdf2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:37.568848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1feb8-3291-490e-bbb1-87925b26bdf2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/roles/622a5c47ef5447bdbbabb0307f6e8f4a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:37.571838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1feb8-3291-490e-bbb1-87925b26bdf2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:delete_role(role_id=622a5c47ef5447bdbbabb0307f6e8f4a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:37.574523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1feb8-3291-490e-bbb1-87925b26bdf2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:37.653183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-91a1feb8-3291-490e-bbb1-87925b26bdf2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invalidating the token cache because role 622a5c47ef5447bdbbabb0307f6e8f4a has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:37.653482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-91a1feb8-3291-490e-bbb1-87925b26bdf2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:37.654196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-91a1feb8-3291-490e-bbb1-87925b26bdf2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:37.664519 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2563/5062] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:37 2022] DELETE /identity/v3/roles/622a5c47ef5447bdbbabb0307f6e8f4a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:37.665627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1feb8-3291-490e-bbb1-87925b26bdf2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:37.668597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-91a1feb8-3291-490e-bbb1-87925b26bdf2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:37.679913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:37.757419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:37.767918 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2500/5063] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:05:37.769031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfee44f-94f8-48d6-9e5b-4d472a5f0fd9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:37.772996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0bfee44f-94f8-48d6-9e5b-4d472a5f0fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:37.814373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfee44f-94f8-48d6-9e5b-4d472a5f0fd9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:37.912034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1feb8-3291-490e-bbb1-87925b26bdf2 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:37.932009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9dac8-72da-4b19-9995-a19839137a96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:37.932738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9dac8-72da-4b19-9995-a19839137a96 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:37.933084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9dac8-72da-4b19-9995-a19839137a96 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:37.933424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9dac8-72da-4b19-9995-a19839137a96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:37.940425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1bb9dac8-72da-4b19-9995-a19839137a96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:37.942152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4ebfa-7305-4854-bd11-d469f9e4a343 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:37.942947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4ebfa-7305-4854-bd11-d469f9e4a343 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:37.943389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4ebfa-7305-4854-bd11-d469f9e4a343 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:37.943746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4ebfa-7305-4854-bd11-d469f9e4a343 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:37.944562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4ebfa-7305-4854-bd11-d469f9e4a343 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:37.946627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4ebfa-7305-4854-bd11-d469f9e4a343 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:37.980818 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2501/5064] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:05:37 2022] POST /identity/v3/roles => generated 240 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:37.980818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4ebfa-7305-4854-bd11-d469f9e4a343 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:38.111755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9dac8-72da-4b19-9995-a19839137a96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:38.114443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9dac8-72da-4b19-9995-a19839137a96 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:38.150692 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2564/5065] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:37 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:05:38.151993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9dac8-72da-4b19-9995-a19839137a96 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:38.168474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9dac8-72da-4b19-9995-a19839137a96 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:38.191577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44f7f7-2029-4f79-965d-70b0b737baf9 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:38.192377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44f7f7-2029-4f79-965d-70b0b737baf9 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:38.192727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44f7f7-2029-4f79-965d-70b0b737baf9 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:38.193709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44f7f7-2029-4f79-965d-70b0b737baf9 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/4b47492528f647a48c4fbe9c927e7e4d/groups/44d87a83a5a3478e8bd9fddf037e80c9/roles/e8391730657b4acd8fc6fb374c908352/inherited_to_projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:38.197323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44f7f7-2029-4f79-965d-70b0b737baf9 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=4b47492528f647a48c4fbe9c927e7e4d, group_id=44d87a83a5a3478e8bd9fddf037e80c9, role_id=e8391730657b4acd8fc6fb374c908352)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:38.200357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44f7f7-2029-4f79-965d-70b0b737baf9 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:38.206260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4ebfa-7305-4854-bd11-d469f9e4a343 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:38.228272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e792f-991d-4623-b87d-c14717f4a5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:38.229057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e792f-991d-4623-b87d-c14717f4a5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:38.229439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e792f-991d-4623-b87d-c14717f4a5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:38.229734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e792f-991d-4623-b87d-c14717f4a5d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:38.230857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e792f-991d-4623-b87d-c14717f4a5d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:05:38.230857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e792f-991d-4623-b87d-c14717f4a5d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:38.240867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e792f-991d-4623-b87d-c14717f4a5d1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:38.258689 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2565/5066] 10.222.0.17 () {60 vars in 1582 bytes} [Sat Oct 1 21:05:38 2022] PUT /identity/v3/OS-INHERIT/domains/4b47492528f647a48c4fbe9c927e7e4d/groups/44d87a83a5a3478e8bd9fddf037e80c9/roles/e8391730657b4acd8fc6fb374c908352/inherited_to_projects => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:38.259919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44f7f7-2029-4f79-965d-70b0b737baf9 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:38.262205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1e44f7f7-2029-4f79-965d-70b0b737baf9 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:38.320337 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2502/5067] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:38.320337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e792f-991d-4623-b87d-c14717f4a5d1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:38.362946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e792f-991d-4623-b87d-c14717f4a5d1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:38.427845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44f7f7-2029-4f79-965d-70b0b737baf9 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:38.444865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d053582-1cbf-467f-9da0-b44d5ba66aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:38.445532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d053582-1cbf-467f-9da0-b44d5ba66aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:38.447463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d053582-1cbf-467f-9da0-b44d5ba66aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:38.447975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d053582-1cbf-467f-9da0-b44d5ba66aaa admin admin] PATH_INFO: `/v3/users/a4e3cd820f0e49cc969a94cb48422e42` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:38.452534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d053582-1cbf-467f-9da0-b44d5ba66aaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4e3cd820f0e49cc969a94cb48422e42)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:38.454392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d053582-1cbf-467f-9da0-b44d5ba66aaa admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:38.539549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0d053582-1cbf-467f-9da0-b44d5ba66aaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4e3cd820f0e49cc969a94cb48422e42'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:38.550851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:38.564143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0d053582-1cbf-467f-9da0-b44d5ba66aaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4e3cd820f0e49cc969a94cb48422e42'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:38.571734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3ce9a-b0a6-45b4-a038-8347563db137 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:38.572452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3ce9a-b0a6-45b4-a038-8347563db137 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:38.572738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3ce9a-b0a6-45b4-a038-8347563db137 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:38.573922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3ce9a-b0a6-45b4-a038-8347563db137 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/4b47492528f647a48c4fbe9c927e7e4d/groups/44d87a83a5a3478e8bd9fddf037e80c9/roles/inherited_to_projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:38.580478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3ce9a-b0a6-45b4-a038-8347563db137 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=4b47492528f647a48c4fbe9c927e7e4d, group_id=44d87a83a5a3478e8bd9fddf037e80c9)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:38.587079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3ce9a-b0a6-45b4-a038-8347563db137 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:38.599187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0d053582-1cbf-467f-9da0-b44d5ba66aaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4e3cd820f0e49cc969a94cb48422e42'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:38.600578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0d053582-1cbf-467f-9da0-b44d5ba66aaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4e3cd820f0e49cc969a94cb48422e42'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:38.612909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:38.617165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0d053582-1cbf-467f-9da0-b44d5ba66aaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4e3cd820f0e49cc969a94cb48422e42'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:38.621216 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2503/5068] 10.222.0.17 () {58 vars in 1464 bytes} [Sat Oct 1 21:05:38 2022] GET /identity/v3/OS-INHERIT/domains/4b47492528f647a48c4fbe9c927e7e4d/groups/44d87a83a5a3478e8bd9fddf037e80c9/roles/inherited_to_projects => generated 448 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:38.625647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3ce9a-b0a6-45b4-a038-8347563db137 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:38.627679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-54a3ce9a-b0a6-45b4-a038-8347563db137 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:38.630641 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2566/5069] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:38 2022] DELETE /identity/v3/users/a4e3cd820f0e49cc969a94cb48422e42 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:38.631747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d053582-1cbf-467f-9da0-b44d5ba66aaa admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:38.634566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0d053582-1cbf-467f-9da0-b44d5ba66aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:38.662828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3ce9a-b0a6-45b4-a038-8347563db137 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:38.759894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d053582-1cbf-467f-9da0-b44d5ba66aaa admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:38.771054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fee870-22b8-4a70-af2f-a39a185b61bc tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:38.771898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fee870-22b8-4a70-af2f-a39a185b61bc tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:38.772288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fee870-22b8-4a70-af2f-a39a185b61bc tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:38.772670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fee870-22b8-4a70-af2f-a39a185b61bc tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/4b47492528f647a48c4fbe9c927e7e4d/groups/44d87a83a5a3478e8bd9fddf037e80c9/roles/e8391730657b4acd8fc6fb374c908352/inherited_to_projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:38.777246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fee870-22b8-4a70-af2f-a39a185b61bc tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=4b47492528f647a48c4fbe9c927e7e4d, group_id=44d87a83a5a3478e8bd9fddf037e80c9, role_id=e8391730657b4acd8fc6fb374c908352)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:38.780218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fee870-22b8-4a70-af2f-a39a185b61bc tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:38.796197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbffd43d-b38f-40d1-841a-21b3aa4f6e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:38.796927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffd43d-b38f-40d1-841a-21b3aa4f6e78 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:38.797283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffd43d-b38f-40d1-841a-21b3aa4f6e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:38.797633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffd43d-b38f-40d1-841a-21b3aa4f6e78 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:38.798347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbffd43d-b38f-40d1-841a-21b3aa4f6e78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:38.799397 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2567/5070] 10.222.0.17 () {58 vars in 1564 bytes} [Sat Oct 1 21:05:38 2022] HEAD /identity/v3/OS-INHERIT/domains/4b47492528f647a48c4fbe9c927e7e4d/groups/44d87a83a5a3478e8bd9fddf037e80c9/roles/e8391730657b4acd8fc6fb374c908352/inherited_to_projects => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:38.801500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbffd43d-b38f-40d1-841a-21b3aa4f6e78 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:38.802474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fda05e-39d1-4fe8-a146-026d1fd45a8f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:38.803010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fda05e-39d1-4fe8-a146-026d1fd45a8f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:38.803443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fda05e-39d1-4fe8-a146-026d1fd45a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:38.842412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f6fda05e-39d1-4fe8-a146-026d1fd45a8f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:38.844286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f6fda05e-39d1-4fe8-a146-026d1fd45a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:38.870451 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2568/5071] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:38.871667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fda05e-39d1-4fe8-a146-026d1fd45a8f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:38.879574 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2504/5072] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:38 2022] POST /identity/v3/projects => generated 379 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:38.884439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbffd43d-b38f-40d1-841a-21b3aa4f6e78 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:38.887679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fda05e-39d1-4fe8-a146-026d1fd45a8f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:38.985589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45042c1-0982-4b2b-b640-892c5cba5c28 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:38.986631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45042c1-0982-4b2b-b640-892c5cba5c28 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:38.987038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45042c1-0982-4b2b-b640-892c5cba5c28 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:38.987455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45042c1-0982-4b2b-b640-892c5cba5c28 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/4b47492528f647a48c4fbe9c927e7e4d/groups/44d87a83a5a3478e8bd9fddf037e80c9/roles/e8391730657b4acd8fc6fb374c908352/inherited_to_projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:38.990897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45042c1-0982-4b2b-b640-892c5cba5c28 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=4b47492528f647a48c4fbe9c927e7e4d, group_id=44d87a83a5a3478e8bd9fddf037e80c9, role_id=e8391730657b4acd8fc6fb374c908352)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:38.993783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45042c1-0982-4b2b-b640-892c5cba5c28 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:39.004932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f45042c1-0982-4b2b-b640-892c5cba5c28 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invalidating the token cache because role e8391730657b4acd8fc6fb374c908352 was removed from group 44d87a83a5a3478e8bd9fddf037e80c9 on domain 4b47492528f647a48c4fbe9c927e7e4d. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:39.005295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f45042c1-0982-4b2b-b640-892c5cba5c28 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:39.006716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f45042c1-0982-4b2b-b640-892c5cba5c28 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:39.037195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbffd43d-b38f-40d1-841a-21b3aa4f6e78 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:39.037737 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2569/5073] 10.222.0.17 () {58 vars in 1566 bytes} [Sat Oct 1 21:05:38 2022] DELETE /identity/v3/OS-INHERIT/domains/4b47492528f647a48c4fbe9c927e7e4d/groups/44d87a83a5a3478e8bd9fddf037e80c9/roles/e8391730657b4acd8fc6fb374c908352/inherited_to_projects => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:39.038705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45042c1-0982-4b2b-b640-892c5cba5c28 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:39.040223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f45042c1-0982-4b2b-b640-892c5cba5c28 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:39.132639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433af3d-800f-43ea-b524-629692b4bf5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:39.133276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433af3d-800f-43ea-b524-629692b4bf5e admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:39.133531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45042c1-0982-4b2b-b640-892c5cba5c28 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:39.133634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433af3d-800f-43ea-b524-629692b4bf5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:39.133822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433af3d-800f-43ea-b524-629692b4bf5e admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:39.135432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433af3d-800f-43ea-b524-629692b4bf5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:39.138256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433af3d-800f-43ea-b524-629692b4bf5e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:39.146851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4de1e-639a-4e06-8e53-e8cfb35bf81e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:39.147831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4de1e-639a-4e06-8e53-e8cfb35bf81e service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:39.148332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4de1e-639a-4e06-8e53-e8cfb35bf81e service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:39.148800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4de1e-639a-4e06-8e53-e8cfb35bf81e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:39.152518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b8d4de1e-639a-4e06-8e53-e8cfb35bf81e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:39.171179 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2505/5074] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:38 2022] POST /identity/v3/users => generated 290 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:39.172138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433af3d-800f-43ea-b524-629692b4bf5e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:39.174200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b433af3d-800f-43ea-b524-629692b4bf5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:39.178942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:39.187438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4de1e-639a-4e06-8e53-e8cfb35bf81e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:39.190588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4de1e-639a-4e06-8e53-e8cfb35bf81e service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:39.216912 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2570/5075] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:05:39.222248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:39.224581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4de1e-639a-4e06-8e53-e8cfb35bf81e service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:39.227177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b8d4de1e-639a-4e06-8e53-e8cfb35bf81e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:39.262029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4de1e-639a-4e06-8e53-e8cfb35bf81e service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:39.271215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ded1b-ccb3-477f-a245-10ebc474e30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:39.272514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ded1b-ccb3-477f-a245-10ebc474e30c admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:39.272895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ded1b-ccb3-477f-a245-10ebc474e30c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:39.273242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ded1b-ccb3-477f-a245-10ebc474e30c admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:39.274166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7ded1b-ccb3-477f-a245-10ebc474e30c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:39.277764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7ded1b-ccb3-477f-a245-10ebc474e30c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:39.294539 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2571/5076] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:39 2022] GET /identity/v3/roles => generated 1651 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:39.298626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ded1b-ccb3-477f-a245-10ebc474e30c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:39.329083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433af3d-800f-43ea-b524-629692b4bf5e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:39.336315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ded1b-ccb3-477f-a245-10ebc474e30c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:39.336489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906f520-f979-44e1-a8fe-e3626427e19d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:39.337271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906f520-f979-44e1-a8fe-e3626427e19d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:39.337636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906f520-f979-44e1-a8fe-e3626427e19d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:39.338007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906f520-f979-44e1-a8fe-e3626427e19d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/roles/e8391730657b4acd8fc6fb374c908352` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:39.339940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906f520-f979-44e1-a8fe-e3626427e19d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:delete_role(role_id=e8391730657b4acd8fc6fb374c908352)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:39.341904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906f520-f979-44e1-a8fe-e3626427e19d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:39.351547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f57c5-ddf7-4ede-902f-4b75ba88dddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:39.353211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f57c5-ddf7-4ede-902f-4b75ba88dddd admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:39.358210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f57c5-ddf7-4ede-902f-4b75ba88dddd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:39.358699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f57c5-ddf7-4ede-902f-4b75ba88dddd admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:39.360204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f57c5-ddf7-4ede-902f-4b75ba88dddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:39.364822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f57c5-ddf7-4ede-902f-4b75ba88dddd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:39.399351 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2572/5077] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:39 2022] GET /identity/v3/roles => generated 1651 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:39.404064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f57c5-ddf7-4ede-902f-4b75ba88dddd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:39.432969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7906f520-f979-44e1-a8fe-e3626427e19d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invalidating the token cache because role e8391730657b4acd8fc6fb374c908352 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:39.433411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7906f520-f979-44e1-a8fe-e3626427e19d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:39.434198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7906f520-f979-44e1-a8fe-e3626427e19d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:39.436977 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2506/5078] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:39 2022] DELETE /identity/v3/roles/e8391730657b4acd8fc6fb374c908352 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:39.441903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906f520-f979-44e1-a8fe-e3626427e19d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:39.444014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7906f520-f979-44e1-a8fe-e3626427e19d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:39.452479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f57c5-ddf7-4ede-902f-4b75ba88dddd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:39.576724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906f520-f979-44e1-a8fe-e3626427e19d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:39.586209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99964eca-ee3b-4bc5-943c-1f650091b74d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:39.588034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99964eca-ee3b-4bc5-943c-1f650091b74d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:39.588404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99964eca-ee3b-4bc5-943c-1f650091b74d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:39.588841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99964eca-ee3b-4bc5-943c-1f650091b74d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:39.589956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99964eca-ee3b-4bc5-943c-1f650091b74d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:39.593240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99964eca-ee3b-4bc5-943c-1f650091b74d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:39.627769 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2507/5079] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:05:39 2022] POST /identity/v3/roles => generated 239 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:39.631934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99964eca-ee3b-4bc5-943c-1f650091b74d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:39.637360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5234e8a-056a-45fd-b45f-b4a917c39cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:39.639986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5234e8a-056a-45fd-b45f-b4a917c39cf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:39.642172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5234e8a-056a-45fd-b45f-b4a917c39cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:39.642598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5234e8a-056a-45fd-b45f-b4a917c39cf6 admin admin] PATH_INFO: `/v3/projects/3242ac5d0c3146b0bed8fd2f2e48584b/users/de8c90c2e17c4fb799678e81e57d9765/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:39.662208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99964eca-ee3b-4bc5-943c-1f650091b74d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:39.677118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf45e4-8537-48ed-bb2f-9528ff28e125 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:39.680176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf45e4-8537-48ed-bb2f-9528ff28e125 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:39.680608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf45e4-8537-48ed-bb2f-9528ff28e125 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:39.681152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf45e4-8537-48ed-bb2f-9528ff28e125 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/4b47492528f647a48c4fbe9c927e7e4d/users/9c9383ae72e647278c4617af86a60544/roles/b00bb84185934cf7885c863bfac0368a/inherited_to_projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:39.685563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5234e8a-056a-45fd-b45f-b4a917c39cf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3242ac5d0c3146b0bed8fd2f2e48584b, user_id=de8c90c2e17c4fb799678e81e57d9765, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:39.686615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf45e4-8537-48ed-bb2f-9528ff28e125 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=4b47492528f647a48c4fbe9c927e7e4d, user_id=9c9383ae72e647278c4617af86a60544, role_id=b00bb84185934cf7885c863bfac0368a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:39.692487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf45e4-8537-48ed-bb2f-9528ff28e125 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:39.693477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5234e8a-056a-45fd-b45f-b4a917c39cf6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:39.718795 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2508/5080] 10.222.0.17 () {60 vars in 1579 bytes} [Sat Oct 1 21:05:39 2022] PUT /identity/v3/OS-INHERIT/domains/4b47492528f647a48c4fbe9c927e7e4d/users/9c9383ae72e647278c4617af86a60544/roles/b00bb84185934cf7885c863bfac0368a/inherited_to_projects => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:39.723101 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2573/5081] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:39 2022] PUT /identity/v3/projects/3242ac5d0c3146b0bed8fd2f2e48584b/users/de8c90c2e17c4fb799678e81e57d9765/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:39.723701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf45e4-8537-48ed-bb2f-9528ff28e125 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:39.725018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35079569-284c-4874-8f87-15411c69f965 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:39.726127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35079569-284c-4874-8f87-15411c69f965 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:39.726127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35079569-284c-4874-8f87-15411c69f965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:39.802171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-35079569-284c-4874-8f87-15411c69f965 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:39.803460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-35079569-284c-4874-8f87-15411c69f965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:39.919177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf45e4-8537-48ed-bb2f-9528ff28e125 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:39.942895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60145c86-33d6-4450-b455-c2a1f0699e58 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:39.943719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60145c86-33d6-4450-b455-c2a1f0699e58 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:39.944197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60145c86-33d6-4450-b455-c2a1f0699e58 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:39.945444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60145c86-33d6-4450-b455-c2a1f0699e58 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/4b47492528f647a48c4fbe9c927e7e4d/users/9c9383ae72e647278c4617af86a60544/roles/inherited_to_projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:39.950927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60145c86-33d6-4450-b455-c2a1f0699e58 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=4b47492528f647a48c4fbe9c927e7e4d, user_id=9c9383ae72e647278c4617af86a60544)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:39.958993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60145c86-33d6-4450-b455-c2a1f0699e58 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:39.974887 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2574/5082] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:39.985069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:39.992014 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2509/5083] 10.222.0.17 () {58 vars in 1461 bytes} [Sat Oct 1 21:05:39 2022] GET /identity/v3/OS-INHERIT/domains/4b47492528f647a48c4fbe9c927e7e4d/users/9c9383ae72e647278c4617af86a60544/roles/inherited_to_projects => generated 446 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:39.995926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35079569-284c-4874-8f87-15411c69f965 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:40.011745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60145c86-33d6-4450-b455-c2a1f0699e58 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:40.012307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35079569-284c-4874-8f87-15411c69f965 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:40.016117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-60145c86-33d6-4450-b455-c2a1f0699e58 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:40.026156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa73be-0029-45e5-8958-302e95bb675a tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:40.027534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa73be-0029-45e5-8958-302e95bb675a tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:40.027819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa73be-0029-45e5-8958-302e95bb675a tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:40.028096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa73be-0029-45e5-8958-302e95bb675a tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/4b47492528f647a48c4fbe9c927e7e4d/users/9c9383ae72e647278c4617af86a60544/roles/b00bb84185934cf7885c863bfac0368a/inherited_to_projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:40.032027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa73be-0029-45e5-8958-302e95bb675a tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=4b47492528f647a48c4fbe9c927e7e4d, user_id=9c9383ae72e647278c4617af86a60544, role_id=b00bb84185934cf7885c863bfac0368a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:40.052927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa73be-0029-45e5-8958-302e95bb675a tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:40.068801 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2575/5084] 10.222.0.17 () {58 vars in 1561 bytes} [Sat Oct 1 21:05:39 2022] HEAD /identity/v3/OS-INHERIT/domains/4b47492528f647a48c4fbe9c927e7e4d/users/9c9383ae72e647278c4617af86a60544/roles/b00bb84185934cf7885c863bfac0368a/inherited_to_projects => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:40.070104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa73be-0029-45e5-8958-302e95bb675a tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:40.100115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa73be-0029-45e5-8958-302e95bb675a tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:40.115675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5f1cd-5e17-4c5c-95cf-40340713d31d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:40.116540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5f1cd-5e17-4c5c-95cf-40340713d31d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:40.118683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5f1cd-5e17-4c5c-95cf-40340713d31d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:40.118987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5f1cd-5e17-4c5c-95cf-40340713d31d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/4b47492528f647a48c4fbe9c927e7e4d/users/9c9383ae72e647278c4617af86a60544/roles/b00bb84185934cf7885c863bfac0368a/inherited_to_projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:40.131815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5f1cd-5e17-4c5c-95cf-40340713d31d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=4b47492528f647a48c4fbe9c927e7e4d, user_id=9c9383ae72e647278c4617af86a60544, role_id=b00bb84185934cf7885c863bfac0368a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:40.136029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5f1cd-5e17-4c5c-95cf-40340713d31d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:40.161251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-97a5f1cd-5e17-4c5c-95cf-40340713d31d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invalidating the token cache because role b00bb84185934cf7885c863bfac0368a was removed from user 9c9383ae72e647278c4617af86a60544 on domain 4b47492528f647a48c4fbe9c927e7e4d. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:40.161706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-97a5f1cd-5e17-4c5c-95cf-40340713d31d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:40.162982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-97a5f1cd-5e17-4c5c-95cf-40340713d31d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:40.172754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60145c86-33d6-4450-b455-c2a1f0699e58 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:40.183984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46a447-d28e-4acb-938a-04bbe1091f1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:40.185990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46a447-d28e-4acb-938a-04bbe1091f1e service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:40.186995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46a447-d28e-4acb-938a-04bbe1091f1e service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:40.187285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46a447-d28e-4acb-938a-04bbe1091f1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:40.191504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4c46a447-d28e-4acb-938a-04bbe1091f1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:40.192735 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2576/5085] 10.222.0.17 () {58 vars in 1563 bytes} [Sat Oct 1 21:05:40 2022] DELETE /identity/v3/OS-INHERIT/domains/4b47492528f647a48c4fbe9c927e7e4d/users/9c9383ae72e647278c4617af86a60544/roles/b00bb84185934cf7885c863bfac0368a/inherited_to_projects => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:40.196735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5f1cd-5e17-4c5c-95cf-40340713d31d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:40.201374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-97a5f1cd-5e17-4c5c-95cf-40340713d31d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:40.239374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46a447-d28e-4acb-938a-04bbe1091f1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:40.246363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46a447-d28e-4acb-938a-04bbe1091f1e service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:40.279473 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2510/5086] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:05:40.297325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:40.356135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5f1cd-5e17-4c5c-95cf-40340713d31d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:40.364972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75b358-4bfc-4a53-a2ab-b50fabb80839 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:40.365968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75b358-4bfc-4a53-a2ab-b50fabb80839 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:40.366332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75b358-4bfc-4a53-a2ab-b50fabb80839 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:40.366705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75b358-4bfc-4a53-a2ab-b50fabb80839 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/roles/b00bb84185934cf7885c863bfac0368a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:40.369001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75b358-4bfc-4a53-a2ab-b50fabb80839 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:delete_role(role_id=b00bb84185934cf7885c863bfac0368a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:40.370841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75b358-4bfc-4a53-a2ab-b50fabb80839 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:40.483408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3f75b358-4bfc-4a53-a2ab-b50fabb80839 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invalidating the token cache because role b00bb84185934cf7885c863bfac0368a has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:40.483892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3f75b358-4bfc-4a53-a2ab-b50fabb80839 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:40.484718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3f75b358-4bfc-4a53-a2ab-b50fabb80839 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:40.493519 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2577/5087] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:40 2022] DELETE /identity/v3/roles/b00bb84185934cf7885c863bfac0368a => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:40.498525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46a447-d28e-4acb-938a-04bbe1091f1e service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:40.523887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46a447-d28e-4acb-938a-04bbe1091f1e service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:40.664936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6b053-3ac9-4265-b8b0-85564f11019f tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:40.667453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6b053-3ac9-4265-b8b0-85564f11019f tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:40.667797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6b053-3ac9-4265-b8b0-85564f11019f tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:40.668390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6b053-3ac9-4265-b8b0-85564f11019f tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:40.670551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b6b053-3ac9-4265-b8b0-85564f11019f tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:40.674289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b6b053-3ac9-4265-b8b0-85564f11019f tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:40.712665 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2511/5088] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:05:40 2022] POST /identity/v3/roles => generated 240 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:40.713239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75b358-4bfc-4a53-a2ab-b50fabb80839 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:40.714804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3f75b358-4bfc-4a53-a2ab-b50fabb80839 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:40.740435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75b358-4bfc-4a53-a2ab-b50fabb80839 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:40.747164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b51bbe-0b77-4d26-9770-251f8cf4a650 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:40.747793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b51bbe-0b77-4d26-9770-251f8cf4a650 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:40.748076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b51bbe-0b77-4d26-9770-251f8cf4a650 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:40.748867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b51bbe-0b77-4d26-9770-251f8cf4a650 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:40.749520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b51bbe-0b77-4d26-9770-251f8cf4a650 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:40.755462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b51bbe-0b77-4d26-9770-251f8cf4a650 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:40.848665 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2578/5089] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:40 2022] POST /identity/v3/projects => generated 409 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:40.852863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6b053-3ac9-4265-b8b0-85564f11019f tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:40.868789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6b053-3ac9-4265-b8b0-85564f11019f tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:40.972664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26145bc-c302-4829-996f-76c902002758 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:40.973499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26145bc-c302-4829-996f-76c902002758 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:40.974118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26145bc-c302-4829-996f-76c902002758 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:40.974266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26145bc-c302-4829-996f-76c902002758 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/4b47492528f647a48c4fbe9c927e7e4d/users/9c9383ae72e647278c4617af86a60544/roles/3473a5848cf740d48957361d4ab21bad/inherited_to_projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:40.976941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26145bc-c302-4829-996f-76c902002758 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=4b47492528f647a48c4fbe9c927e7e4d, user_id=9c9383ae72e647278c4617af86a60544, role_id=3473a5848cf740d48957361d4ab21bad)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:40.981640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26145bc-c302-4829-996f-76c902002758 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:41.005398 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2512/5090] 10.222.0.17 () {60 vars in 1579 bytes} [Sat Oct 1 21:05:40 2022] PUT /identity/v3/OS-INHERIT/domains/4b47492528f647a48c4fbe9c927e7e4d/users/9c9383ae72e647278c4617af86a60544/roles/3473a5848cf740d48957361d4ab21bad/inherited_to_projects => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:41.009557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b51bbe-0b77-4d26-9770-251f8cf4a650 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:41.024908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b51bbe-0b77-4d26-9770-251f8cf4a650 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:41.119707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26145bc-c302-4829-996f-76c902002758 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:41.121667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f26145bc-c302-4829-996f-76c902002758 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:41.127247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3256d0-e3a5-4749-b4a0-8525d0c4ada9 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:41.128533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3256d0-e3a5-4749-b4a0-8525d0c4ada9 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:41.130302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3256d0-e3a5-4749-b4a0-8525d0c4ada9 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:41.130723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3256d0-e3a5-4749-b4a0-8525d0c4ada9 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:41.132671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3256d0-e3a5-4749-b4a0-8525d0c4ada9 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Adding query filter params (scope.project.id=c18eaa6e4f1a46b782e3c975baede48d, user.id=9c9383ae72e647278c4617af86a60544) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:05:41.143323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3256d0-e3a5-4749-b4a0-8525d0c4ada9 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:41.146122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3256d0-e3a5-4749-b4a0-8525d0c4ada9 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:41.183323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:41.231663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26145bc-c302-4829-996f-76c902002758 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:41.239021 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2579/5091] 10.222.0.17 () {58 vars in 1356 bytes} [Sat Oct 1 21:05:41 2022] GET /identity/v3/role_assignments?scope.project.id=c18eaa6e4f1a46b782e3c975baede48d&user.id=9c9383ae72e647278c4617af86a60544&effective => generated 648 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:41.241100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e6875-4301-425c-892f-4ba448dc26cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:41.243106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e6875-4301-425c-892f-4ba448dc26cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:41.243106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e6875-4301-425c-892f-4ba448dc26cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:41.243916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e6875-4301-425c-892f-4ba448dc26cf admin admin] PATH_INFO: `/v3/projects/79942f157bd4443bb422b629eea5b30f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:41.244918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e6875-4301-425c-892f-4ba448dc26cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=79942f157bd4443bb422b629eea5b30f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:41.248348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e6875-4301-425c-892f-4ba448dc26cf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:41.253727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3256d0-e3a5-4749-b4a0-8525d0c4ada9 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:41.273748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3256d0-e3a5-4749-b4a0-8525d0c4ada9 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:41.281842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4638f-acc9-4936-b41f-23c2ee261404 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:41.282898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4638f-acc9-4936-b41f-23c2ee261404 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:41.284023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4638f-acc9-4936-b41f-23c2ee261404 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:41.284522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4638f-acc9-4936-b41f-23c2ee261404 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:41.285804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4638f-acc9-4936-b41f-23c2ee261404 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Adding query filter params (scope.project.id=6e9bbaf81c8b443aa5d0c9c75a3aa7e2, user.id=9c9383ae72e647278c4617af86a60544) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:05:41.286315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4638f-acc9-4936-b41f-23c2ee261404 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:41.293513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4638f-acc9-4936-b41f-23c2ee261404 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:41.325262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-165e6875-4301-425c-892f-4ba448dc26cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79942f157bd4443bb422b629eea5b30f', 'request_id': 'req-165e6875-4301-425c-892f-4ba448dc26cf'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:41.327246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:41.350859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-165e6875-4301-425c-892f-4ba448dc26cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79942f157bd4443bb422b629eea5b30f', 'request_id': 'req-165e6875-4301-425c-892f-4ba448dc26cf'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:41.366394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-165e6875-4301-425c-892f-4ba448dc26cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79942f157bd4443bb422b629eea5b30f', 'request_id': 'req-165e6875-4301-425c-892f-4ba448dc26cf'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:41.398232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-165e6875-4301-425c-892f-4ba448dc26cf admin admin] The token cache is being invalidate because project 79942f157bd4443bb422b629eea5b30f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:41.398549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-165e6875-4301-425c-892f-4ba448dc26cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:41.399249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-165e6875-4301-425c-892f-4ba448dc26cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:41.404028 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2513/5092] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:41 2022] DELETE /identity/v3/projects/79942f157bd4443bb422b629eea5b30f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:41.409526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753ea34-998f-4749-b7cb-a2945823087f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:41.409818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753ea34-998f-4749-b7cb-a2945823087f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:41.410098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753ea34-998f-4749-b7cb-a2945823087f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:41.422770 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2580/5093] 10.222.0.17 () {58 vars in 1356 bytes} [Sat Oct 1 21:05:41 2022] GET /identity/v3/role_assignments?scope.project.id=6e9bbaf81c8b443aa5d0c9c75a3aa7e2&user.id=9c9383ae72e647278c4617af86a60544&effective => generated 648 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:41.424741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4638f-acc9-4936-b41f-23c2ee261404 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:41.427069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-07c4638f-acc9-4936-b41f-23c2ee261404 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:41.446202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4753ea34-998f-4749-b7cb-a2945823087f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:41.447523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4753ea34-998f-4749-b7cb-a2945823087f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:41.464642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4638f-acc9-4936-b41f-23c2ee261404 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:41.583131 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2514/5094] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:41.585441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967e945-e0cd-4e0d-83ca-60d9403afcb1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:41.585830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967e945-e0cd-4e0d-83ca-60d9403afcb1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:41.586211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967e945-e0cd-4e0d-83ca-60d9403afcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:41.628253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e510c-bc53-4df3-936c-7d454361cdf4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:41.630352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e510c-bc53-4df3-936c-7d454361cdf4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:41.630828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e510c-bc53-4df3-936c-7d454361cdf4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:41.631420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e510c-bc53-4df3-936c-7d454361cdf4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/4b47492528f647a48c4fbe9c927e7e4d/users/9c9383ae72e647278c4617af86a60544/roles/3473a5848cf740d48957361d4ab21bad/inherited_to_projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:41.633912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e510c-bc53-4df3-936c-7d454361cdf4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=4b47492528f647a48c4fbe9c927e7e4d, user_id=9c9383ae72e647278c4617af86a60544, role_id=3473a5848cf740d48957361d4ab21bad)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:41.636749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7967e945-e0cd-4e0d-83ca-60d9403afcb1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:41.638116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7967e945-e0cd-4e0d-83ca-60d9403afcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:41.640263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e510c-bc53-4df3-936c-7d454361cdf4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:41.647955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-140e510c-bc53-4df3-936c-7d454361cdf4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invalidating the token cache because role 3473a5848cf740d48957361d4ab21bad was removed from user 9c9383ae72e647278c4617af86a60544 on domain 4b47492528f647a48c4fbe9c927e7e4d. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:41.648521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-140e510c-bc53-4df3-936c-7d454361cdf4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:41.649567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-140e510c-bc53-4df3-936c-7d454361cdf4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:41.679633 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2581/5095] 10.222.0.17 () {58 vars in 1563 bytes} [Sat Oct 1 21:05:41 2022] DELETE /identity/v3/OS-INHERIT/domains/4b47492528f647a48c4fbe9c927e7e4d/users/9c9383ae72e647278c4617af86a60544/roles/3473a5848cf740d48957361d4ab21bad/inherited_to_projects => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:41.686625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e510c-bc53-4df3-936c-7d454361cdf4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:41.691197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-140e510c-bc53-4df3-936c-7d454361cdf4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:41.693704 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2515/5096] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:41.698925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967e945-e0cd-4e0d-83ca-60d9403afcb1 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:41.748185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967e945-e0cd-4e0d-83ca-60d9403afcb1 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:41.884367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e510c-bc53-4df3-936c-7d454361cdf4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:41.922275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa97cc9-ed33-4f73-bb57-f1c06656cd48 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:41.923364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa97cc9-ed33-4f73-bb57-f1c06656cd48 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:41.923795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa97cc9-ed33-4f73-bb57-f1c06656cd48 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:41.924210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa97cc9-ed33-4f73-bb57-f1c06656cd48 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:41.925938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa97cc9-ed33-4f73-bb57-f1c06656cd48 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Adding query filter params (scope.project.id=c18eaa6e4f1a46b782e3c975baede48d, user.id=9c9383ae72e647278c4617af86a60544) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:05:41.926402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa97cc9-ed33-4f73-bb57-f1c06656cd48 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:41.931786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:41.932547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa97cc9-ed33-4f73-bb57-f1c06656cd48 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:41.948310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984db17-54dc-4510-acc8-86b0907cdfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:41.948944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984db17-54dc-4510-acc8-86b0907cdfa2 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:41.949232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984db17-54dc-4510-acc8-86b0907cdfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:41.949505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984db17-54dc-4510-acc8-86b0907cdfa2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:41.950159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984db17-54dc-4510-acc8-86b0907cdfa2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:05:41.950460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984db17-54dc-4510-acc8-86b0907cdfa2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:41.957263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984db17-54dc-4510-acc8-86b0907cdfa2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:41.988452 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2516/5097] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:41.993281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984db17-54dc-4510-acc8-86b0907cdfa2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:42.017759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984db17-54dc-4510-acc8-86b0907cdfa2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:42.023237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f987c-ccb5-4b11-8bb7-b1c368f1e54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:42.023861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f987c-ccb5-4b11-8bb7-b1c368f1e54f admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:42.024141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f987c-ccb5-4b11-8bb7-b1c368f1e54f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:42.024411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f987c-ccb5-4b11-8bb7-b1c368f1e54f admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:42.030359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f987c-ccb5-4b11-8bb7-b1c368f1e54f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:42.030790 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2582/5098] 10.222.0.17 () {58 vars in 1356 bytes} [Sat Oct 1 21:05:41 2022] GET /identity/v3/role_assignments?scope.project.id=c18eaa6e4f1a46b782e3c975baede48d&user.id=9c9383ae72e647278c4617af86a60544&effective => generated 228 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:42.030891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f987c-ccb5-4b11-8bb7-b1c368f1e54f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:42.032773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa97cc9-ed33-4f73-bb57-f1c06656cd48 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:42.060854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa97cc9-ed33-4f73-bb57-f1c06656cd48 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:42.066616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea3bb6-4df1-43e2-8aef-2fd3fd7a7e9d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:42.067771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea3bb6-4df1-43e2-8aef-2fd3fd7a7e9d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:42.068651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea3bb6-4df1-43e2-8aef-2fd3fd7a7e9d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:42.069007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea3bb6-4df1-43e2-8aef-2fd3fd7a7e9d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:42.071304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea3bb6-4df1-43e2-8aef-2fd3fd7a7e9d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Adding query filter params (scope.project.id=6e9bbaf81c8b443aa5d0c9c75a3aa7e2, user.id=9c9383ae72e647278c4617af86a60544) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:05:42.072008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea3bb6-4df1-43e2-8aef-2fd3fd7a7e9d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:42.076111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea3bb6-4df1-43e2-8aef-2fd3fd7a7e9d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:42.091862 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2517/5099] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:41 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:42.093916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f987c-ccb5-4b11-8bb7-b1c368f1e54f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:42.102297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:42.216442 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2583/5100] 10.222.0.17 () {58 vars in 1356 bytes} [Sat Oct 1 21:05:42 2022] GET /identity/v3/role_assignments?scope.project.id=6e9bbaf81c8b443aa5d0c9c75a3aa7e2&user.id=9c9383ae72e647278c4617af86a60544&effective => generated 228 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:05:42.222718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea3bb6-4df1-43e2-8aef-2fd3fd7a7e9d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:42.248085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea3bb6-4df1-43e2-8aef-2fd3fd7a7e9d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:42.279418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f987c-ccb5-4b11-8bb7-b1c368f1e54f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:42.290763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd0186-c2f8-4395-acce-6ac15f1e1a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:42.292642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd0186-c2f8-4395-acce-6ac15f1e1a2c admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:42.293028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd0186-c2f8-4395-acce-6ac15f1e1a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:42.293428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd0186-c2f8-4395-acce-6ac15f1e1a2c admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:42.297733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd0186-c2f8-4395-acce-6ac15f1e1a2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:42.300778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd0186-c2f8-4395-acce-6ac15f1e1a2c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:42.361719 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2518/5101] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:42 2022] POST /identity/v3/users => generated 286 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:42.365543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd0186-c2f8-4395-acce-6ac15f1e1a2c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:42.380064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f93eca-60a4-421e-a4f2-0d46a9f43a8d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:42.380823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f93eca-60a4-421e-a4f2-0d46a9f43a8d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:42.381714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f93eca-60a4-421e-a4f2-0d46a9f43a8d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:42.381998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f93eca-60a4-421e-a4f2-0d46a9f43a8d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/projects/6e9bbaf81c8b443aa5d0c9c75a3aa7e2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:42.383373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f93eca-60a4-421e-a4f2-0d46a9f43a8d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6e9bbaf81c8b443aa5d0c9c75a3aa7e2)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:42.386181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f93eca-60a4-421e-a4f2-0d46a9f43a8d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:42.390441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd0186-c2f8-4395-acce-6ac15f1e1a2c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:42.400470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2ec38-d29a-4ed1-8de1-557ca617e6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:42.410125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2ec38-d29a-4ed1-8de1-557ca617e6ed admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:42.411689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2ec38-d29a-4ed1-8de1-557ca617e6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:42.412062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2ec38-d29a-4ed1-8de1-557ca617e6ed admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:42.412840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2ec38-d29a-4ed1-8de1-557ca617e6ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:42.418833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2ec38-d29a-4ed1-8de1-557ca617e6ed admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:42.448869 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2519/5102] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:42 2022] GET /identity/v3/roles => generated 1651 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:42.451001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2ec38-d29a-4ed1-8de1-557ca617e6ed admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:42.480196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2ec38-d29a-4ed1-8de1-557ca617e6ed admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:42.489580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c8f93eca-60a4-421e-a4f2-0d46a9f43a8d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e9bbaf81c8b443aa5d0c9c75a3aa7e2', 'request_id': 'req-c8f93eca-60a4-421e-a4f2-0d46a9f43a8d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:42.499271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c8f93eca-60a4-421e-a4f2-0d46a9f43a8d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e9bbaf81c8b443aa5d0c9c75a3aa7e2', 'request_id': 'req-c8f93eca-60a4-421e-a4f2-0d46a9f43a8d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:42.509569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c8f93eca-60a4-421e-a4f2-0d46a9f43a8d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e9bbaf81c8b443aa5d0c9c75a3aa7e2', 'request_id': 'req-c8f93eca-60a4-421e-a4f2-0d46a9f43a8d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:42.555582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c8f93eca-60a4-421e-a4f2-0d46a9f43a8d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] The token cache is being invalidate because project 6e9bbaf81c8b443aa5d0c9c75a3aa7e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:42.557826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c8f93eca-60a4-421e-a4f2-0d46a9f43a8d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:42.563348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c8f93eca-60a4-421e-a4f2-0d46a9f43a8d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:42.571125 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2584/5103] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:42 2022] DELETE /identity/v3/projects/6e9bbaf81c8b443aa5d0c9c75a3aa7e2 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:42.576615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f93eca-60a4-421e-a4f2-0d46a9f43a8d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:42.578791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c8f93eca-60a4-421e-a4f2-0d46a9f43a8d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:42.631001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b723e-53f1-4f65-a49e-009464e6c7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:42.636512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b723e-53f1-4f65-a49e-009464e6c7a0 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:42.637181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b723e-53f1-4f65-a49e-009464e6c7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:42.640798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b723e-53f1-4f65-a49e-009464e6c7a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:42.643600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b723e-53f1-4f65-a49e-009464e6c7a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:42.647261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b723e-53f1-4f65-a49e-009464e6c7a0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:42.660768 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2520/5104] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:42 2022] GET /identity/v3/roles => generated 1651 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:42.662326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517c298-abf5-48d1-8299-819c72d8cb68 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:42.662653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517c298-abf5-48d1-8299-819c72d8cb68 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:42.662941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517c298-abf5-48d1-8299-819c72d8cb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:42.702548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2517c298-abf5-48d1-8299-819c72d8cb68 None None] MFA Rules not processed for user `de8c90c2e17c4fb799678e81e57d9765`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:42.704031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2517c298-abf5-48d1-8299-819c72d8cb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:42.718044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f93eca-60a4-421e-a4f2-0d46a9f43a8d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:42.724576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99bba0f-6808-46c0-bb2b-9aa9e92a5709 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:42.725230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99bba0f-6808-46c0-bb2b-9aa9e92a5709 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:42.725516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99bba0f-6808-46c0-bb2b-9aa9e92a5709 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:42.725795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99bba0f-6808-46c0-bb2b-9aa9e92a5709 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/roles/3473a5848cf740d48957361d4ab21bad` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:42.727294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99bba0f-6808-46c0-bb2b-9aa9e92a5709 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:delete_role(role_id=3473a5848cf740d48957361d4ab21bad)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:42.729204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99bba0f-6808-46c0-bb2b-9aa9e92a5709 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:42.734282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:42.763406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:42.794104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a99bba0f-6808-46c0-bb2b-9aa9e92a5709 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invalidating the token cache because role 3473a5848cf740d48957361d4ab21bad has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:42.794412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a99bba0f-6808-46c0-bb2b-9aa9e92a5709 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:42.796157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a99bba0f-6808-46c0-bb2b-9aa9e92a5709 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:42.803207 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2585/5105] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:42 2022] DELETE /identity/v3/roles/3473a5848cf740d48957361d4ab21bad => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:42.804141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99bba0f-6808-46c0-bb2b-9aa9e92a5709 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:42.806416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a99bba0f-6808-46c0-bb2b-9aa9e92a5709 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:42.810263 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2521/5106] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:42.811524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517c298-abf5-48d1-8299-819c72d8cb68 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:42.824994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517c298-abf5-48d1-8299-819c72d8cb68 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:42.930967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a3d24-6bfd-426a-b801-bc394f2d14b1 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:42.931626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a3d24-6bfd-426a-b801-bc394f2d14b1 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:42.933195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a3d24-6bfd-426a-b801-bc394f2d14b1 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:42.933762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a3d24-6bfd-426a-b801-bc394f2d14b1 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:42.937045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a3d24-6bfd-426a-b801-bc394f2d14b1 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:42.938902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99bba0f-6808-46c0-bb2b-9aa9e92a5709 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:42.940881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a3d24-6bfd-426a-b801-bc394f2d14b1 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:42.953084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202154ca-b354-43e5-82d4-ce3f81d4ea55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:42.954169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202154ca-b354-43e5-82d4-ce3f81d4ea55 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:42.954289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202154ca-b354-43e5-82d4-ce3f81d4ea55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:42.954573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202154ca-b354-43e5-82d4-ce3f81d4ea55 admin admin] PATH_INFO: `/v3/projects/eb64a865764d4ec28a9b4a05183cf7a5/users/fb00e8aa19884664aeaad3c78733c9d5/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:42.974690 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2522/5107] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:05:42 2022] POST /identity/v3/roles => generated 240 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:42.975599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a3d24-6bfd-426a-b801-bc394f2d14b1 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:42.980584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e34a3d24-6bfd-426a-b801-bc394f2d14b1 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:42.997544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202154ca-b354-43e5-82d4-ce3f81d4ea55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb64a865764d4ec28a9b4a05183cf7a5, user_id=fb00e8aa19884664aeaad3c78733c9d5, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:43.012790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202154ca-b354-43e5-82d4-ce3f81d4ea55 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:43.026189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a3d24-6bfd-426a-b801-bc394f2d14b1 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:43.034057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a827d-c08b-4bef-bfde-3ffd98beebf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:43.034914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a827d-c08b-4bef-bfde-3ffd98beebf2 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:43.035574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a827d-c08b-4bef-bfde-3ffd98beebf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:43.035711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a827d-c08b-4bef-bfde-3ffd98beebf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:43.039793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a827d-c08b-4bef-bfde-3ffd98beebf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:43.041922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a827d-c08b-4bef-bfde-3ffd98beebf2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:43.050439 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2586/5108] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:42 2022] PUT /identity/v3/projects/eb64a865764d4ec28a9b4a05183cf7a5/users/fb00e8aa19884664aeaad3c78733c9d5/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:43.051503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202154ca-b354-43e5-82d4-ce3f81d4ea55 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:43.068143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-202154ca-b354-43e5-82d4-ce3f81d4ea55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:43.120743 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2523/5109] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:42 2022] POST /identity/v3/projects => generated 379 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:43.124807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f15157-79e5-47fa-b7c2-d382f40dc6f6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:43.125117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f15157-79e5-47fa-b7c2-d382f40dc6f6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:43.125401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f15157-79e5-47fa-b7c2-d382f40dc6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:43.181689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c4f15157-79e5-47fa-b7c2-d382f40dc6f6 None None] MFA Rules not processed for user `fb00e8aa19884664aeaad3c78733c9d5`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:43.183485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c4f15157-79e5-47fa-b7c2-d382f40dc6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:43.215795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202154ca-b354-43e5-82d4-ce3f81d4ea55 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:43.288879 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2524/5110] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:43.290623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f15157-79e5-47fa-b7c2-d382f40dc6f6 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:43.321189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:43.330897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5229d1c-ae9a-4b9a-9309-d119a2227a0c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:43.332594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5229d1c-ae9a-4b9a-9309-d119a2227a0c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:43.332986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5229d1c-ae9a-4b9a-9309-d119a2227a0c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:43.333396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5229d1c-ae9a-4b9a-9309-d119a2227a0c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:43.335008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5229d1c-ae9a-4b9a-9309-d119a2227a0c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:43.338142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5229d1c-ae9a-4b9a-9309-d119a2227a0c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:43.341436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:43.384570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f15157-79e5-47fa-b7c2-d382f40dc6f6 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:43.393516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90038a-6eac-4930-b3fb-dd03a40bf77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:43.394448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90038a-6eac-4930-b3fb-dd03a40bf77a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:43.394920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90038a-6eac-4930-b3fb-dd03a40bf77a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:43.395338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90038a-6eac-4930-b3fb-dd03a40bf77a admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:43.396248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90038a-6eac-4930-b3fb-dd03a40bf77a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:43.398667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90038a-6eac-4930-b3fb-dd03a40bf77a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:43.424268 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2587/5111] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:43 2022] POST /identity/v3/projects => generated 410 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:43.425371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5229d1c-ae9a-4b9a-9309-d119a2227a0c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:43.430881 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2525/5112] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:43 2022] POST /identity/v3/users => generated 296 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:43.434319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90038a-6eac-4930-b3fb-dd03a40bf77a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:43.473573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90038a-6eac-4930-b3fb-dd03a40bf77a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:43.568150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eeab0a-d019-48b5-921b-e221c1631ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:43.569046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eeab0a-d019-48b5-921b-e221c1631ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:43.569464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eeab0a-d019-48b5-921b-e221c1631ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:43.570013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eeab0a-d019-48b5-921b-e221c1631ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:43.570223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5229d1c-ae9a-4b9a-9309-d119a2227a0c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:43.571181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eeab0a-d019-48b5-921b-e221c1631ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:43.573255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eeab0a-d019-48b5-921b-e221c1631ef8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:43.576104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a032964-3d31-4130-a2d5-b9075b0ebb0d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:43.577336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a032964-3d31-4130-a2d5-b9075b0ebb0d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:43.577621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a032964-3d31-4130-a2d5-b9075b0ebb0d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:43.577897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a032964-3d31-4130-a2d5-b9075b0ebb0d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/c18eaa6e4f1a46b782e3c975baede48d/users/9c9383ae72e647278c4617af86a60544/roles/a486fafcb7aa47dc82300b9d0d829efc/inherited_to_projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:43.580320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a032964-3d31-4130-a2d5-b9075b0ebb0d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:create_grant(project_id=c18eaa6e4f1a46b782e3c975baede48d, user_id=9c9383ae72e647278c4617af86a60544, role_id=a486fafcb7aa47dc82300b9d0d829efc)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:43.583992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a032964-3d31-4130-a2d5-b9075b0ebb0d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:43.588607 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2526/5113] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:43 2022] GET /identity/v3/roles => generated 1651 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:05:43.593984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eeab0a-d019-48b5-921b-e221c1631ef8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:43.605552 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2588/5114] 10.222.0.17 () {60 vars in 1582 bytes} [Sat Oct 1 21:05:43 2022] PUT /identity/v3/OS-INHERIT/projects/c18eaa6e4f1a46b782e3c975baede48d/users/9c9383ae72e647278c4617af86a60544/roles/a486fafcb7aa47dc82300b9d0d829efc/inherited_to_projects => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:43.607161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eeab0a-d019-48b5-921b-e221c1631ef8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:43.610739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a032964-3d31-4130-a2d5-b9075b0ebb0d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:43.700387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a032964-3d31-4130-a2d5-b9075b0ebb0d tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:43.706077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296ea64-e46b-4ab5-a1f9-24ce3fe4c407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:43.707509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7743ea-f415-4271-b87d-79dd6f535c26 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:43.707779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7743ea-f415-4271-b87d-79dd6f535c26 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:43.708234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296ea64-e46b-4ab5-a1f9-24ce3fe4c407 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:43.708348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7743ea-f415-4271-b87d-79dd6f535c26 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:43.708631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7743ea-f415-4271-b87d-79dd6f535c26 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:43.708893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296ea64-e46b-4ab5-a1f9-24ce3fe4c407 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:43.709465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7743ea-f415-4271-b87d-79dd6f535c26 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Adding query filter params (scope.project.id=d3d12d9d3eb64361a0a698ecc2237110, user.id=9c9383ae72e647278c4617af86a60544) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:05:43.709661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7743ea-f415-4271-b87d-79dd6f535c26 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:43.710684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296ea64-e46b-4ab5-a1f9-24ce3fe4c407 admin admin] PATH_INFO: `/v3/projects/4640d180e2174808b0351f33abf87e3b/users/5e4c6ff5395246b292941af15d2040c0/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:43.711562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7743ea-f415-4271-b87d-79dd6f535c26 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:43.732879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4296ea64-e46b-4ab5-a1f9-24ce3fe4c407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4640d180e2174808b0351f33abf87e3b, user_id=5e4c6ff5395246b292941af15d2040c0, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:43.740397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4296ea64-e46b-4ab5-a1f9-24ce3fe4c407 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:43.770341 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2527/5115] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:43 2022] PUT /identity/v3/projects/4640d180e2174808b0351f33abf87e3b/users/5e4c6ff5395246b292941af15d2040c0/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:43.773906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296ea64-e46b-4ab5-a1f9-24ce3fe4c407 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:43.856144 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2589/5116] 10.222.0.17 () {58 vars in 1356 bytes} [Sat Oct 1 21:05:43 2022] GET /identity/v3/role_assignments?scope.project.id=d3d12d9d3eb64361a0a698ecc2237110&user.id=9c9383ae72e647278c4617af86a60544&effective => generated 649 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:43.856959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:43.858803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7743ea-f415-4271-b87d-79dd6f535c26 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:43.880833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7743ea-f415-4271-b87d-79dd6f535c26 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:43.911371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:43.930832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296ea64-e46b-4ab5-a1f9-24ce3fe4c407 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:43.946644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b88cbb-680f-4cac-a139-a7ccc278397b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:43.947523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b88cbb-680f-4cac-a139-a7ccc278397b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:43.948350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b88cbb-680f-4cac-a139-a7ccc278397b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:43.949188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b88cbb-680f-4cac-a139-a7ccc278397b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:43.950144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b88cbb-680f-4cac-a139-a7ccc278397b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:43.956630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b88cbb-680f-4cac-a139-a7ccc278397b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:43.989941 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2528/5117] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:43 2022] GET /identity/v3/roles => generated 1651 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:43.991051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b88cbb-680f-4cac-a139-a7ccc278397b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:44.004073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b88cbb-680f-4cac-a139-a7ccc278397b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:44.010611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98156553-e2b3-474d-8f50-e5ed99494bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:44.011293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98156553-e2b3-474d-8f50-e5ed99494bec admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:44.011649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98156553-e2b3-474d-8f50-e5ed99494bec admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:44.012149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98156553-e2b3-474d-8f50-e5ed99494bec admin admin] PATH_INFO: `/v3/projects/4640d180e2174808b0351f33abf87e3b/users/5e4c6ff5395246b292941af15d2040c0/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:44.014921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98156553-e2b3-474d-8f50-e5ed99494bec admin admin] RBAC: Authorizing `identity:create_grant(project_id=4640d180e2174808b0351f33abf87e3b, user_id=5e4c6ff5395246b292941af15d2040c0, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:44.017753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98156553-e2b3-474d-8f50-e5ed99494bec admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:44.024988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f679c-e5e2-4b69-93ae-eebb4421f2e4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:44.026277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f679c-e5e2-4b69-93ae-eebb4421f2e4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:44.028622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f679c-e5e2-4b69-93ae-eebb4421f2e4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:44.029164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f679c-e5e2-4b69-93ae-eebb4421f2e4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/c18eaa6e4f1a46b782e3c975baede48d/users/9c9383ae72e647278c4617af86a60544/roles/a486fafcb7aa47dc82300b9d0d829efc/inherited_to_projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:44.035161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f679c-e5e2-4b69-93ae-eebb4421f2e4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=c18eaa6e4f1a46b782e3c975baede48d, user_id=9c9383ae72e647278c4617af86a60544, role_id=a486fafcb7aa47dc82300b9d0d829efc)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:44.038387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f679c-e5e2-4b69-93ae-eebb4421f2e4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:44.052388 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2529/5118] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:43 2022] PUT /identity/v3/projects/4640d180e2174808b0351f33abf87e3b/users/5e4c6ff5395246b292941af15d2040c0/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:44.055615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-866f679c-e5e2-4b69-93ae-eebb4421f2e4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invalidating the token cache because role a486fafcb7aa47dc82300b9d0d829efc was removed from user 9c9383ae72e647278c4617af86a60544 on project c18eaa6e4f1a46b782e3c975baede48d. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:44.055615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-866f679c-e5e2-4b69-93ae-eebb4421f2e4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:44.056273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-866f679c-e5e2-4b69-93ae-eebb4421f2e4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:44.084018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98156553-e2b3-474d-8f50-e5ed99494bec admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:44.085368 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2590/5119] 10.222.0.17 () {58 vars in 1566 bytes} [Sat Oct 1 21:05:43 2022] DELETE /identity/v3/OS-INHERIT/projects/c18eaa6e4f1a46b782e3c975baede48d/users/9c9383ae72e647278c4617af86a60544/roles/a486fafcb7aa47dc82300b9d0d829efc/inherited_to_projects => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:44.087275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-98156553-e2b3-474d-8f50-e5ed99494bec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:44.225313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98156553-e2b3-474d-8f50-e5ed99494bec admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:44.309833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d19f38-a760-4856-a407-4d7ec7b156a7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:44.310649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d19f38-a760-4856-a407-4d7ec7b156a7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:44.311559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d19f38-a760-4856-a407-4d7ec7b156a7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:44.312018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d19f38-a760-4856-a407-4d7ec7b156a7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:44.313684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d19f38-a760-4856-a407-4d7ec7b156a7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Adding query filter params (scope.project.id=d3d12d9d3eb64361a0a698ecc2237110, user.id=9c9383ae72e647278c4617af86a60544) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:05:44.314220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d19f38-a760-4856-a407-4d7ec7b156a7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:44.318196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d19f38-a760-4856-a407-4d7ec7b156a7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:44.400745 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2530/5120] 10.222.0.17 () {58 vars in 1356 bytes} [Sat Oct 1 21:05:44 2022] GET /identity/v3/role_assignments?scope.project.id=d3d12d9d3eb64361a0a698ecc2237110&user.id=9c9383ae72e647278c4617af86a60544&effective => generated 228 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:44.405209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f679c-e5e2-4b69-93ae-eebb4421f2e4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:44.416383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f679c-e5e2-4b69-93ae-eebb4421f2e4 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:44.422061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988bc86f-3451-4a35-9f96-428e4983e9c7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:44.422803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bc86f-3451-4a35-9f96-428e4983e9c7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:44.423100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bc86f-3451-4a35-9f96-428e4983e9c7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:44.423378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bc86f-3451-4a35-9f96-428e4983e9c7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/projects/d3d12d9d3eb64361a0a698ecc2237110` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:44.424679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988bc86f-3451-4a35-9f96-428e4983e9c7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d3d12d9d3eb64361a0a698ecc2237110)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:44.427071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988bc86f-3451-4a35-9f96-428e4983e9c7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:44.495702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-988bc86f-3451-4a35-9f96-428e4983e9c7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3d12d9d3eb64361a0a698ecc2237110', 'request_id': 'req-988bc86f-3451-4a35-9f96-428e4983e9c7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:44.526641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-988bc86f-3451-4a35-9f96-428e4983e9c7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3d12d9d3eb64361a0a698ecc2237110', 'request_id': 'req-988bc86f-3451-4a35-9f96-428e4983e9c7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:44.540097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-988bc86f-3451-4a35-9f96-428e4983e9c7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3d12d9d3eb64361a0a698ecc2237110', 'request_id': 'req-988bc86f-3451-4a35-9f96-428e4983e9c7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:44.576122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-988bc86f-3451-4a35-9f96-428e4983e9c7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] The token cache is being invalidate because project d3d12d9d3eb64361a0a698ecc2237110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:44.576866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-988bc86f-3451-4a35-9f96-428e4983e9c7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:44.578461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-988bc86f-3451-4a35-9f96-428e4983e9c7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:44.582835 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2591/5121] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:44 2022] DELETE /identity/v3/projects/d3d12d9d3eb64361a0a698ecc2237110 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:44.589233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d19f38-a760-4856-a407-4d7ec7b156a7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:44.591577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e7d19f38-a760-4856-a407-4d7ec7b156a7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:44.617518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d19f38-a760-4856-a407-4d7ec7b156a7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:44.678570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988bc86f-3451-4a35-9f96-428e4983e9c7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:44.679995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:44.681601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-988bc86f-3451-4a35-9f96-428e4983e9c7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:44.730383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daf648-8e33-420b-9cea-4bab3b324fe6 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:44.732129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daf648-8e33-420b-9cea-4bab3b324fe6 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:44.733189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daf648-8e33-420b-9cea-4bab3b324fe6 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:44.733666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daf648-8e33-420b-9cea-4bab3b324fe6 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/roles/a486fafcb7aa47dc82300b9d0d829efc` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:44.735746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66daf648-8e33-420b-9cea-4bab3b324fe6 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:delete_role(role_id=a486fafcb7aa47dc82300b9d0d829efc)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:44.739028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66daf648-8e33-420b-9cea-4bab3b324fe6 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:44.783000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-66daf648-8e33-420b-9cea-4bab3b324fe6 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invalidating the token cache because role a486fafcb7aa47dc82300b9d0d829efc has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:44.783395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-66daf648-8e33-420b-9cea-4bab3b324fe6 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:44.784158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-66daf648-8e33-420b-9cea-4bab3b324fe6 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:44.789965 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2531/5122] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:44 2022] DELETE /identity/v3/roles/a486fafcb7aa47dc82300b9d0d829efc => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:05:44.791596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daf648-8e33-420b-9cea-4bab3b324fe6 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:44.793391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-66daf648-8e33-420b-9cea-4bab3b324fe6 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:44.887804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988bc86f-3451-4a35-9f96-428e4983e9c7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:44.935532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:44.944447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daf648-8e33-420b-9cea-4bab3b324fe6 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:44.963125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932a1d9-9b59-4ae4-9233-1ef518ab966b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:44.963829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932a1d9-9b59-4ae4-9233-1ef518ab966b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:44.965851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932a1d9-9b59-4ae4-9233-1ef518ab966b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:44.966042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932a1d9-9b59-4ae4-9233-1ef518ab966b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/users/9c9383ae72e647278c4617af86a60544` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:44.969020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932a1d9-9b59-4ae4-9233-1ef518ab966b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:delete_user(user_id=9c9383ae72e647278c4617af86a60544)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:44.971201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932a1d9-9b59-4ae4-9233-1ef518ab966b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:45.012290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2af78-eccb-4f95-a698-9963480a226c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:45.012992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2af78-eccb-4f95-a698-9963480a226c service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:45.013203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2af78-eccb-4f95-a698-9963480a226c service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:45.015592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2af78-eccb-4f95-a698-9963480a226c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:45.018880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f6f2af78-eccb-4f95-a698-9963480a226c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:45.064192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d932a1d9-9b59-4ae4-9233-1ef518ab966b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c9383ae72e647278c4617af86a60544'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:45.065210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d932a1d9-9b59-4ae4-9233-1ef518ab966b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c9383ae72e647278c4617af86a60544'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:45.065837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d932a1d9-9b59-4ae4-9233-1ef518ab966b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c9383ae72e647278c4617af86a60544'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:45.082586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d932a1d9-9b59-4ae4-9233-1ef518ab966b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c9383ae72e647278c4617af86a60544'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:45.106513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d932a1d9-9b59-4ae4-9233-1ef518ab966b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c9383ae72e647278c4617af86a60544'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:45.129309 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2532/5123] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:44 2022] DELETE /identity/v3/users/9c9383ae72e647278c4617af86a60544 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:05:45.130424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932a1d9-9b59-4ae4-9233-1ef518ab966b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:45.132123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d932a1d9-9b59-4ae4-9233-1ef518ab966b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:45.148860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2af78-eccb-4f95-a698-9963480a226c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:45.153599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2af78-eccb-4f95-a698-9963480a226c service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:45.169496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:45.170014 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2592/5124] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:05:45.287621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932a1d9-9b59-4ae4-9233-1ef518ab966b tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:45.293262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd1470-fadd-4b98-a0fb-bb863e0a0e6f tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:45.293993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd1470-fadd-4b98-a0fb-bb863e0a0e6f tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:45.294356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd1470-fadd-4b98-a0fb-bb863e0a0e6f tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:45.294745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd1470-fadd-4b98-a0fb-bb863e0a0e6f tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/groups/44d87a83a5a3478e8bd9fddf037e80c9` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:45.297332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd1470-fadd-4b98-a0fb-bb863e0a0e6f tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:delete_group(group_id=44d87a83a5a3478e8bd9fddf037e80c9)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:45.299788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd1470-fadd-4b98-a0fb-bb863e0a0e6f tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:45.392691 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2533/5125] 10.222.0.17 () {58 vars in 1227 bytes} [Sat Oct 1 21:05:45 2022] DELETE /identity/v3/groups/44d87a83a5a3478e8bd9fddf037e80c9 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:45.396440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2af78-eccb-4f95-a698-9963480a226c service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:45.411053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2af78-eccb-4f95-a698-9963480a226c service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:45.516496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c4073-77d7-405d-a693-cf26dc98a832 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:45.517591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c4073-77d7-405d-a693-cf26dc98a832 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:45.517591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c4073-77d7-405d-a693-cf26dc98a832 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:45.518015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c4073-77d7-405d-a693-cf26dc98a832 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/projects/c18eaa6e4f1a46b782e3c975baede48d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:45.519452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c4073-77d7-405d-a693-cf26dc98a832 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c18eaa6e4f1a46b782e3c975baede48d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:45.522061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c4073-77d7-405d-a693-cf26dc98a832 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:45.650561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a5c4073-77d7-405d-a693-cf26dc98a832 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c18eaa6e4f1a46b782e3c975baede48d', 'request_id': 'req-9a5c4073-77d7-405d-a693-cf26dc98a832'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:45.660389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a5c4073-77d7-405d-a693-cf26dc98a832 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c18eaa6e4f1a46b782e3c975baede48d', 'request_id': 'req-9a5c4073-77d7-405d-a693-cf26dc98a832'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:45.669128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a5c4073-77d7-405d-a693-cf26dc98a832 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c18eaa6e4f1a46b782e3c975baede48d', 'request_id': 'req-9a5c4073-77d7-405d-a693-cf26dc98a832'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:45.686862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:45.692430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a5c4073-77d7-405d-a693-cf26dc98a832 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] The token cache is being invalidate because project c18eaa6e4f1a46b782e3c975baede48d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:45.694008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a5c4073-77d7-405d-a693-cf26dc98a832 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:45.694449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a5c4073-77d7-405d-a693-cf26dc98a832 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:45.698993 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2593/5126] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:45 2022] DELETE /identity/v3/projects/c18eaa6e4f1a46b782e3c975baede48d => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:45.701100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd1470-fadd-4b98-a0fb-bb863e0a0e6f tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:45.703771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-55bd1470-fadd-4b98-a0fb-bb863e0a0e6f tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:45.724202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd1470-fadd-4b98-a0fb-bb863e0a0e6f tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:45.834571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4a8b2-ac84-4468-a049-f1df8b25cdce None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:45.835072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4a8b2-ac84-4468-a049-f1df8b25cdce None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:45.835493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4a8b2-ac84-4468-a049-f1df8b25cdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:45.846755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8de13-d3c2-47cf-85b7-cff50baa412c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:45.847696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8de13-d3c2-47cf-85b7-cff50baa412c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:45.850474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8de13-d3c2-47cf-85b7-cff50baa412c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:45.851030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8de13-d3c2-47cf-85b7-cff50baa412c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/domains/4b47492528f647a48c4fbe9c927e7e4d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:45.856230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8de13-d3c2-47cf-85b7-cff50baa412c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=4b47492528f647a48c4fbe9c927e7e4d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:45.858417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8de13-d3c2-47cf-85b7-cff50baa412c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:45.892454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-eba4a8b2-ac84-4468-a049-f1df8b25cdce None None] MFA Rules not processed for user `5e4c6ff5395246b292941af15d2040c0`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:45.895941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-eba4a8b2-ac84-4468-a049-f1df8b25cdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:45.910912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4fe8de13-d3c2-47cf-85b7-cff50baa412c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '4b47492528f647a48c4fbe9c927e7e4d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:45.913609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4fe8de13-d3c2-47cf-85b7-cff50baa412c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '4b47492528f647a48c4fbe9c927e7e4d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:45.966355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4fe8de13-d3c2-47cf-85b7-cff50baa412c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '4b47492528f647a48c4fbe9c927e7e4d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:45.968036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4fe8de13-d3c2-47cf-85b7-cff50baa412c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '4b47492528f647a48c4fbe9c927e7e4d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:45.972147 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2534/5127] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:05:45 2022] PATCH /identity/v3/domains/4b47492528f647a48c4fbe9c927e7e4d => generated 278 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:45.977335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8de13-d3c2-47cf-85b7-cff50baa412c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:45.980483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4fe8de13-d3c2-47cf-85b7-cff50baa412c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:46.063724 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2594/5128] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:05:46.066029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901fd06-f8db-43e3-813c-82aa265f9125 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:46.066325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901fd06-f8db-43e3-813c-82aa265f9125 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:46.066640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901fd06-f8db-43e3-813c-82aa265f9125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:46.087258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:UPDATE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:46.095531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9901fd06-f8db-43e3-813c-82aa265f9125 None None] MFA Rules not processed for user `de8c90c2e17c4fb799678e81e57d9765`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:46.096767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9901fd06-f8db-43e3-813c-82aa265f9125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:46.157253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8de13-d3c2-47cf-85b7-cff50baa412c tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:46.176322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b52786be44f04aaebae93c409a8debeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10aee3c37c164abea2ace622dd851605', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:46.176979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:46.177261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:46.177541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] PATH_INFO: `/v3/domains/4b47492528f647a48c4fbe9c927e7e4d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:46.204434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=4b47492528f647a48c4fbe9c927e7e4d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:46.205782 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2595/5129] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:46.206223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:46.249685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '4b47492528f647a48c4fbe9c927e7e4d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:46.266974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '4b47492528f647a48c4fbe9c927e7e4d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:46.329343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b47492528f647a48c4fbe9c927e7e4d', 'request_id': 'req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:46.353418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b47492528f647a48c4fbe9c927e7e4d', 'request_id': 'req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:46.364101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b47492528f647a48c4fbe9c927e7e4d', 'request_id': 'req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:46.378880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] The token cache is being invalidate because project 4b47492528f647a48c4fbe9c927e7e4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:46.379177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:46.379893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:46.400619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '4b47492528f647a48c4fbe9c927e7e4d', 'request_id': 'req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:46.401710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '4b47492528f647a48c4fbe9c927e7e4d', 'request_id': 'req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:46.406711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7 tempest-InheritsV3TestJSON-1911678894 tempest-InheritsV3TestJSON-1911678894-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '4b47492528f647a48c4fbe9c927e7e4d', 'request_id': 'req-e483b195-c3c1-4b20-9eb8-aa6ad8c48cc7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:46.414657 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2535/5130] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:05:45 2022] DELETE /identity/v3/domains/4b47492528f647a48c4fbe9c927e7e4d => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:46.417941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901fd06-f8db-43e3-813c-82aa265f9125 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:46.420259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9901fd06-f8db-43e3-813c-82aa265f9125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:46.516506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901fd06-f8db-43e3-813c-82aa265f9125 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:46.554681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:46.582732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1576f44-b4c5-4b3b-8b7e-b431f07aad15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:46.584033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1576f44-b4c5-4b3b-8b7e-b431f07aad15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:46.584321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1576f44-b4c5-4b3b-8b7e-b431f07aad15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:46.584612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1576f44-b4c5-4b3b-8b7e-b431f07aad15 admin admin] PATH_INFO: `/v3/users/dbb018c0c0a54b39b879f959a370a63a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:46.586005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1576f44-b4c5-4b3b-8b7e-b431f07aad15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbb018c0c0a54b39b879f959a370a63a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:46.588662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1576f44-b4c5-4b3b-8b7e-b431f07aad15 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:46.651775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e1576f44-b4c5-4b3b-8b7e-b431f07aad15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbb018c0c0a54b39b879f959a370a63a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:46.661876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e1576f44-b4c5-4b3b-8b7e-b431f07aad15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbb018c0c0a54b39b879f959a370a63a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:46.683061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e1576f44-b4c5-4b3b-8b7e-b431f07aad15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbb018c0c0a54b39b879f959a370a63a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:46.683941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e1576f44-b4c5-4b3b-8b7e-b431f07aad15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbb018c0c0a54b39b879f959a370a63a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:46.694666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e1576f44-b4c5-4b3b-8b7e-b431f07aad15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbb018c0c0a54b39b879f959a370a63a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:46.708609 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2596/5131] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:46 2022] DELETE /identity/v3/users/dbb018c0c0a54b39b879f959a370a63a => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:05:46.712893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e85bc3-51f6-4705-8cc2-9aad71b61d3a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:46.713188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e85bc3-51f6-4705-8cc2-9aad71b61d3a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:46.713465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e85bc3-51f6-4705-8cc2-9aad71b61d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:46.750024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-30e85bc3-51f6-4705-8cc2-9aad71b61d3a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:46.751296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-30e85bc3-51f6-4705-8cc2-9aad71b61d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:46.832972 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2536/5132] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:46.849311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1576f44-b4c5-4b3b-8b7e-b431f07aad15 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:46.851415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e1576f44-b4c5-4b3b-8b7e-b431f07aad15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:46.924213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1576f44-b4c5-4b3b-8b7e-b431f07aad15 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:46.928256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887e0c3-43d8-4997-88c6-110b72773c20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:46.928868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887e0c3-43d8-4997-88c6-110b72773c20 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:46.929152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887e0c3-43d8-4997-88c6-110b72773c20 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:46.930009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887e0c3-43d8-4997-88c6-110b72773c20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:46.939855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887e0c3-43d8-4997-88c6-110b72773c20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:46.941491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887e0c3-43d8-4997-88c6-110b72773c20 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:46.954054 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2597/5133] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:05:47.270413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e85bc3-51f6-4705-8cc2-9aad71b61d3a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:47.271932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-30e85bc3-51f6-4705-8cc2-9aad71b61d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:47.285566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e85bc3-51f6-4705-8cc2-9aad71b61d3a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:47.291504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f002b-611f-496e-91d3-f5bbb92b692c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:47.291943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f002b-611f-496e-91d3-f5bbb92b692c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:47.292226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f002b-611f-496e-91d3-f5bbb92b692c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:47.292500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f002b-611f-496e-91d3-f5bbb92b692c admin admin] PATH_INFO: `/v3/projects/5a2f68dc593c49249a1e86a2c0678b5a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:47.299544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f002b-611f-496e-91d3-f5bbb92b692c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a2f68dc593c49249a1e86a2c0678b5a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:47.302517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f002b-611f-496e-91d3-f5bbb92b692c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:47.316760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:47.360161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b06f002b-611f-496e-91d3-f5bbb92b692c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a2f68dc593c49249a1e86a2c0678b5a', 'request_id': 'req-b06f002b-611f-496e-91d3-f5bbb92b692c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:47.368240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b06f002b-611f-496e-91d3-f5bbb92b692c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a2f68dc593c49249a1e86a2c0678b5a', 'request_id': 'req-b06f002b-611f-496e-91d3-f5bbb92b692c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:47.376269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b06f002b-611f-496e-91d3-f5bbb92b692c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a2f68dc593c49249a1e86a2c0678b5a', 'request_id': 'req-b06f002b-611f-496e-91d3-f5bbb92b692c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:47.387887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b06f002b-611f-496e-91d3-f5bbb92b692c admin admin] The token cache is being invalidate because project 5a2f68dc593c49249a1e86a2c0678b5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:47.388188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b06f002b-611f-496e-91d3-f5bbb92b692c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:47.389376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b06f002b-611f-496e-91d3-f5bbb92b692c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:47.391007 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2537/5134] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:47 2022] DELETE /identity/v3/projects/5a2f68dc593c49249a1e86a2c0678b5a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:47.394931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887e0c3-43d8-4997-88c6-110b72773c20 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:47.407598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887e0c3-43d8-4997-88c6-110b72773c20 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:47.425110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f002b-611f-496e-91d3-f5bbb92b692c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:47.430702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b06f002b-611f-496e-91d3-f5bbb92b692c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:47.516051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42300cfb-c839-4fae-b908-1f3f081258c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:47.516824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42300cfb-c839-4fae-b908-1f3f081258c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:47.517107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42300cfb-c839-4fae-b908-1f3f081258c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:47.517381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42300cfb-c839-4fae-b908-1f3f081258c4 admin admin] PATH_INFO: `/v3/users/b52786be44f04aaebae93c409a8debeb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:47.518604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42300cfb-c839-4fae-b908-1f3f081258c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b52786be44f04aaebae93c409a8debeb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:47.520938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42300cfb-c839-4fae-b908-1f3f081258c4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:47.522932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:47.599514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-42300cfb-c839-4fae-b908-1f3f081258c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b52786be44f04aaebae93c409a8debeb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:47.605456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f002b-611f-496e-91d3-f5bbb92b692c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:47.611051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-42300cfb-c839-4fae-b908-1f3f081258c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b52786be44f04aaebae93c409a8debeb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:47.615367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609e7c9-ebd1-4a5c-975c-85f18c0e72c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:47.617079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609e7c9-ebd1-4a5c-975c-85f18c0e72c3 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:47.617581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609e7c9-ebd1-4a5c-975c-85f18c0e72c3 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:47.618455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609e7c9-ebd1-4a5c-975c-85f18c0e72c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:47.621312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4609e7c9-ebd1-4a5c-975c-85f18c0e72c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:47.630338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-42300cfb-c839-4fae-b908-1f3f081258c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b52786be44f04aaebae93c409a8debeb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:47.631908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-42300cfb-c839-4fae-b908-1f3f081258c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b52786be44f04aaebae93c409a8debeb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:47.648799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-42300cfb-c839-4fae-b908-1f3f081258c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b52786be44f04aaebae93c409a8debeb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:47.668820 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2598/5135] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:47 2022] DELETE /identity/v3/users/b52786be44f04aaebae93c409a8debeb => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:05:47.707490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609e7c9-ebd1-4a5c-975c-85f18c0e72c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:47.710874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609e7c9-ebd1-4a5c-975c-85f18c0e72c3 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:47.723260 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2538/5136] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:05:49.115681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42300cfb-c839-4fae-b908-1f3f081258c4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:49.117210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-42300cfb-c839-4fae-b908-1f3f081258c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:49.287832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42300cfb-c839-4fae-b908-1f3f081258c4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:49.309656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cb05f-510d-49bb-b8fd-6277930636b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:49.317193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cb05f-510d-49bb-b8fd-6277930636b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:49.318632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cb05f-510d-49bb-b8fd-6277930636b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:49.319040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cb05f-510d-49bb-b8fd-6277930636b3 admin admin] PATH_INFO: `/v3/projects/10aee3c37c164abea2ace622dd851605` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:49.321635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cb05f-510d-49bb-b8fd-6277930636b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10aee3c37c164abea2ace622dd851605)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:49.323932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cb05f-510d-49bb-b8fd-6277930636b3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:49.436150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-066cb05f-510d-49bb-b8fd-6277930636b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10aee3c37c164abea2ace622dd851605', 'request_id': 'req-066cb05f-510d-49bb-b8fd-6277930636b3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:49.458130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609e7c9-ebd1-4a5c-975c-85f18c0e72c3 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:49.461043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4609e7c9-ebd1-4a5c-975c-85f18c0e72c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:49.471243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-066cb05f-510d-49bb-b8fd-6277930636b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10aee3c37c164abea2ace622dd851605', 'request_id': 'req-066cb05f-510d-49bb-b8fd-6277930636b3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:49.485178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609e7c9-ebd1-4a5c-975c-85f18c0e72c3 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:49.486953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-066cb05f-510d-49bb-b8fd-6277930636b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10aee3c37c164abea2ace622dd851605', 'request_id': 'req-066cb05f-510d-49bb-b8fd-6277930636b3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:49.514716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-066cb05f-510d-49bb-b8fd-6277930636b3 admin admin] The token cache is being invalidate because project 10aee3c37c164abea2ace622dd851605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:49.516874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-066cb05f-510d-49bb-b8fd-6277930636b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:49.519727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-066cb05f-510d-49bb-b8fd-6277930636b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:49.522857 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2599/5137] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:49 2022] DELETE /identity/v3/projects/10aee3c37c164abea2ace622dd851605 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:49.530714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64489f3a-b707-4ec2-a550-7a63025f6a45 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:49.531001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64489f3a-b707-4ec2-a550-7a63025f6a45 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:49.531277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64489f3a-b707-4ec2-a550-7a63025f6a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:49.576641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:49.577589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-64489f3a-b707-4ec2-a550-7a63025f6a45 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:49.584718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-64489f3a-b707-4ec2-a550-7a63025f6a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:49.595645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:49.642315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eaa03e-facb-47a7-91d5-9bca014af01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:49.643009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaa03e-facb-47a7-91d5-9bca014af01e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:49.643453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaa03e-facb-47a7-91d5-9bca014af01e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:49.645773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaa03e-facb-47a7-91d5-9bca014af01e admin admin] PATH_INFO: `/v3/users/fb00e8aa19884664aeaad3c78733c9d5` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:49.649026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eaa03e-facb-47a7-91d5-9bca014af01e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb00e8aa19884664aeaad3c78733c9d5)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:49.651338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eaa03e-facb-47a7-91d5-9bca014af01e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:49.742377 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2600/5138] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:49.744133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26b57e-102a-473b-ae83-4b14cbbb36dc None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:49.744771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26b57e-102a-473b-ae83-4b14cbbb36dc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:49.745290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26b57e-102a-473b-ae83-4b14cbbb36dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:49.789905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89eaa03e-facb-47a7-91d5-9bca014af01e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb00e8aa19884664aeaad3c78733c9d5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:49.790954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89eaa03e-facb-47a7-91d5-9bca014af01e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb00e8aa19884664aeaad3c78733c9d5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:49.792086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89eaa03e-facb-47a7-91d5-9bca014af01e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb00e8aa19884664aeaad3c78733c9d5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:49.812944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89eaa03e-facb-47a7-91d5-9bca014af01e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb00e8aa19884664aeaad3c78733c9d5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:49.835802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0e26b57e-102a-473b-ae83-4b14cbbb36dc None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:49.840584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0e26b57e-102a-473b-ae83-4b14cbbb36dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:49.841102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-89eaa03e-facb-47a7-91d5-9bca014af01e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb00e8aa19884664aeaad3c78733c9d5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:49.869210 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2539/5139] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:49 2022] DELETE /identity/v3/users/fb00e8aa19884664aeaad3c78733c9d5 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:05:49.874835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb6437-7e79-45d0-9434-e9381a84a980 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:49.875435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb6437-7e79-45d0-9434-e9381a84a980 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:49.875882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb6437-7e79-45d0-9434-e9381a84a980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:49.900071 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2601/5140] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:49.902778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7acb6437-7e79-45d0-9434-e9381a84a980 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:49.905187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7acb6437-7e79-45d0-9434-e9381a84a980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:49.906803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26b57e-102a-473b-ae83-4b14cbbb36dc None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:49.929684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26b57e-102a-473b-ae83-4b14cbbb36dc None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:50.014638 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2540/5141] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:05:50.035287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb6437-7e79-45d0-9434-e9381a84a980 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:50.036825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7acb6437-7e79-45d0-9434-e9381a84a980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:50.110202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676636f9-2d52-40fb-87de-4f70ec47e712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:50.110866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676636f9-2d52-40fb-87de-4f70ec47e712 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:50.111148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676636f9-2d52-40fb-87de-4f70ec47e712 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:50.111418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676636f9-2d52-40fb-87de-4f70ec47e712 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:50.112075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676636f9-2d52-40fb-87de-4f70ec47e712 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:05:50.112380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676636f9-2d52-40fb-87de-4f70ec47e712 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:50.113982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676636f9-2d52-40fb-87de-4f70ec47e712 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:50.145199 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2602/5142] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:50.147026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676636f9-2d52-40fb-87de-4f70ec47e712 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:50.163212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676636f9-2d52-40fb-87de-4f70ec47e712 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:50.183653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035b21f-a7ac-4aa0-b4d1-da930f9a220f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:50.184620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035b21f-a7ac-4aa0-b4d1-da930f9a220f admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:50.185179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035b21f-a7ac-4aa0-b4d1-da930f9a220f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:50.185765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035b21f-a7ac-4aa0-b4d1-da930f9a220f admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:50.186606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035b21f-a7ac-4aa0-b4d1-da930f9a220f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:50.188670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035b21f-a7ac-4aa0-b4d1-da930f9a220f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:50.218087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb6437-7e79-45d0-9434-e9381a84a980 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:50.224548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b07bc-926b-432f-92dc-8529964860ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:50.225253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b07bc-926b-432f-92dc-8529964860ba service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:50.225601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b07bc-926b-432f-92dc-8529964860ba service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:50.225880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b07bc-926b-432f-92dc-8529964860ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:50.246451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b07bc-926b-432f-92dc-8529964860ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:50.249188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b07bc-926b-432f-92dc-8529964860ba service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:50.256059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:50.270801 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2603/5143] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:50 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:50.271819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035b21f-a7ac-4aa0-b4d1-da930f9a220f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:50.277232 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2541/5144] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:05:50.394590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035b21f-a7ac-4aa0-b4d1-da930f9a220f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:50.407621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35902ad-c8ff-42ef-b75c-74e12afeb52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:50.408449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35902ad-c8ff-42ef-b75c-74e12afeb52b admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:50.411262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35902ad-c8ff-42ef-b75c-74e12afeb52b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:50.412243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35902ad-c8ff-42ef-b75c-74e12afeb52b admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:50.413039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35902ad-c8ff-42ef-b75c-74e12afeb52b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:50.418261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35902ad-c8ff-42ef-b75c-74e12afeb52b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:50.476811 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2604/5145] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:50 2022] POST /identity/v3/users => generated 294 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:50.480983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b07bc-926b-432f-92dc-8529964860ba service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:50.519153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b07bc-926b-432f-92dc-8529964860ba service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:50.535377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e8c19-0c5e-4478-a7b2-5eac57f85a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:50.536010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e8c19-0c5e-4478-a7b2-5eac57f85a0f admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:50.536295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e8c19-0c5e-4478-a7b2-5eac57f85a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:50.536568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e8c19-0c5e-4478-a7b2-5eac57f85a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:50.537216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e8c19-0c5e-4478-a7b2-5eac57f85a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:50.539688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e8c19-0c5e-4478-a7b2-5eac57f85a0f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:50.551918 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2542/5146] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:50.556268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35902ad-c8ff-42ef-b75c-74e12afeb52b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:50.569625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:50.577296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35902ad-c8ff-42ef-b75c-74e12afeb52b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:50.585628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f130027-b764-4a64-af47-857a6898e16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:50.586257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f130027-b764-4a64-af47-857a6898e16c admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:50.586619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f130027-b764-4a64-af47-857a6898e16c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:50.586902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f130027-b764-4a64-af47-857a6898e16c admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:50.587607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f130027-b764-4a64-af47-857a6898e16c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:50.589609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f130027-b764-4a64-af47-857a6898e16c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:50.603294 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2605/5147] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:50.607283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e8c19-0c5e-4478-a7b2-5eac57f85a0f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:50.620426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e8c19-0c5e-4478-a7b2-5eac57f85a0f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:50.633838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c71e79-189a-46ca-b45b-575c8def33ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:50.634445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c71e79-189a-46ca-b45b-575c8def33ee admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:50.634769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c71e79-189a-46ca-b45b-575c8def33ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:50.635212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c71e79-189a-46ca-b45b-575c8def33ee admin admin] PATH_INFO: `/v3/projects/2b680ef456fc4d508bc3806cf97f728f/users/fd554535bfd44ff88f711612c9916051/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:50.664650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c71e79-189a-46ca-b45b-575c8def33ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b680ef456fc4d508bc3806cf97f728f, user_id=fd554535bfd44ff88f711612c9916051, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:50.669279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c71e79-189a-46ca-b45b-575c8def33ee admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:50.711093 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2543/5148] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:50 2022] PUT /identity/v3/projects/2b680ef456fc4d508bc3806cf97f728f/users/fd554535bfd44ff88f711612c9916051/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:50.714024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e9703-3ba9-45f4-a3f7-62e0cca772b9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:50.714548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e9703-3ba9-45f4-a3f7-62e0cca772b9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:50.715421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e9703-3ba9-45f4-a3f7-62e0cca772b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:50.765780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-066e9703-3ba9-45f4-a3f7-62e0cca772b9 None None] MFA Rules not processed for user `fd554535bfd44ff88f711612c9916051`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:50.767529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-066e9703-3ba9-45f4-a3f7-62e0cca772b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:50.902049 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2606/5149] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:50.903594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c71e79-189a-46ca-b45b-575c8def33ee admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:51.014691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c71e79-189a-46ca-b45b-575c8def33ee admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:51.024348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5dfe5-8c42-4458-afa6-eba5f16d6eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:51.024975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5dfe5-8c42-4458-afa6-eba5f16d6eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:51.025257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5dfe5-8c42-4458-afa6-eba5f16d6eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:51.025531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5dfe5-8c42-4458-afa6-eba5f16d6eb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:51.026170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5dfe5-8c42-4458-afa6-eba5f16d6eb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:51.028186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5dfe5-8c42-4458-afa6-eba5f16d6eb7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:51.072820 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2544/5150] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:50 2022] POST /identity/v3/projects => generated 385 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:51.076929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e9703-3ba9-45f4-a3f7-62e0cca772b9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:51.089163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e9703-3ba9-45f4-a3f7-62e0cca772b9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:51.226573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172cc139-d26c-4e2c-9693-32d43d1694f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:51.227984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172cc139-d26c-4e2c-9693-32d43d1694f0 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:51.228460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172cc139-d26c-4e2c-9693-32d43d1694f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:51.229067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172cc139-d26c-4e2c-9693-32d43d1694f0 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:51.231804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172cc139-d26c-4e2c-9693-32d43d1694f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:51.235325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172cc139-d26c-4e2c-9693-32d43d1694f0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:51.320329 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2607/5151] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:51 2022] POST /identity/v3/users => generated 299 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:51.323901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5dfe5-8c42-4458-afa6-eba5f16d6eb7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:51.334955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5dfe5-8c42-4458-afa6-eba5f16d6eb7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:51.366194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b3011-9234-4c67-8e3d-c9b0a0ca0722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:51.367146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b3011-9234-4c67-8e3d-c9b0a0ca0722 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:51.367587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b3011-9234-4c67-8e3d-c9b0a0ca0722 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:51.368002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b3011-9234-4c67-8e3d-c9b0a0ca0722 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:51.368789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b3011-9234-4c67-8e3d-c9b0a0ca0722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:51.373507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b3011-9234-4c67-8e3d-c9b0a0ca0722 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:51.393886 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2545/5152] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:51.394343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172cc139-d26c-4e2c-9693-32d43d1694f0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:51.412232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172cc139-d26c-4e2c-9693-32d43d1694f0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:51.432630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed25c80-3d15-49fa-91e5-d5a7e0807228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:51.434300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed25c80-3d15-49fa-91e5-d5a7e0807228 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:51.434712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed25c80-3d15-49fa-91e5-d5a7e0807228 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:51.437164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed25c80-3d15-49fa-91e5-d5a7e0807228 admin admin] PATH_INFO: `/v3/projects/7e4c13e3bfd5431fb532127f48da00ba/users/3167b1e5634a4e688359d710c5517093/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:51.480336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed25c80-3d15-49fa-91e5-d5a7e0807228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e4c13e3bfd5431fb532127f48da00ba, user_id=3167b1e5634a4e688359d710c5517093, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:51.483668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed25c80-3d15-49fa-91e5-d5a7e0807228 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:51.510057 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2608/5153] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:51 2022] PUT /identity/v3/projects/7e4c13e3bfd5431fb532127f48da00ba/users/3167b1e5634a4e688359d710c5517093/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:51.514965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b3011-9234-4c67-8e3d-c9b0a0ca0722 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:51.530394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b3011-9234-4c67-8e3d-c9b0a0ca0722 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:51.579393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed25c80-3d15-49fa-91e5-d5a7e0807228 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:51.590912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0ed25c80-3d15-49fa-91e5-d5a7e0807228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:51.627158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a47a6-e854-49c0-9e08-7896dafd6263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:51.629082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a47a6-e854-49c0-9e08-7896dafd6263 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:51.629465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a47a6-e854-49c0-9e08-7896dafd6263 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:51.631646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a47a6-e854-49c0-9e08-7896dafd6263 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:51.632926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a47a6-e854-49c0-9e08-7896dafd6263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:51.636697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a47a6-e854-49c0-9e08-7896dafd6263 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:51.668941 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2546/5154] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:51.673586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a47a6-e854-49c0-9e08-7896dafd6263 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:51.688067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a47a6-e854-49c0-9e08-7896dafd6263 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:51.693667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04699582-e6a7-482a-b541-a349d13240a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:51.694454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04699582-e6a7-482a-b541-a349d13240a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:51.694770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04699582-e6a7-482a-b541-a349d13240a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:51.695046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04699582-e6a7-482a-b541-a349d13240a3 admin admin] PATH_INFO: `/v3/projects/7e4c13e3bfd5431fb532127f48da00ba/users/3167b1e5634a4e688359d710c5517093/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:51.697393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04699582-e6a7-482a-b541-a349d13240a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e4c13e3bfd5431fb532127f48da00ba, user_id=3167b1e5634a4e688359d710c5517093, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:51.700164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04699582-e6a7-482a-b541-a349d13240a3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:51.719133 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2547/5155] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:51 2022] PUT /identity/v3/projects/7e4c13e3bfd5431fb532127f48da00ba/users/3167b1e5634a4e688359d710c5517093/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:05:51.725292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc662e3-7723-4182-8eaa-939ecf3a440e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:51.725578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc662e3-7723-4182-8eaa-939ecf3a440e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:51.725856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc662e3-7723-4182-8eaa-939ecf3a440e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:51.726897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:51.765236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6dc662e3-7723-4182-8eaa-939ecf3a440e None None] MFA Rules not processed for user `3167b1e5634a4e688359d710c5517093`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:51.767353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6dc662e3-7723-4182-8eaa-939ecf3a440e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:51.767896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed25c80-3d15-49fa-91e5-d5a7e0807228 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:51.783243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:51.884457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f8363-d0ec-418d-a9c0-5d43c344d649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:51.885252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f8363-d0ec-418d-a9c0-5d43c344d649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:51.885537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f8363-d0ec-418d-a9c0-5d43c344d649 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:51.885808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f8363-d0ec-418d-a9c0-5d43c344d649 admin admin] PATH_INFO: `/v3/projects/eb64a865764d4ec28a9b4a05183cf7a5` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:51.889938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f8363-d0ec-418d-a9c0-5d43c344d649 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb64a865764d4ec28a9b4a05183cf7a5)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:51.893993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f8363-d0ec-418d-a9c0-5d43c344d649 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:51.908683 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2548/5156] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:05:51.910662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc662e3-7723-4182-8eaa-939ecf3a440e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:51.933944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc662e3-7723-4182-8eaa-939ecf3a440e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:51.945029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed51b1-c895-4560-bf07-db27a64f17f1 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3167b1e5634a4e688359d710c5517093', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4c13e3bfd5431fb532127f48da00ba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:51.945878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed51b1-c895-4560-bf07-db27a64f17f1 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:51.946310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed51b1-c895-4560-bf07-db27a64f17f1 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:51.947110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed51b1-c895-4560-bf07-db27a64f17f1 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:51.947913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed51b1-c895-4560-bf07-db27a64f17f1 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:51.950100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed51b1-c895-4560-bf07-db27a64f17f1 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:52.017934 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2549/5157] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:51 2022] POST /identity/v3/projects => generated 358 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:52.020078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed51b1-c895-4560-bf07-db27a64f17f1 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:52.037372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a5f8363-d0ec-418d-a9c0-5d43c344d649 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb64a865764d4ec28a9b4a05183cf7a5', 'request_id': 'req-9a5f8363-d0ec-418d-a9c0-5d43c344d649'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:52.060610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a5f8363-d0ec-418d-a9c0-5d43c344d649 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb64a865764d4ec28a9b4a05183cf7a5', 'request_id': 'req-9a5f8363-d0ec-418d-a9c0-5d43c344d649'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:52.084336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a5f8363-d0ec-418d-a9c0-5d43c344d649 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb64a865764d4ec28a9b4a05183cf7a5', 'request_id': 'req-9a5f8363-d0ec-418d-a9c0-5d43c344d649'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:52.114401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a5f8363-d0ec-418d-a9c0-5d43c344d649 admin admin] The token cache is being invalidate because project eb64a865764d4ec28a9b4a05183cf7a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:52.114726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a5f8363-d0ec-418d-a9c0-5d43c344d649 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:52.117530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a5f8363-d0ec-418d-a9c0-5d43c344d649 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:52.119040 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2609/5158] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:51 2022] DELETE /identity/v3/projects/eb64a865764d4ec28a9b4a05183cf7a5 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:52.125819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12907b-0a91-4eb6-9399-b05653f904c4 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:52.126116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12907b-0a91-4eb6-9399-b05653f904c4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:52.127208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12907b-0a91-4eb6-9399-b05653f904c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:52.158278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6f12907b-0a91-4eb6-9399-b05653f904c4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:52.158921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6f12907b-0a91-4eb6-9399-b05653f904c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:52.162982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed51b1-c895-4560-bf07-db27a64f17f1 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:52.185897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666c2cf-6e2c-41ad-9e18-46489818f5ed tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3167b1e5634a4e688359d710c5517093', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4c13e3bfd5431fb532127f48da00ba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:52.186600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666c2cf-6e2c-41ad-9e18-46489818f5ed tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:52.186893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666c2cf-6e2c-41ad-9e18-46489818f5ed tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:52.187176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666c2cf-6e2c-41ad-9e18-46489818f5ed tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] PATH_INFO: `/v3/projects/5a7653a154bc499ebcd56d48a48b9fc2/tags/tempest-tag-71076139` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:52.188596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4666c2cf-6e2c-41ad-9e18-46489818f5ed tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: Authorizing `identity:create_project_tag(project_id=5a7653a154bc499ebcd56d48a48b9fc2, value=tempest-tag-71076139)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:52.191365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4666c2cf-6e2c-41ad-9e18-46489818f5ed tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:52.284016 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2610/5159] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:52.292593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6a300-c864-43b9-855f-ee79c8af281e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:52.293027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6a300-c864-43b9-855f-ee79c8af281e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:52.293292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6a300-c864-43b9-855f-ee79c8af281e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:52.306535 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-4666c2cf-6e2c-41ad-9e18-46489818f5ed tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-tag-71076139. To ensure interoperability, identifiers should be a valid uuid. Oct 01 21:05:52.306535 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 01 21:05:52.306535 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:05:52.322146 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2550/5160] 10.222.0.17 () {60 vars in 1327 bytes} [Sat Oct 1 21:05:52 2022] PUT /identity/v3/projects/5a7653a154bc499ebcd56d48a48b9fc2/tags/tempest-tag-71076139 => generated 0 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 289 bytes (1 switches on core 0) Oct 01 21:05:52.324301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666c2cf-6e2c-41ad-9e18-46489818f5ed tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:52.328422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4666c2cf-6e2c-41ad-9e18-46489818f5ed tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:52.329972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fbf6a300-c864-43b9-855f-ee79c8af281e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:52.332383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fbf6a300-c864-43b9-855f-ee79c8af281e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:52.354114 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2611/5161] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:52.358978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6a300-c864-43b9-855f-ee79c8af281e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:52.367578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666c2cf-6e2c-41ad-9e18-46489818f5ed tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:52.370834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6a300-c864-43b9-855f-ee79c8af281e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:52.380188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16a47e-0c65-43db-860c-1c5661724ce0 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3167b1e5634a4e688359d710c5517093', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4c13e3bfd5431fb532127f48da00ba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:52.380880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16a47e-0c65-43db-860c-1c5661724ce0 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:52.381163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16a47e-0c65-43db-860c-1c5661724ce0 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:52.381288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba11ea9-3f2f-4bf6-b89b-19b6dcd3cc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:52.381438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16a47e-0c65-43db-860c-1c5661724ce0 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] PATH_INFO: `/v3/projects/5a7653a154bc499ebcd56d48a48b9fc2/tags/tempest-tag-71076139` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:52.382027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba11ea9-3f2f-4bf6-b89b-19b6dcd3cc50 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:52.382402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba11ea9-3f2f-4bf6-b89b-19b6dcd3cc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:52.382833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba11ea9-3f2f-4bf6-b89b-19b6dcd3cc50 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:52.383917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba11ea9-3f2f-4bf6-b89b-19b6dcd3cc50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:05:52.383917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba11ea9-3f2f-4bf6-b89b-19b6dcd3cc50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:52.387093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba11ea9-3f2f-4bf6-b89b-19b6dcd3cc50 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:52.403430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16a47e-0c65-43db-860c-1c5661724ce0 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: Authorizing `identity:get_project_tag(project_id=5a7653a154bc499ebcd56d48a48b9fc2, value=tempest-tag-71076139)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:52.410207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16a47e-0c65-43db-860c-1c5661724ce0 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:52.413841 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2612/5162] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:52.415754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba11ea9-3f2f-4bf6-b89b-19b6dcd3cc50 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:52.427762 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2551/5163] 10.222.0.17 () {58 vars in 1308 bytes} [Sat Oct 1 21:05:52 2022] GET /identity/v3/projects/5a7653a154bc499ebcd56d48a48b9fc2/tags/tempest-tag-71076139 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:52.428916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16a47e-0c65-43db-860c-1c5661724ce0 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:52.432681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba11ea9-3f2f-4bf6-b89b-19b6dcd3cc50 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:52.440022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3f760-dca5-4bcd-93ea-6544e9ee2837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:52.441151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3f760-dca5-4bcd-93ea-6544e9ee2837 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:52.441329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3f760-dca5-4bcd-93ea-6544e9ee2837 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:52.441739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3f760-dca5-4bcd-93ea-6544e9ee2837 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:52.442693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3f760-dca5-4bcd-93ea-6544e9ee2837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:52.443439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16a47e-0c65-43db-860c-1c5661724ce0 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:52.445341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3f760-dca5-4bcd-93ea-6544e9ee2837 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:52.449083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9130b-5825-40d5-9564-bac71fcbdf27 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3167b1e5634a4e688359d710c5517093', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4c13e3bfd5431fb532127f48da00ba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:52.450593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9130b-5825-40d5-9564-bac71fcbdf27 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:52.451625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9130b-5825-40d5-9564-bac71fcbdf27 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:52.452213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9130b-5825-40d5-9564-bac71fcbdf27 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] PATH_INFO: `/v3/projects/5a7653a154bc499ebcd56d48a48b9fc2/tags` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:52.454031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9130b-5825-40d5-9564-bac71fcbdf27 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: Authorizing `identity:update_project_tags(project_id=5a7653a154bc499ebcd56d48a48b9fc2)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:52.454241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:52.457406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9130b-5825-40d5-9564-bac71fcbdf27 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:52.517754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:52.547946 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2613/5164] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:52 2022] POST /identity/v3/projects => generated 389 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:52.550359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3f760-dca5-4bcd-93ea-6544e9ee2837 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:52.629472 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2552/5165] 10.222.0.17 () {60 vars in 1265 bytes} [Sat Oct 1 21:05:52 2022] PUT /identity/v3/projects/5a7653a154bc499ebcd56d48a48b9fc2/tags => generated 141 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:52.632112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9130b-5825-40d5-9564-bac71fcbdf27 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:52.647862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9130b-5825-40d5-9564-bac71fcbdf27 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:52.726692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3f760-dca5-4bcd-93ea-6544e9ee2837 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:52.745701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3aacd-affa-4e8f-acbc-a119be4e2d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:52.746360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3aacd-affa-4e8f-acbc-a119be4e2d1e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:52.746714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3aacd-affa-4e8f-acbc-a119be4e2d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:52.746932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3aacd-affa-4e8f-acbc-a119be4e2d1e admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:52.747580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3aacd-affa-4e8f-acbc-a119be4e2d1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:52.750172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3aacd-affa-4e8f-acbc-a119be4e2d1e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:52.777637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd0cf71-82c8-4770-be66-3af46514edad tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3167b1e5634a4e688359d710c5517093', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4c13e3bfd5431fb532127f48da00ba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:52.780317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0cf71-82c8-4770-be66-3af46514edad tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:52.780856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0cf71-82c8-4770-be66-3af46514edad tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:52.781588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0cf71-82c8-4770-be66-3af46514edad tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] PATH_INFO: `/v3/projects/5a7653a154bc499ebcd56d48a48b9fc2/tags` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:52.796574 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2614/5166] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:52 2022] POST /identity/v3/users => generated 295 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:52.802071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3aacd-affa-4e8f-acbc-a119be4e2d1e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:52.809241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd0cf71-82c8-4770-be66-3af46514edad tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: Authorizing `identity:list_project_tags(project_id=5a7653a154bc499ebcd56d48a48b9fc2)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:52.814390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd0cf71-82c8-4770-be66-3af46514edad tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:52.823080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3aacd-affa-4e8f-acbc-a119be4e2d1e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:52.828065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3532a-1561-4bf2-a5e0-1163a44d10a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:52.828689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3532a-1561-4bf2-a5e0-1163a44d10a2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:52.828980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3532a-1561-4bf2-a5e0-1163a44d10a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:52.830539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3532a-1561-4bf2-a5e0-1163a44d10a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:52.830755 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2553/5167] 10.222.0.17 () {58 vars in 1245 bytes} [Sat Oct 1 21:05:52 2022] GET /identity/v3/projects/5a7653a154bc499ebcd56d48a48b9fc2/tags => generated 141 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:52.832505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3532a-1561-4bf2-a5e0-1163a44d10a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:52.834048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd0cf71-82c8-4770-be66-3af46514edad tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:52.840208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3532a-1561-4bf2-a5e0-1163a44d10a2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:52.853385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd0cf71-82c8-4770-be66-3af46514edad tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:52.859144 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2615/5168] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:52.862984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3532a-1561-4bf2-a5e0-1163a44d10a2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:52.865174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef42bf43-9a47-4c06-b8c6-98ed9b272c5a tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3167b1e5634a4e688359d710c5517093', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4c13e3bfd5431fb532127f48da00ba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:52.866797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42bf43-9a47-4c06-b8c6-98ed9b272c5a tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:52.868336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42bf43-9a47-4c06-b8c6-98ed9b272c5a tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:52.868841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42bf43-9a47-4c06-b8c6-98ed9b272c5a tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] PATH_INFO: `/v3/projects/5a7653a154bc499ebcd56d48a48b9fc2/tags/tempest-tag-1528700413` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:52.875481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef42bf43-9a47-4c06-b8c6-98ed9b272c5a tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: Authorizing `identity:delete_project_tag(project_id=5a7653a154bc499ebcd56d48a48b9fc2, value=tempest-tag-1528700413)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:52.877652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef42bf43-9a47-4c06-b8c6-98ed9b272c5a tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:52.879672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3532a-1561-4bf2-a5e0-1163a44d10a2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:52.892011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27851f-908b-42da-a840-f54c04f8fad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:52.892859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27851f-908b-42da-a840-f54c04f8fad9 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:52.893229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27851f-908b-42da-a840-f54c04f8fad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:52.893611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27851f-908b-42da-a840-f54c04f8fad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:52.894536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c27851f-908b-42da-a840-f54c04f8fad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:52.897355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c27851f-908b-42da-a840-f54c04f8fad9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:52.931163 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2616/5169] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:05:52.935403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27851f-908b-42da-a840-f54c04f8fad9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:52.938370 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-ef42bf43-9a47-4c06-b8c6-98ed9b272c5a tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-tag-1528700413. To ensure interoperability, identifiers should be a valid uuid. Oct 01 21:05:52.938370 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Oct 01 21:05:52.938370 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:05:52.950227 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2554/5170] 10.222.0.17 () {58 vars in 1317 bytes} [Sat Oct 1 21:05:52 2022] DELETE /identity/v3/projects/5a7653a154bc499ebcd56d48a48b9fc2/tags/tempest-tag-1528700413 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:52.955556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef42bf43-9a47-4c06-b8c6-98ed9b272c5a tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:52.958257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27851f-908b-42da-a840-f54c04f8fad9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:52.974549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8f64c-6604-4c6a-b421-c5c6ff11f241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:52.975322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8f64c-6604-4c6a-b421-c5c6ff11f241 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:52.975678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8f64c-6604-4c6a-b421-c5c6ff11f241 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:52.976058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8f64c-6604-4c6a-b421-c5c6ff11f241 admin admin] PATH_INFO: `/v3/projects/d3434dac76ec46b3a5e10f840b7c2785/users/646f6f5cff204d27b0476d53bd9e6fa0/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:52.978584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef42bf43-9a47-4c06-b8c6-98ed9b272c5a tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:52.993716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ab1d2-6448-42d2-87f8-ffda1e0b5fad tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3167b1e5634a4e688359d710c5517093', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4c13e3bfd5431fb532127f48da00ba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:52.994877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ab1d2-6448-42d2-87f8-ffda1e0b5fad tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:52.995428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ab1d2-6448-42d2-87f8-ffda1e0b5fad tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:52.997132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ab1d2-6448-42d2-87f8-ffda1e0b5fad tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] PATH_INFO: `/v3/projects/5a7653a154bc499ebcd56d48a48b9fc2/tags/tempest-tag-1528700413` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:53.013880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8f64c-6604-4c6a-b421-c5c6ff11f241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3434dac76ec46b3a5e10f840b7c2785, user_id=646f6f5cff204d27b0476d53bd9e6fa0, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:53.016395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8f64c-6604-4c6a-b421-c5c6ff11f241 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:53.019078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ab1d2-6448-42d2-87f8-ffda1e0b5fad tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: Authorizing `identity:get_project_tag(project_id=5a7653a154bc499ebcd56d48a48b9fc2, value=tempest-tag-1528700413)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:53.021945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ab1d2-6448-42d2-87f8-ffda1e0b5fad tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:53.048972 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2617/5171] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:52 2022] PUT /identity/v3/projects/d3434dac76ec46b3a5e10f840b7c2785/users/646f6f5cff204d27b0476d53bd9e6fa0/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:05:53.053828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda20996-3bc6-4af3-913d-4c4755bca4e8 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:53.054128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda20996-3bc6-4af3-913d-4c4755bca4e8 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:53.054690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda20996-3bc6-4af3-913d-4c4755bca4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:53.058901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-035ab1d2-6448-42d2-87f8-ffda1e0b5fad tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Could not find project tag: tempest-tag-1528700413.: keystone.exception.ProjectTagNotFound: Could not find project tag: tempest-tag-1528700413. Oct 01 21:05:53.058901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:05:53.058901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:05:53.058901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:05:53.058901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:05:53.058901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:05:53.058901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:05:53.058901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:05:53.058901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:05:53.058901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:05:53.058901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:05:53.058901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:05:53.058901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 284, in get Oct 01 21:05:53.058901 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application PROVIDERS.resource_api.get_project_tag(project_id, value) Oct 01 21:05:53.059916 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:05:53.059916 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:05:53.059916 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 960, in get_project_tag Oct 01 21:05:53.059916 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectTagNotFound(project_tag=tag_name) Oct 01 21:05:53.059916 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectTagNotFound: Could not find project tag: tempest-tag-1528700413. Oct 01 21:05:53.059916 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:05:53.062892 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2555/5172] 10.222.0.17 () {58 vars in 1314 bytes} [Sat Oct 1 21:05:52 2022] GET /identity/v3/projects/5a7653a154bc499ebcd56d48a48b9fc2/tags/tempest-tag-1528700413 => generated 107 bytes in 109 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:05:53.067447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ab1d2-6448-42d2-87f8-ffda1e0b5fad tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:53.084428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ab1d2-6448-42d2-87f8-ffda1e0b5fad tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:53.096311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fda20996-3bc6-4af3-913d-4c4755bca4e8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:53.102930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fda20996-3bc6-4af3-913d-4c4755bca4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:53.204231 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2618/5173] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:53.218711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a97c6-cc97-430c-a67b-f3b6a5ded92e tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3167b1e5634a4e688359d710c5517093', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4c13e3bfd5431fb532127f48da00ba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:53.219377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a97c6-cc97-430c-a67b-f3b6a5ded92e tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:53.219663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a97c6-cc97-430c-a67b-f3b6a5ded92e tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:53.219943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a97c6-cc97-430c-a67b-f3b6a5ded92e tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] PATH_INFO: `/v3/projects/5a7653a154bc499ebcd56d48a48b9fc2/tags` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:53.221536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a97c6-cc97-430c-a67b-f3b6a5ded92e tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: Authorizing `identity:delete_project_tags(project_id=5a7653a154bc499ebcd56d48a48b9fc2)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:53.224107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a97c6-cc97-430c-a67b-f3b6a5ded92e tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:53.225615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:53.226885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda20996-3bc6-4af3-913d-4c4755bca4e8 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:53.228835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fda20996-3bc6-4af3-913d-4c4755bca4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:53.276391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:53.320548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda20996-3bc6-4af3-913d-4c4755bca4e8 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:53.332669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d6673-d741-4afe-a930-70a0d8e4128a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:53.333703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d6673-d741-4afe-a930-70a0d8e4128a service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:53.334243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d6673-d741-4afe-a930-70a0d8e4128a service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:53.334842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d6673-d741-4afe-a930-70a0d8e4128a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:53.339210 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2556/5174] 10.222.0.17 () {58 vars in 1248 bytes} [Sat Oct 1 21:05:53 2022] DELETE /identity/v3/projects/5a7653a154bc499ebcd56d48a48b9fc2/tags => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:05:53.343408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a97c6-cc97-430c-a67b-f3b6a5ded92e tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:53.350871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d6673-d741-4afe-a930-70a0d8e4128a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:53.352174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d6673-d741-4afe-a930-70a0d8e4128a service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:53.355110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a97c6-cc97-430c-a67b-f3b6a5ded92e tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:53.362125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95bfe9-2473-4601-a884-ab0bfc42eb51 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3167b1e5634a4e688359d710c5517093', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4c13e3bfd5431fb532127f48da00ba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:53.363126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95bfe9-2473-4601-a884-ab0bfc42eb51 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:53.363825 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2619/5175] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:05:53.364259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95bfe9-2473-4601-a884-ab0bfc42eb51 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:53.366550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95bfe9-2473-4601-a884-ab0bfc42eb51 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] PATH_INFO: `/v3/projects/5a7653a154bc499ebcd56d48a48b9fc2/tags` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:53.389012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95bfe9-2473-4601-a884-ab0bfc42eb51 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: Authorizing `identity:list_project_tags(project_id=5a7653a154bc499ebcd56d48a48b9fc2)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:53.393910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95bfe9-2473-4601-a884-ab0bfc42eb51 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:53.411555 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2557/5176] 10.222.0.17 () {58 vars in 1245 bytes} [Sat Oct 1 21:05:53 2022] GET /identity/v3/projects/5a7653a154bc499ebcd56d48a48b9fc2/tags => generated 66 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 01 21:05:53.416113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d6673-d741-4afe-a930-70a0d8e4128a service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:53.432354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d6673-d741-4afe-a930-70a0d8e4128a service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:53.444684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e3644-5cd9-4e8c-8665-05e66c3bc658 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3167b1e5634a4e688359d710c5517093', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4c13e3bfd5431fb532127f48da00ba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:53.445388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e3644-5cd9-4e8c-8665-05e66c3bc658 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:53.446406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e3644-5cd9-4e8c-8665-05e66c3bc658 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:53.446406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e3644-5cd9-4e8c-8665-05e66c3bc658 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] PATH_INFO: `/v3/projects/5a7653a154bc499ebcd56d48a48b9fc2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:53.448365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e3644-5cd9-4e8c-8665-05e66c3bc658 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5a7653a154bc499ebcd56d48a48b9fc2)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:53.451501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e3644-5cd9-4e8c-8665-05e66c3bc658 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:53.595338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f50e3644-5cd9-4e8c-8665-05e66c3bc658 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a7653a154bc499ebcd56d48a48b9fc2', 'request_id': 'req-f50e3644-5cd9-4e8c-8665-05e66c3bc658'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:53.606964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f50e3644-5cd9-4e8c-8665-05e66c3bc658 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a7653a154bc499ebcd56d48a48b9fc2', 'request_id': 'req-f50e3644-5cd9-4e8c-8665-05e66c3bc658'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:53.633580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f50e3644-5cd9-4e8c-8665-05e66c3bc658 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a7653a154bc499ebcd56d48a48b9fc2', 'request_id': 'req-f50e3644-5cd9-4e8c-8665-05e66c3bc658'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:53.672519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f50e3644-5cd9-4e8c-8665-05e66c3bc658 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] The token cache is being invalidate because project 5a7653a154bc499ebcd56d48a48b9fc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:53.672826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f50e3644-5cd9-4e8c-8665-05e66c3bc658 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:53.673589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f50e3644-5cd9-4e8c-8665-05e66c3bc658 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:53.676412 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2620/5177] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:53 2022] DELETE /identity/v3/projects/5a7653a154bc499ebcd56d48a48b9fc2 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:53.681822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95bfe9-2473-4601-a884-ab0bfc42eb51 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:53.684348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6e95bfe9-2473-4601-a884-ab0bfc42eb51 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:53.722964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95bfe9-2473-4601-a884-ab0bfc42eb51 tempest-IdentityV3ProjectTagsTest-404066545 tempest-IdentityV3ProjectTagsTest-404066545-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:53.804962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3395a71-f871-464a-ad16-719391879900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:53.806877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3395a71-f871-464a-ad16-719391879900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:53.807410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3395a71-f871-464a-ad16-719391879900 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:53.807833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3395a71-f871-464a-ad16-719391879900 admin admin] PATH_INFO: `/v3/users/fd554535bfd44ff88f711612c9916051` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:53.809648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3395a71-f871-464a-ad16-719391879900 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd554535bfd44ff88f711612c9916051)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:53.812101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3395a71-f871-464a-ad16-719391879900 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:53.879415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f3395a71-f871-464a-ad16-719391879900 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd554535bfd44ff88f711612c9916051'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:53.881296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f3395a71-f871-464a-ad16-719391879900 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd554535bfd44ff88f711612c9916051'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:53.882412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f3395a71-f871-464a-ad16-719391879900 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd554535bfd44ff88f711612c9916051'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:53.894339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f3395a71-f871-464a-ad16-719391879900 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd554535bfd44ff88f711612c9916051'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:53.914095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f3395a71-f871-464a-ad16-719391879900 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd554535bfd44ff88f711612c9916051'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:53.940994 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2558/5178] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:53 2022] DELETE /identity/v3/users/fd554535bfd44ff88f711612c9916051 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:05:53.946690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf676677-36c5-413c-b0f3-8e795d52fa15 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:53.947268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf676677-36c5-413c-b0f3-8e795d52fa15 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:53.947724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf676677-36c5-413c-b0f3-8e795d52fa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:53.978300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-cf676677-36c5-413c-b0f3-8e795d52fa15 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:53.979975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cf676677-36c5-413c-b0f3-8e795d52fa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:54.080175 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2621/5179] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:54.131990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3395a71-f871-464a-ad16-719391879900 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:54.135382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f3395a71-f871-464a-ad16-719391879900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:54.199799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:54.254294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3395a71-f871-464a-ad16-719391879900 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:54.261624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34df6d-a6ac-43b6-be5d-ab23e04f22fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:54.262256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34df6d-a6ac-43b6-be5d-ab23e04f22fd service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:54.262585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34df6d-a6ac-43b6-be5d-ab23e04f22fd service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:54.263296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34df6d-a6ac-43b6-be5d-ab23e04f22fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:54.274647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34df6d-a6ac-43b6-be5d-ab23e04f22fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:54.276221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34df6d-a6ac-43b6-be5d-ab23e04f22fd service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:54.289251 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2559/5180] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:05:55.336011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf676677-36c5-413c-b0f3-8e795d52fa15 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:55.338431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cf676677-36c5-413c-b0f3-8e795d52fa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:55.352167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf676677-36c5-413c-b0f3-8e795d52fa15 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:55.356995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65241267-4c13-4d6e-a305-0acee075f995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:55.358240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65241267-4c13-4d6e-a305-0acee075f995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:55.358635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65241267-4c13-4d6e-a305-0acee075f995 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:55.358978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65241267-4c13-4d6e-a305-0acee075f995 admin admin] PATH_INFO: `/v3/projects/2b680ef456fc4d508bc3806cf97f728f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:55.361783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65241267-4c13-4d6e-a305-0acee075f995 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b680ef456fc4d508bc3806cf97f728f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:55.363816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65241267-4c13-4d6e-a305-0acee075f995 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:55.474072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-65241267-4c13-4d6e-a305-0acee075f995 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b680ef456fc4d508bc3806cf97f728f', 'request_id': 'req-65241267-4c13-4d6e-a305-0acee075f995'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:55.477068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:55.484651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-65241267-4c13-4d6e-a305-0acee075f995 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b680ef456fc4d508bc3806cf97f728f', 'request_id': 'req-65241267-4c13-4d6e-a305-0acee075f995'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:55.506374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-65241267-4c13-4d6e-a305-0acee075f995 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b680ef456fc4d508bc3806cf97f728f', 'request_id': 'req-65241267-4c13-4d6e-a305-0acee075f995'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:55.529032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-65241267-4c13-4d6e-a305-0acee075f995 admin admin] The token cache is being invalidate because project 2b680ef456fc4d508bc3806cf97f728f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:55.529475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-65241267-4c13-4d6e-a305-0acee075f995 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:55.530413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-65241267-4c13-4d6e-a305-0acee075f995 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:55.534605 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2622/5181] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:55 2022] DELETE /identity/v3/projects/2b680ef456fc4d508bc3806cf97f728f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:55.538200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34df6d-a6ac-43b6-be5d-ab23e04f22fd service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:55.555388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34df6d-a6ac-43b6-be5d-ab23e04f22fd service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:55.713509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203eb96d-3ca3-4943-bd61-22c02cd6025e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:55.714194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203eb96d-3ca3-4943-bd61-22c02cd6025e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:55.714479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203eb96d-3ca3-4943-bd61-22c02cd6025e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:55.714795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203eb96d-3ca3-4943-bd61-22c02cd6025e admin admin] PATH_INFO: `/v3/users/3167b1e5634a4e688359d710c5517093` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:55.718399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203eb96d-3ca3-4943-bd61-22c02cd6025e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3167b1e5634a4e688359d710c5517093)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:55.722965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203eb96d-3ca3-4943-bd61-22c02cd6025e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:55.836092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-203eb96d-3ca3-4943-bd61-22c02cd6025e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3167b1e5634a4e688359d710c5517093'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:55.836841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-203eb96d-3ca3-4943-bd61-22c02cd6025e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3167b1e5634a4e688359d710c5517093'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:55.837580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-203eb96d-3ca3-4943-bd61-22c02cd6025e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3167b1e5634a4e688359d710c5517093'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:55.870427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-203eb96d-3ca3-4943-bd61-22c02cd6025e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3167b1e5634a4e688359d710c5517093'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:55.903918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-203eb96d-3ca3-4943-bd61-22c02cd6025e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3167b1e5634a4e688359d710c5517093'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:55.921356 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2560/5182] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:55 2022] DELETE /identity/v3/users/3167b1e5634a4e688359d710c5517093 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:56.784106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65241267-4c13-4d6e-a305-0acee075f995 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:56.786743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-65241267-4c13-4d6e-a305-0acee075f995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:56.842423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65241267-4c13-4d6e-a305-0acee075f995 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:56.948088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee045061-f2f4-43e0-9003-5a9715ce96a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:56.948828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee045061-f2f4-43e0-9003-5a9715ce96a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:56.949189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee045061-f2f4-43e0-9003-5a9715ce96a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:56.949556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee045061-f2f4-43e0-9003-5a9715ce96a2 admin admin] PATH_INFO: `/v3/projects/7e4c13e3bfd5431fb532127f48da00ba` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:56.951349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee045061-f2f4-43e0-9003-5a9715ce96a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e4c13e3bfd5431fb532127f48da00ba)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:56.954089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee045061-f2f4-43e0-9003-5a9715ce96a2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:57.059456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ee045061-f2f4-43e0-9003-5a9715ce96a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e4c13e3bfd5431fb532127f48da00ba', 'request_id': 'req-ee045061-f2f4-43e0-9003-5a9715ce96a2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:57.074159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ee045061-f2f4-43e0-9003-5a9715ce96a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e4c13e3bfd5431fb532127f48da00ba', 'request_id': 'req-ee045061-f2f4-43e0-9003-5a9715ce96a2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:57.082563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ee045061-f2f4-43e0-9003-5a9715ce96a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e4c13e3bfd5431fb532127f48da00ba', 'request_id': 'req-ee045061-f2f4-43e0-9003-5a9715ce96a2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:57.107760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ee045061-f2f4-43e0-9003-5a9715ce96a2 admin admin] The token cache is being invalidate because project 7e4c13e3bfd5431fb532127f48da00ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:05:57.108158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ee045061-f2f4-43e0-9003-5a9715ce96a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:57.109653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ee045061-f2f4-43e0-9003-5a9715ce96a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:57.118580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867cec7-8b87-4155-a90b-78fc42de304e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:57.119106 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2623/5183] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:05:56 2022] DELETE /identity/v3/projects/7e4c13e3bfd5431fb532127f48da00ba => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:57.119614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867cec7-8b87-4155-a90b-78fc42de304e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:57.120187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867cec7-8b87-4155-a90b-78fc42de304e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:57.171099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8867cec7-8b87-4155-a90b-78fc42de304e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:57.175063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8867cec7-8b87-4155-a90b-78fc42de304e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:57.256833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:57.312314 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2561/5184] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:05:57.322005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9ae72-79c4-42f0-bda6-679ff1b65704 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:57.322306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9ae72-79c4-42f0-bda6-679ff1b65704 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:57.322696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9ae72-79c4-42f0-bda6-679ff1b65704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:57.364003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-40e9ae72-79c4-42f0-bda6-679ff1b65704 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:57.365511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-40e9ae72-79c4-42f0-bda6-679ff1b65704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:57.388888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867cec7-8b87-4155-a90b-78fc42de304e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:57.389230 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2624/5185] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:57.404399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867cec7-8b87-4155-a90b-78fc42de304e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:57.409739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099086d-8905-4cb1-8960-8a81800fde4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:57.410367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099086d-8905-4cb1-8960-8a81800fde4a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:57.410685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099086d-8905-4cb1-8960-8a81800fde4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:57.410961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099086d-8905-4cb1-8960-8a81800fde4a admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:57.411618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099086d-8905-4cb1-8960-8a81800fde4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:05:57.411924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099086d-8905-4cb1-8960-8a81800fde4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:57.414261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099086d-8905-4cb1-8960-8a81800fde4a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:57.430163 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2562/5186] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:57.435412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9ae72-79c4-42f0-bda6-679ff1b65704 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:57.453475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9ae72-79c4-42f0-bda6-679ff1b65704 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:57.471807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092214b-32d2-4f43-a07a-0798fe458e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:57.473583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092214b-32d2-4f43-a07a-0798fe458e80 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:57.475288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092214b-32d2-4f43-a07a-0798fe458e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:57.476594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092214b-32d2-4f43-a07a-0798fe458e80 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:57.479155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092214b-32d2-4f43-a07a-0798fe458e80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:57.487915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092214b-32d2-4f43-a07a-0798fe458e80 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:57.593116 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2625/5187] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:05:57 2022] POST /identity/v3/projects => generated 369 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:57.598882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099086d-8905-4cb1-8960-8a81800fde4a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:57.611973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099086d-8905-4cb1-8960-8a81800fde4a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:57.686455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe80715-9780-4578-bf63-e9c00edc1483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:57.687314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe80715-9780-4578-bf63-e9c00edc1483 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:57.688612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe80715-9780-4578-bf63-e9c00edc1483 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:57.689166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe80715-9780-4578-bf63-e9c00edc1483 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:57.691044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe80715-9780-4578-bf63-e9c00edc1483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:57.693489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe80715-9780-4578-bf63-e9c00edc1483 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:57.727191 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2563/5188] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:05:57 2022] POST /identity/v3/users => generated 285 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:05:57.731469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092214b-32d2-4f43-a07a-0798fe458e80 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:57.749685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092214b-32d2-4f43-a07a-0798fe458e80 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:57.756380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e6ba5-eae5-4c2a-b3b5-18d9b72ea240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:57.757224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e6ba5-eae5-4c2a-b3b5-18d9b72ea240 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:57.758709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e6ba5-eae5-4c2a-b3b5-18d9b72ea240 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:57.759282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e6ba5-eae5-4c2a-b3b5-18d9b72ea240 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:57.760097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e6ba5-eae5-4c2a-b3b5-18d9b72ea240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:57.762066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e6ba5-eae5-4c2a-b3b5-18d9b72ea240 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:57.780902 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2626/5189] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:05:57.785092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe80715-9780-4578-bf63-e9c00edc1483 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:57.802575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe80715-9780-4578-bf63-e9c00edc1483 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:57.813771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2538f-e7c3-4c27-8a89-4b10651d2f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:57.814443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2538f-e7c3-4c27-8a89-4b10651d2f6e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:57.814868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2538f-e7c3-4c27-8a89-4b10651d2f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:57.815283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2538f-e7c3-4c27-8a89-4b10651d2f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:57.816025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2538f-e7c3-4c27-8a89-4b10651d2f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:57.818047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2538f-e7c3-4c27-8a89-4b10651d2f6e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:57.836034 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2564/5190] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:05:57.840117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e6ba5-eae5-4c2a-b3b5-18d9b72ea240 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:57.852154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e6ba5-eae5-4c2a-b3b5-18d9b72ea240 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:57.857420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef52fd-04c1-45c9-9bef-c71c7411001d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:57.858330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef52fd-04c1-45c9-9bef-c71c7411001d admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:57.858875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef52fd-04c1-45c9-9bef-c71c7411001d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:57.859285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef52fd-04c1-45c9-9bef-c71c7411001d admin admin] PATH_INFO: `/v3/projects/e1bf22a392d1466fa92860a3cac7169a/users/b60c825d444d435b863af3ef381164b6/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:57.886623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef52fd-04c1-45c9-9bef-c71c7411001d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1bf22a392d1466fa92860a3cac7169a, user_id=b60c825d444d435b863af3ef381164b6, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:57.893223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef52fd-04c1-45c9-9bef-c71c7411001d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:57.910561 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2627/5191] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:05:57 2022] PUT /identity/v3/projects/e1bf22a392d1466fa92860a3cac7169a/users/b60c825d444d435b863af3ef381164b6/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:57.918869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badc89d-6234-4335-9ddd-5b420d251364 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:57.919476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badc89d-6234-4335-9ddd-5b420d251364 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:57.921061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badc89d-6234-4335-9ddd-5b420d251364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:57.964126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0badc89d-6234-4335-9ddd-5b420d251364 None None] MFA Rules not processed for user `b60c825d444d435b863af3ef381164b6`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:57.965660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0badc89d-6234-4335-9ddd-5b420d251364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:58.063083 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2565/5192] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:58.063885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef52fd-04c1-45c9-9bef-c71c7411001d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:58.072366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:58.080911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef52fd-04c1-45c9-9bef-c71c7411001d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:58.087500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e41a70-c1ac-4c00-bef4-9bc0ad7b9380 tempest-TestApiDiscovery-1596829233 tempest-TestApiDiscovery-1596829233-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60c825d444d435b863af3ef381164b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1bf22a392d1466fa92860a3cac7169a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:58.088158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e41a70-c1ac-4c00-bef4-9bc0ad7b9380 tempest-TestApiDiscovery-1596829233 tempest-TestApiDiscovery-1596829233-project] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:58.088469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e41a70-c1ac-4c00-bef4-9bc0ad7b9380 tempest-TestApiDiscovery-1596829233 tempest-TestApiDiscovery-1596829233-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:58.088722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e41a70-c1ac-4c00-bef4-9bc0ad7b9380 tempest-TestApiDiscovery-1596829233 tempest-TestApiDiscovery-1596829233-project] PATH_INFO: `/v3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:58.090692 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2628/5193] 10.222.0.17 () {58 vars in 1104 bytes} [Sat Oct 1 21:05:58 2022] GET /identity/v3 => generated 255 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:58.095872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badc89d-6234-4335-9ddd-5b420d251364 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:58.105906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badc89d-6234-4335-9ddd-5b420d251364 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:58.110944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a7787-eff6-4f90-b897-ec3df963f3f5 tempest-TestApiDiscovery-1596829233 tempest-TestApiDiscovery-1596829233-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60c825d444d435b863af3ef381164b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1bf22a392d1466fa92860a3cac7169a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:58.111551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a7787-eff6-4f90-b897-ec3df963f3f5 tempest-TestApiDiscovery-1596829233 tempest-TestApiDiscovery-1596829233-project] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:58.111838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a7787-eff6-4f90-b897-ec3df963f3f5 tempest-TestApiDiscovery-1596829233 tempest-TestApiDiscovery-1596829233-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:58.112181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a7787-eff6-4f90-b897-ec3df963f3f5 tempest-TestApiDiscovery-1596829233 tempest-TestApiDiscovery-1596829233-project] PATH_INFO: `/v3` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:58.114536 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2566/5194] 10.222.0.17 () {58 vars in 1104 bytes} [Sat Oct 1 21:05:58 2022] GET /identity/v3 => generated 255 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:58.119060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e41a70-c1ac-4c00-bef4-9bc0ad7b9380 tempest-TestApiDiscovery-1596829233 tempest-TestApiDiscovery-1596829233-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:58.135226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e41a70-c1ac-4c00-bef4-9bc0ad7b9380 tempest-TestApiDiscovery-1596829233 tempest-TestApiDiscovery-1596829233-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:58.142046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7471dd-382d-42b0-bbd9-043e85c1c302 tempest-TestApiDiscovery-1596829233 tempest-TestApiDiscovery-1596829233-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60c825d444d435b863af3ef381164b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1bf22a392d1466fa92860a3cac7169a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:58.143704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7471dd-382d-42b0-bbd9-043e85c1c302 tempest-TestApiDiscovery-1596829233 tempest-TestApiDiscovery-1596829233-project] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:58.143842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7471dd-382d-42b0-bbd9-043e85c1c302 tempest-TestApiDiscovery-1596829233 tempest-TestApiDiscovery-1596829233-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:58.144594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7471dd-382d-42b0-bbd9-043e85c1c302 tempest-TestApiDiscovery-1596829233 tempest-TestApiDiscovery-1596829233-project] PATH_INFO: `/v3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:58.145738 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2629/5195] 10.222.0.17 () {58 vars in 1104 bytes} [Sat Oct 1 21:05:58 2022] GET /identity/v3 => generated 255 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:05:58.153022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f22dc1-77e5-4702-bf4d-32f87f1b511d None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:58.154340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f22dc1-77e5-4702-bf4d-32f87f1b511d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:58.154658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f22dc1-77e5-4702-bf4d-32f87f1b511d None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:58.161413 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2567/5196] 10.222.0.17 () {56 vars in 895 bytes} [Sat Oct 1 21:05:58 2022] GET /identity/ => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 21:05:58.170475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37714600-fe59-4b4d-96d4-d8c07b7db73f None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:58.170820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37714600-fe59-4b4d-96d4-d8c07b7db73f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:58.171102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37714600-fe59-4b4d-96d4-d8c07b7db73f None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:58.178133 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2630/5197] 10.222.0.17 () {56 vars in 895 bytes} [Sat Oct 1 21:05:58 2022] GET /identity/ => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 21:05:58.180618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f22dc1-77e5-4702-bf4d-32f87f1b511d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:58.308804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:58.351997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f22dc1-77e5-4702-bf4d-32f87f1b511d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:58.359612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7290842-8595-4210-99f8-efeee056488b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:58.360976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7290842-8595-4210-99f8-efeee056488b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:58.361406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7290842-8595-4210-99f8-efeee056488b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:58.362222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7290842-8595-4210-99f8-efeee056488b admin admin] PATH_INFO: `/v3/users/b60c825d444d435b863af3ef381164b6` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:58.364045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7290842-8595-4210-99f8-efeee056488b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b60c825d444d435b863af3ef381164b6)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:58.368429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7290842-8595-4210-99f8-efeee056488b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:58.480413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b7290842-8595-4210-99f8-efeee056488b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b60c825d444d435b863af3ef381164b6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:58.482693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b7290842-8595-4210-99f8-efeee056488b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b60c825d444d435b863af3ef381164b6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:58.484442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b7290842-8595-4210-99f8-efeee056488b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b60c825d444d435b863af3ef381164b6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:58.515483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b7290842-8595-4210-99f8-efeee056488b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b60c825d444d435b863af3ef381164b6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:58.568911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b7290842-8595-4210-99f8-efeee056488b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b60c825d444d435b863af3ef381164b6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:05:58.598658 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2568/5198] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:05:58 2022] DELETE /identity/v3/users/b60c825d444d435b863af3ef381164b6 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:05:58.600563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef3106-04f6-4b8e-a1b3-5f30161bac3d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:58.600868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef3106-04f6-4b8e-a1b3-5f30161bac3d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:58.601122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef3106-04f6-4b8e-a1b3-5f30161bac3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:58.631606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-56ef3106-04f6-4b8e-a1b3-5f30161bac3d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:05:58.633414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-56ef3106-04f6-4b8e-a1b3-5f30161bac3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:58.737302 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2631/5199] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:05:58.813024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7290842-8595-4210-99f8-efeee056488b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:05:58.814559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b7290842-8595-4210-99f8-efeee056488b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:05:58.951599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7290842-8595-4210-99f8-efeee056488b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:05:58.965935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911aceb6-ca7d-4817-91aa-cee43c65763a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:05:58.966683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911aceb6-ca7d-4817-91aa-cee43c65763a service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:05:58.967006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911aceb6-ca7d-4817-91aa-cee43c65763a service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:05:58.967282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911aceb6-ca7d-4817-91aa-cee43c65763a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:05:58.996059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911aceb6-ca7d-4817-91aa-cee43c65763a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:05:58.998053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911aceb6-ca7d-4817-91aa-cee43c65763a service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:05:59.033222 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2569/5200] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:05:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:06:01.571656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef3106-04f6-4b8e-a1b3-5f30161bac3d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:01.579508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-56ef3106-04f6-4b8e-a1b3-5f30161bac3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:01.608408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef3106-04f6-4b8e-a1b3-5f30161bac3d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:01.629893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6dfc3-bb35-4f75-a93a-b35a65f97268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:01.630797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6dfc3-bb35-4f75-a93a-b35a65f97268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:01.630898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6dfc3-bb35-4f75-a93a-b35a65f97268 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:01.632474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6dfc3-bb35-4f75-a93a-b35a65f97268 admin admin] PATH_INFO: `/v3/projects/e1bf22a392d1466fa92860a3cac7169a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:01.634367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6dfc3-bb35-4f75-a93a-b35a65f97268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1bf22a392d1466fa92860a3cac7169a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:01.636324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6dfc3-bb35-4f75-a93a-b35a65f97268 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:01.760000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8ae6dfc3-bb35-4f75-a93a-b35a65f97268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1bf22a392d1466fa92860a3cac7169a', 'request_id': 'req-8ae6dfc3-bb35-4f75-a93a-b35a65f97268'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:01.774838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8ae6dfc3-bb35-4f75-a93a-b35a65f97268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1bf22a392d1466fa92860a3cac7169a', 'request_id': 'req-8ae6dfc3-bb35-4f75-a93a-b35a65f97268'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:01.791396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8ae6dfc3-bb35-4f75-a93a-b35a65f97268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1bf22a392d1466fa92860a3cac7169a', 'request_id': 'req-8ae6dfc3-bb35-4f75-a93a-b35a65f97268'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:01.814358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8ae6dfc3-bb35-4f75-a93a-b35a65f97268 admin admin] The token cache is being invalidate because project e1bf22a392d1466fa92860a3cac7169a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:06:01.814358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8ae6dfc3-bb35-4f75-a93a-b35a65f97268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:01.814998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8ae6dfc3-bb35-4f75-a93a-b35a65f97268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:01.827601 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2632/5201] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:06:01 2022] DELETE /identity/v3/projects/e1bf22a392d1466fa92860a3cac7169a => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:01.834120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60eb4a2-961e-49bd-a58c-bf643362b001 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:01.834655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60eb4a2-961e-49bd-a58c-bf643362b001 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:01.834980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60eb4a2-961e-49bd-a58c-bf643362b001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:01.862235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f60eb4a2-961e-49bd-a58c-bf643362b001 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:01.863655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f60eb4a2-961e-49bd-a58c-bf643362b001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:02.001964 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2570/5202] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:02.004353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369c7fd-cf53-4dcf-bb45-89b1306eae8a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:02.004573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369c7fd-cf53-4dcf-bb45-89b1306eae8a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:02.004898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369c7fd-cf53-4dcf-bb45-89b1306eae8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:02.056597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3369c7fd-cf53-4dcf-bb45-89b1306eae8a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:02.058012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3369c7fd-cf53-4dcf-bb45-89b1306eae8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:02.070654 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2633/5203] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:02.074711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60eb4a2-961e-49bd-a58c-bf643362b001 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:02.080552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:02.090150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60eb4a2-961e-49bd-a58c-bf643362b001 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:02.097950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2cde3-7983-46aa-bb6b-babf7e10b954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:02.098763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2cde3-7983-46aa-bb6b-babf7e10b954 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:02.100111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2cde3-7983-46aa-bb6b-babf7e10b954 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:02.100408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2cde3-7983-46aa-bb6b-babf7e10b954 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:02.101312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2cde3-7983-46aa-bb6b-babf7e10b954 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:06:02.101622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2cde3-7983-46aa-bb6b-babf7e10b954 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:02.103472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2cde3-7983-46aa-bb6b-babf7e10b954 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:02.125801 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2571/5204] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:06:02.127072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369c7fd-cf53-4dcf-bb45-89b1306eae8a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:02.142036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369c7fd-cf53-4dcf-bb45-89b1306eae8a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:02.157258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed07ea-5bfb-4154-8371-a3b13d02ecf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:02.158085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed07ea-5bfb-4154-8371-a3b13d02ecf3 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:02.159359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed07ea-5bfb-4154-8371-a3b13d02ecf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:02.159830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed07ea-5bfb-4154-8371-a3b13d02ecf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:02.160807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed07ea-5bfb-4154-8371-a3b13d02ecf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:02.162754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed07ea-5bfb-4154-8371-a3b13d02ecf3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:02.221574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2cde3-7983-46aa-bb6b-babf7e10b954 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:02.222372 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2634/5205] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:06:02 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:02.242857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2cde3-7983-46aa-bb6b-babf7e10b954 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:02.332460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902b460-d19d-4963-b685-d0c1eeeee39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:02.333089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902b460-d19d-4963-b685-d0c1eeeee39d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:02.333372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902b460-d19d-4963-b685-d0c1eeeee39d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:02.333646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902b460-d19d-4963-b685-d0c1eeeee39d admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:02.334283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902b460-d19d-4963-b685-d0c1eeeee39d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:02.336411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902b460-d19d-4963-b685-d0c1eeeee39d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:02.368183 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2572/5206] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:06:02 2022] POST /identity/v3/users => generated 287 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:02.373229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed07ea-5bfb-4154-8371-a3b13d02ecf3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:02.387186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed07ea-5bfb-4154-8371-a3b13d02ecf3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:02.398209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c35e6-dea5-40b9-9a89-1a99592926e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:02.398794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c35e6-dea5-40b9-9a89-1a99592926e2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:02.399072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c35e6-dea5-40b9-9a89-1a99592926e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:02.399342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c35e6-dea5-40b9-9a89-1a99592926e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:02.400025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c35e6-dea5-40b9-9a89-1a99592926e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:02.401615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c35e6-dea5-40b9-9a89-1a99592926e2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:02.430885 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2635/5207] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:02.441289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902b460-d19d-4963-b685-d0c1eeeee39d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:02.480118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902b460-d19d-4963-b685-d0c1eeeee39d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:02.494467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42088c4c-1dc8-4654-8db4-54a2694e547b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:02.495514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42088c4c-1dc8-4654-8db4-54a2694e547b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:02.495796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42088c4c-1dc8-4654-8db4-54a2694e547b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:02.496163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42088c4c-1dc8-4654-8db4-54a2694e547b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:02.496803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42088c4c-1dc8-4654-8db4-54a2694e547b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:02.500280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42088c4c-1dc8-4654-8db4-54a2694e547b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:02.522623 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2573/5208] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:02.527013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c35e6-dea5-40b9-9a89-1a99592926e2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:02.540305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c35e6-dea5-40b9-9a89-1a99592926e2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:02.557996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d599474-5d6d-411f-9f93-ee0a97bfc63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:02.558733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d599474-5d6d-411f-9f93-ee0a97bfc63a admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:02.559560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d599474-5d6d-411f-9f93-ee0a97bfc63a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:02.559793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d599474-5d6d-411f-9f93-ee0a97bfc63a admin admin] PATH_INFO: `/v3/projects/ff94f6d0c75c41eb955991379d67008a/users/f75fd47cfb17461b9e5c12c449ef4590/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:02.583507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d599474-5d6d-411f-9f93-ee0a97bfc63a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff94f6d0c75c41eb955991379d67008a, user_id=f75fd47cfb17461b9e5c12c449ef4590, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:02.586761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d599474-5d6d-411f-9f93-ee0a97bfc63a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:02.616752 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2636/5209] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:06:02 2022] PUT /identity/v3/projects/ff94f6d0c75c41eb955991379d67008a/users/f75fd47cfb17461b9e5c12c449ef4590/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:02.627450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f47be-1fc3-4195-a0e0-f9b866886c9e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:02.628050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f47be-1fc3-4195-a0e0-f9b866886c9e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:02.629424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f47be-1fc3-4195-a0e0-f9b866886c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:02.674060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5e9f47be-1fc3-4195-a0e0-f9b866886c9e None None] MFA Rules not processed for user `f75fd47cfb17461b9e5c12c449ef4590`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:02.675480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5e9f47be-1fc3-4195-a0e0-f9b866886c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:02.762241 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2574/5210] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:02.768037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d599474-5d6d-411f-9f93-ee0a97bfc63a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:02.782813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d599474-5d6d-411f-9f93-ee0a97bfc63a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:02.788034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17bc56-a2e1-4396-a84e-b10f845e11d5 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f75fd47cfb17461b9e5c12c449ef4590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff94f6d0c75c41eb955991379d67008a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:02.788708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17bc56-a2e1-4396-a84e-b10f845e11d5 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:02.789067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17bc56-a2e1-4396-a84e-b10f845e11d5 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:02.790632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17bc56-a2e1-4396-a84e-b10f845e11d5 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] PATH_INFO: `/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:02.791390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17bc56-a2e1-4396-a84e-b10f845e11d5 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=f75fd47cfb17461b9e5c12c449ef4590)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:02.795229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17bc56-a2e1-4396-a84e-b10f845e11d5 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:02.836859 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2637/5211] 10.222.0.17 () {60 vars in 1299 bytes} [Sat Oct 1 21:06:02 2022] POST /identity/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2 => generated 367 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:02.842129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f47be-1fc3-4195-a0e0-f9b866886c9e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:02.853456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f47be-1fc3-4195-a0e0-f9b866886c9e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:02.860639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e5d55-4668-4501-8ea1-0706938d04de tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f75fd47cfb17461b9e5c12c449ef4590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff94f6d0c75c41eb955991379d67008a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:02.861166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e5d55-4668-4501-8ea1-0706938d04de tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:02.861456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e5d55-4668-4501-8ea1-0706938d04de tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:02.861813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e5d55-4668-4501-8ea1-0706938d04de tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] PATH_INFO: `/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2/a971753ab2e843cd8bf4ae4d17f106da` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:02.865322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e5d55-4668-4501-8ea1-0706938d04de tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=f75fd47cfb17461b9e5c12c449ef4590, credential_id=a971753ab2e843cd8bf4ae4d17f106da)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:02.867254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e5d55-4668-4501-8ea1-0706938d04de tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:02.901560 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2575/5212] 10.222.0.17 () {58 vars in 1380 bytes} [Sat Oct 1 21:06:02 2022] DELETE /identity/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2/a971753ab2e843cd8bf4ae4d17f106da => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:02.908526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17bc56-a2e1-4396-a84e-b10f845e11d5 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:02.915404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:02.925529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17bc56-a2e1-4396-a84e-b10f845e11d5 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:02.934326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a929a5-7939-4bfa-a941-cbbeef7c9a21 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f75fd47cfb17461b9e5c12c449ef4590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff94f6d0c75c41eb955991379d67008a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:02.936618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a929a5-7939-4bfa-a941-cbbeef7c9a21 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:02.937220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a929a5-7939-4bfa-a941-cbbeef7c9a21 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:02.937881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a929a5-7939-4bfa-a941-cbbeef7c9a21 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] PATH_INFO: `/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:02.938891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a929a5-7939-4bfa-a941-cbbeef7c9a21 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=f75fd47cfb17461b9e5c12c449ef4590)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:02.941477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a929a5-7939-4bfa-a941-cbbeef7c9a21 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:02.975670 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2638/5213] 10.222.0.17 () {60 vars in 1299 bytes} [Sat Oct 1 21:06:02 2022] POST /identity/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2 => generated 367 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:02.979028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e5d55-4668-4501-8ea1-0706938d04de tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:03.008550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e5d55-4668-4501-8ea1-0706938d04de tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:03.017524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df210db-eb48-4351-b9d3-db9ccacf9f2a tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f75fd47cfb17461b9e5c12c449ef4590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff94f6d0c75c41eb955991379d67008a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:03.018162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df210db-eb48-4351-b9d3-db9ccacf9f2a tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:03.018689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df210db-eb48-4351-b9d3-db9ccacf9f2a tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:03.018973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df210db-eb48-4351-b9d3-db9ccacf9f2a tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] PATH_INFO: `/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2/649d8f365b1844a1b823f2923ba7c12d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:03.021386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df210db-eb48-4351-b9d3-db9ccacf9f2a tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=f75fd47cfb17461b9e5c12c449ef4590, credential_id=649d8f365b1844a1b823f2923ba7c12d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:03.023029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df210db-eb48-4351-b9d3-db9ccacf9f2a tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:03.056725 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2576/5214] 10.222.0.17 () {58 vars in 1380 bytes} [Sat Oct 1 21:06:02 2022] DELETE /identity/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2/649d8f365b1844a1b823f2923ba7c12d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:03.058209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a929a5-7939-4bfa-a941-cbbeef7c9a21 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:03.073025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a929a5-7939-4bfa-a941-cbbeef7c9a21 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:03.078557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c94ea-78f8-47ac-9c10-1513562807c4 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f75fd47cfb17461b9e5c12c449ef4590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff94f6d0c75c41eb955991379d67008a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:03.079607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c94ea-78f8-47ac-9c10-1513562807c4 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:03.080055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c94ea-78f8-47ac-9c10-1513562807c4 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:03.080478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c94ea-78f8-47ac-9c10-1513562807c4 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] PATH_INFO: `/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2/649d8f365b1844a1b823f2923ba7c12d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:03.133679 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-cf2c94ea-78f8-47ac-9c10-1513562807c4 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Could not find credential: 42afc43b89ad39eea66b6844d681da302c127c33009579d4af825b9f76996022.: keystone.exception.CredentialNotFound: Could not find credential: 42afc43b89ad39eea66b6844d681da302c127c33009579d4af825b9f76996022. Oct 01 21:06:03.133679 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:06:03.133679 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:06:03.133679 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:06:03.133679 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:06:03.133679 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:06:03.133679 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:06:03.133679 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:06:03.133679 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:06:03.133679 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:06:03.133679 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:06:03.133679 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:06:03.133679 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 426, in get Oct 01 21:06:03.133679 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Oct 01 21:06:03.134613 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Oct 01 21:06:03.134613 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Oct 01 21:06:03.134613 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 119, in _build_enforcer_target_data_owner_and_user_id_match Oct 01 21:06:03.134613 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref['credential'] = PROVIDERS.credential_api.get_credential( Oct 01 21:06:03.134613 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:06:03.134613 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:06:03.134613 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/core.py", line 119, in get_credential Oct 01 21:06:03.134613 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application credential = self._get_credential(credential_id) Oct 01 21:06:03.134613 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:06:03.134613 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:06:03.134613 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:06:03.134613 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:06:03.134613 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:06:03.134613 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 21:06:03.134613 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:06:03.134613 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:06:03.135882 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:06:03.135882 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:06:03.135882 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:06:03.135882 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:06:03.135882 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:06:03.135882 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:06:03.135882 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/core.py", line 124, in _get_credential Oct 01 21:06:03.135882 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.get_credential(credential_id) Oct 01 21:06:03.135882 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/backends/sql.py", line 88, in get_credential Oct 01 21:06:03.135882 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_credential(session, credential_id).to_dict() Oct 01 21:06:03.135882 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/backends/sql.py", line 83, in _get_credential Oct 01 21:06:03.135882 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.CredentialNotFound(credential_id=credential_id) Oct 01 21:06:03.135882 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.CredentialNotFound: Could not find credential: 42afc43b89ad39eea66b6844d681da302c127c33009579d4af825b9f76996022. Oct 01 21:06:03.135882 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:06:03.139873 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2639/5215] 10.222.0.17 () {58 vars in 1377 bytes} [Sat Oct 1 21:06:03 2022] GET /identity/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2/649d8f365b1844a1b823f2923ba7c12d => generated 148 bytes in 82 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:06:03.140671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df210db-eb48-4351-b9d3-db9ccacf9f2a tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:03.151359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df210db-eb48-4351-b9d3-db9ccacf9f2a tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:03.156449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783f2d2-11a9-4860-83e1-8b6a442e218f tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f75fd47cfb17461b9e5c12c449ef4590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff94f6d0c75c41eb955991379d67008a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:03.157114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783f2d2-11a9-4860-83e1-8b6a442e218f tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:03.157584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783f2d2-11a9-4860-83e1-8b6a442e218f tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:03.157867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783f2d2-11a9-4860-83e1-8b6a442e218f tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] PATH_INFO: `/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:03.158516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783f2d2-11a9-4860-83e1-8b6a442e218f tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=f75fd47cfb17461b9e5c12c449ef4590)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:03.160039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783f2d2-11a9-4860-83e1-8b6a442e218f tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:03.191783 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2577/5216] 10.222.0.17 () {60 vars in 1299 bytes} [Sat Oct 1 21:06:03 2022] POST /identity/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2 => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:03.195782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c94ea-78f8-47ac-9c10-1513562807c4 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:03.209361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c94ea-78f8-47ac-9c10-1513562807c4 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:03.214647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91fe5d-c72b-46fb-b031-c878dc9925e2 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f75fd47cfb17461b9e5c12c449ef4590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff94f6d0c75c41eb955991379d67008a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:03.215346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91fe5d-c72b-46fb-b031-c878dc9925e2 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:03.215632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91fe5d-c72b-46fb-b031-c878dc9925e2 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:03.216816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91fe5d-c72b-46fb-b031-c878dc9925e2 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] PATH_INFO: `/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:03.217562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91fe5d-c72b-46fb-b031-c878dc9925e2 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=f75fd47cfb17461b9e5c12c449ef4590)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:03.219384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91fe5d-c72b-46fb-b031-c878dc9925e2 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:03.259805 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2640/5217] 10.222.0.17 () {60 vars in 1299 bytes} [Sat Oct 1 21:06:03 2022] POST /identity/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2 => generated 367 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:03.264290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783f2d2-11a9-4860-83e1-8b6a442e218f tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:03.280660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783f2d2-11a9-4860-83e1-8b6a442e218f tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:03.285641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c89daf-eb7c-4df8-98c8-ce8fecfd9abc tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f75fd47cfb17461b9e5c12c449ef4590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff94f6d0c75c41eb955991379d67008a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:03.286292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c89daf-eb7c-4df8-98c8-ce8fecfd9abc tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:03.286615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c89daf-eb7c-4df8-98c8-ce8fecfd9abc tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:03.286890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c89daf-eb7c-4df8-98c8-ce8fecfd9abc tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] PATH_INFO: `/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:03.287473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c89daf-eb7c-4df8-98c8-ce8fecfd9abc tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorizing `identity:ec2_list_credentials(user_id=f75fd47cfb17461b9e5c12c449ef4590)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:03.289091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c89daf-eb7c-4df8-98c8-ce8fecfd9abc tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:03.306793 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2578/5218] 10.222.0.17 () {58 vars in 1278 bytes} [Sat Oct 1 21:06:03 2022] GET /identity/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2 => generated 865 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:06:03.311549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91fe5d-c72b-46fb-b031-c878dc9925e2 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:03.330527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91fe5d-c72b-46fb-b031-c878dc9925e2 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:03.337040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295b783-2fd3-4dce-952d-41bbfcba90fe tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f75fd47cfb17461b9e5c12c449ef4590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff94f6d0c75c41eb955991379d67008a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:03.337807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295b783-2fd3-4dce-952d-41bbfcba90fe tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:03.338195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295b783-2fd3-4dce-952d-41bbfcba90fe tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:03.338655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295b783-2fd3-4dce-952d-41bbfcba90fe tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] PATH_INFO: `/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2/03fca290dcd0437686993841433570eb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:03.341513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295b783-2fd3-4dce-952d-41bbfcba90fe tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=f75fd47cfb17461b9e5c12c449ef4590, credential_id=03fca290dcd0437686993841433570eb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:03.343132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295b783-2fd3-4dce-952d-41bbfcba90fe tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:03.376633 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2641/5219] 10.222.0.17 () {58 vars in 1380 bytes} [Sat Oct 1 21:06:03 2022] DELETE /identity/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2/03fca290dcd0437686993841433570eb => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:03.380848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c89daf-eb7c-4df8-98c8-ce8fecfd9abc tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:03.390136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:03.396632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c89daf-eb7c-4df8-98c8-ce8fecfd9abc tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:03.415495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1487e2-2426-4c80-8d65-6381f67a2a9d tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f75fd47cfb17461b9e5c12c449ef4590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff94f6d0c75c41eb955991379d67008a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:03.416367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1487e2-2426-4c80-8d65-6381f67a2a9d tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:03.416830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1487e2-2426-4c80-8d65-6381f67a2a9d tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:03.417121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1487e2-2426-4c80-8d65-6381f67a2a9d tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] PATH_INFO: `/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2/cec8421eacdb46f99fa48f83291c0418` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:03.419682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1487e2-2426-4c80-8d65-6381f67a2a9d tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=f75fd47cfb17461b9e5c12c449ef4590, credential_id=cec8421eacdb46f99fa48f83291c0418)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:03.421460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1487e2-2426-4c80-8d65-6381f67a2a9d tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:03.464917 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2579/5220] 10.222.0.17 () {58 vars in 1380 bytes} [Sat Oct 1 21:06:03 2022] DELETE /identity/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2/cec8421eacdb46f99fa48f83291c0418 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:03.468176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295b783-2fd3-4dce-952d-41bbfcba90fe tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:03.502887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295b783-2fd3-4dce-952d-41bbfcba90fe tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:03.507949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab33ce5-93b9-4c1b-bfb6-859b34148ce6 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f75fd47cfb17461b9e5c12c449ef4590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff94f6d0c75c41eb955991379d67008a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:03.509865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab33ce5-93b9-4c1b-bfb6-859b34148ce6 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:03.510900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab33ce5-93b9-4c1b-bfb6-859b34148ce6 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:03.511387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab33ce5-93b9-4c1b-bfb6-859b34148ce6 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] PATH_INFO: `/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:03.512735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab33ce5-93b9-4c1b-bfb6-859b34148ce6 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=f75fd47cfb17461b9e5c12c449ef4590)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:03.514383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab33ce5-93b9-4c1b-bfb6-859b34148ce6 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:03.545247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512e6a8-4f6f-4d5a-b03c-658f72fd08e8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:03.545552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512e6a8-4f6f-4d5a-b03c-658f72fd08e8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:03.545835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512e6a8-4f6f-4d5a-b03c-658f72fd08e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:03.588043 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2642/5221] 10.222.0.17 () {60 vars in 1299 bytes} [Sat Oct 1 21:06:03 2022] POST /identity/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2 => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:03.592039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab33ce5-93b9-4c1b-bfb6-859b34148ce6 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:03.598745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5512e6a8-4f6f-4d5a-b03c-658f72fd08e8 None None] MFA Rules not processed for user `6d63c633112e49d091421c84cea3422b`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:03.600986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5512e6a8-4f6f-4d5a-b03c-658f72fd08e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:03.603134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab33ce5-93b9-4c1b-bfb6-859b34148ce6 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:03.608934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7832e-fdc2-4d8c-b685-8e5b7e4a1dd1 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f75fd47cfb17461b9e5c12c449ef4590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff94f6d0c75c41eb955991379d67008a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:03.609595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7832e-fdc2-4d8c-b685-8e5b7e4a1dd1 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:03.610372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7832e-fdc2-4d8c-b685-8e5b7e4a1dd1 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:03.610701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7832e-fdc2-4d8c-b685-8e5b7e4a1dd1 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] PATH_INFO: `/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2/e576d33bf061434e8f5cf7842c915a36` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:03.613115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd7832e-fdc2-4d8c-b685-8e5b7e4a1dd1 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorizing `identity:ec2_get_credential(user_id=f75fd47cfb17461b9e5c12c449ef4590, credential_id=e576d33bf061434e8f5cf7842c915a36)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:03.614726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd7832e-fdc2-4d8c-b685-8e5b7e4a1dd1 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:03.621740 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2643/5222] 10.222.0.17 () {58 vars in 1377 bytes} [Sat Oct 1 21:06:03 2022] GET /identity/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2/e576d33bf061434e8f5cf7842c915a36 => generated 367 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:06:03.623575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7832e-fdc2-4d8c-b685-8e5b7e4a1dd1 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:03.638152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7832e-fdc2-4d8c-b685-8e5b7e4a1dd1 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:03.645089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a0f96-e09b-4ff6-8007-9bd3c86cebd2 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f75fd47cfb17461b9e5c12c449ef4590', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff94f6d0c75c41eb955991379d67008a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:03.645661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a0f96-e09b-4ff6-8007-9bd3c86cebd2 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:03.650551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a0f96-e09b-4ff6-8007-9bd3c86cebd2 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:03.651240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a0f96-e09b-4ff6-8007-9bd3c86cebd2 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] PATH_INFO: `/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2/e576d33bf061434e8f5cf7842c915a36` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:03.654771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a0f96-e09b-4ff6-8007-9bd3c86cebd2 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=f75fd47cfb17461b9e5c12c449ef4590, credential_id=e576d33bf061434e8f5cf7842c915a36)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:03.662070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a0f96-e09b-4ff6-8007-9bd3c86cebd2 tempest-EC2CredentialsTest-2059844626 tempest-EC2CredentialsTest-2059844626-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:03.704090 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2580/5223] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:06:03.704287 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2644/5224] 10.222.0.17 () {58 vars in 1380 bytes} [Sat Oct 1 21:06:03 2022] DELETE /identity/v3/users/f75fd47cfb17461b9e5c12c449ef4590/credentials/OS-EC2/e576d33bf061434e8f5cf7842c915a36 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:03.705217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512e6a8-4f6f-4d5a-b03c-658f72fd08e8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:03.833748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512e6a8-4f6f-4d5a-b03c-658f72fd08e8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:03.842824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626b6d2-1082-4851-9f3a-14ceac1bf22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:03.844184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626b6d2-1082-4851-9f3a-14ceac1bf22a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:03.844875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626b6d2-1082-4851-9f3a-14ceac1bf22a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:03.845333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626b6d2-1082-4851-9f3a-14ceac1bf22a admin admin] PATH_INFO: `/v3/users/f75fd47cfb17461b9e5c12c449ef4590` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:03.847386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626b6d2-1082-4851-9f3a-14ceac1bf22a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f75fd47cfb17461b9e5c12c449ef4590)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:03.850579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626b6d2-1082-4851-9f3a-14ceac1bf22a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:03.941368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e626b6d2-1082-4851-9f3a-14ceac1bf22a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f75fd47cfb17461b9e5c12c449ef4590'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:03.942150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e626b6d2-1082-4851-9f3a-14ceac1bf22a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f75fd47cfb17461b9e5c12c449ef4590'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:03.942831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e626b6d2-1082-4851-9f3a-14ceac1bf22a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f75fd47cfb17461b9e5c12c449ef4590'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:03.957480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e626b6d2-1082-4851-9f3a-14ceac1bf22a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f75fd47cfb17461b9e5c12c449ef4590'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:03.976427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e626b6d2-1082-4851-9f3a-14ceac1bf22a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f75fd47cfb17461b9e5c12c449ef4590'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:03.994099 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2581/5225] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:06:03 2022] DELETE /identity/v3/users/f75fd47cfb17461b9e5c12c449ef4590 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:04.000192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf385e8-9d1b-4187-82f1-d06faba23149 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:04.000646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf385e8-9d1b-4187-82f1-d06faba23149 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:04.001041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf385e8-9d1b-4187-82f1-d06faba23149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:04.024178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1bf385e8-9d1b-4187-82f1-d06faba23149 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:04.025524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1bf385e8-9d1b-4187-82f1-d06faba23149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:04.154513 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2645/5226] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:04.183549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626b6d2-1082-4851-9f3a-14ceac1bf22a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:04.185350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e626b6d2-1082-4851-9f3a-14ceac1bf22a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:04.224484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:04.311077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626b6d2-1082-4851-9f3a-14ceac1bf22a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:04.322933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f43404-1ed3-4f3b-85bc-338a7f2f6fb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:04.324404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f43404-1ed3-4f3b-85bc-338a7f2f6fb2 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:04.324831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f43404-1ed3-4f3b-85bc-338a7f2f6fb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:04.325353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f43404-1ed3-4f3b-85bc-338a7f2f6fb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:04.359121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f43404-1ed3-4f3b-85bc-338a7f2f6fb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:04.360932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f43404-1ed3-4f3b-85bc-338a7f2f6fb2 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:04.375706 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2582/5227] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:06:06.011486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf385e8-9d1b-4187-82f1-d06faba23149 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:06.014057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1bf385e8-9d1b-4187-82f1-d06faba23149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:06.033569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf385e8-9d1b-4187-82f1-d06faba23149 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:06.055989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1028977-3ac2-4772-8b58-4302fd309045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:06.056919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1028977-3ac2-4772-8b58-4302fd309045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:06.057363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1028977-3ac2-4772-8b58-4302fd309045 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:06.057795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1028977-3ac2-4772-8b58-4302fd309045 admin admin] PATH_INFO: `/v3/projects/ff94f6d0c75c41eb955991379d67008a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:06.060683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1028977-3ac2-4772-8b58-4302fd309045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff94f6d0c75c41eb955991379d67008a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:06.062901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1028977-3ac2-4772-8b58-4302fd309045 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:06.175659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a1028977-3ac2-4772-8b58-4302fd309045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff94f6d0c75c41eb955991379d67008a', 'request_id': 'req-a1028977-3ac2-4772-8b58-4302fd309045'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:06.185638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a1028977-3ac2-4772-8b58-4302fd309045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff94f6d0c75c41eb955991379d67008a', 'request_id': 'req-a1028977-3ac2-4772-8b58-4302fd309045'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:06.193953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a1028977-3ac2-4772-8b58-4302fd309045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff94f6d0c75c41eb955991379d67008a', 'request_id': 'req-a1028977-3ac2-4772-8b58-4302fd309045'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:06.213092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a1028977-3ac2-4772-8b58-4302fd309045 admin admin] The token cache is being invalidate because project ff94f6d0c75c41eb955991379d67008a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:06:06.213979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a1028977-3ac2-4772-8b58-4302fd309045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:06.215095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a1028977-3ac2-4772-8b58-4302fd309045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:06.218550 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2646/5228] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:06:06 2022] DELETE /identity/v3/projects/ff94f6d0c75c41eb955991379d67008a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:06.224167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647cd615-55e0-483c-a2cf-6ab9698f0d5f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:06.224724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647cd615-55e0-483c-a2cf-6ab9698f0d5f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:06.224724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647cd615-55e0-483c-a2cf-6ab9698f0d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:06.248482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-647cd615-55e0-483c-a2cf-6ab9698f0d5f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:06.250003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-647cd615-55e0-483c-a2cf-6ab9698f0d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:06.329362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1028977-3ac2-4772-8b58-4302fd309045 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:06.331176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a1028977-3ac2-4772-8b58-4302fd309045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:06.338247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:06.367691 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2583/5229] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:06.369540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480dcf8-96a9-462c-8b85-ba458975e6ca None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:06.369828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480dcf8-96a9-462c-8b85-ba458975e6ca None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:06.370382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480dcf8-96a9-462c-8b85-ba458975e6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:06.427611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0480dcf8-96a9-462c-8b85-ba458975e6ca None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:06.428865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0480dcf8-96a9-462c-8b85-ba458975e6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:06.460268 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2584/5230] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:06.461277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480dcf8-96a9-462c-8b85-ba458975e6ca None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:06.465205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0480dcf8-96a9-462c-8b85-ba458975e6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:06.479372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480dcf8-96a9-462c-8b85-ba458975e6ca None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:06.486591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa46a41-918a-48de-9273-ac01e187e122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:06.487246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa46a41-918a-48de-9273-ac01e187e122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:06.487523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa46a41-918a-48de-9273-ac01e187e122 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:06.487796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa46a41-918a-48de-9273-ac01e187e122 admin admin] PATH_INFO: `/v3/users/46c6f0cba68e41eab2d90bac5fba9442` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:06.489583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa46a41-918a-48de-9273-ac01e187e122 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46c6f0cba68e41eab2d90bac5fba9442)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:06.492313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa46a41-918a-48de-9273-ac01e187e122 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:06.533447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1028977-3ac2-4772-8b58-4302fd309045 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:06.550611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb43efb-9322-49d7-b5f3-b68aaefd424a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:06.551269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb43efb-9322-49d7-b5f3-b68aaefd424a service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:06.551552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb43efb-9322-49d7-b5f3-b68aaefd424a service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:06.551827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb43efb-9322-49d7-b5f3-b68aaefd424a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:06.557736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-afb43efb-9322-49d7-b5f3-b68aaefd424a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:06.583229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb43efb-9322-49d7-b5f3-b68aaefd424a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:06.584654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb43efb-9322-49d7-b5f3-b68aaefd424a service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:06.610355 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2647/5231] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:06:06.611219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb43efb-9322-49d7-b5f3-b68aaefd424a service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:06.624096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb43efb-9322-49d7-b5f3-b68aaefd424a service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:06.630238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7d36f-edab-4e09-a4d9-733deff432b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:06.631045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7d36f-edab-4e09-a4d9-733deff432b5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:06.631397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7d36f-edab-4e09-a4d9-733deff432b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:06.631778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7d36f-edab-4e09-a4d9-733deff432b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:06.632492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7d36f-edab-4e09-a4d9-733deff432b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:06:06.632866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7d36f-edab-4e09-a4d9-733deff432b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:06.635549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7d36f-edab-4e09-a4d9-733deff432b5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:06.662920 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2648/5232] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:06:06.662920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7d36f-edab-4e09-a4d9-733deff432b5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:06.679373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1fa46a41-918a-48de-9273-ac01e187e122 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46c6f0cba68e41eab2d90bac5fba9442'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:06.680549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1fa46a41-918a-48de-9273-ac01e187e122 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46c6f0cba68e41eab2d90bac5fba9442'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:06.680794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7d36f-edab-4e09-a4d9-733deff432b5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:06.681609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1fa46a41-918a-48de-9273-ac01e187e122 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46c6f0cba68e41eab2d90bac5fba9442'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:06.694291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b81a5-e568-41f2-a4f2-3e2fe1b5f8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:06.694772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1fa46a41-918a-48de-9273-ac01e187e122 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46c6f0cba68e41eab2d90bac5fba9442'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:06.695165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b81a5-e568-41f2-a4f2-3e2fe1b5f8aa admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:06.695419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b81a5-e568-41f2-a4f2-3e2fe1b5f8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:06.695690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b81a5-e568-41f2-a4f2-3e2fe1b5f8aa admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:06.696384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b81a5-e568-41f2-a4f2-3e2fe1b5f8aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:06.698404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b81a5-e568-41f2-a4f2-3e2fe1b5f8aa admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:06.718397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1fa46a41-918a-48de-9273-ac01e187e122 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46c6f0cba68e41eab2d90bac5fba9442'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:06.736682 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2585/5233] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:06:06 2022] DELETE /identity/v3/users/46c6f0cba68e41eab2d90bac5fba9442 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:06.769333 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2649/5234] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:06:06 2022] POST /identity/v3/projects => generated 391 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:06.773264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa46a41-918a-48de-9273-ac01e187e122 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:06.775463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1fa46a41-918a-48de-9273-ac01e187e122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:06.817006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:06.869927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa46a41-918a-48de-9273-ac01e187e122 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:06.942938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15994a-51a1-4a65-a55b-ababd53cc729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:06.943917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15994a-51a1-4a65-a55b-ababd53cc729 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:06.944352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15994a-51a1-4a65-a55b-ababd53cc729 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:06.944776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15994a-51a1-4a65-a55b-ababd53cc729 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:06.945731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15994a-51a1-4a65-a55b-ababd53cc729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:06.948927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15994a-51a1-4a65-a55b-ababd53cc729 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:06.982599 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2586/5235] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:06:06 2022] POST /identity/v3/users => generated 302 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:06.985614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b81a5-e568-41f2-a4f2-3e2fe1b5f8aa admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:07.001458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b81a5-e568-41f2-a4f2-3e2fe1b5f8aa admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:07.008553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a06342-eba0-4c74-8e9e-c5e30378cf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:07.009659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a06342-eba0-4c74-8e9e-c5e30378cf36 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:07.010202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a06342-eba0-4c74-8e9e-c5e30378cf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:07.010727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a06342-eba0-4c74-8e9e-c5e30378cf36 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:07.011874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a06342-eba0-4c74-8e9e-c5e30378cf36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:07.015771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a06342-eba0-4c74-8e9e-c5e30378cf36 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:07.028958 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2650/5236] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:07.032802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15994a-51a1-4a65-a55b-ababd53cc729 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:07.050660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15994a-51a1-4a65-a55b-ababd53cc729 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:07.058111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ced8f-b5c5-4fe4-9795-63cda699f764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:07.058934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ced8f-b5c5-4fe4-9795-63cda699f764 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:07.059305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ced8f-b5c5-4fe4-9795-63cda699f764 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:07.060172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ced8f-b5c5-4fe4-9795-63cda699f764 admin admin] PATH_INFO: `/v3/projects/76a25a013d104b17b8d7c9bd90e5a311/users/ac1e276605bf4f8f854356f122a4a5e0/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:07.098408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ced8f-b5c5-4fe4-9795-63cda699f764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76a25a013d104b17b8d7c9bd90e5a311, user_id=ac1e276605bf4f8f854356f122a4a5e0, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:07.102753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ced8f-b5c5-4fe4-9795-63cda699f764 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:07.137351 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2587/5237] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:06:07 2022] PUT /identity/v3/projects/76a25a013d104b17b8d7c9bd90e5a311/users/ac1e276605bf4f8f854356f122a4a5e0/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:07.139419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a06342-eba0-4c74-8e9e-c5e30378cf36 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:07.157985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a06342-eba0-4c74-8e9e-c5e30378cf36 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:07.234269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a8a32-c9ac-48f8-8f5a-352789ff9857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:07.234867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a8a32-c9ac-48f8-8f5a-352789ff9857 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:07.235152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a8a32-c9ac-48f8-8f5a-352789ff9857 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:07.235431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a8a32-c9ac-48f8-8f5a-352789ff9857 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:07.236085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a8a32-c9ac-48f8-8f5a-352789ff9857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:07.237686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a8a32-c9ac-48f8-8f5a-352789ff9857 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:07.251136 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2651/5238] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:07.255336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ced8f-b5c5-4fe4-9795-63cda699f764 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:07.272686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ced8f-b5c5-4fe4-9795-63cda699f764 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:07.278781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fdf027-c82c-41e5-8a82-fd153cce7c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:07.280678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdf027-c82c-41e5-8a82-fd153cce7c35 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:07.281066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdf027-c82c-41e5-8a82-fd153cce7c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:07.281414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdf027-c82c-41e5-8a82-fd153cce7c35 admin admin] PATH_INFO: `/v3/projects/76a25a013d104b17b8d7c9bd90e5a311/users/ac1e276605bf4f8f854356f122a4a5e0/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:07.284155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fdf027-c82c-41e5-8a82-fd153cce7c35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76a25a013d104b17b8d7c9bd90e5a311, user_id=ac1e276605bf4f8f854356f122a4a5e0, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:07.287439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fdf027-c82c-41e5-8a82-fd153cce7c35 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:07.299898 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2588/5239] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:06:07 2022] PUT /identity/v3/projects/76a25a013d104b17b8d7c9bd90e5a311/users/ac1e276605bf4f8f854356f122a4a5e0/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:07.312647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a6a27-17ca-403a-93d6-d19699f744b8 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:07.313027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a6a27-17ca-403a-93d6-d19699f744b8 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:07.313326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a6a27-17ca-403a-93d6-d19699f744b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:07.370248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8d6a6a27-17ca-403a-93d6-d19699f744b8 None None] MFA Rules not processed for user `ac1e276605bf4f8f854356f122a4a5e0`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:07.372916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8d6a6a27-17ca-403a-93d6-d19699f744b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:07.414445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:07.548307 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2652/5240] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:07 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:07.549665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fdf027-c82c-41e5-8a82-fd153cce7c35 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:07.739953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fdf027-c82c-41e5-8a82-fd153cce7c35 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:07.763405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771c417-d3aa-4bfe-942e-738f671f45b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:07.763405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771c417-d3aa-4bfe-942e-738f671f45b5 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:07.763405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771c417-d3aa-4bfe-942e-738f671f45b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:07.763405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771c417-d3aa-4bfe-942e-738f671f45b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:07.764335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771c417-d3aa-4bfe-942e-738f671f45b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:07.766525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771c417-d3aa-4bfe-942e-738f671f45b5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:07.776004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a6a27-17ca-403a-93d6-d19699f744b8 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:07.777499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8d6a6a27-17ca-403a-93d6-d19699f744b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:07.796478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a6a27-17ca-403a-93d6-d19699f744b8 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:07.802577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6c86c-6df1-4241-99f3-b116bc8ce36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:07.803543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6c86c-6df1-4241-99f3-b116bc8ce36b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:07.805591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6c86c-6df1-4241-99f3-b116bc8ce36b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:07.806107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6c86c-6df1-4241-99f3-b116bc8ce36b admin admin] PATH_INFO: `/v3/projects/8ea6424dd45e48ad93390e398a6fc926` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:07.807713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6c86c-6df1-4241-99f3-b116bc8ce36b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ea6424dd45e48ad93390e398a6fc926)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:07.811898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6c86c-6df1-4241-99f3-b116bc8ce36b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:07.845409 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2589/5241] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:06:07 2022] POST /identity/v3/projects => generated 391 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:07.853740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771c417-d3aa-4bfe-942e-738f671f45b5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:07.895582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:07.924971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-61a6c86c-6df1-4241-99f3-b116bc8ce36b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ea6424dd45e48ad93390e398a6fc926', 'request_id': 'req-61a6c86c-6df1-4241-99f3-b116bc8ce36b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:07.940969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-61a6c86c-6df1-4241-99f3-b116bc8ce36b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ea6424dd45e48ad93390e398a6fc926', 'request_id': 'req-61a6c86c-6df1-4241-99f3-b116bc8ce36b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:07.959555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-61a6c86c-6df1-4241-99f3-b116bc8ce36b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ea6424dd45e48ad93390e398a6fc926', 'request_id': 'req-61a6c86c-6df1-4241-99f3-b116bc8ce36b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:07.991774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-61a6c86c-6df1-4241-99f3-b116bc8ce36b admin admin] The token cache is being invalidate because project 8ea6424dd45e48ad93390e398a6fc926 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:06:07.992946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-61a6c86c-6df1-4241-99f3-b116bc8ce36b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:07.998418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-61a6c86c-6df1-4241-99f3-b116bc8ce36b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:08.001485 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2653/5242] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:06:07 2022] DELETE /identity/v3/projects/8ea6424dd45e48ad93390e398a6fc926 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:06:08.007305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fe0c8-9514-4f86-b98f-857e925900db None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:08.007834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fe0c8-9514-4f86-b98f-857e925900db None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:08.008544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fe0c8-9514-4f86-b98f-857e925900db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:08.070844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771c417-d3aa-4bfe-942e-738f671f45b5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:08.075962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a94fe0c8-9514-4f86-b98f-857e925900db None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:08.079050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a94fe0c8-9514-4f86-b98f-857e925900db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:08.240846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39395b4d-eeb9-4fb2-a338-78a1250db4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:08.241619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39395b4d-eeb9-4fb2-a338-78a1250db4a2 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:08.242200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39395b4d-eeb9-4fb2-a338-78a1250db4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:08.242578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39395b4d-eeb9-4fb2-a338-78a1250db4a2 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:08.243436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39395b4d-eeb9-4fb2-a338-78a1250db4a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:08.245762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39395b4d-eeb9-4fb2-a338-78a1250db4a2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:08.264372 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2654/5243] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:08.273422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c95ea6-b405-4a94-b647-8c5b074a3c89 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:08.273713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c95ea6-b405-4a94-b647-8c5b074a3c89 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:08.274048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c95ea6-b405-4a94-b647-8c5b074a3c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:08.305404 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2590/5244] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:06:07 2022] POST /identity/v3/users => generated 296 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:08.312185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39395b4d-eeb9-4fb2-a338-78a1250db4a2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:08.316202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-39395b4d-eeb9-4fb2-a338-78a1250db4a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:08.325095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b0c95ea6-b405-4a94-b647-8c5b074a3c89 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:08.326696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b0c95ea6-b405-4a94-b647-8c5b074a3c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:08.354871 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2655/5245] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:08.359954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39395b4d-eeb9-4fb2-a338-78a1250db4a2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:08.361203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c95ea6-b405-4a94-b647-8c5b074a3c89 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:08.377943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d2b2f-b142-4c4d-8b03-4b98fdf531ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:08.378809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d2b2f-b142-4c4d-8b03-4b98fdf531ea admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:08.379102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d2b2f-b142-4c4d-8b03-4b98fdf531ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:08.380617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d2b2f-b142-4c4d-8b03-4b98fdf531ea admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:08.381291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d2b2f-b142-4c4d-8b03-4b98fdf531ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:08.383917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c95ea6-b405-4a94-b647-8c5b074a3c89 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:08.388312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d2b2f-b142-4c4d-8b03-4b98fdf531ea admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:08.395279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53073e2e-e98c-4acd-8179-c2e47a8b3fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:08.396316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53073e2e-e98c-4acd-8179-c2e47a8b3fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:08.396929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53073e2e-e98c-4acd-8179-c2e47a8b3fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:08.397212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53073e2e-e98c-4acd-8179-c2e47a8b3fc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:08.398321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53073e2e-e98c-4acd-8179-c2e47a8b3fc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:06:08.398780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53073e2e-e98c-4acd-8179-c2e47a8b3fc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:08.416112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53073e2e-e98c-4acd-8179-c2e47a8b3fc8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:08.427263 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2591/5246] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:08.430770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d2b2f-b142-4c4d-8b03-4b98fdf531ea admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:08.457672 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2656/5247] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:06:08.461377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d2b2f-b142-4c4d-8b03-4b98fdf531ea admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:08.463314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53073e2e-e98c-4acd-8179-c2e47a8b3fc8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:08.475457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53073e2e-e98c-4acd-8179-c2e47a8b3fc8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:08.481053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564e339-a01b-4152-ad7b-282de86e8e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:08.482375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564e339-a01b-4152-ad7b-282de86e8e5f admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:08.482933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564e339-a01b-4152-ad7b-282de86e8e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:08.483931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564e339-a01b-4152-ad7b-282de86e8e5f admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:08.484880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564e339-a01b-4152-ad7b-282de86e8e5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:08.486959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564e339-a01b-4152-ad7b-282de86e8e5f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:08.495395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356c8af-be4a-47ac-974a-7677f606d3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:08.496207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356c8af-be4a-47ac-974a-7677f606d3a9 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:08.496492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356c8af-be4a-47ac-974a-7677f606d3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:08.497057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356c8af-be4a-47ac-974a-7677f606d3a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:08.498297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2356c8af-be4a-47ac-974a-7677f606d3a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:08.506437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2356c8af-be4a-47ac-974a-7677f606d3a9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:08.516871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:08.536327 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2657/5249] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:06:08 2022] POST /identity/v3/projects => generated 361 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:06:08.536327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564e339-a01b-4152-ad7b-282de86e8e5f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:08.536716 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2592/5249] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:08.538446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356c8af-be4a-47ac-974a-7677f606d3a9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:08.560120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356c8af-be4a-47ac-974a-7677f606d3a9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:08.675381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564e339-a01b-4152-ad7b-282de86e8e5f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:08.687947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7ae04-743f-471e-9dde-227aa4847ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:08.689095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7ae04-743f-471e-9dde-227aa4847ec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:08.689444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7ae04-743f-471e-9dde-227aa4847ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:08.690545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7ae04-743f-471e-9dde-227aa4847ec3 admin admin] PATH_INFO: `/v3/projects/47716815e1534059abaab16df21a4879/users/081b5c8233904f32bac857a6ec721d07/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:08.692105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a0681-e933-43a9-be77-457f703b816b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:08.694204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a0681-e933-43a9-be77-457f703b816b admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:08.695025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a0681-e933-43a9-be77-457f703b816b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:08.695373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a0681-e933-43a9-be77-457f703b816b admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:08.696350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a0681-e933-43a9-be77-457f703b816b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:08.703339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a0681-e933-43a9-be77-457f703b816b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:08.723842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb7ae04-743f-471e-9dde-227aa4847ec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47716815e1534059abaab16df21a4879, user_id=081b5c8233904f32bac857a6ec721d07, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:08.727671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb7ae04-743f-471e-9dde-227aa4847ec3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:08.751433 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2593/5250] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:06:08 2022] POST /identity/v3/users => generated 281 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:08.756085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a0681-e933-43a9-be77-457f703b816b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:08.761902 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2658/5251] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:06:08 2022] PUT /identity/v3/projects/47716815e1534059abaab16df21a4879/users/081b5c8233904f32bac857a6ec721d07/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:08.778264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8b7f0-4a3f-4c2f-bcab-e7afea24de09 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:08.782816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8b7f0-4a3f-4c2f-bcab-e7afea24de09 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:08.784761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8b7f0-4a3f-4c2f-bcab-e7afea24de09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:08.794606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a0681-e933-43a9-be77-457f703b816b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:08.816603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:08.871923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-56a8b7f0-4a3f-4c2f-bcab-e7afea24de09 None None] MFA Rules not processed for user `081b5c8233904f32bac857a6ec721d07`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:08.874199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-56a8b7f0-4a3f-4c2f-bcab-e7afea24de09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:09.002288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d6419f-90aa-4538-8875-2d080b486823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:09.003010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6419f-90aa-4538-8875-2d080b486823 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:09.003291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6419f-90aa-4538-8875-2d080b486823 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:09.003564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6419f-90aa-4538-8875-2d080b486823 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:09.008920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d6419f-90aa-4538-8875-2d080b486823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:09.023508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d6419f-90aa-4538-8875-2d080b486823 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:09.057019 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2659/5252] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:09.068977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8b7f0-4a3f-4c2f-bcab-e7afea24de09 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:09.071617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-56a8b7f0-4a3f-4c2f-bcab-e7afea24de09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:09.075364 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2594/5253] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:09.080757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d6419f-90aa-4538-8875-2d080b486823 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:09.100890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d6419f-90aa-4538-8875-2d080b486823 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:09.115482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd782d-37ff-4ce5-ad60-39e1c3b6e845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:09.116050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd782d-37ff-4ce5-ad60-39e1c3b6e845 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:09.116335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd782d-37ff-4ce5-ad60-39e1c3b6e845 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:09.116609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd782d-37ff-4ce5-ad60-39e1c3b6e845 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:09.117254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd782d-37ff-4ce5-ad60-39e1c3b6e845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:09.119498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd782d-37ff-4ce5-ad60-39e1c3b6e845 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:09.148175 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2595/5254] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:09.152942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd782d-37ff-4ce5-ad60-39e1c3b6e845 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:09.177940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd782d-37ff-4ce5-ad60-39e1c3b6e845 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:09.190284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240d04c-0beb-4f35-967c-7f8cfe793cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:09.190936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240d04c-0beb-4f35-967c-7f8cfe793cab admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:09.191240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240d04c-0beb-4f35-967c-7f8cfe793cab admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:09.191486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240d04c-0beb-4f35-967c-7f8cfe793cab admin admin] PATH_INFO: `/v3/projects/d027a23dba31494b9d3e63be39096a7c/users/0b69650f575c49bfa6da177f24994b3b/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:09.218436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8b7f0-4a3f-4c2f-bcab-e7afea24de09 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:09.223308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b40265-0058-4c3a-bd28-9a131b694492 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:09.224065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b40265-0058-4c3a-bd28-9a131b694492 service glance] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:09.228051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b40265-0058-4c3a-bd28-9a131b694492 service glance] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:09.228051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b40265-0058-4c3a-bd28-9a131b694492 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:09.236196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-58b40265-0058-4c3a-bd28-9a131b694492 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:09.267075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240d04c-0beb-4f35-967c-7f8cfe793cab admin admin] RBAC: Authorizing `identity:create_grant(project_id=d027a23dba31494b9d3e63be39096a7c, user_id=0b69650f575c49bfa6da177f24994b3b, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:09.269942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240d04c-0beb-4f35-967c-7f8cfe793cab admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:09.282974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:09.300998 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2596/5255] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:06:09 2022] PUT /identity/v3/projects/d027a23dba31494b9d3e63be39096a7c/users/0b69650f575c49bfa6da177f24994b3b/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:09.312229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79243b0-f749-4c82-95bb-961e901c7f8d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:09.312865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79243b0-f749-4c82-95bb-961e901c7f8d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:09.312865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79243b0-f749-4c82-95bb-961e901c7f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:09.373973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d79243b0-f749-4c82-95bb-961e901c7f8d None None] MFA Rules not processed for user `0b69650f575c49bfa6da177f24994b3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:09.375249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d79243b0-f749-4c82-95bb-961e901c7f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:09.385568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b40265-0058-4c3a-bd28-9a131b694492 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:09.387767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b40265-0058-4c3a-bd28-9a131b694492 service glance] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:09.402881 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2660/5256] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:09 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:06:09.603100 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2597/5257] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:06:09.898642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c877f-3390-40e1-9e9c-cf5ad7f88a52 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:09.898946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c877f-3390-40e1-9e9c-cf5ad7f88a52 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:09.899240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c877f-3390-40e1-9e9c-cf5ad7f88a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:09.962455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-db1c877f-3390-40e1-9e9c-cf5ad7f88a52 None None] MFA Rules not processed for user `6d63c633112e49d091421c84cea3422b`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:09.963904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-db1c877f-3390-40e1-9e9c-cf5ad7f88a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:10.166117 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2661/5258] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:06:11.179913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79243b0-f749-4c82-95bb-961e901c7f8d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:11.283662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79243b0-f749-4c82-95bb-961e901c7f8d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:11.289211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995add2-a366-4b6e-a114-e4f59554fe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:11.290580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995add2-a366-4b6e-a114-e4f59554fe0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:11.290891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995add2-a366-4b6e-a114-e4f59554fe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:11.291432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995add2-a366-4b6e-a114-e4f59554fe0c admin admin] PATH_INFO: `/v3/users/ac1e276605bf4f8f854356f122a4a5e0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:11.292954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995add2-a366-4b6e-a114-e4f59554fe0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac1e276605bf4f8f854356f122a4a5e0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:11.295436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995add2-a366-4b6e-a114-e4f59554fe0c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:11.303894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:11.404884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a995add2-a366-4b6e-a114-e4f59554fe0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac1e276605bf4f8f854356f122a4a5e0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:11.405560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a995add2-a366-4b6e-a114-e4f59554fe0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac1e276605bf4f8f854356f122a4a5e0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:11.406411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a995add2-a366-4b6e-a114-e4f59554fe0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac1e276605bf4f8f854356f122a4a5e0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:11.415981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a995add2-a366-4b6e-a114-e4f59554fe0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac1e276605bf4f8f854356f122a4a5e0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:11.433976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a995add2-a366-4b6e-a114-e4f59554fe0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac1e276605bf4f8f854356f122a4a5e0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:11.451433 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2598/5259] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:06:11 2022] DELETE /identity/v3/users/ac1e276605bf4f8f854356f122a4a5e0 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:06:11.455897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6683dad-d0ab-40f1-9232-8915aa600486 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:11.456367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6683dad-d0ab-40f1-9232-8915aa600486 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:11.456448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6683dad-d0ab-40f1-9232-8915aa600486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:11.485901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f6683dad-d0ab-40f1-9232-8915aa600486 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:11.487108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f6683dad-d0ab-40f1-9232-8915aa600486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:11.594388 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2662/5260] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:11.604576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995add2-a366-4b6e-a114-e4f59554fe0c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:11.607097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a995add2-a366-4b6e-a114-e4f59554fe0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:11.695985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995add2-a366-4b6e-a114-e4f59554fe0c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:11.702339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41dee49-0610-4103-a9c8-3777a3cb3825 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:11.703059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41dee49-0610-4103-a9c8-3777a3cb3825 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:11.703533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41dee49-0610-4103-a9c8-3777a3cb3825 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:11.703812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41dee49-0610-4103-a9c8-3777a3cb3825 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:11.715184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41dee49-0610-4103-a9c8-3777a3cb3825 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:11.717721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41dee49-0610-4103-a9c8-3777a3cb3825 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:11.729640 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2599/5261] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:06:11.959710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6683dad-d0ab-40f1-9232-8915aa600486 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:11.961537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f6683dad-d0ab-40f1-9232-8915aa600486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:11.985677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6683dad-d0ab-40f1-9232-8915aa600486 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:11.999775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0b280-a573-4a72-94c5-15fd1f7d2c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:12.000734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0b280-a573-4a72-94c5-15fd1f7d2c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:12.001670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0b280-a573-4a72-94c5-15fd1f7d2c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:12.002103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0b280-a573-4a72-94c5-15fd1f7d2c73 admin admin] PATH_INFO: `/v3/users/0b69650f575c49bfa6da177f24994b3b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:12.003724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0b280-a573-4a72-94c5-15fd1f7d2c73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b69650f575c49bfa6da177f24994b3b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:12.006876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0b280-a573-4a72-94c5-15fd1f7d2c73 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:12.100495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-86f0b280-a573-4a72-94c5-15fd1f7d2c73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b69650f575c49bfa6da177f24994b3b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:12.110386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:12.112988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-86f0b280-a573-4a72-94c5-15fd1f7d2c73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b69650f575c49bfa6da177f24994b3b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:12.134706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-86f0b280-a573-4a72-94c5-15fd1f7d2c73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b69650f575c49bfa6da177f24994b3b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:12.136255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-86f0b280-a573-4a72-94c5-15fd1f7d2c73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b69650f575c49bfa6da177f24994b3b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:12.157441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-86f0b280-a573-4a72-94c5-15fd1f7d2c73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b69650f575c49bfa6da177f24994b3b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:12.176162 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2663/5262] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:06:11 2022] DELETE /identity/v3/users/0b69650f575c49bfa6da177f24994b3b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:12.181303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b6e54-600b-4df0-9d35-6b3f7960fad8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:12.182542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b6e54-600b-4df0-9d35-6b3f7960fad8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:12.183027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b6e54-600b-4df0-9d35-6b3f7960fad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:12.212068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-049b6e54-600b-4df0-9d35-6b3f7960fad8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:12.213651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-049b6e54-600b-4df0-9d35-6b3f7960fad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:12.362077 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2600/5263] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:12.407037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0b280-a573-4a72-94c5-15fd1f7d2c73 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:12.410182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-86f0b280-a573-4a72-94c5-15fd1f7d2c73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:12.567054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0b280-a573-4a72-94c5-15fd1f7d2c73 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:12.582140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766fd43e-b387-421e-a299-87064a44e676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:12.583189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fd43e-b387-421e-a299-87064a44e676 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:12.583625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fd43e-b387-421e-a299-87064a44e676 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:12.584049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fd43e-b387-421e-a299-87064a44e676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:12.609888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766fd43e-b387-421e-a299-87064a44e676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:12.611796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766fd43e-b387-421e-a299-87064a44e676 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:12.629595 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2664/5264] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:06:12.988425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b618810-6892-4834-9d44-f35ccfa067c1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:12.988744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b618810-6892-4834-9d44-f35ccfa067c1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:12.989251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b618810-6892-4834-9d44-f35ccfa067c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:13.052957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8b618810-6892-4834-9d44-f35ccfa067c1 None None] MFA Rules not processed for user `646f6f5cff204d27b0476d53bd9e6fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:13.054455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8b618810-6892-4834-9d44-f35ccfa067c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:13.100532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:13.248463 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2601/5265] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:13.522464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766fd43e-b387-421e-a299-87064a44e676 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:13.523985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-766fd43e-b387-421e-a299-87064a44e676 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:13.566757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766fd43e-b387-421e-a299-87064a44e676 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:13.578630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7749ea-9313-4108-9218-61d67a361669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:13.581527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7749ea-9313-4108-9218-61d67a361669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:13.581816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7749ea-9313-4108-9218-61d67a361669 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:13.583186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7749ea-9313-4108-9218-61d67a361669 admin admin] PATH_INFO: `/v3/projects/d027a23dba31494b9d3e63be39096a7c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:13.585771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7749ea-9313-4108-9218-61d67a361669 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d027a23dba31494b9d3e63be39096a7c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:13.590953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7749ea-9313-4108-9218-61d67a361669 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:13.686790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bf7749ea-9313-4108-9218-61d67a361669 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd027a23dba31494b9d3e63be39096a7c', 'request_id': 'req-bf7749ea-9313-4108-9218-61d67a361669'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:13.723808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bf7749ea-9313-4108-9218-61d67a361669 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd027a23dba31494b9d3e63be39096a7c', 'request_id': 'req-bf7749ea-9313-4108-9218-61d67a361669'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:13.741177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b618810-6892-4834-9d44-f35ccfa067c1 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:13.742769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8b618810-6892-4834-9d44-f35ccfa067c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:13.747575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bf7749ea-9313-4108-9218-61d67a361669 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd027a23dba31494b9d3e63be39096a7c', 'request_id': 'req-bf7749ea-9313-4108-9218-61d67a361669'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:13.769278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b618810-6892-4834-9d44-f35ccfa067c1 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:13.771976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bf7749ea-9313-4108-9218-61d67a361669 admin admin] The token cache is being invalidate because project d027a23dba31494b9d3e63be39096a7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:06:13.772803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bf7749ea-9313-4108-9218-61d67a361669 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:13.774506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bf7749ea-9313-4108-9218-61d67a361669 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:13.782803 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2665/5266] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:06:13 2022] DELETE /identity/v3/projects/d027a23dba31494b9d3e63be39096a7c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:13.800652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243921fa-83d5-444b-97ef-82d72979db26 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:13.800939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243921fa-83d5-444b-97ef-82d72979db26 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:13.802867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243921fa-83d5-444b-97ef-82d72979db26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:13.838285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-243921fa-83d5-444b-97ef-82d72979db26 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:13.839975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-243921fa-83d5-444b-97ef-82d72979db26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:13.877119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e046fe-dea2-4dd6-a5ca-325b0e6a3806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:13.878000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e046fe-dea2-4dd6-a5ca-325b0e6a3806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:13.878969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e046fe-dea2-4dd6-a5ca-325b0e6a3806 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:13.879317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e046fe-dea2-4dd6-a5ca-325b0e6a3806 admin admin] PATH_INFO: `/v3/projects/76a25a013d104b17b8d7c9bd90e5a311` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:13.885039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e046fe-dea2-4dd6-a5ca-325b0e6a3806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76a25a013d104b17b8d7c9bd90e5a311)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:13.887424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e046fe-dea2-4dd6-a5ca-325b0e6a3806 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:13.953359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:13.976898 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2666/5267] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:06:13.983366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1e046fe-dea2-4dd6-a5ca-325b0e6a3806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76a25a013d104b17b8d7c9bd90e5a311', 'request_id': 'req-d1e046fe-dea2-4dd6-a5ca-325b0e6a3806'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:13.984779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43effc80-08de-44e3-ac59-62c7657a6219 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:13.985267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43effc80-08de-44e3-ac59-62c7657a6219 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:13.985546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43effc80-08de-44e3-ac59-62c7657a6219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:13.998766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1e046fe-dea2-4dd6-a5ca-325b0e6a3806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76a25a013d104b17b8d7c9bd90e5a311', 'request_id': 'req-d1e046fe-dea2-4dd6-a5ca-325b0e6a3806'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:14.010731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1e046fe-dea2-4dd6-a5ca-325b0e6a3806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76a25a013d104b17b8d7c9bd90e5a311', 'request_id': 'req-d1e046fe-dea2-4dd6-a5ca-325b0e6a3806'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:14.053644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1e046fe-dea2-4dd6-a5ca-325b0e6a3806 admin admin] The token cache is being invalidate because project 76a25a013d104b17b8d7c9bd90e5a311 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:06:14.053943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1e046fe-dea2-4dd6-a5ca-325b0e6a3806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:14.054978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1e046fe-dea2-4dd6-a5ca-325b0e6a3806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:14.060403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-43effc80-08de-44e3-ac59-62c7657a6219 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:14.060567 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2602/5268] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:06:13 2022] DELETE /identity/v3/projects/76a25a013d104b17b8d7c9bd90e5a311 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:14.061709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-43effc80-08de-44e3-ac59-62c7657a6219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:14.061911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e046fe-dea2-4dd6-a5ca-325b0e6a3806 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:14.064334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d1e046fe-dea2-4dd6-a5ca-325b0e6a3806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:14.160897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:14.168271 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2667/5269] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:06:14.171675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c53f0a-1062-46eb-9326-7bd16ef5a77f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:14.171905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c53f0a-1062-46eb-9326-7bd16ef5a77f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:14.172190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c53f0a-1062-46eb-9326-7bd16ef5a77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:14.196544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e046fe-dea2-4dd6-a5ca-325b0e6a3806 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:14.200301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-70c53f0a-1062-46eb-9326-7bd16ef5a77f None None] MFA Rules not processed for user `6d63c633112e49d091421c84cea3422b`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:14.202076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-70c53f0a-1062-46eb-9326-7bd16ef5a77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:14.203047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a286b-1284-4857-954d-b031ec495d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:14.204026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a286b-1284-4857-954d-b031ec495d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:14.205055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a286b-1284-4857-954d-b031ec495d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:14.205572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a286b-1284-4857-954d-b031ec495d3d admin admin] PATH_INFO: `/v3/users/081b5c8233904f32bac857a6ec721d07` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:14.208498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a286b-1284-4857-954d-b031ec495d3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=081b5c8233904f32bac857a6ec721d07)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:14.211605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a286b-1284-4857-954d-b031ec495d3d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:14.293245 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2668/5270] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:14.299296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c53f0a-1062-46eb-9326-7bd16ef5a77f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:14.302190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a75a286b-1284-4857-954d-b031ec495d3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '081b5c8233904f32bac857a6ec721d07'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:14.303080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a75a286b-1284-4857-954d-b031ec495d3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '081b5c8233904f32bac857a6ec721d07'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:14.303814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a75a286b-1284-4857-954d-b031ec495d3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '081b5c8233904f32bac857a6ec721d07'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:14.315011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c53f0a-1062-46eb-9326-7bd16ef5a77f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:14.320153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a75a286b-1284-4857-954d-b031ec495d3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '081b5c8233904f32bac857a6ec721d07'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:14.321797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea2f83-aa68-4f22-acf0-3faa1d10eb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:14.324000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea2f83-aa68-4f22-acf0-3faa1d10eb89 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:14.324414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea2f83-aa68-4f22-acf0-3faa1d10eb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:14.324780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea2f83-aa68-4f22-acf0-3faa1d10eb89 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:14.325823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea2f83-aa68-4f22-acf0-3faa1d10eb89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:06:14.325959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea2f83-aa68-4f22-acf0-3faa1d10eb89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:14.328616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea2f83-aa68-4f22-acf0-3faa1d10eb89 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:14.342448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a75a286b-1284-4857-954d-b031ec495d3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '081b5c8233904f32bac857a6ec721d07'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:14.358940 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2603/5271] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:06:14 2022] DELETE /identity/v3/users/081b5c8233904f32bac857a6ec721d07 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:14.366208 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2669/5272] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:06:14.376562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a286b-1284-4857-954d-b031ec495d3d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:14.378344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a75a286b-1284-4857-954d-b031ec495d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:14.555995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a286b-1284-4857-954d-b031ec495d3d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:14.566458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ffab8a-f715-4350-9127-197b2764d6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:14.569903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ffab8a-f715-4350-9127-197b2764d6a0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:14.570315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ffab8a-f715-4350-9127-197b2764d6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:14.570724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ffab8a-f715-4350-9127-197b2764d6a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:14.571484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ffab8a-f715-4350-9127-197b2764d6a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:14.577068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ffab8a-f715-4350-9127-197b2764d6a0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:14.654967 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2604/5273] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:06:14 2022] POST /identity/v3/projects => generated 377 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:14.660643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea2f83-aa68-4f22-acf0-3faa1d10eb89 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:14.675263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea2f83-aa68-4f22-acf0-3faa1d10eb89 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:14.822425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186de6d0-2422-43d0-ba76-0db516ee1aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:14.823172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186de6d0-2422-43d0-ba76-0db516ee1aea admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:14.823414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186de6d0-2422-43d0-ba76-0db516ee1aea admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:14.823643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186de6d0-2422-43d0-ba76-0db516ee1aea admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:14.824311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186de6d0-2422-43d0-ba76-0db516ee1aea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:14.826906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186de6d0-2422-43d0-ba76-0db516ee1aea admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:14.874324 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2670/5274] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:06:14 2022] POST /identity/v3/users => generated 289 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:14.875766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ffab8a-f715-4350-9127-197b2764d6a0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:14.914917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ffab8a-f715-4350-9127-197b2764d6a0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:14.915693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186de6d0-2422-43d0-ba76-0db516ee1aea admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:14.920190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-186de6d0-2422-43d0-ba76-0db516ee1aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:14.928018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee199e-a1c8-4845-b047-be661d1a24b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:14.928656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee199e-a1c8-4845-b047-be661d1a24b1 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:14.928932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee199e-a1c8-4845-b047-be661d1a24b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:14.929465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee199e-a1c8-4845-b047-be661d1a24b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:14.930118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee199e-a1c8-4845-b047-be661d1a24b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:14.933507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee199e-a1c8-4845-b047-be661d1a24b1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:14.941020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186de6d0-2422-43d0-ba76-0db516ee1aea admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:14.951190 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2605/5275] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:14.951732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee199e-a1c8-4845-b047-be661d1a24b1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:14.954050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-82ee199e-a1c8-4845-b047-be661d1a24b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:14.957536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68bd0a-e707-4228-a902-bf9a9c202572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:14.958416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68bd0a-e707-4228-a902-bf9a9c202572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:14.958942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68bd0a-e707-4228-a902-bf9a9c202572 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:14.959455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68bd0a-e707-4228-a902-bf9a9c202572 admin admin] PATH_INFO: `/v3/projects/47716815e1534059abaab16df21a4879` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:14.961199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68bd0a-e707-4228-a902-bf9a9c202572 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47716815e1534059abaab16df21a4879)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:14.965977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68bd0a-e707-4228-a902-bf9a9c202572 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:15.000596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:15.042336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1a68bd0a-e707-4228-a902-bf9a9c202572 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47716815e1534059abaab16df21a4879', 'request_id': 'req-1a68bd0a-e707-4228-a902-bf9a9c202572'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:15.048874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:15.053280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1a68bd0a-e707-4228-a902-bf9a9c202572 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47716815e1534059abaab16df21a4879', 'request_id': 'req-1a68bd0a-e707-4228-a902-bf9a9c202572'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:15.066949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1a68bd0a-e707-4228-a902-bf9a9c202572 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47716815e1534059abaab16df21a4879', 'request_id': 'req-1a68bd0a-e707-4228-a902-bf9a9c202572'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:15.101879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1a68bd0a-e707-4228-a902-bf9a9c202572 admin admin] The token cache is being invalidate because project 47716815e1534059abaab16df21a4879 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:06:15.102888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1a68bd0a-e707-4228-a902-bf9a9c202572 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:15.105086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1a68bd0a-e707-4228-a902-bf9a9c202572 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:15.109410 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2671/5276] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:06:14 2022] DELETE /identity/v3/projects/47716815e1534059abaab16df21a4879 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:15.114538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68bd0a-e707-4228-a902-bf9a9c202572 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:15.120144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1a68bd0a-e707-4228-a902-bf9a9c202572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:15.126186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee199e-a1c8-4845-b047-be661d1a24b1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:15.253515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68bd0a-e707-4228-a902-bf9a9c202572 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:15.259549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401176f3-0ad0-4ae4-bee0-e28611507ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:15.260281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401176f3-0ad0-4ae4-bee0-e28611507ece admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:15.260625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401176f3-0ad0-4ae4-bee0-e28611507ece admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:15.260907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401176f3-0ad0-4ae4-bee0-e28611507ece admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:15.261560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401176f3-0ad0-4ae4-bee0-e28611507ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:15.263265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401176f3-0ad0-4ae4-bee0-e28611507ece admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:15.283742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48405f2f-0f78-439a-a8ec-d137f1ec9335 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:15.283956 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2672/5277] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:15.284039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48405f2f-0f78-439a-a8ec-d137f1ec9335 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:15.284039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48405f2f-0f78-439a-a8ec-d137f1ec9335 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:15.284039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48405f2f-0f78-439a-a8ec-d137f1ec9335 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:15.284262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278db017-bf31-403d-ac58-5d8c1e07470c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:15.284262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278db017-bf31-403d-ac58-5d8c1e07470c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:15.284262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278db017-bf31-403d-ac58-5d8c1e07470c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:15.286828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-48405f2f-0f78-439a-a8ec-d137f1ec9335 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:15.332880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-278db017-bf31-403d-ac58-5d8c1e07470c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:15.334139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-278db017-bf31-403d-ac58-5d8c1e07470c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:15.348681 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2673/5278] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:15.350725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278db017-bf31-403d-ac58-5d8c1e07470c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:15.370519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278db017-bf31-403d-ac58-5d8c1e07470c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:15.390474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bebce6a-daa0-4a1e-ab2f-c03489942ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:15.393257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebce6a-daa0-4a1e-ab2f-c03489942ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:15.394329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebce6a-daa0-4a1e-ab2f-c03489942ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:15.394953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebce6a-daa0-4a1e-ab2f-c03489942ce7 admin admin] PATH_INFO: `/v3/projects/0b5bb0c08a5145369fc64e13d190c988/users/f74f251052ee4c9a99db0c43bc00da9f/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:15.416408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bebce6a-daa0-4a1e-ab2f-c03489942ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b5bb0c08a5145369fc64e13d190c988, user_id=f74f251052ee4c9a99db0c43bc00da9f, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:15.423421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bebce6a-daa0-4a1e-ab2f-c03489942ce7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:15.427968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48405f2f-0f78-439a-a8ec-d137f1ec9335 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:15.435415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48405f2f-0f78-439a-a8ec-d137f1ec9335 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:15.449722 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2674/5279] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:06:15 2022] PUT /identity/v3/projects/0b5bb0c08a5145369fc64e13d190c988/users/f74f251052ee4c9a99db0c43bc00da9f/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:15.453172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4194ad-b927-438b-8efe-9417d69478ff None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:15.453627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4194ad-b927-438b-8efe-9417d69478ff None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:15.453881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4194ad-b927-438b-8efe-9417d69478ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:15.463530 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2606/5280] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:06:15.475841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3e168-c61e-4005-9327-6e188af6f92d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:15.476128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3e168-c61e-4005-9327-6e188af6f92d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:15.476402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3e168-c61e-4005-9327-6e188af6f92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:15.481311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8e4194ad-b927-438b-8efe-9417d69478ff None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:15.482600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8e4194ad-b927-438b-8efe-9417d69478ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:15.542511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c5e3e168-c61e-4005-9327-6e188af6f92d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:15.544150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c5e3e168-c61e-4005-9327-6e188af6f92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:15.637216 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2675/5281] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:15.637579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4194ad-b927-438b-8efe-9417d69478ff None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:15.664302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4194ad-b927-438b-8efe-9417d69478ff None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:15.677912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f723c9d-8dcc-435c-9925-26c1e1b8f3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:15.678167 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2607/5282] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:15.682340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f723c9d-8dcc-435c-9925-26c1e1b8f3d1 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:15.682689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f723c9d-8dcc-435c-9925-26c1e1b8f3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:15.682980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f723c9d-8dcc-435c-9925-26c1e1b8f3d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:15.683797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f723c9d-8dcc-435c-9925-26c1e1b8f3d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:06:15.684107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f723c9d-8dcc-435c-9925-26c1e1b8f3d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:15.686431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f723c9d-8dcc-435c-9925-26c1e1b8f3d1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:15.691544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:15.708458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3e168-c61e-4005-9327-6e188af6f92d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:15.710315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c5e3e168-c61e-4005-9327-6e188af6f92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:15.711933 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2676/5283] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:06:15.712805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f723c9d-8dcc-435c-9925-26c1e1b8f3d1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:15.732506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f723c9d-8dcc-435c-9925-26c1e1b8f3d1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:15.738279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e596626-89f7-4f62-8841-5211a0e50e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:15.738932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e596626-89f7-4f62-8841-5211a0e50e53 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:15.739275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e596626-89f7-4f62-8841-5211a0e50e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:15.739612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e596626-89f7-4f62-8841-5211a0e50e53 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:15.740324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e596626-89f7-4f62-8841-5211a0e50e53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:15.742467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e596626-89f7-4f62-8841-5211a0e50e53 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:15.751498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:15.795865 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2677/5284] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:06:15 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:15.800568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e596626-89f7-4f62-8841-5211a0e50e53 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:15.903659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3e168-c61e-4005-9327-6e188af6f92d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:15.914201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e596626-89f7-4f62-8841-5211a0e50e53 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:15.927564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ea84b-34e5-4359-9d17-c16210507b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:15.930067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ea84b-34e5-4359-9d17-c16210507b2e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:15.930348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ea84b-34e5-4359-9d17-c16210507b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:15.930648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ea84b-34e5-4359-9d17-c16210507b2e admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:15.931289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ea84b-34e5-4359-9d17-c16210507b2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:15.937474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ea84b-34e5-4359-9d17-c16210507b2e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:15.991451 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2678/5285] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:06:15 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:16.000748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ea84b-34e5-4359-9d17-c16210507b2e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:16.055053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ea84b-34e5-4359-9d17-c16210507b2e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:16.069429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f96439-018b-4a77-afe8-a37a4227616f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:16.070057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f96439-018b-4a77-afe8-a37a4227616f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:16.070558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f96439-018b-4a77-afe8-a37a4227616f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:16.070672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f96439-018b-4a77-afe8-a37a4227616f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:16.071286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f96439-018b-4a77-afe8-a37a4227616f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:16.073249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f96439-018b-4a77-afe8-a37a4227616f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:16.078582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b46543d-a77f-4d11-bdb8-7481436d4274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:16.079210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46543d-a77f-4d11-bdb8-7481436d4274 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:16.079472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46543d-a77f-4d11-bdb8-7481436d4274 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:16.080273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46543d-a77f-4d11-bdb8-7481436d4274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:16.109904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b46543d-a77f-4d11-bdb8-7481436d4274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:16.110607 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2679/5286] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:16.112038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f96439-018b-4a77-afe8-a37a4227616f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:16.113692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b46543d-a77f-4d11-bdb8-7481436d4274 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:16.141381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f96439-018b-4a77-afe8-a37a4227616f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:16.143197 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2608/5287] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:06:16.149413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e39a22-4ba3-49ce-9a6f-3970d10e9c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:16.149946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e39a22-4ba3-49ce-9a6f-3970d10e9c14 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:16.150299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e39a22-4ba3-49ce-9a6f-3970d10e9c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:16.150833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e39a22-4ba3-49ce-9a6f-3970d10e9c14 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:16.151491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e39a22-4ba3-49ce-9a6f-3970d10e9c14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:16.158806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e39a22-4ba3-49ce-9a6f-3970d10e9c14 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:16.172697 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2680/5288] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:16.177566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b46543d-a77f-4d11-bdb8-7481436d4274 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:16.190816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b46543d-a77f-4d11-bdb8-7481436d4274 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:16.206785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5a036-259a-40d5-a96f-76a6ce5b425e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:16.209047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5a036-259a-40d5-a96f-76a6ce5b425e admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:16.210016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5a036-259a-40d5-a96f-76a6ce5b425e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:16.210373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5a036-259a-40d5-a96f-76a6ce5b425e admin admin] PATH_INFO: `/v3/projects/27bf2fb5f6a347fea735471e5f00468a/users/57ef04dfc97d4d64999c2a399e5b7aa5/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:16.233130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5a036-259a-40d5-a96f-76a6ce5b425e admin admin] RBAC: Authorizing `identity:create_grant(project_id=27bf2fb5f6a347fea735471e5f00468a, user_id=57ef04dfc97d4d64999c2a399e5b7aa5, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:16.237547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5a036-259a-40d5-a96f-76a6ce5b425e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:16.273600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada6dc0-4fb6-4de2-a058-665252aae637 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:16.274049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada6dc0-4fb6-4de2-a058-665252aae637 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:16.274583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada6dc0-4fb6-4de2-a058-665252aae637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:16.281018 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2609/5289] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:06:16 2022] PUT /identity/v3/projects/27bf2fb5f6a347fea735471e5f00468a/users/57ef04dfc97d4d64999c2a399e5b7aa5/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:06:16.316611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0ada6dc0-4fb6-4de2-a058-665252aae637 None None] MFA Rules not processed for user `57ef04dfc97d4d64999c2a399e5b7aa5`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:16.318368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0ada6dc0-4fb6-4de2-a058-665252aae637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:16.445552 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2681/5290] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:16.460968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5a036-259a-40d5-a96f-76a6ce5b425e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:16.467714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e0f5a036-259a-40d5-a96f-76a6ce5b425e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:16.597180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5a036-259a-40d5-a96f-76a6ce5b425e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:16.602859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36d308-5c7c-4ce4-8e94-0f4642673358 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:16.603620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36d308-5c7c-4ce4-8e94-0f4642673358 service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:16.603983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36d308-5c7c-4ce4-8e94-0f4642673358 service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:16.604492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36d308-5c7c-4ce4-8e94-0f4642673358 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:16.617095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f36d308-5c7c-4ce4-8e94-0f4642673358 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:16.619866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f36d308-5c7c-4ce4-8e94-0f4642673358 service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:16.632739 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2610/5291] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:06:16.921973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613687ab-4d93-4db8-a82f-91aae750f3d9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:16.922457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613687ab-4d93-4db8-a82f-91aae750f3d9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:16.922583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613687ab-4d93-4db8-a82f-91aae750f3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:16.958748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-613687ab-4d93-4db8-a82f-91aae750f3d9 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:16.962226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-613687ab-4d93-4db8-a82f-91aae750f3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:17.046580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:17.165090 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2682/5292] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:06:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:17.249984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36d308-5c7c-4ce4-8e94-0f4642673358 service glance] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:17.252224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2f36d308-5c7c-4ce4-8e94-0f4642673358 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:17.345661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:17.398026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36d308-5c7c-4ce4-8e94-0f4642673358 service glance] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:17.404731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec06a6-23a1-420a-bf7c-8436e80a48c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:17.405760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec06a6-23a1-420a-bf7c-8436e80a48c8 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:17.406225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec06a6-23a1-420a-bf7c-8436e80a48c8 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:17.406710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec06a6-23a1-420a-bf7c-8436e80a48c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:17.417382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec06a6-23a1-420a-bf7c-8436e80a48c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:17.420260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec06a6-23a1-420a-bf7c-8436e80a48c8 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:17.458314 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2611/5293] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:06:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:06:17.792657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e987608-8a42-4f96-9134-293f7ba89bfb None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:17.792945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e987608-8a42-4f96-9134-293f7ba89bfb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:17.793219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e987608-8a42-4f96-9134-293f7ba89bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:17.838399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2e987608-8a42-4f96-9134-293f7ba89bfb None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:17.839706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2e987608-8a42-4f96-9134-293f7ba89bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:17.848824 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2683/5294] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:06:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:17.865799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec06a6-23a1-420a-bf7c-8436e80a48c8 service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:17.898536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec06a6-23a1-420a-bf7c-8436e80a48c8 service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:17.907365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ef844-d384-42f6-b43f-492d31e54279 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:17.908162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ef844-d384-42f6-b43f-492d31e54279 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:17.908510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ef844-d384-42f6-b43f-492d31e54279 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:17.908904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ef844-d384-42f6-b43f-492d31e54279 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:17.928856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ef844-d384-42f6-b43f-492d31e54279 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:17.933911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ef844-d384-42f6-b43f-492d31e54279 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:17.963325 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2612/5295] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:06:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:06:18.425319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522298f-b6cc-4c7d-91fc-70d16bec5e5a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:18.426541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522298f-b6cc-4c7d-91fc-70d16bec5e5a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:18.426960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522298f-b6cc-4c7d-91fc-70d16bec5e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:18.468196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a522298f-b6cc-4c7d-91fc-70d16bec5e5a None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:18.472747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a522298f-b6cc-4c7d-91fc-70d16bec5e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:18.485534 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2684/5296] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:06:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:06:18.504111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ef844-d384-42f6-b43f-492d31e54279 service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:18.519727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ef844-d384-42f6-b43f-492d31e54279 service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:18.526708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0cc4f-9b0d-4156-bc8f-eb2e757d950e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:18.527682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0cc4f-9b0d-4156-bc8f-eb2e757d950e service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:18.528153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0cc4f-9b0d-4156-bc8f-eb2e757d950e service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:18.528884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0cc4f-9b0d-4156-bc8f-eb2e757d950e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:18.542876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0cc4f-9b0d-4156-bc8f-eb2e757d950e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:18.542876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0cc4f-9b0d-4156-bc8f-eb2e757d950e service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:18.553234 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2613/5297] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:06:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 21:06:19.025863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57654e0f-ee9f-436d-8cb8-01f59268afec None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:19.026155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57654e0f-ee9f-436d-8cb8-01f59268afec None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:19.026527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57654e0f-ee9f-436d-8cb8-01f59268afec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:19.077137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-57654e0f-ee9f-436d-8cb8-01f59268afec None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:19.078398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-57654e0f-ee9f-436d-8cb8-01f59268afec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:19.114988 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2685/5298] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:06:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:19.115740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0cc4f-9b0d-4156-bc8f-eb2e757d950e service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:19.133022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0cc4f-9b0d-4156-bc8f-eb2e757d950e service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:19.153648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a81cd2-70f7-4fcd-9002-72cb66714f26 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:19.154287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a81cd2-70f7-4fcd-9002-72cb66714f26 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:19.154592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a81cd2-70f7-4fcd-9002-72cb66714f26 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:19.154867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a81cd2-70f7-4fcd-9002-72cb66714f26 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:19.181516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a81cd2-70f7-4fcd-9002-72cb66714f26 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:19.189296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a81cd2-70f7-4fcd-9002-72cb66714f26 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:19.211116 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2614/5299] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:06:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:06:19.693480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4e40c-f94b-440c-a67b-dff1fb4fa3c9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:19.694314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4e40c-f94b-440c-a67b-dff1fb4fa3c9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:19.694789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4e40c-f94b-440c-a67b-dff1fb4fa3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:19.733638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f3f4e40c-f94b-440c-a67b-dff1fb4fa3c9 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:19.735624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f3f4e40c-f94b-440c-a67b-dff1fb4fa3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:19.744002 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2686/5300] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:06:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:19.774836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a81cd2-70f7-4fcd-9002-72cb66714f26 service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:19.794359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a81cd2-70f7-4fcd-9002-72cb66714f26 service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:19.801344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303a1bd-72c3-4ddb-a61d-404c83c833ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:19.802362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303a1bd-72c3-4ddb-a61d-404c83c833ab service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:19.802894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303a1bd-72c3-4ddb-a61d-404c83c833ab service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:19.803379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303a1bd-72c3-4ddb-a61d-404c83c833ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:19.815066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303a1bd-72c3-4ddb-a61d-404c83c833ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:19.817196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303a1bd-72c3-4ddb-a61d-404c83c833ab service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:19.833471 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2615/5301] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:06:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 21:06:20.245466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060145c7-1393-4b03-a331-d7321f18b3ae None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:20.246460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060145c7-1393-4b03-a331-d7321f18b3ae None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:20.247126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060145c7-1393-4b03-a331-d7321f18b3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:20.277364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-060145c7-1393-4b03-a331-d7321f18b3ae None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:20.279664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-060145c7-1393-4b03-a331-d7321f18b3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:20.289454 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2687/5302] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:06:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:20.306906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303a1bd-72c3-4ddb-a61d-404c83c833ab service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:20.323736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303a1bd-72c3-4ddb-a61d-404c83c833ab service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:20.333131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba90ec-6b48-4587-bb5e-1a5e1a5dbf07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:20.334017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba90ec-6b48-4587-bb5e-1a5e1a5dbf07 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:20.334701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba90ec-6b48-4587-bb5e-1a5e1a5dbf07 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:20.335173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba90ec-6b48-4587-bb5e-1a5e1a5dbf07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:20.349399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ba90ec-6b48-4587-bb5e-1a5e1a5dbf07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:20.352969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ba90ec-6b48-4587-bb5e-1a5e1a5dbf07 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:20.365488 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2616/5303] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:06:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:06:21.860787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3806e3-b17b-455e-8c4a-5c0590e070ba None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:21.861883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3806e3-b17b-455e-8c4a-5c0590e070ba None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:21.862307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3806e3-b17b-455e-8c4a-5c0590e070ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:21.893326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ab3806e3-b17b-455e-8c4a-5c0590e070ba None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:21.895413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ab3806e3-b17b-455e-8c4a-5c0590e070ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:21.915128 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2688/5304] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:06:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:06:21.936937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba90ec-6b48-4587-bb5e-1a5e1a5dbf07 service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:21.952542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba90ec-6b48-4587-bb5e-1a5e1a5dbf07 service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:21.961095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf44ff2-8d2a-4b8d-bab4-63425a02d0a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:21.962113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf44ff2-8d2a-4b8d-bab4-63425a02d0a2 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:21.962688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf44ff2-8d2a-4b8d-bab4-63425a02d0a2 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:21.963197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf44ff2-8d2a-4b8d-bab4-63425a02d0a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:21.974882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf44ff2-8d2a-4b8d-bab4-63425a02d0a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:21.976480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf44ff2-8d2a-4b8d-bab4-63425a02d0a2 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:21.991716 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2617/5305] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:06:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:06:22.232333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248913e4-f338-4bdd-a320-76b8bd3ddf37 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:22.233173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248913e4-f338-4bdd-a320-76b8bd3ddf37 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:22.233822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248913e4-f338-4bdd-a320-76b8bd3ddf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:22.244241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9b0fa-a08e-4630-9e3d-388523d0e1f4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:22.244534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9b0fa-a08e-4630-9e3d-388523d0e1f4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:22.244815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9b0fa-a08e-4630-9e3d-388523d0e1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:22.259552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-248913e4-f338-4bdd-a320-76b8bd3ddf37 None None] MFA Rules not processed for user `6d63c633112e49d091421c84cea3422b`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:22.261421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-248913e4-f338-4bdd-a320-76b8bd3ddf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:22.284706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fcd9b0fa-a08e-4630-9e3d-388523d0e1f4 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:22.286140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fcd9b0fa-a08e-4630-9e3d-388523d0e1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:22.301853 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2618/5306] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:06:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:06:22.317899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9b0fa-a08e-4630-9e3d-388523d0e1f4 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:22.338745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9b0fa-a08e-4630-9e3d-388523d0e1f4 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:22.345468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c279f-5439-48b7-8c24-29ce79f23660 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:22.347097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c279f-5439-48b7-8c24-29ce79f23660 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:22.347736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c279f-5439-48b7-8c24-29ce79f23660 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:22.348312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c279f-5439-48b7-8c24-29ce79f23660 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:22.351699 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2689/5307] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:22.360311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c279f-5439-48b7-8c24-29ce79f23660 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:22.362163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c279f-5439-48b7-8c24-29ce79f23660 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:22.370937 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2619/5308] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:06:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 21:06:22.648136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72521f1-b1aa-4b92-9238-56af9276a365 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:22.648433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72521f1-b1aa-4b92-9238-56af9276a365 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:22.648714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72521f1-b1aa-4b92-9238-56af9276a365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:22.672771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f72521f1-b1aa-4b92-9238-56af9276a365 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:22.674000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f72521f1-b1aa-4b92-9238-56af9276a365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:22.682951 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2690/5309] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:06:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:22.698473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c279f-5439-48b7-8c24-29ce79f23660 service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:22.717498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c279f-5439-48b7-8c24-29ce79f23660 service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:22.735377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5f255-eef5-4c0a-a130-a28abab5e9c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:22.736519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5f255-eef5-4c0a-a130-a28abab5e9c4 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:22.736948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5f255-eef5-4c0a-a130-a28abab5e9c4 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:22.737804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5f255-eef5-4c0a-a130-a28abab5e9c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:22.756280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5f255-eef5-4c0a-a130-a28abab5e9c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:22.758447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5f255-eef5-4c0a-a130-a28abab5e9c4 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:22.774329 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2620/5310] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:06:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:06:23.128952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d63fb8-4e38-4f5d-a661-fffa431bca9c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:23.129388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d63fb8-4e38-4f5d-a661-fffa431bca9c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:23.129478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d63fb8-4e38-4f5d-a661-fffa431bca9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:23.158962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-10d63fb8-4e38-4f5d-a661-fffa431bca9c None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:23.160259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-10d63fb8-4e38-4f5d-a661-fffa431bca9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:23.174159 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2691/5311] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:06:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:23.193241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5f255-eef5-4c0a-a130-a28abab5e9c4 service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:23.223839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5f255-eef5-4c0a-a130-a28abab5e9c4 service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:23.242900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0f267-cfb7-4671-b6cf-662d10367796 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:23.243880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0f267-cfb7-4671-b6cf-662d10367796 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:23.244377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0f267-cfb7-4671-b6cf-662d10367796 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:23.245046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0f267-cfb7-4671-b6cf-662d10367796 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:23.265516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f0f267-cfb7-4671-b6cf-662d10367796 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:23.271231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f0f267-cfb7-4671-b6cf-662d10367796 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:23.291996 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2621/5312] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:06:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:06:23.556665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d999295-abf4-448d-b5f9-f90536d015ea None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:23.557189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d999295-abf4-448d-b5f9-f90536d015ea None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:23.557819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d999295-abf4-448d-b5f9-f90536d015ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:23.623971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4d999295-abf4-448d-b5f9-f90536d015ea None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:23.624758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4d999295-abf4-448d-b5f9-f90536d015ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:23.651559 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2692/5313] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:06:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:23.674042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0f267-cfb7-4671-b6cf-662d10367796 service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:23.692864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0f267-cfb7-4671-b6cf-662d10367796 service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:23.702584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0c21d-dd3b-42cb-8447-ad10fb3ff4bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:23.704294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0c21d-dd3b-42cb-8447-ad10fb3ff4bb service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:23.704795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0c21d-dd3b-42cb-8447-ad10fb3ff4bb service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:23.705999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0c21d-dd3b-42cb-8447-ad10fb3ff4bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:23.718446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0c21d-dd3b-42cb-8447-ad10fb3ff4bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:23.720854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0c21d-dd3b-42cb-8447-ad10fb3ff4bb service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:23.798733 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2622/5314] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:06:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:06:24.039251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901e26e-0dbc-4907-922d-44310f3a9e93 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:24.039544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901e26e-0dbc-4907-922d-44310f3a9e93 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:24.040171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901e26e-0dbc-4907-922d-44310f3a9e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:24.076854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c901e26e-0dbc-4907-922d-44310f3a9e93 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:24.078535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c901e26e-0dbc-4907-922d-44310f3a9e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:24.087925 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2693/5315] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:06:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:24.107997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0c21d-dd3b-42cb-8447-ad10fb3ff4bb service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:24.125748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0c21d-dd3b-42cb-8447-ad10fb3ff4bb service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:24.140362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7007d61-3f61-4cd0-9275-0672a0187bc8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:24.140997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7007d61-3f61-4cd0-9275-0672a0187bc8 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:24.141274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7007d61-3f61-4cd0-9275-0672a0187bc8 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:24.158031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7007d61-3f61-4cd0-9275-0672a0187bc8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:24.180425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7007d61-3f61-4cd0-9275-0672a0187bc8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:24.182434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7007d61-3f61-4cd0-9275-0672a0187bc8 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:24.197398 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2623/5316] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:06:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 21:06:24.382453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901e26e-0dbc-4907-922d-44310f3a9e93 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:24.519978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901e26e-0dbc-4907-922d-44310f3a9e93 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:24.532085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7bcdc-d82a-44eb-8c82-c4a31f1261d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:24.533114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7bcdc-d82a-44eb-8c82-c4a31f1261d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:24.534873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7bcdc-d82a-44eb-8c82-c4a31f1261d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:24.535321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7bcdc-d82a-44eb-8c82-c4a31f1261d5 admin admin] PATH_INFO: `/v3/users/57ef04dfc97d4d64999c2a399e5b7aa5` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:24.540796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7bcdc-d82a-44eb-8c82-c4a31f1261d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57ef04dfc97d4d64999c2a399e5b7aa5)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:24.543180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7bcdc-d82a-44eb-8c82-c4a31f1261d5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:24.561705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:24.635853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-53f7bcdc-d82a-44eb-8c82-c4a31f1261d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57ef04dfc97d4d64999c2a399e5b7aa5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:24.659413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-53f7bcdc-d82a-44eb-8c82-c4a31f1261d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57ef04dfc97d4d64999c2a399e5b7aa5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:24.680201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-53f7bcdc-d82a-44eb-8c82-c4a31f1261d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57ef04dfc97d4d64999c2a399e5b7aa5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:24.682518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-53f7bcdc-d82a-44eb-8c82-c4a31f1261d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57ef04dfc97d4d64999c2a399e5b7aa5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:24.713109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-53f7bcdc-d82a-44eb-8c82-c4a31f1261d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57ef04dfc97d4d64999c2a399e5b7aa5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:24.726530 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2694/5317] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:06:24 2022] DELETE /identity/v3/users/57ef04dfc97d4d64999c2a399e5b7aa5 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:24.728730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68856d7-8705-4c20-aa64-c97cd7bdd047 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:24.729220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68856d7-8705-4c20-aa64-c97cd7bdd047 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:24.729698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68856d7-8705-4c20-aa64-c97cd7bdd047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:24.772816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f68856d7-8705-4c20-aa64-c97cd7bdd047 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:24.776766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f68856d7-8705-4c20-aa64-c97cd7bdd047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:24.883203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7bcdc-d82a-44eb-8c82-c4a31f1261d5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:24.884904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-53f7bcdc-d82a-44eb-8c82-c4a31f1261d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:24.897688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:24.947865 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2624/5318] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:06:24.974240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68856d7-8705-4c20-aa64-c97cd7bdd047 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:24.976464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f68856d7-8705-4c20-aa64-c97cd7bdd047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:25.067658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7bcdc-d82a-44eb-8c82-c4a31f1261d5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:25.092012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421964c-f08b-4a7b-a72e-41339cb9b1ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:25.092824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421964c-f08b-4a7b-a72e-41339cb9b1ee service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:25.092935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421964c-f08b-4a7b-a72e-41339cb9b1ee service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:25.093225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421964c-f08b-4a7b-a72e-41339cb9b1ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:25.097964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1421964c-f08b-4a7b-a72e-41339cb9b1ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:25.141376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68856d7-8705-4c20-aa64-c97cd7bdd047 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:25.153157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6bf91-6cf9-4d8d-a826-5ba657bcc89d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:25.156632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6bf91-6cf9-4d8d-a826-5ba657bcc89d service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:25.157173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6bf91-6cf9-4d8d-a826-5ba657bcc89d service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:25.157695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6bf91-6cf9-4d8d-a826-5ba657bcc89d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:25.188809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6bf91-6cf9-4d8d-a826-5ba657bcc89d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:25.190667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6bf91-6cf9-4d8d-a826-5ba657bcc89d service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:25.225466 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2625/5319] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:06:25.250099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421964c-f08b-4a7b-a72e-41339cb9b1ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:25.253842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421964c-f08b-4a7b-a72e-41339cb9b1ee service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:25.297743 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2695/5320] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:06:26.322262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6bf91-6cf9-4d8d-a826-5ba657bcc89d service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:26.325173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1de6bf91-6cf9-4d8d-a826-5ba657bcc89d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:26.355446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6bf91-6cf9-4d8d-a826-5ba657bcc89d service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:26.365048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11cd5c3-5cc5-49b4-9fe1-6f89b4f0c846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:26.366651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cd5c3-5cc5-49b4-9fe1-6f89b4f0c846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:26.367115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cd5c3-5cc5-49b4-9fe1-6f89b4f0c846 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:26.367590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cd5c3-5cc5-49b4-9fe1-6f89b4f0c846 admin admin] PATH_INFO: `/v3/projects/27bf2fb5f6a347fea735471e5f00468a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:26.370256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11cd5c3-5cc5-49b4-9fe1-6f89b4f0c846 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27bf2fb5f6a347fea735471e5f00468a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:26.379143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11cd5c3-5cc5-49b4-9fe1-6f89b4f0c846 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:26.557007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c11cd5c3-5cc5-49b4-9fe1-6f89b4f0c846 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27bf2fb5f6a347fea735471e5f00468a', 'request_id': 'req-c11cd5c3-5cc5-49b4-9fe1-6f89b4f0c846'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:26.591594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c11cd5c3-5cc5-49b4-9fe1-6f89b4f0c846 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27bf2fb5f6a347fea735471e5f00468a', 'request_id': 'req-c11cd5c3-5cc5-49b4-9fe1-6f89b4f0c846'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:26.598657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c11cd5c3-5cc5-49b4-9fe1-6f89b4f0c846 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27bf2fb5f6a347fea735471e5f00468a', 'request_id': 'req-c11cd5c3-5cc5-49b4-9fe1-6f89b4f0c846'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:26.630367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c11cd5c3-5cc5-49b4-9fe1-6f89b4f0c846 admin admin] The token cache is being invalidate because project 27bf2fb5f6a347fea735471e5f00468a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:06:26.630709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c11cd5c3-5cc5-49b4-9fe1-6f89b4f0c846 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:26.633969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c11cd5c3-5cc5-49b4-9fe1-6f89b4f0c846 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:26.639007 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2626/5321] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:06:26 2022] DELETE /identity/v3/projects/27bf2fb5f6a347fea735471e5f00468a => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:26.644472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84b448-edd0-4433-8a1a-eb85e1f34536 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:26.645288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84b448-edd0-4433-8a1a-eb85e1f34536 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:26.645873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84b448-edd0-4433-8a1a-eb85e1f34536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:26.692294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-de84b448-edd0-4433-8a1a-eb85e1f34536 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:26.693539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-de84b448-edd0-4433-8a1a-eb85e1f34536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:26.726929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:26.854230 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2696/5322] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:26.855164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84fb86-eced-4154-8153-d9d252880096 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:26.855637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84fb86-eced-4154-8153-d9d252880096 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:26.856617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84fb86-eced-4154-8153-d9d252880096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:26.894917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0a84fb86-eced-4154-8153-d9d252880096 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:26.896092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0a84fb86-eced-4154-8153-d9d252880096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:26.912395 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2627/5323] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:06:26.917445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84b448-edd0-4433-8a1a-eb85e1f34536 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:26.931140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84b448-edd0-4433-8a1a-eb85e1f34536 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:26.936880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dbe31-65c7-4a90-b009-f2ec1a7c6ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:26.938316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dbe31-65c7-4a90-b009-f2ec1a7c6ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:26.938560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dbe31-65c7-4a90-b009-f2ec1a7c6ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:26.939402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dbe31-65c7-4a90-b009-f2ec1a7c6ec8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:26.940261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1dbe31-65c7-4a90-b009-f2ec1a7c6ec8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:06:26.940758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1dbe31-65c7-4a90-b009-f2ec1a7c6ec8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:26.951574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1dbe31-65c7-4a90-b009-f2ec1a7c6ec8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:26.970267 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2697/5324] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:06:26.975705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84fb86-eced-4154-8153-d9d252880096 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:26.988829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84fb86-eced-4154-8153-d9d252880096 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:26.993872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed9b1e-c4aa-40bd-bf60-81dbc675a26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:26.994533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed9b1e-c4aa-40bd-bf60-81dbc675a26a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:26.994834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed9b1e-c4aa-40bd-bf60-81dbc675a26a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:26.995110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed9b1e-c4aa-40bd-bf60-81dbc675a26a admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:26.995751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed9b1e-c4aa-40bd-bf60-81dbc675a26a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:27.003538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed9b1e-c4aa-40bd-bf60-81dbc675a26a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:27.081488 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2628/5325] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:06:26 2022] POST /identity/v3/projects => generated 391 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:27.085322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dbe31-65c7-4a90-b009-f2ec1a7c6ec8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:27.097702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dbe31-65c7-4a90-b009-f2ec1a7c6ec8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:27.226793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa3ba5-b6cb-430d-a9c3-69417e3a3a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:27.229551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa3ba5-b6cb-430d-a9c3-69417e3a3a14 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:27.230574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa3ba5-b6cb-430d-a9c3-69417e3a3a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:27.232020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa3ba5-b6cb-430d-a9c3-69417e3a3a14 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:27.232838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa3ba5-b6cb-430d-a9c3-69417e3a3a14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:27.235951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa3ba5-b6cb-430d-a9c3-69417e3a3a14 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:27.291965 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2698/5326] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:06:27 2022] POST /identity/v3/users => generated 296 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:27.294955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed9b1e-c4aa-40bd-bf60-81dbc675a26a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:27.310852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed9b1e-c4aa-40bd-bf60-81dbc675a26a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:27.331488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df27ce6-d659-4382-a087-38b8c4399783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:27.332239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df27ce6-d659-4382-a087-38b8c4399783 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:27.332628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df27ce6-d659-4382-a087-38b8c4399783 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:27.333007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df27ce6-d659-4382-a087-38b8c4399783 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:27.333765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df27ce6-d659-4382-a087-38b8c4399783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:27.335727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df27ce6-d659-4382-a087-38b8c4399783 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:27.359700 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2629/5327] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:27.364520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa3ba5-b6cb-430d-a9c3-69417e3a3a14 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:27.384163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa3ba5-b6cb-430d-a9c3-69417e3a3a14 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:27.402453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df20b9-b411-4a18-a629-dd35a7c2aaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:27.403957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df20b9-b411-4a18-a629-dd35a7c2aaa5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:27.404301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df20b9-b411-4a18-a629-dd35a7c2aaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:27.405267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df20b9-b411-4a18-a629-dd35a7c2aaa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:27.406467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df20b9-b411-4a18-a629-dd35a7c2aaa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:27.413177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df20b9-b411-4a18-a629-dd35a7c2aaa5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:27.469159 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2699/5328] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:27.474922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df27ce6-d659-4382-a087-38b8c4399783 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:27.501060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df27ce6-d659-4382-a087-38b8c4399783 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:27.510367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171b2c3-2736-434c-9a10-32b0768ebdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:27.511375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171b2c3-2736-434c-9a10-32b0768ebdf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:27.512667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171b2c3-2736-434c-9a10-32b0768ebdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:27.514484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171b2c3-2736-434c-9a10-32b0768ebdf3 admin admin] PATH_INFO: `/v3/projects/d75d4687546f4aa6ae14e09183198f00/users/308faa166f634627958d6d990839f133/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:27.559349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171b2c3-2736-434c-9a10-32b0768ebdf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75d4687546f4aa6ae14e09183198f00, user_id=308faa166f634627958d6d990839f133, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:27.563093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171b2c3-2736-434c-9a10-32b0768ebdf3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:27.602349 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2630/5329] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:06:27 2022] PUT /identity/v3/projects/d75d4687546f4aa6ae14e09183198f00/users/308faa166f634627958d6d990839f133/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:27.616026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933593c8-158c-4259-b3a8-b1353468b10e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:27.616714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933593c8-158c-4259-b3a8-b1353468b10e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:27.617226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933593c8-158c-4259-b3a8-b1353468b10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:27.665802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83bb75-5323-4a51-9b11-0b3d45615885 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:27.666213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83bb75-5323-4a51-9b11-0b3d45615885 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:27.666767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83bb75-5323-4a51-9b11-0b3d45615885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:27.695013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-933593c8-158c-4259-b3a8-b1353468b10e None None] MFA Rules not processed for user `308faa166f634627958d6d990839f133`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:27.696246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-933593c8-158c-4259-b3a8-b1353468b10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:27.713874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3f83bb75-5323-4a51-9b11-0b3d45615885 None None] MFA Rules not processed for user `6d63c633112e49d091421c84cea3422b`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:27.718270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3f83bb75-5323-4a51-9b11-0b3d45615885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:27.731532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:27.833245 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2700/5330] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:27.837649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933593c8-158c-4259-b3a8-b1353468b10e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:27.857889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:27.872311 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2631/5331] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:06:28.158805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933593c8-158c-4259-b3a8-b1353468b10e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:28.169820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c11a75-8166-4931-8f4b-0d96a00b7cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:28.171154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c11a75-8166-4931-8f4b-0d96a00b7cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:28.172117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c11a75-8166-4931-8f4b-0d96a00b7cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:28.172639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c11a75-8166-4931-8f4b-0d96a00b7cf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:28.175158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c11a75-8166-4931-8f4b-0d96a00b7cf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:28.179325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c11a75-8166-4931-8f4b-0d96a00b7cf7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:28.241689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83bb75-5323-4a51-9b11-0b3d45615885 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:28.258063 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2701/5332] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:06:27 2022] POST /identity/v3/projects => generated 389 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:28.271868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83bb75-5323-4a51-9b11-0b3d45615885 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:28.399483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdf279-e484-4905-94a8-cd009045476b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:28.400932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdf279-e484-4905-94a8-cd009045476b admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:28.401285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdf279-e484-4905-94a8-cd009045476b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:28.402089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdf279-e484-4905-94a8-cd009045476b admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:28.403563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecdf279-e484-4905-94a8-cd009045476b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:28.405795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecdf279-e484-4905-94a8-cd009045476b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:28.451933 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2632/5333] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:06:28 2022] POST /identity/v3/users => generated 301 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:28.455035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c11a75-8166-4931-8f4b-0d96a00b7cf7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:28.488715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c11a75-8166-4931-8f4b-0d96a00b7cf7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:28.503281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f8f47-c95b-4143-b112-7b171baabdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:28.503927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f8f47-c95b-4143-b112-7b171baabdcf admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:28.504208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f8f47-c95b-4143-b112-7b171baabdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:28.504488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f8f47-c95b-4143-b112-7b171baabdcf admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:28.505146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f8f47-c95b-4143-b112-7b171baabdcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:28.506844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f8f47-c95b-4143-b112-7b171baabdcf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:28.523665 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2702/5334] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:06:28.528726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdf279-e484-4905-94a8-cd009045476b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:28.550891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdf279-e484-4905-94a8-cd009045476b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:28.558957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f922ee-27b5-4aea-86ed-94aef5451786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:28.560098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f922ee-27b5-4aea-86ed-94aef5451786 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:28.560590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f922ee-27b5-4aea-86ed-94aef5451786 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:28.560878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f922ee-27b5-4aea-86ed-94aef5451786 admin admin] PATH_INFO: `/v3/projects/d6ba34a891e74160a851d61cef7ad5ec/users/9a47e1612ae744ad8a9400b64c44f7c7/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:28.588809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f922ee-27b5-4aea-86ed-94aef5451786 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6ba34a891e74160a851d61cef7ad5ec, user_id=9a47e1612ae744ad8a9400b64c44f7c7, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:28.606238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f922ee-27b5-4aea-86ed-94aef5451786 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:28.637439 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2633/5335] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:06:28 2022] PUT /identity/v3/projects/d6ba34a891e74160a851d61cef7ad5ec/users/9a47e1612ae744ad8a9400b64c44f7c7/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:28.641344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f8f47-c95b-4143-b112-7b171baabdcf admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:28.654797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f8f47-c95b-4143-b112-7b171baabdcf admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:28.762946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b7e27-93c1-4ffd-adf0-d640a7183cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:28.763706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b7e27-93c1-4ffd-adf0-d640a7183cbe admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:28.764462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b7e27-93c1-4ffd-adf0-d640a7183cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:28.764462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b7e27-93c1-4ffd-adf0-d640a7183cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:28.765190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b7e27-93c1-4ffd-adf0-d640a7183cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:28.767740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b7e27-93c1-4ffd-adf0-d640a7183cbe admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:28.787425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f922ee-27b5-4aea-86ed-94aef5451786 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:28.787896 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2703/5336] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:28.799953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f922ee-27b5-4aea-86ed-94aef5451786 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:28.813760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44558ba9-bb80-4925-9e3d-466d59412e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:28.814603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44558ba9-bb80-4925-9e3d-466d59412e60 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:28.815504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44558ba9-bb80-4925-9e3d-466d59412e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:28.815504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44558ba9-bb80-4925-9e3d-466d59412e60 admin admin] PATH_INFO: `/v3/projects/d6ba34a891e74160a851d61cef7ad5ec/users/9a47e1612ae744ad8a9400b64c44f7c7/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:28.817632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44558ba9-bb80-4925-9e3d-466d59412e60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6ba34a891e74160a851d61cef7ad5ec, user_id=9a47e1612ae744ad8a9400b64c44f7c7, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:28.820475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44558ba9-bb80-4925-9e3d-466d59412e60 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:28.855139 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2634/5337] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:06:28 2022] PUT /identity/v3/projects/d6ba34a891e74160a851d61cef7ad5ec/users/9a47e1612ae744ad8a9400b64c44f7c7/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:06:28.866847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3d682-02cf-4316-9f2a-859cae245a08 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:28.867275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3d682-02cf-4316-9f2a-859cae245a08 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:28.867485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3d682-02cf-4316-9f2a-859cae245a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:28.919084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3dd3d682-02cf-4316-9f2a-859cae245a08 None None] MFA Rules not processed for user `9a47e1612ae744ad8a9400b64c44f7c7`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:28.924962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3dd3d682-02cf-4316-9f2a-859cae245a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:29.077918 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2704/5338] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:28 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:29.140595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44558ba9-bb80-4925-9e3d-466d59412e60 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:29.144996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-44558ba9-bb80-4925-9e3d-466d59412e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:29.309881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d094a8-1843-44a0-bb98-2e54eece7d11 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:29.310205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d094a8-1843-44a0-bb98-2e54eece7d11 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:29.310445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d094a8-1843-44a0-bb98-2e54eece7d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:29.318873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:29.360357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44558ba9-bb80-4925-9e3d-466d59412e60 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:29.365459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebd8cb-4998-438e-a9e5-ab2b8cd3ff8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:29.366397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebd8cb-4998-438e-a9e5-ab2b8cd3ff8f service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:29.366848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebd8cb-4998-438e-a9e5-ab2b8cd3ff8f service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:29.367640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebd8cb-4998-438e-a9e5-ab2b8cd3ff8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:29.376887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:29.389843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-42d094a8-1843-44a0-bb98-2e54eece7d11 None None] MFA Rules not processed for user `f74f251052ee4c9a99db0c43bc00da9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:29.391102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-42d094a8-1843-44a0-bb98-2e54eece7d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:29.441063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ebd8cb-4998-438e-a9e5-ab2b8cd3ff8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:29.444104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ebd8cb-4998-438e-a9e5-ab2b8cd3ff8f service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:29.455732 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2635/5339] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:29 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:06:29.485279 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2705/5340] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:31.603279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebd8cb-4998-438e-a9e5-ab2b8cd3ff8f service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:31.608115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-74ebd8cb-4998-438e-a9e5-ab2b8cd3ff8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:31.789525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebd8cb-4998-438e-a9e5-ab2b8cd3ff8f service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:31.799988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c57b9-533f-4b3e-91b6-23a0db11d80d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:31.801830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c57b9-533f-4b3e-91b6-23a0db11d80d service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:31.802171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c57b9-533f-4b3e-91b6-23a0db11d80d service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:31.802635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c57b9-533f-4b3e-91b6-23a0db11d80d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:31.817245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c57b9-533f-4b3e-91b6-23a0db11d80d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:31.819008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c57b9-533f-4b3e-91b6-23a0db11d80d service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:31.844018 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2636/5341] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:06:34.284236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe286b4a-8f2d-4a77-8672-09c4e17531d7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:34.284797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe286b4a-8f2d-4a77-8672-09c4e17531d7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:34.284797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe286b4a-8f2d-4a77-8672-09c4e17531d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:34.307538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fe286b4a-8f2d-4a77-8672-09c4e17531d7 None None] MFA Rules not processed for user `6d63c633112e49d091421c84cea3422b`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:34.310536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fe286b4a-8f2d-4a77-8672-09c4e17531d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:34.446234 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2706/5342] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:34.849500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ffb544-dc42-494e-ae27-73295b5decf7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:34.850162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ffb544-dc42-494e-ae27-73295b5decf7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:34.850162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ffb544-dc42-494e-ae27-73295b5decf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:34.881149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-28ffb544-dc42-494e-ae27-73295b5decf7 None None] MFA Rules not processed for user `f74f251052ee4c9a99db0c43bc00da9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:34.884804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-28ffb544-dc42-494e-ae27-73295b5decf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:34.908020 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2637/5343] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:35.219196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe286b4a-8f2d-4a77-8672-09c4e17531d7 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:35.221765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe286b4a-8f2d-4a77-8672-09c4e17531d7 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:35.225838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfe6cf-474b-4967-92c8-e44fbc8a9152 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:35.226455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfe6cf-474b-4967-92c8-e44fbc8a9152 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:35.226821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfe6cf-474b-4967-92c8-e44fbc8a9152 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:35.227095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfe6cf-474b-4967-92c8-e44fbc8a9152 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:35.240503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bfe6cf-474b-4967-92c8-e44fbc8a9152 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:35.243832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bfe6cf-474b-4967-92c8-e44fbc8a9152 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:35.265084 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2707/5344] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:35 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:06:37.128617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ffb544-dc42-494e-ae27-73295b5decf7 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:37.130876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-28ffb544-dc42-494e-ae27-73295b5decf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:37.292557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ffb544-dc42-494e-ae27-73295b5decf7 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:37.338857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4236e17-5259-4b49-bd4b-69df26525082 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:37.343041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4236e17-5259-4b49-bd4b-69df26525082 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:37.344068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4236e17-5259-4b49-bd4b-69df26525082 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:37.344351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4236e17-5259-4b49-bd4b-69df26525082 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:37.352015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a4236e17-5259-4b49-bd4b-69df26525082 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:37.403681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4236e17-5259-4b49-bd4b-69df26525082 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:37.408487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4236e17-5259-4b49-bd4b-69df26525082 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:37.435399 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2638/5345] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:06:40.048132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfe6cf-474b-4967-92c8-e44fbc8a9152 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:40.054599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-05bfe6cf-474b-4967-92c8-e44fbc8a9152 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:40.070521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfe6cf-474b-4967-92c8-e44fbc8a9152 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:40.077342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a00b75-64e0-4793-ad5c-34709531fe89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:40.078399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a00b75-64e0-4793-ad5c-34709531fe89 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:40.078977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a00b75-64e0-4793-ad5c-34709531fe89 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:40.080532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a00b75-64e0-4793-ad5c-34709531fe89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:40.083843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-40a00b75-64e0-4793-ad5c-34709531fe89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:40.336707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a00b75-64e0-4793-ad5c-34709531fe89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:40.338531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a00b75-64e0-4793-ad5c-34709531fe89 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:40.378220 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2708/5346] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:06:41.144432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4236e17-5259-4b49-bd4b-69df26525082 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:41.159176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:41.165167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4236e17-5259-4b49-bd4b-69df26525082 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:41.172263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3907e0-93f3-476b-af62-1e7582bd086e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:41.173309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3907e0-93f3-476b-af62-1e7582bd086e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:41.173779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3907e0-93f3-476b-af62-1e7582bd086e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:41.174298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3907e0-93f3-476b-af62-1e7582bd086e admin admin] PATH_INFO: `/v3/users/308faa166f634627958d6d990839f133` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:41.178371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3907e0-93f3-476b-af62-1e7582bd086e admin admin] RBAC: Authorizing `identity:delete_user(user_id=308faa166f634627958d6d990839f133)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:41.180671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3907e0-93f3-476b-af62-1e7582bd086e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:41.298637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1f3907e0-93f3-476b-af62-1e7582bd086e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '308faa166f634627958d6d990839f133'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:41.299429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1f3907e0-93f3-476b-af62-1e7582bd086e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '308faa166f634627958d6d990839f133'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:41.300297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1f3907e0-93f3-476b-af62-1e7582bd086e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '308faa166f634627958d6d990839f133'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:41.314978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1f3907e0-93f3-476b-af62-1e7582bd086e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '308faa166f634627958d6d990839f133'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:41.334327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1f3907e0-93f3-476b-af62-1e7582bd086e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '308faa166f634627958d6d990839f133'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:41.351488 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2639/5347] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:06:41 2022] DELETE /identity/v3/users/308faa166f634627958d6d990839f133 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:41.356816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68610b-c3ac-46c9-9911-d7c9ad413bf0 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:41.357319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68610b-c3ac-46c9-9911-d7c9ad413bf0 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:41.357399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68610b-c3ac-46c9-9911-d7c9ad413bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:41.384792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-bc68610b-c3ac-46c9-9911-d7c9ad413bf0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:41.386348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bc68610b-c3ac-46c9-9911-d7c9ad413bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:41.395833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:41.487862 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2709/5348] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:41.499678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3907e0-93f3-476b-af62-1e7582bd086e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:41.503380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1f3907e0-93f3-476b-af62-1e7582bd086e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:41.594764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34355d-4c60-45be-9493-757b16874518 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:41.595411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34355d-4c60-45be-9493-757b16874518 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:41.595711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34355d-4c60-45be-9493-757b16874518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:41.615157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3907e0-93f3-476b-af62-1e7582bd086e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:41.619537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6dda45-be1d-48ba-8c74-57812f9dbcf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:41.620606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6dda45-be1d-48ba-8c74-57812f9dbcf9 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:41.620606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6dda45-be1d-48ba-8c74-57812f9dbcf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:41.621016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6dda45-be1d-48ba-8c74-57812f9dbcf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:41.639672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6dda45-be1d-48ba-8c74-57812f9dbcf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:41.641283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6dda45-be1d-48ba-8c74-57812f9dbcf9 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:41.646437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2d34355d-4c60-45be-9493-757b16874518 None None] MFA Rules not processed for user `6d63c633112e49d091421c84cea3422b`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:41.649004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2d34355d-4c60-45be-9493-757b16874518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:41.657664 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2640/5349] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:06:41.802773 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2710/5350] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:42.134528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6dda45-be1d-48ba-8c74-57812f9dbcf9 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:42.136062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7b6dda45-be1d-48ba-8c74-57812f9dbcf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:42.164580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6dda45-be1d-48ba-8c74-57812f9dbcf9 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:42.179828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7160c-7374-4507-a757-a7a3961c9b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:42.181126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7160c-7374-4507-a757-a7a3961c9b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:42.181595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7160c-7374-4507-a757-a7a3961c9b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:42.182090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7160c-7374-4507-a757-a7a3961c9b5e admin admin] PATH_INFO: `/v3/projects/d75d4687546f4aa6ae14e09183198f00` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:42.192563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f7160c-7374-4507-a757-a7a3961c9b5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d75d4687546f4aa6ae14e09183198f00)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:42.208115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f7160c-7374-4507-a757-a7a3961c9b5e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:42.338909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-94f7160c-7374-4507-a757-a7a3961c9b5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd75d4687546f4aa6ae14e09183198f00', 'request_id': 'req-94f7160c-7374-4507-a757-a7a3961c9b5e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:42.353626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-94f7160c-7374-4507-a757-a7a3961c9b5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd75d4687546f4aa6ae14e09183198f00', 'request_id': 'req-94f7160c-7374-4507-a757-a7a3961c9b5e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:42.364300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-94f7160c-7374-4507-a757-a7a3961c9b5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd75d4687546f4aa6ae14e09183198f00', 'request_id': 'req-94f7160c-7374-4507-a757-a7a3961c9b5e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:42.388641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-94f7160c-7374-4507-a757-a7a3961c9b5e admin admin] The token cache is being invalidate because project d75d4687546f4aa6ae14e09183198f00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:06:42.389135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-94f7160c-7374-4507-a757-a7a3961c9b5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:42.390114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-94f7160c-7374-4507-a757-a7a3961c9b5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:42.392112 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2641/5351] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:06:42 2022] DELETE /identity/v3/projects/d75d4687546f4aa6ae14e09183198f00 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:42.396321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34355d-4c60-45be-9493-757b16874518 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:42.418172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34355d-4c60-45be-9493-757b16874518 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:42.525395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9189e-0012-4c64-8738-46134a303354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:42.526419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9189e-0012-4c64-8738-46134a303354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:42.526999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9189e-0012-4c64-8738-46134a303354 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:42.527438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9189e-0012-4c64-8738-46134a303354 admin admin] PATH_INFO: `/v3/users/9a47e1612ae744ad8a9400b64c44f7c7` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:42.528844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c9189e-0012-4c64-8738-46134a303354 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a47e1612ae744ad8a9400b64c44f7c7)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:42.531008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c9189e-0012-4c64-8738-46134a303354 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:42.640484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-72c9189e-0012-4c64-8738-46134a303354 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a47e1612ae744ad8a9400b64c44f7c7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:42.649084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-72c9189e-0012-4c64-8738-46134a303354 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a47e1612ae744ad8a9400b64c44f7c7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:42.651557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:42.666447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-72c9189e-0012-4c64-8738-46134a303354 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a47e1612ae744ad8a9400b64c44f7c7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:42.668229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-72c9189e-0012-4c64-8738-46134a303354 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a47e1612ae744ad8a9400b64c44f7c7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:42.678871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-72c9189e-0012-4c64-8738-46134a303354 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a47e1612ae744ad8a9400b64c44f7c7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:42.689218 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2711/5352] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:06:42 2022] DELETE /identity/v3/users/9a47e1612ae744ad8a9400b64c44f7c7 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:43.247996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7160c-7374-4507-a757-a7a3961c9b5e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:43.251002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-94f7160c-7374-4507-a757-a7a3961c9b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:43.291023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:43.310863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7160c-7374-4507-a757-a7a3961c9b5e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:43.384819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9189e-0012-4c64-8738-46134a303354 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:43.386968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-72c9189e-0012-4c64-8738-46134a303354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:43.460280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07107366-a134-4f94-ab04-d7761d7b8403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:43.461713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07107366-a134-4f94-ab04-d7761d7b8403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:43.461814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07107366-a134-4f94-ab04-d7761d7b8403 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:43.462074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07107366-a134-4f94-ab04-d7761d7b8403 admin admin] PATH_INFO: `/v3/projects/d6ba34a891e74160a851d61cef7ad5ec` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:43.467099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07107366-a134-4f94-ab04-d7761d7b8403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6ba34a891e74160a851d61cef7ad5ec)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:43.474744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07107366-a134-4f94-ab04-d7761d7b8403 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:43.587371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-07107366-a134-4f94-ab04-d7761d7b8403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6ba34a891e74160a851d61cef7ad5ec', 'request_id': 'req-07107366-a134-4f94-ab04-d7761d7b8403'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:43.600238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-07107366-a134-4f94-ab04-d7761d7b8403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6ba34a891e74160a851d61cef7ad5ec', 'request_id': 'req-07107366-a134-4f94-ab04-d7761d7b8403'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:43.617550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-07107366-a134-4f94-ab04-d7761d7b8403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6ba34a891e74160a851d61cef7ad5ec', 'request_id': 'req-07107366-a134-4f94-ab04-d7761d7b8403'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:43.644803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-07107366-a134-4f94-ab04-d7761d7b8403 admin admin] The token cache is being invalidate because project d6ba34a891e74160a851d61cef7ad5ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:06:43.645046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-07107366-a134-4f94-ab04-d7761d7b8403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:43.646129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-07107366-a134-4f94-ab04-d7761d7b8403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:43.648871 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2642/5353] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:06:43 2022] DELETE /identity/v3/projects/d6ba34a891e74160a851d61cef7ad5ec => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:43.654851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5283e-310d-4294-8b8f-12e2d2cb073a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:43.655361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5283e-310d-4294-8b8f-12e2d2cb073a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:43.657916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5283e-310d-4294-8b8f-12e2d2cb073a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:43.668208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9189e-0012-4c64-8738-46134a303354 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:43.748330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-45f5283e-310d-4294-8b8f-12e2d2cb073a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:43.750029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-45f5283e-310d-4294-8b8f-12e2d2cb073a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:43.847505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028415a-cebe-4ba1-be17-8115a6c39605 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:43.848685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028415a-cebe-4ba1-be17-8115a6c39605 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:43.849111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028415a-cebe-4ba1-be17-8115a6c39605 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:43.849648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028415a-cebe-4ba1-be17-8115a6c39605 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:43.851791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7028415a-cebe-4ba1-be17-8115a6c39605 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:43.866121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028415a-cebe-4ba1-be17-8115a6c39605 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:43.869177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028415a-cebe-4ba1-be17-8115a6c39605 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:43.882764 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2643/5354] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:43.890144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac089dc-eba2-4b5b-9f94-f32aea1bd22d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:43.891496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac089dc-eba2-4b5b-9f94-f32aea1bd22d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:43.892066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac089dc-eba2-4b5b-9f94-f32aea1bd22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:43.900604 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2712/5355] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:06:43.922451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6ac089dc-eba2-4b5b-9f94-f32aea1bd22d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:43.924123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6ac089dc-eba2-4b5b-9f94-f32aea1bd22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:43.938004 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2644/5356] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:06:43.943667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028415a-cebe-4ba1-be17-8115a6c39605 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:43.961466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028415a-cebe-4ba1-be17-8115a6c39605 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:43.983735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9548f7a-e483-4d82-b043-0360204778eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:43.985527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9548f7a-e483-4d82-b043-0360204778eb admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:43.985816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9548f7a-e483-4d82-b043-0360204778eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:43.988874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9548f7a-e483-4d82-b043-0360204778eb admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:43.989556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9548f7a-e483-4d82-b043-0360204778eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:06:43.989868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9548f7a-e483-4d82-b043-0360204778eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:43.996276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9548f7a-e483-4d82-b043-0360204778eb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:44.045469 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2713/5357] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:06:44.049352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac089dc-eba2-4b5b-9f94-f32aea1bd22d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:44.069588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac089dc-eba2-4b5b-9f94-f32aea1bd22d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:44.079939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9af114-9598-4b5d-a108-e62626fd451c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:44.080688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9af114-9598-4b5d-a108-e62626fd451c admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:44.081078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9af114-9598-4b5d-a108-e62626fd451c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:44.081591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9af114-9598-4b5d-a108-e62626fd451c admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:44.082324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9af114-9598-4b5d-a108-e62626fd451c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:44.087051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9af114-9598-4b5d-a108-e62626fd451c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:44.122968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:44.155473 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2645/5358] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:06:44 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:06:44.159577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9548f7a-e483-4d82-b043-0360204778eb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:44.172517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9548f7a-e483-4d82-b043-0360204778eb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:44.272780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:44.289161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ecd1e-f0b5-4cd8-bf4b-c74a39f4bb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:44.290355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ecd1e-f0b5-4cd8-bf4b-c74a39f4bb50 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:44.301884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ecd1e-f0b5-4cd8-bf4b-c74a39f4bb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:44.302522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ecd1e-f0b5-4cd8-bf4b-c74a39f4bb50 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:44.308378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ecd1e-f0b5-4cd8-bf4b-c74a39f4bb50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:44.310683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ecd1e-f0b5-4cd8-bf4b-c74a39f4bb50 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:44.366012 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2714/5359] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:06:44 2022] POST /identity/v3/users => generated 288 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:44.366949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9af114-9598-4b5d-a108-e62626fd451c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:44.383738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9af114-9598-4b5d-a108-e62626fd451c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:44.397249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb067ea8-797a-4a09-b11a-f7caecc20ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:44.398268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb067ea8-797a-4a09-b11a-f7caecc20ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:44.403455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb067ea8-797a-4a09-b11a-f7caecc20ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:44.404079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb067ea8-797a-4a09-b11a-f7caecc20ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:44.405003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb067ea8-797a-4a09-b11a-f7caecc20ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:44.407608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb067ea8-797a-4a09-b11a-f7caecc20ac1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:44.436884 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2646/5360] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:44.438984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ecd1e-f0b5-4cd8-bf4b-c74a39f4bb50 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:44.453951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ecd1e-f0b5-4cd8-bf4b-c74a39f4bb50 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:44.459187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a5d4d-a1e3-479c-b091-1e5ee16c9c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:44.459814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a5d4d-a1e3-479c-b091-1e5ee16c9c74 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:44.460659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a5d4d-a1e3-479c-b091-1e5ee16c9c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:44.462077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a5d4d-a1e3-479c-b091-1e5ee16c9c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:44.463105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a5d4d-a1e3-479c-b091-1e5ee16c9c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:44.464876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a5d4d-a1e3-479c-b091-1e5ee16c9c74 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:44.484291 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2715/5361] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:44.488608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb067ea8-797a-4a09-b11a-f7caecc20ac1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:44.501391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb067ea8-797a-4a09-b11a-f7caecc20ac1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:44.524072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81dbab-12fb-4b61-9043-f9a04f583e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:44.525164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81dbab-12fb-4b61-9043-f9a04f583e8d admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:44.525733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81dbab-12fb-4b61-9043-f9a04f583e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:44.527387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81dbab-12fb-4b61-9043-f9a04f583e8d admin admin] PATH_INFO: `/v3/projects/cb23de34eb1c46d8b8081fa873fd7e4b/users/cd1259e0f93346239c8c8966db3c624a/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:44.559932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81dbab-12fb-4b61-9043-f9a04f583e8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb23de34eb1c46d8b8081fa873fd7e4b, user_id=cd1259e0f93346239c8c8966db3c624a, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:44.564437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81dbab-12fb-4b61-9043-f9a04f583e8d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:44.602359 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2647/5362] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:06:44 2022] PUT /identity/v3/projects/cb23de34eb1c46d8b8081fa873fd7e4b/users/cd1259e0f93346239c8c8966db3c624a/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:44.614233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6783af-1b10-4c72-8d25-1c1d82ff78ec None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:44.614651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6783af-1b10-4c72-8d25-1c1d82ff78ec None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:44.614996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6783af-1b10-4c72-8d25-1c1d82ff78ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:44.646073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3a6783af-1b10-4c72-8d25-1c1d82ff78ec None None] MFA Rules not processed for user `cd1259e0f93346239c8c8966db3c624a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:44.647450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3a6783af-1b10-4c72-8d25-1c1d82ff78ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:44.726775 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2716/5363] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:44.732167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81dbab-12fb-4b61-9043-f9a04f583e8d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:44.847327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81dbab-12fb-4b61-9043-f9a04f583e8d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:44.856109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a2fa6-db4b-4af1-9b2a-9eeca0086233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:44.856780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a2fa6-db4b-4af1-9b2a-9eeca0086233 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:44.857075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a2fa6-db4b-4af1-9b2a-9eeca0086233 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:44.857341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a2fa6-db4b-4af1-9b2a-9eeca0086233 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:44.858520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a2fa6-db4b-4af1-9b2a-9eeca0086233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:44.862595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a2fa6-db4b-4af1-9b2a-9eeca0086233 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:44.909574 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2648/5364] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:06:44 2022] POST /identity/v3/projects => generated 375 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:06:44.913396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6783af-1b10-4c72-8d25-1c1d82ff78ec None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:44.933353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6783af-1b10-4c72-8d25-1c1d82ff78ec None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:45.054327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43555132-4e3b-4834-8ba2-0fb4079730a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:45.055410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43555132-4e3b-4834-8ba2-0fb4079730a3 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:45.056025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43555132-4e3b-4834-8ba2-0fb4079730a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:45.056452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43555132-4e3b-4834-8ba2-0fb4079730a3 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:45.057267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43555132-4e3b-4834-8ba2-0fb4079730a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:45.064461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43555132-4e3b-4834-8ba2-0fb4079730a3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:45.101359 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2717/5365] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:06:44 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:06:45.105053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a2fa6-db4b-4af1-9b2a-9eeca0086233 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:45.118112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a2fa6-db4b-4af1-9b2a-9eeca0086233 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:45.125087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396cc98-2457-46e0-bd45-eef703e81f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:45.126021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396cc98-2457-46e0-bd45-eef703e81f42 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:45.126481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396cc98-2457-46e0-bd45-eef703e81f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:45.126995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396cc98-2457-46e0-bd45-eef703e81f42 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:45.131395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396cc98-2457-46e0-bd45-eef703e81f42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:45.134266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396cc98-2457-46e0-bd45-eef703e81f42 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:45.163500 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2649/5366] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:45.167460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43555132-4e3b-4834-8ba2-0fb4079730a3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:45.181366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43555132-4e3b-4834-8ba2-0fb4079730a3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:45.189478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337109a6-b3d3-4565-b74d-6a4a6a9cad31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:45.191202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337109a6-b3d3-4565-b74d-6a4a6a9cad31 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:45.191660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337109a6-b3d3-4565-b74d-6a4a6a9cad31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:45.192307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337109a6-b3d3-4565-b74d-6a4a6a9cad31 admin admin] PATH_INFO: `/v3/projects/8228a8c9bf064bb8a2b810feeeb4506a/users/cff85250ab4341448b70f4dc9bac0050/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:45.219956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337109a6-b3d3-4565-b74d-6a4a6a9cad31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8228a8c9bf064bb8a2b810feeeb4506a, user_id=cff85250ab4341448b70f4dc9bac0050, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:45.223289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337109a6-b3d3-4565-b74d-6a4a6a9cad31 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:45.301586 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2718/5367] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:06:45 2022] PUT /identity/v3/projects/8228a8c9bf064bb8a2b810feeeb4506a/users/cff85250ab4341448b70f4dc9bac0050/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:06:45.305673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396cc98-2457-46e0-bd45-eef703e81f42 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:45.323476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396cc98-2457-46e0-bd45-eef703e81f42 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:45.456544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec7d47-e587-416b-ba03-a902cc30240d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:45.457169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec7d47-e587-416b-ba03-a902cc30240d admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:45.457495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec7d47-e587-416b-ba03-a902cc30240d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:45.457779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec7d47-e587-416b-ba03-a902cc30240d admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:45.458738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec7d47-e587-416b-ba03-a902cc30240d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:45.460715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec7d47-e587-416b-ba03-a902cc30240d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:45.481117 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2650/5368] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:45.485220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337109a6-b3d3-4565-b74d-6a4a6a9cad31 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:45.502316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337109a6-b3d3-4565-b74d-6a4a6a9cad31 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:45.517986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf61e7-356a-4377-a11d-56483408c91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:45.519043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf61e7-356a-4377-a11d-56483408c91e admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:45.519582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf61e7-356a-4377-a11d-56483408c91e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:45.520251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf61e7-356a-4377-a11d-56483408c91e admin admin] PATH_INFO: `/v3/projects/8228a8c9bf064bb8a2b810feeeb4506a/users/cff85250ab4341448b70f4dc9bac0050/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:45.523386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf61e7-356a-4377-a11d-56483408c91e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8228a8c9bf064bb8a2b810feeeb4506a, user_id=cff85250ab4341448b70f4dc9bac0050, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:45.528361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf61e7-356a-4377-a11d-56483408c91e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:45.543423 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2719/5369] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:06:45 2022] PUT /identity/v3/projects/8228a8c9bf064bb8a2b810feeeb4506a/users/cff85250ab4341448b70f4dc9bac0050/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:45.554185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bed17-47dd-4aec-93d2-34494efa1b4e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:45.554481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bed17-47dd-4aec-93d2-34494efa1b4e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:45.563272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bed17-47dd-4aec-93d2-34494efa1b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:45.628586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6f2bed17-47dd-4aec-93d2-34494efa1b4e None None] MFA Rules not processed for user `cff85250ab4341448b70f4dc9bac0050`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:45.631533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6f2bed17-47dd-4aec-93d2-34494efa1b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:45.661804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:45.767304 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2651/5370] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:45 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:45.807517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf61e7-356a-4377-a11d-56483408c91e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:45.811547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ffaf61e7-356a-4377-a11d-56483408c91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:45.842529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:45.894587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf61e7-356a-4377-a11d-56483408c91e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:45.898445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20a52d-135d-451f-a291-fa6af4c67579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:45.899190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20a52d-135d-451f-a291-fa6af4c67579 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:45.899690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20a52d-135d-451f-a291-fa6af4c67579 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:45.900069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20a52d-135d-451f-a291-fa6af4c67579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:45.922080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20a52d-135d-451f-a291-fa6af4c67579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:45.923789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20a52d-135d-451f-a291-fa6af4c67579 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:45.936757 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2720/5371] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:45 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:06:46.181682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fafeb-1db6-43ba-acba-f535d9537c28 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:46.182089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fafeb-1db6-43ba-acba-f535d9537c28 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:46.182204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fafeb-1db6-43ba-acba-f535d9537c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:46.222994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-199fafeb-1db6-43ba-acba-f535d9537c28 None None] MFA Rules not processed for user `6d63c633112e49d091421c84cea3422b`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:46.224372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-199fafeb-1db6-43ba-acba-f535d9537c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:46.335435 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2652/5372] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:49.933555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20a52d-135d-451f-a291-fa6af4c67579 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:50.112223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20a52d-135d-451f-a291-fa6af4c67579 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:50.126527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916f56d-88fe-48d5-88d6-48f476a04ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:50.127505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916f56d-88fe-48d5-88d6-48f476a04ff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:50.128289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916f56d-88fe-48d5-88d6-48f476a04ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:50.129109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916f56d-88fe-48d5-88d6-48f476a04ff2 admin admin] PATH_INFO: `/v3/users/cd1259e0f93346239c8c8966db3c624a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:50.133051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916f56d-88fe-48d5-88d6-48f476a04ff2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd1259e0f93346239c8c8966db3c624a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:50.136594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916f56d-88fe-48d5-88d6-48f476a04ff2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:50.203560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0916f56d-88fe-48d5-88d6-48f476a04ff2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd1259e0f93346239c8c8966db3c624a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:50.235376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0916f56d-88fe-48d5-88d6-48f476a04ff2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd1259e0f93346239c8c8966db3c624a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:50.275650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0916f56d-88fe-48d5-88d6-48f476a04ff2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd1259e0f93346239c8c8966db3c624a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:50.278542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0916f56d-88fe-48d5-88d6-48f476a04ff2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd1259e0f93346239c8c8966db3c624a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:50.300804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0916f56d-88fe-48d5-88d6-48f476a04ff2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd1259e0f93346239c8c8966db3c624a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:50.316540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377a108-1e33-44b1-bb21-50b5d26ea0e4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:50.317175 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2721/5373] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:06:49 2022] DELETE /identity/v3/users/cd1259e0f93346239c8c8966db3c624a => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:50.317292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377a108-1e33-44b1-bb21-50b5d26ea0e4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:50.317292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377a108-1e33-44b1-bb21-50b5d26ea0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:50.352345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f377a108-1e33-44b1-bb21-50b5d26ea0e4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:50.353906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f377a108-1e33-44b1-bb21-50b5d26ea0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:50.481889 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2653/5374] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:50.511740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916f56d-88fe-48d5-88d6-48f476a04ff2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:50.513381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0916f56d-88fe-48d5-88d6-48f476a04ff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:50.612943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:50.648199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916f56d-88fe-48d5-88d6-48f476a04ff2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:50.659204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93a52a-fc96-4638-9284-64fe9719a452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:50.660041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93a52a-fc96-4638-9284-64fe9719a452 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:50.660260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93a52a-fc96-4638-9284-64fe9719a452 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:50.660522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93a52a-fc96-4638-9284-64fe9719a452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:50.681416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93a52a-fc96-4638-9284-64fe9719a452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:50.691581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93a52a-fc96-4638-9284-64fe9719a452 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:50.707162 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2722/5375] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:06:52.021288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377a108-1e33-44b1-bb21-50b5d26ea0e4 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:52.024378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f377a108-1e33-44b1-bb21-50b5d26ea0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:52.039091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377a108-1e33-44b1-bb21-50b5d26ea0e4 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:52.054098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7767efb-cbc6-4d2a-968b-3d4206d70e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:52.054793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7767efb-cbc6-4d2a-968b-3d4206d70e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:52.055083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7767efb-cbc6-4d2a-968b-3d4206d70e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:52.056011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7767efb-cbc6-4d2a-968b-3d4206d70e51 admin admin] PATH_INFO: `/v3/projects/cb23de34eb1c46d8b8081fa873fd7e4b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:52.057309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7767efb-cbc6-4d2a-968b-3d4206d70e51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb23de34eb1c46d8b8081fa873fd7e4b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:52.059502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7767efb-cbc6-4d2a-968b-3d4206d70e51 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:52.154604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a7767efb-cbc6-4d2a-968b-3d4206d70e51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb23de34eb1c46d8b8081fa873fd7e4b', 'request_id': 'req-a7767efb-cbc6-4d2a-968b-3d4206d70e51'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:52.170734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a7767efb-cbc6-4d2a-968b-3d4206d70e51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb23de34eb1c46d8b8081fa873fd7e4b', 'request_id': 'req-a7767efb-cbc6-4d2a-968b-3d4206d70e51'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:52.173250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:52.182282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a7767efb-cbc6-4d2a-968b-3d4206d70e51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb23de34eb1c46d8b8081fa873fd7e4b', 'request_id': 'req-a7767efb-cbc6-4d2a-968b-3d4206d70e51'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:52.203920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a7767efb-cbc6-4d2a-968b-3d4206d70e51 admin admin] The token cache is being invalidate because project cb23de34eb1c46d8b8081fa873fd7e4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:06:52.204857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a7767efb-cbc6-4d2a-968b-3d4206d70e51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:52.206120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a7767efb-cbc6-4d2a-968b-3d4206d70e51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:52.208095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93a52a-fc96-4638-9284-64fe9719a452 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:52.210593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ef93a52a-fc96-4638-9284-64fe9719a452 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:52.213672 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2654/5376] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:06:52 2022] DELETE /identity/v3/projects/cb23de34eb1c46d8b8081fa873fd7e4b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:52.217601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7767efb-cbc6-4d2a-968b-3d4206d70e51 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:52.219724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a7767efb-cbc6-4d2a-968b-3d4206d70e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:52.228936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93a52a-fc96-4638-9284-64fe9719a452 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:52.341173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802104f-9652-41b2-a3da-ca167b1cc1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:52.342834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802104f-9652-41b2-a3da-ca167b1cc1f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:52.343280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802104f-9652-41b2-a3da-ca167b1cc1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:52.343924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802104f-9652-41b2-a3da-ca167b1cc1f1 admin admin] PATH_INFO: `/v3/users/6bda28994cb14f92b014a147a7e74476` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:52.345403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802104f-9652-41b2-a3da-ca167b1cc1f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bda28994cb14f92b014a147a7e74476)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:52.347523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802104f-9652-41b2-a3da-ca167b1cc1f1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:52.358080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7767efb-cbc6-4d2a-968b-3d4206d70e51 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:52.376773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ad243-70fa-4ea9-9dde-a8844665c753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:52.377618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ad243-70fa-4ea9-9dde-a8844665c753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:52.379060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ad243-70fa-4ea9-9dde-a8844665c753 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:52.379060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ad243-70fa-4ea9-9dde-a8844665c753 admin admin] PATH_INFO: `/v3/users/de8c90c2e17c4fb799678e81e57d9765` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:52.380271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98ad243-70fa-4ea9-9dde-a8844665c753 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de8c90c2e17c4fb799678e81e57d9765)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:52.382525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98ad243-70fa-4ea9-9dde-a8844665c753 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:52.416270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7802104f-9652-41b2-a3da-ca167b1cc1f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bda28994cb14f92b014a147a7e74476'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:52.427588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7802104f-9652-41b2-a3da-ca167b1cc1f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bda28994cb14f92b014a147a7e74476'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:52.451297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7802104f-9652-41b2-a3da-ca167b1cc1f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bda28994cb14f92b014a147a7e74476'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:52.452051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7802104f-9652-41b2-a3da-ca167b1cc1f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bda28994cb14f92b014a147a7e74476'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:52.462830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7802104f-9652-41b2-a3da-ca167b1cc1f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bda28994cb14f92b014a147a7e74476'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:52.471722 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2723/5377] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:06:52 2022] DELETE /identity/v3/users/6bda28994cb14f92b014a147a7e74476 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:52.472772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802104f-9652-41b2-a3da-ca167b1cc1f1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:52.485650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7802104f-9652-41b2-a3da-ca167b1cc1f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:52.493576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a98ad243-70fa-4ea9-9dde-a8844665c753 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de8c90c2e17c4fb799678e81e57d9765'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:52.497203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a98ad243-70fa-4ea9-9dde-a8844665c753 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de8c90c2e17c4fb799678e81e57d9765'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:52.499553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a98ad243-70fa-4ea9-9dde-a8844665c753 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de8c90c2e17c4fb799678e81e57d9765'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:52.516727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a98ad243-70fa-4ea9-9dde-a8844665c753 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de8c90c2e17c4fb799678e81e57d9765'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:52.538854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a98ad243-70fa-4ea9-9dde-a8844665c753 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de8c90c2e17c4fb799678e81e57d9765'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:52.568909 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2655/5378] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:06:52 2022] DELETE /identity/v3/users/de8c90c2e17c4fb799678e81e57d9765 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:52.568909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d5bba-221e-4fe4-8aef-7b064d823078 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:52.568909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d5bba-221e-4fe4-8aef-7b064d823078 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:52.569632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d5bba-221e-4fe4-8aef-7b064d823078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:52.583634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802104f-9652-41b2-a3da-ca167b1cc1f1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:52.588634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:52.604389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1d8d5bba-221e-4fe4-8aef-7b064d823078 None None] MFA Rules not processed for user `6d63c633112e49d091421c84cea3422b`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:52.607243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1d8d5bba-221e-4fe4-8aef-7b064d823078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:52.695523 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2656/5379] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:52.701416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd3616-ab6d-4ce9-9e68-0f8c034d01ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:52.702299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd3616-ab6d-4ce9-9e68-0f8c034d01ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:52.702751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd3616-ab6d-4ce9-9e68-0f8c034d01ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:52.703097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd3616-ab6d-4ce9-9e68-0f8c034d01ec admin admin] PATH_INFO: `/v3/users/cff85250ab4341448b70f4dc9bac0050` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:52.710928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd3616-ab6d-4ce9-9e68-0f8c034d01ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=cff85250ab4341448b70f4dc9bac0050)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:52.713052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd3616-ab6d-4ce9-9e68-0f8c034d01ec admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:52.786181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-91cd3616-ab6d-4ce9-9e68-0f8c034d01ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cff85250ab4341448b70f4dc9bac0050'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:52.794533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-91cd3616-ab6d-4ce9-9e68-0f8c034d01ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cff85250ab4341448b70f4dc9bac0050'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:52.809838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-91cd3616-ab6d-4ce9-9e68-0f8c034d01ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cff85250ab4341448b70f4dc9bac0050'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:52.810462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-91cd3616-ab6d-4ce9-9e68-0f8c034d01ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cff85250ab4341448b70f4dc9bac0050'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:52.820128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-91cd3616-ab6d-4ce9-9e68-0f8c034d01ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cff85250ab4341448b70f4dc9bac0050'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:52.829163 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2724/5380] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:06:52 2022] DELETE /identity/v3/users/cff85250ab4341448b70f4dc9bac0050 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:53.142696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d5bba-221e-4fe4-8aef-7b064d823078 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:53.145078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1d8d5bba-221e-4fe4-8aef-7b064d823078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:53.163240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d5bba-221e-4fe4-8aef-7b064d823078 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:53.169723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:53.253625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0fddb-4196-43c8-99f8-bed36a21e5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:53.254172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0fddb-4196-43c8-99f8-bed36a21e5b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:53.254456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0fddb-4196-43c8-99f8-bed36a21e5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:53.255913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0fddb-4196-43c8-99f8-bed36a21e5b5 admin admin] PATH_INFO: `/v3/projects/10fefee3d6a6430682316cd94b859f1e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:53.256599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0fddb-4196-43c8-99f8-bed36a21e5b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10fefee3d6a6430682316cd94b859f1e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:53.258558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0fddb-4196-43c8-99f8-bed36a21e5b5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:53.317568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fed0fddb-4196-43c8-99f8-bed36a21e5b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10fefee3d6a6430682316cd94b859f1e', 'request_id': 'req-fed0fddb-4196-43c8-99f8-bed36a21e5b5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:53.327572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fed0fddb-4196-43c8-99f8-bed36a21e5b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10fefee3d6a6430682316cd94b859f1e', 'request_id': 'req-fed0fddb-4196-43c8-99f8-bed36a21e5b5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:53.336087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fed0fddb-4196-43c8-99f8-bed36a21e5b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10fefee3d6a6430682316cd94b859f1e', 'request_id': 'req-fed0fddb-4196-43c8-99f8-bed36a21e5b5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:53.350704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fed0fddb-4196-43c8-99f8-bed36a21e5b5 admin admin] The token cache is being invalidate because project 10fefee3d6a6430682316cd94b859f1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:06:53.351033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fed0fddb-4196-43c8-99f8-bed36a21e5b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:53.351868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fed0fddb-4196-43c8-99f8-bed36a21e5b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:53.358604 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2657/5381] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:06:53 2022] DELETE /identity/v3/projects/10fefee3d6a6430682316cd94b859f1e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:53.360321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf15d97-9940-46e6-a177-8a762779f947 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:53.360812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf15d97-9940-46e6-a177-8a762779f947 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:53.360891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf15d97-9940-46e6-a177-8a762779f947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:53.384969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-cdf15d97-9940-46e6-a177-8a762779f947 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:53.386710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cdf15d97-9940-46e6-a177-8a762779f947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:53.470649 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2725/5382] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:06:53.476557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10b8a5-09d3-4dfa-ae59-cfb4d960a252 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:53.477054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10b8a5-09d3-4dfa-ae59-cfb4d960a252 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:53.477452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10b8a5-09d3-4dfa-ae59-cfb4d960a252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:53.500633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-be10b8a5-09d3-4dfa-ae59-cfb4d960a252 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:53.502195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-be10b8a5-09d3-4dfa-ae59-cfb4d960a252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:53.516140 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2658/5383] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:53.520344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf15d97-9940-46e6-a177-8a762779f947 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:53.535149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf15d97-9940-46e6-a177-8a762779f947 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:53.548060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e626c86-db81-4dd3-af4b-b82433a5417f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:53.549645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e626c86-db81-4dd3-af4b-b82433a5417f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:53.549933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e626c86-db81-4dd3-af4b-b82433a5417f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:53.550257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e626c86-db81-4dd3-af4b-b82433a5417f admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:53.551056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e626c86-db81-4dd3-af4b-b82433a5417f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:06:53.551389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e626c86-db81-4dd3-af4b-b82433a5417f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:53.554587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e626c86-db81-4dd3-af4b-b82433a5417f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:53.574561 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2726/5384] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:06:53.577433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10b8a5-09d3-4dfa-ae59-cfb4d960a252 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:53.591032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10b8a5-09d3-4dfa-ae59-cfb4d960a252 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:53.597807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d97c1c-241e-48c2-ab14-eb0596a34c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:53.599103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d97c1c-241e-48c2-ab14-eb0596a34c75 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:53.599570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d97c1c-241e-48c2-ab14-eb0596a34c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:53.600078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d97c1c-241e-48c2-ab14-eb0596a34c75 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:53.600956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d97c1c-241e-48c2-ab14-eb0596a34c75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:53.603215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d97c1c-241e-48c2-ab14-eb0596a34c75 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:53.691219 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2659/5385] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:06:53 2022] POST /identity/v3/projects => generated 405 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:53.696383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e626c86-db81-4dd3-af4b-b82433a5417f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:53.712698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e626c86-db81-4dd3-af4b-b82433a5417f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:53.728329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:53.765851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d97c1c-241e-48c2-ab14-eb0596a34c75 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:53.767792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b3d97c1c-241e-48c2-ab14-eb0596a34c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:53.801396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc54985-4229-4176-93fb-676ada1efe0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:53.802106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc54985-4229-4176-93fb-676ada1efe0e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:53.802441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc54985-4229-4176-93fb-676ada1efe0e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:53.804631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc54985-4229-4176-93fb-676ada1efe0e admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:53.805424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc54985-4229-4176-93fb-676ada1efe0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:53.813057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc54985-4229-4176-93fb-676ada1efe0e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:53.847465 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2727/5386] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:06:53 2022] POST /identity/v3/users => generated 303 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:06:53.850259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc54985-4229-4176-93fb-676ada1efe0e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:53.866013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc54985-4229-4176-93fb-676ada1efe0e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:53.872748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3ef14-51cc-4b6b-98ac-553c824f006a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:53.873582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3ef14-51cc-4b6b-98ac-553c824f006a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:53.874109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3ef14-51cc-4b6b-98ac-553c824f006a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:53.874660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3ef14-51cc-4b6b-98ac-553c824f006a admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:53.875467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c3ef14-51cc-4b6b-98ac-553c824f006a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:53.878937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c3ef14-51cc-4b6b-98ac-553c824f006a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:53.890421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d97c1c-241e-48c2-ab14-eb0596a34c75 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:53.900663 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2728/5387] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:53.906145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3ef14-51cc-4b6b-98ac-553c824f006a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:53.907635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21690528-bf75-4d5c-9411-4e7f314a5d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:53.908300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21690528-bf75-4d5c-9411-4e7f314a5d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:53.908584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21690528-bf75-4d5c-9411-4e7f314a5d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:53.908907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21690528-bf75-4d5c-9411-4e7f314a5d19 admin admin] PATH_INFO: `/v3/projects/3242ac5d0c3146b0bed8fd2f2e48584b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:53.910689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21690528-bf75-4d5c-9411-4e7f314a5d19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3242ac5d0c3146b0bed8fd2f2e48584b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:53.912840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21690528-bf75-4d5c-9411-4e7f314a5d19 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:53.918666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3ef14-51cc-4b6b-98ac-553c824f006a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:53.924791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ebc46-53d8-4b40-82f2-a857afbea719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:53.925696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ebc46-53d8-4b40-82f2-a857afbea719 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:53.926082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ebc46-53d8-4b40-82f2-a857afbea719 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:53.926360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ebc46-53d8-4b40-82f2-a857afbea719 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:53.927043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ebc46-53d8-4b40-82f2-a857afbea719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:53.928998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ebc46-53d8-4b40-82f2-a857afbea719 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:53.931556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:53.946652 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2729/5388] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:53.952022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ebc46-53d8-4b40-82f2-a857afbea719 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:53.966669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ebc46-53d8-4b40-82f2-a857afbea719 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:53.990154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e894f-12b6-492b-9d86-bd6657af07f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:53.990890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e894f-12b6-492b-9d86-bd6657af07f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:53.991178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e894f-12b6-492b-9d86-bd6657af07f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:53.991457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e894f-12b6-492b-9d86-bd6657af07f3 admin admin] PATH_INFO: `/v3/projects/a2d2517d51d0417880e6e3d601c52744/users/e7e3bc6d5e45497bb876d5cbcec61835/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:54.032558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-21690528-bf75-4d5c-9411-4e7f314a5d19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3242ac5d0c3146b0bed8fd2f2e48584b', 'request_id': 'req-21690528-bf75-4d5c-9411-4e7f314a5d19'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:54.044750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e894f-12b6-492b-9d86-bd6657af07f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2d2517d51d0417880e6e3d601c52744, user_id=e7e3bc6d5e45497bb876d5cbcec61835, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:54.049090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-21690528-bf75-4d5c-9411-4e7f314a5d19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3242ac5d0c3146b0bed8fd2f2e48584b', 'request_id': 'req-21690528-bf75-4d5c-9411-4e7f314a5d19'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:54.051226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e894f-12b6-492b-9d86-bd6657af07f3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:54.062994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-21690528-bf75-4d5c-9411-4e7f314a5d19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3242ac5d0c3146b0bed8fd2f2e48584b', 'request_id': 'req-21690528-bf75-4d5c-9411-4e7f314a5d19'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:54.084022 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2730/5389] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:06:53 2022] PUT /identity/v3/projects/a2d2517d51d0417880e6e3d601c52744/users/e7e3bc6d5e45497bb876d5cbcec61835/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:54.091169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-21690528-bf75-4d5c-9411-4e7f314a5d19 admin admin] The token cache is being invalidate because project 3242ac5d0c3146b0bed8fd2f2e48584b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:06:54.091769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-21690528-bf75-4d5c-9411-4e7f314a5d19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:54.091919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182005fd-8d72-43f9-8328-e1fc601fb300 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:54.092869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182005fd-8d72-43f9-8328-e1fc601fb300 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:54.092869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182005fd-8d72-43f9-8328-e1fc601fb300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:54.095095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-21690528-bf75-4d5c-9411-4e7f314a5d19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:54.098934 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2660/5390] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:06:53 2022] DELETE /identity/v3/projects/3242ac5d0c3146b0bed8fd2f2e48584b => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:06:54.102451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21690528-bf75-4d5c-9411-4e7f314a5d19 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:54.104182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-21690528-bf75-4d5c-9411-4e7f314a5d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:54.139984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-182005fd-8d72-43f9-8328-e1fc601fb300 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:54.142686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-182005fd-8d72-43f9-8328-e1fc601fb300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:54.252710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21690528-bf75-4d5c-9411-4e7f314a5d19 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:54.303430 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2731/5391] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:54.305385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182005fd-8d72-43f9-8328-e1fc601fb300 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:54.306756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-182005fd-8d72-43f9-8328-e1fc601fb300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:54.324881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182005fd-8d72-43f9-8328-e1fc601fb300 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:54.337670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a1333-35ce-4056-9706-4b3c3ccb97a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:54.338324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a1333-35ce-4056-9706-4b3c3ccb97a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:54.338680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a1333-35ce-4056-9706-4b3c3ccb97a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:54.339724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a1333-35ce-4056-9706-4b3c3ccb97a4 admin admin] PATH_INFO: `/v3/projects/8228a8c9bf064bb8a2b810feeeb4506a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:54.343284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a1333-35ce-4056-9706-4b3c3ccb97a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8228a8c9bf064bb8a2b810feeeb4506a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:54.345280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a1333-35ce-4056-9706-4b3c3ccb97a4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:54.349471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2cf57f-89c4-47e2-8edd-dc00311e173c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:54.350342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2cf57f-89c4-47e2-8edd-dc00311e173c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:54.350663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2cf57f-89c4-47e2-8edd-dc00311e173c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:54.350938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2cf57f-89c4-47e2-8edd-dc00311e173c admin admin] PATH_INFO: `/v3/users/5e4c6ff5395246b292941af15d2040c0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:54.352488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2cf57f-89c4-47e2-8edd-dc00311e173c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e4c6ff5395246b292941af15d2040c0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:54.355593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2cf57f-89c4-47e2-8edd-dc00311e173c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:54.443350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b74a1333-35ce-4056-9706-4b3c3ccb97a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8228a8c9bf064bb8a2b810feeeb4506a', 'request_id': 'req-b74a1333-35ce-4056-9706-4b3c3ccb97a4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:54.446625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:54.454681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b74a1333-35ce-4056-9706-4b3c3ccb97a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8228a8c9bf064bb8a2b810feeeb4506a', 'request_id': 'req-b74a1333-35ce-4056-9706-4b3c3ccb97a4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:54.466146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4a2cf57f-89c4-47e2-8edd-dc00311e173c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e4c6ff5395246b292941af15d2040c0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:54.467249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4a2cf57f-89c4-47e2-8edd-dc00311e173c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e4c6ff5395246b292941af15d2040c0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:54.467664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b74a1333-35ce-4056-9706-4b3c3ccb97a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8228a8c9bf064bb8a2b810feeeb4506a', 'request_id': 'req-b74a1333-35ce-4056-9706-4b3c3ccb97a4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:54.468261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4a2cf57f-89c4-47e2-8edd-dc00311e173c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e4c6ff5395246b292941af15d2040c0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:54.472655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:54.490054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4a2cf57f-89c4-47e2-8edd-dc00311e173c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e4c6ff5395246b292941af15d2040c0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:54.493894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b74a1333-35ce-4056-9706-4b3c3ccb97a4 admin admin] The token cache is being invalidate because project 8228a8c9bf064bb8a2b810feeeb4506a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:06:54.494568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b74a1333-35ce-4056-9706-4b3c3ccb97a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:54.495727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b74a1333-35ce-4056-9706-4b3c3ccb97a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:54.498842 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2732/5392] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:06:54 2022] DELETE /identity/v3/projects/8228a8c9bf064bb8a2b810feeeb4506a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:54.500161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a1333-35ce-4056-9706-4b3c3ccb97a4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:54.503199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b74a1333-35ce-4056-9706-4b3c3ccb97a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:54.529978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4a2cf57f-89c4-47e2-8edd-dc00311e173c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e4c6ff5395246b292941af15d2040c0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:54.552336 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2661/5393] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:06:54 2022] DELETE /identity/v3/users/5e4c6ff5395246b292941af15d2040c0 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:54.556838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5ff64-499f-409f-a1ce-ebbd9258af67 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:54.556838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5ff64-499f-409f-a1ce-ebbd9258af67 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:54.557114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5ff64-499f-409f-a1ce-ebbd9258af67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:54.600917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d9d5ff64-499f-409f-a1ce-ebbd9258af67 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:54.602140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d9d5ff64-499f-409f-a1ce-ebbd9258af67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:54.658388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a1333-35ce-4056-9706-4b3c3ccb97a4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:54.762834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51375a-aa74-4682-ac5d-a6dba1bf7f60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:54.763449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51375a-aa74-4682-ac5d-a6dba1bf7f60 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:54.763780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51375a-aa74-4682-ac5d-a6dba1bf7f60 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:54.764051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51375a-aa74-4682-ac5d-a6dba1bf7f60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:54.766424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fc51375a-aa74-4682-ac5d-a6dba1bf7f60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:54.769414 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2662/5394] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:54.780526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e08e5-5bf7-4686-966b-de8a5c669d2f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:54.780820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e08e5-5bf7-4686-966b-de8a5c669d2f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:54.781169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e08e5-5bf7-4686-966b-de8a5c669d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:54.793162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51375a-aa74-4682-ac5d-a6dba1bf7f60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:54.794721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51375a-aa74-4682-ac5d-a6dba1bf7f60 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:54.807752 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2733/5395] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:06:54.815870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f00e08e5-5bf7-4686-966b-de8a5c669d2f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:54.817710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f00e08e5-5bf7-4686-966b-de8a5c669d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:54.837393 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2663/5396] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:54.845752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51375a-aa74-4682-ac5d-a6dba1bf7f60 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:54.865524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51375a-aa74-4682-ac5d-a6dba1bf7f60 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:54.874162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f19dd-97c6-4f74-a4e4-91356ff4a212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:54.875210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f19dd-97c6-4f74-a4e4-91356ff4a212 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:54.875594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f19dd-97c6-4f74-a4e4-91356ff4a212 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:54.876010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f19dd-97c6-4f74-a4e4-91356ff4a212 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:54.876872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f19dd-97c6-4f74-a4e4-91356ff4a212 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:06:54.877319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f19dd-97c6-4f74-a4e4-91356ff4a212 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:54.879946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f19dd-97c6-4f74-a4e4-91356ff4a212 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:54.904442 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2734/5397] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:06:54.904615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e08e5-5bf7-4686-966b-de8a5c669d2f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:54.915868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e08e5-5bf7-4686-966b-de8a5c669d2f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:54.929060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb8d64-17eb-4058-994d-6a66ad7664bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:54.929694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb8d64-17eb-4058-994d-6a66ad7664bc admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:54.929975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb8d64-17eb-4058-994d-6a66ad7664bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:54.930249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb8d64-17eb-4058-994d-6a66ad7664bc admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:54.931045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb8d64-17eb-4058-994d-6a66ad7664bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:54.933240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb8d64-17eb-4058-994d-6a66ad7664bc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:54.971911 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2664/5398] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:06:54 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:06:54.974463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f19dd-97c6-4f74-a4e4-91356ff4a212 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:54.988882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f19dd-97c6-4f74-a4e4-91356ff4a212 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:55.099845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5af428-2e3e-470f-bda0-b34754a87ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:55.100683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5af428-2e3e-470f-bda0-b34754a87ba4 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:55.101142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5af428-2e3e-470f-bda0-b34754a87ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:55.101785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5af428-2e3e-470f-bda0-b34754a87ba4 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:55.102590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5af428-2e3e-470f-bda0-b34754a87ba4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:55.106370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5af428-2e3e-470f-bda0-b34754a87ba4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:55.106795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb8d64-17eb-4058-994d-6a66ad7664bc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:55.109292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-77fb8d64-17eb-4058-994d-6a66ad7664bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:55.129477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb8d64-17eb-4058-994d-6a66ad7664bc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:55.142248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8732f9-27b1-41f4-8960-51a62678d346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:55.143078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8732f9-27b1-41f4-8960-51a62678d346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:55.143695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8732f9-27b1-41f4-8960-51a62678d346 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:55.144738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8732f9-27b1-41f4-8960-51a62678d346 admin admin] PATH_INFO: `/v3/projects/4640d180e2174808b0351f33abf87e3b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:55.146662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8732f9-27b1-41f4-8960-51a62678d346 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4640d180e2174808b0351f33abf87e3b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:55.153581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8732f9-27b1-41f4-8960-51a62678d346 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:55.195405 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2735/5399] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:06:54 2022] POST /identity/v3/users => generated 286 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:55.196535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5af428-2e3e-470f-bda0-b34754a87ba4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:55.220491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5af428-2e3e-470f-bda0-b34754a87ba4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:55.230233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd4b62-c6fb-4cf9-a6c3-169acecdea83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:55.231251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd4b62-c6fb-4cf9-a6c3-169acecdea83 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:55.231732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd4b62-c6fb-4cf9-a6c3-169acecdea83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:55.232196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd4b62-c6fb-4cf9-a6c3-169acecdea83 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:55.233082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd4b62-c6fb-4cf9-a6c3-169acecdea83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:55.235067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd4b62-c6fb-4cf9-a6c3-169acecdea83 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:55.270307 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2736/5400] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:55.271546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd4b62-c6fb-4cf9-a6c3-169acecdea83 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:55.281245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:55.283311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8f8732f9-27b1-41f4-8960-51a62678d346 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4640d180e2174808b0351f33abf87e3b', 'request_id': 'req-8f8732f9-27b1-41f4-8960-51a62678d346'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:55.287379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd4b62-c6fb-4cf9-a6c3-169acecdea83 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:55.322696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8f8732f9-27b1-41f4-8960-51a62678d346 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4640d180e2174808b0351f33abf87e3b', 'request_id': 'req-8f8732f9-27b1-41f4-8960-51a62678d346'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:55.336720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8f8732f9-27b1-41f4-8960-51a62678d346 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4640d180e2174808b0351f33abf87e3b', 'request_id': 'req-8f8732f9-27b1-41f4-8960-51a62678d346'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:55.359998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8f8732f9-27b1-41f4-8960-51a62678d346 admin admin] The token cache is being invalidate because project 4640d180e2174808b0351f33abf87e3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:06:55.360474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8f8732f9-27b1-41f4-8960-51a62678d346 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:55.363095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8f8732f9-27b1-41f4-8960-51a62678d346 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:06:55.366059 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2665/5401] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:06:55 2022] DELETE /identity/v3/projects/4640d180e2174808b0351f33abf87e3b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:06:55.372476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31c918-1c59-42bd-b948-20c6b24f22c8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:55.373487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31c918-1c59-42bd-b948-20c6b24f22c8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:55.373777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31c918-1c59-42bd-b948-20c6b24f22c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:55.423134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5e31c918-1c59-42bd-b948-20c6b24f22c8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:55.425559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5e31c918-1c59-42bd-b948-20c6b24f22c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:55.439332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffd09b-7aea-4cda-8328-fe1b01c57dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:55.440105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffd09b-7aea-4cda-8328-fe1b01c57dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:55.440386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffd09b-7aea-4cda-8328-fe1b01c57dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:55.441398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffd09b-7aea-4cda-8328-fe1b01c57dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:55.442612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ffd09b-7aea-4cda-8328-fe1b01c57dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:55.447502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ffd09b-7aea-4cda-8328-fe1b01c57dc9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:55.451616 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2666/5402] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:55.459420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c60af5-2dd5-459f-b135-62e7178ce1d1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:55.459708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c60af5-2dd5-459f-b135-62e7178ce1d1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:55.459984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c60af5-2dd5-459f-b135-62e7178ce1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:55.471436 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2737/5403] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:55.476900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffd09b-7aea-4cda-8328-fe1b01c57dc9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:55.482230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a8ffd09b-7aea-4cda-8328-fe1b01c57dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:55.513878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e8c60af5-2dd5-459f-b135-62e7178ce1d1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:55.517131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e8c60af5-2dd5-459f-b135-62e7178ce1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:55.520557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffd09b-7aea-4cda-8328-fe1b01c57dc9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:55.529149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ddebeb-7883-4303-a4ee-7b60bc2f5d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:55.529831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddebeb-7883-4303-a4ee-7b60bc2f5d4d admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:55.530125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddebeb-7883-4303-a4ee-7b60bc2f5d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:55.530383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddebeb-7883-4303-a4ee-7b60bc2f5d4d admin admin] PATH_INFO: `/v3/projects/1e11cf8be9a34cd899e7162801ae2e4e/users/15cb4da6a69d44dbb76473dbceb0e503/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:55.538526 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2667/5404] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:06:55.543194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c60af5-2dd5-459f-b135-62e7178ce1d1 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:55.565867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c60af5-2dd5-459f-b135-62e7178ce1d1 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:55.571059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ddebeb-7883-4303-a4ee-7b60bc2f5d4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e11cf8be9a34cd899e7162801ae2e4e, user_id=15cb4da6a69d44dbb76473dbceb0e503, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:55.578371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ddebeb-7883-4303-a4ee-7b60bc2f5d4d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:55.580932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faf9fe-260b-4a29-b502-45102f5749ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:55.581569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faf9fe-260b-4a29-b502-45102f5749ea admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:55.581843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faf9fe-260b-4a29-b502-45102f5749ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:55.582112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faf9fe-260b-4a29-b502-45102f5749ea admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:55.582796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3faf9fe-260b-4a29-b502-45102f5749ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:06:55.583103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3faf9fe-260b-4a29-b502-45102f5749ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:55.585956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3faf9fe-260b-4a29-b502-45102f5749ea admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:55.591900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:55.612540 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2738/5405] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:06:55 2022] PUT /identity/v3/projects/1e11cf8be9a34cd899e7162801ae2e4e/users/15cb4da6a69d44dbb76473dbceb0e503/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:55.625336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327bdf3-2732-460d-900d-807cf2278aac None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:55.625822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327bdf3-2732-460d-900d-807cf2278aac None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:55.626219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327bdf3-2732-460d-900d-807cf2278aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:55.629241 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2668/5406] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:06:55.633340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faf9fe-260b-4a29-b502-45102f5749ea admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:55.653104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faf9fe-260b-4a29-b502-45102f5749ea admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:55.684240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a327bdf3-2732-460d-900d-807cf2278aac None None] MFA Rules not processed for user `15cb4da6a69d44dbb76473dbceb0e503`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:55.685654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a327bdf3-2732-460d-900d-807cf2278aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:55.765627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66635d25-9cab-4df5-98b7-3bac5bd1d2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:55.766317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66635d25-9cab-4df5-98b7-3bac5bd1d2f3 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:55.767383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66635d25-9cab-4df5-98b7-3bac5bd1d2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:55.767659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66635d25-9cab-4df5-98b7-3bac5bd1d2f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:55.775853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66635d25-9cab-4df5-98b7-3bac5bd1d2f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:55.779099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66635d25-9cab-4df5-98b7-3bac5bd1d2f3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:55.815471 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2739/5407] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:55.820642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327bdf3-2732-460d-900d-807cf2278aac None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:55.846471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327bdf3-2732-460d-900d-807cf2278aac None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:55.849253 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2669/5408] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:06:55 2022] POST /identity/v3/projects => generated 363 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:55.854475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66635d25-9cab-4df5-98b7-3bac5bd1d2f3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:56.005795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46ffb1-5bfd-46b7-bd2d-3e34caf0f451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:56.006430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46ffb1-5bfd-46b7-bd2d-3e34caf0f451 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:56.008223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46ffb1-5bfd-46b7-bd2d-3e34caf0f451 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:56.008499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46ffb1-5bfd-46b7-bd2d-3e34caf0f451 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:56.010949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46ffb1-5bfd-46b7-bd2d-3e34caf0f451 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:56.014410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46ffb1-5bfd-46b7-bd2d-3e34caf0f451 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:56.032918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66635d25-9cab-4df5-98b7-3bac5bd1d2f3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:56.047719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:56.048275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea4b75-3b83-4166-a113-fda34131cff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:56.049069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea4b75-3b83-4166-a113-fda34131cff3 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:56.049553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea4b75-3b83-4166-a113-fda34131cff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:56.050011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea4b75-3b83-4166-a113-fda34131cff3 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:56.051533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea4b75-3b83-4166-a113-fda34131cff3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:56.056941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea4b75-3b83-4166-a113-fda34131cff3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:56.096521 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2740/5409] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:06:55 2022] POST /identity/v3/projects => generated 373 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:56.097384 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2670/5410] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:06:55 2022] POST /identity/v3/users => generated 282 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:56.101001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46ffb1-5bfd-46b7-bd2d-3e34caf0f451 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:56.103573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea4b75-3b83-4166-a113-fda34131cff3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:56.131180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea4b75-3b83-4166-a113-fda34131cff3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:56.191772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46ffb1-5bfd-46b7-bd2d-3e34caf0f451 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:56.199754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7e85f-6dfe-4312-b78c-c316c29681e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:56.201616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7e85f-6dfe-4312-b78c-c316c29681e4 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:56.202138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7e85f-6dfe-4312-b78c-c316c29681e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:56.202775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7e85f-6dfe-4312-b78c-c316c29681e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:56.204259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7e85f-6dfe-4312-b78c-c316c29681e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:56.207504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7e85f-6dfe-4312-b78c-c316c29681e4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:56.223805 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2741/5411] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:56.229362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7e85f-6dfe-4312-b78c-c316c29681e4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:56.246564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7e85f-6dfe-4312-b78c-c316c29681e4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:56.267671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1eae3-8469-4525-a9fc-8764994b7b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:56.268324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1eae3-8469-4525-a9fc-8764994b7b6f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:56.268609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1eae3-8469-4525-a9fc-8764994b7b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:56.268886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1eae3-8469-4525-a9fc-8764994b7b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:56.271027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1eae3-8469-4525-a9fc-8764994b7b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:56.273441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1eae3-8469-4525-a9fc-8764994b7b6f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:56.286150 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2742/5412] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:56.286369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27f486-9170-47c7-aa87-528b9c0b6ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:56.286987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27f486-9170-47c7-aa87-528b9c0b6ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:56.287276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27f486-9170-47c7-aa87-528b9c0b6ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:56.287547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27f486-9170-47c7-aa87-528b9c0b6ca5 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:56.288172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f27f486-9170-47c7-aa87-528b9c0b6ca5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:56.290194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1eae3-8469-4525-a9fc-8764994b7b6f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:56.290535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f27f486-9170-47c7-aa87-528b9c0b6ca5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:56.305264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1eae3-8469-4525-a9fc-8764994b7b6f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:56.314819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b2c13-2788-406c-becd-c9bfd74fd10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:56.316062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b2c13-2788-406c-becd-c9bfd74fd10d admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:56.316657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b2c13-2788-406c-becd-c9bfd74fd10d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:56.317244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b2c13-2788-406c-becd-c9bfd74fd10d admin admin] PATH_INFO: `/v3/projects/021ea8b4eff84168bc344a6ec2fbb998/users/2318c63cfee34ed8acfe691a7e38a8ef/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:56.326588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:56.343267 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2671/5413] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:06:56 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:56.347082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27f486-9170-47c7-aa87-528b9c0b6ca5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:56.351436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b2c13-2788-406c-becd-c9bfd74fd10d admin admin] RBAC: Authorizing `identity:create_grant(project_id=021ea8b4eff84168bc344a6ec2fbb998, user_id=2318c63cfee34ed8acfe691a7e38a8ef, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:56.355669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b2c13-2788-406c-becd-c9bfd74fd10d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:56.368372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27f486-9170-47c7-aa87-528b9c0b6ca5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:56.385161 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2743/5414] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:06:56 2022] PUT /identity/v3/projects/021ea8b4eff84168bc344a6ec2fbb998/users/2318c63cfee34ed8acfe691a7e38a8ef/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:56.404046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579757d-31f2-4172-8048-bf995c9522a1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:56.407149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579757d-31f2-4172-8048-bf995c9522a1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:56.407757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579757d-31f2-4172-8048-bf995c9522a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:56.436305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5579757d-31f2-4172-8048-bf995c9522a1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:56.438271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5579757d-31f2-4172-8048-bf995c9522a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:56.523207 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2744/5415] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:06:56.542334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb415ff-d914-492b-9970-211350925c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:56.543012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb415ff-d914-492b-9970-211350925c3f admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:56.543352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb415ff-d914-492b-9970-211350925c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:56.543556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb415ff-d914-492b-9970-211350925c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:56.544205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb415ff-d914-492b-9970-211350925c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:56.545913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb415ff-d914-492b-9970-211350925c3f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:56.561701 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2672/5416] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:56.566276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579757d-31f2-4172-8048-bf995c9522a1 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:56.578952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb415ff-d914-492b-9970-211350925c3f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:56.581060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8cb415ff-d914-492b-9970-211350925c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:56.582161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579757d-31f2-4172-8048-bf995c9522a1 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:56.588511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fff1bd-76a0-430c-9f32-705986562745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:56.598788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fff1bd-76a0-430c-9f32-705986562745 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:56.598788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fff1bd-76a0-430c-9f32-705986562745 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:56.598788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fff1bd-76a0-430c-9f32-705986562745 admin admin] PATH_INFO: `/v3/projects/c6421057643c4ab3b1dbe33e2242a763/users/70d479ffa4c74b66bdb97af7335fdc4e/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:56.612675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fff1bd-76a0-430c-9f32-705986562745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6421057643c4ab3b1dbe33e2242a763, user_id=70d479ffa4c74b66bdb97af7335fdc4e, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:56.616173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fff1bd-76a0-430c-9f32-705986562745 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:56.655794 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2745/5417] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:06:56 2022] PUT /identity/v3/projects/c6421057643c4ab3b1dbe33e2242a763/users/70d479ffa4c74b66bdb97af7335fdc4e/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:56.660240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fff1bd-76a0-430c-9f32-705986562745 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:56.680466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb415ff-d914-492b-9970-211350925c3f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:56.743394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:56.802448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc6196-1658-4c37-9e1c-772bae0d6628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:56.802979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc6196-1658-4c37-9e1c-772bae0d6628 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:56.803260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc6196-1658-4c37-9e1c-772bae0d6628 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:56.803788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc6196-1658-4c37-9e1c-772bae0d6628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:56.809244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fff1bd-76a0-430c-9f32-705986562745 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:56.821853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008d949-2c10-4358-89d2-52041a3cb5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:56.822245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008d949-2c10-4358-89d2-52041a3cb5d0 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:56.822547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008d949-2c10-4358-89d2-52041a3cb5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:56.822827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008d949-2c10-4358-89d2-52041a3cb5d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:56.824715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc6196-1658-4c37-9e1c-772bae0d6628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:56.826180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc6196-1658-4c37-9e1c-772bae0d6628 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:56.828393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4008d949-2c10-4358-89d2-52041a3cb5d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:56.830013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4008d949-2c10-4358-89d2-52041a3cb5d0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:56.854465 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2673/5418] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:06:56.864768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc6196-1658-4c37-9e1c-772bae0d6628 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:56.865237 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2746/5419] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:06:56.874352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc6196-1658-4c37-9e1c-772bae0d6628 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:56.880901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2a25c-a914-4c88-a344-c3f9a1132b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:56.881624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2a25c-a914-4c88-a344-c3f9a1132b7c admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:56.881911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2a25c-a914-4c88-a344-c3f9a1132b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:56.882179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2a25c-a914-4c88-a344-c3f9a1132b7c admin admin] PATH_INFO: `/v3/projects/c6421057643c4ab3b1dbe33e2242a763/users/70d479ffa4c74b66bdb97af7335fdc4e/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:56.884435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2a25c-a914-4c88-a344-c3f9a1132b7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6421057643c4ab3b1dbe33e2242a763, user_id=70d479ffa4c74b66bdb97af7335fdc4e, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:56.889877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2a25c-a914-4c88-a344-c3f9a1132b7c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:56.913119 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2674/5420] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:06:56 2022] PUT /identity/v3/projects/c6421057643c4ab3b1dbe33e2242a763/users/70d479ffa4c74b66bdb97af7335fdc4e/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:06:56.915049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616e474-9963-4789-8069-2b2436568475 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:56.915424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616e474-9963-4789-8069-2b2436568475 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:56.915700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616e474-9963-4789-8069-2b2436568475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:56.983199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7616e474-9963-4789-8069-2b2436568475 None None] MFA Rules not processed for user `70d479ffa4c74b66bdb97af7335fdc4e`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:56.985248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7616e474-9963-4789-8069-2b2436568475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:57.184307 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2747/5421] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:06:57.189610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2a25c-a914-4c88-a344-c3f9a1132b7c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:57.207734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2a25c-a914-4c88-a344-c3f9a1132b7c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:57.216767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ed508-6655-4605-a623-8d2224c19eba tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70d479ffa4c74b66bdb97af7335fdc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6421057643c4ab3b1dbe33e2242a763', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:57.217741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ed508-6655-4605-a623-8d2224c19eba tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:57.218209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ed508-6655-4605-a623-8d2224c19eba tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:57.218690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ed508-6655-4605-a623-8d2224c19eba tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:57.220127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ed508-6655-4605-a623-8d2224c19eba tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:06:57.220781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ed508-6655-4605-a623-8d2224c19eba tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:57.222682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ed508-6655-4605-a623-8d2224c19eba tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:57.256184 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2675/5422] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:06:57.258905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616e474-9963-4789-8069-2b2436568475 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:57.274304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616e474-9963-4789-8069-2b2436568475 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:57.286565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a236c4e-1cdf-43b3-91d5-408ce2b37e31 tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70d479ffa4c74b66bdb97af7335fdc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6421057643c4ab3b1dbe33e2242a763', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:57.287523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a236c4e-1cdf-43b3-91d5-408ce2b37e31 tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:57.287955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a236c4e-1cdf-43b3-91d5-408ce2b37e31 tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:57.288377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a236c4e-1cdf-43b3-91d5-408ce2b37e31 tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:57.289196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a236c4e-1cdf-43b3-91d5-408ce2b37e31 tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:57.291541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a236c4e-1cdf-43b3-91d5-408ce2b37e31 tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:57.346187 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2748/5423] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:06:57 2022] POST /identity/v3/projects => generated 349 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:06:57.347829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ed508-6655-4605-a623-8d2224c19eba tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:57.358868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ed508-6655-4605-a623-8d2224c19eba tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:57.397100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:57.451372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2027577-ebc8-4bbb-b5e3-f6478c0326fe tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70d479ffa4c74b66bdb97af7335fdc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6421057643c4ab3b1dbe33e2242a763', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:57.452057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2027577-ebc8-4bbb-b5e3-f6478c0326fe tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:57.452599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2027577-ebc8-4bbb-b5e3-f6478c0326fe tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:57.452712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2027577-ebc8-4bbb-b5e3-f6478c0326fe tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:57.454749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2027577-ebc8-4bbb-b5e3-f6478c0326fe tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:06:57.455169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2027577-ebc8-4bbb-b5e3-f6478c0326fe tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:57.457104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2027577-ebc8-4bbb-b5e3-f6478c0326fe tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:57.485159 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2676/5424] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:06:57.532213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a236c4e-1cdf-43b3-91d5-408ce2b37e31 tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:06:57.685479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a236c4e-1cdf-43b3-91d5-408ce2b37e31 tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:06:57.696798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c83be-adb6-4015-9ba6-4b740726f89c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:06:57.697375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c83be-adb6-4015-9ba6-4b740726f89c service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:57.699087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c83be-adb6-4015-9ba6-4b740726f89c service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:57.699369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c83be-adb6-4015-9ba6-4b740726f89c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:57.714921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c83be-adb6-4015-9ba6-4b740726f89c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:06:57.716436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c83be-adb6-4015-9ba6-4b740726f89c service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:06:57.753513 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2749/5425] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:06:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:06:59.324999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0d8d8-4864-4389-bc71-a01c38a35c5b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:06:59.325854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0d8d8-4864-4389-bc71-a01c38a35c5b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:06:59.326324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0d8d8-4864-4389-bc71-a01c38a35c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:06:59.359675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-36e0d8d8-4864-4389-bc71-a01c38a35c5b None None] MFA Rules not processed for user `6d63c633112e49d091421c84cea3422b`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:06:59.363796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-36e0d8d8-4864-4389-bc71-a01c38a35c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:06:59.454654 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2677/5426] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:06:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:03.881039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c83be-adb6-4015-9ba6-4b740726f89c service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:03.892532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c83be-adb6-4015-9ba6-4b740726f89c service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:03.903297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c42b6-d285-462b-885b-728bbab71a98 tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70d479ffa4c74b66bdb97af7335fdc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6421057643c4ab3b1dbe33e2242a763', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:03.904084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c42b6-d285-462b-885b-728bbab71a98 tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:03.904476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c42b6-d285-462b-885b-728bbab71a98 tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:03.904856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c42b6-d285-462b-885b-728bbab71a98 tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] PATH_INFO: `/v3/projects/c8cf5de478cd47db8ea156275be9175a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:03.907560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c42b6-d285-462b-885b-728bbab71a98 tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c8cf5de478cd47db8ea156275be9175a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:03.911393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c42b6-d285-462b-885b-728bbab71a98 tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:03.966475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:03.990216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac1c42b6-d285-462b-885b-728bbab71a98 tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8cf5de478cd47db8ea156275be9175a', 'request_id': 'req-ac1c42b6-d285-462b-885b-728bbab71a98'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:03.998588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac1c42b6-d285-462b-885b-728bbab71a98 tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8cf5de478cd47db8ea156275be9175a', 'request_id': 'req-ac1c42b6-d285-462b-885b-728bbab71a98'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:04.006591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac1c42b6-d285-462b-885b-728bbab71a98 tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8cf5de478cd47db8ea156275be9175a', 'request_id': 'req-ac1c42b6-d285-462b-885b-728bbab71a98'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:04.051737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac1c42b6-d285-462b-885b-728bbab71a98 tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] The token cache is being invalidate because project c8cf5de478cd47db8ea156275be9175a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:07:04.052055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac1c42b6-d285-462b-885b-728bbab71a98 tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:04.052891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac1c42b6-d285-462b-885b-728bbab71a98 tempest-QuotasNegativeTest-1125322585 tempest-QuotasNegativeTest-1125322585-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:04.056190 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2750/5427] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:07:03 2022] DELETE /identity/v3/projects/c8cf5de478cd47db8ea156275be9175a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:07:04.061585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0d8d8-4864-4389-bc71-a01c38a35c5b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:04.176470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0d8d8-4864-4389-bc71-a01c38a35c5b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:04.337054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6560c28-0b9d-4925-9d15-86f46ecf7a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:04.338097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6560c28-0b9d-4925-9d15-86f46ecf7a68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:04.339800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6560c28-0b9d-4925-9d15-86f46ecf7a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:04.340228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6560c28-0b9d-4925-9d15-86f46ecf7a68 admin admin] PATH_INFO: `/v3/users/15cb4da6a69d44dbb76473dbceb0e503` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:04.344105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6560c28-0b9d-4925-9d15-86f46ecf7a68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15cb4da6a69d44dbb76473dbceb0e503)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:04.347917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6560c28-0b9d-4925-9d15-86f46ecf7a68 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:04.398460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:04.483200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a6560c28-0b9d-4925-9d15-86f46ecf7a68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15cb4da6a69d44dbb76473dbceb0e503'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:04.485567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a6560c28-0b9d-4925-9d15-86f46ecf7a68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15cb4da6a69d44dbb76473dbceb0e503'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:04.488460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a6560c28-0b9d-4925-9d15-86f46ecf7a68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15cb4da6a69d44dbb76473dbceb0e503'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:04.519948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a6560c28-0b9d-4925-9d15-86f46ecf7a68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15cb4da6a69d44dbb76473dbceb0e503'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:04.548562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a6560c28-0b9d-4925-9d15-86f46ecf7a68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15cb4da6a69d44dbb76473dbceb0e503'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:04.579167 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2678/5428] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:07:04 2022] DELETE /identity/v3/users/15cb4da6a69d44dbb76473dbceb0e503 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:07:04.584307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d505985-27c0-4e01-bfdf-54563193bb6d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:04.584307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d505985-27c0-4e01-bfdf-54563193bb6d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:04.584307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d505985-27c0-4e01-bfdf-54563193bb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:04.625439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7d505985-27c0-4e01-bfdf-54563193bb6d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:04.627877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7d505985-27c0-4e01-bfdf-54563193bb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:04.765579 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2751/5429] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:04.804002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6560c28-0b9d-4925-9d15-86f46ecf7a68 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:04.806715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a6560c28-0b9d-4925-9d15-86f46ecf7a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:05.034577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6560c28-0b9d-4925-9d15-86f46ecf7a68 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:05.040041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f139b4-9018-4793-8169-9f358c154877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:05.040904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f139b4-9018-4793-8169-9f358c154877 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:05.041339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f139b4-9018-4793-8169-9f358c154877 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:05.042400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f139b4-9018-4793-8169-9f358c154877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:05.069635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f139b4-9018-4793-8169-9f358c154877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:05.071294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f139b4-9018-4793-8169-9f358c154877 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:05.102535 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2679/5430] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:07:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:07:07.483544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d505985-27c0-4e01-bfdf-54563193bb6d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:07.484976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7d505985-27c0-4e01-bfdf-54563193bb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:07.501495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d505985-27c0-4e01-bfdf-54563193bb6d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:07.529714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904b9bf-7cb0-4bec-8038-a13f5bb2f6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:07.530370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904b9bf-7cb0-4bec-8038-a13f5bb2f6c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:07.530705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904b9bf-7cb0-4bec-8038-a13f5bb2f6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:07.530979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904b9bf-7cb0-4bec-8038-a13f5bb2f6c2 admin admin] PATH_INFO: `/v3/projects/1e11cf8be9a34cd899e7162801ae2e4e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:07.532914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904b9bf-7cb0-4bec-8038-a13f5bb2f6c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e11cf8be9a34cd899e7162801ae2e4e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:07.535203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904b9bf-7cb0-4bec-8038-a13f5bb2f6c2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:07.645273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e904b9bf-7cb0-4bec-8038-a13f5bb2f6c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e11cf8be9a34cd899e7162801ae2e4e', 'request_id': 'req-e904b9bf-7cb0-4bec-8038-a13f5bb2f6c2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:07.660934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e904b9bf-7cb0-4bec-8038-a13f5bb2f6c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e11cf8be9a34cd899e7162801ae2e4e', 'request_id': 'req-e904b9bf-7cb0-4bec-8038-a13f5bb2f6c2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:07.677287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e904b9bf-7cb0-4bec-8038-a13f5bb2f6c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e11cf8be9a34cd899e7162801ae2e4e', 'request_id': 'req-e904b9bf-7cb0-4bec-8038-a13f5bb2f6c2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:07.699219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e904b9bf-7cb0-4bec-8038-a13f5bb2f6c2 admin admin] The token cache is being invalidate because project 1e11cf8be9a34cd899e7162801ae2e4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:07:07.699535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e904b9bf-7cb0-4bec-8038-a13f5bb2f6c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:07.707072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e904b9bf-7cb0-4bec-8038-a13f5bb2f6c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:07.709143 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2752/5431] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:07:07 2022] DELETE /identity/v3/projects/1e11cf8be9a34cd899e7162801ae2e4e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:07:07.712981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f139b4-9018-4793-8169-9f358c154877 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:07.734472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f139b4-9018-4793-8169-9f358c154877 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:07.846818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73e50c-63c1-4a5d-a639-746305dc7ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:07.847797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73e50c-63c1-4a5d-a639-746305dc7ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:07.848331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73e50c-63c1-4a5d-a639-746305dc7ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:07.848812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73e50c-63c1-4a5d-a639-746305dc7ea2 admin admin] PATH_INFO: `/v3/users/70d479ffa4c74b66bdb97af7335fdc4e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:07.850581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73e50c-63c1-4a5d-a639-746305dc7ea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70d479ffa4c74b66bdb97af7335fdc4e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:07.854087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73e50c-63c1-4a5d-a639-746305dc7ea2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:07.967568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:07.989233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3c73e50c-63c1-4a5d-a639-746305dc7ea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70d479ffa4c74b66bdb97af7335fdc4e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:07.990675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3c73e50c-63c1-4a5d-a639-746305dc7ea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70d479ffa4c74b66bdb97af7335fdc4e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:07.993052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3c73e50c-63c1-4a5d-a639-746305dc7ea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70d479ffa4c74b66bdb97af7335fdc4e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:08.035441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3c73e50c-63c1-4a5d-a639-746305dc7ea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70d479ffa4c74b66bdb97af7335fdc4e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:08.069078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3c73e50c-63c1-4a5d-a639-746305dc7ea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70d479ffa4c74b66bdb97af7335fdc4e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:08.109770 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2680/5432] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:07:07 2022] DELETE /identity/v3/users/70d479ffa4c74b66bdb97af7335fdc4e => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:07:10.144682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904b9bf-7cb0-4bec-8038-a13f5bb2f6c2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:10.146328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e904b9bf-7cb0-4bec-8038-a13f5bb2f6c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:10.179196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904b9bf-7cb0-4bec-8038-a13f5bb2f6c2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:10.253906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:10.346944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdba8f-6f1f-4060-b8b6-25bd255b5671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:10.348460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdba8f-6f1f-4060-b8b6-25bd255b5671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:10.348993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdba8f-6f1f-4060-b8b6-25bd255b5671 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:10.349486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdba8f-6f1f-4060-b8b6-25bd255b5671 admin admin] PATH_INFO: `/v3/projects/c6421057643c4ab3b1dbe33e2242a763` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:10.353588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdba8f-6f1f-4060-b8b6-25bd255b5671 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6421057643c4ab3b1dbe33e2242a763)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:10.361926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdba8f-6f1f-4060-b8b6-25bd255b5671 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:10.517313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-95fdba8f-6f1f-4060-b8b6-25bd255b5671 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6421057643c4ab3b1dbe33e2242a763', 'request_id': 'req-95fdba8f-6f1f-4060-b8b6-25bd255b5671'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:10.537660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-95fdba8f-6f1f-4060-b8b6-25bd255b5671 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6421057643c4ab3b1dbe33e2242a763', 'request_id': 'req-95fdba8f-6f1f-4060-b8b6-25bd255b5671'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:10.564063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-95fdba8f-6f1f-4060-b8b6-25bd255b5671 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6421057643c4ab3b1dbe33e2242a763', 'request_id': 'req-95fdba8f-6f1f-4060-b8b6-25bd255b5671'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:10.583579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-95fdba8f-6f1f-4060-b8b6-25bd255b5671 admin admin] The token cache is being invalidate because project c6421057643c4ab3b1dbe33e2242a763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:07:10.584224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-95fdba8f-6f1f-4060-b8b6-25bd255b5671 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:10.585347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-95fdba8f-6f1f-4060-b8b6-25bd255b5671 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:10.587850 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2753/5433] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:07:10 2022] DELETE /identity/v3/projects/c6421057643c4ab3b1dbe33e2242a763 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:07:10.593153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c6226-836a-4dda-a931-f7dbc7021b68 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:10.593610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c6226-836a-4dda-a931-f7dbc7021b68 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:10.593702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c6226-836a-4dda-a931-f7dbc7021b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:10.665629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e91c6226-836a-4dda-a931-f7dbc7021b68 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:10.668214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e91c6226-836a-4dda-a931-f7dbc7021b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:10.804566 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2681/5434] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:10.813160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625bd33-894f-46a3-bce8-cadafb3ddfd3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:10.813643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625bd33-894f-46a3-bce8-cadafb3ddfd3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:10.814095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625bd33-894f-46a3-bce8-cadafb3ddfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:10.902073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d625bd33-894f-46a3-bce8-cadafb3ddfd3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:10.903809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d625bd33-894f-46a3-bce8-cadafb3ddfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:10.939522 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2754/5435] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:11.040319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c6226-836a-4dda-a931-f7dbc7021b68 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:11.068032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c6226-836a-4dda-a931-f7dbc7021b68 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:11.078242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc095f8e-4ccd-4c3d-80b9-8f1e76462bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:11.079165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc095f8e-4ccd-4c3d-80b9-8f1e76462bcd admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:11.079627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc095f8e-4ccd-4c3d-80b9-8f1e76462bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:11.080057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc095f8e-4ccd-4c3d-80b9-8f1e76462bcd admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:11.080883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc095f8e-4ccd-4c3d-80b9-8f1e76462bcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:07:11.081335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc095f8e-4ccd-4c3d-80b9-8f1e76462bcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:11.083154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc095f8e-4ccd-4c3d-80b9-8f1e76462bcd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:11.117467 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2682/5436] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:07:11.127506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d625bd33-894f-46a3-bce8-cadafb3ddfd3 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:11.147357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d625bd33-894f-46a3-bce8-cadafb3ddfd3 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:11.161132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc559d1-8be8-41d7-92c7-74ab3e8f86b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:11.161867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc559d1-8be8-41d7-92c7-74ab3e8f86b2 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:11.162200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc559d1-8be8-41d7-92c7-74ab3e8f86b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:11.162536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc559d1-8be8-41d7-92c7-74ab3e8f86b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:11.163485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc559d1-8be8-41d7-92c7-74ab3e8f86b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:11.165681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc559d1-8be8-41d7-92c7-74ab3e8f86b2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:11.232180 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2755/5437] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:07:11 2022] POST /identity/v3/projects => generated 397 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:07:11.236195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc095f8e-4ccd-4c3d-80b9-8f1e76462bcd admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:11.252319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc095f8e-4ccd-4c3d-80b9-8f1e76462bcd admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:11.398394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a9118-5020-4e98-8ffa-a368fe0d68da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:11.400188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a9118-5020-4e98-8ffa-a368fe0d68da admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:11.400746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a9118-5020-4e98-8ffa-a368fe0d68da admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:11.402472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a9118-5020-4e98-8ffa-a368fe0d68da admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:11.403438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a9118-5020-4e98-8ffa-a368fe0d68da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:11.409607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a9118-5020-4e98-8ffa-a368fe0d68da admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:11.473429 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2683/5438] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:07:11 2022] POST /identity/v3/users => generated 299 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:07:11.477487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc559d1-8be8-41d7-92c7-74ab3e8f86b2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:11.492620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc559d1-8be8-41d7-92c7-74ab3e8f86b2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:11.501879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2a4f8-1ce9-4169-9779-ecaea0774b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:11.502758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2a4f8-1ce9-4169-9779-ecaea0774b2c admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:11.503210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2a4f8-1ce9-4169-9779-ecaea0774b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:11.503637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2a4f8-1ce9-4169-9779-ecaea0774b2c admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:11.504443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2a4f8-1ce9-4169-9779-ecaea0774b2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:11.506210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2a4f8-1ce9-4169-9779-ecaea0774b2c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:11.523275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3363b-aa90-42f9-9069-7a057a74d33e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:11.523681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3363b-aa90-42f9-9069-7a057a74d33e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:11.524041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3363b-aa90-42f9-9069-7a057a74d33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:11.545115 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2756/5439] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:07:11.552177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2a4f8-1ce9-4169-9779-ecaea0774b2c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:11.573902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1bc3363b-aa90-42f9-9069-7a057a74d33e None None] MFA Rules not processed for user `6d63c633112e49d091421c84cea3422b`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:11.576027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1bc3363b-aa90-42f9-9069-7a057a74d33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:11.579323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2a4f8-1ce9-4169-9779-ecaea0774b2c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:11.586581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384684d9-58fe-4a58-bcba-64ecec20f44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:11.587497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384684d9-58fe-4a58-bcba-64ecec20f44b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:11.588037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384684d9-58fe-4a58-bcba-64ecec20f44b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:11.589059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384684d9-58fe-4a58-bcba-64ecec20f44b admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:11.590506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384684d9-58fe-4a58-bcba-64ecec20f44b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:11.592690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384684d9-58fe-4a58-bcba-64ecec20f44b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:11.606799 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2757/5440] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:07:11.612045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384684d9-58fe-4a58-bcba-64ecec20f44b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:11.636718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384684d9-58fe-4a58-bcba-64ecec20f44b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:11.636996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:11.644401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78071941-c581-4b69-8997-e8e32f75c69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:11.646228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78071941-c581-4b69-8997-e8e32f75c69c admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:11.646625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78071941-c581-4b69-8997-e8e32f75c69c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:11.647957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78071941-c581-4b69-8997-e8e32f75c69c admin admin] PATH_INFO: `/v3/projects/c34b7d9e02bb4583b6769768a1265d1c/users/e3f8e2ca415c4a309b036d8406fcc921/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:11.703152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78071941-c581-4b69-8997-e8e32f75c69c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c34b7d9e02bb4583b6769768a1265d1c, user_id=e3f8e2ca415c4a309b036d8406fcc921, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:11.715548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78071941-c581-4b69-8997-e8e32f75c69c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:11.749163 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2758/5441] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:07:11 2022] PUT /identity/v3/projects/c34b7d9e02bb4583b6769768a1265d1c/users/e3f8e2ca415c4a309b036d8406fcc921/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:07:11.750774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352be52c-ed8b-4a04-8404-cad8dca1f1a1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:11.760062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352be52c-ed8b-4a04-8404-cad8dca1f1a1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:11.761745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352be52c-ed8b-4a04-8404-cad8dca1f1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:11.770696 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2684/5442] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:07:11.779405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c6602-2902-476c-a466-48895b886f4e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:11.780521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c6602-2902-476c-a466-48895b886f4e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:11.781646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c6602-2902-476c-a466-48895b886f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:11.847966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-352be52c-ed8b-4a04-8404-cad8dca1f1a1 None None] MFA Rules not processed for user `e7e3bc6d5e45497bb876d5cbcec61835`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:11.851296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-352be52c-ed8b-4a04-8404-cad8dca1f1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:11.908415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ac7c6602-2902-476c-a466-48895b886f4e None None] MFA Rules not processed for user `e3f8e2ca415c4a309b036d8406fcc921`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:11.910240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ac7c6602-2902-476c-a466-48895b886f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:11.933736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:12.040086 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2759/5443] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:11 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:12.040802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352be52c-ed8b-4a04-8404-cad8dca1f1a1 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:12.042881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-352be52c-ed8b-4a04-8404-cad8dca1f1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:12.180168 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2685/5444] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:11 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:12.266841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352be52c-ed8b-4a04-8404-cad8dca1f1a1 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:12.272872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d184cb3-8141-4def-826a-663210b88c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:12.273787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d184cb3-8141-4def-826a-663210b88c1a admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:12.274325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d184cb3-8141-4def-826a-663210b88c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:12.274864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d184cb3-8141-4def-826a-663210b88c1a admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:12.276571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d184cb3-8141-4def-826a-663210b88c1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:12.281438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d184cb3-8141-4def-826a-663210b88c1a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:12.314427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c6602-2902-476c-a466-48895b886f4e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:12.316238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ac7c6602-2902-476c-a466-48895b886f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:12.379515 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2760/5445] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:07:12 2022] POST /identity/v3/projects => generated 405 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:07:12.382329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d184cb3-8141-4def-826a-663210b88c1a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:12.485190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c6602-2902-476c-a466-48895b886f4e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:12.582146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d184cb3-8141-4def-826a-663210b88c1a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:12.595686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfeee1-b4d7-4d94-bc14-5e8c10cb1ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:12.596945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfeee1-b4d7-4d94-bc14-5e8c10cb1ae8 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:12.597209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfeee1-b4d7-4d94-bc14-5e8c10cb1ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:12.597482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfeee1-b4d7-4d94-bc14-5e8c10cb1ae8 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:12.598125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bfeee1-b4d7-4d94-bc14-5e8c10cb1ae8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:12.600098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bfeee1-b4d7-4d94-bc14-5e8c10cb1ae8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:12.653309 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2761/5446] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:07:12 2022] POST /identity/v3/users => generated 303 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:07:12.655706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfeee1-b4d7-4d94-bc14-5e8c10cb1ae8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:12.659704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ad866-adf8-422e-bf0b-6d69b8bfb8ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:12.660368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ad866-adf8-422e-bf0b-6d69b8bfb8ce service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:12.661904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ad866-adf8-422e-bf0b-6d69b8bfb8ce service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:12.662224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ad866-adf8-422e-bf0b-6d69b8bfb8ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:12.691576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfeee1-b4d7-4d94-bc14-5e8c10cb1ae8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:12.705481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d14fb-97da-4c35-b62b-f4594eb5321a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:12.706605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d14fb-97da-4c35-b62b-f4594eb5321a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:12.710017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d14fb-97da-4c35-b62b-f4594eb5321a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:12.710517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d14fb-97da-4c35-b62b-f4594eb5321a admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:12.711321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d14fb-97da-4c35-b62b-f4594eb5321a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:12.713054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d14fb-97da-4c35-b62b-f4594eb5321a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:12.747412 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2762/5447] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:07:12.754627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d14fb-97da-4c35-b62b-f4594eb5321a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:12.756846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:12.780876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d14fb-97da-4c35-b62b-f4594eb5321a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:12.797326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c9f25-91b7-410e-ac57-71d9e76ff298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:12.800200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c9f25-91b7-410e-ac57-71d9e76ff298 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:12.803073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c9f25-91b7-410e-ac57-71d9e76ff298 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:12.804534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c9f25-91b7-410e-ac57-71d9e76ff298 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:12.805496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851c9f25-91b7-410e-ac57-71d9e76ff298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:12.815888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851c9f25-91b7-410e-ac57-71d9e76ff298 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:12.857611 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2763/5448] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:07:12.858984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c9f25-91b7-410e-ac57-71d9e76ff298 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:12.859404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ad866-adf8-422e-bf0b-6d69b8bfb8ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:12.860940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ad866-adf8-422e-bf0b-6d69b8bfb8ce service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:12.886757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c9f25-91b7-410e-ac57-71d9e76ff298 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:12.889852 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2686/5449] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:07:12 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:07:12.904788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea63507-7676-480e-9603-f5f8724c5ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:12.905490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea63507-7676-480e-9603-f5f8724c5ced admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:12.905834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea63507-7676-480e-9603-f5f8724c5ced admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:12.906114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea63507-7676-480e-9603-f5f8724c5ced admin admin] PATH_INFO: `/v3/projects/478d0fd7c1664a6cbd61419bbb335beb/users/af3bc6e921b6454496accdab2890bfc3/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:12.949092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea63507-7676-480e-9603-f5f8724c5ced admin admin] RBAC: Authorizing `identity:create_grant(project_id=478d0fd7c1664a6cbd61419bbb335beb, user_id=af3bc6e921b6454496accdab2890bfc3, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:12.956563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea63507-7676-480e-9603-f5f8724c5ced admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:12.969918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:12.996612 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2764/5450] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:07:12 2022] PUT /identity/v3/projects/478d0fd7c1664a6cbd61419bbb335beb/users/af3bc6e921b6454496accdab2890bfc3/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:07:13.496696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348da89a-1fc5-44c3-8411-47fef17cd9af None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:13.497634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348da89a-1fc5-44c3-8411-47fef17cd9af None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:13.498080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348da89a-1fc5-44c3-8411-47fef17cd9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:13.546199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-348da89a-1fc5-44c3-8411-47fef17cd9af None None] MFA Rules not processed for user `6d63c633112e49d091421c84cea3422b`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:13.547535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-348da89a-1fc5-44c3-8411-47fef17cd9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:13.690652 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2687/5451] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:14.290881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea63507-7676-480e-9603-f5f8724c5ced admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:14.293855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5ea63507-7676-480e-9603-f5f8724c5ced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:14.499783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea63507-7676-480e-9603-f5f8724c5ced admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:14.526131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598b3c7-272b-4bbb-a504-0248c4b3bb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:14.526949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598b3c7-272b-4bbb-a504-0248c4b3bb31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:14.530748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598b3c7-272b-4bbb-a504-0248c4b3bb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:14.531429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598b3c7-272b-4bbb-a504-0248c4b3bb31 admin admin] PATH_INFO: `/v3/users/646f6f5cff204d27b0476d53bd9e6fa0` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:14.538157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598b3c7-272b-4bbb-a504-0248c4b3bb31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=646f6f5cff204d27b0476d53bd9e6fa0)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:14.548781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598b3c7-272b-4bbb-a504-0248c4b3bb31 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:14.695124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a598b3c7-272b-4bbb-a504-0248c4b3bb31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '646f6f5cff204d27b0476d53bd9e6fa0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:14.716200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a598b3c7-272b-4bbb-a504-0248c4b3bb31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '646f6f5cff204d27b0476d53bd9e6fa0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:14.750227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a598b3c7-272b-4bbb-a504-0248c4b3bb31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '646f6f5cff204d27b0476d53bd9e6fa0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:14.751530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a598b3c7-272b-4bbb-a504-0248c4b3bb31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '646f6f5cff204d27b0476d53bd9e6fa0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:14.767818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a598b3c7-272b-4bbb-a504-0248c4b3bb31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '646f6f5cff204d27b0476d53bd9e6fa0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:14.779897 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2765/5452] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:07:14 2022] DELETE /identity/v3/users/646f6f5cff204d27b0476d53bd9e6fa0 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:07:15.543621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644126d-08b3-4fe1-a17f-77ba5cbda7c8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:15.544173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644126d-08b3-4fe1-a17f-77ba5cbda7c8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:15.544173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644126d-08b3-4fe1-a17f-77ba5cbda7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:15.632149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a644126d-08b3-4fe1-a17f-77ba5cbda7c8 None None] MFA Rules not processed for user `2318c63cfee34ed8acfe691a7e38a8ef`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:15.634110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a644126d-08b3-4fe1-a17f-77ba5cbda7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:15.929065 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2688/5453] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:15.934646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598b3c7-272b-4bbb-a504-0248c4b3bb31 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:15.936739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a598b3c7-272b-4bbb-a504-0248c4b3bb31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:16.054116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644126d-08b3-4fe1-a17f-77ba5cbda7c8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:16.056011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a644126d-08b3-4fe1-a17f-77ba5cbda7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:16.112997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598b3c7-272b-4bbb-a504-0248c4b3bb31 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:16.127969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247bb12-cfec-454b-9753-869ef612ff97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:16.128729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247bb12-cfec-454b-9753-869ef612ff97 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:16.129254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247bb12-cfec-454b-9753-869ef612ff97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:16.129720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247bb12-cfec-454b-9753-869ef612ff97 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:16.132955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247bb12-cfec-454b-9753-869ef612ff97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:16.137544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247bb12-cfec-454b-9753-869ef612ff97 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:16.199718 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2766/5454] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:07:15 2022] POST /identity/v3/users => generated 288 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:07:16.204161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247bb12-cfec-454b-9753-869ef612ff97 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:16.223210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:16.231234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247bb12-cfec-454b-9753-869ef612ff97 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:16.247843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650182bc-3643-4df3-83ac-60cd59adcb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:16.249189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650182bc-3643-4df3-83ac-60cd59adcb9f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:16.251997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650182bc-3643-4df3-83ac-60cd59adcb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:16.253834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650182bc-3643-4df3-83ac-60cd59adcb9f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:16.254439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650182bc-3643-4df3-83ac-60cd59adcb9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:16.256166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650182bc-3643-4df3-83ac-60cd59adcb9f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:16.281646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644126d-08b3-4fe1-a17f-77ba5cbda7c8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:16.293138 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2767/5455] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:07:16.296752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e16372-9924-4fde-b2ee-0b715081a712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:16.297116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650182bc-3643-4df3-83ac-60cd59adcb9f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:16.297500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e16372-9924-4fde-b2ee-0b715081a712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:16.297789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e16372-9924-4fde-b2ee-0b715081a712 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:16.301167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e16372-9924-4fde-b2ee-0b715081a712 admin admin] PATH_INFO: `/v3/projects/d3434dac76ec46b3a5e10f840b7c2785` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:16.307801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e16372-9924-4fde-b2ee-0b715081a712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3434dac76ec46b3a5e10f840b7c2785)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:16.316517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e16372-9924-4fde-b2ee-0b715081a712 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:16.330676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650182bc-3643-4df3-83ac-60cd59adcb9f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:16.337451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:16.346386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900dfdc8-3e27-47d0-9d7a-c82b2c8ca79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:16.347230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dfdc8-3e27-47d0-9d7a-c82b2c8ca79b admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:16.347601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dfdc8-3e27-47d0-9d7a-c82b2c8ca79b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:16.348942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dfdc8-3e27-47d0-9d7a-c82b2c8ca79b admin admin] PATH_INFO: `/v3/system/users/c3d98b92c09d48dd80317e5dfafd7082/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:16.390053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900dfdc8-3e27-47d0-9d7a-c82b2c8ca79b admin admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c3d98b92c09d48dd80317e5dfafd7082, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:16.392341 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-900dfdc8-3e27-47d0-9d7a-c82b2c8ca79b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:07:16.392341 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:07:16.392341 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:07:16.393281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900dfdc8-3e27-47d0-9d7a-c82b2c8ca79b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:16.427235 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2768/5456] 10.222.0.17 () {60 vars in 1378 bytes} [Sat Oct 1 21:07:16 2022] PUT /identity/v3/system/users/c3d98b92c09d48dd80317e5dfafd7082/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:07:16.429839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9145fe-030f-4383-af7e-56b988971a36 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:16.430138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9145fe-030f-4383-af7e-56b988971a36 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:16.430647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9145fe-030f-4383-af7e-56b988971a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:16.494269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-cd9145fe-030f-4383-af7e-56b988971a36 None None] MFA Rules not processed for user `c3d98b92c09d48dd80317e5dfafd7082`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:16.494939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-98e16372-9924-4fde-b2ee-0b715081a712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3434dac76ec46b3a5e10f840b7c2785', 'request_id': 'req-98e16372-9924-4fde-b2ee-0b715081a712'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:16.496293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cd9145fe-030f-4383-af7e-56b988971a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:16.515784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-98e16372-9924-4fde-b2ee-0b715081a712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3434dac76ec46b3a5e10f840b7c2785', 'request_id': 'req-98e16372-9924-4fde-b2ee-0b715081a712'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:16.541589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-98e16372-9924-4fde-b2ee-0b715081a712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3434dac76ec46b3a5e10f840b7c2785', 'request_id': 'req-98e16372-9924-4fde-b2ee-0b715081a712'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:16.571431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-98e16372-9924-4fde-b2ee-0b715081a712 admin admin] The token cache is being invalidate because project d3434dac76ec46b3a5e10f840b7c2785 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:07:16.574249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-98e16372-9924-4fde-b2ee-0b715081a712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:16.575722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-98e16372-9924-4fde-b2ee-0b715081a712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:16.580320 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2689/5457] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:07:16 2022] DELETE /identity/v3/projects/d3434dac76ec46b3a5e10f840b7c2785 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:07:16.587651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035083d-4e4b-4ddc-ac3b-64e7c07ac8bc None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:16.588445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035083d-4e4b-4ddc-ac3b-64e7c07ac8bc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:16.589255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035083d-4e4b-4ddc-ac3b-64e7c07ac8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:16.669498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4035083d-4e4b-4ddc-ac3b-64e7c07ac8bc None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:16.670968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4035083d-4e4b-4ddc-ac3b-64e7c07ac8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:16.683472 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2769/5458] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:16 2022] POST /identity/v3/auth/tokens => generated 2415 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 01 21:07:16.688224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9145fe-030f-4383-af7e-56b988971a36 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:16.840601 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2690/5459] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:16.850192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd9028-3b8f-4bc5-a370-8067b441ae85 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:16.850695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd9028-3b8f-4bc5-a370-8067b441ae85 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:16.851039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd9028-3b8f-4bc5-a370-8067b441ae85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:16.900012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8bfd9028-3b8f-4bc5-a370-8067b441ae85 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:16.901522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8bfd9028-3b8f-4bc5-a370-8067b441ae85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:16.922947 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2691/5460] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:07:16.927594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd9028-3b8f-4bc5-a370-8067b441ae85 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:16.942010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:16.954954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd9028-3b8f-4bc5-a370-8067b441ae85 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:16.956405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9145fe-030f-4383-af7e-56b988971a36 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:16.965877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5df63-1dcf-4856-8710-e2e38cc1d933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:16.970902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5df63-1dcf-4856-8710-e2e38cc1d933 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:16.971395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5df63-1dcf-4856-8710-e2e38cc1d933 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:16.971818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5df63-1dcf-4856-8710-e2e38cc1d933 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:16.972750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5df63-1dcf-4856-8710-e2e38cc1d933 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:07:16.973907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5df63-1dcf-4856-8710-e2e38cc1d933 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:16.975931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5df63-1dcf-4856-8710-e2e38cc1d933 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:17.011122 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2692/5461] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:07:17.017003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5df63-1dcf-4856-8710-e2e38cc1d933 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:17.034026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5df63-1dcf-4856-8710-e2e38cc1d933 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:17.050728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf71eb-8ee2-432d-bf9b-345364cb0231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:17.051297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf71eb-8ee2-432d-bf9b-345364cb0231 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:17.051538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf71eb-8ee2-432d-bf9b-345364cb0231 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:17.051957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf71eb-8ee2-432d-bf9b-345364cb0231 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:17.052698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf71eb-8ee2-432d-bf9b-345364cb0231 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:17.055004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf71eb-8ee2-432d-bf9b-345364cb0231 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:17.060185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d461f-7a08-4352-8051-d8009f576125 None tempest-ImageQuotaTest-1362289648-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d98b92c09d48dd80317e5dfafd7082', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:17.061940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d461f-7a08-4352-8051-d8009f576125 None tempest-ImageQuotaTest-1362289648-system-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:17.064792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d461f-7a08-4352-8051-d8009f576125 None tempest-ImageQuotaTest-1362289648-system-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:17.066217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d461f-7a08-4352-8051-d8009f576125 None tempest-ImageQuotaTest-1362289648-system-admin] PATH_INFO: `/v3/services` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:17.067926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106d461f-7a08-4352-8051-d8009f576125 None tempest-ImageQuotaTest-1362289648-system-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:17.070194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106d461f-7a08-4352-8051-d8009f576125 None tempest-ImageQuotaTest-1362289648-system-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:17.087567 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2770/5462] 10.222.0.17 () {58 vars in 1110 bytes} [Sat Oct 1 21:07:16 2022] GET /identity/v3/services => generated 2191 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:07:17.092717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d461f-7a08-4352-8051-d8009f576125 None tempest-ImageQuotaTest-1362289648-system-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:17.136912 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2693/5463] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:07:17 2022] POST /identity/v3/projects => generated 405 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:07:17.142452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf71eb-8ee2-432d-bf9b-345364cb0231 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:17.338976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:17.365400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf71eb-8ee2-432d-bf9b-345364cb0231 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:17.381771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f06eb-87ba-4ba6-b2ce-a21b95ac1e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:17.382473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f06eb-87ba-4ba6-b2ce-a21b95ac1e7d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:17.386348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f06eb-87ba-4ba6-b2ce-a21b95ac1e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:17.386986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f06eb-87ba-4ba6-b2ce-a21b95ac1e7d admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:17.388752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f06eb-87ba-4ba6-b2ce-a21b95ac1e7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:17.397778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f06eb-87ba-4ba6-b2ce-a21b95ac1e7d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:17.455867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d461f-7a08-4352-8051-d8009f576125 None tempest-ImageQuotaTest-1362289648-system-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:17.474029 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2694/5464] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:07:17 2022] POST /identity/v3/users => generated 303 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:07:17.480597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f06eb-87ba-4ba6-b2ce-a21b95ac1e7d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:17.516002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f06eb-87ba-4ba6-b2ce-a21b95ac1e7d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:17.535971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bba353-1289-4b8a-aa50-ac5cebd51dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:17.536709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bba353-1289-4b8a-aa50-ac5cebd51dbb admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:17.537080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bba353-1289-4b8a-aa50-ac5cebd51dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:17.537404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bba353-1289-4b8a-aa50-ac5cebd51dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:17.538485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bba353-1289-4b8a-aa50-ac5cebd51dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:17.542626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bba353-1289-4b8a-aa50-ac5cebd51dbb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:17.547646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae519a-3244-470c-9317-1660feb1cb7a None tempest-ImageQuotaTest-1362289648-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d98b92c09d48dd80317e5dfafd7082', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:17.549197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae519a-3244-470c-9317-1660feb1cb7a None tempest-ImageQuotaTest-1362289648-system-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:17.549748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae519a-3244-470c-9317-1660feb1cb7a None tempest-ImageQuotaTest-1362289648-system-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:17.550134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae519a-3244-470c-9317-1660feb1cb7a None tempest-ImageQuotaTest-1362289648-system-admin] PATH_INFO: `/v3/limits` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:17.551043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae519a-3244-470c-9317-1660feb1cb7a None tempest-ImageQuotaTest-1362289648-system-admin] RBAC: Authorizing `identity:create_limits()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:17.553984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae519a-3244-470c-9317-1660feb1cb7a None tempest-ImageQuotaTest-1362289648-system-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:17.577196 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2695/5465] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:07:17.591591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bba353-1289-4b8a-aa50-ac5cebd51dbb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:17.615528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bba353-1289-4b8a-aa50-ac5cebd51dbb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:17.625466 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-d4ae519a-3244-470c-9317-1660feb1cb7a None tempest-ImageQuotaTest-1362289648-system-admin] Unable to create a limit that has no corresponding registered limit.: keystone.exception.NoLimitReference: Unable to create a limit that has no corresponding registered limit. Oct 01 21:07:17.625466 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:07:17.625466 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:07:17.625466 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:07:17.625466 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:07:17.625466 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:07:17.625466 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:07:17.625466 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:07:17.625466 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:07:17.625466 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:07:17.625466 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:07:17.625466 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:07:17.625466 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/limits.py", line 110, in post Oct 01 21:07:17.625466 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application refs = PROVIDERS.unified_limit_api.create_limits(limits) Oct 01 21:07:17.627860 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:07:17.627860 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:07:17.627860 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/core.py", line 116, in create_limits Oct 01 21:07:17.627860 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.driver.create_limits(limits) Oct 01 21:07:17.627860 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 01 21:07:17.627860 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 01 21:07:17.627860 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/backends/sql.py", line 260, in create_limits Oct 01 21:07:17.627860 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application target = self._check_and_fill_registered_limit_id(limit) Oct 01 21:07:17.627860 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/backends/sql.py", line 249, in _check_and_fill_registered_limit_id Oct 01 21:07:17.627860 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.NoLimitReference Oct 01 21:07:17.627860 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.NoLimitReference: Unable to create a limit that has no corresponding registered limit. Oct 01 21:07:17.627860 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:07:17.628937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14afd25e-c162-418a-adea-1c79347de77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:17.628937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afd25e-c162-418a-adea-1c79347de77b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:17.628937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afd25e-c162-418a-adea-1c79347de77b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:17.629235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afd25e-c162-418a-adea-1c79347de77b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:17.629905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14afd25e-c162-418a-adea-1c79347de77b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:17.632826 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2771/5466] 10.222.0.17 () {60 vars in 1126 bytes} [Sat Oct 1 21:07:17 2022] POST /identity/v3/limits => generated 124 bytes in 541 msecs (HTTP/1.1 403) 5 headers in 184 bytes (2 switches on core 0) Oct 01 21:07:17.633654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14afd25e-c162-418a-adea-1c79347de77b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:17.638202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae519a-3244-470c-9317-1660feb1cb7a None tempest-ImageQuotaTest-1362289648-system-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:17.672467 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2696/5467] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:07:17.679730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14afd25e-c162-418a-adea-1c79347de77b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:17.697953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:17.706275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14afd25e-c162-418a-adea-1c79347de77b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:17.721304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd23f72-4662-4498-ab03-9a4972ff2fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:17.722016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd23f72-4662-4498-ab03-9a4972ff2fbd admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:17.722260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd23f72-4662-4498-ab03-9a4972ff2fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:17.722550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd23f72-4662-4498-ab03-9a4972ff2fbd admin admin] PATH_INFO: `/v3/projects/7a8725bd62074b56b5712f469358728c/users/148cdf56f4a14512aba0659e4c362959/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:17.771864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd23f72-4662-4498-ab03-9a4972ff2fbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a8725bd62074b56b5712f469358728c, user_id=148cdf56f4a14512aba0659e4c362959, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:17.774605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd23f72-4662-4498-ab03-9a4972ff2fbd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:17.800295 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2697/5468] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:07:17 2022] PUT /identity/v3/projects/7a8725bd62074b56b5712f469358728c/users/148cdf56f4a14512aba0659e4c362959/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:07:17.810849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56bd43-4f24-4b89-bf85-943757add804 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:17.811163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56bd43-4f24-4b89-bf85-943757add804 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:17.811435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56bd43-4f24-4b89-bf85-943757add804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:17.848038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ec56bd43-4f24-4b89-bf85-943757add804 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:17.850650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ec56bd43-4f24-4b89-bf85-943757add804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:17.861060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae519a-3244-470c-9317-1660feb1cb7a None tempest-ImageQuotaTest-1362289648-system-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:17.968452 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2698/5469] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:17.982296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c379dc-c4f5-4795-a57d-e374b579d63c None tempest-ImageQuotaTest-1362289648-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d98b92c09d48dd80317e5dfafd7082', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:17.983815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c379dc-c4f5-4795-a57d-e374b579d63c None tempest-ImageQuotaTest-1362289648-system-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:17.984263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c379dc-c4f5-4795-a57d-e374b579d63c None tempest-ImageQuotaTest-1362289648-system-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:17.984994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c379dc-c4f5-4795-a57d-e374b579d63c None tempest-ImageQuotaTest-1362289648-system-admin] PATH_INFO: `/v3/registered_limits` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:17.985514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c379dc-c4f5-4795-a57d-e374b579d63c None tempest-ImageQuotaTest-1362289648-system-admin] RBAC: Authorizing `identity:list_registered_limits()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:17.987654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c379dc-c4f5-4795-a57d-e374b579d63c None tempest-ImageQuotaTest-1362289648-system-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:17.990548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56bd43-4f24-4b89-bf85-943757add804 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:17.992846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ec56bd43-4f24-4b89-bf85-943757add804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:17.997642 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2772/5470] 10.222.0.17 () {58 vars in 1137 bytes} [Sat Oct 1 21:07:17 2022] GET /identity/v3/registered_limits => generated 129 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:07:18.004530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c379dc-c4f5-4795-a57d-e374b579d63c None tempest-ImageQuotaTest-1362289648-system-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:18.191615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56bd43-4f24-4b89-bf85-943757add804 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:18.196262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10f4e8-f3c2-478f-8c84-638708b76890 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:18.196968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10f4e8-f3c2-478f-8c84-638708b76890 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:18.197815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10f4e8-f3c2-478f-8c84-638708b76890 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:18.198143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10f4e8-f3c2-478f-8c84-638708b76890 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:18.212027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:18.214538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f10f4e8-f3c2-478f-8c84-638708b76890 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:18.216892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f10f4e8-f3c2-478f-8c84-638708b76890 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:18.232253 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2699/5471] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:07:18.263092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c379dc-c4f5-4795-a57d-e374b579d63c None tempest-ImageQuotaTest-1362289648-system-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:18.350675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29801e38-e410-43bc-a849-bd1cc9fcaf6c None tempest-ImageQuotaTest-1362289648-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d98b92c09d48dd80317e5dfafd7082', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:18.351474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29801e38-e410-43bc-a849-bd1cc9fcaf6c None tempest-ImageQuotaTest-1362289648-system-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:18.353276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29801e38-e410-43bc-a849-bd1cc9fcaf6c None tempest-ImageQuotaTest-1362289648-system-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:18.353913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29801e38-e410-43bc-a849-bd1cc9fcaf6c None tempest-ImageQuotaTest-1362289648-system-admin] PATH_INFO: `/v3/registered_limits` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:18.355373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29801e38-e410-43bc-a849-bd1cc9fcaf6c None tempest-ImageQuotaTest-1362289648-system-admin] RBAC: Authorizing `identity:list_registered_limits()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:18.356590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29801e38-e410-43bc-a849-bd1cc9fcaf6c None tempest-ImageQuotaTest-1362289648-system-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:18.365433 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2773/5472] 10.222.0.17 () {58 vars in 1137 bytes} [Sat Oct 1 21:07:18 2022] GET /identity/v3/registered_limits => generated 129 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:07:26.513799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f4efd-8f8a-4d3a-87f9-59a842a20992 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:26.514334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f4efd-8f8a-4d3a-87f9-59a842a20992 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:26.514443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f4efd-8f8a-4d3a-87f9-59a842a20992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:26.573877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7b0f4efd-8f8a-4d3a-87f9-59a842a20992 None None] MFA Rules not processed for user `af3bc6e921b6454496accdab2890bfc3`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:26.577502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7b0f4efd-8f8a-4d3a-87f9-59a842a20992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:26.657122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:26.727672 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2700/5473] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:26 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:26.738906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d693d64-ed9e-437a-93c2-bbc8f00e01dc None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:26.740135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d693d64-ed9e-437a-93c2-bbc8f00e01dc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:26.740135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d693d64-ed9e-437a-93c2-bbc8f00e01dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:26.790317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6d693d64-ed9e-437a-93c2-bbc8f00e01dc None None] MFA Rules not processed for user `e7e3bc6d5e45497bb876d5cbcec61835`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:26.795127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6d693d64-ed9e-437a-93c2-bbc8f00e01dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:26.894687 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2774/5474] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:26 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:26.905815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f4efd-8f8a-4d3a-87f9-59a842a20992 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:26.907692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7b0f4efd-8f8a-4d3a-87f9-59a842a20992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:27.157396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f4efd-8f8a-4d3a-87f9-59a842a20992 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:27.171170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5246b79-7478-4f92-951e-a31b4461002a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:27.181264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5246b79-7478-4f92-951e-a31b4461002a service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:27.182871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5246b79-7478-4f92-951e-a31b4461002a service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:27.183395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5246b79-7478-4f92-951e-a31b4461002a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:27.186130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b5246b79-7478-4f92-951e-a31b4461002a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:27.203723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5246b79-7478-4f92-951e-a31b4461002a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:27.207021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5246b79-7478-4f92-951e-a31b4461002a service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:27.233721 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2701/5475] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:07:26 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:07:33.194868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d693d64-ed9e-437a-93c2-bbc8f00e01dc None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:33.198189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6d693d64-ed9e-437a-93c2-bbc8f00e01dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:33.217697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d693d64-ed9e-437a-93c2-bbc8f00e01dc None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:33.225161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a0d87-f6e4-438b-8ac5-ac8a14802835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:33.227451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a0d87-f6e4-438b-8ac5-ac8a14802835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:33.227875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a0d87-f6e4-438b-8ac5-ac8a14802835 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:33.229066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a0d87-f6e4-438b-8ac5-ac8a14802835 admin admin] PATH_INFO: `/v3/users/2318c63cfee34ed8acfe691a7e38a8ef` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:33.232286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a0d87-f6e4-438b-8ac5-ac8a14802835 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2318c63cfee34ed8acfe691a7e38a8ef)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:33.249547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a0d87-f6e4-438b-8ac5-ac8a14802835 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:33.361070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-048a0d87-f6e4-438b-8ac5-ac8a14802835 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2318c63cfee34ed8acfe691a7e38a8ef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:33.386074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-048a0d87-f6e4-438b-8ac5-ac8a14802835 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2318c63cfee34ed8acfe691a7e38a8ef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:33.406456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-048a0d87-f6e4-438b-8ac5-ac8a14802835 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2318c63cfee34ed8acfe691a7e38a8ef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:33.407638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-048a0d87-f6e4-438b-8ac5-ac8a14802835 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2318c63cfee34ed8acfe691a7e38a8ef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:33.426365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-048a0d87-f6e4-438b-8ac5-ac8a14802835 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2318c63cfee34ed8acfe691a7e38a8ef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:33.452404 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2775/5476] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:07:33 2022] DELETE /identity/v3/users/2318c63cfee34ed8acfe691a7e38a8ef => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:07:34.839677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5246b79-7478-4f92-951e-a31b4461002a service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:34.858025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5246b79-7478-4f92-951e-a31b4461002a service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:35.108808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90e00f-2575-4201-93eb-3e684c434ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:35.109486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90e00f-2575-4201-93eb-3e684c434ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:35.109788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90e00f-2575-4201-93eb-3e684c434ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:35.110094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90e00f-2575-4201-93eb-3e684c434ae8 admin admin] PATH_INFO: `/v3/projects/021ea8b4eff84168bc344a6ec2fbb998` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:35.111331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90e00f-2575-4201-93eb-3e684c434ae8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=021ea8b4eff84168bc344a6ec2fbb998)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:35.113322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90e00f-2575-4201-93eb-3e684c434ae8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:35.223558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:35.239467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b90e00f-2575-4201-93eb-3e684c434ae8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '021ea8b4eff84168bc344a6ec2fbb998', 'request_id': 'req-5b90e00f-2575-4201-93eb-3e684c434ae8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:35.260101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b90e00f-2575-4201-93eb-3e684c434ae8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '021ea8b4eff84168bc344a6ec2fbb998', 'request_id': 'req-5b90e00f-2575-4201-93eb-3e684c434ae8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:35.278841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b90e00f-2575-4201-93eb-3e684c434ae8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '021ea8b4eff84168bc344a6ec2fbb998', 'request_id': 'req-5b90e00f-2575-4201-93eb-3e684c434ae8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:35.308406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b90e00f-2575-4201-93eb-3e684c434ae8 admin admin] The token cache is being invalidate because project 021ea8b4eff84168bc344a6ec2fbb998 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:07:35.308765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b90e00f-2575-4201-93eb-3e684c434ae8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:35.309424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b90e00f-2575-4201-93eb-3e684c434ae8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:35.314354 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2702/5477] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:07:34 2022] DELETE /identity/v3/projects/021ea8b4eff84168bc344a6ec2fbb998 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:07:35.316116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a0d87-f6e4-438b-8ac5-ac8a14802835 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:35.318416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-048a0d87-f6e4-438b-8ac5-ac8a14802835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:35.346580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a0d87-f6e4-438b-8ac5-ac8a14802835 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:35.370965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:35.517953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce9f0e-61e6-4d4f-82ab-ac83cb175879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:35.521782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce9f0e-61e6-4d4f-82ab-ac83cb175879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:35.522438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce9f0e-61e6-4d4f-82ab-ac83cb175879 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:35.523046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce9f0e-61e6-4d4f-82ab-ac83cb175879 admin admin] PATH_INFO: `/v3/users/c3d98b92c09d48dd80317e5dfafd7082` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:35.536025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce9f0e-61e6-4d4f-82ab-ac83cb175879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3d98b92c09d48dd80317e5dfafd7082)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:35.538991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce9f0e-61e6-4d4f-82ab-ac83cb175879 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:35.714445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-97ce9f0e-61e6-4d4f-82ab-ac83cb175879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3d98b92c09d48dd80317e5dfafd7082'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:35.732916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-97ce9f0e-61e6-4d4f-82ab-ac83cb175879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3d98b92c09d48dd80317e5dfafd7082'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:35.765092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-97ce9f0e-61e6-4d4f-82ab-ac83cb175879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3d98b92c09d48dd80317e5dfafd7082'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:35.765928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-97ce9f0e-61e6-4d4f-82ab-ac83cb175879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3d98b92c09d48dd80317e5dfafd7082'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:35.785060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-97ce9f0e-61e6-4d4f-82ab-ac83cb175879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3d98b92c09d48dd80317e5dfafd7082'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:35.795491 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2776/5478] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:07:35 2022] DELETE /identity/v3/users/c3d98b92c09d48dd80317e5dfafd7082 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:07:37.872381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267d3ab-1212-477a-b53c-9bcc1e546ffb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:37.874008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267d3ab-1212-477a-b53c-9bcc1e546ffb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:37.880756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267d3ab-1212-477a-b53c-9bcc1e546ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:37.968326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e267d3ab-1212-477a-b53c-9bcc1e546ffb None None] MFA Rules not processed for user `148cdf56f4a14512aba0659e4c362959`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:37.970200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e267d3ab-1212-477a-b53c-9bcc1e546ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:38.124318 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2703/5479] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:37 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:38.762297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce9f0e-61e6-4d4f-82ab-ac83cb175879 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:38.764538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-97ce9f0e-61e6-4d4f-82ab-ac83cb175879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:38.926035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce9f0e-61e6-4d4f-82ab-ac83cb175879 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:38.939020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ed2bc-88c3-4b62-bc39-091110c84a58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:38.940161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ed2bc-88c3-4b62-bc39-091110c84a58 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:38.940705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ed2bc-88c3-4b62-bc39-091110c84a58 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:38.941178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ed2bc-88c3-4b62-bc39-091110c84a58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:38.954853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ed2bc-88c3-4b62-bc39-091110c84a58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:38.956768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ed2bc-88c3-4b62-bc39-091110c84a58 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:38.972934 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2777/5480] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:07:38 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:07:46.815381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c9e9c-3a22-4621-973c-b21b5b03c6ca None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:46.816390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c9e9c-3a22-4621-973c-b21b5b03c6ca None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:46.816820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c9e9c-3a22-4621-973c-b21b5b03c6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:46.866948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2e5c9e9c-3a22-4621-973c-b21b5b03c6ca None None] MFA Rules not processed for user `6d63c633112e49d091421c84cea3422b`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:46.869475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2e5c9e9c-3a22-4621-973c-b21b5b03c6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:47.003010 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2704/5481] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:47.186956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ed2bc-88c3-4b62-bc39-091110c84a58 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:47.188841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-837ed2bc-88c3-4b62-bc39-091110c84a58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:47.214864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:47.336433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ed2bc-88c3-4b62-bc39-091110c84a58 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:47.343123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5cd94-5c77-4290-8fb4-308887dbf744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:47.343854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5cd94-5c77-4290-8fb4-308887dbf744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:47.344160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5cd94-5c77-4290-8fb4-308887dbf744 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:47.344444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5cd94-5c77-4290-8fb4-308887dbf744 admin admin] PATH_INFO: `/v3/users/e3f8e2ca415c4a309b036d8406fcc921` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:47.345774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5cd94-5c77-4290-8fb4-308887dbf744 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3f8e2ca415c4a309b036d8406fcc921)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:47.348284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5cd94-5c77-4290-8fb4-308887dbf744 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:47.447488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-29c5cd94-5c77-4290-8fb4-308887dbf744 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3f8e2ca415c4a309b036d8406fcc921'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:47.464482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-29c5cd94-5c77-4290-8fb4-308887dbf744 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3f8e2ca415c4a309b036d8406fcc921'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:47.496650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-29c5cd94-5c77-4290-8fb4-308887dbf744 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3f8e2ca415c4a309b036d8406fcc921'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:47.499835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-29c5cd94-5c77-4290-8fb4-308887dbf744 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3f8e2ca415c4a309b036d8406fcc921'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:47.519118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-29c5cd94-5c77-4290-8fb4-308887dbf744 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3f8e2ca415c4a309b036d8406fcc921'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:47.534863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1010f8c-3e1e-4658-83ee-ad1bd694a27f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:47.538446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1010f8c-3e1e-4658-83ee-ad1bd694a27f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:47.538997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1010f8c-3e1e-4658-83ee-ad1bd694a27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:47.547242 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2778/5482] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:07:47 2022] DELETE /identity/v3/users/e3f8e2ca415c4a309b036d8406fcc921 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:07:47.596889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f1010f8c-3e1e-4658-83ee-ad1bd694a27f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:47.600211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f1010f8c-3e1e-4658-83ee-ad1bd694a27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:47.705355 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2705/5483] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:47.722978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5cd94-5c77-4290-8fb4-308887dbf744 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:47.725915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-29c5cd94-5c77-4290-8fb4-308887dbf744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:47.862322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5cd94-5c77-4290-8fb4-308887dbf744 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:47.867159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed627fa4-3fbe-49c3-b3dc-4fe697bb1754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:47.867804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed627fa4-3fbe-49c3-b3dc-4fe697bb1754 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:47.868710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed627fa4-3fbe-49c3-b3dc-4fe697bb1754 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:47.868969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed627fa4-3fbe-49c3-b3dc-4fe697bb1754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:47.882278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed627fa4-3fbe-49c3-b3dc-4fe697bb1754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:47.883889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed627fa4-3fbe-49c3-b3dc-4fe697bb1754 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:47.900339 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2779/5484] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:07:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:07:48.488221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1010f8c-3e1e-4658-83ee-ad1bd694a27f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:48.490551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f1010f8c-3e1e-4658-83ee-ad1bd694a27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:48.512981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:48.518826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1010f8c-3e1e-4658-83ee-ad1bd694a27f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:48.532551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8bdee-bba1-4f90-803b-26920f1f6dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:48.534331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8bdee-bba1-4f90-803b-26920f1f6dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:48.534798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8bdee-bba1-4f90-803b-26920f1f6dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:48.535229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8bdee-bba1-4f90-803b-26920f1f6dc3 admin admin] PATH_INFO: `/v3/projects/c34b7d9e02bb4583b6769768a1265d1c` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:48.536690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd8bdee-bba1-4f90-803b-26920f1f6dc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c34b7d9e02bb4583b6769768a1265d1c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:48.540473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd8bdee-bba1-4f90-803b-26920f1f6dc3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:48.621260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1dd8bdee-bba1-4f90-803b-26920f1f6dc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c34b7d9e02bb4583b6769768a1265d1c', 'request_id': 'req-1dd8bdee-bba1-4f90-803b-26920f1f6dc3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:48.633358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1dd8bdee-bba1-4f90-803b-26920f1f6dc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c34b7d9e02bb4583b6769768a1265d1c', 'request_id': 'req-1dd8bdee-bba1-4f90-803b-26920f1f6dc3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:48.641808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1dd8bdee-bba1-4f90-803b-26920f1f6dc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c34b7d9e02bb4583b6769768a1265d1c', 'request_id': 'req-1dd8bdee-bba1-4f90-803b-26920f1f6dc3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:48.657758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1dd8bdee-bba1-4f90-803b-26920f1f6dc3 admin admin] The token cache is being invalidate because project c34b7d9e02bb4583b6769768a1265d1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:07:48.658405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1dd8bdee-bba1-4f90-803b-26920f1f6dc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:48.659349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1dd8bdee-bba1-4f90-803b-26920f1f6dc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:48.661453 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2706/5485] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:07:48 2022] DELETE /identity/v3/projects/c34b7d9e02bb4583b6769768a1265d1c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:07:48.668264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2101ce1-6e37-4fcb-8d34-8947ed6ab601 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:48.668821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2101ce1-6e37-4fcb-8d34-8947ed6ab601 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:48.668821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2101ce1-6e37-4fcb-8d34-8947ed6ab601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:48.701505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c2101ce1-6e37-4fcb-8d34-8947ed6ab601 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:48.702854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c2101ce1-6e37-4fcb-8d34-8947ed6ab601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:48.727799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:48.845668 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2780/5486] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:48.854700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502854d-5461-4a1b-909e-2bcc01eba0a3 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:48.855040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502854d-5461-4a1b-909e-2bcc01eba0a3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:48.855778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502854d-5461-4a1b-909e-2bcc01eba0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:48.877970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4502854d-5461-4a1b-909e-2bcc01eba0a3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:48.879498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4502854d-5461-4a1b-909e-2bcc01eba0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:48.896889 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2707/5487] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:48.902471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2101ce1-6e37-4fcb-8d34-8947ed6ab601 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:48.913341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2101ce1-6e37-4fcb-8d34-8947ed6ab601 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:48.919644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f021334-df92-44ec-ae70-b3164a91e2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:48.920267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f021334-df92-44ec-ae70-b3164a91e2d2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:48.920546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f021334-df92-44ec-ae70-b3164a91e2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:48.921256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f021334-df92-44ec-ae70-b3164a91e2d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:48.921710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f021334-df92-44ec-ae70-b3164a91e2d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:07:48.922019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f021334-df92-44ec-ae70-b3164a91e2d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:48.924166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f021334-df92-44ec-ae70-b3164a91e2d2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:48.942909 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2781/5488] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:07:48.945468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502854d-5461-4a1b-909e-2bcc01eba0a3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:48.957776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502854d-5461-4a1b-909e-2bcc01eba0a3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:48.967472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ab3ca-41b9-477c-a69e-9587300ddd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:48.968147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ab3ca-41b9-477c-a69e-9587300ddd7a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:48.968549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ab3ca-41b9-477c-a69e-9587300ddd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:48.968870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ab3ca-41b9-477c-a69e-9587300ddd7a admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:48.970130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ab3ca-41b9-477c-a69e-9587300ddd7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:48.972504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ab3ca-41b9-477c-a69e-9587300ddd7a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:49.028284 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2708/5489] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:07:48 2022] POST /identity/v3/projects => generated 371 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:07:49.034335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f021334-df92-44ec-ae70-b3164a91e2d2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:49.052842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f021334-df92-44ec-ae70-b3164a91e2d2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:49.128098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad42b5c-6c65-4a53-8f30-b5e3ca2827d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:49.129131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad42b5c-6c65-4a53-8f30-b5e3ca2827d0 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:49.129664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad42b5c-6c65-4a53-8f30-b5e3ca2827d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:49.129936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad42b5c-6c65-4a53-8f30-b5e3ca2827d0 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:49.130673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad42b5c-6c65-4a53-8f30-b5e3ca2827d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:49.132780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad42b5c-6c65-4a53-8f30-b5e3ca2827d0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:49.160377 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2782/5490] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:07:49 2022] POST /identity/v3/users => generated 286 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:07:49.165371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ab3ca-41b9-477c-a69e-9587300ddd7a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:49.176776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ab3ca-41b9-477c-a69e-9587300ddd7a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:49.184485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5def6-4b73-4333-b5d8-8557c70bdcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:49.185197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5def6-4b73-4333-b5d8-8557c70bdcd0 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:49.185654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5def6-4b73-4333-b5d8-8557c70bdcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:49.186041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5def6-4b73-4333-b5d8-8557c70bdcd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:49.186922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5def6-4b73-4333-b5d8-8557c70bdcd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:49.188700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5def6-4b73-4333-b5d8-8557c70bdcd0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:49.202532 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2709/5491] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:07:49.204590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad42b5c-6c65-4a53-8f30-b5e3ca2827d0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:49.218004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad42b5c-6c65-4a53-8f30-b5e3ca2827d0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:49.224576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91b0b1-7d70-431b-9e86-6eaf4935d6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:49.225344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91b0b1-7d70-431b-9e86-6eaf4935d6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:49.225685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91b0b1-7d70-431b-9e86-6eaf4935d6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:49.226028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91b0b1-7d70-431b-9e86-6eaf4935d6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:49.226708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91b0b1-7d70-431b-9e86-6eaf4935d6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:49.229111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91b0b1-7d70-431b-9e86-6eaf4935d6b4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:49.240899 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2783/5492] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:07:49.260216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5def6-4b73-4333-b5d8-8557c70bdcd0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:49.280946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5def6-4b73-4333-b5d8-8557c70bdcd0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:49.287835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92f0b0-2054-4b54-affe-e6aaa08bb5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:49.288748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92f0b0-2054-4b54-affe-e6aaa08bb5ef admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:49.289448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92f0b0-2054-4b54-affe-e6aaa08bb5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:49.290606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92f0b0-2054-4b54-affe-e6aaa08bb5ef admin admin] PATH_INFO: `/v3/projects/f79f720a21e2439ea1eb3d0f509e2a8c/users/a8310ff8e16941399d8000498f243a0b/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:49.311338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92f0b0-2054-4b54-affe-e6aaa08bb5ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=f79f720a21e2439ea1eb3d0f509e2a8c, user_id=a8310ff8e16941399d8000498f243a0b, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:49.314727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92f0b0-2054-4b54-affe-e6aaa08bb5ef admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:49.332833 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2710/5493] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:07:49 2022] PUT /identity/v3/projects/f79f720a21e2439ea1eb3d0f509e2a8c/users/a8310ff8e16941399d8000498f243a0b/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:07:49.346461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c8bf2-d9b4-461e-ba07-68468cafca9b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:49.346882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c8bf2-d9b4-461e-ba07-68468cafca9b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:49.347262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c8bf2-d9b4-461e-ba07-68468cafca9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:49.396026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-696c8bf2-d9b4-461e-ba07-68468cafca9b None None] MFA Rules not processed for user `a8310ff8e16941399d8000498f243a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:49.397404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-696c8bf2-d9b4-461e-ba07-68468cafca9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:49.555970 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2784/5494] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:49.566835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92f0b0-2054-4b54-affe-e6aaa08bb5ef admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:49.570682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4f92f0b0-2054-4b54-affe-e6aaa08bb5ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:49.682629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92f0b0-2054-4b54-affe-e6aaa08bb5ef admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:49.688327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf6cf8-3ebf-4dac-8860-53d2ed03fa13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:49.689786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf6cf8-3ebf-4dac-8860-53d2ed03fa13 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:49.692646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf6cf8-3ebf-4dac-8860-53d2ed03fa13 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:49.693346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf6cf8-3ebf-4dac-8860-53d2ed03fa13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:49.710083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf6cf8-3ebf-4dac-8860-53d2ed03fa13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:49.713295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf6cf8-3ebf-4dac-8860-53d2ed03fa13 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:49.729869 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2711/5495] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:07:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:07:50.084103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c8bf2-d9b4-461e-ba07-68468cafca9b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:50.089471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-696c8bf2-d9b4-461e-ba07-68468cafca9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:50.113627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:50.252902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c8bf2-d9b4-461e-ba07-68468cafca9b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:50.263492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010cba9-1efc-4a62-b0ca-c98583ce4721 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:50.264131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010cba9-1efc-4a62-b0ca-c98583ce4721 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:50.264577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010cba9-1efc-4a62-b0ca-c98583ce4721 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:50.264865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010cba9-1efc-4a62-b0ca-c98583ce4721 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:50.268148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3010cba9-1efc-4a62-b0ca-c98583ce4721 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:50.408792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010cba9-1efc-4a62-b0ca-c98583ce4721 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:50.411704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010cba9-1efc-4a62-b0ca-c98583ce4721 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:50.462277 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2785/5496] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:07:50 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:07:50.671771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf6cf8-3ebf-4dac-8860-53d2ed03fa13 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:50.729731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:50.785574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf6cf8-3ebf-4dac-8860-53d2ed03fa13 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:50.791898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4e220-9f4a-4b31-98bf-417944ea2043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:50.793340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4e220-9f4a-4b31-98bf-417944ea2043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:50.793831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4e220-9f4a-4b31-98bf-417944ea2043 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:50.794305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4e220-9f4a-4b31-98bf-417944ea2043 admin admin] PATH_INFO: `/v3/users/a8310ff8e16941399d8000498f243a0b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:50.796322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4e220-9f4a-4b31-98bf-417944ea2043 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8310ff8e16941399d8000498f243a0b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:50.799037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4e220-9f4a-4b31-98bf-417944ea2043 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:50.880555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bfa4e220-9f4a-4b31-98bf-417944ea2043 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8310ff8e16941399d8000498f243a0b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:50.881822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bfa4e220-9f4a-4b31-98bf-417944ea2043 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8310ff8e16941399d8000498f243a0b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:50.882734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bfa4e220-9f4a-4b31-98bf-417944ea2043 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8310ff8e16941399d8000498f243a0b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:50.895658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bfa4e220-9f4a-4b31-98bf-417944ea2043 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8310ff8e16941399d8000498f243a0b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:50.916109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bfa4e220-9f4a-4b31-98bf-417944ea2043 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8310ff8e16941399d8000498f243a0b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:50.939075 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2712/5497] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:07:50 2022] DELETE /identity/v3/users/a8310ff8e16941399d8000498f243a0b => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:07:50.940763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48dafb-547a-4a2c-8c2a-c1b94f589c44 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:50.941089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48dafb-547a-4a2c-8c2a-c1b94f589c44 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:50.941350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48dafb-547a-4a2c-8c2a-c1b94f589c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:50.969451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-cc48dafb-547a-4a2c-8c2a-c1b94f589c44 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:50.971126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cc48dafb-547a-4a2c-8c2a-c1b94f589c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:51.057477 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2786/5498] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:51.076037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4e220-9f4a-4b31-98bf-417944ea2043 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:51.077666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bfa4e220-9f4a-4b31-98bf-417944ea2043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:51.184659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4e220-9f4a-4b31-98bf-417944ea2043 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:51.189365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032637b-cf2b-426c-9e38-d8424849543b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:51.189987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032637b-cf2b-426c-9e38-d8424849543b service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:51.190801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032637b-cf2b-426c-9e38-d8424849543b service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:51.191077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032637b-cf2b-426c-9e38-d8424849543b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:51.202607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2032637b-cf2b-426c-9e38-d8424849543b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:51.204063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2032637b-cf2b-426c-9e38-d8424849543b service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:51.222159 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2713/5499] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:07:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:07:51.735808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48dafb-547a-4a2c-8c2a-c1b94f589c44 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:51.737260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cc48dafb-547a-4a2c-8c2a-c1b94f589c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:51.766736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48dafb-547a-4a2c-8c2a-c1b94f589c44 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:51.774036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fef6ff-48f3-4266-82b8-1ed94b302528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:51.775646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fef6ff-48f3-4266-82b8-1ed94b302528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:51.776117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fef6ff-48f3-4266-82b8-1ed94b302528 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:51.776791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fef6ff-48f3-4266-82b8-1ed94b302528 admin admin] PATH_INFO: `/v3/projects/f79f720a21e2439ea1eb3d0f509e2a8c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:51.780026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fef6ff-48f3-4266-82b8-1ed94b302528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f79f720a21e2439ea1eb3d0f509e2a8c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:51.783055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fef6ff-48f3-4266-82b8-1ed94b302528 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:51.868570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:51.874754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-27fef6ff-48f3-4266-82b8-1ed94b302528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f79f720a21e2439ea1eb3d0f509e2a8c', 'request_id': 'req-27fef6ff-48f3-4266-82b8-1ed94b302528'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:51.884056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-27fef6ff-48f3-4266-82b8-1ed94b302528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f79f720a21e2439ea1eb3d0f509e2a8c', 'request_id': 'req-27fef6ff-48f3-4266-82b8-1ed94b302528'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:51.894807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-27fef6ff-48f3-4266-82b8-1ed94b302528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f79f720a21e2439ea1eb3d0f509e2a8c', 'request_id': 'req-27fef6ff-48f3-4266-82b8-1ed94b302528'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:51.914307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-27fef6ff-48f3-4266-82b8-1ed94b302528 admin admin] The token cache is being invalidate because project f79f720a21e2439ea1eb3d0f509e2a8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:07:51.915844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-27fef6ff-48f3-4266-82b8-1ed94b302528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:51.916891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-27fef6ff-48f3-4266-82b8-1ed94b302528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:07:51.919099 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2787/5500] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:07:51 2022] DELETE /identity/v3/projects/f79f720a21e2439ea1eb3d0f509e2a8c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:07:51.923790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7512ed5-3372-4524-8e21-36ff578d7a48 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:51.925103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7512ed5-3372-4524-8e21-36ff578d7a48 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:51.925103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7512ed5-3372-4524-8e21-36ff578d7a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:51.963344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b7512ed5-3372-4524-8e21-36ff578d7a48 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:51.967622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b7512ed5-3372-4524-8e21-36ff578d7a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:52.089455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:52.126718 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2714/5501] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:52.135099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a300d0-2036-4687-94ed-6cbac6df636f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:52.135698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a300d0-2036-4687-94ed-6cbac6df636f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:52.136237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a300d0-2036-4687-94ed-6cbac6df636f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:52.208105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-94a300d0-2036-4687-94ed-6cbac6df636f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:52.209526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-94a300d0-2036-4687-94ed-6cbac6df636f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:52.231899 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2788/5502] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:52.237213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7512ed5-3372-4524-8e21-36ff578d7a48 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:52.254765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7512ed5-3372-4524-8e21-36ff578d7a48 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:52.264730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e6553-7815-4e99-8fe0-b5d3b519e7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:52.265491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e6553-7815-4e99-8fe0-b5d3b519e7fb admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:52.266119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e6553-7815-4e99-8fe0-b5d3b519e7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:52.266577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e6553-7815-4e99-8fe0-b5d3b519e7fb admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:52.268636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e6553-7815-4e99-8fe0-b5d3b519e7fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:07:52.269335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e6553-7815-4e99-8fe0-b5d3b519e7fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:52.271440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e6553-7815-4e99-8fe0-b5d3b519e7fb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:52.292022 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2715/5503] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:07:52.297156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a300d0-2036-4687-94ed-6cbac6df636f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:52.310031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a300d0-2036-4687-94ed-6cbac6df636f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:52.317263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d143c1-88d4-4675-9c0b-c4066b792d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:52.318096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d143c1-88d4-4675-9c0b-c4066b792d73 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:52.318384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d143c1-88d4-4675-9c0b-c4066b792d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:52.318757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d143c1-88d4-4675-9c0b-c4066b792d73 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:52.319592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d143c1-88d4-4675-9c0b-c4066b792d73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:52.322900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d143c1-88d4-4675-9c0b-c4066b792d73 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:52.371302 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2789/5504] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:07:52 2022] POST /identity/v3/projects => generated 391 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:07:52.377580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e6553-7815-4e99-8fe0-b5d3b519e7fb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:52.396490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e6553-7815-4e99-8fe0-b5d3b519e7fb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:52.529162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb0312-5675-487a-ac09-362370a199af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:52.529789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb0312-5675-487a-ac09-362370a199af admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:52.530073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb0312-5675-487a-ac09-362370a199af admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:52.531396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb0312-5675-487a-ac09-362370a199af admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:52.532253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb0312-5675-487a-ac09-362370a199af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:52.536055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb0312-5675-487a-ac09-362370a199af admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:52.578662 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2716/5505] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:07:52 2022] POST /identity/v3/users => generated 296 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:07:52.582711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d143c1-88d4-4675-9c0b-c4066b792d73 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:52.598150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d143c1-88d4-4675-9c0b-c4066b792d73 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:52.606933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c42a6-8531-4981-8425-417a72a52f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:52.607964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c42a6-8531-4981-8425-417a72a52f1b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:52.608480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c42a6-8531-4981-8425-417a72a52f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:52.609105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c42a6-8531-4981-8425-417a72a52f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:52.610519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c42a6-8531-4981-8425-417a72a52f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:52.612964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c42a6-8531-4981-8425-417a72a52f1b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:52.630891 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2790/5506] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:07:52.641947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb0312-5675-487a-ac09-362370a199af admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:52.657794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb0312-5675-487a-ac09-362370a199af admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:52.666647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710acf0f-c057-4788-9350-3001ba9b74ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:52.667367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710acf0f-c057-4788-9350-3001ba9b74ed admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:52.667958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710acf0f-c057-4788-9350-3001ba9b74ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:52.668246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710acf0f-c057-4788-9350-3001ba9b74ed admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:52.669041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710acf0f-c057-4788-9350-3001ba9b74ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:52.671756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710acf0f-c057-4788-9350-3001ba9b74ed admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:52.693569 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2717/5507] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:07:52.697463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c42a6-8531-4981-8425-417a72a52f1b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:52.710000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c42a6-8531-4981-8425-417a72a52f1b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:52.718277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b02f5b-e152-413c-9ce0-ba14d46aa50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:52.719193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b02f5b-e152-413c-9ce0-ba14d46aa50f admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:52.719607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b02f5b-e152-413c-9ce0-ba14d46aa50f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:52.719989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b02f5b-e152-413c-9ce0-ba14d46aa50f admin admin] PATH_INFO: `/v3/projects/dad97e3d163e4b939eded21322fbcebe/users/48891c9ba3774c0d9b9189af6c271f67/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:52.781504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b02f5b-e152-413c-9ce0-ba14d46aa50f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dad97e3d163e4b939eded21322fbcebe, user_id=48891c9ba3774c0d9b9189af6c271f67, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:52.784615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b02f5b-e152-413c-9ce0-ba14d46aa50f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:52.819092 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2791/5508] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:07:52 2022] PUT /identity/v3/projects/dad97e3d163e4b939eded21322fbcebe/users/48891c9ba3774c0d9b9189af6c271f67/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:07:52.830906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d6a57-af53-4404-bf2c-1ea05ecd5aee None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:52.831259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d6a57-af53-4404-bf2c-1ea05ecd5aee None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:52.831515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d6a57-af53-4404-bf2c-1ea05ecd5aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:52.878080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-277d6a57-af53-4404-bf2c-1ea05ecd5aee None None] MFA Rules not processed for user `48891c9ba3774c0d9b9189af6c271f67`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:07:52.879909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-277d6a57-af53-4404-bf2c-1ea05ecd5aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:53.042053 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2718/5509] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:07:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:07:53.072801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b02f5b-e152-413c-9ce0-ba14d46aa50f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:07:53.075509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e1b02f5b-e152-413c-9ce0-ba14d46aa50f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:07:53.231707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b02f5b-e152-413c-9ce0-ba14d46aa50f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:07:53.237360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2694fd-6bf9-479d-84aa-15fc85376bc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:07:53.238254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2694fd-6bf9-479d-84aa-15fc85376bc5 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:07:53.239049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2694fd-6bf9-479d-84aa-15fc85376bc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:07:53.239582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2694fd-6bf9-479d-84aa-15fc85376bc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:07:53.255216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2694fd-6bf9-479d-84aa-15fc85376bc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:07:53.257199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2694fd-6bf9-479d-84aa-15fc85376bc5 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:07:53.289983 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2792/5510] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:07:53 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:08:02.235814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d6a57-af53-4404-bf2c-1ea05ecd5aee None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:02.239279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-277d6a57-af53-4404-bf2c-1ea05ecd5aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:02.378985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:02.433491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2694fd-6bf9-479d-84aa-15fc85376bc5 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:02.440184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d6a57-af53-4404-bf2c-1ea05ecd5aee None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:02.447260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bef6b-5da4-46e6-9258-57348f3c0667 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:02.448321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bef6b-5da4-46e6-9258-57348f3c0667 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:02.448807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bef6b-5da4-46e6-9258-57348f3c0667 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:02.449261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bef6b-5da4-46e6-9258-57348f3c0667 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:02.452168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-614bef6b-5da4-46e6-9258-57348f3c0667 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:02.484910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614bef6b-5da4-46e6-9258-57348f3c0667 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:02.488387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614bef6b-5da4-46e6-9258-57348f3c0667 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:02.521464 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2719/5511] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:08:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:08:02.582894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2694fd-6bf9-479d-84aa-15fc85376bc5 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:02.594548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e5e01-24df-4a4b-a260-7b9d529ff850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:02.595302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e5e01-24df-4a4b-a260-7b9d529ff850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:02.596197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e5e01-24df-4a4b-a260-7b9d529ff850 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:02.596483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e5e01-24df-4a4b-a260-7b9d529ff850 admin admin] PATH_INFO: `/v3/users/48891c9ba3774c0d9b9189af6c271f67` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:02.599493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e5e01-24df-4a4b-a260-7b9d529ff850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48891c9ba3774c0d9b9189af6c271f67)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:02.601437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e5e01-24df-4a4b-a260-7b9d529ff850 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:02.646367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:02.703100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cd1e5e01-24df-4a4b-a260-7b9d529ff850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48891c9ba3774c0d9b9189af6c271f67'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:02.714324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cd1e5e01-24df-4a4b-a260-7b9d529ff850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48891c9ba3774c0d9b9189af6c271f67'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:02.739353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cd1e5e01-24df-4a4b-a260-7b9d529ff850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48891c9ba3774c0d9b9189af6c271f67'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:02.741326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cd1e5e01-24df-4a4b-a260-7b9d529ff850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48891c9ba3774c0d9b9189af6c271f67'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:02.767119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cd1e5e01-24df-4a4b-a260-7b9d529ff850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48891c9ba3774c0d9b9189af6c271f67'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:02.776109 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2793/5512] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:08:02 2022] DELETE /identity/v3/users/48891c9ba3774c0d9b9189af6c271f67 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:08:02.779760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d041807-7624-45b0-925a-a3b324d92edd None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:02.781992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d041807-7624-45b0-925a-a3b324d92edd None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:02.782479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d041807-7624-45b0-925a-a3b324d92edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:02.824200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8d041807-7624-45b0-925a-a3b324d92edd None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:02.828738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8d041807-7624-45b0-925a-a3b324d92edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:03.051165 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2720/5513] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:08:03.090183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e5e01-24df-4a4b-a260-7b9d529ff850 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:03.091810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cd1e5e01-24df-4a4b-a260-7b9d529ff850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:03.207616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e5e01-24df-4a4b-a260-7b9d529ff850 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:03.219276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7630e-f932-4919-a308-622c378e6935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:03.221890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7630e-f932-4919-a308-622c378e6935 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:03.222876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7630e-f932-4919-a308-622c378e6935 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:03.225803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7630e-f932-4919-a308-622c378e6935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:03.251811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7630e-f932-4919-a308-622c378e6935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:03.253526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7630e-f932-4919-a308-622c378e6935 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:03.277316 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2794/5514] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:08:03.946904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d041807-7624-45b0-925a-a3b324d92edd None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:03.948696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8d041807-7624-45b0-925a-a3b324d92edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:03.972986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d041807-7624-45b0-925a-a3b324d92edd None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:03.995778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb91b80-b8a2-4ce1-9dc7-e0564842ad20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:03.999639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb91b80-b8a2-4ce1-9dc7-e0564842ad20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:04.001026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb91b80-b8a2-4ce1-9dc7-e0564842ad20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:04.003965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb91b80-b8a2-4ce1-9dc7-e0564842ad20 admin admin] PATH_INFO: `/v3/projects/dad97e3d163e4b939eded21322fbcebe` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:04.009711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb91b80-b8a2-4ce1-9dc7-e0564842ad20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dad97e3d163e4b939eded21322fbcebe)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:04.015116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb91b80-b8a2-4ce1-9dc7-e0564842ad20 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:04.156171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cfb91b80-b8a2-4ce1-9dc7-e0564842ad20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dad97e3d163e4b939eded21322fbcebe', 'request_id': 'req-cfb91b80-b8a2-4ce1-9dc7-e0564842ad20'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:04.175336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cfb91b80-b8a2-4ce1-9dc7-e0564842ad20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dad97e3d163e4b939eded21322fbcebe', 'request_id': 'req-cfb91b80-b8a2-4ce1-9dc7-e0564842ad20'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:04.189554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cfb91b80-b8a2-4ce1-9dc7-e0564842ad20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dad97e3d163e4b939eded21322fbcebe', 'request_id': 'req-cfb91b80-b8a2-4ce1-9dc7-e0564842ad20'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:04.217917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cfb91b80-b8a2-4ce1-9dc7-e0564842ad20 admin admin] The token cache is being invalidate because project dad97e3d163e4b939eded21322fbcebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:08:04.218613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cfb91b80-b8a2-4ce1-9dc7-e0564842ad20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:04.223326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cfb91b80-b8a2-4ce1-9dc7-e0564842ad20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:04.225217 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2721/5515] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:08:03 2022] DELETE /identity/v3/projects/dad97e3d163e4b939eded21322fbcebe => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:08:04.232393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad077f71-036b-4062-95c6-b85709621beb None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:04.233271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad077f71-036b-4062-95c6-b85709621beb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:04.233714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad077f71-036b-4062-95c6-b85709621beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:04.287372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ad077f71-036b-4062-95c6-b85709621beb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:04.288555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ad077f71-036b-4062-95c6-b85709621beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:04.420106 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2795/5516] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:08:04.421902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7675f-86ad-4487-8dcf-ad060011b54a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:04.422390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7675f-86ad-4487-8dcf-ad060011b54a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:04.423821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7675f-86ad-4487-8dcf-ad060011b54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:04.442098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad077f71-036b-4062-95c6-b85709621beb None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:04.443807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ad077f71-036b-4062-95c6-b85709621beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:04.463510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad077f71-036b-4062-95c6-b85709621beb None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:04.463894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-dfc7675f-86ad-4487-8dcf-ad060011b54a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:04.465133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-dfc7675f-86ad-4487-8dcf-ad060011b54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:04.469254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470338d3-93dc-4323-9c78-09cd00be1a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:04.470249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470338d3-93dc-4323-9c78-09cd00be1a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:04.470692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470338d3-93dc-4323-9c78-09cd00be1a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:04.471119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470338d3-93dc-4323-9c78-09cd00be1a26 admin admin] PATH_INFO: `/v3/users/148cdf56f4a14512aba0659e4c362959` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:04.472946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470338d3-93dc-4323-9c78-09cd00be1a26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=148cdf56f4a14512aba0659e4c362959)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:04.475697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470338d3-93dc-4323-9c78-09cd00be1a26 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:04.505745 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2722/5517] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:08:04.510746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7675f-86ad-4487-8dcf-ad060011b54a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:04.526794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7675f-86ad-4487-8dcf-ad060011b54a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:04.537874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9b3ba-23dc-4404-a8ba-3a7fd9af1d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:04.540354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9b3ba-23dc-4404-a8ba-3a7fd9af1d17 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:04.540963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9b3ba-23dc-4404-a8ba-3a7fd9af1d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:04.541469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9b3ba-23dc-4404-a8ba-3a7fd9af1d17 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:04.543721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9b3ba-23dc-4404-a8ba-3a7fd9af1d17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:08:04.544257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9b3ba-23dc-4404-a8ba-3a7fd9af1d17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:04.546567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9b3ba-23dc-4404-a8ba-3a7fd9af1d17 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:04.564651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:04.580457 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2723/5518] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:08:04.586076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9b3ba-23dc-4404-a8ba-3a7fd9af1d17 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:04.599321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-470338d3-93dc-4323-9c78-09cd00be1a26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '148cdf56f4a14512aba0659e4c362959'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:04.608697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9b3ba-23dc-4404-a8ba-3a7fd9af1d17 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:04.627143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b9597-04c8-4d5e-9cc8-4b4790d1c8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:04.627769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b9597-04c8-4d5e-9cc8-4b4790d1c8f7 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:04.628047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b9597-04c8-4d5e-9cc8-4b4790d1c8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:04.628320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b9597-04c8-4d5e-9cc8-4b4790d1c8f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:04.628960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b9597-04c8-4d5e-9cc8-4b4790d1c8f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:04.629818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-470338d3-93dc-4323-9c78-09cd00be1a26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '148cdf56f4a14512aba0659e4c362959'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:04.632502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b9597-04c8-4d5e-9cc8-4b4790d1c8f7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:04.687276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-470338d3-93dc-4323-9c78-09cd00be1a26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '148cdf56f4a14512aba0659e4c362959'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:04.688787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-470338d3-93dc-4323-9c78-09cd00be1a26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '148cdf56f4a14512aba0659e4c362959'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:04.709418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-470338d3-93dc-4323-9c78-09cd00be1a26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '148cdf56f4a14512aba0659e4c362959'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:04.727407 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2796/5519] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:08:04 2022] DELETE /identity/v3/users/148cdf56f4a14512aba0659e4c362959 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:08:04.744960 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2724/5520] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:08:04 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:08:04.752958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470338d3-93dc-4323-9c78-09cd00be1a26 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:04.755333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-470338d3-93dc-4323-9c78-09cd00be1a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:04.969210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470338d3-93dc-4323-9c78-09cd00be1a26 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:05.136022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f68f1-4366-4aba-88f3-d2ebd1121cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:05.136910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f68f1-4366-4aba-88f3-d2ebd1121cbc admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:05.137964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f68f1-4366-4aba-88f3-d2ebd1121cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:05.173454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f68f1-4366-4aba-88f3-d2ebd1121cbc admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:05.174709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f68f1-4366-4aba-88f3-d2ebd1121cbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:05.181780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f68f1-4366-4aba-88f3-d2ebd1121cbc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:05.264134 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2797/5521] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:08:04 2022] POST /identity/v3/users => generated 287 bytes in 514 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:08:05.267290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b9597-04c8-4d5e-9cc8-4b4790d1c8f7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:05.286615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b9597-04c8-4d5e-9cc8-4b4790d1c8f7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:05.298249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36b11c-a1b2-4dd1-ae1d-1fbdec4127b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:05.299477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36b11c-a1b2-4dd1-ae1d-1fbdec4127b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:05.299871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36b11c-a1b2-4dd1-ae1d-1fbdec4127b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:05.300271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36b11c-a1b2-4dd1-ae1d-1fbdec4127b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:05.301029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36b11c-a1b2-4dd1-ae1d-1fbdec4127b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:05.303313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36b11c-a1b2-4dd1-ae1d-1fbdec4127b6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:05.321399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f68f1-4366-4aba-88f3-d2ebd1121cbc admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:05.322885 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2725/5522] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:08:05.335631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f68f1-4366-4aba-88f3-d2ebd1121cbc admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:05.343422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e84527-5c20-4e88-a57d-a146269b471f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:05.344308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e84527-5c20-4e88-a57d-a146269b471f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:05.344875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e84527-5c20-4e88-a57d-a146269b471f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:05.345411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e84527-5c20-4e88-a57d-a146269b471f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:05.346301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e84527-5c20-4e88-a57d-a146269b471f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:05.355507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838af0c9-0e7a-45c9-ac97-e12b00a68b04 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:05.356066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838af0c9-0e7a-45c9-ac97-e12b00a68b04 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:05.356651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838af0c9-0e7a-45c9-ac97-e12b00a68b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:05.359917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e84527-5c20-4e88-a57d-a146269b471f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:05.369568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:05.383271 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2798/5523] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:08:05.386567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e84527-5c20-4e88-a57d-a146269b471f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:05.412554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e84527-5c20-4e88-a57d-a146269b471f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:05.428080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a870543-d242-4566-b4b6-f41998b2e43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:05.430416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a870543-d242-4566-b4b6-f41998b2e43c admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:05.431027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a870543-d242-4566-b4b6-f41998b2e43c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:05.431293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a870543-d242-4566-b4b6-f41998b2e43c admin admin] PATH_INFO: `/v3/projects/6be88826b8c149109fbd51bcf9740936/users/425d0c2af0ad424793fe3606153810b7/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:05.433964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-838af0c9-0e7a-45c9-ac97-e12b00a68b04 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:05.436832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-838af0c9-0e7a-45c9-ac97-e12b00a68b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:05.475708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a870543-d242-4566-b4b6-f41998b2e43c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6be88826b8c149109fbd51bcf9740936, user_id=425d0c2af0ad424793fe3606153810b7, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:05.486163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a870543-d242-4566-b4b6-f41998b2e43c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:05.515024 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2799/5524] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:08:05 2022] PUT /identity/v3/projects/6be88826b8c149109fbd51bcf9740936/users/425d0c2af0ad424793fe3606153810b7/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:08:05.516448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a870543-d242-4566-b4b6-f41998b2e43c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:05.521741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1a870543-d242-4566-b4b6-f41998b2e43c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:05.549279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:05.559031 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2726/5525] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:08:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:08:05.560892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f8b51-b7b9-4519-9d89-2f68d1448125 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:05.561322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f8b51-b7b9-4519-9d89-2f68d1448125 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:05.561698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f8b51-b7b9-4519-9d89-2f68d1448125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:05.632398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6f7f8b51-b7b9-4519-9d89-2f68d1448125 None None] MFA Rules not processed for user `425d0c2af0ad424793fe3606153810b7`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:05.635523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6f7f8b51-b7b9-4519-9d89-2f68d1448125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:05.660625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a870543-d242-4566-b4b6-f41998b2e43c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:05.671835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca00d14-75a0-4b74-9696-f7dd6c1ca99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:05.680884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca00d14-75a0-4b74-9696-f7dd6c1ca99a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:05.680884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca00d14-75a0-4b74-9696-f7dd6c1ca99a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:05.680884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca00d14-75a0-4b74-9696-f7dd6c1ca99a admin admin] PATH_INFO: `/v3/projects/7a8725bd62074b56b5712f469358728c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:05.680884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca00d14-75a0-4b74-9696-f7dd6c1ca99a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a8725bd62074b56b5712f469358728c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:05.680884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca00d14-75a0-4b74-9696-f7dd6c1ca99a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:05.754587 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2727/5526] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:08:05.756547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ca00d14-75a0-4b74-9696-f7dd6c1ca99a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a8725bd62074b56b5712f469358728c', 'request_id': 'req-2ca00d14-75a0-4b74-9696-f7dd6c1ca99a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:05.756841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f8b51-b7b9-4519-9d89-2f68d1448125 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:05.760974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6f7f8b51-b7b9-4519-9d89-2f68d1448125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:05.766523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ca00d14-75a0-4b74-9696-f7dd6c1ca99a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a8725bd62074b56b5712f469358728c', 'request_id': 'req-2ca00d14-75a0-4b74-9696-f7dd6c1ca99a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:05.787683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ca00d14-75a0-4b74-9696-f7dd6c1ca99a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a8725bd62074b56b5712f469358728c', 'request_id': 'req-2ca00d14-75a0-4b74-9696-f7dd6c1ca99a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:05.813240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ca00d14-75a0-4b74-9696-f7dd6c1ca99a admin admin] The token cache is being invalidate because project 7a8725bd62074b56b5712f469358728c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:08:05.813551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ca00d14-75a0-4b74-9696-f7dd6c1ca99a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:05.814237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ca00d14-75a0-4b74-9696-f7dd6c1ca99a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:05.815967 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2800/5527] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:08:05 2022] DELETE /identity/v3/projects/7a8725bd62074b56b5712f469358728c => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:08:05.816749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca00d14-75a0-4b74-9696-f7dd6c1ca99a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:05.823247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2ca00d14-75a0-4b74-9696-f7dd6c1ca99a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:05.943051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f8b51-b7b9-4519-9d89-2f68d1448125 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:05.963555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca00d14-75a0-4b74-9696-f7dd6c1ca99a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:05.969607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be4539-a895-487a-bf19-61d1343e534f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:05.970820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be4539-a895-487a-bf19-61d1343e534f service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:05.971215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be4539-a895-487a-bf19-61d1343e534f service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:05.971596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be4539-a895-487a-bf19-61d1343e534f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:05.975435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-79be4539-a895-487a-bf19-61d1343e534f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:06.078235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:06.140793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc11997-25ab-4ff6-8daf-afe23aadd5cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:06.143556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc11997-25ab-4ff6-8daf-afe23aadd5cb service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:06.143998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc11997-25ab-4ff6-8daf-afe23aadd5cb service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:06.144301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc11997-25ab-4ff6-8daf-afe23aadd5cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:06.149832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:06.151617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ecc11997-25ab-4ff6-8daf-afe23aadd5cb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:06.153862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be4539-a895-487a-bf19-61d1343e534f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:06.162216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be4539-a895-487a-bf19-61d1343e534f service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:06.184331 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2801/5528] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:08:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:08:06.187251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27de93-0745-4e7b-a39e-58e83465f197 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:06.187624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27de93-0745-4e7b-a39e-58e83465f197 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:06.190312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27de93-0745-4e7b-a39e-58e83465f197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:06.240340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0e27de93-0745-4e7b-a39e-58e83465f197 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:06.241574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0e27de93-0745-4e7b-a39e-58e83465f197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:06.288773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc11997-25ab-4ff6-8daf-afe23aadd5cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:06.294707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc11997-25ab-4ff6-8daf-afe23aadd5cb service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:06.326726 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2728/5529] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:08:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:08:06.420508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18173c4-3a59-47d4-8f0a-0f0e02e3df0e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:06.420508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18173c4-3a59-47d4-8f0a-0f0e02e3df0e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:06.421063 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2802/5530] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:08:06.421813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18173c4-3a59-47d4-8f0a-0f0e02e3df0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:06.463479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c18173c4-3a59-47d4-8f0a-0f0e02e3df0e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:06.465684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c18173c4-3a59-47d4-8f0a-0f0e02e3df0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:06.486044 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2729/5531] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:08:06.495843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27de93-0745-4e7b-a39e-58e83465f197 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:06.518733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27de93-0745-4e7b-a39e-58e83465f197 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:06.546961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384967ac-4a98-406f-aaca-0c09c872a4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:06.547616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384967ac-4a98-406f-aaca-0c09c872a4b3 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:06.547900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384967ac-4a98-406f-aaca-0c09c872a4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:06.548177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384967ac-4a98-406f-aaca-0c09c872a4b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:06.554059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384967ac-4a98-406f-aaca-0c09c872a4b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:08:06.554389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384967ac-4a98-406f-aaca-0c09c872a4b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:06.568637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384967ac-4a98-406f-aaca-0c09c872a4b3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:06.599786 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2803/5532] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:08:06.604179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18173c4-3a59-47d4-8f0a-0f0e02e3df0e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:06.618432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18173c4-3a59-47d4-8f0a-0f0e02e3df0e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:06.635178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2b9e2-970f-479f-aa51-971d922869aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:06.636030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2b9e2-970f-479f-aa51-971d922869aa admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:06.639448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2b9e2-970f-479f-aa51-971d922869aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:06.639851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2b9e2-970f-479f-aa51-971d922869aa admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:06.640694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2b9e2-970f-479f-aa51-971d922869aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:06.645784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2b9e2-970f-479f-aa51-971d922869aa admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:06.754478 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2730/5533] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:08:06 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:08:06.757045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384967ac-4a98-406f-aaca-0c09c872a4b3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:06.792906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384967ac-4a98-406f-aaca-0c09c872a4b3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:06.987820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8f1a2-96d7-4adc-a77d-2378acaaaf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:06.988535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8f1a2-96d7-4adc-a77d-2378acaaaf47 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:06.988816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8f1a2-96d7-4adc-a77d-2378acaaaf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:06.989192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8f1a2-96d7-4adc-a77d-2378acaaaf47 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:06.990049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd8f1a2-96d7-4adc-a77d-2378acaaaf47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:06.993286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd8f1a2-96d7-4adc-a77d-2378acaaaf47 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:07.049297 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2804/5534] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:08:06 2022] POST /identity/v3/users => generated 291 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:08:07.054419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2b9e2-970f-479f-aa51-971d922869aa admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:07.083169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2b9e2-970f-479f-aa51-971d922869aa admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:07.132301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34715b-12ee-4215-aeca-e75d324fb71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:07.138046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34715b-12ee-4215-aeca-e75d324fb71d admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:07.146272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34715b-12ee-4215-aeca-e75d324fb71d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:07.147109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34715b-12ee-4215-aeca-e75d324fb71d admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:07.150369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34715b-12ee-4215-aeca-e75d324fb71d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:07.164948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34715b-12ee-4215-aeca-e75d324fb71d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:07.239629 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2731/5535] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:08:07.244627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8f1a2-96d7-4adc-a77d-2378acaaaf47 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:07.267124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8f1a2-96d7-4adc-a77d-2378acaaaf47 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:07.295506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8ed18-e32d-4161-810d-969b84563a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:07.295506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8ed18-e32d-4161-810d-969b84563a26 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:07.295506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8ed18-e32d-4161-810d-969b84563a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:07.295506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8ed18-e32d-4161-810d-969b84563a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:07.297200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8ed18-e32d-4161-810d-969b84563a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:07.297200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8ed18-e32d-4161-810d-969b84563a26 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:07.325454 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2805/5536] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:08:07.330319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34715b-12ee-4215-aeca-e75d324fb71d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:07.363775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34715b-12ee-4215-aeca-e75d324fb71d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:07.405746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72260612-7141-4399-ab63-58706adff623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:07.422837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72260612-7141-4399-ab63-58706adff623 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:07.424578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72260612-7141-4399-ab63-58706adff623 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:07.425122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72260612-7141-4399-ab63-58706adff623 admin admin] PATH_INFO: `/v3/projects/337bb3ce6f304c58902406f7279af3f1/users/d2a67ba740284d5589df8bdad6ab671f/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:07.497580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72260612-7141-4399-ab63-58706adff623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=337bb3ce6f304c58902406f7279af3f1, user_id=d2a67ba740284d5589df8bdad6ab671f, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:07.505442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72260612-7141-4399-ab63-58706adff623 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:07.546514 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2732/5537] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:08:07 2022] PUT /identity/v3/projects/337bb3ce6f304c58902406f7279af3f1/users/d2a67ba740284d5589df8bdad6ab671f/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:08:07.553735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec514ca-e782-46aa-8e78-d358c25662f3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:07.555357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec514ca-e782-46aa-8e78-d358c25662f3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:07.555918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec514ca-e782-46aa-8e78-d358c25662f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:07.591808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5ec514ca-e782-46aa-8e78-d358c25662f3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:07.598737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5ec514ca-e782-46aa-8e78-d358c25662f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:07.749289 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2806/5538] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:08:07.761625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72260612-7141-4399-ab63-58706adff623 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:07.764820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-72260612-7141-4399-ab63-58706adff623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:07.872280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72260612-7141-4399-ab63-58706adff623 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:07.878319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4c55c-b077-4256-a027-4243e66f944f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:07.878809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4c55c-b077-4256-a027-4243e66f944f service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:07.879208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4c55c-b077-4256-a027-4243e66f944f service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:07.879680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4c55c-b077-4256-a027-4243e66f944f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:07.904562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e4c55c-b077-4256-a027-4243e66f944f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:07.908763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e4c55c-b077-4256-a027-4243e66f944f service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:07.929468 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2733/5539] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:08:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:08:11.624971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739a56f-42ed-46a5-b710-1754387ea28d None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:11.625949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739a56f-42ed-46a5-b710-1754387ea28d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:11.626416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739a56f-42ed-46a5-b710-1754387ea28d None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:11.628149 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2807/5540] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 21:08:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 21:08:11.636912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878b82a-56ee-46bf-8925-b2e05bb6fdb8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:11.637511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878b82a-56ee-46bf-8925-b2e05bb6fdb8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:11.637511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878b82a-56ee-46bf-8925-b2e05bb6fdb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:11.680256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:11.711167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3878b82a-56ee-46bf-8925-b2e05bb6fdb8 None None] MFA Rules not processed for user `e17a026f305248878f18eec35e1e02dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:11.712521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3878b82a-56ee-46bf-8925-b2e05bb6fdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:11.724560 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2734/5541] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:08:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:08:11.753353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739a56f-42ed-46a5-b710-1754387ea28d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:11.761378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:11.782662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739a56f-42ed-46a5-b710-1754387ea28d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:11.788012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4f934-d5c2-484d-96b2-382314a2d4c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:11.791388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4f934-d5c2-484d-96b2-382314a2d4c9 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:11.792226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4f934-d5c2-484d-96b2-382314a2d4c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:11.792778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4f934-d5c2-484d-96b2-382314a2d4c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:11.806348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4f934-d5c2-484d-96b2-382314a2d4c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:11.808319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4f934-d5c2-484d-96b2-382314a2d4c9 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:11.823621 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2808/5542] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:08:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:08:21.706267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:21.816745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:22.736821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878b82a-56ee-46bf-8925-b2e05bb6fdb8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:22.739005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3878b82a-56ee-46bf-8925-b2e05bb6fdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:22.775610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878b82a-56ee-46bf-8925-b2e05bb6fdb8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:22.789244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db7b9f-cd3f-4919-8a01-c044eb63970a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:22.790880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db7b9f-cd3f-4919-8a01-c044eb63970a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:22.793551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db7b9f-cd3f-4919-8a01-c044eb63970a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:22.794057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db7b9f-cd3f-4919-8a01-c044eb63970a admin admin] PATH_INFO: `/v3/users/e7e3bc6d5e45497bb876d5cbcec61835` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:22.796246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db7b9f-cd3f-4919-8a01-c044eb63970a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7e3bc6d5e45497bb876d5cbcec61835)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:22.801166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db7b9f-cd3f-4919-8a01-c044eb63970a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:22.951031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d8db7b9f-cd3f-4919-8a01-c044eb63970a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7e3bc6d5e45497bb876d5cbcec61835'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:22.952623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d8db7b9f-cd3f-4919-8a01-c044eb63970a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7e3bc6d5e45497bb876d5cbcec61835'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:22.953961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d8db7b9f-cd3f-4919-8a01-c044eb63970a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7e3bc6d5e45497bb876d5cbcec61835'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:22.971548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d8db7b9f-cd3f-4919-8a01-c044eb63970a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7e3bc6d5e45497bb876d5cbcec61835'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:22.995594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d8db7b9f-cd3f-4919-8a01-c044eb63970a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7e3bc6d5e45497bb876d5cbcec61835'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:23.025007 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2735/5543] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:08:22 2022] DELETE /identity/v3/users/e7e3bc6d5e45497bb876d5cbcec61835 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:08:24.144373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4f934-d5c2-484d-96b2-382314a2d4c9 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:24.173559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4f934-d5c2-484d-96b2-382314a2d4c9 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:24.359137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20214f62-3a74-4934-83f9-ad1df43852e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:24.360260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20214f62-3a74-4934-83f9-ad1df43852e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:24.360796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20214f62-3a74-4934-83f9-ad1df43852e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:24.362416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20214f62-3a74-4934-83f9-ad1df43852e3 admin admin] PATH_INFO: `/v3/projects/a2d2517d51d0417880e6e3d601c52744` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:24.363890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20214f62-3a74-4934-83f9-ad1df43852e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2d2517d51d0417880e6e3d601c52744)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:24.366218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20214f62-3a74-4934-83f9-ad1df43852e3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:24.504204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20214f62-3a74-4934-83f9-ad1df43852e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2d2517d51d0417880e6e3d601c52744', 'request_id': 'req-20214f62-3a74-4934-83f9-ad1df43852e3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:24.520214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20214f62-3a74-4934-83f9-ad1df43852e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2d2517d51d0417880e6e3d601c52744', 'request_id': 'req-20214f62-3a74-4934-83f9-ad1df43852e3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:24.538091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20214f62-3a74-4934-83f9-ad1df43852e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2d2517d51d0417880e6e3d601c52744', 'request_id': 'req-20214f62-3a74-4934-83f9-ad1df43852e3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:24.581306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20214f62-3a74-4934-83f9-ad1df43852e3 admin admin] The token cache is being invalidate because project a2d2517d51d0417880e6e3d601c52744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:08:24.581623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20214f62-3a74-4934-83f9-ad1df43852e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:24.584130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-20214f62-3a74-4934-83f9-ad1df43852e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:24.587618 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2809/5544] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:08:24 2022] DELETE /identity/v3/projects/a2d2517d51d0417880e6e3d601c52744 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:08:24.592200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db7b9f-cd3f-4919-8a01-c044eb63970a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:24.593892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d8db7b9f-cd3f-4919-8a01-c044eb63970a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:24.623930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db7b9f-cd3f-4919-8a01-c044eb63970a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:24.820582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b3d2b-f026-478f-967b-3ccf83a31e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:24.823255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b3d2b-f026-478f-967b-3ccf83a31e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:24.823707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b3d2b-f026-478f-967b-3ccf83a31e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:24.824205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b3d2b-f026-478f-967b-3ccf83a31e27 admin admin] PATH_INFO: `/v3/users/af3bc6e921b6454496accdab2890bfc3` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:24.828492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b3d2b-f026-478f-967b-3ccf83a31e27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af3bc6e921b6454496accdab2890bfc3)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:24.831909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b3d2b-f026-478f-967b-3ccf83a31e27 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:24.997362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-513b3d2b-f026-478f-967b-3ccf83a31e27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af3bc6e921b6454496accdab2890bfc3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:25.000284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-513b3d2b-f026-478f-967b-3ccf83a31e27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af3bc6e921b6454496accdab2890bfc3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:25.001395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-513b3d2b-f026-478f-967b-3ccf83a31e27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af3bc6e921b6454496accdab2890bfc3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:25.030579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-513b3d2b-f026-478f-967b-3ccf83a31e27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af3bc6e921b6454496accdab2890bfc3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:25.072011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-513b3d2b-f026-478f-967b-3ccf83a31e27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af3bc6e921b6454496accdab2890bfc3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:25.075119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:25.098227 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2736/5545] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:08:24 2022] DELETE /identity/v3/users/af3bc6e921b6454496accdab2890bfc3 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:08:25.531391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20214f62-3a74-4934-83f9-ad1df43852e3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:25.532844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-20214f62-3a74-4934-83f9-ad1df43852e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:25.560235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20214f62-3a74-4934-83f9-ad1df43852e3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:25.676525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba00c7a-4dde-4acd-8be8-f82709d91bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:25.680228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba00c7a-4dde-4acd-8be8-f82709d91bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:25.680779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba00c7a-4dde-4acd-8be8-f82709d91bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:25.684590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba00c7a-4dde-4acd-8be8-f82709d91bfe admin admin] PATH_INFO: `/v3/projects/478d0fd7c1664a6cbd61419bbb335beb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:25.687061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba00c7a-4dde-4acd-8be8-f82709d91bfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=478d0fd7c1664a6cbd61419bbb335beb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:25.694506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba00c7a-4dde-4acd-8be8-f82709d91bfe admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:25.781890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:25.830386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5ba00c7a-4dde-4acd-8be8-f82709d91bfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '478d0fd7c1664a6cbd61419bbb335beb', 'request_id': 'req-5ba00c7a-4dde-4acd-8be8-f82709d91bfe'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:25.843425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5ba00c7a-4dde-4acd-8be8-f82709d91bfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '478d0fd7c1664a6cbd61419bbb335beb', 'request_id': 'req-5ba00c7a-4dde-4acd-8be8-f82709d91bfe'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:25.859507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5ba00c7a-4dde-4acd-8be8-f82709d91bfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '478d0fd7c1664a6cbd61419bbb335beb', 'request_id': 'req-5ba00c7a-4dde-4acd-8be8-f82709d91bfe'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:25.889193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5ba00c7a-4dde-4acd-8be8-f82709d91bfe admin admin] The token cache is being invalidate because project 478d0fd7c1664a6cbd61419bbb335beb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:08:25.889740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5ba00c7a-4dde-4acd-8be8-f82709d91bfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:25.892897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5ba00c7a-4dde-4acd-8be8-f82709d91bfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:25.896259 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2810/5546] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:08:25 2022] DELETE /identity/v3/projects/478d0fd7c1664a6cbd61419bbb335beb => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:08:25.900041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6625a7b-dff8-4149-a99f-11ab5d746d16 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:25.900546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6625a7b-dff8-4149-a99f-11ab5d746d16 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:25.900650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6625a7b-dff8-4149-a99f-11ab5d746d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:25.906916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60004309-0872-4225-818a-b94a28fe6ab5 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:25.907699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60004309-0872-4225-818a-b94a28fe6ab5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:25.908179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60004309-0872-4225-818a-b94a28fe6ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:25.984204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-60004309-0872-4225-818a-b94a28fe6ab5 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:25.988426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-60004309-0872-4225-818a-b94a28fe6ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:25.994166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d6625a7b-dff8-4149-a99f-11ab5d746d16 None None] MFA Rules not processed for user `d2a67ba740284d5589df8bdad6ab671f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:25.995780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d6625a7b-dff8-4149-a99f-11ab5d746d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:26.161572 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2811/5547] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:08:26.170877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b74718-07a3-4714-9264-276e9cf7a6dc None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:26.171216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b74718-07a3-4714-9264-276e9cf7a6dc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:26.171828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b74718-07a3-4714-9264-276e9cf7a6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:26.217528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d5b74718-07a3-4714-9264-276e9cf7a6dc None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:26.218767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d5b74718-07a3-4714-9264-276e9cf7a6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:26.229679 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2812/5548] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:08:26.237266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b74718-07a3-4714-9264-276e9cf7a6dc None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:26.249119 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2737/5549] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:08:26.250135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6625a7b-dff8-4149-a99f-11ab5d746d16 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:26.252510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d6625a7b-dff8-4149-a99f-11ab5d746d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:26.287742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b74718-07a3-4714-9264-276e9cf7a6dc None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:26.320620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b49d84-69ac-4b03-8617-0693ed4eeb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:26.323469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b49d84-69ac-4b03-8617-0693ed4eeb4a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:26.323751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b49d84-69ac-4b03-8617-0693ed4eeb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:26.324080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b49d84-69ac-4b03-8617-0693ed4eeb4a admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:26.324755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b49d84-69ac-4b03-8617-0693ed4eeb4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:08:26.325202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b49d84-69ac-4b03-8617-0693ed4eeb4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:26.331292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b49d84-69ac-4b03-8617-0693ed4eeb4a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:26.362378 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2813/5550] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:08:26.364030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b49d84-69ac-4b03-8617-0693ed4eeb4a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:26.388892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b49d84-69ac-4b03-8617-0693ed4eeb4a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:26.405199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbac434-0323-4c9a-806d-585b3e4aaa18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:26.405825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbac434-0323-4c9a-806d-585b3e4aaa18 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:26.406105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbac434-0323-4c9a-806d-585b3e4aaa18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:26.408135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbac434-0323-4c9a-806d-585b3e4aaa18 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:26.409044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbac434-0323-4c9a-806d-585b3e4aaa18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:26.414620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbac434-0323-4c9a-806d-585b3e4aaa18 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:26.478203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6625a7b-dff8-4149-a99f-11ab5d746d16 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:26.504517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a571e-9561-475d-9bef-86e7dce638f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:26.506126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a571e-9561-475d-9bef-86e7dce638f9 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:26.506410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a571e-9561-475d-9bef-86e7dce638f9 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:26.506709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a571e-9561-475d-9bef-86e7dce638f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:26.518046 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2814/5551] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:08:26 2022] POST /identity/v3/projects => generated 387 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:08:26.524005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbac434-0323-4c9a-806d-585b3e4aaa18 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:26.560550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291a571e-9561-475d-9bef-86e7dce638f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:26.566477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291a571e-9561-475d-9bef-86e7dce638f9 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:26.608731 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2738/5552] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:08:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:08:26.754973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbac434-0323-4c9a-806d-585b3e4aaa18 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:26.768211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbafd96-19aa-4da9-83b6-ece8c156ff82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:26.769150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbafd96-19aa-4da9-83b6-ece8c156ff82 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:26.769529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbafd96-19aa-4da9-83b6-ece8c156ff82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:26.769900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbafd96-19aa-4da9-83b6-ece8c156ff82 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:26.771085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbafd96-19aa-4da9-83b6-ece8c156ff82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:26.776597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbafd96-19aa-4da9-83b6-ece8c156ff82 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:26.843322 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2815/5553] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:08:26 2022] POST /identity/v3/users => generated 294 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:08:26.845537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a571e-9561-475d-9bef-86e7dce638f9 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:26.873440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a571e-9561-475d-9bef-86e7dce638f9 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:26.888003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc063f33-e4f0-48d3-9110-f34491568210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:26.890093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc063f33-e4f0-48d3-9110-f34491568210 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:26.891299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc063f33-e4f0-48d3-9110-f34491568210 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:26.891766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc063f33-e4f0-48d3-9110-f34491568210 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:26.892444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc063f33-e4f0-48d3-9110-f34491568210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:26.901104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc063f33-e4f0-48d3-9110-f34491568210 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:26.946641 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2739/5554] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:08:26.947818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbafd96-19aa-4da9-83b6-ece8c156ff82 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:26.963999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbafd96-19aa-4da9-83b6-ece8c156ff82 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:26.997090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af729cf7-1ac7-4ab4-85c8-67d459e3a635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:26.997732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af729cf7-1ac7-4ab4-85c8-67d459e3a635 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:26.998010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af729cf7-1ac7-4ab4-85c8-67d459e3a635 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:27.010574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af729cf7-1ac7-4ab4-85c8-67d459e3a635 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:27.010574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af729cf7-1ac7-4ab4-85c8-67d459e3a635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:27.012554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af729cf7-1ac7-4ab4-85c8-67d459e3a635 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:27.021366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:27.046669 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2816/5555] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:08:27.048407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc063f33-e4f0-48d3-9110-f34491568210 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:27.079834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc063f33-e4f0-48d3-9110-f34491568210 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:27.104518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df35e6-a12e-4e72-be8e-ba6b3d301f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:27.107460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df35e6-a12e-4e72-be8e-ba6b3d301f3a admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:27.107749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df35e6-a12e-4e72-be8e-ba6b3d301f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:27.108038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df35e6-a12e-4e72-be8e-ba6b3d301f3a admin admin] PATH_INFO: `/v3/projects/fa37082733634ec688c9d6272a0d9b1d/users/8d79dff848bf40fc89377e13736c527b/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:27.188440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df35e6-a12e-4e72-be8e-ba6b3d301f3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa37082733634ec688c9d6272a0d9b1d, user_id=8d79dff848bf40fc89377e13736c527b, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:27.199322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df35e6-a12e-4e72-be8e-ba6b3d301f3a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:27.231643 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2740/5556] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:08:27 2022] PUT /identity/v3/projects/fa37082733634ec688c9d6272a0d9b1d/users/8d79dff848bf40fc89377e13736c527b/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:08:27.233120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3774c-d527-4b01-a11a-958c1e3a940a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:27.233425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3774c-d527-4b01-a11a-958c1e3a940a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:27.233697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3774c-d527-4b01-a11a-958c1e3a940a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:27.255899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-add3774c-d527-4b01-a11a-958c1e3a940a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:27.258234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-add3774c-d527-4b01-a11a-958c1e3a940a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:27.384484 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2817/5557] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:08:27.490208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df35e6-a12e-4e72-be8e-ba6b3d301f3a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:27.492117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-69df35e6-a12e-4e72-be8e-ba6b3d301f3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:27.565945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:27.607398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df35e6-a12e-4e72-be8e-ba6b3d301f3a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:27.614884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6e0fd-8ce3-4b32-b92e-ee5bb278f158 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:27.616850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6e0fd-8ce3-4b32-b92e-ee5bb278f158 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:27.617138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6e0fd-8ce3-4b32-b92e-ee5bb278f158 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:27.617407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6e0fd-8ce3-4b32-b92e-ee5bb278f158 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:27.643488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6e0fd-8ce3-4b32-b92e-ee5bb278f158 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:27.648213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6e0fd-8ce3-4b32-b92e-ee5bb278f158 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:27.676653 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2741/5558] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:08:28.240709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c018c-b8e0-4ba4-99a9-448c791d941f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:28.241382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c018c-b8e0-4ba4-99a9-448c791d941f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:28.242191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c018c-b8e0-4ba4-99a9-448c791d941f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:28.301757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-782c018c-b8e0-4ba4-99a9-448c791d941f None None] MFA Rules not processed for user `d2a67ba740284d5589df8bdad6ab671f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:28.305318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-782c018c-b8e0-4ba4-99a9-448c791d941f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:28.459592 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2818/5559] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:08:29.618039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6e0fd-8ce3-4b32-b92e-ee5bb278f158 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:29.622050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-20d6e0fd-8ce3-4b32-b92e-ee5bb278f158 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:29.646530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6e0fd-8ce3-4b32-b92e-ee5bb278f158 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:29.658752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1126592-0c24-4144-bdbc-13ee51bd6daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:29.659138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1126592-0c24-4144-bdbc-13ee51bd6daa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:29.659349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1126592-0c24-4144-bdbc-13ee51bd6daa admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:29.659632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1126592-0c24-4144-bdbc-13ee51bd6daa admin admin] PATH_INFO: `/v3/users/425d0c2af0ad424793fe3606153810b7` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:29.663666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1126592-0c24-4144-bdbc-13ee51bd6daa admin admin] RBAC: Authorizing `identity:delete_user(user_id=425d0c2af0ad424793fe3606153810b7)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:29.672432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1126592-0c24-4144-bdbc-13ee51bd6daa admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:29.778036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1126592-0c24-4144-bdbc-13ee51bd6daa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '425d0c2af0ad424793fe3606153810b7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:29.779287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1126592-0c24-4144-bdbc-13ee51bd6daa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '425d0c2af0ad424793fe3606153810b7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:29.781498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1126592-0c24-4144-bdbc-13ee51bd6daa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '425d0c2af0ad424793fe3606153810b7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:29.797411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1126592-0c24-4144-bdbc-13ee51bd6daa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '425d0c2af0ad424793fe3606153810b7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:29.840071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1126592-0c24-4144-bdbc-13ee51bd6daa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '425d0c2af0ad424793fe3606153810b7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:29.872439 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2742/5560] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:08:29 2022] DELETE /identity/v3/users/425d0c2af0ad424793fe3606153810b7 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:08:29.874271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad61a3-de4b-4d76-b4ea-95095584a1ba None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:29.876628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad61a3-de4b-4d76-b4ea-95095584a1ba None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:29.877765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad61a3-de4b-4d76-b4ea-95095584a1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:29.940952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-57ad61a3-de4b-4d76-b4ea-95095584a1ba None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:29.942728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-57ad61a3-de4b-4d76-b4ea-95095584a1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:30.096283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1126592-0c24-4144-bdbc-13ee51bd6daa admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:30.098581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d1126592-0c24-4144-bdbc-13ee51bd6daa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:30.102534 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2819/5561] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:08:30.211455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1126592-0c24-4144-bdbc-13ee51bd6daa admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:30.224373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34dc2a-7af3-4b7d-a599-8739600def84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:30.226785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34dc2a-7af3-4b7d-a599-8739600def84 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:30.227920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34dc2a-7af3-4b7d-a599-8739600def84 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:30.228440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34dc2a-7af3-4b7d-a599-8739600def84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:30.258451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff34dc2a-7af3-4b7d-a599-8739600def84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:30.261197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff34dc2a-7af3-4b7d-a599-8739600def84 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:30.328737 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2743/5562] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:08:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:08:31.233568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad61a3-de4b-4d76-b4ea-95095584a1ba None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:31.235912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-57ad61a3-de4b-4d76-b4ea-95095584a1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:31.267703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad61a3-de4b-4d76-b4ea-95095584a1ba None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:31.291210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f36fae-22fd-4987-80a0-41a652b16f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:31.291950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f36fae-22fd-4987-80a0-41a652b16f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:31.292405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f36fae-22fd-4987-80a0-41a652b16f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:31.293176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f36fae-22fd-4987-80a0-41a652b16f39 admin admin] PATH_INFO: `/v3/projects/6be88826b8c149109fbd51bcf9740936` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:31.302151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f36fae-22fd-4987-80a0-41a652b16f39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6be88826b8c149109fbd51bcf9740936)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:31.305194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f36fae-22fd-4987-80a0-41a652b16f39 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:31.430608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:31.443335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-03f36fae-22fd-4987-80a0-41a652b16f39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6be88826b8c149109fbd51bcf9740936', 'request_id': 'req-03f36fae-22fd-4987-80a0-41a652b16f39'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:31.452225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-03f36fae-22fd-4987-80a0-41a652b16f39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6be88826b8c149109fbd51bcf9740936', 'request_id': 'req-03f36fae-22fd-4987-80a0-41a652b16f39'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:31.460003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-03f36fae-22fd-4987-80a0-41a652b16f39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6be88826b8c149109fbd51bcf9740936', 'request_id': 'req-03f36fae-22fd-4987-80a0-41a652b16f39'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:31.499060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-03f36fae-22fd-4987-80a0-41a652b16f39 admin admin] The token cache is being invalidate because project 6be88826b8c149109fbd51bcf9740936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:08:31.499602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-03f36fae-22fd-4987-80a0-41a652b16f39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:31.502075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-03f36fae-22fd-4987-80a0-41a652b16f39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:31.508139 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2820/5563] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:08:31 2022] DELETE /identity/v3/projects/6be88826b8c149109fbd51bcf9740936 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:08:31.513213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67f00f-a44c-4353-bae0-4dff6dc469f6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:31.513764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67f00f-a44c-4353-bae0-4dff6dc469f6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:31.513764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67f00f-a44c-4353-bae0-4dff6dc469f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:31.594118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1a67f00f-a44c-4353-bae0-4dff6dc469f6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:31.601366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1a67f00f-a44c-4353-bae0-4dff6dc469f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:31.755689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:31.805820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934283e-d2f9-4956-8304-d4366c541eed None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:31.806112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934283e-d2f9-4956-8304-d4366c541eed None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:31.806393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934283e-d2f9-4956-8304-d4366c541eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:31.816072 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2744/5564] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:08:31.846685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e934283e-d2f9-4956-8304-d4366c541eed None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:31.847914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e934283e-d2f9-4956-8304-d4366c541eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:31.861497 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2821/5565] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:08:31.862064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67f00f-a44c-4353-bae0-4dff6dc469f6 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:31.881204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67f00f-a44c-4353-bae0-4dff6dc469f6 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:31.886628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b950f-2387-4e2b-be2f-03b9f9c3220a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:31.887382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b950f-2387-4e2b-be2f-03b9f9c3220a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:31.887760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b950f-2387-4e2b-be2f-03b9f9c3220a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:31.888129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b950f-2387-4e2b-be2f-03b9f9c3220a admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:31.888881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b950f-2387-4e2b-be2f-03b9f9c3220a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:08:31.889397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b950f-2387-4e2b-be2f-03b9f9c3220a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:31.892166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b950f-2387-4e2b-be2f-03b9f9c3220a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:31.913799 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2745/5566] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:08:31.915300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934283e-d2f9-4956-8304-d4366c541eed None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:31.928533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934283e-d2f9-4956-8304-d4366c541eed None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:31.945407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2963b-390a-4c6b-9933-4affc861d687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:31.946046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2963b-390a-4c6b-9933-4affc861d687 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:31.946330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2963b-390a-4c6b-9933-4affc861d687 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:31.946636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2963b-390a-4c6b-9933-4affc861d687 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:31.947284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2963b-390a-4c6b-9933-4affc861d687 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:31.949224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2963b-390a-4c6b-9933-4affc861d687 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:32.001474 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2822/5567] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:08:31 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:08:32.002179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b950f-2387-4e2b-be2f-03b9f9c3220a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:32.017234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b950f-2387-4e2b-be2f-03b9f9c3220a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:32.152547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b6fd2-ab83-449f-86d1-11abe1fb5894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:32.154211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b6fd2-ab83-449f-86d1-11abe1fb5894 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:32.154518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b6fd2-ab83-449f-86d1-11abe1fb5894 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:32.155351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b6fd2-ab83-449f-86d1-11abe1fb5894 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:32.156102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b6fd2-ab83-449f-86d1-11abe1fb5894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:32.158258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b6fd2-ab83-449f-86d1-11abe1fb5894 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:32.192483 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2746/5568] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:08:32 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:08:32.195656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2963b-390a-4c6b-9933-4affc861d687 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:32.209493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2963b-390a-4c6b-9933-4affc861d687 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:32.215793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34231fd5-6bad-4a6e-9034-6f7eb927ab21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:32.216935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34231fd5-6bad-4a6e-9034-6f7eb927ab21 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:32.217424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34231fd5-6bad-4a6e-9034-6f7eb927ab21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:32.217914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34231fd5-6bad-4a6e-9034-6f7eb927ab21 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:32.219092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34231fd5-6bad-4a6e-9034-6f7eb927ab21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:32.221151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34231fd5-6bad-4a6e-9034-6f7eb927ab21 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:32.257647 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2823/5569] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:08:32.261508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b6fd2-ab83-449f-86d1-11abe1fb5894 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:32.276282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b6fd2-ab83-449f-86d1-11abe1fb5894 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:32.296958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5eb68-e2a4-4a4c-9cdc-85f818a4414d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:32.297594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5eb68-e2a4-4a4c-9cdc-85f818a4414d admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:32.297881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5eb68-e2a4-4a4c-9cdc-85f818a4414d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:32.298158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5eb68-e2a4-4a4c-9cdc-85f818a4414d admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:32.298863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f5eb68-e2a4-4a4c-9cdc-85f818a4414d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:32.301587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f5eb68-e2a4-4a4c-9cdc-85f818a4414d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:32.326799 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2747/5570] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:08:32.330865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34231fd5-6bad-4a6e-9034-6f7eb927ab21 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:32.348031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34231fd5-6bad-4a6e-9034-6f7eb927ab21 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:32.363880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1f764-8996-4f37-8aec-0c1f0d7ffbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:32.364929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1f764-8996-4f37-8aec-0c1f0d7ffbb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:32.365111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1f764-8996-4f37-8aec-0c1f0d7ffbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:32.365341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1f764-8996-4f37-8aec-0c1f0d7ffbb7 admin admin] PATH_INFO: `/v3/projects/2be32d5a16f7453c96de3dbdb05fe6d1/users/e683d459ead14ddca5acb36ae9c1cbed/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:32.433321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1f764-8996-4f37-8aec-0c1f0d7ffbb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2be32d5a16f7453c96de3dbdb05fe6d1, user_id=e683d459ead14ddca5acb36ae9c1cbed, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:32.442464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1f764-8996-4f37-8aec-0c1f0d7ffbb7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:32.487376 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2824/5571] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:08:32 2022] PUT /identity/v3/projects/2be32d5a16f7453c96de3dbdb05fe6d1/users/e683d459ead14ddca5acb36ae9c1cbed/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:08:32.497851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2026cd-fbe9-4d0b-964c-7291fcc484a6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:32.498320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2026cd-fbe9-4d0b-964c-7291fcc484a6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:32.498436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2026cd-fbe9-4d0b-964c-7291fcc484a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:32.561713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9f2026cd-fbe9-4d0b-964c-7291fcc484a6 None None] MFA Rules not processed for user `e683d459ead14ddca5acb36ae9c1cbed`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:32.563075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9f2026cd-fbe9-4d0b-964c-7291fcc484a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:32.823339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1f764-8996-4f37-8aec-0c1f0d7ffbb7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:32.823558 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2748/5572] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:08:33.090618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1f764-8996-4f37-8aec-0c1f0d7ffbb7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:33.110580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8d1c7-7970-4a51-a234-597b5041008d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:33.111213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8d1c7-7970-4a51-a234-597b5041008d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:33.111500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8d1c7-7970-4a51-a234-597b5041008d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:33.111775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8d1c7-7970-4a51-a234-597b5041008d admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:33.112479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8d1c7-7970-4a51-a234-597b5041008d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:33.114436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8d1c7-7970-4a51-a234-597b5041008d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:33.188123 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2825/5573] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:08:32 2022] POST /identity/v3/projects => generated 373 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:08:33.192645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2026cd-fbe9-4d0b-964c-7291fcc484a6 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:33.218118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2026cd-fbe9-4d0b-964c-7291fcc484a6 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:33.313037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:33.335655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e2edf-931f-49e2-96c5-33dcfb8ec6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:33.336292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e2edf-931f-49e2-96c5-33dcfb8ec6ea admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:33.336569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e2edf-931f-49e2-96c5-33dcfb8ec6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:33.336847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e2edf-931f-49e2-96c5-33dcfb8ec6ea admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:33.337537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e2edf-931f-49e2-96c5-33dcfb8ec6ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:33.341081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e2edf-931f-49e2-96c5-33dcfb8ec6ea admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:33.383264 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2749/5574] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:08:33 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:08:33.384522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8d1c7-7970-4a51-a234-597b5041008d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:33.399226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8d1c7-7970-4a51-a234-597b5041008d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:33.404187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b49fb-fba5-4b26-8e1d-54a445aa51eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:33.405913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b49fb-fba5-4b26-8e1d-54a445aa51eb admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:33.406455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b49fb-fba5-4b26-8e1d-54a445aa51eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:33.407238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b49fb-fba5-4b26-8e1d-54a445aa51eb admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:33.408617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b49fb-fba5-4b26-8e1d-54a445aa51eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:33.410423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b49fb-fba5-4b26-8e1d-54a445aa51eb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:33.429579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e2edf-931f-49e2-96c5-33dcfb8ec6ea admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:33.445356 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2826/5575] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:08:33.448066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e2edf-931f-49e2-96c5-33dcfb8ec6ea admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:33.455402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c56fa-a52f-4c29-9a09-eedc570869da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:33.457687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c56fa-a52f-4c29-9a09-eedc570869da admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:33.457687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c56fa-a52f-4c29-9a09-eedc570869da admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:33.458183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c56fa-a52f-4c29-9a09-eedc570869da admin admin] PATH_INFO: `/v3/projects/6c0c5a6199ae4f5d9c11af7f0fe94632/users/02cd0f6eeb904eebaadc8f6d1c1da5bd/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:33.478735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c56fa-a52f-4c29-9a09-eedc570869da admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c0c5a6199ae4f5d9c11af7f0fe94632, user_id=02cd0f6eeb904eebaadc8f6d1c1da5bd, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:33.481918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c56fa-a52f-4c29-9a09-eedc570869da admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:33.529811 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2750/5576] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:08:33 2022] PUT /identity/v3/projects/6c0c5a6199ae4f5d9c11af7f0fe94632/users/02cd0f6eeb904eebaadc8f6d1c1da5bd/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:08:33.532029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b49fb-fba5-4b26-8e1d-54a445aa51eb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:33.548135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b49fb-fba5-4b26-8e1d-54a445aa51eb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:33.651291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:33.713392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba06601-6f45-475e-bfd2-bc41d37ee241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:33.714049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba06601-6f45-475e-bfd2-bc41d37ee241 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:33.714312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba06601-6f45-475e-bfd2-bc41d37ee241 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:33.714621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba06601-6f45-475e-bfd2-bc41d37ee241 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:33.715279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba06601-6f45-475e-bfd2-bc41d37ee241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:33.718478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba06601-6f45-475e-bfd2-bc41d37ee241 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:33.744379 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2827/5577] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:08:33.745818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c56fa-a52f-4c29-9a09-eedc570869da admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:33.763650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c56fa-a52f-4c29-9a09-eedc570869da admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:33.787533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657649f-8be9-42d7-bafd-3d8729b5226e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:33.789621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657649f-8be9-42d7-bafd-3d8729b5226e admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:33.789923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657649f-8be9-42d7-bafd-3d8729b5226e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:33.790188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657649f-8be9-42d7-bafd-3d8729b5226e admin admin] PATH_INFO: `/v3/projects/6c0c5a6199ae4f5d9c11af7f0fe94632/users/02cd0f6eeb904eebaadc8f6d1c1da5bd/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:33.792279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657649f-8be9-42d7-bafd-3d8729b5226e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c0c5a6199ae4f5d9c11af7f0fe94632, user_id=02cd0f6eeb904eebaadc8f6d1c1da5bd, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:33.796109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657649f-8be9-42d7-bafd-3d8729b5226e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:33.816694 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2751/5578] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:08:33 2022] PUT /identity/v3/projects/6c0c5a6199ae4f5d9c11af7f0fe94632/users/02cd0f6eeb904eebaadc8f6d1c1da5bd/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:08:33.833288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2672a6-9ca2-4195-a0f6-10da6adfc491 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:33.836106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2672a6-9ca2-4195-a0f6-10da6adfc491 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:33.836752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2672a6-9ca2-4195-a0f6-10da6adfc491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:33.929448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7e2672a6-9ca2-4195-a0f6-10da6adfc491 None None] MFA Rules not processed for user `02cd0f6eeb904eebaadc8f6d1c1da5bd`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:33.932768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7e2672a6-9ca2-4195-a0f6-10da6adfc491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:34.122148 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2828/5579] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:08:34.129127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657649f-8be9-42d7-bafd-3d8729b5226e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:34.131286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3657649f-8be9-42d7-bafd-3d8729b5226e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:34.238836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657649f-8be9-42d7-bafd-3d8729b5226e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:34.242887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705968b-ea63-40a8-82fc-4b133f7528fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:34.243516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705968b-ea63-40a8-82fc-4b133f7528fa service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:34.244050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705968b-ea63-40a8-82fc-4b133f7528fa service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:34.244050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705968b-ea63-40a8-82fc-4b133f7528fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:34.254596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705968b-ea63-40a8-82fc-4b133f7528fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:34.256090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705968b-ea63-40a8-82fc-4b133f7528fa service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:34.270193 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2752/5580] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:08:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:08:34.419093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2672a6-9ca2-4195-a0f6-10da6adfc491 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:34.453963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2672a6-9ca2-4195-a0f6-10da6adfc491 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:34.471507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cd339-08ad-49b5-ab4b-c9f87c3175b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:34.472788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cd339-08ad-49b5-ab4b-c9f87c3175b0 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:34.472788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cd339-08ad-49b5-ab4b-c9f87c3175b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:34.472788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cd339-08ad-49b5-ab4b-c9f87c3175b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:34.602969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cd339-08ad-49b5-ab4b-c9f87c3175b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:34.604430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cd339-08ad-49b5-ab4b-c9f87c3175b0 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:34.616129 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2829/5581] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:08:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:08:38.961193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705968b-ea63-40a8-82fc-4b133f7528fa service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:38.963076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9705968b-ea63-40a8-82fc-4b133f7528fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:39.096227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705968b-ea63-40a8-82fc-4b133f7528fa service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:39.101453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edffe7-9669-4182-a7fb-8f9d8281837d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:39.102404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edffe7-9669-4182-a7fb-8f9d8281837d service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:39.103167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edffe7-9669-4182-a7fb-8f9d8281837d service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:39.103647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edffe7-9669-4182-a7fb-8f9d8281837d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:39.106118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f6edffe7-9669-4182-a7fb-8f9d8281837d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:39.166684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:39.179928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7f583-5106-4006-bd3d-13ad6afdc69a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:39.180500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7f583-5106-4006-bd3d-13ad6afdc69a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:39.180500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7f583-5106-4006-bd3d-13ad6afdc69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:39.204350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6edffe7-9669-4182-a7fb-8f9d8281837d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:39.206039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6edffe7-9669-4182-a7fb-8f9d8281837d service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:39.223556 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2753/5582] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:08:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:08:39.240557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-bea7f583-5106-4006-bd3d-13ad6afdc69a None None] MFA Rules not processed for user `8d79dff848bf40fc89377e13736c527b`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:39.242623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bea7f583-5106-4006-bd3d-13ad6afdc69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:39.386124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:39.391381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edffe7-9669-4182-a7fb-8f9d8281837d service glance] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:39.393566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f6edffe7-9669-4182-a7fb-8f9d8281837d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:39.416125 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2830/5583] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:08:39.493441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edffe7-9669-4182-a7fb-8f9d8281837d service glance] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:39.497145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e5e13-5531-4573-af4c-7cb1fc6b4e20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:39.497763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e5e13-5531-4573-af4c-7cb1fc6b4e20 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:39.498040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e5e13-5531-4573-af4c-7cb1fc6b4e20 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:39.498308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e5e13-5531-4573-af4c-7cb1fc6b4e20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:39.500908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-be6e5e13-5531-4573-af4c-7cb1fc6b4e20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:39.513148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e5e13-5531-4573-af4c-7cb1fc6b4e20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:39.515012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e5e13-5531-4573-af4c-7cb1fc6b4e20 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:39.528147 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2754/5584] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:08:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:08:39.825993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7f583-5106-4006-bd3d-13ad6afdc69a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:39.828712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bea7f583-5106-4006-bd3d-13ad6afdc69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:39.856772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7f583-5106-4006-bd3d-13ad6afdc69a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:39.885182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf457c0-f55c-4339-9101-b2e2c658a3f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:39.885906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf457c0-f55c-4339-9101-b2e2c658a3f7 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:39.886264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf457c0-f55c-4339-9101-b2e2c658a3f7 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:39.886641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf457c0-f55c-4339-9101-b2e2c658a3f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:39.897965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf457c0-f55c-4339-9101-b2e2c658a3f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:39.899967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf457c0-f55c-4339-9101-b2e2c658a3f7 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:39.923375 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2831/5585] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:08:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:08:42.711135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c381e-693d-4eb9-aed5-34320bd22bef None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:42.711889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c381e-693d-4eb9-aed5-34320bd22bef None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:42.712184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c381e-693d-4eb9-aed5-34320bd22bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:42.734639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a57c381e-693d-4eb9-aed5-34320bd22bef None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:42.736149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a57c381e-693d-4eb9-aed5-34320bd22bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:42.806950 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2755/5586] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:08:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:08:42.821499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf457c0-f55c-4339-9101-b2e2c658a3f7 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:42.824034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0bf457c0-f55c-4339-9101-b2e2c658a3f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:42.920247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf457c0-f55c-4339-9101-b2e2c658a3f7 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:42.925777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33145565-2781-42a7-9a27-6f6e3bb13c14 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:42.926393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33145565-2781-42a7-9a27-6f6e3bb13c14 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:42.926916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33145565-2781-42a7-9a27-6f6e3bb13c14 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:42.927199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33145565-2781-42a7-9a27-6f6e3bb13c14 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:42.937884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33145565-2781-42a7-9a27-6f6e3bb13c14 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:42.939519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33145565-2781-42a7-9a27-6f6e3bb13c14 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:42.948623 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2832/5587] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:08:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:08:43.149353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200515c-05d7-4ed5-a78f-4b3a4742f712 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:43.150041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200515c-05d7-4ed5-a78f-4b3a4742f712 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:43.150513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200515c-05d7-4ed5-a78f-4b3a4742f712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:43.216268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4200515c-05d7-4ed5-a78f-4b3a4742f712 None None] MFA Rules not processed for user `8d79dff848bf40fc89377e13736c527b`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:43.219969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4200515c-05d7-4ed5-a78f-4b3a4742f712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:43.250639 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2756/5588] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:08:52.945590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=44 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:53.191347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=67 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:54.579947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33145565-2781-42a7-9a27-6f6e3bb13c14 service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:54.582161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-33145565-2781-42a7-9a27-6f6e3bb13c14 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:54.735206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33145565-2781-42a7-9a27-6f6e3bb13c14 service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:54.741220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb15fae-3e17-4058-a132-ceddb2895620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:54.741904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb15fae-3e17-4058-a132-ceddb2895620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:54.742149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb15fae-3e17-4058-a132-ceddb2895620 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:54.742423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb15fae-3e17-4058-a132-ceddb2895620 admin admin] PATH_INFO: `/v3/users/33e1635d451448fc9ca07f5ab9b93487` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:54.744481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb15fae-3e17-4058-a132-ceddb2895620 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33e1635d451448fc9ca07f5ab9b93487)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:54.749192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb15fae-3e17-4058-a132-ceddb2895620 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:54.854929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6eb15fae-3e17-4058-a132-ceddb2895620 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33e1635d451448fc9ca07f5ab9b93487'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:54.868760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6eb15fae-3e17-4058-a132-ceddb2895620 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33e1635d451448fc9ca07f5ab9b93487'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:54.890808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6eb15fae-3e17-4058-a132-ceddb2895620 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33e1635d451448fc9ca07f5ab9b93487'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:54.891809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6eb15fae-3e17-4058-a132-ceddb2895620 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33e1635d451448fc9ca07f5ab9b93487'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:54.901709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6eb15fae-3e17-4058-a132-ceddb2895620 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33e1635d451448fc9ca07f5ab9b93487'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:54.909792 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2833/5589] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:08:54 2022] DELETE /identity/v3/users/33e1635d451448fc9ca07f5ab9b93487 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:08:54.915219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7cebef-fe4f-4a27-99b4-c09b98f8955d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:54.915788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7cebef-fe4f-4a27-99b4-c09b98f8955d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:54.915888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7cebef-fe4f-4a27-99b4-c09b98f8955d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:54.955809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6b7cebef-fe4f-4a27-99b4-c09b98f8955d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:54.957193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6b7cebef-fe4f-4a27-99b4-c09b98f8955d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:55.129583 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2757/5590] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:08:55.147889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb15fae-3e17-4058-a132-ceddb2895620 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:55.149948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6eb15fae-3e17-4058-a132-ceddb2895620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:55.242158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb15fae-3e17-4058-a132-ceddb2895620 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:55.246876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6f0f8-b478-42db-8b82-4b33a450ebee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:55.247768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6f0f8-b478-42db-8b82-4b33a450ebee service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:55.248150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6f0f8-b478-42db-8b82-4b33a450ebee service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:55.248529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6f0f8-b478-42db-8b82-4b33a450ebee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:55.273814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6f0f8-b478-42db-8b82-4b33a450ebee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:55.275912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6f0f8-b478-42db-8b82-4b33a450ebee service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:55.293225 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2834/5591] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:08:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:08:55.699716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7cebef-fe4f-4a27-99b4-c09b98f8955d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:55.701113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6b7cebef-fe4f-4a27-99b4-c09b98f8955d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:55.720869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7cebef-fe4f-4a27-99b4-c09b98f8955d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:55.736540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a0422-5081-4358-ae97-22db0a8d33bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:55.738222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a0422-5081-4358-ae97-22db0a8d33bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:55.738222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a0422-5081-4358-ae97-22db0a8d33bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:55.738222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a0422-5081-4358-ae97-22db0a8d33bd admin admin] PATH_INFO: `/v3/projects/fee45202b6504d398abe0081a92ab8dd` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:55.739041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a0422-5081-4358-ae97-22db0a8d33bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fee45202b6504d398abe0081a92ab8dd)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:55.741366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a0422-5081-4358-ae97-22db0a8d33bd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:55.806537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a41a0422-5081-4358-ae97-22db0a8d33bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fee45202b6504d398abe0081a92ab8dd', 'request_id': 'req-a41a0422-5081-4358-ae97-22db0a8d33bd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:55.821412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a41a0422-5081-4358-ae97-22db0a8d33bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fee45202b6504d398abe0081a92ab8dd', 'request_id': 'req-a41a0422-5081-4358-ae97-22db0a8d33bd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:55.829712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a41a0422-5081-4358-ae97-22db0a8d33bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fee45202b6504d398abe0081a92ab8dd', 'request_id': 'req-a41a0422-5081-4358-ae97-22db0a8d33bd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:55.844023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a41a0422-5081-4358-ae97-22db0a8d33bd admin admin] The token cache is being invalidate because project fee45202b6504d398abe0081a92ab8dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:08:55.844389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a41a0422-5081-4358-ae97-22db0a8d33bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:55.845129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a41a0422-5081-4358-ae97-22db0a8d33bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:08:55.847025 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2758/5592] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:08:55 2022] DELETE /identity/v3/projects/fee45202b6504d398abe0081a92ab8dd => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:08:55.854389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81beea88-cc00-43f0-bcf6-82234324e178 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:55.854389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81beea88-cc00-43f0-bcf6-82234324e178 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:55.854389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81beea88-cc00-43f0-bcf6-82234324e178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:55.886470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-81beea88-cc00-43f0-bcf6-82234324e178 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:55.887970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-81beea88-cc00-43f0-bcf6-82234324e178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:55.914178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:55.966527 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2835/5593] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:08:55.973001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbffaa7-d7ca-4df0-ab1f-ac8e09fd1451 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:55.974089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbffaa7-d7ca-4df0-ab1f-ac8e09fd1451 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:55.974819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbffaa7-d7ca-4df0-ab1f-ac8e09fd1451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:56.009110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-dbbffaa7-d7ca-4df0-ab1f-ac8e09fd1451 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:56.014655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-dbbffaa7-d7ca-4df0-ab1f-ac8e09fd1451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:56.026948 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2759/5594] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:08:56.031899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81beea88-cc00-43f0-bcf6-82234324e178 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:56.049820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81beea88-cc00-43f0-bcf6-82234324e178 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:56.055494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27ed3b-19aa-4a01-91a4-a69d9a4c3800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:56.056121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27ed3b-19aa-4a01-91a4-a69d9a4c3800 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:56.056439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27ed3b-19aa-4a01-91a4-a69d9a4c3800 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:56.056671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27ed3b-19aa-4a01-91a4-a69d9a4c3800 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:56.057378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27ed3b-19aa-4a01-91a4-a69d9a4c3800 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:08:56.057679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27ed3b-19aa-4a01-91a4-a69d9a4c3800 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:56.059321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27ed3b-19aa-4a01-91a4-a69d9a4c3800 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:56.074543 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2836/5595] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:08:56.078476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbffaa7-d7ca-4df0-ab1f-ac8e09fd1451 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:56.089256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbffaa7-d7ca-4df0-ab1f-ac8e09fd1451 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:56.095679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4687566-97e5-4cae-8b8b-e6029af14cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:56.096310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4687566-97e5-4cae-8b8b-e6029af14cd8 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:56.096652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4687566-97e5-4cae-8b8b-e6029af14cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:56.096935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4687566-97e5-4cae-8b8b-e6029af14cd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:56.097571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4687566-97e5-4cae-8b8b-e6029af14cd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:56.101831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4687566-97e5-4cae-8b8b-e6029af14cd8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:56.147082 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2760/5596] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:08:56 2022] POST /identity/v3/projects => generated 361 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:08:56.150428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27ed3b-19aa-4a01-91a4-a69d9a4c3800 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:56.160961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27ed3b-19aa-4a01-91a4-a69d9a4c3800 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:56.220323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cdf00-4737-4bec-906c-1e7bb9297ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:56.220941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cdf00-4737-4bec-906c-1e7bb9297ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:56.221224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cdf00-4737-4bec-906c-1e7bb9297ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:56.221500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cdf00-4737-4bec-906c-1e7bb9297ec7 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:56.222144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2cdf00-4737-4bec-906c-1e7bb9297ec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:56.224440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2cdf00-4737-4bec-906c-1e7bb9297ec7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:56.251486 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2837/5597] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:08:56 2022] POST /identity/v3/users => generated 281 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:08:56.253365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4687566-97e5-4cae-8b8b-e6029af14cd8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:56.265232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4687566-97e5-4cae-8b8b-e6029af14cd8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:56.271923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9bc51-1a31-4389-9044-2c9df587d704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:56.272689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9bc51-1a31-4389-9044-2c9df587d704 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:56.273022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9bc51-1a31-4389-9044-2c9df587d704 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:56.273297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9bc51-1a31-4389-9044-2c9df587d704 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:56.274313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9bc51-1a31-4389-9044-2c9df587d704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:56.276213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9bc51-1a31-4389-9044-2c9df587d704 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:56.287095 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2761/5598] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:08:56.291747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cdf00-4737-4bec-906c-1e7bb9297ec7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:56.302397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cdf00-4737-4bec-906c-1e7bb9297ec7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:56.307277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd5304-82d8-41aa-9603-6e6041f297d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:56.307885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd5304-82d8-41aa-9603-6e6041f297d5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:56.308334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd5304-82d8-41aa-9603-6e6041f297d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:56.308461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd5304-82d8-41aa-9603-6e6041f297d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:56.309088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd5304-82d8-41aa-9603-6e6041f297d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:56.310758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd5304-82d8-41aa-9603-6e6041f297d5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:56.322427 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2838/5599] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:08:56.326734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9bc51-1a31-4389-9044-2c9df587d704 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:56.340864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9bc51-1a31-4389-9044-2c9df587d704 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:56.345912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09290f-347f-44a0-a44f-0bc594b6a19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:56.346638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09290f-347f-44a0-a44f-0bc594b6a19a admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:56.346946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09290f-347f-44a0-a44f-0bc594b6a19a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:56.347229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09290f-347f-44a0-a44f-0bc594b6a19a admin admin] PATH_INFO: `/v3/projects/8a008a2473e7463fb4cf73326a84aae2/users/52c19be363304ed8b4fdd2896691effa/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:56.362346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc09290f-347f-44a0-a44f-0bc594b6a19a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a008a2473e7463fb4cf73326a84aae2, user_id=52c19be363304ed8b4fdd2896691effa, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:56.365702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc09290f-347f-44a0-a44f-0bc594b6a19a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:56.380070 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2762/5600] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:08:56 2022] PUT /identity/v3/projects/8a008a2473e7463fb4cf73326a84aae2/users/52c19be363304ed8b4fdd2896691effa/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:08:56.392336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2527350-9bb3-46c9-a7f1-79cd501847fb None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:56.392626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2527350-9bb3-46c9-a7f1-79cd501847fb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:56.392903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2527350-9bb3-46c9-a7f1-79cd501847fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:56.424165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b2527350-9bb3-46c9-a7f1-79cd501847fb None None] MFA Rules not processed for user `52c19be363304ed8b4fdd2896691effa`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:08:56.425508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b2527350-9bb3-46c9-a7f1-79cd501847fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:56.505085 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2839/5601] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:08:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:08:56.562118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09290f-347f-44a0-a44f-0bc594b6a19a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:56.564311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cc09290f-347f-44a0-a44f-0bc594b6a19a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:56.584971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:56.694338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09290f-347f-44a0-a44f-0bc594b6a19a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:56.701210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105e68c-32d6-48fa-a3f3-1b68a64977dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:56.702679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105e68c-32d6-48fa-a3f3-1b68a64977dd service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:56.703175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105e68c-32d6-48fa-a3f3-1b68a64977dd service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:56.704412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105e68c-32d6-48fa-a3f3-1b68a64977dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:56.714262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a105e68c-32d6-48fa-a3f3-1b68a64977dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:56.715818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a105e68c-32d6-48fa-a3f3-1b68a64977dd service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:56.726285 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2763/5602] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:08:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:08:57.094884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2527350-9bb3-46c9-a7f1-79cd501847fb None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:08:57.110644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2527350-9bb3-46c9-a7f1-79cd501847fb None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:08:57.115791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abf4e1-c5c2-4a11-8709-f97302f7544a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:08:57.116286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abf4e1-c5c2-4a11-8709-f97302f7544a service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:08:57.116744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abf4e1-c5c2-4a11-8709-f97302f7544a service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:08:57.117155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abf4e1-c5c2-4a11-8709-f97302f7544a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:08:57.119338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-46abf4e1-c5c2-4a11-8709-f97302f7544a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:08:57.137103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:57.216534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46abf4e1-c5c2-4a11-8709-f97302f7544a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:08:57.218069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46abf4e1-c5c2-4a11-8709-f97302f7544a service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:08:57.233963 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2840/5603] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:08:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:09:03.047038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f123d-91fb-4059-bdd9-9b17f01cd23b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:03.047574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f123d-91fb-4059-bdd9-9b17f01cd23b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:03.047574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f123d-91fb-4059-bdd9-9b17f01cd23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:03.073731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1c0f123d-91fb-4059-bdd9-9b17f01cd23b None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:03.075481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1c0f123d-91fb-4059-bdd9-9b17f01cd23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:03.180262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abf4e1-c5c2-4a11-8709-f97302f7544a service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:03.181697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-46abf4e1-c5c2-4a11-8709-f97302f7544a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:03.188370 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2764/5604] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:09:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:03.345346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abf4e1-c5c2-4a11-8709-f97302f7544a service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:03.355475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdc1da-059a-45e8-88ad-08631d934aad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:03.356432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdc1da-059a-45e8-88ad-08631d934aad service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:03.356935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdc1da-059a-45e8-88ad-08631d934aad service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:03.357420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdc1da-059a-45e8-88ad-08631d934aad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:03.371479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fdc1da-059a-45e8-88ad-08631d934aad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:03.373805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fdc1da-059a-45e8-88ad-08631d934aad service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:03.401465 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2841/5605] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:09:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 21:09:04.180723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e900d9d-c269-4625-b360-3cb4ca38a3af None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:04.182449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e900d9d-c269-4625-b360-3cb4ca38a3af None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:04.182905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e900d9d-c269-4625-b360-3cb4ca38a3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:04.209300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6e900d9d-c269-4625-b360-3cb4ca38a3af None None] MFA Rules not processed for user `d2a67ba740284d5589df8bdad6ab671f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:04.210769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6e900d9d-c269-4625-b360-3cb4ca38a3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:04.316601 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2765/5606] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:09.797748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdc1da-059a-45e8-88ad-08631d934aad service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:09.836965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdc1da-059a-45e8-88ad-08631d934aad service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:09.843105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09ed92-061e-4c35-9364-96b8a3b66988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:09.844033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09ed92-061e-4c35-9364-96b8a3b66988 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:09.844491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09ed92-061e-4c35-9364-96b8a3b66988 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:09.844925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09ed92-061e-4c35-9364-96b8a3b66988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:09.846927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5d09ed92-061e-4c35-9364-96b8a3b66988 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:09.859951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09ed92-061e-4c35-9364-96b8a3b66988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:09.861594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09ed92-061e-4c35-9364-96b8a3b66988 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:09.877954 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2842/5607] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:09:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:09:14.295622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=71 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:16.386130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e900d9d-c269-4625-b360-3cb4ca38a3af None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:16.388534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6e900d9d-c269-4625-b360-3cb4ca38a3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:16.488484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e900d9d-c269-4625-b360-3cb4ca38a3af None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:16.490253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21176892-2d8f-48fd-b886-9072900a284f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:16.491275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21176892-2d8f-48fd-b886-9072900a284f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:16.491561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21176892-2d8f-48fd-b886-9072900a284f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:16.491841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21176892-2d8f-48fd-b886-9072900a284f admin admin] PATH_INFO: `/v3/users/e683d459ead14ddca5acb36ae9c1cbed` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:16.493014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21176892-2d8f-48fd-b886-9072900a284f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e683d459ead14ddca5acb36ae9c1cbed)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:16.496906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21176892-2d8f-48fd-b886-9072900a284f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:16.561359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-21176892-2d8f-48fd-b886-9072900a284f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e683d459ead14ddca5acb36ae9c1cbed'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:16.562023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-21176892-2d8f-48fd-b886-9072900a284f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e683d459ead14ddca5acb36ae9c1cbed'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:16.562911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-21176892-2d8f-48fd-b886-9072900a284f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e683d459ead14ddca5acb36ae9c1cbed'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:16.572174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-21176892-2d8f-48fd-b886-9072900a284f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e683d459ead14ddca5acb36ae9c1cbed'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:16.589637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-21176892-2d8f-48fd-b886-9072900a284f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e683d459ead14ddca5acb36ae9c1cbed'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:16.611297 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2766/5608] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:09:16 2022] DELETE /identity/v3/users/e683d459ead14ddca5acb36ae9c1cbed => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:16.612892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ee553-4310-42f8-94ff-91552d2cb307 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:16.613396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ee553-4310-42f8-94ff-91552d2cb307 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:16.613501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ee553-4310-42f8-94ff-91552d2cb307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:16.642908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a84ee553-4310-42f8-94ff-91552d2cb307 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:16.645055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a84ee553-4310-42f8-94ff-91552d2cb307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:16.748368 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2843/5609] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:16.749250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21176892-2d8f-48fd-b886-9072900a284f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:16.751268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-21176892-2d8f-48fd-b886-9072900a284f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:16.871227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21176892-2d8f-48fd-b886-9072900a284f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:16.878267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a229d3-787c-4dc9-ab14-d9d42f1910e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:16.879051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a229d3-787c-4dc9-ab14-d9d42f1910e0 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:16.879428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a229d3-787c-4dc9-ab14-d9d42f1910e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:16.880515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a229d3-787c-4dc9-ab14-d9d42f1910e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:16.890953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a229d3-787c-4dc9-ab14-d9d42f1910e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:16.893067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a229d3-787c-4dc9-ab14-d9d42f1910e0 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:16.914741 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2767/5610] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:09:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:09:17.498046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ee553-4310-42f8-94ff-91552d2cb307 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:17.499797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a84ee553-4310-42f8-94ff-91552d2cb307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:17.515433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ee553-4310-42f8-94ff-91552d2cb307 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:17.522245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38bee9-8301-41e0-aa01-eb8bb9381220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:17.523322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38bee9-8301-41e0-aa01-eb8bb9381220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:17.523610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38bee9-8301-41e0-aa01-eb8bb9381220 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:17.524028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38bee9-8301-41e0-aa01-eb8bb9381220 admin admin] PATH_INFO: `/v3/projects/2be32d5a16f7453c96de3dbdb05fe6d1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:17.525540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38bee9-8301-41e0-aa01-eb8bb9381220 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2be32d5a16f7453c96de3dbdb05fe6d1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:17.528043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38bee9-8301-41e0-aa01-eb8bb9381220 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:17.568789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:17.636180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ea38bee9-8301-41e0-aa01-eb8bb9381220 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2be32d5a16f7453c96de3dbdb05fe6d1', 'request_id': 'req-ea38bee9-8301-41e0-aa01-eb8bb9381220'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:17.642374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ea38bee9-8301-41e0-aa01-eb8bb9381220 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2be32d5a16f7453c96de3dbdb05fe6d1', 'request_id': 'req-ea38bee9-8301-41e0-aa01-eb8bb9381220'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:17.651028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ea38bee9-8301-41e0-aa01-eb8bb9381220 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2be32d5a16f7453c96de3dbdb05fe6d1', 'request_id': 'req-ea38bee9-8301-41e0-aa01-eb8bb9381220'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:17.669123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ea38bee9-8301-41e0-aa01-eb8bb9381220 admin admin] The token cache is being invalidate because project 2be32d5a16f7453c96de3dbdb05fe6d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:09:17.669522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ea38bee9-8301-41e0-aa01-eb8bb9381220 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:17.670485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ea38bee9-8301-41e0-aa01-eb8bb9381220 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:17.672338 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2844/5611] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:09:17 2022] DELETE /identity/v3/projects/2be32d5a16f7453c96de3dbdb05fe6d1 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:17.676346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a229d3-787c-4dc9-ab14-d9d42f1910e0 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:17.689969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a229d3-787c-4dc9-ab14-d9d42f1910e0 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:17.717187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:17.783655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f529660-754f-4a00-aa40-e88d541856b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:17.785614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f529660-754f-4a00-aa40-e88d541856b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:17.785899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f529660-754f-4a00-aa40-e88d541856b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:17.786862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f529660-754f-4a00-aa40-e88d541856b5 admin admin] PATH_INFO: `/v3/users/02cd0f6eeb904eebaadc8f6d1c1da5bd` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:17.789382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f529660-754f-4a00-aa40-e88d541856b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02cd0f6eeb904eebaadc8f6d1c1da5bd)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:17.792242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f529660-754f-4a00-aa40-e88d541856b5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:17.920006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8f529660-754f-4a00-aa40-e88d541856b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02cd0f6eeb904eebaadc8f6d1c1da5bd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:17.921038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8f529660-754f-4a00-aa40-e88d541856b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02cd0f6eeb904eebaadc8f6d1c1da5bd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:17.921897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8f529660-754f-4a00-aa40-e88d541856b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02cd0f6eeb904eebaadc8f6d1c1da5bd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:17.945789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8f529660-754f-4a00-aa40-e88d541856b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02cd0f6eeb904eebaadc8f6d1c1da5bd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:17.978532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8f529660-754f-4a00-aa40-e88d541856b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02cd0f6eeb904eebaadc8f6d1c1da5bd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:18.010583 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2768/5612] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:09:17 2022] DELETE /identity/v3/users/02cd0f6eeb904eebaadc8f6d1c1da5bd => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:18.643730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38bee9-8301-41e0-aa01-eb8bb9381220 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:18.647441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ea38bee9-8301-41e0-aa01-eb8bb9381220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:18.681989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38bee9-8301-41e0-aa01-eb8bb9381220 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:18.816028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9029b6-c5ec-4db4-a53a-a01c15dd08cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:18.816764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9029b6-c5ec-4db4-a53a-a01c15dd08cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:18.816947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9029b6-c5ec-4db4-a53a-a01c15dd08cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:18.819839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9029b6-c5ec-4db4-a53a-a01c15dd08cf admin admin] PATH_INFO: `/v3/projects/6c0c5a6199ae4f5d9c11af7f0fe94632` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:18.824932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9029b6-c5ec-4db4-a53a-a01c15dd08cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c0c5a6199ae4f5d9c11af7f0fe94632)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:18.828349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9029b6-c5ec-4db4-a53a-a01c15dd08cf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:18.932165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8e9029b6-c5ec-4db4-a53a-a01c15dd08cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c0c5a6199ae4f5d9c11af7f0fe94632', 'request_id': 'req-8e9029b6-c5ec-4db4-a53a-a01c15dd08cf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:18.946261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8e9029b6-c5ec-4db4-a53a-a01c15dd08cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c0c5a6199ae4f5d9c11af7f0fe94632', 'request_id': 'req-8e9029b6-c5ec-4db4-a53a-a01c15dd08cf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:18.961916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8e9029b6-c5ec-4db4-a53a-a01c15dd08cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c0c5a6199ae4f5d9c11af7f0fe94632', 'request_id': 'req-8e9029b6-c5ec-4db4-a53a-a01c15dd08cf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:18.987782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8e9029b6-c5ec-4db4-a53a-a01c15dd08cf admin admin] The token cache is being invalidate because project 6c0c5a6199ae4f5d9c11af7f0fe94632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:09:18.988117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8e9029b6-c5ec-4db4-a53a-a01c15dd08cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:18.994640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8e9029b6-c5ec-4db4-a53a-a01c15dd08cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:18.996466 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2845/5613] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:09:18 2022] DELETE /identity/v3/projects/6c0c5a6199ae4f5d9c11af7f0fe94632 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:19.004299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c66e63-eccc-4826-b22e-f69976993f14 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:19.005547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c66e63-eccc-4826-b22e-f69976993f14 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:19.006149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c66e63-eccc-4826-b22e-f69976993f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:19.078215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-02c66e63-eccc-4826-b22e-f69976993f14 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:19.081429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-02c66e63-eccc-4826-b22e-f69976993f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:19.204080 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2769/5614] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:19.213245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2eddd-879c-49b4-b23c-d0935fa9cc97 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:19.213529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2eddd-879c-49b4-b23c-d0935fa9cc97 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:19.214872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2eddd-879c-49b4-b23c-d0935fa9cc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:19.244116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-24a2eddd-879c-49b4-b23c-d0935fa9cc97 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:19.245363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-24a2eddd-879c-49b4-b23c-d0935fa9cc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:19.253773 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2846/5615] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:19.258643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c66e63-eccc-4826-b22e-f69976993f14 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:19.270975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c66e63-eccc-4826-b22e-f69976993f14 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:19.279375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bb0e8-7640-4b03-a5ab-4abef8c787a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:19.282580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bb0e8-7640-4b03-a5ab-4abef8c787a8 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:19.282580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bb0e8-7640-4b03-a5ab-4abef8c787a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:19.282580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bb0e8-7640-4b03-a5ab-4abef8c787a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:19.282580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bb0e8-7640-4b03-a5ab-4abef8c787a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:09:19.282580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bb0e8-7640-4b03-a5ab-4abef8c787a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:19.284579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bb0e8-7640-4b03-a5ab-4abef8c787a8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:19.303962 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2770/5616] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:09:19.308262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2eddd-879c-49b4-b23c-d0935fa9cc97 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:19.319724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2eddd-879c-49b4-b23c-d0935fa9cc97 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:19.327365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8000c-5238-426c-9e19-b10dc2f2eaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:19.328287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8000c-5238-426c-9e19-b10dc2f2eaaf admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:19.328933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8000c-5238-426c-9e19-b10dc2f2eaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:19.329443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8000c-5238-426c-9e19-b10dc2f2eaaf admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:19.330399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8000c-5238-426c-9e19-b10dc2f2eaaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:19.333715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8000c-5238-426c-9e19-b10dc2f2eaaf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:19.393089 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2847/5617] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:09:19 2022] POST /identity/v3/projects => generated 375 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:09:19.397505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bb0e8-7640-4b03-a5ab-4abef8c787a8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:19.410892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bb0e8-7640-4b03-a5ab-4abef8c787a8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:19.455868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:19.504074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ed439-033b-4eee-99a9-4893c45924d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:19.505520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ed439-033b-4eee-99a9-4893c45924d6 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:19.506082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ed439-033b-4eee-99a9-4893c45924d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:19.506768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ed439-033b-4eee-99a9-4893c45924d6 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:19.507621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ed439-033b-4eee-99a9-4893c45924d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:19.509992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ed439-033b-4eee-99a9-4893c45924d6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:19.551666 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2771/5618] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:09:19 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:09:19.553298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8000c-5238-426c-9e19-b10dc2f2eaaf admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:19.561020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:19.569230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8000c-5238-426c-9e19-b10dc2f2eaaf admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:19.585427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3beb49c-3ad8-4b4b-9e9c-265833196fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:19.586702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3beb49c-3ad8-4b4b-9e9c-265833196fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:19.587475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3beb49c-3ad8-4b4b-9e9c-265833196fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:19.588600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3beb49c-3ad8-4b4b-9e9c-265833196fc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:19.589714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3beb49c-3ad8-4b4b-9e9c-265833196fc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:19.591866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3beb49c-3ad8-4b4b-9e9c-265833196fc9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:19.611762 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2848/5619] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:09:19.612576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ed439-033b-4eee-99a9-4893c45924d6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:19.628466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ed439-033b-4eee-99a9-4893c45924d6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:19.634062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47316817-0870-45e9-b67c-668b39c7f8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:19.634737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47316817-0870-45e9-b67c-668b39c7f8db admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:19.635055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47316817-0870-45e9-b67c-668b39c7f8db admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:19.635302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47316817-0870-45e9-b67c-668b39c7f8db admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:19.636160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47316817-0870-45e9-b67c-668b39c7f8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:19.638112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47316817-0870-45e9-b67c-668b39c7f8db admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:19.651740 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2772/5620] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:09:19.654424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3beb49c-3ad8-4b4b-9e9c-265833196fc9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:19.668787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3beb49c-3ad8-4b4b-9e9c-265833196fc9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:19.680936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79989be1-8a07-45c2-8228-a0609dd98562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:19.681664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79989be1-8a07-45c2-8228-a0609dd98562 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:19.681953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79989be1-8a07-45c2-8228-a0609dd98562 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:19.682238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79989be1-8a07-45c2-8228-a0609dd98562 admin admin] PATH_INFO: `/v3/projects/8fb7c0f568dc47249086aa49600baf74/users/7f5e1c1196ea4a86a24299749d6371da/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:19.702172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79989be1-8a07-45c2-8228-a0609dd98562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fb7c0f568dc47249086aa49600baf74, user_id=7f5e1c1196ea4a86a24299749d6371da, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:19.705212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79989be1-8a07-45c2-8228-a0609dd98562 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:19.725619 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2849/5621] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:09:19 2022] PUT /identity/v3/projects/8fb7c0f568dc47249086aa49600baf74/users/7f5e1c1196ea4a86a24299749d6371da/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:19.737055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260aef0-d57c-4ea4-ad49-3afbef0d8ac9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:19.737475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260aef0-d57c-4ea4-ad49-3afbef0d8ac9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:19.737829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260aef0-d57c-4ea4-ad49-3afbef0d8ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:19.773592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6260aef0-d57c-4ea4-ad49-3afbef0d8ac9 None None] MFA Rules not processed for user `7f5e1c1196ea4a86a24299749d6371da`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:19.775226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6260aef0-d57c-4ea4-ad49-3afbef0d8ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:19.862336 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2773/5622] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:19.867308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79989be1-8a07-45c2-8228-a0609dd98562 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:19.957050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79989be1-8a07-45c2-8228-a0609dd98562 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:19.964207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc4e4b-3e01-4f54-ae56-e495ebeefe4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:19.965609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc4e4b-3e01-4f54-ae56-e495ebeefe4a admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:19.966137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc4e4b-3e01-4f54-ae56-e495ebeefe4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:19.966678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc4e4b-3e01-4f54-ae56-e495ebeefe4a admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:19.968172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc4e4b-3e01-4f54-ae56-e495ebeefe4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:19.971261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc4e4b-3e01-4f54-ae56-e495ebeefe4a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:20.018193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260aef0-d57c-4ea4-ad49-3afbef0d8ac9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:20.019821 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2850/5623] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:09:19 2022] POST /identity/v3/projects => generated 375 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:09:20.037517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260aef0-d57c-4ea4-ad49-3afbef0d8ac9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:20.141268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cb690-89e0-4533-8d47-31ea4fc1dec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:20.142170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cb690-89e0-4533-8d47-31ea4fc1dec2 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:20.142589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cb690-89e0-4533-8d47-31ea4fc1dec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:20.142961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cb690-89e0-4533-8d47-31ea4fc1dec2 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:20.143676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234cb690-89e0-4533-8d47-31ea4fc1dec2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:20.146458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234cb690-89e0-4533-8d47-31ea4fc1dec2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:20.214618 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2774/5624] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:09:20 2022] POST /identity/v3/users => generated 294 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:09:20.216264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc4e4b-3e01-4f54-ae56-e495ebeefe4a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:20.234259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc4e4b-3e01-4f54-ae56-e495ebeefe4a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:20.242474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b580ccd-604b-491d-bd9d-2e8b76d24532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:20.243503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b580ccd-604b-491d-bd9d-2e8b76d24532 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:20.244029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b580ccd-604b-491d-bd9d-2e8b76d24532 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:20.244529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b580ccd-604b-491d-bd9d-2e8b76d24532 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:20.245407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b580ccd-604b-491d-bd9d-2e8b76d24532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:20.249014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b580ccd-604b-491d-bd9d-2e8b76d24532 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:20.268841 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2851/5625] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:09:20.269805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cb690-89e0-4533-8d47-31ea4fc1dec2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:20.288126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cb690-89e0-4533-8d47-31ea4fc1dec2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:20.301039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c227b4-8e2f-42cc-8886-ae1d7de490e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:20.303135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c227b4-8e2f-42cc-8886-ae1d7de490e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:20.320704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c227b4-8e2f-42cc-8886-ae1d7de490e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:20.322121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c227b4-8e2f-42cc-8886-ae1d7de490e4 admin admin] PATH_INFO: `/v3/projects/fe509274efbb4895ac7a553465f58438/users/6e6bc1b159e74b4993b87a43272f853d/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:20.371247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c227b4-8e2f-42cc-8886-ae1d7de490e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe509274efbb4895ac7a553465f58438, user_id=6e6bc1b159e74b4993b87a43272f853d, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:20.373935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c227b4-8e2f-42cc-8886-ae1d7de490e4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:20.397473 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2775/5626] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:09:20 2022] PUT /identity/v3/projects/fe509274efbb4895ac7a553465f58438/users/6e6bc1b159e74b4993b87a43272f853d/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:09:20.400779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b580ccd-604b-491d-bd9d-2e8b76d24532 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:20.429537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b580ccd-604b-491d-bd9d-2e8b76d24532 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:20.557635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa4b1d-bf25-4007-a5b4-cfe541319b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:20.558304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa4b1d-bf25-4007-a5b4-cfe541319b80 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:20.558783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa4b1d-bf25-4007-a5b4-cfe541319b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:20.559123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa4b1d-bf25-4007-a5b4-cfe541319b80 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:20.560334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa4b1d-bf25-4007-a5b4-cfe541319b80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:20.564172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa4b1d-bf25-4007-a5b4-cfe541319b80 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:20.578387 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2852/5627] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:09:20.582203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c227b4-8e2f-42cc-8886-ae1d7de490e4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:20.589199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:20.605576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c227b4-8e2f-42cc-8886-ae1d7de490e4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:20.614969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102d3c2-0d6f-43a4-ae37-adfff2624967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:20.616182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102d3c2-0d6f-43a4-ae37-adfff2624967 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:20.616617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102d3c2-0d6f-43a4-ae37-adfff2624967 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:20.617040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102d3c2-0d6f-43a4-ae37-adfff2624967 admin admin] PATH_INFO: `/v3/projects/fe509274efbb4895ac7a553465f58438/users/6e6bc1b159e74b4993b87a43272f853d/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:20.619703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102d3c2-0d6f-43a4-ae37-adfff2624967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe509274efbb4895ac7a553465f58438, user_id=6e6bc1b159e74b4993b87a43272f853d, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:20.624019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102d3c2-0d6f-43a4-ae37-adfff2624967 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:20.639204 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2776/5628] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:09:20 2022] PUT /identity/v3/projects/fe509274efbb4895ac7a553465f58438/users/6e6bc1b159e74b4993b87a43272f853d/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:20.653173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac2208-25cc-4046-849d-b04f99e709c1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:20.653786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac2208-25cc-4046-849d-b04f99e709c1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:20.654253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac2208-25cc-4046-849d-b04f99e709c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:20.698462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-13ac2208-25cc-4046-849d-b04f99e709c1 None None] MFA Rules not processed for user `6e6bc1b159e74b4993b87a43272f853d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:20.700764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-13ac2208-25cc-4046-849d-b04f99e709c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:20.742593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:20.829289 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2853/5629] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:20.846051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102d3c2-0d6f-43a4-ae37-adfff2624967 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:20.849935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1102d3c2-0d6f-43a4-ae37-adfff2624967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:20.971330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102d3c2-0d6f-43a4-ae37-adfff2624967 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:20.977377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aaa048-7ad0-4ea8-896d-4cfa6c38183e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:20.978904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aaa048-7ad0-4ea8-896d-4cfa6c38183e service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:20.979426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aaa048-7ad0-4ea8-896d-4cfa6c38183e service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:20.980446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aaa048-7ad0-4ea8-896d-4cfa6c38183e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:21.093544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aaa048-7ad0-4ea8-896d-4cfa6c38183e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:21.099710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aaa048-7ad0-4ea8-896d-4cfa6c38183e service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:21.125174 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2777/5630] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:09:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:09:24.247661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac2208-25cc-4046-849d-b04f99e709c1 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:24.259448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac2208-25cc-4046-849d-b04f99e709c1 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:24.263672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c640cb-e153-4744-a021-677bab850b5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:24.264302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c640cb-e153-4744-a021-677bab850b5a service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:24.264579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c640cb-e153-4744-a021-677bab850b5a service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:24.264969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c640cb-e153-4744-a021-677bab850b5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:24.276248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c640cb-e153-4744-a021-677bab850b5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:24.277729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c640cb-e153-4744-a021-677bab850b5a service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:24.288508 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2854/5631] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:09:24 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:09:25.006842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aaa048-7ad0-4ea8-896d-4cfa6c38183e service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:25.008265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-48aaa048-7ad0-4ea8-896d-4cfa6c38183e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:25.159534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aaa048-7ad0-4ea8-896d-4cfa6c38183e service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:25.178978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302f21f-f027-4058-93d0-f543aafafba6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:25.179886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302f21f-f027-4058-93d0-f543aafafba6 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:25.180249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302f21f-f027-4058-93d0-f543aafafba6 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:25.180629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302f21f-f027-4058-93d0-f543aafafba6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:25.184574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3302f21f-f027-4058-93d0-f543aafafba6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:25.222257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3302f21f-f027-4058-93d0-f543aafafba6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:25.226125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3302f21f-f027-4058-93d0-f543aafafba6 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:25.255633 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2778/5632] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:09:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:09:26.564127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c640cb-e153-4744-a021-677bab850b5a service cinder] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:26.581390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c640cb-e153-4744-a021-677bab850b5a service cinder] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:26.588320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de60bcc-46d1-4e2a-9ef1-527cbe2e8454 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:26.588943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de60bcc-46d1-4e2a-9ef1-527cbe2e8454 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:26.589228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de60bcc-46d1-4e2a-9ef1-527cbe2e8454 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:26.589504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de60bcc-46d1-4e2a-9ef1-527cbe2e8454 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:26.591967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4de60bcc-46d1-4e2a-9ef1-527cbe2e8454 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:26.611359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de60bcc-46d1-4e2a-9ef1-527cbe2e8454 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:26.612975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de60bcc-46d1-4e2a-9ef1-527cbe2e8454 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:26.627926 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2855/5633] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:09:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:09:31.814813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302f21f-f027-4058-93d0-f543aafafba6 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:31.816348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3302f21f-f027-4058-93d0-f543aafafba6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:31.876700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:31.932269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302f21f-f027-4058-93d0-f543aafafba6 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:31.941902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f973c-1a53-4231-b2ea-8f781c973134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:31.942644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f973c-1a53-4231-b2ea-8f781c973134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:31.942932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f973c-1a53-4231-b2ea-8f781c973134 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:31.943207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f973c-1a53-4231-b2ea-8f781c973134 admin admin] PATH_INFO: `/v3/users/6d63c633112e49d091421c84cea3422b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:31.952825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f973c-1a53-4231-b2ea-8f781c973134 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d63c633112e49d091421c84cea3422b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:31.955940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f973c-1a53-4231-b2ea-8f781c973134 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:32.026868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4a8f973c-1a53-4231-b2ea-8f781c973134 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d63c633112e49d091421c84cea3422b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:32.027786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4a8f973c-1a53-4231-b2ea-8f781c973134 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d63c633112e49d091421c84cea3422b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:32.028594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4a8f973c-1a53-4231-b2ea-8f781c973134 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d63c633112e49d091421c84cea3422b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:32.039019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4a8f973c-1a53-4231-b2ea-8f781c973134 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d63c633112e49d091421c84cea3422b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:32.058014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4a8f973c-1a53-4231-b2ea-8f781c973134 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d63c633112e49d091421c84cea3422b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:32.081589 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2779/5634] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:09:31 2022] DELETE /identity/v3/users/6d63c633112e49d091421c84cea3422b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:32.454754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de60bcc-46d1-4e2a-9ef1-527cbe2e8454 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:32.472534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de60bcc-46d1-4e2a-9ef1-527cbe2e8454 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:32.595342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055d967-9928-43e0-8d0f-14bacfbf883a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:32.596419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055d967-9928-43e0-8d0f-14bacfbf883a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:32.596798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055d967-9928-43e0-8d0f-14bacfbf883a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:32.597425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055d967-9928-43e0-8d0f-14bacfbf883a admin admin] PATH_INFO: `/v3/projects/6c0ad82595344eb4a1122d294faf15b2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:32.601354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055d967-9928-43e0-8d0f-14bacfbf883a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c0ad82595344eb4a1122d294faf15b2)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:32.605822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055d967-9928-43e0-8d0f-14bacfbf883a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:32.730202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3055d967-9928-43e0-8d0f-14bacfbf883a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c0ad82595344eb4a1122d294faf15b2', 'request_id': 'req-3055d967-9928-43e0-8d0f-14bacfbf883a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:32.739251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3055d967-9928-43e0-8d0f-14bacfbf883a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c0ad82595344eb4a1122d294faf15b2', 'request_id': 'req-3055d967-9928-43e0-8d0f-14bacfbf883a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:32.747998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3055d967-9928-43e0-8d0f-14bacfbf883a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c0ad82595344eb4a1122d294faf15b2', 'request_id': 'req-3055d967-9928-43e0-8d0f-14bacfbf883a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:32.764538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3055d967-9928-43e0-8d0f-14bacfbf883a admin admin] The token cache is being invalidate because project 6c0ad82595344eb4a1122d294faf15b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:09:32.764844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3055d967-9928-43e0-8d0f-14bacfbf883a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:32.765518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3055d967-9928-43e0-8d0f-14bacfbf883a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:32.768429 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2856/5635] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:09:32 2022] DELETE /identity/v3/projects/6c0ad82595344eb4a1122d294faf15b2 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:32.774225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a740fbe-a2c7-4260-8600-79c50f8a9d38 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:32.779550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a740fbe-a2c7-4260-8600-79c50f8a9d38 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:32.782990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a740fbe-a2c7-4260-8600-79c50f8a9d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:32.826162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6a740fbe-a2c7-4260-8600-79c50f8a9d38 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:32.829513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6a740fbe-a2c7-4260-8600-79c50f8a9d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:32.928278 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2780/5636] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:09:32.937204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ccad90-2154-410c-b28f-7766e7be24c4 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:32.937496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ccad90-2154-410c-b28f-7766e7be24c4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:32.937778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ccad90-2154-410c-b28f-7766e7be24c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:32.974772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-63ccad90-2154-410c-b28f-7766e7be24c4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:32.977909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-63ccad90-2154-410c-b28f-7766e7be24c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:32.994170 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2857/5637] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:32.997778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a740fbe-a2c7-4260-8600-79c50f8a9d38 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:33.012031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a740fbe-a2c7-4260-8600-79c50f8a9d38 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:33.019670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6fb37-eb29-4185-8b6c-3747054425be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:33.022955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6fb37-eb29-4185-8b6c-3747054425be admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:33.023573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6fb37-eb29-4185-8b6c-3747054425be admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:33.024021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6fb37-eb29-4185-8b6c-3747054425be admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:33.024876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6fb37-eb29-4185-8b6c-3747054425be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:09:33.025068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6fb37-eb29-4185-8b6c-3747054425be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:33.029019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6fb37-eb29-4185-8b6c-3747054425be admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:33.068919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ccad90-2154-410c-b28f-7766e7be24c4 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:33.069073 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2781/5638] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:09:33.090927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ccad90-2154-410c-b28f-7766e7be24c4 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:33.100648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1e378-edec-46b4-9125-6bfcccfa9cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:33.102295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1e378-edec-46b4-9125-6bfcccfa9cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:33.102609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1e378-edec-46b4-9125-6bfcccfa9cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:33.102887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1e378-edec-46b4-9125-6bfcccfa9cf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:33.104355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd1e378-edec-46b4-9125-6bfcccfa9cf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:33.106874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd1e378-edec-46b4-9125-6bfcccfa9cf5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:33.149895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:33.161985 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2858/5639] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:09:33 2022] POST /identity/v3/projects => generated 365 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:09:33.162757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6fb37-eb29-4185-8b6c-3747054425be admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:33.174247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6fb37-eb29-4185-8b6c-3747054425be admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:33.250234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:33.296186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace48c34-e5ac-4530-a105-342e348fb17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:33.297304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace48c34-e5ac-4530-a105-342e348fb17b admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:33.297304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace48c34-e5ac-4530-a105-342e348fb17b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:33.297540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace48c34-e5ac-4530-a105-342e348fb17b admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:33.298151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace48c34-e5ac-4530-a105-342e348fb17b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:33.302025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace48c34-e5ac-4530-a105-342e348fb17b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:33.345056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1e378-edec-46b4-9125-6bfcccfa9cf5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:33.346288 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2782/5640] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:09:33 2022] POST /identity/v3/users => generated 283 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:09:33.360648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1e378-edec-46b4-9125-6bfcccfa9cf5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:33.367121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880ca14-798f-4c81-b973-16a7fef3a731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:33.368050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880ca14-798f-4c81-b973-16a7fef3a731 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:33.368341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880ca14-798f-4c81-b973-16a7fef3a731 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:33.368625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880ca14-798f-4c81-b973-16a7fef3a731 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:33.369280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880ca14-798f-4c81-b973-16a7fef3a731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:33.371058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880ca14-798f-4c81-b973-16a7fef3a731 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:33.382564 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2859/5641] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:09:33.387139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace48c34-e5ac-4530-a105-342e348fb17b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:33.401279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace48c34-e5ac-4530-a105-342e348fb17b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:33.406679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae348b26-199b-4ba9-be23-d76b47098da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:33.407454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae348b26-199b-4ba9-be23-d76b47098da2 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:33.407924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae348b26-199b-4ba9-be23-d76b47098da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:33.408210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae348b26-199b-4ba9-be23-d76b47098da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:33.408862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae348b26-199b-4ba9-be23-d76b47098da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:33.411105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae348b26-199b-4ba9-be23-d76b47098da2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:33.423469 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2783/5642] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:09:33.427799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880ca14-798f-4c81-b973-16a7fef3a731 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:33.439968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880ca14-798f-4c81-b973-16a7fef3a731 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:33.446231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891a9bb-e38b-4919-81d4-716ad2f06ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:33.447367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891a9bb-e38b-4919-81d4-716ad2f06ad8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:33.447891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891a9bb-e38b-4919-81d4-716ad2f06ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:33.448537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891a9bb-e38b-4919-81d4-716ad2f06ad8 admin admin] PATH_INFO: `/v3/projects/d492e34859f14a87b77ff9ef6a48c927/users/8bc7cf6d8d2247c1ad9ca8c4d377ee75/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:33.474638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891a9bb-e38b-4919-81d4-716ad2f06ad8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d492e34859f14a87b77ff9ef6a48c927, user_id=8bc7cf6d8d2247c1ad9ca8c4d377ee75, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:33.477473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891a9bb-e38b-4919-81d4-716ad2f06ad8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:33.509365 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2860/5643] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:09:33 2022] PUT /identity/v3/projects/d492e34859f14a87b77ff9ef6a48c927/users/8bc7cf6d8d2247c1ad9ca8c4d377ee75/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:33.511652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e2a43-a430-42e0-943a-e2b796a7a30c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:33.511960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e2a43-a430-42e0-943a-e2b796a7a30c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:33.512239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e2a43-a430-42e0-943a-e2b796a7a30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:33.563323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c10e2a43-a430-42e0-943a-e2b796a7a30c None None] MFA Rules not processed for user `8bc7cf6d8d2247c1ad9ca8c4d377ee75`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:33.564547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c10e2a43-a430-42e0-943a-e2b796a7a30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:33.690085 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2784/5644] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:33.709560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891a9bb-e38b-4919-81d4-716ad2f06ad8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:33.713977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c891a9bb-e38b-4919-81d4-716ad2f06ad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:33.901004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891a9bb-e38b-4919-81d4-716ad2f06ad8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:33.907434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250ba18-e80b-4da0-8e63-1663e7c16839 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:33.908202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250ba18-e80b-4da0-8e63-1663e7c16839 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:33.908829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250ba18-e80b-4da0-8e63-1663e7c16839 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:33.909446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250ba18-e80b-4da0-8e63-1663e7c16839 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:33.919890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1250ba18-e80b-4da0-8e63-1663e7c16839 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:33.921850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1250ba18-e80b-4da0-8e63-1663e7c16839 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:33.936174 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2861/5645] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:09:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:09:34.147061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e2a43-a430-42e0-943a-e2b796a7a30c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:34.251118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e2a43-a430-42e0-943a-e2b796a7a30c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:34.259099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5d664-558d-4cd7-86d4-e6a8dc583c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:34.259952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5d664-558d-4cd7-86d4-e6a8dc583c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:34.260295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5d664-558d-4cd7-86d4-e6a8dc583c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:34.260763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5d664-558d-4cd7-86d4-e6a8dc583c5b admin admin] PATH_INFO: `/v3/users/8bc7cf6d8d2247c1ad9ca8c4d377ee75` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:34.262106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f5d664-558d-4cd7-86d4-e6a8dc583c5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bc7cf6d8d2247c1ad9ca8c4d377ee75)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:34.266396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f5d664-558d-4cd7-86d4-e6a8dc583c5b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:34.392313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:34.406178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32f5d664-558d-4cd7-86d4-e6a8dc583c5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bc7cf6d8d2247c1ad9ca8c4d377ee75'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:34.410167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32f5d664-558d-4cd7-86d4-e6a8dc583c5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bc7cf6d8d2247c1ad9ca8c4d377ee75'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:34.411646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32f5d664-558d-4cd7-86d4-e6a8dc583c5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bc7cf6d8d2247c1ad9ca8c4d377ee75'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:34.430981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32f5d664-558d-4cd7-86d4-e6a8dc583c5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bc7cf6d8d2247c1ad9ca8c4d377ee75'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:34.449372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32f5d664-558d-4cd7-86d4-e6a8dc583c5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bc7cf6d8d2247c1ad9ca8c4d377ee75'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:34.525691 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2785/5646] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:09:34 2022] DELETE /identity/v3/users/8bc7cf6d8d2247c1ad9ca8c4d377ee75 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:34.530876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8332f6d-bd71-4847-8012-acc1647eced2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:34.531395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8332f6d-bd71-4847-8012-acc1647eced2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:34.531666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8332f6d-bd71-4847-8012-acc1647eced2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:34.581943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f8332f6d-bd71-4847-8012-acc1647eced2 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:34.584159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f8332f6d-bd71-4847-8012-acc1647eced2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:34.767879 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2862/5647] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:09:34.786856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5d664-558d-4cd7-86d4-e6a8dc583c5b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:34.788516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-32f5d664-558d-4cd7-86d4-e6a8dc583c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:34.934809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5d664-558d-4cd7-86d4-e6a8dc583c5b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:34.941313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601e8b1-63ea-4101-a80a-0d712aacbe5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:34.942601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601e8b1-63ea-4101-a80a-0d712aacbe5a service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:34.942891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601e8b1-63ea-4101-a80a-0d712aacbe5a service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:34.943167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601e8b1-63ea-4101-a80a-0d712aacbe5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:34.962304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d601e8b1-63ea-4101-a80a-0d712aacbe5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:34.966283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d601e8b1-63ea-4101-a80a-0d712aacbe5a service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:34.980490 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2786/5648] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:09:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:09:35.643328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8332f6d-bd71-4847-8012-acc1647eced2 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:35.644958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f8332f6d-bd71-4847-8012-acc1647eced2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:35.673592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8332f6d-bd71-4847-8012-acc1647eced2 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:35.684757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab5837b-e2c0-44d7-835e-a012b8106c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:35.685969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5837b-e2c0-44d7-835e-a012b8106c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:35.686569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5837b-e2c0-44d7-835e-a012b8106c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:35.687049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5837b-e2c0-44d7-835e-a012b8106c31 admin admin] PATH_INFO: `/v3/projects/d492e34859f14a87b77ff9ef6a48c927` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:35.689046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab5837b-e2c0-44d7-835e-a012b8106c31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d492e34859f14a87b77ff9ef6a48c927)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:35.692132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab5837b-e2c0-44d7-835e-a012b8106c31 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:35.787972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-aab5837b-e2c0-44d7-835e-a012b8106c31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd492e34859f14a87b77ff9ef6a48c927', 'request_id': 'req-aab5837b-e2c0-44d7-835e-a012b8106c31'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:35.802809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-aab5837b-e2c0-44d7-835e-a012b8106c31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd492e34859f14a87b77ff9ef6a48c927', 'request_id': 'req-aab5837b-e2c0-44d7-835e-a012b8106c31'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:35.811748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-aab5837b-e2c0-44d7-835e-a012b8106c31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd492e34859f14a87b77ff9ef6a48c927', 'request_id': 'req-aab5837b-e2c0-44d7-835e-a012b8106c31'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:35.814834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:35.844229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-aab5837b-e2c0-44d7-835e-a012b8106c31 admin admin] The token cache is being invalidate because project d492e34859f14a87b77ff9ef6a48c927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:09:35.844431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-aab5837b-e2c0-44d7-835e-a012b8106c31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:35.851667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-aab5837b-e2c0-44d7-835e-a012b8106c31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:35.863929 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2863/5649] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:09:35 2022] DELETE /identity/v3/projects/d492e34859f14a87b77ff9ef6a48c927 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:35.871647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea43b1c-7ce1-48a0-b0d3-afc104c2e96a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:35.873462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea43b1c-7ce1-48a0-b0d3-afc104c2e96a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:35.873941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea43b1c-7ce1-48a0-b0d3-afc104c2e96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:35.925669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-dea43b1c-7ce1-48a0-b0d3-afc104c2e96a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:35.928026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-dea43b1c-7ce1-48a0-b0d3-afc104c2e96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:36.119302 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2787/5650] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:36.126542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7d240-0a91-4f66-ae47-22f565a4d69a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:36.127608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7d240-0a91-4f66-ae47-22f565a4d69a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:36.127608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7d240-0a91-4f66-ae47-22f565a4d69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:36.168965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-04c7d240-0a91-4f66-ae47-22f565a4d69a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:36.174238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-04c7d240-0a91-4f66-ae47-22f565a4d69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:36.201803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea43b1c-7ce1-48a0-b0d3-afc104c2e96a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:36.205387 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2864/5651] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:36.233276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea43b1c-7ce1-48a0-b0d3-afc104c2e96a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:36.251193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2efc8-6a3d-45e0-a70a-df0d0b03a23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:36.254774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2efc8-6a3d-45e0-a70a-df0d0b03a23c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:36.258011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2efc8-6a3d-45e0-a70a-df0d0b03a23c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:36.258416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2efc8-6a3d-45e0-a70a-df0d0b03a23c admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:36.259253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a2efc8-6a3d-45e0-a70a-df0d0b03a23c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:09:36.263002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a2efc8-6a3d-45e0-a70a-df0d0b03a23c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:36.266511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a2efc8-6a3d-45e0-a70a-df0d0b03a23c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:36.297594 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2788/5652] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:09:36.299595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7d240-0a91-4f66-ae47-22f565a4d69a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:36.328495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7d240-0a91-4f66-ae47-22f565a4d69a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:36.334992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5c7c1-864c-4948-8e7b-61bbca7f7f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:36.336214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5c7c1-864c-4948-8e7b-61bbca7f7f13 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:36.336749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5c7c1-864c-4948-8e7b-61bbca7f7f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:36.337366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5c7c1-864c-4948-8e7b-61bbca7f7f13 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:36.338520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5c7c1-864c-4948-8e7b-61bbca7f7f13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:36.340912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5c7c1-864c-4948-8e7b-61bbca7f7f13 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:36.380134 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2865/5653] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:09:36 2022] POST /identity/v3/projects => generated 383 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:09:36.384493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2efc8-6a3d-45e0-a70a-df0d0b03a23c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:36.399204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2efc8-6a3d-45e0-a70a-df0d0b03a23c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:36.478781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:36.500499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83344ce-4d3c-49b1-835b-bf036a93c4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:36.501129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83344ce-4d3c-49b1-835b-bf036a93c4a4 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:36.501436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83344ce-4d3c-49b1-835b-bf036a93c4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:36.501697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83344ce-4d3c-49b1-835b-bf036a93c4a4 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:36.502345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83344ce-4d3c-49b1-835b-bf036a93c4a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:36.504617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83344ce-4d3c-49b1-835b-bf036a93c4a4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:36.551516 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2789/5654] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:09:36 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:09:36.555257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5c7c1-864c-4948-8e7b-61bbca7f7f13 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:36.567847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5c7c1-864c-4948-8e7b-61bbca7f7f13 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:36.585416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e55f7-c6d1-44b3-9758-e7d39e18ef18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:36.586808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e55f7-c6d1-44b3-9758-e7d39e18ef18 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:36.587433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e55f7-c6d1-44b3-9758-e7d39e18ef18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:36.587982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e55f7-c6d1-44b3-9758-e7d39e18ef18 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:36.590808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e55f7-c6d1-44b3-9758-e7d39e18ef18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:36.596914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e55f7-c6d1-44b3-9758-e7d39e18ef18 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:36.626261 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2866/5655] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:09:36.632346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83344ce-4d3c-49b1-835b-bf036a93c4a4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:36.653752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83344ce-4d3c-49b1-835b-bf036a93c4a4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:36.687966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bc4c7-2dfc-463f-aea0-916468ca7dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:36.688788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bc4c7-2dfc-463f-aea0-916468ca7dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:36.689196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bc4c7-2dfc-463f-aea0-916468ca7dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:36.689732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bc4c7-2dfc-463f-aea0-916468ca7dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:36.690413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95bc4c7-2dfc-463f-aea0-916468ca7dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:36.696567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95bc4c7-2dfc-463f-aea0-916468ca7dc7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:36.717755 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2790/5656] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:09:36.719900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e55f7-c6d1-44b3-9758-e7d39e18ef18 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:36.735307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e55f7-c6d1-44b3-9758-e7d39e18ef18 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:36.744267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87ba53-7307-41c7-b7e6-18549c977282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:36.745109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87ba53-7307-41c7-b7e6-18549c977282 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:36.745474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87ba53-7307-41c7-b7e6-18549c977282 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:36.745822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87ba53-7307-41c7-b7e6-18549c977282 admin admin] PATH_INFO: `/v3/projects/24a06f5da36b4488a1b8ea01729b0e6c/users/902ba44d318d4a01a510f732d27467c9/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:36.772876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87ba53-7307-41c7-b7e6-18549c977282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24a06f5da36b4488a1b8ea01729b0e6c, user_id=902ba44d318d4a01a510f732d27467c9, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:36.775573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87ba53-7307-41c7-b7e6-18549c977282 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:36.829494 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2867/5657] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:09:36 2022] PUT /identity/v3/projects/24a06f5da36b4488a1b8ea01729b0e6c/users/902ba44d318d4a01a510f732d27467c9/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:36.836450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e025d-f3e3-41c2-9c44-4c520aec900a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:36.836744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e025d-f3e3-41c2-9c44-4c520aec900a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:36.837023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e025d-f3e3-41c2-9c44-4c520aec900a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:36.891792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c69e025d-f3e3-41c2-9c44-4c520aec900a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:36.898325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c69e025d-f3e3-41c2-9c44-4c520aec900a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:37.064246 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2791/5658] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:37.099093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87ba53-7307-41c7-b7e6-18549c977282 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:37.110011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8c87ba53-7307-41c7-b7e6-18549c977282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:37.247357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87ba53-7307-41c7-b7e6-18549c977282 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:37.257463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165443a-fd22-43fa-8c71-966ef8bc9269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:37.258084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165443a-fd22-43fa-8c71-966ef8bc9269 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:37.258363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165443a-fd22-43fa-8c71-966ef8bc9269 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:37.258671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165443a-fd22-43fa-8c71-966ef8bc9269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:37.269241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165443a-fd22-43fa-8c71-966ef8bc9269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:37.270791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165443a-fd22-43fa-8c71-966ef8bc9269 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:37.282893 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2868/5659] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:09:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:09:40.710859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed587611-d9c9-4fe9-a64d-d6fba62753b1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:40.711361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed587611-d9c9-4fe9-a64d-d6fba62753b1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:40.711446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed587611-d9c9-4fe9-a64d-d6fba62753b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:40.742179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ed587611-d9c9-4fe9-a64d-d6fba62753b1 None None] MFA Rules not processed for user `f74f251052ee4c9a99db0c43bc00da9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:40.743749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ed587611-d9c9-4fe9-a64d-d6fba62753b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:40.861191 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2792/5660] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:41.951668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165443a-fd22-43fa-8c71-966ef8bc9269 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:41.953310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5165443a-fd22-43fa-8c71-966ef8bc9269 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:41.968628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165443a-fd22-43fa-8c71-966ef8bc9269 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:41.976227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84958c-124c-4300-a8fd-ac39076d2708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:41.977381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84958c-124c-4300-a8fd-ac39076d2708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:41.977926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84958c-124c-4300-a8fd-ac39076d2708 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:41.978429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84958c-124c-4300-a8fd-ac39076d2708 admin admin] PATH_INFO: `/v3/users/7f5e1c1196ea4a86a24299749d6371da` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:41.979903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84958c-124c-4300-a8fd-ac39076d2708 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f5e1c1196ea4a86a24299749d6371da)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:41.983905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84958c-124c-4300-a8fd-ac39076d2708 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:42.084429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-de84958c-124c-4300-a8fd-ac39076d2708 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f5e1c1196ea4a86a24299749d6371da'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:42.095815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-de84958c-124c-4300-a8fd-ac39076d2708 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f5e1c1196ea4a86a24299749d6371da'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:42.108898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:42.117224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-de84958c-124c-4300-a8fd-ac39076d2708 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f5e1c1196ea4a86a24299749d6371da'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:42.120028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-de84958c-124c-4300-a8fd-ac39076d2708 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f5e1c1196ea4a86a24299749d6371da'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:42.133284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-de84958c-124c-4300-a8fd-ac39076d2708 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f5e1c1196ea4a86a24299749d6371da'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:42.167179 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2869/5661] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:09:41 2022] DELETE /identity/v3/users/7f5e1c1196ea4a86a24299749d6371da => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:42.174118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d598f8-19b4-435d-bb7b-3556ebc35a5f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:42.174972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d598f8-19b4-435d-bb7b-3556ebc35a5f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:42.175409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d598f8-19b4-435d-bb7b-3556ebc35a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:42.223461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-25d598f8-19b4-435d-bb7b-3556ebc35a5f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:42.228508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-25d598f8-19b4-435d-bb7b-3556ebc35a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:42.345762 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2793/5662] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:42.370748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84958c-124c-4300-a8fd-ac39076d2708 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:42.373644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-de84958c-124c-4300-a8fd-ac39076d2708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:42.501875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84958c-124c-4300-a8fd-ac39076d2708 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:42.506929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b26713-788e-46cc-b60d-87b6fd6efa25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:42.507853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b26713-788e-46cc-b60d-87b6fd6efa25 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:42.508280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b26713-788e-46cc-b60d-87b6fd6efa25 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:42.508650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b26713-788e-46cc-b60d-87b6fd6efa25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:42.520625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b26713-788e-46cc-b60d-87b6fd6efa25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:42.522409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b26713-788e-46cc-b60d-87b6fd6efa25 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:42.536824 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2870/5663] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:09:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:09:42.916013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d598f8-19b4-435d-bb7b-3556ebc35a5f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:42.917596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-25d598f8-19b4-435d-bb7b-3556ebc35a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:42.947107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:43.031574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d598f8-19b4-435d-bb7b-3556ebc35a5f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:43.041276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9b02e-eb8a-448d-8349-b1bfa75ccbd4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:43.042641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9b02e-eb8a-448d-8349-b1bfa75ccbd4 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:43.043322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9b02e-eb8a-448d-8349-b1bfa75ccbd4 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:43.044092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9b02e-eb8a-448d-8349-b1bfa75ccbd4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:43.046874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ebe9b02e-eb8a-448d-8349-b1bfa75ccbd4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:43.072842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9b02e-eb8a-448d-8349-b1bfa75ccbd4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:43.075213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9b02e-eb8a-448d-8349-b1bfa75ccbd4 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:43.089948 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2794/5664] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:09:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:09:43.256126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b26713-788e-46cc-b60d-87b6fd6efa25 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:43.258169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-16b26713-788e-46cc-b60d-87b6fd6efa25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:43.295132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b26713-788e-46cc-b60d-87b6fd6efa25 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:43.306354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0940c9-51cf-4c07-8ea9-71c60ff66ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:43.307222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0940c9-51cf-4c07-8ea9-71c60ff66ee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:43.308485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0940c9-51cf-4c07-8ea9-71c60ff66ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:43.310441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9b02e-eb8a-448d-8349-b1bfa75ccbd4 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:43.319857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0940c9-51cf-4c07-8ea9-71c60ff66ee4 admin admin] PATH_INFO: `/v3/projects/8fb7c0f568dc47249086aa49600baf74` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:43.323793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0940c9-51cf-4c07-8ea9-71c60ff66ee4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fb7c0f568dc47249086aa49600baf74)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:43.325145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9b02e-eb8a-448d-8349-b1bfa75ccbd4 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:43.326559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0940c9-51cf-4c07-8ea9-71c60ff66ee4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:43.336043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad5c3a-fb17-488c-9397-65f06da6b144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:43.336799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad5c3a-fb17-488c-9397-65f06da6b144 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:43.337576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad5c3a-fb17-488c-9397-65f06da6b144 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:43.338004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad5c3a-fb17-488c-9397-65f06da6b144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:43.341030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-73ad5c3a-fb17-488c-9397-65f06da6b144 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:43.368109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad5c3a-fb17-488c-9397-65f06da6b144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:43.372051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad5c3a-fb17-488c-9397-65f06da6b144 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:43.396363 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2795/5665] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:09:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:09:43.433315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4e0940c9-51cf-4c07-8ea9-71c60ff66ee4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fb7c0f568dc47249086aa49600baf74', 'request_id': 'req-4e0940c9-51cf-4c07-8ea9-71c60ff66ee4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:43.446923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4e0940c9-51cf-4c07-8ea9-71c60ff66ee4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fb7c0f568dc47249086aa49600baf74', 'request_id': 'req-4e0940c9-51cf-4c07-8ea9-71c60ff66ee4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:43.460813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4e0940c9-51cf-4c07-8ea9-71c60ff66ee4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fb7c0f568dc47249086aa49600baf74', 'request_id': 'req-4e0940c9-51cf-4c07-8ea9-71c60ff66ee4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:43.485394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4e0940c9-51cf-4c07-8ea9-71c60ff66ee4 admin admin] The token cache is being invalidate because project 8fb7c0f568dc47249086aa49600baf74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:09:43.486071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4e0940c9-51cf-4c07-8ea9-71c60ff66ee4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:43.487659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4e0940c9-51cf-4c07-8ea9-71c60ff66ee4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:43.493236 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2871/5666] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:09:43 2022] DELETE /identity/v3/projects/8fb7c0f568dc47249086aa49600baf74 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:43.499863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad5c3a-fb17-488c-9397-65f06da6b144 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:43.525622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad5c3a-fb17-488c-9397-65f06da6b144 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:43.705210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787d8ad-514d-4439-8b3b-2d9c45d16608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:43.708782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787d8ad-514d-4439-8b3b-2d9c45d16608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:43.709196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787d8ad-514d-4439-8b3b-2d9c45d16608 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:43.710791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787d8ad-514d-4439-8b3b-2d9c45d16608 admin admin] PATH_INFO: `/v3/users/6e6bc1b159e74b4993b87a43272f853d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:43.714190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4787d8ad-514d-4439-8b3b-2d9c45d16608 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e6bc1b159e74b4993b87a43272f853d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:43.717467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4787d8ad-514d-4439-8b3b-2d9c45d16608 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:43.821463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4787d8ad-514d-4439-8b3b-2d9c45d16608 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e6bc1b159e74b4993b87a43272f853d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:43.822603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4787d8ad-514d-4439-8b3b-2d9c45d16608 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e6bc1b159e74b4993b87a43272f853d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:43.823496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4787d8ad-514d-4439-8b3b-2d9c45d16608 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e6bc1b159e74b4993b87a43272f853d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:43.865690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4787d8ad-514d-4439-8b3b-2d9c45d16608 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e6bc1b159e74b4993b87a43272f853d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:43.898480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4787d8ad-514d-4439-8b3b-2d9c45d16608 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e6bc1b159e74b4993b87a43272f853d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:43.921268 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2796/5667] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:09:43 2022] DELETE /identity/v3/users/6e6bc1b159e74b4993b87a43272f853d => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:46.333450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0940c9-51cf-4c07-8ea9-71c60ff66ee4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:46.336856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4e0940c9-51cf-4c07-8ea9-71c60ff66ee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:46.373612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0940c9-51cf-4c07-8ea9-71c60ff66ee4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:46.495329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3383e-dc57-440f-9e91-0093a2f63dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:46.496248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3383e-dc57-440f-9e91-0093a2f63dfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:46.496807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3383e-dc57-440f-9e91-0093a2f63dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:46.497245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3383e-dc57-440f-9e91-0093a2f63dfa admin admin] PATH_INFO: `/v3/projects/fe509274efbb4895ac7a553465f58438` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:46.501927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3383e-dc57-440f-9e91-0093a2f63dfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe509274efbb4895ac7a553465f58438)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:46.504935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3383e-dc57-440f-9e91-0093a2f63dfa admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:46.533059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:46.621172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8ba3383e-dc57-440f-9e91-0093a2f63dfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe509274efbb4895ac7a553465f58438', 'request_id': 'req-8ba3383e-dc57-440f-9e91-0093a2f63dfa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:46.628570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8ba3383e-dc57-440f-9e91-0093a2f63dfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe509274efbb4895ac7a553465f58438', 'request_id': 'req-8ba3383e-dc57-440f-9e91-0093a2f63dfa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:46.643981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8ba3383e-dc57-440f-9e91-0093a2f63dfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe509274efbb4895ac7a553465f58438', 'request_id': 'req-8ba3383e-dc57-440f-9e91-0093a2f63dfa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:46.682381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8ba3383e-dc57-440f-9e91-0093a2f63dfa admin admin] The token cache is being invalidate because project fe509274efbb4895ac7a553465f58438 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:09:46.682868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8ba3383e-dc57-440f-9e91-0093a2f63dfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:46.685257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8ba3383e-dc57-440f-9e91-0093a2f63dfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:46.691842 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2872/5668] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:09:46 2022] DELETE /identity/v3/projects/fe509274efbb4895ac7a553465f58438 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:46.694243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca25e8-5365-4459-b7c0-84af49374f70 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:46.695184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca25e8-5365-4459-b7c0-84af49374f70 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:46.695761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca25e8-5365-4459-b7c0-84af49374f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:46.735749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b1ca25e8-5365-4459-b7c0-84af49374f70 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:46.740891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b1ca25e8-5365-4459-b7c0-84af49374f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:46.765913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:46.890267 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2797/5669] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:46.898872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38242a-f4c2-4a63-95ea-8b5350323f3f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:46.899074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38242a-f4c2-4a63-95ea-8b5350323f3f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:46.899287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38242a-f4c2-4a63-95ea-8b5350323f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:46.955368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1b38242a-f4c2-4a63-95ea-8b5350323f3f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:46.957541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1b38242a-f4c2-4a63-95ea-8b5350323f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:46.974255 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2873/5670] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:09:46.975247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca25e8-5365-4459-b7c0-84af49374f70 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:46.987453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca25e8-5365-4459-b7c0-84af49374f70 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:46.999336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa0767-8173-42b7-ba9e-9db6043d817b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:47.000341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa0767-8173-42b7-ba9e-9db6043d817b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:47.000917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa0767-8173-42b7-ba9e-9db6043d817b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:47.001409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa0767-8173-42b7-ba9e-9db6043d817b admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:47.002307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa0767-8173-42b7-ba9e-9db6043d817b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:09:47.002870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa0767-8173-42b7-ba9e-9db6043d817b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:47.006850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa0767-8173-42b7-ba9e-9db6043d817b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:47.076343 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2798/5671] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:09:47.077702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38242a-f4c2-4a63-95ea-8b5350323f3f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:47.098984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38242a-f4c2-4a63-95ea-8b5350323f3f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:47.131579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26447b9f-4b31-4713-abae-b9a63189af45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:47.139346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26447b9f-4b31-4713-abae-b9a63189af45 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:47.142220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26447b9f-4b31-4713-abae-b9a63189af45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:47.142872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26447b9f-4b31-4713-abae-b9a63189af45 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:47.143705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26447b9f-4b31-4713-abae-b9a63189af45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:47.157961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26447b9f-4b31-4713-abae-b9a63189af45 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:47.249978 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2874/5672] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:09:47 2022] POST /identity/v3/projects => generated 369 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:09:47.251642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa0767-8173-42b7-ba9e-9db6043d817b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:47.283527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa0767-8173-42b7-ba9e-9db6043d817b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:47.417545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae0487-84c7-4cdb-a93c-6b9f7bf0561c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:47.418261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae0487-84c7-4cdb-a93c-6b9f7bf0561c admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:47.418581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae0487-84c7-4cdb-a93c-6b9f7bf0561c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:47.418995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae0487-84c7-4cdb-a93c-6b9f7bf0561c admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:47.419642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae0487-84c7-4cdb-a93c-6b9f7bf0561c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:47.424660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae0487-84c7-4cdb-a93c-6b9f7bf0561c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:47.475112 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2799/5673] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:09:47 2022] POST /identity/v3/users => generated 285 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:09:47.479293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26447b9f-4b31-4713-abae-b9a63189af45 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:47.506158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26447b9f-4b31-4713-abae-b9a63189af45 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:47.511198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985fec6-cf33-427c-bda9-473f0d9b9b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:47.511945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985fec6-cf33-427c-bda9-473f0d9b9b2d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:47.512299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985fec6-cf33-427c-bda9-473f0d9b9b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:47.512582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985fec6-cf33-427c-bda9-473f0d9b9b2d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:47.513238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e985fec6-cf33-427c-bda9-473f0d9b9b2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:47.514882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e985fec6-cf33-427c-bda9-473f0d9b9b2d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:47.526524 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2875/5674] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:09:47.530452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae0487-84c7-4cdb-a93c-6b9f7bf0561c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:47.547277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae0487-84c7-4cdb-a93c-6b9f7bf0561c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:47.552914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e33715-cef8-4ea6-bd80-e2bba17dcf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:47.553519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e33715-cef8-4ea6-bd80-e2bba17dcf23 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:47.553803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e33715-cef8-4ea6-bd80-e2bba17dcf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:47.554156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e33715-cef8-4ea6-bd80-e2bba17dcf23 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:47.554889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e33715-cef8-4ea6-bd80-e2bba17dcf23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:47.556340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e33715-cef8-4ea6-bd80-e2bba17dcf23 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:47.570452 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2800/5675] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:09:47.575469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985fec6-cf33-427c-bda9-473f0d9b9b2d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:47.591827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985fec6-cf33-427c-bda9-473f0d9b9b2d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:47.597320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ac162-33ed-4c7d-a66e-964d96c352f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:47.598284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ac162-33ed-4c7d-a66e-964d96c352f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:47.598803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ac162-33ed-4c7d-a66e-964d96c352f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:47.599435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ac162-33ed-4c7d-a66e-964d96c352f2 admin admin] PATH_INFO: `/v3/projects/10e5449589be455e955adbb668c20887/users/71899288ea64484d965793ee62a74684/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:47.639617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ac162-33ed-4c7d-a66e-964d96c352f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10e5449589be455e955adbb668c20887, user_id=71899288ea64484d965793ee62a74684, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:47.643255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ac162-33ed-4c7d-a66e-964d96c352f2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:47.677015 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2876/5676] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:09:47 2022] PUT /identity/v3/projects/10e5449589be455e955adbb668c20887/users/71899288ea64484d965793ee62a74684/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:47.688670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f723b-1109-4264-a93d-adc518ff08be None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:47.688960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f723b-1109-4264-a93d-adc518ff08be None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:47.689242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f723b-1109-4264-a93d-adc518ff08be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:47.749898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-867f723b-1109-4264-a93d-adc518ff08be None None] MFA Rules not processed for user `71899288ea64484d965793ee62a74684`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:47.751609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-867f723b-1109-4264-a93d-adc518ff08be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:47.886162 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2801/5677] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:09:47.890957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ac162-33ed-4c7d-a66e-964d96c352f2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:48.001411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ac162-33ed-4c7d-a66e-964d96c352f2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:48.010867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56697ac-469c-45b0-8068-a762753e9545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:48.011863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56697ac-469c-45b0-8068-a762753e9545 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:48.012748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56697ac-469c-45b0-8068-a762753e9545 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:48.013174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56697ac-469c-45b0-8068-a762753e9545 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:48.013984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56697ac-469c-45b0-8068-a762753e9545 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:48.016149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56697ac-469c-45b0-8068-a762753e9545 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:48.065121 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2877/5678] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:09:47 2022] POST /identity/v3/projects => generated 369 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:09:48.070207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f723b-1109-4264-a93d-adc518ff08be None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:48.101689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f723b-1109-4264-a93d-adc518ff08be None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:48.117377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:48.209269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8216f-da1a-45f8-b0e3-169e8cdeb928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:48.209948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8216f-da1a-45f8-b0e3-169e8cdeb928 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:48.210233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8216f-da1a-45f8-b0e3-169e8cdeb928 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:48.210577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8216f-da1a-45f8-b0e3-169e8cdeb928 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:48.211248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f8216f-da1a-45f8-b0e3-169e8cdeb928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:48.213576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f8216f-da1a-45f8-b0e3-169e8cdeb928 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:48.264190 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2802/5679] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:09:48 2022] POST /identity/v3/users => generated 291 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:09:48.268188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56697ac-469c-45b0-8068-a762753e9545 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:48.283461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56697ac-469c-45b0-8068-a762753e9545 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:48.288387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414897ed-67c8-4945-bd6e-9c82ee145575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:48.289310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414897ed-67c8-4945-bd6e-9c82ee145575 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:48.289656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414897ed-67c8-4945-bd6e-9c82ee145575 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:48.289937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414897ed-67c8-4945-bd6e-9c82ee145575 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:48.290939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414897ed-67c8-4945-bd6e-9c82ee145575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:48.292279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414897ed-67c8-4945-bd6e-9c82ee145575 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:48.309331 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2878/5680] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:09:48.315000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8216f-da1a-45f8-b0e3-169e8cdeb928 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:48.332312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8216f-da1a-45f8-b0e3-169e8cdeb928 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:48.338290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ed870-0744-4f2f-9b76-ac2dbbe16f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:48.339014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ed870-0744-4f2f-9b76-ac2dbbe16f53 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:48.339740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ed870-0744-4f2f-9b76-ac2dbbe16f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:48.340028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ed870-0744-4f2f-9b76-ac2dbbe16f53 admin admin] PATH_INFO: `/v3/projects/1852a16bf4dd4d5583d420960929c6c5/users/f2b24b0982344e43a5d270cbd8e1bab3/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:48.365656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ed870-0744-4f2f-9b76-ac2dbbe16f53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1852a16bf4dd4d5583d420960929c6c5, user_id=f2b24b0982344e43a5d270cbd8e1bab3, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:48.369645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ed870-0744-4f2f-9b76-ac2dbbe16f53 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:48.399370 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2803/5681] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:09:48 2022] PUT /identity/v3/projects/1852a16bf4dd4d5583d420960929c6c5/users/f2b24b0982344e43a5d270cbd8e1bab3/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:48.404069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414897ed-67c8-4945-bd6e-9c82ee145575 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:48.417438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414897ed-67c8-4945-bd6e-9c82ee145575 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:48.426966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:48.523655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c9319-8eee-4e4d-ad0b-e79a6724d4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:48.525254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c9319-8eee-4e4d-ad0b-e79a6724d4ef admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:48.525766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c9319-8eee-4e4d-ad0b-e79a6724d4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:48.526231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c9319-8eee-4e4d-ad0b-e79a6724d4ef admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:48.527482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c9319-8eee-4e4d-ad0b-e79a6724d4ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:48.530109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c9319-8eee-4e4d-ad0b-e79a6724d4ef admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:48.552448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ed870-0744-4f2f-9b76-ac2dbbe16f53 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:48.558401 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2879/5682] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:09:48.566476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ed870-0744-4f2f-9b76-ac2dbbe16f53 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:48.581008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5604682-c8ab-49d4-a3e4-7dd4a6a0f348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:48.583545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5604682-c8ab-49d4-a3e4-7dd4a6a0f348 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:48.584017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5604682-c8ab-49d4-a3e4-7dd4a6a0f348 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:48.584439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5604682-c8ab-49d4-a3e4-7dd4a6a0f348 admin admin] PATH_INFO: `/v3/projects/1852a16bf4dd4d5583d420960929c6c5/users/f2b24b0982344e43a5d270cbd8e1bab3/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:48.586868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5604682-c8ab-49d4-a3e4-7dd4a6a0f348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1852a16bf4dd4d5583d420960929c6c5, user_id=f2b24b0982344e43a5d270cbd8e1bab3, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:48.590101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5604682-c8ab-49d4-a3e4-7dd4a6a0f348 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:48.606596 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2804/5683] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:09:48 2022] PUT /identity/v3/projects/1852a16bf4dd4d5583d420960929c6c5/users/f2b24b0982344e43a5d270cbd8e1bab3/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:48.619378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc59d55-1589-440f-9279-a4102b8b2711 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:48.619661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc59d55-1589-440f-9279-a4102b8b2711 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:48.620277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc59d55-1589-440f-9279-a4102b8b2711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:48.652288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5604682-c8ab-49d4-a3e4-7dd4a6a0f348 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:48.661438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a5604682-c8ab-49d4-a3e4-7dd4a6a0f348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:48.704127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3dc59d55-1589-440f-9279-a4102b8b2711 None None] MFA Rules not processed for user `f2b24b0982344e43a5d270cbd8e1bab3`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:48.705206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3dc59d55-1589-440f-9279-a4102b8b2711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:48.778235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5604682-c8ab-49d4-a3e4-7dd4a6a0f348 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:48.784274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec7420-f1b9-42de-aa9b-00fbfffac3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:48.785192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec7420-f1b9-42de-aa9b-00fbfffac3ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:48.785399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec7420-f1b9-42de-aa9b-00fbfffac3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:48.785683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec7420-f1b9-42de-aa9b-00fbfffac3ec admin admin] PATH_INFO: `/v3/users/d2a67ba740284d5589df8bdad6ab671f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:48.786905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec7420-f1b9-42de-aa9b-00fbfffac3ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2a67ba740284d5589df8bdad6ab671f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:48.788865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec7420-f1b9-42de-aa9b-00fbfffac3ec admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:48.883119 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2880/5684] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:48.887837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ccec7420-f1b9-42de-aa9b-00fbfffac3ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2a67ba740284d5589df8bdad6ab671f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:48.889516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ccec7420-f1b9-42de-aa9b-00fbfffac3ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2a67ba740284d5589df8bdad6ab671f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:48.890437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ccec7420-f1b9-42de-aa9b-00fbfffac3ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2a67ba740284d5589df8bdad6ab671f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:48.895652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc59d55-1589-440f-9279-a4102b8b2711 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:48.897585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3dc59d55-1589-440f-9279-a4102b8b2711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:48.906316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ccec7420-f1b9-42de-aa9b-00fbfffac3ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2a67ba740284d5589df8bdad6ab671f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:48.920272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=85,keystone:INSERT=6,keystone:DELETE=9 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:48.931259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ccec7420-f1b9-42de-aa9b-00fbfffac3ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2a67ba740284d5589df8bdad6ab671f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:48.958618 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2805/5685] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:09:48 2022] DELETE /identity/v3/users/d2a67ba740284d5589df8bdad6ab671f => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:49.035569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc59d55-1589-440f-9279-a4102b8b2711 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:49.119741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:49.130188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258b54d-03f6-431f-87c6-c8c483100110 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:49.131355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258b54d-03f6-431f-87c6-c8c483100110 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:49.131811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258b54d-03f6-431f-87c6-c8c483100110 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:49.132208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258b54d-03f6-431f-87c6-c8c483100110 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:49.135833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b258b54d-03f6-431f-87c6-c8c483100110 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:49.236807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b258b54d-03f6-431f-87c6-c8c483100110 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:49.240357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b258b54d-03f6-431f-87c6-c8c483100110 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:49.255106 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2881/5686] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:09:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:09:49.317578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec7420-f1b9-42de-aa9b-00fbfffac3ec admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:49.320827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ccec7420-f1b9-42de-aa9b-00fbfffac3ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:49.326857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258b54d-03f6-431f-87c6-c8c483100110 service cinder] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:49.329803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b258b54d-03f6-431f-87c6-c8c483100110 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:49.374072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258b54d-03f6-431f-87c6-c8c483100110 service cinder] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:49.413543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330db70-bbd5-4681-9e75-14608ac9c43e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:49.415048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330db70-bbd5-4681-9e75-14608ac9c43e service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:49.416993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330db70-bbd5-4681-9e75-14608ac9c43e service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:49.416993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330db70-bbd5-4681-9e75-14608ac9c43e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:49.446019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4330db70-bbd5-4681-9e75-14608ac9c43e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:49.544433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec7420-f1b9-42de-aa9b-00fbfffac3ec admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:49.552518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091f20c-e33a-4b29-bd4c-4b4fcfee9450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:49.553209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091f20c-e33a-4b29-bd4c-4b4fcfee9450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:49.553495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091f20c-e33a-4b29-bd4c-4b4fcfee9450 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:49.553798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091f20c-e33a-4b29-bd4c-4b4fcfee9450 admin admin] PATH_INFO: `/v3/projects/337bb3ce6f304c58902406f7279af3f1` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:49.555822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091f20c-e33a-4b29-bd4c-4b4fcfee9450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=337bb3ce6f304c58902406f7279af3f1)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:49.560503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091f20c-e33a-4b29-bd4c-4b4fcfee9450 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:49.587679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330db70-bbd5-4681-9e75-14608ac9c43e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:49.590585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330db70-bbd5-4681-9e75-14608ac9c43e service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:49.601337 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2882/5687] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:09:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:09:49.618017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c091f20c-e33a-4b29-bd4c-4b4fcfee9450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '337bb3ce6f304c58902406f7279af3f1', 'request_id': 'req-c091f20c-e33a-4b29-bd4c-4b4fcfee9450'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:49.627460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c091f20c-e33a-4b29-bd4c-4b4fcfee9450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '337bb3ce6f304c58902406f7279af3f1', 'request_id': 'req-c091f20c-e33a-4b29-bd4c-4b4fcfee9450'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:49.636512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c091f20c-e33a-4b29-bd4c-4b4fcfee9450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '337bb3ce6f304c58902406f7279af3f1', 'request_id': 'req-c091f20c-e33a-4b29-bd4c-4b4fcfee9450'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:49.653611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c091f20c-e33a-4b29-bd4c-4b4fcfee9450 admin admin] The token cache is being invalidate because project 337bb3ce6f304c58902406f7279af3f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:09:49.653907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c091f20c-e33a-4b29-bd4c-4b4fcfee9450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:49.655267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c091f20c-e33a-4b29-bd4c-4b4fcfee9450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:49.657041 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2806/5688] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:09:49 2022] DELETE /identity/v3/projects/337bb3ce6f304c58902406f7279af3f1 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:49.662861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4c68f-aee8-4d1e-887d-427a55c8f693 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:49.663499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4c68f-aee8-4d1e-887d-427a55c8f693 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:49.663819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4c68f-aee8-4d1e-887d-427a55c8f693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:49.687002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-bca4c68f-aee8-4d1e-887d-427a55c8f693 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:49.688244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bca4c68f-aee8-4d1e-887d-427a55c8f693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:49.758610 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2883/5689] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:09:49.766807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0b274-71e0-4d06-818c-fda28ccbd287 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:49.768383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0b274-71e0-4d06-818c-fda28ccbd287 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:49.769713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0b274-71e0-4d06-818c-fda28ccbd287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:49.800839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-aad0b274-71e0-4d06-818c-fda28ccbd287 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:49.803275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-aad0b274-71e0-4d06-818c-fda28ccbd287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:49.813102 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2807/5690] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:49.818104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4c68f-aee8-4d1e-887d-427a55c8f693 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:49.828408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4c68f-aee8-4d1e-887d-427a55c8f693 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:49.840939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6bed25-35f0-4c56-b733-cc0d8c6ade37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:49.841747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bed25-35f0-4c56-b733-cc0d8c6ade37 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:49.841874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bed25-35f0-4c56-b733-cc0d8c6ade37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:49.842274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bed25-35f0-4c56-b733-cc0d8c6ade37 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:49.842826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6bed25-35f0-4c56-b733-cc0d8c6ade37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:09:49.843158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6bed25-35f0-4c56-b733-cc0d8c6ade37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:49.844793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6bed25-35f0-4c56-b733-cc0d8c6ade37 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:49.861958 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2884/5691] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:09:49.866324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0b274-71e0-4d06-818c-fda28ccbd287 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:49.878662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0b274-71e0-4d06-818c-fda28ccbd287 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:49.885202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b475ff-1ec1-451f-8aaf-8607fae56c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:49.886182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b475ff-1ec1-451f-8aaf-8607fae56c03 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:49.886464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b475ff-1ec1-451f-8aaf-8607fae56c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:49.886838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b475ff-1ec1-451f-8aaf-8607fae56c03 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:49.887606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b475ff-1ec1-451f-8aaf-8607fae56c03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:49.891144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b475ff-1ec1-451f-8aaf-8607fae56c03 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:49.939061 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2808/5692] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:09:49 2022] POST /identity/v3/projects => generated 387 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:09:49.943660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6bed25-35f0-4c56-b733-cc0d8c6ade37 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:49.958534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6bed25-35f0-4c56-b733-cc0d8c6ade37 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:49.999749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:50.041162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ddd41f-caa1-4bca-8e24-7cae3f6186d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:50.042004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddd41f-caa1-4bca-8e24-7cae3f6186d2 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:50.042377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddd41f-caa1-4bca-8e24-7cae3f6186d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:50.043252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddd41f-caa1-4bca-8e24-7cae3f6186d2 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:50.044576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ddd41f-caa1-4bca-8e24-7cae3f6186d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:50.046533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ddd41f-caa1-4bca-8e24-7cae3f6186d2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:50.070839 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2885/5693] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:09:49 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:09:50.074803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b475ff-1ec1-451f-8aaf-8607fae56c03 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:50.086741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b475ff-1ec1-451f-8aaf-8607fae56c03 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:50.091560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea634789-3ec2-4457-b1a0-00c700809023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:50.092236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea634789-3ec2-4457-b1a0-00c700809023 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:50.092590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea634789-3ec2-4457-b1a0-00c700809023 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:50.092959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea634789-3ec2-4457-b1a0-00c700809023 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:50.093810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea634789-3ec2-4457-b1a0-00c700809023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:50.096267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea634789-3ec2-4457-b1a0-00c700809023 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:50.107461 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2809/5694] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:09:50.111727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ddd41f-caa1-4bca-8e24-7cae3f6186d2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:50.123161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ddd41f-caa1-4bca-8e24-7cae3f6186d2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:50.128079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935abfd9-a4ec-4ef2-a7a5-564b6e25fa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:50.128771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935abfd9-a4ec-4ef2-a7a5-564b6e25fa98 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:50.129122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935abfd9-a4ec-4ef2-a7a5-564b6e25fa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:50.129403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935abfd9-a4ec-4ef2-a7a5-564b6e25fa98 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:50.130041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935abfd9-a4ec-4ef2-a7a5-564b6e25fa98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:50.131770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935abfd9-a4ec-4ef2-a7a5-564b6e25fa98 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:50.145198 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2886/5695] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:09:50.146295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea634789-3ec2-4457-b1a0-00c700809023 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:50.156384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea634789-3ec2-4457-b1a0-00c700809023 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:50.161713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19d41a-63ec-4332-9539-fbbeb58adb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:50.162374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19d41a-63ec-4332-9539-fbbeb58adb1a admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:50.162691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19d41a-63ec-4332-9539-fbbeb58adb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:50.162968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19d41a-63ec-4332-9539-fbbeb58adb1a admin admin] PATH_INFO: `/v3/projects/1215f2e4b926472794f0e60cefa2d90e/users/4f3c9d35f6b843ef905f43d758a9bb25/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:50.179488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b19d41a-63ec-4332-9539-fbbeb58adb1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1215f2e4b926472794f0e60cefa2d90e, user_id=4f3c9d35f6b843ef905f43d758a9bb25, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:50.182177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b19d41a-63ec-4332-9539-fbbeb58adb1a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:50.196090 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2810/5696] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:09:50 2022] PUT /identity/v3/projects/1215f2e4b926472794f0e60cefa2d90e/users/4f3c9d35f6b843ef905f43d758a9bb25/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:50.201405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0ab20-e786-4968-bace-f001767db56c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:50.201705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0ab20-e786-4968-bace-f001767db56c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:50.201988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0ab20-e786-4968-bace-f001767db56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:50.222002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-94b0ab20-e786-4968-bace-f001767db56c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:50.223787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-94b0ab20-e786-4968-bace-f001767db56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:50.285324 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2887/5697] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:50.295247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19d41a-63ec-4332-9539-fbbeb58adb1a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:50.296990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0b19d41a-63ec-4332-9539-fbbeb58adb1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:50.307781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:50.359103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19d41a-63ec-4332-9539-fbbeb58adb1a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:50.364315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936197c6-6a6a-4375-911c-60101caad39a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:50.365142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936197c6-6a6a-4375-911c-60101caad39a service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:50.365262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936197c6-6a6a-4375-911c-60101caad39a service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:50.365527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936197c6-6a6a-4375-911c-60101caad39a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:50.378207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936197c6-6a6a-4375-911c-60101caad39a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:50.379766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936197c6-6a6a-4375-911c-60101caad39a service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:50.393418 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2811/5698] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:09:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:09:51.670631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba94311-1c72-4070-99d5-5c99607ff9c5 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:51.671139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba94311-1c72-4070-99d5-5c99607ff9c5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:51.671319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba94311-1c72-4070-99d5-5c99607ff9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:51.736275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5ba94311-1c72-4070-99d5-5c99607ff9c5 None None] MFA Rules not processed for user `902ba44d318d4a01a510f732d27467c9`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:51.737553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5ba94311-1c72-4070-99d5-5c99607ff9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:51.883968 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2888/5699] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:51.895006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936197c6-6a6a-4375-911c-60101caad39a service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:51.897913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-936197c6-6a6a-4375-911c-60101caad39a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:52.087745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936197c6-6a6a-4375-911c-60101caad39a service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:52.093746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ea572-84d6-476c-808e-05b04df2ef3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:52.094371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ea572-84d6-476c-808e-05b04df2ef3b service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:52.094720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ea572-84d6-476c-808e-05b04df2ef3b service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:52.095198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ea572-84d6-476c-808e-05b04df2ef3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:52.104265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ea572-84d6-476c-808e-05b04df2ef3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:52.106170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ea572-84d6-476c-808e-05b04df2ef3b service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:52.128891 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2812/5700] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:09:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:09:53.841321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba94311-1c72-4070-99d5-5c99607ff9c5 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:53.843489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5ba94311-1c72-4070-99d5-5c99607ff9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:53.877048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba94311-1c72-4070-99d5-5c99607ff9c5 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:53.898046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0ae76-ca31-42a7-bc4a-84f6acabf4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:53.898754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0ae76-ca31-42a7-bc4a-84f6acabf4fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:53.899210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0ae76-ca31-42a7-bc4a-84f6acabf4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:53.899676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0ae76-ca31-42a7-bc4a-84f6acabf4fb admin admin] PATH_INFO: `/v3/users/71899288ea64484d965793ee62a74684` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:53.902580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0ae76-ca31-42a7-bc4a-84f6acabf4fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=71899288ea64484d965793ee62a74684)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:53.904527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0ae76-ca31-42a7-bc4a-84f6acabf4fb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:53.974695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:54.025617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-30b0ae76-ca31-42a7-bc4a-84f6acabf4fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71899288ea64484d965793ee62a74684'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:54.051838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-30b0ae76-ca31-42a7-bc4a-84f6acabf4fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71899288ea64484d965793ee62a74684'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:54.107408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-30b0ae76-ca31-42a7-bc4a-84f6acabf4fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71899288ea64484d965793ee62a74684'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:54.107408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-30b0ae76-ca31-42a7-bc4a-84f6acabf4fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71899288ea64484d965793ee62a74684'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:54.116094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-30b0ae76-ca31-42a7-bc4a-84f6acabf4fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71899288ea64484d965793ee62a74684'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:54.128786 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2889/5701] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:09:53 2022] DELETE /identity/v3/users/71899288ea64484d965793ee62a74684 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:54.131082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f778bc-4e0d-4595-9592-059db8450fc1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:54.132204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f778bc-4e0d-4595-9592-059db8450fc1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:54.132815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f778bc-4e0d-4595-9592-059db8450fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:54.159102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-49f778bc-4e0d-4595-9592-059db8450fc1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:54.164356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-49f778bc-4e0d-4595-9592-059db8450fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:54.303053 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2813/5702] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:54.360504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0ae76-ca31-42a7-bc4a-84f6acabf4fb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:54.363390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-30b0ae76-ca31-42a7-bc4a-84f6acabf4fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:54.497664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0ae76-ca31-42a7-bc4a-84f6acabf4fb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:54.504182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd76f1e-b7d3-40a8-af40-a68aeed32415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:54.505164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd76f1e-b7d3-40a8-af40-a68aeed32415 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:54.505672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd76f1e-b7d3-40a8-af40-a68aeed32415 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:54.506135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd76f1e-b7d3-40a8-af40-a68aeed32415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:54.533288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd76f1e-b7d3-40a8-af40-a68aeed32415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:54.534933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd76f1e-b7d3-40a8-af40-a68aeed32415 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:54.560486 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2890/5703] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:09:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:09:56.036823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713cb891-6d53-42b6-befa-44b1ed004430 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:56.037518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713cb891-6d53-42b6-befa-44b1ed004430 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:56.037518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713cb891-6d53-42b6-befa-44b1ed004430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:56.066594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-713cb891-6d53-42b6-befa-44b1ed004430 None None] MFA Rules not processed for user `902ba44d318d4a01a510f732d27467c9`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:56.068061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-713cb891-6d53-42b6-befa-44b1ed004430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:56.149071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:56.210445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd76f1e-b7d3-40a8-af40-a68aeed32415 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:56.212682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ffd76f1e-b7d3-40a8-af40-a68aeed32415 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:56.270082 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2814/5704] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:56.292872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd76f1e-b7d3-40a8-af40-a68aeed32415 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:56.308791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37275b88-6f91-4bf6-96d8-ea2d8e00b84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:56.309743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37275b88-6f91-4bf6-96d8-ea2d8e00b84e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:56.310188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37275b88-6f91-4bf6-96d8-ea2d8e00b84e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:56.310697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37275b88-6f91-4bf6-96d8-ea2d8e00b84e admin admin] PATH_INFO: `/v3/projects/10e5449589be455e955adbb668c20887` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:56.312036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37275b88-6f91-4bf6-96d8-ea2d8e00b84e admin admin] RBAC: Authorizing `identity:delete_project(project_id=10e5449589be455e955adbb668c20887)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:56.314846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37275b88-6f91-4bf6-96d8-ea2d8e00b84e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:56.424739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-37275b88-6f91-4bf6-96d8-ea2d8e00b84e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10e5449589be455e955adbb668c20887', 'request_id': 'req-37275b88-6f91-4bf6-96d8-ea2d8e00b84e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:56.431813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-37275b88-6f91-4bf6-96d8-ea2d8e00b84e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10e5449589be455e955adbb668c20887', 'request_id': 'req-37275b88-6f91-4bf6-96d8-ea2d8e00b84e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:56.438414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-37275b88-6f91-4bf6-96d8-ea2d8e00b84e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10e5449589be455e955adbb668c20887', 'request_id': 'req-37275b88-6f91-4bf6-96d8-ea2d8e00b84e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:56.468265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-37275b88-6f91-4bf6-96d8-ea2d8e00b84e admin admin] The token cache is being invalidate because project 10e5449589be455e955adbb668c20887 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:09:56.468601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-37275b88-6f91-4bf6-96d8-ea2d8e00b84e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:56.469332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-37275b88-6f91-4bf6-96d8-ea2d8e00b84e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:56.471824 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2891/5705] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:09:56 2022] DELETE /identity/v3/projects/10e5449589be455e955adbb668c20887 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:56.475582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713cb891-6d53-42b6-befa-44b1ed004430 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:56.489119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713cb891-6d53-42b6-befa-44b1ed004430 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:56.579319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81345789-6c58-48b2-a761-763e0aa393bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:56.579895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81345789-6c58-48b2-a761-763e0aa393bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:56.580173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81345789-6c58-48b2-a761-763e0aa393bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:56.580463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81345789-6c58-48b2-a761-763e0aa393bc admin admin] PATH_INFO: `/v3/users/f2b24b0982344e43a5d270cbd8e1bab3` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:56.582178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81345789-6c58-48b2-a761-763e0aa393bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2b24b0982344e43a5d270cbd8e1bab3)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:56.584168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81345789-6c58-48b2-a761-763e0aa393bc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:56.680240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-81345789-6c58-48b2-a761-763e0aa393bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2b24b0982344e43a5d270cbd8e1bab3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:56.687387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-81345789-6c58-48b2-a761-763e0aa393bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2b24b0982344e43a5d270cbd8e1bab3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:56.688502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-81345789-6c58-48b2-a761-763e0aa393bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2b24b0982344e43a5d270cbd8e1bab3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:56.709034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-81345789-6c58-48b2-a761-763e0aa393bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2b24b0982344e43a5d270cbd8e1bab3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:56.732605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-81345789-6c58-48b2-a761-763e0aa393bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2b24b0982344e43a5d270cbd8e1bab3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:56.759284 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2815/5706] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:09:56 2022] DELETE /identity/v3/users/f2b24b0982344e43a5d270cbd8e1bab3 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:56.981819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37275b88-6f91-4bf6-96d8-ea2d8e00b84e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:56.984461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-37275b88-6f91-4bf6-96d8-ea2d8e00b84e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:57.082332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:57.141371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37275b88-6f91-4bf6-96d8-ea2d8e00b84e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:57.234201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe9771-fd93-44c0-84f9-f1b8d1e88221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:57.235994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe9771-fd93-44c0-84f9-f1b8d1e88221 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:57.236451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe9771-fd93-44c0-84f9-f1b8d1e88221 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:57.236837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe9771-fd93-44c0-84f9-f1b8d1e88221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:57.240032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-97fe9771-fd93-44c0-84f9-f1b8d1e88221 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:57.260146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe9771-fd93-44c0-84f9-f1b8d1e88221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:57.262913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe9771-fd93-44c0-84f9-f1b8d1e88221 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:57.267841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81345789-6c58-48b2-a761-763e0aa393bc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:57.269843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-81345789-6c58-48b2-a761-763e0aa393bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:57.278623 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2892/5707] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:09:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:09:57.448946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81345789-6c58-48b2-a761-763e0aa393bc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:57.459713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228f3a8-2f49-4f7e-8ad3-77c319695df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:57.460733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228f3a8-2f49-4f7e-8ad3-77c319695df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:57.461205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228f3a8-2f49-4f7e-8ad3-77c319695df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:57.461665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228f3a8-2f49-4f7e-8ad3-77c319695df0 admin admin] PATH_INFO: `/v3/projects/1852a16bf4dd4d5583d420960929c6c5` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:57.463195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228f3a8-2f49-4f7e-8ad3-77c319695df0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1852a16bf4dd4d5583d420960929c6c5)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:57.465459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228f3a8-2f49-4f7e-8ad3-77c319695df0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:57.469296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:57.562735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1228f3a8-2f49-4f7e-8ad3-77c319695df0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1852a16bf4dd4d5583d420960929c6c5', 'request_id': 'req-1228f3a8-2f49-4f7e-8ad3-77c319695df0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:57.576846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1228f3a8-2f49-4f7e-8ad3-77c319695df0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1852a16bf4dd4d5583d420960929c6c5', 'request_id': 'req-1228f3a8-2f49-4f7e-8ad3-77c319695df0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:57.587202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1228f3a8-2f49-4f7e-8ad3-77c319695df0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1852a16bf4dd4d5583d420960929c6c5', 'request_id': 'req-1228f3a8-2f49-4f7e-8ad3-77c319695df0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:57.604615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1228f3a8-2f49-4f7e-8ad3-77c319695df0 admin admin] The token cache is being invalidate because project 1852a16bf4dd4d5583d420960929c6c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:09:57.605126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1228f3a8-2f49-4f7e-8ad3-77c319695df0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:57.605990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1228f3a8-2f49-4f7e-8ad3-77c319695df0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:09:57.607895 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2816/5708] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:09:57 2022] DELETE /identity/v3/projects/1852a16bf4dd4d5583d420960929c6c5 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:09:57.612461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26bd5e-e4ea-43c2-b4bf-fa565bc3ee8c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:57.612989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26bd5e-e4ea-43c2-b4bf-fa565bc3ee8c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:57.613362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26bd5e-e4ea-43c2-b4bf-fa565bc3ee8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:57.638766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3d26bd5e-e4ea-43c2-b4bf-fa565bc3ee8c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:57.640268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3d26bd5e-e4ea-43c2-b4bf-fa565bc3ee8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:57.760716 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2893/5709] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:57.771024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18698721-67c4-4f23-981b-34327c37f098 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:57.771446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18698721-67c4-4f23-981b-34327c37f098 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:57.771814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18698721-67c4-4f23-981b-34327c37f098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:57.831011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-18698721-67c4-4f23-981b-34327c37f098 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:57.834163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-18698721-67c4-4f23-981b-34327c37f098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:57.858604 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2817/5710] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:57.864972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26bd5e-e4ea-43c2-b4bf-fa565bc3ee8c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:57.876658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26bd5e-e4ea-43c2-b4bf-fa565bc3ee8c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:57.888906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b3d1b-eef6-48e2-8fbf-d2ac7a8e7298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:57.892273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b3d1b-eef6-48e2-8fbf-d2ac7a8e7298 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:57.892678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b3d1b-eef6-48e2-8fbf-d2ac7a8e7298 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:57.893071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b3d1b-eef6-48e2-8fbf-d2ac7a8e7298 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:57.894471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b3d1b-eef6-48e2-8fbf-d2ac7a8e7298 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:09:57.897016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b3d1b-eef6-48e2-8fbf-d2ac7a8e7298 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:57.899866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b3d1b-eef6-48e2-8fbf-d2ac7a8e7298 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:57.925776 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2894/5711] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:09:57.931483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18698721-67c4-4f23-981b-34327c37f098 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:57.943104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18698721-67c4-4f23-981b-34327c37f098 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:57.954370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2990b1c-0093-4344-9252-898b990953df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:57.956009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2990b1c-0093-4344-9252-898b990953df admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:57.956009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2990b1c-0093-4344-9252-898b990953df admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:57.956009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2990b1c-0093-4344-9252-898b990953df admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:57.957291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2990b1c-0093-4344-9252-898b990953df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:57.963349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2990b1c-0093-4344-9252-898b990953df admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:58.029478 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2818/5712] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:09:57 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:09:58.033828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b3d1b-eef6-48e2-8fbf-d2ac7a8e7298 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:58.052311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b3d1b-eef6-48e2-8fbf-d2ac7a8e7298 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:58.206055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56b79a-6087-4b3f-a2a0-ca96a62b1e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:58.213447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56b79a-6087-4b3f-a2a0-ca96a62b1e35 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:58.214088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56b79a-6087-4b3f-a2a0-ca96a62b1e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:58.217101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56b79a-6087-4b3f-a2a0-ca96a62b1e35 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:58.218145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56b79a-6087-4b3f-a2a0-ca96a62b1e35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:58.225067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56b79a-6087-4b3f-a2a0-ca96a62b1e35 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:58.266699 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2895/5713] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:09:58 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:09:58.269033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2990b1c-0093-4344-9252-898b990953df admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:58.283883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2990b1c-0093-4344-9252-898b990953df admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:58.310527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e272c-6a92-4010-9a87-8f56ac97d167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:58.311206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e272c-6a92-4010-9a87-8f56ac97d167 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:58.311480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e272c-6a92-4010-9a87-8f56ac97d167 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:58.311757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e272c-6a92-4010-9a87-8f56ac97d167 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:58.317473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e272c-6a92-4010-9a87-8f56ac97d167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:58.323722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e272c-6a92-4010-9a87-8f56ac97d167 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:58.355276 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2819/5714] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:09:58.361873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56b79a-6087-4b3f-a2a0-ca96a62b1e35 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:58.375343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56b79a-6087-4b3f-a2a0-ca96a62b1e35 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:58.388635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dfc37-f85f-44c7-814a-510073ab4d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:58.389525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dfc37-f85f-44c7-814a-510073ab4d37 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:58.390039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dfc37-f85f-44c7-814a-510073ab4d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:58.390707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dfc37-f85f-44c7-814a-510073ab4d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:58.391599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484dfc37-f85f-44c7-814a-510073ab4d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:58.395874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484dfc37-f85f-44c7-814a-510073ab4d37 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:58.401242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:58.434175 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2896/5715] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:09:58.438240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e272c-6a92-4010-9a87-8f56ac97d167 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:58.461984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e272c-6a92-4010-9a87-8f56ac97d167 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:58.473891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12e1db-7b37-4054-84e9-95ee78a03c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:58.478295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12e1db-7b37-4054-84e9-95ee78a03c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:58.478610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12e1db-7b37-4054-84e9-95ee78a03c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:58.478893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12e1db-7b37-4054-84e9-95ee78a03c70 admin admin] PATH_INFO: `/v3/projects/438396a805cb4950ba2f788942425ee3/users/50dae5a392834816b8f8bf29c1756438/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:58.508200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b12e1db-7b37-4054-84e9-95ee78a03c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=438396a805cb4950ba2f788942425ee3, user_id=50dae5a392834816b8f8bf29c1756438, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:58.515181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b12e1db-7b37-4054-84e9-95ee78a03c70 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:58.558808 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2820/5716] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:09:58 2022] PUT /identity/v3/projects/438396a805cb4950ba2f788942425ee3/users/50dae5a392834816b8f8bf29c1756438/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:09:58.569828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2279f-8544-4f38-b68b-ed1859e745e7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:58.570158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2279f-8544-4f38-b68b-ed1859e745e7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:58.570388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2279f-8544-4f38-b68b-ed1859e745e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:58.592486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12e1db-7b37-4054-84e9-95ee78a03c70 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:58.595111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4b12e1db-7b37-4054-84e9-95ee78a03c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:58.632356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ecd2279f-8544-4f38-b68b-ed1859e745e7 None None] MFA Rules not processed for user `50dae5a392834816b8f8bf29c1756438`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:58.633598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ecd2279f-8544-4f38-b68b-ed1859e745e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:58.717413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12e1db-7b37-4054-84e9-95ee78a03c70 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:58.727717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1af20-dab2-45c7-bea4-709d897eb6a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:58.727717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1af20-dab2-45c7-bea4-709d897eb6a1 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:58.728161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1af20-dab2-45c7-bea4-709d897eb6a1 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:58.728680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1af20-dab2-45c7-bea4-709d897eb6a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:58.731048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a2a1af20-dab2-45c7-bea4-709d897eb6a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:58.750700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1af20-dab2-45c7-bea4-709d897eb6a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:58.752648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1af20-dab2-45c7-bea4-709d897eb6a1 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:58.768380 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2821/5717] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:09:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:09:58.794220 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2897/5718] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:58.799539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1af20-dab2-45c7-bea4-709d897eb6a1 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:58.819646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:58.933713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1af20-dab2-45c7-bea4-709d897eb6a1 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:58.954085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcdf7c-6565-4c08-90ac-d99e12ceb19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:58.960128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcdf7c-6565-4c08-90ac-d99e12ceb19a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:58.961827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcdf7c-6565-4c08-90ac-d99e12ceb19a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:58.962208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcdf7c-6565-4c08-90ac-d99e12ceb19a admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:58.994803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcdf7c-6565-4c08-90ac-d99e12ceb19a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:58.998344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcdf7c-6565-4c08-90ac-d99e12ceb19a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:59.108026 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2822/5719] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:09:58 2022] POST /identity/v3/projects => generated 387 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:09:59.110335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2279f-8544-4f38-b68b-ed1859e745e7 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:59.132822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2279f-8544-4f38-b68b-ed1859e745e7 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:59.340212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165bb43-ea95-4fb8-bab6-1df5326842f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:59.340883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165bb43-ea95-4fb8-bab6-1df5326842f2 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:59.341108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165bb43-ea95-4fb8-bab6-1df5326842f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:59.343406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165bb43-ea95-4fb8-bab6-1df5326842f2 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:59.344410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165bb43-ea95-4fb8-bab6-1df5326842f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:59.347280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165bb43-ea95-4fb8-bab6-1df5326842f2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:59.398416 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2898/5720] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:09:59 2022] POST /identity/v3/users => generated 294 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:09:59.399571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcdf7c-6565-4c08-90ac-d99e12ceb19a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:59.414371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcdf7c-6565-4c08-90ac-d99e12ceb19a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:59.419858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6f057-c7f3-41a9-823d-b2b2ff66a726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:59.420640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6f057-c7f3-41a9-823d-b2b2ff66a726 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:59.421027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6f057-c7f3-41a9-823d-b2b2ff66a726 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:59.421674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6f057-c7f3-41a9-823d-b2b2ff66a726 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:59.422444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6f057-c7f3-41a9-823d-b2b2ff66a726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:59.424243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6f057-c7f3-41a9-823d-b2b2ff66a726 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:59.441586 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2823/5721] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:09:59.442361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165bb43-ea95-4fb8-bab6-1df5326842f2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:59.455107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165bb43-ea95-4fb8-bab6-1df5326842f2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:59.460496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53302cb-2c85-465a-b9eb-2680a2257969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:59.461346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53302cb-2c85-465a-b9eb-2680a2257969 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:59.461738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53302cb-2c85-465a-b9eb-2680a2257969 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:59.462273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53302cb-2c85-465a-b9eb-2680a2257969 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:59.463329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53302cb-2c85-465a-b9eb-2680a2257969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:59.465039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53302cb-2c85-465a-b9eb-2680a2257969 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:59.481789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6f057-c7f3-41a9-823d-b2b2ff66a726 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:59.489591 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2899/5722] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:09:59.497862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6f057-c7f3-41a9-823d-b2b2ff66a726 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:09:59.503802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b10d903-a862-46e9-957a-ffd8abbd5d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:09:59.504624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b10d903-a862-46e9-957a-ffd8abbd5d14 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:59.505013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b10d903-a862-46e9-957a-ffd8abbd5d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:59.505394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b10d903-a862-46e9-957a-ffd8abbd5d14 admin admin] PATH_INFO: `/v3/projects/dc5012199dee456bb4064da27232e49d/users/1f1b0b24bebd426db6c9b92f3e318036/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:59.538570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b10d903-a862-46e9-957a-ffd8abbd5d14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc5012199dee456bb4064da27232e49d, user_id=1f1b0b24bebd426db6c9b92f3e318036, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:09:59.543003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b10d903-a862-46e9-957a-ffd8abbd5d14 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:09:59.584264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a5e3e-3074-4302-8c10-26f807640a81 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:09:59.584611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a5e3e-3074-4302-8c10-26f807640a81 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:09:59.585527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a5e3e-3074-4302-8c10-26f807640a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:09:59.589549 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2824/5723] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:09:59 2022] PUT /identity/v3/projects/dc5012199dee456bb4064da27232e49d/users/1f1b0b24bebd426db6c9b92f3e318036/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:09:59.633275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-292a5e3e-3074-4302-8c10-26f807640a81 None None] MFA Rules not processed for user `1f1b0b24bebd426db6c9b92f3e318036`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:09:59.635100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-292a5e3e-3074-4302-8c10-26f807640a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:09:59.774654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:59.805595 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2900/5724] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:09:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:09:59.811574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b10d903-a862-46e9-957a-ffd8abbd5d14 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:09:59.984740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b10d903-a862-46e9-957a-ffd8abbd5d14 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:00.004802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe2596-382a-4a99-a71c-22f7cfd661e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:00.005806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe2596-382a-4a99-a71c-22f7cfd661e0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:00.006214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe2596-382a-4a99-a71c-22f7cfd661e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:00.006705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe2596-382a-4a99-a71c-22f7cfd661e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:00.007623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe2596-382a-4a99-a71c-22f7cfd661e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:00.010023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe2596-382a-4a99-a71c-22f7cfd661e0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:00.070278 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2825/5725] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:09:59 2022] POST /identity/v3/projects => generated 385 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:10:00.074244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a5e3e-3074-4302-8c10-26f807640a81 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:00.090329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a5e3e-3074-4302-8c10-26f807640a81 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:00.233809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e78449-fe4e-410e-9527-28d462cb3686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:00.235160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e78449-fe4e-410e-9527-28d462cb3686 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:00.235636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e78449-fe4e-410e-9527-28d462cb3686 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:00.236155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e78449-fe4e-410e-9527-28d462cb3686 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:00.237083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e78449-fe4e-410e-9527-28d462cb3686 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:00.240035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e78449-fe4e-410e-9527-28d462cb3686 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:00.298267 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2901/5726] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:10:00 2022] POST /identity/v3/users => generated 299 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:10:00.301042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe2596-382a-4a99-a71c-22f7cfd661e0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:00.315034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe2596-382a-4a99-a71c-22f7cfd661e0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:00.341686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfed84-096c-4d4e-81b1-987da5ccf9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:00.342324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfed84-096c-4d4e-81b1-987da5ccf9fc admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:00.342636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfed84-096c-4d4e-81b1-987da5ccf9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:00.342917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfed84-096c-4d4e-81b1-987da5ccf9fc admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:00.343579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dfed84-096c-4d4e-81b1-987da5ccf9fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:00.345240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dfed84-096c-4d4e-81b1-987da5ccf9fc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:00.374587 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2826/5727] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:00.379479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e78449-fe4e-410e-9527-28d462cb3686 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:00.405334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e78449-fe4e-410e-9527-28d462cb3686 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:00.419434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70306d4e-2082-411d-b56c-4c240b136973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:00.419434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70306d4e-2082-411d-b56c-4c240b136973 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:00.420075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70306d4e-2082-411d-b56c-4c240b136973 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:00.420368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70306d4e-2082-411d-b56c-4c240b136973 admin admin] PATH_INFO: `/v3/projects/71af754245d74edc8aa25bba18235192/users/c1e4c4fa12184befb9449cad088d7829/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:00.462065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70306d4e-2082-411d-b56c-4c240b136973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71af754245d74edc8aa25bba18235192, user_id=c1e4c4fa12184befb9449cad088d7829, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:00.468617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70306d4e-2082-411d-b56c-4c240b136973 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:00.532756 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2902/5728] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:10:00 2022] PUT /identity/v3/projects/71af754245d74edc8aa25bba18235192/users/c1e4c4fa12184befb9449cad088d7829/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:00.537543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfed84-096c-4d4e-81b1-987da5ccf9fc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:00.561955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfed84-096c-4d4e-81b1-987da5ccf9fc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:00.657339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:00.695622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d69a97-f97a-4eb9-a595-e24cb436e5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:00.696412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d69a97-f97a-4eb9-a595-e24cb436e5dd admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:00.696771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d69a97-f97a-4eb9-a595-e24cb436e5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:00.697098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d69a97-f97a-4eb9-a595-e24cb436e5dd admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:00.697837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d69a97-f97a-4eb9-a595-e24cb436e5dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:00.700877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d69a97-f97a-4eb9-a595-e24cb436e5dd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:00.715038 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2827/5729] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:00.723306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70306d4e-2082-411d-b56c-4c240b136973 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:00.743920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70306d4e-2082-411d-b56c-4c240b136973 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:00.780280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261cb14-f40b-4502-9581-07d261e200b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:00.783927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261cb14-f40b-4502-9581-07d261e200b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:00.785051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261cb14-f40b-4502-9581-07d261e200b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:00.785998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261cb14-f40b-4502-9581-07d261e200b3 admin admin] PATH_INFO: `/v3/projects/71af754245d74edc8aa25bba18235192/users/c1e4c4fa12184befb9449cad088d7829/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:00.809044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261cb14-f40b-4502-9581-07d261e200b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71af754245d74edc8aa25bba18235192, user_id=c1e4c4fa12184befb9449cad088d7829, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:00.821007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261cb14-f40b-4502-9581-07d261e200b3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:00.824698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15681012-6914-4194-8399-653549103dfb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:00.825062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15681012-6914-4194-8399-653549103dfb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:00.825477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15681012-6914-4194-8399-653549103dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:00.854997 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2903/5730] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:10:00 2022] PUT /identity/v3/projects/71af754245d74edc8aa25bba18235192/users/c1e4c4fa12184befb9449cad088d7829/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:00.865629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dc9d6-1b8c-40a7-a841-2c9f17ea0f00 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:00.866386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dc9d6-1b8c-40a7-a841-2c9f17ea0f00 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:00.866890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dc9d6-1b8c-40a7-a841-2c9f17ea0f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:00.930923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-cd6dc9d6-1b8c-40a7-a841-2c9f17ea0f00 None None] MFA Rules not processed for user `c1e4c4fa12184befb9449cad088d7829`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:00.932185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cd6dc9d6-1b8c-40a7-a841-2c9f17ea0f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:00.983795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-15681012-6914-4194-8399-653549103dfb None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:00.989098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-15681012-6914-4194-8399-653549103dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:01.056602 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2904/5731] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:01.058053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590ca79-8fd4-4fdc-8e76-975c5ee732b9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:01.059008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590ca79-8fd4-4fdc-8e76-975c5ee732b9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:01.059289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590ca79-8fd4-4fdc-8e76-975c5ee732b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:01.084296 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2828/5732] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:10:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:10:01.087382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15681012-6914-4194-8399-653549103dfb None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:01.089901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-15681012-6914-4194-8399-653549103dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:01.109657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8590ca79-8fd4-4fdc-8e76-975c5ee732b9 None None] MFA Rules not processed for user `4f3c9d35f6b843ef905f43d758a9bb25`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:01.111334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8590ca79-8fd4-4fdc-8e76-975c5ee732b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:01.138542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:01.177948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15681012-6914-4194-8399-653549103dfb None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:01.185304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd77497-377a-4f32-8804-32dac72729bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:01.186458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd77497-377a-4f32-8804-32dac72729bb service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:01.186934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd77497-377a-4f32-8804-32dac72729bb service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:01.187375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd77497-377a-4f32-8804-32dac72729bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:01.222481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd77497-377a-4f32-8804-32dac72729bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:01.226692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd77497-377a-4f32-8804-32dac72729bb service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:01.231433 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2905/5733] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:01.233583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590ca79-8fd4-4fdc-8e76-975c5ee732b9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:01.239873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8590ca79-8fd4-4fdc-8e76-975c5ee732b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:01.254415 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2829/5734] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:01 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:10:01.431703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590ca79-8fd4-4fdc-8e76-975c5ee732b9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:01.441832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d617d7-646a-4213-bd28-6be72a88a0cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:01.443978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d617d7-646a-4213-bd28-6be72a88a0cb service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:01.444550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d617d7-646a-4213-bd28-6be72a88a0cb service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:01.445069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d617d7-646a-4213-bd28-6be72a88a0cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:01.460520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d617d7-646a-4213-bd28-6be72a88a0cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:01.464075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d617d7-646a-4213-bd28-6be72a88a0cb service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:01.490096 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2906/5735] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:10:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:10:01.783691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd77497-377a-4f32-8804-32dac72729bb service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:01.987981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:02.078646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd77497-377a-4f32-8804-32dac72729bb service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:02.102208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba881717-cfd3-4534-999f-3b1e741015f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:02.104022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba881717-cfd3-4534-999f-3b1e741015f1 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:02.105169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba881717-cfd3-4534-999f-3b1e741015f1 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:02.105452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba881717-cfd3-4534-999f-3b1e741015f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:02.144600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba881717-cfd3-4534-999f-3b1e741015f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:02.146394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba881717-cfd3-4534-999f-3b1e741015f1 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:02.178864 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2830/5736] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:10:02.390912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d617d7-646a-4213-bd28-6be72a88a0cb service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:02.394409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c2d617d7-646a-4213-bd28-6be72a88a0cb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:02.426010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d617d7-646a-4213-bd28-6be72a88a0cb service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:02.434590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32918f2-ebfe-4a17-91b0-30400d2356c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:02.435491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32918f2-ebfe-4a17-91b0-30400d2356c5 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:02.435604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32918f2-ebfe-4a17-91b0-30400d2356c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:02.435825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32918f2-ebfe-4a17-91b0-30400d2356c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:02.553729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32918f2-ebfe-4a17-91b0-30400d2356c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:02.556296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32918f2-ebfe-4a17-91b0-30400d2356c5 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:02.572889 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2907/5737] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:10:08.162023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba881717-cfd3-4534-999f-3b1e741015f1 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:08.177192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba881717-cfd3-4534-999f-3b1e741015f1 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:08.199728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9f45c-da08-43c7-8156-11511ca41880 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:08.200363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9f45c-da08-43c7-8156-11511ca41880 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:08.204155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9f45c-da08-43c7-8156-11511ca41880 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:08.204501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9f45c-da08-43c7-8156-11511ca41880 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:08.401000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b9f45c-da08-43c7-8156-11511ca41880 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:08.405321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b9f45c-da08-43c7-8156-11511ca41880 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:08.451283 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2831/5738] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:08 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:10:09.834842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2967d-1387-4bf8-93e2-d9d38f3a7812 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:09.835389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2967d-1387-4bf8-93e2-d9d38f3a7812 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:09.835389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2967d-1387-4bf8-93e2-d9d38f3a7812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:09.911678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e4c2967d-1387-4bf8-93e2-d9d38f3a7812 None None] MFA Rules not processed for user `4f3c9d35f6b843ef905f43d758a9bb25`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:09.912927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e4c2967d-1387-4bf8-93e2-d9d38f3a7812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:09.952287 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2908/5739] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:11.840955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9f45c-da08-43c7-8156-11511ca41880 service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:11.980957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9f45c-da08-43c7-8156-11511ca41880 service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:11.990583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09be2a-f109-47ba-a6bc-54cacd295858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:11.994811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09be2a-f109-47ba-a6bc-54cacd295858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:11.995352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09be2a-f109-47ba-a6bc-54cacd295858 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:11.995843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09be2a-f109-47ba-a6bc-54cacd295858 admin admin] PATH_INFO: `/v3/users/50dae5a392834816b8f8bf29c1756438` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:11.997782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09be2a-f109-47ba-a6bc-54cacd295858 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50dae5a392834816b8f8bf29c1756438)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:12.003994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09be2a-f109-47ba-a6bc-54cacd295858 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:12.159164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8e09be2a-f109-47ba-a6bc-54cacd295858 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50dae5a392834816b8f8bf29c1756438'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:12.166101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8e09be2a-f109-47ba-a6bc-54cacd295858 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50dae5a392834816b8f8bf29c1756438'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:12.167754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8e09be2a-f109-47ba-a6bc-54cacd295858 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50dae5a392834816b8f8bf29c1756438'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:12.192393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8e09be2a-f109-47ba-a6bc-54cacd295858 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50dae5a392834816b8f8bf29c1756438'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:12.220462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8e09be2a-f109-47ba-a6bc-54cacd295858 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50dae5a392834816b8f8bf29c1756438'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:12.259385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:12.285226 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2832/5740] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:10:11 2022] DELETE /identity/v3/users/50dae5a392834816b8f8bf29c1756438 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:12.292711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff2094-2dd8-4215-8ba6-242ee707e5e6 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:12.293701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff2094-2dd8-4215-8ba6-242ee707e5e6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:12.294164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff2094-2dd8-4215-8ba6-242ee707e5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:12.323180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e3ff2094-2dd8-4215-8ba6-242ee707e5e6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:12.324916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e3ff2094-2dd8-4215-8ba6-242ee707e5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:12.390625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:12.471231 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2909/5741] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:12.556544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09be2a-f109-47ba-a6bc-54cacd295858 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:12.560237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8e09be2a-f109-47ba-a6bc-54cacd295858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:12.739781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09be2a-f109-47ba-a6bc-54cacd295858 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:12.743793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e99b7-d8bc-413d-8a70-64c7e40f0aa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:12.744679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e99b7-d8bc-413d-8a70-64c7e40f0aa2 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:12.745211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e99b7-d8bc-413d-8a70-64c7e40f0aa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:12.745583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e99b7-d8bc-413d-8a70-64c7e40f0aa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:12.758127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e99b7-d8bc-413d-8a70-64c7e40f0aa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:12.759768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e99b7-d8bc-413d-8a70-64c7e40f0aa2 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:12.771807 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2833/5742] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:10:13.269778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff2094-2dd8-4215-8ba6-242ee707e5e6 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:13.271413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e3ff2094-2dd8-4215-8ba6-242ee707e5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:13.310169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff2094-2dd8-4215-8ba6-242ee707e5e6 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:13.315965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a8d125-bc1a-43dd-9b81-0441accfaf3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:13.316849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8d125-bc1a-43dd-9b81-0441accfaf3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:13.317293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8d125-bc1a-43dd-9b81-0441accfaf3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:13.317914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8d125-bc1a-43dd-9b81-0441accfaf3f admin admin] PATH_INFO: `/v3/projects/438396a805cb4950ba2f788942425ee3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:13.320323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a8d125-bc1a-43dd-9b81-0441accfaf3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=438396a805cb4950ba2f788942425ee3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:13.322277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a8d125-bc1a-43dd-9b81-0441accfaf3f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:13.400033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e7a8d125-bc1a-43dd-9b81-0441accfaf3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '438396a805cb4950ba2f788942425ee3', 'request_id': 'req-e7a8d125-bc1a-43dd-9b81-0441accfaf3f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:13.410005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e7a8d125-bc1a-43dd-9b81-0441accfaf3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '438396a805cb4950ba2f788942425ee3', 'request_id': 'req-e7a8d125-bc1a-43dd-9b81-0441accfaf3f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:13.419019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e7a8d125-bc1a-43dd-9b81-0441accfaf3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '438396a805cb4950ba2f788942425ee3', 'request_id': 'req-e7a8d125-bc1a-43dd-9b81-0441accfaf3f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:13.445819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e7a8d125-bc1a-43dd-9b81-0441accfaf3f admin admin] The token cache is being invalidate because project 438396a805cb4950ba2f788942425ee3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:10:13.446191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e7a8d125-bc1a-43dd-9b81-0441accfaf3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:13.447002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e7a8d125-bc1a-43dd-9b81-0441accfaf3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:13.448847 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2910/5743] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:10:13 2022] DELETE /identity/v3/projects/438396a805cb4950ba2f788942425ee3 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:13.452924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e99b7-d8bc-413d-8a70-64c7e40f0aa2 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:13.466809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e99b7-d8bc-413d-8a70-64c7e40f0aa2 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:13.612418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdafae-b702-4508-a87c-059301a2d0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:13.613338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdafae-b702-4508-a87c-059301a2d0de admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:13.613338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdafae-b702-4508-a87c-059301a2d0de admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:13.615182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdafae-b702-4508-a87c-059301a2d0de admin admin] PATH_INFO: `/v3/users/1f1b0b24bebd426db6c9b92f3e318036` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:13.618004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdafae-b702-4508-a87c-059301a2d0de admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f1b0b24bebd426db6c9b92f3e318036)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:13.623065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdafae-b702-4508-a87c-059301a2d0de admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:13.738064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b3fdafae-b702-4508-a87c-059301a2d0de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f1b0b24bebd426db6c9b92f3e318036'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:13.738808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b3fdafae-b702-4508-a87c-059301a2d0de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f1b0b24bebd426db6c9b92f3e318036'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:13.739453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b3fdafae-b702-4508-a87c-059301a2d0de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f1b0b24bebd426db6c9b92f3e318036'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:13.755796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b3fdafae-b702-4508-a87c-059301a2d0de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f1b0b24bebd426db6c9b92f3e318036'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:13.782521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b3fdafae-b702-4508-a87c-059301a2d0de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f1b0b24bebd426db6c9b92f3e318036'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:13.816865 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2834/5744] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:10:13 2022] DELETE /identity/v3/users/1f1b0b24bebd426db6c9b92f3e318036 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:14.423906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a8d125-bc1a-43dd-9b81-0441accfaf3f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:14.425969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e7a8d125-bc1a-43dd-9b81-0441accfaf3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:14.448863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a8d125-bc1a-43dd-9b81-0441accfaf3f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:14.556300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdedfe4-4d91-4971-924f-9bf7735c22d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:14.557390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdedfe4-4d91-4971-924f-9bf7735c22d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:14.558466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdedfe4-4d91-4971-924f-9bf7735c22d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:14.558951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdedfe4-4d91-4971-924f-9bf7735c22d1 admin admin] PATH_INFO: `/v3/projects/dc5012199dee456bb4064da27232e49d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:14.560914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdedfe4-4d91-4971-924f-9bf7735c22d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc5012199dee456bb4064da27232e49d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:14.563318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdedfe4-4d91-4971-924f-9bf7735c22d1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:14.680541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5bdedfe4-4d91-4971-924f-9bf7735c22d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc5012199dee456bb4064da27232e49d', 'request_id': 'req-5bdedfe4-4d91-4971-924f-9bf7735c22d1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:14.692195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5bdedfe4-4d91-4971-924f-9bf7735c22d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc5012199dee456bb4064da27232e49d', 'request_id': 'req-5bdedfe4-4d91-4971-924f-9bf7735c22d1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:14.701893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:14.712054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5bdedfe4-4d91-4971-924f-9bf7735c22d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc5012199dee456bb4064da27232e49d', 'request_id': 'req-5bdedfe4-4d91-4971-924f-9bf7735c22d1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:14.744402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5bdedfe4-4d91-4971-924f-9bf7735c22d1 admin admin] The token cache is being invalidate because project dc5012199dee456bb4064da27232e49d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:10:14.744843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5bdedfe4-4d91-4971-924f-9bf7735c22d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:14.745984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5bdedfe4-4d91-4971-924f-9bf7735c22d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:14.749102 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2911/5745] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:10:14 2022] DELETE /identity/v3/projects/dc5012199dee456bb4064da27232e49d => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:14.753333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdafae-b702-4508-a87c-059301a2d0de admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:14.757457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b3fdafae-b702-4508-a87c-059301a2d0de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:14.782536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdafae-b702-4508-a87c-059301a2d0de admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:14.818051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:14.964019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f3b18-d3a1-45eb-b4a3-932138c43a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:14.965031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f3b18-d3a1-45eb-b4a3-932138c43a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:14.965478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f3b18-d3a1-45eb-b4a3-932138c43a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:14.965888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f3b18-d3a1-45eb-b4a3-932138c43a2a admin admin] PATH_INFO: `/v3/users/c1e4c4fa12184befb9449cad088d7829` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:14.968086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f3b18-d3a1-45eb-b4a3-932138c43a2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1e4c4fa12184befb9449cad088d7829)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:14.970439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f3b18-d3a1-45eb-b4a3-932138c43a2a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:15.074093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ff3f3b18-d3a1-45eb-b4a3-932138c43a2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1e4c4fa12184befb9449cad088d7829'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:15.075094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ff3f3b18-d3a1-45eb-b4a3-932138c43a2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1e4c4fa12184befb9449cad088d7829'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:15.076187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ff3f3b18-d3a1-45eb-b4a3-932138c43a2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1e4c4fa12184befb9449cad088d7829'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:15.099982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ff3f3b18-d3a1-45eb-b4a3-932138c43a2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1e4c4fa12184befb9449cad088d7829'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:15.127992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ff3f3b18-d3a1-45eb-b4a3-932138c43a2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1e4c4fa12184befb9449cad088d7829'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:15.167248 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2835/5746] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:10:14 2022] DELETE /identity/v3/users/c1e4c4fa12184befb9449cad088d7829 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:10:15.665887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdedfe4-4d91-4971-924f-9bf7735c22d1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:15.669426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5bdedfe4-4d91-4971-924f-9bf7735c22d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:15.687191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdedfe4-4d91-4971-924f-9bf7735c22d1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:15.814476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79488c-45ce-4393-87e5-351a879aea0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:15.816417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79488c-45ce-4393-87e5-351a879aea0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:15.816706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79488c-45ce-4393-87e5-351a879aea0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:15.816983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79488c-45ce-4393-87e5-351a879aea0c admin admin] PATH_INFO: `/v3/projects/71af754245d74edc8aa25bba18235192` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:15.818212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79488c-45ce-4393-87e5-351a879aea0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=71af754245d74edc8aa25bba18235192)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:15.821102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79488c-45ce-4393-87e5-351a879aea0c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:15.923525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fa79488c-45ce-4393-87e5-351a879aea0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71af754245d74edc8aa25bba18235192', 'request_id': 'req-fa79488c-45ce-4393-87e5-351a879aea0c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:15.934432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fa79488c-45ce-4393-87e5-351a879aea0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71af754245d74edc8aa25bba18235192', 'request_id': 'req-fa79488c-45ce-4393-87e5-351a879aea0c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:15.947575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fa79488c-45ce-4393-87e5-351a879aea0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71af754245d74edc8aa25bba18235192', 'request_id': 'req-fa79488c-45ce-4393-87e5-351a879aea0c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:15.974472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fa79488c-45ce-4393-87e5-351a879aea0c admin admin] The token cache is being invalidate because project 71af754245d74edc8aa25bba18235192 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:10:15.974949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fa79488c-45ce-4393-87e5-351a879aea0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:15.975774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fa79488c-45ce-4393-87e5-351a879aea0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:15.981691 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2912/5747] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:10:15 2022] DELETE /identity/v3/projects/71af754245d74edc8aa25bba18235192 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:15.984074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41b7c0-2866-4eca-80c6-c62cf8f4c6fa None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:15.984557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41b7c0-2866-4eca-80c6-c62cf8f4c6fa None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:15.984754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41b7c0-2866-4eca-80c6-c62cf8f4c6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:16.015775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8e41b7c0-2866-4eca-80c6-c62cf8f4c6fa None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:16.017746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8e41b7c0-2866-4eca-80c6-c62cf8f4c6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:16.131561 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2836/5748] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:10:16.141814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44f911-6924-4e56-8a6c-7eb9ad74bd38 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:16.142478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44f911-6924-4e56-8a6c-7eb9ad74bd38 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:16.143349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44f911-6924-4e56-8a6c-7eb9ad74bd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:16.180350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3f44f911-6924-4e56-8a6c-7eb9ad74bd38 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:16.182404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3f44f911-6924-4e56-8a6c-7eb9ad74bd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:16.208494 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2913/5749] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:16.212192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41b7c0-2866-4eca-80c6-c62cf8f4c6fa None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:16.226114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41b7c0-2866-4eca-80c6-c62cf8f4c6fa None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:16.241783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a9154-8ac9-4ef0-b959-166e595c2b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:16.242406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a9154-8ac9-4ef0-b959-166e595c2b0b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:16.243896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a9154-8ac9-4ef0-b959-166e595c2b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:16.243896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a9154-8ac9-4ef0-b959-166e595c2b0b admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:16.245152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a9154-8ac9-4ef0-b959-166e595c2b0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:10:16.245529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a9154-8ac9-4ef0-b959-166e595c2b0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:16.249093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a9154-8ac9-4ef0-b959-166e595c2b0b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:16.279388 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2837/5750] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:10:16.285091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44f911-6924-4e56-8a6c-7eb9ad74bd38 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:16.297718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44f911-6924-4e56-8a6c-7eb9ad74bd38 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:16.307902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ae2e5-ad59-456e-b712-d422bbaabe1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:16.308548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ae2e5-ad59-456e-b712-d422bbaabe1d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:16.309211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ae2e5-ad59-456e-b712-d422bbaabe1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:16.309211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ae2e5-ad59-456e-b712-d422bbaabe1d admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:16.309867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ae2e5-ad59-456e-b712-d422bbaabe1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:16.312576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ae2e5-ad59-456e-b712-d422bbaabe1d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:16.370621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a9154-8ac9-4ef0-b959-166e595c2b0b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:16.373003 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2914/5751] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:10:16 2022] POST /identity/v3/projects => generated 381 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:10:16.379102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a9154-8ac9-4ef0-b959-166e595c2b0b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:16.383864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ae2e5-ad59-456e-b712-d422bbaabe1d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:16.386834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8b9ae2e5-ad59-456e-b712-d422bbaabe1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:16.432485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:16.489728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:16.517695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88168466-a6a9-4a82-8dc3-3d5c589004dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:16.519319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88168466-a6a9-4a82-8dc3-3d5c589004dd admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:16.519764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88168466-a6a9-4a82-8dc3-3d5c589004dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:16.520198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88168466-a6a9-4a82-8dc3-3d5c589004dd admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:16.523945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88168466-a6a9-4a82-8dc3-3d5c589004dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:16.526559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88168466-a6a9-4a82-8dc3-3d5c589004dd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:16.560360 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2838/5752] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:10:16 2022] POST /identity/v3/users => generated 291 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:10:16.562614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ae2e5-ad59-456e-b712-d422bbaabe1d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:16.564335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88168466-a6a9-4a82-8dc3-3d5c589004dd admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:16.586318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88168466-a6a9-4a82-8dc3-3d5c589004dd admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:16.594108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d804f7c-fabc-4a2b-82d7-4f9a336bd390 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:16.595440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d804f7c-fabc-4a2b-82d7-4f9a336bd390 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:16.595718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d804f7c-fabc-4a2b-82d7-4f9a336bd390 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:16.596103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d804f7c-fabc-4a2b-82d7-4f9a336bd390 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:16.597946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5d804f7c-fabc-4a2b-82d7-4f9a336bd390 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:16.608415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d23de-26c4-43a8-bb5b-71213b1419e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:16.610177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d23de-26c4-43a8-bb5b-71213b1419e4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:16.611986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d23de-26c4-43a8-bb5b-71213b1419e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:16.612313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d23de-26c4-43a8-bb5b-71213b1419e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:16.613021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d23de-26c4-43a8-bb5b-71213b1419e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:16.614777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d23de-26c4-43a8-bb5b-71213b1419e4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:16.658716 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2839/5753] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:16.661372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d23de-26c4-43a8-bb5b-71213b1419e4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:16.693191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d23de-26c4-43a8-bb5b-71213b1419e4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:16.706219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07708cc9-b8e5-4a79-bf96-805a8936b72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:16.706967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07708cc9-b8e5-4a79-bf96-805a8936b72e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:16.707315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07708cc9-b8e5-4a79-bf96-805a8936b72e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:16.707698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07708cc9-b8e5-4a79-bf96-805a8936b72e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:16.708466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07708cc9-b8e5-4a79-bf96-805a8936b72e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:16.711085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07708cc9-b8e5-4a79-bf96-805a8936b72e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:16.712849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d804f7c-fabc-4a2b-82d7-4f9a336bd390 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:16.714959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d804f7c-fabc-4a2b-82d7-4f9a336bd390 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:16.727376 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2915/5754] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:10:16.730831 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2840/5755] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:16.732777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d804f7c-fabc-4a2b-82d7-4f9a336bd390 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:16.759283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d804f7c-fabc-4a2b-82d7-4f9a336bd390 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:16.765190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce34ce-9d35-4abd-a17b-4f39e88ef32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:16.766928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce34ce-9d35-4abd-a17b-4f39e88ef32b admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:16.767328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce34ce-9d35-4abd-a17b-4f39e88ef32b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:16.767867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce34ce-9d35-4abd-a17b-4f39e88ef32b admin admin] PATH_INFO: `/v3/projects/1807b655b65a40458aa0b9a890371235/users/42218017edd74838a017ebd14610cf5e/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:16.792924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce34ce-9d35-4abd-a17b-4f39e88ef32b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1807b655b65a40458aa0b9a890371235, user_id=42218017edd74838a017ebd14610cf5e, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:16.795967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce34ce-9d35-4abd-a17b-4f39e88ef32b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:16.833671 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2916/5756] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:10:16 2022] PUT /identity/v3/projects/1807b655b65a40458aa0b9a890371235/users/42218017edd74838a017ebd14610cf5e/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:16.837131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7527e7-2ae1-4908-b06e-22ed73c7be4a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:16.837419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7527e7-2ae1-4908-b06e-22ed73c7be4a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:16.837880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7527e7-2ae1-4908-b06e-22ed73c7be4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:16.922815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2d7527e7-2ae1-4908-b06e-22ed73c7be4a None None] MFA Rules not processed for user `42218017edd74838a017ebd14610cf5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:16.924963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2d7527e7-2ae1-4908-b06e-22ed73c7be4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:17.067070 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2841/5757] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:17.072186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce34ce-9d35-4abd-a17b-4f39e88ef32b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:17.192868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce34ce-9d35-4abd-a17b-4f39e88ef32b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:17.197962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06465157-13b4-4f7d-bf64-0103d468b09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:17.198844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06465157-13b4-4f7d-bf64-0103d468b09c admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:17.199232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06465157-13b4-4f7d-bf64-0103d468b09c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:17.200005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06465157-13b4-4f7d-bf64-0103d468b09c admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:17.200738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06465157-13b4-4f7d-bf64-0103d468b09c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:17.202967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06465157-13b4-4f7d-bf64-0103d468b09c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:17.253299 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2917/5758] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:10:17 2022] POST /identity/v3/projects => generated 381 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:10:17.253635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7527e7-2ae1-4908-b06e-22ed73c7be4a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:17.268592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7527e7-2ae1-4908-b06e-22ed73c7be4a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:17.371577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df78b9f-e386-4ab5-b399-d8ecb829aaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:17.372503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df78b9f-e386-4ab5-b399-d8ecb829aaa0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:17.372972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df78b9f-e386-4ab5-b399-d8ecb829aaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:17.373449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df78b9f-e386-4ab5-b399-d8ecb829aaa0 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:17.374834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df78b9f-e386-4ab5-b399-d8ecb829aaa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:17.377090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df78b9f-e386-4ab5-b399-d8ecb829aaa0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:17.414760 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2842/5759] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:10:17 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:10:17.416502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06465157-13b4-4f7d-bf64-0103d468b09c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:17.436598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06465157-13b4-4f7d-bf64-0103d468b09c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:17.444208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914263d-a0b4-4ee5-b085-5b3e55ffef4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:17.445237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914263d-a0b4-4ee5-b085-5b3e55ffef4c admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:17.449417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914263d-a0b4-4ee5-b085-5b3e55ffef4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:17.450015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914263d-a0b4-4ee5-b085-5b3e55ffef4c admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:17.450952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9914263d-a0b4-4ee5-b085-5b3e55ffef4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:17.452772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9914263d-a0b4-4ee5-b085-5b3e55ffef4c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:17.470269 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2918/5760] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:17.471865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df78b9f-e386-4ab5-b399-d8ecb829aaa0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:17.486154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df78b9f-e386-4ab5-b399-d8ecb829aaa0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:17.493542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fb629-88e8-448b-ab01-54b2d78b4935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:17.494777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fb629-88e8-448b-ab01-54b2d78b4935 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:17.495075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fb629-88e8-448b-ab01-54b2d78b4935 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:17.495515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fb629-88e8-448b-ab01-54b2d78b4935 admin admin] PATH_INFO: `/v3/projects/8abe6c1018ce439091ed8482acb17ab7/users/62a0c50295654e668f0a93218f07361c/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:17.532752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:17.540983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87fb629-88e8-448b-ab01-54b2d78b4935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8abe6c1018ce439091ed8482acb17ab7, user_id=62a0c50295654e668f0a93218f07361c, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:17.550273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87fb629-88e8-448b-ab01-54b2d78b4935 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:17.582153 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2843/5761] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:10:17 2022] PUT /identity/v3/projects/8abe6c1018ce439091ed8482acb17ab7/users/62a0c50295654e668f0a93218f07361c/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:10:17.583513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914263d-a0b4-4ee5-b085-5b3e55ffef4c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:17.595617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914263d-a0b4-4ee5-b085-5b3e55ffef4c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:17.639427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:17.679595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2d747-6d1c-4c7d-8df4-994259b4ce81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:17.680506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2d747-6d1c-4c7d-8df4-994259b4ce81 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:17.681156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2d747-6d1c-4c7d-8df4-994259b4ce81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:17.681670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2d747-6d1c-4c7d-8df4-994259b4ce81 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:17.682387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2d747-6d1c-4c7d-8df4-994259b4ce81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:17.693349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2d747-6d1c-4c7d-8df4-994259b4ce81 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:17.719641 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2919/5762] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:10:17.722761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fb629-88e8-448b-ab01-54b2d78b4935 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:17.754964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fb629-88e8-448b-ab01-54b2d78b4935 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:17.763514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ee316-b8e2-455c-b592-39c6df705645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:17.763894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ee316-b8e2-455c-b592-39c6df705645 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:17.764178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ee316-b8e2-455c-b592-39c6df705645 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:17.764449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ee316-b8e2-455c-b592-39c6df705645 admin admin] PATH_INFO: `/v3/projects/8abe6c1018ce439091ed8482acb17ab7/users/62a0c50295654e668f0a93218f07361c/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:17.769333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ee316-b8e2-455c-b592-39c6df705645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8abe6c1018ce439091ed8482acb17ab7, user_id=62a0c50295654e668f0a93218f07361c, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:17.772128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ee316-b8e2-455c-b592-39c6df705645 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:17.791829 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2844/5763] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:10:17 2022] PUT /identity/v3/projects/8abe6c1018ce439091ed8482acb17ab7/users/62a0c50295654e668f0a93218f07361c/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:17.807372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c3ac5-aa41-454e-b1c5-04a0cddb800b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:17.807585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c3ac5-aa41-454e-b1c5-04a0cddb800b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:17.807735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c3ac5-aa41-454e-b1c5-04a0cddb800b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:17.854574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-875c3ac5-aa41-454e-b1c5-04a0cddb800b None None] MFA Rules not processed for user `62a0c50295654e668f0a93218f07361c`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:17.855849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-875c3ac5-aa41-454e-b1c5-04a0cddb800b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:17.947107 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2920/5764] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:17 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:10:17.959348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ee316-b8e2-455c-b592-39c6df705645 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:17.962196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-af6ee316-b8e2-455c-b592-39c6df705645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:18.082231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ee316-b8e2-455c-b592-39c6df705645 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:18.086599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ff022-d9f5-45fc-8498-9d0f61488d18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:18.087304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ff022-d9f5-45fc-8498-9d0f61488d18 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:18.087652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ff022-d9f5-45fc-8498-9d0f61488d18 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:18.087989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ff022-d9f5-45fc-8498-9d0f61488d18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:18.103432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ff022-d9f5-45fc-8498-9d0f61488d18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:18.105316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ff022-d9f5-45fc-8498-9d0f61488d18 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:18.116837 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2845/5765] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:17 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:10:18.662877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c3ac5-aa41-454e-b1c5-04a0cddb800b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:18.772104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c3ac5-aa41-454e-b1c5-04a0cddb800b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:18.782328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7009d0d-4224-4497-a261-e259aaaba9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:18.783282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7009d0d-4224-4497-a261-e259aaaba9f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:18.785220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7009d0d-4224-4497-a261-e259aaaba9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:18.785717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7009d0d-4224-4497-a261-e259aaaba9f7 admin admin] PATH_INFO: `/v3/users/42218017edd74838a017ebd14610cf5e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:18.788137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7009d0d-4224-4497-a261-e259aaaba9f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42218017edd74838a017ebd14610cf5e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:18.793328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7009d0d-4224-4497-a261-e259aaaba9f7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:18.860669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b7009d0d-4224-4497-a261-e259aaaba9f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42218017edd74838a017ebd14610cf5e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:18.873494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b7009d0d-4224-4497-a261-e259aaaba9f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42218017edd74838a017ebd14610cf5e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:18.879106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:18.897208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b7009d0d-4224-4497-a261-e259aaaba9f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42218017edd74838a017ebd14610cf5e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:18.898213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b7009d0d-4224-4497-a261-e259aaaba9f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42218017edd74838a017ebd14610cf5e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:18.910384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b7009d0d-4224-4497-a261-e259aaaba9f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42218017edd74838a017ebd14610cf5e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:18.918750 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2921/5766] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:10:18 2022] DELETE /identity/v3/users/42218017edd74838a017ebd14610cf5e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:18.923820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb351336-b49a-4e31-8166-d2dff765cf2a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:18.924147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb351336-b49a-4e31-8166-d2dff765cf2a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:18.924426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb351336-b49a-4e31-8166-d2dff765cf2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:18.949951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fb351336-b49a-4e31-8166-d2dff765cf2a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:18.951261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fb351336-b49a-4e31-8166-d2dff765cf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:19.037982 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2846/5767] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:19.050725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7009d0d-4224-4497-a261-e259aaaba9f7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:19.053113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b7009d0d-4224-4497-a261-e259aaaba9f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:19.170161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7009d0d-4224-4497-a261-e259aaaba9f7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:19.184001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613adc3-6bb0-4cdc-b158-67c5d59f11c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:19.184675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613adc3-6bb0-4cdc-b158-67c5d59f11c5 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:19.184969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613adc3-6bb0-4cdc-b158-67c5d59f11c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:19.185242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613adc3-6bb0-4cdc-b158-67c5d59f11c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:19.204592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613adc3-6bb0-4cdc-b158-67c5d59f11c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:19.206719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613adc3-6bb0-4cdc-b158-67c5d59f11c5 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:19.218890 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2922/5768] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:10:20.406347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb351336-b49a-4e31-8166-d2dff765cf2a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:20.412146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fb351336-b49a-4e31-8166-d2dff765cf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:20.461344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb351336-b49a-4e31-8166-d2dff765cf2a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:20.483503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a011a-7a95-486a-a0bb-3baf7759448b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:20.491042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a011a-7a95-486a-a0bb-3baf7759448b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:20.492377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a011a-7a95-486a-a0bb-3baf7759448b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:20.493369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a011a-7a95-486a-a0bb-3baf7759448b admin admin] PATH_INFO: `/v3/projects/1807b655b65a40458aa0b9a890371235` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:20.500047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a011a-7a95-486a-a0bb-3baf7759448b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1807b655b65a40458aa0b9a890371235)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:20.510257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a011a-7a95-486a-a0bb-3baf7759448b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:20.729157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7c8a011a-7a95-486a-a0bb-3baf7759448b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1807b655b65a40458aa0b9a890371235', 'request_id': 'req-7c8a011a-7a95-486a-a0bb-3baf7759448b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:20.753723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7c8a011a-7a95-486a-a0bb-3baf7759448b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1807b655b65a40458aa0b9a890371235', 'request_id': 'req-7c8a011a-7a95-486a-a0bb-3baf7759448b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:20.769716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7c8a011a-7a95-486a-a0bb-3baf7759448b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1807b655b65a40458aa0b9a890371235', 'request_id': 'req-7c8a011a-7a95-486a-a0bb-3baf7759448b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:20.807732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7c8a011a-7a95-486a-a0bb-3baf7759448b admin admin] The token cache is being invalidate because project 1807b655b65a40458aa0b9a890371235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:10:20.808228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7c8a011a-7a95-486a-a0bb-3baf7759448b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:20.809685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7c8a011a-7a95-486a-a0bb-3baf7759448b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:20.814239 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2847/5769] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:10:20 2022] DELETE /identity/v3/projects/1807b655b65a40458aa0b9a890371235 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:20.818718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613adc3-6bb0-4cdc-b158-67c5d59f11c5 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:20.833190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613adc3-6bb0-4cdc-b158-67c5d59f11c5 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:20.991028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b0142-fd49-4ec1-ade2-79fbb821cddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:20.991706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b0142-fd49-4ec1-ade2-79fbb821cddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:20.995702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b0142-fd49-4ec1-ade2-79fbb821cddf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:20.996069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b0142-fd49-4ec1-ade2-79fbb821cddf admin admin] PATH_INFO: `/v3/users/62a0c50295654e668f0a93218f07361c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:21.001415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b0142-fd49-4ec1-ade2-79fbb821cddf admin admin] RBAC: Authorizing `identity:delete_user(user_id=62a0c50295654e668f0a93218f07361c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:21.007572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b0142-fd49-4ec1-ade2-79fbb821cddf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:21.178713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0d0b0142-fd49-4ec1-ade2-79fbb821cddf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62a0c50295654e668f0a93218f07361c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:21.214630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0d0b0142-fd49-4ec1-ade2-79fbb821cddf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62a0c50295654e668f0a93218f07361c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:21.274950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0d0b0142-fd49-4ec1-ade2-79fbb821cddf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62a0c50295654e668f0a93218f07361c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:21.275720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0d0b0142-fd49-4ec1-ade2-79fbb821cddf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62a0c50295654e668f0a93218f07361c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:21.292338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:21.300883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0d0b0142-fd49-4ec1-ade2-79fbb821cddf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62a0c50295654e668f0a93218f07361c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:21.324064 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2923/5770] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:10:20 2022] DELETE /identity/v3/users/62a0c50295654e668f0a93218f07361c => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:22.588900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a011a-7a95-486a-a0bb-3baf7759448b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:22.590390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7c8a011a-7a95-486a-a0bb-3baf7759448b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:22.600096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:22.633543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a011a-7a95-486a-a0bb-3baf7759448b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:22.800104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640a946-f1ff-4f71-a61d-201a09a9ecc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:22.801077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640a946-f1ff-4f71-a61d-201a09a9ecc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:22.801077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640a946-f1ff-4f71-a61d-201a09a9ecc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:22.801704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640a946-f1ff-4f71-a61d-201a09a9ecc0 admin admin] PATH_INFO: `/v3/projects/8abe6c1018ce439091ed8482acb17ab7` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:22.803512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640a946-f1ff-4f71-a61d-201a09a9ecc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8abe6c1018ce439091ed8482acb17ab7)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:22.805561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640a946-f1ff-4f71-a61d-201a09a9ecc0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:22.918229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c640a946-f1ff-4f71-a61d-201a09a9ecc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8abe6c1018ce439091ed8482acb17ab7', 'request_id': 'req-c640a946-f1ff-4f71-a61d-201a09a9ecc0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:22.932845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c640a946-f1ff-4f71-a61d-201a09a9ecc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8abe6c1018ce439091ed8482acb17ab7', 'request_id': 'req-c640a946-f1ff-4f71-a61d-201a09a9ecc0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:22.948566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c640a946-f1ff-4f71-a61d-201a09a9ecc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8abe6c1018ce439091ed8482acb17ab7', 'request_id': 'req-c640a946-f1ff-4f71-a61d-201a09a9ecc0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:22.965424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c640a946-f1ff-4f71-a61d-201a09a9ecc0 admin admin] The token cache is being invalidate because project 8abe6c1018ce439091ed8482acb17ab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:10:22.965798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c640a946-f1ff-4f71-a61d-201a09a9ecc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:22.966572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c640a946-f1ff-4f71-a61d-201a09a9ecc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:22.968425 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2848/5771] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:10:22 2022] DELETE /identity/v3/projects/8abe6c1018ce439091ed8482acb17ab7 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:22.973892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abc270-0863-4e91-ad37-c1b137b13104 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:22.974367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abc270-0863-4e91-ad37-c1b137b13104 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:22.974618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abc270-0863-4e91-ad37-c1b137b13104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:23.047796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-17abc270-0863-4e91-ad37-c1b137b13104 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:23.049759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-17abc270-0863-4e91-ad37-c1b137b13104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:23.178064 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2924/5772] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:23.184887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5c1ee-f8dc-45b8-bf79-aa78dc92a5b3 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:23.185208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5c1ee-f8dc-45b8-bf79-aa78dc92a5b3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:23.185479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5c1ee-f8dc-45b8-bf79-aa78dc92a5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:23.221432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-88a5c1ee-f8dc-45b8-bf79-aa78dc92a5b3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:23.226709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-88a5c1ee-f8dc-45b8-bf79-aa78dc92a5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:23.245747 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2849/5773] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:23.250243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17abc270-0863-4e91-ad37-c1b137b13104 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:23.262578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17abc270-0863-4e91-ad37-c1b137b13104 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:23.271945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854194b8-d183-4ae6-8b88-9d318be72298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:23.272660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854194b8-d183-4ae6-8b88-9d318be72298 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:23.273046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854194b8-d183-4ae6-8b88-9d318be72298 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:23.273398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854194b8-d183-4ae6-8b88-9d318be72298 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:23.274193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854194b8-d183-4ae6-8b88-9d318be72298 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:10:23.279523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854194b8-d183-4ae6-8b88-9d318be72298 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:23.282294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854194b8-d183-4ae6-8b88-9d318be72298 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:23.334455 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2925/5774] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:10:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:10:23.338476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5c1ee-f8dc-45b8-bf79-aa78dc92a5b3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:23.365779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5c1ee-f8dc-45b8-bf79-aa78dc92a5b3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:23.386016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7781c-1851-4709-adbb-d723be73f8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:23.389824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7781c-1851-4709-adbb-d723be73f8f8 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:23.391955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7781c-1851-4709-adbb-d723be73f8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:23.393103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7781c-1851-4709-adbb-d723be73f8f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:23.395408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7781c-1851-4709-adbb-d723be73f8f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:23.405993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7781c-1851-4709-adbb-d723be73f8f8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:23.497876 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2850/5775] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:10:23 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:10:23.501688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854194b8-d183-4ae6-8b88-9d318be72298 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:23.525603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854194b8-d183-4ae6-8b88-9d318be72298 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:23.749111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ba15a-6429-4541-bcf9-2debdaa99fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:23.749744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ba15a-6429-4541-bcf9-2debdaa99fcd admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:23.750020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ba15a-6429-4541-bcf9-2debdaa99fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:23.750293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ba15a-6429-4541-bcf9-2debdaa99fcd admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:23.751105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ba15a-6429-4541-bcf9-2debdaa99fcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:23.753545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ba15a-6429-4541-bcf9-2debdaa99fcd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:23.795986 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2926/5776] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:10:23 2022] POST /identity/v3/users => generated 289 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:10:23.800099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7781c-1851-4709-adbb-d723be73f8f8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:23.833774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7781c-1851-4709-adbb-d723be73f8f8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:23.845291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09b8c0-4194-4bf3-9225-07fa877ae96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:23.846685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09b8c0-4194-4bf3-9225-07fa877ae96d admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:23.848157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09b8c0-4194-4bf3-9225-07fa877ae96d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:23.849097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09b8c0-4194-4bf3-9225-07fa877ae96d admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:23.849857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09b8c0-4194-4bf3-9225-07fa877ae96d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:23.852897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09b8c0-4194-4bf3-9225-07fa877ae96d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:23.869278 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2851/5777] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:23.873207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ba15a-6429-4541-bcf9-2debdaa99fcd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:23.916134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ba15a-6429-4541-bcf9-2debdaa99fcd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:23.947025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac968d5-3dc7-4ac5-b5d3-13e5031d9a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:23.948813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac968d5-3dc7-4ac5-b5d3-13e5031d9a44 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:23.952082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac968d5-3dc7-4ac5-b5d3-13e5031d9a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:23.952524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac968d5-3dc7-4ac5-b5d3-13e5031d9a44 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:23.953218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac968d5-3dc7-4ac5-b5d3-13e5031d9a44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:23.957037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac968d5-3dc7-4ac5-b5d3-13e5031d9a44 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:23.977803 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2927/5778] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:23.987516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09b8c0-4194-4bf3-9225-07fa877ae96d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:24.006191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09b8c0-4194-4bf3-9225-07fa877ae96d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:24.012562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf74b4-9127-4b3f-97f7-668650c15985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:24.015009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf74b4-9127-4b3f-97f7-668650c15985 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:24.015411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf74b4-9127-4b3f-97f7-668650c15985 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:24.015767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf74b4-9127-4b3f-97f7-668650c15985 admin admin] PATH_INFO: `/v3/projects/bad8134dd6f842239d31f35a9673a79a/users/39038d5fa6a64e3da5f4e1a135d87c2c/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:24.041935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf74b4-9127-4b3f-97f7-668650c15985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad8134dd6f842239d31f35a9673a79a, user_id=39038d5fa6a64e3da5f4e1a135d87c2c, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:24.045297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf74b4-9127-4b3f-97f7-668650c15985 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:24.069040 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2852/5779] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:10:23 2022] PUT /identity/v3/projects/bad8134dd6f842239d31f35a9673a79a/users/39038d5fa6a64e3da5f4e1a135d87c2c/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:24.080091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416faec-7289-4ed8-ab3c-fc9d86e0e7a2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:24.080895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416faec-7289-4ed8-ab3c-fc9d86e0e7a2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:24.081175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416faec-7289-4ed8-ab3c-fc9d86e0e7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:24.121285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c416faec-7289-4ed8-ab3c-fc9d86e0e7a2 None None] MFA Rules not processed for user `39038d5fa6a64e3da5f4e1a135d87c2c`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:24.122664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c416faec-7289-4ed8-ab3c-fc9d86e0e7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:24.213500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:24.249079 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2928/5780] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:24.253251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf74b4-9127-4b3f-97f7-668650c15985 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:24.301835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:24.351147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf74b4-9127-4b3f-97f7-668650c15985 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:24.357763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37683485-662b-45d3-9b64-c59d2bf3d858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:24.358939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37683485-662b-45d3-9b64-c59d2bf3d858 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:24.359418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37683485-662b-45d3-9b64-c59d2bf3d858 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:24.359910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37683485-662b-45d3-9b64-c59d2bf3d858 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:24.360706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37683485-662b-45d3-9b64-c59d2bf3d858 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:24.363677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37683485-662b-45d3-9b64-c59d2bf3d858 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:24.413933 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2853/5781] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:10:24 2022] POST /identity/v3/projects => generated 375 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:10:24.417485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416faec-7289-4ed8-ab3c-fc9d86e0e7a2 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:24.432654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416faec-7289-4ed8-ab3c-fc9d86e0e7a2 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:24.519490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b7899-44be-4f96-8543-c076258ad030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:24.520472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b7899-44be-4f96-8543-c076258ad030 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:24.520940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b7899-44be-4f96-8543-c076258ad030 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:24.521481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b7899-44be-4f96-8543-c076258ad030 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:24.522441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6b7899-44be-4f96-8543-c076258ad030 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:24.524870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6b7899-44be-4f96-8543-c076258ad030 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:24.568385 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2929/5782] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:10:24 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:10:24.571891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37683485-662b-45d3-9b64-c59d2bf3d858 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:24.594966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37683485-662b-45d3-9b64-c59d2bf3d858 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:24.604672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ab82c-5f3f-45e2-ad34-30b7adea4ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:24.605320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ab82c-5f3f-45e2-ad34-30b7adea4ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:24.605606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ab82c-5f3f-45e2-ad34-30b7adea4ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:24.605887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ab82c-5f3f-45e2-ad34-30b7adea4ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:24.606579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ab82c-5f3f-45e2-ad34-30b7adea4ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:24.608227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ab82c-5f3f-45e2-ad34-30b7adea4ff3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:24.629771 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2854/5783] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:24.638259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b7899-44be-4f96-8543-c076258ad030 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:24.650065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b7899-44be-4f96-8543-c076258ad030 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:24.656325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb99e2-2033-468f-b536-897e02c06c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:24.657169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb99e2-2033-468f-b536-897e02c06c84 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:24.657615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb99e2-2033-468f-b536-897e02c06c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:24.658946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb99e2-2033-468f-b536-897e02c06c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:24.659836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb99e2-2033-468f-b536-897e02c06c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:24.661631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb99e2-2033-468f-b536-897e02c06c84 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:24.679204 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2930/5784] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:24.684965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ab82c-5f3f-45e2-ad34-30b7adea4ff3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:24.699398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ab82c-5f3f-45e2-ad34-30b7adea4ff3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:24.709214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf289a-b045-49f6-a096-11a78e2f73a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:24.710223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf289a-b045-49f6-a096-11a78e2f73a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:24.710852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf289a-b045-49f6-a096-11a78e2f73a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:24.711394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf289a-b045-49f6-a096-11a78e2f73a0 admin admin] PATH_INFO: `/v3/projects/54a8f8c604a044749a8b9fb32e5e64f4/users/cfe1d16426e545b498bccf750d780e84/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:24.749351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf289a-b045-49f6-a096-11a78e2f73a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54a8f8c604a044749a8b9fb32e5e64f4, user_id=cfe1d16426e545b498bccf750d780e84, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:24.752825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf289a-b045-49f6-a096-11a78e2f73a0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:24.777889 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2855/5785] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:10:24 2022] PUT /identity/v3/projects/54a8f8c604a044749a8b9fb32e5e64f4/users/cfe1d16426e545b498bccf750d780e84/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:24.887515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55812e1a-1740-486f-9c37-991ace1dd572 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:24.887880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55812e1a-1740-486f-9c37-991ace1dd572 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:24.888080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55812e1a-1740-486f-9c37-991ace1dd572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:24.928104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-55812e1a-1740-486f-9c37-991ace1dd572 None None] MFA Rules not processed for user `cfe1d16426e545b498bccf750d780e84`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:24.929412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-55812e1a-1740-486f-9c37-991ace1dd572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:25.054818 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2931/5786] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:25.055317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf289a-b045-49f6-a096-11a78e2f73a0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:25.254233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf289a-b045-49f6-a096-11a78e2f73a0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:25.261351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d34c60-55d0-49b5-92a6-6df67f44cd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:25.261982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d34c60-55d0-49b5-92a6-6df67f44cd73 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:25.262452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d34c60-55d0-49b5-92a6-6df67f44cd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:25.262783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d34c60-55d0-49b5-92a6-6df67f44cd73 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:25.264598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d34c60-55d0-49b5-92a6-6df67f44cd73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:25.266160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d34c60-55d0-49b5-92a6-6df67f44cd73 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:25.309721 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2856/5787] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:10:25 2022] POST /identity/v3/projects => generated 377 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:10:25.314305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55812e1a-1740-486f-9c37-991ace1dd572 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:25.349175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55812e1a-1740-486f-9c37-991ace1dd572 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:25.491349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac9428-9e40-4310-a2da-87f9f1bfe594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:25.492177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac9428-9e40-4310-a2da-87f9f1bfe594 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:25.492620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac9428-9e40-4310-a2da-87f9f1bfe594 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:25.493552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac9428-9e40-4310-a2da-87f9f1bfe594 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:25.500064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac9428-9e40-4310-a2da-87f9f1bfe594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:25.503810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac9428-9e40-4310-a2da-87f9f1bfe594 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:25.521515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:25.549483 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2932/5788] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:10:25 2022] POST /identity/v3/users => generated 295 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:10:25.553509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d34c60-55d0-49b5-92a6-6df67f44cd73 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:25.578726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d34c60-55d0-49b5-92a6-6df67f44cd73 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:25.586742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc1d40-1e60-421a-8ae1-e6d277e4a1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:25.587914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc1d40-1e60-421a-8ae1-e6d277e4a1c0 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:25.588604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc1d40-1e60-421a-8ae1-e6d277e4a1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:25.589036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc1d40-1e60-421a-8ae1-e6d277e4a1c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:25.590413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc1d40-1e60-421a-8ae1-e6d277e4a1c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:25.592463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc1d40-1e60-421a-8ae1-e6d277e4a1c0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:25.612791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac9428-9e40-4310-a2da-87f9f1bfe594 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:25.613916 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2857/5789] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:25.632458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac9428-9e40-4310-a2da-87f9f1bfe594 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:25.653590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a1ccb-e866-42b0-8965-83f72c7ee3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:25.655668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a1ccb-e866-42b0-8965-83f72c7ee3b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:25.657067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a1ccb-e866-42b0-8965-83f72c7ee3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:25.657584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a1ccb-e866-42b0-8965-83f72c7ee3b3 admin admin] PATH_INFO: `/v3/projects/00a44fe6804247f78efa9a838bd277e8/users/da5600d04c114fec95d635df1df295e6/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:25.701690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a1ccb-e866-42b0-8965-83f72c7ee3b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00a44fe6804247f78efa9a838bd277e8, user_id=da5600d04c114fec95d635df1df295e6, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:25.703774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a1ccb-e866-42b0-8965-83f72c7ee3b3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:25.723392 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2933/5790] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:10:25 2022] PUT /identity/v3/projects/00a44fe6804247f78efa9a838bd277e8/users/da5600d04c114fec95d635df1df295e6/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:25.727303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc1d40-1e60-421a-8ae1-e6d277e4a1c0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:25.742283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc1d40-1e60-421a-8ae1-e6d277e4a1c0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:25.835619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:25.855977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e4ee2-703c-4100-88aa-b73c88cc3c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:25.856622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e4ee2-703c-4100-88aa-b73c88cc3c59 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:25.856910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e4ee2-703c-4100-88aa-b73c88cc3c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:25.857190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e4ee2-703c-4100-88aa-b73c88cc3c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:25.858518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e4ee2-703c-4100-88aa-b73c88cc3c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:25.860425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e4ee2-703c-4100-88aa-b73c88cc3c59 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:25.888589 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2858/5791] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:25.894464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a1ccb-e866-42b0-8965-83f72c7ee3b3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:25.910623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a1ccb-e866-42b0-8965-83f72c7ee3b3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:25.930195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d959f-7ee4-4740-bb60-082e1bd33acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:25.931355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d959f-7ee4-4740-bb60-082e1bd33acb admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:25.931837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d959f-7ee4-4740-bb60-082e1bd33acb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:25.932455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d959f-7ee4-4740-bb60-082e1bd33acb admin admin] PATH_INFO: `/v3/projects/00a44fe6804247f78efa9a838bd277e8/users/da5600d04c114fec95d635df1df295e6/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:25.936149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d959f-7ee4-4740-bb60-082e1bd33acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=00a44fe6804247f78efa9a838bd277e8, user_id=da5600d04c114fec95d635df1df295e6, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:25.944108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d959f-7ee4-4740-bb60-082e1bd33acb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:25.973133 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2934/5792] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:10:25 2022] PUT /identity/v3/projects/00a44fe6804247f78efa9a838bd277e8/users/da5600d04c114fec95d635df1df295e6/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:25.975620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb26b7b-1ac3-4119-a905-6051701fd935 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:25.975957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb26b7b-1ac3-4119-a905-6051701fd935 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:25.976242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb26b7b-1ac3-4119-a905-6051701fd935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:26.021863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bbb26b7b-1ac3-4119-a905-6051701fd935 None None] MFA Rules not processed for user `da5600d04c114fec95d635df1df295e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:26.024710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bbb26b7b-1ac3-4119-a905-6051701fd935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:26.158852 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2859/5793] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:26.166379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d959f-7ee4-4740-bb60-082e1bd33acb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:26.168434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ee3d959f-7ee4-4740-bb60-082e1bd33acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:26.322282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d959f-7ee4-4740-bb60-082e1bd33acb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:26.332858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e920e94-061b-4e92-868a-f6e77d8f5119 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:26.333503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e920e94-061b-4e92-868a-f6e77d8f5119 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:26.333789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e920e94-061b-4e92-868a-f6e77d8f5119 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:26.339379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e920e94-061b-4e92-868a-f6e77d8f5119 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:26.506468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e920e94-061b-4e92-868a-f6e77d8f5119 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:26.507989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e920e94-061b-4e92-868a-f6e77d8f5119 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:26.534392 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2935/5794] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:10:28.373666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb26b7b-1ac3-4119-a905-6051701fd935 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:28.375283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bbb26b7b-1ac3-4119-a905-6051701fd935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:28.407454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb26b7b-1ac3-4119-a905-6051701fd935 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:28.411697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6d9de-fe28-4c7b-a2a9-556dfbd56978 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:28.413084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6d9de-fe28-4c7b-a2a9-556dfbd56978 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:28.413541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6d9de-fe28-4c7b-a2a9-556dfbd56978 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:28.413959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6d9de-fe28-4c7b-a2a9-556dfbd56978 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:28.696956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6d9de-fe28-4c7b-a2a9-556dfbd56978 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:28.698867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6d9de-fe28-4c7b-a2a9-556dfbd56978 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:28.713066 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2860/5795] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:10:29.350192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e920e94-061b-4e92-868a-f6e77d8f5119 service cinder] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:29.351664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7e920e94-061b-4e92-868a-f6e77d8f5119 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:29.464688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e920e94-061b-4e92-868a-f6e77d8f5119 service cinder] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:29.469458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ec6c0-7f9f-483d-98a7-8cf093dd3561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:29.471249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ec6c0-7f9f-483d-98a7-8cf093dd3561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:29.471543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ec6c0-7f9f-483d-98a7-8cf093dd3561 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:29.472354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ec6c0-7f9f-483d-98a7-8cf093dd3561 admin admin] PATH_INFO: `/v3/users/52c19be363304ed8b4fdd2896691effa` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:29.479791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ec6c0-7f9f-483d-98a7-8cf093dd3561 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52c19be363304ed8b4fdd2896691effa)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:29.481756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ec6c0-7f9f-483d-98a7-8cf093dd3561 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:29.492440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:29.521546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6d9de-fe28-4c7b-a2a9-556dfbd56978 service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:29.535366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6d9de-fe28-4c7b-a2a9-556dfbd56978 service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:29.539303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a54152-9383-4393-82ec-adc1144fdc78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:29.540124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a54152-9383-4393-82ec-adc1144fdc78 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:29.540402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a54152-9383-4393-82ec-adc1144fdc78 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:29.540671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a54152-9383-4393-82ec-adc1144fdc78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:29.546972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cf1ec6c0-7f9f-483d-98a7-8cf093dd3561 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52c19be363304ed8b4fdd2896691effa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:29.565140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a54152-9383-4393-82ec-adc1144fdc78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:29.566697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a54152-9383-4393-82ec-adc1144fdc78 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:29.569957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cf1ec6c0-7f9f-483d-98a7-8cf093dd3561 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52c19be363304ed8b4fdd2896691effa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:29.577605 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2861/5796] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:10:29.588860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cf1ec6c0-7f9f-483d-98a7-8cf093dd3561 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52c19be363304ed8b4fdd2896691effa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:29.589853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cf1ec6c0-7f9f-483d-98a7-8cf093dd3561 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52c19be363304ed8b4fdd2896691effa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:29.600730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cf1ec6c0-7f9f-483d-98a7-8cf093dd3561 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52c19be363304ed8b4fdd2896691effa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:29.608786 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2936/5797] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:10:29 2022] DELETE /identity/v3/users/52c19be363304ed8b4fdd2896691effa => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:29.613426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ec42a-22d4-4411-a12d-17372c0b6cc9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:29.613714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ec42a-22d4-4411-a12d-17372c0b6cc9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:29.613985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ec42a-22d4-4411-a12d-17372c0b6cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:29.635866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-949ec42a-22d4-4411-a12d-17372c0b6cc9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:29.637406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-949ec42a-22d4-4411-a12d-17372c0b6cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:29.722556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:29.736771 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2862/5798] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:29.750057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ec6c0-7f9f-483d-98a7-8cf093dd3561 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:29.752531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cf1ec6c0-7f9f-483d-98a7-8cf093dd3561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:29.856471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ec6c0-7f9f-483d-98a7-8cf093dd3561 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:29.861812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4524c965-900a-4da8-88f9-2efccb8716ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:29.862712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524c965-900a-4da8-88f9-2efccb8716ce service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:29.863222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524c965-900a-4da8-88f9-2efccb8716ce service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:29.863932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524c965-900a-4da8-88f9-2efccb8716ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:29.875230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4524c965-900a-4da8-88f9-2efccb8716ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:29.876979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4524c965-900a-4da8-88f9-2efccb8716ce service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:29.887511 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2937/5799] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:10:30.116343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ec42a-22d4-4411-a12d-17372c0b6cc9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:30.118101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-949ec42a-22d4-4411-a12d-17372c0b6cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:30.133874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ec42a-22d4-4411-a12d-17372c0b6cc9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:30.140553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b420e96-530f-4cae-921b-7762f32f7523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:30.141589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b420e96-530f-4cae-921b-7762f32f7523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:30.143911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b420e96-530f-4cae-921b-7762f32f7523 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:30.144556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b420e96-530f-4cae-921b-7762f32f7523 admin admin] PATH_INFO: `/v3/projects/8a008a2473e7463fb4cf73326a84aae2` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:30.183032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b420e96-530f-4cae-921b-7762f32f7523 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a008a2473e7463fb4cf73326a84aae2)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:30.185239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b420e96-530f-4cae-921b-7762f32f7523 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:30.265214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b420e96-530f-4cae-921b-7762f32f7523 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a008a2473e7463fb4cf73326a84aae2', 'request_id': 'req-5b420e96-530f-4cae-921b-7762f32f7523'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:30.276397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b420e96-530f-4cae-921b-7762f32f7523 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a008a2473e7463fb4cf73326a84aae2', 'request_id': 'req-5b420e96-530f-4cae-921b-7762f32f7523'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:30.286657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b420e96-530f-4cae-921b-7762f32f7523 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a008a2473e7463fb4cf73326a84aae2', 'request_id': 'req-5b420e96-530f-4cae-921b-7762f32f7523'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:30.303510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b420e96-530f-4cae-921b-7762f32f7523 admin admin] The token cache is being invalidate because project 8a008a2473e7463fb4cf73326a84aae2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:10:30.307138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b420e96-530f-4cae-921b-7762f32f7523 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:30.307896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b420e96-530f-4cae-921b-7762f32f7523 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:30.313662 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2863/5800] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:10:30 2022] DELETE /identity/v3/projects/8a008a2473e7463fb4cf73326a84aae2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:30.316081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8300f-5065-4441-a068-f1b55b768d8a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:30.316479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8300f-5065-4441-a068-f1b55b768d8a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:30.316875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8300f-5065-4441-a068-f1b55b768d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:30.346956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b6b8300f-5065-4441-a068-f1b55b768d8a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:30.348753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b6b8300f-5065-4441-a068-f1b55b768d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:30.473762 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2938/5801] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:30.482008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ea68d-80fb-4432-b984-e68a9b982802 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:30.482008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ea68d-80fb-4432-b984-e68a9b982802 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:30.482008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ea68d-80fb-4432-b984-e68a9b982802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:30.503512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-053ea68d-80fb-4432-b984-e68a9b982802 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:30.504990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-053ea68d-80fb-4432-b984-e68a9b982802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:30.517578 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2864/5802] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:30.518575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b8300f-5065-4441-a068-f1b55b768d8a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:30.534211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b8300f-5065-4441-a068-f1b55b768d8a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:30.544456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751195f-7a29-4af0-bd85-4f76d75f742b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:30.545549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751195f-7a29-4af0-bd85-4f76d75f742b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:30.546017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751195f-7a29-4af0-bd85-4f76d75f742b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:30.546894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751195f-7a29-4af0-bd85-4f76d75f742b admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:30.552374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751195f-7a29-4af0-bd85-4f76d75f742b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:10:30.553070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751195f-7a29-4af0-bd85-4f76d75f742b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:30.556099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751195f-7a29-4af0-bd85-4f76d75f742b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:30.573955 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2939/5803] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:10:30.578278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ea68d-80fb-4432-b984-e68a9b982802 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:30.591166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ea68d-80fb-4432-b984-e68a9b982802 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:30.601572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b104d-3eea-41b2-9bb4-fd350f0051b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:30.602202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b104d-3eea-41b2-9bb4-fd350f0051b7 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:30.602483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b104d-3eea-41b2-9bb4-fd350f0051b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:30.602784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b104d-3eea-41b2-9bb4-fd350f0051b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:30.603432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b104d-3eea-41b2-9bb4-fd350f0051b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:30.607360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b104d-3eea-41b2-9bb4-fd350f0051b7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:30.651588 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2865/5804] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:10:30 2022] POST /identity/v3/projects => generated 367 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:10:30.655341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751195f-7a29-4af0-bd85-4f76d75f742b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:30.667549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751195f-7a29-4af0-bd85-4f76d75f742b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:30.683154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:30.743962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa4fba-454b-425e-bb32-39781214ad3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:30.744785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa4fba-454b-425e-bb32-39781214ad3e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:30.745141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa4fba-454b-425e-bb32-39781214ad3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:30.745520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa4fba-454b-425e-bb32-39781214ad3e admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:30.746260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa4fba-454b-425e-bb32-39781214ad3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:30.748380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa4fba-454b-425e-bb32-39781214ad3e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:30.788729 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2940/5805] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:10:30 2022] POST /identity/v3/users => generated 284 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:10:30.794222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b104d-3eea-41b2-9bb4-fd350f0051b7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:30.810294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b104d-3eea-41b2-9bb4-fd350f0051b7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:30.821838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e469a0-e2fd-4b01-aebc-2b6ab6fbcc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:30.822479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e469a0-e2fd-4b01-aebc-2b6ab6fbcc44 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:30.822786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e469a0-e2fd-4b01-aebc-2b6ab6fbcc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:30.823063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e469a0-e2fd-4b01-aebc-2b6ab6fbcc44 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:30.823821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e469a0-e2fd-4b01-aebc-2b6ab6fbcc44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:30.825334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e469a0-e2fd-4b01-aebc-2b6ab6fbcc44 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:30.858321 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2866/5806] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:30.862725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa4fba-454b-425e-bb32-39781214ad3e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:30.887959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa4fba-454b-425e-bb32-39781214ad3e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:30.905157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c4c7b-5af1-484b-b052-7bf1680120aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:30.907278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c4c7b-5af1-484b-b052-7bf1680120aa admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:30.908015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c4c7b-5af1-484b-b052-7bf1680120aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:30.908695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c4c7b-5af1-484b-b052-7bf1680120aa admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:30.909760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c4c7b-5af1-484b-b052-7bf1680120aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:30.911748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c4c7b-5af1-484b-b052-7bf1680120aa admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:30.944534 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2941/5807] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:30.945315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e469a0-e2fd-4b01-aebc-2b6ab6fbcc44 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:30.960431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e469a0-e2fd-4b01-aebc-2b6ab6fbcc44 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:30.973736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7ddbb-800f-4cc9-b648-9a55e2e03ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:30.976953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7ddbb-800f-4cc9-b648-9a55e2e03ae7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:30.977443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7ddbb-800f-4cc9-b648-9a55e2e03ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:30.979050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7ddbb-800f-4cc9-b648-9a55e2e03ae7 admin admin] PATH_INFO: `/v3/projects/ef6dd37a1540437e95131d8bab0afc2a/users/49eb5255ebb548cd83f14ec16d53fda3/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:31.051462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7ddbb-800f-4cc9-b648-9a55e2e03ae7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef6dd37a1540437e95131d8bab0afc2a, user_id=49eb5255ebb548cd83f14ec16d53fda3, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:31.060107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7ddbb-800f-4cc9-b648-9a55e2e03ae7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:31.096681 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2867/5808] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:10:30 2022] PUT /identity/v3/projects/ef6dd37a1540437e95131d8bab0afc2a/users/49eb5255ebb548cd83f14ec16d53fda3/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:10:31.106211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f80812-0392-486a-b053-48c4c4ca46dd None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:31.106678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f80812-0392-486a-b053-48c4c4ca46dd None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:31.107085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f80812-0392-486a-b053-48c4c4ca46dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:31.169180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-51f80812-0392-486a-b053-48c4c4ca46dd None None] MFA Rules not processed for user `49eb5255ebb548cd83f14ec16d53fda3`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:31.170515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-51f80812-0392-486a-b053-48c4c4ca46dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:31.323810 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2942/5809] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:31.339760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7ddbb-800f-4cc9-b648-9a55e2e03ae7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:31.341528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a7f7ddbb-800f-4cc9-b648-9a55e2e03ae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:31.444065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7ddbb-800f-4cc9-b648-9a55e2e03ae7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:31.448347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fa248-b2c8-40a0-9e0d-5b21a98fcb96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:31.449116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fa248-b2c8-40a0-9e0d-5b21a98fcb96 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:31.450020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fa248-b2c8-40a0-9e0d-5b21a98fcb96 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:31.450450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fa248-b2c8-40a0-9e0d-5b21a98fcb96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:31.468677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5fa248-b2c8-40a0-9e0d-5b21a98fcb96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:31.471629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5fa248-b2c8-40a0-9e0d-5b21a98fcb96 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:31.486181 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2868/5810] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:10:33.159826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f80812-0392-486a-b053-48c4c4ca46dd None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:33.181607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f80812-0392-486a-b053-48c4c4ca46dd None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:33.203805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f11ad8-9499-4e59-8e2d-f94901ef4150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:33.205298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f11ad8-9499-4e59-8e2d-f94901ef4150 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:33.206019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f11ad8-9499-4e59-8e2d-f94901ef4150 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:33.206649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f11ad8-9499-4e59-8e2d-f94901ef4150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:33.215142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b1f11ad8-9499-4e59-8e2d-f94901ef4150 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:33.354529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f11ad8-9499-4e59-8e2d-f94901ef4150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:33.355704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f11ad8-9499-4e59-8e2d-f94901ef4150 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:33.371791 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2943/5811] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:10:34.321488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fa248-b2c8-40a0-9e0d-5b21a98fcb96 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:34.325227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-eb5fa248-b2c8-40a0-9e0d-5b21a98fcb96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:34.372062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:34.482276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fa248-b2c8-40a0-9e0d-5b21a98fcb96 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:34.488563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f354e8-8d6e-4259-adb3-abb39d3727e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:34.489431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f354e8-8d6e-4259-adb3-abb39d3727e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:34.489950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f354e8-8d6e-4259-adb3-abb39d3727e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:34.490377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f354e8-8d6e-4259-adb3-abb39d3727e6 admin admin] PATH_INFO: `/v3/users/39038d5fa6a64e3da5f4e1a135d87c2c` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:34.491739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f354e8-8d6e-4259-adb3-abb39d3727e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39038d5fa6a64e3da5f4e1a135d87c2c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:34.493842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f354e8-8d6e-4259-adb3-abb39d3727e6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:34.576050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f7f354e8-8d6e-4259-adb3-abb39d3727e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39038d5fa6a64e3da5f4e1a135d87c2c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:34.576819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f7f354e8-8d6e-4259-adb3-abb39d3727e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39038d5fa6a64e3da5f4e1a135d87c2c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:34.577771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f7f354e8-8d6e-4259-adb3-abb39d3727e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39038d5fa6a64e3da5f4e1a135d87c2c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:34.594053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f7f354e8-8d6e-4259-adb3-abb39d3727e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39038d5fa6a64e3da5f4e1a135d87c2c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:34.621775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f7f354e8-8d6e-4259-adb3-abb39d3727e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39038d5fa6a64e3da5f4e1a135d87c2c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:34.643568 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2869/5812] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:10:34 2022] DELETE /identity/v3/users/39038d5fa6a64e3da5f4e1a135d87c2c => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:34.644927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bc0c2-32a2-496f-bd2c-4aac0a3f7c8c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:34.645398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bc0c2-32a2-496f-bd2c-4aac0a3f7c8c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:34.645481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bc0c2-32a2-496f-bd2c-4aac0a3f7c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:34.681491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3b4bc0c2-32a2-496f-bd2c-4aac0a3f7c8c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:34.683038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3b4bc0c2-32a2-496f-bd2c-4aac0a3f7c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:34.688454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:34.805457 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2944/5813] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:10:34.816378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f354e8-8d6e-4259-adb3-abb39d3727e6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:34.817971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f7f354e8-8d6e-4259-adb3-abb39d3727e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:34.937917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f354e8-8d6e-4259-adb3-abb39d3727e6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:34.943115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26975170-9485-4dd6-b8fd-9ec38e563cda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:34.943878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26975170-9485-4dd6-b8fd-9ec38e563cda service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:34.944252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26975170-9485-4dd6-b8fd-9ec38e563cda service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:34.945040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26975170-9485-4dd6-b8fd-9ec38e563cda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:34.956908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26975170-9485-4dd6-b8fd-9ec38e563cda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:34.958897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26975170-9485-4dd6-b8fd-9ec38e563cda service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:34.974452 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2870/5814] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:10:35.785649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4bc0c2-32a2-496f-bd2c-4aac0a3f7c8c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:35.787154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3b4bc0c2-32a2-496f-bd2c-4aac0a3f7c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:35.799521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4bc0c2-32a2-496f-bd2c-4aac0a3f7c8c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:35.805033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a3a37-ca0f-4ce8-ade8-8164ecfe921f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:35.805925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a3a37-ca0f-4ce8-ade8-8164ecfe921f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:35.806480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a3a37-ca0f-4ce8-ade8-8164ecfe921f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:35.806958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a3a37-ca0f-4ce8-ade8-8164ecfe921f admin admin] PATH_INFO: `/v3/projects/bad8134dd6f842239d31f35a9673a79a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:35.808609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a3a37-ca0f-4ce8-ade8-8164ecfe921f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bad8134dd6f842239d31f35a9673a79a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:35.810954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a3a37-ca0f-4ce8-ade8-8164ecfe921f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:35.895823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f32a3a37-ca0f-4ce8-ade8-8164ecfe921f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bad8134dd6f842239d31f35a9673a79a', 'request_id': 'req-f32a3a37-ca0f-4ce8-ade8-8164ecfe921f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:35.908723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f32a3a37-ca0f-4ce8-ade8-8164ecfe921f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bad8134dd6f842239d31f35a9673a79a', 'request_id': 'req-f32a3a37-ca0f-4ce8-ade8-8164ecfe921f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:35.920695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f32a3a37-ca0f-4ce8-ade8-8164ecfe921f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bad8134dd6f842239d31f35a9673a79a', 'request_id': 'req-f32a3a37-ca0f-4ce8-ade8-8164ecfe921f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:35.937584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f32a3a37-ca0f-4ce8-ade8-8164ecfe921f admin admin] The token cache is being invalidate because project bad8134dd6f842239d31f35a9673a79a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:10:35.937877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f32a3a37-ca0f-4ce8-ade8-8164ecfe921f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:35.938736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f32a3a37-ca0f-4ce8-ade8-8164ecfe921f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:35.941001 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2945/5815] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:10:35 2022] DELETE /identity/v3/projects/bad8134dd6f842239d31f35a9673a79a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:35.947519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26975170-9485-4dd6-b8fd-9ec38e563cda service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:35.962996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26975170-9485-4dd6-b8fd-9ec38e563cda service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:36.039878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:36.098954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197483b-f733-4991-9437-fdd2182adea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:36.099635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197483b-f733-4991-9437-fdd2182adea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:36.099913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197483b-f733-4991-9437-fdd2182adea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:36.100190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197483b-f733-4991-9437-fdd2182adea2 admin admin] PATH_INFO: `/v3/users/cfe1d16426e545b498bccf750d780e84` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:36.102000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197483b-f733-4991-9437-fdd2182adea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfe1d16426e545b498bccf750d780e84)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:36.104109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197483b-f733-4991-9437-fdd2182adea2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:36.187278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1197483b-f733-4991-9437-fdd2182adea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfe1d16426e545b498bccf750d780e84'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:36.188182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1197483b-f733-4991-9437-fdd2182adea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfe1d16426e545b498bccf750d780e84'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:36.189482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1197483b-f733-4991-9437-fdd2182adea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfe1d16426e545b498bccf750d780e84'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:36.199721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1197483b-f733-4991-9437-fdd2182adea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfe1d16426e545b498bccf750d780e84'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:36.221503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1197483b-f733-4991-9437-fdd2182adea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfe1d16426e545b498bccf750d780e84'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:36.238663 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2871/5816] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:10:35 2022] DELETE /identity/v3/users/cfe1d16426e545b498bccf750d780e84 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:36.875012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a3a37-ca0f-4ce8-ade8-8164ecfe921f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:36.876714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f32a3a37-ca0f-4ce8-ade8-8164ecfe921f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:36.894039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a3a37-ca0f-4ce8-ade8-8164ecfe921f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:36.990244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77c16b-800b-4b4a-9e5b-bfbb9e2eb141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:36.991276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77c16b-800b-4b4a-9e5b-bfbb9e2eb141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:36.991738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77c16b-800b-4b4a-9e5b-bfbb9e2eb141 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:36.992215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77c16b-800b-4b4a-9e5b-bfbb9e2eb141 admin admin] PATH_INFO: `/v3/projects/54a8f8c604a044749a8b9fb32e5e64f4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:36.993853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77c16b-800b-4b4a-9e5b-bfbb9e2eb141 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54a8f8c604a044749a8b9fb32e5e64f4)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:36.996400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77c16b-800b-4b4a-9e5b-bfbb9e2eb141 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:37.058582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:37.081190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8f77c16b-800b-4b4a-9e5b-bfbb9e2eb141 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54a8f8c604a044749a8b9fb32e5e64f4', 'request_id': 'req-8f77c16b-800b-4b4a-9e5b-bfbb9e2eb141'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:37.090984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8f77c16b-800b-4b4a-9e5b-bfbb9e2eb141 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54a8f8c604a044749a8b9fb32e5e64f4', 'request_id': 'req-8f77c16b-800b-4b4a-9e5b-bfbb9e2eb141'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:37.094258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0d83b-81b4-47be-b26e-7ba4e7517421 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:37.094829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0d83b-81b4-47be-b26e-7ba4e7517421 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:37.094927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0d83b-81b4-47be-b26e-7ba4e7517421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:37.101005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8f77c16b-800b-4b4a-9e5b-bfbb9e2eb141 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54a8f8c604a044749a8b9fb32e5e64f4', 'request_id': 'req-8f77c16b-800b-4b4a-9e5b-bfbb9e2eb141'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:37.117916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1ac0d83b-81b4-47be-b26e-7ba4e7517421 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:37.119838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1ac0d83b-81b4-47be-b26e-7ba4e7517421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:37.126443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8f77c16b-800b-4b4a-9e5b-bfbb9e2eb141 admin admin] The token cache is being invalidate because project 54a8f8c604a044749a8b9fb32e5e64f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:10:37.126778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8f77c16b-800b-4b4a-9e5b-bfbb9e2eb141 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:37.127828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8f77c16b-800b-4b4a-9e5b-bfbb9e2eb141 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:37.129714 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2946/5817] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:10:36 2022] DELETE /identity/v3/projects/54a8f8c604a044749a8b9fb32e5e64f4 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:37.134708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77c16b-800b-4b4a-9e5b-bfbb9e2eb141 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:37.135920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8f77c16b-800b-4b4a-9e5b-bfbb9e2eb141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:37.244565 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2872/5818] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:10:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:37.245727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0d83b-81b4-47be-b26e-7ba4e7517421 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:37.247493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1ac0d83b-81b4-47be-b26e-7ba4e7517421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:37.297451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77c16b-800b-4b4a-9e5b-bfbb9e2eb141 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:37.309751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcafe71-e941-477d-af09-74735ab56df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:37.310408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcafe71-e941-477d-af09-74735ab56df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:37.310716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcafe71-e941-477d-af09-74735ab56df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:37.310993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcafe71-e941-477d-af09-74735ab56df5 admin admin] PATH_INFO: `/v3/users/da5600d04c114fec95d635df1df295e6` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:37.313369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcafe71-e941-477d-af09-74735ab56df5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da5600d04c114fec95d635df1df295e6)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:37.316705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcafe71-e941-477d-af09-74735ab56df5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:37.339426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0d83b-81b4-47be-b26e-7ba4e7517421 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:37.352330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa035b69-8828-40b1-86f7-72275b2d3a02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:37.353613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa035b69-8828-40b1-86f7-72275b2d3a02 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:37.353613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa035b69-8828-40b1-86f7-72275b2d3a02 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:37.353613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa035b69-8828-40b1-86f7-72275b2d3a02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:37.378396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa035b69-8828-40b1-86f7-72275b2d3a02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:37.381453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa035b69-8828-40b1-86f7-72275b2d3a02 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:37.390659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:37.403758 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2873/5819] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:10:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:10:37.442448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4bcafe71-e941-477d-af09-74735ab56df5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da5600d04c114fec95d635df1df295e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:37.454272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4bcafe71-e941-477d-af09-74735ab56df5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da5600d04c114fec95d635df1df295e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:37.507936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4bcafe71-e941-477d-af09-74735ab56df5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da5600d04c114fec95d635df1df295e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:37.509600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4bcafe71-e941-477d-af09-74735ab56df5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da5600d04c114fec95d635df1df295e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:37.529536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4bcafe71-e941-477d-af09-74735ab56df5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da5600d04c114fec95d635df1df295e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:37.542598 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2947/5820] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:10:37 2022] DELETE /identity/v3/users/da5600d04c114fec95d635df1df295e6 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:38.756909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa035b69-8828-40b1-86f7-72275b2d3a02 service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:38.785400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa035b69-8828-40b1-86f7-72275b2d3a02 service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:39.090201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809802b9-1598-415b-941e-058bfeb93958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:39.093814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809802b9-1598-415b-941e-058bfeb93958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:39.096388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809802b9-1598-415b-941e-058bfeb93958 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:39.099366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809802b9-1598-415b-941e-058bfeb93958 admin admin] PATH_INFO: `/v3/projects/00a44fe6804247f78efa9a838bd277e8` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:39.108134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809802b9-1598-415b-941e-058bfeb93958 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00a44fe6804247f78efa9a838bd277e8)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:39.112845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809802b9-1598-415b-941e-058bfeb93958 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:39.233648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-809802b9-1598-415b-941e-058bfeb93958 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00a44fe6804247f78efa9a838bd277e8', 'request_id': 'req-809802b9-1598-415b-941e-058bfeb93958'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:39.261335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-809802b9-1598-415b-941e-058bfeb93958 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00a44fe6804247f78efa9a838bd277e8', 'request_id': 'req-809802b9-1598-415b-941e-058bfeb93958'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:39.290441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-809802b9-1598-415b-941e-058bfeb93958 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00a44fe6804247f78efa9a838bd277e8', 'request_id': 'req-809802b9-1598-415b-941e-058bfeb93958'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:39.316562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-809802b9-1598-415b-941e-058bfeb93958 admin admin] The token cache is being invalidate because project 00a44fe6804247f78efa9a838bd277e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:10:39.316562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-809802b9-1598-415b-941e-058bfeb93958 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:39.317301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-809802b9-1598-415b-941e-058bfeb93958 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:39.319159 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2874/5821] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:10:38 2022] DELETE /identity/v3/projects/00a44fe6804247f78efa9a838bd277e8 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:39.324673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b07f6-48a6-4bdb-b89b-36d31c0800f0 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:39.325205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b07f6-48a6-4bdb-b89b-36d31c0800f0 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:39.325205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b07f6-48a6-4bdb-b89b-36d31c0800f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:39.369555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f11b07f6-48a6-4bdb-b89b-36d31c0800f0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:39.376141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f11b07f6-48a6-4bdb-b89b-36d31c0800f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:39.481180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:39.584981 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2948/5822] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:39.594526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145d2d3-249f-408f-83ca-292f520ace2d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:39.595916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145d2d3-249f-408f-83ca-292f520ace2d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:39.596196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145d2d3-249f-408f-83ca-292f520ace2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:39.627810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7145d2d3-249f-408f-83ca-292f520ace2d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:39.629919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7145d2d3-249f-408f-83ca-292f520ace2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:39.648563 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2875/5823] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:39.649008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b07f6-48a6-4bdb-b89b-36d31c0800f0 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:39.667656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b07f6-48a6-4bdb-b89b-36d31c0800f0 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:39.677697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10982962-ab63-4542-9e2d-cd079626d377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:39.678990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10982962-ab63-4542-9e2d-cd079626d377 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:39.679520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10982962-ab63-4542-9e2d-cd079626d377 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:39.680077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10982962-ab63-4542-9e2d-cd079626d377 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:39.681180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10982962-ab63-4542-9e2d-cd079626d377 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:10:39.681831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10982962-ab63-4542-9e2d-cd079626d377 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:39.685023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10982962-ab63-4542-9e2d-cd079626d377 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:39.717967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145d2d3-249f-408f-83ca-292f520ace2d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:39.730757 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2949/5824] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:10:39.741683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145d2d3-249f-408f-83ca-292f520ace2d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:39.749183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85936d-3a67-40c6-b3e2-110e58a0b881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:39.750432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85936d-3a67-40c6-b3e2-110e58a0b881 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:39.751177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85936d-3a67-40c6-b3e2-110e58a0b881 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:39.752173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85936d-3a67-40c6-b3e2-110e58a0b881 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:39.753152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c85936d-3a67-40c6-b3e2-110e58a0b881 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:39.757513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c85936d-3a67-40c6-b3e2-110e58a0b881 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:39.814520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10982962-ab63-4542-9e2d-cd079626d377 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:39.827598 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2876/5825] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:10:39 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:10:39.836897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10982962-ab63-4542-9e2d-cd079626d377 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:40.004824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07240c-bd7b-4b1d-86da-336327c6874a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:40.005455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07240c-bd7b-4b1d-86da-336327c6874a admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:40.005741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07240c-bd7b-4b1d-86da-336327c6874a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:40.006016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07240c-bd7b-4b1d-86da-336327c6874a admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:40.006694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd07240c-bd7b-4b1d-86da-336327c6874a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:40.009698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd07240c-bd7b-4b1d-86da-336327c6874a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:40.061779 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2950/5826] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:10:39 2022] POST /identity/v3/users => generated 292 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:10:40.066300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85936d-3a67-40c6-b3e2-110e58a0b881 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:40.083148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85936d-3a67-40c6-b3e2-110e58a0b881 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:40.088154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d26cc2-74fc-4588-9b0e-633cfae1b5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:40.088913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d26cc2-74fc-4588-9b0e-633cfae1b5fc admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:40.089198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d26cc2-74fc-4588-9b0e-633cfae1b5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:40.089475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d26cc2-74fc-4588-9b0e-633cfae1b5fc admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:40.090285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d26cc2-74fc-4588-9b0e-633cfae1b5fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:40.092797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d26cc2-74fc-4588-9b0e-633cfae1b5fc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:40.118425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07240c-bd7b-4b1d-86da-336327c6874a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:40.131250 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2877/5827] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:40.137482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07240c-bd7b-4b1d-86da-336327c6874a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:40.144916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1ede7-cfd8-4f3e-b3fa-e88e9915fe8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:40.145766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1ede7-cfd8-4f3e-b3fa-e88e9915fe8f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:40.146278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1ede7-cfd8-4f3e-b3fa-e88e9915fe8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:40.146715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1ede7-cfd8-4f3e-b3fa-e88e9915fe8f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:40.147450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c1ede7-cfd8-4f3e-b3fa-e88e9915fe8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:40.149543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c1ede7-cfd8-4f3e-b3fa-e88e9915fe8f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:40.165730 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2951/5828] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:40.170573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d26cc2-74fc-4588-9b0e-633cfae1b5fc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:40.203359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d26cc2-74fc-4588-9b0e-633cfae1b5fc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:40.213636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7a8b9-b4f0-4422-a849-827e48b88921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:40.214791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7a8b9-b4f0-4422-a849-827e48b88921 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:40.215761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7a8b9-b4f0-4422-a849-827e48b88921 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:40.216219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7a8b9-b4f0-4422-a849-827e48b88921 admin admin] PATH_INFO: `/v3/projects/65b53a9e343e4ee08e0f9905cc895daa/users/73235ec3cce94f00b9c60cd7a3be5e85/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:40.306468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7a8b9-b4f0-4422-a849-827e48b88921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b53a9e343e4ee08e0f9905cc895daa, user_id=73235ec3cce94f00b9c60cd7a3be5e85, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:40.310965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7a8b9-b4f0-4422-a849-827e48b88921 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:40.342943 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2878/5829] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:10:40 2022] PUT /identity/v3/projects/65b53a9e343e4ee08e0f9905cc895daa/users/73235ec3cce94f00b9c60cd7a3be5e85/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:40.356167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11000e-b7ac-4bad-bfeb-7e156589e122 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:40.357034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11000e-b7ac-4bad-bfeb-7e156589e122 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:40.357535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11000e-b7ac-4bad-bfeb-7e156589e122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:40.399128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ce11000e-b7ac-4bad-bfeb-7e156589e122 None None] MFA Rules not processed for user `73235ec3cce94f00b9c60cd7a3be5e85`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:40.400853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ce11000e-b7ac-4bad-bfeb-7e156589e122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:40.561145 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2952/5830] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:40.571123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7a8b9-b4f0-4422-a849-827e48b88921 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:40.572931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a9c7a8b9-b4f0-4422-a849-827e48b88921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:40.634483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:40.691407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7a8b9-b4f0-4422-a849-827e48b88921 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:40.697991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84737b-5135-4e0b-9729-687fe7138175 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:40.698710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84737b-5135-4e0b-9729-687fe7138175 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:40.698932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84737b-5135-4e0b-9729-687fe7138175 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:40.699907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84737b-5135-4e0b-9729-687fe7138175 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:40.741653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84737b-5135-4e0b-9729-687fe7138175 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:40.743285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84737b-5135-4e0b-9729-687fe7138175 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:40.770380 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2879/5831] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:10:45.380614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11000e-b7ac-4bad-bfeb-7e156589e122 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:45.480848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:45.549027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11000e-b7ac-4bad-bfeb-7e156589e122 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:45.565994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7ea0e-798e-4964-a7d2-b3f558b524dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:45.567468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7ea0e-798e-4964-a7d2-b3f558b524dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:45.568001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7ea0e-798e-4964-a7d2-b3f558b524dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:45.568477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7ea0e-798e-4964-a7d2-b3f558b524dd admin admin] PATH_INFO: `/v3/users/73235ec3cce94f00b9c60cd7a3be5e85` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:45.570065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7ea0e-798e-4964-a7d2-b3f558b524dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=73235ec3cce94f00b9c60cd7a3be5e85)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:45.572209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7ea0e-798e-4964-a7d2-b3f558b524dd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:45.744840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4ee7ea0e-798e-4964-a7d2-b3f558b524dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73235ec3cce94f00b9c60cd7a3be5e85'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:45.766034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4ee7ea0e-798e-4964-a7d2-b3f558b524dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73235ec3cce94f00b9c60cd7a3be5e85'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:45.807129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4ee7ea0e-798e-4964-a7d2-b3f558b524dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73235ec3cce94f00b9c60cd7a3be5e85'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:45.808697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4ee7ea0e-798e-4964-a7d2-b3f558b524dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73235ec3cce94f00b9c60cd7a3be5e85'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:45.822124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4ee7ea0e-798e-4964-a7d2-b3f558b524dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73235ec3cce94f00b9c60cd7a3be5e85'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:45.865207 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2953/5832] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:10:45 2022] DELETE /identity/v3/users/73235ec3cce94f00b9c60cd7a3be5e85 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:45.869815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b45c18-4d91-44fe-9914-257a232b1b61 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:45.870591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b45c18-4d91-44fe-9914-257a232b1b61 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:45.870591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b45c18-4d91-44fe-9914-257a232b1b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:45.913285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c0b45c18-4d91-44fe-9914-257a232b1b61 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:45.914623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c0b45c18-4d91-44fe-9914-257a232b1b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:46.042256 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2880/5833] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:46.062893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7ea0e-798e-4964-a7d2-b3f558b524dd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:46.065089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4ee7ea0e-798e-4964-a7d2-b3f558b524dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:46.183889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7ea0e-798e-4964-a7d2-b3f558b524dd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:46.189819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93ebbf-d734-4a39-8b46-c1f3ce05bd9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:46.190541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93ebbf-d734-4a39-8b46-c1f3ce05bd9c service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:46.190825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93ebbf-d734-4a39-8b46-c1f3ce05bd9c service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:46.191107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93ebbf-d734-4a39-8b46-c1f3ce05bd9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:46.258302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93ebbf-d734-4a39-8b46-c1f3ce05bd9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:46.270177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93ebbf-d734-4a39-8b46-c1f3ce05bd9c service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:46.334655 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2954/5834] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:10:46.917013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b45c18-4d91-44fe-9914-257a232b1b61 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:46.919339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c0b45c18-4d91-44fe-9914-257a232b1b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:46.952002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b45c18-4d91-44fe-9914-257a232b1b61 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:46.959742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a9a96-43c9-4c18-b598-d2dc33f3b5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:46.960402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a9a96-43c9-4c18-b598-d2dc33f3b5a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:46.963675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a9a96-43c9-4c18-b598-d2dc33f3b5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:46.964005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a9a96-43c9-4c18-b598-d2dc33f3b5a2 admin admin] PATH_INFO: `/v3/projects/65b53a9e343e4ee08e0f9905cc895daa` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:46.966225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a9a96-43c9-4c18-b598-d2dc33f3b5a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65b53a9e343e4ee08e0f9905cc895daa)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:46.970775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a9a96-43c9-4c18-b598-d2dc33f3b5a2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:47.158321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-844a9a96-43c9-4c18-b598-d2dc33f3b5a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65b53a9e343e4ee08e0f9905cc895daa', 'request_id': 'req-844a9a96-43c9-4c18-b598-d2dc33f3b5a2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:47.180815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-844a9a96-43c9-4c18-b598-d2dc33f3b5a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65b53a9e343e4ee08e0f9905cc895daa', 'request_id': 'req-844a9a96-43c9-4c18-b598-d2dc33f3b5a2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:47.197867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-844a9a96-43c9-4c18-b598-d2dc33f3b5a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65b53a9e343e4ee08e0f9905cc895daa', 'request_id': 'req-844a9a96-43c9-4c18-b598-d2dc33f3b5a2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:47.233242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-844a9a96-43c9-4c18-b598-d2dc33f3b5a2 admin admin] The token cache is being invalidate because project 65b53a9e343e4ee08e0f9905cc895daa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:10:47.234786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-844a9a96-43c9-4c18-b598-d2dc33f3b5a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:47.237452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-844a9a96-43c9-4c18-b598-d2dc33f3b5a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:47.241021 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2881/5835] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:10:46 2022] DELETE /identity/v3/projects/65b53a9e343e4ee08e0f9905cc895daa => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:47.246906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89695c38-4c62-43c2-a07f-9647b1e158c5 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:47.247420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89695c38-4c62-43c2-a07f-9647b1e158c5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:47.247633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89695c38-4c62-43c2-a07f-9647b1e158c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:47.306299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-89695c38-4c62-43c2-a07f-9647b1e158c5 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:47.307823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-89695c38-4c62-43c2-a07f-9647b1e158c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:47.411178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:47.429453 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2955/5836] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:47.437107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce65f0-8b46-48ae-a625-d69322698ba9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:47.437688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce65f0-8b46-48ae-a625-d69322698ba9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:47.438198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce65f0-8b46-48ae-a625-d69322698ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:47.474034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-44ce65f0-8b46-48ae-a625-d69322698ba9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:47.475572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-44ce65f0-8b46-48ae-a625-d69322698ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:47.500736 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2882/5837] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:47.505082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89695c38-4c62-43c2-a07f-9647b1e158c5 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:47.520255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89695c38-4c62-43c2-a07f-9647b1e158c5 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:47.525647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecba59-5f9e-40ef-a782-f46c6744a7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:47.527599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecba59-5f9e-40ef-a782-f46c6744a7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:47.528326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecba59-5f9e-40ef-a782-f46c6744a7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:47.528754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecba59-5f9e-40ef-a782-f46c6744a7f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:47.529576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cecba59-5f9e-40ef-a782-f46c6744a7f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:10:47.530202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cecba59-5f9e-40ef-a782-f46c6744a7f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:47.532876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cecba59-5f9e-40ef-a782-f46c6744a7f7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:47.569681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce65f0-8b46-48ae-a625-d69322698ba9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:47.575477 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2956/5838] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:10:47.582051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce65f0-8b46-48ae-a625-d69322698ba9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:47.591886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c65114-c58c-4658-9c98-40b2c43e2597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:47.592796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c65114-c58c-4658-9c98-40b2c43e2597 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:47.594370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c65114-c58c-4658-9c98-40b2c43e2597 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:47.595396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c65114-c58c-4658-9c98-40b2c43e2597 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:47.597765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c65114-c58c-4658-9c98-40b2c43e2597 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:47.600742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c65114-c58c-4658-9c98-40b2c43e2597 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:47.665678 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2883/5839] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:10:47 2022] POST /identity/v3/projects => generated 389 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:10:47.668847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecba59-5f9e-40ef-a782-f46c6744a7f7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:47.680973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecba59-5f9e-40ef-a782-f46c6744a7f7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:47.799893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed10b10-df95-4fbd-b205-d8d124d1cb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:47.801512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed10b10-df95-4fbd-b205-d8d124d1cb0c admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:47.802077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed10b10-df95-4fbd-b205-d8d124d1cb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:47.802509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed10b10-df95-4fbd-b205-d8d124d1cb0c admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:47.803694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed10b10-df95-4fbd-b205-d8d124d1cb0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:47.807705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed10b10-df95-4fbd-b205-d8d124d1cb0c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:47.871757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c65114-c58c-4658-9c98-40b2c43e2597 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:47.879693 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2957/5840] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:10:47 2022] POST /identity/v3/users => generated 295 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:10:47.885393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c65114-c58c-4658-9c98-40b2c43e2597 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:47.905666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde324c6-2b38-433e-8261-5e7394e51b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:47.906625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde324c6-2b38-433e-8261-5e7394e51b7a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:47.907060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde324c6-2b38-433e-8261-5e7394e51b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:47.908015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde324c6-2b38-433e-8261-5e7394e51b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:47.908816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde324c6-2b38-433e-8261-5e7394e51b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:47.911096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde324c6-2b38-433e-8261-5e7394e51b7a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:47.920441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed10b10-df95-4fbd-b205-d8d124d1cb0c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:47.922593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0ed10b10-df95-4fbd-b205-d8d124d1cb0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:47.954156 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2884/5841] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:10:47.966483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde324c6-2b38-433e-8261-5e7394e51b7a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:47.991897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde324c6-2b38-433e-8261-5e7394e51b7a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:48.010094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd30793-c77e-4021-adb3-19db955e0309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:48.012657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd30793-c77e-4021-adb3-19db955e0309 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:48.016083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd30793-c77e-4021-adb3-19db955e0309 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:48.016934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd30793-c77e-4021-adb3-19db955e0309 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:48.019687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd30793-c77e-4021-adb3-19db955e0309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:48.021504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd30793-c77e-4021-adb3-19db955e0309 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:48.039683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:48.047194 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2885/5842] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:48.055080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd30793-c77e-4021-adb3-19db955e0309 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:48.072598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd30793-c77e-4021-adb3-19db955e0309 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:48.083330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8561499-5ac1-4fa5-93f2-1fa7b57697e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:48.084621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8561499-5ac1-4fa5-93f2-1fa7b57697e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:48.085222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8561499-5ac1-4fa5-93f2-1fa7b57697e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:48.086179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8561499-5ac1-4fa5-93f2-1fa7b57697e6 admin admin] PATH_INFO: `/v3/projects/6d1eb39d09ce4a87960e258ac890e32c/users/484ee8f965604fa9b7f7f5a25a81d0ef/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:48.103749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed10b10-df95-4fbd-b205-d8d124d1cb0c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:48.107735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970aa221-5ccf-423b-9547-d05f07601139 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:48.115091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970aa221-5ccf-423b-9547-d05f07601139 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:48.115379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970aa221-5ccf-423b-9547-d05f07601139 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:48.115667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970aa221-5ccf-423b-9547-d05f07601139 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:48.127835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-970aa221-5ccf-423b-9547-d05f07601139 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:48.129016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8561499-5ac1-4fa5-93f2-1fa7b57697e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d1eb39d09ce4a87960e258ac890e32c, user_id=484ee8f965604fa9b7f7f5a25a81d0ef, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:48.136463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8561499-5ac1-4fa5-93f2-1fa7b57697e6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:48.148949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970aa221-5ccf-423b-9547-d05f07601139 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:48.150448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970aa221-5ccf-423b-9547-d05f07601139 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:48.177321 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2958/5843] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:10:48.192417 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2886/5844] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:10:48 2022] PUT /identity/v3/projects/6d1eb39d09ce4a87960e258ac890e32c/users/484ee8f965604fa9b7f7f5a25a81d0ef/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:10:48.205338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fab4d4-507a-4c71-a498-3b07d4599c04 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:48.205722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fab4d4-507a-4c71-a498-3b07d4599c04 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:48.206119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fab4d4-507a-4c71-a498-3b07d4599c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:48.317887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-90fab4d4-507a-4c71-a498-3b07d4599c04 None None] MFA Rules not processed for user `484ee8f965604fa9b7f7f5a25a81d0ef`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:48.327985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-90fab4d4-507a-4c71-a498-3b07d4599c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:48.556774 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2959/5845] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:48 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:48.558891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8561499-5ac1-4fa5-93f2-1fa7b57697e6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:48.666923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fab4d4-507a-4c71-a498-3b07d4599c04 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:48.668912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-90fab4d4-507a-4c71-a498-3b07d4599c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:48.686436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:48.756536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8561499-5ac1-4fa5-93f2-1fa7b57697e6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:48.778920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dce222-90f3-4f05-ab69-037b0f059650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:48.780064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dce222-90f3-4f05-ab69-037b0f059650 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:48.781413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dce222-90f3-4f05-ab69-037b0f059650 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:48.783099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dce222-90f3-4f05-ab69-037b0f059650 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:48.784000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dce222-90f3-4f05-ab69-037b0f059650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:48.791129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dce222-90f3-4f05-ab69-037b0f059650 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:48.898069 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2887/5846] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:10:48 2022] POST /identity/v3/projects => generated 389 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:10:48.903705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dce222-90f3-4f05-ab69-037b0f059650 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:48.929984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fab4d4-507a-4c71-a498-3b07d4599c04 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:49.090589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dce222-90f3-4f05-ab69-037b0f059650 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:49.103878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f199e1-eccb-4534-84e1-6496ba4b9b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:49.104513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f199e1-eccb-4534-84e1-6496ba4b9b95 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:49.104916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f199e1-eccb-4534-84e1-6496ba4b9b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:49.105274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f199e1-eccb-4534-84e1-6496ba4b9b95 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:49.106002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f199e1-eccb-4534-84e1-6496ba4b9b95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:49.110128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f199e1-eccb-4534-84e1-6496ba4b9b95 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:49.124583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07a806-b8c2-4148-a5d4-e824f9db3ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:49.125288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07a806-b8c2-4148-a5d4-e824f9db3ff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:49.127107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07a806-b8c2-4148-a5d4-e824f9db3ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:49.127388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07a806-b8c2-4148-a5d4-e824f9db3ff4 admin admin] PATH_INFO: `/v3/users/49eb5255ebb548cd83f14ec16d53fda3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:49.128926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07a806-b8c2-4148-a5d4-e824f9db3ff4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49eb5255ebb548cd83f14ec16d53fda3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:49.131575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07a806-b8c2-4148-a5d4-e824f9db3ff4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:49.166253 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2888/5847] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:10:48 2022] POST /identity/v3/users => generated 295 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:10:49.168143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f199e1-eccb-4534-84e1-6496ba4b9b95 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:49.193067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f199e1-eccb-4534-84e1-6496ba4b9b95 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:49.209403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada3704-20bb-48b3-806b-057fe41c7580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:49.211818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada3704-20bb-48b3-806b-057fe41c7580 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:49.212276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada3704-20bb-48b3-806b-057fe41c7580 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:49.212565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada3704-20bb-48b3-806b-057fe41c7580 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:49.214987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada3704-20bb-48b3-806b-057fe41c7580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:49.219403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada3704-20bb-48b3-806b-057fe41c7580 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:49.230389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cf07a806-b8c2-4148-a5d4-e824f9db3ff4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49eb5255ebb548cd83f14ec16d53fda3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:49.247605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cf07a806-b8c2-4148-a5d4-e824f9db3ff4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49eb5255ebb548cd83f14ec16d53fda3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:49.249010 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2889/5848] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:49.253590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada3704-20bb-48b3-806b-057fe41c7580 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:49.276323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cf07a806-b8c2-4148-a5d4-e824f9db3ff4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49eb5255ebb548cd83f14ec16d53fda3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:49.277979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cf07a806-b8c2-4148-a5d4-e824f9db3ff4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49eb5255ebb548cd83f14ec16d53fda3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:49.278809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada3704-20bb-48b3-806b-057fe41c7580 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:49.293337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cf07a806-b8c2-4148-a5d4-e824f9db3ff4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49eb5255ebb548cd83f14ec16d53fda3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:49.300183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c51e8e-1c82-4baf-a2f8-f1d693881950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:49.301905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c51e8e-1c82-4baf-a2f8-f1d693881950 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:49.302557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c51e8e-1c82-4baf-a2f8-f1d693881950 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:49.303078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c51e8e-1c82-4baf-a2f8-f1d693881950 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:49.303820 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2960/5849] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:10:48 2022] DELETE /identity/v3/users/49eb5255ebb548cd83f14ec16d53fda3 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:49.304300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c51e8e-1c82-4baf-a2f8-f1d693881950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:49.306737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c51e8e-1c82-4baf-a2f8-f1d693881950 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:49.309303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bacb2f-a857-4175-8a9b-26d27a291303 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:49.309696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bacb2f-a857-4175-8a9b-26d27a291303 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:49.310117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bacb2f-a857-4175-8a9b-26d27a291303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:49.319585 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2890/5850] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:49.323625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c51e8e-1c82-4baf-a2f8-f1d693881950 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:49.325735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-05c51e8e-1c82-4baf-a2f8-f1d693881950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:49.334386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e3bacb2f-a857-4175-8a9b-26d27a291303 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:49.335889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e3bacb2f-a857-4175-8a9b-26d27a291303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:49.341679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c51e8e-1c82-4baf-a2f8-f1d693881950 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:49.373744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:49.385353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:49.423091 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2961/5851] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:10:49.432832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bacb2f-a857-4175-8a9b-26d27a291303 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:49.436246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e3bacb2f-a857-4175-8a9b-26d27a291303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:49.439580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23af823-911d-45bc-97d2-e8d5a1151ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:49.440258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23af823-911d-45bc-97d2-e8d5a1151ec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:49.440542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23af823-911d-45bc-97d2-e8d5a1151ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:49.440814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23af823-911d-45bc-97d2-e8d5a1151ec3 admin admin] PATH_INFO: `/v3/projects/9bd8ee9103544a4dbd40fa6f54ae344b/users/3c809773d07c44ee8f296e39613405ef/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:49.461916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23af823-911d-45bc-97d2-e8d5a1151ec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd8ee9103544a4dbd40fa6f54ae344b, user_id=3c809773d07c44ee8f296e39613405ef, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:49.465422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23af823-911d-45bc-97d2-e8d5a1151ec3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:49.492967 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2891/5852] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:10:49 2022] PUT /identity/v3/projects/9bd8ee9103544a4dbd40fa6f54ae344b/users/3c809773d07c44ee8f296e39613405ef/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:49.503239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248937a-38ef-4e1a-84d7-b958f48c6033 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:49.503684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248937a-38ef-4e1a-84d7-b958f48c6033 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:49.504833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248937a-38ef-4e1a-84d7-b958f48c6033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:49.527858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bacb2f-a857-4175-8a9b-26d27a291303 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:49.550165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4248937a-38ef-4e1a-84d7-b958f48c6033 None None] MFA Rules not processed for user `3c809773d07c44ee8f296e39613405ef`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:49.551970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4248937a-38ef-4e1a-84d7-b958f48c6033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:49.623216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccfca6-a3e0-40a7-831d-67469fdb3e87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:49.623946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccfca6-a3e0-40a7-831d-67469fdb3e87 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:49.624303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccfca6-a3e0-40a7-831d-67469fdb3e87 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:49.624523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccfca6-a3e0-40a7-831d-67469fdb3e87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:49.648413 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2892/5853] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:10:49.652714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248937a-38ef-4e1a-84d7-b958f48c6033 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:49.723477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccfca6-a3e0-40a7-831d-67469fdb3e87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:49.725340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccfca6-a3e0-40a7-831d-67469fdb3e87 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:49.733065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248937a-38ef-4e1a-84d7-b958f48c6033 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:49.741167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652ca4f-e4d1-45fc-abac-98a9b06ae31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:49.741908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652ca4f-e4d1-45fc-abac-98a9b06ae31a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:49.742225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652ca4f-e4d1-45fc-abac-98a9b06ae31a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:49.742695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652ca4f-e4d1-45fc-abac-98a9b06ae31a admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:49.743474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652ca4f-e4d1-45fc-abac-98a9b06ae31a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:49.745852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652ca4f-e4d1-45fc-abac-98a9b06ae31a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:49.753042 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2962/5854] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:10:49.787571 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2893/5855] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:10:49 2022] POST /identity/v3/projects => generated 387 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:10:49.788580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccfca6-a3e0-40a7-831d-67469fdb3e87 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:49.811378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccfca6-a3e0-40a7-831d-67469fdb3e87 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:49.912653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:49.940612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39633160-913e-4eb6-9f97-2cdab152eb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:49.941397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39633160-913e-4eb6-9f97-2cdab152eb00 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:49.942016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39633160-913e-4eb6-9f97-2cdab152eb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:49.942329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39633160-913e-4eb6-9f97-2cdab152eb00 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:49.942977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39633160-913e-4eb6-9f97-2cdab152eb00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:49.945560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39633160-913e-4eb6-9f97-2cdab152eb00 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:49.982324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652ca4f-e4d1-45fc-abac-98a9b06ae31a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:49.993462 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2963/5856] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:10:49 2022] POST /identity/v3/users => generated 300 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:10:49.996023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652ca4f-e4d1-45fc-abac-98a9b06ae31a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:50.002674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec02c9a-2730-4237-9ffe-c562cb102b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:50.003554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec02c9a-2730-4237-9ffe-c562cb102b29 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:50.003937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec02c9a-2730-4237-9ffe-c562cb102b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:50.004307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec02c9a-2730-4237-9ffe-c562cb102b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:50.005222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec02c9a-2730-4237-9ffe-c562cb102b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:50.007530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec02c9a-2730-4237-9ffe-c562cb102b29 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:50.037335 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2894/5857] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:50.040710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39633160-913e-4eb6-9f97-2cdab152eb00 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:50.060034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec02c9a-2730-4237-9ffe-c562cb102b29 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:50.070667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39633160-913e-4eb6-9f97-2cdab152eb00 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:50.091406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6ec02c9a-2730-4237-9ffe-c562cb102b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:50.092183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809d659-15c2-42e6-bf9e-6a82ac38c8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:50.092183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809d659-15c2-42e6-bf9e-6a82ac38c8e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:50.092183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809d659-15c2-42e6-bf9e-6a82ac38c8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:50.092183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809d659-15c2-42e6-bf9e-6a82ac38c8e6 admin admin] PATH_INFO: `/v3/projects/83d3f634613a4faeb0252cc6f3a6b9f4/users/cd3ed24e5dec462cb60d62198b9c54f4/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:50.099216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809d659-15c2-42e6-bf9e-6a82ac38c8e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83d3f634613a4faeb0252cc6f3a6b9f4, user_id=cd3ed24e5dec462cb60d62198b9c54f4, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:50.102084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809d659-15c2-42e6-bf9e-6a82ac38c8e6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:50.115279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec02c9a-2730-4237-9ffe-c562cb102b29 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:50.133624 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2964/5858] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:10:50 2022] PUT /identity/v3/projects/83d3f634613a4faeb0252cc6f3a6b9f4/users/cd3ed24e5dec462cb60d62198b9c54f4/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:50.134823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809d659-15c2-42e6-bf9e-6a82ac38c8e6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:50.136894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718f902-8c2d-426a-a9b2-b8ab3e21abe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:50.137558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718f902-8c2d-426a-a9b2-b8ab3e21abe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:50.137840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718f902-8c2d-426a-a9b2-b8ab3e21abe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:50.138173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718f902-8c2d-426a-a9b2-b8ab3e21abe4 admin admin] PATH_INFO: `/v3/projects/ef6dd37a1540437e95131d8bab0afc2a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:50.140098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718f902-8c2d-426a-a9b2-b8ab3e21abe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef6dd37a1540437e95131d8bab0afc2a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:50.142794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718f902-8c2d-426a-a9b2-b8ab3e21abe4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:50.157005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:50.221249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809d659-15c2-42e6-bf9e-6a82ac38c8e6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:50.225791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1718f902-8c2d-426a-a9b2-b8ab3e21abe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef6dd37a1540437e95131d8bab0afc2a', 'request_id': 'req-1718f902-8c2d-426a-a9b2-b8ab3e21abe4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:50.235297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1718f902-8c2d-426a-a9b2-b8ab3e21abe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef6dd37a1540437e95131d8bab0afc2a', 'request_id': 'req-1718f902-8c2d-426a-a9b2-b8ab3e21abe4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:50.244462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1718f902-8c2d-426a-a9b2-b8ab3e21abe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef6dd37a1540437e95131d8bab0afc2a', 'request_id': 'req-1718f902-8c2d-426a-a9b2-b8ab3e21abe4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:50.258627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1718f902-8c2d-426a-a9b2-b8ab3e21abe4 admin admin] The token cache is being invalidate because project ef6dd37a1540437e95131d8bab0afc2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:10:50.259080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1718f902-8c2d-426a-a9b2-b8ab3e21abe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:50.260088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1718f902-8c2d-426a-a9b2-b8ab3e21abe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:10:50.262115 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2895/5859] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:10:50 2022] DELETE /identity/v3/projects/ef6dd37a1540437e95131d8bab0afc2a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:10:50.266959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005392a-ff94-4a81-9e75-901c758cede4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:50.268086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005392a-ff94-4a81-9e75-901c758cede4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:50.268444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005392a-ff94-4a81-9e75-901c758cede4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:50.295086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6005392a-ff94-4a81-9e75-901c758cede4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:50.296564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6005392a-ff94-4a81-9e75-901c758cede4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:50.311898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab203b-4229-4e1e-9ca0-3be0d65e4828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:50.312725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab203b-4229-4e1e-9ca0-3be0d65e4828 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:50.317127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab203b-4229-4e1e-9ca0-3be0d65e4828 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:50.317754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab203b-4229-4e1e-9ca0-3be0d65e4828 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:50.319238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab203b-4229-4e1e-9ca0-3be0d65e4828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:50.321042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab203b-4229-4e1e-9ca0-3be0d65e4828 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:50.339194 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2965/5860] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:50.340781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab203b-4229-4e1e-9ca0-3be0d65e4828 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:50.342792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-17ab203b-4229-4e1e-9ca0-3be0d65e4828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:50.357168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab203b-4229-4e1e-9ca0-3be0d65e4828 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:50.363461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af520b-e0be-417c-a03d-a2818e3ac543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:50.364346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af520b-e0be-417c-a03d-a2818e3ac543 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:50.364796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af520b-e0be-417c-a03d-a2818e3ac543 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:50.365250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af520b-e0be-417c-a03d-a2818e3ac543 admin admin] PATH_INFO: `/v3/projects/83d3f634613a4faeb0252cc6f3a6b9f4/users/cd3ed24e5dec462cb60d62198b9c54f4/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:50.367715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af520b-e0be-417c-a03d-a2818e3ac543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83d3f634613a4faeb0252cc6f3a6b9f4, user_id=cd3ed24e5dec462cb60d62198b9c54f4, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:50.371152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af520b-e0be-417c-a03d-a2818e3ac543 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:50.399238 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2966/5861] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:10:50 2022] PUT /identity/v3/projects/83d3f634613a4faeb0252cc6f3a6b9f4/users/cd3ed24e5dec462cb60d62198b9c54f4/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:50.401421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051e0a8-445b-4cb5-9100-b8d772869d30 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:50.401711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051e0a8-445b-4cb5-9100-b8d772869d30 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:50.401987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051e0a8-445b-4cb5-9100-b8d772869d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:50.426856 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2896/5862] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:50.432573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c48d2-c3ef-4a68-bf44-974a528449d3 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:50.433094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c48d2-c3ef-4a68-bf44-974a528449d3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:50.433617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c48d2-c3ef-4a68-bf44-974a528449d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:50.457113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ec7c48d2-c3ef-4a68-bf44-974a528449d3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:50.458719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ec7c48d2-c3ef-4a68-bf44-974a528449d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:50.484647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5051e0a8-445b-4cb5-9100-b8d772869d30 None None] MFA Rules not processed for user `cd3ed24e5dec462cb60d62198b9c54f4`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:50.492326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5051e0a8-445b-4cb5-9100-b8d772869d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:50.584318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:50.589052 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2897/5863] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:10:50.595323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c48d2-c3ef-4a68-bf44-974a528449d3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:50.614386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c48d2-c3ef-4a68-bf44-974a528449d3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:50.623099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575fd11-736e-42ee-97b3-f59c09bc74e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:50.623790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575fd11-736e-42ee-97b3-f59c09bc74e3 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:50.623992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575fd11-736e-42ee-97b3-f59c09bc74e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:50.624270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575fd11-736e-42ee-97b3-f59c09bc74e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:50.624960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575fd11-736e-42ee-97b3-f59c09bc74e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:10:50.625224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575fd11-736e-42ee-97b3-f59c09bc74e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:50.628083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575fd11-736e-42ee-97b3-f59c09bc74e3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:50.641018 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2898/5864] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:10:50.646693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575fd11-736e-42ee-97b3-f59c09bc74e3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:50.648757 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2967/5865] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:50.655637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051e0a8-445b-4cb5-9100-b8d772869d30 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:50.657103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575fd11-736e-42ee-97b3-f59c09bc74e3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:50.657301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5051e0a8-445b-4cb5-9100-b8d772869d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:50.662698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac08b6-547d-4937-af25-fd0125a6505c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:50.663676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac08b6-547d-4937-af25-fd0125a6505c admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:50.663956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac08b6-547d-4937-af25-fd0125a6505c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:50.664227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac08b6-547d-4937-af25-fd0125a6505c admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:50.666230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac08b6-547d-4937-af25-fd0125a6505c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:50.669591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac08b6-547d-4937-af25-fd0125a6505c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:50.714876 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2899/5866] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:10:50 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:10:50.716312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac08b6-547d-4937-af25-fd0125a6505c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:50.728386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:50.781436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051e0a8-445b-4cb5-9100-b8d772869d30 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:50.805468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac08b6-547d-4937-af25-fd0125a6505c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:50.811679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc6425-1d10-4f62-b544-b5c7d200b225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:50.812325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc6425-1d10-4f62-b544-b5c7d200b225 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:50.812594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc6425-1d10-4f62-b544-b5c7d200b225 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:50.812872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc6425-1d10-4f62-b544-b5c7d200b225 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:50.813805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc6425-1d10-4f62-b544-b5c7d200b225 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:50.816580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc6425-1d10-4f62-b544-b5c7d200b225 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:50.846607 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2900/5867] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:10:50 2022] POST /identity/v3/users => generated 289 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:10:50.852145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc6425-1d10-4f62-b544-b5c7d200b225 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:50.868068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc6425-1d10-4f62-b544-b5c7d200b225 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:50.874522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c50d09-9256-4e52-ae14-eb0f89a84ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:50.875610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c50d09-9256-4e52-ae14-eb0f89a84ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:50.876922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c50d09-9256-4e52-ae14-eb0f89a84ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:50.877430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c50d09-9256-4e52-ae14-eb0f89a84ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:50.878322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c50d09-9256-4e52-ae14-eb0f89a84ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:50.880132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c50d09-9256-4e52-ae14-eb0f89a84ef3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:50.898584 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2901/5868] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:50.898833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1fc18-4b41-4559-a602-cff8814dc44f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:50.899437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1fc18-4b41-4559-a602-cff8814dc44f service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:50.899565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c50d09-9256-4e52-ae14-eb0f89a84ef3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:50.900118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1fc18-4b41-4559-a602-cff8814dc44f service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:50.901109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1fc18-4b41-4559-a602-cff8814dc44f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:50.904408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-32e1fc18-4b41-4559-a602-cff8814dc44f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:50.911771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c50d09-9256-4e52-ae14-eb0f89a84ef3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:50.918432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bfdf3-84db-4b1f-8f52-de70730fd020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:50.921054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bfdf3-84db-4b1f-8f52-de70730fd020 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:50.921341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bfdf3-84db-4b1f-8f52-de70730fd020 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:50.921647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bfdf3-84db-4b1f-8f52-de70730fd020 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:50.923129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726bfdf3-84db-4b1f-8f52-de70730fd020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:50.925531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726bfdf3-84db-4b1f-8f52-de70730fd020 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:50.945550 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2902/5869] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:10:50.951258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bfdf3-84db-4b1f-8f52-de70730fd020 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:50.963543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bfdf3-84db-4b1f-8f52-de70730fd020 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:50.968992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98913175-0f98-4897-8b73-2970763a6d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:50.970205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98913175-0f98-4897-8b73-2970763a6d18 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:50.970529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98913175-0f98-4897-8b73-2970763a6d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:50.970792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98913175-0f98-4897-8b73-2970763a6d18 admin admin] PATH_INFO: `/v3/projects/bfdac2fb886a4c3db79b662978ff5be1/users/14da1359f28d43e8b670eb44b02a5614/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:50.989643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98913175-0f98-4897-8b73-2970763a6d18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfdac2fb886a4c3db79b662978ff5be1, user_id=14da1359f28d43e8b670eb44b02a5614, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:50.993689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98913175-0f98-4897-8b73-2970763a6d18 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:51.003206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e1fc18-4b41-4559-a602-cff8814dc44f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:51.005500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e1fc18-4b41-4559-a602-cff8814dc44f service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:51.012479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:51.019829 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2903/5870] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:10:50 2022] PUT /identity/v3/projects/bfdac2fb886a4c3db79b662978ff5be1/users/14da1359f28d43e8b670eb44b02a5614/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:10:51.021543 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2968/5871] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:50 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:10:51.035646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017e778-2977-443a-94ea-7130a72eb317 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:51.036063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017e778-2977-443a-94ea-7130a72eb317 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:51.036437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017e778-2977-443a-94ea-7130a72eb317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:51.078721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0017e778-2977-443a-94ea-7130a72eb317 None None] MFA Rules not processed for user `14da1359f28d43e8b670eb44b02a5614`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:51.080748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0017e778-2977-443a-94ea-7130a72eb317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:51.226582 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2904/5872] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:10:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:51.242253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1fc18-4b41-4559-a602-cff8814dc44f service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:51.243763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-32e1fc18-4b41-4559-a602-cff8814dc44f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:51.329414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1fc18-4b41-4559-a602-cff8814dc44f service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:51.396408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f5d3f-1fd0-4580-bbc2-d8ad0aa63807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:51.397081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f5d3f-1fd0-4580-bbc2-d8ad0aa63807 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:51.397288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f5d3f-1fd0-4580-bbc2-d8ad0aa63807 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:51.397562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f5d3f-1fd0-4580-bbc2-d8ad0aa63807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:51.406627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f5d3f-1fd0-4580-bbc2-d8ad0aa63807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:51.408097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f5d3f-1fd0-4580-bbc2-d8ad0aa63807 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:51.419783 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2969/5873] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:10:54.087288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017e778-2977-443a-94ea-7130a72eb317 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:54.088731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0017e778-2977-443a-94ea-7130a72eb317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:54.109510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017e778-2977-443a-94ea-7130a72eb317 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:54.125802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69472efd-97e2-4a75-b204-2e9f50e55c42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:54.127176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69472efd-97e2-4a75-b204-2e9f50e55c42 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:54.127368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69472efd-97e2-4a75-b204-2e9f50e55c42 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:54.127751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69472efd-97e2-4a75-b204-2e9f50e55c42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:54.209707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:54.255758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69472efd-97e2-4a75-b204-2e9f50e55c42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:54.258440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69472efd-97e2-4a75-b204-2e9f50e55c42 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:54.272960 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2905/5874] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:10:54 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:10:58.549470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8bffb-9acd-4f20-929e-7e63b4c3674a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:58.550387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8bffb-9acd-4f20-929e-7e63b4c3674a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:58.551133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8bffb-9acd-4f20-929e-7e63b4c3674a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:58.601038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-dbc8bffb-9acd-4f20-929e-7e63b4c3674a None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:10:58.604117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-dbc8bffb-9acd-4f20-929e-7e63b4c3674a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:58.791189 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2970/5875] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:10:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:10:58.792498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69472efd-97e2-4a75-b204-2e9f50e55c42 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:10:58.794290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-69472efd-97e2-4a75-b204-2e9f50e55c42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:10:58.977499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69472efd-97e2-4a75-b204-2e9f50e55c42 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:10:59.003896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8480aeb-0dbc-4e8a-84f2-e443fe9bed2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:10:59.005046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8480aeb-0dbc-4e8a-84f2-e443fe9bed2d service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:10:59.007991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8480aeb-0dbc-4e8a-84f2-e443fe9bed2d service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:10:59.008964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8480aeb-0dbc-4e8a-84f2-e443fe9bed2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:10:59.035448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8480aeb-0dbc-4e8a-84f2-e443fe9bed2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:10:59.037405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8480aeb-0dbc-4e8a-84f2-e443fe9bed2d service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:10:59.057995 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2906/5876] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:10:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:11:03.432279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daed48-58dc-4b4e-89c2-abacb78409af None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:03.432798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daed48-58dc-4b4e-89c2-abacb78409af None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:03.432910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daed48-58dc-4b4e-89c2-abacb78409af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:03.476909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-83daed48-58dc-4b4e-89c2-abacb78409af None None] MFA Rules not processed for user `902ba44d318d4a01a510f732d27467c9`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:03.490360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-83daed48-58dc-4b4e-89c2-abacb78409af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:03.599380 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2971/5877] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:11:04.082007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9274b4-71b3-4820-83d7-dfec6c5f9a7e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:04.082623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9274b4-71b3-4820-83d7-dfec6c5f9a7e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:04.082961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9274b4-71b3-4820-83d7-dfec6c5f9a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:04.113768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daed48-58dc-4b4e-89c2-abacb78409af None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:04.116208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-83daed48-58dc-4b4e-89c2-abacb78409af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:04.120230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-cb9274b4-71b3-4820-83d7-dfec6c5f9a7e None None] MFA Rules not processed for user `484ee8f965604fa9b7f7f5a25a81d0ef`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:04.122095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cb9274b4-71b3-4820-83d7-dfec6c5f9a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:04.146319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:04.272670 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2907/5878] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:04.308087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daed48-58dc-4b4e-89c2-abacb78409af None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:04.320353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c592f3-1f1c-4303-9469-5f55eb7e35ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:04.321841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c592f3-1f1c-4303-9469-5f55eb7e35ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:04.322161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c592f3-1f1c-4303-9469-5f55eb7e35ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:04.323515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c592f3-1f1c-4303-9469-5f55eb7e35ad admin admin] PATH_INFO: `/v3/users/9b5f90f4ec8b4b5cb4b64a2b99ca9beb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:04.326306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c592f3-1f1c-4303-9469-5f55eb7e35ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b5f90f4ec8b4b5cb4b64a2b99ca9beb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:04.331184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c592f3-1f1c-4303-9469-5f55eb7e35ad admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:04.446511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-78c592f3-1f1c-4303-9469-5f55eb7e35ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b5f90f4ec8b4b5cb4b64a2b99ca9beb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:04.457171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-78c592f3-1f1c-4303-9469-5f55eb7e35ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b5f90f4ec8b4b5cb4b64a2b99ca9beb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:04.475470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-78c592f3-1f1c-4303-9469-5f55eb7e35ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b5f90f4ec8b4b5cb4b64a2b99ca9beb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:04.477552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-78c592f3-1f1c-4303-9469-5f55eb7e35ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b5f90f4ec8b4b5cb4b64a2b99ca9beb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:04.498085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-78c592f3-1f1c-4303-9469-5f55eb7e35ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b5f90f4ec8b4b5cb4b64a2b99ca9beb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:04.513992 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2972/5879] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:04 2022] DELETE /identity/v3/users/9b5f90f4ec8b4b5cb4b64a2b99ca9beb => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:04.773519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9274b4-71b3-4820-83d7-dfec6c5f9a7e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:04.802299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9274b4-71b3-4820-83d7-dfec6c5f9a7e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:04.876265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b2b10-8206-4997-a82e-19c5437b5da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:04.877250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b2b10-8206-4997-a82e-19c5437b5da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:04.877628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b2b10-8206-4997-a82e-19c5437b5da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:04.877939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b2b10-8206-4997-a82e-19c5437b5da2 admin admin] PATH_INFO: `/v3/users/14da1359f28d43e8b670eb44b02a5614` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:04.879204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b2b10-8206-4997-a82e-19c5437b5da2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14da1359f28d43e8b670eb44b02a5614)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:04.895012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b2b10-8206-4997-a82e-19c5437b5da2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:04.924797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:04.937618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c592f3-1f1c-4303-9469-5f55eb7e35ad admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:04.941269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-78c592f3-1f1c-4303-9469-5f55eb7e35ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:04.967217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c592f3-1f1c-4303-9469-5f55eb7e35ad admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:04.975096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee87284-67c8-4c54-8e23-52f9456b70b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:04.976123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee87284-67c8-4c54-8e23-52f9456b70b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:04.976628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee87284-67c8-4c54-8e23-52f9456b70b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:04.977094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee87284-67c8-4c54-8e23-52f9456b70b7 admin admin] PATH_INFO: `/v3/projects/0bf01814edfe46d5b1dd62f42c2fa45d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:04.979400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee87284-67c8-4c54-8e23-52f9456b70b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bf01814edfe46d5b1dd62f42c2fa45d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:04.994268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee87284-67c8-4c54-8e23-52f9456b70b7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:05.049804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-208b2b10-8206-4997-a82e-19c5437b5da2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14da1359f28d43e8b670eb44b02a5614'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:05.050523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-208b2b10-8206-4997-a82e-19c5437b5da2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14da1359f28d43e8b670eb44b02a5614'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:05.052048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-208b2b10-8206-4997-a82e-19c5437b5da2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14da1359f28d43e8b670eb44b02a5614'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:05.063485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-208b2b10-8206-4997-a82e-19c5437b5da2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14da1359f28d43e8b670eb44b02a5614'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:05.083712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-208b2b10-8206-4997-a82e-19c5437b5da2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14da1359f28d43e8b670eb44b02a5614'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:05.093271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7ee87284-67c8-4c54-8e23-52f9456b70b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bf01814edfe46d5b1dd62f42c2fa45d', 'request_id': 'req-7ee87284-67c8-4c54-8e23-52f9456b70b7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:05.104243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7ee87284-67c8-4c54-8e23-52f9456b70b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bf01814edfe46d5b1dd62f42c2fa45d', 'request_id': 'req-7ee87284-67c8-4c54-8e23-52f9456b70b7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:05.122044 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2908/5880] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:04 2022] DELETE /identity/v3/users/14da1359f28d43e8b670eb44b02a5614 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:05.123972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa189a1-91e7-43da-aba7-6c24415480c7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:05.124751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa189a1-91e7-43da-aba7-6c24415480c7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:05.125346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa189a1-91e7-43da-aba7-6c24415480c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:05.138839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7ee87284-67c8-4c54-8e23-52f9456b70b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bf01814edfe46d5b1dd62f42c2fa45d', 'request_id': 'req-7ee87284-67c8-4c54-8e23-52f9456b70b7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:05.161832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8fa189a1-91e7-43da-aba7-6c24415480c7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:05.164507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8fa189a1-91e7-43da-aba7-6c24415480c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:05.170643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7ee87284-67c8-4c54-8e23-52f9456b70b7 admin admin] The token cache is being invalidate because project 0bf01814edfe46d5b1dd62f42c2fa45d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:05.173280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7ee87284-67c8-4c54-8e23-52f9456b70b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:05.174170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7ee87284-67c8-4c54-8e23-52f9456b70b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:05.176263 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2973/5881] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:04 2022] DELETE /identity/v3/projects/0bf01814edfe46d5b1dd62f42c2fa45d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:05.180113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee87284-67c8-4c54-8e23-52f9456b70b7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:05.181816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7ee87284-67c8-4c54-8e23-52f9456b70b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:05.204368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:05.271248 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2909/5882] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:05.283514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa189a1-91e7-43da-aba7-6c24415480c7 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:05.286833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8fa189a1-91e7-43da-aba7-6c24415480c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:05.358567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee87284-67c8-4c54-8e23-52f9456b70b7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:05.374840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3536e-fdad-44ba-bc65-d0544142f1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:05.375876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3536e-fdad-44ba-bc65-d0544142f1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:05.376163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3536e-fdad-44ba-bc65-d0544142f1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:05.377419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3536e-fdad-44ba-bc65-d0544142f1e3 admin admin] PATH_INFO: `/v3/users/f6f982c714554445b6510ca5f726d126` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:05.382725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f3536e-fdad-44ba-bc65-d0544142f1e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6f982c714554445b6510ca5f726d126)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:05.390216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f3536e-fdad-44ba-bc65-d0544142f1e3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:05.415665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa189a1-91e7-43da-aba7-6c24415480c7 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:05.420612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f3599-b639-4b06-80eb-f80ba867fe83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:05.421334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f3599-b639-4b06-80eb-f80ba867fe83 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:05.423107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f3599-b639-4b06-80eb-f80ba867fe83 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:05.423271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f3599-b639-4b06-80eb-f80ba867fe83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:05.449183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f3599-b639-4b06-80eb-f80ba867fe83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:05.451706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f3599-b639-4b06-80eb-f80ba867fe83 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:05.480178 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2910/5883] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:11:05.526029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a1f3536e-fdad-44ba-bc65-d0544142f1e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6f982c714554445b6510ca5f726d126'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:05.548251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a1f3536e-fdad-44ba-bc65-d0544142f1e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6f982c714554445b6510ca5f726d126'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:05.604507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a1f3536e-fdad-44ba-bc65-d0544142f1e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6f982c714554445b6510ca5f726d126'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:05.605739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a1f3536e-fdad-44ba-bc65-d0544142f1e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6f982c714554445b6510ca5f726d126'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:05.625011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a1f3536e-fdad-44ba-bc65-d0544142f1e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6f982c714554445b6510ca5f726d126'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:05.645245 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2974/5884] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:05 2022] DELETE /identity/v3/users/f6f982c714554445b6510ca5f726d126 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:11:05.799030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f3599-b639-4b06-80eb-f80ba867fe83 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:05.800593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-799f3599-b639-4b06-80eb-f80ba867fe83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:05.819638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f3599-b639-4b06-80eb-f80ba867fe83 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:05.883447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:05.959013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0318b35-0c50-4640-bb14-32daa0afb64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:05.961390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0318b35-0c50-4640-bb14-32daa0afb64d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:05.961487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0318b35-0c50-4640-bb14-32daa0afb64d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:05.961762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0318b35-0c50-4640-bb14-32daa0afb64d admin admin] PATH_INFO: `/v3/projects/bfdac2fb886a4c3db79b662978ff5be1` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:05.964044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0318b35-0c50-4640-bb14-32daa0afb64d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfdac2fb886a4c3db79b662978ff5be1)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:05.966294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0318b35-0c50-4640-bb14-32daa0afb64d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:06.062446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e0318b35-0c50-4640-bb14-32daa0afb64d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfdac2fb886a4c3db79b662978ff5be1', 'request_id': 'req-e0318b35-0c50-4640-bb14-32daa0afb64d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:06.077807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e0318b35-0c50-4640-bb14-32daa0afb64d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfdac2fb886a4c3db79b662978ff5be1', 'request_id': 'req-e0318b35-0c50-4640-bb14-32daa0afb64d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:06.090805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e0318b35-0c50-4640-bb14-32daa0afb64d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfdac2fb886a4c3db79b662978ff5be1', 'request_id': 'req-e0318b35-0c50-4640-bb14-32daa0afb64d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:06.107386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e0318b35-0c50-4640-bb14-32daa0afb64d admin admin] The token cache is being invalidate because project bfdac2fb886a4c3db79b662978ff5be1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:06.107771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e0318b35-0c50-4640-bb14-32daa0afb64d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:06.108645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e0318b35-0c50-4640-bb14-32daa0afb64d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:06.113687 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2911/5885] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:05 2022] DELETE /identity/v3/projects/bfdac2fb886a4c3db79b662978ff5be1 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:06.115552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba85404-21d9-4456-94a1-4773e640ab53 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:06.115886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba85404-21d9-4456-94a1-4773e640ab53 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:06.116174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba85404-21d9-4456-94a1-4773e640ab53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:06.142025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8ba85404-21d9-4456-94a1-4773e640ab53 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:06.143418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8ba85404-21d9-4456-94a1-4773e640ab53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:06.180414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0318b35-0c50-4640-bb14-32daa0afb64d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:06.183145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e0318b35-0c50-4640-bb14-32daa0afb64d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:06.295287 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2975/5886] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:06.297687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0318b35-0c50-4640-bb14-32daa0afb64d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:06.297827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340efb98-5c9f-44aa-a706-612e9eb48e1b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:06.298434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340efb98-5c9f-44aa-a706-612e9eb48e1b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:06.299639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340efb98-5c9f-44aa-a706-612e9eb48e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:06.304675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fdd6d-07d9-4a75-be28-8f941efc0639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:06.305402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fdd6d-07d9-4a75-be28-8f941efc0639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:06.305750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fdd6d-07d9-4a75-be28-8f941efc0639 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:06.307102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fdd6d-07d9-4a75-be28-8f941efc0639 admin admin] PATH_INFO: `/v3/projects/921b1307c8974dcdbef3150b7712ec41` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:06.308433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60fdd6d-07d9-4a75-be28-8f941efc0639 admin admin] RBAC: Authorizing `identity:delete_project(project_id=921b1307c8974dcdbef3150b7712ec41)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:06.310428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60fdd6d-07d9-4a75-be28-8f941efc0639 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:06.334686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-340efb98-5c9f-44aa-a706-612e9eb48e1b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:06.337890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-340efb98-5c9f-44aa-a706-612e9eb48e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:06.353971 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2976/5887] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:06.360650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340efb98-5c9f-44aa-a706-612e9eb48e1b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:06.396058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340efb98-5c9f-44aa-a706-612e9eb48e1b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:06.400173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c60fdd6d-07d9-4a75-be28-8f941efc0639 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '921b1307c8974dcdbef3150b7712ec41', 'request_id': 'req-c60fdd6d-07d9-4a75-be28-8f941efc0639'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:06.413069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c60fdd6d-07d9-4a75-be28-8f941efc0639 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '921b1307c8974dcdbef3150b7712ec41', 'request_id': 'req-c60fdd6d-07d9-4a75-be28-8f941efc0639'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:06.418383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:06.421501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c60fdd6d-07d9-4a75-be28-8f941efc0639 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '921b1307c8974dcdbef3150b7712ec41', 'request_id': 'req-c60fdd6d-07d9-4a75-be28-8f941efc0639'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:06.438620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c60fdd6d-07d9-4a75-be28-8f941efc0639 admin admin] The token cache is being invalidate because project 921b1307c8974dcdbef3150b7712ec41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:06.439108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c60fdd6d-07d9-4a75-be28-8f941efc0639 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:06.439942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c60fdd6d-07d9-4a75-be28-8f941efc0639 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:06.441983 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2912/5888] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:06 2022] DELETE /identity/v3/projects/921b1307c8974dcdbef3150b7712ec41 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:06.457528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:06.458613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82217480-1b6f-490e-92f1-73714899308d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:06.458883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82217480-1b6f-490e-92f1-73714899308d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:06.459163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82217480-1b6f-490e-92f1-73714899308d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:06.500863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-82217480-1b6f-490e-92f1-73714899308d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:06.502683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-82217480-1b6f-490e-92f1-73714899308d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:06.556636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbd3bf-4678-45ab-8b50-56b5582fb3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:06.557289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbd3bf-4678-45ab-8b50-56b5582fb3dd admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:06.557578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbd3bf-4678-45ab-8b50-56b5582fb3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:06.558113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbd3bf-4678-45ab-8b50-56b5582fb3dd admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:06.560349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbd3bf-4678-45ab-8b50-56b5582fb3dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:11:06.561470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbd3bf-4678-45ab-8b50-56b5582fb3dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:06.564556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbd3bf-4678-45ab-8b50-56b5582fb3dd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:06.601076 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2977/5889] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:11:06.601678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbd3bf-4678-45ab-8b50-56b5582fb3dd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:06.604397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-06fbd3bf-4678-45ab-8b50-56b5582fb3dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:06.644043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbd3bf-4678-45ab-8b50-56b5582fb3dd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:06.654959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7909f5-2ece-4993-b17e-96233d88fb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:06.656025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7909f5-2ece-4993-b17e-96233d88fb78 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:06.656578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7909f5-2ece-4993-b17e-96233d88fb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:06.657588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7909f5-2ece-4993-b17e-96233d88fb78 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:06.658686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7909f5-2ece-4993-b17e-96233d88fb78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:06.660388 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2913/5890] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:06.661293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7909f5-2ece-4993-b17e-96233d88fb78 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:06.667778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c73244-ecc5-4870-bb66-e9294f6e9da1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:06.668395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c73244-ecc5-4870-bb66-e9294f6e9da1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:06.668891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c73244-ecc5-4870-bb66-e9294f6e9da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:06.694439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-55c73244-ecc5-4870-bb66-e9294f6e9da1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:06.696674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-55c73244-ecc5-4870-bb66-e9294f6e9da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:06.723352 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2914/5891] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:06.727294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c73244-ecc5-4870-bb66-e9294f6e9da1 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:06.745309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c73244-ecc5-4870-bb66-e9294f6e9da1 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:06.751859 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2978/5892] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:06 2022] POST /identity/v3/projects => generated 375 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:06.754635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7909f5-2ece-4993-b17e-96233d88fb78 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:06.896969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1401e2-07b8-43fd-ae80-e0c4ec869c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:06.897659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1401e2-07b8-43fd-ae80-e0c4ec869c57 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:06.898041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1401e2-07b8-43fd-ae80-e0c4ec869c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:06.898605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1401e2-07b8-43fd-ae80-e0c4ec869c57 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:06.899586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1401e2-07b8-43fd-ae80-e0c4ec869c57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:11:06.900601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1401e2-07b8-43fd-ae80-e0c4ec869c57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:06.903784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1401e2-07b8-43fd-ae80-e0c4ec869c57 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:06.941124 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2915/5893] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:11:06.943958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1401e2-07b8-43fd-ae80-e0c4ec869c57 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:06.947336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7909f5-2ece-4993-b17e-96233d88fb78 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:06.958876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854d5c2-89c8-4967-9f39-80b1b2adc3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:06.960024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854d5c2-89c8-4967-9f39-80b1b2adc3a9 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:06.960646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854d5c2-89c8-4967-9f39-80b1b2adc3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:06.961160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854d5c2-89c8-4967-9f39-80b1b2adc3a9 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:06.962179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854d5c2-89c8-4967-9f39-80b1b2adc3a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:06.965802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854d5c2-89c8-4967-9f39-80b1b2adc3a9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:06.969187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1401e2-07b8-43fd-ae80-e0c4ec869c57 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:06.983227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc3c7b-8761-4bc8-94d9-0fddae5c97e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:06.987700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc3c7b-8761-4bc8-94d9-0fddae5c97e0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:06.988128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc3c7b-8761-4bc8-94d9-0fddae5c97e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:06.988513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc3c7b-8761-4bc8-94d9-0fddae5c97e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:06.998025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc3c7b-8761-4bc8-94d9-0fddae5c97e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:07.009272 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2979/5894] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:06 2022] POST /identity/v3/users => generated 288 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:07.009541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc3c7b-8761-4bc8-94d9-0fddae5c97e0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:07.011511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854d5c2-89c8-4967-9f39-80b1b2adc3a9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:07.031060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854d5c2-89c8-4967-9f39-80b1b2adc3a9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:07.039727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be74d41-82ce-4a45-8c75-1a07b9508640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:07.040717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be74d41-82ce-4a45-8c75-1a07b9508640 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:07.041187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be74d41-82ce-4a45-8c75-1a07b9508640 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:07.041662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be74d41-82ce-4a45-8c75-1a07b9508640 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:07.042480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be74d41-82ce-4a45-8c75-1a07b9508640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:07.045236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be74d41-82ce-4a45-8c75-1a07b9508640 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:07.060524 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2916/5895] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:06 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:07.068149 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2980/5896] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:07.069981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc3c7b-8761-4bc8-94d9-0fddae5c97e0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:07.072107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be74d41-82ce-4a45-8c75-1a07b9508640 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:07.091425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be74d41-82ce-4a45-8c75-1a07b9508640 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:07.214355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:07.215631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98f492-d963-479d-88d8-50b721b92fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:07.216781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98f492-d963-479d-88d8-50b721b92fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:07.217171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98f492-d963-479d-88d8-50b721b92fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:07.217621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98f492-d963-479d-88d8-50b721b92fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:07.218395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98f492-d963-479d-88d8-50b721b92fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:07.221352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98f492-d963-479d-88d8-50b721b92fd9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:07.224793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc3c7b-8761-4bc8-94d9-0fddae5c97e0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:07.239044 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2981/5897] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:07.239279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ea55c-3624-4035-ba12-7084b9b21938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:07.239279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ea55c-3624-4035-ba12-7084b9b21938 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:07.239279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ea55c-3624-4035-ba12-7084b9b21938 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:07.239977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ea55c-3624-4035-ba12-7084b9b21938 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:07.240942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ea55c-3624-4035-ba12-7084b9b21938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:07.241478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98f492-d963-479d-88d8-50b721b92fd9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:07.243783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ea55c-3624-4035-ba12-7084b9b21938 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:07.278323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98f492-d963-479d-88d8-50b721b92fd9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:07.282206 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2917/5898] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:07 2022] POST /identity/v3/users => generated 290 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:07.289930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ea55c-3624-4035-ba12-7084b9b21938 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:07.291630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fa76a-6a92-4899-9ac5-03719a8c17ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:07.292760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fa76a-6a92-4899-9ac5-03719a8c17ff admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:07.293153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fa76a-6a92-4899-9ac5-03719a8c17ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:07.293498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fa76a-6a92-4899-9ac5-03719a8c17ff admin admin] PATH_INFO: `/v3/projects/9a8c08a222ec47eb9573991926aba5fd/users/90707ccf791a416c8805706ad6b33750/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:07.299169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:07.318248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ea55c-3624-4035-ba12-7084b9b21938 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:07.321372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562fa76a-6a92-4899-9ac5-03719a8c17ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a8c08a222ec47eb9573991926aba5fd, user_id=90707ccf791a416c8805706ad6b33750, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:07.324748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562fa76a-6a92-4899-9ac5-03719a8c17ff admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:07.332874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ed922-0b24-4c7a-9b3b-ca024e237ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:07.333992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ed922-0b24-4c7a-9b3b-ca024e237ddd admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:07.335427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ed922-0b24-4c7a-9b3b-ca024e237ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:07.335427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ed922-0b24-4c7a-9b3b-ca024e237ddd admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:07.335683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ed922-0b24-4c7a-9b3b-ca024e237ddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:07.337604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ed922-0b24-4c7a-9b3b-ca024e237ddd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:07.351983 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2982/5899] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:07 2022] PUT /identity/v3/projects/9a8c08a222ec47eb9573991926aba5fd/users/90707ccf791a416c8805706ad6b33750/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:07.362272 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2918/5900] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:11:07.364010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25c308-f26d-4d46-a30e-d7a1235b839b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:07.364337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25c308-f26d-4d46-a30e-d7a1235b839b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:07.364637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25c308-f26d-4d46-a30e-d7a1235b839b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:07.372871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ed922-0b24-4c7a-9b3b-ca024e237ddd admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:07.403766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ed922-0b24-4c7a-9b3b-ca024e237ddd admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:07.422626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ec25c308-f26d-4d46-a30e-d7a1235b839b None None] MFA Rules not processed for user `90707ccf791a416c8805706ad6b33750`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:07.424859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ec25c308-f26d-4d46-a30e-d7a1235b839b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:07.567133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412bf08-e893-4b3b-8dc9-cad3bcd241fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:07.568141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412bf08-e893-4b3b-8dc9-cad3bcd241fa admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:07.568359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412bf08-e893-4b3b-8dc9-cad3bcd241fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:07.568609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412bf08-e893-4b3b-8dc9-cad3bcd241fa admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:07.571181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c412bf08-e893-4b3b-8dc9-cad3bcd241fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:07.574709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c412bf08-e893-4b3b-8dc9-cad3bcd241fa admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:07.604231 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2919/5901] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:07.605160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412bf08-e893-4b3b-8dc9-cad3bcd241fa admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:07.612030 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2983/5902] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:07.633980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412bf08-e893-4b3b-8dc9-cad3bcd241fa admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:07.644685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683946a0-b1f8-4c33-95e9-a1e496a107a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:07.645038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25c308-f26d-4d46-a30e-d7a1235b839b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:07.645703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683946a0-b1f8-4c33-95e9-a1e496a107a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:07.646244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683946a0-b1f8-4c33-95e9-a1e496a107a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:07.646657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683946a0-b1f8-4c33-95e9-a1e496a107a1 admin admin] PATH_INFO: `/v3/projects/e763f5a0a739414b96c9346b82154f41/users/f91d413b46d44b2ba9257de2f6b54955/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:07.647089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ec25c308-f26d-4d46-a30e-d7a1235b839b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:07.689094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683946a0-b1f8-4c33-95e9-a1e496a107a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e763f5a0a739414b96c9346b82154f41, user_id=f91d413b46d44b2ba9257de2f6b54955, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:07.699835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683946a0-b1f8-4c33-95e9-a1e496a107a1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:07.751559 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2920/5903] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:07 2022] PUT /identity/v3/projects/e763f5a0a739414b96c9346b82154f41/users/f91d413b46d44b2ba9257de2f6b54955/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:07.757036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e29ee9-a50e-49e7-9264-28f85fdf2112 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:07.757546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e29ee9-a50e-49e7-9264-28f85fdf2112 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:07.758532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e29ee9-a50e-49e7-9264-28f85fdf2112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:07.782768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25c308-f26d-4d46-a30e-d7a1235b839b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:07.787495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-08e29ee9-a50e-49e7-9264-28f85fdf2112 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:07.791154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-08e29ee9-a50e-49e7-9264-28f85fdf2112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:07.883304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73afcb7-16fb-4f55-8dbc-6eba060f7a67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:07.884266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73afcb7-16fb-4f55-8dbc-6eba060f7a67 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:07.884743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73afcb7-16fb-4f55-8dbc-6eba060f7a67 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:07.885490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73afcb7-16fb-4f55-8dbc-6eba060f7a67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:07.924289 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2921/5904] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:07.944729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e29ee9-a50e-49e7-9264-28f85fdf2112 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:07.962446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e29ee9-a50e-49e7-9264-28f85fdf2112 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:07.969321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:07.975322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709041e-4908-4b8c-8075-8a77771010c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:07.977125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709041e-4908-4b8c-8075-8a77771010c7 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:07.977774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709041e-4908-4b8c-8075-8a77771010c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:07.978341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709041e-4908-4b8c-8075-8a77771010c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:07.995340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73afcb7-16fb-4f55-8dbc-6eba060f7a67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:07.999842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f709041e-4908-4b8c-8075-8a77771010c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:08.000090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73afcb7-16fb-4f55-8dbc-6eba060f7a67 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:08.002664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f709041e-4908-4b8c-8075-8a77771010c7 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:08.022895 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2922/5905] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:11:08.023769 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2984/5906] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:11:08.043031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709041e-4908-4b8c-8075-8a77771010c7 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:08.065184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709041e-4908-4b8c-8075-8a77771010c7 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:08.071778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d75e84-3aa5-4cd8-8714-4ba060a268a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:08.072773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d75e84-3aa5-4cd8-8714-4ba060a268a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:08.073459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d75e84-3aa5-4cd8-8714-4ba060a268a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:08.074451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d75e84-3aa5-4cd8-8714-4ba060a268a3 admin admin] PATH_INFO: `/v3/users/90707ccf791a416c8805706ad6b33750` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:08.075847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d75e84-3aa5-4cd8-8714-4ba060a268a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90707ccf791a416c8805706ad6b33750)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:08.078517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d75e84-3aa5-4cd8-8714-4ba060a268a3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:08.129106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:08.165091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-43d75e84-3aa5-4cd8-8714-4ba060a268a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90707ccf791a416c8805706ad6b33750'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:08.166234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-43d75e84-3aa5-4cd8-8714-4ba060a268a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90707ccf791a416c8805706ad6b33750'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:08.166953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-43d75e84-3aa5-4cd8-8714-4ba060a268a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90707ccf791a416c8805706ad6b33750'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:08.186041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-43d75e84-3aa5-4cd8-8714-4ba060a268a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90707ccf791a416c8805706ad6b33750'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:08.209812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-43d75e84-3aa5-4cd8-8714-4ba060a268a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90707ccf791a416c8805706ad6b33750'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:08.227966 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2923/5907] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:08 2022] DELETE /identity/v3/users/90707ccf791a416c8805706ad6b33750 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:08.232750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da389f-77ae-4c90-aafe-9295a6a86cc5 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:08.233675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da389f-77ae-4c90-aafe-9295a6a86cc5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:08.233675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da389f-77ae-4c90-aafe-9295a6a86cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:08.267329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e0da389f-77ae-4c90-aafe-9295a6a86cc5 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:08.268192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e0da389f-77ae-4c90-aafe-9295a6a86cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:08.398387 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2985/5908] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:08.411616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d75e84-3aa5-4cd8-8714-4ba060a268a3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:08.414307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-43d75e84-3aa5-4cd8-8714-4ba060a268a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:08.502645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d75e84-3aa5-4cd8-8714-4ba060a268a3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:08.507292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80859cd-340c-4998-be3a-8cbf0a3ff60b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:08.507904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80859cd-340c-4998-be3a-8cbf0a3ff60b service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:08.508191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80859cd-340c-4998-be3a-8cbf0a3ff60b service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:08.508481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80859cd-340c-4998-be3a-8cbf0a3ff60b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:08.526663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80859cd-340c-4998-be3a-8cbf0a3ff60b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:08.528623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80859cd-340c-4998-be3a-8cbf0a3ff60b service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:08.543631 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2924/5909] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:11:09.895669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da389f-77ae-4c90-aafe-9295a6a86cc5 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:09.897793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e0da389f-77ae-4c90-aafe-9295a6a86cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:09.950720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da389f-77ae-4c90-aafe-9295a6a86cc5 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:09.957687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bd555-d4a8-49c8-b9dc-cdf6c8de26aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:09.958816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bd555-d4a8-49c8-b9dc-cdf6c8de26aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:09.959405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bd555-d4a8-49c8-b9dc-cdf6c8de26aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:09.960095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bd555-d4a8-49c8-b9dc-cdf6c8de26aa admin admin] PATH_INFO: `/v3/projects/9a8c08a222ec47eb9573991926aba5fd` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:09.961779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bd555-d4a8-49c8-b9dc-cdf6c8de26aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a8c08a222ec47eb9573991926aba5fd)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:09.964285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bd555-d4a8-49c8-b9dc-cdf6c8de26aa admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:10.130265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d58bd555-d4a8-49c8-b9dc-cdf6c8de26aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a8c08a222ec47eb9573991926aba5fd', 'request_id': 'req-d58bd555-d4a8-49c8-b9dc-cdf6c8de26aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:10.138647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d58bd555-d4a8-49c8-b9dc-cdf6c8de26aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a8c08a222ec47eb9573991926aba5fd', 'request_id': 'req-d58bd555-d4a8-49c8-b9dc-cdf6c8de26aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:10.147928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d58bd555-d4a8-49c8-b9dc-cdf6c8de26aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a8c08a222ec47eb9573991926aba5fd', 'request_id': 'req-d58bd555-d4a8-49c8-b9dc-cdf6c8de26aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:10.186670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d58bd555-d4a8-49c8-b9dc-cdf6c8de26aa admin admin] The token cache is being invalidate because project 9a8c08a222ec47eb9573991926aba5fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:10.187091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d58bd555-d4a8-49c8-b9dc-cdf6c8de26aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:10.189052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d58bd555-d4a8-49c8-b9dc-cdf6c8de26aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:10.192274 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2986/5910] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:09 2022] DELETE /identity/v3/projects/9a8c08a222ec47eb9573991926aba5fd => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:10.200764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635526d0-9915-4846-b3e6-ff18d77f3bd9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:10.201528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635526d0-9915-4846-b3e6-ff18d77f3bd9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:10.201965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635526d0-9915-4846-b3e6-ff18d77f3bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:10.235906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-635526d0-9915-4846-b3e6-ff18d77f3bd9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:10.237650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-635526d0-9915-4846-b3e6-ff18d77f3bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:10.346755 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2925/5911] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:11:10.356819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df707957-4050-4cea-b953-f1707b276f8a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:10.357403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df707957-4050-4cea-b953-f1707b276f8a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:10.358131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df707957-4050-4cea-b953-f1707b276f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:10.398941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-df707957-4050-4cea-b953-f1707b276f8a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:10.402545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-df707957-4050-4cea-b953-f1707b276f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:10.425067 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2987/5912] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:11:10.430353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635526d0-9915-4846-b3e6-ff18d77f3bd9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:10.462038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635526d0-9915-4846-b3e6-ff18d77f3bd9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:10.476988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe42f86-12dc-481d-a5af-90ff22e6426e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:10.477837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe42f86-12dc-481d-a5af-90ff22e6426e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:10.478121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe42f86-12dc-481d-a5af-90ff22e6426e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:10.478393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe42f86-12dc-481d-a5af-90ff22e6426e admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:10.479345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe42f86-12dc-481d-a5af-90ff22e6426e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:11:10.479643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe42f86-12dc-481d-a5af-90ff22e6426e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:10.481262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe42f86-12dc-481d-a5af-90ff22e6426e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:10.523523 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2926/5913] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:11:10.528048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df707957-4050-4cea-b953-f1707b276f8a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:10.545929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df707957-4050-4cea-b953-f1707b276f8a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:10.558311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7681d5-2920-4181-8d16-9d3d001a3f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:10.559140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7681d5-2920-4181-8d16-9d3d001a3f68 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:10.560233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7681d5-2920-4181-8d16-9d3d001a3f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:10.560622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7681d5-2920-4181-8d16-9d3d001a3f68 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:10.562077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7681d5-2920-4181-8d16-9d3d001a3f68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:10.564245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7681d5-2920-4181-8d16-9d3d001a3f68 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:10.662247 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2988/5914] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:10 2022] POST /identity/v3/projects => generated 353 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:10.665667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe42f86-12dc-481d-a5af-90ff22e6426e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:10.685258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe42f86-12dc-481d-a5af-90ff22e6426e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:10.764674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:10.813541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5dca5a-40e5-4fa0-971d-75f46ff55370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:10.814467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5dca5a-40e5-4fa0-971d-75f46ff55370 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:10.814935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5dca5a-40e5-4fa0-971d-75f46ff55370 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:10.815390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5dca5a-40e5-4fa0-971d-75f46ff55370 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:10.816375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5dca5a-40e5-4fa0-971d-75f46ff55370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:10.818669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5dca5a-40e5-4fa0-971d-75f46ff55370 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:10.851279 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2927/5915] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:10 2022] POST /identity/v3/users => generated 277 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:10.855291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7681d5-2920-4181-8d16-9d3d001a3f68 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:10.877237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7681d5-2920-4181-8d16-9d3d001a3f68 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:10.882827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4dd30-5fba-4cf8-8d90-a4b88c00cc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:10.883594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4dd30-5fba-4cf8-8d90-a4b88c00cc76 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:10.883891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4dd30-5fba-4cf8-8d90-a4b88c00cc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:10.884210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4dd30-5fba-4cf8-8d90-a4b88c00cc76 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:10.885087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4dd30-5fba-4cf8-8d90-a4b88c00cc76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:10.889515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4dd30-5fba-4cf8-8d90-a4b88c00cc76 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:10.901814 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2989/5916] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:10.905877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5dca5a-40e5-4fa0-971d-75f46ff55370 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:10.922350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5dca5a-40e5-4fa0-971d-75f46ff55370 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:10.930067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb84f2-8915-4e11-aece-b20c7b8b1796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:10.930931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb84f2-8915-4e11-aece-b20c7b8b1796 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:10.931261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb84f2-8915-4e11-aece-b20c7b8b1796 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:10.931874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb84f2-8915-4e11-aece-b20c7b8b1796 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:10.932808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb84f2-8915-4e11-aece-b20c7b8b1796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:10.934603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb84f2-8915-4e11-aece-b20c7b8b1796 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:10.978699 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2928/5917] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:10.982908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4dd30-5fba-4cf8-8d90-a4b88c00cc76 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:10.997392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4dd30-5fba-4cf8-8d90-a4b88c00cc76 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:11.014358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4156c88-a3cd-4852-bab7-d044ca242585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:11.015288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4156c88-a3cd-4852-bab7-d044ca242585 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:11.015672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4156c88-a3cd-4852-bab7-d044ca242585 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:11.016128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4156c88-a3cd-4852-bab7-d044ca242585 admin admin] PATH_INFO: `/v3/projects/06940ad486584ae391c5891858e8af7a/users/bf57979b45f14ca0a58a89961875f6cd/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:11.073968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4156c88-a3cd-4852-bab7-d044ca242585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06940ad486584ae391c5891858e8af7a, user_id=bf57979b45f14ca0a58a89961875f6cd, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:11.080372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4156c88-a3cd-4852-bab7-d044ca242585 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:11.121941 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2990/5918] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:10 2022] PUT /identity/v3/projects/06940ad486584ae391c5891858e8af7a/users/bf57979b45f14ca0a58a89961875f6cd/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:11.137510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b3b30-c6d1-4861-bd24-45eb206e874b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:11.138393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b3b30-c6d1-4861-bd24-45eb206e874b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:11.138791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b3b30-c6d1-4861-bd24-45eb206e874b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:11.225270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2f3b3b30-c6d1-4861-bd24-45eb206e874b None None] MFA Rules not processed for user `bf57979b45f14ca0a58a89961875f6cd`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:11.229110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2f3b3b30-c6d1-4861-bd24-45eb206e874b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:11.409773 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2929/5919] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:11 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:11.414305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4156c88-a3cd-4852-bab7-d044ca242585 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:11.426532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:11.517559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4156c88-a3cd-4852-bab7-d044ca242585 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:11.525369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4168518-cb84-485b-b46f-3d5e57453753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:11.526732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4168518-cb84-485b-b46f-3d5e57453753 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:11.527469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4168518-cb84-485b-b46f-3d5e57453753 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:11.528139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4168518-cb84-485b-b46f-3d5e57453753 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:11.528939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4168518-cb84-485b-b46f-3d5e57453753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:11.532418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4168518-cb84-485b-b46f-3d5e57453753 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:11.572577 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2991/5920] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:11 2022] POST /identity/v3/projects => generated 357 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:11.576816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b3b30-c6d1-4861-bd24-45eb206e874b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:11.590759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b3b30-c6d1-4861-bd24-45eb206e874b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:11.691631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000feab-0a90-4579-ab40-fabe98e325e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:11.692610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000feab-0a90-4579-ab40-fabe98e325e3 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:11.692898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000feab-0a90-4579-ab40-fabe98e325e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:11.693175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000feab-0a90-4579-ab40-fabe98e325e3 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:11.693812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d000feab-0a90-4579-ab40-fabe98e325e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:11.696172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d000feab-0a90-4579-ab40-fabe98e325e3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:11.739752 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2930/5921] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:11 2022] POST /identity/v3/users => generated 285 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:11.744334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4168518-cb84-485b-b46f-3d5e57453753 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:11.773225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4168518-cb84-485b-b46f-3d5e57453753 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:11.782119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bab38-beaf-476f-8b8f-07e9efd81d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:11.818919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bab38-beaf-476f-8b8f-07e9efd81d4f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:11.821616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bab38-beaf-476f-8b8f-07e9efd81d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:11.822063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bab38-beaf-476f-8b8f-07e9efd81d4f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:11.823618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bab38-beaf-476f-8b8f-07e9efd81d4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:11.831027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bab38-beaf-476f-8b8f-07e9efd81d4f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:11.852649 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2992/5922] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:11.858332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000feab-0a90-4579-ab40-fabe98e325e3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:11.875898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000feab-0a90-4579-ab40-fabe98e325e3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:11.888372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f2cfe-9408-4f1e-b4c2-0b5c96d4bb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:11.889127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f2cfe-9408-4f1e-b4c2-0b5c96d4bb5c admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:11.889420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f2cfe-9408-4f1e-b4c2-0b5c96d4bb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:11.889741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f2cfe-9408-4f1e-b4c2-0b5c96d4bb5c admin admin] PATH_INFO: `/v3/projects/1a5366898e7d496fbbaf6c1f1b932ba2/users/c03f4441acb3476bb03057521692d8dd/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:11.915964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f2cfe-9408-4f1e-b4c2-0b5c96d4bb5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5366898e7d496fbbaf6c1f1b932ba2, user_id=c03f4441acb3476bb03057521692d8dd, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:11.919597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f2cfe-9408-4f1e-b4c2-0b5c96d4bb5c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:11.945390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bab38-beaf-476f-8b8f-07e9efd81d4f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:11.945939 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2931/5923] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:11 2022] PUT /identity/v3/projects/1a5366898e7d496fbbaf6c1f1b932ba2/users/c03f4441acb3476bb03057521692d8dd/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:11.958118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bab38-beaf-476f-8b8f-07e9efd81d4f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:12.085886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e315a-0590-43e2-bbe2-bb898830225d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:12.086743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e315a-0590-43e2-bbe2-bb898830225d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:12.087123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e315a-0590-43e2-bbe2-bb898830225d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:12.087602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e315a-0590-43e2-bbe2-bb898830225d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:12.088471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e315a-0590-43e2-bbe2-bb898830225d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:12.090285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e315a-0590-43e2-bbe2-bb898830225d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:12.106943 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2993/5924] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:12.111342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f2cfe-9408-4f1e-b4c2-0b5c96d4bb5c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:12.123713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f2cfe-9408-4f1e-b4c2-0b5c96d4bb5c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:12.131041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23da26f-1689-4c37-843d-7a6e779bb326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:12.131945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23da26f-1689-4c37-843d-7a6e779bb326 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:12.132348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23da26f-1689-4c37-843d-7a6e779bb326 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:12.132838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23da26f-1689-4c37-843d-7a6e779bb326 admin admin] PATH_INFO: `/v3/projects/1a5366898e7d496fbbaf6c1f1b932ba2/users/c03f4441acb3476bb03057521692d8dd/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:12.137611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23da26f-1689-4c37-843d-7a6e779bb326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5366898e7d496fbbaf6c1f1b932ba2, user_id=c03f4441acb3476bb03057521692d8dd, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:12.141948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23da26f-1689-4c37-843d-7a6e779bb326 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:12.145488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:12.159192 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2932/5925] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:12 2022] PUT /identity/v3/projects/1a5366898e7d496fbbaf6c1f1b932ba2/users/c03f4441acb3476bb03057521692d8dd/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:12.169520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c793255-ef1a-466c-8b44-eec0b810b902 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:12.169768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c793255-ef1a-466c-8b44-eec0b810b902 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:12.170064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c793255-ef1a-466c-8b44-eec0b810b902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:12.210162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0c793255-ef1a-466c-8b44-eec0b810b902 None None] MFA Rules not processed for user `c03f4441acb3476bb03057521692d8dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:12.211923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0c793255-ef1a-466c-8b44-eec0b810b902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:12.327742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:12.337842 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2994/5926] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:12 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:12.346550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23da26f-1689-4c37-843d-7a6e779bb326 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:12.348279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f23da26f-1689-4c37-843d-7a6e779bb326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:12.446924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23da26f-1689-4c37-843d-7a6e779bb326 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:12.451651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c732c-6a0a-4356-9500-c080f6e49875 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:12.452839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c732c-6a0a-4356-9500-c080f6e49875 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:12.452839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c732c-6a0a-4356-9500-c080f6e49875 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:12.453704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c732c-6a0a-4356-9500-c080f6e49875 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:12.465260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c732c-6a0a-4356-9500-c080f6e49875 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:12.466822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c732c-6a0a-4356-9500-c080f6e49875 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:12.492077 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2933/5927] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:12 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:11:12.613458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c793255-ef1a-466c-8b44-eec0b810b902 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:12.621233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0c793255-ef1a-466c-8b44-eec0b810b902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:12.648740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c793255-ef1a-466c-8b44-eec0b810b902 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:12.655069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56003f9-7b38-4d60-9cde-1869b4106c80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:12.657598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56003f9-7b38-4d60-9cde-1869b4106c80 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:12.658194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56003f9-7b38-4d60-9cde-1869b4106c80 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:12.659187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56003f9-7b38-4d60-9cde-1869b4106c80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:12.774166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56003f9-7b38-4d60-9cde-1869b4106c80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:12.779049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56003f9-7b38-4d60-9cde-1869b4106c80 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:12.799996 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2995/5928] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:12 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:11:16.421121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c732c-6a0a-4356-9500-c080f6e49875 service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:16.423609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-377c732c-6a0a-4356-9500-c080f6e49875 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:16.570538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c732c-6a0a-4356-9500-c080f6e49875 service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:16.582843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d9b25-43bc-44eb-ac9a-7020d9c4bf08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:16.585615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d9b25-43bc-44eb-ac9a-7020d9c4bf08 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:16.586057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d9b25-43bc-44eb-ac9a-7020d9c4bf08 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:16.586807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d9b25-43bc-44eb-ac9a-7020d9c4bf08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:16.590703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-387d9b25-43bc-44eb-ac9a-7020d9c4bf08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:16.774894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d9b25-43bc-44eb-ac9a-7020d9c4bf08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:16.776877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d9b25-43bc-44eb-ac9a-7020d9c4bf08 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:16.809004 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2934/5929] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:11:17.328019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db33e2-66e9-471a-a551-f84baf58b8c9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:17.328612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db33e2-66e9-471a-a551-f84baf58b8c9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:17.328612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db33e2-66e9-471a-a551-f84baf58b8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:17.390222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e4db33e2-66e9-471a-a551-f84baf58b8c9 None None] MFA Rules not processed for user `f91d413b46d44b2ba9257de2f6b54955`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:17.395381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e4db33e2-66e9-471a-a551-f84baf58b8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:17.551990 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2996/5930] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:17.562031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0516cf3-3beb-456d-9932-36a12eee6616 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:17.563155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0516cf3-3beb-456d-9932-36a12eee6616 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:17.563244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0516cf3-3beb-456d-9932-36a12eee6616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:17.588253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d0516cf3-3beb-456d-9932-36a12eee6616 None None] MFA Rules not processed for user `f91d413b46d44b2ba9257de2f6b54955`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:17.589968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d0516cf3-3beb-456d-9932-36a12eee6616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:17.601118 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2935/5931] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:17.817418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db33e2-66e9-471a-a551-f84baf58b8c9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:17.826471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e4db33e2-66e9-471a-a551-f84baf58b8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:18.031842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db33e2-66e9-471a-a551-f84baf58b8c9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:18.045081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fa96f-bfab-4550-91c9-abfed791c3a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:18.045904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fa96f-bfab-4550-91c9-abfed791c3a5 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:18.047077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fa96f-bfab-4550-91c9-abfed791c3a5 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:18.047077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fa96f-bfab-4550-91c9-abfed791c3a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:18.069624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fa96f-bfab-4550-91c9-abfed791c3a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:18.071441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fa96f-bfab-4550-91c9-abfed791c3a5 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:18.088563 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2997/5932] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:11:19.992359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0516cf3-3beb-456d-9932-36a12eee6616 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:19.996481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d0516cf3-3beb-456d-9932-36a12eee6616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:20.072298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fa96f-bfab-4550-91c9-abfed791c3a5 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:20.074132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0f0fa96f-bfab-4550-91c9-abfed791c3a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:20.107435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:20.122744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fa96f-bfab-4550-91c9-abfed791c3a5 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:20.144355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bc5b2-26a6-49f9-804c-659fcb88fbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:20.147230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bc5b2-26a6-49f9-804c-659fcb88fbaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:20.147510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bc5b2-26a6-49f9-804c-659fcb88fbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:20.150179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bc5b2-26a6-49f9-804c-659fcb88fbaa admin admin] PATH_INFO: `/v3/users/f74f251052ee4c9a99db0c43bc00da9f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:20.157621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bc5b2-26a6-49f9-804c-659fcb88fbaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f74f251052ee4c9a99db0c43bc00da9f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:20.162610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0516cf3-3beb-456d-9932-36a12eee6616 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:20.166103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bc5b2-26a6-49f9-804c-659fcb88fbaa admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:20.184938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd067a9e-c08b-4d04-bf1b-d6066dc46bfd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:20.186064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd067a9e-c08b-4d04-bf1b-d6066dc46bfd service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:20.186861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd067a9e-c08b-4d04-bf1b-d6066dc46bfd service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:20.188245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd067a9e-c08b-4d04-bf1b-d6066dc46bfd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:20.190369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bd067a9e-c08b-4d04-bf1b-d6066dc46bfd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:20.194109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:20.210077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd067a9e-c08b-4d04-bf1b-d6066dc46bfd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:20.213892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd067a9e-c08b-4d04-bf1b-d6066dc46bfd service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:20.230139 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2936/5933] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:11:20.313426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0a5bc5b2-26a6-49f9-804c-659fcb88fbaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f74f251052ee4c9a99db0c43bc00da9f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:20.323856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0a5bc5b2-26a6-49f9-804c-659fcb88fbaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f74f251052ee4c9a99db0c43bc00da9f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:20.347927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0a5bc5b2-26a6-49f9-804c-659fcb88fbaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f74f251052ee4c9a99db0c43bc00da9f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:20.349417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0a5bc5b2-26a6-49f9-804c-659fcb88fbaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f74f251052ee4c9a99db0c43bc00da9f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:20.366808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0a5bc5b2-26a6-49f9-804c-659fcb88fbaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f74f251052ee4c9a99db0c43bc00da9f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:20.404475 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2998/5934] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:20 2022] DELETE /identity/v3/users/f74f251052ee4c9a99db0c43bc00da9f => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:11:20.710705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd067a9e-c08b-4d04-bf1b-d6066dc46bfd service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:20.733703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd067a9e-c08b-4d04-bf1b-d6066dc46bfd service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:20.916018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38180f5a-3eb0-4880-837f-0aa6968e6e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:20.918860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38180f5a-3eb0-4880-837f-0aa6968e6e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:20.919377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38180f5a-3eb0-4880-837f-0aa6968e6e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:20.922351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38180f5a-3eb0-4880-837f-0aa6968e6e7d admin admin] PATH_INFO: `/v3/projects/0b5bb0c08a5145369fc64e13d190c988` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:20.930232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38180f5a-3eb0-4880-837f-0aa6968e6e7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b5bb0c08a5145369fc64e13d190c988)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:20.938996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38180f5a-3eb0-4880-837f-0aa6968e6e7d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:21.102182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-38180f5a-3eb0-4880-837f-0aa6968e6e7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b5bb0c08a5145369fc64e13d190c988', 'request_id': 'req-38180f5a-3eb0-4880-837f-0aa6968e6e7d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:21.128641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-38180f5a-3eb0-4880-837f-0aa6968e6e7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b5bb0c08a5145369fc64e13d190c988', 'request_id': 'req-38180f5a-3eb0-4880-837f-0aa6968e6e7d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:21.155516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-38180f5a-3eb0-4880-837f-0aa6968e6e7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b5bb0c08a5145369fc64e13d190c988', 'request_id': 'req-38180f5a-3eb0-4880-837f-0aa6968e6e7d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:21.240238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-38180f5a-3eb0-4880-837f-0aa6968e6e7d admin admin] The token cache is being invalidate because project 0b5bb0c08a5145369fc64e13d190c988 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:21.249492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-38180f5a-3eb0-4880-837f-0aa6968e6e7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:21.254764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-38180f5a-3eb0-4880-837f-0aa6968e6e7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:21.261670 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2937/5935] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:20 2022] DELETE /identity/v3/projects/0b5bb0c08a5145369fc64e13d190c988 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:21.273974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a63a8-35e3-4b92-bb54-00a866655e88 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:21.274630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a63a8-35e3-4b92-bb54-00a866655e88 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:21.274963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a63a8-35e3-4b92-bb54-00a866655e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:21.378347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b75a63a8-35e3-4b92-bb54-00a866655e88 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:21.388190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b75a63a8-35e3-4b92-bb54-00a866655e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:21.608194 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 2999/5936] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:21.618628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6a54d-4f6e-4867-8425-fd6a2765f2eb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:21.619033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6a54d-4f6e-4867-8425-fd6a2765f2eb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:21.619319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6a54d-4f6e-4867-8425-fd6a2765f2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:21.688114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-22b6a54d-4f6e-4867-8425-fd6a2765f2eb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:21.690467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-22b6a54d-4f6e-4867-8425-fd6a2765f2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:21.732657 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2938/5937] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:21.737789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a63a8-35e3-4b92-bb54-00a866655e88 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:21.757838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a63a8-35e3-4b92-bb54-00a866655e88 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:21.770023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122152e-0eb7-4946-bfde-7603f333e8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:21.771203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122152e-0eb7-4946-bfde-7603f333e8c3 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:21.771764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122152e-0eb7-4946-bfde-7603f333e8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:21.772278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122152e-0eb7-4946-bfde-7603f333e8c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:21.773441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122152e-0eb7-4946-bfde-7603f333e8c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:11:21.777015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122152e-0eb7-4946-bfde-7603f333e8c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:21.780945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122152e-0eb7-4946-bfde-7603f333e8c3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:21.803696 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3000/5938] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:11:21.809149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6a54d-4f6e-4867-8425-fd6a2765f2eb None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:21.827437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6a54d-4f6e-4867-8425-fd6a2765f2eb None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:21.834929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c2742-d384-4d17-8aad-38ff2a6faff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:21.836149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c2742-d384-4d17-8aad-38ff2a6faff4 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:21.836727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c2742-d384-4d17-8aad-38ff2a6faff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:21.837210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c2742-d384-4d17-8aad-38ff2a6faff4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:21.838207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c2742-d384-4d17-8aad-38ff2a6faff4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:21.840609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c2742-d384-4d17-8aad-38ff2a6faff4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:21.886361 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2939/5939] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:21 2022] POST /identity/v3/projects => generated 367 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:21.891043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122152e-0eb7-4946-bfde-7603f333e8c3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:21.907910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122152e-0eb7-4946-bfde-7603f333e8c3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:22.098028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ab715-4f82-480f-9f1b-8f8291db5b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:22.098722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ab715-4f82-480f-9f1b-8f8291db5b5a admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:22.098994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ab715-4f82-480f-9f1b-8f8291db5b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:22.099267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ab715-4f82-480f-9f1b-8f8291db5b5a admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:22.099916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ab715-4f82-480f-9f1b-8f8291db5b5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:22.107449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ab715-4f82-480f-9f1b-8f8291db5b5a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:22.185729 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3001/5940] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:21 2022] POST /identity/v3/users => generated 284 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:22.187854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c2742-d384-4d17-8aad-38ff2a6faff4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:22.204692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c2742-d384-4d17-8aad-38ff2a6faff4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:22.219085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82d81f-738f-4070-8a11-62b9e7e0e976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:22.222008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82d81f-738f-4070-8a11-62b9e7e0e976 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:22.225708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82d81f-738f-4070-8a11-62b9e7e0e976 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:22.228381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82d81f-738f-4070-8a11-62b9e7e0e976 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:22.229830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82d81f-738f-4070-8a11-62b9e7e0e976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:22.230973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82d81f-738f-4070-8a11-62b9e7e0e976 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:22.272457 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2940/5941] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:11:22.272926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ab715-4f82-480f-9f1b-8f8291db5b5a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:22.278755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:22.288216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ab715-4f82-480f-9f1b-8f8291db5b5a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:22.304808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53afebad-8044-487d-b168-410a746e9158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:22.305413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53afebad-8044-487d-b168-410a746e9158 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:22.305890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53afebad-8044-487d-b168-410a746e9158 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:22.306343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53afebad-8044-487d-b168-410a746e9158 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:22.307176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53afebad-8044-487d-b168-410a746e9158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:22.308870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53afebad-8044-487d-b168-410a746e9158 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:22.332612 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3002/5942] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:22.336929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82d81f-738f-4070-8a11-62b9e7e0e976 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:22.350920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82d81f-738f-4070-8a11-62b9e7e0e976 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:22.372375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eaff43-5c1a-4f8e-8e30-f3e553756e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:22.373412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eaff43-5c1a-4f8e-8e30-f3e553756e5b admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:22.373878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eaff43-5c1a-4f8e-8e30-f3e553756e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:22.377451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eaff43-5c1a-4f8e-8e30-f3e553756e5b admin admin] PATH_INFO: `/v3/projects/f4ea767db4294419a4df5a3471f50130/users/52ee557ae4fa434f96e162012f66a367/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:22.430478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eaff43-5c1a-4f8e-8e30-f3e553756e5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4ea767db4294419a4df5a3471f50130, user_id=52ee557ae4fa434f96e162012f66a367, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:22.436000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eaff43-5c1a-4f8e-8e30-f3e553756e5b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:22.456106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:22.466781 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2941/5943] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:22 2022] PUT /identity/v3/projects/f4ea767db4294419a4df5a3471f50130/users/52ee557ae4fa434f96e162012f66a367/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:22.480605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006ec95-ab45-44dc-9050-b18db40ea785 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:22.481588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006ec95-ab45-44dc-9050-b18db40ea785 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:22.482121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006ec95-ab45-44dc-9050-b18db40ea785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:22.521345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0006ec95-ab45-44dc-9050-b18db40ea785 None None] MFA Rules not processed for user `52ee557ae4fa434f96e162012f66a367`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:22.524787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0006ec95-ab45-44dc-9050-b18db40ea785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:22.619087 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3003/5944] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:22.623441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eaff43-5c1a-4f8e-8e30-f3e553756e5b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:22.761356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eaff43-5c1a-4f8e-8e30-f3e553756e5b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:22.772626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4e67c-5c2d-4e2e-9f91-d5f460f26091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:22.773547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4e67c-5c2d-4e2e-9f91-d5f460f26091 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:22.774248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4e67c-5c2d-4e2e-9f91-d5f460f26091 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:22.774732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4e67c-5c2d-4e2e-9f91-d5f460f26091 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:22.775778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4e67c-5c2d-4e2e-9f91-d5f460f26091 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:22.779818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4e67c-5c2d-4e2e-9f91-d5f460f26091 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:22.822432 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2942/5945] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:22 2022] POST /identity/v3/projects => generated 367 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:22.827075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006ec95-ab45-44dc-9050-b18db40ea785 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:22.852381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006ec95-ab45-44dc-9050-b18db40ea785 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:22.996262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033790c4-8da9-4326-a835-1b445768741e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:22.997259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033790c4-8da9-4326-a835-1b445768741e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:22.997989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033790c4-8da9-4326-a835-1b445768741e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:22.999014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033790c4-8da9-4326-a835-1b445768741e admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:23.001270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033790c4-8da9-4326-a835-1b445768741e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:23.007955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033790c4-8da9-4326-a835-1b445768741e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:23.059922 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3004/5946] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:22 2022] POST /identity/v3/users => generated 290 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:23.061584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4e67c-5c2d-4e2e-9f91-d5f460f26091 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:23.081625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4e67c-5c2d-4e2e-9f91-d5f460f26091 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:23.098988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126cf2d-1318-43cb-b317-7b232e936ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:23.100019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126cf2d-1318-43cb-b317-7b232e936ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:23.100475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126cf2d-1318-43cb-b317-7b232e936ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:23.100914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126cf2d-1318-43cb-b317-7b232e936ba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:23.101741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8126cf2d-1318-43cb-b317-7b232e936ba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:23.105588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8126cf2d-1318-43cb-b317-7b232e936ba5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:23.123653 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2943/5947] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:23.128543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033790c4-8da9-4326-a835-1b445768741e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:23.144469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033790c4-8da9-4326-a835-1b445768741e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:23.154293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51174c44-8a48-4224-b240-04285393be31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:23.156048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51174c44-8a48-4224-b240-04285393be31 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:23.157244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51174c44-8a48-4224-b240-04285393be31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:23.157486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51174c44-8a48-4224-b240-04285393be31 admin admin] PATH_INFO: `/v3/projects/7707b99285934b36b593374a8970137e/users/b371b8278d14465b86edcacf48c6c8a7/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:23.201963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51174c44-8a48-4224-b240-04285393be31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7707b99285934b36b593374a8970137e, user_id=b371b8278d14465b86edcacf48c6c8a7, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:23.207135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51174c44-8a48-4224-b240-04285393be31 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:23.242412 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3005/5948] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:23 2022] PUT /identity/v3/projects/7707b99285934b36b593374a8970137e/users/b371b8278d14465b86edcacf48c6c8a7/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:23.244037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126cf2d-1318-43cb-b317-7b232e936ba5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:23.266019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126cf2d-1318-43cb-b317-7b232e936ba5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:23.410956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e46b41-e67f-4d10-8a4b-3a30f3631017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:23.413198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e46b41-e67f-4d10-8a4b-3a30f3631017 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:23.413969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e46b41-e67f-4d10-8a4b-3a30f3631017 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:23.422667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e46b41-e67f-4d10-8a4b-3a30f3631017 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:23.430020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e46b41-e67f-4d10-8a4b-3a30f3631017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:23.436645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e46b41-e67f-4d10-8a4b-3a30f3631017 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:23.495116 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2944/5949] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:23.499220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51174c44-8a48-4224-b240-04285393be31 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:23.518757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51174c44-8a48-4224-b240-04285393be31 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:23.535211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a34ab3-e20e-4999-9f6e-8015f6900b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:23.544037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a34ab3-e20e-4999-9f6e-8015f6900b51 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:23.545684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a34ab3-e20e-4999-9f6e-8015f6900b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:23.548887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a34ab3-e20e-4999-9f6e-8015f6900b51 admin admin] PATH_INFO: `/v3/projects/7707b99285934b36b593374a8970137e/users/b371b8278d14465b86edcacf48c6c8a7/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:23.554817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a34ab3-e20e-4999-9f6e-8015f6900b51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7707b99285934b36b593374a8970137e, user_id=b371b8278d14465b86edcacf48c6c8a7, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:23.559970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a34ab3-e20e-4999-9f6e-8015f6900b51 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:23.617532 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3006/5950] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:23 2022] PUT /identity/v3/projects/7707b99285934b36b593374a8970137e/users/b371b8278d14465b86edcacf48c6c8a7/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:23.628774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ce5a8-3d7c-44c8-8ad3-9a7ebce0b951 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:23.629049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ce5a8-3d7c-44c8-8ad3-9a7ebce0b951 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:23.629288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ce5a8-3d7c-44c8-8ad3-9a7ebce0b951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:23.733018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fe0ce5a8-3d7c-44c8-8ad3-9a7ebce0b951 None None] MFA Rules not processed for user `b371b8278d14465b86edcacf48c6c8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:23.736058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fe0ce5a8-3d7c-44c8-8ad3-9a7ebce0b951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:23.978518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:24.013197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a34ab3-e20e-4999-9f6e-8015f6900b51 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:24.029313 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2945/5951] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:24.032018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a34ab3-e20e-4999-9f6e-8015f6900b51 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:24.037899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62863500-216b-469c-922c-3db3b4a4214a tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b371b8278d14465b86edcacf48c6c8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7707b99285934b36b593374a8970137e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:24.039300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62863500-216b-469c-922c-3db3b4a4214a tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:24.039821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62863500-216b-469c-922c-3db3b4a4214a tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:24.040318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62863500-216b-469c-922c-3db3b4a4214a tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] PATH_INFO: `/v3/regions` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:24.041257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62863500-216b-469c-922c-3db3b4a4214a tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:24.043086 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-62863500-216b-469c-922c-3db3b4a4214a tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:11:24.043086 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:11:24.043086 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:11:24.044490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62863500-216b-469c-922c-3db3b4a4214a tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:24.147066 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3007/5952] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:11:24 2022] POST /identity/v3/regions => generated 224 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:24.151443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ce5a8-3d7c-44c8-8ad3-9a7ebce0b951 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:24.236054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ce5a8-3d7c-44c8-8ad3-9a7ebce0b951 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:24.274793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a91474-432b-47aa-8001-0d14c65a8a10 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b371b8278d14465b86edcacf48c6c8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7707b99285934b36b593374a8970137e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:24.276492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a91474-432b-47aa-8001-0d14c65a8a10 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:24.277051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a91474-432b-47aa-8001-0d14c65a8a10 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:24.277483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a91474-432b-47aa-8001-0d14c65a8a10 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] PATH_INFO: `/v3/regions` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:24.282843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a91474-432b-47aa-8001-0d14c65a8a10 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:24.285918 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-99a91474-432b-47aa-8001-0d14c65a8a10 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:11:24.285918 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:11:24.285918 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:11:24.287618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a91474-432b-47aa-8001-0d14c65a8a10 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:24.398831 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2946/5953] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:11:24 2022] POST /identity/v3/regions => generated 224 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:24.399433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62863500-216b-469c-922c-3db3b4a4214a tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:24.412988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62863500-216b-469c-922c-3db3b4a4214a tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:24.433086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ccee7-ab0e-44db-8c93-edf909b52b64 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b371b8278d14465b86edcacf48c6c8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7707b99285934b36b593374a8970137e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:24.434031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ccee7-ab0e-44db-8c93-edf909b52b64 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:24.435421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ccee7-ab0e-44db-8c93-edf909b52b64 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:24.435900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ccee7-ab0e-44db-8c93-edf909b52b64 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] PATH_INFO: `/v3/regions/d806a567-968e-46f8-bbe8-f8af69f82486` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:24.437506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ccee7-ab0e-44db-8c93-edf909b52b64 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorizing `identity:create_region(region_id=d806a567-968e-46f8-bbe8-f8af69f82486)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:24.439245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ccee7-ab0e-44db-8c93-edf909b52b64 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:24.445410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:24.470094 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3008/5954] 10.222.0.17 () {60 vars in 1259 bytes} [Sat Oct 1 21:11:24 2022] PUT /identity/v3/regions/d806a567-968e-46f8-bbe8-f8af69f82486 => generated 232 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:24.474251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a91474-432b-47aa-8001-0d14c65a8a10 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:24.493457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a91474-432b-47aa-8001-0d14c65a8a10 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:24.524993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e314fbc-a02c-4052-b141-e07dc58232bb tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b371b8278d14465b86edcacf48c6c8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7707b99285934b36b593374a8970137e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:24.526370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e314fbc-a02c-4052-b141-e07dc58232bb tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:24.527533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e314fbc-a02c-4052-b141-e07dc58232bb tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:24.528084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e314fbc-a02c-4052-b141-e07dc58232bb tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] PATH_INFO: `/v3/regions/d806a567-968e-46f8-bbe8-f8af69f82486` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:24.528901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e314fbc-a02c-4052-b141-e07dc58232bb tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorizing `identity:delete_region(region_id=d806a567-968e-46f8-bbe8-f8af69f82486)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:24.531786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e314fbc-a02c-4052-b141-e07dc58232bb tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:24.565198 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2947/5955] 10.222.0.17 () {58 vars in 1242 bytes} [Sat Oct 1 21:11:24 2022] DELETE /identity/v3/regions/d806a567-968e-46f8-bbe8-f8af69f82486 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:24.566688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ccee7-ab0e-44db-8c93-edf909b52b64 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:24.580363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ccee7-ab0e-44db-8c93-edf909b52b64 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:24.637169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27226652-f3bd-468e-b1b1-1455234b8ae9 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b371b8278d14465b86edcacf48c6c8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7707b99285934b36b593374a8970137e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:24.637831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27226652-f3bd-468e-b1b1-1455234b8ae9 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:24.638522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27226652-f3bd-468e-b1b1-1455234b8ae9 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:24.639065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27226652-f3bd-468e-b1b1-1455234b8ae9 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] PATH_INFO: `/v3/regions` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:24.641624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27226652-f3bd-468e-b1b1-1455234b8ae9 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:24.643864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27226652-f3bd-468e-b1b1-1455234b8ae9 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:24.694386 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3009/5956] 10.222.0.17 () {60 vars in 1150 bytes} [Sat Oct 1 21:11:24 2022] POST /identity/v3/regions => generated 254 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:11:24.698512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e314fbc-a02c-4052-b141-e07dc58232bb tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:24.719788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e314fbc-a02c-4052-b141-e07dc58232bb tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:24.755579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc90595-6cd7-4738-a3a1-cbff9d4b4bd8 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b371b8278d14465b86edcacf48c6c8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7707b99285934b36b593374a8970137e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:24.756234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc90595-6cd7-4738-a3a1-cbff9d4b4bd8 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:24.756514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc90595-6cd7-4738-a3a1-cbff9d4b4bd8 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:24.756784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc90595-6cd7-4738-a3a1-cbff9d4b4bd8 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] PATH_INFO: `/v3/regions/06e92e0c63b147e582d6246b6ccf8c63` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:24.757584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc90595-6cd7-4738-a3a1-cbff9d4b4bd8 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorizing `identity:update_region(region_id=06e92e0c63b147e582d6246b6ccf8c63)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:24.758620 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-1fc90595-6cd7-4738-a3a1-cbff9d4b4bd8 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:11:24.758620 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:11:24.758620 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:11:24.759614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc90595-6cd7-4738-a3a1-cbff9d4b4bd8 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:24.802000 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2948/5957] 10.222.0.17 () {60 vars in 1250 bytes} [Sat Oct 1 21:11:24 2022] PATCH /identity/v3/regions/06e92e0c63b147e582d6246b6ccf8c63 => generated 253 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:11:24.806622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27226652-f3bd-468e-b1b1-1455234b8ae9 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:24.830525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27226652-f3bd-468e-b1b1-1455234b8ae9 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:24.872731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77002954-037d-4240-8c90-914241287889 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b371b8278d14465b86edcacf48c6c8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7707b99285934b36b593374a8970137e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:24.875071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77002954-037d-4240-8c90-914241287889 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:24.875412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77002954-037d-4240-8c90-914241287889 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:24.877589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77002954-037d-4240-8c90-914241287889 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] PATH_INFO: `/v3/regions/06e92e0c63b147e582d6246b6ccf8c63` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:24.878407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77002954-037d-4240-8c90-914241287889 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorizing `identity:get_region(region_id=06e92e0c63b147e582d6246b6ccf8c63)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:24.880150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77002954-037d-4240-8c90-914241287889 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:24.896509 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3010/5958] 10.222.0.17 () {58 vars in 1227 bytes} [Sat Oct 1 21:11:24 2022] GET /identity/v3/regions/06e92e0c63b147e582d6246b6ccf8c63 => generated 253 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:11:24.900730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc90595-6cd7-4738-a3a1-cbff9d4b4bd8 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:24.920334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc90595-6cd7-4738-a3a1-cbff9d4b4bd8 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:24.938959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8586e-2a96-4edf-a85e-6593861f12c9 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b371b8278d14465b86edcacf48c6c8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7707b99285934b36b593374a8970137e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:24.939651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8586e-2a96-4edf-a85e-6593861f12c9 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:24.940044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8586e-2a96-4edf-a85e-6593861f12c9 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:24.940323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8586e-2a96-4edf-a85e-6593861f12c9 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] PATH_INFO: `/v3/regions/06e92e0c63b147e582d6246b6ccf8c63` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:24.945023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8586e-2a96-4edf-a85e-6593861f12c9 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorizing `identity:delete_region(region_id=06e92e0c63b147e582d6246b6ccf8c63)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:24.947076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8586e-2a96-4edf-a85e-6593861f12c9 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:25.010284 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2949/5959] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:11:24 2022] DELETE /identity/v3/regions/06e92e0c63b147e582d6246b6ccf8c63 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:11:25.015284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77002954-037d-4240-8c90-914241287889 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:25.033348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77002954-037d-4240-8c90-914241287889 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:25.073545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f539ebe-3379-4b8f-a164-6a7b197bbc7e tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b371b8278d14465b86edcacf48c6c8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7707b99285934b36b593374a8970137e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:25.074513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f539ebe-3379-4b8f-a164-6a7b197bbc7e tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:25.075219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f539ebe-3379-4b8f-a164-6a7b197bbc7e tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:25.075408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f539ebe-3379-4b8f-a164-6a7b197bbc7e tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] PATH_INFO: `/v3/regions` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:25.076017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f539ebe-3379-4b8f-a164-6a7b197bbc7e tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:25.080091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f539ebe-3379-4b8f-a164-6a7b197bbc7e tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:25.089276 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3011/5960] 10.222.0.17 () {58 vars in 1128 bytes} [Sat Oct 1 21:11:25 2022] GET /identity/v3/regions => generated 670 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:11:25.093215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8586e-2a96-4edf-a85e-6593861f12c9 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:25.110337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8586e-2a96-4edf-a85e-6593861f12c9 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:25.116925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b888e8-2dc0-4a92-b144-c614ed3c69d2 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b371b8278d14465b86edcacf48c6c8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7707b99285934b36b593374a8970137e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:25.117905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b888e8-2dc0-4a92-b144-c614ed3c69d2 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:25.119413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b888e8-2dc0-4a92-b144-c614ed3c69d2 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:25.119897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b888e8-2dc0-4a92-b144-c614ed3c69d2 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] PATH_INFO: `/v3/regions/06e92e0c63b147e582d6246b6ccf8c63` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:25.120689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b888e8-2dc0-4a92-b144-c614ed3c69d2 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorizing `identity:delete_region(region_id=06e92e0c63b147e582d6246b6ccf8c63)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:25.122607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b888e8-2dc0-4a92-b144-c614ed3c69d2 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:25.141001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-d9b888e8-2dc0-4a92-b144-c614ed3c69d2 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Could not find region: 06e92e0c63b147e582d6246b6ccf8c63.: keystone.exception.RegionNotFound: Could not find region: 06e92e0c63b147e582d6246b6ccf8c63. Oct 01 21:11:25.141001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:11:25.141001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 129, in delete_region Oct 01 21:11:25.141001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ret = self.driver.delete_region(region_id) Oct 01 21:11:25.141001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 152, in delete_region Oct 01 21:11:25.141001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = self._get_region(session, region_id) Oct 01 21:11:25.141001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 103, in _get_region Oct 01 21:11:25.141001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RegionNotFound(region_id=region_id) Oct 01 21:11:25.141001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RegionNotFound: Could not find region: 06e92e0c63b147e582d6246b6ccf8c63. Oct 01 21:11:25.141001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:11:25.141001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:11:25.141001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:11:25.141001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:11:25.141001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:11:25.141001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:11:25.141001 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:11:25.142426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:11:25.142426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:11:25.142426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:11:25.142426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:11:25.142426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:11:25.142426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:11:25.142426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:11:25.142426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/regions.py", line 92, in delete Oct 01 21:11:25.142426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return PROVIDERS.catalog_api.delete_region( Oct 01 21:11:25.142426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:11:25.142426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:11:25.142426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 135, in delete_region Oct 01 21:11:25.142426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RegionNotFound(region_id=region_id) Oct 01 21:11:25.142426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RegionNotFound: Could not find region: 06e92e0c63b147e582d6246b6ccf8c63. Oct 01 21:11:25.142426 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:11:25.146363 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2950/5961] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:11:25 2022] DELETE /identity/v3/regions/06e92e0c63b147e582d6246b6ccf8c63 => generated 112 bytes in 54 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Oct 01 21:11:25.152274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f539ebe-3379-4b8f-a164-6a7b197bbc7e tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:25.170354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f539ebe-3379-4b8f-a164-6a7b197bbc7e tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:25.182769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b83e04-b243-45f7-a412-a46b36bb4207 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b371b8278d14465b86edcacf48c6c8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7707b99285934b36b593374a8970137e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:25.183719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b83e04-b243-45f7-a412-a46b36bb4207 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:25.184192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b83e04-b243-45f7-a412-a46b36bb4207 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:25.184703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b83e04-b243-45f7-a412-a46b36bb4207 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] PATH_INFO: `/v3/regions` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:25.185940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b83e04-b243-45f7-a412-a46b36bb4207 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:25.187439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b83e04-b243-45f7-a412-a46b36bb4207 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:25.196544 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3012/5962] 10.222.0.17 () {58 vars in 1128 bytes} [Sat Oct 1 21:11:25 2022] GET /identity/v3/regions => generated 670 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:11:25.203720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b888e8-2dc0-4a92-b144-c614ed3c69d2 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:25.229822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b888e8-2dc0-4a92-b144-c614ed3c69d2 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:25.244215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ac531-702d-4936-9d34-9c6fdfb3f030 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b371b8278d14465b86edcacf48c6c8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7707b99285934b36b593374a8970137e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:25.245227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ac531-702d-4936-9d34-9c6fdfb3f030 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:25.245973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ac531-702d-4936-9d34-9c6fdfb3f030 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:25.246257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ac531-702d-4936-9d34-9c6fdfb3f030 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] PATH_INFO: `/v3/regions` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:25.249276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ac531-702d-4936-9d34-9c6fdfb3f030 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:25.251238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ac531-702d-4936-9d34-9c6fdfb3f030 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:25.339800 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2951/5963] 10.222.0.17 () {60 vars in 1150 bytes} [Sat Oct 1 21:11:25 2022] POST /identity/v3/regions => generated 254 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:11:25.345843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b83e04-b243-45f7-a412-a46b36bb4207 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:25.357239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b83e04-b243-45f7-a412-a46b36bb4207 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:25.404234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb5fc1-1f42-40b5-aaff-29880c387089 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b371b8278d14465b86edcacf48c6c8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7707b99285934b36b593374a8970137e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:25.406102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb5fc1-1f42-40b5-aaff-29880c387089 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:25.406396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb5fc1-1f42-40b5-aaff-29880c387089 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:25.409115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb5fc1-1f42-40b5-aaff-29880c387089 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] PATH_INFO: `/v3/regions` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:25.411202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb5fc1-1f42-40b5-aaff-29880c387089 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Adding query filter params (parent_region_id=868605ab0e544b2381aea6f67d4cd2ea) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:11:25.413037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb5fc1-1f42-40b5-aaff-29880c387089 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:25.415532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb5fc1-1f42-40b5-aaff-29880c387089 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:25.429886 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3013/5964] 10.222.0.17 () {58 vars in 1227 bytes} [Sat Oct 1 21:11:25 2022] GET /identity/v3/regions?parent_region_id=868605ab0e544b2381aea6f67d4cd2ea => generated 400 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:11:25.434090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ac531-702d-4936-9d34-9c6fdfb3f030 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:25.452135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ac531-702d-4936-9d34-9c6fdfb3f030 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:25.466399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab58b3-9080-4777-9810-c358c46a349c tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b371b8278d14465b86edcacf48c6c8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7707b99285934b36b593374a8970137e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:25.467367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab58b3-9080-4777-9810-c358c46a349c tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:25.468607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab58b3-9080-4777-9810-c358c46a349c tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:25.470121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab58b3-9080-4777-9810-c358c46a349c tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] PATH_INFO: `/v3/regions/be6efa647b2a478c89635d60c2901b94` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:25.470958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab58b3-9080-4777-9810-c358c46a349c tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorizing `identity:delete_region(region_id=be6efa647b2a478c89635d60c2901b94)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:25.474881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab58b3-9080-4777-9810-c358c46a349c tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:25.520392 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2952/5965] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:11:25 2022] DELETE /identity/v3/regions/be6efa647b2a478c89635d60c2901b94 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:25.525217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb5fc1-1f42-40b5-aaff-29880c387089 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:25.549720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb5fc1-1f42-40b5-aaff-29880c387089 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:25.606541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ed837-4f4f-4d77-923e-e7d7db4dd5f7 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b371b8278d14465b86edcacf48c6c8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7707b99285934b36b593374a8970137e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:25.607200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ed837-4f4f-4d77-923e-e7d7db4dd5f7 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:25.607467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ed837-4f4f-4d77-923e-e7d7db4dd5f7 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:25.607747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ed837-4f4f-4d77-923e-e7d7db4dd5f7 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] PATH_INFO: `/v3/regions/8176dcb2bc1a4ff9adf7f18e124768c2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:25.608347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ed837-4f4f-4d77-923e-e7d7db4dd5f7 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorizing `identity:delete_region(region_id=8176dcb2bc1a4ff9adf7f18e124768c2)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:25.610519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ed837-4f4f-4d77-923e-e7d7db4dd5f7 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:25.670685 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3014/5966] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:11:25 2022] DELETE /identity/v3/regions/8176dcb2bc1a4ff9adf7f18e124768c2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:11:25.675236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab58b3-9080-4777-9810-c358c46a349c tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:25.689457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab58b3-9080-4777-9810-c358c46a349c tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:25.738717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0ad0d-2540-4489-8330-8e15a8cf9839 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b371b8278d14465b86edcacf48c6c8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7707b99285934b36b593374a8970137e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:25.739866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0ad0d-2540-4489-8330-8e15a8cf9839 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:25.740346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0ad0d-2540-4489-8330-8e15a8cf9839 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:25.740878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0ad0d-2540-4489-8330-8e15a8cf9839 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] PATH_INFO: `/v3/regions/868605ab0e544b2381aea6f67d4cd2ea` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:25.741669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0ad0d-2540-4489-8330-8e15a8cf9839 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorizing `identity:delete_region(region_id=868605ab0e544b2381aea6f67d4cd2ea)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:25.743782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0ad0d-2540-4489-8330-8e15a8cf9839 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:25.790817 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2953/5967] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:11:25 2022] DELETE /identity/v3/regions/868605ab0e544b2381aea6f67d4cd2ea => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:25.795763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ed837-4f4f-4d77-923e-e7d7db4dd5f7 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:25.951790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ed837-4f4f-4d77-923e-e7d7db4dd5f7 tempest-RegionsTestJSON-1052083703 tempest-RegionsTestJSON-1052083703-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:26.013260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff520f-ec61-455a-ad76-6b2b225fd6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:26.014333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff520f-ec61-455a-ad76-6b2b225fd6c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:26.014866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff520f-ec61-455a-ad76-6b2b225fd6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:26.016101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff520f-ec61-455a-ad76-6b2b225fd6c3 admin admin] PATH_INFO: `/v3/users/52ee557ae4fa434f96e162012f66a367` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:26.022055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff520f-ec61-455a-ad76-6b2b225fd6c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52ee557ae4fa434f96e162012f66a367)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:26.024647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff520f-ec61-455a-ad76-6b2b225fd6c3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:26.062102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:26.130347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b2ff520f-ec61-455a-ad76-6b2b225fd6c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52ee557ae4fa434f96e162012f66a367'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:26.145425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b2ff520f-ec61-455a-ad76-6b2b225fd6c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52ee557ae4fa434f96e162012f66a367'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:26.170726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b2ff520f-ec61-455a-ad76-6b2b225fd6c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52ee557ae4fa434f96e162012f66a367'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:26.171849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b2ff520f-ec61-455a-ad76-6b2b225fd6c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52ee557ae4fa434f96e162012f66a367'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:26.191780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b2ff520f-ec61-455a-ad76-6b2b225fd6c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52ee557ae4fa434f96e162012f66a367'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:26.202410 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3015/5968] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:25 2022] DELETE /identity/v3/users/52ee557ae4fa434f96e162012f66a367 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:26.208185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f726b-636a-444c-a878-1f02072e7692 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:26.208505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f726b-636a-444c-a878-1f02072e7692 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:26.208780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f726b-636a-444c-a878-1f02072e7692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:26.251425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d29f726b-636a-444c-a878-1f02072e7692 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:26.253051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d29f726b-636a-444c-a878-1f02072e7692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:26.289230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=4,keystone:UPDATE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:26.386040 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2954/5969] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:26.398428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff520f-ec61-455a-ad76-6b2b225fd6c3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:26.400243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b2ff520f-ec61-455a-ad76-6b2b225fd6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:26.528367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff520f-ec61-455a-ad76-6b2b225fd6c3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:26.535213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9f15a-8307-46bc-971f-f2d0625f6c58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:26.536718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9f15a-8307-46bc-971f-f2d0625f6c58 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:26.536935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9f15a-8307-46bc-971f-f2d0625f6c58 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:26.538670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9f15a-8307-46bc-971f-f2d0625f6c58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:26.552569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9f15a-8307-46bc-971f-f2d0625f6c58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:26.554406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9f15a-8307-46bc-971f-f2d0625f6c58 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:26.580955 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3016/5970] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:11:27.167693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f726b-636a-444c-a878-1f02072e7692 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:27.171988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d29f726b-636a-444c-a878-1f02072e7692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:27.199667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f726b-636a-444c-a878-1f02072e7692 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:27.213275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb9e1a-98e6-4aa4-9d79-0154cd1444bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:27.214194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb9e1a-98e6-4aa4-9d79-0154cd1444bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:27.214711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb9e1a-98e6-4aa4-9d79-0154cd1444bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:27.215594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb9e1a-98e6-4aa4-9d79-0154cd1444bf admin admin] PATH_INFO: `/v3/projects/f4ea767db4294419a4df5a3471f50130` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:27.219327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb9e1a-98e6-4aa4-9d79-0154cd1444bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4ea767db4294419a4df5a3471f50130)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:27.221512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb9e1a-98e6-4aa4-9d79-0154cd1444bf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:27.291144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-06fb9e1a-98e6-4aa4-9d79-0154cd1444bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4ea767db4294419a4df5a3471f50130', 'request_id': 'req-06fb9e1a-98e6-4aa4-9d79-0154cd1444bf'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:27.301505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-06fb9e1a-98e6-4aa4-9d79-0154cd1444bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4ea767db4294419a4df5a3471f50130', 'request_id': 'req-06fb9e1a-98e6-4aa4-9d79-0154cd1444bf'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:27.310378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-06fb9e1a-98e6-4aa4-9d79-0154cd1444bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4ea767db4294419a4df5a3471f50130', 'request_id': 'req-06fb9e1a-98e6-4aa4-9d79-0154cd1444bf'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:27.326245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-06fb9e1a-98e6-4aa4-9d79-0154cd1444bf admin admin] The token cache is being invalidate because project f4ea767db4294419a4df5a3471f50130 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:27.326913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-06fb9e1a-98e6-4aa4-9d79-0154cd1444bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:27.328949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-06fb9e1a-98e6-4aa4-9d79-0154cd1444bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:27.330623 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2955/5971] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:27 2022] DELETE /identity/v3/projects/f4ea767db4294419a4df5a3471f50130 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:27.334696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9f15a-8307-46bc-971f-f2d0625f6c58 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:27.346784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9f15a-8307-46bc-971f-f2d0625f6c58 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:27.434553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a0a2c-3713-409c-b9c3-cbf053f490f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:27.435355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a0a2c-3713-409c-b9c3-cbf053f490f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:27.435955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a0a2c-3713-409c-b9c3-cbf053f490f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:27.435955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a0a2c-3713-409c-b9c3-cbf053f490f4 admin admin] PATH_INFO: `/v3/users/b371b8278d14465b86edcacf48c6c8a7` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:27.437485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a0a2c-3713-409c-b9c3-cbf053f490f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b371b8278d14465b86edcacf48c6c8a7)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:27.440771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a0a2c-3713-409c-b9c3-cbf053f490f4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:27.484043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:27.509113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5e9a0a2c-3713-409c-b9c3-cbf053f490f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b371b8278d14465b86edcacf48c6c8a7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:27.518422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5e9a0a2c-3713-409c-b9c3-cbf053f490f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b371b8278d14465b86edcacf48c6c8a7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:27.532790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5e9a0a2c-3713-409c-b9c3-cbf053f490f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b371b8278d14465b86edcacf48c6c8a7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:27.533537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5e9a0a2c-3713-409c-b9c3-cbf053f490f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b371b8278d14465b86edcacf48c6c8a7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:27.543218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5e9a0a2c-3713-409c-b9c3-cbf053f490f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b371b8278d14465b86edcacf48c6c8a7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:27.553064 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3017/5972] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:27 2022] DELETE /identity/v3/users/b371b8278d14465b86edcacf48c6c8a7 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:28.272563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb9e1a-98e6-4aa4-9d79-0154cd1444bf admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:28.275515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-06fb9e1a-98e6-4aa4-9d79-0154cd1444bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:28.292587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb9e1a-98e6-4aa4-9d79-0154cd1444bf admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:28.401121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62cbb6-371e-470b-9698-05bc078df3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:28.402636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62cbb6-371e-470b-9698-05bc078df3d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:28.403330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62cbb6-371e-470b-9698-05bc078df3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:28.404522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62cbb6-371e-470b-9698-05bc078df3d8 admin admin] PATH_INFO: `/v3/projects/7707b99285934b36b593374a8970137e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:28.407440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62cbb6-371e-470b-9698-05bc078df3d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7707b99285934b36b593374a8970137e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:28.411109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62cbb6-371e-470b-9698-05bc078df3d8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:28.523111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dd62cbb6-371e-470b-9698-05bc078df3d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7707b99285934b36b593374a8970137e', 'request_id': 'req-dd62cbb6-371e-470b-9698-05bc078df3d8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:28.528021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:28.552494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dd62cbb6-371e-470b-9698-05bc078df3d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7707b99285934b36b593374a8970137e', 'request_id': 'req-dd62cbb6-371e-470b-9698-05bc078df3d8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:28.564902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dd62cbb6-371e-470b-9698-05bc078df3d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7707b99285934b36b593374a8970137e', 'request_id': 'req-dd62cbb6-371e-470b-9698-05bc078df3d8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:28.584845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dd62cbb6-371e-470b-9698-05bc078df3d8 admin admin] The token cache is being invalidate because project 7707b99285934b36b593374a8970137e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:28.585148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dd62cbb6-371e-470b-9698-05bc078df3d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:28.585827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-dd62cbb6-371e-470b-9698-05bc078df3d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:28.591263 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2956/5973] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:28 2022] DELETE /identity/v3/projects/7707b99285934b36b593374a8970137e => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:28.593342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcffd7-d860-432c-a81d-f77f336ae85a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:28.594265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcffd7-d860-432c-a81d-f77f336ae85a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:28.594668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcffd7-d860-432c-a81d-f77f336ae85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:28.622665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5ebcffd7-d860-432c-a81d-f77f336ae85a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:28.624808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5ebcffd7-d860-432c-a81d-f77f336ae85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:28.712788 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3018/5974] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:28.720844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6bbbb-361b-4b20-b4f5-c80aa06398f9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:28.721231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6bbbb-361b-4b20-b4f5-c80aa06398f9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:28.721663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6bbbb-361b-4b20-b4f5-c80aa06398f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:28.746558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-66a6bbbb-361b-4b20-b4f5-c80aa06398f9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:28.748064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-66a6bbbb-361b-4b20-b4f5-c80aa06398f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:28.757932 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2957/5975] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:11:28.763347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebcffd7-d860-432c-a81d-f77f336ae85a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:28.774332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebcffd7-d860-432c-a81d-f77f336ae85a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:28.784958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d06d5-1c74-474d-a842-9a4b930e9f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:28.786032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d06d5-1c74-474d-a842-9a4b930e9f52 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:28.786729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d06d5-1c74-474d-a842-9a4b930e9f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:28.787258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d06d5-1c74-474d-a842-9a4b930e9f52 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:28.788213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d06d5-1c74-474d-a842-9a4b930e9f52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:11:28.788851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d06d5-1c74-474d-a842-9a4b930e9f52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:28.790902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d06d5-1c74-474d-a842-9a4b930e9f52 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:28.817955 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3019/5976] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:11:28.822520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6bbbb-361b-4b20-b4f5-c80aa06398f9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:28.838319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6bbbb-361b-4b20-b4f5-c80aa06398f9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:28.847767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055e796-eefa-49f5-a7f9-ecd53e4c9f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:28.848319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055e796-eefa-49f5-a7f9-ecd53e4c9f32 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:28.848599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055e796-eefa-49f5-a7f9-ecd53e4c9f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:28.848872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055e796-eefa-49f5-a7f9-ecd53e4c9f32 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:28.849505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055e796-eefa-49f5-a7f9-ecd53e4c9f32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:28.851463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055e796-eefa-49f5-a7f9-ecd53e4c9f32 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:28.887369 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2958/5977] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:28 2022] POST /identity/v3/projects => generated 369 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:28.894646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d06d5-1c74-474d-a842-9a4b930e9f52 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:28.914235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d06d5-1c74-474d-a842-9a4b930e9f52 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:28.989942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e49f25-2181-418e-9a74-081ea2ac1aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:28.990607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e49f25-2181-418e-9a74-081ea2ac1aea admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:28.990896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e49f25-2181-418e-9a74-081ea2ac1aea admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:28.991176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e49f25-2181-418e-9a74-081ea2ac1aea admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:28.991861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e49f25-2181-418e-9a74-081ea2ac1aea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:28.994614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e49f25-2181-418e-9a74-081ea2ac1aea admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:29.027177 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3020/5978] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:28 2022] POST /identity/v3/users => generated 285 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:29.027616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055e796-eefa-49f5-a7f9-ecd53e4c9f32 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:29.053615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055e796-eefa-49f5-a7f9-ecd53e4c9f32 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:29.058905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6aefaa-340f-4e75-b3ed-ff451bc3833b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:29.059958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6aefaa-340f-4e75-b3ed-ff451bc3833b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:29.060100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6aefaa-340f-4e75-b3ed-ff451bc3833b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:29.060764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6aefaa-340f-4e75-b3ed-ff451bc3833b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:29.061825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6aefaa-340f-4e75-b3ed-ff451bc3833b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:29.063675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6aefaa-340f-4e75-b3ed-ff451bc3833b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:29.075106 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2959/5979] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:11:29.079494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e49f25-2181-418e-9a74-081ea2ac1aea admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:29.091485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e49f25-2181-418e-9a74-081ea2ac1aea admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:29.103354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d19fe1-fdd6-4d8c-84df-2929a8c0f19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:29.103977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d19fe1-fdd6-4d8c-84df-2929a8c0f19f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:29.104274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d19fe1-fdd6-4d8c-84df-2929a8c0f19f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:29.104762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d19fe1-fdd6-4d8c-84df-2929a8c0f19f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:29.105187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d19fe1-fdd6-4d8c-84df-2929a8c0f19f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:29.107519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d19fe1-fdd6-4d8c-84df-2929a8c0f19f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:29.119710 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3021/5980] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:11:29.123760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6aefaa-340f-4e75-b3ed-ff451bc3833b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:29.135103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6aefaa-340f-4e75-b3ed-ff451bc3833b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:29.141568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2325a-462e-4495-ba4b-9dfd9f30bc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:29.142339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2325a-462e-4495-ba4b-9dfd9f30bc89 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:29.142747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2325a-462e-4495-ba4b-9dfd9f30bc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:29.143170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2325a-462e-4495-ba4b-9dfd9f30bc89 admin admin] PATH_INFO: `/v3/projects/e6232288e7984cacbbb313b328129b77/users/c3b96a4d620442979eef570a5b93dffa/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:29.160532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a2325a-462e-4495-ba4b-9dfd9f30bc89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6232288e7984cacbbb313b328129b77, user_id=c3b96a4d620442979eef570a5b93dffa, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:29.164489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a2325a-462e-4495-ba4b-9dfd9f30bc89 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:29.178854 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2960/5981] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:29 2022] PUT /identity/v3/projects/e6232288e7984cacbbb313b328129b77/users/c3b96a4d620442979eef570a5b93dffa/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:29.189638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55200a2f-b0ad-4265-ae67-7f33e735f445 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:29.189938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55200a2f-b0ad-4265-ae67-7f33e735f445 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:29.190213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55200a2f-b0ad-4265-ae67-7f33e735f445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:29.212675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:29.225666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-55200a2f-b0ad-4265-ae67-7f33e735f445 None None] MFA Rules not processed for user `c3b96a4d620442979eef570a5b93dffa`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:29.227490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-55200a2f-b0ad-4265-ae67-7f33e735f445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:29.293843 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3022/5982] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:11:29.298284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2325a-462e-4495-ba4b-9dfd9f30bc89 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:29.364191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2325a-462e-4495-ba4b-9dfd9f30bc89 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:29.368938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64a750-d6c6-41a8-8883-a1029cff802e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:29.369641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64a750-d6c6-41a8-8883-a1029cff802e admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:29.369921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64a750-d6c6-41a8-8883-a1029cff802e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:29.370192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64a750-d6c6-41a8-8883-a1029cff802e admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:29.370853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64a750-d6c6-41a8-8883-a1029cff802e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:29.372783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64a750-d6c6-41a8-8883-a1029cff802e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:29.400254 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2961/5983] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:29 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:29.404385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55200a2f-b0ad-4265-ae67-7f33e735f445 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:29.415520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55200a2f-b0ad-4265-ae67-7f33e735f445 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:29.485174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3c12a-818a-4b89-bc2c-f9c87b0a8e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:29.486796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3c12a-818a-4b89-bc2c-f9c87b0a8e6b admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:29.487100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3c12a-818a-4b89-bc2c-f9c87b0a8e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:29.487372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3c12a-818a-4b89-bc2c-f9c87b0a8e6b admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:29.488009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3c12a-818a-4b89-bc2c-f9c87b0a8e6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:29.489926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3c12a-818a-4b89-bc2c-f9c87b0a8e6b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:29.519311 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3023/5984] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:29 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:11:29.520981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64a750-d6c6-41a8-8883-a1029cff802e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:29.533364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64a750-d6c6-41a8-8883-a1029cff802e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:29.539304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6349d0-b58b-4967-b452-b06ecfd68544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:29.540137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6349d0-b58b-4967-b452-b06ecfd68544 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:29.542569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6349d0-b58b-4967-b452-b06ecfd68544 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:29.542858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6349d0-b58b-4967-b452-b06ecfd68544 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:29.543512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6349d0-b58b-4967-b452-b06ecfd68544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:29.545322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6349d0-b58b-4967-b452-b06ecfd68544 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:29.563169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3c12a-818a-4b89-bc2c-f9c87b0a8e6b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:29.566655 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2962/5985] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:29.575362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3c12a-818a-4b89-bc2c-f9c87b0a8e6b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:29.580770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c17cb-fdd6-4903-8e12-a92e3b793f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:29.581620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c17cb-fdd6-4903-8e12-a92e3b793f69 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:29.581740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c17cb-fdd6-4903-8e12-a92e3b793f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:29.581963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c17cb-fdd6-4903-8e12-a92e3b793f69 admin admin] PATH_INFO: `/v3/projects/27cdb54c8a6247bbbf0cf6c9d7e7e2a9/users/46664441c62f451b86e802f303196b4e/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:29.602253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c17cb-fdd6-4903-8e12-a92e3b793f69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27cdb54c8a6247bbbf0cf6c9d7e7e2a9, user_id=46664441c62f451b86e802f303196b4e, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:29.605134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c17cb-fdd6-4903-8e12-a92e3b793f69 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:29.646234 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3024/5986] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:29 2022] PUT /identity/v3/projects/27cdb54c8a6247bbbf0cf6c9d7e7e2a9/users/46664441c62f451b86e802f303196b4e/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:29.647451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6349d0-b58b-4967-b452-b06ecfd68544 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:29.657790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6349d0-b58b-4967-b452-b06ecfd68544 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:29.745225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:29.767724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71be477-43eb-432f-95eb-0130d65d0c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:29.767724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71be477-43eb-432f-95eb-0130d65d0c5b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:29.768018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71be477-43eb-432f-95eb-0130d65d0c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:29.768226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71be477-43eb-432f-95eb-0130d65d0c5b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:29.768936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71be477-43eb-432f-95eb-0130d65d0c5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:29.770581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71be477-43eb-432f-95eb-0130d65d0c5b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:29.784022 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2963/5987] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:29.787705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c17cb-fdd6-4903-8e12-a92e3b793f69 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:29.809061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c17cb-fdd6-4903-8e12-a92e3b793f69 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:29.823836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21587ea7-9026-464d-a8ea-91a10651d7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:29.825957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21587ea7-9026-464d-a8ea-91a10651d7c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:29.826683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21587ea7-9026-464d-a8ea-91a10651d7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:29.829338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21587ea7-9026-464d-a8ea-91a10651d7c4 admin admin] PATH_INFO: `/v3/projects/27cdb54c8a6247bbbf0cf6c9d7e7e2a9/users/46664441c62f451b86e802f303196b4e/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:29.834383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21587ea7-9026-464d-a8ea-91a10651d7c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27cdb54c8a6247bbbf0cf6c9d7e7e2a9, user_id=46664441c62f451b86e802f303196b4e, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:29.839306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21587ea7-9026-464d-a8ea-91a10651d7c4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:29.872345 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3025/5988] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:29 2022] PUT /identity/v3/projects/27cdb54c8a6247bbbf0cf6c9d7e7e2a9/users/46664441c62f451b86e802f303196b4e/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:29.884210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf25e13-c776-4835-9319-2b63f368af1e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:29.886197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf25e13-c776-4835-9319-2b63f368af1e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:29.886598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf25e13-c776-4835-9319-2b63f368af1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:29.945852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-cdf25e13-c776-4835-9319-2b63f368af1e None None] MFA Rules not processed for user `46664441c62f451b86e802f303196b4e`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:29.947144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cdf25e13-c776-4835-9319-2b63f368af1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:30.175257 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2964/5989] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:29 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:30.179118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21587ea7-9026-464d-a8ea-91a10651d7c4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:30.196636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21587ea7-9026-464d-a8ea-91a10651d7c4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:30.208713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae3c6e-72b8-45c5-b88f-add77e6fff71 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46664441c62f451b86e802f303196b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27cdb54c8a6247bbbf0cf6c9d7e7e2a9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:30.209900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae3c6e-72b8-45c5-b88f-add77e6fff71 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:30.210408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae3c6e-72b8-45c5-b88f-add77e6fff71 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:30.210985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae3c6e-72b8-45c5-b88f-add77e6fff71 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:30.212392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae3c6e-72b8-45c5-b88f-add77e6fff71 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:30.215577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae3c6e-72b8-45c5-b88f-add77e6fff71 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:30.288818 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3026/5990] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:30 2022] POST /identity/v3/projects => generated 360 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:30.290080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf25e13-c776-4835-9319-2b63f368af1e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:30.305716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf25e13-c776-4835-9319-2b63f368af1e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:30.429606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab28dfa-0ed3-43bc-8926-77d15bfd8203 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46664441c62f451b86e802f303196b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27cdb54c8a6247bbbf0cf6c9d7e7e2a9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:30.430334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab28dfa-0ed3-43bc-8926-77d15bfd8203 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:30.430721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab28dfa-0ed3-43bc-8926-77d15bfd8203 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:30.431791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab28dfa-0ed3-43bc-8926-77d15bfd8203 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:30.433216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab28dfa-0ed3-43bc-8926-77d15bfd8203 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:30.435753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab28dfa-0ed3-43bc-8926-77d15bfd8203 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:30.498508 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2965/5991] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:30 2022] POST /identity/v3/users => generated 373 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:30.502984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae3c6e-72b8-45c5-b88f-add77e6fff71 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:30.521236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae3c6e-72b8-45c5-b88f-add77e6fff71 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:30.530082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafe169-6950-4424-8b02-c7498f0f6452 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46664441c62f451b86e802f303196b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27cdb54c8a6247bbbf0cf6c9d7e7e2a9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:30.531150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafe169-6950-4424-8b02-c7498f0f6452 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:30.531404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafe169-6950-4424-8b02-c7498f0f6452 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:30.531770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafe169-6950-4424-8b02-c7498f0f6452 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:30.532551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafe169-6950-4424-8b02-c7498f0f6452 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:11:30.532844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafe169-6950-4424-8b02-c7498f0f6452 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:30.534032 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-2bafe169-6950-4424-8b02-c7498f0f6452 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:11:30.534032 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:11:30.534032 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:11:30.535205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafe169-6950-4424-8b02-c7498f0f6452 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:30.597770 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3027/5992] 10.222.0.17 () {58 vars in 1157 bytes} [Sat Oct 1 21:11:30 2022] GET /identity/v3/users?domain_id=default => generated 8297 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:30.599427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab28dfa-0ed3-43bc-8926-77d15bfd8203 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:30.614746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab28dfa-0ed3-43bc-8926-77d15bfd8203 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:30.631380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7db71d-2531-4301-a26b-3eb923d76c0d tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46664441c62f451b86e802f303196b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27cdb54c8a6247bbbf0cf6c9d7e7e2a9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:30.632089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7db71d-2531-4301-a26b-3eb923d76c0d tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:30.635123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7db71d-2531-4301-a26b-3eb923d76c0d tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:30.635457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7db71d-2531-4301-a26b-3eb923d76c0d tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] PATH_INFO: `/v3/users/41af177d3f5c4e6e9f826b5b119dcea0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:30.670013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7db71d-2531-4301-a26b-3eb923d76c0d tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: Authorizing `identity:update_user(user_id=41af177d3f5c4e6e9f826b5b119dcea0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:30.674218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7db71d-2531-4301-a26b-3eb923d76c0d tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:30.736471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1b7db71d-2531-4301-a26b-3eb923d76c0d tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '41af177d3f5c4e6e9f826b5b119dcea0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:30.760995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1b7db71d-2531-4301-a26b-3eb923d76c0d tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Invalidating the token cache because user 41af177d3f5c4e6e9f826b5b119dcea0 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:30.761320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1b7db71d-2531-4301-a26b-3eb923d76c0d tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:30.762158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1b7db71d-2531-4301-a26b-3eb923d76c0d tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:30.764340 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2966/5993] 10.222.0.17 () {60 vars in 1244 bytes} [Sat Oct 1 21:11:30 2022] PATCH /identity/v3/users/41af177d3f5c4e6e9f826b5b119dcea0 => generated 488 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:11:30.769906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5578e3e-2749-4b13-9e34-f9c5940f4bbc None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:30.771417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5578e3e-2749-4b13-9e34-f9c5940f4bbc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:30.771417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5578e3e-2749-4b13-9e34-f9c5940f4bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:30.809186 ncdv-master-1 devstack@keystone.service[99139]: WARNING keystone.auth.plugins.core [None req-e5578e3e-2749-4b13-9e34-f9c5940f4bbc None None] User is disabled: 41af177d3f5c4e6e9f826b5b119dcea0: AssertionError: User is disabled: 41af177d3f5c4e6e9f826b5b119dcea0 Oct 01 21:11:30.830763 ncdv-master-1 devstack@keystone.service[99139]: WARNING keystone.server.flask.application [None req-e5578e3e-2749-4b13-9e34-f9c5940f4bbc None None] Authorization failed. The request you have made requires authentication. from 10.222.0.17: keystone.exception.Unauthorized: The request you have made requires authentication. Oct 01 21:11:30.832599 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3028/5994] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:30 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 64 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Oct 01 21:11:30.837067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7db71d-2531-4301-a26b-3eb923d76c0d tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:30.839880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1b7db71d-2531-4301-a26b-3eb923d76c0d tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:30.867135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7db71d-2531-4301-a26b-3eb923d76c0d tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:30.879036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd814e6-aa02-4d14-b9c8-92dcc2f645e8 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46664441c62f451b86e802f303196b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27cdb54c8a6247bbbf0cf6c9d7e7e2a9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:30.879698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd814e6-aa02-4d14-b9c8-92dcc2f645e8 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:30.879981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd814e6-aa02-4d14-b9c8-92dcc2f645e8 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:30.880255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd814e6-aa02-4d14-b9c8-92dcc2f645e8 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] PATH_INFO: `/v3/users/41af177d3f5c4e6e9f826b5b119dcea0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:30.914415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd814e6-aa02-4d14-b9c8-92dcc2f645e8 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: Authorizing `identity:delete_user(user_id=41af177d3f5c4e6e9f826b5b119dcea0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:30.916433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd814e6-aa02-4d14-b9c8-92dcc2f645e8 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:30.924230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:31.006825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8dd814e6-aa02-4d14-b9c8-92dcc2f645e8 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41af177d3f5c4e6e9f826b5b119dcea0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:31.011617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8dd814e6-aa02-4d14-b9c8-92dcc2f645e8 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41af177d3f5c4e6e9f826b5b119dcea0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:31.016397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8dd814e6-aa02-4d14-b9c8-92dcc2f645e8 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41af177d3f5c4e6e9f826b5b119dcea0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:31.053677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8dd814e6-aa02-4d14-b9c8-92dcc2f645e8 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41af177d3f5c4e6e9f826b5b119dcea0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:31.084642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8dd814e6-aa02-4d14-b9c8-92dcc2f645e8 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41af177d3f5c4e6e9f826b5b119dcea0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:31.137754 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2967/5995] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:30 2022] DELETE /identity/v3/users/41af177d3f5c4e6e9f826b5b119dcea0 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:31.139855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5578e3e-2749-4b13-9e34-f9c5940f4bbc None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:31.143370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e5578e3e-2749-4b13-9e34-f9c5940f4bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:31.173238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:31.277850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5578e3e-2749-4b13-9e34-f9c5940f4bbc None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:31.294163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e2d2e-2c92-4cff-a059-0992cae96ed7 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46664441c62f451b86e802f303196b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27cdb54c8a6247bbbf0cf6c9d7e7e2a9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:31.295396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e2d2e-2c92-4cff-a059-0992cae96ed7 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:31.295897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e2d2e-2c92-4cff-a059-0992cae96ed7 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:31.297057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e2d2e-2c92-4cff-a059-0992cae96ed7 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] PATH_INFO: `/v3/projects/2a2e9817101c4d5eab9a36315df0adb7` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:31.298684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e2d2e-2c92-4cff-a059-0992cae96ed7 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2a2e9817101c4d5eab9a36315df0adb7)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:31.302068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e2d2e-2c92-4cff-a059-0992cae96ed7 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:31.379444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-011e2d2e-2c92-4cff-a059-0992cae96ed7 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a2e9817101c4d5eab9a36315df0adb7', 'request_id': 'req-011e2d2e-2c92-4cff-a059-0992cae96ed7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:31.390551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-011e2d2e-2c92-4cff-a059-0992cae96ed7 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a2e9817101c4d5eab9a36315df0adb7', 'request_id': 'req-011e2d2e-2c92-4cff-a059-0992cae96ed7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:31.402636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-011e2d2e-2c92-4cff-a059-0992cae96ed7 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a2e9817101c4d5eab9a36315df0adb7', 'request_id': 'req-011e2d2e-2c92-4cff-a059-0992cae96ed7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:31.417919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-011e2d2e-2c92-4cff-a059-0992cae96ed7 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] The token cache is being invalidate because project 2a2e9817101c4d5eab9a36315df0adb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:31.418287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-011e2d2e-2c92-4cff-a059-0992cae96ed7 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:31.419230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-011e2d2e-2c92-4cff-a059-0992cae96ed7 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:31.425909 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3029/5996] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:31 2022] DELETE /identity/v3/projects/2a2e9817101c4d5eab9a36315df0adb7 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:31.431642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd814e6-aa02-4d14-b9c8-92dcc2f645e8 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:31.443920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd814e6-aa02-4d14-b9c8-92dcc2f645e8 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:31.510513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10600d53-414e-442d-8349-d1602d724a2a tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46664441c62f451b86e802f303196b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27cdb54c8a6247bbbf0cf6c9d7e7e2a9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:31.511247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10600d53-414e-442d-8349-d1602d724a2a tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:31.511543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10600d53-414e-442d-8349-d1602d724a2a tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:31.511820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10600d53-414e-442d-8349-d1602d724a2a tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:31.512462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10600d53-414e-442d-8349-d1602d724a2a tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:31.514417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10600d53-414e-442d-8349-d1602d724a2a tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:31.529742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-10600d53-414e-442d-8349-d1602d724a2a tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Could not find domain: 70a23bb4d3fc4dbf9afa2899b7a7f14e.: keystone.exception.DomainNotFound: Could not find domain: 70a23bb4d3fc4dbf9afa2899b7a7f14e. Oct 01 21:11:31.529742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:11:31.529742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 01 21:11:31.529742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 01 21:11:31.529742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 21:11:31.529742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 21:11:31.529742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 21:11:31.529742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 21:11:31.529742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 70a23bb4d3fc4dbf9afa2899b7a7f14e. Oct 01 21:11:31.529742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:11:31.529742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:11:31.529742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:11:31.529742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:11:31.529742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:11:31.529742 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:11:31.531265 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:11:31.531265 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:11:31.531265 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:11:31.531265 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:11:31.531265 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:11:31.531265 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:11:31.531265 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:11:31.531265 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:11:31.531265 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 01 21:11:31.531265 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 01 21:11:31.531265 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:11:31.531265 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:11:31.531265 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 21:11:31.531265 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 21:11:31.531265 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 21:11:31.531265 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 21:11:31.531265 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 987, in create_user Oct 01 21:11:31.532670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application PROVIDERS.resource_api.get_domain(domain_id) Oct 01 21:11:31.532670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:11:31.532670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:11:31.532670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:11:31.532670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:11:31.532670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:11:31.532670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:11:31.532670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:11:31.532670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 21:11:31.532670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:11:31.532670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:11:31.532670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:11:31.532670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:11:31.532670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:11:31.532670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:11:31.532670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:11:31.532670 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:11:31.533950 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 01 21:11:31.533950 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 21:11:31.533950 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 70a23bb4d3fc4dbf9afa2899b7a7f14e. Oct 01 21:11:31.533950 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:11:31.533950 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2968/5997] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:31 2022] POST /identity/v3/users => generated 112 bytes in 100 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:11:31.535804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e2d2e-2c92-4cff-a059-0992cae96ed7 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:31.537421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-011e2d2e-2c92-4cff-a059-0992cae96ed7 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:31.627431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e2d2e-2c92-4cff-a059-0992cae96ed7 tempest-UsersNegativeTest-2037388105 tempest-UsersNegativeTest-2037388105-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:31.633416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912a6b9-95a2-4932-9806-84d5ade95beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:31.634203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912a6b9-95a2-4932-9806-84d5ade95beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:31.634692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912a6b9-95a2-4932-9806-84d5ade95beb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:31.635080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912a6b9-95a2-4932-9806-84d5ade95beb admin admin] PATH_INFO: `/v3/users/c3b96a4d620442979eef570a5b93dffa` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:31.636402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912a6b9-95a2-4932-9806-84d5ade95beb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3b96a4d620442979eef570a5b93dffa)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:31.638947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912a6b9-95a2-4932-9806-84d5ade95beb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:31.693036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:31.699125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b912a6b9-95a2-4932-9806-84d5ade95beb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3b96a4d620442979eef570a5b93dffa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:31.709753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b912a6b9-95a2-4932-9806-84d5ade95beb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3b96a4d620442979eef570a5b93dffa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:31.723082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b912a6b9-95a2-4932-9806-84d5ade95beb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3b96a4d620442979eef570a5b93dffa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:31.723955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b912a6b9-95a2-4932-9806-84d5ade95beb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3b96a4d620442979eef570a5b93dffa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:31.732638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b912a6b9-95a2-4932-9806-84d5ade95beb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3b96a4d620442979eef570a5b93dffa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:31.739932 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3030/5998] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:31 2022] DELETE /identity/v3/users/c3b96a4d620442979eef570a5b93dffa => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:31.744499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77669de0-f39b-4b8d-961f-2edde62e8ad0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:31.744781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77669de0-f39b-4b8d-961f-2edde62e8ad0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:31.745371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77669de0-f39b-4b8d-961f-2edde62e8ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:31.764614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-77669de0-f39b-4b8d-961f-2edde62e8ad0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:31.765903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-77669de0-f39b-4b8d-961f-2edde62e8ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:31.824038 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2969/5999] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:11:31.833742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912a6b9-95a2-4932-9806-84d5ade95beb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:31.835150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b912a6b9-95a2-4932-9806-84d5ade95beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:31.894063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912a6b9-95a2-4932-9806-84d5ade95beb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:31.897621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a45ba9-400d-45a7-befa-acdbba2c7b67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:31.898224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a45ba9-400d-45a7-befa-acdbba2c7b67 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:31.898528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a45ba9-400d-45a7-befa-acdbba2c7b67 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:31.898807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a45ba9-400d-45a7-befa-acdbba2c7b67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:31.907872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a45ba9-400d-45a7-befa-acdbba2c7b67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:31.909324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a45ba9-400d-45a7-befa-acdbba2c7b67 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:31.919020 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3031/6000] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:11:32.190714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77669de0-f39b-4b8d-961f-2edde62e8ad0 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:32.192136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-77669de0-f39b-4b8d-961f-2edde62e8ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:32.208060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77669de0-f39b-4b8d-961f-2edde62e8ad0 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:32.214030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb402e2-e07d-41c1-893b-e31509aa44c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:32.214750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb402e2-e07d-41c1-893b-e31509aa44c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:32.215067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb402e2-e07d-41c1-893b-e31509aa44c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:32.215224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb402e2-e07d-41c1-893b-e31509aa44c7 admin admin] PATH_INFO: `/v3/projects/e6232288e7984cacbbb313b328129b77` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:32.216485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb402e2-e07d-41c1-893b-e31509aa44c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6232288e7984cacbbb313b328129b77)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:32.218955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb402e2-e07d-41c1-893b-e31509aa44c7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:32.235787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:32.266539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0fb402e2-e07d-41c1-893b-e31509aa44c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6232288e7984cacbbb313b328129b77', 'request_id': 'req-0fb402e2-e07d-41c1-893b-e31509aa44c7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:32.274071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0fb402e2-e07d-41c1-893b-e31509aa44c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6232288e7984cacbbb313b328129b77', 'request_id': 'req-0fb402e2-e07d-41c1-893b-e31509aa44c7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:32.279917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0fb402e2-e07d-41c1-893b-e31509aa44c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6232288e7984cacbbb313b328129b77', 'request_id': 'req-0fb402e2-e07d-41c1-893b-e31509aa44c7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:32.291338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0fb402e2-e07d-41c1-893b-e31509aa44c7 admin admin] The token cache is being invalidate because project e6232288e7984cacbbb313b328129b77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:32.291627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0fb402e2-e07d-41c1-893b-e31509aa44c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:32.292252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0fb402e2-e07d-41c1-893b-e31509aa44c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:32.293705 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2970/6001] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:32 2022] DELETE /identity/v3/projects/e6232288e7984cacbbb313b328129b77 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:32.298185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a45ba9-400d-45a7-befa-acdbba2c7b67 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:32.308408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a45ba9-400d-45a7-befa-acdbba2c7b67 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:32.369359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ea064-5962-4b1b-986d-0b3addbea52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:32.370085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ea064-5962-4b1b-986d-0b3addbea52c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:32.370508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ea064-5962-4b1b-986d-0b3addbea52c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:32.370943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ea064-5962-4b1b-986d-0b3addbea52c admin admin] PATH_INFO: `/v3/users/46664441c62f451b86e802f303196b4e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:32.372081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ea064-5962-4b1b-986d-0b3addbea52c admin admin] RBAC: Authorizing `identity:delete_user(user_id=46664441c62f451b86e802f303196b4e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:32.374206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ea064-5962-4b1b-986d-0b3addbea52c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:32.425114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-af7ea064-5962-4b1b-986d-0b3addbea52c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46664441c62f451b86e802f303196b4e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:32.433731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-af7ea064-5962-4b1b-986d-0b3addbea52c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46664441c62f451b86e802f303196b4e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:32.444657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:32.453456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-af7ea064-5962-4b1b-986d-0b3addbea52c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46664441c62f451b86e802f303196b4e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:32.454757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-af7ea064-5962-4b1b-986d-0b3addbea52c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46664441c62f451b86e802f303196b4e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:32.466022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-af7ea064-5962-4b1b-986d-0b3addbea52c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46664441c62f451b86e802f303196b4e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:32.486435 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3032/6002] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:32 2022] DELETE /identity/v3/users/46664441c62f451b86e802f303196b4e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:11:32.628376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb402e2-e07d-41c1-893b-e31509aa44c7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:32.630712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0fb402e2-e07d-41c1-893b-e31509aa44c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:32.647723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb402e2-e07d-41c1-893b-e31509aa44c7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:32.731780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e441be-a04b-45fc-b8cb-8cc41a45f059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:32.732916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e441be-a04b-45fc-b8cb-8cc41a45f059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:32.733902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e441be-a04b-45fc-b8cb-8cc41a45f059 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:32.734523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e441be-a04b-45fc-b8cb-8cc41a45f059 admin admin] PATH_INFO: `/v3/users/bf57979b45f14ca0a58a89961875f6cd` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:32.737656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e441be-a04b-45fc-b8cb-8cc41a45f059 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf57979b45f14ca0a58a89961875f6cd)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:32.740213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e441be-a04b-45fc-b8cb-8cc41a45f059 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:32.808971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-29e441be-a04b-45fc-b8cb-8cc41a45f059 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf57979b45f14ca0a58a89961875f6cd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:32.809660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-29e441be-a04b-45fc-b8cb-8cc41a45f059 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf57979b45f14ca0a58a89961875f6cd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:32.810277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-29e441be-a04b-45fc-b8cb-8cc41a45f059 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf57979b45f14ca0a58a89961875f6cd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:32.822430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-29e441be-a04b-45fc-b8cb-8cc41a45f059 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf57979b45f14ca0a58a89961875f6cd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:32.854747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ea064-5962-4b1b-986d-0b3addbea52c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:32.856789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-af7ea064-5962-4b1b-986d-0b3addbea52c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:32.872591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-29e441be-a04b-45fc-b8cb-8cc41a45f059 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf57979b45f14ca0a58a89961875f6cd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:32.889551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ea064-5962-4b1b-986d-0b3addbea52c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:32.911463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c02afd-547c-41d2-a50b-59844e1312de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:32.912183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c02afd-547c-41d2-a50b-59844e1312de admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:32.912386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c02afd-547c-41d2-a50b-59844e1312de admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:32.912668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c02afd-547c-41d2-a50b-59844e1312de admin admin] PATH_INFO: `/v3/projects/27cdb54c8a6247bbbf0cf6c9d7e7e2a9` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:32.915140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c02afd-547c-41d2-a50b-59844e1312de admin admin] RBAC: Authorizing `identity:delete_project(project_id=27cdb54c8a6247bbbf0cf6c9d7e7e2a9)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:32.915540 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2971/6003] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:32 2022] DELETE /identity/v3/users/bf57979b45f14ca0a58a89961875f6cd => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:32.917574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c02afd-547c-41d2-a50b-59844e1312de admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:32.917702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a64083-90d8-4403-bf69-999ca35ab662 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:32.917971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a64083-90d8-4403-bf69-999ca35ab662 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:32.918309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a64083-90d8-4403-bf69-999ca35ab662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:32.954351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-36a64083-90d8-4403-bf69-999ca35ab662 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:32.955699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-36a64083-90d8-4403-bf69-999ca35ab662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:32.989881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e6c02afd-547c-41d2-a50b-59844e1312de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27cdb54c8a6247bbbf0cf6c9d7e7e2a9', 'request_id': 'req-e6c02afd-547c-41d2-a50b-59844e1312de'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:32.991908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:33.001889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e6c02afd-547c-41d2-a50b-59844e1312de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27cdb54c8a6247bbbf0cf6c9d7e7e2a9', 'request_id': 'req-e6c02afd-547c-41d2-a50b-59844e1312de'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:33.014341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e6c02afd-547c-41d2-a50b-59844e1312de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27cdb54c8a6247bbbf0cf6c9d7e7e2a9', 'request_id': 'req-e6c02afd-547c-41d2-a50b-59844e1312de'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:33.032836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e6c02afd-547c-41d2-a50b-59844e1312de admin admin] The token cache is being invalidate because project 27cdb54c8a6247bbbf0cf6c9d7e7e2a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:33.033177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e6c02afd-547c-41d2-a50b-59844e1312de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:33.034097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e6c02afd-547c-41d2-a50b-59844e1312de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:33.036481 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3033/6004] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:32 2022] DELETE /identity/v3/projects/27cdb54c8a6247bbbf0cf6c9d7e7e2a9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:33.043803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ff896-80d0-431c-8e84-2dc852396a3f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:33.044203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ff896-80d0-431c-8e84-2dc852396a3f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:33.044576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ff896-80d0-431c-8e84-2dc852396a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:33.082773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-182ff896-80d0-431c-8e84-2dc852396a3f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:33.084047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-182ff896-80d0-431c-8e84-2dc852396a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:33.087413 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2972/6005] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:33.097347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a64083-90d8-4403-bf69-999ca35ab662 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:33.099393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-36a64083-90d8-4403-bf69-999ca35ab662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:33.166900 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3034/6006] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:33.177113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3afe88-adfb-4432-82c9-d296eff400fc None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:33.177406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3afe88-adfb-4432-82c9-d296eff400fc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:33.177680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3afe88-adfb-4432-82c9-d296eff400fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:33.196505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a64083-90d8-4403-bf69-999ca35ab662 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:33.198859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7d3afe88-adfb-4432-82c9-d296eff400fc None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:33.200067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7d3afe88-adfb-4432-82c9-d296eff400fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:33.210581 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3035/6007] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:33.215271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3afe88-adfb-4432-82c9-d296eff400fc None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:33.230724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3afe88-adfb-4432-82c9-d296eff400fc None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:33.246610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e490396-8740-4bbb-810e-7e549c87853b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:33.247229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e490396-8740-4bbb-810e-7e549c87853b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:33.247510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e490396-8740-4bbb-810e-7e549c87853b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:33.247775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e490396-8740-4bbb-810e-7e549c87853b admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:33.249297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e490396-8740-4bbb-810e-7e549c87853b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:11:33.249297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e490396-8740-4bbb-810e-7e549c87853b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:33.252197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e490396-8740-4bbb-810e-7e549c87853b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:33.268975 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3036/6008] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:11:33.273807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e490396-8740-4bbb-810e-7e549c87853b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:33.278625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d490ac8-4386-4b98-a724-eb79c653f8f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:33.279883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d490ac8-4386-4b98-a724-eb79c653f8f6 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:33.279883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d490ac8-4386-4b98-a724-eb79c653f8f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:33.279883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d490ac8-4386-4b98-a724-eb79c653f8f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:33.300219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d490ac8-4386-4b98-a724-eb79c653f8f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:33.302075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d490ac8-4386-4b98-a724-eb79c653f8f6 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:33.306615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e490396-8740-4bbb-810e-7e549c87853b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:33.317073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07072771-484a-464c-ab3e-4b07b26da401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:33.317709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07072771-484a-464c-ab3e-4b07b26da401 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:33.318183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07072771-484a-464c-ab3e-4b07b26da401 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:33.318426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07072771-484a-464c-ab3e-4b07b26da401 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:33.319214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07072771-484a-464c-ab3e-4b07b26da401 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:33.320312 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2973/6009] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:11:33.322112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07072771-484a-464c-ab3e-4b07b26da401 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:33.363033 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3037/6010] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:33 2022] POST /identity/v3/projects => generated 377 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:33.366629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d490ac8-4386-4b98-a724-eb79c653f8f6 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:33.384248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d490ac8-4386-4b98-a724-eb79c653f8f6 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:33.485693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16859571-a189-4d00-9112-fd615414b3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:33.486784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16859571-a189-4d00-9112-fd615414b3da admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:33.486784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16859571-a189-4d00-9112-fd615414b3da admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:33.487023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16859571-a189-4d00-9112-fd615414b3da admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:33.487878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16859571-a189-4d00-9112-fd615414b3da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:33.489743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16859571-a189-4d00-9112-fd615414b3da admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:33.520289 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2974/6011] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:33 2022] POST /identity/v3/users => generated 295 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:33.521597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07072771-484a-464c-ab3e-4b07b26da401 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:33.552590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07072771-484a-464c-ab3e-4b07b26da401 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:33.558652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa84757a-7a7c-4182-bb9e-7d932705b239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:33.559293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84757a-7a7c-4182-bb9e-7d932705b239 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:33.559554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84757a-7a7c-4182-bb9e-7d932705b239 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:33.559886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84757a-7a7c-4182-bb9e-7d932705b239 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:33.560544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa84757a-7a7c-4182-bb9e-7d932705b239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:33.562246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa84757a-7a7c-4182-bb9e-7d932705b239 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:33.572994 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3038/6012] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:33.577699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16859571-a189-4d00-9112-fd615414b3da admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:33.588178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16859571-a189-4d00-9112-fd615414b3da admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:33.593511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d932a-bfb0-45f2-9221-dc112d3a27c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:33.594169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d932a-bfb0-45f2-9221-dc112d3a27c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:33.594460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d932a-bfb0-45f2-9221-dc112d3a27c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:33.594841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d932a-bfb0-45f2-9221-dc112d3a27c2 admin admin] PATH_INFO: `/v3/projects/131de40e35214e14888471984a615940/users/ea4deb791cab44a1964af31ae0c26f5b/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:33.606189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:33.619723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d932a-bfb0-45f2-9221-dc112d3a27c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=131de40e35214e14888471984a615940, user_id=ea4deb791cab44a1964af31ae0c26f5b, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:33.622364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d932a-bfb0-45f2-9221-dc112d3a27c2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:33.638237 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2975/6013] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:33 2022] PUT /identity/v3/projects/131de40e35214e14888471984a615940/users/ea4deb791cab44a1964af31ae0c26f5b/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:33.643093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa84757a-7a7c-4182-bb9e-7d932705b239 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:33.653285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa84757a-7a7c-4182-bb9e-7d932705b239 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:33.662918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:33.691337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d932a-bfb0-45f2-9221-dc112d3a27c2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:33.693129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b83d932a-bfb0-45f2-9221-dc112d3a27c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:33.727464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5f232-5783-47fc-aff6-d3127f2b49ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:33.728079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5f232-5783-47fc-aff6-d3127f2b49ef admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:33.728363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5f232-5783-47fc-aff6-d3127f2b49ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:33.728637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5f232-5783-47fc-aff6-d3127f2b49ef admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:33.729280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5f232-5783-47fc-aff6-d3127f2b49ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:33.730974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5f232-5783-47fc-aff6-d3127f2b49ef admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:33.742855 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3039/6014] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:33.747319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5f232-5783-47fc-aff6-d3127f2b49ef admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:33.757770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5f232-5783-47fc-aff6-d3127f2b49ef admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:33.762527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e99b88-d72f-4779-a116-1d94d4829452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:33.763163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e99b88-d72f-4779-a116-1d94d4829452 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:33.763443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e99b88-d72f-4779-a116-1d94d4829452 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:33.764232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e99b88-d72f-4779-a116-1d94d4829452 admin admin] PATH_INFO: `/v3/projects/131de40e35214e14888471984a615940/users/ea4deb791cab44a1964af31ae0c26f5b/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:33.766271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e99b88-d72f-4779-a116-1d94d4829452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=131de40e35214e14888471984a615940, user_id=ea4deb791cab44a1964af31ae0c26f5b, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:33.768095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d932a-bfb0-45f2-9221-dc112d3a27c2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:33.768901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e99b88-d72f-4779-a116-1d94d4829452 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:33.772802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98484a70-5a47-4bed-8425-3435b7149582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:33.773426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98484a70-5a47-4bed-8425-3435b7149582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:33.773703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98484a70-5a47-4bed-8425-3435b7149582 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:33.773978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98484a70-5a47-4bed-8425-3435b7149582 admin admin] PATH_INFO: `/v3/projects/06940ad486584ae391c5891858e8af7a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:33.775131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98484a70-5a47-4bed-8425-3435b7149582 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06940ad486584ae391c5891858e8af7a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:33.777028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98484a70-5a47-4bed-8425-3435b7149582 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:33.781228 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3040/6015] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:33 2022] PUT /identity/v3/projects/131de40e35214e14888471984a615940/users/ea4deb791cab44a1964af31ae0c26f5b/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:33.791254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf541f-724b-485a-bb3f-bdcfe1fcdea3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:33.791540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf541f-724b-485a-bb3f-bdcfe1fcdea3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:33.791813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf541f-724b-485a-bb3f-bdcfe1fcdea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:33.822266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0ccf541f-724b-485a-bb3f-bdcfe1fcdea3 None None] MFA Rules not processed for user `ea4deb791cab44a1964af31ae0c26f5b`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:33.823536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0ccf541f-724b-485a-bb3f-bdcfe1fcdea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:33.828106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-98484a70-5a47-4bed-8425-3435b7149582 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06940ad486584ae391c5891858e8af7a', 'request_id': 'req-98484a70-5a47-4bed-8425-3435b7149582'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:33.837021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-98484a70-5a47-4bed-8425-3435b7149582 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06940ad486584ae391c5891858e8af7a', 'request_id': 'req-98484a70-5a47-4bed-8425-3435b7149582'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:33.843020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-98484a70-5a47-4bed-8425-3435b7149582 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06940ad486584ae391c5891858e8af7a', 'request_id': 'req-98484a70-5a47-4bed-8425-3435b7149582'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:33.858442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-98484a70-5a47-4bed-8425-3435b7149582 admin admin] The token cache is being invalidate because project 06940ad486584ae391c5891858e8af7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:33.858801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-98484a70-5a47-4bed-8425-3435b7149582 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:33.859449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-98484a70-5a47-4bed-8425-3435b7149582 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:33.860995 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2976/6016] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:33 2022] DELETE /identity/v3/projects/06940ad486584ae391c5891858e8af7a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:11:33.864782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98484a70-5a47-4bed-8425-3435b7149582 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:33.866073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-98484a70-5a47-4bed-8425-3435b7149582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:33.901949 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3041/6017] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:11:33.906360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf541f-724b-485a-bb3f-bdcfe1fcdea3 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:33.907795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0ccf541f-724b-485a-bb3f-bdcfe1fcdea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:33.934300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98484a70-5a47-4bed-8425-3435b7149582 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:33.940077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dcbc1b-de1d-4baf-8604-ba2eeab75836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:33.940742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcbc1b-de1d-4baf-8604-ba2eeab75836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:33.941052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcbc1b-de1d-4baf-8604-ba2eeab75836 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:33.941587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcbc1b-de1d-4baf-8604-ba2eeab75836 admin admin] PATH_INFO: `/v3/users/c03f4441acb3476bb03057521692d8dd` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:33.943446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dcbc1b-de1d-4baf-8604-ba2eeab75836 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c03f4441acb3476bb03057521692d8dd)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:33.945346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dcbc1b-de1d-4baf-8604-ba2eeab75836 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:33.969965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:33.980860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf541f-724b-485a-bb3f-bdcfe1fcdea3 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:33.987000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61aab6-9dc1-4599-9922-9764a49e41fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:33.988170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61aab6-9dc1-4599-9922-9764a49e41fa admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:33.988715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61aab6-9dc1-4599-9922-9764a49e41fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:33.989173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61aab6-9dc1-4599-9922-9764a49e41fa admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:33.989919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61aab6-9dc1-4599-9922-9764a49e41fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:33.991962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61aab6-9dc1-4599-9922-9764a49e41fa admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:34.003737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e1dcbc1b-de1d-4baf-8604-ba2eeab75836 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c03f4441acb3476bb03057521692d8dd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:34.004422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e1dcbc1b-de1d-4baf-8604-ba2eeab75836 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c03f4441acb3476bb03057521692d8dd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:34.005042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e1dcbc1b-de1d-4baf-8604-ba2eeab75836 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c03f4441acb3476bb03057521692d8dd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:34.014795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e1dcbc1b-de1d-4baf-8604-ba2eeab75836 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c03f4441acb3476bb03057521692d8dd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:34.025161 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3042/6018] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:33 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:11:34.026548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61aab6-9dc1-4599-9922-9764a49e41fa admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:34.030708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e1dcbc1b-de1d-4baf-8604-ba2eeab75836 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c03f4441acb3476bb03057521692d8dd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:34.032924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2f61aab6-9dc1-4599-9922-9764a49e41fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:34.044938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:34.047199 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2977/6019] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:33 2022] DELETE /identity/v3/users/c03f4441acb3476bb03057521692d8dd => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:11:34.132317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61aab6-9dc1-4599-9922-9764a49e41fa admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:34.220825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26df38-b8ec-4cee-91b3-217340b92f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:34.222274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26df38-b8ec-4cee-91b3-217340b92f50 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:34.222274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26df38-b8ec-4cee-91b3-217340b92f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:34.222484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26df38-b8ec-4cee-91b3-217340b92f50 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:34.222995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26df38-b8ec-4cee-91b3-217340b92f50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:34.225606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26df38-b8ec-4cee-91b3-217340b92f50 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:34.254659 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3043/6020] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:34 2022] POST /identity/v3/users => generated 290 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:34.257127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dcbc1b-de1d-4baf-8604-ba2eeab75836 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:34.270792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dcbc1b-de1d-4baf-8604-ba2eeab75836 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:34.275510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a777eb5-8f4c-4507-9078-b87d91b1814f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:34.276190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a777eb5-8f4c-4507-9078-b87d91b1814f admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:34.276484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a777eb5-8f4c-4507-9078-b87d91b1814f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:34.276729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a777eb5-8f4c-4507-9078-b87d91b1814f admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:34.277411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a777eb5-8f4c-4507-9078-b87d91b1814f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:34.279041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a777eb5-8f4c-4507-9078-b87d91b1814f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:34.290415 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2978/6021] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:34.294387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26df38-b8ec-4cee-91b3-217340b92f50 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:34.308628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26df38-b8ec-4cee-91b3-217340b92f50 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:34.314339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef53bd-6eae-4310-a3ce-6662a894f1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:34.314985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef53bd-6eae-4310-a3ce-6662a894f1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:34.315264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef53bd-6eae-4310-a3ce-6662a894f1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:34.315572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef53bd-6eae-4310-a3ce-6662a894f1d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:34.316230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef53bd-6eae-4310-a3ce-6662a894f1d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:34.317814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef53bd-6eae-4310-a3ce-6662a894f1d5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:34.332845 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3044/6022] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:34.334039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a777eb5-8f4c-4507-9078-b87d91b1814f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:34.347926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a777eb5-8f4c-4507-9078-b87d91b1814f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:34.353666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc94e17-585b-42ab-8608-7b37a7c517af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:34.354334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc94e17-585b-42ab-8608-7b37a7c517af admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:34.354789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc94e17-585b-42ab-8608-7b37a7c517af admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:34.355121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc94e17-585b-42ab-8608-7b37a7c517af admin admin] PATH_INFO: `/v3/projects/8c9fbb0cb7b947e091db66edf82dda2a/users/a11a5739bf6b4f778be7d8ebccae72b1/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:34.367444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef53bd-6eae-4310-a3ce-6662a894f1d5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:34.369347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3cef53bd-6eae-4310-a3ce-6662a894f1d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:34.376828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc94e17-585b-42ab-8608-7b37a7c517af admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c9fbb0cb7b947e091db66edf82dda2a, user_id=a11a5739bf6b4f778be7d8ebccae72b1, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:34.380160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc94e17-585b-42ab-8608-7b37a7c517af admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:34.389368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef53bd-6eae-4310-a3ce-6662a894f1d5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:34.399124 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2979/6023] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:34 2022] PUT /identity/v3/projects/8c9fbb0cb7b947e091db66edf82dda2a/users/a11a5739bf6b4f778be7d8ebccae72b1/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:34.410032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9b07d-b3f9-4e1a-9d6b-a2d2e4fc3eec None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:34.410383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9b07d-b3f9-4e1a-9d6b-a2d2e4fc3eec None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:34.410698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9b07d-b3f9-4e1a-9d6b-a2d2e4fc3eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:34.441846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8fe9b07d-b3f9-4e1a-9d6b-a2d2e4fc3eec None None] MFA Rules not processed for user `a11a5739bf6b4f778be7d8ebccae72b1`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:34.443316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8fe9b07d-b3f9-4e1a-9d6b-a2d2e4fc3eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:34.460171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce1a8f-9cfb-4890-8a5a-db13f3709810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:34.460810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce1a8f-9cfb-4890-8a5a-db13f3709810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:34.461097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce1a8f-9cfb-4890-8a5a-db13f3709810 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:34.461375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce1a8f-9cfb-4890-8a5a-db13f3709810 admin admin] PATH_INFO: `/v3/projects/1a5366898e7d496fbbaf6c1f1b932ba2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:34.462737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce1a8f-9cfb-4890-8a5a-db13f3709810 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a5366898e7d496fbbaf6c1f1b932ba2)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:34.464698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce1a8f-9cfb-4890-8a5a-db13f3709810 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:34.506667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:34.515997 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2980/6024] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:11:34.528944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9b07d-b3f9-4e1a-9d6b-a2d2e4fc3eec None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:34.530202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-45ce1a8f-9cfb-4890-8a5a-db13f3709810 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a5366898e7d496fbbaf6c1f1b932ba2', 'request_id': 'req-45ce1a8f-9cfb-4890-8a5a-db13f3709810'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:34.530567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8fe9b07d-b3f9-4e1a-9d6b-a2d2e4fc3eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:34.539613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-45ce1a8f-9cfb-4890-8a5a-db13f3709810 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a5366898e7d496fbbaf6c1f1b932ba2', 'request_id': 'req-45ce1a8f-9cfb-4890-8a5a-db13f3709810'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:34.549039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-45ce1a8f-9cfb-4890-8a5a-db13f3709810 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a5366898e7d496fbbaf6c1f1b932ba2', 'request_id': 'req-45ce1a8f-9cfb-4890-8a5a-db13f3709810'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:34.571528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-45ce1a8f-9cfb-4890-8a5a-db13f3709810 admin admin] The token cache is being invalidate because project 1a5366898e7d496fbbaf6c1f1b932ba2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:34.574166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-45ce1a8f-9cfb-4890-8a5a-db13f3709810 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:34.575037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-45ce1a8f-9cfb-4890-8a5a-db13f3709810 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:34.577819 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3045/6025] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:34 2022] DELETE /identity/v3/projects/1a5366898e7d496fbbaf6c1f1b932ba2 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:11:34.582555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a9f0e-6311-4265-8fc6-65c8ed285b0a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:34.582980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a9f0e-6311-4265-8fc6-65c8ed285b0a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:34.583254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a9f0e-6311-4265-8fc6-65c8ed285b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:34.610340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-cb2a9f0e-6311-4265-8fc6-65c8ed285b0a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:34.611977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cb2a9f0e-6311-4265-8fc6-65c8ed285b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:34.637918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:34.690834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9b07d-b3f9-4e1a-9d6b-a2d2e4fc3eec None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:34.728005 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3046/6026] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:34.729858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1112776-012d-44f6-8bcd-861479f2e67d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:34.730413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1112776-012d-44f6-8bcd-861479f2e67d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:34.730824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1112776-012d-44f6-8bcd-861479f2e67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:34.759195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d1112776-012d-44f6-8bcd-861479f2e67d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:34.760536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d1112776-012d-44f6-8bcd-861479f2e67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:34.772297 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3047/6027] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:11:34.776915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1112776-012d-44f6-8bcd-861479f2e67d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:34.795363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1112776-012d-44f6-8bcd-861479f2e67d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:34.802800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07a23f-d9ab-4384-ab5c-9d01f6e691b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:34.803609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07a23f-d9ab-4384-ab5c-9d01f6e691b7 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:34.804268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07a23f-d9ab-4384-ab5c-9d01f6e691b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:34.804607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07a23f-d9ab-4384-ab5c-9d01f6e691b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:34.805301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07a23f-d9ab-4384-ab5c-9d01f6e691b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:11:34.805618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07a23f-d9ab-4384-ab5c-9d01f6e691b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:34.807703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07a23f-d9ab-4384-ab5c-9d01f6e691b7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:34.811137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b5838-0c99-40ef-9df9-0f6f0ab727ed service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:34.812852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b5838-0c99-40ef-9df9-0f6f0ab727ed service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:34.812978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b5838-0c99-40ef-9df9-0f6f0ab727ed service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:34.813530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b5838-0c99-40ef-9df9-0f6f0ab727ed service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:34.815406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2f7b5838-0c99-40ef-9df9-0f6f0ab727ed service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:34.825304 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3048/6028] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:11:34.826201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07a23f-d9ab-4384-ab5c-9d01f6e691b7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:34.837844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07a23f-d9ab-4384-ab5c-9d01f6e691b7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:34.845944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2da78f-0f21-43dd-894a-bd9bbe2c8c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:34.846899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2da78f-0f21-43dd-894a-bd9bbe2c8c2d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:34.847218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2da78f-0f21-43dd-894a-bd9bbe2c8c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:34.847512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2da78f-0f21-43dd-894a-bd9bbe2c8c2d admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:34.848200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2da78f-0f21-43dd-894a-bd9bbe2c8c2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:34.850536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2da78f-0f21-43dd-894a-bd9bbe2c8c2d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:34.903857 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3049/6029] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:34 2022] POST /identity/v3/projects => generated 365 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:34.905406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2da78f-0f21-43dd-894a-bd9bbe2c8c2d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:34.916751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b5838-0c99-40ef-9df9-0f6f0ab727ed service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:34.918201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b5838-0c99-40ef-9df9-0f6f0ab727ed service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:34.931002 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2981/6030] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:11:35.013530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2da78f-0f21-43dd-894a-bd9bbe2c8c2d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:35.020889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf1bb5-f3d2-4fc1-805e-1b6a5af81dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:35.022598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf1bb5-f3d2-4fc1-805e-1b6a5af81dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:35.022901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf1bb5-f3d2-4fc1-805e-1b6a5af81dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:35.023188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf1bb5-f3d2-4fc1-805e-1b6a5af81dc3 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:35.023834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf1bb5-f3d2-4fc1-805e-1b6a5af81dc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:35.026011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf1bb5-f3d2-4fc1-805e-1b6a5af81dc3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:35.068628 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3050/6031] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:34 2022] POST /identity/v3/users => generated 283 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:35.073553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b5838-0c99-40ef-9df9-0f6f0ab727ed service glance] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:35.090107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b5838-0c99-40ef-9df9-0f6f0ab727ed service glance] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:35.097687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51accfb3-3d1c-4891-8c37-018f6e89082b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:35.097874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51accfb3-3d1c-4891-8c37-018f6e89082b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:35.098166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51accfb3-3d1c-4891-8c37-018f6e89082b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:35.098446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51accfb3-3d1c-4891-8c37-018f6e89082b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:35.099119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51accfb3-3d1c-4891-8c37-018f6e89082b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:35.101911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51accfb3-3d1c-4891-8c37-018f6e89082b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:35.118648 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2982/6032] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:11:35.122221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf1bb5-f3d2-4fc1-805e-1b6a5af81dc3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:35.139055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf1bb5-f3d2-4fc1-805e-1b6a5af81dc3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:35.144703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fd26a-8380-4dab-a4f7-7b6dbb658d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:35.145626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fd26a-8380-4dab-a4f7-7b6dbb658d54 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:35.145776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fd26a-8380-4dab-a4f7-7b6dbb658d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:35.146011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fd26a-8380-4dab-a4f7-7b6dbb658d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:35.146790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fd26a-8380-4dab-a4f7-7b6dbb658d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:35.148557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fd26a-8380-4dab-a4f7-7b6dbb658d54 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:35.155806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:35.163130 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3051/6033] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:35.167347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51accfb3-3d1c-4891-8c37-018f6e89082b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:35.180704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51accfb3-3d1c-4891-8c37-018f6e89082b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:35.187656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054ea95-b4d6-47ce-acaa-11ab343f292e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:35.188496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054ea95-b4d6-47ce-acaa-11ab343f292e admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:35.188842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054ea95-b4d6-47ce-acaa-11ab343f292e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:35.189185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054ea95-b4d6-47ce-acaa-11ab343f292e admin admin] PATH_INFO: `/v3/projects/2e8942500e3c4be7ae908a64552336f9/users/280a7a3c459a4aefa587c577ae177c61/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:35.211911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054ea95-b4d6-47ce-acaa-11ab343f292e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e8942500e3c4be7ae908a64552336f9, user_id=280a7a3c459a4aefa587c577ae177c61, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:35.216445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054ea95-b4d6-47ce-acaa-11ab343f292e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:35.234895 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2983/6034] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:35 2022] PUT /identity/v3/projects/2e8942500e3c4be7ae908a64552336f9/users/280a7a3c459a4aefa587c577ae177c61/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:35.246258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7af855-5cbf-41d6-a94d-4a41776e0648 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:35.246729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7af855-5cbf-41d6-a94d-4a41776e0648 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:35.247120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7af855-5cbf-41d6-a94d-4a41776e0648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:35.285508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8d7af855-5cbf-41d6-a94d-4a41776e0648 None None] MFA Rules not processed for user `280a7a3c459a4aefa587c577ae177c61`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:35.287080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8d7af855-5cbf-41d6-a94d-4a41776e0648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:35.395030 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3052/6035] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:35.399572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054ea95-b4d6-47ce-acaa-11ab343f292e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:35.468470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054ea95-b4d6-47ce-acaa-11ab343f292e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:35.473243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b8e5b-7a28-4de9-bf83-348625f75e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:35.473848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b8e5b-7a28-4de9-bf83-348625f75e03 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:35.474124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b8e5b-7a28-4de9-bf83-348625f75e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:35.474431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b8e5b-7a28-4de9-bf83-348625f75e03 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:35.475225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b8e5b-7a28-4de9-bf83-348625f75e03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:35.477344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b8e5b-7a28-4de9-bf83-348625f75e03 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:35.487271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:35.516946 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2984/6036] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:35 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:35.518028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7af855-5cbf-41d6-a94d-4a41776e0648 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:35.529515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7af855-5cbf-41d6-a94d-4a41776e0648 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:35.593537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abf7c6-2638-4afa-8ae6-82dab39deec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:35.594154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abf7c6-2638-4afa-8ae6-82dab39deec7 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:35.594438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abf7c6-2638-4afa-8ae6-82dab39deec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:35.594754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abf7c6-2638-4afa-8ae6-82dab39deec7 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:35.595385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8abf7c6-2638-4afa-8ae6-82dab39deec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:35.597297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8abf7c6-2638-4afa-8ae6-82dab39deec7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:35.625404 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3053/6037] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:35 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:11:35.629442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b8e5b-7a28-4de9-bf83-348625f75e03 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:35.648881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b8e5b-7a28-4de9-bf83-348625f75e03 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:35.653393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abf7c6-2638-4afa-8ae6-82dab39deec7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:35.653834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b720672-46cb-47fc-8e4b-bcba1a2bb3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:35.654445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b720672-46cb-47fc-8e4b-bcba1a2bb3c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:35.654763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b720672-46cb-47fc-8e4b-bcba1a2bb3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:35.655041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b720672-46cb-47fc-8e4b-bcba1a2bb3c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:35.656159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f8abf7c6-2638-4afa-8ae6-82dab39deec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:35.656327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b720672-46cb-47fc-8e4b-bcba1a2bb3c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:35.657978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b720672-46cb-47fc-8e4b-bcba1a2bb3c7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:35.669761 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2985/6038] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:35.670907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abf7c6-2638-4afa-8ae6-82dab39deec7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:35.673795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b720672-46cb-47fc-8e4b-bcba1a2bb3c7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:35.677060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d844981-4da6-404c-aa04-a149dbc4015b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:35.677705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d844981-4da6-404c-aa04-a149dbc4015b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:35.677987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d844981-4da6-404c-aa04-a149dbc4015b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:35.678409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d844981-4da6-404c-aa04-a149dbc4015b admin admin] PATH_INFO: `/v3/users/ea4deb791cab44a1964af31ae0c26f5b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:35.679999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d844981-4da6-404c-aa04-a149dbc4015b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea4deb791cab44a1964af31ae0c26f5b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:35.682239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d844981-4da6-404c-aa04-a149dbc4015b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:35.684697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b720672-46cb-47fc-8e4b-bcba1a2bb3c7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:35.690597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402747a-e5bd-4378-9104-afd64cdd7325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:35.691249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402747a-e5bd-4378-9104-afd64cdd7325 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:35.691799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402747a-e5bd-4378-9104-afd64cdd7325 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:35.691799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402747a-e5bd-4378-9104-afd64cdd7325 admin admin] PATH_INFO: `/v3/projects/65f2e592ee3b420e9cb0e93cc2ccb9cb/users/130bbc81ba9548cc8b09adb13cc7e7aa/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:35.708880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402747a-e5bd-4378-9104-afd64cdd7325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65f2e592ee3b420e9cb0e93cc2ccb9cb, user_id=130bbc81ba9548cc8b09adb13cc7e7aa, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:35.711715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402747a-e5bd-4378-9104-afd64cdd7325 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:35.733324 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2986/6039] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:35 2022] PUT /identity/v3/projects/65f2e592ee3b420e9cb0e93cc2ccb9cb/users/130bbc81ba9548cc8b09adb13cc7e7aa/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:11:35.737245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402747a-e5bd-4378-9104-afd64cdd7325 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:35.741617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2d844981-4da6-404c-aa04-a149dbc4015b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea4deb791cab44a1964af31ae0c26f5b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:35.751306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2d844981-4da6-404c-aa04-a149dbc4015b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea4deb791cab44a1964af31ae0c26f5b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:35.766708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2d844981-4da6-404c-aa04-a149dbc4015b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea4deb791cab44a1964af31ae0c26f5b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:35.767575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2d844981-4da6-404c-aa04-a149dbc4015b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea4deb791cab44a1964af31ae0c26f5b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:35.769354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:35.778184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2d844981-4da6-404c-aa04-a149dbc4015b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea4deb791cab44a1964af31ae0c26f5b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:35.786554 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3054/6040] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:35 2022] DELETE /identity/v3/users/ea4deb791cab44a1964af31ae0c26f5b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:35.791438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad6c72-f972-4fe6-958d-9c537149ea5b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:35.791732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad6c72-f972-4fe6-958d-9c537149ea5b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:35.792017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad6c72-f972-4fe6-958d-9c537149ea5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:35.814483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402747a-e5bd-4378-9104-afd64cdd7325 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:35.815170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9fad6c72-f972-4fe6-958d-9c537149ea5b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:35.816386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9fad6c72-f972-4fe6-958d-9c537149ea5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:35.885323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40ee27-71fe-4811-a389-387a848fd040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:35.886252 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3055/6041] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:11:35.886402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40ee27-71fe-4811-a389-387a848fd040 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:35.886402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40ee27-71fe-4811-a389-387a848fd040 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:35.886654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40ee27-71fe-4811-a389-387a848fd040 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:35.887308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40ee27-71fe-4811-a389-387a848fd040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:35.889077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40ee27-71fe-4811-a389-387a848fd040 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:35.898418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad6c72-f972-4fe6-958d-9c537149ea5b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:35.900034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9fad6c72-f972-4fe6-958d-9c537149ea5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:35.901638 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2987/6042] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:11:35.905516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40ee27-71fe-4811-a389-387a848fd040 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:35.907371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-de40ee27-71fe-4811-a389-387a848fd040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:35.920273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40ee27-71fe-4811-a389-387a848fd040 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:35.927000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aecb67e-3300-4070-913c-c34da1e35602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:35.927760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aecb67e-3300-4070-913c-c34da1e35602 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:35.928064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aecb67e-3300-4070-913c-c34da1e35602 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:35.928473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aecb67e-3300-4070-913c-c34da1e35602 admin admin] PATH_INFO: `/v3/projects/65f2e592ee3b420e9cb0e93cc2ccb9cb/users/130bbc81ba9548cc8b09adb13cc7e7aa/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:35.930560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aecb67e-3300-4070-913c-c34da1e35602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65f2e592ee3b420e9cb0e93cc2ccb9cb, user_id=130bbc81ba9548cc8b09adb13cc7e7aa, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:35.933241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aecb67e-3300-4070-913c-c34da1e35602 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:35.945593 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2988/6043] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:35 2022] PUT /identity/v3/projects/65f2e592ee3b420e9cb0e93cc2ccb9cb/users/130bbc81ba9548cc8b09adb13cc7e7aa/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:11:35.962071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76613a-b7cb-44f1-9c2f-145d07bed619 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:35.962363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76613a-b7cb-44f1-9c2f-145d07bed619 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:35.962761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76613a-b7cb-44f1-9c2f-145d07bed619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:35.975348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad6c72-f972-4fe6-958d-9c537149ea5b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:35.996889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ba76613a-b7cb-44f1-9c2f-145d07bed619 None None] MFA Rules not processed for user `130bbc81ba9548cc8b09adb13cc7e7aa`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:35.998655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ba76613a-b7cb-44f1-9c2f-145d07bed619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:36.034241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8089c1b-9757-442a-b73f-fa13a9e30dc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:36.034903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8089c1b-9757-442a-b73f-fa13a9e30dc7 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:36.035206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8089c1b-9757-442a-b73f-fa13a9e30dc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:36.035488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8089c1b-9757-442a-b73f-fa13a9e30dc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:36.036278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:36.078745 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2989/6044] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:11:36.090971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76613a-b7cb-44f1-9c2f-145d07bed619 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:36.092461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ba76613a-b7cb-44f1-9c2f-145d07bed619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:36.106456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8089c1b-9757-442a-b73f-fa13a9e30dc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:36.107967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8089c1b-9757-442a-b73f-fa13a9e30dc7 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:36.111985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:36.119295 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3056/6045] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:11:36.169005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76613a-b7cb-44f1-9c2f-145d07bed619 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:36.172824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c13cde-c242-4c2c-bb14-b10c3048bfdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:36.173436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c13cde-c242-4c2c-bb14-b10c3048bfdf service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:36.173720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c13cde-c242-4c2c-bb14-b10c3048bfdf service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:36.174001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c13cde-c242-4c2c-bb14-b10c3048bfdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:36.188337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c13cde-c242-4c2c-bb14-b10c3048bfdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:36.189797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c13cde-c242-4c2c-bb14-b10c3048bfdf service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:36.200613 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2990/6046] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:36 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:11:36.285913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8089c1b-9757-442a-b73f-fa13a9e30dc7 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:36.298936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8089c1b-9757-442a-b73f-fa13a9e30dc7 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:36.305674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22569d1e-f6ab-41a1-984b-a26a110c2e6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:36.306706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22569d1e-f6ab-41a1-984b-a26a110c2e6c service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:36.306894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22569d1e-f6ab-41a1-984b-a26a110c2e6c service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:36.307205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22569d1e-f6ab-41a1-984b-a26a110c2e6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:36.309087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-22569d1e-f6ab-41a1-984b-a26a110c2e6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:36.384042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22569d1e-f6ab-41a1-984b-a26a110c2e6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:36.385674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22569d1e-f6ab-41a1-984b-a26a110c2e6c service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:36.398226 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3057/6047] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:11:36.442727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c13cde-c242-4c2c-bb14-b10c3048bfdf service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:36.444925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d3c13cde-c242-4c2c-bb14-b10c3048bfdf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:36.466472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c13cde-c242-4c2c-bb14-b10c3048bfdf service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:36.473521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6ccd4-d3bb-46ff-a148-97556cd399f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:36.474246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6ccd4-d3bb-46ff-a148-97556cd399f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:36.474640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6ccd4-d3bb-46ff-a148-97556cd399f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:36.474936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6ccd4-d3bb-46ff-a148-97556cd399f8 admin admin] PATH_INFO: `/v3/projects/131de40e35214e14888471984a615940` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:36.476220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e6ccd4-d3bb-46ff-a148-97556cd399f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=131de40e35214e14888471984a615940)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:36.479703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e6ccd4-d3bb-46ff-a148-97556cd399f8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:36.559107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-17e6ccd4-d3bb-46ff-a148-97556cd399f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '131de40e35214e14888471984a615940', 'request_id': 'req-17e6ccd4-d3bb-46ff-a148-97556cd399f8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:36.576667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-17e6ccd4-d3bb-46ff-a148-97556cd399f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '131de40e35214e14888471984a615940', 'request_id': 'req-17e6ccd4-d3bb-46ff-a148-97556cd399f8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:36.588331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-17e6ccd4-d3bb-46ff-a148-97556cd399f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '131de40e35214e14888471984a615940', 'request_id': 'req-17e6ccd4-d3bb-46ff-a148-97556cd399f8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:36.615054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-17e6ccd4-d3bb-46ff-a148-97556cd399f8 admin admin] The token cache is being invalidate because project 131de40e35214e14888471984a615940 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:36.615353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-17e6ccd4-d3bb-46ff-a148-97556cd399f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:36.616078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-17e6ccd4-d3bb-46ff-a148-97556cd399f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:36.617560 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2991/6048] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:36 2022] DELETE /identity/v3/projects/131de40e35214e14888471984a615940 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:36.624966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22569d1e-f6ab-41a1-984b-a26a110c2e6c service cinder] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:36.636748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22569d1e-f6ab-41a1-984b-a26a110c2e6c service cinder] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:36.698981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972fb4d-b167-42ca-985e-01b2f6067a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:36.699624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972fb4d-b167-42ca-985e-01b2f6067a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:36.699913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972fb4d-b167-42ca-985e-01b2f6067a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:36.700191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972fb4d-b167-42ca-985e-01b2f6067a27 admin admin] PATH_INFO: `/v3/users/a11a5739bf6b4f778be7d8ebccae72b1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:36.701539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972fb4d-b167-42ca-985e-01b2f6067a27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a11a5739bf6b4f778be7d8ebccae72b1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:36.703546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972fb4d-b167-42ca-985e-01b2f6067a27 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:36.755050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4972fb4d-b167-42ca-985e-01b2f6067a27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a11a5739bf6b4f778be7d8ebccae72b1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:36.762832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4972fb4d-b167-42ca-985e-01b2f6067a27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a11a5739bf6b4f778be7d8ebccae72b1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:36.775372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4972fb4d-b167-42ca-985e-01b2f6067a27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a11a5739bf6b4f778be7d8ebccae72b1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:36.776239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4972fb4d-b167-42ca-985e-01b2f6067a27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a11a5739bf6b4f778be7d8ebccae72b1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:36.784561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4972fb4d-b167-42ca-985e-01b2f6067a27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a11a5739bf6b4f778be7d8ebccae72b1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:36.790845 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3058/6049] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:36 2022] DELETE /identity/v3/users/a11a5739bf6b4f778be7d8ebccae72b1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:11:37.350356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6ccd4-d3bb-46ff-a148-97556cd399f8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:37.352825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-17e6ccd4-d3bb-46ff-a148-97556cd399f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:37.399750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6ccd4-d3bb-46ff-a148-97556cd399f8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:37.472391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:37.515393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ba925-33fa-4a08-8d30-a2168eb3d087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:37.517675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ba925-33fa-4a08-8d30-a2168eb3d087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:37.517675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ba925-33fa-4a08-8d30-a2168eb3d087 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:37.518078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ba925-33fa-4a08-8d30-a2168eb3d087 admin admin] PATH_INFO: `/v3/projects/8c9fbb0cb7b947e091db66edf82dda2a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:37.520030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ba925-33fa-4a08-8d30-a2168eb3d087 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c9fbb0cb7b947e091db66edf82dda2a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:37.522430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ba925-33fa-4a08-8d30-a2168eb3d087 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:37.533815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d09a5-9508-4fe2-9c0f-3460321c6446 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:37.534356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d09a5-9508-4fe2-9c0f-3460321c6446 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:37.534609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d09a5-9508-4fe2-9c0f-3460321c6446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:37.570717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b28d09a5-9508-4fe2-9c0f-3460321c6446 None None] MFA Rules not processed for user `f91d413b46d44b2ba9257de2f6b54955`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:37.573023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b28d09a5-9508-4fe2-9c0f-3460321c6446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:37.591952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:37.650692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7f1ba925-33fa-4a08-8d30-a2168eb3d087 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c9fbb0cb7b947e091db66edf82dda2a', 'request_id': 'req-7f1ba925-33fa-4a08-8d30-a2168eb3d087'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:37.672193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7f1ba925-33fa-4a08-8d30-a2168eb3d087 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c9fbb0cb7b947e091db66edf82dda2a', 'request_id': 'req-7f1ba925-33fa-4a08-8d30-a2168eb3d087'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:37.683229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7f1ba925-33fa-4a08-8d30-a2168eb3d087 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c9fbb0cb7b947e091db66edf82dda2a', 'request_id': 'req-7f1ba925-33fa-4a08-8d30-a2168eb3d087'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:37.699599 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3059/6050] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:11:37.712260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7f1ba925-33fa-4a08-8d30-a2168eb3d087 admin admin] The token cache is being invalidate because project 8c9fbb0cb7b947e091db66edf82dda2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:37.712570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7f1ba925-33fa-4a08-8d30-a2168eb3d087 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:37.713356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7f1ba925-33fa-4a08-8d30-a2168eb3d087 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:37.722935 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2992/6051] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:37 2022] DELETE /identity/v3/projects/8c9fbb0cb7b947e091db66edf82dda2a => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:37.726951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf36a3-8c77-4bf4-a61b-5868a284fe4c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:37.727346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf36a3-8c77-4bf4-a61b-5868a284fe4c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:37.727923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf36a3-8c77-4bf4-a61b-5868a284fe4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:37.759555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6ecf36a3-8c77-4bf4-a61b-5868a284fe4c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:37.761174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6ecf36a3-8c77-4bf4-a61b-5868a284fe4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:37.876887 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3060/6052] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:11:37.885787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab05af4-972d-47db-9cf1-17f98ca7f890 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:37.886103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab05af4-972d-47db-9cf1-17f98ca7f890 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:37.886451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab05af4-972d-47db-9cf1-17f98ca7f890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:37.929481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5ab05af4-972d-47db-9cf1-17f98ca7f890 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:37.931268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5ab05af4-972d-47db-9cf1-17f98ca7f890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:37.945210 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2993/6053] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:37.949904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf36a3-8c77-4bf4-a61b-5868a284fe4c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:37.966831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf36a3-8c77-4bf4-a61b-5868a284fe4c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:37.991372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae82556-7b60-49a6-9730-6e70bebdb7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:37.994593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae82556-7b60-49a6-9730-6e70bebdb7ba admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:37.994968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae82556-7b60-49a6-9730-6e70bebdb7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:37.995226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae82556-7b60-49a6-9730-6e70bebdb7ba admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:38.006590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae82556-7b60-49a6-9730-6e70bebdb7ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:11:38.007023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae82556-7b60-49a6-9730-6e70bebdb7ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:38.010719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae82556-7b60-49a6-9730-6e70bebdb7ba admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:38.056695 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3061/6054] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:11:38.057641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab05af4-972d-47db-9cf1-17f98ca7f890 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:38.071864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab05af4-972d-47db-9cf1-17f98ca7f890 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:38.080475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21334235-3f5f-49bd-964e-ed71d7b94de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:38.083789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21334235-3f5f-49bd-964e-ed71d7b94de3 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:38.084004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21334235-3f5f-49bd-964e-ed71d7b94de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:38.084277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21334235-3f5f-49bd-964e-ed71d7b94de3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:38.089163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21334235-3f5f-49bd-964e-ed71d7b94de3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:38.091583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21334235-3f5f-49bd-964e-ed71d7b94de3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:38.147253 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2994/6055] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:38 2022] POST /identity/v3/projects => generated 383 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:38.152125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae82556-7b60-49a6-9730-6e70bebdb7ba admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:38.167084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae82556-7b60-49a6-9730-6e70bebdb7ba admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:38.350417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565c0a0-487b-4cf3-b533-024c4c7bd599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:38.351318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565c0a0-487b-4cf3-b533-024c4c7bd599 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:38.351431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565c0a0-487b-4cf3-b533-024c4c7bd599 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:38.351614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565c0a0-487b-4cf3-b533-024c4c7bd599 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:38.352471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565c0a0-487b-4cf3-b533-024c4c7bd599 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:38.354454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565c0a0-487b-4cf3-b533-024c4c7bd599 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:38.409143 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3062/6056] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:38 2022] POST /identity/v3/users => generated 292 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:38.423690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21334235-3f5f-49bd-964e-ed71d7b94de3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:38.437628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21334235-3f5f-49bd-964e-ed71d7b94de3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:38.444071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043a754-fa3c-4035-af20-95a01502f12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:38.444974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043a754-fa3c-4035-af20-95a01502f12b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:38.445260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043a754-fa3c-4035-af20-95a01502f12b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:38.446141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043a754-fa3c-4035-af20-95a01502f12b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:38.447210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043a754-fa3c-4035-af20-95a01502f12b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:38.455146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043a754-fa3c-4035-af20-95a01502f12b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:38.478047 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2995/6057] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:38.479389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565c0a0-487b-4cf3-b533-024c4c7bd599 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:38.494850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565c0a0-487b-4cf3-b533-024c4c7bd599 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:38.507428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f4431-3ba0-4d4e-af56-f81c98ee067e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:38.508168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f4431-3ba0-4d4e-af56-f81c98ee067e admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:38.508436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f4431-3ba0-4d4e-af56-f81c98ee067e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:38.508710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f4431-3ba0-4d4e-af56-f81c98ee067e admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:38.509356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f4431-3ba0-4d4e-af56-f81c98ee067e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:38.511031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f4431-3ba0-4d4e-af56-f81c98ee067e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:38.528103 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3063/6058] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:38.530900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043a754-fa3c-4035-af20-95a01502f12b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:38.551183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043a754-fa3c-4035-af20-95a01502f12b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:38.559373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e76f2-ebdf-4f54-8dd3-fcf76c4a23c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:38.560124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e76f2-ebdf-4f54-8dd3-fcf76c4a23c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:38.560405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e76f2-ebdf-4f54-8dd3-fcf76c4a23c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:38.560682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e76f2-ebdf-4f54-8dd3-fcf76c4a23c0 admin admin] PATH_INFO: `/v3/projects/26e628d0f18541c6abc1e8614bce827e/users/f64843a62c46411e9d8dbcba77215cf2/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:38.600705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e76f2-ebdf-4f54-8dd3-fcf76c4a23c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26e628d0f18541c6abc1e8614bce827e, user_id=f64843a62c46411e9d8dbcba77215cf2, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:38.603524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e76f2-ebdf-4f54-8dd3-fcf76c4a23c0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:38.629983 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2996/6059] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:38 2022] PUT /identity/v3/projects/26e628d0f18541c6abc1e8614bce827e/users/f64843a62c46411e9d8dbcba77215cf2/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:38.641034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f0149-3870-41e7-ba93-099b85a68541 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:38.641529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f0149-3870-41e7-ba93-099b85a68541 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:38.641620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f0149-3870-41e7-ba93-099b85a68541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:38.682455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-713f0149-3870-41e7-ba93-099b85a68541 None None] MFA Rules not processed for user `f64843a62c46411e9d8dbcba77215cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:38.685581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-713f0149-3870-41e7-ba93-099b85a68541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:38.714689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:38.803065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e76f2-ebdf-4f54-8dd3-fcf76c4a23c0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:38.805746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2f2e76f2-ebdf-4f54-8dd3-fcf76c4a23c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:38.820208 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3064/6060] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:38.824755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f0149-3870-41e7-ba93-099b85a68541 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:38.913258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f0149-3870-41e7-ba93-099b85a68541 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:38.928868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4b95e-5d7d-45b0-88dc-ab17123dbca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:38.930730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4b95e-5d7d-45b0-88dc-ab17123dbca1 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:38.931041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4b95e-5d7d-45b0-88dc-ab17123dbca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:38.931337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4b95e-5d7d-45b0-88dc-ab17123dbca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:38.934325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4b95e-5d7d-45b0-88dc-ab17123dbca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:38.941174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4b95e-5d7d-45b0-88dc-ab17123dbca1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:38.948743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:38.954058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e76f2-ebdf-4f54-8dd3-fcf76c4a23c0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:38.962753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813fb48-9dca-4a11-9c36-1f5935a938e1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:38.963954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813fb48-9dca-4a11-9c36-1f5935a938e1 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:38.965595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813fb48-9dca-4a11-9c36-1f5935a938e1 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:38.966987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813fb48-9dca-4a11-9c36-1f5935a938e1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:38.970706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7813fb48-9dca-4a11-9c36-1f5935a938e1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:38.996663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7813fb48-9dca-4a11-9c36-1f5935a938e1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:39.000605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7813fb48-9dca-4a11-9c36-1f5935a938e1 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:39.012862 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3065/6061] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:38 2022] POST /identity/v3/projects => generated 385 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:11:39.018014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4b95e-5d7d-45b0-88dc-ab17123dbca1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:39.027518 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2997/6062] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:11:39.141904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4b95e-5d7d-45b0-88dc-ab17123dbca1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:39.153821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5041b65-f53a-4fed-87e8-b8312620473e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:39.156664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5041b65-f53a-4fed-87e8-b8312620473e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:39.157185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5041b65-f53a-4fed-87e8-b8312620473e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:39.159852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5041b65-f53a-4fed-87e8-b8312620473e admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:39.161673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5041b65-f53a-4fed-87e8-b8312620473e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:39.167449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5041b65-f53a-4fed-87e8-b8312620473e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:39.223055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813fb48-9dca-4a11-9c36-1f5935a938e1 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:39.229297 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3066/6063] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:39 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:39.240004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813fb48-9dca-4a11-9c36-1f5935a938e1 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:39.248491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413908d-1e3e-4e05-ae0c-0dbdcdd35b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:39.251843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413908d-1e3e-4e05-ae0c-0dbdcdd35b5e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:39.251843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413908d-1e3e-4e05-ae0c-0dbdcdd35b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:39.252122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413908d-1e3e-4e05-ae0c-0dbdcdd35b5e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:39.253551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413908d-1e3e-4e05-ae0c-0dbdcdd35b5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:39.260712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413908d-1e3e-4e05-ae0c-0dbdcdd35b5e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:39.294530 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2998/6064] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:11:39.298006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5041b65-f53a-4fed-87e8-b8312620473e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:39.311900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5041b65-f53a-4fed-87e8-b8312620473e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:39.323852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41892924-3901-433d-a5f9-aaecb56ee5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:39.324489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41892924-3901-433d-a5f9-aaecb56ee5bd admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:39.324823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41892924-3901-433d-a5f9-aaecb56ee5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:39.325148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41892924-3901-433d-a5f9-aaecb56ee5bd admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:39.327589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41892924-3901-433d-a5f9-aaecb56ee5bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:39.339386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41892924-3901-433d-a5f9-aaecb56ee5bd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:39.367510 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3067/6065] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:11:39.372736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413908d-1e3e-4e05-ae0c-0dbdcdd35b5e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:39.389034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413908d-1e3e-4e05-ae0c-0dbdcdd35b5e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:39.400505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f234a-02f6-4396-a414-997492f4f5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:39.401674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f234a-02f6-4396-a414-997492f4f5f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:39.402357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f234a-02f6-4396-a414-997492f4f5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:39.403351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f234a-02f6-4396-a414-997492f4f5f7 admin admin] PATH_INFO: `/v3/projects/71d9714d046a44ccaacc94040fae53db/users/1c286400f0bb491a840bd003249bc731/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:39.430666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f234a-02f6-4396-a414-997492f4f5f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71d9714d046a44ccaacc94040fae53db, user_id=1c286400f0bb491a840bd003249bc731, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:39.434453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f234a-02f6-4396-a414-997492f4f5f7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:39.464086 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 2999/6066] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:39 2022] PUT /identity/v3/projects/71d9714d046a44ccaacc94040fae53db/users/1c286400f0bb491a840bd003249bc731/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:11:39.477031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46d082-d6ea-4183-995d-9c1bd1c15a83 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:39.478305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46d082-d6ea-4183-995d-9c1bd1c15a83 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:39.480155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46d082-d6ea-4183-995d-9c1bd1c15a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:39.562317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:39.581131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-df46d082-d6ea-4183-995d-9c1bd1c15a83 None None] MFA Rules not processed for user `1c286400f0bb491a840bd003249bc731`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:39.583057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-df46d082-d6ea-4183-995d-9c1bd1c15a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:39.809701 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3068/6067] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:39.837759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f234a-02f6-4396-a414-997492f4f5f7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:39.842456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d76f234a-02f6-4396-a414-997492f4f5f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:40.124727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f234a-02f6-4396-a414-997492f4f5f7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:40.156233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499bb13b-a0ab-4102-a11d-9ab88f081963 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:40.157591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bb13b-a0ab-4102-a11d-9ab88f081963 service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:40.159063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bb13b-a0ab-4102-a11d-9ab88f081963 service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:40.159423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bb13b-a0ab-4102-a11d-9ab88f081963 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:40.418207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499bb13b-a0ab-4102-a11d-9ab88f081963 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:40.420391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499bb13b-a0ab-4102-a11d-9ab88f081963 service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:40.465118 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3000/6068] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:11:40.888479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46d082-d6ea-4183-995d-9c1bd1c15a83 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:40.929304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46d082-d6ea-4183-995d-9c1bd1c15a83 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:40.952664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919002d-96f5-4471-a056-21838f8c2fb4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:40.973880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919002d-96f5-4471-a056-21838f8c2fb4 service glance] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:40.974222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919002d-96f5-4471-a056-21838f8c2fb4 service glance] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:40.978908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919002d-96f5-4471-a056-21838f8c2fb4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:41.041690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a919002d-96f5-4471-a056-21838f8c2fb4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:41.051951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a919002d-96f5-4471-a056-21838f8c2fb4 service glance] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:41.075580 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3069/6069] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:11:41.713659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499bb13b-a0ab-4102-a11d-9ab88f081963 service glance] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:41.830633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499bb13b-a0ab-4102-a11d-9ab88f081963 service glance] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:41.837841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b87ba3-7cc1-4927-9ad4-b9fad9a57541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:41.838758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b87ba3-7cc1-4927-9ad4-b9fad9a57541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:41.840990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b87ba3-7cc1-4927-9ad4-b9fad9a57541 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:41.841752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b87ba3-7cc1-4927-9ad4-b9fad9a57541 admin admin] PATH_INFO: `/v3/users/f64843a62c46411e9d8dbcba77215cf2` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:41.844314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b87ba3-7cc1-4927-9ad4-b9fad9a57541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f64843a62c46411e9d8dbcba77215cf2)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:41.846746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b87ba3-7cc1-4927-9ad4-b9fad9a57541 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:41.864870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:41.926970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-66b87ba3-7cc1-4927-9ad4-b9fad9a57541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f64843a62c46411e9d8dbcba77215cf2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:41.927842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-66b87ba3-7cc1-4927-9ad4-b9fad9a57541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f64843a62c46411e9d8dbcba77215cf2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:41.928615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-66b87ba3-7cc1-4927-9ad4-b9fad9a57541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f64843a62c46411e9d8dbcba77215cf2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:41.938161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-66b87ba3-7cc1-4927-9ad4-b9fad9a57541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f64843a62c46411e9d8dbcba77215cf2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:41.960945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-66b87ba3-7cc1-4927-9ad4-b9fad9a57541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f64843a62c46411e9d8dbcba77215cf2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:41.985342 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3001/6070] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:41 2022] DELETE /identity/v3/users/f64843a62c46411e9d8dbcba77215cf2 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:41.990583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8cfd1-28e5-4e60-bc76-ed6d9b8f3926 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:41.991119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8cfd1-28e5-4e60-bc76-ed6d9b8f3926 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:41.991297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8cfd1-28e5-4e60-bc76-ed6d9b8f3926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:42.041337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9ea8cfd1-28e5-4e60-bc76-ed6d9b8f3926 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:42.042757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9ea8cfd1-28e5-4e60-bc76-ed6d9b8f3926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:42.127645 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3070/6071] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:42.139006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b87ba3-7cc1-4927-9ad4-b9fad9a57541 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:42.140886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-66b87ba3-7cc1-4927-9ad4-b9fad9a57541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:42.226340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b87ba3-7cc1-4927-9ad4-b9fad9a57541 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:42.230722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d84e12-66a9-4cca-8956-05ea21e4afc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:42.231353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d84e12-66a9-4cca-8956-05ea21e4afc1 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:42.231663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d84e12-66a9-4cca-8956-05ea21e4afc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:42.231967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d84e12-66a9-4cca-8956-05ea21e4afc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:42.243762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d84e12-66a9-4cca-8956-05ea21e4afc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:42.245198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d84e12-66a9-4cca-8956-05ea21e4afc1 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:42.256724 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3002/6072] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:11:43.352048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8cfd1-28e5-4e60-bc76-ed6d9b8f3926 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:43.354716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9ea8cfd1-28e5-4e60-bc76-ed6d9b8f3926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:43.375708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8cfd1-28e5-4e60-bc76-ed6d9b8f3926 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:43.383193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86e111-776e-41d7-914c-7ed673180a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:43.384203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86e111-776e-41d7-914c-7ed673180a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:43.384721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86e111-776e-41d7-914c-7ed673180a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:43.385307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86e111-776e-41d7-914c-7ed673180a99 admin admin] PATH_INFO: `/v3/projects/26e628d0f18541c6abc1e8614bce827e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:43.386909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86e111-776e-41d7-914c-7ed673180a99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26e628d0f18541c6abc1e8614bce827e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:43.390411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86e111-776e-41d7-914c-7ed673180a99 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:43.466361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2d86e111-776e-41d7-914c-7ed673180a99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26e628d0f18541c6abc1e8614bce827e', 'request_id': 'req-2d86e111-776e-41d7-914c-7ed673180a99'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:43.473201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2d86e111-776e-41d7-914c-7ed673180a99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26e628d0f18541c6abc1e8614bce827e', 'request_id': 'req-2d86e111-776e-41d7-914c-7ed673180a99'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:43.479881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2d86e111-776e-41d7-914c-7ed673180a99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26e628d0f18541c6abc1e8614bce827e', 'request_id': 'req-2d86e111-776e-41d7-914c-7ed673180a99'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:43.499897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2d86e111-776e-41d7-914c-7ed673180a99 admin admin] The token cache is being invalidate because project 26e628d0f18541c6abc1e8614bce827e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:43.500188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2d86e111-776e-41d7-914c-7ed673180a99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:43.500879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2d86e111-776e-41d7-914c-7ed673180a99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:43.503446 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3071/6073] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:43 2022] DELETE /identity/v3/projects/26e628d0f18541c6abc1e8614bce827e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:11:43.507161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d84e12-66a9-4cca-8956-05ea21e4afc1 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:43.517956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d84e12-66a9-4cca-8956-05ea21e4afc1 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:43.594792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0246ea9-e45e-4f8c-895c-4d400a75e8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:43.595448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0246ea9-e45e-4f8c-895c-4d400a75e8ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:43.595736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0246ea9-e45e-4f8c-895c-4d400a75e8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:43.596158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0246ea9-e45e-4f8c-895c-4d400a75e8ed admin admin] PATH_INFO: `/v3/users/1c286400f0bb491a840bd003249bc731` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:43.597361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0246ea9-e45e-4f8c-895c-4d400a75e8ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c286400f0bb491a840bd003249bc731)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:43.599337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0246ea9-e45e-4f8c-895c-4d400a75e8ed admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:43.647189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:43.663243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b0246ea9-e45e-4f8c-895c-4d400a75e8ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c286400f0bb491a840bd003249bc731'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:43.666595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b0246ea9-e45e-4f8c-895c-4d400a75e8ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c286400f0bb491a840bd003249bc731'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:43.667002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b0246ea9-e45e-4f8c-895c-4d400a75e8ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c286400f0bb491a840bd003249bc731'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:43.677549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b0246ea9-e45e-4f8c-895c-4d400a75e8ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c286400f0bb491a840bd003249bc731'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:43.695434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b0246ea9-e45e-4f8c-895c-4d400a75e8ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c286400f0bb491a840bd003249bc731'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:43.712418 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3003/6074] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:43 2022] DELETE /identity/v3/users/1c286400f0bb491a840bd003249bc731 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:43.813392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec83c2-99d4-4418-8daa-fb951747f695 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:43.813814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec83c2-99d4-4418-8daa-fb951747f695 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:43.814264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec83c2-99d4-4418-8daa-fb951747f695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:43.837398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-76ec83c2-99d4-4418-8daa-fb951747f695 None None] MFA Rules not processed for user `8d79dff848bf40fc89377e13736c527b`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:43.838712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-76ec83c2-99d4-4418-8daa-fb951747f695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:43.844905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:43.942163 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3072/6075] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:44.980593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0246ea9-e45e-4f8c-895c-4d400a75e8ed admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:44.983265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b0246ea9-e45e-4f8c-895c-4d400a75e8ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:45.117736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0246ea9-e45e-4f8c-895c-4d400a75e8ed admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:45.130778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c05b6-6c14-4629-a8ac-39a8b65ae5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:45.134762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c05b6-6c14-4629-a8ac-39a8b65ae5ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:45.135093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c05b6-6c14-4629-a8ac-39a8b65ae5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:45.135380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c05b6-6c14-4629-a8ac-39a8b65ae5ca admin admin] PATH_INFO: `/v3/projects/71d9714d046a44ccaacc94040fae53db` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:45.137703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c05b6-6c14-4629-a8ac-39a8b65ae5ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=71d9714d046a44ccaacc94040fae53db)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:45.143415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c05b6-6c14-4629-a8ac-39a8b65ae5ca admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:45.236742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f23c05b6-6c14-4629-a8ac-39a8b65ae5ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71d9714d046a44ccaacc94040fae53db', 'request_id': 'req-f23c05b6-6c14-4629-a8ac-39a8b65ae5ca'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:45.267686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f23c05b6-6c14-4629-a8ac-39a8b65ae5ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71d9714d046a44ccaacc94040fae53db', 'request_id': 'req-f23c05b6-6c14-4629-a8ac-39a8b65ae5ca'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:45.285650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f23c05b6-6c14-4629-a8ac-39a8b65ae5ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71d9714d046a44ccaacc94040fae53db', 'request_id': 'req-f23c05b6-6c14-4629-a8ac-39a8b65ae5ca'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:45.314055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f23c05b6-6c14-4629-a8ac-39a8b65ae5ca admin admin] The token cache is being invalidate because project 71d9714d046a44ccaacc94040fae53db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:45.314391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f23c05b6-6c14-4629-a8ac-39a8b65ae5ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:45.315279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f23c05b6-6c14-4629-a8ac-39a8b65ae5ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:45.328185 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3004/6076] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:44 2022] DELETE /identity/v3/projects/71d9714d046a44ccaacc94040fae53db => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:45.329900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444f424-c32b-4558-b02d-fd68cedcab9d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:45.330742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444f424-c32b-4558-b02d-fd68cedcab9d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:45.331209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444f424-c32b-4558-b02d-fd68cedcab9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:45.387795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1444f424-c32b-4558-b02d-fd68cedcab9d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:45.393934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1444f424-c32b-4558-b02d-fd68cedcab9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:45.588701 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3073/6077] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:11:45.719961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ef616-90fe-46ba-b3a1-994793f4983b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:45.720365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ef616-90fe-46ba-b3a1-994793f4983b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:45.720505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ef616-90fe-46ba-b3a1-994793f4983b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:45.790691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ab5ef616-90fe-46ba-b3a1-994793f4983b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:45.796380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ab5ef616-90fe-46ba-b3a1-994793f4983b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:45.817797 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3005/6078] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:11:45.823326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444f424-c32b-4558-b02d-fd68cedcab9d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:45.844927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444f424-c32b-4558-b02d-fd68cedcab9d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:45.859473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8c00a-0ddb-412b-b1ee-fe1009f19c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:45.860109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8c00a-0ddb-412b-b1ee-fe1009f19c99 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:45.860390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8c00a-0ddb-412b-b1ee-fe1009f19c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:45.863585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8c00a-0ddb-412b-b1ee-fe1009f19c99 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:45.864476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8c00a-0ddb-412b-b1ee-fe1009f19c99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:11:45.866452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8c00a-0ddb-412b-b1ee-fe1009f19c99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:45.870622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8c00a-0ddb-412b-b1ee-fe1009f19c99 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:45.902042 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3074/6079] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:11:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:11:45.906803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ef616-90fe-46ba-b3a1-994793f4983b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:45.918071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ef616-90fe-46ba-b3a1-994793f4983b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:45.940482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba1a8d-7c3b-4fe2-846a-babb379892e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:45.941088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba1a8d-7c3b-4fe2-846a-babb379892e9 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:45.941368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba1a8d-7c3b-4fe2-846a-babb379892e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:45.941648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba1a8d-7c3b-4fe2-846a-babb379892e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:45.944848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba1a8d-7c3b-4fe2-846a-babb379892e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:45.948130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba1a8d-7c3b-4fe2-846a-babb379892e9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:45.989185 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3006/6080] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:45 2022] POST /identity/v3/projects => generated 389 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:45.993337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8c00a-0ddb-412b-b1ee-fe1009f19c99 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:46.008775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8c00a-0ddb-412b-b1ee-fe1009f19c99 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:46.149048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18da46-0918-4c2b-b653-9defb3f33021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:46.150041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18da46-0918-4c2b-b653-9defb3f33021 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:46.150578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18da46-0918-4c2b-b653-9defb3f33021 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:46.151043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18da46-0918-4c2b-b653-9defb3f33021 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:46.152125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18da46-0918-4c2b-b653-9defb3f33021 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:46.155408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18da46-0918-4c2b-b653-9defb3f33021 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:46.208875 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3075/6081] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:45 2022] POST /identity/v3/users => generated 295 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:46.210731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba1a8d-7c3b-4fe2-846a-babb379892e9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:46.226924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba1a8d-7c3b-4fe2-846a-babb379892e9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:46.240930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148479a8-d323-4fd5-8b6c-ada3729f4a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:46.242780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148479a8-d323-4fd5-8b6c-ada3729f4a22 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:46.243069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148479a8-d323-4fd5-8b6c-ada3729f4a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:46.243350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148479a8-d323-4fd5-8b6c-ada3729f4a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:46.244102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148479a8-d323-4fd5-8b6c-ada3729f4a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:46.245903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148479a8-d323-4fd5-8b6c-ada3729f4a22 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:46.273048 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3007/6082] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:46.280057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18da46-0918-4c2b-b653-9defb3f33021 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:46.292471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18da46-0918-4c2b-b653-9defb3f33021 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:46.298776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556e52b-9596-41ab-b855-48b5ced20582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:46.300009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556e52b-9596-41ab-b855-48b5ced20582 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:46.300009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556e52b-9596-41ab-b855-48b5ced20582 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:46.300859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556e52b-9596-41ab-b855-48b5ced20582 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:46.308861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556e52b-9596-41ab-b855-48b5ced20582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:46.311606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556e52b-9596-41ab-b855-48b5ced20582 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:46.327127 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3076/6083] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:46.331501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148479a8-d323-4fd5-8b6c-ada3729f4a22 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:46.338953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:46.348486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148479a8-d323-4fd5-8b6c-ada3729f4a22 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:46.359042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2a7ae-c022-4662-9a04-afea3119ca98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:46.360431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2a7ae-c022-4662-9a04-afea3119ca98 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:46.360943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2a7ae-c022-4662-9a04-afea3119ca98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:46.361565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2a7ae-c022-4662-9a04-afea3119ca98 admin admin] PATH_INFO: `/v3/projects/1e00ee15f5e1440cb75594f06cfb9fe6/users/0d9b01929f0541c1a94831518c529a71/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:46.395019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea2a7ae-c022-4662-9a04-afea3119ca98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e00ee15f5e1440cb75594f06cfb9fe6, user_id=0d9b01929f0541c1a94831518c529a71, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:46.398943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea2a7ae-c022-4662-9a04-afea3119ca98 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:46.425163 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3008/6084] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:46 2022] PUT /identity/v3/projects/1e00ee15f5e1440cb75594f06cfb9fe6/users/0d9b01929f0541c1a94831518c529a71/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:46.437787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f7a8b-5e25-476e-9668-b72eed905bee None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:46.438571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f7a8b-5e25-476e-9668-b72eed905bee None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:46.438571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f7a8b-5e25-476e-9668-b72eed905bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:46.515232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:46.568501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-063f7a8b-5e25-476e-9668-b72eed905bee None None] MFA Rules not processed for user `0d9b01929f0541c1a94831518c529a71`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:46.570866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-063f7a8b-5e25-476e-9668-b72eed905bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:46.614572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2a7ae-c022-4662-9a04-afea3119ca98 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:46.616655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7ea2a7ae-c022-4662-9a04-afea3119ca98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:46.775793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2a7ae-c022-4662-9a04-afea3119ca98 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:46.794182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db51116-15ea-4530-9844-c560f75ef237 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:46.794859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db51116-15ea-4530-9844-c560f75ef237 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:46.795197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db51116-15ea-4530-9844-c560f75ef237 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:46.796125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db51116-15ea-4530-9844-c560f75ef237 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:46.800501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0db51116-15ea-4530-9844-c560f75ef237 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:46.828878 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3077/6085] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:46.832397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db51116-15ea-4530-9844-c560f75ef237 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:46.834420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db51116-15ea-4530-9844-c560f75ef237 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:46.861198 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3009/6086] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:11:46.868646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f7a8b-5e25-476e-9668-b72eed905bee None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:46.870409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-063f7a8b-5e25-476e-9668-b72eed905bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:47.060648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f7a8b-5e25-476e-9668-b72eed905bee None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:47.080101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41205bdf-18a1-4536-be1d-114e471cdca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:47.082689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41205bdf-18a1-4536-be1d-114e471cdca0 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:47.082845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41205bdf-18a1-4536-be1d-114e471cdca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:47.085404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41205bdf-18a1-4536-be1d-114e471cdca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:47.112251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41205bdf-18a1-4536-be1d-114e471cdca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:47.116687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41205bdf-18a1-4536-be1d-114e471cdca0 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:47.152160 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3078/6087] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:46 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:11:48.238021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db51116-15ea-4530-9844-c560f75ef237 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:48.240128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0db51116-15ea-4530-9844-c560f75ef237 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:48.375170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db51116-15ea-4530-9844-c560f75ef237 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:48.381025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7356d52-9530-4af8-b58d-25a8eea01a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:48.382021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7356d52-9530-4af8-b58d-25a8eea01a4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:48.382762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7356d52-9530-4af8-b58d-25a8eea01a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:48.383347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7356d52-9530-4af8-b58d-25a8eea01a4b admin admin] PATH_INFO: `/v3/users/280a7a3c459a4aefa587c577ae177c61` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:48.386333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7356d52-9530-4af8-b58d-25a8eea01a4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=280a7a3c459a4aefa587c577ae177c61)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:48.388910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7356d52-9530-4af8-b58d-25a8eea01a4b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:48.478560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c7356d52-9530-4af8-b58d-25a8eea01a4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '280a7a3c459a4aefa587c577ae177c61'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:48.479997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c7356d52-9530-4af8-b58d-25a8eea01a4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '280a7a3c459a4aefa587c577ae177c61'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:48.480785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c7356d52-9530-4af8-b58d-25a8eea01a4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '280a7a3c459a4aefa587c577ae177c61'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:48.493585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c7356d52-9530-4af8-b58d-25a8eea01a4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '280a7a3c459a4aefa587c577ae177c61'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:48.515451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:48.518676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c7356d52-9530-4af8-b58d-25a8eea01a4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '280a7a3c459a4aefa587c577ae177c61'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:48.555078 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3010/6088] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:48 2022] DELETE /identity/v3/users/280a7a3c459a4aefa587c577ae177c61 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:48.556840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbb438-a399-4ba1-b5e0-d71de638c6ca None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:48.557438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbb438-a399-4ba1-b5e0-d71de638c6ca None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:48.557764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbb438-a399-4ba1-b5e0-d71de638c6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:48.586558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-abdbb438-a399-4ba1-b5e0-d71de638c6ca None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:48.587929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-abdbb438-a399-4ba1-b5e0-d71de638c6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:48.694059 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3079/6089] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:48.715365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7356d52-9530-4af8-b58d-25a8eea01a4b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:48.716851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c7356d52-9530-4af8-b58d-25a8eea01a4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:48.808284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7356d52-9530-4af8-b58d-25a8eea01a4b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:48.813927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df940e32-77af-4471-a4da-79175ea0f964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:48.814821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df940e32-77af-4471-a4da-79175ea0f964 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:48.815169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df940e32-77af-4471-a4da-79175ea0f964 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:48.815835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df940e32-77af-4471-a4da-79175ea0f964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:48.826444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df940e32-77af-4471-a4da-79175ea0f964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:48.828332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df940e32-77af-4471-a4da-79175ea0f964 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:48.838823 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3011/6090] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:11:49.463318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbb438-a399-4ba1-b5e0-d71de638c6ca None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:49.464871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-abdbb438-a399-4ba1-b5e0-d71de638c6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:49.479790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbb438-a399-4ba1-b5e0-d71de638c6ca None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:49.485861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9285a-f352-4937-87a0-d4608e6c934c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:49.486624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9285a-f352-4937-87a0-d4608e6c934c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:49.486979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9285a-f352-4937-87a0-d4608e6c934c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:49.487328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9285a-f352-4937-87a0-d4608e6c934c admin admin] PATH_INFO: `/v3/projects/2e8942500e3c4be7ae908a64552336f9` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:49.489179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9285a-f352-4937-87a0-d4608e6c934c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e8942500e3c4be7ae908a64552336f9)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:49.491483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9285a-f352-4937-87a0-d4608e6c934c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:49.573738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:49.585187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cac9285a-f352-4937-87a0-d4608e6c934c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e8942500e3c4be7ae908a64552336f9', 'request_id': 'req-cac9285a-f352-4937-87a0-d4608e6c934c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:49.597805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cac9285a-f352-4937-87a0-d4608e6c934c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e8942500e3c4be7ae908a64552336f9', 'request_id': 'req-cac9285a-f352-4937-87a0-d4608e6c934c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:49.612056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cac9285a-f352-4937-87a0-d4608e6c934c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e8942500e3c4be7ae908a64552336f9', 'request_id': 'req-cac9285a-f352-4937-87a0-d4608e6c934c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:49.659617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cac9285a-f352-4937-87a0-d4608e6c934c admin admin] The token cache is being invalidate because project 2e8942500e3c4be7ae908a64552336f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:49.660471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cac9285a-f352-4937-87a0-d4608e6c934c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:49.663107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cac9285a-f352-4937-87a0-d4608e6c934c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:49.665605 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3080/6091] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:49 2022] DELETE /identity/v3/projects/2e8942500e3c4be7ae908a64552336f9 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:11:49.669542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df940e32-77af-4471-a4da-79175ea0f964 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:49.683209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df940e32-77af-4471-a4da-79175ea0f964 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:49.754982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d237c00-3569-4a0b-adeb-1585315e6322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:49.756170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d237c00-3569-4a0b-adeb-1585315e6322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:49.756826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d237c00-3569-4a0b-adeb-1585315e6322 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:49.757310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d237c00-3569-4a0b-adeb-1585315e6322 admin admin] PATH_INFO: `/v3/users/130bbc81ba9548cc8b09adb13cc7e7aa` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:49.759873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d237c00-3569-4a0b-adeb-1585315e6322 admin admin] RBAC: Authorizing `identity:delete_user(user_id=130bbc81ba9548cc8b09adb13cc7e7aa)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:49.763164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d237c00-3569-4a0b-adeb-1585315e6322 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:49.864738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1d237c00-3569-4a0b-adeb-1585315e6322 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '130bbc81ba9548cc8b09adb13cc7e7aa'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:49.865850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1d237c00-3569-4a0b-adeb-1585315e6322 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '130bbc81ba9548cc8b09adb13cc7e7aa'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:49.866043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1d237c00-3569-4a0b-adeb-1585315e6322 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '130bbc81ba9548cc8b09adb13cc7e7aa'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:49.889340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1d237c00-3569-4a0b-adeb-1585315e6322 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '130bbc81ba9548cc8b09adb13cc7e7aa'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:49.906457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1d237c00-3569-4a0b-adeb-1585315e6322 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '130bbc81ba9548cc8b09adb13cc7e7aa'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:49.922542 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3012/6092] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:49 2022] DELETE /identity/v3/users/130bbc81ba9548cc8b09adb13cc7e7aa => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:50.915138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9285a-f352-4937-87a0-d4608e6c934c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:50.917138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cac9285a-f352-4937-87a0-d4608e6c934c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:50.936924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9285a-f352-4937-87a0-d4608e6c934c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:51.023813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8b290-82a5-407b-9d36-ea17a780162e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:51.024885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8b290-82a5-407b-9d36-ea17a780162e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:51.025471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8b290-82a5-407b-9d36-ea17a780162e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:51.025969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8b290-82a5-407b-9d36-ea17a780162e admin admin] PATH_INFO: `/v3/projects/65f2e592ee3b420e9cb0e93cc2ccb9cb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:51.028299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a8b290-82a5-407b-9d36-ea17a780162e admin admin] RBAC: Authorizing `identity:delete_project(project_id=65f2e592ee3b420e9cb0e93cc2ccb9cb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:51.031743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a8b290-82a5-407b-9d36-ea17a780162e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:51.135438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-52a8b290-82a5-407b-9d36-ea17a780162e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65f2e592ee3b420e9cb0e93cc2ccb9cb', 'request_id': 'req-52a8b290-82a5-407b-9d36-ea17a780162e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:51.144415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-52a8b290-82a5-407b-9d36-ea17a780162e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65f2e592ee3b420e9cb0e93cc2ccb9cb', 'request_id': 'req-52a8b290-82a5-407b-9d36-ea17a780162e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:51.155873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-52a8b290-82a5-407b-9d36-ea17a780162e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65f2e592ee3b420e9cb0e93cc2ccb9cb', 'request_id': 'req-52a8b290-82a5-407b-9d36-ea17a780162e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:51.176885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-52a8b290-82a5-407b-9d36-ea17a780162e admin admin] The token cache is being invalidate because project 65f2e592ee3b420e9cb0e93cc2ccb9cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:51.177653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-52a8b290-82a5-407b-9d36-ea17a780162e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:51.178845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-52a8b290-82a5-407b-9d36-ea17a780162e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:51.184917 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3081/6093] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:50 2022] DELETE /identity/v3/projects/65f2e592ee3b420e9cb0e93cc2ccb9cb => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:51.190512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee5c5c-8d10-4023-a6f5-bf4da4d0ab62 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:51.191538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee5c5c-8d10-4023-a6f5-bf4da4d0ab62 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:51.191966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee5c5c-8d10-4023-a6f5-bf4da4d0ab62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:51.215262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d9ee5c5c-8d10-4023-a6f5-bf4da4d0ab62 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:51.216997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d9ee5c5c-8d10-4023-a6f5-bf4da4d0ab62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:51.224852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:51.393489 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3013/6094] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:51.400901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a136e6-9d26-4726-bbf6-f4081406b387 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:51.401192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a136e6-9d26-4726-bbf6-f4081406b387 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:51.401473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a136e6-9d26-4726-bbf6-f4081406b387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:51.445204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-89a136e6-9d26-4726-bbf6-f4081406b387 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:51.448046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-89a136e6-9d26-4726-bbf6-f4081406b387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:51.464146 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3082/6095] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:51.468109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee5c5c-8d10-4023-a6f5-bf4da4d0ab62 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:51.492430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee5c5c-8d10-4023-a6f5-bf4da4d0ab62 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:51.511153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8db5cf-822f-434c-9b80-dba367dca80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:51.514822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8db5cf-822f-434c-9b80-dba367dca80d admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:51.515216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8db5cf-822f-434c-9b80-dba367dca80d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:51.515727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8db5cf-822f-434c-9b80-dba367dca80d admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:51.517563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8db5cf-822f-434c-9b80-dba367dca80d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:11:51.517917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8db5cf-822f-434c-9b80-dba367dca80d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:51.524241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8db5cf-822f-434c-9b80-dba367dca80d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:51.547600 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3014/6096] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:11:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:11:51.551314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a136e6-9d26-4726-bbf6-f4081406b387 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:51.567715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a136e6-9d26-4726-bbf6-f4081406b387 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:51.574632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d5bdd-01f3-4def-8981-d9ef8b62ae12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:51.575614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d5bdd-01f3-4def-8981-d9ef8b62ae12 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:51.576096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d5bdd-01f3-4def-8981-d9ef8b62ae12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:51.576919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d5bdd-01f3-4def-8981-d9ef8b62ae12 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:51.578675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d5bdd-01f3-4def-8981-d9ef8b62ae12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:51.581054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d5bdd-01f3-4def-8981-d9ef8b62ae12 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:51.640863 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3083/6097] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:51 2022] POST /identity/v3/projects => generated 367 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:11:51.645078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8db5cf-822f-434c-9b80-dba367dca80d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:51.663871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8db5cf-822f-434c-9b80-dba367dca80d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:51.767237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6a3ad-1f58-4f48-a81e-67ebb65e571b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:51.775506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6a3ad-1f58-4f48-a81e-67ebb65e571b admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:51.775909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6a3ad-1f58-4f48-a81e-67ebb65e571b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:51.776290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6a3ad-1f58-4f48-a81e-67ebb65e571b admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:51.777233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6a3ad-1f58-4f48-a81e-67ebb65e571b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:51.779690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6a3ad-1f58-4f48-a81e-67ebb65e571b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:51.815748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d5bdd-01f3-4def-8981-d9ef8b62ae12 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:51.825069 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3015/6098] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:51 2022] POST /identity/v3/users => generated 284 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:51.834247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d5bdd-01f3-4def-8981-d9ef8b62ae12 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:51.840220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f681dc-9df4-45d2-a579-2031445595d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:51.841126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f681dc-9df4-45d2-a579-2031445595d8 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:51.841589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f681dc-9df4-45d2-a579-2031445595d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:51.842439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f681dc-9df4-45d2-a579-2031445595d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:51.843402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f681dc-9df4-45d2-a579-2031445595d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:51.850201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f681dc-9df4-45d2-a579-2031445595d8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:51.897638 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3084/6099] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:11:51.902260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6a3ad-1f58-4f48-a81e-67ebb65e571b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:51.923991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6a3ad-1f58-4f48-a81e-67ebb65e571b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:51.930275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed435120-25a6-4336-bf29-aa5f0d196104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:51.931061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed435120-25a6-4336-bf29-aa5f0d196104 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:51.931398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed435120-25a6-4336-bf29-aa5f0d196104 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:51.931824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed435120-25a6-4336-bf29-aa5f0d196104 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:51.932483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed435120-25a6-4336-bf29-aa5f0d196104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:51.934997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed435120-25a6-4336-bf29-aa5f0d196104 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:51.949730 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3016/6100] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:51.955309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f681dc-9df4-45d2-a579-2031445595d8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:51.971728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f681dc-9df4-45d2-a579-2031445595d8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:52.002392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44dad67-3c5c-4781-b99d-52bec4fb1a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:52.003572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44dad67-3c5c-4781-b99d-52bec4fb1a3a admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:52.004031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44dad67-3c5c-4781-b99d-52bec4fb1a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:52.004513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44dad67-3c5c-4781-b99d-52bec4fb1a3a admin admin] PATH_INFO: `/v3/projects/31754c550bc246f682ca1f7b67875fdf/users/61561c9fb88446d1b3e43ebbbb9fcf7a/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:52.071914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44dad67-3c5c-4781-b99d-52bec4fb1a3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=31754c550bc246f682ca1f7b67875fdf, user_id=61561c9fb88446d1b3e43ebbbb9fcf7a, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:52.075023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44dad67-3c5c-4781-b99d-52bec4fb1a3a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:52.080588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:52.110927 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3085/6101] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:51 2022] PUT /identity/v3/projects/31754c550bc246f682ca1f7b67875fdf/users/61561c9fb88446d1b3e43ebbbb9fcf7a/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:52.120718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9596d-301b-468d-a008-a3b08109c312 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:52.120982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9596d-301b-468d-a008-a3b08109c312 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:52.121252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9596d-301b-468d-a008-a3b08109c312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:52.193651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-38e9596d-301b-468d-a008-a3b08109c312 None None] MFA Rules not processed for user `61561c9fb88446d1b3e43ebbbb9fcf7a`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:52.195679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-38e9596d-301b-468d-a008-a3b08109c312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:52.324100 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3017/6102] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:52.329037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44dad67-3c5c-4781-b99d-52bec4fb1a3a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:52.478349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44dad67-3c5c-4781-b99d-52bec4fb1a3a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:52.503248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1407011-0f31-47eb-8b2c-da9576811d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:52.503882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1407011-0f31-47eb-8b2c-da9576811d23 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:52.504164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1407011-0f31-47eb-8b2c-da9576811d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:52.504831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1407011-0f31-47eb-8b2c-da9576811d23 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:52.505066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1407011-0f31-47eb-8b2c-da9576811d23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:52.514590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1407011-0f31-47eb-8b2c-da9576811d23 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:52.580537 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3086/6103] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:52 2022] POST /identity/v3/projects => generated 367 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:52.584258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9596d-301b-468d-a008-a3b08109c312 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:52.592071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:52.597045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9596d-301b-468d-a008-a3b08109c312 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:52.704295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9b166-62e8-4c85-8703-2177442ca093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:52.705335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9b166-62e8-4c85-8703-2177442ca093 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:52.705851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9b166-62e8-4c85-8703-2177442ca093 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:52.707109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9b166-62e8-4c85-8703-2177442ca093 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:52.708276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad9b166-62e8-4c85-8703-2177442ca093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:52.710610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad9b166-62e8-4c85-8703-2177442ca093 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:52.754111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1407011-0f31-47eb-8b2c-da9576811d23 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:52.763104 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3018/6104] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:52 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:52.770247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1407011-0f31-47eb-8b2c-da9576811d23 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:52.778971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223820c-f7cc-41f0-9a11-5fb2d998c4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:52.780353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223820c-f7cc-41f0-9a11-5fb2d998c4af admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:52.781910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223820c-f7cc-41f0-9a11-5fb2d998c4af admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:52.782624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223820c-f7cc-41f0-9a11-5fb2d998c4af admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:52.784598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223820c-f7cc-41f0-9a11-5fb2d998c4af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:52.793637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223820c-f7cc-41f0-9a11-5fb2d998c4af admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:52.822405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9b166-62e8-4c85-8703-2177442ca093 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:52.833457 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3087/6105] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:52.844063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9b166-62e8-4c85-8703-2177442ca093 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:52.856979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc087a-824b-4d6a-b9a3-cfa1d4dc858c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:52.858524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc087a-824b-4d6a-b9a3-cfa1d4dc858c admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:52.858946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc087a-824b-4d6a-b9a3-cfa1d4dc858c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:52.859816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc087a-824b-4d6a-b9a3-cfa1d4dc858c admin admin] PATH_INFO: `/v3/projects/008a915d72854ac383e8b9f08f106795/users/aff43b0d937549e78be18060a70f522a/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:52.885813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc087a-824b-4d6a-b9a3-cfa1d4dc858c admin admin] RBAC: Authorizing `identity:create_grant(project_id=008a915d72854ac383e8b9f08f106795, user_id=aff43b0d937549e78be18060a70f522a, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:52.889388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc087a-824b-4d6a-b9a3-cfa1d4dc858c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:52.913030 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3019/6106] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:52 2022] PUT /identity/v3/projects/008a915d72854ac383e8b9f08f106795/users/aff43b0d937549e78be18060a70f522a/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:52.917078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223820c-f7cc-41f0-9a11-5fb2d998c4af admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:52.929335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223820c-f7cc-41f0-9a11-5fb2d998c4af admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:53.011067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed164c-b215-4575-bede-226812f5b36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:53.014733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed164c-b215-4575-bede-226812f5b36d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:53.014943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed164c-b215-4575-bede-226812f5b36d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:53.015227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed164c-b215-4575-bede-226812f5b36d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:53.016534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed164c-b215-4575-bede-226812f5b36d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:53.019206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed164c-b215-4575-bede-226812f5b36d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:53.051636 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3088/6107] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:53.055554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc087a-824b-4d6a-b9a3-cfa1d4dc858c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:53.074946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc087a-824b-4d6a-b9a3-cfa1d4dc858c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:53.083391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757fb0a-75ae-4092-b701-3645b9e62459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:53.084496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757fb0a-75ae-4092-b701-3645b9e62459 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:53.085084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757fb0a-75ae-4092-b701-3645b9e62459 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:53.085515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757fb0a-75ae-4092-b701-3645b9e62459 admin admin] PATH_INFO: `/v3/projects/008a915d72854ac383e8b9f08f106795/users/aff43b0d937549e78be18060a70f522a/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:53.087867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757fb0a-75ae-4092-b701-3645b9e62459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=008a915d72854ac383e8b9f08f106795, user_id=aff43b0d937549e78be18060a70f522a, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:53.094449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757fb0a-75ae-4092-b701-3645b9e62459 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:53.113216 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3020/6108] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:53 2022] PUT /identity/v3/projects/008a915d72854ac383e8b9f08f106795/users/aff43b0d937549e78be18060a70f522a/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:11:53.125711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c2584-9d98-4454-bb29-19d82b968e0e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:53.126991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c2584-9d98-4454-bb29-19d82b968e0e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:53.127539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c2584-9d98-4454-bb29-19d82b968e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:53.168182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5c1c2584-9d98-4454-bb29-19d82b968e0e None None] MFA Rules not processed for user `aff43b0d937549e78be18060a70f522a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:53.169838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5c1c2584-9d98-4454-bb29-19d82b968e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:53.284663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:53.360996 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3089/6109] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:53.378287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757fb0a-75ae-4092-b701-3645b9e62459 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:53.381925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e757fb0a-75ae-4092-b701-3645b9e62459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:53.570548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757fb0a-75ae-4092-b701-3645b9e62459 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:53.575592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3013f-98c0-49f0-8d38-4695f3b8c619 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:53.576332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3013f-98c0-49f0-8d38-4695f3b8c619 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:53.576618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3013f-98c0-49f0-8d38-4695f3b8c619 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:53.576890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3013f-98c0-49f0-8d38-4695f3b8c619 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:53.595205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3013f-98c0-49f0-8d38-4695f3b8c619 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:53.595433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3013f-98c0-49f0-8d38-4695f3b8c619 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:53.621160 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3021/6110] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:53 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:11:55.843583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c2584-9d98-4454-bb29-19d82b968e0e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:55.936431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c2584-9d98-4454-bb29-19d82b968e0e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:55.941882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccabc56-6476-45ce-97bc-da486c1dc01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:55.944856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccabc56-6476-45ce-97bc-da486c1dc01e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:55.945568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccabc56-6476-45ce-97bc-da486c1dc01e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:55.945845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccabc56-6476-45ce-97bc-da486c1dc01e admin admin] PATH_INFO: `/v3/users/61561c9fb88446d1b3e43ebbbb9fcf7a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:55.947101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccabc56-6476-45ce-97bc-da486c1dc01e admin admin] RBAC: Authorizing `identity:delete_user(user_id=61561c9fb88446d1b3e43ebbbb9fcf7a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:55.949443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccabc56-6476-45ce-97bc-da486c1dc01e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:56.015730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4ccabc56-6476-45ce-97bc-da486c1dc01e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61561c9fb88446d1b3e43ebbbb9fcf7a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:56.025610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4ccabc56-6476-45ce-97bc-da486c1dc01e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61561c9fb88446d1b3e43ebbbb9fcf7a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:56.044159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4ccabc56-6476-45ce-97bc-da486c1dc01e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61561c9fb88446d1b3e43ebbbb9fcf7a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:56.045537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4ccabc56-6476-45ce-97bc-da486c1dc01e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61561c9fb88446d1b3e43ebbbb9fcf7a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:56.063024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4ccabc56-6476-45ce-97bc-da486c1dc01e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61561c9fb88446d1b3e43ebbbb9fcf7a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:56.071521 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3090/6111] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:55 2022] DELETE /identity/v3/users/61561c9fb88446d1b3e43ebbbb9fcf7a => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:56.076928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7471eb1-94a9-4e02-bd29-56982db98766 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:56.077539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7471eb1-94a9-4e02-bd29-56982db98766 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:56.077815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7471eb1-94a9-4e02-bd29-56982db98766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:56.107316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a7471eb1-94a9-4e02-bd29-56982db98766 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:56.109498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a7471eb1-94a9-4e02-bd29-56982db98766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:56.202295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:56.214828 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3022/6112] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:11:56.230565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccabc56-6476-45ce-97bc-da486c1dc01e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:56.233709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4ccabc56-6476-45ce-97bc-da486c1dc01e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:56.304346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccabc56-6476-45ce-97bc-da486c1dc01e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:56.308158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ddf15-39c8-4d88-9da1-9ae38e60bac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:56.308868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ddf15-39c8-4d88-9da1-9ae38e60bac4 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:56.309215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ddf15-39c8-4d88-9da1-9ae38e60bac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:56.309551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ddf15-39c8-4d88-9da1-9ae38e60bac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:56.321182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ddf15-39c8-4d88-9da1-9ae38e60bac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:56.322744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ddf15-39c8-4d88-9da1-9ae38e60bac4 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:56.334521 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3091/6113] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:11:57.126002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7471eb1-94a9-4e02-bd29-56982db98766 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:57.128813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a7471eb1-94a9-4e02-bd29-56982db98766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:57.148705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7471eb1-94a9-4e02-bd29-56982db98766 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:57.156375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84fe62-2152-42c4-aed4-feff28983aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:57.157224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84fe62-2152-42c4-aed4-feff28983aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:57.157663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84fe62-2152-42c4-aed4-feff28983aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:57.158117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84fe62-2152-42c4-aed4-feff28983aa9 admin admin] PATH_INFO: `/v3/projects/31754c550bc246f682ca1f7b67875fdf` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:57.160660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e84fe62-2152-42c4-aed4-feff28983aa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31754c550bc246f682ca1f7b67875fdf)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:57.164589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e84fe62-2152-42c4-aed4-feff28983aa9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:57.315148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5e84fe62-2152-42c4-aed4-feff28983aa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31754c550bc246f682ca1f7b67875fdf', 'request_id': 'req-5e84fe62-2152-42c4-aed4-feff28983aa9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:57.320831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ddf15-39c8-4d88-9da1-9ae38e60bac4 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:57.322914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-230ddf15-39c8-4d88-9da1-9ae38e60bac4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:57.327786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5e84fe62-2152-42c4-aed4-feff28983aa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31754c550bc246f682ca1f7b67875fdf', 'request_id': 'req-5e84fe62-2152-42c4-aed4-feff28983aa9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:57.343301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ddf15-39c8-4d88-9da1-9ae38e60bac4 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:57.350352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5e84fe62-2152-42c4-aed4-feff28983aa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31754c550bc246f682ca1f7b67875fdf', 'request_id': 'req-5e84fe62-2152-42c4-aed4-feff28983aa9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:57.376246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:57.382692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5e84fe62-2152-42c4-aed4-feff28983aa9 admin admin] The token cache is being invalidate because project 31754c550bc246f682ca1f7b67875fdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:57.383050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5e84fe62-2152-42c4-aed4-feff28983aa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:57.384021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5e84fe62-2152-42c4-aed4-feff28983aa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:57.387252 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3023/6114] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:57 2022] DELETE /identity/v3/projects/31754c550bc246f682ca1f7b67875fdf => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:57.395537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84fe62-2152-42c4-aed4-feff28983aa9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:57.397529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5e84fe62-2152-42c4-aed4-feff28983aa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:57.471338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35939a2d-3843-4f77-858b-1b9d6dd1b9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:57.472071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35939a2d-3843-4f77-858b-1b9d6dd1b9c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:57.472555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35939a2d-3843-4f77-858b-1b9d6dd1b9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:57.474301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35939a2d-3843-4f77-858b-1b9d6dd1b9c4 admin admin] PATH_INFO: `/v3/users/4f3c9d35f6b843ef905f43d758a9bb25` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:57.475866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35939a2d-3843-4f77-858b-1b9d6dd1b9c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f3c9d35f6b843ef905f43d758a9bb25)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:57.478868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35939a2d-3843-4f77-858b-1b9d6dd1b9c4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:57.512516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84fe62-2152-42c4-aed4-feff28983aa9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:57.519133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1d4a0-dd55-4bf6-99b9-c25c4c94cfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:57.520201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1d4a0-dd55-4bf6-99b9-c25c4c94cfb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:57.520518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1d4a0-dd55-4bf6-99b9-c25c4c94cfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:57.520813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1d4a0-dd55-4bf6-99b9-c25c4c94cfb9 admin admin] PATH_INFO: `/v3/users/aff43b0d937549e78be18060a70f522a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:57.523522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1d4a0-dd55-4bf6-99b9-c25c4c94cfb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aff43b0d937549e78be18060a70f522a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:57.526662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1d4a0-dd55-4bf6-99b9-c25c4c94cfb9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:57.566481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-35939a2d-3843-4f77-858b-1b9d6dd1b9c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f3c9d35f6b843ef905f43d758a9bb25'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:57.577968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-35939a2d-3843-4f77-858b-1b9d6dd1b9c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f3c9d35f6b843ef905f43d758a9bb25'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:57.594643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-35939a2d-3843-4f77-858b-1b9d6dd1b9c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f3c9d35f6b843ef905f43d758a9bb25'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:57.595719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-35939a2d-3843-4f77-858b-1b9d6dd1b9c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f3c9d35f6b843ef905f43d758a9bb25'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:57.598344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6da1d4a0-dd55-4bf6-99b9-c25c4c94cfb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aff43b0d937549e78be18060a70f522a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:57.599491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6da1d4a0-dd55-4bf6-99b9-c25c4c94cfb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aff43b0d937549e78be18060a70f522a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:57.601351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6da1d4a0-dd55-4bf6-99b9-c25c4c94cfb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aff43b0d937549e78be18060a70f522a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:57.604241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-35939a2d-3843-4f77-858b-1b9d6dd1b9c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f3c9d35f6b843ef905f43d758a9bb25'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:57.612821 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3092/6115] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:57 2022] DELETE /identity/v3/users/4f3c9d35f6b843ef905f43d758a9bb25 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:57.616282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6da1d4a0-dd55-4bf6-99b9-c25c4c94cfb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aff43b0d937549e78be18060a70f522a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:57.632423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6da1d4a0-dd55-4bf6-99b9-c25c4c94cfb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aff43b0d937549e78be18060a70f522a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:57.648715 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3024/6116] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:11:57 2022] DELETE /identity/v3/users/aff43b0d937549e78be18060a70f522a => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:57.997743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35939a2d-3843-4f77-858b-1b9d6dd1b9c4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:57.999595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-35939a2d-3843-4f77-858b-1b9d6dd1b9c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:58.101649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35939a2d-3843-4f77-858b-1b9d6dd1b9c4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:58.147118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:58.170378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab49ee-085d-44f7-8d45-b4fdf88b5c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:58.171253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab49ee-085d-44f7-8d45-b4fdf88b5c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:58.171539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab49ee-085d-44f7-8d45-b4fdf88b5c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:58.171945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab49ee-085d-44f7-8d45-b4fdf88b5c10 admin admin] PATH_INFO: `/v3/projects/1215f2e4b926472794f0e60cefa2d90e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:58.173110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fab49ee-085d-44f7-8d45-b4fdf88b5c10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1215f2e4b926472794f0e60cefa2d90e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:58.175375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fab49ee-085d-44f7-8d45-b4fdf88b5c10 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:58.227909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1d4a0-dd55-4bf6-99b9-c25c4c94cfb9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:58.229663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6da1d4a0-dd55-4bf6-99b9-c25c4c94cfb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:58.241044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:58.264245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7fab49ee-085d-44f7-8d45-b4fdf88b5c10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1215f2e4b926472794f0e60cefa2d90e', 'request_id': 'req-7fab49ee-085d-44f7-8d45-b4fdf88b5c10'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:58.273174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7fab49ee-085d-44f7-8d45-b4fdf88b5c10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1215f2e4b926472794f0e60cefa2d90e', 'request_id': 'req-7fab49ee-085d-44f7-8d45-b4fdf88b5c10'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:58.276710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1d4a0-dd55-4bf6-99b9-c25c4c94cfb9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:58.282570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7fab49ee-085d-44f7-8d45-b4fdf88b5c10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1215f2e4b926472794f0e60cefa2d90e', 'request_id': 'req-7fab49ee-085d-44f7-8d45-b4fdf88b5c10'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:58.301760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7fab49ee-085d-44f7-8d45-b4fdf88b5c10 admin admin] The token cache is being invalidate because project 1215f2e4b926472794f0e60cefa2d90e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:58.302417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7fab49ee-085d-44f7-8d45-b4fdf88b5c10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:58.303637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7fab49ee-085d-44f7-8d45-b4fdf88b5c10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:58.306212 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3093/6117] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:57 2022] DELETE /identity/v3/projects/1215f2e4b926472794f0e60cefa2d90e => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:58.311676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5472ab6-3244-4cdc-9dba-a3d832b05a85 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:58.312291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5472ab6-3244-4cdc-9dba-a3d832b05a85 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:58.312729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5472ab6-3244-4cdc-9dba-a3d832b05a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:58.340977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d5472ab6-3244-4cdc-9dba-a3d832b05a85 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:58.344183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d5472ab6-3244-4cdc-9dba-a3d832b05a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:58.353688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00079243-6355-40a6-8c93-89d3a833730d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:58.354517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00079243-6355-40a6-8c93-89d3a833730d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:58.354886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00079243-6355-40a6-8c93-89d3a833730d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:58.355515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00079243-6355-40a6-8c93-89d3a833730d admin admin] PATH_INFO: `/v3/projects/008a915d72854ac383e8b9f08f106795` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:58.358184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00079243-6355-40a6-8c93-89d3a833730d admin admin] RBAC: Authorizing `identity:delete_project(project_id=008a915d72854ac383e8b9f08f106795)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:58.360335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00079243-6355-40a6-8c93-89d3a833730d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:58.425590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00079243-6355-40a6-8c93-89d3a833730d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '008a915d72854ac383e8b9f08f106795', 'request_id': 'req-00079243-6355-40a6-8c93-89d3a833730d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:58.439446 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3094/6118] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:58.440222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00079243-6355-40a6-8c93-89d3a833730d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '008a915d72854ac383e8b9f08f106795', 'request_id': 'req-00079243-6355-40a6-8c93-89d3a833730d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:58.443176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3fa61-b1d8-4b2a-9c27-a7c02674da5b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:58.443471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3fa61-b1d8-4b2a-9c27-a7c02674da5b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:58.443753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3fa61-b1d8-4b2a-9c27-a7c02674da5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:58.448756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00079243-6355-40a6-8c93-89d3a833730d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '008a915d72854ac383e8b9f08f106795', 'request_id': 'req-00079243-6355-40a6-8c93-89d3a833730d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:58.462746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00079243-6355-40a6-8c93-89d3a833730d admin admin] The token cache is being invalidate because project 008a915d72854ac383e8b9f08f106795 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:11:58.463040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00079243-6355-40a6-8c93-89d3a833730d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:58.463972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00079243-6355-40a6-8c93-89d3a833730d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:11:58.467586 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3025/6119] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:11:58 2022] DELETE /identity/v3/projects/008a915d72854ac383e8b9f08f106795 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:11:58.474664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5e97b-5b4e-4bf8-a7df-bf28e9feb38b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:58.475629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5e97b-5b4e-4bf8-a7df-bf28e9feb38b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:58.476072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5e97b-5b4e-4bf8-a7df-bf28e9feb38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:58.478033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-21f3fa61-b1d8-4b2a-9c27-a7c02674da5b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:58.479718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-21f3fa61-b1d8-4b2a-9c27-a7c02674da5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:58.510347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ebd5e97b-5b4e-4bf8-a7df-bf28e9feb38b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:58.513011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ebd5e97b-5b4e-4bf8-a7df-bf28e9feb38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:58.563988 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3095/6120] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:58.568107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3fa61-b1d8-4b2a-9c27-a7c02674da5b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:58.580030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3fa61-b1d8-4b2a-9c27-a7c02674da5b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:58.585806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d592f67-eb9c-4016-8318-ac39fd2b9394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:58.586647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d592f67-eb9c-4016-8318-ac39fd2b9394 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:58.587005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d592f67-eb9c-4016-8318-ac39fd2b9394 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:58.587468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d592f67-eb9c-4016-8318-ac39fd2b9394 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:58.588209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d592f67-eb9c-4016-8318-ac39fd2b9394 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:11:58.588576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d592f67-eb9c-4016-8318-ac39fd2b9394 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:58.590382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d592f67-eb9c-4016-8318-ac39fd2b9394 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:58.606944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:58.620930 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3026/6121] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:58.629056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c858c56-01be-43ba-9fc3-7a1a93fc59a3 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:58.629484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c858c56-01be-43ba-9fc3-7a1a93fc59a3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:58.629850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c858c56-01be-43ba-9fc3-7a1a93fc59a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:58.638892 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3096/6122] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:11:58.645296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d592f67-eb9c-4016-8318-ac39fd2b9394 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:58.672995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d592f67-eb9c-4016-8318-ac39fd2b9394 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:58.677190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9c858c56-01be-43ba-9fc3-7a1a93fc59a3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:58.678731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9c858c56-01be-43ba-9fc3-7a1a93fc59a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:58.681107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4257a-962b-4144-93fc-c1e7b6dfbfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:58.686557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4257a-962b-4144-93fc-c1e7b6dfbfa6 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:58.686557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4257a-962b-4144-93fc-c1e7b6dfbfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:58.686557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4257a-962b-4144-93fc-c1e7b6dfbfa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:58.686557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4257a-962b-4144-93fc-c1e7b6dfbfa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:58.689541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4257a-962b-4144-93fc-c1e7b6dfbfa6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:58.702523 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3027/6123] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:58.706861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c858c56-01be-43ba-9fc3-7a1a93fc59a3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:58.719259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c858c56-01be-43ba-9fc3-7a1a93fc59a3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:58.729458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183618e-8ed4-490d-a085-ff9b078660b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:58.730217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183618e-8ed4-490d-a085-ff9b078660b2 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:58.733325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183618e-8ed4-490d-a085-ff9b078660b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:58.733704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183618e-8ed4-490d-a085-ff9b078660b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:58.734592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183618e-8ed4-490d-a085-ff9b078660b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:11:58.734994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183618e-8ed4-490d-a085-ff9b078660b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:58.738998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183618e-8ed4-490d-a085-ff9b078660b2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:58.762410 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3097/6124] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:58 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:58.767424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4257a-962b-4144-93fc-c1e7b6dfbfa6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:58.770919 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3028/6125] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:11:58.775522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183618e-8ed4-490d-a085-ff9b078660b2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:58.796119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183618e-8ed4-490d-a085-ff9b078660b2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:58.816029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:58.889957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4257a-962b-4144-93fc-c1e7b6dfbfa6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:58.898869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083c381-5025-47bb-b105-eeb931c73d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:58.899651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083c381-5025-47bb-b105-eeb931c73d31 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:58.899651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083c381-5025-47bb-b105-eeb931c73d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:58.899948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083c381-5025-47bb-b105-eeb931c73d31 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:58.900583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083c381-5025-47bb-b105-eeb931c73d31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:58.902557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083c381-5025-47bb-b105-eeb931c73d31 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:58.925365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc77c35-7322-4fea-ba33-55ae1589948a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:58.926247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc77c35-7322-4fea-ba33-55ae1589948a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:58.926484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc77c35-7322-4fea-ba33-55ae1589948a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:58.926880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc77c35-7322-4fea-ba33-55ae1589948a admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:58.929189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc77c35-7322-4fea-ba33-55ae1589948a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:58.931426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc77c35-7322-4fea-ba33-55ae1589948a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:58.933548 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3098/6126] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:58 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:58.938378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083c381-5025-47bb-b105-eeb931c73d31 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:58.951454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083c381-5025-47bb-b105-eeb931c73d31 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:58.957920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa71cfe-a36a-418b-8b8e-667bfb78aa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:58.958580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa71cfe-a36a-418b-8b8e-667bfb78aa69 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:58.958871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa71cfe-a36a-418b-8b8e-667bfb78aa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:58.959150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa71cfe-a36a-418b-8b8e-667bfb78aa69 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:58.959855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa71cfe-a36a-418b-8b8e-667bfb78aa69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:58.961900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa71cfe-a36a-418b-8b8e-667bfb78aa69 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:58.975961 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3099/6127] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:11:58.980195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa71cfe-a36a-418b-8b8e-667bfb78aa69 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:58.986196 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3029/6128] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:58 2022] POST /identity/v3/projects => generated 365 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:58.987216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc77c35-7322-4fea-ba33-55ae1589948a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:58.995007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa71cfe-a36a-418b-8b8e-667bfb78aa69 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:59.074702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc77c35-7322-4fea-ba33-55ae1589948a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:59.080456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124e42e-d7db-483b-8273-cf78fc7e5cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:59.081292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124e42e-d7db-483b-8273-cf78fc7e5cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:59.081906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124e42e-d7db-483b-8273-cf78fc7e5cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:59.082386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124e42e-d7db-483b-8273-cf78fc7e5cc8 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:59.083136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124e42e-d7db-483b-8273-cf78fc7e5cc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:59.085390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124e42e-d7db-483b-8273-cf78fc7e5cc8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:59.091507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4301b-418d-4e27-a2bf-e9fdae0b2cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:59.092739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4301b-418d-4e27-a2bf-e9fdae0b2cdb admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:59.093020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4301b-418d-4e27-a2bf-e9fdae0b2cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:59.093291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4301b-418d-4e27-a2bf-e9fdae0b2cdb admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:59.094466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4301b-418d-4e27-a2bf-e9fdae0b2cdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:59.095793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4301b-418d-4e27-a2bf-e9fdae0b2cdb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:59.109240 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3100/6129] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:58 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:11:59.114160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4301b-418d-4e27-a2bf-e9fdae0b2cdb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:59.122611 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3030/6130] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:58 2022] POST /identity/v3/users => generated 283 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:59.126119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4301b-418d-4e27-a2bf-e9fdae0b2cdb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:59.126735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124e42e-d7db-483b-8273-cf78fc7e5cc8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:59.132014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c33ee1-9d3b-462c-8677-069094e93baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:59.132695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c33ee1-9d3b-462c-8677-069094e93baa admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:59.132977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c33ee1-9d3b-462c-8677-069094e93baa admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:59.133254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c33ee1-9d3b-462c-8677-069094e93baa admin admin] PATH_INFO: `/v3/projects/8f92b50dc0364842980ce41af830ba1a/users/a2745d6923e347939e2c67015aefadda/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:59.143531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124e42e-d7db-483b-8273-cf78fc7e5cc8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:59.150118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678cdbe-5516-476b-9bd4-70b1bc7e4acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:59.151489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678cdbe-5516-476b-9bd4-70b1bc7e4acd admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:59.151938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678cdbe-5516-476b-9bd4-70b1bc7e4acd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:59.152344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678cdbe-5516-476b-9bd4-70b1bc7e4acd admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:59.152619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c33ee1-9d3b-462c-8677-069094e93baa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f92b50dc0364842980ce41af830ba1a, user_id=a2745d6923e347939e2c67015aefadda, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:59.152994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678cdbe-5516-476b-9bd4-70b1bc7e4acd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:59.155310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678cdbe-5516-476b-9bd4-70b1bc7e4acd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:59.156520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c33ee1-9d3b-462c-8677-069094e93baa admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:59.166708 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3031/6131] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:11:59.171190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678cdbe-5516-476b-9bd4-70b1bc7e4acd admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:59.172483 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3101/6132] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:59 2022] PUT /identity/v3/projects/8f92b50dc0364842980ce41af830ba1a/users/a2745d6923e347939e2c67015aefadda/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:59.177438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5d03c-007d-4c04-8fc0-c9cf6dc0965f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:59.177730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5d03c-007d-4c04-8fc0-c9cf6dc0965f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:59.178012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5d03c-007d-4c04-8fc0-c9cf6dc0965f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:59.185549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678cdbe-5516-476b-9bd4-70b1bc7e4acd admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:59.199355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7ff5d03c-007d-4c04-8fc0-c9cf6dc0965f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:59.200602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7ff5d03c-007d-4c04-8fc0-c9cf6dc0965f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:59.227743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:59.314390 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3102/6133] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:11:59.327029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5d03c-007d-4c04-8fc0-c9cf6dc0965f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:59.328508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7ff5d03c-007d-4c04-8fc0-c9cf6dc0965f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:59.335402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aba3c8-ff0c-4c56-88b1-0f69033739f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:59.343433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aba3c8-ff0c-4c56-88b1-0f69033739f8 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:59.343970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aba3c8-ff0c-4c56-88b1-0f69033739f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:59.344425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aba3c8-ff0c-4c56-88b1-0f69033739f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:59.345468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aba3c8-ff0c-4c56-88b1-0f69033739f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:59.347418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aba3c8-ff0c-4c56-88b1-0f69033739f8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:59.370197 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3032/6134] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:11:59.374178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aba3c8-ff0c-4c56-88b1-0f69033739f8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:59.392613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aba3c8-ff0c-4c56-88b1-0f69033739f8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:59.404994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448fba8-4e92-456c-b4c4-fbdb77f3ebf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:59.405680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448fba8-4e92-456c-b4c4-fbdb77f3ebf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:59.405960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448fba8-4e92-456c-b4c4-fbdb77f3ebf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:59.406221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448fba8-4e92-456c-b4c4-fbdb77f3ebf7 admin admin] PATH_INFO: `/v3/projects/6034100b31904b83870cd7da3e8dfc0c/users/40f73d17e08d4ca58a70fa38024fc78d/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:59.417080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5d03c-007d-4c04-8fc0-c9cf6dc0965f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:59.422617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:59.425617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896158d6-c50f-4a3e-868e-5349afecf23f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:59.426395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896158d6-c50f-4a3e-868e-5349afecf23f service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:59.426783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896158d6-c50f-4a3e-868e-5349afecf23f service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:59.427252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896158d6-c50f-4a3e-868e-5349afecf23f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:59.428897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448fba8-4e92-456c-b4c4-fbdb77f3ebf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6034100b31904b83870cd7da3e8dfc0c, user_id=40f73d17e08d4ca58a70fa38024fc78d, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:59.432046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448fba8-4e92-456c-b4c4-fbdb77f3ebf7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:59.442953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896158d6-c50f-4a3e-868e-5349afecf23f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:59.444944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896158d6-c50f-4a3e-868e-5349afecf23f service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:59.453450 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3033/6135] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:59 2022] PUT /identity/v3/projects/6034100b31904b83870cd7da3e8dfc0c/users/40f73d17e08d4ca58a70fa38024fc78d/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:11:59.457192 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3103/6136] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:11:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:11:59.464115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b7b31-7d15-4aec-9849-5cd952f49c72 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:59.464492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b7b31-7d15-4aec-9849-5cd952f49c72 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:59.464963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b7b31-7d15-4aec-9849-5cd952f49c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:59.495657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-745b7b31-7d15-4aec-9849-5cd952f49c72 None None] MFA Rules not processed for user `40f73d17e08d4ca58a70fa38024fc78d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:11:59.497063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-745b7b31-7d15-4aec-9849-5cd952f49c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:11:59.604101 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3034/6137] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:11:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:11:59.605547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896158d6-c50f-4a3e-868e-5349afecf23f service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:59.625601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896158d6-c50f-4a3e-868e-5349afecf23f service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:59.693047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e2637-19e5-4e0c-8d03-52354025615f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:59.693667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e2637-19e5-4e0c-8d03-52354025615f admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:59.693942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e2637-19e5-4e0c-8d03-52354025615f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:59.694214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e2637-19e5-4e0c-8d03-52354025615f admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:59.694890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e2637-19e5-4e0c-8d03-52354025615f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:59.696802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e2637-19e5-4e0c-8d03-52354025615f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:59.737617 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3104/6138] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:11:59 2022] POST /identity/v3/projects => generated 367 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:11:59.738792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b7b31-7d15-4aec-9849-5cd952f49c72 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:59.759136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b7b31-7d15-4aec-9849-5cd952f49c72 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:59.832253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa14e2-a3af-4e2d-82cb-e070cbff7438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:59.833589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa14e2-a3af-4e2d-82cb-e070cbff7438 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:59.834062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa14e2-a3af-4e2d-82cb-e070cbff7438 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:59.834387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa14e2-a3af-4e2d-82cb-e070cbff7438 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:59.835144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa14e2-a3af-4e2d-82cb-e070cbff7438 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:59.837643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa14e2-a3af-4e2d-82cb-e070cbff7438 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:59.864541 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3035/6139] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:11:59 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:11:59.868760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e2637-19e5-4e0c-8d03-52354025615f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:59.881261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e2637-19e5-4e0c-8d03-52354025615f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:59.887145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747f96e-978f-4321-980f-bfb95fa1a4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:59.887792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747f96e-978f-4321-980f-bfb95fa1a4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:59.888071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747f96e-978f-4321-980f-bfb95fa1a4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:59.889100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747f96e-978f-4321-980f-bfb95fa1a4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:59.889586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747f96e-978f-4321-980f-bfb95fa1a4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:59.891229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747f96e-978f-4321-980f-bfb95fa1a4e4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:59.903289 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3105/6140] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:11:59.910754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa14e2-a3af-4e2d-82cb-e070cbff7438 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:11:59.921911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa14e2-a3af-4e2d-82cb-e070cbff7438 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:11:59.928141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aeabc5-c645-4a18-943a-0a283d88ca5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:11:59.928884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aeabc5-c645-4a18-943a-0a283d88ca5b admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:11:59.929295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aeabc5-c645-4a18-943a-0a283d88ca5b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:11:59.930270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aeabc5-c645-4a18-943a-0a283d88ca5b admin admin] PATH_INFO: `/v3/projects/53c9b23ba6ad4b789507a03b98640676/users/643d5939b803416c95e9aa1c381b97fc/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:11:59.948871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aeabc5-c645-4a18-943a-0a283d88ca5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=53c9b23ba6ad4b789507a03b98640676, user_id=643d5939b803416c95e9aa1c381b97fc, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:11:59.952381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aeabc5-c645-4a18-943a-0a283d88ca5b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:11:59.975756 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3036/6141] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:11:59 2022] PUT /identity/v3/projects/53c9b23ba6ad4b789507a03b98640676/users/643d5939b803416c95e9aa1c381b97fc/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:11:59.979959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747f96e-978f-4321-980f-bfb95fa1a4e4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:00.003142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747f96e-978f-4321-980f-bfb95fa1a4e4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:00.054453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:00.075341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c92195-f7a5-471a-88f3-342c4b1877f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:00.075898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c92195-f7a5-471a-88f3-342c4b1877f4 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:00.078106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c92195-f7a5-471a-88f3-342c4b1877f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:00.078396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c92195-f7a5-471a-88f3-342c4b1877f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:00.079089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c92195-f7a5-471a-88f3-342c4b1877f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:00.081721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c92195-f7a5-471a-88f3-342c4b1877f4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:00.095304 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3106/6142] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:12:00.099602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aeabc5-c645-4a18-943a-0a283d88ca5b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:00.126525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aeabc5-c645-4a18-943a-0a283d88ca5b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:00.139799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209738d-8165-41f5-b458-f27db3e1ddfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:00.140564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209738d-8165-41f5-b458-f27db3e1ddfd admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:00.141184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209738d-8165-41f5-b458-f27db3e1ddfd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:00.141539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209738d-8165-41f5-b458-f27db3e1ddfd admin admin] PATH_INFO: `/v3/projects/53c9b23ba6ad4b789507a03b98640676/users/643d5939b803416c95e9aa1c381b97fc/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:00.146092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209738d-8165-41f5-b458-f27db3e1ddfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=53c9b23ba6ad4b789507a03b98640676, user_id=643d5939b803416c95e9aa1c381b97fc, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:00.153069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209738d-8165-41f5-b458-f27db3e1ddfd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:00.195557 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3037/6143] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:12:00 2022] PUT /identity/v3/projects/53c9b23ba6ad4b789507a03b98640676/users/643d5939b803416c95e9aa1c381b97fc/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:00.197109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4d60b-e70a-46fc-ae8c-b0abb6e0d770 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:00.197109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4d60b-e70a-46fc-ae8c-b0abb6e0d770 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:00.197335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4d60b-e70a-46fc-ae8c-b0abb6e0d770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:00.283419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9ea4d60b-e70a-46fc-ae8c-b0abb6e0d770 None None] MFA Rules not processed for user `643d5939b803416c95e9aa1c381b97fc`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:00.299748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9ea4d60b-e70a-46fc-ae8c-b0abb6e0d770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:00.523450 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3107/6144] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:00.530287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209738d-8165-41f5-b458-f27db3e1ddfd admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:00.532214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2209738d-8165-41f5-b458-f27db3e1ddfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:00.618231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:00.649067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209738d-8165-41f5-b458-f27db3e1ddfd admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:00.653433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1524b-c2f5-453b-8b10-7ff6bb1522b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:00.654766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1524b-c2f5-453b-8b10-7ff6bb1522b3 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:00.655119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1524b-c2f5-453b-8b10-7ff6bb1522b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:00.655400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1524b-c2f5-453b-8b10-7ff6bb1522b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:00.668737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1524b-c2f5-453b-8b10-7ff6bb1522b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:00.670671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1524b-c2f5-453b-8b10-7ff6bb1522b3 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:00.700268 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3038/6145] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:12:00 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:12:00.829610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4d60b-e70a-46fc-ae8c-b0abb6e0d770 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:00.849182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4d60b-e70a-46fc-ae8c-b0abb6e0d770 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:00.859661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e214f-cf50-4a0b-817d-ca4b82356c9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:00.861666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e214f-cf50-4a0b-817d-ca4b82356c9a service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:00.861978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e214f-cf50-4a0b-817d-ca4b82356c9a service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:00.862925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e214f-cf50-4a0b-817d-ca4b82356c9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:01.014136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e214f-cf50-4a0b-817d-ca4b82356c9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:01.017427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e214f-cf50-4a0b-817d-ca4b82356c9a service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:01.055213 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3108/6146] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:12:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:12:07.240801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1524b-c2f5-453b-8b10-7ff6bb1522b3 service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:07.347727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1524b-c2f5-453b-8b10-7ff6bb1522b3 service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:07.376682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b565b9-203c-4257-a283-eb17264028e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:07.379127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b565b9-203c-4257-a283-eb17264028e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:07.380559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b565b9-203c-4257-a283-eb17264028e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:07.380840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b565b9-203c-4257-a283-eb17264028e9 admin admin] PATH_INFO: `/v3/users/40f73d17e08d4ca58a70fa38024fc78d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:07.383601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b565b9-203c-4257-a283-eb17264028e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40f73d17e08d4ca58a70fa38024fc78d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:07.390521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b565b9-203c-4257-a283-eb17264028e9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:07.492671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-62b565b9-203c-4257-a283-eb17264028e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40f73d17e08d4ca58a70fa38024fc78d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:07.493609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-62b565b9-203c-4257-a283-eb17264028e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40f73d17e08d4ca58a70fa38024fc78d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:07.494400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-62b565b9-203c-4257-a283-eb17264028e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40f73d17e08d4ca58a70fa38024fc78d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:07.515218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-62b565b9-203c-4257-a283-eb17264028e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40f73d17e08d4ca58a70fa38024fc78d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:07.556929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-62b565b9-203c-4257-a283-eb17264028e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40f73d17e08d4ca58a70fa38024fc78d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:07.583554 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3039/6147] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:12:07 2022] DELETE /identity/v3/users/40f73d17e08d4ca58a70fa38024fc78d => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:07.588924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a16fc4-ac7c-4fd3-a7c3-ab2f4706dc14 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:07.589533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a16fc4-ac7c-4fd3-a7c3-ab2f4706dc14 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:07.589733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a16fc4-ac7c-4fd3-a7c3-ab2f4706dc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:07.634405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-28a16fc4-ac7c-4fd3-a7c3-ab2f4706dc14 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:07.636279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-28a16fc4-ac7c-4fd3-a7c3-ab2f4706dc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:07.741087 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3109/6148] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:07.785103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b565b9-203c-4257-a283-eb17264028e9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:07.786620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-62b565b9-203c-4257-a283-eb17264028e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:07.896375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b565b9-203c-4257-a283-eb17264028e9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:07.906451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53921a46-f762-41f2-8263-e4105b225c30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:07.907115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53921a46-f762-41f2-8263-e4105b225c30 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:07.907395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53921a46-f762-41f2-8263-e4105b225c30 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:07.907666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53921a46-f762-41f2-8263-e4105b225c30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:07.921567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:07.924684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53921a46-f762-41f2-8263-e4105b225c30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:07.927495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53921a46-f762-41f2-8263-e4105b225c30 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:07.940950 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3040/6149] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:12:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:12:09.953057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a16fc4-ac7c-4fd3-a7c3-ab2f4706dc14 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:09.954560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-28a16fc4-ac7c-4fd3-a7c3-ab2f4706dc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:09.965151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:09.973226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a16fc4-ac7c-4fd3-a7c3-ab2f4706dc14 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:09.981826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57e570-5932-4db7-9099-738829355f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:10.000803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57e570-5932-4db7-9099-738829355f56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:10.001294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57e570-5932-4db7-9099-738829355f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:10.001716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57e570-5932-4db7-9099-738829355f56 admin admin] PATH_INFO: `/v3/projects/6034100b31904b83870cd7da3e8dfc0c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:10.004228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f57e570-5932-4db7-9099-738829355f56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6034100b31904b83870cd7da3e8dfc0c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:10.007039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f57e570-5932-4db7-9099-738829355f56 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:10.131232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5f57e570-5932-4db7-9099-738829355f56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6034100b31904b83870cd7da3e8dfc0c', 'request_id': 'req-5f57e570-5932-4db7-9099-738829355f56'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:10.142406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5f57e570-5932-4db7-9099-738829355f56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6034100b31904b83870cd7da3e8dfc0c', 'request_id': 'req-5f57e570-5932-4db7-9099-738829355f56'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:10.151029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5f57e570-5932-4db7-9099-738829355f56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6034100b31904b83870cd7da3e8dfc0c', 'request_id': 'req-5f57e570-5932-4db7-9099-738829355f56'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:10.211195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5f57e570-5932-4db7-9099-738829355f56 admin admin] The token cache is being invalidate because project 6034100b31904b83870cd7da3e8dfc0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:12:10.211677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5f57e570-5932-4db7-9099-738829355f56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:10.220880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5f57e570-5932-4db7-9099-738829355f56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:10.223884 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3110/6150] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:12:09 2022] DELETE /identity/v3/projects/6034100b31904b83870cd7da3e8dfc0c => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:10.227564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53921a46-f762-41f2-8263-e4105b225c30 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:10.246918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53921a46-f762-41f2-8263-e4105b225c30 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:10.375035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b458144-9c1e-4e4c-a27b-8f233cc95eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:10.375956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b458144-9c1e-4e4c-a27b-8f233cc95eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:10.376395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b458144-9c1e-4e4c-a27b-8f233cc95eba admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:10.376828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b458144-9c1e-4e4c-a27b-8f233cc95eba admin admin] PATH_INFO: `/v3/users/643d5939b803416c95e9aa1c381b97fc` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:10.378370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b458144-9c1e-4e4c-a27b-8f233cc95eba admin admin] RBAC: Authorizing `identity:delete_user(user_id=643d5939b803416c95e9aa1c381b97fc)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:10.384656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b458144-9c1e-4e4c-a27b-8f233cc95eba admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:10.500862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9b458144-9c1e-4e4c-a27b-8f233cc95eba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '643d5939b803416c95e9aa1c381b97fc'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:10.502511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9b458144-9c1e-4e4c-a27b-8f233cc95eba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '643d5939b803416c95e9aa1c381b97fc'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:10.503977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9b458144-9c1e-4e4c-a27b-8f233cc95eba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '643d5939b803416c95e9aa1c381b97fc'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:10.518888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9b458144-9c1e-4e4c-a27b-8f233cc95eba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '643d5939b803416c95e9aa1c381b97fc'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:10.541758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9b458144-9c1e-4e4c-a27b-8f233cc95eba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '643d5939b803416c95e9aa1c381b97fc'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:10.589257 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3041/6151] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:12:10 2022] DELETE /identity/v3/users/643d5939b803416c95e9aa1c381b97fc => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:12:11.713923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57e570-5932-4db7-9099-738829355f56 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:11.718367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5f57e570-5932-4db7-9099-738829355f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:11.732732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b458144-9c1e-4e4c-a27b-8f233cc95eba admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:11.734894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9b458144-9c1e-4e4c-a27b-8f233cc95eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:11.783974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57e570-5932-4db7-9099-738829355f56 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:11.879311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b458144-9c1e-4e4c-a27b-8f233cc95eba admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:11.893522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d23c86-1ec9-42e8-a8ee-2eead6605987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:11.895097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d23c86-1ec9-42e8-a8ee-2eead6605987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:11.896094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d23c86-1ec9-42e8-a8ee-2eead6605987 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:11.897791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d23c86-1ec9-42e8-a8ee-2eead6605987 admin admin] PATH_INFO: `/v3/projects/53c9b23ba6ad4b789507a03b98640676` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:11.901655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d23c86-1ec9-42e8-a8ee-2eead6605987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53c9b23ba6ad4b789507a03b98640676)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:11.904242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f239e-b16d-4aaa-8581-0434aebec3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:11.905198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f239e-b16d-4aaa-8581-0434aebec3a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:11.906060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d23c86-1ec9-42e8-a8ee-2eead6605987 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:11.906618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f239e-b16d-4aaa-8581-0434aebec3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:11.907004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f239e-b16d-4aaa-8581-0434aebec3a7 admin admin] PATH_INFO: `/v3/users/0d9b01929f0541c1a94831518c529a71` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:11.910142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f239e-b16d-4aaa-8581-0434aebec3a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d9b01929f0541c1a94831518c529a71)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:11.913432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f239e-b16d-4aaa-8581-0434aebec3a7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:11.946536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:12.015498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7f8f239e-b16d-4aaa-8581-0434aebec3a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d9b01929f0541c1a94831518c529a71'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:12.031028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7f8f239e-b16d-4aaa-8581-0434aebec3a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d9b01929f0541c1a94831518c529a71'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:12.035669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-56d23c86-1ec9-42e8-a8ee-2eead6605987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53c9b23ba6ad4b789507a03b98640676', 'request_id': 'req-56d23c86-1ec9-42e8-a8ee-2eead6605987'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:12.050328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7f8f239e-b16d-4aaa-8581-0434aebec3a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d9b01929f0541c1a94831518c529a71'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:12.051275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7f8f239e-b16d-4aaa-8581-0434aebec3a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d9b01929f0541c1a94831518c529a71'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:12.062252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-56d23c86-1ec9-42e8-a8ee-2eead6605987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53c9b23ba6ad4b789507a03b98640676', 'request_id': 'req-56d23c86-1ec9-42e8-a8ee-2eead6605987'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:12.066734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7f8f239e-b16d-4aaa-8581-0434aebec3a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d9b01929f0541c1a94831518c529a71'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:12.072863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-56d23c86-1ec9-42e8-a8ee-2eead6605987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53c9b23ba6ad4b789507a03b98640676', 'request_id': 'req-56d23c86-1ec9-42e8-a8ee-2eead6605987'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:12.076684 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3111/6152] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:12:11 2022] DELETE /identity/v3/users/0d9b01929f0541c1a94831518c529a71 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:12.081246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28049dcc-8fae-44a8-afd9-990be4f971d4 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:12.082226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28049dcc-8fae-44a8-afd9-990be4f971d4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:12.082226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28049dcc-8fae-44a8-afd9-990be4f971d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:12.105489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-56d23c86-1ec9-42e8-a8ee-2eead6605987 admin admin] The token cache is being invalidate because project 53c9b23ba6ad4b789507a03b98640676 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:12:12.105937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-56d23c86-1ec9-42e8-a8ee-2eead6605987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:12.106924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-56d23c86-1ec9-42e8-a8ee-2eead6605987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:12.108553 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3042/6153] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:12:11 2022] DELETE /identity/v3/projects/53c9b23ba6ad4b789507a03b98640676 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:12.111401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:12.113332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c73ab-4c7b-4f1d-ab9a-3366018b7048 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:12.113639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c73ab-4c7b-4f1d-ab9a-3366018b7048 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:12.114269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c73ab-4c7b-4f1d-ab9a-3366018b7048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:12.124376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-28049dcc-8fae-44a8-afd9-990be4f971d4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:12.127008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-28049dcc-8fae-44a8-afd9-990be4f971d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:12.161513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4e6c73ab-4c7b-4f1d-ab9a-3366018b7048 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:12.163517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4e6c73ab-4c7b-4f1d-ab9a-3366018b7048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:12.238949 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3112/6154] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:12.251869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28049dcc-8fae-44a8-afd9-990be4f971d4 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:12.254290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-28049dcc-8fae-44a8-afd9-990be4f971d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:12.302871 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3043/6155] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:12.312491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e9432-2925-487d-85e3-dab6ba0f8207 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:12.316599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e9432-2925-487d-85e3-dab6ba0f8207 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:12.317025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e9432-2925-487d-85e3-dab6ba0f8207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:12.334317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28049dcc-8fae-44a8-afd9-990be4f971d4 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:12.348144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-408e9432-2925-487d-85e3-dab6ba0f8207 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:12.350470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-408e9432-2925-487d-85e3-dab6ba0f8207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:12.359205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a465b4-f1c4-4fdb-be33-be110b03b339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:12.360111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a465b4-f1c4-4fdb-be33-be110b03b339 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:12.360397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a465b4-f1c4-4fdb-be33-be110b03b339 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:12.360679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a465b4-f1c4-4fdb-be33-be110b03b339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:12.364371 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3044/6156] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:12.369336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e9432-2925-487d-85e3-dab6ba0f8207 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:12.380191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a465b4-f1c4-4fdb-be33-be110b03b339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:12.393343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a465b4-f1c4-4fdb-be33-be110b03b339 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:12.394479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e9432-2925-487d-85e3-dab6ba0f8207 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:12.400264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051d945-962c-4a93-b05d-6af1fccb77c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:12.400929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051d945-962c-4a93-b05d-6af1fccb77c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:12.401217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051d945-962c-4a93-b05d-6af1fccb77c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:12.401491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051d945-962c-4a93-b05d-6af1fccb77c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:12.403424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4051d945-962c-4a93-b05d-6af1fccb77c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:12:12.403903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4051d945-962c-4a93-b05d-6af1fccb77c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:12.406185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4051d945-962c-4a93-b05d-6af1fccb77c7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:12.409531 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3113/6157] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:12:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:12:12.433162 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3045/6158] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:12:12.434050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a465b4-f1c4-4fdb-be33-be110b03b339 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:12.449580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a465b4-f1c4-4fdb-be33-be110b03b339 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:12.460080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aab0b1-16e1-4952-b0e5-61988951ea8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:12.460882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aab0b1-16e1-4952-b0e5-61988951ea8e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:12.461235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aab0b1-16e1-4952-b0e5-61988951ea8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:12.461663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aab0b1-16e1-4952-b0e5-61988951ea8e admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:12.462385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aab0b1-16e1-4952-b0e5-61988951ea8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:12.464424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aab0b1-16e1-4952-b0e5-61988951ea8e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:12.499881 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3114/6159] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:12:12 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:12:12.503673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051d945-962c-4a93-b05d-6af1fccb77c7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:12.515531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051d945-962c-4a93-b05d-6af1fccb77c7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:12.587341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc9cc3-d53a-4d1a-8830-9817e1409746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:12.591740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc9cc3-d53a-4d1a-8830-9817e1409746 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:12.592302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc9cc3-d53a-4d1a-8830-9817e1409746 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:12.592585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc9cc3-d53a-4d1a-8830-9817e1409746 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:12.593248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc9cc3-d53a-4d1a-8830-9817e1409746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:12.595240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc9cc3-d53a-4d1a-8830-9817e1409746 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:12.620183 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3046/6160] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:12:12 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:12:12.624460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aab0b1-16e1-4952-b0e5-61988951ea8e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:12.637035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aab0b1-16e1-4952-b0e5-61988951ea8e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:12.647239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c97857-84c1-4e7d-82a8-2f6d62fb47ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:12.649049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c97857-84c1-4e7d-82a8-2f6d62fb47ac admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:12.649351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c97857-84c1-4e7d-82a8-2f6d62fb47ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:12.649635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c97857-84c1-4e7d-82a8-2f6d62fb47ac admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:12.650302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c97857-84c1-4e7d-82a8-2f6d62fb47ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:12.653198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c97857-84c1-4e7d-82a8-2f6d62fb47ac admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:12.669098 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3115/6161] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:12:12.672811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc9cc3-d53a-4d1a-8830-9817e1409746 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:12.685340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc9cc3-d53a-4d1a-8830-9817e1409746 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:12.691486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc09b7-6d38-4fed-9ecb-1bf7b15ab51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:12.692706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc09b7-6d38-4fed-9ecb-1bf7b15ab51d admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:12.692706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc09b7-6d38-4fed-9ecb-1bf7b15ab51d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:12.693008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc09b7-6d38-4fed-9ecb-1bf7b15ab51d admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:12.693735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc09b7-6d38-4fed-9ecb-1bf7b15ab51d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:12.695863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc09b7-6d38-4fed-9ecb-1bf7b15ab51d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:12.712492 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3047/6162] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:12:12.717498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c97857-84c1-4e7d-82a8-2f6d62fb47ac admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:12.730747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c97857-84c1-4e7d-82a8-2f6d62fb47ac admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:12.736274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd9481-f79c-435a-8bec-d077c2c2c74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:12.737071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd9481-f79c-435a-8bec-d077c2c2c74e admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:12.738265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd9481-f79c-435a-8bec-d077c2c2c74e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:12.738265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd9481-f79c-435a-8bec-d077c2c2c74e admin admin] PATH_INFO: `/v3/projects/55864f798646443bbdfe30f8b9fbefb1/users/78cc9dbf1b344a35b9be09ad32549a62/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:12.749275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc09b7-6d38-4fed-9ecb-1bf7b15ab51d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:12.751144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a2cc09b7-6d38-4fed-9ecb-1bf7b15ab51d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:12.770231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:12.775362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc09b7-6d38-4fed-9ecb-1bf7b15ab51d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:12.782061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd9481-f79c-435a-8bec-d077c2c2c74e admin admin] RBAC: Authorizing `identity:create_grant(project_id=55864f798646443bbdfe30f8b9fbefb1, user_id=78cc9dbf1b344a35b9be09ad32549a62, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:12.784037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b1a10-c9fa-4ebf-b4ba-272ede6a3398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:12.785182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b1a10-c9fa-4ebf-b4ba-272ede6a3398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:12.785912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b1a10-c9fa-4ebf-b4ba-272ede6a3398 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:12.786760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b1a10-c9fa-4ebf-b4ba-272ede6a3398 admin admin] PATH_INFO: `/v3/projects/1e00ee15f5e1440cb75594f06cfb9fe6` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:12.788694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd9481-f79c-435a-8bec-d077c2c2c74e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:12.789357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b1a10-c9fa-4ebf-b4ba-272ede6a3398 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e00ee15f5e1440cb75594f06cfb9fe6)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:12.792261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b1a10-c9fa-4ebf-b4ba-272ede6a3398 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:12.809515 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3116/6163] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:12:12 2022] PUT /identity/v3/projects/55864f798646443bbdfe30f8b9fbefb1/users/78cc9dbf1b344a35b9be09ad32549a62/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:12.816479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d037692-ad21-49cc-89c7-60894f9cb74b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:12.817006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d037692-ad21-49cc-89c7-60894f9cb74b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:12.817497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d037692-ad21-49cc-89c7-60894f9cb74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:12.846153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2d037692-ad21-49cc-89c7-60894f9cb74b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:12.848363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2d037692-ad21-49cc-89c7-60894f9cb74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:12.887037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f55b1a10-c9fa-4ebf-b4ba-272ede6a3398 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e00ee15f5e1440cb75594f06cfb9fe6', 'request_id': 'req-f55b1a10-c9fa-4ebf-b4ba-272ede6a3398'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:12.898396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f55b1a10-c9fa-4ebf-b4ba-272ede6a3398 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e00ee15f5e1440cb75594f06cfb9fe6', 'request_id': 'req-f55b1a10-c9fa-4ebf-b4ba-272ede6a3398'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:12.907606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f55b1a10-c9fa-4ebf-b4ba-272ede6a3398 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e00ee15f5e1440cb75594f06cfb9fe6', 'request_id': 'req-f55b1a10-c9fa-4ebf-b4ba-272ede6a3398'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:12.926290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f55b1a10-c9fa-4ebf-b4ba-272ede6a3398 admin admin] The token cache is being invalidate because project 1e00ee15f5e1440cb75594f06cfb9fe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:12:12.926615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f55b1a10-c9fa-4ebf-b4ba-272ede6a3398 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:12.931637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f55b1a10-c9fa-4ebf-b4ba-272ede6a3398 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:12.938673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:12.939375 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3048/6164] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:12:12 2022] DELETE /identity/v3/projects/1e00ee15f5e1440cb75594f06cfb9fe6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:12:12.941270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab6cac-8141-43a3-81c3-d2414f634e69 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:12.941558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab6cac-8141-43a3-81c3-d2414f634e69 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:12.941916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab6cac-8141-43a3-81c3-d2414f634e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:12.959523 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3117/6165] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:12:12.968972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d037692-ad21-49cc-89c7-60894f9cb74b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:12.971129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2d037692-ad21-49cc-89c7-60894f9cb74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:12.980711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3fab6cac-8141-43a3-81c3-d2414f634e69 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:12.983294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3fab6cac-8141-43a3-81c3-d2414f634e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:13.006418 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3049/6166] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:13.018964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15405c5-052a-4023-9840-d60d82d4b3c7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:13.019472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15405c5-052a-4023-9840-d60d82d4b3c7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:13.019854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15405c5-052a-4023-9840-d60d82d4b3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:13.084242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d15405c5-052a-4023-9840-d60d82d4b3c7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:13.085721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d15405c5-052a-4023-9840-d60d82d4b3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:13.090032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d037692-ad21-49cc-89c7-60894f9cb74b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:13.097217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150b742-7702-4c95-8376-86769296078a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:13.098250 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3050/6167] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:13.098781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150b742-7702-4c95-8376-86769296078a service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:13.099149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15405c5-052a-4023-9840-d60d82d4b3c7 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:13.099620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150b742-7702-4c95-8376-86769296078a service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:13.100350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150b742-7702-4c95-8376-86769296078a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:13.100635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d15405c5-052a-4023-9840-d60d82d4b3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:13.119435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150b742-7702-4c95-8376-86769296078a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:13.121330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15405c5-052a-4023-9840-d60d82d4b3c7 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:13.121510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150b742-7702-4c95-8376-86769296078a service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:13.128485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768dd196-2296-4b30-a4d1-fb607d9a6e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:13.129438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768dd196-2296-4b30-a4d1-fb607d9a6e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:13.129877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768dd196-2296-4b30-a4d1-fb607d9a6e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:13.130327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768dd196-2296-4b30-a4d1-fb607d9a6e16 admin admin] PATH_INFO: `/v3/users/f91d413b46d44b2ba9257de2f6b54955` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:13.132173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768dd196-2296-4b30-a4d1-fb607d9a6e16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f91d413b46d44b2ba9257de2f6b54955)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:13.134838 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3118/6168] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:12:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:12:13.136204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150b742-7702-4c95-8376-86769296078a service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:13.149199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768dd196-2296-4b30-a4d1-fb607d9a6e16 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:13.150912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150b742-7702-4c95-8376-86769296078a service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:13.166520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc3e48-ca1e-40d9-803b-368e4839b452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:13.167357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc3e48-ca1e-40d9-803b-368e4839b452 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:13.168315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc3e48-ca1e-40d9-803b-368e4839b452 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:13.168315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc3e48-ca1e-40d9-803b-368e4839b452 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:13.168924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc3e48-ca1e-40d9-803b-368e4839b452 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:12:13.169801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc3e48-ca1e-40d9-803b-368e4839b452 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:13.172464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc3e48-ca1e-40d9-803b-368e4839b452 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:13.194656 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3119/6169] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:12:13.196063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc3e48-ca1e-40d9-803b-368e4839b452 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:13.211857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc3e48-ca1e-40d9-803b-368e4839b452 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:13.221114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003e5d1-5371-4c11-92db-239e1825c894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:13.221796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003e5d1-5371-4c11-92db-239e1825c894 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:13.222106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003e5d1-5371-4c11-92db-239e1825c894 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:13.222930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003e5d1-5371-4c11-92db-239e1825c894 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:13.223739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003e5d1-5371-4c11-92db-239e1825c894 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:13.226129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003e5d1-5371-4c11-92db-239e1825c894 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:13.248510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-768dd196-2296-4b30-a4d1-fb607d9a6e16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f91d413b46d44b2ba9257de2f6b54955'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:13.249677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-768dd196-2296-4b30-a4d1-fb607d9a6e16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f91d413b46d44b2ba9257de2f6b54955'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:13.250471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-768dd196-2296-4b30-a4d1-fb607d9a6e16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f91d413b46d44b2ba9257de2f6b54955'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:13.278747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-768dd196-2296-4b30-a4d1-fb607d9a6e16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f91d413b46d44b2ba9257de2f6b54955'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:13.287314 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3120/6170] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:12:13 2022] POST /identity/v3/projects => generated 379 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:12:13.293192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003e5d1-5371-4c11-92db-239e1825c894 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:13.297034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3003e5d1-5371-4c11-92db-239e1825c894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:13.299580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-768dd196-2296-4b30-a4d1-fb607d9a6e16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f91d413b46d44b2ba9257de2f6b54955'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:13.322336 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3051/6171] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:12:13 2022] DELETE /identity/v3/users/f91d413b46d44b2ba9257de2f6b54955 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:13.396686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003e5d1-5371-4c11-92db-239e1825c894 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:13.478006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9bd63-da0d-460f-9c96-993e54b1490b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:13.479027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9bd63-da0d-460f-9c96-993e54b1490b admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:13.479400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9bd63-da0d-460f-9c96-993e54b1490b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:13.479813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9bd63-da0d-460f-9c96-993e54b1490b admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:13.480547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9bd63-da0d-460f-9c96-993e54b1490b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:13.483696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9bd63-da0d-460f-9c96-993e54b1490b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:13.505016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:13.520096 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3121/6172] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:12:13 2022] POST /identity/v3/users => generated 290 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:12:13.521216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768dd196-2296-4b30-a4d1-fb607d9a6e16 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:13.544921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768dd196-2296-4b30-a4d1-fb607d9a6e16 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:13.565527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949f492-df24-4598-a12e-b49ae5ff2045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:13.566548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949f492-df24-4598-a12e-b49ae5ff2045 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:13.566835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949f492-df24-4598-a12e-b49ae5ff2045 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:13.567115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949f492-df24-4598-a12e-b49ae5ff2045 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:13.568002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949f492-df24-4598-a12e-b49ae5ff2045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:13.569434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949f492-df24-4598-a12e-b49ae5ff2045 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:13.603815 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3052/6173] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:12:13.608344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9bd63-da0d-460f-9c96-993e54b1490b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:13.619904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9bd63-da0d-460f-9c96-993e54b1490b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:13.638947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e625a-4d03-4a07-ad1a-b2f5a0f062da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:13.656768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e625a-4d03-4a07-ad1a-b2f5a0f062da admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:13.657149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e625a-4d03-4a07-ad1a-b2f5a0f062da admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:13.657611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e625a-4d03-4a07-ad1a-b2f5a0f062da admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:13.658397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e625a-4d03-4a07-ad1a-b2f5a0f062da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:13.660337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e625a-4d03-4a07-ad1a-b2f5a0f062da admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:13.682542 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3122/6174] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:12:13.688268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949f492-df24-4598-a12e-b49ae5ff2045 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:13.706474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949f492-df24-4598-a12e-b49ae5ff2045 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:13.713514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405896b-37a8-45ca-9985-4bcc44c4178e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:13.714196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405896b-37a8-45ca-9985-4bcc44c4178e admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:13.714479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405896b-37a8-45ca-9985-4bcc44c4178e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:13.714859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405896b-37a8-45ca-9985-4bcc44c4178e admin admin] PATH_INFO: `/v3/projects/60d493f07a5148aba21c9a903ec5eb68/users/269b477a369347f991547be38db227bd/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:13.750324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405896b-37a8-45ca-9985-4bcc44c4178e admin admin] RBAC: Authorizing `identity:create_grant(project_id=60d493f07a5148aba21c9a903ec5eb68, user_id=269b477a369347f991547be38db227bd, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:13.754062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405896b-37a8-45ca-9985-4bcc44c4178e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:13.783160 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3053/6175] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:12:13 2022] PUT /identity/v3/projects/60d493f07a5148aba21c9a903ec5eb68/users/269b477a369347f991547be38db227bd/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:13.787989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181147a-57af-4e80-8598-a2b07ae0c869 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:13.788565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181147a-57af-4e80-8598-a2b07ae0c869 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:13.788682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181147a-57af-4e80-8598-a2b07ae0c869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:13.838423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405896b-37a8-45ca-9985-4bcc44c4178e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:13.840727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1405896b-37a8-45ca-9985-4bcc44c4178e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:13.893084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2181147a-57af-4e80-8598-a2b07ae0c869 None None] MFA Rules not processed for user `269b477a369347f991547be38db227bd`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:13.894348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2181147a-57af-4e80-8598-a2b07ae0c869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:13.906344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:14.076303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405896b-37a8-45ca-9985-4bcc44c4178e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:14.088860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2cbce-4875-43d3-b3da-516ba0c32bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:14.089569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2cbce-4875-43d3-b3da-516ba0c32bdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:14.089915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2cbce-4875-43d3-b3da-516ba0c32bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:14.090204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2cbce-4875-43d3-b3da-516ba0c32bdd admin admin] PATH_INFO: `/v3/projects/e763f5a0a739414b96c9346b82154f41` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:14.092123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2cbce-4875-43d3-b3da-516ba0c32bdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e763f5a0a739414b96c9346b82154f41)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:14.094387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2cbce-4875-43d3-b3da-516ba0c32bdd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:14.125075 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3123/6176] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:14.155454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181147a-57af-4e80-8598-a2b07ae0c869 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:14.156910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2181147a-57af-4e80-8598-a2b07ae0c869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:14.195148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-63e2cbce-4875-43d3-b3da-516ba0c32bdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e763f5a0a739414b96c9346b82154f41', 'request_id': 'req-63e2cbce-4875-43d3-b3da-516ba0c32bdd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:14.217239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-63e2cbce-4875-43d3-b3da-516ba0c32bdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e763f5a0a739414b96c9346b82154f41', 'request_id': 'req-63e2cbce-4875-43d3-b3da-516ba0c32bdd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:14.227231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-63e2cbce-4875-43d3-b3da-516ba0c32bdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e763f5a0a739414b96c9346b82154f41', 'request_id': 'req-63e2cbce-4875-43d3-b3da-516ba0c32bdd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:14.256231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-63e2cbce-4875-43d3-b3da-516ba0c32bdd admin admin] The token cache is being invalidate because project e763f5a0a739414b96c9346b82154f41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:12:14.257955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-63e2cbce-4875-43d3-b3da-516ba0c32bdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:14.259857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-63e2cbce-4875-43d3-b3da-516ba0c32bdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:14.263218 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3054/6177] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:12:13 2022] DELETE /identity/v3/projects/e763f5a0a739414b96c9346b82154f41 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:12:14.269254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee05a67-d873-495a-bd59-6cc0b236d3ff None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:14.270000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee05a67-d873-495a-bd59-6cc0b236d3ff None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:14.270460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee05a67-d873-495a-bd59-6cc0b236d3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:14.304428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181147a-57af-4e80-8598-a2b07ae0c869 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:14.330784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-eee05a67-d873-495a-bd59-6cc0b236d3ff None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:14.336180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-eee05a67-d873-495a-bd59-6cc0b236d3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:14.420055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648ea52-ab6f-48d8-bb4d-4508be807669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:14.420688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648ea52-ab6f-48d8-bb4d-4508be807669 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:14.421335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648ea52-ab6f-48d8-bb4d-4508be807669 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:14.421459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648ea52-ab6f-48d8-bb4d-4508be807669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:14.424474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0648ea52-ab6f-48d8-bb4d-4508be807669 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:14.477407 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3055/6178] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:14.485970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ce1d9-7171-46e0-bcbc-9dbecf1e2e43 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:14.486281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ce1d9-7171-46e0-bcbc-9dbecf1e2e43 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:14.486596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ce1d9-7171-46e0-bcbc-9dbecf1e2e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:14.511687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-948ce1d9-7171-46e0-bcbc-9dbecf1e2e43 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:14.513563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-948ce1d9-7171-46e0-bcbc-9dbecf1e2e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:14.519768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:14.523465 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3056/6179] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:14.527638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ce1d9-7171-46e0-bcbc-9dbecf1e2e43 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:14.542970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ce1d9-7171-46e0-bcbc-9dbecf1e2e43 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:14.545594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648ea52-ab6f-48d8-bb4d-4508be807669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:14.548183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648ea52-ab6f-48d8-bb4d-4508be807669 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:14.548790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fa606-8fe9-40c8-a54d-da229ef43369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:14.549454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fa606-8fe9-40c8-a54d-da229ef43369 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:14.549735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fa606-8fe9-40c8-a54d-da229ef43369 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:14.550323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fa606-8fe9-40c8-a54d-da229ef43369 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:14.551066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34fa606-8fe9-40c8-a54d-da229ef43369 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:12:14.551377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34fa606-8fe9-40c8-a54d-da229ef43369 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:14.553042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34fa606-8fe9-40c8-a54d-da229ef43369 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:14.568376 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3124/6180] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:12:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:12:14.569339 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3057/6181] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:12:14.574235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648ea52-ab6f-48d8-bb4d-4508be807669 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:14.586928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648ea52-ab6f-48d8-bb4d-4508be807669 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:14.592502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca245bf-ee77-4b83-bb46-0e13d1682ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:14.593589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca245bf-ee77-4b83-bb46-0e13d1682ebc admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:14.593589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca245bf-ee77-4b83-bb46-0e13d1682ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:14.593808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca245bf-ee77-4b83-bb46-0e13d1682ebc admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:14.594289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca245bf-ee77-4b83-bb46-0e13d1682ebc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:14.596519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca245bf-ee77-4b83-bb46-0e13d1682ebc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:14.634860 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3125/6182] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:12:14 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:12:14.641162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fa606-8fe9-40c8-a54d-da229ef43369 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:14.649960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5fa01-bf52-4331-824b-46415ed16447 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:14.650541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5fa01-bf52-4331-824b-46415ed16447 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:14.651071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5fa01-bf52-4331-824b-46415ed16447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:14.677874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fa606-8fe9-40c8-a54d-da229ef43369 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:14.704389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:14.722578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a2e5fa01-bf52-4331-824b-46415ed16447 None None] MFA Rules not processed for user `a2745d6923e347939e2c67015aefadda`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:14.724117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a2e5fa01-bf52-4331-824b-46415ed16447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:14.794965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006a74d-f1d5-4117-b3a3-80a89bf93ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:14.795801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006a74d-f1d5-4117-b3a3-80a89bf93ec2 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:14.796386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006a74d-f1d5-4117-b3a3-80a89bf93ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:14.796944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006a74d-f1d5-4117-b3a3-80a89bf93ec2 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:14.797696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006a74d-f1d5-4117-b3a3-80a89bf93ec2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:14.799789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006a74d-f1d5-4117-b3a3-80a89bf93ec2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:14.838664 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3058/6183] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:12:14 2022] POST /identity/v3/users => generated 287 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:12:14.843002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006a74d-f1d5-4117-b3a3-80a89bf93ec2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:14.863186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006a74d-f1d5-4117-b3a3-80a89bf93ec2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:14.874762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef527742-afe2-4003-b9cf-f61dd9fcb17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:14.875729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef527742-afe2-4003-b9cf-f61dd9fcb17b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:14.876107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef527742-afe2-4003-b9cf-f61dd9fcb17b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:14.876637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef527742-afe2-4003-b9cf-f61dd9fcb17b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:14.876823 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3126/6184] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:14.887429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e888110-c2a2-48e5-9461-f8640eb5ce9c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:14.887623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef527742-afe2-4003-b9cf-f61dd9fcb17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:14.888691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e888110-c2a2-48e5-9461-f8640eb5ce9c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:14.889274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e888110-c2a2-48e5-9461-f8640eb5ce9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:14.890749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef527742-afe2-4003-b9cf-f61dd9fcb17b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:14.904962 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3059/6185] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:12:14.908643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef527742-afe2-4003-b9cf-f61dd9fcb17b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:14.923531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7e888110-c2a2-48e5-9461-f8640eb5ce9c None None] MFA Rules not processed for user `a2745d6923e347939e2c67015aefadda`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:14.925650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7e888110-c2a2-48e5-9461-f8640eb5ce9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:14.936741 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3127/6186] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:14.940790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef527742-afe2-4003-b9cf-f61dd9fcb17b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:14.949158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e888110-c2a2-48e5-9461-f8640eb5ce9c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:14.951422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7e888110-c2a2-48e5-9461-f8640eb5ce9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:14.956912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906de39-3cb9-4990-ac66-fe6a9158c280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:14.957542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906de39-3cb9-4990-ac66-fe6a9158c280 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:14.957821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906de39-3cb9-4990-ac66-fe6a9158c280 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:14.958091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906de39-3cb9-4990-ac66-fe6a9158c280 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:14.958860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906de39-3cb9-4990-ac66-fe6a9158c280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:14.962704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906de39-3cb9-4990-ac66-fe6a9158c280 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:14.993145 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3060/6187] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:12:14.999504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906de39-3cb9-4990-ac66-fe6a9158c280 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:15.037894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906de39-3cb9-4990-ac66-fe6a9158c280 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:15.060574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1b995-050d-41f1-85c2-604853e4ce3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:15.061292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1b995-050d-41f1-85c2-604853e4ce3e admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:15.061574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1b995-050d-41f1-85c2-604853e4ce3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:15.061852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1b995-050d-41f1-85c2-604853e4ce3e admin admin] PATH_INFO: `/v3/projects/6548b7e8d5a244f58fabbd12126808ac/users/3ba55fd90c52436d96ceaa933cf78a26/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:15.078956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e888110-c2a2-48e5-9461-f8640eb5ce9c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:15.103198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e1b995-050d-41f1-85c2-604853e4ce3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6548b7e8d5a244f58fabbd12126808ac, user_id=3ba55fd90c52436d96ceaa933cf78a26, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:15.104217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8ec5e-da2d-4709-bfe0-290d73936e5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:15.105784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8ec5e-da2d-4709-bfe0-290d73936e5c service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:15.106081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8ec5e-da2d-4709-bfe0-290d73936e5c service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:15.107014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8ec5e-da2d-4709-bfe0-290d73936e5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:15.119474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e1b995-050d-41f1-85c2-604853e4ce3e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:15.132233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8ec5e-da2d-4709-bfe0-290d73936e5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:15.137488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8ec5e-da2d-4709-bfe0-290d73936e5c service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:15.147760 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3061/6188] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:12:14 2022] PUT /identity/v3/projects/6548b7e8d5a244f58fabbd12126808ac/users/3ba55fd90c52436d96ceaa933cf78a26/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:12:15.147760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92d74c-7aa8-4723-b408-2e511903fdc7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:15.147760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92d74c-7aa8-4723-b408-2e511903fdc7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:15.147760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92d74c-7aa8-4723-b408-2e511903fdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:15.151706 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3128/6189] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:12:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:12:15.153123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68411ed3-abd5-4b47-896b-52092f2d7716 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:15.154061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68411ed3-abd5-4b47-896b-52092f2d7716 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:15.154420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68411ed3-abd5-4b47-896b-52092f2d7716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:15.178309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-68411ed3-abd5-4b47-896b-52092f2d7716 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:15.180064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-68411ed3-abd5-4b47-896b-52092f2d7716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:15.228479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:15.265371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ba92d74c-7aa8-4723-b408-2e511903fdc7 None None] MFA Rules not processed for user `78cc9dbf1b344a35b9be09ad32549a62`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:15.266942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ba92d74c-7aa8-4723-b408-2e511903fdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:15.348191 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3129/6190] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:15.386437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68411ed3-abd5-4b47-896b-52092f2d7716 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:15.389087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-68411ed3-abd5-4b47-896b-52092f2d7716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:15.447497 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3062/6191] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:15.448441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92d74c-7aa8-4723-b408-2e511903fdc7 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:15.451005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ba92d74c-7aa8-4723-b408-2e511903fdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:15.485371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92d74c-7aa8-4723-b408-2e511903fdc7 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:15.502279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fef1e-f150-4476-b584-e4ac66aca4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:15.502938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fef1e-f150-4476-b584-e4ac66aca4e1 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:15.503224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fef1e-f150-4476-b584-e4ac66aca4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:15.508065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fef1e-f150-4476-b584-e4ac66aca4e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:15.509351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7fef1e-f150-4476-b584-e4ac66aca4e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:15.517123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7fef1e-f150-4476-b584-e4ac66aca4e1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:15.552495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68411ed3-abd5-4b47-896b-52092f2d7716 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:15.568434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc2521-049a-48af-92fb-42437616ccf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:15.569122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc2521-049a-48af-92fb-42437616ccf5 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:15.569418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc2521-049a-48af-92fb-42437616ccf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:15.570132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc2521-049a-48af-92fb-42437616ccf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:15.607280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc2521-049a-48af-92fb-42437616ccf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:15.609193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc2521-049a-48af-92fb-42437616ccf5 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:15.613200 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3063/6192] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:12:15 2022] POST /identity/v3/projects => generated 371 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:12:15.617793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fef1e-f150-4476-b584-e4ac66aca4e1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:15.653920 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3130/6193] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:12:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:12:15.729455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:15.835514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fef1e-f150-4476-b584-e4ac66aca4e1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:15.853292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bc176-9186-49f4-a571-b4bd33d70c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:15.854166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bc176-9186-49f4-a571-b4bd33d70c50 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:15.854457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bc176-9186-49f4-a571-b4bd33d70c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:15.854768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bc176-9186-49f4-a571-b4bd33d70c50 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:15.855417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6bc176-9186-49f4-a571-b4bd33d70c50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:15.868111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6bc176-9186-49f4-a571-b4bd33d70c50 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:15.931102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc2521-049a-48af-92fb-42437616ccf5 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:15.931918 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3064/6194] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:12:15 2022] POST /identity/v3/users => generated 292 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:12:15.949430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc2521-049a-48af-92fb-42437616ccf5 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:15.964881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057eb35-9e03-4878-aa8d-a7f2659b7a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:15.965515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057eb35-9e03-4878-aa8d-a7f2659b7a3f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:15.965800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057eb35-9e03-4878-aa8d-a7f2659b7a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:15.966109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057eb35-9e03-4878-aa8d-a7f2659b7a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:15.966794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057eb35-9e03-4878-aa8d-a7f2659b7a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:15.976282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057eb35-9e03-4878-aa8d-a7f2659b7a3f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:16.017521 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3131/6195] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:12:16.023906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bc176-9186-49f4-a571-b4bd33d70c50 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:16.045562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bc176-9186-49f4-a571-b4bd33d70c50 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:16.061350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372d202-f989-4698-adec-7c05f87ab4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:16.062767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372d202-f989-4698-adec-7c05f87ab4b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:16.063371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372d202-f989-4698-adec-7c05f87ab4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:16.063868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372d202-f989-4698-adec-7c05f87ab4b2 admin admin] PATH_INFO: `/v3/projects/316a7457d230463dbea005d8ba1d4019/users/e4c5c1b5f5e74dd1a8defcdd94c73da4/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:16.118058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3372d202-f989-4698-adec-7c05f87ab4b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=316a7457d230463dbea005d8ba1d4019, user_id=e4c5c1b5f5e74dd1a8defcdd94c73da4, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:16.121449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3372d202-f989-4698-adec-7c05f87ab4b2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:16.161608 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3065/6196] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:12:16 2022] PUT /identity/v3/projects/316a7457d230463dbea005d8ba1d4019/users/e4c5c1b5f5e74dd1a8defcdd94c73da4/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:16.162016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057eb35-9e03-4878-aa8d-a7f2659b7a3f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:16.179692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057eb35-9e03-4878-aa8d-a7f2659b7a3f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:16.312887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7b3ed-5ca6-4bd7-b49c-1bd477a3ffe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:16.313663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7b3ed-5ca6-4bd7-b49c-1bd477a3ffe0 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:16.313948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7b3ed-5ca6-4bd7-b49c-1bd477a3ffe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:16.314224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7b3ed-5ca6-4bd7-b49c-1bd477a3ffe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:16.314907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7b3ed-5ca6-4bd7-b49c-1bd477a3ffe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:16.316522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7b3ed-5ca6-4bd7-b49c-1bd477a3ffe0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:16.339836 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3132/6197] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:12:16.341649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372d202-f989-4698-adec-7c05f87ab4b2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:16.354393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372d202-f989-4698-adec-7c05f87ab4b2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:16.367816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a1e89-ea6f-491e-99b0-809e97c2d7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:16.368576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a1e89-ea6f-491e-99b0-809e97c2d7c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:16.368793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a1e89-ea6f-491e-99b0-809e97c2d7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:16.369073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a1e89-ea6f-491e-99b0-809e97c2d7c7 admin admin] PATH_INFO: `/v3/projects/316a7457d230463dbea005d8ba1d4019/users/e4c5c1b5f5e74dd1a8defcdd94c73da4/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:16.372708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a1e89-ea6f-491e-99b0-809e97c2d7c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=316a7457d230463dbea005d8ba1d4019, user_id=e4c5c1b5f5e74dd1a8defcdd94c73da4, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:16.375451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a1e89-ea6f-491e-99b0-809e97c2d7c7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:16.412217 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3066/6198] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:12:16 2022] PUT /identity/v3/projects/316a7457d230463dbea005d8ba1d4019/users/e4c5c1b5f5e74dd1a8defcdd94c73da4/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:21.555840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcb71d-13ca-45a6-9d5c-f74d8096f1ac None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:21.556459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcb71d-13ca-45a6-9d5c-f74d8096f1ac None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:21.556459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcb71d-13ca-45a6-9d5c-f74d8096f1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:21.632850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-aebcb71d-13ca-45a6-9d5c-f74d8096f1ac None None] MFA Rules not processed for user `e4c5c1b5f5e74dd1a8defcdd94c73da4`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:21.634117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-aebcb71d-13ca-45a6-9d5c-f74d8096f1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:21.734538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:21.792411 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3133/6199] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:21.802690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcf5b6-8ca4-4bc6-a501-7a6164ce0955 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:21.803188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcf5b6-8ca4-4bc6-a501-7a6164ce0955 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:21.803334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcf5b6-8ca4-4bc6-a501-7a6164ce0955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:21.837128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7cdcf5b6-8ca4-4bc6-a501-7a6164ce0955 None None] MFA Rules not processed for user `78cc9dbf1b344a35b9be09ad32549a62`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:21.838357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7cdcf5b6-8ca4-4bc6-a501-7a6164ce0955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:21.994660 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3067/6200] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:22.004504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcb71d-13ca-45a6-9d5c-f74d8096f1ac None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:22.231365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcb71d-13ca-45a6-9d5c-f74d8096f1ac None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:22.242582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6fba9-e964-4f39-9b8c-63fb9d65f564 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:22.243505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6fba9-e964-4f39-9b8c-63fb9d65f564 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:22.244003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6fba9-e964-4f39-9b8c-63fb9d65f564 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:22.244452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6fba9-e964-4f39-9b8c-63fb9d65f564 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:22.256771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6fba9-e964-4f39-9b8c-63fb9d65f564 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:22.259659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6fba9-e964-4f39-9b8c-63fb9d65f564 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:22.302956 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3134/6201] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:12:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:12:25.247591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb520b79-7ff7-45a3-ae5d-ce6f0dc084eb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:25.248562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb520b79-7ff7-45a3-ae5d-ce6f0dc084eb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:25.249031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb520b79-7ff7-45a3-ae5d-ce6f0dc084eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:25.292280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fb520b79-7ff7-45a3-ae5d-ce6f0dc084eb None None] MFA Rules not processed for user `484ee8f965604fa9b7f7f5a25a81d0ef`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:25.293861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fb520b79-7ff7-45a3-ae5d-ce6f0dc084eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:25.490933 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3068/6202] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:29.399387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fcb95-012e-4e5a-b611-409fd0659319 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:29.400655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fcb95-012e-4e5a-b611-409fd0659319 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:29.401092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fcb95-012e-4e5a-b611-409fd0659319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:29.444974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-656fcb95-012e-4e5a-b611-409fd0659319 None None] MFA Rules not processed for user `902ba44d318d4a01a510f732d27467c9`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:29.448744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-656fcb95-012e-4e5a-b611-409fd0659319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:29.641078 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3135/6203] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:34.979438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b6b6c-3da4-40d3-83ca-420ac87c75b4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:34.979983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b6b6c-3da4-40d3-83ca-420ac87c75b4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:34.979983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b6b6c-3da4-40d3-83ca-420ac87c75b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:35.048968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-969b6b6c-3da4-40d3-83ca-420ac87c75b4 None None] MFA Rules not processed for user `3ba55fd90c52436d96ceaa933cf78a26`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:35.050572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-969b6b6c-3da4-40d3-83ca-420ac87c75b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:35.081726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:35.189171 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3069/6204] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:35.901777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fcb95-012e-4e5a-b611-409fd0659319 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:35.916512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fcb95-012e-4e5a-b611-409fd0659319 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:35.923366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d44a6-90eb-445b-87bf-364e35765d11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:35.924308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d44a6-90eb-445b-87bf-364e35765d11 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:35.925066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d44a6-90eb-445b-87bf-364e35765d11 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:35.925560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d44a6-90eb-445b-87bf-364e35765d11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:35.934747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d44a6-90eb-445b-87bf-364e35765d11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:35.936759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d44a6-90eb-445b-87bf-364e35765d11 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:35.947615 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3136/6205] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:12:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:12:42.323579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc23732-c08c-4ce0-9c1c-0fe57c783ddf None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:42.324050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc23732-c08c-4ce0-9c1c-0fe57c783ddf None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:42.324155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc23732-c08c-4ce0-9c1c-0fe57c783ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:42.348904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5fc23732-c08c-4ce0-9c1c-0fe57c783ddf None None] MFA Rules not processed for user `3ba55fd90c52436d96ceaa933cf78a26`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:42.350799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5fc23732-c08c-4ce0-9c1c-0fe57c783ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:42.362120 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3070/6206] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:43.502989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c388458-496a-40ae-a740-3ef9f8919bcc None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:43.504133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c388458-496a-40ae-a740-3ef9f8919bcc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:43.504828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c388458-496a-40ae-a740-3ef9f8919bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:43.553333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8c388458-496a-40ae-a740-3ef9f8919bcc None None] MFA Rules not processed for user `a2745d6923e347939e2c67015aefadda`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:43.555950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8c388458-496a-40ae-a740-3ef9f8919bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:43.618845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:43.639956 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3137/6207] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:46.113324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc23732-c08c-4ce0-9c1c-0fe57c783ddf None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:46.114979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5fc23732-c08c-4ce0-9c1c-0fe57c783ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:46.249602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc23732-c08c-4ce0-9c1c-0fe57c783ddf None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:46.258225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6ef09-12ae-41a4-a40c-cd34ff4def93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:46.258829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6ef09-12ae-41a4-a40c-cd34ff4def93 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:46.259495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6ef09-12ae-41a4-a40c-cd34ff4def93 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:46.259950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6ef09-12ae-41a4-a40c-cd34ff4def93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:46.274401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6ef09-12ae-41a4-a40c-cd34ff4def93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:46.276349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6ef09-12ae-41a4-a40c-cd34ff4def93 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:46.309367 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3071/6208] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:12:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:12:47.197185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c388458-496a-40ae-a740-3ef9f8919bcc None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:47.199809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8c388458-496a-40ae-a740-3ef9f8919bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:47.334262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c388458-496a-40ae-a740-3ef9f8919bcc None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:47.339931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5bada-c34b-41ef-a6c7-e9d839c79127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:47.340851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5bada-c34b-41ef-a6c7-e9d839c79127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:47.341465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5bada-c34b-41ef-a6c7-e9d839c79127 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:47.341949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5bada-c34b-41ef-a6c7-e9d839c79127 admin admin] PATH_INFO: `/v3/users/269b477a369347f991547be38db227bd` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:47.343589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a5bada-c34b-41ef-a6c7-e9d839c79127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=269b477a369347f991547be38db227bd)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:47.346463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a5bada-c34b-41ef-a6c7-e9d839c79127 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:47.438132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-60a5bada-c34b-41ef-a6c7-e9d839c79127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '269b477a369347f991547be38db227bd'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:47.465677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-60a5bada-c34b-41ef-a6c7-e9d839c79127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '269b477a369347f991547be38db227bd'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:47.483412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-60a5bada-c34b-41ef-a6c7-e9d839c79127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '269b477a369347f991547be38db227bd'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:47.484716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-60a5bada-c34b-41ef-a6c7-e9d839c79127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '269b477a369347f991547be38db227bd'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:47.494844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-60a5bada-c34b-41ef-a6c7-e9d839c79127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '269b477a369347f991547be38db227bd'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:47.503564 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3138/6209] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:12:47 2022] DELETE /identity/v3/users/269b477a369347f991547be38db227bd => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:47.507954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8f3c2-f9d0-4b74-9923-2d11887c6245 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:47.508453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8f3c2-f9d0-4b74-9923-2d11887c6245 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:47.508598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8f3c2-f9d0-4b74-9923-2d11887c6245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:47.529938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b8c8f3c2-f9d0-4b74-9923-2d11887c6245 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:47.531468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b8c8f3c2-f9d0-4b74-9923-2d11887c6245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:47.605141 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3072/6210] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:47.634641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5bada-c34b-41ef-a6c7-e9d839c79127 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:47.634641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-60a5bada-c34b-41ef-a6c7-e9d839c79127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:47.710507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5bada-c34b-41ef-a6c7-e9d839c79127 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:47.714343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe891612-f7f2-477c-af8f-733ae84b54bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:47.715574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe891612-f7f2-477c-af8f-733ae84b54bb service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:47.715852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe891612-f7f2-477c-af8f-733ae84b54bb service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:47.716119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe891612-f7f2-477c-af8f-733ae84b54bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:47.726056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe891612-f7f2-477c-af8f-733ae84b54bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:47.727599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe891612-f7f2-477c-af8f-733ae84b54bb service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:47.740552 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3139/6211] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:12:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:12:48.267855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8f3c2-f9d0-4b74-9923-2d11887c6245 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:48.270541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b8c8f3c2-f9d0-4b74-9923-2d11887c6245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:48.285173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8f3c2-f9d0-4b74-9923-2d11887c6245 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:48.290472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267baca7-fc75-448b-b743-cfaf2adae106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:48.291782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267baca7-fc75-448b-b743-cfaf2adae106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:48.292226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267baca7-fc75-448b-b743-cfaf2adae106 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:48.292659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267baca7-fc75-448b-b743-cfaf2adae106 admin admin] PATH_INFO: `/v3/projects/60d493f07a5148aba21c9a903ec5eb68` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:48.294004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267baca7-fc75-448b-b743-cfaf2adae106 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60d493f07a5148aba21c9a903ec5eb68)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:48.296755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267baca7-fc75-448b-b743-cfaf2adae106 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:48.372162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:48.374742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-267baca7-fc75-448b-b743-cfaf2adae106 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60d493f07a5148aba21c9a903ec5eb68', 'request_id': 'req-267baca7-fc75-448b-b743-cfaf2adae106'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:48.391929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-267baca7-fc75-448b-b743-cfaf2adae106 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60d493f07a5148aba21c9a903ec5eb68', 'request_id': 'req-267baca7-fc75-448b-b743-cfaf2adae106'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:48.398544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-267baca7-fc75-448b-b743-cfaf2adae106 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60d493f07a5148aba21c9a903ec5eb68', 'request_id': 'req-267baca7-fc75-448b-b743-cfaf2adae106'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:48.412283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-267baca7-fc75-448b-b743-cfaf2adae106 admin admin] The token cache is being invalidate because project 60d493f07a5148aba21c9a903ec5eb68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:12:48.413222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-267baca7-fc75-448b-b743-cfaf2adae106 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:48.413913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-267baca7-fc75-448b-b743-cfaf2adae106 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:48.415560 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3073/6212] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:12:48 2022] DELETE /identity/v3/projects/60d493f07a5148aba21c9a903ec5eb68 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:48.420506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe0001-a021-4bc4-b59b-4cb2b04b5195 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:48.421400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe0001-a021-4bc4-b59b-4cb2b04b5195 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:48.421749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe0001-a021-4bc4-b59b-4cb2b04b5195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:48.444158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2afe0001-a021-4bc4-b59b-4cb2b04b5195 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:48.445496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2afe0001-a021-4bc4-b59b-4cb2b04b5195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:48.457689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:48.535713 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3140/6213] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:48.544925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bfd15-7a60-4ca7-9c25-5ec7d4125adb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:48.545852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bfd15-7a60-4ca7-9c25-5ec7d4125adb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:48.546135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bfd15-7a60-4ca7-9c25-5ec7d4125adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:48.571549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-af3bfd15-7a60-4ca7-9c25-5ec7d4125adb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:48.572988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-af3bfd15-7a60-4ca7-9c25-5ec7d4125adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:48.582841 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3074/6214] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:12:48.586753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe0001-a021-4bc4-b59b-4cb2b04b5195 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:48.599100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe0001-a021-4bc4-b59b-4cb2b04b5195 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:48.604995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44aaf4-cc9f-46f8-853f-c59b6b6315f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:48.605575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44aaf4-cc9f-46f8-853f-c59b6b6315f2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:48.606049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44aaf4-cc9f-46f8-853f-c59b6b6315f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:48.606327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44aaf4-cc9f-46f8-853f-c59b6b6315f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:48.607050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44aaf4-cc9f-46f8-853f-c59b6b6315f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:12:48.607612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44aaf4-cc9f-46f8-853f-c59b6b6315f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:48.608951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44aaf4-cc9f-46f8-853f-c59b6b6315f2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:48.623818 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3141/6215] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:12:48.628153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3bfd15-7a60-4ca7-9c25-5ec7d4125adb None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:48.641331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3bfd15-7a60-4ca7-9c25-5ec7d4125adb None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:48.646467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4922ac-605b-4886-b37e-20ceb14335fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:48.647211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4922ac-605b-4886-b37e-20ceb14335fd admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:48.647572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4922ac-605b-4886-b37e-20ceb14335fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:48.647914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4922ac-605b-4886-b37e-20ceb14335fd admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:48.648615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4922ac-605b-4886-b37e-20ceb14335fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:48.650814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4922ac-605b-4886-b37e-20ceb14335fd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:48.683120 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3075/6216] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:12:48 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:12:48.691861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44aaf4-cc9f-46f8-853f-c59b6b6315f2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:48.704044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44aaf4-cc9f-46f8-853f-c59b6b6315f2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:48.774060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ab4d4-ab97-40b0-ae52-3d2430ff39d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:48.774723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ab4d4-ab97-40b0-ae52-3d2430ff39d6 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:48.775003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ab4d4-ab97-40b0-ae52-3d2430ff39d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:48.775278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ab4d4-ab97-40b0-ae52-3d2430ff39d6 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:48.775943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ab4d4-ab97-40b0-ae52-3d2430ff39d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:48.777881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ab4d4-ab97-40b0-ae52-3d2430ff39d6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:48.806754 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3142/6217] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:12:48 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:12:48.811443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4922ac-605b-4886-b37e-20ceb14335fd admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:48.822728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4922ac-605b-4886-b37e-20ceb14335fd admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:48.829070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c3759-3591-400e-9bfc-3a9d05a7e946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:48.829739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c3759-3591-400e-9bfc-3a9d05a7e946 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:48.830199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c3759-3591-400e-9bfc-3a9d05a7e946 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:48.830542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c3759-3591-400e-9bfc-3a9d05a7e946 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:48.831284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655c3759-3591-400e-9bfc-3a9d05a7e946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:48.833041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655c3759-3591-400e-9bfc-3a9d05a7e946 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:48.844099 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3076/6218] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:12:48.847928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ab4d4-ab97-40b0-ae52-3d2430ff39d6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:48.858139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ab4d4-ab97-40b0-ae52-3d2430ff39d6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:48.863395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfa8f1-b997-47f0-ad08-6342e5f8f88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:48.864015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfa8f1-b997-47f0-ad08-6342e5f8f88e admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:48.864282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfa8f1-b997-47f0-ad08-6342e5f8f88e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:48.864550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfa8f1-b997-47f0-ad08-6342e5f8f88e admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:48.865185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cfa8f1-b997-47f0-ad08-6342e5f8f88e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:48.866835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cfa8f1-b997-47f0-ad08-6342e5f8f88e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:48.881914 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3143/6219] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:12:48.883470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c3759-3591-400e-9bfc-3a9d05a7e946 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:48.894678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c3759-3591-400e-9bfc-3a9d05a7e946 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:48.899628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbf4ad-4050-4c81-920c-1cacaeed5acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:48.900373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbf4ad-4050-4c81-920c-1cacaeed5acf admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:48.900877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbf4ad-4050-4c81-920c-1cacaeed5acf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:48.901229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbf4ad-4050-4c81-920c-1cacaeed5acf admin admin] PATH_INFO: `/v3/projects/dc9e416e0c96447a8f002d713111cb60/users/7a1af4e7a11e49da9fb0c6edd9b7b253/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:48.917922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbf4ad-4050-4c81-920c-1cacaeed5acf admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc9e416e0c96447a8f002d713111cb60, user_id=7a1af4e7a11e49da9fb0c6edd9b7b253, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:48.920713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbf4ad-4050-4c81-920c-1cacaeed5acf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:48.934874 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3077/6220] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:12:48 2022] PUT /identity/v3/projects/dc9e416e0c96447a8f002d713111cb60/users/7a1af4e7a11e49da9fb0c6edd9b7b253/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:48.945368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248bf5b-0c8c-40bb-b79f-ab557aa2359b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:48.946822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248bf5b-0c8c-40bb-b79f-ab557aa2359b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:48.947241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248bf5b-0c8c-40bb-b79f-ab557aa2359b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:48.981265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5248bf5b-0c8c-40bb-b79f-ab557aa2359b None None] MFA Rules not processed for user `7a1af4e7a11e49da9fb0c6edd9b7b253`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:48.983086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5248bf5b-0c8c-40bb-b79f-ab557aa2359b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:49.103742 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3144/6221] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:12:49.119866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbf4ad-4050-4c81-920c-1cacaeed5acf admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:49.122177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3dbbf4ad-4050-4c81-920c-1cacaeed5acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:49.194869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbf4ad-4050-4c81-920c-1cacaeed5acf admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:49.198708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff66da-38a8-4073-ac6e-312d6a7d6ddf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:49.199435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff66da-38a8-4073-ac6e-312d6a7d6ddf service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:49.199781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff66da-38a8-4073-ac6e-312d6a7d6ddf service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:49.200409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff66da-38a8-4073-ac6e-312d6a7d6ddf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:49.214115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff66da-38a8-4073-ac6e-312d6a7d6ddf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:49.215859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff66da-38a8-4073-ac6e-312d6a7d6ddf service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:49.225628 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3078/6222] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:12:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:12:49.806378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248bf5b-0c8c-40bb-b79f-ab557aa2359b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:49.823073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:49.890517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248bf5b-0c8c-40bb-b79f-ab557aa2359b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:49.895470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718e2de-e5b7-469b-bd02-d906dc135b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:49.896323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718e2de-e5b7-469b-bd02-d906dc135b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:49.896917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718e2de-e5b7-469b-bd02-d906dc135b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:49.897380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718e2de-e5b7-469b-bd02-d906dc135b3d admin admin] PATH_INFO: `/v3/users/7a1af4e7a11e49da9fb0c6edd9b7b253` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:49.898762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718e2de-e5b7-469b-bd02-d906dc135b3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a1af4e7a11e49da9fb0c6edd9b7b253)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:49.901290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718e2de-e5b7-469b-bd02-d906dc135b3d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:49.956876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f718e2de-e5b7-469b-bd02-d906dc135b3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a1af4e7a11e49da9fb0c6edd9b7b253'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:49.964984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f718e2de-e5b7-469b-bd02-d906dc135b3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a1af4e7a11e49da9fb0c6edd9b7b253'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:49.978149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f718e2de-e5b7-469b-bd02-d906dc135b3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a1af4e7a11e49da9fb0c6edd9b7b253'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:49.978924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f718e2de-e5b7-469b-bd02-d906dc135b3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a1af4e7a11e49da9fb0c6edd9b7b253'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:49.991829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f718e2de-e5b7-469b-bd02-d906dc135b3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a1af4e7a11e49da9fb0c6edd9b7b253'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:49.999537 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3145/6223] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:12:49 2022] DELETE /identity/v3/users/7a1af4e7a11e49da9fb0c6edd9b7b253 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:12:50.004003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b026914-9ec6-426f-afee-f9404cfa5437 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:50.004533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b026914-9ec6-426f-afee-f9404cfa5437 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:50.004533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b026914-9ec6-426f-afee-f9404cfa5437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:50.025064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0b026914-9ec6-426f-afee-f9404cfa5437 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:50.026378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0b026914-9ec6-426f-afee-f9404cfa5437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:50.105812 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3079/6224] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:50.119108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718e2de-e5b7-469b-bd02-d906dc135b3d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:50.122628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f718e2de-e5b7-469b-bd02-d906dc135b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:50.194166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718e2de-e5b7-469b-bd02-d906dc135b3d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:50.201084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4610391-3b85-4eec-896a-b9fdf4a24d16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:50.202567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4610391-3b85-4eec-896a-b9fdf4a24d16 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:50.202567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4610391-3b85-4eec-896a-b9fdf4a24d16 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:50.202815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4610391-3b85-4eec-896a-b9fdf4a24d16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:50.215753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4610391-3b85-4eec-896a-b9fdf4a24d16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:50.217374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4610391-3b85-4eec-896a-b9fdf4a24d16 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:50.232520 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3146/6225] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:12:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:12:50.676368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b026914-9ec6-426f-afee-f9404cfa5437 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:50.677963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0b026914-9ec6-426f-afee-f9404cfa5437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:50.696217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b026914-9ec6-426f-afee-f9404cfa5437 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:50.703930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b074869-4c89-4f84-b1bf-49be9a3a564c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:50.705673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b074869-4c89-4f84-b1bf-49be9a3a564c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:50.706369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b074869-4c89-4f84-b1bf-49be9a3a564c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:50.706918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b074869-4c89-4f84-b1bf-49be9a3a564c admin admin] PATH_INFO: `/v3/projects/dc9e416e0c96447a8f002d713111cb60` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:50.708416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b074869-4c89-4f84-b1bf-49be9a3a564c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc9e416e0c96447a8f002d713111cb60)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:50.711684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b074869-4c89-4f84-b1bf-49be9a3a564c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:50.729023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:50.812930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b074869-4c89-4f84-b1bf-49be9a3a564c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc9e416e0c96447a8f002d713111cb60', 'request_id': 'req-5b074869-4c89-4f84-b1bf-49be9a3a564c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:50.822324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b074869-4c89-4f84-b1bf-49be9a3a564c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc9e416e0c96447a8f002d713111cb60', 'request_id': 'req-5b074869-4c89-4f84-b1bf-49be9a3a564c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:50.833740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b074869-4c89-4f84-b1bf-49be9a3a564c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc9e416e0c96447a8f002d713111cb60', 'request_id': 'req-5b074869-4c89-4f84-b1bf-49be9a3a564c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:50.898610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b074869-4c89-4f84-b1bf-49be9a3a564c admin admin] The token cache is being invalidate because project dc9e416e0c96447a8f002d713111cb60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:12:50.901549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b074869-4c89-4f84-b1bf-49be9a3a564c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:50.903441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5b074869-4c89-4f84-b1bf-49be9a3a564c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:50.907552 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3080/6226] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:12:50 2022] DELETE /identity/v3/projects/dc9e416e0c96447a8f002d713111cb60 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:50.913129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdf98c-90c3-47dc-b3bb-853e60ed5ce4 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:50.913644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdf98c-90c3-47dc-b3bb-853e60ed5ce4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:50.913766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdf98c-90c3-47dc-b3bb-853e60ed5ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:50.941809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-07bdf98c-90c3-47dc-b3bb-853e60ed5ce4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:50.943469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-07bdf98c-90c3-47dc-b3bb-853e60ed5ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:50.982109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:51.042605 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3147/6227] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:51.049519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d051ef-4158-40e0-98ee-4cdf57291e73 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:51.050251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d051ef-4158-40e0-98ee-4cdf57291e73 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:51.050251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d051ef-4158-40e0-98ee-4cdf57291e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:51.079866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f4d051ef-4158-40e0-98ee-4cdf57291e73 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:51.081214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f4d051ef-4158-40e0-98ee-4cdf57291e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:51.098320 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3081/6228] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:51.099305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdf98c-90c3-47dc-b3bb-853e60ed5ce4 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:51.111626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdf98c-90c3-47dc-b3bb-853e60ed5ce4 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:51.117850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae5f53-21d3-49d4-9fb9-435fc33dde49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:51.118786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae5f53-21d3-49d4-9fb9-435fc33dde49 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:51.119237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae5f53-21d3-49d4-9fb9-435fc33dde49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:51.119694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae5f53-21d3-49d4-9fb9-435fc33dde49 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:51.120368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae5f53-21d3-49d4-9fb9-435fc33dde49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:12:51.121197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae5f53-21d3-49d4-9fb9-435fc33dde49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:51.123422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae5f53-21d3-49d4-9fb9-435fc33dde49 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:51.143440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d051ef-4158-40e0-98ee-4cdf57291e73 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:51.150193 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3148/6229] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:12:51.167507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d051ef-4158-40e0-98ee-4cdf57291e73 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:51.167507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be2dd3-97c8-41f0-bb7b-587c15dd30a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:51.167507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be2dd3-97c8-41f0-bb7b-587c15dd30a8 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:51.167507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be2dd3-97c8-41f0-bb7b-587c15dd30a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:51.168282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be2dd3-97c8-41f0-bb7b-587c15dd30a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:51.168282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61be2dd3-97c8-41f0-bb7b-587c15dd30a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:51.168282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61be2dd3-97c8-41f0-bb7b-587c15dd30a8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:51.202527 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3082/6230] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:12:51 2022] POST /identity/v3/projects => generated 363 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:12:51.204801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae5f53-21d3-49d4-9fb9-435fc33dde49 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:51.215695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae5f53-21d3-49d4-9fb9-435fc33dde49 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:51.280975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93800b-0d5c-492d-94ff-484809fca69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:51.281597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93800b-0d5c-492d-94ff-484809fca69d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:51.281873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93800b-0d5c-492d-94ff-484809fca69d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:51.282174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93800b-0d5c-492d-94ff-484809fca69d admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:51.283121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93800b-0d5c-492d-94ff-484809fca69d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:51.286154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93800b-0d5c-492d-94ff-484809fca69d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:51.312922 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3149/6231] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:12:51 2022] POST /identity/v3/users => generated 282 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:12:51.315438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be2dd3-97c8-41f0-bb7b-587c15dd30a8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:51.330900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be2dd3-97c8-41f0-bb7b-587c15dd30a8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:51.338899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43556e36-89b6-4ee8-b201-ccd7b69f9ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:51.340010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43556e36-89b6-4ee8-b201-ccd7b69f9ab1 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:51.340010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43556e36-89b6-4ee8-b201-ccd7b69f9ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:51.340262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43556e36-89b6-4ee8-b201-ccd7b69f9ab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:51.340931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43556e36-89b6-4ee8-b201-ccd7b69f9ab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:51.342942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43556e36-89b6-4ee8-b201-ccd7b69f9ab1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:51.354049 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3083/6232] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:12:51.359091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93800b-0d5c-492d-94ff-484809fca69d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:51.370134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93800b-0d5c-492d-94ff-484809fca69d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:51.376050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42536eb6-577e-4ea3-b6b6-50ab86ffbb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:51.376669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42536eb6-577e-4ea3-b6b6-50ab86ffbb72 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:51.376978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42536eb6-577e-4ea3-b6b6-50ab86ffbb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:51.377220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42536eb6-577e-4ea3-b6b6-50ab86ffbb72 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:51.377860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42536eb6-577e-4ea3-b6b6-50ab86ffbb72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:51.379560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42536eb6-577e-4ea3-b6b6-50ab86ffbb72 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:51.394280 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3150/6233] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:12:51.395455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43556e36-89b6-4ee8-b201-ccd7b69f9ab1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:51.406093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43556e36-89b6-4ee8-b201-ccd7b69f9ab1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:51.410987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b391f34-23e5-4316-88db-c70b723d0afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:51.411725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b391f34-23e5-4316-88db-c70b723d0afc admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:51.412086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b391f34-23e5-4316-88db-c70b723d0afc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:51.412440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b391f34-23e5-4316-88db-c70b723d0afc admin admin] PATH_INFO: `/v3/projects/b189b99f52ce4bbdb449e7c244a9d7f3/users/888357a150af4495bf4570ebf6b172c3/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:51.429119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b391f34-23e5-4316-88db-c70b723d0afc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b189b99f52ce4bbdb449e7c244a9d7f3, user_id=888357a150af4495bf4570ebf6b172c3, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:51.431954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b391f34-23e5-4316-88db-c70b723d0afc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:51.449671 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3084/6234] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:12:51 2022] PUT /identity/v3/projects/b189b99f52ce4bbdb449e7c244a9d7f3/users/888357a150af4495bf4570ebf6b172c3/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:51.459576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e53c84-0b90-4eec-861b-07147f483529 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:51.459762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e53c84-0b90-4eec-861b-07147f483529 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:51.460479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e53c84-0b90-4eec-861b-07147f483529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:51.490339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-98e53c84-0b90-4eec-861b-07147f483529 None None] MFA Rules not processed for user `888357a150af4495bf4570ebf6b172c3`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:51.492074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-98e53c84-0b90-4eec-861b-07147f483529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:51.559571 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3151/6235] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:51.563811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b391f34-23e5-4316-88db-c70b723d0afc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:51.626743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b391f34-23e5-4316-88db-c70b723d0afc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:51.631707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806c598-fcdd-447f-b065-e94e32e3782a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:51.632268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806c598-fcdd-447f-b065-e94e32e3782a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:51.632551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806c598-fcdd-447f-b065-e94e32e3782a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:51.632827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806c598-fcdd-447f-b065-e94e32e3782a admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:51.633593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806c598-fcdd-447f-b065-e94e32e3782a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:51.635628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806c598-fcdd-447f-b065-e94e32e3782a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:51.665991 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3085/6236] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:12:51 2022] POST /identity/v3/projects => generated 363 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:12:51.669882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e53c84-0b90-4eec-861b-07147f483529 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:51.681482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e53c84-0b90-4eec-861b-07147f483529 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:51.709787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:51.743487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97680a08-f7cf-47c2-873d-933f01ce8387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:51.744109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97680a08-f7cf-47c2-873d-933f01ce8387 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:51.744391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97680a08-f7cf-47c2-873d-933f01ce8387 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:51.744674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97680a08-f7cf-47c2-873d-933f01ce8387 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:51.745399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97680a08-f7cf-47c2-873d-933f01ce8387 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:51.748147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97680a08-f7cf-47c2-873d-933f01ce8387 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:51.774107 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3152/6237] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:12:51 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:12:51.778885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806c598-fcdd-447f-b065-e94e32e3782a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:51.792234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806c598-fcdd-447f-b065-e94e32e3782a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:51.798821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b48829-4810-4983-a6de-b7cc078fb867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:51.799419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b48829-4810-4983-a6de-b7cc078fb867 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:51.800145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b48829-4810-4983-a6de-b7cc078fb867 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:51.800542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b48829-4810-4983-a6de-b7cc078fb867 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:51.801219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b48829-4810-4983-a6de-b7cc078fb867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:51.802940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b48829-4810-4983-a6de-b7cc078fb867 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:51.826065 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3086/6238] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:12:51.827932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97680a08-f7cf-47c2-873d-933f01ce8387 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:51.840669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97680a08-f7cf-47c2-873d-933f01ce8387 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:51.849782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b9436-5e28-4924-b488-588235028411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:51.850964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b9436-5e28-4924-b488-588235028411 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:51.851242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b9436-5e28-4924-b488-588235028411 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:51.851518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b9436-5e28-4924-b488-588235028411 admin admin] PATH_INFO: `/v3/projects/327f3dc849db4891a81b4e2d7d16c927/users/64b3ffe848664433a7aaeaf8eea18653/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:51.869638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b9436-5e28-4924-b488-588235028411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=327f3dc849db4891a81b4e2d7d16c927, user_id=64b3ffe848664433a7aaeaf8eea18653, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:51.872911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b9436-5e28-4924-b488-588235028411 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:51.891334 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3153/6239] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:12:51 2022] PUT /identity/v3/projects/327f3dc849db4891a81b4e2d7d16c927/users/64b3ffe848664433a7aaeaf8eea18653/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:51.895506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b48829-4810-4983-a6de-b7cc078fb867 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:51.909107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b48829-4810-4983-a6de-b7cc078fb867 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:51.915866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:51.983301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1bdc5d-f7d6-4cf6-8dfb-7a67d0692ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:51.984010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bdc5d-f7d6-4cf6-8dfb-7a67d0692ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:51.984368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bdc5d-f7d6-4cf6-8dfb-7a67d0692ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:51.984774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bdc5d-f7d6-4cf6-8dfb-7a67d0692ae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:51.985465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1bdc5d-f7d6-4cf6-8dfb-7a67d0692ae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:51.987171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1bdc5d-f7d6-4cf6-8dfb-7a67d0692ae8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:51.999504 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3087/6240] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:12:52.004197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b9436-5e28-4924-b488-588235028411 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:52.017949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b9436-5e28-4924-b488-588235028411 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:52.025382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147bec3-8a84-4d62-848c-91571dc5074b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:52.026118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147bec3-8a84-4d62-848c-91571dc5074b admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:52.026390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147bec3-8a84-4d62-848c-91571dc5074b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:52.026751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147bec3-8a84-4d62-848c-91571dc5074b admin admin] PATH_INFO: `/v3/projects/327f3dc849db4891a81b4e2d7d16c927/users/64b3ffe848664433a7aaeaf8eea18653/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:52.029867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147bec3-8a84-4d62-848c-91571dc5074b admin admin] RBAC: Authorizing `identity:create_grant(project_id=327f3dc849db4891a81b4e2d7d16c927, user_id=64b3ffe848664433a7aaeaf8eea18653, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:52.033052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147bec3-8a84-4d62-848c-91571dc5074b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:52.047163 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3154/6241] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:12:52 2022] PUT /identity/v3/projects/327f3dc849db4891a81b4e2d7d16c927/users/64b3ffe848664433a7aaeaf8eea18653/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:52.057470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c73041-6b02-40b9-af99-7446a3d77a4d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:52.057916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c73041-6b02-40b9-af99-7446a3d77a4d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:52.058186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c73041-6b02-40b9-af99-7446a3d77a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:52.097258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d2c73041-6b02-40b9-af99-7446a3d77a4d None None] MFA Rules not processed for user `64b3ffe848664433a7aaeaf8eea18653`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:52.098534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d2c73041-6b02-40b9-af99-7446a3d77a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:52.182446 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3088/6242] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:52.197902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147bec3-8a84-4d62-848c-91571dc5074b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:52.200973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e147bec3-8a84-4d62-848c-91571dc5074b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:52.272555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147bec3-8a84-4d62-848c-91571dc5074b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:52.278764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ac1c5-163d-45b5-a379-d0c8705ee41b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:52.279710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ac1c5-163d-45b5-a379-d0c8705ee41b service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:52.279710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ac1c5-163d-45b5-a379-d0c8705ee41b service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:52.280304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ac1c5-163d-45b5-a379-d0c8705ee41b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:52.290125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ac1c5-163d-45b5-a379-d0c8705ee41b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:52.291801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ac1c5-163d-45b5-a379-d0c8705ee41b service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:52.302965 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3155/6243] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:12:52 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:12:52.985751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c73041-6b02-40b9-af99-7446a3d77a4d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:53.115589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c73041-6b02-40b9-af99-7446a3d77a4d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:53.121862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2679324-a8d9-4db2-b0f3-e2601dbf3adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:53.122630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2679324-a8d9-4db2-b0f3-e2601dbf3adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:53.122979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2679324-a8d9-4db2-b0f3-e2601dbf3adb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:53.123653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2679324-a8d9-4db2-b0f3-e2601dbf3adb admin admin] PATH_INFO: `/v3/users/888357a150af4495bf4570ebf6b172c3` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:53.125184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2679324-a8d9-4db2-b0f3-e2601dbf3adb admin admin] RBAC: Authorizing `identity:delete_user(user_id=888357a150af4495bf4570ebf6b172c3)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:53.138846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2679324-a8d9-4db2-b0f3-e2601dbf3adb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:53.211169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:53.246350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f2679324-a8d9-4db2-b0f3-e2601dbf3adb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '888357a150af4495bf4570ebf6b172c3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:53.247672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f2679324-a8d9-4db2-b0f3-e2601dbf3adb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '888357a150af4495bf4570ebf6b172c3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:53.248715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f2679324-a8d9-4db2-b0f3-e2601dbf3adb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '888357a150af4495bf4570ebf6b172c3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:53.263004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f2679324-a8d9-4db2-b0f3-e2601dbf3adb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '888357a150af4495bf4570ebf6b172c3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:53.300354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f2679324-a8d9-4db2-b0f3-e2601dbf3adb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '888357a150af4495bf4570ebf6b172c3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:53.332309 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3089/6244] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:12:52 2022] DELETE /identity/v3/users/888357a150af4495bf4570ebf6b172c3 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:53.337802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febba593-3b3a-4480-90eb-7b9e5435b5e1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:53.338327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febba593-3b3a-4480-90eb-7b9e5435b5e1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:53.338526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febba593-3b3a-4480-90eb-7b9e5435b5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:53.375552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-febba593-3b3a-4480-90eb-7b9e5435b5e1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:53.377131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-febba593-3b3a-4480-90eb-7b9e5435b5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:53.489715 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3156/6245] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:53.501642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2679324-a8d9-4db2-b0f3-e2601dbf3adb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:53.503010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f2679324-a8d9-4db2-b0f3-e2601dbf3adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:53.615701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2679324-a8d9-4db2-b0f3-e2601dbf3adb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:53.621466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ee68f-3dcd-4bc2-be5a-7f1dc0d8252b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:53.623728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ee68f-3dcd-4bc2-be5a-7f1dc0d8252b service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:53.624090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ee68f-3dcd-4bc2-be5a-7f1dc0d8252b service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:53.624501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ee68f-3dcd-4bc2-be5a-7f1dc0d8252b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:53.635730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ee68f-3dcd-4bc2-be5a-7f1dc0d8252b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:53.638195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ee68f-3dcd-4bc2-be5a-7f1dc0d8252b service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:53.650337 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3090/6246] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:12:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:12:54.245683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febba593-3b3a-4480-90eb-7b9e5435b5e1 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:54.247959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-febba593-3b3a-4480-90eb-7b9e5435b5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:54.282168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febba593-3b3a-4480-90eb-7b9e5435b5e1 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:54.303648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b476352-a4b8-4fee-ae3c-f03f073423da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:54.304420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b476352-a4b8-4fee-ae3c-f03f073423da admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:54.304792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b476352-a4b8-4fee-ae3c-f03f073423da admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:54.305142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b476352-a4b8-4fee-ae3c-f03f073423da admin admin] PATH_INFO: `/v3/projects/b189b99f52ce4bbdb449e7c244a9d7f3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:54.308118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b476352-a4b8-4fee-ae3c-f03f073423da admin admin] RBAC: Authorizing `identity:delete_project(project_id=b189b99f52ce4bbdb449e7c244a9d7f3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:54.310572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b476352-a4b8-4fee-ae3c-f03f073423da admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:54.356362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:54.423159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1b476352-a4b8-4fee-ae3c-f03f073423da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b189b99f52ce4bbdb449e7c244a9d7f3', 'request_id': 'req-1b476352-a4b8-4fee-ae3c-f03f073423da'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:54.442606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1b476352-a4b8-4fee-ae3c-f03f073423da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b189b99f52ce4bbdb449e7c244a9d7f3', 'request_id': 'req-1b476352-a4b8-4fee-ae3c-f03f073423da'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:54.461721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1b476352-a4b8-4fee-ae3c-f03f073423da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b189b99f52ce4bbdb449e7c244a9d7f3', 'request_id': 'req-1b476352-a4b8-4fee-ae3c-f03f073423da'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:54.486316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1b476352-a4b8-4fee-ae3c-f03f073423da admin admin] The token cache is being invalidate because project b189b99f52ce4bbdb449e7c244a9d7f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:12:54.487694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1b476352-a4b8-4fee-ae3c-f03f073423da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:54.490176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1b476352-a4b8-4fee-ae3c-f03f073423da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:54.494127 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3157/6247] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:12:54 2022] DELETE /identity/v3/projects/b189b99f52ce4bbdb449e7c244a9d7f3 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:54.498648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ee68f-3dcd-4bc2-be5a-7f1dc0d8252b service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:54.517958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ee68f-3dcd-4bc2-be5a-7f1dc0d8252b service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:54.571927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85100c-068a-40f7-a7fd-f82109875166 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:54.572247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85100c-068a-40f7-a7fd-f82109875166 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:54.572986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85100c-068a-40f7-a7fd-f82109875166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:54.626216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8d85100c-068a-40f7-a7fd-f82109875166 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:54.630973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8d85100c-068a-40f7-a7fd-f82109875166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:54.646063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d62302-c7a1-407b-9707-6793d17c9d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:54.646736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d62302-c7a1-407b-9707-6793d17c9d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:54.648507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d62302-c7a1-407b-9707-6793d17c9d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:54.650476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d62302-c7a1-407b-9707-6793d17c9d91 admin admin] PATH_INFO: `/v3/users/64b3ffe848664433a7aaeaf8eea18653` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:54.654129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d62302-c7a1-407b-9707-6793d17c9d91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64b3ffe848664433a7aaeaf8eea18653)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:54.660944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d62302-c7a1-407b-9707-6793d17c9d91 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:54.797379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-57d62302-c7a1-407b-9707-6793d17c9d91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64b3ffe848664433a7aaeaf8eea18653'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:54.799404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-57d62302-c7a1-407b-9707-6793d17c9d91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64b3ffe848664433a7aaeaf8eea18653'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:54.800611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-57d62302-c7a1-407b-9707-6793d17c9d91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64b3ffe848664433a7aaeaf8eea18653'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:54.816770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-57d62302-c7a1-407b-9707-6793d17c9d91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64b3ffe848664433a7aaeaf8eea18653'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:54.819894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:54.860772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-57d62302-c7a1-407b-9707-6793d17c9d91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64b3ffe848664433a7aaeaf8eea18653'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:54.866548 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3158/6248] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:12:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:12:54.893976 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3091/6249] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:12:54 2022] DELETE /identity/v3/users/64b3ffe848664433a7aaeaf8eea18653 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:12:54.949745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85100c-068a-40f7-a7fd-f82109875166 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:54.951430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8d85100c-068a-40f7-a7fd-f82109875166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:55.104355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85100c-068a-40f7-a7fd-f82109875166 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:55.190881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:55.246784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2f0d2-6d73-442f-bc49-e737b94382fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:55.251025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2f0d2-6d73-442f-bc49-e737b94382fa service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:55.251489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2f0d2-6d73-442f-bc49-e737b94382fa service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:55.251890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2f0d2-6d73-442f-bc49-e737b94382fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:55.344539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2f0d2-6d73-442f-bc49-e737b94382fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:55.346393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2f0d2-6d73-442f-bc49-e737b94382fa service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:55.360606 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3159/6250] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:12:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:12:56.257279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d62302-c7a1-407b-9707-6793d17c9d91 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:56.261817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-57d62302-c7a1-407b-9707-6793d17c9d91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:56.451674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d62302-c7a1-407b-9707-6793d17c9d91 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:56.463946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932480b3-fd6b-4855-a6a5-4e87428f1a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:56.470730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932480b3-fd6b-4855-a6a5-4e87428f1a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:56.471293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932480b3-fd6b-4855-a6a5-4e87428f1a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:56.471767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932480b3-fd6b-4855-a6a5-4e87428f1a34 admin admin] PATH_INFO: `/v3/projects/327f3dc849db4891a81b4e2d7d16c927` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:56.473274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932480b3-fd6b-4855-a6a5-4e87428f1a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=327f3dc849db4891a81b4e2d7d16c927)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:56.475589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932480b3-fd6b-4855-a6a5-4e87428f1a34 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:56.605571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-932480b3-fd6b-4855-a6a5-4e87428f1a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '327f3dc849db4891a81b4e2d7d16c927', 'request_id': 'req-932480b3-fd6b-4855-a6a5-4e87428f1a34'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:56.623174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-932480b3-fd6b-4855-a6a5-4e87428f1a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '327f3dc849db4891a81b4e2d7d16c927', 'request_id': 'req-932480b3-fd6b-4855-a6a5-4e87428f1a34'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:56.637032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-932480b3-fd6b-4855-a6a5-4e87428f1a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '327f3dc849db4891a81b4e2d7d16c927', 'request_id': 'req-932480b3-fd6b-4855-a6a5-4e87428f1a34'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:56.669595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-932480b3-fd6b-4855-a6a5-4e87428f1a34 admin admin] The token cache is being invalidate because project 327f3dc849db4891a81b4e2d7d16c927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:12:56.669899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-932480b3-fd6b-4855-a6a5-4e87428f1a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:56.670627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-932480b3-fd6b-4855-a6a5-4e87428f1a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:12:56.676280 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3092/6251] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:12:56 2022] DELETE /identity/v3/projects/327f3dc849db4891a81b4e2d7d16c927 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:56.680274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826ead1-27c4-444d-9132-ea1b524c5e6b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:56.680852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826ead1-27c4-444d-9132-ea1b524c5e6b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:56.680852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826ead1-27c4-444d-9132-ea1b524c5e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:56.721213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b826ead1-27c4-444d-9132-ea1b524c5e6b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:56.724297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b826ead1-27c4-444d-9132-ea1b524c5e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:56.867477 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3160/6252] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:56.875404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a09c1c-2d35-4665-82a8-85bb00f31af0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:56.876098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a09c1c-2d35-4665-82a8-85bb00f31af0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:56.876377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a09c1c-2d35-4665-82a8-85bb00f31af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:56.900896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-03a09c1c-2d35-4665-82a8-85bb00f31af0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:56.902925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-03a09c1c-2d35-4665-82a8-85bb00f31af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:56.918808 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3093/6253] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:56.919197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826ead1-27c4-444d-9132-ea1b524c5e6b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:56.933097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826ead1-27c4-444d-9132-ea1b524c5e6b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:56.944866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b9ef7-85b6-4f1d-82a5-dcc8f9dd0475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:56.945763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b9ef7-85b6-4f1d-82a5-dcc8f9dd0475 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:56.951106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b9ef7-85b6-4f1d-82a5-dcc8f9dd0475 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:56.951674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b9ef7-85b6-4f1d-82a5-dcc8f9dd0475 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:56.952548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b9ef7-85b6-4f1d-82a5-dcc8f9dd0475 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:12:56.953037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b9ef7-85b6-4f1d-82a5-dcc8f9dd0475 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:56.954914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b9ef7-85b6-4f1d-82a5-dcc8f9dd0475 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:56.985641 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3161/6254] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:12:56.991655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a09c1c-2d35-4665-82a8-85bb00f31af0 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:57.012451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a09c1c-2d35-4665-82a8-85bb00f31af0 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:57.049820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544c785-d074-4917-9e21-46e1270fe62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:57.050389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544c785-d074-4917-9e21-46e1270fe62e admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:57.050738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544c785-d074-4917-9e21-46e1270fe62e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:57.051007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544c785-d074-4917-9e21-46e1270fe62e admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:57.051642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544c785-d074-4917-9e21-46e1270fe62e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:57.053588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544c785-d074-4917-9e21-46e1270fe62e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:57.111706 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3094/6255] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:12:56 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:12:57.113393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b9ef7-85b6-4f1d-82a5-dcc8f9dd0475 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:57.133440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b9ef7-85b6-4f1d-82a5-dcc8f9dd0475 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:57.288016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcef7fa-c477-4351-a4ac-06c6ad513595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:57.292914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcef7fa-c477-4351-a4ac-06c6ad513595 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:57.293548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcef7fa-c477-4351-a4ac-06c6ad513595 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:57.294079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcef7fa-c477-4351-a4ac-06c6ad513595 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:57.295217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcef7fa-c477-4351-a4ac-06c6ad513595 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:57.301400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcef7fa-c477-4351-a4ac-06c6ad513595 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:57.335945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:57.372247 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3162/6256] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:12:57 2022] POST /identity/v3/users => generated 292 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:12:57.374331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544c785-d074-4917-9e21-46e1270fe62e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:57.398433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544c785-d074-4917-9e21-46e1270fe62e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:57.432773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f86d7-32da-4c50-ba44-db8d6f2f5d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:57.433413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f86d7-32da-4c50-ba44-db8d6f2f5d15 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:57.433693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f86d7-32da-4c50-ba44-db8d6f2f5d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:57.434377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f86d7-32da-4c50-ba44-db8d6f2f5d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:57.435058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f86d7-32da-4c50-ba44-db8d6f2f5d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:57.442465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f86d7-32da-4c50-ba44-db8d6f2f5d15 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:57.458618 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3095/6257] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:12:57.462684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcef7fa-c477-4351-a4ac-06c6ad513595 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:57.477811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcef7fa-c477-4351-a4ac-06c6ad513595 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:57.492498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12daa5-7697-43b6-a34b-2dac59da892a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:57.495225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12daa5-7697-43b6-a34b-2dac59da892a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:57.496175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12daa5-7697-43b6-a34b-2dac59da892a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:57.498298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12daa5-7697-43b6-a34b-2dac59da892a admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:57.502821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12daa5-7697-43b6-a34b-2dac59da892a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:57.505069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12daa5-7697-43b6-a34b-2dac59da892a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:57.529405 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3163/6258] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:12:57.533523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f86d7-32da-4c50-ba44-db8d6f2f5d15 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:57.549950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f86d7-32da-4c50-ba44-db8d6f2f5d15 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:57.557860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa2f44-c113-4fd9-9f8b-cd15b0b88967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:57.559294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa2f44-c113-4fd9-9f8b-cd15b0b88967 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:57.559764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa2f44-c113-4fd9-9f8b-cd15b0b88967 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:57.560203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa2f44-c113-4fd9-9f8b-cd15b0b88967 admin admin] PATH_INFO: `/v3/projects/62aa9b4263524351843f7136dd44a9d0/users/b38792f636ab4f36abce0bf90faaef80/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:57.630536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa2f44-c113-4fd9-9f8b-cd15b0b88967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62aa9b4263524351843f7136dd44a9d0, user_id=b38792f636ab4f36abce0bf90faaef80, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:57.632721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa2f44-c113-4fd9-9f8b-cd15b0b88967 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:57.664317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:57.683447 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3096/6259] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:12:57 2022] PUT /identity/v3/projects/62aa9b4263524351843f7136dd44a9d0/users/b38792f636ab4f36abce0bf90faaef80/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:57.693703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fa23f-e2bf-4cf8-a505-facdc119c25b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:57.697106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fa23f-e2bf-4cf8-a505-facdc119c25b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:57.697418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fa23f-e2bf-4cf8-a505-facdc119c25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:57.768211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4d4fa23f-e2bf-4cf8-a505-facdc119c25b None None] MFA Rules not processed for user `b38792f636ab4f36abce0bf90faaef80`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:57.769675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4d4fa23f-e2bf-4cf8-a505-facdc119c25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:57.910103 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3164/6260] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:57.914819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa2f44-c113-4fd9-9f8b-cd15b0b88967 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:58.020255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa2f44-c113-4fd9-9f8b-cd15b0b88967 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:58.035480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1ff7b-fcd4-4575-a90f-de059d6fb74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:58.036156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1ff7b-fcd4-4575-a90f-de059d6fb74f admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:58.036835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1ff7b-fcd4-4575-a90f-de059d6fb74f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:58.037230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1ff7b-fcd4-4575-a90f-de059d6fb74f admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:58.037949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1ff7b-fcd4-4575-a90f-de059d6fb74f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:58.048257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1ff7b-fcd4-4575-a90f-de059d6fb74f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:58.114838 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3097/6261] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:12:57 2022] POST /identity/v3/projects => generated 381 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:12:58.118558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fa23f-e2bf-4cf8-a505-facdc119c25b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:58.138045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fa23f-e2bf-4cf8-a505-facdc119c25b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:58.227272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f815c-d433-4026-aee3-282b6ba1ffa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:58.228250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f815c-d433-4026-aee3-282b6ba1ffa5 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:58.228645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f815c-d433-4026-aee3-282b6ba1ffa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:58.229046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f815c-d433-4026-aee3-282b6ba1ffa5 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:58.230986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6f815c-d433-4026-aee3-282b6ba1ffa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:58.233489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6f815c-d433-4026-aee3-282b6ba1ffa5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:58.275415 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3165/6262] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:12:58 2022] POST /identity/v3/users => generated 297 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:12:58.279557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1ff7b-fcd4-4575-a90f-de059d6fb74f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:58.302847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1ff7b-fcd4-4575-a90f-de059d6fb74f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:58.313834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a6381-9dd8-4651-99aa-0d034703f083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:58.315371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a6381-9dd8-4651-99aa-0d034703f083 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:58.315834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a6381-9dd8-4651-99aa-0d034703f083 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:58.316184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a6381-9dd8-4651-99aa-0d034703f083 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:58.316908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a6381-9dd8-4651-99aa-0d034703f083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:58.318831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a6381-9dd8-4651-99aa-0d034703f083 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:58.340811 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3098/6263] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:12:58.344970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f815c-d433-4026-aee3-282b6ba1ffa5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:58.364367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f815c-d433-4026-aee3-282b6ba1ffa5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:58.370082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0083905-8d1b-4998-9ba3-2ce12320de88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:58.372229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0083905-8d1b-4998-9ba3-2ce12320de88 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:58.372579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0083905-8d1b-4998-9ba3-2ce12320de88 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:58.372930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0083905-8d1b-4998-9ba3-2ce12320de88 admin admin] PATH_INFO: `/v3/projects/b5774d4ffcb340288863dc5027074f28/users/75e652c5013948f58e2aa4737eeeddff/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:58.399975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0083905-8d1b-4998-9ba3-2ce12320de88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5774d4ffcb340288863dc5027074f28, user_id=75e652c5013948f58e2aa4737eeeddff, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:58.403045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0083905-8d1b-4998-9ba3-2ce12320de88 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:58.421093 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3166/6264] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:12:58 2022] PUT /identity/v3/projects/b5774d4ffcb340288863dc5027074f28/users/75e652c5013948f58e2aa4737eeeddff/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:58.426589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a6381-9dd8-4651-99aa-0d034703f083 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:58.440505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a6381-9dd8-4651-99aa-0d034703f083 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:58.548307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbcb060-c9bd-49fa-b1c1-8b92da76b11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:58.549103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcb060-c9bd-49fa-b1c1-8b92da76b11f admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:58.549790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcb060-c9bd-49fa-b1c1-8b92da76b11f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:58.550199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcb060-c9bd-49fa-b1c1-8b92da76b11f admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:58.551151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbcb060-c9bd-49fa-b1c1-8b92da76b11f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:58.553955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbcb060-c9bd-49fa-b1c1-8b92da76b11f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:58.569860 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3099/6265] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:12:58.574327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0083905-8d1b-4998-9ba3-2ce12320de88 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:58.588303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0083905-8d1b-4998-9ba3-2ce12320de88 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:58.595880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8949e-ff43-44c2-bbd6-b2849d7270f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:58.596902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8949e-ff43-44c2-bbd6-b2849d7270f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:58.597276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8949e-ff43-44c2-bbd6-b2849d7270f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:58.597669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8949e-ff43-44c2-bbd6-b2849d7270f7 admin admin] PATH_INFO: `/v3/projects/b5774d4ffcb340288863dc5027074f28/users/75e652c5013948f58e2aa4737eeeddff/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:58.600458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8949e-ff43-44c2-bbd6-b2849d7270f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5774d4ffcb340288863dc5027074f28, user_id=75e652c5013948f58e2aa4737eeeddff, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:58.603699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8949e-ff43-44c2-bbd6-b2849d7270f7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:58.618727 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3167/6266] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:12:58 2022] PUT /identity/v3/projects/b5774d4ffcb340288863dc5027074f28/users/75e652c5013948f58e2aa4737eeeddff/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:12:58.630925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57910f3-306a-4633-acd0-38d2bd00e243 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:58.631336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57910f3-306a-4633-acd0-38d2bd00e243 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:58.631711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57910f3-306a-4633-acd0-38d2bd00e243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:58.666011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f57910f3-306a-4633-acd0-38d2bd00e243 None None] MFA Rules not processed for user `75e652c5013948f58e2aa4737eeeddff`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:12:58.667627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f57910f3-306a-4633-acd0-38d2bd00e243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:58.752290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:58.768098 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3100/6267] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:12:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:12:58.780240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8949e-ff43-44c2-bbd6-b2849d7270f7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:58.782455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-96e8949e-ff43-44c2-bbd6-b2849d7270f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:12:58.868905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:58.884158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8949e-ff43-44c2-bbd6-b2849d7270f7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:58.888275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958f028-f853-44a4-9112-aa4e6bce5acf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:58.889253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958f028-f853-44a4-9112-aa4e6bce5acf service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:58.889671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958f028-f853-44a4-9112-aa4e6bce5acf service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:58.890469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958f028-f853-44a4-9112-aa4e6bce5acf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:58.901349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958f028-f853-44a4-9112-aa4e6bce5acf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:58.903458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958f028-f853-44a4-9112-aa4e6bce5acf service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:58.915148 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3168/6268] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:12:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:12:59.071110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57910f3-306a-4633-acd0-38d2bd00e243 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:12:59.083149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57910f3-306a-4633-acd0-38d2bd00e243 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:12:59.089744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d7d9e-b2ac-4f14-9e32-2723c162172b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:12:59.090696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d7d9e-b2ac-4f14-9e32-2723c162172b service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:12:59.091152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d7d9e-b2ac-4f14-9e32-2723c162172b service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:12:59.091152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d7d9e-b2ac-4f14-9e32-2723c162172b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:12:59.252340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d7d9e-b2ac-4f14-9e32-2723c162172b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:12:59.255057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d7d9e-b2ac-4f14-9e32-2723c162172b service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:12:59.275641 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3101/6269] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:12:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:03.957831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958f028-f853-44a4-9112-aa4e6bce5acf service cinder] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:04.124657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958f028-f853-44a4-9112-aa4e6bce5acf service cinder] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:04.134867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af080f3-2b71-4846-9764-c00bca6e7ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:04.136630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af080f3-2b71-4846-9764-c00bca6e7ebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:04.139277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af080f3-2b71-4846-9764-c00bca6e7ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:04.139945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af080f3-2b71-4846-9764-c00bca6e7ebf admin admin] PATH_INFO: `/v3/users/b38792f636ab4f36abce0bf90faaef80` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:04.144980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af080f3-2b71-4846-9764-c00bca6e7ebf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b38792f636ab4f36abce0bf90faaef80)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:04.147275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af080f3-2b71-4846-9764-c00bca6e7ebf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:04.277932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7af080f3-2b71-4846-9764-c00bca6e7ebf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b38792f636ab4f36abce0bf90faaef80'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:04.304714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7af080f3-2b71-4846-9764-c00bca6e7ebf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b38792f636ab4f36abce0bf90faaef80'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:04.339528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7af080f3-2b71-4846-9764-c00bca6e7ebf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b38792f636ab4f36abce0bf90faaef80'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:04.342058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7af080f3-2b71-4846-9764-c00bca6e7ebf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b38792f636ab4f36abce0bf90faaef80'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:04.376140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7af080f3-2b71-4846-9764-c00bca6e7ebf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b38792f636ab4f36abce0bf90faaef80'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:04.406480 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3169/6270] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:03 2022] DELETE /identity/v3/users/b38792f636ab4f36abce0bf90faaef80 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:04.412276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546d9b1-0538-4970-81d6-eff115253ef4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:04.412770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546d9b1-0538-4970-81d6-eff115253ef4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:04.413107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546d9b1-0538-4970-81d6-eff115253ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:04.453767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1546d9b1-0538-4970-81d6-eff115253ef4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:04.455534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1546d9b1-0538-4970-81d6-eff115253ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:04.593922 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3102/6271] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:04.626063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af080f3-2b71-4846-9764-c00bca6e7ebf admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:04.627801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7af080f3-2b71-4846-9764-c00bca6e7ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:04.751218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:04.764780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af080f3-2b71-4846-9764-c00bca6e7ebf admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:04.772738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1c48d-46f5-47df-a0f4-94710abf2b18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:04.773285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1c48d-46f5-47df-a0f4-94710abf2b18 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:04.773575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1c48d-46f5-47df-a0f4-94710abf2b18 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:04.773855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1c48d-46f5-47df-a0f4-94710abf2b18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:04.790476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1c48d-46f5-47df-a0f4-94710abf2b18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:04.793312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1c48d-46f5-47df-a0f4-94710abf2b18 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:04.821737 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3170/6272] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:05.809168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546d9b1-0538-4970-81d6-eff115253ef4 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:05.810875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1546d9b1-0538-4970-81d6-eff115253ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:05.833383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546d9b1-0538-4970-81d6-eff115253ef4 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:05.850484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c00bdf-53f2-464a-9951-2645fd085224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:05.851173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c00bdf-53f2-464a-9951-2645fd085224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:05.851468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c00bdf-53f2-464a-9951-2645fd085224 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:05.851745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c00bdf-53f2-464a-9951-2645fd085224 admin admin] PATH_INFO: `/v3/projects/62aa9b4263524351843f7136dd44a9d0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:05.857225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c00bdf-53f2-464a-9951-2645fd085224 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62aa9b4263524351843f7136dd44a9d0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:05.869529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c00bdf-53f2-464a-9951-2645fd085224 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:06.043347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-96c00bdf-53f2-464a-9951-2645fd085224 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62aa9b4263524351843f7136dd44a9d0', 'request_id': 'req-96c00bdf-53f2-464a-9951-2645fd085224'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:06.066297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-96c00bdf-53f2-464a-9951-2645fd085224 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62aa9b4263524351843f7136dd44a9d0', 'request_id': 'req-96c00bdf-53f2-464a-9951-2645fd085224'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:06.077921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-96c00bdf-53f2-464a-9951-2645fd085224 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62aa9b4263524351843f7136dd44a9d0', 'request_id': 'req-96c00bdf-53f2-464a-9951-2645fd085224'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:06.094227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-96c00bdf-53f2-464a-9951-2645fd085224 admin admin] The token cache is being invalidate because project 62aa9b4263524351843f7136dd44a9d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:06.095565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-96c00bdf-53f2-464a-9951-2645fd085224 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:06.119709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-96c00bdf-53f2-464a-9951-2645fd085224 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:06.122124 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3103/6273] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:05 2022] DELETE /identity/v3/projects/62aa9b4263524351843f7136dd44a9d0 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:06.126822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1c48d-46f5-47df-a0f4-94710abf2b18 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:06.145034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1c48d-46f5-47df-a0f4-94710abf2b18 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:06.310383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ea8c3-fccc-4247-ab16-383d59c980fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:06.311064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ea8c3-fccc-4247-ab16-383d59c980fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:06.311352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ea8c3-fccc-4247-ab16-383d59c980fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:06.311631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ea8c3-fccc-4247-ab16-383d59c980fc admin admin] PATH_INFO: `/v3/users/75e652c5013948f58e2aa4737eeeddff` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:06.318822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ea8c3-fccc-4247-ab16-383d59c980fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=75e652c5013948f58e2aa4737eeeddff)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:06.326175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ea8c3-fccc-4247-ab16-383d59c980fc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:06.456926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-356ea8c3-fccc-4247-ab16-383d59c980fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75e652c5013948f58e2aa4737eeeddff'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:06.475380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-356ea8c3-fccc-4247-ab16-383d59c980fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75e652c5013948f58e2aa4737eeeddff'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:06.521587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-356ea8c3-fccc-4247-ab16-383d59c980fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75e652c5013948f58e2aa4737eeeddff'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:06.523712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-356ea8c3-fccc-4247-ab16-383d59c980fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75e652c5013948f58e2aa4737eeeddff'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:06.545492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-356ea8c3-fccc-4247-ab16-383d59c980fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75e652c5013948f58e2aa4737eeeddff'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:06.560794 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3171/6274] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:06 2022] DELETE /identity/v3/users/75e652c5013948f58e2aa4737eeeddff => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:07.244344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c00bdf-53f2-464a-9951-2645fd085224 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:07.246005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-96c00bdf-53f2-464a-9951-2645fd085224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:07.287175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c00bdf-53f2-464a-9951-2645fd085224 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:07.331140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:07.468126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c45871-a5e5-4fec-b33d-764d05de2f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:07.468840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c45871-a5e5-4fec-b33d-764d05de2f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:07.469846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c45871-a5e5-4fec-b33d-764d05de2f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:07.470129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c45871-a5e5-4fec-b33d-764d05de2f85 admin admin] PATH_INFO: `/v3/projects/b5774d4ffcb340288863dc5027074f28` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:07.472440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c45871-a5e5-4fec-b33d-764d05de2f85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5774d4ffcb340288863dc5027074f28)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:07.479106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c45871-a5e5-4fec-b33d-764d05de2f85 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:07.641616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-78c45871-a5e5-4fec-b33d-764d05de2f85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5774d4ffcb340288863dc5027074f28', 'request_id': 'req-78c45871-a5e5-4fec-b33d-764d05de2f85'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:07.662065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-78c45871-a5e5-4fec-b33d-764d05de2f85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5774d4ffcb340288863dc5027074f28', 'request_id': 'req-78c45871-a5e5-4fec-b33d-764d05de2f85'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:07.680316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-78c45871-a5e5-4fec-b33d-764d05de2f85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5774d4ffcb340288863dc5027074f28', 'request_id': 'req-78c45871-a5e5-4fec-b33d-764d05de2f85'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:07.748938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-78c45871-a5e5-4fec-b33d-764d05de2f85 admin admin] The token cache is being invalidate because project b5774d4ffcb340288863dc5027074f28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:07.749241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-78c45871-a5e5-4fec-b33d-764d05de2f85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:07.755631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-78c45871-a5e5-4fec-b33d-764d05de2f85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:07.766397 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3104/6275] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:07 2022] DELETE /identity/v3/projects/b5774d4ffcb340288863dc5027074f28 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:07.769527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6d371-ee05-4621-8318-5962d3f10b39 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:07.770040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6d371-ee05-4621-8318-5962d3f10b39 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:07.770158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6d371-ee05-4621-8318-5962d3f10b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:07.844435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a0c6d371-ee05-4621-8318-5962d3f10b39 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:07.845817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a0c6d371-ee05-4621-8318-5962d3f10b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:07.988239 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3172/6276] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:07.995979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee33168-d901-4e43-927f-a3ddda479d01 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:07.996909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee33168-d901-4e43-927f-a3ddda479d01 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:07.997438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee33168-d901-4e43-927f-a3ddda479d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:08.038937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-dee33168-d901-4e43-927f-a3ddda479d01 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:08.040266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-dee33168-d901-4e43-927f-a3ddda479d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:08.073175 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3105/6277] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:08.077111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6d371-ee05-4621-8318-5962d3f10b39 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:08.090682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6d371-ee05-4621-8318-5962d3f10b39 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:08.100638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba35d834-bd1a-41b3-95df-3a400d6fa5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:08.101607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35d834-bd1a-41b3-95df-3a400d6fa5c1 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:08.102250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35d834-bd1a-41b3-95df-3a400d6fa5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:08.102818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35d834-bd1a-41b3-95df-3a400d6fa5c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:08.103717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba35d834-bd1a-41b3-95df-3a400d6fa5c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:13:08.104254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba35d834-bd1a-41b3-95df-3a400d6fa5c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:08.107409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba35d834-bd1a-41b3-95df-3a400d6fa5c1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:08.111462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:08.135556 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3173/6278] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:13:08.139653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee33168-d901-4e43-927f-a3ddda479d01 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:08.158425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee33168-d901-4e43-927f-a3ddda479d01 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:08.173085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168535e-62ad-40d5-889b-509c42f6b245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:08.173085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168535e-62ad-40d5-889b-509c42f6b245 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:08.173621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168535e-62ad-40d5-889b-509c42f6b245 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:08.174010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168535e-62ad-40d5-889b-509c42f6b245 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:08.176042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168535e-62ad-40d5-889b-509c42f6b245 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:08.178974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168535e-62ad-40d5-889b-509c42f6b245 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:08.250875 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3106/6279] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:08 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:08.253449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba35d834-bd1a-41b3-95df-3a400d6fa5c1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:08.275725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba35d834-bd1a-41b3-95df-3a400d6fa5c1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:08.413522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec558a-03d4-4ff8-a1c9-6ed0ce60b6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:08.414412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec558a-03d4-4ff8-a1c9-6ed0ce60b6f7 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:08.415461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec558a-03d4-4ff8-a1c9-6ed0ce60b6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:08.415916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec558a-03d4-4ff8-a1c9-6ed0ce60b6f7 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:08.417467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec558a-03d4-4ff8-a1c9-6ed0ce60b6f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:08.420913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec558a-03d4-4ff8-a1c9-6ed0ce60b6f7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:08.456062 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3174/6280] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:08 2022] POST /identity/v3/users => generated 290 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:08.460105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168535e-62ad-40d5-889b-509c42f6b245 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:08.476982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168535e-62ad-40d5-889b-509c42f6b245 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:08.486598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ece3bd-a5ae-488e-97cf-2d8c14bfe54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:08.487656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ece3bd-a5ae-488e-97cf-2d8c14bfe54b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:08.488308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ece3bd-a5ae-488e-97cf-2d8c14bfe54b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:08.488699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ece3bd-a5ae-488e-97cf-2d8c14bfe54b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:08.489575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ece3bd-a5ae-488e-97cf-2d8c14bfe54b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:08.491445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ece3bd-a5ae-488e-97cf-2d8c14bfe54b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:08.506476 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3107/6281] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:08.510057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec558a-03d4-4ff8-a1c9-6ed0ce60b6f7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:08.533826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec558a-03d4-4ff8-a1c9-6ed0ce60b6f7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:08.540140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797dd836-03da-4bae-b8bf-e6197ea0bc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:08.541157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dd836-03da-4bae-b8bf-e6197ea0bc14 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:08.541790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dd836-03da-4bae-b8bf-e6197ea0bc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:08.542385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dd836-03da-4bae-b8bf-e6197ea0bc14 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:08.543517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797dd836-03da-4bae-b8bf-e6197ea0bc14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:08.545991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797dd836-03da-4bae-b8bf-e6197ea0bc14 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:08.564749 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3175/6282] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:13:08.568546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ece3bd-a5ae-488e-97cf-2d8c14bfe54b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:08.584684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ece3bd-a5ae-488e-97cf-2d8c14bfe54b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:08.595524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d31e3e-9476-4161-80b5-cb58cdcf57bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:08.596792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d31e3e-9476-4161-80b5-cb58cdcf57bb admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:08.597543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d31e3e-9476-4161-80b5-cb58cdcf57bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:08.597828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d31e3e-9476-4161-80b5-cb58cdcf57bb admin admin] PATH_INFO: `/v3/projects/fdc98fc841664be2bc220da1a2f0a9fe/users/f8d10af82ef249feb8bfb058a6633730/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:08.624385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d31e3e-9476-4161-80b5-cb58cdcf57bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdc98fc841664be2bc220da1a2f0a9fe, user_id=f8d10af82ef249feb8bfb058a6633730, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:08.627397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d31e3e-9476-4161-80b5-cb58cdcf57bb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:08.645126 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3108/6283] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:08 2022] PUT /identity/v3/projects/fdc98fc841664be2bc220da1a2f0a9fe/users/f8d10af82ef249feb8bfb058a6633730/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:08.654904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd5071-c6f0-4a4f-8464-9b87d705852f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:08.655238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd5071-c6f0-4a4f-8464-9b87d705852f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:08.655617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd5071-c6f0-4a4f-8464-9b87d705852f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:08.705589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4efd5071-c6f0-4a4f-8464-9b87d705852f None None] MFA Rules not processed for user `f8d10af82ef249feb8bfb058a6633730`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:08.707038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4efd5071-c6f0-4a4f-8464-9b87d705852f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:08.846378 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3176/6284] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:08.850441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d31e3e-9476-4161-80b5-cb58cdcf57bb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:08.987838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:09.063996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d31e3e-9476-4161-80b5-cb58cdcf57bb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:09.083147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07e97d-aa6a-4703-8b0f-fa00821c45e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:09.083946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07e97d-aa6a-4703-8b0f-fa00821c45e1 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:09.084846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07e97d-aa6a-4703-8b0f-fa00821c45e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:09.085414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07e97d-aa6a-4703-8b0f-fa00821c45e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:09.086211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab07e97d-aa6a-4703-8b0f-fa00821c45e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:09.098907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab07e97d-aa6a-4703-8b0f-fa00821c45e1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:09.179481 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3109/6285] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:08 2022] POST /identity/v3/projects => generated 379 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:09.184712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd5071-c6f0-4a4f-8464-9b87d705852f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:09.217887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd5071-c6f0-4a4f-8464-9b87d705852f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:09.380409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263f41d-662a-4ae7-a082-77f999977742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:09.381073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263f41d-662a-4ae7-a082-77f999977742 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:09.381358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263f41d-662a-4ae7-a082-77f999977742 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:09.384636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263f41d-662a-4ae7-a082-77f999977742 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:09.384636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3263f41d-662a-4ae7-a082-77f999977742 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:09.393548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3263f41d-662a-4ae7-a082-77f999977742 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:09.425877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:09.447300 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3177/6286] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:09 2022] POST /identity/v3/users => generated 290 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:09.451081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07e97d-aa6a-4703-8b0f-fa00821c45e1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:09.464855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07e97d-aa6a-4703-8b0f-fa00821c45e1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:09.471038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd9de1-c89b-4d72-9487-4f6799d60e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:09.471710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd9de1-c89b-4d72-9487-4f6799d60e39 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:09.472005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd9de1-c89b-4d72-9487-4f6799d60e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:09.472310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd9de1-c89b-4d72-9487-4f6799d60e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:09.482592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd9de1-c89b-4d72-9487-4f6799d60e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:09.482592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd9de1-c89b-4d72-9487-4f6799d60e39 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:09.499897 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3110/6287] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:13:09.503792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263f41d-662a-4ae7-a082-77f999977742 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:09.524333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263f41d-662a-4ae7-a082-77f999977742 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:09.539748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c68e5-ebde-4b04-b2b2-c4b696a980da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:09.540392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c68e5-ebde-4b04-b2b2-c4b696a980da admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:09.540679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c68e5-ebde-4b04-b2b2-c4b696a980da admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:09.540958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c68e5-ebde-4b04-b2b2-c4b696a980da admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:09.541624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c68e5-ebde-4b04-b2b2-c4b696a980da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:09.543448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c68e5-ebde-4b04-b2b2-c4b696a980da admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:09.575546 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3178/6288] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:09.576804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd9de1-c89b-4d72-9487-4f6799d60e39 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:09.590701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd9de1-c89b-4d72-9487-4f6799d60e39 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:09.623491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88461e84-67e6-4c1b-b411-616eb430e81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:09.624947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88461e84-67e6-4c1b-b411-616eb430e81b admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:09.625297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88461e84-67e6-4c1b-b411-616eb430e81b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:09.625585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88461e84-67e6-4c1b-b411-616eb430e81b admin admin] PATH_INFO: `/v3/projects/6aa7e9d55eee4849ad5b38e01f85f480/users/c6d81496ff5144969c4ff654eb547b73/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:09.665553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88461e84-67e6-4c1b-b411-616eb430e81b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aa7e9d55eee4849ad5b38e01f85f480, user_id=c6d81496ff5144969c4ff654eb547b73, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:09.668690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88461e84-67e6-4c1b-b411-616eb430e81b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:09.710790 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3111/6289] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:09 2022] PUT /identity/v3/projects/6aa7e9d55eee4849ad5b38e01f85f480/users/c6d81496ff5144969c4ff654eb547b73/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:09.713033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f7839-ae04-46f1-a207-00619c4f967c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:09.713327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f7839-ae04-46f1-a207-00619c4f967c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:09.713613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f7839-ae04-46f1-a207-00619c4f967c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:09.783124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f20f7839-ae04-46f1-a207-00619c4f967c None None] MFA Rules not processed for user `c6d81496ff5144969c4ff654eb547b73`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:09.785191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f20f7839-ae04-46f1-a207-00619c4f967c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:09.915595 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3179/6290] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:09.919513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88461e84-67e6-4c1b-b411-616eb430e81b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:10.275838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88461e84-67e6-4c1b-b411-616eb430e81b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:10.312244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65252499-a240-4e21-80bf-88e29646e20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:10.312873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65252499-a240-4e21-80bf-88e29646e20c admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:10.313153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65252499-a240-4e21-80bf-88e29646e20c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:10.313523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65252499-a240-4e21-80bf-88e29646e20c admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:10.314710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65252499-a240-4e21-80bf-88e29646e20c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:10.319538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65252499-a240-4e21-80bf-88e29646e20c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:10.378281 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3112/6291] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:09 2022] POST /identity/v3/projects => generated 379 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:10.380799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f7839-ae04-46f1-a207-00619c4f967c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:10.398139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f7839-ae04-46f1-a207-00619c4f967c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:10.668233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3df24-8dab-49f9-a1af-029ce333dab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:10.671149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3df24-8dab-49f9-a1af-029ce333dab8 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:10.671644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3df24-8dab-49f9-a1af-029ce333dab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:10.672677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3df24-8dab-49f9-a1af-029ce333dab8 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:10.675634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3df24-8dab-49f9-a1af-029ce333dab8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:10.684614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3df24-8dab-49f9-a1af-029ce333dab8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:10.766289 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3180/6292] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:10 2022] POST /identity/v3/users => generated 296 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:10.782940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65252499-a240-4e21-80bf-88e29646e20c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:10.835295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65252499-a240-4e21-80bf-88e29646e20c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:10.873761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f3a2a-7d31-4f63-b1b2-eae883c997a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:10.874393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f3a2a-7d31-4f63-b1b2-eae883c997a1 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:10.877189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f3a2a-7d31-4f63-b1b2-eae883c997a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:10.877526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f3a2a-7d31-4f63-b1b2-eae883c997a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:10.878180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f3a2a-7d31-4f63-b1b2-eae883c997a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:10.879909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f3a2a-7d31-4f63-b1b2-eae883c997a1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:10.903230 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3113/6293] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:10.906998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3df24-8dab-49f9-a1af-029ce333dab8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:10.956309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3df24-8dab-49f9-a1af-029ce333dab8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:10.993610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65d212-4e18-4a7f-ad43-cc5d9be76706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:10.994339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65d212-4e18-4a7f-ad43-cc5d9be76706 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:10.994717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65d212-4e18-4a7f-ad43-cc5d9be76706 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:10.995602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65d212-4e18-4a7f-ad43-cc5d9be76706 admin admin] PATH_INFO: `/v3/projects/58597598ab8947e497b37172b49971c3/users/bc70398b1ade4941a5bf129f87a3b1d1/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:11.060247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65d212-4e18-4a7f-ad43-cc5d9be76706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58597598ab8947e497b37172b49971c3, user_id=bc70398b1ade4941a5bf129f87a3b1d1, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:11.075887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65d212-4e18-4a7f-ad43-cc5d9be76706 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:11.106790 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3181/6294] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:10 2022] PUT /identity/v3/projects/58597598ab8947e497b37172b49971c3/users/bc70398b1ade4941a5bf129f87a3b1d1/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:11.110802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f3a2a-7d31-4f63-b1b2-eae883c997a1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:11.125859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f3a2a-7d31-4f63-b1b2-eae883c997a1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:11.266310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194179c-eb12-4952-ab20-36944a8b349a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:11.266961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194179c-eb12-4952-ab20-36944a8b349a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:11.267234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194179c-eb12-4952-ab20-36944a8b349a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:11.267728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194179c-eb12-4952-ab20-36944a8b349a admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:11.268748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f194179c-eb12-4952-ab20-36944a8b349a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:11.270559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f194179c-eb12-4952-ab20-36944a8b349a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:11.311359 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3114/6295] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:11.313029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65d212-4e18-4a7f-ad43-cc5d9be76706 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:11.334420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65d212-4e18-4a7f-ad43-cc5d9be76706 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:11.344459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34032c-9499-4b1b-b61b-e493c97d0ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:11.345275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34032c-9499-4b1b-b61b-e493c97d0ad8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:11.345498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34032c-9499-4b1b-b61b-e493c97d0ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:11.345748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34032c-9499-4b1b-b61b-e493c97d0ad8 admin admin] PATH_INFO: `/v3/projects/58597598ab8947e497b37172b49971c3/users/bc70398b1ade4941a5bf129f87a3b1d1/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:11.353178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34032c-9499-4b1b-b61b-e493c97d0ad8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58597598ab8947e497b37172b49971c3, user_id=bc70398b1ade4941a5bf129f87a3b1d1, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:11.355310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34032c-9499-4b1b-b61b-e493c97d0ad8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:11.391989 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3182/6296] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:11 2022] PUT /identity/v3/projects/58597598ab8947e497b37172b49971c3/users/bc70398b1ade4941a5bf129f87a3b1d1/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:11.396715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cc93c-a0d6-4e3c-aae7-738c3ad91a6a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:11.397263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cc93c-a0d6-4e3c-aae7-738c3ad91a6a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:11.397723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cc93c-a0d6-4e3c-aae7-738c3ad91a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:11.410563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:11.474220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-465cc93c-a0d6-4e3c-aae7-738c3ad91a6a None None] MFA Rules not processed for user `bc70398b1ade4941a5bf129f87a3b1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:11.478923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-465cc93c-a0d6-4e3c-aae7-738c3ad91a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:11.691766 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3115/6297] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:11 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:11.710187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34032c-9499-4b1b-b61b-e493c97d0ad8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:11.713088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3b34032c-9499-4b1b-b61b-e493c97d0ad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:11.785843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:11.831803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34032c-9499-4b1b-b61b-e493c97d0ad8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:11.837764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5acba-4716-492b-bed4-cf72d4e726ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:11.839234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5acba-4716-492b-bed4-cf72d4e726ed service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:11.839682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5acba-4716-492b-bed4-cf72d4e726ed service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:11.840142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5acba-4716-492b-bed4-cf72d4e726ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:11.853331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5acba-4716-492b-bed4-cf72d4e726ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:11.855206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5acba-4716-492b-bed4-cf72d4e726ed service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:11.884020 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3183/6298] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:11 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:11.892232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cc93c-a0d6-4e3c-aae7-738c3ad91a6a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:11.893800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-465cc93c-a0d6-4e3c-aae7-738c3ad91a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:11.977375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5acba-4716-492b-bed4-cf72d4e726ed service cinder] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:11.977375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5af5acba-4716-492b-bed4-cf72d4e726ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:11.999637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5acba-4716-492b-bed4-cf72d4e726ed service cinder] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:12.008229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ea2a4-a4a7-4636-b82d-66bb57698c93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:12.009504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ea2a4-a4a7-4636-b82d-66bb57698c93 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:12.010773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ea2a4-a4a7-4636-b82d-66bb57698c93 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:12.011717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ea2a4-a4a7-4636-b82d-66bb57698c93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:12.106344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cc93c-a0d6-4e3c-aae7-738c3ad91a6a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:12.113350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d0ac9-3c1c-4224-91b2-e579c891c00f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:12.114315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d0ac9-3c1c-4224-91b2-e579c891c00f service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:12.114914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d0ac9-3c1c-4224-91b2-e579c891c00f service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:12.116113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d0ac9-3c1c-4224-91b2-e579c891c00f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:12.120352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-495d0ac9-3c1c-4224-91b2-e579c891c00f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:12.147406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ea2a4-a4a7-4636-b82d-66bb57698c93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:12.149446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ea2a4-a4a7-4636-b82d-66bb57698c93 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:12.154361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d0ac9-3c1c-4224-91b2-e579c891c00f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:12.156632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d0ac9-3c1c-4224-91b2-e579c891c00f service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:12.178143 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3184/6299] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:12.179930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ea2a4-a4a7-4636-b82d-66bb57698c93 service cinder] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:12.187959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-061ea2a4-a4a7-4636-b82d-66bb57698c93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:12.199825 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3116/6300] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:12.369329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ea2a4-a4a7-4636-b82d-66bb57698c93 service cinder] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:12.387438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49a4b0-4949-4b2a-afb4-63f8df407d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:12.388447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49a4b0-4949-4b2a-afb4-63f8df407d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:12.389099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49a4b0-4949-4b2a-afb4-63f8df407d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:12.389559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49a4b0-4949-4b2a-afb4-63f8df407d8d admin admin] PATH_INFO: `/v3/users/3ba55fd90c52436d96ceaa933cf78a26` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:12.395382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49a4b0-4949-4b2a-afb4-63f8df407d8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ba55fd90c52436d96ceaa933cf78a26)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:12.400102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49a4b0-4949-4b2a-afb4-63f8df407d8d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:12.517901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9f49a4b0-4949-4b2a-afb4-63f8df407d8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ba55fd90c52436d96ceaa933cf78a26'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:12.529099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9f49a4b0-4949-4b2a-afb4-63f8df407d8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ba55fd90c52436d96ceaa933cf78a26'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:12.539977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:12.555923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9f49a4b0-4949-4b2a-afb4-63f8df407d8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ba55fd90c52436d96ceaa933cf78a26'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:12.558518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9f49a4b0-4949-4b2a-afb4-63f8df407d8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ba55fd90c52436d96ceaa933cf78a26'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:12.573599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9f49a4b0-4949-4b2a-afb4-63f8df407d8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ba55fd90c52436d96ceaa933cf78a26'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:12.593168 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3185/6301] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:12 2022] DELETE /identity/v3/users/3ba55fd90c52436d96ceaa933cf78a26 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:13.670960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d0ac9-3c1c-4224-91b2-e579c891c00f service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:13.710829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d0ac9-3c1c-4224-91b2-e579c891c00f service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:13.843245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f62ef-7835-4e4e-8b50-c18050d57ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:13.843908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f62ef-7835-4e4e-8b50-c18050d57ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:13.844376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f62ef-7835-4e4e-8b50-c18050d57ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:13.844811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f62ef-7835-4e4e-8b50-c18050d57ecb admin admin] PATH_INFO: `/v3/projects/6548b7e8d5a244f58fabbd12126808ac` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:13.846686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f62ef-7835-4e4e-8b50-c18050d57ecb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6548b7e8d5a244f58fabbd12126808ac)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:13.849597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f62ef-7835-4e4e-8b50-c18050d57ecb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:13.898174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:13.966672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2f1f62ef-7835-4e4e-8b50-c18050d57ecb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6548b7e8d5a244f58fabbd12126808ac', 'request_id': 'req-2f1f62ef-7835-4e4e-8b50-c18050d57ecb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:13.991502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2f1f62ef-7835-4e4e-8b50-c18050d57ecb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6548b7e8d5a244f58fabbd12126808ac', 'request_id': 'req-2f1f62ef-7835-4e4e-8b50-c18050d57ecb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:14.000848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2f1f62ef-7835-4e4e-8b50-c18050d57ecb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6548b7e8d5a244f58fabbd12126808ac', 'request_id': 'req-2f1f62ef-7835-4e4e-8b50-c18050d57ecb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:14.021839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2f1f62ef-7835-4e4e-8b50-c18050d57ecb admin admin] The token cache is being invalidate because project 6548b7e8d5a244f58fabbd12126808ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:14.022434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2f1f62ef-7835-4e4e-8b50-c18050d57ecb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:14.023601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2f1f62ef-7835-4e4e-8b50-c18050d57ecb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:14.037505 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3117/6302] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:13 2022] DELETE /identity/v3/projects/6548b7e8d5a244f58fabbd12126808ac => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:14.045212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506d26d-d335-4400-a83e-2eb12d1d003d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:14.046060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506d26d-d335-4400-a83e-2eb12d1d003d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:14.046690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506d26d-d335-4400-a83e-2eb12d1d003d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:14.071293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7506d26d-d335-4400-a83e-2eb12d1d003d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:14.072545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7506d26d-d335-4400-a83e-2eb12d1d003d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:14.189622 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3186/6303] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:14.199685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ca891-b87c-43dc-b602-e89f202622dd None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:14.199979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ca891-b87c-43dc-b602-e89f202622dd None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:14.200255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ca891-b87c-43dc-b602-e89f202622dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:14.258222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-627ca891-b87c-43dc-b602-e89f202622dd None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:14.264402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-627ca891-b87c-43dc-b602-e89f202622dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:14.329870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506d26d-d335-4400-a83e-2eb12d1d003d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:14.340560 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3118/6304] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:14.345665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506d26d-d335-4400-a83e-2eb12d1d003d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:14.365801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448daaf-e916-4561-a122-bd61c2a9be94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:14.367479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448daaf-e916-4561-a122-bd61c2a9be94 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:14.370920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448daaf-e916-4561-a122-bd61c2a9be94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:14.371324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448daaf-e916-4561-a122-bd61c2a9be94 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:14.374432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448daaf-e916-4561-a122-bd61c2a9be94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:13:14.375505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448daaf-e916-4561-a122-bd61c2a9be94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:14.396557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448daaf-e916-4561-a122-bd61c2a9be94 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:14.443769 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3187/6305] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:13:14.444595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ca891-b87c-43dc-b602-e89f202622dd None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:14.461566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ca891-b87c-43dc-b602-e89f202622dd None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:14.471411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd32ff-8179-4c00-95e4-bcb8b60a058a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:14.473009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd32ff-8179-4c00-95e4-bcb8b60a058a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:14.473298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd32ff-8179-4c00-95e4-bcb8b60a058a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:14.473570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd32ff-8179-4c00-95e4-bcb8b60a058a admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:14.474216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd32ff-8179-4c00-95e4-bcb8b60a058a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:14.478351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd32ff-8179-4c00-95e4-bcb8b60a058a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:14.547994 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3119/6306] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:14 2022] POST /identity/v3/projects => generated 367 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:14.548664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448daaf-e916-4561-a122-bd61c2a9be94 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:14.564687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448daaf-e916-4561-a122-bd61c2a9be94 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:14.684246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beefaa7-41a1-4501-821d-d075884f7f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:14.684877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beefaa7-41a1-4501-821d-d075884f7f6d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:14.685955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beefaa7-41a1-4501-821d-d075884f7f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:14.686265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beefaa7-41a1-4501-821d-d075884f7f6d admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:14.688225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beefaa7-41a1-4501-821d-d075884f7f6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:14.699234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beefaa7-41a1-4501-821d-d075884f7f6d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:14.751701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd32ff-8179-4c00-95e4-bcb8b60a058a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:14.762736 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3188/6307] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:14 2022] POST /identity/v3/users => generated 284 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:13:14.774780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd32ff-8179-4c00-95e4-bcb8b60a058a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:14.791543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96329ec9-0e98-4b4c-a53b-ef92058a21f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:14.793233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96329ec9-0e98-4b4c-a53b-ef92058a21f5 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:14.794375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96329ec9-0e98-4b4c-a53b-ef92058a21f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:14.795769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96329ec9-0e98-4b4c-a53b-ef92058a21f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:14.817068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96329ec9-0e98-4b4c-a53b-ef92058a21f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:14.825948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96329ec9-0e98-4b4c-a53b-ef92058a21f5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:14.872647 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3120/6308] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:14.883988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beefaa7-41a1-4501-821d-d075884f7f6d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:14.904694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beefaa7-41a1-4501-821d-d075884f7f6d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:14.927065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa83e89-0090-4f11-a92d-139c6c2c4011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:14.927692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa83e89-0090-4f11-a92d-139c6c2c4011 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:14.939994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa83e89-0090-4f11-a92d-139c6c2c4011 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:14.940303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa83e89-0090-4f11-a92d-139c6c2c4011 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:14.941019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa83e89-0090-4f11-a92d-139c6c2c4011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:14.953485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa83e89-0090-4f11-a92d-139c6c2c4011 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:15.013564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96329ec9-0e98-4b4c-a53b-ef92058a21f5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:15.014439 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3189/6309] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:15.027801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96329ec9-0e98-4b4c-a53b-ef92058a21f5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:15.043105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ce823-0dd7-406d-998a-2d1993484354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:15.044597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ce823-0dd7-406d-998a-2d1993484354 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:15.044992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ce823-0dd7-406d-998a-2d1993484354 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:15.045375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ce823-0dd7-406d-998a-2d1993484354 admin admin] PATH_INFO: `/v3/projects/588321452fd442ee8b78e7d7e4fa4638/users/20dc4cd847944bf09273b91c1cf6952e/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:15.077566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ce823-0dd7-406d-998a-2d1993484354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=588321452fd442ee8b78e7d7e4fa4638, user_id=20dc4cd847944bf09273b91c1cf6952e, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:15.082235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ce823-0dd7-406d-998a-2d1993484354 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:15.107072 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3121/6310] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:15 2022] PUT /identity/v3/projects/588321452fd442ee8b78e7d7e4fa4638/users/20dc4cd847944bf09273b91c1cf6952e/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:15.120879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3e166-bda2-4665-a9e1-5f906bb34201 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:15.121758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3e166-bda2-4665-a9e1-5f906bb34201 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:15.122226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3e166-bda2-4665-a9e1-5f906bb34201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:15.162864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a4f3e166-bda2-4665-a9e1-5f906bb34201 None None] MFA Rules not processed for user `20dc4cd847944bf09273b91c1cf6952e`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:15.165202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a4f3e166-bda2-4665-a9e1-5f906bb34201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:15.239140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:15.313925 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3190/6311] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:15.328181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ce823-0dd7-406d-998a-2d1993484354 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:15.330815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1a0ce823-0dd7-406d-998a-2d1993484354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:15.524473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ce823-0dd7-406d-998a-2d1993484354 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:15.534836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ecafdd-075f-4509-a4dd-65484d9241ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:15.535487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecafdd-075f-4509-a4dd-65484d9241ec service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:15.535770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecafdd-075f-4509-a4dd-65484d9241ec service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:15.536049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecafdd-075f-4509-a4dd-65484d9241ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:15.568670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ecafdd-075f-4509-a4dd-65484d9241ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:15.570341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ecafdd-075f-4509-a4dd-65484d9241ec service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:15.609137 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3122/6312] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:16.074580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f3e166-bda2-4665-a9e1-5f906bb34201 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:16.076852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a4f3e166-bda2-4665-a9e1-5f906bb34201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:16.144544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef3782-fa72-4d95-bde5-7cb1795fc60c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:16.145267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef3782-fa72-4d95-bde5-7cb1795fc60c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:16.145267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef3782-fa72-4d95-bde5-7cb1795fc60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:16.191823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b3ef3782-fa72-4d95-bde5-7cb1795fc60c None None] MFA Rules not processed for user `a2745d6923e347939e2c67015aefadda`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:16.195542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b3ef3782-fa72-4d95-bde5-7cb1795fc60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:16.214697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f3e166-bda2-4665-a9e1-5f906bb34201 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:16.225498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c42dd-7c59-4913-b2a8-447351ad4335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:16.228017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c42dd-7c59-4913-b2a8-447351ad4335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:16.228779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c42dd-7c59-4913-b2a8-447351ad4335 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:16.230041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c42dd-7c59-4913-b2a8-447351ad4335 admin admin] PATH_INFO: `/v3/users/78cc9dbf1b344a35b9be09ad32549a62` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:16.232669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c42dd-7c59-4913-b2a8-447351ad4335 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78cc9dbf1b344a35b9be09ad32549a62)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:16.238165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c42dd-7c59-4913-b2a8-447351ad4335 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:16.293228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:16.359008 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3123/6313] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:16.363313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef3782-fa72-4d95-bde5-7cb1795fc60c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:16.364848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b3ef3782-fa72-4d95-bde5-7cb1795fc60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:16.387633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-455c42dd-7c59-4913-b2a8-447351ad4335 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78cc9dbf1b344a35b9be09ad32549a62'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:16.418255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-455c42dd-7c59-4913-b2a8-447351ad4335 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78cc9dbf1b344a35b9be09ad32549a62'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:16.426417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef3782-fa72-4d95-bde5-7cb1795fc60c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:16.436933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d71ce-cba1-4aa2-b2e4-392de1ecc334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:16.439872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d71ce-cba1-4aa2-b2e4-392de1ecc334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:16.439976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d71ce-cba1-4aa2-b2e4-392de1ecc334 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:16.440243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d71ce-cba1-4aa2-b2e4-392de1ecc334 admin admin] PATH_INFO: `/v3/users/f8d10af82ef249feb8bfb058a6633730` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:16.442245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d71ce-cba1-4aa2-b2e4-392de1ecc334 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8d10af82ef249feb8bfb058a6633730)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:16.446777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d71ce-cba1-4aa2-b2e4-392de1ecc334 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:16.466165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-455c42dd-7c59-4913-b2a8-447351ad4335 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78cc9dbf1b344a35b9be09ad32549a62'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:16.467082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-455c42dd-7c59-4913-b2a8-447351ad4335 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78cc9dbf1b344a35b9be09ad32549a62'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:16.482193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-455c42dd-7c59-4913-b2a8-447351ad4335 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78cc9dbf1b344a35b9be09ad32549a62'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:16.506201 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3191/6314] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:16 2022] DELETE /identity/v3/users/78cc9dbf1b344a35b9be09ad32549a62 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:13:16.579714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7e1d71ce-cba1-4aa2-b2e4-392de1ecc334 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8d10af82ef249feb8bfb058a6633730'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:16.581002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7e1d71ce-cba1-4aa2-b2e4-392de1ecc334 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8d10af82ef249feb8bfb058a6633730'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:16.581649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7e1d71ce-cba1-4aa2-b2e4-392de1ecc334 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8d10af82ef249feb8bfb058a6633730'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:16.599464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7e1d71ce-cba1-4aa2-b2e4-392de1ecc334 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8d10af82ef249feb8bfb058a6633730'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:16.622935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7e1d71ce-cba1-4aa2-b2e4-392de1ecc334 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8d10af82ef249feb8bfb058a6633730'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:16.649449 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3124/6315] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:16 2022] DELETE /identity/v3/users/f8d10af82ef249feb8bfb058a6633730 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:16.654693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacba62a-5d81-40d6-8f2a-d00bf1a2567e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:16.655221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacba62a-5d81-40d6-8f2a-d00bf1a2567e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:16.655726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacba62a-5d81-40d6-8f2a-d00bf1a2567e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:16.694285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-cacba62a-5d81-40d6-8f2a-d00bf1a2567e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:16.698654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cacba62a-5d81-40d6-8f2a-d00bf1a2567e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:16.821860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:16.851734 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3192/6316] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:16.860986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d71ce-cba1-4aa2-b2e4-392de1ecc334 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:16.862483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7e1d71ce-cba1-4aa2-b2e4-392de1ecc334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:16.988901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d71ce-cba1-4aa2-b2e4-392de1ecc334 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:17.003155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474163f8-88c3-4c4c-8636-78e787e5fd32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:17.005004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474163f8-88c3-4c4c-8636-78e787e5fd32 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:17.005283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474163f8-88c3-4c4c-8636-78e787e5fd32 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:17.005565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474163f8-88c3-4c4c-8636-78e787e5fd32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:17.038401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474163f8-88c3-4c4c-8636-78e787e5fd32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:17.040709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474163f8-88c3-4c4c-8636-78e787e5fd32 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:17.069897 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3125/6317] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:17.641904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacba62a-5d81-40d6-8f2a-d00bf1a2567e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:17.644176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cacba62a-5d81-40d6-8f2a-d00bf1a2567e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:17.669972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacba62a-5d81-40d6-8f2a-d00bf1a2567e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:17.684582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65185e6-8bf1-4673-9034-ee158c8ac5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:17.686705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65185e6-8bf1-4673-9034-ee158c8ac5a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:17.687781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65185e6-8bf1-4673-9034-ee158c8ac5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:17.688281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65185e6-8bf1-4673-9034-ee158c8ac5a5 admin admin] PATH_INFO: `/v3/projects/55864f798646443bbdfe30f8b9fbefb1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:17.689757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65185e6-8bf1-4673-9034-ee158c8ac5a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55864f798646443bbdfe30f8b9fbefb1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:17.692630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65185e6-8bf1-4673-9034-ee158c8ac5a5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:17.779310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b65185e6-8bf1-4673-9034-ee158c8ac5a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55864f798646443bbdfe30f8b9fbefb1', 'request_id': 'req-b65185e6-8bf1-4673-9034-ee158c8ac5a5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:17.788778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b65185e6-8bf1-4673-9034-ee158c8ac5a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55864f798646443bbdfe30f8b9fbefb1', 'request_id': 'req-b65185e6-8bf1-4673-9034-ee158c8ac5a5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:17.808407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b65185e6-8bf1-4673-9034-ee158c8ac5a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55864f798646443bbdfe30f8b9fbefb1', 'request_id': 'req-b65185e6-8bf1-4673-9034-ee158c8ac5a5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:17.841817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b65185e6-8bf1-4673-9034-ee158c8ac5a5 admin admin] The token cache is being invalidate because project 55864f798646443bbdfe30f8b9fbefb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:17.842258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b65185e6-8bf1-4673-9034-ee158c8ac5a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:17.845713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b65185e6-8bf1-4673-9034-ee158c8ac5a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:17.883957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474163f8-88c3-4c4c-8636-78e787e5fd32 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:17.902855 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3193/6318] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:17 2022] DELETE /identity/v3/projects/55864f798646443bbdfe30f8b9fbefb1 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:17.907593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474163f8-88c3-4c4c-8636-78e787e5fd32 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:18.040614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4204e8-2c6d-4010-82a6-45d7c47e96d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:18.041609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4204e8-2c6d-4010-82a6-45d7c47e96d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:18.042015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4204e8-2c6d-4010-82a6-45d7c47e96d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:18.042243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4204e8-2c6d-4010-82a6-45d7c47e96d4 admin admin] PATH_INFO: `/v3/users/e4c5c1b5f5e74dd1a8defcdd94c73da4` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:18.044816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4204e8-2c6d-4010-82a6-45d7c47e96d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4c5c1b5f5e74dd1a8defcdd94c73da4)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:18.047939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4204e8-2c6d-4010-82a6-45d7c47e96d4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:18.069848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:18.157174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-df4204e8-2c6d-4010-82a6-45d7c47e96d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4c5c1b5f5e74dd1a8defcdd94c73da4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:18.158247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-df4204e8-2c6d-4010-82a6-45d7c47e96d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4c5c1b5f5e74dd1a8defcdd94c73da4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:18.159075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-df4204e8-2c6d-4010-82a6-45d7c47e96d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4c5c1b5f5e74dd1a8defcdd94c73da4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:18.179513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-df4204e8-2c6d-4010-82a6-45d7c47e96d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4c5c1b5f5e74dd1a8defcdd94c73da4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:18.213989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-df4204e8-2c6d-4010-82a6-45d7c47e96d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4c5c1b5f5e74dd1a8defcdd94c73da4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:18.243574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65185e6-8bf1-4673-9034-ee158c8ac5a5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:18.246332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b65185e6-8bf1-4673-9034-ee158c8ac5a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:18.266676 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3126/6319] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:17 2022] DELETE /identity/v3/users/e4c5c1b5f5e74dd1a8defcdd94c73da4 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:18.271421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65185e6-8bf1-4673-9034-ee158c8ac5a5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:18.381408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc3d00-41fb-4a5d-863b-823a9a909642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:18.382336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc3d00-41fb-4a5d-863b-823a9a909642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:18.383039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc3d00-41fb-4a5d-863b-823a9a909642 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:18.383662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc3d00-41fb-4a5d-863b-823a9a909642 admin admin] PATH_INFO: `/v3/projects/fdc98fc841664be2bc220da1a2f0a9fe` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:18.385222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc3d00-41fb-4a5d-863b-823a9a909642 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdc98fc841664be2bc220da1a2f0a9fe)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:18.388696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc3d00-41fb-4a5d-863b-823a9a909642 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:18.487065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-33bc3d00-41fb-4a5d-863b-823a9a909642 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdc98fc841664be2bc220da1a2f0a9fe', 'request_id': 'req-33bc3d00-41fb-4a5d-863b-823a9a909642'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:18.501685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-33bc3d00-41fb-4a5d-863b-823a9a909642 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdc98fc841664be2bc220da1a2f0a9fe', 'request_id': 'req-33bc3d00-41fb-4a5d-863b-823a9a909642'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:18.529750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-33bc3d00-41fb-4a5d-863b-823a9a909642 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdc98fc841664be2bc220da1a2f0a9fe', 'request_id': 'req-33bc3d00-41fb-4a5d-863b-823a9a909642'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:18.580033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-33bc3d00-41fb-4a5d-863b-823a9a909642 admin admin] The token cache is being invalidate because project fdc98fc841664be2bc220da1a2f0a9fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:18.581871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-33bc3d00-41fb-4a5d-863b-823a9a909642 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:18.585329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-33bc3d00-41fb-4a5d-863b-823a9a909642 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:18.594985 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3194/6320] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:18 2022] DELETE /identity/v3/projects/fdc98fc841664be2bc220da1a2f0a9fe => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:18.599161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4204e8-2c6d-4010-82a6-45d7c47e96d4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:18.600642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-df4204e8-2c6d-4010-82a6-45d7c47e96d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:18.632008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc3d00-41fb-4a5d-863b-823a9a909642 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:18.634784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-33bc3d00-41fb-4a5d-863b-823a9a909642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:18.640050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4204e8-2c6d-4010-82a6-45d7c47e96d4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:18.670259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:18.793539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a4dde-a522-44f1-a651-47f0f58e0461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:18.794379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a4dde-a522-44f1-a651-47f0f58e0461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:18.794694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a4dde-a522-44f1-a651-47f0f58e0461 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:18.794975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a4dde-a522-44f1-a651-47f0f58e0461 admin admin] PATH_INFO: `/v3/users/c6d81496ff5144969c4ff654eb547b73` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:18.802466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a4dde-a522-44f1-a651-47f0f58e0461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6d81496ff5144969c4ff654eb547b73)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:18.804688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a4dde-a522-44f1-a651-47f0f58e0461 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:18.850266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc3d00-41fb-4a5d-863b-823a9a909642 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:18.861806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4757ee5d-1e86-4263-a028-226cc686c0be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:18.862472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757ee5d-1e86-4263-a028-226cc686c0be service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:18.862990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757ee5d-1e86-4263-a028-226cc686c0be service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:18.863300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757ee5d-1e86-4263-a028-226cc686c0be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:18.874551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4757ee5d-1e86-4263-a028-226cc686c0be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:18.918864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4757ee5d-1e86-4263-a028-226cc686c0be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:18.922991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4757ee5d-1e86-4263-a028-226cc686c0be service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:18.925565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f41a4dde-a522-44f1-a651-47f0f58e0461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6d81496ff5144969c4ff654eb547b73'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:18.927202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f41a4dde-a522-44f1-a651-47f0f58e0461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6d81496ff5144969c4ff654eb547b73'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:18.934962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f41a4dde-a522-44f1-a651-47f0f58e0461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6d81496ff5144969c4ff654eb547b73'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:18.964721 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3195/6321] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:18.972652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f41a4dde-a522-44f1-a651-47f0f58e0461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6d81496ff5144969c4ff654eb547b73'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:19.009909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f41a4dde-a522-44f1-a651-47f0f58e0461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6d81496ff5144969c4ff654eb547b73'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:19.057586 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3127/6322] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:18 2022] DELETE /identity/v3/users/c6d81496ff5144969c4ff654eb547b73 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:19.659734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4757ee5d-1e86-4263-a028-226cc686c0be service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:19.661337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4757ee5d-1e86-4263-a028-226cc686c0be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:19.679275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4757ee5d-1e86-4263-a028-226cc686c0be service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:19.696353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a4dde-a522-44f1-a651-47f0f58e0461 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:19.699833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f41a4dde-a522-44f1-a651-47f0f58e0461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:19.710872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:19.776414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89682731-4169-4131-9982-e5ab9bf9b711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:19.777227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89682731-4169-4131-9982-e5ab9bf9b711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:19.778710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89682731-4169-4131-9982-e5ab9bf9b711 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:19.779077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89682731-4169-4131-9982-e5ab9bf9b711 admin admin] PATH_INFO: `/v3/projects/316a7457d230463dbea005d8ba1d4019` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:19.781752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89682731-4169-4131-9982-e5ab9bf9b711 admin admin] RBAC: Authorizing `identity:delete_project(project_id=316a7457d230463dbea005d8ba1d4019)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:19.785545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89682731-4169-4131-9982-e5ab9bf9b711 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:19.856929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a4dde-a522-44f1-a651-47f0f58e0461 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:19.876170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-89682731-4169-4131-9982-e5ab9bf9b711 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '316a7457d230463dbea005d8ba1d4019', 'request_id': 'req-89682731-4169-4131-9982-e5ab9bf9b711'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:19.884077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-89682731-4169-4131-9982-e5ab9bf9b711 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '316a7457d230463dbea005d8ba1d4019', 'request_id': 'req-89682731-4169-4131-9982-e5ab9bf9b711'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:19.892398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-89682731-4169-4131-9982-e5ab9bf9b711 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '316a7457d230463dbea005d8ba1d4019', 'request_id': 'req-89682731-4169-4131-9982-e5ab9bf9b711'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:19.914316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-89682731-4169-4131-9982-e5ab9bf9b711 admin admin] The token cache is being invalidate because project 316a7457d230463dbea005d8ba1d4019 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:19.914662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-89682731-4169-4131-9982-e5ab9bf9b711 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:19.915392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-89682731-4169-4131-9982-e5ab9bf9b711 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:19.918975 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3196/6323] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:19 2022] DELETE /identity/v3/projects/316a7457d230463dbea005d8ba1d4019 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:19.924486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421136d8-7f5b-49b8-9c7e-d09a1a0d3511 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:19.925225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421136d8-7f5b-49b8-9c7e-d09a1a0d3511 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:19.926144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421136d8-7f5b-49b8-9c7e-d09a1a0d3511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:19.979459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e25ece1-2b9b-4239-8f10-03af5c59d16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:19.981180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25ece1-2b9b-4239-8f10-03af5c59d16a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:19.981584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25ece1-2b9b-4239-8f10-03af5c59d16a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:19.982996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25ece1-2b9b-4239-8f10-03af5c59d16a admin admin] PATH_INFO: `/v3/projects/6aa7e9d55eee4849ad5b38e01f85f480` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:19.986705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e25ece1-2b9b-4239-8f10-03af5c59d16a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aa7e9d55eee4849ad5b38e01f85f480)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:19.990181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e25ece1-2b9b-4239-8f10-03af5c59d16a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:19.993014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-421136d8-7f5b-49b8-9c7e-d09a1a0d3511 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:19.994242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-421136d8-7f5b-49b8-9c7e-d09a1a0d3511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:20.013004 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3197/6324] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:20.014246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421136d8-7f5b-49b8-9c7e-d09a1a0d3511 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:20.016237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-421136d8-7f5b-49b8-9c7e-d09a1a0d3511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:20.043322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421136d8-7f5b-49b8-9c7e-d09a1a0d3511 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:20.049590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06146b55-1816-46b1-afa7-9831665e13b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:20.050245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06146b55-1816-46b1-afa7-9831665e13b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:20.050688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06146b55-1816-46b1-afa7-9831665e13b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:20.050972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06146b55-1816-46b1-afa7-9831665e13b4 admin admin] PATH_INFO: `/v3/users/484ee8f965604fa9b7f7f5a25a81d0ef` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:20.052463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06146b55-1816-46b1-afa7-9831665e13b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=484ee8f965604fa9b7f7f5a25a81d0ef)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:20.054143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06146b55-1816-46b1-afa7-9831665e13b4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:20.070674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6e25ece1-2b9b-4239-8f10-03af5c59d16a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aa7e9d55eee4849ad5b38e01f85f480', 'request_id': 'req-6e25ece1-2b9b-4239-8f10-03af5c59d16a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:20.086527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6e25ece1-2b9b-4239-8f10-03af5c59d16a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aa7e9d55eee4849ad5b38e01f85f480', 'request_id': 'req-6e25ece1-2b9b-4239-8f10-03af5c59d16a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:20.091234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:20.095317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6e25ece1-2b9b-4239-8f10-03af5c59d16a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aa7e9d55eee4849ad5b38e01f85f480', 'request_id': 'req-6e25ece1-2b9b-4239-8f10-03af5c59d16a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:20.108436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6e25ece1-2b9b-4239-8f10-03af5c59d16a admin admin] The token cache is being invalidate because project 6aa7e9d55eee4849ad5b38e01f85f480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:20.108735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6e25ece1-2b9b-4239-8f10-03af5c59d16a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:20.109643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6e25ece1-2b9b-4239-8f10-03af5c59d16a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:20.111414 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3128/6325] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:19 2022] DELETE /identity/v3/projects/6aa7e9d55eee4849ad5b38e01f85f480 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:20.116581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b9103-15c3-4f30-afc4-f6ac91e82a99 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:20.117050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b9103-15c3-4f30-afc4-f6ac91e82a99 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:20.117335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b9103-15c3-4f30-afc4-f6ac91e82a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:20.138030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-06146b55-1816-46b1-afa7-9831665e13b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '484ee8f965604fa9b7f7f5a25a81d0ef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:20.146254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-06146b55-1816-46b1-afa7-9831665e13b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '484ee8f965604fa9b7f7f5a25a81d0ef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:20.149273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-dc5b9103-15c3-4f30-afc4-f6ac91e82a99 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:20.151382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-dc5b9103-15c3-4f30-afc4-f6ac91e82a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:20.164587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-06146b55-1816-46b1-afa7-9831665e13b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '484ee8f965604fa9b7f7f5a25a81d0ef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:20.165391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-06146b55-1816-46b1-afa7-9831665e13b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '484ee8f965604fa9b7f7f5a25a81d0ef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:20.176220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-06146b55-1816-46b1-afa7-9831665e13b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '484ee8f965604fa9b7f7f5a25a81d0ef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:20.185749 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3198/6326] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:20 2022] DELETE /identity/v3/users/484ee8f965604fa9b7f7f5a25a81d0ef => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:20.186910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06146b55-1816-46b1-afa7-9831665e13b4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:20.188605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-06146b55-1816-46b1-afa7-9831665e13b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:20.206642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:20.229651 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3129/6327] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:20.231098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b550e68-e776-4bd1-ad0e-f4dbb2abdb4d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:20.231387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b550e68-e776-4bd1-ad0e-f4dbb2abdb4d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:20.231663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b550e68-e776-4bd1-ad0e-f4dbb2abdb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:20.251279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3b550e68-e776-4bd1-ad0e-f4dbb2abdb4d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:20.252524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3b550e68-e776-4bd1-ad0e-f4dbb2abdb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:20.264948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06146b55-1816-46b1-afa7-9831665e13b4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:20.265822 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3130/6328] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:20.267329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b550e68-e776-4bd1-ad0e-f4dbb2abdb4d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:20.273363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531dd922-06a3-418d-99e4-35f83bd0fd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:20.274186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531dd922-06a3-418d-99e4-35f83bd0fd3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:20.274478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531dd922-06a3-418d-99e4-35f83bd0fd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:20.274798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531dd922-06a3-418d-99e4-35f83bd0fd3b admin admin] PATH_INFO: `/v3/users/bc70398b1ade4941a5bf129f87a3b1d1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:20.276015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531dd922-06a3-418d-99e4-35f83bd0fd3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc70398b1ade4941a5bf129f87a3b1d1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:20.278281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531dd922-06a3-418d-99e4-35f83bd0fd3b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:20.278692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b550e68-e776-4bd1-ad0e-f4dbb2abdb4d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:20.284875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f915a-7309-4e5a-8fc6-32d6c4a460c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:20.286525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f915a-7309-4e5a-8fc6-32d6c4a460c4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:20.286830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f915a-7309-4e5a-8fc6-32d6c4a460c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:20.287109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f915a-7309-4e5a-8fc6-32d6c4a460c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:20.287776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f915a-7309-4e5a-8fc6-32d6c4a460c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:13:20.288114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f915a-7309-4e5a-8fc6-32d6c4a460c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:20.289794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f915a-7309-4e5a-8fc6-32d6c4a460c4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:20.306413 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3131/6329] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:13:20.308419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f915a-7309-4e5a-8fc6-32d6c4a460c4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:20.310823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d24f915a-7309-4e5a-8fc6-32d6c4a460c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:20.339124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-531dd922-06a3-418d-99e4-35f83bd0fd3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc70398b1ade4941a5bf129f87a3b1d1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:20.346987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-531dd922-06a3-418d-99e4-35f83bd0fd3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc70398b1ade4941a5bf129f87a3b1d1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:20.364801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-531dd922-06a3-418d-99e4-35f83bd0fd3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc70398b1ade4941a5bf129f87a3b1d1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:20.365887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-531dd922-06a3-418d-99e4-35f83bd0fd3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc70398b1ade4941a5bf129f87a3b1d1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:20.375737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-531dd922-06a3-418d-99e4-35f83bd0fd3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc70398b1ade4941a5bf129f87a3b1d1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:20.381103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f915a-7309-4e5a-8fc6-32d6c4a460c4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:20.386734 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3199/6330] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:20 2022] DELETE /identity/v3/users/bc70398b1ade4941a5bf129f87a3b1d1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:20.387589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531dd922-06a3-418d-99e4-35f83bd0fd3b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:20.389283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-531dd922-06a3-418d-99e4-35f83bd0fd3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:20.435648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:20.457954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded4f2a-9809-4dc0-9d0d-6c67865d970c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:20.458673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded4f2a-9809-4dc0-9d0d-6c67865d970c service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:20.459252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded4f2a-9809-4dc0-9d0d-6c67865d970c service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:20.459252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded4f2a-9809-4dc0-9d0d-6c67865d970c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:20.465545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9ded4f2a-9809-4dc0-9d0d-6c67865d970c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:20.507644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531dd922-06a3-418d-99e4-35f83bd0fd3b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:20.516120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c642b-2a6b-4e09-9538-0dca8a1ee9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:20.517779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c642b-2a6b-4e09-9538-0dca8a1ee9a7 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:20.518297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c642b-2a6b-4e09-9538-0dca8a1ee9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:20.520755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c642b-2a6b-4e09-9538-0dca8a1ee9a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:20.522125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c642b-2a6b-4e09-9538-0dca8a1ee9a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:20.527098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c642b-2a6b-4e09-9538-0dca8a1ee9a7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:20.571947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded4f2a-9809-4dc0-9d0d-6c67865d970c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:20.574871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded4f2a-9809-4dc0-9d0d-6c67865d970c service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:20.588100 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3132/6331] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:20.589567 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3200/6332] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:20 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:20.593652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded4f2a-9809-4dc0-9d0d-6c67865d970c service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:20.605285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded4f2a-9809-4dc0-9d0d-6c67865d970c service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:20.640379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:20.692118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd81d7-cbf2-4cb0-b2a4-5a6930cf5897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:20.692789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad339d8-7ac7-4768-85ba-1952094c01f0 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:20.693239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd81d7-cbf2-4cb0-b2a4-5a6930cf5897 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:20.693345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad339d8-7ac7-4768-85ba-1952094c01f0 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:20.693456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd81d7-cbf2-4cb0-b2a4-5a6930cf5897 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:20.693556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad339d8-7ac7-4768-85ba-1952094c01f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:20.693708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd81d7-cbf2-4cb0-b2a4-5a6930cf5897 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:20.694404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd81d7-cbf2-4cb0-b2a4-5a6930cf5897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:20.697513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd81d7-cbf2-4cb0-b2a4-5a6930cf5897 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:20.728702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6ad339d8-7ac7-4768-85ba-1952094c01f0 None None] MFA Rules not processed for user `a2745d6923e347939e2c67015aefadda`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:20.730042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6ad339d8-7ac7-4768-85ba-1952094c01f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:20.735211 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3133/6333] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:20 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:20.739291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd81d7-cbf2-4cb0-b2a4-5a6930cf5897 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:20.759378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd81d7-cbf2-4cb0-b2a4-5a6930cf5897 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:20.766482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d85be-7cc0-4b70-afee-9b448e728fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:20.767791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d85be-7cc0-4b70-afee-9b448e728fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:20.768526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d85be-7cc0-4b70-afee-9b448e728fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:20.768913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d85be-7cc0-4b70-afee-9b448e728fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:20.770240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d85be-7cc0-4b70-afee-9b448e728fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:20.773084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d85be-7cc0-4b70-afee-9b448e728fa5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:20.787476 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3134/6334] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:20.791153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d85be-7cc0-4b70-afee-9b448e728fa5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:20.803915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d85be-7cc0-4b70-afee-9b448e728fa5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:20.810479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3144a3-3867-404c-99e1-7fef1b6c03b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:20.811639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3144a3-3867-404c-99e1-7fef1b6c03b5 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:20.811904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3144a3-3867-404c-99e1-7fef1b6c03b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:20.812310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3144a3-3867-404c-99e1-7fef1b6c03b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:20.812852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3144a3-3867-404c-99e1-7fef1b6c03b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:20.814974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3144a3-3867-404c-99e1-7fef1b6c03b5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:20.829322 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3135/6335] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:20.835444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3144a3-3867-404c-99e1-7fef1b6c03b5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:20.837725 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3201/6336] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:20.865516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3144a3-3867-404c-99e1-7fef1b6c03b5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:20.874690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3d9dc-67c0-49a7-8352-4c8b375e40fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:20.875641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3d9dc-67c0-49a7-8352-4c8b375e40fb admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:20.876000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3d9dc-67c0-49a7-8352-4c8b375e40fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:20.876345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3d9dc-67c0-49a7-8352-4c8b375e40fb admin admin] PATH_INFO: `/v3/projects/f1168e0d1a9b4d0aba476f95c4e6213e/users/5ba8405d7413464e8055529b65da4bc6/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:20.893878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad339d8-7ac7-4768-85ba-1952094c01f0 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:20.895956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6ad339d8-7ac7-4768-85ba-1952094c01f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:20.918785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad339d8-7ac7-4768-85ba-1952094c01f0 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:20.924739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3d9dc-67c0-49a7-8352-4c8b375e40fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1168e0d1a9b4d0aba476f95c4e6213e, user_id=5ba8405d7413464e8055529b65da4bc6, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:20.925951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bd8a3-4369-473a-8dc2-cf9a398bb75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:20.927193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bd8a3-4369-473a-8dc2-cf9a398bb75f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:20.927933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bd8a3-4369-473a-8dc2-cf9a398bb75f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:20.928256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bd8a3-4369-473a-8dc2-cf9a398bb75f admin admin] PATH_INFO: `/v3/projects/58597598ab8947e497b37172b49971c3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:20.929573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3d9dc-67c0-49a7-8352-4c8b375e40fb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:20.931149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bd8a3-4369-473a-8dc2-cf9a398bb75f admin admin] RBAC: Authorizing `identity:delete_project(project_id=58597598ab8947e497b37172b49971c3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:20.938186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bd8a3-4369-473a-8dc2-cf9a398bb75f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:20.960839 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3136/6337] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:20 2022] PUT /identity/v3/projects/f1168e0d1a9b4d0aba476f95c4e6213e/users/5ba8405d7413464e8055529b65da4bc6/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:20.969111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3d9dc-67c0-49a7-8352-4c8b375e40fb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:20.972341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2cb3d9dc-67c0-49a7-8352-4c8b375e40fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:21.052466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e51bd8a3-4369-473a-8dc2-cf9a398bb75f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58597598ab8947e497b37172b49971c3', 'request_id': 'req-e51bd8a3-4369-473a-8dc2-cf9a398bb75f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:21.063972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e51bd8a3-4369-473a-8dc2-cf9a398bb75f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58597598ab8947e497b37172b49971c3', 'request_id': 'req-e51bd8a3-4369-473a-8dc2-cf9a398bb75f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:21.070527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e51bd8a3-4369-473a-8dc2-cf9a398bb75f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58597598ab8947e497b37172b49971c3', 'request_id': 'req-e51bd8a3-4369-473a-8dc2-cf9a398bb75f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:21.081362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3d9dc-67c0-49a7-8352-4c8b375e40fb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:21.089072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e51bd8a3-4369-473a-8dc2-cf9a398bb75f admin admin] The token cache is being invalidate because project 58597598ab8947e497b37172b49971c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:21.089361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e51bd8a3-4369-473a-8dc2-cf9a398bb75f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:21.089998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e51bd8a3-4369-473a-8dc2-cf9a398bb75f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:21.095950 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3202/6338] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:20 2022] DELETE /identity/v3/projects/58597598ab8947e497b37172b49971c3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:21.095950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393c10f-7881-4d9a-9f02-4d2eb006f664 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:21.095950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393c10f-7881-4d9a-9f02-4d2eb006f664 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:21.095950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393c10f-7881-4d9a-9f02-4d2eb006f664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:21.126616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0393c10f-7881-4d9a-9f02-4d2eb006f664 None None] MFA Rules not processed for user `5ba8405d7413464e8055529b65da4bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:21.128836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0393c10f-7881-4d9a-9f02-4d2eb006f664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:21.146753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86d8ac-a6f0-43e2-8c2c-53ce9b69292f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:21.147492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86d8ac-a6f0-43e2-8c2c-53ce9b69292f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:21.148797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86d8ac-a6f0-43e2-8c2c-53ce9b69292f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:21.148961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86d8ac-a6f0-43e2-8c2c-53ce9b69292f admin admin] PATH_INFO: `/v3/projects/6d1eb39d09ce4a87960e258ac890e32c` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:21.151180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:21.151324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86d8ac-a6f0-43e2-8c2c-53ce9b69292f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d1eb39d09ce4a87960e258ac890e32c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:21.153398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86d8ac-a6f0-43e2-8c2c-53ce9b69292f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:21.188248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:21.208807 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3203/6339] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:21.210156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad32a1-9a6b-4d32-83ae-93f440117edd None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:21.210439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad32a1-9a6b-4d32-83ae-93f440117edd None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:21.210742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad32a1-9a6b-4d32-83ae-93f440117edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:21.224915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0b86d8ac-a6f0-43e2-8c2c-53ce9b69292f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d1eb39d09ce4a87960e258ac890e32c', 'request_id': 'req-0b86d8ac-a6f0-43e2-8c2c-53ce9b69292f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:21.231476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-aaad32a1-9a6b-4d32-83ae-93f440117edd None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:21.232738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-aaad32a1-9a6b-4d32-83ae-93f440117edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:21.235938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0b86d8ac-a6f0-43e2-8c2c-53ce9b69292f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d1eb39d09ce4a87960e258ac890e32c', 'request_id': 'req-0b86d8ac-a6f0-43e2-8c2c-53ce9b69292f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:21.244027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0b86d8ac-a6f0-43e2-8c2c-53ce9b69292f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d1eb39d09ce4a87960e258ac890e32c', 'request_id': 'req-0b86d8ac-a6f0-43e2-8c2c-53ce9b69292f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:21.261721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0b86d8ac-a6f0-43e2-8c2c-53ce9b69292f admin admin] The token cache is being invalidate because project 6d1eb39d09ce4a87960e258ac890e32c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:21.262320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0b86d8ac-a6f0-43e2-8c2c-53ce9b69292f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:21.263048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0b86d8ac-a6f0-43e2-8c2c-53ce9b69292f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:21.264783 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3137/6340] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:20 2022] DELETE /identity/v3/projects/6d1eb39d09ce4a87960e258ac890e32c => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:21.265979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86d8ac-a6f0-43e2-8c2c-53ce9b69292f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:21.267471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0b86d8ac-a6f0-43e2-8c2c-53ce9b69292f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:21.305504 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3204/6341] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:21.306685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad32a1-9a6b-4d32-83ae-93f440117edd None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:21.308218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-aaad32a1-9a6b-4d32-83ae-93f440117edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:21.321239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad32a1-9a6b-4d32-83ae-93f440117edd None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:21.326775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f236432-5332-45fe-ac2f-a62a7a6b659a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:21.327382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f236432-5332-45fe-ac2f-a62a7a6b659a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:21.327666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f236432-5332-45fe-ac2f-a62a7a6b659a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:21.328024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f236432-5332-45fe-ac2f-a62a7a6b659a admin admin] PATH_INFO: `/v3/users/3c809773d07c44ee8f296e39613405ef` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:21.329081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f236432-5332-45fe-ac2f-a62a7a6b659a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c809773d07c44ee8f296e39613405ef)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:21.331079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f236432-5332-45fe-ac2f-a62a7a6b659a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:21.342594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86d8ac-a6f0-43e2-8c2c-53ce9b69292f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:21.348636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce34dc-a8fb-4d0a-8268-49016cff8e7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:21.349363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce34dc-a8fb-4d0a-8268-49016cff8e7b service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:21.349744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce34dc-a8fb-4d0a-8268-49016cff8e7b service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:21.350096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce34dc-a8fb-4d0a-8268-49016cff8e7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:21.353153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-35ce34dc-a8fb-4d0a-8268-49016cff8e7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:21.384123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7f236432-5332-45fe-ac2f-a62a7a6b659a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c809773d07c44ee8f296e39613405ef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:21.392430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7f236432-5332-45fe-ac2f-a62a7a6b659a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c809773d07c44ee8f296e39613405ef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:21.408869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7f236432-5332-45fe-ac2f-a62a7a6b659a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c809773d07c44ee8f296e39613405ef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:21.410196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7f236432-5332-45fe-ac2f-a62a7a6b659a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c809773d07c44ee8f296e39613405ef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:21.419359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7f236432-5332-45fe-ac2f-a62a7a6b659a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c809773d07c44ee8f296e39613405ef'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:21.425130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce34dc-a8fb-4d0a-8268-49016cff8e7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:21.426976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce34dc-a8fb-4d0a-8268-49016cff8e7b service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:21.427689 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3205/6342] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:21 2022] DELETE /identity/v3/users/3c809773d07c44ee8f296e39613405ef => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:21.428911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c5247-5f9d-42aa-88e6-8dc99abcf8f7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:21.429298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c5247-5f9d-42aa-88e6-8dc99abcf8f7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:21.429675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c5247-5f9d-42aa-88e6-8dc99abcf8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:21.453416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-349c5247-5f9d-42aa-88e6-8dc99abcf8f7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:21.454890 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3138/6343] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:21.463076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-349c5247-5f9d-42aa-88e6-8dc99abcf8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:21.475906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce34dc-a8fb-4d0a-8268-49016cff8e7b service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:21.477763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-35ce34dc-a8fb-4d0a-8268-49016cff8e7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:21.497115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce34dc-a8fb-4d0a-8268-49016cff8e7b service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:21.520440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:21.576476 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3206/6344] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:21.580350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c5247-5f9d-42aa-88e6-8dc99abcf8f7 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:21.593161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c5247-5f9d-42aa-88e6-8dc99abcf8f7 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:21.599106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03f383-8621-41dd-953a-60922d3c400f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:21.600019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03f383-8621-41dd-953a-60922d3c400f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:21.600401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03f383-8621-41dd-953a-60922d3c400f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:21.600779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03f383-8621-41dd-953a-60922d3c400f admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:21.601769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03f383-8621-41dd-953a-60922d3c400f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:13:21.602177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03f383-8621-41dd-953a-60922d3c400f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:21.604471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03f383-8621-41dd-953a-60922d3c400f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:21.633335 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3207/6345] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:13:21.637818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03f383-8621-41dd-953a-60922d3c400f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:21.663556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03f383-8621-41dd-953a-60922d3c400f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:21.676274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f7f11-8980-48ac-9c10-e51bbb4240ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:21.676452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937393f-303e-4bcc-b48d-5df50ed44801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:21.677077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937393f-303e-4bcc-b48d-5df50ed44801 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:21.677206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f7f11-8980-48ac-9c10-e51bbb4240ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:21.677411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937393f-303e-4bcc-b48d-5df50ed44801 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:21.677527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f7f11-8980-48ac-9c10-e51bbb4240ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:21.677617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937393f-303e-4bcc-b48d-5df50ed44801 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:21.677787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f7f11-8980-48ac-9c10-e51bbb4240ea admin admin] PATH_INFO: `/v3/users/5ba8405d7413464e8055529b65da4bc6` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:21.680768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937393f-303e-4bcc-b48d-5df50ed44801 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:21.681729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f7f11-8980-48ac-9c10-e51bbb4240ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ba8405d7413464e8055529b65da4bc6)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:21.685953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f7f11-8980-48ac-9c10-e51bbb4240ea admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:21.688051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937393f-303e-4bcc-b48d-5df50ed44801 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:21.708977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:21.795957 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3208/6346] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:21 2022] POST /identity/v3/projects => generated 387 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:13:21.801621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937393f-303e-4bcc-b48d-5df50ed44801 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:21.802041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-069f7f11-8980-48ac-9c10-e51bbb4240ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ba8405d7413464e8055529b65da4bc6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:21.803002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-069f7f11-8980-48ac-9c10-e51bbb4240ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ba8405d7413464e8055529b65da4bc6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:21.803795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-069f7f11-8980-48ac-9c10-e51bbb4240ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ba8405d7413464e8055529b65da4bc6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:21.815312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-069f7f11-8980-48ac-9c10-e51bbb4240ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ba8405d7413464e8055529b65da4bc6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:21.846995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-069f7f11-8980-48ac-9c10-e51bbb4240ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ba8405d7413464e8055529b65da4bc6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:21.890539 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3139/6347] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:21 2022] DELETE /identity/v3/users/5ba8405d7413464e8055529b65da4bc6 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:21.890539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f7f11-8980-48ac-9c10-e51bbb4240ea admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:21.899121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-069f7f11-8980-48ac-9c10-e51bbb4240ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:21.960501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937393f-303e-4bcc-b48d-5df50ed44801 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:22.017459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f7f11-8980-48ac-9c10-e51bbb4240ea admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:22.024269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584a3aa-245e-4ff1-9b82-b84a6475e2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:22.025054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584a3aa-245e-4ff1-9b82-b84a6475e2bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:22.025337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584a3aa-245e-4ff1-9b82-b84a6475e2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:22.025615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584a3aa-245e-4ff1-9b82-b84a6475e2bd admin admin] PATH_INFO: `/v3/projects/9bd8ee9103544a4dbd40fa6f54ae344b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:22.029950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d584a3aa-245e-4ff1-9b82-b84a6475e2bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bd8ee9103544a4dbd40fa6f54ae344b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:22.031980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d584a3aa-245e-4ff1-9b82-b84a6475e2bd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:22.039677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086d783-963e-4190-a240-336ee91e8dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:22.040552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086d783-963e-4190-a240-336ee91e8dae admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:22.041140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086d783-963e-4190-a240-336ee91e8dae admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:22.042076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086d783-963e-4190-a240-336ee91e8dae admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:22.042990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086d783-963e-4190-a240-336ee91e8dae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:22.045065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086d783-963e-4190-a240-336ee91e8dae admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:22.091324 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3209/6348] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:21 2022] POST /identity/v3/users => generated 294 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:13:22.093442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11b15e-d562-474e-98d3-1e6450b9188e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:22.093442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11b15e-d562-474e-98d3-1e6450b9188e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:22.093442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11b15e-d562-474e-98d3-1e6450b9188e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:22.120512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d584a3aa-245e-4ff1-9b82-b84a6475e2bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bd8ee9103544a4dbd40fa6f54ae344b', 'request_id': 'req-d584a3aa-245e-4ff1-9b82-b84a6475e2bd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:22.129692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d584a3aa-245e-4ff1-9b82-b84a6475e2bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bd8ee9103544a4dbd40fa6f54ae344b', 'request_id': 'req-d584a3aa-245e-4ff1-9b82-b84a6475e2bd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:22.130884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-bf11b15e-d562-474e-98d3-1e6450b9188e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:22.137215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bf11b15e-d562-474e-98d3-1e6450b9188e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:22.137851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d584a3aa-245e-4ff1-9b82-b84a6475e2bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bd8ee9103544a4dbd40fa6f54ae344b', 'request_id': 'req-d584a3aa-245e-4ff1-9b82-b84a6475e2bd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:22.155080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d584a3aa-245e-4ff1-9b82-b84a6475e2bd admin admin] The token cache is being invalidate because project 9bd8ee9103544a4dbd40fa6f54ae344b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:22.155080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d584a3aa-245e-4ff1-9b82-b84a6475e2bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:22.155080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d584a3aa-245e-4ff1-9b82-b84a6475e2bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:22.157562 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3140/6349] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:21 2022] DELETE /identity/v3/projects/9bd8ee9103544a4dbd40fa6f54ae344b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:22.158478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584a3aa-245e-4ff1-9b82-b84a6475e2bd admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:22.159953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d584a3aa-245e-4ff1-9b82-b84a6475e2bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:22.175903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:22.205046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:22.261428 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3210/6350] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:22.262872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11b15e-d562-474e-98d3-1e6450b9188e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:22.265813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bf11b15e-d562-474e-98d3-1e6450b9188e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:22.275379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584a3aa-245e-4ff1-9b82-b84a6475e2bd admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:22.283993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11b15e-d562-474e-98d3-1e6450b9188e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:22.288922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85756689-425e-4de6-90ba-457931e35da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:22.289559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85756689-425e-4de6-90ba-457931e35da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:22.289898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85756689-425e-4de6-90ba-457931e35da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:22.290110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85756689-425e-4de6-90ba-457931e35da2 admin admin] PATH_INFO: `/v3/users/cd3ed24e5dec462cb60d62198b9c54f4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:22.291350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85756689-425e-4de6-90ba-457931e35da2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd3ed24e5dec462cb60d62198b9c54f4)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:22.293253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85756689-425e-4de6-90ba-457931e35da2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:22.295316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b81f9-241c-4006-88cf-bedbcd92288b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:22.295950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b81f9-241c-4006-88cf-bedbcd92288b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:22.296243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b81f9-241c-4006-88cf-bedbcd92288b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:22.296949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b81f9-241c-4006-88cf-bedbcd92288b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:22.297481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b81f9-241c-4006-88cf-bedbcd92288b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:22.299772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b81f9-241c-4006-88cf-bedbcd92288b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:22.318968 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3141/6351] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:22.320935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b81f9-241c-4006-88cf-bedbcd92288b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:22.324897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3c1b81f9-241c-4006-88cf-bedbcd92288b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:22.376015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-85756689-425e-4de6-90ba-457931e35da2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd3ed24e5dec462cb60d62198b9c54f4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:22.389251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-85756689-425e-4de6-90ba-457931e35da2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd3ed24e5dec462cb60d62198b9c54f4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:22.409573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-85756689-425e-4de6-90ba-457931e35da2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd3ed24e5dec462cb60d62198b9c54f4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:22.410692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-85756689-425e-4de6-90ba-457931e35da2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd3ed24e5dec462cb60d62198b9c54f4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:22.424089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b81f9-241c-4006-88cf-bedbcd92288b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:22.427868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-85756689-425e-4de6-90ba-457931e35da2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd3ed24e5dec462cb60d62198b9c54f4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:22.428408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946d647-6b90-48b1-8504-7b7546c7e40d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:22.429020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946d647-6b90-48b1-8504-7b7546c7e40d service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:22.429369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946d647-6b90-48b1-8504-7b7546c7e40d service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:22.429782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946d647-6b90-48b1-8504-7b7546c7e40d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:22.431616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6946d647-6b90-48b1-8504-7b7546c7e40d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:22.435083 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3211/6352] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:22 2022] DELETE /identity/v3/users/cd3ed24e5dec462cb60d62198b9c54f4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:22.447172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85756689-425e-4de6-90ba-457931e35da2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:22.449114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946d647-6b90-48b1-8504-7b7546c7e40d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:22.450851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946d647-6b90-48b1-8504-7b7546c7e40d service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:22.461871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-85756689-425e-4de6-90ba-457931e35da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:22.464264 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3142/6353] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:22.587629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85756689-425e-4de6-90ba-457931e35da2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:22.598567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0993-d2db-4cce-a5ac-646f6bc358ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:22.599534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0993-d2db-4cce-a5ac-646f6bc358ba admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:22.600845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0993-d2db-4cce-a5ac-646f6bc358ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:22.601225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0993-d2db-4cce-a5ac-646f6bc358ba admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:22.601976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a0993-d2db-4cce-a5ac-646f6bc358ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:22.606541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a0993-d2db-4cce-a5ac-646f6bc358ba admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:22.623966 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3212/6354] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:22.628569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946d647-6b90-48b1-8504-7b7546c7e40d service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:22.644002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946d647-6b90-48b1-8504-7b7546c7e40d service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:22.652142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e3d0c-e4da-4902-aff6-bb5ac683bb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:22.652832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e3d0c-e4da-4902-aff6-bb5ac683bb39 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:22.653535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e3d0c-e4da-4902-aff6-bb5ac683bb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:22.653535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e3d0c-e4da-4902-aff6-bb5ac683bb39 admin admin] PATH_INFO: `/v3/projects/3fdf2a69b9294b5283858cc0a173676c/users/7a31f27f3bdf4a8ab606779999ddd894/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:22.677738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e3d0c-e4da-4902-aff6-bb5ac683bb39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fdf2a69b9294b5283858cc0a173676c, user_id=7a31f27f3bdf4a8ab606779999ddd894, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:22.681510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e3d0c-e4da-4902-aff6-bb5ac683bb39 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:22.702159 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3143/6355] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:22 2022] PUT /identity/v3/projects/3fdf2a69b9294b5283858cc0a173676c/users/7a31f27f3bdf4a8ab606779999ddd894/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:22.711303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0993-d2db-4cce-a5ac-646f6bc358ba admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:22.714286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7e9a0993-d2db-4cce-a5ac-646f6bc358ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:22.731632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec79d5-22b5-4c83-9648-7a0dbdcd9f19 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:22.732018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec79d5-22b5-4c83-9648-7a0dbdcd9f19 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:22.732619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec79d5-22b5-4c83-9648-7a0dbdcd9f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:22.753963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0993-d2db-4cce-a5ac-646f6bc358ba admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:22.787953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2bec79d5-22b5-4c83-9648-7a0dbdcd9f19 None None] MFA Rules not processed for user `7a31f27f3bdf4a8ab606779999ddd894`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:22.802457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2bec79d5-22b5-4c83-9648-7a0dbdcd9f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:22.827812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:22.850970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79a0a7-92ba-450d-8404-f57d59865c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:22.851903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79a0a7-92ba-450d-8404-f57d59865c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:22.852350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79a0a7-92ba-450d-8404-f57d59865c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:22.852788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79a0a7-92ba-450d-8404-f57d59865c18 admin admin] PATH_INFO: `/v3/projects/83d3f634613a4faeb0252cc6f3a6b9f4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:22.854203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79a0a7-92ba-450d-8404-f57d59865c18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83d3f634613a4faeb0252cc6f3a6b9f4)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:22.857300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79a0a7-92ba-450d-8404-f57d59865c18 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:22.933393 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3144/6356] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:22.938415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec79d5-22b5-4c83-9648-7a0dbdcd9f19 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:22.943279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fe79a0a7-92ba-450d-8404-f57d59865c18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83d3f634613a4faeb0252cc6f3a6b9f4', 'request_id': 'req-fe79a0a7-92ba-450d-8404-f57d59865c18'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:22.957716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fe79a0a7-92ba-450d-8404-f57d59865c18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83d3f634613a4faeb0252cc6f3a6b9f4', 'request_id': 'req-fe79a0a7-92ba-450d-8404-f57d59865c18'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:22.967531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec79d5-22b5-4c83-9648-7a0dbdcd9f19 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:22.967758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fe79a0a7-92ba-450d-8404-f57d59865c18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83d3f634613a4faeb0252cc6f3a6b9f4', 'request_id': 'req-fe79a0a7-92ba-450d-8404-f57d59865c18'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:22.986479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fe79a0a7-92ba-450d-8404-f57d59865c18 admin admin] The token cache is being invalidate because project 83d3f634613a4faeb0252cc6f3a6b9f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:22.987141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fe79a0a7-92ba-450d-8404-f57d59865c18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:22.989387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fe79a0a7-92ba-450d-8404-f57d59865c18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:22.991521 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3213/6357] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:22 2022] DELETE /identity/v3/projects/83d3f634613a4faeb0252cc6f3a6b9f4 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:13:22.993059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79a0a7-92ba-450d-8404-f57d59865c18 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:22.994837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:23.019833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fe79a0a7-92ba-450d-8404-f57d59865c18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:23.140480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b70f7-fa59-4607-bff8-9d0586e5e913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:23.144056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b70f7-fa59-4607-bff8-9d0586e5e913 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:23.144645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b70f7-fa59-4607-bff8-9d0586e5e913 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:23.147332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b70f7-fa59-4607-bff8-9d0586e5e913 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:23.148387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b70f7-fa59-4607-bff8-9d0586e5e913 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:23.151056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b70f7-fa59-4607-bff8-9d0586e5e913 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:23.196044 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3145/6358] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:22 2022] POST /identity/v3/projects => generated 385 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:23.200623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b70f7-fa59-4607-bff8-9d0586e5e913 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:23.202609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-480b70f7-fa59-4607-bff8-9d0586e5e913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:23.229152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79a0a7-92ba-450d-8404-f57d59865c18 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:23.337750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa171b-8243-4c83-ab28-1037f0f4c8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:23.339724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa171b-8243-4c83-ab28-1037f0f4c8de admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:23.340687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa171b-8243-4c83-ab28-1037f0f4c8de admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:23.341098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa171b-8243-4c83-ab28-1037f0f4c8de admin admin] PATH_INFO: `/v3/projects/f1168e0d1a9b4d0aba476f95c4e6213e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:23.343258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa171b-8243-4c83-ab28-1037f0f4c8de admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1168e0d1a9b4d0aba476f95c4e6213e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:23.347580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa171b-8243-4c83-ab28-1037f0f4c8de admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:23.353207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b70f7-fa59-4607-bff8-9d0586e5e913 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:23.373078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da470a-db63-4c0f-a313-987d277fc6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:23.373719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da470a-db63-4c0f-a313-987d277fc6a1 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:23.376418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da470a-db63-4c0f-a313-987d277fc6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:23.376972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da470a-db63-4c0f-a313-987d277fc6a1 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:23.377645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da470a-db63-4c0f-a313-987d277fc6a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:23.379858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da470a-db63-4c0f-a313-987d277fc6a1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:23.403064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:23.419283 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3146/6359] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:23 2022] POST /identity/v3/users => generated 299 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:13:23.424840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da470a-db63-4c0f-a313-987d277fc6a1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:23.440097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da470a-db63-4c0f-a313-987d277fc6a1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:23.442820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-46fa171b-8243-4c83-ab28-1037f0f4c8de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1168e0d1a9b4d0aba476f95c4e6213e', 'request_id': 'req-46fa171b-8243-4c83-ab28-1037f0f4c8de'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:23.452057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-46fa171b-8243-4c83-ab28-1037f0f4c8de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1168e0d1a9b4d0aba476f95c4e6213e', 'request_id': 'req-46fa171b-8243-4c83-ab28-1037f0f4c8de'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:23.459021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-46fa171b-8243-4c83-ab28-1037f0f4c8de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1168e0d1a9b4d0aba476f95c4e6213e', 'request_id': 'req-46fa171b-8243-4c83-ab28-1037f0f4c8de'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:23.477912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-46fa171b-8243-4c83-ab28-1037f0f4c8de admin admin] The token cache is being invalidate because project f1168e0d1a9b4d0aba476f95c4e6213e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:23.478361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-46fa171b-8243-4c83-ab28-1037f0f4c8de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:23.495196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-46fa171b-8243-4c83-ab28-1037f0f4c8de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:23.497266 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3214/6360] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:22 2022] DELETE /identity/v3/projects/f1168e0d1a9b4d0aba476f95c4e6213e => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:23.502375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f443a2a-eff7-46db-8928-c6d97d9fea5f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:23.502375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f443a2a-eff7-46db-8928-c6d97d9fea5f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:23.503129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f443a2a-eff7-46db-8928-c6d97d9fea5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:23.556472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9f443a2a-eff7-46db-8928-c6d97d9fea5f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:23.557994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9f443a2a-eff7-46db-8928-c6d97d9fea5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:23.561491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c165814-b7e5-4c95-b11f-a8ec7c8bc653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:23.562609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c165814-b7e5-4c95-b11f-a8ec7c8bc653 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:23.563144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c165814-b7e5-4c95-b11f-a8ec7c8bc653 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:23.563759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c165814-b7e5-4c95-b11f-a8ec7c8bc653 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:23.564632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c165814-b7e5-4c95-b11f-a8ec7c8bc653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:23.566706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c165814-b7e5-4c95-b11f-a8ec7c8bc653 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:23.568687 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3215/6361] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:23.577422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58072877-7bb0-42a9-a45c-68efb5f2ab67 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:23.578031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58072877-7bb0-42a9-a45c-68efb5f2ab67 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:23.578213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58072877-7bb0-42a9-a45c-68efb5f2ab67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:23.589106 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3147/6362] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:23.590250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c165814-b7e5-4c95-b11f-a8ec7c8bc653 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:23.592079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1c165814-b7e5-4c95-b11f-a8ec7c8bc653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:23.606018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-58072877-7bb0-42a9-a45c-68efb5f2ab67 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:23.607296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c165814-b7e5-4c95-b11f-a8ec7c8bc653 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:23.607425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-58072877-7bb0-42a9-a45c-68efb5f2ab67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:23.615779 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3216/6363] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:23.619750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1e071-a5cc-460b-8814-7d170e7c01c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:23.620504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1e071-a5cc-460b-8814-7d170e7c01c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:23.620785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1e071-a5cc-460b-8814-7d170e7c01c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:23.621140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1e071-a5cc-460b-8814-7d170e7c01c2 admin admin] PATH_INFO: `/v3/projects/0b19ee6c3f2540e6a298648052eec5ca/users/456ed7c7a5d6404eb4ffd35dd37ac981/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:23.622821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58072877-7bb0-42a9-a45c-68efb5f2ab67 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:23.634052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58072877-7bb0-42a9-a45c-68efb5f2ab67 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:23.644014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1e071-a5cc-460b-8814-7d170e7c01c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b19ee6c3f2540e6a298648052eec5ca, user_id=456ed7c7a5d6404eb4ffd35dd37ac981, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:23.647468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1e071-a5cc-460b-8814-7d170e7c01c2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:23.672803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278a738-e33f-42ad-9008-8a96c662312b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:23.673586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278a738-e33f-42ad-9008-8a96c662312b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:23.679265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278a738-e33f-42ad-9008-8a96c662312b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:23.681391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278a738-e33f-42ad-9008-8a96c662312b admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:23.682404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278a738-e33f-42ad-9008-8a96c662312b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:13:23.682991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278a738-e33f-42ad-9008-8a96c662312b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:23.684818 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3148/6364] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:23 2022] PUT /identity/v3/projects/0b19ee6c3f2540e6a298648052eec5ca/users/456ed7c7a5d6404eb4ffd35dd37ac981/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:13:23.684970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278a738-e33f-42ad-9008-8a96c662312b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:23.688462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1e071-a5cc-460b-8814-7d170e7c01c2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:23.696778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:23.704191 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3217/6365] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:13:23.708472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278a738-e33f-42ad-9008-8a96c662312b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:23.723619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278a738-e33f-42ad-9008-8a96c662312b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:23.787281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1e071-a5cc-460b-8814-7d170e7c01c2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:23.790026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480be4a-1bfe-4a82-9391-6dd157133fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:23.790681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480be4a-1bfe-4a82-9391-6dd157133fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:23.790964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480be4a-1bfe-4a82-9391-6dd157133fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:23.791093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25835a68-de3e-400f-bfa2-622ea523de26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:23.792033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480be4a-1bfe-4a82-9391-6dd157133fe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:23.792033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480be4a-1bfe-4a82-9391-6dd157133fe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:23.792258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25835a68-de3e-400f-bfa2-622ea523de26 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:23.792356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25835a68-de3e-400f-bfa2-622ea523de26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:23.792749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25835a68-de3e-400f-bfa2-622ea523de26 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:23.793477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25835a68-de3e-400f-bfa2-622ea523de26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:23.793674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480be4a-1bfe-4a82-9391-6dd157133fe0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:23.795555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25835a68-de3e-400f-bfa2-622ea523de26 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:23.805285 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3149/6366] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:13:23.815737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480be4a-1bfe-4a82-9391-6dd157133fe0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:23.831180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480be4a-1bfe-4a82-9391-6dd157133fe0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:23.836018 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3218/6367] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:23 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:23.841115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25835a68-de3e-400f-bfa2-622ea523de26 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:23.844606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8cc316-65d7-4d0b-acea-861c5e97e303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:23.845541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cc316-65d7-4d0b-acea-861c5e97e303 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:23.846026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cc316-65d7-4d0b-acea-861c5e97e303 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:23.846529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cc316-65d7-4d0b-acea-861c5e97e303 admin admin] PATH_INFO: `/v3/projects/0b19ee6c3f2540e6a298648052eec5ca/users/456ed7c7a5d6404eb4ffd35dd37ac981/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:23.854337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8cc316-65d7-4d0b-acea-861c5e97e303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b19ee6c3f2540e6a298648052eec5ca, user_id=456ed7c7a5d6404eb4ffd35dd37ac981, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:23.859764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8cc316-65d7-4d0b-acea-861c5e97e303 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:23.881199 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3150/6368] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:23 2022] PUT /identity/v3/projects/0b19ee6c3f2540e6a298648052eec5ca/users/456ed7c7a5d6404eb4ffd35dd37ac981/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:23.893334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d681f4-c93a-4fca-9d0e-670b2a13bd3f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:23.893679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d681f4-c93a-4fca-9d0e-670b2a13bd3f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:23.894145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d681f4-c93a-4fca-9d0e-670b2a13bd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:23.934023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d6d681f4-c93a-4fca-9d0e-670b2a13bd3f None None] MFA Rules not processed for user `456ed7c7a5d6404eb4ffd35dd37ac981`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:23.935747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d6d681f4-c93a-4fca-9d0e-670b2a13bd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:23.948530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25835a68-de3e-400f-bfa2-622ea523de26 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:23.970026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:24.052923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1f1c4-290f-4348-a189-6f388a1aa8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:24.053558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1f1c4-290f-4348-a189-6f388a1aa8af admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:24.055195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1f1c4-290f-4348-a189-6f388a1aa8af admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:24.055501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1f1c4-290f-4348-a189-6f388a1aa8af admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:24.056148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1f1c4-290f-4348-a189-6f388a1aa8af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:24.058589 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3151/6369] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:23 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:24.059920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1f1c4-290f-4348-a189-6f388a1aa8af admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:24.066196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d681f4-c93a-4fca-9d0e-670b2a13bd3f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:24.068410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d6d681f4-c93a-4fca-9d0e-670b2a13bd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:24.091667 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3219/6370] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:23 2022] POST /identity/v3/users => generated 292 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:24.095855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1f1c4-290f-4348-a189-6f388a1aa8af admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:24.111580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1f1c4-290f-4348-a189-6f388a1aa8af admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:24.126819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d165a-be67-4b2c-b1a4-0307515a190a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:24.128349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d165a-be67-4b2c-b1a4-0307515a190a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:24.128793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d165a-be67-4b2c-b1a4-0307515a190a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:24.129077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d165a-be67-4b2c-b1a4-0307515a190a admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:24.129766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2d165a-be67-4b2c-b1a4-0307515a190a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:24.136056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2d165a-be67-4b2c-b1a4-0307515a190a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:24.151920 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3220/6371] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:24.156358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d165a-be67-4b2c-b1a4-0307515a190a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:24.164019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d681f4-c93a-4fca-9d0e-670b2a13bd3f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:24.168433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a1ff3-3c14-446a-9aa9-00720c4550c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:24.169134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d165a-be67-4b2c-b1a4-0307515a190a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:24.169276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a1ff3-3c14-446a-9aa9-00720c4550c2 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:24.169625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a1ff3-3c14-446a-9aa9-00720c4550c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:24.169969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a1ff3-3c14-446a-9aa9-00720c4550c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:24.175380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc754c6-95ec-4dda-a23b-46b1afa82975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:24.177360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc754c6-95ec-4dda-a23b-46b1afa82975 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:24.177819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc754c6-95ec-4dda-a23b-46b1afa82975 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:24.178251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc754c6-95ec-4dda-a23b-46b1afa82975 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:24.179409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc754c6-95ec-4dda-a23b-46b1afa82975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:24.181124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a1ff3-3c14-446a-9aa9-00720c4550c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:24.181381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc754c6-95ec-4dda-a23b-46b1afa82975 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:24.183373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a1ff3-3c14-446a-9aa9-00720c4550c2 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:24.198705 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3152/6372] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:13:24.204241 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3221/6373] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:13:24.207619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a1ff3-3c14-446a-9aa9-00720c4550c2 service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:24.234166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a1ff3-3c14-446a-9aa9-00720c4550c2 service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:24.240940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed0ad7-46ee-4c4d-b670-d87dcd6cd8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:24.241668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed0ad7-46ee-4c4d-b670-d87dcd6cd8a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:24.241951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed0ad7-46ee-4c4d-b670-d87dcd6cd8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:24.242225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed0ad7-46ee-4c4d-b670-d87dcd6cd8a5 admin admin] PATH_INFO: `/v3/projects/9d2f356873b84cca908a01cc30d7cac0/users/9399e9638e464708bc007502e2f571a2/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:24.268464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:24.273050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed0ad7-46ee-4c4d-b670-d87dcd6cd8a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d2f356873b84cca908a01cc30d7cac0, user_id=9399e9638e464708bc007502e2f571a2, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:24.277609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed0ad7-46ee-4c4d-b670-d87dcd6cd8a5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:24.300052 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3153/6374] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:24 2022] PUT /identity/v3/projects/9d2f356873b84cca908a01cc30d7cac0/users/9399e9638e464708bc007502e2f571a2/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:24.311329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af06ea-8da4-465b-8236-fe194f182caf None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:24.311637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af06ea-8da4-465b-8236-fe194f182caf None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:24.312008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af06ea-8da4-465b-8236-fe194f182caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:24.355845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-28af06ea-8da4-465b-8236-fe194f182caf None None] MFA Rules not processed for user `9399e9638e464708bc007502e2f571a2`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:24.358221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-28af06ea-8da4-465b-8236-fe194f182caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:24.478538 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3222/6375] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:24.522744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed0ad7-46ee-4c4d-b670-d87dcd6cd8a5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:24.599081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed0ad7-46ee-4c4d-b670-d87dcd6cd8a5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:24.601994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983c609-2398-4c73-a0ec-2c1ffd331b76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:24.602714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983c609-2398-4c73-a0ec-2c1ffd331b76 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:24.603064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983c609-2398-4c73-a0ec-2c1ffd331b76 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:24.603487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983c609-2398-4c73-a0ec-2c1ffd331b76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:24.613283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983c609-2398-4c73-a0ec-2c1ffd331b76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:24.614796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983c609-2398-4c73-a0ec-2c1ffd331b76 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:24.632383 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3154/6376] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:25.485178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af06ea-8da4-465b-8236-fe194f182caf None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:25.600001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af06ea-8da4-465b-8236-fe194f182caf None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:25.606624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7a1a9-4c01-4720-9b52-f750a3a14f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:25.607612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7a1a9-4c01-4720-9b52-f750a3a14f35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:25.608052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7a1a9-4c01-4720-9b52-f750a3a14f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:25.609252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7a1a9-4c01-4720-9b52-f750a3a14f35 admin admin] PATH_INFO: `/v3/users/7a31f27f3bdf4a8ab606779999ddd894` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:25.610471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7a1a9-4c01-4720-9b52-f750a3a14f35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a31f27f3bdf4a8ab606779999ddd894)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:25.612511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7a1a9-4c01-4720-9b52-f750a3a14f35 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:25.630284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:25.677052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c4d7a1a9-4c01-4720-9b52-f750a3a14f35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a31f27f3bdf4a8ab606779999ddd894'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:25.688804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c4d7a1a9-4c01-4720-9b52-f750a3a14f35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a31f27f3bdf4a8ab606779999ddd894'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:25.702159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c4d7a1a9-4c01-4720-9b52-f750a3a14f35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a31f27f3bdf4a8ab606779999ddd894'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:25.703001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c4d7a1a9-4c01-4720-9b52-f750a3a14f35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a31f27f3bdf4a8ab606779999ddd894'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:25.710689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c4d7a1a9-4c01-4720-9b52-f750a3a14f35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a31f27f3bdf4a8ab606779999ddd894'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:25.717674 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3223/6377] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:25 2022] DELETE /identity/v3/users/7a31f27f3bdf4a8ab606779999ddd894 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:25.722379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa347f-2722-4177-bd56-13cee8fad49b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:25.722918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa347f-2722-4177-bd56-13cee8fad49b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:25.723094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa347f-2722-4177-bd56-13cee8fad49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:25.750007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-16fa347f-2722-4177-bd56-13cee8fad49b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:25.751329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-16fa347f-2722-4177-bd56-13cee8fad49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:25.827345 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3155/6378] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:25.834592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7a1a9-4c01-4720-9b52-f750a3a14f35 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:25.836106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c4d7a1a9-4c01-4720-9b52-f750a3a14f35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:25.909818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7a1a9-4c01-4720-9b52-f750a3a14f35 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:25.914973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f3f66-bcbd-40e6-9243-6d1beb794b7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:25.915434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f3f66-bcbd-40e6-9243-6d1beb794b7c service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:25.915604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f3f66-bcbd-40e6-9243-6d1beb794b7c service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:25.917214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f3f66-bcbd-40e6-9243-6d1beb794b7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:25.933497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f3f66-bcbd-40e6-9243-6d1beb794b7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:25.935040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f3f66-bcbd-40e6-9243-6d1beb794b7c service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:25.947173 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3224/6379] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:26.395547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa347f-2722-4177-bd56-13cee8fad49b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:26.397253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-16fa347f-2722-4177-bd56-13cee8fad49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:26.413979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa347f-2722-4177-bd56-13cee8fad49b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:26.423166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d1ad2-1ff3-4915-ac0b-129c3325b2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:26.424043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d1ad2-1ff3-4915-ac0b-129c3325b2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:26.425324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d1ad2-1ff3-4915-ac0b-129c3325b2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:26.425324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d1ad2-1ff3-4915-ac0b-129c3325b2e8 admin admin] PATH_INFO: `/v3/projects/3fdf2a69b9294b5283858cc0a173676c` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:26.427009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d1ad2-1ff3-4915-ac0b-129c3325b2e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fdf2a69b9294b5283858cc0a173676c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:26.429792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d1ad2-1ff3-4915-ac0b-129c3325b2e8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:26.503973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4d0d1ad2-1ff3-4915-ac0b-129c3325b2e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fdf2a69b9294b5283858cc0a173676c', 'request_id': 'req-4d0d1ad2-1ff3-4915-ac0b-129c3325b2e8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:26.517749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4d0d1ad2-1ff3-4915-ac0b-129c3325b2e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fdf2a69b9294b5283858cc0a173676c', 'request_id': 'req-4d0d1ad2-1ff3-4915-ac0b-129c3325b2e8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:26.534336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4d0d1ad2-1ff3-4915-ac0b-129c3325b2e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fdf2a69b9294b5283858cc0a173676c', 'request_id': 'req-4d0d1ad2-1ff3-4915-ac0b-129c3325b2e8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:26.553541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4d0d1ad2-1ff3-4915-ac0b-129c3325b2e8 admin admin] The token cache is being invalidate because project 3fdf2a69b9294b5283858cc0a173676c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:26.553816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4d0d1ad2-1ff3-4915-ac0b-129c3325b2e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:26.554830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4d0d1ad2-1ff3-4915-ac0b-129c3325b2e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:26.560778 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3156/6380] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:26 2022] DELETE /identity/v3/projects/3fdf2a69b9294b5283858cc0a173676c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:26.564776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f3f66-bcbd-40e6-9243-6d1beb794b7c service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:26.577342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f3f66-bcbd-40e6-9243-6d1beb794b7c service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:26.658121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc989c-3966-402e-9b59-e02d037fca64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:26.658816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc989c-3966-402e-9b59-e02d037fca64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:26.659092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc989c-3966-402e-9b59-e02d037fca64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:26.659366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc989c-3966-402e-9b59-e02d037fca64 admin admin] PATH_INFO: `/v3/users/456ed7c7a5d6404eb4ffd35dd37ac981` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:26.660674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc989c-3966-402e-9b59-e02d037fca64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=456ed7c7a5d6404eb4ffd35dd37ac981)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:26.663133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc989c-3966-402e-9b59-e02d037fca64 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:26.710974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:26.719482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a1bc989c-3966-402e-9b59-e02d037fca64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '456ed7c7a5d6404eb4ffd35dd37ac981'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:26.727360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a1bc989c-3966-402e-9b59-e02d037fca64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '456ed7c7a5d6404eb4ffd35dd37ac981'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:26.741361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a1bc989c-3966-402e-9b59-e02d037fca64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '456ed7c7a5d6404eb4ffd35dd37ac981'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:26.742082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a1bc989c-3966-402e-9b59-e02d037fca64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '456ed7c7a5d6404eb4ffd35dd37ac981'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:26.749278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a1bc989c-3966-402e-9b59-e02d037fca64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '456ed7c7a5d6404eb4ffd35dd37ac981'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:26.755485 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3225/6381] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:26 2022] DELETE /identity/v3/users/456ed7c7a5d6404eb4ffd35dd37ac981 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:27.225023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d1ad2-1ff3-4915-ac0b-129c3325b2e8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:27.228150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4d0d1ad2-1ff3-4915-ac0b-129c3325b2e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:27.266015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d1ad2-1ff3-4915-ac0b-129c3325b2e8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:27.297665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:27.391965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788383d-f5e4-45bd-9db1-4cc4d3081aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:27.392653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788383d-f5e4-45bd-9db1-4cc4d3081aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:27.392933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788383d-f5e4-45bd-9db1-4cc4d3081aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:27.393275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788383d-f5e4-45bd-9db1-4cc4d3081aa2 admin admin] PATH_INFO: `/v3/projects/0b19ee6c3f2540e6a298648052eec5ca` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:27.394376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a788383d-f5e4-45bd-9db1-4cc4d3081aa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b19ee6c3f2540e6a298648052eec5ca)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:27.396861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a788383d-f5e4-45bd-9db1-4cc4d3081aa2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:27.469693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a788383d-f5e4-45bd-9db1-4cc4d3081aa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b19ee6c3f2540e6a298648052eec5ca', 'request_id': 'req-a788383d-f5e4-45bd-9db1-4cc4d3081aa2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:27.478479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a788383d-f5e4-45bd-9db1-4cc4d3081aa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b19ee6c3f2540e6a298648052eec5ca', 'request_id': 'req-a788383d-f5e4-45bd-9db1-4cc4d3081aa2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:27.486099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a788383d-f5e4-45bd-9db1-4cc4d3081aa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b19ee6c3f2540e6a298648052eec5ca', 'request_id': 'req-a788383d-f5e4-45bd-9db1-4cc4d3081aa2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:27.499966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a788383d-f5e4-45bd-9db1-4cc4d3081aa2 admin admin] The token cache is being invalidate because project 0b19ee6c3f2540e6a298648052eec5ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:27.500248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a788383d-f5e4-45bd-9db1-4cc4d3081aa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:27.501088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a788383d-f5e4-45bd-9db1-4cc4d3081aa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:27.502770 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3157/6382] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:27 2022] DELETE /identity/v3/projects/0b19ee6c3f2540e6a298648052eec5ca => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:27.512520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc278e1-6df5-40ef-b0df-208c84396496 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:27.512822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc278e1-6df5-40ef-b0df-208c84396496 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:27.513113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc278e1-6df5-40ef-b0df-208c84396496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:27.542026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-bfc278e1-6df5-40ef-b0df-208c84396496 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:27.543336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bfc278e1-6df5-40ef-b0df-208c84396496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:27.628997 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3226/6383] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:27.637970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db85e6c-cfaa-4307-8c1f-8de71ba57961 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:27.638379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db85e6c-cfaa-4307-8c1f-8de71ba57961 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:27.638771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db85e6c-cfaa-4307-8c1f-8de71ba57961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:27.676840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0db85e6c-cfaa-4307-8c1f-8de71ba57961 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:27.678574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0db85e6c-cfaa-4307-8c1f-8de71ba57961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:27.695896 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3158/6384] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:27.701811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc278e1-6df5-40ef-b0df-208c84396496 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:27.719535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc278e1-6df5-40ef-b0df-208c84396496 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:27.735270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c20aff-500f-4d4f-ab38-2c42556f338a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:27.735902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c20aff-500f-4d4f-ab38-2c42556f338a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:27.736907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c20aff-500f-4d4f-ab38-2c42556f338a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:27.737280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c20aff-500f-4d4f-ab38-2c42556f338a admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:27.737941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c20aff-500f-4d4f-ab38-2c42556f338a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:13:27.739475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c20aff-500f-4d4f-ab38-2c42556f338a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:27.742194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c20aff-500f-4d4f-ab38-2c42556f338a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:27.757922 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3227/6385] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:13:27.762832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db85e6c-cfaa-4307-8c1f-8de71ba57961 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:27.777652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db85e6c-cfaa-4307-8c1f-8de71ba57961 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:27.786021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0b680-0330-4656-917e-f6de4ffc75bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:27.786959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0b680-0330-4656-917e-f6de4ffc75bb admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:27.787316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0b680-0330-4656-917e-f6de4ffc75bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:27.787609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0b680-0330-4656-917e-f6de4ffc75bb admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:27.789094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0b680-0330-4656-917e-f6de4ffc75bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:27.791871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0b680-0330-4656-917e-f6de4ffc75bb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:27.838718 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3159/6386] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:27 2022] POST /identity/v3/projects => generated 375 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:27.842629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c20aff-500f-4d4f-ab38-2c42556f338a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:27.853926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c20aff-500f-4d4f-ab38-2c42556f338a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:27.952237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1391e-ed20-4228-b151-4d32b1041724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:27.953030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1391e-ed20-4228-b151-4d32b1041724 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:27.953319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1391e-ed20-4228-b151-4d32b1041724 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:27.953602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1391e-ed20-4228-b151-4d32b1041724 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:27.954448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c1391e-ed20-4228-b151-4d32b1041724 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:27.956733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c1391e-ed20-4228-b151-4d32b1041724 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:27.996585 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3228/6387] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:27 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:13:28.002461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0b680-0330-4656-917e-f6de4ffc75bb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:28.015875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0b680-0330-4656-917e-f6de4ffc75bb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:28.023482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1addf3-f861-4a70-8b5e-9376aa192b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:28.025163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1addf3-f861-4a70-8b5e-9376aa192b6f admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:28.025483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1addf3-f861-4a70-8b5e-9376aa192b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:28.030743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1addf3-f861-4a70-8b5e-9376aa192b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:28.030743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1addf3-f861-4a70-8b5e-9376aa192b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:28.030743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1addf3-f861-4a70-8b5e-9376aa192b6f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:28.043488 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3160/6388] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:28.045176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1391e-ed20-4228-b151-4d32b1041724 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:28.057463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1391e-ed20-4228-b151-4d32b1041724 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:28.062519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc30dff-3e5c-48b9-accb-a8cd35ff8119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:28.063232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc30dff-3e5c-48b9-accb-a8cd35ff8119 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:28.063516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc30dff-3e5c-48b9-accb-a8cd35ff8119 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:28.063788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc30dff-3e5c-48b9-accb-a8cd35ff8119 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:28.064427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc30dff-3e5c-48b9-accb-a8cd35ff8119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:28.066172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc30dff-3e5c-48b9-accb-a8cd35ff8119 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:28.077535 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3229/6389] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:28.081994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1addf3-f861-4a70-8b5e-9376aa192b6f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:28.093817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1addf3-f861-4a70-8b5e-9376aa192b6f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:28.105491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd2b98-87af-4264-9fca-4bbdf50f48fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:28.105910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd2b98-87af-4264-9fca-4bbdf50f48fa admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:28.106316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd2b98-87af-4264-9fca-4bbdf50f48fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:28.106698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd2b98-87af-4264-9fca-4bbdf50f48fa admin admin] PATH_INFO: `/v3/projects/c6b627e68c7b42c18cb91b2e8308b3a6/users/f542f3958542490a811c4dd8fe8fe04d/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:28.135137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd2b98-87af-4264-9fca-4bbdf50f48fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6b627e68c7b42c18cb91b2e8308b3a6, user_id=f542f3958542490a811c4dd8fe8fe04d, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:28.137765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd2b98-87af-4264-9fca-4bbdf50f48fa admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:28.160234 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3161/6390] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:28 2022] PUT /identity/v3/projects/c6b627e68c7b42c18cb91b2e8308b3a6/users/f542f3958542490a811c4dd8fe8fe04d/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:13:28.172187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3b36d-ef47-4e59-813b-cf71fed446ee None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:28.172458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3b36d-ef47-4e59-813b-cf71fed446ee None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:28.172656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3b36d-ef47-4e59-813b-cf71fed446ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:28.210714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-78f3b36d-ef47-4e59-813b-cf71fed446ee None None] MFA Rules not processed for user `f542f3958542490a811c4dd8fe8fe04d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:28.211961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-78f3b36d-ef47-4e59-813b-cf71fed446ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:28.222684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:28.300697 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3230/6391] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:28.311785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd2b98-87af-4264-9fca-4bbdf50f48fa admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:28.314183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4fdd2b98-87af-4264-9fca-4bbdf50f48fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:28.373221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:28.391218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd2b98-87af-4264-9fca-4bbdf50f48fa admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:28.397019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4dfdb8-f24d-4ef5-8299-9fa8106065ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:28.398199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4dfdb8-f24d-4ef5-8299-9fa8106065ba service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:28.398706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4dfdb8-f24d-4ef5-8299-9fa8106065ba service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:28.399140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4dfdb8-f24d-4ef5-8299-9fa8106065ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:28.415099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4dfdb8-f24d-4ef5-8299-9fa8106065ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:28.416926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4dfdb8-f24d-4ef5-8299-9fa8106065ba service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:28.429684 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3162/6392] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:28.489279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3b36d-ef47-4e59-813b-cf71fed446ee None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:28.491074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-78f3b36d-ef47-4e59-813b-cf71fed446ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:28.591051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3b36d-ef47-4e59-813b-cf71fed446ee None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:28.596709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590072c-3cc8-4ecf-a749-54f943dd5860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:28.597430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590072c-3cc8-4ecf-a749-54f943dd5860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:28.597736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590072c-3cc8-4ecf-a749-54f943dd5860 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:28.598592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590072c-3cc8-4ecf-a749-54f943dd5860 admin admin] PATH_INFO: `/v3/users/20dc4cd847944bf09273b91c1cf6952e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:28.599656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590072c-3cc8-4ecf-a749-54f943dd5860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20dc4cd847944bf09273b91c1cf6952e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:28.602639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590072c-3cc8-4ecf-a749-54f943dd5860 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:28.662239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a590072c-3cc8-4ecf-a749-54f943dd5860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20dc4cd847944bf09273b91c1cf6952e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:28.678033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a590072c-3cc8-4ecf-a749-54f943dd5860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20dc4cd847944bf09273b91c1cf6952e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:28.700621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a590072c-3cc8-4ecf-a749-54f943dd5860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20dc4cd847944bf09273b91c1cf6952e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:28.701632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a590072c-3cc8-4ecf-a749-54f943dd5860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20dc4cd847944bf09273b91c1cf6952e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:28.718121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a590072c-3cc8-4ecf-a749-54f943dd5860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20dc4cd847944bf09273b91c1cf6952e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:28.730259 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3231/6393] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:28 2022] DELETE /identity/v3/users/20dc4cd847944bf09273b91c1cf6952e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:28.734994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15aaa3b-6187-4ee0-b881-d33c675bf876 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:28.735310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15aaa3b-6187-4ee0-b881-d33c675bf876 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:28.735695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15aaa3b-6187-4ee0-b881-d33c675bf876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:28.763602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c15aaa3b-6187-4ee0-b881-d33c675bf876 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:28.764831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c15aaa3b-6187-4ee0-b881-d33c675bf876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:28.851772 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3163/6394] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:28.864799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590072c-3cc8-4ecf-a749-54f943dd5860 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:28.867241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a590072c-3cc8-4ecf-a749-54f943dd5860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:28.917048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:28.960973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590072c-3cc8-4ecf-a749-54f943dd5860 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:28.965341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd9c1c-d749-43a9-ae4c-af328e5a92c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:28.965954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd9c1c-d749-43a9-ae4c-af328e5a92c0 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:28.966238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd9c1c-d749-43a9-ae4c-af328e5a92c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:28.966536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd9c1c-d749-43a9-ae4c-af328e5a92c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:28.979454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd9c1c-d749-43a9-ae4c-af328e5a92c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:28.981177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd9c1c-d749-43a9-ae4c-af328e5a92c0 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:28.993809 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3232/6395] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:29.052991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463189e-5ff4-4fa2-a6ea-7f304ff6af62 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:29.053584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463189e-5ff4-4fa2-a6ea-7f304ff6af62 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:29.053857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463189e-5ff4-4fa2-a6ea-7f304ff6af62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:29.087760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c463189e-5ff4-4fa2-a6ea-7f304ff6af62 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:29.090544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c463189e-5ff4-4fa2-a6ea-7f304ff6af62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:29.170955 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3164/6396] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:13:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:29.185782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd9c1c-d749-43a9-ae4c-af328e5a92c0 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:29.189168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-48cd9c1c-d749-43a9-ae4c-af328e5a92c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:29.366875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd9c1c-d749-43a9-ae4c-af328e5a92c0 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:29.390075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fbf78-5eb2-4810-bfb4-e0e7498bdf11 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:29.391055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fbf78-5eb2-4810-bfb4-e0e7498bdf11 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:29.391055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fbf78-5eb2-4810-bfb4-e0e7498bdf11 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:29.391308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fbf78-5eb2-4810-bfb4-e0e7498bdf11 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:29.401231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463189e-5ff4-4fa2-a6ea-7f304ff6af62 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:29.402843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c463189e-5ff4-4fa2-a6ea-7f304ff6af62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:29.412118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fbf78-5eb2-4810-bfb4-e0e7498bdf11 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:29.416324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fbf78-5eb2-4810-bfb4-e0e7498bdf11 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:29.421945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463189e-5ff4-4fa2-a6ea-7f304ff6af62 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:29.428903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47693860-571a-40d9-b176-ddc30d155e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:29.429926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47693860-571a-40d9-b176-ddc30d155e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:29.431355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47693860-571a-40d9-b176-ddc30d155e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:29.432694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47693860-571a-40d9-b176-ddc30d155e1d admin admin] PATH_INFO: `/v3/projects/588321452fd442ee8b78e7d7e4fa4638` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:29.435264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47693860-571a-40d9-b176-ddc30d155e1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=588321452fd442ee8b78e7d7e4fa4638)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:29.439797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47693860-571a-40d9-b176-ddc30d155e1d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:29.443492 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3233/6397] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:13:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:13:29.537691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-47693860-571a-40d9-b176-ddc30d155e1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '588321452fd442ee8b78e7d7e4fa4638', 'request_id': 'req-47693860-571a-40d9-b176-ddc30d155e1d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:29.557937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-47693860-571a-40d9-b176-ddc30d155e1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '588321452fd442ee8b78e7d7e4fa4638', 'request_id': 'req-47693860-571a-40d9-b176-ddc30d155e1d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:29.570006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-47693860-571a-40d9-b176-ddc30d155e1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '588321452fd442ee8b78e7d7e4fa4638', 'request_id': 'req-47693860-571a-40d9-b176-ddc30d155e1d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:29.592031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-47693860-571a-40d9-b176-ddc30d155e1d admin admin] The token cache is being invalidate because project 588321452fd442ee8b78e7d7e4fa4638 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:29.592362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-47693860-571a-40d9-b176-ddc30d155e1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:29.598765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-47693860-571a-40d9-b176-ddc30d155e1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:29.605615 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3165/6398] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:29 2022] DELETE /identity/v3/projects/588321452fd442ee8b78e7d7e4fa4638 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:13:29.610011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3cad8-89a4-44cc-9b2a-678ea3945035 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:29.612958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3cad8-89a4-44cc-9b2a-678ea3945035 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:29.613827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3cad8-89a4-44cc-9b2a-678ea3945035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:29.694405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fcf3cad8-89a4-44cc-9b2a-678ea3945035 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:29.696412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fcf3cad8-89a4-44cc-9b2a-678ea3945035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:29.880548 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3234/6399] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:29.882316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a617910-3def-418d-a21e-d93229c65d7c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:29.882629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a617910-3def-418d-a21e-d93229c65d7c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:29.882901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a617910-3def-418d-a21e-d93229c65d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:29.927364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1a617910-3def-418d-a21e-d93229c65d7c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:29.928624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1a617910-3def-418d-a21e-d93229c65d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:29.952982 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3166/6400] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:29.957023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3cad8-89a4-44cc-9b2a-678ea3945035 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:29.972694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3cad8-89a4-44cc-9b2a-678ea3945035 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:29.979456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4c6e3-39ab-41de-b239-1594ede49468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:29.980146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4c6e3-39ab-41de-b239-1594ede49468 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:29.980487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4c6e3-39ab-41de-b239-1594ede49468 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:29.981013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4c6e3-39ab-41de-b239-1594ede49468 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:29.981734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4c6e3-39ab-41de-b239-1594ede49468 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:13:29.982040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4c6e3-39ab-41de-b239-1594ede49468 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:29.984407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4c6e3-39ab-41de-b239-1594ede49468 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:30.014154 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3235/6401] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:13:30.018632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a617910-3def-418d-a21e-d93229c65d7c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:30.037986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a617910-3def-418d-a21e-d93229c65d7c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:30.060966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e62b3-bcb8-491a-ad09-51a8a3ce1976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:30.062005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e62b3-bcb8-491a-ad09-51a8a3ce1976 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:30.065367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e62b3-bcb8-491a-ad09-51a8a3ce1976 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:30.068680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e62b3-bcb8-491a-ad09-51a8a3ce1976 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:30.069441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e62b3-bcb8-491a-ad09-51a8a3ce1976 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:30.075669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e62b3-bcb8-491a-ad09-51a8a3ce1976 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:30.103984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:30.170680 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3167/6402] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:30 2022] POST /identity/v3/projects => generated 401 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:13:30.177199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4c6e3-39ab-41de-b239-1594ede49468 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:30.210250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4c6e3-39ab-41de-b239-1594ede49468 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:30.346814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:30.374986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779a961-979a-4f40-a3af-f3dbb093317b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:30.375669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779a961-979a-4f40-a3af-f3dbb093317b admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:30.375957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779a961-979a-4f40-a3af-f3dbb093317b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:30.376283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779a961-979a-4f40-a3af-f3dbb093317b admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:30.382968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779a961-979a-4f40-a3af-f3dbb093317b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:30.385640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779a961-979a-4f40-a3af-f3dbb093317b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:30.450511 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3236/6403] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:30 2022] POST /identity/v3/users => generated 301 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:30.452613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e62b3-bcb8-491a-ad09-51a8a3ce1976 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:30.484124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e62b3-bcb8-491a-ad09-51a8a3ce1976 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:30.524849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b0afa-1a79-407e-aea6-f9cc233d8adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:30.526054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b0afa-1a79-407e-aea6-f9cc233d8adc admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:30.528986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b0afa-1a79-407e-aea6-f9cc233d8adc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:30.529295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b0afa-1a79-407e-aea6-f9cc233d8adc admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:30.529982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b0afa-1a79-407e-aea6-f9cc233d8adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:30.539947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b0afa-1a79-407e-aea6-f9cc233d8adc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:30.590923 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3168/6404] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:13:30.592652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779a961-979a-4f40-a3af-f3dbb093317b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:30.647333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779a961-979a-4f40-a3af-f3dbb093317b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:30.671637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97198fe7-d106-4cd7-981d-0971f245b838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:30.673759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97198fe7-d106-4cd7-981d-0971f245b838 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:30.675212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97198fe7-d106-4cd7-981d-0971f245b838 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:30.675525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97198fe7-d106-4cd7-981d-0971f245b838 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:30.678847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97198fe7-d106-4cd7-981d-0971f245b838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:30.686372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97198fe7-d106-4cd7-981d-0971f245b838 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:30.733815 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3237/6405] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:30.736814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b0afa-1a79-407e-aea6-f9cc233d8adc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:30.758722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b0afa-1a79-407e-aea6-f9cc233d8adc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:30.769760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5f229-46c2-4ab5-8b86-585c04d4bd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:30.770742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5f229-46c2-4ab5-8b86-585c04d4bd78 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:30.771266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5f229-46c2-4ab5-8b86-585c04d4bd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:30.771759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5f229-46c2-4ab5-8b86-585c04d4bd78 admin admin] PATH_INFO: `/v3/projects/8d3b3bbc444c40cd87c001df49cde2f3/users/de7b0236737e4af29a591aa56c4de04d/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:30.821752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97198fe7-d106-4cd7-981d-0971f245b838 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:30.823931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-97198fe7-d106-4cd7-981d-0971f245b838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:30.836098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb5f229-46c2-4ab5-8b86-585c04d4bd78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d3b3bbc444c40cd87c001df49cde2f3, user_id=de7b0236737e4af29a591aa56c4de04d, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:30.848327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb5f229-46c2-4ab5-8b86-585c04d4bd78 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:30.858871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97198fe7-d106-4cd7-981d-0971f245b838 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:30.878080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781869ee-9a3e-4bb3-9615-80c9a1879c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:30.879155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781869ee-9a3e-4bb3-9615-80c9a1879c9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:30.880366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781869ee-9a3e-4bb3-9615-80c9a1879c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:30.880980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781869ee-9a3e-4bb3-9615-80c9a1879c9a admin admin] PATH_INFO: `/v3/users/9399e9638e464708bc007502e2f571a2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:30.886934 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3169/6406] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:30 2022] PUT /identity/v3/projects/8d3b3bbc444c40cd87c001df49cde2f3/users/de7b0236737e4af29a591aa56c4de04d/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:30.888112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781869ee-9a3e-4bb3-9615-80c9a1879c9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9399e9638e464708bc007502e2f571a2)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:30.890574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781869ee-9a3e-4bb3-9615-80c9a1879c9a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:30.897788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b3afa-6ce9-4183-be37-8595a8f14758 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:30.898071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b3afa-6ce9-4183-be37-8595a8f14758 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:30.898449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b3afa-6ce9-4183-be37-8595a8f14758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:30.981682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-975b3afa-6ce9-4183-be37-8595a8f14758 None None] MFA Rules not processed for user `de7b0236737e4af29a591aa56c4de04d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:30.983754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-975b3afa-6ce9-4183-be37-8595a8f14758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:30.999797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-781869ee-9a3e-4bb3-9615-80c9a1879c9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9399e9638e464708bc007502e2f571a2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:31.010569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-781869ee-9a3e-4bb3-9615-80c9a1879c9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9399e9638e464708bc007502e2f571a2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:31.049083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-781869ee-9a3e-4bb3-9615-80c9a1879c9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9399e9638e464708bc007502e2f571a2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:31.050772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-781869ee-9a3e-4bb3-9615-80c9a1879c9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9399e9638e464708bc007502e2f571a2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:31.071524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-781869ee-9a3e-4bb3-9615-80c9a1879c9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9399e9638e464708bc007502e2f571a2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:31.094662 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3238/6407] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:30 2022] DELETE /identity/v3/users/9399e9638e464708bc007502e2f571a2 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:31.096888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94886d70-3a64-4515-9845-bf7def315dba None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:31.097173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94886d70-3a64-4515-9845-bf7def315dba None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:31.097442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94886d70-3a64-4515-9845-bf7def315dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:31.158950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-94886d70-3a64-4515-9845-bf7def315dba None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:31.160204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-94886d70-3a64-4515-9845-bf7def315dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:31.173427 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3170/6408] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:30 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:31.178325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b3afa-6ce9-4183-be37-8595a8f14758 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:31.179791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-975b3afa-6ce9-4183-be37-8595a8f14758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:31.358863 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3239/6409] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:31.371947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94886d70-3a64-4515-9845-bf7def315dba None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:31.377358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-94886d70-3a64-4515-9845-bf7def315dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:31.397982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b3afa-6ce9-4183-be37-8595a8f14758 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:31.420822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9195e94-dc15-435e-a4e8-6a56fc8e8139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:31.421049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9195e94-dc15-435e-a4e8-6a56fc8e8139 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:31.421339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9195e94-dc15-435e-a4e8-6a56fc8e8139 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:31.421621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9195e94-dc15-435e-a4e8-6a56fc8e8139 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:31.424744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9195e94-dc15-435e-a4e8-6a56fc8e8139 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:31.441102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9195e94-dc15-435e-a4e8-6a56fc8e8139 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:31.503099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:31.518461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94886d70-3a64-4515-9845-bf7def315dba None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:31.522759 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3171/6410] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:31 2022] POST /identity/v3/projects => generated 403 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:31.524832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9195e94-dc15-435e-a4e8-6a56fc8e8139 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:31.619981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:31.703908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f92fa8-d73d-4cbe-878a-afbf009a8fb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:31.704550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f92fa8-d73d-4cbe-878a-afbf009a8fb2 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:31.704833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f92fa8-d73d-4cbe-878a-afbf009a8fb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:31.705109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f92fa8-d73d-4cbe-878a-afbf009a8fb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:31.721378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9195e94-dc15-435e-a4e8-6a56fc8e8139 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:31.723175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f92fa8-d73d-4cbe-878a-afbf009a8fb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:31.725043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f92fa8-d73d-4cbe-878a-afbf009a8fb2 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:31.741828 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3240/6411] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:31.745317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38392056-ccda-4877-a9f6-e0112ace1fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:31.747272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38392056-ccda-4877-a9f6-e0112ace1fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:31.747807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38392056-ccda-4877-a9f6-e0112ace1fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:31.750391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38392056-ccda-4877-a9f6-e0112ace1fd3 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:31.751332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38392056-ccda-4877-a9f6-e0112ace1fd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:31.753480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38392056-ccda-4877-a9f6-e0112ace1fd3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:31.805325 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3172/6412] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:31 2022] POST /identity/v3/users => generated 308 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:13:31.805495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f92fa8-d73d-4cbe-878a-afbf009a8fb2 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:31.823645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f92fa8-d73d-4cbe-878a-afbf009a8fb2 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:31.838852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9301c2-4e0e-41e1-9f03-e3c64c32c540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:31.843231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9301c2-4e0e-41e1-9f03-e3c64c32c540 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:31.843731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9301c2-4e0e-41e1-9f03-e3c64c32c540 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:31.844180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9301c2-4e0e-41e1-9f03-e3c64c32c540 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:31.845012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9301c2-4e0e-41e1-9f03-e3c64c32c540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:31.852212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9301c2-4e0e-41e1-9f03-e3c64c32c540 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:31.876033 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3241/6413] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:31.881118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38392056-ccda-4877-a9f6-e0112ace1fd3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:31.898830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38392056-ccda-4877-a9f6-e0112ace1fd3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:31.915052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f3f84-f0f6-448a-b7ef-daa2a23a5668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:31.919229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f3f84-f0f6-448a-b7ef-daa2a23a5668 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:31.920383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f3f84-f0f6-448a-b7ef-daa2a23a5668 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:31.921740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f3f84-f0f6-448a-b7ef-daa2a23a5668 admin admin] PATH_INFO: `/v3/projects/b037c9cdf4ec4cf9a595bd0e316b1248/users/bb521f26aa7f482e97f482a88d5c65ef/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:31.971413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f3f84-f0f6-448a-b7ef-daa2a23a5668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b037c9cdf4ec4cf9a595bd0e316b1248, user_id=bb521f26aa7f482e97f482a88d5c65ef, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:31.984784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f3f84-f0f6-448a-b7ef-daa2a23a5668 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:32.019232 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3173/6414] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:31 2022] PUT /identity/v3/projects/b037c9cdf4ec4cf9a595bd0e316b1248/users/bb521f26aa7f482e97f482a88d5c65ef/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:32.023773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9301c2-4e0e-41e1-9f03-e3c64c32c540 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:32.042263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9301c2-4e0e-41e1-9f03-e3c64c32c540 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:32.337471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329cd65-db8a-47c8-812c-2fbe8a58090f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:32.338104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329cd65-db8a-47c8-812c-2fbe8a58090f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:32.338435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329cd65-db8a-47c8-812c-2fbe8a58090f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:32.338815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329cd65-db8a-47c8-812c-2fbe8a58090f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:32.340762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8329cd65-db8a-47c8-812c-2fbe8a58090f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:32.342411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8329cd65-db8a-47c8-812c-2fbe8a58090f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:32.387041 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3242/6415] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:32.391953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f3f84-f0f6-448a-b7ef-daa2a23a5668 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:32.410685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f3f84-f0f6-448a-b7ef-daa2a23a5668 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:32.422538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9774a7-4df9-4eea-8dfb-90f150bb8217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:32.423302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9774a7-4df9-4eea-8dfb-90f150bb8217 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:32.423610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9774a7-4df9-4eea-8dfb-90f150bb8217 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:32.423872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9774a7-4df9-4eea-8dfb-90f150bb8217 admin admin] PATH_INFO: `/v3/projects/b037c9cdf4ec4cf9a595bd0e316b1248/users/bb521f26aa7f482e97f482a88d5c65ef/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:32.431257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9774a7-4df9-4eea-8dfb-90f150bb8217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b037c9cdf4ec4cf9a595bd0e316b1248, user_id=bb521f26aa7f482e97f482a88d5c65ef, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:32.436399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9774a7-4df9-4eea-8dfb-90f150bb8217 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:32.463572 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3174/6416] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:32 2022] PUT /identity/v3/projects/b037c9cdf4ec4cf9a595bd0e316b1248/users/bb521f26aa7f482e97f482a88d5c65ef/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:32.477749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fbe0b-4f5b-4492-8f3b-365664b71a7f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:32.478038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fbe0b-4f5b-4492-8f3b-365664b71a7f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:32.478351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fbe0b-4f5b-4492-8f3b-365664b71a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:32.525104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fd2fbe0b-4f5b-4492-8f3b-365664b71a7f None None] MFA Rules not processed for user `bb521f26aa7f482e97f482a88d5c65ef`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:32.527009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fd2fbe0b-4f5b-4492-8f3b-365664b71a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:32.567258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9774a7-4df9-4eea-8dfb-90f150bb8217 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:32.570900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3e9774a7-4df9-4eea-8dfb-90f150bb8217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:32.690364 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3243/6417] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:32 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:32.699698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2fbe0b-4f5b-4492-8f3b-365664b71a7f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:32.718879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2fbe0b-4f5b-4492-8f3b-365664b71a7f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:32.738864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c916e83-2eac-4613-9a82-d2a6e23545cb tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb521f26aa7f482e97f482a88d5c65ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b037c9cdf4ec4cf9a595bd0e316b1248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:32.739616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c916e83-2eac-4613-9a82-d2a6e23545cb tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:32.739941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c916e83-2eac-4613-9a82-d2a6e23545cb tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:32.740078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c916e83-2eac-4613-9a82-d2a6e23545cb tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:32.740741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c916e83-2eac-4613-9a82-d2a6e23545cb tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:32.743032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c916e83-2eac-4613-9a82-d2a6e23545cb tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:32.762866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:32.777744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9774a7-4df9-4eea-8dfb-90f150bb8217 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:32.778305 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3244/6418] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:13:32 2022] POST /identity/v3/roles => generated 240 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:32.779482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c916e83-2eac-4613-9a82-d2a6e23545cb tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:32.781376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5c916e83-2eac-4613-9a82-d2a6e23545cb tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:32.791732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56035106-ca09-432c-b0ed-0d47b8a67a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:32.792385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56035106-ca09-432c-b0ed-0d47b8a67a59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:32.792661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56035106-ca09-432c-b0ed-0d47b8a67a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:32.797576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56035106-ca09-432c-b0ed-0d47b8a67a59 admin admin] PATH_INFO: `/v3/users/f542f3958542490a811c4dd8fe8fe04d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:32.801704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c916e83-2eac-4613-9a82-d2a6e23545cb tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:32.803234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56035106-ca09-432c-b0ed-0d47b8a67a59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f542f3958542490a811c4dd8fe8fe04d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:32.808618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56035106-ca09-432c-b0ed-0d47b8a67a59 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:32.817890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8a194-7e05-43db-adb4-353d56a9238b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:32.820344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8a194-7e05-43db-adb4-353d56a9238b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:32.820702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8a194-7e05-43db-adb4-353d56a9238b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:32.820996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8a194-7e05-43db-adb4-353d56a9238b admin admin] PATH_INFO: `/v3/projects/9d2f356873b84cca908a01cc30d7cac0` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:32.825571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8a194-7e05-43db-adb4-353d56a9238b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d2f356873b84cca908a01cc30d7cac0)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:32.832978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8a194-7e05-43db-adb4-353d56a9238b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:32.931299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0a8a194-7e05-43db-adb4-353d56a9238b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d2f356873b84cca908a01cc30d7cac0', 'request_id': 'req-a0a8a194-7e05-43db-adb4-353d56a9238b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:32.933528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-56035106-ca09-432c-b0ed-0d47b8a67a59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f542f3958542490a811c4dd8fe8fe04d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:32.934195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-56035106-ca09-432c-b0ed-0d47b8a67a59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f542f3958542490a811c4dd8fe8fe04d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:32.934833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-56035106-ca09-432c-b0ed-0d47b8a67a59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f542f3958542490a811c4dd8fe8fe04d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:32.943458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-56035106-ca09-432c-b0ed-0d47b8a67a59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f542f3958542490a811c4dd8fe8fe04d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:32.951031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0a8a194-7e05-43db-adb4-353d56a9238b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d2f356873b84cca908a01cc30d7cac0', 'request_id': 'req-a0a8a194-7e05-43db-adb4-353d56a9238b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:32.961466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-56035106-ca09-432c-b0ed-0d47b8a67a59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f542f3958542490a811c4dd8fe8fe04d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:32.965735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0a8a194-7e05-43db-adb4-353d56a9238b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d2f356873b84cca908a01cc30d7cac0', 'request_id': 'req-a0a8a194-7e05-43db-adb4-353d56a9238b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:32.973026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:32.981980 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3175/6419] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:32 2022] DELETE /identity/v3/users/f542f3958542490a811c4dd8fe8fe04d => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:32.983230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56035106-ca09-432c-b0ed-0d47b8a67a59 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:32.985825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-56035106-ca09-432c-b0ed-0d47b8a67a59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:32.992154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0a8a194-7e05-43db-adb4-353d56a9238b admin admin] The token cache is being invalidate because project 9d2f356873b84cca908a01cc30d7cac0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:32.992542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0a8a194-7e05-43db-adb4-353d56a9238b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:33.004803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0a8a194-7e05-43db-adb4-353d56a9238b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:33.009811 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3245/6420] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:32 2022] DELETE /identity/v3/projects/9d2f356873b84cca908a01cc30d7cac0 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:33.012217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d2bcd-75b0-4572-88a9-2b9c0bba9a99 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:33.015801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d2bcd-75b0-4572-88a9-2b9c0bba9a99 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:33.017324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d2bcd-75b0-4572-88a9-2b9c0bba9a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:33.083054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-572d2bcd-75b0-4572-88a9-2b9c0bba9a99 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:33.096504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56035106-ca09-432c-b0ed-0d47b8a67a59 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:33.098604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-572d2bcd-75b0-4572-88a9-2b9c0bba9a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:33.105905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9154cee-3b5d-46db-a54b-f1a3cae190a3 tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb521f26aa7f482e97f482a88d5c65ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b037c9cdf4ec4cf9a595bd0e316b1248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:33.107051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9154cee-3b5d-46db-a54b-f1a3cae190a3 tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:33.107331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9154cee-3b5d-46db-a54b-f1a3cae190a3 tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:33.107667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9154cee-3b5d-46db-a54b-f1a3cae190a3 tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] PATH_INFO: `/v3/projects/8d3b3bbc444c40cd87c001df49cde2f3/users/de7b0236737e4af29a591aa56c4de04d/roles/36ca23c3a5ef465b8781c0fbd1e660cb` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:33.111777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9154cee-3b5d-46db-a54b-f1a3cae190a3 tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] RBAC: Authorizing `identity:create_grant(project_id=8d3b3bbc444c40cd87c001df49cde2f3, user_id=de7b0236737e4af29a591aa56c4de04d, role_id=36ca23c3a5ef465b8781c0fbd1e660cb)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:33.116081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9154cee-3b5d-46db-a54b-f1a3cae190a3 tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:33.152808 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3176/6421] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:32 2022] PUT /identity/v3/projects/8d3b3bbc444c40cd87c001df49cde2f3/users/de7b0236737e4af29a591aa56c4de04d/roles/36ca23c3a5ef465b8781c0fbd1e660cb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:33.157060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1a07a-4d61-4850-90b4-2f414335c5bf None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:33.157349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1a07a-4d61-4850-90b4-2f414335c5bf None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:33.157634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1a07a-4d61-4850-90b4-2f414335c5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:33.214229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fbc1a07a-4d61-4850-90b4-2f414335c5bf None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:33.216596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fbc1a07a-4d61-4850-90b4-2f414335c5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:33.263372 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3246/6422] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:33.264317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d2bcd-75b0-4572-88a9-2b9c0bba9a99 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:33.279261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-572d2bcd-75b0-4572-88a9-2b9c0bba9a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:33.347228 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3177/6423] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:33.347228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1a07a-4d61-4850-90b4-2f414335c5bf None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:33.347228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fbc1a07a-4d61-4850-90b4-2f414335c5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:33.419681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d2bcd-75b0-4572-88a9-2b9c0bba9a99 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:33.443654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1a07a-4d61-4850-90b4-2f414335c5bf None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:33.449776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045aa39-a72c-4466-8505-3b0e901f654d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:33.450727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045aa39-a72c-4466-8505-3b0e901f654d service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:33.451018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045aa39-a72c-4466-8505-3b0e901f654d service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:33.451427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045aa39-a72c-4466-8505-3b0e901f654d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:33.465716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045aa39-a72c-4466-8505-3b0e901f654d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:33.468419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045aa39-a72c-4466-8505-3b0e901f654d service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:33.484500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:33.488568 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3178/6424] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:33.489998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5996a44-0e5c-4d98-9e66-5201e4b6a93a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:33.490377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5996a44-0e5c-4d98-9e66-5201e4b6a93a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:33.490799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5996a44-0e5c-4d98-9e66-5201e4b6a93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:33.519095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f5996a44-0e5c-4d98-9e66-5201e4b6a93a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:33.522110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f5996a44-0e5c-4d98-9e66-5201e4b6a93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:33.537107 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3179/6425] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:33.542679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5996a44-0e5c-4d98-9e66-5201e4b6a93a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:33.571092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5996a44-0e5c-4d98-9e66-5201e4b6a93a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:33.579879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56905658-8a4c-4f8f-a9d8-648380507562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:33.580851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966eeb1-acc6-4180-9c77-00153521159a tempest-ApplicationCredentialsV3AdminTest-318842069 tempest-ApplicationCredentialsV3AdminTest-318842069-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de7b0236737e4af29a591aa56c4de04d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d3b3bbc444c40cd87c001df49cde2f3', 'project_domain_id': 'default', 'roles': ['member', 'tempest-test_role-1518444412', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:33.581454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56905658-8a4c-4f8f-a9d8-648380507562 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:33.581591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966eeb1-acc6-4180-9c77-00153521159a tempest-ApplicationCredentialsV3AdminTest-318842069 tempest-ApplicationCredentialsV3AdminTest-318842069-project] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:33.581837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966eeb1-acc6-4180-9c77-00153521159a tempest-ApplicationCredentialsV3AdminTest-318842069 tempest-ApplicationCredentialsV3AdminTest-318842069-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:33.582335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966eeb1-acc6-4180-9c77-00153521159a tempest-ApplicationCredentialsV3AdminTest-318842069 tempest-ApplicationCredentialsV3AdminTest-318842069-project] PATH_INFO: `/v3/users/de7b0236737e4af29a591aa56c4de04d/application_credentials` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:33.582736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56905658-8a4c-4f8f-a9d8-648380507562 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:33.582995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966eeb1-acc6-4180-9c77-00153521159a tempest-ApplicationCredentialsV3AdminTest-318842069 tempest-ApplicationCredentialsV3AdminTest-318842069-project] RBAC: Authorizing `identity:create_application_credential(user_id=de7b0236737e4af29a591aa56c4de04d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:33.583517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56905658-8a4c-4f8f-a9d8-648380507562 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:33.584518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966eeb1-acc6-4180-9c77-00153521159a tempest-ApplicationCredentialsV3AdminTest-318842069 tempest-ApplicationCredentialsV3AdminTest-318842069-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:33.584955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56905658-8a4c-4f8f-a9d8-648380507562 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:13:33.585267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56905658-8a4c-4f8f-a9d8-648380507562 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:33.588710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56905658-8a4c-4f8f-a9d8-648380507562 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:33.612960 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3180/6426] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:13:33.617524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56905658-8a4c-4f8f-a9d8-648380507562 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:33.637610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56905658-8a4c-4f8f-a9d8-648380507562 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:33.652156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105ef45-4816-474e-b244-55ad8f7eba59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:33.653274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105ef45-4816-474e-b244-55ad8f7eba59 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:33.653811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105ef45-4816-474e-b244-55ad8f7eba59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:33.654742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105ef45-4816-474e-b244-55ad8f7eba59 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:33.655722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105ef45-4816-474e-b244-55ad8f7eba59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:33.658355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105ef45-4816-474e-b244-55ad8f7eba59 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:33.701420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:33.704603 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3247/6427] 10.222.0.17 () {60 vars in 1315 bytes} [Sat Oct 1 21:13:33 2022] POST /identity/v3/users/de7b0236737e4af29a591aa56c4de04d/application_credentials => generated 659 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:33.710091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51816295-b921-46ca-a8a6-302e702917a2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:33.710476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51816295-b921-46ca-a8a6-302e702917a2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:33.711951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51816295-b921-46ca-a8a6-302e702917a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:33.733634 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3181/6428] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:33 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:33.735270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105ef45-4816-474e-b244-55ad8f7eba59 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:33.771749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-51816295-b921-46ca-a8a6-302e702917a2 None None] MFA Rules not processed for user `de7b0236737e4af29a591aa56c4de04d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:33.773269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-51816295-b921-46ca-a8a6-302e702917a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:33.932343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105ef45-4816-474e-b244-55ad8f7eba59 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:33.950373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49ecad-405f-4b2b-8417-44118305228d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:33.951546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49ecad-405f-4b2b-8417-44118305228d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:33.951991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49ecad-405f-4b2b-8417-44118305228d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:33.954525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49ecad-405f-4b2b-8417-44118305228d admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:33.959646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49ecad-405f-4b2b-8417-44118305228d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:33.965600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49ecad-405f-4b2b-8417-44118305228d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:34.030779 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3182/6429] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:33 2022] POST /identity/v3/users => generated 289 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:34.032048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49ecad-405f-4b2b-8417-44118305228d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:34.052653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49ecad-405f-4b2b-8417-44118305228d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:34.087356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87601b70-46e8-46d5-8f5d-e5b10fa67b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:34.088961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87601b70-46e8-46d5-8f5d-e5b10fa67b9e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:34.089425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87601b70-46e8-46d5-8f5d-e5b10fa67b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:34.089875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87601b70-46e8-46d5-8f5d-e5b10fa67b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:34.090984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87601b70-46e8-46d5-8f5d-e5b10fa67b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:34.092857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87601b70-46e8-46d5-8f5d-e5b10fa67b9e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:34.108401 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3248/6430] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:33 2022] POST /identity/v3/auth/tokens => generated 3394 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Oct 01 21:13:34.108655 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3183/6431] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:34 2022] GET /identity/v3/roles => generated 1651 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:34.109812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51816295-b921-46ca-a8a6-302e702917a2 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:34.113501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87601b70-46e8-46d5-8f5d-e5b10fa67b9e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:34.125442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:34.127412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87601b70-46e8-46d5-8f5d-e5b10fa67b9e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:34.140223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716698f-500d-4b07-9bfd-48dfb8b66653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:34.141307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716698f-500d-4b07-9bfd-48dfb8b66653 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:34.141970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716698f-500d-4b07-9bfd-48dfb8b66653 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:34.142690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716698f-500d-4b07-9bfd-48dfb8b66653 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:34.143889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716698f-500d-4b07-9bfd-48dfb8b66653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:34.145989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716698f-500d-4b07-9bfd-48dfb8b66653 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:34.162298 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3184/6432] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:34 2022] GET /identity/v3/roles => generated 1651 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:34.162298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716698f-500d-4b07-9bfd-48dfb8b66653 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:34.162298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a716698f-500d-4b07-9bfd-48dfb8b66653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:34.188059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716698f-500d-4b07-9bfd-48dfb8b66653 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:34.195577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81defc0-6876-4cc7-98b5-d6b0e781b9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:34.196377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81defc0-6876-4cc7-98b5-d6b0e781b9bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:34.196684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81defc0-6876-4cc7-98b5-d6b0e781b9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:34.197050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81defc0-6876-4cc7-98b5-d6b0e781b9bd admin admin] PATH_INFO: `/v3/projects/c6b627e68c7b42c18cb91b2e8308b3a6` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:34.199466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81defc0-6876-4cc7-98b5-d6b0e781b9bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6b627e68c7b42c18cb91b2e8308b3a6)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:34.202025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81defc0-6876-4cc7-98b5-d6b0e781b9bd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:34.268072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51816295-b921-46ca-a8a6-302e702917a2 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:34.286940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c81defc0-6876-4cc7-98b5-d6b0e781b9bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6b627e68c7b42c18cb91b2e8308b3a6', 'request_id': 'req-c81defc0-6876-4cc7-98b5-d6b0e781b9bd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:34.309230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c81defc0-6876-4cc7-98b5-d6b0e781b9bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6b627e68c7b42c18cb91b2e8308b3a6', 'request_id': 'req-c81defc0-6876-4cc7-98b5-d6b0e781b9bd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:34.319374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c81defc0-6876-4cc7-98b5-d6b0e781b9bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6b627e68c7b42c18cb91b2e8308b3a6', 'request_id': 'req-c81defc0-6876-4cc7-98b5-d6b0e781b9bd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:34.335359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c81defc0-6876-4cc7-98b5-d6b0e781b9bd admin admin] The token cache is being invalidate because project c6b627e68c7b42c18cb91b2e8308b3a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:34.335755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c81defc0-6876-4cc7-98b5-d6b0e781b9bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:34.336518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c81defc0-6876-4cc7-98b5-d6b0e781b9bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:34.339239 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3185/6433] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:34 2022] DELETE /identity/v3/projects/c6b627e68c7b42c18cb91b2e8308b3a6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:34.340189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81defc0-6876-4cc7-98b5-d6b0e781b9bd admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:34.342017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c81defc0-6876-4cc7-98b5-d6b0e781b9bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:34.353060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8aac7-d036-4448-929c-073ca5332b1c tempest-ApplicationCredentialsV3AdminTest-318842069 tempest-ApplicationCredentialsV3AdminTest-318842069-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de7b0236737e4af29a591aa56c4de04d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d3b3bbc444c40cd87c001df49cde2f3', 'project_domain_id': 'default', 'roles': ['member', 'tempest-test_role-1518444412', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:34.354265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8aac7-d036-4448-929c-073ca5332b1c tempest-ApplicationCredentialsV3AdminTest-318842069 tempest-ApplicationCredentialsV3AdminTest-318842069-project] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:34.354652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8aac7-d036-4448-929c-073ca5332b1c tempest-ApplicationCredentialsV3AdminTest-318842069 tempest-ApplicationCredentialsV3AdminTest-318842069-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:34.354941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8aac7-d036-4448-929c-073ca5332b1c tempest-ApplicationCredentialsV3AdminTest-318842069 tempest-ApplicationCredentialsV3AdminTest-318842069-project] PATH_INFO: `/v3/users/de7b0236737e4af29a591aa56c4de04d/application_credentials/4b90f687d69748a3bc546944dbd1b70d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:34.356198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8aac7-d036-4448-929c-073ca5332b1c tempest-ApplicationCredentialsV3AdminTest-318842069 tempest-ApplicationCredentialsV3AdminTest-318842069-project] RBAC: Authorizing `identity:delete_application_credential(user_id=de7b0236737e4af29a591aa56c4de04d, application_credential_id=4b90f687d69748a3bc546944dbd1b70d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:34.358334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8aac7-d036-4448-929c-073ca5332b1c tempest-ApplicationCredentialsV3AdminTest-318842069 tempest-ApplicationCredentialsV3AdminTest-318842069-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:34.395377 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3249/6434] 10.222.0.17 () {58 vars in 1395 bytes} [Sat Oct 1 21:13:34 2022] DELETE /identity/v3/users/de7b0236737e4af29a591aa56c4de04d/application_credentials/4b90f687d69748a3bc546944dbd1b70d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:34.397866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f46703-ce55-4cc3-b3b3-17bab3ecc2e7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:34.398977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f46703-ce55-4cc3-b3b3-17bab3ecc2e7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:34.399318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f46703-ce55-4cc3-b3b3-17bab3ecc2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:34.436540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-41f46703-ce55-4cc3-b3b3-17bab3ecc2e7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:34.438285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-41f46703-ce55-4cc3-b3b3-17bab3ecc2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:34.452855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:34.510514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81defc0-6876-4cc7-98b5-d6b0e781b9bd admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:34.524699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f757cc-18b4-4796-86b5-44ba7d1589bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:34.525558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f757cc-18b4-4796-86b5-44ba7d1589bc admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:34.525905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f757cc-18b4-4796-86b5-44ba7d1589bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:34.526261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f757cc-18b4-4796-86b5-44ba7d1589bc admin admin] PATH_INFO: `/v3/projects/424cbcca79d947f7a629d36a2cc90847/users/6e8b32fe37d342c2911d6e1ede9f37ee/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:34.552481 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3250/6435] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:34.552481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f46703-ce55-4cc3-b3b3-17bab3ecc2e7 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:34.552481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-41f46703-ce55-4cc3-b3b3-17bab3ecc2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:34.559738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f757cc-18b4-4796-86b5-44ba7d1589bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=424cbcca79d947f7a629d36a2cc90847, user_id=6e8b32fe37d342c2911d6e1ede9f37ee, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:34.562596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f757cc-18b4-4796-86b5-44ba7d1589bc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:34.592017 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3186/6436] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:34 2022] PUT /identity/v3/projects/424cbcca79d947f7a629d36a2cc90847/users/6e8b32fe37d342c2911d6e1ede9f37ee/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:34.593430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29215397-3b4b-4536-a53d-9b0d62e510f7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:34.593626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29215397-3b4b-4536-a53d-9b0d62e510f7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:34.593999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29215397-3b4b-4536-a53d-9b0d62e510f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:34.641165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-29215397-3b4b-4536-a53d-9b0d62e510f7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:34.642921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-29215397-3b4b-4536-a53d-9b0d62e510f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:34.707135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:34.768740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f46703-ce55-4cc3-b3b3-17bab3ecc2e7 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:34.808303 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3187/6437] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:34.808303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2fe04-192f-47c1-812e-85d7c059694e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:34.808303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2fe04-192f-47c1-812e-85d7c059694e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:34.808303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2fe04-192f-47c1-812e-85d7c059694e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:34.863999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-eea2fe04-192f-47c1-812e-85d7c059694e None None] MFA Rules not processed for user `6e8b32fe37d342c2911d6e1ede9f37ee`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:34.865227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-eea2fe04-192f-47c1-812e-85d7c059694e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:34.879692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13aea9d-1f4a-4243-8324-5a88c9eeb03b tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb521f26aa7f482e97f482a88d5c65ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b037c9cdf4ec4cf9a595bd0e316b1248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:34.881227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13aea9d-1f4a-4243-8324-5a88c9eeb03b tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:34.882132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13aea9d-1f4a-4243-8324-5a88c9eeb03b tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:34.882694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13aea9d-1f4a-4243-8324-5a88c9eeb03b tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] PATH_INFO: `/v3/roles/36ca23c3a5ef465b8781c0fbd1e660cb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:34.884825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13aea9d-1f4a-4243-8324-5a88c9eeb03b tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] RBAC: Authorizing `identity:delete_role(role_id=36ca23c3a5ef465b8781c0fbd1e660cb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:34.886830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13aea9d-1f4a-4243-8324-5a88c9eeb03b tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:34.969187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a13aea9d-1f4a-4243-8324-5a88c9eeb03b tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] Invalidating the token cache because role 36ca23c3a5ef465b8781c0fbd1e660cb has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:34.970305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a13aea9d-1f4a-4243-8324-5a88c9eeb03b tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:34.971361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a13aea9d-1f4a-4243-8324-5a88c9eeb03b tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:34.974060 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3251/6438] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:34 2022] DELETE /identity/v3/roles/36ca23c3a5ef465b8781c0fbd1e660cb => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:34.975734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13aea9d-1f4a-4243-8324-5a88c9eeb03b tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:34.980865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a13aea9d-1f4a-4243-8324-5a88c9eeb03b tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:35.005098 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3188/6439] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:35.005760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2fe04-192f-47c1-812e-85d7c059694e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:35.010999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-eea2fe04-192f-47c1-812e-85d7c059694e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:35.063678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2fe04-192f-47c1-812e-85d7c059694e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:35.109378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13aea9d-1f4a-4243-8324-5a88c9eeb03b tempest-ApplicationCredentialsV3AdminTest-1962310855 tempest-ApplicationCredentialsV3AdminTest-1962310855-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:35.125698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fe214-25db-4c4d-9cef-c88263c7f6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:35.126804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fe214-25db-4c4d-9cef-c88263c7f6f5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:35.129816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fe214-25db-4c4d-9cef-c88263c7f6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:35.130932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fe214-25db-4c4d-9cef-c88263c7f6f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:35.131826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fe214-25db-4c4d-9cef-c88263c7f6f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:13:35.133853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fe214-25db-4c4d-9cef-c88263c7f6f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:35.141151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fe214-25db-4c4d-9cef-c88263c7f6f5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:35.177281 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3252/6440] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:13:35.178941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fe214-25db-4c4d-9cef-c88263c7f6f5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:35.181602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-930fe214-25db-4c4d-9cef-c88263c7f6f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:35.200337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:35.208753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e373b3-77e4-48ac-8ab3-f17e3b54ed57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:35.209586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e373b3-77e4-48ac-8ab3-f17e3b54ed57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:35.210048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e373b3-77e4-48ac-8ab3-f17e3b54ed57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:35.212248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e373b3-77e4-48ac-8ab3-f17e3b54ed57 admin admin] PATH_INFO: `/v3/users/de7b0236737e4af29a591aa56c4de04d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:35.215553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e373b3-77e4-48ac-8ab3-f17e3b54ed57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de7b0236737e4af29a591aa56c4de04d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:35.218043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e373b3-77e4-48ac-8ab3-f17e3b54ed57 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:35.335907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e0e373b3-77e4-48ac-8ab3-f17e3b54ed57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de7b0236737e4af29a591aa56c4de04d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:35.340175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e0e373b3-77e4-48ac-8ab3-f17e3b54ed57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de7b0236737e4af29a591aa56c4de04d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:35.342427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e0e373b3-77e4-48ac-8ab3-f17e3b54ed57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de7b0236737e4af29a591aa56c4de04d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:35.351376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:35.376946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fe214-25db-4c4d-9cef-c88263c7f6f5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:35.379505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e0e373b3-77e4-48ac-8ab3-f17e3b54ed57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de7b0236737e4af29a591aa56c4de04d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:35.387395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473ee23-1bfe-499a-9874-ed4bf33165bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:35.388022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473ee23-1bfe-499a-9874-ed4bf33165bd service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:35.388293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473ee23-1bfe-499a-9874-ed4bf33165bd service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:35.388573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473ee23-1bfe-499a-9874-ed4bf33165bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:35.392153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e473ee23-1bfe-499a-9874-ed4bf33165bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:35.400272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e0e373b3-77e4-48ac-8ab3-f17e3b54ed57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de7b0236737e4af29a591aa56c4de04d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:35.418040 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3189/6441] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:35 2022] DELETE /identity/v3/users/de7b0236737e4af29a591aa56c4de04d => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:35.418850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e373b3-77e4-48ac-8ab3-f17e3b54ed57 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:35.420915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e0e373b3-77e4-48ac-8ab3-f17e3b54ed57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:35.522211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473ee23-1bfe-499a-9874-ed4bf33165bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:35.524802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473ee23-1bfe-499a-9874-ed4bf33165bd service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:35.527177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e373b3-77e4-48ac-8ab3-f17e3b54ed57 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:35.534962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8718e-1da7-41e3-864b-458533305223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:35.536589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8718e-1da7-41e3-864b-458533305223 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:35.537755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8718e-1da7-41e3-864b-458533305223 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:35.538693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8718e-1da7-41e3-864b-458533305223 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:35.540430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8718e-1da7-41e3-864b-458533305223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:35.546411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8718e-1da7-41e3-864b-458533305223 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:35.553913 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3253/6442] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:35.555306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1f1cc-9742-4c21-b342-6c2fae7608ab None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:35.555510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1f1cc-9742-4c21-b342-6c2fae7608ab None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:35.555785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1f1cc-9742-4c21-b342-6c2fae7608ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:35.584669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-77b1f1cc-9742-4c21-b342-6c2fae7608ab None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:35.588376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-77b1f1cc-9742-4c21-b342-6c2fae7608ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:35.603951 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3190/6443] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:35 2022] POST /identity/v3/projects => generated 383 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:35.605841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8718e-1da7-41e3-864b-458533305223 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:35.612230 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3254/6444] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:35.620771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1f1cc-9742-4c21-b342-6c2fae7608ab None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:35.622314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-77b1f1cc-9742-4c21-b342-6c2fae7608ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:35.776571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8718e-1da7-41e3-864b-458533305223 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:35.785001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1f1cc-9742-4c21-b342-6c2fae7608ab None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:35.799614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df32815-1de0-46ac-9451-254ce827f89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:35.800245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df32815-1de0-46ac-9451-254ce827f89f admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:35.800526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df32815-1de0-46ac-9451-254ce827f89f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:35.800798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df32815-1de0-46ac-9451-254ce827f89f admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:35.801525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df32815-1de0-46ac-9451-254ce827f89f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:35.803618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df32815-1de0-46ac-9451-254ce827f89f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:35.829744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:35.852545 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3191/6445] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:35 2022] POST /identity/v3/users => generated 292 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:13:35.853767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df32815-1de0-46ac-9451-254ce827f89f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:35.872030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df32815-1de0-46ac-9451-254ce827f89f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:35.885315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218437f-21b5-497a-84f5-4f8d168847f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:35.885951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218437f-21b5-497a-84f5-4f8d168847f8 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:35.886233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218437f-21b5-497a-84f5-4f8d168847f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:35.886529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218437f-21b5-497a-84f5-4f8d168847f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:35.887220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6218437f-21b5-497a-84f5-4f8d168847f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:35.889038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6218437f-21b5-497a-84f5-4f8d168847f8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:35.904248 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3192/6446] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:35.908300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218437f-21b5-497a-84f5-4f8d168847f8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:35.912780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4d43a-09dc-44ca-9c0c-10e6d69ab219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:35.915324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4d43a-09dc-44ca-9c0c-10e6d69ab219 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:35.915912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4d43a-09dc-44ca-9c0c-10e6d69ab219 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:35.916366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4d43a-09dc-44ca-9c0c-10e6d69ab219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:35.929987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4d43a-09dc-44ca-9c0c-10e6d69ab219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:35.932994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4d43a-09dc-44ca-9c0c-10e6d69ab219 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:35.936931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218437f-21b5-497a-84f5-4f8d168847f8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:35.946477 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3255/6447] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:35.953313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179da33-71f4-4706-923b-a2c795122f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:35.954833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179da33-71f4-4706-923b-a2c795122f54 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:35.955712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179da33-71f4-4706-923b-a2c795122f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:35.956173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179da33-71f4-4706-923b-a2c795122f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:35.957445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d179da33-71f4-4706-923b-a2c795122f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:35.959911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d179da33-71f4-4706-923b-a2c795122f54 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:35.978704 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3193/6448] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:35.982797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4d43a-09dc-44ca-9c0c-10e6d69ab219 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:35.998307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4d43a-09dc-44ca-9c0c-10e6d69ab219 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:36.032868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2be00e-2503-4aaf-a2ba-163cdc18d30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:36.032868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2be00e-2503-4aaf-a2ba-163cdc18d30a admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:36.032868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2be00e-2503-4aaf-a2ba-163cdc18d30a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:36.032868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2be00e-2503-4aaf-a2ba-163cdc18d30a admin admin] PATH_INFO: `/v3/projects/99af99b8903e4bcdb0f03bf35a6002c6/users/5b53ea073e824fc5ba6583298c8ad9c8/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:36.070380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2be00e-2503-4aaf-a2ba-163cdc18d30a admin admin] RBAC: Authorizing `identity:create_grant(project_id=99af99b8903e4bcdb0f03bf35a6002c6, user_id=5b53ea073e824fc5ba6583298c8ad9c8, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:36.073709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2be00e-2503-4aaf-a2ba-163cdc18d30a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:36.098261 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3256/6449] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:35 2022] PUT /identity/v3/projects/99af99b8903e4bcdb0f03bf35a6002c6/users/5b53ea073e824fc5ba6583298c8ad9c8/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:36.127238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760f397-77a6-4421-acfc-128580062d01 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:36.128720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760f397-77a6-4421-acfc-128580062d01 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:36.129009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760f397-77a6-4421-acfc-128580062d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:36.223990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7760f397-77a6-4421-acfc-128580062d01 None None] MFA Rules not processed for user `5b53ea073e824fc5ba6583298c8ad9c8`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:36.229976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7760f397-77a6-4421-acfc-128580062d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:36.491229 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3194/6450] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:36.495772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2be00e-2503-4aaf-a2ba-163cdc18d30a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:36.558434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:36.608169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760f397-77a6-4421-acfc-128580062d01 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:36.611281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7760f397-77a6-4421-acfc-128580062d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:36.656752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2be00e-2503-4aaf-a2ba-163cdc18d30a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:36.672821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da5c41-c807-4191-9e63-e046157dd416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:36.673653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da5c41-c807-4191-9e63-e046157dd416 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:36.675049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da5c41-c807-4191-9e63-e046157dd416 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:36.675963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da5c41-c807-4191-9e63-e046157dd416 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:36.678104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da5c41-c807-4191-9e63-e046157dd416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:36.682444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da5c41-c807-4191-9e63-e046157dd416 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:36.758569 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3257/6451] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:36 2022] POST /identity/v3/projects => generated 383 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:36.761582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da5c41-c807-4191-9e63-e046157dd416 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:36.818327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760f397-77a6-4421-acfc-128580062d01 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:36.932714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da5c41-c807-4191-9e63-e046157dd416 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:36.940651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f69fc0-68a6-4987-a4d0-71808f8bcd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:36.941441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f69fc0-68a6-4987-a4d0-71808f8bcd0c admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:36.941817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f69fc0-68a6-4987-a4d0-71808f8bcd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:36.942182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f69fc0-68a6-4987-a4d0-71808f8bcd0c admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:36.943976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f69fc0-68a6-4987-a4d0-71808f8bcd0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:36.947874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f69fc0-68a6-4987-a4d0-71808f8bcd0c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:36.972152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30cb11-5dbd-4a9e-a34e-98af5dfe6dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:36.974928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30cb11-5dbd-4a9e-a34e-98af5dfe6dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:36.975314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30cb11-5dbd-4a9e-a34e-98af5dfe6dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:36.975605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30cb11-5dbd-4a9e-a34e-98af5dfe6dc3 admin admin] PATH_INFO: `/v3/projects/8d3b3bbc444c40cd87c001df49cde2f3` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:36.979338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30cb11-5dbd-4a9e-a34e-98af5dfe6dc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d3b3bbc444c40cd87c001df49cde2f3)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:36.984102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30cb11-5dbd-4a9e-a34e-98af5dfe6dc3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:36.984294 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3258/6452] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:36 2022] POST /identity/v3/users => generated 292 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:36.985177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f69fc0-68a6-4987-a4d0-71808f8bcd0c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:36.998783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:37.023514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f69fc0-68a6-4987-a4d0-71808f8bcd0c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:37.041741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5961b-9a1a-4cef-b412-e0a529d1d08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:37.042320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5961b-9a1a-4cef-b412-e0a529d1d08e admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:37.042616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5961b-9a1a-4cef-b412-e0a529d1d08e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:37.042899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5961b-9a1a-4cef-b412-e0a529d1d08e admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:37.043559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5961b-9a1a-4cef-b412-e0a529d1d08e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:37.045190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5961b-9a1a-4cef-b412-e0a529d1d08e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:37.067572 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3259/6453] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:37.071546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5961b-9a1a-4cef-b412-e0a529d1d08e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:37.085222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cf30cb11-5dbd-4a9e-a34e-98af5dfe6dc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d3b3bbc444c40cd87c001df49cde2f3', 'request_id': 'req-cf30cb11-5dbd-4a9e-a34e-98af5dfe6dc3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:37.086875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5961b-9a1a-4cef-b412-e0a529d1d08e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:37.114408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cf30cb11-5dbd-4a9e-a34e-98af5dfe6dc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d3b3bbc444c40cd87c001df49cde2f3', 'request_id': 'req-cf30cb11-5dbd-4a9e-a34e-98af5dfe6dc3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:37.123660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cf30cb11-5dbd-4a9e-a34e-98af5dfe6dc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d3b3bbc444c40cd87c001df49cde2f3', 'request_id': 'req-cf30cb11-5dbd-4a9e-a34e-98af5dfe6dc3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:37.141079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cf30cb11-5dbd-4a9e-a34e-98af5dfe6dc3 admin admin] The token cache is being invalidate because project 8d3b3bbc444c40cd87c001df49cde2f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:37.141593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cf30cb11-5dbd-4a9e-a34e-98af5dfe6dc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:37.142862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cf30cb11-5dbd-4a9e-a34e-98af5dfe6dc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:37.144497 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3195/6454] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:36 2022] DELETE /identity/v3/projects/8d3b3bbc444c40cd87c001df49cde2f3 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:37.148099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30cb11-5dbd-4a9e-a34e-98af5dfe6dc3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:37.149638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cf30cb11-5dbd-4a9e-a34e-98af5dfe6dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:37.222387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025a03d-958b-4e63-b9ce-c18d2bf3c42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:37.223071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025a03d-958b-4e63-b9ce-c18d2bf3c42c admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:37.223352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025a03d-958b-4e63-b9ce-c18d2bf3c42c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:37.223625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025a03d-958b-4e63-b9ce-c18d2bf3c42c admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:37.224287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025a03d-958b-4e63-b9ce-c18d2bf3c42c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:37.226032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025a03d-958b-4e63-b9ce-c18d2bf3c42c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:37.242068 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3260/6455] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:37.246256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025a03d-958b-4e63-b9ce-c18d2bf3c42c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:37.248214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a025a03d-958b-4e63-b9ce-c18d2bf3c42c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:37.273420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30cb11-5dbd-4a9e-a34e-98af5dfe6dc3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:37.282152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9c8ed-5010-4ec4-aae6-dd771b9b8798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:37.284743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025a03d-958b-4e63-b9ce-c18d2bf3c42c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:37.285267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9c8ed-5010-4ec4-aae6-dd771b9b8798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:37.285727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9c8ed-5010-4ec4-aae6-dd771b9b8798 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:37.286157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9c8ed-5010-4ec4-aae6-dd771b9b8798 admin admin] PATH_INFO: `/v3/users/bb521f26aa7f482e97f482a88d5c65ef` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:37.287694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9c8ed-5010-4ec4-aae6-dd771b9b8798 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb521f26aa7f482e97f482a88d5c65ef)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:37.290245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9c8ed-5010-4ec4-aae6-dd771b9b8798 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:37.298124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3671b-47ce-40bc-b277-f48a15eb74a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:37.298787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3671b-47ce-40bc-b277-f48a15eb74a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:37.299319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3671b-47ce-40bc-b277-f48a15eb74a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:37.299608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3671b-47ce-40bc-b277-f48a15eb74a3 admin admin] PATH_INFO: `/v3/projects/c3cca72997054a53bba0852d70d70ee1/users/240f6cf0aba34a739a85a6e15925613a/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:37.318801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c3671b-47ce-40bc-b277-f48a15eb74a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3cca72997054a53bba0852d70d70ee1, user_id=240f6cf0aba34a739a85a6e15925613a, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:37.321837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c3671b-47ce-40bc-b277-f48a15eb74a3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:37.349174 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3261/6456] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:37 2022] PUT /identity/v3/projects/c3cca72997054a53bba0852d70d70ee1/users/240f6cf0aba34a739a85a6e15925613a/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:37.353249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39fbad-e176-4025-84bc-ee13bddb0a0a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:37.353549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39fbad-e176-4025-84bc-ee13bddb0a0a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:37.353973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39fbad-e176-4025-84bc-ee13bddb0a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:37.366621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2ec9c8ed-5010-4ec4-aae6-dd771b9b8798 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb521f26aa7f482e97f482a88d5c65ef'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:37.368212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:37.369989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2ec9c8ed-5010-4ec4-aae6-dd771b9b8798 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb521f26aa7f482e97f482a88d5c65ef'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:37.371209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2ec9c8ed-5010-4ec4-aae6-dd771b9b8798 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb521f26aa7f482e97f482a88d5c65ef'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:37.381342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2ec9c8ed-5010-4ec4-aae6-dd771b9b8798 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb521f26aa7f482e97f482a88d5c65ef'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:37.409255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ee39fbad-e176-4025-84bc-ee13bddb0a0a None None] MFA Rules not processed for user `240f6cf0aba34a739a85a6e15925613a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:37.410643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ee39fbad-e176-4025-84bc-ee13bddb0a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:37.417501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2ec9c8ed-5010-4ec4-aae6-dd771b9b8798 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb521f26aa7f482e97f482a88d5c65ef'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:37.440962 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3196/6457] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:37 2022] DELETE /identity/v3/users/bb521f26aa7f482e97f482a88d5c65ef => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:13:37.569185 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3262/6458] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:37.573344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9c8ed-5010-4ec4-aae6-dd771b9b8798 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:37.574817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2ec9c8ed-5010-4ec4-aae6-dd771b9b8798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:37.601588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:37.675561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9c8ed-5010-4ec4-aae6-dd771b9b8798 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:37.682764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2a40c-8538-4b29-b349-76dc56292564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:37.683630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2a40c-8538-4b29-b349-76dc56292564 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:37.683780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2a40c-8538-4b29-b349-76dc56292564 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:37.683973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2a40c-8538-4b29-b349-76dc56292564 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:37.684658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2a40c-8538-4b29-b349-76dc56292564 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:37.686606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2a40c-8538-4b29-b349-76dc56292564 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:37.738805 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3197/6459] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:37 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:37.743623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39fbad-e176-4025-84bc-ee13bddb0a0a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:37.823785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39fbad-e176-4025-84bc-ee13bddb0a0a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:37.854299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2a40c-8538-4b29-b349-76dc56292564 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:37.856353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-68f2a40c-8538-4b29-b349-76dc56292564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:37.897267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37293c37-311a-4d4a-b24f-304e03beea6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:37.897879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37293c37-311a-4d4a-b24f-304e03beea6f admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:37.898247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37293c37-311a-4d4a-b24f-304e03beea6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:37.898554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37293c37-311a-4d4a-b24f-304e03beea6f admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:37.899178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37293c37-311a-4d4a-b24f-304e03beea6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:37.901701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37293c37-311a-4d4a-b24f-304e03beea6f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:37.928701 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3263/6460] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:37 2022] POST /identity/v3/users => generated 298 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:13:37.930046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37293c37-311a-4d4a-b24f-304e03beea6f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:37.934314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2a40c-8538-4b29-b349-76dc56292564 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:37.939428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810278b2-43e8-446e-b258-f6f6224b5a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:37.940058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810278b2-43e8-446e-b258-f6f6224b5a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:37.940340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810278b2-43e8-446e-b258-f6f6224b5a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:37.940539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37293c37-311a-4d4a-b24f-304e03beea6f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:37.941018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810278b2-43e8-446e-b258-f6f6224b5a0e admin admin] PATH_INFO: `/v3/projects/b037c9cdf4ec4cf9a595bd0e316b1248` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:37.941822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810278b2-43e8-446e-b258-f6f6224b5a0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b037c9cdf4ec4cf9a595bd0e316b1248)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:37.943782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810278b2-43e8-446e-b258-f6f6224b5a0e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:37.945689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e7b141-b9da-4a20-ba52-5967fe5e448b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:37.946482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7b141-b9da-4a20-ba52-5967fe5e448b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:37.946793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7b141-b9da-4a20-ba52-5967fe5e448b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:37.947589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7b141-b9da-4a20-ba52-5967fe5e448b admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:37.948225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e7b141-b9da-4a20-ba52-5967fe5e448b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:37.949830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e7b141-b9da-4a20-ba52-5967fe5e448b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:37.959987 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3264/6461] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:37.964501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e7b141-b9da-4a20-ba52-5967fe5e448b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:37.975074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e7b141-b9da-4a20-ba52-5967fe5e448b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:37.980357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60e35d-b348-47ea-bf27-706c5ebfe52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:37.981011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60e35d-b348-47ea-bf27-706c5ebfe52d admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:37.981297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60e35d-b348-47ea-bf27-706c5ebfe52d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:37.981572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60e35d-b348-47ea-bf27-706c5ebfe52d admin admin] PATH_INFO: `/v3/projects/5e9e7e768c964f2e9856471cb26c4754/users/32363c7dc0ec4e33bdbe898136962a02/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:37.993828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-810278b2-43e8-446e-b258-f6f6224b5a0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b037c9cdf4ec4cf9a595bd0e316b1248', 'request_id': 'req-810278b2-43e8-446e-b258-f6f6224b5a0e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:37.997533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60e35d-b348-47ea-bf27-706c5ebfe52d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e9e7e768c964f2e9856471cb26c4754, user_id=32363c7dc0ec4e33bdbe898136962a02, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:38.001058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60e35d-b348-47ea-bf27-706c5ebfe52d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:38.001738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-810278b2-43e8-446e-b258-f6f6224b5a0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b037c9cdf4ec4cf9a595bd0e316b1248', 'request_id': 'req-810278b2-43e8-446e-b258-f6f6224b5a0e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:38.004505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:38.008428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-810278b2-43e8-446e-b258-f6f6224b5a0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b037c9cdf4ec4cf9a595bd0e316b1248', 'request_id': 'req-810278b2-43e8-446e-b258-f6f6224b5a0e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:38.018163 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3265/6462] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:37 2022] PUT /identity/v3/projects/5e9e7e768c964f2e9856471cb26c4754/users/32363c7dc0ec4e33bdbe898136962a02/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:13:38.020622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-810278b2-43e8-446e-b258-f6f6224b5a0e admin admin] The token cache is being invalidate because project b037c9cdf4ec4cf9a595bd0e316b1248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:38.020916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-810278b2-43e8-446e-b258-f6f6224b5a0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:38.021732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-810278b2-43e8-446e-b258-f6f6224b5a0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:38.022451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60e35d-b348-47ea-bf27-706c5ebfe52d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:38.023768 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3198/6463] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:37 2022] DELETE /identity/v3/projects/b037c9cdf4ec4cf9a595bd0e316b1248 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:38.024573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6d60e35d-b348-47ea-bf27-706c5ebfe52d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:38.032652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e32b00-3853-4914-b945-c46d91d17c51 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:38.033388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e32b00-3853-4914-b945-c46d91d17c51 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:38.034209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e32b00-3853-4914-b945-c46d91d17c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:38.056081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a4e32b00-3853-4914-b945-c46d91d17c51 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:38.058082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a4e32b00-3853-4914-b945-c46d91d17c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:38.083573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:38.099687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60e35d-b348-47ea-bf27-706c5ebfe52d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:38.105220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fd653-a2ef-4377-84c6-d2dcb860375f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:38.105829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fd653-a2ef-4377-84c6-d2dcb860375f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:38.106118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fd653-a2ef-4377-84c6-d2dcb860375f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:38.106395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fd653-a2ef-4377-84c6-d2dcb860375f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:38.107074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3fd653-a2ef-4377-84c6-d2dcb860375f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:38.108667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3fd653-a2ef-4377-84c6-d2dcb860375f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:38.118861 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3266/6464] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:13:38.122111 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3199/6465] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:38.123242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fd653-a2ef-4377-84c6-d2dcb860375f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:38.129569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614903c-0e51-4313-b38f-29693b163e43 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:38.129852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614903c-0e51-4313-b38f-29693b163e43 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:38.130135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614903c-0e51-4313-b38f-29693b163e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:38.133382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fd653-a2ef-4377-84c6-d2dcb860375f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:38.137993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5581b-e9ca-41fe-a5d4-1afcf418731e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:38.138660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5581b-e9ca-41fe-a5d4-1afcf418731e admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:38.138943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5581b-e9ca-41fe-a5d4-1afcf418731e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:38.139225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5581b-e9ca-41fe-a5d4-1afcf418731e admin admin] PATH_INFO: `/v3/projects/5e9e7e768c964f2e9856471cb26c4754/users/32363c7dc0ec4e33bdbe898136962a02/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:38.141302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e5581b-e9ca-41fe-a5d4-1afcf418731e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e9e7e768c964f2e9856471cb26c4754, user_id=32363c7dc0ec4e33bdbe898136962a02, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:38.144194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e5581b-e9ca-41fe-a5d4-1afcf418731e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:38.149761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7614903c-0e51-4313-b38f-29693b163e43 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:38.151518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7614903c-0e51-4313-b38f-29693b163e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:38.155367 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3267/6466] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:38 2022] PUT /identity/v3/projects/5e9e7e768c964f2e9856471cb26c4754/users/32363c7dc0ec4e33bdbe898136962a02/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:38.166905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f37a3-01bd-4a71-aaf4-459e409c7165 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:38.167190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f37a3-01bd-4a71-aaf4-459e409c7165 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:38.167471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f37a3-01bd-4a71-aaf4-459e409c7165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:38.206974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6c2f37a3-01bd-4a71-aaf4-459e409c7165 None None] MFA Rules not processed for user `32363c7dc0ec4e33bdbe898136962a02`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:38.208938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6c2f37a3-01bd-4a71-aaf4-459e409c7165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:38.234873 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3200/6467] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:38.236222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614903c-0e51-4313-b38f-29693b163e43 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:38.250957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614903c-0e51-4313-b38f-29693b163e43 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:38.257323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2712e9-948f-47f1-991c-2e36646a0f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:38.258200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2712e9-948f-47f1-991c-2e36646a0f7c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:38.258551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2712e9-948f-47f1-991c-2e36646a0f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:38.258829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2712e9-948f-47f1-991c-2e36646a0f7c admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:38.259532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2712e9-948f-47f1-991c-2e36646a0f7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:13:38.260093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2712e9-948f-47f1-991c-2e36646a0f7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:38.262644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2712e9-948f-47f1-991c-2e36646a0f7c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:38.291838 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3201/6468] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:13:38.293030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2712e9-948f-47f1-991c-2e36646a0f7c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:38.385705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2712e9-948f-47f1-991c-2e36646a0f7c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:38.391590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1257aed-dc9d-4f43-8337-99ae2ede6e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:38.392507 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3268/6469] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:38.394520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1257aed-dc9d-4f43-8337-99ae2ede6e91 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:38.394822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1257aed-dc9d-4f43-8337-99ae2ede6e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:38.395099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1257aed-dc9d-4f43-8337-99ae2ede6e91 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:38.395881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1257aed-dc9d-4f43-8337-99ae2ede6e91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:38.397936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1257aed-dc9d-4f43-8337-99ae2ede6e91 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:38.412842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f37a3-01bd-4a71-aaf4-459e409c7165 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:38.417436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6c2f37a3-01bd-4a71-aaf4-459e409c7165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:38.476929 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3202/6470] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:38 2022] POST /identity/v3/projects => generated 375 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:38.492153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1257aed-dc9d-4f43-8337-99ae2ede6e91 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:38.541483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:38.576180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f37a3-01bd-4a71-aaf4-459e409c7165 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:38.638118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1257aed-dc9d-4f43-8337-99ae2ede6e91 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:38.673475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92502035-3d57-4fbf-b6b2-6dfdc2e4bd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:38.674360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92502035-3d57-4fbf-b6b2-6dfdc2e4bd5a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:38.674770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92502035-3d57-4fbf-b6b2-6dfdc2e4bd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:38.675045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92502035-3d57-4fbf-b6b2-6dfdc2e4bd5a admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:38.676311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92502035-3d57-4fbf-b6b2-6dfdc2e4bd5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:38.691513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92502035-3d57-4fbf-b6b2-6dfdc2e4bd5a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:38.706241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da918f5-b65e-4c84-8320-a0b64e7aae33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:38.707868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da918f5-b65e-4c84-8320-a0b64e7aae33 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:38.708427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da918f5-b65e-4c84-8320-a0b64e7aae33 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:38.709036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da918f5-b65e-4c84-8320-a0b64e7aae33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:38.711491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0da918f5-b65e-4c84-8320-a0b64e7aae33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:38.721681 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3203/6471] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:38 2022] POST /identity/v3/users => generated 288 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:38.725884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92502035-3d57-4fbf-b6b2-6dfdc2e4bd5a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:38.739116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:38.763893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92502035-3d57-4fbf-b6b2-6dfdc2e4bd5a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:38.783813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498c068-d762-4ba9-8bfd-e2916062f57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:38.784458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498c068-d762-4ba9-8bfd-e2916062f57d admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:38.784731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498c068-d762-4ba9-8bfd-e2916062f57d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:38.788588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498c068-d762-4ba9-8bfd-e2916062f57d admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:38.790343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498c068-d762-4ba9-8bfd-e2916062f57d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:38.798900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498c068-d762-4ba9-8bfd-e2916062f57d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:38.832452 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3204/6472] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:38.833894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498c068-d762-4ba9-8bfd-e2916062f57d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:38.855460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498c068-d762-4ba9-8bfd-e2916062f57d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:38.861912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475df83c-4a8d-44e1-a2ae-28a598556dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:38.862673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475df83c-4a8d-44e1-a2ae-28a598556dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:38.862822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475df83c-4a8d-44e1-a2ae-28a598556dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:38.863081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475df83c-4a8d-44e1-a2ae-28a598556dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:38.863723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475df83c-4a8d-44e1-a2ae-28a598556dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:38.869836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475df83c-4a8d-44e1-a2ae-28a598556dd9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:38.906474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da918f5-b65e-4c84-8320-a0b64e7aae33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:38.908338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da918f5-b65e-4c84-8320-a0b64e7aae33 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:38.917803 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3205/6473] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:38.918744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475df83c-4a8d-44e1-a2ae-28a598556dd9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:38.936287 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3269/6474] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:38.940396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475df83c-4a8d-44e1-a2ae-28a598556dd9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:38.953618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660d014-be6d-482e-b1d7-38d2fc7b71a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:38.956635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660d014-be6d-482e-b1d7-38d2fc7b71a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:38.956635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660d014-be6d-482e-b1d7-38d2fc7b71a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:38.956635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660d014-be6d-482e-b1d7-38d2fc7b71a3 admin admin] PATH_INFO: `/v3/projects/4714a8f6f9194d399961603f33749dfc/users/c65b273e951a49698f65518120c3df63/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:38.999795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660d014-be6d-482e-b1d7-38d2fc7b71a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4714a8f6f9194d399961603f33749dfc, user_id=c65b273e951a49698f65518120c3df63, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:39.007147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660d014-be6d-482e-b1d7-38d2fc7b71a3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:39.048794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85f3ef-6449-46d0-91f6-92a77c937c62 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:39.049621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85f3ef-6449-46d0-91f6-92a77c937c62 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:39.049621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85f3ef-6449-46d0-91f6-92a77c937c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:39.051398 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3206/6475] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:38 2022] PUT /identity/v3/projects/4714a8f6f9194d399961603f33749dfc/users/c65b273e951a49698f65518120c3df63/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:39.145309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5f85f3ef-6449-46d0-91f6-92a77c937c62 None None] MFA Rules not processed for user `c65b273e951a49698f65518120c3df63`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:39.152686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5f85f3ef-6449-46d0-91f6-92a77c937c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:39.295706 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3270/6476] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:39.300565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660d014-be6d-482e-b1d7-38d2fc7b71a3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:39.447664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660d014-be6d-482e-b1d7-38d2fc7b71a3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:39.456180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5dbf34-bd9a-4279-9011-9d9a88376a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:39.456880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5dbf34-bd9a-4279-9011-9d9a88376a21 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:39.457228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5dbf34-bd9a-4279-9011-9d9a88376a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:39.457508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5dbf34-bd9a-4279-9011-9d9a88376a21 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:39.458230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5dbf34-bd9a-4279-9011-9d9a88376a21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:39.460518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5dbf34-bd9a-4279-9011-9d9a88376a21 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:39.504762 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3207/6477] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:39 2022] POST /identity/v3/projects => generated 375 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:39.508893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85f3ef-6449-46d0-91f6-92a77c937c62 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:39.540884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85f3ef-6449-46d0-91f6-92a77c937c62 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:39.686346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:39.708328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97243ad0-04eb-4574-8ac9-b8413fa8d7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:39.709045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97243ad0-04eb-4574-8ac9-b8413fa8d7ca admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:39.709321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97243ad0-04eb-4574-8ac9-b8413fa8d7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:39.709757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97243ad0-04eb-4574-8ac9-b8413fa8d7ca admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:39.710446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97243ad0-04eb-4574-8ac9-b8413fa8d7ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:39.712532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97243ad0-04eb-4574-8ac9-b8413fa8d7ca admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:39.758034 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3271/6478] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:39 2022] POST /identity/v3/users => generated 294 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:13:39.763168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5dbf34-bd9a-4279-9011-9d9a88376a21 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:39.790192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5dbf34-bd9a-4279-9011-9d9a88376a21 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:39.808456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30d128-7ab7-4871-b175-c886cd3d9d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:39.810905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30d128-7ab7-4871-b175-c886cd3d9d3f admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:39.811445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30d128-7ab7-4871-b175-c886cd3d9d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:39.811814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30d128-7ab7-4871-b175-c886cd3d9d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:39.812627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30d128-7ab7-4871-b175-c886cd3d9d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:39.814721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30d128-7ab7-4871-b175-c886cd3d9d3f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:39.836515 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3208/6479] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:39.840588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97243ad0-04eb-4574-8ac9-b8413fa8d7ca admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:39.854859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97243ad0-04eb-4574-8ac9-b8413fa8d7ca admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:39.867016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90f0c5-13c3-4f47-9e4e-729025127551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:39.868083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90f0c5-13c3-4f47-9e4e-729025127551 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:39.868497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90f0c5-13c3-4f47-9e4e-729025127551 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:39.869738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90f0c5-13c3-4f47-9e4e-729025127551 admin admin] PATH_INFO: `/v3/projects/abdd7ccdaec14ed78b14f5e14b29cfca/users/f4982c8f32c84971a10543c3fc74d23f/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:39.926437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90f0c5-13c3-4f47-9e4e-729025127551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abdd7ccdaec14ed78b14f5e14b29cfca, user_id=f4982c8f32c84971a10543c3fc74d23f, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:39.934415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90f0c5-13c3-4f47-9e4e-729025127551 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:39.970947 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3272/6480] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:39 2022] PUT /identity/v3/projects/abdd7ccdaec14ed78b14f5e14b29cfca/users/f4982c8f32c84971a10543c3fc74d23f/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:39.972137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30d128-7ab7-4871-b175-c886cd3d9d3f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:39.984812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30d128-7ab7-4871-b175-c886cd3d9d3f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:40.093025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90f0c5-13c3-4f47-9e4e-729025127551 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:40.095243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-da90f0c5-13c3-4f47-9e4e-729025127551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:40.141272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc2cec-ba32-4a3c-90f7-4aaff041427e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:40.142119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc2cec-ba32-4a3c-90f7-4aaff041427e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:40.142318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc2cec-ba32-4a3c-90f7-4aaff041427e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:40.142686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc2cec-ba32-4a3c-90f7-4aaff041427e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:40.143398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc2cec-ba32-4a3c-90f7-4aaff041427e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:40.145083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc2cec-ba32-4a3c-90f7-4aaff041427e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:40.167689 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3209/6481] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:40.171922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc2cec-ba32-4a3c-90f7-4aaff041427e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:40.204146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc2cec-ba32-4a3c-90f7-4aaff041427e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:40.214938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90f0c5-13c3-4f47-9e4e-729025127551 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:40.215361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29487be2-27cc-4c6e-8e73-29ec0c6eca78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:40.216063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29487be2-27cc-4c6e-8e73-29ec0c6eca78 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:40.216339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29487be2-27cc-4c6e-8e73-29ec0c6eca78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:40.216618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29487be2-27cc-4c6e-8e73-29ec0c6eca78 admin admin] PATH_INFO: `/v3/projects/abdd7ccdaec14ed78b14f5e14b29cfca/users/f4982c8f32c84971a10543c3fc74d23f/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:40.228809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29487be2-27cc-4c6e-8e73-29ec0c6eca78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abdd7ccdaec14ed78b14f5e14b29cfca, user_id=f4982c8f32c84971a10543c3fc74d23f, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:40.228809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29487be2-27cc-4c6e-8e73-29ec0c6eca78 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:40.238020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2259f15-3208-4aa9-ad48-afa3ae8be8fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:40.241139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2259f15-3208-4aa9-ad48-afa3ae8be8fa service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:40.242305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2259f15-3208-4aa9-ad48-afa3ae8be8fa service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:40.242910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2259f15-3208-4aa9-ad48-afa3ae8be8fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:40.246265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e2259f15-3208-4aa9-ad48-afa3ae8be8fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:40.249778 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3210/6482] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:40 2022] PUT /identity/v3/projects/abdd7ccdaec14ed78b14f5e14b29cfca/users/f4982c8f32c84971a10543c3fc74d23f/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:13:40.261128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cb89e-a4ab-422f-9980-8e6dd3196964 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:40.261418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cb89e-a4ab-422f-9980-8e6dd3196964 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:40.261716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cb89e-a4ab-422f-9980-8e6dd3196964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:40.335860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:40.342873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5f9cb89e-a4ab-422f-9980-8e6dd3196964 None None] MFA Rules not processed for user `f4982c8f32c84971a10543c3fc74d23f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:40.344669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5f9cb89e-a4ab-422f-9980-8e6dd3196964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:40.391896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2259f15-3208-4aa9-ad48-afa3ae8be8fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:40.394429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2259f15-3208-4aa9-ad48-afa3ae8be8fa service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:40.453780 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3273/6483] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:40.509301 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3211/6484] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:40.513832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2259f15-3208-4aa9-ad48-afa3ae8be8fa service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:40.626904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:40.703797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2259f15-3208-4aa9-ad48-afa3ae8be8fa service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:40.710582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e468f4f-bb04-4fd4-a036-fe67c558aad6 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4982c8f32c84971a10543c3fc74d23f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abdd7ccdaec14ed78b14f5e14b29cfca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:40.711212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e468f4f-bb04-4fd4-a036-fe67c558aad6 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:40.711733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e468f4f-bb04-4fd4-a036-fe67c558aad6 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:40.712169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e468f4f-bb04-4fd4-a036-fe67c558aad6 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:40.713641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e468f4f-bb04-4fd4-a036-fe67c558aad6 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:40.716488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e468f4f-bb04-4fd4-a036-fe67c558aad6 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:40.780510 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3274/6485] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:40 2022] POST /identity/v3/projects => generated 334 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:40.783993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9cb89e-a4ab-422f-9980-8e6dd3196964 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:40.797110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9cb89e-a4ab-422f-9980-8e6dd3196964 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:40.916095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e468f4f-bb04-4fd4-a036-fe67c558aad6 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:40.971717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc55248-c314-48fb-94ac-9695d2bf1eb8 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4982c8f32c84971a10543c3fc74d23f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abdd7ccdaec14ed78b14f5e14b29cfca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:40.972353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc55248-c314-48fb-94ac-9695d2bf1eb8 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:40.972642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc55248-c314-48fb-94ac-9695d2bf1eb8 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:40.972920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc55248-c314-48fb-94ac-9695d2bf1eb8 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:40.974053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc55248-c314-48fb-94ac-9695d2bf1eb8 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:40.975986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc55248-c314-48fb-94ac-9695d2bf1eb8 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:41.024288 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3212/6486] 10.222.0.17 () {60 vars in 1152 bytes} [Sat Oct 1 21:13:40 2022] POST /identity/v3/projects => generated 332 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:41.033713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc55248-c314-48fb-94ac-9695d2bf1eb8 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:41.036162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e468f4f-bb04-4fd4-a036-fe67c558aad6 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:41.108524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240874c8-fcc3-49f9-9df8-5c2b0ccbff47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:41.109112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240874c8-fcc3-49f9-9df8-5c2b0ccbff47 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:41.109661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240874c8-fcc3-49f9-9df8-5c2b0ccbff47 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:41.109983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240874c8-fcc3-49f9-9df8-5c2b0ccbff47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:41.112373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-240874c8-fcc3-49f9-9df8-5c2b0ccbff47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:41.115065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc55248-c314-48fb-94ac-9695d2bf1eb8 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:41.121241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c576a3f-167b-42f9-bcf3-65eb62da48b9 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4982c8f32c84971a10543c3fc74d23f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abdd7ccdaec14ed78b14f5e14b29cfca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:41.122611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c576a3f-167b-42f9-bcf3-65eb62da48b9 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:41.122899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c576a3f-167b-42f9-bcf3-65eb62da48b9 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:41.123199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c576a3f-167b-42f9-bcf3-65eb62da48b9 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:41.124251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c576a3f-167b-42f9-bcf3-65eb62da48b9 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:41.126273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c576a3f-167b-42f9-bcf3-65eb62da48b9 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:41.181936 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3213/6487] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:41 2022] POST /identity/v3/projects => generated 358 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:41.186363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c576a3f-167b-42f9-bcf3-65eb62da48b9 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:41.191869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:41.221270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240874c8-fcc3-49f9-9df8-5c2b0ccbff47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:41.223231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240874c8-fcc3-49f9-9df8-5c2b0ccbff47 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:41.236480 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3275/6488] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:41.300523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c576a3f-167b-42f9-bcf3-65eb62da48b9 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:41.307474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06346539-4385-4550-b48b-f267e715856e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4982c8f32c84971a10543c3fc74d23f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abdd7ccdaec14ed78b14f5e14b29cfca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:41.308065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06346539-4385-4550-b48b-f267e715856e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:41.308355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06346539-4385-4550-b48b-f267e715856e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:41.308707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06346539-4385-4550-b48b-f267e715856e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:41.309303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06346539-4385-4550-b48b-f267e715856e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:41.312102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06346539-4385-4550-b48b-f267e715856e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:41.344813 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3214/6489] 10.222.0.17 () {58 vars in 1131 bytes} [Sat Oct 1 21:13:41 2022] GET /identity/v3/projects => generated 6856 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:41.350999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240874c8-fcc3-49f9-9df8-5c2b0ccbff47 service cinder] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:41.364978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240874c8-fcc3-49f9-9df8-5c2b0ccbff47 service cinder] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:41.370256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1cb42f-1d42-4a5b-9ee5-bf8353852c4a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4982c8f32c84971a10543c3fc74d23f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abdd7ccdaec14ed78b14f5e14b29cfca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:41.371157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cb42f-1d42-4a5b-9ee5-bf8353852c4a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:41.371400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cb42f-1d42-4a5b-9ee5-bf8353852c4a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:41.371782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cb42f-1d42-4a5b-9ee5-bf8353852c4a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:41.372706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1cb42f-1d42-4a5b-9ee5-bf8353852c4a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: Adding query filter params (enabled=False) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:13:41.373144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1cb42f-1d42-4a5b-9ee5-bf8353852c4a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:41.375965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1cb42f-1d42-4a5b-9ee5-bf8353852c4a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:41.400426 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3276/6490] 10.222.0.17 () {58 vars in 1158 bytes} [Sat Oct 1 21:13:41 2022] GET /identity/v3/projects?enabled=False => generated 445 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:13:41.420198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06346539-4385-4550-b48b-f267e715856e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:41.432844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06346539-4385-4550-b48b-f267e715856e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:41.438566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307089cd-4b59-4a4d-9171-235544c6fe28 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4982c8f32c84971a10543c3fc74d23f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abdd7ccdaec14ed78b14f5e14b29cfca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:41.439320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307089cd-4b59-4a4d-9171-235544c6fe28 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:41.439737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307089cd-4b59-4a4d-9171-235544c6fe28 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:41.440015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307089cd-4b59-4a4d-9171-235544c6fe28 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:41.440900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307089cd-4b59-4a4d-9171-235544c6fe28 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: Adding query filter params (parent_id=8dccf65146d048bba8721d9d37d63aca) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:13:41.441146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307089cd-4b59-4a4d-9171-235544c6fe28 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:41.443081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307089cd-4b59-4a4d-9171-235544c6fe28 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:41.467515 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3215/6491] 10.222.0.17 () {58 vars in 1216 bytes} [Sat Oct 1 21:13:41 2022] GET /identity/v3/projects?parent_id=8dccf65146d048bba8721d9d37d63aca => generated 498 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:13:41.468493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1cb42f-1d42-4a5b-9ee5-bf8353852c4a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:41.481222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1cb42f-1d42-4a5b-9ee5-bf8353852c4a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:41.488374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240c0c7-ec51-49d9-b4d2-f402ea4d218e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4982c8f32c84971a10543c3fc74d23f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abdd7ccdaec14ed78b14f5e14b29cfca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:41.490671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240c0c7-ec51-49d9-b4d2-f402ea4d218e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:41.493050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240c0c7-ec51-49d9-b4d2-f402ea4d218e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:41.494032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240c0c7-ec51-49d9-b4d2-f402ea4d218e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] PATH_INFO: `/v3/projects/7f952885f8a74c6aa87387b25b476b68` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:41.496335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240c0c7-ec51-49d9-b4d2-f402ea4d218e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7f952885f8a74c6aa87387b25b476b68)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:41.500288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240c0c7-ec51-49d9-b4d2-f402ea4d218e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:41.543083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:41.587120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9240c0c7-ec51-49d9-b4d2-f402ea4d218e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f952885f8a74c6aa87387b25b476b68', 'request_id': 'req-9240c0c7-ec51-49d9-b4d2-f402ea4d218e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:41.597073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9240c0c7-ec51-49d9-b4d2-f402ea4d218e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f952885f8a74c6aa87387b25b476b68', 'request_id': 'req-9240c0c7-ec51-49d9-b4d2-f402ea4d218e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:41.605571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9240c0c7-ec51-49d9-b4d2-f402ea4d218e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f952885f8a74c6aa87387b25b476b68', 'request_id': 'req-9240c0c7-ec51-49d9-b4d2-f402ea4d218e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:41.622368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9240c0c7-ec51-49d9-b4d2-f402ea4d218e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] The token cache is being invalidate because project 7f952885f8a74c6aa87387b25b476b68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:41.623514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9240c0c7-ec51-49d9-b4d2-f402ea4d218e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:41.624193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9240c0c7-ec51-49d9-b4d2-f402ea4d218e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:41.626598 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3277/6492] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:41 2022] DELETE /identity/v3/projects/7f952885f8a74c6aa87387b25b476b68 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:41.631175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307089cd-4b59-4a4d-9171-235544c6fe28 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:41.634864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-307089cd-4b59-4a4d-9171-235544c6fe28 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:41.653017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307089cd-4b59-4a4d-9171-235544c6fe28 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:41.726153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefc09d-bfd7-4ecd-8ad4-7a2d2ce71790 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4982c8f32c84971a10543c3fc74d23f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abdd7ccdaec14ed78b14f5e14b29cfca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:41.726927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefc09d-bfd7-4ecd-8ad4-7a2d2ce71790 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:41.727110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefc09d-bfd7-4ecd-8ad4-7a2d2ce71790 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:41.727375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefc09d-bfd7-4ecd-8ad4-7a2d2ce71790 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] PATH_INFO: `/v3/projects/8dccf65146d048bba8721d9d37d63aca` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:41.728641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cefc09d-bfd7-4ecd-8ad4-7a2d2ce71790 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8dccf65146d048bba8721d9d37d63aca)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:41.730748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cefc09d-bfd7-4ecd-8ad4-7a2d2ce71790 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:41.787769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2cefc09d-bfd7-4ecd-8ad4-7a2d2ce71790 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dccf65146d048bba8721d9d37d63aca', 'request_id': 'req-2cefc09d-bfd7-4ecd-8ad4-7a2d2ce71790'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:41.800173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2cefc09d-bfd7-4ecd-8ad4-7a2d2ce71790 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dccf65146d048bba8721d9d37d63aca', 'request_id': 'req-2cefc09d-bfd7-4ecd-8ad4-7a2d2ce71790'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:41.807447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2cefc09d-bfd7-4ecd-8ad4-7a2d2ce71790 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dccf65146d048bba8721d9d37d63aca', 'request_id': 'req-2cefc09d-bfd7-4ecd-8ad4-7a2d2ce71790'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:41.822555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2cefc09d-bfd7-4ecd-8ad4-7a2d2ce71790 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] The token cache is being invalidate because project 8dccf65146d048bba8721d9d37d63aca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:41.822555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2cefc09d-bfd7-4ecd-8ad4-7a2d2ce71790 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:41.822555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2cefc09d-bfd7-4ecd-8ad4-7a2d2ce71790 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:41.827378 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3216/6493] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:41 2022] DELETE /identity/v3/projects/8dccf65146d048bba8721d9d37d63aca => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:41.828367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240c0c7-ec51-49d9-b4d2-f402ea4d218e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:41.842757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240c0c7-ec51-49d9-b4d2-f402ea4d218e tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:41.920244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9e0a6-cabc-4dbc-8fcc-e850e7ffc33a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4982c8f32c84971a10543c3fc74d23f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abdd7ccdaec14ed78b14f5e14b29cfca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:41.921321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9e0a6-cabc-4dbc-8fcc-e850e7ffc33a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:41.921786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9e0a6-cabc-4dbc-8fcc-e850e7ffc33a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:41.922156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9e0a6-cabc-4dbc-8fcc-e850e7ffc33a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] PATH_INFO: `/v3/projects/4e0b84e87f6d440ba2f0ec782aefe81a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:41.924873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9e0a6-cabc-4dbc-8fcc-e850e7ffc33a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4e0b84e87f6d440ba2f0ec782aefe81a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:41.926928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9e0a6-cabc-4dbc-8fcc-e850e7ffc33a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:41.982645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b0a9e0a6-cabc-4dbc-8fcc-e850e7ffc33a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e0b84e87f6d440ba2f0ec782aefe81a', 'request_id': 'req-b0a9e0a6-cabc-4dbc-8fcc-e850e7ffc33a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:41.990140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b0a9e0a6-cabc-4dbc-8fcc-e850e7ffc33a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e0b84e87f6d440ba2f0ec782aefe81a', 'request_id': 'req-b0a9e0a6-cabc-4dbc-8fcc-e850e7ffc33a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:41.998177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefc09d-bfd7-4ecd-8ad4-7a2d2ce71790 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:41.998521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b0a9e0a6-cabc-4dbc-8fcc-e850e7ffc33a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e0b84e87f6d440ba2f0ec782aefe81a', 'request_id': 'req-b0a9e0a6-cabc-4dbc-8fcc-e850e7ffc33a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:41.999964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2cefc09d-bfd7-4ecd-8ad4-7a2d2ce71790 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:42.016267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b0a9e0a6-cabc-4dbc-8fcc-e850e7ffc33a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] The token cache is being invalidate because project 4e0b84e87f6d440ba2f0ec782aefe81a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:42.016679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b0a9e0a6-cabc-4dbc-8fcc-e850e7ffc33a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:42.017419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b0a9e0a6-cabc-4dbc-8fcc-e850e7ffc33a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:42.018650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:42.023108 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3278/6494] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:41 2022] DELETE /identity/v3/projects/4e0b84e87f6d440ba2f0ec782aefe81a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:13:42.024787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9e0a6-cabc-4dbc-8fcc-e850e7ffc33a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:42.026702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b0a9e0a6-cabc-4dbc-8fcc-e850e7ffc33a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:42.075198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefc09d-bfd7-4ecd-8ad4-7a2d2ce71790 tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:42.084956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:42.115827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9e0a6-cabc-4dbc-8fcc-e850e7ffc33a tempest-ListProjectsTestJSON-696233045 tempest-ListProjectsTestJSON-696233045-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:42.125718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48972294-794a-4ebd-bfc9-9d7c286124a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:42.126256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48972294-794a-4ebd-bfc9-9d7c286124a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:42.127891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48972294-794a-4ebd-bfc9-9d7c286124a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:42.129322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48972294-794a-4ebd-bfc9-9d7c286124a5 admin admin] PATH_INFO: `/v3/users/c65b273e951a49698f65518120c3df63` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:42.130193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48972294-794a-4ebd-bfc9-9d7c286124a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c65b273e951a49698f65518120c3df63)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:42.132322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48972294-794a-4ebd-bfc9-9d7c286124a5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:42.146435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3153690a-a432-49a8-9547-3adec88a8964 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:42.147362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153690a-a432-49a8-9547-3adec88a8964 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:42.147746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153690a-a432-49a8-9547-3adec88a8964 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:42.148124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153690a-a432-49a8-9547-3adec88a8964 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:42.150297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3153690a-a432-49a8-9547-3adec88a8964 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:42.211032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48972294-794a-4ebd-bfc9-9d7c286124a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c65b273e951a49698f65518120c3df63'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:42.222746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48972294-794a-4ebd-bfc9-9d7c286124a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c65b273e951a49698f65518120c3df63'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:42.231828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153690a-a432-49a8-9547-3adec88a8964 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:42.233344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153690a-a432-49a8-9547-3adec88a8964 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:42.246575 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3217/6495] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:13:42.257235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48972294-794a-4ebd-bfc9-9d7c286124a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c65b273e951a49698f65518120c3df63'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:42.259490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48972294-794a-4ebd-bfc9-9d7c286124a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c65b273e951a49698f65518120c3df63'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:42.297935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48972294-794a-4ebd-bfc9-9d7c286124a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c65b273e951a49698f65518120c3df63'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:42.311905 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3279/6496] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:42 2022] DELETE /identity/v3/users/c65b273e951a49698f65518120c3df63 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:42.315414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241035-1db4-4fb0-aa7b-d545e76050a9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:42.315696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241035-1db4-4fb0-aa7b-d545e76050a9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:42.315970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241035-1db4-4fb0-aa7b-d545e76050a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:42.400891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9a241035-1db4-4fb0-aa7b-d545e76050a9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:42.402244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9a241035-1db4-4fb0-aa7b-d545e76050a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:42.563954 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3218/6497] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:42.571251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48972294-794a-4ebd-bfc9-9d7c286124a5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:42.573353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-48972294-794a-4ebd-bfc9-9d7c286124a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:42.663061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48972294-794a-4ebd-bfc9-9d7c286124a5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:42.669470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c2ede-7e8a-4a69-9748-8fc9c5ac6799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:42.670409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c2ede-7e8a-4a69-9748-8fc9c5ac6799 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:42.670625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c2ede-7e8a-4a69-9748-8fc9c5ac6799 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:42.671023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c2ede-7e8a-4a69-9748-8fc9c5ac6799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:42.684836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c2ede-7e8a-4a69-9748-8fc9c5ac6799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:42.686981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c2ede-7e8a-4a69-9748-8fc9c5ac6799 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:42.701407 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3280/6498] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:13:43.079395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a241035-1db4-4fb0-aa7b-d545e76050a9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:43.081295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9a241035-1db4-4fb0-aa7b-d545e76050a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:43.113822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a241035-1db4-4fb0-aa7b-d545e76050a9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:43.130277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8893d4-b77f-441e-8470-8083cbbf284e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:43.130990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8893d4-b77f-441e-8470-8083cbbf284e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:43.131607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8893d4-b77f-441e-8470-8083cbbf284e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:43.132054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8893d4-b77f-441e-8470-8083cbbf284e admin admin] PATH_INFO: `/v3/projects/4714a8f6f9194d399961603f33749dfc` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:43.143359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8893d4-b77f-441e-8470-8083cbbf284e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4714a8f6f9194d399961603f33749dfc)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:43.145598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8893d4-b77f-441e-8470-8083cbbf284e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:43.168321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:43.207752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-da8893d4-b77f-441e-8470-8083cbbf284e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4714a8f6f9194d399961603f33749dfc', 'request_id': 'req-da8893d4-b77f-441e-8470-8083cbbf284e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:43.218289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-da8893d4-b77f-441e-8470-8083cbbf284e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4714a8f6f9194d399961603f33749dfc', 'request_id': 'req-da8893d4-b77f-441e-8470-8083cbbf284e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:43.227208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-da8893d4-b77f-441e-8470-8083cbbf284e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4714a8f6f9194d399961603f33749dfc', 'request_id': 'req-da8893d4-b77f-441e-8470-8083cbbf284e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:43.239366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-da8893d4-b77f-441e-8470-8083cbbf284e admin admin] The token cache is being invalidate because project 4714a8f6f9194d399961603f33749dfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:43.239728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-da8893d4-b77f-441e-8470-8083cbbf284e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:43.240409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-da8893d4-b77f-441e-8470-8083cbbf284e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:43.242131 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3219/6499] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:43 2022] DELETE /identity/v3/projects/4714a8f6f9194d399961603f33749dfc => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:43.246244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c2ede-7e8a-4a69-9748-8fc9c5ac6799 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:43.257173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c2ede-7e8a-4a69-9748-8fc9c5ac6799 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:43.321564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49157de-2184-4cfb-b66e-646466b00823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:43.322950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49157de-2184-4cfb-b66e-646466b00823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:43.323333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49157de-2184-4cfb-b66e-646466b00823 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:43.323715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49157de-2184-4cfb-b66e-646466b00823 admin admin] PATH_INFO: `/v3/users/f4982c8f32c84971a10543c3fc74d23f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:43.325072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49157de-2184-4cfb-b66e-646466b00823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4982c8f32c84971a10543c3fc74d23f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:43.327211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49157de-2184-4cfb-b66e-646466b00823 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:43.329936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:43.402170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f9bdb-5b99-474d-a1c0-7280a9b86e0e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:43.402624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f9bdb-5b99-474d-a1c0-7280a9b86e0e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:43.402785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f9bdb-5b99-474d-a1c0-7280a9b86e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:43.417227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b49157de-2184-4cfb-b66e-646466b00823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4982c8f32c84971a10543c3fc74d23f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:43.430300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b49157de-2184-4cfb-b66e-646466b00823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4982c8f32c84971a10543c3fc74d23f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:43.431699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1f3f9bdb-5b99-474d-a1c0-7280a9b86e0e None None] MFA Rules not processed for user `a2745d6923e347939e2c67015aefadda`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:43.433148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1f3f9bdb-5b99-474d-a1c0-7280a9b86e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:43.469027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b49157de-2184-4cfb-b66e-646466b00823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4982c8f32c84971a10543c3fc74d23f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:43.469739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b49157de-2184-4cfb-b66e-646466b00823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4982c8f32c84971a10543c3fc74d23f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:43.482101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b49157de-2184-4cfb-b66e-646466b00823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4982c8f32c84971a10543c3fc74d23f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:43.490564 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3281/6500] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:43 2022] DELETE /identity/v3/users/f4982c8f32c84971a10543c3fc74d23f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:43.551211 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3220/6501] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:44.085469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49157de-2184-4cfb-b66e-646466b00823 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:44.087144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b49157de-2184-4cfb-b66e-646466b00823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:44.179723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49157de-2184-4cfb-b66e-646466b00823 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:44.186869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8fac07-5e40-4af7-82f0-24ca79faf98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:44.187730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fac07-5e40-4af7-82f0-24ca79faf98e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:44.188228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fac07-5e40-4af7-82f0-24ca79faf98e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:44.188652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fac07-5e40-4af7-82f0-24ca79faf98e admin admin] PATH_INFO: `/v3/projects/abdd7ccdaec14ed78b14f5e14b29cfca` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:44.190457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8fac07-5e40-4af7-82f0-24ca79faf98e admin admin] RBAC: Authorizing `identity:delete_project(project_id=abdd7ccdaec14ed78b14f5e14b29cfca)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:44.193526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8fac07-5e40-4af7-82f0-24ca79faf98e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:44.251393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac8fac07-5e40-4af7-82f0-24ca79faf98e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abdd7ccdaec14ed78b14f5e14b29cfca', 'request_id': 'req-ac8fac07-5e40-4af7-82f0-24ca79faf98e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:44.258909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac8fac07-5e40-4af7-82f0-24ca79faf98e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abdd7ccdaec14ed78b14f5e14b29cfca', 'request_id': 'req-ac8fac07-5e40-4af7-82f0-24ca79faf98e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:44.266410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac8fac07-5e40-4af7-82f0-24ca79faf98e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abdd7ccdaec14ed78b14f5e14b29cfca', 'request_id': 'req-ac8fac07-5e40-4af7-82f0-24ca79faf98e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:44.283243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac8fac07-5e40-4af7-82f0-24ca79faf98e admin admin] The token cache is being invalidate because project abdd7ccdaec14ed78b14f5e14b29cfca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:44.283623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac8fac07-5e40-4af7-82f0-24ca79faf98e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:44.284575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac8fac07-5e40-4af7-82f0-24ca79faf98e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:44.286148 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3282/6502] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:44 2022] DELETE /identity/v3/projects/abdd7ccdaec14ed78b14f5e14b29cfca => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:44.291704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df1aec-d3a8-402d-9a0d-8835d34aba4f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:44.292176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df1aec-d3a8-402d-9a0d-8835d34aba4f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:44.292295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df1aec-d3a8-402d-9a0d-8835d34aba4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:44.316223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-35df1aec-d3a8-402d-9a0d-8835d34aba4f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:44.317532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-35df1aec-d3a8-402d-9a0d-8835d34aba4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:44.431579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8bc43-79a7-4d63-a816-d830affe7377 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:44.431899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8bc43-79a7-4d63-a816-d830affe7377 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:44.432698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8bc43-79a7-4d63-a816-d830affe7377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:44.434671 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3221/6503] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:44.460691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c5b8bc43-79a7-4d63-a816-d830affe7377 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:44.462020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c5b8bc43-79a7-4d63-a816-d830affe7377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:44.475633 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3283/6504] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:44.476494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df1aec-d3a8-402d-9a0d-8835d34aba4f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:44.487785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df1aec-d3a8-402d-9a0d-8835d34aba4f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:44.496649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59909427-9773-486a-b68b-a32909bfb12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:44.497477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59909427-9773-486a-b68b-a32909bfb12a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:44.498037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59909427-9773-486a-b68b-a32909bfb12a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:44.498478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59909427-9773-486a-b68b-a32909bfb12a admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:44.499441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59909427-9773-486a-b68b-a32909bfb12a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:13:44.499888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59909427-9773-486a-b68b-a32909bfb12a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:44.501571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59909427-9773-486a-b68b-a32909bfb12a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:44.517279 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3222/6505] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:13:44.521789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8bc43-79a7-4d63-a816-d830affe7377 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:44.528554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:44.537595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8bc43-79a7-4d63-a816-d830affe7377 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:44.544976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bdbbc-79b1-466a-80cd-6a3f085f7e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:44.545639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bdbbc-79b1-466a-80cd-6a3f085f7e88 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:44.545983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bdbbc-79b1-466a-80cd-6a3f085f7e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:44.546516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bdbbc-79b1-466a-80cd-6a3f085f7e88 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:44.547351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495bdbbc-79b1-466a-80cd-6a3f085f7e88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:44.549747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495bdbbc-79b1-466a-80cd-6a3f085f7e88 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:44.583488 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3284/6506] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:44 2022] POST /identity/v3/projects => generated 369 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:44.589758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59909427-9773-486a-b68b-a32909bfb12a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:44.600622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59909427-9773-486a-b68b-a32909bfb12a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:44.638049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:44.669151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faac74-8623-4ce7-a646-fdc411b8dc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:44.669779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faac74-8623-4ce7-a646-fdc411b8dc3f admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:44.670061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faac74-8623-4ce7-a646-fdc411b8dc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:44.672408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faac74-8623-4ce7-a646-fdc411b8dc3f admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:44.672408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56faac74-8623-4ce7-a646-fdc411b8dc3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:44.673064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56faac74-8623-4ce7-a646-fdc411b8dc3f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:44.706241 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3223/6507] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:44 2022] POST /identity/v3/users => generated 285 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:44.712216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bdbbc-79b1-466a-80cd-6a3f085f7e88 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:44.724477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bdbbc-79b1-466a-80cd-6a3f085f7e88 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:44.731556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025fe191-d7a6-44b8-8c67-5cd611430457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:44.732263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fe191-d7a6-44b8-8c67-5cd611430457 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:44.732621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fe191-d7a6-44b8-8c67-5cd611430457 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:44.732971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fe191-d7a6-44b8-8c67-5cd611430457 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:44.733767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025fe191-d7a6-44b8-8c67-5cd611430457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:44.735491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025fe191-d7a6-44b8-8c67-5cd611430457 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:44.758022 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3285/6508] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:13:44.759447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faac74-8623-4ce7-a646-fdc411b8dc3f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:44.773483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faac74-8623-4ce7-a646-fdc411b8dc3f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:44.781779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b93f5-9480-4385-93bd-9f8100105d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:44.782625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b93f5-9480-4385-93bd-9f8100105d30 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:44.783410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b93f5-9480-4385-93bd-9f8100105d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:44.783791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b93f5-9480-4385-93bd-9f8100105d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:44.785415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b93f5-9480-4385-93bd-9f8100105d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:44.787320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b93f5-9480-4385-93bd-9f8100105d30 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:44.800897 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3224/6509] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:44.805852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025fe191-d7a6-44b8-8c67-5cd611430457 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:44.818477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025fe191-d7a6-44b8-8c67-5cd611430457 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:44.824167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f5c73-644d-42e0-a9e9-659a0feb5cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:44.824854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f5c73-644d-42e0-a9e9-659a0feb5cc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:44.825485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f5c73-644d-42e0-a9e9-659a0feb5cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:44.826049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f5c73-644d-42e0-a9e9-659a0feb5cc5 admin admin] PATH_INFO: `/v3/projects/5946f2c858e54d42ab4f184fd2f4d87c/users/871645dc61df4f818082b21626ff2eb1/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:44.847177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f5c73-644d-42e0-a9e9-659a0feb5cc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5946f2c858e54d42ab4f184fd2f4d87c, user_id=871645dc61df4f818082b21626ff2eb1, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:44.851557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f5c73-644d-42e0-a9e9-659a0feb5cc5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:44.874700 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3286/6510] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:44 2022] PUT /identity/v3/projects/5946f2c858e54d42ab4f184fd2f4d87c/users/871645dc61df4f818082b21626ff2eb1/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:13:44.887150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6b556-79ba-4a2c-8dbf-4afe94774162 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:44.887796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6b556-79ba-4a2c-8dbf-4afe94774162 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:44.888676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6b556-79ba-4a2c-8dbf-4afe94774162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:44.926532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8fe6b556-79ba-4a2c-8dbf-4afe94774162 None None] MFA Rules not processed for user `871645dc61df4f818082b21626ff2eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:44.928037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8fe6b556-79ba-4a2c-8dbf-4afe94774162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:44.942899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f5c73-644d-42e0-a9e9-659a0feb5cc5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:44.945144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-166f5c73-644d-42e0-a9e9-659a0feb5cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:45.063501 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3225/6511] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:45.069417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6b556-79ba-4a2c-8dbf-4afe94774162 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:45.153874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f5c73-644d-42e0-a9e9-659a0feb5cc5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:45.160219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef3258-057a-4fe5-8121-98962a664a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:45.161512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef3258-057a-4fe5-8121-98962a664a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:45.162009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef3258-057a-4fe5-8121-98962a664a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:45.162357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef3258-057a-4fe5-8121-98962a664a09 admin admin] PATH_INFO: `/v3/users/6e8b32fe37d342c2911d6e1ede9f37ee` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:45.166355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef3258-057a-4fe5-8121-98962a664a09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e8b32fe37d342c2911d6e1ede9f37ee)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:45.169407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef3258-057a-4fe5-8121-98962a664a09 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:45.211851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6b556-79ba-4a2c-8dbf-4afe94774162 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:45.243181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a934bb6-ea64-4c45-bb14-aab8610cdfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:45.246002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a934bb6-ea64-4c45-bb14-aab8610cdfbb admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:45.246324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a934bb6-ea64-4c45-bb14-aab8610cdfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:45.246795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a934bb6-ea64-4c45-bb14-aab8610cdfbb admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:45.247668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a934bb6-ea64-4c45-bb14-aab8610cdfbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:45.250400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a934bb6-ea64-4c45-bb14-aab8610cdfbb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:45.296822 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3226/6512] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:45 2022] POST /identity/v3/projects => generated 367 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:45.301456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a934bb6-ea64-4c45-bb14-aab8610cdfbb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:45.335173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b8ef3258-057a-4fe5-8121-98962a664a09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e8b32fe37d342c2911d6e1ede9f37ee'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:45.354896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:45.369499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b8ef3258-057a-4fe5-8121-98962a664a09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e8b32fe37d342c2911d6e1ede9f37ee'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:45.400909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b8ef3258-057a-4fe5-8121-98962a664a09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e8b32fe37d342c2911d6e1ede9f37ee'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:45.403049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b8ef3258-057a-4fe5-8121-98962a664a09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e8b32fe37d342c2911d6e1ede9f37ee'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:45.406984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a934bb6-ea64-4c45-bb14-aab8610cdfbb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:45.420737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b8ef3258-057a-4fe5-8121-98962a664a09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e8b32fe37d342c2911d6e1ede9f37ee'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:45.424344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f5fdf-b261-4cf7-8d3c-bce103bed8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:45.425445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f5fdf-b261-4cf7-8d3c-bce103bed8d9 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:45.425987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f5fdf-b261-4cf7-8d3c-bce103bed8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:45.426615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f5fdf-b261-4cf7-8d3c-bce103bed8d9 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:45.427880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f5fdf-b261-4cf7-8d3c-bce103bed8d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:45.429974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f5fdf-b261-4cf7-8d3c-bce103bed8d9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:45.432245 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3287/6513] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:44 2022] DELETE /identity/v3/users/6e8b32fe37d342c2911d6e1ede9f37ee => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:45.436975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea329bb2-6a0f-4947-a9a0-25d1d3187cc1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:45.437385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea329bb2-6a0f-4947-a9a0-25d1d3187cc1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:45.437778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea329bb2-6a0f-4947-a9a0-25d1d3187cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:45.461797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ea329bb2-6a0f-4947-a9a0-25d1d3187cc1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:45.465004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ea329bb2-6a0f-4947-a9a0-25d1d3187cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:45.493226 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3227/6514] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:45 2022] POST /identity/v3/users => generated 290 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:45.495595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f5fdf-b261-4cf7-8d3c-bce103bed8d9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:45.496775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:45.498748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ed6f5fdf-b261-4cf7-8d3c-bce103bed8d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:45.521880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f5fdf-b261-4cf7-8d3c-bce103bed8d9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:45.577813 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3288/6515] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:45.587226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea329bb2-6a0f-4947-a9a0-25d1d3187cc1 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:45.590301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ea329bb2-6a0f-4947-a9a0-25d1d3187cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:45.609512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24396472-ab87-4f4f-960e-c22f4860e9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:45.610769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24396472-ab87-4f4f-960e-c22f4860e9e4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:45.611154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24396472-ab87-4f4f-960e-c22f4860e9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:45.611437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24396472-ab87-4f4f-960e-c22f4860e9e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:45.612772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24396472-ab87-4f4f-960e-c22f4860e9e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:45.614463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24396472-ab87-4f4f-960e-c22f4860e9e4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:45.627026 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3228/6516] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:45.631373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24396472-ab87-4f4f-960e-c22f4860e9e4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:45.663440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24396472-ab87-4f4f-960e-c22f4860e9e4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:45.679671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20af161-9a97-4348-b478-e1315bae10be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:45.681426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20af161-9a97-4348-b478-e1315bae10be admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:45.681722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20af161-9a97-4348-b478-e1315bae10be admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:45.682008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20af161-9a97-4348-b478-e1315bae10be admin admin] PATH_INFO: `/v3/projects/cd1c8aa3b1794a56b9120a7655d138b7/users/42c2d21ecbfb4378a6ea22b3d7d294d9/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:45.729470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20af161-9a97-4348-b478-e1315bae10be admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd1c8aa3b1794a56b9120a7655d138b7, user_id=42c2d21ecbfb4378a6ea22b3d7d294d9, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:45.732599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20af161-9a97-4348-b478-e1315bae10be admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:45.772561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea329bb2-6a0f-4947-a9a0-25d1d3187cc1 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:45.791953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0403623-5770-4b02-a24c-14d65db777c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:45.792583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0403623-5770-4b02-a24c-14d65db777c6 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:45.792864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0403623-5770-4b02-a24c-14d65db777c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:45.793138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0403623-5770-4b02-a24c-14d65db777c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:45.811995 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3229/6517] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:45 2022] PUT /identity/v3/projects/cd1c8aa3b1794a56b9120a7655d138b7/users/42c2d21ecbfb4378a6ea22b3d7d294d9/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:45.812957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20af161-9a97-4348-b478-e1315bae10be admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:45.837441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0403623-5770-4b02-a24c-14d65db777c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:45.839847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0403623-5770-4b02-a24c-14d65db777c6 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:45.861381 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3289/6518] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:45.962063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20af161-9a97-4348-b478-e1315bae10be admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:45.973483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e4342-6e6e-43e2-a7c4-f461f780a047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:45.974892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e4342-6e6e-43e2-a7c4-f461f780a047 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:45.975600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e4342-6e6e-43e2-a7c4-f461f780a047 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:45.976120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e4342-6e6e-43e2-a7c4-f461f780a047 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:45.977132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e4342-6e6e-43e2-a7c4-f461f780a047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:45.980007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e4342-6e6e-43e2-a7c4-f461f780a047 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:46.004337 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3230/6519] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:46.008307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0403623-5770-4b02-a24c-14d65db777c6 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:46.038017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0403623-5770-4b02-a24c-14d65db777c6 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:46.066551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc92ac9-bd92-41d4-84f6-ea3b16fa6e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:46.067453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc92ac9-bd92-41d4-84f6-ea3b16fa6e3a admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:46.067453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc92ac9-bd92-41d4-84f6-ea3b16fa6e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:46.067834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc92ac9-bd92-41d4-84f6-ea3b16fa6e3a admin admin] PATH_INFO: `/v3/projects/cd1c8aa3b1794a56b9120a7655d138b7/users/42c2d21ecbfb4378a6ea22b3d7d294d9/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:46.074679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc92ac9-bd92-41d4-84f6-ea3b16fa6e3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd1c8aa3b1794a56b9120a7655d138b7, user_id=42c2d21ecbfb4378a6ea22b3d7d294d9, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:46.083148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc92ac9-bd92-41d4-84f6-ea3b16fa6e3a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:46.117067 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3290/6520] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:46 2022] PUT /identity/v3/projects/cd1c8aa3b1794a56b9120a7655d138b7/users/42c2d21ecbfb4378a6ea22b3d7d294d9/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:46.123563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e4411-79ad-44e0-92d1-eab0dbd92db9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:46.125997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e4411-79ad-44e0-92d1-eab0dbd92db9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:46.126816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e4411-79ad-44e0-92d1-eab0dbd92db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:46.176116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-088e4411-79ad-44e0-92d1-eab0dbd92db9 None None] MFA Rules not processed for user `42c2d21ecbfb4378a6ea22b3d7d294d9`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:46.178163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-088e4411-79ad-44e0-92d1-eab0dbd92db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:46.196795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:46.278802 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3231/6521] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:46.283188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc92ac9-bd92-41d4-84f6-ea3b16fa6e3a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:46.372553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc92ac9-bd92-41d4-84f6-ea3b16fa6e3a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:46.385614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2348b0-9abb-41d8-9688-eb56c35db9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:46.386396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2348b0-9abb-41d8-9688-eb56c35db9d4 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:46.386845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2348b0-9abb-41d8-9688-eb56c35db9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:46.387341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2348b0-9abb-41d8-9688-eb56c35db9d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:46.388515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2348b0-9abb-41d8-9688-eb56c35db9d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:46.391284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2348b0-9abb-41d8-9688-eb56c35db9d4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:46.440868 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3291/6522] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:46 2022] POST /identity/v3/projects => generated 369 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:13:46.443474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e4411-79ad-44e0-92d1-eab0dbd92db9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:46.462736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e4411-79ad-44e0-92d1-eab0dbd92db9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:46.560192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a8ba6-ec03-4eaa-b5c6-8d2e66010c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:46.560623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a8ba6-ec03-4eaa-b5c6-8d2e66010c19 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:46.561034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a8ba6-ec03-4eaa-b5c6-8d2e66010c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:46.561314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a8ba6-ec03-4eaa-b5c6-8d2e66010c19 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:46.561981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a8ba6-ec03-4eaa-b5c6-8d2e66010c19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:46.566435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2348b0-9abb-41d8-9688-eb56c35db9d4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:46.566741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a8ba6-ec03-4eaa-b5c6-8d2e66010c19 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:46.568793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9c2348b0-9abb-41d8-9688-eb56c35db9d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:46.591463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2348b0-9abb-41d8-9688-eb56c35db9d4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:46.597026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff9c5a-d2ae-4518-ace4-602e3e2eade1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:46.597739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff9c5a-d2ae-4518-ace4-602e3e2eade1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:46.597985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff9c5a-d2ae-4518-ace4-602e3e2eade1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:46.598340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff9c5a-d2ae-4518-ace4-602e3e2eade1 admin admin] PATH_INFO: `/v3/projects/424cbcca79d947f7a629d36a2cc90847` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:46.599628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff9c5a-d2ae-4518-ace4-602e3e2eade1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=424cbcca79d947f7a629d36a2cc90847)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:46.602301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff9c5a-d2ae-4518-ace4-602e3e2eade1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:46.608774 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3232/6523] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:46 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:46.613935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a8ba6-ec03-4eaa-b5c6-8d2e66010c19 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:46.628667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a8ba6-ec03-4eaa-b5c6-8d2e66010c19 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:46.635355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d0a88-52ea-4ce2-ab26-e4c193a30e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:46.635993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d0a88-52ea-4ce2-ab26-e4c193a30e89 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:46.636651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d0a88-52ea-4ce2-ab26-e4c193a30e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:46.637100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d0a88-52ea-4ce2-ab26-e4c193a30e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:46.637999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d0a88-52ea-4ce2-ab26-e4c193a30e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:46.640847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d0a88-52ea-4ce2-ab26-e4c193a30e89 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:46.650992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:46.656594 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3233/6524] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:46.658286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d0a88-52ea-4ce2-ab26-e4c193a30e89 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:46.665653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-85ff9c5a-d2ae-4518-ace4-602e3e2eade1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '424cbcca79d947f7a629d36a2cc90847', 'request_id': 'req-85ff9c5a-d2ae-4518-ace4-602e3e2eade1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:46.671143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d0a88-52ea-4ce2-ab26-e4c193a30e89 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:46.678541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-85ff9c5a-d2ae-4518-ace4-602e3e2eade1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '424cbcca79d947f7a629d36a2cc90847', 'request_id': 'req-85ff9c5a-d2ae-4518-ace4-602e3e2eade1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:46.684579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-85ff9c5a-d2ae-4518-ace4-602e3e2eade1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '424cbcca79d947f7a629d36a2cc90847', 'request_id': 'req-85ff9c5a-d2ae-4518-ace4-602e3e2eade1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:46.701339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-85ff9c5a-d2ae-4518-ace4-602e3e2eade1 admin admin] The token cache is being invalidate because project 424cbcca79d947f7a629d36a2cc90847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:46.701618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-85ff9c5a-d2ae-4518-ace4-602e3e2eade1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:46.702660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-85ff9c5a-d2ae-4518-ace4-602e3e2eade1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:46.704475 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3292/6525] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:46 2022] DELETE /identity/v3/projects/424cbcca79d947f7a629d36a2cc90847 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:13:46.708927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b4216-b3de-46bd-b74f-dcb41c5bf8b6 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:46.709256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b4216-b3de-46bd-b74f-dcb41c5bf8b6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:46.709481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b4216-b3de-46bd-b74f-dcb41c5bf8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:46.732295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c74b4216-b3de-46bd-b74f-dcb41c5bf8b6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:46.733806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c74b4216-b3de-46bd-b74f-dcb41c5bf8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:46.751187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e054f4-b814-43ad-bfde-3985432b3b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:46.751920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e054f4-b814-43ad-bfde-3985432b3b94 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:46.752264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e054f4-b814-43ad-bfde-3985432b3b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:46.752541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e054f4-b814-43ad-bfde-3985432b3b94 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:46.753187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e054f4-b814-43ad-bfde-3985432b3b94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:46.754845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e054f4-b814-43ad-bfde-3985432b3b94 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:46.765668 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3234/6526] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:46.769807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e054f4-b814-43ad-bfde-3985432b3b94 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:46.771604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d9e054f4-b814-43ad-bfde-3985432b3b94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:46.785253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e054f4-b814-43ad-bfde-3985432b3b94 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:46.791268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5419f3d-a286-4385-949e-7e76464a2963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:46.791946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5419f3d-a286-4385-949e-7e76464a2963 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:46.792232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5419f3d-a286-4385-949e-7e76464a2963 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:46.792492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5419f3d-a286-4385-949e-7e76464a2963 admin admin] PATH_INFO: `/v3/projects/e08ef2b377884a51a7ef815ddef1be8c/users/eb153aa122af4ab9a652798bee5516ee/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:46.811188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5419f3d-a286-4385-949e-7e76464a2963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e08ef2b377884a51a7ef815ddef1be8c, user_id=eb153aa122af4ab9a652798bee5516ee, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:46.813766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5419f3d-a286-4385-949e-7e76464a2963 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:46.818347 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3293/6527] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:46.827007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35386f44-7a3d-4b7d-8d02-32dad87273d3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:46.827591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35386f44-7a3d-4b7d-8d02-32dad87273d3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:46.835353 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3235/6528] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:46 2022] PUT /identity/v3/projects/e08ef2b377884a51a7ef815ddef1be8c/users/eb153aa122af4ab9a652798bee5516ee/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:46.839803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35386f44-7a3d-4b7d-8d02-32dad87273d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:46.845851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be4128-94e7-4768-ae0e-04c0e69a9890 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:46.846136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be4128-94e7-4768-ae0e-04c0e69a9890 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:46.846410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be4128-94e7-4768-ae0e-04c0e69a9890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:46.854101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:46.871704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-35386f44-7a3d-4b7d-8d02-32dad87273d3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:46.872961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-35386f44-7a3d-4b7d-8d02-32dad87273d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:46.879895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-85be4128-94e7-4768-ae0e-04c0e69a9890 None None] MFA Rules not processed for user `eb153aa122af4ab9a652798bee5516ee`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:46.881114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-85be4128-94e7-4768-ae0e-04c0e69a9890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:46.959636 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3294/6529] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:46.963706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35386f44-7a3d-4b7d-8d02-32dad87273d3 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:46.975367 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3236/6530] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:46.980683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be4128-94e7-4768-ae0e-04c0e69a9890 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:46.982225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35386f44-7a3d-4b7d-8d02-32dad87273d3 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:46.982376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-85be4128-94e7-4768-ae0e-04c0e69a9890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:46.988161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14764434-132d-4db2-8e2d-e641d3c0aae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:46.988784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14764434-132d-4db2-8e2d-e641d3c0aae5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:46.989067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14764434-132d-4db2-8e2d-e641d3c0aae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:46.989348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14764434-132d-4db2-8e2d-e641d3c0aae5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:46.990013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14764434-132d-4db2-8e2d-e641d3c0aae5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:13:46.990466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14764434-132d-4db2-8e2d-e641d3c0aae5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:46.992303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14764434-132d-4db2-8e2d-e641d3c0aae5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:47.008303 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3295/6531] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:13:47.009893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14764434-132d-4db2-8e2d-e641d3c0aae5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:47.011963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-14764434-132d-4db2-8e2d-e641d3c0aae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:47.027598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14764434-132d-4db2-8e2d-e641d3c0aae5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:47.041518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629e55e-1e01-4417-8aca-c381838ab5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:47.042162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629e55e-1e01-4417-8aca-c381838ab5fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:47.042439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629e55e-1e01-4417-8aca-c381838ab5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:47.042767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629e55e-1e01-4417-8aca-c381838ab5fb admin admin] PATH_INFO: `/v3/users/5b53ea073e824fc5ba6583298c8ad9c8` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:47.044365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629e55e-1e01-4417-8aca-c381838ab5fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b53ea073e824fc5ba6583298c8ad9c8)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:47.046278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629e55e-1e01-4417-8aca-c381838ab5fb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:47.078349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be4128-94e7-4768-ae0e-04c0e69a9890 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:47.089303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6b54a-4424-43dc-92ea-0ff1fb39c503 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:47.091538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6b54a-4424-43dc-92ea-0ff1fb39c503 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:47.092019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6b54a-4424-43dc-92ea-0ff1fb39c503 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:47.092783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6b54a-4424-43dc-92ea-0ff1fb39c503 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:47.093014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6b54a-4424-43dc-92ea-0ff1fb39c503 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:47.095098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6b54a-4424-43dc-92ea-0ff1fb39c503 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:47.122615 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3237/6532] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:13:46 2022] POST /identity/v3/domains => generated 277 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:13:47.123815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6b54a-4424-43dc-92ea-0ff1fb39c503 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:47.140611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b629e55e-1e01-4417-8aca-c381838ab5fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b53ea073e824fc5ba6583298c8ad9c8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:47.142614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:47.160171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b629e55e-1e01-4417-8aca-c381838ab5fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b53ea073e824fc5ba6583298c8ad9c8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:47.189884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b629e55e-1e01-4417-8aca-c381838ab5fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b53ea073e824fc5ba6583298c8ad9c8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:47.191119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b629e55e-1e01-4417-8aca-c381838ab5fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b53ea073e824fc5ba6583298c8ad9c8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:47.213474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b629e55e-1e01-4417-8aca-c381838ab5fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b53ea073e824fc5ba6583298c8ad9c8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:47.229643 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3296/6533] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:47 2022] DELETE /identity/v3/users/5b53ea073e824fc5ba6583298c8ad9c8 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:47.232164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629e55e-1e01-4417-8aca-c381838ab5fb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:47.235746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b629e55e-1e01-4417-8aca-c381838ab5fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:47.263131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6b54a-4424-43dc-92ea-0ff1fb39c503 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:47.311327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:47.357939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629e55e-1e01-4417-8aca-c381838ab5fb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:47.364615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c73d9-dfe8-421c-abee-39df28a58014 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:47.365568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c73d9-dfe8-421c-abee-39df28a58014 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:47.365968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c73d9-dfe8-421c-abee-39df28a58014 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:47.366350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c73d9-dfe8-421c-abee-39df28a58014 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:47.367227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c73d9-dfe8-421c-abee-39df28a58014 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:47.369583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c73d9-dfe8-421c-abee-39df28a58014 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:47.369811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3af800-6d95-4950-a15d-b4f922a95bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:47.370423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3af800-6d95-4950-a15d-b4f922a95bfa admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:47.370817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3af800-6d95-4950-a15d-b4f922a95bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:47.371096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3af800-6d95-4950-a15d-b4f922a95bfa admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:47.372456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3af800-6d95-4950-a15d-b4f922a95bfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:47.375041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3af800-6d95-4950-a15d-b4f922a95bfa admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:47.398343 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3297/6534] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:13:47 2022] POST /identity/v3/roles => generated 238 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:47.399863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663bf86-f4ed-400b-932f-3047ff1b3d14 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:47.400412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663bf86-f4ed-400b-932f-3047ff1b3d14 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:47.400750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663bf86-f4ed-400b-932f-3047ff1b3d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:47.422984 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3238/6535] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:47 2022] POST /identity/v3/projects => generated 383 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:47.425506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3af800-6d95-4950-a15d-b4f922a95bfa admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:47.433606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2663bf86-f4ed-400b-932f-3047ff1b3d14 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:47.435478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2663bf86-f4ed-400b-932f-3047ff1b3d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:47.506224 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3298/6536] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:47.510545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663bf86-f4ed-400b-932f-3047ff1b3d14 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:47.512087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2663bf86-f4ed-400b-932f-3047ff1b3d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:47.516063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3af800-6d95-4950-a15d-b4f922a95bfa admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:47.522617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30613f5-bc7b-4663-80d7-b08099a70a8f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:47.523513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30613f5-bc7b-4663-80d7-b08099a70a8f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:47.523962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30613f5-bc7b-4663-80d7-b08099a70a8f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:47.524264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663bf86-f4ed-400b-932f-3047ff1b3d14 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:47.524670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30613f5-bc7b-4663-80d7-b08099a70a8f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains/cb0cf8cf2a9c43cd80a9746768468584/users/eb153aa122af4ab9a652798bee5516ee/roles/f30f45eafbdd4adc8ea84540d367735d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:47.531751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37464546-24e2-43fe-b86a-6db8cbf8c99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:47.532824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37464546-24e2-43fe-b86a-6db8cbf8c99f admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:47.533293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37464546-24e2-43fe-b86a-6db8cbf8c99f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:47.533856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37464546-24e2-43fe-b86a-6db8cbf8c99f admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:47.534686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37464546-24e2-43fe-b86a-6db8cbf8c99f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:47.537063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37464546-24e2-43fe-b86a-6db8cbf8c99f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:47.542990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30613f5-bc7b-4663-80d7-b08099a70a8f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=cb0cf8cf2a9c43cd80a9746768468584, user_id=eb153aa122af4ab9a652798bee5516ee, role_id=f30f45eafbdd4adc8ea84540d367735d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:47.545672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30613f5-bc7b-4663-80d7-b08099a70a8f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:47.562333 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3239/6537] 10.222.0.17 () {60 vars in 1480 bytes} [Sat Oct 1 21:13:47 2022] PUT /identity/v3/domains/cb0cf8cf2a9c43cd80a9746768468584/users/eb153aa122af4ab9a652798bee5516ee/roles/f30f45eafbdd4adc8ea84540d367735d => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:47.566239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30613f5-bc7b-4663-80d7-b08099a70a8f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:47.568338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c30613f5-bc7b-4663-80d7-b08099a70a8f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:47.572385 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3299/6538] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:47 2022] POST /identity/v3/users => generated 292 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:47.573388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37464546-24e2-43fe-b86a-6db8cbf8c99f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:47.588564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37464546-24e2-43fe-b86a-6db8cbf8c99f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:47.652647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30613f5-bc7b-4663-80d7-b08099a70a8f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:47.657827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f19533-060b-4df1-bd4d-b10a72835ba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:47.658614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f19533-060b-4df1-bd4d-b10a72835ba8 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:47.658957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f19533-060b-4df1-bd4d-b10a72835ba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:47.659755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f19533-060b-4df1-bd4d-b10a72835ba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:47.673377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:47.718776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed530876-a2fc-4b5a-a0a7-3eb03f6c62f9 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:47.719409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed530876-a2fc-4b5a-a0a7-3eb03f6c62f9 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:47.719693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed530876-a2fc-4b5a-a0a7-3eb03f6c62f9 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:47.719972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed530876-a2fc-4b5a-a0a7-3eb03f6c62f9 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:47.720900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed530876-a2fc-4b5a-a0a7-3eb03f6c62f9 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:47.727492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed530876-a2fc-4b5a-a0a7-3eb03f6c62f9 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:47.760763 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3300/6539] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:13:47 2022] POST /identity/v3/domains => generated 275 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:47.761652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed530876-a2fc-4b5a-a0a7-3eb03f6c62f9 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:47.832394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:47.845633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f19533-060b-4df1-bd4d-b10a72835ba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:47.847253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f19533-060b-4df1-bd4d-b10a72835ba8 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:47.849918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed530876-a2fc-4b5a-a0a7-3eb03f6c62f9 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:47.856560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df00c5f-7beb-4323-b684-ad546362bba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:47.857942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df00c5f-7beb-4323-b684-ad546362bba6 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:47.858433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df00c5f-7beb-4323-b684-ad546362bba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:47.858944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df00c5f-7beb-4323-b684-ad546362bba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:47.860047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df00c5f-7beb-4323-b684-ad546362bba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:47.862611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df00c5f-7beb-4323-b684-ad546362bba6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:47.877991 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3301/6540] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:47 2022] GET /identity/v3/roles => generated 1649 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:47.879574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df00c5f-7beb-4323-b684-ad546362bba6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:47.885870 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3240/6541] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:47.890810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f19533-060b-4df1-bd4d-b10a72835ba8 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:47.919274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f19533-060b-4df1-bd4d-b10a72835ba8 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:47.934780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5c8d6-a55e-4965-9d6d-adafe0f8f71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:47.935767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5c8d6-a55e-4965-9d6d-adafe0f8f71c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:47.936055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5c8d6-a55e-4965-9d6d-adafe0f8f71c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:47.937374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5c8d6-a55e-4965-9d6d-adafe0f8f71c admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:47.939731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5c8d6-a55e-4965-9d6d-adafe0f8f71c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:47.941428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5c8d6-a55e-4965-9d6d-adafe0f8f71c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:47.968397 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3241/6542] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:47 2022] GET /identity/v3/roles => generated 1649 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:47.969698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5c8d6-a55e-4965-9d6d-adafe0f8f71c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:47.981991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5c8d6-a55e-4965-9d6d-adafe0f8f71c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:47.987179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f81c63-6d1c-4972-a261-8e8746ade81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:47.988208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f81c63-6d1c-4972-a261-8e8746ade81b admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:47.988775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f81c63-6d1c-4972-a261-8e8746ade81b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:47.989124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f81c63-6d1c-4972-a261-8e8746ade81b admin admin] PATH_INFO: `/v3/projects/06d9722f2159421eac66bffde6d5a901/users/5eab4b929e0f4c0db0a5e48245d3df4b/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:47.991186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df00c5f-7beb-4323-b684-ad546362bba6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:47.998138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cdbf2-6590-4eb2-870c-0abc32d163ab tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:47.999016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cdbf2-6590-4eb2-870c-0abc32d163ab tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:47.999213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cdbf2-6590-4eb2-870c-0abc32d163ab tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:47.999470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cdbf2-6590-4eb2-870c-0abc32d163ab tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:48.000139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815cdbf2-6590-4eb2-870c-0abc32d163ab tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:48.001809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815cdbf2-6590-4eb2-870c-0abc32d163ab tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:48.009809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f81c63-6d1c-4972-a261-8e8746ade81b admin admin] RBAC: Authorizing `identity:create_grant(project_id=06d9722f2159421eac66bffde6d5a901, user_id=5eab4b929e0f4c0db0a5e48245d3df4b, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:48.013296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f81c63-6d1c-4972-a261-8e8746ade81b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:48.033796 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3302/6543] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:13:47 2022] POST /identity/v3/roles => generated 239 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:48.038181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cdbf2-6590-4eb2-870c-0abc32d163ab tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:48.044773 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3242/6544] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:47 2022] PUT /identity/v3/projects/06d9722f2159421eac66bffde6d5a901/users/5eab4b929e0f4c0db0a5e48245d3df4b/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:48.062143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f9f70-363b-4cab-bcf0-f5760c7c5149 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:48.062526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f9f70-363b-4cab-bcf0-f5760c7c5149 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:48.064132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f9f70-363b-4cab-bcf0-f5760c7c5149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:48.083112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cdbf2-6590-4eb2-870c-0abc32d163ab tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:48.127627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-164f9f70-363b-4cab-bcf0-f5760c7c5149 None None] MFA Rules not processed for user `5eab4b929e0f4c0db0a5e48245d3df4b`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:48.129641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-164f9f70-363b-4cab-bcf0-f5760c7c5149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:48.179456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e12b3c-3996-410c-aa77-6da5c65ed4e8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:48.180099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e12b3c-3996-410c-aa77-6da5c65ed4e8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:48.180381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e12b3c-3996-410c-aa77-6da5c65ed4e8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:48.180657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e12b3c-3996-410c-aa77-6da5c65ed4e8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/groups` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:48.181819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e12b3c-3996-410c-aa77-6da5c65ed4e8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:48.183760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e12b3c-3996-410c-aa77-6da5c65ed4e8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:48.195573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:48.219515 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3303/6545] 10.222.0.17 () {60 vars in 1147 bytes} [Sat Oct 1 21:13:48 2022] POST /identity/v3/groups => generated 326 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:48.220567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e12b3c-3996-410c-aa77-6da5c65ed4e8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:48.231818 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3243/6546] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:48.235736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f9f70-363b-4cab-bcf0-f5760c7c5149 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:48.244689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e12b3c-3996-410c-aa77-6da5c65ed4e8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:48.254051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd511c-c43c-4a38-8665-66348d194a1a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:48.255566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd511c-c43c-4a38-8665-66348d194a1a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:48.256235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd511c-c43c-4a38-8665-66348d194a1a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:48.256980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd511c-c43c-4a38-8665-66348d194a1a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/groups/ea531154be934235bdbd27a1930044fb/users/eb153aa122af4ab9a652798bee5516ee` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:48.264585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd511c-c43c-4a38-8665-66348d194a1a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=ea531154be934235bdbd27a1930044fb, user_id=eb153aa122af4ab9a652798bee5516ee)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:48.267986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd511c-c43c-4a38-8665-66348d194a1a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:48.308199 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3304/6547] 10.222.0.17 () {60 vars in 1360 bytes} [Sat Oct 1 21:13:48 2022] PUT /identity/v3/groups/ea531154be934235bdbd27a1930044fb/users/eb153aa122af4ab9a652798bee5516ee => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:48.311097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd511c-c43c-4a38-8665-66348d194a1a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:48.379841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f9f70-363b-4cab-bcf0-f5760c7c5149 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:48.401397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd511c-c43c-4a38-8665-66348d194a1a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:48.409036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e56e02-509f-4a68-93f7-69a9f7b87a0a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:48.410038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e56e02-509f-4a68-93f7-69a9f7b87a0a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:48.410335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e56e02-509f-4a68-93f7-69a9f7b87a0a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:48.410659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e56e02-509f-4a68-93f7-69a9f7b87a0a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains/134961015f02403fa8cc7397b10945f2/groups/ea531154be934235bdbd27a1930044fb/roles/1c3c664618dd4d9a9c7c737243a42528` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:48.413187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e56e02-509f-4a68-93f7-69a9f7b87a0a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=134961015f02403fa8cc7397b10945f2, group_id=ea531154be934235bdbd27a1930044fb, role_id=1c3c664618dd4d9a9c7c737243a42528)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:48.417089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e56e02-509f-4a68-93f7-69a9f7b87a0a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:48.448033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:48.449714 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3305/6548] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:48 2022] PUT /identity/v3/domains/134961015f02403fa8cc7397b10945f2/groups/ea531154be934235bdbd27a1930044fb/roles/1c3c664618dd4d9a9c7c737243a42528 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:48.451782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e56e02-509f-4a68-93f7-69a9f7b87a0a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:48.459033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c08171-7c10-4050-9b92-e38d59cb55fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:48.459869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c08171-7c10-4050-9b92-e38d59cb55fa admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:48.460277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c08171-7c10-4050-9b92-e38d59cb55fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:48.460665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c08171-7c10-4050-9b92-e38d59cb55fa admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:48.461390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c08171-7c10-4050-9b92-e38d59cb55fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:48.463776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c08171-7c10-4050-9b92-e38d59cb55fa admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:48.511744 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3244/6549] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:48 2022] POST /identity/v3/projects => generated 385 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:48.514152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c08171-7c10-4050-9b92-e38d59cb55fa admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:48.567143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e56e02-509f-4a68-93f7-69a9f7b87a0a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:48.629732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c08171-7c10-4050-9b92-e38d59cb55fa admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:48.637638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31d10a-4dc6-4f49-b615-41afc4359e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:48.639033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31d10a-4dc6-4f49-b615-41afc4359e5f admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:48.639325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31d10a-4dc6-4f49-b615-41afc4359e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:48.639602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31d10a-4dc6-4f49-b615-41afc4359e5f admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:48.640444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31d10a-4dc6-4f49-b615-41afc4359e5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:48.643200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31d10a-4dc6-4f49-b615-41afc4359e5f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:48.645468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d124a-768a-4765-ba00-8e595e3de097 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:48.646204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d124a-768a-4765-ba00-8e595e3de097 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:48.646553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d124a-768a-4765-ba00-8e595e3de097 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:48.646847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d124a-768a-4765-ba00-8e595e3de097 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:48.647690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d124a-768a-4765-ba00-8e595e3de097 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:48.649337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d124a-768a-4765-ba00-8e595e3de097 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:48.667501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:48.687274 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3245/6550] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:48 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:48.688630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31d10a-4dc6-4f49-b615-41afc4359e5f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:48.691585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8e31d10a-4dc6-4f49-b615-41afc4359e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:48.701429 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3306/6551] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:13:48 2022] POST /identity/v3/domains => generated 276 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:48.706351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d124a-768a-4765-ba00-8e595e3de097 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:48.707819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31d10a-4dc6-4f49-b615-41afc4359e5f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:48.798070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d124a-768a-4765-ba00-8e595e3de097 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:48.803915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e042984-8748-475a-b636-2bd6bd3bb034 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:48.807155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e042984-8748-475a-b636-2bd6bd3bb034 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:48.807424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e042984-8748-475a-b636-2bd6bd3bb034 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:48.807682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e042984-8748-475a-b636-2bd6bd3bb034 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:48.808377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e042984-8748-475a-b636-2bd6bd3bb034 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:48.810791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e435b-e6cb-431a-99c0-1a134b4ec4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:48.811743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e435b-e6cb-431a-99c0-1a134b4ec4d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:48.812263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e435b-e6cb-431a-99c0-1a134b4ec4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:48.812778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e435b-e6cb-431a-99c0-1a134b4ec4d2 admin admin] PATH_INFO: `/v3/projects/99af99b8903e4bcdb0f03bf35a6002c6` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:48.814288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e435b-e6cb-431a-99c0-1a134b4ec4d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99af99b8903e4bcdb0f03bf35a6002c6)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:48.818072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e042984-8748-475a-b636-2bd6bd3bb034 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:48.819377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e435b-e6cb-431a-99c0-1a134b4ec4d2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:48.856000 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3307/6552] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:13:48 2022] POST /identity/v3/roles => generated 239 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:48.860990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e042984-8748-475a-b636-2bd6bd3bb034 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:48.875578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e042984-8748-475a-b636-2bd6bd3bb034 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:48.883067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d3ebd-2028-4dd1-aa16-01b9818d649d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:48.884164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d3ebd-2028-4dd1-aa16-01b9818d649d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:48.884672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d3ebd-2028-4dd1-aa16-01b9818d649d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:48.885347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d3ebd-2028-4dd1-aa16-01b9818d649d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:48.886297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d3ebd-2028-4dd1-aa16-01b9818d649d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:48.888202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d3ebd-2028-4dd1-aa16-01b9818d649d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:48.904370 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3308/6553] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:48 2022] GET /identity/v3/roles => generated 2109 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:48.905712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d3ebd-2028-4dd1-aa16-01b9818d649d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:48.921160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d3ebd-2028-4dd1-aa16-01b9818d649d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:48.936455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cb0e435b-e6cb-431a-99c0-1a134b4ec4d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99af99b8903e4bcdb0f03bf35a6002c6', 'request_id': 'req-cb0e435b-e6cb-431a-99c0-1a134b4ec4d2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:48.947535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cb0e435b-e6cb-431a-99c0-1a134b4ec4d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99af99b8903e4bcdb0f03bf35a6002c6', 'request_id': 'req-cb0e435b-e6cb-431a-99c0-1a134b4ec4d2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:48.956726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cb0e435b-e6cb-431a-99c0-1a134b4ec4d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99af99b8903e4bcdb0f03bf35a6002c6', 'request_id': 'req-cb0e435b-e6cb-431a-99c0-1a134b4ec4d2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:48.970181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:48.975460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cb0e435b-e6cb-431a-99c0-1a134b4ec4d2 admin admin] The token cache is being invalidate because project 99af99b8903e4bcdb0f03bf35a6002c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:48.976197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cb0e435b-e6cb-431a-99c0-1a134b4ec4d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:48.977598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-cb0e435b-e6cb-431a-99c0-1a134b4ec4d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:48.981158 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3246/6554] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:48 2022] DELETE /identity/v3/projects/99af99b8903e4bcdb0f03bf35a6002c6 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:48.983287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e435b-e6cb-431a-99c0-1a134b4ec4d2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:48.985065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cb0e435b-e6cb-431a-99c0-1a134b4ec4d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:49.105789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca45e2e-ea61-4ec0-9b93-0a685afc1b1f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:49.106534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca45e2e-ea61-4ec0-9b93-0a685afc1b1f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:49.106866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca45e2e-ea61-4ec0-9b93-0a685afc1b1f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:49.107149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca45e2e-ea61-4ec0-9b93-0a685afc1b1f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains/c0f0fa967587446cb0bfc99bdeabc526/users/eb153aa122af4ab9a652798bee5516ee/roles/b380574d48f848c68b979d70b0180290` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:49.119508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e435b-e6cb-431a-99c0-1a134b4ec4d2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:49.120224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca45e2e-ea61-4ec0-9b93-0a685afc1b1f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=c0f0fa967587446cb0bfc99bdeabc526, user_id=eb153aa122af4ab9a652798bee5516ee, role_id=b380574d48f848c68b979d70b0180290)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:49.122921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca45e2e-ea61-4ec0-9b93-0a685afc1b1f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:49.124759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae801eb5-d3a8-4738-ae88-b694aa4d6b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:49.125586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae801eb5-d3a8-4738-ae88-b694aa4d6b8c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:49.125867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae801eb5-d3a8-4738-ae88-b694aa4d6b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:49.126700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae801eb5-d3a8-4738-ae88-b694aa4d6b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:49.127490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae801eb5-d3a8-4738-ae88-b694aa4d6b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:49.130094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae801eb5-d3a8-4738-ae88-b694aa4d6b8c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:49.144505 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3309/6555] 10.222.0.17 () {60 vars in 1480 bytes} [Sat Oct 1 21:13:48 2022] PUT /identity/v3/domains/c0f0fa967587446cb0bfc99bdeabc526/users/eb153aa122af4ab9a652798bee5516ee/roles/b380574d48f848c68b979d70b0180290 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:49.146268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995049a-3130-43f1-8b13-0e0c48ee7d7b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:49.146674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995049a-3130-43f1-8b13-0e0c48ee7d7b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:49.147028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995049a-3130-43f1-8b13-0e0c48ee7d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:49.152923 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3247/6556] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:48 2022] GET /identity/v3/roles => generated 2109 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:49.152923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae801eb5-d3a8-4738-ae88-b694aa4d6b8c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:49.153675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ae801eb5-d3a8-4738-ae88-b694aa4d6b8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:49.173522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae801eb5-d3a8-4738-ae88-b694aa4d6b8c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:49.193874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f995049a-3130-43f1-8b13-0e0c48ee7d7b None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:49.195170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f995049a-3130-43f1-8b13-0e0c48ee7d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:49.282411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a6971-dcc6-4d61-bcdc-d8b92acc7b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:49.283458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a6971-dcc6-4d61-bcdc-d8b92acc7b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:49.284121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a6971-dcc6-4d61-bcdc-d8b92acc7b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:49.284410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a6971-dcc6-4d61-bcdc-d8b92acc7b50 admin admin] PATH_INFO: `/v3/users/240f6cf0aba34a739a85a6e15925613a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:49.287357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a6971-dcc6-4d61-bcdc-d8b92acc7b50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=240f6cf0aba34a739a85a6e15925613a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:49.289545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a6971-dcc6-4d61-bcdc-d8b92acc7b50 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:49.303983 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3310/6557] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:13:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:49.304987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995049a-3130-43f1-8b13-0e0c48ee7d7b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:49.307193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f995049a-3130-43f1-8b13-0e0c48ee7d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:49.324827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:49.359667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-616a6971-dcc6-4d61-bcdc-d8b92acc7b50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '240f6cf0aba34a739a85a6e15925613a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:49.360701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-616a6971-dcc6-4d61-bcdc-d8b92acc7b50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '240f6cf0aba34a739a85a6e15925613a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:49.361336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-616a6971-dcc6-4d61-bcdc-d8b92acc7b50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '240f6cf0aba34a739a85a6e15925613a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:49.372126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-616a6971-dcc6-4d61-bcdc-d8b92acc7b50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '240f6cf0aba34a739a85a6e15925613a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:49.388159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995049a-3130-43f1-8b13-0e0c48ee7d7b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:49.391379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-616a6971-dcc6-4d61-bcdc-d8b92acc7b50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '240f6cf0aba34a739a85a6e15925613a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:49.394882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418dbc95-c444-4cf0-98d1-3cd7bb3f49fd tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:49.395758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418dbc95-c444-4cf0-98d1-3cd7bb3f49fd tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:49.397156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418dbc95-c444-4cf0-98d1-3cd7bb3f49fd tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:49.397652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418dbc95-c444-4cf0-98d1-3cd7bb3f49fd tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:49.398624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418dbc95-c444-4cf0-98d1-3cd7bb3f49fd tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:49.401000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418dbc95-c444-4cf0-98d1-3cd7bb3f49fd tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:49.408867 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3248/6558] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:49 2022] DELETE /identity/v3/users/240f6cf0aba34a739a85a6e15925613a => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:49.409971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a6971-dcc6-4d61-bcdc-d8b92acc7b50 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:49.411364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-616a6971-dcc6-4d61-bcdc-d8b92acc7b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:49.425232 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3311/6559] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:13:49 2022] POST /identity/v3/domains => generated 275 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:49.425232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd239ca0-15f7-4a53-a761-c515acbdc919 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:49.425232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd239ca0-15f7-4a53-a761-c515acbdc919 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:49.425232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd239ca0-15f7-4a53-a761-c515acbdc919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:49.440076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:49.450205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-cd239ca0-15f7-4a53-a761-c515acbdc919 None None] MFA Rules not processed for user `902ba44d318d4a01a510f732d27467c9`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:49.451931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cd239ca0-15f7-4a53-a761-c515acbdc919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:49.521362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a6971-dcc6-4d61-bcdc-d8b92acc7b50 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:49.545031 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3312/6560] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:49.547786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd239ca0-15f7-4a53-a761-c515acbdc919 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:49.551253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cd239ca0-15f7-4a53-a761-c515acbdc919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:49.623375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e836d9-22ff-4291-8f59-b8b37490188b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:49.623989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e836d9-22ff-4291-8f59-b8b37490188b admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:49.624793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e836d9-22ff-4291-8f59-b8b37490188b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:49.625505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e836d9-22ff-4291-8f59-b8b37490188b admin admin] PATH_INFO: `/v3/projects/1fa38b276bd845648ee01a473ae7f4c5/users/8a6d1d77d0e84fb884bd2cefdec645a1/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:49.656921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e836d9-22ff-4291-8f59-b8b37490188b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fa38b276bd845648ee01a473ae7f4c5, user_id=8a6d1d77d0e84fb884bd2cefdec645a1, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:49.679648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e836d9-22ff-4291-8f59-b8b37490188b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:49.701625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd239ca0-15f7-4a53-a761-c515acbdc919 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:49.710057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6315c-0308-46d1-b8f3-01f03b8a19d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:49.711092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6315c-0308-46d1-b8f3-01f03b8a19d1 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:49.711592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6315c-0308-46d1-b8f3-01f03b8a19d1 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:49.712214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6315c-0308-46d1-b8f3-01f03b8a19d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:49.714651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-88b6315c-0308-46d1-b8f3-01f03b8a19d1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:49.715143 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3249/6561] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:49 2022] PUT /identity/v3/projects/1fa38b276bd845648ee01a473ae7f4c5/users/8a6d1d77d0e84fb884bd2cefdec645a1/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:49.716653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e836d9-22ff-4291-8f59-b8b37490188b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:49.728903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-14e836d9-22ff-4291-8f59-b8b37490188b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:49.812117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:49.817002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b6315c-0308-46d1-b8f3-01f03b8a19d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:49.819147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b6315c-0308-46d1-b8f3-01f03b8a19d1 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:49.827718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e836d9-22ff-4291-8f59-b8b37490188b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:49.832204 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3313/6562] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:13:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:13:49.835517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeee5ea-e1cb-4d9c-b582-cabf29ae92b7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:49.836051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeee5ea-e1cb-4d9c-b582-cabf29ae92b7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:49.836541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeee5ea-e1cb-4d9c-b582-cabf29ae92b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:49.836762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61e51e-9a8c-4a70-a534-259301b1720c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:49.837936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61e51e-9a8c-4a70-a534-259301b1720c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:49.838225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61e51e-9a8c-4a70-a534-259301b1720c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:49.838521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61e51e-9a8c-4a70-a534-259301b1720c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:49.844417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61e51e-9a8c-4a70-a534-259301b1720c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:49.846221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61e51e-9a8c-4a70-a534-259301b1720c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:49.868024 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3250/6563] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:13:49 2022] POST /identity/v3/roles => generated 240 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:49.871772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61e51e-9a8c-4a70-a534-259301b1720c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:49.873899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6aeee5ea-e1cb-4d9c-b582-cabf29ae92b7 None None] MFA Rules not processed for user `8a6d1d77d0e84fb884bd2cefdec645a1`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:49.875242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6aeee5ea-e1cb-4d9c-b582-cabf29ae92b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:49.890702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61e51e-9a8c-4a70-a534-259301b1720c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:49.909763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04490724-5296-4e96-ad79-c37b974baf0b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:49.910465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04490724-5296-4e96-ad79-c37b974baf0b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:49.910802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04490724-5296-4e96-ad79-c37b974baf0b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:49.914812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04490724-5296-4e96-ad79-c37b974baf0b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/groups` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:49.915670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04490724-5296-4e96-ad79-c37b974baf0b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:49.920718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04490724-5296-4e96-ad79-c37b974baf0b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:49.965408 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3251/6564] 10.222.0.17 () {60 vars in 1147 bytes} [Sat Oct 1 21:13:49 2022] POST /identity/v3/groups => generated 325 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:13:49.966167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04490724-5296-4e96-ad79-c37b974baf0b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:49.969310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-04490724-5296-4e96-ad79-c37b974baf0b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:49.983008 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3314/6565] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:49.984156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeee5ea-e1cb-4d9c-b582-cabf29ae92b7 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:50.002845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeee5ea-e1cb-4d9c-b582-cabf29ae92b7 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:50.009608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:50.013151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f2a15-eef5-4139-9ea3-3973f0e844de tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:50.013848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f2a15-eef5-4139-9ea3-3973f0e844de tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:50.014261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f2a15-eef5-4139-9ea3-3973f0e844de tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:50.014776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f2a15-eef5-4139-9ea3-3973f0e844de tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/groups/4a696e4953dd4ba483aa32e75dbe206f/users/eb153aa122af4ab9a652798bee5516ee` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:50.031192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f2a15-eef5-4139-9ea3-3973f0e844de tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=4a696e4953dd4ba483aa32e75dbe206f, user_id=eb153aa122af4ab9a652798bee5516ee)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:50.033358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f2a15-eef5-4139-9ea3-3973f0e844de tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:50.083128 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3315/6566] 10.222.0.17 () {60 vars in 1360 bytes} [Sat Oct 1 21:13:49 2022] PUT /identity/v3/groups/4a696e4953dd4ba483aa32e75dbe206f/users/eb153aa122af4ab9a652798bee5516ee => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:50.084160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f2a15-eef5-4139-9ea3-3973f0e844de tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:50.128933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04490724-5296-4e96-ad79-c37b974baf0b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:50.167686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f2a15-eef5-4139-9ea3-3973f0e844de tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:50.181843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a700a-5271-46ea-a3cf-3d771dff8291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:50.182683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a700a-5271-46ea-a3cf-3d771dff8291 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:50.182981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a700a-5271-46ea-a3cf-3d771dff8291 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:50.183246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a700a-5271-46ea-a3cf-3d771dff8291 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:50.184210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a700a-5271-46ea-a3cf-3d771dff8291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:50.191095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a700a-5271-46ea-a3cf-3d771dff8291 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:50.212737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf52a98-3dc4-4330-bf3c-5e5a0321fc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:50.214310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf52a98-3dc4-4330-bf3c-5e5a0321fc49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:50.214670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf52a98-3dc4-4330-bf3c-5e5a0321fc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:50.215218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf52a98-3dc4-4330-bf3c-5e5a0321fc49 admin admin] PATH_INFO: `/v3/projects/c3cca72997054a53bba0852d70d70ee1` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:50.217070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf52a98-3dc4-4330-bf3c-5e5a0321fc49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3cca72997054a53bba0852d70d70ee1)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:50.219679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf52a98-3dc4-4330-bf3c-5e5a0321fc49 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:50.236157 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3316/6567] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:50 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:50.236157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a700a-5271-46ea-a3cf-3d771dff8291 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:50.281586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-faf52a98-3dc4-4330-bf3c-5e5a0321fc49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3cca72997054a53bba0852d70d70ee1', 'request_id': 'req-faf52a98-3dc4-4330-bf3c-5e5a0321fc49'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:50.294193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-faf52a98-3dc4-4330-bf3c-5e5a0321fc49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3cca72997054a53bba0852d70d70ee1', 'request_id': 'req-faf52a98-3dc4-4330-bf3c-5e5a0321fc49'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:50.303766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-faf52a98-3dc4-4330-bf3c-5e5a0321fc49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3cca72997054a53bba0852d70d70ee1', 'request_id': 'req-faf52a98-3dc4-4330-bf3c-5e5a0321fc49'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:50.316367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:50.323693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-faf52a98-3dc4-4330-bf3c-5e5a0321fc49 admin admin] The token cache is being invalidate because project c3cca72997054a53bba0852d70d70ee1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:50.323889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-faf52a98-3dc4-4330-bf3c-5e5a0321fc49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:50.325666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-faf52a98-3dc4-4330-bf3c-5e5a0321fc49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:50.329443 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3252/6568] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:49 2022] DELETE /identity/v3/projects/c3cca72997054a53bba0852d70d70ee1 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:50.331108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf52a98-3dc4-4330-bf3c-5e5a0321fc49 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:50.333330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-faf52a98-3dc4-4330-bf3c-5e5a0321fc49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:50.348751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a700a-5271-46ea-a3cf-3d771dff8291 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:50.378410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:50.429251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a4c7f-6504-431c-8966-d776993ce5a3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:50.430126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a4c7f-6504-431c-8966-d776993ce5a3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:50.430541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a4c7f-6504-431c-8966-d776993ce5a3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:50.432414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a4c7f-6504-431c-8966-d776993ce5a3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains/5362b3c879354830a63155868fb44159/groups/4a696e4953dd4ba483aa32e75dbe206f/roles/0988cee68a8c43f7a486f147d81b9d58` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:50.436056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf52a98-3dc4-4330-bf3c-5e5a0321fc49 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:50.436434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a4c7f-6504-431c-8966-d776993ce5a3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=5362b3c879354830a63155868fb44159, group_id=4a696e4953dd4ba483aa32e75dbe206f, role_id=0988cee68a8c43f7a486f147d81b9d58)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:50.439584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a4c7f-6504-431c-8966-d776993ce5a3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:50.449972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005e5cb-c9dc-4791-84dc-0ba6e9e9fea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:50.450632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005e5cb-c9dc-4791-84dc-0ba6e9e9fea6 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:50.450959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005e5cb-c9dc-4791-84dc-0ba6e9e9fea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:50.451236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005e5cb-c9dc-4791-84dc-0ba6e9e9fea6 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:50.451881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d005e5cb-c9dc-4791-84dc-0ba6e9e9fea6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:50.454790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d005e5cb-c9dc-4791-84dc-0ba6e9e9fea6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:50.464000 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3317/6569] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:50 2022] PUT /identity/v3/domains/5362b3c879354830a63155868fb44159/groups/4a696e4953dd4ba483aa32e75dbe206f/roles/0988cee68a8c43f7a486f147d81b9d58 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:50.465094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a4c7f-6504-431c-8966-d776993ce5a3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:50.467309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cb9a4c7f-6504-431c-8966-d776993ce5a3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:50.485234 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3253/6570] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:50 2022] POST /identity/v3/users => generated 299 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:50.485234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005e5cb-c9dc-4791-84dc-0ba6e9e9fea6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:50.486306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d005e5cb-c9dc-4791-84dc-0ba6e9e9fea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:50.551736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a4c7f-6504-431c-8966-d776993ce5a3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:50.559369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217e297-3008-4735-8fb6-2ebb5a383c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:50.560012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217e297-3008-4735-8fb6-2ebb5a383c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:50.560280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217e297-3008-4735-8fb6-2ebb5a383c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:50.560551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217e297-3008-4735-8fb6-2ebb5a383c75 admin admin] PATH_INFO: `/v3/users/32363c7dc0ec4e33bdbe898136962a02` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:50.563005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217e297-3008-4735-8fb6-2ebb5a383c75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32363c7dc0ec4e33bdbe898136962a02)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:50.564940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217e297-3008-4735-8fb6-2ebb5a383c75 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:50.595581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005e5cb-c9dc-4791-84dc-0ba6e9e9fea6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:50.634764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b217e297-3008-4735-8fb6-2ebb5a383c75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32363c7dc0ec4e33bdbe898136962a02'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:50.645802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b217e297-3008-4735-8fb6-2ebb5a383c75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32363c7dc0ec4e33bdbe898136962a02'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:50.665819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b217e297-3008-4735-8fb6-2ebb5a383c75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32363c7dc0ec4e33bdbe898136962a02'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:50.666954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b217e297-3008-4735-8fb6-2ebb5a383c75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32363c7dc0ec4e33bdbe898136962a02'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:50.680630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b217e297-3008-4735-8fb6-2ebb5a383c75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32363c7dc0ec4e33bdbe898136962a02'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:50.692413 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3318/6571] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:50 2022] DELETE /identity/v3/users/32363c7dc0ec4e33bdbe898136962a02 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:50.692985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217e297-3008-4735-8fb6-2ebb5a383c75 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:50.694337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b217e297-3008-4735-8fb6-2ebb5a383c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:50.702458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:50.730284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3899293-8dfb-4db7-a23b-dea3ff24c24b tempest-TokensV3TestJSON-1837527138 tempest-TokensV3TestJSON-1837527138-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb153aa122af4ab9a652798bee5516ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08ef2b377884a51a7ef815ddef1be8c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:50.731076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3899293-8dfb-4db7-a23b-dea3ff24c24b tempest-TokensV3TestJSON-1837527138 tempest-TokensV3TestJSON-1837527138-project] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:50.731344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3899293-8dfb-4db7-a23b-dea3ff24c24b tempest-TokensV3TestJSON-1837527138 tempest-TokensV3TestJSON-1837527138-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:50.731925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3899293-8dfb-4db7-a23b-dea3ff24c24b tempest-TokensV3TestJSON-1837527138 tempest-TokensV3TestJSON-1837527138-project] PATH_INFO: `/v3/auth/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:50.732511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3899293-8dfb-4db7-a23b-dea3ff24c24b tempest-TokensV3TestJSON-1837527138 tempest-TokensV3TestJSON-1837527138-project] RBAC: Authorizing `identity:get_auth_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:50.733734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3899293-8dfb-4db7-a23b-dea3ff24c24b tempest-TokensV3TestJSON-1837527138 tempest-TokensV3TestJSON-1837527138-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:50.766422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:50.807663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217e297-3008-4735-8fb6-2ebb5a383c75 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:50.813814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4d93e-f9a5-41c7-9a92-d2bb865ece49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:50.814463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4d93e-f9a5-41c7-9a92-d2bb865ece49 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:50.814824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4d93e-f9a5-41c7-9a92-d2bb865ece49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:50.815256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4d93e-f9a5-41c7-9a92-d2bb865ece49 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:50.815923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4d93e-f9a5-41c7-9a92-d2bb865ece49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:50.817937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4d93e-f9a5-41c7-9a92-d2bb865ece49 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:50.832416 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3319/6572] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:50 2022] GET /identity/v3/roles => generated 2340 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:50.837630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4d93e-f9a5-41c7-9a92-d2bb865ece49 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:50.861613 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3254/6573] 10.222.0.17 () {58 vars in 1143 bytes} [Sat Oct 1 21:13:50 2022] GET /identity/v3/auth/domains => generated 1171 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:50.862088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4d93e-f9a5-41c7-9a92-d2bb865ece49 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:50.865688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3899293-8dfb-4db7-a23b-dea3ff24c24b tempest-TokensV3TestJSON-1837527138 tempest-TokensV3TestJSON-1837527138-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:50.867838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f3899293-8dfb-4db7-a23b-dea3ff24c24b tempest-TokensV3TestJSON-1837527138 tempest-TokensV3TestJSON-1837527138-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:50.873964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9741cf9-88dd-46ca-97f5-4d62bfbd6493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:50.874924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9741cf9-88dd-46ca-97f5-4d62bfbd6493 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:50.875492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9741cf9-88dd-46ca-97f5-4d62bfbd6493 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:50.877381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9741cf9-88dd-46ca-97f5-4d62bfbd6493 admin admin] PATH_INFO: `/v3/projects/5b09cbe898f04fa199f05d7122f76e11/users/0d76c2f60dde4f51aa8762150ffe71fa/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:50.916158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9741cf9-88dd-46ca-97f5-4d62bfbd6493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b09cbe898f04fa199f05d7122f76e11, user_id=0d76c2f60dde4f51aa8762150ffe71fa, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:50.919327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9741cf9-88dd-46ca-97f5-4d62bfbd6493 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:50.942536 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3320/6574] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:50 2022] PUT /identity/v3/projects/5b09cbe898f04fa199f05d7122f76e11/users/0d76c2f60dde4f51aa8762150ffe71fa/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:50.948401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9741cf9-88dd-46ca-97f5-4d62bfbd6493 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:51.029016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3899293-8dfb-4db7-a23b-dea3ff24c24b tempest-TokensV3TestJSON-1837527138 tempest-TokensV3TestJSON-1837527138-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:51.081913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9741cf9-88dd-46ca-97f5-4d62bfbd6493 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:51.093850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987e3cc-335b-46f7-8a1f-92a0fc939fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:51.094510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987e3cc-335b-46f7-8a1f-92a0fc939fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:51.094789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987e3cc-335b-46f7-8a1f-92a0fc939fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:51.097753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987e3cc-335b-46f7-8a1f-92a0fc939fe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:51.098434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987e3cc-335b-46f7-8a1f-92a0fc939fe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:51.102529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987e3cc-335b-46f7-8a1f-92a0fc939fe0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:51.124416 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3321/6575] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:50 2022] GET /identity/v3/roles => generated 2340 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:13:51.129900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987e3cc-335b-46f7-8a1f-92a0fc939fe0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:51.143998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f7c08-8836-44b7-a7a1-9c0a0c80c727 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:51.147198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f7c08-8836-44b7-a7a1-9c0a0c80c727 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:51.147559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f7c08-8836-44b7-a7a1-9c0a0c80c727 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:51.147998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f7c08-8836-44b7-a7a1-9c0a0c80c727 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains/5362b3c879354830a63155868fb44159/groups/4a696e4953dd4ba483aa32e75dbe206f/roles/0988cee68a8c43f7a486f147d81b9d58` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:51.171098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987e3cc-335b-46f7-8a1f-92a0fc939fe0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:51.176836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824f204-14a6-42e9-88d9-53b9d5218262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:51.177530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824f204-14a6-42e9-88d9-53b9d5218262 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:51.177814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824f204-14a6-42e9-88d9-53b9d5218262 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:51.178093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824f204-14a6-42e9-88d9-53b9d5218262 admin admin] PATH_INFO: `/v3/projects/5b09cbe898f04fa199f05d7122f76e11/users/0d76c2f60dde4f51aa8762150ffe71fa/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:51.181538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824f204-14a6-42e9-88d9-53b9d5218262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b09cbe898f04fa199f05d7122f76e11, user_id=0d76c2f60dde4f51aa8762150ffe71fa, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:51.184278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824f204-14a6-42e9-88d9-53b9d5218262 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:51.196057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f7c08-8836-44b7-a7a1-9c0a0c80c727 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=5362b3c879354830a63155868fb44159, group_id=4a696e4953dd4ba483aa32e75dbe206f, role_id=0988cee68a8c43f7a486f147d81b9d58)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:51.200773 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3322/6576] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:51 2022] PUT /identity/v3/projects/5b09cbe898f04fa199f05d7122f76e11/users/0d76c2f60dde4f51aa8762150ffe71fa/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:51.201702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824f204-14a6-42e9-88d9-53b9d5218262 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:51.202550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f7c08-8836-44b7-a7a1-9c0a0c80c727 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:51.215357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d824f204-14a6-42e9-88d9-53b9d5218262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:51.229696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a61f7c08-8836-44b7-a7a1-9c0a0c80c727 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invalidating the token cache because role 0988cee68a8c43f7a486f147d81b9d58 was removed from group 4a696e4953dd4ba483aa32e75dbe206f on domain 5362b3c879354830a63155868fb44159. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:51.230399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a61f7c08-8836-44b7-a7a1-9c0a0c80c727 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:51.232563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a61f7c08-8836-44b7-a7a1-9c0a0c80c727 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:51.257765 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3255/6577] 10.222.0.17 () {58 vars in 1467 bytes} [Sat Oct 1 21:13:50 2022] DELETE /identity/v3/domains/5362b3c879354830a63155868fb44159/groups/4a696e4953dd4ba483aa32e75dbe206f/roles/0988cee68a8c43f7a486f147d81b9d58 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:13:51.259971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813225c-ebaf-47c9-a19c-38e43a0eef1c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:51.261778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813225c-ebaf-47c9-a19c-38e43a0eef1c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:51.262314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813225c-ebaf-47c9-a19c-38e43a0eef1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:51.316914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:51.320553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b813225c-ebaf-47c9-a19c-38e43a0eef1c None None] MFA Rules not processed for user `0d76c2f60dde4f51aa8762150ffe71fa`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:51.323008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824f204-14a6-42e9-88d9-53b9d5218262 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:51.324912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b813225c-ebaf-47c9-a19c-38e43a0eef1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:51.394593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:51.395402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3b504-47ed-4369-b46e-4662a08ce0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:51.396182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3b504-47ed-4369-b46e-4662a08ce0ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:51.396496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3b504-47ed-4369-b46e-4662a08ce0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:51.396974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3b504-47ed-4369-b46e-4662a08ce0ac admin admin] PATH_INFO: `/v3/projects/5e9e7e768c964f2e9856471cb26c4754` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:51.398193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3b504-47ed-4369-b46e-4662a08ce0ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e9e7e768c964f2e9856471cb26c4754)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:51.401272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3b504-47ed-4369-b46e-4662a08ce0ac admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:51.430265 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3256/6578] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:51.431760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813225c-ebaf-47c9-a19c-38e43a0eef1c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:51.432977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b813225c-ebaf-47c9-a19c-38e43a0eef1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:51.475621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f1d3b504-47ed-4369-b46e-4662a08ce0ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e9e7e768c964f2e9856471cb26c4754', 'request_id': 'req-f1d3b504-47ed-4369-b46e-4662a08ce0ac'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:51.486122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f1d3b504-47ed-4369-b46e-4662a08ce0ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e9e7e768c964f2e9856471cb26c4754', 'request_id': 'req-f1d3b504-47ed-4369-b46e-4662a08ce0ac'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:51.494794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f1d3b504-47ed-4369-b46e-4662a08ce0ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e9e7e768c964f2e9856471cb26c4754', 'request_id': 'req-f1d3b504-47ed-4369-b46e-4662a08ce0ac'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:51.521423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f1d3b504-47ed-4369-b46e-4662a08ce0ac admin admin] The token cache is being invalidate because project 5e9e7e768c964f2e9856471cb26c4754 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:51.521754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f1d3b504-47ed-4369-b46e-4662a08ce0ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:51.522662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f1d3b504-47ed-4369-b46e-4662a08ce0ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:51.525932 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3323/6579] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:51 2022] DELETE /identity/v3/projects/5e9e7e768c964f2e9856471cb26c4754 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:51.527158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3b504-47ed-4369-b46e-4662a08ce0ac admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:51.529015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f1d3b504-47ed-4369-b46e-4662a08ce0ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:51.529925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813225c-ebaf-47c9-a19c-38e43a0eef1c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:51.604354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2ee37-5ed4-4b02-9587-cfe2708ce330 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:51.605127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2ee37-5ed4-4b02-9587-cfe2708ce330 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:51.605407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2ee37-5ed4-4b02-9587-cfe2708ce330 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:51.606232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2ee37-5ed4-4b02-9587-cfe2708ce330 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/groups/4a696e4953dd4ba483aa32e75dbe206f/users/eb153aa122af4ab9a652798bee5516ee` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:51.608796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2ee37-5ed4-4b02-9587-cfe2708ce330 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=4a696e4953dd4ba483aa32e75dbe206f, user_id=eb153aa122af4ab9a652798bee5516ee)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:51.609779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3b504-47ed-4369-b46e-4662a08ce0ac admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:51.611298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2ee37-5ed4-4b02-9587-cfe2708ce330 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:51.614005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1cbbf-bde9-4e44-b642-a35375ead6d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:51.615026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1cbbf-bde9-4e44-b642-a35375ead6d0 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:51.615330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1cbbf-bde9-4e44-b642-a35375ead6d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:51.615730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1cbbf-bde9-4e44-b642-a35375ead6d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:51.617556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-73f1cbbf-bde9-4e44-b642-a35375ead6d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:51.630881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d7d2ee37-5ed4-4b02-9587-cfe2708ce330 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'eb153aa122af4ab9a652798bee5516ee'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:51.651878 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3257/6580] 10.222.0.17 () {58 vars in 1344 bytes} [Sat Oct 1 21:13:51 2022] DELETE /identity/v3/groups/4a696e4953dd4ba483aa32e75dbe206f/users/eb153aa122af4ab9a652798bee5516ee => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:51.653836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0da059-3d9d-4f3d-8932-83426e2878e4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:51.654340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0da059-3d9d-4f3d-8932-83426e2878e4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:51.654909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0da059-3d9d-4f3d-8932-83426e2878e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:51.682419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:51.697177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-db0da059-3d9d-4f3d-8932-83426e2878e4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:51.698416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-db0da059-3d9d-4f3d-8932-83426e2878e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:51.708881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1cbbf-bde9-4e44-b642-a35375ead6d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:51.710667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1cbbf-bde9-4e44-b642-a35375ead6d0 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:51.724821 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3324/6581] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:51.726148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1cbbf-bde9-4e44-b642-a35375ead6d0 service cinder] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:51.727559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-73f1cbbf-bde9-4e44-b642-a35375ead6d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:51.762062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1cbbf-bde9-4e44-b642-a35375ead6d0 service cinder] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:51.777770 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3258/6582] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:51.785640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e4e77-3266-4b9b-a16e-8afe2951461a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:51.786687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e4e77-3266-4b9b-a16e-8afe2951461a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:51.787649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e4e77-3266-4b9b-a16e-8afe2951461a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:51.814719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-cd8e4e77-3266-4b9b-a16e-8afe2951461a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:51.816068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cd8e4e77-3266-4b9b-a16e-8afe2951461a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:51.831906 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3259/6583] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:51.835586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e4e77-3266-4b9b-a16e-8afe2951461a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:51.841901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057588f4-0b9c-4bb3-81d6-dfce9224e4a1 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:51.842575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057588f4-0b9c-4bb3-81d6-dfce9224e4a1 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:51.843023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057588f4-0b9c-4bb3-81d6-dfce9224e4a1 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:51.843305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057588f4-0b9c-4bb3-81d6-dfce9224e4a1 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/groups/4a696e4953dd4ba483aa32e75dbe206f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:51.845775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057588f4-0b9c-4bb3-81d6-dfce9224e4a1 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:delete_group(group_id=4a696e4953dd4ba483aa32e75dbe206f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:51.845934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e4e77-3266-4b9b-a16e-8afe2951461a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:51.848439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057588f4-0b9c-4bb3-81d6-dfce9224e4a1 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:51.853469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8707cf-5d80-461e-b698-f4416f5d6467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:51.854324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8707cf-5d80-461e-b698-f4416f5d6467 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:51.854914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8707cf-5d80-461e-b698-f4416f5d6467 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:51.855195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8707cf-5d80-461e-b698-f4416f5d6467 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:51.855856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8707cf-5d80-461e-b698-f4416f5d6467 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:13:51.856162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8707cf-5d80-461e-b698-f4416f5d6467 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:51.858020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8707cf-5d80-461e-b698-f4416f5d6467 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:51.870367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:51.879228 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3260/6584] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:13:51.883571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8707cf-5d80-461e-b698-f4416f5d6467 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:51.897035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8707cf-5d80-461e-b698-f4416f5d6467 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:51.902180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c79ab9-f538-46ba-a7da-6819870d0cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:51.903044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c79ab9-f538-46ba-a7da-6819870d0cdf admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:51.903382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c79ab9-f538-46ba-a7da-6819870d0cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:51.903838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c79ab9-f538-46ba-a7da-6819870d0cdf admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:51.904526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c79ab9-f538-46ba-a7da-6819870d0cdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:51.905707 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3325/6585] 10.222.0.17 () {58 vars in 1227 bytes} [Sat Oct 1 21:13:51 2022] DELETE /identity/v3/groups/4a696e4953dd4ba483aa32e75dbe206f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:51.906727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c79ab9-f538-46ba-a7da-6819870d0cdf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:51.909853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057588f4-0b9c-4bb3-81d6-dfce9224e4a1 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:51.943608 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3261/6586] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:51 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:51.948249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c79ab9-f538-46ba-a7da-6819870d0cdf admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:51.985508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057588f4-0b9c-4bb3-81d6-dfce9224e4a1 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:52.028330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c79ab9-f538-46ba-a7da-6819870d0cdf admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:52.033306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57797085-fba7-472c-80fa-1115ae5e47a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:52.034010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57797085-fba7-472c-80fa-1115ae5e47a2 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:52.034396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57797085-fba7-472c-80fa-1115ae5e47a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:52.034923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57797085-fba7-472c-80fa-1115ae5e47a2 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:52.035661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57797085-fba7-472c-80fa-1115ae5e47a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:52.037679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57797085-fba7-472c-80fa-1115ae5e47a2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:52.070536 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3262/6587] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:51 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:52.076321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57797085-fba7-472c-80fa-1115ae5e47a2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:52.081456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:52.096239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66319dae-391b-4059-b9f0-b7c0c0b888f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:52.096916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66319dae-391b-4059-b9f0-b7c0c0b888f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:52.097170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66319dae-391b-4059-b9f0-b7c0c0b888f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:52.097439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66319dae-391b-4059-b9f0-b7c0c0b888f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/roles/0988cee68a8c43f7a486f147d81b9d58` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:52.098082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57797085-fba7-472c-80fa-1115ae5e47a2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:52.099699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66319dae-391b-4059-b9f0-b7c0c0b888f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:delete_role(role_id=0988cee68a8c43f7a486f147d81b9d58)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:52.101494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66319dae-391b-4059-b9f0-b7c0c0b888f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:52.107316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077e2c0-8f5d-4350-9f28-502f41a1561e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:52.108184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077e2c0-8f5d-4350-9f28-502f41a1561e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:52.108695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077e2c0-8f5d-4350-9f28-502f41a1561e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:52.109134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077e2c0-8f5d-4350-9f28-502f41a1561e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:52.110063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077e2c0-8f5d-4350-9f28-502f41a1561e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:52.112164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077e2c0-8f5d-4350-9f28-502f41a1561e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:52.128752 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3263/6588] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:52 2022] GET /identity/v3/roles => generated 2340 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:52.133699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077e2c0-8f5d-4350-9f28-502f41a1561e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:52.149933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077e2c0-8f5d-4350-9f28-502f41a1561e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:52.154121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-66319dae-391b-4059-b9f0-b7c0c0b888f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invalidating the token cache because role 0988cee68a8c43f7a486f147d81b9d58 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:52.154335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-66319dae-391b-4059-b9f0-b7c0c0b888f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.155101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-66319dae-391b-4059-b9f0-b7c0c0b888f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.161287 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3326/6589] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:51 2022] DELETE /identity/v3/roles/0988cee68a8c43f7a486f147d81b9d58 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:13:52.162650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ac938-03e6-4cf1-8dfe-52f312252996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:52.163430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ac938-03e6-4cf1-8dfe-52f312252996 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:52.163882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ac938-03e6-4cf1-8dfe-52f312252996 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:52.164046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66319dae-391b-4059-b9f0-b7c0c0b888f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:52.164340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ac938-03e6-4cf1-8dfe-52f312252996 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:52.165830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ac938-03e6-4cf1-8dfe-52f312252996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:52.167893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-66319dae-391b-4059-b9f0-b7c0c0b888f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:52.168037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ac938-03e6-4cf1-8dfe-52f312252996 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:52.202404 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3264/6590] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:52 2022] GET /identity/v3/roles => generated 2109 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:52.206633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ac938-03e6-4cf1-8dfe-52f312252996 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:52.208539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ca8ac938-03e6-4cf1-8dfe-52f312252996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:52.224729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ac938-03e6-4cf1-8dfe-52f312252996 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:52.285331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66319dae-391b-4059-b9f0-b7c0c0b888f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:52.299639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edc212-e1fc-4f34-b847-58d4315acda3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:52.300303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edc212-e1fc-4f34-b847-58d4315acda3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:52.300582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edc212-e1fc-4f34-b847-58d4315acda3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:52.300854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edc212-e1fc-4f34-b847-58d4315acda3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains/5362b3c879354830a63155868fb44159` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:52.302461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46edc212-e1fc-4f34-b847-58d4315acda3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=5362b3c879354830a63155868fb44159)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:52.304119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46edc212-e1fc-4f34-b847-58d4315acda3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:52.321228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbe814-aa02-4d4a-9477-e31981f717c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:52.321961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbe814-aa02-4d4a-9477-e31981f717c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:52.322372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbe814-aa02-4d4a-9477-e31981f717c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:52.322683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbe814-aa02-4d4a-9477-e31981f717c3 admin admin] PATH_INFO: `/v3/projects/96a930f687c1474f94ad64cc3667e5c5/users/11e8063d10a84bf0a44095c831c0f08c/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:52.328129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-46edc212-e1fc-4f34-b847-58d4315acda3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '5362b3c879354830a63155868fb44159'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.329804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-46edc212-e1fc-4f34-b847-58d4315acda3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '5362b3c879354830a63155868fb44159'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.359643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fbe814-aa02-4d4a-9477-e31981f717c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96a930f687c1474f94ad64cc3667e5c5, user_id=11e8063d10a84bf0a44095c831c0f08c, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:52.365229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fbe814-aa02-4d4a-9477-e31981f717c3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:52.371943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-46edc212-e1fc-4f34-b847-58d4315acda3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '5362b3c879354830a63155868fb44159'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.375334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-46edc212-e1fc-4f34-b847-58d4315acda3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '5362b3c879354830a63155868fb44159'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.382400 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3327/6591] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:13:52 2022] PATCH /identity/v3/domains/5362b3c879354830a63155868fb44159 => generated 276 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:13:52.385498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edc212-e1fc-4f34-b847-58d4315acda3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:52.390083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-46edc212-e1fc-4f34-b847-58d4315acda3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:52.390286 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3265/6592] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:52 2022] PUT /identity/v3/projects/96a930f687c1474f94ad64cc3667e5c5/users/11e8063d10a84bf0a44095c831c0f08c/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:13:52.401529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93866c-028e-4c52-8136-b95b456ef429 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:52.402196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93866c-028e-4c52-8136-b95b456ef429 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:52.402857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93866c-028e-4c52-8136-b95b456ef429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:52.451114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7b93866c-028e-4c52-8136-b95b456ef429 None None] MFA Rules not processed for user `11e8063d10a84bf0a44095c831c0f08c`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:52.452800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7b93866c-028e-4c52-8136-b95b456ef429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:52.473484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:52.531261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edc212-e1fc-4f34-b847-58d4315acda3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:52.569475 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3266/6593] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:52.582197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93866c-028e-4c52-8136-b95b456ef429 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:52.583776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7b93866c-028e-4c52-8136-b95b456ef429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:52.595614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:UPDATE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:52.628491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdbebc-ff8b-460a-beeb-60cc865d4501 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:52.629136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdbebc-ff8b-460a-beeb-60cc865d4501 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:52.629515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdbebc-ff8b-460a-beeb-60cc865d4501 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:52.630399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdbebc-ff8b-460a-beeb-60cc865d4501 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains/5362b3c879354830a63155868fb44159` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:52.642022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdbebc-ff8b-460a-beeb-60cc865d4501 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=5362b3c879354830a63155868fb44159)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:52.643686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdbebc-ff8b-460a-beeb-60cc865d4501 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:52.667397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93866c-028e-4c52-8136-b95b456ef429 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:52.670369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96cdbebc-ff8b-460a-beeb-60cc865d4501 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '5362b3c879354830a63155868fb44159'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.672614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30feed-fe5f-47aa-903c-56ad704bf42a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:52.673233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30feed-fe5f-47aa-903c-56ad704bf42a service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:52.673512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30feed-fe5f-47aa-903c-56ad704bf42a service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:52.673786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30feed-fe5f-47aa-903c-56ad704bf42a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:52.676090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96cdbebc-ff8b-460a-beeb-60cc865d4501 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '5362b3c879354830a63155868fb44159'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.684870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30feed-fe5f-47aa-903c-56ad704bf42a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:52.686601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30feed-fe5f-47aa-903c-56ad704bf42a service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:52.696665 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3267/6594] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:52.728625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30feed-fe5f-47aa-903c-56ad704bf42a service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:52.730745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-be30feed-fe5f-47aa-903c-56ad704bf42a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:52.791923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96cdbebc-ff8b-460a-beeb-60cc865d4501 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5362b3c879354830a63155868fb44159', 'request_id': 'req-96cdbebc-ff8b-460a-beeb-60cc865d4501'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.791923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96cdbebc-ff8b-460a-beeb-60cc865d4501 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5362b3c879354830a63155868fb44159', 'request_id': 'req-96cdbebc-ff8b-460a-beeb-60cc865d4501'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.791923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96cdbebc-ff8b-460a-beeb-60cc865d4501 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5362b3c879354830a63155868fb44159', 'request_id': 'req-96cdbebc-ff8b-460a-beeb-60cc865d4501'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.799851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96cdbebc-ff8b-460a-beeb-60cc865d4501 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] The token cache is being invalidate because project 5362b3c879354830a63155868fb44159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:52.800142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96cdbebc-ff8b-460a-beeb-60cc865d4501 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.800827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96cdbebc-ff8b-460a-beeb-60cc865d4501 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.809556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96cdbebc-ff8b-460a-beeb-60cc865d4501 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '5362b3c879354830a63155868fb44159', 'request_id': 'req-96cdbebc-ff8b-460a-beeb-60cc865d4501'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.810712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96cdbebc-ff8b-460a-beeb-60cc865d4501 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '5362b3c879354830a63155868fb44159', 'request_id': 'req-96cdbebc-ff8b-460a-beeb-60cc865d4501'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.811554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96cdbebc-ff8b-460a-beeb-60cc865d4501 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '5362b3c879354830a63155868fb44159', 'request_id': 'req-96cdbebc-ff8b-460a-beeb-60cc865d4501'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.823154 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3328/6595] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:13:52 2022] DELETE /identity/v3/domains/5362b3c879354830a63155868fb44159 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:52.827191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdbebc-ff8b-460a-beeb-60cc865d4501 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:52.828604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-96cdbebc-ff8b-460a-beeb-60cc865d4501 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:52.852071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30feed-fe5f-47aa-903c-56ad704bf42a service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:52.905183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdbebc-ff8b-460a-beeb-60cc865d4501 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:52.912178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991305b-e720-4d55-b904-952b49dbfb3f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:52.912779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991305b-e720-4d55-b904-952b49dbfb3f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:52.913333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991305b-e720-4d55-b904-952b49dbfb3f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:52.913589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991305b-e720-4d55-b904-952b49dbfb3f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains/c0f0fa967587446cb0bfc99bdeabc526/users/eb153aa122af4ab9a652798bee5516ee/roles/b380574d48f848c68b979d70b0180290` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:52.914516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b558d-4783-44fe-8316-de3f80771932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:52.915474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b558d-4783-44fe-8316-de3f80771932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:52.915829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b558d-4783-44fe-8316-de3f80771932 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:52.916168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b558d-4783-44fe-8316-de3f80771932 admin admin] PATH_INFO: `/v3/users/11e8063d10a84bf0a44095c831c0f08c` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:52.917076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991305b-e720-4d55-b904-952b49dbfb3f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=c0f0fa967587446cb0bfc99bdeabc526, user_id=eb153aa122af4ab9a652798bee5516ee, role_id=b380574d48f848c68b979d70b0180290)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:52.917757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b558d-4783-44fe-8316-de3f80771932 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11e8063d10a84bf0a44095c831c0f08c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:52.919459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b558d-4783-44fe-8316-de3f80771932 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:52.920170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991305b-e720-4d55-b904-952b49dbfb3f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:52.926790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:52.928357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0991305b-e720-4d55-b904-952b49dbfb3f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invalidating the token cache because role b380574d48f848c68b979d70b0180290 was removed from user eb153aa122af4ab9a652798bee5516ee on domain c0f0fa967587446cb0bfc99bdeabc526. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:52.928476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0991305b-e720-4d55-b904-952b49dbfb3f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.929356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0991305b-e720-4d55-b904-952b49dbfb3f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.941566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:52.947845 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3329/6596] 10.222.0.17 () {58 vars in 1464 bytes} [Sat Oct 1 21:13:52 2022] DELETE /identity/v3/domains/c0f0fa967587446cb0bfc99bdeabc526/users/eb153aa122af4ab9a652798bee5516ee/roles/b380574d48f848c68b979d70b0180290 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:52.952061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991305b-e720-4d55-b904-952b49dbfb3f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:52.957258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0991305b-e720-4d55-b904-952b49dbfb3f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:52.978017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c35b558d-4783-44fe-8316-de3f80771932 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11e8063d10a84bf0a44095c831c0f08c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.978745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c35b558d-4783-44fe-8316-de3f80771932 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11e8063d10a84bf0a44095c831c0f08c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.979394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c35b558d-4783-44fe-8316-de3f80771932 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11e8063d10a84bf0a44095c831c0f08c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:52.989914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c35b558d-4783-44fe-8316-de3f80771932 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11e8063d10a84bf0a44095c831c0f08c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:53.011273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c35b558d-4783-44fe-8316-de3f80771932 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11e8063d10a84bf0a44095c831c0f08c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:53.040846 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3268/6597] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:52 2022] DELETE /identity/v3/users/11e8063d10a84bf0a44095c831c0f08c => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:53.041779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991305b-e720-4d55-b904-952b49dbfb3f tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:53.046029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75bbe4-5e6f-4d7d-94f1-035ad840ba47 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:53.048687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75bbe4-5e6f-4d7d-94f1-035ad840ba47 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:53.049314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75bbe4-5e6f-4d7d-94f1-035ad840ba47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:53.088683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ed75bbe4-5e6f-4d7d-94f1-035ad840ba47 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:53.090147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ed75bbe4-5e6f-4d7d-94f1-035ad840ba47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:53.139400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb263c81-7168-4f7c-94ca-ab60d077c3cd tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:53.139606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb263c81-7168-4f7c-94ca-ab60d077c3cd tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:53.139860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb263c81-7168-4f7c-94ca-ab60d077c3cd tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:53.140145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb263c81-7168-4f7c-94ca-ab60d077c3cd tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/roles/b380574d48f848c68b979d70b0180290` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:53.146081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb263c81-7168-4f7c-94ca-ab60d077c3cd tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:delete_role(role_id=b380574d48f848c68b979d70b0180290)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:53.148597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb263c81-7168-4f7c-94ca-ab60d077c3cd tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:53.161258 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3269/6598] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:53.172064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75bbe4-5e6f-4d7d-94f1-035ad840ba47 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:53.175692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ed75bbe4-5e6f-4d7d-94f1-035ad840ba47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:53.240057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bb263c81-7168-4f7c-94ca-ab60d077c3cd tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invalidating the token cache because role b380574d48f848c68b979d70b0180290 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:53.240423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bb263c81-7168-4f7c-94ca-ab60d077c3cd tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:53.241370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bb263c81-7168-4f7c-94ca-ab60d077c3cd tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:53.248538 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3330/6599] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:52 2022] DELETE /identity/v3/roles/b380574d48f848c68b979d70b0180290 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:13:53.251210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb263c81-7168-4f7c-94ca-ab60d077c3cd tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:53.253725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bb263c81-7168-4f7c-94ca-ab60d077c3cd tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:53.296911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75bbe4-5e6f-4d7d-94f1-035ad840ba47 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:53.402983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:53.429232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef6a3b-1523-4fa3-8691-55c6b2543e2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:53.430158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef6a3b-1523-4fa3-8691-55c6b2543e2d service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:53.431084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef6a3b-1523-4fa3-8691-55c6b2543e2d service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:53.431834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef6a3b-1523-4fa3-8691-55c6b2543e2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:53.434242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-20ef6a3b-1523-4fa3-8691-55c6b2543e2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:53.475742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb263c81-7168-4f7c-94ca-ab60d077c3cd tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:53.486190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb424731-1b88-4b1b-afe6-2222a556765c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:53.487048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb424731-1b88-4b1b-afe6-2222a556765c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:53.487334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb424731-1b88-4b1b-afe6-2222a556765c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:53.487608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb424731-1b88-4b1b-afe6-2222a556765c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains/c0f0fa967587446cb0bfc99bdeabc526` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:53.489775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb424731-1b88-4b1b-afe6-2222a556765c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=c0f0fa967587446cb0bfc99bdeabc526)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:53.491278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb424731-1b88-4b1b-afe6-2222a556765c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:53.548453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:53.558104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cb424731-1b88-4b1b-afe6-2222a556765c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'c0f0fa967587446cb0bfc99bdeabc526'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:53.561020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cb424731-1b88-4b1b-afe6-2222a556765c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'c0f0fa967587446cb0bfc99bdeabc526'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:53.565390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef6a3b-1523-4fa3-8691-55c6b2543e2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:53.570064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef6a3b-1523-4fa3-8691-55c6b2543e2d service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:53.587114 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3270/6600] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:53.588249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef6a3b-1523-4fa3-8691-55c6b2543e2d service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:53.589601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-20ef6a3b-1523-4fa3-8691-55c6b2543e2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:53.624818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cb424731-1b88-4b1b-afe6-2222a556765c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'c0f0fa967587446cb0bfc99bdeabc526'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:53.626535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cb424731-1b88-4b1b-afe6-2222a556765c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'c0f0fa967587446cb0bfc99bdeabc526'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:53.633482 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3331/6601] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:13:53 2022] PATCH /identity/v3/domains/c0f0fa967587446cb0bfc99bdeabc526 => generated 277 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:13:53.634638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb424731-1b88-4b1b-afe6-2222a556765c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:53.638930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cb424731-1b88-4b1b-afe6-2222a556765c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:53.735346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef6a3b-1523-4fa3-8691-55c6b2543e2d service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:53.777512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb424731-1b88-4b1b-afe6-2222a556765c tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:53.785879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8088908-e435-45df-9936-36791e5736ea tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:53.787223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8088908-e435-45df-9936-36791e5736ea tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:53.787619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8088908-e435-45df-9936-36791e5736ea tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:53.788027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8088908-e435-45df-9936-36791e5736ea tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains/c0f0fa967587446cb0bfc99bdeabc526` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:53.805818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8088908-e435-45df-9936-36791e5736ea tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=c0f0fa967587446cb0bfc99bdeabc526)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:53.807811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8088908-e435-45df-9936-36791e5736ea tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:53.841579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65556db-a6dc-4135-bc36-1d8111842f8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:53.842890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65556db-a6dc-4135-bc36-1d8111842f8d service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:53.843165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65556db-a6dc-4135-bc36-1d8111842f8d service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:53.843537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65556db-a6dc-4135-bc36-1d8111842f8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:53.846374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e65556db-a6dc-4135-bc36-1d8111842f8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:53.859569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8088908-e435-45df-9936-36791e5736ea tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'c0f0fa967587446cb0bfc99bdeabc526'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:53.867084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8088908-e435-45df-9936-36791e5736ea tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'c0f0fa967587446cb0bfc99bdeabc526'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:53.933373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65556db-a6dc-4135-bc36-1d8111842f8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:53.935651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65556db-a6dc-4135-bc36-1d8111842f8d service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:53.946322 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3271/6602] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:53.970422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8088908-e435-45df-9936-36791e5736ea tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0f0fa967587446cb0bfc99bdeabc526', 'request_id': 'req-d8088908-e435-45df-9936-36791e5736ea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:53.983503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8088908-e435-45df-9936-36791e5736ea tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0f0fa967587446cb0bfc99bdeabc526', 'request_id': 'req-d8088908-e435-45df-9936-36791e5736ea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:53.996396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8088908-e435-45df-9936-36791e5736ea tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0f0fa967587446cb0bfc99bdeabc526', 'request_id': 'req-d8088908-e435-45df-9936-36791e5736ea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:54.009263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65556db-a6dc-4135-bc36-1d8111842f8d service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:54.010956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e65556db-a6dc-4135-bc36-1d8111842f8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:54.017291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:54.033279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8088908-e435-45df-9936-36791e5736ea tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] The token cache is being invalidate because project c0f0fa967587446cb0bfc99bdeabc526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:54.033279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8088908-e435-45df-9936-36791e5736ea tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:54.038369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8088908-e435-45df-9936-36791e5736ea tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:54.088748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8088908-e435-45df-9936-36791e5736ea tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'c0f0fa967587446cb0bfc99bdeabc526', 'request_id': 'req-d8088908-e435-45df-9936-36791e5736ea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:54.091134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8088908-e435-45df-9936-36791e5736ea tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'c0f0fa967587446cb0bfc99bdeabc526', 'request_id': 'req-d8088908-e435-45df-9936-36791e5736ea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:54.093226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8088908-e435-45df-9936-36791e5736ea tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'c0f0fa967587446cb0bfc99bdeabc526', 'request_id': 'req-d8088908-e435-45df-9936-36791e5736ea'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:54.106398 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3332/6603] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:13:53 2022] DELETE /identity/v3/domains/c0f0fa967587446cb0bfc99bdeabc526 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:54.110824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8088908-e435-45df-9936-36791e5736ea tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:54.113031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d8088908-e435-45df-9936-36791e5736ea tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:54.139198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65556db-a6dc-4135-bc36-1d8111842f8d service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:54.144179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=87,keystone:UPDATE=1,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:54.217064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b530895-6b9b-4f7f-9323-4f81232a7352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:54.217959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b530895-6b9b-4f7f-9323-4f81232a7352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:54.218618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b530895-6b9b-4f7f-9323-4f81232a7352 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:54.219059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b530895-6b9b-4f7f-9323-4f81232a7352 admin admin] PATH_INFO: `/v3/projects/96a930f687c1474f94ad64cc3667e5c5` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:54.220465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b530895-6b9b-4f7f-9323-4f81232a7352 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96a930f687c1474f94ad64cc3667e5c5)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:54.223006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b530895-6b9b-4f7f-9323-4f81232a7352 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:54.236709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8088908-e435-45df-9936-36791e5736ea tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:54.243087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc238f0f-e4e2-48f3-8ecc-794e3ed56178 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:54.243882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc238f0f-e4e2-48f3-8ecc-794e3ed56178 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:54.244275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc238f0f-e4e2-48f3-8ecc-794e3ed56178 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:54.244780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc238f0f-e4e2-48f3-8ecc-794e3ed56178 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains/134961015f02403fa8cc7397b10945f2/groups/ea531154be934235bdbd27a1930044fb/roles/1c3c664618dd4d9a9c7c737243a42528` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:54.248557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc238f0f-e4e2-48f3-8ecc-794e3ed56178 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=134961015f02403fa8cc7397b10945f2, group_id=ea531154be934235bdbd27a1930044fb, role_id=1c3c664618dd4d9a9c7c737243a42528)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:54.252143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc238f0f-e4e2-48f3-8ecc-794e3ed56178 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:54.260348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bc238f0f-e4e2-48f3-8ecc-794e3ed56178 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invalidating the token cache because role 1c3c664618dd4d9a9c7c737243a42528 was removed from group ea531154be934235bdbd27a1930044fb on domain 134961015f02403fa8cc7397b10945f2. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:54.260784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bc238f0f-e4e2-48f3-8ecc-794e3ed56178 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:54.262303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bc238f0f-e4e2-48f3-8ecc-794e3ed56178 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:54.286439 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3333/6604] 10.222.0.17 () {58 vars in 1467 bytes} [Sat Oct 1 21:13:54 2022] DELETE /identity/v3/domains/134961015f02403fa8cc7397b10945f2/groups/ea531154be934235bdbd27a1930044fb/roles/1c3c664618dd4d9a9c7c737243a42528 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:13:54.292068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc238f0f-e4e2-48f3-8ecc-794e3ed56178 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:54.297440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6b530895-6b9b-4f7f-9323-4f81232a7352 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96a930f687c1474f94ad64cc3667e5c5', 'request_id': 'req-6b530895-6b9b-4f7f-9323-4f81232a7352'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:54.298457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bc238f0f-e4e2-48f3-8ecc-794e3ed56178 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:54.325624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6b530895-6b9b-4f7f-9323-4f81232a7352 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96a930f687c1474f94ad64cc3667e5c5', 'request_id': 'req-6b530895-6b9b-4f7f-9323-4f81232a7352'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:54.341275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6b530895-6b9b-4f7f-9323-4f81232a7352 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96a930f687c1474f94ad64cc3667e5c5', 'request_id': 'req-6b530895-6b9b-4f7f-9323-4f81232a7352'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:54.382128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6b530895-6b9b-4f7f-9323-4f81232a7352 admin admin] The token cache is being invalidate because project 96a930f687c1474f94ad64cc3667e5c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:54.382451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6b530895-6b9b-4f7f-9323-4f81232a7352 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:54.383153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6b530895-6b9b-4f7f-9323-4f81232a7352 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:54.385117 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3272/6605] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:54 2022] DELETE /identity/v3/projects/96a930f687c1474f94ad64cc3667e5c5 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:54.390080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d452bb6-cd16-4a2e-bee8-58b68dbd86eb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:54.390376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d452bb6-cd16-4a2e-bee8-58b68dbd86eb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:54.390827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d452bb6-cd16-4a2e-bee8-58b68dbd86eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:54.397315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc238f0f-e4e2-48f3-8ecc-794e3ed56178 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:54.404699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249ac51-3c27-4742-9d6e-616ba4a46cd3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:54.405836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249ac51-3c27-4742-9d6e-616ba4a46cd3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:54.406365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249ac51-3c27-4742-9d6e-616ba4a46cd3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:54.406997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249ac51-3c27-4742-9d6e-616ba4a46cd3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/groups/ea531154be934235bdbd27a1930044fb/users/eb153aa122af4ab9a652798bee5516ee` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:54.409814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c249ac51-3c27-4742-9d6e-616ba4a46cd3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=ea531154be934235bdbd27a1930044fb, user_id=eb153aa122af4ab9a652798bee5516ee)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:54.413145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c249ac51-3c27-4742-9d6e-616ba4a46cd3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:54.416536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1d452bb6-cd16-4a2e-bee8-58b68dbd86eb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:54.418174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1d452bb6-cd16-4a2e-bee8-58b68dbd86eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:54.427935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c249ac51-3c27-4742-9d6e-616ba4a46cd3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'eb153aa122af4ab9a652798bee5516ee'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:54.454155 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3334/6606] 10.222.0.17 () {58 vars in 1344 bytes} [Sat Oct 1 21:13:54 2022] DELETE /identity/v3/groups/ea531154be934235bdbd27a1930044fb/users/eb153aa122af4ab9a652798bee5516ee => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:13:54.459025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249ac51-3c27-4742-9d6e-616ba4a46cd3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:54.461629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c249ac51-3c27-4742-9d6e-616ba4a46cd3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:54.467575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:54.518549 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3273/6607] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:54.524328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc657610-2574-4112-a098-b1f348ee69d9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:54.525070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc657610-2574-4112-a098-b1f348ee69d9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:54.526711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc657610-2574-4112-a098-b1f348ee69d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:54.554867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249ac51-3c27-4742-9d6e-616ba4a46cd3 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:54.561168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f80f9-abe1-4959-8157-5a82e6b044d4 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:54.561821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f80f9-abe1-4959-8157-5a82e6b044d4 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:54.562258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f80f9-abe1-4959-8157-5a82e6b044d4 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:54.562661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f80f9-abe1-4959-8157-5a82e6b044d4 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/groups/ea531154be934235bdbd27a1930044fb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:54.563150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bc657610-2574-4112-a098-b1f348ee69d9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:54.564356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69f80f9-abe1-4959-8157-5a82e6b044d4 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:delete_group(group_id=ea531154be934235bdbd27a1930044fb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:54.564498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bc657610-2574-4112-a098-b1f348ee69d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:54.566357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69f80f9-abe1-4959-8157-5a82e6b044d4 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:54.611231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:54.632622 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3335/6608] 10.222.0.17 () {58 vars in 1227 bytes} [Sat Oct 1 21:13:54 2022] DELETE /identity/v3/groups/ea531154be934235bdbd27a1930044fb => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:13:54.637143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f80f9-abe1-4959-8157-5a82e6b044d4 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:54.667784 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3274/6609] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:54.668753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc657610-2574-4112-a098-b1f348ee69d9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:54.687100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc657610-2574-4112-a098-b1f348ee69d9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:54.764575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f80f9-abe1-4959-8157-5a82e6b044d4 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:54.772438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac0c25-bca5-4b84-8cfc-615efbca75eb tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:54.775035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac0c25-bca5-4b84-8cfc-615efbca75eb tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:54.775391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac0c25-bca5-4b84-8cfc-615efbca75eb tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:54.775888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac0c25-bca5-4b84-8cfc-615efbca75eb tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/roles/1c3c664618dd4d9a9c7c737243a42528` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:54.777617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac0c25-bca5-4b84-8cfc-615efbca75eb tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:delete_role(role_id=1c3c664618dd4d9a9c7c737243a42528)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:54.780551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac0c25-bca5-4b84-8cfc-615efbca75eb tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:54.796207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d2743-ad27-445a-9f2e-f7dea7d0fed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:54.796927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d2743-ad27-445a-9f2e-f7dea7d0fed2 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:54.797272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d2743-ad27-445a-9f2e-f7dea7d0fed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:54.797662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d2743-ad27-445a-9f2e-f7dea7d0fed2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:54.798380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d2743-ad27-445a-9f2e-f7dea7d0fed2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:13:54.798806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d2743-ad27-445a-9f2e-f7dea7d0fed2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:54.800973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d2743-ad27-445a-9f2e-f7dea7d0fed2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:54.818623 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3275/6610] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:13:54.819572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d2743-ad27-445a-9f2e-f7dea7d0fed2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:54.821571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-187d2743-ad27-445a-9f2e-f7dea7d0fed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:54.830863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ceac0c25-bca5-4b84-8cfc-615efbca75eb tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invalidating the token cache because role 1c3c664618dd4d9a9c7c737243a42528 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:54.831406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ceac0c25-bca5-4b84-8cfc-615efbca75eb tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:54.832152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ceac0c25-bca5-4b84-8cfc-615efbca75eb tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:54.839134 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3336/6611] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:54 2022] DELETE /identity/v3/roles/1c3c664618dd4d9a9c7c737243a42528 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:13:54.840316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac0c25-bca5-4b84-8cfc-615efbca75eb tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:54.841871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ceac0c25-bca5-4b84-8cfc-615efbca75eb tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:54.933711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d2743-ad27-445a-9f2e-f7dea7d0fed2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:54.946632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac0c25-bca5-4b84-8cfc-615efbca75eb tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:54.952207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be828bb-0b80-4aac-930c-609b4f766791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:54.952839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be828bb-0b80-4aac-930c-609b4f766791 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:54.953131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be828bb-0b80-4aac-930c-609b4f766791 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:54.953404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be828bb-0b80-4aac-930c-609b4f766791 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:54.954447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be828bb-0b80-4aac-930c-609b4f766791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:54.957013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be828bb-0b80-4aac-930c-609b4f766791 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:54.966273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:54.995944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d81d06-c5cf-4c20-93de-1b079e99f7d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:54.996747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d81d06-c5cf-4c20-93de-1b079e99f7d8 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:54.997629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d81d06-c5cf-4c20-93de-1b079e99f7d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:54.998036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d81d06-c5cf-4c20-93de-1b079e99f7d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:55.000293 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3337/6612] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:54 2022] POST /identity/v3/projects => generated 391 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:55.001533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be828bb-0b80-4aac-930c-609b4f766791 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:55.003238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9be828bb-0b80-4aac-930c-609b4f766791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:55.003818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-71d81d06-c5cf-4c20-93de-1b079e99f7d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:55.081975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:55.110817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d81d06-c5cf-4c20-93de-1b079e99f7d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:55.112287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d81d06-c5cf-4c20-93de-1b079e99f7d8 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:55.117757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be828bb-0b80-4aac-930c-609b4f766791 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:55.122679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f308c40-e2f3-47ed-8a68-765bfa817a1b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:55.123682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f308c40-e2f3-47ed-8a68-765bfa817a1b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:55.123682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f308c40-e2f3-47ed-8a68-765bfa817a1b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:55.123894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f308c40-e2f3-47ed-8a68-765bfa817a1b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains/134961015f02403fa8cc7397b10945f2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:55.125134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f308c40-e2f3-47ed-8a68-765bfa817a1b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=134961015f02403fa8cc7397b10945f2)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:55.127010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f308c40-e2f3-47ed-8a68-765bfa817a1b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:55.130823 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3276/6613] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:55.131564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d81d06-c5cf-4c20-93de-1b079e99f7d8 service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:55.146797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d81d06-c5cf-4c20-93de-1b079e99f7d8 service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:55.153453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9f308c40-e2f3-47ed-8a68-765bfa817a1b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '134961015f02403fa8cc7397b10945f2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:55.154190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9f308c40-e2f3-47ed-8a68-765bfa817a1b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '134961015f02403fa8cc7397b10945f2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:55.190386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9f308c40-e2f3-47ed-8a68-765bfa817a1b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '134961015f02403fa8cc7397b10945f2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:55.191086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9f308c40-e2f3-47ed-8a68-765bfa817a1b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '134961015f02403fa8cc7397b10945f2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:55.199259 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3338/6614] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:13:55 2022] PATCH /identity/v3/domains/134961015f02403fa8cc7397b10945f2 => generated 276 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:13:55.200263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f308c40-e2f3-47ed-8a68-765bfa817a1b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:55.201690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9f308c40-e2f3-47ed-8a68-765bfa817a1b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:55.278009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299114c3-385c-4124-9dea-c1f76c57e920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:55.278827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299114c3-385c-4124-9dea-c1f76c57e920 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:55.278940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299114c3-385c-4124-9dea-c1f76c57e920 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:55.279354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299114c3-385c-4124-9dea-c1f76c57e920 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:55.279926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299114c3-385c-4124-9dea-c1f76c57e920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:55.282176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299114c3-385c-4124-9dea-c1f76c57e920 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:55.296525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f308c40-e2f3-47ed-8a68-765bfa817a1b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:55.312450 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3277/6615] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:55 2022] POST /identity/v3/users => generated 296 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:55.315958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299114c3-385c-4124-9dea-c1f76c57e920 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:55.316417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb9fa1-28b2-4153-b713-c4795c3325bc tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:55.317132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb9fa1-28b2-4153-b713-c4795c3325bc tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:55.317372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb9fa1-28b2-4153-b713-c4795c3325bc tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:55.317649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb9fa1-28b2-4153-b713-c4795c3325bc tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains/134961015f02403fa8cc7397b10945f2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:55.318731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-299114c3-385c-4124-9dea-c1f76c57e920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:55.332202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb9fa1-28b2-4153-b713-c4795c3325bc tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=134961015f02403fa8cc7397b10945f2)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:55.334443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb9fa1-28b2-4153-b713-c4795c3325bc tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:55.334690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299114c3-385c-4124-9dea-c1f76c57e920 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:55.371638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e4bb9fa1-28b2-4153-b713-c4795c3325bc tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '134961015f02403fa8cc7397b10945f2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:55.376579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e4bb9fa1-28b2-4153-b713-c4795c3325bc tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '134961015f02403fa8cc7397b10945f2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:55.450299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922dd9d-6430-4c9f-b65e-df01ae92a765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:55.450983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922dd9d-6430-4c9f-b65e-df01ae92a765 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:55.451859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922dd9d-6430-4c9f-b65e-df01ae92a765 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:55.452155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922dd9d-6430-4c9f-b65e-df01ae92a765 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:55.452824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922dd9d-6430-4c9f-b65e-df01ae92a765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:55.454987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922dd9d-6430-4c9f-b65e-df01ae92a765 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:55.463724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e4bb9fa1-28b2-4153-b713-c4795c3325bc tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '134961015f02403fa8cc7397b10945f2', 'request_id': 'req-e4bb9fa1-28b2-4153-b713-c4795c3325bc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:55.468419 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3278/6616] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:55 2022] GET /identity/v3/roles => generated 1649 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:55.471734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922dd9d-6430-4c9f-b65e-df01ae92a765 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:55.488274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922dd9d-6430-4c9f-b65e-df01ae92a765 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:55.494640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e4bb9fa1-28b2-4153-b713-c4795c3325bc tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '134961015f02403fa8cc7397b10945f2', 'request_id': 'req-e4bb9fa1-28b2-4153-b713-c4795c3325bc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:55.495583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:55.505851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e4bb9fa1-28b2-4153-b713-c4795c3325bc tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '134961015f02403fa8cc7397b10945f2', 'request_id': 'req-e4bb9fa1-28b2-4153-b713-c4795c3325bc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:55.529405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e4bb9fa1-28b2-4153-b713-c4795c3325bc tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] The token cache is being invalidate because project 134961015f02403fa8cc7397b10945f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:55.529701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e4bb9fa1-28b2-4153-b713-c4795c3325bc tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:55.531239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e4bb9fa1-28b2-4153-b713-c4795c3325bc tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:55.533973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:UPDATE=1,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:55.549206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e4bb9fa1-28b2-4153-b713-c4795c3325bc tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '134961015f02403fa8cc7397b10945f2', 'request_id': 'req-e4bb9fa1-28b2-4153-b713-c4795c3325bc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:55.550067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e4bb9fa1-28b2-4153-b713-c4795c3325bc tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '134961015f02403fa8cc7397b10945f2', 'request_id': 'req-e4bb9fa1-28b2-4153-b713-c4795c3325bc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:55.550912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e4bb9fa1-28b2-4153-b713-c4795c3325bc tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '134961015f02403fa8cc7397b10945f2', 'request_id': 'req-e4bb9fa1-28b2-4153-b713-c4795c3325bc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:55.580534 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3339/6617] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:13:55 2022] DELETE /identity/v3/domains/134961015f02403fa8cc7397b10945f2 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:55.584758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb9fa1-28b2-4153-b713-c4795c3325bc tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:55.590679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e4bb9fa1-28b2-4153-b713-c4795c3325bc tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:55.619309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc9e3e-d860-489a-a31e-f0bd0a6a4d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:55.620148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc9e3e-d860-489a-a31e-f0bd0a6a4d11 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:55.620441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc9e3e-d860-489a-a31e-f0bd0a6a4d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:55.621120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc9e3e-d860-489a-a31e-f0bd0a6a4d11 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:55.621563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc9e3e-d860-489a-a31e-f0bd0a6a4d11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:55.623487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc9e3e-d860-489a-a31e-f0bd0a6a4d11 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:55.637781 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3279/6618] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:55 2022] GET /identity/v3/roles => generated 1649 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:13:55.642736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc9e3e-d860-489a-a31e-f0bd0a6a4d11 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:55.645230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-46bc9e3e-d860-489a-a31e-f0bd0a6a4d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:55.661866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc9e3e-d860-489a-a31e-f0bd0a6a4d11 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:55.667210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf47655-b066-4da1-87b1-3dca442053e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:55.667962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf47655-b066-4da1-87b1-3dca442053e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:55.668312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf47655-b066-4da1-87b1-3dca442053e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:55.668657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf47655-b066-4da1-87b1-3dca442053e7 admin admin] PATH_INFO: `/v3/projects/a2c9800616974afa9c35148898db5316/users/bf76b289d770472cb0141e987a3eeb2c/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:55.698261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf47655-b066-4da1-87b1-3dca442053e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2c9800616974afa9c35148898db5316, user_id=bf76b289d770472cb0141e987a3eeb2c, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:55.702470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf47655-b066-4da1-87b1-3dca442053e7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:55.739946 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3280/6619] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:55 2022] PUT /identity/v3/projects/a2c9800616974afa9c35148898db5316/users/bf76b289d770472cb0141e987a3eeb2c/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:55.741665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0964a-cc48-4316-85e7-968e369bcadc None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:55.742183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0964a-cc48-4316-85e7-968e369bcadc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:55.742619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0964a-cc48-4316-85e7-968e369bcadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:55.762666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb9fa1-28b2-4153-b713-c4795c3325bc tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:55.802932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-31a0964a-cc48-4316-85e7-968e369bcadc None None] MFA Rules not processed for user `bf76b289d770472cb0141e987a3eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:55.808292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-31a0964a-cc48-4316-85e7-968e369bcadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:55.849156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f2a1d-aa7b-4d38-b559-c6e9d649ab40 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:55.850085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f2a1d-aa7b-4d38-b559-c6e9d649ab40 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:55.851224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f2a1d-aa7b-4d38-b559-c6e9d649ab40 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:55.851362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f2a1d-aa7b-4d38-b559-c6e9d649ab40 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains/cb0cf8cf2a9c43cd80a9746768468584/users/eb153aa122af4ab9a652798bee5516ee/roles/f30f45eafbdd4adc8ea84540d367735d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:55.856786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f2a1d-aa7b-4d38-b559-c6e9d649ab40 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=cb0cf8cf2a9c43cd80a9746768468584, user_id=eb153aa122af4ab9a652798bee5516ee, role_id=f30f45eafbdd4adc8ea84540d367735d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:55.859608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f2a1d-aa7b-4d38-b559-c6e9d649ab40 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:55.866790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-412f2a1d-aa7b-4d38-b559-c6e9d649ab40 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invalidating the token cache because role f30f45eafbdd4adc8ea84540d367735d was removed from user eb153aa122af4ab9a652798bee5516ee on domain cb0cf8cf2a9c43cd80a9746768468584. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:55.867095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-412f2a1d-aa7b-4d38-b559-c6e9d649ab40 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:55.868833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-412f2a1d-aa7b-4d38-b559-c6e9d649ab40 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:55.897204 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3340/6620] 10.222.0.17 () {58 vars in 1464 bytes} [Sat Oct 1 21:13:55 2022] DELETE /identity/v3/domains/cb0cf8cf2a9c43cd80a9746768468584/users/eb153aa122af4ab9a652798bee5516ee/roles/f30f45eafbdd4adc8ea84540d367735d => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:55.898716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f2a1d-aa7b-4d38-b559-c6e9d649ab40 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:55.900859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-412f2a1d-aa7b-4d38-b559-c6e9d649ab40 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:55.921455 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3281/6621] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:55.932527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0964a-cc48-4316-85e7-968e369bcadc None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:55.934847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-31a0964a-cc48-4316-85e7-968e369bcadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:55.981942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f2a1d-aa7b-4d38-b559-c6e9d649ab40 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:55.987518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec72552-2879-410e-9121-785a15b20b06 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:55.988294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec72552-2879-410e-9121-785a15b20b06 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:55.988591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec72552-2879-410e-9121-785a15b20b06 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:55.988863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec72552-2879-410e-9121-785a15b20b06 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/roles/f30f45eafbdd4adc8ea84540d367735d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:55.990329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec72552-2879-410e-9121-785a15b20b06 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:delete_role(role_id=f30f45eafbdd4adc8ea84540d367735d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:55.991958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec72552-2879-410e-9121-785a15b20b06 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:56.011958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=7,keystone:SELECT=93 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:56.013018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0964a-cc48-4316-85e7-968e369bcadc None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:56.047881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6ec72552-2879-410e-9121-785a15b20b06 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invalidating the token cache because role f30f45eafbdd4adc8ea84540d367735d has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:56.048484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6ec72552-2879-410e-9121-785a15b20b06 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:56.049669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6ec72552-2879-410e-9121-785a15b20b06 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:56.057818 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3341/6622] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:55 2022] DELETE /identity/v3/roles/f30f45eafbdd4adc8ea84540d367735d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:56.059295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec72552-2879-410e-9121-785a15b20b06 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:56.061017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6ec72552-2879-410e-9121-785a15b20b06 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:56.067361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:56.113068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768cad3-7713-46aa-a776-1178efaa8000 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:56.113767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768cad3-7713-46aa-a776-1178efaa8000 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:56.114173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768cad3-7713-46aa-a776-1178efaa8000 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:56.114515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768cad3-7713-46aa-a776-1178efaa8000 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:56.116671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4768cad3-7713-46aa-a776-1178efaa8000 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:56.139924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec72552-2879-410e-9121-785a15b20b06 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:56.145312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7e8b7-caed-4c77-ab17-55fbdb347ca0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:56.145953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7e8b7-caed-4c77-ab17-55fbdb347ca0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:56.146238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7e8b7-caed-4c77-ab17-55fbdb347ca0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:56.147029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7e8b7-caed-4c77-ab17-55fbdb347ca0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains/cb0cf8cf2a9c43cd80a9746768468584` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:56.148691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab7e8b7-caed-4c77-ab17-55fbdb347ca0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=cb0cf8cf2a9c43cd80a9746768468584)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:56.150320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab7e8b7-caed-4c77-ab17-55fbdb347ca0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:56.174337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dab7e8b7-caed-4c77-ab17-55fbdb347ca0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'cb0cf8cf2a9c43cd80a9746768468584'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:56.175069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dab7e8b7-caed-4c77-ab17-55fbdb347ca0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'cb0cf8cf2a9c43cd80a9746768468584'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:56.179556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4768cad3-7713-46aa-a776-1178efaa8000 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:56.181576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4768cad3-7713-46aa-a776-1178efaa8000 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:56.191745 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3282/6623] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:55 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:56.201533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dab7e8b7-caed-4c77-ab17-55fbdb347ca0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'cb0cf8cf2a9c43cd80a9746768468584'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:56.202165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dab7e8b7-caed-4c77-ab17-55fbdb347ca0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'cb0cf8cf2a9c43cd80a9746768468584'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:56.207703 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3342/6624] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:13:56 2022] PATCH /identity/v3/domains/cb0cf8cf2a9c43cd80a9746768468584 => generated 278 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:13:56.208852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768cad3-7713-46aa-a776-1178efaa8000 service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:56.220377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768cad3-7713-46aa-a776-1178efaa8000 service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:56.280975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:56.281624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:56.281910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:56.282189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains/cb0cf8cf2a9c43cd80a9746768468584` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:56.296764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=cb0cf8cf2a9c43cd80a9746768468584)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:56.298423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:56.327567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'cb0cf8cf2a9c43cd80a9746768468584'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:56.341652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'cb0cf8cf2a9c43cd80a9746768468584'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:56.400302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:56.402267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb0cf8cf2a9c43cd80a9746768468584', 'request_id': 'req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:56.413012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb0cf8cf2a9c43cd80a9746768468584', 'request_id': 'req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:56.421416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb0cf8cf2a9c43cd80a9746768468584', 'request_id': 'req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:56.443422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] The token cache is being invalidate because project cb0cf8cf2a9c43cd80a9746768468584 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:56.443898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:56.445090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:56.460581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'cb0cf8cf2a9c43cd80a9746768468584', 'request_id': 'req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:56.461910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'cb0cf8cf2a9c43cd80a9746768468584', 'request_id': 'req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:56.467702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'cb0cf8cf2a9c43cd80a9746768468584', 'request_id': 'req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:56.479756 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3283/6625] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:13:56 2022] DELETE /identity/v3/domains/cb0cf8cf2a9c43cd80a9746768468584 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:56.488158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7e8b7-caed-4c77-ab17-55fbdb347ca0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:56.490812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-dab7e8b7-caed-4c77-ab17-55fbdb347ca0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:56.518626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7e8b7-caed-4c77-ab17-55fbdb347ca0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:56.666257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de555a-7200-4b8c-8f5f-3bd52c203a2e tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:56.667006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de555a-7200-4b8c-8f5f-3bd52c203a2e tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:56.667667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de555a-7200-4b8c-8f5f-3bd52c203a2e tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:56.668011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de555a-7200-4b8c-8f5f-3bd52c203a2e tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:56.669792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de555a-7200-4b8c-8f5f-3bd52c203a2e tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:56.671831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de555a-7200-4b8c-8f5f-3bd52c203a2e tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:56.706531 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3343/6626] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:13:56.709762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:56.712428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:56.728953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac3f1a-f86b-4bbe-91da-98c01bd49eb8 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:56.740918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddb570-d0f7-4bb4-8068-c01726087220 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:56.741706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddb570-d0f7-4bb4-8068-c01726087220 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:56.742060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddb570-d0f7-4bb4-8068-c01726087220 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:56.742460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddb570-d0f7-4bb4-8068-c01726087220 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/projects/5946f2c858e54d42ab4f184fd2f4d87c/users/42c2d21ecbfb4378a6ea22b3d7d294d9/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:56.745755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdddb570-d0f7-4bb4-8068-c01726087220 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_grant(project_id=5946f2c858e54d42ab4f184fd2f4d87c, user_id=42c2d21ecbfb4378a6ea22b3d7d294d9, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:56.749684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdddb570-d0f7-4bb4-8068-c01726087220 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:56.768865 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3284/6627] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:56 2022] PUT /identity/v3/projects/5946f2c858e54d42ab4f184fd2f4d87c/users/42c2d21ecbfb4378a6ea22b3d7d294d9/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:56.773106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de555a-7200-4b8c-8f5f-3bd52c203a2e tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:56.784134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de555a-7200-4b8c-8f5f-3bd52c203a2e tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:56.836048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:UPDATE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:56.864770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da0aad-253b-4b09-bcb5-c7df61786d0a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:56.865728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da0aad-253b-4b09-bcb5-c7df61786d0a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:56.866278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da0aad-253b-4b09-bcb5-c7df61786d0a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:56.870913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da0aad-253b-4b09-bcb5-c7df61786d0a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/auth/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:56.871866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da0aad-253b-4b09-bcb5-c7df61786d0a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:get_auth_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:56.873062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da0aad-253b-4b09-bcb5-c7df61786d0a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:56.965378 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3344/6628] 10.222.0.17 () {58 vars in 1146 bytes} [Sat Oct 1 21:13:56 2022] GET /identity/v3/auth/projects => generated 826 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:13:56.966820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddb570-d0f7-4bb4-8068-c01726087220 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:56.979776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddb570-d0f7-4bb4-8068-c01726087220 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:56.984701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e99a26-4287-4791-86f8-b851fd6d9691 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:56.985525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e99a26-4287-4791-86f8-b851fd6d9691 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:56.985809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e99a26-4287-4791-86f8-b851fd6d9691 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:56.986090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e99a26-4287-4791-86f8-b851fd6d9691 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/projects/5946f2c858e54d42ab4f184fd2f4d87c/users/42c2d21ecbfb4378a6ea22b3d7d294d9/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:56.988061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e99a26-4287-4791-86f8-b851fd6d9691 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=5946f2c858e54d42ab4f184fd2f4d87c, user_id=42c2d21ecbfb4378a6ea22b3d7d294d9, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:56.990839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e99a26-4287-4791-86f8-b851fd6d9691 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:56.997969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-43e99a26-4287-4791-86f8-b851fd6d9691 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invalidating the token cache because role 1c50896511054a5f994bee3d99fa0619 was removed from user 42c2d21ecbfb4378a6ea22b3d7d294d9 on project 5946f2c858e54d42ab4f184fd2f4d87c. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:56.998268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-43e99a26-4287-4791-86f8-b851fd6d9691 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:56.999010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-43e99a26-4287-4791-86f8-b851fd6d9691 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:57.018077 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3285/6629] 10.222.0.17 () {58 vars in 1467 bytes} [Sat Oct 1 21:13:56 2022] DELETE /identity/v3/projects/5946f2c858e54d42ab4f184fd2f4d87c/users/42c2d21ecbfb4378a6ea22b3d7d294d9/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:57.025159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da0aad-253b-4b09-bcb5-c7df61786d0a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:57.029071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-26da0aad-253b-4b09-bcb5-c7df61786d0a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:57.054892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da0aad-253b-4b09-bcb5-c7df61786d0a tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:57.142964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64896aa3-185c-4bac-bac3-0d40036ae3cf tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:57.143532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64896aa3-185c-4bac-bac3-0d40036ae3cf tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:57.143807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64896aa3-185c-4bac-bac3-0d40036ae3cf tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:57.144084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64896aa3-185c-4bac-bac3-0d40036ae3cf tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:57.145361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64896aa3-185c-4bac-bac3-0d40036ae3cf tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:57.147322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64896aa3-185c-4bac-bac3-0d40036ae3cf tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:57.172208 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3345/6630] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:13:57 2022] POST /identity/v3/users => generated 321 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:57.173541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e99a26-4287-4791-86f8-b851fd6d9691 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:57.187639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e99a26-4287-4791-86f8-b851fd6d9691 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:57.193564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7f564-5c32-4933-a4c3-1bd9b6803b12 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:57.194446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7f564-5c32-4933-a4c3-1bd9b6803b12 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:57.194902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7f564-5c32-4933-a4c3-1bd9b6803b12 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:57.195372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7f564-5c32-4933-a4c3-1bd9b6803b12 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:57.196702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7f564-5c32-4933-a4c3-1bd9b6803b12 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:57.199852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7f564-5c32-4933-a4c3-1bd9b6803b12 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:57.233952 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3286/6631] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:57 2022] POST /identity/v3/projects => generated 362 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:57.235177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64896aa3-185c-4bac-bac3-0d40036ae3cf tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:57.246305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64896aa3-185c-4bac-bac3-0d40036ae3cf tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:57.308330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881e3fe-3781-41b8-932b-552bfff4cf33 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:57.308943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881e3fe-3781-41b8-932b-552bfff4cf33 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:57.309222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881e3fe-3781-41b8-932b-552bfff4cf33 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:57.309495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881e3fe-3781-41b8-932b-552bfff4cf33 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:57.310118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881e3fe-3781-41b8-932b-552bfff4cf33 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:57.312033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881e3fe-3781-41b8-932b-552bfff4cf33 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:57.340270 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3346/6632] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:13:57 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:57.344575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7f564-5c32-4933-a4c3-1bd9b6803b12 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:57.355428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7f564-5c32-4933-a4c3-1bd9b6803b12 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:57.414061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3012a-6b29-4d8e-85b8-451f5ac9c828 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:57.414646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3012a-6b29-4d8e-85b8-451f5ac9c828 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:57.415471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3012a-6b29-4d8e-85b8-451f5ac9c828 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:57.415751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3012a-6b29-4d8e-85b8-451f5ac9c828 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:57.416409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3012a-6b29-4d8e-85b8-451f5ac9c828 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:57.418015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3012a-6b29-4d8e-85b8-451f5ac9c828 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:57.434101 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3287/6633] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:13:57 2022] POST /identity/v3/roles => generated 239 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:13:57.438117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881e3fe-3781-41b8-932b-552bfff4cf33 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:57.448988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881e3fe-3781-41b8-932b-552bfff4cf33 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:57.453632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1d31d-d7e0-43fc-bfee-cada8e27dbf7 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:57.454290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1d31d-d7e0-43fc-bfee-cada8e27dbf7 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:57.454599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1d31d-d7e0-43fc-bfee-cada8e27dbf7 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:57.454898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1d31d-d7e0-43fc-bfee-cada8e27dbf7 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/projects/55673eb17a06402a96cc28fabe9e61f7/users/7e11b8949a244530918efb30d612677a/roles/afb6c0e5d58b42f18bbe66c0bfe7374e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:57.470386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a1d31d-d7e0-43fc-bfee-cada8e27dbf7 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_grant(project_id=55673eb17a06402a96cc28fabe9e61f7, user_id=7e11b8949a244530918efb30d612677a, role_id=afb6c0e5d58b42f18bbe66c0bfe7374e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:57.473057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a1d31d-d7e0-43fc-bfee-cada8e27dbf7 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:57.482520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:57.488742 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3347/6634] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:57 2022] PUT /identity/v3/projects/55673eb17a06402a96cc28fabe9e61f7/users/7e11b8949a244530918efb30d612677a/roles/afb6c0e5d58b42f18bbe66c0bfe7374e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:57.492918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3012a-6b29-4d8e-85b8-451f5ac9c828 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:57.503672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3012a-6b29-4d8e-85b8-451f5ac9c828 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:57.553973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:57.561166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2b3db-2da9-443d-8b7d-ea94ff6ffe56 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:57.561866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2b3db-2da9-443d-8b7d-ea94ff6ffe56 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:57.562168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2b3db-2da9-443d-8b7d-ea94ff6ffe56 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:57.562835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2b3db-2da9-443d-8b7d-ea94ff6ffe56 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/projects/742ff5907abf4267ad132311bd5d0b2f/users/7e11b8949a244530918efb30d612677a/roles/afb6c0e5d58b42f18bbe66c0bfe7374e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:57.565465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2b3db-2da9-443d-8b7d-ea94ff6ffe56 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:create_grant(project_id=742ff5907abf4267ad132311bd5d0b2f, user_id=7e11b8949a244530918efb30d612677a, role_id=afb6c0e5d58b42f18bbe66c0bfe7374e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:57.568472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2b3db-2da9-443d-8b7d-ea94ff6ffe56 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:57.580997 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3288/6635] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:13:57 2022] PUT /identity/v3/projects/742ff5907abf4267ad132311bd5d0b2f/users/7e11b8949a244530918efb30d612677a/roles/afb6c0e5d58b42f18bbe66c0bfe7374e => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:57.585447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cccf59-24cd-43f6-9762-f227d35808d8 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:57.585733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cccf59-24cd-43f6-9762-f227d35808d8 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:57.586034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cccf59-24cd-43f6-9762-f227d35808d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:57.613171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f9cccf59-24cd-43f6-9762-f227d35808d8 None None] MFA Rules not processed for user `7e11b8949a244530918efb30d612677a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:57.614407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f9cccf59-24cd-43f6-9762-f227d35808d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:57.617823 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3348/6636] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:57 2022] POST /identity/v3/auth/tokens => generated 334 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 01 21:13:57.622119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb2bfe-0ae6-4871-a137-ee8920f930d8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:57.622585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb2bfe-0ae6-4871-a137-ee8920f930d8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:57.622983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb2bfe-0ae6-4871-a137-ee8920f930d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:57.635561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7adb2bfe-0ae6-4871-a137-ee8920f930d8 None None] MFA Rules not processed for user `7e11b8949a244530918efb30d612677a`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:57.636772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7adb2bfe-0ae6-4871-a137-ee8920f930d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:57.704466 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3289/6637] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:57 2022] POST /identity/v3/auth/tokens => generated 3223 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Oct 01 21:13:57.708401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cccf59-24cd-43f6-9762-f227d35808d8 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:57.770572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb2bfe-0ae6-4871-a137-ee8920f930d8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:57.770572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7adb2bfe-0ae6-4871-a137-ee8920f930d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:57.795958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cccf59-24cd-43f6-9762-f227d35808d8 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:57.802202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1cf77-5b31-4b4a-91c9-2268ff0d792b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:57.803059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1cf77-5b31-4b4a-91c9-2268ff0d792b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:57.803509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1cf77-5b31-4b4a-91c9-2268ff0d792b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:57.804576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1cf77-5b31-4b4a-91c9-2268ff0d792b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:57.817774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1cf77-5b31-4b4a-91c9-2268ff0d792b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:revoke_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:57.821688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1cf77-5b31-4b4a-91c9-2268ff0d792b tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:57.847993 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3349/6638] 10.222.0.17 () {56 vars in 1308 bytes} [Sat Oct 1 21:13:57 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:57.855886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb2bfe-0ae6-4871-a137-ee8920f930d8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:57.859361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0789f-30e1-4a49-91db-df2ca1c76659 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:57.859659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0789f-30e1-4a49-91db-df2ca1c76659 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:57.859790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09d90d-2263-4e8f-ba38-8e5d805ab57f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:57.859928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0789f-30e1-4a49-91db-df2ca1c76659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:57.860333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09d90d-2263-4e8f-ba38-8e5d805ab57f service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:57.860615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09d90d-2263-4e8f-ba38-8e5d805ab57f service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:57.860886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09d90d-2263-4e8f-ba38-8e5d805ab57f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:57.862991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4a09d90d-2263-4e8f-ba38-8e5d805ab57f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:57.878226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-78b0789f-30e1-4a49-91db-df2ca1c76659 None None] MFA Rules not processed for user `7e11b8949a244530918efb30d612677a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:57.879915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-78b0789f-30e1-4a49-91db-df2ca1c76659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:57.961909 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3350/6639] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:57 2022] POST /identity/v3/auth/tokens => generated 3223 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Oct 01 21:13:57.966772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0789f-30e1-4a49-91db-df2ca1c76659 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:57.983206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0789f-30e1-4a49-91db-df2ca1c76659 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:57.989948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81571ad4-46ea-4b94-87f0-e49ff1b5c23d tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:57.990805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81571ad4-46ea-4b94-87f0-e49ff1b5c23d tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:57.991124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81571ad4-46ea-4b94-87f0-e49ff1b5c23d tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:57.991345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81571ad4-46ea-4b94-87f0-e49ff1b5c23d tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/roles/afb6c0e5d58b42f18bbe66c0bfe7374e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:57.993676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81571ad4-46ea-4b94-87f0-e49ff1b5c23d tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:delete_role(role_id=afb6c0e5d58b42f18bbe66c0bfe7374e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:57.994381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09d90d-2263-4e8f-ba38-8e5d805ab57f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:57.995432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81571ad4-46ea-4b94-87f0-e49ff1b5c23d tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:57.996198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09d90d-2263-4e8f-ba38-8e5d805ab57f service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:58.012631 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3290/6640] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:13:58.054220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-81571ad4-46ea-4b94-87f0-e49ff1b5c23d tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invalidating the token cache because role afb6c0e5d58b42f18bbe66c0bfe7374e has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:58.054693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-81571ad4-46ea-4b94-87f0-e49ff1b5c23d tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:58.055400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-81571ad4-46ea-4b94-87f0-e49ff1b5c23d tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:58.058845 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3351/6641] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:57 2022] DELETE /identity/v3/roles/afb6c0e5d58b42f18bbe66c0bfe7374e => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:58.063245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09d90d-2263-4e8f-ba38-8e5d805ab57f service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:58.075325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09d90d-2263-4e8f-ba38-8e5d805ab57f service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:58.117887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:58.157147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de5b23-6c97-4239-8076-89c96fde3a51 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:58.157805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de5b23-6c97-4239-8076-89c96fde3a51 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:58.158063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de5b23-6c97-4239-8076-89c96fde3a51 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:58.158337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de5b23-6c97-4239-8076-89c96fde3a51 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/projects/742ff5907abf4267ad132311bd5d0b2f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:58.162720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de5b23-6c97-4239-8076-89c96fde3a51 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:delete_project(project_id=742ff5907abf4267ad132311bd5d0b2f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:58.165375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de5b23-6c97-4239-8076-89c96fde3a51 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:58.248154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32de5b23-6c97-4239-8076-89c96fde3a51 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '742ff5907abf4267ad132311bd5d0b2f', 'request_id': 'req-32de5b23-6c97-4239-8076-89c96fde3a51'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:58.268179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32de5b23-6c97-4239-8076-89c96fde3a51 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '742ff5907abf4267ad132311bd5d0b2f', 'request_id': 'req-32de5b23-6c97-4239-8076-89c96fde3a51'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:58.280893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32de5b23-6c97-4239-8076-89c96fde3a51 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '742ff5907abf4267ad132311bd5d0b2f', 'request_id': 'req-32de5b23-6c97-4239-8076-89c96fde3a51'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:58.295263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32de5b23-6c97-4239-8076-89c96fde3a51 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] The token cache is being invalidate because project 742ff5907abf4267ad132311bd5d0b2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:58.295571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32de5b23-6c97-4239-8076-89c96fde3a51 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:58.296249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32de5b23-6c97-4239-8076-89c96fde3a51 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:58.298042 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3291/6642] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:58 2022] DELETE /identity/v3/projects/742ff5907abf4267ad132311bd5d0b2f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:58.302176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81571ad4-46ea-4b94-87f0-e49ff1b5c23d tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:58.303934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-81571ad4-46ea-4b94-87f0-e49ff1b5c23d tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:58.318375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81571ad4-46ea-4b94-87f0-e49ff1b5c23d tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:58.364936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:58.396750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a8459-0800-4891-a15a-aa06c1b4a2f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:58.397006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a8459-0800-4891-a15a-aa06c1b4a2f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:58.397332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a8459-0800-4891-a15a-aa06c1b4a2f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:58.397616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a8459-0800-4891-a15a-aa06c1b4a2f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/projects/742ff5907abf4267ad132311bd5d0b2f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:58.406806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6a8459-0800-4891-a15a-aa06c1b4a2f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:delete_project(project_id=742ff5907abf4267ad132311bd5d0b2f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:58.409139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6a8459-0800-4891-a15a-aa06c1b4a2f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:58.423126 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-4e6a8459-0800-4891-a15a-aa06c1b4a2f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Could not find project: 742ff5907abf4267ad132311bd5d0b2f.: keystone.exception.ProjectNotFound: Could not find project: 742ff5907abf4267ad132311bd5d0b2f. Oct 01 21:13:58.423126 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:13:58.423126 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:13:58.423126 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:13:58.423126 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:13:58.423126 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:13:58.423126 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:13:58.423126 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:13:58.423126 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:13:58.423126 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:13:58.423126 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:13:58.423126 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:13:58.423126 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 211, in delete Oct 01 21:13:58.423126 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_project( Oct 01 21:13:58.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:13:58.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:13:58.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 478, in delete_project Oct 01 21:13:58.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = self.driver.get_project(project_id) Oct 01 21:13:58.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 21:13:58.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 21:13:58.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 21:13:58.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 21:13:58.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 742ff5907abf4267ad132311bd5d0b2f. Oct 01 21:13:58.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:13:58.424969 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3352/6643] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:58 2022] DELETE /identity/v3/projects/742ff5907abf4267ad132311bd5d0b2f => generated 113 bytes in 123 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:13:58.428671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de5b23-6c97-4239-8076-89c96fde3a51 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:58.430232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-32de5b23-6c97-4239-8076-89c96fde3a51 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:58.444437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de5b23-6c97-4239-8076-89c96fde3a51 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:58.450751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df67e8-5d62-48eb-9d9f-a87491f261b5 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:58.451395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df67e8-5d62-48eb-9d9f-a87491f261b5 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:58.451681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df67e8-5d62-48eb-9d9f-a87491f261b5 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:58.451965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df67e8-5d62-48eb-9d9f-a87491f261b5 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/projects/55673eb17a06402a96cc28fabe9e61f7` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:58.453226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df67e8-5d62-48eb-9d9f-a87491f261b5 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:delete_project(project_id=55673eb17a06402a96cc28fabe9e61f7)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:58.455513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df67e8-5d62-48eb-9d9f-a87491f261b5 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:58.515478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f5df67e8-5d62-48eb-9d9f-a87491f261b5 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55673eb17a06402a96cc28fabe9e61f7', 'request_id': 'req-f5df67e8-5d62-48eb-9d9f-a87491f261b5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:58.524970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f5df67e8-5d62-48eb-9d9f-a87491f261b5 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55673eb17a06402a96cc28fabe9e61f7', 'request_id': 'req-f5df67e8-5d62-48eb-9d9f-a87491f261b5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:58.532126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f5df67e8-5d62-48eb-9d9f-a87491f261b5 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55673eb17a06402a96cc28fabe9e61f7', 'request_id': 'req-f5df67e8-5d62-48eb-9d9f-a87491f261b5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:58.546226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f5df67e8-5d62-48eb-9d9f-a87491f261b5 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] The token cache is being invalidate because project 55673eb17a06402a96cc28fabe9e61f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:58.546682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f5df67e8-5d62-48eb-9d9f-a87491f261b5 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:58.547582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f5df67e8-5d62-48eb-9d9f-a87491f261b5 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:58.549977 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3292/6644] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:58 2022] DELETE /identity/v3/projects/55673eb17a06402a96cc28fabe9e61f7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:13:58.554406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a8459-0800-4891-a15a-aa06c1b4a2f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:58.557184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4e6a8459-0800-4891-a15a-aa06c1b4a2f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:58.570923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a8459-0800-4891-a15a-aa06c1b4a2f0 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:58.640927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb170bc3-61e9-495d-a6d3-a348097867da tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:58.641618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb170bc3-61e9-495d-a6d3-a348097867da tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:58.641901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb170bc3-61e9-495d-a6d3-a348097867da tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:58.642174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb170bc3-61e9-495d-a6d3-a348097867da tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/users/7e11b8949a244530918efb30d612677a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:58.643707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb170bc3-61e9-495d-a6d3-a348097867da tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorizing `identity:delete_user(user_id=7e11b8949a244530918efb30d612677a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:58.645843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb170bc3-61e9-495d-a6d3-a348097867da tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:58.716101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fb170bc3-61e9-495d-a6d3-a348097867da tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e11b8949a244530918efb30d612677a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:58.724529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fb170bc3-61e9-495d-a6d3-a348097867da tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e11b8949a244530918efb30d612677a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:58.740867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fb170bc3-61e9-495d-a6d3-a348097867da tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e11b8949a244530918efb30d612677a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:58.741715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fb170bc3-61e9-495d-a6d3-a348097867da tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e11b8949a244530918efb30d612677a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:58.755780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fb170bc3-61e9-495d-a6d3-a348097867da tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e11b8949a244530918efb30d612677a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:58.768170 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3353/6645] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:58 2022] DELETE /identity/v3/users/7e11b8949a244530918efb30d612677a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:58.774525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df67e8-5d62-48eb-9d9f-a87491f261b5 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:58.785957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df67e8-5d62-48eb-9d9f-a87491f261b5 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:58.833225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:58.850742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26a7e2-65ce-4e3d-a5a7-13bcfdd156b9 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:58.851381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26a7e2-65ce-4e3d-a5a7-13bcfdd156b9 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:58.851668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26a7e2-65ce-4e3d-a5a7-13bcfdd156b9 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:58.851944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26a7e2-65ce-4e3d-a5a7-13bcfdd156b9 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains/5362b3c879354830a63155868fb44159` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:58.859548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26a7e2-65ce-4e3d-a5a7-13bcfdd156b9 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 21:13:58.864476 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-bf26a7e2-65ce-4e3d-a5a7-13bcfdd156b9 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Could not find domain: 5362b3c879354830a63155868fb44159.: keystone.exception.DomainNotFound: Could not find domain: 5362b3c879354830a63155868fb44159. Oct 01 21:13:58.864476 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:13:58.864476 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 01 21:13:58.864476 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 01 21:13:58.864476 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 21:13:58.864476 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 21:13:58.864476 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 21:13:58.864476 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 21:13:58.864476 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 5362b3c879354830a63155868fb44159. Oct 01 21:13:58.864476 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:13:58.864476 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:13:58.864476 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:13:58.864476 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:13:58.864476 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:13:58.864476 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:13:58.864476 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:13:58.865360 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:13:58.865360 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:13:58.865360 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:13:58.865360 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:13:58.865360 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:13:58.865360 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:13:58.865360 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:13:58.865360 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 01 21:13:58.865360 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 01 21:13:58.865360 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 21:13:58.865360 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 21:13:58.865360 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 21:13:58.865360 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 21:13:58.865360 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:13:58.865360 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:13:58.865360 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:13:58.867140 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:13:58.867140 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:13:58.867140 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:13:58.867140 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:13:58.867140 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 21:13:58.867140 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:13:58.867140 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:13:58.867140 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:13:58.867140 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:13:58.867140 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:13:58.867140 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:13:58.867140 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:13:58.867140 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:13:58.867140 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 01 21:13:58.867140 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 21:13:58.867140 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 5362b3c879354830a63155868fb44159. Oct 01 21:13:58.867140 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:13:58.868670 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3293/6646] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:13:58 2022] PATCH /identity/v3/domains/5362b3c879354830a63155868fb44159 => generated 112 bytes in 92 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:13:58.870185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb170bc3-61e9-495d-a6d3-a348097867da tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:58.871781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fb170bc3-61e9-495d-a6d3-a348097867da tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:58.885125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb170bc3-61e9-495d-a6d3-a348097867da tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:58.889972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47584e9-ca91-472f-9d29-cfa7a338980e tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:58.890631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47584e9-ca91-472f-9d29-cfa7a338980e tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:58.890914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47584e9-ca91-472f-9d29-cfa7a338980e tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:58.891218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47584e9-ca91-472f-9d29-cfa7a338980e tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains/c0f0fa967587446cb0bfc99bdeabc526` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:58.898222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47584e9-ca91-472f-9d29-cfa7a338980e tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 21:13:58.903118 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-a47584e9-ca91-472f-9d29-cfa7a338980e tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Could not find domain: c0f0fa967587446cb0bfc99bdeabc526.: keystone.exception.DomainNotFound: Could not find domain: c0f0fa967587446cb0bfc99bdeabc526. Oct 01 21:13:58.903118 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:13:58.903118 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 01 21:13:58.903118 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 01 21:13:58.903118 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 21:13:58.903118 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 21:13:58.903118 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 21:13:58.903118 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 21:13:58.903118 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: c0f0fa967587446cb0bfc99bdeabc526. Oct 01 21:13:58.903118 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:13:58.903118 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:13:58.903118 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:13:58.903118 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:13:58.903118 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:13:58.903118 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:13:58.903118 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:13:58.904163 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:13:58.904163 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:13:58.904163 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:13:58.904163 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:13:58.904163 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:13:58.904163 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:13:58.904163 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:13:58.904163 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 01 21:13:58.904163 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 01 21:13:58.904163 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 21:13:58.904163 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 21:13:58.904163 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 21:13:58.904163 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 21:13:58.904163 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:13:58.904163 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:13:58.904163 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:13:58.904963 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:13:58.904963 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:13:58.904963 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:13:58.904963 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:13:58.904963 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 21:13:58.904963 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:13:58.904963 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:13:58.904963 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:13:58.904963 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:13:58.904963 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:13:58.904963 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:13:58.904963 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:13:58.904963 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:13:58.904963 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 01 21:13:58.904963 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 21:13:58.904963 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: c0f0fa967587446cb0bfc99bdeabc526. Oct 01 21:13:58.904963 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:13:58.905766 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3354/6647] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:13:58 2022] PATCH /identity/v3/domains/c0f0fa967587446cb0bfc99bdeabc526 => generated 112 bytes in 35 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:13:58.908383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26a7e2-65ce-4e3d-a5a7-13bcfdd156b9 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:58.919950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26a7e2-65ce-4e3d-a5a7-13bcfdd156b9 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:58.924839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28cffe-2591-4d97-8247-c0e9ede757ee tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:58.925560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28cffe-2591-4d97-8247-c0e9ede757ee tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:58.925742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28cffe-2591-4d97-8247-c0e9ede757ee tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:58.926018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28cffe-2591-4d97-8247-c0e9ede757ee tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains/134961015f02403fa8cc7397b10945f2` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:58.933781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28cffe-2591-4d97-8247-c0e9ede757ee tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 21:13:58.939058 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-db28cffe-2591-4d97-8247-c0e9ede757ee tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Could not find domain: 134961015f02403fa8cc7397b10945f2.: keystone.exception.DomainNotFound: Could not find domain: 134961015f02403fa8cc7397b10945f2. Oct 01 21:13:58.939058 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:13:58.939058 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 01 21:13:58.939058 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 01 21:13:58.939058 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 21:13:58.939058 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 21:13:58.939058 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 21:13:58.939058 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 21:13:58.939058 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 134961015f02403fa8cc7397b10945f2. Oct 01 21:13:58.939058 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:13:58.939058 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:13:58.939058 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:13:58.939058 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:13:58.939058 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:13:58.939058 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:13:58.939058 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:13:58.940197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:13:58.940197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:13:58.940197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:13:58.940197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:13:58.940197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:13:58.940197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:13:58.940197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:13:58.940197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 01 21:13:58.940197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 01 21:13:58.940197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 21:13:58.940197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 21:13:58.940197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 21:13:58.940197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 21:13:58.940197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:13:58.940197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:13:58.940197 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:13:58.941112 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:13:58.941112 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:13:58.941112 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:13:58.941112 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:13:58.941112 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 21:13:58.941112 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:13:58.941112 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:13:58.941112 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:13:58.941112 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:13:58.941112 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:13:58.941112 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:13:58.941112 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:13:58.941112 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:13:58.941112 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 01 21:13:58.941112 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 21:13:58.941112 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 134961015f02403fa8cc7397b10945f2. Oct 01 21:13:58.941112 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:13:58.942648 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3294/6648] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:13:58 2022] PATCH /identity/v3/domains/134961015f02403fa8cc7397b10945f2 => generated 112 bytes in 35 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:13:58.943937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47584e9-ca91-472f-9d29-cfa7a338980e tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:58.955310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47584e9-ca91-472f-9d29-cfa7a338980e tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:58.960104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54b602-7e1f-4091-974b-2ccf688189c5 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c2d21ecbfb4378a6ea22b3d7d294d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1c8aa3b1794a56b9120a7655d138b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:58.960713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54b602-7e1f-4091-974b-2ccf688189c5 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:58.960997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54b602-7e1f-4091-974b-2ccf688189c5 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:58.961275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54b602-7e1f-4091-974b-2ccf688189c5 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] PATH_INFO: `/v3/domains/cb0cf8cf2a9c43cd80a9746768468584` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:58.969781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54b602-7e1f-4091-974b-2ccf688189c5 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 21:13:58.975101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-1c54b602-7e1f-4091-974b-2ccf688189c5 tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Could not find domain: cb0cf8cf2a9c43cd80a9746768468584.: keystone.exception.DomainNotFound: Could not find domain: cb0cf8cf2a9c43cd80a9746768468584. Oct 01 21:13:58.975101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:13:58.975101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 01 21:13:58.975101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 01 21:13:58.975101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 21:13:58.975101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 21:13:58.975101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 21:13:58.975101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 21:13:58.975101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: cb0cf8cf2a9c43cd80a9746768468584. Oct 01 21:13:58.975101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:13:58.975101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:13:58.975101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:13:58.975101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:13:58.975101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:13:58.975101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:13:58.975101 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:13:58.976119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:13:58.976119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:13:58.976119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:13:58.976119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:13:58.976119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:13:58.976119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:13:58.976119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:13:58.976119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 01 21:13:58.976119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 01 21:13:58.976119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 21:13:58.976119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 21:13:58.976119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 21:13:58.976119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 21:13:58.976119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:13:58.976119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:13:58.976119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:13:58.979391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:13:58.979391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:13:58.979391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:13:58.979391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:13:58.979391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 21:13:58.979391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:13:58.979391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:13:58.979391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:13:58.979391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:13:58.979391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:13:58.979391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:13:58.979391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:13:58.979391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:13:58.979391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 01 21:13:58.979391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 21:13:58.979391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: cb0cf8cf2a9c43cd80a9746768468584. Oct 01 21:13:58.979391 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:13:58.980353 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3355/6649] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:13:58 2022] PATCH /identity/v3/domains/cb0cf8cf2a9c43cd80a9746768468584 => generated 112 bytes in 33 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Oct 01 21:13:58.980891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28cffe-2591-4d97-8247-c0e9ede757ee tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:58.982606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-db28cffe-2591-4d97-8247-c0e9ede757ee tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:59.097503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28cffe-2591-4d97-8247-c0e9ede757ee tempest-TokensV3TestJSON-791946367 tempest-TokensV3TestJSON-791946367-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:59.105208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c0dd4-4615-4408-82e7-6c4f52df7a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:59.105909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c0dd4-4615-4408-82e7-6c4f52df7a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:59.106788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c0dd4-4615-4408-82e7-6c4f52df7a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:59.106788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c0dd4-4615-4408-82e7-6c4f52df7a4d admin admin] PATH_INFO: `/v3/users/871645dc61df4f818082b21626ff2eb1` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:59.107742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c0dd4-4615-4408-82e7-6c4f52df7a4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=871645dc61df4f818082b21626ff2eb1)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:59.109780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c0dd4-4615-4408-82e7-6c4f52df7a4d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:59.201721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bb7c0dd4-4615-4408-82e7-6c4f52df7a4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '871645dc61df4f818082b21626ff2eb1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:59.202457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bb7c0dd4-4615-4408-82e7-6c4f52df7a4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '871645dc61df4f818082b21626ff2eb1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:59.203151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bb7c0dd4-4615-4408-82e7-6c4f52df7a4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '871645dc61df4f818082b21626ff2eb1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:59.214457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bb7c0dd4-4615-4408-82e7-6c4f52df7a4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '871645dc61df4f818082b21626ff2eb1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:59.234469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bb7c0dd4-4615-4408-82e7-6c4f52df7a4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '871645dc61df4f818082b21626ff2eb1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:59.261088 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3295/6650] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:58 2022] DELETE /identity/v3/users/871645dc61df4f818082b21626ff2eb1 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:59.262608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c938363-044e-4ea8-9f29-0af6eb503613 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:59.262916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c938363-044e-4ea8-9f29-0af6eb503613 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:59.263189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c938363-044e-4ea8-9f29-0af6eb503613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:59.305893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8c938363-044e-4ea8-9f29-0af6eb503613 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:59.307234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8c938363-044e-4ea8-9f29-0af6eb503613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:59.337114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:59.365191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c0dd4-4615-4408-82e7-6c4f52df7a4d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:59.366752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bb7c0dd4-4615-4408-82e7-6c4f52df7a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:59.388021 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3356/6651] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:13:59.398063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c938363-044e-4ea8-9f29-0af6eb503613 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:59.400643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8c938363-044e-4ea8-9f29-0af6eb503613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:59.458811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:59.468307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c0dd4-4615-4408-82e7-6c4f52df7a4d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:59.474997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c938363-044e-4ea8-9f29-0af6eb503613 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:59.477908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc8a18-de99-4076-8ac5-6228d209bb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:59.478604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc8a18-de99-4076-8ac5-6228d209bb8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:59.479093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc8a18-de99-4076-8ac5-6228d209bb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:59.479321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc8a18-de99-4076-8ac5-6228d209bb8b admin admin] PATH_INFO: `/v3/users/5eab4b929e0f4c0db0a5e48245d3df4b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:59.479638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d35c08-bae7-478b-9f90-0301a7eeeb69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:59.480244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d35c08-bae7-478b-9f90-0301a7eeeb69 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:59.480500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbc8a18-de99-4076-8ac5-6228d209bb8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eab4b929e0f4c0db0a5e48245d3df4b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:59.480609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d35c08-bae7-478b-9f90-0301a7eeeb69 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:59.480778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d35c08-bae7-478b-9f90-0301a7eeeb69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:59.482515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbc8a18-de99-4076-8ac5-6228d209bb8b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:59.490396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d35c08-bae7-478b-9f90-0301a7eeeb69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:59.491862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d35c08-bae7-478b-9f90-0301a7eeeb69 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:59.502848 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3357/6652] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:13:59.536343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2bbc8a18-de99-4076-8ac5-6228d209bb8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eab4b929e0f4c0db0a5e48245d3df4b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:59.537003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2bbc8a18-de99-4076-8ac5-6228d209bb8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eab4b929e0f4c0db0a5e48245d3df4b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:59.537626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2bbc8a18-de99-4076-8ac5-6228d209bb8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eab4b929e0f4c0db0a5e48245d3df4b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:59.544652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2bbc8a18-de99-4076-8ac5-6228d209bb8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eab4b929e0f4c0db0a5e48245d3df4b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:59.558267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2bbc8a18-de99-4076-8ac5-6228d209bb8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eab4b929e0f4c0db0a5e48245d3df4b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:59.575719 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3296/6653] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:59 2022] DELETE /identity/v3/users/5eab4b929e0f4c0db0a5e48245d3df4b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:59.577399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a2ec-7740-482b-9c34-d0e67a897f15 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:59.577693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a2ec-7740-482b-9c34-d0e67a897f15 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:59.577987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a2ec-7740-482b-9c34-d0e67a897f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:59.602608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-84e8a2ec-7740-482b-9c34-d0e67a897f15 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:13:59.604126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-84e8a2ec-7740-482b-9c34-d0e67a897f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:59.690755 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3358/6654] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:13:59.701687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc8a18-de99-4076-8ac5-6228d209bb8b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:59.703437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2bbc8a18-de99-4076-8ac5-6228d209bb8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:59.779873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc8a18-de99-4076-8ac5-6228d209bb8b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:59.783612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219bfe8-5e70-4992-90e9-ab05446b4f75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:59.784266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219bfe8-5e70-4992-90e9-ab05446b4f75 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:59.784524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219bfe8-5e70-4992-90e9-ab05446b4f75 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:59.784810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219bfe8-5e70-4992-90e9-ab05446b4f75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:59.794286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2219bfe8-5e70-4992-90e9-ab05446b4f75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:59.795978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2219bfe8-5e70-4992-90e9-ab05446b4f75 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:59.806813 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3297/6655] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:13:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:13:59.840353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a2ec-7740-482b-9c34-d0e67a897f15 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:59.842373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-84e8a2ec-7740-482b-9c34-d0e67a897f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:13:59.857972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a2ec-7740-482b-9c34-d0e67a897f15 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:13:59.863076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deee273-403e-4e90-8b53-e37a10bb0b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:13:59.864111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deee273-403e-4e90-8b53-e37a10bb0b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:13:59.864426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deee273-403e-4e90-8b53-e37a10bb0b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:13:59.864726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deee273-403e-4e90-8b53-e37a10bb0b70 admin admin] PATH_INFO: `/v3/projects/5946f2c858e54d42ab4f184fd2f4d87c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:13:59.866865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deee273-403e-4e90-8b53-e37a10bb0b70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5946f2c858e54d42ab4f184fd2f4d87c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:13:59.869412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deee273-403e-4e90-8b53-e37a10bb0b70 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:13:59.923785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3deee273-403e-4e90-8b53-e37a10bb0b70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5946f2c858e54d42ab4f184fd2f4d87c', 'request_id': 'req-3deee273-403e-4e90-8b53-e37a10bb0b70'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:59.931737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3deee273-403e-4e90-8b53-e37a10bb0b70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5946f2c858e54d42ab4f184fd2f4d87c', 'request_id': 'req-3deee273-403e-4e90-8b53-e37a10bb0b70'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:59.938939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3deee273-403e-4e90-8b53-e37a10bb0b70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5946f2c858e54d42ab4f184fd2f4d87c', 'request_id': 'req-3deee273-403e-4e90-8b53-e37a10bb0b70'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:59.943601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:59.954970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3deee273-403e-4e90-8b53-e37a10bb0b70 admin admin] The token cache is being invalidate because project 5946f2c858e54d42ab4f184fd2f4d87c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:13:59.955345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3deee273-403e-4e90-8b53-e37a10bb0b70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:59.956230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3deee273-403e-4e90-8b53-e37a10bb0b70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:13:59.958162 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3359/6656] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:13:59 2022] DELETE /identity/v3/projects/5946f2c858e54d42ab4f184fd2f4d87c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:13:59.962327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219bfe8-5e70-4992-90e9-ab05446b4f75 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:13:59.978859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219bfe8-5e70-4992-90e9-ab05446b4f75 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:00.060614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a524e2c-50ac-4b02-b8b7-6818e8e6b3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:00.060854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a524e2c-50ac-4b02-b8b7-6818e8e6b3ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:00.061137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a524e2c-50ac-4b02-b8b7-6818e8e6b3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:00.061404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a524e2c-50ac-4b02-b8b7-6818e8e6b3ff admin admin] PATH_INFO: `/v3/users/42c2d21ecbfb4378a6ea22b3d7d294d9` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:00.062571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a524e2c-50ac-4b02-b8b7-6818e8e6b3ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=42c2d21ecbfb4378a6ea22b3d7d294d9)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:00.064559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a524e2c-50ac-4b02-b8b7-6818e8e6b3ff admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:00.093156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:00.127288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deee273-403e-4e90-8b53-e37a10bb0b70 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:00.128952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3deee273-403e-4e90-8b53-e37a10bb0b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:00.139668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9a524e2c-50ac-4b02-b8b7-6818e8e6b3ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42c2d21ecbfb4378a6ea22b3d7d294d9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.140102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9a524e2c-50ac-4b02-b8b7-6818e8e6b3ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42c2d21ecbfb4378a6ea22b3d7d294d9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.140646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9a524e2c-50ac-4b02-b8b7-6818e8e6b3ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42c2d21ecbfb4378a6ea22b3d7d294d9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.145922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deee273-403e-4e90-8b53-e37a10bb0b70 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:00.151537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9a524e2c-50ac-4b02-b8b7-6818e8e6b3ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42c2d21ecbfb4378a6ea22b3d7d294d9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.152060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31adec86-322e-4536-a1c2-d7f5b22bae23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:00.152854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adec86-322e-4536-a1c2-d7f5b22bae23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:00.153404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adec86-322e-4536-a1c2-d7f5b22bae23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:00.153689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adec86-322e-4536-a1c2-d7f5b22bae23 admin admin] PATH_INFO: `/v3/projects/06d9722f2159421eac66bffde6d5a901` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:00.155272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31adec86-322e-4536-a1c2-d7f5b22bae23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06d9722f2159421eac66bffde6d5a901)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:00.157684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31adec86-322e-4536-a1c2-d7f5b22bae23 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:00.166453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9a524e2c-50ac-4b02-b8b7-6818e8e6b3ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42c2d21ecbfb4378a6ea22b3d7d294d9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.189084 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3298/6657] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:13:59 2022] DELETE /identity/v3/users/42c2d21ecbfb4378a6ea22b3d7d294d9 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:00.226366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-31adec86-322e-4536-a1c2-d7f5b22bae23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06d9722f2159421eac66bffde6d5a901', 'request_id': 'req-31adec86-322e-4536-a1c2-d7f5b22bae23'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.235938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-31adec86-322e-4536-a1c2-d7f5b22bae23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06d9722f2159421eac66bffde6d5a901', 'request_id': 'req-31adec86-322e-4536-a1c2-d7f5b22bae23'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.245846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-31adec86-322e-4536-a1c2-d7f5b22bae23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06d9722f2159421eac66bffde6d5a901', 'request_id': 'req-31adec86-322e-4536-a1c2-d7f5b22bae23'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.262814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-31adec86-322e-4536-a1c2-d7f5b22bae23 admin admin] The token cache is being invalidate because project 06d9722f2159421eac66bffde6d5a901 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:00.263287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-31adec86-322e-4536-a1c2-d7f5b22bae23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.264032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-31adec86-322e-4536-a1c2-d7f5b22bae23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.265969 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3360/6658] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:00 2022] DELETE /identity/v3/projects/06d9722f2159421eac66bffde6d5a901 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:00.271044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a524e2c-50ac-4b02-b8b7-6818e8e6b3ff admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:00.272337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9a524e2c-50ac-4b02-b8b7-6818e8e6b3ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:00.346742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a524e2c-50ac-4b02-b8b7-6818e8e6b3ff admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:00.421088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d77068-d0b3-4440-98c0-403fb1e7c3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:00.421732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d77068-d0b3-4440-98c0-403fb1e7c3e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:00.422013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d77068-d0b3-4440-98c0-403fb1e7c3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:00.422288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d77068-d0b3-4440-98c0-403fb1e7c3e5 admin admin] PATH_INFO: `/v3/users/8a6d1d77d0e84fb884bd2cefdec645a1` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:00.423464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d77068-d0b3-4440-98c0-403fb1e7c3e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a6d1d77d0e84fb884bd2cefdec645a1)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:00.425650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d77068-d0b3-4440-98c0-403fb1e7c3e5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:00.468746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31adec86-322e-4536-a1c2-d7f5b22bae23 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:00.470261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-31adec86-322e-4536-a1c2-d7f5b22bae23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:00.479634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95d77068-d0b3-4440-98c0-403fb1e7c3e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a6d1d77d0e84fb884bd2cefdec645a1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.480591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95d77068-d0b3-4440-98c0-403fb1e7c3e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a6d1d77d0e84fb884bd2cefdec645a1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.481416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95d77068-d0b3-4440-98c0-403fb1e7c3e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a6d1d77d0e84fb884bd2cefdec645a1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.484419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:00.488504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31adec86-322e-4536-a1c2-d7f5b22bae23 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:00.495945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95d77068-d0b3-4440-98c0-403fb1e7c3e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a6d1d77d0e84fb884bd2cefdec645a1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.496042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42688894-355f-4106-9b4d-fc62ed72e38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:00.497645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42688894-355f-4106-9b4d-fc62ed72e38b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:00.497921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42688894-355f-4106-9b4d-fc62ed72e38b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:00.498302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42688894-355f-4106-9b4d-fc62ed72e38b admin admin] PATH_INFO: `/v3/projects/cd1c8aa3b1794a56b9120a7655d138b7` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:00.499505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42688894-355f-4106-9b4d-fc62ed72e38b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd1c8aa3b1794a56b9120a7655d138b7)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:00.501428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42688894-355f-4106-9b4d-fc62ed72e38b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:00.512519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95d77068-d0b3-4440-98c0-403fb1e7c3e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a6d1d77d0e84fb884bd2cefdec645a1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.526307 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3299/6659] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:00 2022] DELETE /identity/v3/users/8a6d1d77d0e84fb884bd2cefdec645a1 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:00.555489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-42688894-355f-4106-9b4d-fc62ed72e38b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd1c8aa3b1794a56b9120a7655d138b7', 'request_id': 'req-42688894-355f-4106-9b4d-fc62ed72e38b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.562170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-42688894-355f-4106-9b4d-fc62ed72e38b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd1c8aa3b1794a56b9120a7655d138b7', 'request_id': 'req-42688894-355f-4106-9b4d-fc62ed72e38b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.568332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-42688894-355f-4106-9b4d-fc62ed72e38b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd1c8aa3b1794a56b9120a7655d138b7', 'request_id': 'req-42688894-355f-4106-9b4d-fc62ed72e38b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.581318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-42688894-355f-4106-9b4d-fc62ed72e38b admin admin] The token cache is being invalidate because project cd1c8aa3b1794a56b9120a7655d138b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:00.581608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-42688894-355f-4106-9b4d-fc62ed72e38b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.582249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-42688894-355f-4106-9b4d-fc62ed72e38b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.584006 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3361/6660] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:00 2022] DELETE /identity/v3/projects/cd1c8aa3b1794a56b9120a7655d138b7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:00.588337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d77068-d0b3-4440-98c0-403fb1e7c3e5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:00.589755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-95d77068-d0b3-4440-98c0-403fb1e7c3e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:00.663505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d77068-d0b3-4440-98c0-403fb1e7c3e5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:00.739106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc0c32-db89-458d-b17c-3c7dc02052fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:00.740195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc0c32-db89-458d-b17c-3c7dc02052fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:00.740489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc0c32-db89-458d-b17c-3c7dc02052fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:00.740769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc0c32-db89-458d-b17c-3c7dc02052fd admin admin] PATH_INFO: `/v3/users/eb153aa122af4ab9a652798bee5516ee` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:00.742600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc0c32-db89-458d-b17c-3c7dc02052fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb153aa122af4ab9a652798bee5516ee)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:00.745363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc0c32-db89-458d-b17c-3c7dc02052fd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:00.803990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-88fc0c32-db89-458d-b17c-3c7dc02052fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb153aa122af4ab9a652798bee5516ee'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.807779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-88fc0c32-db89-458d-b17c-3c7dc02052fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb153aa122af4ab9a652798bee5516ee'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.808483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-88fc0c32-db89-458d-b17c-3c7dc02052fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb153aa122af4ab9a652798bee5516ee'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.812757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42688894-355f-4106-9b4d-fc62ed72e38b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:00.814711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-42688894-355f-4106-9b4d-fc62ed72e38b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:00.819901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-88fc0c32-db89-458d-b17c-3c7dc02052fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb153aa122af4ab9a652798bee5516ee'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.829522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42688894-355f-4106-9b4d-fc62ed72e38b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:00.835335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db9e1a-f24c-4e10-a7ff-598e94cb50dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:00.836028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db9e1a-f24c-4e10-a7ff-598e94cb50dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:00.836307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db9e1a-f24c-4e10-a7ff-598e94cb50dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:00.836633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db9e1a-f24c-4e10-a7ff-598e94cb50dc admin admin] PATH_INFO: `/v3/projects/1fa38b276bd845648ee01a473ae7f4c5` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:00.838360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db9e1a-f24c-4e10-a7ff-598e94cb50dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fa38b276bd845648ee01a473ae7f4c5)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:00.840248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db9e1a-f24c-4e10-a7ff-598e94cb50dc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:00.841733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-88fc0c32-db89-458d-b17c-3c7dc02052fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb153aa122af4ab9a652798bee5516ee'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.844022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:00.859062 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3300/6661] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:00 2022] DELETE /identity/v3/users/eb153aa122af4ab9a652798bee5516ee => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:00.893721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f2db9e1a-f24c-4e10-a7ff-598e94cb50dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fa38b276bd845648ee01a473ae7f4c5', 'request_id': 'req-f2db9e1a-f24c-4e10-a7ff-598e94cb50dc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.900166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f2db9e1a-f24c-4e10-a7ff-598e94cb50dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fa38b276bd845648ee01a473ae7f4c5', 'request_id': 'req-f2db9e1a-f24c-4e10-a7ff-598e94cb50dc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.902128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:00.906975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f2db9e1a-f24c-4e10-a7ff-598e94cb50dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fa38b276bd845648ee01a473ae7f4c5', 'request_id': 'req-f2db9e1a-f24c-4e10-a7ff-598e94cb50dc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.924130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f2db9e1a-f24c-4e10-a7ff-598e94cb50dc admin admin] The token cache is being invalidate because project 1fa38b276bd845648ee01a473ae7f4c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:00.924429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f2db9e1a-f24c-4e10-a7ff-598e94cb50dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.925087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f2db9e1a-f24c-4e10-a7ff-598e94cb50dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:00.926842 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3362/6662] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:00 2022] DELETE /identity/v3/projects/1fa38b276bd845648ee01a473ae7f4c5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:00.931178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc0c32-db89-458d-b17c-3c7dc02052fd admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:00.932606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-88fc0c32-db89-458d-b17c-3c7dc02052fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:01.003982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc0c32-db89-458d-b17c-3c7dc02052fd admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:01.080426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c0b42-ae7a-4bb0-b15f-c4726ff594bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:01.081139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c0b42-ae7a-4bb0-b15f-c4726ff594bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:01.081420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c0b42-ae7a-4bb0-b15f-c4726ff594bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:01.081699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c0b42-ae7a-4bb0-b15f-c4726ff594bf admin admin] PATH_INFO: `/v3/users/0d76c2f60dde4f51aa8762150ffe71fa` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:01.082864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c0b42-ae7a-4bb0-b15f-c4726ff594bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d76c2f60dde4f51aa8762150ffe71fa)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:01.084816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c0b42-ae7a-4bb0-b15f-c4726ff594bf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:01.151775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2a2c0b42-ae7a-4bb0-b15f-c4726ff594bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d76c2f60dde4f51aa8762150ffe71fa'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:01.152587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2a2c0b42-ae7a-4bb0-b15f-c4726ff594bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d76c2f60dde4f51aa8762150ffe71fa'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:01.154038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2a2c0b42-ae7a-4bb0-b15f-c4726ff594bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d76c2f60dde4f51aa8762150ffe71fa'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:01.160575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db9e1a-f24c-4e10-a7ff-598e94cb50dc admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:01.162870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f2db9e1a-f24c-4e10-a7ff-598e94cb50dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:01.185875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2a2c0b42-ae7a-4bb0-b15f-c4726ff594bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d76c2f60dde4f51aa8762150ffe71fa'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:01.186580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db9e1a-f24c-4e10-a7ff-598e94cb50dc admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:01.193560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9a703-d6c9-4d57-884c-9fdc70580dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:01.193560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9a703-d6c9-4d57-884c-9fdc70580dcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:01.193560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9a703-d6c9-4d57-884c-9fdc70580dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:01.193560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9a703-d6c9-4d57-884c-9fdc70580dcf admin admin] PATH_INFO: `/v3/projects/e08ef2b377884a51a7ef815ddef1be8c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:01.195272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff9a703-d6c9-4d57-884c-9fdc70580dcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e08ef2b377884a51a7ef815ddef1be8c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:01.197847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff9a703-d6c9-4d57-884c-9fdc70580dcf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:01.219656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2a2c0b42-ae7a-4bb0-b15f-c4726ff594bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d76c2f60dde4f51aa8762150ffe71fa'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:01.254288 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3301/6663] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:00 2022] DELETE /identity/v3/users/0d76c2f60dde4f51aa8762150ffe71fa => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:01.270885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eff9a703-d6c9-4d57-884c-9fdc70580dcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e08ef2b377884a51a7ef815ddef1be8c', 'request_id': 'req-eff9a703-d6c9-4d57-884c-9fdc70580dcf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:01.282433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eff9a703-d6c9-4d57-884c-9fdc70580dcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e08ef2b377884a51a7ef815ddef1be8c', 'request_id': 'req-eff9a703-d6c9-4d57-884c-9fdc70580dcf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:01.293954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eff9a703-d6c9-4d57-884c-9fdc70580dcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e08ef2b377884a51a7ef815ddef1be8c', 'request_id': 'req-eff9a703-d6c9-4d57-884c-9fdc70580dcf'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:01.318184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eff9a703-d6c9-4d57-884c-9fdc70580dcf admin admin] The token cache is being invalidate because project e08ef2b377884a51a7ef815ddef1be8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:01.318905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eff9a703-d6c9-4d57-884c-9fdc70580dcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:01.319781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eff9a703-d6c9-4d57-884c-9fdc70580dcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:01.324196 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3363/6664] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:01 2022] DELETE /identity/v3/projects/e08ef2b377884a51a7ef815ddef1be8c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:01.334282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ec296-0dff-4500-b11c-bf02172dab67 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:01.334725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ec296-0dff-4500-b11c-bf02172dab67 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:01.335102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ec296-0dff-4500-b11c-bf02172dab67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:01.389497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-484ec296-0dff-4500-b11c-bf02172dab67 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:01.391113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-484ec296-0dff-4500-b11c-bf02172dab67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:01.412362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:01.518048 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3302/6665] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:01.530306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4c348-b362-4d9b-9ea8-fd14eb438946 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:01.530306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4c348-b362-4d9b-9ea8-fd14eb438946 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:01.530306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4c348-b362-4d9b-9ea8-fd14eb438946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:01.561053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-56d4c348-b362-4d9b-9ea8-fd14eb438946 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:01.563172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-56d4c348-b362-4d9b-9ea8-fd14eb438946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:01.576321 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3364/6666] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:01.577701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ec296-0dff-4500-b11c-bf02172dab67 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:01.590344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ec296-0dff-4500-b11c-bf02172dab67 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:01.596746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce363d-86b5-4f47-8ce7-9b49ecbab4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:01.597746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce363d-86b5-4f47-8ce7-9b49ecbab4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:01.598636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce363d-86b5-4f47-8ce7-9b49ecbab4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:01.598636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce363d-86b5-4f47-8ce7-9b49ecbab4a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:01.598992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce363d-86b5-4f47-8ce7-9b49ecbab4a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:14:01.599300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce363d-86b5-4f47-8ce7-9b49ecbab4a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:01.600918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce363d-86b5-4f47-8ce7-9b49ecbab4a8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:01.615927 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3303/6667] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:14:01.620167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4c348-b362-4d9b-9ea8-fd14eb438946 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:01.636184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4c348-b362-4d9b-9ea8-fd14eb438946 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:01.645065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10b966-6971-472e-9ecb-5839dbd85b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:01.645957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10b966-6971-472e-9ecb-5839dbd85b98 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:01.646343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10b966-6971-472e-9ecb-5839dbd85b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:01.646699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10b966-6971-472e-9ecb-5839dbd85b98 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:01.647490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10b966-6971-472e-9ecb-5839dbd85b98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:01.651715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10b966-6971-472e-9ecb-5839dbd85b98 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:01.685226 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3365/6668] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:01 2022] POST /identity/v3/projects => generated 387 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:01.689636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce363d-86b5-4f47-8ce7-9b49ecbab4a8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:01.701596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce363d-86b5-4f47-8ce7-9b49ecbab4a8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:01.702934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10b966-6971-472e-9ecb-5839dbd85b98 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:01.704602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ea10b966-6971-472e-9ecb-5839dbd85b98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:01.764130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d139705-5578-4f7b-96be-e6b93edd7bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:01.764673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d139705-5578-4f7b-96be-e6b93edd7bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:01.764956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d139705-5578-4f7b-96be-e6b93edd7bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:01.765229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d139705-5578-4f7b-96be-e6b93edd7bb8 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:01.765852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d139705-5578-4f7b-96be-e6b93edd7bb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:01.767816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d139705-5578-4f7b-96be-e6b93edd7bb8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:01.784294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10b966-6971-472e-9ecb-5839dbd85b98 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:01.789312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c392b91-b34e-4343-8824-719037233423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:01.789986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c392b91-b34e-4343-8824-719037233423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:01.793265 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3304/6669] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:01 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:14:01.793380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c392b91-b34e-4343-8824-719037233423 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:01.793380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c392b91-b34e-4343-8824-719037233423 admin admin] PATH_INFO: `/v3/projects/5b09cbe898f04fa199f05d7122f76e11` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:01.793380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c392b91-b34e-4343-8824-719037233423 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b09cbe898f04fa199f05d7122f76e11)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:01.793687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c392b91-b34e-4343-8824-719037233423 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:01.794268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d139705-5578-4f7b-96be-e6b93edd7bb8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:01.804202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d139705-5578-4f7b-96be-e6b93edd7bb8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:01.810010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61384191-0101-4422-be96-e5744c0e3c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:01.810701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61384191-0101-4422-be96-e5744c0e3c5d admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:01.811302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61384191-0101-4422-be96-e5744c0e3c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:01.813768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61384191-0101-4422-be96-e5744c0e3c5d admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:01.814426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61384191-0101-4422-be96-e5744c0e3c5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:01.816044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61384191-0101-4422-be96-e5744c0e3c5d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:01.829484 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3305/6670] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:14:01.834169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61384191-0101-4422-be96-e5744c0e3c5d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:01.850617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61384191-0101-4422-be96-e5744c0e3c5d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:01.851024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:01.859013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4c392b91-b34e-4343-8824-719037233423 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b09cbe898f04fa199f05d7122f76e11', 'request_id': 'req-4c392b91-b34e-4343-8824-719037233423'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:01.868541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4c392b91-b34e-4343-8824-719037233423 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b09cbe898f04fa199f05d7122f76e11', 'request_id': 'req-4c392b91-b34e-4343-8824-719037233423'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:01.875954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4c392b91-b34e-4343-8824-719037233423 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b09cbe898f04fa199f05d7122f76e11', 'request_id': 'req-4c392b91-b34e-4343-8824-719037233423'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:01.890767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4c392b91-b34e-4343-8824-719037233423 admin admin] The token cache is being invalidate because project 5b09cbe898f04fa199f05d7122f76e11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:01.891065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4c392b91-b34e-4343-8824-719037233423 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:01.891726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4c392b91-b34e-4343-8824-719037233423 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:01.893297 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3366/6671] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:01 2022] DELETE /identity/v3/projects/5b09cbe898f04fa199f05d7122f76e11 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:01.898134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67628e5-32f1-4903-becf-a5c49623ff9e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:01.898726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67628e5-32f1-4903-becf-a5c49623ff9e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:01.898820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67628e5-32f1-4903-becf-a5c49623ff9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:01.920835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e67628e5-32f1-4903-becf-a5c49623ff9e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:01.922361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e67628e5-32f1-4903-becf-a5c49623ff9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:01.930379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dba75-9b8e-4efa-b4d6-6828d22ae9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:01.931078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dba75-9b8e-4efa-b4d6-6828d22ae9bf admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:01.931358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dba75-9b8e-4efa-b4d6-6828d22ae9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:01.931634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dba75-9b8e-4efa-b4d6-6828d22ae9bf admin admin] PATH_INFO: `/v3/projects/0ec3561fe5f14c388b97b07b49adc725/users/24792a32b2144d3ca0d44e621595e35f/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:01.949726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16dba75-9b8e-4efa-b4d6-6828d22ae9bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ec3561fe5f14c388b97b07b49adc725, user_id=24792a32b2144d3ca0d44e621595e35f, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:01.952507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16dba75-9b8e-4efa-b4d6-6828d22ae9bf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:01.968848 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3306/6672] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:01 2022] PUT /identity/v3/projects/0ec3561fe5f14c388b97b07b49adc725/users/24792a32b2144d3ca0d44e621595e35f/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:01.972803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dba75-9b8e-4efa-b4d6-6828d22ae9bf admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:01.974703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a16dba75-9b8e-4efa-b4d6-6828d22ae9bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:01.986362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:02.000454 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3367/6673] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:02.007932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf9c06-ecfa-4bf4-9088-affd11f37444 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:02.008226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf9c06-ecfa-4bf4-9088-affd11f37444 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:02.009147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf9c06-ecfa-4bf4-9088-affd11f37444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:02.030821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3bbf9c06-ecfa-4bf4-9088-affd11f37444 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:02.032098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3bbf9c06-ecfa-4bf4-9088-affd11f37444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:02.060379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dba75-9b8e-4efa-b4d6-6828d22ae9bf admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:02.065163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901ac17-4c13-4058-8bb0-c145aa9d0155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:02.065764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901ac17-4c13-4058-8bb0-c145aa9d0155 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:02.066051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901ac17-4c13-4058-8bb0-c145aa9d0155 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:02.066324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901ac17-4c13-4058-8bb0-c145aa9d0155 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:02.067067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6901ac17-4c13-4058-8bb0-c145aa9d0155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:02.068652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6901ac17-4c13-4058-8bb0-c145aa9d0155 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:02.078911 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3307/6674] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:14:02.083407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901ac17-4c13-4058-8bb0-c145aa9d0155 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:02.093565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901ac17-4c13-4058-8bb0-c145aa9d0155 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:02.096151 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3368/6675] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:02.098441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b65a6-10af-412d-a713-a2ff0b1afa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:02.099202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b65a6-10af-412d-a713-a2ff0b1afa85 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:02.099561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b65a6-10af-412d-a713-a2ff0b1afa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:02.099847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b65a6-10af-412d-a713-a2ff0b1afa85 admin admin] PATH_INFO: `/v3/projects/0ec3561fe5f14c388b97b07b49adc725/users/24792a32b2144d3ca0d44e621595e35f/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:02.100367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf9c06-ecfa-4bf4-9088-affd11f37444 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:02.101850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b65a6-10af-412d-a713-a2ff0b1afa85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ec3561fe5f14c388b97b07b49adc725, user_id=24792a32b2144d3ca0d44e621595e35f, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:02.104795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b65a6-10af-412d-a713-a2ff0b1afa85 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:02.109620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf9c06-ecfa-4bf4-9088-affd11f37444 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:02.115134 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3308/6676] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:02 2022] PUT /identity/v3/projects/0ec3561fe5f14c388b97b07b49adc725/users/24792a32b2144d3ca0d44e621595e35f/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:02.125736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ab515-dab9-4f5a-b148-cda6d19b28d4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:02.126031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ab515-dab9-4f5a-b148-cda6d19b28d4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:02.126311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ab515-dab9-4f5a-b148-cda6d19b28d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:02.156725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9b3ab515-dab9-4f5a-b148-cda6d19b28d4 None None] MFA Rules not processed for user `24792a32b2144d3ca0d44e621595e35f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:02.157933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9b3ab515-dab9-4f5a-b148-cda6d19b28d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:02.167050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35538148-a185-45c3-82da-8338ef0342d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:02.167655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35538148-a185-45c3-82da-8338ef0342d6 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:02.167936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35538148-a185-45c3-82da-8338ef0342d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:02.168211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35538148-a185-45c3-82da-8338ef0342d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:02.168862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35538148-a185-45c3-82da-8338ef0342d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:14:02.169167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35538148-a185-45c3-82da-8338ef0342d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:02.170777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35538148-a185-45c3-82da-8338ef0342d6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:02.186075 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3369/6677] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:14:02.187141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35538148-a185-45c3-82da-8338ef0342d6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:02.193108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:02.197613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35538148-a185-45c3-82da-8338ef0342d6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:02.204473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e9276-f8c9-4c0c-bc80-5d79007b98d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:02.205110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e9276-f8c9-4c0c-bc80-5d79007b98d8 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:02.205644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e9276-f8c9-4c0c-bc80-5d79007b98d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:02.205985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e9276-f8c9-4c0c-bc80-5d79007b98d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:02.206719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e9276-f8c9-4c0c-bc80-5d79007b98d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:02.209109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e9276-f8c9-4c0c-bc80-5d79007b98d8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:02.241946 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3309/6678] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:02.246683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ab515-dab9-4f5a-b148-cda6d19b28d4 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:02.247139 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3370/6679] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:02 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:14:02.251175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e9276-f8c9-4c0c-bc80-5d79007b98d8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:02.262847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ab515-dab9-4f5a-b148-cda6d19b28d4 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:02.329899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954abf69-cf96-4237-ae38-dfca48756b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:02.330537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954abf69-cf96-4237-ae38-dfca48756b60 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:02.330845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954abf69-cf96-4237-ae38-dfca48756b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:02.331120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954abf69-cf96-4237-ae38-dfca48756b60 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:02.331740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954abf69-cf96-4237-ae38-dfca48756b60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:02.333646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954abf69-cf96-4237-ae38-dfca48756b60 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:02.336139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e9276-f8c9-4c0c-bc80-5d79007b98d8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:02.340928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29864028-a28f-4c2c-b36c-a81adf3e2e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:02.341561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29864028-a28f-4c2c-b36c-a81adf3e2e84 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:02.341845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29864028-a28f-4c2c-b36c-a81adf3e2e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:02.342118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29864028-a28f-4c2c-b36c-a81adf3e2e84 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:02.342799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29864028-a28f-4c2c-b36c-a81adf3e2e84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:02.344729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29864028-a28f-4c2c-b36c-a81adf3e2e84 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:02.354733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:02.363686 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3310/6680] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:02 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:14:02.368388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954abf69-cf96-4237-ae38-dfca48756b60 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:02.370679 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3371/6681] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:02 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:02.374312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29864028-a28f-4c2c-b36c-a81adf3e2e84 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:02.384637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29864028-a28f-4c2c-b36c-a81adf3e2e84 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:02.435920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954abf69-cf96-4237-ae38-dfca48756b60 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:02.440781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb638410-7ca8-4e3a-8529-7e4c122f2bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:02.441386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb638410-7ca8-4e3a-8529-7e4c122f2bc3 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:02.441665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb638410-7ca8-4e3a-8529-7e4c122f2bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:02.441937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb638410-7ca8-4e3a-8529-7e4c122f2bc3 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:02.442585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb638410-7ca8-4e3a-8529-7e4c122f2bc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:02.444492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb638410-7ca8-4e3a-8529-7e4c122f2bc3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:02.450372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef25c7-7c5c-4095-a9e2-090194538fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:02.451020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef25c7-7c5c-4095-a9e2-090194538fef admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:02.451298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef25c7-7c5c-4095-a9e2-090194538fef admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:02.451561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef25c7-7c5c-4095-a9e2-090194538fef admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:02.454266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef25c7-7c5c-4095-a9e2-090194538fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:02.454266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef25c7-7c5c-4095-a9e2-090194538fef admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:02.465074 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3372/6682] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:14:02.468747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef25c7-7c5c-4095-a9e2-090194538fef admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:02.472259 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3311/6683] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:02 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:14:02.473187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb638410-7ca8-4e3a-8529-7e4c122f2bc3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:02.479592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef25c7-7c5c-4095-a9e2-090194538fef admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:02.485100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb638410-7ca8-4e3a-8529-7e4c122f2bc3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:02.487515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742b6e3-2667-499a-a8e4-84b44ee9109d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:02.488350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742b6e3-2667-499a-a8e4-84b44ee9109d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:02.488657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742b6e3-2667-499a-a8e4-84b44ee9109d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:02.488899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742b6e3-2667-499a-a8e4-84b44ee9109d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:02.489599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742b6e3-2667-499a-a8e4-84b44ee9109d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:02.491326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742b6e3-2667-499a-a8e4-84b44ee9109d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:02.491520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382eb15-9c52-4d7d-a297-ea58faac0946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:02.492236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382eb15-9c52-4d7d-a297-ea58faac0946 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:02.492587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382eb15-9c52-4d7d-a297-ea58faac0946 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:02.493538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382eb15-9c52-4d7d-a297-ea58faac0946 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:02.494265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0382eb15-9c52-4d7d-a297-ea58faac0946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:02.496110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0382eb15-9c52-4d7d-a297-ea58faac0946 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:02.504245 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3373/6684] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:14:02.508337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742b6e3-2667-499a-a8e4-84b44ee9109d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:02.510791 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3312/6685] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:02.511720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382eb15-9c52-4d7d-a297-ea58faac0946 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:02.518601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742b6e3-2667-499a-a8e4-84b44ee9109d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:02.522190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382eb15-9c52-4d7d-a297-ea58faac0946 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:02.523817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9df83d-4623-44dc-bd03-a548e637e616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:02.524388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9df83d-4623-44dc-bd03-a548e637e616 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:02.524746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9df83d-4623-44dc-bd03-a548e637e616 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:02.525097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9df83d-4623-44dc-bd03-a548e637e616 admin admin] PATH_INFO: `/v3/projects/817ad1c30bfc4b9c9030f36c70e07781/users/b049b104d5d6487da4849ef06cda8075/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:02.527525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d5dd2-4843-499f-97bf-16599f5fb1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:02.528137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d5dd2-4843-499f-97bf-16599f5fb1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:02.528421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d5dd2-4843-499f-97bf-16599f5fb1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:02.528700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d5dd2-4843-499f-97bf-16599f5fb1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:02.529728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d5dd2-4843-499f-97bf-16599f5fb1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:02.532049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d5dd2-4843-499f-97bf-16599f5fb1d8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:02.545207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9df83d-4623-44dc-bd03-a548e637e616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=817ad1c30bfc4b9c9030f36c70e07781, user_id=b049b104d5d6487da4849ef06cda8075, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:02.548058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9df83d-4623-44dc-bd03-a548e637e616 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:02.553622 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3313/6686] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:02.557993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d5dd2-4843-499f-97bf-16599f5fb1d8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:02.578764 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3374/6687] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:02 2022] PUT /identity/v3/projects/817ad1c30bfc4b9c9030f36c70e07781/users/b049b104d5d6487da4849ef06cda8075/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:02.587576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ae60b-4ae2-466d-abfb-dcaa4afae6ff None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:02.587964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ae60b-4ae2-466d-abfb-dcaa4afae6ff None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:02.588650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ae60b-4ae2-466d-abfb-dcaa4afae6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:02.597175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d5dd2-4843-499f-97bf-16599f5fb1d8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:02.622170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3e1ae60b-4ae2-466d-abfb-dcaa4afae6ff None None] MFA Rules not processed for user `b049b104d5d6487da4849ef06cda8075`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:02.623472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3e1ae60b-4ae2-466d-abfb-dcaa4afae6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:02.662760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:02.683132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b16895-e0cf-487d-9824-a2d09cfef0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:02.685098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b16895-e0cf-487d-9824-a2d09cfef0af admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:02.685098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b16895-e0cf-487d-9824-a2d09cfef0af admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:02.685098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b16895-e0cf-487d-9824-a2d09cfef0af admin admin] PATH_INFO: `/v3/projects/7a1d70115ea44685a2dd7df9f3f61e8d/users/a0f8c8d665bc46a695fda4aaa199670a/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:02.700690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b16895-e0cf-487d-9824-a2d09cfef0af admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1d70115ea44685a2dd7df9f3f61e8d, user_id=a0f8c8d665bc46a695fda4aaa199670a, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:02.703652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b16895-e0cf-487d-9824-a2d09cfef0af admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:02.722612 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3314/6688] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:02 2022] PUT /identity/v3/projects/7a1d70115ea44685a2dd7df9f3f61e8d/users/a0f8c8d665bc46a695fda4aaa199670a/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:02.725817 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3375/6689] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:02.731552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8b320-95f2-4a7d-8d51-106b44a0dc07 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:02.732017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8b320-95f2-4a7d-8d51-106b44a0dc07 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:02.732407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8b320-95f2-4a7d-8d51-106b44a0dc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:02.737328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ae60b-4ae2-466d-abfb-dcaa4afae6ff None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:02.739043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3e1ae60b-4ae2-466d-abfb-dcaa4afae6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:02.764015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b0c8b320-95f2-4a7d-8d51-106b44a0dc07 None None] MFA Rules not processed for user `a0f8c8d665bc46a695fda4aaa199670a`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:02.765363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b0c8b320-95f2-4a7d-8d51-106b44a0dc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:02.811906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:02.822139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ae60b-4ae2-466d-abfb-dcaa4afae6ff None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:02.846200 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3315/6690] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:02.859421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8b320-95f2-4a7d-8d51-106b44a0dc07 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:02.860980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b0c8b320-95f2-4a7d-8d51-106b44a0dc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:02.929406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865e069-d9e8-4160-bb60-8e6eaab7da44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:02.930154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865e069-d9e8-4160-bb60-8e6eaab7da44 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:02.930570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865e069-d9e8-4160-bb60-8e6eaab7da44 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:02.931012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865e069-d9e8-4160-bb60-8e6eaab7da44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:02.937996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8b320-95f2-4a7d-8d51-106b44a0dc07 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:02.942798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0ee1c-426c-4cf7-a1f9-9a5daa5d8230 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:02.943640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0ee1c-426c-4cf7-a1f9-9a5daa5d8230 service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:02.943931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0ee1c-426c-4cf7-a1f9-9a5daa5d8230 service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:02.944210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0ee1c-426c-4cf7-a1f9-9a5daa5d8230 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:03.002174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865e069-d9e8-4160-bb60-8e6eaab7da44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:03.003818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865e069-d9e8-4160-bb60-8e6eaab7da44 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:03.019609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0ee1c-426c-4cf7-a1f9-9a5daa5d8230 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:03.021639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0ee1c-426c-4cf7-a1f9-9a5daa5d8230 service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:03.040886 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3376/6691] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:14:03.051470 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3316/6692] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:14:03.457721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865e069-d9e8-4160-bb60-8e6eaab7da44 service cinder] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:03.523775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:03.575506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865e069-d9e8-4160-bb60-8e6eaab7da44 service cinder] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:03.580319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08bf9b-7ad8-4f76-88d1-166abb655523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:03.580956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08bf9b-7ad8-4f76-88d1-166abb655523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:03.581256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08bf9b-7ad8-4f76-88d1-166abb655523 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:03.581527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08bf9b-7ad8-4f76-88d1-166abb655523 admin admin] PATH_INFO: `/v3/users/24792a32b2144d3ca0d44e621595e35f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:03.582929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08bf9b-7ad8-4f76-88d1-166abb655523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24792a32b2144d3ca0d44e621595e35f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:03.584922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08bf9b-7ad8-4f76-88d1-166abb655523 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:03.668810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9b08bf9b-7ad8-4f76-88d1-166abb655523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24792a32b2144d3ca0d44e621595e35f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:03.677822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9b08bf9b-7ad8-4f76-88d1-166abb655523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24792a32b2144d3ca0d44e621595e35f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:03.693776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9b08bf9b-7ad8-4f76-88d1-166abb655523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24792a32b2144d3ca0d44e621595e35f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:03.694724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9b08bf9b-7ad8-4f76-88d1-166abb655523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24792a32b2144d3ca0d44e621595e35f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:03.705823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9b08bf9b-7ad8-4f76-88d1-166abb655523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24792a32b2144d3ca0d44e621595e35f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:03.713357 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3377/6693] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:03 2022] DELETE /identity/v3/users/24792a32b2144d3ca0d44e621595e35f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:03.720196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f65f8-7ad3-4322-850b-103e8b5f98f8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:03.720662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f65f8-7ad3-4322-850b-103e8b5f98f8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:03.720798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f65f8-7ad3-4322-850b-103e8b5f98f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:03.743379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-239f65f8-7ad3-4322-850b-103e8b5f98f8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:03.744727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-239f65f8-7ad3-4322-850b-103e8b5f98f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:03.818848 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3317/6694] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:03.830663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08bf9b-7ad8-4f76-88d1-166abb655523 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:03.832613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9b08bf9b-7ad8-4f76-88d1-166abb655523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:03.915747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08bf9b-7ad8-4f76-88d1-166abb655523 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:03.919470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea73075-1baf-4946-9f60-cc264f01b38a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:03.920160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea73075-1baf-4946-9f60-cc264f01b38a service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:03.920449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea73075-1baf-4946-9f60-cc264f01b38a service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:03.920723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea73075-1baf-4946-9f60-cc264f01b38a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:03.930437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea73075-1baf-4946-9f60-cc264f01b38a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:03.931913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea73075-1baf-4946-9f60-cc264f01b38a service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:03.941762 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3378/6695] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:04.226808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f65f8-7ad3-4322-850b-103e8b5f98f8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:04.228517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-239f65f8-7ad3-4322-850b-103e8b5f98f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:04.248858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f65f8-7ad3-4322-850b-103e8b5f98f8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:04.267030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b622f7a-a77c-4d2d-8e12-ab7c49843e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:04.267839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b622f7a-a77c-4d2d-8e12-ab7c49843e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:04.268182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b622f7a-a77c-4d2d-8e12-ab7c49843e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:04.268482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b622f7a-a77c-4d2d-8e12-ab7c49843e44 admin admin] PATH_INFO: `/v3/projects/0ec3561fe5f14c388b97b07b49adc725` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:04.270252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b622f7a-a77c-4d2d-8e12-ab7c49843e44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ec3561fe5f14c388b97b07b49adc725)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:04.275919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b622f7a-a77c-4d2d-8e12-ab7c49843e44 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:04.318675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:04.346909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6b622f7a-a77c-4d2d-8e12-ab7c49843e44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ec3561fe5f14c388b97b07b49adc725', 'request_id': 'req-6b622f7a-a77c-4d2d-8e12-ab7c49843e44'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:04.357850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6b622f7a-a77c-4d2d-8e12-ab7c49843e44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ec3561fe5f14c388b97b07b49adc725', 'request_id': 'req-6b622f7a-a77c-4d2d-8e12-ab7c49843e44'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:04.368263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6b622f7a-a77c-4d2d-8e12-ab7c49843e44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ec3561fe5f14c388b97b07b49adc725', 'request_id': 'req-6b622f7a-a77c-4d2d-8e12-ab7c49843e44'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:04.384526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6b622f7a-a77c-4d2d-8e12-ab7c49843e44 admin admin] The token cache is being invalidate because project 0ec3561fe5f14c388b97b07b49adc725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:04.385196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6b622f7a-a77c-4d2d-8e12-ab7c49843e44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:04.385504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6b622f7a-a77c-4d2d-8e12-ab7c49843e44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:04.390003 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3318/6696] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:04 2022] DELETE /identity/v3/projects/0ec3561fe5f14c388b97b07b49adc725 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:04.391824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea73075-1baf-4946-9f60-cc264f01b38a service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:04.404109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea73075-1baf-4946-9f60-cc264f01b38a service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:04.464027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:04.472318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e17f0-926d-4a86-a761-9785690b7fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:04.473327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e17f0-926d-4a86-a761-9785690b7fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:04.474040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e17f0-926d-4a86-a761-9785690b7fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:04.474699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e17f0-926d-4a86-a761-9785690b7fe3 admin admin] PATH_INFO: `/v3/users/a0f8c8d665bc46a695fda4aaa199670a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:04.476236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e17f0-926d-4a86-a761-9785690b7fe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0f8c8d665bc46a695fda4aaa199670a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:04.478628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e17f0-926d-4a86-a761-9785690b7fe3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:04.538787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-227e17f0-926d-4a86-a761-9785690b7fe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0f8c8d665bc46a695fda4aaa199670a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:04.549027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-227e17f0-926d-4a86-a761-9785690b7fe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0f8c8d665bc46a695fda4aaa199670a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:04.566777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-227e17f0-926d-4a86-a761-9785690b7fe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0f8c8d665bc46a695fda4aaa199670a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:04.568006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-227e17f0-926d-4a86-a761-9785690b7fe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0f8c8d665bc46a695fda4aaa199670a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:04.579278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-227e17f0-926d-4a86-a761-9785690b7fe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0f8c8d665bc46a695fda4aaa199670a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:04.592274 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3379/6697] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:04 2022] DELETE /identity/v3/users/a0f8c8d665bc46a695fda4aaa199670a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:04.954894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b622f7a-a77c-4d2d-8e12-ab7c49843e44 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:04.956502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6b622f7a-a77c-4d2d-8e12-ab7c49843e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:04.984696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b622f7a-a77c-4d2d-8e12-ab7c49843e44 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:05.130569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394cbfc1-b9b6-43ed-8c7f-9a8ea860f92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:05.132687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cbfc1-b9b6-43ed-8c7f-9a8ea860f92e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:05.132975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cbfc1-b9b6-43ed-8c7f-9a8ea860f92e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:05.133293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cbfc1-b9b6-43ed-8c7f-9a8ea860f92e admin admin] PATH_INFO: `/v3/projects/7a1d70115ea44685a2dd7df9f3f61e8d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:05.135361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394cbfc1-b9b6-43ed-8c7f-9a8ea860f92e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a1d70115ea44685a2dd7df9f3f61e8d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:05.138347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394cbfc1-b9b6-43ed-8c7f-9a8ea860f92e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:05.243184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-394cbfc1-b9b6-43ed-8c7f-9a8ea860f92e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a1d70115ea44685a2dd7df9f3f61e8d', 'request_id': 'req-394cbfc1-b9b6-43ed-8c7f-9a8ea860f92e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:05.254059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-394cbfc1-b9b6-43ed-8c7f-9a8ea860f92e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a1d70115ea44685a2dd7df9f3f61e8d', 'request_id': 'req-394cbfc1-b9b6-43ed-8c7f-9a8ea860f92e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:05.264308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-394cbfc1-b9b6-43ed-8c7f-9a8ea860f92e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a1d70115ea44685a2dd7df9f3f61e8d', 'request_id': 'req-394cbfc1-b9b6-43ed-8c7f-9a8ea860f92e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:05.285311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-394cbfc1-b9b6-43ed-8c7f-9a8ea860f92e admin admin] The token cache is being invalidate because project 7a1d70115ea44685a2dd7df9f3f61e8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:05.285698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-394cbfc1-b9b6-43ed-8c7f-9a8ea860f92e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:05.287220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-394cbfc1-b9b6-43ed-8c7f-9a8ea860f92e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:05.288875 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3319/6698] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:04 2022] DELETE /identity/v3/projects/7a1d70115ea44685a2dd7df9f3f61e8d => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:05.300200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced73ba-048f-47e8-a3f1-f6543bf1f800 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:05.300797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced73ba-048f-47e8-a3f1-f6543bf1f800 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:05.301296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced73ba-048f-47e8-a3f1-f6543bf1f800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:05.341637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-dced73ba-048f-47e8-a3f1-f6543bf1f800 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:05.342977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-dced73ba-048f-47e8-a3f1-f6543bf1f800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:05.422952 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3380/6699] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:05.433855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787d287-8e13-477d-8d57-ce1d3d979492 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:05.434149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787d287-8e13-477d-8d57-ce1d3d979492 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:05.434434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787d287-8e13-477d-8d57-ce1d3d979492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:05.462299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b787d287-8e13-477d-8d57-ce1d3d979492 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:05.464598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b787d287-8e13-477d-8d57-ce1d3d979492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:05.473904 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3320/6700] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:05.478941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced73ba-048f-47e8-a3f1-f6543bf1f800 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:05.492333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced73ba-048f-47e8-a3f1-f6543bf1f800 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:05.498196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f846c2-b823-4f88-989c-1f9b344a0a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:05.498880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f846c2-b823-4f88-989c-1f9b344a0a8f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:05.499245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f846c2-b823-4f88-989c-1f9b344a0a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:05.499525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f846c2-b823-4f88-989c-1f9b344a0a8f admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:05.500830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f846c2-b823-4f88-989c-1f9b344a0a8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:14:05.501788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f846c2-b823-4f88-989c-1f9b344a0a8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:05.504117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f846c2-b823-4f88-989c-1f9b344a0a8f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:05.526524 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3381/6701] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:14:05.526697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787d287-8e13-477d-8d57-ce1d3d979492 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:05.537840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787d287-8e13-477d-8d57-ce1d3d979492 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:05.543054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f4880-662b-4520-88a6-5f3afdce36cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:05.543290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f4880-662b-4520-88a6-5f3afdce36cc admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:05.543577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f4880-662b-4520-88a6-5f3afdce36cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:05.543855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f4880-662b-4520-88a6-5f3afdce36cc admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:05.544521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f4880-662b-4520-88a6-5f3afdce36cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:05.546514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f4880-662b-4520-88a6-5f3afdce36cc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:05.579890 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3321/6702] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:05 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:14:05.583684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f846c2-b823-4f88-989c-1f9b344a0a8f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:05.595622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f846c2-b823-4f88-989c-1f9b344a0a8f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:05.689857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691b247-86ca-484f-b34d-5447bce53a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:05.690546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691b247-86ca-484f-b34d-5447bce53a80 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:05.690838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691b247-86ca-484f-b34d-5447bce53a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:05.691244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691b247-86ca-484f-b34d-5447bce53a80 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:05.691817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691b247-86ca-484f-b34d-5447bce53a80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:05.694003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691b247-86ca-484f-b34d-5447bce53a80 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:05.706726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:05.733172 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3382/6703] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:05 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:05.742026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f4880-662b-4520-88a6-5f3afdce36cc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:05.755953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f4880-662b-4520-88a6-5f3afdce36cc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:05.764919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf54f7-16c6-41db-bc83-05842759ba18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:05.766125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf54f7-16c6-41db-bc83-05842759ba18 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:05.767131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf54f7-16c6-41db-bc83-05842759ba18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:05.767870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf54f7-16c6-41db-bc83-05842759ba18 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:05.768825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedf54f7-16c6-41db-bc83-05842759ba18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:05.771277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedf54f7-16c6-41db-bc83-05842759ba18 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:05.789204 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3322/6704] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:05.793508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691b247-86ca-484f-b34d-5447bce53a80 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:05.808599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691b247-86ca-484f-b34d-5447bce53a80 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:05.816083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e21d51-b67c-4f53-b023-e6111405ebb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:05.816700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e21d51-b67c-4f53-b023-e6111405ebb1 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:05.817050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e21d51-b67c-4f53-b023-e6111405ebb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:05.817321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e21d51-b67c-4f53-b023-e6111405ebb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:05.817960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e21d51-b67c-4f53-b023-e6111405ebb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:05.819729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e21d51-b67c-4f53-b023-e6111405ebb1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:05.833101 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3383/6705] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:05.837871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf54f7-16c6-41db-bc83-05842759ba18 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:05.844454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:05.849653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf54f7-16c6-41db-bc83-05842759ba18 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:05.856503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120441ba-36c4-4aca-9124-8179ef5609c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:05.857217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120441ba-36c4-4aca-9124-8179ef5609c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:05.857672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120441ba-36c4-4aca-9124-8179ef5609c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:05.858029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120441ba-36c4-4aca-9124-8179ef5609c5 admin admin] PATH_INFO: `/v3/projects/d11d8a87dba14231a359abcb3b26059e/users/f54221040bea4aa3869b6693ce7a3645/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:05.874785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120441ba-36c4-4aca-9124-8179ef5609c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d11d8a87dba14231a359abcb3b26059e, user_id=f54221040bea4aa3869b6693ce7a3645, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:05.877489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120441ba-36c4-4aca-9124-8179ef5609c5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:05.893829 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3323/6706] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:05 2022] PUT /identity/v3/projects/d11d8a87dba14231a359abcb3b26059e/users/f54221040bea4aa3869b6693ce7a3645/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:05.904863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bdb8c-ab3c-43be-bb1c-af8fdafd4c63 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:05.905253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bdb8c-ab3c-43be-bb1c-af8fdafd4c63 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:05.905624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bdb8c-ab3c-43be-bb1c-af8fdafd4c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:05.940286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-474bdb8c-ab3c-43be-bb1c-af8fdafd4c63 None None] MFA Rules not processed for user `f54221040bea4aa3869b6693ce7a3645`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:05.941550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-474bdb8c-ab3c-43be-bb1c-af8fdafd4c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:06.014907 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3384/6707] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:06.027742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120441ba-36c4-4aca-9124-8179ef5609c5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:06.029508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-120441ba-36c4-4aca-9124-8179ef5609c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:06.091224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120441ba-36c4-4aca-9124-8179ef5609c5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:06.095191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beda770-642f-47df-be3a-64f9947642df service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:06.095822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beda770-642f-47df-be3a-64f9947642df service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:06.096154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beda770-642f-47df-be3a-64f9947642df service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:06.096438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beda770-642f-47df-be3a-64f9947642df service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:06.106058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beda770-642f-47df-be3a-64f9947642df service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:06.107571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beda770-642f-47df-be3a-64f9947642df service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:06.116939 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3324/6708] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:06.263977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc341dc7-1dc4-47a2-bbe4-4405c9dc1b95 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:06.264272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc341dc7-1dc4-47a2-bbe4-4405c9dc1b95 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:06.264546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc341dc7-1dc4-47a2-bbe4-4405c9dc1b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:06.284359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fc341dc7-1dc4-47a2-bbe4-4405c9dc1b95 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:06.285565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fc341dc7-1dc4-47a2-bbe4-4405c9dc1b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:06.373551 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3385/6709] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:14:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:06.400262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beda770-642f-47df-be3a-64f9947642df service glance] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:06.405942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4beda770-642f-47df-be3a-64f9947642df service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:06.489379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beda770-642f-47df-be3a-64f9947642df service glance] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:06.495349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db914182-dcfa-4079-a71c-4c92da599fba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:06.495866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db914182-dcfa-4079-a71c-4c92da599fba service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:06.496145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db914182-dcfa-4079-a71c-4c92da599fba service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:06.496417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db914182-dcfa-4079-a71c-4c92da599fba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:06.507025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db914182-dcfa-4079-a71c-4c92da599fba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:06.509297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db914182-dcfa-4079-a71c-4c92da599fba service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:06.519347 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3325/6710] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:14:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:14:06.938164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c231da-bc98-4034-bd71-eac4c4850c32 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:06.938554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c231da-bc98-4034-bd71-eac4c4850c32 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:06.938842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c231da-bc98-4034-bd71-eac4c4850c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:06.961648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f3c231da-bc98-4034-bd71-eac4c4850c32 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:06.962863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f3c231da-bc98-4034-bd71-eac4c4850c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:06.971085 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3386/6711] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:14:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:06.984638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db914182-dcfa-4079-a71c-4c92da599fba service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:06.997978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db914182-dcfa-4079-a71c-4c92da599fba service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:07.003815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da115f34-71bd-4642-93b2-87b88d35658b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:07.004412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da115f34-71bd-4642-93b2-87b88d35658b service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:07.004697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da115f34-71bd-4642-93b2-87b88d35658b service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:07.004974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da115f34-71bd-4642-93b2-87b88d35658b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:07.014808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da115f34-71bd-4642-93b2-87b88d35658b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:07.016514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da115f34-71bd-4642-93b2-87b88d35658b service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:07.026567 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3326/6712] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:14:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 21:14:07.429413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567f2ab-a3f0-40b3-8aa6-ef0311497c5b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:07.429917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567f2ab-a3f0-40b3-8aa6-ef0311497c5b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:07.431287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567f2ab-a3f0-40b3-8aa6-ef0311497c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:07.451665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d567f2ab-a3f0-40b3-8aa6-ef0311497c5b None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:07.452861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d567f2ab-a3f0-40b3-8aa6-ef0311497c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:07.460426 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3387/6713] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:14:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:07.472836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da115f34-71bd-4642-93b2-87b88d35658b service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:07.485002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da115f34-71bd-4642-93b2-87b88d35658b service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:07.489997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f237c7d-992c-4e83-a3d1-b9f9e4af4fdc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:07.490636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f237c7d-992c-4e83-a3d1-b9f9e4af4fdc service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:07.490917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f237c7d-992c-4e83-a3d1-b9f9e4af4fdc service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:07.491187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f237c7d-992c-4e83-a3d1-b9f9e4af4fdc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:07.500726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f237c7d-992c-4e83-a3d1-b9f9e4af4fdc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:07.502673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f237c7d-992c-4e83-a3d1-b9f9e4af4fdc service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:07.511551 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3327/6714] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:14:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:14:07.664099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211068de-025c-4c36-81f0-38f89aa3b7ee None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:07.664440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211068de-025c-4c36-81f0-38f89aa3b7ee None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:07.664728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211068de-025c-4c36-81f0-38f89aa3b7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:07.684709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-211068de-025c-4c36-81f0-38f89aa3b7ee None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:07.685982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-211068de-025c-4c36-81f0-38f89aa3b7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:07.693257 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3388/6715] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:14:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:07.707268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f237c7d-992c-4e83-a3d1-b9f9e4af4fdc service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:07.719872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f237c7d-992c-4e83-a3d1-b9f9e4af4fdc service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:07.724956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392360f5-248a-47b4-ba56-e54362b49ef4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:07.725553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392360f5-248a-47b4-ba56-e54362b49ef4 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:07.725832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392360f5-248a-47b4-ba56-e54362b49ef4 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:07.726106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392360f5-248a-47b4-ba56-e54362b49ef4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:07.735245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392360f5-248a-47b4-ba56-e54362b49ef4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:07.736827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392360f5-248a-47b4-ba56-e54362b49ef4 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:07.745685 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3328/6716] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:14:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:14:07.928726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211068de-025c-4c36-81f0-38f89aa3b7ee None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:07.930807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-211068de-025c-4c36-81f0-38f89aa3b7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:08.004111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:08.011363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211068de-025c-4c36-81f0-38f89aa3b7ee None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:08.017434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0857c3f-e5db-4f01-afcd-928d9de4959d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:08.018143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0857c3f-e5db-4f01-afcd-928d9de4959d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:08.018582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0857c3f-e5db-4f01-afcd-928d9de4959d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:08.018879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0857c3f-e5db-4f01-afcd-928d9de4959d admin admin] PATH_INFO: `/v3/users/bf76b289d770472cb0141e987a3eeb2c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:08.021940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0857c3f-e5db-4f01-afcd-928d9de4959d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf76b289d770472cb0141e987a3eeb2c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:08.024028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0857c3f-e5db-4f01-afcd-928d9de4959d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:08.096832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e0857c3f-e5db-4f01-afcd-928d9de4959d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf76b289d770472cb0141e987a3eeb2c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:08.108559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e0857c3f-e5db-4f01-afcd-928d9de4959d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf76b289d770472cb0141e987a3eeb2c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:08.124037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e0857c3f-e5db-4f01-afcd-928d9de4959d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf76b289d770472cb0141e987a3eeb2c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:08.124867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e0857c3f-e5db-4f01-afcd-928d9de4959d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf76b289d770472cb0141e987a3eeb2c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:08.135098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a8d6e-8571-4fe5-848e-065f81f06678 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:08.135507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a8d6e-8571-4fe5-848e-065f81f06678 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:08.135885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a8d6e-8571-4fe5-848e-065f81f06678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:08.136366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e0857c3f-e5db-4f01-afcd-928d9de4959d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf76b289d770472cb0141e987a3eeb2c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:08.143251 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3389/6717] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:07 2022] DELETE /identity/v3/users/bf76b289d770472cb0141e987a3eeb2c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:08.148288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb0a72-6500-40e0-98da-285e5ce16f6d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:08.148668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb0a72-6500-40e0-98da-285e5ce16f6d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:08.149046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb0a72-6500-40e0-98da-285e5ce16f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:08.158549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b05a8d6e-8571-4fe5-848e-065f81f06678 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:08.159934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b05a8d6e-8571-4fe5-848e-065f81f06678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:08.194803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:08.226646 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3329/6718] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:14:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:08.239771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a8d6e-8571-4fe5-848e-065f81f06678 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:08.241251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b05a8d6e-8571-4fe5-848e-065f81f06678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:08.323361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b6bb0a72-6500-40e0-98da-285e5ce16f6d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:08.324592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b6bb0a72-6500-40e0-98da-285e5ce16f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:08.347637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a8d6e-8571-4fe5-848e-065f81f06678 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:08.353421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052d473-8ff9-41a6-ae3c-3cc841847186 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:08.354102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052d473-8ff9-41a6-ae3c-3cc841847186 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:08.354479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052d473-8ff9-41a6-ae3c-3cc841847186 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:08.354880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052d473-8ff9-41a6-ae3c-3cc841847186 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:08.366077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052d473-8ff9-41a6-ae3c-3cc841847186 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:08.366077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052d473-8ff9-41a6-ae3c-3cc841847186 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:08.376444 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3330/6719] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:14:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 21:14:08.397615 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3390/6720] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:08.407245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052d473-8ff9-41a6-ae3c-3cc841847186 service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:08.408717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1052d473-8ff9-41a6-ae3c-3cc841847186 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:08.481617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052d473-8ff9-41a6-ae3c-3cc841847186 service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:08.485632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aedebd-117c-4588-a8d2-7e26b37dfdb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:08.486465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aedebd-117c-4588-a8d2-7e26b37dfdb5 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:08.486938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aedebd-117c-4588-a8d2-7e26b37dfdb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:08.487172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aedebd-117c-4588-a8d2-7e26b37dfdb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:08.499435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aedebd-117c-4588-a8d2-7e26b37dfdb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:08.501348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aedebd-117c-4588-a8d2-7e26b37dfdb5 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:08.512866 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3331/6721] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:08.602809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfe7b4-0306-40a4-bc8c-229e82584515 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:08.603293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfe7b4-0306-40a4-bc8c-229e82584515 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:08.603366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfe7b4-0306-40a4-bc8c-229e82584515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:08.624466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-55dfe7b4-0306-40a4-bc8c-229e82584515 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:08.626263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-55dfe7b4-0306-40a4-bc8c-229e82584515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:08.643312 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3391/6722] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:14:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:08.655853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aedebd-117c-4588-a8d2-7e26b37dfdb5 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:08.681875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aedebd-117c-4588-a8d2-7e26b37dfdb5 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:08.699190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa63ee0-13fc-4bff-9fcf-c32031200ffa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:08.700119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa63ee0-13fc-4bff-9fcf-c32031200ffa service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:08.700657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa63ee0-13fc-4bff-9fcf-c32031200ffa service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:08.701282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa63ee0-13fc-4bff-9fcf-c32031200ffa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:08.717929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa63ee0-13fc-4bff-9fcf-c32031200ffa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:08.720183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa63ee0-13fc-4bff-9fcf-c32031200ffa service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:08.735044 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3332/6723] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:14:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:14:08.875272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dfe7b4-0306-40a4-bc8c-229e82584515 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:08.876936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-55dfe7b4-0306-40a4-bc8c-229e82584515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:08.885702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa63ee0-13fc-4bff-9fcf-c32031200ffa service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:08.887389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-caa63ee0-13fc-4bff-9fcf-c32031200ffa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:08.896652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dfe7b4-0306-40a4-bc8c-229e82584515 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:08.903213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7879a48-41fd-40a6-9eb1-5ec7c6e2288c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:08.904044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7879a48-41fd-40a6-9eb1-5ec7c6e2288c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:08.904388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7879a48-41fd-40a6-9eb1-5ec7c6e2288c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:08.904793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7879a48-41fd-40a6-9eb1-5ec7c6e2288c admin admin] PATH_INFO: `/v3/projects/a2c9800616974afa9c35148898db5316` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:08.906221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7879a48-41fd-40a6-9eb1-5ec7c6e2288c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2c9800616974afa9c35148898db5316)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:08.909449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa63ee0-13fc-4bff-9fcf-c32031200ffa service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:08.916582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7879a48-41fd-40a6-9eb1-5ec7c6e2288c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:08.916857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cdb740-29e5-4bf8-a3dc-c5f40bf8b1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:08.917499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdb740-29e5-4bf8-a3dc-c5f40bf8b1e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:08.917781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdb740-29e5-4bf8-a3dc-c5f40bf8b1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:08.918053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdb740-29e5-4bf8-a3dc-c5f40bf8b1e6 admin admin] PATH_INFO: `/v3/users/f54221040bea4aa3869b6693ce7a3645` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:08.919757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cdb740-29e5-4bf8-a3dc-c5f40bf8b1e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f54221040bea4aa3869b6693ce7a3645)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:08.928222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cdb740-29e5-4bf8-a3dc-c5f40bf8b1e6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:08.981371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f7879a48-41fd-40a6-9eb1-5ec7c6e2288c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2c9800616974afa9c35148898db5316', 'request_id': 'req-f7879a48-41fd-40a6-9eb1-5ec7c6e2288c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:08.981558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32cdb740-29e5-4bf8-a3dc-c5f40bf8b1e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f54221040bea4aa3869b6693ce7a3645'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:08.983776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32cdb740-29e5-4bf8-a3dc-c5f40bf8b1e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f54221040bea4aa3869b6693ce7a3645'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:08.984319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32cdb740-29e5-4bf8-a3dc-c5f40bf8b1e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f54221040bea4aa3869b6693ce7a3645'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:08.988490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f7879a48-41fd-40a6-9eb1-5ec7c6e2288c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2c9800616974afa9c35148898db5316', 'request_id': 'req-f7879a48-41fd-40a6-9eb1-5ec7c6e2288c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:08.988627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:08.994354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32cdb740-29e5-4bf8-a3dc-c5f40bf8b1e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f54221040bea4aa3869b6693ce7a3645'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:08.996675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f7879a48-41fd-40a6-9eb1-5ec7c6e2288c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2c9800616974afa9c35148898db5316', 'request_id': 'req-f7879a48-41fd-40a6-9eb1-5ec7c6e2288c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:08.999570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:09.013844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f7879a48-41fd-40a6-9eb1-5ec7c6e2288c admin admin] The token cache is being invalidate because project a2c9800616974afa9c35148898db5316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:09.014770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f7879a48-41fd-40a6-9eb1-5ec7c6e2288c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:09.015456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f7879a48-41fd-40a6-9eb1-5ec7c6e2288c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:09.017038 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3392/6724] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:08 2022] DELETE /identity/v3/projects/a2c9800616974afa9c35148898db5316 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:09.021016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-32cdb740-29e5-4bf8-a3dc-c5f40bf8b1e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f54221040bea4aa3869b6693ce7a3645'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:09.022171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a48fb5-27f4-4da4-822e-9e862f947773 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:09.022454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a48fb5-27f4-4da4-822e-9e862f947773 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:09.022786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a48fb5-27f4-4da4-822e-9e862f947773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:09.041324 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3333/6725] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:08 2022] DELETE /identity/v3/users/f54221040bea4aa3869b6693ce7a3645 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:09.045731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceb208-b0f3-44aa-b2fb-fdea5df58250 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:09.050542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceb208-b0f3-44aa-b2fb-fdea5df58250 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:09.050864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceb208-b0f3-44aa-b2fb-fdea5df58250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:09.061256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-08a48fb5-27f4-4da4-822e-9e862f947773 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:09.062595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-08a48fb5-27f4-4da4-822e-9e862f947773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:09.070804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7bceb208-b0f3-44aa-b2fb-fdea5df58250 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:09.072205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7bceb208-b0f3-44aa-b2fb-fdea5df58250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:09.138533 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3393/6727] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:09.138739 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3334/6727] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:09.157042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bceb208-b0f3-44aa-b2fb-fdea5df58250 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:09.158741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7bceb208-b0f3-44aa-b2fb-fdea5df58250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:09.232549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bceb208-b0f3-44aa-b2fb-fdea5df58250 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:09.236149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4931d-4164-4255-a57f-1db90568a604 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:09.236768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4931d-4164-4255-a57f-1db90568a604 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:09.237204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4931d-4164-4255-a57f-1db90568a604 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:09.237498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4931d-4164-4255-a57f-1db90568a604 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:09.248791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea38c7-dfb0-4a38-b5d1-02c8c722e4ae None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:09.249071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea38c7-dfb0-4a38-b5d1-02c8c722e4ae None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:09.249341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea38c7-dfb0-4a38-b5d1-02c8c722e4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:09.256099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4931d-4164-4255-a57f-1db90568a604 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:09.256099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4931d-4164-4255-a57f-1db90568a604 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:09.267908 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3335/6728] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:09.271000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f6ea38c7-dfb0-4a38-b5d1-02c8c722e4ae None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:09.272195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f6ea38c7-dfb0-4a38-b5d1-02c8c722e4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:09.280452 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3394/6729] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:09.284562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4931d-4164-4255-a57f-1db90568a604 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:09.296911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4931d-4164-4255-a57f-1db90568a604 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:09.301910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b8f2e-dc89-4267-b821-7859620c2db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:09.302535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b8f2e-dc89-4267-b821-7859620c2db4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:09.302817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b8f2e-dc89-4267-b821-7859620c2db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:09.303153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b8f2e-dc89-4267-b821-7859620c2db4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:09.304165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b8f2e-dc89-4267-b821-7859620c2db4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:14:09.304468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b8f2e-dc89-4267-b821-7859620c2db4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:09.306704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b8f2e-dc89-4267-b821-7859620c2db4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:09.319712 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3336/6730] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:14:09.324645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea38c7-dfb0-4a38-b5d1-02c8c722e4ae None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:09.338959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea38c7-dfb0-4a38-b5d1-02c8c722e4ae None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:09.345808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecccee0-f33c-494a-bd33-6d8c623d6d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:09.346690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecccee0-f33c-494a-bd33-6d8c623d6d7e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:09.346798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecccee0-f33c-494a-bd33-6d8c623d6d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:09.347202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecccee0-f33c-494a-bd33-6d8c623d6d7e admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:09.348039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecccee0-f33c-494a-bd33-6d8c623d6d7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:09.350534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecccee0-f33c-494a-bd33-6d8c623d6d7e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:09.387636 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3395/6731] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:09 2022] POST /identity/v3/projects => generated 393 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:09.388802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b8f2e-dc89-4267-b821-7859620c2db4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:09.402361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b8f2e-dc89-4267-b821-7859620c2db4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:09.473217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1549c-0597-46d2-95b9-cd352e64c214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:09.473947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1549c-0597-46d2-95b9-cd352e64c214 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:09.474131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1549c-0597-46d2-95b9-cd352e64c214 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:09.474411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1549c-0597-46d2-95b9-cd352e64c214 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:09.475275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb1549c-0597-46d2-95b9-cd352e64c214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:09.477870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb1549c-0597-46d2-95b9-cd352e64c214 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:09.491866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:09.513835 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3337/6732] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:09 2022] POST /identity/v3/users => generated 297 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:09.515270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecccee0-f33c-494a-bd33-6d8c623d6d7e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:09.527000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecccee0-f33c-494a-bd33-6d8c623d6d7e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:09.534423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47512d76-2bf8-4c7b-a197-db8c19ad47f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:09.535208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47512d76-2bf8-4c7b-a197-db8c19ad47f8 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:09.535493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47512d76-2bf8-4c7b-a197-db8c19ad47f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:09.535824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47512d76-2bf8-4c7b-a197-db8c19ad47f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:09.536415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47512d76-2bf8-4c7b-a197-db8c19ad47f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:09.538571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47512d76-2bf8-4c7b-a197-db8c19ad47f8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:09.564764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1549c-0597-46d2-95b9-cd352e64c214 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:09.564872 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3396/6733] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:09.577221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47512d76-2bf8-4c7b-a197-db8c19ad47f8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:09.580243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1549c-0597-46d2-95b9-cd352e64c214 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:09.583850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-47512d76-2bf8-4c7b-a197-db8c19ad47f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:09.589117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2f78b-8317-45fb-9f68-3957aba23114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:09.590243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2f78b-8317-45fb-9f68-3957aba23114 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:09.590817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2f78b-8317-45fb-9f68-3957aba23114 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:09.591521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2f78b-8317-45fb-9f68-3957aba23114 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:09.593234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2f78b-8317-45fb-9f68-3957aba23114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:09.596317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2f78b-8317-45fb-9f68-3957aba23114 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:09.605791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47512d76-2bf8-4c7b-a197-db8c19ad47f8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:09.611474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f43e95-87a5-4451-a44d-dfabe1c72d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:09.612114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f43e95-87a5-4451-a44d-dfabe1c72d08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:09.612448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f43e95-87a5-4451-a44d-dfabe1c72d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:09.612731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f43e95-87a5-4451-a44d-dfabe1c72d08 admin admin] PATH_INFO: `/v3/projects/d11d8a87dba14231a359abcb3b26059e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:09.614046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f43e95-87a5-4451-a44d-dfabe1c72d08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d11d8a87dba14231a359abcb3b26059e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:09.616122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f43e95-87a5-4451-a44d-dfabe1c72d08 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:09.624572 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3338/6734] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:14:09.630193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2f78b-8317-45fb-9f68-3957aba23114 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:09.642913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2f78b-8317-45fb-9f68-3957aba23114 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:09.651826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa48f6-e470-4492-9450-02a8df8d8156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:09.652717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa48f6-e470-4492-9450-02a8df8d8156 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:09.653106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa48f6-e470-4492-9450-02a8df8d8156 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:09.653454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa48f6-e470-4492-9450-02a8df8d8156 admin admin] PATH_INFO: `/v3/projects/71511e85c1b947e3b03501b27d70d560/users/f77486ef55a0469a99c3795b626bee1a/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:09.678556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa48f6-e470-4492-9450-02a8df8d8156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71511e85c1b947e3b03501b27d70d560, user_id=f77486ef55a0469a99c3795b626bee1a, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:09.679940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d9f43e95-87a5-4451-a44d-dfabe1c72d08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd11d8a87dba14231a359abcb3b26059e', 'request_id': 'req-d9f43e95-87a5-4451-a44d-dfabe1c72d08'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:09.681638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa48f6-e470-4492-9450-02a8df8d8156 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:09.691671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d9f43e95-87a5-4451-a44d-dfabe1c72d08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd11d8a87dba14231a359abcb3b26059e', 'request_id': 'req-d9f43e95-87a5-4451-a44d-dfabe1c72d08'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:09.698246 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3339/6735] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:09 2022] PUT /identity/v3/projects/71511e85c1b947e3b03501b27d70d560/users/f77486ef55a0469a99c3795b626bee1a/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:09.708022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d9f43e95-87a5-4451-a44d-dfabe1c72d08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd11d8a87dba14231a359abcb3b26059e', 'request_id': 'req-d9f43e95-87a5-4451-a44d-dfabe1c72d08'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:09.712057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321fd71-8183-4665-8b09-53ab6a591c35 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:09.712450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321fd71-8183-4665-8b09-53ab6a591c35 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:09.712812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321fd71-8183-4665-8b09-53ab6a591c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:09.723178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d9f43e95-87a5-4451-a44d-dfabe1c72d08 admin admin] The token cache is being invalidate because project d11d8a87dba14231a359abcb3b26059e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:09.727706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d9f43e95-87a5-4451-a44d-dfabe1c72d08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:09.727706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d9f43e95-87a5-4451-a44d-dfabe1c72d08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:09.727706 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3397/6736] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:09 2022] DELETE /identity/v3/projects/d11d8a87dba14231a359abcb3b26059e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:09.731327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2529d-7223-4839-af81-6e44b46cb892 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:09.731718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2529d-7223-4839-af81-6e44b46cb892 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:09.732222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2529d-7223-4839-af81-6e44b46cb892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:09.749692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b321fd71-8183-4665-8b09-53ab6a591c35 None None] MFA Rules not processed for user `f77486ef55a0469a99c3795b626bee1a`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:09.750956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b321fd71-8183-4665-8b09-53ab6a591c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:09.754622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-12a2529d-7223-4839-af81-6e44b46cb892 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:09.756283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-12a2529d-7223-4839-af81-6e44b46cb892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:09.793619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=3,keystone:DELETE=8,keystone:SELECT=89 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:09.837696 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3340/6737] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:09.842010 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3398/6738] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:09.848552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321fd71-8183-4665-8b09-53ab6a591c35 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:09.849684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b321fd71-8183-4665-8b09-53ab6a591c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:09.850022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16988b77-be6f-4ed5-9590-24e28e5a25cf None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:09.850405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16988b77-be6f-4ed5-9590-24e28e5a25cf None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:09.850764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16988b77-be6f-4ed5-9590-24e28e5a25cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:09.873585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321fd71-8183-4665-8b09-53ab6a591c35 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:09.878149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-16988b77-be6f-4ed5-9590-24e28e5a25cf None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:09.879601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-16988b77-be6f-4ed5-9590-24e28e5a25cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:09.885659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15050970-2e67-4508-90ab-de8086f465ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:09.886282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15050970-2e67-4508-90ab-de8086f465ca admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:09.886603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15050970-2e67-4508-90ab-de8086f465ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:09.886882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15050970-2e67-4508-90ab-de8086f465ca admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:09.887510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15050970-2e67-4508-90ab-de8086f465ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:09.889505 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3399/6739] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:09.890181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15050970-2e67-4508-90ab-de8086f465ca admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:09.893809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16988b77-be6f-4ed5-9590-24e28e5a25cf None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:09.912875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16988b77-be6f-4ed5-9590-24e28e5a25cf None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:09.919582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9a1cf-de04-4762-89f0-65ca502278da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:09.920566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9a1cf-de04-4762-89f0-65ca502278da admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:09.921043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9a1cf-de04-4762-89f0-65ca502278da admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:09.921786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9a1cf-de04-4762-89f0-65ca502278da admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:09.922787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9a1cf-de04-4762-89f0-65ca502278da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:14:09.923251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9a1cf-de04-4762-89f0-65ca502278da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:09.927039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9a1cf-de04-4762-89f0-65ca502278da admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:09.935868 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3341/6740] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:09 2022] POST /identity/v3/projects => generated 391 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:14:09.943869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15050970-2e67-4508-90ab-de8086f465ca admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:09.952221 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3400/6741] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:14:09.953887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9a1cf-de04-4762-89f0-65ca502278da admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:09.967697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9a1cf-de04-4762-89f0-65ca502278da admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:10.041688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15050970-2e67-4508-90ab-de8086f465ca admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:10.052860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2c5fd-bf09-4bc8-a6a7-d6722642f84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:10.053538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2c5fd-bf09-4bc8-a6a7-d6722642f84f admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:10.053989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2c5fd-bf09-4bc8-a6a7-d6722642f84f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:10.054278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2c5fd-bf09-4bc8-a6a7-d6722642f84f admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:10.055008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2c5fd-bf09-4bc8-a6a7-d6722642f84f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:10.057025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2c5fd-bf09-4bc8-a6a7-d6722642f84f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:10.057517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9be3b-b374-401d-b61b-c31965cd76a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:10.058124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9be3b-b374-401d-b61b-c31965cd76a0 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:10.058403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9be3b-b374-401d-b61b-c31965cd76a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:10.058710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9be3b-b374-401d-b61b-c31965cd76a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:10.059332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9be3b-b374-401d-b61b-c31965cd76a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:10.061356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9be3b-b374-401d-b61b-c31965cd76a0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:10.091277 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3342/6742] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:09 2022] POST /identity/v3/users => generated 302 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:10.092338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2c5fd-bf09-4bc8-a6a7-d6722642f84f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:10.105706 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3401/6743] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:09 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:10.106959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9be3b-b374-401d-b61b-c31965cd76a0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:10.107825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2c5fd-bf09-4bc8-a6a7-d6722642f84f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:10.116690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:10.172906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9be3b-b374-401d-b61b-c31965cd76a0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:10.178447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5615c-fa9e-498d-b8c1-915b97f521d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:10.179152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5615c-fa9e-498d-b8c1-915b97f521d7 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:10.179500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5615c-fa9e-498d-b8c1-915b97f521d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:10.179613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e768f8-ddbc-49a0-a7af-886303d48141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:10.179921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5615c-fa9e-498d-b8c1-915b97f521d7 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:10.180638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5615c-fa9e-498d-b8c1-915b97f521d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:10.180772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e768f8-ddbc-49a0-a7af-886303d48141 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:10.181062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e768f8-ddbc-49a0-a7af-886303d48141 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:10.181341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e768f8-ddbc-49a0-a7af-886303d48141 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:10.181993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e768f8-ddbc-49a0-a7af-886303d48141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:10.182827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5615c-fa9e-498d-b8c1-915b97f521d7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:10.183612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e768f8-ddbc-49a0-a7af-886303d48141 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:10.196879 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3343/6744] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:10.201083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e768f8-ddbc-49a0-a7af-886303d48141 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:10.208143 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3402/6745] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:10 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:10.212382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5615c-fa9e-498d-b8c1-915b97f521d7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:10.212795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e768f8-ddbc-49a0-a7af-886303d48141 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:10.217807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccca71-2d62-4a52-9e28-797366ca17e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:10.218612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccca71-2d62-4a52-9e28-797366ca17e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:10.218900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccca71-2d62-4a52-9e28-797366ca17e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:10.219179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccca71-2d62-4a52-9e28-797366ca17e0 admin admin] PATH_INFO: `/v3/projects/8de5bb9a57014d588d6a62e872f0ec3c/users/39e3f059dc9c4fd78a6799b5b209d710/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:10.234748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5615c-fa9e-498d-b8c1-915b97f521d7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:10.234748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f265b-a8c0-4d74-9904-58315ee1ea7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:10.244152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f265b-a8c0-4d74-9904-58315ee1ea7b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:10.244152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f265b-a8c0-4d74-9904-58315ee1ea7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:10.244152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f265b-a8c0-4d74-9904-58315ee1ea7b admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:10.244690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ccca71-2d62-4a52-9e28-797366ca17e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8de5bb9a57014d588d6a62e872f0ec3c, user_id=39e3f059dc9c4fd78a6799b5b209d710, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:10.244780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f265b-a8c0-4d74-9904-58315ee1ea7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:10.244780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f265b-a8c0-4d74-9904-58315ee1ea7b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:10.244958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ccca71-2d62-4a52-9e28-797366ca17e0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:10.249701 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3403/6746] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:10.255247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f265b-a8c0-4d74-9904-58315ee1ea7b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:10.259247 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3344/6747] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:10 2022] PUT /identity/v3/projects/8de5bb9a57014d588d6a62e872f0ec3c/users/39e3f059dc9c4fd78a6799b5b209d710/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:10.260194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccca71-2d62-4a52-9e28-797366ca17e0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:10.266402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f265b-a8c0-4d74-9904-58315ee1ea7b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:10.286337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:10.337102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f60c48-e2a7-4280-bd46-c994194ee58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:10.338462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f60c48-e2a7-4280-bd46-c994194ee58e admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:10.338842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f60c48-e2a7-4280-bd46-c994194ee58e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:10.339181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f60c48-e2a7-4280-bd46-c994194ee58e admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:10.339802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccca71-2d62-4a52-9e28-797366ca17e0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:10.339969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f60c48-e2a7-4280-bd46-c994194ee58e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:10.341876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f60c48-e2a7-4280-bd46-c994194ee58e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:10.344671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9576f-2b9e-48e5-95ea-bd1aaa04fb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:10.345279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9576f-2b9e-48e5-95ea-bd1aaa04fb96 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:10.345561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9576f-2b9e-48e5-95ea-bd1aaa04fb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:10.345835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9576f-2b9e-48e5-95ea-bd1aaa04fb96 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:10.346476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9576f-2b9e-48e5-95ea-bd1aaa04fb96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:10.348246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9576f-2b9e-48e5-95ea-bd1aaa04fb96 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:10.360870 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3404/6748] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:14:10.361029 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3345/6749] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:14:10.362064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f60c48-e2a7-4280-bd46-c994194ee58e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:10.362651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9576f-2b9e-48e5-95ea-bd1aaa04fb96 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:10.373421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f60c48-e2a7-4280-bd46-c994194ee58e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:10.373565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9576f-2b9e-48e5-95ea-bd1aaa04fb96 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:10.378194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4421d7f-67ca-492f-bb50-8621e703bb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:10.378555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d6ff3-7d80-4a8b-8418-fb525c6506d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:10.378980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4421d7f-67ca-492f-bb50-8621e703bb91 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:10.379462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d6ff3-7d80-4a8b-8418-fb525c6506d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:10.379670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4421d7f-67ca-492f-bb50-8621e703bb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:10.379789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d6ff3-7d80-4a8b-8418-fb525c6506d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:10.380053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d6ff3-7d80-4a8b-8418-fb525c6506d9 admin admin] PATH_INFO: `/v3/projects/7e4e4ae8ef404433b10c730c58cea332/users/b0976775875b4b7db4e9d6a1134cbc8a/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:10.380171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4421d7f-67ca-492f-bb50-8621e703bb91 admin admin] PATH_INFO: `/v3/projects/8de5bb9a57014d588d6a62e872f0ec3c/users/39e3f059dc9c4fd78a6799b5b209d710/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:10.382649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4421d7f-67ca-492f-bb50-8621e703bb91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8de5bb9a57014d588d6a62e872f0ec3c, user_id=39e3f059dc9c4fd78a6799b5b209d710, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:10.386022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4421d7f-67ca-492f-bb50-8621e703bb91 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:10.396494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911d6ff3-7d80-4a8b-8418-fb525c6506d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e4e4ae8ef404433b10c730c58cea332, user_id=b0976775875b4b7db4e9d6a1134cbc8a, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:10.406372 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3346/6750] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:10 2022] PUT /identity/v3/projects/8de5bb9a57014d588d6a62e872f0ec3c/users/39e3f059dc9c4fd78a6799b5b209d710/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:10.407392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911d6ff3-7d80-4a8b-8418-fb525c6506d9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:10.407886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d51c1-307b-4290-8b3c-505df2d20ad3 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:10.408197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d51c1-307b-4290-8b3c-505df2d20ad3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:10.408497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d51c1-307b-4290-8b3c-505df2d20ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:10.432974 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3405/6751] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:10 2022] PUT /identity/v3/projects/7e4e4ae8ef404433b10c730c58cea332/users/b0976775875b4b7db4e9d6a1134cbc8a/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:10.444626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c73d51c1-307b-4290-8b3c-505df2d20ad3 None None] MFA Rules not processed for user `39e3f059dc9c4fd78a6799b5b209d710`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:10.446530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c73d51c1-307b-4290-8b3c-505df2d20ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:10.519566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:10.531846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e191924-77d1-4765-adfe-bb12a89b4a60 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:10.532155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e191924-77d1-4765-adfe-bb12a89b4a60 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:10.532569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e191924-77d1-4765-adfe-bb12a89b4a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:10.541913 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3347/6752] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:10 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:10.557820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d51c1-307b-4290-8b3c-505df2d20ad3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:10.559396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c73d51c1-307b-4290-8b3c-505df2d20ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:10.564512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2e191924-77d1-4765-adfe-bb12a89b4a60 None None] MFA Rules not processed for user `b0976775875b4b7db4e9d6a1134cbc8a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:10.565947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2e191924-77d1-4765-adfe-bb12a89b4a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:10.642918 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3406/6753] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:10.652048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d51c1-307b-4290-8b3c-505df2d20ad3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:10.656531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e191924-77d1-4765-adfe-bb12a89b4a60 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:10.658363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2e191924-77d1-4765-adfe-bb12a89b4a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:10.658531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a78e1-ea5b-4ebb-a29f-3646982faceb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:10.659101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a78e1-ea5b-4ebb-a29f-3646982faceb service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:10.659332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a78e1-ea5b-4ebb-a29f-3646982faceb service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:10.659602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a78e1-ea5b-4ebb-a29f-3646982faceb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:10.728061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a78e1-ea5b-4ebb-a29f-3646982faceb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:10.728296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e191924-77d1-4765-adfe-bb12a89b4a60 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:10.729669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a78e1-ea5b-4ebb-a29f-3646982faceb service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:10.732050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef896e38-ce4f-4264-bdd6-0fadb7538c93 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:10.732831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef896e38-ce4f-4264-bdd6-0fadb7538c93 service glance] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:10.733115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef896e38-ce4f-4264-bdd6-0fadb7538c93 service glance] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:10.733394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef896e38-ce4f-4264-bdd6-0fadb7538c93 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:10.739179 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3348/6754] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:10 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:14:10.743020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef896e38-ce4f-4264-bdd6-0fadb7538c93 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:10.744442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef896e38-ce4f-4264-bdd6-0fadb7538c93 service glance] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:10.754019 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3407/6755] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:14:10.871241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a78e1-ea5b-4ebb-a29f-3646982faceb service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:10.943804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a78e1-ea5b-4ebb-a29f-3646982faceb service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:10.949059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28562c1-c853-4aa0-92ac-1806f05a8d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:10.949802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28562c1-c853-4aa0-92ac-1806f05a8d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:10.950149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28562c1-c853-4aa0-92ac-1806f05a8d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:10.950517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28562c1-c853-4aa0-92ac-1806f05a8d5b admin admin] PATH_INFO: `/v3/users/b0976775875b4b7db4e9d6a1134cbc8a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:10.951740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28562c1-c853-4aa0-92ac-1806f05a8d5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0976775875b4b7db4e9d6a1134cbc8a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:10.953741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28562c1-c853-4aa0-92ac-1806f05a8d5b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:10.990800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:11.011583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e28562c1-c853-4aa0-92ac-1806f05a8d5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0976775875b4b7db4e9d6a1134cbc8a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:11.012079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e28562c1-c853-4aa0-92ac-1806f05a8d5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0976775875b4b7db4e9d6a1134cbc8a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:11.012720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e28562c1-c853-4aa0-92ac-1806f05a8d5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0976775875b4b7db4e9d6a1134cbc8a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:11.023105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e28562c1-c853-4aa0-92ac-1806f05a8d5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0976775875b4b7db4e9d6a1134cbc8a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:11.052689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e28562c1-c853-4aa0-92ac-1806f05a8d5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0976775875b4b7db4e9d6a1134cbc8a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:11.068962 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3349/6756] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:10 2022] DELETE /identity/v3/users/b0976775875b4b7db4e9d6a1134cbc8a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:11.073914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2e23f-dbd4-4f25-95dc-20b8bd2a9f44 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:11.074205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2e23f-dbd4-4f25-95dc-20b8bd2a9f44 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:11.074747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2e23f-dbd4-4f25-95dc-20b8bd2a9f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:11.102850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a0b2e23f-dbd4-4f25-95dc-20b8bd2a9f44 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:11.103989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a0b2e23f-dbd4-4f25-95dc-20b8bd2a9f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:11.123593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:11.179278 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3408/6757] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:11.189525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28562c1-c853-4aa0-92ac-1806f05a8d5b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:11.191178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e28562c1-c853-4aa0-92ac-1806f05a8d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:11.254230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28562c1-c853-4aa0-92ac-1806f05a8d5b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:11.257929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d4bee-fb32-4941-908b-507fc5057e29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:11.259157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d4bee-fb32-4941-908b-507fc5057e29 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:11.259441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d4bee-fb32-4941-908b-507fc5057e29 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:11.259749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d4bee-fb32-4941-908b-507fc5057e29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:11.269344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d4bee-fb32-4941-908b-507fc5057e29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:11.270839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d4bee-fb32-4941-908b-507fc5057e29 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:11.281822 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3350/6758] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:11.595740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2e23f-dbd4-4f25-95dc-20b8bd2a9f44 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:11.597268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a0b2e23f-dbd4-4f25-95dc-20b8bd2a9f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:11.614439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2e23f-dbd4-4f25-95dc-20b8bd2a9f44 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:11.622145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659e9d8-5763-4e8a-84a8-90ff882d7ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:11.623072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659e9d8-5763-4e8a-84a8-90ff882d7ac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:11.624326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659e9d8-5763-4e8a-84a8-90ff882d7ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:11.624711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659e9d8-5763-4e8a-84a8-90ff882d7ac6 admin admin] PATH_INFO: `/v3/projects/7e4e4ae8ef404433b10c730c58cea332` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:11.626753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659e9d8-5763-4e8a-84a8-90ff882d7ac6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e4e4ae8ef404433b10c730c58cea332)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:11.629290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659e9d8-5763-4e8a-84a8-90ff882d7ac6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:11.698617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f659e9d8-5763-4e8a-84a8-90ff882d7ac6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e4e4ae8ef404433b10c730c58cea332', 'request_id': 'req-f659e9d8-5763-4e8a-84a8-90ff882d7ac6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:11.707411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f659e9d8-5763-4e8a-84a8-90ff882d7ac6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e4e4ae8ef404433b10c730c58cea332', 'request_id': 'req-f659e9d8-5763-4e8a-84a8-90ff882d7ac6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:11.714246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f659e9d8-5763-4e8a-84a8-90ff882d7ac6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e4e4ae8ef404433b10c730c58cea332', 'request_id': 'req-f659e9d8-5763-4e8a-84a8-90ff882d7ac6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:11.732591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f659e9d8-5763-4e8a-84a8-90ff882d7ac6 admin admin] The token cache is being invalidate because project 7e4e4ae8ef404433b10c730c58cea332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:11.733048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f659e9d8-5763-4e8a-84a8-90ff882d7ac6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:11.733993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f659e9d8-5763-4e8a-84a8-90ff882d7ac6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:11.736268 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3409/6759] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:11 2022] DELETE /identity/v3/projects/7e4e4ae8ef404433b10c730c58cea332 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:11.741175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daedf6-fbdd-475a-8abf-9c3ebe206b7a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:11.741676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daedf6-fbdd-475a-8abf-9c3ebe206b7a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:11.741764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daedf6-fbdd-475a-8abf-9c3ebe206b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:11.765769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-03daedf6-fbdd-475a-8abf-9c3ebe206b7a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:11.767282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-03daedf6-fbdd-475a-8abf-9c3ebe206b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:11.847761 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3351/6760] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:11.855967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d474c4b-0475-492d-a3bb-0387981ac6aa None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:11.856184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d474c4b-0475-492d-a3bb-0387981ac6aa None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:11.856534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d474c4b-0475-492d-a3bb-0387981ac6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:11.879398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8d474c4b-0475-492d-a3bb-0387981ac6aa None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:11.880654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8d474c4b-0475-492d-a3bb-0387981ac6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:11.889556 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3410/6761] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:11.893720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03daedf6-fbdd-475a-8abf-9c3ebe206b7a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:11.906330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03daedf6-fbdd-475a-8abf-9c3ebe206b7a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:11.912357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e8574-44fc-42cd-9ea7-34dd14f244d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:11.913000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e8574-44fc-42cd-9ea7-34dd14f244d9 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:11.913281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e8574-44fc-42cd-9ea7-34dd14f244d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:11.913554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e8574-44fc-42cd-9ea7-34dd14f244d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:11.914792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e8574-44fc-42cd-9ea7-34dd14f244d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:14:11.915407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e8574-44fc-42cd-9ea7-34dd14f244d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:11.917711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e8574-44fc-42cd-9ea7-34dd14f244d9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:11.941768 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3352/6762] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:14:11.943347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d474c4b-0475-492d-a3bb-0387981ac6aa None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:11.955367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d474c4b-0475-492d-a3bb-0387981ac6aa None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:11.961817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb510d-44dd-44bb-8806-31256293e572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:11.962441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb510d-44dd-44bb-8806-31256293e572 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:11.962764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb510d-44dd-44bb-8806-31256293e572 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:11.963578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb510d-44dd-44bb-8806-31256293e572 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:11.963784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb510d-44dd-44bb-8806-31256293e572 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:11.965668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb510d-44dd-44bb-8806-31256293e572 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:12.003365 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3411/6763] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:11 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:12.004749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e8574-44fc-42cd-9ea7-34dd14f244d9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:12.016276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e8574-44fc-42cd-9ea7-34dd14f244d9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:12.076212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:12.115366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5083951-2835-4d8c-9369-4135753a4de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:12.116121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5083951-2835-4d8c-9369-4135753a4de0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:12.116496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5083951-2835-4d8c-9369-4135753a4de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:12.117055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5083951-2835-4d8c-9369-4135753a4de0 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:12.118289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5083951-2835-4d8c-9369-4135753a4de0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:12.120514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5083951-2835-4d8c-9369-4135753a4de0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:12.144549 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3353/6764] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:12 2022] POST /identity/v3/users => generated 286 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:12.148844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb510d-44dd-44bb-8806-31256293e572 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:12.159798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb510d-44dd-44bb-8806-31256293e572 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:12.164715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f19a74-43ab-43dd-aaae-8f00ca61c98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:12.165323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f19a74-43ab-43dd-aaae-8f00ca61c98e admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:12.165631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f19a74-43ab-43dd-aaae-8f00ca61c98e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:12.165904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f19a74-43ab-43dd-aaae-8f00ca61c98e admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:12.166602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f19a74-43ab-43dd-aaae-8f00ca61c98e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:12.168165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f19a74-43ab-43dd-aaae-8f00ca61c98e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:12.178891 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3412/6765] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:12.183386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5083951-2835-4d8c-9369-4135753a4de0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:12.193680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5083951-2835-4d8c-9369-4135753a4de0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:12.199407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9103a-cf05-41b6-beeb-9218a4b2ebda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:12.200186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9103a-cf05-41b6-beeb-9218a4b2ebda admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:12.200538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9103a-cf05-41b6-beeb-9218a4b2ebda admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:12.200863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9103a-cf05-41b6-beeb-9218a4b2ebda admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:12.201604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9103a-cf05-41b6-beeb-9218a4b2ebda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:12.203320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9103a-cf05-41b6-beeb-9218a4b2ebda admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:12.216734 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3354/6766] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:12.220722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f19a74-43ab-43dd-aaae-8f00ca61c98e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:12.231624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f19a74-43ab-43dd-aaae-8f00ca61c98e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:12.236325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14c534-cb32-4f58-aa36-50aded218119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:12.236977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14c534-cb32-4f58-aa36-50aded218119 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:12.237256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14c534-cb32-4f58-aa36-50aded218119 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:12.237529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14c534-cb32-4f58-aa36-50aded218119 admin admin] PATH_INFO: `/v3/projects/ecbe00d8f2584463ab0d87931b832e18/users/fae6dc37a58b4cc3b0a33775e96da406/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:12.253393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b14c534-cb32-4f58-aa36-50aded218119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecbe00d8f2584463ab0d87931b832e18, user_id=fae6dc37a58b4cc3b0a33775e96da406, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:12.256179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b14c534-cb32-4f58-aa36-50aded218119 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:12.271742 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3413/6767] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:12 2022] PUT /identity/v3/projects/ecbe00d8f2584463ab0d87931b832e18/users/fae6dc37a58b4cc3b0a33775e96da406/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:12.284216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdd7c4-0c4c-4155-a9c7-7bef79cb8904 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:12.284596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdd7c4-0c4c-4155-a9c7-7bef79cb8904 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:12.285060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdd7c4-0c4c-4155-a9c7-7bef79cb8904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:12.315987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-74bdd7c4-0c4c-4155-a9c7-7bef79cb8904 None None] MFA Rules not processed for user `fae6dc37a58b4cc3b0a33775e96da406`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:12.317238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-74bdd7c4-0c4c-4155-a9c7-7bef79cb8904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:12.428102 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3355/6768] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:12.448748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14c534-cb32-4f58-aa36-50aded218119 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:12.450827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2b14c534-cb32-4f58-aa36-50aded218119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:12.513983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:12.531159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14c534-cb32-4f58-aa36-50aded218119 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:12.535019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a33774-77ce-4b86-9c4f-10772485d685 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:12.535628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a33774-77ce-4b86-9c4f-10772485d685 service glance] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:12.535909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a33774-77ce-4b86-9c4f-10772485d685 service glance] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:12.536187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a33774-77ce-4b86-9c4f-10772485d685 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:12.545239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a33774-77ce-4b86-9c4f-10772485d685 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:12.546795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a33774-77ce-4b86-9c4f-10772485d685 service glance] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:12.556490 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3414/6769] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:13.487210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdd7c4-0c4c-4155-a9c7-7bef79cb8904 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:13.566939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdd7c4-0c4c-4155-a9c7-7bef79cb8904 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:13.572453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a92eb-a592-47db-9b3b-4958bf53a37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:13.573239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a92eb-a592-47db-9b3b-4958bf53a37a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:13.573351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a92eb-a592-47db-9b3b-4958bf53a37a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:13.573634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a92eb-a592-47db-9b3b-4958bf53a37a admin admin] PATH_INFO: `/v3/users/fae6dc37a58b4cc3b0a33775e96da406` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:13.574837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a92eb-a592-47db-9b3b-4958bf53a37a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fae6dc37a58b4cc3b0a33775e96da406)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:13.576774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a92eb-a592-47db-9b3b-4958bf53a37a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:13.636612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f95a92eb-a592-47db-9b3b-4958bf53a37a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fae6dc37a58b4cc3b0a33775e96da406'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:13.637710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f95a92eb-a592-47db-9b3b-4958bf53a37a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fae6dc37a58b4cc3b0a33775e96da406'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:13.638275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f95a92eb-a592-47db-9b3b-4958bf53a37a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fae6dc37a58b4cc3b0a33775e96da406'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:13.640297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:13.649614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f95a92eb-a592-47db-9b3b-4958bf53a37a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fae6dc37a58b4cc3b0a33775e96da406'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:13.671548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f95a92eb-a592-47db-9b3b-4958bf53a37a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fae6dc37a58b4cc3b0a33775e96da406'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:13.695256 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3356/6770] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:13 2022] DELETE /identity/v3/users/fae6dc37a58b4cc3b0a33775e96da406 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:13.700164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c437355-ff24-4a64-9a79-e82c601d03e3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:13.700645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c437355-ff24-4a64-9a79-e82c601d03e3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:13.700830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c437355-ff24-4a64-9a79-e82c601d03e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:13.719813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4c437355-ff24-4a64-9a79-e82c601d03e3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:13.721114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4c437355-ff24-4a64-9a79-e82c601d03e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:13.782421 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3415/6771] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:13.792930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a92eb-a592-47db-9b3b-4958bf53a37a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:13.794342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f95a92eb-a592-47db-9b3b-4958bf53a37a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:13.845563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c437355-ff24-4a64-9a79-e82c601d03e3 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:13.847278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4c437355-ff24-4a64-9a79-e82c601d03e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:13.860577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c437355-ff24-4a64-9a79-e82c601d03e3 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:13.861065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a92eb-a592-47db-9b3b-4958bf53a37a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:13.864813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0627b-9e24-479d-a3af-6159d805f9cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:13.865434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0627b-9e24-479d-a3af-6159d805f9cf service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:13.865713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0627b-9e24-479d-a3af-6159d805f9cf service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:13.865813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e8053-985a-4c1c-8648-b2e03df8ed4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:13.865981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0627b-9e24-479d-a3af-6159d805f9cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:13.866650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e8053-985a-4c1c-8648-b2e03df8ed4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:13.866947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e8053-985a-4c1c-8648-b2e03df8ed4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:13.867226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e8053-985a-4c1c-8648-b2e03df8ed4a admin admin] PATH_INFO: `/v3/users/b049b104d5d6487da4849ef06cda8075` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:13.869070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e8053-985a-4c1c-8648-b2e03df8ed4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b049b104d5d6487da4849ef06cda8075)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:13.871052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e8053-985a-4c1c-8648-b2e03df8ed4a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:13.880477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b0627b-9e24-479d-a3af-6159d805f9cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:13.882641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b0627b-9e24-479d-a3af-6159d805f9cf service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:13.898923 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3357/6772] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:13.928311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cc9e8053-985a-4c1c-8648-b2e03df8ed4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b049b104d5d6487da4849ef06cda8075'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:13.937755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cc9e8053-985a-4c1c-8648-b2e03df8ed4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b049b104d5d6487da4849ef06cda8075'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:13.956166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cc9e8053-985a-4c1c-8648-b2e03df8ed4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b049b104d5d6487da4849ef06cda8075'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:13.957045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cc9e8053-985a-4c1c-8648-b2e03df8ed4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b049b104d5d6487da4849ef06cda8075'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:13.968108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cc9e8053-985a-4c1c-8648-b2e03df8ed4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b049b104d5d6487da4849ef06cda8075'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:13.981711 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3416/6773] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:13 2022] DELETE /identity/v3/users/b049b104d5d6487da4849ef06cda8075 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:13.983866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f62858-b854-4c73-b55a-106b8eb427a7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:13.984280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f62858-b854-4c73-b55a-106b8eb427a7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:13.984654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f62858-b854-4c73-b55a-106b8eb427a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:14.009476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-46f62858-b854-4c73-b55a-106b8eb427a7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:14.010980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-46f62858-b854-4c73-b55a-106b8eb427a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:14.129453 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3358/6774] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:14.146858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e8053-985a-4c1c-8648-b2e03df8ed4a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:14.149003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cc9e8053-985a-4c1c-8648-b2e03df8ed4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:14.218968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e8053-985a-4c1c-8648-b2e03df8ed4a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:14.223983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba06ab6-e981-4fee-b5ad-4aa2b03ae92c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:14.224153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba06ab6-e981-4fee-b5ad-4aa2b03ae92c service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:14.224432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba06ab6-e981-4fee-b5ad-4aa2b03ae92c service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:14.224838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba06ab6-e981-4fee-b5ad-4aa2b03ae92c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:14.240317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f62858-b854-4c73-b55a-106b8eb427a7 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:14.241859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-46f62858-b854-4c73-b55a-106b8eb427a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:14.245402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba06ab6-e981-4fee-b5ad-4aa2b03ae92c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:14.247283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba06ab6-e981-4fee-b5ad-4aa2b03ae92c service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:14.256952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f62858-b854-4c73-b55a-106b8eb427a7 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:14.264813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2f321-2791-47b1-8e61-4edb73f39d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:14.265796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2f321-2791-47b1-8e61-4edb73f39d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:14.266087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2f321-2791-47b1-8e61-4edb73f39d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:14.266431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2f321-2791-47b1-8e61-4edb73f39d69 admin admin] PATH_INFO: `/v3/projects/ecbe00d8f2584463ab0d87931b832e18` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:14.267740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2f321-2791-47b1-8e61-4edb73f39d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecbe00d8f2584463ab0d87931b832e18)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:14.279387 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3417/6775] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:14:14.280349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2f321-2791-47b1-8e61-4edb73f39d69 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:14.283595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba06ab6-e981-4fee-b5ad-4aa2b03ae92c service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:14.285620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7ba06ab6-e981-4fee-b5ad-4aa2b03ae92c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:14.293922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:14.305209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba06ab6-e981-4fee-b5ad-4aa2b03ae92c service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:14.312412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676ecbb-afce-415d-8011-299cbdee7c2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:14.313242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676ecbb-afce-415d-8011-299cbdee7c2e service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:14.313670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676ecbb-afce-415d-8011-299cbdee7c2e service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:14.314083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676ecbb-afce-415d-8011-299cbdee7c2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:14.316607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4676ecbb-afce-415d-8011-299cbdee7c2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:14.358781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-79e2f321-2791-47b1-8e61-4edb73f39d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecbe00d8f2584463ab0d87931b832e18', 'request_id': 'req-79e2f321-2791-47b1-8e61-4edb73f39d69'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:14.369278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-79e2f321-2791-47b1-8e61-4edb73f39d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecbe00d8f2584463ab0d87931b832e18', 'request_id': 'req-79e2f321-2791-47b1-8e61-4edb73f39d69'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:14.377756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:14.379107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-79e2f321-2791-47b1-8e61-4edb73f39d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecbe00d8f2584463ab0d87931b832e18', 'request_id': 'req-79e2f321-2791-47b1-8e61-4edb73f39d69'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:14.405353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-79e2f321-2791-47b1-8e61-4edb73f39d69 admin admin] The token cache is being invalidate because project ecbe00d8f2584463ab0d87931b832e18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:14.405698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-79e2f321-2791-47b1-8e61-4edb73f39d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:14.407076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-79e2f321-2791-47b1-8e61-4edb73f39d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:14.413942 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3359/6776] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:14 2022] DELETE /identity/v3/projects/ecbe00d8f2584463ab0d87931b832e18 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:14.419956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba93da5-5dbf-4a58-8796-93fad9867651 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:14.420242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba93da5-5dbf-4a58-8796-93fad9867651 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:14.422742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba93da5-5dbf-4a58-8796-93fad9867651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:14.426810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4676ecbb-afce-415d-8011-299cbdee7c2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:14.428597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4676ecbb-afce-415d-8011-299cbdee7c2e service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:14.441657 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3418/6777] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:14 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:14.450351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5ba93da5-5dbf-4a58-8796-93fad9867651 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:14.452049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5ba93da5-5dbf-4a58-8796-93fad9867651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:14.553140 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3360/6778] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:14.562263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f4a1c-289d-47c6-96bf-234e0195a282 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:14.562757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f4a1c-289d-47c6-96bf-234e0195a282 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:14.563145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f4a1c-289d-47c6-96bf-234e0195a282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:14.585954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-752f4a1c-289d-47c6-96bf-234e0195a282 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:14.587353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-752f4a1c-289d-47c6-96bf-234e0195a282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:14.596883 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3419/6779] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:14.603152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba93da5-5dbf-4a58-8796-93fad9867651 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:14.612706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba93da5-5dbf-4a58-8796-93fad9867651 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:14.621211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26becd-b192-4216-bf9c-54428db87b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:14.622067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26becd-b192-4216-bf9c-54428db87b43 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:14.622480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26becd-b192-4216-bf9c-54428db87b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:14.622929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26becd-b192-4216-bf9c-54428db87b43 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:14.623801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26becd-b192-4216-bf9c-54428db87b43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:14:14.624147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26becd-b192-4216-bf9c-54428db87b43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:14.626075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26becd-b192-4216-bf9c-54428db87b43 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:14.636323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f4a1c-289d-47c6-96bf-234e0195a282 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:14.637835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-752f4a1c-289d-47c6-96bf-234e0195a282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:14.659592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f4a1c-289d-47c6-96bf-234e0195a282 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:14.667729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3255526-ea00-4489-9730-bf9bbc5caf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:14.668386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3255526-ea00-4489-9730-bf9bbc5caf05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:14.668664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3255526-ea00-4489-9730-bf9bbc5caf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:14.668933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3255526-ea00-4489-9730-bf9bbc5caf05 admin admin] PATH_INFO: `/v3/projects/817ad1c30bfc4b9c9030f36c70e07781` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:14.670205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3255526-ea00-4489-9730-bf9bbc5caf05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=817ad1c30bfc4b9c9030f36c70e07781)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:14.673305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3255526-ea00-4489-9730-bf9bbc5caf05 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:14.681916 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3361/6780] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:14:14.702723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26becd-b192-4216-bf9c-54428db87b43 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:14.713128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26becd-b192-4216-bf9c-54428db87b43 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:14.718748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d43d30-aa20-4eb0-9e71-6dece372d4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:14.719489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d43d30-aa20-4eb0-9e71-6dece372d4ae admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:14.719842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d43d30-aa20-4eb0-9e71-6dece372d4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:14.720222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d43d30-aa20-4eb0-9e71-6dece372d4ae admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:14.721058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d43d30-aa20-4eb0-9e71-6dece372d4ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:14.723562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d43d30-aa20-4eb0-9e71-6dece372d4ae admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:14.751714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3255526-ea00-4489-9730-bf9bbc5caf05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '817ad1c30bfc4b9c9030f36c70e07781', 'request_id': 'req-e3255526-ea00-4489-9730-bf9bbc5caf05'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:14.761284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3255526-ea00-4489-9730-bf9bbc5caf05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '817ad1c30bfc4b9c9030f36c70e07781', 'request_id': 'req-e3255526-ea00-4489-9730-bf9bbc5caf05'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:14.781399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3255526-ea00-4489-9730-bf9bbc5caf05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '817ad1c30bfc4b9c9030f36c70e07781', 'request_id': 'req-e3255526-ea00-4489-9730-bf9bbc5caf05'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:14.787291 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3362/6781] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:14 2022] POST /identity/v3/projects => generated 361 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:14.792767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d43d30-aa20-4eb0-9e71-6dece372d4ae admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:14.826330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3255526-ea00-4489-9730-bf9bbc5caf05 admin admin] The token cache is being invalidate because project 817ad1c30bfc4b9c9030f36c70e07781 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:14.826681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3255526-ea00-4489-9730-bf9bbc5caf05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:14.828228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3255526-ea00-4489-9730-bf9bbc5caf05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:14.844621 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3420/6782] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:14 2022] DELETE /identity/v3/projects/817ad1c30bfc4b9c9030f36c70e07781 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:14.918088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d43d30-aa20-4eb0-9e71-6dece372d4ae admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:14.925650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84917899-6c97-4af1-9863-c5acf5f6bb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:14.926755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84917899-6c97-4af1-9863-c5acf5f6bb80 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:14.927284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84917899-6c97-4af1-9863-c5acf5f6bb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:14.927827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84917899-6c97-4af1-9863-c5acf5f6bb80 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:14.928793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84917899-6c97-4af1-9863-c5acf5f6bb80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:14.931831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84917899-6c97-4af1-9863-c5acf5f6bb80 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:14.969975 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3363/6783] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:14 2022] POST /identity/v3/users => generated 281 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:14:14.974639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3255526-ea00-4489-9730-bf9bbc5caf05 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:14.976750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e3255526-ea00-4489-9730-bf9bbc5caf05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:15.110184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3255526-ea00-4489-9730-bf9bbc5caf05 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:15.116438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd37eb-760c-4ba7-94eb-b11a24d665a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:15.116438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd37eb-760c-4ba7-94eb-b11a24d665a2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:15.116720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd37eb-760c-4ba7-94eb-b11a24d665a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:15.117137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd37eb-760c-4ba7-94eb-b11a24d665a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:15.118422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd37eb-760c-4ba7-94eb-b11a24d665a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:15.120147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd37eb-760c-4ba7-94eb-b11a24d665a2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:15.139339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84917899-6c97-4af1-9863-c5acf5f6bb80 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:15.142084 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3421/6784] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:15.154864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84917899-6c97-4af1-9863-c5acf5f6bb80 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:15.164090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e714c2c-5ec4-47ce-9b1d-39a2597bd27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:15.165395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e714c2c-5ec4-47ce-9b1d-39a2597bd27c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:15.165914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e714c2c-5ec4-47ce-9b1d-39a2597bd27c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:15.166729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e714c2c-5ec4-47ce-9b1d-39a2597bd27c admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:15.167622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e714c2c-5ec4-47ce-9b1d-39a2597bd27c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:15.171682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e714c2c-5ec4-47ce-9b1d-39a2597bd27c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:15.197558 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3364/6785] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:14:15.202383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd37eb-760c-4ba7-94eb-b11a24d665a2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:15.222867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd37eb-760c-4ba7-94eb-b11a24d665a2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:15.230961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c869f-7a28-4dcb-9150-1c9d508f121c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:15.232654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c869f-7a28-4dcb-9150-1c9d508f121c admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:15.233172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c869f-7a28-4dcb-9150-1c9d508f121c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:15.234410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c869f-7a28-4dcb-9150-1c9d508f121c admin admin] PATH_INFO: `/v3/projects/70deb74154df46398fed12e6d9f12bf8/users/57f3c29a02f5418e82e216b8200272fb/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:15.246017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:15.286424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c869f-7a28-4dcb-9150-1c9d508f121c admin admin] RBAC: Authorizing `identity:create_grant(project_id=70deb74154df46398fed12e6d9f12bf8, user_id=57f3c29a02f5418e82e216b8200272fb, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:15.290396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c869f-7a28-4dcb-9150-1c9d508f121c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:15.312211 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3422/6786] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:15 2022] PUT /identity/v3/projects/70deb74154df46398fed12e6d9f12bf8/users/57f3c29a02f5418e82e216b8200272fb/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:15.322722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3503762-d780-4a3e-b303-1991ed91ead3 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:15.324304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3503762-d780-4a3e-b303-1991ed91ead3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:15.324728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3503762-d780-4a3e-b303-1991ed91ead3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:15.400316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e3503762-d780-4a3e-b303-1991ed91ead3 None None] MFA Rules not processed for user `57f3c29a02f5418e82e216b8200272fb`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:15.401583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e3503762-d780-4a3e-b303-1991ed91ead3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:15.452435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:15.521259 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3365/6787] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:15.542463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c869f-7a28-4dcb-9150-1c9d508f121c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:15.625927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c869f-7a28-4dcb-9150-1c9d508f121c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:15.634565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab694b-3ec2-47be-9adb-47b9dd3f2263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:15.635375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab694b-3ec2-47be-9adb-47b9dd3f2263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:15.635752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab694b-3ec2-47be-9adb-47b9dd3f2263 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:15.636165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab694b-3ec2-47be-9adb-47b9dd3f2263 admin admin] PATH_INFO: `/v3/users/57f3c29a02f5418e82e216b8200272fb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:15.638456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab694b-3ec2-47be-9adb-47b9dd3f2263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57f3c29a02f5418e82e216b8200272fb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:15.640757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab694b-3ec2-47be-9adb-47b9dd3f2263 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:15.710082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e6ab694b-3ec2-47be-9adb-47b9dd3f2263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57f3c29a02f5418e82e216b8200272fb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:15.720011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e6ab694b-3ec2-47be-9adb-47b9dd3f2263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57f3c29a02f5418e82e216b8200272fb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:15.735949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e6ab694b-3ec2-47be-9adb-47b9dd3f2263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57f3c29a02f5418e82e216b8200272fb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:15.736861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e6ab694b-3ec2-47be-9adb-47b9dd3f2263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57f3c29a02f5418e82e216b8200272fb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:15.747815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e6ab694b-3ec2-47be-9adb-47b9dd3f2263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57f3c29a02f5418e82e216b8200272fb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:15.755606 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3423/6788] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:15 2022] DELETE /identity/v3/users/57f3c29a02f5418e82e216b8200272fb => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:15.761245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcb114-3745-43b6-9561-5182bdc0ad26 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:15.761863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcb114-3745-43b6-9561-5182bdc0ad26 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:15.762277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcb114-3745-43b6-9561-5182bdc0ad26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:15.791585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9dfcb114-3745-43b6-9561-5182bdc0ad26 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:15.793469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9dfcb114-3745-43b6-9561-5182bdc0ad26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:15.873757 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3366/6789] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:15.885514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab694b-3ec2-47be-9adb-47b9dd3f2263 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:15.887750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e6ab694b-3ec2-47be-9adb-47b9dd3f2263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:15.970825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab694b-3ec2-47be-9adb-47b9dd3f2263 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:15.984425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693e9a2-aaaa-4b19-99f7-1a52b82bdbdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:15.986189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693e9a2-aaaa-4b19-99f7-1a52b82bdbdd service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:15.986689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693e9a2-aaaa-4b19-99f7-1a52b82bdbdd service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:15.987070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693e9a2-aaaa-4b19-99f7-1a52b82bdbdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:16.001593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693e9a2-aaaa-4b19-99f7-1a52b82bdbdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:16.003421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693e9a2-aaaa-4b19-99f7-1a52b82bdbdd service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:16.016113 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3424/6790] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:16.650726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcb114-3745-43b6-9561-5182bdc0ad26 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:16.652436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9dfcb114-3745-43b6-9561-5182bdc0ad26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:16.674813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcb114-3745-43b6-9561-5182bdc0ad26 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:16.683626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f42fc-b963-4542-b020-d4e57b325d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:16.684361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f42fc-b963-4542-b020-d4e57b325d7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:16.684837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f42fc-b963-4542-b020-d4e57b325d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:16.685002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f42fc-b963-4542-b020-d4e57b325d7d admin admin] PATH_INFO: `/v3/projects/70deb74154df46398fed12e6d9f12bf8` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:16.686451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f42fc-b963-4542-b020-d4e57b325d7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=70deb74154df46398fed12e6d9f12bf8)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:16.688573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f42fc-b963-4542-b020-d4e57b325d7d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:16.751683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a25f42fc-b963-4542-b020-d4e57b325d7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70deb74154df46398fed12e6d9f12bf8', 'request_id': 'req-a25f42fc-b963-4542-b020-d4e57b325d7d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:16.762770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a25f42fc-b963-4542-b020-d4e57b325d7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70deb74154df46398fed12e6d9f12bf8', 'request_id': 'req-a25f42fc-b963-4542-b020-d4e57b325d7d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:16.769871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a25f42fc-b963-4542-b020-d4e57b325d7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70deb74154df46398fed12e6d9f12bf8', 'request_id': 'req-a25f42fc-b963-4542-b020-d4e57b325d7d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:16.785467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a25f42fc-b963-4542-b020-d4e57b325d7d admin admin] The token cache is being invalidate because project 70deb74154df46398fed12e6d9f12bf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:16.785867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a25f42fc-b963-4542-b020-d4e57b325d7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:16.786692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a25f42fc-b963-4542-b020-d4e57b325d7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:16.788812 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3367/6791] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:16 2022] DELETE /identity/v3/projects/70deb74154df46398fed12e6d9f12bf8 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:16.795938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bc655-f899-47b8-bac9-fc0e7457e598 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:16.796457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bc655-f899-47b8-bac9-fc0e7457e598 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:16.796533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bc655-f899-47b8-bac9-fc0e7457e598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:16.818916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:16.830839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-128bc655-f899-47b8-bac9-fc0e7457e598 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:16.833696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-128bc655-f899-47b8-bac9-fc0e7457e598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:16.915529 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3425/6792] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:16.925239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d23089-efdb-4ae3-8875-c3e25772a412 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:16.926076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d23089-efdb-4ae3-8875-c3e25772a412 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:16.926662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d23089-efdb-4ae3-8875-c3e25772a412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:16.975714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-49d23089-efdb-4ae3-8875-c3e25772a412 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:16.984673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-49d23089-efdb-4ae3-8875-c3e25772a412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:16.994232 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3368/6793] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:17.001171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128bc655-f899-47b8-bac9-fc0e7457e598 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:17.030973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128bc655-f899-47b8-bac9-fc0e7457e598 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:17.039168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b55f3b-0ba1-4184-b6e5-fbffa478cc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:17.040344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b55f3b-0ba1-4184-b6e5-fbffa478cc0a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:17.040944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b55f3b-0ba1-4184-b6e5-fbffa478cc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:17.041444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b55f3b-0ba1-4184-b6e5-fbffa478cc0a admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:17.042444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b55f3b-0ba1-4184-b6e5-fbffa478cc0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:14:17.042989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b55f3b-0ba1-4184-b6e5-fbffa478cc0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:17.044917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b55f3b-0ba1-4184-b6e5-fbffa478cc0a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:17.072351 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3426/6794] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:14:17.077111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d23089-efdb-4ae3-8875-c3e25772a412 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:17.093484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d23089-efdb-4ae3-8875-c3e25772a412 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:17.102165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbde0d1-dcd0-454e-a54e-b97364fa9056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:17.103694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbde0d1-dcd0-454e-a54e-b97364fa9056 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:17.104151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbde0d1-dcd0-454e-a54e-b97364fa9056 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:17.104568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbde0d1-dcd0-454e-a54e-b97364fa9056 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:17.106418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbde0d1-dcd0-454e-a54e-b97364fa9056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:17.110130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbde0d1-dcd0-454e-a54e-b97364fa9056 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:17.152077 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3369/6795] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:17 2022] POST /identity/v3/projects => generated 397 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:17.159499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b55f3b-0ba1-4184-b6e5-fbffa478cc0a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:17.175502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b55f3b-0ba1-4184-b6e5-fbffa478cc0a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:17.283502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d914d93-c2c5-4425-a400-dff5a204e96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:17.284063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d914d93-c2c5-4425-a400-dff5a204e96b admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:17.284341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d914d93-c2c5-4425-a400-dff5a204e96b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:17.284630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d914d93-c2c5-4425-a400-dff5a204e96b admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:17.285269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d914d93-c2c5-4425-a400-dff5a204e96b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:17.287224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d914d93-c2c5-4425-a400-dff5a204e96b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:17.321063 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3427/6796] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:17 2022] POST /identity/v3/users => generated 299 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:17.322798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbde0d1-dcd0-454e-a54e-b97364fa9056 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:17.340006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbde0d1-dcd0-454e-a54e-b97364fa9056 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:17.348753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b34f9-0bb3-49eb-920c-275d19c8988e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:17.348753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b34f9-0bb3-49eb-920c-275d19c8988e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:17.349361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b34f9-0bb3-49eb-920c-275d19c8988e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:17.349566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b34f9-0bb3-49eb-920c-275d19c8988e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:17.352955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b34f9-0bb3-49eb-920c-275d19c8988e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:17.355271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b34f9-0bb3-49eb-920c-275d19c8988e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:17.369453 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3370/6797] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:17.373621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d914d93-c2c5-4425-a400-dff5a204e96b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:17.387098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d914d93-c2c5-4425-a400-dff5a204e96b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:17.397566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1a1c2-2d37-4065-9012-ae49a1010155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:17.398154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1a1c2-2d37-4065-9012-ae49a1010155 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:17.399094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1a1c2-2d37-4065-9012-ae49a1010155 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:17.399507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1a1c2-2d37-4065-9012-ae49a1010155 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:17.400228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a1a1c2-2d37-4065-9012-ae49a1010155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:17.401960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a1a1c2-2d37-4065-9012-ae49a1010155 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:17.420024 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3428/6798] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:17.421135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b34f9-0bb3-49eb-920c-275d19c8988e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:17.433221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b34f9-0bb3-49eb-920c-275d19c8988e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:17.439841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d37b9-8bcd-4bdd-8d0b-923dc2632784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:17.440674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d37b9-8bcd-4bdd-8d0b-923dc2632784 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:17.441277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d37b9-8bcd-4bdd-8d0b-923dc2632784 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:17.441555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d37b9-8bcd-4bdd-8d0b-923dc2632784 admin admin] PATH_INFO: `/v3/projects/9124bfad91f54d86843f0e33d1461054/users/d0aae3f2844c4e17a11cb54c0aadeb6b/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:17.446975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:17.485625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d37b9-8bcd-4bdd-8d0b-923dc2632784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9124bfad91f54d86843f0e33d1461054, user_id=d0aae3f2844c4e17a11cb54c0aadeb6b, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:17.488349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d37b9-8bcd-4bdd-8d0b-923dc2632784 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:17.509454 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3371/6799] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:17 2022] PUT /identity/v3/projects/9124bfad91f54d86843f0e33d1461054/users/d0aae3f2844c4e17a11cb54c0aadeb6b/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:17.520880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c0cf3-0255-4aa3-b291-70ce936bcb97 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:17.521161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c0cf3-0255-4aa3-b291-70ce936bcb97 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:17.521437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c0cf3-0255-4aa3-b291-70ce936bcb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:17.577813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3e1c0cf3-0255-4aa3-b291-70ce936bcb97 None None] MFA Rules not processed for user `d0aae3f2844c4e17a11cb54c0aadeb6b`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:17.579091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3e1c0cf3-0255-4aa3-b291-70ce936bcb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:17.726070 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3429/6800] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:17 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:17.731440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d37b9-8bcd-4bdd-8d0b-923dc2632784 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:17.829870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d37b9-8bcd-4bdd-8d0b-923dc2632784 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:17.837282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45d10e-6c33-46be-adea-ec58fd0f9f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:17.839012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45d10e-6c33-46be-adea-ec58fd0f9f3d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:17.839985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45d10e-6c33-46be-adea-ec58fd0f9f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:17.839985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45d10e-6c33-46be-adea-ec58fd0f9f3d admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:17.840753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45d10e-6c33-46be-adea-ec58fd0f9f3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:17.843028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45d10e-6c33-46be-adea-ec58fd0f9f3d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:17.877126 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3372/6801] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:17 2022] POST /identity/v3/projects => generated 397 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:17.880670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c0cf3-0255-4aa3-b291-70ce936bcb97 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:17.888396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:17.895306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c0cf3-0255-4aa3-b291-70ce936bcb97 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:17.968246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a1659-e3c9-4823-a38c-d27bd6b0cd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:17.969179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a1659-e3c9-4823-a38c-d27bd6b0cd28 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:17.969380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a1659-e3c9-4823-a38c-d27bd6b0cd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:17.969643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a1659-e3c9-4823-a38c-d27bd6b0cd28 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:17.972297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a1659-e3c9-4823-a38c-d27bd6b0cd28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:17.980012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a1659-e3c9-4823-a38c-d27bd6b0cd28 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:18.019549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45d10e-6c33-46be-adea-ec58fd0f9f3d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:18.030759 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3430/6802] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:17 2022] POST /identity/v3/users => generated 305 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:18.038697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45d10e-6c33-46be-adea-ec58fd0f9f3d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:18.045864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec9e57-1c4e-494b-9706-cbe599dc012e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:18.046684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec9e57-1c4e-494b-9706-cbe599dc012e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:18.047674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec9e57-1c4e-494b-9706-cbe599dc012e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:18.047958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec9e57-1c4e-494b-9706-cbe599dc012e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:18.049946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec9e57-1c4e-494b-9706-cbe599dc012e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:18.051816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec9e57-1c4e-494b-9706-cbe599dc012e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:18.064259 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3373/6803] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:18.069167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a1659-e3c9-4823-a38c-d27bd6b0cd28 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:18.094749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a1659-e3c9-4823-a38c-d27bd6b0cd28 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:18.095539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92efbb-882b-486b-9d42-d1db4b20bd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:18.096448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92efbb-882b-486b-9d42-d1db4b20bd7f admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:18.096726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92efbb-882b-486b-9d42-d1db4b20bd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:18.096994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92efbb-882b-486b-9d42-d1db4b20bd7f admin admin] PATH_INFO: `/v3/projects/0cc3c153553d44678c7ec04423356cce/users/726fd11410744562a954d3424a4d1358/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:18.101556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ef397-7510-46e0-9f8e-069e8d414871 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:18.101977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ef397-7510-46e0-9f8e-069e8d414871 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:18.102359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ef397-7510-46e0-9f8e-069e8d414871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:18.115204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92efbb-882b-486b-9d42-d1db4b20bd7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cc3c153553d44678c7ec04423356cce, user_id=726fd11410744562a954d3424a4d1358, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:18.119085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92efbb-882b-486b-9d42-d1db4b20bd7f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:18.125658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ec3ef397-7510-46e0-9f8e-069e8d414871 None None] MFA Rules not processed for user `f77486ef55a0469a99c3795b626bee1a`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:18.127221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ec3ef397-7510-46e0-9f8e-069e8d414871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:18.143559 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3431/6804] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:18 2022] PUT /identity/v3/projects/0cc3c153553d44678c7ec04423356cce/users/726fd11410744562a954d3424a4d1358/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:18.144905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92efbb-882b-486b-9d42-d1db4b20bd7f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:18.202158 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3374/6805] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:18.220382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92efbb-882b-486b-9d42-d1db4b20bd7f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:18.228816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59fbd2-821f-4a2a-8c17-1414875040b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:18.229720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59fbd2-821f-4a2a-8c17-1414875040b4 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:18.230992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59fbd2-821f-4a2a-8c17-1414875040b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:18.231440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59fbd2-821f-4a2a-8c17-1414875040b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:18.232446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59fbd2-821f-4a2a-8c17-1414875040b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:18.236573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59fbd2-821f-4a2a-8c17-1414875040b4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:18.262590 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3432/6806] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:18.265078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ef397-7510-46e0-9f8e-069e8d414871 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:18.281594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ef397-7510-46e0-9f8e-069e8d414871 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:18.290037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d79294-5d69-4b64-ae2d-2e57eb68add5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:18.290802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d79294-5d69-4b64-ae2d-2e57eb68add5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:18.291230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d79294-5d69-4b64-ae2d-2e57eb68add5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:18.291513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d79294-5d69-4b64-ae2d-2e57eb68add5 admin admin] PATH_INFO: `/v3/projects/0cc3c153553d44678c7ec04423356cce/users/726fd11410744562a954d3424a4d1358/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:18.294248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d79294-5d69-4b64-ae2d-2e57eb68add5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cc3c153553d44678c7ec04423356cce, user_id=726fd11410744562a954d3424a4d1358, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:18.297452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d79294-5d69-4b64-ae2d-2e57eb68add5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:18.314646 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3375/6807] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:18 2022] PUT /identity/v3/projects/0cc3c153553d44678c7ec04423356cce/users/726fd11410744562a954d3424a4d1358/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:18.325360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959ffc5-d0d5-421b-b96c-00067495f76c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:18.325650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959ffc5-d0d5-421b-b96c-00067495f76c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:18.325923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959ffc5-d0d5-421b-b96c-00067495f76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:18.363670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3959ffc5-d0d5-421b-b96c-00067495f76c None None] MFA Rules not processed for user `726fd11410744562a954d3424a4d1358`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:18.365173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3959ffc5-d0d5-421b-b96c-00067495f76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:18.430738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:18.455943 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3433/6808] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:18 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:18.466538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d79294-5d69-4b64-ae2d-2e57eb68add5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:18.468632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b0d79294-5d69-4b64-ae2d-2e57eb68add5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:18.554388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d79294-5d69-4b64-ae2d-2e57eb68add5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:18.558463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e77f1-9bab-46dd-b8f8-95979f8bce82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:18.559124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e77f1-9bab-46dd-b8f8-95979f8bce82 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:18.559406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e77f1-9bab-46dd-b8f8-95979f8bce82 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:18.559838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e77f1-9bab-46dd-b8f8-95979f8bce82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:18.564947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:18.640480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e77f1-9bab-46dd-b8f8-95979f8bce82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:18.642611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e77f1-9bab-46dd-b8f8-95979f8bce82 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:18.655630 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3376/6809] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:18 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:19.223334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959ffc5-d0d5-421b-b96c-00067495f76c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:19.225400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3959ffc5-d0d5-421b-b96c-00067495f76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:19.326628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959ffc5-d0d5-421b-b96c-00067495f76c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:19.333146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13440d-e38f-4aef-abb0-5155a634c374 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:19.333859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13440d-e38f-4aef-abb0-5155a634c374 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:19.334419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13440d-e38f-4aef-abb0-5155a634c374 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:19.334785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13440d-e38f-4aef-abb0-5155a634c374 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:19.345361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13440d-e38f-4aef-abb0-5155a634c374 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:19.347107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13440d-e38f-4aef-abb0-5155a634c374 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:19.359100 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3434/6810] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:19 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:21.513661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e77f1-9bab-46dd-b8f8-95979f8bce82 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:21.515178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-211e77f1-9bab-46dd-b8f8-95979f8bce82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:21.550043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e77f1-9bab-46dd-b8f8-95979f8bce82 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:21.555755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d5414-11d5-47b6-b569-a876fdd8456a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:21.556393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d5414-11d5-47b6-b569-a876fdd8456a service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:21.556673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d5414-11d5-47b6-b569-a876fdd8456a service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:21.556942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d5414-11d5-47b6-b569-a876fdd8456a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:21.559689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-014d5414-11d5-47b6-b569-a876fdd8456a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:21.572647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d5414-11d5-47b6-b569-a876fdd8456a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:21.574150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d5414-11d5-47b6-b569-a876fdd8456a service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:21.586907 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3377/6811] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:25.085071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13440d-e38f-4aef-abb0-5155a634c374 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:25.148284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13440d-e38f-4aef-abb0-5155a634c374 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:25.152952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477e074-b597-4b7d-901f-f34e97675461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:25.153575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477e074-b597-4b7d-901f-f34e97675461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:25.153858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477e074-b597-4b7d-901f-f34e97675461 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:25.154133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477e074-b597-4b7d-901f-f34e97675461 admin admin] PATH_INFO: `/v3/users/d0aae3f2844c4e17a11cb54c0aadeb6b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:25.155288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6477e074-b597-4b7d-901f-f34e97675461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0aae3f2844c4e17a11cb54c0aadeb6b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:25.157201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6477e074-b597-4b7d-901f-f34e97675461 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:25.205014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6477e074-b597-4b7d-901f-f34e97675461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0aae3f2844c4e17a11cb54c0aadeb6b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:25.211950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6477e074-b597-4b7d-901f-f34e97675461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0aae3f2844c4e17a11cb54c0aadeb6b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:25.230201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6477e074-b597-4b7d-901f-f34e97675461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0aae3f2844c4e17a11cb54c0aadeb6b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:25.231089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6477e074-b597-4b7d-901f-f34e97675461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0aae3f2844c4e17a11cb54c0aadeb6b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:25.233729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:25.240824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6477e074-b597-4b7d-901f-f34e97675461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0aae3f2844c4e17a11cb54c0aadeb6b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:25.249305 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3435/6812] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:25 2022] DELETE /identity/v3/users/d0aae3f2844c4e17a11cb54c0aadeb6b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:25.253897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507d41e-36b9-4aeb-ac16-3fb99589ec99 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:25.254380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507d41e-36b9-4aeb-ac16-3fb99589ec99 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:25.254467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507d41e-36b9-4aeb-ac16-3fb99589ec99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:25.273490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c507d41e-36b9-4aeb-ac16-3fb99589ec99 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:25.274730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c507d41e-36b9-4aeb-ac16-3fb99589ec99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:25.349692 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3378/6813] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:25.359868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477e074-b597-4b7d-901f-f34e97675461 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:25.361388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6477e074-b597-4b7d-901f-f34e97675461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:25.430821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477e074-b597-4b7d-901f-f34e97675461 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:25.435165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7562ec6-7992-4d50-bbf3-36faf3281633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:25.435777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7562ec6-7992-4d50-bbf3-36faf3281633 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:25.436058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7562ec6-7992-4d50-bbf3-36faf3281633 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:25.436331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7562ec6-7992-4d50-bbf3-36faf3281633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:25.446715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7562ec6-7992-4d50-bbf3-36faf3281633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:25.448266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7562ec6-7992-4d50-bbf3-36faf3281633 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:25.459228 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3436/6814] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:25.659300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507d41e-36b9-4aeb-ac16-3fb99589ec99 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:25.660770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c507d41e-36b9-4aeb-ac16-3fb99589ec99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:25.683660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507d41e-36b9-4aeb-ac16-3fb99589ec99 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:25.688949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e545d-05ac-4fb0-bf0b-34d6efb992d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:25.690081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e545d-05ac-4fb0-bf0b-34d6efb992d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:25.690081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e545d-05ac-4fb0-bf0b-34d6efb992d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:25.690286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e545d-05ac-4fb0-bf0b-34d6efb992d4 admin admin] PATH_INFO: `/v3/projects/9124bfad91f54d86843f0e33d1461054` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:25.691317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e545d-05ac-4fb0-bf0b-34d6efb992d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9124bfad91f54d86843f0e33d1461054)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:25.693304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e545d-05ac-4fb0-bf0b-34d6efb992d4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:25.775631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c24e545d-05ac-4fb0-bf0b-34d6efb992d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9124bfad91f54d86843f0e33d1461054', 'request_id': 'req-c24e545d-05ac-4fb0-bf0b-34d6efb992d4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:25.786009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c24e545d-05ac-4fb0-bf0b-34d6efb992d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9124bfad91f54d86843f0e33d1461054', 'request_id': 'req-c24e545d-05ac-4fb0-bf0b-34d6efb992d4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:25.792632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c24e545d-05ac-4fb0-bf0b-34d6efb992d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9124bfad91f54d86843f0e33d1461054', 'request_id': 'req-c24e545d-05ac-4fb0-bf0b-34d6efb992d4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:25.806766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c24e545d-05ac-4fb0-bf0b-34d6efb992d4 admin admin] The token cache is being invalidate because project 9124bfad91f54d86843f0e33d1461054 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:25.807061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c24e545d-05ac-4fb0-bf0b-34d6efb992d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:25.807726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c24e545d-05ac-4fb0-bf0b-34d6efb992d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:25.809423 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3379/6815] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:25 2022] DELETE /identity/v3/projects/9124bfad91f54d86843f0e33d1461054 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:25.813258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7562ec6-7992-4d50-bbf3-36faf3281633 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:25.825861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7562ec6-7992-4d50-bbf3-36faf3281633 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:25.916875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc605d3-03bd-4587-86cf-119b82aa7911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:25.917656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc605d3-03bd-4587-86cf-119b82aa7911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:25.917939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc605d3-03bd-4587-86cf-119b82aa7911 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:25.918256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc605d3-03bd-4587-86cf-119b82aa7911 admin admin] PATH_INFO: `/v3/users/726fd11410744562a954d3424a4d1358` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:25.919540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc605d3-03bd-4587-86cf-119b82aa7911 admin admin] RBAC: Authorizing `identity:delete_user(user_id=726fd11410744562a954d3424a4d1358)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:25.925217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc605d3-03bd-4587-86cf-119b82aa7911 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:25.994353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7bc605d3-03bd-4587-86cf-119b82aa7911 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '726fd11410744562a954d3424a4d1358'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:26.005869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7bc605d3-03bd-4587-86cf-119b82aa7911 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '726fd11410744562a954d3424a4d1358'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:26.025439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7bc605d3-03bd-4587-86cf-119b82aa7911 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '726fd11410744562a954d3424a4d1358'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:26.026323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7bc605d3-03bd-4587-86cf-119b82aa7911 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '726fd11410744562a954d3424a4d1358'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:26.038134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7bc605d3-03bd-4587-86cf-119b82aa7911 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '726fd11410744562a954d3424a4d1358'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:26.055758 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3437/6816] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:25 2022] DELETE /identity/v3/users/726fd11410744562a954d3424a4d1358 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:26.258873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e545d-05ac-4fb0-bf0b-34d6efb992d4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:26.260323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c24e545d-05ac-4fb0-bf0b-34d6efb992d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:26.279364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e545d-05ac-4fb0-bf0b-34d6efb992d4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:26.293204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:26.354021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3a97e-db24-4ef2-8020-ac145768cc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:26.354791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3a97e-db24-4ef2-8020-ac145768cc4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:26.355168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3a97e-db24-4ef2-8020-ac145768cc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:26.356096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3a97e-db24-4ef2-8020-ac145768cc4f admin admin] PATH_INFO: `/v3/projects/0cc3c153553d44678c7ec04423356cce` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:26.358170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3a97e-db24-4ef2-8020-ac145768cc4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cc3c153553d44678c7ec04423356cce)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:26.361798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3a97e-db24-4ef2-8020-ac145768cc4f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:26.429603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-02e3a97e-db24-4ef2-8020-ac145768cc4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cc3c153553d44678c7ec04423356cce', 'request_id': 'req-02e3a97e-db24-4ef2-8020-ac145768cc4f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:26.438737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-02e3a97e-db24-4ef2-8020-ac145768cc4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cc3c153553d44678c7ec04423356cce', 'request_id': 'req-02e3a97e-db24-4ef2-8020-ac145768cc4f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:26.448216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-02e3a97e-db24-4ef2-8020-ac145768cc4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cc3c153553d44678c7ec04423356cce', 'request_id': 'req-02e3a97e-db24-4ef2-8020-ac145768cc4f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:26.465641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-02e3a97e-db24-4ef2-8020-ac145768cc4f admin admin] The token cache is being invalidate because project 0cc3c153553d44678c7ec04423356cce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:26.465641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-02e3a97e-db24-4ef2-8020-ac145768cc4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:26.466254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-02e3a97e-db24-4ef2-8020-ac145768cc4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:26.468055 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3380/6817] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:26 2022] DELETE /identity/v3/projects/0cc3c153553d44678c7ec04423356cce => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:26.473085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135ed94-243d-4b0c-b076-199981ff9845 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:26.473572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135ed94-243d-4b0c-b076-199981ff9845 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:26.473572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135ed94-243d-4b0c-b076-199981ff9845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:26.505545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3135ed94-243d-4b0c-b076-199981ff9845 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:26.506918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3135ed94-243d-4b0c-b076-199981ff9845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:26.519002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:26.592435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67777b7b-e902-490f-9592-691a03ec0f58 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:26.593119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67777b7b-e902-490f-9592-691a03ec0f58 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:26.594008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67777b7b-e902-490f-9592-691a03ec0f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:26.594633 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3438/6818] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:26.603120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527485c-af43-4836-afdc-201398692a15 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:26.603413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527485c-af43-4836-afdc-201398692a15 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:26.603704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527485c-af43-4836-afdc-201398692a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:26.626750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-67777b7b-e902-490f-9592-691a03ec0f58 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:26.628494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-67777b7b-e902-490f-9592-691a03ec0f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:26.628618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f527485c-af43-4836-afdc-201398692a15 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:26.629948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f527485c-af43-4836-afdc-201398692a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:26.642542 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3439/6819] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:26.645372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527485c-af43-4836-afdc-201398692a15 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:26.657473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527485c-af43-4836-afdc-201398692a15 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:26.666845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042be97-1780-4c42-958b-945fde63f172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:26.668482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042be97-1780-4c42-958b-945fde63f172 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:26.668898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042be97-1780-4c42-958b-945fde63f172 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:26.669273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042be97-1780-4c42-958b-945fde63f172 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:26.670053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042be97-1780-4c42-958b-945fde63f172 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:14:26.670354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042be97-1780-4c42-958b-945fde63f172 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:26.674235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042be97-1780-4c42-958b-945fde63f172 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:26.685225 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3440/6820] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:14:26.689235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042be97-1780-4c42-958b-945fde63f172 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:26.706449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042be97-1780-4c42-958b-945fde63f172 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:26.707472 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3381/6821] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:14:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:26.714160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363db838-8cb6-41b8-9174-2e92a11eb804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:26.714905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363db838-8cb6-41b8-9174-2e92a11eb804 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:26.715254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363db838-8cb6-41b8-9174-2e92a11eb804 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:26.715618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363db838-8cb6-41b8-9174-2e92a11eb804 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:26.716585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363db838-8cb6-41b8-9174-2e92a11eb804 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:26.718811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363db838-8cb6-41b8-9174-2e92a11eb804 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:26.726075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67777b7b-e902-490f-9592-691a03ec0f58 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:26.728608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-67777b7b-e902-490f-9592-691a03ec0f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:26.753698 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3441/6822] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:26 2022] POST /identity/v3/projects => generated 389 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:26.757541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363db838-8cb6-41b8-9174-2e92a11eb804 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:26.800894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:26.835357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67777b7b-e902-490f-9592-691a03ec0f58 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:26.842725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363db838-8cb6-41b8-9174-2e92a11eb804 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:26.847819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262c33e-cfa0-4fac-96f0-2df9e1ed27ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:26.848432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262c33e-cfa0-4fac-96f0-2df9e1ed27ce admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:26.848711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262c33e-cfa0-4fac-96f0-2df9e1ed27ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:26.849015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262c33e-cfa0-4fac-96f0-2df9e1ed27ce admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:26.849646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262c33e-cfa0-4fac-96f0-2df9e1ed27ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:26.851836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262c33e-cfa0-4fac-96f0-2df9e1ed27ce admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:26.876528 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3442/6823] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:26 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:14:26.877484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262c33e-cfa0-4fac-96f0-2df9e1ed27ce admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:26.879420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d262c33e-cfa0-4fac-96f0-2df9e1ed27ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:26.924431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215fe052-f3e3-43b9-8987-755a5acb7e29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:26.925206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fe052-f3e3-43b9-8987-755a5acb7e29 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:26.925595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fe052-f3e3-43b9-8987-755a5acb7e29 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:26.925975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fe052-f3e3-43b9-8987-755a5acb7e29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:26.951119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262c33e-cfa0-4fac-96f0-2df9e1ed27ce admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:26.956221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3180569-2193-4e08-baee-6fce324d33ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:26.956979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3180569-2193-4e08-baee-6fce324d33ac service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:26.957275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3180569-2193-4e08-baee-6fce324d33ac service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:26.958195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3180569-2193-4e08-baee-6fce324d33ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:26.960171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f3180569-2193-4e08-baee-6fce324d33ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:26.977666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3180569-2193-4e08-baee-6fce324d33ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:26.978942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3180569-2193-4e08-baee-6fce324d33ac service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:26.994290 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3443/6824] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:26.995367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3180569-2193-4e08-baee-6fce324d33ac service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:27.010066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3180569-2193-4e08-baee-6fce324d33ac service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:27.015206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001b654-46c4-4940-869b-f917e7185871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:27.015370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215fe052-f3e3-43b9-8987-755a5acb7e29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:27.015845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001b654-46c4-4940-869b-f917e7185871 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:27.016129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001b654-46c4-4940-869b-f917e7185871 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:27.016400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001b654-46c4-4940-869b-f917e7185871 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:27.016939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215fe052-f3e3-43b9-8987-755a5acb7e29 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:27.017048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001b654-46c4-4940-869b-f917e7185871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:27.018683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001b654-46c4-4940-869b-f917e7185871 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:27.020494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:27.057237 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3444/6825] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:27.061935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001b654-46c4-4940-869b-f917e7185871 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:27.065286 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3382/6826] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:14:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:14:27.074622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001b654-46c4-4940-869b-f917e7185871 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:27.084088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ab311-c9fa-4d92-9b06-aa691be0792e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:27.085424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ab311-c9fa-4d92-9b06-aa691be0792e admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:27.085795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ab311-c9fa-4d92-9b06-aa691be0792e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:27.086094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ab311-c9fa-4d92-9b06-aa691be0792e admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:27.086863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ab311-c9fa-4d92-9b06-aa691be0792e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:27.089016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ab311-c9fa-4d92-9b06-aa691be0792e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:27.129991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215fe052-f3e3-43b9-8987-755a5acb7e29 service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:27.137541 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3445/6827] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:14:27.145033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215fe052-f3e3-43b9-8987-755a5acb7e29 service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:27.150060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02dfbc6-7298-4b43-8401-a6e5c938811d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:27.153146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02dfbc6-7298-4b43-8401-a6e5c938811d admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:27.153591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02dfbc6-7298-4b43-8401-a6e5c938811d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:27.154193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02dfbc6-7298-4b43-8401-a6e5c938811d admin admin] PATH_INFO: `/v3/projects/bd52ba49c73c4538ac7e9e790e817025/users/d543ecdd580f4cf38a8b1e9abf8a0a4e/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:27.197964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02dfbc6-7298-4b43-8401-a6e5c938811d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd52ba49c73c4538ac7e9e790e817025, user_id=d543ecdd580f4cf38a8b1e9abf8a0a4e, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:27.205967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02dfbc6-7298-4b43-8401-a6e5c938811d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:27.209816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ab311-c9fa-4d92-9b06-aa691be0792e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:27.211865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-de7ab311-c9fa-4d92-9b06-aa691be0792e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:27.232854 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3383/6828] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:27 2022] PUT /identity/v3/projects/bd52ba49c73c4538ac7e9e790e817025/users/d543ecdd580f4cf38a8b1e9abf8a0a4e/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:27.254597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914fd2d-bef0-4cfe-9233-390a7f0cd37a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:27.254890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914fd2d-bef0-4cfe-9233-390a7f0cd37a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:27.257349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914fd2d-bef0-4cfe-9233-390a7f0cd37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:27.314596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2914fd2d-bef0-4cfe-9233-390a7f0cd37a None None] MFA Rules not processed for user `d543ecdd580f4cf38a8b1e9abf8a0a4e`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:27.316618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2914fd2d-bef0-4cfe-9233-390a7f0cd37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:27.429034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ab311-c9fa-4d92-9b06-aa691be0792e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:27.481994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:27.494799 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3384/6829] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:27.503501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914fd2d-bef0-4cfe-9233-390a7f0cd37a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:27.594800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914fd2d-bef0-4cfe-9233-390a7f0cd37a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:27.600565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcd9cb-8e45-4870-82b6-b0cd73e67b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:27.601148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcd9cb-8e45-4870-82b6-b0cd73e67b4b service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:27.601534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcd9cb-8e45-4870-82b6-b0cd73e67b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:27.601829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcd9cb-8e45-4870-82b6-b0cd73e67b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:27.636853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eec5c3-6b5e-41a3-9f06-c6dbfaf0020c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:27.637597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eec5c3-6b5e-41a3-9f06-c6dbfaf0020c service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:27.637869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eec5c3-6b5e-41a3-9f06-c6dbfaf0020c service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:27.638148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eec5c3-6b5e-41a3-9f06-c6dbfaf0020c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:27.640139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcd9cb-8e45-4870-82b6-b0cd73e67b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:27.644899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcd9cb-8e45-4870-82b6-b0cd73e67b4b service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:27.648263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-75eec5c3-6b5e-41a3-9f06-c6dbfaf0020c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:27.691293 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3385/6830] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:27 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:27.703899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eec5c3-6b5e-41a3-9f06-c6dbfaf0020c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:27.706034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eec5c3-6b5e-41a3-9f06-c6dbfaf0020c service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:27.726405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcd9cb-8e45-4870-82b6-b0cd73e67b4b service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:27.745568 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3446/6831] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:14:27.841212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcd9cb-8e45-4870-82b6-b0cd73e67b4b service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:27.847260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b49f9-fd65-456e-b9e5-2f9390004250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:27.848873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b49f9-fd65-456e-b9e5-2f9390004250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:27.849521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b49f9-fd65-456e-b9e5-2f9390004250 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:27.849800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b49f9-fd65-456e-b9e5-2f9390004250 admin admin] PATH_INFO: `/v3/users/d543ecdd580f4cf38a8b1e9abf8a0a4e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:27.851486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b49f9-fd65-456e-b9e5-2f9390004250 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d543ecdd580f4cf38a8b1e9abf8a0a4e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:27.853648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b49f9-fd65-456e-b9e5-2f9390004250 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:27.937997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-172b49f9-fd65-456e-b9e5-2f9390004250 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd543ecdd580f4cf38a8b1e9abf8a0a4e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:27.938951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-172b49f9-fd65-456e-b9e5-2f9390004250 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd543ecdd580f4cf38a8b1e9abf8a0a4e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:27.939561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-172b49f9-fd65-456e-b9e5-2f9390004250 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd543ecdd580f4cf38a8b1e9abf8a0a4e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:27.953450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-172b49f9-fd65-456e-b9e5-2f9390004250 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd543ecdd580f4cf38a8b1e9abf8a0a4e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:27.983301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-172b49f9-fd65-456e-b9e5-2f9390004250 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd543ecdd580f4cf38a8b1e9abf8a0a4e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:28.022114 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3386/6832] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:27 2022] DELETE /identity/v3/users/d543ecdd580f4cf38a8b1e9abf8a0a4e => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:28.023835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0c0e7-ac7a-4909-af75-fb0e9d51c144 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:28.024580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0c0e7-ac7a-4909-af75-fb0e9d51c144 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:28.024859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0c0e7-ac7a-4909-af75-fb0e9d51c144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:28.059567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c2e0c0e7-ac7a-4909-af75-fb0e9d51c144 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:28.061405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c2e0c0e7-ac7a-4909-af75-fb0e9d51c144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:28.271520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b49f9-fd65-456e-b9e5-2f9390004250 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:28.276567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-172b49f9-fd65-456e-b9e5-2f9390004250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:28.289974 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3447/6833] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:28.301003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0c0e7-ac7a-4909-af75-fb0e9d51c144 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:28.302525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c2e0c0e7-ac7a-4909-af75-fb0e9d51c144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:28.330295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:28.373931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:28.454525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b49f9-fd65-456e-b9e5-2f9390004250 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:28.463115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0c0e7-ac7a-4909-af75-fb0e9d51c144 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:28.467727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f092f-1614-4150-a31f-b80eec7f6e0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:28.468391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f092f-1614-4150-a31f-b80eec7f6e0a service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:28.471220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f092f-1614-4150-a31f-b80eec7f6e0a service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:28.471507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f092f-1614-4150-a31f-b80eec7f6e0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:28.473723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d0c95-e12d-460b-9842-22e44b5d2422 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:28.475600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d0c95-e12d-460b-9842-22e44b5d2422 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:28.477324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d0c95-e12d-460b-9842-22e44b5d2422 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:28.477998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d0c95-e12d-460b-9842-22e44b5d2422 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:28.480102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cd5f092f-1614-4150-a31f-b80eec7f6e0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:28.494836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d0c95-e12d-460b-9842-22e44b5d2422 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:28.496676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d0c95-e12d-460b-9842-22e44b5d2422 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:28.511785 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3448/6834] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:28.664822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f092f-1614-4150-a31f-b80eec7f6e0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:28.669705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f092f-1614-4150-a31f-b80eec7f6e0a service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:28.719317 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3387/6835] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:29.342817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d0c95-e12d-460b-9842-22e44b5d2422 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:29.345190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-368d0c95-e12d-460b-9842-22e44b5d2422 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:29.388793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d0c95-e12d-460b-9842-22e44b5d2422 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:29.436260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0aebc0-8560-4693-b9da-69f762f5b931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:29.437028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aebc0-8560-4693-b9da-69f762f5b931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:29.437318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aebc0-8560-4693-b9da-69f762f5b931 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:29.437600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aebc0-8560-4693-b9da-69f762f5b931 admin admin] PATH_INFO: `/v3/projects/bd52ba49c73c4538ac7e9e790e817025` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:29.439014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0aebc0-8560-4693-b9da-69f762f5b931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd52ba49c73c4538ac7e9e790e817025)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:29.441027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0aebc0-8560-4693-b9da-69f762f5b931 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:29.526812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2e0aebc0-8560-4693-b9da-69f762f5b931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd52ba49c73c4538ac7e9e790e817025', 'request_id': 'req-2e0aebc0-8560-4693-b9da-69f762f5b931'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:29.535672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2e0aebc0-8560-4693-b9da-69f762f5b931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd52ba49c73c4538ac7e9e790e817025', 'request_id': 'req-2e0aebc0-8560-4693-b9da-69f762f5b931'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:29.545155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2e0aebc0-8560-4693-b9da-69f762f5b931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd52ba49c73c4538ac7e9e790e817025', 'request_id': 'req-2e0aebc0-8560-4693-b9da-69f762f5b931'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:29.576724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2e0aebc0-8560-4693-b9da-69f762f5b931 admin admin] The token cache is being invalidate because project bd52ba49c73c4538ac7e9e790e817025 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:29.577323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2e0aebc0-8560-4693-b9da-69f762f5b931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:29.578148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2e0aebc0-8560-4693-b9da-69f762f5b931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:29.580822 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3449/6836] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:29 2022] DELETE /identity/v3/projects/bd52ba49c73c4538ac7e9e790e817025 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:29.586004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d81245-44a3-4799-b23e-5ee4a53569ff None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:29.586856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d81245-44a3-4799-b23e-5ee4a53569ff None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:29.587310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d81245-44a3-4799-b23e-5ee4a53569ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:29.635921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-84d81245-44a3-4799-b23e-5ee4a53569ff None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:29.638146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-84d81245-44a3-4799-b23e-5ee4a53569ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:29.736059 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3388/6837] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:29.743836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d28a3-4b3d-4086-af8f-de4cf108a7af None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:29.744415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d28a3-4b3d-4086-af8f-de4cf108a7af None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:29.744700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d28a3-4b3d-4086-af8f-de4cf108a7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:29.773110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1d8d28a3-4b3d-4086-af8f-de4cf108a7af None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:29.774754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1d8d28a3-4b3d-4086-af8f-de4cf108a7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:29.797212 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3450/6838] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:29.798972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d81245-44a3-4799-b23e-5ee4a53569ff None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:29.818480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d81245-44a3-4799-b23e-5ee4a53569ff None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:29.828929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15f0e0-3c76-41fc-ae0e-dff164f815eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:29.829858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15f0e0-3c76-41fc-ae0e-dff164f815eb admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:29.830308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15f0e0-3c76-41fc-ae0e-dff164f815eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:29.830753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15f0e0-3c76-41fc-ae0e-dff164f815eb admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:29.832021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15f0e0-3c76-41fc-ae0e-dff164f815eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:14:29.832427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15f0e0-3c76-41fc-ae0e-dff164f815eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:29.834096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15f0e0-3c76-41fc-ae0e-dff164f815eb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:29.849006 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3389/6839] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:14:29.853561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d28a3-4b3d-4086-af8f-de4cf108a7af None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:29.864345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d28a3-4b3d-4086-af8f-de4cf108a7af None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:29.870108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9550417-63ba-4c10-b575-cd3bb5e674b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:29.871008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9550417-63ba-4c10-b575-cd3bb5e674b8 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:29.871324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9550417-63ba-4c10-b575-cd3bb5e674b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:29.871820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9550417-63ba-4c10-b575-cd3bb5e674b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:29.872716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9550417-63ba-4c10-b575-cd3bb5e674b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:29.874556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9550417-63ba-4c10-b575-cd3bb5e674b8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:29.913514 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3451/6840] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:29 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:29.915935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15f0e0-3c76-41fc-ae0e-dff164f815eb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:29.930507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15f0e0-3c76-41fc-ae0e-dff164f815eb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:30.003020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:30.020740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e516182-150a-4b7d-88d4-20b5f2f95478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:30.021521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e516182-150a-4b7d-88d4-20b5f2f95478 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:30.021864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e516182-150a-4b7d-88d4-20b5f2f95478 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:30.022276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e516182-150a-4b7d-88d4-20b5f2f95478 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:30.024365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e516182-150a-4b7d-88d4-20b5f2f95478 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:30.025998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e516182-150a-4b7d-88d4-20b5f2f95478 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:30.075436 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3390/6841] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:29 2022] POST /identity/v3/users => generated 290 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:30.075667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9550417-63ba-4c10-b575-cd3bb5e674b8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:30.091977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9550417-63ba-4c10-b575-cd3bb5e674b8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:30.106287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7ee3e-798a-43fc-89e5-f61e76028a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:30.106929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7ee3e-798a-43fc-89e5-f61e76028a6d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:30.107206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7ee3e-798a-43fc-89e5-f61e76028a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:30.107472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7ee3e-798a-43fc-89e5-f61e76028a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:30.108120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7ee3e-798a-43fc-89e5-f61e76028a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:30.109705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7ee3e-798a-43fc-89e5-f61e76028a6d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:30.155129 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3452/6842] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:30.158269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e516182-150a-4b7d-88d4-20b5f2f95478 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:30.170945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e516182-150a-4b7d-88d4-20b5f2f95478 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:30.178534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97589c-1a6e-461a-b515-18cc368f0d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:30.178534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97589c-1a6e-461a-b515-18cc368f0d8f admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:30.178534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97589c-1a6e-461a-b515-18cc368f0d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:30.178534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97589c-1a6e-461a-b515-18cc368f0d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:30.194008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97589c-1a6e-461a-b515-18cc368f0d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:30.195854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97589c-1a6e-461a-b515-18cc368f0d8f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:30.225265 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3391/6843] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:30.229984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7ee3e-798a-43fc-89e5-f61e76028a6d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:30.240704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7ee3e-798a-43fc-89e5-f61e76028a6d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:30.258167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ddd2f-8648-413c-b4b4-0e491329d569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:30.258892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ddd2f-8648-413c-b4b4-0e491329d569 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:30.259172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ddd2f-8648-413c-b4b4-0e491329d569 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:30.259443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ddd2f-8648-413c-b4b4-0e491329d569 admin admin] PATH_INFO: `/v3/projects/11c9ba909762485fa02968b280eb986c/users/771b0d8ca48c4a5283a74d2797db57a4/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:30.309519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ddd2f-8648-413c-b4b4-0e491329d569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11c9ba909762485fa02968b280eb986c, user_id=771b0d8ca48c4a5283a74d2797db57a4, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:30.312237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ddd2f-8648-413c-b4b4-0e491329d569 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:30.334266 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3453/6844] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:30 2022] PUT /identity/v3/projects/11c9ba909762485fa02968b280eb986c/users/771b0d8ca48c4a5283a74d2797db57a4/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:30.345446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7450ef-2fe8-4d62-b79b-aed0b2fa68de None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:30.345962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7450ef-2fe8-4d62-b79b-aed0b2fa68de None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:30.346241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7450ef-2fe8-4d62-b79b-aed0b2fa68de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:30.402735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3f7450ef-2fe8-4d62-b79b-aed0b2fa68de None None] MFA Rules not processed for user `771b0d8ca48c4a5283a74d2797db57a4`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:30.403982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3f7450ef-2fe8-4d62-b79b-aed0b2fa68de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:30.555342 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3392/6845] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:30.567698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ddd2f-8648-413c-b4b4-0e491329d569 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:30.569466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-071ddd2f-8648-413c-b4b4-0e491329d569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:30.617967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:30.670589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ddd2f-8648-413c-b4b4-0e491329d569 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:30.674670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261652ef-2fb3-4b1d-bb0b-45f39259c208 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:30.675287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261652ef-2fb3-4b1d-bb0b-45f39259c208 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:30.675566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261652ef-2fb3-4b1d-bb0b-45f39259c208 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:30.675838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261652ef-2fb3-4b1d-bb0b-45f39259c208 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:30.690273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261652ef-2fb3-4b1d-bb0b-45f39259c208 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:30.691898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261652ef-2fb3-4b1d-bb0b-45f39259c208 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:30.702086 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3454/6846] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:31.905025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7450ef-2fe8-4d62-b79b-aed0b2fa68de None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:31.918850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7450ef-2fe8-4d62-b79b-aed0b2fa68de None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:31.925125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fdc0b-eaab-4597-826b-cf2576fe6891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:31.926269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fdc0b-eaab-4597-826b-cf2576fe6891 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:31.926591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fdc0b-eaab-4597-826b-cf2576fe6891 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:31.926867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fdc0b-eaab-4597-826b-cf2576fe6891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:31.928723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fd1fdc0b-eaab-4597-826b-cf2576fe6891 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:32.098822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fdc0b-eaab-4597-826b-cf2576fe6891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:32.100406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fdc0b-eaab-4597-826b-cf2576fe6891 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:32.131335 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3393/6847] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:39.094602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261652ef-2fb3-4b1d-bb0b-45f39259c208 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:39.096168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-261652ef-2fb3-4b1d-bb0b-45f39259c208 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:39.246759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261652ef-2fb3-4b1d-bb0b-45f39259c208 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:39.262976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e8b92-1fa1-4eb5-8165-7471da6c24e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:39.263604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e8b92-1fa1-4eb5-8165-7471da6c24e2 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:39.263880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e8b92-1fa1-4eb5-8165-7471da6c24e2 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:39.264148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e8b92-1fa1-4eb5-8165-7471da6c24e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:39.267845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d02e8b92-1fa1-4eb5-8165-7471da6c24e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:39.302000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e8b92-1fa1-4eb5-8165-7471da6c24e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:39.303960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e8b92-1fa1-4eb5-8165-7471da6c24e2 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:39.329810 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3455/6848] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:40.520648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fdc0b-eaab-4597-826b-cf2576fe6891 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:40.523675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fd1fdc0b-eaab-4597-826b-cf2576fe6891 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:40.568747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fdc0b-eaab-4597-826b-cf2576fe6891 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:40.589520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c27f1-9816-4245-8a90-eb3cff7b6a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:40.590672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c27f1-9816-4245-8a90-eb3cff7b6a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:40.591334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c27f1-9816-4245-8a90-eb3cff7b6a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:40.591801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c27f1-9816-4245-8a90-eb3cff7b6a9a admin admin] PATH_INFO: `/v3/users/8d79dff848bf40fc89377e13736c527b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:40.596516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c27f1-9816-4245-8a90-eb3cff7b6a9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d79dff848bf40fc89377e13736c527b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:40.599092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c27f1-9816-4245-8a90-eb3cff7b6a9a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:40.721325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1c0c27f1-9816-4245-8a90-eb3cff7b6a9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d79dff848bf40fc89377e13736c527b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:40.722257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1c0c27f1-9816-4245-8a90-eb3cff7b6a9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d79dff848bf40fc89377e13736c527b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:40.723040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1c0c27f1-9816-4245-8a90-eb3cff7b6a9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d79dff848bf40fc89377e13736c527b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:40.727567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:40.747073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1c0c27f1-9816-4245-8a90-eb3cff7b6a9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d79dff848bf40fc89377e13736c527b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:40.787085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1c0c27f1-9816-4245-8a90-eb3cff7b6a9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d79dff848bf40fc89377e13736c527b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:40.824603 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3394/6849] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:40 2022] DELETE /identity/v3/users/8d79dff848bf40fc89377e13736c527b => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:41.136297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e8b92-1fa1-4eb5-8165-7471da6c24e2 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:41.151030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e8b92-1fa1-4eb5-8165-7471da6c24e2 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:41.305840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbeb756-fbf3-49d4-bf49-e1db0f119eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:41.306515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbeb756-fbf3-49d4-bf49-e1db0f119eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:41.306799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbeb756-fbf3-49d4-bf49-e1db0f119eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:41.307580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbeb756-fbf3-49d4-bf49-e1db0f119eb4 admin admin] PATH_INFO: `/v3/projects/fa37082733634ec688c9d6272a0d9b1d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:41.309285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbeb756-fbf3-49d4-bf49-e1db0f119eb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa37082733634ec688c9d6272a0d9b1d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:41.311735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbeb756-fbf3-49d4-bf49-e1db0f119eb4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:41.406161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ffbeb756-fbf3-49d4-bf49-e1db0f119eb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa37082733634ec688c9d6272a0d9b1d', 'request_id': 'req-ffbeb756-fbf3-49d4-bf49-e1db0f119eb4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:41.434282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ffbeb756-fbf3-49d4-bf49-e1db0f119eb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa37082733634ec688c9d6272a0d9b1d', 'request_id': 'req-ffbeb756-fbf3-49d4-bf49-e1db0f119eb4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:41.447694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:41.449029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ffbeb756-fbf3-49d4-bf49-e1db0f119eb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa37082733634ec688c9d6272a0d9b1d', 'request_id': 'req-ffbeb756-fbf3-49d4-bf49-e1db0f119eb4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:41.494215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ffbeb756-fbf3-49d4-bf49-e1db0f119eb4 admin admin] The token cache is being invalidate because project fa37082733634ec688c9d6272a0d9b1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:41.496601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ffbeb756-fbf3-49d4-bf49-e1db0f119eb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:41.497439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ffbeb756-fbf3-49d4-bf49-e1db0f119eb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:41.507772 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3456/6850] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:41 2022] DELETE /identity/v3/projects/fa37082733634ec688c9d6272a0d9b1d => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:41.509481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f8970-b909-4222-bca8-ee30ed3d6548 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:41.509792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f8970-b909-4222-bca8-ee30ed3d6548 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:41.510075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f8970-b909-4222-bca8-ee30ed3d6548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:41.537825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2c8f8970-b909-4222-bca8-ee30ed3d6548 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:41.539830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2c8f8970-b909-4222-bca8-ee30ed3d6548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:41.622697 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3395/6851] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:41.624357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e6a54-b685-4d16-871e-64a9efc71c57 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:41.624644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e6a54-b685-4d16-871e-64a9efc71c57 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:41.624920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e6a54-b685-4d16-871e-64a9efc71c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:41.656827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5d5e6a54-b685-4d16-871e-64a9efc71c57 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:41.658440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5d5e6a54-b685-4d16-871e-64a9efc71c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:41.671480 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3457/6852] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:41.672921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f8970-b909-4222-bca8-ee30ed3d6548 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:41.700882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f8970-b909-4222-bca8-ee30ed3d6548 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:41.710451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e1e5e-8c98-4a85-91f8-8a314a5587d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:41.711108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e1e5e-8c98-4a85-91f8-8a314a5587d9 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:41.713886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e1e5e-8c98-4a85-91f8-8a314a5587d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:41.714210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e1e5e-8c98-4a85-91f8-8a314a5587d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:41.716771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e1e5e-8c98-4a85-91f8-8a314a5587d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:14:41.717106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e1e5e-8c98-4a85-91f8-8a314a5587d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:41.718942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e1e5e-8c98-4a85-91f8-8a314a5587d9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:41.749947 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3396/6853] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:14:41.751338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e6a54-b685-4d16-871e-64a9efc71c57 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:41.764672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e6a54-b685-4d16-871e-64a9efc71c57 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:41.773607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658817e-eeb4-4208-af9f-ac98e62014c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:41.774249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658817e-eeb4-4208-af9f-ac98e62014c9 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:41.776828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658817e-eeb4-4208-af9f-ac98e62014c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:41.777405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658817e-eeb4-4208-af9f-ac98e62014c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:41.778832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658817e-eeb4-4208-af9f-ac98e62014c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:41.783843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658817e-eeb4-4208-af9f-ac98e62014c9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:41.877748 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3458/6854] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:41 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:14:41.879021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e1e5e-8c98-4a85-91f8-8a314a5587d9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:41.901233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e1e5e-8c98-4a85-91f8-8a314a5587d9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:42.015470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3fd842-067d-4c3e-9df3-79e368444261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:42.016310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fd842-067d-4c3e-9df3-79e368444261 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:42.016768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fd842-067d-4c3e-9df3-79e368444261 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:42.017048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fd842-067d-4c3e-9df3-79e368444261 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:42.017682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3fd842-067d-4c3e-9df3-79e368444261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:42.019700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3fd842-067d-4c3e-9df3-79e368444261 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:42.093757 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3397/6855] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:41 2022] POST /identity/v3/users => generated 292 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:42.104221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658817e-eeb4-4208-af9f-ac98e62014c9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:42.114923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658817e-eeb4-4208-af9f-ac98e62014c9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:42.120574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8787bf-97a6-445b-8982-1c126a51460f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:42.121249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8787bf-97a6-445b-8982-1c126a51460f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:42.121807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8787bf-97a6-445b-8982-1c126a51460f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:42.121938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8787bf-97a6-445b-8982-1c126a51460f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:42.122679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8787bf-97a6-445b-8982-1c126a51460f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:42.125061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8787bf-97a6-445b-8982-1c126a51460f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:42.138536 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3459/6856] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:42.140419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3fd842-067d-4c3e-9df3-79e368444261 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:42.152011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3fd842-067d-4c3e-9df3-79e368444261 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:42.157188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607775fc-38c6-4fe7-893f-74f01f74b4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:42.158009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607775fc-38c6-4fe7-893f-74f01f74b4fd admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:42.158289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607775fc-38c6-4fe7-893f-74f01f74b4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:42.158599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607775fc-38c6-4fe7-893f-74f01f74b4fd admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:42.159276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607775fc-38c6-4fe7-893f-74f01f74b4fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:42.160937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607775fc-38c6-4fe7-893f-74f01f74b4fd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:42.178558 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3398/6857] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:14:42.181477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8787bf-97a6-445b-8982-1c126a51460f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:42.193007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8787bf-97a6-445b-8982-1c126a51460f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:42.198409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ef7d8-865b-4af0-9b68-be85e89a9779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:42.199135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ef7d8-865b-4af0-9b68-be85e89a9779 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:42.199416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ef7d8-865b-4af0-9b68-be85e89a9779 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:42.199694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ef7d8-865b-4af0-9b68-be85e89a9779 admin admin] PATH_INFO: `/v3/projects/96f05f07401c48ada3722a1e5f118696/users/dcea499a458e4ef6a5a1d5dd983af075/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:42.218778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ef7d8-865b-4af0-9b68-be85e89a9779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96f05f07401c48ada3722a1e5f118696, user_id=dcea499a458e4ef6a5a1d5dd983af075, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:42.222158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ef7d8-865b-4af0-9b68-be85e89a9779 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:42.264158 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3460/6858] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:42 2022] PUT /identity/v3/projects/96f05f07401c48ada3722a1e5f118696/users/dcea499a458e4ef6a5a1d5dd983af075/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:42.268512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399ba70-5af3-480e-8588-034cf6c567c9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:42.268949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399ba70-5af3-480e-8588-034cf6c567c9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:42.269092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399ba70-5af3-480e-8588-034cf6c567c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:42.367189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a399ba70-5af3-480e-8588-034cf6c567c9 None None] MFA Rules not processed for user `dcea499a458e4ef6a5a1d5dd983af075`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:42.368449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a399ba70-5af3-480e-8588-034cf6c567c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:42.400043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:42.482999 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3399/6859] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:42.486905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ef7d8-865b-4af0-9b68-be85e89a9779 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:42.565406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ef7d8-865b-4af0-9b68-be85e89a9779 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:42.571651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83cf070-e635-48cd-be90-77ab2c829da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:42.572369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cf070-e635-48cd-be90-77ab2c829da9 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:42.572722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cf070-e635-48cd-be90-77ab2c829da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:42.572951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cf070-e635-48cd-be90-77ab2c829da9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:42.573698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83cf070-e635-48cd-be90-77ab2c829da9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:42.587528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83cf070-e635-48cd-be90-77ab2c829da9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:42.634959 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3461/6860] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:42 2022] POST /identity/v3/projects => generated 383 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:14:42.639174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399ba70-5af3-480e-8588-034cf6c567c9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:42.658870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399ba70-5af3-480e-8588-034cf6c567c9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:42.753115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eee658-ae2f-48ed-be16-b00aa3e2deb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:42.754258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eee658-ae2f-48ed-be16-b00aa3e2deb8 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:42.754718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eee658-ae2f-48ed-be16-b00aa3e2deb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:42.755134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eee658-ae2f-48ed-be16-b00aa3e2deb8 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:42.755906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eee658-ae2f-48ed-be16-b00aa3e2deb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:42.758177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eee658-ae2f-48ed-be16-b00aa3e2deb8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:42.796734 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3400/6861] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:42 2022] POST /identity/v3/users => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:42.802980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83cf070-e635-48cd-be90-77ab2c829da9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:42.816550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83cf070-e635-48cd-be90-77ab2c829da9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:42.823158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd05a4b-b6b8-4a2e-989c-0abad1841117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:42.824257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd05a4b-b6b8-4a2e-989c-0abad1841117 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:42.824617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd05a4b-b6b8-4a2e-989c-0abad1841117 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:42.824988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd05a4b-b6b8-4a2e-989c-0abad1841117 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:42.825893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd05a4b-b6b8-4a2e-989c-0abad1841117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:42.828167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd05a4b-b6b8-4a2e-989c-0abad1841117 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:42.845806 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3462/6862] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:42.849906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eee658-ae2f-48ed-be16-b00aa3e2deb8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:42.863083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eee658-ae2f-48ed-be16-b00aa3e2deb8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:42.876583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd6302-8a8f-4ae6-b736-f54761b7ff86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:42.876583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd6302-8a8f-4ae6-b736-f54761b7ff86 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:42.876583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd6302-8a8f-4ae6-b736-f54761b7ff86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:42.876583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd6302-8a8f-4ae6-b736-f54761b7ff86 admin admin] PATH_INFO: `/v3/projects/03e428202a7e4fe4a29a787284d6470a/users/5fafa83cf4bf40b2a94fc65dbba98f5d/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:42.938545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd6302-8a8f-4ae6-b736-f54761b7ff86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e428202a7e4fe4a29a787284d6470a, user_id=5fafa83cf4bf40b2a94fc65dbba98f5d, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:42.941792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd6302-8a8f-4ae6-b736-f54761b7ff86 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:42.987008 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3401/6863] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:42 2022] PUT /identity/v3/projects/03e428202a7e4fe4a29a787284d6470a/users/5fafa83cf4bf40b2a94fc65dbba98f5d/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:42.988760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd05a4b-b6b8-4a2e-989c-0abad1841117 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:43.003051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd05a4b-b6b8-4a2e-989c-0abad1841117 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:43.124254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2bac2-722f-4cd4-9820-810df9b915c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:43.124887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2bac2-722f-4cd4-9820-810df9b915c1 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:43.125321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2bac2-722f-4cd4-9820-810df9b915c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:43.125594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2bac2-722f-4cd4-9820-810df9b915c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:43.126241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2bac2-722f-4cd4-9820-810df9b915c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:43.128584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2bac2-722f-4cd4-9820-810df9b915c1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:43.131489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:43.151298 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3463/6864] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:14:43.152887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd6302-8a8f-4ae6-b736-f54761b7ff86 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:43.177533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd6302-8a8f-4ae6-b736-f54761b7ff86 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:43.195359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632cf42-b10c-4b7f-9610-50357da67c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:43.195928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632cf42-b10c-4b7f-9610-50357da67c64 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:43.196207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632cf42-b10c-4b7f-9610-50357da67c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:43.196504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632cf42-b10c-4b7f-9610-50357da67c64 admin admin] PATH_INFO: `/v3/projects/03e428202a7e4fe4a29a787284d6470a/users/5fafa83cf4bf40b2a94fc65dbba98f5d/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:43.204009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c632cf42-b10c-4b7f-9610-50357da67c64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e428202a7e4fe4a29a787284d6470a, user_id=5fafa83cf4bf40b2a94fc65dbba98f5d, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:43.216210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c632cf42-b10c-4b7f-9610-50357da67c64 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:43.248045 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3402/6865] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:43 2022] PUT /identity/v3/projects/03e428202a7e4fe4a29a787284d6470a/users/5fafa83cf4bf40b2a94fc65dbba98f5d/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:43.258168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcd045-313a-42e7-98ee-f0f46eb2a53e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:43.258600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcd045-313a-42e7-98ee-f0f46eb2a53e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:43.259127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcd045-313a-42e7-98ee-f0f46eb2a53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:43.322801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-10dcd045-313a-42e7-98ee-f0f46eb2a53e None None] MFA Rules not processed for user `5fafa83cf4bf40b2a94fc65dbba98f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:43.324053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-10dcd045-313a-42e7-98ee-f0f46eb2a53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:43.437507 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3464/6866] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:43.437892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632cf42-b10c-4b7f-9610-50357da67c64 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:43.457603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632cf42-b10c-4b7f-9610-50357da67c64 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:43.472277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fd7f9-8fb4-4322-9b0c-9021e9745059 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fafa83cf4bf40b2a94fc65dbba98f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03e428202a7e4fe4a29a787284d6470a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:43.473004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fd7f9-8fb4-4322-9b0c-9021e9745059 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:43.473295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fd7f9-8fb4-4322-9b0c-9021e9745059 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:43.482625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fd7f9-8fb4-4322-9b0c-9021e9745059 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:43.482625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fd7f9-8fb4-4322-9b0c-9021e9745059 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:43.482625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fd7f9-8fb4-4322-9b0c-9021e9745059 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-d56fd7f9-8fb4-4322-9b0c-9021e9745059 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invalid input for field 'name': '' is too short Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: Failed validating 'minLength' in schema['properties']['name']: Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: On instance['name']: Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: '': keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: Failed validating 'minLength' in schema['properties']['name']: Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: On instance['name']: Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: '' Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise error Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: '' is too short Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application On instance['name']: Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application '' Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:14:43.495456 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:14:43.497043 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:14:43.497043 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:14:43.497043 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:14:43.497043 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:14:43.497043 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:14:43.497043 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:14:43.497043 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:14:43.497043 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:14:43.497043 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:14:43.497043 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:14:43.497043 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:14:43.497043 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:14:43.497043 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 115, in post Oct 01 21:14:43.497043 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application validation.lazy_validate(schema.domain_create, domain) Oct 01 21:14:43.497043 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Oct 01 21:14:43.497043 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Oct 01 21:14:43.497043 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Oct 01 21:14:43.497963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Oct 01 21:14:43.497963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Oct 01 21:14:43.497963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:14:43.497963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Oct 01 21:14:43.497963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 01 21:14:43.497963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:14:43.497963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application On instance['name']: Oct 01 21:14:43.497963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application '' Oct 01 21:14:43.497963 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:14:43.499801 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3403/6867] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:14:43 2022] POST /identity/v3/domains => generated 283 bytes in 63 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Oct 01 21:14:43.516551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dcd045-313a-42e7-98ee-f0f46eb2a53e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:43.532058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dcd045-313a-42e7-98ee-f0f46eb2a53e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:43.553184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b6092-31ac-4d4d-b36c-46fb213d2eb5 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fafa83cf4bf40b2a94fc65dbba98f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03e428202a7e4fe4a29a787284d6470a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:43.553184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b6092-31ac-4d4d-b36c-46fb213d2eb5 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:43.553184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b6092-31ac-4d4d-b36c-46fb213d2eb5 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:43.554396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b6092-31ac-4d4d-b36c-46fb213d2eb5 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:43.554396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b6092-31ac-4d4d-b36c-46fb213d2eb5 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:43.554396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b6092-31ac-4d4d-b36c-46fb213d2eb5 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:43.556578 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-e16b6092-31ac-4d4d-b36c-46fb213d2eb5 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 01 21:14:43.556578 ncdv-master-1 devstack@keystone.service[99139]: Failed validating 'maxLength' in schema['properties']['name']: Oct 01 21:14:43.556578 ncdv-master-1 devstack@keystone.service[99139]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 01 21:14:43.556578 ncdv-master-1 devstack@keystone.service[99139]: On instance['name']: Oct 01 21:14:43.556578 ncdv-master-1 devstack@keystone.service[99139]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 01 21:14:43.556578 ncdv-master-1 devstack@keystone.service[99139]: Failed validating 'maxLength' in schema['properties']['name']: Oct 01 21:14:43.556578 ncdv-master-1 devstack@keystone.service[99139]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 01 21:14:43.556578 ncdv-master-1 devstack@keystone.service[99139]: On instance['name']: Oct 01 21:14:43.556578 ncdv-master-1 devstack@keystone.service[99139]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Oct 01 21:14:43.556578 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:14:43.556578 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Oct 01 21:14:43.556578 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Oct 01 21:14:43.556578 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 01 21:14:43.556578 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise error Oct 01 21:14:43.556578 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 01 21:14:43.556578 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:14:43.556578 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Oct 01 21:14:43.556578 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 01 21:14:43.556578 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:14:43.557876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application On instance['name']: Oct 01 21:14:43.557876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Oct 01 21:14:43.557876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:14:43.557876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:14:43.557876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:14:43.557876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:14:43.557876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:14:43.557876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:14:43.557876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:14:43.557876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:14:43.557876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:14:43.557876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:14:43.557876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:14:43.557876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:14:43.557876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:14:43.557876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:14:43.557876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 115, in post Oct 01 21:14:43.557876 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application validation.lazy_validate(schema.domain_create, domain) Oct 01 21:14:43.559004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Oct 01 21:14:43.559004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Oct 01 21:14:43.559004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Oct 01 21:14:43.559004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Oct 01 21:14:43.559004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 01 21:14:43.559004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:14:43.559004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Oct 01 21:14:43.559004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Oct 01 21:14:43.559004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:14:43.559004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application On instance['name']: Oct 01 21:14:43.559004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Oct 01 21:14:43.559004 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:14:43.559004 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3465/6868] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:14:43 2022] POST /identity/v3/domains => generated 412 bytes in 42 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 01 21:14:43.564716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fd7f9-8fb4-4322-9b0c-9021e9745059 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:43.577164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fd7f9-8fb4-4322-9b0c-9021e9745059 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:43.582170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d50d3-a97d-47aa-a51a-5b185ee70b81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fafa83cf4bf40b2a94fc65dbba98f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03e428202a7e4fe4a29a787284d6470a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:43.582814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d50d3-a97d-47aa-a51a-5b185ee70b81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:43.583098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d50d3-a97d-47aa-a51a-5b185ee70b81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:43.583374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d50d3-a97d-47aa-a51a-5b185ee70b81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:43.584221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d50d3-a97d-47aa-a51a-5b185ee70b81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:43.586807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d50d3-a97d-47aa-a51a-5b185ee70b81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:43.614423 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3404/6869] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:14:43 2022] POST /identity/v3/domains => generated 276 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:43.615848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b6092-31ac-4d4d-b36c-46fb213d2eb5 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:43.631811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b6092-31ac-4d4d-b36c-46fb213d2eb5 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:43.702302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b35b9-f755-43b6-bb51-76620cad1e23 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fafa83cf4bf40b2a94fc65dbba98f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03e428202a7e4fe4a29a787284d6470a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:43.703164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b35b9-f755-43b6-bb51-76620cad1e23 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:43.703581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b35b9-f755-43b6-bb51-76620cad1e23 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:43.704042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b35b9-f755-43b6-bb51-76620cad1e23 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] PATH_INFO: `/v3/domains/89c99c87ad7d40399adcde13c83e5d3f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:43.720280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b35b9-f755-43b6-bb51-76620cad1e23 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=89c99c87ad7d40399adcde13c83e5d3f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:43.722245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b35b9-f755-43b6-bb51-76620cad1e23 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:43.740662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-7d8b35b9-f755-43b6-bb51-76620cad1e23 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Cannot delete a domain that is enabled, please disable it first.: keystone.exception.ForbiddenNotSecurity: Cannot delete a domain that is enabled, please disable it first. Oct 01 21:14:43.740662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:14:43.740662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:14:43.740662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:14:43.740662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:14:43.740662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:14:43.740662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:14:43.740662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:14:43.740662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:14:43.740662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:14:43.740662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:14:43.740662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:14:43.740662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 151, in delete Oct 01 21:14:43.740662 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_domain( Oct 01 21:14:43.741568 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:14:43.741568 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:14:43.741568 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 802, in delete_domain Oct 01 21:14:43.741568 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._delete_domain(domain, initiator) Oct 01 21:14:43.741568 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 815, in _delete_domain Oct 01 21:14:43.741568 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ForbiddenNotSecurity( Oct 01 21:14:43.741568 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ForbiddenNotSecurity: Cannot delete a domain that is enabled, please disable it first. Oct 01 21:14:43.741568 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:14:43.746672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d50d3-a97d-47aa-a51a-5b185ee70b81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:43.753349 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3466/6870] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:14:43 2022] DELETE /identity/v3/domains/89c99c87ad7d40399adcde13c83e5d3f => generated 120 bytes in 137 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:14:43.758842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d50d3-a97d-47aa-a51a-5b185ee70b81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:43.766092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f0ec0-b73f-4da2-8278-ebe217dc5b20 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fafa83cf4bf40b2a94fc65dbba98f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03e428202a7e4fe4a29a787284d6470a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:43.766777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f0ec0-b73f-4da2-8278-ebe217dc5b20 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:43.767064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f0ec0-b73f-4da2-8278-ebe217dc5b20 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:43.767358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f0ec0-b73f-4da2-8278-ebe217dc5b20 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] PATH_INFO: `/v3/domains/89c99c87ad7d40399adcde13c83e5d3f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:43.768816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f0ec0-b73f-4da2-8278-ebe217dc5b20 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=89c99c87ad7d40399adcde13c83e5d3f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:43.770586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f0ec0-b73f-4da2-8278-ebe217dc5b20 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:43.804525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-395f0ec0-b73f-4da2-8278-ebe217dc5b20 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '89c99c87ad7d40399adcde13c83e5d3f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:43.806681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-395f0ec0-b73f-4da2-8278-ebe217dc5b20 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '89c99c87ad7d40399adcde13c83e5d3f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:43.868421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-395f0ec0-b73f-4da2-8278-ebe217dc5b20 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '89c99c87ad7d40399adcde13c83e5d3f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:43.869080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-395f0ec0-b73f-4da2-8278-ebe217dc5b20 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '89c99c87ad7d40399adcde13c83e5d3f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:43.875482 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3405/6871] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:14:43 2022] PATCH /identity/v3/domains/89c99c87ad7d40399adcde13c83e5d3f => generated 277 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:14:43.876846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b35b9-f755-43b6-bb51-76620cad1e23 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:43.879873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7d8b35b9-f755-43b6-bb51-76620cad1e23 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:43.894351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b35b9-f755-43b6-bb51-76620cad1e23 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:43.966098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77711fb8-184b-4206-96f7-be6a18032d81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fafa83cf4bf40b2a94fc65dbba98f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03e428202a7e4fe4a29a787284d6470a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:43.966774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77711fb8-184b-4206-96f7-be6a18032d81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:43.967057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77711fb8-184b-4206-96f7-be6a18032d81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:43.967331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77711fb8-184b-4206-96f7-be6a18032d81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] PATH_INFO: `/v3/domains/89c99c87ad7d40399adcde13c83e5d3f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:43.971032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:43.983462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77711fb8-184b-4206-96f7-be6a18032d81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=89c99c87ad7d40399adcde13c83e5d3f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:43.985446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77711fb8-184b-4206-96f7-be6a18032d81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:44.019869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77711fb8-184b-4206-96f7-be6a18032d81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '89c99c87ad7d40399adcde13c83e5d3f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.024286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77711fb8-184b-4206-96f7-be6a18032d81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '89c99c87ad7d40399adcde13c83e5d3f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.082137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77711fb8-184b-4206-96f7-be6a18032d81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89c99c87ad7d40399adcde13c83e5d3f', 'request_id': 'req-77711fb8-184b-4206-96f7-be6a18032d81'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.089919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77711fb8-184b-4206-96f7-be6a18032d81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89c99c87ad7d40399adcde13c83e5d3f', 'request_id': 'req-77711fb8-184b-4206-96f7-be6a18032d81'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.096810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77711fb8-184b-4206-96f7-be6a18032d81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89c99c87ad7d40399adcde13c83e5d3f', 'request_id': 'req-77711fb8-184b-4206-96f7-be6a18032d81'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.128018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f0ec0-b73f-4da2-8278-ebe217dc5b20 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:44.130055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-395f0ec0-b73f-4da2-8278-ebe217dc5b20 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:44.131364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77711fb8-184b-4206-96f7-be6a18032d81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] The token cache is being invalidate because project 89c99c87ad7d40399adcde13c83e5d3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:44.132627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77711fb8-184b-4206-96f7-be6a18032d81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.133219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77711fb8-184b-4206-96f7-be6a18032d81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.144079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77711fb8-184b-4206-96f7-be6a18032d81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '89c99c87ad7d40399adcde13c83e5d3f', 'request_id': 'req-77711fb8-184b-4206-96f7-be6a18032d81'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.144813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77711fb8-184b-4206-96f7-be6a18032d81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '89c99c87ad7d40399adcde13c83e5d3f', 'request_id': 'req-77711fb8-184b-4206-96f7-be6a18032d81'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.145452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77711fb8-184b-4206-96f7-be6a18032d81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '89c99c87ad7d40399adcde13c83e5d3f', 'request_id': 'req-77711fb8-184b-4206-96f7-be6a18032d81'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.152413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:UPDATE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:44.156571 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3467/6872] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:14:43 2022] DELETE /identity/v3/domains/89c99c87ad7d40399adcde13c83e5d3f => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:44.161658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77711fb8-184b-4206-96f7-be6a18032d81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:44.168453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-77711fb8-184b-4206-96f7-be6a18032d81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:44.237977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f0ec0-b73f-4da2-8278-ebe217dc5b20 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:44.298689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77711fb8-184b-4206-96f7-be6a18032d81 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:44.305404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af790f-f2a6-4345-9248-129f618264a1 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fafa83cf4bf40b2a94fc65dbba98f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03e428202a7e4fe4a29a787284d6470a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:44.306431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af790f-f2a6-4345-9248-129f618264a1 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:44.306906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af790f-f2a6-4345-9248-129f618264a1 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:44.308039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af790f-f2a6-4345-9248-129f618264a1 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] PATH_INFO: `/v3/domains/9222bae300ba4108a4142e8bbe50b596` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:44.317627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af790f-f2a6-4345-9248-129f618264a1 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 21:14:44.323624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb101e-eb1e-416e-b882-9ccbbbe83aa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:44.324421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb101e-eb1e-416e-b882-9ccbbbe83aa5 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:44.324801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb101e-eb1e-416e-b882-9ccbbbe83aa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:44.325142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb101e-eb1e-416e-b882-9ccbbbe83aa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:44.326742 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-55af790f-f2a6-4345-9248-129f618264a1 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Could not find domain: 9222bae300ba4108a4142e8bbe50b596.: keystone.exception.DomainNotFound: Could not find domain: 9222bae300ba4108a4142e8bbe50b596. Oct 01 21:14:44.326742 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:14:44.326742 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 01 21:14:44.326742 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 01 21:14:44.326742 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 21:14:44.326742 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 21:14:44.326742 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 21:14:44.326742 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 21:14:44.326742 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 9222bae300ba4108a4142e8bbe50b596. Oct 01 21:14:44.326742 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:14:44.326742 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:14:44.326742 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:14:44.326742 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:14:44.326742 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:14:44.326742 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:14:44.327847 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:14:44.327847 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:14:44.327847 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:14:44.327847 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:14:44.327847 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:14:44.327847 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:14:44.327847 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:14:44.327847 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:14:44.327847 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 150, in delete Oct 01 21:14:44.327847 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_domain') Oct 01 21:14:44.327847 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 21:14:44.327847 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 21:14:44.327847 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 21:14:44.327847 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 21:14:44.327847 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:14:44.327847 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:14:44.328927 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:14:44.328927 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:14:44.328927 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:14:44.328927 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:14:44.328927 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:14:44.328927 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 21:14:44.328927 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:14:44.328927 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:14:44.328927 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:14:44.328927 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:14:44.328927 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:14:44.328927 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:14:44.328927 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:14:44.328927 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:14:44.328927 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 01 21:14:44.328927 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 21:14:44.328927 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 9222bae300ba4108a4142e8bbe50b596. Oct 01 21:14:44.330064 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:14:44.330436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-18bb101e-eb1e-416e-b882-9ccbbbe83aa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:44.336498 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3468/6873] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:14:44 2022] DELETE /identity/v3/domains/9222bae300ba4108a4142e8bbe50b596 => generated 112 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:14:44.337726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af790f-f2a6-4345-9248-129f618264a1 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:44.346297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb101e-eb1e-416e-b882-9ccbbbe83aa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:44.348638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb101e-eb1e-416e-b882-9ccbbbe83aa5 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:44.353152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af790f-f2a6-4345-9248-129f618264a1 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:44.358851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4daedf-c85e-4afd-8fd6-e0c83c7169e2 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fafa83cf4bf40b2a94fc65dbba98f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03e428202a7e4fe4a29a787284d6470a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:44.359453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4daedf-c85e-4afd-8fd6-e0c83c7169e2 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:44.359738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4daedf-c85e-4afd-8fd6-e0c83c7169e2 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:44.360015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4daedf-c85e-4afd-8fd6-e0c83c7169e2 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:44.361008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4daedf-c85e-4afd-8fd6-e0c83c7169e2 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:44.363655 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3406/6874] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:44.364805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4daedf-c85e-4afd-8fd6-e0c83c7169e2 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:44.385527 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3469/6875] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:14:44 2022] POST /identity/v3/domains => generated 252 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:44.389292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb101e-eb1e-416e-b882-9ccbbbe83aa5 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:44.401424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb101e-eb1e-416e-b882-9ccbbbe83aa5 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:44.465644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3741a7-3370-43cb-ae01-025342de15de tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fafa83cf4bf40b2a94fc65dbba98f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03e428202a7e4fe4a29a787284d6470a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:44.466265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3741a7-3370-43cb-ae01-025342de15de tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:44.466578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3741a7-3370-43cb-ae01-025342de15de tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:44.466858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3741a7-3370-43cb-ae01-025342de15de tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:44.467512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3741a7-3370-43cb-ae01-025342de15de tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:44.469119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3741a7-3370-43cb-ae01-025342de15de tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:44.477863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.sql.core [None req-da3741a7-3370-43cb-ae01-025342de15de tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-tempest-domain-dup-965610683' for key 'project.ixu_project_name_domain_id'") Oct 01 21:14:44.477863 ncdv-master-1 devstack@keystone.service[99140]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 01 21:14:44.477863 ncdv-master-1 devstack@keystone.service[99140]: [parameters: {'id': '7079bbe3d4a241d5a44c21d22b02d538', 'name': 'tempest-domain-dup-965610683', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 01 21:14:44.477863 ncdv-master-1 devstack@keystone.service[99140]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99140) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 01 21:14:44.485981 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-da3741a7-3370-43cb-ae01-025342de15de tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-965610683.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-965610683. Oct 01 21:14:44.485981 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:14:44.485981 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 01 21:14:44.485981 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 01 21:14:44.485981 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 01 21:14:44.485981 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 01 21:14:44.485981 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 01 21:14:44.485981 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application result = self._query(query) Oct 01 21:14:44.485981 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 01 21:14:44.485981 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application conn.query(q) Oct 01 21:14:44.485981 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 01 21:14:44.485981 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 01 21:14:44.485981 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 01 21:14:44.486987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application result.read() Oct 01 21:14:44.486987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 01 21:14:44.486987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 01 21:14:44.486987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 01 21:14:44.486987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 01 21:14:44.486987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 01 21:14:44.486987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 01 21:14:44.486987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 01 21:14:44.486987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 01 21:14:44.486987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-tempest-domain-dup-965610683' for key 'project.ixu_project_name_domain_id'") Oct 01 21:14:44.486987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:14:44.486987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 01 21:14:44.486987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:14:44.486987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:14:44.486987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 01 21:14:44.486987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 01 21:14:44.486987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 01 21:14:44.486987 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 01 21:14:44.487940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 01 21:14:44.487940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application next(self.gen) Oct 01 21:14:44.487940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 01 21:14:44.487940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application yield resource Oct 01 21:14:44.487940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 01 21:14:44.487940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application next(self.gen) Oct 01 21:14:44.487940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 01 21:14:44.487940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.session.rollback() Oct 01 21:14:44.487940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 01 21:14:44.487940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.force_reraise() Oct 01 21:14:44.487940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 01 21:14:44.487940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise self.value Oct 01 21:14:44.487940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 01 21:14:44.487940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 01 21:14:44.487940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 01 21:14:44.487940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application session.commit() Oct 01 21:14:44.487940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 01 21:14:44.488908 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 01 21:14:44.488908 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 01 21:14:44.488908 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._prepare_impl() Oct 01 21:14:44.488908 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 01 21:14:44.488908 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.session.flush() Oct 01 21:14:44.488908 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 01 21:14:44.488908 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._flush(objects) Oct 01 21:14:44.488908 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 01 21:14:44.488908 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 01 21:14:44.488908 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 01 21:14:44.488908 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application compat.raise_( Oct 01 21:14:44.488908 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 01 21:14:44.488908 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception Oct 01 21:14:44.488908 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 01 21:14:44.488908 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application flush_context.execute() Oct 01 21:14:44.488908 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 01 21:14:44.488908 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rec.execute(self) Oct 01 21:14:44.489817 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 01 21:14:44.489817 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 01 21:14:44.489817 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 01 21:14:44.489817 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 01 21:14:44.489817 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 01 21:14:44.489817 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 01 21:14:44.489817 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 01 21:14:44.489817 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 01 21:14:44.489817 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 01 21:14:44.489817 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 01 21:14:44.489817 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 01 21:14:44.489817 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 01 21:14:44.489817 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 01 21:14:44.489817 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 01 21:14:44.489817 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 01 21:14:44.490669 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 01 21:14:44.490669 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 01 21:14:44.490669 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception Oct 01 21:14:44.490669 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 01 21:14:44.490669 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 01 21:14:44.490669 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 01 21:14:44.490669 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 01 21:14:44.490669 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 01 21:14:44.490669 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application result = self._query(query) Oct 01 21:14:44.490669 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 01 21:14:44.490669 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application conn.query(q) Oct 01 21:14:44.490669 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 01 21:14:44.490669 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 01 21:14:44.490669 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 01 21:14:44.490669 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application result.read() Oct 01 21:14:44.490669 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 01 21:14:44.490669 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 01 21:14:44.491612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 01 21:14:44.491612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 01 21:14:44.491612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 01 21:14:44.491612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 01 21:14:44.491612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 01 21:14:44.491612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 01 21:14:44.491612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-tempest-domain-dup-965610683' for key 'project.ixu_project_name_domain_id'") Oct 01 21:14:44.491612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 01 21:14:44.491612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [parameters: {'id': '7079bbe3d4a241d5a44c21d22b02d538', 'name': 'tempest-domain-dup-965610683', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 01 21:14:44.491612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 01 21:14:44.491612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:14:44.491612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:14:44.491612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:14:44.491612 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:14:44.492337 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 01 21:14:44.492337 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 01 21:14:44.492337 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 01 21:14:44.492337 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 01 21:14:44.492337 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name tempest-domain-dup-965610683. Oct 01 21:14:44.492337 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:14:44.492337 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:14:44.492337 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:14:44.492337 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:14:44.492337 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:14:44.492337 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:14:44.492337 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:14:44.492337 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:14:44.492337 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:14:44.492337 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:14:44.492337 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:14:44.492337 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:14:44.493248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:14:44.493248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:14:44.493248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 01 21:14:44.493248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 01 21:14:44.493248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:14:44.493248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:14:44.493248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 01 21:14:44.493248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 01 21:14:44.493248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:14:44.493248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:14:44.493248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 01 21:14:44.493248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 01 21:14:44.493248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-965610683. Oct 01 21:14:44.493248 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:14:44.493248 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3407/6876] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:14:44 2022] POST /identity/v3/domains => generated 211 bytes in 99 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 01 21:14:44.495661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4daedf-c85e-4afd-8fd6-e0c83c7169e2 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:44.507193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4daedf-c85e-4afd-8fd6-e0c83c7169e2 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:44.512057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a297334-03cc-4cad-a31f-679011dbfc26 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fafa83cf4bf40b2a94fc65dbba98f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03e428202a7e4fe4a29a787284d6470a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:44.512727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a297334-03cc-4cad-a31f-679011dbfc26 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:44.512975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a297334-03cc-4cad-a31f-679011dbfc26 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:44.513254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a297334-03cc-4cad-a31f-679011dbfc26 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] PATH_INFO: `/v3/domains/65a63a77663d428ea0950343f66480f3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:44.528864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a297334-03cc-4cad-a31f-679011dbfc26 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=65a63a77663d428ea0950343f66480f3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:44.530509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a297334-03cc-4cad-a31f-679011dbfc26 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:44.550877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:44.562710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5a297334-03cc-4cad-a31f-679011dbfc26 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '65a63a77663d428ea0950343f66480f3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.563664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5a297334-03cc-4cad-a31f-679011dbfc26 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '65a63a77663d428ea0950343f66480f3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.603975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5a297334-03cc-4cad-a31f-679011dbfc26 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '65a63a77663d428ea0950343f66480f3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.605006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5a297334-03cc-4cad-a31f-679011dbfc26 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '65a63a77663d428ea0950343f66480f3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.608601 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3470/6877] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:14:44 2022] PATCH /identity/v3/domains/65a63a77663d428ea0950343f66480f3 => generated 253 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:14:44.615497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3741a7-3370-43cb-ae01-025342de15de tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:44.617441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-da3741a7-3370-43cb-ae01-025342de15de tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:44.644625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3741a7-3370-43cb-ae01-025342de15de tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:44.751282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d126242-ef44-4039-963a-f06716e3efd4 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fafa83cf4bf40b2a94fc65dbba98f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03e428202a7e4fe4a29a787284d6470a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:44.752002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d126242-ef44-4039-963a-f06716e3efd4 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:44.752378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d126242-ef44-4039-963a-f06716e3efd4 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:44.752744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d126242-ef44-4039-963a-f06716e3efd4 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] PATH_INFO: `/v3/domains/65a63a77663d428ea0950343f66480f3` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:44.768298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d126242-ef44-4039-963a-f06716e3efd4 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=65a63a77663d428ea0950343f66480f3)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:44.770180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d126242-ef44-4039-963a-f06716e3efd4 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:44.773381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:44.821124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8d126242-ef44-4039-963a-f06716e3efd4 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '65a63a77663d428ea0950343f66480f3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.855823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8d126242-ef44-4039-963a-f06716e3efd4 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '65a63a77663d428ea0950343f66480f3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.901700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8d126242-ef44-4039-963a-f06716e3efd4 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65a63a77663d428ea0950343f66480f3', 'request_id': 'req-8d126242-ef44-4039-963a-f06716e3efd4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.916809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8d126242-ef44-4039-963a-f06716e3efd4 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65a63a77663d428ea0950343f66480f3', 'request_id': 'req-8d126242-ef44-4039-963a-f06716e3efd4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.926715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8d126242-ef44-4039-963a-f06716e3efd4 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65a63a77663d428ea0950343f66480f3', 'request_id': 'req-8d126242-ef44-4039-963a-f06716e3efd4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.948364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8d126242-ef44-4039-963a-f06716e3efd4 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] The token cache is being invalidate because project 65a63a77663d428ea0950343f66480f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:44.948938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8d126242-ef44-4039-963a-f06716e3efd4 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.949823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8d126242-ef44-4039-963a-f06716e3efd4 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.961404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8d126242-ef44-4039-963a-f06716e3efd4 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '65a63a77663d428ea0950343f66480f3', 'request_id': 'req-8d126242-ef44-4039-963a-f06716e3efd4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.962398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8d126242-ef44-4039-963a-f06716e3efd4 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '65a63a77663d428ea0950343f66480f3', 'request_id': 'req-8d126242-ef44-4039-963a-f06716e3efd4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.966275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8d126242-ef44-4039-963a-f06716e3efd4 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '65a63a77663d428ea0950343f66480f3', 'request_id': 'req-8d126242-ef44-4039-963a-f06716e3efd4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:44.973529 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3408/6878] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:14:44 2022] DELETE /identity/v3/domains/65a63a77663d428ea0950343f66480f3 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:44.979078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a297334-03cc-4cad-a31f-679011dbfc26 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:45.000995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a297334-03cc-4cad-a31f-679011dbfc26 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:45.106137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db7e45-59f2-4442-b1fa-e965d4838a38 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fafa83cf4bf40b2a94fc65dbba98f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03e428202a7e4fe4a29a787284d6470a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:45.107397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db7e45-59f2-4442-b1fa-e965d4838a38 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:45.108036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db7e45-59f2-4442-b1fa-e965d4838a38 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:45.108268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db7e45-59f2-4442-b1fa-e965d4838a38 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] PATH_INFO: `/v3/domains/89c99c87ad7d40399adcde13c83e5d3f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:45.120622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db7e45-59f2-4442-b1fa-e965d4838a38 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 21:14:45.122184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-f5db7e45-59f2-4442-b1fa-e965d4838a38 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Could not find domain: 89c99c87ad7d40399adcde13c83e5d3f.: keystone.exception.DomainNotFound: Could not find domain: 89c99c87ad7d40399adcde13c83e5d3f. Oct 01 21:14:45.122184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:14:45.122184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 01 21:14:45.122184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 01 21:14:45.122184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 21:14:45.122184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 21:14:45.122184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 21:14:45.122184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 21:14:45.122184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 89c99c87ad7d40399adcde13c83e5d3f. Oct 01 21:14:45.122184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:14:45.122184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:14:45.122184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:14:45.122184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:14:45.122184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:14:45.122184 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:14:45.123092 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:14:45.123092 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:14:45.123092 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:14:45.123092 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:14:45.123092 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:14:45.123092 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:14:45.123092 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:14:45.123092 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:14:45.123092 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 01 21:14:45.123092 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 01 21:14:45.123092 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 21:14:45.123092 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 21:14:45.123092 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 21:14:45.123092 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 21:14:45.123092 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:14:45.123092 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:14:45.127077 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:14:45.127077 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:14:45.127077 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:14:45.127077 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:14:45.127077 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:14:45.127077 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 21:14:45.127077 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:14:45.127077 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:14:45.127077 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:14:45.127077 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:14:45.127077 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:14:45.127077 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:14:45.127077 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:14:45.127077 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:14:45.127077 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 01 21:14:45.127077 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 21:14:45.127077 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 89c99c87ad7d40399adcde13c83e5d3f. Oct 01 21:14:45.128117 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:14:45.128117 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3471/6879] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:14:44 2022] PATCH /identity/v3/domains/89c99c87ad7d40399adcde13c83e5d3f => generated 112 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:14:45.128265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d126242-ef44-4039-963a-f06716e3efd4 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:45.139549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8d126242-ef44-4039-963a-f06716e3efd4 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:45.209410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d126242-ef44-4039-963a-f06716e3efd4 tempest-DomainsNegativeTestJSON-1145319215 tempest-DomainsNegativeTestJSON-1145319215-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:45.214968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5291d-2453-49f0-92ae-e05a31204f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:45.215614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5291d-2453-49f0-92ae-e05a31204f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:45.215890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5291d-2453-49f0-92ae-e05a31204f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:45.216161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5291d-2453-49f0-92ae-e05a31204f29 admin admin] PATH_INFO: `/v3/users/dcea499a458e4ef6a5a1d5dd983af075` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:45.217329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5291d-2453-49f0-92ae-e05a31204f29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcea499a458e4ef6a5a1d5dd983af075)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:45.219425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5291d-2453-49f0-92ae-e05a31204f29 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:45.277827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f5e5291d-2453-49f0-92ae-e05a31204f29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcea499a458e4ef6a5a1d5dd983af075'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:45.279045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f5e5291d-2453-49f0-92ae-e05a31204f29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcea499a458e4ef6a5a1d5dd983af075'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:45.280130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f5e5291d-2453-49f0-92ae-e05a31204f29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcea499a458e4ef6a5a1d5dd983af075'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:45.284497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:45.292825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f5e5291d-2453-49f0-92ae-e05a31204f29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcea499a458e4ef6a5a1d5dd983af075'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:45.309622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f5e5291d-2453-49f0-92ae-e05a31204f29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcea499a458e4ef6a5a1d5dd983af075'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:45.328209 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3409/6880] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:45 2022] DELETE /identity/v3/users/dcea499a458e4ef6a5a1d5dd983af075 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:45.333637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeaeac-73fb-4195-8e66-74eb6dc98bd8 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:45.334109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeaeac-73fb-4195-8e66-74eb6dc98bd8 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:45.334216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeaeac-73fb-4195-8e66-74eb6dc98bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:45.356673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-debeaeac-73fb-4195-8e66-74eb6dc98bd8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:45.358233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-debeaeac-73fb-4195-8e66-74eb6dc98bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:45.441267 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3472/6881] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:45.451953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5291d-2453-49f0-92ae-e05a31204f29 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:45.454534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f5e5291d-2453-49f0-92ae-e05a31204f29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:45.527353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5291d-2453-49f0-92ae-e05a31204f29 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:45.531746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1687e92-e446-483b-9920-548464da392e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:45.533003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1687e92-e446-483b-9920-548464da392e service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:45.533286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1687e92-e446-483b-9920-548464da392e service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:45.533560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1687e92-e446-483b-9920-548464da392e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:45.549415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1687e92-e446-483b-9920-548464da392e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:45.551783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1687e92-e446-483b-9920-548464da392e service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:45.563509 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3410/6882] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:45.799229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debeaeac-73fb-4195-8e66-74eb6dc98bd8 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:45.800877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-debeaeac-73fb-4195-8e66-74eb6dc98bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:45.819071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debeaeac-73fb-4195-8e66-74eb6dc98bd8 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:45.824795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e83a2-a039-44bc-b527-573bce801154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:45.825427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e83a2-a039-44bc-b527-573bce801154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:45.825702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e83a2-a039-44bc-b527-573bce801154 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:45.825972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e83a2-a039-44bc-b527-573bce801154 admin admin] PATH_INFO: `/v3/users/771b0d8ca48c4a5283a74d2797db57a4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:45.827113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e83a2-a039-44bc-b527-573bce801154 admin admin] RBAC: Authorizing `identity:delete_user(user_id=771b0d8ca48c4a5283a74d2797db57a4)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:45.829028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e83a2-a039-44bc-b527-573bce801154 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:45.887766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a66e83a2-a039-44bc-b527-573bce801154 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '771b0d8ca48c4a5283a74d2797db57a4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:45.895473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a66e83a2-a039-44bc-b527-573bce801154 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '771b0d8ca48c4a5283a74d2797db57a4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:45.905332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:UPDATE=1,keystone:DELETE=8 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:45.911291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a66e83a2-a039-44bc-b527-573bce801154 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '771b0d8ca48c4a5283a74d2797db57a4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:45.911899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a66e83a2-a039-44bc-b527-573bce801154 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '771b0d8ca48c4a5283a74d2797db57a4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:45.921564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a66e83a2-a039-44bc-b527-573bce801154 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '771b0d8ca48c4a5283a74d2797db57a4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:45.929819 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3473/6883] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:45 2022] DELETE /identity/v3/users/771b0d8ca48c4a5283a74d2797db57a4 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:45.933956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545acf2-9ea8-4c50-bbb5-51ca9808d65b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:45.934452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545acf2-9ea8-4c50-bbb5-51ca9808d65b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:45.934656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545acf2-9ea8-4c50-bbb5-51ca9808d65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:45.966908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9545acf2-9ea8-4c50-bbb5-51ca9808d65b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:45.968004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9545acf2-9ea8-4c50-bbb5-51ca9808d65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:46.062692 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3411/6884] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:46.075246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e83a2-a039-44bc-b527-573bce801154 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:46.076950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a66e83a2-a039-44bc-b527-573bce801154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:46.121006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545acf2-9ea8-4c50-bbb5-51ca9808d65b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:46.122632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9545acf2-9ea8-4c50-bbb5-51ca9808d65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:46.137045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545acf2-9ea8-4c50-bbb5-51ca9808d65b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:46.143394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b8f2a-aa36-496f-8f63-dab0475f94c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:46.143607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b8f2a-aa36-496f-8f63-dab0475f94c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:46.144015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b8f2a-aa36-496f-8f63-dab0475f94c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:46.144805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b8f2a-aa36-496f-8f63-dab0475f94c1 admin admin] PATH_INFO: `/v3/projects/96f05f07401c48ada3722a1e5f118696` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:46.146061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b8f2a-aa36-496f-8f63-dab0475f94c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96f05f07401c48ada3722a1e5f118696)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:46.148267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b8f2a-aa36-496f-8f63-dab0475f94c1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:46.156705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e83a2-a039-44bc-b527-573bce801154 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:46.166864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b6979-8fa6-47dc-8b84-0b11b6692fc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:46.167660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b6979-8fa6-47dc-8b84-0b11b6692fc4 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:46.168146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b6979-8fa6-47dc-8b84-0b11b6692fc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:46.168487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b6979-8fa6-47dc-8b84-0b11b6692fc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:46.195580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b6979-8fa6-47dc-8b84-0b11b6692fc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:46.199071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b6979-8fa6-47dc-8b84-0b11b6692fc4 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:46.220654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f30b8f2a-aa36-496f-8f63-dab0475f94c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96f05f07401c48ada3722a1e5f118696', 'request_id': 'req-f30b8f2a-aa36-496f-8f63-dab0475f94c1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:46.220799 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3474/6885] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:46.231402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f30b8f2a-aa36-496f-8f63-dab0475f94c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96f05f07401c48ada3722a1e5f118696', 'request_id': 'req-f30b8f2a-aa36-496f-8f63-dab0475f94c1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:46.239284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f30b8f2a-aa36-496f-8f63-dab0475f94c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96f05f07401c48ada3722a1e5f118696', 'request_id': 'req-f30b8f2a-aa36-496f-8f63-dab0475f94c1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:46.240441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:46.259824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f30b8f2a-aa36-496f-8f63-dab0475f94c1 admin admin] The token cache is being invalidate because project 96f05f07401c48ada3722a1e5f118696 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:46.260128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f30b8f2a-aa36-496f-8f63-dab0475f94c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:46.260888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f30b8f2a-aa36-496f-8f63-dab0475f94c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:46.265237 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3412/6886] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:46 2022] DELETE /identity/v3/projects/96f05f07401c48ada3722a1e5f118696 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:46.266725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b6979-8fa6-47dc-8b84-0b11b6692fc4 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:46.280445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b6979-8fa6-47dc-8b84-0b11b6692fc4 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:46.354141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b517a-4635-44ed-ba69-97ceaa850507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:46.354820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b517a-4635-44ed-ba69-97ceaa850507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:46.355085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b517a-4635-44ed-ba69-97ceaa850507 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:46.355350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b517a-4635-44ed-ba69-97ceaa850507 admin admin] PATH_INFO: `/v3/users/5fafa83cf4bf40b2a94fc65dbba98f5d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:46.356472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b517a-4635-44ed-ba69-97ceaa850507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fafa83cf4bf40b2a94fc65dbba98f5d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:46.358369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b517a-4635-44ed-ba69-97ceaa850507 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:46.406834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-437b517a-4635-44ed-ba69-97ceaa850507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fafa83cf4bf40b2a94fc65dbba98f5d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:46.424742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-437b517a-4635-44ed-ba69-97ceaa850507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fafa83cf4bf40b2a94fc65dbba98f5d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:46.434808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b8f2a-aa36-496f-8f63-dab0475f94c1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:46.436238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f30b8f2a-aa36-496f-8f63-dab0475f94c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:46.440608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-437b517a-4635-44ed-ba69-97ceaa850507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fafa83cf4bf40b2a94fc65dbba98f5d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:46.441344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-437b517a-4635-44ed-ba69-97ceaa850507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fafa83cf4bf40b2a94fc65dbba98f5d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:46.451236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-437b517a-4635-44ed-ba69-97ceaa850507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fafa83cf4bf40b2a94fc65dbba98f5d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:46.460312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b8f2a-aa36-496f-8f63-dab0475f94c1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:46.465827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fe4dc-9ec9-493b-8898-a12ff4b12bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:46.466452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fe4dc-9ec9-493b-8898-a12ff4b12bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:46.466763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fe4dc-9ec9-493b-8898-a12ff4b12bba admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:46.467033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fe4dc-9ec9-493b-8898-a12ff4b12bba admin admin] PATH_INFO: `/v3/projects/11c9ba909762485fa02968b280eb986c` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:46.467274 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3475/6887] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:46 2022] DELETE /identity/v3/users/5fafa83cf4bf40b2a94fc65dbba98f5d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:46.468147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5fe4dc-9ec9-493b-8898-a12ff4b12bba admin admin] RBAC: Authorizing `identity:delete_project(project_id=11c9ba909762485fa02968b280eb986c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:46.470043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5fe4dc-9ec9-493b-8898-a12ff4b12bba admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:46.523009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bf5fe4dc-9ec9-493b-8898-a12ff4b12bba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11c9ba909762485fa02968b280eb986c', 'request_id': 'req-bf5fe4dc-9ec9-493b-8898-a12ff4b12bba'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:46.531463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bf5fe4dc-9ec9-493b-8898-a12ff4b12bba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11c9ba909762485fa02968b280eb986c', 'request_id': 'req-bf5fe4dc-9ec9-493b-8898-a12ff4b12bba'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:46.538242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bf5fe4dc-9ec9-493b-8898-a12ff4b12bba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11c9ba909762485fa02968b280eb986c', 'request_id': 'req-bf5fe4dc-9ec9-493b-8898-a12ff4b12bba'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:46.550260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bf5fe4dc-9ec9-493b-8898-a12ff4b12bba admin admin] The token cache is being invalidate because project 11c9ba909762485fa02968b280eb986c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:46.550585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bf5fe4dc-9ec9-493b-8898-a12ff4b12bba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:46.551250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bf5fe4dc-9ec9-493b-8898-a12ff4b12bba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:46.552738 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3413/6888] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:46 2022] DELETE /identity/v3/projects/11c9ba909762485fa02968b280eb986c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:46.558287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1000794-0b2b-411b-aa31-427394d54171 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:46.558616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1000794-0b2b-411b-aa31-427394d54171 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:46.558896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1000794-0b2b-411b-aa31-427394d54171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:46.574082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:46.581957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e1000794-0b2b-411b-aa31-427394d54171 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:46.584174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e1000794-0b2b-411b-aa31-427394d54171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:46.656268 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3476/6889] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:46.660431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f08d72-3917-4ef1-aae7-70cfc3517a32 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:46.661139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f08d72-3917-4ef1-aae7-70cfc3517a32 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:46.661419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f08d72-3917-4ef1-aae7-70cfc3517a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:46.682341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e5f08d72-3917-4ef1-aae7-70cfc3517a32 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:46.683599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e5f08d72-3917-4ef1-aae7-70cfc3517a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:46.692307 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3414/6890] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:46.696909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1000794-0b2b-411b-aa31-427394d54171 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:46.706851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1000794-0b2b-411b-aa31-427394d54171 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:46.711461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cc817-82ce-42c3-b068-ac5194dc31c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:46.712058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cc817-82ce-42c3-b068-ac5194dc31c1 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:46.712336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cc817-82ce-42c3-b068-ac5194dc31c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:46.712607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cc817-82ce-42c3-b068-ac5194dc31c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:46.713249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cc817-82ce-42c3-b068-ac5194dc31c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:14:46.713550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cc817-82ce-42c3-b068-ac5194dc31c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:46.715286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cc817-82ce-42c3-b068-ac5194dc31c1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:46.728774 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3477/6891] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:14:46.733383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f08d72-3917-4ef1-aae7-70cfc3517a32 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:46.744228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f08d72-3917-4ef1-aae7-70cfc3517a32 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:46.749581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72accd56-53fd-4b80-83cb-2bae14e6862c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:46.750268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72accd56-53fd-4b80-83cb-2bae14e6862c admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:46.750588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72accd56-53fd-4b80-83cb-2bae14e6862c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:46.750867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72accd56-53fd-4b80-83cb-2bae14e6862c admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:46.751501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72accd56-53fd-4b80-83cb-2bae14e6862c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:46.753397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72accd56-53fd-4b80-83cb-2bae14e6862c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:46.760982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cc817-82ce-42c3-b068-ac5194dc31c1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:46.767333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-506cc817-82ce-42c3-b068-ac5194dc31c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:46.783770 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3415/6892] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:46 2022] POST /identity/v3/projects => generated 357 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:14:46.788437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72accd56-53fd-4b80-83cb-2bae14e6862c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:46.801124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cc817-82ce-42c3-b068-ac5194dc31c1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:46.863684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72accd56-53fd-4b80-83cb-2bae14e6862c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:46.872432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4cd78e-f16d-42b6-94fd-48d8096c556e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:46.873173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cd78e-f16d-42b6-94fd-48d8096c556e admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:46.873545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cd78e-f16d-42b6-94fd-48d8096c556e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:46.873958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cd78e-f16d-42b6-94fd-48d8096c556e admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:46.874980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4cd78e-f16d-42b6-94fd-48d8096c556e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:46.875699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b51ac-4252-494e-a0c7-5a3a98060374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:46.876335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b51ac-4252-494e-a0c7-5a3a98060374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:46.876619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b51ac-4252-494e-a0c7-5a3a98060374 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:46.876895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b51ac-4252-494e-a0c7-5a3a98060374 admin admin] PATH_INFO: `/v3/projects/03e428202a7e4fe4a29a787284d6470a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:46.877490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4cd78e-f16d-42b6-94fd-48d8096c556e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:46.878031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b51ac-4252-494e-a0c7-5a3a98060374 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03e428202a7e4fe4a29a787284d6470a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:46.879955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b51ac-4252-494e-a0c7-5a3a98060374 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:46.904070 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3416/6893] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:46 2022] POST /identity/v3/users => generated 279 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:14:46.907259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4cd78e-f16d-42b6-94fd-48d8096c556e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:46.920181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4cd78e-f16d-42b6-94fd-48d8096c556e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:46.934461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b27b51ac-4252-494e-a0c7-5a3a98060374 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03e428202a7e4fe4a29a787284d6470a', 'request_id': 'req-b27b51ac-4252-494e-a0c7-5a3a98060374'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:46.940870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b27b51ac-4252-494e-a0c7-5a3a98060374 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03e428202a7e4fe4a29a787284d6470a', 'request_id': 'req-b27b51ac-4252-494e-a0c7-5a3a98060374'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:46.946710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b27b51ac-4252-494e-a0c7-5a3a98060374 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03e428202a7e4fe4a29a787284d6470a', 'request_id': 'req-b27b51ac-4252-494e-a0c7-5a3a98060374'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:46.957221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:46.959805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b27b51ac-4252-494e-a0c7-5a3a98060374 admin admin] The token cache is being invalidate because project 03e428202a7e4fe4a29a787284d6470a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:46.960108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b27b51ac-4252-494e-a0c7-5a3a98060374 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:46.960776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b27b51ac-4252-494e-a0c7-5a3a98060374 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:46.962293 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3478/6894] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:46 2022] DELETE /identity/v3/projects/03e428202a7e4fe4a29a787284d6470a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:46.967656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6725af-404f-455a-b311-2728cc040e60 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:46.968510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6725af-404f-455a-b311-2728cc040e60 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:46.968891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6725af-404f-455a-b311-2728cc040e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:46.983910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e9495-b75a-4b7f-b7b0-734b9ec46d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:46.984521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e9495-b75a-4b7f-b7b0-734b9ec46d0f admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:46.984806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e9495-b75a-4b7f-b7b0-734b9ec46d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:46.985088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e9495-b75a-4b7f-b7b0-734b9ec46d0f admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:46.985781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e9495-b75a-4b7f-b7b0-734b9ec46d0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:46.987466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e9495-b75a-4b7f-b7b0-734b9ec46d0f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:46.993053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6a6725af-404f-455a-b311-2728cc040e60 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:46.994341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6a6725af-404f-455a-b311-2728cc040e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:47.001292 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3417/6895] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:47.002374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e9495-b75a-4b7f-b7b0-734b9ec46d0f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:47.004201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c85e9495-b75a-4b7f-b7b0-734b9ec46d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:47.014725 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3479/6896] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:47.017040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcdf43-f935-4451-8b7b-3d2db6f0788b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:47.017452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcdf43-f935-4451-8b7b-3d2db6f0788b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:47.017822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcdf43-f935-4451-8b7b-3d2db6f0788b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:47.017918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e9495-b75a-4b7f-b7b0-734b9ec46d0f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:47.023038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60549dd-6c7a-4b31-84f2-add64cdfe4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:47.023645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60549dd-6c7a-4b31-84f2-add64cdfe4a6 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:47.023932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60549dd-6c7a-4b31-84f2-add64cdfe4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:47.024211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60549dd-6c7a-4b31-84f2-add64cdfe4a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:47.024851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60549dd-6c7a-4b31-84f2-add64cdfe4a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:47.026456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60549dd-6c7a-4b31-84f2-add64cdfe4a6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:47.038554 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3418/6897] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:47.042769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60549dd-6c7a-4b31-84f2-add64cdfe4a6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:47.057406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60549dd-6c7a-4b31-84f2-add64cdfe4a6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:47.062369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-07bcdf43-f935-4451-8b7b-3d2db6f0788b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:47.062724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1574c1-ef08-4a06-b4b3-d4c8e1660eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:47.063389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1574c1-ef08-4a06-b4b3-d4c8e1660eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:47.063648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-07bcdf43-f935-4451-8b7b-3d2db6f0788b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:47.063740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1574c1-ef08-4a06-b4b3-d4c8e1660eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:47.063972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1574c1-ef08-4a06-b4b3-d4c8e1660eb5 admin admin] PATH_INFO: `/v3/projects/d97a25f0219e4b5d9fba692f03ee5a62/users/d9693d6502f34c7aa0604f840abb2a56/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:47.081477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1574c1-ef08-4a06-b4b3-d4c8e1660eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d97a25f0219e4b5d9fba692f03ee5a62, user_id=d9693d6502f34c7aa0604f840abb2a56, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:47.083057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1574c1-ef08-4a06-b4b3-d4c8e1660eb5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:47.085114 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3480/6898] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:47.089147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcdf43-f935-4451-8b7b-3d2db6f0788b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:47.099090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcdf43-f935-4451-8b7b-3d2db6f0788b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:47.100165 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3419/6899] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:47 2022] PUT /identity/v3/projects/d97a25f0219e4b5d9fba692f03ee5a62/users/d9693d6502f34c7aa0604f840abb2a56/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:47.110703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:47.111219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66433f5e-6872-4e30-81a3-90eb81550fac None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:47.111528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66433f5e-6872-4e30-81a3-90eb81550fac None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:47.111868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66433f5e-6872-4e30-81a3-90eb81550fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:47.148569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-66433f5e-6872-4e30-81a3-90eb81550fac None None] MFA Rules not processed for user `d9693d6502f34c7aa0604f840abb2a56`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:47.149910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-66433f5e-6872-4e30-81a3-90eb81550fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:47.168088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed23764-5144-4766-b4bf-9e81c8da8047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:47.168698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed23764-5144-4766-b4bf-9e81c8da8047 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:47.168978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed23764-5144-4766-b4bf-9e81c8da8047 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:47.169248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed23764-5144-4766-b4bf-9e81c8da8047 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:47.169896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed23764-5144-4766-b4bf-9e81c8da8047 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:14:47.170195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed23764-5144-4766-b4bf-9e81c8da8047 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:47.171813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed23764-5144-4766-b4bf-9e81c8da8047 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:47.186768 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3481/6900] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:14:47.191000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed23764-5144-4766-b4bf-9e81c8da8047 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:47.201602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed23764-5144-4766-b4bf-9e81c8da8047 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:47.206379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473676e-1945-4ba3-9efa-9df9b9d94e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:47.207013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473676e-1945-4ba3-9efa-9df9b9d94e92 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:47.207514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473676e-1945-4ba3-9efa-9df9b9d94e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:47.207812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473676e-1945-4ba3-9efa-9df9b9d94e92 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:47.208470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473676e-1945-4ba3-9efa-9df9b9d94e92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:47.210915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473676e-1945-4ba3-9efa-9df9b9d94e92 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:47.239240 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3420/6901] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:47 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:47.250293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66433f5e-6872-4e30-81a3-90eb81550fac None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:47.252119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-66433f5e-6872-4e30-81a3-90eb81550fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:47.261770 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3482/6902] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:47 2022] POST /identity/v3/projects => generated 367 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:47.263536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473676e-1945-4ba3-9efa-9df9b9d94e92 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:47.342577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66433f5e-6872-4e30-81a3-90eb81550fac None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:47.347027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473676e-1945-4ba3-9efa-9df9b9d94e92 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:47.353166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f66ae-1b35-44cb-89ff-dddcb11d89b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:47.353775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f66ae-1b35-44cb-89ff-dddcb11d89b1 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:47.354054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f66ae-1b35-44cb-89ff-dddcb11d89b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:47.354337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f66ae-1b35-44cb-89ff-dddcb11d89b1 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:47.355004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f66ae-1b35-44cb-89ff-dddcb11d89b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:47.356924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f66ae-1b35-44cb-89ff-dddcb11d89b1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:47.380946 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3483/6903] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:47 2022] POST /identity/v3/users => generated 284 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:47.386253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f66ae-1b35-44cb-89ff-dddcb11d89b1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:47.394184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:47.399542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f66ae-1b35-44cb-89ff-dddcb11d89b1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:47.404571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e02cc-888e-4d9c-8206-4ce029efcadb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:47.405196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e02cc-888e-4d9c-8206-4ce029efcadb service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:47.405476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e02cc-888e-4d9c-8206-4ce029efcadb service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:47.405752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e02cc-888e-4d9c-8206-4ce029efcadb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:47.406881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa227ce-0a38-4564-814f-d5f7e897be71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:47.407483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa227ce-0a38-4564-814f-d5f7e897be71 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:47.407758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa227ce-0a38-4564-814f-d5f7e897be71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:47.408028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa227ce-0a38-4564-814f-d5f7e897be71 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:47.408673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa227ce-0a38-4564-814f-d5f7e897be71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:47.410298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa227ce-0a38-4564-814f-d5f7e897be71 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:47.428096 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3484/6904] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:14:47.430343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa227ce-0a38-4564-814f-d5f7e897be71 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:47.455910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa227ce-0a38-4564-814f-d5f7e897be71 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:47.470303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46364c-6d4c-4130-b3d9-f70bf191a52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:47.471355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46364c-6d4c-4130-b3d9-f70bf191a52b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:47.471845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46364c-6d4c-4130-b3d9-f70bf191a52b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:47.472264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46364c-6d4c-4130-b3d9-f70bf191a52b admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:47.473060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46364c-6d4c-4130-b3d9-f70bf191a52b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:47.476392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46364c-6d4c-4130-b3d9-f70bf191a52b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:47.487970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e02cc-888e-4d9c-8206-4ce029efcadb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:47.489584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e02cc-888e-4d9c-8206-4ce029efcadb service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:47.493452 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3485/6905] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:14:47.499464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46364c-6d4c-4130-b3d9-f70bf191a52b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:47.511659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46364c-6d4c-4130-b3d9-f70bf191a52b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:47.512904 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3421/6906] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:47.518425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcdf83f-7853-4981-ae5d-4942c0267469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:47.519230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcdf83f-7853-4981-ae5d-4942c0267469 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:47.519604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcdf83f-7853-4981-ae5d-4942c0267469 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:47.520258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcdf83f-7853-4981-ae5d-4942c0267469 admin admin] PATH_INFO: `/v3/projects/f88f20f15e4b4718b1650395982b5daa/users/df19f2c2abb042398dcd1128119a9fac/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:47.528598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ea7fb-ca35-4201-9f15-cb9fee0a4c4f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:47.528890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ea7fb-ca35-4201-9f15-cb9fee0a4c4f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:47.529322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ea7fb-ca35-4201-9f15-cb9fee0a4c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:47.541187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcdf83f-7853-4981-ae5d-4942c0267469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f88f20f15e4b4718b1650395982b5daa, user_id=df19f2c2abb042398dcd1128119a9fac, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:47.543829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcdf83f-7853-4981-ae5d-4942c0267469 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:47.561803 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3486/6907] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:47 2022] PUT /identity/v3/projects/f88f20f15e4b4718b1650395982b5daa/users/df19f2c2abb042398dcd1128119a9fac/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:47.571303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-861ea7fb-ca35-4201-9f15-cb9fee0a4c4f None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:47.572805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-861ea7fb-ca35-4201-9f15-cb9fee0a4c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:47.573534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190c363-a1e3-45f3-8182-7a36df5467d5 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:47.573983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190c363-a1e3-45f3-8182-7a36df5467d5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:47.574264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190c363-a1e3-45f3-8182-7a36df5467d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:47.627126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e190c363-a1e3-45f3-8182-7a36df5467d5 None None] MFA Rules not processed for user `df19f2c2abb042398dcd1128119a9fac`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:47.630026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e190c363-a1e3-45f3-8182-7a36df5467d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:47.652608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:47.656354 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3422/6908] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:14:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:47.672949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ea7fb-ca35-4201-9f15-cb9fee0a4c4f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:47.674737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-861ea7fb-ca35-4201-9f15-cb9fee0a4c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:47.746455 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3487/6909] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:47.747612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190c363-a1e3-45f3-8182-7a36df5467d5 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:47.780655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ea7fb-ca35-4201-9f15-cb9fee0a4c4f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:47.788298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e15254-af26-4bb0-a41e-a0a7db20972d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:47.790173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e15254-af26-4bb0-a41e-a0a7db20972d service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:47.791179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e15254-af26-4bb0-a41e-a0a7db20972d service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:47.791681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e15254-af26-4bb0-a41e-a0a7db20972d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:47.807772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e15254-af26-4bb0-a41e-a0a7db20972d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:47.810786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e15254-af26-4bb0-a41e-a0a7db20972d service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:47.823266 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3423/6910] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:14:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 21:14:47.837038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190c363-a1e3-45f3-8182-7a36df5467d5 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:47.845775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b49b4-9456-41e8-9e55-059747d6f1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:47.846651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b49b4-9456-41e8-9e55-059747d6f1ad admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:47.847051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b49b4-9456-41e8-9e55-059747d6f1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:47.847455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b49b4-9456-41e8-9e55-059747d6f1ad admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:47.848272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b49b4-9456-41e8-9e55-059747d6f1ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:47.851668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b49b4-9456-41e8-9e55-059747d6f1ad admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:47.907841 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3488/6911] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:47 2022] POST /identity/v3/projects => generated 367 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:47.911914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e15254-af26-4bb0-a41e-a0a7db20972d service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:47.942332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e15254-af26-4bb0-a41e-a0a7db20972d service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:47.979108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:48.014858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f602df9-a150-4fbf-b637-8b15a6969b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:48.015556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f602df9-a150-4fbf-b637-8b15a6969b8d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:48.015908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f602df9-a150-4fbf-b637-8b15a6969b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:48.016268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f602df9-a150-4fbf-b637-8b15a6969b8d admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:48.016965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f602df9-a150-4fbf-b637-8b15a6969b8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:48.019670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f602df9-a150-4fbf-b637-8b15a6969b8d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:48.059260 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3424/6912] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:47 2022] POST /identity/v3/users => generated 290 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:14:48.064097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b49b4-9456-41e8-9e55-059747d6f1ad admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:48.077489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b49b4-9456-41e8-9e55-059747d6f1ad admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:48.083028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06c226-9824-4e10-921c-0f6e9e11a8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:48.083805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06c226-9824-4e10-921c-0f6e9e11a8ae admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:48.084175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06c226-9824-4e10-921c-0f6e9e11a8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:48.085088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06c226-9824-4e10-921c-0f6e9e11a8ae admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:48.085806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06c226-9824-4e10-921c-0f6e9e11a8ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:48.087556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06c226-9824-4e10-921c-0f6e9e11a8ae admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:48.101973 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3489/6913] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:48.106384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f602df9-a150-4fbf-b637-8b15a6969b8d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:48.121825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f602df9-a150-4fbf-b637-8b15a6969b8d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:48.127024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b0605-7aa5-40bc-a51c-663d40d11e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:48.127796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b0605-7aa5-40bc-a51c-663d40d11e04 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:48.128158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b0605-7aa5-40bc-a51c-663d40d11e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:48.128506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b0605-7aa5-40bc-a51c-663d40d11e04 admin admin] PATH_INFO: `/v3/projects/97b9ba86b6fc4899ac71ee9a664cc9bc/users/9bc9cad00323458b8a895a12ed46dd87/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:48.146811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551b0605-7aa5-40bc-a51c-663d40d11e04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97b9ba86b6fc4899ac71ee9a664cc9bc, user_id=9bc9cad00323458b8a895a12ed46dd87, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:48.149791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551b0605-7aa5-40bc-a51c-663d40d11e04 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:48.169071 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3425/6914] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:48 2022] PUT /identity/v3/projects/97b9ba86b6fc4899ac71ee9a664cc9bc/users/9bc9cad00323458b8a895a12ed46dd87/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:48.171055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06c226-9824-4e10-921c-0f6e9e11a8ae admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:48.183531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06c226-9824-4e10-921c-0f6e9e11a8ae admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:48.249838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbdf43-cc35-4584-ad06-17cf40a7580f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:48.250769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbdf43-cc35-4584-ad06-17cf40a7580f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:48.251128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbdf43-cc35-4584-ad06-17cf40a7580f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:48.251534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbdf43-cc35-4584-ad06-17cf40a7580f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:48.253545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbdf43-cc35-4584-ad06-17cf40a7580f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:48.254299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbdf43-cc35-4584-ad06-17cf40a7580f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:48.265827 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3490/6915] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:48.270405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b0605-7aa5-40bc-a51c-663d40d11e04 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:48.284840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b0605-7aa5-40bc-a51c-663d40d11e04 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:48.293157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c56a3e-b4f2-412a-adac-3a1895d48128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:48.294042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c56a3e-b4f2-412a-adac-3a1895d48128 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:48.294744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c56a3e-b4f2-412a-adac-3a1895d48128 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:48.295099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c56a3e-b4f2-412a-adac-3a1895d48128 admin admin] PATH_INFO: `/v3/projects/97b9ba86b6fc4899ac71ee9a664cc9bc/users/9bc9cad00323458b8a895a12ed46dd87/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:48.297537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c56a3e-b4f2-412a-adac-3a1895d48128 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97b9ba86b6fc4899ac71ee9a664cc9bc, user_id=9bc9cad00323458b8a895a12ed46dd87, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:48.307804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c56a3e-b4f2-412a-adac-3a1895d48128 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:48.338705 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3426/6916] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:48 2022] PUT /identity/v3/projects/97b9ba86b6fc4899ac71ee9a664cc9bc/users/9bc9cad00323458b8a895a12ed46dd87/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:48.340847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3abd66-32a3-4d68-89a0-395186bedcec None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:48.341139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3abd66-32a3-4d68-89a0-395186bedcec None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:48.341422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3abd66-32a3-4d68-89a0-395186bedcec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:48.380688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4a3abd66-32a3-4d68-89a0-395186bedcec None None] MFA Rules not processed for user `9bc9cad00323458b8a895a12ed46dd87`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:48.382208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4a3abd66-32a3-4d68-89a0-395186bedcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:48.408357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:48.510902 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3491/6917] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:48 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:48.514914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c56a3e-b4f2-412a-adac-3a1895d48128 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:48.529476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c56a3e-b4f2-412a-adac-3a1895d48128 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:48.534923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf8220-fb86-4f4b-a015-9cd6a4e6c33e tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bc9cad00323458b8a895a12ed46dd87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97b9ba86b6fc4899ac71ee9a664cc9bc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:48.536455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf8220-fb86-4f4b-a015-9cd6a4e6c33e tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:48.536904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf8220-fb86-4f4b-a015-9cd6a4e6c33e tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:48.537322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf8220-fb86-4f4b-a015-9cd6a4e6c33e tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:48.539446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf8220-fb86-4f4b-a015-9cd6a4e6c33e tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:48.541455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf8220-fb86-4f4b-a015-9cd6a4e6c33e tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:48.581977 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3427/6918] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:14:48 2022] POST /identity/v3/domains => generated 274 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:48.587787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3abd66-32a3-4d68-89a0-395186bedcec None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:48.597085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3abd66-32a3-4d68-89a0-395186bedcec None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:48.722680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac9773-a7a6-4552-a3d0-d0a0a834d698 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bc9cad00323458b8a895a12ed46dd87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97b9ba86b6fc4899ac71ee9a664cc9bc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:48.723917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac9773-a7a6-4552-a3d0-d0a0a834d698 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:48.724268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac9773-a7a6-4552-a3d0-d0a0a834d698 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:48.724734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac9773-a7a6-4552-a3d0-d0a0a834d698 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:48.726103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac9773-a7a6-4552-a3d0-d0a0a834d698 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:48.730183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac9773-a7a6-4552-a3d0-d0a0a834d698 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:48.782077 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3492/6919] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:48 2022] POST /identity/v3/users => generated 347 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:48.787072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf8220-fb86-4f4b-a015-9cd6a4e6c33e tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:48.806726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf8220-fb86-4f4b-a015-9cd6a4e6c33e tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:48.816302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa8e21-26d9-44b2-a8a5-71653ac66846 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bc9cad00323458b8a895a12ed46dd87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97b9ba86b6fc4899ac71ee9a664cc9bc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:48.817081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa8e21-26d9-44b2-a8a5-71653ac66846 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:48.817899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa8e21-26d9-44b2-a8a5-71653ac66846 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:48.818055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa8e21-26d9-44b2-a8a5-71653ac66846 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:48.819104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa8e21-26d9-44b2-a8a5-71653ac66846 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:48.821393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa8e21-26d9-44b2-a8a5-71653ac66846 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:48.869119 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3428/6920] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:48 2022] POST /identity/v3/users => generated 323 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:14:48.869946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac9773-a7a6-4552-a3d0-d0a0a834d698 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:48.884571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac9773-a7a6-4552-a3d0-d0a0a834d698 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:48.890408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45134f-1011-4c9f-970a-0dd22fd56fc9 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bc9cad00323458b8a895a12ed46dd87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97b9ba86b6fc4899ac71ee9a664cc9bc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:48.891135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45134f-1011-4c9f-970a-0dd22fd56fc9 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:48.891418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45134f-1011-4c9f-970a-0dd22fd56fc9 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:48.891695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45134f-1011-4c9f-970a-0dd22fd56fc9 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] PATH_INFO: `/v3/users/a025ca197c1d44bf83af56ffd6c12def` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:48.909153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45134f-1011-4c9f-970a-0dd22fd56fc9 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorizing `identity:get_user(user_id=a025ca197c1d44bf83af56ffd6c12def)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:48.911460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45134f-1011-4c9f-970a-0dd22fd56fc9 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:48.913984 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3493/6921] 10.222.0.17 () {58 vars in 1221 bytes} [Sat Oct 1 21:14:48 2022] GET /identity/v3/users/a025ca197c1d44bf83af56ffd6c12def => generated 347 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:14:48.920825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa8e21-26d9-44b2-a8a5-71653ac66846 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:48.937565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa8e21-26d9-44b2-a8a5-71653ac66846 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:48.944985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350e549-2046-49c6-ade3-c5725768a4fd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bc9cad00323458b8a895a12ed46dd87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97b9ba86b6fc4899ac71ee9a664cc9bc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:48.945736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350e549-2046-49c6-ade3-c5725768a4fd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:48.946147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350e549-2046-49c6-ade3-c5725768a4fd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:48.946583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350e549-2046-49c6-ade3-c5725768a4fd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:48.947472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350e549-2046-49c6-ade3-c5725768a4fd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Adding query filter params (domain_id=78b28af29d32438e95bcde204c1b82f3) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:14:48.947884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350e549-2046-49c6-ade3-c5725768a4fd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:48.949397 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-d350e549-2046-49c6-ade3-c5725768a4fd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:14:48.949397 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:14:48.949397 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:14:48.952371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350e549-2046-49c6-ade3-c5725768a4fd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:48.967801 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3429/6922] 10.222.0.17 () {58 vars in 1207 bytes} [Sat Oct 1 21:14:48 2022] GET /identity/v3/users?domain_id=78b28af29d32438e95bcde204c1b82f3 => generated 484 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:14:48.973060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45134f-1011-4c9f-970a-0dd22fd56fc9 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:48.989597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45134f-1011-4c9f-970a-0dd22fd56fc9 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:49.001099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b168c-89e1-467a-b08e-19aa61ae6767 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bc9cad00323458b8a895a12ed46dd87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97b9ba86b6fc4899ac71ee9a664cc9bc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:49.001759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b168c-89e1-467a-b08e-19aa61ae6767 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:49.002044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b168c-89e1-467a-b08e-19aa61ae6767 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:49.002934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b168c-89e1-467a-b08e-19aa61ae6767 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:49.003559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b168c-89e1-467a-b08e-19aa61ae6767 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:49.005925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b168c-89e1-467a-b08e-19aa61ae6767 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:49.081788 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3494/6923] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:48 2022] GET /identity/v3/users => generated 7739 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:49.083019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350e549-2046-49c6-ade3-c5725768a4fd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:49.097441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350e549-2046-49c6-ade3-c5725768a4fd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:49.109905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb191b0e-54d2-4b1e-adda-d6b495235b46 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bc9cad00323458b8a895a12ed46dd87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97b9ba86b6fc4899ac71ee9a664cc9bc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:49.110662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb191b0e-54d2-4b1e-adda-d6b495235b46 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:49.111073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb191b0e-54d2-4b1e-adda-d6b495235b46 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:49.111490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb191b0e-54d2-4b1e-adda-d6b495235b46 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:49.112674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb191b0e-54d2-4b1e-adda-d6b495235b46 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Adding query filter params (name=tempest-test_user-1126778871) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:14:49.112982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb191b0e-54d2-4b1e-adda-d6b495235b46 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:49.121213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb191b0e-54d2-4b1e-adda-d6b495235b46 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:49.188579 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3430/6924] 10.222.0.17 () {58 vars in 1189 bytes} [Sat Oct 1 21:14:49 2022] GET /identity/v3/users?name=tempest-test_user-1126778871 => generated 475 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:14:49.189003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b168c-89e1-467a-b08e-19aa61ae6767 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:49.202114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b168c-89e1-467a-b08e-19aa61ae6767 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:49.226526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ec670-ccf6-40f4-b7dd-2fa5b3c20ce6 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bc9cad00323458b8a895a12ed46dd87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97b9ba86b6fc4899ac71ee9a664cc9bc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:49.227163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ec670-ccf6-40f4-b7dd-2fa5b3c20ce6 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:49.227454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ec670-ccf6-40f4-b7dd-2fa5b3c20ce6 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:49.227735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ec670-ccf6-40f4-b7dd-2fa5b3c20ce6 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:49.228478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ec670-ccf6-40f4-b7dd-2fa5b3c20ce6 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Adding query filter params (enabled=False) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:14:49.228787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ec670-ccf6-40f4-b7dd-2fa5b3c20ce6 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:49.230743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ec670-ccf6-40f4-b7dd-2fa5b3c20ce6 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:49.266060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb191b0e-54d2-4b1e-adda-d6b495235b46 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:49.272303 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3495/6925] 10.222.0.17 () {58 vars in 1149 bytes} [Sat Oct 1 21:14:49 2022] GET /identity/v3/users?enabled=False => generated 431 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:14:49.283229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb191b0e-54d2-4b1e-adda-d6b495235b46 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:49.291738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55e21e-4106-46af-9cf9-a38f06db87bd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bc9cad00323458b8a895a12ed46dd87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97b9ba86b6fc4899ac71ee9a664cc9bc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:49.292958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55e21e-4106-46af-9cf9-a38f06db87bd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:49.293331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55e21e-4106-46af-9cf9-a38f06db87bd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:49.293609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55e21e-4106-46af-9cf9-a38f06db87bd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] PATH_INFO: `/v3/users/dfc44f0e53724c689da7429b78679e38` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:49.328182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55e21e-4106-46af-9cf9-a38f06db87bd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorizing `identity:delete_user(user_id=dfc44f0e53724c689da7429b78679e38)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:49.331796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55e21e-4106-46af-9cf9-a38f06db87bd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:49.408053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6c55e21e-4106-46af-9cf9-a38f06db87bd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfc44f0e53724c689da7429b78679e38'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:49.409326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6c55e21e-4106-46af-9cf9-a38f06db87bd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfc44f0e53724c689da7429b78679e38'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:49.410023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6c55e21e-4106-46af-9cf9-a38f06db87bd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfc44f0e53724c689da7429b78679e38'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:49.424954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6c55e21e-4106-46af-9cf9-a38f06db87bd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfc44f0e53724c689da7429b78679e38'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:49.458715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc1eee-b106-466a-9425-a85ec1bd2639 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:49.459674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc1eee-b106-466a-9425-a85ec1bd2639 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:49.460215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc1eee-b106-466a-9425-a85ec1bd2639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:49.461734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6c55e21e-4106-46af-9cf9-a38f06db87bd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfc44f0e53724c689da7429b78679e38'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:49.463221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=80,keystone:INSERT=10,keystone:DELETE=10 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:49.485653 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3431/6926] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:49 2022] DELETE /identity/v3/users/dfc44f0e53724c689da7429b78679e38 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:49.487608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55e21e-4106-46af-9cf9-a38f06db87bd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:49.492326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6c55e21e-4106-46af-9cf9-a38f06db87bd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:49.498415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8dfc1eee-b106-466a-9425-a85ec1bd2639 None None] MFA Rules not processed for user `a2745d6923e347939e2c67015aefadda`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:49.499734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8dfc1eee-b106-466a-9425-a85ec1bd2639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:49.581103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:49.587331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55e21e-4106-46af-9cf9-a38f06db87bd tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:49.592528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26febefa-73c2-409c-8ff7-b9bf3e4dcc34 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bc9cad00323458b8a895a12ed46dd87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97b9ba86b6fc4899ac71ee9a664cc9bc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:49.593180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26febefa-73c2-409c-8ff7-b9bf3e4dcc34 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:49.593452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26febefa-73c2-409c-8ff7-b9bf3e4dcc34 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:49.594084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26febefa-73c2-409c-8ff7-b9bf3e4dcc34 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] PATH_INFO: `/v3/users/a025ca197c1d44bf83af56ffd6c12def` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:49.595316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26febefa-73c2-409c-8ff7-b9bf3e4dcc34 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorizing `identity:delete_user(user_id=a025ca197c1d44bf83af56ffd6c12def)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:49.597249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26febefa-73c2-409c-8ff7-b9bf3e4dcc34 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:49.614223 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3496/6927] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:49.698200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-26febefa-73c2-409c-8ff7-b9bf3e4dcc34 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a025ca197c1d44bf83af56ffd6c12def'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:49.699047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-26febefa-73c2-409c-8ff7-b9bf3e4dcc34 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a025ca197c1d44bf83af56ffd6c12def'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:49.699675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-26febefa-73c2-409c-8ff7-b9bf3e4dcc34 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a025ca197c1d44bf83af56ffd6c12def'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:49.717254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-26febefa-73c2-409c-8ff7-b9bf3e4dcc34 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a025ca197c1d44bf83af56ffd6c12def'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:49.820827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-26febefa-73c2-409c-8ff7-b9bf3e4dcc34 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a025ca197c1d44bf83af56ffd6c12def'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:49.855430 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3432/6928] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:49 2022] DELETE /identity/v3/users/a025ca197c1d44bf83af56ffd6c12def => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:49.860148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc1eee-b106-466a-9425-a85ec1bd2639 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:49.882268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc1eee-b106-466a-9425-a85ec1bd2639 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:49.955423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853eeaf8-d36c-46de-8a60-bad404876e80 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bc9cad00323458b8a895a12ed46dd87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97b9ba86b6fc4899ac71ee9a664cc9bc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:49.956184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853eeaf8-d36c-46de-8a60-bad404876e80 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:49.956649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853eeaf8-d36c-46de-8a60-bad404876e80 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:49.957133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853eeaf8-d36c-46de-8a60-bad404876e80 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] PATH_INFO: `/v3/domains/78b28af29d32438e95bcde204c1b82f3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:49.960490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853eeaf8-d36c-46de-8a60-bad404876e80 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=78b28af29d32438e95bcde204c1b82f3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:49.963277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853eeaf8-d36c-46de-8a60-bad404876e80 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:49.997788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-853eeaf8-d36c-46de-8a60-bad404876e80 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '78b28af29d32438e95bcde204c1b82f3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:49.999785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-853eeaf8-d36c-46de-8a60-bad404876e80 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '78b28af29d32438e95bcde204c1b82f3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:50.046732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-853eeaf8-d36c-46de-8a60-bad404876e80 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '78b28af29d32438e95bcde204c1b82f3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:50.047607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-853eeaf8-d36c-46de-8a60-bad404876e80 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '78b28af29d32438e95bcde204c1b82f3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:50.050730 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3497/6929] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:14:49 2022] PATCH /identity/v3/domains/78b28af29d32438e95bcde204c1b82f3 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:14:50.055767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26febefa-73c2-409c-8ff7-b9bf3e4dcc34 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:50.057570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-26febefa-73c2-409c-8ff7-b9bf3e4dcc34 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:50.072611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26febefa-73c2-409c-8ff7-b9bf3e4dcc34 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:50.159523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd05662-30f2-47f3-b221-9957d953bdc1 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bc9cad00323458b8a895a12ed46dd87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97b9ba86b6fc4899ac71ee9a664cc9bc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:50.160464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd05662-30f2-47f3-b221-9957d953bdc1 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:50.160867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd05662-30f2-47f3-b221-9957d953bdc1 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:50.161314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd05662-30f2-47f3-b221-9957d953bdc1 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] PATH_INFO: `/v3/domains/78b28af29d32438e95bcde204c1b82f3` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:50.188921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd05662-30f2-47f3-b221-9957d953bdc1 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=78b28af29d32438e95bcde204c1b82f3)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:50.191367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd05662-30f2-47f3-b221-9957d953bdc1 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:50.200127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:50.252544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-acd05662-30f2-47f3-b221-9957d953bdc1 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '78b28af29d32438e95bcde204c1b82f3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:50.265541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-acd05662-30f2-47f3-b221-9957d953bdc1 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '78b28af29d32438e95bcde204c1b82f3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:50.339569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-acd05662-30f2-47f3-b221-9957d953bdc1 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78b28af29d32438e95bcde204c1b82f3', 'request_id': 'req-acd05662-30f2-47f3-b221-9957d953bdc1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:50.352278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-acd05662-30f2-47f3-b221-9957d953bdc1 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78b28af29d32438e95bcde204c1b82f3', 'request_id': 'req-acd05662-30f2-47f3-b221-9957d953bdc1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:50.360949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-acd05662-30f2-47f3-b221-9957d953bdc1 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78b28af29d32438e95bcde204c1b82f3', 'request_id': 'req-acd05662-30f2-47f3-b221-9957d953bdc1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:50.379029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-acd05662-30f2-47f3-b221-9957d953bdc1 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] The token cache is being invalidate because project 78b28af29d32438e95bcde204c1b82f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:50.379355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-acd05662-30f2-47f3-b221-9957d953bdc1 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:50.380323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-acd05662-30f2-47f3-b221-9957d953bdc1 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:50.394597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-acd05662-30f2-47f3-b221-9957d953bdc1 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '78b28af29d32438e95bcde204c1b82f3', 'request_id': 'req-acd05662-30f2-47f3-b221-9957d953bdc1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:50.395385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-acd05662-30f2-47f3-b221-9957d953bdc1 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '78b28af29d32438e95bcde204c1b82f3', 'request_id': 'req-acd05662-30f2-47f3-b221-9957d953bdc1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:50.402818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-acd05662-30f2-47f3-b221-9957d953bdc1 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '78b28af29d32438e95bcde204c1b82f3', 'request_id': 'req-acd05662-30f2-47f3-b221-9957d953bdc1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:50.414545 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3433/6930] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:14:50 2022] DELETE /identity/v3/domains/78b28af29d32438e95bcde204c1b82f3 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:50.417111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853eeaf8-d36c-46de-8a60-bad404876e80 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:50.418356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-853eeaf8-d36c-46de-8a60-bad404876e80 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:50.535050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853eeaf8-d36c-46de-8a60-bad404876e80 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:50.634452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed95c6-c21e-4683-b409-555b1560dcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:50.635252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed95c6-c21e-4683-b409-555b1560dcb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:50.635535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed95c6-c21e-4683-b409-555b1560dcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:50.636030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed95c6-c21e-4683-b409-555b1560dcb1 admin admin] PATH_INFO: `/v3/users/df19f2c2abb042398dcd1128119a9fac` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:50.637895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed95c6-c21e-4683-b409-555b1560dcb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df19f2c2abb042398dcd1128119a9fac)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:50.640956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed95c6-c21e-4683-b409-555b1560dcb1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:50.660741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:50.719054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d1ed95c6-c21e-4683-b409-555b1560dcb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df19f2c2abb042398dcd1128119a9fac'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:50.729708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d1ed95c6-c21e-4683-b409-555b1560dcb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df19f2c2abb042398dcd1128119a9fac'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:50.750872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d1ed95c6-c21e-4683-b409-555b1560dcb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df19f2c2abb042398dcd1128119a9fac'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:50.750872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d1ed95c6-c21e-4683-b409-555b1560dcb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df19f2c2abb042398dcd1128119a9fac'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:50.761281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd05662-30f2-47f3-b221-9957d953bdc1 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:50.763071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-acd05662-30f2-47f3-b221-9957d953bdc1 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:50.772595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d1ed95c6-c21e-4683-b409-555b1560dcb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df19f2c2abb042398dcd1128119a9fac'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:50.784418 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3498/6931] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:50 2022] DELETE /identity/v3/users/df19f2c2abb042398dcd1128119a9fac => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:50.786703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3edaf-d646-4492-99a4-7ff4b0a28b29 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:50.787282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3edaf-d646-4492-99a4-7ff4b0a28b29 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:50.787398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3edaf-d646-4492-99a4-7ff4b0a28b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:50.826372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-afd3edaf-d646-4492-99a4-7ff4b0a28b29 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:50.827796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-afd3edaf-d646-4492-99a4-7ff4b0a28b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:50.907831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd05662-30f2-47f3-b221-9957d953bdc1 tempest-UsersV3TestJSON-1104156543 tempest-UsersV3TestJSON-1104156543-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:50.913835 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3499/6932] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:50.923034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd3edaf-d646-4492-99a4-7ff4b0a28b29 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:50.925679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-afd3edaf-d646-4492-99a4-7ff4b0a28b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:50.988619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:51.012735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638062f9-d5d8-488b-a630-45f1a27702d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:51.013461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638062f9-d5d8-488b-a630-45f1a27702d9 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:51.013751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638062f9-d5d8-488b-a630-45f1a27702d9 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:51.014031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638062f9-d5d8-488b-a630-45f1a27702d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:51.016186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-638062f9-d5d8-488b-a630-45f1a27702d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:51.055379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638062f9-d5d8-488b-a630-45f1a27702d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:51.057092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638062f9-d5d8-488b-a630-45f1a27702d9 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:51.075241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd3edaf-d646-4492-99a4-7ff4b0a28b29 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:51.083403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12d74a-9789-4f50-b247-85b70f0bd091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:51.084031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12d74a-9789-4f50-b247-85b70f0bd091 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:51.084344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12d74a-9789-4f50-b247-85b70f0bd091 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:51.085195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12d74a-9789-4f50-b247-85b70f0bd091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:51.085849 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3434/6933] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:14:51.096413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12d74a-9789-4f50-b247-85b70f0bd091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:51.098243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12d74a-9789-4f50-b247-85b70f0bd091 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:51.110350 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3500/6934] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:14:51.533344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638062f9-d5d8-488b-a630-45f1a27702d9 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:51.534909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-638062f9-d5d8-488b-a630-45f1a27702d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:51.555508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638062f9-d5d8-488b-a630-45f1a27702d9 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:51.560968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb1b63-98c3-4158-bfca-1821e3ed6f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:51.561777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb1b63-98c3-4158-bfca-1821e3ed6f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:51.562327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb1b63-98c3-4158-bfca-1821e3ed6f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:51.562762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb1b63-98c3-4158-bfca-1821e3ed6f33 admin admin] PATH_INFO: `/v3/projects/f88f20f15e4b4718b1650395982b5daa` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:51.564390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb1b63-98c3-4158-bfca-1821e3ed6f33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f88f20f15e4b4718b1650395982b5daa)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:51.566544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb1b63-98c3-4158-bfca-1821e3ed6f33 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:51.571541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12d74a-9789-4f50-b247-85b70f0bd091 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:51.573066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bd12d74a-9789-4f50-b247-85b70f0bd091 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:51.656414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:51.672029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b0cb1b63-98c3-4158-bfca-1821e3ed6f33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f88f20f15e4b4718b1650395982b5daa', 'request_id': 'req-b0cb1b63-98c3-4158-bfca-1821e3ed6f33'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:51.693390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b0cb1b63-98c3-4158-bfca-1821e3ed6f33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f88f20f15e4b4718b1650395982b5daa', 'request_id': 'req-b0cb1b63-98c3-4158-bfca-1821e3ed6f33'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:51.708412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b0cb1b63-98c3-4158-bfca-1821e3ed6f33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f88f20f15e4b4718b1650395982b5daa', 'request_id': 'req-b0cb1b63-98c3-4158-bfca-1821e3ed6f33'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:51.739923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b0cb1b63-98c3-4158-bfca-1821e3ed6f33 admin admin] The token cache is being invalidate because project f88f20f15e4b4718b1650395982b5daa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:51.740222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b0cb1b63-98c3-4158-bfca-1821e3ed6f33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:51.748497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b0cb1b63-98c3-4158-bfca-1821e3ed6f33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:51.751334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12d74a-9789-4f50-b247-85b70f0bd091 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:51.759047 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3435/6935] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:51 2022] DELETE /identity/v3/projects/f88f20f15e4b4718b1650395982b5daa => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:51.760181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb1b63-98c3-4158-bfca-1821e3ed6f33 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:51.764606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b0cb1b63-98c3-4158-bfca-1821e3ed6f33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:51.910995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb1b63-98c3-4158-bfca-1821e3ed6f33 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:51.916295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379590b-a024-498a-a942-0298ba0d1e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:51.917114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379590b-a024-498a-a942-0298ba0d1e7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:51.917399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379590b-a024-498a-a942-0298ba0d1e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:51.917809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379590b-a024-498a-a942-0298ba0d1e7e admin admin] PATH_INFO: `/v3/users/9bc9cad00323458b8a895a12ed46dd87` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:51.919141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d379590b-a024-498a-a942-0298ba0d1e7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bc9cad00323458b8a895a12ed46dd87)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:51.921320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d379590b-a024-498a-a942-0298ba0d1e7e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:51.923689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2de8e9-04f3-4e98-bf54-9a6e2b8dbcd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:51.924401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2de8e9-04f3-4e98-bf54-9a6e2b8dbcd9 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:51.924692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2de8e9-04f3-4e98-bf54-9a6e2b8dbcd9 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:51.924972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2de8e9-04f3-4e98-bf54-9a6e2b8dbcd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:51.926917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4b2de8e9-04f3-4e98-bf54-9a6e2b8dbcd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:52.020720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d379590b-a024-498a-a942-0298ba0d1e7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bc9cad00323458b8a895a12ed46dd87'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:52.022279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d379590b-a024-498a-a942-0298ba0d1e7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bc9cad00323458b8a895a12ed46dd87'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:52.023388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d379590b-a024-498a-a942-0298ba0d1e7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bc9cad00323458b8a895a12ed46dd87'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:52.038103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d379590b-a024-498a-a942-0298ba0d1e7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bc9cad00323458b8a895a12ed46dd87'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:52.049270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2de8e9-04f3-4e98-bf54-9a6e2b8dbcd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:52.051502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2de8e9-04f3-4e98-bf54-9a6e2b8dbcd9 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:52.054641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:52.065581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d379590b-a024-498a-a942-0298ba0d1e7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bc9cad00323458b8a895a12ed46dd87'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:52.065720 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3501/6936] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:14:52.066516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2de8e9-04f3-4e98-bf54-9a6e2b8dbcd9 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:52.068195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4b2de8e9-04f3-4e98-bf54-9a6e2b8dbcd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:52.102533 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3436/6937] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:51 2022] DELETE /identity/v3/users/9bc9cad00323458b8a895a12ed46dd87 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:52.159752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2de8e9-04f3-4e98-bf54-9a6e2b8dbcd9 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:52.192986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:52.223668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0153a-f451-48c8-a804-bce85d1edf36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:52.224289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0153a-f451-48c8-a804-bce85d1edf36 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:52.224570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0153a-f451-48c8-a804-bce85d1edf36 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:52.224844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0153a-f451-48c8-a804-bce85d1edf36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:52.227469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-63c0153a-f451-48c8-a804-bce85d1edf36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:52.308837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0153a-f451-48c8-a804-bce85d1edf36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:52.310353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0153a-f451-48c8-a804-bce85d1edf36 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:52.323435 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3502/6938] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:52.721961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379590b-a024-498a-a942-0298ba0d1e7e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:52.723997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d379590b-a024-498a-a942-0298ba0d1e7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:52.827617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379590b-a024-498a-a942-0298ba0d1e7e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:52.832580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c93239-72dd-439f-9bbd-8a9b66f5298d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:52.833225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c93239-72dd-439f-9bbd-8a9b66f5298d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:52.833504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c93239-72dd-439f-9bbd-8a9b66f5298d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:52.833778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c93239-72dd-439f-9bbd-8a9b66f5298d admin admin] PATH_INFO: `/v3/projects/97b9ba86b6fc4899ac71ee9a664cc9bc` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:52.834966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c93239-72dd-439f-9bbd-8a9b66f5298d admin admin] RBAC: Authorizing `identity:delete_project(project_id=97b9ba86b6fc4899ac71ee9a664cc9bc)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:52.837985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c93239-72dd-439f-9bbd-8a9b66f5298d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:52.896045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d9c93239-72dd-439f-9bbd-8a9b66f5298d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97b9ba86b6fc4899ac71ee9a664cc9bc', 'request_id': 'req-d9c93239-72dd-439f-9bbd-8a9b66f5298d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:52.913048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d9c93239-72dd-439f-9bbd-8a9b66f5298d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97b9ba86b6fc4899ac71ee9a664cc9bc', 'request_id': 'req-d9c93239-72dd-439f-9bbd-8a9b66f5298d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:52.920865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d9c93239-72dd-439f-9bbd-8a9b66f5298d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97b9ba86b6fc4899ac71ee9a664cc9bc', 'request_id': 'req-d9c93239-72dd-439f-9bbd-8a9b66f5298d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:52.934692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d9c93239-72dd-439f-9bbd-8a9b66f5298d admin admin] The token cache is being invalidate because project 97b9ba86b6fc4899ac71ee9a664cc9bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:52.935094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d9c93239-72dd-439f-9bbd-8a9b66f5298d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:52.935829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d9c93239-72dd-439f-9bbd-8a9b66f5298d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:52.942175 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3437/6939] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:52 2022] DELETE /identity/v3/projects/97b9ba86b6fc4899ac71ee9a664cc9bc => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:52.944122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be6a9f-8d20-4151-bdeb-915770c35f0d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:52.944642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be6a9f-8d20-4151-bdeb-915770c35f0d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:52.944642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be6a9f-8d20-4151-bdeb-915770c35f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:52.972065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-18be6a9f-8d20-4151-bdeb-915770c35f0d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:52.973354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-18be6a9f-8d20-4151-bdeb-915770c35f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:53.096338 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3503/6940] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:53.109864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f242fd4-cd88-4436-98f0-aaafa2b07a9f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:53.110531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f242fd4-cd88-4436-98f0-aaafa2b07a9f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:53.111791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f242fd4-cd88-4436-98f0-aaafa2b07a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:53.161741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5f242fd4-cd88-4436-98f0-aaafa2b07a9f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:53.164576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5f242fd4-cd88-4436-98f0-aaafa2b07a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:53.184694 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3438/6941] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:53.189901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be6a9f-8d20-4151-bdeb-915770c35f0d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:53.202230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be6a9f-8d20-4151-bdeb-915770c35f0d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:53.207456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65823c07-791c-45d7-960e-8b80e5f33f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:53.208082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65823c07-791c-45d7-960e-8b80e5f33f3b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:53.208430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65823c07-791c-45d7-960e-8b80e5f33f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:53.208715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65823c07-791c-45d7-960e-8b80e5f33f3b admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:53.211147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65823c07-791c-45d7-960e-8b80e5f33f3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:14:53.211493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65823c07-791c-45d7-960e-8b80e5f33f3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:53.213427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65823c07-791c-45d7-960e-8b80e5f33f3b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:53.234014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f242fd4-cd88-4436-98f0-aaafa2b07a9f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:53.234242 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3504/6942] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:14:53.243861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f242fd4-cd88-4436-98f0-aaafa2b07a9f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:53.249034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c3c23-963a-47fc-8818-3654f19e3e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:53.249721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c3c23-963a-47fc-8818-3654f19e3e91 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:53.250074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c3c23-963a-47fc-8818-3654f19e3e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:53.250357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c3c23-963a-47fc-8818-3654f19e3e91 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:53.251499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c3c23-963a-47fc-8818-3654f19e3e91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:53.253428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c3c23-963a-47fc-8818-3654f19e3e91 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:53.302040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65823c07-791c-45d7-960e-8b80e5f33f3b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:53.302971 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3439/6943] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:53 2022] POST /identity/v3/projects => generated 391 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:53.313286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65823c07-791c-45d7-960e-8b80e5f33f3b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:53.378872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40273ce8-d60d-49ae-9e54-2de68a3561de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:53.379541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40273ce8-d60d-49ae-9e54-2de68a3561de admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:53.379827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40273ce8-d60d-49ae-9e54-2de68a3561de admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:53.380108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40273ce8-d60d-49ae-9e54-2de68a3561de admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:53.380744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40273ce8-d60d-49ae-9e54-2de68a3561de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:53.383337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40273ce8-d60d-49ae-9e54-2de68a3561de admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:53.415081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:53.428759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c3c23-963a-47fc-8818-3654f19e3e91 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:53.429108 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3505/6944] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:53 2022] POST /identity/v3/users => generated 302 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:53.443115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c3c23-963a-47fc-8818-3654f19e3e91 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:53.450543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f769810-5cf9-466e-87ab-28c7784b4f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:53.451446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f769810-5cf9-466e-87ab-28c7784b4f96 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:53.451813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f769810-5cf9-466e-87ab-28c7784b4f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:53.452004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f769810-5cf9-466e-87ab-28c7784b4f96 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:53.453021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f769810-5cf9-466e-87ab-28c7784b4f96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:53.454747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f769810-5cf9-466e-87ab-28c7784b4f96 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:53.468202 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3440/6945] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:53.472421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40273ce8-d60d-49ae-9e54-2de68a3561de admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:53.487067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40273ce8-d60d-49ae-9e54-2de68a3561de admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:53.493686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb30920-24ea-437d-af90-8dc42559c145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:53.494363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb30920-24ea-437d-af90-8dc42559c145 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:53.494689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb30920-24ea-437d-af90-8dc42559c145 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:53.494971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb30920-24ea-437d-af90-8dc42559c145 admin admin] PATH_INFO: `/v3/projects/dda263d940fd48a8985ba2af5ca3331d/users/be47e96378bd4afaa5876c16f5d02b02/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:53.513123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb30920-24ea-437d-af90-8dc42559c145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dda263d940fd48a8985ba2af5ca3331d, user_id=be47e96378bd4afaa5876c16f5d02b02, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:53.518239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb30920-24ea-437d-af90-8dc42559c145 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:53.544900 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3506/6946] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:53 2022] PUT /identity/v3/projects/dda263d940fd48a8985ba2af5ca3331d/users/be47e96378bd4afaa5876c16f5d02b02/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:53.549355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f769810-5cf9-466e-87ab-28c7784b4f96 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:53.590118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f769810-5cf9-466e-87ab-28c7784b4f96 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:53.642636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:53.689423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a155f-b0d7-4469-9ec3-0fe2b0228c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:53.690170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a155f-b0d7-4469-9ec3-0fe2b0228c44 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:53.690575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a155f-b0d7-4469-9ec3-0fe2b0228c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:53.690950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a155f-b0d7-4469-9ec3-0fe2b0228c44 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:53.693327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a155f-b0d7-4469-9ec3-0fe2b0228c44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:53.695930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a155f-b0d7-4469-9ec3-0fe2b0228c44 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:53.710938 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3441/6947] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:53.712704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb30920-24ea-437d-af90-8dc42559c145 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:53.731083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb30920-24ea-437d-af90-8dc42559c145 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:53.736421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f105b-100b-4c34-a75e-369c748484ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:53.737194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f105b-100b-4c34-a75e-369c748484ce admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:53.737573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f105b-100b-4c34-a75e-369c748484ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:53.738985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f105b-100b-4c34-a75e-369c748484ce admin admin] PATH_INFO: `/v3/projects/dda263d940fd48a8985ba2af5ca3331d/users/be47e96378bd4afaa5876c16f5d02b02/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:53.740505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f105b-100b-4c34-a75e-369c748484ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=dda263d940fd48a8985ba2af5ca3331d, user_id=be47e96378bd4afaa5876c16f5d02b02, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:53.743716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f105b-100b-4c34-a75e-369c748484ce admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:53.757931 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3507/6948] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:53 2022] PUT /identity/v3/projects/dda263d940fd48a8985ba2af5ca3331d/users/be47e96378bd4afaa5876c16f5d02b02/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:53.767908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee7d00-a69c-453f-b4bb-cc726235370b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:53.768426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee7d00-a69c-453f-b4bb-cc726235370b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:53.768733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee7d00-a69c-453f-b4bb-cc726235370b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:53.799719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a9ee7d00-a69c-453f-b4bb-cc726235370b None None] MFA Rules not processed for user `be47e96378bd4afaa5876c16f5d02b02`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:53.801125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a9ee7d00-a69c-453f-b4bb-cc726235370b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:53.890245 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3442/6949] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:53 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:53.894938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f105b-100b-4c34-a75e-369c748484ce admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:54.018022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f105b-100b-4c34-a75e-369c748484ce admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:54.028248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f5cde-191b-43f6-bf20-b52d702ef83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:54.029175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f5cde-191b-43f6-bf20-b52d702ef83b admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:54.029612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f5cde-191b-43f6-bf20-b52d702ef83b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:54.030465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f5cde-191b-43f6-bf20-b52d702ef83b admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:54.035399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f5cde-191b-43f6-bf20-b52d702ef83b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:54.039569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f5cde-191b-43f6-bf20-b52d702ef83b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:54.090717 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3508/6950] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:53 2022] POST /identity/v3/projects => generated 395 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:54.095366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee7d00-a69c-453f-b4bb-cc726235370b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:54.108963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee7d00-a69c-453f-b4bb-cc726235370b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:54.203797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f9e47-4680-43d7-90bf-313572c5a944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:54.204959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f9e47-4680-43d7-90bf-313572c5a944 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:54.205450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f9e47-4680-43d7-90bf-313572c5a944 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:54.205951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f9e47-4680-43d7-90bf-313572c5a944 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:54.206867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f9e47-4680-43d7-90bf-313572c5a944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:54.209084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f9e47-4680-43d7-90bf-313572c5a944 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:54.264605 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3443/6951] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:54 2022] POST /identity/v3/users => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:54.265949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f5cde-191b-43f6-bf20-b52d702ef83b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:54.278749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f5cde-191b-43f6-bf20-b52d702ef83b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:54.283738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81808def-a0a0-4c36-a7bd-d35128792a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:54.284364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81808def-a0a0-4c36-a7bd-d35128792a5a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:54.284650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81808def-a0a0-4c36-a7bd-d35128792a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:54.285220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81808def-a0a0-4c36-a7bd-d35128792a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:54.285887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81808def-a0a0-4c36-a7bd-d35128792a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:54.287890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81808def-a0a0-4c36-a7bd-d35128792a5a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:54.325602 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3509/6952] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:54.326830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f9e47-4680-43d7-90bf-313572c5a944 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:54.342333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f9e47-4680-43d7-90bf-313572c5a944 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:54.352469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0f19d-4e5c-46cc-9952-f74c3e27396b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:54.353088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0f19d-4e5c-46cc-9952-f74c3e27396b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:54.353369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0f19d-4e5c-46cc-9952-f74c3e27396b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:54.353650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0f19d-4e5c-46cc-9952-f74c3e27396b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:54.354294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0f19d-4e5c-46cc-9952-f74c3e27396b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:54.356187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0f19d-4e5c-46cc-9952-f74c3e27396b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:54.371151 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3444/6953] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:54.372165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81808def-a0a0-4c36-a7bd-d35128792a5a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:54.384306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81808def-a0a0-4c36-a7bd-d35128792a5a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:54.395540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa931bb-345a-4cc1-8401-bf70f22a5e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:54.396459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa931bb-345a-4cc1-8401-bf70f22a5e93 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:54.396919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa931bb-345a-4cc1-8401-bf70f22a5e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:54.397303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa931bb-345a-4cc1-8401-bf70f22a5e93 admin admin] PATH_INFO: `/v3/projects/dccc170747824c19ab6af6b8460d03c5/users/44e0a163496c4a67819944a50375ee86/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:54.421314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa931bb-345a-4cc1-8401-bf70f22a5e93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dccc170747824c19ab6af6b8460d03c5, user_id=44e0a163496c4a67819944a50375ee86, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:54.424493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa931bb-345a-4cc1-8401-bf70f22a5e93 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:54.452296 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3510/6954] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:54 2022] PUT /identity/v3/projects/dccc170747824c19ab6af6b8460d03c5/users/44e0a163496c4a67819944a50375ee86/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:54.453059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af277c5-4d6b-4333-85de-19bb7ca87062 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:54.453417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af277c5-4d6b-4333-85de-19bb7ca87062 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:54.453767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af277c5-4d6b-4333-85de-19bb7ca87062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:54.494368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8af277c5-4d6b-4333-85de-19bb7ca87062 None None] MFA Rules not processed for user `44e0a163496c4a67819944a50375ee86`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:54.495380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8af277c5-4d6b-4333-85de-19bb7ca87062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:54.516949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:54.575894 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3445/6955] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:54 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:54.589183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa931bb-345a-4cc1-8401-bf70f22a5e93 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:54.591041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1aa931bb-345a-4cc1-8401-bf70f22a5e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:54.720355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa931bb-345a-4cc1-8401-bf70f22a5e93 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:54.724417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db892360-095b-4ca2-935a-cb8a9c06d4a6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:54.725180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db892360-095b-4ca2-935a-cb8a9c06d4a6 service glance] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:54.725511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db892360-095b-4ca2-935a-cb8a9c06d4a6 service glance] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:54.725858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db892360-095b-4ca2-935a-cb8a9c06d4a6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:54.766413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:54.848154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db892360-095b-4ca2-935a-cb8a9c06d4a6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:54.849707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db892360-095b-4ca2-935a-cb8a9c06d4a6 service glance] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:54.880682 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3511/6956] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:54 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:55.462531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af277c5-4d6b-4333-85de-19bb7ca87062 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:55.548846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af277c5-4d6b-4333-85de-19bb7ca87062 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:55.555042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfaedb-57fc-4ab8-a5da-484a52f0ce2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:55.556076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfaedb-57fc-4ab8-a5da-484a52f0ce2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:55.556430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfaedb-57fc-4ab8-a5da-484a52f0ce2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:55.556773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfaedb-57fc-4ab8-a5da-484a52f0ce2d admin admin] PATH_INFO: `/v3/users/be47e96378bd4afaa5876c16f5d02b02` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:55.558159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfaedb-57fc-4ab8-a5da-484a52f0ce2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=be47e96378bd4afaa5876c16f5d02b02)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:55.560266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfaedb-57fc-4ab8-a5da-484a52f0ce2d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:55.625902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e8cfaedb-57fc-4ab8-a5da-484a52f0ce2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be47e96378bd4afaa5876c16f5d02b02'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:55.626654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e8cfaedb-57fc-4ab8-a5da-484a52f0ce2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be47e96378bd4afaa5876c16f5d02b02'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:55.627275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e8cfaedb-57fc-4ab8-a5da-484a52f0ce2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be47e96378bd4afaa5876c16f5d02b02'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:55.639487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e8cfaedb-57fc-4ab8-a5da-484a52f0ce2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be47e96378bd4afaa5876c16f5d02b02'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:55.658708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e8cfaedb-57fc-4ab8-a5da-484a52f0ce2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be47e96378bd4afaa5876c16f5d02b02'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:55.674918 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3446/6957] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:55 2022] DELETE /identity/v3/users/be47e96378bd4afaa5876c16f5d02b02 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:55.679567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2cdbc-df37-4ca8-a0cc-eeb2a92ec296 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:55.680041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2cdbc-df37-4ca8-a0cc-eeb2a92ec296 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:55.680121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2cdbc-df37-4ca8-a0cc-eeb2a92ec296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:55.705213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3ae2cdbc-df37-4ca8-a0cc-eeb2a92ec296 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:55.707925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3ae2cdbc-df37-4ca8-a0cc-eeb2a92ec296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:55.844930 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3512/6958] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:55.923143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfaedb-57fc-4ab8-a5da-484a52f0ce2d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:55.924843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e8cfaedb-57fc-4ab8-a5da-484a52f0ce2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:55.976380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:56.000272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfaedb-57fc-4ab8-a5da-484a52f0ce2d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:56.023874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35510590-2664-47a0-8a9d-63b9c1eb04ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:56.026198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35510590-2664-47a0-8a9d-63b9c1eb04ad service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:56.027034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35510590-2664-47a0-8a9d-63b9c1eb04ad service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:56.027542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35510590-2664-47a0-8a9d-63b9c1eb04ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:56.066171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35510590-2664-47a0-8a9d-63b9c1eb04ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:56.072781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35510590-2664-47a0-8a9d-63b9c1eb04ad service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:56.095094 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3447/6959] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:56.925170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2cdbc-df37-4ca8-a0cc-eeb2a92ec296 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:56.926782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3ae2cdbc-df37-4ca8-a0cc-eeb2a92ec296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:56.941046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2cdbc-df37-4ca8-a0cc-eeb2a92ec296 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:56.947344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c16a3-95d1-4344-a2b4-300f19bd9a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:56.948110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c16a3-95d1-4344-a2b4-300f19bd9a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:56.948536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c16a3-95d1-4344-a2b4-300f19bd9a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:56.949032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c16a3-95d1-4344-a2b4-300f19bd9a37 admin admin] PATH_INFO: `/v3/projects/dda263d940fd48a8985ba2af5ca3331d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:56.950343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c16a3-95d1-4344-a2b4-300f19bd9a37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dda263d940fd48a8985ba2af5ca3331d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:56.952848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c16a3-95d1-4344-a2b4-300f19bd9a37 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:57.038312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8d1c16a3-95d1-4344-a2b4-300f19bd9a37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dda263d940fd48a8985ba2af5ca3331d', 'request_id': 'req-8d1c16a3-95d1-4344-a2b4-300f19bd9a37'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:57.058719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8d1c16a3-95d1-4344-a2b4-300f19bd9a37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dda263d940fd48a8985ba2af5ca3331d', 'request_id': 'req-8d1c16a3-95d1-4344-a2b4-300f19bd9a37'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:57.068409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8d1c16a3-95d1-4344-a2b4-300f19bd9a37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dda263d940fd48a8985ba2af5ca3331d', 'request_id': 'req-8d1c16a3-95d1-4344-a2b4-300f19bd9a37'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:57.087940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8d1c16a3-95d1-4344-a2b4-300f19bd9a37 admin admin] The token cache is being invalidate because project dda263d940fd48a8985ba2af5ca3331d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:57.088279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8d1c16a3-95d1-4344-a2b4-300f19bd9a37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:57.088857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8d1c16a3-95d1-4344-a2b4-300f19bd9a37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:57.090517 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3513/6960] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:56 2022] DELETE /identity/v3/projects/dda263d940fd48a8985ba2af5ca3331d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:57.094555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35510590-2664-47a0-8a9d-63b9c1eb04ad service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:57.109036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35510590-2664-47a0-8a9d-63b9c1eb04ad service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:57.254563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8878b-ca30-41e4-a60d-100b5ffac89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:57.255821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8878b-ca30-41e4-a60d-100b5ffac89f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:57.256777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8878b-ca30-41e4-a60d-100b5ffac89f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:57.257403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8878b-ca30-41e4-a60d-100b5ffac89f admin admin] PATH_INFO: `/v3/users/44e0a163496c4a67819944a50375ee86` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:57.262737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8878b-ca30-41e4-a60d-100b5ffac89f admin admin] RBAC: Authorizing `identity:delete_user(user_id=44e0a163496c4a67819944a50375ee86)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:57.265602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8878b-ca30-41e4-a60d-100b5ffac89f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:57.373639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a3b8878b-ca30-41e4-a60d-100b5ffac89f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44e0a163496c4a67819944a50375ee86'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:57.374343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a3b8878b-ca30-41e4-a60d-100b5ffac89f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44e0a163496c4a67819944a50375ee86'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:57.375214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a3b8878b-ca30-41e4-a60d-100b5ffac89f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44e0a163496c4a67819944a50375ee86'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:57.396095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a3b8878b-ca30-41e4-a60d-100b5ffac89f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44e0a163496c4a67819944a50375ee86'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:57.421032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a3b8878b-ca30-41e4-a60d-100b5ffac89f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44e0a163496c4a67819944a50375ee86'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:57.441661 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3448/6961] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:57 2022] DELETE /identity/v3/users/44e0a163496c4a67819944a50375ee86 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:57.960163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c16a3-95d1-4344-a2b4-300f19bd9a37 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:57.961723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8d1c16a3-95d1-4344-a2b4-300f19bd9a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:57.992478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c16a3-95d1-4344-a2b4-300f19bd9a37 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:58.043810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:58.068598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accddde9-3d31-4010-b022-6f8f085763c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:58.070181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accddde9-3d31-4010-b022-6f8f085763c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:58.070941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accddde9-3d31-4010-b022-6f8f085763c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:58.071687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accddde9-3d31-4010-b022-6f8f085763c3 admin admin] PATH_INFO: `/v3/projects/dccc170747824c19ab6af6b8460d03c5` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:58.073890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accddde9-3d31-4010-b022-6f8f085763c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dccc170747824c19ab6af6b8460d03c5)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:58.076226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accddde9-3d31-4010-b022-6f8f085763c3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:58.143020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8878b-ca30-41e4-a60d-100b5ffac89f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:58.144988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a3b8878b-ca30-41e4-a60d-100b5ffac89f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:58.147350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-accddde9-3d31-4010-b022-6f8f085763c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dccc170747824c19ab6af6b8460d03c5', 'request_id': 'req-accddde9-3d31-4010-b022-6f8f085763c3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:58.155446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-accddde9-3d31-4010-b022-6f8f085763c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dccc170747824c19ab6af6b8460d03c5', 'request_id': 'req-accddde9-3d31-4010-b022-6f8f085763c3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:58.161630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-accddde9-3d31-4010-b022-6f8f085763c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dccc170747824c19ab6af6b8460d03c5', 'request_id': 'req-accddde9-3d31-4010-b022-6f8f085763c3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:58.165117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8878b-ca30-41e4-a60d-100b5ffac89f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:58.186949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-accddde9-3d31-4010-b022-6f8f085763c3 admin admin] The token cache is being invalidate because project dccc170747824c19ab6af6b8460d03c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:58.187243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-accddde9-3d31-4010-b022-6f8f085763c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:58.188039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-accddde9-3d31-4010-b022-6f8f085763c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:58.189705 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3514/6962] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:57 2022] DELETE /identity/v3/projects/dccc170747824c19ab6af6b8460d03c5 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:58.194608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8639fe-e2a3-4476-8ba8-25541efcb134 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:58.194900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8639fe-e2a3-4476-8ba8-25541efcb134 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:58.195176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8639fe-e2a3-4476-8ba8-25541efcb134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:58.217078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ed8639fe-e2a3-4476-8ba8-25541efcb134 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:58.218682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ed8639fe-e2a3-4476-8ba8-25541efcb134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:58.233965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bd2af-8917-4581-a759-ca11c9a996f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:58.234772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bd2af-8917-4581-a759-ca11c9a996f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:58.235098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bd2af-8917-4581-a759-ca11c9a996f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:58.235808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bd2af-8917-4581-a759-ca11c9a996f2 admin admin] PATH_INFO: `/v3/users/d9693d6502f34c7aa0604f840abb2a56` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:58.237169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28bd2af-8917-4581-a759-ca11c9a996f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9693d6502f34c7aa0604f840abb2a56)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:58.239282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28bd2af-8917-4581-a759-ca11c9a996f2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:58.242896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:58.297821 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3515/6963] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:58.306321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b28bd2af-8917-4581-a759-ca11c9a996f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9693d6502f34c7aa0604f840abb2a56'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:58.307131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b28bd2af-8917-4581-a759-ca11c9a996f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9693d6502f34c7aa0604f840abb2a56'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:58.308196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b28bd2af-8917-4581-a759-ca11c9a996f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9693d6502f34c7aa0604f840abb2a56'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:58.308415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625e6dd-f52a-4d53-abc0-d07d7e5c83c1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:58.308751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625e6dd-f52a-4d53-abc0-d07d7e5c83c1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:58.309104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625e6dd-f52a-4d53-abc0-d07d7e5c83c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:58.318940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b28bd2af-8917-4581-a759-ca11c9a996f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9693d6502f34c7aa0604f840abb2a56'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:58.333007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b28bd2af-8917-4581-a759-ca11c9a996f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9693d6502f34c7aa0604f840abb2a56'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:58.339573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f625e6dd-f52a-4d53-abc0-d07d7e5c83c1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:58.340820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f625e6dd-f52a-4d53-abc0-d07d7e5c83c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:58.353225 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3516/6964] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:58.354305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625e6dd-f52a-4d53-abc0-d07d7e5c83c1 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:58.356996 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3449/6965] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:14:58 2022] DELETE /identity/v3/users/d9693d6502f34c7aa0604f840abb2a56 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:58.361661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b331dbf-aa53-4b6b-aa5d-3ea2b7f83d78 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:58.362631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b331dbf-aa53-4b6b-aa5d-3ea2b7f83d78 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:58.368723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b331dbf-aa53-4b6b-aa5d-3ea2b7f83d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:58.372809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625e6dd-f52a-4d53-abc0-d07d7e5c83c1 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:58.393158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3b331dbf-aa53-4b6b-aa5d-3ea2b7f83d78 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:58.394522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3b331dbf-aa53-4b6b-aa5d-3ea2b7f83d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:58.456626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0efe2-6f78-4d8d-9613-c38971233fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:58.456756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0efe2-6f78-4d8d-9613-c38971233fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:58.457695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0efe2-6f78-4d8d-9613-c38971233fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:58.458011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0efe2-6f78-4d8d-9613-c38971233fc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:58.458749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0efe2-6f78-4d8d-9613-c38971233fc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:14:58.459594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0efe2-6f78-4d8d-9613-c38971233fc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:58.461964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0efe2-6f78-4d8d-9613-c38971233fc0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:58.476516 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3517/6966] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:14:58.481228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0efe2-6f78-4d8d-9613-c38971233fc0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:58.490679 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3450/6967] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:58.493960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:58.503377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0efe2-6f78-4d8d-9613-c38971233fc0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:58.506749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b331dbf-aa53-4b6b-aa5d-3ea2b7f83d78 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:58.508634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3b331dbf-aa53-4b6b-aa5d-3ea2b7f83d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:58.510019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c089cdc-f9ac-4231-9f11-cf06506b2ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:58.510827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c089cdc-f9ac-4231-9f11-cf06506b2ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:58.511476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c089cdc-f9ac-4231-9f11-cf06506b2ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:58.512072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c089cdc-f9ac-4231-9f11-cf06506b2ec3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:58.512947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c089cdc-f9ac-4231-9f11-cf06506b2ec3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:58.515656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c089cdc-f9ac-4231-9f11-cf06506b2ec3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:58.572461 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3518/6968] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:58 2022] POST /identity/v3/projects => generated 369 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:14:58.576251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c089cdc-f9ac-4231-9f11-cf06506b2ec3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:58.590190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b331dbf-aa53-4b6b-aa5d-3ea2b7f83d78 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:58.638828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:58.665148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae57b88-7dca-4962-806d-12abc9611b45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:58.666408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae57b88-7dca-4962-806d-12abc9611b45 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:58.667041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae57b88-7dca-4962-806d-12abc9611b45 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:58.667475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae57b88-7dca-4962-806d-12abc9611b45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:58.668950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c089cdc-f9ac-4231-9f11-cf06506b2ec3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:58.676472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f80b05-02ab-4f87-80fe-36908cb4f88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:58.678374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f80b05-02ab-4f87-80fe-36908cb4f88d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:58.678374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f80b05-02ab-4f87-80fe-36908cb4f88d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:58.678926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f80b05-02ab-4f87-80fe-36908cb4f88d admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:58.679897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f80b05-02ab-4f87-80fe-36908cb4f88d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:58.680559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae57b88-7dca-4962-806d-12abc9611b45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:58.682916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae57b88-7dca-4962-806d-12abc9611b45 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:58.683127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f80b05-02ab-4f87-80fe-36908cb4f88d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:58.694404 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3451/6969] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:14:58.715425 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3519/6970] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:58 2022] POST /identity/v3/users => generated 285 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:58.716467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae57b88-7dca-4962-806d-12abc9611b45 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:58.729099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae57b88-7dca-4962-806d-12abc9611b45 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:58.736502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9862da-a4cb-49a0-97f0-d938c6dddff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:58.737107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9862da-a4cb-49a0-97f0-d938c6dddff8 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:58.737393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9862da-a4cb-49a0-97f0-d938c6dddff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:58.737758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9862da-a4cb-49a0-97f0-d938c6dddff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:58.738386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9862da-a4cb-49a0-97f0-d938c6dddff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:58.740107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9862da-a4cb-49a0-97f0-d938c6dddff8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:58.770743 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3452/6971] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:14:58.772474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f80b05-02ab-4f87-80fe-36908cb4f88d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:58.785500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f80b05-02ab-4f87-80fe-36908cb4f88d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:58.792369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce6b32-b110-4a17-8cf7-3ca807754bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:58.793180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce6b32-b110-4a17-8cf7-3ca807754bbf admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:58.793687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce6b32-b110-4a17-8cf7-3ca807754bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:58.793963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce6b32-b110-4a17-8cf7-3ca807754bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:58.794713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce6b32-b110-4a17-8cf7-3ca807754bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:58.797068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce6b32-b110-4a17-8cf7-3ca807754bbf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:58.810850 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3520/6972] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:14:58.815438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9862da-a4cb-49a0-97f0-d938c6dddff8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:58.827538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9862da-a4cb-49a0-97f0-d938c6dddff8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:58.832370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57497c-2ef2-4926-86f4-2e000258a34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:58.833035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57497c-2ef2-4926-86f4-2e000258a34e admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:58.833363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57497c-2ef2-4926-86f4-2e000258a34e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:58.833643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57497c-2ef2-4926-86f4-2e000258a34e admin admin] PATH_INFO: `/v3/projects/64375fec27684654b4f5ff2f2c1cbde0/users/57058bac9d114a5a8551e7551d09921d/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:58.849596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57497c-2ef2-4926-86f4-2e000258a34e admin admin] RBAC: Authorizing `identity:create_grant(project_id=64375fec27684654b4f5ff2f2c1cbde0, user_id=57058bac9d114a5a8551e7551d09921d, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:58.852832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57497c-2ef2-4926-86f4-2e000258a34e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:58.869207 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3453/6973] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:58 2022] PUT /identity/v3/projects/64375fec27684654b4f5ff2f2c1cbde0/users/57058bac9d114a5a8551e7551d09921d/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:58.880456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d780de-42df-4ffb-8486-d4decf28c255 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:58.880857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d780de-42df-4ffb-8486-d4decf28c255 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:58.881263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d780de-42df-4ffb-8486-d4decf28c255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:58.905976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57497c-2ef2-4926-86f4-2e000258a34e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:58.908207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7a57497c-2ef2-4926-86f4-2e000258a34e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:58.916716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-96d780de-42df-4ffb-8486-d4decf28c255 None None] MFA Rules not processed for user `57058bac9d114a5a8551e7551d09921d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:58.918225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-96d780de-42df-4ffb-8486-d4decf28c255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:59.007564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57497c-2ef2-4926-86f4-2e000258a34e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:59.013007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a997f-54eb-49c6-9358-4080e90152e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:59.013647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a997f-54eb-49c6-9358-4080e90152e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:59.013932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a997f-54eb-49c6-9358-4080e90152e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:59.014209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a997f-54eb-49c6-9358-4080e90152e7 admin admin] PATH_INFO: `/v3/projects/d97a25f0219e4b5d9fba692f03ee5a62` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:59.015356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a997f-54eb-49c6-9358-4080e90152e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d97a25f0219e4b5d9fba692f03ee5a62)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:59.018209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a997f-54eb-49c6-9358-4080e90152e7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:59.039258 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3521/6974] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:59.042058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d780de-42df-4ffb-8486-d4decf28c255 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:59.064591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d780de-42df-4ffb-8486-d4decf28c255 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:59.071896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fded3-7257-4d9e-9ce2-63d15c1e2817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:59.072684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fded3-7257-4d9e-9ce2-63d15c1e2817 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:59.072971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fded3-7257-4d9e-9ce2-63d15c1e2817 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:59.073362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fded3-7257-4d9e-9ce2-63d15c1e2817 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:59.074045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fded3-7257-4d9e-9ce2-63d15c1e2817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:59.076030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fded3-7257-4d9e-9ce2-63d15c1e2817 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:59.083106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4a9a997f-54eb-49c6-9358-4080e90152e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd97a25f0219e4b5d9fba692f03ee5a62', 'request_id': 'req-4a9a997f-54eb-49c6-9358-4080e90152e7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:59.091954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4a9a997f-54eb-49c6-9358-4080e90152e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd97a25f0219e4b5d9fba692f03ee5a62', 'request_id': 'req-4a9a997f-54eb-49c6-9358-4080e90152e7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:59.098368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4a9a997f-54eb-49c6-9358-4080e90152e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd97a25f0219e4b5d9fba692f03ee5a62', 'request_id': 'req-4a9a997f-54eb-49c6-9358-4080e90152e7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:59.110445 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3522/6975] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:59 2022] POST /identity/v3/projects => generated 369 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:59.110651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4a9a997f-54eb-49c6-9358-4080e90152e7 admin admin] The token cache is being invalidate because project d97a25f0219e4b5d9fba692f03ee5a62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:14:59.110906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4a9a997f-54eb-49c6-9358-4080e90152e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:59.111693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fded3-7257-4d9e-9ce2-63d15c1e2817 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:59.111821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4a9a997f-54eb-49c6-9358-4080e90152e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:14:59.113270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9a5fded3-7257-4d9e-9ce2-63d15c1e2817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:59.113386 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3454/6976] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:14:58 2022] DELETE /identity/v3/projects/d97a25f0219e4b5d9fba692f03ee5a62 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:59.118512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91dfa2e-1680-482e-a2c3-db5c5f6f86ab None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:59.118809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91dfa2e-1680-482e-a2c3-db5c5f6f86ab None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:59.119101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91dfa2e-1680-482e-a2c3-db5c5f6f86ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:59.141717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f91dfa2e-1680-482e-a2c3-db5c5f6f86ab None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:59.143207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f91dfa2e-1680-482e-a2c3-db5c5f6f86ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:59.149226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:59.176403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:59.189409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fded3-7257-4d9e-9ce2-63d15c1e2817 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:59.196376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82607b25-8c82-406a-a282-7bfe5eaf0843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:59.197049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82607b25-8c82-406a-a282-7bfe5eaf0843 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:59.197903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82607b25-8c82-406a-a282-7bfe5eaf0843 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:59.198182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82607b25-8c82-406a-a282-7bfe5eaf0843 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:59.198854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82607b25-8c82-406a-a282-7bfe5eaf0843 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:59.200890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82607b25-8c82-406a-a282-7bfe5eaf0843 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:59.215378 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3455/6977] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:59.224452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0603e70-9f9d-45d8-b73c-4fd624534e14 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:59.224631 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3523/6978] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:59 2022] POST /identity/v3/users => generated 285 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:59.227545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0603e70-9f9d-45d8-b73c-4fd624534e14 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:59.227831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0603e70-9f9d-45d8-b73c-4fd624534e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:59.228424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82607b25-8c82-406a-a282-7bfe5eaf0843 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:59.238661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82607b25-8c82-406a-a282-7bfe5eaf0843 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:59.244122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79125cf8-8611-4fa3-89a5-1c7b081a0cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:59.244722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79125cf8-8611-4fa3-89a5-1c7b081a0cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:59.245009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79125cf8-8611-4fa3-89a5-1c7b081a0cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:59.245414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79125cf8-8611-4fa3-89a5-1c7b081a0cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:59.246197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79125cf8-8611-4fa3-89a5-1c7b081a0cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:59.247842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79125cf8-8611-4fa3-89a5-1c7b081a0cd0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:59.252187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f0603e70-9f9d-45d8-b73c-4fd624534e14 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:59.253396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f0603e70-9f9d-45d8-b73c-4fd624534e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:59.261410 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3524/6979] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:14:59.262037 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3456/6980] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:59.262463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79125cf8-8611-4fa3-89a5-1c7b081a0cd0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:59.265860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0603e70-9f9d-45d8-b73c-4fd624534e14 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:59.273124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79125cf8-8611-4fa3-89a5-1c7b081a0cd0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:59.274836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0603e70-9f9d-45d8-b73c-4fd624534e14 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:59.277878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bc3d1-2e2c-495d-b0ed-776f448dafad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:59.278462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bc3d1-2e2c-495d-b0ed-776f448dafad admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:59.278782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bc3d1-2e2c-495d-b0ed-776f448dafad admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:59.279057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bc3d1-2e2c-495d-b0ed-776f448dafad admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:59.279413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaed8db-1c34-4b5a-bc42-41dcc26009f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:59.279694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bc3d1-2e2c-495d-b0ed-776f448dafad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:59.279990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaed8db-1c34-4b5a-bc42-41dcc26009f1 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:59.280272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaed8db-1c34-4b5a-bc42-41dcc26009f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:59.280550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaed8db-1c34-4b5a-bc42-41dcc26009f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:59.281187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaed8db-1c34-4b5a-bc42-41dcc26009f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:14:59.281288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bc3d1-2e2c-495d-b0ed-776f448dafad admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:59.281491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaed8db-1c34-4b5a-bc42-41dcc26009f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:59.283117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaed8db-1c34-4b5a-bc42-41dcc26009f1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:59.295494 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3525/6981] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:59.295650 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3457/6982] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:14:59.296671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bc3d1-2e2c-495d-b0ed-776f448dafad admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:59.299220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaed8db-1c34-4b5a-bc42-41dcc26009f1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:59.307330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bc3d1-2e2c-495d-b0ed-776f448dafad admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:59.309221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaed8db-1c34-4b5a-bc42-41dcc26009f1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:59.312079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3fb4a-34de-49dd-883e-1782d65dc4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:59.312665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3fb4a-34de-49dd-883e-1782d65dc4b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:59.312941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3fb4a-34de-49dd-883e-1782d65dc4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:59.313213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3fb4a-34de-49dd-883e-1782d65dc4b5 admin admin] PATH_INFO: `/v3/projects/dbeef91d920246d2a3548469b6ffa22d/users/2fd53be8056140dc8edc89aa80b7ecb5/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:59.314401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca06240-8391-4168-bcf3-cdbbf936a627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:59.315088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca06240-8391-4168-bcf3-cdbbf936a627 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:59.315441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca06240-8391-4168-bcf3-cdbbf936a627 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:59.315784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca06240-8391-4168-bcf3-cdbbf936a627 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:59.316467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca06240-8391-4168-bcf3-cdbbf936a627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:59.318383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca06240-8391-4168-bcf3-cdbbf936a627 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:59.328872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3fb4a-34de-49dd-883e-1782d65dc4b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbeef91d920246d2a3548469b6ffa22d, user_id=2fd53be8056140dc8edc89aa80b7ecb5, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:59.331478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3fb4a-34de-49dd-883e-1782d65dc4b5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:59.344880 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3526/6983] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:59 2022] PUT /identity/v3/projects/dbeef91d920246d2a3548469b6ffa22d/users/2fd53be8056140dc8edc89aa80b7ecb5/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:14:59.347298 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3458/6984] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:59 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:14:59.351678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3fb4a-34de-49dd-883e-1782d65dc4b5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:59.355739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351483e-92d1-475e-9146-8804df83eeee None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:59.356020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351483e-92d1-475e-9146-8804df83eeee None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:59.356298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351483e-92d1-475e-9146-8804df83eeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:59.384716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d351483e-92d1-475e-9146-8804df83eeee None None] MFA Rules not processed for user `2fd53be8056140dc8edc89aa80b7ecb5`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:59.386028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d351483e-92d1-475e-9146-8804df83eeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:59.415827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3fb4a-34de-49dd-883e-1782d65dc4b5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:59.453590 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3459/6985] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:14:59.465850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351483e-92d1-475e-9146-8804df83eeee None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:59.467305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d351483e-92d1-475e-9146-8804df83eeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:59.474361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340fdb8-31de-414d-b372-a283cb52d829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:59.475683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340fdb8-31de-414d-b372-a283cb52d829 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:59.475960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340fdb8-31de-414d-b372-a283cb52d829 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:59.476236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340fdb8-31de-414d-b372-a283cb52d829 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:59.477035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2340fdb8-31de-414d-b372-a283cb52d829 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:59.478847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2340fdb8-31de-414d-b372-a283cb52d829 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:59.501149 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3527/6986] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:59 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:14:59.505396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340fdb8-31de-414d-b372-a283cb52d829 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:59.515733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340fdb8-31de-414d-b372-a283cb52d829 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:59.520363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e0cf2-2172-49bd-9887-7b84aa4e9d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:59.520953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e0cf2-2172-49bd-9887-7b84aa4e9d9a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:59.521230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e0cf2-2172-49bd-9887-7b84aa4e9d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:59.521498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e0cf2-2172-49bd-9887-7b84aa4e9d9a admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:59.522801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e0cf2-2172-49bd-9887-7b84aa4e9d9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:59.524983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e0cf2-2172-49bd-9887-7b84aa4e9d9a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:59.526280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:59.530185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351483e-92d1-475e-9146-8804df83eeee None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:59.534389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c43fd8-3ce3-4ff6-842d-76266a4d9d6b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:59.535028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c43fd8-3ce3-4ff6-842d-76266a4d9d6b service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:59.535320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c43fd8-3ce3-4ff6-842d-76266a4d9d6b service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:59.535579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c43fd8-3ce3-4ff6-842d-76266a4d9d6b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:59.536088 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3528/6987] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:14:59.537401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-09c43fd8-3ce3-4ff6-842d-76266a4d9d6b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:59.540070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e0cf2-2172-49bd-9887-7b84aa4e9d9a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:59.550676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e0cf2-2172-49bd-9887-7b84aa4e9d9a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:59.555530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd37c3-610d-49bb-b2e1-58d22011fde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:59.556129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd37c3-610d-49bb-b2e1-58d22011fde3 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:59.556420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd37c3-610d-49bb-b2e1-58d22011fde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:59.556698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd37c3-610d-49bb-b2e1-58d22011fde3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:59.557388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd37c3-610d-49bb-b2e1-58d22011fde3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:59.559043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd37c3-610d-49bb-b2e1-58d22011fde3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:59.572419 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3529/6988] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:14:59.576501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd37c3-610d-49bb-b2e1-58d22011fde3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:59.581199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:59.587364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd37c3-610d-49bb-b2e1-58d22011fde3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:59.596277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d3ade-a4ec-4fec-8b5e-98cf9f7cb078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:59.597006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d3ade-a4ec-4fec-8b5e-98cf9f7cb078 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:59.597367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d3ade-a4ec-4fec-8b5e-98cf9f7cb078 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:59.597715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d3ade-a4ec-4fec-8b5e-98cf9f7cb078 admin admin] PATH_INFO: `/v3/projects/357f6ddcb8264bf08fe97f45b3ba336c/users/a83c5e2d125e4f89803ec6104f6aed05/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:59.605968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c43fd8-3ce3-4ff6-842d-76266a4d9d6b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:59.607617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c43fd8-3ce3-4ff6-842d-76266a4d9d6b service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:59.616460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d3ade-a4ec-4fec-8b5e-98cf9f7cb078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=357f6ddcb8264bf08fe97f45b3ba336c, user_id=a83c5e2d125e4f89803ec6104f6aed05, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:59.617854 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3460/6989] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:14:59.619186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d3ade-a4ec-4fec-8b5e-98cf9f7cb078 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:59.637571 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3530/6990] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:14:59 2022] PUT /identity/v3/projects/357f6ddcb8264bf08fe97f45b3ba336c/users/a83c5e2d125e4f89803ec6104f6aed05/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:14:59.644189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecb231-fab8-453d-a01f-58f32bd692ff None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:59.644481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecb231-fab8-453d-a01f-58f32bd692ff None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:59.644790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecb231-fab8-453d-a01f-58f32bd692ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:59.676102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f2ecb231-fab8-453d-a01f-58f32bd692ff None None] MFA Rules not processed for user `a83c5e2d125e4f89803ec6104f6aed05`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:14:59.677508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f2ecb231-fab8-453d-a01f-58f32bd692ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:14:59.756663 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3461/6991] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:14:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:14:59.761605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d3ade-a4ec-4fec-8b5e-98cf9f7cb078 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:59.773903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ecb231-fab8-453d-a01f-58f32bd692ff None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:14:59.879725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d3ade-a4ec-4fec-8b5e-98cf9f7cb078 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:59.885978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a7ff1-1e1b-4cb5-8516-d2fa86ce549e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:59.886913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a7ff1-1e1b-4cb5-8516-d2fa86ce549e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:59.887815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a7ff1-1e1b-4cb5-8516-d2fa86ce549e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:59.887815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a7ff1-1e1b-4cb5-8516-d2fa86ce549e admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:59.888678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a7ff1-1e1b-4cb5-8516-d2fa86ce549e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:14:59.895608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a7ff1-1e1b-4cb5-8516-d2fa86ce549e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:14:59.932225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ecb231-fab8-453d-a01f-58f32bd692ff None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:14:59.939967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81ea01-48b5-4d3a-9d49-e5c94544c9d3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:14:59.940577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81ea01-48b5-4d3a-9d49-e5c94544c9d3 service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:14:59.940856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81ea01-48b5-4d3a-9d49-e5c94544c9d3 service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:14:59.941129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81ea01-48b5-4d3a-9d49-e5c94544c9d3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:14:59.943357 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3531/6992] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:14:59 2022] POST /identity/v3/projects => generated 373 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:14:59.947854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a7ff1-1e1b-4cb5-8516-d2fa86ce549e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:00.045260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81ea01-48b5-4d3a-9d49-e5c94544c9d3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:00.049458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81ea01-48b5-4d3a-9d49-e5c94544c9d3 service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:00.062121 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3462/6993] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:14:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:15:00.089878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a7ff1-1e1b-4cb5-8516-d2fa86ce549e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:00.100726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a9379-b042-4456-88f5-ae8adcf5a22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:00.102386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a9379-b042-4456-88f5-ae8adcf5a22e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:00.103534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a9379-b042-4456-88f5-ae8adcf5a22e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:00.106376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a9379-b042-4456-88f5-ae8adcf5a22e admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:00.107923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a9379-b042-4456-88f5-ae8adcf5a22e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:00.110454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a9379-b042-4456-88f5-ae8adcf5a22e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:00.139022 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3532/6994] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:14:59 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:00.143309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81ea01-48b5-4d3a-9d49-e5c94544c9d3 service glance] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:00.162071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81ea01-48b5-4d3a-9d49-e5c94544c9d3 service glance] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:00.168365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a91e85-35de-417c-9f3f-b1ccdde5d3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:00.169707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a91e85-35de-417c-9f3f-b1ccdde5d3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:00.170002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a91e85-35de-417c-9f3f-b1ccdde5d3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:00.170420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a91e85-35de-417c-9f3f-b1ccdde5d3d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:00.171359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a91e85-35de-417c-9f3f-b1ccdde5d3d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:00.174531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a91e85-35de-417c-9f3f-b1ccdde5d3d9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:00.188523 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3463/6995] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:00.193357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a9379-b042-4456-88f5-ae8adcf5a22e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:00.205718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a9379-b042-4456-88f5-ae8adcf5a22e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:00.211975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba7b8dc-ee73-4c84-a80a-7b5178e8a5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:00.212643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7b8dc-ee73-4c84-a80a-7b5178e8a5e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:00.212923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7b8dc-ee73-4c84-a80a-7b5178e8a5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:00.213199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7b8dc-ee73-4c84-a80a-7b5178e8a5e8 admin admin] PATH_INFO: `/v3/projects/eb66dcac57a94081ace839dfddcd1d59/users/6d24598438104e2fb19dc1fff9e7bd26/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:00.230929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba7b8dc-ee73-4c84-a80a-7b5178e8a5e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb66dcac57a94081ace839dfddcd1d59, user_id=6d24598438104e2fb19dc1fff9e7bd26, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:00.233978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba7b8dc-ee73-4c84-a80a-7b5178e8a5e8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:00.253343 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3533/6996] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:00 2022] PUT /identity/v3/projects/eb66dcac57a94081ace839dfddcd1d59/users/6d24598438104e2fb19dc1fff9e7bd26/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:00.255058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a91e85-35de-417c-9f3f-b1ccdde5d3d9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:00.266692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a91e85-35de-417c-9f3f-b1ccdde5d3d9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:00.281051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:00.365061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d9656-edb6-4c05-aedd-b3b927d5f5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:00.365736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d9656-edb6-4c05-aedd-b3b927d5f5cf admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:00.366087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d9656-edb6-4c05-aedd-b3b927d5f5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:00.369566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d9656-edb6-4c05-aedd-b3b927d5f5cf admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:00.370266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d9656-edb6-4c05-aedd-b3b927d5f5cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:00.372237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d9656-edb6-4c05-aedd-b3b927d5f5cf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:00.386263 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3464/6997] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:00.392163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba7b8dc-ee73-4c84-a80a-7b5178e8a5e8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:00.408123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba7b8dc-ee73-4c84-a80a-7b5178e8a5e8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:00.414535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cde5d6-1247-41ec-90b4-ec07b7a010b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:00.416030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cde5d6-1247-41ec-90b4-ec07b7a010b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:00.416501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cde5d6-1247-41ec-90b4-ec07b7a010b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:00.417006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cde5d6-1247-41ec-90b4-ec07b7a010b9 admin admin] PATH_INFO: `/v3/projects/eb66dcac57a94081ace839dfddcd1d59/users/6d24598438104e2fb19dc1fff9e7bd26/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:00.420274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cde5d6-1247-41ec-90b4-ec07b7a010b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb66dcac57a94081ace839dfddcd1d59, user_id=6d24598438104e2fb19dc1fff9e7bd26, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:00.424607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cde5d6-1247-41ec-90b4-ec07b7a010b9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:00.434360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:00.452243 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3534/6998] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:00 2022] PUT /identity/v3/projects/eb66dcac57a94081ace839dfddcd1d59/users/6d24598438104e2fb19dc1fff9e7bd26/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:00.462890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a0624-61db-4a39-adaa-fdd42d5105b0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:00.463343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a0624-61db-4a39-adaa-fdd42d5105b0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:00.464291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a0624-61db-4a39-adaa-fdd42d5105b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:00.495790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ac5a0624-61db-4a39-adaa-fdd42d5105b0 None None] MFA Rules not processed for user `6d24598438104e2fb19dc1fff9e7bd26`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:00.497324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ac5a0624-61db-4a39-adaa-fdd42d5105b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:00.578900 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3465/6999] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:00.592125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cde5d6-1247-41ec-90b4-ec07b7a010b9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:00.593828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-00cde5d6-1247-41ec-90b4-ec07b7a010b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:00.671065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cde5d6-1247-41ec-90b4-ec07b7a010b9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:00.675504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be466f-2237-4a8d-9b67-dfbc7a396e78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:00.676123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be466f-2237-4a8d-9b67-dfbc7a396e78 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:00.676409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be466f-2237-4a8d-9b67-dfbc7a396e78 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:00.676690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be466f-2237-4a8d-9b67-dfbc7a396e78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:00.686954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be466f-2237-4a8d-9b67-dfbc7a396e78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:00.688385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be466f-2237-4a8d-9b67-dfbc7a396e78 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:00.699200 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3535/7000] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:15:01.136969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a0624-61db-4a39-adaa-fdd42d5105b0 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:01.138389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ac5a0624-61db-4a39-adaa-fdd42d5105b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:01.173054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be466f-2237-4a8d-9b67-dfbc7a396e78 service cinder] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:01.234967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a0624-61db-4a39-adaa-fdd42d5105b0 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:01.241776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f847dd-8188-4ed3-babb-24ce6d489846 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:01.242708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f847dd-8188-4ed3-babb-24ce6d489846 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:01.242902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f847dd-8188-4ed3-babb-24ce6d489846 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:01.243391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f847dd-8188-4ed3-babb-24ce6d489846 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:01.247100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-37f847dd-8188-4ed3-babb-24ce6d489846 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:01.270478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be466f-2237-4a8d-9b67-dfbc7a396e78 service cinder] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:01.275761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6e752-0457-4f46-80cb-90330e6bc18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:01.276394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6e752-0457-4f46-80cb-90330e6bc18f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:01.276674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6e752-0457-4f46-80cb-90330e6bc18f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:01.276946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6e752-0457-4f46-80cb-90330e6bc18f admin admin] PATH_INFO: `/v3/users/a83c5e2d125e4f89803ec6104f6aed05` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:01.278081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6e752-0457-4f46-80cb-90330e6bc18f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a83c5e2d125e4f89803ec6104f6aed05)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:01.280662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6e752-0457-4f46-80cb-90330e6bc18f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:01.336103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:01.343695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-47b6e752-0457-4f46-80cb-90330e6bc18f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a83c5e2d125e4f89803ec6104f6aed05'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:01.352739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-47b6e752-0457-4f46-80cb-90330e6bc18f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a83c5e2d125e4f89803ec6104f6aed05'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:01.356110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f847dd-8188-4ed3-babb-24ce6d489846 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:01.358097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f847dd-8188-4ed3-babb-24ce6d489846 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:01.365346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-47b6e752-0457-4f46-80cb-90330e6bc18f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a83c5e2d125e4f89803ec6104f6aed05'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:01.366079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-47b6e752-0457-4f46-80cb-90330e6bc18f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a83c5e2d125e4f89803ec6104f6aed05'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:01.374163 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3466/7001] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:15:01.375823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-47b6e752-0457-4f46-80cb-90330e6bc18f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a83c5e2d125e4f89803ec6104f6aed05'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:01.384343 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3536/7002] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:01 2022] DELETE /identity/v3/users/a83c5e2d125e4f89803ec6104f6aed05 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:01.389494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d3675-4084-47f2-8a09-1d5f578d3211 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:01.390738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d3675-4084-47f2-8a09-1d5f578d3211 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:01.391190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d3675-4084-47f2-8a09-1d5f578d3211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:01.413650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7c0d3675-4084-47f2-8a09-1d5f578d3211 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:01.415900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7c0d3675-4084-47f2-8a09-1d5f578d3211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:01.493672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6e752-0457-4f46-80cb-90330e6bc18f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:01.495261 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3467/7003] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:01.495390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-47b6e752-0457-4f46-80cb-90330e6bc18f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:01.504654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d3675-4084-47f2-8a09-1d5f578d3211 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:01.506184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7c0d3675-4084-47f2-8a09-1d5f578d3211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:01.510334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6e752-0457-4f46-80cb-90330e6bc18f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:01.516260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f8cea-826c-4a3c-ad4c-5ff872cd219a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:01.517179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f8cea-826c-4a3c-ad4c-5ff872cd219a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:01.517626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f8cea-826c-4a3c-ad4c-5ff872cd219a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:01.517902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f8cea-826c-4a3c-ad4c-5ff872cd219a admin admin] PATH_INFO: `/v3/users/57058bac9d114a5a8551e7551d09921d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:01.519379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f8cea-826c-4a3c-ad4c-5ff872cd219a admin admin] RBAC: Authorizing `identity:delete_user(user_id=57058bac9d114a5a8551e7551d09921d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:01.521303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f8cea-826c-4a3c-ad4c-5ff872cd219a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:01.524520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:01.578982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d3675-4084-47f2-8a09-1d5f578d3211 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:01.581330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-974f8cea-826c-4a3c-ad4c-5ff872cd219a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57058bac9d114a5a8551e7551d09921d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:01.584686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953f272-ccb3-4d4c-8593-cca85bef7589 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:01.585682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953f272-ccb3-4d4c-8593-cca85bef7589 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:01.585960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953f272-ccb3-4d4c-8593-cca85bef7589 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:01.586808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953f272-ccb3-4d4c-8593-cca85bef7589 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:01.589259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-974f8cea-826c-4a3c-ad4c-5ff872cd219a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57058bac9d114a5a8551e7551d09921d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:01.598817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953f272-ccb3-4d4c-8593-cca85bef7589 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:01.601490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953f272-ccb3-4d4c-8593-cca85bef7589 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:01.603913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-974f8cea-826c-4a3c-ad4c-5ff872cd219a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57058bac9d114a5a8551e7551d09921d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:01.604804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-974f8cea-826c-4a3c-ad4c-5ff872cd219a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57058bac9d114a5a8551e7551d09921d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:01.612109 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3468/7004] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:15:01.615368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-974f8cea-826c-4a3c-ad4c-5ff872cd219a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57058bac9d114a5a8551e7551d09921d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:01.622074 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3537/7005] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:01 2022] DELETE /identity/v3/users/57058bac9d114a5a8551e7551d09921d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:01.626481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813746e-bb5d-495f-b482-48904e1a0424 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:01.626794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813746e-bb5d-495f-b482-48904e1a0424 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:01.627060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813746e-bb5d-495f-b482-48904e1a0424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:01.649454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8813746e-bb5d-495f-b482-48904e1a0424 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:01.650251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8813746e-bb5d-495f-b482-48904e1a0424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:01.745512 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3469/7006] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:01.784941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f8cea-826c-4a3c-ad4c-5ff872cd219a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:01.786520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-974f8cea-826c-4a3c-ad4c-5ff872cd219a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:01.867650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f8cea-826c-4a3c-ad4c-5ff872cd219a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:01.871914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fb311-edab-42e0-aa8e-94a3ce4f0a18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:01.872691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fb311-edab-42e0-aa8e-94a3ce4f0a18 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:01.873200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fb311-edab-42e0-aa8e-94a3ce4f0a18 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:01.873707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fb311-edab-42e0-aa8e-94a3ce4f0a18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:01.884264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6fb311-edab-42e0-aa8e-94a3ce4f0a18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:01.885914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6fb311-edab-42e0-aa8e-94a3ce4f0a18 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:01.899083 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3538/7007] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:15:01.963043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813746e-bb5d-495f-b482-48904e1a0424 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:01.964542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8813746e-bb5d-495f-b482-48904e1a0424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:02.021208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fb311-edab-42e0-aa8e-94a3ce4f0a18 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:02.024418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5a6fb311-edab-42e0-aa8e-94a3ce4f0a18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:02.037810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:02.060768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fb311-edab-42e0-aa8e-94a3ce4f0a18 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:02.077484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9fa60-b2c4-4a94-9c7e-5269a5caa5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:02.078148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9fa60-b2c4-4a94-9c7e-5269a5caa5c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:02.078881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9fa60-b2c4-4a94-9c7e-5269a5caa5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:02.078881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9fa60-b2c4-4a94-9c7e-5269a5caa5c2 admin admin] PATH_INFO: `/v3/projects/357f6ddcb8264bf08fe97f45b3ba336c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:02.083857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9fa60-b2c4-4a94-9c7e-5269a5caa5c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=357f6ddcb8264bf08fe97f45b3ba336c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:02.086384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9fa60-b2c4-4a94-9c7e-5269a5caa5c2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:02.132574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813746e-bb5d-495f-b482-48904e1a0424 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:02.142600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233af6f-c1d9-48d8-ac03-d65ce187ce89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:02.142600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233af6f-c1d9-48d8-ac03-d65ce187ce89 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:02.143114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233af6f-c1d9-48d8-ac03-d65ce187ce89 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:02.143450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233af6f-c1d9-48d8-ac03-d65ce187ce89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:02.145555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a233af6f-c1d9-48d8-ac03-d65ce187ce89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:02.156500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ee9fa60-b2c4-4a94-9c7e-5269a5caa5c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '357f6ddcb8264bf08fe97f45b3ba336c', 'request_id': 'req-2ee9fa60-b2c4-4a94-9c7e-5269a5caa5c2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.164641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ee9fa60-b2c4-4a94-9c7e-5269a5caa5c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '357f6ddcb8264bf08fe97f45b3ba336c', 'request_id': 'req-2ee9fa60-b2c4-4a94-9c7e-5269a5caa5c2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.165633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233af6f-c1d9-48d8-ac03-d65ce187ce89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:02.167991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233af6f-c1d9-48d8-ac03-d65ce187ce89 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:02.171938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ee9fa60-b2c4-4a94-9c7e-5269a5caa5c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '357f6ddcb8264bf08fe97f45b3ba336c', 'request_id': 'req-2ee9fa60-b2c4-4a94-9c7e-5269a5caa5c2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.183905 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3470/7008] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:15:02.190480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ee9fa60-b2c4-4a94-9c7e-5269a5caa5c2 admin admin] The token cache is being invalidate because project 357f6ddcb8264bf08fe97f45b3ba336c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:02.190889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ee9fa60-b2c4-4a94-9c7e-5269a5caa5c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.191644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ee9fa60-b2c4-4a94-9c7e-5269a5caa5c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.196982 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3539/7009] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:02 2022] DELETE /identity/v3/projects/357f6ddcb8264bf08fe97f45b3ba336c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:02.198929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233af6f-c1d9-48d8-ac03-d65ce187ce89 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:02.211731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233af6f-c1d9-48d8-ac03-d65ce187ce89 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:02.299625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3cbb5-b2fb-4836-862d-9c779c3fe0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:02.300783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3cbb5-b2fb-4836-862d-9c779c3fe0ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:02.300897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3cbb5-b2fb-4836-862d-9c779c3fe0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:02.301432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3cbb5-b2fb-4836-862d-9c779c3fe0ae admin admin] PATH_INFO: `/v3/users/6d24598438104e2fb19dc1fff9e7bd26` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:02.303630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3cbb5-b2fb-4836-862d-9c779c3fe0ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d24598438104e2fb19dc1fff9e7bd26)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:02.305835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3cbb5-b2fb-4836-862d-9c779c3fe0ae admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:02.351940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9fa60-b2c4-4a94-9c7e-5269a5caa5c2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:02.353551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2ee9fa60-b2c4-4a94-9c7e-5269a5caa5c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:02.376963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-44b3cbb5-b2fb-4836-862d-9c779c3fe0ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d24598438104e2fb19dc1fff9e7bd26'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.377838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-44b3cbb5-b2fb-4836-862d-9c779c3fe0ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d24598438104e2fb19dc1fff9e7bd26'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.378680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-44b3cbb5-b2fb-4836-862d-9c779c3fe0ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d24598438104e2fb19dc1fff9e7bd26'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.392226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-44b3cbb5-b2fb-4836-862d-9c779c3fe0ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d24598438104e2fb19dc1fff9e7bd26'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.401815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9fa60-b2c4-4a94-9c7e-5269a5caa5c2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:02.409842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06215382-85af-4ec0-a05b-4e59f5c0298d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:02.410951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06215382-85af-4ec0-a05b-4e59f5c0298d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:02.413067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06215382-85af-4ec0-a05b-4e59f5c0298d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:02.413877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06215382-85af-4ec0-a05b-4e59f5c0298d admin admin] PATH_INFO: `/v3/projects/64375fec27684654b4f5ff2f2c1cbde0` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:02.414599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-44b3cbb5-b2fb-4836-862d-9c779c3fe0ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d24598438104e2fb19dc1fff9e7bd26'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.416960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06215382-85af-4ec0-a05b-4e59f5c0298d admin admin] RBAC: Authorizing `identity:delete_project(project_id=64375fec27684654b4f5ff2f2c1cbde0)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:02.420591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06215382-85af-4ec0-a05b-4e59f5c0298d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:02.422704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:02.431091 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3471/7010] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:02 2022] DELETE /identity/v3/users/6d24598438104e2fb19dc1fff9e7bd26 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:02.486592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-06215382-85af-4ec0-a05b-4e59f5c0298d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64375fec27684654b4f5ff2f2c1cbde0', 'request_id': 'req-06215382-85af-4ec0-a05b-4e59f5c0298d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.499131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-06215382-85af-4ec0-a05b-4e59f5c0298d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64375fec27684654b4f5ff2f2c1cbde0', 'request_id': 'req-06215382-85af-4ec0-a05b-4e59f5c0298d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.507905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-06215382-85af-4ec0-a05b-4e59f5c0298d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64375fec27684654b4f5ff2f2c1cbde0', 'request_id': 'req-06215382-85af-4ec0-a05b-4e59f5c0298d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.526243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-06215382-85af-4ec0-a05b-4e59f5c0298d admin admin] The token cache is being invalidate because project 64375fec27684654b4f5ff2f2c1cbde0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:02.526427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-06215382-85af-4ec0-a05b-4e59f5c0298d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.527114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-06215382-85af-4ec0-a05b-4e59f5c0298d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.528869 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3540/7011] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:02 2022] DELETE /identity/v3/projects/64375fec27684654b4f5ff2f2c1cbde0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:02.533013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3cbb5-b2fb-4836-862d-9c779c3fe0ae admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:02.534474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-44b3cbb5-b2fb-4836-862d-9c779c3fe0ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:02.575421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:02.619986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3cbb5-b2fb-4836-862d-9c779c3fe0ae admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:02.691115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c88121-c8e1-4ad8-92f5-0b31c6fce96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:02.691898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c88121-c8e1-4ad8-92f5-0b31c6fce96b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:02.692280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c88121-c8e1-4ad8-92f5-0b31c6fce96b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:02.692575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c88121-c8e1-4ad8-92f5-0b31c6fce96b admin admin] PATH_INFO: `/v3/users/2fd53be8056140dc8edc89aa80b7ecb5` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:02.693833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c88121-c8e1-4ad8-92f5-0b31c6fce96b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fd53be8056140dc8edc89aa80b7ecb5)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:02.695817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c88121-c8e1-4ad8-92f5-0b31c6fce96b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:02.749323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06215382-85af-4ec0-a05b-4e59f5c0298d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:02.750995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-06215382-85af-4ec0-a05b-4e59f5c0298d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:02.756371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f2c88121-c8e1-4ad8-92f5-0b31c6fce96b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fd53be8056140dc8edc89aa80b7ecb5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.757661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f2c88121-c8e1-4ad8-92f5-0b31c6fce96b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fd53be8056140dc8edc89aa80b7ecb5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.758622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f2c88121-c8e1-4ad8-92f5-0b31c6fce96b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fd53be8056140dc8edc89aa80b7ecb5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.765201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06215382-85af-4ec0-a05b-4e59f5c0298d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:02.770992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e258c6-db78-4bf1-9fe2-07ea3e9873b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:02.771784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e258c6-db78-4bf1-9fe2-07ea3e9873b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:02.772071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e258c6-db78-4bf1-9fe2-07ea3e9873b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:02.772348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e258c6-db78-4bf1-9fe2-07ea3e9873b4 admin admin] PATH_INFO: `/v3/projects/eb66dcac57a94081ace839dfddcd1d59` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:02.773835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e258c6-db78-4bf1-9fe2-07ea3e9873b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb66dcac57a94081ace839dfddcd1d59)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:02.776163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e258c6-db78-4bf1-9fe2-07ea3e9873b4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:02.784810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f2c88121-c8e1-4ad8-92f5-0b31c6fce96b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fd53be8056140dc8edc89aa80b7ecb5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.801267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f2c88121-c8e1-4ad8-92f5-0b31c6fce96b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fd53be8056140dc8edc89aa80b7ecb5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.821151 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3472/7012] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:02 2022] DELETE /identity/v3/users/2fd53be8056140dc8edc89aa80b7ecb5 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:02.836311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-39e258c6-db78-4bf1-9fe2-07ea3e9873b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb66dcac57a94081ace839dfddcd1d59', 'request_id': 'req-39e258c6-db78-4bf1-9fe2-07ea3e9873b4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.843451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-39e258c6-db78-4bf1-9fe2-07ea3e9873b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb66dcac57a94081ace839dfddcd1d59', 'request_id': 'req-39e258c6-db78-4bf1-9fe2-07ea3e9873b4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.850901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-39e258c6-db78-4bf1-9fe2-07ea3e9873b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb66dcac57a94081ace839dfddcd1d59', 'request_id': 'req-39e258c6-db78-4bf1-9fe2-07ea3e9873b4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.865747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-39e258c6-db78-4bf1-9fe2-07ea3e9873b4 admin admin] The token cache is being invalidate because project eb66dcac57a94081ace839dfddcd1d59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:02.866276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-39e258c6-db78-4bf1-9fe2-07ea3e9873b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.867101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-39e258c6-db78-4bf1-9fe2-07ea3e9873b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:02.869041 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3541/7013] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:02 2022] DELETE /identity/v3/projects/eb66dcac57a94081ace839dfddcd1d59 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:02.873756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125a472-78ae-4ce4-8a3a-30df6dd6f3d5 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:02.874178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125a472-78ae-4ce4-8a3a-30df6dd6f3d5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:02.874462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125a472-78ae-4ce4-8a3a-30df6dd6f3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:02.895543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4125a472-78ae-4ce4-8a3a-30df6dd6f3d5 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:02.897346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4125a472-78ae-4ce4-8a3a-30df6dd6f3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:02.960107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:02.979304 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3473/7014] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:02.980925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d1d1d-258d-4510-ada6-41371ef2d3ea None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:02.981044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d1d1d-258d-4510-ada6-41371ef2d3ea None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:02.981250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d1d1d-258d-4510-ada6-41371ef2d3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:03.013533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8d4d1d1d-258d-4510-ada6-41371ef2d3ea None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:03.015097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8d4d1d1d-258d-4510-ada6-41371ef2d3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:03.038014 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3542/7015] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:03.042692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125a472-78ae-4ce4-8a3a-30df6dd6f3d5 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:03.056887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125a472-78ae-4ce4-8a3a-30df6dd6f3d5 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:03.062898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671de9d7-0042-4a09-bd72-6fe3255ad2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:03.068637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671de9d7-0042-4a09-bd72-6fe3255ad2a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:03.068982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671de9d7-0042-4a09-bd72-6fe3255ad2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:03.069260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671de9d7-0042-4a09-bd72-6fe3255ad2a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:03.069973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671de9d7-0042-4a09-bd72-6fe3255ad2a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:15:03.070334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671de9d7-0042-4a09-bd72-6fe3255ad2a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:03.072875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671de9d7-0042-4a09-bd72-6fe3255ad2a7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:03.090815 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3474/7016] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:03.091454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d1d1d-258d-4510-ada6-41371ef2d3ea None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:03.102528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d1d1d-258d-4510-ada6-41371ef2d3ea None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:03.109180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697667e-b944-4580-945e-662009a71eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:03.110732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697667e-b944-4580-945e-662009a71eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:03.111253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697667e-b944-4580-945e-662009a71eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:03.111721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697667e-b944-4580-945e-662009a71eb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:03.112562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697667e-b944-4580-945e-662009a71eb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:03.116036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697667e-b944-4580-945e-662009a71eb6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:03.166701 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3543/7017] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:03 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:03.170928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671de9d7-0042-4a09-bd72-6fe3255ad2a7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:03.183481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671de9d7-0042-4a09-bd72-6fe3255ad2a7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:03.197716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697667e-b944-4580-945e-662009a71eb6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:03.199642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2697667e-b944-4580-945e-662009a71eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:03.269447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53e354-bd5b-4a2e-9e44-97c4e67ec2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:03.270100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53e354-bd5b-4a2e-9e44-97c4e67ec2db admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:03.270382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53e354-bd5b-4a2e-9e44-97c4e67ec2db admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:03.270694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53e354-bd5b-4a2e-9e44-97c4e67ec2db admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:03.271349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53e354-bd5b-4a2e-9e44-97c4e67ec2db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:03.274195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53e354-bd5b-4a2e-9e44-97c4e67ec2db admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:03.283415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:03.303702 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3475/7018] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:03 2022] POST /identity/v3/users => generated 286 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:03.305457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53e354-bd5b-4a2e-9e44-97c4e67ec2db admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:03.309921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697667e-b944-4580-945e-662009a71eb6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:03.315422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90003377-d420-41fd-945a-898df57d0d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:03.316068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90003377-d420-41fd-945a-898df57d0d61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:03.316749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90003377-d420-41fd-945a-898df57d0d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:03.316749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90003377-d420-41fd-945a-898df57d0d61 admin admin] PATH_INFO: `/v3/projects/dbeef91d920246d2a3548469b6ffa22d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:03.317876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90003377-d420-41fd-945a-898df57d0d61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbeef91d920246d2a3548469b6ffa22d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:03.320126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53e354-bd5b-4a2e-9e44-97c4e67ec2db admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:03.320483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90003377-d420-41fd-945a-898df57d0d61 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:03.325860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dabe2b-f424-4aaf-9dfd-afd53964baf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:03.326512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dabe2b-f424-4aaf-9dfd-afd53964baf5 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:03.326801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dabe2b-f424-4aaf-9dfd-afd53964baf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:03.327076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dabe2b-f424-4aaf-9dfd-afd53964baf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:03.327719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dabe2b-f424-4aaf-9dfd-afd53964baf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:03.329305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dabe2b-f424-4aaf-9dfd-afd53964baf5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:03.343945 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3476/7019] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:15:03.348225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dabe2b-f424-4aaf-9dfd-afd53964baf5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:03.360927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dabe2b-f424-4aaf-9dfd-afd53964baf5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:03.365559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f277ba-6121-4d34-a017-68e4a2697ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:03.366158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f277ba-6121-4d34-a017-68e4a2697ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:03.366435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f277ba-6121-4d34-a017-68e4a2697ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:03.366730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f277ba-6121-4d34-a017-68e4a2697ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:03.367367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f277ba-6121-4d34-a017-68e4a2697ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:03.368953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f277ba-6121-4d34-a017-68e4a2697ea3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:03.379308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-90003377-d420-41fd-945a-898df57d0d61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbeef91d920246d2a3548469b6ffa22d', 'request_id': 'req-90003377-d420-41fd-945a-898df57d0d61'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:03.383690 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3477/7020] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:03.385366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f277ba-6121-4d34-a017-68e4a2697ea3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:03.390828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-90003377-d420-41fd-945a-898df57d0d61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbeef91d920246d2a3548469b6ffa22d', 'request_id': 'req-90003377-d420-41fd-945a-898df57d0d61'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:03.401415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f277ba-6121-4d34-a017-68e4a2697ea3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:03.401598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-90003377-d420-41fd-945a-898df57d0d61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbeef91d920246d2a3548469b6ffa22d', 'request_id': 'req-90003377-d420-41fd-945a-898df57d0d61'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:03.420010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-90003377-d420-41fd-945a-898df57d0d61 admin admin] The token cache is being invalidate because project dbeef91d920246d2a3548469b6ffa22d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:03.420332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-90003377-d420-41fd-945a-898df57d0d61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:03.421010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-90003377-d420-41fd-945a-898df57d0d61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:03.422729 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3544/7021] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:03 2022] DELETE /identity/v3/projects/dbeef91d920246d2a3548469b6ffa22d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:03.428689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc7525-6775-48b9-804e-3ce9933c7ca3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:03.429026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc7525-6775-48b9-804e-3ce9933c7ca3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:03.429312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc7525-6775-48b9-804e-3ce9933c7ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:03.452363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5bdc7525-6775-48b9-804e-3ce9933c7ca3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:03.453594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5bdc7525-6775-48b9-804e-3ce9933c7ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:03.499381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3b0e0-8222-49fb-9912-ac3bd359b745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:03.501376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3b0e0-8222-49fb-9912-ac3bd359b745 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:03.501812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3b0e0-8222-49fb-9912-ac3bd359b745 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:03.502257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3b0e0-8222-49fb-9912-ac3bd359b745 admin admin] PATH_INFO: `/v3/projects/aa31359109754fcba8ad848702356b65/users/39af6e701f564a2888b09d1eafe3478d/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:03.528001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3b0e0-8222-49fb-9912-ac3bd359b745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa31359109754fcba8ad848702356b65, user_id=39af6e701f564a2888b09d1eafe3478d, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:03.532956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3b0e0-8222-49fb-9912-ac3bd359b745 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:03.554645 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3545/7022] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:03.564238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19679d-93db-4000-8fa9-3a480194a82f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:03.566032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19679d-93db-4000-8fa9-3a480194a82f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:03.567023 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3478/7023] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:03 2022] PUT /identity/v3/projects/aa31359109754fcba8ad848702356b65/users/39af6e701f564a2888b09d1eafe3478d/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:03.567311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19679d-93db-4000-8fa9-3a480194a82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:03.587330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129b1db-a583-4396-8f83-1ff443f51232 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:03.587895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129b1db-a583-4396-8f83-1ff443f51232 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:03.588412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129b1db-a583-4396-8f83-1ff443f51232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:03.595731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ff19679d-93db-4000-8fa9-3a480194a82f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:03.597222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ff19679d-93db-4000-8fa9-3a480194a82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:03.620467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9129b1db-a583-4396-8f83-1ff443f51232 None None] MFA Rules not processed for user `39af6e701f564a2888b09d1eafe3478d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:03.622628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9129b1db-a583-4396-8f83-1ff443f51232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:03.674609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:03.691328 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3546/7024] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:15:03.697251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff19679d-93db-4000-8fa9-3a480194a82f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:03.710514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff19679d-93db-4000-8fa9-3a480194a82f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:03.719115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b9139-863e-49bd-8318-8df60f66a5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:03.720137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b9139-863e-49bd-8318-8df60f66a5e7 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:03.720477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b9139-863e-49bd-8318-8df60f66a5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:03.720860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b9139-863e-49bd-8318-8df60f66a5e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:03.721659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891b9139-863e-49bd-8318-8df60f66a5e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:15:03.722063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891b9139-863e-49bd-8318-8df60f66a5e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:03.723911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891b9139-863e-49bd-8318-8df60f66a5e7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:03.743182 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3547/7025] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:03.744941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b9139-863e-49bd-8318-8df60f66a5e7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:03.753426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:03.762155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b9139-863e-49bd-8318-8df60f66a5e7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:03.777392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f162f28-bf49-4188-99ec-7609919cc475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:03.778702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f162f28-bf49-4188-99ec-7609919cc475 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:03.780323 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3479/7026] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:03.780445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f162f28-bf49-4188-99ec-7609919cc475 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:03.780814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f162f28-bf49-4188-99ec-7609919cc475 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:03.781532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f162f28-bf49-4188-99ec-7609919cc475 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:03.784645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f162f28-bf49-4188-99ec-7609919cc475 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:03.796248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129b1db-a583-4396-8f83-1ff443f51232 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:03.799063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9129b1db-a583-4396-8f83-1ff443f51232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:03.843789 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3548/7027] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:03 2022] POST /identity/v3/projects => generated 409 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:03.845168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f162f28-bf49-4188-99ec-7609919cc475 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:03.882431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129b1db-a583-4396-8f83-1ff443f51232 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:03.934476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f162f28-bf49-4188-99ec-7609919cc475 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:03.940394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c1c23-9638-4c0b-82cc-7adfb8a3d979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:03.941159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c1c23-9638-4c0b-82cc-7adfb8a3d979 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:03.941819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c1c23-9638-4c0b-82cc-7adfb8a3d979 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:03.941819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c1c23-9638-4c0b-82cc-7adfb8a3d979 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:03.942989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c1c23-9638-4c0b-82cc-7adfb8a3d979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:03.945965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c1c23-9638-4c0b-82cc-7adfb8a3d979 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:03.968697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec082d95-1394-4b4d-a0ba-72156b14d619 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:03.969333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec082d95-1394-4b4d-a0ba-72156b14d619 service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:03.970851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec082d95-1394-4b4d-a0ba-72156b14d619 service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:03.971440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec082d95-1394-4b4d-a0ba-72156b14d619 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:03.978024 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3549/7028] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:03 2022] POST /identity/v3/users => generated 305 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:03.982521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c1c23-9638-4c0b-82cc-7adfb8a3d979 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:03.995399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c1c23-9638-4c0b-82cc-7adfb8a3d979 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:04.001158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8180c-aaaf-4716-80dd-ded61be55317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:04.002308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8180c-aaaf-4716-80dd-ded61be55317 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:04.002960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8180c-aaaf-4716-80dd-ded61be55317 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:04.003454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8180c-aaaf-4716-80dd-ded61be55317 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:04.004597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8180c-aaaf-4716-80dd-ded61be55317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:04.007119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8180c-aaaf-4716-80dd-ded61be55317 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:04.018900 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3550/7029] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:04.028109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8180c-aaaf-4716-80dd-ded61be55317 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:04.047649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8180c-aaaf-4716-80dd-ded61be55317 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:04.051445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec082d95-1394-4b4d-a0ba-72156b14d619 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:04.054048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec082d95-1394-4b4d-a0ba-72156b14d619 service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:04.062086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58319dc-53a6-4cfb-a90d-b32d0bbc1aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:04.062860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58319dc-53a6-4cfb-a90d-b32d0bbc1aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:04.063253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58319dc-53a6-4cfb-a90d-b32d0bbc1aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:04.063474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58319dc-53a6-4cfb-a90d-b32d0bbc1aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:04.065188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58319dc-53a6-4cfb-a90d-b32d0bbc1aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:04.067709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58319dc-53a6-4cfb-a90d-b32d0bbc1aa8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:04.067881 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3480/7030] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:15:04.087333 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3551/7031] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:04.091898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec082d95-1394-4b4d-a0ba-72156b14d619 service glance] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:04.107583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec082d95-1394-4b4d-a0ba-72156b14d619 service glance] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:04.115873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181deba-c5ce-4f21-9510-bdef640b1cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:04.116744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181deba-c5ce-4f21-9510-bdef640b1cc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:04.117035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181deba-c5ce-4f21-9510-bdef640b1cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:04.117385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181deba-c5ce-4f21-9510-bdef640b1cc5 admin admin] PATH_INFO: `/v3/projects/21a72991a0f641dc8f513a5cc55b8ad9/users/4ef51ab5ea664ee1bcd815a50cff316f/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:04.144591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181deba-c5ce-4f21-9510-bdef640b1cc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21a72991a0f641dc8f513a5cc55b8ad9, user_id=4ef51ab5ea664ee1bcd815a50cff316f, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:04.150219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181deba-c5ce-4f21-9510-bdef640b1cc5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:04.179763 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3481/7032] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:04 2022] PUT /identity/v3/projects/21a72991a0f641dc8f513a5cc55b8ad9/users/4ef51ab5ea664ee1bcd815a50cff316f/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:04.195690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dc10d-4a01-4a6c-818f-5e71ccdc45aa None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:04.196058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dc10d-4a01-4a6c-818f-5e71ccdc45aa None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:04.196349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dc10d-4a01-4a6c-818f-5e71ccdc45aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:04.258285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0b4dc10d-4a01-4a6c-818f-5e71ccdc45aa None None] MFA Rules not processed for user `4ef51ab5ea664ee1bcd815a50cff316f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:04.262438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0b4dc10d-4a01-4a6c-818f-5e71ccdc45aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:04.372711 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3552/7033] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:04 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:04.373706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181deba-c5ce-4f21-9510-bdef640b1cc5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:04.400069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:04.474648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181deba-c5ce-4f21-9510-bdef640b1cc5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:04.491525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd85064-5ef9-481e-8215-03bcd378d61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:04.492201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd85064-5ef9-481e-8215-03bcd378d61e admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:04.492510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd85064-5ef9-481e-8215-03bcd378d61e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:04.492786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd85064-5ef9-481e-8215-03bcd378d61e admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:04.494251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd85064-5ef9-481e-8215-03bcd378d61e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:04.496408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd85064-5ef9-481e-8215-03bcd378d61e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:04.532487 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3482/7034] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:04 2022] POST /identity/v3/projects => generated 409 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:04.536631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dc10d-4a01-4a6c-818f-5e71ccdc45aa None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:04.550520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dc10d-4a01-4a6c-818f-5e71ccdc45aa None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:04.618734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:04.653673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63729636-1334-41e5-9297-adb612de2a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:04.654414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63729636-1334-41e5-9297-adb612de2a41 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:04.654746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63729636-1334-41e5-9297-adb612de2a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:04.655019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63729636-1334-41e5-9297-adb612de2a41 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:04.655749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63729636-1334-41e5-9297-adb612de2a41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:04.657785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63729636-1334-41e5-9297-adb612de2a41 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:04.685642 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3553/7035] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:04 2022] POST /identity/v3/users => generated 311 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:04.690017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd85064-5ef9-481e-8215-03bcd378d61e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:04.702626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd85064-5ef9-481e-8215-03bcd378d61e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:04.707511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c58627-0adb-444d-9e98-3ff82e7e2e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:04.708267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c58627-0adb-444d-9e98-3ff82e7e2e47 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:04.708433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c58627-0adb-444d-9e98-3ff82e7e2e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:04.708642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c58627-0adb-444d-9e98-3ff82e7e2e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:04.709820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c58627-0adb-444d-9e98-3ff82e7e2e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:04.711522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c58627-0adb-444d-9e98-3ff82e7e2e47 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:04.728665 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3483/7036] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:15:04.730561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63729636-1334-41e5-9297-adb612de2a41 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:04.741444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63729636-1334-41e5-9297-adb612de2a41 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:04.747195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1868e6-c3a6-44b8-bd6d-82bf461407ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:04.747912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1868e6-c3a6-44b8-bd6d-82bf461407ef admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:04.748207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1868e6-c3a6-44b8-bd6d-82bf461407ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:04.748522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1868e6-c3a6-44b8-bd6d-82bf461407ef admin admin] PATH_INFO: `/v3/projects/d7054b3618e342a38ad5fbe0133b1de9/users/bfa962c2e6f349f283bbdd7af31a1f8e/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:04.767927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1868e6-c3a6-44b8-bd6d-82bf461407ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7054b3618e342a38ad5fbe0133b1de9, user_id=bfa962c2e6f349f283bbdd7af31a1f8e, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:04.771075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1868e6-c3a6-44b8-bd6d-82bf461407ef admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:04.791672 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3554/7037] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:04 2022] PUT /identity/v3/projects/d7054b3618e342a38ad5fbe0133b1de9/users/bfa962c2e6f349f283bbdd7af31a1f8e/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:04.795694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c58627-0adb-444d-9e98-3ff82e7e2e47 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:04.809238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c58627-0adb-444d-9e98-3ff82e7e2e47 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:04.900934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62dd6b8-beb6-4797-8c89-698ce1a0332c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:04.901556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dd6b8-beb6-4797-8c89-698ce1a0332c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:04.901836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dd6b8-beb6-4797-8c89-698ce1a0332c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:04.902112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dd6b8-beb6-4797-8c89-698ce1a0332c admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:04.902857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62dd6b8-beb6-4797-8c89-698ce1a0332c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:04.904477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62dd6b8-beb6-4797-8c89-698ce1a0332c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:04.918829 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3484/7038] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:04.921153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1868e6-c3a6-44b8-bd6d-82bf461407ef admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:04.937589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1868e6-c3a6-44b8-bd6d-82bf461407ef admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:04.948696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5e835-163e-4015-bd62-7114332a5747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:04.948696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e835-163e-4015-bd62-7114332a5747 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:04.948696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e835-163e-4015-bd62-7114332a5747 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:04.948696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e835-163e-4015-bd62-7114332a5747 admin admin] PATH_INFO: `/v3/projects/d7054b3618e342a38ad5fbe0133b1de9/users/bfa962c2e6f349f283bbdd7af31a1f8e/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:04.949951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5e835-163e-4015-bd62-7114332a5747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7054b3618e342a38ad5fbe0133b1de9, user_id=bfa962c2e6f349f283bbdd7af31a1f8e, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:04.952802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5e835-163e-4015-bd62-7114332a5747 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:04.964384 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3555/7039] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:04 2022] PUT /identity/v3/projects/d7054b3618e342a38ad5fbe0133b1de9/users/bfa962c2e6f349f283bbdd7af31a1f8e/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:04.976267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a02b24-02cb-45dd-a44b-9dab10b3ec36 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:04.976620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a02b24-02cb-45dd-a44b-9dab10b3ec36 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:04.977159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a02b24-02cb-45dd-a44b-9dab10b3ec36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:05.014280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-89a02b24-02cb-45dd-a44b-9dab10b3ec36 None None] MFA Rules not processed for user `bfa962c2e6f349f283bbdd7af31a1f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:05.015596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-89a02b24-02cb-45dd-a44b-9dab10b3ec36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:05.106376 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3485/7040] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:04 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:05.116739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5e835-163e-4015-bd62-7114332a5747 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:05.118542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ebf5e835-163e-4015-bd62-7114332a5747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:05.184283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5e835-163e-4015-bd62-7114332a5747 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:05.189193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2924903-0051-49f1-b9eb-5b5101e07647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:05.190528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2924903-0051-49f1-b9eb-5b5101e07647 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:05.190885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2924903-0051-49f1-b9eb-5b5101e07647 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:05.191226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2924903-0051-49f1-b9eb-5b5101e07647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:05.204634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2924903-0051-49f1-b9eb-5b5101e07647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:05.206971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2924903-0051-49f1-b9eb-5b5101e07647 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:05.220823 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3556/7041] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:05 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:15:06.025373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a02b24-02cb-45dd-a44b-9dab10b3ec36 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:06.030950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-89a02b24-02cb-45dd-a44b-9dab10b3ec36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:06.051116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:06.134435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a02b24-02cb-45dd-a44b-9dab10b3ec36 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:06.139552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5dc30e-db4e-4d70-a5ac-9fc7e00923af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:06.140185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5dc30e-db4e-4d70-a5ac-9fc7e00923af admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:06.140470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5dc30e-db4e-4d70-a5ac-9fc7e00923af admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:06.140746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5dc30e-db4e-4d70-a5ac-9fc7e00923af admin admin] PATH_INFO: `/v3/users/902ba44d318d4a01a510f732d27467c9` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:06.142263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5dc30e-db4e-4d70-a5ac-9fc7e00923af admin admin] RBAC: Authorizing `identity:delete_user(user_id=902ba44d318d4a01a510f732d27467c9)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:06.144252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5dc30e-db4e-4d70-a5ac-9fc7e00923af admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:06.197200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5a5dc30e-db4e-4d70-a5ac-9fc7e00923af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '902ba44d318d4a01a510f732d27467c9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:06.197859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5a5dc30e-db4e-4d70-a5ac-9fc7e00923af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '902ba44d318d4a01a510f732d27467c9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:06.198469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5a5dc30e-db4e-4d70-a5ac-9fc7e00923af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '902ba44d318d4a01a510f732d27467c9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:06.206127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5a5dc30e-db4e-4d70-a5ac-9fc7e00923af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '902ba44d318d4a01a510f732d27467c9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:06.218858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5a5dc30e-db4e-4d70-a5ac-9fc7e00923af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '902ba44d318d4a01a510f732d27467c9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:06.233056 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3486/7042] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:06 2022] DELETE /identity/v3/users/902ba44d318d4a01a510f732d27467c9 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:06.423975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2924903-0051-49f1-b9eb-5b5101e07647 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:06.443938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2924903-0051-49f1-b9eb-5b5101e07647 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:06.511395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d8980-47cd-4869-b1ca-8f6a86333974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:06.512026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d8980-47cd-4869-b1ca-8f6a86333974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:06.512339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d8980-47cd-4869-b1ca-8f6a86333974 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:06.512612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d8980-47cd-4869-b1ca-8f6a86333974 admin admin] PATH_INFO: `/v3/projects/24a06f5da36b4488a1b8ea01729b0e6c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:06.513765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d8980-47cd-4869-b1ca-8f6a86333974 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24a06f5da36b4488a1b8ea01729b0e6c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:06.515811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d8980-47cd-4869-b1ca-8f6a86333974 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:06.555068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:06.571521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e91d8980-47cd-4869-b1ca-8f6a86333974 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24a06f5da36b4488a1b8ea01729b0e6c', 'request_id': 'req-e91d8980-47cd-4869-b1ca-8f6a86333974'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:06.578021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e91d8980-47cd-4869-b1ca-8f6a86333974 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24a06f5da36b4488a1b8ea01729b0e6c', 'request_id': 'req-e91d8980-47cd-4869-b1ca-8f6a86333974'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:06.584565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e91d8980-47cd-4869-b1ca-8f6a86333974 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24a06f5da36b4488a1b8ea01729b0e6c', 'request_id': 'req-e91d8980-47cd-4869-b1ca-8f6a86333974'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:06.597876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e91d8980-47cd-4869-b1ca-8f6a86333974 admin admin] The token cache is being invalidate because project 24a06f5da36b4488a1b8ea01729b0e6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:06.598172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e91d8980-47cd-4869-b1ca-8f6a86333974 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:06.598821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e91d8980-47cd-4869-b1ca-8f6a86333974 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:06.600339 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3557/7043] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:06 2022] DELETE /identity/v3/projects/24a06f5da36b4488a1b8ea01729b0e6c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:06.606035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f17e9-b69c-44dc-8917-49cab6554e12 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:06.606328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f17e9-b69c-44dc-8917-49cab6554e12 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:06.606668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f17e9-b69c-44dc-8917-49cab6554e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:06.627662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c90f17e9-b69c-44dc-8917-49cab6554e12 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:06.628940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c90f17e9-b69c-44dc-8917-49cab6554e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:06.692448 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3487/7044] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:06.700890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395e8ff-62a7-4d07-87f6-d6e286df8438 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:06.701280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395e8ff-62a7-4d07-87f6-d6e286df8438 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:06.701626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395e8ff-62a7-4d07-87f6-d6e286df8438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:06.722713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6395e8ff-62a7-4d07-87f6-d6e286df8438 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:06.723936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6395e8ff-62a7-4d07-87f6-d6e286df8438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:06.735491 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3558/7045] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:15:06.739808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f17e9-b69c-44dc-8917-49cab6554e12 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:06.751322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f17e9-b69c-44dc-8917-49cab6554e12 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:06.756935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea785e-ada0-4130-b72b-2abe2fa7a4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:06.757543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea785e-ada0-4130-b72b-2abe2fa7a4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:06.757833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea785e-ada0-4130-b72b-2abe2fa7a4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:06.758110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea785e-ada0-4130-b72b-2abe2fa7a4c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:06.758790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea785e-ada0-4130-b72b-2abe2fa7a4c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:15:06.759132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea785e-ada0-4130-b72b-2abe2fa7a4c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:06.762818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea785e-ada0-4130-b72b-2abe2fa7a4c8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:06.777238 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3488/7046] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:15:06.778465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395e8ff-62a7-4d07-87f6-d6e286df8438 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:06.788123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395e8ff-62a7-4d07-87f6-d6e286df8438 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:06.792797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cb35f-230f-4908-8f55-e1103ca9104d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:06.793400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cb35f-230f-4908-8f55-e1103ca9104d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:06.793681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cb35f-230f-4908-8f55-e1103ca9104d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:06.793953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cb35f-230f-4908-8f55-e1103ca9104d admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:06.794618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55cb35f-230f-4908-8f55-e1103ca9104d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:06.796512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55cb35f-230f-4908-8f55-e1103ca9104d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:06.829921 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3559/7047] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:06 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:06.831100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea785e-ada0-4130-b72b-2abe2fa7a4c8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:06.842409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea785e-ada0-4130-b72b-2abe2fa7a4c8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:06.852091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:06.907735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abeaff2-e290-432b-8ba6-e33a40038fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:06.910126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abeaff2-e290-432b-8ba6-e33a40038fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:06.910402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abeaff2-e290-432b-8ba6-e33a40038fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:06.910734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abeaff2-e290-432b-8ba6-e33a40038fa8 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:06.911358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abeaff2-e290-432b-8ba6-e33a40038fa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:06.913290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abeaff2-e290-432b-8ba6-e33a40038fa8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:06.941111 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3489/7048] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:06 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:06.945878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cb35f-230f-4908-8f55-e1103ca9104d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:06.958109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cb35f-230f-4908-8f55-e1103ca9104d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:06.964244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6629c-ab8a-489c-8eac-c4b41281c75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:06.965062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6629c-ab8a-489c-8eac-c4b41281c75a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:06.965494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6629c-ab8a-489c-8eac-c4b41281c75a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:06.965924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6629c-ab8a-489c-8eac-c4b41281c75a admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:06.966900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6629c-ab8a-489c-8eac-c4b41281c75a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:06.968655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6629c-ab8a-489c-8eac-c4b41281c75a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:06.980496 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3560/7049] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:06.985154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abeaff2-e290-432b-8ba6-e33a40038fa8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:06.998679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abeaff2-e290-432b-8ba6-e33a40038fa8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:07.023057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0dbaf-8544-4e43-944c-f802398896cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:07.023057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0dbaf-8544-4e43-944c-f802398896cc admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:07.023057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0dbaf-8544-4e43-944c-f802398896cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:07.023057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0dbaf-8544-4e43-944c-f802398896cc admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:07.047746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0dbaf-8544-4e43-944c-f802398896cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:07.048929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0dbaf-8544-4e43-944c-f802398896cc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:07.081410 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3490/7050] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:07.092449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6629c-ab8a-489c-8eac-c4b41281c75a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:07.114591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6629c-ab8a-489c-8eac-c4b41281c75a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:07.120086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c84074-3235-4cd3-a708-0b81062727d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:07.120759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c84074-3235-4cd3-a708-0b81062727d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:07.121033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c84074-3235-4cd3-a708-0b81062727d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:07.121431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c84074-3235-4cd3-a708-0b81062727d0 admin admin] PATH_INFO: `/v3/projects/ec4188e52f7b496db0700400cf04b200/users/6222fb0a13cb47c9a44173dd3e37b5c4/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:07.140325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c84074-3235-4cd3-a708-0b81062727d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec4188e52f7b496db0700400cf04b200, user_id=6222fb0a13cb47c9a44173dd3e37b5c4, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:07.143190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c84074-3235-4cd3-a708-0b81062727d0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:07.170424 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3561/7051] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:07 2022] PUT /identity/v3/projects/ec4188e52f7b496db0700400cf04b200/users/6222fb0a13cb47c9a44173dd3e37b5c4/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:07.182927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc9113-0543-4acb-8329-a69a1f1b1f02 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:07.184227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc9113-0543-4acb-8329-a69a1f1b1f02 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:07.184227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc9113-0543-4acb-8329-a69a1f1b1f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:07.217033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-64fc9113-0543-4acb-8329-a69a1f1b1f02 None None] MFA Rules not processed for user `6222fb0a13cb47c9a44173dd3e37b5c4`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:07.218554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-64fc9113-0543-4acb-8329-a69a1f1b1f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:07.259301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c84074-3235-4cd3-a708-0b81062727d0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:07.261916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-77c84074-3235-4cd3-a708-0b81062727d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:07.296481 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3491/7052] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:07.301269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc9113-0543-4acb-8329-a69a1f1b1f02 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:07.343384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c84074-3235-4cd3-a708-0b81062727d0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:07.348547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e727316-0117-4fdc-a649-062775ebc4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:07.350053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e727316-0117-4fdc-a649-062775ebc4f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:07.350409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e727316-0117-4fdc-a649-062775ebc4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:07.350794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e727316-0117-4fdc-a649-062775ebc4f8 admin admin] PATH_INFO: `/v3/users/4ef51ab5ea664ee1bcd815a50cff316f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:07.352147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e727316-0117-4fdc-a649-062775ebc4f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ef51ab5ea664ee1bcd815a50cff316f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:07.354081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e727316-0117-4fdc-a649-062775ebc4f8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:07.375920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc9113-0543-4acb-8329-a69a1f1b1f02 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:07.382126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f1dc3-d997-45b4-9d1e-7e81b6016418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:07.383201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f1dc3-d997-45b4-9d1e-7e81b6016418 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:07.383201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f1dc3-d997-45b4-9d1e-7e81b6016418 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:07.383520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f1dc3-d997-45b4-9d1e-7e81b6016418 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:07.384205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f1dc3-d997-45b4-9d1e-7e81b6016418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:07.386627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f1dc3-d997-45b4-9d1e-7e81b6016418 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:07.411643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8e727316-0117-4fdc-a649-062775ebc4f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ef51ab5ea664ee1bcd815a50cff316f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:07.432601 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3492/7053] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:07 2022] POST /identity/v3/projects => generated 375 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:07.436544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f1dc3-d997-45b4-9d1e-7e81b6016418 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:07.438314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8e727316-0117-4fdc-a649-062775ebc4f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ef51ab5ea664ee1bcd815a50cff316f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:07.447638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:07.454875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:07.468489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8e727316-0117-4fdc-a649-062775ebc4f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ef51ab5ea664ee1bcd815a50cff316f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:07.472554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8e727316-0117-4fdc-a649-062775ebc4f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ef51ab5ea664ee1bcd815a50cff316f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:07.482701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8e727316-0117-4fdc-a649-062775ebc4f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ef51ab5ea664ee1bcd815a50cff316f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:07.493333 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3562/7054] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:07 2022] DELETE /identity/v3/users/4ef51ab5ea664ee1bcd815a50cff316f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:07.504787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98c15d-1fa8-4fa6-a76f-bdde25430abb None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:07.504947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98c15d-1fa8-4fa6-a76f-bdde25430abb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:07.505228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98c15d-1fa8-4fa6-a76f-bdde25430abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:07.523167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f1dc3-d997-45b4-9d1e-7e81b6016418 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:07.535593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5f98c15d-1fa8-4fa6-a76f-bdde25430abb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:07.537110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5f98c15d-1fa8-4fa6-a76f-bdde25430abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:07.589917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e863dea-bb84-46f5-bb9c-9c448ae67d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:07.590860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e863dea-bb84-46f5-bb9c-9c448ae67d4c admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:07.591253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e863dea-bb84-46f5-bb9c-9c448ae67d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:07.591650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e863dea-bb84-46f5-bb9c-9c448ae67d4c admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:07.592376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e863dea-bb84-46f5-bb9c-9c448ae67d4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:07.594665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e863dea-bb84-46f5-bb9c-9c448ae67d4c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:07.610559 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3563/7055] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:15:07.620093 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3493/7056] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:07 2022] POST /identity/v3/users => generated 294 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:07.620351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98c15d-1fa8-4fa6-a76f-bdde25430abb None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:07.621831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5f98c15d-1fa8-4fa6-a76f-bdde25430abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:07.625161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e863dea-bb84-46f5-bb9c-9c448ae67d4c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:07.628140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8e863dea-bb84-46f5-bb9c-9c448ae67d4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:07.641413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e863dea-bb84-46f5-bb9c-9c448ae67d4c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:07.648578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe074f7-a14d-4099-b499-e58b3e25d94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:07.649200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe074f7-a14d-4099-b499-e58b3e25d94f admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:07.649484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe074f7-a14d-4099-b499-e58b3e25d94f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:07.649887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe074f7-a14d-4099-b499-e58b3e25d94f admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:07.650767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe074f7-a14d-4099-b499-e58b3e25d94f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:07.652545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe074f7-a14d-4099-b499-e58b3e25d94f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:07.664419 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3494/7057] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:07.669761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe074f7-a14d-4099-b499-e58b3e25d94f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:07.681159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe074f7-a14d-4099-b499-e58b3e25d94f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:07.687248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca90e7-dd5f-46e1-b535-0baf4cc23aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:07.688056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca90e7-dd5f-46e1-b535-0baf4cc23aac admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:07.688411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca90e7-dd5f-46e1-b535-0baf4cc23aac admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:07.688770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca90e7-dd5f-46e1-b535-0baf4cc23aac admin admin] PATH_INFO: `/v3/projects/40d2a6596d6746bea3cab26333dbdb74/users/55615b46899c44249f5940428e5ff261/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:07.695704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98c15d-1fa8-4fa6-a76f-bdde25430abb None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:07.700432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b0842-81f4-4b6c-8994-50abcc70ea2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:07.701298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b0842-81f4-4b6c-8994-50abcc70ea2b service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:07.701585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b0842-81f4-4b6c-8994-50abcc70ea2b service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:07.701908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b0842-81f4-4b6c-8994-50abcc70ea2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:07.706186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca90e7-dd5f-46e1-b535-0baf4cc23aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d2a6596d6746bea3cab26333dbdb74, user_id=55615b46899c44249f5940428e5ff261, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:07.710684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca90e7-dd5f-46e1-b535-0baf4cc23aac admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:07.713770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b0842-81f4-4b6c-8994-50abcc70ea2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:07.716811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b0842-81f4-4b6c-8994-50abcc70ea2b service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:07.730402 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3564/7058] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:15:07.733232 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3495/7059] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:07 2022] PUT /identity/v3/projects/40d2a6596d6746bea3cab26333dbdb74/users/55615b46899c44249f5940428e5ff261/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:07.738096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b0842-81f4-4b6c-8994-50abcc70ea2b service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:07.748822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b0842-81f4-4b6c-8994-50abcc70ea2b service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:07.809042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975aa84f-04ba-4d1c-8d00-6cf08b4ca7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:07.809742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975aa84f-04ba-4d1c-8d00-6cf08b4ca7dc admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:07.810090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975aa84f-04ba-4d1c-8d00-6cf08b4ca7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:07.810445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975aa84f-04ba-4d1c-8d00-6cf08b4ca7dc admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:07.811120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975aa84f-04ba-4d1c-8d00-6cf08b4ca7dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:07.812710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975aa84f-04ba-4d1c-8d00-6cf08b4ca7dc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:07.823773 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3565/7060] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:07.827968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca90e7-dd5f-46e1-b535-0baf4cc23aac admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:07.839572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca90e7-dd5f-46e1-b535-0baf4cc23aac admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:07.844311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d919de8-343c-4194-8bb0-8a3a67b364c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:07.844961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d919de8-343c-4194-8bb0-8a3a67b364c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:07.845240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d919de8-343c-4194-8bb0-8a3a67b364c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:07.845522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d919de8-343c-4194-8bb0-8a3a67b364c8 admin admin] PATH_INFO: `/v3/projects/40d2a6596d6746bea3cab26333dbdb74/users/55615b46899c44249f5940428e5ff261/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:07.847486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d919de8-343c-4194-8bb0-8a3a67b364c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d2a6596d6746bea3cab26333dbdb74, user_id=55615b46899c44249f5940428e5ff261, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:07.850182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d919de8-343c-4194-8bb0-8a3a67b364c8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:07.860837 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3496/7061] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:07 2022] PUT /identity/v3/projects/40d2a6596d6746bea3cab26333dbdb74/users/55615b46899c44249f5940428e5ff261/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:07.871085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54576737-a29f-499a-8597-f59baee32908 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:07.871372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54576737-a29f-499a-8597-f59baee32908 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:07.871641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54576737-a29f-499a-8597-f59baee32908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:07.902336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-54576737-a29f-499a-8597-f59baee32908 None None] MFA Rules not processed for user `55615b46899c44249f5940428e5ff261`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:07.903616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-54576737-a29f-499a-8597-f59baee32908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:07.909784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:07.979217 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3566/7062] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:15:07.984283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d919de8-343c-4194-8bb0-8a3a67b364c8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:07.996094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d919de8-343c-4194-8bb0-8a3a67b364c8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:08.001331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c13af-d90f-4302-93bb-057be54c8ced tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55615b46899c44249f5940428e5ff261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40d2a6596d6746bea3cab26333dbdb74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:08.002018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c13af-d90f-4302-93bb-057be54c8ced tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:08.002364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c13af-d90f-4302-93bb-057be54c8ced tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:08.002720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c13af-d90f-4302-93bb-057be54c8ced tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:08.003493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c13af-d90f-4302-93bb-057be54c8ced tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:08.005209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c13af-d90f-4302-93bb-057be54c8ced tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:08.012344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54576737-a29f-499a-8597-f59baee32908 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:08.013931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-54576737-a29f-499a-8597-f59baee32908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:08.035760 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3497/7063] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:15:07 2022] POST /identity/v3/domains => generated 246 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:08.040654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c13af-d90f-4302-93bb-057be54c8ced tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:08.090901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:08.106900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54576737-a29f-499a-8597-f59baee32908 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:08.114018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c13af-d90f-4302-93bb-057be54c8ced tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:08.119683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bb7c0-c897-4d45-b212-a20eb38bd1ce tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55615b46899c44249f5940428e5ff261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40d2a6596d6746bea3cab26333dbdb74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:08.120530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bb7c0-c897-4d45-b212-a20eb38bd1ce tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:08.120889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bb7c0-c897-4d45-b212-a20eb38bd1ce tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:08.121253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bb7c0-c897-4d45-b212-a20eb38bd1ce tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:08.122073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0bb7c0-c897-4d45-b212-a20eb38bd1ce tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:08.124638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0bb7c0-c897-4d45-b212-a20eb38bd1ce tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:08.164124 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3498/7064] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:08 2022] POST /identity/v3/projects => generated 410 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:08.168290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bb7c0-c897-4d45-b212-a20eb38bd1ce tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:08.168915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ce542-2c01-4484-b745-a7e110caa09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:08.169561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ce542-2c01-4484-b745-a7e110caa09f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:08.169842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ce542-2c01-4484-b745-a7e110caa09f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:08.170118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ce542-2c01-4484-b745-a7e110caa09f admin admin] PATH_INFO: `/v3/projects/21a72991a0f641dc8f513a5cc55b8ad9` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:08.171294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ce542-2c01-4484-b745-a7e110caa09f admin admin] RBAC: Authorizing `identity:delete_project(project_id=21a72991a0f641dc8f513a5cc55b8ad9)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:08.173453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ce542-2c01-4484-b745-a7e110caa09f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:08.228030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c87ce542-2c01-4484-b745-a7e110caa09f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21a72991a0f641dc8f513a5cc55b8ad9', 'request_id': 'req-c87ce542-2c01-4484-b745-a7e110caa09f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:08.231509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:08.239554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bb7c0-c897-4d45-b212-a20eb38bd1ce tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:08.240612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c87ce542-2c01-4484-b745-a7e110caa09f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21a72991a0f641dc8f513a5cc55b8ad9', 'request_id': 'req-c87ce542-2c01-4484-b745-a7e110caa09f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:08.248692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c87ce542-2c01-4484-b745-a7e110caa09f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21a72991a0f641dc8f513a5cc55b8ad9', 'request_id': 'req-c87ce542-2c01-4484-b745-a7e110caa09f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:08.261916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c87ce542-2c01-4484-b745-a7e110caa09f admin admin] The token cache is being invalidate because project 21a72991a0f641dc8f513a5cc55b8ad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:08.262335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c87ce542-2c01-4484-b745-a7e110caa09f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:08.263075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c87ce542-2c01-4484-b745-a7e110caa09f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:08.264919 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3567/7065] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:08 2022] DELETE /identity/v3/projects/21a72991a0f641dc8f513a5cc55b8ad9 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:08.268498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ce542-2c01-4484-b745-a7e110caa09f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:08.269972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c87ce542-2c01-4484-b745-a7e110caa09f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:08.309668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7aa3eb-ffae-47f0-b6db-b760c09d0051 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55615b46899c44249f5940428e5ff261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40d2a6596d6746bea3cab26333dbdb74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:08.310283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7aa3eb-ffae-47f0-b6db-b760c09d0051 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:08.310599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7aa3eb-ffae-47f0-b6db-b760c09d0051 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:08.310878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7aa3eb-ffae-47f0-b6db-b760c09d0051 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:08.311919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7aa3eb-ffae-47f0-b6db-b760c09d0051 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:08.313911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7aa3eb-ffae-47f0-b6db-b760c09d0051 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:08.337867 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3499/7066] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:08 2022] POST /identity/v3/users => generated 346 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:08.341936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7aa3eb-ffae-47f0-b6db-b760c09d0051 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:08.343688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3f7aa3eb-ffae-47f0-b6db-b760c09d0051 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:08.357558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7aa3eb-ffae-47f0-b6db-b760c09d0051 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:08.362462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ed2de-521b-41f0-9f78-0abfa38a12d9 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55615b46899c44249f5940428e5ff261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40d2a6596d6746bea3cab26333dbdb74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:08.363108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ed2de-521b-41f0-9f78-0abfa38a12d9 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:08.363391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ed2de-521b-41f0-9f78-0abfa38a12d9 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:08.363685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ed2de-521b-41f0-9f78-0abfa38a12d9 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:08.364387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ed2de-521b-41f0-9f78-0abfa38a12d9 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:08.366678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ed2de-521b-41f0-9f78-0abfa38a12d9 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:08.367185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ce542-2c01-4484-b745-a7e110caa09f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:08.377872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3df562-419e-4661-b396-29ee01884282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:08.385268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3df562-419e-4661-b396-29ee01884282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:08.385268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3df562-419e-4661-b396-29ee01884282 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:08.385268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3df562-419e-4661-b396-29ee01884282 admin admin] PATH_INFO: `/v3/users/bfa962c2e6f349f283bbdd7af31a1f8e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:08.385517 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3500/7067] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:15:08.386211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3df562-419e-4661-b396-29ee01884282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfa962c2e6f349f283bbdd7af31a1f8e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:08.386886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ed2de-521b-41f0-9f78-0abfa38a12d9 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:08.388363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3df562-419e-4661-b396-29ee01884282 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:08.396801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ed2de-521b-41f0-9f78-0abfa38a12d9 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:08.401868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2252730-e873-44e2-8c30-a20817020951 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55615b46899c44249f5940428e5ff261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40d2a6596d6746bea3cab26333dbdb74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:08.402547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2252730-e873-44e2-8c30-a20817020951 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:08.402831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2252730-e873-44e2-8c30-a20817020951 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:08.403108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2252730-e873-44e2-8c30-a20817020951 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] PATH_INFO: `/v3/projects/fa421578911a4724b83b1177e8479f63/users/337b86a4ed13402f8b0a32b6bc8760dc/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:08.417514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2252730-e873-44e2-8c30-a20817020951 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: Authorizing `identity:create_grant(project_id=fa421578911a4724b83b1177e8479f63, user_id=337b86a4ed13402f8b0a32b6bc8760dc, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:08.420191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2252730-e873-44e2-8c30-a20817020951 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:08.435461 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3501/7068] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:08 2022] PUT /identity/v3/projects/fa421578911a4724b83b1177e8479f63/users/337b86a4ed13402f8b0a32b6bc8760dc/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:08.439884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474daff-a22d-4242-9178-436933e91be5 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:08.440194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474daff-a22d-4242-9178-436933e91be5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:08.440474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474daff-a22d-4242-9178-436933e91be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:08.446920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-af3df562-419e-4661-b396-29ee01884282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfa962c2e6f349f283bbdd7af31a1f8e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:08.454461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-af3df562-419e-4661-b396-29ee01884282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfa962c2e6f349f283bbdd7af31a1f8e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:08.467458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-af3df562-419e-4661-b396-29ee01884282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfa962c2e6f349f283bbdd7af31a1f8e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:08.468144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-af3df562-419e-4661-b396-29ee01884282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfa962c2e6f349f283bbdd7af31a1f8e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:08.469252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:08.476813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-af3df562-419e-4661-b396-29ee01884282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfa962c2e6f349f283bbdd7af31a1f8e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:08.484284 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3568/7069] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:08 2022] DELETE /identity/v3/users/bfa962c2e6f349f283bbdd7af31a1f8e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:08.549188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6474daff-a22d-4242-9178-436933e91be5 None None] MFA Rules not processed for user `337b86a4ed13402f8b0a32b6bc8760dc`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:08.550583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6474daff-a22d-4242-9178-436933e91be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:08.579763 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3502/7070] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:08 2022] POST /identity/v3/auth/tokens => generated 3361 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:15:08.587490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9ca41-dd83-4df2-9c0f-5da0b54262a6 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:08.587706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9ca41-dd83-4df2-9c0f-5da0b54262a6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:08.587981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9ca41-dd83-4df2-9c0f-5da0b54262a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:08.608774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-36e9ca41-dd83-4df2-9c0f-5da0b54262a6 None None] MFA Rules not processed for user `337b86a4ed13402f8b0a32b6bc8760dc`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:08.609992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-36e9ca41-dd83-4df2-9c0f-5da0b54262a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:08.617691 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3569/7071] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:08 2022] POST /identity/v3/auth/tokens => generated 3361 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:08.621606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474daff-a22d-4242-9178-436933e91be5 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:08.631984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474daff-a22d-4242-9178-436933e91be5 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:08.636866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76318c44-2095-4504-9510-8b5f1cc35334 tempest-test_project-2031014046 tempest-user-98649429] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '337b86a4ed13402f8b0a32b6bc8760dc', 'user_domain_id': '6902352497e8459eacd873420303d711', 'system_scope': None, 'project_id': 'fa421578911a4724b83b1177e8479f63', 'project_domain_id': '6902352497e8459eacd873420303d711', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:08.637407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76318c44-2095-4504-9510-8b5f1cc35334 tempest-test_project-2031014046 tempest-user-98649429] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:08.637687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76318c44-2095-4504-9510-8b5f1cc35334 tempest-test_project-2031014046 tempest-user-98649429] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:08.637965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76318c44-2095-4504-9510-8b5f1cc35334 tempest-test_project-2031014046 tempest-user-98649429] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:08.651358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76318c44-2095-4504-9510-8b5f1cc35334 tempest-test_project-2031014046 tempest-user-98649429] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:08.653035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76318c44-2095-4504-9510-8b5f1cc35334 tempest-test_project-2031014046 tempest-user-98649429] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:08.663926 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3503/7072] 10.222.0.17 () {56 vars in 1284 bytes} [Sat Oct 1 21:15:08 2022] GET /identity/v3/auth/tokens => generated 3361 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:15:08.671284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9ca41-dd83-4df2-9c0f-5da0b54262a6 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:08.672842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-36e9ca41-dd83-4df2-9c0f-5da0b54262a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:08.676986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76318c44-2095-4504-9510-8b5f1cc35334 tempest-test_project-2031014046 tempest-user-98649429] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:08.680820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-76318c44-2095-4504-9510-8b5f1cc35334 tempest-test_project-2031014046 tempest-user-98649429] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:08.750630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:08.770563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9ca41-dd83-4df2-9c0f-5da0b54262a6 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:08.774627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76318c44-2095-4504-9510-8b5f1cc35334 tempest-test_project-2031014046 tempest-user-98649429] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:08.778684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efabb19-b95e-427e-97a3-7b315e23ac09 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55615b46899c44249f5940428e5ff261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40d2a6596d6746bea3cab26333dbdb74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:08.779617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efabb19-b95e-427e-97a3-7b315e23ac09 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:08.779970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efabb19-b95e-427e-97a3-7b315e23ac09 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:08.780242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab8567-f6cd-419a-acff-58c81fe0303e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:08.780551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efabb19-b95e-427e-97a3-7b315e23ac09 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] PATH_INFO: `/v3/users/337b86a4ed13402f8b0a32b6bc8760dc` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:08.781048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab8567-f6cd-419a-acff-58c81fe0303e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:08.781393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab8567-f6cd-419a-acff-58c81fe0303e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:08.781675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab8567-f6cd-419a-acff-58c81fe0303e admin admin] PATH_INFO: `/v3/projects/d7054b3618e342a38ad5fbe0133b1de9` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:08.782516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efabb19-b95e-427e-97a3-7b315e23ac09 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: Authorizing `identity:delete_user(user_id=337b86a4ed13402f8b0a32b6bc8760dc)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:08.782872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dab8567-f6cd-419a-acff-58c81fe0303e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7054b3618e342a38ad5fbe0133b1de9)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:08.784961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dab8567-f6cd-419a-acff-58c81fe0303e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:08.786063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efabb19-b95e-427e-97a3-7b315e23ac09 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:08.850150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3efabb19-b95e-427e-97a3-7b315e23ac09 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '337b86a4ed13402f8b0a32b6bc8760dc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:08.858180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3efabb19-b95e-427e-97a3-7b315e23ac09 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '337b86a4ed13402f8b0a32b6bc8760dc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:08.859461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2dab8567-f6cd-419a-acff-58c81fe0303e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7054b3618e342a38ad5fbe0133b1de9', 'request_id': 'req-2dab8567-f6cd-419a-acff-58c81fe0303e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:08.869892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2dab8567-f6cd-419a-acff-58c81fe0303e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7054b3618e342a38ad5fbe0133b1de9', 'request_id': 'req-2dab8567-f6cd-419a-acff-58c81fe0303e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:08.873493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3efabb19-b95e-427e-97a3-7b315e23ac09 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '337b86a4ed13402f8b0a32b6bc8760dc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:08.874297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3efabb19-b95e-427e-97a3-7b315e23ac09 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '337b86a4ed13402f8b0a32b6bc8760dc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:08.876901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2dab8567-f6cd-419a-acff-58c81fe0303e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7054b3618e342a38ad5fbe0133b1de9', 'request_id': 'req-2dab8567-f6cd-419a-acff-58c81fe0303e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:08.883641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3efabb19-b95e-427e-97a3-7b315e23ac09 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '337b86a4ed13402f8b0a32b6bc8760dc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:08.890515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2dab8567-f6cd-419a-acff-58c81fe0303e admin admin] The token cache is being invalidate because project d7054b3618e342a38ad5fbe0133b1de9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:08.890670 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3570/7073] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:08 2022] DELETE /identity/v3/users/337b86a4ed13402f8b0a32b6bc8760dc => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:08.890820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2dab8567-f6cd-419a-acff-58c81fe0303e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:08.892996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2dab8567-f6cd-419a-acff-58c81fe0303e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:08.894652 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3504/7074] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:08 2022] DELETE /identity/v3/projects/d7054b3618e342a38ad5fbe0133b1de9 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:08.899260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efabb19-b95e-427e-97a3-7b315e23ac09 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:08.900041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad891f-71e4-41c2-9451-f663d26fadd2 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:08.900277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad891f-71e4-41c2-9451-f663d26fadd2 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:08.900547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad891f-71e4-41c2-9451-f663d26fadd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:08.900922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3efabb19-b95e-427e-97a3-7b315e23ac09 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:08.914831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:08.926050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-48ad891f-71e4-41c2-9451-f663d26fadd2 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:08.928260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-48ad891f-71e4-41c2-9451-f663d26fadd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:08.971520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efabb19-b95e-427e-97a3-7b315e23ac09 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:08.977073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3578dd3-511f-48e3-9fbe-491f749c7d79 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55615b46899c44249f5940428e5ff261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40d2a6596d6746bea3cab26333dbdb74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:08.977704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3578dd3-511f-48e3-9fbe-491f749c7d79 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:08.977988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3578dd3-511f-48e3-9fbe-491f749c7d79 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:08.978266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3578dd3-511f-48e3-9fbe-491f749c7d79 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] PATH_INFO: `/v3/projects/fa421578911a4724b83b1177e8479f63` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:08.979432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3578dd3-511f-48e3-9fbe-491f749c7d79 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: Authorizing `identity:delete_project(project_id=fa421578911a4724b83b1177e8479f63)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:08.981349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3578dd3-511f-48e3-9fbe-491f749c7d79 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:08.989371 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3505/7075] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:15:08.996779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ab6e7-d89f-419d-b835-08fb5384e08f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:08.997137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ab6e7-d89f-419d-b835-08fb5384e08f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:08.997508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ab6e7-d89f-419d-b835-08fb5384e08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:09.016929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2c3ab6e7-d89f-419d-b835-08fb5384e08f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:09.018050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2c3ab6e7-d89f-419d-b835-08fb5384e08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:09.029653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3578dd3-511f-48e3-9fbe-491f749c7d79 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa421578911a4724b83b1177e8479f63', 'request_id': 'req-e3578dd3-511f-48e3-9fbe-491f749c7d79'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.036171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3578dd3-511f-48e3-9fbe-491f749c7d79 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa421578911a4724b83b1177e8479f63', 'request_id': 'req-e3578dd3-511f-48e3-9fbe-491f749c7d79'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.051224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3578dd3-511f-48e3-9fbe-491f749c7d79 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa421578911a4724b83b1177e8479f63', 'request_id': 'req-e3578dd3-511f-48e3-9fbe-491f749c7d79'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.064568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3578dd3-511f-48e3-9fbe-491f749c7d79 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] The token cache is being invalidate because project fa421578911a4724b83b1177e8479f63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:09.064862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3578dd3-511f-48e3-9fbe-491f749c7d79 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.065534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3578dd3-511f-48e3-9fbe-491f749c7d79 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.067109 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3571/7076] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:08 2022] DELETE /identity/v3/projects/fa421578911a4724b83b1177e8479f63 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:09.071341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3578dd3-511f-48e3-9fbe-491f749c7d79 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:09.072675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e3578dd3-511f-48e3-9fbe-491f749c7d79 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:09.077626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:09.086126 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3506/7077] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:15:09.090388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ab6e7-d89f-419d-b835-08fb5384e08f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:09.100056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ab6e7-d89f-419d-b835-08fb5384e08f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:09.104945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e65a3-e486-4691-aa13-f242578dbc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:09.105569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e65a3-e486-4691-aa13-f242578dbc4d admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:09.105874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e65a3-e486-4691-aa13-f242578dbc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:09.106269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e65a3-e486-4691-aa13-f242578dbc4d admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:09.106981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e65a3-e486-4691-aa13-f242578dbc4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:15:09.107284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e65a3-e486-4691-aa13-f242578dbc4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:09.108877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e65a3-e486-4691-aa13-f242578dbc4d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:09.122539 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3507/7078] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:15:09.126048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e65a3-e486-4691-aa13-f242578dbc4d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:09.137089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e65a3-e486-4691-aa13-f242578dbc4d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:09.142978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3578dd3-511f-48e3-9fbe-491f749c7d79 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:09.144464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2565e4d-ac0a-472c-8350-f995be38f68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:09.145071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2565e4d-ac0a-472c-8350-f995be38f68b admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:09.145350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2565e4d-ac0a-472c-8350-f995be38f68b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:09.145624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2565e4d-ac0a-472c-8350-f995be38f68b admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:09.146801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2565e4d-ac0a-472c-8350-f995be38f68b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:09.147877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a5216-3227-400c-b5ab-1f541366a336 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55615b46899c44249f5940428e5ff261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40d2a6596d6746bea3cab26333dbdb74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:09.148700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2565e4d-ac0a-472c-8350-f995be38f68b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:09.148801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a5216-3227-400c-b5ab-1f541366a336 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:09.149057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a5216-3227-400c-b5ab-1f541366a336 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:09.149321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a5216-3227-400c-b5ab-1f541366a336 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] PATH_INFO: `/v3/domains/6902352497e8459eacd873420303d711` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:09.150590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a5216-3227-400c-b5ab-1f541366a336 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=6902352497e8459eacd873420303d711)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:09.152154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a5216-3227-400c-b5ab-1f541366a336 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:09.177186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e50a5216-3227-400c-b5ab-1f541366a336 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '6902352497e8459eacd873420303d711'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.177953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e50a5216-3227-400c-b5ab-1f541366a336 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '6902352497e8459eacd873420303d711'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.181377 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3508/7079] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:09 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:09.182116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2565e4d-ac0a-472c-8350-f995be38f68b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:09.183699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b2565e4d-ac0a-472c-8350-f995be38f68b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:09.204427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e50a5216-3227-400c-b5ab-1f541366a336 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '6902352497e8459eacd873420303d711'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.205401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e50a5216-3227-400c-b5ab-1f541366a336 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '6902352497e8459eacd873420303d711'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.208239 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3572/7080] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:15:09 2022] PATCH /identity/v3/domains/6902352497e8459eacd873420303d711 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:15:09.212649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a5216-3227-400c-b5ab-1f541366a336 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:09.213995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e50a5216-3227-400c-b5ab-1f541366a336 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:09.252570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2565e4d-ac0a-472c-8350-f995be38f68b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:09.274449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:09.291478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a5216-3227-400c-b5ab-1f541366a336 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:09.302132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55615b46899c44249f5940428e5ff261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40d2a6596d6746bea3cab26333dbdb74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:09.307008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:09.307633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:09.307633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] PATH_INFO: `/v3/domains/6902352497e8459eacd873420303d711` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:09.323786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=6902352497e8459eacd873420303d711)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:09.325894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:09.329961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e68c23-b048-4bf7-adb4-b4cf78e59151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:09.330402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e68c23-b048-4bf7-adb4-b4cf78e59151 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:09.330784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e68c23-b048-4bf7-adb4-b4cf78e59151 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:09.331082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e68c23-b048-4bf7-adb4-b4cf78e59151 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:09.331824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e68c23-b048-4bf7-adb4-b4cf78e59151 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:09.334683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e68c23-b048-4bf7-adb4-b4cf78e59151 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:09.359772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '6902352497e8459eacd873420303d711'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.366014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '6902352497e8459eacd873420303d711'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.372978 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3509/7081] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:09 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:09.374895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e68c23-b048-4bf7-adb4-b4cf78e59151 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:09.378836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-70e68c23-b048-4bf7-adb4-b4cf78e59151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:09.394096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e68c23-b048-4bf7-adb4-b4cf78e59151 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:09.401319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b0f20-1517-49e5-a78f-9b9b80efbef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:09.401997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b0f20-1517-49e5-a78f-9b9b80efbef2 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:09.402313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b0f20-1517-49e5-a78f-9b9b80efbef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:09.402636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b0f20-1517-49e5-a78f-9b9b80efbef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:09.403283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b0f20-1517-49e5-a78f-9b9b80efbef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:09.406053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b0f20-1517-49e5-a78f-9b9b80efbef2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:09.421361 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3510/7082] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:09.422433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b0f20-1517-49e5-a78f-9b9b80efbef2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:09.431129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:UPDATE=1,keystone:DELETE=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:09.434005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b0f20-1517-49e5-a78f-9b9b80efbef2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:09.448174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6902352497e8459eacd873420303d711', 'request_id': 'req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.457786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6902352497e8459eacd873420303d711', 'request_id': 'req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.466265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6902352497e8459eacd873420303d711', 'request_id': 'req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.484248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] The token cache is being invalidate because project 6902352497e8459eacd873420303d711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:09.484658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.485889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.495393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '6902352497e8459eacd873420303d711', 'request_id': 'req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.497603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '6902352497e8459eacd873420303d711', 'request_id': 'req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.498402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '6902352497e8459eacd873420303d711', 'request_id': 'req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.511320 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3573/7083] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:15:09 2022] DELETE /identity/v3/domains/6902352497e8459eacd873420303d711 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:09.517657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:09.519005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc023ac9-6ba1-4c70-853f-71ebd4ff9834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:09.519486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:09.519948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc023ac9-6ba1-4c70-853f-71ebd4ff9834 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:09.520295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc023ac9-6ba1-4c70-853f-71ebd4ff9834 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:09.520636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc023ac9-6ba1-4c70-853f-71ebd4ff9834 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:09.521923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc023ac9-6ba1-4c70-853f-71ebd4ff9834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:09.523813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc023ac9-6ba1-4c70-853f-71ebd4ff9834 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:09.540557 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3511/7084] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:09.541813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc023ac9-6ba1-4c70-853f-71ebd4ff9834 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:09.548436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ba7aa-2a93-4fbd-9cd2-033694334fc3 tempest-TestDefaultProjectId-844681069 tempest-TestDefaultProjectId-844681069-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:09.548566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cc023ac9-6ba1-4c70-853f-71ebd4ff9834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:09.560158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c37ac-c994-4b68-992f-e9d213e2cafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:09.560898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c37ac-c994-4b68-992f-e9d213e2cafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:09.561184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c37ac-c994-4b68-992f-e9d213e2cafd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:09.561456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c37ac-c994-4b68-992f-e9d213e2cafd admin admin] PATH_INFO: `/v3/users/6222fb0a13cb47c9a44173dd3e37b5c4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:09.563285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c37ac-c994-4b68-992f-e9d213e2cafd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6222fb0a13cb47c9a44173dd3e37b5c4)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:09.565503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c37ac-c994-4b68-992f-e9d213e2cafd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:09.580780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc023ac9-6ba1-4c70-853f-71ebd4ff9834 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:09.589650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf48131-f16a-4dd3-9392-a65ff62ce96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:09.592325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf48131-f16a-4dd3-9392-a65ff62ce96b admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:09.592761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf48131-f16a-4dd3-9392-a65ff62ce96b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:09.593123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf48131-f16a-4dd3-9392-a65ff62ce96b admin admin] PATH_INFO: `/v3/projects/4e066910195742e68f06bb037f578789/users/0141ea4ade274df1a6f9780ae2afe252/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:09.615504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf48131-f16a-4dd3-9392-a65ff62ce96b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e066910195742e68f06bb037f578789, user_id=0141ea4ade274df1a6f9780ae2afe252, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:09.621125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf48131-f16a-4dd3-9392-a65ff62ce96b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:09.637905 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3512/7085] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:09 2022] PUT /identity/v3/projects/4e066910195742e68f06bb037f578789/users/0141ea4ade274df1a6f9780ae2afe252/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:09.647911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680ec3f-cde2-4fe9-9366-d5ef2dc7ebea None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:09.648108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-254c37ac-c994-4b68-992f-e9d213e2cafd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6222fb0a13cb47c9a44173dd3e37b5c4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.648195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680ec3f-cde2-4fe9-9366-d5ef2dc7ebea None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:09.648448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680ec3f-cde2-4fe9-9366-d5ef2dc7ebea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:09.655723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-254c37ac-c994-4b68-992f-e9d213e2cafd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6222fb0a13cb47c9a44173dd3e37b5c4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.668516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-254c37ac-c994-4b68-992f-e9d213e2cafd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6222fb0a13cb47c9a44173dd3e37b5c4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.669379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-254c37ac-c994-4b68-992f-e9d213e2cafd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6222fb0a13cb47c9a44173dd3e37b5c4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.680194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-254c37ac-c994-4b68-992f-e9d213e2cafd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6222fb0a13cb47c9a44173dd3e37b5c4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:09.683524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4680ec3f-cde2-4fe9-9366-d5ef2dc7ebea None None] MFA Rules not processed for user `0141ea4ade274df1a6f9780ae2afe252`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:09.684833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4680ec3f-cde2-4fe9-9366-d5ef2dc7ebea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:09.687474 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3574/7086] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:09 2022] DELETE /identity/v3/users/6222fb0a13cb47c9a44173dd3e37b5c4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:09.692058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f2e5a-8756-4179-acdd-c0f7366ee96a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:09.692666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f2e5a-8756-4179-acdd-c0f7366ee96a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:09.692666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f2e5a-8756-4179-acdd-c0f7366ee96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:09.713117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e55f2e5a-8756-4179-acdd-c0f7366ee96a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:09.714319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e55f2e5a-8756-4179-acdd-c0f7366ee96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:09.764305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:09.771872 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3513/7087] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:15:09.778476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680ec3f-cde2-4fe9-9366-d5ef2dc7ebea None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:09.780038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4680ec3f-cde2-4fe9-9366-d5ef2dc7ebea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:09.786217 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3575/7088] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:09.796889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f2e5a-8756-4179-acdd-c0f7366ee96a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:09.798662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e55f2e5a-8756-4179-acdd-c0f7366ee96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:09.837226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:09.875060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680ec3f-cde2-4fe9-9366-d5ef2dc7ebea None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:09.880134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576d201-4386-4953-99b6-eeac80fc1161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:09.881391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576d201-4386-4953-99b6-eeac80fc1161 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:09.881668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576d201-4386-4953-99b6-eeac80fc1161 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:09.881926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576d201-4386-4953-99b6-eeac80fc1161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:09.891626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576d201-4386-4953-99b6-eeac80fc1161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:09.893069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576d201-4386-4953-99b6-eeac80fc1161 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:09.906847 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3514/7089] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:15:09.909754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f2e5a-8756-4179-acdd-c0f7366ee96a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:09.915400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee66a93-4b88-4ea3-a8bc-597de54107eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:09.917292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee66a93-4b88-4ea3-a8bc-597de54107eb service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:09.917514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee66a93-4b88-4ea3-a8bc-597de54107eb service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:09.918133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee66a93-4b88-4ea3-a8bc-597de54107eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:09.933700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee66a93-4b88-4ea3-a8bc-597de54107eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:09.935527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee66a93-4b88-4ea3-a8bc-597de54107eb service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:09.947474 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3576/7090] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:15:10.208212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576d201-4386-4953-99b6-eeac80fc1161 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:10.209683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6576d201-4386-4953-99b6-eeac80fc1161 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:10.295306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576d201-4386-4953-99b6-eeac80fc1161 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:10.303495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27915faa-3338-4a88-b119-17b2b3b9d9e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:10.304389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27915faa-3338-4a88-b119-17b2b3b9d9e2 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:10.304662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27915faa-3338-4a88-b119-17b2b3b9d9e2 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:10.304931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27915faa-3338-4a88-b119-17b2b3b9d9e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:10.306908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-27915faa-3338-4a88-b119-17b2b3b9d9e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:10.323127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27915faa-3338-4a88-b119-17b2b3b9d9e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:10.324871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27915faa-3338-4a88-b119-17b2b3b9d9e2 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:10.338227 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3515/7091] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:15:10.498240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee66a93-4b88-4ea3-a8bc-597de54107eb service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:10.499776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-dee66a93-4b88-4ea3-a8bc-597de54107eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:10.513341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee66a93-4b88-4ea3-a8bc-597de54107eb service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:10.517981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce8485-c61a-4529-8079-e8581ceb9233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:10.518646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce8485-c61a-4529-8079-e8581ceb9233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:10.518973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce8485-c61a-4529-8079-e8581ceb9233 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:10.519199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce8485-c61a-4529-8079-e8581ceb9233 admin admin] PATH_INFO: `/v3/projects/ec4188e52f7b496db0700400cf04b200` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:10.520345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce8485-c61a-4529-8079-e8581ceb9233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec4188e52f7b496db0700400cf04b200)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:10.522269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce8485-c61a-4529-8079-e8581ceb9233 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:10.574158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e1ce8485-c61a-4529-8079-e8581ceb9233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec4188e52f7b496db0700400cf04b200', 'request_id': 'req-e1ce8485-c61a-4529-8079-e8581ceb9233'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:10.580594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e1ce8485-c61a-4529-8079-e8581ceb9233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec4188e52f7b496db0700400cf04b200', 'request_id': 'req-e1ce8485-c61a-4529-8079-e8581ceb9233'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:10.587070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e1ce8485-c61a-4529-8079-e8581ceb9233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec4188e52f7b496db0700400cf04b200', 'request_id': 'req-e1ce8485-c61a-4529-8079-e8581ceb9233'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:10.600335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e1ce8485-c61a-4529-8079-e8581ceb9233 admin admin] The token cache is being invalidate because project ec4188e52f7b496db0700400cf04b200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:10.600640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e1ce8485-c61a-4529-8079-e8581ceb9233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:10.601270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e1ce8485-c61a-4529-8079-e8581ceb9233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:10.605758 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3577/7092] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:10 2022] DELETE /identity/v3/projects/ec4188e52f7b496db0700400cf04b200 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:10.606879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27915faa-3338-4a88-b119-17b2b3b9d9e2 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:10.617936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27915faa-3338-4a88-b119-17b2b3b9d9e2 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:10.685233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff06bed-f6e3-4473-9302-a44ac3c8353d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:10.686305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff06bed-f6e3-4473-9302-a44ac3c8353d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:10.686849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff06bed-f6e3-4473-9302-a44ac3c8353d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:10.687355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff06bed-f6e3-4473-9302-a44ac3c8353d admin admin] PATH_INFO: `/v3/users/55615b46899c44249f5940428e5ff261` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:10.689323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff06bed-f6e3-4473-9302-a44ac3c8353d admin admin] RBAC: Authorizing `identity:delete_user(user_id=55615b46899c44249f5940428e5ff261)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:10.691337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff06bed-f6e3-4473-9302-a44ac3c8353d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:10.743555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:10.756449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-eff06bed-f6e3-4473-9302-a44ac3c8353d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55615b46899c44249f5940428e5ff261'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:10.757842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-eff06bed-f6e3-4473-9302-a44ac3c8353d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55615b46899c44249f5940428e5ff261'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:10.758616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-eff06bed-f6e3-4473-9302-a44ac3c8353d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55615b46899c44249f5940428e5ff261'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:10.772337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-eff06bed-f6e3-4473-9302-a44ac3c8353d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55615b46899c44249f5940428e5ff261'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:10.788105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-eff06bed-f6e3-4473-9302-a44ac3c8353d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55615b46899c44249f5940428e5ff261'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:10.807660 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3516/7093] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:10 2022] DELETE /identity/v3/users/55615b46899c44249f5940428e5ff261 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:11.271751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce8485-c61a-4529-8079-e8581ceb9233 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:11.273291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e1ce8485-c61a-4529-8079-e8581ceb9233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:11.300143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce8485-c61a-4529-8079-e8581ceb9233 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:11.399426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8388a73-8f71-4067-b751-3ba4b100c180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:11.400077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8388a73-8f71-4067-b751-3ba4b100c180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:11.400369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8388a73-8f71-4067-b751-3ba4b100c180 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:11.400648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8388a73-8f71-4067-b751-3ba4b100c180 admin admin] PATH_INFO: `/v3/projects/40d2a6596d6746bea3cab26333dbdb74` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:11.401858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8388a73-8f71-4067-b751-3ba4b100c180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40d2a6596d6746bea3cab26333dbdb74)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:11.403925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8388a73-8f71-4067-b751-3ba4b100c180 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:11.457994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:11.461563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b8388a73-8f71-4067-b751-3ba4b100c180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40d2a6596d6746bea3cab26333dbdb74', 'request_id': 'req-b8388a73-8f71-4067-b751-3ba4b100c180'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:11.472501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b8388a73-8f71-4067-b751-3ba4b100c180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40d2a6596d6746bea3cab26333dbdb74', 'request_id': 'req-b8388a73-8f71-4067-b751-3ba4b100c180'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:11.497678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b8388a73-8f71-4067-b751-3ba4b100c180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40d2a6596d6746bea3cab26333dbdb74', 'request_id': 'req-b8388a73-8f71-4067-b751-3ba4b100c180'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:11.524031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b8388a73-8f71-4067-b751-3ba4b100c180 admin admin] The token cache is being invalidate because project 40d2a6596d6746bea3cab26333dbdb74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:11.524603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b8388a73-8f71-4067-b751-3ba4b100c180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:11.527354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b8388a73-8f71-4067-b751-3ba4b100c180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:11.533691 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3578/7094] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:11 2022] DELETE /identity/v3/projects/40d2a6596d6746bea3cab26333dbdb74 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:11.535675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb001e2-a3cb-46ac-b822-764a7f9e27ec None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:11.536163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb001e2-a3cb-46ac-b822-764a7f9e27ec None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:11.536280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb001e2-a3cb-46ac-b822-764a7f9e27ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:11.569767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-edb001e2-a3cb-46ac-b822-764a7f9e27ec None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:11.571336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-edb001e2-a3cb-46ac-b822-764a7f9e27ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:11.577014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8388a73-8f71-4067-b751-3ba4b100c180 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:11.578587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b8388a73-8f71-4067-b751-3ba4b100c180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:11.654902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8388a73-8f71-4067-b751-3ba4b100c180 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:11.659157 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3517/7095] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:11.659836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66344464-d4c0-475d-ad06-4f33ea657ab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:11.660462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66344464-d4c0-475d-ad06-4f33ea657ab4 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:11.660866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66344464-d4c0-475d-ad06-4f33ea657ab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:11.661155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66344464-d4c0-475d-ad06-4f33ea657ab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:11.663885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-66344464-d4c0-475d-ad06-4f33ea657ab4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:11.667642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1736f7-489d-4fce-9e38-5aa26ee557f0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:11.668407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1736f7-489d-4fce-9e38-5aa26ee557f0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:11.669217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1736f7-489d-4fce-9e38-5aa26ee557f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:11.682237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66344464-d4c0-475d-ad06-4f33ea657ab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:11.685552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66344464-d4c0-475d-ad06-4f33ea657ab4 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:11.698728 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3579/7096] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:15:11.701553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-dd1736f7-489d-4fce-9e38-5aa26ee557f0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:11.703072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-dd1736f7-489d-4fce-9e38-5aa26ee557f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:11.712797 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3518/7097] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:15:11.716814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66344464-d4c0-475d-ad06-4f33ea657ab4 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:11.746046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66344464-d4c0-475d-ad06-4f33ea657ab4 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:11.760621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daef250-bfd1-426c-bfe8-c1788d147659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:11.762434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daef250-bfd1-426c-bfe8-c1788d147659 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:11.762434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daef250-bfd1-426c-bfe8-c1788d147659 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:11.762434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daef250-bfd1-426c-bfe8-c1788d147659 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:11.762807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daef250-bfd1-426c-bfe8-c1788d147659 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:15:11.763119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daef250-bfd1-426c-bfe8-c1788d147659 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:11.765731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daef250-bfd1-426c-bfe8-c1788d147659 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:11.780845 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3580/7098] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:11.785510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1736f7-489d-4fce-9e38-5aa26ee557f0 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:11.797703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1736f7-489d-4fce-9e38-5aa26ee557f0 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:11.804849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2613681-43bb-4110-a954-e49ae64b023d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:11.806195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2613681-43bb-4110-a954-e49ae64b023d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:11.807146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2613681-43bb-4110-a954-e49ae64b023d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:11.807502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2613681-43bb-4110-a954-e49ae64b023d admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:11.808531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2613681-43bb-4110-a954-e49ae64b023d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:11.815663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2613681-43bb-4110-a954-e49ae64b023d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:11.818632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daef250-bfd1-426c-bfe8-c1788d147659 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:11.821174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5daef250-bfd1-426c-bfe8-c1788d147659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:11.868292 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3519/7099] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:11 2022] POST /identity/v3/projects => generated 391 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:11.872117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2613681-43bb-4110-a954-e49ae64b023d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:11.975544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daef250-bfd1-426c-bfe8-c1788d147659 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:12.012869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2613681-43bb-4110-a954-e49ae64b023d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:12.034721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90106f-a33b-4a2f-953e-88ecdb81c924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:12.035351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90106f-a33b-4a2f-953e-88ecdb81c924 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:12.035628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90106f-a33b-4a2f-953e-88ecdb81c924 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:12.035909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90106f-a33b-4a2f-953e-88ecdb81c924 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:12.036588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea90106f-a33b-4a2f-953e-88ecdb81c924 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:12.043007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea90106f-a33b-4a2f-953e-88ecdb81c924 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:12.088631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ddcfd-bee5-4ad2-a7bb-cbf458108232 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:12.089539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ddcfd-bee5-4ad2-a7bb-cbf458108232 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:12.090046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ddcfd-bee5-4ad2-a7bb-cbf458108232 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:12.090405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ddcfd-bee5-4ad2-a7bb-cbf458108232 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:12.092742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bd0ddcfd-bee5-4ad2-a7bb-cbf458108232 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:12.095351 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3520/7100] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:11 2022] POST /identity/v3/users => generated 296 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:12.100314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90106f-a33b-4a2f-953e-88ecdb81c924 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:12.111111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:12.116049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90106f-a33b-4a2f-953e-88ecdb81c924 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:12.117861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ddcfd-bee5-4ad2-a7bb-cbf458108232 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:12.120803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ddcfd-bee5-4ad2-a7bb-cbf458108232 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:12.121959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5793d18-3d9d-40b5-8d48-766a4abda89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:12.122706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5793d18-3d9d-40b5-8d48-766a4abda89d admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:12.123064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5793d18-3d9d-40b5-8d48-766a4abda89d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:12.123485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5793d18-3d9d-40b5-8d48-766a4abda89d admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:12.124873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5793d18-3d9d-40b5-8d48-766a4abda89d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:12.127422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5793d18-3d9d-40b5-8d48-766a4abda89d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:12.141170 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3521/7101] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:12.147415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5793d18-3d9d-40b5-8d48-766a4abda89d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:12.166731 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3581/7102] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:15:12.169813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:12.177164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5793d18-3d9d-40b5-8d48-766a4abda89d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:12.194108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab53a8-3aba-4576-a232-c29d7d8c5072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:12.195003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab53a8-3aba-4576-a232-c29d7d8c5072 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:12.195475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab53a8-3aba-4576-a232-c29d7d8c5072 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:12.195753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab53a8-3aba-4576-a232-c29d7d8c5072 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:12.196404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab53a8-3aba-4576-a232-c29d7d8c5072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:12.198026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab53a8-3aba-4576-a232-c29d7d8c5072 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:12.217611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ddcfd-bee5-4ad2-a7bb-cbf458108232 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:12.225401 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3522/7103] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:12.230390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ddcfd-bee5-4ad2-a7bb-cbf458108232 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:12.235538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c9a75-c8c7-4d51-975a-2e57f799510e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:12.236379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c9a75-c8c7-4d51-975a-2e57f799510e admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:12.236732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c9a75-c8c7-4d51-975a-2e57f799510e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:12.237086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c9a75-c8c7-4d51-975a-2e57f799510e admin admin] PATH_INFO: `/v3/projects/476d81be34d84fd28bef57ae94442665/users/3f6aedb7db234cdf8e665581f76584f7/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:12.259407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c9a75-c8c7-4d51-975a-2e57f799510e admin admin] RBAC: Authorizing `identity:create_grant(project_id=476d81be34d84fd28bef57ae94442665, user_id=3f6aedb7db234cdf8e665581f76584f7, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:12.265014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c9a75-c8c7-4d51-975a-2e57f799510e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:12.300776 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3582/7104] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:12 2022] PUT /identity/v3/projects/476d81be34d84fd28bef57ae94442665/users/3f6aedb7db234cdf8e665581f76584f7/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:12.313120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506a283-7573-41aa-a825-7fd0d50ac6a3 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:12.313120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506a283-7573-41aa-a825-7fd0d50ac6a3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:12.313387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506a283-7573-41aa-a825-7fd0d50ac6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:12.377938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d506a283-7573-41aa-a825-7fd0d50ac6a3 None None] MFA Rules not processed for user `3f6aedb7db234cdf8e665581f76584f7`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:12.379646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d506a283-7573-41aa-a825-7fd0d50ac6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:12.462398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c9a75-c8c7-4d51-975a-2e57f799510e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:12.491383 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3523/7105] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:12.496074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506a283-7573-41aa-a825-7fd0d50ac6a3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:12.553628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c9a75-c8c7-4d51-975a-2e57f799510e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:12.557840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd771e-a512-46e7-a07c-bd146e9d9106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:12.558606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd771e-a512-46e7-a07c-bd146e9d9106 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:12.558962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd771e-a512-46e7-a07c-bd146e9d9106 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:12.559309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd771e-a512-46e7-a07c-bd146e9d9106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:12.561386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f2dd771e-a512-46e7-a07c-bd146e9d9106 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:12.588744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506a283-7573-41aa-a825-7fd0d50ac6a3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:12.595320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfcb407-6a3f-4039-add2-4aab350afd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:12.595895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcb407-6a3f-4039-add2-4aab350afd6f admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:12.596248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcb407-6a3f-4039-add2-4aab350afd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:12.596597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcb407-6a3f-4039-add2-4aab350afd6f admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:12.597338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfcb407-6a3f-4039-add2-4aab350afd6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:12.599622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfcb407-6a3f-4039-add2-4aab350afd6f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:12.632894 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3524/7106] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:12 2022] POST /identity/v3/projects => generated 389 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:12.637937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfcb407-6a3f-4039-add2-4aab350afd6f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:12.650853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd771e-a512-46e7-a07c-bd146e9d9106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:12.652798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd771e-a512-46e7-a07c-bd146e9d9106 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:12.669193 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3583/7107] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:15:12.720391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfcb407-6a3f-4039-add2-4aab350afd6f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:12.725986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265792e-5a35-4905-98a0-c851db214b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:12.726909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265792e-5a35-4905-98a0-c851db214b44 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:12.727366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265792e-5a35-4905-98a0-c851db214b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:12.727730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265792e-5a35-4905-98a0-c851db214b44 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:12.728465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265792e-5a35-4905-98a0-c851db214b44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:12.730529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265792e-5a35-4905-98a0-c851db214b44 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:12.745365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:12.772533 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3525/7108] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:12 2022] POST /identity/v3/users => generated 301 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:12.772900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd771e-a512-46e7-a07c-bd146e9d9106 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:12.788990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd771e-a512-46e7-a07c-bd146e9d9106 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:12.803165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec6fdc3-6563-4dc1-b4ef-260d8cc7173e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:12.803942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6fdc3-6563-4dc1-b4ef-260d8cc7173e admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:12.804233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6fdc3-6563-4dc1-b4ef-260d8cc7173e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:12.804511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6fdc3-6563-4dc1-b4ef-260d8cc7173e admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:12.805248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec6fdc3-6563-4dc1-b4ef-260d8cc7173e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:12.807836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec6fdc3-6563-4dc1-b4ef-260d8cc7173e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:12.820461 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3584/7109] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:12.824716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265792e-5a35-4905-98a0-c851db214b44 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:12.837862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265792e-5a35-4905-98a0-c851db214b44 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:12.848922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00f142-7036-43c2-8a0a-20d3ac3c759d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:12.849809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00f142-7036-43c2-8a0a-20d3ac3c759d admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:12.850214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00f142-7036-43c2-8a0a-20d3ac3c759d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:12.850705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00f142-7036-43c2-8a0a-20d3ac3c759d admin admin] PATH_INFO: `/v3/projects/6abf8585ff6f423ea79b064961d74d4e/users/50d2e5c113f44efea24d6925be461afc/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:12.912248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc00f142-7036-43c2-8a0a-20d3ac3c759d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6abf8585ff6f423ea79b064961d74d4e, user_id=50d2e5c113f44efea24d6925be461afc, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:12.915722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc00f142-7036-43c2-8a0a-20d3ac3c759d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:12.936888 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3526/7110] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:12 2022] PUT /identity/v3/projects/6abf8585ff6f423ea79b064961d74d4e/users/50d2e5c113f44efea24d6925be461afc/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:12.941751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec6fdc3-6563-4dc1-b4ef-260d8cc7173e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:12.960861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec6fdc3-6563-4dc1-b4ef-260d8cc7173e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:13.073353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868d8cb-798b-4cc7-a54b-e72190972487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:13.073926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868d8cb-798b-4cc7-a54b-e72190972487 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:13.074218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868d8cb-798b-4cc7-a54b-e72190972487 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:13.074524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868d8cb-798b-4cc7-a54b-e72190972487 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:13.075171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868d8cb-798b-4cc7-a54b-e72190972487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:13.081974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868d8cb-798b-4cc7-a54b-e72190972487 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:13.105707 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3585/7111] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:13.106140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:13.106851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00f142-7036-43c2-8a0a-20d3ac3c759d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:13.123638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00f142-7036-43c2-8a0a-20d3ac3c759d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:13.141785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1d224-8eab-44cc-ba59-c11f91911bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:13.142767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1d224-8eab-44cc-ba59-c11f91911bb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:13.143154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1d224-8eab-44cc-ba59-c11f91911bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:13.143600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1d224-8eab-44cc-ba59-c11f91911bb2 admin admin] PATH_INFO: `/v3/projects/6abf8585ff6f423ea79b064961d74d4e/users/50d2e5c113f44efea24d6925be461afc/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:13.146665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1d224-8eab-44cc-ba59-c11f91911bb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6abf8585ff6f423ea79b064961d74d4e, user_id=50d2e5c113f44efea24d6925be461afc, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:13.152485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1d224-8eab-44cc-ba59-c11f91911bb2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:13.180585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9acf6-2dbb-4bf9-a7df-f25605170be5 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:13.180990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9acf6-2dbb-4bf9-a7df-f25605170be5 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:13.181539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9acf6-2dbb-4bf9-a7df-f25605170be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:13.185130 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3527/7112] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:13 2022] PUT /identity/v3/projects/6abf8585ff6f423ea79b064961d74d4e/users/50d2e5c113f44efea24d6925be461afc/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:13.220969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c2e9acf6-2dbb-4bf9-a7df-f25605170be5 None None] MFA Rules not processed for user `50d2e5c113f44efea24d6925be461afc`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:13.222557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c2e9acf6-2dbb-4bf9-a7df-f25605170be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:13.319355 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3586/7113] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:13 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:13.324420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1d224-8eab-44cc-ba59-c11f91911bb2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:13.349857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1d224-8eab-44cc-ba59-c11f91911bb2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:13.357824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7cd41-63d4-49b4-a41f-02a541a1f28f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:13.358633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7cd41-63d4-49b4-a41f-02a541a1f28f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:13.363119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7cd41-63d4-49b4-a41f-02a541a1f28f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:13.363456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7cd41-63d4-49b4-a41f-02a541a1f28f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:13.364184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7cd41-63d4-49b4-a41f-02a541a1f28f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:13.366807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7cd41-63d4-49b4-a41f-02a541a1f28f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:13.405195 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3528/7114] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:15:13 2022] POST /identity/v3/domains => generated 276 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:13.410228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9acf6-2dbb-4bf9-a7df-f25605170be5 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:13.432654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9acf6-2dbb-4bf9-a7df-f25605170be5 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:13.577115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5765fb-c422-4b87-92b1-034822b28000 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:13.578829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5765fb-c422-4b87-92b1-034822b28000 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:13.579718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5765fb-c422-4b87-92b1-034822b28000 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:13.581106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5765fb-c422-4b87-92b1-034822b28000 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/f1c42bf0a35a4930b74abc7577572c7c/config` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:13.582048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5765fb-c422-4b87-92b1-034822b28000 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=f1c42bf0a35a4930b74abc7577572c7c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:13.584072 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-df5765fb-c422-4b87-92b1-034822b28000 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:15:13.584072 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:15:13.584072 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:15:13.586678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5765fb-c422-4b87-92b1-034822b28000 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:13.673515 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3587/7115] 10.222.0.17 () {60 vars in 1269 bytes} [Sat Oct 1 21:15:13 2022] PUT /identity/v3/domains/f1c42bf0a35a4930b74abc7577572c7c/config => generated 138 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:13.677691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7cd41-63d4-49b4-a41f-02a541a1f28f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:13.693925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7cd41-63d4-49b4-a41f-02a541a1f28f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:13.704897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bcfb45-130d-42ca-a8f1-e25d87edcebf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:13.707255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcfb45-130d-42ca-a8f1-e25d87edcebf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:13.707756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcfb45-130d-42ca-a8f1-e25d87edcebf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:13.710159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcfb45-130d-42ca-a8f1-e25d87edcebf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/f1c42bf0a35a4930b74abc7577572c7c/config/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:13.711680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bcfb45-130d-42ca-a8f1-e25d87edcebf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=f1c42bf0a35a4930b74abc7577572c7c, group=identity)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:13.712878 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-f6bcfb45-130d-42ca-a8f1-e25d87edcebf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:15:13.712878 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:15:13.712878 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:15:13.714430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bcfb45-130d-42ca-a8f1-e25d87edcebf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:13.726050 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3529/7116] 10.222.0.17 () {58 vars in 1275 bytes} [Sat Oct 1 21:15:13 2022] GET /identity/v3/domains/f1c42bf0a35a4930b74abc7577572c7c/config/identity => generated 45 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:13.730301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5765fb-c422-4b87-92b1-034822b28000 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:13.746413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5765fb-c422-4b87-92b1-034822b28000 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:13.760152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c989ef-af30-4d76-a6ee-6b59b6954dcf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:13.760983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c989ef-af30-4d76-a6ee-6b59b6954dcf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:13.761486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c989ef-af30-4d76-a6ee-6b59b6954dcf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:13.761861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c989ef-af30-4d76-a6ee-6b59b6954dcf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/f1c42bf0a35a4930b74abc7577572c7c/config/identity/driver` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:13.763015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c989ef-af30-4d76-a6ee-6b59b6954dcf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=f1c42bf0a35a4930b74abc7577572c7c, group=identity, option=driver)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:13.764714 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-79c989ef-af30-4d76-a6ee-6b59b6954dcf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:15:13.764714 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:15:13.764714 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:15:13.765607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c989ef-af30-4d76-a6ee-6b59b6954dcf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:13.775544 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3588/7117] 10.222.0.17 () {58 vars in 1296 bytes} [Sat Oct 1 21:15:13 2022] GET /identity/v3/domains/f1c42bf0a35a4930b74abc7577572c7c/config/identity/driver => generated 31 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:13.779769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bcfb45-130d-42ca-a8f1-e25d87edcebf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:13.795859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bcfb45-130d-42ca-a8f1-e25d87edcebf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:13.808238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22face80-ad63-4d34-9a12-cf52113a9c6b tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:13.810844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22face80-ad63-4d34-9a12-cf52113a9c6b tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:13.811600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22face80-ad63-4d34-9a12-cf52113a9c6b tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:13.811948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22face80-ad63-4d34-9a12-cf52113a9c6b tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/f1c42bf0a35a4930b74abc7577572c7c/config/ldap` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:13.815338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22face80-ad63-4d34-9a12-cf52113a9c6b tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=f1c42bf0a35a4930b74abc7577572c7c, group=ldap)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:13.820484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22face80-ad63-4d34-9a12-cf52113a9c6b tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:13.835042 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3530/7118] 10.222.0.17 () {58 vars in 1263 bytes} [Sat Oct 1 21:15:13 2022] GET /identity/v3/domains/f1c42bf0a35a4930b74abc7577572c7c/config/ldap => generated 106 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:13.836313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c989ef-af30-4d76-a6ee-6b59b6954dcf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:13.860871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c989ef-af30-4d76-a6ee-6b59b6954dcf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:13.870620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b95fa-6fe5-4695-885b-ffce63bf4cfc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:13.871702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b95fa-6fe5-4695-885b-ffce63bf4cfc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:13.871702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b95fa-6fe5-4695-885b-ffce63bf4cfc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:13.872117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b95fa-6fe5-4695-885b-ffce63bf4cfc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/f1c42bf0a35a4930b74abc7577572c7c/config/ldap/url` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:13.873386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b95fa-6fe5-4695-885b-ffce63bf4cfc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=f1c42bf0a35a4930b74abc7577572c7c, group=ldap, option=url)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:13.875791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b95fa-6fe5-4695-885b-ffce63bf4cfc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:13.888722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22face80-ad63-4d34-9a12-cf52113a9c6b tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:13.900325 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3589/7119] 10.222.0.17 () {58 vars in 1275 bytes} [Sat Oct 1 21:15:13 2022] GET /identity/v3/domains/f1c42bf0a35a4930b74abc7577572c7c/config/ldap/url => generated 46 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 01 21:15:13.902215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22face80-ad63-4d34-9a12-cf52113a9c6b tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:13.907894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf2725-0adb-45c7-b732-b3ba2b3542c3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:13.908560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf2725-0adb-45c7-b732-b3ba2b3542c3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:13.908844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf2725-0adb-45c7-b732-b3ba2b3542c3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:13.909122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf2725-0adb-45c7-b732-b3ba2b3542c3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/f1c42bf0a35a4930b74abc7577572c7c/config/ldap/user_tree_dn` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:13.910736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf2725-0adb-45c7-b732-b3ba2b3542c3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=f1c42bf0a35a4930b74abc7577572c7c, group=ldap, option=user_tree_dn)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:13.913033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf2725-0adb-45c7-b732-b3ba2b3542c3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:13.920070 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3531/7120] 10.222.0.17 () {58 vars in 1302 bytes} [Sat Oct 1 21:15:13 2022] GET /identity/v3/domains/f1c42bf0a35a4930b74abc7577572c7c/config/ldap/user_tree_dn => generated 63 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:13.924536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b95fa-6fe5-4695-885b-ffce63bf4cfc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:13.943261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b95fa-6fe5-4695-885b-ffce63bf4cfc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:13.952346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61db40-9e38-42d2-bfdf-eac7bd821a6f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:13.953047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61db40-9e38-42d2-bfdf-eac7bd821a6f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:13.953396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61db40-9e38-42d2-bfdf-eac7bd821a6f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:13.954184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61db40-9e38-42d2-bfdf-eac7bd821a6f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/f1c42bf0a35a4930b74abc7577572c7c/config` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:13.954976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61db40-9e38-42d2-bfdf-eac7bd821a6f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=f1c42bf0a35a4930b74abc7577572c7c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:13.956077 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-1c61db40-9e38-42d2-bfdf-eac7bd821a6f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:15:13.956077 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:15:13.956077 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:15:13.957496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61db40-9e38-42d2-bfdf-eac7bd821a6f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:13.979907 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3590/7121] 10.222.0.17 () {58 vars in 1251 bytes} [Sat Oct 1 21:15:13 2022] DELETE /identity/v3/domains/f1c42bf0a35a4930b74abc7577572c7c/config => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:13.984352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf2725-0adb-45c7-b732-b3ba2b3542c3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:13.998476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf2725-0adb-45c7-b732-b3ba2b3542c3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:14.004119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e939f0c-5eed-4636-9ffe-31e4452ffb48 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:14.004384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e939f0c-5eed-4636-9ffe-31e4452ffb48 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:14.004870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e939f0c-5eed-4636-9ffe-31e4452ffb48 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:14.005221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e939f0c-5eed-4636-9ffe-31e4452ffb48 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/f1c42bf0a35a4930b74abc7577572c7c` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:14.007056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e939f0c-5eed-4636-9ffe-31e4452ffb48 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=f1c42bf0a35a4930b74abc7577572c7c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:14.008826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e939f0c-5eed-4636-9ffe-31e4452ffb48 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:14.078672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5e939f0c-5eed-4636-9ffe-31e4452ffb48 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'f1c42bf0a35a4930b74abc7577572c7c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:14.079503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5e939f0c-5eed-4636-9ffe-31e4452ffb48 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'f1c42bf0a35a4930b74abc7577572c7c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:14.135325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5e939f0c-5eed-4636-9ffe-31e4452ffb48 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'f1c42bf0a35a4930b74abc7577572c7c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:14.136203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5e939f0c-5eed-4636-9ffe-31e4452ffb48 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'f1c42bf0a35a4930b74abc7577572c7c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:14.143360 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3532/7122] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:15:13 2022] PATCH /identity/v3/domains/f1c42bf0a35a4930b74abc7577572c7c => generated 277 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:14.149820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61db40-9e38-42d2-bfdf-eac7bd821a6f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:14.155432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1c61db40-9e38-42d2-bfdf-eac7bd821a6f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:14.179377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61db40-9e38-42d2-bfdf-eac7bd821a6f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:14.233299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:14.284034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1b13e-9440-4a8a-a239-cc3504afcb60 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:14.284868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1b13e-9440-4a8a-a239-cc3504afcb60 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:14.285038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1b13e-9440-4a8a-a239-cc3504afcb60 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:14.285324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1b13e-9440-4a8a-a239-cc3504afcb60 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/f1c42bf0a35a4930b74abc7577572c7c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:14.301680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b1b13e-9440-4a8a-a239-cc3504afcb60 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=f1c42bf0a35a4930b74abc7577572c7c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:14.303188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b1b13e-9440-4a8a-a239-cc3504afcb60 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:14.345332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0b1b13e-9440-4a8a-a239-cc3504afcb60 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'f1c42bf0a35a4930b74abc7577572c7c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:14.358187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0b1b13e-9440-4a8a-a239-cc3504afcb60 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'f1c42bf0a35a4930b74abc7577572c7c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:14.429182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0b1b13e-9440-4a8a-a239-cc3504afcb60 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1c42bf0a35a4930b74abc7577572c7c', 'request_id': 'req-a0b1b13e-9440-4a8a-a239-cc3504afcb60'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:14.441210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0b1b13e-9440-4a8a-a239-cc3504afcb60 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1c42bf0a35a4930b74abc7577572c7c', 'request_id': 'req-a0b1b13e-9440-4a8a-a239-cc3504afcb60'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:14.471714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0b1b13e-9440-4a8a-a239-cc3504afcb60 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1c42bf0a35a4930b74abc7577572c7c', 'request_id': 'req-a0b1b13e-9440-4a8a-a239-cc3504afcb60'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:14.501678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0b1b13e-9440-4a8a-a239-cc3504afcb60 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] The token cache is being invalidate because project f1c42bf0a35a4930b74abc7577572c7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:14.502132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0b1b13e-9440-4a8a-a239-cc3504afcb60 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:14.503062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0b1b13e-9440-4a8a-a239-cc3504afcb60 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:14.517175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0b1b13e-9440-4a8a-a239-cc3504afcb60 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'f1c42bf0a35a4930b74abc7577572c7c', 'request_id': 'req-a0b1b13e-9440-4a8a-a239-cc3504afcb60'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:14.518055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0b1b13e-9440-4a8a-a239-cc3504afcb60 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'f1c42bf0a35a4930b74abc7577572c7c', 'request_id': 'req-a0b1b13e-9440-4a8a-a239-cc3504afcb60'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:14.518788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0b1b13e-9440-4a8a-a239-cc3504afcb60 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'f1c42bf0a35a4930b74abc7577572c7c', 'request_id': 'req-a0b1b13e-9440-4a8a-a239-cc3504afcb60'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:14.529727 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3591/7123] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:15:14 2022] DELETE /identity/v3/domains/f1c42bf0a35a4930b74abc7577572c7c => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:14.535257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e939f0c-5eed-4636-9ffe-31e4452ffb48 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:14.547494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e939f0c-5eed-4636-9ffe-31e4452ffb48 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:14.640973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade9503-c9c1-4110-bc33-80c2d79f6273 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:14.641799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade9503-c9c1-4110-bc33-80c2d79f6273 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:14.642228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade9503-c9c1-4110-bc33-80c2d79f6273 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:14.642651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade9503-c9c1-4110-bc33-80c2d79f6273 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:14.643514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade9503-c9c1-4110-bc33-80c2d79f6273 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:14.646701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade9503-c9c1-4110-bc33-80c2d79f6273 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:14.673987 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3533/7124] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:15:14 2022] POST /identity/v3/domains => generated 274 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:14.676002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1b13e-9440-4a8a-a239-cc3504afcb60 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:14.678321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a0b1b13e-9440-4a8a-a239-cc3504afcb60 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:14.696466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1b13e-9440-4a8a-a239-cc3504afcb60 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:14.803642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce4e17-19a9-496d-9d16-c1239886c1b3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:14.804642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce4e17-19a9-496d-9d16-c1239886c1b3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:14.816686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce4e17-19a9-496d-9d16-c1239886c1b3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:14.818374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce4e17-19a9-496d-9d16-c1239886c1b3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/e24a461ad43444b4a6c8f689c06d983a/config` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:14.819344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce4e17-19a9-496d-9d16-c1239886c1b3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=e24a461ad43444b4a6c8f689c06d983a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:14.821491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce4e17-19a9-496d-9d16-c1239886c1b3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:14.837882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:14.889670 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3592/7125] 10.222.0.17 () {60 vars in 1269 bytes} [Sat Oct 1 21:15:14 2022] PUT /identity/v3/domains/e24a461ad43444b4a6c8f689c06d983a/config => generated 138 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:14.894796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade9503-c9c1-4110-bc33-80c2d79f6273 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:14.910533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade9503-c9c1-4110-bc33-80c2d79f6273 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:14.934164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d607350-62c8-49b3-8b94-2bf316b93b81 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:14.935026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d607350-62c8-49b3-8b94-2bf316b93b81 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:14.939507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d607350-62c8-49b3-8b94-2bf316b93b81 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:14.940785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d607350-62c8-49b3-8b94-2bf316b93b81 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/e24a461ad43444b4a6c8f689c06d983a/config` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:14.941706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d607350-62c8-49b3-8b94-2bf316b93b81 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=e24a461ad43444b4a6c8f689c06d983a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:14.945968 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-7d607350-62c8-49b3-8b94-2bf316b93b81 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:15:14.945968 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:15:14.945968 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:15:14.948638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d607350-62c8-49b3-8b94-2bf316b93b81 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:15.037392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:UPDATE=1,keystone:DELETE=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:15.047314 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3534/7126] 10.222.0.17 () {60 vars in 1271 bytes} [Sat Oct 1 21:15:14 2022] PATCH /identity/v3/domains/e24a461ad43444b4a6c8f689c06d983a/config => generated 142 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:15:15.051975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce4e17-19a9-496d-9d16-c1239886c1b3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:15.080048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce4e17-19a9-496d-9d16-c1239886c1b3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:15.092160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e84366-61d8-448f-bce8-311cdc79b419 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:15.099668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e84366-61d8-448f-bce8-311cdc79b419 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:15.100483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e84366-61d8-448f-bce8-311cdc79b419 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:15.100955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e84366-61d8-448f-bce8-311cdc79b419 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/e24a461ad43444b4a6c8f689c06d983a/config` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:15.103839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e84366-61d8-448f-bce8-311cdc79b419 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=e24a461ad43444b4a6c8f689c06d983a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:15.107876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e84366-61d8-448f-bce8-311cdc79b419 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:15.116986 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3593/7127] 10.222.0.17 () {58 vars in 1248 bytes} [Sat Oct 1 21:15:15 2022] GET /identity/v3/domains/e24a461ad43444b4a6c8f689c06d983a/config => generated 142 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:15:15.120854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d607350-62c8-49b3-8b94-2bf316b93b81 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:15.136839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d607350-62c8-49b3-8b94-2bf316b93b81 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:15.142218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade77366-6288-4766-9cb1-f4ee78001b3d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:15.142840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade77366-6288-4766-9cb1-f4ee78001b3d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:15.143122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade77366-6288-4766-9cb1-f4ee78001b3d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:15.143398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade77366-6288-4766-9cb1-f4ee78001b3d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/e24a461ad43444b4a6c8f689c06d983a/config` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:15.144623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade77366-6288-4766-9cb1-f4ee78001b3d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=e24a461ad43444b4a6c8f689c06d983a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:15.145706 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-ade77366-6288-4766-9cb1-f4ee78001b3d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:15:15.145706 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:15:15.145706 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:15:15.147301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade77366-6288-4766-9cb1-f4ee78001b3d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:15.167846 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3535/7128] 10.222.0.17 () {58 vars in 1251 bytes} [Sat Oct 1 21:15:15 2022] DELETE /identity/v3/domains/e24a461ad43444b4a6c8f689c06d983a/config => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:15.169357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e84366-61d8-448f-bce8-311cdc79b419 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:15.182613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e84366-61d8-448f-bce8-311cdc79b419 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:15.191246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35890f2c-ef02-4e27-875d-5081ec45f5f5 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:15.191968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35890f2c-ef02-4e27-875d-5081ec45f5f5 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:15.192295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35890f2c-ef02-4e27-875d-5081ec45f5f5 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:15.192579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35890f2c-ef02-4e27-875d-5081ec45f5f5 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/e24a461ad43444b4a6c8f689c06d983a/config` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:15.194296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35890f2c-ef02-4e27-875d-5081ec45f5f5 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=e24a461ad43444b4a6c8f689c06d983a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:15.196597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35890f2c-ef02-4e27-875d-5081ec45f5f5 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:15.207772 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-35890f2c-ef02-4e27-875d-5081ec45f5f5 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Could not find any options in domain configuration for domain e24a461ad43444b4a6c8f689c06d983a.: keystone.exception.DomainConfigNotFound: Could not find any options in domain configuration for domain e24a461ad43444b4a6c8f689c06d983a. Oct 01 21:15:15.207772 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:15.207772 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:15:15.207772 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:15:15.207772 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:15:15.207772 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:15:15.207772 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:15:15.207772 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:15:15.207772 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:15:15.207772 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:15:15.207772 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:15:15.207772 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:15:15.207772 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Oct 01 21:15:15.207772 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Oct 01 21:15:15.209389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Oct 01 21:15:15.209389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Oct 01 21:15:15.209389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:15:15.209389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:15:15.209389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Oct 01 21:15:15.209389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Oct 01 21:15:15.209389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find any options in domain configuration for domain e24a461ad43444b4a6c8f689c06d983a. Oct 01 21:15:15.209389 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:15:15.214143 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3594/7129] 10.222.0.17 () {58 vars in 1248 bytes} [Sat Oct 1 21:15:15 2022] GET /identity/v3/domains/e24a461ad43444b4a6c8f689c06d983a/config => generated 151 bytes in 41 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:15:15.217577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade77366-6288-4766-9cb1-f4ee78001b3d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:15.243129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade77366-6288-4766-9cb1-f4ee78001b3d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:15.252398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657da13-dfc8-406f-b772-79aa619f6248 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:15.253224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657da13-dfc8-406f-b772-79aa619f6248 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:15.253535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657da13-dfc8-406f-b772-79aa619f6248 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:15.253917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657da13-dfc8-406f-b772-79aa619f6248 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/e24a461ad43444b4a6c8f689c06d983a/config` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:15.254675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657da13-dfc8-406f-b772-79aa619f6248 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=e24a461ad43444b4a6c8f689c06d983a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:15.258540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657da13-dfc8-406f-b772-79aa619f6248 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:15.278801 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3536/7130] 10.222.0.17 () {58 vars in 1251 bytes} [Sat Oct 1 21:15:15 2022] DELETE /identity/v3/domains/e24a461ad43444b4a6c8f689c06d983a/config => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:15.280336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35890f2c-ef02-4e27-875d-5081ec45f5f5 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:15.316995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35890f2c-ef02-4e27-875d-5081ec45f5f5 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:15.326145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6314467-1869-4a88-bb30-ca20cc11b8c6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:15.327184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6314467-1869-4a88-bb30-ca20cc11b8c6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:15.328774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6314467-1869-4a88-bb30-ca20cc11b8c6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:15.329222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6314467-1869-4a88-bb30-ca20cc11b8c6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/e24a461ad43444b4a6c8f689c06d983a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:15.332253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6314467-1869-4a88-bb30-ca20cc11b8c6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=e24a461ad43444b4a6c8f689c06d983a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:15.335930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6314467-1869-4a88-bb30-ca20cc11b8c6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:15.394125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a6314467-1869-4a88-bb30-ca20cc11b8c6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'e24a461ad43444b4a6c8f689c06d983a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:15.394651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a6314467-1869-4a88-bb30-ca20cc11b8c6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'e24a461ad43444b4a6c8f689c06d983a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:15.428469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a6314467-1869-4a88-bb30-ca20cc11b8c6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'e24a461ad43444b4a6c8f689c06d983a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:15.429200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a6314467-1869-4a88-bb30-ca20cc11b8c6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'e24a461ad43444b4a6c8f689c06d983a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:15.432008 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3595/7131] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:15:15 2022] PATCH /identity/v3/domains/e24a461ad43444b4a6c8f689c06d983a => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:15.435943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657da13-dfc8-406f-b772-79aa619f6248 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:15.437888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d657da13-dfc8-406f-b772-79aa619f6248 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:15.452633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657da13-dfc8-406f-b772-79aa619f6248 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:15.519760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d096d-e8ae-4054-8172-ece087b2119a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:15.520576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d096d-e8ae-4054-8172-ece087b2119a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:15.520661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d096d-e8ae-4054-8172-ece087b2119a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:15.521317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d096d-e8ae-4054-8172-ece087b2119a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/e24a461ad43444b4a6c8f689c06d983a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:15.533400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d096d-e8ae-4054-8172-ece087b2119a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=e24a461ad43444b4a6c8f689c06d983a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:15.535857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d096d-e8ae-4054-8172-ece087b2119a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:15.563498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-df9d096d-e8ae-4054-8172-ece087b2119a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'e24a461ad43444b4a6c8f689c06d983a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:15.587634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-df9d096d-e8ae-4054-8172-ece087b2119a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'e24a461ad43444b4a6c8f689c06d983a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:15.646451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-df9d096d-e8ae-4054-8172-ece087b2119a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e24a461ad43444b4a6c8f689c06d983a', 'request_id': 'req-df9d096d-e8ae-4054-8172-ece087b2119a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:15.656037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-df9d096d-e8ae-4054-8172-ece087b2119a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e24a461ad43444b4a6c8f689c06d983a', 'request_id': 'req-df9d096d-e8ae-4054-8172-ece087b2119a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:15.663231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-df9d096d-e8ae-4054-8172-ece087b2119a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e24a461ad43444b4a6c8f689c06d983a', 'request_id': 'req-df9d096d-e8ae-4054-8172-ece087b2119a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:15.676715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-df9d096d-e8ae-4054-8172-ece087b2119a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] The token cache is being invalidate because project e24a461ad43444b4a6c8f689c06d983a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:15.677022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-df9d096d-e8ae-4054-8172-ece087b2119a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:15.678229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-df9d096d-e8ae-4054-8172-ece087b2119a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:15.689065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-df9d096d-e8ae-4054-8172-ece087b2119a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'e24a461ad43444b4a6c8f689c06d983a', 'request_id': 'req-df9d096d-e8ae-4054-8172-ece087b2119a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:15.689803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-df9d096d-e8ae-4054-8172-ece087b2119a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'e24a461ad43444b4a6c8f689c06d983a', 'request_id': 'req-df9d096d-e8ae-4054-8172-ece087b2119a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:15.694557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-df9d096d-e8ae-4054-8172-ece087b2119a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'e24a461ad43444b4a6c8f689c06d983a', 'request_id': 'req-df9d096d-e8ae-4054-8172-ece087b2119a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:15.700573 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3537/7132] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:15:15 2022] DELETE /identity/v3/domains/e24a461ad43444b4a6c8f689c06d983a => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:15.708089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6314467-1869-4a88-bb30-ca20cc11b8c6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:15.720328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6314467-1869-4a88-bb30-ca20cc11b8c6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:15.811955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fbab0-7e15-4b34-8344-7990ab40d4ed tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:15.812735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fbab0-7e15-4b34-8344-7990ab40d4ed tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:15.813092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fbab0-7e15-4b34-8344-7990ab40d4ed tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:15.813589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fbab0-7e15-4b34-8344-7990ab40d4ed tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:15.814443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00fbab0-7e15-4b34-8344-7990ab40d4ed tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:15.817053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00fbab0-7e15-4b34-8344-7990ab40d4ed tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:15.841686 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3596/7133] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:15:15 2022] POST /identity/v3/domains => generated 275 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:15.843042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d096d-e8ae-4054-8172-ece087b2119a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:15.844633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-df9d096d-e8ae-4054-8172-ece087b2119a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:15.857999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d096d-e8ae-4054-8172-ece087b2119a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:15.871906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:15.922231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7299db4-84fe-456c-9b26-dbace9a7f272 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:15.922964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7299db4-84fe-456c-9b26-dbace9a7f272 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:15.923362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7299db4-84fe-456c-9b26-dbace9a7f272 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:15.923644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7299db4-84fe-456c-9b26-dbace9a7f272 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/beb063b82edc46adac6f7350b2c1d79c/config` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:15.924328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7299db4-84fe-456c-9b26-dbace9a7f272 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=beb063b82edc46adac6f7350b2c1d79c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:15.925358 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-c7299db4-84fe-456c-9b26-dbace9a7f272 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:15:15.925358 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:15:15.925358 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:15:15.926285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7299db4-84fe-456c-9b26-dbace9a7f272 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:15.967326 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3538/7134] 10.222.0.17 () {60 vars in 1269 bytes} [Sat Oct 1 21:15:15 2022] PUT /identity/v3/domains/beb063b82edc46adac6f7350b2c1d79c/config => generated 138 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:15.968758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fbab0-7e15-4b34-8344-7990ab40d4ed tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:15.979981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fbab0-7e15-4b34-8344-7990ab40d4ed tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:15.984816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91564fcb-8099-4b72-b94f-6a65bd557e02 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:15.985484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91564fcb-8099-4b72-b94f-6a65bd557e02 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:15.985769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91564fcb-8099-4b72-b94f-6a65bd557e02 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:15.986049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91564fcb-8099-4b72-b94f-6a65bd557e02 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/beb063b82edc46adac6f7350b2c1d79c/config/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:15.987029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91564fcb-8099-4b72-b94f-6a65bd557e02 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=beb063b82edc46adac6f7350b2c1d79c, group=identity)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:15.988110 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-91564fcb-8099-4b72-b94f-6a65bd557e02 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:15:15.988110 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:15:15.988110 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:15:15.989020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91564fcb-8099-4b72-b94f-6a65bd557e02 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:16.027757 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3597/7135] 10.222.0.17 () {60 vars in 1297 bytes} [Sat Oct 1 21:15:15 2022] PATCH /identity/v3/domains/beb063b82edc46adac6f7350b2c1d79c/config/identity => generated 176 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:16.029178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7299db4-84fe-456c-9b26-dbace9a7f272 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:16.040296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7299db4-84fe-456c-9b26-dbace9a7f272 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:16.045812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48437e-d7a9-4817-b227-563b29cfef9f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:16.046514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48437e-d7a9-4817-b227-563b29cfef9f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:16.046879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48437e-d7a9-4817-b227-563b29cfef9f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:16.047955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48437e-d7a9-4817-b227-563b29cfef9f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/beb063b82edc46adac6f7350b2c1d79c/config/identity/driver` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:16.049114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c48437e-d7a9-4817-b227-563b29cfef9f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=beb063b82edc46adac6f7350b2c1d79c, group=identity, option=driver)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:16.051347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c48437e-d7a9-4817-b227-563b29cfef9f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:16.085182 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3539/7136] 10.222.0.17 () {60 vars in 1318 bytes} [Sat Oct 1 21:15:16 2022] PATCH /identity/v3/domains/beb063b82edc46adac6f7350b2c1d79c/config/identity/driver => generated 176 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:16.089286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91564fcb-8099-4b72-b94f-6a65bd557e02 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:16.100380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91564fcb-8099-4b72-b94f-6a65bd557e02 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:16.105275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a6a91-bbdc-41ef-8a64-f8d519d165c3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:16.105952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a6a91-bbdc-41ef-8a64-f8d519d165c3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:16.106239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a6a91-bbdc-41ef-8a64-f8d519d165c3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:16.106551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a6a91-bbdc-41ef-8a64-f8d519d165c3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/beb063b82edc46adac6f7350b2c1d79c/config/identity/driver` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:16.107166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a6a91-bbdc-41ef-8a64-f8d519d165c3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=beb063b82edc46adac6f7350b2c1d79c, group=identity, option=driver)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:16.108781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a6a91-bbdc-41ef-8a64-f8d519d165c3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:16.126304 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3598/7137] 10.222.0.17 () {58 vars in 1299 bytes} [Sat Oct 1 21:15:16 2022] DELETE /identity/v3/domains/beb063b82edc46adac6f7350b2c1d79c/config/identity/driver => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:16.131735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48437e-d7a9-4817-b227-563b29cfef9f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:16.142617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48437e-d7a9-4817-b227-563b29cfef9f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:16.148167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031ecf9-78ba-4d6f-bb73-0c9db203bd97 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:16.148837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031ecf9-78ba-4d6f-bb73-0c9db203bd97 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:16.149126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031ecf9-78ba-4d6f-bb73-0c9db203bd97 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:16.149405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031ecf9-78ba-4d6f-bb73-0c9db203bd97 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/beb063b82edc46adac6f7350b2c1d79c/config/identity/driver` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:16.150603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b031ecf9-78ba-4d6f-bb73-0c9db203bd97 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=beb063b82edc46adac6f7350b2c1d79c, group=identity, option=driver)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:16.152273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b031ecf9-78ba-4d6f-bb73-0c9db203bd97 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:16.161183 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-b031ecf9-78ba-4d6f-bb73-0c9db203bd97 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Could not find option driver in group identity in domain configuration for domain beb063b82edc46adac6f7350b2c1d79c.: keystone.exception.DomainConfigNotFound: Could not find option driver in group identity in domain configuration for domain beb063b82edc46adac6f7350b2c1d79c. Oct 01 21:15:16.161183 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:16.161183 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:15:16.161183 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:15:16.161183 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:15:16.161183 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:15:16.161183 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:15:16.161183 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:15:16.161183 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:15:16.161183 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:15:16.161183 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:15:16.161183 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:15:16.161183 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Oct 01 21:15:16.162116 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Oct 01 21:15:16.162116 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Oct 01 21:15:16.162116 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Oct 01 21:15:16.162116 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:15:16.162116 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:15:16.162116 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Oct 01 21:15:16.162116 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Oct 01 21:15:16.162116 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find option driver in group identity in domain configuration for domain beb063b82edc46adac6f7350b2c1d79c. Oct 01 21:15:16.162116 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:15:16.162680 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3540/7138] 10.222.0.17 () {58 vars in 1296 bytes} [Sat Oct 1 21:15:16 2022] GET /identity/v3/domains/beb063b82edc46adac6f7350b2c1d79c/config/identity/driver => generated 171 bytes in 32 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:15:16.166516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a6a91-bbdc-41ef-8a64-f8d519d165c3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:16.177232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a6a91-bbdc-41ef-8a64-f8d519d165c3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:16.183658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fe6d1-bf85-443c-9593-efac8cffeeb2 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:16.184313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fe6d1-bf85-443c-9593-efac8cffeeb2 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:16.184599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fe6d1-bf85-443c-9593-efac8cffeeb2 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:16.184966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fe6d1-bf85-443c-9593-efac8cffeeb2 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/beb063b82edc46adac6f7350b2c1d79c/config/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:16.185562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fe6d1-bf85-443c-9593-efac8cffeeb2 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=beb063b82edc46adac6f7350b2c1d79c, group=identity)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:16.187204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fe6d1-bf85-443c-9593-efac8cffeeb2 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:16.197584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=4,keystone:UPDATE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:16.204853 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3599/7139] 10.222.0.17 () {58 vars in 1278 bytes} [Sat Oct 1 21:15:16 2022] DELETE /identity/v3/domains/beb063b82edc46adac6f7350b2c1d79c/config/identity => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:16.209389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031ecf9-78ba-4d6f-bb73-0c9db203bd97 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:16.220335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031ecf9-78ba-4d6f-bb73-0c9db203bd97 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:16.224945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79136bfc-c7b1-4caa-929d-cf4bbc8a0776 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:16.225586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79136bfc-c7b1-4caa-929d-cf4bbc8a0776 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:16.226172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79136bfc-c7b1-4caa-929d-cf4bbc8a0776 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:16.226172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79136bfc-c7b1-4caa-929d-cf4bbc8a0776 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/beb063b82edc46adac6f7350b2c1d79c/config/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:16.227357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79136bfc-c7b1-4caa-929d-cf4bbc8a0776 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=beb063b82edc46adac6f7350b2c1d79c, group=identity)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:16.228866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79136bfc-c7b1-4caa-929d-cf4bbc8a0776 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:16.237672 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-79136bfc-c7b1-4caa-929d-cf4bbc8a0776 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Could not find group identity in domain configuration for domain beb063b82edc46adac6f7350b2c1d79c.: keystone.exception.DomainConfigNotFound: Could not find group identity in domain configuration for domain beb063b82edc46adac6f7350b2c1d79c. Oct 01 21:15:16.237672 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:16.237672 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:15:16.237672 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:15:16.237672 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:15:16.237672 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:15:16.237672 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:15:16.237672 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:15:16.237672 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:15:16.237672 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:15:16.237672 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:15:16.237672 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:15:16.237672 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Oct 01 21:15:16.238671 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Oct 01 21:15:16.238671 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Oct 01 21:15:16.238671 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Oct 01 21:15:16.238671 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:15:16.238671 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:15:16.238671 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Oct 01 21:15:16.238671 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Oct 01 21:15:16.238671 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find group identity in domain configuration for domain beb063b82edc46adac6f7350b2c1d79c. Oct 01 21:15:16.238671 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:15:16.239243 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3541/7140] 10.222.0.17 () {58 vars in 1275 bytes} [Sat Oct 1 21:15:16 2022] GET /identity/v3/domains/beb063b82edc46adac6f7350b2c1d79c/config/identity => generated 154 bytes in 31 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:15:16.243415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fe6d1-bf85-443c-9593-efac8cffeeb2 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:16.253751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fe6d1-bf85-443c-9593-efac8cffeeb2 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:16.259019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e955376-08bd-437c-a815-3cd2dffe926d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:16.259714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e955376-08bd-437c-a815-3cd2dffe926d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:16.260063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e955376-08bd-437c-a815-3cd2dffe926d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:16.260403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e955376-08bd-437c-a815-3cd2dffe926d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/beb063b82edc46adac6f7350b2c1d79c/config` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:16.261230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e955376-08bd-437c-a815-3cd2dffe926d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=beb063b82edc46adac6f7350b2c1d79c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:16.263295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e955376-08bd-437c-a815-3cd2dffe926d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:16.273636 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3600/7141] 10.222.0.17 () {58 vars in 1251 bytes} [Sat Oct 1 21:15:16 2022] DELETE /identity/v3/domains/beb063b82edc46adac6f7350b2c1d79c/config => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:16.280835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79136bfc-c7b1-4caa-929d-cf4bbc8a0776 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:16.296370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79136bfc-c7b1-4caa-929d-cf4bbc8a0776 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:16.305707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ed07a-a203-4835-bbc3-74a61cb48cb9 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:16.306448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ed07a-a203-4835-bbc3-74a61cb48cb9 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:16.306911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ed07a-a203-4835-bbc3-74a61cb48cb9 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:16.307173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ed07a-a203-4835-bbc3-74a61cb48cb9 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/beb063b82edc46adac6f7350b2c1d79c` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:16.308547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ed07a-a203-4835-bbc3-74a61cb48cb9 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=beb063b82edc46adac6f7350b2c1d79c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:16.310278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ed07a-a203-4835-bbc3-74a61cb48cb9 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:16.339928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-081ed07a-a203-4835-bbc3-74a61cb48cb9 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'beb063b82edc46adac6f7350b2c1d79c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:16.340677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-081ed07a-a203-4835-bbc3-74a61cb48cb9 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'beb063b82edc46adac6f7350b2c1d79c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:16.380131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-081ed07a-a203-4835-bbc3-74a61cb48cb9 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'beb063b82edc46adac6f7350b2c1d79c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:16.381147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-081ed07a-a203-4835-bbc3-74a61cb48cb9 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'beb063b82edc46adac6f7350b2c1d79c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:16.385141 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3542/7142] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:15:16 2022] PATCH /identity/v3/domains/beb063b82edc46adac6f7350b2c1d79c => generated 276 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:16.389847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e955376-08bd-437c-a815-3cd2dffe926d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:16.391845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2e955376-08bd-437c-a815-3cd2dffe926d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:16.411039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e955376-08bd-437c-a815-3cd2dffe926d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:16.512205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2c11a-ba97-4fad-af9c-c3e81d681920 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:16.512856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2c11a-ba97-4fad-af9c-c3e81d681920 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:16.513138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2c11a-ba97-4fad-af9c-c3e81d681920 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:16.513408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2c11a-ba97-4fad-af9c-c3e81d681920 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/beb063b82edc46adac6f7350b2c1d79c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:16.531380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2c11a-ba97-4fad-af9c-c3e81d681920 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=beb063b82edc46adac6f7350b2c1d79c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:16.533169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2c11a-ba97-4fad-af9c-c3e81d681920 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:16.568441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-63d2c11a-ba97-4fad-af9c-c3e81d681920 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'beb063b82edc46adac6f7350b2c1d79c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:16.575590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-63d2c11a-ba97-4fad-af9c-c3e81d681920 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'beb063b82edc46adac6f7350b2c1d79c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:16.634729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-63d2c11a-ba97-4fad-af9c-c3e81d681920 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'beb063b82edc46adac6f7350b2c1d79c', 'request_id': 'req-63d2c11a-ba97-4fad-af9c-c3e81d681920'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:16.642071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-63d2c11a-ba97-4fad-af9c-c3e81d681920 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'beb063b82edc46adac6f7350b2c1d79c', 'request_id': 'req-63d2c11a-ba97-4fad-af9c-c3e81d681920'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:16.648986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-63d2c11a-ba97-4fad-af9c-c3e81d681920 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'beb063b82edc46adac6f7350b2c1d79c', 'request_id': 'req-63d2c11a-ba97-4fad-af9c-c3e81d681920'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:16.668516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-63d2c11a-ba97-4fad-af9c-c3e81d681920 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] The token cache is being invalidate because project beb063b82edc46adac6f7350b2c1d79c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:16.668815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-63d2c11a-ba97-4fad-af9c-c3e81d681920 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:16.669540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-63d2c11a-ba97-4fad-af9c-c3e81d681920 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:16.679064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-63d2c11a-ba97-4fad-af9c-c3e81d681920 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'beb063b82edc46adac6f7350b2c1d79c', 'request_id': 'req-63d2c11a-ba97-4fad-af9c-c3e81d681920'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:16.679932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-63d2c11a-ba97-4fad-af9c-c3e81d681920 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'beb063b82edc46adac6f7350b2c1d79c', 'request_id': 'req-63d2c11a-ba97-4fad-af9c-c3e81d681920'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:16.680449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-63d2c11a-ba97-4fad-af9c-c3e81d681920 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'beb063b82edc46adac6f7350b2c1d79c', 'request_id': 'req-63d2c11a-ba97-4fad-af9c-c3e81d681920'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:16.693806 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3601/7143] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:15:16 2022] DELETE /identity/v3/domains/beb063b82edc46adac6f7350b2c1d79c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:16.700161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ed07a-a203-4835-bbc3-74a61cb48cb9 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:16.713312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ed07a-a203-4835-bbc3-74a61cb48cb9 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:16.784967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2580ae-7320-4d9e-8e1e-d8afd20b4abf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:16.785741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2580ae-7320-4d9e-8e1e-d8afd20b4abf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:16.786030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2580ae-7320-4d9e-8e1e-d8afd20b4abf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:16.786306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2580ae-7320-4d9e-8e1e-d8afd20b4abf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:16.787616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2580ae-7320-4d9e-8e1e-d8afd20b4abf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:16.788894 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-5b2580ae-7320-4d9e-8e1e-d8afd20b4abf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config_default": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:15:16.788894 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:15:16.788894 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:15:16.790401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2580ae-7320-4d9e-8e1e-d8afd20b4abf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:16.797835 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3543/7144] 10.222.0.17 () {58 vars in 1173 bytes} [Sat Oct 1 21:15:16 2022] GET /identity/v3/domains/config/default => generated 1527 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:16.799904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2c11a-ba97-4fad-af9c-c3e81d681920 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:16.801648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-63d2c11a-ba97-4fad-af9c-c3e81d681920 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:16.816617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2c11a-ba97-4fad-af9c-c3e81d681920 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:16.827893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f62fe-6e31-43f9-bfff-bd6820ce2177 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:16.829807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f62fe-6e31-43f9-bfff-bd6820ce2177 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:16.830094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f62fe-6e31-43f9-bfff-bd6820ce2177 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:16.830390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f62fe-6e31-43f9-bfff-bd6820ce2177 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/identity/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:16.831317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f62fe-6e31-43f9-bfff-bd6820ce2177 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:16.832663 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-f46f62fe-6e31-43f9-bfff-bd6820ce2177 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config_default": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:15:16.832663 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:15:16.832663 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:15:16.834074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f62fe-6e31-43f9-bfff-bd6820ce2177 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:16.836000 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3602/7145] 10.222.0.17 () {58 vars in 1200 bytes} [Sat Oct 1 21:15:16 2022] GET /identity/v3/domains/config/identity/default => generated 64 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 01 21:15:16.839466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2580ae-7320-4d9e-8e1e-d8afd20b4abf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:16.848036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:16.855895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2580ae-7320-4d9e-8e1e-d8afd20b4abf tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:16.876542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f163e78-4a1c-4f8e-8fd1-747cab679b4d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:16.877729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f163e78-4a1c-4f8e-8fd1-747cab679b4d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:16.878047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f163e78-4a1c-4f8e-8fd1-747cab679b4d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:16.878412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f163e78-4a1c-4f8e-8fd1-747cab679b4d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/identity/driver/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:16.879265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f163e78-4a1c-4f8e-8fd1-747cab679b4d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity, option=driver)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:16.880879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f62fe-6e31-43f9-bfff-bd6820ce2177 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:16.882204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f163e78-4a1c-4f8e-8fd1-747cab679b4d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:16.884108 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3544/7146] 10.222.0.17 () {58 vars in 1221 bytes} [Sat Oct 1 21:15:16 2022] GET /identity/v3/domains/config/identity/driver/default => generated 30 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:16.884992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f46f62fe-6e31-43f9-bfff-bd6820ce2177 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:16.887783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f163e78-4a1c-4f8e-8fd1-747cab679b4d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:16.902775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f163e78-4a1c-4f8e-8fd1-747cab679b4d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:16.911416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d85c4-ae19-4c1e-9413-c8d51ade1bc7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:16.912335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d85c4-ae19-4c1e-9413-c8d51ade1bc7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:16.912756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d85c4-ae19-4c1e-9413-c8d51ade1bc7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:16.913222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d85c4-ae19-4c1e-9413-c8d51ade1bc7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/identity/list_limit/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:16.914153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d85c4-ae19-4c1e-9413-c8d51ade1bc7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity, option=list_limit)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:16.916175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d85c4-ae19-4c1e-9413-c8d51ade1bc7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:16.918936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:16.930333 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3545/7147] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:16 2022] GET /identity/v3/domains/config/identity/list_limit/default => generated 33 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:16.933891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d85c4-ae19-4c1e-9413-c8d51ade1bc7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:16.952255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d85c4-ae19-4c1e-9413-c8d51ade1bc7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:16.960318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e8dd7-b747-4356-a558-fa36ce28458f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:16.960973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e8dd7-b747-4356-a558-fa36ce28458f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:16.961551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e8dd7-b747-4356-a558-fa36ce28458f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:16.961882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e8dd7-b747-4356-a558-fa36ce28458f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:16.962626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e8dd7-b747-4356-a558-fa36ce28458f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:16.964425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e8dd7-b747-4356-a558-fa36ce28458f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:16.966620 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3546/7148] 10.222.0.17 () {58 vars in 1188 bytes} [Sat Oct 1 21:15:16 2022] GET /identity/v3/domains/config/ldap/default => generated 1476 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:15:16.974701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e8dd7-b747-4356-a558-fa36ce28458f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:16.986721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e8dd7-b747-4356-a558-fa36ce28458f tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:16.992305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45069608-89ff-4fe0-af4a-da89e9139ba8 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:16.993457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45069608-89ff-4fe0-af4a-da89e9139ba8 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:16.993457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45069608-89ff-4fe0-af4a-da89e9139ba8 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:16.993997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45069608-89ff-4fe0-af4a-da89e9139ba8 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/url/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:16.994758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45069608-89ff-4fe0-af4a-da89e9139ba8 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=url)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:16.996725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45069608-89ff-4fe0-af4a-da89e9139ba8 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.014608 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3547/7149] 10.222.0.17 () {58 vars in 1200 bytes} [Sat Oct 1 21:15:16 2022] GET /identity/v3/domains/config/ldap/url/default => generated 40 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:17.014851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f62fe-6e31-43f9-bfff-bd6820ce2177 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.014956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45069608-89ff-4fe0-af4a-da89e9139ba8 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.032357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1083f-b6c7-4184-945d-078a6719271f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.032357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1083f-b6c7-4184-945d-078a6719271f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.032357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1083f-b6c7-4184-945d-078a6719271f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.032357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1083f-b6c7-4184-945d-078a6719271f admin admin] PATH_INFO: `/v3/users/39af6e701f564a2888b09d1eafe3478d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.032855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1083f-b6c7-4184-945d-078a6719271f admin admin] RBAC: Authorizing `identity:delete_user(user_id=39af6e701f564a2888b09d1eafe3478d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.032855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1083f-b6c7-4184-945d-078a6719271f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.033010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45069608-89ff-4fe0-af4a-da89e9139ba8 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.049950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13832bac-9d66-4351-bb08-295664b9dc13 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.051350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13832bac-9d66-4351-bb08-295664b9dc13 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.051706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13832bac-9d66-4351-bb08-295664b9dc13 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.052740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13832bac-9d66-4351-bb08-295664b9dc13 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/user/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.053562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13832bac-9d66-4351-bb08-295664b9dc13 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.056775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13832bac-9d66-4351-bb08-295664b9dc13 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.058097 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3548/7150] 10.222.0.17 () {58 vars in 1203 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/domains/config/ldap/user/default => generated 27 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:17.062113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13832bac-9d66-4351-bb08-295664b9dc13 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.077963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13832bac-9d66-4351-bb08-295664b9dc13 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.084444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae44efd-a0bf-4d61-922f-ebcc36bbcb75 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.085496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae44efd-a0bf-4d61-922f-ebcc36bbcb75 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.085863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae44efd-a0bf-4d61-922f-ebcc36bbcb75 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.086211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae44efd-a0bf-4d61-922f-ebcc36bbcb75 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/suffix/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.086902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae44efd-a0bf-4d61-922f-ebcc36bbcb75 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=suffix)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.088568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae44efd-a0bf-4d61-922f-ebcc36bbcb75 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.089836 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3549/7151] 10.222.0.17 () {58 vars in 1209 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/domains/config/ldap/suffix/default => generated 44 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 01 21:15:17.094255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae44efd-a0bf-4d61-922f-ebcc36bbcb75 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.105155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae44efd-a0bf-4d61-922f-ebcc36bbcb75 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.108027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-00a1083f-b6c7-4184-945d-078a6719271f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39af6e701f564a2888b09d1eafe3478d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:17.109804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483329e-7a66-47b5-921e-0132f735dee1 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.110448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483329e-7a66-47b5-921e-0132f735dee1 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.110753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483329e-7a66-47b5-921e-0132f735dee1 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.111148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483329e-7a66-47b5-921e-0132f735dee1 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/query_scope/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.111724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483329e-7a66-47b5-921e-0132f735dee1 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=query_scope)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.113300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483329e-7a66-47b5-921e-0132f735dee1 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.114533 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3550/7152] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/domains/config/ldap/query_scope/default => generated 35 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 01 21:15:17.116603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-00a1083f-b6c7-4184-945d-078a6719271f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39af6e701f564a2888b09d1eafe3478d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:17.119625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483329e-7a66-47b5-921e-0132f735dee1 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.135116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-00a1083f-b6c7-4184-945d-078a6719271f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39af6e701f564a2888b09d1eafe3478d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:17.135418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483329e-7a66-47b5-921e-0132f735dee1 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.135886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-00a1083f-b6c7-4184-945d-078a6719271f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39af6e701f564a2888b09d1eafe3478d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:17.140460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12435d-fc50-4781-8edf-4ede05b30f16 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.141170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12435d-fc50-4781-8edf-4ede05b30f16 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.141607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12435d-fc50-4781-8edf-4ede05b30f16 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.141723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12435d-fc50-4781-8edf-4ede05b30f16 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/page_size/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.142559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12435d-fc50-4781-8edf-4ede05b30f16 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=page_size)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.144171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12435d-fc50-4781-8edf-4ede05b30f16 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.145130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-00a1083f-b6c7-4184-945d-078a6719271f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39af6e701f564a2888b09d1eafe3478d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:17.145595 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3551/7153] 10.222.0.17 () {58 vars in 1218 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/domains/config/ldap/page_size/default => generated 29 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 01 21:15:17.149115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12435d-fc50-4781-8edf-4ede05b30f16 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.151077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3d12435d-fc50-4781-8edf-4ede05b30f16 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:17.155630 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3603/7154] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:16 2022] DELETE /identity/v3/users/39af6e701f564a2888b09d1eafe3478d => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:17.159642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f561e9-ec63-4c0a-888e-838182aa145b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.160699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f561e9-ec63-4c0a-888e-838182aa145b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.160699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f561e9-ec63-4c0a-888e-838182aa145b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.165090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12435d-fc50-4781-8edf-4ede05b30f16 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.183851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b6f561e9-ec63-4c0a-888e-838182aa145b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:17.185175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b6f561e9-ec63-4c0a-888e-838182aa145b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:17.241211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab96893-f7be-4f32-8507-efecb2c0bfad tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.241861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab96893-f7be-4f32-8507-efecb2c0bfad tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.242383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab96893-f7be-4f32-8507-efecb2c0bfad tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.242816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab96893-f7be-4f32-8507-efecb2c0bfad tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/alias_dereferencing/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.243486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab96893-f7be-4f32-8507-efecb2c0bfad tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=alias_dereferencing)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.245307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab96893-f7be-4f32-8507-efecb2c0bfad tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.246574 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3552/7155] 10.222.0.17 () {58 vars in 1248 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/domains/config/ldap/alias_dereferencing/default => generated 47 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:17.247341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab96893-f7be-4f32-8507-efecb2c0bfad tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.248908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0ab96893-f7be-4f32-8507-efecb2c0bfad tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:17.253598 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3604/7156] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:15:17.254406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f561e9-ec63-4c0a-888e-838182aa145b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.265344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f561e9-ec63-4c0a-888e-838182aa145b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.266151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab96893-f7be-4f32-8507-efecb2c0bfad tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.270226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d8660-6cd0-440e-b75c-d5079da2921e tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.271004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d8660-6cd0-440e-b75c-d5079da2921e tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.271178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18710043-33cf-4d1e-b4d3-b0b5678038fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.271288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d8660-6cd0-440e-b75c-d5079da2921e tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.271550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d8660-6cd0-440e-b75c-d5079da2921e tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/debug_level/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.271811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18710043-33cf-4d1e-b4d3-b0b5678038fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.272096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18710043-33cf-4d1e-b4d3-b0b5678038fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.272207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d8660-6cd0-440e-b75c-d5079da2921e tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=debug_level)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.272397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18710043-33cf-4d1e-b4d3-b0b5678038fa admin admin] PATH_INFO: `/v3/users/0141ea4ade274df1a6f9780ae2afe252` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.273567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18710043-33cf-4d1e-b4d3-b0b5678038fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=0141ea4ade274df1a6f9780ae2afe252)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.273705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d8660-6cd0-440e-b75c-d5079da2921e tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.277641 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3605/7157] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/domains/config/ldap/debug_level/default => generated 34 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:17.277765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18710043-33cf-4d1e-b4d3-b0b5678038fa admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.277857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d8660-6cd0-440e-b75c-d5079da2921e tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.278759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-156d8660-6cd0-440e-b75c-d5079da2921e tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:17.324391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-18710043-33cf-4d1e-b4d3-b0b5678038fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0141ea4ade274df1a6f9780ae2afe252'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:17.324868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:17.325378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-18710043-33cf-4d1e-b4d3-b0b5678038fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0141ea4ade274df1a6f9780ae2afe252'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:17.326412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-18710043-33cf-4d1e-b4d3-b0b5678038fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0141ea4ade274df1a6f9780ae2afe252'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:17.335943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-18710043-33cf-4d1e-b4d3-b0b5678038fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0141ea4ade274df1a6f9780ae2afe252'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:17.346698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d8660-6cd0-440e-b75c-d5079da2921e tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.348332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-18710043-33cf-4d1e-b4d3-b0b5678038fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0141ea4ade274df1a6f9780ae2afe252'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:17.351154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e2b47-9063-497a-81b6-ac0c4a80feef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.351757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e2b47-9063-497a-81b6-ac0c4a80feef service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.352044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e2b47-9063-497a-81b6-ac0c4a80feef service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.352322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e2b47-9063-497a-81b6-ac0c4a80feef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.354139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e85e2b47-9063-497a-81b6-ac0c4a80feef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:17.360930 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3553/7158] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:17 2022] DELETE /identity/v3/users/0141ea4ade274df1a6f9780ae2afe252 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:17.361962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18710043-33cf-4d1e-b4d3-b0b5678038fa admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.363536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-18710043-33cf-4d1e-b4d3-b0b5678038fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:17.369363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e2b47-9063-497a-81b6-ac0c4a80feef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.370944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e2b47-9063-497a-81b6-ac0c4a80feef service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.381413 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3606/7159] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:15:17.382683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32582a-d944-4baa-a1c0-4b35052481e6 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.382969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32582a-d944-4baa-a1c0-4b35052481e6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.383244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32582a-d944-4baa-a1c0-4b35052481e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.405602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7b32582a-d944-4baa-a1c0-4b35052481e6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:17.406972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7b32582a-d944-4baa-a1c0-4b35052481e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:17.429672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:17.454089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18710043-33cf-4d1e-b4d3-b0b5678038fa admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.459636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5dc4bb-542c-491a-8303-1a346ff3f014 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.460231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dc4bb-542c-491a-8303-1a346ff3f014 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.460518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dc4bb-542c-491a-8303-1a346ff3f014 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.461029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dc4bb-542c-491a-8303-1a346ff3f014 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/chase_referrals/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.461553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5dc4bb-542c-491a-8303-1a346ff3f014 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=chase_referrals)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.463248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5dc4bb-542c-491a-8303-1a346ff3f014 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.464354 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3554/7160] 10.222.0.17 () {58 vars in 1236 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/domains/config/ldap/chase_referrals/default => generated 38 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:17.468786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5dc4bb-542c-491a-8303-1a346ff3f014 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.481136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5dc4bb-542c-491a-8303-1a346ff3f014 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.487298 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3607/7161] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:17.487439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d4bf1-c85b-497c-901d-6bb26abe07b3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.487439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d4bf1-c85b-497c-901d-6bb26abe07b3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.487439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d4bf1-c85b-497c-901d-6bb26abe07b3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.487747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d4bf1-c85b-497c-901d-6bb26abe07b3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_tree_dn/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.487908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d4bf1-c85b-497c-901d-6bb26abe07b3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_tree_dn)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.489529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d4bf1-c85b-497c-901d-6bb26abe07b3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.491302 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3555/7162] 10.222.0.17 () {58 vars in 1227 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/domains/config/ldap/user_tree_dn/default => generated 35 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:17.495128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32582a-d944-4baa-a1c0-4b35052481e6 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.498005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d4bf1-c85b-497c-901d-6bb26abe07b3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.499889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c67d4bf1-c85b-497c-901d-6bb26abe07b3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:17.507163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32582a-d944-4baa-a1c0-4b35052481e6 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.512593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dda110-6872-49ab-b25e-81ba4f5d8f42 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.513254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dda110-6872-49ab-b25e-81ba4f5d8f42 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.513943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dda110-6872-49ab-b25e-81ba4f5d8f42 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.513943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dda110-6872-49ab-b25e-81ba4f5d8f42 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_filter/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.514401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dda110-6872-49ab-b25e-81ba4f5d8f42 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_filter)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.516044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dda110-6872-49ab-b25e-81ba4f5d8f42 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.517222 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3608/7163] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/domains/config/ldap/user_filter/default => generated 34 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 01 21:15:17.521599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dda110-6872-49ab-b25e-81ba4f5d8f42 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.532064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dda110-6872-49ab-b25e-81ba4f5d8f42 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.537175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c943902-4114-4092-ae76-19f705ea2065 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.537944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c943902-4114-4092-ae76-19f705ea2065 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.538358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c943902-4114-4092-ae76-19f705ea2065 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.539164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c943902-4114-4092-ae76-19f705ea2065 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_objectclass/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.539475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c943902-4114-4092-ae76-19f705ea2065 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_objectclass)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.541156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c943902-4114-4092-ae76-19f705ea2065 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.545259 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3609/7164] 10.222.0.17 () {58 vars in 1239 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/domains/config/ldap/user_objectclass/default => generated 50 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:17.557970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c943902-4114-4092-ae76-19f705ea2065 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.574247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c943902-4114-4092-ae76-19f705ea2065 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.577107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d4bf1-c85b-497c-901d-6bb26abe07b3 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.581054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf43f66-6850-4e7a-bb53-ad409c62d1bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.581747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf43f66-6850-4e7a-bb53-ad409c62d1bb service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.582102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf43f66-6850-4e7a-bb53-ad409c62d1bb service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.582450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf43f66-6850-4e7a-bb53-ad409c62d1bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.582934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5da9ee-73fc-4616-8250-a97cbfe439f7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.584144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5da9ee-73fc-4616-8250-a97cbfe439f7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.584427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5da9ee-73fc-4616-8250-a97cbfe439f7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.584700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5da9ee-73fc-4616-8250-a97cbfe439f7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_id_attribute/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.585404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5da9ee-73fc-4616-8250-a97cbfe439f7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_id_attribute)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.587008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5da9ee-73fc-4616-8250-a97cbfe439f7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.588386 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3610/7165] 10.222.0.17 () {58 vars in 1242 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/domains/config/ldap/user_id_attribute/default => generated 40 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 01 21:15:17.592112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5da9ee-73fc-4616-8250-a97cbfe439f7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.594013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf43f66-6850-4e7a-bb53-ad409c62d1bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.595499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf43f66-6850-4e7a-bb53-ad409c62d1bb service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.604113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5da9ee-73fc-4616-8250-a97cbfe439f7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.606440 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3556/7166] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:15:17.609548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a16b5e-5cf6-4d22-a2bc-f262447f9bb0 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.610183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a16b5e-5cf6-4d22-a2bc-f262447f9bb0 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.610471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a16b5e-5cf6-4d22-a2bc-f262447f9bb0 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.610784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a16b5e-5cf6-4d22-a2bc-f262447f9bb0 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_name_attribute/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.611370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a16b5e-5cf6-4d22-a2bc-f262447f9bb0 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_name_attribute)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.612938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a16b5e-5cf6-4d22-a2bc-f262447f9bb0 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.614119 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3611/7167] 10.222.0.17 () {58 vars in 1248 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/domains/config/ldap/user_name_attribute/default => generated 42 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 01 21:15:17.618851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf43f66-6850-4e7a-bb53-ad409c62d1bb service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.630327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf43f66-6850-4e7a-bb53-ad409c62d1bb service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.641489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9963f7d-292c-495d-a463-5941c62729a8 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.642171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9963f7d-292c-495d-a463-5941c62729a8 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.642454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9963f7d-292c-495d-a463-5941c62729a8 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.642769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9963f7d-292c-495d-a463-5941c62729a8 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_mail_attribute/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.643365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9963f7d-292c-495d-a463-5941c62729a8 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_mail_attribute)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.645000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9963f7d-292c-495d-a463-5941c62729a8 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.646221 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3557/7168] 10.222.0.17 () {58 vars in 1248 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/domains/config/ldap/user_mail_attribute/default => generated 44 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:17.650358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a16b5e-5cf6-4d22-a2bc-f262447f9bb0 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.662879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a16b5e-5cf6-4d22-a2bc-f262447f9bb0 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.669647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a235a5-921a-4834-b110-00bb7b1aedcc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.670624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a235a5-921a-4834-b110-00bb7b1aedcc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.671102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a235a5-921a-4834-b110-00bb7b1aedcc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.671568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a235a5-921a-4834-b110-00bb7b1aedcc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_description_attribute/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.672429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a235a5-921a-4834-b110-00bb7b1aedcc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_description_attribute)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.674583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a235a5-921a-4834-b110-00bb7b1aedcc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.676603 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3612/7169] 10.222.0.17 () {58 vars in 1269 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/domains/config/ldap/user_description_attribute/default => generated 58 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:17.680968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9963f7d-292c-495d-a463-5941c62729a8 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.699808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9963f7d-292c-495d-a463-5941c62729a8 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.706181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f50829-1f56-4405-a287-025584e00857 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.707311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f50829-1f56-4405-a287-025584e00857 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.707841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f50829-1f56-4405-a287-025584e00857 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.708232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f50829-1f56-4405-a287-025584e00857 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_pass_attribute/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.709612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f50829-1f56-4405-a287-025584e00857 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_pass_attribute)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.711339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f50829-1f56-4405-a287-025584e00857 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.712784 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3558/7170] 10.222.0.17 () {58 vars in 1248 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/domains/config/ldap/user_pass_attribute/default => generated 52 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:17.718814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a235a5-921a-4834-b110-00bb7b1aedcc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.718974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f50829-1f56-4405-a287-025584e00857 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.721803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-25f50829-1f56-4405-a287-025584e00857 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:17.733462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a235a5-921a-4834-b110-00bb7b1aedcc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.740993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f50829-1f56-4405-a287-025584e00857 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.741870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d417db-e208-4119-bb81-fe81782a36f6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.743376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d417db-e208-4119-bb81-fe81782a36f6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.743776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d417db-e208-4119-bb81-fe81782a36f6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.744257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d417db-e208-4119-bb81-fe81782a36f6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_attribute/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.747799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4e297-d606-40dc-9583-626c704037f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.748677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4e297-d606-40dc-9583-626c704037f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.748874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4e297-d606-40dc-9583-626c704037f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.749076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4e297-d606-40dc-9583-626c704037f3 admin admin] PATH_INFO: `/v3/projects/aa31359109754fcba8ad848702356b65` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.751757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d417db-e208-4119-bb81-fe81782a36f6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_attribute)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.754151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d417db-e208-4119-bb81-fe81782a36f6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.755082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c4e297-d606-40dc-9583-626c704037f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa31359109754fcba8ad848702356b65)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.759241 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3613/7171] 10.222.0.17 () {58 vars in 1257 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/domains/config/ldap/user_enabled_attribute/default => generated 50 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:17.760107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d417db-e208-4119-bb81-fe81782a36f6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.760306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c4e297-d606-40dc-9583-626c704037f3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.772615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d417db-e208-4119-bb81-fe81782a36f6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.778431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16036e-6aed-4cc4-8f87-bcead3c185a1 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.779494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16036e-6aed-4cc4-8f87-bcead3c185a1 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.779930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16036e-6aed-4cc4-8f87-bcead3c185a1 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.780357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16036e-6aed-4cc4-8f87-bcead3c185a1 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_invert/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.781257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16036e-6aed-4cc4-8f87-bcead3c185a1 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_invert)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.783205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16036e-6aed-4cc4-8f87-bcead3c185a1 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.784604 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3614/7172] 10.222.0.17 () {58 vars in 1248 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/domains/config/ldap/user_enabled_invert/default => generated 43 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:17.788329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16036e-6aed-4cc4-8f87-bcead3c185a1 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.799962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16036e-6aed-4cc4-8f87-bcead3c185a1 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.805394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bde333-76b8-4937-a7df-fa169f9a5c97 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.806040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bde333-76b8-4937-a7df-fa169f9a5c97 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.806508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bde333-76b8-4937-a7df-fa169f9a5c97 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.806801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bde333-76b8-4937-a7df-fa169f9a5c97 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_mask/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.807399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bde333-76b8-4937-a7df-fa169f9a5c97 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_mask)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.808988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bde333-76b8-4937-a7df-fa169f9a5c97 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.810362 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3615/7173] 10.222.0.17 () {58 vars in 1242 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/domains/config/ldap/user_enabled_mask/default => generated 37 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 01 21:15:17.814875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bde333-76b8-4937-a7df-fa169f9a5c97 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.823370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-64c4e297-d606-40dc-9583-626c704037f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa31359109754fcba8ad848702356b65', 'request_id': 'req-64c4e297-d606-40dc-9583-626c704037f3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:17.825915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bde333-76b8-4937-a7df-fa169f9a5c97 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:17.832672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-64c4e297-d606-40dc-9583-626c704037f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa31359109754fcba8ad848702356b65', 'request_id': 'req-64c4e297-d606-40dc-9583-626c704037f3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:17.841747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-64c4e297-d606-40dc-9583-626c704037f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa31359109754fcba8ad848702356b65', 'request_id': 'req-64c4e297-d606-40dc-9583-626c704037f3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:17.855546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-64c4e297-d606-40dc-9583-626c704037f3 admin admin] The token cache is being invalidate because project aa31359109754fcba8ad848702356b65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:17.856271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-64c4e297-d606-40dc-9583-626c704037f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:17.857386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-64c4e297-d606-40dc-9583-626c704037f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:17.859657 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3559/7174] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:17 2022] DELETE /identity/v3/projects/aa31359109754fcba8ad848702356b65 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:17.864446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c65ad4-5e8a-477b-8fc0-6e8e34776d0d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.865279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c65ad4-5e8a-477b-8fc0-6e8e34776d0d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.865565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c65ad4-5e8a-477b-8fc0-6e8e34776d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.897999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-92c65ad4-5e8a-477b-8fc0-6e8e34776d0d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:17.899541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-92c65ad4-5e8a-477b-8fc0-6e8e34776d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:17.917970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed730e4c-777c-480f-a766-3dba9573dca6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:17.918773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed730e4c-777c-480f-a766-3dba9573dca6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:17.919422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed730e4c-777c-480f-a766-3dba9573dca6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:17.919607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed730e4c-777c-480f-a766-3dba9573dca6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_default/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:17.920290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed730e4c-777c-480f-a766-3dba9573dca6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:17.924832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed730e4c-777c-480f-a766-3dba9573dca6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:17.924832 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3616/7175] 10.222.0.17 () {58 vars in 1251 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/domains/config/ldap/user_enabled_default/default => generated 45 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:17.927821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed730e4c-777c-480f-a766-3dba9573dca6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.930965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ed730e4c-777c-480f-a766-3dba9573dca6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:17.967950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:17.978988 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3560/7176] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:17.979819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c65ad4-5e8a-477b-8fc0-6e8e34776d0d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:17.981819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-92c65ad4-5e8a-477b-8fc0-6e8e34776d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:17.991881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:17.995396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c65ad4-5e8a-477b-8fc0-6e8e34776d0d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.000273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d811cae-8d77-427f-901d-2cac93b505b4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.000920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d811cae-8d77-427f-901d-2cac93b505b4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.001204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d811cae-8d77-427f-901d-2cac93b505b4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.001482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d811cae-8d77-427f-901d-2cac93b505b4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_attribute_ignore/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.002065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d811cae-8d77-427f-901d-2cac93b505b4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_attribute_ignore)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.003681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d811cae-8d77-427f-901d-2cac93b505b4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.004765 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3561/7177] 10.222.0.17 () {58 vars in 1254 bytes} [Sat Oct 1 21:15:17 2022] GET /identity/v3/domains/config/ldap/user_attribute_ignore/default => generated 62 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:18.006522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b3b23-c668-422f-83b1-61083d5be6b4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.006900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b3b23-c668-422f-83b1-61083d5be6b4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.007181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b3b23-c668-422f-83b1-61083d5be6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.010807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed730e4c-777c-480f-a766-3dba9573dca6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.015649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe67e9d-d66d-4104-bc7b-90076f85aa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.016159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe67e9d-d66d-4104-bc7b-90076f85aa9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.018105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe67e9d-d66d-4104-bc7b-90076f85aa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.018105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe67e9d-d66d-4104-bc7b-90076f85aa9a admin admin] PATH_INFO: `/v3/projects/4e066910195742e68f06bb037f578789` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.018880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe67e9d-d66d-4104-bc7b-90076f85aa9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e066910195742e68f06bb037f578789)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.020854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe67e9d-d66d-4104-bc7b-90076f85aa9a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.027052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-cc7b3b23-c668-422f-83b1-61083d5be6b4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:18.028253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cc7b3b23-c668-422f-83b1-61083d5be6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:18.050126 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3562/7178] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:18.050126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b3b23-c668-422f-83b1-61083d5be6b4 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.062252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b3b23-c668-422f-83b1-61083d5be6b4 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.076911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dbe67e9d-d66d-4104-bc7b-90076f85aa9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e066910195742e68f06bb037f578789', 'request_id': 'req-dbe67e9d-d66d-4104-bc7b-90076f85aa9a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:18.087381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dbe67e9d-d66d-4104-bc7b-90076f85aa9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e066910195742e68f06bb037f578789', 'request_id': 'req-dbe67e9d-d66d-4104-bc7b-90076f85aa9a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:18.093606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dbe67e9d-d66d-4104-bc7b-90076f85aa9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e066910195742e68f06bb037f578789', 'request_id': 'req-dbe67e9d-d66d-4104-bc7b-90076f85aa9a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:18.112119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dbe67e9d-d66d-4104-bc7b-90076f85aa9a admin admin] The token cache is being invalidate because project 4e066910195742e68f06bb037f578789 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:18.112779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dbe67e9d-d66d-4104-bc7b-90076f85aa9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:18.113531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dbe67e9d-d66d-4104-bc7b-90076f85aa9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:18.119182 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3617/7179] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:17 2022] DELETE /identity/v3/projects/4e066910195742e68f06bb037f578789 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:18.119182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe67e9d-d66d-4104-bc7b-90076f85aa9a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.119182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-dbe67e9d-d66d-4104-bc7b-90076f85aa9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:18.134431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f367fb9-bb54-4117-b70c-6e41f48562be tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.135127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f367fb9-bb54-4117-b70c-6e41f48562be tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.135410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f367fb9-bb54-4117-b70c-6e41f48562be tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.135688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f367fb9-bb54-4117-b70c-6e41f48562be tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_default_project_id_attribute/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.136426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f367fb9-bb54-4117-b70c-6e41f48562be tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_default_project_id_attribute)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.138030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f367fb9-bb54-4117-b70c-6e41f48562be tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.139154 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3563/7180] 10.222.0.17 () {58 vars in 1290 bytes} [Sat Oct 1 21:15:18 2022] GET /identity/v3/domains/config/ldap/user_default_project_id_attribute/default => generated 56 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:18.142570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86c5f1-b1de-40ec-9b8a-bef1fa2529f6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.142570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86c5f1-b1de-40ec-9b8a-bef1fa2529f6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.142570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86c5f1-b1de-40ec-9b8a-bef1fa2529f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.161706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ba86c5f1-b1de-40ec-9b8a-bef1fa2529f6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:18.162952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ba86c5f1-b1de-40ec-9b8a-bef1fa2529f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:18.200014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe67e9d-d66d-4104-bc7b-90076f85aa9a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.206301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac302b0c-6fde-4e6b-bbc2-0484bce96a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.207078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac302b0c-6fde-4e6b-bbc2-0484bce96a69 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.207466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac302b0c-6fde-4e6b-bbc2-0484bce96a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.207846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac302b0c-6fde-4e6b-bbc2-0484bce96a69 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.208767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac302b0c-6fde-4e6b-bbc2-0484bce96a69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:15:18.209337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac302b0c-6fde-4e6b-bbc2-0484bce96a69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.211073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac302b0c-6fde-4e6b-bbc2-0484bce96a69 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.225232 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3618/7181] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:18.226072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac302b0c-6fde-4e6b-bbc2-0484bce96a69 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.227799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ac302b0c-6fde-4e6b-bbc2-0484bce96a69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:18.241565 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3564/7182] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:18.241807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac302b0c-6fde-4e6b-bbc2-0484bce96a69 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.242680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86c5f1-b1de-40ec-9b8a-bef1fa2529f6 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.246778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c709b-6f96-4b6e-96de-83496a14becc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.247421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c709b-6f96-4b6e-96de-83496a14becc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.247704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c709b-6f96-4b6e-96de-83496a14becc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.247975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c709b-6f96-4b6e-96de-83496a14becc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.248649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c709b-6f96-4b6e-96de-83496a14becc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.250228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c709b-6f96-4b6e-96de-83496a14becc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.251640 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3619/7183] 10.222.0.17 () {58 vars in 1257 bytes} [Sat Oct 1 21:15:18 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation/default => generated 46 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:18.252614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc31d33-12ab-456c-929e-8b1a7c6f0304 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.252891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc31d33-12ab-456c-929e-8b1a7c6f0304 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.253383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc31d33-12ab-456c-929e-8b1a7c6f0304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.258465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86c5f1-b1de-40ec-9b8a-bef1fa2529f6 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.263386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e7540-65d8-49ce-8030-4c844687f7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.263987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e7540-65d8-49ce-8030-4c844687f7b4 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.264271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e7540-65d8-49ce-8030-4c844687f7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.264548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e7540-65d8-49ce-8030-4c844687f7b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.265202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e7540-65d8-49ce-8030-4c844687f7b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.267177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e7540-65d8-49ce-8030-4c844687f7b4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.273989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5cc31d33-12ab-456c-929e-8b1a7c6f0304 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:18.275270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5cc31d33-12ab-456c-929e-8b1a7c6f0304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:18.288900 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3620/7184] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:18.288900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc31d33-12ab-456c-929e-8b1a7c6f0304 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.298306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc31d33-12ab-456c-929e-8b1a7c6f0304 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.300247 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3565/7185] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:18 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:18.301224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e7540-65d8-49ce-8030-4c844687f7b4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.358749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74bc6b-6a63-4f77-a416-ea1a64ce90ae tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.359402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74bc6b-6a63-4f77-a416-ea1a64ce90ae tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.359690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74bc6b-6a63-4f77-a416-ea1a64ce90ae tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.359963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74bc6b-6a63-4f77-a416-ea1a64ce90ae tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation_dn/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.360552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e74bc6b-6a63-4f77-a416-ea1a64ce90ae tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation_dn)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.362159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e74bc6b-6a63-4f77-a416-ea1a64ce90ae tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.366988 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3621/7186] 10.222.0.17 () {58 vars in 1266 bytes} [Sat Oct 1 21:15:18 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation_dn/default => generated 48 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:18.366988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74bc6b-6a63-4f77-a416-ea1a64ce90ae tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.367255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e7540-65d8-49ce-8030-4c844687f7b4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.370941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd668b04-a5da-4fd5-b193-432aa9cc5af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.371565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd668b04-a5da-4fd5-b193-432aa9cc5af1 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.371829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd668b04-a5da-4fd5-b193-432aa9cc5af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.372106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd668b04-a5da-4fd5-b193-432aa9cc5af1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.372246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:18.372754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd668b04-a5da-4fd5-b193-432aa9cc5af1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:15:18.373073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd668b04-a5da-4fd5-b193-432aa9cc5af1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.374847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd668b04-a5da-4fd5-b193-432aa9cc5af1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.380110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74bc6b-6a63-4f77-a416-ea1a64ce90ae tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.386130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f484726-8fec-4de6-bf0b-bf40b9b4f183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.386838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f484726-8fec-4de6-bf0b-bf40b9b4f183 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.387198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f484726-8fec-4de6-bf0b-bf40b9b4f183 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.387515 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3566/7187] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:18.387648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f484726-8fec-4de6-bf0b-bf40b9b4f183 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.388335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f484726-8fec-4de6-bf0b-bf40b9b4f183 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.390267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f484726-8fec-4de6-bf0b-bf40b9b4f183 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.391548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd668b04-a5da-4fd5-b193-432aa9cc5af1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.398963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:18.403614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd668b04-a5da-4fd5-b193-432aa9cc5af1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.409987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef2941-9e1d-4527-973a-81c6e3eb4504 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.410721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef2941-9e1d-4527-973a-81c6e3eb4504 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.411053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef2941-9e1d-4527-973a-81c6e3eb4504 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.411904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef2941-9e1d-4527-973a-81c6e3eb4504 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation_use_group_config/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.412576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef2941-9e1d-4527-973a-81c6e3eb4504 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation_use_group_config)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.414348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef2941-9e1d-4527-973a-81c6e3eb4504 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.414877 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3622/7188] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:18 2022] POST /identity/v3/users => generated 290 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:18.418758 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3567/7189] 10.222.0.17 () {58 vars in 1308 bytes} [Sat Oct 1 21:15:18 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation_use_group_config/default => generated 63 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:18.418758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef2941-9e1d-4527-973a-81c6e3eb4504 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.418976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f484726-8fec-4de6-bf0b-bf40b9b4f183 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.429708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f484726-8fec-4de6-bf0b-bf40b9b4f183 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.429876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef2941-9e1d-4527-973a-81c6e3eb4504 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.434382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359d8e3-ec18-428c-ab87-bf5d18f4ea57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.434571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e7182-ab69-42ad-9ab8-c9e14b7d45c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.435014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359d8e3-ec18-428c-ab87-bf5d18f4ea57 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.435161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e7182-ab69-42ad-9ab8-c9e14b7d45c4 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.435294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359d8e3-ec18-428c-ab87-bf5d18f4ea57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.435447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e7182-ab69-42ad-9ab8-c9e14b7d45c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.435552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359d8e3-ec18-428c-ab87-bf5d18f4ea57 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.435726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e7182-ab69-42ad-9ab8-c9e14b7d45c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.436172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0359d8e3-ec18-428c-ab87-bf5d18f4ea57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.436370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e7182-ab69-42ad-9ab8-c9e14b7d45c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.437984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e7182-ab69-42ad-9ab8-c9e14b7d45c4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.438100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0359d8e3-ec18-428c-ab87-bf5d18f4ea57 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.451654 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3623/7190] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:15:18.451654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e7182-ab69-42ad-9ab8-c9e14b7d45c4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.461461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e7182-ab69-42ad-9ab8-c9e14b7d45c4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.466699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60e3df-dce2-45c7-a1a9-148966f78a8d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.467340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60e3df-dce2-45c7-a1a9-148966f78a8d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.467629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60e3df-dce2-45c7-a1a9-148966f78a8d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.467909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60e3df-dce2-45c7-a1a9-148966f78a8d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_additional_attribute_mapping/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.468500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60e3df-dce2-45c7-a1a9-148966f78a8d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_additional_attribute_mapping)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.472923 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3568/7191] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:18 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:18.473058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60e3df-dce2-45c7-a1a9-148966f78a8d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.473139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359d8e3-ec18-428c-ab87-bf5d18f4ea57 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.473212 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3624/7192] 10.222.0.17 () {58 vars in 1290 bytes} [Sat Oct 1 21:15:18 2022] GET /identity/v3/domains/config/ldap/user_additional_attribute_mapping/default => generated 54 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:18.473757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60e3df-dce2-45c7-a1a9-148966f78a8d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.484739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60e3df-dce2-45c7-a1a9-148966f78a8d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.536238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359d8e3-ec18-428c-ab87-bf5d18f4ea57 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.541145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6dc52c-f490-4c4a-806f-bbcb5e699669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.541416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fc409-4afb-47cc-b57a-3e5623bd2400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.541753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dc52c-f490-4c4a-806f-bbcb5e699669 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.542081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fc409-4afb-47cc-b57a-3e5623bd2400 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.542192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dc52c-f490-4c4a-806f-bbcb5e699669 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.542279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fc409-4afb-47cc-b57a-3e5623bd2400 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.542362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dc52c-f490-4c4a-806f-bbcb5e699669 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.542695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fc409-4afb-47cc-b57a-3e5623bd2400 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.542937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6dc52c-f490-4c4a-806f-bbcb5e699669 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.543338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fc409-4afb-47cc-b57a-3e5623bd2400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.544832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6dc52c-f490-4c4a-806f-bbcb5e699669 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.544959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fc409-4afb-47cc-b57a-3e5623bd2400 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.555701 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3569/7193] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:18.556715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fc409-4afb-47cc-b57a-3e5623bd2400 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.568032 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3625/7194] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:18 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:18.569074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6dc52c-f490-4c4a-806f-bbcb5e699669 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.578838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6dc52c-f490-4c4a-806f-bbcb5e699669 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.584024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252e3a7-df6c-499b-b325-970ab062ad62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.584656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252e3a7-df6c-499b-b325-970ab062ad62 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.584955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252e3a7-df6c-499b-b325-970ab062ad62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.585215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252e3a7-df6c-499b-b325-970ab062ad62 admin admin] PATH_INFO: `/v3/projects/2fa455091de94f1abda18743c51fb284/users/9476b7020b2c40e9abb14d11b7481f77/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.600462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252e3a7-df6c-499b-b325-970ab062ad62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fa455091de94f1abda18743c51fb284, user_id=9476b7020b2c40e9abb14d11b7481f77, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.603126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252e3a7-df6c-499b-b325-970ab062ad62 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.616337 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3626/7195] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:18 2022] PUT /identity/v3/projects/2fa455091de94f1abda18743c51fb284/users/9476b7020b2c40e9abb14d11b7481f77/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:18.617207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252e3a7-df6c-499b-b325-970ab062ad62 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.623738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fc409-4afb-47cc-b57a-3e5623bd2400 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.680715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf2760-2927-4711-94ac-20109c84b4e7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.681366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf2760-2927-4711-94ac-20109c84b4e7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.681648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf2760-2927-4711-94ac-20109c84b4e7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.681925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf2760-2927-4711-94ac-20109c84b4e7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_tree_dn/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.682538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf2760-2927-4711-94ac-20109c84b4e7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_tree_dn)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.684181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf2760-2927-4711-94ac-20109c84b4e7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.685466 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3570/7196] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:15:18 2022] GET /identity/v3/domains/config/ldap/group_tree_dn/default => generated 36 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:18.686115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252e3a7-df6c-499b-b325-970ab062ad62 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.686419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34508c5a-f4a8-4615-b3ff-2129a60359ad None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.686731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34508c5a-f4a8-4615-b3ff-2129a60359ad None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.686988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34508c5a-f4a8-4615-b3ff-2129a60359ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.692604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c884e10-c5e7-4d87-b5cb-6ed4659f9a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.693296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c884e10-c5e7-4d87-b5cb-6ed4659f9a60 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.693578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c884e10-c5e7-4d87-b5cb-6ed4659f9a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.693851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c884e10-c5e7-4d87-b5cb-6ed4659f9a60 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.694527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c884e10-c5e7-4d87-b5cb-6ed4659f9a60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.696136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c884e10-c5e7-4d87-b5cb-6ed4659f9a60 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.705840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-34508c5a-f4a8-4615-b3ff-2129a60359ad None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:18.709716 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3627/7197] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:18.709716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c884e10-c5e7-4d87-b5cb-6ed4659f9a60 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.709928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-34508c5a-f4a8-4615-b3ff-2129a60359ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:18.715401 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3571/7198] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:18.716071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34508c5a-f4a8-4615-b3ff-2129a60359ad None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.717866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c884e10-c5e7-4d87-b5cb-6ed4659f9a60 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.725965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34508c5a-f4a8-4615-b3ff-2129a60359ad None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.728342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d582f8d-8f40-4343-8754-c5cb248dd2dc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.728993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d582f8d-8f40-4343-8754-c5cb248dd2dc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.729274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d582f8d-8f40-4343-8754-c5cb248dd2dc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.729548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d582f8d-8f40-4343-8754-c5cb248dd2dc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_filter/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.730133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d582f8d-8f40-4343-8754-c5cb248dd2dc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_filter)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.730659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2dbf34-f981-4da7-981c-8735998d7eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.731247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dbf34-f981-4da7-981c-8735998d7eae admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.731527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dbf34-f981-4da7-981c-8735998d7eae admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.731804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dbf34-f981-4da7-981c-8735998d7eae admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.731898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d582f8d-8f40-4343-8754-c5cb248dd2dc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.732436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2dbf34-f981-4da7-981c-8735998d7eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.733070 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3628/7199] 10.222.0.17 () {58 vars in 1227 bytes} [Sat Oct 1 21:15:18 2022] GET /identity/v3/domains/config/ldap/group_filter/default => generated 35 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:18.733888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d582f8d-8f40-4343-8754-c5cb248dd2dc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.734000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2dbf34-f981-4da7-981c-8735998d7eae admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.735549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5d582f8d-8f40-4343-8754-c5cb248dd2dc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:18.744800 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3572/7200] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:18.745447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2dbf34-f981-4da7-981c-8735998d7eae admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.756337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2dbf34-f981-4da7-981c-8735998d7eae admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.761107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6735b9-1fc5-4b29-9799-bee7e5a5ef53 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.761726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6735b9-1fc5-4b29-9799-bee7e5a5ef53 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.762007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6735b9-1fc5-4b29-9799-bee7e5a5ef53 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.762616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6735b9-1fc5-4b29-9799-bee7e5a5ef53 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_objectclass/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.762892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6735b9-1fc5-4b29-9799-bee7e5a5ef53 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_objectclass)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.764456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6735b9-1fc5-4b29-9799-bee7e5a5ef53 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.765490 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3573/7201] 10.222.0.17 () {58 vars in 1242 bytes} [Sat Oct 1 21:15:18 2022] GET /identity/v3/domains/config/ldap/group_objectclass/default => generated 50 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:18.766201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6735b9-1fc5-4b29-9799-bee7e5a5ef53 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.767121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:18.771522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:18.777566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6735b9-1fc5-4b29-9799-bee7e5a5ef53 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.783444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d4e9e-23ac-402e-9595-15fbb72d14c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.784079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d4e9e-23ac-402e-9595-15fbb72d14c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.784356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d4e9e-23ac-402e-9595-15fbb72d14c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.784627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d4e9e-23ac-402e-9595-15fbb72d14c9 admin admin] PATH_INFO: `/v3/projects/5bad539f140a49398ccd85fa28753aab/users/71c5be0b64074fa58b525ce69ee0ad1d/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.798415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d582f8d-8f40-4343-8754-c5cb248dd2dc tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.799768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d4e9e-23ac-402e-9595-15fbb72d14c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bad539f140a49398ccd85fa28753aab, user_id=71c5be0b64074fa58b525ce69ee0ad1d, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.802821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cc8c2-9e0f-48df-a3dc-cf3b00fcdccb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.803281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d4e9e-23ac-402e-9595-15fbb72d14c9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.803423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cc8c2-9e0f-48df-a3dc-cf3b00fcdccb service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.803703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cc8c2-9e0f-48df-a3dc-cf3b00fcdccb service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.803999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cc8c2-9e0f-48df-a3dc-cf3b00fcdccb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.813737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05cc8c2-9e0f-48df-a3dc-cf3b00fcdccb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.815209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05cc8c2-9e0f-48df-a3dc-cf3b00fcdccb service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.816450 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3574/7202] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:18 2022] PUT /identity/v3/projects/5bad539f140a49398ccd85fa28753aab/users/71c5be0b64074fa58b525ce69ee0ad1d/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:18.817200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d4e9e-23ac-402e-9595-15fbb72d14c9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.825792 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3629/7203] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:15:18.827051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecce0f-fa22-4e67-8de8-43cd81225b4c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.827340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecce0f-fa22-4e67-8de8-43cd81225b4c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.827618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecce0f-fa22-4e67-8de8-43cd81225b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.862007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-61ecce0f-fa22-4e67-8de8-43cd81225b4c None None] MFA Rules not processed for user `71c5be0b64074fa58b525ce69ee0ad1d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:18.862952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-61ecce0f-fa22-4e67-8de8-43cd81225b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:18.896767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d4e9e-23ac-402e-9595-15fbb72d14c9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.904990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30759099-f8d5-42a2-b0a9-2d9c298c9d14 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.905454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30759099-f8d5-42a2-b0a9-2d9c298c9d14 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.905779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30759099-f8d5-42a2-b0a9-2d9c298c9d14 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.906882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30759099-f8d5-42a2-b0a9-2d9c298c9d14 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_id_attribute/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.907714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30759099-f8d5-42a2-b0a9-2d9c298c9d14 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_id_attribute)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.911179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30759099-f8d5-42a2-b0a9-2d9c298c9d14 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.913663 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3575/7204] 10.222.0.17 () {58 vars in 1245 bytes} [Sat Oct 1 21:15:18 2022] GET /identity/v3/domains/config/ldap/group_id_attribute/default => generated 41 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:18.917079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30759099-f8d5-42a2-b0a9-2d9c298c9d14 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.931635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30759099-f8d5-42a2-b0a9-2d9c298c9d14 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.940514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2c747-b563-49d2-9882-bca6c72e1ed4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.942058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2c747-b563-49d2-9882-bca6c72e1ed4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.942343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2c747-b563-49d2-9882-bca6c72e1ed4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.942696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2c747-b563-49d2-9882-bca6c72e1ed4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_name_attribute/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.943979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d2c747-b563-49d2-9882-bca6c72e1ed4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_name_attribute)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.944226 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3630/7205] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:18.945801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d2c747-b563-49d2-9882-bca6c72e1ed4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.947243 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3576/7206] 10.222.0.17 () {58 vars in 1251 bytes} [Sat Oct 1 21:15:18 2022] GET /identity/v3/domains/config/ldap/group_name_attribute/default => generated 43 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 01 21:15:18.952255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ecce0f-fa22-4e67-8de8-43cd81225b4c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.957810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2c747-b563-49d2-9882-bca6c72e1ed4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:18.961230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-60d2c747-b563-49d2-9882-bca6c72e1ed4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:18.977564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ecce0f-fa22-4e67-8de8-43cd81225b4c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:18.983762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2794c-3aa8-44b1-8b9f-c9cdc42842f2 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:18.985000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2794c-3aa8-44b1-8b9f-c9cdc42842f2 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:18.985367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2794c-3aa8-44b1-8b9f-c9cdc42842f2 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:18.985710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2794c-3aa8-44b1-8b9f-c9cdc42842f2 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_members_are_ids/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:18.986355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2794c-3aa8-44b1-8b9f-c9cdc42842f2 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_members_are_ids)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:18.988309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2794c-3aa8-44b1-8b9f-c9cdc42842f2 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:18.990380 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3631/7207] 10.222.0.17 () {58 vars in 1254 bytes} [Sat Oct 1 21:15:18 2022] GET /identity/v3/domains/config/ldap/group_members_are_ids/default => generated 45 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:18.995291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2794c-3aa8-44b1-8b9f-c9cdc42842f2 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.006743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2794c-3aa8-44b1-8b9f-c9cdc42842f2 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.012531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9323e-4b6a-49b0-974b-2829fe4ddbe4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.013122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9323e-4b6a-49b0-974b-2829fe4ddbe4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.013473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9323e-4b6a-49b0-974b-2829fe4ddbe4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.013822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9323e-4b6a-49b0-974b-2829fe4ddbe4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_member_attribute/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.014908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9323e-4b6a-49b0-974b-2829fe4ddbe4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_member_attribute)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:19.016941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9323e-4b6a-49b0-974b-2829fe4ddbe4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:19.018188 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3632/7208] 10.222.0.17 () {58 vars in 1257 bytes} [Sat Oct 1 21:15:18 2022] GET /identity/v3/domains/config/ldap/group_member_attribute/default => generated 49 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:19.022446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9323e-4b6a-49b0-974b-2829fe4ddbe4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.037404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9323e-4b6a-49b0-974b-2829fe4ddbe4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.045191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0249f9-1ce5-47e6-b16c-07a184159d33 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.045814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0249f9-1ce5-47e6-b16c-07a184159d33 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.046873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0249f9-1ce5-47e6-b16c-07a184159d33 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.046873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0249f9-1ce5-47e6-b16c-07a184159d33 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_desc_attribute/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.047045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2c747-b563-49d2-9882-bca6c72e1ed4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.047475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0249f9-1ce5-47e6-b16c-07a184159d33 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_desc_attribute)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:19.052676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826086a-b177-4d09-9d62-90c173b5f77d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.052880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0249f9-1ce5-47e6-b16c-07a184159d33 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:19.053497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826086a-b177-4d09-9d62-90c173b5f77d service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.053782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826086a-b177-4d09-9d62-90c173b5f77d service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.054055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826086a-b177-4d09-9d62-90c173b5f77d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.058357 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3633/7209] 10.222.0.17 () {58 vars in 1251 bytes} [Sat Oct 1 21:15:19 2022] GET /identity/v3/domains/config/ldap/group_desc_attribute/default => generated 52 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:19.059329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0249f9-1ce5-47e6-b16c-07a184159d33 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.064965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826086a-b177-4d09-9d62-90c173b5f77d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:19.066484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826086a-b177-4d09-9d62-90c173b5f77d service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:19.070837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0249f9-1ce5-47e6-b16c-07a184159d33 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.075704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240387a4-4064-4f9a-a2fa-4fb5dadb1210 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.076495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240387a4-4064-4f9a-a2fa-4fb5dadb1210 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.076794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240387a4-4064-4f9a-a2fa-4fb5dadb1210 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.077112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240387a4-4064-4f9a-a2fa-4fb5dadb1210 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_attribute_ignore/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.077709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240387a4-4064-4f9a-a2fa-4fb5dadb1210 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_attribute_ignore)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:19.079378 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240387a4-4064-4f9a-a2fa-4fb5dadb1210 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:19.080731 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3634/7210] 10.222.0.17 () {58 vars in 1257 bytes} [Sat Oct 1 21:15:19 2022] GET /identity/v3/domains/config/ldap/group_attribute_ignore/default => generated 43 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:19.084978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240387a4-4064-4f9a-a2fa-4fb5dadb1210 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.094586 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3577/7211] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:15:19.095660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240387a4-4064-4f9a-a2fa-4fb5dadb1210 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.100852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc8bcbc-1b0d-4906-823f-6da914312ff7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.101508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8bcbc-1b0d-4906-823f-6da914312ff7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.101795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8bcbc-1b0d-4906-823f-6da914312ff7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.102077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8bcbc-1b0d-4906-823f-6da914312ff7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_additional_attribute_mapping/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.102715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc8bcbc-1b0d-4906-823f-6da914312ff7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_additional_attribute_mapping)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:19.104313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc8bcbc-1b0d-4906-823f-6da914312ff7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:19.106739 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3635/7212] 10.222.0.17 () {58 vars in 1293 bytes} [Sat Oct 1 21:15:19 2022] GET /identity/v3/domains/config/ldap/group_additional_attribute_mapping/default => generated 55 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 01 21:15:19.110672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826086a-b177-4d09-9d62-90c173b5f77d service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.122880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826086a-b177-4d09-9d62-90c173b5f77d service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.129034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04975021-4919-41a2-9612-487d1e7e400a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.130034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04975021-4919-41a2-9612-487d1e7e400a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.130714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04975021-4919-41a2-9612-487d1e7e400a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.131121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04975021-4919-41a2-9612-487d1e7e400a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_cacertfile/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.131767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04975021-4919-41a2-9612-487d1e7e400a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_cacertfile)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:19.133994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04975021-4919-41a2-9612-487d1e7e400a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:19.135372 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3578/7213] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:19 2022] GET /identity/v3/domains/config/ldap/tls_cacertfile/default => generated 37 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 01 21:15:19.139975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc8bcbc-1b0d-4906-823f-6da914312ff7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.155891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc8bcbc-1b0d-4906-823f-6da914312ff7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.163291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e866e-539c-4fdc-9565-14cce4c19cbd tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.163919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e866e-539c-4fdc-9565-14cce4c19cbd tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.164209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e866e-539c-4fdc-9565-14cce4c19cbd tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.164490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e866e-539c-4fdc-9565-14cce4c19cbd tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_cacertdir/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.165083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e866e-539c-4fdc-9565-14cce4c19cbd tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_cacertdir)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:19.166879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e866e-539c-4fdc-9565-14cce4c19cbd tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:19.175389 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3636/7214] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:15:19 2022] GET /identity/v3/domains/config/ldap/tls_cacertdir/default => generated 36 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:19.177121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04975021-4919-41a2-9612-487d1e7e400a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.190389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04975021-4919-41a2-9612-487d1e7e400a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.196827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2fdf8-c166-4a01-840a-14c0c30e118b tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.197925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2fdf8-c166-4a01-840a-14c0c30e118b tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.198455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2fdf8-c166-4a01-840a-14c0c30e118b tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.199007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2fdf8-c166-4a01-840a-14c0c30e118b tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_tls/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.199809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec2fdf8-c166-4a01-840a-14c0c30e118b tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_tls)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:19.202550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec2fdf8-c166-4a01-840a-14c0c30e118b tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:19.204151 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3579/7215] 10.222.0.17 () {58 vars in 1212 bytes} [Sat Oct 1 21:15:19 2022] GET /identity/v3/domains/config/ldap/use_tls/default => generated 31 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:19.208170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e866e-539c-4fdc-9565-14cce4c19cbd tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.231710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e866e-539c-4fdc-9565-14cce4c19cbd tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.246453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d798b8-b086-48ee-b2cb-7ba8a74195b0 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.247278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d798b8-b086-48ee-b2cb-7ba8a74195b0 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.247583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d798b8-b086-48ee-b2cb-7ba8a74195b0 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.247962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d798b8-b086-48ee-b2cb-7ba8a74195b0 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_req_cert/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.248708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d798b8-b086-48ee-b2cb-7ba8a74195b0 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_req_cert)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:19.250580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d798b8-b086-48ee-b2cb-7ba8a74195b0 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:19.254891 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3637/7216] 10.222.0.17 () {58 vars in 1227 bytes} [Sat Oct 1 21:15:19 2022] GET /identity/v3/domains/config/ldap/tls_req_cert/default => generated 39 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:19.257813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2fdf8-c166-4a01-840a-14c0c30e118b tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.276876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2fdf8-c166-4a01-840a-14c0c30e118b tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.285808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff313e8-f5ac-44fa-bf3b-750ead5dc8d6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.287016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff313e8-f5ac-44fa-bf3b-750ead5dc8d6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.287159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff313e8-f5ac-44fa-bf3b-750ead5dc8d6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.287511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff313e8-f5ac-44fa-bf3b-750ead5dc8d6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_pool/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.288188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff313e8-f5ac-44fa-bf3b-750ead5dc8d6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_pool)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:19.289884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff313e8-f5ac-44fa-bf3b-750ead5dc8d6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:19.291325 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3580/7217] 10.222.0.17 () {58 vars in 1215 bytes} [Sat Oct 1 21:15:19 2022] GET /identity/v3/domains/config/ldap/use_pool/default => generated 31 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:19.295567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d798b8-b086-48ee-b2cb-7ba8a74195b0 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.306924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d798b8-b086-48ee-b2cb-7ba8a74195b0 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.313306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02634eb4-fba8-48a8-92ea-ace570586c1a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.313306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02634eb4-fba8-48a8-92ea-ace570586c1a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.313306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02634eb4-fba8-48a8-92ea-ace570586c1a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.313530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02634eb4-fba8-48a8-92ea-ace570586c1a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_size/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.314062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02634eb4-fba8-48a8-92ea-ace570586c1a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_size)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:19.315755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02634eb4-fba8-48a8-92ea-ace570586c1a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:19.318234 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3638/7218] 10.222.0.17 () {58 vars in 1218 bytes} [Sat Oct 1 21:15:19 2022] GET /identity/v3/domains/config/ldap/pool_size/default => generated 30 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 01 21:15:19.322691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff313e8-f5ac-44fa-bf3b-750ead5dc8d6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.334807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff313e8-f5ac-44fa-bf3b-750ead5dc8d6 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.340314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b6aee-847e-4f57-8ee5-dc6afb5969ef tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.341443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b6aee-847e-4f57-8ee5-dc6afb5969ef tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.341766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b6aee-847e-4f57-8ee5-dc6afb5969ef tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.342247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b6aee-847e-4f57-8ee5-dc6afb5969ef tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_retry_max/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.343134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b6aee-847e-4f57-8ee5-dc6afb5969ef tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_retry_max)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:19.345408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b6aee-847e-4f57-8ee5-dc6afb5969ef tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:19.349338 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3581/7219] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:19 2022] GET /identity/v3/domains/config/ldap/pool_retry_max/default => generated 34 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:19.350795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02634eb4-fba8-48a8-92ea-ace570586c1a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.366962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02634eb4-fba8-48a8-92ea-ace570586c1a tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.374573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8654f-d592-4da9-9a1c-a62dc02ea678 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.375302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8654f-d592-4da9-9a1c-a62dc02ea678 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.375830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8654f-d592-4da9-9a1c-a62dc02ea678 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.376193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8654f-d592-4da9-9a1c-a62dc02ea678 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_retry_delay/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.376946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c8654f-d592-4da9-9a1c-a62dc02ea678 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_retry_delay)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:19.378772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c8654f-d592-4da9-9a1c-a62dc02ea678 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:19.381273 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3639/7220] 10.222.0.17 () {58 vars in 1239 bytes} [Sat Oct 1 21:15:19 2022] GET /identity/v3/domains/config/ldap/pool_retry_delay/default => generated 38 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:19.384760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b6aee-847e-4f57-8ee5-dc6afb5969ef tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.395638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b6aee-847e-4f57-8ee5-dc6afb5969ef tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.401381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4acb9b-9fe8-4d91-b427-02a4c8641d6d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.402038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4acb9b-9fe8-4d91-b427-02a4c8641d6d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.402320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4acb9b-9fe8-4d91-b427-02a4c8641d6d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.402628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4acb9b-9fe8-4d91-b427-02a4c8641d6d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_connection_timeout/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.403217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4acb9b-9fe8-4d91-b427-02a4c8641d6d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_connection_timeout)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:19.404992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4acb9b-9fe8-4d91-b427-02a4c8641d6d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:19.410427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8654f-d592-4da9-9a1c-a62dc02ea678 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.416249 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3582/7221] 10.222.0.17 () {58 vars in 1260 bytes} [Sat Oct 1 21:15:19 2022] GET /identity/v3/domains/config/ldap/pool_connection_timeout/default => generated 44 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:19.423473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8654f-d592-4da9-9a1c-a62dc02ea678 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.431049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18da03-38e7-48b3-8faa-9d2ac86fbbea tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.431489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18da03-38e7-48b3-8faa-9d2ac86fbbea tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.431889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18da03-38e7-48b3-8faa-9d2ac86fbbea tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.432448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18da03-38e7-48b3-8faa-9d2ac86fbbea tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_connection_lifetime/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.433100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18da03-38e7-48b3-8faa-9d2ac86fbbea tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_connection_lifetime)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:19.435407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18da03-38e7-48b3-8faa-9d2ac86fbbea tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:19.437259 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3640/7222] 10.222.0.17 () {58 vars in 1263 bytes} [Sat Oct 1 21:15:19 2022] GET /identity/v3/domains/config/ldap/pool_connection_lifetime/default => generated 46 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 01 21:15:19.441287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4acb9b-9fe8-4d91-b427-02a4c8641d6d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.454263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4acb9b-9fe8-4d91-b427-02a4c8641d6d tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.460681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83605652-7a74-4074-acf1-7095a52f8131 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.462177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83605652-7a74-4074-acf1-7095a52f8131 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.462462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83605652-7a74-4074-acf1-7095a52f8131 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.462787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83605652-7a74-4074-acf1-7095a52f8131 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_auth_pool/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.463517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83605652-7a74-4074-acf1-7095a52f8131 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_auth_pool)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:19.465156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83605652-7a74-4074-acf1-7095a52f8131 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:19.470199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18da03-38e7-48b3-8faa-9d2ac86fbbea tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.470785 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3583/7223] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:15:19 2022] GET /identity/v3/domains/config/ldap/use_auth_pool/default => generated 36 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:19.482591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18da03-38e7-48b3-8faa-9d2ac86fbbea tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.488329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de7fa3-e945-45c4-b26c-723693469b40 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.488918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de7fa3-e945-45c4-b26c-723693469b40 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.489207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de7fa3-e945-45c4-b26c-723693469b40 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.490589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de7fa3-e945-45c4-b26c-723693469b40 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/auth_pool_size/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.491216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de7fa3-e945-45c4-b26c-723693469b40 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=auth_pool_size)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:19.492874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de7fa3-e945-45c4-b26c-723693469b40 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:19.497686 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3641/7224] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:19 2022] GET /identity/v3/domains/config/ldap/auth_pool_size/default => generated 36 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Oct 01 21:15:19.498399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83605652-7a74-4074-acf1-7095a52f8131 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.519655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83605652-7a74-4074-acf1-7095a52f8131 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.537821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa513a18-4673-45aa-8069-c81516919d42 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.538576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa513a18-4673-45aa-8069-c81516919d42 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.538957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa513a18-4673-45aa-8069-c81516919d42 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.540488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa513a18-4673-45aa-8069-c81516919d42 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/config/ldap/auth_pool_connection_lifetime/default` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.541248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa513a18-4673-45aa-8069-c81516919d42 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=auth_pool_connection_lifetime)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:19.545417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa513a18-4673-45aa-8069-c81516919d42 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:19.551289 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3584/7225] 10.222.0.17 () {58 vars in 1278 bytes} [Sat Oct 1 21:15:19 2022] GET /identity/v3/domains/config/ldap/auth_pool_connection_lifetime/default => generated 50 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Oct 01 21:15:19.554449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de7fa3-e945-45c4-b26c-723693469b40 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.568320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de7fa3-e945-45c4-b26c-723693469b40 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.573143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d97ad6-1e79-414e-8bc0-6216752a26c7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.573790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d97ad6-1e79-414e-8bc0-6216752a26c7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.574070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d97ad6-1e79-414e-8bc0-6216752a26c7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.574344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d97ad6-1e79-414e-8bc0-6216752a26c7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/beb063b82edc46adac6f7350b2c1d79c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.584585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d97ad6-1e79-414e-8bc0-6216752a26c7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 21:15:19.590397 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-99d97ad6-1e79-414e-8bc0-6216752a26c7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Could not find domain: beb063b82edc46adac6f7350b2c1d79c.: keystone.exception.DomainNotFound: Could not find domain: beb063b82edc46adac6f7350b2c1d79c. Oct 01 21:15:19.590397 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:19.590397 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 01 21:15:19.590397 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 01 21:15:19.590397 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 21:15:19.590397 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 21:15:19.590397 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 21:15:19.590397 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 21:15:19.590397 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: beb063b82edc46adac6f7350b2c1d79c. Oct 01 21:15:19.590397 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:15:19.590397 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:15:19.590397 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:15:19.590397 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:19.590397 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:15:19.590397 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:15:19.591651 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:15:19.591651 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:15:19.591651 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:15:19.591651 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:15:19.591651 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:15:19.591651 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:15:19.591651 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:15:19.591651 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:15:19.591651 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 01 21:15:19.591651 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 01 21:15:19.591651 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 21:15:19.591651 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 21:15:19.591651 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 21:15:19.591651 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 21:15:19.591651 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:15:19.591651 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:15:19.592650 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:15:19.592650 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:15:19.592650 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:15:19.592650 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:15:19.592650 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:15:19.592650 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 21:15:19.592650 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:15:19.592650 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:15:19.592650 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:15:19.592650 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:15:19.592650 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:15:19.592650 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:15:19.592650 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:15:19.592650 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:15:19.592650 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 01 21:15:19.592650 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 21:15:19.592650 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: beb063b82edc46adac6f7350b2c1d79c. Oct 01 21:15:19.593715 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:15:19.593715 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3642/7226] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:15:19 2022] PATCH /identity/v3/domains/beb063b82edc46adac6f7350b2c1d79c => generated 112 bytes in 38 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:15:19.596048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa513a18-4673-45aa-8069-c81516919d42 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.607201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa513a18-4673-45aa-8069-c81516919d42 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.612579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76023757-877d-4d51-a7c1-a089a062afc4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.613269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76023757-877d-4d51-a7c1-a089a062afc4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.613488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76023757-877d-4d51-a7c1-a089a062afc4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.613759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76023757-877d-4d51-a7c1-a089a062afc4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/e24a461ad43444b4a6c8f689c06d983a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.622310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76023757-877d-4d51-a7c1-a089a062afc4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 21:15:19.628057 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-76023757-877d-4d51-a7c1-a089a062afc4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Could not find domain: e24a461ad43444b4a6c8f689c06d983a.: keystone.exception.DomainNotFound: Could not find domain: e24a461ad43444b4a6c8f689c06d983a. Oct 01 21:15:19.628057 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:19.628057 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 01 21:15:19.628057 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 01 21:15:19.628057 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 21:15:19.628057 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 21:15:19.628057 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 21:15:19.628057 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 21:15:19.628057 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: e24a461ad43444b4a6c8f689c06d983a. Oct 01 21:15:19.628057 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:15:19.628057 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:15:19.628057 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:15:19.628057 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:19.628057 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:15:19.628057 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:15:19.631011 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:15:19.631011 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:15:19.631011 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:15:19.631011 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:15:19.631011 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:15:19.631011 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:15:19.631011 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:15:19.631011 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:15:19.631011 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 01 21:15:19.631011 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 01 21:15:19.631011 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 21:15:19.631011 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 21:15:19.631011 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 21:15:19.631011 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 21:15:19.631011 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:15:19.631011 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:15:19.631879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:15:19.631879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:15:19.631879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:15:19.631879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:15:19.631879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:15:19.631879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 21:15:19.631879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:15:19.631879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:15:19.631879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:15:19.631879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:15:19.631879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:15:19.631879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:15:19.631879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:15:19.631879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:15:19.631879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 01 21:15:19.631879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 21:15:19.631879 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: e24a461ad43444b4a6c8f689c06d983a. Oct 01 21:15:19.632949 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:15:19.632949 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3585/7227] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:15:19 2022] PATCH /identity/v3/domains/e24a461ad43444b4a6c8f689c06d983a => generated 112 bytes in 34 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:15:19.634248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d97ad6-1e79-414e-8bc0-6216752a26c7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.646181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d97ad6-1e79-414e-8bc0-6216752a26c7 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.651607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369231f7-7e8c-42aa-98f2-7a5f20093304 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d2e5c113f44efea24d6925be461afc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6abf8585ff6f423ea79b064961d74d4e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.652361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369231f7-7e8c-42aa-98f2-7a5f20093304 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.652648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369231f7-7e8c-42aa-98f2-7a5f20093304 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.652915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369231f7-7e8c-42aa-98f2-7a5f20093304 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] PATH_INFO: `/v3/domains/f1c42bf0a35a4930b74abc7577572c7c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.661202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369231f7-7e8c-42aa-98f2-7a5f20093304 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 21:15:19.666863 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-369231f7-7e8c-42aa-98f2-7a5f20093304 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Could not find domain: f1c42bf0a35a4930b74abc7577572c7c.: keystone.exception.DomainNotFound: Could not find domain: f1c42bf0a35a4930b74abc7577572c7c. Oct 01 21:15:19.666863 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:19.666863 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 01 21:15:19.666863 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 01 21:15:19.666863 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 21:15:19.666863 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 21:15:19.666863 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 21:15:19.666863 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 21:15:19.666863 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: f1c42bf0a35a4930b74abc7577572c7c. Oct 01 21:15:19.666863 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:15:19.666863 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:15:19.666863 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:15:19.666863 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:19.666863 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:15:19.666863 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:15:19.671178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:15:19.671178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:15:19.671178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:15:19.671178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:15:19.671178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:15:19.671178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:15:19.671178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:15:19.671178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:15:19.671178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 01 21:15:19.671178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 01 21:15:19.671178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 21:15:19.671178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 21:15:19.671178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 21:15:19.671178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 21:15:19.671178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:15:19.671178 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:15:19.672055 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:15:19.672055 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:15:19.672055 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:15:19.672055 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:15:19.672055 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:15:19.672055 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 21:15:19.672055 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:15:19.672055 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:15:19.672055 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:15:19.672055 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:15:19.672055 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:15:19.672055 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:15:19.672055 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:15:19.672055 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:15:19.672055 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 01 21:15:19.672055 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 21:15:19.672055 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: f1c42bf0a35a4930b74abc7577572c7c. Oct 01 21:15:19.672953 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:15:19.672953 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3643/7228] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:15:19 2022] PATCH /identity/v3/domains/f1c42bf0a35a4930b74abc7577572c7c => generated 112 bytes in 34 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:15:19.673348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76023757-877d-4d51-a7c1-a089a062afc4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.676094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-76023757-877d-4d51-a7c1-a089a062afc4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:19.725683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:19.750668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76023757-877d-4d51-a7c1-a089a062afc4 tempest-DomainConfigurationTestJSON-424798474 tempest-DomainConfigurationTestJSON-424798474-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:19.756441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8bb4f-35d9-43ed-8dbd-f545d0554dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:19.757077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8bb4f-35d9-43ed-8dbd-f545d0554dbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.757356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8bb4f-35d9-43ed-8dbd-f545d0554dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.757627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8bb4f-35d9-43ed-8dbd-f545d0554dbb admin admin] PATH_INFO: `/v3/users/3f6aedb7db234cdf8e665581f76584f7` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.758811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e8bb4f-35d9-43ed-8dbd-f545d0554dbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f6aedb7db234cdf8e665581f76584f7)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:19.760735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e8bb4f-35d9-43ed-8dbd-f545d0554dbb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:19.811087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-09e8bb4f-35d9-43ed-8dbd-f545d0554dbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f6aedb7db234cdf8e665581f76584f7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:19.814670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-09e8bb4f-35d9-43ed-8dbd-f545d0554dbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f6aedb7db234cdf8e665581f76584f7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:19.815509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-09e8bb4f-35d9-43ed-8dbd-f545d0554dbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f6aedb7db234cdf8e665581f76584f7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:19.825452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-09e8bb4f-35d9-43ed-8dbd-f545d0554dbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f6aedb7db234cdf8e665581f76584f7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:19.840063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-09e8bb4f-35d9-43ed-8dbd-f545d0554dbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f6aedb7db234cdf8e665581f76584f7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:19.865069 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3586/7229] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:19 2022] DELETE /identity/v3/users/3f6aedb7db234cdf8e665581f76584f7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:19.866853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078e606-2ffe-4270-b75d-8c95c10c3f10 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:19.867637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078e606-2ffe-4270-b75d-8c95c10c3f10 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:19.869038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078e606-2ffe-4270-b75d-8c95c10c3f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:19.890907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1078e606-2ffe-4270-b75d-8c95c10c3f10 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:19.892228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1078e606-2ffe-4270-b75d-8c95c10c3f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:19.955962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:19.978019 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3644/7230] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:19.997357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8bb4f-35d9-43ed-8dbd-f545d0554dbb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:19.999202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-09e8bb4f-35d9-43ed-8dbd-f545d0554dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:20.082895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8bb4f-35d9-43ed-8dbd-f545d0554dbb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:20.090022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3d921-cfe1-4326-9d36-71714f30e08d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:20.090746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3d921-cfe1-4326-9d36-71714f30e08d service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:20.091037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3d921-cfe1-4326-9d36-71714f30e08d service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:20.091316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3d921-cfe1-4326-9d36-71714f30e08d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:20.105906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb3d921-cfe1-4326-9d36-71714f30e08d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:20.109493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb3d921-cfe1-4326-9d36-71714f30e08d service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:20.133002 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3587/7231] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:15:20.629077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078e606-2ffe-4270-b75d-8c95c10c3f10 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:20.633020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1078e606-2ffe-4270-b75d-8c95c10c3f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:20.669572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078e606-2ffe-4270-b75d-8c95c10c3f10 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:20.687164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6523c0-e32d-4890-8ad0-66e6f54f043c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:20.688877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6523c0-e32d-4890-8ad0-66e6f54f043c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:20.689169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6523c0-e32d-4890-8ad0-66e6f54f043c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:20.689495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6523c0-e32d-4890-8ad0-66e6f54f043c admin admin] PATH_INFO: `/v3/projects/476d81be34d84fd28bef57ae94442665` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:20.690698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6523c0-e32d-4890-8ad0-66e6f54f043c admin admin] RBAC: Authorizing `identity:delete_project(project_id=476d81be34d84fd28bef57ae94442665)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:20.693958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6523c0-e32d-4890-8ad0-66e6f54f043c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:20.760437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ab6523c0-e32d-4890-8ad0-66e6f54f043c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '476d81be34d84fd28bef57ae94442665', 'request_id': 'req-ab6523c0-e32d-4890-8ad0-66e6f54f043c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:20.775337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ab6523c0-e32d-4890-8ad0-66e6f54f043c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '476d81be34d84fd28bef57ae94442665', 'request_id': 'req-ab6523c0-e32d-4890-8ad0-66e6f54f043c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:20.807577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ab6523c0-e32d-4890-8ad0-66e6f54f043c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '476d81be34d84fd28bef57ae94442665', 'request_id': 'req-ab6523c0-e32d-4890-8ad0-66e6f54f043c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:20.825815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ab6523c0-e32d-4890-8ad0-66e6f54f043c admin admin] The token cache is being invalidate because project 476d81be34d84fd28bef57ae94442665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:20.826146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ab6523c0-e32d-4890-8ad0-66e6f54f043c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:20.827285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ab6523c0-e32d-4890-8ad0-66e6f54f043c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:20.834448 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3645/7232] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:20 2022] DELETE /identity/v3/projects/476d81be34d84fd28bef57ae94442665 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:20.835165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3d921-cfe1-4326-9d36-71714f30e08d service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:20.849853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3d921-cfe1-4326-9d36-71714f30e08d service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:20.965778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed8181-10fa-4708-94a4-680e5590aa64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:20.966520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed8181-10fa-4708-94a4-680e5590aa64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:20.967007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed8181-10fa-4708-94a4-680e5590aa64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:20.967404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed8181-10fa-4708-94a4-680e5590aa64 admin admin] PATH_INFO: `/v3/users/50d2e5c113f44efea24d6925be461afc` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:20.968706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed8181-10fa-4708-94a4-680e5590aa64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50d2e5c113f44efea24d6925be461afc)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:20.971095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed8181-10fa-4708-94a4-680e5590aa64 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:20.987304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:21.112071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e4ed8181-10fa-4708-94a4-680e5590aa64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50d2e5c113f44efea24d6925be461afc'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:21.113582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e4ed8181-10fa-4708-94a4-680e5590aa64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50d2e5c113f44efea24d6925be461afc'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:21.114939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e4ed8181-10fa-4708-94a4-680e5590aa64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50d2e5c113f44efea24d6925be461afc'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:21.127763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e4ed8181-10fa-4708-94a4-680e5590aa64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50d2e5c113f44efea24d6925be461afc'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:21.147501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e4ed8181-10fa-4708-94a4-680e5590aa64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50d2e5c113f44efea24d6925be461afc'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:21.172842 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3588/7233] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:20 2022] DELETE /identity/v3/users/50d2e5c113f44efea24d6925be461afc => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:21.876545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6523c0-e32d-4890-8ad0-66e6f54f043c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:21.878032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ab6523c0-e32d-4890-8ad0-66e6f54f043c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:21.905746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6523c0-e32d-4890-8ad0-66e6f54f043c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:22.120257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc7f9c-2d90-41c5-a331-bf7c54701e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:22.126342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc7f9c-2d90-41c5-a331-bf7c54701e9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:22.126966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc7f9c-2d90-41c5-a331-bf7c54701e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:22.127493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc7f9c-2d90-41c5-a331-bf7c54701e9e admin admin] PATH_INFO: `/v3/projects/6abf8585ff6f423ea79b064961d74d4e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:22.129553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc7f9c-2d90-41c5-a331-bf7c54701e9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6abf8585ff6f423ea79b064961d74d4e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:22.132453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc7f9c-2d90-41c5-a331-bf7c54701e9e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:22.205829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-41cc7f9c-2d90-41c5-a331-bf7c54701e9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6abf8585ff6f423ea79b064961d74d4e', 'request_id': 'req-41cc7f9c-2d90-41c5-a331-bf7c54701e9e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:22.227625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-41cc7f9c-2d90-41c5-a331-bf7c54701e9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6abf8585ff6f423ea79b064961d74d4e', 'request_id': 'req-41cc7f9c-2d90-41c5-a331-bf7c54701e9e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:22.246094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-41cc7f9c-2d90-41c5-a331-bf7c54701e9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6abf8585ff6f423ea79b064961d74d4e', 'request_id': 'req-41cc7f9c-2d90-41c5-a331-bf7c54701e9e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:22.275027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-41cc7f9c-2d90-41c5-a331-bf7c54701e9e admin admin] The token cache is being invalidate because project 6abf8585ff6f423ea79b064961d74d4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:22.275439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-41cc7f9c-2d90-41c5-a331-bf7c54701e9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:22.278000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-41cc7f9c-2d90-41c5-a331-bf7c54701e9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:22.285496 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3646/7234] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:21 2022] DELETE /identity/v3/projects/6abf8585ff6f423ea79b064961d74d4e => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:22.286704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21c8be-8542-49f6-92c3-a934204e03e5 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:22.287142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21c8be-8542-49f6-92c3-a934204e03e5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:22.287391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21c8be-8542-49f6-92c3-a934204e03e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:22.330725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1d21c8be-8542-49f6-92c3-a934204e03e5 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:22.332100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1d21c8be-8542-49f6-92c3-a934204e03e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:22.447683 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3589/7235] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:22.448540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ede7b-c18d-4a32-98a7-b82fc4894c57 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:22.448868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ede7b-c18d-4a32-98a7-b82fc4894c57 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:22.449141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ede7b-c18d-4a32-98a7-b82fc4894c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:22.473191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:22.491461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ca5ede7b-c18d-4a32-98a7-b82fc4894c57 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:22.494417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ca5ede7b-c18d-4a32-98a7-b82fc4894c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:22.522023 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3647/7236] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:22.524168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21c8be-8542-49f6-92c3-a934204e03e5 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:22.547266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21c8be-8542-49f6-92c3-a934204e03e5 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:22.555133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0121261-8dc2-4121-aac7-b5c3aac5dd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:22.555768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0121261-8dc2-4121-aac7-b5c3aac5dd95 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:22.556607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0121261-8dc2-4121-aac7-b5c3aac5dd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:22.557698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0121261-8dc2-4121-aac7-b5c3aac5dd95 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:22.558562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0121261-8dc2-4121-aac7-b5c3aac5dd95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:15:22.559773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0121261-8dc2-4121-aac7-b5c3aac5dd95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:22.561484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0121261-8dc2-4121-aac7-b5c3aac5dd95 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:22.582468 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3590/7237] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:22.587080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ede7b-c18d-4a32-98a7-b82fc4894c57 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:22.601792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ede7b-c18d-4a32-98a7-b82fc4894c57 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:22.610936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af984f7f-793b-4e2a-a64a-c12178674c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:22.611561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af984f7f-793b-4e2a-a64a-c12178674c59 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:22.611848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af984f7f-793b-4e2a-a64a-c12178674c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:22.612129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af984f7f-793b-4e2a-a64a-c12178674c59 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:22.612798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af984f7f-793b-4e2a-a64a-c12178674c59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:22.620204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af984f7f-793b-4e2a-a64a-c12178674c59 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:22.690332 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3648/7238] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:22 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:22.691745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0121261-8dc2-4121-aac7-b5c3aac5dd95 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:22.717428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0121261-8dc2-4121-aac7-b5c3aac5dd95 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:22.791738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164ce9c-f6ed-4e4a-9221-c9939970c465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:22.793438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164ce9c-f6ed-4e4a-9221-c9939970c465 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:22.793732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164ce9c-f6ed-4e4a-9221-c9939970c465 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:22.794326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164ce9c-f6ed-4e4a-9221-c9939970c465 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:22.795324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164ce9c-f6ed-4e4a-9221-c9939970c465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:22.798978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164ce9c-f6ed-4e4a-9221-c9939970c465 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:22.824611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:22.836301 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3591/7239] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:22 2022] POST /identity/v3/users => generated 285 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:22.839773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af984f7f-793b-4e2a-a64a-c12178674c59 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:22.853786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af984f7f-793b-4e2a-a64a-c12178674c59 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:22.862557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712b940-56dd-4ca4-8b39-b3c1edd51686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:22.863337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712b940-56dd-4ca4-8b39-b3c1edd51686 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:22.865283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712b940-56dd-4ca4-8b39-b3c1edd51686 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:22.865575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712b940-56dd-4ca4-8b39-b3c1edd51686 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:22.866847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712b940-56dd-4ca4-8b39-b3c1edd51686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:22.869917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712b940-56dd-4ca4-8b39-b3c1edd51686 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:22.895061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164ce9c-f6ed-4e4a-9221-c9939970c465 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:22.905499 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3649/7240] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:22.907195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164ce9c-f6ed-4e4a-9221-c9939970c465 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:22.912907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932cc6ff-f856-4e06-89a4-f62cd6b0b9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:22.913539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932cc6ff-f856-4e06-89a4-f62cd6b0b9ef admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:22.913820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932cc6ff-f856-4e06-89a4-f62cd6b0b9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:22.914251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932cc6ff-f856-4e06-89a4-f62cd6b0b9ef admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:22.915032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932cc6ff-f856-4e06-89a4-f62cd6b0b9ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:22.917334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932cc6ff-f856-4e06-89a4-f62cd6b0b9ef admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:22.954267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712b940-56dd-4ca4-8b39-b3c1edd51686 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:22.962873 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3592/7241] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:22.967902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712b940-56dd-4ca4-8b39-b3c1edd51686 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:22.992023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc890427-89de-477b-857f-e21bcc9ab9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:22.992800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc890427-89de-477b-857f-e21bcc9ab9dd admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:22.993156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc890427-89de-477b-857f-e21bcc9ab9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:22.993508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc890427-89de-477b-857f-e21bcc9ab9dd admin admin] PATH_INFO: `/v3/projects/b758708abef04a00a2922680d116198a/users/dcf9e137e3de486eba2cafcad5651464/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:23.021219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc890427-89de-477b-857f-e21bcc9ab9dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b758708abef04a00a2922680d116198a, user_id=dcf9e137e3de486eba2cafcad5651464, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:23.032113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc890427-89de-477b-857f-e21bcc9ab9dd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:23.073956 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3650/7242] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:22 2022] PUT /identity/v3/projects/b758708abef04a00a2922680d116198a/users/dcf9e137e3de486eba2cafcad5651464/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:23.085043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8d6c0-8bd7-4125-a49b-4a89586362ac None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:23.085340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8d6c0-8bd7-4125-a49b-4a89586362ac None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:23.085617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8d6c0-8bd7-4125-a49b-4a89586362ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:23.116385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-17b8d6c0-8bd7-4125-a49b-4a89586362ac None None] MFA Rules not processed for user `dcf9e137e3de486eba2cafcad5651464`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:23.117633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-17b8d6c0-8bd7-4125-a49b-4a89586362ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:23.214806 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3593/7243] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:23.219311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc890427-89de-477b-857f-e21bcc9ab9dd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:23.355149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc890427-89de-477b-857f-e21bcc9ab9dd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:23.361781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ade734-efb7-44c8-8ee5-a8ab17cec9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:23.362792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ade734-efb7-44c8-8ee5-a8ab17cec9a6 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:23.363180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ade734-efb7-44c8-8ee5-a8ab17cec9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:23.363645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ade734-efb7-44c8-8ee5-a8ab17cec9a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:23.365105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ade734-efb7-44c8-8ee5-a8ab17cec9a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:23.368112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ade734-efb7-44c8-8ee5-a8ab17cec9a6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:23.440505 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3651/7244] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:23 2022] POST /identity/v3/projects => generated 369 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:23.441646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8d6c0-8bd7-4125-a49b-4a89586362ac None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:23.459919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8d6c0-8bd7-4125-a49b-4a89586362ac None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:23.557737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84650353-b860-4347-85c6-6f08030677f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:23.558654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84650353-b860-4347-85c6-6f08030677f6 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:23.559123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84650353-b860-4347-85c6-6f08030677f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:23.559468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84650353-b860-4347-85c6-6f08030677f6 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:23.560289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84650353-b860-4347-85c6-6f08030677f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:23.562376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84650353-b860-4347-85c6-6f08030677f6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:23.605198 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3594/7245] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:23 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:23.606447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ade734-efb7-44c8-8ee5-a8ab17cec9a6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:23.618211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ade734-efb7-44c8-8ee5-a8ab17cec9a6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:23.623342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c900f-2188-4e8e-8656-d2bf61f1b010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:23.623970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c900f-2188-4e8e-8656-d2bf61f1b010 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:23.624253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c900f-2188-4e8e-8656-d2bf61f1b010 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:23.624521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c900f-2188-4e8e-8656-d2bf61f1b010 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:23.625170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c900f-2188-4e8e-8656-d2bf61f1b010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:23.626921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c900f-2188-4e8e-8656-d2bf61f1b010 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:23.644063 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3652/7246] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:15:23.644600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84650353-b860-4347-85c6-6f08030677f6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:23.659730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84650353-b860-4347-85c6-6f08030677f6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:23.666158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828de41f-6d24-4b8a-96be-543f45f6a1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:23.666690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828de41f-6d24-4b8a-96be-543f45f6a1bc admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:23.667329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828de41f-6d24-4b8a-96be-543f45f6a1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:23.667329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828de41f-6d24-4b8a-96be-543f45f6a1bc admin admin] PATH_INFO: `/v3/projects/3ab8b62281af4862a9cb86af23aa8371/users/1747c0e3418242238fdce39ab1d923fb/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:23.687601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828de41f-6d24-4b8a-96be-543f45f6a1bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ab8b62281af4862a9cb86af23aa8371, user_id=1747c0e3418242238fdce39ab1d923fb, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:23.691107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828de41f-6d24-4b8a-96be-543f45f6a1bc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:23.720452 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3595/7247] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:23 2022] PUT /identity/v3/projects/3ab8b62281af4862a9cb86af23aa8371/users/1747c0e3418242238fdce39ab1d923fb/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:23.726393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c900f-2188-4e8e-8656-d2bf61f1b010 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:23.742253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c900f-2188-4e8e-8656-d2bf61f1b010 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:23.846643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a097da2-1ff7-46e5-a322-ae30c4b336ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:23.847275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a097da2-1ff7-46e5-a322-ae30c4b336ec admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:23.849122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a097da2-1ff7-46e5-a322-ae30c4b336ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:23.849369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a097da2-1ff7-46e5-a322-ae30c4b336ec admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:23.850034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a097da2-1ff7-46e5-a322-ae30c4b336ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:23.851670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a097da2-1ff7-46e5-a322-ae30c4b336ec admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:23.880625 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3653/7248] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:23.883578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828de41f-6d24-4b8a-96be-543f45f6a1bc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:23.897194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828de41f-6d24-4b8a-96be-543f45f6a1bc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:23.902730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c652c-4e12-4066-8b38-a184b4ee4534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:23.903707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c652c-4e12-4066-8b38-a184b4ee4534 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:23.904130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c652c-4e12-4066-8b38-a184b4ee4534 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:23.904654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c652c-4e12-4066-8b38-a184b4ee4534 admin admin] PATH_INFO: `/v3/projects/3ab8b62281af4862a9cb86af23aa8371/users/1747c0e3418242238fdce39ab1d923fb/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:23.906958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c652c-4e12-4066-8b38-a184b4ee4534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ab8b62281af4862a9cb86af23aa8371, user_id=1747c0e3418242238fdce39ab1d923fb, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:23.910038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c652c-4e12-4066-8b38-a184b4ee4534 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:23.926608 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3596/7249] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:23 2022] PUT /identity/v3/projects/3ab8b62281af4862a9cb86af23aa8371/users/1747c0e3418242238fdce39ab1d923fb/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:23.941029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d522e6a-83a9-40cd-b8ad-68f135bc7a81 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:23.941518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d522e6a-83a9-40cd-b8ad-68f135bc7a81 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:23.942158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d522e6a-83a9-40cd-b8ad-68f135bc7a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:23.969868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:23.973227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0d522e6a-83a9-40cd-b8ad-68f135bc7a81 None None] MFA Rules not processed for user `1747c0e3418242238fdce39ab1d923fb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:23.974895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0d522e6a-83a9-40cd-b8ad-68f135bc7a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:24.105478 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3654/7250] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:24.111912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c652c-4e12-4066-8b38-a184b4ee4534 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:24.139654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c652c-4e12-4066-8b38-a184b4ee4534 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:24.152466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82c45d-c51a-4412-9e03-d93aaa4bbb2c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:24.154385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82c45d-c51a-4412-9e03-d93aaa4bbb2c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:24.155242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82c45d-c51a-4412-9e03-d93aaa4bbb2c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:24.156275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82c45d-c51a-4412-9e03-d93aaa4bbb2c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:24.157285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c82c45d-c51a-4412-9e03-d93aaa4bbb2c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:24.167855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c82c45d-c51a-4412-9e03-d93aaa4bbb2c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:24.231654 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3597/7251] 10.222.0.17 () {60 vars in 1152 bytes} [Sat Oct 1 21:15:24 2022] POST /identity/v3/projects => generated 328 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:24.236155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d522e6a-83a9-40cd-b8ad-68f135bc7a81 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:24.253512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d522e6a-83a9-40cd-b8ad-68f135bc7a81 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:24.334602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5ab9f-2f39-4761-b043-163dff88aa19 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:24.335472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5ab9f-2f39-4761-b043-163dff88aa19 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:24.337362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5ab9f-2f39-4761-b043-163dff88aa19 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:24.337942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5ab9f-2f39-4761-b043-163dff88aa19 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:24.338767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5ab9f-2f39-4761-b043-163dff88aa19 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:24.341076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5ab9f-2f39-4761-b043-163dff88aa19 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:24.372125 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3655/7252] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:24 2022] POST /identity/v3/users => generated 413 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:24.376958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82c45d-c51a-4412-9e03-d93aaa4bbb2c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:24.390355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82c45d-c51a-4412-9e03-d93aaa4bbb2c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:24.396174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7c00e-8412-42a4-937b-9d586cb0b6ae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:24.396789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7c00e-8412-42a4-937b-9d586cb0b6ae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:24.397071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7c00e-8412-42a4-937b-9d586cb0b6ae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:24.397485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7c00e-8412-42a4-937b-9d586cb0b6ae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:24.398651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e7c00e-8412-42a4-937b-9d586cb0b6ae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:24.414248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e7c00e-8412-42a4-937b-9d586cb0b6ae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:24.419188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:24.452550 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3598/7253] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:15:24 2022] POST /identity/v3/roles => generated 244 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:24.458744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5ab9f-2f39-4761-b043-163dff88aa19 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:24.474880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5ab9f-2f39-4761-b043-163dff88aa19 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:24.493871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0875b2a-d792-4f7c-82ac-c5de50d9028a tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:24.494979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0875b2a-d792-4f7c-82ac-c5de50d9028a tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:24.495528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0875b2a-d792-4f7c-82ac-c5de50d9028a tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:24.496008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0875b2a-d792-4f7c-82ac-c5de50d9028a tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:24.497522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0875b2a-d792-4f7c-82ac-c5de50d9028a tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:24.501764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0875b2a-d792-4f7c-82ac-c5de50d9028a tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:24.520539 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3656/7254] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:15:24 2022] POST /identity/v3/roles => generated 247 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:24.526528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7c00e-8412-42a4-937b-9d586cb0b6ae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:24.542205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7c00e-8412-42a4-937b-9d586cb0b6ae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:24.548358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f9b38-fd49-437f-9f2b-056b654aee0b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:24.549033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f9b38-fd49-437f-9f2b-056b654aee0b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:24.549316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f9b38-fd49-437f-9f2b-056b654aee0b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:24.549620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f9b38-fd49-437f-9f2b-056b654aee0b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects/81bc6d3965bd43ebb794e4ec08c3051c/users/046f89a576a74e099b22bfc04c1b8451/roles/54806e8b7475469c908b578a645d0120` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:24.595494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f9b38-fd49-437f-9f2b-056b654aee0b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_grant(project_id=81bc6d3965bd43ebb794e4ec08c3051c, user_id=046f89a576a74e099b22bfc04c1b8451, role_id=54806e8b7475469c908b578a645d0120)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:24.599589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f9b38-fd49-437f-9f2b-056b654aee0b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:24.615661 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3599/7255] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:24 2022] PUT /identity/v3/projects/81bc6d3965bd43ebb794e4ec08c3051c/users/046f89a576a74e099b22bfc04c1b8451/roles/54806e8b7475469c908b578a645d0120 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:24.620396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0875b2a-d792-4f7c-82ac-c5de50d9028a tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:24.637270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0875b2a-d792-4f7c-82ac-c5de50d9028a tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:24.785844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428ca94-6bf4-4b57-ba42-6a9f781d72d2 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:24.787358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428ca94-6bf4-4b57-ba42-6a9f781d72d2 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:24.787821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428ca94-6bf4-4b57-ba42-6a9f781d72d2 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:24.788278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428ca94-6bf4-4b57-ba42-6a9f781d72d2 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects/81bc6d3965bd43ebb794e4ec08c3051c/users/046f89a576a74e099b22bfc04c1b8451/roles/bf269a7c52874649ae3a56dc70fdb1f9` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:24.797051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428ca94-6bf4-4b57-ba42-6a9f781d72d2 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_grant(project_id=81bc6d3965bd43ebb794e4ec08c3051c, user_id=046f89a576a74e099b22bfc04c1b8451, role_id=bf269a7c52874649ae3a56dc70fdb1f9)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:24.800038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428ca94-6bf4-4b57-ba42-6a9f781d72d2 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:24.839790 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3657/7256] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:24 2022] PUT /identity/v3/projects/81bc6d3965bd43ebb794e4ec08c3051c/users/046f89a576a74e099b22bfc04c1b8451/roles/bf269a7c52874649ae3a56dc70fdb1f9 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:24.840926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f9b38-fd49-437f-9f2b-056b654aee0b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:24.871389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f9b38-fd49-437f-9f2b-056b654aee0b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:24.953896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0bf26-6122-4d44-a018-429be534a0d0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:24.955102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0bf26-6122-4d44-a018-429be534a0d0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:24.955686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0bf26-6122-4d44-a018-429be534a0d0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:24.956182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0bf26-6122-4d44-a018-429be534a0d0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:24.957351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d0bf26-6122-4d44-a018-429be534a0d0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:24.960078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d0bf26-6122-4d44-a018-429be534a0d0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:25.049828 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3600/7257] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:24 2022] GET /identity/v3/users => generated 7496 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:25.054705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9c1a5-2128-4ce3-adae-a7bd4414c3e1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:25.055268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9c1a5-2128-4ce3-adae-a7bd4414c3e1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:25.055549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9c1a5-2128-4ce3-adae-a7bd4414c3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:25.118574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:25.165770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-bff9c1a5-2128-4ce3-adae-a7bd4414c3e1 None None] MFA Rules not processed for user `046f89a576a74e099b22bfc04c1b8451`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:25.167166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bff9c1a5-2128-4ce3-adae-a7bd4414c3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:25.316665 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3658/7258] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:25 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:25.332184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f0939-46d2-42f1-b4c2-a14168153a48 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:25.332477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f0939-46d2-42f1-b4c2-a14168153a48 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:25.332907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f0939-46d2-42f1-b4c2-a14168153a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:25.337293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fa6df-b0ca-4273-be8d-79c004ef02af None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:25.337797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fa6df-b0ca-4273-be8d-79c004ef02af None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:25.338339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fa6df-b0ca-4273-be8d-79c004ef02af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:25.371692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bd2f0939-46d2-42f1-b4c2-a14168153a48 None None] MFA Rules not processed for user `046f89a576a74e099b22bfc04c1b8451`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:25.373102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bd2f0939-46d2-42f1-b4c2-a14168153a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:25.379157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4c5fa6df-b0ca-4273-be8d-79c004ef02af None None] MFA Rules not processed for user `9476b7020b2c40e9abb14d11b7481f77`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:25.382172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4c5fa6df-b0ca-4273-be8d-79c004ef02af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:25.401328 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3601/7259] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:25 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:25.405278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f0939-46d2-42f1-b4c2-a14168153a48 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:25.415362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f0939-46d2-42f1-b4c2-a14168153a48 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:25.421427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b32e7f-8739-46ad-82a3-4ad0eea411af tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '046f89a576a74e099b22bfc04c1b8451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81bc6d3965bd43ebb794e4ec08c3051c', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1662087544', 'tempest-DelegatedRole-2035643220'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:25.422157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b32e7f-8739-46ad-82a3-4ad0eea411af tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:25.422517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b32e7f-8739-46ad-82a3-4ad0eea411af tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:25.423146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b32e7f-8739-46ad-82a3-4ad0eea411af tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:25.423895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b32e7f-8739-46ad-82a3-4ad0eea411af tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] RBAC: Authorizing `identity:create_trust()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:25.425882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b32e7f-8739-46ad-82a3-4ad0eea411af tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:25.485232 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3659/7260] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:25.490174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fa6df-b0ca-4273-be8d-79c004ef02af None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:25.497227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4c5fa6df-b0ca-4273-be8d-79c004ef02af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:25.577276 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3602/7261] 10.222.0.17 () {60 vars in 1174 bytes} [Sat Oct 1 21:15:25 2022] POST /identity/v3/OS-TRUST/trusts => generated 834 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:25.583973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b32e7f-8739-46ad-82a3-4ad0eea411af tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:25.599383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b32e7f-8739-46ad-82a3-4ad0eea411af tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:25.605254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec64ac4-5ce4-4d82-b513-ee18ec969ae8 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:25.605893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec64ac4-5ce4-4d82-b513-ee18ec969ae8 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:25.606276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec64ac4-5ce4-4d82-b513-ee18ec969ae8 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:25.606727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec64ac4-5ce4-4d82-b513-ee18ec969ae8 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:25.607847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec64ac4-5ce4-4d82-b513-ee18ec969ae8 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:list_trusts()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:25.608775 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-dec64ac4-5ce4-4d82-b513-ee18ec969ae8 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_trusts": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:15:25.608775 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:15:25.608775 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:15:25.610681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec64ac4-5ce4-4d82-b513-ee18ec969ae8 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:25.628536 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3603/7262] 10.222.0.17 () {58 vars in 1155 bytes} [Sat Oct 1 21:15:25 2022] GET /identity/v3/OS-TRUST/trusts/ => generated 562 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:25.634690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec64ac4-5ce4-4d82-b513-ee18ec969ae8 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:25.646839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec64ac4-5ce4-4d82-b513-ee18ec969ae8 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:25.651449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55210bd5-244a-4cd9-86e6-602e717a9e2d tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '046f89a576a74e099b22bfc04c1b8451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81bc6d3965bd43ebb794e4ec08c3051c', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1662087544', 'tempest-DelegatedRole-2035643220'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:25.652026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55210bd5-244a-4cd9-86e6-602e717a9e2d tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:25.652305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55210bd5-244a-4cd9-86e6-602e717a9e2d tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:25.652645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55210bd5-244a-4cd9-86e6-602e717a9e2d tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] PATH_INFO: `/v3/OS-TRUST/trusts/a769bd0c58e54040ae0516b1b902c03a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:25.659846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fa6df-b0ca-4273-be8d-79c004ef02af None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:25.667072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55210bd5-244a-4cd9-86e6-602e717a9e2d tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] RBAC: Authorizing `identity:delete_trust(trust_id=a769bd0c58e54040ae0516b1b902c03a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:25.668719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55210bd5-244a-4cd9-86e6-602e717a9e2d tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:25.675351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f3572-9be5-4469-865c-0e60e4bb8d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:25.675995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f3572-9be5-4469-865c-0e60e4bb8d2d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:25.676507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f3572-9be5-4469-865c-0e60e4bb8d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:25.676614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f3572-9be5-4469-865c-0e60e4bb8d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:25.677699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f3572-9be5-4469-865c-0e60e4bb8d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:25.679902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f3572-9be5-4469-865c-0e60e4bb8d2d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:25.727074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-55210bd5-244a-4cd9-86e6-602e717a9e2d tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'a769bd0c58e54040ae0516b1b902c03a', 'request_id': 'req-55210bd5-244a-4cd9-86e6-602e717a9e2d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:25.729611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-55210bd5-244a-4cd9-86e6-602e717a9e2d tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': 'a769bd0c58e54040ae0516b1b902c03a', 'request_id': 'req-55210bd5-244a-4cd9-86e6-602e717a9e2d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:25.741445 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3660/7263] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:25 2022] POST /identity/v3/projects => generated 379 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:25.750436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f3572-9be5-4469-865c-0e60e4bb8d2d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:25.750939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-55210bd5-244a-4cd9-86e6-602e717a9e2d tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'a769bd0c58e54040ae0516b1b902c03a', 'request_id': 'req-55210bd5-244a-4cd9-86e6-602e717a9e2d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:25.754060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bb0f3572-9be5-4469-865c-0e60e4bb8d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:25.765501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:INSERT=5,keystone:SELECT=93,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:25.770473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:25.772024 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3604/7264] 10.222.0.17 () {58 vars in 1254 bytes} [Sat Oct 1 21:15:25 2022] DELETE /identity/v3/OS-TRUST/trusts/a769bd0c58e54040ae0516b1b902c03a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:25.776627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55210bd5-244a-4cd9-86e6-602e717a9e2d tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:25.783237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-55210bd5-244a-4cd9-86e6-602e717a9e2d tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:25.803890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55210bd5-244a-4cd9-86e6-602e717a9e2d tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:25.928230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f3572-9be5-4469-865c-0e60e4bb8d2d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:25.931580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77227ef-6a4b-4ccd-a774-83faefb1c47f tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '046f89a576a74e099b22bfc04c1b8451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '81bc6d3965bd43ebb794e4ec08c3051c', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1662087544', 'tempest-DelegatedRole-2035643220'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:25.932234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77227ef-6a4b-4ccd-a774-83faefb1c47f tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:25.932504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77227ef-6a4b-4ccd-a774-83faefb1c47f tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:25.932776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77227ef-6a4b-4ccd-a774-83faefb1c47f tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] PATH_INFO: `/v3/OS-TRUST/trusts/a769bd0c58e54040ae0516b1b902c03a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:25.940121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dde561-d164-475d-8822-4af637cc8f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:25.941759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dde561-d164-475d-8822-4af637cc8f24 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:25.942045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dde561-d164-475d-8822-4af637cc8f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:25.942345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dde561-d164-475d-8822-4af637cc8f24 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:25.944477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dde561-d164-475d-8822-4af637cc8f24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:25.950446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dde561-d164-475d-8822-4af637cc8f24 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:25.967754 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-a77227ef-6a4b-4ccd-a774-83faefb1c47f tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] Could not find trust: a769bd0c58e54040ae0516b1b902c03a.: keystone.exception.TrustNotFound: Could not find trust: a769bd0c58e54040ae0516b1b902c03a. Oct 01 21:15:25.967754 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:25.967754 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:15:25.967754 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:15:25.967754 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:15:25.967754 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:15:25.967754 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:15:25.967754 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:15:25.967754 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:15:25.967754 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:15:25.967754 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:15:25.967754 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:15:25.967754 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Oct 01 21:15:25.967754 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Oct 01 21:15:25.967754 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Oct 01 21:15:25.968819 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Oct 01 21:15:25.968819 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Oct 01 21:15:25.968819 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Oct 01 21:15:25.968819 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Oct 01 21:15:25.968819 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Oct 01 21:15:25.968819 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:15:25.968819 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:15:25.968819 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Oct 01 21:15:25.968819 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Oct 01 21:15:25.968819 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Oct 01 21:15:25.968819 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Oct 01 21:15:25.968819 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: a769bd0c58e54040ae0516b1b902c03a. Oct 01 21:15:25.968819 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:15:25.975004 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3605/7265] 10.222.0.17 () {58 vars in 1251 bytes} [Sat Oct 1 21:15:25 2022] GET /identity/v3/OS-TRUST/trusts/a769bd0c58e54040ae0516b1b902c03a => generated 111 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:15:25.988016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77227ef-6a4b-4ccd-a774-83faefb1c47f tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:25.989157 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3661/7266] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:25 2022] POST /identity/v3/users => generated 296 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:25.994636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dde561-d164-475d-8822-4af637cc8f24 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:25.999614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a77227ef-6a4b-4ccd-a774-83faefb1c47f tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:26.008360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dde561-d164-475d-8822-4af637cc8f24 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:26.014448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6e223-825c-4d2c-b3fe-72b445caffbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:26.015837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6e223-825c-4d2c-b3fe-72b445caffbb admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:26.021310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6e223-825c-4d2c-b3fe-72b445caffbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:26.024638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6e223-825c-4d2c-b3fe-72b445caffbb admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:26.025606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6e223-825c-4d2c-b3fe-72b445caffbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:26.028440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6e223-825c-4d2c-b3fe-72b445caffbb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:26.056713 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3662/7267] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:25 2022] GET /identity/v3/roles => generated 1893 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:26.062430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6e223-825c-4d2c-b3fe-72b445caffbb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:26.082624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6e223-825c-4d2c-b3fe-72b445caffbb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:26.090563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4675437-47f9-4d08-9ce6-50604d69eaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:26.092159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4675437-47f9-4d08-9ce6-50604d69eaf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:26.092676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4675437-47f9-4d08-9ce6-50604d69eaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:26.093159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4675437-47f9-4d08-9ce6-50604d69eaf0 admin admin] PATH_INFO: `/v3/projects/3a4e16bc67d745f7864ddf366c0a067f/users/f07b4382a49046bfb100fbfbbb4b896a/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:26.110924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4675437-47f9-4d08-9ce6-50604d69eaf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a4e16bc67d745f7864ddf366c0a067f, user_id=f07b4382a49046bfb100fbfbbb4b896a, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:26.115493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4675437-47f9-4d08-9ce6-50604d69eaf0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:26.140582 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3663/7268] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:26 2022] PUT /identity/v3/projects/3a4e16bc67d745f7864ddf366c0a067f/users/f07b4382a49046bfb100fbfbbb4b896a/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:26.146166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4675437-47f9-4d08-9ce6-50604d69eaf0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:26.167870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77227ef-6a4b-4ccd-a774-83faefb1c47f tempest-TrustsV3TestJSON-956325957 tempest-user-551649634] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:26.236223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4675437-47f9-4d08-9ce6-50604d69eaf0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:26.242013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e932c-00cf-419e-bd52-052338c86610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:26.242731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e932c-00cf-419e-bd52-052338c86610 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:26.243079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e932c-00cf-419e-bd52-052338c86610 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:26.243423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e932c-00cf-419e-bd52-052338c86610 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:26.244129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e932c-00cf-419e-bd52-052338c86610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:26.245791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e932c-00cf-419e-bd52-052338c86610 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:26.245997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0f4b0-0900-445c-9750-50108c82abae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:26.246679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0f4b0-0900-445c-9750-50108c82abae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:26.246956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0f4b0-0900-445c-9750-50108c82abae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:26.247242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0f4b0-0900-445c-9750-50108c82abae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles/bf269a7c52874649ae3a56dc70fdb1f9` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:26.248668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0f4b0-0900-445c-9750-50108c82abae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_role(role_id=bf269a7c52874649ae3a56dc70fdb1f9)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:26.250321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0f4b0-0900-445c-9750-50108c82abae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:26.262206 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3664/7269] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:26 2022] GET /identity/v3/roles => generated 1893 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:26.263250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e932c-00cf-419e-bd52-052338c86610 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:26.277627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e932c-00cf-419e-bd52-052338c86610 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:26.286097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265f280-c9d7-4ff4-bd36-c91921b31ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:26.287738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265f280-c9d7-4ff4-bd36-c91921b31ab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:26.288719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265f280-c9d7-4ff4-bd36-c91921b31ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:26.289118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265f280-c9d7-4ff4-bd36-c91921b31ab1 admin admin] PATH_INFO: `/v3/projects/3a4e16bc67d745f7864ddf366c0a067f/users/f07b4382a49046bfb100fbfbbb4b896a/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:26.291456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3265f280-c9d7-4ff4-bd36-c91921b31ab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a4e16bc67d745f7864ddf366c0a067f, user_id=f07b4382a49046bfb100fbfbbb4b896a, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:26.294876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3265f280-c9d7-4ff4-bd36-c91921b31ab1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:26.311876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-39f0f4b0-0900-445c-9750-50108c82abae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invalidating the token cache because role bf269a7c52874649ae3a56dc70fdb1f9 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:26.312220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-39f0f4b0-0900-445c-9750-50108c82abae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:26.312918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-39f0f4b0-0900-445c-9750-50108c82abae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:26.314572 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3665/7270] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:26 2022] PUT /identity/v3/projects/3a4e16bc67d745f7864ddf366c0a067f/users/f07b4382a49046bfb100fbfbbb4b896a/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:26.321372 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3606/7271] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:25 2022] DELETE /identity/v3/roles/bf269a7c52874649ae3a56dc70fdb1f9 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:26.322406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265f280-c9d7-4ff4-bd36-c91921b31ab1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:26.324331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3265f280-c9d7-4ff4-bd36-c91921b31ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:26.391606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:26.421432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265f280-c9d7-4ff4-bd36-c91921b31ab1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:26.555821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53bea0f-283c-43fc-b692-8d7a815dfdcb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:26.556977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bea0f-283c-43fc-b692-8d7a815dfdcb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:26.557263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bea0f-283c-43fc-b692-8d7a815dfdcb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:26.557541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bea0f-283c-43fc-b692-8d7a815dfdcb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles/54806e8b7475469c908b578a645d0120` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:26.559053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53bea0f-283c-43fc-b692-8d7a815dfdcb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_role(role_id=54806e8b7475469c908b578a645d0120)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:26.560638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53bea0f-283c-43fc-b692-8d7a815dfdcb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:26.616504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c53bea0f-283c-43fc-b692-8d7a815dfdcb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invalidating the token cache because role 54806e8b7475469c908b578a645d0120 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:26.616881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c53bea0f-283c-43fc-b692-8d7a815dfdcb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:26.617858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c53bea0f-283c-43fc-b692-8d7a815dfdcb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:26.619908 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3666/7272] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:26 2022] DELETE /identity/v3/roles/54806e8b7475469c908b578a645d0120 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:26.624089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0f4b0-0900-445c-9750-50108c82abae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:26.636309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0f4b0-0900-445c-9750-50108c82abae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:26.650839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:26.709928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12578e-b63d-4b8a-90c9-7878547d8bf5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:26.710929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12578e-b63d-4b8a-90c9-7878547d8bf5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:26.711324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12578e-b63d-4b8a-90c9-7878547d8bf5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:26.711799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12578e-b63d-4b8a-90c9-7878547d8bf5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/users/046f89a576a74e099b22bfc04c1b8451` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:26.713568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e12578e-b63d-4b8a-90c9-7878547d8bf5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_user(user_id=046f89a576a74e099b22bfc04c1b8451)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:26.715995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e12578e-b63d-4b8a-90c9-7878547d8bf5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:26.795469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2e12578e-b63d-4b8a-90c9-7878547d8bf5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '046f89a576a74e099b22bfc04c1b8451'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:26.796450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2e12578e-b63d-4b8a-90c9-7878547d8bf5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '046f89a576a74e099b22bfc04c1b8451'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:26.797096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2e12578e-b63d-4b8a-90c9-7878547d8bf5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '046f89a576a74e099b22bfc04c1b8451'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:26.808929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2e12578e-b63d-4b8a-90c9-7878547d8bf5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '046f89a576a74e099b22bfc04c1b8451'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:26.836025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2e12578e-b63d-4b8a-90c9-7878547d8bf5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '046f89a576a74e099b22bfc04c1b8451'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:26.852335 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3607/7273] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:26 2022] DELETE /identity/v3/users/046f89a576a74e099b22bfc04c1b8451 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:26.856454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53bea0f-283c-43fc-b692-8d7a815dfdcb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:26.857899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c53bea0f-283c-43fc-b692-8d7a815dfdcb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:26.876947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53bea0f-283c-43fc-b692-8d7a815dfdcb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:27.002817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b646d52-f7f2-406e-9b4a-53fe5412ab70 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:27.003416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b646d52-f7f2-406e-9b4a-53fe5412ab70 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:27.003703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b646d52-f7f2-406e-9b4a-53fe5412ab70 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:27.003974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b646d52-f7f2-406e-9b4a-53fe5412ab70 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects/81bc6d3965bd43ebb794e4ec08c3051c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:27.011161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b646d52-f7f2-406e-9b4a-53fe5412ab70 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_project(project_id=81bc6d3965bd43ebb794e4ec08c3051c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:27.014317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b646d52-f7f2-406e-9b4a-53fe5412ab70 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:27.190650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7b646d52-f7f2-406e-9b4a-53fe5412ab70 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81bc6d3965bd43ebb794e4ec08c3051c', 'request_id': 'req-7b646d52-f7f2-406e-9b4a-53fe5412ab70'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:27.201900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7b646d52-f7f2-406e-9b4a-53fe5412ab70 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81bc6d3965bd43ebb794e4ec08c3051c', 'request_id': 'req-7b646d52-f7f2-406e-9b4a-53fe5412ab70'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:27.205191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:UPDATE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:27.213818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7b646d52-f7f2-406e-9b4a-53fe5412ab70 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81bc6d3965bd43ebb794e4ec08c3051c', 'request_id': 'req-7b646d52-f7f2-406e-9b4a-53fe5412ab70'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:27.248916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7b646d52-f7f2-406e-9b4a-53fe5412ab70 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] The token cache is being invalidate because project 81bc6d3965bd43ebb794e4ec08c3051c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:27.248916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7b646d52-f7f2-406e-9b4a-53fe5412ab70 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:27.249887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7b646d52-f7f2-406e-9b4a-53fe5412ab70 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:27.252378 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3667/7274] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:26 2022] DELETE /identity/v3/projects/81bc6d3965bd43ebb794e4ec08c3051c => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:27.260269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12578e-b63d-4b8a-90c9-7878547d8bf5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:27.262924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2e12578e-b63d-4b8a-90c9-7878547d8bf5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:27.285448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12578e-b63d-4b8a-90c9-7878547d8bf5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:27.404913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d32a8-2a8c-4309-b086-c50d99d5b3e6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:27.406566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d32a8-2a8c-4309-b086-c50d99d5b3e6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:27.407434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d32a8-2a8c-4309-b086-c50d99d5b3e6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:27.408527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d32a8-2a8c-4309-b086-c50d99d5b3e6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:27.409472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8d32a8-2a8c-4309-b086-c50d99d5b3e6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:27.419991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8d32a8-2a8c-4309-b086-c50d99d5b3e6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:27.474327 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3608/7275] 10.222.0.17 () {60 vars in 1152 bytes} [Sat Oct 1 21:15:27 2022] POST /identity/v3/projects => generated 328 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:27.475309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b646d52-f7f2-406e-9b4a-53fe5412ab70 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:27.476804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7b646d52-f7f2-406e-9b4a-53fe5412ab70 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:27.500316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b646d52-f7f2-406e-9b4a-53fe5412ab70 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:27.611014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f27b05-e96d-4d68-a666-1b88116d97bf tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:27.611647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f27b05-e96d-4d68-a666-1b88116d97bf tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:27.611931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f27b05-e96d-4d68-a666-1b88116d97bf tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:27.612259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f27b05-e96d-4d68-a666-1b88116d97bf tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:27.613771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f27b05-e96d-4d68-a666-1b88116d97bf tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:27.616743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f27b05-e96d-4d68-a666-1b88116d97bf tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:27.651160 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3668/7276] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:27 2022] POST /identity/v3/users => generated 416 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:27.652352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d32a8-2a8c-4309-b086-c50d99d5b3e6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:27.688109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d32a8-2a8c-4309-b086-c50d99d5b3e6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:27.702909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b486501-b774-42b6-98ca-5b1ca6fc0eeb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:27.703452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b486501-b774-42b6-98ca-5b1ca6fc0eeb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:27.703788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b486501-b774-42b6-98ca-5b1ca6fc0eeb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:27.704237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b486501-b774-42b6-98ca-5b1ca6fc0eeb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:27.706472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b486501-b774-42b6-98ca-5b1ca6fc0eeb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:27.708367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b486501-b774-42b6-98ca-5b1ca6fc0eeb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:27.747119 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3609/7277] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:15:27 2022] POST /identity/v3/roles => generated 244 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:27.752412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f27b05-e96d-4d68-a666-1b88116d97bf tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:27.770116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f27b05-e96d-4d68-a666-1b88116d97bf tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:27.775422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c919184-f86c-483f-a14f-2ae67beb9105 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:27.776387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c919184-f86c-483f-a14f-2ae67beb9105 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:27.776823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c919184-f86c-483f-a14f-2ae67beb9105 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:27.777328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c919184-f86c-483f-a14f-2ae67beb9105 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:27.778225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c919184-f86c-483f-a14f-2ae67beb9105 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:27.780131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c919184-f86c-483f-a14f-2ae67beb9105 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:27.817892 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3669/7278] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:15:27 2022] POST /identity/v3/roles => generated 246 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:27.822327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b486501-b774-42b6-98ca-5b1ca6fc0eeb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:27.844197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b486501-b774-42b6-98ca-5b1ca6fc0eeb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:27.872146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036cbc39-92a2-496c-86c4-5335fb71215d tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:27.874150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cbc39-92a2-496c-86c4-5335fb71215d tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:27.874150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cbc39-92a2-496c-86c4-5335fb71215d tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:27.874150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cbc39-92a2-496c-86c4-5335fb71215d tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects/364b59857ff542a0bb5f584f12a0e46e/users/638e4cb82aa24393a817aec2d4b31d2c/roles/490d7b733f1c4ff0acee117956a93ee6` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:27.877683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:27.905557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036cbc39-92a2-496c-86c4-5335fb71215d tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_grant(project_id=364b59857ff542a0bb5f584f12a0e46e, user_id=638e4cb82aa24393a817aec2d4b31d2c, role_id=490d7b733f1c4ff0acee117956a93ee6)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:27.909134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036cbc39-92a2-496c-86c4-5335fb71215d tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:27.935234 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3610/7279] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:27 2022] PUT /identity/v3/projects/364b59857ff542a0bb5f584f12a0e46e/users/638e4cb82aa24393a817aec2d4b31d2c/roles/490d7b733f1c4ff0acee117956a93ee6 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:27.939828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c919184-f86c-483f-a14f-2ae67beb9105 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:27.960597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c919184-f86c-483f-a14f-2ae67beb9105 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:28.089132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b4a47-3abd-4def-a731-c6d2e88c8203 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:28.089822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b4a47-3abd-4def-a731-c6d2e88c8203 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:28.090106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b4a47-3abd-4def-a731-c6d2e88c8203 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:28.090381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b4a47-3abd-4def-a731-c6d2e88c8203 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects/364b59857ff542a0bb5f584f12a0e46e/users/638e4cb82aa24393a817aec2d4b31d2c/roles/275358a0b0bf4c4f8d76af2b6b7f3b4d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:28.093241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b4a47-3abd-4def-a731-c6d2e88c8203 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_grant(project_id=364b59857ff542a0bb5f584f12a0e46e, user_id=638e4cb82aa24393a817aec2d4b31d2c, role_id=275358a0b0bf4c4f8d76af2b6b7f3b4d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:28.097996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b4a47-3abd-4def-a731-c6d2e88c8203 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:28.125033 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3670/7280] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:27 2022] PUT /identity/v3/projects/364b59857ff542a0bb5f584f12a0e46e/users/638e4cb82aa24393a817aec2d4b31d2c/roles/275358a0b0bf4c4f8d76af2b6b7f3b4d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:28.128968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036cbc39-92a2-496c-86c4-5335fb71215d tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:28.142935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036cbc39-92a2-496c-86c4-5335fb71215d tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:28.256546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d7937-aff2-443a-8dee-1712e8dc5794 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:28.258726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d7937-aff2-443a-8dee-1712e8dc5794 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:28.259278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d7937-aff2-443a-8dee-1712e8dc5794 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:28.261591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d7937-aff2-443a-8dee-1712e8dc5794 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:28.263537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d7937-aff2-443a-8dee-1712e8dc5794 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:28.267591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d7937-aff2-443a-8dee-1712e8dc5794 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:28.496790 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3611/7281] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:28 2022] GET /identity/v3/users => generated 7786 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:28.501675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4b5ba-8393-4d4b-8299-b9746fb63ccb None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:28.502330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4b5ba-8393-4d4b-8299-b9746fb63ccb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:28.502330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4b5ba-8393-4d4b-8299-b9746fb63ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:28.570027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-eab4b5ba-8393-4d4b-8299-b9746fb63ccb None None] MFA Rules not processed for user `638e4cb82aa24393a817aec2d4b31d2c`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:28.571364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-eab4b5ba-8393-4d4b-8299-b9746fb63ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:28.626898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=2,keystone:INSERT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:28.666398 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3671/7282] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:28 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:28.677614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26947f-cdc7-4152-a0f3-0d5ed6ab9204 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:28.678193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26947f-cdc7-4152-a0f3-0d5ed6ab9204 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:28.678193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26947f-cdc7-4152-a0f3-0d5ed6ab9204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:28.724930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2d26947f-cdc7-4152-a0f3-0d5ed6ab9204 None None] MFA Rules not processed for user `638e4cb82aa24393a817aec2d4b31d2c`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:28.729185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2d26947f-cdc7-4152-a0f3-0d5ed6ab9204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:28.759053 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3612/7283] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:28 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:15:28.764277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4b5ba-8393-4d4b-8299-b9746fb63ccb None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:28.778998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4b5ba-8393-4d4b-8299-b9746fb63ccb None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:28.791150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fc21f-b264-4ea6-8ddb-b25f8d472f9a tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '638e4cb82aa24393a817aec2d4b31d2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '364b59857ff542a0bb5f584f12a0e46e', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-497900542', 'tempest-DelegatedRole-1760350693'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:28.791800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fc21f-b264-4ea6-8ddb-b25f8d472f9a tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:28.792084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fc21f-b264-4ea6-8ddb-b25f8d472f9a tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:28.792359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fc21f-b264-4ea6-8ddb-b25f8d472f9a tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:28.793140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8fc21f-b264-4ea6-8ddb-b25f8d472f9a tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] RBAC: Authorizing `identity:create_trust()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:28.794332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8fc21f-b264-4ea6-8ddb-b25f8d472f9a tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:28.944604 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3672/7284] 10.222.0.17 () {60 vars in 1174 bytes} [Sat Oct 1 21:15:28 2022] POST /identity/v3/OS-TRUST/trusts => generated 834 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:28.948427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26947f-cdc7-4152-a0f3-0d5ed6ab9204 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:28.967360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26947f-cdc7-4152-a0f3-0d5ed6ab9204 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:28.987819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da82c5-5534-4a25-866e-8a5f6096309d tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '638e4cb82aa24393a817aec2d4b31d2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '364b59857ff542a0bb5f584f12a0e46e', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-497900542', 'tempest-DelegatedRole-1760350693'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:28.990406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da82c5-5534-4a25-866e-8a5f6096309d tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:28.990784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da82c5-5534-4a25-866e-8a5f6096309d tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:28.991593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da82c5-5534-4a25-866e-8a5f6096309d tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:28.991945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da82c5-5534-4a25-866e-8a5f6096309d tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] RBAC: Authorizing `identity:list_trusts_for_trustor()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:28.994635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da82c5-5534-4a25-866e-8a5f6096309d tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:29.025837 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3613/7285] 10.222.0.17 () {58 vars in 1252 bytes} [Sat Oct 1 21:15:28 2022] GET /identity/v3/OS-TRUST/trusts/?trustor_user_id=638e4cb82aa24393a817aec2d4b31d2c => generated 611 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:29.027542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fc21f-b264-4ea6-8ddb-b25f8d472f9a tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:29.043469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fc21f-b264-4ea6-8ddb-b25f8d472f9a tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:29.058203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964462c-70a1-427e-88a1-b4b887608231 tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '638e4cb82aa24393a817aec2d4b31d2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '364b59857ff542a0bb5f584f12a0e46e', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-497900542', 'tempest-DelegatedRole-1760350693'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:29.061254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964462c-70a1-427e-88a1-b4b887608231 tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:29.061254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964462c-70a1-427e-88a1-b4b887608231 tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:29.061254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964462c-70a1-427e-88a1-b4b887608231 tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] PATH_INFO: `/v3/OS-TRUST/trusts/86bcd3de2c124722be624a9db3a3db52` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:29.083547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964462c-70a1-427e-88a1-b4b887608231 tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] RBAC: Authorizing `identity:delete_trust(trust_id=86bcd3de2c124722be624a9db3a3db52)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:29.085249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964462c-70a1-427e-88a1-b4b887608231 tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:29.127910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0964462c-70a1-427e-88a1-b4b887608231 tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '86bcd3de2c124722be624a9db3a3db52', 'request_id': 'req-0964462c-70a1-427e-88a1-b4b887608231'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:29.128743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0964462c-70a1-427e-88a1-b4b887608231 tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '86bcd3de2c124722be624a9db3a3db52', 'request_id': 'req-0964462c-70a1-427e-88a1-b4b887608231'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:29.129669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0964462c-70a1-427e-88a1-b4b887608231 tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '86bcd3de2c124722be624a9db3a3db52', 'request_id': 'req-0964462c-70a1-427e-88a1-b4b887608231'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:29.157897 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3673/7286] 10.222.0.17 () {58 vars in 1254 bytes} [Sat Oct 1 21:15:29 2022] DELETE /identity/v3/OS-TRUST/trusts/86bcd3de2c124722be624a9db3a3db52 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:29.158811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da82c5-5534-4a25-866e-8a5f6096309d tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:29.161746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-94da82c5-5534-4a25-866e-8a5f6096309d tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:29.180650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da82c5-5534-4a25-866e-8a5f6096309d tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:29.187259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd0b87-aad0-4ce6-88cf-ad099d71c931 tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '638e4cb82aa24393a817aec2d4b31d2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '364b59857ff542a0bb5f584f12a0e46e', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-497900542', 'tempest-DelegatedRole-1760350693'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:29.188252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd0b87-aad0-4ce6-88cf-ad099d71c931 tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:29.190070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd0b87-aad0-4ce6-88cf-ad099d71c931 tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:29.190731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd0b87-aad0-4ce6-88cf-ad099d71c931 tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] PATH_INFO: `/v3/OS-TRUST/trusts/86bcd3de2c124722be624a9db3a3db52` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:29.202514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-4ffd0b87-aad0-4ce6-88cf-ad099d71c931 tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] Could not find trust: 86bcd3de2c124722be624a9db3a3db52.: keystone.exception.TrustNotFound: Could not find trust: 86bcd3de2c124722be624a9db3a3db52. Oct 01 21:15:29.202514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:29.202514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:15:29.202514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:15:29.202514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:15:29.202514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:15:29.202514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:15:29.202514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:15:29.202514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:15:29.202514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:15:29.202514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:15:29.202514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:15:29.202514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Oct 01 21:15:29.202514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Oct 01 21:15:29.202514 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Oct 01 21:15:29.203593 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Oct 01 21:15:29.203593 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Oct 01 21:15:29.203593 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Oct 01 21:15:29.203593 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Oct 01 21:15:29.203593 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Oct 01 21:15:29.203593 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:15:29.203593 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:15:29.203593 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Oct 01 21:15:29.203593 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Oct 01 21:15:29.203593 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Oct 01 21:15:29.203593 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Oct 01 21:15:29.203593 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 86bcd3de2c124722be624a9db3a3db52. Oct 01 21:15:29.203593 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:15:29.206433 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3614/7287] 10.222.0.17 () {58 vars in 1251 bytes} [Sat Oct 1 21:15:29 2022] GET /identity/v3/OS-TRUST/trusts/86bcd3de2c124722be624a9db3a3db52 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:15:29.212954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964462c-70a1-427e-88a1-b4b887608231 tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:29.215020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0964462c-70a1-427e-88a1-b4b887608231 tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:29.244289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964462c-70a1-427e-88a1-b4b887608231 tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:29.249589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32726350-29f6-42f1-b47c-90edb56817a0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:29.250290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32726350-29f6-42f1-b47c-90edb56817a0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:29.250626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32726350-29f6-42f1-b47c-90edb56817a0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:29.250909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32726350-29f6-42f1-b47c-90edb56817a0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles/275358a0b0bf4c4f8d76af2b6b7f3b4d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:29.252952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32726350-29f6-42f1-b47c-90edb56817a0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_role(role_id=275358a0b0bf4c4f8d76af2b6b7f3b4d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:29.255383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32726350-29f6-42f1-b47c-90edb56817a0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:29.332996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-32726350-29f6-42f1-b47c-90edb56817a0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invalidating the token cache because role 275358a0b0bf4c4f8d76af2b6b7f3b4d has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:29.333448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-32726350-29f6-42f1-b47c-90edb56817a0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:29.334242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-32726350-29f6-42f1-b47c-90edb56817a0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:29.341629 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3674/7288] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:29 2022] DELETE /identity/v3/roles/275358a0b0bf4c4f8d76af2b6b7f3b4d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:29.342127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd0b87-aad0-4ce6-88cf-ad099d71c931 tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:29.343928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4ffd0b87-aad0-4ce6-88cf-ad099d71c931 tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:29.374843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd0b87-aad0-4ce6-88cf-ad099d71c931 tempest-TrustsV3TestJSON-310231059 tempest-user-1476182589] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:29.498079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5159d-de37-4553-b7a4-6c90913d940b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:29.498829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5159d-de37-4553-b7a4-6c90913d940b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:29.499310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5159d-de37-4553-b7a4-6c90913d940b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:29.499415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5159d-de37-4553-b7a4-6c90913d940b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles/490d7b733f1c4ff0acee117956a93ee6` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:29.501864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5159d-de37-4553-b7a4-6c90913d940b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_role(role_id=490d7b733f1c4ff0acee117956a93ee6)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:29.515227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5159d-de37-4553-b7a4-6c90913d940b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:29.588068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c0d5159d-de37-4553-b7a4-6c90913d940b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invalidating the token cache because role 490d7b733f1c4ff0acee117956a93ee6 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:29.588375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c0d5159d-de37-4553-b7a4-6c90913d940b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:29.591311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c0d5159d-de37-4553-b7a4-6c90913d940b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:29.594531 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3615/7289] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:29 2022] DELETE /identity/v3/roles/490d7b733f1c4ff0acee117956a93ee6 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:29.599898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32726350-29f6-42f1-b47c-90edb56817a0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:29.618900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32726350-29f6-42f1-b47c-90edb56817a0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:29.689666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:UPDATE=1,keystone:DELETE=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:29.725557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766e182-f391-4f73-8e40-09240e07c5ff tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:29.726614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766e182-f391-4f73-8e40-09240e07c5ff tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:29.739931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766e182-f391-4f73-8e40-09240e07c5ff tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:29.740874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766e182-f391-4f73-8e40-09240e07c5ff tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/users/638e4cb82aa24393a817aec2d4b31d2c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:29.743447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766e182-f391-4f73-8e40-09240e07c5ff tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_user(user_id=638e4cb82aa24393a817aec2d4b31d2c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:29.747532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766e182-f391-4f73-8e40-09240e07c5ff tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:29.905181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2766e182-f391-4f73-8e40-09240e07c5ff tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '638e4cb82aa24393a817aec2d4b31d2c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:29.923600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2766e182-f391-4f73-8e40-09240e07c5ff tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '638e4cb82aa24393a817aec2d4b31d2c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:29.948908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2766e182-f391-4f73-8e40-09240e07c5ff tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '638e4cb82aa24393a817aec2d4b31d2c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:29.955306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2766e182-f391-4f73-8e40-09240e07c5ff tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '638e4cb82aa24393a817aec2d4b31d2c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:29.969084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2766e182-f391-4f73-8e40-09240e07c5ff tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '638e4cb82aa24393a817aec2d4b31d2c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:29.982287 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3675/7290] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:29 2022] DELETE /identity/v3/users/638e4cb82aa24393a817aec2d4b31d2c => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:29.983645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5159d-de37-4553-b7a4-6c90913d940b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:29.985108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c0d5159d-de37-4553-b7a4-6c90913d940b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:29.998981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5159d-de37-4553-b7a4-6c90913d940b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:30.019501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:30.107843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72984209-464c-4b25-bc12-bc5e0f291940 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:30.108557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72984209-464c-4b25-bc12-bc5e0f291940 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:30.108843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72984209-464c-4b25-bc12-bc5e0f291940 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:30.109122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72984209-464c-4b25-bc12-bc5e0f291940 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects/364b59857ff542a0bb5f584f12a0e46e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:30.111149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72984209-464c-4b25-bc12-bc5e0f291940 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_project(project_id=364b59857ff542a0bb5f584f12a0e46e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:30.113387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72984209-464c-4b25-bc12-bc5e0f291940 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:30.184457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-72984209-464c-4b25-bc12-bc5e0f291940 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '364b59857ff542a0bb5f584f12a0e46e', 'request_id': 'req-72984209-464c-4b25-bc12-bc5e0f291940'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:30.196470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-72984209-464c-4b25-bc12-bc5e0f291940 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '364b59857ff542a0bb5f584f12a0e46e', 'request_id': 'req-72984209-464c-4b25-bc12-bc5e0f291940'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:30.206251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-72984209-464c-4b25-bc12-bc5e0f291940 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '364b59857ff542a0bb5f584f12a0e46e', 'request_id': 'req-72984209-464c-4b25-bc12-bc5e0f291940'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:30.221450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-72984209-464c-4b25-bc12-bc5e0f291940 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] The token cache is being invalidate because project 364b59857ff542a0bb5f584f12a0e46e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:30.221450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-72984209-464c-4b25-bc12-bc5e0f291940 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:30.222025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-72984209-464c-4b25-bc12-bc5e0f291940 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:30.224249 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3616/7291] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:29 2022] DELETE /identity/v3/projects/364b59857ff542a0bb5f584f12a0e46e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:30.230462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766e182-f391-4f73-8e40-09240e07c5ff tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:30.231224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2766e182-f391-4f73-8e40-09240e07c5ff tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:30.245475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766e182-f391-4f73-8e40-09240e07c5ff tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:30.322966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfa951-8e3e-49e8-b2f2-271bd5b180f3 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:30.323874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfa951-8e3e-49e8-b2f2-271bd5b180f3 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:30.324356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfa951-8e3e-49e8-b2f2-271bd5b180f3 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:30.324818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfa951-8e3e-49e8-b2f2-271bd5b180f3 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:30.325628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfa951-8e3e-49e8-b2f2-271bd5b180f3 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:30.328107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfa951-8e3e-49e8-b2f2-271bd5b180f3 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:30.374930 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3676/7292] 10.222.0.17 () {60 vars in 1152 bytes} [Sat Oct 1 21:15:30 2022] POST /identity/v3/projects => generated 328 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:30.378868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72984209-464c-4b25-bc12-bc5e0f291940 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:30.381759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-72984209-464c-4b25-bc12-bc5e0f291940 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:30.417707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72984209-464c-4b25-bc12-bc5e0f291940 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:30.496929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbf901-d2f9-4d06-826b-ea5f9f50527c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:30.498868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbf901-d2f9-4d06-826b-ea5f9f50527c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:30.499161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbf901-d2f9-4d06-826b-ea5f9f50527c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:30.499491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbf901-d2f9-4d06-826b-ea5f9f50527c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:30.501224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbf901-d2f9-4d06-826b-ea5f9f50527c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:30.503307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbf901-d2f9-4d06-826b-ea5f9f50527c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:30.560039 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3617/7293] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:30 2022] POST /identity/v3/users => generated 413 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:30.565047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfa951-8e3e-49e8-b2f2-271bd5b180f3 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:30.580038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfa951-8e3e-49e8-b2f2-271bd5b180f3 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:30.591655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5facd-7792-44a2-bf73-a3f38caef91d tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:30.592463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5facd-7792-44a2-bf73-a3f38caef91d tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:30.593107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5facd-7792-44a2-bf73-a3f38caef91d tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:30.593469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5facd-7792-44a2-bf73-a3f38caef91d tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:30.595643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd5facd-7792-44a2-bf73-a3f38caef91d tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:30.598604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd5facd-7792-44a2-bf73-a3f38caef91d tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:30.706542 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3677/7294] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:15:30 2022] POST /identity/v3/roles => generated 243 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:30.709116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbf901-d2f9-4d06-826b-ea5f9f50527c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:30.730545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbf901-d2f9-4d06-826b-ea5f9f50527c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:30.738159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66859e98-8783-4544-ba6a-386bfe62a3bb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:30.738819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66859e98-8783-4544-ba6a-386bfe62a3bb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:30.739684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66859e98-8783-4544-ba6a-386bfe62a3bb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:30.739967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66859e98-8783-4544-ba6a-386bfe62a3bb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:30.742127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66859e98-8783-4544-ba6a-386bfe62a3bb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:30.749682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66859e98-8783-4544-ba6a-386bfe62a3bb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:30.827521 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3618/7295] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:15:30 2022] POST /identity/v3/roles => generated 247 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:30.831708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5facd-7792-44a2-bf73-a3f38caef91d tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:30.887440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5facd-7792-44a2-bf73-a3f38caef91d tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:30.919184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb22176-2837-4116-8263-c3344f8ec152 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:30.919878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb22176-2837-4116-8263-c3344f8ec152 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:30.920162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb22176-2837-4116-8263-c3344f8ec152 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:30.920450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb22176-2837-4116-8263-c3344f8ec152 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects/306916f9bdd34819a2112cedf981e069/users/17de92c5266e47c6b373da74d23a7044/roles/ee0f311cb91a4805b8d1611f0f7dac7d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:30.974750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb22176-2837-4116-8263-c3344f8ec152 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_grant(project_id=306916f9bdd34819a2112cedf981e069, user_id=17de92c5266e47c6b373da74d23a7044, role_id=ee0f311cb91a4805b8d1611f0f7dac7d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:30.977716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb22176-2837-4116-8263-c3344f8ec152 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:31.015860 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3678/7296] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:30 2022] PUT /identity/v3/projects/306916f9bdd34819a2112cedf981e069/users/17de92c5266e47c6b373da74d23a7044/roles/ee0f311cb91a4805b8d1611f0f7dac7d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:31.017418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66859e98-8783-4544-ba6a-386bfe62a3bb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:31.040725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66859e98-8783-4544-ba6a-386bfe62a3bb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:31.112691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:UPDATE=1,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:31.226539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964bd0a5-02fc-434b-8064-35522ef07257 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:31.227465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bd0a5-02fc-434b-8064-35522ef07257 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:31.228020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bd0a5-02fc-434b-8064-35522ef07257 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:31.228442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bd0a5-02fc-434b-8064-35522ef07257 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects/306916f9bdd34819a2112cedf981e069/users/17de92c5266e47c6b373da74d23a7044/roles/cce0848485bc434fa70d0546d095dede` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:31.231869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964bd0a5-02fc-434b-8064-35522ef07257 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_grant(project_id=306916f9bdd34819a2112cedf981e069, user_id=17de92c5266e47c6b373da74d23a7044, role_id=cce0848485bc434fa70d0546d095dede)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:31.235603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964bd0a5-02fc-434b-8064-35522ef07257 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:31.256221 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3619/7297] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:31 2022] PUT /identity/v3/projects/306916f9bdd34819a2112cedf981e069/users/17de92c5266e47c6b373da74d23a7044/roles/cce0848485bc434fa70d0546d095dede => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:31.260912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb22176-2837-4116-8263-c3344f8ec152 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:31.279286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb22176-2837-4116-8263-c3344f8ec152 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:31.317103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:31.405674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636a001-2807-42fb-a01f-a98ebc096cae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:31.406763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636a001-2807-42fb-a01f-a98ebc096cae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:31.407239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636a001-2807-42fb-a01f-a98ebc096cae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:31.407702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636a001-2807-42fb-a01f-a98ebc096cae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:31.408692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636a001-2807-42fb-a01f-a98ebc096cae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:31.412763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636a001-2807-42fb-a01f-a98ebc096cae tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:31.477475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2e4fc-7d0c-4a86-8f7d-68114c0ab62c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:31.478090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2e4fc-7d0c-4a86-8f7d-68114c0ab62c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:31.478628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2e4fc-7d0c-4a86-8f7d-68114c0ab62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:31.491654 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3679/7298] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:31 2022] GET /identity/v3/users => generated 7783 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:15:31.556340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9ad2e4fc-7d0c-4a86-8f7d-68114c0ab62c None None] MFA Rules not processed for user `17de92c5266e47c6b373da74d23a7044`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:31.557746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9ad2e4fc-7d0c-4a86-8f7d-68114c0ab62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:31.659825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa5514-114b-4529-9a18-b5931033ae0e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:31.660115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa5514-114b-4529-9a18-b5931033ae0e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:31.660398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa5514-114b-4529-9a18-b5931033ae0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:31.668667 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3620/7299] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:31 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:31.689629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7faa5514-114b-4529-9a18-b5931033ae0e None None] MFA Rules not processed for user `17de92c5266e47c6b373da74d23a7044`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:31.691349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7faa5514-114b-4529-9a18-b5931033ae0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:31.706728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2e4fc-7d0c-4a86-8f7d-68114c0ab62c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:31.721763 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3680/7300] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:31 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:31.723089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2e4fc-7d0c-4a86-8f7d-68114c0ab62c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:31.730058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a0863-307d-4b7f-b2f8-1eb598bad679 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17de92c5266e47c6b373da74d23a7044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '306916f9bdd34819a2112cedf981e069', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1704165163', 'tempest-DelegatedRole-632600839'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:31.730911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a0863-307d-4b7f-b2f8-1eb598bad679 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:31.731241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a0863-307d-4b7f-b2f8-1eb598bad679 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:31.731657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a0863-307d-4b7f-b2f8-1eb598bad679 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:31.732346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a0863-307d-4b7f-b2f8-1eb598bad679 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: Authorizing `identity:create_trust()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:31.733549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a0863-307d-4b7f-b2f8-1eb598bad679 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:31.893720 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3621/7301] 10.222.0.17 () {60 vars in 1174 bytes} [Sat Oct 1 21:15:31 2022] POST /identity/v3/OS-TRUST/trusts => generated 858 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:31.894522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa5514-114b-4529-9a18-b5931033ae0e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:31.916906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa5514-114b-4529-9a18-b5931033ae0e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:31.946528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a74c89-2b5b-47ca-a0f8-8c320ec63c42 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17de92c5266e47c6b373da74d23a7044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '306916f9bdd34819a2112cedf981e069', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1704165163', 'tempest-DelegatedRole-632600839'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:31.946891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a74c89-2b5b-47ca-a0f8-8c320ec63c42 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:31.947153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a74c89-2b5b-47ca-a0f8-8c320ec63c42 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:31.947430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a74c89-2b5b-47ca-a0f8-8c320ec63c42 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] PATH_INFO: `/v3/OS-TRUST/trusts/08b0189d22b146e1b58326474cf74f29` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:31.956162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a74c89-2b5b-47ca-a0f8-8c320ec63c42 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: Authorizing `identity:get_trust(trust_id=08b0189d22b146e1b58326474cf74f29)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:31.964193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a74c89-2b5b-47ca-a0f8-8c320ec63c42 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:31.991481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a0863-307d-4b7f-b2f8-1eb598bad679 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:32.003882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a0863-307d-4b7f-b2f8-1eb598bad679 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:32.004168 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3681/7302] 10.222.0.17 () {58 vars in 1251 bytes} [Sat Oct 1 21:15:31 2022] GET /identity/v3/OS-TRUST/trusts/08b0189d22b146e1b58326474cf74f29 => generated 858 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:32.008992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a596be-7f1c-4e42-bca0-b958aecbef0d tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17de92c5266e47c6b373da74d23a7044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '306916f9bdd34819a2112cedf981e069', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1704165163', 'tempest-DelegatedRole-632600839'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:32.009948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a596be-7f1c-4e42-bca0-b958aecbef0d tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:32.010305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a596be-7f1c-4e42-bca0-b958aecbef0d tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:32.010681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a596be-7f1c-4e42-bca0-b958aecbef0d tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] PATH_INFO: `/v3/OS-TRUST/trusts/08b0189d22b146e1b58326474cf74f29/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:32.017986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a596be-7f1c-4e42-bca0-b958aecbef0d tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=08b0189d22b146e1b58326474cf74f29)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:32.019654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a596be-7f1c-4e42-bca0-b958aecbef0d tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:32.033660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a74c89-2b5b-47ca-a0f8-8c320ec63c42 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:32.044857 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3622/7303] 10.222.0.17 () {58 vars in 1269 bytes} [Sat Oct 1 21:15:31 2022] GET /identity/v3/OS-TRUST/trusts/08b0189d22b146e1b58326474cf74f29/roles => generated 370 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:32.048260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a74c89-2b5b-47ca-a0f8-8c320ec63c42 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:32.054473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ceccd-f15f-4a2f-91ed-e51f17c23af6 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17de92c5266e47c6b373da74d23a7044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '306916f9bdd34819a2112cedf981e069', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1704165163', 'tempest-DelegatedRole-632600839'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:32.056407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ceccd-f15f-4a2f-91ed-e51f17c23af6 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:32.057053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ceccd-f15f-4a2f-91ed-e51f17c23af6 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:32.057458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ceccd-f15f-4a2f-91ed-e51f17c23af6 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] PATH_INFO: `/v3/OS-TRUST/trusts/08b0189d22b146e1b58326474cf74f29/roles/ee0f311cb91a4805b8d1611f0f7dac7d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:32.070061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ceccd-f15f-4a2f-91ed-e51f17c23af6 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: Authorizing `identity:get_role_for_trust(trust_id=08b0189d22b146e1b58326474cf74f29, role_id=ee0f311cb91a4805b8d1611f0f7dac7d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:32.072574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ceccd-f15f-4a2f-91ed-e51f17c23af6 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:32.092743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a596be-7f1c-4e42-bca0-b958aecbef0d tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:32.103835 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3682/7304] 10.222.0.17 () {58 vars in 1368 bytes} [Sat Oct 1 21:15:32 2022] GET /identity/v3/OS-TRUST/trusts/08b0189d22b146e1b58326474cf74f29/roles/ee0f311cb91a4805b8d1611f0f7dac7d => generated 243 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:32.105247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a596be-7f1c-4e42-bca0-b958aecbef0d tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:32.110062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac68fde-c970-43a7-8cf8-51957d5ce165 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17de92c5266e47c6b373da74d23a7044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '306916f9bdd34819a2112cedf981e069', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1704165163', 'tempest-DelegatedRole-632600839'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:32.111304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac68fde-c970-43a7-8cf8-51957d5ce165 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] REQUEST_METHOD: `HEAD` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:32.111654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac68fde-c970-43a7-8cf8-51957d5ce165 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:32.111936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac68fde-c970-43a7-8cf8-51957d5ce165 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] PATH_INFO: `/v3/OS-TRUST/trusts/08b0189d22b146e1b58326474cf74f29/roles/ee0f311cb91a4805b8d1611f0f7dac7d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:32.120413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac68fde-c970-43a7-8cf8-51957d5ce165 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: Authorizing `identity:get_role_for_trust(trust_id=08b0189d22b146e1b58326474cf74f29, role_id=ee0f311cb91a4805b8d1611f0f7dac7d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:32.121884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac68fde-c970-43a7-8cf8-51957d5ce165 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:32.123404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:32.135925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ceccd-f15f-4a2f-91ed-e51f17c23af6 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:32.136404 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3623/7305] 10.222.0.17 () {58 vars in 1369 bytes} [Sat Oct 1 21:15:32 2022] HEAD /identity/v3/OS-TRUST/trusts/08b0189d22b146e1b58326474cf74f29/roles/ee0f311cb91a4805b8d1611f0f7dac7d => generated 0 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:32.148084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ceccd-f15f-4a2f-91ed-e51f17c23af6 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:32.153932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192d6d3-56c2-49c5-94f8-f93ad9ec9d92 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17de92c5266e47c6b373da74d23a7044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '306916f9bdd34819a2112cedf981e069', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1704165163', 'tempest-DelegatedRole-632600839'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:32.154805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192d6d3-56c2-49c5-94f8-f93ad9ec9d92 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:32.155168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192d6d3-56c2-49c5-94f8-f93ad9ec9d92 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:32.155655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192d6d3-56c2-49c5-94f8-f93ad9ec9d92 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] PATH_INFO: `/v3/OS-TRUST/trusts/08b0189d22b146e1b58326474cf74f29/roles/cce0848485bc434fa70d0546d095dede` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:32.164138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192d6d3-56c2-49c5-94f8-f93ad9ec9d92 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: Authorizing `identity:get_role_for_trust(trust_id=08b0189d22b146e1b58326474cf74f29, role_id=cce0848485bc434fa70d0546d095dede)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:32.166455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192d6d3-56c2-49c5-94f8-f93ad9ec9d92 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:32.189829 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-c192d6d3-56c2-49c5-94f8-f93ad9ec9d92 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Could not find role: cce0848485bc434fa70d0546d095dede.: keystone.exception.RoleNotFound: Could not find role: cce0848485bc434fa70d0546d095dede. Oct 01 21:15:32.189829 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:32.189829 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:15:32.189829 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:15:32.189829 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:15:32.189829 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:15:32.189829 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:15:32.189829 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:15:32.189829 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:15:32.189829 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:15:32.189829 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:15:32.189829 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:15:32.189829 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Oct 01 21:15:32.189829 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 21:15:32.189829 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: cce0848485bc434fa70d0546d095dede. Oct 01 21:15:32.190799 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:15:32.197162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac68fde-c970-43a7-8cf8-51957d5ce165 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:32.208743 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3683/7306] 10.222.0.17 () {58 vars in 1368 bytes} [Sat Oct 1 21:15:32 2022] GET /identity/v3/OS-TRUST/trusts/08b0189d22b146e1b58326474cf74f29/roles/cce0848485bc434fa70d0546d095dede => generated 110 bytes in 73 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:15:32.210189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac68fde-c970-43a7-8cf8-51957d5ce165 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:32.216025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe18970-87d4-42de-9bb6-ee26581e2282 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17de92c5266e47c6b373da74d23a7044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '306916f9bdd34819a2112cedf981e069', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1704165163', 'tempest-DelegatedRole-632600839'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:32.216960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe18970-87d4-42de-9bb6-ee26581e2282 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] REQUEST_METHOD: `HEAD` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:32.217254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe18970-87d4-42de-9bb6-ee26581e2282 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:32.217529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe18970-87d4-42de-9bb6-ee26581e2282 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] PATH_INFO: `/v3/OS-TRUST/trusts/08b0189d22b146e1b58326474cf74f29/roles/cce0848485bc434fa70d0546d095dede` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:32.225128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe18970-87d4-42de-9bb6-ee26581e2282 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: Authorizing `identity:get_role_for_trust(trust_id=08b0189d22b146e1b58326474cf74f29, role_id=cce0848485bc434fa70d0546d095dede)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:32.227052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe18970-87d4-42de-9bb6-ee26581e2282 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:32.240731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-9fe18970-87d4-42de-9bb6-ee26581e2282 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Could not find role: cce0848485bc434fa70d0546d095dede.: keystone.exception.RoleNotFound: Could not find role: cce0848485bc434fa70d0546d095dede. Oct 01 21:15:32.240731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:32.240731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:15:32.240731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:15:32.240731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:15:32.240731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:15:32.240731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:15:32.240731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:15:32.240731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:15:32.240731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:15:32.240731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:15:32.240731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:15:32.240731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Oct 01 21:15:32.240731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 21:15:32.240731 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: cce0848485bc434fa70d0546d095dede. Oct 01 21:15:32.241703 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:15:32.242031 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3624/7307] 10.222.0.17 () {58 vars in 1369 bytes} [Sat Oct 1 21:15:32 2022] HEAD /identity/v3/OS-TRUST/trusts/08b0189d22b146e1b58326474cf74f29/roles/cce0848485bc434fa70d0546d095dede => generated 0 bytes in 47 msecs (HTTP/1.1 404) 5 headers in 184 bytes (0 switches on core 0) Oct 01 21:15:32.247810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192d6d3-56c2-49c5-94f8-f93ad9ec9d92 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:32.262339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192d6d3-56c2-49c5-94f8-f93ad9ec9d92 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:32.267346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da691ab1-c246-4c0f-9792-9f32d4c2e818 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17de92c5266e47c6b373da74d23a7044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '306916f9bdd34819a2112cedf981e069', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1704165163', 'tempest-DelegatedRole-632600839'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:32.268724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da691ab1-c246-4c0f-9792-9f32d4c2e818 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:32.268931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da691ab1-c246-4c0f-9792-9f32d4c2e818 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:32.269216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da691ab1-c246-4c0f-9792-9f32d4c2e818 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] PATH_INFO: `/v3/OS-TRUST/trusts/08b0189d22b146e1b58326474cf74f29` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:32.278180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da691ab1-c246-4c0f-9792-9f32d4c2e818 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: Authorizing `identity:delete_trust(trust_id=08b0189d22b146e1b58326474cf74f29)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:32.279896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da691ab1-c246-4c0f-9792-9f32d4c2e818 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:32.303125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-da691ab1-c246-4c0f-9792-9f32d4c2e818 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '08b0189d22b146e1b58326474cf74f29', 'request_id': 'req-da691ab1-c246-4c0f-9792-9f32d4c2e818'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:32.304108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-da691ab1-c246-4c0f-9792-9f32d4c2e818 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '08b0189d22b146e1b58326474cf74f29', 'request_id': 'req-da691ab1-c246-4c0f-9792-9f32d4c2e818'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:32.304932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-da691ab1-c246-4c0f-9792-9f32d4c2e818 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '08b0189d22b146e1b58326474cf74f29', 'request_id': 'req-da691ab1-c246-4c0f-9792-9f32d4c2e818'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:32.335812 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3684/7308] 10.222.0.17 () {58 vars in 1254 bytes} [Sat Oct 1 21:15:32 2022] DELETE /identity/v3/OS-TRUST/trusts/08b0189d22b146e1b58326474cf74f29 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:32.337196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe18970-87d4-42de-9bb6-ee26581e2282 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:32.339624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9fe18970-87d4-42de-9bb6-ee26581e2282 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:32.362911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe18970-87d4-42de-9bb6-ee26581e2282 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:32.369996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e36352-6ba4-4e6c-abee-d382092544fa tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17de92c5266e47c6b373da74d23a7044', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '306916f9bdd34819a2112cedf981e069', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1704165163', 'tempest-DelegatedRole-632600839'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:32.370782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e36352-6ba4-4e6c-abee-d382092544fa tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:32.371108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e36352-6ba4-4e6c-abee-d382092544fa tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:32.371392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e36352-6ba4-4e6c-abee-d382092544fa tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] PATH_INFO: `/v3/OS-TRUST/trusts/08b0189d22b146e1b58326474cf74f29` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:32.383628 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-47e36352-6ba4-4e6c-abee-d382092544fa tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Could not find trust: 08b0189d22b146e1b58326474cf74f29.: keystone.exception.TrustNotFound: Could not find trust: 08b0189d22b146e1b58326474cf74f29. Oct 01 21:15:32.383628 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:32.383628 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:15:32.383628 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:15:32.383628 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:15:32.383628 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:15:32.383628 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:15:32.383628 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:15:32.383628 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:15:32.383628 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:15:32.383628 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:15:32.383628 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:15:32.383628 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Oct 01 21:15:32.383628 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Oct 01 21:15:32.383628 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Oct 01 21:15:32.384842 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Oct 01 21:15:32.384842 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Oct 01 21:15:32.384842 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Oct 01 21:15:32.384842 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Oct 01 21:15:32.384842 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Oct 01 21:15:32.384842 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:15:32.384842 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:15:32.384842 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Oct 01 21:15:32.384842 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Oct 01 21:15:32.384842 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Oct 01 21:15:32.384842 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Oct 01 21:15:32.384842 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 08b0189d22b146e1b58326474cf74f29. Oct 01 21:15:32.384842 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:15:32.397121 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3625/7309] 10.222.0.17 () {58 vars in 1251 bytes} [Sat Oct 1 21:15:32 2022] GET /identity/v3/OS-TRUST/trusts/08b0189d22b146e1b58326474cf74f29 => generated 111 bytes in 60 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:15:32.401198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da691ab1-c246-4c0f-9792-9f32d4c2e818 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:32.410178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-da691ab1-c246-4c0f-9792-9f32d4c2e818 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:32.459137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da691ab1-c246-4c0f-9792-9f32d4c2e818 tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:32.466038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fbe33e-6535-4661-8eaa-8b2686e12a1e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:32.466892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbe33e-6535-4661-8eaa-8b2686e12a1e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:32.467302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbe33e-6535-4661-8eaa-8b2686e12a1e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:32.468142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbe33e-6535-4661-8eaa-8b2686e12a1e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles/cce0848485bc434fa70d0546d095dede` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:32.470641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fbe33e-6535-4661-8eaa-8b2686e12a1e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_role(role_id=cce0848485bc434fa70d0546d095dede)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:32.472540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fbe33e-6535-4661-8eaa-8b2686e12a1e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:32.568403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d4fbe33e-6535-4661-8eaa-8b2686e12a1e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invalidating the token cache because role cce0848485bc434fa70d0546d095dede has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:32.569264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d4fbe33e-6535-4661-8eaa-8b2686e12a1e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:32.571885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d4fbe33e-6535-4661-8eaa-8b2686e12a1e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:32.578112 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3685/7310] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:32 2022] DELETE /identity/v3/roles/cce0848485bc434fa70d0546d095dede => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:32.579650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e36352-6ba4-4e6c-abee-d382092544fa tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:32.581800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-47e36352-6ba4-4e6c-abee-d382092544fa tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:32.604718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e36352-6ba4-4e6c-abee-d382092544fa tempest-TrustsV3TestJSON-825942563 tempest-user-332997439] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:32.711652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c4c7c-3b90-43bd-9bd1-46e4079bf3b5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:32.712522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c4c7c-3b90-43bd-9bd1-46e4079bf3b5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:32.712925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c4c7c-3b90-43bd-9bd1-46e4079bf3b5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:32.713211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c4c7c-3b90-43bd-9bd1-46e4079bf3b5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles/ee0f311cb91a4805b8d1611f0f7dac7d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:32.716185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c4c7c-3b90-43bd-9bd1-46e4079bf3b5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_role(role_id=ee0f311cb91a4805b8d1611f0f7dac7d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:32.718134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c4c7c-3b90-43bd-9bd1-46e4079bf3b5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:32.806889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0f0c4c7c-3b90-43bd-9bd1-46e4079bf3b5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invalidating the token cache because role ee0f311cb91a4805b8d1611f0f7dac7d has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:32.807320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0f0c4c7c-3b90-43bd-9bd1-46e4079bf3b5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:32.808153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0f0c4c7c-3b90-43bd-9bd1-46e4079bf3b5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:32.824328 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3626/7311] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:32 2022] DELETE /identity/v3/roles/ee0f311cb91a4805b8d1611f0f7dac7d => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:32.828267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fbe33e-6535-4661-8eaa-8b2686e12a1e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:32.850099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fbe33e-6535-4661-8eaa-8b2686e12a1e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:32.941151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:33.052737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a80a6-2b11-491d-b39a-db40eb14a588 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:33.053404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a80a6-2b11-491d-b39a-db40eb14a588 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:33.053684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a80a6-2b11-491d-b39a-db40eb14a588 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:33.053962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a80a6-2b11-491d-b39a-db40eb14a588 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/users/17de92c5266e47c6b373da74d23a7044` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:33.056968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a80a6-2b11-491d-b39a-db40eb14a588 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_user(user_id=17de92c5266e47c6b373da74d23a7044)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:33.059005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a80a6-2b11-491d-b39a-db40eb14a588 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:33.176946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4b7a80a6-2b11-491d-b39a-db40eb14a588 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17de92c5266e47c6b373da74d23a7044'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:33.186876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4b7a80a6-2b11-491d-b39a-db40eb14a588 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17de92c5266e47c6b373da74d23a7044'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:33.209369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4b7a80a6-2b11-491d-b39a-db40eb14a588 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17de92c5266e47c6b373da74d23a7044'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:33.210281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4b7a80a6-2b11-491d-b39a-db40eb14a588 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17de92c5266e47c6b373da74d23a7044'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:33.221715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4b7a80a6-2b11-491d-b39a-db40eb14a588 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17de92c5266e47c6b373da74d23a7044'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:33.231315 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3686/7312] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:32 2022] DELETE /identity/v3/users/17de92c5266e47c6b373da74d23a7044 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:33.235557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c4c7c-3b90-43bd-9bd1-46e4079bf3b5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:33.237150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0f0c4c7c-3b90-43bd-9bd1-46e4079bf3b5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:33.258682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c4c7c-3b90-43bd-9bd1-46e4079bf3b5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:33.402820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82b465-fa59-4083-8b84-7b2f5de96338 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:33.404095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82b465-fa59-4083-8b84-7b2f5de96338 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:33.404461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82b465-fa59-4083-8b84-7b2f5de96338 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:33.404831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82b465-fa59-4083-8b84-7b2f5de96338 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects/306916f9bdd34819a2112cedf981e069` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:33.409161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f82b465-fa59-4083-8b84-7b2f5de96338 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_project(project_id=306916f9bdd34819a2112cedf981e069)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:33.415216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f82b465-fa59-4083-8b84-7b2f5de96338 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:33.481004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:33.505648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4f82b465-fa59-4083-8b84-7b2f5de96338 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '306916f9bdd34819a2112cedf981e069', 'request_id': 'req-4f82b465-fa59-4083-8b84-7b2f5de96338'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:33.517945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4f82b465-fa59-4083-8b84-7b2f5de96338 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '306916f9bdd34819a2112cedf981e069', 'request_id': 'req-4f82b465-fa59-4083-8b84-7b2f5de96338'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:33.527826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4f82b465-fa59-4083-8b84-7b2f5de96338 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '306916f9bdd34819a2112cedf981e069', 'request_id': 'req-4f82b465-fa59-4083-8b84-7b2f5de96338'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:33.561429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4f82b465-fa59-4083-8b84-7b2f5de96338 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] The token cache is being invalidate because project 306916f9bdd34819a2112cedf981e069 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:33.561925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4f82b465-fa59-4083-8b84-7b2f5de96338 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:33.564215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4f82b465-fa59-4083-8b84-7b2f5de96338 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:33.571642 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3627/7313] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:33 2022] DELETE /identity/v3/projects/306916f9bdd34819a2112cedf981e069 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:33.573271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a80a6-2b11-491d-b39a-db40eb14a588 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:33.575572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4b7a80a6-2b11-491d-b39a-db40eb14a588 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:33.601827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a80a6-2b11-491d-b39a-db40eb14a588 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:33.736360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b0a51-13f1-4dbe-8e1a-79ffdab587d7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:33.737099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b0a51-13f1-4dbe-8e1a-79ffdab587d7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:33.737373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b0a51-13f1-4dbe-8e1a-79ffdab587d7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:33.737649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b0a51-13f1-4dbe-8e1a-79ffdab587d7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:33.738281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b0a51-13f1-4dbe-8e1a-79ffdab587d7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:33.740296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b0a51-13f1-4dbe-8e1a-79ffdab587d7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:33.788882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82b465-fa59-4083-8b84-7b2f5de96338 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:33.790432 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3687/7314] 10.222.0.17 () {60 vars in 1152 bytes} [Sat Oct 1 21:15:33 2022] POST /identity/v3/projects => generated 329 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:33.790731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4f82b465-fa59-4083-8b84-7b2f5de96338 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:33.816016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82b465-fa59-4083-8b84-7b2f5de96338 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:33.905619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e912ae5d-7d52-41a0-811f-1fddd5bc0891 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:33.906245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912ae5d-7d52-41a0-811f-1fddd5bc0891 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:33.906590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912ae5d-7d52-41a0-811f-1fddd5bc0891 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:33.906840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912ae5d-7d52-41a0-811f-1fddd5bc0891 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:33.907630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e912ae5d-7d52-41a0-811f-1fddd5bc0891 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:33.909584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e912ae5d-7d52-41a0-811f-1fddd5bc0891 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:33.943197 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3628/7315] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:33 2022] POST /identity/v3/users => generated 413 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:33.945019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b0a51-13f1-4dbe-8e1a-79ffdab587d7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:33.975711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b0a51-13f1-4dbe-8e1a-79ffdab587d7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:33.984321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228a66d-27e5-45dd-9f39-cbbbb133f875 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:33.985735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228a66d-27e5-45dd-9f39-cbbbb133f875 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:33.985924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228a66d-27e5-45dd-9f39-cbbbb133f875 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:33.986323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228a66d-27e5-45dd-9f39-cbbbb133f875 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:33.989550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228a66d-27e5-45dd-9f39-cbbbb133f875 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:33.996555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228a66d-27e5-45dd-9f39-cbbbb133f875 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:34.022060 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3688/7316] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:15:33 2022] POST /identity/v3/roles => generated 244 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:34.027656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e912ae5d-7d52-41a0-811f-1fddd5bc0891 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:34.070334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e912ae5d-7d52-41a0-811f-1fddd5bc0891 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:34.086602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc98e04-4f24-4d9d-a99e-51a1ede3c8ba tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:34.088361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc98e04-4f24-4d9d-a99e-51a1ede3c8ba tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:34.089449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc98e04-4f24-4d9d-a99e-51a1ede3c8ba tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:34.090522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc98e04-4f24-4d9d-a99e-51a1ede3c8ba tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:34.091339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc98e04-4f24-4d9d-a99e-51a1ede3c8ba tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:34.093068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc98e04-4f24-4d9d-a99e-51a1ede3c8ba tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:34.115086 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3629/7317] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:15:34 2022] POST /identity/v3/roles => generated 246 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:34.119497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228a66d-27e5-45dd-9f39-cbbbb133f875 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:34.140020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228a66d-27e5-45dd-9f39-cbbbb133f875 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:34.146922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115c88c-b583-4856-a44e-cd1b433febdc tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:34.147813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115c88c-b583-4856-a44e-cd1b433febdc tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:34.148884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115c88c-b583-4856-a44e-cd1b433febdc tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:34.149271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115c88c-b583-4856-a44e-cd1b433febdc tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects/777f624acef0485d8cba942de9930dac/users/6ebe5f76dcc14df19ceab0a36a3ab26d/roles/87197ac4ad7a4218a5891db73d059da9` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:34.171316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5115c88c-b583-4856-a44e-cd1b433febdc tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_grant(project_id=777f624acef0485d8cba942de9930dac, user_id=6ebe5f76dcc14df19ceab0a36a3ab26d, role_id=87197ac4ad7a4218a5891db73d059da9)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:34.174700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5115c88c-b583-4856-a44e-cd1b433febdc tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:34.179052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:34.198166 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3689/7318] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:34 2022] PUT /identity/v3/projects/777f624acef0485d8cba942de9930dac/users/6ebe5f76dcc14df19ceab0a36a3ab26d/roles/87197ac4ad7a4218a5891db73d059da9 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:34.203115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc98e04-4f24-4d9d-a99e-51a1ede3c8ba tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:34.219866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc98e04-4f24-4d9d-a99e-51a1ede3c8ba tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:34.344016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349d3cb-e365-4370-99ee-34f1a1ff77e9 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:34.344705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349d3cb-e365-4370-99ee-34f1a1ff77e9 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:34.344991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349d3cb-e365-4370-99ee-34f1a1ff77e9 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:34.345279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349d3cb-e365-4370-99ee-34f1a1ff77e9 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects/777f624acef0485d8cba942de9930dac/users/6ebe5f76dcc14df19ceab0a36a3ab26d/roles/ea2ec53422ae45a59ca2249dbd69676f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:34.349822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2349d3cb-e365-4370-99ee-34f1a1ff77e9 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_grant(project_id=777f624acef0485d8cba942de9930dac, user_id=6ebe5f76dcc14df19ceab0a36a3ab26d, role_id=ea2ec53422ae45a59ca2249dbd69676f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:34.356412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2349d3cb-e365-4370-99ee-34f1a1ff77e9 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:34.400871 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3630/7319] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:34 2022] PUT /identity/v3/projects/777f624acef0485d8cba942de9930dac/users/6ebe5f76dcc14df19ceab0a36a3ab26d/roles/ea2ec53422ae45a59ca2249dbd69676f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:34.404666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115c88c-b583-4856-a44e-cd1b433febdc tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:34.434377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115c88c-b583-4856-a44e-cd1b433febdc tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:34.554005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f4c35-0678-4500-9ab0-c0339076efe7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:34.555128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f4c35-0678-4500-9ab0-c0339076efe7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:34.555283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f4c35-0678-4500-9ab0-c0339076efe7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:34.555559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f4c35-0678-4500-9ab0-c0339076efe7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:34.559366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f4c35-0678-4500-9ab0-c0339076efe7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:34.560556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f4c35-0678-4500-9ab0-c0339076efe7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:34.597150 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3690/7320] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:34 2022] GET /identity/v3/users => generated 7783 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:34.602990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f76ca-167f-4cfe-b4cf-a7842b06bb2f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:34.603915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f76ca-167f-4cfe-b4cf-a7842b06bb2f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:34.604411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f76ca-167f-4cfe-b4cf-a7842b06bb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:34.661524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e04f76ca-167f-4cfe-b4cf-a7842b06bb2f None None] MFA Rules not processed for user `6ebe5f76dcc14df19ceab0a36a3ab26d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:34.663384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e04f76ca-167f-4cfe-b4cf-a7842b06bb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:34.695417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:UPDATE=1,keystone:DELETE=4,keystone:INSERT=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:34.813934 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3631/7321] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:34 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:34.816435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd70075-7223-4ae7-977b-a6a5b476d32b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:34.816816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd70075-7223-4ae7-977b-a6a5b476d32b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:34.817142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd70075-7223-4ae7-977b-a6a5b476d32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:34.860531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-dbd70075-7223-4ae7-977b-a6a5b476d32b None None] MFA Rules not processed for user `6ebe5f76dcc14df19ceab0a36a3ab26d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:34.861968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-dbd70075-7223-4ae7-977b-a6a5b476d32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:34.885083 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3691/7322] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:34 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:15:34.886870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f76ca-167f-4cfe-b4cf-a7842b06bb2f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:34.898029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f76ca-167f-4cfe-b4cf-a7842b06bb2f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:34.903475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcaa8c-671a-4f6e-940c-af89e2a59c53 tempest-TrustsV3TestJSON-1479908146 tempest-user-743485810] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebe5f76dcc14df19ceab0a36a3ab26d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '777f624acef0485d8cba942de9930dac', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1345871178', 'tempest-NotDelegatedRole-885631475'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:34.903475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcaa8c-671a-4f6e-940c-af89e2a59c53 tempest-TrustsV3TestJSON-1479908146 tempest-user-743485810] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:34.903764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcaa8c-671a-4f6e-940c-af89e2a59c53 tempest-TrustsV3TestJSON-1479908146 tempest-user-743485810] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:34.904088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcaa8c-671a-4f6e-940c-af89e2a59c53 tempest-TrustsV3TestJSON-1479908146 tempest-user-743485810] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:34.904873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debcaa8c-671a-4f6e-940c-af89e2a59c53 tempest-TrustsV3TestJSON-1479908146 tempest-user-743485810] RBAC: Authorizing `identity:create_trust()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:34.906354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debcaa8c-671a-4f6e-940c-af89e2a59c53 tempest-TrustsV3TestJSON-1479908146 tempest-user-743485810] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:35.082455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-debcaa8c-671a-4f6e-940c-af89e2a59c53 tempest-TrustsV3TestJSON-1479908146 tempest-user-743485810] Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationTimeStampError: Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 01 21:15:35.082455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:35.082455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py", line 44, in parse_isotime Oct 01 21:15:35.082455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return iso8601.parse_date(timestr) Oct 01 21:15:35.082455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/iso8601/iso8601.py", line 127, in parse_date Oct 01 21:15:35.082455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise ParseError(f"Unable to parse date string {datestring!r}") Oct 01 21:15:35.082455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application iso8601.iso8601.ParseError: Unable to parse date string 'bad.123Z' Oct 01 21:15:35.082455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:15:35.082455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:15:35.082455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:15:35.082455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:35.082455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/utils.py", line 346, in parse_expiration_date Oct 01 21:15:35.082455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application expiration_time = timeutils.parse_isotime(expiration_date) Oct 01 21:15:35.082455 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py", line 46, in parse_isotime Oct 01 21:15:35.084329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise ValueError(str(e)) Oct 01 21:15:35.084329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ValueError: Unable to parse date string 'bad.123Z' Oct 01 21:15:35.084329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:15:35.084329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:15:35.084329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:15:35.084329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:35.084329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:15:35.084329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:15:35.084329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:15:35.084329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:15:35.084329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:15:35.084329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:15:35.084329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:15:35.084329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:15:35.084329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:15:35.084329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:15:35.084329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 296, in post Oct 01 21:15:35.084329 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application trust['expires_at'] = self._parse_expiration_date( Oct 01 21:15:35.085572 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 127, in _parse_expiration_date Oct 01 21:15:35.085572 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return utils.parse_expiration_date(expiration_date) Oct 01 21:15:35.085572 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/utils.py", line 348, in parse_expiration_date Oct 01 21:15:35.085572 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ValidationTimeStampError() Oct 01 21:15:35.085572 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ValidationTimeStampError: Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 01 21:15:35.085572 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:15:35.088358 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3632/7323] 10.222.0.17 () {60 vars in 1174 bytes} [Sat Oct 1 21:15:34 2022] POST /identity/v3/OS-TRUST/trusts => generated 227 bytes in 202 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 01 21:15:35.092584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd70075-7223-4ae7-977b-a6a5b476d32b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:35.114821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd70075-7223-4ae7-977b-a6a5b476d32b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:35.137477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f72c40-24da-46e6-b2bc-942878fa6fc7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:35.138154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f72c40-24da-46e6-b2bc-942878fa6fc7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:35.138448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f72c40-24da-46e6-b2bc-942878fa6fc7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:35.138799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f72c40-24da-46e6-b2bc-942878fa6fc7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles/ea2ec53422ae45a59ca2249dbd69676f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:35.140344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f72c40-24da-46e6-b2bc-942878fa6fc7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_role(role_id=ea2ec53422ae45a59ca2249dbd69676f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:35.142136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f72c40-24da-46e6-b2bc-942878fa6fc7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:35.215585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-49f72c40-24da-46e6-b2bc-942878fa6fc7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invalidating the token cache because role ea2ec53422ae45a59ca2249dbd69676f has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:35.215883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-49f72c40-24da-46e6-b2bc-942878fa6fc7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:35.218636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-49f72c40-24da-46e6-b2bc-942878fa6fc7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:35.229072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcaa8c-671a-4f6e-940c-af89e2a59c53 tempest-TrustsV3TestJSON-1479908146 tempest-user-743485810] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:35.236574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-debcaa8c-671a-4f6e-940c-af89e2a59c53 tempest-TrustsV3TestJSON-1479908146 tempest-user-743485810] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:35.244382 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3692/7324] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:35 2022] DELETE /identity/v3/roles/ea2ec53422ae45a59ca2249dbd69676f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:35.285291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcaa8c-671a-4f6e-940c-af89e2a59c53 tempest-TrustsV3TestJSON-1479908146 tempest-user-743485810] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:35.429135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed32c4-4b82-4bff-920d-265ec279f21e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:35.432068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed32c4-4b82-4bff-920d-265ec279f21e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:35.432515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed32c4-4b82-4bff-920d-265ec279f21e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:35.432960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed32c4-4b82-4bff-920d-265ec279f21e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles/87197ac4ad7a4218a5891db73d059da9` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:35.439773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed32c4-4b82-4bff-920d-265ec279f21e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_role(role_id=87197ac4ad7a4218a5891db73d059da9)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:35.446083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed32c4-4b82-4bff-920d-265ec279f21e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:35.528528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-90ed32c4-4b82-4bff-920d-265ec279f21e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invalidating the token cache because role 87197ac4ad7a4218a5891db73d059da9 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:35.528839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-90ed32c4-4b82-4bff-920d-265ec279f21e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:35.531669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-90ed32c4-4b82-4bff-920d-265ec279f21e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:35.537369 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3633/7325] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:35 2022] DELETE /identity/v3/roles/87197ac4ad7a4218a5891db73d059da9 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:35.539623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f72c40-24da-46e6-b2bc-942878fa6fc7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:35.565057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f72c40-24da-46e6-b2bc-942878fa6fc7 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:35.688863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b51e24-994e-4b15-b6c9-bad800b50697 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:35.690567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b51e24-994e-4b15-b6c9-bad800b50697 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:35.690867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b51e24-994e-4b15-b6c9-bad800b50697 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:35.691153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b51e24-994e-4b15-b6c9-bad800b50697 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/users/6ebe5f76dcc14df19ceab0a36a3ab26d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:35.693676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b51e24-994e-4b15-b6c9-bad800b50697 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_user(user_id=6ebe5f76dcc14df19ceab0a36a3ab26d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:35.696798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b51e24-994e-4b15-b6c9-bad800b50697 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:35.784828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=10 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:35.791064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-50b51e24-994e-4b15-b6c9-bad800b50697 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ebe5f76dcc14df19ceab0a36a3ab26d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:35.812560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-50b51e24-994e-4b15-b6c9-bad800b50697 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ebe5f76dcc14df19ceab0a36a3ab26d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:35.849392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-50b51e24-994e-4b15-b6c9-bad800b50697 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ebe5f76dcc14df19ceab0a36a3ab26d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:35.850512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-50b51e24-994e-4b15-b6c9-bad800b50697 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ebe5f76dcc14df19ceab0a36a3ab26d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:35.868679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-50b51e24-994e-4b15-b6c9-bad800b50697 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ebe5f76dcc14df19ceab0a36a3ab26d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:35.876688 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3693/7326] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:35 2022] DELETE /identity/v3/users/6ebe5f76dcc14df19ceab0a36a3ab26d => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:35.881129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed32c4-4b82-4bff-920d-265ec279f21e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:35.884350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-90ed32c4-4b82-4bff-920d-265ec279f21e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:35.899647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed32c4-4b82-4bff-920d-265ec279f21e tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:35.934771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:36.013331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc992c-5cf6-4d0a-b9c1-f61fb3ed7107 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:36.013956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc992c-5cf6-4d0a-b9c1-f61fb3ed7107 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:36.014246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc992c-5cf6-4d0a-b9c1-f61fb3ed7107 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:36.015355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc992c-5cf6-4d0a-b9c1-f61fb3ed7107 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects/777f624acef0485d8cba942de9930dac` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:36.017386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc992c-5cf6-4d0a-b9c1-f61fb3ed7107 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_project(project_id=777f624acef0485d8cba942de9930dac)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:36.021033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc992c-5cf6-4d0a-b9c1-f61fb3ed7107 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:36.103099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c3dc992c-5cf6-4d0a-b9c1-f61fb3ed7107 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '777f624acef0485d8cba942de9930dac', 'request_id': 'req-c3dc992c-5cf6-4d0a-b9c1-f61fb3ed7107'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:36.116490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c3dc992c-5cf6-4d0a-b9c1-f61fb3ed7107 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '777f624acef0485d8cba942de9930dac', 'request_id': 'req-c3dc992c-5cf6-4d0a-b9c1-f61fb3ed7107'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:36.129478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c3dc992c-5cf6-4d0a-b9c1-f61fb3ed7107 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '777f624acef0485d8cba942de9930dac', 'request_id': 'req-c3dc992c-5cf6-4d0a-b9c1-f61fb3ed7107'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:36.151256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c3dc992c-5cf6-4d0a-b9c1-f61fb3ed7107 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] The token cache is being invalidate because project 777f624acef0485d8cba942de9930dac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:36.153914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c3dc992c-5cf6-4d0a-b9c1-f61fb3ed7107 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:36.156852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c3dc992c-5cf6-4d0a-b9c1-f61fb3ed7107 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:36.158427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cdcd2-ff8f-40fc-a695-0fa2f997d839 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:36.158939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cdcd2-ff8f-40fc-a695-0fa2f997d839 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:36.159036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cdcd2-ff8f-40fc-a695-0fa2f997d839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:36.163224 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3634/7327] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:35 2022] DELETE /identity/v3/projects/777f624acef0485d8cba942de9930dac => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:36.168692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc992c-5cf6-4d0a-b9c1-f61fb3ed7107 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:36.170881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c3dc992c-5cf6-4d0a-b9c1-f61fb3ed7107 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:36.200056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c28cdcd2-ff8f-40fc-a695-0fa2f997d839 None None] MFA Rules not processed for user `a2745d6923e347939e2c67015aefadda`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:36.201365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c28cdcd2-ff8f-40fc-a695-0fa2f997d839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:36.276152 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3694/7328] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:15:36.286596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc992c-5cf6-4d0a-b9c1-f61fb3ed7107 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:36.293142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d4ff1-8fc0-455f-b29d-0b52c6c1ed55 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:36.293841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d4ff1-8fc0-455f-b29d-0b52c6c1ed55 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:36.294126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d4ff1-8fc0-455f-b29d-0b52c6c1ed55 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:36.295279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d4ff1-8fc0-455f-b29d-0b52c6c1ed55 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:36.296224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d4ff1-8fc0-455f-b29d-0b52c6c1ed55 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:36.298616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d4ff1-8fc0-455f-b29d-0b52c6c1ed55 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:36.335240 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3635/7329] 10.222.0.17 () {60 vars in 1152 bytes} [Sat Oct 1 21:15:36 2022] POST /identity/v3/projects => generated 329 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:36.339414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28cdcd2-ff8f-40fc-a695-0fa2f997d839 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:36.366616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28cdcd2-ff8f-40fc-a695-0fa2f997d839 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:36.446284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519523df-804e-4dbd-8734-f2ee5e0a6056 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:36.446638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519523df-804e-4dbd-8734-f2ee5e0a6056 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:36.446937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519523df-804e-4dbd-8734-f2ee5e0a6056 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:36.447546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519523df-804e-4dbd-8734-f2ee5e0a6056 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:36.447864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519523df-804e-4dbd-8734-f2ee5e0a6056 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:36.452246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519523df-804e-4dbd-8734-f2ee5e0a6056 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:36.492920 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3695/7330] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:36 2022] POST /identity/v3/users => generated 416 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:36.494721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d4ff1-8fc0-455f-b29d-0b52c6c1ed55 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:36.509605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d4ff1-8fc0-455f-b29d-0b52c6c1ed55 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:36.516124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3f691-5534-4948-b48c-30ccd000da3a tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:36.516681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3f691-5534-4948-b48c-30ccd000da3a tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:36.516965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3f691-5534-4948-b48c-30ccd000da3a tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:36.517650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3f691-5534-4948-b48c-30ccd000da3a tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:36.517951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf3f691-5534-4948-b48c-30ccd000da3a tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:36.519780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf3f691-5534-4948-b48c-30ccd000da3a tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:36.547454 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3636/7331] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:15:36 2022] POST /identity/v3/roles => generated 244 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:36.551835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519523df-804e-4dbd-8734-f2ee5e0a6056 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:36.566383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519523df-804e-4dbd-8734-f2ee5e0a6056 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:36.574196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92af92-67c2-435e-83aa-5d49c625180b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:36.576823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92af92-67c2-435e-83aa-5d49c625180b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:36.577446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92af92-67c2-435e-83aa-5d49c625180b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:36.577948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92af92-67c2-435e-83aa-5d49c625180b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:36.579009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c92af92-67c2-435e-83aa-5d49c625180b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:36.581648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c92af92-67c2-435e-83aa-5d49c625180b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:36.637028 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3696/7332] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:15:36 2022] POST /identity/v3/roles => generated 247 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:36.637844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3f691-5534-4948-b48c-30ccd000da3a tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:36.651690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3f691-5534-4948-b48c-30ccd000da3a tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:36.663975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e7523-b1ea-40cd-b543-99819359ef3f tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:36.664751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e7523-b1ea-40cd-b543-99819359ef3f tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:36.665028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e7523-b1ea-40cd-b543-99819359ef3f tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:36.665298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e7523-b1ea-40cd-b543-99819359ef3f tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects/2864b5090843408daf48d9d64b0f1f0d/users/e80d473bc8ae44ebb0c55627c0aefddf/roles/45f7cab98e95416d86d2777d31c1777e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:36.708177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e7523-b1ea-40cd-b543-99819359ef3f tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_grant(project_id=2864b5090843408daf48d9d64b0f1f0d, user_id=e80d473bc8ae44ebb0c55627c0aefddf, role_id=45f7cab98e95416d86d2777d31c1777e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:36.713296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e7523-b1ea-40cd-b543-99819359ef3f tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:36.728055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:36.742389 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3637/7333] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:36 2022] PUT /identity/v3/projects/2864b5090843408daf48d9d64b0f1f0d/users/e80d473bc8ae44ebb0c55627c0aefddf/roles/45f7cab98e95416d86d2777d31c1777e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:36.743913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92af92-67c2-435e-83aa-5d49c625180b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:36.759579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92af92-67c2-435e-83aa-5d49c625180b tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:36.846112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:36.901624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e764ae-8030-48bb-a366-a2ebbcf206a3 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:36.902368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e764ae-8030-48bb-a366-a2ebbcf206a3 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:36.902716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e764ae-8030-48bb-a366-a2ebbcf206a3 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:36.903045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e764ae-8030-48bb-a366-a2ebbcf206a3 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects/2864b5090843408daf48d9d64b0f1f0d/users/e80d473bc8ae44ebb0c55627c0aefddf/roles/d20f5b34e87249bd8d9058252a9d7f7e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:36.910584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e764ae-8030-48bb-a366-a2ebbcf206a3 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_grant(project_id=2864b5090843408daf48d9d64b0f1f0d, user_id=e80d473bc8ae44ebb0c55627c0aefddf, role_id=d20f5b34e87249bd8d9058252a9d7f7e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:36.913730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e764ae-8030-48bb-a366-a2ebbcf206a3 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:36.946738 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3697/7334] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:36 2022] PUT /identity/v3/projects/2864b5090843408daf48d9d64b0f1f0d/users/e80d473bc8ae44ebb0c55627c0aefddf/roles/d20f5b34e87249bd8d9058252a9d7f7e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:36.951055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e7523-b1ea-40cd-b543-99819359ef3f tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:36.971002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e7523-b1ea-40cd-b543-99819359ef3f tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:37.113460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64625dc-48e4-48fe-bbdd-3491d8503e04 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:37.114540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64625dc-48e4-48fe-bbdd-3491d8503e04 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:37.115005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64625dc-48e4-48fe-bbdd-3491d8503e04 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:37.115452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64625dc-48e4-48fe-bbdd-3491d8503e04 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:37.116294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64625dc-48e4-48fe-bbdd-3491d8503e04 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:37.118255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64625dc-48e4-48fe-bbdd-3491d8503e04 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:37.161149 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3638/7335] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:36 2022] GET /identity/v3/users => generated 7786 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:37.166395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b09be-52f8-4d36-99e6-7bfab39809f4 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:37.166880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b09be-52f8-4d36-99e6-7bfab39809f4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:37.166959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b09be-52f8-4d36-99e6-7bfab39809f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:37.294343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-911b09be-52f8-4d36-99e6-7bfab39809f4 None None] MFA Rules not processed for user `e80d473bc8ae44ebb0c55627c0aefddf`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:37.296107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-911b09be-52f8-4d36-99e6-7bfab39809f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:37.484655 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3698/7336] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:37 2022] POST /identity/v3/auth/tokens => generated 3281 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:37.495666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf482f5a-b90b-4343-acd4-3589ba8a82b7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:37.496119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf482f5a-b90b-4343-acd4-3589ba8a82b7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:37.496405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf482f5a-b90b-4343-acd4-3589ba8a82b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:37.542008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-cf482f5a-b90b-4343-acd4-3589ba8a82b7 None None] MFA Rules not processed for user `e80d473bc8ae44ebb0c55627c0aefddf`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:37.545324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cf482f5a-b90b-4343-acd4-3589ba8a82b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:37.557192 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3639/7337] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:37 2022] POST /identity/v3/auth/tokens => generated 3281 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:37.561733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b09be-52f8-4d36-99e6-7bfab39809f4 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:37.575426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b09be-52f8-4d36-99e6-7bfab39809f4 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:37.590431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08745c2-5d47-4d91-9b20-43c71fb04110 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80d473bc8ae44ebb0c55627c0aefddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2864b5090843408daf48d9d64b0f1f0d', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2089535339', 'tempest-NotDelegatedRole-1152993550'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:37.591336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08745c2-5d47-4d91-9b20-43c71fb04110 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:37.591813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08745c2-5d47-4d91-9b20-43c71fb04110 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:37.592233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08745c2-5d47-4d91-9b20-43c71fb04110 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:37.593045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08745c2-5d47-4d91-9b20-43c71fb04110 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: Authorizing `identity:create_trust()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:37.594947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08745c2-5d47-4d91-9b20-43c71fb04110 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:37.741408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf482f5a-b90b-4343-acd4-3589ba8a82b7 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:37.747634 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3699/7338] 10.222.0.17 () {60 vars in 1174 bytes} [Sat Oct 1 21:15:37 2022] POST /identity/v3/OS-TRUST/trusts => generated 834 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:37.752625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf482f5a-b90b-4343-acd4-3589ba8a82b7 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:37.765502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c510b12-ca9a-42b3-aec2-c2e302b06558 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80d473bc8ae44ebb0c55627c0aefddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2864b5090843408daf48d9d64b0f1f0d', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2089535339', 'tempest-NotDelegatedRole-1152993550'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:37.766307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c510b12-ca9a-42b3-aec2-c2e302b06558 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:37.766639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c510b12-ca9a-42b3-aec2-c2e302b06558 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:37.767233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c510b12-ca9a-42b3-aec2-c2e302b06558 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] PATH_INFO: `/v3/OS-TRUST/trusts/122d9d9677c246b8848f654a657f17df` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:37.783948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c510b12-ca9a-42b3-aec2-c2e302b06558 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: Authorizing `identity:get_trust(trust_id=122d9d9677c246b8848f654a657f17df)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:37.786573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c510b12-ca9a-42b3-aec2-c2e302b06558 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:37.818272 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3640/7339] 10.222.0.17 () {58 vars in 1251 bytes} [Sat Oct 1 21:15:37 2022] GET /identity/v3/OS-TRUST/trusts/122d9d9677c246b8848f654a657f17df => generated 834 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:15:37.819061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08745c2-5d47-4d91-9b20-43c71fb04110 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:37.832182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08745c2-5d47-4d91-9b20-43c71fb04110 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:37.843730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c2742-6a1a-4bce-897b-f305bbeaeb70 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80d473bc8ae44ebb0c55627c0aefddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2864b5090843408daf48d9d64b0f1f0d', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2089535339', 'tempest-NotDelegatedRole-1152993550'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:37.844452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c2742-6a1a-4bce-897b-f305bbeaeb70 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:37.844678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c2742-6a1a-4bce-897b-f305bbeaeb70 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:37.844963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c2742-6a1a-4bce-897b-f305bbeaeb70 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] PATH_INFO: `/v3/OS-TRUST/trusts/122d9d9677c246b8848f654a657f17df/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:37.865665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c2742-6a1a-4bce-897b-f305bbeaeb70 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=122d9d9677c246b8848f654a657f17df)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:37.867317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c2742-6a1a-4bce-897b-f305bbeaeb70 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:37.883446 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3700/7340] 10.222.0.17 () {58 vars in 1269 bytes} [Sat Oct 1 21:15:37 2022] GET /identity/v3/OS-TRUST/trusts/122d9d9677c246b8848f654a657f17df/roles => generated 371 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:37.888525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c510b12-ca9a-42b3-aec2-c2e302b06558 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:37.912744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c510b12-ca9a-42b3-aec2-c2e302b06558 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:37.922033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327b5bd-e475-4251-bbb8-47a20104aa6e tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80d473bc8ae44ebb0c55627c0aefddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2864b5090843408daf48d9d64b0f1f0d', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2089535339', 'tempest-NotDelegatedRole-1152993550'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:37.924528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327b5bd-e475-4251-bbb8-47a20104aa6e tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:37.924754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327b5bd-e475-4251-bbb8-47a20104aa6e tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:37.925742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327b5bd-e475-4251-bbb8-47a20104aa6e tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] PATH_INFO: `/v3/OS-TRUST/trusts/122d9d9677c246b8848f654a657f17df/roles/45f7cab98e95416d86d2777d31c1777e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:37.951126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327b5bd-e475-4251-bbb8-47a20104aa6e tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: Authorizing `identity:get_role_for_trust(trust_id=122d9d9677c246b8848f654a657f17df, role_id=45f7cab98e95416d86d2777d31c1777e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:37.953480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327b5bd-e475-4251-bbb8-47a20104aa6e tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:37.962881 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3641/7341] 10.222.0.17 () {58 vars in 1368 bytes} [Sat Oct 1 21:15:37 2022] GET /identity/v3/OS-TRUST/trusts/122d9d9677c246b8848f654a657f17df/roles/45f7cab98e95416d86d2777d31c1777e => generated 244 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:37.968651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c2742-6a1a-4bce-897b-f305bbeaeb70 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:37.984469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c2742-6a1a-4bce-897b-f305bbeaeb70 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:37.991141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b0e7d-b86b-4ba4-9029-81ebdaad1ba9 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80d473bc8ae44ebb0c55627c0aefddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2864b5090843408daf48d9d64b0f1f0d', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2089535339', 'tempest-NotDelegatedRole-1152993550'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:37.991970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b0e7d-b86b-4ba4-9029-81ebdaad1ba9 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] REQUEST_METHOD: `HEAD` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:37.993095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b0e7d-b86b-4ba4-9029-81ebdaad1ba9 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:37.993444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b0e7d-b86b-4ba4-9029-81ebdaad1ba9 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] PATH_INFO: `/v3/OS-TRUST/trusts/122d9d9677c246b8848f654a657f17df/roles/45f7cab98e95416d86d2777d31c1777e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:38.016569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b0e7d-b86b-4ba4-9029-81ebdaad1ba9 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: Authorizing `identity:get_role_for_trust(trust_id=122d9d9677c246b8848f654a657f17df, role_id=45f7cab98e95416d86d2777d31c1777e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:38.017510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b0e7d-b86b-4ba4-9029-81ebdaad1ba9 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:38.032800 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3701/7342] 10.222.0.17 () {58 vars in 1369 bytes} [Sat Oct 1 21:15:37 2022] HEAD /identity/v3/OS-TRUST/trusts/122d9d9677c246b8848f654a657f17df/roles/45f7cab98e95416d86d2777d31c1777e => generated 0 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Oct 01 21:15:38.035749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327b5bd-e475-4251-bbb8-47a20104aa6e tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:38.071652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327b5bd-e475-4251-bbb8-47a20104aa6e tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:38.080180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353859ca-3427-48ee-afda-deb6b65c6ff1 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80d473bc8ae44ebb0c55627c0aefddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2864b5090843408daf48d9d64b0f1f0d', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2089535339', 'tempest-NotDelegatedRole-1152993550'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:38.081470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353859ca-3427-48ee-afda-deb6b65c6ff1 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:38.081944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353859ca-3427-48ee-afda-deb6b65c6ff1 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:38.082404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353859ca-3427-48ee-afda-deb6b65c6ff1 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] PATH_INFO: `/v3/OS-TRUST/trusts/122d9d9677c246b8848f654a657f17df/roles/d20f5b34e87249bd8d9058252a9d7f7e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:38.107822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353859ca-3427-48ee-afda-deb6b65c6ff1 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: Authorizing `identity:get_role_for_trust(trust_id=122d9d9677c246b8848f654a657f17df, role_id=d20f5b34e87249bd8d9058252a9d7f7e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:38.114451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353859ca-3427-48ee-afda-deb6b65c6ff1 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:38.137090 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-353859ca-3427-48ee-afda-deb6b65c6ff1 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Could not find role: d20f5b34e87249bd8d9058252a9d7f7e.: keystone.exception.RoleNotFound: Could not find role: d20f5b34e87249bd8d9058252a9d7f7e. Oct 01 21:15:38.137090 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:38.137090 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:15:38.137090 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:15:38.137090 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:15:38.137090 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:15:38.137090 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:15:38.137090 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:15:38.137090 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:15:38.137090 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:15:38.137090 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:15:38.137090 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:15:38.137090 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Oct 01 21:15:38.137090 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 21:15:38.137090 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: d20f5b34e87249bd8d9058252a9d7f7e. Oct 01 21:15:38.138071 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:15:38.144283 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3642/7343] 10.222.0.17 () {58 vars in 1368 bytes} [Sat Oct 1 21:15:38 2022] GET /identity/v3/OS-TRUST/trusts/122d9d9677c246b8848f654a657f17df/roles/d20f5b34e87249bd8d9058252a9d7f7e => generated 110 bytes in 107 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:15:38.145856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b0e7d-b86b-4ba4-9029-81ebdaad1ba9 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:38.157812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b0e7d-b86b-4ba4-9029-81ebdaad1ba9 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:38.167151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e291ee-887b-4e81-8c71-43b048272d6d tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80d473bc8ae44ebb0c55627c0aefddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2864b5090843408daf48d9d64b0f1f0d', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2089535339', 'tempest-NotDelegatedRole-1152993550'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:38.168969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e291ee-887b-4e81-8c71-43b048272d6d tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] REQUEST_METHOD: `HEAD` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:38.169253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e291ee-887b-4e81-8c71-43b048272d6d tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:38.171121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e291ee-887b-4e81-8c71-43b048272d6d tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] PATH_INFO: `/v3/OS-TRUST/trusts/122d9d9677c246b8848f654a657f17df/roles/d20f5b34e87249bd8d9058252a9d7f7e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:38.191670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e291ee-887b-4e81-8c71-43b048272d6d tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: Authorizing `identity:get_role_for_trust(trust_id=122d9d9677c246b8848f654a657f17df, role_id=d20f5b34e87249bd8d9058252a9d7f7e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:38.194781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e291ee-887b-4e81-8c71-43b048272d6d tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:38.198052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:38.212904 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-83e291ee-887b-4e81-8c71-43b048272d6d tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Could not find role: d20f5b34e87249bd8d9058252a9d7f7e.: keystone.exception.RoleNotFound: Could not find role: d20f5b34e87249bd8d9058252a9d7f7e. Oct 01 21:15:38.212904 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:38.212904 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:15:38.212904 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:15:38.212904 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:15:38.212904 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:15:38.212904 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:15:38.212904 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:15:38.212904 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:15:38.212904 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:15:38.212904 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:15:38.212904 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:15:38.212904 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Oct 01 21:15:38.212904 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 21:15:38.212904 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: d20f5b34e87249bd8d9058252a9d7f7e. Oct 01 21:15:38.214128 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:15:38.215973 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3702/7344] 10.222.0.17 () {58 vars in 1369 bytes} [Sat Oct 1 21:15:38 2022] HEAD /identity/v3/OS-TRUST/trusts/122d9d9677c246b8848f654a657f17df/roles/d20f5b34e87249bd8d9058252a9d7f7e => generated 0 bytes in 70 msecs (HTTP/1.1 404) 5 headers in 184 bytes (0 switches on core 0) Oct 01 21:15:38.219986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353859ca-3427-48ee-afda-deb6b65c6ff1 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:38.232679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353859ca-3427-48ee-afda-deb6b65c6ff1 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:38.237338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5594d-fe40-4234-95c1-7dc6b4740e19 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80d473bc8ae44ebb0c55627c0aefddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2864b5090843408daf48d9d64b0f1f0d', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2089535339', 'tempest-NotDelegatedRole-1152993550'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:38.237990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5594d-fe40-4234-95c1-7dc6b4740e19 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:38.238265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5594d-fe40-4234-95c1-7dc6b4740e19 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:38.238561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5594d-fe40-4234-95c1-7dc6b4740e19 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] PATH_INFO: `/v3/OS-TRUST/trusts/122d9d9677c246b8848f654a657f17df` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:38.254274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5594d-fe40-4234-95c1-7dc6b4740e19 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: Authorizing `identity:delete_trust(trust_id=122d9d9677c246b8848f654a657f17df)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:38.256532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5594d-fe40-4234-95c1-7dc6b4740e19 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:38.295942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3ad5594d-fe40-4234-95c1-7dc6b4740e19 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '122d9d9677c246b8848f654a657f17df', 'request_id': 'req-3ad5594d-fe40-4234-95c1-7dc6b4740e19'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:38.297214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3ad5594d-fe40-4234-95c1-7dc6b4740e19 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '122d9d9677c246b8848f654a657f17df', 'request_id': 'req-3ad5594d-fe40-4234-95c1-7dc6b4740e19'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:38.309841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3ad5594d-fe40-4234-95c1-7dc6b4740e19 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '122d9d9677c246b8848f654a657f17df', 'request_id': 'req-3ad5594d-fe40-4234-95c1-7dc6b4740e19'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:38.322020 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3643/7345] 10.222.0.17 () {58 vars in 1254 bytes} [Sat Oct 1 21:15:38 2022] DELETE /identity/v3/OS-TRUST/trusts/122d9d9677c246b8848f654a657f17df => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:38.324761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e291ee-887b-4e81-8c71-43b048272d6d tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:38.326756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-83e291ee-887b-4e81-8c71-43b048272d6d tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:38.347502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e291ee-887b-4e81-8c71-43b048272d6d tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:38.355503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e893603-6fe3-4e00-ac18-3f0813d4b512 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e80d473bc8ae44ebb0c55627c0aefddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2864b5090843408daf48d9d64b0f1f0d', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2089535339', 'tempest-NotDelegatedRole-1152993550'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:38.356164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e893603-6fe3-4e00-ac18-3f0813d4b512 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:38.356446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e893603-6fe3-4e00-ac18-3f0813d4b512 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:38.356724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e893603-6fe3-4e00-ac18-3f0813d4b512 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] PATH_INFO: `/v3/OS-TRUST/trusts/122d9d9677c246b8848f654a657f17df` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:38.368735 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-8e893603-6fe3-4e00-ac18-3f0813d4b512 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Could not find trust: 122d9d9677c246b8848f654a657f17df.: keystone.exception.TrustNotFound: Could not find trust: 122d9d9677c246b8848f654a657f17df. Oct 01 21:15:38.368735 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:38.368735 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:15:38.368735 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:15:38.368735 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:15:38.368735 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:15:38.368735 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:15:38.368735 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:15:38.368735 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:15:38.368735 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:15:38.368735 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:15:38.368735 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:15:38.368735 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Oct 01 21:15:38.368735 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Oct 01 21:15:38.368735 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Oct 01 21:15:38.369671 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Oct 01 21:15:38.369671 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Oct 01 21:15:38.369671 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Oct 01 21:15:38.369671 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Oct 01 21:15:38.369671 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Oct 01 21:15:38.369671 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:15:38.369671 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:15:38.369671 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Oct 01 21:15:38.369671 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Oct 01 21:15:38.369671 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Oct 01 21:15:38.369671 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Oct 01 21:15:38.369671 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 122d9d9677c246b8848f654a657f17df. Oct 01 21:15:38.369671 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:15:38.370948 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3703/7346] 10.222.0.17 () {58 vars in 1251 bytes} [Sat Oct 1 21:15:38 2022] GET /identity/v3/OS-TRUST/trusts/122d9d9677c246b8848f654a657f17df => generated 111 bytes in 49 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:15:38.374678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5594d-fe40-4234-95c1-7dc6b4740e19 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:38.376211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3ad5594d-fe40-4234-95c1-7dc6b4740e19 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:38.402657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5594d-fe40-4234-95c1-7dc6b4740e19 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:38.417493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcb5ed-f373-47ff-a613-4324c7e00151 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:38.418238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcb5ed-f373-47ff-a613-4324c7e00151 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:38.418737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcb5ed-f373-47ff-a613-4324c7e00151 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:38.418979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcb5ed-f373-47ff-a613-4324c7e00151 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles/d20f5b34e87249bd8d9058252a9d7f7e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:38.420896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fcb5ed-f373-47ff-a613-4324c7e00151 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_role(role_id=d20f5b34e87249bd8d9058252a9d7f7e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:38.422636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fcb5ed-f373-47ff-a613-4324c7e00151 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:38.479434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b9fcb5ed-f373-47ff-a613-4324c7e00151 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invalidating the token cache because role d20f5b34e87249bd8d9058252a9d7f7e has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:38.479434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b9fcb5ed-f373-47ff-a613-4324c7e00151 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:38.479919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b9fcb5ed-f373-47ff-a613-4324c7e00151 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:38.482434 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3644/7347] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:38 2022] DELETE /identity/v3/roles/d20f5b34e87249bd8d9058252a9d7f7e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:38.486603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e893603-6fe3-4e00-ac18-3f0813d4b512 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:38.488594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8e893603-6fe3-4e00-ac18-3f0813d4b512 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:38.505292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e893603-6fe3-4e00-ac18-3f0813d4b512 tempest-TrustsV3TestJSON-1267067645 tempest-user-1343183299] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:38.582799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac600a-73be-41de-961a-9cd754fb0f49 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:38.583448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac600a-73be-41de-961a-9cd754fb0f49 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:38.583738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac600a-73be-41de-961a-9cd754fb0f49 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:38.584058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac600a-73be-41de-961a-9cd754fb0f49 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles/45f7cab98e95416d86d2777d31c1777e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:38.586211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac600a-73be-41de-961a-9cd754fb0f49 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_role(role_id=45f7cab98e95416d86d2777d31c1777e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:38.588397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac600a-73be-41de-961a-9cd754fb0f49 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:38.658165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7aac600a-73be-41de-961a-9cd754fb0f49 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invalidating the token cache because role 45f7cab98e95416d86d2777d31c1777e has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:38.659050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7aac600a-73be-41de-961a-9cd754fb0f49 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:38.659873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7aac600a-73be-41de-961a-9cd754fb0f49 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:38.663197 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3704/7348] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:38 2022] DELETE /identity/v3/roles/45f7cab98e95416d86d2777d31c1777e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:38.667050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcb5ed-f373-47ff-a613-4324c7e00151 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:38.689210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcb5ed-f373-47ff-a613-4324c7e00151 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:38.719069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:38.816490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a7468-437c-457c-8c18-7912c1ea7fb6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:38.817819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a7468-437c-457c-8c18-7912c1ea7fb6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:38.817964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a7468-437c-457c-8c18-7912c1ea7fb6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:38.818238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a7468-437c-457c-8c18-7912c1ea7fb6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/users/e80d473bc8ae44ebb0c55627c0aefddf` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:38.820037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4a7468-437c-457c-8c18-7912c1ea7fb6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_user(user_id=e80d473bc8ae44ebb0c55627c0aefddf)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:38.821713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4a7468-437c-457c-8c18-7912c1ea7fb6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:38.891626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bf4a7468-437c-457c-8c18-7912c1ea7fb6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e80d473bc8ae44ebb0c55627c0aefddf'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:38.892296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bf4a7468-437c-457c-8c18-7912c1ea7fb6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e80d473bc8ae44ebb0c55627c0aefddf'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:38.893143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bf4a7468-437c-457c-8c18-7912c1ea7fb6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e80d473bc8ae44ebb0c55627c0aefddf'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:38.904928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bf4a7468-437c-457c-8c18-7912c1ea7fb6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e80d473bc8ae44ebb0c55627c0aefddf'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:38.934321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bf4a7468-437c-457c-8c18-7912c1ea7fb6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e80d473bc8ae44ebb0c55627c0aefddf'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:38.959981 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3645/7349] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:38 2022] DELETE /identity/v3/users/e80d473bc8ae44ebb0c55627c0aefddf => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:38.961316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac600a-73be-41de-961a-9cd754fb0f49 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:38.963044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7aac600a-73be-41de-961a-9cd754fb0f49 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:38.978619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac600a-73be-41de-961a-9cd754fb0f49 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:39.088379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfff94-a65f-478c-bb0d-9f18ec2e3971 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:39.088661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfff94-a65f-478c-bb0d-9f18ec2e3971 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:39.088943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfff94-a65f-478c-bb0d-9f18ec2e3971 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:39.089229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfff94-a65f-478c-bb0d-9f18ec2e3971 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects/2864b5090843408daf48d9d64b0f1f0d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:39.090729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfff94-a65f-478c-bb0d-9f18ec2e3971 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2864b5090843408daf48d9d64b0f1f0d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:39.092943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfff94-a65f-478c-bb0d-9f18ec2e3971 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:39.173022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5dcfff94-a65f-478c-bb0d-9f18ec2e3971 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2864b5090843408daf48d9d64b0f1f0d', 'request_id': 'req-5dcfff94-a65f-478c-bb0d-9f18ec2e3971'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:39.186047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5dcfff94-a65f-478c-bb0d-9f18ec2e3971 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2864b5090843408daf48d9d64b0f1f0d', 'request_id': 'req-5dcfff94-a65f-478c-bb0d-9f18ec2e3971'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:39.187702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:UPDATE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:39.196498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5dcfff94-a65f-478c-bb0d-9f18ec2e3971 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2864b5090843408daf48d9d64b0f1f0d', 'request_id': 'req-5dcfff94-a65f-478c-bb0d-9f18ec2e3971'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:39.220628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5dcfff94-a65f-478c-bb0d-9f18ec2e3971 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] The token cache is being invalidate because project 2864b5090843408daf48d9d64b0f1f0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:39.220990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5dcfff94-a65f-478c-bb0d-9f18ec2e3971 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:39.221679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5dcfff94-a65f-478c-bb0d-9f18ec2e3971 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:39.228677 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3705/7350] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:38 2022] DELETE /identity/v3/projects/2864b5090843408daf48d9d64b0f1f0d => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:39.230990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a7468-437c-457c-8c18-7912c1ea7fb6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:39.233744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bf4a7468-437c-457c-8c18-7912c1ea7fb6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:39.248513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a7468-437c-457c-8c18-7912c1ea7fb6 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:39.327704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd946cf-6ad3-476e-a080-ff5388bfb5b5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:39.328400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd946cf-6ad3-476e-a080-ff5388bfb5b5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:39.328694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd946cf-6ad3-476e-a080-ff5388bfb5b5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:39.328974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd946cf-6ad3-476e-a080-ff5388bfb5b5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:39.329994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd946cf-6ad3-476e-a080-ff5388bfb5b5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:39.333153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd946cf-6ad3-476e-a080-ff5388bfb5b5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:39.369308 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3646/7351] 10.222.0.17 () {60 vars in 1152 bytes} [Sat Oct 1 21:15:39 2022] POST /identity/v3/projects => generated 328 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:39.373417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfff94-a65f-478c-bb0d-9f18ec2e3971 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:39.375281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5dcfff94-a65f-478c-bb0d-9f18ec2e3971 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:39.393743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfff94-a65f-478c-bb0d-9f18ec2e3971 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:39.470561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176cbbd-2bf5-4954-8b4c-78c2bd766933 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:39.471241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176cbbd-2bf5-4954-8b4c-78c2bd766933 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:39.471583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176cbbd-2bf5-4954-8b4c-78c2bd766933 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:39.471917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176cbbd-2bf5-4954-8b4c-78c2bd766933 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:39.472644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176cbbd-2bf5-4954-8b4c-78c2bd766933 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:39.475437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176cbbd-2bf5-4954-8b4c-78c2bd766933 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:39.499282 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3706/7352] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:39 2022] POST /identity/v3/users => generated 413 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:39.503544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd946cf-6ad3-476e-a080-ff5388bfb5b5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:39.517386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd946cf-6ad3-476e-a080-ff5388bfb5b5 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:39.522799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0553efd-b084-4f11-811f-cab5e591ced0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:39.523460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0553efd-b084-4f11-811f-cab5e591ced0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:39.523772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0553efd-b084-4f11-811f-cab5e591ced0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:39.524052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0553efd-b084-4f11-811f-cab5e591ced0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:39.524993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0553efd-b084-4f11-811f-cab5e591ced0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:39.526326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0553efd-b084-4f11-811f-cab5e591ced0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:39.558996 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3647/7353] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:15:39 2022] POST /identity/v3/roles => generated 243 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:39.559383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176cbbd-2bf5-4954-8b4c-78c2bd766933 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:39.573629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176cbbd-2bf5-4954-8b4c-78c2bd766933 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:39.581702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26db611-ca1b-4543-8840-1712bf4d7767 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:39.583037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26db611-ca1b-4543-8840-1712bf4d7767 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:39.583333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26db611-ca1b-4543-8840-1712bf4d7767 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:39.583618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26db611-ca1b-4543-8840-1712bf4d7767 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:39.584273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26db611-ca1b-4543-8840-1712bf4d7767 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:39.589812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26db611-ca1b-4543-8840-1712bf4d7767 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:39.624085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0553efd-b084-4f11-811f-cab5e591ced0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:39.635943 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3707/7354] 10.222.0.17 () {60 vars in 1143 bytes} [Sat Oct 1 21:15:39 2022] POST /identity/v3/roles => generated 247 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:39.638632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0553efd-b084-4f11-811f-cab5e591ced0 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:39.644398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b6ce0-20a3-41c4-83a9-96c3a02e93af tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:39.646003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b6ce0-20a3-41c4-83a9-96c3a02e93af tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:39.646435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b6ce0-20a3-41c4-83a9-96c3a02e93af tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:39.646974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b6ce0-20a3-41c4-83a9-96c3a02e93af tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects/1d7cba69100d4ac3b05288f44b7fd5d1/users/a07f559eefd1477d9b854fcc4d73bb2f/roles/06c67addecf24f1aa1a67ea1bb9ddd39` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:39.649941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:39.674074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b6ce0-20a3-41c4-83a9-96c3a02e93af tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_grant(project_id=1d7cba69100d4ac3b05288f44b7fd5d1, user_id=a07f559eefd1477d9b854fcc4d73bb2f, role_id=06c67addecf24f1aa1a67ea1bb9ddd39)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:39.676872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b6ce0-20a3-41c4-83a9-96c3a02e93af tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:39.692867 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3648/7355] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:39 2022] PUT /identity/v3/projects/1d7cba69100d4ac3b05288f44b7fd5d1/users/a07f559eefd1477d9b854fcc4d73bb2f/roles/06c67addecf24f1aa1a67ea1bb9ddd39 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:39.697303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26db611-ca1b-4543-8840-1712bf4d7767 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:39.710391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26db611-ca1b-4543-8840-1712bf4d7767 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:39.780554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f19cd-3a9e-45b5-8005-d4dd585f478c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:39.781226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f19cd-3a9e-45b5-8005-d4dd585f478c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:39.781514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f19cd-3a9e-45b5-8005-d4dd585f478c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:39.781793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f19cd-3a9e-45b5-8005-d4dd585f478c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects/1d7cba69100d4ac3b05288f44b7fd5d1/users/a07f559eefd1477d9b854fcc4d73bb2f/roles/b7fdfed9c5c046fa945f5c7fe2e731bb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:39.788049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f19cd-3a9e-45b5-8005-d4dd585f478c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:create_grant(project_id=1d7cba69100d4ac3b05288f44b7fd5d1, user_id=a07f559eefd1477d9b854fcc4d73bb2f, role_id=b7fdfed9c5c046fa945f5c7fe2e731bb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:39.790846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f19cd-3a9e-45b5-8005-d4dd585f478c tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:39.814576 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3708/7356] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:39 2022] PUT /identity/v3/projects/1d7cba69100d4ac3b05288f44b7fd5d1/users/a07f559eefd1477d9b854fcc4d73bb2f/roles/b7fdfed9c5c046fa945f5c7fe2e731bb => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:39.818729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b6ce0-20a3-41c4-83a9-96c3a02e93af tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:39.841665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b6ce0-20a3-41c4-83a9-96c3a02e93af tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:39.862908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97a19a-543c-473b-b85c-c2eb644ca8bd None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:39.863449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97a19a-543c-473b-b85c-c2eb644ca8bd None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:39.863947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97a19a-543c-473b-b85c-c2eb644ca8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:39.903990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0a97a19a-543c-473b-b85c-c2eb644ca8bd None None] MFA Rules not processed for user `f07b4382a49046bfb100fbfbbb4b896a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:39.905307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0a97a19a-543c-473b-b85c-c2eb644ca8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:39.926147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b83405-1af5-4804-b3f0-fb2fac81f13f tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:39.931346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b83405-1af5-4804-b3f0-fb2fac81f13f tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:39.931346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b83405-1af5-4804-b3f0-fb2fac81f13f tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:39.931577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b83405-1af5-4804-b3f0-fb2fac81f13f tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:39.932243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b83405-1af5-4804-b3f0-fb2fac81f13f tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:39.935308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b83405-1af5-4804-b3f0-fb2fac81f13f tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:39.968351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=3,keystone:SELECT=91,keystone:INSERT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:39.969872 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3649/7357] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:39 2022] GET /identity/v3/users => generated 7783 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:39.975485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cdfb7-1cc6-4afd-84a4-e09fc1f018fc None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:39.976048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cdfb7-1cc6-4afd-84a4-e09fc1f018fc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:39.976610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cdfb7-1cc6-4afd-84a4-e09fc1f018fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:40.019671 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3709/7358] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:39 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:15:40.044017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a97a19a-543c-473b-b85c-c2eb644ca8bd None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:40.045460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0a97a19a-543c-473b-b85c-c2eb644ca8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:40.067666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7d9cdfb7-1cc6-4afd-84a4-e09fc1f018fc None None] MFA Rules not processed for user `a07f559eefd1477d9b854fcc4d73bb2f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:40.071204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7d9cdfb7-1cc6-4afd-84a4-e09fc1f018fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:40.161758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a97a19a-543c-473b-b85c-c2eb644ca8bd None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:40.168644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143e27d-0d73-4555-9ccc-af79bd6cca7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:40.170198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143e27d-0d73-4555-9ccc-af79bd6cca7b service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:40.170485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143e27d-0d73-4555-9ccc-af79bd6cca7b service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:40.170784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143e27d-0d73-4555-9ccc-af79bd6cca7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:40.182147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9143e27d-0d73-4555-9ccc-af79bd6cca7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:40.184844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9143e27d-0d73-4555-9ccc-af79bd6cca7b service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:40.201053 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3710/7359] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:40 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:15:40.221208 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3650/7360] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:39 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:40.224667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d58b9a-5e85-4217-9bf9-73d39075d0f9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:40.225097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d58b9a-5e85-4217-9bf9-73d39075d0f9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:40.225197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d58b9a-5e85-4217-9bf9-73d39075d0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:40.263306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-09d58b9a-5e85-4217-9bf9-73d39075d0f9 None None] MFA Rules not processed for user `a07f559eefd1477d9b854fcc4d73bb2f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:40.265823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-09d58b9a-5e85-4217-9bf9-73d39075d0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:40.291678 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3711/7361] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:40 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:40.292948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cdfb7-1cc6-4afd-84a4-e09fc1f018fc None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:40.306271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cdfb7-1cc6-4afd-84a4-e09fc1f018fc None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:40.322876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3cd8a8-1a49-4cc9-957a-ade3f2b67e43 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07f559eefd1477d9b854fcc4d73bb2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7cba69100d4ac3b05288f44b7fd5d1', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-326344493', 'tempest-NotDelegatedRole-1863472341'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:40.323500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3cd8a8-1a49-4cc9-957a-ade3f2b67e43 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:40.323782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3cd8a8-1a49-4cc9-957a-ade3f2b67e43 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:40.324056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3cd8a8-1a49-4cc9-957a-ade3f2b67e43 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:40.341353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3cd8a8-1a49-4cc9-957a-ade3f2b67e43 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: Authorizing `identity:create_trust()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:40.342839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3cd8a8-1a49-4cc9-957a-ade3f2b67e43 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:40.537559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:40.568019 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3651/7362] 10.222.0.17 () {60 vars in 1174 bytes} [Sat Oct 1 21:15:40 2022] POST /identity/v3/OS-TRUST/trusts => generated 834 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:40.572231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d58b9a-5e85-4217-9bf9-73d39075d0f9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:40.585812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d58b9a-5e85-4217-9bf9-73d39075d0f9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:40.591810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27375d6-d990-42d4-9f8a-c5229a129a64 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07f559eefd1477d9b854fcc4d73bb2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7cba69100d4ac3b05288f44b7fd5d1', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-326344493', 'tempest-NotDelegatedRole-1863472341'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:40.592720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27375d6-d990-42d4-9f8a-c5229a129a64 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:40.593027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27375d6-d990-42d4-9f8a-c5229a129a64 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:40.593473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27375d6-d990-42d4-9f8a-c5229a129a64 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] PATH_INFO: `/v3/OS-TRUST/trusts/314e47c742394abfadb72657a8628bb4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:40.607174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27375d6-d990-42d4-9f8a-c5229a129a64 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: Authorizing `identity:get_trust(trust_id=314e47c742394abfadb72657a8628bb4)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:40.610241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27375d6-d990-42d4-9f8a-c5229a129a64 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:40.627571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3cd8a8-1a49-4cc9-957a-ade3f2b67e43 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:40.631176 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3712/7363] 10.222.0.17 () {58 vars in 1251 bytes} [Sat Oct 1 21:15:40 2022] GET /identity/v3/OS-TRUST/trusts/314e47c742394abfadb72657a8628bb4 => generated 834 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:40.640346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3cd8a8-1a49-4cc9-957a-ade3f2b67e43 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:40.659680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7547d-95ee-4de8-9715-f82971acadce tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07f559eefd1477d9b854fcc4d73bb2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7cba69100d4ac3b05288f44b7fd5d1', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-326344493', 'tempest-NotDelegatedRole-1863472341'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:40.660385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7547d-95ee-4de8-9715-f82971acadce tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:40.660708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7547d-95ee-4de8-9715-f82971acadce tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:40.660988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7547d-95ee-4de8-9715-f82971acadce tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] PATH_INFO: `/v3/OS-TRUST/trusts/314e47c742394abfadb72657a8628bb4/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:40.676833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d7547d-95ee-4de8-9715-f82971acadce tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=314e47c742394abfadb72657a8628bb4)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:40.677823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d7547d-95ee-4de8-9715-f82971acadce tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:40.697547 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3652/7364] 10.222.0.17 () {58 vars in 1269 bytes} [Sat Oct 1 21:15:40 2022] GET /identity/v3/OS-TRUST/trusts/314e47c742394abfadb72657a8628bb4/roles => generated 370 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:15:40.698956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27375d6-d990-42d4-9f8a-c5229a129a64 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:40.724668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27375d6-d990-42d4-9f8a-c5229a129a64 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:40.739241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f20fed7-f4e0-48fa-bd91-d1cc8229d7a1 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07f559eefd1477d9b854fcc4d73bb2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7cba69100d4ac3b05288f44b7fd5d1', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-326344493', 'tempest-NotDelegatedRole-1863472341'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:40.739967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20fed7-f4e0-48fa-bd91-d1cc8229d7a1 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:40.742732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20fed7-f4e0-48fa-bd91-d1cc8229d7a1 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:40.743021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20fed7-f4e0-48fa-bd91-d1cc8229d7a1 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] PATH_INFO: `/v3/OS-TRUST/trusts/314e47c742394abfadb72657a8628bb4/roles/06c67addecf24f1aa1a67ea1bb9ddd39` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:40.780480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f20fed7-f4e0-48fa-bd91-d1cc8229d7a1 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: Authorizing `identity:get_role_for_trust(trust_id=314e47c742394abfadb72657a8628bb4, role_id=06c67addecf24f1aa1a67ea1bb9ddd39)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:40.782239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f20fed7-f4e0-48fa-bd91-d1cc8229d7a1 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:40.816956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7547d-95ee-4de8-9715-f82971acadce tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:40.818724 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3713/7365] 10.222.0.17 () {58 vars in 1368 bytes} [Sat Oct 1 21:15:40 2022] GET /identity/v3/OS-TRUST/trusts/314e47c742394abfadb72657a8628bb4/roles/06c67addecf24f1aa1a67ea1bb9ddd39 => generated 243 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:40.832060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7547d-95ee-4de8-9715-f82971acadce tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:40.854951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b10a2e-bcfe-4eb8-8381-44309977c980 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07f559eefd1477d9b854fcc4d73bb2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7cba69100d4ac3b05288f44b7fd5d1', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-326344493', 'tempest-NotDelegatedRole-1863472341'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:40.855762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b10a2e-bcfe-4eb8-8381-44309977c980 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] REQUEST_METHOD: `HEAD` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:40.856221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b10a2e-bcfe-4eb8-8381-44309977c980 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:40.856645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b10a2e-bcfe-4eb8-8381-44309977c980 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] PATH_INFO: `/v3/OS-TRUST/trusts/314e47c742394abfadb72657a8628bb4/roles/06c67addecf24f1aa1a67ea1bb9ddd39` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:40.881055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b10a2e-bcfe-4eb8-8381-44309977c980 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: Authorizing `identity:get_role_for_trust(trust_id=314e47c742394abfadb72657a8628bb4, role_id=06c67addecf24f1aa1a67ea1bb9ddd39)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:40.882671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b10a2e-bcfe-4eb8-8381-44309977c980 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:40.900981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f20fed7-f4e0-48fa-bd91-d1cc8229d7a1 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:40.901959 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3653/7366] 10.222.0.17 () {58 vars in 1369 bytes} [Sat Oct 1 21:15:40 2022] HEAD /identity/v3/OS-TRUST/trusts/314e47c742394abfadb72657a8628bb4/roles/06c67addecf24f1aa1a67ea1bb9ddd39 => generated 0 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Oct 01 21:15:40.915264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f20fed7-f4e0-48fa-bd91-d1cc8229d7a1 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:40.925951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e53b0-ac12-4445-bb6d-3bde79529737 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07f559eefd1477d9b854fcc4d73bb2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7cba69100d4ac3b05288f44b7fd5d1', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-326344493', 'tempest-NotDelegatedRole-1863472341'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:40.927953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e53b0-ac12-4445-bb6d-3bde79529737 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:40.929063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e53b0-ac12-4445-bb6d-3bde79529737 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:40.930917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e53b0-ac12-4445-bb6d-3bde79529737 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] PATH_INFO: `/v3/OS-TRUST/trusts/314e47c742394abfadb72657a8628bb4/roles/b7fdfed9c5c046fa945f5c7fe2e731bb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:40.945976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e53b0-ac12-4445-bb6d-3bde79529737 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: Authorizing `identity:get_role_for_trust(trust_id=314e47c742394abfadb72657a8628bb4, role_id=b7fdfed9c5c046fa945f5c7fe2e731bb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:40.947637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e53b0-ac12-4445-bb6d-3bde79529737 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:40.964119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-ab7e53b0-ac12-4445-bb6d-3bde79529737 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Could not find role: b7fdfed9c5c046fa945f5c7fe2e731bb.: keystone.exception.RoleNotFound: Could not find role: b7fdfed9c5c046fa945f5c7fe2e731bb. Oct 01 21:15:40.964119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:40.964119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:15:40.964119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:15:40.964119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:15:40.964119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:15:40.964119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:15:40.964119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:15:40.964119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:15:40.964119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:15:40.964119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:15:40.964119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:15:40.964119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Oct 01 21:15:40.964119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 21:15:40.964119 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: b7fdfed9c5c046fa945f5c7fe2e731bb. Oct 01 21:15:40.965299 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:15:40.966558 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3714/7367] 10.222.0.17 () {58 vars in 1368 bytes} [Sat Oct 1 21:15:40 2022] GET /identity/v3/OS-TRUST/trusts/314e47c742394abfadb72657a8628bb4/roles/b7fdfed9c5c046fa945f5c7fe2e731bb => generated 110 bytes in 66 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Oct 01 21:15:40.970946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b10a2e-bcfe-4eb8-8381-44309977c980 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:40.985181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b10a2e-bcfe-4eb8-8381-44309977c980 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:40.993917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa726eb-d287-4967-a5e5-eaf400b371be tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07f559eefd1477d9b854fcc4d73bb2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7cba69100d4ac3b05288f44b7fd5d1', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-326344493', 'tempest-NotDelegatedRole-1863472341'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:40.997738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa726eb-d287-4967-a5e5-eaf400b371be tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] REQUEST_METHOD: `HEAD` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:40.998776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa726eb-d287-4967-a5e5-eaf400b371be tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:40.999233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa726eb-d287-4967-a5e5-eaf400b371be tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] PATH_INFO: `/v3/OS-TRUST/trusts/314e47c742394abfadb72657a8628bb4/roles/b7fdfed9c5c046fa945f5c7fe2e731bb` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:41.013341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa726eb-d287-4967-a5e5-eaf400b371be tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: Authorizing `identity:get_role_for_trust(trust_id=314e47c742394abfadb72657a8628bb4, role_id=b7fdfed9c5c046fa945f5c7fe2e731bb)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:41.016964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa726eb-d287-4967-a5e5-eaf400b371be tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:41.043441 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-baa726eb-d287-4967-a5e5-eaf400b371be tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Could not find role: b7fdfed9c5c046fa945f5c7fe2e731bb.: keystone.exception.RoleNotFound: Could not find role: b7fdfed9c5c046fa945f5c7fe2e731bb. Oct 01 21:15:41.043441 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:41.043441 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:15:41.043441 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:15:41.043441 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:15:41.043441 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:15:41.043441 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:15:41.043441 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:15:41.043441 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:15:41.043441 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:15:41.043441 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:15:41.043441 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:15:41.043441 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Oct 01 21:15:41.043441 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 01 21:15:41.043441 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: b7fdfed9c5c046fa945f5c7fe2e731bb. Oct 01 21:15:41.044393 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:15:41.053845 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3654/7368] 10.222.0.17 () {58 vars in 1369 bytes} [Sat Oct 1 21:15:40 2022] HEAD /identity/v3/OS-TRUST/trusts/314e47c742394abfadb72657a8628bb4/roles/b7fdfed9c5c046fa945f5c7fe2e731bb => generated 0 bytes in 79 msecs (HTTP/1.1 404) 5 headers in 184 bytes (0 switches on core 0) Oct 01 21:15:41.054886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e53b0-ac12-4445-bb6d-3bde79529737 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:41.069259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e53b0-ac12-4445-bb6d-3bde79529737 tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:41.082415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d025b1-fa5c-412c-bfbd-cb4eb69e937d tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07f559eefd1477d9b854fcc4d73bb2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7cba69100d4ac3b05288f44b7fd5d1', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-326344493', 'tempest-NotDelegatedRole-1863472341'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:41.083992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d025b1-fa5c-412c-bfbd-cb4eb69e937d tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:41.084482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d025b1-fa5c-412c-bfbd-cb4eb69e937d tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:41.085596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d025b1-fa5c-412c-bfbd-cb4eb69e937d tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] PATH_INFO: `/v3/OS-TRUST/trusts/314e47c742394abfadb72657a8628bb4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:41.094836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d025b1-fa5c-412c-bfbd-cb4eb69e937d tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: Authorizing `identity:delete_trust(trust_id=314e47c742394abfadb72657a8628bb4)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:41.099272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d025b1-fa5c-412c-bfbd-cb4eb69e937d tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:41.137296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-25d025b1-fa5c-412c-bfbd-cb4eb69e937d tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '314e47c742394abfadb72657a8628bb4', 'request_id': 'req-25d025b1-fa5c-412c-bfbd-cb4eb69e937d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:41.138435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-25d025b1-fa5c-412c-bfbd-cb4eb69e937d tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '314e47c742394abfadb72657a8628bb4', 'request_id': 'req-25d025b1-fa5c-412c-bfbd-cb4eb69e937d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:41.139472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-25d025b1-fa5c-412c-bfbd-cb4eb69e937d tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '314e47c742394abfadb72657a8628bb4', 'request_id': 'req-25d025b1-fa5c-412c-bfbd-cb4eb69e937d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:41.160182 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3715/7369] 10.222.0.17 () {58 vars in 1254 bytes} [Sat Oct 1 21:15:41 2022] DELETE /identity/v3/OS-TRUST/trusts/314e47c742394abfadb72657a8628bb4 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:41.164473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa726eb-d287-4967-a5e5-eaf400b371be tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:41.166427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-baa726eb-d287-4967-a5e5-eaf400b371be tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:41.196015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa726eb-d287-4967-a5e5-eaf400b371be tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:41.213963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e083522-5084-4403-861e-b8340afad11d tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a07f559eefd1477d9b854fcc4d73bb2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7cba69100d4ac3b05288f44b7fd5d1', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-326344493', 'tempest-NotDelegatedRole-1863472341'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:41.214553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e083522-5084-4403-861e-b8340afad11d tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:41.214849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e083522-5084-4403-861e-b8340afad11d tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:41.215112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e083522-5084-4403-861e-b8340afad11d tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] PATH_INFO: `/v3/OS-TRUST/trusts/314e47c742394abfadb72657a8628bb4` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:41.241125 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-7e083522-5084-4403-861e-b8340afad11d tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Could not find trust: 314e47c742394abfadb72657a8628bb4.: keystone.exception.TrustNotFound: Could not find trust: 314e47c742394abfadb72657a8628bb4. Oct 01 21:15:41.241125 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:41.241125 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:15:41.241125 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:15:41.241125 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:15:41.241125 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:15:41.241125 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:15:41.241125 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:15:41.241125 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:15:41.241125 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:15:41.241125 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:15:41.241125 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:15:41.241125 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Oct 01 21:15:41.241125 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Oct 01 21:15:41.241125 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Oct 01 21:15:41.242038 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Oct 01 21:15:41.242038 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Oct 01 21:15:41.242038 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Oct 01 21:15:41.242038 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Oct 01 21:15:41.242038 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Oct 01 21:15:41.242038 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:15:41.242038 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:15:41.242038 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Oct 01 21:15:41.242038 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Oct 01 21:15:41.242038 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Oct 01 21:15:41.242038 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Oct 01 21:15:41.242038 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 314e47c742394abfadb72657a8628bb4. Oct 01 21:15:41.242038 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:15:41.242797 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3655/7370] 10.222.0.17 () {58 vars in 1251 bytes} [Sat Oct 1 21:15:41 2022] GET /identity/v3/OS-TRUST/trusts/314e47c742394abfadb72657a8628bb4 => generated 111 bytes in 78 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:15:41.250084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d025b1-fa5c-412c-bfbd-cb4eb69e937d tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:41.253579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-25d025b1-fa5c-412c-bfbd-cb4eb69e937d tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:41.291051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d025b1-fa5c-412c-bfbd-cb4eb69e937d tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:41.299896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae64d0-5376-4135-ac5a-bf47cafa39ce tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:41.300994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae64d0-5376-4135-ac5a-bf47cafa39ce tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:41.302369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae64d0-5376-4135-ac5a-bf47cafa39ce tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:41.302993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae64d0-5376-4135-ac5a-bf47cafa39ce tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles/b7fdfed9c5c046fa945f5c7fe2e731bb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:41.305474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae64d0-5376-4135-ac5a-bf47cafa39ce tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_role(role_id=b7fdfed9c5c046fa945f5c7fe2e731bb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:41.307634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae64d0-5376-4135-ac5a-bf47cafa39ce tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:41.358876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:41.404634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3bae64d0-5376-4135-ac5a-bf47cafa39ce tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invalidating the token cache because role b7fdfed9c5c046fa945f5c7fe2e731bb has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:41.404939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3bae64d0-5376-4135-ac5a-bf47cafa39ce tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:41.407809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3bae64d0-5376-4135-ac5a-bf47cafa39ce tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:41.410021 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3716/7371] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:41 2022] DELETE /identity/v3/roles/b7fdfed9c5c046fa945f5c7fe2e731bb => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:41.414521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e083522-5084-4403-861e-b8340afad11d tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:41.416557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7e083522-5084-4403-861e-b8340afad11d tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:41.440183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e083522-5084-4403-861e-b8340afad11d tempest-TrustsV3TestJSON-418843387 tempest-user-707087023] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:41.520982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd24845-e907-4876-af6d-738b8f989314 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:41.522080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd24845-e907-4876-af6d-738b8f989314 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:41.522698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd24845-e907-4876-af6d-738b8f989314 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:41.523167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd24845-e907-4876-af6d-738b8f989314 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/roles/06c67addecf24f1aa1a67ea1bb9ddd39` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:41.524957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd24845-e907-4876-af6d-738b8f989314 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_role(role_id=06c67addecf24f1aa1a67ea1bb9ddd39)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:41.527183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd24845-e907-4876-af6d-738b8f989314 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:41.606891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6bd24845-e907-4876-af6d-738b8f989314 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invalidating the token cache because role 06c67addecf24f1aa1a67ea1bb9ddd39 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:41.606891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6bd24845-e907-4876-af6d-738b8f989314 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:41.608165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6bd24845-e907-4876-af6d-738b8f989314 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:41.610656 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3656/7372] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:41 2022] DELETE /identity/v3/roles/06c67addecf24f1aa1a67ea1bb9ddd39 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:41.615024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae64d0-5376-4135-ac5a-bf47cafa39ce tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:41.627832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae64d0-5376-4135-ac5a-bf47cafa39ce tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:41.743933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaba016-1a10-4ade-a9f0-e7df84b0a0df tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:41.744815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaba016-1a10-4ade-a9f0-e7df84b0a0df tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:41.745251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaba016-1a10-4ade-a9f0-e7df84b0a0df tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:41.745679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaba016-1a10-4ade-a9f0-e7df84b0a0df tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/users/a07f559eefd1477d9b854fcc4d73bb2f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:41.749486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaba016-1a10-4ade-a9f0-e7df84b0a0df tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_user(user_id=a07f559eefd1477d9b854fcc4d73bb2f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:41.752046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaba016-1a10-4ade-a9f0-e7df84b0a0df tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:41.847008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-beaba016-1a10-4ade-a9f0-e7df84b0a0df tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a07f559eefd1477d9b854fcc4d73bb2f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:41.864904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-beaba016-1a10-4ade-a9f0-e7df84b0a0df tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a07f559eefd1477d9b854fcc4d73bb2f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:41.892295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-beaba016-1a10-4ade-a9f0-e7df84b0a0df tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a07f559eefd1477d9b854fcc4d73bb2f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:41.893102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-beaba016-1a10-4ade-a9f0-e7df84b0a0df tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a07f559eefd1477d9b854fcc4d73bb2f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:41.918505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-beaba016-1a10-4ade-a9f0-e7df84b0a0df tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a07f559eefd1477d9b854fcc4d73bb2f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:41.935138 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3717/7373] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:41 2022] DELETE /identity/v3/users/a07f559eefd1477d9b854fcc4d73bb2f => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:41.940752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd24845-e907-4876-af6d-738b8f989314 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:41.947964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6bd24845-e907-4876-af6d-738b8f989314 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:41.972965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd24845-e907-4876-af6d-738b8f989314 tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:42.053425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:INSERT=2,keystone:SELECT=95,keystone:DELETE=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:42.071053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829f6ec-9171-47b9-a502-5b0769da27cb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1747c0e3418242238fdce39ab1d923fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab8b62281af4862a9cb86af23aa8371', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:42.072886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829f6ec-9171-47b9-a502-5b0769da27cb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:42.073489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829f6ec-9171-47b9-a502-5b0769da27cb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:42.074232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829f6ec-9171-47b9-a502-5b0769da27cb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] PATH_INFO: `/v3/projects/1d7cba69100d4ac3b05288f44b7fd5d1` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:42.079726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829f6ec-9171-47b9-a502-5b0769da27cb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1d7cba69100d4ac3b05288f44b7fd5d1)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:42.083571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829f6ec-9171-47b9-a502-5b0769da27cb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:42.166312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4829f6ec-9171-47b9-a502-5b0769da27cb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d7cba69100d4ac3b05288f44b7fd5d1', 'request_id': 'req-4829f6ec-9171-47b9-a502-5b0769da27cb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:42.178455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4829f6ec-9171-47b9-a502-5b0769da27cb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d7cba69100d4ac3b05288f44b7fd5d1', 'request_id': 'req-4829f6ec-9171-47b9-a502-5b0769da27cb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:42.187313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4829f6ec-9171-47b9-a502-5b0769da27cb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d7cba69100d4ac3b05288f44b7fd5d1', 'request_id': 'req-4829f6ec-9171-47b9-a502-5b0769da27cb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:42.226004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4829f6ec-9171-47b9-a502-5b0769da27cb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] The token cache is being invalidate because project 1d7cba69100d4ac3b05288f44b7fd5d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:42.226004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4829f6ec-9171-47b9-a502-5b0769da27cb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:42.226004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4829f6ec-9171-47b9-a502-5b0769da27cb tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:42.235893 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3657/7374] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:41 2022] DELETE /identity/v3/projects/1d7cba69100d4ac3b05288f44b7fd5d1 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:42.240885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaba016-1a10-4ade-a9f0-e7df84b0a0df tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:42.243417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-beaba016-1a10-4ade-a9f0-e7df84b0a0df tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:42.409532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaba016-1a10-4ade-a9f0-e7df84b0a0df tempest-TrustsV3TestJSON-1803548874 tempest-TrustsV3TestJSON-1803548874-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:42.414924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:42.491433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8f2e5-5676-4cee-ad72-c8ee48f07959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:42.492365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8f2e5-5676-4cee-ad72-c8ee48f07959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:42.492653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8f2e5-5676-4cee-ad72-c8ee48f07959 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:42.492930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8f2e5-5676-4cee-ad72-c8ee48f07959 admin admin] PATH_INFO: `/v3/users/dcf9e137e3de486eba2cafcad5651464` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:42.494159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8f2e5-5676-4cee-ad72-c8ee48f07959 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcf9e137e3de486eba2cafcad5651464)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:42.497165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8f2e5-5676-4cee-ad72-c8ee48f07959 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:42.596066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9ee8f2e5-5676-4cee-ad72-c8ee48f07959 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcf9e137e3de486eba2cafcad5651464'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:42.611419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9ee8f2e5-5676-4cee-ad72-c8ee48f07959 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcf9e137e3de486eba2cafcad5651464'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:42.634892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9ee8f2e5-5676-4cee-ad72-c8ee48f07959 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcf9e137e3de486eba2cafcad5651464'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:42.635935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9ee8f2e5-5676-4cee-ad72-c8ee48f07959 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcf9e137e3de486eba2cafcad5651464'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:42.647027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9ee8f2e5-5676-4cee-ad72-c8ee48f07959 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcf9e137e3de486eba2cafcad5651464'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:42.655675 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3718/7375] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:42 2022] DELETE /identity/v3/users/dcf9e137e3de486eba2cafcad5651464 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:42.662989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654e979-3502-44c2-8313-265093bff7fb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:42.663460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654e979-3502-44c2-8313-265093bff7fb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:42.663845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654e979-3502-44c2-8313-265093bff7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:42.695021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8654e979-3502-44c2-8313-265093bff7fb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:42.696318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8654e979-3502-44c2-8313-265093bff7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:42.792694 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3658/7376] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:42.836476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8f2e5-5676-4cee-ad72-c8ee48f07959 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:42.836476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9ee8f2e5-5676-4cee-ad72-c8ee48f07959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:42.963976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8f2e5-5676-4cee-ad72-c8ee48f07959 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:42.972821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b9ea4-e086-4c04-9946-0bf63d1592fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:42.975206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b9ea4-e086-4c04-9946-0bf63d1592fc service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:42.975430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b9ea4-e086-4c04-9946-0bf63d1592fc service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:42.975707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b9ea4-e086-4c04-9946-0bf63d1592fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:42.999081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b9ea4-e086-4c04-9946-0bf63d1592fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:43.001501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b9ea4-e086-4c04-9946-0bf63d1592fc service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:43.036548 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3719/7377] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:15:44.300421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654e979-3502-44c2-8313-265093bff7fb None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:44.302130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8654e979-3502-44c2-8313-265093bff7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:44.318374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654e979-3502-44c2-8313-265093bff7fb None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:44.323810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051315e0-9b7f-47a6-839b-a255d26385b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:44.324571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051315e0-9b7f-47a6-839b-a255d26385b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:44.325286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051315e0-9b7f-47a6-839b-a255d26385b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:44.325286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051315e0-9b7f-47a6-839b-a255d26385b5 admin admin] PATH_INFO: `/v3/projects/b758708abef04a00a2922680d116198a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:44.326474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051315e0-9b7f-47a6-839b-a255d26385b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b758708abef04a00a2922680d116198a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:44.328458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051315e0-9b7f-47a6-839b-a255d26385b5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:44.416665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-051315e0-9b7f-47a6-839b-a255d26385b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b758708abef04a00a2922680d116198a', 'request_id': 'req-051315e0-9b7f-47a6-839b-a255d26385b5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:44.425835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-051315e0-9b7f-47a6-839b-a255d26385b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b758708abef04a00a2922680d116198a', 'request_id': 'req-051315e0-9b7f-47a6-839b-a255d26385b5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:44.436347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-051315e0-9b7f-47a6-839b-a255d26385b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b758708abef04a00a2922680d116198a', 'request_id': 'req-051315e0-9b7f-47a6-839b-a255d26385b5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:44.453772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-051315e0-9b7f-47a6-839b-a255d26385b5 admin admin] The token cache is being invalidate because project b758708abef04a00a2922680d116198a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:44.454413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-051315e0-9b7f-47a6-839b-a255d26385b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:44.455362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-051315e0-9b7f-47a6-839b-a255d26385b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:44.462171 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3659/7378] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:44 2022] DELETE /identity/v3/projects/b758708abef04a00a2922680d116198a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:44.463058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b9ea4-e086-4c04-9946-0bf63d1592fc service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:44.474515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b9ea4-e086-4c04-9946-0bf63d1592fc service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:44.507322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:44.560759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a13a12-9d09-422b-94a1-4d258a320f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:44.560934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a13a12-9d09-422b-94a1-4d258a320f19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:44.561817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a13a12-9d09-422b-94a1-4d258a320f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:44.562103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a13a12-9d09-422b-94a1-4d258a320f19 admin admin] PATH_INFO: `/v3/users/1747c0e3418242238fdce39ab1d923fb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:44.563797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a13a12-9d09-422b-94a1-4d258a320f19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1747c0e3418242238fdce39ab1d923fb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:44.566303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a13a12-9d09-422b-94a1-4d258a320f19 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:44.647326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d6a13a12-9d09-422b-94a1-4d258a320f19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1747c0e3418242238fdce39ab1d923fb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:44.668850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d6a13a12-9d09-422b-94a1-4d258a320f19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1747c0e3418242238fdce39ab1d923fb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:44.686079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d6a13a12-9d09-422b-94a1-4d258a320f19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1747c0e3418242238fdce39ab1d923fb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:44.687051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d6a13a12-9d09-422b-94a1-4d258a320f19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1747c0e3418242238fdce39ab1d923fb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:44.695197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d6a13a12-9d09-422b-94a1-4d258a320f19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1747c0e3418242238fdce39ab1d923fb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:44.701755 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3720/7379] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:44 2022] DELETE /identity/v3/users/1747c0e3418242238fdce39ab1d923fb => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:45.013612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051315e0-9b7f-47a6-839b-a255d26385b5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:45.015314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-051315e0-9b7f-47a6-839b-a255d26385b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:45.032347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051315e0-9b7f-47a6-839b-a255d26385b5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:45.037918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:UPDATE=1,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:45.129313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7720255-b80e-4eaa-b3b9-48701da4a606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:45.130149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7720255-b80e-4eaa-b3b9-48701da4a606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:45.130587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7720255-b80e-4eaa-b3b9-48701da4a606 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:45.130966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7720255-b80e-4eaa-b3b9-48701da4a606 admin admin] PATH_INFO: `/v3/projects/3ab8b62281af4862a9cb86af23aa8371` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:45.132255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7720255-b80e-4eaa-b3b9-48701da4a606 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ab8b62281af4862a9cb86af23aa8371)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:45.135435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7720255-b80e-4eaa-b3b9-48701da4a606 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:45.219120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c7720255-b80e-4eaa-b3b9-48701da4a606 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ab8b62281af4862a9cb86af23aa8371', 'request_id': 'req-c7720255-b80e-4eaa-b3b9-48701da4a606'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:45.230667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c7720255-b80e-4eaa-b3b9-48701da4a606 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ab8b62281af4862a9cb86af23aa8371', 'request_id': 'req-c7720255-b80e-4eaa-b3b9-48701da4a606'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:45.239004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c7720255-b80e-4eaa-b3b9-48701da4a606 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ab8b62281af4862a9cb86af23aa8371', 'request_id': 'req-c7720255-b80e-4eaa-b3b9-48701da4a606'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:45.257148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c7720255-b80e-4eaa-b3b9-48701da4a606 admin admin] The token cache is being invalidate because project 3ab8b62281af4862a9cb86af23aa8371 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:45.257511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c7720255-b80e-4eaa-b3b9-48701da4a606 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:45.258324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c7720255-b80e-4eaa-b3b9-48701da4a606 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:45.260859 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3660/7380] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:45 2022] DELETE /identity/v3/projects/3ab8b62281af4862a9cb86af23aa8371 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:45.270626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83a103-5de1-4fd4-92ea-4075b4cac226 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:45.275047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83a103-5de1-4fd4-92ea-4075b4cac226 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:45.276295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83a103-5de1-4fd4-92ea-4075b4cac226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:45.319834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6d83a103-5de1-4fd4-92ea-4075b4cac226 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:45.321382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6d83a103-5de1-4fd4-92ea-4075b4cac226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:45.399792 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3721/7381] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:45.409335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92d78f-c8ef-4de6-bbc0-8b27d91e1667 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:45.409845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92d78f-c8ef-4de6-bbc0-8b27d91e1667 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:45.409928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92d78f-c8ef-4de6-bbc0-8b27d91e1667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:45.434780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5a92d78f-c8ef-4de6-bbc0-8b27d91e1667 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:45.436033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5a92d78f-c8ef-4de6-bbc0-8b27d91e1667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:45.452047 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3661/7382] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:45.456019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83a103-5de1-4fd4-92ea-4075b4cac226 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:45.466863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83a103-5de1-4fd4-92ea-4075b4cac226 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:45.474771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df234e-0c71-4d50-b140-5546920924ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:45.475802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df234e-0c71-4d50-b140-5546920924ae admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:45.476091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df234e-0c71-4d50-b140-5546920924ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:45.476368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df234e-0c71-4d50-b140-5546920924ae admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:45.477031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df234e-0c71-4d50-b140-5546920924ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:15:45.478344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df234e-0c71-4d50-b140-5546920924ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:45.483432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df234e-0c71-4d50-b140-5546920924ae admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:45.500046 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3722/7383] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:45.505434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92d78f-c8ef-4de6-bbc0-8b27d91e1667 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:45.519289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92d78f-c8ef-4de6-bbc0-8b27d91e1667 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:45.525817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ee936-b342-4956-b4c8-4b921963310c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:45.526481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ee936-b342-4956-b4c8-4b921963310c admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:45.526800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ee936-b342-4956-b4c8-4b921963310c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:45.527069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ee936-b342-4956-b4c8-4b921963310c admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:45.527700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ee936-b342-4956-b4c8-4b921963310c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:45.529605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ee936-b342-4956-b4c8-4b921963310c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:45.578211 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3662/7384] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:45 2022] POST /identity/v3/projects => generated 369 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:45.584360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df234e-0c71-4d50-b140-5546920924ae admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:45.608532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df234e-0c71-4d50-b140-5546920924ae admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:45.670817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:45.742028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a5dec-8134-4bc3-842c-0dd357ae1524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:45.742740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a5dec-8134-4bc3-842c-0dd357ae1524 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:45.743014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a5dec-8134-4bc3-842c-0dd357ae1524 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:45.743286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a5dec-8134-4bc3-842c-0dd357ae1524 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:45.743925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a5dec-8134-4bc3-842c-0dd357ae1524 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:45.745864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a5dec-8134-4bc3-842c-0dd357ae1524 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:45.778317 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3723/7385] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:45 2022] POST /identity/v3/users => generated 285 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:45.779967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ee936-b342-4956-b4c8-4b921963310c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:45.808883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ee936-b342-4956-b4c8-4b921963310c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:45.821156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0acc0b-307a-4a38-9ad3-72f5df34e599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:45.822041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0acc0b-307a-4a38-9ad3-72f5df34e599 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:45.822750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0acc0b-307a-4a38-9ad3-72f5df34e599 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:45.823203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0acc0b-307a-4a38-9ad3-72f5df34e599 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:45.825090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0acc0b-307a-4a38-9ad3-72f5df34e599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:45.826826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0acc0b-307a-4a38-9ad3-72f5df34e599 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:45.844558 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3663/7386] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:45.845887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a5dec-8134-4bc3-842c-0dd357ae1524 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:45.858429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a5dec-8134-4bc3-842c-0dd357ae1524 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:45.864208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176630a-32b0-485f-bebb-5df2bf2a2ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:45.865101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176630a-32b0-485f-bebb-5df2bf2a2ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:45.865823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176630a-32b0-485f-bebb-5df2bf2a2ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:45.866587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176630a-32b0-485f-bebb-5df2bf2a2ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:45.867498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176630a-32b0-485f-bebb-5df2bf2a2ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:45.869289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176630a-32b0-485f-bebb-5df2bf2a2ca3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:45.882567 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3724/7387] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:45.887346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0acc0b-307a-4a38-9ad3-72f5df34e599 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:45.898505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0acc0b-307a-4a38-9ad3-72f5df34e599 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:45.904432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343196a-65d7-49d6-9e86-0e83ccd598bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:45.905120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343196a-65d7-49d6-9e86-0e83ccd598bf admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:45.906290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343196a-65d7-49d6-9e86-0e83ccd598bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:45.906727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343196a-65d7-49d6-9e86-0e83ccd598bf admin admin] PATH_INFO: `/v3/projects/8a8f6d58ed794cf7b6d1579407df6260/users/59259b8a3cf1490cb0d7ee084c5056e0/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:45.927901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343196a-65d7-49d6-9e86-0e83ccd598bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a8f6d58ed794cf7b6d1579407df6260, user_id=59259b8a3cf1490cb0d7ee084c5056e0, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:45.930967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343196a-65d7-49d6-9e86-0e83ccd598bf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:45.953271 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3664/7388] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:45 2022] PUT /identity/v3/projects/8a8f6d58ed794cf7b6d1579407df6260/users/59259b8a3cf1490cb0d7ee084c5056e0/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:46.058843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a8792-29f4-4e01-b94b-d1b76b251b4e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:46.059746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a8792-29f4-4e01-b94b-d1b76b251b4e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:46.060397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a8792-29f4-4e01-b94b-d1b76b251b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:46.120085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-938a8792-29f4-4e01-b94b-d1b76b251b4e None None] MFA Rules not processed for user `59259b8a3cf1490cb0d7ee084c5056e0`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:46.122163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-938a8792-29f4-4e01-b94b-d1b76b251b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:46.273524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343196a-65d7-49d6-9e86-0e83ccd598bf admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:46.273672 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3725/7389] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:46.288598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343196a-65d7-49d6-9e86-0e83ccd598bf admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:46.296382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc698e0-7bc6-4b08-8fa4-1424853a290d tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59259b8a3cf1490cb0d7ee084c5056e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a8f6d58ed794cf7b6d1579407df6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:46.297193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc698e0-7bc6-4b08-8fa4-1424853a290d tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:46.297950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc698e0-7bc6-4b08-8fa4-1424853a290d tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:46.299001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc698e0-7bc6-4b08-8fa4-1424853a290d tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] PATH_INFO: `/v3/users/59259b8a3cf1490cb0d7ee084c5056e0/application_credentials` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:46.300154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc698e0-7bc6-4b08-8fa4-1424853a290d tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: Authorizing `identity:create_application_credential(user_id=59259b8a3cf1490cb0d7ee084c5056e0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:46.302164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc698e0-7bc6-4b08-8fa4-1424853a290d tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:46.366023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:46.479910 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3665/7390] 10.222.0.17 () {60 vars in 1315 bytes} [Sat Oct 1 21:15:46 2022] POST /identity/v3/users/59259b8a3cf1490cb0d7ee084c5056e0/application_credentials => generated 920 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:46.489268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a8792-29f4-4e01-b94b-d1b76b251b4e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:46.508699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a8792-29f4-4e01-b94b-d1b76b251b4e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:46.520403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d98fd-7d1f-45d4-87a8-8b6ccfd98c91 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59259b8a3cf1490cb0d7ee084c5056e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a8f6d58ed794cf7b6d1579407df6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:46.521836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d98fd-7d1f-45d4-87a8-8b6ccfd98c91 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:46.522236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d98fd-7d1f-45d4-87a8-8b6ccfd98c91 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:46.522720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d98fd-7d1f-45d4-87a8-8b6ccfd98c91 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] PATH_INFO: `/v3/users/59259b8a3cf1490cb0d7ee084c5056e0/access_rules/03d8c4888a1643e3a395fd47121a3bc1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:46.524401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d98fd-7d1f-45d4-87a8-8b6ccfd98c91 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: Authorizing `identity:delete_access_rule(user_id=59259b8a3cf1490cb0d7ee084c5056e0, access_rule_id=03d8c4888a1643e3a395fd47121a3bc1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:46.525971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d98fd-7d1f-45d4-87a8-8b6ccfd98c91 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:46.634021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-309d98fd-7d1f-45d4-87a8-8b6ccfd98c91 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] May not delete access rule in use: keystone.exception.ForbiddenNotSecurity: May not delete access rule in use Oct 01 21:15:46.634021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:46.634021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/backends/sql.py", line 256, in delete_access_rule Oct 01 21:15:46.634021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application session.delete(ref) Oct 01 21:15:46.634021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 01 21:15:46.634021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application next(self.gen) Oct 01 21:15:46.634021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 01 21:15:46.634021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application yield resource Oct 01 21:15:46.634021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 01 21:15:46.634021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application next(self.gen) Oct 01 21:15:46.634021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 01 21:15:46.634021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.session.rollback() Oct 01 21:15:46.634021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 01 21:15:46.634021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.force_reraise() Oct 01 21:15:46.634021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 01 21:15:46.634021 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise self.value Oct 01 21:15:46.635161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 01 21:15:46.635161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 01 21:15:46.635161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 01 21:15:46.635161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application session.commit() Oct 01 21:15:46.635161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 01 21:15:46.635161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 01 21:15:46.635161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 01 21:15:46.635161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._prepare_impl() Oct 01 21:15:46.635161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 01 21:15:46.635161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.session.flush() Oct 01 21:15:46.635161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 01 21:15:46.635161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._flush(objects) Oct 01 21:15:46.635161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 01 21:15:46.635161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 01 21:15:46.635161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 01 21:15:46.635161 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application compat.raise_( Oct 01 21:15:46.640674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 01 21:15:46.640674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception Oct 01 21:15:46.640674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 01 21:15:46.640674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application flush_context.execute() Oct 01 21:15:46.640674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 01 21:15:46.640674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rec.execute(self) Oct 01 21:15:46.640674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 577, in execute Oct 01 21:15:46.640674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.dependency_processor.process_deletes(uow, states) Oct 01 21:15:46.640674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/dependency.py", line 552, in process_deletes Oct 01 21:15:46.640674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._synchronize( Oct 01 21:15:46.640674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/dependency.py", line 610, in _synchronize Oct 01 21:15:46.640674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application sync.clear(dest, self.mapper, self.prop.synchronize_pairs) Oct 01 21:15:46.640674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/sync.py", line 86, in clear Oct 01 21:15:46.640674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise AssertionError( Oct 01 21:15:46.640674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application AssertionError: Dependency rule tried to blank-out primary key column 'application_credential_access_rule.access_rule_id' on instance '' Oct 01 21:15:46.640674 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:15:46.642726 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:15:46.642726 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:15:46.642726 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:15:46.642726 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:15:46.642726 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:15:46.642726 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:15:46.642726 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:15:46.642726 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:15:46.642726 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:15:46.642726 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:15:46.642726 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:15:46.642726 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:15:46.642726 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:15:46.642726 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 763, in delete Oct 01 21:15:46.642726 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application PROVIDERS.application_credential_api.delete_access_rule( Oct 01 21:15:46.642726 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:15:46.642726 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:15:46.645084 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/core.py", line 252, in delete_access_rule Oct 01 21:15:46.645084 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self.driver.delete_access_rule(access_rule_id) Oct 01 21:15:46.645084 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/backends/sql.py", line 258, in delete_access_rule Oct 01 21:15:46.645084 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ForbiddenNotSecurity( Oct 01 21:15:46.645084 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ForbiddenNotSecurity: May not delete access rule in use Oct 01 21:15:46.645084 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:15:46.645084 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3726/7391] 10.222.0.17 () {58 vars in 1362 bytes} [Sat Oct 1 21:15:46 2022] DELETE /identity/v3/users/59259b8a3cf1490cb0d7ee084c5056e0/access_rules/03d8c4888a1643e3a395fd47121a3bc1 => generated 89 bytes in 148 msecs (HTTP/1.1 403) 5 headers in 183 bytes (1 switches on core 0) Oct 01 21:15:46.646032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc698e0-7bc6-4b08-8fa4-1424853a290d tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:46.651999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc698e0-7bc6-4b08-8fa4-1424853a290d tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:46.668418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6eebb-71ed-4ce2-9605-02e4eaf2a3d0 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59259b8a3cf1490cb0d7ee084c5056e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a8f6d58ed794cf7b6d1579407df6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:46.669303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6eebb-71ed-4ce2-9605-02e4eaf2a3d0 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:46.669686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6eebb-71ed-4ce2-9605-02e4eaf2a3d0 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:46.670059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6eebb-71ed-4ce2-9605-02e4eaf2a3d0 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] PATH_INFO: `/v3/users/59259b8a3cf1490cb0d7ee084c5056e0/application_credentials/2ecf3772498541ada4f767c6d2c484e1` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:46.702247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6eebb-71ed-4ce2-9605-02e4eaf2a3d0 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: Authorizing `identity:delete_application_credential(user_id=59259b8a3cf1490cb0d7ee084c5056e0, application_credential_id=2ecf3772498541ada4f767c6d2c484e1)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:46.704026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6eebb-71ed-4ce2-9605-02e4eaf2a3d0 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:46.770229 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3666/7392] 10.222.0.17 () {58 vars in 1395 bytes} [Sat Oct 1 21:15:46 2022] DELETE /identity/v3/users/59259b8a3cf1490cb0d7ee084c5056e0/application_credentials/2ecf3772498541ada4f767c6d2c484e1 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:46.774048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d98fd-7d1f-45d4-87a8-8b6ccfd98c91 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:46.800336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d98fd-7d1f-45d4-87a8-8b6ccfd98c91 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:46.816734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947115b-640b-4900-a170-f6b3f22d234e tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59259b8a3cf1490cb0d7ee084c5056e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a8f6d58ed794cf7b6d1579407df6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:46.817550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947115b-640b-4900-a170-f6b3f22d234e tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:46.817929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947115b-640b-4900-a170-f6b3f22d234e tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:46.818301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947115b-640b-4900-a170-f6b3f22d234e tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] PATH_INFO: `/v3/users/59259b8a3cf1490cb0d7ee084c5056e0/access_rules` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:46.819885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947115b-640b-4900-a170-f6b3f22d234e tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: Authorizing `identity:list_access_rules(user_id=59259b8a3cf1490cb0d7ee084c5056e0)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:46.821628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947115b-640b-4900-a170-f6b3f22d234e tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:46.829745 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3727/7393] 10.222.0.17 () {58 vars in 1260 bytes} [Sat Oct 1 21:15:46 2022] GET /identity/v3/users/59259b8a3cf1490cb0d7ee084c5056e0/access_rules => generated 369 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:46.832702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6eebb-71ed-4ce2-9605-02e4eaf2a3d0 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:46.854242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6eebb-71ed-4ce2-9605-02e4eaf2a3d0 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:46.897905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec73441-9c2a-4dfb-aa85-d0c826bb80a8 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59259b8a3cf1490cb0d7ee084c5056e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a8f6d58ed794cf7b6d1579407df6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:46.899517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec73441-9c2a-4dfb-aa85-d0c826bb80a8 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:46.899847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec73441-9c2a-4dfb-aa85-d0c826bb80a8 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:46.900128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec73441-9c2a-4dfb-aa85-d0c826bb80a8 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] PATH_INFO: `/v3/users/59259b8a3cf1490cb0d7ee084c5056e0/access_rules/03d8c4888a1643e3a395fd47121a3bc1` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:46.906373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec73441-9c2a-4dfb-aa85-d0c826bb80a8 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: Authorizing `identity:delete_access_rule(user_id=59259b8a3cf1490cb0d7ee084c5056e0, access_rule_id=03d8c4888a1643e3a395fd47121a3bc1)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:46.910152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec73441-9c2a-4dfb-aa85-d0c826bb80a8 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:46.933212 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3667/7394] 10.222.0.17 () {58 vars in 1362 bytes} [Sat Oct 1 21:15:46 2022] DELETE /identity/v3/users/59259b8a3cf1490cb0d7ee084c5056e0/access_rules/03d8c4888a1643e3a395fd47121a3bc1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:46.944580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947115b-640b-4900-a170-f6b3f22d234e tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:46.962230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947115b-640b-4900-a170-f6b3f22d234e tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:46.966958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36947df-6e2d-4d07-8928-9fb0dfdf394a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59259b8a3cf1490cb0d7ee084c5056e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a8f6d58ed794cf7b6d1579407df6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:46.967616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36947df-6e2d-4d07-8928-9fb0dfdf394a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:46.967906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36947df-6e2d-4d07-8928-9fb0dfdf394a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:46.968186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36947df-6e2d-4d07-8928-9fb0dfdf394a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] PATH_INFO: `/v3/users/59259b8a3cf1490cb0d7ee084c5056e0/access_rules` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:46.969684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36947df-6e2d-4d07-8928-9fb0dfdf394a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: Authorizing `identity:list_access_rules(user_id=59259b8a3cf1490cb0d7ee084c5056e0)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:46.971231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36947df-6e2d-4d07-8928-9fb0dfdf394a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:46.983182 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3728/7395] 10.222.0.17 () {58 vars in 1260 bytes} [Sat Oct 1 21:15:46 2022] GET /identity/v3/users/59259b8a3cf1490cb0d7ee084c5056e0/access_rules => generated 158 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:46.984084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec73441-9c2a-4dfb-aa85-d0c826bb80a8 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:47.009535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec73441-9c2a-4dfb-aa85-d0c826bb80a8 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:47.028153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607d0e3-d722-4e27-bb40-d2b81bfdd8d8 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59259b8a3cf1490cb0d7ee084c5056e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a8f6d58ed794cf7b6d1579407df6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:47.031390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607d0e3-d722-4e27-bb40-d2b81bfdd8d8 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:47.032356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607d0e3-d722-4e27-bb40-d2b81bfdd8d8 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:47.032799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607d0e3-d722-4e27-bb40-d2b81bfdd8d8 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] PATH_INFO: `/v3/users/59259b8a3cf1490cb0d7ee084c5056e0/application_credentials` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:47.036671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b607d0e3-d722-4e27-bb40-d2b81bfdd8d8 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: Authorizing `identity:create_application_credential(user_id=59259b8a3cf1490cb0d7ee084c5056e0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:47.041142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b607d0e3-d722-4e27-bb40-d2b81bfdd8d8 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:47.277149 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3668/7396] 10.222.0.17 () {60 vars in 1315 bytes} [Sat Oct 1 21:15:46 2022] POST /identity/v3/users/59259b8a3cf1490cb0d7ee084c5056e0/application_credentials => generated 920 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:47.281577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36947df-6e2d-4d07-8928-9fb0dfdf394a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:47.307346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36947df-6e2d-4d07-8928-9fb0dfdf394a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:47.313170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9e340-2377-4118-a0e9-38146fbda33a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59259b8a3cf1490cb0d7ee084c5056e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a8f6d58ed794cf7b6d1579407df6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:47.314233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9e340-2377-4118-a0e9-38146fbda33a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:47.315042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9e340-2377-4118-a0e9-38146fbda33a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:47.316082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9e340-2377-4118-a0e9-38146fbda33a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] PATH_INFO: `/v3/users/59259b8a3cf1490cb0d7ee084c5056e0/access_rules` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:47.318598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9e340-2377-4118-a0e9-38146fbda33a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: Authorizing `identity:list_access_rules(user_id=59259b8a3cf1490cb0d7ee084c5056e0)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:47.320680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9e340-2377-4118-a0e9-38146fbda33a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:47.332856 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3729/7397] 10.222.0.17 () {58 vars in 1260 bytes} [Sat Oct 1 21:15:47 2022] GET /identity/v3/users/59259b8a3cf1490cb0d7ee084c5056e0/access_rules => generated 369 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:47.334920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607d0e3-d722-4e27-bb40-d2b81bfdd8d8 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:47.344484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607d0e3-d722-4e27-bb40-d2b81bfdd8d8 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:47.358725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797a34d-7136-4389-ba75-86c2a308850a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59259b8a3cf1490cb0d7ee084c5056e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a8f6d58ed794cf7b6d1579407df6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:47.359716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797a34d-7136-4389-ba75-86c2a308850a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:47.360336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797a34d-7136-4389-ba75-86c2a308850a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:47.360532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797a34d-7136-4389-ba75-86c2a308850a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] PATH_INFO: `/v3/users/59259b8a3cf1490cb0d7ee084c5056e0/application_credentials` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:47.361152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1797a34d-7136-4389-ba75-86c2a308850a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: Authorizing `identity:create_application_credential(user_id=59259b8a3cf1490cb0d7ee084c5056e0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:47.362616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1797a34d-7136-4389-ba75-86c2a308850a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:47.460042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:INSERT=8,keystone:DELETE=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:47.524765 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3669/7398] 10.222.0.17 () {60 vars in 1315 bytes} [Sat Oct 1 21:15:47 2022] POST /identity/v3/users/59259b8a3cf1490cb0d7ee084c5056e0/application_credentials => generated 921 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:47.528996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9e340-2377-4118-a0e9-38146fbda33a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:47.541997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9e340-2377-4118-a0e9-38146fbda33a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:47.551919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f019f1d-b76b-4d05-a6d2-ae968196e906 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59259b8a3cf1490cb0d7ee084c5056e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a8f6d58ed794cf7b6d1579407df6260', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:47.552844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f019f1d-b76b-4d05-a6d2-ae968196e906 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:47.553280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f019f1d-b76b-4d05-a6d2-ae968196e906 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:47.553737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f019f1d-b76b-4d05-a6d2-ae968196e906 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] PATH_INFO: `/v3/users/59259b8a3cf1490cb0d7ee084c5056e0/access_rules/9c9f95c39572412790d9d055815441f8` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:47.556259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f019f1d-b76b-4d05-a6d2-ae968196e906 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: Authorizing `identity:get_access_rule(user_id=59259b8a3cf1490cb0d7ee084c5056e0, access_rule_id=9c9f95c39572412790d9d055815441f8)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:47.568969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f019f1d-b76b-4d05-a6d2-ae968196e906 tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:47.580203 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3730/7399] 10.222.0.17 () {58 vars in 1359 bytes} [Sat Oct 1 21:15:47 2022] GET /identity/v3/users/59259b8a3cf1490cb0d7ee084c5056e0/access_rules/9c9f95c39572412790d9d055815441f8 => generated 229 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:47.587021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797a34d-7136-4389-ba75-86c2a308850a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:47.742873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797a34d-7136-4389-ba75-86c2a308850a tempest-AccessRulesV3Test-724850984 tempest-AccessRulesV3Test-724850984-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:47.747790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4343ff3-4f6d-405f-b103-97cbcda24932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:47.748478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4343ff3-4f6d-405f-b103-97cbcda24932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:47.748761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4343ff3-4f6d-405f-b103-97cbcda24932 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:47.749919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4343ff3-4f6d-405f-b103-97cbcda24932 admin admin] PATH_INFO: `/v3/users/59259b8a3cf1490cb0d7ee084c5056e0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:47.750228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4343ff3-4f6d-405f-b103-97cbcda24932 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59259b8a3cf1490cb0d7ee084c5056e0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:47.752251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4343ff3-4f6d-405f-b103-97cbcda24932 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:47.835677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c4343ff3-4f6d-405f-b103-97cbcda24932 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59259b8a3cf1490cb0d7ee084c5056e0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:47.836437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c4343ff3-4f6d-405f-b103-97cbcda24932 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59259b8a3cf1490cb0d7ee084c5056e0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:47.837064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c4343ff3-4f6d-405f-b103-97cbcda24932 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59259b8a3cf1490cb0d7ee084c5056e0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:47.848818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c4343ff3-4f6d-405f-b103-97cbcda24932 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59259b8a3cf1490cb0d7ee084c5056e0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:47.932540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c4343ff3-4f6d-405f-b103-97cbcda24932 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59259b8a3cf1490cb0d7ee084c5056e0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:47.965447 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3670/7400] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:47 2022] DELETE /identity/v3/users/59259b8a3cf1490cb0d7ee084c5056e0 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:47.967296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fcb54-3eff-40a3-8059-20039aee7948 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:47.968346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fcb54-3eff-40a3-8059-20039aee7948 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:47.968833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fcb54-3eff-40a3-8059-20039aee7948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:48.002124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c16fcb54-3eff-40a3-8059-20039aee7948 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:48.003648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c16fcb54-3eff-40a3-8059-20039aee7948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:48.115457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:48.131582 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3731/7401] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:15:48.147448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4343ff3-4f6d-405f-b103-97cbcda24932 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:48.149350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c4343ff3-4f6d-405f-b103-97cbcda24932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:48.224367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:48.270144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4343ff3-4f6d-405f-b103-97cbcda24932 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:48.275579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21238471-0c21-494b-97c5-f4b1ac902743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:48.276793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21238471-0c21-494b-97c5-f4b1ac902743 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:48.277185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21238471-0c21-494b-97c5-f4b1ac902743 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:48.277723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21238471-0c21-494b-97c5-f4b1ac902743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:48.304563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21238471-0c21-494b-97c5-f4b1ac902743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:48.306269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21238471-0c21-494b-97c5-f4b1ac902743 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:48.320032 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3671/7402] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:15:48.950509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fcb54-3eff-40a3-8059-20039aee7948 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:48.952550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c16fcb54-3eff-40a3-8059-20039aee7948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:48.978076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fcb54-3eff-40a3-8059-20039aee7948 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:48.991446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0dfcb1-e2b3-40b0-8ded-fd0a1066349c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:48.992940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0dfcb1-e2b3-40b0-8ded-fd0a1066349c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:48.993640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0dfcb1-e2b3-40b0-8ded-fd0a1066349c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:48.994599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0dfcb1-e2b3-40b0-8ded-fd0a1066349c admin admin] PATH_INFO: `/v3/projects/8a8f6d58ed794cf7b6d1579407df6260` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:48.996191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0dfcb1-e2b3-40b0-8ded-fd0a1066349c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a8f6d58ed794cf7b6d1579407df6260)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:49.001206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0dfcb1-e2b3-40b0-8ded-fd0a1066349c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:49.120677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2e0dfcb1-e2b3-40b0-8ded-fd0a1066349c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a8f6d58ed794cf7b6d1579407df6260', 'request_id': 'req-2e0dfcb1-e2b3-40b0-8ded-fd0a1066349c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:49.130266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2e0dfcb1-e2b3-40b0-8ded-fd0a1066349c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a8f6d58ed794cf7b6d1579407df6260', 'request_id': 'req-2e0dfcb1-e2b3-40b0-8ded-fd0a1066349c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:49.137172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2e0dfcb1-e2b3-40b0-8ded-fd0a1066349c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a8f6d58ed794cf7b6d1579407df6260', 'request_id': 'req-2e0dfcb1-e2b3-40b0-8ded-fd0a1066349c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:49.161335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2e0dfcb1-e2b3-40b0-8ded-fd0a1066349c admin admin] The token cache is being invalidate because project 8a8f6d58ed794cf7b6d1579407df6260 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:49.161723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2e0dfcb1-e2b3-40b0-8ded-fd0a1066349c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:49.162450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2e0dfcb1-e2b3-40b0-8ded-fd0a1066349c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:49.165362 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3732/7403] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:48 2022] DELETE /identity/v3/projects/8a8f6d58ed794cf7b6d1579407df6260 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:49.170127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75894054-ed37-48a6-bc6f-d6bb1e3569ad None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:49.170678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75894054-ed37-48a6-bc6f-d6bb1e3569ad None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:49.171018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75894054-ed37-48a6-bc6f-d6bb1e3569ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:49.200725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-75894054-ed37-48a6-bc6f-d6bb1e3569ad None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:49.202375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-75894054-ed37-48a6-bc6f-d6bb1e3569ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:49.287234 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3672/7404] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:49.294726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231609a-65c3-457d-9c7e-b37993e7b472 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:49.295421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231609a-65c3-457d-9c7e-b37993e7b472 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:49.295905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231609a-65c3-457d-9c7e-b37993e7b472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:49.328005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b231609a-65c3-457d-9c7e-b37993e7b472 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:49.329848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b231609a-65c3-457d-9c7e-b37993e7b472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:49.346453 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3733/7405] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:49.350834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75894054-ed37-48a6-bc6f-d6bb1e3569ad None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:49.362128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75894054-ed37-48a6-bc6f-d6bb1e3569ad None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:49.370101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29191892-dc61-4c5a-aef7-a12e2fc9c02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:49.371133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29191892-dc61-4c5a-aef7-a12e2fc9c02b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:49.371797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29191892-dc61-4c5a-aef7-a12e2fc9c02b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:49.372293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29191892-dc61-4c5a-aef7-a12e2fc9c02b admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:49.373134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29191892-dc61-4c5a-aef7-a12e2fc9c02b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:15:49.373620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29191892-dc61-4c5a-aef7-a12e2fc9c02b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:49.380292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29191892-dc61-4c5a-aef7-a12e2fc9c02b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:49.411313 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3673/7406] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:49.415912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231609a-65c3-457d-9c7e-b37993e7b472 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:49.432701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231609a-65c3-457d-9c7e-b37993e7b472 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:49.448464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d98ca-fec5-476b-b88f-1c924e18be0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:49.451548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d98ca-fec5-476b-b88f-1c924e18be0a admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:49.452078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d98ca-fec5-476b-b88f-1c924e18be0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:49.452727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d98ca-fec5-476b-b88f-1c924e18be0a admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:49.453629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d98ca-fec5-476b-b88f-1c924e18be0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:49.455962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d98ca-fec5-476b-b88f-1c924e18be0a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:49.512594 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3734/7407] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:49 2022] POST /identity/v3/projects => generated 393 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:49.517186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29191892-dc61-4c5a-aef7-a12e2fc9c02b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:49.535554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29191892-dc61-4c5a-aef7-a12e2fc9c02b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:49.638921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03aedb0-9936-4515-8892-cd28654198de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:49.639685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03aedb0-9936-4515-8892-cd28654198de admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:49.640084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03aedb0-9936-4515-8892-cd28654198de admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:49.640432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03aedb0-9936-4515-8892-cd28654198de admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:49.641131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03aedb0-9936-4515-8892-cd28654198de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:49.643238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03aedb0-9936-4515-8892-cd28654198de admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:49.674444 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3674/7408] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:49 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:49.675756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d98ca-fec5-476b-b88f-1c924e18be0a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:49.691773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d98ca-fec5-476b-b88f-1c924e18be0a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:49.699140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04bddf3-3b10-46b9-8802-04941195e28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:49.699780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bddf3-3b10-46b9-8802-04941195e28d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:49.700062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bddf3-3b10-46b9-8802-04941195e28d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:49.700337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bddf3-3b10-46b9-8802-04941195e28d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:49.701664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04bddf3-3b10-46b9-8802-04941195e28d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:49.703961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04bddf3-3b10-46b9-8802-04941195e28d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:49.719590 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3735/7409] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:49.726384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03aedb0-9936-4515-8892-cd28654198de admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:49.738561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03aedb0-9936-4515-8892-cd28654198de admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:49.743568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7a8f0-5440-434b-be86-a08591fac3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:49.744203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7a8f0-5440-434b-be86-a08591fac3e5 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:49.744483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7a8f0-5440-434b-be86-a08591fac3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:49.744866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7a8f0-5440-434b-be86-a08591fac3e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:49.745403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7a8f0-5440-434b-be86-a08591fac3e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:49.748515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7a8f0-5440-434b-be86-a08591fac3e5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:49.765118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04bddf3-3b10-46b9-8802-04941195e28d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:49.771377 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3675/7410] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:49.777442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04bddf3-3b10-46b9-8802-04941195e28d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:49.790621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c09b0-4408-460e-a9f5-6c5f7f4ff569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:49.791680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c09b0-4408-460e-a9f5-6c5f7f4ff569 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:49.792920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c09b0-4408-460e-a9f5-6c5f7f4ff569 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:49.794130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c09b0-4408-460e-a9f5-6c5f7f4ff569 admin admin] PATH_INFO: `/v3/projects/6739feb3963749bfafe2cfc0b87a5d58/users/2679f307212b420ca10b339aa487bf08/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:49.820160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c09b0-4408-460e-a9f5-6c5f7f4ff569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6739feb3963749bfafe2cfc0b87a5d58, user_id=2679f307212b420ca10b339aa487bf08, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:49.825341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c09b0-4408-460e-a9f5-6c5f7f4ff569 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:49.846192 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3736/7411] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:49 2022] PUT /identity/v3/projects/6739feb3963749bfafe2cfc0b87a5d58/users/2679f307212b420ca10b339aa487bf08/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:49.856785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786356f-46f1-4913-be6c-6d653764dd4b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:49.857139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786356f-46f1-4913-be6c-6d653764dd4b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:49.857348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786356f-46f1-4913-be6c-6d653764dd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:49.922594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e786356f-46f1-4913-be6c-6d653764dd4b None None] MFA Rules not processed for user `2679f307212b420ca10b339aa487bf08`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:49.926748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e786356f-46f1-4913-be6c-6d653764dd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:49.996010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:50.067602 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3676/7412] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:49 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:50.072645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c09b0-4408-460e-a9f5-6c5f7f4ff569 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:50.092010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c09b0-4408-460e-a9f5-6c5f7f4ff569 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:50.106280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf88ed-97c4-40e7-b2af-30a4c4453803 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2679f307212b420ca10b339aa487bf08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6739feb3963749bfafe2cfc0b87a5d58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:50.106901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf88ed-97c4-40e7-b2af-30a4c4453803 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:50.107283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf88ed-97c4-40e7-b2af-30a4c4453803 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:50.107567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf88ed-97c4-40e7-b2af-30a4c4453803 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] PATH_INFO: `/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:50.108239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf88ed-97c4-40e7-b2af-30a4c4453803 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorizing `identity:create_application_credential(user_id=2679f307212b420ca10b339aa487bf08)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:50.109408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf88ed-97c4-40e7-b2af-30a4c4453803 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:50.222873 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3737/7413] 10.222.0.17 () {60 vars in 1314 bytes} [Sat Oct 1 21:15:50 2022] POST /identity/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials => generated 788 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:50.226663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e786356f-46f1-4913-be6c-6d653764dd4b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:50.239756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e786356f-46f1-4913-be6c-6d653764dd4b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:50.244924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d3489-7a37-4e62-b88f-781928230a67 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2679f307212b420ca10b339aa487bf08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6739feb3963749bfafe2cfc0b87a5d58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:50.246237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d3489-7a37-4e62-b88f-781928230a67 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:50.246237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d3489-7a37-4e62-b88f-781928230a67 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:50.246237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d3489-7a37-4e62-b88f-781928230a67 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] PATH_INFO: `/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials/28b2f6f0d83246838cbc004401c2726c` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:50.257201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d3489-7a37-4e62-b88f-781928230a67 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorizing `identity:get_application_credential(user_id=2679f307212b420ca10b339aa487bf08, application_credential_id=28b2f6f0d83246838cbc004401c2726c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:50.259083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d3489-7a37-4e62-b88f-781928230a67 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:50.262665 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3677/7414] 10.222.0.17 () {58 vars in 1392 bytes} [Sat Oct 1 21:15:50 2022] GET /identity/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials/28b2f6f0d83246838cbc004401c2726c => generated 688 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:50.267120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319b8b5-540a-44ed-81eb-f4bd65405e5d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:50.267529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319b8b5-540a-44ed-81eb-f4bd65405e5d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:50.268074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319b8b5-540a-44ed-81eb-f4bd65405e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:50.291750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1319b8b5-540a-44ed-81eb-f4bd65405e5d None None] MFA Rules not processed for user `2679f307212b420ca10b339aa487bf08`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:50.293363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1319b8b5-540a-44ed-81eb-f4bd65405e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:50.303212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:50.442914 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3738/7415] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:50 2022] POST /identity/v3/auth/tokens => generated 3426 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Oct 01 21:15:50.446816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d3489-7a37-4e62-b88f-781928230a67 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:50.457701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d3489-7a37-4e62-b88f-781928230a67 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:50.462930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94893050-b0cb-4689-8eed-27c26f907e20 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2679f307212b420ca10b339aa487bf08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6739feb3963749bfafe2cfc0b87a5d58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:50.463800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94893050-b0cb-4689-8eed-27c26f907e20 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:50.464048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94893050-b0cb-4689-8eed-27c26f907e20 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:50.464326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94893050-b0cb-4689-8eed-27c26f907e20 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] PATH_INFO: `/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials/28b2f6f0d83246838cbc004401c2726c` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:50.465603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94893050-b0cb-4689-8eed-27c26f907e20 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorizing `identity:delete_application_credential(user_id=2679f307212b420ca10b339aa487bf08, application_credential_id=28b2f6f0d83246838cbc004401c2726c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:50.467208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94893050-b0cb-4689-8eed-27c26f907e20 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:50.497703 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3678/7416] 10.222.0.17 () {58 vars in 1395 bytes} [Sat Oct 1 21:15:50 2022] DELETE /identity/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials/28b2f6f0d83246838cbc004401c2726c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:50.505612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319b8b5-540a-44ed-81eb-f4bd65405e5d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:50.516662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319b8b5-540a-44ed-81eb-f4bd65405e5d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:50.523258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735e4ba-bd13-4694-a35d-8a13ea174087 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2679f307212b420ca10b339aa487bf08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6739feb3963749bfafe2cfc0b87a5d58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:50.524056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735e4ba-bd13-4694-a35d-8a13ea174087 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:50.524467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735e4ba-bd13-4694-a35d-8a13ea174087 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:50.524858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735e4ba-bd13-4694-a35d-8a13ea174087 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] PATH_INFO: `/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:50.525863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735e4ba-bd13-4694-a35d-8a13ea174087 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorizing `identity:create_application_credential(user_id=2679f307212b420ca10b339aa487bf08)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:50.530719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735e4ba-bd13-4694-a35d-8a13ea174087 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:50.642975 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3739/7417] 10.222.0.17 () {60 vars in 1315 bytes} [Sat Oct 1 21:15:50 2022] POST /identity/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials => generated 921 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:50.647086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94893050-b0cb-4689-8eed-27c26f907e20 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:50.659910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94893050-b0cb-4689-8eed-27c26f907e20 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:50.664435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f22c0-2c90-4a40-97be-23bfb446ceeb tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2679f307212b420ca10b339aa487bf08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6739feb3963749bfafe2cfc0b87a5d58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:50.665571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f22c0-2c90-4a40-97be-23bfb446ceeb tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:50.665843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f22c0-2c90-4a40-97be-23bfb446ceeb tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:50.666122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f22c0-2c90-4a40-97be-23bfb446ceeb tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] PATH_INFO: `/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials/c5152b96cee34473999248078dba4434` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:50.681288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f22c0-2c90-4a40-97be-23bfb446ceeb tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorizing `identity:delete_application_credential(user_id=2679f307212b420ca10b339aa487bf08, application_credential_id=c5152b96cee34473999248078dba4434)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:50.683276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f22c0-2c90-4a40-97be-23bfb446ceeb tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:50.715882 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3679/7418] 10.222.0.17 () {58 vars in 1395 bytes} [Sat Oct 1 21:15:50 2022] DELETE /identity/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials/c5152b96cee34473999248078dba4434 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:50.721580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735e4ba-bd13-4694-a35d-8a13ea174087 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:50.746782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735e4ba-bd13-4694-a35d-8a13ea174087 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:50.760084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba2f47-ecc4-4f84-8be1-e655d81f0cd1 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2679f307212b420ca10b339aa487bf08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6739feb3963749bfafe2cfc0b87a5d58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:50.763214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba2f47-ecc4-4f84-8be1-e655d81f0cd1 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:50.763784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba2f47-ecc4-4f84-8be1-e655d81f0cd1 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:50.764251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba2f47-ecc4-4f84-8be1-e655d81f0cd1 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] PATH_INFO: `/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:50.766297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba2f47-ecc4-4f84-8be1-e655d81f0cd1 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorizing `identity:create_application_credential(user_id=2679f307212b420ca10b339aa487bf08)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:50.768129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba2f47-ecc4-4f84-8be1-e655d81f0cd1 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:50.872725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:50.911912 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3740/7419] 10.222.0.17 () {60 vars in 1315 bytes} [Sat Oct 1 21:15:50 2022] POST /identity/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials => generated 813 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:50.916816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f22c0-2c90-4a40-97be-23bfb446ceeb tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:50.931222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f22c0-2c90-4a40-97be-23bfb446ceeb tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:50.944341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8e39d-0820-4bfe-bc01-f8668074dd3e tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2679f307212b420ca10b339aa487bf08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6739feb3963749bfafe2cfc0b87a5d58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:50.945134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8e39d-0820-4bfe-bc01-f8668074dd3e tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:50.945622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8e39d-0820-4bfe-bc01-f8668074dd3e tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:50.945973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8e39d-0820-4bfe-bc01-f8668074dd3e tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] PATH_INFO: `/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials/4f8d262f9eba480c9468310ca576844d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:50.964501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8e39d-0820-4bfe-bc01-f8668074dd3e tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorizing `identity:delete_application_credential(user_id=2679f307212b420ca10b339aa487bf08, application_credential_id=4f8d262f9eba480c9468310ca576844d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:50.966293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8e39d-0820-4bfe-bc01-f8668074dd3e tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:51.007914 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3680/7420] 10.222.0.17 () {58 vars in 1395 bytes} [Sat Oct 1 21:15:50 2022] DELETE /identity/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials/4f8d262f9eba480c9468310ca576844d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:51.013440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba2f47-ecc4-4f84-8be1-e655d81f0cd1 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:51.056982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba2f47-ecc4-4f84-8be1-e655d81f0cd1 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:51.068042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd279a2-b95f-4820-b0d6-a33627ca7ff9 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2679f307212b420ca10b339aa487bf08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6739feb3963749bfafe2cfc0b87a5d58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:51.068832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd279a2-b95f-4820-b0d6-a33627ca7ff9 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:51.069084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd279a2-b95f-4820-b0d6-a33627ca7ff9 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:51.069362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd279a2-b95f-4820-b0d6-a33627ca7ff9 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] PATH_INFO: `/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:51.071017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd279a2-b95f-4820-b0d6-a33627ca7ff9 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorizing `identity:create_application_credential(user_id=2679f307212b420ca10b339aa487bf08)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:51.072215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd279a2-b95f-4820-b0d6-a33627ca7ff9 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:51.242138 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3741/7421] 10.222.0.17 () {60 vars in 1314 bytes} [Sat Oct 1 21:15:51 2022] POST /identity/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials => generated 788 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:51.243041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8e39d-0820-4bfe-bc01-f8668074dd3e tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:51.254364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8e39d-0820-4bfe-bc01-f8668074dd3e tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:51.258904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5cb59-aa72-41dd-aca4-ad1d444f3c3a tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2679f307212b420ca10b339aa487bf08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6739feb3963749bfafe2cfc0b87a5d58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:51.259717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5cb59-aa72-41dd-aca4-ad1d444f3c3a tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:51.260302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5cb59-aa72-41dd-aca4-ad1d444f3c3a tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:51.260645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5cb59-aa72-41dd-aca4-ad1d444f3c3a tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] PATH_INFO: `/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:51.261399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd5cb59-aa72-41dd-aca4-ad1d444f3c3a tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorizing `identity:create_application_credential(user_id=2679f307212b420ca10b339aa487bf08)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:51.262670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd5cb59-aa72-41dd-aca4-ad1d444f3c3a tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:51.359991 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3681/7422] 10.222.0.17 () {60 vars in 1314 bytes} [Sat Oct 1 21:15:51 2022] POST /identity/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials => generated 789 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:51.364164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd279a2-b95f-4820-b0d6-a33627ca7ff9 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:51.377220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd279a2-b95f-4820-b0d6-a33627ca7ff9 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:51.408759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743a385-c02a-411a-b007-2d3d7b0d2fbd tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2679f307212b420ca10b339aa487bf08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6739feb3963749bfafe2cfc0b87a5d58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:51.409543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743a385-c02a-411a-b007-2d3d7b0d2fbd tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:51.409826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743a385-c02a-411a-b007-2d3d7b0d2fbd tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:51.410548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743a385-c02a-411a-b007-2d3d7b0d2fbd tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] PATH_INFO: `/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:51.411086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743a385-c02a-411a-b007-2d3d7b0d2fbd tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorizing `identity:list_application_credentials(user_id=2679f307212b420ca10b339aa487bf08)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:51.412835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743a385-c02a-411a-b007-2d3d7b0d2fbd tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:51.443861 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3742/7423] 10.222.0.17 () {58 vars in 1293 bytes} [Sat Oct 1 21:15:51 2022] GET /identity/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials => generated 1501 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:51.449072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5cb59-aa72-41dd-aca4-ad1d444f3c3a tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:51.461647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5cb59-aa72-41dd-aca4-ad1d444f3c3a tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:51.480514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaef1b3-1551-4130-bfed-5653a002a65b tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2679f307212b420ca10b339aa487bf08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6739feb3963749bfafe2cfc0b87a5d58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:51.481190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaef1b3-1551-4130-bfed-5653a002a65b tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:51.481728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaef1b3-1551-4130-bfed-5653a002a65b tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:51.481728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaef1b3-1551-4130-bfed-5653a002a65b tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] PATH_INFO: `/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials/7d347f6374b348b8a45664dfc9a1a88b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:51.494767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaef1b3-1551-4130-bfed-5653a002a65b tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorizing `identity:delete_application_credential(user_id=2679f307212b420ca10b339aa487bf08, application_credential_id=7d347f6374b348b8a45664dfc9a1a88b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:51.497309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaef1b3-1551-4130-bfed-5653a002a65b tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:51.510678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:51.529059 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3682/7424] 10.222.0.17 () {58 vars in 1395 bytes} [Sat Oct 1 21:15:51 2022] DELETE /identity/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials/7d347f6374b348b8a45664dfc9a1a88b => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:51.533038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743a385-c02a-411a-b007-2d3d7b0d2fbd tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:51.545313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743a385-c02a-411a-b007-2d3d7b0d2fbd tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:51.553277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e169b7-f9c5-4757-95d0-5221a2abd46b tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2679f307212b420ca10b339aa487bf08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6739feb3963749bfafe2cfc0b87a5d58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:51.553967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e169b7-f9c5-4757-95d0-5221a2abd46b tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:51.554255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e169b7-f9c5-4757-95d0-5221a2abd46b tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:51.554572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e169b7-f9c5-4757-95d0-5221a2abd46b tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] PATH_INFO: `/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials/8f6b231bab2b4e05903296fa1155a98e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:51.583896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e169b7-f9c5-4757-95d0-5221a2abd46b tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorizing `identity:delete_application_credential(user_id=2679f307212b420ca10b339aa487bf08, application_credential_id=8f6b231bab2b4e05903296fa1155a98e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:51.585870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e169b7-f9c5-4757-95d0-5221a2abd46b tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:51.634523 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3743/7425] 10.222.0.17 () {58 vars in 1395 bytes} [Sat Oct 1 21:15:51 2022] DELETE /identity/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials/8f6b231bab2b4e05903296fa1155a98e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:51.641322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaef1b3-1551-4130-bfed-5653a002a65b tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:51.664315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaef1b3-1551-4130-bfed-5653a002a65b tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:51.678455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01287f1a-6783-4144-a92b-2d9c4f517d40 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2679f307212b420ca10b339aa487bf08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6739feb3963749bfafe2cfc0b87a5d58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:51.678735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01287f1a-6783-4144-a92b-2d9c4f517d40 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:51.679076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01287f1a-6783-4144-a92b-2d9c4f517d40 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:51.679316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01287f1a-6783-4144-a92b-2d9c4f517d40 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] PATH_INFO: `/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:51.679930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01287f1a-6783-4144-a92b-2d9c4f517d40 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorizing `identity:create_application_credential(user_id=2679f307212b420ca10b339aa487bf08)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:51.683784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01287f1a-6783-4144-a92b-2d9c4f517d40 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:51.806867 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3683/7426] 10.222.0.17 () {60 vars in 1314 bytes} [Sat Oct 1 21:15:51 2022] POST /identity/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials => generated 789 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:51.808115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e169b7-f9c5-4757-95d0-5221a2abd46b tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:51.820311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e169b7-f9c5-4757-95d0-5221a2abd46b tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:51.825158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd1e61-4473-4df1-a776-9ca42f3e9e8f tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2679f307212b420ca10b339aa487bf08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6739feb3963749bfafe2cfc0b87a5d58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:51.825819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd1e61-4473-4df1-a776-9ca42f3e9e8f tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:51.826106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd1e61-4473-4df1-a776-9ca42f3e9e8f tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:51.826438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd1e61-4473-4df1-a776-9ca42f3e9e8f tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] PATH_INFO: `/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:51.827444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd1e61-4473-4df1-a776-9ca42f3e9e8f tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorizing `identity:create_application_credential(user_id=2679f307212b420ca10b339aa487bf08)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:51.828622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd1e61-4473-4df1-a776-9ca42f3e9e8f tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:51.950589 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3744/7427] 10.222.0.17 () {60 vars in 1314 bytes} [Sat Oct 1 21:15:51 2022] POST /identity/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials => generated 788 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:51.953069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01287f1a-6783-4144-a92b-2d9c4f517d40 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:51.963244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01287f1a-6783-4144-a92b-2d9c4f517d40 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:51.967669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99602a4d-debd-4c8a-b703-dd55717c8efe tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2679f307212b420ca10b339aa487bf08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6739feb3963749bfafe2cfc0b87a5d58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:51.968279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99602a4d-debd-4c8a-b703-dd55717c8efe tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:51.968560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99602a4d-debd-4c8a-b703-dd55717c8efe tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:51.968834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99602a4d-debd-4c8a-b703-dd55717c8efe tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] PATH_INFO: `/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:51.970019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99602a4d-debd-4c8a-b703-dd55717c8efe tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Adding query filter params (name=tempest-application_credential-989762067) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:15:51.970019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99602a4d-debd-4c8a-b703-dd55717c8efe tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorizing `identity:list_application_credentials(user_id=2679f307212b420ca10b339aa487bf08)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:51.971730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99602a4d-debd-4c8a-b703-dd55717c8efe tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:51.990773 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3684/7428] 10.222.0.17 () {58 vars in 1384 bytes} [Sat Oct 1 21:15:51 2022] GET /identity/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials?name=tempest-application_credential-989762067 => generated 885 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:51.991874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd1e61-4473-4df1-a776-9ca42f3e9e8f tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:52.010355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd1e61-4473-4df1-a776-9ca42f3e9e8f tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:52.015592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7b82f-0488-4789-a028-ca7af6d9d10f tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2679f307212b420ca10b339aa487bf08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6739feb3963749bfafe2cfc0b87a5d58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:52.016749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7b82f-0488-4789-a028-ca7af6d9d10f tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:52.017353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7b82f-0488-4789-a028-ca7af6d9d10f tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:52.017714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7b82f-0488-4789-a028-ca7af6d9d10f tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] PATH_INFO: `/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials/98a0c9c7082e4c21a50bf22b1f88eeb1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:52.034862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7b82f-0488-4789-a028-ca7af6d9d10f tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorizing `identity:delete_application_credential(user_id=2679f307212b420ca10b339aa487bf08, application_credential_id=98a0c9c7082e4c21a50bf22b1f88eeb1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:52.037254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7b82f-0488-4789-a028-ca7af6d9d10f tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:52.092467 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3745/7429] 10.222.0.17 () {58 vars in 1395 bytes} [Sat Oct 1 21:15:51 2022] DELETE /identity/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials/98a0c9c7082e4c21a50bf22b1f88eeb1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:52.097008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99602a4d-debd-4c8a-b703-dd55717c8efe tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:52.120355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99602a4d-debd-4c8a-b703-dd55717c8efe tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:52.130073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0ca6b-479a-49c7-b189-3587f00d9340 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2679f307212b420ca10b339aa487bf08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6739feb3963749bfafe2cfc0b87a5d58', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:52.131163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0ca6b-479a-49c7-b189-3587f00d9340 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:52.131549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0ca6b-479a-49c7-b189-3587f00d9340 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:52.132053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0ca6b-479a-49c7-b189-3587f00d9340 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] PATH_INFO: `/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials/2ed05b6497ed4f42955647485ab8b263` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:52.142450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0ca6b-479a-49c7-b189-3587f00d9340 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorizing `identity:delete_application_credential(user_id=2679f307212b420ca10b339aa487bf08, application_credential_id=2ed05b6497ed4f42955647485ab8b263)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:52.144496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0ca6b-479a-49c7-b189-3587f00d9340 tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:52.181763 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3685/7430] 10.222.0.17 () {58 vars in 1395 bytes} [Sat Oct 1 21:15:52 2022] DELETE /identity/v3/users/2679f307212b420ca10b339aa487bf08/application_credentials/2ed05b6497ed4f42955647485ab8b263 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:52.186785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7b82f-0488-4789-a028-ca7af6d9d10f tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:52.195198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=6,keystone:SELECT=90,keystone:DELETE=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:52.361965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7b82f-0488-4789-a028-ca7af6d9d10f tempest-ApplicationCredentialsV3Test-1627204436 tempest-ApplicationCredentialsV3Test-1627204436-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:52.367995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b61aa-ed7f-4de0-9bdb-6c7642aefb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:52.368743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b61aa-ed7f-4de0-9bdb-6c7642aefb7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:52.369123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b61aa-ed7f-4de0-9bdb-6c7642aefb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:52.369471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b61aa-ed7f-4de0-9bdb-6c7642aefb7f admin admin] PATH_INFO: `/v3/users/2679f307212b420ca10b339aa487bf08` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:52.370803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b61aa-ed7f-4de0-9bdb-6c7642aefb7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2679f307212b420ca10b339aa487bf08)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:52.377703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b61aa-ed7f-4de0-9bdb-6c7642aefb7f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:52.526101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac8b61aa-ed7f-4de0-9bdb-6c7642aefb7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2679f307212b420ca10b339aa487bf08'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:52.542390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac8b61aa-ed7f-4de0-9bdb-6c7642aefb7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2679f307212b420ca10b339aa487bf08'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:52.574866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac8b61aa-ed7f-4de0-9bdb-6c7642aefb7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2679f307212b420ca10b339aa487bf08'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:52.576235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac8b61aa-ed7f-4de0-9bdb-6c7642aefb7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2679f307212b420ca10b339aa487bf08'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:52.589455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ac8b61aa-ed7f-4de0-9bdb-6c7642aefb7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2679f307212b420ca10b339aa487bf08'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:52.602769 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3746/7431] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:52 2022] DELETE /identity/v3/users/2679f307212b420ca10b339aa487bf08 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:52.608423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74fee5-5d1c-4e49-a83d-bf92f7678441 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:52.608423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74fee5-5d1c-4e49-a83d-bf92f7678441 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:52.608423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74fee5-5d1c-4e49-a83d-bf92f7678441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:52.641811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ee74fee5-5d1c-4e49-a83d-bf92f7678441 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:52.643252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ee74fee5-5d1c-4e49-a83d-bf92f7678441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:52.723003 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3686/7432] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:52.755758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b61aa-ed7f-4de0-9bdb-6c7642aefb7f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:52.758069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ac8b61aa-ed7f-4de0-9bdb-6c7642aefb7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:52.911841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b61aa-ed7f-4de0-9bdb-6c7642aefb7f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:52.916274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8c323-da52-44a9-80e8-8e57fc0bbdfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:52.916911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8c323-da52-44a9-80e8-8e57fc0bbdfb service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:52.917981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8c323-da52-44a9-80e8-8e57fc0bbdfb service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:52.918915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8c323-da52-44a9-80e8-8e57fc0bbdfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:52.935571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8c323-da52-44a9-80e8-8e57fc0bbdfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:52.937041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8c323-da52-44a9-80e8-8e57fc0bbdfb service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:52.952777 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3747/7433] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:15:53.259108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74fee5-5d1c-4e49-a83d-bf92f7678441 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:53.262275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ee74fee5-5d1c-4e49-a83d-bf92f7678441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:53.304303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74fee5-5d1c-4e49-a83d-bf92f7678441 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:53.316791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e447ee-3c83-44e4-93ce-3d7484d42042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:53.318854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e447ee-3c83-44e4-93ce-3d7484d42042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:53.319356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e447ee-3c83-44e4-93ce-3d7484d42042 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:53.319736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e447ee-3c83-44e4-93ce-3d7484d42042 admin admin] PATH_INFO: `/v3/users/9476b7020b2c40e9abb14d11b7481f77` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:53.321309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e447ee-3c83-44e4-93ce-3d7484d42042 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9476b7020b2c40e9abb14d11b7481f77)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:53.324145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e447ee-3c83-44e4-93ce-3d7484d42042 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:53.428761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:53.436586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b6e447ee-3c83-44e4-93ce-3d7484d42042 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9476b7020b2c40e9abb14d11b7481f77'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:53.438182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b6e447ee-3c83-44e4-93ce-3d7484d42042 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9476b7020b2c40e9abb14d11b7481f77'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:53.439940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b6e447ee-3c83-44e4-93ce-3d7484d42042 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9476b7020b2c40e9abb14d11b7481f77'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:53.456110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b6e447ee-3c83-44e4-93ce-3d7484d42042 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9476b7020b2c40e9abb14d11b7481f77'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:53.479015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b6e447ee-3c83-44e4-93ce-3d7484d42042 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9476b7020b2c40e9abb14d11b7481f77'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:53.507658 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3687/7434] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:53 2022] DELETE /identity/v3/users/9476b7020b2c40e9abb14d11b7481f77 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:53.818901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8c323-da52-44a9-80e8-8e57fc0bbdfb service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:53.820394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-48a8c323-da52-44a9-80e8-8e57fc0bbdfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:53.837862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8c323-da52-44a9-80e8-8e57fc0bbdfb service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:53.882300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:53.929561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30c7c8-dd44-46e0-950a-76d771a0603e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:53.930199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30c7c8-dd44-46e0-950a-76d771a0603e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:53.930481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30c7c8-dd44-46e0-950a-76d771a0603e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:53.930802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30c7c8-dd44-46e0-950a-76d771a0603e admin admin] PATH_INFO: `/v3/projects/6739feb3963749bfafe2cfc0b87a5d58` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:53.931950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30c7c8-dd44-46e0-950a-76d771a0603e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6739feb3963749bfafe2cfc0b87a5d58)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:53.933846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30c7c8-dd44-46e0-950a-76d771a0603e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:53.995419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1c30c7c8-dd44-46e0-950a-76d771a0603e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6739feb3963749bfafe2cfc0b87a5d58', 'request_id': 'req-1c30c7c8-dd44-46e0-950a-76d771a0603e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:54.002685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1c30c7c8-dd44-46e0-950a-76d771a0603e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6739feb3963749bfafe2cfc0b87a5d58', 'request_id': 'req-1c30c7c8-dd44-46e0-950a-76d771a0603e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:54.012406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1c30c7c8-dd44-46e0-950a-76d771a0603e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6739feb3963749bfafe2cfc0b87a5d58', 'request_id': 'req-1c30c7c8-dd44-46e0-950a-76d771a0603e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:54.044130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1c30c7c8-dd44-46e0-950a-76d771a0603e admin admin] The token cache is being invalidate because project 6739feb3963749bfafe2cfc0b87a5d58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:54.044899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1c30c7c8-dd44-46e0-950a-76d771a0603e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:54.046941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1c30c7c8-dd44-46e0-950a-76d771a0603e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:54.052455 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3748/7435] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:53 2022] DELETE /identity/v3/projects/6739feb3963749bfafe2cfc0b87a5d58 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:54.057981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e68ddf-b293-4c0c-b5bb-dd5033a972bd None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:54.058479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e68ddf-b293-4c0c-b5bb-dd5033a972bd None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:54.058993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e68ddf-b293-4c0c-b5bb-dd5033a972bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:54.081353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-37e68ddf-b293-4c0c-b5bb-dd5033a972bd None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:54.082635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-37e68ddf-b293-4c0c-b5bb-dd5033a972bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:54.096730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30c7c8-dd44-46e0-950a-76d771a0603e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:54.098352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1c30c7c8-dd44-46e0-950a-76d771a0603e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:54.185918 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3688/7436] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:15:54.188947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba1a5c-9e93-4605-96bd-81c36dd9e7fb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:54.189619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba1a5c-9e93-4605-96bd-81c36dd9e7fb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:54.189619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba1a5c-9e93-4605-96bd-81c36dd9e7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:54.222278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30c7c8-dd44-46e0-950a-76d771a0603e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:54.228164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83f6cb-d990-4406-bc9d-2d1818052e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:54.228950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83f6cb-d990-4406-bc9d-2d1818052e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:54.229067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b5ba1a5c-9e93-4605-96bd-81c36dd9e7fb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:54.229258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83f6cb-d990-4406-bc9d-2d1818052e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:54.229504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83f6cb-d990-4406-bc9d-2d1818052e17 admin admin] PATH_INFO: `/v3/projects/2fa455091de94f1abda18743c51fb284` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:54.230198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b5ba1a5c-9e93-4605-96bd-81c36dd9e7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:54.230702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83f6cb-d990-4406-bc9d-2d1818052e17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fa455091de94f1abda18743c51fb284)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:54.232718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83f6cb-d990-4406-bc9d-2d1818052e17 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:54.253294 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3689/7437] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:54.254538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba1a5c-9e93-4605-96bd-81c36dd9e7fb None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:54.266965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba1a5c-9e93-4605-96bd-81c36dd9e7fb None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:54.277108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bf073-0e3c-4bc2-b325-742c3f8711f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:54.277894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bf073-0e3c-4bc2-b325-742c3f8711f0 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:54.278177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bf073-0e3c-4bc2-b325-742c3f8711f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:54.278530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bf073-0e3c-4bc2-b325-742c3f8711f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:54.279196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bf073-0e3c-4bc2-b325-742c3f8711f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:15:54.279496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bf073-0e3c-4bc2-b325-742c3f8711f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:54.281106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bf073-0e3c-4bc2-b325-742c3f8711f0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:54.295518 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3690/7438] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:54.300219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bf073-0e3c-4bc2-b325-742c3f8711f0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:54.312510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bf073-0e3c-4bc2-b325-742c3f8711f0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:54.313452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1e83f6cb-d990-4406-bc9d-2d1818052e17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fa455091de94f1abda18743c51fb284', 'request_id': 'req-1e83f6cb-d990-4406-bc9d-2d1818052e17'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:54.320667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1e83f6cb-d990-4406-bc9d-2d1818052e17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fa455091de94f1abda18743c51fb284', 'request_id': 'req-1e83f6cb-d990-4406-bc9d-2d1818052e17'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:54.322162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:54.330674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1e83f6cb-d990-4406-bc9d-2d1818052e17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fa455091de94f1abda18743c51fb284', 'request_id': 'req-1e83f6cb-d990-4406-bc9d-2d1818052e17'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:54.355061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1e83f6cb-d990-4406-bc9d-2d1818052e17 admin admin] The token cache is being invalidate because project 2fa455091de94f1abda18743c51fb284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:54.355665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1e83f6cb-d990-4406-bc9d-2d1818052e17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:54.356913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1e83f6cb-d990-4406-bc9d-2d1818052e17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:54.364050 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3749/7439] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:54 2022] DELETE /identity/v3/projects/2fa455091de94f1abda18743c51fb284 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:54.365399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83f6cb-d990-4406-bc9d-2d1818052e17 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:54.367916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1e83f6cb-d990-4406-bc9d-2d1818052e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:54.410444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdb709-4337-4b93-a37e-aec3bcf76576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:54.411558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdb709-4337-4b93-a37e-aec3bcf76576 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:54.411904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdb709-4337-4b93-a37e-aec3bcf76576 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:54.412368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdb709-4337-4b93-a37e-aec3bcf76576 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:54.413276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fdb709-4337-4b93-a37e-aec3bcf76576 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:54.415359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fdb709-4337-4b93-a37e-aec3bcf76576 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:54.443444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83f6cb-d990-4406-bc9d-2d1818052e17 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:54.448933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef40ee-cecf-4a26-966b-acbdbe31e5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:54.449571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef40ee-cecf-4a26-966b-acbdbe31e5e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:54.449857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef40ee-cecf-4a26-966b-acbdbe31e5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:54.450135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef40ee-cecf-4a26-966b-acbdbe31e5e6 admin admin] PATH_INFO: `/v3/users/f07b4382a49046bfb100fbfbbb4b896a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:54.451094 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3691/7440] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:54 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:54.451544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef40ee-cecf-4a26-966b-acbdbe31e5e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f07b4382a49046bfb100fbfbbb4b896a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:54.453439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef40ee-cecf-4a26-966b-acbdbe31e5e6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:54.454916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdb709-4337-4b93-a37e-aec3bcf76576 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:54.456703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-42fdb709-4337-4b93-a37e-aec3bcf76576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:54.490259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:54.514712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-94ef40ee-cecf-4a26-966b-acbdbe31e5e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f07b4382a49046bfb100fbfbbb4b896a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:54.522876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-94ef40ee-cecf-4a26-966b-acbdbe31e5e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f07b4382a49046bfb100fbfbbb4b896a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:54.538920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-94ef40ee-cecf-4a26-966b-acbdbe31e5e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f07b4382a49046bfb100fbfbbb4b896a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:54.540034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-94ef40ee-cecf-4a26-966b-acbdbe31e5e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f07b4382a49046bfb100fbfbbb4b896a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:54.545384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdb709-4337-4b93-a37e-aec3bcf76576 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:54.549696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-94ef40ee-cecf-4a26-966b-acbdbe31e5e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f07b4382a49046bfb100fbfbbb4b896a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:54.553868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bebc9-8fd7-4b36-a184-bc104c2f95e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:54.554706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bebc9-8fd7-4b36-a184-bc104c2f95e0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:54.554990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bebc9-8fd7-4b36-a184-bc104c2f95e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:54.555369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bebc9-8fd7-4b36-a184-bc104c2f95e0 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:54.555875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556bebc9-8fd7-4b36-a184-bc104c2f95e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:54.558017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556bebc9-8fd7-4b36-a184-bc104c2f95e0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:54.559158 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3750/7441] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:54 2022] DELETE /identity/v3/users/f07b4382a49046bfb100fbfbbb4b896a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:54.586377 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3692/7442] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:54 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:54.591688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef40ee-cecf-4a26-966b-acbdbe31e5e6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:54.593217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-94ef40ee-cecf-4a26-966b-acbdbe31e5e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:54.687948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef40ee-cecf-4a26-966b-acbdbe31e5e6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:54.693979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ca440-ff53-401d-af10-9589c8f1d17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:54.694637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ca440-ff53-401d-af10-9589c8f1d17d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:54.694923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ca440-ff53-401d-af10-9589c8f1d17d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:54.695206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ca440-ff53-401d-af10-9589c8f1d17d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:54.695854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ca440-ff53-401d-af10-9589c8f1d17d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:54.697456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ca440-ff53-401d-af10-9589c8f1d17d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:54.700858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:54.712168 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3751/7443] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:15:54.715958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bebc9-8fd7-4b36-a184-bc104c2f95e0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:54.729210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bebc9-8fd7-4b36-a184-bc104c2f95e0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:54.735683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910fe21e-98fb-4a99-ae5b-1d884dd90713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:54.736377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fe21e-98fb-4a99-ae5b-1d884dd90713 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:54.737265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fe21e-98fb-4a99-ae5b-1d884dd90713 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:54.737265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fe21e-98fb-4a99-ae5b-1d884dd90713 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:54.740484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910fe21e-98fb-4a99-ae5b-1d884dd90713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:54.742172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910fe21e-98fb-4a99-ae5b-1d884dd90713 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:54.753575 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3693/7444] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:54.758090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ca440-ff53-401d-af10-9589c8f1d17d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:54.774268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ca440-ff53-401d-af10-9589c8f1d17d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:54.781507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910fe21e-98fb-4a99-ae5b-1d884dd90713 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:54.783928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-910fe21e-98fb-4a99-ae5b-1d884dd90713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:54.784456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975a5e2-82e5-4c1c-98f8-9f73d62fcaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:54.785340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975a5e2-82e5-4c1c-98f8-9f73d62fcaae admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:54.785674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975a5e2-82e5-4c1c-98f8-9f73d62fcaae admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:54.785956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975a5e2-82e5-4c1c-98f8-9f73d62fcaae admin admin] PATH_INFO: `/v3/projects/62412fd0bdc04170b42bca3ffa0f141c/users/36769adab18c4d629689b3dd81009842/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:54.814397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975a5e2-82e5-4c1c-98f8-9f73d62fcaae admin admin] RBAC: Authorizing `identity:create_grant(project_id=62412fd0bdc04170b42bca3ffa0f141c, user_id=36769adab18c4d629689b3dd81009842, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:54.817655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975a5e2-82e5-4c1c-98f8-9f73d62fcaae admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:54.823129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910fe21e-98fb-4a99-ae5b-1d884dd90713 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:54.829144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be7d800-4cf4-437d-9679-e31bf4dc7ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:54.829895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7d800-4cf4-437d-9679-e31bf4dc7ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:54.830181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7d800-4cf4-437d-9679-e31bf4dc7ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:54.830459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7d800-4cf4-437d-9679-e31bf4dc7ac7 admin admin] PATH_INFO: `/v3/projects/3a4e16bc67d745f7864ddf366c0a067f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:54.831658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be7d800-4cf4-437d-9679-e31bf4dc7ac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a4e16bc67d745f7864ddf366c0a067f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:54.833618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be7d800-4cf4-437d-9679-e31bf4dc7ac7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:54.836348 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3752/7445] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:54 2022] PUT /identity/v3/projects/62412fd0bdc04170b42bca3ffa0f141c/users/36769adab18c4d629689b3dd81009842/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:54.846697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8da22-7b9f-4e1b-889e-da5c8f6f4dfc None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:54.847129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8da22-7b9f-4e1b-889e-da5c8f6f4dfc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:54.850864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8da22-7b9f-4e1b-889e-da5c8f6f4dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:54.890867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-bdb8da22-7b9f-4e1b-889e-da5c8f6f4dfc None None] MFA Rules not processed for user `36769adab18c4d629689b3dd81009842`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:54.892176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bdb8da22-7b9f-4e1b-889e-da5c8f6f4dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:54.905056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4be7d800-4cf4-437d-9679-e31bf4dc7ac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a4e16bc67d745f7864ddf366c0a067f', 'request_id': 'req-4be7d800-4cf4-437d-9679-e31bf4dc7ac7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:54.914026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4be7d800-4cf4-437d-9679-e31bf4dc7ac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a4e16bc67d745f7864ddf366c0a067f', 'request_id': 'req-4be7d800-4cf4-437d-9679-e31bf4dc7ac7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:54.923242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4be7d800-4cf4-437d-9679-e31bf4dc7ac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a4e16bc67d745f7864ddf366c0a067f', 'request_id': 'req-4be7d800-4cf4-437d-9679-e31bf4dc7ac7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:54.940280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4be7d800-4cf4-437d-9679-e31bf4dc7ac7 admin admin] The token cache is being invalidate because project 3a4e16bc67d745f7864ddf366c0a067f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:54.941871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4be7d800-4cf4-437d-9679-e31bf4dc7ac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:54.942952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4be7d800-4cf4-437d-9679-e31bf4dc7ac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:54.949552 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3694/7446] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:54 2022] DELETE /identity/v3/projects/3a4e16bc67d745f7864ddf366c0a067f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:54.956606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2886039-c9f0-47f1-845b-4ce39388efda None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:54.956938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2886039-c9f0-47f1-845b-4ce39388efda None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:54.957213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2886039-c9f0-47f1-845b-4ce39388efda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:54.981573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d2886039-c9f0-47f1-845b-4ce39388efda None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:54.983821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d2886039-c9f0-47f1-845b-4ce39388efda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:55.020188 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3753/7447] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:55.023837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8da22-7b9f-4e1b-889e-da5c8f6f4dfc None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:55.051159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8da22-7b9f-4e1b-889e-da5c8f6f4dfc None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:55.066580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c53b50-6e3d-4936-a8d8-eca980741e79 tempest-DefaultDomainTestJSON-2099176732 tempest-DefaultDomainTestJSON-2099176732-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36769adab18c4d629689b3dd81009842', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62412fd0bdc04170b42bca3ffa0f141c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:55.071486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c53b50-6e3d-4936-a8d8-eca980741e79 tempest-DefaultDomainTestJSON-2099176732 tempest-DefaultDomainTestJSON-2099176732-project] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:55.072611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c53b50-6e3d-4936-a8d8-eca980741e79 tempest-DefaultDomainTestJSON-2099176732 tempest-DefaultDomainTestJSON-2099176732-project] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:55.072611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c53b50-6e3d-4936-a8d8-eca980741e79 tempest-DefaultDomainTestJSON-2099176732 tempest-DefaultDomainTestJSON-2099176732-project] PATH_INFO: `/v3/domains/default` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:55.074479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c53b50-6e3d-4936-a8d8-eca980741e79 tempest-DefaultDomainTestJSON-2099176732 tempest-DefaultDomainTestJSON-2099176732-project] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:55.077654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c53b50-6e3d-4936-a8d8-eca980741e79 tempest-DefaultDomainTestJSON-2099176732 tempest-DefaultDomainTestJSON-2099176732-project] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:55.085725 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3754/7448] 10.222.0.17 () {58 vars in 1152 bytes} [Sat Oct 1 21:15:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:55.087955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c53b50-6e3d-4936-a8d8-eca980741e79 tempest-DefaultDomainTestJSON-2099176732 tempest-DefaultDomainTestJSON-2099176732-project] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:55.094790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-82c53b50-6e3d-4936-a8d8-eca980741e79 tempest-DefaultDomainTestJSON-2099176732 tempest-DefaultDomainTestJSON-2099176732-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:55.110093 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3695/7449] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:15:55.122432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13718fa-f122-4db1-b1a8-0e347c9a908c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:55.122772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13718fa-f122-4db1-b1a8-0e347c9a908c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:55.123055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13718fa-f122-4db1-b1a8-0e347c9a908c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:55.130058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c53b50-6e3d-4936-a8d8-eca980741e79 tempest-DefaultDomainTestJSON-2099176732 tempest-DefaultDomainTestJSON-2099176732-project] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:55.146659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d13718fa-f122-4db1-b1a8-0e347c9a908c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:55.148710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d13718fa-f122-4db1-b1a8-0e347c9a908c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:55.165158 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3696/7450] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:55.167550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13718fa-f122-4db1-b1a8-0e347c9a908c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:55.167795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e7953-220c-42b1-9e9b-2e8187bd0f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:55.171235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e7953-220c-42b1-9e9b-2e8187bd0f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:55.171735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e7953-220c-42b1-9e9b-2e8187bd0f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:55.173111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e7953-220c-42b1-9e9b-2e8187bd0f97 admin admin] PATH_INFO: `/v3/users/36769adab18c4d629689b3dd81009842` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:55.175376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e7953-220c-42b1-9e9b-2e8187bd0f97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36769adab18c4d629689b3dd81009842)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:55.176827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13718fa-f122-4db1-b1a8-0e347c9a908c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:55.179823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e7953-220c-42b1-9e9b-2e8187bd0f97 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:55.182124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb7057-9aa7-4795-955c-3f618c0ee39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:55.182831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb7057-9aa7-4795-955c-3f618c0ee39e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:55.183507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb7057-9aa7-4795-955c-3f618c0ee39e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:55.183849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb7057-9aa7-4795-955c-3f618c0ee39e admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:55.184496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb7057-9aa7-4795-955c-3f618c0ee39e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:15:55.184830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb7057-9aa7-4795-955c-3f618c0ee39e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:55.188153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb7057-9aa7-4795-955c-3f618c0ee39e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:55.202183 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3697/7451] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:15:55.206925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb7057-9aa7-4795-955c-3f618c0ee39e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:55.227113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb7057-9aa7-4795-955c-3f618c0ee39e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:55.237685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37201c-800e-43c7-88b4-fec4f463e3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:55.239241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37201c-800e-43c7-88b4-fec4f463e3ae admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:55.239797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37201c-800e-43c7-88b4-fec4f463e3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:55.240737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37201c-800e-43c7-88b4-fec4f463e3ae admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:55.242299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37201c-800e-43c7-88b4-fec4f463e3ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:55.248553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37201c-800e-43c7-88b4-fec4f463e3ae admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:55.262323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:55.275009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cd3e7953-220c-42b1-9e9b-2e8187bd0f97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36769adab18c4d629689b3dd81009842'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:55.294626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cd3e7953-220c-42b1-9e9b-2e8187bd0f97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36769adab18c4d629689b3dd81009842'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:55.302963 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3698/7452] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:55 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:55.308993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37201c-800e-43c7-88b4-fec4f463e3ae admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:55.317013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cd3e7953-220c-42b1-9e9b-2e8187bd0f97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36769adab18c4d629689b3dd81009842'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:55.317963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cd3e7953-220c-42b1-9e9b-2e8187bd0f97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36769adab18c4d629689b3dd81009842'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:55.331625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cd3e7953-220c-42b1-9e9b-2e8187bd0f97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36769adab18c4d629689b3dd81009842'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:55.348718 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3755/7453] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:55 2022] DELETE /identity/v3/users/36769adab18c4d629689b3dd81009842 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:55.353441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dd561-2109-474f-a764-6bc8ef2e9e1c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:55.353876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dd561-2109-474f-a764-6bc8ef2e9e1c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:55.353989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dd561-2109-474f-a764-6bc8ef2e9e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:55.389854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-794dd561-2109-474f-a764-6bc8ef2e9e1c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:55.391131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-794dd561-2109-474f-a764-6bc8ef2e9e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:55.416408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37201c-800e-43c7-88b4-fec4f463e3ae admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:55.432963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:55.481242 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3756/7454] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:55.492369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794dd561-2109-474f-a764-6bc8ef2e9e1c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:55.493867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-794dd561-2109-474f-a764-6bc8ef2e9e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:55.496910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5332c4-65e5-4c09-8e94-67e83dbd5a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:55.498104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5332c4-65e5-4c09-8e94-67e83dbd5a94 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:55.498632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5332c4-65e5-4c09-8e94-67e83dbd5a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:55.499093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5332c4-65e5-4c09-8e94-67e83dbd5a94 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:55.500037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5332c4-65e5-4c09-8e94-67e83dbd5a94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:55.503391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5332c4-65e5-4c09-8e94-67e83dbd5a94 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:55.540168 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3699/7455] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:55 2022] POST /identity/v3/users => generated 287 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:15:55.543952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5332c4-65e5-4c09-8e94-67e83dbd5a94 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:55.545802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6c5332c4-65e5-4c09-8e94-67e83dbd5a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:55.560748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5332c4-65e5-4c09-8e94-67e83dbd5a94 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:55.567071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db66c89-f0bc-43d3-9521-2ccb38502658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:55.568441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db66c89-f0bc-43d3-9521-2ccb38502658 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:55.568801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db66c89-f0bc-43d3-9521-2ccb38502658 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:55.569154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db66c89-f0bc-43d3-9521-2ccb38502658 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:55.570012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db66c89-f0bc-43d3-9521-2ccb38502658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:55.572047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db66c89-f0bc-43d3-9521-2ccb38502658 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:55.572621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794dd561-2109-474f-a764-6bc8ef2e9e1c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:55.578309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a38236-bb92-41ff-b5d8-0c86a4791a20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:55.579171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a38236-bb92-41ff-b5d8-0c86a4791a20 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:55.579783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a38236-bb92-41ff-b5d8-0c86a4791a20 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:55.580205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a38236-bb92-41ff-b5d8-0c86a4791a20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:55.584845 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3700/7456] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:55.589704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db66c89-f0bc-43d3-9521-2ccb38502658 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:55.592001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a38236-bb92-41ff-b5d8-0c86a4791a20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:55.594006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a38236-bb92-41ff-b5d8-0c86a4791a20 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:55.602731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db66c89-f0bc-43d3-9521-2ccb38502658 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:55.609778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c6af4-056b-4624-91b3-3aa620022c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:55.609778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c6af4-056b-4624-91b3-3aa620022c11 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:55.609778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c6af4-056b-4624-91b3-3aa620022c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:55.609778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c6af4-056b-4624-91b3-3aa620022c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:55.610314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c6af4-056b-4624-91b3-3aa620022c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:55.612024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c6af4-056b-4624-91b3-3aa620022c11 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:55.626618 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3701/7457] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:55.632970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c6af4-056b-4624-91b3-3aa620022c11 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:55.637328 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3757/7458] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:15:55.656338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c6af4-056b-4624-91b3-3aa620022c11 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:55.661701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11a110-5684-4e8c-8421-97dea2a1a079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:55.662436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11a110-5684-4e8c-8421-97dea2a1a079 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:55.663197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11a110-5684-4e8c-8421-97dea2a1a079 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:55.663197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11a110-5684-4e8c-8421-97dea2a1a079 admin admin] PATH_INFO: `/v3/projects/d4d77c36520447b3bd3bf8984e9326e2/users/3066b652821f47f6909892c9b04f4236/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:55.682111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11a110-5684-4e8c-8421-97dea2a1a079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4d77c36520447b3bd3bf8984e9326e2, user_id=3066b652821f47f6909892c9b04f4236, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:55.685012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11a110-5684-4e8c-8421-97dea2a1a079 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:55.700235 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3702/7459] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:55 2022] PUT /identity/v3/projects/d4d77c36520447b3bd3bf8984e9326e2/users/3066b652821f47f6909892c9b04f4236/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:55.712765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70293ec1-bae0-4fae-ba91-a90ae1214ca4 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:55.713083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70293ec1-bae0-4fae-ba91-a90ae1214ca4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:55.713384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70293ec1-bae0-4fae-ba91-a90ae1214ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:55.748004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-70293ec1-bae0-4fae-ba91-a90ae1214ca4 None None] MFA Rules not processed for user `3066b652821f47f6909892c9b04f4236`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:55.749993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-70293ec1-bae0-4fae-ba91-a90ae1214ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:55.832825 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3758/7460] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:55.834075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11a110-5684-4e8c-8421-97dea2a1a079 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:55.886695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:55.927963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11a110-5684-4e8c-8421-97dea2a1a079 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:55.939248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3489e5-c6fe-47de-9c22-1e2d1aee2216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:55.940273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3489e5-c6fe-47de-9c22-1e2d1aee2216 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:55.940743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3489e5-c6fe-47de-9c22-1e2d1aee2216 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:55.942477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3489e5-c6fe-47de-9c22-1e2d1aee2216 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:55.943328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3489e5-c6fe-47de-9c22-1e2d1aee2216 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:55.945487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3489e5-c6fe-47de-9c22-1e2d1aee2216 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:55.989124 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3703/7461] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:55 2022] POST /identity/v3/projects => generated 375 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:55.992823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70293ec1-bae0-4fae-ba91-a90ae1214ca4 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:56.010070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70293ec1-bae0-4fae-ba91-a90ae1214ca4 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:56.114008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbd297-1fb9-4f30-9abb-849cb9f1a561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:56.115515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbd297-1fb9-4f30-9abb-849cb9f1a561 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:56.116948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbd297-1fb9-4f30-9abb-849cb9f1a561 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:56.117611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbd297-1fb9-4f30-9abb-849cb9f1a561 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:56.119546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbd297-1fb9-4f30-9abb-849cb9f1a561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:56.124820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbd297-1fb9-4f30-9abb-849cb9f1a561 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:56.166767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:56.178101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3489e5-c6fe-47de-9c22-1e2d1aee2216 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:56.181732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9d3489e5-c6fe-47de-9c22-1e2d1aee2216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:56.202556 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3759/7462] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:55 2022] POST /identity/v3/users => generated 294 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:56.205305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbd297-1fb9-4f30-9abb-849cb9f1a561 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:56.226610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3489e5-c6fe-47de-9c22-1e2d1aee2216 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:56.236600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbd297-1fb9-4f30-9abb-849cb9f1a561 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:56.243579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7b55b-fc70-4487-bd83-5b2ceedfacf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:56.244199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7b55b-fc70-4487-bd83-5b2ceedfacf2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:56.244481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7b55b-fc70-4487-bd83-5b2ceedfacf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:56.244755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7b55b-fc70-4487-bd83-5b2ceedfacf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:56.245390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e7b55b-fc70-4487-bd83-5b2ceedfacf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:56.247177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e7b55b-fc70-4487-bd83-5b2ceedfacf2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:56.247296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c41c15-9cc6-456e-b3cc-fdfbcfad35d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:56.247704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c41c15-9cc6-456e-b3cc-fdfbcfad35d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:56.247989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c41c15-9cc6-456e-b3cc-fdfbcfad35d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:56.250763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c41c15-9cc6-456e-b3cc-fdfbcfad35d0 admin admin] PATH_INFO: `/v3/projects/62412fd0bdc04170b42bca3ffa0f141c` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:56.252396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c41c15-9cc6-456e-b3cc-fdfbcfad35d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62412fd0bdc04170b42bca3ffa0f141c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:56.256111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c41c15-9cc6-456e-b3cc-fdfbcfad35d0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:56.261751 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3760/7463] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:56.263665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7b55b-fc70-4487-bd83-5b2ceedfacf2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:56.280422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7b55b-fc70-4487-bd83-5b2ceedfacf2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:56.285541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e019e6-c81b-4841-98b1-6da58177b4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:56.286245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e019e6-c81b-4841-98b1-6da58177b4e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:56.286571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e019e6-c81b-4841-98b1-6da58177b4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:56.286847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e019e6-c81b-4841-98b1-6da58177b4e8 admin admin] PATH_INFO: `/v3/projects/4d383d8983284b36a205307ea45425ec/users/4c6f15b326704a2999b906a49cf2b514/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:56.306029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e019e6-c81b-4841-98b1-6da58177b4e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d383d8983284b36a205307ea45425ec, user_id=4c6f15b326704a2999b906a49cf2b514, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:56.309066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e019e6-c81b-4841-98b1-6da58177b4e8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:56.326988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-68c41c15-9cc6-456e-b3cc-fdfbcfad35d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62412fd0bdc04170b42bca3ffa0f141c', 'request_id': 'req-68c41c15-9cc6-456e-b3cc-fdfbcfad35d0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:56.345127 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3761/7464] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:56 2022] PUT /identity/v3/projects/4d383d8983284b36a205307ea45425ec/users/4c6f15b326704a2999b906a49cf2b514/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:56.345328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-68c41c15-9cc6-456e-b3cc-fdfbcfad35d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62412fd0bdc04170b42bca3ffa0f141c', 'request_id': 'req-68c41c15-9cc6-456e-b3cc-fdfbcfad35d0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:56.346991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e019e6-c81b-4841-98b1-6da58177b4e8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:56.352500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-68c41c15-9cc6-456e-b3cc-fdfbcfad35d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62412fd0bdc04170b42bca3ffa0f141c', 'request_id': 'req-68c41c15-9cc6-456e-b3cc-fdfbcfad35d0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:56.373018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-68c41c15-9cc6-456e-b3cc-fdfbcfad35d0 admin admin] The token cache is being invalidate because project 62412fd0bdc04170b42bca3ffa0f141c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:15:56.373018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-68c41c15-9cc6-456e-b3cc-fdfbcfad35d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:56.373757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-68c41c15-9cc6-456e-b3cc-fdfbcfad35d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:56.376658 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3704/7465] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:15:56 2022] DELETE /identity/v3/projects/62412fd0bdc04170b42bca3ffa0f141c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:15:56.384164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4402bf-d090-4bbb-8948-084a7b1e07fc None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:56.385053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4402bf-d090-4bbb-8948-084a7b1e07fc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:56.385586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4402bf-d090-4bbb-8948-084a7b1e07fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:56.407143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ed4402bf-d090-4bbb-8948-084a7b1e07fc None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:56.409085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ed4402bf-d090-4bbb-8948-084a7b1e07fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:56.433705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e019e6-c81b-4841-98b1-6da58177b4e8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:56.439918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a493b1-7932-48c4-80ea-4b2041a3ab3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:56.440750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a493b1-7932-48c4-80ea-4b2041a3ab3f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:56.441668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a493b1-7932-48c4-80ea-4b2041a3ab3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:56.442011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a493b1-7932-48c4-80ea-4b2041a3ab3f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:56.442808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a493b1-7932-48c4-80ea-4b2041a3ab3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:56.444533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a493b1-7932-48c4-80ea-4b2041a3ab3f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:56.462330 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3762/7466] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:56.464321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a493b1-7932-48c4-80ea-4b2041a3ab3f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:56.467002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-38a493b1-7932-48c4-80ea-4b2041a3ab3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:56.479191 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3705/7467] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:15:56.481106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a493b1-7932-48c4-80ea-4b2041a3ab3f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:56.486318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439e901-e6fe-43ec-8549-d5ab25874564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:56.486781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc8517-de61-4c01-88a2-1d1f4e73b513 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:56.487023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439e901-e6fe-43ec-8549-d5ab25874564 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:56.487156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc8517-de61-4c01-88a2-1d1f4e73b513 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:56.487311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439e901-e6fe-43ec-8549-d5ab25874564 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:56.487412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc8517-de61-4c01-88a2-1d1f4e73b513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:56.487579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439e901-e6fe-43ec-8549-d5ab25874564 admin admin] PATH_INFO: `/v3/projects/4d383d8983284b36a205307ea45425ec/users/4c6f15b326704a2999b906a49cf2b514/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:56.490358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439e901-e6fe-43ec-8549-d5ab25874564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d383d8983284b36a205307ea45425ec, user_id=4c6f15b326704a2999b906a49cf2b514, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:56.493033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439e901-e6fe-43ec-8549-d5ab25874564 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:56.506059 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3763/7468] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:56 2022] PUT /identity/v3/projects/4d383d8983284b36a205307ea45425ec/users/4c6f15b326704a2999b906a49cf2b514/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:56.508864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-32bc8517-de61-4c01-88a2-1d1f4e73b513 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:56.510065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-32bc8517-de61-4c01-88a2-1d1f4e73b513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:56.518116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc1e55-156d-4860-8883-24bf7f8b593b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:56.518116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc1e55-156d-4860-8883-24bf7f8b593b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:56.518313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc1e55-156d-4860-8883-24bf7f8b593b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:56.570522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:56.572825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-05fc1e55-156d-4860-8883-24bf7f8b593b None None] MFA Rules not processed for user `4c6f15b326704a2999b906a49cf2b514`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:56.574195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-05fc1e55-156d-4860-8883-24bf7f8b593b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:56.601183 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3706/7469] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:15:56.605170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc8517-de61-4c01-88a2-1d1f4e73b513 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:56.615279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc8517-de61-4c01-88a2-1d1f4e73b513 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:56.622050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ab6e2-65ef-4be4-b993-cabd9b2519f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:56.622930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ab6e2-65ef-4be4-b993-cabd9b2519f0 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:56.623215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ab6e2-65ef-4be4-b993-cabd9b2519f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:56.623489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ab6e2-65ef-4be4-b993-cabd9b2519f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:56.625004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ab6e2-65ef-4be4-b993-cabd9b2519f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:15:56.625315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ab6e2-65ef-4be4-b993-cabd9b2519f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:56.627013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ab6e2-65ef-4be4-b993-cabd9b2519f0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:56.645035 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3707/7470] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:15:56.646083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ab6e2-65ef-4be4-b993-cabd9b2519f0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:56.657572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ab6e2-65ef-4be4-b993-cabd9b2519f0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:56.663656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46393e8-b272-4b2b-ab6a-a39f9136fdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:56.664347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46393e8-b272-4b2b-ab6a-a39f9136fdc9 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:56.664622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46393e8-b272-4b2b-ab6a-a39f9136fdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:56.664893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46393e8-b272-4b2b-ab6a-a39f9136fdc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:56.665510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46393e8-b272-4b2b-ab6a-a39f9136fdc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:56.667504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46393e8-b272-4b2b-ab6a-a39f9136fdc9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:56.677289 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3764/7471] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:56.681276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc1e55-156d-4860-8883-24bf7f8b593b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:56.685170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-05fc1e55-156d-4860-8883-24bf7f8b593b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:56.720935 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3708/7472] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:56 2022] POST /identity/v3/projects => generated 397 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:56.725638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46393e8-b272-4b2b-ab6a-a39f9136fdc9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:56.729200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:56.779175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc1e55-156d-4860-8883-24bf7f8b593b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:56.827901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46393e8-b272-4b2b-ab6a-a39f9136fdc9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:56.836230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093ca36-6e66-4c06-a9e8-94353180f909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:56.837000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093ca36-6e66-4c06-a9e8-94353180f909 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:56.837288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093ca36-6e66-4c06-a9e8-94353180f909 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:56.837569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093ca36-6e66-4c06-a9e8-94353180f909 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:56.838209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093ca36-6e66-4c06-a9e8-94353180f909 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:56.840221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093ca36-6e66-4c06-a9e8-94353180f909 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:56.873317 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3709/7473] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:56 2022] POST /identity/v3/users => generated 305 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:56.874958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a549b6-bb36-4783-af6d-dbe9097a1b27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:56.875278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a549b6-bb36-4783-af6d-dbe9097a1b27 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:56.875578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a549b6-bb36-4783-af6d-dbe9097a1b27 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:56.875871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a549b6-bb36-4783-af6d-dbe9097a1b27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:56.880098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093ca36-6e66-4c06-a9e8-94353180f909 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:56.883454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c3a549b6-bb36-4783-af6d-dbe9097a1b27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:56.892009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093ca36-6e66-4c06-a9e8-94353180f909 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:56.914270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882de926-0ee9-4be4-8761-7ac5855246c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:56.915132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882de926-0ee9-4be4-8761-7ac5855246c6 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:56.915412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882de926-0ee9-4be4-8761-7ac5855246c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:56.915777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882de926-0ee9-4be4-8761-7ac5855246c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:56.916422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882de926-0ee9-4be4-8761-7ac5855246c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:56.918099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882de926-0ee9-4be4-8761-7ac5855246c6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:56.934415 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3710/7474] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:56.942914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882de926-0ee9-4be4-8761-7ac5855246c6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:56.963927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882de926-0ee9-4be4-8761-7ac5855246c6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:56.970215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd02177-bf4d-4ce3-a616-03ec393086d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:56.971535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd02177-bf4d-4ce3-a616-03ec393086d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:56.972084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd02177-bf4d-4ce3-a616-03ec393086d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:56.973411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd02177-bf4d-4ce3-a616-03ec393086d4 admin admin] PATH_INFO: `/v3/projects/2ff4104a8f51480eb4e9b12dcdcb1e67/users/9cb5d67818414ed2be4cd8034868fd75/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:56.990813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a549b6-bb36-4783-af6d-dbe9097a1b27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:56.993032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a549b6-bb36-4783-af6d-dbe9097a1b27 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:56.999263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd02177-bf4d-4ce3-a616-03ec393086d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ff4104a8f51480eb4e9b12dcdcb1e67, user_id=9cb5d67818414ed2be4cd8034868fd75, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:57.006986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd02177-bf4d-4ce3-a616-03ec393086d4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:57.008067 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3765/7475] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:15:57.046346 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3711/7476] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:56 2022] PUT /identity/v3/projects/2ff4104a8f51480eb4e9b12dcdcb1e67/users/9cb5d67818414ed2be4cd8034868fd75/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:57.049210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a549b6-bb36-4783-af6d-dbe9097a1b27 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:57.068166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a549b6-bb36-4783-af6d-dbe9097a1b27 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:57.180729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230a3ca-ab7d-4b2a-b576-40799ba3eeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:57.183154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230a3ca-ab7d-4b2a-b576-40799ba3eeae admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:57.183624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230a3ca-ab7d-4b2a-b576-40799ba3eeae admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:57.184087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230a3ca-ab7d-4b2a-b576-40799ba3eeae admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:57.187060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230a3ca-ab7d-4b2a-b576-40799ba3eeae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:57.189281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230a3ca-ab7d-4b2a-b576-40799ba3eeae admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:57.214811 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3766/7477] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:57.218480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd02177-bf4d-4ce3-a616-03ec393086d4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:57.232892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd02177-bf4d-4ce3-a616-03ec393086d4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:57.249232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44660e-a823-45e9-b1ec-e61831761288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:57.250292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44660e-a823-45e9-b1ec-e61831761288 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:57.250837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44660e-a823-45e9-b1ec-e61831761288 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:57.251326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44660e-a823-45e9-b1ec-e61831761288 admin admin] PATH_INFO: `/v3/projects/2ff4104a8f51480eb4e9b12dcdcb1e67/users/9cb5d67818414ed2be4cd8034868fd75/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:57.253668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44660e-a823-45e9-b1ec-e61831761288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ff4104a8f51480eb4e9b12dcdcb1e67, user_id=9cb5d67818414ed2be4cd8034868fd75, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:57.256937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44660e-a823-45e9-b1ec-e61831761288 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:57.271444 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3712/7478] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:57 2022] PUT /identity/v3/projects/2ff4104a8f51480eb4e9b12dcdcb1e67/users/9cb5d67818414ed2be4cd8034868fd75/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:57.283272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55e6d9-c4e5-42fc-8159-15d9239ef7b6 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:57.283594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55e6d9-c4e5-42fc-8159-15d9239ef7b6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:57.283837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55e6d9-c4e5-42fc-8159-15d9239ef7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:57.370149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1b55e6d9-c4e5-42fc-8159-15d9239ef7b6 None None] MFA Rules not processed for user `9cb5d67818414ed2be4cd8034868fd75`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:57.371702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1b55e6d9-c4e5-42fc-8159-15d9239ef7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:57.409720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:57.478472 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3767/7479] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:57 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:57.483316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44660e-a823-45e9-b1ec-e61831761288 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:57.586272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:57.610812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44660e-a823-45e9-b1ec-e61831761288 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:57.621516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090648f2-78e7-47ad-8362-e8e6cd549321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:57.622226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090648f2-78e7-47ad-8362-e8e6cd549321 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:57.622876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090648f2-78e7-47ad-8362-e8e6cd549321 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:57.623167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090648f2-78e7-47ad-8362-e8e6cd549321 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:57.624109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090648f2-78e7-47ad-8362-e8e6cd549321 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:57.628046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090648f2-78e7-47ad-8362-e8e6cd549321 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:57.675260 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3713/7480] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:15:57 2022] POST /identity/v3/projects => generated 399 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:57.679158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55e6d9-c4e5-42fc-8159-15d9239ef7b6 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:57.696671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55e6d9-c4e5-42fc-8159-15d9239ef7b6 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:57.788241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5000d-3a67-4fea-a542-19a21d1583d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:57.788974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5000d-3a67-4fea-a542-19a21d1583d4 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:57.789260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5000d-3a67-4fea-a542-19a21d1583d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:57.789925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5000d-3a67-4fea-a542-19a21d1583d4 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:57.790574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5000d-3a67-4fea-a542-19a21d1583d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:57.798125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5000d-3a67-4fea-a542-19a21d1583d4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:57.836965 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3768/7481] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:15:57 2022] POST /identity/v3/users => generated 300 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:15:57.838288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090648f2-78e7-47ad-8362-e8e6cd549321 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:57.853760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090648f2-78e7-47ad-8362-e8e6cd549321 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:57.860114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c11b51-8a2a-4438-ab82-af0e4cd5938c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:57.860499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c11b51-8a2a-4438-ab82-af0e4cd5938c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:57.860834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c11b51-8a2a-4438-ab82-af0e4cd5938c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:57.861116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c11b51-8a2a-4438-ab82-af0e4cd5938c admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:57.861928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c11b51-8a2a-4438-ab82-af0e4cd5938c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:57.863716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c11b51-8a2a-4438-ab82-af0e4cd5938c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:57.878612 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3714/7482] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:57.879847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5000d-3a67-4fea-a542-19a21d1583d4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:57.898060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5000d-3a67-4fea-a542-19a21d1583d4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:57.904442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da1ad5-5798-47b1-8f03-468482e3b1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:57.905594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da1ad5-5798-47b1-8f03-468482e3b1bb admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:57.905594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da1ad5-5798-47b1-8f03-468482e3b1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:57.905863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da1ad5-5798-47b1-8f03-468482e3b1bb admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:57.906714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da1ad5-5798-47b1-8f03-468482e3b1bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:57.908640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da1ad5-5798-47b1-8f03-468482e3b1bb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:57.924922 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3769/7483] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:15:57.928978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c11b51-8a2a-4438-ab82-af0e4cd5938c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:57.953605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c11b51-8a2a-4438-ab82-af0e4cd5938c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:57.967600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afefb72-8d59-42a7-a1fc-b94b61c4ab07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:57.968590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afefb72-8d59-42a7-a1fc-b94b61c4ab07 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:57.969044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afefb72-8d59-42a7-a1fc-b94b61c4ab07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:57.969488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afefb72-8d59-42a7-a1fc-b94b61c4ab07 admin admin] PATH_INFO: `/v3/projects/fa36b030a2eb4340ae48565977e7f9fa/users/7f4775a80cc148abac5a3ef4e19d9c33/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:58.009945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afefb72-8d59-42a7-a1fc-b94b61c4ab07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa36b030a2eb4340ae48565977e7f9fa, user_id=7f4775a80cc148abac5a3ef4e19d9c33, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:58.013318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afefb72-8d59-42a7-a1fc-b94b61c4ab07 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:58.070022 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3715/7484] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:15:57 2022] PUT /identity/v3/projects/fa36b030a2eb4340ae48565977e7f9fa/users/7f4775a80cc148abac5a3ef4e19d9c33/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:58.072520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581b7a8-4be8-4e22-882d-f21c1514d243 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:58.072987 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581b7a8-4be8-4e22-882d-f21c1514d243 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:58.073083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581b7a8-4be8-4e22-882d-f21c1514d243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:58.113842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9581b7a8-4be8-4e22-882d-f21c1514d243 None None] MFA Rules not processed for user `7f4775a80cc148abac5a3ef4e19d9c33`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:58.114922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9581b7a8-4be8-4e22-882d-f21c1514d243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:58.248408 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3770/7485] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:58 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:58.261802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afefb72-8d59-42a7-a1fc-b94b61c4ab07 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:58.263829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2afefb72-8d59-42a7-a1fc-b94b61c4ab07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:58.376906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afefb72-8d59-42a7-a1fc-b94b61c4ab07 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:58.390565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00faa7e4-6b00-4cf7-bdba-5fda1cb001ab service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:58.391232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00faa7e4-6b00-4cf7-bdba-5fda1cb001ab service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:58.391482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00faa7e4-6b00-4cf7-bdba-5fda1cb001ab service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:58.391761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00faa7e4-6b00-4cf7-bdba-5fda1cb001ab service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:58.559664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00faa7e4-6b00-4cf7-bdba-5fda1cb001ab service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:58.565716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00faa7e4-6b00-4cf7-bdba-5fda1cb001ab service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:58.581469 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3716/7486] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:58 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:15:59.313445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581b7a8-4be8-4e22-882d-f21c1514d243 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:59.367247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:59.403994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581b7a8-4be8-4e22-882d-f21c1514d243 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:59.408978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c41a9-abf5-42e0-823a-4cdce494ce1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:59.409622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c41a9-abf5-42e0-823a-4cdce494ce1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:59.409934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c41a9-abf5-42e0-823a-4cdce494ce1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:59.410206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c41a9-abf5-42e0-823a-4cdce494ce1e admin admin] PATH_INFO: `/v3/users/9cb5d67818414ed2be4cd8034868fd75` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:59.411372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2c41a9-abf5-42e0-823a-4cdce494ce1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cb5d67818414ed2be4cd8034868fd75)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:59.413286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2c41a9-abf5-42e0-823a-4cdce494ce1e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:59.501259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-de2c41a9-abf5-42e0-823a-4cdce494ce1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cb5d67818414ed2be4cd8034868fd75'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:59.509967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-de2c41a9-abf5-42e0-823a-4cdce494ce1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cb5d67818414ed2be4cd8034868fd75'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:59.524199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-de2c41a9-abf5-42e0-823a-4cdce494ce1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cb5d67818414ed2be4cd8034868fd75'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:59.525159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-de2c41a9-abf5-42e0-823a-4cdce494ce1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cb5d67818414ed2be4cd8034868fd75'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:59.551197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-de2c41a9-abf5-42e0-823a-4cdce494ce1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cb5d67818414ed2be4cd8034868fd75'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:15:59.555283 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3771/7487] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:15:59 2022] DELETE /identity/v3/users/9cb5d67818414ed2be4cd8034868fd75 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:15:59.559436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760fe77-ff5f-4a6b-8c9f-77a3a78340e5 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:59.559897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760fe77-ff5f-4a6b-8c9f-77a3a78340e5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:59.560086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760fe77-ff5f-4a6b-8c9f-77a3a78340e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:59.596502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6760fe77-ff5f-4a6b-8c9f-77a3a78340e5 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:15:59.599081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6760fe77-ff5f-4a6b-8c9f-77a3a78340e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:59.710007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:59.730727 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3717/7488] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:15:59.770858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c41a9-abf5-42e0-823a-4cdce494ce1e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:15:59.773838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-de2c41a9-abf5-42e0-823a-4cdce494ce1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:15:59.862163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c41a9-abf5-42e0-823a-4cdce494ce1e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:15:59.867545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23623d86-6e33-4be4-a8a8-60118b44df73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:15:59.868182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23623d86-6e33-4be4-a8a8-60118b44df73 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:15:59.868466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23623d86-6e33-4be4-a8a8-60118b44df73 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:15:59.868784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23623d86-6e33-4be4-a8a8-60118b44df73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:15:59.900834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23623d86-6e33-4be4-a8a8-60118b44df73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:15:59.902286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23623d86-6e33-4be4-a8a8-60118b44df73 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:15:59.914960 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3772/7489] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:16:00.535331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760fe77-ff5f-4a6b-8c9f-77a3a78340e5 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:00.562157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760fe77-ff5f-4a6b-8c9f-77a3a78340e5 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:00.564475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23623d86-6e33-4be4-a8a8-60118b44df73 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:00.567462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-23623d86-6e33-4be4-a8a8-60118b44df73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:00.575377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72016b3-b69a-4cf3-9956-e78c9b7e0293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:00.576307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72016b3-b69a-4cf3-9956-e78c9b7e0293 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:00.576821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72016b3-b69a-4cf3-9956-e78c9b7e0293 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:00.577321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72016b3-b69a-4cf3-9956-e78c9b7e0293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:00.586172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b72016b3-b69a-4cf3-9956-e78c9b7e0293 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:00.599819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23623d86-6e33-4be4-a8a8-60118b44df73 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:00.615535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed87e42-6200-4540-909a-0e8deabba354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:00.616135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed87e42-6200-4540-909a-0e8deabba354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:00.616729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed87e42-6200-4540-909a-0e8deabba354 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:00.617036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed87e42-6200-4540-909a-0e8deabba354 admin admin] PATH_INFO: `/v3/projects/2ff4104a8f51480eb4e9b12dcdcb1e67` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:00.619824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed87e42-6200-4540-909a-0e8deabba354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ff4104a8f51480eb4e9b12dcdcb1e67)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:00.623089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed87e42-6200-4540-909a-0e8deabba354 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:00.698804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:00.704979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7ed87e42-6200-4540-909a-0e8deabba354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ff4104a8f51480eb4e9b12dcdcb1e67', 'request_id': 'req-7ed87e42-6200-4540-909a-0e8deabba354'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:00.726234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7ed87e42-6200-4540-909a-0e8deabba354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ff4104a8f51480eb4e9b12dcdcb1e67', 'request_id': 'req-7ed87e42-6200-4540-909a-0e8deabba354'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:00.739580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7ed87e42-6200-4540-909a-0e8deabba354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ff4104a8f51480eb4e9b12dcdcb1e67', 'request_id': 'req-7ed87e42-6200-4540-909a-0e8deabba354'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:00.773485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7ed87e42-6200-4540-909a-0e8deabba354 admin admin] The token cache is being invalidate because project 2ff4104a8f51480eb4e9b12dcdcb1e67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:16:00.774168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7ed87e42-6200-4540-909a-0e8deabba354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:00.777086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7ed87e42-6200-4540-909a-0e8deabba354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:00.779602 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3773/7490] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:16:00 2022] DELETE /identity/v3/projects/2ff4104a8f51480eb4e9b12dcdcb1e67 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:00.783392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed87e42-6200-4540-909a-0e8deabba354 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:00.786295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7ed87e42-6200-4540-909a-0e8deabba354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:00.790838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72016b3-b69a-4cf3-9956-e78c9b7e0293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:00.795329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72016b3-b69a-4cf3-9956-e78c9b7e0293 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:00.818421 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3718/7491] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:16:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:16:00.885645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed87e42-6200-4540-909a-0e8deabba354 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:00.893784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b871092-8c13-44f3-a43a-f96283d99964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:00.895286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b871092-8c13-44f3-a43a-f96283d99964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:00.895676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b871092-8c13-44f3-a43a-f96283d99964 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:00.896074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b871092-8c13-44f3-a43a-f96283d99964 admin admin] PATH_INFO: `/v3/users/7f4775a80cc148abac5a3ef4e19d9c33` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:00.898207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b871092-8c13-44f3-a43a-f96283d99964 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f4775a80cc148abac5a3ef4e19d9c33)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:00.900640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b871092-8c13-44f3-a43a-f96283d99964 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:00.983405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1b871092-8c13-44f3-a43a-f96283d99964 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f4775a80cc148abac5a3ef4e19d9c33'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:00.994467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1b871092-8c13-44f3-a43a-f96283d99964 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f4775a80cc148abac5a3ef4e19d9c33'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:01.014070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1b871092-8c13-44f3-a43a-f96283d99964 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f4775a80cc148abac5a3ef4e19d9c33'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:01.015545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1b871092-8c13-44f3-a43a-f96283d99964 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f4775a80cc148abac5a3ef4e19d9c33'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:01.030635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1b871092-8c13-44f3-a43a-f96283d99964 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f4775a80cc148abac5a3ef4e19d9c33'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:01.052366 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3774/7492] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:16:00 2022] DELETE /identity/v3/users/7f4775a80cc148abac5a3ef4e19d9c33 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:01.669901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72016b3-b69a-4cf3-9956-e78c9b7e0293 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:01.693245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72016b3-b69a-4cf3-9956-e78c9b7e0293 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:01.795354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eeffa2-14cc-48ed-bc89-595ca364adf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:01.796255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eeffa2-14cc-48ed-bc89-595ca364adf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:01.796711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eeffa2-14cc-48ed-bc89-595ca364adf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:01.797152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eeffa2-14cc-48ed-bc89-595ca364adf3 admin admin] PATH_INFO: `/v3/projects/fa36b030a2eb4340ae48565977e7f9fa` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:01.800889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eeffa2-14cc-48ed-bc89-595ca364adf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa36b030a2eb4340ae48565977e7f9fa)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:01.803052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eeffa2-14cc-48ed-bc89-595ca364adf3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:01.868273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-09eeffa2-14cc-48ed-bc89-595ca364adf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa36b030a2eb4340ae48565977e7f9fa', 'request_id': 'req-09eeffa2-14cc-48ed-bc89-595ca364adf3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:01.879008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-09eeffa2-14cc-48ed-bc89-595ca364adf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa36b030a2eb4340ae48565977e7f9fa', 'request_id': 'req-09eeffa2-14cc-48ed-bc89-595ca364adf3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:01.888716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-09eeffa2-14cc-48ed-bc89-595ca364adf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa36b030a2eb4340ae48565977e7f9fa', 'request_id': 'req-09eeffa2-14cc-48ed-bc89-595ca364adf3'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:01.902889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-09eeffa2-14cc-48ed-bc89-595ca364adf3 admin admin] The token cache is being invalidate because project fa36b030a2eb4340ae48565977e7f9fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:16:01.903185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-09eeffa2-14cc-48ed-bc89-595ca364adf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:01.904004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-09eeffa2-14cc-48ed-bc89-595ca364adf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:01.905791 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3719/7493] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:16:01 2022] DELETE /identity/v3/projects/fa36b030a2eb4340ae48565977e7f9fa => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:01.911113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f23fb-f791-4154-ba9f-52b53f31cc5f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:01.911628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f23fb-f791-4154-ba9f-52b53f31cc5f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:01.911741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f23fb-f791-4154-ba9f-52b53f31cc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:01.934135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-bc7f23fb-f791-4154-ba9f-52b53f31cc5f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:01.935509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bc7f23fb-f791-4154-ba9f-52b53f31cc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:01.990667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:02.014640 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3775/7494] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:02.021885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9018f49-d2ee-45d7-93c1-56b86550a0e0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:02.022479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9018f49-d2ee-45d7-93c1-56b86550a0e0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:02.023075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9018f49-d2ee-45d7-93c1-56b86550a0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:02.051281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c9018f49-d2ee-45d7-93c1-56b86550a0e0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:02.052624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c9018f49-d2ee-45d7-93c1-56b86550a0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:02.061851 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3720/7495] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:02.066725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f23fb-f791-4154-ba9f-52b53f31cc5f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:02.080975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f23fb-f791-4154-ba9f-52b53f31cc5f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:02.086860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891a24f-8683-4972-a56e-0c430f739b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:02.087683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891a24f-8683-4972-a56e-0c430f739b7a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:02.088218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891a24f-8683-4972-a56e-0c430f739b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:02.088628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891a24f-8683-4972-a56e-0c430f739b7a admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:02.089414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891a24f-8683-4972-a56e-0c430f739b7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:16:02.089932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891a24f-8683-4972-a56e-0c430f739b7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:02.091735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891a24f-8683-4972-a56e-0c430f739b7a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:02.108911 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3776/7496] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:16:02.109745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9018f49-d2ee-45d7-93c1-56b86550a0e0 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:02.121870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9018f49-d2ee-45d7-93c1-56b86550a0e0 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:02.127064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0205ffb-d0a9-4c48-a50d-2f7699079157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:02.127707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0205ffb-d0a9-4c48-a50d-2f7699079157 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:02.127989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0205ffb-d0a9-4c48-a50d-2f7699079157 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:02.128261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0205ffb-d0a9-4c48-a50d-2f7699079157 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:02.129651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0205ffb-d0a9-4c48-a50d-2f7699079157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:02.131596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0205ffb-d0a9-4c48-a50d-2f7699079157 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:02.144666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:02.169657 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3721/7497] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:16:02 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:16:02.174085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891a24f-8683-4972-a56e-0c430f739b7a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:02.186048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891a24f-8683-4972-a56e-0c430f739b7a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:02.292493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400f6ea-9e70-46bd-a2ae-54e33da28f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:02.293275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400f6ea-9e70-46bd-a2ae-54e33da28f74 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:02.293557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400f6ea-9e70-46bd-a2ae-54e33da28f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:02.293828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400f6ea-9e70-46bd-a2ae-54e33da28f74 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:02.294475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400f6ea-9e70-46bd-a2ae-54e33da28f74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:02.301552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400f6ea-9e70-46bd-a2ae-54e33da28f74 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:02.380850 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3777/7498] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:16:02 2022] POST /identity/v3/users => generated 283 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:16:02.385091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0205ffb-d0a9-4c48-a50d-2f7699079157 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:02.416901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0205ffb-d0a9-4c48-a50d-2f7699079157 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:02.441709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998decb-fc15-4396-9ee0-bfc6e1c06a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:02.442350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998decb-fc15-4396-9ee0-bfc6e1c06a2e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:02.442654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998decb-fc15-4396-9ee0-bfc6e1c06a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:02.442928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998decb-fc15-4396-9ee0-bfc6e1c06a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:02.443730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998decb-fc15-4396-9ee0-bfc6e1c06a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:02.445448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998decb-fc15-4396-9ee0-bfc6e1c06a2e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:02.466239 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3722/7499] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:16:02.471197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400f6ea-9e70-46bd-a2ae-54e33da28f74 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:02.487282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400f6ea-9e70-46bd-a2ae-54e33da28f74 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:02.501025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdc31b-b5a0-4814-97c4-753a196eacee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:02.501824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdc31b-b5a0-4814-97c4-753a196eacee admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:02.502181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdc31b-b5a0-4814-97c4-753a196eacee admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:02.502556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdc31b-b5a0-4814-97c4-753a196eacee admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:02.505358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdc31b-b5a0-4814-97c4-753a196eacee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:02.507152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdc31b-b5a0-4814-97c4-753a196eacee admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:02.532642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998decb-fc15-4396-9ee0-bfc6e1c06a2e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:02.532857 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3778/7500] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:16:02.557976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998decb-fc15-4396-9ee0-bfc6e1c06a2e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:02.563466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7be25-3a52-486a-af50-dddb3d2cff41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:02.564147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7be25-3a52-486a-af50-dddb3d2cff41 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:02.564427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7be25-3a52-486a-af50-dddb3d2cff41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:02.564745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7be25-3a52-486a-af50-dddb3d2cff41 admin admin] PATH_INFO: `/v3/projects/57e4fa7838ff4550baf3944117646774/users/a887e636cafb4c419af658b3320187ee/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:02.591382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7be25-3a52-486a-af50-dddb3d2cff41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57e4fa7838ff4550baf3944117646774, user_id=a887e636cafb4c419af658b3320187ee, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:02.593994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7be25-3a52-486a-af50-dddb3d2cff41 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:02.658855 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3723/7501] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:16:02 2022] PUT /identity/v3/projects/57e4fa7838ff4550baf3944117646774/users/a887e636cafb4c419af658b3320187ee/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:16:02.662574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0aba1-fa90-4882-8ad1-e404b7e2eb05 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:02.662877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0aba1-fa90-4882-8ad1-e404b7e2eb05 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:02.663160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0aba1-fa90-4882-8ad1-e404b7e2eb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:02.735190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-92e0aba1-fa90-4882-8ad1-e404b7e2eb05 None None] MFA Rules not processed for user `a887e636cafb4c419af658b3320187ee`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:02.741935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-92e0aba1-fa90-4882-8ad1-e404b7e2eb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:02.858431 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3779/7502] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:02.877888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7be25-3a52-486a-af50-dddb3d2cff41 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:02.880553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fec7be25-3a52-486a-af50-dddb3d2cff41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:02.984732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7be25-3a52-486a-af50-dddb3d2cff41 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:03.004624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a500f62-f685-4c12-8a1a-e6451896b738 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:03.005326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a500f62-f685-4c12-8a1a-e6451896b738 service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:03.005678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a500f62-f685-4c12-8a1a-e6451896b738 service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:03.006030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a500f62-f685-4c12-8a1a-e6451896b738 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:03.031329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a500f62-f685-4c12-8a1a-e6451896b738 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:03.033827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a500f62-f685-4c12-8a1a-e6451896b738 service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:03.060696 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3724/7503] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:16:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:16:03.728632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0aba1-fa90-4882-8ad1-e404b7e2eb05 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:03.813794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:03.848636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0aba1-fa90-4882-8ad1-e404b7e2eb05 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:03.855859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a8009-466b-4051-bc98-d5a7ed6fba9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:03.857216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a8009-466b-4051-bc98-d5a7ed6fba9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:03.857216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a8009-466b-4051-bc98-d5a7ed6fba9d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:03.857216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a8009-466b-4051-bc98-d5a7ed6fba9d admin admin] PATH_INFO: `/v3/users/a887e636cafb4c419af658b3320187ee` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:03.858666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a8009-466b-4051-bc98-d5a7ed6fba9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a887e636cafb4c419af658b3320187ee)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:03.861601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a8009-466b-4051-bc98-d5a7ed6fba9d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:03.952943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e14a8009-466b-4051-bc98-d5a7ed6fba9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a887e636cafb4c419af658b3320187ee'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:03.968444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e14a8009-466b-4051-bc98-d5a7ed6fba9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a887e636cafb4c419af658b3320187ee'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:03.992017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e14a8009-466b-4051-bc98-d5a7ed6fba9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a887e636cafb4c419af658b3320187ee'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:03.993822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e14a8009-466b-4051-bc98-d5a7ed6fba9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a887e636cafb4c419af658b3320187ee'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:04.004890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e14a8009-466b-4051-bc98-d5a7ed6fba9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a887e636cafb4c419af658b3320187ee'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:04.019436 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3780/7504] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:16:03 2022] DELETE /identity/v3/users/a887e636cafb4c419af658b3320187ee => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:04.020113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d2fa0-5fea-412a-ad19-f5921c7750b2 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:04.020581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d2fa0-5fea-412a-ad19-f5921c7750b2 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:04.020687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d2fa0-5fea-412a-ad19-f5921c7750b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:04.057625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-be8d2fa0-5fea-412a-ad19-f5921c7750b2 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:04.061068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-be8d2fa0-5fea-412a-ad19-f5921c7750b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:04.205077 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3725/7505] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:04.230829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a8009-466b-4051-bc98-d5a7ed6fba9d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:04.234384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e14a8009-466b-4051-bc98-d5a7ed6fba9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:04.402959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a8009-466b-4051-bc98-d5a7ed6fba9d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:04.406891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a694ddb-5abe-4d54-a18a-e0b3f921ca5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:04.412639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a694ddb-5abe-4d54-a18a-e0b3f921ca5e service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:04.412937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a694ddb-5abe-4d54-a18a-e0b3f921ca5e service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:04.413215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a694ddb-5abe-4d54-a18a-e0b3f921ca5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:04.442670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a694ddb-5abe-4d54-a18a-e0b3f921ca5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:04.445845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a694ddb-5abe-4d54-a18a-e0b3f921ca5e service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:04.470181 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3781/7506] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:16:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:16:05.338151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d2fa0-5fea-412a-ad19-f5921c7750b2 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:05.340621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-be8d2fa0-5fea-412a-ad19-f5921c7750b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:05.356437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d2fa0-5fea-412a-ad19-f5921c7750b2 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:05.368553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25789778-970a-4d45-904c-30d03fe8fa0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:05.369624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25789778-970a-4d45-904c-30d03fe8fa0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:05.370141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25789778-970a-4d45-904c-30d03fe8fa0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:05.370754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25789778-970a-4d45-904c-30d03fe8fa0a admin admin] PATH_INFO: `/v3/projects/57e4fa7838ff4550baf3944117646774` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:05.372519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25789778-970a-4d45-904c-30d03fe8fa0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=57e4fa7838ff4550baf3944117646774)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:05.375787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25789778-970a-4d45-904c-30d03fe8fa0a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:05.479102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-25789778-970a-4d45-904c-30d03fe8fa0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57e4fa7838ff4550baf3944117646774', 'request_id': 'req-25789778-970a-4d45-904c-30d03fe8fa0a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:05.482791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:05.494876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-25789778-970a-4d45-904c-30d03fe8fa0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57e4fa7838ff4550baf3944117646774', 'request_id': 'req-25789778-970a-4d45-904c-30d03fe8fa0a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:05.510580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-25789778-970a-4d45-904c-30d03fe8fa0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57e4fa7838ff4550baf3944117646774', 'request_id': 'req-25789778-970a-4d45-904c-30d03fe8fa0a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:05.526562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-25789778-970a-4d45-904c-30d03fe8fa0a admin admin] The token cache is being invalidate because project 57e4fa7838ff4550baf3944117646774 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:16:05.526860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-25789778-970a-4d45-904c-30d03fe8fa0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:05.527744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-25789778-970a-4d45-904c-30d03fe8fa0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:05.529458 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3726/7507] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:16:05 2022] DELETE /identity/v3/projects/57e4fa7838ff4550baf3944117646774 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:05.535596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c3bfa-f41e-418d-972f-c4ea4e15a14f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:05.536065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c3bfa-f41e-418d-972f-c4ea4e15a14f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:05.536506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c3bfa-f41e-418d-972f-c4ea4e15a14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:05.567072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0b2c3bfa-f41e-418d-972f-c4ea4e15a14f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:05.568393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0b2c3bfa-f41e-418d-972f-c4ea4e15a14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:05.681821 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3782/7508] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:05.685304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b890379-cd17-4f75-afa1-7f1dee5b7c8c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:05.685855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b890379-cd17-4f75-afa1-7f1dee5b7c8c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:05.686313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b890379-cd17-4f75-afa1-7f1dee5b7c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:05.718414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2b890379-cd17-4f75-afa1-7f1dee5b7c8c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:05.719967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2b890379-cd17-4f75-afa1-7f1dee5b7c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:05.735191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c3bfa-f41e-418d-972f-c4ea4e15a14f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:05.745883 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3727/7509] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:05.748371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c3bfa-f41e-418d-972f-c4ea4e15a14f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:05.754966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216ac85-b242-47b4-ae81-534f5f19f4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:05.756008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216ac85-b242-47b4-ae81-534f5f19f4c2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:05.756524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216ac85-b242-47b4-ae81-534f5f19f4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:05.757058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216ac85-b242-47b4-ae81-534f5f19f4c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:05.757906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216ac85-b242-47b4-ae81-534f5f19f4c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:16:05.758547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216ac85-b242-47b4-ae81-534f5f19f4c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:05.760588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216ac85-b242-47b4-ae81-534f5f19f4c2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:05.763778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:05.784216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b890379-cd17-4f75-afa1-7f1dee5b7c8c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:05.803568 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3783/7510] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:16:05.806189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b890379-cd17-4f75-afa1-7f1dee5b7c8c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:05.813755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b0805-0f4f-4709-a501-6efd858a2fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:05.814436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b0805-0f4f-4709-a501-6efd858a2fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:05.814749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b0805-0f4f-4709-a501-6efd858a2fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:05.815023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b0805-0f4f-4709-a501-6efd858a2fe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:05.815788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b0805-0f4f-4709-a501-6efd858a2fe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:05.823966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b0805-0f4f-4709-a501-6efd858a2fe5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:05.870669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216ac85-b242-47b4-ae81-534f5f19f4c2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:05.883849 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3728/7511] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:16:05 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:16:05.884220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216ac85-b242-47b4-ae81-534f5f19f4c2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:05.979390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71bec46-e8b5-4908-9b16-51f885241947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:05.980093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bec46-e8b5-4908-9b16-51f885241947 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:05.980381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bec46-e8b5-4908-9b16-51f885241947 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:05.980660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bec46-e8b5-4908-9b16-51f885241947 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:05.981927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71bec46-e8b5-4908-9b16-51f885241947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:05.984130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71bec46-e8b5-4908-9b16-51f885241947 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:06.038439 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3784/7512] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:16:05 2022] POST /identity/v3/users => generated 290 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:16:06.042457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b0805-0f4f-4709-a501-6efd858a2fe5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:06.062857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b0805-0f4f-4709-a501-6efd858a2fe5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:06.079542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d5a95-bb10-4649-b409-592075633d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:06.080443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d5a95-bb10-4649-b409-592075633d24 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:06.080922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d5a95-bb10-4649-b409-592075633d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:06.081896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d5a95-bb10-4649-b409-592075633d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:06.082724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d5a95-bb10-4649-b409-592075633d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:06.085834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d5a95-bb10-4649-b409-592075633d24 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:06.106131 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3729/7513] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:16:06.107510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71bec46-e8b5-4908-9b16-51f885241947 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:06.119204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71bec46-e8b5-4908-9b16-51f885241947 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:06.124605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bedf7-8eb5-4201-a3d7-7516f01db1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:06.125465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bedf7-8eb5-4201-a3d7-7516f01db1de admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:06.125897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bedf7-8eb5-4201-a3d7-7516f01db1de admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:06.126348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bedf7-8eb5-4201-a3d7-7516f01db1de admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:06.127226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bedf7-8eb5-4201-a3d7-7516f01db1de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:06.129239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bedf7-8eb5-4201-a3d7-7516f01db1de admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:06.157273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d5a95-bb10-4649-b409-592075633d24 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:06.178849 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3785/7514] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:16:06.185469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d5a95-bb10-4649-b409-592075633d24 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:06.193160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ffef5-d0a2-4825-8ff6-02503febd11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:06.194212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ffef5-d0a2-4825-8ff6-02503febd11c admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:06.194669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ffef5-d0a2-4825-8ff6-02503febd11c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:06.195121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ffef5-d0a2-4825-8ff6-02503febd11c admin admin] PATH_INFO: `/v3/projects/364f70244ca2486c80d34622147c6d49/users/8c91c63645be41819d9a4dd6dec87fae/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:06.223632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ffef5-d0a2-4825-8ff6-02503febd11c admin admin] RBAC: Authorizing `identity:create_grant(project_id=364f70244ca2486c80d34622147c6d49, user_id=8c91c63645be41819d9a4dd6dec87fae, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:06.227193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ffef5-d0a2-4825-8ff6-02503febd11c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:06.257939 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3730/7515] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:16:06 2022] PUT /identity/v3/projects/364f70244ca2486c80d34622147c6d49/users/8c91c63645be41819d9a4dd6dec87fae/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:06.269501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f6fd2-2a1c-4214-a082-8080fa45c94d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:06.269797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f6fd2-2a1c-4214-a082-8080fa45c94d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:06.270075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f6fd2-2a1c-4214-a082-8080fa45c94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:06.334621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f47f6fd2-2a1c-4214-a082-8080fa45c94d None None] MFA Rules not processed for user `8c91c63645be41819d9a4dd6dec87fae`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:06.336419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f47f6fd2-2a1c-4214-a082-8080fa45c94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:06.479833 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3786/7516] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:06.493782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ffef5-d0a2-4825-8ff6-02503febd11c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:06.496122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2e3ffef5-d0a2-4825-8ff6-02503febd11c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:06.631928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ffef5-d0a2-4825-8ff6-02503febd11c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:06.639405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75099a6-9918-4aaf-bc6e-f0fae9701957 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:06.640240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75099a6-9918-4aaf-bc6e-f0fae9701957 service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:06.640518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75099a6-9918-4aaf-bc6e-f0fae9701957 service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:06.640798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75099a6-9918-4aaf-bc6e-f0fae9701957 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:06.666290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75099a6-9918-4aaf-bc6e-f0fae9701957 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:06.667795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75099a6-9918-4aaf-bc6e-f0fae9701957 service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:06.689388 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3731/7517] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:16:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:16:07.230074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f6fd2-2a1c-4214-a082-8080fa45c94d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:07.415930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f6fd2-2a1c-4214-a082-8080fa45c94d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:07.429695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148edb3-52f3-4710-ae9f-5dfe8860ee70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:07.430600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148edb3-52f3-4710-ae9f-5dfe8860ee70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:07.431048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148edb3-52f3-4710-ae9f-5dfe8860ee70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:07.436875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148edb3-52f3-4710-ae9f-5dfe8860ee70 admin admin] PATH_INFO: `/v3/users/8c91c63645be41819d9a4dd6dec87fae` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:07.438665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148edb3-52f3-4710-ae9f-5dfe8860ee70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c91c63645be41819d9a4dd6dec87fae)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:07.440826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148edb3-52f3-4710-ae9f-5dfe8860ee70 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:07.462389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:07.578354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f148edb3-52f3-4710-ae9f-5dfe8860ee70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c91c63645be41819d9a4dd6dec87fae'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:07.597886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f148edb3-52f3-4710-ae9f-5dfe8860ee70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c91c63645be41819d9a4dd6dec87fae'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:07.641870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f148edb3-52f3-4710-ae9f-5dfe8860ee70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c91c63645be41819d9a4dd6dec87fae'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:07.642812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f148edb3-52f3-4710-ae9f-5dfe8860ee70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c91c63645be41819d9a4dd6dec87fae'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:07.664124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f148edb3-52f3-4710-ae9f-5dfe8860ee70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c91c63645be41819d9a4dd6dec87fae'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:07.672838 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3787/7518] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:16:07 2022] DELETE /identity/v3/users/8c91c63645be41819d9a4dd6dec87fae => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:07.678257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b9761-ad39-4a93-9ec7-fde112e252df None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:07.678790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b9761-ad39-4a93-9ec7-fde112e252df None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:07.678874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b9761-ad39-4a93-9ec7-fde112e252df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:07.729667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-358b9761-ad39-4a93-9ec7-fde112e252df None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:07.738727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-358b9761-ad39-4a93-9ec7-fde112e252df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:07.890538 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3732/7519] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:07.913561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148edb3-52f3-4710-ae9f-5dfe8860ee70 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:07.915428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f148edb3-52f3-4710-ae9f-5dfe8860ee70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:08.052400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148edb3-52f3-4710-ae9f-5dfe8860ee70 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:08.062974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e7b29-6776-4350-9d30-bdb8e2f99ca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:08.067099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e7b29-6776-4350-9d30-bdb8e2f99ca9 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:08.067951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e7b29-6776-4350-9d30-bdb8e2f99ca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:08.068392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e7b29-6776-4350-9d30-bdb8e2f99ca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:08.098882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e7b29-6776-4350-9d30-bdb8e2f99ca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:08.100513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e7b29-6776-4350-9d30-bdb8e2f99ca9 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:08.124527 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3788/7520] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:16:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:16:09.498475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b9761-ad39-4a93-9ec7-fde112e252df None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:09.500317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-358b9761-ad39-4a93-9ec7-fde112e252df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:09.522449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b9761-ad39-4a93-9ec7-fde112e252df None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:09.541499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470251bf-2a60-404b-83ae-6d99eb3f1f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:09.542468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470251bf-2a60-404b-83ae-6d99eb3f1f47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:09.543136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470251bf-2a60-404b-83ae-6d99eb3f1f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:09.543750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470251bf-2a60-404b-83ae-6d99eb3f1f47 admin admin] PATH_INFO: `/v3/users/a2745d6923e347939e2c67015aefadda` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:09.545261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470251bf-2a60-404b-83ae-6d99eb3f1f47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2745d6923e347939e2c67015aefadda)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:09.547806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470251bf-2a60-404b-83ae-6d99eb3f1f47 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:09.596617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:09.679882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-470251bf-2a60-404b-83ae-6d99eb3f1f47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2745d6923e347939e2c67015aefadda'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:09.682274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-470251bf-2a60-404b-83ae-6d99eb3f1f47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2745d6923e347939e2c67015aefadda'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:09.683454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-470251bf-2a60-404b-83ae-6d99eb3f1f47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2745d6923e347939e2c67015aefadda'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:09.707826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-470251bf-2a60-404b-83ae-6d99eb3f1f47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2745d6923e347939e2c67015aefadda'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:09.738678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e7b29-6776-4350-9d30-bdb8e2f99ca9 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:09.740200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-de6e7b29-6776-4350-9d30-bdb8e2f99ca9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:09.741217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-470251bf-2a60-404b-83ae-6d99eb3f1f47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2745d6923e347939e2c67015aefadda'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:09.761779 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3733/7521] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:16:09 2022] DELETE /identity/v3/users/a2745d6923e347939e2c67015aefadda => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:09.770522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e7b29-6776-4350-9d30-bdb8e2f99ca9 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:09.904322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2461c-7f34-4d8a-b976-09bb52a0c9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:09.905059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2461c-7f34-4d8a-b976-09bb52a0c9f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:09.905346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2461c-7f34-4d8a-b976-09bb52a0c9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:09.905619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2461c-7f34-4d8a-b976-09bb52a0c9f2 admin admin] PATH_INFO: `/v3/projects/364f70244ca2486c80d34622147c6d49` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:09.908428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2461c-7f34-4d8a-b976-09bb52a0c9f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=364f70244ca2486c80d34622147c6d49)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:09.910958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2461c-7f34-4d8a-b976-09bb52a0c9f2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:10.001883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:10.054736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c3c2461c-7f34-4d8a-b976-09bb52a0c9f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '364f70244ca2486c80d34622147c6d49', 'request_id': 'req-c3c2461c-7f34-4d8a-b976-09bb52a0c9f2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:10.055822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de737b-e0a7-4a12-8fda-6c80f81a46a3 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:10.056009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de737b-e0a7-4a12-8fda-6c80f81a46a3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:10.056285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de737b-e0a7-4a12-8fda-6c80f81a46a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:10.065178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c3c2461c-7f34-4d8a-b976-09bb52a0c9f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '364f70244ca2486c80d34622147c6d49', 'request_id': 'req-c3c2461c-7f34-4d8a-b976-09bb52a0c9f2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:10.076699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c3c2461c-7f34-4d8a-b976-09bb52a0c9f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '364f70244ca2486c80d34622147c6d49', 'request_id': 'req-c3c2461c-7f34-4d8a-b976-09bb52a0c9f2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:10.088412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-64de737b-e0a7-4a12-8fda-6c80f81a46a3 None None] MFA Rules not processed for user `3066b652821f47f6909892c9b04f4236`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:10.089646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-64de737b-e0a7-4a12-8fda-6c80f81a46a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:10.095899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c3c2461c-7f34-4d8a-b976-09bb52a0c9f2 admin admin] The token cache is being invalidate because project 364f70244ca2486c80d34622147c6d49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:16:10.096501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c3c2461c-7f34-4d8a-b976-09bb52a0c9f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:10.098012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c3c2461c-7f34-4d8a-b976-09bb52a0c9f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:10.102921 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3789/7522] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:16:09 2022] DELETE /identity/v3/projects/364f70244ca2486c80d34622147c6d49 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:10.121769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5fca5-636a-4537-8cd5-3b9ffaad3d17 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:10.122218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5fca5-636a-4537-8cd5-3b9ffaad3d17 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:10.122526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5fca5-636a-4537-8cd5-3b9ffaad3d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:10.189144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0fd5fca5-636a-4537-8cd5-3b9ffaad3d17 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:10.190411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0fd5fca5-636a-4537-8cd5-3b9ffaad3d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:10.219360 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3734/7523] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:10.344685 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3790/7524] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:10.352727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ad013-cf94-484e-a614-cf0ca1118906 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:10.353094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ad013-cf94-484e-a614-cf0ca1118906 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:10.353309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ad013-cf94-484e-a614-cf0ca1118906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:10.395496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-861ad013-cf94-484e-a614-cf0ca1118906 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:10.397184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-861ad013-cf94-484e-a614-cf0ca1118906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:10.411070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5fca5-636a-4537-8cd5-3b9ffaad3d17 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:10.420207 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3735/7525] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:10.421807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5fca5-636a-4537-8cd5-3b9ffaad3d17 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:10.427975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28869e40-5b0c-4cc5-8694-b42e7901fff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:10.428520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28869e40-5b0c-4cc5-8694-b42e7901fff7 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:10.428804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28869e40-5b0c-4cc5-8694-b42e7901fff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:10.429083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28869e40-5b0c-4cc5-8694-b42e7901fff7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:10.429755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28869e40-5b0c-4cc5-8694-b42e7901fff7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:16:10.430066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28869e40-5b0c-4cc5-8694-b42e7901fff7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:10.431729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28869e40-5b0c-4cc5-8694-b42e7901fff7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:10.457708 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3791/7526] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:16:10.462795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ad013-cf94-484e-a614-cf0ca1118906 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:10.482317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ad013-cf94-484e-a614-cf0ca1118906 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:10.489203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26c11d-80a0-43cf-8824-6650107071a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:10.489970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26c11d-80a0-43cf-8824-6650107071a8 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:10.490371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26c11d-80a0-43cf-8824-6650107071a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:10.490785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26c11d-80a0-43cf-8824-6650107071a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:10.491487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26c11d-80a0-43cf-8824-6650107071a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:10.494082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26c11d-80a0-43cf-8824-6650107071a8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:10.515856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28869e40-5b0c-4cc5-8694-b42e7901fff7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:10.517764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-28869e40-5b0c-4cc5-8694-b42e7901fff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:10.545216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28869e40-5b0c-4cc5-8694-b42e7901fff7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:10.553170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5e2e0-863e-42b7-9486-de0260fdc1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:10.554235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5e2e0-863e-42b7-9486-de0260fdc1f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:10.556375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5e2e0-863e-42b7-9486-de0260fdc1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:10.557544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5e2e0-863e-42b7-9486-de0260fdc1f5 admin admin] PATH_INFO: `/v3/projects/8f92b50dc0364842980ce41af830ba1a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:10.559482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5e2e0-863e-42b7-9486-de0260fdc1f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f92b50dc0364842980ce41af830ba1a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:10.562300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5e2e0-863e-42b7-9486-de0260fdc1f5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:10.563740 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3736/7527] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:16:10 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:16:10.564761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26c11d-80a0-43cf-8824-6650107071a8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:10.672710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-99a5e2e0-863e-42b7-9486-de0260fdc1f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f92b50dc0364842980ce41af830ba1a', 'request_id': 'req-99a5e2e0-863e-42b7-9486-de0260fdc1f5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:10.692429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-99a5e2e0-863e-42b7-9486-de0260fdc1f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f92b50dc0364842980ce41af830ba1a', 'request_id': 'req-99a5e2e0-863e-42b7-9486-de0260fdc1f5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:10.719224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-99a5e2e0-863e-42b7-9486-de0260fdc1f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f92b50dc0364842980ce41af830ba1a', 'request_id': 'req-99a5e2e0-863e-42b7-9486-de0260fdc1f5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:10.720369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26c11d-80a0-43cf-8824-6650107071a8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:10.742374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-99a5e2e0-863e-42b7-9486-de0260fdc1f5 admin admin] The token cache is being invalidate because project 8f92b50dc0364842980ce41af830ba1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:16:10.743057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-99a5e2e0-863e-42b7-9486-de0260fdc1f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:10.743503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:10.751055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-99a5e2e0-863e-42b7-9486-de0260fdc1f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:10.753416 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3792/7528] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:16:10 2022] DELETE /identity/v3/projects/8f92b50dc0364842980ce41af830ba1a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:10.758034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6975c-7d10-4b3c-b9e1-3d5be485ee82 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:10.758324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6975c-7d10-4b3c-b9e1-3d5be485ee82 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:10.758646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6975c-7d10-4b3c-b9e1-3d5be485ee82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:10.790875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0ee6975c-7d10-4b3c-b9e1-3d5be485ee82 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:10.793048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0ee6975c-7d10-4b3c-b9e1-3d5be485ee82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:10.818262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca40188-6208-4624-b7cc-20e1cf698e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:10.818973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca40188-6208-4624-b7cc-20e1cf698e6f admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:10.819486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca40188-6208-4624-b7cc-20e1cf698e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:10.819965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca40188-6208-4624-b7cc-20e1cf698e6f admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:10.820821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca40188-6208-4624-b7cc-20e1cf698e6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:10.821622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:10.823005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca40188-6208-4624-b7cc-20e1cf698e6f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:10.865585 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3737/7529] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:16:10 2022] POST /identity/v3/users => generated 293 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:16:10.870283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca40188-6208-4624-b7cc-20e1cf698e6f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:10.873357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4ca40188-6208-4624-b7cc-20e1cf698e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:10.888755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca40188-6208-4624-b7cc-20e1cf698e6f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:10.890168 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3793/7530] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:16:10.894419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de0c1a-5be8-4c9a-87fe-36ecc8a41689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:10.895322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de0c1a-5be8-4c9a-87fe-36ecc8a41689 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:10.895766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de0c1a-5be8-4c9a-87fe-36ecc8a41689 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:10.896040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de0c1a-5be8-4c9a-87fe-36ecc8a41689 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:10.896688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de0c1a-5be8-4c9a-87fe-36ecc8a41689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:10.898316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de0c1a-5be8-4c9a-87fe-36ecc8a41689 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:10.900047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b4774-0aa5-4f02-9180-b16af14c700f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:10.900333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b4774-0aa5-4f02-9180-b16af14c700f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:10.900620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b4774-0aa5-4f02-9180-b16af14c700f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:10.914548 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3738/7531] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:16:10.916027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de0c1a-5be8-4c9a-87fe-36ecc8a41689 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:10.929508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de0c1a-5be8-4c9a-87fe-36ecc8a41689 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:10.936440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-cf9b4774-0aa5-4f02-9180-b16af14c700f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:10.937633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33028ed-4cda-4f1d-b58f-18a26780407c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:10.938267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33028ed-4cda-4f1d-b58f-18a26780407c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:10.938597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33028ed-4cda-4f1d-b58f-18a26780407c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:10.938877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33028ed-4cda-4f1d-b58f-18a26780407c admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:10.939491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cf9b4774-0aa5-4f02-9180-b16af14c700f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:10.939604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33028ed-4cda-4f1d-b58f-18a26780407c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:10.942796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33028ed-4cda-4f1d-b58f-18a26780407c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:10.954221 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3794/7532] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:10.955469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b4774-0aa5-4f02-9180-b16af14c700f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:10.960402 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3739/7533] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:16:10.964650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33028ed-4cda-4f1d-b58f-18a26780407c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:10.973913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b4774-0aa5-4f02-9180-b16af14c700f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:10.979144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aff6f0-f786-4e99-8834-6d689c26edd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:10.979589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aff6f0-f786-4e99-8834-6d689c26edd2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:10.979873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aff6f0-f786-4e99-8834-6d689c26edd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:10.980148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aff6f0-f786-4e99-8834-6d689c26edd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:10.980847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aff6f0-f786-4e99-8834-6d689c26edd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:16:10.981158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aff6f0-f786-4e99-8834-6d689c26edd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:10.982788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aff6f0-f786-4e99-8834-6d689c26edd2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:10.984947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33028ed-4cda-4f1d-b58f-18a26780407c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:10.992909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87d592-8a67-42c0-82f7-5f6af2f3b59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:10.994607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d592-8a67-42c0-82f7-5f6af2f3b59c admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:10.994929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d592-8a67-42c0-82f7-5f6af2f3b59c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:10.995601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d592-8a67-42c0-82f7-5f6af2f3b59c admin admin] PATH_INFO: `/v3/projects/a1dd25b43b47445cb97e9fd7c6a2c5b1/users/40865ecd1e6c4ca6bbdd0a181a8b2c65/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:11.000714 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3795/7534] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:16:11.002021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aff6f0-f786-4e99-8834-6d689c26edd2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:11.013181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aff6f0-f786-4e99-8834-6d689c26edd2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:11.019148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4040eac-3ee6-4125-aff9-c13886a1f869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:11.019487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87d592-8a67-42c0-82f7-5f6af2f3b59c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1dd25b43b47445cb97e9fd7c6a2c5b1, user_id=40865ecd1e6c4ca6bbdd0a181a8b2c65, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:11.020045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4040eac-3ee6-4125-aff9-c13886a1f869 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:11.020323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4040eac-3ee6-4125-aff9-c13886a1f869 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:11.020664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4040eac-3ee6-4125-aff9-c13886a1f869 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:11.021368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4040eac-3ee6-4125-aff9-c13886a1f869 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:11.022735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87d592-8a67-42c0-82f7-5f6af2f3b59c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:11.023689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4040eac-3ee6-4125-aff9-c13886a1f869 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:11.068775 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3740/7536] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:16:10 2022] PUT /identity/v3/projects/a1dd25b43b47445cb97e9fd7c6a2c5b1/users/40865ecd1e6c4ca6bbdd0a181a8b2c65/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:11.069065 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3796/7536] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:16:11 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:16:11.070288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4040eac-3ee6-4125-aff9-c13886a1f869 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:11.070726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8d688-86f9-4c80-a519-666f4eaf69f8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:11.071112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8d688-86f9-4c80-a519-666f4eaf69f8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:11.071579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8d688-86f9-4c80-a519-666f4eaf69f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:11.106698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bdc8d688-86f9-4c80-a519-666f4eaf69f8 None None] MFA Rules not processed for user `40865ecd1e6c4ca6bbdd0a181a8b2c65`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:11.108063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bdc8d688-86f9-4c80-a519-666f4eaf69f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:11.176848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4040eac-3ee6-4125-aff9-c13886a1f869 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:11.184499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c0af2-bcdf-421d-870f-6ede69dd14fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:11.185685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c0af2-bcdf-421d-870f-6ede69dd14fd admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:11.186279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c0af2-bcdf-421d-870f-6ede69dd14fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:11.186787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c0af2-bcdf-421d-870f-6ede69dd14fd admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:11.187761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c0af2-bcdf-421d-870f-6ede69dd14fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:11.191316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c0af2-bcdf-421d-870f-6ede69dd14fd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:11.215626 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3741/7537] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:11.220416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8d688-86f9-4c80-a519-666f4eaf69f8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:11.234050 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3797/7538] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:16:11 2022] POST /identity/v3/users => generated 291 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:16:11.239814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c0af2-bcdf-421d-870f-6ede69dd14fd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:11.246911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8d688-86f9-4c80-a519-666f4eaf69f8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:11.256386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50f79a-e223-4886-8f5c-bb6b33498b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:11.257095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50f79a-e223-4886-8f5c-bb6b33498b2d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:11.257443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50f79a-e223-4886-8f5c-bb6b33498b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:11.257834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50f79a-e223-4886-8f5c-bb6b33498b2d admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:11.258181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c0af2-bcdf-421d-870f-6ede69dd14fd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:11.258646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50f79a-e223-4886-8f5c-bb6b33498b2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:11.260893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50f79a-e223-4886-8f5c-bb6b33498b2d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:11.278505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4c7f5-b278-4823-a345-634ab5e5d8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:11.279521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4c7f5-b278-4823-a345-634ab5e5d8a6 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:11.280033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4c7f5-b278-4823-a345-634ab5e5d8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:11.280458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4c7f5-b278-4823-a345-634ab5e5d8a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:11.281692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4c7f5-b278-4823-a345-634ab5e5d8a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:11.284242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4c7f5-b278-4823-a345-634ab5e5d8a6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:11.303187 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3798/7539] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:16:11.303702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4c7f5-b278-4823-a345-634ab5e5d8a6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:11.304333 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3742/7540] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:16:11 2022] POST /identity/v3/projects => generated 385 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:16:11.306022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50f79a-e223-4886-8f5c-bb6b33498b2d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:11.318888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4c7f5-b278-4823-a345-634ab5e5d8a6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:11.347519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:11.401916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50f79a-e223-4886-8f5c-bb6b33498b2d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:11.408228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c168cfe-c46c-4afa-ab4b-0e62193ecc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:11.409363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c168cfe-c46c-4afa-ab4b-0e62193ecc8a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:11.409822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c168cfe-c46c-4afa-ab4b-0e62193ecc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:11.410115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c168cfe-c46c-4afa-ab4b-0e62193ecc8a admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:11.415206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c168cfe-c46c-4afa-ab4b-0e62193ecc8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:11.417913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c168cfe-c46c-4afa-ab4b-0e62193ecc8a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:11.423314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bff14-8a4d-4c6e-a852-cf4fa90c89bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:11.423946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bff14-8a4d-4c6e-a852-cf4fa90c89bf admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:11.424230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bff14-8a4d-4c6e-a852-cf4fa90c89bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:11.424508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bff14-8a4d-4c6e-a852-cf4fa90c89bf admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:11.426707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314bff14-8a4d-4c6e-a852-cf4fa90c89bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:11.428930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314bff14-8a4d-4c6e-a852-cf4fa90c89bf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:11.437058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:11.447276 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3799/7541] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:16:11.448307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bff14-8a4d-4c6e-a852-cf4fa90c89bf admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:11.454109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-314bff14-8a4d-4c6e-a852-cf4fa90c89bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:11.455056 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3743/7542] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:16:11 2022] POST /identity/v3/users => generated 299 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:16:11.455056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c168cfe-c46c-4afa-ab4b-0e62193ecc8a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:11.470327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c168cfe-c46c-4afa-ab4b-0e62193ecc8a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:11.476327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cacaaa6-7aee-47c7-9ca4-37b43a5bd877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:11.477332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacaaa6-7aee-47c7-9ca4-37b43a5bd877 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:11.477332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacaaa6-7aee-47c7-9ca4-37b43a5bd877 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:11.477614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacaaa6-7aee-47c7-9ca4-37b43a5bd877 admin admin] PATH_INFO: `/v3/projects/ee4f57ea97c64af5958543447819ce8d/users/a554c62c88c94526aea50fe59333aa29/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:11.500102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cacaaa6-7aee-47c7-9ca4-37b43a5bd877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee4f57ea97c64af5958543447819ce8d, user_id=a554c62c88c94526aea50fe59333aa29, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:11.503323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cacaaa6-7aee-47c7-9ca4-37b43a5bd877 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:11.533794 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3744/7543] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:16:11 2022] PUT /identity/v3/projects/ee4f57ea97c64af5958543447819ce8d/users/a554c62c88c94526aea50fe59333aa29/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:11.536184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cacaaa6-7aee-47c7-9ca4-37b43a5bd877 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:11.538619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bff14-8a4d-4c6e-a852-cf4fa90c89bf admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:11.604130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f2cca-9a04-419b-ae5e-a9997cc929b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:11.604790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f2cca-9a04-419b-ae5e-a9997cc929b5 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:11.605204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f2cca-9a04-419b-ae5e-a9997cc929b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:11.605488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f2cca-9a04-419b-ae5e-a9997cc929b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:11.607549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4e2f2cca-9a04-419b-ae5e-a9997cc929b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:11.622267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f2cca-9a04-419b-ae5e-a9997cc929b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:11.623275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f2cca-9a04-419b-ae5e-a9997cc929b5 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:11.626381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cacaaa6-7aee-47c7-9ca4-37b43a5bd877 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:11.636635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb8e89-0352-46f3-85be-e36aa6067741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:11.637202 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3800/7544] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:16:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:16:11.638598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb8e89-0352-46f3-85be-e36aa6067741 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:11.638988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb8e89-0352-46f3-85be-e36aa6067741 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:11.639343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a670e80-73fa-4b10-9eff-9f54e9fe0057 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:11.639650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a670e80-73fa-4b10-9eff-9f54e9fe0057 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:11.640031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb8e89-0352-46f3-85be-e36aa6067741 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:11.640168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a670e80-73fa-4b10-9eff-9f54e9fe0057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:11.640923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb8e89-0352-46f3-85be-e36aa6067741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:11.643015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb8e89-0352-46f3-85be-e36aa6067741 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:11.658396 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3745/7545] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:16:11.662738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb8e89-0352-46f3-85be-e36aa6067741 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:11.666678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9a670e80-73fa-4b10-9eff-9f54e9fe0057 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:11.667896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9a670e80-73fa-4b10-9eff-9f54e9fe0057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:11.677378 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3801/7546] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:11.679145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb8e89-0352-46f3-85be-e36aa6067741 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:11.685097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc2bc6-863e-4930-8039-7d9d10b81c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:11.685779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc2bc6-863e-4930-8039-7d9d10b81c7e admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:11.686065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc2bc6-863e-4930-8039-7d9d10b81c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:11.686344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc2bc6-863e-4930-8039-7d9d10b81c7e admin admin] PATH_INFO: `/v3/projects/90a969c31ee846ac8daf915407e18287/users/ec9c2469a51940fab3bdc23c44cac11d/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:11.692782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a670e80-73fa-4b10-9eff-9f54e9fe0057 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:11.694441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9a670e80-73fa-4b10-9eff-9f54e9fe0057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:11.705550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc2bc6-863e-4930-8039-7d9d10b81c7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a969c31ee846ac8daf915407e18287, user_id=ec9c2469a51940fab3bdc23c44cac11d, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:11.708811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc2bc6-863e-4930-8039-7d9d10b81c7e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:11.711197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a670e80-73fa-4b10-9eff-9f54e9fe0057 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:11.715149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c7bc1-d89f-4810-acaa-db3207866b1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:11.715983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c7bc1-d89f-4810-acaa-db3207866b1a service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:11.716448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c7bc1-d89f-4810-acaa-db3207866b1a service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:11.716796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c7bc1-d89f-4810-acaa-db3207866b1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:11.729903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c7bc1-d89f-4810-acaa-db3207866b1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:11.732362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c7bc1-d89f-4810-acaa-db3207866b1a service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:11.735457 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3746/7547] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:16:11 2022] PUT /identity/v3/projects/90a969c31ee846ac8daf915407e18287/users/ec9c2469a51940fab3bdc23c44cac11d/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:11.739488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc2bc6-863e-4930-8039-7d9d10b81c7e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:11.755899 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3802/7548] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:16:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:16:11.833888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc2bc6-863e-4930-8039-7d9d10b81c7e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:11.842073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d664f-4d7a-4208-8345-5d1aa6c37078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:11.842980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d664f-4d7a-4208-8345-5d1aa6c37078 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:11.843289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d664f-4d7a-4208-8345-5d1aa6c37078 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:11.843597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d664f-4d7a-4208-8345-5d1aa6c37078 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:11.844400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d664f-4d7a-4208-8345-5d1aa6c37078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:11.847980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d664f-4d7a-4208-8345-5d1aa6c37078 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:11.857048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:11.866595 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3747/7549] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:16:11.871135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c7bc1-d89f-4810-acaa-db3207866b1a service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:11.883505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c7bc1-d89f-4810-acaa-db3207866b1a service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:11.894422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c73f93-360d-4d0a-b8fa-b0ad7b2a60de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:11.895663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c73f93-360d-4d0a-b8fa-b0ad7b2a60de admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:11.896547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c73f93-360d-4d0a-b8fa-b0ad7b2a60de admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:11.897550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c73f93-360d-4d0a-b8fa-b0ad7b2a60de admin admin] PATH_INFO: `/v3/projects/90a969c31ee846ac8daf915407e18287/users/ec9c2469a51940fab3bdc23c44cac11d/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:11.900188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c73f93-360d-4d0a-b8fa-b0ad7b2a60de admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a969c31ee846ac8daf915407e18287, user_id=ec9c2469a51940fab3bdc23c44cac11d, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:11.903287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c73f93-360d-4d0a-b8fa-b0ad7b2a60de admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:11.919394 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3803/7550] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:16:11 2022] PUT /identity/v3/projects/90a969c31ee846ac8daf915407e18287/users/ec9c2469a51940fab3bdc23c44cac11d/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:11.932589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f1544-2068-402f-8661-5931fc88f988 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:11.932895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f1544-2068-402f-8661-5931fc88f988 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:11.933164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f1544-2068-402f-8661-5931fc88f988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:11.982614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-123f1544-2068-402f-8661-5931fc88f988 None None] MFA Rules not processed for user `ec9c2469a51940fab3bdc23c44cac11d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:11.983925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-123f1544-2068-402f-8661-5931fc88f988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:12.121712 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3748/7551] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:12.151009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c73f93-360d-4d0a-b8fa-b0ad7b2a60de admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:12.277778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c73f93-360d-4d0a-b8fa-b0ad7b2a60de admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:12.284713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc33195-dd94-4fc6-a80f-060c5965c027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:12.295077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc33195-dd94-4fc6-a80f-060c5965c027 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:12.295918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc33195-dd94-4fc6-a80f-060c5965c027 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:12.296304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc33195-dd94-4fc6-a80f-060c5965c027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:12.364095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:12.396915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc33195-dd94-4fc6-a80f-060c5965c027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:12.400242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc33195-dd94-4fc6-a80f-060c5965c027 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:12.438886 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3804/7552] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:16:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:16:13.406448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f1544-2068-402f-8661-5931fc88f988 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:13.423411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f1544-2068-402f-8661-5931fc88f988 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:13.431947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c38b9-a7e4-4813-9046-45399775592e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:13.433959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c38b9-a7e4-4813-9046-45399775592e service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:13.433959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c38b9-a7e4-4813-9046-45399775592e service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:13.434329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c38b9-a7e4-4813-9046-45399775592e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:13.458896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c38b9-a7e4-4813-9046-45399775592e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:13.460522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c38b9-a7e4-4813-9046-45399775592e service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:13.484751 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3749/7553] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:16:13 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:16:21.344161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5285c7d-475e-4cec-a66f-d0bf94ab06e3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:21.347330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5285c7d-475e-4cec-a66f-d0bf94ab06e3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:21.348397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5285c7d-475e-4cec-a66f-d0bf94ab06e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:21.407558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c5285c7d-475e-4cec-a66f-d0bf94ab06e3 None None] MFA Rules not processed for user `a554c62c88c94526aea50fe59333aa29`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:21.409168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c5285c7d-475e-4cec-a66f-d0bf94ab06e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:21.521463 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3805/7554] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:21.537664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fda07e-095b-4b6d-927c-58a0e0a2cabc None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:21.538526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fda07e-095b-4b6d-927c-58a0e0a2cabc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:21.539329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fda07e-095b-4b6d-927c-58a0e0a2cabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:21.575141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-69fda07e-095b-4b6d-927c-58a0e0a2cabc None None] MFA Rules not processed for user `a554c62c88c94526aea50fe59333aa29`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:21.577506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-69fda07e-095b-4b6d-927c-58a0e0a2cabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:21.588018 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3750/7555] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:16:21.600610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5285c7d-475e-4cec-a66f-d0bf94ab06e3 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:21.602442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c5285c7d-475e-4cec-a66f-d0bf94ab06e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:21.762350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5285c7d-475e-4cec-a66f-d0bf94ab06e3 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:21.772972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da140d-9f30-4aad-a36a-a997fb64320c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:21.773746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da140d-9f30-4aad-a36a-a997fb64320c service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:21.787749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da140d-9f30-4aad-a36a-a997fb64320c service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:21.788081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da140d-9f30-4aad-a36a-a997fb64320c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:21.801507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da140d-9f30-4aad-a36a-a997fb64320c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:21.803971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da140d-9f30-4aad-a36a-a997fb64320c service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:21.829112 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3806/7556] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:16:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:16:23.332790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fda07e-095b-4b6d-927c-58a0e0a2cabc None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:23.334523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-69fda07e-095b-4b6d-927c-58a0e0a2cabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:23.572903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fda07e-095b-4b6d-927c-58a0e0a2cabc None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:23.591157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4c76e-f37e-4a04-97ea-0438c27c13e1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:23.593544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4c76e-f37e-4a04-97ea-0438c27c13e1 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:23.594786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4c76e-f37e-4a04-97ea-0438c27c13e1 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:23.595140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4c76e-f37e-4a04-97ea-0438c27c13e1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:23.600683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-52d4c76e-f37e-4a04-97ea-0438c27c13e1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:23.642569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4c76e-f37e-4a04-97ea-0438c27c13e1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:23.644429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4c76e-f37e-4a04-97ea-0438c27c13e1 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:23.682678 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3751/7557] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:16:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:16:31.809772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=75 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:32.784809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da140d-9f30-4aad-a36a-a997fb64320c service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:32.869226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da140d-9f30-4aad-a36a-a997fb64320c service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:32.874347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b42b28-18df-4469-8598-07ab163175d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:32.875602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b42b28-18df-4469-8598-07ab163175d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:32.876005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b42b28-18df-4469-8598-07ab163175d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:32.876470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b42b28-18df-4469-8598-07ab163175d5 admin admin] PATH_INFO: `/v3/users/40865ecd1e6c4ca6bbdd0a181a8b2c65` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:32.878048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b42b28-18df-4469-8598-07ab163175d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40865ecd1e6c4ca6bbdd0a181a8b2c65)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:32.880208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b42b28-18df-4469-8598-07ab163175d5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:32.934207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c0b42b28-18df-4469-8598-07ab163175d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40865ecd1e6c4ca6bbdd0a181a8b2c65'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:32.944862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c0b42b28-18df-4469-8598-07ab163175d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40865ecd1e6c4ca6bbdd0a181a8b2c65'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:32.963009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c0b42b28-18df-4469-8598-07ab163175d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40865ecd1e6c4ca6bbdd0a181a8b2c65'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:32.965229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c0b42b28-18df-4469-8598-07ab163175d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40865ecd1e6c4ca6bbdd0a181a8b2c65'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:32.973775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c0b42b28-18df-4469-8598-07ab163175d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40865ecd1e6c4ca6bbdd0a181a8b2c65'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:32.981360 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3807/7558] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:16:32 2022] DELETE /identity/v3/users/40865ecd1e6c4ca6bbdd0a181a8b2c65 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:32.987715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39201fce-84d8-4a8d-85ac-aa87ea28499f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:32.988365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39201fce-84d8-4a8d-85ac-aa87ea28499f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:32.988629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39201fce-84d8-4a8d-85ac-aa87ea28499f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:33.024437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-39201fce-84d8-4a8d-85ac-aa87ea28499f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:33.025819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-39201fce-84d8-4a8d-85ac-aa87ea28499f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:33.088363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:33.111944 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3752/7559] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:16:33.125806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b42b28-18df-4469-8598-07ab163175d5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:33.128039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c0b42b28-18df-4469-8598-07ab163175d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:33.225374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b42b28-18df-4469-8598-07ab163175d5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:33.231165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb93f01-cc9d-459b-8a23-f4fb316a7dbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:33.232102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb93f01-cc9d-459b-8a23-f4fb316a7dbe service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:33.232102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb93f01-cc9d-459b-8a23-f4fb316a7dbe service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:33.232355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb93f01-cc9d-459b-8a23-f4fb316a7dbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:33.245704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb93f01-cc9d-459b-8a23-f4fb316a7dbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:33.248692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb93f01-cc9d-459b-8a23-f4fb316a7dbe service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:33.264907 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3808/7560] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:16:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:16:33.929530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39201fce-84d8-4a8d-85ac-aa87ea28499f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:33.931386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-39201fce-84d8-4a8d-85ac-aa87ea28499f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:33.946406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39201fce-84d8-4a8d-85ac-aa87ea28499f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:33.951565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1746d-ec50-470c-ab20-62b5428a6c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:33.952228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1746d-ec50-470c-ab20-62b5428a6c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:33.952513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1746d-ec50-470c-ab20-62b5428a6c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:33.952796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1746d-ec50-470c-ab20-62b5428a6c22 admin admin] PATH_INFO: `/v3/projects/a1dd25b43b47445cb97e9fd7c6a2c5b1` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:33.953971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1746d-ec50-470c-ab20-62b5428a6c22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1dd25b43b47445cb97e9fd7c6a2c5b1)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:33.956077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1746d-ec50-470c-ab20-62b5428a6c22 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:34.020044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-05b1746d-ec50-470c-ab20-62b5428a6c22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1dd25b43b47445cb97e9fd7c6a2c5b1', 'request_id': 'req-05b1746d-ec50-470c-ab20-62b5428a6c22'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:34.043023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-05b1746d-ec50-470c-ab20-62b5428a6c22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1dd25b43b47445cb97e9fd7c6a2c5b1', 'request_id': 'req-05b1746d-ec50-470c-ab20-62b5428a6c22'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:34.059218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-05b1746d-ec50-470c-ab20-62b5428a6c22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1dd25b43b47445cb97e9fd7c6a2c5b1', 'request_id': 'req-05b1746d-ec50-470c-ab20-62b5428a6c22'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:34.102324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-05b1746d-ec50-470c-ab20-62b5428a6c22 admin admin] The token cache is being invalidate because project a1dd25b43b47445cb97e9fd7c6a2c5b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:16:34.102912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-05b1746d-ec50-470c-ab20-62b5428a6c22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:34.104405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-05b1746d-ec50-470c-ab20-62b5428a6c22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:34.107084 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3753/7561] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:16:33 2022] DELETE /identity/v3/projects/a1dd25b43b47445cb97e9fd7c6a2c5b1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:34.112384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb93f01-cc9d-459b-8a23-f4fb316a7dbe service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:34.163102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb93f01-cc9d-459b-8a23-f4fb316a7dbe service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:34.171827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:34.305232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b00af7-b4f1-4b71-88b0-4e8adb6073f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:34.306438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b00af7-b4f1-4b71-88b0-4e8adb6073f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:34.306438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b00af7-b4f1-4b71-88b0-4e8adb6073f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:34.306745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b00af7-b4f1-4b71-88b0-4e8adb6073f2 admin admin] PATH_INFO: `/v3/users/ec9c2469a51940fab3bdc23c44cac11d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:34.307877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b00af7-b4f1-4b71-88b0-4e8adb6073f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec9c2469a51940fab3bdc23c44cac11d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:34.309857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b00af7-b4f1-4b71-88b0-4e8adb6073f2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:34.416717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77b00af7-b4f1-4b71-88b0-4e8adb6073f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec9c2469a51940fab3bdc23c44cac11d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:34.430678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77b00af7-b4f1-4b71-88b0-4e8adb6073f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec9c2469a51940fab3bdc23c44cac11d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:34.449008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77b00af7-b4f1-4b71-88b0-4e8adb6073f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec9c2469a51940fab3bdc23c44cac11d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:34.449865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77b00af7-b4f1-4b71-88b0-4e8adb6073f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec9c2469a51940fab3bdc23c44cac11d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:34.461843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-77b00af7-b4f1-4b71-88b0-4e8adb6073f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec9c2469a51940fab3bdc23c44cac11d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:34.489941 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3809/7562] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:16:34 2022] DELETE /identity/v3/users/ec9c2469a51940fab3bdc23c44cac11d => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:35.392200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1746d-ec50-470c-ab20-62b5428a6c22 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:35.394317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-05b1746d-ec50-470c-ab20-62b5428a6c22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:35.409612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1746d-ec50-470c-ab20-62b5428a6c22 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:35.500331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70f73e-b5d5-44da-92de-07f616232031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:35.501128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70f73e-b5d5-44da-92de-07f616232031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:35.501485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70f73e-b5d5-44da-92de-07f616232031 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:35.502036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70f73e-b5d5-44da-92de-07f616232031 admin admin] PATH_INFO: `/v3/projects/90a969c31ee846ac8daf915407e18287` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:35.504374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70f73e-b5d5-44da-92de-07f616232031 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90a969c31ee846ac8daf915407e18287)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:35.507670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70f73e-b5d5-44da-92de-07f616232031 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:35.602739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ea70f73e-b5d5-44da-92de-07f616232031 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90a969c31ee846ac8daf915407e18287', 'request_id': 'req-ea70f73e-b5d5-44da-92de-07f616232031'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:35.617524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ea70f73e-b5d5-44da-92de-07f616232031 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90a969c31ee846ac8daf915407e18287', 'request_id': 'req-ea70f73e-b5d5-44da-92de-07f616232031'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:35.629249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ea70f73e-b5d5-44da-92de-07f616232031 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90a969c31ee846ac8daf915407e18287', 'request_id': 'req-ea70f73e-b5d5-44da-92de-07f616232031'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:35.642680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:35.650314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ea70f73e-b5d5-44da-92de-07f616232031 admin admin] The token cache is being invalidate because project 90a969c31ee846ac8daf915407e18287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:16:35.651818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ea70f73e-b5d5-44da-92de-07f616232031 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:35.653047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ea70f73e-b5d5-44da-92de-07f616232031 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:35.661182 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3754/7563] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:16:35 2022] DELETE /identity/v3/projects/90a969c31ee846ac8daf915407e18287 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:35.663350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae3595-9941-404c-bc3d-ca1e66aec430 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:35.665243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae3595-9941-404c-bc3d-ca1e66aec430 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:35.665703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae3595-9941-404c-bc3d-ca1e66aec430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:35.701291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-21ae3595-9941-404c-bc3d-ca1e66aec430 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:35.703110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-21ae3595-9941-404c-bc3d-ca1e66aec430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:35.809030 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3810/7564] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:35.817503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864e94b-f1da-45ea-a021-f87c047fa8aa None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:35.817793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864e94b-f1da-45ea-a021-f87c047fa8aa None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:35.818076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864e94b-f1da-45ea-a021-f87c047fa8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:35.850250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0864e94b-f1da-45ea-a021-f87c047fa8aa None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:35.855992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0864e94b-f1da-45ea-a021-f87c047fa8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:35.870398 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3755/7565] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:35.874481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae3595-9941-404c-bc3d-ca1e66aec430 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:35.885362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae3595-9941-404c-bc3d-ca1e66aec430 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:35.890384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed0287-7a89-41b5-8979-97a242b3f38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:35.891206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed0287-7a89-41b5-8979-97a242b3f38f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:35.891582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed0287-7a89-41b5-8979-97a242b3f38f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:35.891953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed0287-7a89-41b5-8979-97a242b3f38f admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:35.892702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed0287-7a89-41b5-8979-97a242b3f38f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:16:35.893104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed0287-7a89-41b5-8979-97a242b3f38f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:35.894805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed0287-7a89-41b5-8979-97a242b3f38f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:35.912712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864e94b-f1da-45ea-a021-f87c047fa8aa None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:35.917234 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3811/7566] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:16:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:16:35.929513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864e94b-f1da-45ea-a021-f87c047fa8aa None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:35.938660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579036c-25c6-42c0-8d09-0d54fccf0214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:35.940105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579036c-25c6-42c0-8d09-0d54fccf0214 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:35.940917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579036c-25c6-42c0-8d09-0d54fccf0214 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:35.941357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579036c-25c6-42c0-8d09-0d54fccf0214 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:35.942434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579036c-25c6-42c0-8d09-0d54fccf0214 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:35.945020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579036c-25c6-42c0-8d09-0d54fccf0214 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:35.988594 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3756/7567] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:16:35 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:16:35.990204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed0287-7a89-41b5-8979-97a242b3f38f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:36.006964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed0287-7a89-41b5-8979-97a242b3f38f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:36.019522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:36.093860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60256a-4f4d-4675-93dc-b2dbe826531d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:36.094507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60256a-4f4d-4675-93dc-b2dbe826531d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:36.094802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60256a-4f4d-4675-93dc-b2dbe826531d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:36.095079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60256a-4f4d-4675-93dc-b2dbe826531d admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:36.096536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60256a-4f4d-4675-93dc-b2dbe826531d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:36.098550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60256a-4f4d-4675-93dc-b2dbe826531d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:36.126686 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3812/7568] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:16:35 2022] POST /identity/v3/users => generated 288 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:16:36.128403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579036c-25c6-42c0-8d09-0d54fccf0214 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:36.148601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579036c-25c6-42c0-8d09-0d54fccf0214 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:36.158994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8303966-032f-484d-b491-bb265f7849d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:36.160261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8303966-032f-484d-b491-bb265f7849d2 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:36.160558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8303966-032f-484d-b491-bb265f7849d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:36.160835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8303966-032f-484d-b491-bb265f7849d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:36.161492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8303966-032f-484d-b491-bb265f7849d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:36.163165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8303966-032f-484d-b491-bb265f7849d2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:36.189452 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3757/7569] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:16:36.192699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60256a-4f4d-4675-93dc-b2dbe826531d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:36.220098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60256a-4f4d-4675-93dc-b2dbe826531d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:36.240717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85894f8f-d8f9-46ff-9ad5-f36f12f8a729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:36.245011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85894f8f-d8f9-46ff-9ad5-f36f12f8a729 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:36.245353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85894f8f-d8f9-46ff-9ad5-f36f12f8a729 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:36.245632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85894f8f-d8f9-46ff-9ad5-f36f12f8a729 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:36.246340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85894f8f-d8f9-46ff-9ad5-f36f12f8a729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:36.248018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85894f8f-d8f9-46ff-9ad5-f36f12f8a729 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:36.291094 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3813/7570] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:16:36.292634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8303966-032f-484d-b491-bb265f7849d2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:36.313650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8303966-032f-484d-b491-bb265f7849d2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:36.333452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa64e0a-0bfd-4531-914e-b388ace8c81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:36.339967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa64e0a-0bfd-4531-914e-b388ace8c81d admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:36.340512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa64e0a-0bfd-4531-914e-b388ace8c81d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:36.340934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa64e0a-0bfd-4531-914e-b388ace8c81d admin admin] PATH_INFO: `/v3/projects/43d00fbe47e7422aa5b769d21b1138f9/users/8961b1d81e444c42b5a735f87de3ff29/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:36.373582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa64e0a-0bfd-4531-914e-b388ace8c81d admin admin] RBAC: Authorizing `identity:create_grant(project_id=43d00fbe47e7422aa5b769d21b1138f9, user_id=8961b1d81e444c42b5a735f87de3ff29, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:36.376601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa64e0a-0bfd-4531-914e-b388ace8c81d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:36.405766 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3758/7571] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:16:36 2022] PUT /identity/v3/projects/43d00fbe47e7422aa5b769d21b1138f9/users/8961b1d81e444c42b5a735f87de3ff29/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:36.417054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cb008-f7ce-4731-b9e5-f2834cc0b06b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:36.417420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cb008-f7ce-4731-b9e5-f2834cc0b06b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:36.417706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cb008-f7ce-4731-b9e5-f2834cc0b06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:36.482611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e62cb008-f7ce-4731-b9e5-f2834cc0b06b None None] MFA Rules not processed for user `8961b1d81e444c42b5a735f87de3ff29`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:36.483913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e62cb008-f7ce-4731-b9e5-f2834cc0b06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:36.652411 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3814/7572] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:36.653743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa64e0a-0bfd-4531-914e-b388ace8c81d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:36.767694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa64e0a-0bfd-4531-914e-b388ace8c81d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:36.776051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee416ac4-e9af-49e4-9fb2-7b7c9260e463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:36.777006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee416ac4-e9af-49e4-9fb2-7b7c9260e463 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:36.777513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee416ac4-e9af-49e4-9fb2-7b7c9260e463 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:36.778002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee416ac4-e9af-49e4-9fb2-7b7c9260e463 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:36.778917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee416ac4-e9af-49e4-9fb2-7b7c9260e463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:36.781241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee416ac4-e9af-49e4-9fb2-7b7c9260e463 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:36.853817 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3759/7573] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:16:36 2022] POST /identity/v3/projects => generated 377 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:16:36.855058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62cb008-f7ce-4731-b9e5-f2834cc0b06b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:36.877822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62cb008-f7ce-4731-b9e5-f2834cc0b06b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:37.015719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34193e-4dde-42a5-a3bb-e7816d8a2755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:37.016019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34193e-4dde-42a5-a3bb-e7816d8a2755 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:37.016307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34193e-4dde-42a5-a3bb-e7816d8a2755 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:37.016606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34193e-4dde-42a5-a3bb-e7816d8a2755 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:37.017861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34193e-4dde-42a5-a3bb-e7816d8a2755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:37.021123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34193e-4dde-42a5-a3bb-e7816d8a2755 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:37.073070 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3815/7574] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:16:36 2022] POST /identity/v3/users => generated 295 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:16:37.078918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee416ac4-e9af-49e4-9fb2-7b7c9260e463 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:37.093840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee416ac4-e9af-49e4-9fb2-7b7c9260e463 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:37.107906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753d206-73c5-4b12-9c6d-42767039476b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:37.109031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753d206-73c5-4b12-9c6d-42767039476b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:37.109420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753d206-73c5-4b12-9c6d-42767039476b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:37.110701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753d206-73c5-4b12-9c6d-42767039476b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:37.111812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753d206-73c5-4b12-9c6d-42767039476b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:37.113832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753d206-73c5-4b12-9c6d-42767039476b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:37.134563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34193e-4dde-42a5-a3bb-e7816d8a2755 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:37.134845 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3760/7575] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:16:37.157899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34193e-4dde-42a5-a3bb-e7816d8a2755 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:37.164642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fba7a7-6e25-4301-bfae-7b4a6d2119fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:37.166339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba7a7-6e25-4301-bfae-7b4a6d2119fb admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:37.166804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba7a7-6e25-4301-bfae-7b4a6d2119fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:37.167183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba7a7-6e25-4301-bfae-7b4a6d2119fb admin admin] PATH_INFO: `/v3/projects/d488228fb1ff41c09b30577b9bbc786c/users/6c060a55ad964d84b010d9b44777051a/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:37.179573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:37.204913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fba7a7-6e25-4301-bfae-7b4a6d2119fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d488228fb1ff41c09b30577b9bbc786c, user_id=6c060a55ad964d84b010d9b44777051a, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:37.213724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fba7a7-6e25-4301-bfae-7b4a6d2119fb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:37.247016 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3816/7576] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:16:37 2022] PUT /identity/v3/projects/d488228fb1ff41c09b30577b9bbc786c/users/6c060a55ad964d84b010d9b44777051a/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:37.251750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753d206-73c5-4b12-9c6d-42767039476b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:37.277796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753d206-73c5-4b12-9c6d-42767039476b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:37.439013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ee99c-9c77-4ef2-8d1f-b54c29d6488a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:37.440196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ee99c-9c77-4ef2-8d1f-b54c29d6488a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:37.440687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ee99c-9c77-4ef2-8d1f-b54c29d6488a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:37.441147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ee99c-9c77-4ef2-8d1f-b54c29d6488a admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:37.442051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ee99c-9c77-4ef2-8d1f-b54c29d6488a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:37.443909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ee99c-9c77-4ef2-8d1f-b54c29d6488a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:37.468911 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3761/7577] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:16:37.475196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fba7a7-6e25-4301-bfae-7b4a6d2119fb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:37.489595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fba7a7-6e25-4301-bfae-7b4a6d2119fb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:37.510044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f72b7-0292-456e-bbd8-448fd5471642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:37.511066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f72b7-0292-456e-bbd8-448fd5471642 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:37.511567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f72b7-0292-456e-bbd8-448fd5471642 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:37.512080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f72b7-0292-456e-bbd8-448fd5471642 admin admin] PATH_INFO: `/v3/projects/d488228fb1ff41c09b30577b9bbc786c/users/6c060a55ad964d84b010d9b44777051a/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:37.514772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f72b7-0292-456e-bbd8-448fd5471642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d488228fb1ff41c09b30577b9bbc786c, user_id=6c060a55ad964d84b010d9b44777051a, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:37.518530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f72b7-0292-456e-bbd8-448fd5471642 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:37.542539 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3817/7578] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:16:37 2022] PUT /identity/v3/projects/d488228fb1ff41c09b30577b9bbc786c/users/6c060a55ad964d84b010d9b44777051a/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:37.553964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c0f9d-693d-4cae-898d-ffecfd56ca9c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:37.554479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c0f9d-693d-4cae-898d-ffecfd56ca9c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:37.554975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c0f9d-693d-4cae-898d-ffecfd56ca9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:37.571406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:37.612903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b18c0f9d-693d-4cae-898d-ffecfd56ca9c None None] MFA Rules not processed for user `6c060a55ad964d84b010d9b44777051a`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:37.615027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b18c0f9d-693d-4cae-898d-ffecfd56ca9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:37.786413 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3762/7579] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:16:37.825185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f72b7-0292-456e-bbd8-448fd5471642 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:37.828168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-966f72b7-0292-456e-bbd8-448fd5471642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:37.942728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f72b7-0292-456e-bbd8-448fd5471642 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:37.953592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cb0d4-e9b3-4afd-b255-9988a89b3256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:37.954265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cb0d4-e9b3-4afd-b255-9988a89b3256 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:37.954809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cb0d4-e9b3-4afd-b255-9988a89b3256 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:37.955759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cb0d4-e9b3-4afd-b255-9988a89b3256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:37.968925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cb0d4-e9b3-4afd-b255-9988a89b3256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:37.970911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cb0d4-e9b3-4afd-b255-9988a89b3256 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:37.987054 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3818/7580] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:16:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:16:41.595151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8348cd-f610-41f1-8e9c-039fc665c12b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:41.596620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8348cd-f610-41f1-8e9c-039fc665c12b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:41.597126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8348cd-f610-41f1-8e9c-039fc665c12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:41.619496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ca8348cd-f610-41f1-8e9c-039fc665c12b None None] MFA Rules not processed for user `f77486ef55a0469a99c3795b626bee1a`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:41.621120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ca8348cd-f610-41f1-8e9c-039fc665c12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:41.772480 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3763/7581] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:41 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:47.369384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cb0d4-e9b3-4afd-b255-9988a89b3256 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:47.375995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bc8cb0d4-e9b3-4afd-b255-9988a89b3256 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:47.587897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cb0d4-e9b3-4afd-b255-9988a89b3256 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:47.599316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05c401-c742-46e5-9be9-ab7576a9dc8e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:47.600247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05c401-c742-46e5-9be9-ab7576a9dc8e service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:47.600681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05c401-c742-46e5-9be9-ab7576a9dc8e service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:47.601108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05c401-c742-46e5-9be9-ab7576a9dc8e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:47.603579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2d05c401-c742-46e5-9be9-ab7576a9dc8e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:47.624914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05c401-c742-46e5-9be9-ab7576a9dc8e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:47.627269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05c401-c742-46e5-9be9-ab7576a9dc8e service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:47.646949 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3819/7582] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:16:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:16:47.704015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8348cd-f610-41f1-8e9c-039fc665c12b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:47.723924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8348cd-f610-41f1-8e9c-039fc665c12b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:47.764746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c255d61-d4e1-4aa8-a8c7-1965fa1e0c2c tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c060a55ad964d84b010d9b44777051a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488228fb1ff41c09b30577b9bbc786c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:47.766041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c255d61-d4e1-4aa8-a8c7-1965fa1e0c2c tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:47.766552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c255d61-d4e1-4aa8-a8c7-1965fa1e0c2c tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:47.767741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c255d61-d4e1-4aa8-a8c7-1965fa1e0c2c tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:47.768668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c255d61-d4e1-4aa8-a8c7-1965fa1e0c2c tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:16:47.769173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c255d61-d4e1-4aa8-a8c7-1965fa1e0c2c tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:47.772097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c255d61-d4e1-4aa8-a8c7-1965fa1e0c2c tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:47.829202 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3764/7583] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:16:47.833878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05c401-c742-46e5-9be9-ab7576a9dc8e service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:47.856936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05c401-c742-46e5-9be9-ab7576a9dc8e service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:47.874734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dd898-11ed-407a-9fc9-5e08a015f488 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c060a55ad964d84b010d9b44777051a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488228fb1ff41c09b30577b9bbc786c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:47.875620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dd898-11ed-407a-9fc9-5e08a015f488 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:47.875845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dd898-11ed-407a-9fc9-5e08a015f488 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:47.876125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dd898-11ed-407a-9fc9-5e08a015f488 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:47.877334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52dd898-11ed-407a-9fc9-5e08a015f488 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:47.879415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52dd898-11ed-407a-9fc9-5e08a015f488 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:47.929912 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3820/7584] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:16:47 2022] POST /identity/v3/projects => generated 348 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:16:47.934221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c255d61-d4e1-4aa8-a8c7-1965fa1e0c2c tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:47.948375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c255d61-d4e1-4aa8-a8c7-1965fa1e0c2c tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:48.018895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e95e7f-6c86-4f62-9faa-a79c0d31d571 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c060a55ad964d84b010d9b44777051a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488228fb1ff41c09b30577b9bbc786c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:48.019587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e95e7f-6c86-4f62-9faa-a79c0d31d571 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:48.019882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e95e7f-6c86-4f62-9faa-a79c0d31d571 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:48.020513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e95e7f-6c86-4f62-9faa-a79c0d31d571 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:48.021245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e95e7f-6c86-4f62-9faa-a79c0d31d571 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:16:48.021615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e95e7f-6c86-4f62-9faa-a79c0d31d571 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:48.023619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e95e7f-6c86-4f62-9faa-a79c0d31d571 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:48.046928 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3765/7585] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:16:48.556262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dd898-11ed-407a-9fc9-5e08a015f488 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:48.576643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dd898-11ed-407a-9fc9-5e08a015f488 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:48.584533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e6aa4-5852-422f-b41b-7d563ff0fd03 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c060a55ad964d84b010d9b44777051a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd488228fb1ff41c09b30577b9bbc786c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:48.585199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e6aa4-5852-422f-b41b-7d563ff0fd03 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:48.585489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e6aa4-5852-422f-b41b-7d563ff0fd03 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:48.585766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e6aa4-5852-422f-b41b-7d563ff0fd03 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] PATH_INFO: `/v3/projects/477d2eb667234b61b7d040caf5639f48` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:48.587614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e6aa4-5852-422f-b41b-7d563ff0fd03 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] RBAC: Authorizing `identity:delete_project(project_id=477d2eb667234b61b7d040caf5639f48)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:48.592284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e6aa4-5852-422f-b41b-7d563ff0fd03 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:48.647583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:48.681003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-992e6aa4-5852-422f-b41b-7d563ff0fd03 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '477d2eb667234b61b7d040caf5639f48', 'request_id': 'req-992e6aa4-5852-422f-b41b-7d563ff0fd03'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:48.693049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-992e6aa4-5852-422f-b41b-7d563ff0fd03 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '477d2eb667234b61b7d040caf5639f48', 'request_id': 'req-992e6aa4-5852-422f-b41b-7d563ff0fd03'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:48.708975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-992e6aa4-5852-422f-b41b-7d563ff0fd03 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '477d2eb667234b61b7d040caf5639f48', 'request_id': 'req-992e6aa4-5852-422f-b41b-7d563ff0fd03'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:48.741388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-992e6aa4-5852-422f-b41b-7d563ff0fd03 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] The token cache is being invalidate because project 477d2eb667234b61b7d040caf5639f48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:16:48.741803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-992e6aa4-5852-422f-b41b-7d563ff0fd03 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:48.742621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-992e6aa4-5852-422f-b41b-7d563ff0fd03 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:48.745686 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3821/7586] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:16:48 2022] DELETE /identity/v3/projects/477d2eb667234b61b7d040caf5639f48 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:48.772639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e95e7f-6c86-4f62-9faa-a79c0d31d571 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:48.774779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-55e95e7f-6c86-4f62-9faa-a79c0d31d571 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:48.809352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:48.883644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e95e7f-6c86-4f62-9faa-a79c0d31d571 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:48.977987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c13112-01bc-4edd-8ea4-577ecb097d22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:48.978814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c13112-01bc-4edd-8ea4-577ecb097d22 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:48.979185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c13112-01bc-4edd-8ea4-577ecb097d22 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:48.979357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c13112-01bc-4edd-8ea4-577ecb097d22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:48.982234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-32c13112-01bc-4edd-8ea4-577ecb097d22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:49.066469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c13112-01bc-4edd-8ea4-577ecb097d22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:49.068255 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c13112-01bc-4edd-8ea4-577ecb097d22 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:49.081885 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3766/7587] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:16:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:16:51.596009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e6aa4-5852-422f-b41b-7d563ff0fd03 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:51.600560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-992e6aa4-5852-422f-b41b-7d563ff0fd03 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:51.722382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e6aa4-5852-422f-b41b-7d563ff0fd03 tempest-RoutersIpV6AdminTest-1086575812 tempest-RoutersIpV6AdminTest-1086575812-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:51.731420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4fbe4-b9cc-477c-b715-85def8f6df6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:51.732411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4fbe4-b9cc-477c-b715-85def8f6df6e service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:51.732700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4fbe4-b9cc-477c-b715-85def8f6df6e service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:51.732976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4fbe4-b9cc-477c-b715-85def8f6df6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:51.735970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-79a4fbe4-b9cc-477c-b715-85def8f6df6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:51.758868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a4fbe4-b9cc-477c-b715-85def8f6df6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:51.760493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a4fbe4-b9cc-477c-b715-85def8f6df6e service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:51.779682 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3822/7588] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:16:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:16:53.288011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c13112-01bc-4edd-8ea4-577ecb097d22 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:53.289461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-32c13112-01bc-4edd-8ea4-577ecb097d22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:53.388286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c13112-01bc-4edd-8ea4-577ecb097d22 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:53.396036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475a8bd-c1bb-4e3c-9bcb-c7c48ceb2480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:53.396945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475a8bd-c1bb-4e3c-9bcb-c7c48ceb2480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:53.397234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475a8bd-c1bb-4e3c-9bcb-c7c48ceb2480 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:53.397516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475a8bd-c1bb-4e3c-9bcb-c7c48ceb2480 admin admin] PATH_INFO: `/v3/users/a554c62c88c94526aea50fe59333aa29` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:53.398751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8475a8bd-c1bb-4e3c-9bcb-c7c48ceb2480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a554c62c88c94526aea50fe59333aa29)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:53.400984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8475a8bd-c1bb-4e3c-9bcb-c7c48ceb2480 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:53.452464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:53.473213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8475a8bd-c1bb-4e3c-9bcb-c7c48ceb2480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a554c62c88c94526aea50fe59333aa29'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:53.474277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8475a8bd-c1bb-4e3c-9bcb-c7c48ceb2480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a554c62c88c94526aea50fe59333aa29'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:53.475123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8475a8bd-c1bb-4e3c-9bcb-c7c48ceb2480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a554c62c88c94526aea50fe59333aa29'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:53.493469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8475a8bd-c1bb-4e3c-9bcb-c7c48ceb2480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a554c62c88c94526aea50fe59333aa29'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:53.511503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8475a8bd-c1bb-4e3c-9bcb-c7c48ceb2480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a554c62c88c94526aea50fe59333aa29'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:53.537891 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3767/7589] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:16:53 2022] DELETE /identity/v3/users/a554c62c88c94526aea50fe59333aa29 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:53.786791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4fbe4-b9cc-477c-b715-85def8f6df6e service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:53.815153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4fbe4-b9cc-477c-b715-85def8f6df6e service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:53.933773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b43955-3d04-40d0-8884-0dd39b54ea76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:53.934633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b43955-3d04-40d0-8884-0dd39b54ea76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:53.935052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b43955-3d04-40d0-8884-0dd39b54ea76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:53.935395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b43955-3d04-40d0-8884-0dd39b54ea76 admin admin] PATH_INFO: `/v3/projects/ee4f57ea97c64af5958543447819ce8d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:53.936781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b43955-3d04-40d0-8884-0dd39b54ea76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee4f57ea97c64af5958543447819ce8d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:53.940651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b43955-3d04-40d0-8884-0dd39b54ea76 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:54.003931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:54.007324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f5b43955-3d04-40d0-8884-0dd39b54ea76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee4f57ea97c64af5958543447819ce8d', 'request_id': 'req-f5b43955-3d04-40d0-8884-0dd39b54ea76'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:54.017299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f5b43955-3d04-40d0-8884-0dd39b54ea76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee4f57ea97c64af5958543447819ce8d', 'request_id': 'req-f5b43955-3d04-40d0-8884-0dd39b54ea76'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:54.026164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f5b43955-3d04-40d0-8884-0dd39b54ea76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee4f57ea97c64af5958543447819ce8d', 'request_id': 'req-f5b43955-3d04-40d0-8884-0dd39b54ea76'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:54.046293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f5b43955-3d04-40d0-8884-0dd39b54ea76 admin admin] The token cache is being invalidate because project ee4f57ea97c64af5958543447819ce8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:16:54.049661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f5b43955-3d04-40d0-8884-0dd39b54ea76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:54.049661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f5b43955-3d04-40d0-8884-0dd39b54ea76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:54.053643 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3823/7590] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:16:53 2022] DELETE /identity/v3/projects/ee4f57ea97c64af5958543447819ce8d => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:54.055359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598d152-3fc0-4c82-b1d7-a1244329e228 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:54.056400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598d152-3fc0-4c82-b1d7-a1244329e228 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:54.056400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598d152-3fc0-4c82-b1d7-a1244329e228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:54.112269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7598d152-3fc0-4c82-b1d7-a1244329e228 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:54.113984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7598d152-3fc0-4c82-b1d7-a1244329e228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:54.221552 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3768/7591] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:16:54.229176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e8958-81b5-4170-92c0-844110f6dbeb None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:54.229647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e8958-81b5-4170-92c0-844110f6dbeb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:54.230053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e8958-81b5-4170-92c0-844110f6dbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:54.265478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-537e8958-81b5-4170-92c0-844110f6dbeb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:54.266818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-537e8958-81b5-4170-92c0-844110f6dbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:54.299995 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3824/7592] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:54.304115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598d152-3fc0-4c82-b1d7-a1244329e228 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:54.318173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598d152-3fc0-4c82-b1d7-a1244329e228 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:54.330756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac021f-0164-4991-9dbc-eb02d7a615c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:54.331675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac021f-0164-4991-9dbc-eb02d7a615c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:54.335606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac021f-0164-4991-9dbc-eb02d7a615c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:54.336080 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac021f-0164-4991-9dbc-eb02d7a615c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:54.337108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac021f-0164-4991-9dbc-eb02d7a615c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:16:54.337576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac021f-0164-4991-9dbc-eb02d7a615c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:54.339423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac021f-0164-4991-9dbc-eb02d7a615c7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:54.366244 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3769/7593] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:16:54.370142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e8958-81b5-4170-92c0-844110f6dbeb None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:54.384734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e8958-81b5-4170-92c0-844110f6dbeb None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:54.391425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f013829f-3fa2-4806-9f15-e158eb6be1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:54.413002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013829f-3fa2-4806-9f15-e158eb6be1fe admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:54.414848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013829f-3fa2-4806-9f15-e158eb6be1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:54.420231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013829f-3fa2-4806-9f15-e158eb6be1fe admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:54.421709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f013829f-3fa2-4806-9f15-e158eb6be1fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:54.437739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f013829f-3fa2-4806-9f15-e158eb6be1fe admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:54.513744 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3825/7594] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:16:54 2022] POST /identity/v3/projects => generated 391 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:16:54.517668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac021f-0164-4991-9dbc-eb02d7a615c7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:54.529729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac021f-0164-4991-9dbc-eb02d7a615c7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:54.664088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbaa24-3043-4e26-8e48-c1ec98f97517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:54.664703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbaa24-3043-4e26-8e48-c1ec98f97517 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:54.664980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbaa24-3043-4e26-8e48-c1ec98f97517 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:54.665248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbaa24-3043-4e26-8e48-c1ec98f97517 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:54.665872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dbaa24-3043-4e26-8e48-c1ec98f97517 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:54.668012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dbaa24-3043-4e26-8e48-c1ec98f97517 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:54.720880 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3770/7595] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:16:54 2022] POST /identity/v3/users => generated 296 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:16:54.724595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f013829f-3fa2-4806-9f15-e158eb6be1fe admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:54.738569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f013829f-3fa2-4806-9f15-e158eb6be1fe admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:54.744680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b75f03-ecd2-49d8-95c4-a0801724f4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:54.745522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b75f03-ecd2-49d8-95c4-a0801724f4c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:54.745808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b75f03-ecd2-49d8-95c4-a0801724f4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:54.746073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b75f03-ecd2-49d8-95c4-a0801724f4c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:54.746818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b75f03-ecd2-49d8-95c4-a0801724f4c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:54.748505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b75f03-ecd2-49d8-95c4-a0801724f4c7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:54.775644 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3826/7596] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:16:54.776744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbaa24-3043-4e26-8e48-c1ec98f97517 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:54.791614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbaa24-3043-4e26-8e48-c1ec98f97517 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:54.799374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9278df-80da-48a5-87f0-848866f3929b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:54.800372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9278df-80da-48a5-87f0-848866f3929b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:54.801149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9278df-80da-48a5-87f0-848866f3929b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:54.801607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9278df-80da-48a5-87f0-848866f3929b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:54.802595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9278df-80da-48a5-87f0-848866f3929b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:54.805311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9278df-80da-48a5-87f0-848866f3929b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:54.818763 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3771/7597] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:16:54.822942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b75f03-ecd2-49d8-95c4-a0801724f4c7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:54.837425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b75f03-ecd2-49d8-95c4-a0801724f4c7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:54.847122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96408c99-570a-4d66-8079-2112535354aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:54.848384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96408c99-570a-4d66-8079-2112535354aa admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:54.849039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96408c99-570a-4d66-8079-2112535354aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:54.849544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96408c99-570a-4d66-8079-2112535354aa admin admin] PATH_INFO: `/v3/projects/a4c1094d71344db7a70dc610b22bcc08/users/52f6bf6dd22549699fd1398ec8c3e8e6/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:54.908888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96408c99-570a-4d66-8079-2112535354aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4c1094d71344db7a70dc610b22bcc08, user_id=52f6bf6dd22549699fd1398ec8c3e8e6, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:54.913150 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96408c99-570a-4d66-8079-2112535354aa admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:54.937074 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3827/7598] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:16:54 2022] PUT /identity/v3/projects/a4c1094d71344db7a70dc610b22bcc08/users/52f6bf6dd22549699fd1398ec8c3e8e6/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:16:54.945020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd657e-7fa4-4dcf-b3eb-5c7c6f12ecd7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:54.945020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd657e-7fa4-4dcf-b3eb-5c7c6f12ecd7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:54.945020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd657e-7fa4-4dcf-b3eb-5c7c6f12ecd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:54.955331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:54.977816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-67fd657e-7fa4-4dcf-b3eb-5c7c6f12ecd7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:54.979590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-67fd657e-7fa4-4dcf-b3eb-5c7c6f12ecd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:55.089764 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3772/7599] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:55.105272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96408c99-570a-4d66-8079-2112535354aa admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:55.107306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-96408c99-570a-4d66-8079-2112535354aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:55.172993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96408c99-570a-4d66-8079-2112535354aa admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:55.178110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10143e7e-8c42-4220-a45e-06b2b43024ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:55.178630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10143e7e-8c42-4220-a45e-06b2b43024ee service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:55.178905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10143e7e-8c42-4220-a45e-06b2b43024ee service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:55.179179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10143e7e-8c42-4220-a45e-06b2b43024ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:55.189586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10143e7e-8c42-4220-a45e-06b2b43024ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:55.191072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10143e7e-8c42-4220-a45e-06b2b43024ee service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:55.202236 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3828/7600] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:16:55.317935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd657e-7fa4-4dcf-b3eb-5c7c6f12ecd7 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:55.319467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-67fd657e-7fa4-4dcf-b3eb-5c7c6f12ecd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:55.335680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd657e-7fa4-4dcf-b3eb-5c7c6f12ecd7 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:55.343957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3d3d5-f8c5-4561-a07c-bfc0d6000c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:55.346363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3d3d5-f8c5-4561-a07c-bfc0d6000c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:55.346989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3d3d5-f8c5-4561-a07c-bfc0d6000c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:55.347580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3d3d5-f8c5-4561-a07c-bfc0d6000c2b admin admin] PATH_INFO: `/v3/users/71c5be0b64074fa58b525ce69ee0ad1d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:55.352123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b3d3d5-f8c5-4561-a07c-bfc0d6000c2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=71c5be0b64074fa58b525ce69ee0ad1d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:55.355760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b3d3d5-f8c5-4561-a07c-bfc0d6000c2b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:55.415700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-90b3d3d5-f8c5-4561-a07c-bfc0d6000c2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71c5be0b64074fa58b525ce69ee0ad1d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:55.416460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-90b3d3d5-f8c5-4561-a07c-bfc0d6000c2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71c5be0b64074fa58b525ce69ee0ad1d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:55.417160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-90b3d3d5-f8c5-4561-a07c-bfc0d6000c2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71c5be0b64074fa58b525ce69ee0ad1d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:55.427339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-90b3d3d5-f8c5-4561-a07c-bfc0d6000c2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71c5be0b64074fa58b525ce69ee0ad1d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:55.441699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-90b3d3d5-f8c5-4561-a07c-bfc0d6000c2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71c5be0b64074fa58b525ce69ee0ad1d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:55.458088 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3773/7601] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:16:55 2022] DELETE /identity/v3/users/71c5be0b64074fa58b525ce69ee0ad1d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:55.462374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a84fab-0181-4167-b8f9-c073329054fc None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:55.462746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a84fab-0181-4167-b8f9-c073329054fc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:55.462973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a84fab-0181-4167-b8f9-c073329054fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:55.488662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-42a84fab-0181-4167-b8f9-c073329054fc None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:55.489701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-42a84fab-0181-4167-b8f9-c073329054fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:55.572198 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3829/7602] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:55.584151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3d3d5-f8c5-4561-a07c-bfc0d6000c2b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:55.585803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-90b3d3d5-f8c5-4561-a07c-bfc0d6000c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:55.664834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3d3d5-f8c5-4561-a07c-bfc0d6000c2b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:55.670163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c60ee9-f564-4f7d-bf92-799d3e35a70d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:55.671593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c60ee9-f564-4f7d-bf92-799d3e35a70d service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:55.673128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c60ee9-f564-4f7d-bf92-799d3e35a70d service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:55.673504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c60ee9-f564-4f7d-bf92-799d3e35a70d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:55.690728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c60ee9-f564-4f7d-bf92-799d3e35a70d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:55.693369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c60ee9-f564-4f7d-bf92-799d3e35a70d service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:55.707555 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3774/7603] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:16:56.167709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a84fab-0181-4167-b8f9-c073329054fc None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:56.169973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-42a84fab-0181-4167-b8f9-c073329054fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:56.200676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a84fab-0181-4167-b8f9-c073329054fc None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:56.214344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb7c10-92c6-47c0-a47a-fa44704b8951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:56.215527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb7c10-92c6-47c0-a47a-fa44704b8951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:56.216335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb7c10-92c6-47c0-a47a-fa44704b8951 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:56.216724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb7c10-92c6-47c0-a47a-fa44704b8951 admin admin] PATH_INFO: `/v3/projects/5bad539f140a49398ccd85fa28753aab` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:56.219258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb7c10-92c6-47c0-a47a-fa44704b8951 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bad539f140a49398ccd85fa28753aab)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:56.221617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb7c10-92c6-47c0-a47a-fa44704b8951 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:56.243051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:56.327535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7fbb7c10-92c6-47c0-a47a-fa44704b8951 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bad539f140a49398ccd85fa28753aab', 'request_id': 'req-7fbb7c10-92c6-47c0-a47a-fa44704b8951'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:56.343272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7fbb7c10-92c6-47c0-a47a-fa44704b8951 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bad539f140a49398ccd85fa28753aab', 'request_id': 'req-7fbb7c10-92c6-47c0-a47a-fa44704b8951'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:56.356226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7fbb7c10-92c6-47c0-a47a-fa44704b8951 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bad539f140a49398ccd85fa28753aab', 'request_id': 'req-7fbb7c10-92c6-47c0-a47a-fa44704b8951'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:56.397132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7fbb7c10-92c6-47c0-a47a-fa44704b8951 admin admin] The token cache is being invalidate because project 5bad539f140a49398ccd85fa28753aab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:16:56.397965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7fbb7c10-92c6-47c0-a47a-fa44704b8951 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:56.399690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7fbb7c10-92c6-47c0-a47a-fa44704b8951 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:16:56.402427 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3830/7604] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:16:56 2022] DELETE /identity/v3/projects/5bad539f140a49398ccd85fa28753aab => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:56.407127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c403b-524c-4f38-a209-6751893b97b2 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:56.407998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c403b-524c-4f38-a209-6751893b97b2 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:56.408446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c403b-524c-4f38-a209-6751893b97b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:56.447282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-578c403b-524c-4f38-a209-6751893b97b2 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:56.448865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-578c403b-524c-4f38-a209-6751893b97b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:56.466776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:56.568091 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3775/7605] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:56.577141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09cefc-2501-4722-9199-b7b406ed773e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:56.577625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09cefc-2501-4722-9199-b7b406ed773e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:56.578000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09cefc-2501-4722-9199-b7b406ed773e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:56.603793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fe09cefc-2501-4722-9199-b7b406ed773e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:56.606133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fe09cefc-2501-4722-9199-b7b406ed773e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:56.624092 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3831/7606] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:16:56.629377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c403b-524c-4f38-a209-6751893b97b2 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:56.645108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c403b-524c-4f38-a209-6751893b97b2 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:56.667161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec031b-537f-4f25-bb8c-b760fee5f2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:56.670183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec031b-537f-4f25-bb8c-b760fee5f2b9 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:56.670630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec031b-537f-4f25-bb8c-b760fee5f2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:56.670999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec031b-537f-4f25-bb8c-b760fee5f2b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:56.673768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aec031b-537f-4f25-bb8c-b760fee5f2b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:16:56.674211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aec031b-537f-4f25-bb8c-b760fee5f2b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:56.682327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aec031b-537f-4f25-bb8c-b760fee5f2b9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:56.716751 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3776/7607] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:16:56.717643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09cefc-2501-4722-9199-b7b406ed773e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:56.729797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09cefc-2501-4722-9199-b7b406ed773e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:56.741696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c5337-7bfa-4036-8af9-380ce1217815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:56.742669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c5337-7bfa-4036-8af9-380ce1217815 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:56.743067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c5337-7bfa-4036-8af9-380ce1217815 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:56.743354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c5337-7bfa-4036-8af9-380ce1217815 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:56.743995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c5337-7bfa-4036-8af9-380ce1217815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:56.751673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c5337-7bfa-4036-8af9-380ce1217815 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:56.798729 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3832/7608] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:16:56 2022] POST /identity/v3/projects => generated 365 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:16:56.799487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec031b-537f-4f25-bb8c-b760fee5f2b9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:56.812766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec031b-537f-4f25-bb8c-b760fee5f2b9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:56.910859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ba8e7-9d7c-4051-acac-022d7be375f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:56.911503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ba8e7-9d7c-4051-acac-022d7be375f1 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:56.911844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ba8e7-9d7c-4051-acac-022d7be375f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:56.912380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ba8e7-9d7c-4051-acac-022d7be375f1 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:56.913081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4ba8e7-9d7c-4051-acac-022d7be375f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:56.915852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4ba8e7-9d7c-4051-acac-022d7be375f1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:56.956086 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3777/7609] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:16:56 2022] POST /identity/v3/users => generated 283 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:16:56.956580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c5337-7bfa-4036-8af9-380ce1217815 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:56.971359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c5337-7bfa-4036-8af9-380ce1217815 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:56.979201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9ce39-04be-4817-a770-3745ad78943e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:56.979924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9ce39-04be-4817-a770-3745ad78943e admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:56.980283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9ce39-04be-4817-a770-3745ad78943e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:56.980665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9ce39-04be-4817-a770-3745ad78943e admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:56.981399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e9ce39-04be-4817-a770-3745ad78943e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:56.983185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e9ce39-04be-4817-a770-3745ad78943e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:56.998437 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3833/7610] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:16:57.002717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ba8e7-9d7c-4051-acac-022d7be375f1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:57.014397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ba8e7-9d7c-4051-acac-022d7be375f1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:57.019939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9733f-4bf0-4475-bb46-2a8d9ef5378b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:57.020630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9733f-4bf0-4475-bb46-2a8d9ef5378b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:57.021018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9733f-4bf0-4475-bb46-2a8d9ef5378b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:57.021333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9733f-4bf0-4475-bb46-2a8d9ef5378b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:57.022060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9733f-4bf0-4475-bb46-2a8d9ef5378b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:57.023798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9733f-4bf0-4475-bb46-2a8d9ef5378b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:57.056549 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3778/7611] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:16:57.058452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9ce39-04be-4817-a770-3745ad78943e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:57.076573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9ce39-04be-4817-a770-3745ad78943e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:57.085340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec3327-8339-47d0-a51a-2372ac9b1568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:57.086399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec3327-8339-47d0-a51a-2372ac9b1568 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:57.086702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec3327-8339-47d0-a51a-2372ac9b1568 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:57.087054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec3327-8339-47d0-a51a-2372ac9b1568 admin admin] PATH_INFO: `/v3/projects/919ae1be1ab1486389836316fd2e367f/users/0243052320b649a98e1329ca4014bdc8/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:57.114597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec3327-8339-47d0-a51a-2372ac9b1568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=919ae1be1ab1486389836316fd2e367f, user_id=0243052320b649a98e1329ca4014bdc8, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:57.117374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec3327-8339-47d0-a51a-2372ac9b1568 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:57.155005 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3834/7612] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:16:57 2022] PUT /identity/v3/projects/919ae1be1ab1486389836316fd2e367f/users/0243052320b649a98e1329ca4014bdc8/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:16:57.156180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e87f26-3bd1-4e1e-82e6-a1ba62c29e5f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:57.156405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e87f26-3bd1-4e1e-82e6-a1ba62c29e5f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:57.156881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e87f26-3bd1-4e1e-82e6-a1ba62c29e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:57.194303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c4e87f26-3bd1-4e1e-82e6-a1ba62c29e5f None None] MFA Rules not processed for user `0243052320b649a98e1329ca4014bdc8`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:16:57.196591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c4e87f26-3bd1-4e1e-82e6-a1ba62c29e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:57.307949 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3779/7613] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:16:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:16:57.644843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec3327-8339-47d0-a51a-2372ac9b1568 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:16:57.646899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-79ec3327-8339-47d0-a51a-2372ac9b1568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:16:57.795201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec3327-8339-47d0-a51a-2372ac9b1568 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:16:57.800552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7f449-1405-458d-b947-2d132e8ab7d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:16:57.801193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7f449-1405-458d-b947-2d132e8ab7d8 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:16:57.801475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7f449-1405-458d-b947-2d132e8ab7d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:16:57.802151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7f449-1405-458d-b947-2d132e8ab7d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:16:57.833557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de7f449-1405-458d-b947-2d132e8ab7d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:16:57.836506 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de7f449-1405-458d-b947-2d132e8ab7d8 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:16:57.849414 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3835/7614] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:16:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:17:01.534522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc6b9a-6d2f-4e21-a70b-a8ef5a35b012 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:01.535096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc6b9a-6d2f-4e21-a70b-a8ef5a35b012 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:01.535096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc6b9a-6d2f-4e21-a70b-a8ef5a35b012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:01.573422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3dbc6b9a-6d2f-4e21-a70b-a8ef5a35b012 None None] MFA Rules not processed for user `52f6bf6dd22549699fd1398ec8c3e8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:17:01.575350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3dbc6b9a-6d2f-4e21-a70b-a8ef5a35b012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:01.581440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:01.686895 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3780/7615] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:17:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:17:01.782779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896bc4d-f13e-4871-b415-693bddf19a31 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:01.783346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896bc4d-f13e-4871-b415-693bddf19a31 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:01.783346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896bc4d-f13e-4871-b415-693bddf19a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:01.809025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5896bc4d-f13e-4871-b415-693bddf19a31 None None] MFA Rules not processed for user `52f6bf6dd22549699fd1398ec8c3e8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:17:01.810622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5896bc4d-f13e-4871-b415-693bddf19a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:01.819042 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3836/7616] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:17:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:17:01.831329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc6b9a-6d2f-4e21-a70b-a8ef5a35b012 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:01.832800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3dbc6b9a-6d2f-4e21-a70b-a8ef5a35b012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:01.922653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc6b9a-6d2f-4e21-a70b-a8ef5a35b012 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:01.928673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd10a2a-3f52-4c92-a3a7-4410e8d4831c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:01.929467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd10a2a-3f52-4c92-a3a7-4410e8d4831c service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:01.929902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd10a2a-3f52-4c92-a3a7-4410e8d4831c service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:01.930324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd10a2a-3f52-4c92-a3a7-4410e8d4831c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:01.941835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd10a2a-3f52-4c92-a3a7-4410e8d4831c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:01.943647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd10a2a-3f52-4c92-a3a7-4410e8d4831c service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:01.954926 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3781/7617] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:17:01 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:17:11.798917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:11.949055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=59 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:13.437297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4740b-4805-48c0-8a29-1c4eeb2839dc None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:13.438453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4740b-4805-48c0-8a29-1c4eeb2839dc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:13.439318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4740b-4805-48c0-8a29-1c4eeb2839dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:13.461804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-70e4740b-4805-48c0-8a29-1c4eeb2839dc None None] MFA Rules not processed for user `3066b652821f47f6909892c9b04f4236`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:17:13.463102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-70e4740b-4805-48c0-8a29-1c4eeb2839dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:13.531934 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3837/7618] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:17:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:17:23.524985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=25 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:30.516738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd10a2a-3f52-4c92-a3a7-4410e8d4831c service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:30.533067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0cd10a2a-3f52-4c92-a3a7-4410e8d4831c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:30.622692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd10a2a-3f52-4c92-a3a7-4410e8d4831c service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:30.631192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5a135-5060-4273-aaeb-2a86750fe7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:30.632923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5a135-5060-4273-aaeb-2a86750fe7e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:30.633430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5a135-5060-4273-aaeb-2a86750fe7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:30.633843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5a135-5060-4273-aaeb-2a86750fe7e1 admin admin] PATH_INFO: `/v3/users/52f6bf6dd22549699fd1398ec8c3e8e6` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:30.635303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5a135-5060-4273-aaeb-2a86750fe7e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52f6bf6dd22549699fd1398ec8c3e8e6)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:30.637605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5a135-5060-4273-aaeb-2a86750fe7e1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:30.700557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-82c5a135-5060-4273-aaeb-2a86750fe7e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52f6bf6dd22549699fd1398ec8c3e8e6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:30.701229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-82c5a135-5060-4273-aaeb-2a86750fe7e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52f6bf6dd22549699fd1398ec8c3e8e6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:30.701837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-82c5a135-5060-4273-aaeb-2a86750fe7e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52f6bf6dd22549699fd1398ec8c3e8e6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:30.711954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-82c5a135-5060-4273-aaeb-2a86750fe7e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52f6bf6dd22549699fd1398ec8c3e8e6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:30.728052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-82c5a135-5060-4273-aaeb-2a86750fe7e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52f6bf6dd22549699fd1398ec8c3e8e6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:30.745249 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3782/7619] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:17:30 2022] DELETE /identity/v3/users/52f6bf6dd22549699fd1398ec8c3e8e6 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:17:31.197666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4740b-4805-48c0-8a29-1c4eeb2839dc None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:31.217964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4740b-4805-48c0-8a29-1c4eeb2839dc None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:31.327077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469f085-3ae1-472e-a984-5baebe6e6d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:31.329101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469f085-3ae1-472e-a984-5baebe6e6d9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:31.329609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469f085-3ae1-472e-a984-5baebe6e6d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:31.330169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469f085-3ae1-472e-a984-5baebe6e6d9f admin admin] PATH_INFO: `/v3/projects/a4c1094d71344db7a70dc610b22bcc08` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:31.331665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469f085-3ae1-472e-a984-5baebe6e6d9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4c1094d71344db7a70dc610b22bcc08)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:31.334265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469f085-3ae1-472e-a984-5baebe6e6d9f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:31.399499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6469f085-3ae1-472e-a984-5baebe6e6d9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4c1094d71344db7a70dc610b22bcc08', 'request_id': 'req-6469f085-3ae1-472e-a984-5baebe6e6d9f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:31.417002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6469f085-3ae1-472e-a984-5baebe6e6d9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4c1094d71344db7a70dc610b22bcc08', 'request_id': 'req-6469f085-3ae1-472e-a984-5baebe6e6d9f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:31.426733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6469f085-3ae1-472e-a984-5baebe6e6d9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4c1094d71344db7a70dc610b22bcc08', 'request_id': 'req-6469f085-3ae1-472e-a984-5baebe6e6d9f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:31.450691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6469f085-3ae1-472e-a984-5baebe6e6d9f admin admin] The token cache is being invalidate because project a4c1094d71344db7a70dc610b22bcc08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:17:31.451244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6469f085-3ae1-472e-a984-5baebe6e6d9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:31.451961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6469f085-3ae1-472e-a984-5baebe6e6d9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:31.454299 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3838/7620] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:17:31 2022] DELETE /identity/v3/projects/a4c1094d71344db7a70dc610b22bcc08 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:17:31.459206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a88e8-c01f-4871-a66d-e53a369b78ff None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:31.459757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a88e8-c01f-4871-a66d-e53a369b78ff None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:31.460252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a88e8-c01f-4871-a66d-e53a369b78ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:31.529383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4c6a88e8-c01f-4871-a66d-e53a369b78ff None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:17:31.532099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4c6a88e8-c01f-4871-a66d-e53a369b78ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:31.613122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:31.626720 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3783/7621] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:17:31.634207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263bc44-28c9-4d43-8f79-cc4756d2a4ce None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:31.634512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263bc44-28c9-4d43-8f79-cc4756d2a4ce None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:31.634808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263bc44-28c9-4d43-8f79-cc4756d2a4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:31.663748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4263bc44-28c9-4d43-8f79-cc4756d2a4ce None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:17:31.665156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4263bc44-28c9-4d43-8f79-cc4756d2a4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:31.679777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a88e8-c01f-4871-a66d-e53a369b78ff None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:31.685564 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3839/7622] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:17:31.692831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a88e8-c01f-4871-a66d-e53a369b78ff None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:31.697991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969e851-4b52-47e9-8b37-066e228c05be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:31.699633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969e851-4b52-47e9-8b37-066e228c05be admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:31.699633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969e851-4b52-47e9-8b37-066e228c05be admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:31.699895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969e851-4b52-47e9-8b37-066e228c05be admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:31.700586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8969e851-4b52-47e9-8b37-066e228c05be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:17:31.700758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8969e851-4b52-47e9-8b37-066e228c05be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:31.702401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8969e851-4b52-47e9-8b37-066e228c05be admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:31.715579 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3784/7623] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:17:31.720620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263bc44-28c9-4d43-8f79-cc4756d2a4ce None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:31.730822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263bc44-28c9-4d43-8f79-cc4756d2a4ce None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:31.737651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60044eb9-f6a6-455d-b33c-fb75f47769f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:31.738683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60044eb9-f6a6-455d-b33c-fb75f47769f6 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:31.739018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60044eb9-f6a6-455d-b33c-fb75f47769f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:31.739421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60044eb9-f6a6-455d-b33c-fb75f47769f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:31.740304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60044eb9-f6a6-455d-b33c-fb75f47769f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:31.743501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60044eb9-f6a6-455d-b33c-fb75f47769f6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:31.801718 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3840/7624] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:17:31 2022] POST /identity/v3/projects => generated 379 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:17:31.806166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969e851-4b52-47e9-8b37-066e228c05be admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:31.826342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969e851-4b52-47e9-8b37-066e228c05be admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:31.915762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8e9df-e7a8-44ea-839f-111f03ca05fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:31.916817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8e9df-e7a8-44ea-839f-111f03ca05fe admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:31.917269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8e9df-e7a8-44ea-839f-111f03ca05fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:31.917859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8e9df-e7a8-44ea-839f-111f03ca05fe admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:31.920329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8e9df-e7a8-44ea-839f-111f03ca05fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:31.922973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8e9df-e7a8-44ea-839f-111f03ca05fe admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:31.955745 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3785/7625] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:17:31 2022] POST /identity/v3/users => generated 290 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:17:31.957064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60044eb9-f6a6-455d-b33c-fb75f47769f6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:31.976413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60044eb9-f6a6-455d-b33c-fb75f47769f6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:31.993635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3c612-8ca8-4d82-95f0-5efa661f87ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:31.994267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3c612-8ca8-4d82-95f0-5efa661f87ce admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:31.994572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3c612-8ca8-4d82-95f0-5efa661f87ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:31.994869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3c612-8ca8-4d82-95f0-5efa661f87ce admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:31.995499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3c612-8ca8-4d82-95f0-5efa661f87ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:31.997099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3c612-8ca8-4d82-95f0-5efa661f87ce admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:32.035892 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3841/7626] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:17:32.040465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8e9df-e7a8-44ea-839f-111f03ca05fe admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:32.053143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8e9df-e7a8-44ea-839f-111f03ca05fe admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:32.071059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c400bd7-ec8a-45e0-8070-6f8002159830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:32.071948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c400bd7-ec8a-45e0-8070-6f8002159830 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:32.072790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c400bd7-ec8a-45e0-8070-6f8002159830 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:32.073651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c400bd7-ec8a-45e0-8070-6f8002159830 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:32.074081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c400bd7-ec8a-45e0-8070-6f8002159830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:32.076639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c400bd7-ec8a-45e0-8070-6f8002159830 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:32.108906 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3786/7627] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:17:32.112989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3c612-8ca8-4d82-95f0-5efa661f87ce admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:32.125459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3c612-8ca8-4d82-95f0-5efa661f87ce admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:32.132738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25d6b1-cf76-4987-bf97-f31487d5a352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:32.132738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25d6b1-cf76-4987-bf97-f31487d5a352 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:32.133094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25d6b1-cf76-4987-bf97-f31487d5a352 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:32.134156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25d6b1-cf76-4987-bf97-f31487d5a352 admin admin] PATH_INFO: `/v3/projects/9e09544743724cf18d2ab2aeb4180d81/users/6ac020dc323042c48ccce2a8826bc6be/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:32.151906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25d6b1-cf76-4987-bf97-f31487d5a352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e09544743724cf18d2ab2aeb4180d81, user_id=6ac020dc323042c48ccce2a8826bc6be, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:32.154801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25d6b1-cf76-4987-bf97-f31487d5a352 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:32.180117 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3842/7628] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:17:32 2022] PUT /identity/v3/projects/9e09544743724cf18d2ab2aeb4180d81/users/6ac020dc323042c48ccce2a8826bc6be/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:17:32.181692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08d7a0-bac3-4c10-b287-227d9c7ebcdb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:32.181981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08d7a0-bac3-4c10-b287-227d9c7ebcdb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:32.182257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08d7a0-bac3-4c10-b287-227d9c7ebcdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:32.219535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3d08d7a0-bac3-4c10-b287-227d9c7ebcdb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:17:32.220840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3d08d7a0-bac3-4c10-b287-227d9c7ebcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:32.294521 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3787/7629] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:17:32.309214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25d6b1-cf76-4987-bf97-f31487d5a352 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:32.310918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2d25d6b1-cf76-4987-bf97-f31487d5a352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:32.348104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:32.392681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25d6b1-cf76-4987-bf97-f31487d5a352 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:32.396818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3331cc8-b0d2-4c7f-bf64-67390f1356b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:32.397808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3331cc8-b0d2-4c7f-bf64-67390f1356b8 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:32.401562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3331cc8-b0d2-4c7f-bf64-67390f1356b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:32.401976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3331cc8-b0d2-4c7f-bf64-67390f1356b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:32.412883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3331cc8-b0d2-4c7f-bf64-67390f1356b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:32.414515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3331cc8-b0d2-4c7f-bf64-67390f1356b8 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:32.431281 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3843/7630] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:17:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:17:34.754961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08d7a0-bac3-4c10-b287-227d9c7ebcdb None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:34.757120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3d08d7a0-bac3-4c10-b287-227d9c7ebcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:34.780450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08d7a0-bac3-4c10-b287-227d9c7ebcdb None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:34.784790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb72a5-924f-4fc1-8725-105cf8cc99bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:34.786475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb72a5-924f-4fc1-8725-105cf8cc99bb service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:34.786965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb72a5-924f-4fc1-8725-105cf8cc99bb service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:34.787400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb72a5-924f-4fc1-8725-105cf8cc99bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:34.790197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e6eb72a5-924f-4fc1-8725-105cf8cc99bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:34.804079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb72a5-924f-4fc1-8725-105cf8cc99bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:34.806427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb72a5-924f-4fc1-8725-105cf8cc99bb service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:34.820503 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3788/7631] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:17:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:17:35.467908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3331cc8-b0d2-4c7f-bf64-67390f1356b8 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:35.469413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a3331cc8-b0d2-4c7f-bf64-67390f1356b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:35.484008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3331cc8-b0d2-4c7f-bf64-67390f1356b8 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:35.500896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464986c5-ad37-4927-9d3b-c570dafaffe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:35.502334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464986c5-ad37-4927-9d3b-c570dafaffe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:35.503766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464986c5-ad37-4927-9d3b-c570dafaffe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:35.504228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464986c5-ad37-4927-9d3b-c570dafaffe5 admin admin] PATH_INFO: `/v3/users/8961b1d81e444c42b5a735f87de3ff29` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:35.511019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464986c5-ad37-4927-9d3b-c570dafaffe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8961b1d81e444c42b5a735f87de3ff29)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:35.515132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464986c5-ad37-4927-9d3b-c570dafaffe5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:35.582945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb72a5-924f-4fc1-8725-105cf8cc99bb service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:35.602041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb72a5-924f-4fc1-8725-105cf8cc99bb service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:35.613696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db86d9-e8f4-427f-a50f-2d24c3c4f5fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:35.614937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db86d9-e8f4-427f-a50f-2d24c3c4f5fb service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:35.620292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db86d9-e8f4-427f-a50f-2d24c3c4f5fb service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:35.620785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db86d9-e8f4-427f-a50f-2d24c3c4f5fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:35.627737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-36db86d9-e8f4-427f-a50f-2d24c3c4f5fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:35.628870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-464986c5-ad37-4927-9d3b-c570dafaffe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8961b1d81e444c42b5a735f87de3ff29'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:35.656943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-464986c5-ad37-4927-9d3b-c570dafaffe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8961b1d81e444c42b5a735f87de3ff29'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:35.681260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-464986c5-ad37-4927-9d3b-c570dafaffe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8961b1d81e444c42b5a735f87de3ff29'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:35.682046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-464986c5-ad37-4927-9d3b-c570dafaffe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8961b1d81e444c42b5a735f87de3ff29'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:35.695735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-464986c5-ad37-4927-9d3b-c570dafaffe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8961b1d81e444c42b5a735f87de3ff29'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:35.708310 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3844/7632] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:17:35 2022] DELETE /identity/v3/users/8961b1d81e444c42b5a735f87de3ff29 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:17:35.710065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee82d3f-e286-4bbe-9d01-900013b40818 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:35.710475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee82d3f-e286-4bbe-9d01-900013b40818 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:35.711387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee82d3f-e286-4bbe-9d01-900013b40818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:35.738886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:35.741165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-dee82d3f-e286-4bbe-9d01-900013b40818 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:17:35.743609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-dee82d3f-e286-4bbe-9d01-900013b40818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:35.831156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db86d9-e8f4-427f-a50f-2d24c3c4f5fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:35.833573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db86d9-e8f4-427f-a50f-2d24c3c4f5fb service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:35.846754 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3845/7633] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:17:35.855167 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3789/7634] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:17:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:17:35.863169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee82d3f-e286-4bbe-9d01-900013b40818 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:35.865683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-dee82d3f-e286-4bbe-9d01-900013b40818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:35.940739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:35.969425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee82d3f-e286-4bbe-9d01-900013b40818 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:35.978883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a9465-1446-43ab-84e9-5e3780c22b40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:35.980198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a9465-1446-43ab-84e9-5e3780c22b40 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:35.981847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a9465-1446-43ab-84e9-5e3780c22b40 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:35.981847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a9465-1446-43ab-84e9-5e3780c22b40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:35.992828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a9465-1446-43ab-84e9-5e3780c22b40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:35.994586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a9465-1446-43ab-84e9-5e3780c22b40 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:36.007314 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3846/7635] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:17:36.847671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db86d9-e8f4-427f-a50f-2d24c3c4f5fb service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:36.871975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db86d9-e8f4-427f-a50f-2d24c3c4f5fb service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:36.881133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3377d0c-31ac-4632-b578-9d4a48c7e69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:36.882825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3377d0c-31ac-4632-b578-9d4a48c7e69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:36.883337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3377d0c-31ac-4632-b578-9d4a48c7e69e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:36.883792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3377d0c-31ac-4632-b578-9d4a48c7e69e admin admin] PATH_INFO: `/v3/projects/43d00fbe47e7422aa5b769d21b1138f9` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:36.890556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3377d0c-31ac-4632-b578-9d4a48c7e69e admin admin] RBAC: Authorizing `identity:delete_project(project_id=43d00fbe47e7422aa5b769d21b1138f9)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:36.893551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3377d0c-31ac-4632-b578-9d4a48c7e69e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:37.017444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b3377d0c-31ac-4632-b578-9d4a48c7e69e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43d00fbe47e7422aa5b769d21b1138f9', 'request_id': 'req-b3377d0c-31ac-4632-b578-9d4a48c7e69e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:37.051123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b3377d0c-31ac-4632-b578-9d4a48c7e69e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43d00fbe47e7422aa5b769d21b1138f9', 'request_id': 'req-b3377d0c-31ac-4632-b578-9d4a48c7e69e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:37.065191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b3377d0c-31ac-4632-b578-9d4a48c7e69e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43d00fbe47e7422aa5b769d21b1138f9', 'request_id': 'req-b3377d0c-31ac-4632-b578-9d4a48c7e69e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:37.112882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b3377d0c-31ac-4632-b578-9d4a48c7e69e admin admin] The token cache is being invalidate because project 43d00fbe47e7422aa5b769d21b1138f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:17:37.115147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b3377d0c-31ac-4632-b578-9d4a48c7e69e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:37.119154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b3377d0c-31ac-4632-b578-9d4a48c7e69e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:37.124045 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3790/7636] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:17:36 2022] DELETE /identity/v3/projects/43d00fbe47e7422aa5b769d21b1138f9 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:17:37.127355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a9465-1446-43ab-84e9-5e3780c22b40 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:37.128870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-075a9465-1446-43ab-84e9-5e3780c22b40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:37.152070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a9465-1446-43ab-84e9-5e3780c22b40 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:37.262998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac44a23-28d3-48eb-9ff1-7bcec5387550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:37.263804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac44a23-28d3-48eb-9ff1-7bcec5387550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:37.264036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac44a23-28d3-48eb-9ff1-7bcec5387550 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:37.264309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac44a23-28d3-48eb-9ff1-7bcec5387550 admin admin] PATH_INFO: `/v3/users/6c060a55ad964d84b010d9b44777051a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:37.267954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac44a23-28d3-48eb-9ff1-7bcec5387550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c060a55ad964d84b010d9b44777051a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:37.269971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac44a23-28d3-48eb-9ff1-7bcec5387550 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:37.411300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1ac44a23-28d3-48eb-9ff1-7bcec5387550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c060a55ad964d84b010d9b44777051a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:37.425289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1ac44a23-28d3-48eb-9ff1-7bcec5387550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c060a55ad964d84b010d9b44777051a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:37.453048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1ac44a23-28d3-48eb-9ff1-7bcec5387550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c060a55ad964d84b010d9b44777051a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:37.454405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1ac44a23-28d3-48eb-9ff1-7bcec5387550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c060a55ad964d84b010d9b44777051a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:37.469644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-1ac44a23-28d3-48eb-9ff1-7bcec5387550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c060a55ad964d84b010d9b44777051a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:37.478384 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3847/7637] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:17:37 2022] DELETE /identity/v3/users/6c060a55ad964d84b010d9b44777051a => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:17:38.805384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3377d0c-31ac-4632-b578-9d4a48c7e69e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:38.808645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b3377d0c-31ac-4632-b578-9d4a48c7e69e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:38.837543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3377d0c-31ac-4632-b578-9d4a48c7e69e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:39.000742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274be063-1b81-4365-80c0-9d4dc584a897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:39.001798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274be063-1b81-4365-80c0-9d4dc584a897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:39.002335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274be063-1b81-4365-80c0-9d4dc584a897 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:39.002968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274be063-1b81-4365-80c0-9d4dc584a897 admin admin] PATH_INFO: `/v3/projects/d488228fb1ff41c09b30577b9bbc786c` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:39.006558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274be063-1b81-4365-80c0-9d4dc584a897 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d488228fb1ff41c09b30577b9bbc786c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:39.010414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274be063-1b81-4365-80c0-9d4dc584a897 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:39.111781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:39.116959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-274be063-1b81-4365-80c0-9d4dc584a897 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd488228fb1ff41c09b30577b9bbc786c', 'request_id': 'req-274be063-1b81-4365-80c0-9d4dc584a897'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:39.136317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-274be063-1b81-4365-80c0-9d4dc584a897 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd488228fb1ff41c09b30577b9bbc786c', 'request_id': 'req-274be063-1b81-4365-80c0-9d4dc584a897'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:39.146035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-274be063-1b81-4365-80c0-9d4dc584a897 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd488228fb1ff41c09b30577b9bbc786c', 'request_id': 'req-274be063-1b81-4365-80c0-9d4dc584a897'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:39.181657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-274be063-1b81-4365-80c0-9d4dc584a897 admin admin] The token cache is being invalidate because project d488228fb1ff41c09b30577b9bbc786c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:17:39.181883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-274be063-1b81-4365-80c0-9d4dc584a897 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:39.182602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-274be063-1b81-4365-80c0-9d4dc584a897 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:39.184487 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3791/7638] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:17:38 2022] DELETE /identity/v3/projects/d488228fb1ff41c09b30577b9bbc786c => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:17:39.189904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617ddda-b9a1-49b4-b0cc-1488b41286bc None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:39.191304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617ddda-b9a1-49b4-b0cc-1488b41286bc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:39.192084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617ddda-b9a1-49b4-b0cc-1488b41286bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:39.221425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e617ddda-b9a1-49b4-b0cc-1488b41286bc None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:17:39.223537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e617ddda-b9a1-49b4-b0cc-1488b41286bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:39.311620 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3848/7639] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:17:39.320348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272a8cf-0463-4a28-8c21-594a99cd2ac0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:39.321127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272a8cf-0463-4a28-8c21-594a99cd2ac0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:39.321785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272a8cf-0463-4a28-8c21-594a99cd2ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:39.354728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9272a8cf-0463-4a28-8c21-594a99cd2ac0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:17:39.356231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9272a8cf-0463-4a28-8c21-594a99cd2ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:39.384981 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3792/7640] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:17:39.387602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617ddda-b9a1-49b4-b0cc-1488b41286bc None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:39.398221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:39.409532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617ddda-b9a1-49b4-b0cc-1488b41286bc None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:39.424317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404c421-2f20-46fe-83a3-749c583283cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:39.425376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404c421-2f20-46fe-83a3-749c583283cf admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:39.425727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404c421-2f20-46fe-83a3-749c583283cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:39.426071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404c421-2f20-46fe-83a3-749c583283cf admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:39.427162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404c421-2f20-46fe-83a3-749c583283cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:17:39.427659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404c421-2f20-46fe-83a3-749c583283cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:39.435748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404c421-2f20-46fe-83a3-749c583283cf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:39.461958 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3849/7641] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:17:39.466193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272a8cf-0463-4a28-8c21-594a99cd2ac0 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:39.480508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272a8cf-0463-4a28-8c21-594a99cd2ac0 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:39.487371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e0c68-3696-4f0d-8cfb-02c6650dbbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:39.489307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e0c68-3696-4f0d-8cfb-02c6650dbbd3 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:39.489872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e0c68-3696-4f0d-8cfb-02c6650dbbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:39.490306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e0c68-3696-4f0d-8cfb-02c6650dbbd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:39.491207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e0c68-3696-4f0d-8cfb-02c6650dbbd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:39.493546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e0c68-3696-4f0d-8cfb-02c6650dbbd3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:39.555576 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3793/7642] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:17:39 2022] POST /identity/v3/projects => generated 363 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:17:39.559720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404c421-2f20-46fe-83a3-749c583283cf admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:39.586911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404c421-2f20-46fe-83a3-749c583283cf admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:39.742232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fb6e5-fa62-4067-a1d3-76b74eddfa14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:39.743037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fb6e5-fa62-4067-a1d3-76b74eddfa14 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:39.743322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fb6e5-fa62-4067-a1d3-76b74eddfa14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:39.743597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fb6e5-fa62-4067-a1d3-76b74eddfa14 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:39.744235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fb6e5-fa62-4067-a1d3-76b74eddfa14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:39.746938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fb6e5-fa62-4067-a1d3-76b74eddfa14 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:39.807259 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3850/7643] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:17:39 2022] POST /identity/v3/users => generated 282 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:17:39.811315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e0c68-3696-4f0d-8cfb-02c6650dbbd3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:39.838303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e0c68-3696-4f0d-8cfb-02c6650dbbd3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:39.849195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abbe1f3-c3e5-456f-962b-1f30ac9008ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:39.849836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abbe1f3-c3e5-456f-962b-1f30ac9008ad admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:39.850110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abbe1f3-c3e5-456f-962b-1f30ac9008ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:39.851538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abbe1f3-c3e5-456f-962b-1f30ac9008ad admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:39.852409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abbe1f3-c3e5-456f-962b-1f30ac9008ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:39.856671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abbe1f3-c3e5-456f-962b-1f30ac9008ad admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:39.882420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fb6e5-fa62-4067-a1d3-76b74eddfa14 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:39.882643 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3794/7644] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:17:39.898027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fb6e5-fa62-4067-a1d3-76b74eddfa14 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:39.914083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecbcacb-db9d-47a2-b4ff-dc69cdc48f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:39.914761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbcacb-db9d-47a2-b4ff-dc69cdc48f10 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:39.915044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbcacb-db9d-47a2-b4ff-dc69cdc48f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:39.915316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbcacb-db9d-47a2-b4ff-dc69cdc48f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:39.915977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecbcacb-db9d-47a2-b4ff-dc69cdc48f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:39.917733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecbcacb-db9d-47a2-b4ff-dc69cdc48f10 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:39.958449 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3851/7645] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:17:39.959725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abbe1f3-c3e5-456f-962b-1f30ac9008ad admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:39.982215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abbe1f3-c3e5-456f-962b-1f30ac9008ad admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:39.998321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c10093-19a2-4280-a4e7-b16a3adb3365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:39.999130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c10093-19a2-4280-a4e7-b16a3adb3365 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:39.999452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c10093-19a2-4280-a4e7-b16a3adb3365 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:39.999876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c10093-19a2-4280-a4e7-b16a3adb3365 admin admin] PATH_INFO: `/v3/projects/cca4882f1479428ab2fd140cf39d2ca7/users/05b76b9e9dd54c9f887594348e9e2cb1/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:40.042616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c10093-19a2-4280-a4e7-b16a3adb3365 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca4882f1479428ab2fd140cf39d2ca7, user_id=05b76b9e9dd54c9f887594348e9e2cb1, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:40.045311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c10093-19a2-4280-a4e7-b16a3adb3365 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:40.072731 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3795/7646] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:17:39 2022] PUT /identity/v3/projects/cca4882f1479428ab2fd140cf39d2ca7/users/05b76b9e9dd54c9f887594348e9e2cb1/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:17:40.084206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9517fc-dde4-4405-ad86-b04659ab08f3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:40.084628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9517fc-dde4-4405-ad86-b04659ab08f3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:40.085073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9517fc-dde4-4405-ad86-b04659ab08f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:40.147835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ef9517fc-dde4-4405-ad86-b04659ab08f3 None None] MFA Rules not processed for user `05b76b9e9dd54c9f887594348e9e2cb1`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:17:40.149384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ef9517fc-dde4-4405-ad86-b04659ab08f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:40.284651 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3852/7647] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:17:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:17:40.285559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c10093-19a2-4280-a4e7-b16a3adb3365 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:40.427598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c10093-19a2-4280-a4e7-b16a3adb3365 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:40.433829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecd678-1588-4ea1-baf0-9ebf3b18e7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:40.434836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecd678-1588-4ea1-baf0-9ebf3b18e7ed admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:40.435862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecd678-1588-4ea1-baf0-9ebf3b18e7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:40.436304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecd678-1588-4ea1-baf0-9ebf3b18e7ed admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:40.438116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ecd678-1588-4ea1-baf0-9ebf3b18e7ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:40.441804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ecd678-1588-4ea1-baf0-9ebf3b18e7ed admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:40.547538 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3796/7648] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:17:40 2022] POST /identity/v3/projects => generated 363 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:17:40.549432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9517fc-dde4-4405-ad86-b04659ab08f3 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:40.567865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9517fc-dde4-4405-ad86-b04659ab08f3 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:40.745352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc8eb3-7afb-4448-8d60-7addcd4b5c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:40.747978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc8eb3-7afb-4448-8d60-7addcd4b5c00 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:40.748499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc8eb3-7afb-4448-8d60-7addcd4b5c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:40.749973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc8eb3-7afb-4448-8d60-7addcd4b5c00 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:40.750747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc8eb3-7afb-4448-8d60-7addcd4b5c00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:40.758265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc8eb3-7afb-4448-8d60-7addcd4b5c00 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:40.779465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:40.812294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecd678-1588-4ea1-baf0-9ebf3b18e7ed admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:40.812835 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3853/7649] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:17:40 2022] POST /identity/v3/users => generated 288 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:17:40.832187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecd678-1588-4ea1-baf0-9ebf3b18e7ed admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:40.837739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec379070-0453-4c08-90e9-e94c42519eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:40.838803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec379070-0453-4c08-90e9-e94c42519eaf admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:40.839363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec379070-0453-4c08-90e9-e94c42519eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:40.839799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec379070-0453-4c08-90e9-e94c42519eaf admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:40.841550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec379070-0453-4c08-90e9-e94c42519eaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:40.843971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec379070-0453-4c08-90e9-e94c42519eaf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:40.857600 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3797/7650] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:17:40.863350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc8eb3-7afb-4448-8d60-7addcd4b5c00 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:40.877630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc8eb3-7afb-4448-8d60-7addcd4b5c00 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:40.888315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8ed28-f67a-4ee4-b4c2-2dc2d4647aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:40.889205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8ed28-f67a-4ee4-b4c2-2dc2d4647aaa admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:40.889355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8ed28-f67a-4ee4-b4c2-2dc2d4647aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:40.889536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8ed28-f67a-4ee4-b4c2-2dc2d4647aaa admin admin] PATH_INFO: `/v3/projects/596f2608bf9942eaba418edbd4bab150/users/e56e025713c34c5aab0a106058ed4203/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:40.912145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8ed28-f67a-4ee4-b4c2-2dc2d4647aaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=596f2608bf9942eaba418edbd4bab150, user_id=e56e025713c34c5aab0a106058ed4203, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:40.915096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8ed28-f67a-4ee4-b4c2-2dc2d4647aaa admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:40.938805 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3854/7651] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:17:40 2022] PUT /identity/v3/projects/596f2608bf9942eaba418edbd4bab150/users/e56e025713c34c5aab0a106058ed4203/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:17:40.939475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec379070-0453-4c08-90e9-e94c42519eaf admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:40.952942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec379070-0453-4c08-90e9-e94c42519eaf admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:41.035404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:41.094873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a78b5-8dfe-4036-a424-09b271459750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:41.095597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a78b5-8dfe-4036-a424-09b271459750 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:41.096223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a78b5-8dfe-4036-a424-09b271459750 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:41.096719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a78b5-8dfe-4036-a424-09b271459750 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:41.098290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a78b5-8dfe-4036-a424-09b271459750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:41.100955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a78b5-8dfe-4036-a424-09b271459750 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:41.123495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8ed28-f67a-4ee4-b4c2-2dc2d4647aaa admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:41.139579 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3798/7652] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:17:41.144671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8ed28-f67a-4ee4-b4c2-2dc2d4647aaa admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:41.159878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff6d0c-fc67-4ded-a06b-b08be257d97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:41.160510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff6d0c-fc67-4ded-a06b-b08be257d97d admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:41.160858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff6d0c-fc67-4ded-a06b-b08be257d97d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:41.161283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff6d0c-fc67-4ded-a06b-b08be257d97d admin admin] PATH_INFO: `/v3/projects/596f2608bf9942eaba418edbd4bab150/users/e56e025713c34c5aab0a106058ed4203/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:41.163805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff6d0c-fc67-4ded-a06b-b08be257d97d admin admin] RBAC: Authorizing `identity:create_grant(project_id=596f2608bf9942eaba418edbd4bab150, user_id=e56e025713c34c5aab0a106058ed4203, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:41.168237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff6d0c-fc67-4ded-a06b-b08be257d97d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:41.208049 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3855/7653] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:17:41 2022] PUT /identity/v3/projects/596f2608bf9942eaba418edbd4bab150/users/e56e025713c34c5aab0a106058ed4203/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:17:41.209238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559d203-2796-4420-92a3-eb6244a57ae0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:41.209681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559d203-2796-4420-92a3-eb6244a57ae0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:41.210125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559d203-2796-4420-92a3-eb6244a57ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:41.282085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b559d203-2796-4420-92a3-eb6244a57ae0 None None] MFA Rules not processed for user `e56e025713c34c5aab0a106058ed4203`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:17:41.283478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b559d203-2796-4420-92a3-eb6244a57ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:41.440890 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3799/7654] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:17:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:17:41.443628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd17e9d-f04c-4dcf-81d6-d4502131c855 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:41.444089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd17e9d-f04c-4dcf-81d6-d4502131c855 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:41.444488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd17e9d-f04c-4dcf-81d6-d4502131c855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:41.465114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559d203-2796-4420-92a3-eb6244a57ae0 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:41.466818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b559d203-2796-4420-92a3-eb6244a57ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:41.530129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-afd17e9d-f04c-4dcf-81d6-d4502131c855 None None] MFA Rules not processed for user `6ac020dc323042c48ccce2a8826bc6be`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:17:41.541424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-afd17e9d-f04c-4dcf-81d6-d4502131c855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:41.584055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559d203-2796-4420-92a3-eb6244a57ae0 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:41.590095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397c850-3357-4523-acfd-c8ce919d0e9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:41.590764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397c850-3357-4523-acfd-c8ce919d0e9f service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:41.591043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397c850-3357-4523-acfd-c8ce919d0e9f service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:41.591322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397c850-3357-4523-acfd-c8ce919d0e9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:41.603065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397c850-3357-4523-acfd-c8ce919d0e9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:41.605415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397c850-3357-4523-acfd-c8ce919d0e9f service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:41.618016 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3800/7655] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:17:41 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:17:41.653046 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3856/7656] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:17:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:17:41.673130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397c850-3357-4523-acfd-c8ce919d0e9f service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:41.674612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3397c850-3357-4523-acfd-c8ce919d0e9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:41.756249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:41.764030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397c850-3357-4523-acfd-c8ce919d0e9f service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:41.770820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08322ab4-a811-4d26-aa58-618a284b3919 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:41.771515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08322ab4-a811-4d26-aa58-618a284b3919 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:41.771871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08322ab4-a811-4d26-aa58-618a284b3919 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:41.772249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08322ab4-a811-4d26-aa58-618a284b3919 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:41.781505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08322ab4-a811-4d26-aa58-618a284b3919 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:41.783386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08322ab4-a811-4d26-aa58-618a284b3919 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:41.794262 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3801/7657] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:17:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:17:43.837519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035aead5-edb8-42ff-87a0-6215556be349 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:43.838023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035aead5-edb8-42ff-87a0-6215556be349 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:43.838355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035aead5-edb8-42ff-87a0-6215556be349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:43.864191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-035aead5-edb8-42ff-87a0-6215556be349 None None] MFA Rules not processed for user `6ac020dc323042c48ccce2a8826bc6be`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:17:43.865620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-035aead5-edb8-42ff-87a0-6215556be349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:43.880385 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3857/7658] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:17:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:17:44.742885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08322ab4-a811-4d26-aa58-618a284b3919 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:44.745795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-08322ab4-a811-4d26-aa58-618a284b3919 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:44.775592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08322ab4-a811-4d26-aa58-618a284b3919 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:44.782224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145cbcb-7990-4a77-99d6-8b196650228b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:44.782824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145cbcb-7990-4a77-99d6-8b196650228b service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:44.783274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145cbcb-7990-4a77-99d6-8b196650228b service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:44.783559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145cbcb-7990-4a77-99d6-8b196650228b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:44.887689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145cbcb-7990-4a77-99d6-8b196650228b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:44.890749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145cbcb-7990-4a77-99d6-8b196650228b service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:44.915260 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3802/7659] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:17:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:17:46.344983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035aead5-edb8-42ff-87a0-6215556be349 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:46.444326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035aead5-edb8-42ff-87a0-6215556be349 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:46.449640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a111455-9304-430a-a436-510af7b7e14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:46.450704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a111455-9304-430a-a436-510af7b7e14d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:46.451774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a111455-9304-430a-a436-510af7b7e14d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:46.452766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a111455-9304-430a-a436-510af7b7e14d admin admin] PATH_INFO: `/v3/users/05b76b9e9dd54c9f887594348e9e2cb1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:46.454168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a111455-9304-430a-a436-510af7b7e14d admin admin] RBAC: Authorizing `identity:delete_user(user_id=05b76b9e9dd54c9f887594348e9e2cb1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:46.456927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a111455-9304-430a-a436-510af7b7e14d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:46.524498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=5,keystone:SELECT=90,keystone:DELETE=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:46.542363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a111455-9304-430a-a436-510af7b7e14d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05b76b9e9dd54c9f887594348e9e2cb1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:46.554184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a111455-9304-430a-a436-510af7b7e14d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05b76b9e9dd54c9f887594348e9e2cb1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:46.571494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a111455-9304-430a-a436-510af7b7e14d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05b76b9e9dd54c9f887594348e9e2cb1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:46.572404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a111455-9304-430a-a436-510af7b7e14d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05b76b9e9dd54c9f887594348e9e2cb1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:46.583470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-8a111455-9304-430a-a436-510af7b7e14d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05b76b9e9dd54c9f887594348e9e2cb1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:46.603731 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3858/7660] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:17:46 2022] DELETE /identity/v3/users/05b76b9e9dd54c9f887594348e9e2cb1 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:17:46.605818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b517c8a-f9f2-42ef-8f30-da1b83e9b297 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:46.606342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b517c8a-f9f2-42ef-8f30-da1b83e9b297 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:46.606459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b517c8a-f9f2-42ef-8f30-da1b83e9b297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:46.646145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7b517c8a-f9f2-42ef-8f30-da1b83e9b297 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:17:46.648849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7b517c8a-f9f2-42ef-8f30-da1b83e9b297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:46.750537 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3803/7661] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:17:46.762586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a111455-9304-430a-a436-510af7b7e14d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:46.764813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8a111455-9304-430a-a436-510af7b7e14d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:46.841269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a111455-9304-430a-a436-510af7b7e14d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:46.845604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d5c26-bcfb-467d-9144-96aff922c483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:46.848167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d5c26-bcfb-467d-9144-96aff922c483 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:46.848167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d5c26-bcfb-467d-9144-96aff922c483 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:46.848167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d5c26-bcfb-467d-9144-96aff922c483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:46.858194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d5c26-bcfb-467d-9144-96aff922c483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:46.860434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d5c26-bcfb-467d-9144-96aff922c483 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:46.874975 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3859/7662] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:17:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:17:47.586952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b517c8a-f9f2-42ef-8f30-da1b83e9b297 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:47.588648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7b517c8a-f9f2-42ef-8f30-da1b83e9b297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:47.602120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b517c8a-f9f2-42ef-8f30-da1b83e9b297 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:47.607851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9ecd9-d51d-4b73-9cd6-b90fa45eff6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:47.609221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9ecd9-d51d-4b73-9cd6-b90fa45eff6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:47.609498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9ecd9-d51d-4b73-9cd6-b90fa45eff6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:47.609772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9ecd9-d51d-4b73-9cd6-b90fa45eff6f admin admin] PATH_INFO: `/v3/projects/cca4882f1479428ab2fd140cf39d2ca7` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:47.610997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9ecd9-d51d-4b73-9cd6-b90fa45eff6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cca4882f1479428ab2fd140cf39d2ca7)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:47.612978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9ecd9-d51d-4b73-9cd6-b90fa45eff6f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:47.705036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-35f9ecd9-d51d-4b73-9cd6-b90fa45eff6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cca4882f1479428ab2fd140cf39d2ca7', 'request_id': 'req-35f9ecd9-d51d-4b73-9cd6-b90fa45eff6f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:47.718674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-35f9ecd9-d51d-4b73-9cd6-b90fa45eff6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cca4882f1479428ab2fd140cf39d2ca7', 'request_id': 'req-35f9ecd9-d51d-4b73-9cd6-b90fa45eff6f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:47.728574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-35f9ecd9-d51d-4b73-9cd6-b90fa45eff6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cca4882f1479428ab2fd140cf39d2ca7', 'request_id': 'req-35f9ecd9-d51d-4b73-9cd6-b90fa45eff6f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:47.749787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-35f9ecd9-d51d-4b73-9cd6-b90fa45eff6f admin admin] The token cache is being invalidate because project cca4882f1479428ab2fd140cf39d2ca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:17:47.750375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-35f9ecd9-d51d-4b73-9cd6-b90fa45eff6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:47.751597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-35f9ecd9-d51d-4b73-9cd6-b90fa45eff6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:47.757118 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3804/7663] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:17:47 2022] DELETE /identity/v3/projects/cca4882f1479428ab2fd140cf39d2ca7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:17:47.758360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d5c26-bcfb-467d-9144-96aff922c483 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:47.768747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d5c26-bcfb-467d-9144-96aff922c483 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:47.849817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86366544-e4d3-45d7-96a0-13ffd7cf78e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:47.851125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86366544-e4d3-45d7-96a0-13ffd7cf78e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:47.851449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86366544-e4d3-45d7-96a0-13ffd7cf78e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:47.851725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86366544-e4d3-45d7-96a0-13ffd7cf78e6 admin admin] PATH_INFO: `/v3/users/e56e025713c34c5aab0a106058ed4203` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:47.853111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86366544-e4d3-45d7-96a0-13ffd7cf78e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e56e025713c34c5aab0a106058ed4203)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:47.855486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86366544-e4d3-45d7-96a0-13ffd7cf78e6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:47.944773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-86366544-e4d3-45d7-96a0-13ffd7cf78e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e56e025713c34c5aab0a106058ed4203'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:47.961704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-86366544-e4d3-45d7-96a0-13ffd7cf78e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e56e025713c34c5aab0a106058ed4203'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:47.964250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:47.982029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-86366544-e4d3-45d7-96a0-13ffd7cf78e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e56e025713c34c5aab0a106058ed4203'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:47.983137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-86366544-e4d3-45d7-96a0-13ffd7cf78e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e56e025713c34c5aab0a106058ed4203'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:47.992289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-86366544-e4d3-45d7-96a0-13ffd7cf78e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e56e025713c34c5aab0a106058ed4203'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:48.000369 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3860/7664] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:17:47 2022] DELETE /identity/v3/users/e56e025713c34c5aab0a106058ed4203 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:17:48.662001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9ecd9-d51d-4b73-9cd6-b90fa45eff6f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:48.664108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-35f9ecd9-d51d-4b73-9cd6-b90fa45eff6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:48.698252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9ecd9-d51d-4b73-9cd6-b90fa45eff6f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:48.724137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:48.815204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779ab0b-3a87-4c98-a45f-b45a88747029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:48.816399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779ab0b-3a87-4c98-a45f-b45a88747029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:48.816882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779ab0b-3a87-4c98-a45f-b45a88747029 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:48.817220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779ab0b-3a87-4c98-a45f-b45a88747029 admin admin] PATH_INFO: `/v3/projects/596f2608bf9942eaba418edbd4bab150` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:48.818484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779ab0b-3a87-4c98-a45f-b45a88747029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=596f2608bf9942eaba418edbd4bab150)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:48.820658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779ab0b-3a87-4c98-a45f-b45a88747029 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:48.874351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c779ab0b-3a87-4c98-a45f-b45a88747029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '596f2608bf9942eaba418edbd4bab150', 'request_id': 'req-c779ab0b-3a87-4c98-a45f-b45a88747029'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:48.882606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c779ab0b-3a87-4c98-a45f-b45a88747029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '596f2608bf9942eaba418edbd4bab150', 'request_id': 'req-c779ab0b-3a87-4c98-a45f-b45a88747029'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:48.888799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c779ab0b-3a87-4c98-a45f-b45a88747029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '596f2608bf9942eaba418edbd4bab150', 'request_id': 'req-c779ab0b-3a87-4c98-a45f-b45a88747029'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:48.901828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c779ab0b-3a87-4c98-a45f-b45a88747029 admin admin] The token cache is being invalidate because project 596f2608bf9942eaba418edbd4bab150 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:17:48.902206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c779ab0b-3a87-4c98-a45f-b45a88747029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:48.902972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c779ab0b-3a87-4c98-a45f-b45a88747029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:17:48.904537 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3805/7665] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:17:48 2022] DELETE /identity/v3/projects/596f2608bf9942eaba418edbd4bab150 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:17:48.910090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e6f57-2459-4afa-a0d9-3c4fbe5b1598 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:48.910667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e6f57-2459-4afa-a0d9-3c4fbe5b1598 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:48.910667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e6f57-2459-4afa-a0d9-3c4fbe5b1598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:48.931564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-240e6f57-2459-4afa-a0d9-3c4fbe5b1598 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:17:48.932845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-240e6f57-2459-4afa-a0d9-3c4fbe5b1598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:49.011182 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3861/7666] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:17:49.019699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747451c0-05c7-4867-b19a-6183ad910372 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:49.020060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747451c0-05c7-4867-b19a-6183ad910372 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:49.020779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747451c0-05c7-4867-b19a-6183ad910372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:49.056366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-747451c0-05c7-4867-b19a-6183ad910372 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:17:49.058759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-747451c0-05c7-4867-b19a-6183ad910372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:49.070016 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3806/7667] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:17:49.074073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e6f57-2459-4afa-a0d9-3c4fbe5b1598 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:49.087678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e6f57-2459-4afa-a0d9-3c4fbe5b1598 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:49.093117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190513b-4306-4683-afe4-a45dc2680913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:49.094975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190513b-4306-4683-afe4-a45dc2680913 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:49.095338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190513b-4306-4683-afe4-a45dc2680913 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:49.095854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190513b-4306-4683-afe4-a45dc2680913 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:49.096627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190513b-4306-4683-afe4-a45dc2680913 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:17:49.097041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190513b-4306-4683-afe4-a45dc2680913 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:49.098975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190513b-4306-4683-afe4-a45dc2680913 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:49.115668 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3862/7668] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:17:49.117276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747451c0-05c7-4867-b19a-6183ad910372 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:49.128844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747451c0-05c7-4867-b19a-6183ad910372 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:49.139188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998c2d5-a9da-45bd-aec6-b19da35b9988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:49.140458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998c2d5-a9da-45bd-aec6-b19da35b9988 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:49.140953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998c2d5-a9da-45bd-aec6-b19da35b9988 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:49.141341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998c2d5-a9da-45bd-aec6-b19da35b9988 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:49.142066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998c2d5-a9da-45bd-aec6-b19da35b9988 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:49.144242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998c2d5-a9da-45bd-aec6-b19da35b9988 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:49.182532 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3807/7669] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:17:49 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:17:49.187306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190513b-4306-4683-afe4-a45dc2680913 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:49.198191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190513b-4306-4683-afe4-a45dc2680913 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:49.286123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74114fc8-3887-4a84-b700-1c64a2bf32cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:49.286779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74114fc8-3887-4a84-b700-1c64a2bf32cb admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:49.287066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74114fc8-3887-4a84-b700-1c64a2bf32cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:49.287860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74114fc8-3887-4a84-b700-1c64a2bf32cb admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:49.287982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74114fc8-3887-4a84-b700-1c64a2bf32cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:49.289899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74114fc8-3887-4a84-b700-1c64a2bf32cb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:49.317883 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3863/7670] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:17:49 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:17:49.321932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998c2d5-a9da-45bd-aec6-b19da35b9988 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:49.336109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998c2d5-a9da-45bd-aec6-b19da35b9988 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:49.341009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe1192-f1fd-44c5-83cb-c839c93f9a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:49.341619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe1192-f1fd-44c5-83cb-c839c93f9a7e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:49.341904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe1192-f1fd-44c5-83cb-c839c93f9a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:49.342178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe1192-f1fd-44c5-83cb-c839c93f9a7e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:49.342961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe1192-f1fd-44c5-83cb-c839c93f9a7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:49.344744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe1192-f1fd-44c5-83cb-c839c93f9a7e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:49.356340 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3808/7671] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:17:49.361026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74114fc8-3887-4a84-b700-1c64a2bf32cb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:49.372284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74114fc8-3887-4a84-b700-1c64a2bf32cb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:49.378452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71164367-d84f-441d-a90f-fef8274e2f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:49.379099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71164367-d84f-441d-a90f-fef8274e2f81 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:49.379385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71164367-d84f-441d-a90f-fef8274e2f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:49.379651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71164367-d84f-441d-a90f-fef8274e2f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:49.380287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71164367-d84f-441d-a90f-fef8274e2f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:49.382019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71164367-d84f-441d-a90f-fef8274e2f81 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:49.394294 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3864/7672] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:17:49.398244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe1192-f1fd-44c5-83cb-c839c93f9a7e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:49.410039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe1192-f1fd-44c5-83cb-c839c93f9a7e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:49.415602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09bdfd9-dc9d-490e-955e-b8c90643d562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:49.416285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bdfd9-dc9d-490e-955e-b8c90643d562 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:49.416569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bdfd9-dc9d-490e-955e-b8c90643d562 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:49.416848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bdfd9-dc9d-490e-955e-b8c90643d562 admin admin] PATH_INFO: `/v3/projects/f83b01cadbf346cfabbbae9bb30b2202/users/355d7696d7194e119c72d9105cc8778a/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:49.439266 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09bdfd9-dc9d-490e-955e-b8c90643d562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f83b01cadbf346cfabbbae9bb30b2202, user_id=355d7696d7194e119c72d9105cc8778a, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:49.442034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09bdfd9-dc9d-490e-955e-b8c90643d562 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:49.470631 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3809/7673] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:17:49 2022] PUT /identity/v3/projects/f83b01cadbf346cfabbbae9bb30b2202/users/355d7696d7194e119c72d9105cc8778a/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:17:49.485225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2f66d-0e33-44de-85ce-364fef8f930f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:49.487337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2f66d-0e33-44de-85ce-364fef8f930f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:49.488166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2f66d-0e33-44de-85ce-364fef8f930f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:49.532263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7af2f66d-0e33-44de-85ce-364fef8f930f None None] MFA Rules not processed for user `355d7696d7194e119c72d9105cc8778a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:17:49.533544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7af2f66d-0e33-44de-85ce-364fef8f930f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:49.591024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:49.649451 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3865/7674] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:17:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:17:49.680747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09bdfd9-dc9d-490e-955e-b8c90643d562 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:17:49.689142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c09bdfd9-dc9d-490e-955e-b8c90643d562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:17:49.746474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:49.792571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09bdfd9-dc9d-490e-955e-b8c90643d562 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:17:49.799728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3b8f4-28ee-45e1-8247-da2a4ab7e26d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:17:49.800520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3b8f4-28ee-45e1-8247-da2a4ab7e26d service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:17:49.800880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3b8f4-28ee-45e1-8247-da2a4ab7e26d service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:17:49.801261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3b8f4-28ee-45e1-8247-da2a4ab7e26d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:17:49.812667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3b8f4-28ee-45e1-8247-da2a4ab7e26d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:17:49.814551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3b8f4-28ee-45e1-8247-da2a4ab7e26d service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:17:49.826243 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3810/7675] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:17:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:17:59.647994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=13 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:59.821431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=13 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:16.439516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af2f66d-0e33-44de-85ce-364fef8f930f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:16.442969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7af2f66d-0e33-44de-85ce-364fef8f930f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:16.652348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af2f66d-0e33-44de-85ce-364fef8f930f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:16.671576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c22035-e372-4de5-a2ad-5bff20cda3c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:16.672215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c22035-e372-4de5-a2ad-5bff20cda3c4 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:16.672495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c22035-e372-4de5-a2ad-5bff20cda3c4 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:16.672768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c22035-e372-4de5-a2ad-5bff20cda3c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:16.674725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-53c22035-e372-4de5-a2ad-5bff20cda3c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:16.693349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c22035-e372-4de5-a2ad-5bff20cda3c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:16.695747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c22035-e372-4de5-a2ad-5bff20cda3c4 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:16.722525 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3866/7676] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:18:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:18:22.136428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3b8f4-28ee-45e1-8247-da2a4ab7e26d service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:22.240223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3b8f4-28ee-45e1-8247-da2a4ab7e26d service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:22.245361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b94673-c9c3-40a7-bbc4-9bab01bf6b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:22.246024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b94673-c9c3-40a7-bbc4-9bab01bf6b76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:22.246307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b94673-c9c3-40a7-bbc4-9bab01bf6b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:22.246630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b94673-c9c3-40a7-bbc4-9bab01bf6b76 admin admin] PATH_INFO: `/v3/users/355d7696d7194e119c72d9105cc8778a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:22.248418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b94673-c9c3-40a7-bbc4-9bab01bf6b76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=355d7696d7194e119c72d9105cc8778a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:22.250411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b94673-c9c3-40a7-bbc4-9bab01bf6b76 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:22.331162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-86b94673-c9c3-40a7-bbc4-9bab01bf6b76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '355d7696d7194e119c72d9105cc8778a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:22.331861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-86b94673-c9c3-40a7-bbc4-9bab01bf6b76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '355d7696d7194e119c72d9105cc8778a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:22.332498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-86b94673-c9c3-40a7-bbc4-9bab01bf6b76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '355d7696d7194e119c72d9105cc8778a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:22.345947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-86b94673-c9c3-40a7-bbc4-9bab01bf6b76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '355d7696d7194e119c72d9105cc8778a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:22.364826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-86b94673-c9c3-40a7-bbc4-9bab01bf6b76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '355d7696d7194e119c72d9105cc8778a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:22.385961 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3811/7677] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:18:22 2022] DELETE /identity/v3/users/355d7696d7194e119c72d9105cc8778a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:22.391511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6fa72-724e-4a5d-8c18-8ea59b2d69d9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:22.392006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6fa72-724e-4a5d-8c18-8ea59b2d69d9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:22.392180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6fa72-724e-4a5d-8c18-8ea59b2d69d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:22.417226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8ed6fa72-724e-4a5d-8c18-8ea59b2d69d9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:22.418755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8ed6fa72-724e-4a5d-8c18-8ea59b2d69d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:22.507696 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3867/7678] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:22.515833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b94673-c9c3-40a7-bbc4-9bab01bf6b76 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:22.517935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-86b94673-c9c3-40a7-bbc4-9bab01bf6b76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:22.628240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b94673-c9c3-40a7-bbc4-9bab01bf6b76 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:22.634055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873bd78-cf42-449b-8f04-0f568a4acc5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:22.635602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873bd78-cf42-449b-8f04-0f568a4acc5c service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:22.636809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873bd78-cf42-449b-8f04-0f568a4acc5c service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:22.637465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873bd78-cf42-449b-8f04-0f568a4acc5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:22.650818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873bd78-cf42-449b-8f04-0f568a4acc5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:22.652286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873bd78-cf42-449b-8f04-0f568a4acc5c service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:22.678747 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3812/7679] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:18:22.891188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6fa72-724e-4a5d-8c18-8ea59b2d69d9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:22.892718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8ed6fa72-724e-4a5d-8c18-8ea59b2d69d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:22.909675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6fa72-724e-4a5d-8c18-8ea59b2d69d9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:22.917289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb57ef-5aa4-4d67-93ab-b561a911483c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:22.918385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb57ef-5aa4-4d67-93ab-b561a911483c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:22.918921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb57ef-5aa4-4d67-93ab-b561a911483c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:22.919591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb57ef-5aa4-4d67-93ab-b561a911483c admin admin] PATH_INFO: `/v3/projects/f83b01cadbf346cfabbbae9bb30b2202` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:22.921682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb57ef-5aa4-4d67-93ab-b561a911483c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f83b01cadbf346cfabbbae9bb30b2202)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:22.923967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb57ef-5aa4-4d67-93ab-b561a911483c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:22.983651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-88bb57ef-5aa4-4d67-93ab-b561a911483c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f83b01cadbf346cfabbbae9bb30b2202', 'request_id': 'req-88bb57ef-5aa4-4d67-93ab-b561a911483c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:22.992236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-88bb57ef-5aa4-4d67-93ab-b561a911483c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f83b01cadbf346cfabbbae9bb30b2202', 'request_id': 'req-88bb57ef-5aa4-4d67-93ab-b561a911483c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:23.003808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-88bb57ef-5aa4-4d67-93ab-b561a911483c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f83b01cadbf346cfabbbae9bb30b2202', 'request_id': 'req-88bb57ef-5aa4-4d67-93ab-b561a911483c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:23.023667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-88bb57ef-5aa4-4d67-93ab-b561a911483c admin admin] The token cache is being invalidate because project f83b01cadbf346cfabbbae9bb30b2202 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:18:23.024234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-88bb57ef-5aa4-4d67-93ab-b561a911483c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:23.025362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-88bb57ef-5aa4-4d67-93ab-b561a911483c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:23.027558 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3868/7680] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:18:22 2022] DELETE /identity/v3/projects/f83b01cadbf346cfabbbae9bb30b2202 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:23.033302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8247a11-66f0-40c3-8f29-4ed061a375d4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:23.033698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8247a11-66f0-40c3-8f29-4ed061a375d4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:23.034097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8247a11-66f0-40c3-8f29-4ed061a375d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:23.066104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c8247a11-66f0-40c3-8f29-4ed061a375d4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:23.068326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c8247a11-66f0-40c3-8f29-4ed061a375d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:23.099566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:23.177698 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3813/7681] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:23.185572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a1083-4406-42b2-aaa4-7dbedfdecf7b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:23.185875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a1083-4406-42b2-aaa4-7dbedfdecf7b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:23.186131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a1083-4406-42b2-aaa4-7dbedfdecf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:23.210589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-835a1083-4406-42b2-aaa4-7dbedfdecf7b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:23.212249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-835a1083-4406-42b2-aaa4-7dbedfdecf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:23.228766 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3869/7682] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:23.232302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8247a11-66f0-40c3-8f29-4ed061a375d4 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:23.254862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8247a11-66f0-40c3-8f29-4ed061a375d4 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:23.262640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877592d-333c-4d65-bc85-a8ed49f20411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:23.264062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877592d-333c-4d65-bc85-a8ed49f20411 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:23.264347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877592d-333c-4d65-bc85-a8ed49f20411 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:23.264618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877592d-333c-4d65-bc85-a8ed49f20411 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:23.265271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877592d-333c-4d65-bc85-a8ed49f20411 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:18:23.265624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877592d-333c-4d65-bc85-a8ed49f20411 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:23.268020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877592d-333c-4d65-bc85-a8ed49f20411 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:23.290902 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3814/7683] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:18:23.295677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a1083-4406-42b2-aaa4-7dbedfdecf7b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:23.311541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a1083-4406-42b2-aaa4-7dbedfdecf7b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:23.321768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c161b57-6c17-451c-bc8a-c265e88b4284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:23.322392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c161b57-6c17-451c-bc8a-c265e88b4284 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:23.322734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c161b57-6c17-451c-bc8a-c265e88b4284 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:23.323454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c161b57-6c17-451c-bc8a-c265e88b4284 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:23.323653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c161b57-6c17-451c-bc8a-c265e88b4284 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:23.326882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c161b57-6c17-451c-bc8a-c265e88b4284 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:23.332229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:23.366301 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3870/7684] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:18:23 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:18:23.370607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877592d-333c-4d65-bc85-a8ed49f20411 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:23.385422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877592d-333c-4d65-bc85-a8ed49f20411 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:23.456244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251989a5-4233-47dd-80a1-32b9e2ff6028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:23.456867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251989a5-4233-47dd-80a1-32b9e2ff6028 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:23.457154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251989a5-4233-47dd-80a1-32b9e2ff6028 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:23.457436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251989a5-4233-47dd-80a1-32b9e2ff6028 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:23.458071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251989a5-4233-47dd-80a1-32b9e2ff6028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:23.461191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251989a5-4233-47dd-80a1-32b9e2ff6028 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:23.508471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c161b57-6c17-451c-bc8a-c265e88b4284 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:23.509023 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3815/7685] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:18:23 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:18:23.523393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c161b57-6c17-451c-bc8a-c265e88b4284 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:23.529363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068f182-e5c1-4a74-b1dd-c44c39f6bd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:23.530381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068f182-e5c1-4a74-b1dd-c44c39f6bd71 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:23.530765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068f182-e5c1-4a74-b1dd-c44c39f6bd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:23.531122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068f182-e5c1-4a74-b1dd-c44c39f6bd71 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:23.531823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2068f182-e5c1-4a74-b1dd-c44c39f6bd71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:23.533889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2068f182-e5c1-4a74-b1dd-c44c39f6bd71 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:23.550621 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3871/7686] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:18:23.551564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251989a5-4233-47dd-80a1-32b9e2ff6028 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:23.563061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251989a5-4233-47dd-80a1-32b9e2ff6028 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:23.568094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85716b-ed16-4373-8a2c-9606dde5d178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:23.568802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85716b-ed16-4373-8a2c-9606dde5d178 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:23.569217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85716b-ed16-4373-8a2c-9606dde5d178 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:23.569569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85716b-ed16-4373-8a2c-9606dde5d178 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:23.570292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85716b-ed16-4373-8a2c-9606dde5d178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:23.572128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85716b-ed16-4373-8a2c-9606dde5d178 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:23.588485 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3816/7687] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:18:23.590672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068f182-e5c1-4a74-b1dd-c44c39f6bd71 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:23.604941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068f182-e5c1-4a74-b1dd-c44c39f6bd71 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:23.610799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1beda51-ed67-498f-a2e5-333a3eaef7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:23.611616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beda51-ed67-498f-a2e5-333a3eaef7c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:23.612348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beda51-ed67-498f-a2e5-333a3eaef7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:23.612907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beda51-ed67-498f-a2e5-333a3eaef7c0 admin admin] PATH_INFO: `/v3/projects/1934974f010a4acbac1a25b56ddfeb8d/users/d4dc4507da524fae837ebd6c775ec5f8/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:23.631163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1beda51-ed67-498f-a2e5-333a3eaef7c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1934974f010a4acbac1a25b56ddfeb8d, user_id=d4dc4507da524fae837ebd6c775ec5f8, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:23.633971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1beda51-ed67-498f-a2e5-333a3eaef7c0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:23.674701 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3872/7688] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:18:23 2022] PUT /identity/v3/projects/1934974f010a4acbac1a25b56ddfeb8d/users/d4dc4507da524fae837ebd6c775ec5f8/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:23.677449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4af37-1155-41ef-ac5e-1889f2d5a49d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:23.677748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4af37-1155-41ef-ac5e-1889f2d5a49d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:23.678028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4af37-1155-41ef-ac5e-1889f2d5a49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:23.722128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-97f4af37-1155-41ef-ac5e-1889f2d5a49d None None] MFA Rules not processed for user `d4dc4507da524fae837ebd6c775ec5f8`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:23.723105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-97f4af37-1155-41ef-ac5e-1889f2d5a49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:23.812155 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3817/7689] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:18:23.823605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1beda51-ed67-498f-a2e5-333a3eaef7c0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:23.826877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b1beda51-ed67-498f-a2e5-333a3eaef7c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:23.919647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1beda51-ed67-498f-a2e5-333a3eaef7c0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:23.925508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50f8d1-3b06-4bc0-b2ed-45b586ba2160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:23.927018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50f8d1-3b06-4bc0-b2ed-45b586ba2160 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:23.927393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50f8d1-3b06-4bc0-b2ed-45b586ba2160 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:23.927754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50f8d1-3b06-4bc0-b2ed-45b586ba2160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:23.941372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50f8d1-3b06-4bc0-b2ed-45b586ba2160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:23.942886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50f8d1-3b06-4bc0-b2ed-45b586ba2160 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:23.957562 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3873/7690] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:18:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:18:24.320158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4af37-1155-41ef-ac5e-1889f2d5a49d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:24.348296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:24.407700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4af37-1155-41ef-ac5e-1889f2d5a49d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:24.418202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9295e53-f643-4ef4-a07f-5e42529701b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:24.419149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9295e53-f643-4ef4-a07f-5e42529701b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:24.420546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9295e53-f643-4ef4-a07f-5e42529701b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:24.421520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9295e53-f643-4ef4-a07f-5e42529701b0 admin admin] PATH_INFO: `/v3/users/d4dc4507da524fae837ebd6c775ec5f8` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:24.423267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9295e53-f643-4ef4-a07f-5e42529701b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4dc4507da524fae837ebd6c775ec5f8)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:24.427100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9295e53-f643-4ef4-a07f-5e42529701b0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:24.490318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e9295e53-f643-4ef4-a07f-5e42529701b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4dc4507da524fae837ebd6c775ec5f8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:24.491139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e9295e53-f643-4ef4-a07f-5e42529701b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4dc4507da524fae837ebd6c775ec5f8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:24.491824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e9295e53-f643-4ef4-a07f-5e42529701b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4dc4507da524fae837ebd6c775ec5f8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:24.501840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e9295e53-f643-4ef4-a07f-5e42529701b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4dc4507da524fae837ebd6c775ec5f8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:24.520682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e9295e53-f643-4ef4-a07f-5e42529701b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4dc4507da524fae837ebd6c775ec5f8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:24.535601 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3818/7691] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:18:24 2022] DELETE /identity/v3/users/d4dc4507da524fae837ebd6c775ec5f8 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:24.539948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d770c-c687-478d-8fbd-8423c93d5650 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:24.540456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d770c-c687-478d-8fbd-8423c93d5650 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:24.540640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d770c-c687-478d-8fbd-8423c93d5650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:24.563213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9e9d770c-c687-478d-8fbd-8423c93d5650 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:24.564583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9e9d770c-c687-478d-8fbd-8423c93d5650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:24.631691 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3874/7692] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:24.642217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9295e53-f643-4ef4-a07f-5e42529701b0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:24.643729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e9295e53-f643-4ef4-a07f-5e42529701b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:24.732985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9295e53-f643-4ef4-a07f-5e42529701b0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:24.739641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d860e41-a28f-4a1e-9053-af005dfffd5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:24.741290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d860e41-a28f-4a1e-9053-af005dfffd5b service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:24.742100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d860e41-a28f-4a1e-9053-af005dfffd5b service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:24.742100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d860e41-a28f-4a1e-9053-af005dfffd5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:24.753288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d860e41-a28f-4a1e-9053-af005dfffd5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:24.755022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d860e41-a28f-4a1e-9053-af005dfffd5b service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:24.768741 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3819/7693] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:18:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:18:25.116723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d770c-c687-478d-8fbd-8423c93d5650 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:25.118418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9e9d770c-c687-478d-8fbd-8423c93d5650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:25.131782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d770c-c687-478d-8fbd-8423c93d5650 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:25.152511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c647e8-07f8-40e2-a21a-d8380dd7228f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:25.153330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c647e8-07f8-40e2-a21a-d8380dd7228f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:25.154153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c647e8-07f8-40e2-a21a-d8380dd7228f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:25.154547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c647e8-07f8-40e2-a21a-d8380dd7228f admin admin] PATH_INFO: `/v3/projects/1934974f010a4acbac1a25b56ddfeb8d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:25.155665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c647e8-07f8-40e2-a21a-d8380dd7228f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1934974f010a4acbac1a25b56ddfeb8d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:25.158103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c647e8-07f8-40e2-a21a-d8380dd7228f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:25.219469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:25.221737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48c647e8-07f8-40e2-a21a-d8380dd7228f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1934974f010a4acbac1a25b56ddfeb8d', 'request_id': 'req-48c647e8-07f8-40e2-a21a-d8380dd7228f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:25.230732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48c647e8-07f8-40e2-a21a-d8380dd7228f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1934974f010a4acbac1a25b56ddfeb8d', 'request_id': 'req-48c647e8-07f8-40e2-a21a-d8380dd7228f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:25.240971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48c647e8-07f8-40e2-a21a-d8380dd7228f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1934974f010a4acbac1a25b56ddfeb8d', 'request_id': 'req-48c647e8-07f8-40e2-a21a-d8380dd7228f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:25.266297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48c647e8-07f8-40e2-a21a-d8380dd7228f admin admin] The token cache is being invalidate because project 1934974f010a4acbac1a25b56ddfeb8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:18:25.266689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48c647e8-07f8-40e2-a21a-d8380dd7228f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:25.267365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48c647e8-07f8-40e2-a21a-d8380dd7228f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:25.268888 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3875/7694] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:18:25 2022] DELETE /identity/v3/projects/1934974f010a4acbac1a25b56ddfeb8d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:18:25.273733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77b23f-a513-4417-99e0-44ac31b43f23 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:25.274023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77b23f-a513-4417-99e0-44ac31b43f23 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:25.274301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77b23f-a513-4417-99e0-44ac31b43f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:25.303253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-da77b23f-a513-4417-99e0-44ac31b43f23 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:25.308000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-da77b23f-a513-4417-99e0-44ac31b43f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:25.379093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:25.424661 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3820/7695] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:18:25.434854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c715a5-e4b2-4846-b169-b352a1925d8e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:25.435141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c715a5-e4b2-4846-b169-b352a1925d8e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:25.435421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c715a5-e4b2-4846-b169-b352a1925d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:25.473169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-66c715a5-e4b2-4846-b169-b352a1925d8e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:25.474614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-66c715a5-e4b2-4846-b169-b352a1925d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:25.499411 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3876/7696] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:25.503937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77b23f-a513-4417-99e0-44ac31b43f23 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:25.520938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77b23f-a513-4417-99e0-44ac31b43f23 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:25.530911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d7cc3-1ac2-41c2-93bf-24497c0d44d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:25.531645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d7cc3-1ac2-41c2-93bf-24497c0d44d3 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:25.532677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d7cc3-1ac2-41c2-93bf-24497c0d44d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:25.534209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d7cc3-1ac2-41c2-93bf-24497c0d44d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:25.534993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d7cc3-1ac2-41c2-93bf-24497c0d44d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:18:25.536741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d7cc3-1ac2-41c2-93bf-24497c0d44d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:25.538456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d7cc3-1ac2-41c2-93bf-24497c0d44d3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:25.572637 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3821/7697] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:18:25.577394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c715a5-e4b2-4846-b169-b352a1925d8e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:25.594582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c715a5-e4b2-4846-b169-b352a1925d8e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:25.604196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dcc08e-d232-4586-adae-cebda9b3bb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:25.604819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dcc08e-d232-4586-adae-cebda9b3bb95 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:25.605223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dcc08e-d232-4586-adae-cebda9b3bb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:25.605577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dcc08e-d232-4586-adae-cebda9b3bb95 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:25.606287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dcc08e-d232-4586-adae-cebda9b3bb95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:25.608840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dcc08e-d232-4586-adae-cebda9b3bb95 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:25.668965 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3877/7698] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:18:25 2022] POST /identity/v3/projects => generated 381 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:18:25.673934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d7cc3-1ac2-41c2-93bf-24497c0d44d3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:25.695776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d7cc3-1ac2-41c2-93bf-24497c0d44d3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:25.846723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7f2c0-3da4-4543-a790-7c1688a03d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:25.848105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7f2c0-3da4-4543-a790-7c1688a03d1a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:25.848435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7f2c0-3da4-4543-a790-7c1688a03d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:25.848750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7f2c0-3da4-4543-a790-7c1688a03d1a admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:25.852557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7f2c0-3da4-4543-a790-7c1688a03d1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:25.858119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7f2c0-3da4-4543-a790-7c1688a03d1a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:25.905119 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3822/7699] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:18:25 2022] POST /identity/v3/users => generated 291 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:18:25.909436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dcc08e-d232-4586-adae-cebda9b3bb95 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:25.925602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dcc08e-d232-4586-adae-cebda9b3bb95 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:25.932494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19f7d7-0789-4b18-8aea-cdddae39a437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:25.933778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19f7d7-0789-4b18-8aea-cdddae39a437 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:25.935374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19f7d7-0789-4b18-8aea-cdddae39a437 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:25.935862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19f7d7-0789-4b18-8aea-cdddae39a437 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:25.937186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19f7d7-0789-4b18-8aea-cdddae39a437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:25.941600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19f7d7-0789-4b18-8aea-cdddae39a437 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:25.957136 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3878/7700] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:18:25.957913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7f2c0-3da4-4543-a790-7c1688a03d1a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:25.971979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7f2c0-3da4-4543-a790-7c1688a03d1a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:25.978974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a9d65-fc34-4197-921a-5d892782acaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:25.980163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a9d65-fc34-4197-921a-5d892782acaf admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:25.980529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a9d65-fc34-4197-921a-5d892782acaf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:25.980879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a9d65-fc34-4197-921a-5d892782acaf admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:25.981758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a9d65-fc34-4197-921a-5d892782acaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:25.984353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a9d65-fc34-4197-921a-5d892782acaf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:26.002203 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3823/7701] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:18:26.005179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19f7d7-0789-4b18-8aea-cdddae39a437 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:26.024355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19f7d7-0789-4b18-8aea-cdddae39a437 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:26.051323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1b1ee-2d83-4f6d-9b35-f4a5e3f3a375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:26.052024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1b1ee-2d83-4f6d-9b35-f4a5e3f3a375 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:26.052310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1b1ee-2d83-4f6d-9b35-f4a5e3f3a375 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:26.052591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1b1ee-2d83-4f6d-9b35-f4a5e3f3a375 admin admin] PATH_INFO: `/v3/projects/9b0215aa115b42dca0c6d62b3eaf0977/users/287985707071436b9b74aac5f990b453/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:26.082196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de1b1ee-2d83-4f6d-9b35-f4a5e3f3a375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b0215aa115b42dca0c6d62b3eaf0977, user_id=287985707071436b9b74aac5f990b453, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:26.089229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de1b1ee-2d83-4f6d-9b35-f4a5e3f3a375 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:26.125267 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3879/7702] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:18:26 2022] PUT /identity/v3/projects/9b0215aa115b42dca0c6d62b3eaf0977/users/287985707071436b9b74aac5f990b453/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:26.137458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56c182-774d-454d-bdab-3b9f02e9f0b3 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:26.138784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56c182-774d-454d-bdab-3b9f02e9f0b3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:26.139219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56c182-774d-454d-bdab-3b9f02e9f0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:26.180796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-df56c182-774d-454d-bdab-3b9f02e9f0b3 None None] MFA Rules not processed for user `287985707071436b9b74aac5f990b453`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:26.183228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-df56c182-774d-454d-bdab-3b9f02e9f0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:26.296449 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3824/7703] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:18:26.301756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1b1ee-2d83-4f6d-9b35-f4a5e3f3a375 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:26.372212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1b1ee-2d83-4f6d-9b35-f4a5e3f3a375 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:26.377573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea1f59-d8cf-4736-8fe0-22a0ffff4db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:26.378201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea1f59-d8cf-4736-8fe0-22a0ffff4db9 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:26.378508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea1f59-d8cf-4736-8fe0-22a0ffff4db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:26.378784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea1f59-d8cf-4736-8fe0-22a0ffff4db9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:26.379445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea1f59-d8cf-4736-8fe0-22a0ffff4db9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:26.381366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea1f59-d8cf-4736-8fe0-22a0ffff4db9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:26.418075 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3880/7704] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:18:26 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:18:26.423777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56c182-774d-454d-bdab-3b9f02e9f0b3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:26.441504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56c182-774d-454d-bdab-3b9f02e9f0b3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:26.527106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:26.574781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b41359-cad9-4caa-92e8-e11373dfcb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:26.575466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b41359-cad9-4caa-92e8-e11373dfcb8e admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:26.576349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b41359-cad9-4caa-92e8-e11373dfcb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:26.576621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b41359-cad9-4caa-92e8-e11373dfcb8e admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:26.577248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b41359-cad9-4caa-92e8-e11373dfcb8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:26.579835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b41359-cad9-4caa-92e8-e11373dfcb8e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:26.616478 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3825/7705] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:18:26 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:18:26.620281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea1f59-d8cf-4736-8fe0-22a0ffff4db9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:26.636615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea1f59-d8cf-4736-8fe0-22a0ffff4db9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:26.648633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1386e-3152-4a06-85d4-dcb68dce7638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:26.649533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1386e-3152-4a06-85d4-dcb68dce7638 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:26.650007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1386e-3152-4a06-85d4-dcb68dce7638 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:26.650474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1386e-3152-4a06-85d4-dcb68dce7638 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:26.651346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1386e-3152-4a06-85d4-dcb68dce7638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:26.655082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1386e-3152-4a06-85d4-dcb68dce7638 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:26.685631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b41359-cad9-4caa-92e8-e11373dfcb8e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:26.686030 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3881/7706] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:18:26.718982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b41359-cad9-4caa-92e8-e11373dfcb8e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:26.733783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980e28d-eb64-48ed-90ac-cd0c9698a680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:26.735560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980e28d-eb64-48ed-90ac-cd0c9698a680 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:26.736527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980e28d-eb64-48ed-90ac-cd0c9698a680 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:26.736527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980e28d-eb64-48ed-90ac-cd0c9698a680 admin admin] PATH_INFO: `/v3/projects/d4d39b386a334518816667ae969afe65/users/3ba0acd0e63843eab5e1492dbed9c0b9/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:26.775587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980e28d-eb64-48ed-90ac-cd0c9698a680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4d39b386a334518816667ae969afe65, user_id=3ba0acd0e63843eab5e1492dbed9c0b9, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:26.780602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980e28d-eb64-48ed-90ac-cd0c9698a680 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:26.813535 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3826/7707] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:18:26 2022] PUT /identity/v3/projects/d4d39b386a334518816667ae969afe65/users/3ba0acd0e63843eab5e1492dbed9c0b9/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:18:26.818814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1386e-3152-4a06-85d4-dcb68dce7638 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:26.834636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1386e-3152-4a06-85d4-dcb68dce7638 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:26.960299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:26.990931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18633d36-121b-4b51-a505-be9d7f24e2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:26.991597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18633d36-121b-4b51-a505-be9d7f24e2ea admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:26.991880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18633d36-121b-4b51-a505-be9d7f24e2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:26.992152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18633d36-121b-4b51-a505-be9d7f24e2ea admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:26.992815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18633d36-121b-4b51-a505-be9d7f24e2ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:26.994450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18633d36-121b-4b51-a505-be9d7f24e2ea admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:27.021439 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3882/7708] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:18:27.022828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980e28d-eb64-48ed-90ac-cd0c9698a680 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:27.033771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980e28d-eb64-48ed-90ac-cd0c9698a680 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:27.049961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45aa687-292b-496c-a482-5246e482742f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:27.050679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45aa687-292b-496c-a482-5246e482742f admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:27.050980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45aa687-292b-496c-a482-5246e482742f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:27.051240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45aa687-292b-496c-a482-5246e482742f admin admin] PATH_INFO: `/v3/projects/d4d39b386a334518816667ae969afe65/users/3ba0acd0e63843eab5e1492dbed9c0b9/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:27.053215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45aa687-292b-496c-a482-5246e482742f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4d39b386a334518816667ae969afe65, user_id=3ba0acd0e63843eab5e1492dbed9c0b9, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:27.059591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45aa687-292b-496c-a482-5246e482742f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:27.084014 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3827/7709] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:18:27 2022] PUT /identity/v3/projects/d4d39b386a334518816667ae969afe65/users/3ba0acd0e63843eab5e1492dbed9c0b9/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:27.094607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a25cc-70c7-4aa3-a98a-f9613fb6bb90 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:27.094959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a25cc-70c7-4aa3-a98a-f9613fb6bb90 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:27.095197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a25cc-70c7-4aa3-a98a-f9613fb6bb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:27.136899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c28a25cc-70c7-4aa3-a98a-f9613fb6bb90 None None] MFA Rules not processed for user `3ba0acd0e63843eab5e1492dbed9c0b9`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:27.138253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c28a25cc-70c7-4aa3-a98a-f9613fb6bb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:27.274870 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3883/7710] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:27.304210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45aa687-292b-496c-a482-5246e482742f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:27.310340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c45aa687-292b-496c-a482-5246e482742f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:27.403118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45aa687-292b-496c-a482-5246e482742f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:27.407833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155b4bf-e094-4eeb-a06b-c4539e636657 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:27.408257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155b4bf-e094-4eeb-a06b-c4539e636657 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:27.408523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155b4bf-e094-4eeb-a06b-c4539e636657 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:27.408804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155b4bf-e094-4eeb-a06b-c4539e636657 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:27.420660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155b4bf-e094-4eeb-a06b-c4539e636657 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:27.422457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155b4bf-e094-4eeb-a06b-c4539e636657 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:27.448556 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3828/7711] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:18:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:18:28.174093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a25cc-70c7-4aa3-a98a-f9613fb6bb90 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:28.188667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a25cc-70c7-4aa3-a98a-f9613fb6bb90 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:28.195857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6625f-eb12-4424-ba1a-d6d5c4b16e71 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba0acd0e63843eab5e1492dbed9c0b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4d39b386a334518816667ae969afe65', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:28.197466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6625f-eb12-4424-ba1a-d6d5c4b16e71 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:28.198008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6625f-eb12-4424-ba1a-d6d5c4b16e71 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:28.198629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6625f-eb12-4424-ba1a-d6d5c4b16e71 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:28.200089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6625f-eb12-4424-ba1a-d6d5c4b16e71 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:18:28.201437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6625f-eb12-4424-ba1a-d6d5c4b16e71 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:28.204143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6625f-eb12-4424-ba1a-d6d5c4b16e71 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:28.223430 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3884/7712] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:18:28.227443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155b4bf-e094-4eeb-a06b-c4539e636657 service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:28.242223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155b4bf-e094-4eeb-a06b-c4539e636657 service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:28.252280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c95241-dd8c-4955-907e-b77524e340ab tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba0acd0e63843eab5e1492dbed9c0b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4d39b386a334518816667ae969afe65', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:28.252844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c95241-dd8c-4955-907e-b77524e340ab tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:28.253134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c95241-dd8c-4955-907e-b77524e340ab tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:28.253413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c95241-dd8c-4955-907e-b77524e340ab tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:28.254229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c95241-dd8c-4955-907e-b77524e340ab tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:28.256438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c95241-dd8c-4955-907e-b77524e340ab tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:28.292883 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3829/7713] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:18:28 2022] POST /identity/v3/projects => generated 355 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:18:28.297081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6625f-eb12-4424-ba1a-d6d5c4b16e71 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:28.313877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6625f-eb12-4424-ba1a-d6d5c4b16e71 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:28.478522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521762ec-756c-407a-b0a0-ae0549af91dc tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba0acd0e63843eab5e1492dbed9c0b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4d39b386a334518816667ae969afe65', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:28.485155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521762ec-756c-407a-b0a0-ae0549af91dc tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:28.485766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521762ec-756c-407a-b0a0-ae0549af91dc tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:28.486146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521762ec-756c-407a-b0a0-ae0549af91dc tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:28.487007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521762ec-756c-407a-b0a0-ae0549af91dc tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:18:28.487845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521762ec-756c-407a-b0a0-ae0549af91dc tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:28.490010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521762ec-756c-407a-b0a0-ae0549af91dc tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:28.521837 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3885/7714] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:18:28.589248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c95241-dd8c-4955-907e-b77524e340ab tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:28.604919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c95241-dd8c-4955-907e-b77524e340ab tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:28.613425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9238d51-8d37-437f-890d-1f2efcda6782 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba0acd0e63843eab5e1492dbed9c0b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4d39b386a334518816667ae969afe65', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:28.616135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9238d51-8d37-437f-890d-1f2efcda6782 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:28.616686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9238d51-8d37-437f-890d-1f2efcda6782 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:28.617187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9238d51-8d37-437f-890d-1f2efcda6782 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] PATH_INFO: `/v3/projects/b12941ee261f4441a34abb597af292c5` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:28.619937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9238d51-8d37-437f-890d-1f2efcda6782 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b12941ee261f4441a34abb597af292c5)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:28.624145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9238d51-8d37-437f-890d-1f2efcda6782 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:28.779778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f9238d51-8d37-437f-890d-1f2efcda6782 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b12941ee261f4441a34abb597af292c5', 'request_id': 'req-f9238d51-8d37-437f-890d-1f2efcda6782'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:28.790789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f9238d51-8d37-437f-890d-1f2efcda6782 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b12941ee261f4441a34abb597af292c5', 'request_id': 'req-f9238d51-8d37-437f-890d-1f2efcda6782'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:28.799532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:28.804504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f9238d51-8d37-437f-890d-1f2efcda6782 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b12941ee261f4441a34abb597af292c5', 'request_id': 'req-f9238d51-8d37-437f-890d-1f2efcda6782'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:28.838466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f9238d51-8d37-437f-890d-1f2efcda6782 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] The token cache is being invalidate because project b12941ee261f4441a34abb597af292c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:18:28.838938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f9238d51-8d37-437f-890d-1f2efcda6782 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:28.839755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f9238d51-8d37-437f-890d-1f2efcda6782 tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:28.843510 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3830/7715] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:18:28 2022] DELETE /identity/v3/projects/b12941ee261f4441a34abb597af292c5 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:29.197664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521762ec-756c-407a-b0a0-ae0549af91dc tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:29.199554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-521762ec-756c-407a-b0a0-ae0549af91dc tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:29.295298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:29.313981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521762ec-756c-407a-b0a0-ae0549af91dc tempest-VolumeQuotaClassesTest-1366503645 tempest-VolumeQuotaClassesTest-1366503645-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:29.475493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9698af5-d6ec-4188-a08d-5eb03ef66e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:29.477068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9698af5-d6ec-4188-a08d-5eb03ef66e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:29.477068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9698af5-d6ec-4188-a08d-5eb03ef66e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:29.477068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9698af5-d6ec-4188-a08d-5eb03ef66e60 admin admin] PATH_INFO: `/v3/users/287985707071436b9b74aac5f990b453` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:29.479659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9698af5-d6ec-4188-a08d-5eb03ef66e60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=287985707071436b9b74aac5f990b453)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:29.482396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9698af5-d6ec-4188-a08d-5eb03ef66e60 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:29.576762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c9698af5-d6ec-4188-a08d-5eb03ef66e60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '287985707071436b9b74aac5f990b453'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:29.587381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c9698af5-d6ec-4188-a08d-5eb03ef66e60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '287985707071436b9b74aac5f990b453'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:29.621247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c9698af5-d6ec-4188-a08d-5eb03ef66e60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '287985707071436b9b74aac5f990b453'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:29.622043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c9698af5-d6ec-4188-a08d-5eb03ef66e60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '287985707071436b9b74aac5f990b453'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:29.631007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c9698af5-d6ec-4188-a08d-5eb03ef66e60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '287985707071436b9b74aac5f990b453'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:29.648267 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3886/7716] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:18:29 2022] DELETE /identity/v3/users/287985707071436b9b74aac5f990b453 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:29.649789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaffe4-1810-4164-b11c-01a953b8ee50 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:29.650950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaffe4-1810-4164-b11c-01a953b8ee50 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:29.651451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaffe4-1810-4164-b11c-01a953b8ee50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:29.689945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e3aaffe4-1810-4164-b11c-01a953b8ee50 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:29.691657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e3aaffe4-1810-4164-b11c-01a953b8ee50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:29.787421 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3831/7717] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:18:29.796332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9698af5-d6ec-4188-a08d-5eb03ef66e60 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:29.798160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c9698af5-d6ec-4188-a08d-5eb03ef66e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:29.924059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9698af5-d6ec-4188-a08d-5eb03ef66e60 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:29.932256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61489a05-a3e4-46e3-b1eb-b23e9c2c289d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:29.932953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61489a05-a3e4-46e3-b1eb-b23e9c2c289d service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:29.933246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61489a05-a3e4-46e3-b1eb-b23e9c2c289d service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:29.933578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61489a05-a3e4-46e3-b1eb-b23e9c2c289d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:29.947116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61489a05-a3e4-46e3-b1eb-b23e9c2c289d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:29.948681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61489a05-a3e4-46e3-b1eb-b23e9c2c289d service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:29.961845 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3887/7718] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:18:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:18:30.401237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aaffe4-1810-4164-b11c-01a953b8ee50 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:30.402731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e3aaffe4-1810-4164-b11c-01a953b8ee50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:30.422624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aaffe4-1810-4164-b11c-01a953b8ee50 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:30.430794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65283a-938b-433f-9fdf-e06f35f0987b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:30.431558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65283a-938b-433f-9fdf-e06f35f0987b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:30.431851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65283a-938b-433f-9fdf-e06f35f0987b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:30.432181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65283a-938b-433f-9fdf-e06f35f0987b admin admin] PATH_INFO: `/v3/projects/9b0215aa115b42dca0c6d62b3eaf0977` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:30.433386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c65283a-938b-433f-9fdf-e06f35f0987b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b0215aa115b42dca0c6d62b3eaf0977)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:30.437867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c65283a-938b-433f-9fdf-e06f35f0987b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:30.571262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1c65283a-938b-433f-9fdf-e06f35f0987b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b0215aa115b42dca0c6d62b3eaf0977', 'request_id': 'req-1c65283a-938b-433f-9fdf-e06f35f0987b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:30.593045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1c65283a-938b-433f-9fdf-e06f35f0987b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b0215aa115b42dca0c6d62b3eaf0977', 'request_id': 'req-1c65283a-938b-433f-9fdf-e06f35f0987b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:30.602136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1c65283a-938b-433f-9fdf-e06f35f0987b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b0215aa115b42dca0c6d62b3eaf0977', 'request_id': 'req-1c65283a-938b-433f-9fdf-e06f35f0987b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:30.620765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1c65283a-938b-433f-9fdf-e06f35f0987b admin admin] The token cache is being invalidate because project 9b0215aa115b42dca0c6d62b3eaf0977 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:18:30.621793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1c65283a-938b-433f-9fdf-e06f35f0987b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:30.623432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1c65283a-938b-433f-9fdf-e06f35f0987b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:30.626802 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3832/7719] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:18:30 2022] DELETE /identity/v3/projects/9b0215aa115b42dca0c6d62b3eaf0977 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:30.630417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61489a05-a3e4-46e3-b1eb-b23e9c2c289d service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:30.646475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61489a05-a3e4-46e3-b1eb-b23e9c2c289d service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:30.662838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:30.721859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c114e-1df0-4687-a894-c77960ec6c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:30.722583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c114e-1df0-4687-a894-c77960ec6c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:30.723052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c114e-1df0-4687-a894-c77960ec6c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:30.723279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c114e-1df0-4687-a894-c77960ec6c02 admin admin] PATH_INFO: `/v3/users/3ba0acd0e63843eab5e1492dbed9c0b9` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:30.725322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c114e-1df0-4687-a894-c77960ec6c02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ba0acd0e63843eab5e1492dbed9c0b9)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:30.727628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c114e-1df0-4687-a894-c77960ec6c02 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:30.797086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e57c114e-1df0-4687-a894-c77960ec6c02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ba0acd0e63843eab5e1492dbed9c0b9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:30.805155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e57c114e-1df0-4687-a894-c77960ec6c02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ba0acd0e63843eab5e1492dbed9c0b9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:30.823177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e57c114e-1df0-4687-a894-c77960ec6c02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ba0acd0e63843eab5e1492dbed9c0b9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:30.823989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e57c114e-1df0-4687-a894-c77960ec6c02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ba0acd0e63843eab5e1492dbed9c0b9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:30.839120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e57c114e-1df0-4687-a894-c77960ec6c02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ba0acd0e63843eab5e1492dbed9c0b9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:30.850839 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3888/7720] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:18:30 2022] DELETE /identity/v3/users/3ba0acd0e63843eab5e1492dbed9c0b9 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:31.314344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65283a-938b-433f-9fdf-e06f35f0987b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:31.316154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1c65283a-938b-433f-9fdf-e06f35f0987b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:31.348322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65283a-938b-433f-9fdf-e06f35f0987b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:31.508993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fdbf2-bf46-49ca-b46f-112de5d79852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:31.509968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fdbf2-bf46-49ca-b46f-112de5d79852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:31.510462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fdbf2-bf46-49ca-b46f-112de5d79852 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:31.512425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fdbf2-bf46-49ca-b46f-112de5d79852 admin admin] PATH_INFO: `/v3/projects/d4d39b386a334518816667ae969afe65` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:31.514813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fdbf2-bf46-49ca-b46f-112de5d79852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4d39b386a334518816667ae969afe65)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:31.517385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fdbf2-bf46-49ca-b46f-112de5d79852 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:31.555759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:31.647302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-816fdbf2-bf46-49ca-b46f-112de5d79852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4d39b386a334518816667ae969afe65', 'request_id': 'req-816fdbf2-bf46-49ca-b46f-112de5d79852'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:31.671585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-816fdbf2-bf46-49ca-b46f-112de5d79852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4d39b386a334518816667ae969afe65', 'request_id': 'req-816fdbf2-bf46-49ca-b46f-112de5d79852'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:31.680556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-816fdbf2-bf46-49ca-b46f-112de5d79852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4d39b386a334518816667ae969afe65', 'request_id': 'req-816fdbf2-bf46-49ca-b46f-112de5d79852'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:31.697551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-816fdbf2-bf46-49ca-b46f-112de5d79852 admin admin] The token cache is being invalidate because project d4d39b386a334518816667ae969afe65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:18:31.697857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-816fdbf2-bf46-49ca-b46f-112de5d79852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:31.698663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-816fdbf2-bf46-49ca-b46f-112de5d79852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:31.705073 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3833/7721] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:18:31 2022] DELETE /identity/v3/projects/d4d39b386a334518816667ae969afe65 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:31.706742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238319d-7f7c-4298-af96-cd14c9c4ef27 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:31.707564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238319d-7f7c-4298-af96-cd14c9c4ef27 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:31.708036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238319d-7f7c-4298-af96-cd14c9c4ef27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:31.737290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5238319d-7f7c-4298-af96-cd14c9c4ef27 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:31.739159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5238319d-7f7c-4298-af96-cd14c9c4ef27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:31.872944 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3889/7722] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:18:31.883023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcadfe-fb35-48ec-927a-1c3b9aaae466 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:31.883464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcadfe-fb35-48ec-927a-1c3b9aaae466 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:31.883744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcadfe-fb35-48ec-927a-1c3b9aaae466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:31.925690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a0dcadfe-fb35-48ec-927a-1c3b9aaae466 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:31.927072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a0dcadfe-fb35-48ec-927a-1c3b9aaae466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:31.944878 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3834/7723] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:31.945874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238319d-7f7c-4298-af96-cd14c9c4ef27 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:31.959298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238319d-7f7c-4298-af96-cd14c9c4ef27 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:31.966152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0769c4-ac82-4631-9c3e-4e8fa12c6d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:31.967270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0769c4-ac82-4631-9c3e-4e8fa12c6d63 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:31.967788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0769c4-ac82-4631-9c3e-4e8fa12c6d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:31.968274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0769c4-ac82-4631-9c3e-4e8fa12c6d63 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:31.969726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0769c4-ac82-4631-9c3e-4e8fa12c6d63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:18:31.970256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0769c4-ac82-4631-9c3e-4e8fa12c6d63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:31.975570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0769c4-ac82-4631-9c3e-4e8fa12c6d63 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:31.998069 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3890/7724] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:18:32.001986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dcadfe-fb35-48ec-927a-1c3b9aaae466 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:32.019648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dcadfe-fb35-48ec-927a-1c3b9aaae466 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:32.032453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4a8ff-cd84-4bce-9899-31b765c21473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:32.035902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4a8ff-cd84-4bce-9899-31b765c21473 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:32.036488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4a8ff-cd84-4bce-9899-31b765c21473 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:32.036917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4a8ff-cd84-4bce-9899-31b765c21473 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:32.037998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4a8ff-cd84-4bce-9899-31b765c21473 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:32.043264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4a8ff-cd84-4bce-9899-31b765c21473 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:32.098152 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3835/7725] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:18:32 2022] POST /identity/v3/projects => generated 393 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:18:32.099657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0769c4-ac82-4631-9c3e-4e8fa12c6d63 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:32.116613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0769c4-ac82-4631-9c3e-4e8fa12c6d63 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:32.161159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:32.237638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35fbe5-e76e-4f26-9d13-9c5397a67c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:32.238484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35fbe5-e76e-4f26-9d13-9c5397a67c99 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:32.238958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35fbe5-e76e-4f26-9d13-9c5397a67c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:32.239576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35fbe5-e76e-4f26-9d13-9c5397a67c99 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:32.240331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35fbe5-e76e-4f26-9d13-9c5397a67c99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:32.242717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35fbe5-e76e-4f26-9d13-9c5397a67c99 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:32.277537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4a8ff-cd84-4bce-9899-31b765c21473 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:32.288084 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3891/7726] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:18:32 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:18:32.291903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4a8ff-cd84-4bce-9899-31b765c21473 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:32.297957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5c5da-20a3-47d8-9cbd-821a6253f669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:32.298616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5c5da-20a3-47d8-9cbd-821a6253f669 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:32.298897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5c5da-20a3-47d8-9cbd-821a6253f669 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:32.299165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5c5da-20a3-47d8-9cbd-821a6253f669 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:32.299955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5c5da-20a3-47d8-9cbd-821a6253f669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:32.301581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5c5da-20a3-47d8-9cbd-821a6253f669 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:32.321172 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3836/7727] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:18:32.324986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35fbe5-e76e-4f26-9d13-9c5397a67c99 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:32.344132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35fbe5-e76e-4f26-9d13-9c5397a67c99 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:32.352837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cae133-884e-4d1b-a927-527936f90707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:32.355528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cae133-884e-4d1b-a927-527936f90707 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:32.356715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cae133-884e-4d1b-a927-527936f90707 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:32.356715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cae133-884e-4d1b-a927-527936f90707 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:32.358350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cae133-884e-4d1b-a927-527936f90707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:32.360647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cae133-884e-4d1b-a927-527936f90707 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:32.382532 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3892/7728] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:18:32.383569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5c5da-20a3-47d8-9cbd-821a6253f669 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:32.399449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5c5da-20a3-47d8-9cbd-821a6253f669 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:32.407416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57accc-4095-4c69-a623-a00757908175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:32.408799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57accc-4095-4c69-a623-a00757908175 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:32.408799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57accc-4095-4c69-a623-a00757908175 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:32.409237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57accc-4095-4c69-a623-a00757908175 admin admin] PATH_INFO: `/v3/projects/2cc0a640e91c4ca8ac4593d20fc4114c/users/b5012db78b0149b1aa7dcd2a71b660a7/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:32.433020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57accc-4095-4c69-a623-a00757908175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cc0a640e91c4ca8ac4593d20fc4114c, user_id=b5012db78b0149b1aa7dcd2a71b660a7, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:32.436394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57accc-4095-4c69-a623-a00757908175 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:32.466935 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3837/7729] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:18:32 2022] PUT /identity/v3/projects/2cc0a640e91c4ca8ac4593d20fc4114c/users/b5012db78b0149b1aa7dcd2a71b660a7/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:32.469316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427425c-9e33-403b-bc7a-33586ce024fd None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:32.469916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427425c-9e33-403b-bc7a-33586ce024fd None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:32.470397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427425c-9e33-403b-bc7a-33586ce024fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:32.501868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b427425c-9e33-403b-bc7a-33586ce024fd None None] MFA Rules not processed for user `b5012db78b0149b1aa7dcd2a71b660a7`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:32.504248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b427425c-9e33-403b-bc7a-33586ce024fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:32.604246 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3893/7730] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:32.605007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57accc-4095-4c69-a623-a00757908175 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:32.734005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57accc-4095-4c69-a623-a00757908175 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:32.740137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18c1ea-ede7-4bc8-92e4-144b763136a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:32.740965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18c1ea-ede7-4bc8-92e4-144b763136a5 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:32.741320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18c1ea-ede7-4bc8-92e4-144b763136a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:32.741669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18c1ea-ede7-4bc8-92e4-144b763136a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:32.743332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18c1ea-ede7-4bc8-92e4-144b763136a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:32.745500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18c1ea-ede7-4bc8-92e4-144b763136a5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:32.791292 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3838/7731] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:18:32 2022] POST /identity/v3/projects => generated 393 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:18:32.796665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427425c-9e33-403b-bc7a-33586ce024fd None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:32.812040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427425c-9e33-403b-bc7a-33586ce024fd None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:32.915449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5b61b-ec5d-4486-9dd6-f901783e1ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:32.916095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5b61b-ec5d-4486-9dd6-f901783e1ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:32.916383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5b61b-ec5d-4486-9dd6-f901783e1ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:32.916662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5b61b-ec5d-4486-9dd6-f901783e1ba6 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:32.917415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5b61b-ec5d-4486-9dd6-f901783e1ba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:32.919590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5b61b-ec5d-4486-9dd6-f901783e1ba6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:32.969729 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3894/7732] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:18:32 2022] POST /identity/v3/users => generated 303 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:18:32.971062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18c1ea-ede7-4bc8-92e4-144b763136a5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:32.984800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18c1ea-ede7-4bc8-92e4-144b763136a5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:33.001354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9c3c6-6dbc-472f-badd-9f1820172efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:33.002045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9c3c6-6dbc-472f-badd-9f1820172efb admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:33.002679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9c3c6-6dbc-472f-badd-9f1820172efb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:33.003095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9c3c6-6dbc-472f-badd-9f1820172efb admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:33.004159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9c3c6-6dbc-472f-badd-9f1820172efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:33.005586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9c3c6-6dbc-472f-badd-9f1820172efb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:33.034060 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3839/7733] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:18:33.036144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5b61b-ec5d-4486-9dd6-f901783e1ba6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:33.051295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5b61b-ec5d-4486-9dd6-f901783e1ba6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:33.058780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a0bee-884d-4468-8638-ae8dca3bbadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:33.059483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a0bee-884d-4468-8638-ae8dca3bbadb admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:33.059880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a0bee-884d-4468-8638-ae8dca3bbadb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:33.060219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a0bee-884d-4468-8638-ae8dca3bbadb admin admin] PATH_INFO: `/v3/projects/acfec70863e54957a46174536e9a0c3e/users/c96380523e8c463b91d8474cd930fb24/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:33.104201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a0bee-884d-4468-8638-ae8dca3bbadb admin admin] RBAC: Authorizing `identity:create_grant(project_id=acfec70863e54957a46174536e9a0c3e, user_id=c96380523e8c463b91d8474cd930fb24, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:33.106519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a0bee-884d-4468-8638-ae8dca3bbadb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:33.131497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:33.148886 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3895/7734] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:18:33 2022] PUT /identity/v3/projects/acfec70863e54957a46174536e9a0c3e/users/c96380523e8c463b91d8474cd930fb24/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:33.156342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9c3c6-6dbc-472f-badd-9f1820172efb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:33.180669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9c3c6-6dbc-472f-badd-9f1820172efb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:33.210606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:33.282088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3147c05-c851-4ea7-a0bf-27a8d4fb89a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:33.282841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3147c05-c851-4ea7-a0bf-27a8d4fb89a2 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:33.283338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3147c05-c851-4ea7-a0bf-27a8d4fb89a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:33.283693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3147c05-c851-4ea7-a0bf-27a8d4fb89a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:33.284418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3147c05-c851-4ea7-a0bf-27a8d4fb89a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:33.286138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3147c05-c851-4ea7-a0bf-27a8d4fb89a2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:33.300957 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3840/7735] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:18:33.304852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a0bee-884d-4468-8638-ae8dca3bbadb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:33.317633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a0bee-884d-4468-8638-ae8dca3bbadb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:33.331776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d25e9-b0d7-445c-b83a-69833e42b082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:33.333049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d25e9-b0d7-445c-b83a-69833e42b082 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:33.333465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d25e9-b0d7-445c-b83a-69833e42b082 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:33.334963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d25e9-b0d7-445c-b83a-69833e42b082 admin admin] PATH_INFO: `/v3/projects/acfec70863e54957a46174536e9a0c3e/users/c96380523e8c463b91d8474cd930fb24/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:33.343871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d25e9-b0d7-445c-b83a-69833e42b082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acfec70863e54957a46174536e9a0c3e, user_id=c96380523e8c463b91d8474cd930fb24, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:33.351075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d25e9-b0d7-445c-b83a-69833e42b082 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:33.383384 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3896/7736] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:18:33 2022] PUT /identity/v3/projects/acfec70863e54957a46174536e9a0c3e/users/c96380523e8c463b91d8474cd930fb24/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:33.397288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa633cb-10cb-415e-8722-fb808857b5dd None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:33.397813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa633cb-10cb-415e-8722-fb808857b5dd None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:33.397813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa633cb-10cb-415e-8722-fb808857b5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:33.468759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0aa633cb-10cb-415e-8722-fb808857b5dd None None] MFA Rules not processed for user `c96380523e8c463b91d8474cd930fb24`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:33.471213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0aa633cb-10cb-415e-8722-fb808857b5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:33.640629 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3841/7737] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:33 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:33.658773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d25e9-b0d7-445c-b83a-69833e42b082 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:33.660747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c97d25e9-b0d7-445c-b83a-69833e42b082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:33.802352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d25e9-b0d7-445c-b83a-69833e42b082 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:33.806245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a41a14-21fb-4b79-8bbf-2db3f2df0e10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:33.806882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a41a14-21fb-4b79-8bbf-2db3f2df0e10 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:33.807159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a41a14-21fb-4b79-8bbf-2db3f2df0e10 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:33.811257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a41a14-21fb-4b79-8bbf-2db3f2df0e10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:33.824273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a41a14-21fb-4b79-8bbf-2db3f2df0e10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:33.825853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a41a14-21fb-4b79-8bbf-2db3f2df0e10 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:33.839581 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3897/7738] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:18:33 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:18:33.930845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa633cb-10cb-415e-8722-fb808857b5dd None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:33.932679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0aa633cb-10cb-415e-8722-fb808857b5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:33.945690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b1cad-19e3-49bd-b9da-5de5d9da2ef3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:33.946133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b1cad-19e3-49bd-b9da-5de5d9da2ef3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:33.946409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b1cad-19e3-49bd-b9da-5de5d9da2ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:33.957691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa633cb-10cb-415e-8722-fb808857b5dd None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:33.964323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b811c7-bebd-4c21-9d55-95db848fcfdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:33.965346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b811c7-bebd-4c21-9d55-95db848fcfdd service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:33.965656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b811c7-bebd-4c21-9d55-95db848fcfdd service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:33.966621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b811c7-bebd-4c21-9d55-95db848fcfdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:33.990638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-384b1cad-19e3-49bd-b9da-5de5d9da2ef3 None None] MFA Rules not processed for user `3066b652821f47f6909892c9b04f4236`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:33.991950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-384b1cad-19e3-49bd-b9da-5de5d9da2ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:34.067560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b811c7-bebd-4c21-9d55-95db848fcfdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:34.069467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b811c7-bebd-4c21-9d55-95db848fcfdd service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:34.091508 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3842/7739] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:18:33 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:18:34.130253 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3898/7740] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:18:38.558768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b811c7-bebd-4c21-9d55-95db848fcfdd service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:38.644464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:38.716568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b811c7-bebd-4c21-9d55-95db848fcfdd service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:38.723385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba6c55-00c0-48ba-8600-2319b81b347b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:38.724283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba6c55-00c0-48ba-8600-2319b81b347b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:38.724725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba6c55-00c0-48ba-8600-2319b81b347b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:38.725171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba6c55-00c0-48ba-8600-2319b81b347b admin admin] PATH_INFO: `/v3/users/b5012db78b0149b1aa7dcd2a71b660a7` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:38.727031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba6c55-00c0-48ba-8600-2319b81b347b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5012db78b0149b1aa7dcd2a71b660a7)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:38.730468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba6c55-00c0-48ba-8600-2319b81b347b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:38.836157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1dba6c55-00c0-48ba-8600-2319b81b347b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5012db78b0149b1aa7dcd2a71b660a7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:38.837106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1dba6c55-00c0-48ba-8600-2319b81b347b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5012db78b0149b1aa7dcd2a71b660a7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:38.838063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1dba6c55-00c0-48ba-8600-2319b81b347b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5012db78b0149b1aa7dcd2a71b660a7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:38.857735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1dba6c55-00c0-48ba-8600-2319b81b347b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5012db78b0149b1aa7dcd2a71b660a7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:38.889868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1dba6c55-00c0-48ba-8600-2319b81b347b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5012db78b0149b1aa7dcd2a71b660a7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:38.908057 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3843/7741] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:18:38 2022] DELETE /identity/v3/users/b5012db78b0149b1aa7dcd2a71b660a7 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:38.912684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9141f7-57ce-46f9-9523-c4750bd346fe None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:38.913174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9141f7-57ce-46f9-9523-c4750bd346fe None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:38.913294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9141f7-57ce-46f9-9523-c4750bd346fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:38.937117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5a9141f7-57ce-46f9-9523-c4750bd346fe None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:38.938576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5a9141f7-57ce-46f9-9523-c4750bd346fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:39.021271 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3899/7742] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:39.029018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba6c55-00c0-48ba-8600-2319b81b347b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:39.030462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1dba6c55-00c0-48ba-8600-2319b81b347b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:39.106029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba6c55-00c0-48ba-8600-2319b81b347b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:39.110714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f5fce-0add-49ca-9010-43a9e0fad36c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:39.111340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f5fce-0add-49ca-9010-43a9e0fad36c service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:39.111626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f5fce-0add-49ca-9010-43a9e0fad36c service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:39.111901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f5fce-0add-49ca-9010-43a9e0fad36c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:39.134130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f5fce-0add-49ca-9010-43a9e0fad36c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:39.135850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f5fce-0add-49ca-9010-43a9e0fad36c service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:39.147652 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3844/7743] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:18:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:18:39.772981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9141f7-57ce-46f9-9523-c4750bd346fe None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:39.776168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5a9141f7-57ce-46f9-9523-c4750bd346fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:39.806699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9141f7-57ce-46f9-9523-c4750bd346fe None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:39.836807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79fdc8-fca5-40e0-bf2c-121d4cfbe0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:39.837866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79fdc8-fca5-40e0-bf2c-121d4cfbe0c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:39.838255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79fdc8-fca5-40e0-bf2c-121d4cfbe0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:39.839044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79fdc8-fca5-40e0-bf2c-121d4cfbe0c3 admin admin] PATH_INFO: `/v3/projects/2cc0a640e91c4ca8ac4593d20fc4114c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:39.841792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79fdc8-fca5-40e0-bf2c-121d4cfbe0c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cc0a640e91c4ca8ac4593d20fc4114c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:39.844218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79fdc8-fca5-40e0-bf2c-121d4cfbe0c3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:39.905609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:39.926037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3d79fdc8-fca5-40e0-bf2c-121d4cfbe0c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cc0a640e91c4ca8ac4593d20fc4114c', 'request_id': 'req-3d79fdc8-fca5-40e0-bf2c-121d4cfbe0c3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:39.933309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3d79fdc8-fca5-40e0-bf2c-121d4cfbe0c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cc0a640e91c4ca8ac4593d20fc4114c', 'request_id': 'req-3d79fdc8-fca5-40e0-bf2c-121d4cfbe0c3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:39.942962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3d79fdc8-fca5-40e0-bf2c-121d4cfbe0c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cc0a640e91c4ca8ac4593d20fc4114c', 'request_id': 'req-3d79fdc8-fca5-40e0-bf2c-121d4cfbe0c3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:39.958725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3d79fdc8-fca5-40e0-bf2c-121d4cfbe0c3 admin admin] The token cache is being invalidate because project 2cc0a640e91c4ca8ac4593d20fc4114c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:18:39.959419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3d79fdc8-fca5-40e0-bf2c-121d4cfbe0c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:39.960395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3d79fdc8-fca5-40e0-bf2c-121d4cfbe0c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:39.962519 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3900/7744] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:18:39 2022] DELETE /identity/v3/projects/2cc0a640e91c4ca8ac4593d20fc4114c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:39.967259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f5fce-0add-49ca-9010-43a9e0fad36c service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:39.999171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f5fce-0add-49ca-9010-43a9e0fad36c service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:40.098607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:40.115078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e9ab0-7a54-4ced-8612-dd8291dbcb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:40.119364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e9ab0-7a54-4ced-8612-dd8291dbcb16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:40.119987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e9ab0-7a54-4ced-8612-dd8291dbcb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:40.120452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e9ab0-7a54-4ced-8612-dd8291dbcb16 admin admin] PATH_INFO: `/v3/users/c96380523e8c463b91d8474cd930fb24` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:40.122241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e9ab0-7a54-4ced-8612-dd8291dbcb16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c96380523e8c463b91d8474cd930fb24)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:40.124948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e9ab0-7a54-4ced-8612-dd8291dbcb16 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:40.210239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a65e9ab0-7a54-4ced-8612-dd8291dbcb16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c96380523e8c463b91d8474cd930fb24'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:40.210955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a65e9ab0-7a54-4ced-8612-dd8291dbcb16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c96380523e8c463b91d8474cd930fb24'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:40.218572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a65e9ab0-7a54-4ced-8612-dd8291dbcb16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c96380523e8c463b91d8474cd930fb24'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:40.230215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a65e9ab0-7a54-4ced-8612-dd8291dbcb16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c96380523e8c463b91d8474cd930fb24'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:40.252247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a65e9ab0-7a54-4ced-8612-dd8291dbcb16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c96380523e8c463b91d8474cd930fb24'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:40.269998 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3845/7745] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:18:39 2022] DELETE /identity/v3/users/c96380523e8c463b91d8474cd930fb24 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:40.289341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841afb6-f43c-4f52-b3c4-ad26025bcedc None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:40.289807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841afb6-f43c-4f52-b3c4-ad26025bcedc None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:40.290093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841afb6-f43c-4f52-b3c4-ad26025bcedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:40.327893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d841afb6-f43c-4f52-b3c4-ad26025bcedc None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:40.329424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d841afb6-f43c-4f52-b3c4-ad26025bcedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:40.428861 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3901/7746] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:18:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:18:40.513346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e9ab0-7a54-4ced-8612-dd8291dbcb16 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:40.516385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a65e9ab0-7a54-4ced-8612-dd8291dbcb16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:40.727820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e9ab0-7a54-4ced-8612-dd8291dbcb16 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:40.734848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d8b73-4dd7-4986-b33e-07a68afe13d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:40.735608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d8b73-4dd7-4986-b33e-07a68afe13d8 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:40.735983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d8b73-4dd7-4986-b33e-07a68afe13d8 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:40.736367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d8b73-4dd7-4986-b33e-07a68afe13d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:40.747802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d8b73-4dd7-4986-b33e-07a68afe13d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:40.749612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d8b73-4dd7-4986-b33e-07a68afe13d8 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:40.760054 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3846/7747] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:18:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 21:18:41.035287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841afb6-f43c-4f52-b3c4-ad26025bcedc None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:41.036839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d841afb6-f43c-4f52-b3c4-ad26025bcedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:41.196874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841afb6-f43c-4f52-b3c4-ad26025bcedc None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:41.204873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca7042-e3a1-4129-b42a-480d89e2637a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:41.205769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca7042-e3a1-4129-b42a-480d89e2637a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:41.206227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca7042-e3a1-4129-b42a-480d89e2637a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:41.206722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca7042-e3a1-4129-b42a-480d89e2637a admin admin] PATH_INFO: `/v3/projects/acfec70863e54957a46174536e9a0c3e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:41.208307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca7042-e3a1-4129-b42a-480d89e2637a admin admin] RBAC: Authorizing `identity:delete_project(project_id=acfec70863e54957a46174536e9a0c3e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:41.214286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca7042-e3a1-4129-b42a-480d89e2637a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:41.382248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eaca7042-e3a1-4129-b42a-480d89e2637a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acfec70863e54957a46174536e9a0c3e', 'request_id': 'req-eaca7042-e3a1-4129-b42a-480d89e2637a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:41.391363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eaca7042-e3a1-4129-b42a-480d89e2637a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acfec70863e54957a46174536e9a0c3e', 'request_id': 'req-eaca7042-e3a1-4129-b42a-480d89e2637a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:41.395765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:41.404510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eaca7042-e3a1-4129-b42a-480d89e2637a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acfec70863e54957a46174536e9a0c3e', 'request_id': 'req-eaca7042-e3a1-4129-b42a-480d89e2637a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:41.445517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eaca7042-e3a1-4129-b42a-480d89e2637a admin admin] The token cache is being invalidate because project acfec70863e54957a46174536e9a0c3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:18:41.445517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eaca7042-e3a1-4129-b42a-480d89e2637a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:41.445517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-eaca7042-e3a1-4129-b42a-480d89e2637a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:41.446956 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3902/7748] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:18:41 2022] DELETE /identity/v3/projects/acfec70863e54957a46174536e9a0c3e => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:41.449234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4858348-5cad-46e3-8d75-3a3c69bdd6ca None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:41.449234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4858348-5cad-46e3-8d75-3a3c69bdd6ca None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:41.449234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4858348-5cad-46e3-8d75-3a3c69bdd6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:41.489413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c4858348-5cad-46e3-8d75-3a3c69bdd6ca None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:41.490605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c4858348-5cad-46e3-8d75-3a3c69bdd6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:41.617740 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3847/7749] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:41.619682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406313ec-a376-4088-8706-33047d573096 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:41.620189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406313ec-a376-4088-8706-33047d573096 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:41.620691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406313ec-a376-4088-8706-33047d573096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:41.685412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-406313ec-a376-4088-8706-33047d573096 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:41.687054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-406313ec-a376-4088-8706-33047d573096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:41.696052 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3903/7750] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:41.703128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4858348-5cad-46e3-8d75-3a3c69bdd6ca None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:41.713547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4858348-5cad-46e3-8d75-3a3c69bdd6ca None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:41.718674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c868d-fedd-478f-a1b1-b0b5a55143d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:41.719286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c868d-fedd-478f-a1b1-b0b5a55143d3 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:41.719589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c868d-fedd-478f-a1b1-b0b5a55143d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:41.719847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c868d-fedd-478f-a1b1-b0b5a55143d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:41.720497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c868d-fedd-478f-a1b1-b0b5a55143d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:18:41.720800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c868d-fedd-478f-a1b1-b0b5a55143d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:41.722410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c868d-fedd-478f-a1b1-b0b5a55143d3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:41.724114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:41.738551 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3848/7751] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:18:41.742732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406313ec-a376-4088-8706-33047d573096 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:41.758083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406313ec-a376-4088-8706-33047d573096 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:41.765999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437e983-e3ca-4918-8c1f-bccb34886cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:41.766749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437e983-e3ca-4918-8c1f-bccb34886cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:41.767122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437e983-e3ca-4918-8c1f-bccb34886cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:41.767509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437e983-e3ca-4918-8c1f-bccb34886cc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:41.768393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437e983-e3ca-4918-8c1f-bccb34886cc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:41.770440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437e983-e3ca-4918-8c1f-bccb34886cc8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:41.819415 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3904/7752] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:18:41 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:18:41.822468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c868d-fedd-478f-a1b1-b0b5a55143d3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:41.837383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c868d-fedd-478f-a1b1-b0b5a55143d3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:41.919675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7d7de-c5e0-4915-9990-27654e78d683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:41.920555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7d7de-c5e0-4915-9990-27654e78d683 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:41.921264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7d7de-c5e0-4915-9990-27654e78d683 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:41.921686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7d7de-c5e0-4915-9990-27654e78d683 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:41.922513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7d7de-c5e0-4915-9990-27654e78d683 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:41.924598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7d7de-c5e0-4915-9990-27654e78d683 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:41.959141 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3849/7753] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:18:41 2022] POST /identity/v3/users => generated 287 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:18:41.960760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437e983-e3ca-4918-8c1f-bccb34886cc8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:41.981662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437e983-e3ca-4918-8c1f-bccb34886cc8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:41.987191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38991058-5a30-467e-89c0-61f83f42b9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:41.988166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38991058-5a30-467e-89c0-61f83f42b9fe admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:41.988558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38991058-5a30-467e-89c0-61f83f42b9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:41.988937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38991058-5a30-467e-89c0-61f83f42b9fe admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:41.989752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38991058-5a30-467e-89c0-61f83f42b9fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:41.991543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38991058-5a30-467e-89c0-61f83f42b9fe admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:42.010404 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3905/7754] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:18:42.011878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7d7de-c5e0-4915-9990-27654e78d683 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:42.039464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7d7de-c5e0-4915-9990-27654e78d683 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:42.045303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddad0c-dba7-41fb-9786-3d2a1917eb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:42.046182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddad0c-dba7-41fb-9786-3d2a1917eb12 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:42.046461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddad0c-dba7-41fb-9786-3d2a1917eb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:42.046767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddad0c-dba7-41fb-9786-3d2a1917eb12 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:42.047409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddad0c-dba7-41fb-9786-3d2a1917eb12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:42.049010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddad0c-dba7-41fb-9786-3d2a1917eb12 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:42.068741 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3850/7755] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:18:42.068951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38991058-5a30-467e-89c0-61f83f42b9fe admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:42.081030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38991058-5a30-467e-89c0-61f83f42b9fe admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:42.086709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc261342-5c6c-44f9-9b75-b31576d6c68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:42.086921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc261342-5c6c-44f9-9b75-b31576d6c68d admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:42.087095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc261342-5c6c-44f9-9b75-b31576d6c68d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:42.087369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc261342-5c6c-44f9-9b75-b31576d6c68d admin admin] PATH_INFO: `/v3/projects/f349ee99765b4d12a33b415bf5d07458/users/8b876c2dc2f440d0bc1a929f5a89b544/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:42.112944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc261342-5c6c-44f9-9b75-b31576d6c68d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f349ee99765b4d12a33b415bf5d07458, user_id=8b876c2dc2f440d0bc1a929f5a89b544, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:42.116496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc261342-5c6c-44f9-9b75-b31576d6c68d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:42.136435 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3906/7756] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:18:42 2022] PUT /identity/v3/projects/f349ee99765b4d12a33b415bf5d07458/users/8b876c2dc2f440d0bc1a929f5a89b544/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:42.148446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b92cb-c2ef-4ab0-8999-f23180b19c59 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:42.148750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b92cb-c2ef-4ab0-8999-f23180b19c59 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:42.149032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b92cb-c2ef-4ab0-8999-f23180b19c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:42.188536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f64b92cb-c2ef-4ab0-8999-f23180b19c59 None None] MFA Rules not processed for user `8b876c2dc2f440d0bc1a929f5a89b544`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:42.190118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f64b92cb-c2ef-4ab0-8999-f23180b19c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:42.288942 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3851/7757] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:42.300525 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc261342-5c6c-44f9-9b75-b31576d6c68d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:42.304279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cc261342-5c6c-44f9-9b75-b31576d6c68d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:42.408508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc261342-5c6c-44f9-9b75-b31576d6c68d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:42.413022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddbb58-4fde-45a0-a0d6-92a08cf3611d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:42.413913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddbb58-4fde-45a0-a0d6-92a08cf3611d service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:42.414840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddbb58-4fde-45a0-a0d6-92a08cf3611d service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:42.415307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddbb58-4fde-45a0-a0d6-92a08cf3611d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:42.428077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddbb58-4fde-45a0-a0d6-92a08cf3611d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:42.429785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddbb58-4fde-45a0-a0d6-92a08cf3611d service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:42.440418 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3907/7758] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:18:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:18:42.467472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b92cb-c2ef-4ab0-8999-f23180b19c59 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:42.540515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b92cb-c2ef-4ab0-8999-f23180b19c59 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:42.547571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69b644-b080-4dc9-a00f-e655ff1edaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:42.548474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69b644-b080-4dc9-a00f-e655ff1edaa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:42.548823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69b644-b080-4dc9-a00f-e655ff1edaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:42.549114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69b644-b080-4dc9-a00f-e655ff1edaa7 admin admin] PATH_INFO: `/v3/users/8b876c2dc2f440d0bc1a929f5a89b544` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:42.550464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69b644-b080-4dc9-a00f-e655ff1edaa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b876c2dc2f440d0bc1a929f5a89b544)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:42.552786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69b644-b080-4dc9-a00f-e655ff1edaa7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:42.556180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:42.626179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ed69b644-b080-4dc9-a00f-e655ff1edaa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b876c2dc2f440d0bc1a929f5a89b544'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:42.627844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ed69b644-b080-4dc9-a00f-e655ff1edaa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b876c2dc2f440d0bc1a929f5a89b544'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:42.628543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ed69b644-b080-4dc9-a00f-e655ff1edaa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b876c2dc2f440d0bc1a929f5a89b544'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:42.640480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ed69b644-b080-4dc9-a00f-e655ff1edaa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b876c2dc2f440d0bc1a929f5a89b544'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:42.656151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ed69b644-b080-4dc9-a00f-e655ff1edaa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b876c2dc2f440d0bc1a929f5a89b544'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:42.675820 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3852/7759] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:18:42 2022] DELETE /identity/v3/users/8b876c2dc2f440d0bc1a929f5a89b544 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:42.681089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608dc493-9736-44df-8f69-1fb0a6e7e222 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:42.681372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608dc493-9736-44df-8f69-1fb0a6e7e222 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:42.682199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608dc493-9736-44df-8f69-1fb0a6e7e222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:42.702923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-608dc493-9736-44df-8f69-1fb0a6e7e222 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:42.704142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-608dc493-9736-44df-8f69-1fb0a6e7e222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:42.813385 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3908/7760] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:42.823704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69b644-b080-4dc9-a00f-e655ff1edaa7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:42.825206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ed69b644-b080-4dc9-a00f-e655ff1edaa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:42.963037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69b644-b080-4dc9-a00f-e655ff1edaa7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:42.983227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8edaa7-5b47-4ed0-a4eb-cb0ac92f20b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:42.984330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8edaa7-5b47-4ed0-a4eb-cb0ac92f20b0 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:42.985396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8edaa7-5b47-4ed0-a4eb-cb0ac92f20b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:42.985836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8edaa7-5b47-4ed0-a4eb-cb0ac92f20b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:43.008273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8edaa7-5b47-4ed0-a4eb-cb0ac92f20b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:43.010231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8edaa7-5b47-4ed0-a4eb-cb0ac92f20b0 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:43.040522 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3853/7761] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:18:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:18:43.531192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608dc493-9736-44df-8f69-1fb0a6e7e222 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:43.533976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-608dc493-9736-44df-8f69-1fb0a6e7e222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:43.568455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608dc493-9736-44df-8f69-1fb0a6e7e222 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:43.602695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5d7ab-410a-41b5-815b-e048145e49f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:43.603472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5d7ab-410a-41b5-815b-e048145e49f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:43.603848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5d7ab-410a-41b5-815b-e048145e49f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:43.604238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5d7ab-410a-41b5-815b-e048145e49f2 admin admin] PATH_INFO: `/v3/projects/f349ee99765b4d12a33b415bf5d07458` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:43.605590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5d7ab-410a-41b5-815b-e048145e49f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f349ee99765b4d12a33b415bf5d07458)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:43.613240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5d7ab-410a-41b5-815b-e048145e49f2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:43.673948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:43.730807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8e5d7ab-410a-41b5-815b-e048145e49f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f349ee99765b4d12a33b415bf5d07458', 'request_id': 'req-d8e5d7ab-410a-41b5-815b-e048145e49f2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:43.744020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8e5d7ab-410a-41b5-815b-e048145e49f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f349ee99765b4d12a33b415bf5d07458', 'request_id': 'req-d8e5d7ab-410a-41b5-815b-e048145e49f2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:43.754851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8e5d7ab-410a-41b5-815b-e048145e49f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f349ee99765b4d12a33b415bf5d07458', 'request_id': 'req-d8e5d7ab-410a-41b5-815b-e048145e49f2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:43.783295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8e5d7ab-410a-41b5-815b-e048145e49f2 admin admin] The token cache is being invalidate because project f349ee99765b4d12a33b415bf5d07458 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:18:43.783672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8e5d7ab-410a-41b5-815b-e048145e49f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:43.784501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d8e5d7ab-410a-41b5-815b-e048145e49f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:43.792261 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3909/7762] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:18:43 2022] DELETE /identity/v3/projects/f349ee99765b4d12a33b415bf5d07458 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:43.793660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2ecb7-43ab-4379-87ff-e322c7244e26 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:43.794151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2ecb7-43ab-4379-87ff-e322c7244e26 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:43.794254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2ecb7-43ab-4379-87ff-e322c7244e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:43.826361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1ba2ecb7-43ab-4379-87ff-e322c7244e26 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:43.828481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1ba2ecb7-43ab-4379-87ff-e322c7244e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:43.916766 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3854/7763] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:43.926463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9a740-f2ea-4df8-aa48-b0a058e5470a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:43.926792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9a740-f2ea-4df8-aa48-b0a058e5470a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:43.927058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9a740-f2ea-4df8-aa48-b0a058e5470a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:43.957780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fca9a740-f2ea-4df8-aa48-b0a058e5470a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:43.960064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fca9a740-f2ea-4df8-aa48-b0a058e5470a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:43.998687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2ecb7-43ab-4379-87ff-e322c7244e26 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:44.006136 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3910/7764] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:44.011581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2ecb7-43ab-4379-87ff-e322c7244e26 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:44.022537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589728ae-701a-41bd-9c7d-3899d45aedd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:44.023103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589728ae-701a-41bd-9c7d-3899d45aedd4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:44.023383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589728ae-701a-41bd-9c7d-3899d45aedd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:44.023652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589728ae-701a-41bd-9c7d-3899d45aedd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:44.024310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589728ae-701a-41bd-9c7d-3899d45aedd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:18:44.024610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589728ae-701a-41bd-9c7d-3899d45aedd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:44.030757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589728ae-701a-41bd-9c7d-3899d45aedd4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:44.072408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9a740-f2ea-4df8-aa48-b0a058e5470a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:44.082130 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3855/7765] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:18:44.089961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9a740-f2ea-4df8-aa48-b0a058e5470a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:44.097956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae7413-c792-4b4e-9ae3-155530193498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:44.099212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae7413-c792-4b4e-9ae3-155530193498 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:44.099212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae7413-c792-4b4e-9ae3-155530193498 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:44.099212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae7413-c792-4b4e-9ae3-155530193498 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:44.100010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae7413-c792-4b4e-9ae3-155530193498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:44.102021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae7413-c792-4b4e-9ae3-155530193498 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:44.155155 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3911/7766] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:18:44 2022] POST /identity/v3/projects => generated 377 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:18:44.157404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589728ae-701a-41bd-9c7d-3899d45aedd4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:44.172176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589728ae-701a-41bd-9c7d-3899d45aedd4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:44.182483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:44.341955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b266e-b9e6-4382-8aaf-1f8f748a5818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:44.342618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b266e-b9e6-4382-8aaf-1f8f748a5818 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:44.342898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b266e-b9e6-4382-8aaf-1f8f748a5818 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:44.343297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b266e-b9e6-4382-8aaf-1f8f748a5818 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:44.343951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b266e-b9e6-4382-8aaf-1f8f748a5818 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:44.345879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b266e-b9e6-4382-8aaf-1f8f748a5818 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:44.390868 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3856/7767] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:18:44 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:18:44.395021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae7413-c792-4b4e-9ae3-155530193498 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:44.413020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae7413-c792-4b4e-9ae3-155530193498 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:44.429443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d1506-98a7-4e6a-8ed7-31b4a754c10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:44.430078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d1506-98a7-4e6a-8ed7-31b4a754c10c admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:44.430360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d1506-98a7-4e6a-8ed7-31b4a754c10c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:44.430668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d1506-98a7-4e6a-8ed7-31b4a754c10c admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:44.431325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d1506-98a7-4e6a-8ed7-31b4a754c10c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:44.432917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d1506-98a7-4e6a-8ed7-31b4a754c10c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:44.459228 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3912/7768] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:18:44.462862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b266e-b9e6-4382-8aaf-1f8f748a5818 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:44.476599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b266e-b9e6-4382-8aaf-1f8f748a5818 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:44.483454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccf4b1-09bc-4e8c-ba17-a135fb725b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:44.484492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccf4b1-09bc-4e8c-ba17-a135fb725b25 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:44.484991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccf4b1-09bc-4e8c-ba17-a135fb725b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:44.485371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccf4b1-09bc-4e8c-ba17-a135fb725b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:44.486259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccf4b1-09bc-4e8c-ba17-a135fb725b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:44.488112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccf4b1-09bc-4e8c-ba17-a135fb725b25 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:44.505773 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3857/7769] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:18:44.506864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d1506-98a7-4e6a-8ed7-31b4a754c10c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:44.521133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d1506-98a7-4e6a-8ed7-31b4a754c10c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:44.529498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13c255-3d3e-4921-bea0-21740754cab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:44.530567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13c255-3d3e-4921-bea0-21740754cab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:44.531040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13c255-3d3e-4921-bea0-21740754cab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:44.531482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13c255-3d3e-4921-bea0-21740754cab5 admin admin] PATH_INFO: `/v3/projects/d3f7e49e1524439394e7a96d619c0c6d/users/33f3c126be294b1f8cd46f25d4087af0/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:44.555733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13c255-3d3e-4921-bea0-21740754cab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3f7e49e1524439394e7a96d619c0c6d, user_id=33f3c126be294b1f8cd46f25d4087af0, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:44.562067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13c255-3d3e-4921-bea0-21740754cab5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:44.587516 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3913/7770] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:18:44 2022] PUT /identity/v3/projects/d3f7e49e1524439394e7a96d619c0c6d/users/33f3c126be294b1f8cd46f25d4087af0/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:18:44.597944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e92e70-c491-4dee-bbe5-e2c534e00798 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:44.598227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e92e70-c491-4dee-bbe5-e2c534e00798 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:44.598516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e92e70-c491-4dee-bbe5-e2c534e00798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:44.633657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f1e92e70-c491-4dee-bbe5-e2c534e00798 None None] MFA Rules not processed for user `33f3c126be294b1f8cd46f25d4087af0`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:44.635130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f1e92e70-c491-4dee-bbe5-e2c534e00798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:44.717850 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3858/7771] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:44.727168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13c255-3d3e-4921-bea0-21740754cab5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:44.729115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bb13c255-3d3e-4921-bea0-21740754cab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:44.816398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13c255-3d3e-4921-bea0-21740754cab5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:44.821807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46db1f-67c9-4454-9a4c-2c74c63b106f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:44.823314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46db1f-67c9-4454-9a4c-2c74c63b106f service glance] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:44.823811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46db1f-67c9-4454-9a4c-2c74c63b106f service glance] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:44.824290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46db1f-67c9-4454-9a4c-2c74c63b106f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:44.839974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46db1f-67c9-4454-9a4c-2c74c63b106f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:44.843054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46db1f-67c9-4454-9a4c-2c74c63b106f service glance] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:44.854615 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3914/7772] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:18:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:18:48.328130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e92e70-c491-4dee-bbe5-e2c534e00798 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:48.329811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f1e92e70-c491-4dee-bbe5-e2c534e00798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:48.433256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e92e70-c491-4dee-bbe5-e2c534e00798 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:48.443734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b4495-6f4f-4d16-b584-690254afab58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:48.444402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b4495-6f4f-4d16-b584-690254afab58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:48.444684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b4495-6f4f-4d16-b584-690254afab58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:48.445350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b4495-6f4f-4d16-b584-690254afab58 admin admin] PATH_INFO: `/v3/users/0243052320b649a98e1329ca4014bdc8` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:48.447109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b4495-6f4f-4d16-b584-690254afab58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0243052320b649a98e1329ca4014bdc8)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:48.450445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b4495-6f4f-4d16-b584-690254afab58 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:48.485887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:48.588968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7b7b4495-6f4f-4d16-b584-690254afab58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0243052320b649a98e1329ca4014bdc8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:48.592111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7b7b4495-6f4f-4d16-b584-690254afab58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0243052320b649a98e1329ca4014bdc8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:48.606758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7b7b4495-6f4f-4d16-b584-690254afab58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0243052320b649a98e1329ca4014bdc8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:48.619085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7b7b4495-6f4f-4d16-b584-690254afab58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0243052320b649a98e1329ca4014bdc8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:48.635395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7b7b4495-6f4f-4d16-b584-690254afab58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0243052320b649a98e1329ca4014bdc8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:48.654304 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3859/7773] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:18:48 2022] DELETE /identity/v3/users/0243052320b649a98e1329ca4014bdc8 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:48.658479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df005ee-64ef-4fbb-a145-13884145e2ff None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:48.659080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df005ee-64ef-4fbb-a145-13884145e2ff None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:48.659080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df005ee-64ef-4fbb-a145-13884145e2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:48.700548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6df005ee-64ef-4fbb-a145-13884145e2ff None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:48.706206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6df005ee-64ef-4fbb-a145-13884145e2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:48.769280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:48.795744 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3915/7774] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:48.805842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b4495-6f4f-4d16-b584-690254afab58 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:48.807990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7b7b4495-6f4f-4d16-b584-690254afab58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:48.916439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b4495-6f4f-4d16-b584-690254afab58 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:48.923359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636242e0-728d-4924-82e2-7f070e6558cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:48.924418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636242e0-728d-4924-82e2-7f070e6558cc service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:48.924911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636242e0-728d-4924-82e2-7f070e6558cc service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:48.925565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636242e0-728d-4924-82e2-7f070e6558cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:48.939099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636242e0-728d-4924-82e2-7f070e6558cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:48.943695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636242e0-728d-4924-82e2-7f070e6558cc service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:48.960262 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3860/7775] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:18:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:18:49.220555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df005ee-64ef-4fbb-a145-13884145e2ff None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:49.222064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6df005ee-64ef-4fbb-a145-13884145e2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:49.244691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df005ee-64ef-4fbb-a145-13884145e2ff None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:49.268538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a3151-5e48-41e4-aaea-dc99d0711b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:49.269900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a3151-5e48-41e4-aaea-dc99d0711b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:49.270051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a3151-5e48-41e4-aaea-dc99d0711b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:49.270262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a3151-5e48-41e4-aaea-dc99d0711b16 admin admin] PATH_INFO: `/v3/projects/919ae1be1ab1486389836316fd2e367f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:49.271991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a3151-5e48-41e4-aaea-dc99d0711b16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=919ae1be1ab1486389836316fd2e367f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:49.273713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a3151-5e48-41e4-aaea-dc99d0711b16 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:49.365744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a4a3151-5e48-41e4-aaea-dc99d0711b16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '919ae1be1ab1486389836316fd2e367f', 'request_id': 'req-9a4a3151-5e48-41e4-aaea-dc99d0711b16'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:49.384284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a4a3151-5e48-41e4-aaea-dc99d0711b16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '919ae1be1ab1486389836316fd2e367f', 'request_id': 'req-9a4a3151-5e48-41e4-aaea-dc99d0711b16'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:49.396238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a4a3151-5e48-41e4-aaea-dc99d0711b16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '919ae1be1ab1486389836316fd2e367f', 'request_id': 'req-9a4a3151-5e48-41e4-aaea-dc99d0711b16'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:49.424463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a4a3151-5e48-41e4-aaea-dc99d0711b16 admin admin] The token cache is being invalidate because project 919ae1be1ab1486389836316fd2e367f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:18:49.424780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a4a3151-5e48-41e4-aaea-dc99d0711b16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:49.427550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9a4a3151-5e48-41e4-aaea-dc99d0711b16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:49.430717 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3916/7776] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:18:49 2022] DELETE /identity/v3/projects/919ae1be1ab1486389836316fd2e367f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:49.435573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb5992-9cbf-42b0-b807-24a6d0922b03 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:49.437554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb5992-9cbf-42b0-b807-24a6d0922b03 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:49.438028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb5992-9cbf-42b0-b807-24a6d0922b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:49.478268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-23cb5992-9cbf-42b0-b807-24a6d0922b03 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:49.479517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-23cb5992-9cbf-42b0-b807-24a6d0922b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:49.597503 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3861/7777] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:49.607140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05c480-d7da-4af6-a546-d4cc0b342034 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:49.607337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05c480-d7da-4af6-a546-d4cc0b342034 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:49.607618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05c480-d7da-4af6-a546-d4cc0b342034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:49.649425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6d05c480-d7da-4af6-a546-d4cc0b342034 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:49.651734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6d05c480-d7da-4af6-a546-d4cc0b342034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:49.665625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb5992-9cbf-42b0-b807-24a6d0922b03 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:49.667275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-23cb5992-9cbf-42b0-b807-24a6d0922b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:49.674545 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3917/7778] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:49.680087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05c480-d7da-4af6-a546-d4cc0b342034 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:49.681368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb5992-9cbf-42b0-b807-24a6d0922b03 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:49.688319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fecd34-d3d1-4a3e-825b-6700946f24f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:49.689049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fecd34-d3d1-4a3e-825b-6700946f24f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:49.689404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fecd34-d3d1-4a3e-825b-6700946f24f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:49.689798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fecd34-d3d1-4a3e-825b-6700946f24f5 admin admin] PATH_INFO: `/v3/users/33f3c126be294b1f8cd46f25d4087af0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:49.691762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fecd34-d3d1-4a3e-825b-6700946f24f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33f3c126be294b1f8cd46f25d4087af0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:49.701346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fecd34-d3d1-4a3e-825b-6700946f24f5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:49.708080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05c480-d7da-4af6-a546-d4cc0b342034 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:49.717253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecc35e-b9ea-4f29-954b-6b7913532eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:49.718311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecc35e-b9ea-4f29-954b-6b7913532eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:49.718993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecc35e-b9ea-4f29-954b-6b7913532eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:49.719567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecc35e-b9ea-4f29-954b-6b7913532eb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:49.720740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecc35e-b9ea-4f29-954b-6b7913532eb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:18:49.721364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecc35e-b9ea-4f29-954b-6b7913532eb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:49.729842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecc35e-b9ea-4f29-954b-6b7913532eb4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:49.754724 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3918/7779] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:18:49.762986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecc35e-b9ea-4f29-954b-6b7913532eb4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:49.768336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:49.780456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-12fecd34-d3d1-4a3e-825b-6700946f24f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33f3c126be294b1f8cd46f25d4087af0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:49.781232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-12fecd34-d3d1-4a3e-825b-6700946f24f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33f3c126be294b1f8cd46f25d4087af0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:49.781927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-12fecd34-d3d1-4a3e-825b-6700946f24f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33f3c126be294b1f8cd46f25d4087af0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:49.785641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecc35e-b9ea-4f29-954b-6b7913532eb4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:49.794203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39474f1-ac0a-4827-9b3a-d85155f45cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:49.795111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-12fecd34-d3d1-4a3e-825b-6700946f24f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33f3c126be294b1f8cd46f25d4087af0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:49.795642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39474f1-ac0a-4827-9b3a-d85155f45cf9 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:49.796437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39474f1-ac0a-4827-9b3a-d85155f45cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:49.796921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39474f1-ac0a-4827-9b3a-d85155f45cf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:49.798116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39474f1-ac0a-4827-9b3a-d85155f45cf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:49.800791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39474f1-ac0a-4827-9b3a-d85155f45cf9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:49.816091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-12fecd34-d3d1-4a3e-825b-6700946f24f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33f3c126be294b1f8cd46f25d4087af0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:49.838887 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3862/7780] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:18:49 2022] DELETE /identity/v3/users/33f3c126be294b1f8cd46f25d4087af0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:18:49.843573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce7505-2e97-4063-bb5f-4eba81996cfb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:49.844962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce7505-2e97-4063-bb5f-4eba81996cfb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:49.845454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce7505-2e97-4063-bb5f-4eba81996cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:49.876029 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3919/7781] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:18:49 2022] POST /identity/v3/projects => generated 359 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:18:49.879887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39474f1-ac0a-4827-9b3a-d85155f45cf9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:49.883712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a39474f1-ac0a-4827-9b3a-d85155f45cf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:49.885674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1dce7505-2e97-4063-bb5f-4eba81996cfb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:49.891558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1dce7505-2e97-4063-bb5f-4eba81996cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:49.975114 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3863/7782] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:49.999580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce7505-2e97-4063-bb5f-4eba81996cfb None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:50.001385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1dce7505-2e97-4063-bb5f-4eba81996cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:50.012748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39474f1-ac0a-4827-9b3a-d85155f45cf9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:50.032000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3f2b4-4c03-433a-abe6-827272c41d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:50.032639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3f2b4-4c03-433a-abe6-827272c41d2e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:50.032922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3f2b4-4c03-433a-abe6-827272c41d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:50.033196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3f2b4-4c03-433a-abe6-827272c41d2e admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:50.033845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3f2b4-4c03-433a-abe6-827272c41d2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:50.035867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3f2b4-4c03-433a-abe6-827272c41d2e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:50.084955 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3920/7783] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:18:49 2022] POST /identity/v3/users => generated 280 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:18:50.089744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3f2b4-4c03-433a-abe6-827272c41d2e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:50.106921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce7505-2e97-4063-bb5f-4eba81996cfb None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:50.107562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3f2b4-4c03-433a-abe6-827272c41d2e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:50.111168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48705e-37e5-496c-839e-50ecab5bc8dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:50.112098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48705e-37e5-496c-839e-50ecab5bc8dd service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:50.112244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48705e-37e5-496c-839e-50ecab5bc8dd service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:50.112478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48705e-37e5-496c-839e-50ecab5bc8dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:50.116311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c30a5-b085-4ccf-859c-bd8bdf64d07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:50.117199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c30a5-b085-4ccf-859c-bd8bdf64d07b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:50.117734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c30a5-b085-4ccf-859c-bd8bdf64d07b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:50.118219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c30a5-b085-4ccf-859c-bd8bdf64d07b admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:50.119087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c30a5-b085-4ccf-859c-bd8bdf64d07b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:50.120943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c30a5-b085-4ccf-859c-bd8bdf64d07b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:50.122823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48705e-37e5-496c-839e-50ecab5bc8dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:50.124493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48705e-37e5-496c-839e-50ecab5bc8dd service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:50.140023 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3921/7784] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:18:50.151438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c30a5-b085-4ccf-859c-bd8bdf64d07b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:50.152247 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3864/7785] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:18:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:18:50.164432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:50.174827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c30a5-b085-4ccf-859c-bd8bdf64d07b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:50.195975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736340b4-fc98-4f1b-86e4-aefbc206b96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:50.196881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736340b4-fc98-4f1b-86e4-aefbc206b96e admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:50.197953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736340b4-fc98-4f1b-86e4-aefbc206b96e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:50.198307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736340b4-fc98-4f1b-86e4-aefbc206b96e admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:50.203566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736340b4-fc98-4f1b-86e4-aefbc206b96e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:50.205489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736340b4-fc98-4f1b-86e4-aefbc206b96e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:50.227764 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3922/7786] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:18:50.233498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48705e-37e5-496c-839e-50ecab5bc8dd service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:50.256083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48705e-37e5-496c-839e-50ecab5bc8dd service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:50.267166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac978b-5fef-4e92-9c41-c4484dc28507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:50.268083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac978b-5fef-4e92-9c41-c4484dc28507 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:50.268478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac978b-5fef-4e92-9c41-c4484dc28507 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:50.268855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac978b-5fef-4e92-9c41-c4484dc28507 admin admin] PATH_INFO: `/v3/projects/f4fbc14980d642ab80d82945dcf337fb/users/af560738f1684441acb597d35753e24a/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:50.321232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac978b-5fef-4e92-9c41-c4484dc28507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4fbc14980d642ab80d82945dcf337fb, user_id=af560738f1684441acb597d35753e24a, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:50.323976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac978b-5fef-4e92-9c41-c4484dc28507 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:50.365569 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3865/7787] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:18:50 2022] PUT /identity/v3/projects/f4fbc14980d642ab80d82945dcf337fb/users/af560738f1684441acb597d35753e24a/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:50.376488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcb5a0-4720-4d41-a8ee-4891680ee935 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:50.376782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcb5a0-4720-4d41-a8ee-4891680ee935 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:50.377057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcb5a0-4720-4d41-a8ee-4891680ee935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:50.409159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-88dcb5a0-4720-4d41-a8ee-4891680ee935 None None] MFA Rules not processed for user `af560738f1684441acb597d35753e24a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:50.410561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-88dcb5a0-4720-4d41-a8ee-4891680ee935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:50.523602 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3923/7788] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:50 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:50.549055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac978b-5fef-4e92-9c41-c4484dc28507 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:50.550877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e9ac978b-5fef-4e92-9c41-c4484dc28507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:50.615672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:50.623790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dcb5a0-4720-4d41-a8ee-4891680ee935 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:50.625446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-88dcb5a0-4720-4d41-a8ee-4891680ee935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:50.691506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac978b-5fef-4e92-9c41-c4484dc28507 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:50.701390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdfb095-e42c-49dd-bc3b-7c5e1fc39b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:50.701390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfb095-e42c-49dd-bc3b-7c5e1fc39b8f service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:50.701734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfb095-e42c-49dd-bc3b-7c5e1fc39b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:50.701734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfb095-e42c-49dd-bc3b-7c5e1fc39b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:50.718732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdfb095-e42c-49dd-bc3b-7c5e1fc39b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:50.721047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdfb095-e42c-49dd-bc3b-7c5e1fc39b8f service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:50.735707 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3866/7789] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:18:50 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:18:50.770747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dcb5a0-4720-4d41-a8ee-4891680ee935 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:50.790232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02253bc1-62a6-468c-ac49-09e1423a02e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:50.790942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02253bc1-62a6-468c-ac49-09e1423a02e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:50.792302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02253bc1-62a6-468c-ac49-09e1423a02e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:50.794872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02253bc1-62a6-468c-ac49-09e1423a02e6 admin admin] PATH_INFO: `/v3/projects/d3f7e49e1524439394e7a96d619c0c6d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:50.799005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02253bc1-62a6-468c-ac49-09e1423a02e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3f7e49e1524439394e7a96d619c0c6d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:50.801016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02253bc1-62a6-468c-ac49-09e1423a02e6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:50.969550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-02253bc1-62a6-468c-ac49-09e1423a02e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3f7e49e1524439394e7a96d619c0c6d', 'request_id': 'req-02253bc1-62a6-468c-ac49-09e1423a02e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:50.989648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-02253bc1-62a6-468c-ac49-09e1423a02e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3f7e49e1524439394e7a96d619c0c6d', 'request_id': 'req-02253bc1-62a6-468c-ac49-09e1423a02e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:51.011519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-02253bc1-62a6-468c-ac49-09e1423a02e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3f7e49e1524439394e7a96d619c0c6d', 'request_id': 'req-02253bc1-62a6-468c-ac49-09e1423a02e6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:51.038235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-02253bc1-62a6-468c-ac49-09e1423a02e6 admin admin] The token cache is being invalidate because project d3f7e49e1524439394e7a96d619c0c6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:18:51.040549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-02253bc1-62a6-468c-ac49-09e1423a02e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:51.041827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-02253bc1-62a6-468c-ac49-09e1423a02e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:51.045359 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3924/7790] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:18:50 2022] DELETE /identity/v3/projects/d3f7e49e1524439394e7a96d619c0c6d => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:18:51.051378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154fb4b-31f9-4479-848d-e948a4b137e9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:51.052672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154fb4b-31f9-4479-848d-e948a4b137e9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:51.053198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154fb4b-31f9-4479-848d-e948a4b137e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:51.087877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5154fb4b-31f9-4479-848d-e948a4b137e9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:51.090190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5154fb4b-31f9-4479-848d-e948a4b137e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:51.277134 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3867/7791] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:51.285357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114413d-b315-488d-b1a4-278ea87337a4 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:51.285647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114413d-b315-488d-b1a4-278ea87337a4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:51.285932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114413d-b315-488d-b1a4-278ea87337a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:51.312305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2114413d-b315-488d-b1a4-278ea87337a4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:51.313683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2114413d-b315-488d-b1a4-278ea87337a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:51.327809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154fb4b-31f9-4479-848d-e948a4b137e9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:51.334228 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3925/7792] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:51.350997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154fb4b-31f9-4479-848d-e948a4b137e9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:51.369772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98053af3-c835-4b26-8559-ecbe92d69b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:51.370417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98053af3-c835-4b26-8559-ecbe92d69b9e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:51.370732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98053af3-c835-4b26-8559-ecbe92d69b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:51.371026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98053af3-c835-4b26-8559-ecbe92d69b9e admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:51.371741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98053af3-c835-4b26-8559-ecbe92d69b9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:18:51.372051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98053af3-c835-4b26-8559-ecbe92d69b9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:51.374129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98053af3-c835-4b26-8559-ecbe92d69b9e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:51.397282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114413d-b315-488d-b1a4-278ea87337a4 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:51.404009 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3868/7793] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:18:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:18:51.409163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114413d-b315-488d-b1a4-278ea87337a4 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:51.415761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e33bcd-ddf6-4d7a-bd76-d7857a10c756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:51.416721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e33bcd-ddf6-4d7a-bd76-d7857a10c756 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:51.417109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e33bcd-ddf6-4d7a-bd76-d7857a10c756 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:51.417481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e33bcd-ddf6-4d7a-bd76-d7857a10c756 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:51.418954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e33bcd-ddf6-4d7a-bd76-d7857a10c756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:51.421596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e33bcd-ddf6-4d7a-bd76-d7857a10c756 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:51.426764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:51.462837 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3926/7794] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:18:51 2022] POST /identity/v3/projects => generated 361 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:18:51.467464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98053af3-c835-4b26-8559-ecbe92d69b9e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:51.483163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98053af3-c835-4b26-8559-ecbe92d69b9e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:51.581688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee103d8c-0ff7-4350-bcda-f4acc32b697f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:51.583181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee103d8c-0ff7-4350-bcda-f4acc32b697f admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:51.583466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee103d8c-0ff7-4350-bcda-f4acc32b697f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:51.583746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee103d8c-0ff7-4350-bcda-f4acc32b697f admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:51.585070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee103d8c-0ff7-4350-bcda-f4acc32b697f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:51.587789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee103d8c-0ff7-4350-bcda-f4acc32b697f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:51.629307 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3869/7795] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:18:51 2022] POST /identity/v3/users => generated 281 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:18:51.631202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e33bcd-ddf6-4d7a-bd76-d7857a10c756 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:51.647688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e33bcd-ddf6-4d7a-bd76-d7857a10c756 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:51.656466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29b9c1-18b9-4eff-922b-43c1f6806732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:51.657330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29b9c1-18b9-4eff-922b-43c1f6806732 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:51.659758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29b9c1-18b9-4eff-922b-43c1f6806732 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:51.660727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29b9c1-18b9-4eff-922b-43c1f6806732 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:51.662476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29b9c1-18b9-4eff-922b-43c1f6806732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:51.665401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29b9c1-18b9-4eff-922b-43c1f6806732 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:51.681156 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3927/7796] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:18:51.684914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee103d8c-0ff7-4350-bcda-f4acc32b697f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:51.698227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee103d8c-0ff7-4350-bcda-f4acc32b697f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:51.720597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e016e-9763-475f-91e5-42a72ef91240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:51.721284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e016e-9763-475f-91e5-42a72ef91240 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:51.721569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e016e-9763-475f-91e5-42a72ef91240 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:51.721845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e016e-9763-475f-91e5-42a72ef91240 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:51.723647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e016e-9763-475f-91e5-42a72ef91240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:51.729548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e016e-9763-475f-91e5-42a72ef91240 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:51.743003 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3870/7797] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:18:51.747565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29b9c1-18b9-4eff-922b-43c1f6806732 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:51.776686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29b9c1-18b9-4eff-922b-43c1f6806732 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:51.791121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f63d7-9a15-4c60-bd1d-4cce3b353da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:51.792095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f63d7-9a15-4c60-bd1d-4cce3b353da8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:51.792544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f63d7-9a15-4c60-bd1d-4cce3b353da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:51.792997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f63d7-9a15-4c60-bd1d-4cce3b353da8 admin admin] PATH_INFO: `/v3/projects/4d0fd9d1068446ac9ba3eba70b042736/users/53ef2c63406148b7ace2ef22b5567919/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:51.823020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f63d7-9a15-4c60-bd1d-4cce3b353da8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d0fd9d1068446ac9ba3eba70b042736, user_id=53ef2c63406148b7ace2ef22b5567919, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:51.826895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f63d7-9a15-4c60-bd1d-4cce3b353da8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:51.849800 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3928/7798] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:18:51 2022] PUT /identity/v3/projects/4d0fd9d1068446ac9ba3eba70b042736/users/53ef2c63406148b7ace2ef22b5567919/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:51.861505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856d63b-4b1f-40d5-97fd-d0c96b4cef4d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:51.861803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856d63b-4b1f-40d5-97fd-d0c96b4cef4d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:51.862189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856d63b-4b1f-40d5-97fd-d0c96b4cef4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:51.935746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f856d63b-4b1f-40d5-97fd-d0c96b4cef4d None None] MFA Rules not processed for user `53ef2c63406148b7ace2ef22b5567919`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:51.938133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f856d63b-4b1f-40d5-97fd-d0c96b4cef4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:52.018358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:52.114996 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3871/7799] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:18:52.151183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f63d7-9a15-4c60-bd1d-4cce3b353da8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:52.154910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e94f63d7-9a15-4c60-bd1d-4cce3b353da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:52.284952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f63d7-9a15-4c60-bd1d-4cce3b353da8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:52.296554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8c2d6-0912-4310-ad23-2ac5384b914e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:52.297271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8c2d6-0912-4310-ad23-2ac5384b914e service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:52.297832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8c2d6-0912-4310-ad23-2ac5384b914e service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:52.298249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8c2d6-0912-4310-ad23-2ac5384b914e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:52.337501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d8c2d6-0912-4310-ad23-2ac5384b914e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:52.339927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d8c2d6-0912-4310-ad23-2ac5384b914e service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:52.355210 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3929/7800] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:18:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:18:52.383803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856d63b-4b1f-40d5-97fd-d0c96b4cef4d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:52.487731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856d63b-4b1f-40d5-97fd-d0c96b4cef4d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:52.513152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f6495-d8cf-41bd-9e55-9394d4889650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:52.513898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f6495-d8cf-41bd-9e55-9394d4889650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:52.514246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f6495-d8cf-41bd-9e55-9394d4889650 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:52.514586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f6495-d8cf-41bd-9e55-9394d4889650 admin admin] PATH_INFO: `/v3/users/53ef2c63406148b7ace2ef22b5567919` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:52.527270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f6495-d8cf-41bd-9e55-9394d4889650 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53ef2c63406148b7ace2ef22b5567919)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:52.536079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f6495-d8cf-41bd-9e55-9394d4889650 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:52.668793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e69f6495-d8cf-41bd-9e55-9394d4889650 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53ef2c63406148b7ace2ef22b5567919'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:52.671305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e69f6495-d8cf-41bd-9e55-9394d4889650 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53ef2c63406148b7ace2ef22b5567919'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:52.674999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e69f6495-d8cf-41bd-9e55-9394d4889650 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53ef2c63406148b7ace2ef22b5567919'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:52.690561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e69f6495-d8cf-41bd-9e55-9394d4889650 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53ef2c63406148b7ace2ef22b5567919'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:52.712445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e69f6495-d8cf-41bd-9e55-9394d4889650 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53ef2c63406148b7ace2ef22b5567919'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:52.747293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e70b8-6614-4745-8870-390ea17a4968 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:52.747466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e70b8-6614-4745-8870-390ea17a4968 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:52.747753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e70b8-6614-4745-8870-390ea17a4968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:52.756899 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3872/7801] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:18:52 2022] DELETE /identity/v3/users/53ef2c63406148b7ace2ef22b5567919 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:52.784677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-178e70b8-6614-4745-8870-390ea17a4968 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:52.787946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-178e70b8-6614-4745-8870-390ea17a4968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:52.926357 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3930/7802] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:52.941042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f6495-d8cf-41bd-9e55-9394d4889650 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:52.948200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e69f6495-d8cf-41bd-9e55-9394d4889650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:53.131124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f6495-d8cf-41bd-9e55-9394d4889650 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:53.139601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6dead9-090b-4d09-98f8-c9ab12cd6c05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:53.140748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dead9-090b-4d09-98f8-c9ab12cd6c05 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:53.141206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dead9-090b-4d09-98f8-c9ab12cd6c05 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:53.141484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dead9-090b-4d09-98f8-c9ab12cd6c05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:53.155429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:53.165230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6dead9-090b-4d09-98f8-c9ab12cd6c05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:53.166728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6dead9-090b-4d09-98f8-c9ab12cd6c05 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:53.187920 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3873/7803] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:18:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:18:54.334100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e70b8-6614-4745-8870-390ea17a4968 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:54.342207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-178e70b8-6614-4745-8870-390ea17a4968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:54.392028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e70b8-6614-4745-8870-390ea17a4968 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:54.407708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbedc7-a15f-41b7-8844-2c6c7df88f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:54.409862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbedc7-a15f-41b7-8844-2c6c7df88f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:54.411564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbedc7-a15f-41b7-8844-2c6c7df88f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:54.412410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbedc7-a15f-41b7-8844-2c6c7df88f0b admin admin] PATH_INFO: `/v3/projects/4d0fd9d1068446ac9ba3eba70b042736` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:54.417805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dbedc7-a15f-41b7-8844-2c6c7df88f0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d0fd9d1068446ac9ba3eba70b042736)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:54.426071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dbedc7-a15f-41b7-8844-2c6c7df88f0b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:54.538195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:54.540737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c0dbedc7-a15f-41b7-8844-2c6c7df88f0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d0fd9d1068446ac9ba3eba70b042736', 'request_id': 'req-c0dbedc7-a15f-41b7-8844-2c6c7df88f0b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:54.556291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c0dbedc7-a15f-41b7-8844-2c6c7df88f0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d0fd9d1068446ac9ba3eba70b042736', 'request_id': 'req-c0dbedc7-a15f-41b7-8844-2c6c7df88f0b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:54.569446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c0dbedc7-a15f-41b7-8844-2c6c7df88f0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d0fd9d1068446ac9ba3eba70b042736', 'request_id': 'req-c0dbedc7-a15f-41b7-8844-2c6c7df88f0b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:54.592419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c0dbedc7-a15f-41b7-8844-2c6c7df88f0b admin admin] The token cache is being invalidate because project 4d0fd9d1068446ac9ba3eba70b042736 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:18:54.593009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c0dbedc7-a15f-41b7-8844-2c6c7df88f0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:54.593841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c0dbedc7-a15f-41b7-8844-2c6c7df88f0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:18:54.595721 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3931/7804] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:18:54 2022] DELETE /identity/v3/projects/4d0fd9d1068446ac9ba3eba70b042736 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:54.600828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd6be1-bbcd-47ee-856b-95f9401206fc None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:54.601664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd6be1-bbcd-47ee-856b-95f9401206fc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:54.602125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd6be1-bbcd-47ee-856b-95f9401206fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:54.647891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7afd6be1-bbcd-47ee-856b-95f9401206fc None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:54.649841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7afd6be1-bbcd-47ee-856b-95f9401206fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:54.791519 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3874/7805] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:54.799712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd779078-0d7e-47a2-9066-c1a40a089653 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:54.800261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd779078-0d7e-47a2-9066-c1a40a089653 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:54.800793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd779078-0d7e-47a2-9066-c1a40a089653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:54.841740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-dd779078-0d7e-47a2-9066-c1a40a089653 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:54.846604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-dd779078-0d7e-47a2-9066-c1a40a089653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:54.868427 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3932/7806] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:54.873286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd6be1-bbcd-47ee-856b-95f9401206fc None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:54.886185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd6be1-bbcd-47ee-856b-95f9401206fc None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:54.911453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c4775-05fd-462f-be70-57935f56d82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:54.912334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c4775-05fd-462f-be70-57935f56d82d admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:54.912723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c4775-05fd-462f-be70-57935f56d82d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:54.913068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c4775-05fd-462f-be70-57935f56d82d admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:54.913799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c4775-05fd-462f-be70-57935f56d82d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:18:54.914180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c4775-05fd-462f-be70-57935f56d82d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:54.915975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c4775-05fd-462f-be70-57935f56d82d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:54.958451 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3875/7807] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:18:54.960629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd779078-0d7e-47a2-9066-c1a40a089653 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:54.978197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd779078-0d7e-47a2-9066-c1a40a089653 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:54.993458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e4d9c-235c-4bac-866e-1aed86d57ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:54.994182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e4d9c-235c-4bac-866e-1aed86d57ded admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:54.994555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e4d9c-235c-4bac-866e-1aed86d57ded admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:54.994905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e4d9c-235c-4bac-866e-1aed86d57ded admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:54.995623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e4d9c-235c-4bac-866e-1aed86d57ded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:54.998213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e4d9c-235c-4bac-866e-1aed86d57ded admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:55.079263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c4775-05fd-462f-be70-57935f56d82d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:55.093870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c4775-05fd-462f-be70-57935f56d82d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:55.109771 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3933/7808] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:18:54 2022] POST /identity/v3/projects => generated 371 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:18:55.240724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821413e-9e44-4cd2-a40a-29819d4a0470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:55.241394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821413e-9e44-4cd2-a40a-29819d4a0470 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:55.241668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821413e-9e44-4cd2-a40a-29819d4a0470 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:55.241945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821413e-9e44-4cd2-a40a-29819d4a0470 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:55.246621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821413e-9e44-4cd2-a40a-29819d4a0470 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:55.249960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821413e-9e44-4cd2-a40a-29819d4a0470 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:55.291968 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3876/7809] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:18:55 2022] POST /identity/v3/users => generated 286 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:18:55.297462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e4d9c-235c-4bac-866e-1aed86d57ded admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:55.312845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e4d9c-235c-4bac-866e-1aed86d57ded admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:55.321403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec630f5-5d22-4676-bbfb-2a5132ae5683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:55.322039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec630f5-5d22-4676-bbfb-2a5132ae5683 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:55.322317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec630f5-5d22-4676-bbfb-2a5132ae5683 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:55.322614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec630f5-5d22-4676-bbfb-2a5132ae5683 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:55.323261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec630f5-5d22-4676-bbfb-2a5132ae5683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:55.325420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec630f5-5d22-4676-bbfb-2a5132ae5683 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:55.364629 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3934/7810] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:18:55.365883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821413e-9e44-4cd2-a40a-29819d4a0470 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:55.391390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821413e-9e44-4cd2-a40a-29819d4a0470 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:55.405640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd22e69-44fa-47eb-b537-af79735fcbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:55.406579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd22e69-44fa-47eb-b537-af79735fcbfa admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:55.407016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd22e69-44fa-47eb-b537-af79735fcbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:55.407445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd22e69-44fa-47eb-b537-af79735fcbfa admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:55.408268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd22e69-44fa-47eb-b537-af79735fcbfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:55.410066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd22e69-44fa-47eb-b537-af79735fcbfa admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:55.431642 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3877/7811] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:18:55.432960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec630f5-5d22-4676-bbfb-2a5132ae5683 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:55.458975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec630f5-5d22-4676-bbfb-2a5132ae5683 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:55.481183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55487f22-dbfa-4d13-92ec-ce28da6b4adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:55.483152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55487f22-dbfa-4d13-92ec-ce28da6b4adf admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:55.485243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55487f22-dbfa-4d13-92ec-ce28da6b4adf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:55.487482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55487f22-dbfa-4d13-92ec-ce28da6b4adf admin admin] PATH_INFO: `/v3/projects/1e8b95027abe453085cf757cc0b64874/users/31d66b5c448f4914a1bb2a58715b699d/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:55.515171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55487f22-dbfa-4d13-92ec-ce28da6b4adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8b95027abe453085cf757cc0b64874, user_id=31d66b5c448f4914a1bb2a58715b699d, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:55.518725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55487f22-dbfa-4d13-92ec-ce28da6b4adf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:55.543702 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3935/7812] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:18:55 2022] PUT /identity/v3/projects/1e8b95027abe453085cf757cc0b64874/users/31d66b5c448f4914a1bb2a58715b699d/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:55.554979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfb82e-5412-403a-8955-70e80b316b42 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:55.555351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfb82e-5412-403a-8955-70e80b316b42 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:55.555528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfb82e-5412-403a-8955-70e80b316b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:55.622012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-30dfb82e-5412-403a-8955-70e80b316b42 None None] MFA Rules not processed for user `31d66b5c448f4914a1bb2a58715b699d`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:55.625162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-30dfb82e-5412-403a-8955-70e80b316b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:55.747529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:55.788598 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3878/7813] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:55.793579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55487f22-dbfa-4d13-92ec-ce28da6b4adf admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:55.909437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55487f22-dbfa-4d13-92ec-ce28da6b4adf admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:55.920446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19989b3f-6144-47ee-ac70-3891bdc7dbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:55.921684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19989b3f-6144-47ee-ac70-3891bdc7dbe0 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:55.922270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19989b3f-6144-47ee-ac70-3891bdc7dbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:55.923156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19989b3f-6144-47ee-ac70-3891bdc7dbe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:55.924225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19989b3f-6144-47ee-ac70-3891bdc7dbe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:55.926425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19989b3f-6144-47ee-ac70-3891bdc7dbe0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:56.007815 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3936/7814] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:18:55 2022] POST /identity/v3/projects => generated 373 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:18:56.012390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfb82e-5412-403a-8955-70e80b316b42 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:56.050029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfb82e-5412-403a-8955-70e80b316b42 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:56.166221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991543ba-0bde-44be-bf64-5b997e5ba016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:56.166887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991543ba-0bde-44be-bf64-5b997e5ba016 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:56.167168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991543ba-0bde-44be-bf64-5b997e5ba016 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:56.167440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991543ba-0bde-44be-bf64-5b997e5ba016 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:56.173071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991543ba-0bde-44be-bf64-5b997e5ba016 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:56.180549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991543ba-0bde-44be-bf64-5b997e5ba016 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:56.255537 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3879/7815] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:18:56 2022] POST /identity/v3/users => generated 293 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:18:56.259831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19989b3f-6144-47ee-ac70-3891bdc7dbe0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:56.279531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19989b3f-6144-47ee-ac70-3891bdc7dbe0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:56.289326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0154f1-6a4a-4734-82c8-c29b9c08014b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:56.291677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0154f1-6a4a-4734-82c8-c29b9c08014b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:56.292075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0154f1-6a4a-4734-82c8-c29b9c08014b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:56.292362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0154f1-6a4a-4734-82c8-c29b9c08014b admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:56.295638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0154f1-6a4a-4734-82c8-c29b9c08014b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:56.299471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0154f1-6a4a-4734-82c8-c29b9c08014b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:56.335262 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3937/7816] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:18:56.336590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991543ba-0bde-44be-bf64-5b997e5ba016 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:56.353061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991543ba-0bde-44be-bf64-5b997e5ba016 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:56.373072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff4db3-0376-486e-99ea-7275f6e8a134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:56.373718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff4db3-0376-486e-99ea-7275f6e8a134 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:56.374086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff4db3-0376-486e-99ea-7275f6e8a134 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:56.374372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff4db3-0376-486e-99ea-7275f6e8a134 admin admin] PATH_INFO: `/v3/projects/a9ac3f8831e443d8b79cd027e3dcc357/users/be34493b7af94a49a76835c2afde6f01/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:56.431050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff4db3-0376-486e-99ea-7275f6e8a134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9ac3f8831e443d8b79cd027e3dcc357, user_id=be34493b7af94a49a76835c2afde6f01, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:56.435734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff4db3-0376-486e-99ea-7275f6e8a134 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:56.475180 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3880/7817] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:18:56 2022] PUT /identity/v3/projects/a9ac3f8831e443d8b79cd027e3dcc357/users/be34493b7af94a49a76835c2afde6f01/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:56.478931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0154f1-6a4a-4734-82c8-c29b9c08014b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:56.522034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0154f1-6a4a-4734-82c8-c29b9c08014b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:56.659353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:56.686691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878625c7-2d77-4b40-bfdd-ad85e66d32a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:56.687310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878625c7-2d77-4b40-bfdd-ad85e66d32a4 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:56.687900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878625c7-2d77-4b40-bfdd-ad85e66d32a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:56.688019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878625c7-2d77-4b40-bfdd-ad85e66d32a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:56.688629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878625c7-2d77-4b40-bfdd-ad85e66d32a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:56.691422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878625c7-2d77-4b40-bfdd-ad85e66d32a4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:56.711771 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3938/7818] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:18:56.713026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff4db3-0376-486e-99ea-7275f6e8a134 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:56.726063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff4db3-0376-486e-99ea-7275f6e8a134 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:56.740490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc8a5d-d7be-42af-a05d-b7582696f874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:56.741186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc8a5d-d7be-42af-a05d-b7582696f874 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:56.741835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc8a5d-d7be-42af-a05d-b7582696f874 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:56.742125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc8a5d-d7be-42af-a05d-b7582696f874 admin admin] PATH_INFO: `/v3/projects/a9ac3f8831e443d8b79cd027e3dcc357/users/be34493b7af94a49a76835c2afde6f01/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:56.746375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc8a5d-d7be-42af-a05d-b7582696f874 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9ac3f8831e443d8b79cd027e3dcc357, user_id=be34493b7af94a49a76835c2afde6f01, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:56.751387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc8a5d-d7be-42af-a05d-b7582696f874 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:56.781348 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3881/7819] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:18:56 2022] PUT /identity/v3/projects/a9ac3f8831e443d8b79cd027e3dcc357/users/be34493b7af94a49a76835c2afde6f01/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:18:56.794354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc610112-66d7-400a-93f3-4c8962f2f468 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:56.794719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc610112-66d7-400a-93f3-4c8962f2f468 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:56.795103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc610112-66d7-400a-93f3-4c8962f2f468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:56.874201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-cc610112-66d7-400a-93f3-4c8962f2f468 None None] MFA Rules not processed for user `be34493b7af94a49a76835c2afde6f01`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:18:56.875833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cc610112-66d7-400a-93f3-4c8962f2f468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:57.104993 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3939/7820] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:18:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:18:57.119372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc8a5d-d7be-42af-a05d-b7582696f874 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:18:57.120741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-73cc8a5d-d7be-42af-a05d-b7582696f874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:18:57.279613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc8a5d-d7be-42af-a05d-b7582696f874 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:18:57.295025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31f0c2-dbb3-49ab-9db0-cc4a32fab777 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:18:57.296604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31f0c2-dbb3-49ab-9db0-cc4a32fab777 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:18:57.296898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31f0c2-dbb3-49ab-9db0-cc4a32fab777 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:18:57.297424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31f0c2-dbb3-49ab-9db0-cc4a32fab777 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:18:57.403263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:57.406879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31f0c2-dbb3-49ab-9db0-cc4a32fab777 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:18:57.408516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31f0c2-dbb3-49ab-9db0-cc4a32fab777 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:18:57.443950 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3882/7821] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:18:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:19:00.701395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc610112-66d7-400a-93f3-4c8962f2f468 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:00.703317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cc610112-66d7-400a-93f3-4c8962f2f468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:00.825762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31f0c2-dbb3-49ab-9db0-cc4a32fab777 service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:00.826844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc610112-66d7-400a-93f3-4c8962f2f468 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:00.834434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270fe35-a952-459b-832d-2fa21f32721e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:00.835085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270fe35-a952-459b-832d-2fa21f32721e service glance] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:00.835365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270fe35-a952-459b-832d-2fa21f32721e service glance] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:00.835636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270fe35-a952-459b-832d-2fa21f32721e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:00.842428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1270fe35-a952-459b-832d-2fa21f32721e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:00.906896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31f0c2-dbb3-49ab-9db0-cc4a32fab777 service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:00.914303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d4ee9-2fe9-4960-a8f5-a0b1fce2dfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:00.915392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d4ee9-2fe9-4960-a8f5-a0b1fce2dfb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:00.915898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d4ee9-2fe9-4960-a8f5-a0b1fce2dfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:00.916385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d4ee9-2fe9-4960-a8f5-a0b1fce2dfb8 admin admin] PATH_INFO: `/v3/users/31d66b5c448f4914a1bb2a58715b699d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:00.918651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d4ee9-2fe9-4960-a8f5-a0b1fce2dfb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31d66b5c448f4914a1bb2a58715b699d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:00.921840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d4ee9-2fe9-4960-a8f5-a0b1fce2dfb8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:01.014534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fe8d4ee9-2fe9-4960-a8f5-a0b1fce2dfb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31d66b5c448f4914a1bb2a58715b699d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:01.015372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fe8d4ee9-2fe9-4960-a8f5-a0b1fce2dfb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31d66b5c448f4914a1bb2a58715b699d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:01.016151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fe8d4ee9-2fe9-4960-a8f5-a0b1fce2dfb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31d66b5c448f4914a1bb2a58715b699d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:01.036313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fe8d4ee9-2fe9-4960-a8f5-a0b1fce2dfb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31d66b5c448f4914a1bb2a58715b699d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:01.068366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fe8d4ee9-2fe9-4960-a8f5-a0b1fce2dfb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31d66b5c448f4914a1bb2a58715b699d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:01.108308 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3883/7822] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:19:00 2022] DELETE /identity/v3/users/31d66b5c448f4914a1bb2a58715b699d => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:19:01.109958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944e19d-9c00-4ee6-b326-c346d1506afe None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:01.111834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944e19d-9c00-4ee6-b326-c346d1506afe None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:01.115997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944e19d-9c00-4ee6-b326-c346d1506afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:01.131724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270fe35-a952-459b-832d-2fa21f32721e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:01.139573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270fe35-a952-459b-832d-2fa21f32721e service glance] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:01.162094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9944e19d-9c00-4ee6-b326-c346d1506afe None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:19:01.163808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9944e19d-9c00-4ee6-b326-c346d1506afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:01.168894 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3940/7823] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:19:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:19:01.296771 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3884/7824] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:19:01.329885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270fe35-a952-459b-832d-2fa21f32721e service glance] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:01.331566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1270fe35-a952-459b-832d-2fa21f32721e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:01.350180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:19:01.463171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270fe35-a952-459b-832d-2fa21f32721e service glance] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:01.602893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b791b-15d8-4edd-a5d3-728041f750b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:01.602893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b791b-15d8-4edd-a5d3-728041f750b5 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:01.602893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b791b-15d8-4edd-a5d3-728041f750b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:01.602893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b791b-15d8-4edd-a5d3-728041f750b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:01.632993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b791b-15d8-4edd-a5d3-728041f750b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:01.634644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b791b-15d8-4edd-a5d3-728041f750b5 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:01.649273 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3941/7825] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:19:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:19:02.340391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944e19d-9c00-4ee6-b326-c346d1506afe None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:02.342070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9944e19d-9c00-4ee6-b326-c346d1506afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:02.362569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944e19d-9c00-4ee6-b326-c346d1506afe None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:02.381301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3abf79-c9d3-4353-861b-9f5aad241ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:02.381850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3abf79-c9d3-4353-861b-9f5aad241ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:02.382136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3abf79-c9d3-4353-861b-9f5aad241ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:02.382598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3abf79-c9d3-4353-861b-9f5aad241ca8 admin admin] PATH_INFO: `/v3/projects/1e8b95027abe453085cf757cc0b64874` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:02.383781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3abf79-c9d3-4353-861b-9f5aad241ca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e8b95027abe453085cf757cc0b64874)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:02.386707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3abf79-c9d3-4353-861b-9f5aad241ca8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:02.414071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:19:02.485189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ab3abf79-c9d3-4353-861b-9f5aad241ca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e8b95027abe453085cf757cc0b64874', 'request_id': 'req-ab3abf79-c9d3-4353-861b-9f5aad241ca8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:02.499521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ab3abf79-c9d3-4353-861b-9f5aad241ca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e8b95027abe453085cf757cc0b64874', 'request_id': 'req-ab3abf79-c9d3-4353-861b-9f5aad241ca8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:02.507034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ab3abf79-c9d3-4353-861b-9f5aad241ca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e8b95027abe453085cf757cc0b64874', 'request_id': 'req-ab3abf79-c9d3-4353-861b-9f5aad241ca8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:02.531894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ab3abf79-c9d3-4353-861b-9f5aad241ca8 admin admin] The token cache is being invalidate because project 1e8b95027abe453085cf757cc0b64874 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:19:02.532202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ab3abf79-c9d3-4353-861b-9f5aad241ca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:02.535755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ab3abf79-c9d3-4353-861b-9f5aad241ca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:02.538586 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3885/7826] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:19:02 2022] DELETE /identity/v3/projects/1e8b95027abe453085cf757cc0b64874 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:19:02.542348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b791b-15d8-4edd-a5d3-728041f750b5 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:02.555344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b791b-15d8-4edd-a5d3-728041f750b5 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:02.667600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552a063-29f2-422c-afbd-a200710042fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:02.668662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552a063-29f2-422c-afbd-a200710042fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:02.669114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552a063-29f2-422c-afbd-a200710042fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:02.669486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552a063-29f2-422c-afbd-a200710042fe admin admin] PATH_INFO: `/v3/users/be34493b7af94a49a76835c2afde6f01` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:02.672183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552a063-29f2-422c-afbd-a200710042fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=be34493b7af94a49a76835c2afde6f01)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:02.675011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552a063-29f2-422c-afbd-a200710042fe admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:02.765753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c552a063-29f2-422c-afbd-a200710042fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be34493b7af94a49a76835c2afde6f01'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:02.778362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c552a063-29f2-422c-afbd-a200710042fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be34493b7af94a49a76835c2afde6f01'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:02.828373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:19:02.834455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c552a063-29f2-422c-afbd-a200710042fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be34493b7af94a49a76835c2afde6f01'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:02.836643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c552a063-29f2-422c-afbd-a200710042fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be34493b7af94a49a76835c2afde6f01'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:02.851903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-c552a063-29f2-422c-afbd-a200710042fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be34493b7af94a49a76835c2afde6f01'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:02.865000 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3942/7827] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:19:02 2022] DELETE /identity/v3/users/be34493b7af94a49a76835c2afde6f01 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:19:03.653327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3abf79-c9d3-4353-861b-9f5aad241ca8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:03.654805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ab3abf79-c9d3-4353-861b-9f5aad241ca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:03.669123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3abf79-c9d3-4353-861b-9f5aad241ca8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:03.762117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d7864f-de06-4c93-b4df-8601cd9647cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:03.762815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7864f-de06-4c93-b4df-8601cd9647cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:03.763096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7864f-de06-4c93-b4df-8601cd9647cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:03.763511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7864f-de06-4c93-b4df-8601cd9647cd admin admin] PATH_INFO: `/v3/projects/a9ac3f8831e443d8b79cd027e3dcc357` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:03.765339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d7864f-de06-4c93-b4df-8601cd9647cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9ac3f8831e443d8b79cd027e3dcc357)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:03.767778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d7864f-de06-4c93-b4df-8601cd9647cd admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:03.845820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-13d7864f-de06-4c93-b4df-8601cd9647cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9ac3f8831e443d8b79cd027e3dcc357', 'request_id': 'req-13d7864f-de06-4c93-b4df-8601cd9647cd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:03.860022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-13d7864f-de06-4c93-b4df-8601cd9647cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9ac3f8831e443d8b79cd027e3dcc357', 'request_id': 'req-13d7864f-de06-4c93-b4df-8601cd9647cd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:03.872955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-13d7864f-de06-4c93-b4df-8601cd9647cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9ac3f8831e443d8b79cd027e3dcc357', 'request_id': 'req-13d7864f-de06-4c93-b4df-8601cd9647cd'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:03.890238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-13d7864f-de06-4c93-b4df-8601cd9647cd admin admin] The token cache is being invalidate because project a9ac3f8831e443d8b79cd027e3dcc357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:19:03.890560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-13d7864f-de06-4c93-b4df-8601cd9647cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:03.891842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-13d7864f-de06-4c93-b4df-8601cd9647cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:03.900484 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3886/7828] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:19:03 2022] DELETE /identity/v3/projects/a9ac3f8831e443d8b79cd027e3dcc357 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:19:03.904611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6d14c-e3e5-4541-9850-c2d543ddcab7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:03.905453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6d14c-e3e5-4541-9850-c2d543ddcab7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:03.905952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6d14c-e3e5-4541-9850-c2d543ddcab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:03.936217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3ef6d14c-e3e5-4541-9850-c2d543ddcab7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:19:03.937509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3ef6d14c-e3e5-4541-9850-c2d543ddcab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:04.030216 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3943/7829] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:19:04.038233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a5d79-856c-4ee2-8d09-7094d2ec0bda None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:04.038546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a5d79-856c-4ee2-8d09-7094d2ec0bda None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:04.038821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a5d79-856c-4ee2-8d09-7094d2ec0bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:04.071407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1d9a5d79-856c-4ee2-8d09-7094d2ec0bda None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:19:04.072733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1d9a5d79-856c-4ee2-8d09-7094d2ec0bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:04.088386 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3887/7830] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:19:04.092887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6d14c-e3e5-4541-9850-c2d543ddcab7 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:04.108761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6d14c-e3e5-4541-9850-c2d543ddcab7 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:04.115365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1f96c-51aa-47e7-aff1-fb174223b36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:04.116013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1f96c-51aa-47e7-aff1-fb174223b36e admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:04.116508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1f96c-51aa-47e7-aff1-fb174223b36e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:04.117027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1f96c-51aa-47e7-aff1-fb174223b36e admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:04.117714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1f96c-51aa-47e7-aff1-fb174223b36e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:19:04.118194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1f96c-51aa-47e7-aff1-fb174223b36e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:04.119863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1f96c-51aa-47e7-aff1-fb174223b36e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:04.144661 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3944/7831] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:19:04.149026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a5d79-856c-4ee2-8d09-7094d2ec0bda None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:04.163286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a5d79-856c-4ee2-8d09-7094d2ec0bda None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:04.168655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d134582f-3529-41af-8cf6-4fcde00f445a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:04.169460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134582f-3529-41af-8cf6-4fcde00f445a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:04.169750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134582f-3529-41af-8cf6-4fcde00f445a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:04.170027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134582f-3529-41af-8cf6-4fcde00f445a admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:04.170950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d134582f-3529-41af-8cf6-4fcde00f445a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:04.172887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d134582f-3529-41af-8cf6-4fcde00f445a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:04.203720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:19:04.227661 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3888/7832] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:19:04 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:19:04.231625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1f96c-51aa-47e7-aff1-fb174223b36e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:04.243076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1f96c-51aa-47e7-aff1-fb174223b36e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:04.315931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02c066-c08a-45c2-9f3c-001f14d5ba60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:04.316881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02c066-c08a-45c2-9f3c-001f14d5ba60 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:04.317146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02c066-c08a-45c2-9f3c-001f14d5ba60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:04.317485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02c066-c08a-45c2-9f3c-001f14d5ba60 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:04.318181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b02c066-c08a-45c2-9f3c-001f14d5ba60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:04.320543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b02c066-c08a-45c2-9f3c-001f14d5ba60 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:04.347016 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3945/7833] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:19:04 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:19:04.350948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d134582f-3529-41af-8cf6-4fcde00f445a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:04.363116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d134582f-3529-41af-8cf6-4fcde00f445a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:04.369902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217b1b5-4864-4ee3-9f00-b36362fad56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:04.370714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217b1b5-4864-4ee3-9f00-b36362fad56a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:04.371667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217b1b5-4864-4ee3-9f00-b36362fad56a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:04.371936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217b1b5-4864-4ee3-9f00-b36362fad56a admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:04.372713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217b1b5-4864-4ee3-9f00-b36362fad56a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:04.374591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217b1b5-4864-4ee3-9f00-b36362fad56a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:04.388474 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3889/7834] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:19:04.389724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02c066-c08a-45c2-9f3c-001f14d5ba60 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:04.400253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02c066-c08a-45c2-9f3c-001f14d5ba60 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:04.406102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307fdba-1f3a-4dc2-8f34-5e02611a4a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:04.408133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307fdba-1f3a-4dc2-8f34-5e02611a4a48 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:04.408133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307fdba-1f3a-4dc2-8f34-5e02611a4a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:04.408133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307fdba-1f3a-4dc2-8f34-5e02611a4a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:04.408812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307fdba-1f3a-4dc2-8f34-5e02611a4a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:04.410879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307fdba-1f3a-4dc2-8f34-5e02611a4a48 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:04.426906 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3946/7835] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:19:04.428002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217b1b5-4864-4ee3-9f00-b36362fad56a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:04.440015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217b1b5-4864-4ee3-9f00-b36362fad56a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:04.447937 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efef2c1-876a-4967-8a3b-6d623484dff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:04.448739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efef2c1-876a-4967-8a3b-6d623484dff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:04.449135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efef2c1-876a-4967-8a3b-6d623484dff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:04.449491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efef2c1-876a-4967-8a3b-6d623484dff0 admin admin] PATH_INFO: `/v3/projects/5f12200ca5c649cd993499b7cb7e382f/users/ecdf7bba776f4b9c94f2653679505303/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:04.470413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efef2c1-876a-4967-8a3b-6d623484dff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f12200ca5c649cd993499b7cb7e382f, user_id=ecdf7bba776f4b9c94f2653679505303, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:04.473220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efef2c1-876a-4967-8a3b-6d623484dff0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:04.492521 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3890/7836] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:19:04 2022] PUT /identity/v3/projects/5f12200ca5c649cd993499b7cb7e382f/users/ecdf7bba776f4b9c94f2653679505303/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:19:04.498197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1dc37-9dfa-47c0-b2e8-6d08eaba93f3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:04.498599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1dc37-9dfa-47c0-b2e8-6d08eaba93f3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:04.498970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1dc37-9dfa-47c0-b2e8-6d08eaba93f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:04.523921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fce1dc37-9dfa-47c0-b2e8-6d08eaba93f3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:19:04.526086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fce1dc37-9dfa-47c0-b2e8-6d08eaba93f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:04.607818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:19:04.626669 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3947/7837] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:19:04.653042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efef2c1-876a-4967-8a3b-6d623484dff0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:04.655940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3efef2c1-876a-4967-8a3b-6d623484dff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:04.751355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efef2c1-876a-4967-8a3b-6d623484dff0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:04.762681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801101b-3930-4ee2-955c-a34512906dbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:04.763081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801101b-3930-4ee2-955c-a34512906dbd service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:04.763276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801101b-3930-4ee2-955c-a34512906dbd service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:04.763552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801101b-3930-4ee2-955c-a34512906dbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:04.778454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801101b-3930-4ee2-955c-a34512906dbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:04.780078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801101b-3930-4ee2-955c-a34512906dbd service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:04.793501 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3891/7838] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:19:05.524537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce19bcf-fd89-4e80-abe7-d951fd2f4d58 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:05.525037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce19bcf-fd89-4e80-abe7-d951fd2f4d58 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:05.525227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce19bcf-fd89-4e80-abe7-d951fd2f4d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:05.547461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3ce19bcf-fd89-4e80-abe7-d951fd2f4d58 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:19:05.549805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3ce19bcf-fd89-4e80-abe7-d951fd2f4d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:05.607175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082eb19d-aed6-4229-9892-12a5ed25e7d0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:05.607683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082eb19d-aed6-4229-9892-12a5ed25e7d0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:05.608610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082eb19d-aed6-4229-9892-12a5ed25e7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:05.617852 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3948/7839] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:19:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:19:05.639075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce19bcf-fd89-4e80-abe7-d951fd2f4d58 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:05.640668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3ce19bcf-fd89-4e80-abe7-d951fd2f4d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:05.662994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-082eb19d-aed6-4229-9892-12a5ed25e7d0 None None] MFA Rules not processed for user `ecdf7bba776f4b9c94f2653679505303`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:19:05.664364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-082eb19d-aed6-4229-9892-12a5ed25e7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:05.751882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce19bcf-fd89-4e80-abe7-d951fd2f4d58 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:05.754851 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3892/7840] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:19:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:19:05.762231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74604938-dba2-446e-9a73-c956e9437d0d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:05.762900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74604938-dba2-446e-9a73-c956e9437d0d service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:05.763235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74604938-dba2-446e-9a73-c956e9437d0d service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:05.763515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74604938-dba2-446e-9a73-c956e9437d0d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:05.767345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082eb19d-aed6-4229-9892-12a5ed25e7d0 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:05.769492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-082eb19d-aed6-4229-9892-12a5ed25e7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:05.778302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74604938-dba2-446e-9a73-c956e9437d0d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:05.780093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74604938-dba2-446e-9a73-c956e9437d0d service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:05.799723 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3949/7841] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:19:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:19:05.896919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:19:05.905545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082eb19d-aed6-4229-9892-12a5ed25e7d0 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:05.912426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306a1bf-5a97-4822-8f8e-82a8a729be4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:05.913137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306a1bf-5a97-4822-8f8e-82a8a729be4a service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:05.913606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306a1bf-5a97-4822-8f8e-82a8a729be4a service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:05.913948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306a1bf-5a97-4822-8f8e-82a8a729be4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:05.928985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3306a1bf-5a97-4822-8f8e-82a8a729be4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:05.931552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3306a1bf-5a97-4822-8f8e-82a8a729be4a service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:05.965141 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3893/7842] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:19:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:19:07.557797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a8a90-10a0-4b33-8d5e-56982aaab67e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:07.559387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a8a90-10a0-4b33-8d5e-56982aaab67e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:07.559884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a8a90-10a0-4b33-8d5e-56982aaab67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:07.592314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-805a8a90-10a0-4b33-8d5e-56982aaab67e None None] MFA Rules not processed for user `ecdf7bba776f4b9c94f2653679505303`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:19:07.594327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-805a8a90-10a0-4b33-8d5e-56982aaab67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:07.603646 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3950/7843] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:19:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:19:11.000451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306a1bf-5a97-4822-8f8e-82a8a729be4a service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:11.018396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306a1bf-5a97-4822-8f8e-82a8a729be4a service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:11.030907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6573b-c5cf-4177-84c0-40f1540b6699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:11.033894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6573b-c5cf-4177-84c0-40f1540b6699 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:11.034435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6573b-c5cf-4177-84c0-40f1540b6699 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:11.036923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6573b-c5cf-4177-84c0-40f1540b6699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:11.040405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f5e6573b-c5cf-4177-84c0-40f1540b6699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:11.224857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e6573b-c5cf-4177-84c0-40f1540b6699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:11.226518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e6573b-c5cf-4177-84c0-40f1540b6699 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:11.278724 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3894/7844] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:19:10 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:19:15.595235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f789991-aba2-4a45-8c2e-ce7838d0be84 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:15.596291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f789991-aba2-4a45-8c2e-ce7838d0be84 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:15.596793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f789991-aba2-4a45-8c2e-ce7838d0be84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:15.632256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4f789991-aba2-4a45-8c2e-ce7838d0be84 None None] MFA Rules not processed for user `f77486ef55a0469a99c3795b626bee1a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:19:15.635256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4f789991-aba2-4a45-8c2e-ce7838d0be84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:15.656908 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3951/7845] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:19:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:19:21.257166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=32 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:19:23.018673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6573b-c5cf-4177-84c0-40f1540b6699 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:23.020457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f5e6573b-c5cf-4177-84c0-40f1540b6699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:23.042116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6573b-c5cf-4177-84c0-40f1540b6699 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:23.063275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4c364-9cbf-40e3-8dc2-f1db12223ff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:23.064063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4c364-9cbf-40e3-8dc2-f1db12223ff9 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:23.064390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4c364-9cbf-40e3-8dc2-f1db12223ff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:23.064890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4c364-9cbf-40e3-8dc2-f1db12223ff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:23.068056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f1e4c364-9cbf-40e3-8dc2-f1db12223ff9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:23.097282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e4c364-9cbf-40e3-8dc2-f1db12223ff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:23.102535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e4c364-9cbf-40e3-8dc2-f1db12223ff9 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:23.120543 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3895/7846] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:19:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:19:25.616891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=66 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:19:28.799239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f789991-aba2-4a45-8c2e-ce7838d0be84 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:28.801429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4f789991-aba2-4a45-8c2e-ce7838d0be84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:28.965589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f789991-aba2-4a45-8c2e-ce7838d0be84 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:28.981442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e24ab9-3113-4f6a-b253-ae57d871e68c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:28.981442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e24ab9-3113-4f6a-b253-ae57d871e68c service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:28.981442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e24ab9-3113-4f6a-b253-ae57d871e68c service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:28.981442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e24ab9-3113-4f6a-b253-ae57d871e68c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:28.983544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-28e24ab9-3113-4f6a-b253-ae57d871e68c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:29.020332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e24ab9-3113-4f6a-b253-ae57d871e68c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:29.022389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e24ab9-3113-4f6a-b253-ae57d871e68c service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:29.047704 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3952/7847] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:19:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:19:29.712623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4c364-9cbf-40e3-8dc2-f1db12223ff9 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:29.724751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4c364-9cbf-40e3-8dc2-f1db12223ff9 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:29.728996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbb024-ede1-4f8b-aa95-6f88f8b13030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:29.730407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbb024-ede1-4f8b-aa95-6f88f8b13030 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:29.730805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbb024-ede1-4f8b-aa95-6f88f8b13030 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:29.731171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbb024-ede1-4f8b-aa95-6f88f8b13030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:29.733180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c4dbb024-ede1-4f8b-aa95-6f88f8b13030 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:29.752626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbb024-ede1-4f8b-aa95-6f88f8b13030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:29.754151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbb024-ede1-4f8b-aa95-6f88f8b13030 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:29.768957 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3896/7848] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:19:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:19:30.548300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f32bd-df41-42d0-856f-db4f544ac7c3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:30.548824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f32bd-df41-42d0-856f-db4f544ac7c3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:30.548922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f32bd-df41-42d0-856f-db4f544ac7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:30.600847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d30f32bd-df41-42d0-856f-db4f544ac7c3 None None] MFA Rules not processed for user `ecdf7bba776f4b9c94f2653679505303`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:19:30.605788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d30f32bd-df41-42d0-856f-db4f544ac7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:30.635630 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3953/7849] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:19:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:19:30.725510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbb024-ede1-4f8b-aa95-6f88f8b13030 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:30.765434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbb024-ede1-4f8b-aa95-6f88f8b13030 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:30.784085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f514cb-00d7-431a-8587-d4155f862ac8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:30.786030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f514cb-00d7-431a-8587-d4155f862ac8 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:30.786247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f514cb-00d7-431a-8587-d4155f862ac8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:30.786542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f514cb-00d7-431a-8587-d4155f862ac8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:30.788589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-62f514cb-00d7-431a-8587-d4155f862ac8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:30.901630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f514cb-00d7-431a-8587-d4155f862ac8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:30.907840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f514cb-00d7-431a-8587-d4155f862ac8 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:30.921820 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3897/7850] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:19:30 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:19:31.172735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f32bd-df41-42d0-856f-db4f544ac7c3 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:31.195651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f32bd-df41-42d0-856f-db4f544ac7c3 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:31.201164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa572ce-7392-4091-a595-ae452c24b7e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:31.202211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa572ce-7392-4091-a595-ae452c24b7e3 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:31.203343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa572ce-7392-4091-a595-ae452c24b7e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:31.203799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa572ce-7392-4091-a595-ae452c24b7e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:31.206062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9fa572ce-7392-4091-a595-ae452c24b7e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:31.220823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa572ce-7392-4091-a595-ae452c24b7e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:31.222577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa572ce-7392-4091-a595-ae452c24b7e3 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:31.237839 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3954/7851] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:19:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:19:32.187404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32be71-074b-40d0-8565-12df0c490880 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:32.188289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32be71-074b-40d0-8565-12df0c490880 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:32.189459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32be71-074b-40d0-8565-12df0c490880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:32.227139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-af32be71-074b-40d0-8565-12df0c490880 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:19:32.228601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-af32be71-074b-40d0-8565-12df0c490880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:32.245741 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3898/7852] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:19:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:19:32.256581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa572ce-7392-4091-a595-ae452c24b7e3 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:32.280320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa572ce-7392-4091-a595-ae452c24b7e3 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:32.294369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeccd503-28a1-4029-a3f1-c814b22eff76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:32.296555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccd503-28a1-4029-a3f1-c814b22eff76 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:32.297008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccd503-28a1-4029-a3f1-c814b22eff76 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:32.298245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccd503-28a1-4029-a3f1-c814b22eff76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:32.337897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeccd503-28a1-4029-a3f1-c814b22eff76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:32.340204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeccd503-28a1-4029-a3f1-c814b22eff76 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:32.368216 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3955/7853] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:19:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:19:35.873608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32be71-074b-40d0-8565-12df0c490880 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:35.875656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-af32be71-074b-40d0-8565-12df0c490880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:35.907476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32be71-074b-40d0-8565-12df0c490880 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:35.936355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3d231-e4b5-4417-8025-a97980a98bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:35.940180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3d231-e4b5-4417-8025-a97980a98bd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:35.943974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3d231-e4b5-4417-8025-a97980a98bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:35.947208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3d231-e4b5-4417-8025-a97980a98bd5 admin admin] PATH_INFO: `/v3/users/3066b652821f47f6909892c9b04f4236` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:35.951958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3d231-e4b5-4417-8025-a97980a98bd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3066b652821f47f6909892c9b04f4236)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:35.964732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3d231-e4b5-4417-8025-a97980a98bd5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:36.074873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fda3d231-e4b5-4417-8025-a97980a98bd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3066b652821f47f6909892c9b04f4236'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:36.075553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fda3d231-e4b5-4417-8025-a97980a98bd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3066b652821f47f6909892c9b04f4236'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:36.076217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fda3d231-e4b5-4417-8025-a97980a98bd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3066b652821f47f6909892c9b04f4236'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:36.098552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fda3d231-e4b5-4417-8025-a97980a98bd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3066b652821f47f6909892c9b04f4236'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:36.142381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fda3d231-e4b5-4417-8025-a97980a98bd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3066b652821f47f6909892c9b04f4236'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:36.174218 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3899/7854] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:19:35 2022] DELETE /identity/v3/users/3066b652821f47f6909892c9b04f4236 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:19:36.178939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f35af-d0b1-4d1b-934e-b72221ffd499 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:36.179896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f35af-d0b1-4d1b-934e-b72221ffd499 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:36.180490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f35af-d0b1-4d1b-934e-b72221ffd499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:36.234896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c17f35af-d0b1-4d1b-934e-b72221ffd499 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:19:36.237511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c17f35af-d0b1-4d1b-934e-b72221ffd499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:36.423561 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3956/7855] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:19:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:19:36.467684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3d231-e4b5-4417-8025-a97980a98bd5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:36.469654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fda3d231-e4b5-4417-8025-a97980a98bd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:36.549077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:19:36.609797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3d231-e4b5-4417-8025-a97980a98bd5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:36.620930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e180b-da7d-46ef-b06e-0d592093bb2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:36.621830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e180b-da7d-46ef-b06e-0d592093bb2b service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:36.622301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e180b-da7d-46ef-b06e-0d592093bb2b service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:36.622754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e180b-da7d-46ef-b06e-0d592093bb2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:36.647501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e180b-da7d-46ef-b06e-0d592093bb2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:36.649640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e180b-da7d-46ef-b06e-0d592093bb2b service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:36.673746 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3900/7856] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:19:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:19:37.100492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f35af-d0b1-4d1b-934e-b72221ffd499 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:37.103601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c17f35af-d0b1-4d1b-934e-b72221ffd499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:37.120771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f35af-d0b1-4d1b-934e-b72221ffd499 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:37.127667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0d1c7-2bd8-4eaa-9649-bbae3bd07635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:37.128551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0d1c7-2bd8-4eaa-9649-bbae3bd07635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:37.129002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0d1c7-2bd8-4eaa-9649-bbae3bd07635 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:37.129438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0d1c7-2bd8-4eaa-9649-bbae3bd07635 admin admin] PATH_INFO: `/v3/projects/d4d77c36520447b3bd3bf8984e9326e2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:37.131100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0d1c7-2bd8-4eaa-9649-bbae3bd07635 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4d77c36520447b3bd3bf8984e9326e2)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:37.133957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0d1c7-2bd8-4eaa-9649-bbae3bd07635 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:37.228612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:19:37.231948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bcc0d1c7-2bd8-4eaa-9649-bbae3bd07635 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4d77c36520447b3bd3bf8984e9326e2', 'request_id': 'req-bcc0d1c7-2bd8-4eaa-9649-bbae3bd07635'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:37.247410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bcc0d1c7-2bd8-4eaa-9649-bbae3bd07635 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4d77c36520447b3bd3bf8984e9326e2', 'request_id': 'req-bcc0d1c7-2bd8-4eaa-9649-bbae3bd07635'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:37.261740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bcc0d1c7-2bd8-4eaa-9649-bbae3bd07635 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4d77c36520447b3bd3bf8984e9326e2', 'request_id': 'req-bcc0d1c7-2bd8-4eaa-9649-bbae3bd07635'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:37.292426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bcc0d1c7-2bd8-4eaa-9649-bbae3bd07635 admin admin] The token cache is being invalidate because project d4d77c36520447b3bd3bf8984e9326e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:19:37.292669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bcc0d1c7-2bd8-4eaa-9649-bbae3bd07635 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:37.294960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-bcc0d1c7-2bd8-4eaa-9649-bbae3bd07635 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:37.297226 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3957/7857] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:19:37 2022] DELETE /identity/v3/projects/d4d77c36520447b3bd3bf8984e9326e2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:19:37.301193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e180b-da7d-46ef-b06e-0d592093bb2b service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:37.321950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e180b-da7d-46ef-b06e-0d592093bb2b service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:37.487526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1a215-9066-4f34-b7f9-05c38614f3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:37.489113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1a215-9066-4f34-b7f9-05c38614f3f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:37.490942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1a215-9066-4f34-b7f9-05c38614f3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:37.497499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1a215-9066-4f34-b7f9-05c38614f3f3 admin admin] PATH_INFO: `/v3/users/4c6f15b326704a2999b906a49cf2b514` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:37.499152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def1a215-9066-4f34-b7f9-05c38614f3f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c6f15b326704a2999b906a49cf2b514)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:37.501262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def1a215-9066-4f34-b7f9-05c38614f3f3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:37.617295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-def1a215-9066-4f34-b7f9-05c38614f3f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c6f15b326704a2999b906a49cf2b514'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:37.621737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-def1a215-9066-4f34-b7f9-05c38614f3f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c6f15b326704a2999b906a49cf2b514'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:37.623674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-def1a215-9066-4f34-b7f9-05c38614f3f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c6f15b326704a2999b906a49cf2b514'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:37.709887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-def1a215-9066-4f34-b7f9-05c38614f3f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c6f15b326704a2999b906a49cf2b514'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:37.757722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-def1a215-9066-4f34-b7f9-05c38614f3f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c6f15b326704a2999b906a49cf2b514'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:37.800032 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3901/7858] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:19:37 2022] DELETE /identity/v3/users/4c6f15b326704a2999b906a49cf2b514 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:19:38.847474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0d1c7-2bd8-4eaa-9649-bbae3bd07635 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:38.881333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bcc0d1c7-2bd8-4eaa-9649-bbae3bd07635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:38.904397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0d1c7-2bd8-4eaa-9649-bbae3bd07635 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:39.079543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5055a587-c30a-458f-aee2-218e09ebf4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:39.080178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055a587-c30a-458f-aee2-218e09ebf4bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:39.080663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055a587-c30a-458f-aee2-218e09ebf4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:39.081232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055a587-c30a-458f-aee2-218e09ebf4bd admin admin] PATH_INFO: `/v3/projects/4d383d8983284b36a205307ea45425ec` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:39.083280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5055a587-c30a-458f-aee2-218e09ebf4bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d383d8983284b36a205307ea45425ec)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:39.085686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5055a587-c30a-458f-aee2-218e09ebf4bd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:39.188956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5055a587-c30a-458f-aee2-218e09ebf4bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d383d8983284b36a205307ea45425ec', 'request_id': 'req-5055a587-c30a-458f-aee2-218e09ebf4bd'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:39.216873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5055a587-c30a-458f-aee2-218e09ebf4bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d383d8983284b36a205307ea45425ec', 'request_id': 'req-5055a587-c30a-458f-aee2-218e09ebf4bd'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:39.229878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5055a587-c30a-458f-aee2-218e09ebf4bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d383d8983284b36a205307ea45425ec', 'request_id': 'req-5055a587-c30a-458f-aee2-218e09ebf4bd'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:39.284771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5055a587-c30a-458f-aee2-218e09ebf4bd admin admin] The token cache is being invalidate because project 4d383d8983284b36a205307ea45425ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:19:39.286279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5055a587-c30a-458f-aee2-218e09ebf4bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:39.288863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5055a587-c30a-458f-aee2-218e09ebf4bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:19:39.297196 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3958/7859] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:19:38 2022] DELETE /identity/v3/projects/4d383d8983284b36a205307ea45425ec => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:19:39.299031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208c388-3b62-49c3-ade7-d902d6281126 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:39.299507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208c388-3b62-49c3-ade7-d902d6281126 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:39.299770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208c388-3b62-49c3-ade7-d902d6281126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:39.340705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b208c388-3b62-49c3-ade7-d902d6281126 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:19:39.355007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b208c388-3b62-49c3-ade7-d902d6281126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:39.468985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:19:39.500003 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3902/7860] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:19:39.505518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bafbe-46c8-4a05-a24e-8856a1cbe77f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:39.506779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bafbe-46c8-4a05-a24e-8856a1cbe77f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:39.507221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bafbe-46c8-4a05-a24e-8856a1cbe77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:39.534555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-004bafbe-46c8-4a05-a24e-8856a1cbe77f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:19:39.535800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-004bafbe-46c8-4a05-a24e-8856a1cbe77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:39.550043 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3959/7861] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:19:39.552435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208c388-3b62-49c3-ade7-d902d6281126 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:39.565733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208c388-3b62-49c3-ade7-d902d6281126 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:39.575678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395edb5-b3f5-4ba8-9a3e-d46965f4f24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:39.576339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395edb5-b3f5-4ba8-9a3e-d46965f4f24e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:39.576617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395edb5-b3f5-4ba8-9a3e-d46965f4f24e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:39.577165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395edb5-b3f5-4ba8-9a3e-d46965f4f24e admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:39.577714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395edb5-b3f5-4ba8-9a3e-d46965f4f24e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:19:39.578051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395edb5-b3f5-4ba8-9a3e-d46965f4f24e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:39.579715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395edb5-b3f5-4ba8-9a3e-d46965f4f24e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:39.605579 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3903/7862] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:19:39.610407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004bafbe-46c8-4a05-a24e-8856a1cbe77f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:39.625669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004bafbe-46c8-4a05-a24e-8856a1cbe77f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:39.630694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edaa852-2168-43e2-9c41-94d50a3e879c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:39.631310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edaa852-2168-43e2-9c41-94d50a3e879c admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:39.631589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edaa852-2168-43e2-9c41-94d50a3e879c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:39.631861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edaa852-2168-43e2-9c41-94d50a3e879c admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:39.632484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edaa852-2168-43e2-9c41-94d50a3e879c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:39.635324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edaa852-2168-43e2-9c41-94d50a3e879c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:39.677507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395edb5-b3f5-4ba8-9a3e-d46965f4f24e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:39.679214 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3960/7863] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:19:39 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:19:39.692126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395edb5-b3f5-4ba8-9a3e-d46965f4f24e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:39.779954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d08c814-a7a4-42eb-ada9-ead36ce0e5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:39.780668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08c814-a7a4-42eb-ada9-ead36ce0e5ed admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:39.780959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08c814-a7a4-42eb-ada9-ead36ce0e5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:39.781234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08c814-a7a4-42eb-ada9-ead36ce0e5ed admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:39.781870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d08c814-a7a4-42eb-ada9-ead36ce0e5ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:39.783973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d08c814-a7a4-42eb-ada9-ead36ce0e5ed admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:39.825081 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3904/7864] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:19:39 2022] POST /identity/v3/users => generated 287 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:19:39.829108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edaa852-2168-43e2-9c41-94d50a3e879c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:39.848437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edaa852-2168-43e2-9c41-94d50a3e879c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:39.859095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f7c15-b9ae-4418-9d2d-7f9bd39108a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:39.859811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f7c15-b9ae-4418-9d2d-7f9bd39108a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:39.860393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f7c15-b9ae-4418-9d2d-7f9bd39108a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:39.860393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f7c15-b9ae-4418-9d2d-7f9bd39108a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:39.861158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f7c15-b9ae-4418-9d2d-7f9bd39108a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:39.862859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f7c15-b9ae-4418-9d2d-7f9bd39108a7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:39.884634 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3961/7865] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:19:39.892168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d08c814-a7a4-42eb-ada9-ead36ce0e5ed admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:39.923059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d08c814-a7a4-42eb-ada9-ead36ce0e5ed admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:39.927900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda4989-60e0-40b9-9509-fdc4c214a6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:39.928509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda4989-60e0-40b9-9509-fdc4c214a6ec admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:39.928790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda4989-60e0-40b9-9509-fdc4c214a6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:39.929098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda4989-60e0-40b9-9509-fdc4c214a6ec admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:39.929741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda4989-60e0-40b9-9509-fdc4c214a6ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:39.931369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda4989-60e0-40b9-9509-fdc4c214a6ec admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:39.944087 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3905/7866] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:19:39.948256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f7c15-b9ae-4418-9d2d-7f9bd39108a7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:39.961233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f7c15-b9ae-4418-9d2d-7f9bd39108a7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:39.967307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d29e08-2c57-42e8-ab16-d47fdde77432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:39.968044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d29e08-2c57-42e8-ab16-d47fdde77432 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:39.968407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d29e08-2c57-42e8-ab16-d47fdde77432 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:39.968750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d29e08-2c57-42e8-ab16-d47fdde77432 admin admin] PATH_INFO: `/v3/projects/45d45087fbd24f2c90470e455d620178/users/c97a8a29cea043f5878205407dafc100/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:39.987173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d29e08-2c57-42e8-ab16-d47fdde77432 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d45087fbd24f2c90470e455d620178, user_id=c97a8a29cea043f5878205407dafc100, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:39.990160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d29e08-2c57-42e8-ab16-d47fdde77432 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:40.008981 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3962/7867] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:19:39 2022] PUT /identity/v3/projects/45d45087fbd24f2c90470e455d620178/users/c97a8a29cea043f5878205407dafc100/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:19:40.014916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8986620-3da8-4d38-863c-a800e9d0b812 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:40.015355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8986620-3da8-4d38-863c-a800e9d0b812 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:40.015914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8986620-3da8-4d38-863c-a800e9d0b812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:40.045648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a8986620-3da8-4d38-863c-a800e9d0b812 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:19:40.047489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a8986620-3da8-4d38-863c-a800e9d0b812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:40.129548 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3906/7868] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:19:40.140065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d29e08-2c57-42e8-ab16-d47fdde77432 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:40.142141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-49d29e08-2c57-42e8-ab16-d47fdde77432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:40.150068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:19:40.226780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d29e08-2c57-42e8-ab16-d47fdde77432 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:40.231826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de1637-eec4-41be-ae53-47d66603e705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:40.232440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de1637-eec4-41be-ae53-47d66603e705 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:40.232729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de1637-eec4-41be-ae53-47d66603e705 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:40.232996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de1637-eec4-41be-ae53-47d66603e705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:40.245821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de1637-eec4-41be-ae53-47d66603e705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:40.245821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de1637-eec4-41be-ae53-47d66603e705 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:40.257366 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3963/7869] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:19:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:19:47.143087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08a51b-5a65-4ef1-a82e-26ded450c52c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:47.143969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08a51b-5a65-4ef1-a82e-26ded450c52c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:47.144436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08a51b-5a65-4ef1-a82e-26ded450c52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:47.235037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ea08a51b-5a65-4ef1-a82e-26ded450c52c None None] MFA Rules not processed for user `c97a8a29cea043f5878205407dafc100`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:19:47.237849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ea08a51b-5a65-4ef1-a82e-26ded450c52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:47.397163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:19:47.439825 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3907/7870] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:19:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:19:47.444467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de1637-eec4-41be-ae53-47d66603e705 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:47.458539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de1637-eec4-41be-ae53-47d66603e705 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:47.480532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48ad3e-1a68-4ae3-b0d2-efbd0e5b687e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:47.480532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48ad3e-1a68-4ae3-b0d2-efbd0e5b687e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:47.487133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48ad3e-1a68-4ae3-b0d2-efbd0e5b687e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:47.487133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48ad3e-1a68-4ae3-b0d2-efbd0e5b687e admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:47.489971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d48ad3e-1a68-4ae3-b0d2-efbd0e5b687e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:47.497750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d48ad3e-1a68-4ae3-b0d2-efbd0e5b687e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:47.595281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08a51b-5a65-4ef1-a82e-26ded450c52c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:47.596304 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3964/7871] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:19:47 2022] POST /identity/v3/projects => generated 375 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:19:47.617760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08a51b-5a65-4ef1-a82e-26ded450c52c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:47.772191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc168a-95f6-4e98-9c50-e0efc95072d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:47.773153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc168a-95f6-4e98-9c50-e0efc95072d9 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:47.773616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc168a-95f6-4e98-9c50-e0efc95072d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:47.774162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc168a-95f6-4e98-9c50-e0efc95072d9 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:47.774992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc168a-95f6-4e98-9c50-e0efc95072d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:47.777742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc168a-95f6-4e98-9c50-e0efc95072d9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:47.881099 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3908/7872] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:19:47 2022] POST /identity/v3/users => generated 294 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:19:47.885335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48ad3e-1a68-4ae3-b0d2-efbd0e5b687e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:47.904505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48ad3e-1a68-4ae3-b0d2-efbd0e5b687e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:47.911369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc0d42-5005-41d0-9820-33b4bb4d2694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:47.912077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc0d42-5005-41d0-9820-33b4bb4d2694 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:47.912435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc0d42-5005-41d0-9820-33b4bb4d2694 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:47.912781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc0d42-5005-41d0-9820-33b4bb4d2694 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:47.913514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc0d42-5005-41d0-9820-33b4bb4d2694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:47.915218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc0d42-5005-41d0-9820-33b4bb4d2694 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:47.937855 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3965/7873] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:19:47.942689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc168a-95f6-4e98-9c50-e0efc95072d9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:47.976032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc168a-95f6-4e98-9c50-e0efc95072d9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:47.991848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c73984-f053-49f2-a021-2fee950199be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:47.992681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c73984-f053-49f2-a021-2fee950199be admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:47.993110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c73984-f053-49f2-a021-2fee950199be admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:47.993464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c73984-f053-49f2-a021-2fee950199be admin admin] PATH_INFO: `/v3/projects/9441e87384c5410bb11278bead2a3761/users/bbe6040ce96544c0a301331101db7267/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:48.038359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c73984-f053-49f2-a021-2fee950199be admin admin] RBAC: Authorizing `identity:create_grant(project_id=9441e87384c5410bb11278bead2a3761, user_id=bbe6040ce96544c0a301331101db7267, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:48.041907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c73984-f053-49f2-a021-2fee950199be admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:48.099170 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3909/7874] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:19:47 2022] PUT /identity/v3/projects/9441e87384c5410bb11278bead2a3761/users/bbe6040ce96544c0a301331101db7267/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:19:48.100363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc0d42-5005-41d0-9820-33b4bb4d2694 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:48.126598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc0d42-5005-41d0-9820-33b4bb4d2694 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:48.288899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7a578-fae2-45bf-92dc-dab88104083d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:48.289617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7a578-fae2-45bf-92dc-dab88104083d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:48.292903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7a578-fae2-45bf-92dc-dab88104083d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:48.293414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7a578-fae2-45bf-92dc-dab88104083d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:48.294843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a7a578-fae2-45bf-92dc-dab88104083d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:48.301528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a7a578-fae2-45bf-92dc-dab88104083d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:48.330741 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3966/7875] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:19:48.334768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c73984-f053-49f2-a021-2fee950199be admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:48.354968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c73984-f053-49f2-a021-2fee950199be admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:48.364795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff42fee-7a15-494a-8b84-28657e4b56e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:48.366144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff42fee-7a15-494a-8b84-28657e4b56e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:48.366812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff42fee-7a15-494a-8b84-28657e4b56e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:48.367369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff42fee-7a15-494a-8b84-28657e4b56e4 admin admin] PATH_INFO: `/v3/projects/9441e87384c5410bb11278bead2a3761/users/bbe6040ce96544c0a301331101db7267/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:48.379218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff42fee-7a15-494a-8b84-28657e4b56e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9441e87384c5410bb11278bead2a3761, user_id=bbe6040ce96544c0a301331101db7267, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:48.387792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff42fee-7a15-494a-8b84-28657e4b56e4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:48.413802 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3910/7876] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:19:48 2022] PUT /identity/v3/projects/9441e87384c5410bb11278bead2a3761/users/bbe6040ce96544c0a301331101db7267/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:19:53.469352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47393cca-7ca4-4fa5-95db-5f6396761fb0 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:53.470474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47393cca-7ca4-4fa5-95db-5f6396761fb0 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:53.471020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47393cca-7ca4-4fa5-95db-5f6396761fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:53.498389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-47393cca-7ca4-4fa5-95db-5f6396761fb0 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:19:53.500653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-47393cca-7ca4-4fa5-95db-5f6396761fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:53.630283 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3967/7877] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:19:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:19:53.639391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff42fee-7a15-494a-8b84-28657e4b56e4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:53.642317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-aff42fee-7a15-494a-8b84-28657e4b56e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:53.808504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff42fee-7a15-494a-8b84-28657e4b56e4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:53.816021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca450965-7a1b-47c6-9ee2-1d24a5293797 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:53.816889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca450965-7a1b-47c6-9ee2-1d24a5293797 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:53.817345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca450965-7a1b-47c6-9ee2-1d24a5293797 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:53.817789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca450965-7a1b-47c6-9ee2-1d24a5293797 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:53.847119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca450965-7a1b-47c6-9ee2-1d24a5293797 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:53.850061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca450965-7a1b-47c6-9ee2-1d24a5293797 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:53.868473 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3911/7878] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:19:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:19:55.409029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47393cca-7ca4-4fa5-95db-5f6396761fb0 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:55.437031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:19:55.547824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47393cca-7ca4-4fa5-95db-5f6396761fb0 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:55.552912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cd970-9be3-4538-a860-235dcf2e0144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:55.554023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cd970-9be3-4538-a860-235dcf2e0144 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:55.554824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cd970-9be3-4538-a860-235dcf2e0144 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:55.557380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cd970-9be3-4538-a860-235dcf2e0144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:55.560969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-216cd970-9be3-4538-a860-235dcf2e0144 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:55.607027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cd970-9be3-4538-a860-235dcf2e0144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:55.611521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cd970-9be3-4538-a860-235dcf2e0144 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:55.680307 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3968/7879] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:19:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:19:55.778058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3938a1-03d0-4236-8333-a012d4ec1780 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:55.778561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3938a1-03d0-4236-8333-a012d4ec1780 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:55.778683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3938a1-03d0-4236-8333-a012d4ec1780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:55.855451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fb3938a1-03d0-4236-8333-a012d4ec1780 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:19:55.859990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fb3938a1-03d0-4236-8333-a012d4ec1780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:55.890862 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3912/7880] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:19:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:19:55.903782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cd970-9be3-4538-a860-235dcf2e0144 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:55.937279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cd970-9be3-4538-a860-235dcf2e0144 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:55.963080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db298a-147c-4ef6-bac9-711fb39d6662 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:55.963897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db298a-147c-4ef6-bac9-711fb39d6662 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:55.965626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db298a-147c-4ef6-bac9-711fb39d6662 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:55.967308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db298a-147c-4ef6-bac9-711fb39d6662 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:55.990206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db298a-147c-4ef6-bac9-711fb39d6662 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:55.992577 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db298a-147c-4ef6-bac9-711fb39d6662 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:56.017606 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3969/7881] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:19:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 21:19:56.740446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208666aa-da33-46d8-a6bd-60723d09e1c1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:56.740746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208666aa-da33-46d8-a6bd-60723d09e1c1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:56.741022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208666aa-da33-46d8-a6bd-60723d09e1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:56.803889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-208666aa-da33-46d8-a6bd-60723d09e1c1 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:19:56.811755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-208666aa-da33-46d8-a6bd-60723d09e1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:56.837084 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3913/7882] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:19:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:19:56.850949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db298a-147c-4ef6-bac9-711fb39d6662 service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:56.871231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db298a-147c-4ef6-bac9-711fb39d6662 service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:56.897028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa867f4b-3a08-4d63-8e4a-b93fb80c1400 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:56.898103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa867f4b-3a08-4d63-8e4a-b93fb80c1400 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:56.899576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa867f4b-3a08-4d63-8e4a-b93fb80c1400 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:56.900369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa867f4b-3a08-4d63-8e4a-b93fb80c1400 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:56.922321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa867f4b-3a08-4d63-8e4a-b93fb80c1400 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:56.926039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa867f4b-3a08-4d63-8e4a-b93fb80c1400 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:56.960932 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3970/7883] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:19:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:19:57.187566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2710df-856d-4527-9191-3e19e6122057 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:57.187856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2710df-856d-4527-9191-3e19e6122057 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:57.188132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2710df-856d-4527-9191-3e19e6122057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:57.225682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2a2710df-856d-4527-9191-3e19e6122057 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:19:57.228999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2a2710df-856d-4527-9191-3e19e6122057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:19:57.260379 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3914/7884] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:19:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:19:57.283741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa867f4b-3a08-4d63-8e4a-b93fb80c1400 service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:19:57.299409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa867f4b-3a08-4d63-8e4a-b93fb80c1400 service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:19:57.334062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09847309-e398-4489-8447-7a64a11cb1dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:19:57.334724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09847309-e398-4489-8447-7a64a11cb1dc service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:19:57.335012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09847309-e398-4489-8447-7a64a11cb1dc service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:19:57.335285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09847309-e398-4489-8447-7a64a11cb1dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:19:57.373170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09847309-e398-4489-8447-7a64a11cb1dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:19:57.377863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09847309-e398-4489-8447-7a64a11cb1dc service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:19:57.406563 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3971/7885] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:19:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:20:00.902898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a9c95-c9f0-4ef7-bfc4-22b02154deab None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:00.906897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a9c95-c9f0-4ef7-bfc4-22b02154deab None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:00.907830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a9c95-c9f0-4ef7-bfc4-22b02154deab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:00.987245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:20:00.994444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-dc1a9c95-c9f0-4ef7-bfc4-22b02154deab None None] MFA Rules not processed for user `bbe6040ce96544c0a301331101db7267`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:20:00.997491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-dc1a9c95-c9f0-4ef7-bfc4-22b02154deab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:20:01.182070 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3915/7886] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:20:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:20:01.191628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09847309-e398-4489-8447-7a64a11cb1dc service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:20:01.194749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-09847309-e398-4489-8447-7a64a11cb1dc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:20:01.438430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09847309-e398-4489-8447-7a64a11cb1dc service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:20:01.457650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc5af7-34c3-4400-91e8-b1e3cdf86987 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:20:01.458800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc5af7-34c3-4400-91e8-b1e3cdf86987 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:01.459180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc5af7-34c3-4400-91e8-b1e3cdf86987 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:01.462597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc5af7-34c3-4400-91e8-b1e3cdf86987 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:01.493017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbc5af7-34c3-4400-91e8-b1e3cdf86987 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:20:01.495467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbc5af7-34c3-4400-91e8-b1e3cdf86987 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:20:01.517341 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3972/7887] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:20:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:20:01.661098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120c7aa-f7ea-4f5c-9dd8-2cf00eae35bb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:01.664168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120c7aa-f7ea-4f5c-9dd8-2cf00eae35bb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:01.666565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120c7aa-f7ea-4f5c-9dd8-2cf00eae35bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:01.714298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3120c7aa-f7ea-4f5c-9dd8-2cf00eae35bb None None] MFA Rules not processed for user `c97a8a29cea043f5878205407dafc100`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:20:01.716035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3120c7aa-f7ea-4f5c-9dd8-2cf00eae35bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:20:01.945702 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3916/7888] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:20:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:20:03.763275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc5af7-34c3-4400-91e8-b1e3cdf86987 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:20:03.766345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ffbc5af7-34c3-4400-91e8-b1e3cdf86987 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:20:03.906435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:20:03.921954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc5af7-34c3-4400-91e8-b1e3cdf86987 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:20:03.928213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51148c3-1cf8-4b0e-8a1f-251d8fda6d53 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:20:03.929106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51148c3-1cf8-4b0e-8a1f-251d8fda6d53 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:03.929560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51148c3-1cf8-4b0e-8a1f-251d8fda6d53 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:03.930112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51148c3-1cf8-4b0e-8a1f-251d8fda6d53 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:03.937263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e51148c3-1cf8-4b0e-8a1f-251d8fda6d53 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:20:03.956523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51148c3-1cf8-4b0e-8a1f-251d8fda6d53 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:20:03.958191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51148c3-1cf8-4b0e-8a1f-251d8fda6d53 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:20:04.007767 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3973/7889] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:20:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:20:05.966564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018e635-5718-4b7a-8a00-3e195186ade6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:05.967153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018e635-5718-4b7a-8a00-3e195186ade6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:05.967153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018e635-5718-4b7a-8a00-3e195186ade6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:06.016135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4018e635-5718-4b7a-8a00-3e195186ade6 None None] MFA Rules not processed for user `6ac020dc323042c48ccce2a8826bc6be`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:20:06.033336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4018e635-5718-4b7a-8a00-3e195186ade6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:20:06.302965 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3917/7890] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:20:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:20:13.967632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:20:15.888222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51148c3-1cf8-4b0e-8a1f-251d8fda6d53 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:20:15.917078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51148c3-1cf8-4b0e-8a1f-251d8fda6d53 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:20:15.941131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f0d8d-5120-4003-8364-1b16f0233c13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:20:15.943286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f0d8d-5120-4003-8364-1b16f0233c13 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:15.946019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f0d8d-5120-4003-8364-1b16f0233c13 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:15.950689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f0d8d-5120-4003-8364-1b16f0233c13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:15.955953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-013f0d8d-5120-4003-8364-1b16f0233c13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:20:16.020137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f0d8d-5120-4003-8364-1b16f0233c13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:20:16.025979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f0d8d-5120-4003-8364-1b16f0233c13 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:20:16.083626 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3974/7891] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:20:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:20:16.265853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=78 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:20:18.391456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018e635-5718-4b7a-8a00-3e195186ade6 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:20:18.392926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4018e635-5718-4b7a-8a00-3e195186ade6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:20:18.456831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018e635-5718-4b7a-8a00-3e195186ade6 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:20:18.487592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b49bb-948e-4054-a5e5-f7b7fbd28753 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:20:18.488242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b49bb-948e-4054-a5e5-f7b7fbd28753 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:18.488521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b49bb-948e-4054-a5e5-f7b7fbd28753 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:18.488796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b49bb-948e-4054-a5e5-f7b7fbd28753 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:18.501853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5c9b49bb-948e-4054-a5e5-f7b7fbd28753 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:20:18.562410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b49bb-948e-4054-a5e5-f7b7fbd28753 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:20:18.564070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b49bb-948e-4054-a5e5-f7b7fbd28753 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:20:18.642940 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3918/7892] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:20:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:20:20.087396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad893fc-d4ce-419f-b9ab-cac20863aa03 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:20.088024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad893fc-d4ce-419f-b9ab-cac20863aa03 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:20.088024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad893fc-d4ce-419f-b9ab-cac20863aa03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:20.112573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1ad893fc-d4ce-419f-b9ab-cac20863aa03 None None] MFA Rules not processed for user `c97a8a29cea043f5878205407dafc100`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:20:20.115722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1ad893fc-d4ce-419f-b9ab-cac20863aa03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:20:20.130988 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3975/7893] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:20:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:20:28.599967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:20:30.106042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=9 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:20:39.621456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b86d3c-d705-4f2e-8470-4f17b4d19da3 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:39.622553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b86d3c-d705-4f2e-8470-4f17b4d19da3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:39.623058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b86d3c-d705-4f2e-8470-4f17b4d19da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:39.688267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-30b86d3c-d705-4f2e-8470-4f17b4d19da3 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:20:39.690783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-30b86d3c-d705-4f2e-8470-4f17b4d19da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:20:39.701744 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3919/7894] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:20:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:20:39.717442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad893fc-d4ce-419f-b9ab-cac20863aa03 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:20:39.738080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad893fc-d4ce-419f-b9ab-cac20863aa03 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:20:39.745933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d87466-46ab-4bc1-a64e-5ea667dff22c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:20:39.747246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d87466-46ab-4bc1-a64e-5ea667dff22c service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:39.747756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d87466-46ab-4bc1-a64e-5ea667dff22c service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:39.748294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d87466-46ab-4bc1-a64e-5ea667dff22c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:39.761448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d87466-46ab-4bc1-a64e-5ea667dff22c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:20:39.764326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d87466-46ab-4bc1-a64e-5ea667dff22c service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:20:39.789465 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3976/7895] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:20:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:20:43.481781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b86d3c-d705-4f2e-8470-4f17b4d19da3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:20:43.483360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-30b86d3c-d705-4f2e-8470-4f17b4d19da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:20:43.616898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b86d3c-d705-4f2e-8470-4f17b4d19da3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:20:43.622106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eed73f-c3b1-436b-875c-91f65ef6ecf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:20:43.622783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eed73f-c3b1-436b-875c-91f65ef6ecf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:43.623281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eed73f-c3b1-436b-875c-91f65ef6ecf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:43.623566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eed73f-c3b1-436b-875c-91f65ef6ecf7 admin admin] PATH_INFO: `/v3/users/ecdf7bba776f4b9c94f2653679505303` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:43.624710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eed73f-c3b1-436b-875c-91f65ef6ecf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecdf7bba776f4b9c94f2653679505303)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:20:43.626777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eed73f-c3b1-436b-875c-91f65ef6ecf7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:20:43.720806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-63eed73f-c3b1-436b-875c-91f65ef6ecf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecdf7bba776f4b9c94f2653679505303'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:20:43.722067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-63eed73f-c3b1-436b-875c-91f65ef6ecf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecdf7bba776f4b9c94f2653679505303'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:20:43.723045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-63eed73f-c3b1-436b-875c-91f65ef6ecf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecdf7bba776f4b9c94f2653679505303'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:20:43.734772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-63eed73f-c3b1-436b-875c-91f65ef6ecf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecdf7bba776f4b9c94f2653679505303'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:20:43.763141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-63eed73f-c3b1-436b-875c-91f65ef6ecf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecdf7bba776f4b9c94f2653679505303'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:20:43.790576 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3920/7896] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:20:43 2022] DELETE /identity/v3/users/ecdf7bba776f4b9c94f2653679505303 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:20:44.092404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d87466-46ab-4bc1-a64e-5ea667dff22c service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:20:44.115997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d87466-46ab-4bc1-a64e-5ea667dff22c service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:20:44.188227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377b852-d464-45ce-b7fe-bed6ad76c6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:20:44.189025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377b852-d464-45ce-b7fe-bed6ad76c6d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:44.189720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377b852-d464-45ce-b7fe-bed6ad76c6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:44.190097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377b852-d464-45ce-b7fe-bed6ad76c6d1 admin admin] PATH_INFO: `/v3/projects/5f12200ca5c649cd993499b7cb7e382f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:44.191470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377b852-d464-45ce-b7fe-bed6ad76c6d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f12200ca5c649cd993499b7cb7e382f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:20:44.193584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377b852-d464-45ce-b7fe-bed6ad76c6d1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:20:44.261672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0377b852-d464-45ce-b7fe-bed6ad76c6d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f12200ca5c649cd993499b7cb7e382f', 'request_id': 'req-0377b852-d464-45ce-b7fe-bed6ad76c6d1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:20:44.268894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0377b852-d464-45ce-b7fe-bed6ad76c6d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f12200ca5c649cd993499b7cb7e382f', 'request_id': 'req-0377b852-d464-45ce-b7fe-bed6ad76c6d1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:20:44.276383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0377b852-d464-45ce-b7fe-bed6ad76c6d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f12200ca5c649cd993499b7cb7e382f', 'request_id': 'req-0377b852-d464-45ce-b7fe-bed6ad76c6d1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:20:44.303845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0377b852-d464-45ce-b7fe-bed6ad76c6d1 admin admin] The token cache is being invalidate because project 5f12200ca5c649cd993499b7cb7e382f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:20:44.304837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0377b852-d464-45ce-b7fe-bed6ad76c6d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:20:44.309412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0377b852-d464-45ce-b7fe-bed6ad76c6d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:20:44.316675 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3977/7897] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:20:44 2022] DELETE /identity/v3/projects/5f12200ca5c649cd993499b7cb7e382f => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:20:44.317831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d270753-272d-4830-8707-c949eb59ac5e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:44.318180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d270753-272d-4830-8707-c949eb59ac5e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:44.318527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d270753-272d-4830-8707-c949eb59ac5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:44.377556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1d270753-272d-4830-8707-c949eb59ac5e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:20:44.381799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1d270753-272d-4830-8707-c949eb59ac5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:20:44.513216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:20:44.537517 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3921/7898] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:20:44.545645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd12755-0724-4d43-ab34-db185fb2c603 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:44.546313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd12755-0724-4d43-ab34-db185fb2c603 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:44.546836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd12755-0724-4d43-ab34-db185fb2c603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:44.572561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0fd12755-0724-4d43-ab34-db185fb2c603 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:20:44.573945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0fd12755-0724-4d43-ab34-db185fb2c603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:20:44.583424 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3978/7899] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:20:44.587957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d270753-272d-4830-8707-c949eb59ac5e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:20:44.602197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d270753-272d-4830-8707-c949eb59ac5e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:20:44.608195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17959d7e-626d-4e5c-b362-39b3f09397b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:20:44.608848 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17959d7e-626d-4e5c-b362-39b3f09397b0 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:44.609141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17959d7e-626d-4e5c-b362-39b3f09397b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:44.609468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17959d7e-626d-4e5c-b362-39b3f09397b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:44.610285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17959d7e-626d-4e5c-b362-39b3f09397b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:20:44.610626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17959d7e-626d-4e5c-b362-39b3f09397b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:20:44.612653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17959d7e-626d-4e5c-b362-39b3f09397b0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:20:44.630731 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3922/7900] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:20:44.632089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd12755-0724-4d43-ab34-db185fb2c603 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:20:44.642645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd12755-0724-4d43-ab34-db185fb2c603 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:20:44.647600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efa1cb-650b-4816-a4da-38839e668a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:20:44.648258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efa1cb-650b-4816-a4da-38839e668a08 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:44.648542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efa1cb-650b-4816-a4da-38839e668a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:44.649190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efa1cb-650b-4816-a4da-38839e668a08 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:44.649984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efa1cb-650b-4816-a4da-38839e668a08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:20:44.652079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efa1cb-650b-4816-a4da-38839e668a08 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:20:44.687334 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3979/7901] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:20:44 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:20:44.687635 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17959d7e-626d-4e5c-b362-39b3f09397b0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:20:44.705028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17959d7e-626d-4e5c-b362-39b3f09397b0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:20:44.777451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0c6f7-c3bf-4226-9f6b-783c39674abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:20:44.777990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0c6f7-c3bf-4226-9f6b-783c39674abf admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:44.778274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0c6f7-c3bf-4226-9f6b-783c39674abf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:44.778589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0c6f7-c3bf-4226-9f6b-783c39674abf admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:44.779261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a0c6f7-c3bf-4226-9f6b-783c39674abf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:20:44.781464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a0c6f7-c3bf-4226-9f6b-783c39674abf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:20:44.857903 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3923/7902] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:20:44 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:20:44.859425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efa1cb-650b-4816-a4da-38839e668a08 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:20:44.875687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efa1cb-650b-4816-a4da-38839e668a08 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:20:44.892590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ff0ad-3665-4be5-b4eb-0adacee3a2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:20:44.895082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ff0ad-3665-4be5-b4eb-0adacee3a2d0 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:44.895578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ff0ad-3665-4be5-b4eb-0adacee3a2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:44.895865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ff0ad-3665-4be5-b4eb-0adacee3a2d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:44.896540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ff0ad-3665-4be5-b4eb-0adacee3a2d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:20:44.898978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ff0ad-3665-4be5-b4eb-0adacee3a2d0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:20:44.915362 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3980/7903] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:20:44.917947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0c6f7-c3bf-4226-9f6b-783c39674abf admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:20:44.934066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0c6f7-c3bf-4226-9f6b-783c39674abf admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:20:44.941165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a899e6-58b7-4dd6-9f7b-58a9723b4a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:20:44.942033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a899e6-58b7-4dd6-9f7b-58a9723b4a45 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:44.942478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a899e6-58b7-4dd6-9f7b-58a9723b4a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:44.942962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a899e6-58b7-4dd6-9f7b-58a9723b4a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:44.943813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a899e6-58b7-4dd6-9f7b-58a9723b4a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:20:44.946529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a899e6-58b7-4dd6-9f7b-58a9723b4a45 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:20:44.961809 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3924/7904] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:20:44.965612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ff0ad-3665-4be5-b4eb-0adacee3a2d0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:20:44.984781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ff0ad-3665-4be5-b4eb-0adacee3a2d0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:20:44.997010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40c286-97a3-4ba1-97fc-5eff53f171d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:20:44.998116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40c286-97a3-4ba1-97fc-5eff53f171d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:44.999403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40c286-97a3-4ba1-97fc-5eff53f171d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:44.999684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40c286-97a3-4ba1-97fc-5eff53f171d5 admin admin] PATH_INFO: `/v3/projects/df0e56b020444dd28af868590b1bccf4/users/eb0bf5fce885439eaa16a0871c2992ac/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:45.034421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e40c286-97a3-4ba1-97fc-5eff53f171d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df0e56b020444dd28af868590b1bccf4, user_id=eb0bf5fce885439eaa16a0871c2992ac, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:20:45.039536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e40c286-97a3-4ba1-97fc-5eff53f171d5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:20:45.067468 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3981/7905] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:20:44 2022] PUT /identity/v3/projects/df0e56b020444dd28af868590b1bccf4/users/eb0bf5fce885439eaa16a0871c2992ac/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:20:45.069098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f4cc7-9acd-4105-93e0-0b24cf29dbf6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:45.069394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f4cc7-9acd-4105-93e0-0b24cf29dbf6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:45.069774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f4cc7-9acd-4105-93e0-0b24cf29dbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:45.098645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b73f4cc7-9acd-4105-93e0-0b24cf29dbf6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:20:45.099947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b73f4cc7-9acd-4105-93e0-0b24cf29dbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:20:45.177258 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3925/7906] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:20:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:20:45.194921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40c286-97a3-4ba1-97fc-5eff53f171d5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:20:45.196957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9e40c286-97a3-4ba1-97fc-5eff53f171d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:20:45.213316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:20:45.279075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40c286-97a3-4ba1-97fc-5eff53f171d5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:20:45.283282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191652b1-f37e-4a5d-869a-1367e20c2770 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:20:45.284103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191652b1-f37e-4a5d-869a-1367e20c2770 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:45.284545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191652b1-f37e-4a5d-869a-1367e20c2770 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:45.284832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191652b1-f37e-4a5d-869a-1367e20c2770 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:45.298434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191652b1-f37e-4a5d-869a-1367e20c2770 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:20:45.300226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191652b1-f37e-4a5d-869a-1367e20c2770 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:20:45.312950 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3982/7907] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:20:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:20:55.168477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=72,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:20:55.308005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=20 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:20:55.364281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270eddf-5d7e-4e5e-aa39-c29e11048eda None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:55.364799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270eddf-5d7e-4e5e-aa39-c29e11048eda None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:55.365292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270eddf-5d7e-4e5e-aa39-c29e11048eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:55.442366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7270eddf-5d7e-4e5e-aa39-c29e11048eda None None] MFA Rules not processed for user `eb0bf5fce885439eaa16a0871c2992ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:20:55.443665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7270eddf-5d7e-4e5e-aa39-c29e11048eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:20:55.559330 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3926/7908] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:20:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:20:55.564676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191652b1-f37e-4a5d-869a-1367e20c2770 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:20:55.591845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191652b1-f37e-4a5d-869a-1367e20c2770 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:20:55.597899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e3682-d886-4e63-a24a-1326192e88e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:20:55.599085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e3682-d886-4e63-a24a-1326192e88e1 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:55.599565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e3682-d886-4e63-a24a-1326192e88e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:55.599997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e3682-d886-4e63-a24a-1326192e88e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:55.601328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e3682-d886-4e63-a24a-1326192e88e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:20:55.603462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e3682-d886-4e63-a24a-1326192e88e1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:20:55.665692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270eddf-5d7e-4e5e-aa39-c29e11048eda None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:20:55.673683 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3983/7909] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:20:55 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:20:55.679473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270eddf-5d7e-4e5e-aa39-c29e11048eda None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:20:55.810565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36543703-8df3-41b6-aa6e-8d13b9df9014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:20:55.810810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36543703-8df3-41b6-aa6e-8d13b9df9014 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:55.811050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36543703-8df3-41b6-aa6e-8d13b9df9014 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:55.811605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36543703-8df3-41b6-aa6e-8d13b9df9014 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:55.812847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36543703-8df3-41b6-aa6e-8d13b9df9014 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:20:55.815277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36543703-8df3-41b6-aa6e-8d13b9df9014 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:20:55.849020 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3927/7910] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:20:55 2022] POST /identity/v3/users => generated 296 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:20:55.853640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e3682-d886-4e63-a24a-1326192e88e1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:20:55.909496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e3682-d886-4e63-a24a-1326192e88e1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:20:55.936853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a6efb-373b-4796-bbb4-6f274f50a423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:20:55.937868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a6efb-373b-4796-bbb4-6f274f50a423 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:55.939888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a6efb-373b-4796-bbb4-6f274f50a423 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:55.942201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a6efb-373b-4796-bbb4-6f274f50a423 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:55.943180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a6efb-373b-4796-bbb4-6f274f50a423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:20:55.954323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a6efb-373b-4796-bbb4-6f274f50a423 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:20:56.007298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36543703-8df3-41b6-aa6e-8d13b9df9014 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:20:56.008988 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3984/7911] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:20:56.030783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36543703-8df3-41b6-aa6e-8d13b9df9014 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:20:56.067275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2744ff9-f22e-47e8-bc2c-31790d241dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:20:56.068332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2744ff9-f22e-47e8-bc2c-31790d241dd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:56.068781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2744ff9-f22e-47e8-bc2c-31790d241dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:56.069913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2744ff9-f22e-47e8-bc2c-31790d241dd0 admin admin] PATH_INFO: `/v3/projects/a5e5603923f64ece8cb7a80f4e2dac66/users/299e38c0298743ab8096d3c9c70ad9b5/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:56.113471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2744ff9-f22e-47e8-bc2c-31790d241dd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5e5603923f64ece8cb7a80f4e2dac66, user_id=299e38c0298743ab8096d3c9c70ad9b5, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:20:56.116732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2744ff9-f22e-47e8-bc2c-31790d241dd0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:20:56.153410 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3928/7912] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:20:56 2022] PUT /identity/v3/projects/a5e5603923f64ece8cb7a80f4e2dac66/users/299e38c0298743ab8096d3c9c70ad9b5/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:20:56.158106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a6efb-373b-4796-bbb4-6f274f50a423 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:20:56.175466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a6efb-373b-4796-bbb4-6f274f50a423 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:20:56.301956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f7e3d-84c9-4932-9e47-57dd6947787b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:20:56.302834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f7e3d-84c9-4932-9e47-57dd6947787b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:56.303298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f7e3d-84c9-4932-9e47-57dd6947787b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:56.304679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f7e3d-84c9-4932-9e47-57dd6947787b admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:56.305642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66f7e3d-84c9-4932-9e47-57dd6947787b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:20:56.308664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66f7e3d-84c9-4932-9e47-57dd6947787b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:20:56.328580 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3985/7913] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:20:56.328908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2744ff9-f22e-47e8-bc2c-31790d241dd0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:20:56.348656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2744ff9-f22e-47e8-bc2c-31790d241dd0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:20:56.358854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3532e-1df1-435f-81f4-8fabeb679aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:20:56.362564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3532e-1df1-435f-81f4-8fabeb679aa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:56.362914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3532e-1df1-435f-81f4-8fabeb679aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:56.363478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3532e-1df1-435f-81f4-8fabeb679aa4 admin admin] PATH_INFO: `/v3/projects/a5e5603923f64ece8cb7a80f4e2dac66/users/299e38c0298743ab8096d3c9c70ad9b5/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:56.368160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3532e-1df1-435f-81f4-8fabeb679aa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5e5603923f64ece8cb7a80f4e2dac66, user_id=299e38c0298743ab8096d3c9c70ad9b5, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:20:56.372412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3532e-1df1-435f-81f4-8fabeb679aa4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:20:56.407129 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3929/7914] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:20:56 2022] PUT /identity/v3/projects/a5e5603923f64ece8cb7a80f4e2dac66/users/299e38c0298743ab8096d3c9c70ad9b5/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:20:59.301052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384601c-645a-4c65-a431-205716630d8f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:20:59.301891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384601c-645a-4c65-a431-205716630d8f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:20:59.302324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384601c-645a-4c65-a431-205716630d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:20:59.347699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b384601c-645a-4c65-a431-205716630d8f None None] MFA Rules not processed for user `f77486ef55a0469a99c3795b626bee1a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:20:59.349099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b384601c-645a-4c65-a431-205716630d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:20:59.511937 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3986/7915] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:20:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:01.102067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d803a-d89c-4ef2-956f-dc1eba86d77d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:01.103236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d803a-d89c-4ef2-956f-dc1eba86d77d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:01.103838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d803a-d89c-4ef2-956f-dc1eba86d77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:01.132597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f11d803a-d89c-4ef2-956f-dc1eba86d77d None None] MFA Rules not processed for user `6ac020dc323042c48ccce2a8826bc6be`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:01.139512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f11d803a-d89c-4ef2-956f-dc1eba86d77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:01.243451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:01.269638 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3930/7916] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:04.924908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384601c-645a-4c65-a431-205716630d8f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:04.926475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b384601c-645a-4c65-a431-205716630d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:05.085101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384601c-645a-4c65-a431-205716630d8f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:05.090721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bac7ca-2108-46db-8c50-d74c340880bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:05.091600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bac7ca-2108-46db-8c50-d74c340880bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:05.092056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bac7ca-2108-46db-8c50-d74c340880bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:05.092498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bac7ca-2108-46db-8c50-d74c340880bd admin admin] PATH_INFO: `/v3/users/af560738f1684441acb597d35753e24a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:05.094451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bac7ca-2108-46db-8c50-d74c340880bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=af560738f1684441acb597d35753e24a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:05.097574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bac7ca-2108-46db-8c50-d74c340880bd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:05.113021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:05.179453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-88bac7ca-2108-46db-8c50-d74c340880bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af560738f1684441acb597d35753e24a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:05.193091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-88bac7ca-2108-46db-8c50-d74c340880bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af560738f1684441acb597d35753e24a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:05.209435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-88bac7ca-2108-46db-8c50-d74c340880bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af560738f1684441acb597d35753e24a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:05.210465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-88bac7ca-2108-46db-8c50-d74c340880bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af560738f1684441acb597d35753e24a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:05.227133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-88bac7ca-2108-46db-8c50-d74c340880bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af560738f1684441acb597d35753e24a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:05.235048 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3987/7917] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:21:04 2022] DELETE /identity/v3/users/af560738f1684441acb597d35753e24a => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:05.239526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4826b-30b2-4744-bfcb-3774cba846cb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:05.240359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4826b-30b2-4744-bfcb-3774cba846cb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:05.241379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4826b-30b2-4744-bfcb-3774cba846cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:05.274665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0fd4826b-30b2-4744-bfcb-3774cba846cb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:05.277377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0fd4826b-30b2-4744-bfcb-3774cba846cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:05.377062 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3931/7918] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:05.388179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bac7ca-2108-46db-8c50-d74c340880bd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:05.391251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-88bac7ca-2108-46db-8c50-d74c340880bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:05.495092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bac7ca-2108-46db-8c50-d74c340880bd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:05.507624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596b530-ba1f-403f-91ba-aa8c0b07dff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:05.508704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596b530-ba1f-403f-91ba-aa8c0b07dff9 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:05.508992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596b530-ba1f-403f-91ba-aa8c0b07dff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:05.509259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596b530-ba1f-403f-91ba-aa8c0b07dff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:05.520368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596b530-ba1f-403f-91ba-aa8c0b07dff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:05.523824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596b530-ba1f-403f-91ba-aa8c0b07dff9 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:05.546949 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3988/7919] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:21:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:21:05.768809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4826b-30b2-4744-bfcb-3774cba846cb None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:05.773922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0fd4826b-30b2-4744-bfcb-3774cba846cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:05.797686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4826b-30b2-4744-bfcb-3774cba846cb None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:05.804858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33c154-681d-4ca3-b23e-d7396f4883c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:05.805810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33c154-681d-4ca3-b23e-d7396f4883c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:05.806672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33c154-681d-4ca3-b23e-d7396f4883c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:05.807127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33c154-681d-4ca3-b23e-d7396f4883c8 admin admin] PATH_INFO: `/v3/projects/f4fbc14980d642ab80d82945dcf337fb` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:05.808900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33c154-681d-4ca3-b23e-d7396f4883c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4fbc14980d642ab80d82945dcf337fb)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:05.811463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33c154-681d-4ca3-b23e-d7396f4883c8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:05.895601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1c33c154-681d-4ca3-b23e-d7396f4883c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4fbc14980d642ab80d82945dcf337fb', 'request_id': 'req-1c33c154-681d-4ca3-b23e-d7396f4883c8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:05.906003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1c33c154-681d-4ca3-b23e-d7396f4883c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4fbc14980d642ab80d82945dcf337fb', 'request_id': 'req-1c33c154-681d-4ca3-b23e-d7396f4883c8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:05.914014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1c33c154-681d-4ca3-b23e-d7396f4883c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4fbc14980d642ab80d82945dcf337fb', 'request_id': 'req-1c33c154-681d-4ca3-b23e-d7396f4883c8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:05.933374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1c33c154-681d-4ca3-b23e-d7396f4883c8 admin admin] The token cache is being invalidate because project f4fbc14980d642ab80d82945dcf337fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:21:05.933775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1c33c154-681d-4ca3-b23e-d7396f4883c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:05.934344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1c33c154-681d-4ca3-b23e-d7396f4883c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:05.941238 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3932/7920] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:21:05 2022] DELETE /identity/v3/projects/f4fbc14980d642ab80d82945dcf337fb => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:05.942481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc834d0-79d3-43aa-b8eb-8c845835d4be None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:05.942982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc834d0-79d3-43aa-b8eb-8c845835d4be None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:05.943287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc834d0-79d3-43aa-b8eb-8c845835d4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:05.995561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ddc834d0-79d3-43aa-b8eb-8c845835d4be None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:05.997788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ddc834d0-79d3-43aa-b8eb-8c845835d4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:06.114016 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3989/7921] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:06.121633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdcc9c-5340-4066-96df-13a9d37c8bf7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:06.122146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdcc9c-5340-4066-96df-13a9d37c8bf7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:06.122624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdcc9c-5340-4066-96df-13a9d37c8bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:06.144190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1acdcc9c-5340-4066-96df-13a9d37c8bf7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:06.145412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1acdcc9c-5340-4066-96df-13a9d37c8bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:06.157978 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3933/7922] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:06.159679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc834d0-79d3-43aa-b8eb-8c845835d4be None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:06.175560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc834d0-79d3-43aa-b8eb-8c845835d4be None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:06.186473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f951c-98b7-4c1e-adfb-9b780dff8b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:06.187636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f951c-98b7-4c1e-adfb-9b780dff8b6d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:06.188265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f951c-98b7-4c1e-adfb-9b780dff8b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:06.188714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f951c-98b7-4c1e-adfb-9b780dff8b6d admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:06.190974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f951c-98b7-4c1e-adfb-9b780dff8b6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:21:06.191780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f951c-98b7-4c1e-adfb-9b780dff8b6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:06.194739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f951c-98b7-4c1e-adfb-9b780dff8b6d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:06.237380 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3990/7923] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:21:06.238987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acdcc9c-5340-4066-96df-13a9d37c8bf7 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:06.253147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acdcc9c-5340-4066-96df-13a9d37c8bf7 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:06.263281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913af87d-0642-4a41-ac96-69a0f91eb802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:06.264545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913af87d-0642-4a41-ac96-69a0f91eb802 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:06.264839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913af87d-0642-4a41-ac96-69a0f91eb802 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:06.265118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913af87d-0642-4a41-ac96-69a0f91eb802 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:06.265764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913af87d-0642-4a41-ac96-69a0f91eb802 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:06.274854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913af87d-0642-4a41-ac96-69a0f91eb802 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:06.348590 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3934/7924] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:21:06 2022] POST /identity/v3/projects => generated 353 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:21:06.352544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f951c-98b7-4c1e-adfb-9b780dff8b6d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:06.385919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f951c-98b7-4c1e-adfb-9b780dff8b6d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:06.443615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:06.618755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71a068-d0a5-40ac-a5be-aa6bc5893b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:06.619390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71a068-d0a5-40ac-a5be-aa6bc5893b3d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:06.619675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71a068-d0a5-40ac-a5be-aa6bc5893b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:06.619955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71a068-d0a5-40ac-a5be-aa6bc5893b3d admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:06.620647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71a068-d0a5-40ac-a5be-aa6bc5893b3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:06.623444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71a068-d0a5-40ac-a5be-aa6bc5893b3d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:06.678817 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3991/7925] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:21:06 2022] POST /identity/v3/users => generated 277 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:21:06.682998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913af87d-0642-4a41-ac96-69a0f91eb802 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:06.698686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913af87d-0642-4a41-ac96-69a0f91eb802 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:06.703946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032d85d-bce2-434a-96af-f464f1996c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:06.704799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032d85d-bce2-434a-96af-f464f1996c16 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:06.705511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032d85d-bce2-434a-96af-f464f1996c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:06.705953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032d85d-bce2-434a-96af-f464f1996c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:06.706896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032d85d-bce2-434a-96af-f464f1996c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:06.709930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032d85d-bce2-434a-96af-f464f1996c16 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:06.737564 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3935/7926] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:06.738461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71a068-d0a5-40ac-a5be-aa6bc5893b3d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:06.756632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71a068-d0a5-40ac-a5be-aa6bc5893b3d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:06.766142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc377ccc-4311-4393-8494-ac4566bca2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:06.766731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc377ccc-4311-4393-8494-ac4566bca2af admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:06.767016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc377ccc-4311-4393-8494-ac4566bca2af admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:06.767292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc377ccc-4311-4393-8494-ac4566bca2af admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:06.767949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc377ccc-4311-4393-8494-ac4566bca2af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:06.769558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc377ccc-4311-4393-8494-ac4566bca2af admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:06.799476 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3992/7927] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:21:06.801271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032d85d-bce2-434a-96af-f464f1996c16 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:06.818602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032d85d-bce2-434a-96af-f464f1996c16 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:06.827590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5def4-e82e-4368-82b5-a3e71fb7a195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:06.829818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5def4-e82e-4368-82b5-a3e71fb7a195 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:06.831469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5def4-e82e-4368-82b5-a3e71fb7a195 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:06.835254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5def4-e82e-4368-82b5-a3e71fb7a195 admin admin] PATH_INFO: `/v3/projects/93e6926f4c0948849d50014b8a063ccc/users/631a64c9af58490bbeb268d0e3004597/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:06.867616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5def4-e82e-4368-82b5-a3e71fb7a195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93e6926f4c0948849d50014b8a063ccc, user_id=631a64c9af58490bbeb268d0e3004597, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:06.872313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5def4-e82e-4368-82b5-a3e71fb7a195 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:06.904772 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3936/7928] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:21:06 2022] PUT /identity/v3/projects/93e6926f4c0948849d50014b8a063ccc/users/631a64c9af58490bbeb268d0e3004597/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:06.916414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd34fe0-4b51-4e26-91a6-59fdf6708669 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:06.917140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd34fe0-4b51-4e26-91a6-59fdf6708669 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:06.917625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd34fe0-4b51-4e26-91a6-59fdf6708669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:06.984033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5fd34fe0-4b51-4e26-91a6-59fdf6708669 None None] MFA Rules not processed for user `631a64c9af58490bbeb268d0e3004597`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:06.986424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5fd34fe0-4b51-4e26-91a6-59fdf6708669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:07.135167 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3993/7929] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:06 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:07.152901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5def4-e82e-4368-82b5-a3e71fb7a195 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:07.155729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-11a5def4-e82e-4368-82b5-a3e71fb7a195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:07.177097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:07.312781 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5def4-e82e-4368-82b5-a3e71fb7a195 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:07.329381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452f30e-2cea-4993-8ab6-657f869af678 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:07.330146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452f30e-2cea-4993-8ab6-657f869af678 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:07.330645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452f30e-2cea-4993-8ab6-657f869af678 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:07.330800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452f30e-2cea-4993-8ab6-657f869af678 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:07.351569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452f30e-2cea-4993-8ab6-657f869af678 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:07.353312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452f30e-2cea-4993-8ab6-657f869af678 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:07.375976 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3937/7930] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:21:07 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:21:07.773010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd34fe0-4b51-4e26-91a6-59fdf6708669 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:07.775358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5fd34fe0-4b51-4e26-91a6-59fdf6708669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:07.953451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd34fe0-4b51-4e26-91a6-59fdf6708669 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:07.960881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297077f9-6e3c-4c51-9c71-b36c02ad4079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:07.962973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297077f9-6e3c-4c51-9c71-b36c02ad4079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:07.976017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297077f9-6e3c-4c51-9c71-b36c02ad4079 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:07.976356 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297077f9-6e3c-4c51-9c71-b36c02ad4079 admin admin] PATH_INFO: `/v3/users/c97a8a29cea043f5878205407dafc100` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:07.979823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297077f9-6e3c-4c51-9c71-b36c02ad4079 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c97a8a29cea043f5878205407dafc100)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:07.997828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297077f9-6e3c-4c51-9c71-b36c02ad4079 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:08.085738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:08.154343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-297077f9-6e3c-4c51-9c71-b36c02ad4079 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c97a8a29cea043f5878205407dafc100'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:08.166703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-297077f9-6e3c-4c51-9c71-b36c02ad4079 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c97a8a29cea043f5878205407dafc100'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:08.205091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-297077f9-6e3c-4c51-9c71-b36c02ad4079 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c97a8a29cea043f5878205407dafc100'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:08.207290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-297077f9-6e3c-4c51-9c71-b36c02ad4079 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c97a8a29cea043f5878205407dafc100'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:08.225810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-297077f9-6e3c-4c51-9c71-b36c02ad4079 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c97a8a29cea043f5878205407dafc100'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:08.234138 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3994/7931] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:21:07 2022] DELETE /identity/v3/users/c97a8a29cea043f5878205407dafc100 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:08.762002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452f30e-2cea-4993-8ab6-657f869af678 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:08.776836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452f30e-2cea-4993-8ab6-657f869af678 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:08.913426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3b35c-a116-48fc-95c5-bedf868c6f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:08.914263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3b35c-a116-48fc-95c5-bedf868c6f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:08.918478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3b35c-a116-48fc-95c5-bedf868c6f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:08.918918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3b35c-a116-48fc-95c5-bedf868c6f45 admin admin] PATH_INFO: `/v3/projects/45d45087fbd24f2c90470e455d620178` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:08.920708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3b35c-a116-48fc-95c5-bedf868c6f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45d45087fbd24f2c90470e455d620178)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:08.925262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3b35c-a116-48fc-95c5-bedf868c6f45 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:09.025200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9bc3b35c-a116-48fc-95c5-bedf868c6f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45d45087fbd24f2c90470e455d620178', 'request_id': 'req-9bc3b35c-a116-48fc-95c5-bedf868c6f45'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:09.038377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9bc3b35c-a116-48fc-95c5-bedf868c6f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45d45087fbd24f2c90470e455d620178', 'request_id': 'req-9bc3b35c-a116-48fc-95c5-bedf868c6f45'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:09.048192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9bc3b35c-a116-48fc-95c5-bedf868c6f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45d45087fbd24f2c90470e455d620178', 'request_id': 'req-9bc3b35c-a116-48fc-95c5-bedf868c6f45'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:09.061795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9bc3b35c-a116-48fc-95c5-bedf868c6f45 admin admin] The token cache is being invalidate because project 45d45087fbd24f2c90470e455d620178 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:21:09.062458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9bc3b35c-a116-48fc-95c5-bedf868c6f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:09.063453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9bc3b35c-a116-48fc-95c5-bedf868c6f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:09.065729 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3938/7932] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:21:08 2022] DELETE /identity/v3/projects/45d45087fbd24f2c90470e455d620178 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:09.073647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297077f9-6e3c-4c51-9c71-b36c02ad4079 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:09.075469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-297077f9-6e3c-4c51-9c71-b36c02ad4079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:09.091756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297077f9-6e3c-4c51-9c71-b36c02ad4079 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:09.166115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66908b5c-6fc8-44d9-9dbd-606771f5e0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:09.166800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66908b5c-6fc8-44d9-9dbd-606771f5e0bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:09.167424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66908b5c-6fc8-44d9-9dbd-606771f5e0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:09.167912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66908b5c-6fc8-44d9-9dbd-606771f5e0bf admin admin] PATH_INFO: `/v3/users/bbe6040ce96544c0a301331101db7267` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:09.169835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66908b5c-6fc8-44d9-9dbd-606771f5e0bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbe6040ce96544c0a301331101db7267)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:09.172126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66908b5c-6fc8-44d9-9dbd-606771f5e0bf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:09.228298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-66908b5c-6fc8-44d9-9dbd-606771f5e0bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbe6040ce96544c0a301331101db7267'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:09.251252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-66908b5c-6fc8-44d9-9dbd-606771f5e0bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbe6040ce96544c0a301331101db7267'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:09.267157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-66908b5c-6fc8-44d9-9dbd-606771f5e0bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbe6040ce96544c0a301331101db7267'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:09.268785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-66908b5c-6fc8-44d9-9dbd-606771f5e0bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbe6040ce96544c0a301331101db7267'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:09.280480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-66908b5c-6fc8-44d9-9dbd-606771f5e0bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbe6040ce96544c0a301331101db7267'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:09.296061 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3995/7933] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:21:09 2022] DELETE /identity/v3/users/bbe6040ce96544c0a301331101db7267 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:09.732387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3b35c-a116-48fc-95c5-bedf868c6f45 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:09.734043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9bc3b35c-a116-48fc-95c5-bedf868c6f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:09.754563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3b35c-a116-48fc-95c5-bedf868c6f45 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:09.800871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:09.823037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3db16-50a2-4556-88d9-4cffc75a86ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:09.823723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3db16-50a2-4556-88d9-4cffc75a86ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:09.824017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3db16-50a2-4556-88d9-4cffc75a86ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:09.824323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3db16-50a2-4556-88d9-4cffc75a86ef admin admin] PATH_INFO: `/v3/projects/9441e87384c5410bb11278bead2a3761` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:09.825476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3db16-50a2-4556-88d9-4cffc75a86ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=9441e87384c5410bb11278bead2a3761)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:09.827449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3db16-50a2-4556-88d9-4cffc75a86ef admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:09.883858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f4b3db16-50a2-4556-88d9-4cffc75a86ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9441e87384c5410bb11278bead2a3761', 'request_id': 'req-f4b3db16-50a2-4556-88d9-4cffc75a86ef'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:09.892044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f4b3db16-50a2-4556-88d9-4cffc75a86ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9441e87384c5410bb11278bead2a3761', 'request_id': 'req-f4b3db16-50a2-4556-88d9-4cffc75a86ef'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:09.904027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f4b3db16-50a2-4556-88d9-4cffc75a86ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9441e87384c5410bb11278bead2a3761', 'request_id': 'req-f4b3db16-50a2-4556-88d9-4cffc75a86ef'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:09.916257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f4b3db16-50a2-4556-88d9-4cffc75a86ef admin admin] The token cache is being invalidate because project 9441e87384c5410bb11278bead2a3761 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:21:09.916556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f4b3db16-50a2-4556-88d9-4cffc75a86ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:09.917673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-f4b3db16-50a2-4556-88d9-4cffc75a86ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:09.926573 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3939/7934] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:21:09 2022] DELETE /identity/v3/projects/9441e87384c5410bb11278bead2a3761 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:09.929715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dde70-f963-4d3d-bae3-6a15ae3c7695 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:09.930310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dde70-f963-4d3d-bae3-6a15ae3c7695 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:09.930714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dde70-f963-4d3d-bae3-6a15ae3c7695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:09.967491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-047dde70-f963-4d3d-bae3-6a15ae3c7695 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:09.968771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-047dde70-f963-4d3d-bae3-6a15ae3c7695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:09.991026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:10.081957 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3996/7935] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:21:10.090733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad2046-229c-459e-878a-4028bfedd28e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:10.091277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad2046-229c-459e-878a-4028bfedd28e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:10.091838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad2046-229c-459e-878a-4028bfedd28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:10.118712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-43ad2046-229c-459e-878a-4028bfedd28e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:10.122178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-43ad2046-229c-459e-878a-4028bfedd28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:10.133223 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3940/7936] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:10.137232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dde70-f963-4d3d-bae3-6a15ae3c7695 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:10.148256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dde70-f963-4d3d-bae3-6a15ae3c7695 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:10.153155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a3356-5336-419d-aea8-635d8e102db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:10.153776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a3356-5336-419d-aea8-635d8e102db5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:10.154293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a3356-5336-419d-aea8-635d8e102db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:10.154555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a3356-5336-419d-aea8-635d8e102db5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:10.155389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a3356-5336-419d-aea8-635d8e102db5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:21:10.156057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a3356-5336-419d-aea8-635d8e102db5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:10.159754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a3356-5336-419d-aea8-635d8e102db5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:10.179644 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3997/7937] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:21:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:21:10.183731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad2046-229c-459e-878a-4028bfedd28e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:10.197239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad2046-229c-459e-878a-4028bfedd28e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:10.206897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295ac79-5a64-457a-b526-881d9d90746a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:10.208334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295ac79-5a64-457a-b526-881d9d90746a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:10.208783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295ac79-5a64-457a-b526-881d9d90746a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:10.209778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295ac79-5a64-457a-b526-881d9d90746a admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:10.210866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295ac79-5a64-457a-b526-881d9d90746a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:10.214241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295ac79-5a64-457a-b526-881d9d90746a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:10.277886 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3941/7938] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:21:10 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:21:10.282031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a3356-5336-419d-aea8-635d8e102db5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:10.293808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a3356-5336-419d-aea8-635d8e102db5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:10.357065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024b2f1-d172-4d2e-b1e0-c62c2905b255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:10.357692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024b2f1-d172-4d2e-b1e0-c62c2905b255 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:10.358201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024b2f1-d172-4d2e-b1e0-c62c2905b255 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:10.358717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024b2f1-d172-4d2e-b1e0-c62c2905b255 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:10.359441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024b2f1-d172-4d2e-b1e0-c62c2905b255 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:10.361625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024b2f1-d172-4d2e-b1e0-c62c2905b255 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:10.389136 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3998/7939] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:21:10 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:21:10.393261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295ac79-5a64-457a-b526-881d9d90746a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:10.405455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295ac79-5a64-457a-b526-881d9d90746a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:10.411201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4364d0b-ecae-49ea-957b-3002b4b0f1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:10.411837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4364d0b-ecae-49ea-957b-3002b4b0f1bf admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:10.412270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4364d0b-ecae-49ea-957b-3002b4b0f1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:10.412592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4364d0b-ecae-49ea-957b-3002b4b0f1bf admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:10.413251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4364d0b-ecae-49ea-957b-3002b4b0f1bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:10.415353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4364d0b-ecae-49ea-957b-3002b4b0f1bf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:10.427220 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3942/7940] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:10.445598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024b2f1-d172-4d2e-b1e0-c62c2905b255 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:10.461707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024b2f1-d172-4d2e-b1e0-c62c2905b255 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:10.467111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82810a-2580-4f53-91e1-17832f04d37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:10.467691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82810a-2580-4f53-91e1-17832f04d37e admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:10.468181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82810a-2580-4f53-91e1-17832f04d37e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:10.468653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82810a-2580-4f53-91e1-17832f04d37e admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:10.469216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb82810a-2580-4f53-91e1-17832f04d37e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:10.471043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb82810a-2580-4f53-91e1-17832f04d37e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:10.491549 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 3999/7941] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:10.496452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4364d0b-ecae-49ea-957b-3002b4b0f1bf admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:10.524877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4364d0b-ecae-49ea-957b-3002b4b0f1bf admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:10.537495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befba800-8cb7-452b-9d52-eaea327f4b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:10.538316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befba800-8cb7-452b-9d52-eaea327f4b89 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:10.538701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befba800-8cb7-452b-9d52-eaea327f4b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:10.539048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befba800-8cb7-452b-9d52-eaea327f4b89 admin admin] PATH_INFO: `/v3/projects/90cb48440cb446ea9abfc00ed0f53779/users/55dadd491ec1441aac4171eff46bd679/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:10.557442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befba800-8cb7-452b-9d52-eaea327f4b89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90cb48440cb446ea9abfc00ed0f53779, user_id=55dadd491ec1441aac4171eff46bd679, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:10.560546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befba800-8cb7-452b-9d52-eaea327f4b89 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:10.587291 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3943/7942] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:21:10 2022] PUT /identity/v3/projects/90cb48440cb446ea9abfc00ed0f53779/users/55dadd491ec1441aac4171eff46bd679/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:21:10.592736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71926d57-5e39-4b8d-bb45-19d2ff624057 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:10.593136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71926d57-5e39-4b8d-bb45-19d2ff624057 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:10.593482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71926d57-5e39-4b8d-bb45-19d2ff624057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:10.622911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-71926d57-5e39-4b8d-bb45-19d2ff624057 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:10.624148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-71926d57-5e39-4b8d-bb45-19d2ff624057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:10.701912 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4000/7943] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:10.713358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befba800-8cb7-452b-9d52-eaea327f4b89 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:10.715202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-befba800-8cb7-452b-9d52-eaea327f4b89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:10.789239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befba800-8cb7-452b-9d52-eaea327f4b89 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:10.793302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b4125-c551-4cc3-812a-9046ebde1ce8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:10.793939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b4125-c551-4cc3-812a-9046ebde1ce8 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:10.794226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b4125-c551-4cc3-812a-9046ebde1ce8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:10.794793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b4125-c551-4cc3-812a-9046ebde1ce8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:10.806566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b4125-c551-4cc3-812a-9046ebde1ce8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:10.808056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b4125-c551-4cc3-812a-9046ebde1ce8 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:10.820202 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3944/7944] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:21:11.219930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71926d57-5e39-4b8d-bb45-19d2ff624057 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:11.224211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-71926d57-5e39-4b8d-bb45-19d2ff624057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:11.270691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71926d57-5e39-4b8d-bb45-19d2ff624057 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:11.279435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0875b3a-b64e-4f7f-addd-80125ab07ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:11.280851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0875b3a-b64e-4f7f-addd-80125ab07ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:11.281264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0875b3a-b64e-4f7f-addd-80125ab07ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:11.282803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0875b3a-b64e-4f7f-addd-80125ab07ffc admin admin] PATH_INFO: `/v3/users/631a64c9af58490bbeb268d0e3004597` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:11.287878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0875b3a-b64e-4f7f-addd-80125ab07ffc admin admin] RBAC: Authorizing `identity:delete_user(user_id=631a64c9af58490bbeb268d0e3004597)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:11.290999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0875b3a-b64e-4f7f-addd-80125ab07ffc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:11.347275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:11.376058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b0875b3a-b64e-4f7f-addd-80125ab07ffc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '631a64c9af58490bbeb268d0e3004597'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:11.385457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b0875b3a-b64e-4f7f-addd-80125ab07ffc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '631a64c9af58490bbeb268d0e3004597'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:11.401636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b0875b3a-b64e-4f7f-addd-80125ab07ffc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '631a64c9af58490bbeb268d0e3004597'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:11.402913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b0875b3a-b64e-4f7f-addd-80125ab07ffc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '631a64c9af58490bbeb268d0e3004597'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:11.413435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b0875b3a-b64e-4f7f-addd-80125ab07ffc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '631a64c9af58490bbeb268d0e3004597'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:11.422681 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4001/7945] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:21:11 2022] DELETE /identity/v3/users/631a64c9af58490bbeb268d0e3004597 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:11.427395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01bd51-e71d-49ad-8ce9-d1f7ec6f06c3 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:11.428575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01bd51-e71d-49ad-8ce9-d1f7ec6f06c3 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:11.428808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01bd51-e71d-49ad-8ce9-d1f7ec6f06c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:11.454200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7e01bd51-e71d-49ad-8ce9-d1f7ec6f06c3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:11.456177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7e01bd51-e71d-49ad-8ce9-d1f7ec6f06c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:11.463099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:11.559634 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3945/7946] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:11.661465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0875b3a-b64e-4f7f-addd-80125ab07ffc admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:11.663097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b0875b3a-b64e-4f7f-addd-80125ab07ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:11.747367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0875b3a-b64e-4f7f-addd-80125ab07ffc admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:11.751947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa244c-2921-4920-985e-a901ef7399d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:11.752194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa244c-2921-4920-985e-a901ef7399d0 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:11.752481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa244c-2921-4920-985e-a901ef7399d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:11.752931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa244c-2921-4920-985e-a901ef7399d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:11.766393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa244c-2921-4920-985e-a901ef7399d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:11.768429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa244c-2921-4920-985e-a901ef7399d0 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:11.783940 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4002/7947] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:21:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:21:12.499811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01bd51-e71d-49ad-8ce9-d1f7ec6f06c3 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:12.501907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7e01bd51-e71d-49ad-8ce9-d1f7ec6f06c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:12.515033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01bd51-e71d-49ad-8ce9-d1f7ec6f06c3 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:12.520549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcae3d-f402-49a8-80e6-aa2c83fe7e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:12.521656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcae3d-f402-49a8-80e6-aa2c83fe7e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:12.522107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcae3d-f402-49a8-80e6-aa2c83fe7e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:12.522645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcae3d-f402-49a8-80e6-aa2c83fe7e11 admin admin] PATH_INFO: `/v3/projects/93e6926f4c0948849d50014b8a063ccc` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:12.524004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcae3d-f402-49a8-80e6-aa2c83fe7e11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93e6926f4c0948849d50014b8a063ccc)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:12.526659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcae3d-f402-49a8-80e6-aa2c83fe7e11 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:12.613486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9bbcae3d-f402-49a8-80e6-aa2c83fe7e11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93e6926f4c0948849d50014b8a063ccc', 'request_id': 'req-9bbcae3d-f402-49a8-80e6-aa2c83fe7e11'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:12.640796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9bbcae3d-f402-49a8-80e6-aa2c83fe7e11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93e6926f4c0948849d50014b8a063ccc', 'request_id': 'req-9bbcae3d-f402-49a8-80e6-aa2c83fe7e11'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:12.650149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9bbcae3d-f402-49a8-80e6-aa2c83fe7e11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93e6926f4c0948849d50014b8a063ccc', 'request_id': 'req-9bbcae3d-f402-49a8-80e6-aa2c83fe7e11'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:12.667118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9bbcae3d-f402-49a8-80e6-aa2c83fe7e11 admin admin] The token cache is being invalidate because project 93e6926f4c0948849d50014b8a063ccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:21:12.667413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9bbcae3d-f402-49a8-80e6-aa2c83fe7e11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:12.668416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9bbcae3d-f402-49a8-80e6-aa2c83fe7e11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:12.670355 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3946/7948] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:21:12 2022] DELETE /identity/v3/projects/93e6926f4c0948849d50014b8a063ccc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:12.675793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b958958-a883-48e6-9c5c-0d17794dc3c9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:12.676404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b958958-a883-48e6-9c5c-0d17794dc3c9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:12.677435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b958958-a883-48e6-9c5c-0d17794dc3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:12.712240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6b958958-a883-48e6-9c5c-0d17794dc3c9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:12.718066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6b958958-a883-48e6-9c5c-0d17794dc3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:12.848722 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4003/7949] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:12.857444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205a163-773a-4fe7-ae6e-471fb33fc53e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:12.857728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205a163-773a-4fe7-ae6e-471fb33fc53e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:12.858007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205a163-773a-4fe7-ae6e-471fb33fc53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:12.906029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e205a163-773a-4fe7-ae6e-471fb33fc53e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:12.907897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e205a163-773a-4fe7-ae6e-471fb33fc53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:12.921041 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3947/7950] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:12.925422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b958958-a883-48e6-9c5c-0d17794dc3c9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:12.943051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b958958-a883-48e6-9c5c-0d17794dc3c9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:12.955430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037fad5-c379-453f-b5c2-c74e26bf2079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:12.956314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037fad5-c379-453f-b5c2-c74e26bf2079 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:12.956692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037fad5-c379-453f-b5c2-c74e26bf2079 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:12.957194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037fad5-c379-453f-b5c2-c74e26bf2079 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:12.958019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037fad5-c379-453f-b5c2-c74e26bf2079 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:21:12.958366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037fad5-c379-453f-b5c2-c74e26bf2079 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:12.960765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037fad5-c379-453f-b5c2-c74e26bf2079 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:13.000551 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4004/7951] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:21:13.002625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205a163-773a-4fe7-ae6e-471fb33fc53e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:13.032851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205a163-773a-4fe7-ae6e-471fb33fc53e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:13.038405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48973639-7ce0-4294-8672-cb41450ce4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:13.039068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48973639-7ce0-4294-8672-cb41450ce4e4 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:13.039361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48973639-7ce0-4294-8672-cb41450ce4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:13.045329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48973639-7ce0-4294-8672-cb41450ce4e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:13.045329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48973639-7ce0-4294-8672-cb41450ce4e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:13.045329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48973639-7ce0-4294-8672-cb41450ce4e4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:13.076981 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3948/7952] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:21:13 2022] POST /identity/v3/projects => generated 383 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:21:13.082406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037fad5-c379-453f-b5c2-c74e26bf2079 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:13.093207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6c23e-8829-43c9-b9f0-490c35495646 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:13.093503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6c23e-8829-43c9-b9f0-490c35495646 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:13.093782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6c23e-8829-43c9-b9f0-490c35495646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:13.107580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037fad5-c379-453f-b5c2-c74e26bf2079 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:13.133898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-36c6c23e-8829-43c9-b9f0-490c35495646 None None] MFA Rules not processed for user `299e38c0298743ab8096d3c9c70ad9b5`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:13.135434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-36c6c23e-8829-43c9-b9f0-490c35495646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:13.216952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:13.270428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe3279-bdb3-4341-b15f-4e0d0dd2e77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:13.272208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe3279-bdb3-4341-b15f-4e0d0dd2e77d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:13.272639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe3279-bdb3-4341-b15f-4e0d0dd2e77d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:13.273035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe3279-bdb3-4341-b15f-4e0d0dd2e77d admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:13.275228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:13.275466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe3279-bdb3-4341-b15f-4e0d0dd2e77d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:13.278787 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe3279-bdb3-4341-b15f-4e0d0dd2e77d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:13.319466 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4005/7953] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:21:13 2022] POST /identity/v3/users => generated 292 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:21:13.320953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe3279-bdb3-4341-b15f-4e0d0dd2e77d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:13.328776 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3949/7954] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:13.346618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6c23e-8829-43c9-b9f0-490c35495646 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:13.348338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-36c6c23e-8829-43c9-b9f0-490c35495646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:13.355231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe3279-bdb3-4341-b15f-4e0d0dd2e77d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:13.396994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8653847-9be7-4a09-9e00-bf481c9250e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:13.397281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8653847-9be7-4a09-9e00-bf481c9250e8 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:13.397402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8653847-9be7-4a09-9e00-bf481c9250e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:13.397666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8653847-9be7-4a09-9e00-bf481c9250e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:13.400519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8653847-9be7-4a09-9e00-bf481c9250e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:13.406142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8653847-9be7-4a09-9e00-bf481c9250e8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:13.437070 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4006/7955] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:13.446417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8653847-9be7-4a09-9e00-bf481c9250e8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:13.478456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8653847-9be7-4a09-9e00-bf481c9250e8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:13.485630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11208d9a-6543-4372-8df5-2a15d0ee2532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:13.487200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11208d9a-6543-4372-8df5-2a15d0ee2532 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:13.487682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11208d9a-6543-4372-8df5-2a15d0ee2532 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:13.488203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11208d9a-6543-4372-8df5-2a15d0ee2532 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:13.489720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11208d9a-6543-4372-8df5-2a15d0ee2532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:13.499200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11208d9a-6543-4372-8df5-2a15d0ee2532 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:13.513804 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4007/7956] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:13.518802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11208d9a-6543-4372-8df5-2a15d0ee2532 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:13.526773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6c23e-8829-43c9-b9f0-490c35495646 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:13.534872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11208d9a-6543-4372-8df5-2a15d0ee2532 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:13.542174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7471af-22a3-4dc6-90d2-e91a74709d52 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:13.543052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7471af-22a3-4dc6-90d2-e91a74709d52 service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:13.543376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7471af-22a3-4dc6-90d2-e91a74709d52 service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:13.543740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7471af-22a3-4dc6-90d2-e91a74709d52 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:13.544648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4bc2e-8c21-49d1-8b19-b4fab966aaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:13.545447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4bc2e-8c21-49d1-8b19-b4fab966aaa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:13.545753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4bc2e-8c21-49d1-8b19-b4fab966aaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:13.546024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4bc2e-8c21-49d1-8b19-b4fab966aaa7 admin admin] PATH_INFO: `/v3/projects/453b7427926c421fa590be9814e291be/users/79df0d8b21af4f76b18c8de9c27c9b3f/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:13.547212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5c7471af-22a3-4dc6-90d2-e91a74709d52 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:13.563440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4bc2e-8c21-49d1-8b19-b4fab966aaa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=453b7427926c421fa590be9814e291be, user_id=79df0d8b21af4f76b18c8de9c27c9b3f, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:13.566208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4bc2e-8c21-49d1-8b19-b4fab966aaa7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:13.596925 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4008/7957] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:21:13 2022] PUT /identity/v3/projects/453b7427926c421fa590be9814e291be/users/79df0d8b21af4f76b18c8de9c27c9b3f/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:13.608188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2e76a-2670-4335-992a-62385ae1f4e3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:13.608662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2e76a-2670-4335-992a-62385ae1f4e3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:13.609127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2e76a-2670-4335-992a-62385ae1f4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:13.664666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7471af-22a3-4dc6-90d2-e91a74709d52 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:13.666284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7471af-22a3-4dc6-90d2-e91a74709d52 service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:13.671607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3ad2e76a-2670-4335-992a-62385ae1f4e3 None None] MFA Rules not processed for user `79df0d8b21af4f76b18c8de9c27c9b3f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:13.672835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3ad2e76a-2670-4335-992a-62385ae1f4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:13.678658 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3950/7958] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:21:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:21:13.786233 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4009/7959] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:21:13.791704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7471af-22a3-4dc6-90d2-e91a74709d52 service glance] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:13.813068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7471af-22a3-4dc6-90d2-e91a74709d52 service glance] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:13.945389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de79b60-c1a5-4298-9acc-e3ae38c767c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:13.946034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de79b60-c1a5-4298-9acc-e3ae38c767c4 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:13.946322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de79b60-c1a5-4298-9acc-e3ae38c767c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:13.946628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de79b60-c1a5-4298-9acc-e3ae38c767c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:13.947421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de79b60-c1a5-4298-9acc-e3ae38c767c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:13.950672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de79b60-c1a5-4298-9acc-e3ae38c767c4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:14.010549 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3951/7960] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:21:13 2022] POST /identity/v3/projects => generated 387 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:21:14.011899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad2e76a-2670-4335-992a-62385ae1f4e3 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:14.035006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad2e76a-2670-4335-992a-62385ae1f4e3 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:14.158300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706dc0f1-1ba8-4f35-8a22-369cd0772da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:14.158974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dc0f1-1ba8-4f35-8a22-369cd0772da9 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:14.159304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dc0f1-1ba8-4f35-8a22-369cd0772da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:14.159528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dc0f1-1ba8-4f35-8a22-369cd0772da9 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:14.160177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706dc0f1-1ba8-4f35-8a22-369cd0772da9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:14.164104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706dc0f1-1ba8-4f35-8a22-369cd0772da9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:14.191505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:14.227815 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4010/7961] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:21:14 2022] POST /identity/v3/users => generated 300 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:21:14.231608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de79b60-c1a5-4298-9acc-e3ae38c767c4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:14.246609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de79b60-c1a5-4298-9acc-e3ae38c767c4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:14.256290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb2449-711d-4bda-9a4f-8712679d74a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:14.257736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb2449-711d-4bda-9a4f-8712679d74a2 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:14.258026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb2449-711d-4bda-9a4f-8712679d74a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:14.258308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb2449-711d-4bda-9a4f-8712679d74a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:14.258992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb2449-711d-4bda-9a4f-8712679d74a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:14.263947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb2449-711d-4bda-9a4f-8712679d74a2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:14.299980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706dc0f1-1ba8-4f35-8a22-369cd0772da9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:14.307030 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3952/7962] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:14.326553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706dc0f1-1ba8-4f35-8a22-369cd0772da9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:14.346458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0fbed-f65f-468b-a712-d62efbea8f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:14.348320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0fbed-f65f-468b-a712-d62efbea8f2b admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:14.348608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0fbed-f65f-468b-a712-d62efbea8f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:14.350536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0fbed-f65f-468b-a712-d62efbea8f2b admin admin] PATH_INFO: `/v3/projects/a41472740df14c7fb3f9e78f82bdf7d5/users/16b024a712bf4757b134ee2040b772c6/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:14.390840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0fbed-f65f-468b-a712-d62efbea8f2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a41472740df14c7fb3f9e78f82bdf7d5, user_id=16b024a712bf4757b134ee2040b772c6, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:14.394210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0fbed-f65f-468b-a712-d62efbea8f2b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:14.423299 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4011/7963] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:21:14 2022] PUT /identity/v3/projects/a41472740df14c7fb3f9e78f82bdf7d5/users/16b024a712bf4757b134ee2040b772c6/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:14.424630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb2449-711d-4bda-9a4f-8712679d74a2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:14.444904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb2449-711d-4bda-9a4f-8712679d74a2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:14.474973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:14.644564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68416b79-eee0-4a4e-8588-8cf3c7413cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:14.645247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68416b79-eee0-4a4e-8588-8cf3c7413cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:14.645457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68416b79-eee0-4a4e-8588-8cf3c7413cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:14.645720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68416b79-eee0-4a4e-8588-8cf3c7413cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:14.646738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68416b79-eee0-4a4e-8588-8cf3c7413cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:14.654381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68416b79-eee0-4a4e-8588-8cf3c7413cc5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:14.691079 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3953/7964] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:14.695966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0fbed-f65f-468b-a712-d62efbea8f2b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:14.716230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0fbed-f65f-468b-a712-d62efbea8f2b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:14.737286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6add3-36cd-4ab0-9446-7e5803524d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:14.738657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6add3-36cd-4ab0-9446-7e5803524d09 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:14.738947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6add3-36cd-4ab0-9446-7e5803524d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:14.739255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6add3-36cd-4ab0-9446-7e5803524d09 admin admin] PATH_INFO: `/v3/projects/a41472740df14c7fb3f9e78f82bdf7d5/users/16b024a712bf4757b134ee2040b772c6/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:14.744418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e6add3-36cd-4ab0-9446-7e5803524d09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a41472740df14c7fb3f9e78f82bdf7d5, user_id=16b024a712bf4757b134ee2040b772c6, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:14.757605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e6add3-36cd-4ab0-9446-7e5803524d09 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:14.793039 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4012/7965] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:21:14 2022] PUT /identity/v3/projects/a41472740df14c7fb3f9e78f82bdf7d5/users/16b024a712bf4757b134ee2040b772c6/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:14.911563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285869fa-c443-46da-9d1f-2303a9ff9174 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:14.911859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285869fa-c443-46da-9d1f-2303a9ff9174 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:14.913119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285869fa-c443-46da-9d1f-2303a9ff9174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:14.968188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-285869fa-c443-46da-9d1f-2303a9ff9174 None None] MFA Rules not processed for user `16b024a712bf4757b134ee2040b772c6`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:14.969456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-285869fa-c443-46da-9d1f-2303a9ff9174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:15.165173 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3954/7966] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:14 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:15.174233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6add3-36cd-4ab0-9446-7e5803524d09 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:15.178744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-22e6add3-36cd-4ab0-9446-7e5803524d09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:15.325599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6add3-36cd-4ab0-9446-7e5803524d09 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:15.337922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec611344-b279-4582-8353-b85f13c4ea2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:15.338714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec611344-b279-4582-8353-b85f13c4ea2c service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:15.339004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec611344-b279-4582-8353-b85f13c4ea2c service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:15.339281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec611344-b279-4582-8353-b85f13c4ea2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:15.359924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec611344-b279-4582-8353-b85f13c4ea2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:15.361448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec611344-b279-4582-8353-b85f13c4ea2c service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:15.383737 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4013/7967] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:21:15 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:21:15.673478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285869fa-c443-46da-9d1f-2303a9ff9174 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:15.807334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285869fa-c443-46da-9d1f-2303a9ff9174 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:15.813433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c0f2f-6fe6-4269-a369-307126231a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:15.814457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c0f2f-6fe6-4269-a369-307126231a2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:15.814790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c0f2f-6fe6-4269-a369-307126231a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:15.815118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c0f2f-6fe6-4269-a369-307126231a2f admin admin] PATH_INFO: `/v3/users/79df0d8b21af4f76b18c8de9c27c9b3f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:15.816637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c0f2f-6fe6-4269-a369-307126231a2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=79df0d8b21af4f76b18c8de9c27c9b3f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:15.818980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c0f2f-6fe6-4269-a369-307126231a2f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:15.923512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:15.940254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ba0c0f2f-6fe6-4269-a369-307126231a2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79df0d8b21af4f76b18c8de9c27c9b3f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:15.942915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ba0c0f2f-6fe6-4269-a369-307126231a2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79df0d8b21af4f76b18c8de9c27c9b3f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:15.945722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ba0c0f2f-6fe6-4269-a369-307126231a2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79df0d8b21af4f76b18c8de9c27c9b3f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:15.971443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ba0c0f2f-6fe6-4269-a369-307126231a2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79df0d8b21af4f76b18c8de9c27c9b3f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:16.009312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ba0c0f2f-6fe6-4269-a369-307126231a2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79df0d8b21af4f76b18c8de9c27c9b3f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:16.050349 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3955/7968] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:21:15 2022] DELETE /identity/v3/users/79df0d8b21af4f76b18c8de9c27c9b3f => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:16.059836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb8bd0-42c6-4e58-835a-e18f45642a89 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:16.062147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb8bd0-42c6-4e58-835a-e18f45642a89 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:16.063842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb8bd0-42c6-4e58-835a-e18f45642a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:16.093527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fdbb8bd0-42c6-4e58-835a-e18f45642a89 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:16.095270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fdbb8bd0-42c6-4e58-835a-e18f45642a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:16.253707 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4014/7969] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:16.264990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c0f2f-6fe6-4269-a369-307126231a2f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:16.267230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ba0c0f2f-6fe6-4269-a369-307126231a2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:16.387402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c0f2f-6fe6-4269-a369-307126231a2f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:16.393026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1da79-489e-4e62-bcff-c22d329e6f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:16.393739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1da79-489e-4e62-bcff-c22d329e6f90 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:16.394141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1da79-489e-4e62-bcff-c22d329e6f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:16.395125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1da79-489e-4e62-bcff-c22d329e6f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:16.411379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1da79-489e-4e62-bcff-c22d329e6f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:16.413251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1da79-489e-4e62-bcff-c22d329e6f90 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:16.453163 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3956/7970] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:21:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:21:17.289387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb8bd0-42c6-4e58-835a-e18f45642a89 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:17.291162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fdbb8bd0-42c6-4e58-835a-e18f45642a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:17.310602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb8bd0-42c6-4e58-835a-e18f45642a89 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:17.339398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d16828-3e79-4b5a-9853-790ec85f9836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:17.341129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d16828-3e79-4b5a-9853-790ec85f9836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:17.342023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d16828-3e79-4b5a-9853-790ec85f9836 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:17.342929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d16828-3e79-4b5a-9853-790ec85f9836 admin admin] PATH_INFO: `/v3/projects/453b7427926c421fa590be9814e291be` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:17.344419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d16828-3e79-4b5a-9853-790ec85f9836 admin admin] RBAC: Authorizing `identity:delete_project(project_id=453b7427926c421fa590be9814e291be)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:17.346648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d16828-3e79-4b5a-9853-790ec85f9836 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:17.456217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-70d16828-3e79-4b5a-9853-790ec85f9836 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '453b7427926c421fa590be9814e291be', 'request_id': 'req-70d16828-3e79-4b5a-9853-790ec85f9836'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:17.464783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-70d16828-3e79-4b5a-9853-790ec85f9836 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '453b7427926c421fa590be9814e291be', 'request_id': 'req-70d16828-3e79-4b5a-9853-790ec85f9836'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:17.467620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:17.472663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-70d16828-3e79-4b5a-9853-790ec85f9836 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '453b7427926c421fa590be9814e291be', 'request_id': 'req-70d16828-3e79-4b5a-9853-790ec85f9836'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:17.492635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-70d16828-3e79-4b5a-9853-790ec85f9836 admin admin] The token cache is being invalidate because project 453b7427926c421fa590be9814e291be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:21:17.492936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-70d16828-3e79-4b5a-9853-790ec85f9836 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:17.495040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-70d16828-3e79-4b5a-9853-790ec85f9836 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:17.498154 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4015/7971] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:21:17 2022] DELETE /identity/v3/projects/453b7427926c421fa590be9814e291be => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:17.502125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1da79-489e-4e62-bcff-c22d329e6f90 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:17.515997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1da79-489e-4e62-bcff-c22d329e6f90 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:17.638081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab757f6-7ab5-480a-ba3d-56a8e0e2e8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:17.638930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab757f6-7ab5-480a-ba3d-56a8e0e2e8ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:17.640300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab757f6-7ab5-480a-ba3d-56a8e0e2e8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:17.640794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab757f6-7ab5-480a-ba3d-56a8e0e2e8ea admin admin] PATH_INFO: `/v3/users/16b024a712bf4757b134ee2040b772c6` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:17.642473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab757f6-7ab5-480a-ba3d-56a8e0e2e8ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=16b024a712bf4757b134ee2040b772c6)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:17.645105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab757f6-7ab5-480a-ba3d-56a8e0e2e8ea admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:17.760735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2ab757f6-7ab5-480a-ba3d-56a8e0e2e8ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16b024a712bf4757b134ee2040b772c6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:17.761644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2ab757f6-7ab5-480a-ba3d-56a8e0e2e8ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16b024a712bf4757b134ee2040b772c6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:17.762093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2ab757f6-7ab5-480a-ba3d-56a8e0e2e8ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16b024a712bf4757b134ee2040b772c6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:17.802485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2ab757f6-7ab5-480a-ba3d-56a8e0e2e8ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16b024a712bf4757b134ee2040b772c6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:17.814584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:17.846696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2ab757f6-7ab5-480a-ba3d-56a8e0e2e8ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16b024a712bf4757b134ee2040b772c6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:17.863667 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3957/7972] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:21:17 2022] DELETE /identity/v3/users/16b024a712bf4757b134ee2040b772c6 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:18.625486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d16828-3e79-4b5a-9853-790ec85f9836 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:18.627239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-70d16828-3e79-4b5a-9853-790ec85f9836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:18.654371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d16828-3e79-4b5a-9853-790ec85f9836 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:18.745560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2322b79-e5b7-4004-9f93-ef4325aeeadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:18.746702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2322b79-e5b7-4004-9f93-ef4325aeeadc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:18.747152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2322b79-e5b7-4004-9f93-ef4325aeeadc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:18.747583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2322b79-e5b7-4004-9f93-ef4325aeeadc admin admin] PATH_INFO: `/v3/projects/a41472740df14c7fb3f9e78f82bdf7d5` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:18.749090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2322b79-e5b7-4004-9f93-ef4325aeeadc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a41472740df14c7fb3f9e78f82bdf7d5)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:18.753036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2322b79-e5b7-4004-9f93-ef4325aeeadc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:18.820932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a2322b79-e5b7-4004-9f93-ef4325aeeadc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a41472740df14c7fb3f9e78f82bdf7d5', 'request_id': 'req-a2322b79-e5b7-4004-9f93-ef4325aeeadc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:18.829095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a2322b79-e5b7-4004-9f93-ef4325aeeadc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a41472740df14c7fb3f9e78f82bdf7d5', 'request_id': 'req-a2322b79-e5b7-4004-9f93-ef4325aeeadc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:18.837702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a2322b79-e5b7-4004-9f93-ef4325aeeadc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a41472740df14c7fb3f9e78f82bdf7d5', 'request_id': 'req-a2322b79-e5b7-4004-9f93-ef4325aeeadc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:18.859568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a2322b79-e5b7-4004-9f93-ef4325aeeadc admin admin] The token cache is being invalidate because project a41472740df14c7fb3f9e78f82bdf7d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:21:18.859904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a2322b79-e5b7-4004-9f93-ef4325aeeadc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:18.860550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a2322b79-e5b7-4004-9f93-ef4325aeeadc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:18.862929 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4016/7973] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:21:18 2022] DELETE /identity/v3/projects/a41472740df14c7fb3f9e78f82bdf7d5 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:18.868139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23979ce-95a3-4487-b7dc-9d6725693953 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:18.868654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23979ce-95a3-4487-b7dc-9d6725693953 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:18.868958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23979ce-95a3-4487-b7dc-9d6725693953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:18.903549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e23979ce-95a3-4487-b7dc-9d6725693953 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:18.908440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e23979ce-95a3-4487-b7dc-9d6725693953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:19.011991 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3958/7974] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:19.020253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af05ea1-75f1-498a-969c-22b9fe7f58b3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:19.020675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af05ea1-75f1-498a-969c-22b9fe7f58b3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:19.021021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af05ea1-75f1-498a-969c-22b9fe7f58b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:19.067958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2af05ea1-75f1-498a-969c-22b9fe7f58b3 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:19.070054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2af05ea1-75f1-498a-969c-22b9fe7f58b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:19.080366 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4017/7975] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:19.084789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23979ce-95a3-4487-b7dc-9d6725693953 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:19.096095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23979ce-95a3-4487-b7dc-9d6725693953 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:19.101683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c72767-0c26-4d49-87cc-77c1addfd876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:19.103288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c72767-0c26-4d49-87cc-77c1addfd876 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:19.103554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c72767-0c26-4d49-87cc-77c1addfd876 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:19.107289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c72767-0c26-4d49-87cc-77c1addfd876 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:19.108021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c72767-0c26-4d49-87cc-77c1addfd876 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:21:19.108531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c72767-0c26-4d49-87cc-77c1addfd876 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:19.110484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c72767-0c26-4d49-87cc-77c1addfd876 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:19.140237 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3959/7976] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:21:19.140487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0550ef-0442-4d86-b02c-cd061bbb4b97 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:19.140487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0550ef-0442-4d86-b02c-cd061bbb4b97 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:19.140487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0550ef-0442-4d86-b02c-cd061bbb4b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:19.143024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c72767-0c26-4d49-87cc-77c1addfd876 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:19.154315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c72767-0c26-4d49-87cc-77c1addfd876 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:19.165996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694eaf53-4bc2-4e37-acb2-48a9b4b04427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:19.166375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694eaf53-4bc2-4e37-acb2-48a9b4b04427 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:19.166719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694eaf53-4bc2-4e37-acb2-48a9b4b04427 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:19.166994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694eaf53-4bc2-4e37-acb2-48a9b4b04427 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:19.167640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694eaf53-4bc2-4e37-acb2-48a9b4b04427 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:19.169803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694eaf53-4bc2-4e37-acb2-48a9b4b04427 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:19.187933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0c0550ef-0442-4d86-b02c-cd061bbb4b97 None None] MFA Rules not processed for user `55dadd491ec1441aac4171eff46bd679`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:19.189560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0c0550ef-0442-4d86-b02c-cd061bbb4b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:19.209501 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3960/7977] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:21:19 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:21:19.211026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694eaf53-4bc2-4e37-acb2-48a9b4b04427 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:19.333711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:19.346983 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4018/7978] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:19.350723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694eaf53-4bc2-4e37-acb2-48a9b4b04427 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:19.351687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50ebcd-08ee-44d2-be8b-8c5af36a845b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:19.352701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50ebcd-08ee-44d2-be8b-8c5af36a845b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:19.352973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50ebcd-08ee-44d2-be8b-8c5af36a845b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:19.361221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3e16e-b5a8-4981-8e13-3b0c1bb87201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:19.362291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3e16e-b5a8-4981-8e13-3b0c1bb87201 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:19.362766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3e16e-b5a8-4981-8e13-3b0c1bb87201 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:19.363200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3e16e-b5a8-4981-8e13-3b0c1bb87201 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:19.364045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3e16e-b5a8-4981-8e13-3b0c1bb87201 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:19.366745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3e16e-b5a8-4981-8e13-3b0c1bb87201 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:19.376377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0c50ebcd-08ee-44d2-be8b-8c5af36a845b None None] MFA Rules not processed for user `eb0bf5fce885439eaa16a0871c2992ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:19.377579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0c50ebcd-08ee-44d2-be8b-8c5af36a845b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:19.395916 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3961/7979] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:21:19 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:21:19.396901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3e16e-b5a8-4981-8e13-3b0c1bb87201 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:19.398846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f8e3e16e-b5a8-4981-8e13-3b0c1bb87201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:19.413302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3e16e-b5a8-4981-8e13-3b0c1bb87201 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:19.418142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab8ca8-112e-438e-b63a-0aaef90a5fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:19.418794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab8ca8-112e-438e-b63a-0aaef90a5fb7 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:19.419079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab8ca8-112e-438e-b63a-0aaef90a5fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:19.419599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab8ca8-112e-438e-b63a-0aaef90a5fb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:19.420228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab8ca8-112e-438e-b63a-0aaef90a5fb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:19.423151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab8ca8-112e-438e-b63a-0aaef90a5fb7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:19.458854 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4019/7980] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:19.458854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50ebcd-08ee-44d2-be8b-8c5af36a845b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:19.469011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50ebcd-08ee-44d2-be8b-8c5af36a845b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:19.475461 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3962/7981] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:21:19 2022] POST /identity/v3/projects => generated 381 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:21:19.476212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab8ca8-112e-438e-b63a-0aaef90a5fb7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:19.499942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:DELETE=1,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:19.573248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6b71d-2725-4cab-8e75-4c3b63b11ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:19.573881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6b71d-2725-4cab-8e75-4c3b63b11ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:19.574165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6b71d-2725-4cab-8e75-4c3b63b11ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:19.574442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6b71d-2725-4cab-8e75-4c3b63b11ac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:19.575129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6b71d-2725-4cab-8e75-4c3b63b11ac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:19.576803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6b71d-2725-4cab-8e75-4c3b63b11ac8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:19.577641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab8ca8-112e-438e-b63a-0aaef90a5fb7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:19.596252 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4020/7982] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:19.598598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6b71d-2725-4cab-8e75-4c3b63b11ac8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:19.598734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58e3a65-f265-4c86-8ce7-ba891a6d5b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:19.600014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e3a65-f265-4c86-8ce7-ba891a6d5b85 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:19.600450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e3a65-f265-4c86-8ce7-ba891a6d5b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:19.600965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e3a65-f265-4c86-8ce7-ba891a6d5b85 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:19.602990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58e3a65-f265-4c86-8ce7-ba891a6d5b85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:19.608514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58e3a65-f265-4c86-8ce7-ba891a6d5b85 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:19.612715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6b71d-2725-4cab-8e75-4c3b63b11ac8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:19.620256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffa319-abfc-4d16-b70c-72bf7601457a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:19.620954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffa319-abfc-4d16-b70c-72bf7601457a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:19.621253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffa319-abfc-4d16-b70c-72bf7601457a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:19.621553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffa319-abfc-4d16-b70c-72bf7601457a admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:19.622181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffa319-abfc-4d16-b70c-72bf7601457a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:19.623882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffa319-abfc-4d16-b70c-72bf7601457a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:19.643303 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4021/7983] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:19.646149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffa319-abfc-4d16-b70c-72bf7601457a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:19.655193 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3963/7984] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:21:19 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:21:19.658138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58e3a65-f265-4c86-8ce7-ba891a6d5b85 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:19.664082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffa319-abfc-4d16-b70c-72bf7601457a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:19.671254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d60a1-5c68-453c-b40b-12bf9039d62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:19.672156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d60a1-5c68-453c-b40b-12bf9039d62c admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:19.672609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d60a1-5c68-453c-b40b-12bf9039d62c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:19.673330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d60a1-5c68-453c-b40b-12bf9039d62c admin admin] PATH_INFO: `/v3/projects/f8007ae0f001409a8a3c9d1e0b01bde1/users/ea4d4af3136e49d88739a370f46baacb/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:19.686161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58e3a65-f265-4c86-8ce7-ba891a6d5b85 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:19.691799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6010cb-5f5d-450b-9f9a-bb88c649cbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:19.692591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6010cb-5f5d-450b-9f9a-bb88c649cbec admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:19.692897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6010cb-5f5d-450b-9f9a-bb88c649cbec admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:19.693159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6010cb-5f5d-450b-9f9a-bb88c649cbec admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:19.694136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d60a1-5c68-453c-b40b-12bf9039d62c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8007ae0f001409a8a3c9d1e0b01bde1, user_id=ea4d4af3136e49d88739a370f46baacb, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:19.694435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6010cb-5f5d-450b-9f9a-bb88c649cbec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:19.696110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6010cb-5f5d-450b-9f9a-bb88c649cbec admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:19.698046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d60a1-5c68-453c-b40b-12bf9039d62c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:19.718855 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3964/7985] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:19.727387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6010cb-5f5d-450b-9f9a-bb88c649cbec admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:19.736786 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4022/7986] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:21:19 2022] PUT /identity/v3/projects/f8007ae0f001409a8a3c9d1e0b01bde1/users/ea4d4af3136e49d88739a370f46baacb/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:19.747347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8594cf9-d5fa-407c-8634-1cfab1e1acd6 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:19.748280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8594cf9-d5fa-407c-8634-1cfab1e1acd6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:19.748661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8594cf9-d5fa-407c-8634-1cfab1e1acd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:19.750939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6010cb-5f5d-450b-9f9a-bb88c649cbec admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:19.797856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c8594cf9-d5fa-407c-8634-1cfab1e1acd6 None None] MFA Rules not processed for user `ea4d4af3136e49d88739a370f46baacb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:19.799796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c8594cf9-d5fa-407c-8634-1cfab1e1acd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:19.867794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd9b11-a909-4f95-9e46-19e946557c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:19.868477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd9b11-a909-4f95-9e46-19e946557c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:19.869037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd9b11-a909-4f95-9e46-19e946557c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:19.869142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd9b11-a909-4f95-9e46-19e946557c1d admin admin] PATH_INFO: `/v3/projects/9dca212c4531456e96b84afb30119345/users/7ca7e6f15e574640b899f6f47ea0bfd4/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:19.893828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd9b11-a909-4f95-9e46-19e946557c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dca212c4531456e96b84afb30119345, user_id=7ca7e6f15e574640b899f6f47ea0bfd4, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:19.906732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd9b11-a909-4f95-9e46-19e946557c1d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:19.944267 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3965/7987] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:21:19 2022] PUT /identity/v3/projects/9dca212c4531456e96b84afb30119345/users/7ca7e6f15e574640b899f6f47ea0bfd4/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:19.950871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd9b11-a909-4f95-9e46-19e946557c1d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:19.961369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:19.968107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b8fd9b11-a909-4f95-9e46-19e946557c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:19.985142 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4023/7988] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:19.988449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8594cf9-d5fa-407c-8634-1cfab1e1acd6 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:20.015572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8594cf9-d5fa-407c-8634-1cfab1e1acd6 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:20.074018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd9b11-a909-4f95-9e46-19e946557c1d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:20.078096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342561b-5f24-42fc-9a0a-570d01047da1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:20.078782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342561b-5f24-42fc-9a0a-570d01047da1 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:20.079061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342561b-5f24-42fc-9a0a-570d01047da1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:20.079331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342561b-5f24-42fc-9a0a-570d01047da1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:20.086050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5342561b-5f24-42fc-9a0a-570d01047da1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:20.102406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342561b-5f24-42fc-9a0a-570d01047da1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:20.104050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342561b-5f24-42fc-9a0a-570d01047da1 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:20.109719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15703d-2ac1-4ded-9eb8-7e9acf94d692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:20.110443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15703d-2ac1-4ded-9eb8-7e9acf94d692 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:20.110880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15703d-2ac1-4ded-9eb8-7e9acf94d692 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:20.111364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15703d-2ac1-4ded-9eb8-7e9acf94d692 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:20.112143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15703d-2ac1-4ded-9eb8-7e9acf94d692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:20.113856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15703d-2ac1-4ded-9eb8-7e9acf94d692 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:20.117917 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3966/7989] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:21:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:21:20.119498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342561b-5f24-42fc-9a0a-570d01047da1 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:20.126061 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4024/7990] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:20.130350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15703d-2ac1-4ded-9eb8-7e9acf94d692 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:20.136473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342561b-5f24-42fc-9a0a-570d01047da1 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:20.141613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15703d-2ac1-4ded-9eb8-7e9acf94d692 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:20.143525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aceaf0-b38f-45b8-b7ac-6414acc326af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:20.144175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aceaf0-b38f-45b8-b7ac-6414acc326af admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:20.144743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aceaf0-b38f-45b8-b7ac-6414acc326af admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:20.145051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aceaf0-b38f-45b8-b7ac-6414acc326af admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:20.146318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aceaf0-b38f-45b8-b7ac-6414acc326af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:20.148311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aceaf0-b38f-45b8-b7ac-6414acc326af admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:20.152402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8893e-b502-48c0-a85f-29f29e1b3190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:20.153080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8893e-b502-48c0-a85f-29f29e1b3190 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:20.153361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8893e-b502-48c0-a85f-29f29e1b3190 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:20.153639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8893e-b502-48c0-a85f-29f29e1b3190 admin admin] PATH_INFO: `/v3/projects/9dca212c4531456e96b84afb30119345/users/7ca7e6f15e574640b899f6f47ea0bfd4/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:20.156108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8893e-b502-48c0-a85f-29f29e1b3190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dca212c4531456e96b84afb30119345, user_id=7ca7e6f15e574640b899f6f47ea0bfd4, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:20.159114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8893e-b502-48c0-a85f-29f29e1b3190 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:20.169847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:20.172860 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4025/7991] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:21:20 2022] PUT /identity/v3/projects/9dca212c4531456e96b84afb30119345/users/7ca7e6f15e574640b899f6f47ea0bfd4/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:21:20.191786 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3967/7992] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:21:20 2022] POST /identity/v3/projects => generated 387 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:21:20.195984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8893e-b502-48c0-a85f-29f29e1b3190 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:20.279226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8893e-b502-48c0-a85f-29f29e1b3190 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:20.387953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10805b8a-d4d0-40b2-b0cb-37b8a6846e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:20.388852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10805b8a-d4d0-40b2-b0cb-37b8a6846e99 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:20.406845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10805b8a-d4d0-40b2-b0cb-37b8a6846e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:20.408092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10805b8a-d4d0-40b2-b0cb-37b8a6846e99 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:20.408789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10805b8a-d4d0-40b2-b0cb-37b8a6846e99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:20.416013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10805b8a-d4d0-40b2-b0cb-37b8a6846e99 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:20.488171 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4026/7993] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:21:20 2022] POST /identity/v3/users => generated 300 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:21:20.491563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aceaf0-b38f-45b8-b7ac-6414acc326af admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:20.533702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aceaf0-b38f-45b8-b7ac-6414acc326af admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:20.540506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e2b61-05c8-42ca-8a57-b8997af812e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:20.541241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e2b61-05c8-42ca-8a57-b8997af812e5 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:20.541527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e2b61-05c8-42ca-8a57-b8997af812e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:20.541818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e2b61-05c8-42ca-8a57-b8997af812e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:20.542444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e2b61-05c8-42ca-8a57-b8997af812e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:20.544508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e2b61-05c8-42ca-8a57-b8997af812e5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:20.574527 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3968/7994] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:20.575763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10805b8a-d4d0-40b2-b0cb-37b8a6846e99 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:20.587006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10805b8a-d4d0-40b2-b0cb-37b8a6846e99 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:20.609553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097d06e-2d33-4a58-9b88-cbf309d978ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:20.613867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097d06e-2d33-4a58-9b88-cbf309d978ad admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:20.615639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097d06e-2d33-4a58-9b88-cbf309d978ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:20.627187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097d06e-2d33-4a58-9b88-cbf309d978ad admin admin] PATH_INFO: `/v3/projects/7a36be1ef1f24b27bc338a165be0df7f/users/4d1f2a6966ef4b27bfc0e6f4168c0dda/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:20.654194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:20.657933 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097d06e-2d33-4a58-9b88-cbf309d978ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a36be1ef1f24b27bc338a165be0df7f, user_id=4d1f2a6966ef4b27bfc0e6f4168c0dda, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:20.688960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097d06e-2d33-4a58-9b88-cbf309d978ad admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:20.717873 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4027/7995] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:21:20 2022] PUT /identity/v3/projects/7a36be1ef1f24b27bc338a165be0df7f/users/4d1f2a6966ef4b27bfc0e6f4168c0dda/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:20.723068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e2b61-05c8-42ca-8a57-b8997af812e5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:20.737861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e2b61-05c8-42ca-8a57-b8997af812e5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:20.838088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437355f4-2662-405a-b92d-085ce2944922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:20.839069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437355f4-2662-405a-b92d-085ce2944922 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:20.839541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437355f4-2662-405a-b92d-085ce2944922 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:20.840002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437355f4-2662-405a-b92d-085ce2944922 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:20.841077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437355f4-2662-405a-b92d-085ce2944922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:20.843356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437355f4-2662-405a-b92d-085ce2944922 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:20.882247 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3969/7996] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:20.886420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097d06e-2d33-4a58-9b88-cbf309d978ad admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:20.920464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097d06e-2d33-4a58-9b88-cbf309d978ad admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:20.933683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a874fa8-4d68-4d19-bc33-5353f9d833b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:20.936472 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a874fa8-4d68-4d19-bc33-5353f9d833b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:20.937367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a874fa8-4d68-4d19-bc33-5353f9d833b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:20.939444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a874fa8-4d68-4d19-bc33-5353f9d833b0 admin admin] PATH_INFO: `/v3/projects/7a36be1ef1f24b27bc338a165be0df7f/users/4d1f2a6966ef4b27bfc0e6f4168c0dda/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:20.944037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a874fa8-4d68-4d19-bc33-5353f9d833b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a36be1ef1f24b27bc338a165be0df7f, user_id=4d1f2a6966ef4b27bfc0e6f4168c0dda, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:20.948050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a874fa8-4d68-4d19-bc33-5353f9d833b0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:20.964925 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4028/7997] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:21:20 2022] PUT /identity/v3/projects/7a36be1ef1f24b27bc338a165be0df7f/users/4d1f2a6966ef4b27bfc0e6f4168c0dda/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:20.976089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94f05b-ee8a-4444-bfc2-863158d26e90 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:20.976633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94f05b-ee8a-4444-bfc2-863158d26e90 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:20.977114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94f05b-ee8a-4444-bfc2-863158d26e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:21.013794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9d94f05b-ee8a-4444-bfc2-863158d26e90 None None] MFA Rules not processed for user `4d1f2a6966ef4b27bfc0e6f4168c0dda`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:21.015431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9d94f05b-ee8a-4444-bfc2-863158d26e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:21.146035 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3970/7998] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:20 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:21.159234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a874fa8-4d68-4d19-bc33-5353f9d833b0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:21.162395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1a874fa8-4d68-4d19-bc33-5353f9d833b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:21.272223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a874fa8-4d68-4d19-bc33-5353f9d833b0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:21.276420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5975e-e222-4976-824a-51a366a6770b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:21.277079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5975e-e222-4976-824a-51a366a6770b service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:21.277365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5975e-e222-4976-824a-51a366a6770b service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:21.277637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5975e-e222-4976-824a-51a366a6770b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:21.289180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5975e-e222-4976-824a-51a366a6770b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:21.290737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5975e-e222-4976-824a-51a366a6770b service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:21.303470 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4029/7999] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:21:21 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:21:21.390088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94f05b-ee8a-4444-bfc2-863158d26e90 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:21.479120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94f05b-ee8a-4444-bfc2-863158d26e90 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:21.485316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00346978-c0b4-47fe-8c34-ddcefae9e141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:21.486054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00346978-c0b4-47fe-8c34-ddcefae9e141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:21.486401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00346978-c0b4-47fe-8c34-ddcefae9e141 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:21.486777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00346978-c0b4-47fe-8c34-ddcefae9e141 admin admin] PATH_INFO: `/v3/users/ea4d4af3136e49d88739a370f46baacb` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:21.487998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00346978-c0b4-47fe-8c34-ddcefae9e141 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea4d4af3136e49d88739a370f46baacb)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:21.490847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00346978-c0b4-47fe-8c34-ddcefae9e141 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:21.498205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:21.559705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00346978-c0b4-47fe-8c34-ddcefae9e141 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea4d4af3136e49d88739a370f46baacb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:21.560444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00346978-c0b4-47fe-8c34-ddcefae9e141 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea4d4af3136e49d88739a370f46baacb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:21.561916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00346978-c0b4-47fe-8c34-ddcefae9e141 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea4d4af3136e49d88739a370f46baacb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:21.573225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00346978-c0b4-47fe-8c34-ddcefae9e141 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea4d4af3136e49d88739a370f46baacb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:21.592860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-00346978-c0b4-47fe-8c34-ddcefae9e141 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea4d4af3136e49d88739a370f46baacb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:21.614042 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3971/8000] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:21:21 2022] DELETE /identity/v3/users/ea4d4af3136e49d88739a370f46baacb => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:21.616930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbb06d-34a9-4ca2-abc0-2b20ff487461 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:21.617369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbb06d-34a9-4ca2-abc0-2b20ff487461 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:21.617607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbb06d-34a9-4ca2-abc0-2b20ff487461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:21.649049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-98dbb06d-34a9-4ca2-abc0-2b20ff487461 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:21.650618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-98dbb06d-34a9-4ca2-abc0-2b20ff487461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:21.738538 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4030/8001] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:21:21.747862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00346978-c0b4-47fe-8c34-ddcefae9e141 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:21.751001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-00346978-c0b4-47fe-8c34-ddcefae9e141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:21.838873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00346978-c0b4-47fe-8c34-ddcefae9e141 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:21.842883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92808c3-4139-4359-8386-59ae972ddf11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:21.843568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92808c3-4139-4359-8386-59ae972ddf11 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:21.843932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92808c3-4139-4359-8386-59ae972ddf11 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:21.844195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92808c3-4139-4359-8386-59ae972ddf11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:21.856887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92808c3-4139-4359-8386-59ae972ddf11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:21.858532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92808c3-4139-4359-8386-59ae972ddf11 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:21.871114 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3972/8002] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:21:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:21:22.727143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dbb06d-34a9-4ca2-abc0-2b20ff487461 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:22.728839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-98dbb06d-34a9-4ca2-abc0-2b20ff487461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:22.746216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dbb06d-34a9-4ca2-abc0-2b20ff487461 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:22.758916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4a879-1313-42f6-85a5-e74dab492ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:22.759974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4a879-1313-42f6-85a5-e74dab492ffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:22.760554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4a879-1313-42f6-85a5-e74dab492ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:22.761172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4a879-1313-42f6-85a5-e74dab492ffa admin admin] PATH_INFO: `/v3/projects/f8007ae0f001409a8a3c9d1e0b01bde1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:22.763424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b4a879-1313-42f6-85a5-e74dab492ffa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8007ae0f001409a8a3c9d1e0b01bde1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:22.766353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b4a879-1313-42f6-85a5-e74dab492ffa admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:22.840111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d0b4a879-1313-42f6-85a5-e74dab492ffa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8007ae0f001409a8a3c9d1e0b01bde1', 'request_id': 'req-d0b4a879-1313-42f6-85a5-e74dab492ffa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:22.848077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d0b4a879-1313-42f6-85a5-e74dab492ffa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8007ae0f001409a8a3c9d1e0b01bde1', 'request_id': 'req-d0b4a879-1313-42f6-85a5-e74dab492ffa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:22.855794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d0b4a879-1313-42f6-85a5-e74dab492ffa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8007ae0f001409a8a3c9d1e0b01bde1', 'request_id': 'req-d0b4a879-1313-42f6-85a5-e74dab492ffa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:22.878894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d0b4a879-1313-42f6-85a5-e74dab492ffa admin admin] The token cache is being invalidate because project f8007ae0f001409a8a3c9d1e0b01bde1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:21:22.879390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d0b4a879-1313-42f6-85a5-e74dab492ffa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:22.881164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d0b4a879-1313-42f6-85a5-e74dab492ffa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:22.883706 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4031/8003] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:21:22 2022] DELETE /identity/v3/projects/f8007ae0f001409a8a3c9d1e0b01bde1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:22.887745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92808c3-4139-4359-8386-59ae972ddf11 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:22.911016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92808c3-4139-4359-8386-59ae972ddf11 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:23.112718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c52c5c-3909-4cea-a981-4e599db03a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:23.113829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c52c5c-3909-4cea-a981-4e599db03a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:23.114784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c52c5c-3909-4cea-a981-4e599db03a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:23.115837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c52c5c-3909-4cea-a981-4e599db03a81 admin admin] PATH_INFO: `/v3/users/4d1f2a6966ef4b27bfc0e6f4168c0dda` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:23.118856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c52c5c-3909-4cea-a981-4e599db03a81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d1f2a6966ef4b27bfc0e6f4168c0dda)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:23.121339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c52c5c-3909-4cea-a981-4e599db03a81 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:23.191514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:23.213863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e0c52c5c-3909-4cea-a981-4e599db03a81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d1f2a6966ef4b27bfc0e6f4168c0dda'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:23.214646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e0c52c5c-3909-4cea-a981-4e599db03a81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d1f2a6966ef4b27bfc0e6f4168c0dda'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:23.215462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e0c52c5c-3909-4cea-a981-4e599db03a81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d1f2a6966ef4b27bfc0e6f4168c0dda'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:23.227225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e0c52c5c-3909-4cea-a981-4e599db03a81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d1f2a6966ef4b27bfc0e6f4168c0dda'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:23.249859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e0c52c5c-3909-4cea-a981-4e599db03a81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d1f2a6966ef4b27bfc0e6f4168c0dda'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:23.280252 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3973/8004] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:21:22 2022] DELETE /identity/v3/users/4d1f2a6966ef4b27bfc0e6f4168c0dda => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:24.619646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4a879-1313-42f6-85a5-e74dab492ffa admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:24.621234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d0b4a879-1313-42f6-85a5-e74dab492ffa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:24.633018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:24.657423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4a879-1313-42f6-85a5-e74dab492ffa admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:24.816029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e293c-fabc-46ed-bb93-f8045c0bdf89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:24.817444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e293c-fabc-46ed-bb93-f8045c0bdf89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:24.817802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e293c-fabc-46ed-bb93-f8045c0bdf89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:24.818700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e293c-fabc-46ed-bb93-f8045c0bdf89 admin admin] PATH_INFO: `/v3/projects/7a36be1ef1f24b27bc338a165be0df7f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:24.821282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e293c-fabc-46ed-bb93-f8045c0bdf89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a36be1ef1f24b27bc338a165be0df7f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:24.828730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e293c-fabc-46ed-bb93-f8045c0bdf89 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:24.950977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-670e293c-fabc-46ed-bb93-f8045c0bdf89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a36be1ef1f24b27bc338a165be0df7f', 'request_id': 'req-670e293c-fabc-46ed-bb93-f8045c0bdf89'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:24.983184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-670e293c-fabc-46ed-bb93-f8045c0bdf89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a36be1ef1f24b27bc338a165be0df7f', 'request_id': 'req-670e293c-fabc-46ed-bb93-f8045c0bdf89'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:25.002098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-670e293c-fabc-46ed-bb93-f8045c0bdf89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a36be1ef1f24b27bc338a165be0df7f', 'request_id': 'req-670e293c-fabc-46ed-bb93-f8045c0bdf89'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:25.035040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-670e293c-fabc-46ed-bb93-f8045c0bdf89 admin admin] The token cache is being invalidate because project 7a36be1ef1f24b27bc338a165be0df7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:21:25.035742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-670e293c-fabc-46ed-bb93-f8045c0bdf89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:25.038082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-670e293c-fabc-46ed-bb93-f8045c0bdf89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:25.055289 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4032/8005] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:21:24 2022] DELETE /identity/v3/projects/7a36be1ef1f24b27bc338a165be0df7f => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:25.060069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953368dc-9b5c-45c3-84f8-c253ae03f225 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:25.061376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953368dc-9b5c-45c3-84f8-c253ae03f225 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:25.062170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953368dc-9b5c-45c3-84f8-c253ae03f225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:25.111658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-953368dc-9b5c-45c3-84f8-c253ae03f225 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:25.112944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-953368dc-9b5c-45c3-84f8-c253ae03f225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:25.258912 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3974/8006] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:25.261061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364b937-1dde-4593-bfe8-c1cfe32ceaff None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:25.262180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364b937-1dde-4593-bfe8-c1cfe32ceaff None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:25.262558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364b937-1dde-4593-bfe8-c1cfe32ceaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:25.317743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1364b937-1dde-4593-bfe8-c1cfe32ceaff None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:25.319792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1364b937-1dde-4593-bfe8-c1cfe32ceaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:25.335271 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4033/8007] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:25.336974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953368dc-9b5c-45c3-84f8-c253ae03f225 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:25.356075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953368dc-9b5c-45c3-84f8-c253ae03f225 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:25.364456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80831ea-ef36-477e-949c-77c8c6b8db52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:25.368637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80831ea-ef36-477e-949c-77c8c6b8db52 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:25.369025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80831ea-ef36-477e-949c-77c8c6b8db52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:25.369405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80831ea-ef36-477e-949c-77c8c6b8db52 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:25.370182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80831ea-ef36-477e-949c-77c8c6b8db52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:21:25.373298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80831ea-ef36-477e-949c-77c8c6b8db52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:25.374601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80831ea-ef36-477e-949c-77c8c6b8db52 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:25.403829 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3975/8008] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:21:25.408417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364b937-1dde-4593-bfe8-c1cfe32ceaff None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:25.424178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364b937-1dde-4593-bfe8-c1cfe32ceaff None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:25.435169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818b0d2-fae7-4e67-98c2-e7871f8e6508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:25.436084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818b0d2-fae7-4e67-98c2-e7871f8e6508 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:25.436469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818b0d2-fae7-4e67-98c2-e7871f8e6508 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:25.436846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818b0d2-fae7-4e67-98c2-e7871f8e6508 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:25.437966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818b0d2-fae7-4e67-98c2-e7871f8e6508 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:25.441814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818b0d2-fae7-4e67-98c2-e7871f8e6508 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:25.524859 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4034/8009] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:21:25 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:21:25.528642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80831ea-ef36-477e-949c-77c8c6b8db52 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:25.553054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80831ea-ef36-477e-949c-77c8c6b8db52 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:25.734571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f759ae-d682-4caa-b4af-a46a01a1224c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:25.735257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f759ae-d682-4caa-b4af-a46a01a1224c admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:25.735544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f759ae-d682-4caa-b4af-a46a01a1224c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:25.735813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f759ae-d682-4caa-b4af-a46a01a1224c admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:25.736462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f759ae-d682-4caa-b4af-a46a01a1224c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:25.738568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f759ae-d682-4caa-b4af-a46a01a1224c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:25.792997 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3976/8010] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:21:25 2022] POST /identity/v3/users => generated 294 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:21:25.795186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818b0d2-fae7-4e67-98c2-e7871f8e6508 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:25.809383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818b0d2-fae7-4e67-98c2-e7871f8e6508 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:25.826058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c28e1-2134-4ad4-a604-5a369752ab7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:25.826785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c28e1-2134-4ad4-a604-5a369752ab7a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:25.826986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c28e1-2134-4ad4-a604-5a369752ab7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:25.827562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c28e1-2134-4ad4-a604-5a369752ab7a admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:25.827950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c28e1-2134-4ad4-a604-5a369752ab7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:25.829590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c28e1-2134-4ad4-a604-5a369752ab7a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:25.849910 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4035/8011] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:25.853703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f759ae-d682-4caa-b4af-a46a01a1224c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:25.867702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f759ae-d682-4caa-b4af-a46a01a1224c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:25.883262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb63b8-2dc3-489f-9ad3-97b8cb22bbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:25.883889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb63b8-2dc3-489f-9ad3-97b8cb22bbe6 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:25.884168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb63b8-2dc3-489f-9ad3-97b8cb22bbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:25.884440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb63b8-2dc3-489f-9ad3-97b8cb22bbe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:25.885092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb63b8-2dc3-489f-9ad3-97b8cb22bbe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:25.886805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb63b8-2dc3-489f-9ad3-97b8cb22bbe6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:25.914966 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3977/8012] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:25.919725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c28e1-2134-4ad4-a604-5a369752ab7a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:25.942314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c28e1-2134-4ad4-a604-5a369752ab7a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:25.948854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04270a1-fdc1-4f56-bfbc-a307de4a4346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:25.949747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04270a1-fdc1-4f56-bfbc-a307de4a4346 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:25.950037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04270a1-fdc1-4f56-bfbc-a307de4a4346 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:25.950319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04270a1-fdc1-4f56-bfbc-a307de4a4346 admin admin] PATH_INFO: `/v3/projects/df92a298c5864d28b59ae18fd1e9f329/users/5381cd9578ad4a7f96ca896dee3cf5c8/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:25.986484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04270a1-fdc1-4f56-bfbc-a307de4a4346 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df92a298c5864d28b59ae18fd1e9f329, user_id=5381cd9578ad4a7f96ca896dee3cf5c8, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:25.990534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04270a1-fdc1-4f56-bfbc-a307de4a4346 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:26.013891 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4036/8013] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:21:25 2022] PUT /identity/v3/projects/df92a298c5864d28b59ae18fd1e9f329/users/5381cd9578ad4a7f96ca896dee3cf5c8/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:26.025727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682a6b-d947-4f2f-b03c-a3d3d58bff1d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:26.026067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682a6b-d947-4f2f-b03c-a3d3d58bff1d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:26.026353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682a6b-d947-4f2f-b03c-a3d3d58bff1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:26.123274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:26.133004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-0d682a6b-d947-4f2f-b03c-a3d3d58bff1d None None] MFA Rules not processed for user `5381cd9578ad4a7f96ca896dee3cf5c8`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:26.138718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0d682a6b-d947-4f2f-b03c-a3d3d58bff1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:26.307453 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3978/8014] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:26.312180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04270a1-fdc1-4f56-bfbc-a307de4a4346 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:26.349081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:26.443697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04270a1-fdc1-4f56-bfbc-a307de4a4346 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:26.453303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15722e6-882d-4efa-8b54-f1bea4199f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:26.454454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15722e6-882d-4efa-8b54-f1bea4199f6e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:26.454454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15722e6-882d-4efa-8b54-f1bea4199f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:26.454454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15722e6-882d-4efa-8b54-f1bea4199f6e admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:26.455883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15722e6-882d-4efa-8b54-f1bea4199f6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:26.465196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15722e6-882d-4efa-8b54-f1bea4199f6e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:26.526787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d682a6b-d947-4f2f-b03c-a3d3d58bff1d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:26.527104 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4037/8015] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:21:26 2022] POST /identity/v3/projects => generated 389 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:21:26.545149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d682a6b-d947-4f2f-b03c-a3d3d58bff1d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:26.680248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951bc71-cbc2-416c-8a31-6dfd78c6e3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:26.683459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951bc71-cbc2-416c-8a31-6dfd78c6e3e2 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:26.683922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951bc71-cbc2-416c-8a31-6dfd78c6e3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:26.684372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951bc71-cbc2-416c-8a31-6dfd78c6e3e2 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:26.685208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951bc71-cbc2-416c-8a31-6dfd78c6e3e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:26.688296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951bc71-cbc2-416c-8a31-6dfd78c6e3e2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:26.735807 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3979/8016] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:21:26 2022] POST /identity/v3/users => generated 301 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:21:26.739876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15722e6-882d-4efa-8b54-f1bea4199f6e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:26.751193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15722e6-882d-4efa-8b54-f1bea4199f6e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:26.764469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d7e3cb-dff8-443b-b85b-07a1544b3fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:26.765099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7e3cb-dff8-443b-b85b-07a1544b3fed admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:26.765397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7e3cb-dff8-443b-b85b-07a1544b3fed admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:26.765673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7e3cb-dff8-443b-b85b-07a1544b3fed admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:26.766337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d7e3cb-dff8-443b-b85b-07a1544b3fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:26.768137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d7e3cb-dff8-443b-b85b-07a1544b3fed admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:26.782780 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4038/8017] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:26.788164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951bc71-cbc2-416c-8a31-6dfd78c6e3e2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:26.805132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951bc71-cbc2-416c-8a31-6dfd78c6e3e2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:26.822357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34b21e-2fd9-4c34-98e2-9f2092646ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:26.823012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34b21e-2fd9-4c34-98e2-9f2092646ae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:26.823296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34b21e-2fd9-4c34-98e2-9f2092646ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:26.823576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34b21e-2fd9-4c34-98e2-9f2092646ae6 admin admin] PATH_INFO: `/v3/projects/2ec671eb45254e688446b1caa2c784a8/users/bfce09ced01f4684936e9fb15f80f903/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:26.870133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34b21e-2fd9-4c34-98e2-9f2092646ae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ec671eb45254e688446b1caa2c784a8, user_id=bfce09ced01f4684936e9fb15f80f903, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:26.872875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34b21e-2fd9-4c34-98e2-9f2092646ae6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:26.914920 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3980/8018] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:21:26 2022] PUT /identity/v3/projects/2ec671eb45254e688446b1caa2c784a8/users/bfce09ced01f4684936e9fb15f80f903/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:21:26.920148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d7e3cb-dff8-443b-b85b-07a1544b3fed admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:26.934239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d7e3cb-dff8-443b-b85b-07a1544b3fed admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:27.020715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db44aa-6ac2-4967-932b-b39d77877b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:27.022045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db44aa-6ac2-4967-932b-b39d77877b8e admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:27.022383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db44aa-6ac2-4967-932b-b39d77877b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:27.022709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db44aa-6ac2-4967-932b-b39d77877b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:27.023455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db44aa-6ac2-4967-932b-b39d77877b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:27.025188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db44aa-6ac2-4967-932b-b39d77877b8e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:27.039904 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4039/8019] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:27.046693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34b21e-2fd9-4c34-98e2-9f2092646ae6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:27.065619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34b21e-2fd9-4c34-98e2-9f2092646ae6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:27.077795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdccae2-a392-44ff-840c-108d25e4558a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:27.078519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdccae2-a392-44ff-840c-108d25e4558a admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:27.078813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdccae2-a392-44ff-840c-108d25e4558a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:27.079089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdccae2-a392-44ff-840c-108d25e4558a admin admin] PATH_INFO: `/v3/projects/2ec671eb45254e688446b1caa2c784a8/users/bfce09ced01f4684936e9fb15f80f903/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:27.088087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdccae2-a392-44ff-840c-108d25e4558a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ec671eb45254e688446b1caa2c784a8, user_id=bfce09ced01f4684936e9fb15f80f903, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:27.091085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdccae2-a392-44ff-840c-108d25e4558a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:27.116634 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3981/8020] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:21:27 2022] PUT /identity/v3/projects/2ec671eb45254e688446b1caa2c784a8/users/bfce09ced01f4684936e9fb15f80f903/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:27.129383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb433be-dd00-4e95-aa9c-a84a4e8109aa None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:27.129383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb433be-dd00-4e95-aa9c-a84a4e8109aa None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:27.129383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb433be-dd00-4e95-aa9c-a84a4e8109aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:27.167441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2eb433be-dd00-4e95-aa9c-a84a4e8109aa None None] MFA Rules not processed for user `bfce09ced01f4684936e9fb15f80f903`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:27.168877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2eb433be-dd00-4e95-aa9c-a84a4e8109aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:27.340545 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4040/8021] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:27 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:27.352314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdccae2-a392-44ff-840c-108d25e4558a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:27.354753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-abdccae2-a392-44ff-840c-108d25e4558a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:27.478034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdccae2-a392-44ff-840c-108d25e4558a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:27.485459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabacdb-6dbf-4e60-86b9-f0c9ce7e880a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:27.486031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabacdb-6dbf-4e60-86b9-f0c9ce7e880a service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:27.486472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabacdb-6dbf-4e60-86b9-f0c9ce7e880a service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:27.486909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabacdb-6dbf-4e60-86b9-f0c9ce7e880a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:27.500731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beabacdb-6dbf-4e60-86b9-f0c9ce7e880a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:27.502475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beabacdb-6dbf-4e60-86b9-f0c9ce7e880a service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:27.508760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:27.525847 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3982/8022] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:21:27 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:21:27.758875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb433be-dd00-4e95-aa9c-a84a4e8109aa None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:27.787741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:27.855695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb433be-dd00-4e95-aa9c-a84a4e8109aa None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:27.863920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5fac0-02e8-40a0-9ca5-e86ad61a8dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:27.864291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5fac0-02e8-40a0-9ca5-e86ad61a8dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:27.864507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5fac0-02e8-40a0-9ca5-e86ad61a8dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:27.865351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5fac0-02e8-40a0-9ca5-e86ad61a8dd7 admin admin] PATH_INFO: `/v3/users/5381cd9578ad4a7f96ca896dee3cf5c8` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:27.866543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa5fac0-02e8-40a0-9ca5-e86ad61a8dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5381cd9578ad4a7f96ca896dee3cf5c8)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:27.872660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa5fac0-02e8-40a0-9ca5-e86ad61a8dd7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:27.985203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6fa5fac0-02e8-40a0-9ca5-e86ad61a8dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5381cd9578ad4a7f96ca896dee3cf5c8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:27.994176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6fa5fac0-02e8-40a0-9ca5-e86ad61a8dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5381cd9578ad4a7f96ca896dee3cf5c8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:28.026193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6fa5fac0-02e8-40a0-9ca5-e86ad61a8dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5381cd9578ad4a7f96ca896dee3cf5c8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:28.027007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6fa5fac0-02e8-40a0-9ca5-e86ad61a8dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5381cd9578ad4a7f96ca896dee3cf5c8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:28.042447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6fa5fac0-02e8-40a0-9ca5-e86ad61a8dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5381cd9578ad4a7f96ca896dee3cf5c8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:28.072298 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4041/8023] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:21:27 2022] DELETE /identity/v3/users/5381cd9578ad4a7f96ca896dee3cf5c8 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:28.077855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae51ec-ff68-4da3-b1a3-8697807592a1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:28.078410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae51ec-ff68-4da3-b1a3-8697807592a1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:28.078939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae51ec-ff68-4da3-b1a3-8697807592a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:28.109895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-35ae51ec-ff68-4da3-b1a3-8697807592a1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:28.111786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-35ae51ec-ff68-4da3-b1a3-8697807592a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:28.293405 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3983/8024] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:21:28.306366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5fac0-02e8-40a0-9ca5-e86ad61a8dd7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:28.308261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6fa5fac0-02e8-40a0-9ca5-e86ad61a8dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:28.469935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5fac0-02e8-40a0-9ca5-e86ad61a8dd7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:28.476040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6142e3-0c87-44b2-8c31-b16bef02fb53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:28.476040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6142e3-0c87-44b2-8c31-b16bef02fb53 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:28.476323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6142e3-0c87-44b2-8c31-b16bef02fb53 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:28.476608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6142e3-0c87-44b2-8c31-b16bef02fb53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:28.489516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6142e3-0c87-44b2-8c31-b16bef02fb53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:28.492051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6142e3-0c87-44b2-8c31-b16bef02fb53 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:28.507132 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4042/8025] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:21:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:21:28.950153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae51ec-ff68-4da3-b1a3-8697807592a1 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:28.951855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-35ae51ec-ff68-4da3-b1a3-8697807592a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:28.979831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae51ec-ff68-4da3-b1a3-8697807592a1 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:28.990712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecdffc-e5d5-45f8-80ab-15c3abc7220d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:28.991582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecdffc-e5d5-45f8-80ab-15c3abc7220d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:28.992296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecdffc-e5d5-45f8-80ab-15c3abc7220d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:28.993195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecdffc-e5d5-45f8-80ab-15c3abc7220d admin admin] PATH_INFO: `/v3/projects/df92a298c5864d28b59ae18fd1e9f329` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:28.995926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecdffc-e5d5-45f8-80ab-15c3abc7220d admin admin] RBAC: Authorizing `identity:delete_project(project_id=df92a298c5864d28b59ae18fd1e9f329)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:28.999995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecdffc-e5d5-45f8-80ab-15c3abc7220d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:29.086211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2cecdffc-e5d5-45f8-80ab-15c3abc7220d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df92a298c5864d28b59ae18fd1e9f329', 'request_id': 'req-2cecdffc-e5d5-45f8-80ab-15c3abc7220d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:29.108788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2cecdffc-e5d5-45f8-80ab-15c3abc7220d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df92a298c5864d28b59ae18fd1e9f329', 'request_id': 'req-2cecdffc-e5d5-45f8-80ab-15c3abc7220d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:29.131947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2cecdffc-e5d5-45f8-80ab-15c3abc7220d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df92a298c5864d28b59ae18fd1e9f329', 'request_id': 'req-2cecdffc-e5d5-45f8-80ab-15c3abc7220d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:29.155521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2cecdffc-e5d5-45f8-80ab-15c3abc7220d admin admin] The token cache is being invalidate because project df92a298c5864d28b59ae18fd1e9f329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:21:29.156071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2cecdffc-e5d5-45f8-80ab-15c3abc7220d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:29.156935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2cecdffc-e5d5-45f8-80ab-15c3abc7220d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:29.160463 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3984/8026] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:21:28 2022] DELETE /identity/v3/projects/df92a298c5864d28b59ae18fd1e9f329 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:29.164774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6142e3-0c87-44b2-8c31-b16bef02fb53 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:29.178915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6142e3-0c87-44b2-8c31-b16bef02fb53 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:29.217273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:29.279599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65208c30-3c0e-464e-ba09-50fd6d96437b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:29.280258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65208c30-3c0e-464e-ba09-50fd6d96437b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:29.280535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65208c30-3c0e-464e-ba09-50fd6d96437b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:29.280807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65208c30-3c0e-464e-ba09-50fd6d96437b admin admin] PATH_INFO: `/v3/users/bfce09ced01f4684936e9fb15f80f903` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:29.281972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65208c30-3c0e-464e-ba09-50fd6d96437b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfce09ced01f4684936e9fb15f80f903)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:29.284034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65208c30-3c0e-464e-ba09-50fd6d96437b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:29.347670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-65208c30-3c0e-464e-ba09-50fd6d96437b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfce09ced01f4684936e9fb15f80f903'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:29.357511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-65208c30-3c0e-464e-ba09-50fd6d96437b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfce09ced01f4684936e9fb15f80f903'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:29.376770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-65208c30-3c0e-464e-ba09-50fd6d96437b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfce09ced01f4684936e9fb15f80f903'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:29.377758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-65208c30-3c0e-464e-ba09-50fd6d96437b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfce09ced01f4684936e9fb15f80f903'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:29.388665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-65208c30-3c0e-464e-ba09-50fd6d96437b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfce09ced01f4684936e9fb15f80f903'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:29.396588 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4043/8027] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:21:29 2022] DELETE /identity/v3/users/bfce09ced01f4684936e9fb15f80f903 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:29.710205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecdffc-e5d5-45f8-80ab-15c3abc7220d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:29.711926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2cecdffc-e5d5-45f8-80ab-15c3abc7220d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:29.757975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecdffc-e5d5-45f8-80ab-15c3abc7220d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:29.831178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f856c-8f28-412f-aaa6-82c337b0a3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:29.831930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f856c-8f28-412f-aaa6-82c337b0a3df admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:29.832318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f856c-8f28-412f-aaa6-82c337b0a3df admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:29.832658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f856c-8f28-412f-aaa6-82c337b0a3df admin admin] PATH_INFO: `/v3/projects/2ec671eb45254e688446b1caa2c784a8` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:29.834021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f856c-8f28-412f-aaa6-82c337b0a3df admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ec671eb45254e688446b1caa2c784a8)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:29.836142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f856c-8f28-412f-aaa6-82c337b0a3df admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:29.866531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:29.903709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fa6f856c-8f28-412f-aaa6-82c337b0a3df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ec671eb45254e688446b1caa2c784a8', 'request_id': 'req-fa6f856c-8f28-412f-aaa6-82c337b0a3df'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:29.913867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fa6f856c-8f28-412f-aaa6-82c337b0a3df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ec671eb45254e688446b1caa2c784a8', 'request_id': 'req-fa6f856c-8f28-412f-aaa6-82c337b0a3df'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:29.921704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fa6f856c-8f28-412f-aaa6-82c337b0a3df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ec671eb45254e688446b1caa2c784a8', 'request_id': 'req-fa6f856c-8f28-412f-aaa6-82c337b0a3df'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:29.934537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fa6f856c-8f28-412f-aaa6-82c337b0a3df admin admin] The token cache is being invalidate because project 2ec671eb45254e688446b1caa2c784a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:21:29.934836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fa6f856c-8f28-412f-aaa6-82c337b0a3df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:29.935495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fa6f856c-8f28-412f-aaa6-82c337b0a3df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:29.937333 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3985/8028] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:21:29 2022] DELETE /identity/v3/projects/2ec671eb45254e688446b1caa2c784a8 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:29.942029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a05f0b-eaff-4051-b6a3-a827157f4168 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:29.942319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a05f0b-eaff-4051-b6a3-a827157f4168 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:29.942662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a05f0b-eaff-4051-b6a3-a827157f4168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:29.965241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-09a05f0b-eaff-4051-b6a3-a827157f4168 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:29.966470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-09a05f0b-eaff-4051-b6a3-a827157f4168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:29.984691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69f952-3b23-47ac-9a30-bd796c62624a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:29.985262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69f952-3b23-47ac-9a30-bd796c62624a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:29.985262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69f952-3b23-47ac-9a30-bd796c62624a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:30.026574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-1a69f952-3b23-47ac-9a30-bd796c62624a None None] MFA Rules not processed for user `7ca7e6f15e574640b899f6f47ea0bfd4`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:30.028817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1a69f952-3b23-47ac-9a30-bd796c62624a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:30.047394 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4044/8029] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:30.054928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d269d-86a5-44c9-ae2c-87c9f55c0a6b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:30.055342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d269d-86a5-44c9-ae2c-87c9f55c0a6b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:30.055815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d269d-86a5-44c9-ae2c-87c9f55c0a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:30.085973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8a9d269d-86a5-44c9-ae2c-87c9f55c0a6b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:30.089460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8a9d269d-86a5-44c9-ae2c-87c9f55c0a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:30.097602 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4045/8030] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:30.103713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d269d-86a5-44c9-ae2c-87c9f55c0a6b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:30.122624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d269d-86a5-44c9-ae2c-87c9f55c0a6b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:30.135025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe0b82-e7c8-4dd2-a17e-7f2d5e718add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:30.135783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe0b82-e7c8-4dd2-a17e-7f2d5e718add admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:30.136082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe0b82-e7c8-4dd2-a17e-7f2d5e718add admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:30.136349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe0b82-e7c8-4dd2-a17e-7f2d5e718add admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:30.137020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe0b82-e7c8-4dd2-a17e-7f2d5e718add admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:21:30.137329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe0b82-e7c8-4dd2-a17e-7f2d5e718add admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:30.139037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe0b82-e7c8-4dd2-a17e-7f2d5e718add admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:30.160256 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4046/8031] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:21:30.164851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe0b82-e7c8-4dd2-a17e-7f2d5e718add admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:30.178879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe0b82-e7c8-4dd2-a17e-7f2d5e718add admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:30.190458 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3986/8032] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:30.199694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69f952-3b23-47ac-9a30-bd796c62624a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:30.201522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-1a69f952-3b23-47ac-9a30-bd796c62624a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:30.213086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdfe4e-86c3-4623-85f9-e065ff50c964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:30.213734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdfe4e-86c3-4623-85f9-e065ff50c964 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:30.215114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdfe4e-86c3-4623-85f9-e065ff50c964 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:30.215406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdfe4e-86c3-4623-85f9-e065ff50c964 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:30.216174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdfe4e-86c3-4623-85f9-e065ff50c964 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:30.218248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdfe4e-86c3-4623-85f9-e065ff50c964 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:30.268959 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4047/8033] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:21:30 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:21:30.273748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdfe4e-86c3-4623-85f9-e065ff50c964 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:30.292439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:30.311267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69f952-3b23-47ac-9a30-bd796c62624a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:30.379804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdfe4e-86c3-4623-85f9-e065ff50c964 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:30.385418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61933c0a-10a3-4d78-82ae-5daca37afc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:30.386052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61933c0a-10a3-4d78-82ae-5daca37afc35 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:30.386335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61933c0a-10a3-4d78-82ae-5daca37afc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:30.386808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61933c0a-10a3-4d78-82ae-5daca37afc35 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:30.387449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61933c0a-10a3-4d78-82ae-5daca37afc35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:30.390264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61933c0a-10a3-4d78-82ae-5daca37afc35 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:30.409990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc1a16-f287-48ba-a250-75115632636c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:30.410657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc1a16-f287-48ba-a250-75115632636c service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:30.411018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc1a16-f287-48ba-a250-75115632636c service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:30.412108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc1a16-f287-48ba-a250-75115632636c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:30.414648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-accc1a16-f287-48ba-a250-75115632636c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:30.428941 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4048/8034] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:21:30 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:21:30.433440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61933c0a-10a3-4d78-82ae-5daca37afc35 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:30.436902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:30.462527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61933c0a-10a3-4d78-82ae-5daca37afc35 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:30.471481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b80d0c-62ee-45a0-94c0-0ab3a6fe9c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:30.472343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b80d0c-62ee-45a0-94c0-0ab3a6fe9c47 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:30.472459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b80d0c-62ee-45a0-94c0-0ab3a6fe9c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:30.472661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b80d0c-62ee-45a0-94c0-0ab3a6fe9c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:30.473439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b80d0c-62ee-45a0-94c0-0ab3a6fe9c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:30.476920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b80d0c-62ee-45a0-94c0-0ab3a6fe9c47 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:30.500546 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4049/8035] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:30.501808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b80d0c-62ee-45a0-94c0-0ab3a6fe9c47 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:30.515504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b80d0c-62ee-45a0-94c0-0ab3a6fe9c47 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:30.522096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8a3f6-e8d9-48c8-aec7-799c904e0fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:30.522882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8a3f6-e8d9-48c8-aec7-799c904e0fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:30.523231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8a3f6-e8d9-48c8-aec7-799c904e0fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:30.523567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8a3f6-e8d9-48c8-aec7-799c904e0fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:30.524361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8a3f6-e8d9-48c8-aec7-799c904e0fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:30.526020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8a3f6-e8d9-48c8-aec7-799c904e0fc5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:30.535864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc1a16-f287-48ba-a250-75115632636c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:30.538157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc1a16-f287-48ba-a250-75115632636c service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:30.539794 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4050/8036] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:30.545187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8a3f6-e8d9-48c8-aec7-799c904e0fc5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:30.554160 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3987/8037] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:21:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:21:30.559720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8a3f6-e8d9-48c8-aec7-799c904e0fc5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:30.566207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b5b5e-9de2-4031-a850-9921aca94edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:30.567233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b5b5e-9de2-4031-a850-9921aca94edf admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:30.567602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b5b5e-9de2-4031-a850-9921aca94edf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:30.568497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b5b5e-9de2-4031-a850-9921aca94edf admin admin] PATH_INFO: `/v3/projects/f748158a60e842d4b1bb6598eb610212/users/21add4fa26f94702a091703c6d98e855/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:30.606187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b5b5e-9de2-4031-a850-9921aca94edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f748158a60e842d4b1bb6598eb610212, user_id=21add4fa26f94702a091703c6d98e855, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:30.615503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b5b5e-9de2-4031-a850-9921aca94edf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:30.658927 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4051/8038] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:21:30 2022] PUT /identity/v3/projects/f748158a60e842d4b1bb6598eb610212/users/21add4fa26f94702a091703c6d98e855/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:30.671554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68483283-b641-4327-8add-417ab1d3bcaa None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:30.672048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68483283-b641-4327-8add-417ab1d3bcaa None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:30.672506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68483283-b641-4327-8add-417ab1d3bcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:30.740274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-68483283-b641-4327-8add-417ab1d3bcaa None None] MFA Rules not processed for user `21add4fa26f94702a091703c6d98e855`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:30.743308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-68483283-b641-4327-8add-417ab1d3bcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:30.946852 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3988/8039] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:30.949578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b5b5e-9de2-4031-a850-9921aca94edf admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:31.074864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b5b5e-9de2-4031-a850-9921aca94edf admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:31.081801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e359d-4ba6-4fad-ad99-910dddb47a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:31.082725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e359d-4ba6-4fad-ad99-910dddb47a20 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:31.083071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e359d-4ba6-4fad-ad99-910dddb47a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:31.083569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e359d-4ba6-4fad-ad99-910dddb47a20 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:31.084500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e359d-4ba6-4fad-ad99-910dddb47a20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:31.086785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e359d-4ba6-4fad-ad99-910dddb47a20 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:31.131351 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4052/8040] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:21:30 2022] POST /identity/v3/projects => generated 383 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:21:31.135889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68483283-b641-4327-8add-417ab1d3bcaa None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:31.158100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68483283-b641-4327-8add-417ab1d3bcaa None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:31.338511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9036fe8-d54f-43c1-86ce-ae8847e2dc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:31.339308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9036fe8-d54f-43c1-86ce-ae8847e2dc6c admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:31.340240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9036fe8-d54f-43c1-86ce-ae8847e2dc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:31.340240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9036fe8-d54f-43c1-86ce-ae8847e2dc6c admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:31.340704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9036fe8-d54f-43c1-86ce-ae8847e2dc6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:31.342806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9036fe8-d54f-43c1-86ce-ae8847e2dc6c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:31.392428 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3989/8041] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:21:31 2022] POST /identity/v3/users => generated 298 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:21:31.397143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e359d-4ba6-4fad-ad99-910dddb47a20 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:31.409682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e359d-4ba6-4fad-ad99-910dddb47a20 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:31.416576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789af79-725c-4f41-bb36-f0920bab014f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:31.417333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789af79-725c-4f41-bb36-f0920bab014f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:31.417710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789af79-725c-4f41-bb36-f0920bab014f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:31.418082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789af79-725c-4f41-bb36-f0920bab014f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:31.418898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7789af79-725c-4f41-bb36-f0920bab014f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:31.420937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7789af79-725c-4f41-bb36-f0920bab014f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:31.435751 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4053/8042] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:31.439719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9036fe8-d54f-43c1-86ce-ae8847e2dc6c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:31.453811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9036fe8-d54f-43c1-86ce-ae8847e2dc6c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:31.463834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33012a36-2894-4dca-aff9-b9981c1b121b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:31.464964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33012a36-2894-4dca-aff9-b9981c1b121b admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:31.465400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33012a36-2894-4dca-aff9-b9981c1b121b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:31.465852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33012a36-2894-4dca-aff9-b9981c1b121b admin admin] PATH_INFO: `/v3/projects/8376d8cb5f81436b98821522b7319e1b/users/7f09d39189554ff696660125bf3ba12e/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:31.488051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33012a36-2894-4dca-aff9-b9981c1b121b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8376d8cb5f81436b98821522b7319e1b, user_id=7f09d39189554ff696660125bf3ba12e, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:31.491387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33012a36-2894-4dca-aff9-b9981c1b121b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:31.509774 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3990/8043] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:21:31 2022] PUT /identity/v3/projects/8376d8cb5f81436b98821522b7319e1b/users/7f09d39189554ff696660125bf3ba12e/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:31.513727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789af79-725c-4f41-bb36-f0920bab014f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:31.526889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789af79-725c-4f41-bb36-f0920bab014f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:31.565024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:31.612328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba3c4b-4c19-4221-827b-3a97cce1a053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:31.613675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba3c4b-4c19-4221-827b-3a97cce1a053 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:31.618325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba3c4b-4c19-4221-827b-3a97cce1a053 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:31.618880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba3c4b-4c19-4221-827b-3a97cce1a053 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:31.619776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba3c4b-4c19-4221-827b-3a97cce1a053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:31.622397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba3c4b-4c19-4221-827b-3a97cce1a053 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:31.640043 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4054/8044] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:31.641823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33012a36-2894-4dca-aff9-b9981c1b121b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:31.647833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:31.653505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33012a36-2894-4dca-aff9-b9981c1b121b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:31.664275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e8ed0-3330-4758-bbae-907b077363e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:31.665421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e8ed0-3330-4758-bbae-907b077363e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:31.665421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e8ed0-3330-4758-bbae-907b077363e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:31.666562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e8ed0-3330-4758-bbae-907b077363e8 admin admin] PATH_INFO: `/v3/projects/8376d8cb5f81436b98821522b7319e1b/users/7f09d39189554ff696660125bf3ba12e/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:31.669966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e8ed0-3330-4758-bbae-907b077363e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8376d8cb5f81436b98821522b7319e1b, user_id=7f09d39189554ff696660125bf3ba12e, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:31.674760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e8ed0-3330-4758-bbae-907b077363e8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:31.689345 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3991/8045] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:21:31 2022] PUT /identity/v3/projects/8376d8cb5f81436b98821522b7319e1b/users/7f09d39189554ff696660125bf3ba12e/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:31.700597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e762a-b81d-4906-9fa7-dd78f73e2dab None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:31.700784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e762a-b81d-4906-9fa7-dd78f73e2dab None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:31.701119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e762a-b81d-4906-9fa7-dd78f73e2dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:31.744825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-416e762a-b81d-4906-9fa7-dd78f73e2dab None None] MFA Rules not processed for user `7f09d39189554ff696660125bf3ba12e`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:31.746432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-416e762a-b81d-4906-9fa7-dd78f73e2dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:31.846846 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4055/8046] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:31 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:31.859568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e8ed0-3330-4758-bbae-907b077363e8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:31.862234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cc5e8ed0-3330-4758-bbae-907b077363e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:31.947818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e8ed0-3330-4758-bbae-907b077363e8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:31.955147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7b755-59e3-43f8-84d4-149a6b051008 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:31.955784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7b755-59e3-43f8-84d4-149a6b051008 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:31.956064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7b755-59e3-43f8-84d4-149a6b051008 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:31.956336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7b755-59e3-43f8-84d4-149a6b051008 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:31.967659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7b755-59e3-43f8-84d4-149a6b051008 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:31.969310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7b755-59e3-43f8-84d4-149a6b051008 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:31.980843 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3992/8047] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:21:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:21:32.098312 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e762a-b81d-4906-9fa7-dd78f73e2dab None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:32.120990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e762a-b81d-4906-9fa7-dd78f73e2dab None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:32.125502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee44e39-c755-43e1-850b-1d6bf9bb3b92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:32.126007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee44e39-c755-43e1-850b-1d6bf9bb3b92 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:32.126376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee44e39-c755-43e1-850b-1d6bf9bb3b92 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:32.126678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee44e39-c755-43e1-850b-1d6bf9bb3b92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:32.217843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee44e39-c755-43e1-850b-1d6bf9bb3b92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:32.219736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee44e39-c755-43e1-850b-1d6bf9bb3b92 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:32.232285 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4056/8048] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:21:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:21:32.751278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7b755-59e3-43f8-84d4-149a6b051008 service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:32.830310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7b755-59e3-43f8-84d4-149a6b051008 service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:32.844906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae8e66-2cb7-450f-b4fa-1304e13911b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:32.845594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae8e66-2cb7-450f-b4fa-1304e13911b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:32.845931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae8e66-2cb7-450f-b4fa-1304e13911b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:32.846141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae8e66-2cb7-450f-b4fa-1304e13911b9 admin admin] PATH_INFO: `/v3/users/21add4fa26f94702a091703c6d98e855` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:32.851573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae8e66-2cb7-450f-b4fa-1304e13911b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21add4fa26f94702a091703c6d98e855)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:32.853579 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae8e66-2cb7-450f-b4fa-1304e13911b9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:32.917390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2eae8e66-2cb7-450f-b4fa-1304e13911b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21add4fa26f94702a091703c6d98e855'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:32.917950 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2eae8e66-2cb7-450f-b4fa-1304e13911b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21add4fa26f94702a091703c6d98e855'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:32.918628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2eae8e66-2cb7-450f-b4fa-1304e13911b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21add4fa26f94702a091703c6d98e855'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:32.928165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2eae8e66-2cb7-450f-b4fa-1304e13911b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21add4fa26f94702a091703c6d98e855'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:32.942014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2eae8e66-2cb7-450f-b4fa-1304e13911b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21add4fa26f94702a091703c6d98e855'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:32.955953 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3993/8049] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:21:32 2022] DELETE /identity/v3/users/21add4fa26f94702a091703c6d98e855 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:21:32.960534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ae7a6-1001-44fa-84d7-ef14f3e87e46 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:32.960994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ae7a6-1001-44fa-84d7-ef14f3e87e46 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:32.961133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ae7a6-1001-44fa-84d7-ef14f3e87e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:32.981598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3c4ae7a6-1001-44fa-84d7-ef14f3e87e46 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:32.985328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3c4ae7a6-1001-44fa-84d7-ef14f3e87e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:33.033912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:33.057415 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4057/8050] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:21:33.067491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae8e66-2cb7-450f-b4fa-1304e13911b9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:33.069011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2eae8e66-2cb7-450f-b4fa-1304e13911b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:33.083554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:33.137637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae8e66-2cb7-450f-b4fa-1304e13911b9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:33.141531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32bf578-c360-45d1-8d07-6c5a3498bd2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:33.142141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bf578-c360-45d1-8d07-6c5a3498bd2b service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:33.142422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bf578-c360-45d1-8d07-6c5a3498bd2b service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:33.143531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bf578-c360-45d1-8d07-6c5a3498bd2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:33.168147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32bf578-c360-45d1-8d07-6c5a3498bd2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:33.169856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32bf578-c360-45d1-8d07-6c5a3498bd2b service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:33.188031 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3994/8051] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:21:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:21:33.509508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ae7a6-1001-44fa-84d7-ef14f3e87e46 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:33.511250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3c4ae7a6-1001-44fa-84d7-ef14f3e87e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:33.539345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ae7a6-1001-44fa-84d7-ef14f3e87e46 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:33.549189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281ca24-3af0-41e6-80d0-cc266a412bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:33.549920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281ca24-3af0-41e6-80d0-cc266a412bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:33.550413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281ca24-3af0-41e6-80d0-cc266a412bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:33.550700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281ca24-3af0-41e6-80d0-cc266a412bf6 admin admin] PATH_INFO: `/v3/projects/f748158a60e842d4b1bb6598eb610212` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:33.552565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5281ca24-3af0-41e6-80d0-cc266a412bf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f748158a60e842d4b1bb6598eb610212)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:33.554745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5281ca24-3af0-41e6-80d0-cc266a412bf6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:33.628872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5281ca24-3af0-41e6-80d0-cc266a412bf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f748158a60e842d4b1bb6598eb610212', 'request_id': 'req-5281ca24-3af0-41e6-80d0-cc266a412bf6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:33.641745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5281ca24-3af0-41e6-80d0-cc266a412bf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f748158a60e842d4b1bb6598eb610212', 'request_id': 'req-5281ca24-3af0-41e6-80d0-cc266a412bf6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:33.650030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5281ca24-3af0-41e6-80d0-cc266a412bf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f748158a60e842d4b1bb6598eb610212', 'request_id': 'req-5281ca24-3af0-41e6-80d0-cc266a412bf6'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:33.669944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5281ca24-3af0-41e6-80d0-cc266a412bf6 admin admin] The token cache is being invalidate because project f748158a60e842d4b1bb6598eb610212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:21:33.670452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5281ca24-3af0-41e6-80d0-cc266a412bf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:33.671476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5281ca24-3af0-41e6-80d0-cc266a412bf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:33.674291 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4058/8052] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:21:33 2022] DELETE /identity/v3/projects/f748158a60e842d4b1bb6598eb610212 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:21:33.679866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32bf578-c360-45d1-8d07-6c5a3498bd2b service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:33.692198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32bf578-c360-45d1-8d07-6c5a3498bd2b service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:33.765188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff3b7f-d033-4feb-8cd2-8c553f1aea49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:33.766992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff3b7f-d033-4feb-8cd2-8c553f1aea49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:33.767485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff3b7f-d033-4feb-8cd2-8c553f1aea49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:33.768136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff3b7f-d033-4feb-8cd2-8c553f1aea49 admin admin] PATH_INFO: `/v3/users/7f09d39189554ff696660125bf3ba12e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:33.769553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff3b7f-d033-4feb-8cd2-8c553f1aea49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f09d39189554ff696660125bf3ba12e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:33.775467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff3b7f-d033-4feb-8cd2-8c553f1aea49 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:33.872432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-10ff3b7f-d033-4feb-8cd2-8c553f1aea49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f09d39189554ff696660125bf3ba12e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:33.879158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-10ff3b7f-d033-4feb-8cd2-8c553f1aea49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f09d39189554ff696660125bf3ba12e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:33.881000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-10ff3b7f-d033-4feb-8cd2-8c553f1aea49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f09d39189554ff696660125bf3ba12e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:33.899924 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-10ff3b7f-d033-4feb-8cd2-8c553f1aea49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f09d39189554ff696660125bf3ba12e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:33.917036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-10ff3b7f-d033-4feb-8cd2-8c553f1aea49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f09d39189554ff696660125bf3ba12e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:33.935525 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3995/8053] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:21:33 2022] DELETE /identity/v3/users/7f09d39189554ff696660125bf3ba12e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:34.244889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281ca24-3af0-41e6-80d0-cc266a412bf6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:34.246386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5281ca24-3af0-41e6-80d0-cc266a412bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:34.269654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281ca24-3af0-41e6-80d0-cc266a412bf6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:34.347712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f4cc8-3352-49c1-832a-6fcbd09196a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:34.348507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f4cc8-3352-49c1-832a-6fcbd09196a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:34.348792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f4cc8-3352-49c1-832a-6fcbd09196a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:34.349785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f4cc8-3352-49c1-832a-6fcbd09196a3 admin admin] PATH_INFO: `/v3/projects/8376d8cb5f81436b98821522b7319e1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:34.356072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f4cc8-3352-49c1-832a-6fcbd09196a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8376d8cb5f81436b98821522b7319e1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:34.357996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f4cc8-3352-49c1-832a-6fcbd09196a3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:34.423932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-773f4cc8-3352-49c1-832a-6fcbd09196a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8376d8cb5f81436b98821522b7319e1b', 'request_id': 'req-773f4cc8-3352-49c1-832a-6fcbd09196a3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:34.431497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-773f4cc8-3352-49c1-832a-6fcbd09196a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8376d8cb5f81436b98821522b7319e1b', 'request_id': 'req-773f4cc8-3352-49c1-832a-6fcbd09196a3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:34.441600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-773f4cc8-3352-49c1-832a-6fcbd09196a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8376d8cb5f81436b98821522b7319e1b', 'request_id': 'req-773f4cc8-3352-49c1-832a-6fcbd09196a3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:34.458107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:34.462930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-773f4cc8-3352-49c1-832a-6fcbd09196a3 admin admin] The token cache is being invalidate because project 8376d8cb5f81436b98821522b7319e1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:21:34.463557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-773f4cc8-3352-49c1-832a-6fcbd09196a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:34.464243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-773f4cc8-3352-49c1-832a-6fcbd09196a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:21:34.466071 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4059/8054] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:21:34 2022] DELETE /identity/v3/projects/8376d8cb5f81436b98821522b7319e1b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:34.471079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa45fc9-88a7-49a2-95b4-4950f075861d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:34.471388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa45fc9-88a7-49a2-95b4-4950f075861d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:34.471647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa45fc9-88a7-49a2-95b4-4950f075861d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:34.493567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4aa45fc9-88a7-49a2-95b4-4950f075861d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:34.494808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4aa45fc9-88a7-49a2-95b4-4950f075861d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:34.523716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:34.558640 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3996/8055] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:34.566592 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59234f58-f5e0-419e-a049-e01260aa7fff None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:34.566887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59234f58-f5e0-419e-a049-e01260aa7fff None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:34.567166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59234f58-f5e0-419e-a049-e01260aa7fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:34.587122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-59234f58-f5e0-419e-a049-e01260aa7fff None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:34.588335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-59234f58-f5e0-419e-a049-e01260aa7fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:34.596691 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4060/8056] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:34.600982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa45fc9-88a7-49a2-95b4-4950f075861d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:34.610874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa45fc9-88a7-49a2-95b4-4950f075861d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:34.616201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c83c5-e435-44a3-855d-b8798ca05480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:34.616793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c83c5-e435-44a3-855d-b8798ca05480 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:34.617074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c83c5-e435-44a3-855d-b8798ca05480 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:34.617348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c83c5-e435-44a3-855d-b8798ca05480 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:34.617986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c83c5-e435-44a3-855d-b8798ca05480 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:21:34.618292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c83c5-e435-44a3-855d-b8798ca05480 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:34.619919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c83c5-e435-44a3-855d-b8798ca05480 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:34.635006 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3997/8057] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:21:34.636085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59234f58-f5e0-419e-a049-e01260aa7fff None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:34.645399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59234f58-f5e0-419e-a049-e01260aa7fff None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:34.650099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd2abf-bf81-4995-aad4-9687362edf50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:34.650729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd2abf-bf81-4995-aad4-9687362edf50 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:34.651012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd2abf-bf81-4995-aad4-9687362edf50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:34.651282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd2abf-bf81-4995-aad4-9687362edf50 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:34.651896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfd2abf-bf81-4995-aad4-9687362edf50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:34.653796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfd2abf-bf81-4995-aad4-9687362edf50 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:34.685124 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4061/8058] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:21:34 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:21:34.686263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c83c5-e435-44a3-855d-b8798ca05480 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:34.696559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c83c5-e435-44a3-855d-b8798ca05480 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:34.752900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5177b57-673c-4ef1-9a9e-abe3c7a171a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:34.753502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5177b57-673c-4ef1-9a9e-abe3c7a171a0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:34.753778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5177b57-673c-4ef1-9a9e-abe3c7a171a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:34.754047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5177b57-673c-4ef1-9a9e-abe3c7a171a0 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:34.754703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5177b57-673c-4ef1-9a9e-abe3c7a171a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:34.756648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5177b57-673c-4ef1-9a9e-abe3c7a171a0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:34.784485 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3998/8059] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:21:34 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:21:34.789316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd2abf-bf81-4995-aad4-9687362edf50 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:34.802579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd2abf-bf81-4995-aad4-9687362edf50 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:34.807450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d530d1-b850-4c24-b3fa-7000b2beb911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:34.808519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d530d1-b850-4c24-b3fa-7000b2beb911 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:34.808843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d530d1-b850-4c24-b3fa-7000b2beb911 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:34.809643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d530d1-b850-4c24-b3fa-7000b2beb911 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:34.810387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d530d1-b850-4c24-b3fa-7000b2beb911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:34.812526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d530d1-b850-4c24-b3fa-7000b2beb911 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:34.827250 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4062/8060] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:21:34.829314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5177b57-673c-4ef1-9a9e-abe3c7a171a0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:34.843035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5177b57-673c-4ef1-9a9e-abe3c7a171a0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:34.850067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f860705-55c2-4820-aa5e-3f531860e902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:34.851661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f860705-55c2-4820-aa5e-3f531860e902 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:34.851661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f860705-55c2-4820-aa5e-3f531860e902 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:34.851898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f860705-55c2-4820-aa5e-3f531860e902 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:34.852545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f860705-55c2-4820-aa5e-3f531860e902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:34.854540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f860705-55c2-4820-aa5e-3f531860e902 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:34.866444 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 3999/8061] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:34.870921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d530d1-b850-4c24-b3fa-7000b2beb911 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:34.881633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d530d1-b850-4c24-b3fa-7000b2beb911 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:34.886650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d877f53-f02e-4515-b9d4-ff14bb5e4a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:34.887404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d877f53-f02e-4515-b9d4-ff14bb5e4a32 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:34.887752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d877f53-f02e-4515-b9d4-ff14bb5e4a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:34.888436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d877f53-f02e-4515-b9d4-ff14bb5e4a32 admin admin] PATH_INFO: `/v3/projects/cca952fbe7c84efda0ecaf0dfb56a658/users/733cf56b323f4844b6578ea1f9a46cfb/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:34.903805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d877f53-f02e-4515-b9d4-ff14bb5e4a32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca952fbe7c84efda0ecaf0dfb56a658, user_id=733cf56b323f4844b6578ea1f9a46cfb, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:34.906575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d877f53-f02e-4515-b9d4-ff14bb5e4a32 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:34.926127 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4063/8062] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:21:34 2022] PUT /identity/v3/projects/cca952fbe7c84efda0ecaf0dfb56a658/users/733cf56b323f4844b6578ea1f9a46cfb/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:21:34.946460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb6bff-b40a-4f5f-803c-2ddaf32f3aee None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:34.946460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb6bff-b40a-4f5f-803c-2ddaf32f3aee None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:34.946460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb6bff-b40a-4f5f-803c-2ddaf32f3aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:34.999882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2adb6bff-b40a-4f5f-803c-2ddaf32f3aee None None] MFA Rules not processed for user `733cf56b323f4844b6578ea1f9a46cfb`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:35.001383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2adb6bff-b40a-4f5f-803c-2ddaf32f3aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:35.140558 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4000/8063] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:35.144083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d877f53-f02e-4515-b9d4-ff14bb5e4a32 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:35.236480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d877f53-f02e-4515-b9d4-ff14bb5e4a32 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:35.244336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7260e-07ef-4d14-9514-072be5909bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:35.245364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7260e-07ef-4d14-9514-072be5909bbe admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:35.246381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7260e-07ef-4d14-9514-072be5909bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:35.246942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7260e-07ef-4d14-9514-072be5909bbe admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:35.247605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d7260e-07ef-4d14-9514-072be5909bbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:35.249576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d7260e-07ef-4d14-9514-072be5909bbe admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:35.285986 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4064/8064] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:21:35 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:21:35.290417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb6bff-b40a-4f5f-803c-2ddaf32f3aee None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:35.303687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb6bff-b40a-4f5f-803c-2ddaf32f3aee None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:35.334585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:35.374379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdffc61-34f6-4c45-b132-bf5d0a4af832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:35.375039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdffc61-34f6-4c45-b132-bf5d0a4af832 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:35.375340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdffc61-34f6-4c45-b132-bf5d0a4af832 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:35.375620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdffc61-34f6-4c45-b132-bf5d0a4af832 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:35.376249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdffc61-34f6-4c45-b132-bf5d0a4af832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:35.378473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdffc61-34f6-4c45-b132-bf5d0a4af832 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:35.404159 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4001/8065] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:21:35 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:21:35.405674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7260e-07ef-4d14-9514-072be5909bbe admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:35.420451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7260e-07ef-4d14-9514-072be5909bbe admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:35.425986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244a127-12e8-4b7c-83c7-bef63f4315d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:35.426717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244a127-12e8-4b7c-83c7-bef63f4315d0 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:35.427702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244a127-12e8-4b7c-83c7-bef63f4315d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:35.428040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244a127-12e8-4b7c-83c7-bef63f4315d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:35.428771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1244a127-12e8-4b7c-83c7-bef63f4315d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:35.430485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1244a127-12e8-4b7c-83c7-bef63f4315d0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:35.448196 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4065/8066] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:35.452404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdffc61-34f6-4c45-b132-bf5d0a4af832 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:35.466182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdffc61-34f6-4c45-b132-bf5d0a4af832 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:35.471199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c8bfe-8aba-46d7-8f22-dadd874f7e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:35.471871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c8bfe-8aba-46d7-8f22-dadd874f7e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:35.472188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c8bfe-8aba-46d7-8f22-dadd874f7e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:35.472522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c8bfe-8aba-46d7-8f22-dadd874f7e64 admin admin] PATH_INFO: `/v3/projects/1964fa936eba471c88f88c7df5dd559d/users/84fc36afafa04944b67aa2988c51efca/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:35.490186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c8bfe-8aba-46d7-8f22-dadd874f7e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1964fa936eba471c88f88c7df5dd559d, user_id=84fc36afafa04944b67aa2988c51efca, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:35.493115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c8bfe-8aba-46d7-8f22-dadd874f7e64 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:35.511862 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4002/8067] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:21:35 2022] PUT /identity/v3/projects/1964fa936eba471c88f88c7df5dd559d/users/84fc36afafa04944b67aa2988c51efca/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:21:35.515833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244a127-12e8-4b7c-83c7-bef63f4315d0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:35.531340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244a127-12e8-4b7c-83c7-bef63f4315d0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:35.596432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb536112-b650-4d46-a5bf-83e63dc38f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:35.597042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb536112-b650-4d46-a5bf-83e63dc38f02 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:35.597319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb536112-b650-4d46-a5bf-83e63dc38f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:35.597590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb536112-b650-4d46-a5bf-83e63dc38f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:35.598255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb536112-b650-4d46-a5bf-83e63dc38f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:35.600412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb536112-b650-4d46-a5bf-83e63dc38f02 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:35.614620 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4066/8068] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:21:35.615368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c8bfe-8aba-46d7-8f22-dadd874f7e64 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:35.626682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c8bfe-8aba-46d7-8f22-dadd874f7e64 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:35.631825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0875d6-a1bd-468c-a235-d2b4e741ea29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:35.632453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0875d6-a1bd-468c-a235-d2b4e741ea29 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:35.632731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0875d6-a1bd-468c-a235-d2b4e741ea29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:35.633176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0875d6-a1bd-468c-a235-d2b4e741ea29 admin admin] PATH_INFO: `/v3/projects/1964fa936eba471c88f88c7df5dd559d/users/84fc36afafa04944b67aa2988c51efca/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:35.635203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0875d6-a1bd-468c-a235-d2b4e741ea29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1964fa936eba471c88f88c7df5dd559d, user_id=84fc36afafa04944b67aa2988c51efca, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:35.638479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0875d6-a1bd-468c-a235-d2b4e741ea29 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:35.650331 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4003/8069] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:21:35 2022] PUT /identity/v3/projects/1964fa936eba471c88f88c7df5dd559d/users/84fc36afafa04944b67aa2988c51efca/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:21:35.660885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f750d5e-f155-41d8-b8be-4610ae19f49e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:35.661027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f750d5e-f155-41d8-b8be-4610ae19f49e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:35.661890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f750d5e-f155-41d8-b8be-4610ae19f49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:35.671632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:35.707104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1f750d5e-f155-41d8-b8be-4610ae19f49e None None] MFA Rules not processed for user `84fc36afafa04944b67aa2988c51efca`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:35.708869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1f750d5e-f155-41d8-b8be-4610ae19f49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:35.784770 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4067/8070] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:35.799790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0875d6-a1bd-468c-a235-d2b4e741ea29 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:35.802188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6d0875d6-a1bd-468c-a235-d2b4e741ea29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:35.886812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0875d6-a1bd-468c-a235-d2b4e741ea29 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:35.891052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c144e-a3cf-41ae-b703-52ce5e7a1fca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:35.891948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c144e-a3cf-41ae-b703-52ce5e7a1fca service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:35.892298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c144e-a3cf-41ae-b703-52ce5e7a1fca service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:35.892636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c144e-a3cf-41ae-b703-52ce5e7a1fca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:35.968872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c144e-a3cf-41ae-b703-52ce5e7a1fca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:35.970732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c144e-a3cf-41ae-b703-52ce5e7a1fca service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:35.982873 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4004/8071] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:21:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:21:43.289390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f750d5e-f155-41d8-b8be-4610ae19f49e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:43.290979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1f750d5e-f155-41d8-b8be-4610ae19f49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:43.320848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f750d5e-f155-41d8-b8be-4610ae19f49e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:43.324685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ecc61-a3a8-4740-8d3a-a2db7a737e51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:43.325294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ecc61-a3a8-4740-8d3a-a2db7a737e51 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:43.325575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ecc61-a3a8-4740-8d3a-a2db7a737e51 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:43.325848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ecc61-a3a8-4740-8d3a-a2db7a737e51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:43.335070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ecc61-a3a8-4740-8d3a-a2db7a737e51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:43.336474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ecc61-a3a8-4740-8d3a-a2db7a737e51 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:43.345960 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4068/8072] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:21:43 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:21:45.978135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=79,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:48.318415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88386f-407c-4666-9d7e-95805233e15b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:48.319698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88386f-407c-4666-9d7e-95805233e15b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:48.319999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88386f-407c-4666-9d7e-95805233e15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:48.349367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ca88386f-407c-4666-9d7e-95805233e15b None None] MFA Rules not processed for user `eb0bf5fce885439eaa16a0871c2992ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:48.351104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ca88386f-407c-4666-9d7e-95805233e15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:48.464173 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4005/8073] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:50.543422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ecc61-a3a8-4740-8d3a-a2db7a737e51 service cinder] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:50.545751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f82ecc61-a3a8-4740-8d3a-a2db7a737e51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:50.763112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ecc61-a3a8-4740-8d3a-a2db7a737e51 service cinder] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:50.777455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d1274-d9a8-425d-9959-f6da0f09b78d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:50.778673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d1274-d9a8-425d-9959-f6da0f09b78d service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:50.778997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d1274-d9a8-425d-9959-f6da0f09b78d service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:50.779268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d1274-d9a8-425d-9959-f6da0f09b78d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:50.788655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1b7d1274-d9a8-425d-9959-f6da0f09b78d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:50.817555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d1274-d9a8-425d-9959-f6da0f09b78d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:50.819697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d1274-d9a8-425d-9959-f6da0f09b78d service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:50.854479 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4069/8074] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:21:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:21:55.407435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e8def-4850-43e5-a588-758bb038e648 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:55.408332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e8def-4850-43e5-a588-758bb038e648 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:55.409619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e8def-4850-43e5-a588-758bb038e648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:55.442208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6d9e8def-4850-43e5-a588-758bb038e648 None None] MFA Rules not processed for user `55dadd491ec1441aac4171eff46bd679`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:21:55.443469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6d9e8def-4850-43e5-a588-758bb038e648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:55.521255 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4006/8075] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:21:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:21:59.652843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d1274-d9a8-425d-9959-f6da0f09b78d service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:21:59.654788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1b7d1274-d9a8-425d-9959-f6da0f09b78d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:59.673959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d1274-d9a8-425d-9959-f6da0f09b78d service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:21:59.679915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6af2a-dfbc-4265-a610-78a1e32bb0ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:21:59.680739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6af2a-dfbc-4265-a610-78a1e32bb0ad service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:21:59.680850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6af2a-dfbc-4265-a610-78a1e32bb0ad service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:21:59.681116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6af2a-dfbc-4265-a610-78a1e32bb0ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:21:59.683024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-24f6af2a-dfbc-4265-a610-78a1e32bb0ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:21:59.699590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6af2a-dfbc-4265-a610-78a1e32bb0ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:21:59.701260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6af2a-dfbc-4265-a610-78a1e32bb0ad service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:21:59.715645 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4070/8076] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:21:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:22:02.899381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e8def-4850-43e5-a588-758bb038e648 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:02.975483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e8def-4850-43e5-a588-758bb038e648 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:02.980676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d4c83-a073-4e6c-a068-4be06e4b7414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:02.981299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d4c83-a073-4e6c-a068-4be06e4b7414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:02.981767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d4c83-a073-4e6c-a068-4be06e4b7414 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:02.982045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d4c83-a073-4e6c-a068-4be06e4b7414 admin admin] PATH_INFO: `/v3/users/733cf56b323f4844b6578ea1f9a46cfb` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:02.983378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d4c83-a073-4e6c-a068-4be06e4b7414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=733cf56b323f4844b6578ea1f9a46cfb)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:02.985289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d4c83-a073-4e6c-a068-4be06e4b7414 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:03.050142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-af0d4c83-a073-4e6c-a068-4be06e4b7414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '733cf56b323f4844b6578ea1f9a46cfb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:03.050895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-af0d4c83-a073-4e6c-a068-4be06e4b7414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '733cf56b323f4844b6578ea1f9a46cfb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:03.051744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-af0d4c83-a073-4e6c-a068-4be06e4b7414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '733cf56b323f4844b6578ea1f9a46cfb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:03.055882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:03.061442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-af0d4c83-a073-4e6c-a068-4be06e4b7414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '733cf56b323f4844b6578ea1f9a46cfb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:03.075745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-af0d4c83-a073-4e6c-a068-4be06e4b7414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '733cf56b323f4844b6578ea1f9a46cfb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:03.090811 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4007/8077] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:22:02 2022] DELETE /identity/v3/users/733cf56b323f4844b6578ea1f9a46cfb => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:22:03.095433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd4779-2640-47e6-9684-8f5db7f37f5b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:03.095920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd4779-2640-47e6-9684-8f5db7f37f5b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:03.095920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd4779-2640-47e6-9684-8f5db7f37f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:03.115517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-32fd4779-2640-47e6-9684-8f5db7f37f5b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:22:03.116718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-32fd4779-2640-47e6-9684-8f5db7f37f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:03.147654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:03.190345 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4071/8078] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:22:03.202736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d4c83-a073-4e6c-a068-4be06e4b7414 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:03.205055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-af0d4c83-a073-4e6c-a068-4be06e4b7414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:03.280235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d4c83-a073-4e6c-a068-4be06e4b7414 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:03.284134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db452fd3-6b23-421e-b6a4-77df4989dd4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:03.284858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db452fd3-6b23-421e-b6a4-77df4989dd4e service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:03.285258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db452fd3-6b23-421e-b6a4-77df4989dd4e service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:03.285619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db452fd3-6b23-421e-b6a4-77df4989dd4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:03.296773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db452fd3-6b23-421e-b6a4-77df4989dd4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:03.298515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db452fd3-6b23-421e-b6a4-77df4989dd4e service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:03.310900 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4008/8079] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:22:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:22:03.525662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c9b2a-4eef-439d-8a15-d0f1d8a1b566 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:03.526031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c9b2a-4eef-439d-8a15-d0f1d8a1b566 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:03.526386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c9b2a-4eef-439d-8a15-d0f1d8a1b566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:03.546670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ea6c9b2a-4eef-439d-8a15-d0f1d8a1b566 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:22:03.547884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ea6c9b2a-4eef-439d-8a15-d0f1d8a1b566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:03.597044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db452fd3-6b23-421e-b6a4-77df4989dd4e service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:03.599395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-db452fd3-6b23-421e-b6a4-77df4989dd4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:03.613281 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4072/8080] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:22:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:22:03.628949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db452fd3-6b23-421e-b6a4-77df4989dd4e service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:03.635615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef2aed-1f88-4b80-9e12-2a4206333e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:03.636350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef2aed-1f88-4b80-9e12-2a4206333e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:03.636632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef2aed-1f88-4b80-9e12-2a4206333e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:03.636920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef2aed-1f88-4b80-9e12-2a4206333e3e admin admin] PATH_INFO: `/v3/projects/cca952fbe7c84efda0ecaf0dfb56a658` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:03.638437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef2aed-1f88-4b80-9e12-2a4206333e3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cca952fbe7c84efda0ecaf0dfb56a658)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:03.640031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef2aed-1f88-4b80-9e12-2a4206333e3e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:03.670800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c9b2a-4eef-439d-8a15-d0f1d8a1b566 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:03.673097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ea6c9b2a-4eef-439d-8a15-d0f1d8a1b566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:03.691283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-10ef2aed-1f88-4b80-9e12-2a4206333e3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cca952fbe7c84efda0ecaf0dfb56a658', 'request_id': 'req-10ef2aed-1f88-4b80-9e12-2a4206333e3e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:03.699759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-10ef2aed-1f88-4b80-9e12-2a4206333e3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cca952fbe7c84efda0ecaf0dfb56a658', 'request_id': 'req-10ef2aed-1f88-4b80-9e12-2a4206333e3e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:03.707201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-10ef2aed-1f88-4b80-9e12-2a4206333e3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cca952fbe7c84efda0ecaf0dfb56a658', 'request_id': 'req-10ef2aed-1f88-4b80-9e12-2a4206333e3e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:03.718156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-10ef2aed-1f88-4b80-9e12-2a4206333e3e admin admin] The token cache is being invalidate because project cca952fbe7c84efda0ecaf0dfb56a658 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:22:03.718444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-10ef2aed-1f88-4b80-9e12-2a4206333e3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:03.719119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-10ef2aed-1f88-4b80-9e12-2a4206333e3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:03.720594 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4009/8081] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:22:03 2022] DELETE /identity/v3/projects/cca952fbe7c84efda0ecaf0dfb56a658 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:22:03.725083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef2aed-1f88-4b80-9e12-2a4206333e3e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:03.726447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-10ef2aed-1f88-4b80-9e12-2a4206333e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:03.764744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c9b2a-4eef-439d-8a15-d0f1d8a1b566 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:03.797865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:03.808325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef2aed-1f88-4b80-9e12-2a4206333e3e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:03.815252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb7bc8-dc0f-4fb0-acf6-73f2f3bc3921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:03.816319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb7bc8-dc0f-4fb0-acf6-73f2f3bc3921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:03.817064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb7bc8-dc0f-4fb0-acf6-73f2f3bc3921 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:03.817453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb7bc8-dc0f-4fb0-acf6-73f2f3bc3921 admin admin] PATH_INFO: `/v3/users/84fc36afafa04944b67aa2988c51efca` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:03.819176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb7bc8-dc0f-4fb0-acf6-73f2f3bc3921 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84fc36afafa04944b67aa2988c51efca)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:03.821791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb7bc8-dc0f-4fb0-acf6-73f2f3bc3921 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:03.843219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cfe9ad-212d-4379-8e08-d3d2da3a2ca2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:03.844018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfe9ad-212d-4379-8e08-d3d2da3a2ca2 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:03.844166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfe9ad-212d-4379-8e08-d3d2da3a2ca2 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:03.844375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfe9ad-212d-4379-8e08-d3d2da3a2ca2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:03.846600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-29cfe9ad-212d-4379-8e08-d3d2da3a2ca2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:03.862105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:03.902380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-52eb7bc8-dc0f-4fb0-acf6-73f2f3bc3921 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84fc36afafa04944b67aa2988c51efca'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:03.903859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-52eb7bc8-dc0f-4fb0-acf6-73f2f3bc3921 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84fc36afafa04944b67aa2988c51efca'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:03.904611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-52eb7bc8-dc0f-4fb0-acf6-73f2f3bc3921 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84fc36afafa04944b67aa2988c51efca'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:03.916335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-52eb7bc8-dc0f-4fb0-acf6-73f2f3bc3921 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84fc36afafa04944b67aa2988c51efca'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:03.928189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cfe9ad-212d-4379-8e08-d3d2da3a2ca2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:03.931144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cfe9ad-212d-4379-8e08-d3d2da3a2ca2 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:03.932111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-52eb7bc8-dc0f-4fb0-acf6-73f2f3bc3921 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84fc36afafa04944b67aa2988c51efca'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:03.943914 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4073/8082] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:22:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 21:22:03.969723 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4010/8083] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:22:03 2022] DELETE /identity/v3/users/84fc36afafa04944b67aa2988c51efca => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:22:04.350076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bbb90-13d4-4956-ad43-d10ac1b2c34a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:04.350684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bbb90-13d4-4956-ad43-d10ac1b2c34a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:04.350684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bbb90-13d4-4956-ad43-d10ac1b2c34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:04.382427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f77bbb90-13d4-4956-ad43-d10ac1b2c34a None None] MFA Rules not processed for user `eb0bf5fce885439eaa16a0871c2992ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:22:04.384359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f77bbb90-13d4-4956-ad43-d10ac1b2c34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:04.426942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb7bc8-dc0f-4fb0-acf6-73f2f3bc3921 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:04.428376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-52eb7bc8-dc0f-4fb0-acf6-73f2f3bc3921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:04.491717 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4074/8084] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:22:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:22:04.539454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb7bc8-dc0f-4fb0-acf6-73f2f3bc3921 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:04.546667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68815e-4b78-43a6-91ef-c3f1b5a3b9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:04.547664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68815e-4b78-43a6-91ef-c3f1b5a3b9c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:04.547983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68815e-4b78-43a6-91ef-c3f1b5a3b9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:04.548277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68815e-4b78-43a6-91ef-c3f1b5a3b9c2 admin admin] PATH_INFO: `/v3/projects/1964fa936eba471c88f88c7df5dd559d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:04.549738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68815e-4b78-43a6-91ef-c3f1b5a3b9c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1964fa936eba471c88f88c7df5dd559d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:04.552667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68815e-4b78-43a6-91ef-c3f1b5a3b9c2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:04.607852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8e68815e-4b78-43a6-91ef-c3f1b5a3b9c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1964fa936eba471c88f88c7df5dd559d', 'request_id': 'req-8e68815e-4b78-43a6-91ef-c3f1b5a3b9c2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:04.615808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8e68815e-4b78-43a6-91ef-c3f1b5a3b9c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1964fa936eba471c88f88c7df5dd559d', 'request_id': 'req-8e68815e-4b78-43a6-91ef-c3f1b5a3b9c2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:04.621505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8e68815e-4b78-43a6-91ef-c3f1b5a3b9c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1964fa936eba471c88f88c7df5dd559d', 'request_id': 'req-8e68815e-4b78-43a6-91ef-c3f1b5a3b9c2'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:04.633791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8e68815e-4b78-43a6-91ef-c3f1b5a3b9c2 admin admin] The token cache is being invalidate because project 1964fa936eba471c88f88c7df5dd559d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:22:04.634085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8e68815e-4b78-43a6-91ef-c3f1b5a3b9c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:04.635201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8e68815e-4b78-43a6-91ef-c3f1b5a3b9c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:04.636585 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4011/8085] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:22:04 2022] DELETE /identity/v3/projects/1964fa936eba471c88f88c7df5dd559d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:22:04.641488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a8c87-2a5f-4b45-8bda-8c408291fd74 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:04.641774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a8c87-2a5f-4b45-8bda-8c408291fd74 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:04.642074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a8c87-2a5f-4b45-8bda-8c408291fd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:04.662626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7b2a8c87-2a5f-4b45-8bda-8c408291fd74 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:22:04.663846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7b2a8c87-2a5f-4b45-8bda-8c408291fd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:04.726384 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4075/8086] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:22:04.735325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523604c1-9e10-43c5-8d91-dd5444899c62 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:04.735623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523604c1-9e10-43c5-8d91-dd5444899c62 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:04.735904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523604c1-9e10-43c5-8d91-dd5444899c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:04.744767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:04.757426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-523604c1-9e10-43c5-8d91-dd5444899c62 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:22:04.759495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-523604c1-9e10-43c5-8d91-dd5444899c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:04.767687 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4012/8087] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:22:04.772426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a8c87-2a5f-4b45-8bda-8c408291fd74 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:04.782379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a8c87-2a5f-4b45-8bda-8c408291fd74 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:04.788505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a27b60-3c1e-47e6-a457-7561893f42be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:04.789200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a27b60-3c1e-47e6-a457-7561893f42be admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:04.789552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a27b60-3c1e-47e6-a457-7561893f42be admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:04.789900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a27b60-3c1e-47e6-a457-7561893f42be admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:04.790693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a27b60-3c1e-47e6-a457-7561893f42be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:22:04.791068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a27b60-3c1e-47e6-a457-7561893f42be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:04.792755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a27b60-3c1e-47e6-a457-7561893f42be admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:04.805451 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4076/8088] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:22:04.809777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523604c1-9e10-43c5-8d91-dd5444899c62 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:04.819182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523604c1-9e10-43c5-8d91-dd5444899c62 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:04.823880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38d4e6-4db9-4b6a-949e-2c0345501f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:04.824472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38d4e6-4db9-4b6a-949e-2c0345501f93 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:04.824756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38d4e6-4db9-4b6a-949e-2c0345501f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:04.825030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38d4e6-4db9-4b6a-949e-2c0345501f93 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:04.825647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38d4e6-4db9-4b6a-949e-2c0345501f93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:04.827618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38d4e6-4db9-4b6a-949e-2c0345501f93 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:04.860857 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4013/8089] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:22:04 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:22:04.862051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a27b60-3c1e-47e6-a457-7561893f42be admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:04.872183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a27b60-3c1e-47e6-a457-7561893f42be admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:04.920075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:04.940020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d24dbc-6b7f-4a96-bc3c-3481a84f0dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:04.940649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d24dbc-6b7f-4a96-bc3c-3481a84f0dec admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:04.941021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d24dbc-6b7f-4a96-bc3c-3481a84f0dec admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:04.941353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d24dbc-6b7f-4a96-bc3c-3481a84f0dec admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:04.941902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d24dbc-6b7f-4a96-bc3c-3481a84f0dec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:04.943922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d24dbc-6b7f-4a96-bc3c-3481a84f0dec admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:04.966293 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4077/8090] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:22:04 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:22:04.970451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38d4e6-4db9-4b6a-949e-2c0345501f93 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:04.981753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38d4e6-4db9-4b6a-949e-2c0345501f93 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:04.986894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f6055-9779-4415-bfbc-591254c2c2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:04.987627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f6055-9779-4415-bfbc-591254c2c2f6 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:04.987998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f6055-9779-4415-bfbc-591254c2c2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:04.988366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f6055-9779-4415-bfbc-591254c2c2f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:04.989066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f6055-9779-4415-bfbc-591254c2c2f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:04.990744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f6055-9779-4415-bfbc-591254c2c2f6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:05.002527 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4014/8091] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:22:05.005027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d24dbc-6b7f-4a96-bc3c-3481a84f0dec admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:05.015085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d24dbc-6b7f-4a96-bc3c-3481a84f0dec admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:05.020441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34e311-91c6-4321-a664-77b6cc5f1aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:05.021074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34e311-91c6-4321-a664-77b6cc5f1aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:05.021425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34e311-91c6-4321-a664-77b6cc5f1aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:05.021763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34e311-91c6-4321-a664-77b6cc5f1aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:05.022475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34e311-91c6-4321-a664-77b6cc5f1aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:05.024152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34e311-91c6-4321-a664-77b6cc5f1aa1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:05.038527 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4078/8092] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:22:05.043698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f6055-9779-4415-bfbc-591254c2c2f6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:05.055117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f6055-9779-4415-bfbc-591254c2c2f6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:05.060638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd38570-75ae-4a99-af22-482a8fd54240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:05.061355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd38570-75ae-4a99-af22-482a8fd54240 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:05.061785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd38570-75ae-4a99-af22-482a8fd54240 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:05.062072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd38570-75ae-4a99-af22-482a8fd54240 admin admin] PATH_INFO: `/v3/projects/410af66d56c34146994cac962b73fbfd/users/22c02150477f4e98a9aa7fd0247def02/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:05.082610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd38570-75ae-4a99-af22-482a8fd54240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=410af66d56c34146994cac962b73fbfd, user_id=22c02150477f4e98a9aa7fd0247def02, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:05.085707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd38570-75ae-4a99-af22-482a8fd54240 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:05.104513 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4015/8093] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:22:05 2022] PUT /identity/v3/projects/410af66d56c34146994cac962b73fbfd/users/22c02150477f4e98a9aa7fd0247def02/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:22:05.115302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae8ac6-7aa4-4289-97b0-97d169624e9d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:05.115515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae8ac6-7aa4-4289-97b0-97d169624e9d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:05.115772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae8ac6-7aa4-4289-97b0-97d169624e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:05.149534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-18ae8ac6-7aa4-4289-97b0-97d169624e9d None None] MFA Rules not processed for user `22c02150477f4e98a9aa7fd0247def02`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:22:05.150789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-18ae8ac6-7aa4-4289-97b0-97d169624e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:05.236373 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4079/8094] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:22:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:22:05.240962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd38570-75ae-4a99-af22-482a8fd54240 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:05.318580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd38570-75ae-4a99-af22-482a8fd54240 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:05.325250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e59bbd-de20-4584-93b0-d352bc5696d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:05.325933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e59bbd-de20-4584-93b0-d352bc5696d1 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:05.326223 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e59bbd-de20-4584-93b0-d352bc5696d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:05.326484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e59bbd-de20-4584-93b0-d352bc5696d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:05.327279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e59bbd-de20-4584-93b0-d352bc5696d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:05.329767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e59bbd-de20-4584-93b0-d352bc5696d1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:05.360402 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4016/8095] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:22:05 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:22:05.364642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae8ac6-7aa4-4289-97b0-97d169624e9d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:05.383550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae8ac6-7aa4-4289-97b0-97d169624e9d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:05.453189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18184bda-9450-421e-b9b8-e58ea9299edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:05.454039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18184bda-9450-421e-b9b8-e58ea9299edc admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:05.454518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18184bda-9450-421e-b9b8-e58ea9299edc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:05.454953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18184bda-9450-421e-b9b8-e58ea9299edc admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:05.456398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18184bda-9450-421e-b9b8-e58ea9299edc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:05.459909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18184bda-9450-421e-b9b8-e58ea9299edc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:05.489410 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4080/8096] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:22:05 2022] POST /identity/v3/users => generated 298 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:22:05.493115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e59bbd-de20-4584-93b0-d352bc5696d1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:05.504884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e59bbd-de20-4584-93b0-d352bc5696d1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:05.510794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb9ea5-0cec-4b7a-8629-555f23fed715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:05.511405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb9ea5-0cec-4b7a-8629-555f23fed715 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:05.511684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb9ea5-0cec-4b7a-8629-555f23fed715 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:05.511951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb9ea5-0cec-4b7a-8629-555f23fed715 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:05.512588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb9ea5-0cec-4b7a-8629-555f23fed715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:05.514339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb9ea5-0cec-4b7a-8629-555f23fed715 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:05.525714 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4017/8097] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:22:05.530024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18184bda-9450-421e-b9b8-e58ea9299edc admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:05.540006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18184bda-9450-421e-b9b8-e58ea9299edc admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:05.544963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d2cc5-c1ae-4163-aafd-6c938f1901ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:05.545617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d2cc5-c1ae-4163-aafd-6c938f1901ee admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:05.545894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d2cc5-c1ae-4163-aafd-6c938f1901ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:05.546166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d2cc5-c1ae-4163-aafd-6c938f1901ee admin admin] PATH_INFO: `/v3/projects/12e7b51181e04f3eb5dcc48e45dabde5/users/51642c736773414886ad03bd940c8778/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:05.562572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d2cc5-c1ae-4163-aafd-6c938f1901ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=12e7b51181e04f3eb5dcc48e45dabde5, user_id=51642c736773414886ad03bd940c8778, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:05.565255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d2cc5-c1ae-4163-aafd-6c938f1901ee admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:05.578714 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4081/8098] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:22:05 2022] PUT /identity/v3/projects/12e7b51181e04f3eb5dcc48e45dabde5/users/51642c736773414886ad03bd940c8778/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:22:05.583392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb9ea5-0cec-4b7a-8629-555f23fed715 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:05.593766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb9ea5-0cec-4b7a-8629-555f23fed715 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:05.656778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe519f8-16b7-4e34-a122-60e753c00447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:05.657298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe519f8-16b7-4e34-a122-60e753c00447 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:05.657578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe519f8-16b7-4e34-a122-60e753c00447 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:05.657847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe519f8-16b7-4e34-a122-60e753c00447 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:05.658502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe519f8-16b7-4e34-a122-60e753c00447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:05.660121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe519f8-16b7-4e34-a122-60e753c00447 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:05.671569 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4018/8099] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:22:05.676197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d2cc5-c1ae-4163-aafd-6c938f1901ee admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:05.686700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d2cc5-c1ae-4163-aafd-6c938f1901ee admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:05.692015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb18c1d-bd4e-48a0-8d54-4fcc376702be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:05.692653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb18c1d-bd4e-48a0-8d54-4fcc376702be admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:05.693233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb18c1d-bd4e-48a0-8d54-4fcc376702be admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:05.693678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb18c1d-bd4e-48a0-8d54-4fcc376702be admin admin] PATH_INFO: `/v3/projects/12e7b51181e04f3eb5dcc48e45dabde5/users/51642c736773414886ad03bd940c8778/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:05.696578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb18c1d-bd4e-48a0-8d54-4fcc376702be admin admin] RBAC: Authorizing `identity:create_grant(project_id=12e7b51181e04f3eb5dcc48e45dabde5, user_id=51642c736773414886ad03bd940c8778, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:05.699428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb18c1d-bd4e-48a0-8d54-4fcc376702be admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:05.710580 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4082/8100] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:22:05 2022] PUT /identity/v3/projects/12e7b51181e04f3eb5dcc48e45dabde5/users/51642c736773414886ad03bd940c8778/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:22:05.723208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d8fa0-0110-4a75-8e95-1690ba0d9b2f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:05.723208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d8fa0-0110-4a75-8e95-1690ba0d9b2f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:05.723208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d8fa0-0110-4a75-8e95-1690ba0d9b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:05.745092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:05.758102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-eb2d8fa0-0110-4a75-8e95-1690ba0d9b2f None None] MFA Rules not processed for user `51642c736773414886ad03bd940c8778`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:22:05.759409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-eb2d8fa0-0110-4a75-8e95-1690ba0d9b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:05.839091 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4019/8101] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:22:05 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:22:05.851299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb18c1d-bd4e-48a0-8d54-4fcc376702be admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:05.853123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-deb18c1d-bd4e-48a0-8d54-4fcc376702be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:05.873927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:05.943704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb18c1d-bd4e-48a0-8d54-4fcc376702be admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:05.948627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4adc5-9579-445a-9bca-5d5b0bb6ca7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:05.949319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4adc5-9579-445a-9bca-5d5b0bb6ca7b service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:05.949675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4adc5-9579-445a-9bca-5d5b0bb6ca7b service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:05.950028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4adc5-9579-445a-9bca-5d5b0bb6ca7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:06.026342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4adc5-9579-445a-9bca-5d5b0bb6ca7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:06.027880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4adc5-9579-445a-9bca-5d5b0bb6ca7b service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:06.037203 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4083/8102] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:22:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:22:08.039841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d8fa0-0110-4a75-8e95-1690ba0d9b2f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:08.041371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-eb2d8fa0-0110-4a75-8e95-1690ba0d9b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:08.115471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d8fa0-0110-4a75-8e95-1690ba0d9b2f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:08.119647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146d44e-9fe8-4540-b7b6-f4f05bbbed9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:08.120292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146d44e-9fe8-4540-b7b6-f4f05bbbed9e service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:08.120577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146d44e-9fe8-4540-b7b6-f4f05bbbed9e service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:08.120851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146d44e-9fe8-4540-b7b6-f4f05bbbed9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:08.122684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6146d44e-9fe8-4540-b7b6-f4f05bbbed9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:08.220795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146d44e-9fe8-4540-b7b6-f4f05bbbed9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:08.222356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146d44e-9fe8-4540-b7b6-f4f05bbbed9e service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:08.234642 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4020/8103] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:22:08 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:22:10.611434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4adc5-9579-445a-9bca-5d5b0bb6ca7b service cinder] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:10.625621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4adc5-9579-445a-9bca-5d5b0bb6ca7b service cinder] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:10.633304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00500a-0ab0-48d8-98b1-8e2b1263ac52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:10.633586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00500a-0ab0-48d8-98b1-8e2b1263ac52 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:10.633982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00500a-0ab0-48d8-98b1-8e2b1263ac52 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:10.634325 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00500a-0ab0-48d8-98b1-8e2b1263ac52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:10.654534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00500a-0ab0-48d8-98b1-8e2b1263ac52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:10.656208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00500a-0ab0-48d8-98b1-8e2b1263ac52 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:10.668400 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4084/8104] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:22:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:22:15.820057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146d44e-9fe8-4540-b7b6-f4f05bbbed9e service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:15.868655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:15.883045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146d44e-9fe8-4540-b7b6-f4f05bbbed9e service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:15.887676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d3d88-e39c-4b8c-a48c-383cf89c7925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:15.888302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d3d88-e39c-4b8c-a48c-383cf89c7925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:15.888578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d3d88-e39c-4b8c-a48c-383cf89c7925 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:15.888849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d3d88-e39c-4b8c-a48c-383cf89c7925 admin admin] PATH_INFO: `/v3/users/22c02150477f4e98a9aa7fd0247def02` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:15.889958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d3d88-e39c-4b8c-a48c-383cf89c7925 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22c02150477f4e98a9aa7fd0247def02)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:15.891890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d3d88-e39c-4b8c-a48c-383cf89c7925 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:15.942377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ce6d3d88-e39c-4b8c-a48c-383cf89c7925 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22c02150477f4e98a9aa7fd0247def02'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:15.943063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ce6d3d88-e39c-4b8c-a48c-383cf89c7925 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22c02150477f4e98a9aa7fd0247def02'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:15.943669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ce6d3d88-e39c-4b8c-a48c-383cf89c7925 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22c02150477f4e98a9aa7fd0247def02'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:15.951127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ce6d3d88-e39c-4b8c-a48c-383cf89c7925 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22c02150477f4e98a9aa7fd0247def02'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:15.964383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ce6d3d88-e39c-4b8c-a48c-383cf89c7925 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22c02150477f4e98a9aa7fd0247def02'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:15.978312 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4021/8105] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:22:15 2022] DELETE /identity/v3/users/22c02150477f4e98a9aa7fd0247def02 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:22:15.982975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be7fd9-1610-430f-a2a2-b652cbdea52b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:15.983447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be7fd9-1610-430f-a2a2-b652cbdea52b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:15.983574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be7fd9-1610-430f-a2a2-b652cbdea52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:16.003903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-59be7fd9-1610-430f-a2a2-b652cbdea52b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:22:16.005343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-59be7fd9-1610-430f-a2a2-b652cbdea52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:16.073605 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4085/8106] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:22:16.085129 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d3d88-e39c-4b8c-a48c-383cf89c7925 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:16.086392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ce6d3d88-e39c-4b8c-a48c-383cf89c7925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:16.159320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d3d88-e39c-4b8c-a48c-383cf89c7925 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:16.163684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af34fc-d9c3-4c5a-936e-634e25063c3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:16.164385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af34fc-d9c3-4c5a-936e-634e25063c3a service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:16.164739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af34fc-d9c3-4c5a-936e-634e25063c3a service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:16.165084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af34fc-d9c3-4c5a-936e-634e25063c3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:16.175940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af34fc-d9c3-4c5a-936e-634e25063c3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:16.177594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af34fc-d9c3-4c5a-936e-634e25063c3a service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:16.194700 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4022/8107] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:22:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:22:16.480632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be7fd9-1610-430f-a2a2-b652cbdea52b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:16.482081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-59be7fd9-1610-430f-a2a2-b652cbdea52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:16.495024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be7fd9-1610-430f-a2a2-b652cbdea52b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:16.499922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e78bd-03cf-4fa5-a0d8-e86f7bb81d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:16.500427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e78bd-03cf-4fa5-a0d8-e86f7bb81d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:16.500709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e78bd-03cf-4fa5-a0d8-e86f7bb81d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:16.500983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e78bd-03cf-4fa5-a0d8-e86f7bb81d9e admin admin] PATH_INFO: `/v3/projects/410af66d56c34146994cac962b73fbfd` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:16.502136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9e78bd-03cf-4fa5-a0d8-e86f7bb81d9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=410af66d56c34146994cac962b73fbfd)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:16.504111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9e78bd-03cf-4fa5-a0d8-e86f7bb81d9e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:16.554181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:16.558340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7a9e78bd-03cf-4fa5-a0d8-e86f7bb81d9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '410af66d56c34146994cac962b73fbfd', 'request_id': 'req-7a9e78bd-03cf-4fa5-a0d8-e86f7bb81d9e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:16.566795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7a9e78bd-03cf-4fa5-a0d8-e86f7bb81d9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '410af66d56c34146994cac962b73fbfd', 'request_id': 'req-7a9e78bd-03cf-4fa5-a0d8-e86f7bb81d9e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:16.572489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7a9e78bd-03cf-4fa5-a0d8-e86f7bb81d9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '410af66d56c34146994cac962b73fbfd', 'request_id': 'req-7a9e78bd-03cf-4fa5-a0d8-e86f7bb81d9e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:16.589825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7a9e78bd-03cf-4fa5-a0d8-e86f7bb81d9e admin admin] The token cache is being invalidate because project 410af66d56c34146994cac962b73fbfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:22:16.590124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7a9e78bd-03cf-4fa5-a0d8-e86f7bb81d9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:16.590792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7a9e78bd-03cf-4fa5-a0d8-e86f7bb81d9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:16.592298 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4086/8108] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:22:16 2022] DELETE /identity/v3/projects/410af66d56c34146994cac962b73fbfd => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:22:16.596356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af34fc-d9c3-4c5a-936e-634e25063c3a service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:16.606808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af34fc-d9c3-4c5a-936e-634e25063c3a service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:16.667595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c186cf3-94df-4049-8b01-abe67c7a5d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:16.668290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c186cf3-94df-4049-8b01-abe67c7a5d57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:16.668511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c186cf3-94df-4049-8b01-abe67c7a5d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:16.669045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c186cf3-94df-4049-8b01-abe67c7a5d57 admin admin] PATH_INFO: `/v3/users/51642c736773414886ad03bd940c8778` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:16.670166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c186cf3-94df-4049-8b01-abe67c7a5d57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51642c736773414886ad03bd940c8778)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:16.672178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c186cf3-94df-4049-8b01-abe67c7a5d57 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:16.682310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:16.720186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4c186cf3-94df-4049-8b01-abe67c7a5d57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51642c736773414886ad03bd940c8778'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:16.720847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4c186cf3-94df-4049-8b01-abe67c7a5d57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51642c736773414886ad03bd940c8778'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:16.721466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4c186cf3-94df-4049-8b01-abe67c7a5d57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51642c736773414886ad03bd940c8778'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:16.728804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4c186cf3-94df-4049-8b01-abe67c7a5d57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51642c736773414886ad03bd940c8778'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:16.741075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4c186cf3-94df-4049-8b01-abe67c7a5d57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51642c736773414886ad03bd940c8778'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:16.753313 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4023/8109] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:22:16 2022] DELETE /identity/v3/users/51642c736773414886ad03bd940c8778 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:22:17.022857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e78bd-03cf-4fa5-a0d8-e86f7bb81d9e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:17.024322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7a9e78bd-03cf-4fa5-a0d8-e86f7bb81d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:17.037762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e78bd-03cf-4fa5-a0d8-e86f7bb81d9e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:17.102064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f84c6-0dd0-4b92-b9c5-2dd9d9339bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:17.102723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f84c6-0dd0-4b92-b9c5-2dd9d9339bc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:17.103004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f84c6-0dd0-4b92-b9c5-2dd9d9339bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:17.103276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f84c6-0dd0-4b92-b9c5-2dd9d9339bc9 admin admin] PATH_INFO: `/v3/projects/12e7b51181e04f3eb5dcc48e45dabde5` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:17.104386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f84c6-0dd0-4b92-b9c5-2dd9d9339bc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12e7b51181e04f3eb5dcc48e45dabde5)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:17.106265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f84c6-0dd0-4b92-b9c5-2dd9d9339bc9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:17.153771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a8f84c6-0dd0-4b92-b9c5-2dd9d9339bc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12e7b51181e04f3eb5dcc48e45dabde5', 'request_id': 'req-6a8f84c6-0dd0-4b92-b9c5-2dd9d9339bc9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:17.160231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a8f84c6-0dd0-4b92-b9c5-2dd9d9339bc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12e7b51181e04f3eb5dcc48e45dabde5', 'request_id': 'req-6a8f84c6-0dd0-4b92-b9c5-2dd9d9339bc9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:17.165763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a8f84c6-0dd0-4b92-b9c5-2dd9d9339bc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12e7b51181e04f3eb5dcc48e45dabde5', 'request_id': 'req-6a8f84c6-0dd0-4b92-b9c5-2dd9d9339bc9'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:17.179358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a8f84c6-0dd0-4b92-b9c5-2dd9d9339bc9 admin admin] The token cache is being invalidate because project 12e7b51181e04f3eb5dcc48e45dabde5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:22:17.179690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a8f84c6-0dd0-4b92-b9c5-2dd9d9339bc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:17.180862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6a8f84c6-0dd0-4b92-b9c5-2dd9d9339bc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:22:17.182362 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4087/8110] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:22:17 2022] DELETE /identity/v3/projects/12e7b51181e04f3eb5dcc48e45dabde5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:22:17.188691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b5102-18a3-40e4-a98d-15c31a9718ad None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:17.189215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b5102-18a3-40e4-a98d-15c31a9718ad None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:17.189215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b5102-18a3-40e4-a98d-15c31a9718ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:17.208862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-261b5102-18a3-40e4-a98d-15c31a9718ad None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:22:17.210055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-261b5102-18a3-40e4-a98d-15c31a9718ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:17.273185 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4024/8111] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:22:17.281574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844d1b0-b6cb-4e9b-a819-e40d9cf8f0fa None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:17.281861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844d1b0-b6cb-4e9b-a819-e40d9cf8f0fa None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:17.282183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844d1b0-b6cb-4e9b-a819-e40d9cf8f0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:17.303896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d844d1b0-b6cb-4e9b-a819-e40d9cf8f0fa None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:22:17.305088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d844d1b0-b6cb-4e9b-a819-e40d9cf8f0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:17.313332 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4088/8112] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:22:17.317783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b5102-18a3-40e4-a98d-15c31a9718ad None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:17.327738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b5102-18a3-40e4-a98d-15c31a9718ad None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:17.332448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b396b-4b80-4785-a06e-a99ba2b876ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:17.333050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b396b-4b80-4785-a06e-a99ba2b876ee admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:17.333371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b396b-4b80-4785-a06e-a99ba2b876ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:17.333647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b396b-4b80-4785-a06e-a99ba2b876ee admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:17.334300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b396b-4b80-4785-a06e-a99ba2b876ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:22:17.334644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b396b-4b80-4785-a06e-a99ba2b876ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:17.336307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b396b-4b80-4785-a06e-a99ba2b876ee admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:17.348932 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4025/8113] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:22:17.353510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844d1b0-b6cb-4e9b-a819-e40d9cf8f0fa None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:17.363446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844d1b0-b6cb-4e9b-a819-e40d9cf8f0fa None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:17.368149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df8260-1243-4c2b-a8f6-8131ed5b515a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:17.368749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df8260-1243-4c2b-a8f6-8131ed5b515a admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:17.369031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df8260-1243-4c2b-a8f6-8131ed5b515a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:17.369304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df8260-1243-4c2b-a8f6-8131ed5b515a admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:17.369944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df8260-1243-4c2b-a8f6-8131ed5b515a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:17.371856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df8260-1243-4c2b-a8f6-8131ed5b515a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:17.400745 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4089/8114] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:22:17 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:22:17.404973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b396b-4b80-4785-a06e-a99ba2b876ee admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:17.415773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b396b-4b80-4785-a06e-a99ba2b876ee admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:17.473871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a486e61-dd24-4cbf-8356-b2dda23e53b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:17.474435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a486e61-dd24-4cbf-8356-b2dda23e53b7 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:17.474748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a486e61-dd24-4cbf-8356-b2dda23e53b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:17.475041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a486e61-dd24-4cbf-8356-b2dda23e53b7 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:17.475650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a486e61-dd24-4cbf-8356-b2dda23e53b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:17.477553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a486e61-dd24-4cbf-8356-b2dda23e53b7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:17.498456 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4026/8115] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:22:17 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:22:17.502723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df8260-1243-4c2b-a8f6-8131ed5b515a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:17.513001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df8260-1243-4c2b-a8f6-8131ed5b515a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:17.517647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c6e3d-799e-4b45-9e96-73311a4229f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:17.518241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c6e3d-799e-4b45-9e96-73311a4229f5 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:17.518551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c6e3d-799e-4b45-9e96-73311a4229f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:17.518829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c6e3d-799e-4b45-9e96-73311a4229f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:17.519993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c6e3d-799e-4b45-9e96-73311a4229f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:17.521570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c6e3d-799e-4b45-9e96-73311a4229f5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:17.531641 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4090/8116] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:22:17.535983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a486e61-dd24-4cbf-8356-b2dda23e53b7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:17.542952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:17.546999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a486e61-dd24-4cbf-8356-b2dda23e53b7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:17.552707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86937e-bb91-4e9f-bdb1-65d9cc917016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:17.553633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86937e-bb91-4e9f-bdb1-65d9cc917016 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:17.554095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86937e-bb91-4e9f-bdb1-65d9cc917016 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:17.554417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86937e-bb91-4e9f-bdb1-65d9cc917016 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:17.555242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86937e-bb91-4e9f-bdb1-65d9cc917016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:17.556847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86937e-bb91-4e9f-bdb1-65d9cc917016 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:17.567596 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4027/8117] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:22:17.571526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c6e3d-799e-4b45-9e96-73311a4229f5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:17.582075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c6e3d-799e-4b45-9e96-73311a4229f5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:17.586902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661c774-b993-4287-aa18-55cd6bfefba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:17.587557 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661c774-b993-4287-aa18-55cd6bfefba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:17.587846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661c774-b993-4287-aa18-55cd6bfefba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:17.588125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661c774-b993-4287-aa18-55cd6bfefba4 admin admin] PATH_INFO: `/v3/projects/0bb47fed88494698a3be2a0b34761464/users/2f6a9ae0c03d4328be8a4665cc503005/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:17.604068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661c774-b993-4287-aa18-55cd6bfefba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bb47fed88494698a3be2a0b34761464, user_id=2f6a9ae0c03d4328be8a4665cc503005, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:17.606843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661c774-b993-4287-aa18-55cd6bfefba4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:17.614753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:17.623843 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4091/8118] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:22:17 2022] PUT /identity/v3/projects/0bb47fed88494698a3be2a0b34761464/users/2f6a9ae0c03d4328be8a4665cc503005/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:22:17.633721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36821fb2-d626-4975-8692-4474ade1edec None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:17.634067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36821fb2-d626-4975-8692-4474ade1edec None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:17.634339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36821fb2-d626-4975-8692-4474ade1edec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:17.663012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-36821fb2-d626-4975-8692-4474ade1edec None None] MFA Rules not processed for user `2f6a9ae0c03d4328be8a4665cc503005`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:22:17.664237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-36821fb2-d626-4975-8692-4474ade1edec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:17.733478 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4028/8119] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:22:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:22:17.746137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661c774-b993-4287-aa18-55cd6bfefba4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:22:17.748385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8661c774-b993-4287-aa18-55cd6bfefba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:17.822589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661c774-b993-4287-aa18-55cd6bfefba4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:22:17.826341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118d5b5-9a63-4a4d-ba3a-be4c12c772e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:22:17.827015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118d5b5-9a63-4a4d-ba3a-be4c12c772e5 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:17.827299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118d5b5-9a63-4a4d-ba3a-be4c12c772e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:17.827574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118d5b5-9a63-4a4d-ba3a-be4c12c772e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:17.838326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118d5b5-9a63-4a4d-ba3a-be4c12c772e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:22:17.839801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118d5b5-9a63-4a4d-ba3a-be4c12c772e5 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:22:17.851107 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4092/8120] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:22:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:22:19.243323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adaae7-0eeb-4129-887f-cc50ff9cad9d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:22:19.243877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adaae7-0eeb-4129-887f-cc50ff9cad9d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:22:19.243877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adaae7-0eeb-4129-887f-cc50ff9cad9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:22:19.263540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b8adaae7-0eeb-4129-887f-cc50ff9cad9d None None] MFA Rules not processed for user `2f6a9ae0c03d4328be8a4665cc503005`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:22:19.264869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b8adaae7-0eeb-4129-887f-cc50ff9cad9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:22:19.273398 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4029/8121] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:22:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:22:27.845331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=27 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:29.257732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=38 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:07.068819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118d5b5-9a63-4a4d-ba3a-be4c12c772e5 service cinder] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:07.070238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1118d5b5-9a63-4a4d-ba3a-be4c12c772e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:07.244067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118d5b5-9a63-4a4d-ba3a-be4c12c772e5 service cinder] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:07.249929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad321a72-1aa2-4deb-bfaa-24819101a5f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:07.250736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad321a72-1aa2-4deb-bfaa-24819101a5f2 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:07.251019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad321a72-1aa2-4deb-bfaa-24819101a5f2 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:07.251294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad321a72-1aa2-4deb-bfaa-24819101a5f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:07.253250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ad321a72-1aa2-4deb-bfaa-24819101a5f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:07.279498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830b788-3585-4c50-bf92-c76a06e04984 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:07.280086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830b788-3585-4c50-bf92-c76a06e04984 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:07.280252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830b788-3585-4c50-bf92-c76a06e04984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:07.304368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5830b788-3585-4c50-bf92-c76a06e04984 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:23:07.306396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5830b788-3585-4c50-bf92-c76a06e04984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:07.352135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad321a72-1aa2-4deb-bfaa-24819101a5f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:07.355609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad321a72-1aa2-4deb-bfaa-24819101a5f2 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:07.369370 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4093/8122] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:23:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:23:07.454548 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4030/8123] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:23:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:23:07.522142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad321a72-1aa2-4deb-bfaa-24819101a5f2 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:07.528295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ad321a72-1aa2-4deb-bfaa-24819101a5f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:07.703348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad321a72-1aa2-4deb-bfaa-24819101a5f2 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:07.709839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb6350-b3c6-4a39-a0ac-4fee667548ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:07.710648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb6350-b3c6-4a39-a0ac-4fee667548ba service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:07.711030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb6350-b3c6-4a39-a0ac-4fee667548ba service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:07.711514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb6350-b3c6-4a39-a0ac-4fee667548ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:07.721332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb6350-b3c6-4a39-a0ac-4fee667548ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:07.723243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb6350-b3c6-4a39-a0ac-4fee667548ba service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:07.732013 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4094/8124] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:23:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:23:08.412598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c8220-027c-4b2e-bf0f-b4b646841a03 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:08.413078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c8220-027c-4b2e-bf0f-b4b646841a03 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:08.413601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c8220-027c-4b2e-bf0f-b4b646841a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:08.439440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c98c8220-027c-4b2e-bf0f-b4b646841a03 None None] MFA Rules not processed for user `6ac020dc323042c48ccce2a8826bc6be`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:23:08.440935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c98c8220-027c-4b2e-bf0f-b4b646841a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:08.450396 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4031/8125] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:23:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:23:14.299446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dfd26-3a78-4fa2-8ddf-a610556c984b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:14.300000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dfd26-3a78-4fa2-8ddf-a610556c984b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:14.300000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dfd26-3a78-4fa2-8ddf-a610556c984b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:14.324290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-367dfd26-3a78-4fa2-8ddf-a610556c984b None None] MFA Rules not processed for user `eb0bf5fce885439eaa16a0871c2992ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:23:14.325710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-367dfd26-3a78-4fa2-8ddf-a610556c984b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:14.332043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:14.407676 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4095/8126] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:23:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:23:15.061831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c8220-027c-4b2e-bf0f-b4b646841a03 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:15.159366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c8220-027c-4b2e-bf0f-b4b646841a03 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:15.164202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f0fe3-3eb0-48e1-92ab-a777c9790482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:15.164913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f0fe3-3eb0-48e1-92ab-a777c9790482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:15.165775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f0fe3-3eb0-48e1-92ab-a777c9790482 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:15.166042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f0fe3-3eb0-48e1-92ab-a777c9790482 admin admin] PATH_INFO: `/v3/users/2f6a9ae0c03d4328be8a4665cc503005` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:15.167659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f0fe3-3eb0-48e1-92ab-a777c9790482 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f6a9ae0c03d4328be8a4665cc503005)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:15.170951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f0fe3-3eb0-48e1-92ab-a777c9790482 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:15.249083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-166f0fe3-3eb0-48e1-92ab-a777c9790482 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f6a9ae0c03d4328be8a4665cc503005'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:15.249953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-166f0fe3-3eb0-48e1-92ab-a777c9790482 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f6a9ae0c03d4328be8a4665cc503005'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:15.250720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-166f0fe3-3eb0-48e1-92ab-a777c9790482 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f6a9ae0c03d4328be8a4665cc503005'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:15.261137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-166f0fe3-3eb0-48e1-92ab-a777c9790482 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f6a9ae0c03d4328be8a4665cc503005'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:15.279303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-166f0fe3-3eb0-48e1-92ab-a777c9790482 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f6a9ae0c03d4328be8a4665cc503005'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:15.324742 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4032/8127] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:23:15 2022] DELETE /identity/v3/users/2f6a9ae0c03d4328be8a4665cc503005 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:23:15.329427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b72356-e4fe-4cbf-9bf2-3a9ea6fb4b91 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:15.329967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b72356-e4fe-4cbf-9bf2-3a9ea6fb4b91 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:15.330251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b72356-e4fe-4cbf-9bf2-3a9ea6fb4b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:15.370637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d3b72356-e4fe-4cbf-9bf2-3a9ea6fb4b91 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:23:15.374727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d3b72356-e4fe-4cbf-9bf2-3a9ea6fb4b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:15.494099 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4096/8128] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:23:15.501303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f0fe3-3eb0-48e1-92ab-a777c9790482 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:15.504583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-166f0fe3-3eb0-48e1-92ab-a777c9790482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:15.524361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:15.641474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f0fe3-3eb0-48e1-92ab-a777c9790482 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:15.646206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245abb6-34c0-49e6-b36b-0909089ed129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:15.647320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245abb6-34c0-49e6-b36b-0909089ed129 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:15.647774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245abb6-34c0-49e6-b36b-0909089ed129 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:15.648367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245abb6-34c0-49e6-b36b-0909089ed129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:15.664117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245abb6-34c0-49e6-b36b-0909089ed129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:15.671830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245abb6-34c0-49e6-b36b-0909089ed129 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:15.694309 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4033/8129] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:23:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:23:16.302702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b72356-e4fe-4cbf-9bf2-3a9ea6fb4b91 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:16.307163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d3b72356-e4fe-4cbf-9bf2-3a9ea6fb4b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:16.359722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b72356-e4fe-4cbf-9bf2-3a9ea6fb4b91 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:16.370257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e034d-7578-4606-8c48-5199c0c8ffd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:16.372566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e034d-7578-4606-8c48-5199c0c8ffd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:16.374484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e034d-7578-4606-8c48-5199c0c8ffd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:16.374981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e034d-7578-4606-8c48-5199c0c8ffd2 admin admin] PATH_INFO: `/v3/projects/0bb47fed88494698a3be2a0b34761464` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:16.386801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e034d-7578-4606-8c48-5199c0c8ffd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bb47fed88494698a3be2a0b34761464)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:16.393477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e034d-7578-4606-8c48-5199c0c8ffd2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:16.450075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245abb6-34c0-49e6-b36b-0909089ed129 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:16.453598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e245abb6-34c0-49e6-b36b-0909089ed129 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:16.483064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245abb6-34c0-49e6-b36b-0909089ed129 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:16.489553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9b1e034d-7578-4606-8c48-5199c0c8ffd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bb47fed88494698a3be2a0b34761464', 'request_id': 'req-9b1e034d-7578-4606-8c48-5199c0c8ffd2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:16.497469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9b1e034d-7578-4606-8c48-5199c0c8ffd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bb47fed88494698a3be2a0b34761464', 'request_id': 'req-9b1e034d-7578-4606-8c48-5199c0c8ffd2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:16.510986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9b1e034d-7578-4606-8c48-5199c0c8ffd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bb47fed88494698a3be2a0b34761464', 'request_id': 'req-9b1e034d-7578-4606-8c48-5199c0c8ffd2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:16.549732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9b1e034d-7578-4606-8c48-5199c0c8ffd2 admin admin] The token cache is being invalidate because project 0bb47fed88494698a3be2a0b34761464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:23:16.550942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9b1e034d-7578-4606-8c48-5199c0c8ffd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:16.552366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9b1e034d-7578-4606-8c48-5199c0c8ffd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:16.554776 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4097/8130] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:23:16 2022] DELETE /identity/v3/projects/0bb47fed88494698a3be2a0b34761464 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:23:16.560837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7494fe9-df0e-4f97-8d4d-930ba8e6aec7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:16.561603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7494fe9-df0e-4f97-8d4d-930ba8e6aec7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:16.562092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7494fe9-df0e-4f97-8d4d-930ba8e6aec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:16.594021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f7494fe9-df0e-4f97-8d4d-930ba8e6aec7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:23:16.596329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f7494fe9-df0e-4f97-8d4d-930ba8e6aec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:16.608171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a91ba5-092b-44bc-bdb3-ce2082a1adb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:16.608838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a91ba5-092b-44bc-bdb3-ce2082a1adb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:16.609122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a91ba5-092b-44bc-bdb3-ce2082a1adb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:16.609400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a91ba5-092b-44bc-bdb3-ce2082a1adb5 admin admin] PATH_INFO: `/v3/users/f77486ef55a0469a99c3795b626bee1a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:16.612004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a91ba5-092b-44bc-bdb3-ce2082a1adb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f77486ef55a0469a99c3795b626bee1a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:16.617488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a91ba5-092b-44bc-bdb3-ce2082a1adb5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:16.695880 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:16.726453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e7a91ba5-092b-44bc-bdb3-ce2082a1adb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f77486ef55a0469a99c3795b626bee1a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:16.727335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e7a91ba5-092b-44bc-bdb3-ce2082a1adb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f77486ef55a0469a99c3795b626bee1a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:16.728031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e7a91ba5-092b-44bc-bdb3-ce2082a1adb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f77486ef55a0469a99c3795b626bee1a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:16.756059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e7a91ba5-092b-44bc-bdb3-ce2082a1adb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f77486ef55a0469a99c3795b626bee1a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:16.780992 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4098/8131] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:23:16.782773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e7a91ba5-092b-44bc-bdb3-ce2082a1adb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f77486ef55a0469a99c3795b626bee1a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:16.789028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf878ae2-bf4f-4346-971e-b2cd2b810e57 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:16.789819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf878ae2-bf4f-4346-971e-b2cd2b810e57 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:16.790110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf878ae2-bf4f-4346-971e-b2cd2b810e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:16.810937 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4034/8132] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:23:16 2022] DELETE /identity/v3/users/f77486ef55a0469a99c3795b626bee1a => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:23:16.815756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f1cda-06a0-46bd-b9ef-25378c9d906c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:16.816397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f1cda-06a0-46bd-b9ef-25378c9d906c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:16.816836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f1cda-06a0-46bd-b9ef-25378c9d906c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:16.839251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-187f1cda-06a0-46bd-b9ef-25378c9d906c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:23:16.840796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-187f1cda-06a0-46bd-b9ef-25378c9d906c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:16.845116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-bf878ae2-bf4f-4346-971e-b2cd2b810e57 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:23:16.849857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bf878ae2-bf4f-4346-971e-b2cd2b810e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:16.924422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:16.983568 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4099/8133] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:23:16.987846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf878ae2-bf4f-4346-971e-b2cd2b810e57 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:17.001137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf878ae2-bf4f-4346-971e-b2cd2b810e57 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:17.009781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b20201-69e1-4b07-90c2-6ca11f66b64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:17.010852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b20201-69e1-4b07-90c2-6ca11f66b64c admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:17.011328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b20201-69e1-4b07-90c2-6ca11f66b64c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:17.011768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b20201-69e1-4b07-90c2-6ca11f66b64c admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:17.012743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b20201-69e1-4b07-90c2-6ca11f66b64c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:23:17.013215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b20201-69e1-4b07-90c2-6ca11f66b64c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:17.015020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b20201-69e1-4b07-90c2-6ca11f66b64c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:17.030082 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4100/8134] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:23:17.035164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b20201-69e1-4b07-90c2-6ca11f66b64c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:17.048318 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4035/8135] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:23:17.049655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b20201-69e1-4b07-90c2-6ca11f66b64c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:17.055261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62f16c-875c-4884-a8a2-8a4c21388102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:17.056656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62f16c-875c-4884-a8a2-8a4c21388102 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:17.057148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62f16c-875c-4884-a8a2-8a4c21388102 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:17.057590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62f16c-875c-4884-a8a2-8a4c21388102 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:17.058386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62f16c-875c-4884-a8a2-8a4c21388102 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:17.059709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f1cda-06a0-46bd-b9ef-25378c9d906c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:17.060755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62f16c-875c-4884-a8a2-8a4c21388102 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:17.061163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-187f1cda-06a0-46bd-b9ef-25378c9d906c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:17.099956 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4101/8136] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:23:17 2022] POST /identity/v3/projects => generated 365 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:23:17.104173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62f16c-875c-4884-a8a2-8a4c21388102 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:17.187442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f1cda-06a0-46bd-b9ef-25378c9d906c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:17.228733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62f16c-875c-4884-a8a2-8a4c21388102 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:17.241800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c1ca4-0605-4ef6-962a-2c6b1067750a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:17.242466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c1ca4-0605-4ef6-962a-2c6b1067750a admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:17.243183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c1ca4-0605-4ef6-962a-2c6b1067750a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:17.243463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c1ca4-0605-4ef6-962a-2c6b1067750a admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:17.248853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c1ca4-0605-4ef6-962a-2c6b1067750a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:17.252510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c1ca4-0605-4ef6-962a-2c6b1067750a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:17.299804 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4102/8137] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:23:17 2022] POST /identity/v3/users => generated 283 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:23:17.303631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c1ca4-0605-4ef6-962a-2c6b1067750a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:17.304480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a0de8-d408-4add-a1f1-24d92ba56268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:17.306866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a0de8-d408-4add-a1f1-24d92ba56268 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:17.307314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a0de8-d408-4add-a1f1-24d92ba56268 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:17.307749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a0de8-d408-4add-a1f1-24d92ba56268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:17.316638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c1ca4-0605-4ef6-962a-2c6b1067750a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:17.323940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46018676-1742-451c-b90c-37b8ea3a43f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:17.324735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46018676-1742-451c-b90c-37b8ea3a43f3 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:17.325123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46018676-1742-451c-b90c-37b8ea3a43f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:17.325423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a0de8-d408-4add-a1f1-24d92ba56268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:17.325639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46018676-1742-451c-b90c-37b8ea3a43f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:17.326542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46018676-1742-451c-b90c-37b8ea3a43f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:17.327446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a0de8-d408-4add-a1f1-24d92ba56268 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:17.329109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46018676-1742-451c-b90c-37b8ea3a43f3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:17.340893 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4036/8138] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:23:17.343389 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4103/8139] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:23:17.347306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a0de8-d408-4add-a1f1-24d92ba56268 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:17.360079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a0de8-d408-4add-a1f1-24d92ba56268 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:17.365946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5bd24-c439-42b7-a339-bc2416975f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:17.366845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5bd24-c439-42b7-a339-bc2416975f6f admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:17.367288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5bd24-c439-42b7-a339-bc2416975f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:17.367756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5bd24-c439-42b7-a339-bc2416975f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:17.370323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5bd24-c439-42b7-a339-bc2416975f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:17.372268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5bd24-c439-42b7-a339-bc2416975f6f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:17.388314 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4037/8140] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:23:17.393129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46018676-1742-451c-b90c-37b8ea3a43f3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:17.408059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46018676-1742-451c-b90c-37b8ea3a43f3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:17.413208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2eb26-ac5a-4422-9bfc-fb275872b988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:17.414015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2eb26-ac5a-4422-9bfc-fb275872b988 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:17.414432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2eb26-ac5a-4422-9bfc-fb275872b988 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:17.414877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2eb26-ac5a-4422-9bfc-fb275872b988 admin admin] PATH_INFO: `/v3/projects/243e5655a8a04805942d9fe4603bf8e5/users/c8d29756718f45c0a8a715a00daa1505/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:17.435777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2eb26-ac5a-4422-9bfc-fb275872b988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=243e5655a8a04805942d9fe4603bf8e5, user_id=c8d29756718f45c0a8a715a00daa1505, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:17.439034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2eb26-ac5a-4422-9bfc-fb275872b988 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:17.454055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:17.463115 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4104/8141] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:23:17 2022] PUT /identity/v3/projects/243e5655a8a04805942d9fe4603bf8e5/users/c8d29756718f45c0a8a715a00daa1505/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:23:17.474534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77353d73-ffa6-40ca-a266-f28ed2dfaf35 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:17.475094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77353d73-ffa6-40ca-a266-f28ed2dfaf35 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:17.475094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77353d73-ffa6-40ca-a266-f28ed2dfaf35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:17.546858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-77353d73-ffa6-40ca-a266-f28ed2dfaf35 None None] MFA Rules not processed for user `c8d29756718f45c0a8a715a00daa1505`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:23:17.548233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-77353d73-ffa6-40ca-a266-f28ed2dfaf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:17.641653 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4038/8142] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:23:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:23:17.656030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2eb26-ac5a-4422-9bfc-fb275872b988 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:17.657840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-68a2eb26-ac5a-4422-9bfc-fb275872b988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:17.756144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2eb26-ac5a-4422-9bfc-fb275872b988 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:17.763208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14fabb-20c5-4731-a1f3-12ac79dc9cb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:17.763920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14fabb-20c5-4731-a1f3-12ac79dc9cb4 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:17.764276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14fabb-20c5-4731-a1f3-12ac79dc9cb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:17.764617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14fabb-20c5-4731-a1f3-12ac79dc9cb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:17.776824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14fabb-20c5-4731-a1f3-12ac79dc9cb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:17.779154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14fabb-20c5-4731-a1f3-12ac79dc9cb4 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:17.793308 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4105/8143] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:23:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:23:18.131175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77353d73-ffa6-40ca-a266-f28ed2dfaf35 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:18.132712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-77353d73-ffa6-40ca-a266-f28ed2dfaf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:18.152127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:18.314099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77353d73-ffa6-40ca-a266-f28ed2dfaf35 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:18.333594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5c9d5-a2fc-4bed-b26e-03b6b22f9cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:18.335720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5c9d5-a2fc-4bed-b26e-03b6b22f9cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:18.336007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5c9d5-a2fc-4bed-b26e-03b6b22f9cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:18.336286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5c9d5-a2fc-4bed-b26e-03b6b22f9cc9 admin admin] PATH_INFO: `/v3/projects/71511e85c1b947e3b03501b27d70d560` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:18.344323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5c9d5-a2fc-4bed-b26e-03b6b22f9cc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71511e85c1b947e3b03501b27d70d560)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:18.355871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5c9d5-a2fc-4bed-b26e-03b6b22f9cc9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:18.462707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-81f5c9d5-a2fc-4bed-b26e-03b6b22f9cc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71511e85c1b947e3b03501b27d70d560', 'request_id': 'req-81f5c9d5-a2fc-4bed-b26e-03b6b22f9cc9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:18.478651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-81f5c9d5-a2fc-4bed-b26e-03b6b22f9cc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71511e85c1b947e3b03501b27d70d560', 'request_id': 'req-81f5c9d5-a2fc-4bed-b26e-03b6b22f9cc9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:18.487228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-81f5c9d5-a2fc-4bed-b26e-03b6b22f9cc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71511e85c1b947e3b03501b27d70d560', 'request_id': 'req-81f5c9d5-a2fc-4bed-b26e-03b6b22f9cc9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:18.516386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-81f5c9d5-a2fc-4bed-b26e-03b6b22f9cc9 admin admin] The token cache is being invalidate because project 71511e85c1b947e3b03501b27d70d560 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:23:18.519831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-81f5c9d5-a2fc-4bed-b26e-03b6b22f9cc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:18.521596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-81f5c9d5-a2fc-4bed-b26e-03b6b22f9cc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:18.526524 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4039/8144] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:23:18 2022] DELETE /identity/v3/projects/71511e85c1b947e3b03501b27d70d560 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:23:18.530921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14fabb-20c5-4731-a1f3-12ac79dc9cb4 service cinder] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:18.554938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14fabb-20c5-4731-a1f3-12ac79dc9cb4 service cinder] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:18.669943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae7657-0cf2-4d28-b74e-206bc49fcc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:18.670853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae7657-0cf2-4d28-b74e-206bc49fcc31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:18.671363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae7657-0cf2-4d28-b74e-206bc49fcc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:18.671796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae7657-0cf2-4d28-b74e-206bc49fcc31 admin admin] PATH_INFO: `/v3/users/39e3f059dc9c4fd78a6799b5b209d710` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:18.673197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae7657-0cf2-4d28-b74e-206bc49fcc31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39e3f059dc9c4fd78a6799b5b209d710)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:18.688602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae7657-0cf2-4d28-b74e-206bc49fcc31 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:18.773622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4eae7657-0cf2-4d28-b74e-206bc49fcc31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39e3f059dc9c4fd78a6799b5b209d710'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:18.783792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4eae7657-0cf2-4d28-b74e-206bc49fcc31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39e3f059dc9c4fd78a6799b5b209d710'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:18.799773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4eae7657-0cf2-4d28-b74e-206bc49fcc31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39e3f059dc9c4fd78a6799b5b209d710'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:18.801027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4eae7657-0cf2-4d28-b74e-206bc49fcc31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39e3f059dc9c4fd78a6799b5b209d710'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:18.814957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4eae7657-0cf2-4d28-b74e-206bc49fcc31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39e3f059dc9c4fd78a6799b5b209d710'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:18.834900 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4106/8145] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:23:18 2022] DELETE /identity/v3/users/39e3f059dc9c4fd78a6799b5b209d710 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:23:19.093054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5c9d5-a2fc-4bed-b26e-03b6b22f9cc9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:19.095164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-81f5c9d5-a2fc-4bed-b26e-03b6b22f9cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:19.116396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5c9d5-a2fc-4bed-b26e-03b6b22f9cc9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:19.203700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da0f2f-9362-4a06-82fe-fdd36fb8b8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:19.204959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da0f2f-9362-4a06-82fe-fdd36fb8b8d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:19.204959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da0f2f-9362-4a06-82fe-fdd36fb8b8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:19.204959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da0f2f-9362-4a06-82fe-fdd36fb8b8d1 admin admin] PATH_INFO: `/v3/projects/8de5bb9a57014d588d6a62e872f0ec3c` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:19.206192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da0f2f-9362-4a06-82fe-fdd36fb8b8d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8de5bb9a57014d588d6a62e872f0ec3c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:19.208543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da0f2f-9362-4a06-82fe-fdd36fb8b8d1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:19.262078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:19.272712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-69da0f2f-9362-4a06-82fe-fdd36fb8b8d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8de5bb9a57014d588d6a62e872f0ec3c', 'request_id': 'req-69da0f2f-9362-4a06-82fe-fdd36fb8b8d1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:19.281765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-69da0f2f-9362-4a06-82fe-fdd36fb8b8d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8de5bb9a57014d588d6a62e872f0ec3c', 'request_id': 'req-69da0f2f-9362-4a06-82fe-fdd36fb8b8d1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:19.288850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-69da0f2f-9362-4a06-82fe-fdd36fb8b8d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8de5bb9a57014d588d6a62e872f0ec3c', 'request_id': 'req-69da0f2f-9362-4a06-82fe-fdd36fb8b8d1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:19.302024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-69da0f2f-9362-4a06-82fe-fdd36fb8b8d1 admin admin] The token cache is being invalidate because project 8de5bb9a57014d588d6a62e872f0ec3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:23:19.302533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-69da0f2f-9362-4a06-82fe-fdd36fb8b8d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:19.303114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-69da0f2f-9362-4a06-82fe-fdd36fb8b8d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:19.304673 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4040/8146] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:23:19 2022] DELETE /identity/v3/projects/8de5bb9a57014d588d6a62e872f0ec3c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:23:20.709420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae7657-0cf2-4d28-b74e-206bc49fcc31 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:20.711580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4eae7657-0cf2-4d28-b74e-206bc49fcc31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:20.736742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:20.798561 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae7657-0cf2-4d28-b74e-206bc49fcc31 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:20.804433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da0f2f-9362-4a06-82fe-fdd36fb8b8d1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:20.807268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-69da0f2f-9362-4a06-82fe-fdd36fb8b8d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:20.874120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfcebe-8b03-4123-a844-b25c1e089aae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:20.874783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfcebe-8b03-4123-a844-b25c1e089aae service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:20.875314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfcebe-8b03-4123-a844-b25c1e089aae service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:20.875493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfcebe-8b03-4123-a844-b25c1e089aae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:20.877804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-46bfcebe-8b03-4123-a844-b25c1e089aae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:20.885157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da0f2f-9362-4a06-82fe-fdd36fb8b8d1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:20.890789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b79815-ab60-4a78-9cdd-730637e7969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:20.891505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b79815-ab60-4a78-9cdd-730637e7969c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:20.891863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b79815-ab60-4a78-9cdd-730637e7969c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:20.892210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b79815-ab60-4a78-9cdd-730637e7969c admin admin] PATH_INFO: `/v3/users/55dadd491ec1441aac4171eff46bd679` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:20.893428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b79815-ab60-4a78-9cdd-730637e7969c admin admin] RBAC: Authorizing `identity:delete_user(user_id=55dadd491ec1441aac4171eff46bd679)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:20.895560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b79815-ab60-4a78-9cdd-730637e7969c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:20.895801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfcebe-8b03-4123-a844-b25c1e089aae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:20.898169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfcebe-8b03-4123-a844-b25c1e089aae service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:20.913494 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4107/8147] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:23:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:23:20.958897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-22b79815-ab60-4a78-9cdd-730637e7969c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55dadd491ec1441aac4171eff46bd679'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:20.959657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-22b79815-ab60-4a78-9cdd-730637e7969c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55dadd491ec1441aac4171eff46bd679'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:20.960285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-22b79815-ab60-4a78-9cdd-730637e7969c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55dadd491ec1441aac4171eff46bd679'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:20.968773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-22b79815-ab60-4a78-9cdd-730637e7969c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55dadd491ec1441aac4171eff46bd679'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:20.982801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-22b79815-ab60-4a78-9cdd-730637e7969c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55dadd491ec1441aac4171eff46bd679'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:20.997254 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4041/8148] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:23:20 2022] DELETE /identity/v3/users/55dadd491ec1441aac4171eff46bd679 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:23:21.176200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfcebe-8b03-4123-a844-b25c1e089aae service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:21.199948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfcebe-8b03-4123-a844-b25c1e089aae service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:21.313781 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc4043-b39a-4328-b32f-c1d4b41804c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:21.314709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc4043-b39a-4328-b32f-c1d4b41804c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:21.315151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc4043-b39a-4328-b32f-c1d4b41804c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:21.315613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc4043-b39a-4328-b32f-c1d4b41804c8 admin admin] PATH_INFO: `/v3/projects/90cb48440cb446ea9abfc00ed0f53779` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:21.316950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc4043-b39a-4328-b32f-c1d4b41804c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90cb48440cb446ea9abfc00ed0f53779)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:21.337578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc4043-b39a-4328-b32f-c1d4b41804c8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:21.435911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ffc4043-b39a-4328-b32f-c1d4b41804c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90cb48440cb446ea9abfc00ed0f53779', 'request_id': 'req-2ffc4043-b39a-4328-b32f-c1d4b41804c8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:21.445883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ffc4043-b39a-4328-b32f-c1d4b41804c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90cb48440cb446ea9abfc00ed0f53779', 'request_id': 'req-2ffc4043-b39a-4328-b32f-c1d4b41804c8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:21.447318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:21.461253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ffc4043-b39a-4328-b32f-c1d4b41804c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90cb48440cb446ea9abfc00ed0f53779', 'request_id': 'req-2ffc4043-b39a-4328-b32f-c1d4b41804c8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:21.503231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ffc4043-b39a-4328-b32f-c1d4b41804c8 admin admin] The token cache is being invalidate because project 90cb48440cb446ea9abfc00ed0f53779 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:23:21.503531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ffc4043-b39a-4328-b32f-c1d4b41804c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:21.504381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2ffc4043-b39a-4328-b32f-c1d4b41804c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:21.507048 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4108/8149] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:23:21 2022] DELETE /identity/v3/projects/90cb48440cb446ea9abfc00ed0f53779 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:23:21.510254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b79815-ab60-4a78-9cdd-730637e7969c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:21.512169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-22b79815-ab60-4a78-9cdd-730637e7969c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:21.541618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b79815-ab60-4a78-9cdd-730637e7969c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:21.609695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:21.626765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057094c7-5941-4cb6-a1ed-1218bdc37028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:21.628093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057094c7-5941-4cb6-a1ed-1218bdc37028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:21.630250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057094c7-5941-4cb6-a1ed-1218bdc37028 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:21.630724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057094c7-5941-4cb6-a1ed-1218bdc37028 admin admin] PATH_INFO: `/v3/users/7ca7e6f15e574640b899f6f47ea0bfd4` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:21.632364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057094c7-5941-4cb6-a1ed-1218bdc37028 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ca7e6f15e574640b899f6f47ea0bfd4)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:21.635467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057094c7-5941-4cb6-a1ed-1218bdc37028 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:21.688835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-057094c7-5941-4cb6-a1ed-1218bdc37028 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ca7e6f15e574640b899f6f47ea0bfd4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:21.689486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-057094c7-5941-4cb6-a1ed-1218bdc37028 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ca7e6f15e574640b899f6f47ea0bfd4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:21.690761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-057094c7-5941-4cb6-a1ed-1218bdc37028 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ca7e6f15e574640b899f6f47ea0bfd4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:21.698301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-057094c7-5941-4cb6-a1ed-1218bdc37028 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ca7e6f15e574640b899f6f47ea0bfd4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:21.713865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-057094c7-5941-4cb6-a1ed-1218bdc37028 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ca7e6f15e574640b899f6f47ea0bfd4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:21.733005 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4042/8150] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:23:21 2022] DELETE /identity/v3/users/7ca7e6f15e574640b899f6f47ea0bfd4 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:23:21.956915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc4043-b39a-4328-b32f-c1d4b41804c8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:21.958382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2ffc4043-b39a-4328-b32f-c1d4b41804c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:21.971902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc4043-b39a-4328-b32f-c1d4b41804c8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:22.044408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628d269-746d-4188-9dcf-926b46cff04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:22.045135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628d269-746d-4188-9dcf-926b46cff04c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:22.045494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628d269-746d-4188-9dcf-926b46cff04c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:22.045831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628d269-746d-4188-9dcf-926b46cff04c admin admin] PATH_INFO: `/v3/projects/9dca212c4531456e96b84afb30119345` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:22.047074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4628d269-746d-4188-9dcf-926b46cff04c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dca212c4531456e96b84afb30119345)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:22.049031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4628d269-746d-4188-9dcf-926b46cff04c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:22.095023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4628d269-746d-4188-9dcf-926b46cff04c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dca212c4531456e96b84afb30119345', 'request_id': 'req-4628d269-746d-4188-9dcf-926b46cff04c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:22.101105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4628d269-746d-4188-9dcf-926b46cff04c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dca212c4531456e96b84afb30119345', 'request_id': 'req-4628d269-746d-4188-9dcf-926b46cff04c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:22.107382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4628d269-746d-4188-9dcf-926b46cff04c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dca212c4531456e96b84afb30119345', 'request_id': 'req-4628d269-746d-4188-9dcf-926b46cff04c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:22.119997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4628d269-746d-4188-9dcf-926b46cff04c admin admin] The token cache is being invalidate because project 9dca212c4531456e96b84afb30119345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:23:22.120316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4628d269-746d-4188-9dcf-926b46cff04c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:22.121030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4628d269-746d-4188-9dcf-926b46cff04c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:22.122741 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4109/8151] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:23:21 2022] DELETE /identity/v3/projects/9dca212c4531456e96b84afb30119345 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:23:24.353892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811cf4c-2122-490e-9b35-f215b6aa41a5 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:24.354556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811cf4c-2122-490e-9b35-f215b6aa41a5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:24.354556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811cf4c-2122-490e-9b35-f215b6aa41a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:24.375332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a811cf4c-2122-490e-9b35-f215b6aa41a5 None None] MFA Rules not processed for user `6ac020dc323042c48ccce2a8826bc6be`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:23:24.376664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a811cf4c-2122-490e-9b35-f215b6aa41a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:24.440037 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4043/8152] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:23:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:23:29.131512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628d269-746d-4188-9dcf-926b46cff04c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:29.132946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4628d269-746d-4188-9dcf-926b46cff04c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:29.193327 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628d269-746d-4188-9dcf-926b46cff04c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:29.197534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5495b1b-9d88-4d18-982d-4ac825d1b37b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:29.198202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5495b1b-9d88-4d18-982d-4ac825d1b37b service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:29.198499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5495b1b-9d88-4d18-982d-4ac825d1b37b service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:29.198801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5495b1b-9d88-4d18-982d-4ac825d1b37b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:29.200512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b5495b1b-9d88-4d18-982d-4ac825d1b37b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:29.230870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:29.269402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5495b1b-9d88-4d18-982d-4ac825d1b37b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:29.270922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5495b1b-9d88-4d18-982d-4ac825d1b37b service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:29.281343 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4110/8153] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:23:29 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:23:34.432616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=56,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:39.276771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=17 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:39.341418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811cf4c-2122-490e-9b35-f215b6aa41a5 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:39.342933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a811cf4c-2122-490e-9b35-f215b6aa41a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:39.434294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504fdb5-e5a7-4f9f-a3b2-ada4197a4613 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:39.434743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504fdb5-e5a7-4f9f-a3b2-ada4197a4613 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:39.435106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504fdb5-e5a7-4f9f-a3b2-ada4197a4613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:39.447769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811cf4c-2122-490e-9b35-f215b6aa41a5 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:39.452770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec5353-6128-45e7-b543-b9cca332328a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:39.453403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec5353-6128-45e7-b543-b9cca332328a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:39.453709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec5353-6128-45e7-b543-b9cca332328a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:39.453962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec5353-6128-45e7-b543-b9cca332328a admin admin] PATH_INFO: `/v3/users/c8d29756718f45c0a8a715a00daa1505` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:39.455133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec5353-6128-45e7-b543-b9cca332328a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8d29756718f45c0a8a715a00daa1505)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:39.456531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4504fdb5-e5a7-4f9f-a3b2-ada4197a4613 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:23:39.457097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec5353-6128-45e7-b543-b9cca332328a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:39.457788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4504fdb5-e5a7-4f9f-a3b2-ada4197a4613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:39.508721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-16ec5353-6128-45e7-b543-b9cca332328a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8d29756718f45c0a8a715a00daa1505'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:39.509715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-16ec5353-6128-45e7-b543-b9cca332328a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8d29756718f45c0a8a715a00daa1505'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:39.510380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-16ec5353-6128-45e7-b543-b9cca332328a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8d29756718f45c0a8a715a00daa1505'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:39.518237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-16ec5353-6128-45e7-b543-b9cca332328a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8d29756718f45c0a8a715a00daa1505'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:39.520372 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4111/8154] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:23:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:23:39.530687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-16ec5353-6128-45e7-b543-b9cca332328a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8d29756718f45c0a8a715a00daa1505'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:39.533173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504fdb5-e5a7-4f9f-a3b2-ada4197a4613 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:39.534841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4504fdb5-e5a7-4f9f-a3b2-ada4197a4613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:39.544300 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4044/8155] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:23:39 2022] DELETE /identity/v3/users/c8d29756718f45c0a8a715a00daa1505 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:23:39.548667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de7fd7-b0a2-436d-b55d-ba8b5e56afd7 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:39.548986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de7fd7-b0a2-436d-b55d-ba8b5e56afd7 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:39.549405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de7fd7-b0a2-436d-b55d-ba8b5e56afd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:39.570592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-14de7fd7-b0a2-436d-b55d-ba8b5e56afd7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:23:39.571860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-14de7fd7-b0a2-436d-b55d-ba8b5e56afd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:39.613754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504fdb5-e5a7-4f9f-a3b2-ada4197a4613 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:39.623388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:39.636599 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4045/8156] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:23:39.646603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de7fd7-b0a2-436d-b55d-ba8b5e56afd7 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:39.648202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-14de7fd7-b0a2-436d-b55d-ba8b5e56afd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:39.681218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec759b8-f708-42d6-88ab-6a35273b8fe1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:39.682009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec759b8-f708-42d6-88ab-6a35273b8fe1 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:39.682293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec759b8-f708-42d6-88ab-6a35273b8fe1 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:39.682612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec759b8-f708-42d6-88ab-6a35273b8fe1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:39.710656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de7fd7-b0a2-436d-b55d-ba8b5e56afd7 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:39.714342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffafd74-c1f2-491f-a22c-9954e3a99a47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:39.714988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffafd74-c1f2-491f-a22c-9954e3a99a47 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:39.715263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffafd74-c1f2-491f-a22c-9954e3a99a47 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:39.715535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffafd74-c1f2-491f-a22c-9954e3a99a47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:39.724978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffafd74-c1f2-491f-a22c-9954e3a99a47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:39.726416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffafd74-c1f2-491f-a22c-9954e3a99a47 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:39.733143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:39.737372 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4046/8157] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:23:39.744407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec759b8-f708-42d6-88ab-6a35273b8fe1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:39.746188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec759b8-f708-42d6-88ab-6a35273b8fe1 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:39.755260 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4112/8158] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:23:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 21:23:40.081809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffafd74-c1f2-491f-a22c-9954e3a99a47 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:40.084018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cffafd74-c1f2-491f-a22c-9954e3a99a47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:40.107019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffafd74-c1f2-491f-a22c-9954e3a99a47 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:40.115886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee3999-1aba-43a5-a850-cab081de6775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:40.117210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee3999-1aba-43a5-a850-cab081de6775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:40.117797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee3999-1aba-43a5-a850-cab081de6775 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:40.122514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee3999-1aba-43a5-a850-cab081de6775 admin admin] PATH_INFO: `/v3/projects/243e5655a8a04805942d9fe4603bf8e5` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:40.125006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee3999-1aba-43a5-a850-cab081de6775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=243e5655a8a04805942d9fe4603bf8e5)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:40.129228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee3999-1aba-43a5-a850-cab081de6775 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:40.218277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bcee3999-1aba-43a5-a850-cab081de6775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '243e5655a8a04805942d9fe4603bf8e5', 'request_id': 'req-bcee3999-1aba-43a5-a850-cab081de6775'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:40.232967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bcee3999-1aba-43a5-a850-cab081de6775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '243e5655a8a04805942d9fe4603bf8e5', 'request_id': 'req-bcee3999-1aba-43a5-a850-cab081de6775'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:40.250767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bcee3999-1aba-43a5-a850-cab081de6775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '243e5655a8a04805942d9fe4603bf8e5', 'request_id': 'req-bcee3999-1aba-43a5-a850-cab081de6775'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:40.275491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bcee3999-1aba-43a5-a850-cab081de6775 admin admin] The token cache is being invalidate because project 243e5655a8a04805942d9fe4603bf8e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:23:40.364816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bcee3999-1aba-43a5-a850-cab081de6775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:40.364816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bcee3999-1aba-43a5-a850-cab081de6775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:23:40.364816 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4047/8159] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:23:40 2022] DELETE /identity/v3/projects/243e5655a8a04805942d9fe4603bf8e5 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:23:40.365688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55728637-c040-43a1-a4f8-7ce9931be180 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:40.365688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55728637-c040-43a1-a4f8-7ce9931be180 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:40.365688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55728637-c040-43a1-a4f8-7ce9931be180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:40.365688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-55728637-c040-43a1-a4f8-7ce9931be180 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:23:40.365688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-55728637-c040-43a1-a4f8-7ce9931be180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:40.477185 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4113/8160] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:23:40.484775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d16534-8e58-47bb-b667-282b5107f82e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:40.485136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d16534-8e58-47bb-b667-282b5107f82e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:40.485972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d16534-8e58-47bb-b667-282b5107f82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:40.511241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-93d16534-8e58-47bb-b667-282b5107f82e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:23:40.512543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-93d16534-8e58-47bb-b667-282b5107f82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:40.529717 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4048/8161] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:23:40.533688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55728637-c040-43a1-a4f8-7ce9931be180 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:40.544936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55728637-c040-43a1-a4f8-7ce9931be180 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:40.550470 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1049335-aa62-4245-b7bf-080513d0c35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:40.551347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1049335-aa62-4245-b7bf-080513d0c35f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:40.551788 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1049335-aa62-4245-b7bf-080513d0c35f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:40.552299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1049335-aa62-4245-b7bf-080513d0c35f admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:40.553143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1049335-aa62-4245-b7bf-080513d0c35f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:23:40.553612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1049335-aa62-4245-b7bf-080513d0c35f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:40.556020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1049335-aa62-4245-b7bf-080513d0c35f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:40.570528 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4114/8162] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:23:40.574320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d16534-8e58-47bb-b667-282b5107f82e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:40.589895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d16534-8e58-47bb-b667-282b5107f82e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:40.594883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de91e6a-d21c-4763-adec-b796beb6e8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:40.595701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de91e6a-d21c-4763-adec-b796beb6e8d6 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:40.596023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de91e6a-d21c-4763-adec-b796beb6e8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:40.596354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de91e6a-d21c-4763-adec-b796beb6e8d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:40.597037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de91e6a-d21c-4763-adec-b796beb6e8d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:40.599199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de91e6a-d21c-4763-adec-b796beb6e8d6 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:40.646301 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4049/8163] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:23:40 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:23:40.650951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1049335-aa62-4245-b7bf-080513d0c35f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:40.666042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1049335-aa62-4245-b7bf-080513d0c35f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:40.734155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5252f8-92c1-4cdd-96e5-2420549767ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:40.734829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5252f8-92c1-4cdd-96e5-2420549767ab admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:40.735177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5252f8-92c1-4cdd-96e5-2420549767ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:40.735462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5252f8-92c1-4cdd-96e5-2420549767ab admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:40.736104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5252f8-92c1-4cdd-96e5-2420549767ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:40.738204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5252f8-92c1-4cdd-96e5-2420549767ab admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:40.763612 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4115/8164] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:23:40 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:23:40.767872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de91e6a-d21c-4763-adec-b796beb6e8d6 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:40.784677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de91e6a-d21c-4763-adec-b796beb6e8d6 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:40.789923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe03d5f-235d-4fcf-a36e-69de1de4471f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:40.793550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe03d5f-235d-4fcf-a36e-69de1de4471f admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:40.793550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe03d5f-235d-4fcf-a36e-69de1de4471f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:40.793550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe03d5f-235d-4fcf-a36e-69de1de4471f admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:40.794345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe03d5f-235d-4fcf-a36e-69de1de4471f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:40.796730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe03d5f-235d-4fcf-a36e-69de1de4471f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:40.810581 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4050/8165] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:23:40.811829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5252f8-92c1-4cdd-96e5-2420549767ab admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:40.825574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5252f8-92c1-4cdd-96e5-2420549767ab admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:40.830541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d35b69-4edc-4a60-abf5-56d6fbd5cceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:40.831151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d35b69-4edc-4a60-abf5-56d6fbd5cceb admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:40.831432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d35b69-4edc-4a60-abf5-56d6fbd5cceb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:40.831707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d35b69-4edc-4a60-abf5-56d6fbd5cceb admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:40.832388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d35b69-4edc-4a60-abf5-56d6fbd5cceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:40.835759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d35b69-4edc-4a60-abf5-56d6fbd5cceb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:40.848587 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4116/8166] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:23:40.853596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe03d5f-235d-4fcf-a36e-69de1de4471f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:40.865802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe03d5f-235d-4fcf-a36e-69de1de4471f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:40.873877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebaae4-a912-45e4-ae22-66f575f1ae14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:40.874931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebaae4-a912-45e4-ae22-66f575f1ae14 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:40.875218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebaae4-a912-45e4-ae22-66f575f1ae14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:40.875493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebaae4-a912-45e4-ae22-66f575f1ae14 admin admin] PATH_INFO: `/v3/projects/3702ba8854c041f4ad1a3de3347dd5b3/users/fb9b3c506e6244b28569e68fb36ad5fc/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:40.898183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ebaae4-a912-45e4-ae22-66f575f1ae14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3702ba8854c041f4ad1a3de3347dd5b3, user_id=fb9b3c506e6244b28569e68fb36ad5fc, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:40.901609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ebaae4-a912-45e4-ae22-66f575f1ae14 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:40.919476 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4051/8167] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:23:40 2022] PUT /identity/v3/projects/3702ba8854c041f4ad1a3de3347dd5b3/users/fb9b3c506e6244b28569e68fb36ad5fc/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:23:40.932133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dd27b-5727-4f93-8d28-db95f00392f1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:40.932423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dd27b-5727-4f93-8d28-db95f00392f1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:40.932713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dd27b-5727-4f93-8d28-db95f00392f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:40.969888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4a9dd27b-5727-4f93-8d28-db95f00392f1 None None] MFA Rules not processed for user `fb9b3c506e6244b28569e68fb36ad5fc`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:23:40.971170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4a9dd27b-5727-4f93-8d28-db95f00392f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:41.034790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:41.068737 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4117/8168] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:23:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:23:41.080465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebaae4-a912-45e4-ae22-66f575f1ae14 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:23:41.082282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-69ebaae4-a912-45e4-ae22-66f575f1ae14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:41.106856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:41.161387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebaae4-a912-45e4-ae22-66f575f1ae14 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:23:41.169286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15184a-1f0c-4d46-baa7-d66c4473e171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:23:41.169921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15184a-1f0c-4d46-baa7-d66c4473e171 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:41.170189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15184a-1f0c-4d46-baa7-d66c4473e171 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:41.170458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15184a-1f0c-4d46-baa7-d66c4473e171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:41.179784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15184a-1f0c-4d46-baa7-d66c4473e171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:23:41.181919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15184a-1f0c-4d46-baa7-d66c4473e171 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:23:41.192942 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4052/8169] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:23:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:23:42.652424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a93de-ab1a-447a-b828-5ea2aa047331 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:23:42.652943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a93de-ab1a-447a-b828-5ea2aa047331 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:23:42.653038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a93de-ab1a-447a-b828-5ea2aa047331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:23:42.675418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8d8a93de-ab1a-447a-b828-5ea2aa047331 None None] MFA Rules not processed for user `fb9b3c506e6244b28569e68fb36ad5fc`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:23:42.676684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8d8a93de-ab1a-447a-b828-5ea2aa047331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:23:42.684446 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4118/8170] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:23:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:23:51.188729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=18 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:52.669549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=10 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:24:00.585585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c406fcf-d898-4822-b983-e25c1b38950b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:00.586046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c406fcf-d898-4822-b983-e25c1b38950b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:00.586245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c406fcf-d898-4822-b983-e25c1b38950b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:00.609524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4c406fcf-d898-4822-b983-e25c1b38950b None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:00.610804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4c406fcf-d898-4822-b983-e25c1b38950b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:00.677682 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4053/8171] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:24:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:24:00.690594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a93de-ab1a-447a-b828-5ea2aa047331 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:00.692149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8d8a93de-ab1a-447a-b828-5ea2aa047331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:00.774821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a93de-ab1a-447a-b828-5ea2aa047331 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:00.781063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2274c5-a6f8-4071-a3da-c82711f76d34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:00.781829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2274c5-a6f8-4071-a3da-c82711f76d34 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:00.782127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2274c5-a6f8-4071-a3da-c82711f76d34 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:00.782458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2274c5-a6f8-4071-a3da-c82711f76d34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:00.794438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2274c5-a6f8-4071-a3da-c82711f76d34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:00.796138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2274c5-a6f8-4071-a3da-c82711f76d34 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:00.805740 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4119/8172] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:24:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:24:01.348628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e54dd-50ba-4a1c-a002-bdf7026761a4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:01.348986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e54dd-50ba-4a1c-a002-bdf7026761a4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:01.349360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e54dd-50ba-4a1c-a002-bdf7026761a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:01.370313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8d3e54dd-50ba-4a1c-a002-bdf7026761a4 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:01.371563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8d3e54dd-50ba-4a1c-a002-bdf7026761a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:01.378820 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4054/8173] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:24:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:24:01.392519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2274c5-a6f8-4071-a3da-c82711f76d34 service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:01.406275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2274c5-a6f8-4071-a3da-c82711f76d34 service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:01.411514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44eb635-4895-4213-bb9f-039a44989fc2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:01.412112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44eb635-4895-4213-bb9f-039a44989fc2 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:01.412393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44eb635-4895-4213-bb9f-039a44989fc2 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:01.412669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44eb635-4895-4213-bb9f-039a44989fc2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:01.421384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44eb635-4895-4213-bb9f-039a44989fc2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:01.423096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44eb635-4895-4213-bb9f-039a44989fc2 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:01.432001 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4120/8174] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:24:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:24:01.684680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abf535-28bb-4042-bb69-d6935b40abdc None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:01.685192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abf535-28bb-4042-bb69-d6935b40abdc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:01.685265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abf535-28bb-4042-bb69-d6935b40abdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:01.710351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d6abf535-28bb-4042-bb69-d6935b40abdc None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:01.712013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d6abf535-28bb-4042-bb69-d6935b40abdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:01.720685 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4055/8175] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:24:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:24:01.735313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44eb635-4895-4213-bb9f-039a44989fc2 service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:01.750524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44eb635-4895-4213-bb9f-039a44989fc2 service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:01.756597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c3139-60f7-4aa7-ae89-eaa20074c597 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:01.757209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c3139-60f7-4aa7-ae89-eaa20074c597 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:01.757511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c3139-60f7-4aa7-ae89-eaa20074c597 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:01.757786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c3139-60f7-4aa7-ae89-eaa20074c597 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:01.767674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985c3139-60f7-4aa7-ae89-eaa20074c597 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:01.769516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985c3139-60f7-4aa7-ae89-eaa20074c597 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:01.779165 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4121/8176] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:24:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:24:02.124659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abf535-28bb-4042-bb69-d6935b40abdc None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:02.126718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d6abf535-28bb-4042-bb69-d6935b40abdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:02.138178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b609f-a31a-4287-b4b8-fcd3a08b34ea None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:02.138681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b609f-a31a-4287-b4b8-fcd3a08b34ea None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:02.139012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b609f-a31a-4287-b4b8-fcd3a08b34ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:02.161024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7f2b609f-a31a-4287-b4b8-fcd3a08b34ea None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:02.162451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7f2b609f-a31a-4287-b4b8-fcd3a08b34ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:02.171030 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4122/8177] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:24:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:24:02.185925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b609f-a31a-4287-b4b8-fcd3a08b34ea None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:02.201974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abf535-28bb-4042-bb69-d6935b40abdc None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:02.202842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b609f-a31a-4287-b4b8-fcd3a08b34ea None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:02.207687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2561b459-087f-4b7a-8f24-21ecc89b1157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:02.208360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561b459-087f-4b7a-8f24-21ecc89b1157 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:02.208647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561b459-087f-4b7a-8f24-21ecc89b1157 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:02.209035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561b459-087f-4b7a-8f24-21ecc89b1157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:02.210442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95299c0-f016-4605-839c-d1548bbda3b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:02.211278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95299c0-f016-4605-839c-d1548bbda3b9 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:02.211559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95299c0-f016-4605-839c-d1548bbda3b9 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:02.211774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2561b459-087f-4b7a-8f24-21ecc89b1157 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:02.211867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95299c0-f016-4605-839c-d1548bbda3b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:02.221829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95299c0-f016-4605-839c-d1548bbda3b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:02.223406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95299c0-f016-4605-839c-d1548bbda3b9 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:02.232187 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4123/8178] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:24:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 21:24:02.315549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2561b459-087f-4b7a-8f24-21ecc89b1157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:02.317084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2561b459-087f-4b7a-8f24-21ecc89b1157 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:02.334351 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4056/8179] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:24:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:24:04.476989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95299c0-f016-4605-839c-d1548bbda3b9 service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:04.563373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95299c0-f016-4605-839c-d1548bbda3b9 service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:04.568038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a7505-05be-4c9f-ac2d-2c50f0fda6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:04.568704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a7505-05be-4c9f-ac2d-2c50f0fda6aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:04.568985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a7505-05be-4c9f-ac2d-2c50f0fda6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:04.569260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a7505-05be-4c9f-ac2d-2c50f0fda6aa admin admin] PATH_INFO: `/v3/users/fb9b3c506e6244b28569e68fb36ad5fc` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:04.570416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a7505-05be-4c9f-ac2d-2c50f0fda6aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb9b3c506e6244b28569e68fb36ad5fc)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:04.573536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a7505-05be-4c9f-ac2d-2c50f0fda6aa admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:04.617976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:24:04.629712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fe3a7505-05be-4c9f-ac2d-2c50f0fda6aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb9b3c506e6244b28569e68fb36ad5fc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:04.637844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fe3a7505-05be-4c9f-ac2d-2c50f0fda6aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb9b3c506e6244b28569e68fb36ad5fc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:04.651097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fe3a7505-05be-4c9f-ac2d-2c50f0fda6aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb9b3c506e6244b28569e68fb36ad5fc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:04.651883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fe3a7505-05be-4c9f-ac2d-2c50f0fda6aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb9b3c506e6244b28569e68fb36ad5fc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:04.660192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fe3a7505-05be-4c9f-ac2d-2c50f0fda6aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb9b3c506e6244b28569e68fb36ad5fc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:04.668808 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4124/8180] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:24:04 2022] DELETE /identity/v3/users/fb9b3c506e6244b28569e68fb36ad5fc => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:24:04.673425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8962b11-38bb-41f6-9cc0-7386ce9fdd24 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:04.673980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8962b11-38bb-41f6-9cc0-7386ce9fdd24 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:04.674067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8962b11-38bb-41f6-9cc0-7386ce9fdd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:04.713734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b8962b11-38bb-41f6-9cc0-7386ce9fdd24 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:04.715130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b8962b11-38bb-41f6-9cc0-7386ce9fdd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:04.733460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:24:04.782369 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4057/8181] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:24:04.793181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a7505-05be-4c9f-ac2d-2c50f0fda6aa admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:04.794761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fe3a7505-05be-4c9f-ac2d-2c50f0fda6aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:04.865892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a7505-05be-4c9f-ac2d-2c50f0fda6aa admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:04.870741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8c56a-2adf-4962-b854-d0411bb24b92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:04.871412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8c56a-2adf-4962-b854-d0411bb24b92 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:04.871706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8c56a-2adf-4962-b854-d0411bb24b92 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:04.871984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8c56a-2adf-4962-b854-d0411bb24b92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:04.882021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8c56a-2adf-4962-b854-d0411bb24b92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:04.883721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8c56a-2adf-4962-b854-d0411bb24b92 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:04.895743 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4125/8182] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:24:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:24:05.130025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8962b11-38bb-41f6-9cc0-7386ce9fdd24 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:05.132120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b8962b11-38bb-41f6-9cc0-7386ce9fdd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:05.146346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8962b11-38bb-41f6-9cc0-7386ce9fdd24 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:05.168431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679379f7-ed88-461f-b4c5-be16c7a0ad95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:05.168569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679379f7-ed88-461f-b4c5-be16c7a0ad95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:05.168737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679379f7-ed88-461f-b4c5-be16c7a0ad95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:05.169095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679379f7-ed88-461f-b4c5-be16c7a0ad95 admin admin] PATH_INFO: `/v3/projects/3702ba8854c041f4ad1a3de3347dd5b3` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:05.170653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679379f7-ed88-461f-b4c5-be16c7a0ad95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3702ba8854c041f4ad1a3de3347dd5b3)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:05.173289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679379f7-ed88-461f-b4c5-be16c7a0ad95 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:05.236989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-679379f7-ed88-461f-b4c5-be16c7a0ad95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3702ba8854c041f4ad1a3de3347dd5b3', 'request_id': 'req-679379f7-ed88-461f-b4c5-be16c7a0ad95'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:05.245928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-679379f7-ed88-461f-b4c5-be16c7a0ad95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3702ba8854c041f4ad1a3de3347dd5b3', 'request_id': 'req-679379f7-ed88-461f-b4c5-be16c7a0ad95'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:05.253846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-679379f7-ed88-461f-b4c5-be16c7a0ad95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3702ba8854c041f4ad1a3de3347dd5b3', 'request_id': 'req-679379f7-ed88-461f-b4c5-be16c7a0ad95'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:05.279250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-679379f7-ed88-461f-b4c5-be16c7a0ad95 admin admin] The token cache is being invalidate because project 3702ba8854c041f4ad1a3de3347dd5b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:24:05.279250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-679379f7-ed88-461f-b4c5-be16c7a0ad95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:05.282692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-679379f7-ed88-461f-b4c5-be16c7a0ad95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:05.285934 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4058/8183] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:24:05 2022] DELETE /identity/v3/projects/3702ba8854c041f4ad1a3de3347dd5b3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:24:05.300603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd288e-856b-4a0a-8b55-62c43adbfe0a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:05.300917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd288e-856b-4a0a-8b55-62c43adbfe0a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:05.301212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd288e-856b-4a0a-8b55-62c43adbfe0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:05.332926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fbcd288e-856b-4a0a-8b55-62c43adbfe0a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:05.334238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fbcd288e-856b-4a0a-8b55-62c43adbfe0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:05.406887 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4126/8184] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:24:05.414816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f01d9a-1863-4dd8-b7b2-dd6ae676079c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:05.415256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f01d9a-1863-4dd8-b7b2-dd6ae676079c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:05.415335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f01d9a-1863-4dd8-b7b2-dd6ae676079c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:05.437763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b4f01d9a-1863-4dd8-b7b2-dd6ae676079c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:05.439051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b4f01d9a-1863-4dd8-b7b2-dd6ae676079c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:05.448208 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4059/8185] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:24:05.453195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd288e-856b-4a0a-8b55-62c43adbfe0a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:05.465571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd288e-856b-4a0a-8b55-62c43adbfe0a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:05.470946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6a569-e810-4dfc-80a6-39e29731dbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:05.471552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6a569-e810-4dfc-80a6-39e29731dbed admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:05.471868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6a569-e810-4dfc-80a6-39e29731dbed admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:05.472429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6a569-e810-4dfc-80a6-39e29731dbed admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:05.472820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6a569-e810-4dfc-80a6-39e29731dbed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:24:05.473164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6a569-e810-4dfc-80a6-39e29731dbed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:05.474953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6a569-e810-4dfc-80a6-39e29731dbed admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:05.490611 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4127/8186] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:24:05.492954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f01d9a-1863-4dd8-b7b2-dd6ae676079c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:05.503403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f01d9a-1863-4dd8-b7b2-dd6ae676079c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:05.509459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0a75c-0087-455f-bd04-eb05158b267c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:05.510127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0a75c-0087-455f-bd04-eb05158b267c admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:05.510455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0a75c-0087-455f-bd04-eb05158b267c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:05.510801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0a75c-0087-455f-bd04-eb05158b267c admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:05.511449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0a75c-0087-455f-bd04-eb05158b267c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:05.513424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0a75c-0087-455f-bd04-eb05158b267c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:05.548031 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4060/8187] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:24:05 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:24:05.552487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6a569-e810-4dfc-80a6-39e29731dbed admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:05.564318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6a569-e810-4dfc-80a6-39e29731dbed admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:05.618139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:24:05.637104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ae11d-faec-4b48-bf65-acb64b9a241a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:05.637675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ae11d-faec-4b48-bf65-acb64b9a241a admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:05.637965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ae11d-faec-4b48-bf65-acb64b9a241a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:05.638221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ae11d-faec-4b48-bf65-acb64b9a241a admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:05.638876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ae11d-faec-4b48-bf65-acb64b9a241a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:05.641279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ae11d-faec-4b48-bf65-acb64b9a241a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:05.667840 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4128/8188] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:24:05 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:24:05.672435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0a75c-0087-455f-bd04-eb05158b267c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:05.684388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0a75c-0087-455f-bd04-eb05158b267c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:05.692249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90980a1f-e306-4abf-bf4e-133c4815ac78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:05.693064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90980a1f-e306-4abf-bf4e-133c4815ac78 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:05.693497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90980a1f-e306-4abf-bf4e-133c4815ac78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:05.693805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90980a1f-e306-4abf-bf4e-133c4815ac78 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:05.694839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90980a1f-e306-4abf-bf4e-133c4815ac78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:05.697423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90980a1f-e306-4abf-bf4e-133c4815ac78 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:05.711971 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4061/8189] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:24:05.716844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ae11d-faec-4b48-bf65-acb64b9a241a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:05.728907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ae11d-faec-4b48-bf65-acb64b9a241a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:05.735441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd457cc-e9b5-46bd-a037-548e2d51b9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:05.736174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd457cc-e9b5-46bd-a037-548e2d51b9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:05.736461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd457cc-e9b5-46bd-a037-548e2d51b9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:05.736732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd457cc-e9b5-46bd-a037-548e2d51b9c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:05.737379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd457cc-e9b5-46bd-a037-548e2d51b9c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:05.739015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd457cc-e9b5-46bd-a037-548e2d51b9c8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:05.751246 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4129/8190] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:24:05.755549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90980a1f-e306-4abf-bf4e-133c4815ac78 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:05.766932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90980a1f-e306-4abf-bf4e-133c4815ac78 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:05.771874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0b629-6574-48df-9873-0598edf047a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:05.772738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0b629-6574-48df-9873-0598edf047a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:05.772964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0b629-6574-48df-9873-0598edf047a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:05.773243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0b629-6574-48df-9873-0598edf047a8 admin admin] PATH_INFO: `/v3/projects/dd6e51791d934c73bf7d92e7ed1a8909/users/98361240b014405cb20cf9c145061be6/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:05.789682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0b629-6574-48df-9873-0598edf047a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd6e51791d934c73bf7d92e7ed1a8909, user_id=98361240b014405cb20cf9c145061be6, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:05.792928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0b629-6574-48df-9873-0598edf047a8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:05.806051 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4062/8191] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:24:05 2022] PUT /identity/v3/projects/dd6e51791d934c73bf7d92e7ed1a8909/users/98361240b014405cb20cf9c145061be6/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:24:05.811589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953919d8-43cc-4700-a9b5-9b202cd3d652 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:05.811911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953919d8-43cc-4700-a9b5-9b202cd3d652 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:05.812224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953919d8-43cc-4700-a9b5-9b202cd3d652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:05.832410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-953919d8-43cc-4700-a9b5-9b202cd3d652 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:05.833617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-953919d8-43cc-4700-a9b5-9b202cd3d652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:05.905321 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4130/8192] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:24:05.915245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0b629-6574-48df-9873-0598edf047a8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:05.917281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-dba0b629-6574-48df-9873-0598edf047a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:05.969307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:24:05.985746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0b629-6574-48df-9873-0598edf047a8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:05.989999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00bcf5-3b25-48a7-ae23-7d333b871dfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:05.990680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00bcf5-3b25-48a7-ae23-7d333b871dfd service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:05.990967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00bcf5-3b25-48a7-ae23-7d333b871dfd service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:05.991357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00bcf5-3b25-48a7-ae23-7d333b871dfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:06.002556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00bcf5-3b25-48a7-ae23-7d333b871dfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:06.004015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00bcf5-3b25-48a7-ae23-7d333b871dfd service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:06.014593 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4063/8193] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:24:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:24:06.791529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fbe78-b633-444d-a9f6-d06b2bd8657a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:06.792133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fbe78-b633-444d-a9f6-d06b2bd8657a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:06.792133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fbe78-b633-444d-a9f6-d06b2bd8657a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:06.831320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6c9fbe78-b633-444d-a9f6-d06b2bd8657a None None] MFA Rules not processed for user `98361240b014405cb20cf9c145061be6`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:06.832832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6c9fbe78-b633-444d-a9f6-d06b2bd8657a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:06.928696 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4131/8194] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:24:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:24:06.941331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00bcf5-3b25-48a7-ae23-7d333b871dfd service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:06.943882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ab00bcf5-3b25-48a7-ae23-7d333b871dfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:07.038145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00bcf5-3b25-48a7-ae23-7d333b871dfd service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:07.046464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d9442-633d-4ac0-8a5e-4998adad8b02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:07.047240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d9442-633d-4ac0-8a5e-4998adad8b02 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:07.047591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d9442-633d-4ac0-8a5e-4998adad8b02 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:07.047900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d9442-633d-4ac0-8a5e-4998adad8b02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:07.059337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d9442-633d-4ac0-8a5e-4998adad8b02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:07.060966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d9442-633d-4ac0-8a5e-4998adad8b02 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:07.071451 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4064/8195] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:24:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:24:10.738606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34391676-156f-4a46-af55-520f2a0defec None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:10.739021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34391676-156f-4a46-af55-520f2a0defec None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:10.739309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34391676-156f-4a46-af55-520f2a0defec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:10.758949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-34391676-156f-4a46-af55-520f2a0defec None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:10.760703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-34391676-156f-4a46-af55-520f2a0defec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:10.808455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:24:10.817978 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4132/8196] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:24:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:24:10.842014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d9442-633d-4ac0-8a5e-4998adad8b02 service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:10.843473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e34d9442-633d-4ac0-8a5e-4998adad8b02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:10.924518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d9442-633d-4ac0-8a5e-4998adad8b02 service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:10.929554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d7979-75cd-4949-982f-7c6647af10f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:10.930155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d7979-75cd-4949-982f-7c6647af10f6 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:10.930450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d7979-75cd-4949-982f-7c6647af10f6 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:10.930744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d7979-75cd-4949-982f-7c6647af10f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:10.941100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d7979-75cd-4949-982f-7c6647af10f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:10.942923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d7979-75cd-4949-982f-7c6647af10f6 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:10.952806 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4065/8197] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:24:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:24:12.011083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34391676-156f-4a46-af55-520f2a0defec None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:12.012765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-34391676-156f-4a46-af55-520f2a0defec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:12.161696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34391676-156f-4a46-af55-520f2a0defec None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:12.172499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd42154-a31b-4c04-a80a-5367e4ac8aff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:12.173289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd42154-a31b-4c04-a80a-5367e4ac8aff service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:12.173662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd42154-a31b-4c04-a80a-5367e4ac8aff service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:12.173949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd42154-a31b-4c04-a80a-5367e4ac8aff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:12.182145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fbd42154-a31b-4c04-a80a-5367e4ac8aff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:12.210207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd42154-a31b-4c04-a80a-5367e4ac8aff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:12.212656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd42154-a31b-4c04-a80a-5367e4ac8aff service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:12.242729 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4133/8198] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:24:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:24:20.325551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82588e4b-5c1b-4184-a59a-afbc9969d8a2 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:20.326065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82588e4b-5c1b-4184-a59a-afbc9969d8a2 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:20.326151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82588e4b-5c1b-4184-a59a-afbc9969d8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:20.346527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-82588e4b-5c1b-4184-a59a-afbc9969d8a2 None None] MFA Rules not processed for user `98361240b014405cb20cf9c145061be6`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:20.347740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-82588e4b-5c1b-4184-a59a-afbc9969d8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:20.355253 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4066/8199] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:24:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:24:22.223174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=33 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:24:28.359275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd42154-a31b-4c04-a80a-5367e4ac8aff service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:28.371806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd42154-a31b-4c04-a80a-5367e4ac8aff service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:28.375548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83ea8d-0fbb-4ae0-914b-ab7518dd074b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:28.376152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83ea8d-0fbb-4ae0-914b-ab7518dd074b service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:28.376434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83ea8d-0fbb-4ae0-914b-ab7518dd074b service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:28.376746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83ea8d-0fbb-4ae0-914b-ab7518dd074b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:28.378474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7e83ea8d-0fbb-4ae0-914b-ab7518dd074b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:28.391000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e83ea8d-0fbb-4ae0-914b-ab7518dd074b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:28.392413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e83ea8d-0fbb-4ae0-914b-ab7518dd074b service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:28.403331 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4134/8200] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:24:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:24:30.340957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=68 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:24:30.438368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82588e4b-5c1b-4184-a59a-afbc9969d8a2 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:30.451662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82588e4b-5c1b-4184-a59a-afbc9969d8a2 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:30.455621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1324d-5969-4c24-905e-5524ac5a0243 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:30.456434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1324d-5969-4c24-905e-5524ac5a0243 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:30.456735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1324d-5969-4c24-905e-5524ac5a0243 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:30.456998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1324d-5969-4c24-905e-5524ac5a0243 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:30.459068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f0a1324d-5969-4c24-905e-5524ac5a0243 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:30.537887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a1324d-5969-4c24-905e-5524ac5a0243 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:30.539416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a1324d-5969-4c24-905e-5524ac5a0243 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:30.555868 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4067/8201] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:24:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:24:31.554460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83ea8d-0fbb-4ae0-914b-ab7518dd074b service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:31.556780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7e83ea8d-0fbb-4ae0-914b-ab7518dd074b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:31.579597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83ea8d-0fbb-4ae0-914b-ab7518dd074b service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:31.587954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71fe00-ccc8-4af6-bc2b-b893fb2ed5ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:31.588581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71fe00-ccc8-4af6-bc2b-b893fb2ed5ed service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:31.588861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71fe00-ccc8-4af6-bc2b-b893fb2ed5ed service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:31.589134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71fe00-ccc8-4af6-bc2b-b893fb2ed5ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:31.594578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6f71fe00-ccc8-4af6-bc2b-b893fb2ed5ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:31.616899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71fe00-ccc8-4af6-bc2b-b893fb2ed5ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:31.619195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71fe00-ccc8-4af6-bc2b-b893fb2ed5ed service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:31.641248 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4135/8202] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:24:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:24:31.653227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1324d-5969-4c24-905e-5524ac5a0243 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:31.655472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f0a1324d-5969-4c24-905e-5524ac5a0243 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:31.682062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1324d-5969-4c24-905e-5524ac5a0243 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:31.687659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cae778-349e-473f-92e5-0fc3e73794be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:31.688509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cae778-349e-473f-92e5-0fc3e73794be admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:31.688944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cae778-349e-473f-92e5-0fc3e73794be admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:31.689217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cae778-349e-473f-92e5-0fc3e73794be admin admin] PATH_INFO: `/v3/users/6ac020dc323042c48ccce2a8826bc6be` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:31.690355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cae778-349e-473f-92e5-0fc3e73794be admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ac020dc323042c48ccce2a8826bc6be)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:31.692292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cae778-349e-473f-92e5-0fc3e73794be admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:31.739853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c6cae778-349e-473f-92e5-0fc3e73794be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ac020dc323042c48ccce2a8826bc6be'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:31.740510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c6cae778-349e-473f-92e5-0fc3e73794be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ac020dc323042c48ccce2a8826bc6be'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:31.741121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c6cae778-349e-473f-92e5-0fc3e73794be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ac020dc323042c48ccce2a8826bc6be'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:31.748498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c6cae778-349e-473f-92e5-0fc3e73794be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ac020dc323042c48ccce2a8826bc6be'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:31.761013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c6cae778-349e-473f-92e5-0fc3e73794be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ac020dc323042c48ccce2a8826bc6be'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:31.774247 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4068/8203] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:24:31 2022] DELETE /identity/v3/users/6ac020dc323042c48ccce2a8826bc6be => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:24:31.947155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71fe00-ccc8-4af6-bc2b-b893fb2ed5ed service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:31.958097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71fe00-ccc8-4af6-bc2b-b893fb2ed5ed service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:32.015194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc10e8-b68f-4bd6-b180-f5fb91ecd7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:32.015825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc10e8-b68f-4bd6-b180-f5fb91ecd7be admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:32.016110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc10e8-b68f-4bd6-b180-f5fb91ecd7be admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:32.016388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc10e8-b68f-4bd6-b180-f5fb91ecd7be admin admin] PATH_INFO: `/v3/projects/9e09544743724cf18d2ab2aeb4180d81` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:32.017511 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc10e8-b68f-4bd6-b180-f5fb91ecd7be admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e09544743724cf18d2ab2aeb4180d81)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:32.019456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc10e8-b68f-4bd6-b180-f5fb91ecd7be admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:32.066144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0abc10e8-b68f-4bd6-b180-f5fb91ecd7be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e09544743724cf18d2ab2aeb4180d81', 'request_id': 'req-0abc10e8-b68f-4bd6-b180-f5fb91ecd7be'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:32.072982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0abc10e8-b68f-4bd6-b180-f5fb91ecd7be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e09544743724cf18d2ab2aeb4180d81', 'request_id': 'req-0abc10e8-b68f-4bd6-b180-f5fb91ecd7be'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:32.078622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0abc10e8-b68f-4bd6-b180-f5fb91ecd7be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e09544743724cf18d2ab2aeb4180d81', 'request_id': 'req-0abc10e8-b68f-4bd6-b180-f5fb91ecd7be'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:32.091982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0abc10e8-b68f-4bd6-b180-f5fb91ecd7be admin admin] The token cache is being invalidate because project 9e09544743724cf18d2ab2aeb4180d81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:24:32.092272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0abc10e8-b68f-4bd6-b180-f5fb91ecd7be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:32.093063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-0abc10e8-b68f-4bd6-b180-f5fb91ecd7be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:32.094837 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4136/8204] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:24:31 2022] DELETE /identity/v3/projects/9e09544743724cf18d2ab2aeb4180d81 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:24:32.099601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56c231-c1f4-4780-bf9a-e94e472ee9db None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:32.099889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56c231-c1f4-4780-bf9a-e94e472ee9db None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:32.100165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56c231-c1f4-4780-bf9a-e94e472ee9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:32.119351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ad56c231-c1f4-4780-bf9a-e94e472ee9db None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:32.120530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ad56c231-c1f4-4780-bf9a-e94e472ee9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:32.171745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:24:32.183472 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4069/8205] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:24:32.191090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cdd1b-03ed-4640-9d4f-a83bbebb1e64 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:32.191377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cdd1b-03ed-4640-9d4f-a83bbebb1e64 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:32.191651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cdd1b-03ed-4640-9d4f-a83bbebb1e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:32.211847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-340cdd1b-03ed-4640-9d4f-a83bbebb1e64 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:32.213035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-340cdd1b-03ed-4640-9d4f-a83bbebb1e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:32.221928 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4137/8206] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:24:32.225143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56c231-c1f4-4780-bf9a-e94e472ee9db None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:32.235503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56c231-c1f4-4780-bf9a-e94e472ee9db None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:32.240087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0099b2-44dc-46c2-a828-28f7baf6b1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:32.240697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0099b2-44dc-46c2-a828-28f7baf6b1a3 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:32.240973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0099b2-44dc-46c2-a828-28f7baf6b1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:32.241242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0099b2-44dc-46c2-a828-28f7baf6b1a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:32.241884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0099b2-44dc-46c2-a828-28f7baf6b1a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:24:32.242185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0099b2-44dc-46c2-a828-28f7baf6b1a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:32.244576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0099b2-44dc-46c2-a828-28f7baf6b1a3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:32.256737 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4070/8207] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:24:32.261560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340cdd1b-03ed-4640-9d4f-a83bbebb1e64 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:32.274245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340cdd1b-03ed-4640-9d4f-a83bbebb1e64 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:32.279794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fe547-708b-4cca-90ec-a38781d58e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:32.280407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fe547-708b-4cca-90ec-a38781d58e0c admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:32.280690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fe547-708b-4cca-90ec-a38781d58e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:32.280962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fe547-708b-4cca-90ec-a38781d58e0c admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:32.282251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6fe547-708b-4cca-90ec-a38781d58e0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:32.283515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6fe547-708b-4cca-90ec-a38781d58e0c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:32.314535 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4138/8208] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:24:32 2022] POST /identity/v3/projects => generated 397 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:24:32.318163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0099b2-44dc-46c2-a828-28f7baf6b1a3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:32.328478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0099b2-44dc-46c2-a828-28f7baf6b1a3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:32.386582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e49a7f-734a-427c-815d-bc0468ac5af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:32.387186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e49a7f-734a-427c-815d-bc0468ac5af9 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:32.387466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e49a7f-734a-427c-815d-bc0468ac5af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:32.387732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e49a7f-734a-427c-815d-bc0468ac5af9 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:32.388351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e49a7f-734a-427c-815d-bc0468ac5af9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:32.390291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e49a7f-734a-427c-815d-bc0468ac5af9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:32.416833 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4071/8209] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:24:32 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:24:32.421428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fe547-708b-4cca-90ec-a38781d58e0c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:32.432837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fe547-708b-4cca-90ec-a38781d58e0c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:32.437340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95869f6f-a0f3-432c-bafc-a858417c70b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:32.437943 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95869f6f-a0f3-432c-bafc-a858417c70b2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:32.438225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95869f6f-a0f3-432c-bafc-a858417c70b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:32.438629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95869f6f-a0f3-432c-bafc-a858417c70b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:32.439269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95869f6f-a0f3-432c-bafc-a858417c70b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:32.440845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95869f6f-a0f3-432c-bafc-a858417c70b2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:32.451644 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4139/8210] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:24:32.455758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e49a7f-734a-427c-815d-bc0468ac5af9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:32.466406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e49a7f-734a-427c-815d-bc0468ac5af9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:32.472292 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25d092-2647-4120-b634-2ceeeab82ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:32.473168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25d092-2647-4120-b634-2ceeeab82ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:32.473455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25d092-2647-4120-b634-2ceeeab82ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:32.473726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25d092-2647-4120-b634-2ceeeab82ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:32.474394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25d092-2647-4120-b634-2ceeeab82ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:32.476539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25d092-2647-4120-b634-2ceeeab82ef7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:32.492552 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4072/8211] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:24:32.493264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95869f6f-a0f3-432c-bafc-a858417c70b2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:32.504100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95869f6f-a0f3-432c-bafc-a858417c70b2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:32.509589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d4bf8-3a3f-4c9c-9876-1376100383cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:32.510484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d4bf8-3a3f-4c9c-9876-1376100383cc admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:32.510821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d4bf8-3a3f-4c9c-9876-1376100383cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:32.511098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d4bf8-3a3f-4c9c-9876-1376100383cc admin admin] PATH_INFO: `/v3/projects/b6bf21b68848471081c62ade9d6ec793/users/32fcb91152c84fd8b7260e4febb07ade/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:32.527020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d4bf8-3a3f-4c9c-9876-1376100383cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6bf21b68848471081c62ade9d6ec793, user_id=32fcb91152c84fd8b7260e4febb07ade, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:32.529850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d4bf8-3a3f-4c9c-9876-1376100383cc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:32.538965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:24:32.545328 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4140/8212] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:24:32 2022] PUT /identity/v3/projects/b6bf21b68848471081c62ade9d6ec793/users/32fcb91152c84fd8b7260e4febb07ade/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:24:32.550787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f742b-b9b1-457f-b972-25365e837797 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:32.551072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f742b-b9b1-457f-b972-25365e837797 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:32.551345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f742b-b9b1-457f-b972-25365e837797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:32.573763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-162f742b-b9b1-457f-b972-25365e837797 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:32.575091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-162f742b-b9b1-457f-b972-25365e837797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:32.639047 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4073/8213] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:24:32.649536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d4bf8-3a3f-4c9c-9876-1376100383cc admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:32.651481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-203d4bf8-3a3f-4c9c-9876-1376100383cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:32.713836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d4bf8-3a3f-4c9c-9876-1376100383cc admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:32.717688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a8c6b-4b33-42e2-aacd-b4dff9cdb418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:32.718324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a8c6b-4b33-42e2-aacd-b4dff9cdb418 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:32.718642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a8c6b-4b33-42e2-aacd-b4dff9cdb418 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:32.718916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a8c6b-4b33-42e2-aacd-b4dff9cdb418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:32.728148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a8c6b-4b33-42e2-aacd-b4dff9cdb418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:32.729563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a8c6b-4b33-42e2-aacd-b4dff9cdb418 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:32.739140 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4141/8214] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:24:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:24:37.053529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d7923-209f-4b29-8134-e532b694667d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:37.054046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d7923-209f-4b29-8134-e532b694667d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:37.054440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d7923-209f-4b29-8134-e532b694667d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:37.089159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-596d7923-209f-4b29-8134-e532b694667d None None] MFA Rules not processed for user `32fcb91152c84fd8b7260e4febb07ade`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:37.090526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-596d7923-209f-4b29-8134-e532b694667d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:37.162783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:24:37.177468 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4074/8215] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:24:37 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:24:37.189852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4343a8c-77c1-4c73-9610-177d32cfaab6 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:37.190289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4343a8c-77c1-4c73-9610-177d32cfaab6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:37.190424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4343a8c-77c1-4c73-9610-177d32cfaab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:37.213624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e4343a8c-77c1-4c73-9610-177d32cfaab6 None None] MFA Rules not processed for user `32fcb91152c84fd8b7260e4febb07ade`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:37.214939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e4343a8c-77c1-4c73-9610-177d32cfaab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:37.223975 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4142/8216] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:24:37 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:24:37.235067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d7923-209f-4b29-8134-e532b694667d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:37.236735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-596d7923-209f-4b29-8134-e532b694667d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:37.333631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d7923-209f-4b29-8134-e532b694667d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:37.341541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974964d9-ea75-4838-9dc1-2228557b9116 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:37.342518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974964d9-ea75-4838-9dc1-2228557b9116 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:37.342818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974964d9-ea75-4838-9dc1-2228557b9116 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:37.343143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974964d9-ea75-4838-9dc1-2228557b9116 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:37.353458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974964d9-ea75-4838-9dc1-2228557b9116 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:37.355231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974964d9-ea75-4838-9dc1-2228557b9116 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:37.377644 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4075/8217] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:24:37 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:24:37.661999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4343a8c-77c1-4c73-9610-177d32cfaab6 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:37.663478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e4343a8c-77c1-4c73-9610-177d32cfaab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:37.678351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4343a8c-77c1-4c73-9610-177d32cfaab6 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:37.682438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8277cf4-96bd-4e75-8f60-713976c60180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:37.683115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8277cf4-96bd-4e75-8f60-713976c60180 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:37.683429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8277cf4-96bd-4e75-8f60-713976c60180 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:37.683669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8277cf4-96bd-4e75-8f60-713976c60180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:37.685397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c8277cf4-96bd-4e75-8f60-713976c60180 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:37.698911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8277cf4-96bd-4e75-8f60-713976c60180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:37.700282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8277cf4-96bd-4e75-8f60-713976c60180 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:37.711973 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4143/8218] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:24:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:24:40.315742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974964d9-ea75-4838-9dc1-2228557b9116 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:40.332230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974964d9-ea75-4838-9dc1-2228557b9116 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:40.337573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afba8f-e14e-48be-8221-da2505ad0ebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:40.338441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afba8f-e14e-48be-8221-da2505ad0ebd service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:40.338829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afba8f-e14e-48be-8221-da2505ad0ebd service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:40.339317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afba8f-e14e-48be-8221-da2505ad0ebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:40.341258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e9afba8f-e14e-48be-8221-da2505ad0ebd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:40.372376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afba8f-e14e-48be-8221-da2505ad0ebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:40.374218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afba8f-e14e-48be-8221-da2505ad0ebd service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:40.391009 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4076/8219] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:24:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:24:41.219735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8277cf4-96bd-4e75-8f60-713976c60180 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:41.249682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8277cf4-96bd-4e75-8f60-713976c60180 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:41.258613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17923e34-ded3-49cd-a0fb-e67225830518 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:41.259500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17923e34-ded3-49cd-a0fb-e67225830518 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:41.259881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17923e34-ded3-49cd-a0fb-e67225830518 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:41.260245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17923e34-ded3-49cd-a0fb-e67225830518 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:41.262471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-17923e34-ded3-49cd-a0fb-e67225830518 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:41.282363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17923e34-ded3-49cd-a0fb-e67225830518 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:41.284250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17923e34-ded3-49cd-a0fb-e67225830518 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:41.300015 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4144/8220] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:24:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:24:50.379359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=43 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:24:51.291651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=41 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:24:54.572455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afba8f-e14e-48be-8221-da2505ad0ebd service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:54.592094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afba8f-e14e-48be-8221-da2505ad0ebd service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:54.596825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3169b6f-b32a-4a6b-84a5-331c9bcafabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:54.597464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3169b6f-b32a-4a6b-84a5-331c9bcafabb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:54.597747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3169b6f-b32a-4a6b-84a5-331c9bcafabb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:54.598035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3169b6f-b32a-4a6b-84a5-331c9bcafabb admin admin] PATH_INFO: `/v3/users/32fcb91152c84fd8b7260e4febb07ade` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:54.599153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3169b6f-b32a-4a6b-84a5-331c9bcafabb admin admin] RBAC: Authorizing `identity:delete_user(user_id=32fcb91152c84fd8b7260e4febb07ade)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:54.601275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3169b6f-b32a-4a6b-84a5-331c9bcafabb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:54.649761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c3169b6f-b32a-4a6b-84a5-331c9bcafabb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32fcb91152c84fd8b7260e4febb07ade'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:54.650452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c3169b6f-b32a-4a6b-84a5-331c9bcafabb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32fcb91152c84fd8b7260e4febb07ade'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:54.651119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c3169b6f-b32a-4a6b-84a5-331c9bcafabb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32fcb91152c84fd8b7260e4febb07ade'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:54.659079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c3169b6f-b32a-4a6b-84a5-331c9bcafabb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32fcb91152c84fd8b7260e4febb07ade'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:54.672686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c3169b6f-b32a-4a6b-84a5-331c9bcafabb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32fcb91152c84fd8b7260e4febb07ade'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:54.685386 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4077/8221] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:24:54 2022] DELETE /identity/v3/users/32fcb91152c84fd8b7260e4febb07ade => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:24:54.858963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17923e34-ded3-49cd-a0fb-e67225830518 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:54.870344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17923e34-ded3-49cd-a0fb-e67225830518 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:54.929321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341303fa-f7ce-43d1-bfee-27fb824214c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:54.929994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341303fa-f7ce-43d1-bfee-27fb824214c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:54.930278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341303fa-f7ce-43d1-bfee-27fb824214c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:54.930659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341303fa-f7ce-43d1-bfee-27fb824214c3 admin admin] PATH_INFO: `/v3/projects/b6bf21b68848471081c62ade9d6ec793` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:54.931817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341303fa-f7ce-43d1-bfee-27fb824214c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6bf21b68848471081c62ade9d6ec793)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:54.933700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341303fa-f7ce-43d1-bfee-27fb824214c3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:54.979789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-341303fa-f7ce-43d1-bfee-27fb824214c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6bf21b68848471081c62ade9d6ec793', 'request_id': 'req-341303fa-f7ce-43d1-bfee-27fb824214c3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:54.985800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-341303fa-f7ce-43d1-bfee-27fb824214c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6bf21b68848471081c62ade9d6ec793', 'request_id': 'req-341303fa-f7ce-43d1-bfee-27fb824214c3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:54.991601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-341303fa-f7ce-43d1-bfee-27fb824214c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6bf21b68848471081c62ade9d6ec793', 'request_id': 'req-341303fa-f7ce-43d1-bfee-27fb824214c3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:55.004384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-341303fa-f7ce-43d1-bfee-27fb824214c3 admin admin] The token cache is being invalidate because project b6bf21b68848471081c62ade9d6ec793 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:24:55.004666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-341303fa-f7ce-43d1-bfee-27fb824214c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:55.005286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-341303fa-f7ce-43d1-bfee-27fb824214c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:24:55.006819 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4145/8222] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:24:54 2022] DELETE /identity/v3/projects/b6bf21b68848471081c62ade9d6ec793 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:24:55.011771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355ddcf-7fbc-491f-9246-5952f7ae3d58 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:55.012050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355ddcf-7fbc-491f-9246-5952f7ae3d58 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:55.012362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355ddcf-7fbc-491f-9246-5952f7ae3d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:55.033007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2355ddcf-7fbc-491f-9246-5952f7ae3d58 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:55.034135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2355ddcf-7fbc-491f-9246-5952f7ae3d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:55.096008 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4078/8223] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:24:55.103828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4a15f-0736-4b89-ae72-b8912ac038ac None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:55.104123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4a15f-0736-4b89-ae72-b8912ac038ac None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:55.104402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4a15f-0736-4b89-ae72-b8912ac038ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:55.123833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b4b4a15f-0736-4b89-ae72-b8912ac038ac None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:55.125059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b4b4a15f-0736-4b89-ae72-b8912ac038ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:55.134102 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4146/8224] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:24:55.138710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355ddcf-7fbc-491f-9246-5952f7ae3d58 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:55.149815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355ddcf-7fbc-491f-9246-5952f7ae3d58 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:55.154659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d18379-6fb3-4103-a4be-52823b190c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:55.155261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d18379-6fb3-4103-a4be-52823b190c65 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:55.155537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d18379-6fb3-4103-a4be-52823b190c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:55.155810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d18379-6fb3-4103-a4be-52823b190c65 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:55.156477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d18379-6fb3-4103-a4be-52823b190c65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:24:55.156782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d18379-6fb3-4103-a4be-52823b190c65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:55.158397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d18379-6fb3-4103-a4be-52823b190c65 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:55.170283 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4079/8225] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:24:55.174527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4a15f-0736-4b89-ae72-b8912ac038ac None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:55.186055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4a15f-0736-4b89-ae72-b8912ac038ac None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:55.190801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069c73d-496b-4463-9c3e-fcaf567e32c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:55.191393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069c73d-496b-4463-9c3e-fcaf567e32c8 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:55.191669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069c73d-496b-4463-9c3e-fcaf567e32c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:55.191936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069c73d-496b-4463-9c3e-fcaf567e32c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:55.192562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069c73d-496b-4463-9c3e-fcaf567e32c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:55.194436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069c73d-496b-4463-9c3e-fcaf567e32c8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:55.222574 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4147/8226] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:24:55 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:24:55.226702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d18379-6fb3-4103-a4be-52823b190c65 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:55.237323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d18379-6fb3-4103-a4be-52823b190c65 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:55.293774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f357f42-9b49-4bf2-b8b7-f111a615a64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:55.294312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f357f42-9b49-4bf2-b8b7-f111a615a64d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:55.294626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f357f42-9b49-4bf2-b8b7-f111a615a64d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:55.294904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f357f42-9b49-4bf2-b8b7-f111a615a64d admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:55.295523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f357f42-9b49-4bf2-b8b7-f111a615a64d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:55.297453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f357f42-9b49-4bf2-b8b7-f111a615a64d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:55.305576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:24:55.319838 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4080/8227] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:24:55 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:24:55.324566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069c73d-496b-4463-9c3e-fcaf567e32c8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:55.334871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069c73d-496b-4463-9c3e-fcaf567e32c8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:55.340090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f9dfb-f949-4df9-8e43-57fb32632fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:55.340683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f9dfb-f949-4df9-8e43-57fb32632fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:55.340961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f9dfb-f949-4df9-8e43-57fb32632fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:55.341233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f9dfb-f949-4df9-8e43-57fb32632fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:55.341870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f9dfb-f949-4df9-8e43-57fb32632fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:55.343478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f9dfb-f949-4df9-8e43-57fb32632fd6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:55.354075 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4148/8228] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:24:55.357944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f357f42-9b49-4bf2-b8b7-f111a615a64d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:55.368705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f357f42-9b49-4bf2-b8b7-f111a615a64d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:55.373468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef3afd-9c32-41ac-9660-780572f6795e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:55.374071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef3afd-9c32-41ac-9660-780572f6795e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:55.374349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef3afd-9c32-41ac-9660-780572f6795e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:55.374686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef3afd-9c32-41ac-9660-780572f6795e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:55.375293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef3afd-9c32-41ac-9660-780572f6795e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:55.376867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef3afd-9c32-41ac-9660-780572f6795e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:55.387611 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4081/8229] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:24:55.391755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f9dfb-f949-4df9-8e43-57fb32632fd6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:55.401843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f9dfb-f949-4df9-8e43-57fb32632fd6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:55.406553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3dc74-346f-45bb-800b-66266a8ad0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:55.407199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3dc74-346f-45bb-800b-66266a8ad0f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:55.407479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3dc74-346f-45bb-800b-66266a8ad0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:55.407750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3dc74-346f-45bb-800b-66266a8ad0f8 admin admin] PATH_INFO: `/v3/projects/70dcbab212774655a7094ebc0cb9d800/users/3f7614694df347bea9be492787dbd455/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:55.422712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3dc74-346f-45bb-800b-66266a8ad0f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70dcbab212774655a7094ebc0cb9d800, user_id=3f7614694df347bea9be492787dbd455, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:55.425346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3dc74-346f-45bb-800b-66266a8ad0f8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:55.438767 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4149/8230] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:24:55 2022] PUT /identity/v3/projects/70dcbab212774655a7094ebc0cb9d800/users/3f7614694df347bea9be492787dbd455/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:24:55.449101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed258e7c-c6e7-4509-93a8-17a2c34fff5a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:55.449386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed258e7c-c6e7-4509-93a8-17a2c34fff5a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:55.449663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed258e7c-c6e7-4509-93a8-17a2c34fff5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:55.478227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ed258e7c-c6e7-4509-93a8-17a2c34fff5a None None] MFA Rules not processed for user `3f7614694df347bea9be492787dbd455`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:55.479460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ed258e7c-c6e7-4509-93a8-17a2c34fff5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:55.542467 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4082/8231] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:24:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:24:55.553961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0de84-7194-4239-92e5-fda87b265ee1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:55.554191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0de84-7194-4239-92e5-fda87b265ee1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:55.554465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0de84-7194-4239-92e5-fda87b265ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:55.573060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e8a0de84-7194-4239-92e5-fda87b265ee1 None None] MFA Rules not processed for user `3f7614694df347bea9be492787dbd455`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:55.574257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e8a0de84-7194-4239-92e5-fda87b265ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:55.581156 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4150/8232] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:24:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:24:55.593537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed258e7c-c6e7-4509-93a8-17a2c34fff5a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:24:55.595008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ed258e7c-c6e7-4509-93a8-17a2c34fff5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:55.671120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed258e7c-c6e7-4509-93a8-17a2c34fff5a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:24:55.676655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9619b-269a-46d8-b476-9f01ac56dde2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:24:55.677256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9619b-269a-46d8-b476-9f01ac56dde2 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:55.677539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9619b-269a-46d8-b476-9f01ac56dde2 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:55.677813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9619b-269a-46d8-b476-9f01ac56dde2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:55.685515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9619b-269a-46d8-b476-9f01ac56dde2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:24:55.687138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9619b-269a-46d8-b476-9f01ac56dde2 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:24:55.695916 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4083/8233] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:24:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:24:55.974450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d9e52-f43a-4aca-aa2b-6378f949750c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:24:55.974964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d9e52-f43a-4aca-aa2b-6378f949750c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:24:55.975075 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d9e52-f43a-4aca-aa2b-6378f949750c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:24:55.995173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a36d9e52-f43a-4aca-aa2b-6378f949750c None None] MFA Rules not processed for user `3f7614694df347bea9be492787dbd455`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:24:55.996371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a36d9e52-f43a-4aca-aa2b-6378f949750c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:24:56.003414 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4151/8234] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:24:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:25:02.803516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9619b-269a-46d8-b476-9f01ac56dde2 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:02.805646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-23a9619b-269a-46d8-b476-9f01ac56dde2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:02.863366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9619b-269a-46d8-b476-9f01ac56dde2 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:02.867746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545c9b0-55d7-4cea-80a4-cf71141fad98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:02.868336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545c9b0-55d7-4cea-80a4-cf71141fad98 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:02.868612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545c9b0-55d7-4cea-80a4-cf71141fad98 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:02.868904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545c9b0-55d7-4cea-80a4-cf71141fad98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:02.870625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3545c9b0-55d7-4cea-80a4-cf71141fad98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:02.877609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:02.883086 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545c9b0-55d7-4cea-80a4-cf71141fad98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:02.884765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545c9b0-55d7-4cea-80a4-cf71141fad98 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:02.895399 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4084/8235] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:25:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:25:05.989832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:12.889517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:38.610928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d9e52-f43a-4aca-aa2b-6378f949750c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:38.680995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d9e52-f43a-4aca-aa2b-6378f949750c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:38.685751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad443e3-d6e5-4425-8600-a3da6aa1aa4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:38.686377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad443e3-d6e5-4425-8600-a3da6aa1aa4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:38.686689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad443e3-d6e5-4425-8600-a3da6aa1aa4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:38.686961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad443e3-d6e5-4425-8600-a3da6aa1aa4c admin admin] PATH_INFO: `/v3/users/3f7614694df347bea9be492787dbd455` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:38.688087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad443e3-d6e5-4425-8600-a3da6aa1aa4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f7614694df347bea9be492787dbd455)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:38.689983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad443e3-d6e5-4425-8600-a3da6aa1aa4c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:38.738984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5ad443e3-d6e5-4425-8600-a3da6aa1aa4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f7614694df347bea9be492787dbd455'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:38.746398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5ad443e3-d6e5-4425-8600-a3da6aa1aa4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f7614694df347bea9be492787dbd455'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:38.758763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5ad443e3-d6e5-4425-8600-a3da6aa1aa4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f7614694df347bea9be492787dbd455'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:38.759548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5ad443e3-d6e5-4425-8600-a3da6aa1aa4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f7614694df347bea9be492787dbd455'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:38.768582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5ad443e3-d6e5-4425-8600-a3da6aa1aa4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f7614694df347bea9be492787dbd455'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:38.775008 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4152/8236] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:25:38 2022] DELETE /identity/v3/users/3f7614694df347bea9be492787dbd455 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:25:38.780012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96f45b-0b0a-45b7-8516-fb3602081679 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:38.780505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96f45b-0b0a-45b7-8516-fb3602081679 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:38.780618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96f45b-0b0a-45b7-8516-fb3602081679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:38.800284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3f96f45b-0b0a-45b7-8516-fb3602081679 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:38.801416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3f96f45b-0b0a-45b7-8516-fb3602081679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:38.861491 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4085/8237] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:25:38.872037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad443e3-d6e5-4425-8600-a3da6aa1aa4c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:38.873521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5ad443e3-d6e5-4425-8600-a3da6aa1aa4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:38.936849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad443e3-d6e5-4425-8600-a3da6aa1aa4c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:38.940797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1d9ae-d53c-4aa0-8a24-f7f64c113ce0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:38.941409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1d9ae-d53c-4aa0-8a24-f7f64c113ce0 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:38.941692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1d9ae-d53c-4aa0-8a24-f7f64c113ce0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:38.941970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1d9ae-d53c-4aa0-8a24-f7f64c113ce0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:38.951984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1d9ae-d53c-4aa0-8a24-f7f64c113ce0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:38.953420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1d9ae-d53c-4aa0-8a24-f7f64c113ce0 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:38.963569 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4153/8238] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:25:39.269804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96f45b-0b0a-45b7-8516-fb3602081679 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:39.271993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3f96f45b-0b0a-45b7-8516-fb3602081679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:39.288379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96f45b-0b0a-45b7-8516-fb3602081679 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:39.295249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458769d-8c1b-467f-8140-3d8cce7aebe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:39.295911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458769d-8c1b-467f-8140-3d8cce7aebe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:39.296192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458769d-8c1b-467f-8140-3d8cce7aebe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:39.296470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458769d-8c1b-467f-8140-3d8cce7aebe9 admin admin] PATH_INFO: `/v3/projects/70dcbab212774655a7094ebc0cb9d800` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:39.297648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6458769d-8c1b-467f-8140-3d8cce7aebe9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70dcbab212774655a7094ebc0cb9d800)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:39.299731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6458769d-8c1b-467f-8140-3d8cce7aebe9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:39.360560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6458769d-8c1b-467f-8140-3d8cce7aebe9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70dcbab212774655a7094ebc0cb9d800', 'request_id': 'req-6458769d-8c1b-467f-8140-3d8cce7aebe9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:39.371809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6458769d-8c1b-467f-8140-3d8cce7aebe9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70dcbab212774655a7094ebc0cb9d800', 'request_id': 'req-6458769d-8c1b-467f-8140-3d8cce7aebe9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:39.384393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6458769d-8c1b-467f-8140-3d8cce7aebe9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70dcbab212774655a7094ebc0cb9d800', 'request_id': 'req-6458769d-8c1b-467f-8140-3d8cce7aebe9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:39.399358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6458769d-8c1b-467f-8140-3d8cce7aebe9 admin admin] The token cache is being invalidate because project 70dcbab212774655a7094ebc0cb9d800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:39.399780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6458769d-8c1b-467f-8140-3d8cce7aebe9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:39.400523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6458769d-8c1b-467f-8140-3d8cce7aebe9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:39.402136 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4086/8239] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:39 2022] DELETE /identity/v3/projects/70dcbab212774655a7094ebc0cb9d800 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:39.407092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4154a-a96f-438b-a864-0cd373a0df29 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:39.407488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4154a-a96f-438b-a864-0cd373a0df29 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:39.407863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4154a-a96f-438b-a864-0cd373a0df29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:39.432133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8fa4154a-a96f-438b-a864-0cd373a0df29 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:39.434129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8fa4154a-a96f-438b-a864-0cd373a0df29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:39.452922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:39.504800 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4154/8240] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:25:39.515228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573398e-0d92-4552-a84f-d27bafb640ea None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:39.515507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573398e-0d92-4552-a84f-d27bafb640ea None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:39.516324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573398e-0d92-4552-a84f-d27bafb640ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:39.538845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4573398e-0d92-4552-a84f-d27bafb640ea None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:39.540487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4573398e-0d92-4552-a84f-d27bafb640ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:39.557822 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4087/8241] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:25:39.559729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4154a-a96f-438b-a864-0cd373a0df29 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:39.572281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4154a-a96f-438b-a864-0cd373a0df29 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:39.577719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2459c1b-c9ec-4636-9c22-1871e5ae10c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:39.578696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2459c1b-c9ec-4636-9c22-1871e5ae10c3 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:39.579199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2459c1b-c9ec-4636-9c22-1871e5ae10c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:39.579671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2459c1b-c9ec-4636-9c22-1871e5ae10c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:39.580703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2459c1b-c9ec-4636-9c22-1871e5ae10c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:25:39.581220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2459c1b-c9ec-4636-9c22-1871e5ae10c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:39.583222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2459c1b-c9ec-4636-9c22-1871e5ae10c3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:39.599384 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4155/8242] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:39.603843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573398e-0d92-4552-a84f-d27bafb640ea None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:39.628451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573398e-0d92-4552-a84f-d27bafb640ea None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:39.637832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae1eb0-3294-4951-a3dc-dfa3b9e4e7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:39.638598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae1eb0-3294-4951-a3dc-dfa3b9e4e7ff admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:39.638951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae1eb0-3294-4951-a3dc-dfa3b9e4e7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:39.639284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae1eb0-3294-4951-a3dc-dfa3b9e4e7ff admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:39.639987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae1eb0-3294-4951-a3dc-dfa3b9e4e7ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:39.642078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae1eb0-3294-4951-a3dc-dfa3b9e4e7ff admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:39.689819 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4088/8243] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:25:39 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:39.694989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2459c1b-c9ec-4636-9c22-1871e5ae10c3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:39.706729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2459c1b-c9ec-4636-9c22-1871e5ae10c3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:39.775042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba88629-f418-4b5f-b1d7-e454c42bb0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:39.775666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba88629-f418-4b5f-b1d7-e454c42bb0b0 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:39.775950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba88629-f418-4b5f-b1d7-e454c42bb0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:39.776219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba88629-f418-4b5f-b1d7-e454c42bb0b0 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:39.777383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba88629-f418-4b5f-b1d7-e454c42bb0b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:39.779349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba88629-f418-4b5f-b1d7-e454c42bb0b0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:39.805467 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4156/8244] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:25:39 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:39.809231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae1eb0-3294-4951-a3dc-dfa3b9e4e7ff admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:39.821024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae1eb0-3294-4951-a3dc-dfa3b9e4e7ff admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:39.825985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e57f44-2549-440e-8a97-6543215ff108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:39.826632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e57f44-2549-440e-8a97-6543215ff108 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:39.826920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e57f44-2549-440e-8a97-6543215ff108 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:39.827196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e57f44-2549-440e-8a97-6543215ff108 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:39.827840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e57f44-2549-440e-8a97-6543215ff108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:39.829458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e57f44-2549-440e-8a97-6543215ff108 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:39.842122 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4089/8245] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:25:39.846407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba88629-f418-4b5f-b1d7-e454c42bb0b0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:39.857944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba88629-f418-4b5f-b1d7-e454c42bb0b0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:39.862983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b11f2-5c10-4417-a2bb-916f4ae3816f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:39.863613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b11f2-5c10-4417-a2bb-916f4ae3816f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:39.863901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b11f2-5c10-4417-a2bb-916f4ae3816f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:39.864174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b11f2-5c10-4417-a2bb-916f4ae3816f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:39.864836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7b11f2-5c10-4417-a2bb-916f4ae3816f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:39.866394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7b11f2-5c10-4417-a2bb-916f4ae3816f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:39.877926 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4157/8246] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:25:39.882310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e57f44-2549-440e-8a97-6543215ff108 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:39.895273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e57f44-2549-440e-8a97-6543215ff108 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:39.900330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d3674-0f68-4555-b250-421ed0bf86a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:39.901064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d3674-0f68-4555-b250-421ed0bf86a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:39.901524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d3674-0f68-4555-b250-421ed0bf86a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:39.901631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d3674-0f68-4555-b250-421ed0bf86a8 admin admin] PATH_INFO: `/v3/projects/47beac8bc27c419981753ac43892f566/users/cfa00e77721d4c8ea7e32aaa5ab427e1/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:39.920998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d3674-0f68-4555-b250-421ed0bf86a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47beac8bc27c419981753ac43892f566, user_id=cfa00e77721d4c8ea7e32aaa5ab427e1, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:39.923697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d3674-0f68-4555-b250-421ed0bf86a8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:39.936678 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4090/8247] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:25:39 2022] PUT /identity/v3/projects/47beac8bc27c419981753ac43892f566/users/cfa00e77721d4c8ea7e32aaa5ab427e1/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:39.946924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ced95-d135-43e5-8489-8df9588a8391 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:39.947885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ced95-d135-43e5-8489-8df9588a8391 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:39.948410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ced95-d135-43e5-8489-8df9588a8391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:39.978519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0b8ced95-d135-43e5-8489-8df9588a8391 None None] MFA Rules not processed for user `cfa00e77721d4c8ea7e32aaa5ab427e1`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:39.979830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0b8ced95-d135-43e5-8489-8df9588a8391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:40.047097 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4158/8248] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:25:40.051636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d3674-0f68-4555-b250-421ed0bf86a8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:40.073117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:40.120057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d3674-0f68-4555-b250-421ed0bf86a8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:40.124800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbad35-a758-4035-b524-c002767797f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:40.125408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbad35-a758-4035-b524-c002767797f0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:40.125698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbad35-a758-4035-b524-c002767797f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:40.126054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbad35-a758-4035-b524-c002767797f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:40.126603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbad35-a758-4035-b524-c002767797f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:40.129614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbad35-a758-4035-b524-c002767797f0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:40.163647 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4091/8249] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:25:40 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:40.168450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ced95-d135-43e5-8489-8df9588a8391 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:40.184421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ced95-d135-43e5-8489-8df9588a8391 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:40.200181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:40.259160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf28848-7e53-46c9-8dae-8ca966080207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:40.259830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf28848-7e53-46c9-8dae-8ca966080207 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:40.260129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf28848-7e53-46c9-8dae-8ca966080207 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:40.260473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf28848-7e53-46c9-8dae-8ca966080207 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:40.262030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf28848-7e53-46c9-8dae-8ca966080207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:40.264040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf28848-7e53-46c9-8dae-8ca966080207 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:40.290256 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4159/8250] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:25:40 2022] POST /identity/v3/users => generated 294 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:40.291725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbad35-a758-4035-b524-c002767797f0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:40.303485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbad35-a758-4035-b524-c002767797f0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:40.309310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db215f-08b6-49a3-9951-12e5d39c61f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:40.309940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db215f-08b6-49a3-9951-12e5d39c61f4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:40.310225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db215f-08b6-49a3-9951-12e5d39c61f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:40.310533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db215f-08b6-49a3-9951-12e5d39c61f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:40.311187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db215f-08b6-49a3-9951-12e5d39c61f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:40.313448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db215f-08b6-49a3-9951-12e5d39c61f4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:40.326088 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4092/8251] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:25:40.330374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf28848-7e53-46c9-8dae-8ca966080207 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:40.341252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf28848-7e53-46c9-8dae-8ca966080207 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:40.346317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b0813-310d-4c2d-bf5a-cf76454db6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:40.347128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b0813-310d-4c2d-bf5a-cf76454db6a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:40.347481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b0813-310d-4c2d-bf5a-cf76454db6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:40.347828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b0813-310d-4c2d-bf5a-cf76454db6a6 admin admin] PATH_INFO: `/v3/projects/8745adc4f77c49fb83158f914a1c34bf/users/0fda9f515b794bacbac55873ec49ee43/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:40.364065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b0813-310d-4c2d-bf5a-cf76454db6a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8745adc4f77c49fb83158f914a1c34bf, user_id=0fda9f515b794bacbac55873ec49ee43, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:40.366757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b0813-310d-4c2d-bf5a-cf76454db6a6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:40.382598 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4160/8252] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:25:40 2022] PUT /identity/v3/projects/8745adc4f77c49fb83158f914a1c34bf/users/0fda9f515b794bacbac55873ec49ee43/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:40.387382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db215f-08b6-49a3-9951-12e5d39c61f4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:40.398603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db215f-08b6-49a3-9951-12e5d39c61f4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:40.440230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036bc7e-0b91-4b31-9c0e-dd4bc4ae8584 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:40.440519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036bc7e-0b91-4b31-9c0e-dd4bc4ae8584 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:40.440797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036bc7e-0b91-4b31-9c0e-dd4bc4ae8584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:40.469598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3036bc7e-0b91-4b31-9c0e-dd4bc4ae8584 None None] MFA Rules not processed for user `eb0bf5fce885439eaa16a0871c2992ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:40.470870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3036bc7e-0b91-4b31-9c0e-dd4bc4ae8584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:40.475388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85042d09-aea3-4bdc-97fb-b93d63ac3ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:40.476099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85042d09-aea3-4bdc-97fb-b93d63ac3ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:40.476927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85042d09-aea3-4bdc-97fb-b93d63ac3ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:40.477214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85042d09-aea3-4bdc-97fb-b93d63ac3ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:40.478179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85042d09-aea3-4bdc-97fb-b93d63ac3ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:40.479902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85042d09-aea3-4bdc-97fb-b93d63ac3ae1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:40.491895 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4093/8253] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:25:40.496986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85042d09-aea3-4bdc-97fb-b93d63ac3ae1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:40.509716 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85042d09-aea3-4bdc-97fb-b93d63ac3ae1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:40.515280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b46ac-5da0-4c7d-a034-01ee05c5a213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:40.515935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b46ac-5da0-4c7d-a034-01ee05c5a213 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:40.516214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b46ac-5da0-4c7d-a034-01ee05c5a213 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:40.516488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b46ac-5da0-4c7d-a034-01ee05c5a213 admin admin] PATH_INFO: `/v3/projects/8745adc4f77c49fb83158f914a1c34bf/users/0fda9f515b794bacbac55873ec49ee43/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:40.518420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b46ac-5da0-4c7d-a034-01ee05c5a213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8745adc4f77c49fb83158f914a1c34bf, user_id=0fda9f515b794bacbac55873ec49ee43, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:40.521085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b46ac-5da0-4c7d-a034-01ee05c5a213 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:40.531917 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4094/8254] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:25:40 2022] PUT /identity/v3/projects/8745adc4f77c49fb83158f914a1c34bf/users/0fda9f515b794bacbac55873ec49ee43/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:25:40.539191 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4161/8255] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:25:40.544510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5647b4e-33c4-4cf6-b546-fc2f743d6df9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:40.544801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5647b4e-33c4-4cf6-b546-fc2f743d6df9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:40.545084 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5647b4e-33c4-4cf6-b546-fc2f743d6df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:40.575768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c5647b4e-33c4-4cf6-b546-fc2f743d6df9 None None] MFA Rules not processed for user `0fda9f515b794bacbac55873ec49ee43`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:40.576967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c5647b4e-33c4-4cf6-b546-fc2f743d6df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:40.657577 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4095/8256] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:25:40.662528 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036bc7e-0b91-4b31-9c0e-dd4bc4ae8584 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:40.741727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036bc7e-0b91-4b31-9c0e-dd4bc4ae8584 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:40.741727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d73cd-c6f6-45a3-a5d1-605d3e1a6b52 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fda9f515b794bacbac55873ec49ee43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8745adc4f77c49fb83158f914a1c34bf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:40.742402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d73cd-c6f6-45a3-a5d1-605d3e1a6b52 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:40.742402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d73cd-c6f6-45a3-a5d1-605d3e1a6b52 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:40.742617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d73cd-c6f6-45a3-a5d1-605d3e1a6b52 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:40.743255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d73cd-c6f6-45a3-a5d1-605d3e1a6b52 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:40.746540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d73cd-c6f6-45a3-a5d1-605d3e1a6b52 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:40.786386 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4162/8257] 10.222.0.17 () {60 vars in 1152 bytes} [Sat Oct 1 21:25:40 2022] POST /identity/v3/projects => generated 349 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:40.790229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5647b4e-33c4-4cf6-b546-fc2f743d6df9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:40.796069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:40.802562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5647b4e-33c4-4cf6-b546-fc2f743d6df9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:40.872611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc084002-886f-4007-abff-6568b7241417 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fda9f515b794bacbac55873ec49ee43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8745adc4f77c49fb83158f914a1c34bf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:40.874005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc084002-886f-4007-abff-6568b7241417 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:40.874005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc084002-886f-4007-abff-6568b7241417 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:40.874230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc084002-886f-4007-abff-6568b7241417 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:40.874970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc084002-886f-4007-abff-6568b7241417 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:40.877190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc084002-886f-4007-abff-6568b7241417 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:40.913731 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4096/8258] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:25:40 2022] POST /identity/v3/projects => generated 350 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:40.919134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d73cd-c6f6-45a3-a5d1-605d3e1a6b52 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:40.931304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d73cd-c6f6-45a3-a5d1-605d3e1a6b52 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:40.944405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:41.006835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8af957-c135-410b-ab63-10fe85c42a54 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fda9f515b794bacbac55873ec49ee43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8745adc4f77c49fb83158f914a1c34bf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:41.007568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8af957-c135-410b-ab63-10fe85c42a54 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:41.007934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8af957-c135-410b-ab63-10fe85c42a54 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:41.008283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8af957-c135-410b-ab63-10fe85c42a54 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] PATH_INFO: `/v3/users/cfa00e77721d4c8ea7e32aaa5ab427e1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:41.009499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8af957-c135-410b-ab63-10fe85c42a54 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorizing `identity:get_user(user_id=cfa00e77721d4c8ea7e32aaa5ab427e1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:41.012746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8af957-c135-410b-ab63-10fe85c42a54 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:41.014729 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4163/8259] 10.222.0.17 () {58 vars in 1221 bytes} [Sat Oct 1 21:25:40 2022] GET /identity/v3/users/cfa00e77721d4c8ea7e32aaa5ab427e1 => generated 288 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:25:41.019642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc084002-886f-4007-abff-6568b7241417 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:41.037345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc084002-886f-4007-abff-6568b7241417 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:41.042680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab49bd-fdfc-46e9-b48f-1a72ed15900c tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fda9f515b794bacbac55873ec49ee43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8745adc4f77c49fb83158f914a1c34bf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:41.043335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab49bd-fdfc-46e9-b48f-1a72ed15900c tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:41.043622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab49bd-fdfc-46e9-b48f-1a72ed15900c tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:41.043903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab49bd-fdfc-46e9-b48f-1a72ed15900c tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] PATH_INFO: `/v3/credentials` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:41.044562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab49bd-fdfc-46e9-b48f-1a72ed15900c tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:41.046465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab49bd-fdfc-46e9-b48f-1a72ed15900c tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:41.064341 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4097/8260] 10.222.0.17 () {60 vars in 1162 bytes} [Sat Oct 1 21:25:41 2022] POST /identity/v3/credentials => generated 429 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:41.068565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8af957-c135-410b-ab63-10fe85c42a54 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:41.079888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8af957-c135-410b-ab63-10fe85c42a54 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:41.084757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc3826-0c4d-46c1-a5de-cc24d8541288 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fda9f515b794bacbac55873ec49ee43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8745adc4f77c49fb83158f914a1c34bf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:41.085424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc3826-0c4d-46c1-a5de-cc24d8541288 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:41.085717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc3826-0c4d-46c1-a5de-cc24d8541288 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:41.086001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc3826-0c4d-46c1-a5de-cc24d8541288 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] PATH_INFO: `/v3/credentials/a5b116d46512c41086694ccf80538bef4d1af200b6397f624ea5594e04cf04ff` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:41.088334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc3826-0c4d-46c1-a5de-cc24d8541288 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorizing `identity:update_credential(credential_id=a5b116d46512c41086694ccf80538bef4d1af200b6397f624ea5594e04cf04ff)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:41.090258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc3826-0c4d-46c1-a5de-cc24d8541288 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:41.093425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc3826-0c4d-46c1-a5de-cc24d8541288 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorizing `identity:update_credential(credential_id=a5b116d46512c41086694ccf80538bef4d1af200b6397f624ea5594e04cf04ff)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:41.094966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc3826-0c4d-46c1-a5de-cc24d8541288 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:41.107777 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4164/8261] 10.222.0.17 () {60 vars in 1358 bytes} [Sat Oct 1 21:25:41 2022] PATCH /identity/v3/credentials/a5b116d46512c41086694ccf80538bef4d1af200b6397f624ea5594e04cf04ff => generated 437 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:25:41.112156 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab49bd-fdfc-46e9-b48f-1a72ed15900c tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:41.123227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab49bd-fdfc-46e9-b48f-1a72ed15900c tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:41.128287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67156c3-f6d1-4322-bbd3-ef4287f7d41e tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fda9f515b794bacbac55873ec49ee43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8745adc4f77c49fb83158f914a1c34bf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:41.129173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67156c3-f6d1-4322-bbd3-ef4287f7d41e tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:41.129503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67156c3-f6d1-4322-bbd3-ef4287f7d41e tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:41.129793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67156c3-f6d1-4322-bbd3-ef4287f7d41e tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] PATH_INFO: `/v3/credentials/a5b116d46512c41086694ccf80538bef4d1af200b6397f624ea5594e04cf04ff` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:41.132269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67156c3-f6d1-4322-bbd3-ef4287f7d41e tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorizing `identity:get_credential(credential_id=a5b116d46512c41086694ccf80538bef4d1af200b6397f624ea5594e04cf04ff)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:41.134057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67156c3-f6d1-4322-bbd3-ef4287f7d41e tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:41.140178 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4098/8262] 10.222.0.17 () {58 vars in 1335 bytes} [Sat Oct 1 21:25:41 2022] GET /identity/v3/credentials/a5b116d46512c41086694ccf80538bef4d1af200b6397f624ea5594e04cf04ff => generated 437 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:41.141180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc3826-0c4d-46c1-a5de-cc24d8541288 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:41.151414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc3826-0c4d-46c1-a5de-cc24d8541288 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:41.156224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c09bd8-6aba-4d15-8762-8f4ab228733d tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fda9f515b794bacbac55873ec49ee43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8745adc4f77c49fb83158f914a1c34bf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:41.156850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c09bd8-6aba-4d15-8762-8f4ab228733d tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:41.157138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c09bd8-6aba-4d15-8762-8f4ab228733d tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:41.157421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c09bd8-6aba-4d15-8762-8f4ab228733d tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] PATH_INFO: `/v3/credentials/a5b116d46512c41086694ccf80538bef4d1af200b6397f624ea5594e04cf04ff` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:41.159369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c09bd8-6aba-4d15-8762-8f4ab228733d tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=a5b116d46512c41086694ccf80538bef4d1af200b6397f624ea5594e04cf04ff)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:41.161106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c09bd8-6aba-4d15-8762-8f4ab228733d tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:41.178392 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4165/8263] 10.222.0.17 () {58 vars in 1338 bytes} [Sat Oct 1 21:25:41 2022] DELETE /identity/v3/credentials/a5b116d46512c41086694ccf80538bef4d1af200b6397f624ea5594e04cf04ff => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:41.184046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67156c3-f6d1-4322-bbd3-ef4287f7d41e tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:41.194596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67156c3-f6d1-4322-bbd3-ef4287f7d41e tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:41.200188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6fbf9-12a1-4758-866f-2851767d4091 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fda9f515b794bacbac55873ec49ee43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8745adc4f77c49fb83158f914a1c34bf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:41.200849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6fbf9-12a1-4758-866f-2851767d4091 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:41.201230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6fbf9-12a1-4758-866f-2851767d4091 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:41.201566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6fbf9-12a1-4758-866f-2851767d4091 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] PATH_INFO: `/v3/credentials` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:41.202299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6fbf9-12a1-4758-866f-2851767d4091 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:41.204325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6fbf9-12a1-4758-866f-2851767d4091 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:41.225184 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4099/8264] 10.222.0.17 () {60 vars in 1162 bytes} [Sat Oct 1 21:25:41 2022] POST /identity/v3/credentials => generated 430 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:41.226736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c09bd8-6aba-4d15-8762-8f4ab228733d tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:41.237199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c09bd8-6aba-4d15-8762-8f4ab228733d tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:41.241936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53da7c-6e6e-4ea6-810d-f321279635de tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fda9f515b794bacbac55873ec49ee43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8745adc4f77c49fb83158f914a1c34bf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:41.242571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53da7c-6e6e-4ea6-810d-f321279635de tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:41.242857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53da7c-6e6e-4ea6-810d-f321279635de tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:41.243131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53da7c-6e6e-4ea6-810d-f321279635de tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] PATH_INFO: `/v3/credentials` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:41.243791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee53da7c-6e6e-4ea6-810d-f321279635de tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:41.245508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee53da7c-6e6e-4ea6-810d-f321279635de tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:41.264143 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4166/8265] 10.222.0.17 () {60 vars in 1162 bytes} [Sat Oct 1 21:25:41 2022] POST /identity/v3/credentials => generated 428 bytes in 38 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:41.268162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6fbf9-12a1-4758-866f-2851767d4091 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:41.279479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6fbf9-12a1-4758-866f-2851767d4091 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:41.284265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a447fe-d811-478b-a495-daacd6d65541 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fda9f515b794bacbac55873ec49ee43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8745adc4f77c49fb83158f914a1c34bf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:41.284899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a447fe-d811-478b-a495-daacd6d65541 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:41.285186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a447fe-d811-478b-a495-daacd6d65541 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:41.285452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a447fe-d811-478b-a495-daacd6d65541 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] PATH_INFO: `/v3/credentials` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:41.286046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a447fe-d811-478b-a495-daacd6d65541 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorizing `identity:list_credentials()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:41.287609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a447fe-d811-478b-a495-daacd6d65541 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:41.295526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a447fe-d811-478b-a495-daacd6d65541 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorizing `identity:get_credential()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:41.297124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a447fe-d811-478b-a495-daacd6d65541 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:41.298792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a447fe-d811-478b-a495-daacd6d65541 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorizing `identity:get_credential()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:41.300441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a447fe-d811-478b-a495-daacd6d65541 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:41.302985 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4100/8266] 10.222.0.17 () {58 vars in 1140 bytes} [Sat Oct 1 21:25:41 2022] GET /identity/v3/credentials => generated 943 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:41.307297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53da7c-6e6e-4ea6-810d-f321279635de tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:41.318639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53da7c-6e6e-4ea6-810d-f321279635de tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:41.323391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cace58-4d2e-46ee-99bc-8210bc3c6576 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fda9f515b794bacbac55873ec49ee43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8745adc4f77c49fb83158f914a1c34bf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:41.324040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cace58-4d2e-46ee-99bc-8210bc3c6576 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:41.324321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cace58-4d2e-46ee-99bc-8210bc3c6576 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:41.324595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cace58-4d2e-46ee-99bc-8210bc3c6576 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] PATH_INFO: `/v3/credentials/e5a77ba2d5b54128516851762f0087771075725ac720c62f84af6af0e4f95c7a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:41.326514 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cace58-4d2e-46ee-99bc-8210bc3c6576 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=e5a77ba2d5b54128516851762f0087771075725ac720c62f84af6af0e4f95c7a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:41.328296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cace58-4d2e-46ee-99bc-8210bc3c6576 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:41.344234 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4167/8267] 10.222.0.17 () {58 vars in 1338 bytes} [Sat Oct 1 21:25:41 2022] DELETE /identity/v3/credentials/e5a77ba2d5b54128516851762f0087771075725ac720c62f84af6af0e4f95c7a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:41.348314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a447fe-d811-478b-a495-daacd6d65541 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:41.359353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a447fe-d811-478b-a495-daacd6d65541 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:41.364680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10372a-d5f2-4307-9d2a-9263275de739 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fda9f515b794bacbac55873ec49ee43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8745adc4f77c49fb83158f914a1c34bf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:41.365316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10372a-d5f2-4307-9d2a-9263275de739 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:41.365600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10372a-d5f2-4307-9d2a-9263275de739 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:41.365883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10372a-d5f2-4307-9d2a-9263275de739 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] PATH_INFO: `/v3/credentials/3aa2145549e6edae773eadcc6641c107a4d20cfb5a8cb6bbf2b027350bc2518c` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:41.367844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10372a-d5f2-4307-9d2a-9263275de739 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=3aa2145549e6edae773eadcc6641c107a4d20cfb5a8cb6bbf2b027350bc2518c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:41.369592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10372a-d5f2-4307-9d2a-9263275de739 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:41.389726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cace58-4d2e-46ee-99bc-8210bc3c6576 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:41.391869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-96cace58-4d2e-46ee-99bc-8210bc3c6576 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:41.393789 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4101/8268] 10.222.0.17 () {58 vars in 1338 bytes} [Sat Oct 1 21:25:41 2022] DELETE /identity/v3/credentials/3aa2145549e6edae773eadcc6641c107a4d20cfb5a8cb6bbf2b027350bc2518c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:41.398394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10372a-d5f2-4307-9d2a-9263275de739 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:41.409133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10372a-d5f2-4307-9d2a-9263275de739 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:41.414462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a64398-5c89-446a-9a1d-e95e2d7b4cb1 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fda9f515b794bacbac55873ec49ee43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8745adc4f77c49fb83158f914a1c34bf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:41.415052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a64398-5c89-446a-9a1d-e95e2d7b4cb1 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:41.415402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a64398-5c89-446a-9a1d-e95e2d7b4cb1 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:41.415744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a64398-5c89-446a-9a1d-e95e2d7b4cb1 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] PATH_INFO: `/v3/projects/3d3d18b1885f48b18dad73650c5c3958` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:41.417232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a64398-5c89-446a-9a1d-e95e2d7b4cb1 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3d3d18b1885f48b18dad73650c5c3958)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:41.420114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a64398-5c89-446a-9a1d-e95e2d7b4cb1 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:41.471767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d8a64398-5c89-446a-9a1d-e95e2d7b4cb1 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d3d18b1885f48b18dad73650c5c3958', 'request_id': 'req-d8a64398-5c89-446a-9a1d-e95e2d7b4cb1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:41.480347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d8a64398-5c89-446a-9a1d-e95e2d7b4cb1 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d3d18b1885f48b18dad73650c5c3958', 'request_id': 'req-d8a64398-5c89-446a-9a1d-e95e2d7b4cb1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:41.488338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d8a64398-5c89-446a-9a1d-e95e2d7b4cb1 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d3d18b1885f48b18dad73650c5c3958', 'request_id': 'req-d8a64398-5c89-446a-9a1d-e95e2d7b4cb1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:41.491485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cace58-4d2e-46ee-99bc-8210bc3c6576 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:41.500441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d8a64398-5c89-446a-9a1d-e95e2d7b4cb1 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] The token cache is being invalidate because project 3d3d18b1885f48b18dad73650c5c3958 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:41.500731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d8a64398-5c89-446a-9a1d-e95e2d7b4cb1 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:41.501480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d8a64398-5c89-446a-9a1d-e95e2d7b4cb1 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:41.503068 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4102/8269] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:41 2022] DELETE /identity/v3/projects/3d3d18b1885f48b18dad73650c5c3958 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:41.506959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a64398-5c89-446a-9a1d-e95e2d7b4cb1 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:41.508357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d8a64398-5c89-446a-9a1d-e95e2d7b4cb1 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:41.527641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:41.554381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305a726-7bc4-40bd-bc9b-e02c83330126 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:41.555034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305a726-7bc4-40bd-bc9b-e02c83330126 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:41.555339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305a726-7bc4-40bd-bc9b-e02c83330126 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:41.555620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305a726-7bc4-40bd-bc9b-e02c83330126 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:41.557428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a305a726-7bc4-40bd-bc9b-e02c83330126 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:41.565326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=96,keystone:UPDATE=1,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:41.572605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305a726-7bc4-40bd-bc9b-e02c83330126 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:41.575858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305a726-7bc4-40bd-bc9b-e02c83330126 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:41.577486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a64398-5c89-446a-9a1d-e95e2d7b4cb1 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:41.582356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cdd95c-5e5c-4607-813d-5ffa49b5d3e9 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fda9f515b794bacbac55873ec49ee43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8745adc4f77c49fb83158f914a1c34bf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:41.583019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdd95c-5e5c-4607-813d-5ffa49b5d3e9 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:41.583299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdd95c-5e5c-4607-813d-5ffa49b5d3e9 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:41.583571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdd95c-5e5c-4607-813d-5ffa49b5d3e9 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] PATH_INFO: `/v3/projects/a127132c4fb148bbb9c2ceaa9be2018a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:41.584690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cdd95c-5e5c-4607-813d-5ffa49b5d3e9 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a127132c4fb148bbb9c2ceaa9be2018a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:41.586620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cdd95c-5e5c-4607-813d-5ffa49b5d3e9 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:41.589088 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4168/8270] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:25:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:25:41.638480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-57cdd95c-5e5c-4607-813d-5ffa49b5d3e9 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a127132c4fb148bbb9c2ceaa9be2018a', 'request_id': 'req-57cdd95c-5e5c-4607-813d-5ffa49b5d3e9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:41.649999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-57cdd95c-5e5c-4607-813d-5ffa49b5d3e9 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a127132c4fb148bbb9c2ceaa9be2018a', 'request_id': 'req-57cdd95c-5e5c-4607-813d-5ffa49b5d3e9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:41.656284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-57cdd95c-5e5c-4607-813d-5ffa49b5d3e9 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a127132c4fb148bbb9c2ceaa9be2018a', 'request_id': 'req-57cdd95c-5e5c-4607-813d-5ffa49b5d3e9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:41.668146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-57cdd95c-5e5c-4607-813d-5ffa49b5d3e9 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] The token cache is being invalidate because project a127132c4fb148bbb9c2ceaa9be2018a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:41.668443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-57cdd95c-5e5c-4607-813d-5ffa49b5d3e9 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:41.669118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-57cdd95c-5e5c-4607-813d-5ffa49b5d3e9 tempest-CredentialsTestJSON-1082298562 tempest-CredentialsTestJSON-1082298562-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:41.670727 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4103/8271] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:41 2022] DELETE /identity/v3/projects/a127132c4fb148bbb9c2ceaa9be2018a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:25:41.674612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305a726-7bc4-40bd-bc9b-e02c83330126 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:41.676099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a305a726-7bc4-40bd-bc9b-e02c83330126 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:41.746010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305a726-7bc4-40bd-bc9b-e02c83330126 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:41.810582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca740a-ea5e-4339-8c38-863ec8ffa08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:41.811230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca740a-ea5e-4339-8c38-863ec8ffa08f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:41.811513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca740a-ea5e-4339-8c38-863ec8ffa08f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:41.811809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca740a-ea5e-4339-8c38-863ec8ffa08f admin admin] PATH_INFO: `/v3/users/cfa00e77721d4c8ea7e32aaa5ab427e1` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:41.812955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca740a-ea5e-4339-8c38-863ec8ffa08f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfa00e77721d4c8ea7e32aaa5ab427e1)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:41.815132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca740a-ea5e-4339-8c38-863ec8ffa08f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:41.863498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-40ca740a-ea5e-4339-8c38-863ec8ffa08f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfa00e77721d4c8ea7e32aaa5ab427e1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:41.871647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-40ca740a-ea5e-4339-8c38-863ec8ffa08f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfa00e77721d4c8ea7e32aaa5ab427e1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:41.885007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-40ca740a-ea5e-4339-8c38-863ec8ffa08f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfa00e77721d4c8ea7e32aaa5ab427e1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:41.885808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-40ca740a-ea5e-4339-8c38-863ec8ffa08f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfa00e77721d4c8ea7e32aaa5ab427e1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:41.894040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-40ca740a-ea5e-4339-8c38-863ec8ffa08f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfa00e77721d4c8ea7e32aaa5ab427e1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:41.900311 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4169/8272] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:25:41 2022] DELETE /identity/v3/users/cfa00e77721d4c8ea7e32aaa5ab427e1 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:41.904936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e5d5d-02e3-4027-953f-d7e9ebe87bec None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:41.905434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e5d5d-02e3-4027-953f-d7e9ebe87bec None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:41.905528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e5d5d-02e3-4027-953f-d7e9ebe87bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:41.927094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-927e5d5d-02e3-4027-953f-d7e9ebe87bec None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:41.928309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-927e5d5d-02e3-4027-953f-d7e9ebe87bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:41.990412 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4104/8273] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:25:42.001308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca740a-ea5e-4339-8c38-863ec8ffa08f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:42.002800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-40ca740a-ea5e-4339-8c38-863ec8ffa08f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:42.026894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:42.074659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca740a-ea5e-4339-8c38-863ec8ffa08f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:42.078923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ea7c0-0b23-4f9f-8996-7fb30f72f2b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:42.079627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ea7c0-0b23-4f9f-8996-7fb30f72f2b2 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:42.079808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ea7c0-0b23-4f9f-8996-7fb30f72f2b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:42.080084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ea7c0-0b23-4f9f-8996-7fb30f72f2b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:42.089652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ea7c0-0b23-4f9f-8996-7fb30f72f2b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:42.091111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ea7c0-0b23-4f9f-8996-7fb30f72f2b2 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:42.100984 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4170/8274] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:25:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:25:42.379670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e5d5d-02e3-4027-953f-d7e9ebe87bec None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:42.381162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-927e5d5d-02e3-4027-953f-d7e9ebe87bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:42.394215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e5d5d-02e3-4027-953f-d7e9ebe87bec None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:42.398869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438dcc2f-910d-4ed5-9811-b7efd36a2de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:42.399506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438dcc2f-910d-4ed5-9811-b7efd36a2de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:42.399785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438dcc2f-910d-4ed5-9811-b7efd36a2de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:42.400058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438dcc2f-910d-4ed5-9811-b7efd36a2de1 admin admin] PATH_INFO: `/v3/projects/47beac8bc27c419981753ac43892f566` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:42.401218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438dcc2f-910d-4ed5-9811-b7efd36a2de1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47beac8bc27c419981753ac43892f566)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:42.403725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438dcc2f-910d-4ed5-9811-b7efd36a2de1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:42.444759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:42.450973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-438dcc2f-910d-4ed5-9811-b7efd36a2de1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47beac8bc27c419981753ac43892f566', 'request_id': 'req-438dcc2f-910d-4ed5-9811-b7efd36a2de1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:42.459180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-438dcc2f-910d-4ed5-9811-b7efd36a2de1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47beac8bc27c419981753ac43892f566', 'request_id': 'req-438dcc2f-910d-4ed5-9811-b7efd36a2de1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:42.466078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-438dcc2f-910d-4ed5-9811-b7efd36a2de1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47beac8bc27c419981753ac43892f566', 'request_id': 'req-438dcc2f-910d-4ed5-9811-b7efd36a2de1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:42.477562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-438dcc2f-910d-4ed5-9811-b7efd36a2de1 admin admin] The token cache is being invalidate because project 47beac8bc27c419981753ac43892f566 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:42.477855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-438dcc2f-910d-4ed5-9811-b7efd36a2de1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:42.478704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-438dcc2f-910d-4ed5-9811-b7efd36a2de1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:42.480079 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4105/8275] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:42 2022] DELETE /identity/v3/projects/47beac8bc27c419981753ac43892f566 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:42.484314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ea7c0-0b23-4f9f-8996-7fb30f72f2b2 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:42.494148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ea7c0-0b23-4f9f-8996-7fb30f72f2b2 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:42.574651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c08cf0-f739-41fd-bd3d-404671b355b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:42.574651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c08cf0-f739-41fd-bd3d-404671b355b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:42.574651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c08cf0-f739-41fd-bd3d-404671b355b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:42.574651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c08cf0-f739-41fd-bd3d-404671b355b7 admin admin] PATH_INFO: `/v3/users/0fda9f515b794bacbac55873ec49ee43` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:42.575037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c08cf0-f739-41fd-bd3d-404671b355b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fda9f515b794bacbac55873ec49ee43)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:42.575037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c08cf0-f739-41fd-bd3d-404671b355b7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:42.626288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b9c08cf0-f739-41fd-bd3d-404671b355b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fda9f515b794bacbac55873ec49ee43'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:42.636766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b9c08cf0-f739-41fd-bd3d-404671b355b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fda9f515b794bacbac55873ec49ee43'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:42.649405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b9c08cf0-f739-41fd-bd3d-404671b355b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fda9f515b794bacbac55873ec49ee43'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:42.650158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b9c08cf0-f739-41fd-bd3d-404671b355b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fda9f515b794bacbac55873ec49ee43'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:42.661194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b9c08cf0-f739-41fd-bd3d-404671b355b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fda9f515b794bacbac55873ec49ee43'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:42.672363 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4171/8276] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:25:42 2022] DELETE /identity/v3/users/0fda9f515b794bacbac55873ec49ee43 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:43.123234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438dcc2f-910d-4ed5-9811-b7efd36a2de1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:43.125349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-438dcc2f-910d-4ed5-9811-b7efd36a2de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:43.147673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438dcc2f-910d-4ed5-9811-b7efd36a2de1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:43.251423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553bd0c-d6e6-4752-87e0-22cf72bbe5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:43.252208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553bd0c-d6e6-4752-87e0-22cf72bbe5cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:43.252621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553bd0c-d6e6-4752-87e0-22cf72bbe5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:43.252918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553bd0c-d6e6-4752-87e0-22cf72bbe5cf admin admin] PATH_INFO: `/v3/projects/8745adc4f77c49fb83158f914a1c34bf` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:43.254944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553bd0c-d6e6-4752-87e0-22cf72bbe5cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8745adc4f77c49fb83158f914a1c34bf)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:43.257144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553bd0c-d6e6-4752-87e0-22cf72bbe5cf admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:43.318246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3553bd0c-d6e6-4752-87e0-22cf72bbe5cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8745adc4f77c49fb83158f914a1c34bf', 'request_id': 'req-3553bd0c-d6e6-4752-87e0-22cf72bbe5cf'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:43.327271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3553bd0c-d6e6-4752-87e0-22cf72bbe5cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8745adc4f77c49fb83158f914a1c34bf', 'request_id': 'req-3553bd0c-d6e6-4752-87e0-22cf72bbe5cf'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:43.333553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3553bd0c-d6e6-4752-87e0-22cf72bbe5cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8745adc4f77c49fb83158f914a1c34bf', 'request_id': 'req-3553bd0c-d6e6-4752-87e0-22cf72bbe5cf'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:43.348493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3553bd0c-d6e6-4752-87e0-22cf72bbe5cf admin admin] The token cache is being invalidate because project 8745adc4f77c49fb83158f914a1c34bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:43.348865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3553bd0c-d6e6-4752-87e0-22cf72bbe5cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:43.349600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3553bd0c-d6e6-4752-87e0-22cf72bbe5cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:43.351332 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4106/8277] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:43 2022] DELETE /identity/v3/projects/8745adc4f77c49fb83158f914a1c34bf => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:43.356343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15522d44-e851-424e-9ca7-07755b5aed33 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:43.356797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15522d44-e851-424e-9ca7-07755b5aed33 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:43.356896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15522d44-e851-424e-9ca7-07755b5aed33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:43.378281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-15522d44-e851-424e-9ca7-07755b5aed33 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:43.379560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-15522d44-e851-424e-9ca7-07755b5aed33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:43.435412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:43.464094 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4172/8278] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:25:43.473090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19971d-a44e-4340-bb2d-302c71c290a6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:43.473382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19971d-a44e-4340-bb2d-302c71c290a6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:43.473663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19971d-a44e-4340-bb2d-302c71c290a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:43.497328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6f19971d-a44e-4340-bb2d-302c71c290a6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:43.498564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6f19971d-a44e-4340-bb2d-302c71c290a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:43.513911 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4107/8279] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:25:43.517951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15522d44-e851-424e-9ca7-07755b5aed33 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:43.533578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15522d44-e851-424e-9ca7-07755b5aed33 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:43.539465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10af41-b167-452e-8e0d-d45972480ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:43.540102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10af41-b167-452e-8e0d-d45972480ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:43.540462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10af41-b167-452e-8e0d-d45972480ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:43.540636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10af41-b167-452e-8e0d-d45972480ae4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:43.542025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10af41-b167-452e-8e0d-d45972480ae4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:25:43.542377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10af41-b167-452e-8e0d-d45972480ae4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:43.544047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10af41-b167-452e-8e0d-d45972480ae4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:43.561856 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4173/8280] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:43.563672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19971d-a44e-4340-bb2d-302c71c290a6 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:43.577958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19971d-a44e-4340-bb2d-302c71c290a6 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:43.584603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8985ba-405e-4888-80f5-9f8f14ec8b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:43.585647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8985ba-405e-4888-80f5-9f8f14ec8b7e admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:43.586055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8985ba-405e-4888-80f5-9f8f14ec8b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:43.586786 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8985ba-405e-4888-80f5-9f8f14ec8b7e admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:43.588119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8985ba-405e-4888-80f5-9f8f14ec8b7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:43.590414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8985ba-405e-4888-80f5-9f8f14ec8b7e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:43.653712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10af41-b167-452e-8e0d-d45972480ae4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:43.654741 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4108/8281] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:25:43 2022] POST /identity/v3/projects => generated 365 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:43.674264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10af41-b167-452e-8e0d-d45972480ae4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:43.779468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d669be1-7aae-4136-9d66-4ad5e5ac68d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:43.780168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d669be1-7aae-4136-9d66-4ad5e5ac68d1 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:43.780524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d669be1-7aae-4136-9d66-4ad5e5ac68d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:43.780911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d669be1-7aae-4136-9d66-4ad5e5ac68d1 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:43.781613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d669be1-7aae-4136-9d66-4ad5e5ac68d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:43.784068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d669be1-7aae-4136-9d66-4ad5e5ac68d1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:43.840693 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4174/8282] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:25:43 2022] POST /identity/v3/users => generated 283 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:43.855225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8985ba-405e-4888-80f5-9f8f14ec8b7e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:43.872625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8985ba-405e-4888-80f5-9f8f14ec8b7e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:43.880615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab575df-7585-4026-93fb-88531a96c934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:43.881498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab575df-7585-4026-93fb-88531a96c934 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:43.881917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab575df-7585-4026-93fb-88531a96c934 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:43.882409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab575df-7585-4026-93fb-88531a96c934 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:43.883331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab575df-7585-4026-93fb-88531a96c934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:43.885323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab575df-7585-4026-93fb-88531a96c934 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:43.909316 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4109/8283] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:25:43.913407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d669be1-7aae-4136-9d66-4ad5e5ac68d1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:43.933668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d669be1-7aae-4136-9d66-4ad5e5ac68d1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:43.940316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5474e32-6cc2-499f-a9d5-dce5d5caacae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:43.941056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5474e32-6cc2-499f-a9d5-dce5d5caacae admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:43.941218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5474e32-6cc2-499f-a9d5-dce5d5caacae admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:43.941500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5474e32-6cc2-499f-a9d5-dce5d5caacae admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:43.942152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5474e32-6cc2-499f-a9d5-dce5d5caacae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:43.951830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5474e32-6cc2-499f-a9d5-dce5d5caacae admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:43.979484 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4175/8284] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:25:43.983921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab575df-7585-4026-93fb-88531a96c934 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:43.997826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab575df-7585-4026-93fb-88531a96c934 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:44.003136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c5587-25ef-42b9-9ecc-facc0852ab00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:44.004123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c5587-25ef-42b9-9ecc-facc0852ab00 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:44.004503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c5587-25ef-42b9-9ecc-facc0852ab00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:44.004882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c5587-25ef-42b9-9ecc-facc0852ab00 admin admin] PATH_INFO: `/v3/projects/4fc5f66263594b21ba1ef9b9ae79db25/users/ced61e0fee084ba5ba286e26f385a04b/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:44.036073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c5587-25ef-42b9-9ecc-facc0852ab00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fc5f66263594b21ba1ef9b9ae79db25, user_id=ced61e0fee084ba5ba286e26f385a04b, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:44.039432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c5587-25ef-42b9-9ecc-facc0852ab00 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:44.043219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:44.062313 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4110/8285] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:25:43 2022] PUT /identity/v3/projects/4fc5f66263594b21ba1ef9b9ae79db25/users/ced61e0fee084ba5ba286e26f385a04b/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:44.072873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661b411-053c-47e8-87d7-8d52a32df344 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:44.074680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661b411-053c-47e8-87d7-8d52a32df344 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:44.075152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661b411-053c-47e8-87d7-8d52a32df344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:44.119595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8661b411-053c-47e8-87d7-8d52a32df344 None None] MFA Rules not processed for user `ced61e0fee084ba5ba286e26f385a04b`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:44.122170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8661b411-053c-47e8-87d7-8d52a32df344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:44.206176 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4176/8286] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:25:44.210569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c5587-25ef-42b9-9ecc-facc0852ab00 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:44.276570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c5587-25ef-42b9-9ecc-facc0852ab00 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:44.281814 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a40564-7941-4a9d-9bdd-fda8ca8c7af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:44.282416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a40564-7941-4a9d-9bdd-fda8ca8c7af0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:44.282739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a40564-7941-4a9d-9bdd-fda8ca8c7af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:44.283222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a40564-7941-4a9d-9bdd-fda8ca8c7af0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:44.283857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a40564-7941-4a9d-9bdd-fda8ca8c7af0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:44.285765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a40564-7941-4a9d-9bdd-fda8ca8c7af0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:44.313504 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4111/8287] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:25:44 2022] POST /identity/v3/projects => generated 361 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:25:44.317609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661b411-053c-47e8-87d7-8d52a32df344 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:44.329593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661b411-053c-47e8-87d7-8d52a32df344 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:44.363049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:44.390465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4c831-fc25-46bb-b08d-b06656e003f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:44.391103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4c831-fc25-46bb-b08d-b06656e003f6 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:44.391385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4c831-fc25-46bb-b08d-b06656e003f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:44.391659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4c831-fc25-46bb-b08d-b06656e003f6 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:44.392309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4c831-fc25-46bb-b08d-b06656e003f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:44.394225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4c831-fc25-46bb-b08d-b06656e003f6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:44.427827 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4177/8288] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:25:44 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:25:44.431885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a40564-7941-4a9d-9bdd-fda8ca8c7af0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:44.442651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a40564-7941-4a9d-9bdd-fda8ca8c7af0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:44.447876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296fa95-443b-4706-925f-d4ce7b4901e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:44.449427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296fa95-443b-4706-925f-d4ce7b4901e5 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:44.449545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296fa95-443b-4706-925f-d4ce7b4901e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:44.449825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296fa95-443b-4706-925f-d4ce7b4901e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:44.450583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296fa95-443b-4706-925f-d4ce7b4901e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:44.452260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296fa95-443b-4706-925f-d4ce7b4901e5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:44.463859 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4112/8289] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:25:44.468698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4c831-fc25-46bb-b08d-b06656e003f6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:44.480699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4c831-fc25-46bb-b08d-b06656e003f6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:44.488049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dab827-4441-4f04-b71f-35b8b1a39cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:44.488670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab827-4441-4f04-b71f-35b8b1a39cc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:44.488961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab827-4441-4f04-b71f-35b8b1a39cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:44.489238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab827-4441-4f04-b71f-35b8b1a39cc0 admin admin] PATH_INFO: `/v3/projects/5b0792b17b1b43e8a5cf92d41acc827d/users/ab7b55af965f49099672be7e3d14dae4/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:44.507109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dab827-4441-4f04-b71f-35b8b1a39cc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b0792b17b1b43e8a5cf92d41acc827d, user_id=ab7b55af965f49099672be7e3d14dae4, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:44.510782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dab827-4441-4f04-b71f-35b8b1a39cc0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:44.526598 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4178/8290] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:25:44 2022] PUT /identity/v3/projects/5b0792b17b1b43e8a5cf92d41acc827d/users/ab7b55af965f49099672be7e3d14dae4/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:44.531103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296fa95-443b-4706-925f-d4ce7b4901e5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:44.541855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296fa95-443b-4706-925f-d4ce7b4901e5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:44.602312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74ded9-cdd0-4645-bec1-32dd8837a4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:44.602987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74ded9-cdd0-4645-bec1-32dd8837a4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:44.603270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74ded9-cdd0-4645-bec1-32dd8837a4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:44.603542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74ded9-cdd0-4645-bec1-32dd8837a4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:44.604175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74ded9-cdd0-4645-bec1-32dd8837a4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:44.605821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74ded9-cdd0-4645-bec1-32dd8837a4d4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:44.618827 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4113/8291] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:25:44.620030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dab827-4441-4f04-b71f-35b8b1a39cc0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:44.630900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dab827-4441-4f04-b71f-35b8b1a39cc0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:44.636636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8003c-4a1a-4ba2-9148-629f6753f116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:44.637422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8003c-4a1a-4ba2-9148-629f6753f116 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:44.637779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8003c-4a1a-4ba2-9148-629f6753f116 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:44.637976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8003c-4a1a-4ba2-9148-629f6753f116 admin admin] PATH_INFO: `/v3/projects/5b0792b17b1b43e8a5cf92d41acc827d/users/ab7b55af965f49099672be7e3d14dae4/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:44.640119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8003c-4a1a-4ba2-9148-629f6753f116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b0792b17b1b43e8a5cf92d41acc827d, user_id=ab7b55af965f49099672be7e3d14dae4, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:44.642806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8003c-4a1a-4ba2-9148-629f6753f116 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:44.654598 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4179/8292] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:25:44 2022] PUT /identity/v3/projects/5b0792b17b1b43e8a5cf92d41acc827d/users/ab7b55af965f49099672be7e3d14dae4/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:44.665645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f3e3a-888a-4f22-b2f7-6a2fd0b0ce66 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:44.665932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f3e3a-888a-4f22-b2f7-6a2fd0b0ce66 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:44.666272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f3e3a-888a-4f22-b2f7-6a2fd0b0ce66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:44.694958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b78f3e3a-888a-4f22-b2f7-6a2fd0b0ce66 None None] MFA Rules not processed for user `ab7b55af965f49099672be7e3d14dae4`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:44.696366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b78f3e3a-888a-4f22-b2f7-6a2fd0b0ce66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:44.752649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:44.773705 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4114/8293] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:44 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:25:44.778648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8003c-4a1a-4ba2-9148-629f6753f116 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:44.790930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8003c-4a1a-4ba2-9148-629f6753f116 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:44.796363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc967bb-7b8f-446c-bb94-e62c6bca0ac5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:44.796974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc967bb-7b8f-446c-bb94-e62c6bca0ac5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:44.797258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc967bb-7b8f-446c-bb94-e62c6bca0ac5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:44.797555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc967bb-7b8f-446c-bb94-e62c6bca0ac5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:44.798231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc967bb-7b8f-446c-bb94-e62c6bca0ac5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:44.799858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc967bb-7b8f-446c-bb94-e62c6bca0ac5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:44.822548 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4180/8294] 10.222.0.17 () {60 vars in 1150 bytes} [Sat Oct 1 21:25:44 2022] POST /identity/v3/domains => generated 276 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:25:44.824681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f3e3a-888a-4f22-b2f7-6a2fd0b0ce66 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:44.834251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f3e3a-888a-4f22-b2f7-6a2fd0b0ce66 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:44.902434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e53ec-ee59-4bf7-8248-db0fe39881cc tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:44.903149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e53ec-ee59-4bf7-8248-db0fe39881cc tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:44.903693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e53ec-ee59-4bf7-8248-db0fe39881cc tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:44.904348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e53ec-ee59-4bf7-8248-db0fe39881cc tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:44.904677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e53ec-ee59-4bf7-8248-db0fe39881cc tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:44.906281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e53ec-ee59-4bf7-8248-db0fe39881cc tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:44.927990 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4115/8295] 10.222.0.17 () {60 vars in 1150 bytes} [Sat Oct 1 21:25:44 2022] POST /identity/v3/domains => generated 277 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:44.931176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc967bb-7b8f-446c-bb94-e62c6bca0ac5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:44.945022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc967bb-7b8f-446c-bb94-e62c6bca0ac5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:45.008205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3878d-d3c2-4e61-909d-37f05278a0a3 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:45.008827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3878d-d3c2-4e61-909d-37f05278a0a3 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:45.009110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3878d-d3c2-4e61-909d-37f05278a0a3 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:45.009383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3878d-d3c2-4e61-909d-37f05278a0a3 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:45.010046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3878d-d3c2-4e61-909d-37f05278a0a3 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:45.011735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3878d-d3c2-4e61-909d-37f05278a0a3 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:45.030839 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4181/8296] 10.222.0.17 () {60 vars in 1150 bytes} [Sat Oct 1 21:25:44 2022] POST /identity/v3/domains => generated 278 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:45.035433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e53ec-ee59-4bf7-8248-db0fe39881cc tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:45.047041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e53ec-ee59-4bf7-8248-db0fe39881cc tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:45.105245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b2b70-5603-4b39-9d8a-37ba320de07a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:45.105851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b2b70-5603-4b39-9d8a-37ba320de07a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:45.106135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b2b70-5603-4b39-9d8a-37ba320de07a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:45.106410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b2b70-5603-4b39-9d8a-37ba320de07a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:45.107094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b2b70-5603-4b39-9d8a-37ba320de07a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:45.108695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b2b70-5603-4b39-9d8a-37ba320de07a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:45.124979 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4116/8297] 10.222.0.17 () {60 vars in 1150 bytes} [Sat Oct 1 21:25:45 2022] POST /identity/v3/domains => generated 278 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:45.129396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3878d-d3c2-4e61-909d-37f05278a0a3 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:45.140926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3878d-d3c2-4e61-909d-37f05278a0a3 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:45.199197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf4be5-cf05-4b62-8735-23a291480788 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:45.199826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf4be5-cf05-4b62-8735-23a291480788 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:45.200107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf4be5-cf05-4b62-8735-23a291480788 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:45.200381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf4be5-cf05-4b62-8735-23a291480788 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains/49e641876f2b4196b513cc29fc3ebbd2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:45.211330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf4be5-cf05-4b62-8735-23a291480788 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=49e641876f2b4196b513cc29fc3ebbd2)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:45.212937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf4be5-cf05-4b62-8735-23a291480788 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:45.215211 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:45.236450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-81cf4be5-cf05-4b62-8735-23a291480788 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '49e641876f2b4196b513cc29fc3ebbd2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.239789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-81cf4be5-cf05-4b62-8735-23a291480788 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '49e641876f2b4196b513cc29fc3ebbd2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.286426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-81cf4be5-cf05-4b62-8735-23a291480788 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49e641876f2b4196b513cc29fc3ebbd2', 'request_id': 'req-81cf4be5-cf05-4b62-8735-23a291480788'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.293770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-81cf4be5-cf05-4b62-8735-23a291480788 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49e641876f2b4196b513cc29fc3ebbd2', 'request_id': 'req-81cf4be5-cf05-4b62-8735-23a291480788'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.299444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-81cf4be5-cf05-4b62-8735-23a291480788 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49e641876f2b4196b513cc29fc3ebbd2', 'request_id': 'req-81cf4be5-cf05-4b62-8735-23a291480788'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.312817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-81cf4be5-cf05-4b62-8735-23a291480788 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] The token cache is being invalidate because project 49e641876f2b4196b513cc29fc3ebbd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:45.313111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-81cf4be5-cf05-4b62-8735-23a291480788 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.313812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-81cf4be5-cf05-4b62-8735-23a291480788 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.321418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-81cf4be5-cf05-4b62-8735-23a291480788 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '49e641876f2b4196b513cc29fc3ebbd2', 'request_id': 'req-81cf4be5-cf05-4b62-8735-23a291480788'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.322087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-81cf4be5-cf05-4b62-8735-23a291480788 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '49e641876f2b4196b513cc29fc3ebbd2', 'request_id': 'req-81cf4be5-cf05-4b62-8735-23a291480788'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.322796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-81cf4be5-cf05-4b62-8735-23a291480788 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '49e641876f2b4196b513cc29fc3ebbd2', 'request_id': 'req-81cf4be5-cf05-4b62-8735-23a291480788'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.331908 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4182/8298] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:25:45 2022] DELETE /identity/v3/domains/49e641876f2b4196b513cc29fc3ebbd2 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:45.336962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b2b70-5603-4b39-9d8a-37ba320de07a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:45.339006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-393b2b70-5603-4b39-9d8a-37ba320de07a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:45.352811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b2b70-5603-4b39-9d8a-37ba320de07a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:45.416384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44174b52-bb88-4366-9bfb-8c4d0c693ddb tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:45.417886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44174b52-bb88-4366-9bfb-8c4d0c693ddb tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:45.418305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44174b52-bb88-4366-9bfb-8c4d0c693ddb tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:45.418647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44174b52-bb88-4366-9bfb-8c4d0c693ddb tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:45.419990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44174b52-bb88-4366-9bfb-8c4d0c693ddb tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:45.421071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44174b52-bb88-4366-9bfb-8c4d0c693ddb tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:45.441391 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4117/8299] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:25:45 2022] POST /identity/v3/domains => generated 247 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:45.442713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf4be5-cf05-4b62-8735-23a291480788 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:45.453760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf4be5-cf05-4b62-8735-23a291480788 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:45.512232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd694d65-f643-4f45-ac3a-6884dabf02bf tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:45.512907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd694d65-f643-4f45-ac3a-6884dabf02bf tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:45.513188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd694d65-f643-4f45-ac3a-6884dabf02bf tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:45.513468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd694d65-f643-4f45-ac3a-6884dabf02bf tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains/676ed9945cb241e994db2df4be942f91` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:45.524973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd694d65-f643-4f45-ac3a-6884dabf02bf tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=676ed9945cb241e994db2df4be942f91)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:45.526709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd694d65-f643-4f45-ac3a-6884dabf02bf tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:45.548446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dd694d65-f643-4f45-ac3a-6884dabf02bf tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '676ed9945cb241e994db2df4be942f91'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.549110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dd694d65-f643-4f45-ac3a-6884dabf02bf tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '676ed9945cb241e994db2df4be942f91'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.574644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dd694d65-f643-4f45-ac3a-6884dabf02bf tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '676ed9945cb241e994db2df4be942f91'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.575281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dd694d65-f643-4f45-ac3a-6884dabf02bf tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '676ed9945cb241e994db2df4be942f91'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.577834 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4183/8300] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:25:45 2022] PATCH /identity/v3/domains/676ed9945cb241e994db2df4be942f91 => generated 248 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:45.581886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44174b52-bb88-4366-9bfb-8c4d0c693ddb tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:45.583608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-44174b52-bb88-4366-9bfb-8c4d0c693ddb tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:45.596676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44174b52-bb88-4366-9bfb-8c4d0c693ddb tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:45.606527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:45.658971 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44728a31-0495-4ef2-9829-b2b3e280309f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:45.659616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44728a31-0495-4ef2-9829-b2b3e280309f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:45.659901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44728a31-0495-4ef2-9829-b2b3e280309f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:45.660271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44728a31-0495-4ef2-9829-b2b3e280309f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains/676ed9945cb241e994db2df4be942f91` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:45.671799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44728a31-0495-4ef2-9829-b2b3e280309f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=676ed9945cb241e994db2df4be942f91)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:45.673414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44728a31-0495-4ef2-9829-b2b3e280309f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:45.695455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-44728a31-0495-4ef2-9829-b2b3e280309f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '676ed9945cb241e994db2df4be942f91'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.705162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-44728a31-0495-4ef2-9829-b2b3e280309f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '676ed9945cb241e994db2df4be942f91'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.747262 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-44728a31-0495-4ef2-9829-b2b3e280309f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '676ed9945cb241e994db2df4be942f91', 'request_id': 'req-44728a31-0495-4ef2-9829-b2b3e280309f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.757751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-44728a31-0495-4ef2-9829-b2b3e280309f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '676ed9945cb241e994db2df4be942f91', 'request_id': 'req-44728a31-0495-4ef2-9829-b2b3e280309f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.764667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-44728a31-0495-4ef2-9829-b2b3e280309f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '676ed9945cb241e994db2df4be942f91', 'request_id': 'req-44728a31-0495-4ef2-9829-b2b3e280309f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.779552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-44728a31-0495-4ef2-9829-b2b3e280309f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] The token cache is being invalidate because project 676ed9945cb241e994db2df4be942f91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:45.779904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-44728a31-0495-4ef2-9829-b2b3e280309f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.780617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-44728a31-0495-4ef2-9829-b2b3e280309f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.789038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-44728a31-0495-4ef2-9829-b2b3e280309f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '676ed9945cb241e994db2df4be942f91', 'request_id': 'req-44728a31-0495-4ef2-9829-b2b3e280309f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.789725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-44728a31-0495-4ef2-9829-b2b3e280309f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '676ed9945cb241e994db2df4be942f91', 'request_id': 'req-44728a31-0495-4ef2-9829-b2b3e280309f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.794047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-44728a31-0495-4ef2-9829-b2b3e280309f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '676ed9945cb241e994db2df4be942f91', 'request_id': 'req-44728a31-0495-4ef2-9829-b2b3e280309f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:45.800058 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4118/8301] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:25:45 2022] DELETE /identity/v3/domains/676ed9945cb241e994db2df4be942f91 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:45.804952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd694d65-f643-4f45-ac3a-6884dabf02bf tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:45.811361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:45.815771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd694d65-f643-4f45-ac3a-6884dabf02bf tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:45.880587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccf8fa-9f96-481f-a35d-47431c247a48 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:45.881199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccf8fa-9f96-481f-a35d-47431c247a48 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:45.881491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccf8fa-9f96-481f-a35d-47431c247a48 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:45.881769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccf8fa-9f96-481f-a35d-47431c247a48 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:45.882429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccf8fa-9f96-481f-a35d-47431c247a48 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:45.884070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccf8fa-9f96-481f-a35d-47431c247a48 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:45.899512 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4184/8302] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:25:45 2022] POST /identity/v3/domains => generated 279 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:25:45.904162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44728a31-0495-4ef2-9829-b2b3e280309f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:45.905572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-44728a31-0495-4ef2-9829-b2b3e280309f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:45.918983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44728a31-0495-4ef2-9829-b2b3e280309f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:45.966377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:45.978108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bae5b7-6ca5-4705-83c1-178673fe20a0 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:45.978795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bae5b7-6ca5-4705-83c1-178673fe20a0 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:45.979081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bae5b7-6ca5-4705-83c1-178673fe20a0 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:45.979359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bae5b7-6ca5-4705-83c1-178673fe20a0 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains/801b74f6b3264365bcf367a7e081bff8` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:45.990011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bae5b7-6ca5-4705-83c1-178673fe20a0 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=801b74f6b3264365bcf367a7e081bff8)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:45.991679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bae5b7-6ca5-4705-83c1-178673fe20a0 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:46.013211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a2bae5b7-6ca5-4705-83c1-178673fe20a0 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '801b74f6b3264365bcf367a7e081bff8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:46.013887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a2bae5b7-6ca5-4705-83c1-178673fe20a0 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '801b74f6b3264365bcf367a7e081bff8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:46.044304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a2bae5b7-6ca5-4705-83c1-178673fe20a0 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '801b74f6b3264365bcf367a7e081bff8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:46.044941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a2bae5b7-6ca5-4705-83c1-178673fe20a0 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '801b74f6b3264365bcf367a7e081bff8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:46.050470 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4119/8303] 10.222.0.17 () {60 vars in 1250 bytes} [Sat Oct 1 21:25:45 2022] PATCH /identity/v3/domains/801b74f6b3264365bcf367a7e081bff8 => generated 278 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:46.051452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccf8fa-9f96-481f-a35d-47431c247a48 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:46.053140 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e7ccf8fa-9f96-481f-a35d-47431c247a48 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:46.065982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccf8fa-9f96-481f-a35d-47431c247a48 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:46.270129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627eea29-3b08-4355-aef8-a78269561b22 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:46.270794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627eea29-3b08-4355-aef8-a78269561b22 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:46.271079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627eea29-3b08-4355-aef8-a78269561b22 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:46.271371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627eea29-3b08-4355-aef8-a78269561b22 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains/801b74f6b3264365bcf367a7e081bff8` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:46.282804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627eea29-3b08-4355-aef8-a78269561b22 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:get_domain(domain_id=801b74f6b3264365bcf367a7e081bff8)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:46.284735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627eea29-3b08-4355-aef8-a78269561b22 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:46.286373 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4185/8304] 10.222.0.17 () {58 vars in 1227 bytes} [Sat Oct 1 21:25:46 2022] GET /identity/v3/domains/801b74f6b3264365bcf367a7e081bff8 => generated 278 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:46.290463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bae5b7-6ca5-4705-83c1-178673fe20a0 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:46.303634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bae5b7-6ca5-4705-83c1-178673fe20a0 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:46.309081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86752d69-9ed3-48b3-8470-a745febfaef4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:46.309710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86752d69-9ed3-48b3-8470-a745febfaef4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:46.309995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86752d69-9ed3-48b3-8470-a745febfaef4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:46.310274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86752d69-9ed3-48b3-8470-a745febfaef4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains/801b74f6b3264365bcf367a7e081bff8` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:46.311468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86752d69-9ed3-48b3-8470-a745febfaef4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=801b74f6b3264365bcf367a7e081bff8)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:46.313083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86752d69-9ed3-48b3-8470-a745febfaef4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:46.336526 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-86752d69-9ed3-48b3-8470-a745febfaef4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '801b74f6b3264365bcf367a7e081bff8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:46.346430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-86752d69-9ed3-48b3-8470-a745febfaef4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '801b74f6b3264365bcf367a7e081bff8'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:46.385761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-86752d69-9ed3-48b3-8470-a745febfaef4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '801b74f6b3264365bcf367a7e081bff8', 'request_id': 'req-86752d69-9ed3-48b3-8470-a745febfaef4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:46.393582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-86752d69-9ed3-48b3-8470-a745febfaef4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '801b74f6b3264365bcf367a7e081bff8', 'request_id': 'req-86752d69-9ed3-48b3-8470-a745febfaef4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:46.399296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-86752d69-9ed3-48b3-8470-a745febfaef4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '801b74f6b3264365bcf367a7e081bff8', 'request_id': 'req-86752d69-9ed3-48b3-8470-a745febfaef4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:46.411991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-86752d69-9ed3-48b3-8470-a745febfaef4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] The token cache is being invalidate because project 801b74f6b3264365bcf367a7e081bff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:46.412308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-86752d69-9ed3-48b3-8470-a745febfaef4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:46.412953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-86752d69-9ed3-48b3-8470-a745febfaef4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:46.420466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-86752d69-9ed3-48b3-8470-a745febfaef4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '801b74f6b3264365bcf367a7e081bff8', 'request_id': 'req-86752d69-9ed3-48b3-8470-a745febfaef4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:46.421100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-86752d69-9ed3-48b3-8470-a745febfaef4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '801b74f6b3264365bcf367a7e081bff8', 'request_id': 'req-86752d69-9ed3-48b3-8470-a745febfaef4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:46.424658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-86752d69-9ed3-48b3-8470-a745febfaef4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '801b74f6b3264365bcf367a7e081bff8', 'request_id': 'req-86752d69-9ed3-48b3-8470-a745febfaef4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:46.430588 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4120/8305] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:25:46 2022] DELETE /identity/v3/domains/801b74f6b3264365bcf367a7e081bff8 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:46.435189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627eea29-3b08-4355-aef8-a78269561b22 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:46.436882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-627eea29-3b08-4355-aef8-a78269561b22 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:46.449972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627eea29-3b08-4355-aef8-a78269561b22 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:46.504487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a13a8-c058-4acd-ae3b-ddcd953f171a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:46.505085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a13a8-c058-4acd-ae3b-ddcd953f171a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:46.505366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a13a8-c058-4acd-ae3b-ddcd953f171a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:46.505642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a13a8-c058-4acd-ae3b-ddcd953f171a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:46.506286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a13a8-c058-4acd-ae3b-ddcd953f171a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:46.507881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a13a8-c058-4acd-ae3b-ddcd953f171a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:46.525880 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4186/8306] 10.222.0.17 () {58 vars in 1128 bytes} [Sat Oct 1 21:25:46 2022] GET /identity/v3/domains => generated 1349 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:25:46.530120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86752d69-9ed3-48b3-8470-a745febfaef4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:46.540252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86752d69-9ed3-48b3-8470-a745febfaef4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:46.545167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc891da-8e6a-411d-a2c0-27d4e44e4e3c tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:46.545790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc891da-8e6a-411d-a2c0-27d4e44e4e3c tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:46.546075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc891da-8e6a-411d-a2c0-27d4e44e4e3c tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:46.546353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc891da-8e6a-411d-a2c0-27d4e44e4e3c tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains/801b74f6b3264365bcf367a7e081bff8` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:46.553263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc891da-8e6a-411d-a2c0-27d4e44e4e3c tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 21:25:46.558777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-4bc891da-8e6a-411d-a2c0-27d4e44e4e3c tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Could not find domain: 801b74f6b3264365bcf367a7e081bff8.: keystone.exception.DomainNotFound: Could not find domain: 801b74f6b3264365bcf367a7e081bff8. Oct 01 21:25:46.558777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:25:46.558777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 01 21:25:46.558777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 01 21:25:46.558777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 21:25:46.558777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 21:25:46.558777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 21:25:46.558777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 21:25:46.558777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 801b74f6b3264365bcf367a7e081bff8. Oct 01 21:25:46.558777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:25:46.558777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:25:46.558777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:25:46.558777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:25:46.558777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:25:46.558777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:25:46.558777 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:25:46.559940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:25:46.559940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:25:46.559940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:25:46.559940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:25:46.559940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:25:46.559940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:25:46.559940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:25:46.559940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 01 21:25:46.559940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 01 21:25:46.559940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 21:25:46.559940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 21:25:46.559940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 21:25:46.559940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 21:25:46.559940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:25:46.559940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:25:46.559940 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:25:46.561054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:25:46.561054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:25:46.561054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:25:46.561054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:25:46.561054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 21:25:46.561054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:25:46.561054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:25:46.561054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:25:46.561054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:25:46.561054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:25:46.561054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:25:46.561054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:25:46.561054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:25:46.561054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 01 21:25:46.561054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 21:25:46.561054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 801b74f6b3264365bcf367a7e081bff8. Oct 01 21:25:46.561054 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:25:46.562234 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4121/8307] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:25:46 2022] PATCH /identity/v3/domains/801b74f6b3264365bcf367a7e081bff8 => generated 112 bytes in 30 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:25:46.565237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a13a8-c058-4acd-ae3b-ddcd953f171a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:46.575941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a13a8-c058-4acd-ae3b-ddcd953f171a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:46.580684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febce62-2a10-44f5-9320-5d815c25b102 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:46.581279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febce62-2a10-44f5-9320-5d815c25b102 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:46.581565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febce62-2a10-44f5-9320-5d815c25b102 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:46.581847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febce62-2a10-44f5-9320-5d815c25b102 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:46.582539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febce62-2a10-44f5-9320-5d815c25b102 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:46.584154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febce62-2a10-44f5-9320-5d815c25b102 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:46.599632 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4187/8308] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:25:46 2022] POST /identity/v3/domains => generated 274 bytes in 35 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:46.603792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc891da-8e6a-411d-a2c0-27d4e44e4e3c tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:46.614291 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc891da-8e6a-411d-a2c0-27d4e44e4e3c tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:46.647567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=87,keystone:UPDATE=1,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:46.673093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ba54a-6558-40cd-819f-a57326c72c48 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:46.673696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ba54a-6558-40cd-819f-a57326c72c48 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:46.673974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ba54a-6558-40cd-819f-a57326c72c48 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:46.674245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ba54a-6558-40cd-819f-a57326c72c48 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:46.674896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ba54a-6558-40cd-819f-a57326c72c48 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:46.676793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ba54a-6558-40cd-819f-a57326c72c48 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:46.707305 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4122/8309] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:25:46 2022] POST /identity/v3/users => generated 344 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:46.711576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febce62-2a10-44f5-9320-5d815c25b102 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:46.721685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febce62-2a10-44f5-9320-5d815c25b102 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:46.726395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1500461-3df2-447b-9ff6-8fc77e780beb tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:46.727003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1500461-3df2-447b-9ff6-8fc77e780beb tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:46.727286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1500461-3df2-447b-9ff6-8fc77e780beb tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:46.727560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1500461-3df2-447b-9ff6-8fc77e780beb tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/groups` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:46.728176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1500461-3df2-447b-9ff6-8fc77e780beb tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:46.730667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1500461-3df2-447b-9ff6-8fc77e780beb tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:46.740590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:46.746812 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4188/8310] 10.222.0.17 () {60 vars in 1147 bytes} [Sat Oct 1 21:25:46 2022] POST /identity/v3/groups => generated 324 bytes in 36 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:46.750747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ba54a-6558-40cd-819f-a57326c72c48 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:46.763282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ba54a-6558-40cd-819f-a57326c72c48 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:46.770179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ea29a-85fc-4867-8566-53d4b60b6057 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:46.770841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ea29a-85fc-4867-8566-53d4b60b6057 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:46.771123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ea29a-85fc-4867-8566-53d4b60b6057 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:46.771398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ea29a-85fc-4867-8566-53d4b60b6057 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains/70c0a3e53e704eb1bedc43491074c722` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:46.772653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ea29a-85fc-4867-8566-53d4b60b6057 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=70c0a3e53e704eb1bedc43491074c722)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:46.774251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ea29a-85fc-4867-8566-53d4b60b6057 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:46.796118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-849ea29a-85fc-4867-8566-53d4b60b6057 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '70c0a3e53e704eb1bedc43491074c722'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:46.796822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-849ea29a-85fc-4867-8566-53d4b60b6057 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '70c0a3e53e704eb1bedc43491074c722'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:46.821637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-849ea29a-85fc-4867-8566-53d4b60b6057 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '70c0a3e53e704eb1bedc43491074c722'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:46.822563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-849ea29a-85fc-4867-8566-53d4b60b6057 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '70c0a3e53e704eb1bedc43491074c722'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:46.825238 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4123/8311] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:25:46 2022] PATCH /identity/v3/domains/70c0a3e53e704eb1bedc43491074c722 => generated 275 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:46.829978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1500461-3df2-447b-9ff6-8fc77e780beb tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:46.831928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b1500461-3df2-447b-9ff6-8fc77e780beb tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:46.844503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1500461-3df2-447b-9ff6-8fc77e780beb tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:46.901927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:46.902598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:46.902883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:46.903160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains/70c0a3e53e704eb1bedc43491074c722` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:46.913777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=70c0a3e53e704eb1bedc43491074c722)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:46.915392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:46.938538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '70c0a3e53e704eb1bedc43491074c722'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:46.941861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '70c0a3e53e704eb1bedc43491074c722'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.069547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69f0483b15a647339c3e13c4e029f44e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.080488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69f0483b15a647339c3e13c4e029f44e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.085763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=12 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:47.094130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69f0483b15a647339c3e13c4e029f44e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.094905 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69f0483b15a647339c3e13c4e029f44e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.102031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69f0483b15a647339c3e13c4e029f44e'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.144852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70c0a3e53e704eb1bedc43491074c722', 'request_id': 'req-5d6a61c7-57c4-4f14-bf8e-35cf74337816'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.151270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70c0a3e53e704eb1bedc43491074c722', 'request_id': 'req-5d6a61c7-57c4-4f14-bf8e-35cf74337816'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.157021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70c0a3e53e704eb1bedc43491074c722', 'request_id': 'req-5d6a61c7-57c4-4f14-bf8e-35cf74337816'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.169719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] The token cache is being invalidate because project 70c0a3e53e704eb1bedc43491074c722 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:47.170038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.170718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.177934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '70c0a3e53e704eb1bedc43491074c722', 'request_id': 'req-5d6a61c7-57c4-4f14-bf8e-35cf74337816'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.178600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '70c0a3e53e704eb1bedc43491074c722', 'request_id': 'req-5d6a61c7-57c4-4f14-bf8e-35cf74337816'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.179219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '70c0a3e53e704eb1bedc43491074c722', 'request_id': 'req-5d6a61c7-57c4-4f14-bf8e-35cf74337816'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.188031 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4189/8312] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:25:46 2022] DELETE /identity/v3/domains/70c0a3e53e704eb1bedc43491074c722 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:47.192359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ea29a-85fc-4867-8566-53d4b60b6057 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:47.203296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ea29a-85fc-4867-8566-53d4b60b6057 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:47.263491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946b1b3-b434-460d-9dc7-515aa2834e9e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:47.264136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946b1b3-b434-460d-9dc7-515aa2834e9e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:47.264424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946b1b3-b434-460d-9dc7-515aa2834e9e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:47.264707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946b1b3-b434-460d-9dc7-515aa2834e9e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains/70c0a3e53e704eb1bedc43491074c722` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:47.272391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946b1b3-b434-460d-9dc7-515aa2834e9e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:get_domain(domain_id=70c0a3e53e704eb1bedc43491074c722)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:47.274553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946b1b3-b434-460d-9dc7-515aa2834e9e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:47.286828 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-d946b1b3-b434-460d-9dc7-515aa2834e9e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Could not find domain: 70c0a3e53e704eb1bedc43491074c722.: keystone.exception.DomainNotFound: Could not find domain: 70c0a3e53e704eb1bedc43491074c722. Oct 01 21:25:47.286828 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:25:47.286828 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 01 21:25:47.286828 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 01 21:25:47.286828 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 21:25:47.286828 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 21:25:47.286828 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 21:25:47.286828 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 21:25:47.286828 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 70c0a3e53e704eb1bedc43491074c722. Oct 01 21:25:47.286828 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:25:47.286828 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:25:47.286828 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:25:47.286828 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:25:47.286828 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:25:47.286828 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:25:47.286828 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:25:47.288003 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:25:47.288003 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:25:47.288003 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:25:47.288003 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:25:47.288003 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:25:47.288003 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:25:47.288003 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:25:47.288003 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 01 21:25:47.288003 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 01 21:25:47.288003 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 01 21:25:47.288003 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 01 21:25:47.288003 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:25:47.288003 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:25:47.288003 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:25:47.288003 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:25:47.288003 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:25:47.288003 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:25:47.289072 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:25:47.289072 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 21:25:47.289072 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:25:47.289072 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:25:47.289072 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:25:47.289072 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:25:47.289072 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:25:47.289072 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:25:47.289072 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:25:47.289072 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:25:47.289072 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 01 21:25:47.289072 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 21:25:47.289072 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 70c0a3e53e704eb1bedc43491074c722. Oct 01 21:25:47.289072 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:25:47.289072 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4124/8313] 10.222.0.17 () {58 vars in 1227 bytes} [Sat Oct 1 21:25:47 2022] GET /identity/v3/domains/70c0a3e53e704eb1bedc43491074c722 => generated 112 bytes in 96 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:25:47.291980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:47.293596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:47.306439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a61c7-57c4-4f14-bf8e-35cf74337816 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:47.313330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be02f0-aa82-4d8e-b4ef-510adba7a20f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:47.313901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be02f0-aa82-4d8e-b4ef-510adba7a20f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:47.314256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be02f0-aa82-4d8e-b4ef-510adba7a20f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:47.314466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be02f0-aa82-4d8e-b4ef-510adba7a20f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/users/69f0483b15a647339c3e13c4e029f44e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:47.324657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be02f0-aa82-4d8e-b4ef-510adba7a20f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:get_user(user_id=69f0483b15a647339c3e13c4e029f44e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:47.326731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be02f0-aa82-4d8e-b4ef-510adba7a20f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:47.338518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-43be02f0-aa82-4d8e-b4ef-510adba7a20f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Could not find user: 69f0483b15a647339c3e13c4e029f44e.: keystone.exception.UserNotFound: Could not find user: 69f0483b15a647339c3e13c4e029f44e. Oct 01 21:25:47.338518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:25:47.338518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:25:47.338518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:25:47.338518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:25:47.338518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:25:47.338518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:25:47.338518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:25:47.338518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:25:47.338518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:25:47.338518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:25:47.338518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:25:47.338518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 01 21:25:47.338518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 01 21:25:47.338518 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 01 21:25:47.339886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 01 21:25:47.339886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:25:47.339886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:25:47.339886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 21:25:47.339886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 21:25:47.339886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 21:25:47.339886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 21:25:47.339886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:25:47.339886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:25:47.339886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:25:47.339886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:25:47.339886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:25:47.339886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 21:25:47.339886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:25:47.339886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:25:47.339886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:25:47.339886 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:25:47.340974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:25:47.340974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:25:47.340974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:25:47.340974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:25:47.340974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 21:25:47.340974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 21:25:47.340974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 21:25:47.340974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 21:25:47.340974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 21:25:47.340974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 21:25:47.340974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: 69f0483b15a647339c3e13c4e029f44e. Oct 01 21:25:47.340974 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:25:47.340974 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4190/8314] 10.222.0.17 () {58 vars in 1221 bytes} [Sat Oct 1 21:25:47 2022] GET /identity/v3/users/69f0483b15a647339c3e13c4e029f44e => generated 110 bytes in 49 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Oct 01 21:25:47.343866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946b1b3-b434-460d-9dc7-515aa2834e9e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:47.354132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946b1b3-b434-460d-9dc7-515aa2834e9e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:47.359051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608b288-2744-40a3-a6d7-703535a8780a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:47.359693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608b288-2744-40a3-a6d7-703535a8780a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:47.359977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608b288-2744-40a3-a6d7-703535a8780a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:47.360254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608b288-2744-40a3-a6d7-703535a8780a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/groups/9ee93ee83edd4f8d94cb6638e5df219e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:47.368237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608b288-2744-40a3-a6d7-703535a8780a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:get_group(group_id=9ee93ee83edd4f8d94cb6638e5df219e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:47.370113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608b288-2744-40a3-a6d7-703535a8780a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:47.387837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-6608b288-2744-40a3-a6d7-703535a8780a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Could not find group: 9ee93ee83edd4f8d94cb6638e5df219e.: keystone.exception.GroupNotFound: Could not find group: 9ee93ee83edd4f8d94cb6638e5df219e. Oct 01 21:25:47.387837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:25:47.387837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:25:47.387837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:25:47.387837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:25:47.387837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:25:47.387837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:25:47.387837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:25:47.387837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:25:47.387837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:25:47.387837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:25:47.387837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:25:47.387837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 58, in get Oct 01 21:25:47.387837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_group(group_id) Oct 01 21:25:47.387837 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 70, in _get_group Oct 01 21:25:47.388825 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.identity_api.get_group(group_id)) Oct 01 21:25:47.388825 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:25:47.388825 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:25:47.388825 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 21:25:47.388825 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 21:25:47.388825 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 21:25:47.388825 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 21:25:47.388825 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:25:47.388825 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:25:47.388825 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:25:47.388825 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:25:47.388825 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:25:47.388825 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 21:25:47.388825 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:25:47.388825 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:25:47.388825 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:25:47.388825 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:25:47.389730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:25:47.389730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:25:47.389730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:25:47.389730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:25:47.389730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1262, in get_group Oct 01 21:25:47.389730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_group(entity_id) Oct 01 21:25:47.389730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 417, in get_group Oct 01 21:25:47.389730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_group(session, group_id).to_dict() Oct 01 21:25:47.389730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 412, in _get_group Oct 01 21:25:47.389730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.GroupNotFound(group_id=group_id) Oct 01 21:25:47.389730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.GroupNotFound: Could not find group: 9ee93ee83edd4f8d94cb6638e5df219e. Oct 01 21:25:47.389730 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:25:47.389730 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4125/8315] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:25:47 2022] GET /identity/v3/groups/9ee93ee83edd4f8d94cb6638e5df219e => generated 111 bytes in 46 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:25:47.393444 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be02f0-aa82-4d8e-b4ef-510adba7a20f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:47.404353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be02f0-aa82-4d8e-b4ef-510adba7a20f tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:47.409336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c109e-20f8-4686-a772-fef645b58e42 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:47.409972 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c109e-20f8-4686-a772-fef645b58e42 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:47.410256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c109e-20f8-4686-a772-fef645b58e42 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:47.410614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c109e-20f8-4686-a772-fef645b58e42 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/groups/9ee93ee83edd4f8d94cb6638e5df219e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:47.416709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c109e-20f8-4686-a772-fef645b58e42 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 21:25:47.422413 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-493c109e-20f8-4686-a772-fef645b58e42 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Could not find group: 9ee93ee83edd4f8d94cb6638e5df219e.: keystone.exception.GroupNotFound: Could not find group: 9ee93ee83edd4f8d94cb6638e5df219e. Oct 01 21:25:47.422413 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:25:47.422413 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:25:47.422413 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:25:47.422413 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:25:47.422413 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:25:47.422413 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:25:47.422413 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:25:47.422413 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:25:47.422413 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:25:47.422413 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:25:47.422413 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:25:47.422413 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 133, in delete Oct 01 21:25:47.422413 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_group') Oct 01 21:25:47.423343 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 21:25:47.423343 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 21:25:47.423343 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 21:25:47.423343 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 21:25:47.423343 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:25:47.423343 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:25:47.423343 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 21:25:47.423343 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 21:25:47.423343 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 21:25:47.423343 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 21:25:47.423343 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:25:47.423343 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:25:47.423343 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:25:47.423343 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:25:47.423343 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:25:47.423343 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 21:25:47.423343 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:25:47.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:25:47.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:25:47.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:25:47.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:25:47.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:25:47.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:25:47.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:25:47.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1262, in get_group Oct 01 21:25:47.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ref = driver.get_group(entity_id) Oct 01 21:25:47.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 417, in get_group Oct 01 21:25:47.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_group(session, group_id).to_dict() Oct 01 21:25:47.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 412, in _get_group Oct 01 21:25:47.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.GroupNotFound(group_id=group_id) Oct 01 21:25:47.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.GroupNotFound: Could not find group: 9ee93ee83edd4f8d94cb6638e5df219e. Oct 01 21:25:47.424215 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:25:47.427375 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4191/8316] 10.222.0.17 () {58 vars in 1227 bytes} [Sat Oct 1 21:25:47 2022] DELETE /identity/v3/groups/9ee93ee83edd4f8d94cb6638e5df219e => generated 111 bytes in 34 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:25:47.428795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608b288-2744-40a3-a6d7-703535a8780a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:47.441463 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608b288-2744-40a3-a6d7-703535a8780a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:47.451021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefa177-992f-4db0-8e98-317c886d0cc5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:47.451740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefa177-992f-4db0-8e98-317c886d0cc5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:47.452105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefa177-992f-4db0-8e98-317c886d0cc5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:47.452811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefa177-992f-4db0-8e98-317c886d0cc5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/users/69f0483b15a647339c3e13c4e029f44e` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:47.460359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffefa177-992f-4db0-8e98-317c886d0cc5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:delete_user(user_id=69f0483b15a647339c3e13c4e029f44e)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:47.462252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffefa177-992f-4db0-8e98-317c886d0cc5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:47.475062 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-ffefa177-992f-4db0-8e98-317c886d0cc5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Could not find user: 69f0483b15a647339c3e13c4e029f44e.: keystone.exception.UserNotFound: Could not find user: 69f0483b15a647339c3e13c4e029f44e. Oct 01 21:25:47.475062 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:25:47.475062 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:25:47.475062 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:25:47.475062 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:25:47.475062 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:25:47.475062 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:25:47.475062 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:25:47.475062 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:25:47.475062 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:25:47.475062 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:25:47.475062 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:25:47.475062 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 275, in delete Oct 01 21:25:47.475062 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application PROVIDERS.identity_api.delete_user(user_id) Oct 01 21:25:47.477193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:25:47.477193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:25:47.477193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 21:25:47.477193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 21:25:47.477193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 21:25:47.477193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 21:25:47.477193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1207, in delete_user Oct 01 21:25:47.477193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application user_old = self.get_user(user_id) Oct 01 21:25:47.477193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:25:47.477193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:25:47.477193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 01 21:25:47.477193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 21:25:47.477193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 01 21:25:47.477193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 01 21:25:47.477193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:25:47.477193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:25:47.477193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:25:47.477193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:25:47.478339 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:25:47.478339 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 21:25:47.478339 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:25:47.478339 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:25:47.478339 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:25:47.478339 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:25:47.478339 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:25:47.478339 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:25:47.478339 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:25:47.478339 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:25:47.478339 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 01 21:25:47.478339 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 01 21:25:47.478339 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 01 21:25:47.478339 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 01 21:25:47.478339 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 01 21:25:47.478339 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 01 21:25:47.478339 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: 69f0483b15a647339c3e13c4e029f44e. Oct 01 21:25:47.478339 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:25:47.480438 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4126/8317] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:25:47 2022] DELETE /identity/v3/users/69f0483b15a647339c3e13c4e029f44e => generated 110 bytes in 53 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:25:47.484484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c109e-20f8-4686-a772-fef645b58e42 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:47.494755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c109e-20f8-4686-a772-fef645b58e42 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:47.499883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f3e47-df20-4e7a-abb3-8947d02ed8c4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:47.500495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f3e47-df20-4e7a-abb3-8947d02ed8c4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:47.500779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f3e47-df20-4e7a-abb3-8947d02ed8c4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:47.501055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f3e47-df20-4e7a-abb3-8947d02ed8c4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains/70c0a3e53e704eb1bedc43491074c722` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:47.508517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f3e47-df20-4e7a-abb3-8947d02ed8c4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 21:25:47.513368 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-de5f3e47-df20-4e7a-abb3-8947d02ed8c4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Could not find domain: 70c0a3e53e704eb1bedc43491074c722.: keystone.exception.DomainNotFound: Could not find domain: 70c0a3e53e704eb1bedc43491074c722. Oct 01 21:25:47.513368 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:25:47.513368 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 01 21:25:47.513368 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 01 21:25:47.513368 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 21:25:47.513368 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 21:25:47.513368 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 21:25:47.513368 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 21:25:47.513368 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 70c0a3e53e704eb1bedc43491074c722. Oct 01 21:25:47.513368 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:25:47.513368 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:25:47.513368 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:25:47.513368 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:25:47.513368 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:25:47.513368 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:25:47.513368 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:25:47.514356 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:25:47.514356 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:25:47.514356 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:25:47.514356 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:25:47.514356 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:25:47.514356 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:25:47.514356 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:25:47.514356 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 01 21:25:47.514356 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 01 21:25:47.514356 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 21:25:47.514356 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 21:25:47.514356 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 21:25:47.514356 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 21:25:47.514356 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:25:47.514356 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:25:47.514356 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:25:47.515310 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:25:47.515310 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:25:47.515310 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:25:47.515310 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:25:47.515310 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 21:25:47.515310 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:25:47.515310 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:25:47.515310 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:25:47.515310 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:25:47.515310 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:25:47.515310 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:25:47.515310 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:25:47.515310 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:25:47.515310 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 01 21:25:47.515310 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 21:25:47.515310 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 70c0a3e53e704eb1bedc43491074c722. Oct 01 21:25:47.515310 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:25:47.516504 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4192/8318] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:25:47 2022] PATCH /identity/v3/domains/70c0a3e53e704eb1bedc43491074c722 => generated 112 bytes in 30 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Oct 01 21:25:47.519339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefa177-992f-4db0-8e98-317c886d0cc5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:47.529350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefa177-992f-4db0-8e98-317c886d0cc5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:47.533982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0014dc-5f09-4e45-8ea0-b254e38569f5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:47.534612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0014dc-5f09-4e45-8ea0-b254e38569f5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:47.534901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0014dc-5f09-4e45-8ea0-b254e38569f5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:47.535187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0014dc-5f09-4e45-8ea0-b254e38569f5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:47.535801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0014dc-5f09-4e45-8ea0-b254e38569f5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:47.537387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0014dc-5f09-4e45-8ea0-b254e38569f5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:47.556017 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4127/8319] 10.222.0.17 () {58 vars in 1128 bytes} [Sat Oct 1 21:25:47 2022] GET /identity/v3/domains => generated 1349 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:25:47.561349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f3e47-df20-4e7a-abb3-8947d02ed8c4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:47.571435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f3e47-df20-4e7a-abb3-8947d02ed8c4 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:47.576026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb219fc-6045-44b5-beb3-962867e79729 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:47.576613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb219fc-6045-44b5-beb3-962867e79729 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:47.576895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb219fc-6045-44b5-beb3-962867e79729 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:47.577167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb219fc-6045-44b5-beb3-962867e79729 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:47.577802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb219fc-6045-44b5-beb3-962867e79729 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Adding query filter params (enabled=True) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:25:47.578103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb219fc-6045-44b5-beb3-962867e79729 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:47.579727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb219fc-6045-44b5-beb3-962867e79729 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:47.597816 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4193/8320] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:25:47 2022] GET /identity/v3/domains?enabled=True => generated 1095 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:25:47.602655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0014dc-5f09-4e45-8ea0-b254e38569f5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:47.612989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0014dc-5f09-4e45-8ea0-b254e38569f5 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:47.617532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e5908-903e-4db8-b590-fd094940a920 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:47.618116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e5908-903e-4db8-b590-fd094940a920 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:47.618396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e5908-903e-4db8-b590-fd094940a920 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:47.618710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e5908-903e-4db8-b590-fd094940a920 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:47.619341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e5908-903e-4db8-b590-fd094940a920 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Adding query filter params (name=tempest-test_domain-409086401) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:25:47.619645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e5908-903e-4db8-b590-fd094940a920 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:47.621832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e5908-903e-4db8-b590-fd094940a920 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:47.633362 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4128/8321] 10.222.0.17 () {58 vars in 1197 bytes} [Sat Oct 1 21:25:47 2022] GET /identity/v3/domains?name=tempest-test_domain-409086401 => generated 407 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:47.637860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb219fc-6045-44b5-beb3-962867e79729 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:47.647754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb219fc-6045-44b5-beb3-962867e79729 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:47.652343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108d463-50ca-421a-a679-62d38ad89801 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:47.652952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108d463-50ca-421a-a679-62d38ad89801 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:47.653238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108d463-50ca-421a-a679-62d38ad89801 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:47.653518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108d463-50ca-421a-a679-62d38ad89801 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains/70c0a3e53e704eb1bedc43491074c722` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:47.660377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108d463-50ca-421a-a679-62d38ad89801 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 21:25:47.665254 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-6108d463-50ca-421a-a679-62d38ad89801 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Could not find domain: 70c0a3e53e704eb1bedc43491074c722.: keystone.exception.DomainNotFound: Could not find domain: 70c0a3e53e704eb1bedc43491074c722. Oct 01 21:25:47.665254 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:25:47.665254 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 01 21:25:47.665254 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 01 21:25:47.665254 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 21:25:47.665254 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 21:25:47.665254 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 21:25:47.665254 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 21:25:47.665254 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 70c0a3e53e704eb1bedc43491074c722. Oct 01 21:25:47.665254 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:25:47.665254 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:25:47.665254 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:25:47.665254 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:25:47.665254 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:25:47.665254 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:25:47.665254 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:25:47.666232 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:25:47.666232 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:25:47.666232 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:25:47.666232 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:25:47.666232 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:25:47.666232 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:25:47.666232 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:25:47.666232 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 01 21:25:47.666232 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 01 21:25:47.666232 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 21:25:47.666232 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 21:25:47.666232 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 21:25:47.666232 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 21:25:47.666232 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:25:47.666232 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:25:47.666232 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:25:47.667859 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:25:47.667859 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:25:47.667859 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:25:47.667859 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:25:47.667859 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 21:25:47.667859 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:25:47.667859 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:25:47.667859 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:25:47.667859 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:25:47.667859 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:25:47.667859 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:25:47.667859 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:25:47.667859 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:25:47.667859 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 01 21:25:47.667859 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 21:25:47.667859 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 70c0a3e53e704eb1bedc43491074c722. Oct 01 21:25:47.667859 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:25:47.668741 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4194/8322] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:25:47 2022] PATCH /identity/v3/domains/70c0a3e53e704eb1bedc43491074c722 => generated 112 bytes in 29 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Oct 01 21:25:47.670755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e5908-903e-4db8-b590-fd094940a920 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:47.680736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e5908-903e-4db8-b590-fd094940a920 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:47.686213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6789c-6d00-4ca2-a34f-6fa2b4bad091 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:47.686860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6789c-6d00-4ca2-a34f-6fa2b4bad091 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:47.687143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6789c-6d00-4ca2-a34f-6fa2b4bad091 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:47.687418 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6789c-6d00-4ca2-a34f-6fa2b4bad091 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains/773e800cea204351b6fb8bd5a842a67d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:47.699114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf6789c-6d00-4ca2-a34f-6fa2b4bad091 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=773e800cea204351b6fb8bd5a842a67d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:47.700727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf6789c-6d00-4ca2-a34f-6fa2b4bad091 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:47.704861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:UPDATE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:47.744834 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4129/8323] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:25:47 2022] PATCH /identity/v3/domains/773e800cea204351b6fb8bd5a842a67d => generated 278 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:47.749341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108d463-50ca-421a-a679-62d38ad89801 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:47.762131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108d463-50ca-421a-a679-62d38ad89801 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:47.766954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:47.767601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:47.767883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:47.768161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains/773e800cea204351b6fb8bd5a842a67d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:47.779404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=773e800cea204351b6fb8bd5a842a67d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:47.781010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:47.803560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '773e800cea204351b6fb8bd5a842a67d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.806803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '773e800cea204351b6fb8bd5a842a67d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.836723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:47.855416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '773e800cea204351b6fb8bd5a842a67d', 'request_id': 'req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.861828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '773e800cea204351b6fb8bd5a842a67d', 'request_id': 'req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.867477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '773e800cea204351b6fb8bd5a842a67d', 'request_id': 'req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.879492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] The token cache is being invalidate because project 773e800cea204351b6fb8bd5a842a67d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:47.879783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.880436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.888094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '773e800cea204351b6fb8bd5a842a67d', 'request_id': 'req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.888747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '773e800cea204351b6fb8bd5a842a67d', 'request_id': 'req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.889403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '773e800cea204351b6fb8bd5a842a67d', 'request_id': 'req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:47.897753 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4195/8324] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:25:47 2022] DELETE /identity/v3/domains/773e800cea204351b6fb8bd5a842a67d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:47.902284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6789c-6d00-4ca2-a34f-6fa2b4bad091 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:47.904066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cbf6789c-6d00-4ca2-a34f-6fa2b4bad091 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:47.916889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6789c-6d00-4ca2-a34f-6fa2b4bad091 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:47.972286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d33c63-0a87-4b44-bdae-3123e0de332e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:47.972918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d33c63-0a87-4b44-bdae-3123e0de332e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:47.973205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d33c63-0a87-4b44-bdae-3123e0de332e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:47.973483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d33c63-0a87-4b44-bdae-3123e0de332e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains/8bf1b630251946b28ce5356a0cfad222` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:47.984708 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d33c63-0a87-4b44-bdae-3123e0de332e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=8bf1b630251946b28ce5356a0cfad222)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:47.986434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d33c63-0a87-4b44-bdae-3123e0de332e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:48.008113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-90d33c63-0a87-4b44-bdae-3123e0de332e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '8bf1b630251946b28ce5356a0cfad222'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.008789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-90d33c63-0a87-4b44-bdae-3123e0de332e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '8bf1b630251946b28ce5356a0cfad222'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.033847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-90d33c63-0a87-4b44-bdae-3123e0de332e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '8bf1b630251946b28ce5356a0cfad222'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.034483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-90d33c63-0a87-4b44-bdae-3123e0de332e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '8bf1b630251946b28ce5356a0cfad222'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.036910 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4130/8325] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:25:47 2022] PATCH /identity/v3/domains/8bf1b630251946b28ce5356a0cfad222 => generated 278 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:48.040990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:48.051824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9aae1-4f8d-411f-81f3-a67c6aa8bb16 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:48.109872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:48.110533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:48.110819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:48.111124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains/8bf1b630251946b28ce5356a0cfad222` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:48.121835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=8bf1b630251946b28ce5356a0cfad222)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:48.123453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:48.146218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '8bf1b630251946b28ce5356a0cfad222'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.149582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '8bf1b630251946b28ce5356a0cfad222'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.195346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bf1b630251946b28ce5356a0cfad222', 'request_id': 'req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.202559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bf1b630251946b28ce5356a0cfad222', 'request_id': 'req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.208087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bf1b630251946b28ce5356a0cfad222', 'request_id': 'req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.209700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=16,keystone:SELECT=83,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:48.223028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] The token cache is being invalidate because project 8bf1b630251946b28ce5356a0cfad222 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:48.223315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.223945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.231824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '8bf1b630251946b28ce5356a0cfad222', 'request_id': 'req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.232469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '8bf1b630251946b28ce5356a0cfad222', 'request_id': 'req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.233086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '8bf1b630251946b28ce5356a0cfad222', 'request_id': 'req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.241062 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4196/8326] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:25:48 2022] DELETE /identity/v3/domains/8bf1b630251946b28ce5356a0cfad222 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:48.245024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d33c63-0a87-4b44-bdae-3123e0de332e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:48.246620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-90d33c63-0a87-4b44-bdae-3123e0de332e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:48.258965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d33c63-0a87-4b44-bdae-3123e0de332e tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:48.314502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710b742-95f4-43c4-8e88-e0a2e5a11d5a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:48.315145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710b742-95f4-43c4-8e88-e0a2e5a11d5a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:48.315431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710b742-95f4-43c4-8e88-e0a2e5a11d5a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:48.315709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710b742-95f4-43c4-8e88-e0a2e5a11d5a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains/8a6e61addf2e4bb49baba9ed7c10fcf0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:48.326648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710b742-95f4-43c4-8e88-e0a2e5a11d5a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=8a6e61addf2e4bb49baba9ed7c10fcf0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:48.328240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710b742-95f4-43c4-8e88-e0a2e5a11d5a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:48.349189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3710b742-95f4-43c4-8e88-e0a2e5a11d5a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '8a6e61addf2e4bb49baba9ed7c10fcf0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.349772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3710b742-95f4-43c4-8e88-e0a2e5a11d5a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '8a6e61addf2e4bb49baba9ed7c10fcf0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.351683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:48.380267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3710b742-95f4-43c4-8e88-e0a2e5a11d5a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '8a6e61addf2e4bb49baba9ed7c10fcf0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.381242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3710b742-95f4-43c4-8e88-e0a2e5a11d5a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '8a6e61addf2e4bb49baba9ed7c10fcf0'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.387252 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4131/8327] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:25:48 2022] PATCH /identity/v3/domains/8a6e61addf2e4bb49baba9ed7c10fcf0 => generated 277 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:48.388892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:48.390311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:48.406074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe51cfb-9167-47dd-bbaa-d125b0e4bd57 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:48.465779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388c607-f83e-4e9a-a25c-c3625411f853 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab7b55af965f49099672be7e3d14dae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b0792b17b1b43e8a5cf92d41acc827d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:48.466415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388c607-f83e-4e9a-a25c-c3625411f853 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:48.466822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388c607-f83e-4e9a-a25c-c3625411f853 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:48.467112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388c607-f83e-4e9a-a25c-c3625411f853 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] PATH_INFO: `/v3/domains/8a6e61addf2e4bb49baba9ed7c10fcf0` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:48.478179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388c607-f83e-4e9a-a25c-c3625411f853 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=8a6e61addf2e4bb49baba9ed7c10fcf0)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:48.479835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388c607-f83e-4e9a-a25c-c3625411f853 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:48.502371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2388c607-f83e-4e9a-a25c-c3625411f853 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '8a6e61addf2e4bb49baba9ed7c10fcf0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.505847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2388c607-f83e-4e9a-a25c-c3625411f853 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '8a6e61addf2e4bb49baba9ed7c10fcf0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.553974 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2388c607-f83e-4e9a-a25c-c3625411f853 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a6e61addf2e4bb49baba9ed7c10fcf0', 'request_id': 'req-2388c607-f83e-4e9a-a25c-c3625411f853'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.560875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2388c607-f83e-4e9a-a25c-c3625411f853 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a6e61addf2e4bb49baba9ed7c10fcf0', 'request_id': 'req-2388c607-f83e-4e9a-a25c-c3625411f853'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.566413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2388c607-f83e-4e9a-a25c-c3625411f853 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a6e61addf2e4bb49baba9ed7c10fcf0', 'request_id': 'req-2388c607-f83e-4e9a-a25c-c3625411f853'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.581399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2388c607-f83e-4e9a-a25c-c3625411f853 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] The token cache is being invalidate because project 8a6e61addf2e4bb49baba9ed7c10fcf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:48.581841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2388c607-f83e-4e9a-a25c-c3625411f853 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.582663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2388c607-f83e-4e9a-a25c-c3625411f853 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.590205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2388c607-f83e-4e9a-a25c-c3625411f853 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '8a6e61addf2e4bb49baba9ed7c10fcf0', 'request_id': 'req-2388c607-f83e-4e9a-a25c-c3625411f853'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.590898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2388c607-f83e-4e9a-a25c-c3625411f853 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '8a6e61addf2e4bb49baba9ed7c10fcf0', 'request_id': 'req-2388c607-f83e-4e9a-a25c-c3625411f853'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.591515 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2388c607-f83e-4e9a-a25c-c3625411f853 tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '8a6e61addf2e4bb49baba9ed7c10fcf0', 'request_id': 'req-2388c607-f83e-4e9a-a25c-c3625411f853'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.599878 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4197/8328] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:25:48 2022] DELETE /identity/v3/domains/8a6e61addf2e4bb49baba9ed7c10fcf0 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:48.603796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710b742-95f4-43c4-8e88-e0a2e5a11d5a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:48.605298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3710b742-95f4-43c4-8e88-e0a2e5a11d5a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:48.673477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710b742-95f4-43c4-8e88-e0a2e5a11d5a tempest-DomainsTestJSON-4967138 tempest-DomainsTestJSON-4967138-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:48.728921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdcaba0-d231-420f-9d19-4a534db4b8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:48.729550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdcaba0-d231-420f-9d19-4a534db4b8c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:48.729856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdcaba0-d231-420f-9d19-4a534db4b8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:48.730128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdcaba0-d231-420f-9d19-4a534db4b8c0 admin admin] PATH_INFO: `/v3/users/ced61e0fee084ba5ba286e26f385a04b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:48.731270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdcaba0-d231-420f-9d19-4a534db4b8c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ced61e0fee084ba5ba286e26f385a04b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:48.733215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdcaba0-d231-420f-9d19-4a534db4b8c0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:48.778733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ecdcaba0-d231-420f-9d19-4a534db4b8c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ced61e0fee084ba5ba286e26f385a04b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.779378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ecdcaba0-d231-420f-9d19-4a534db4b8c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ced61e0fee084ba5ba286e26f385a04b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.779979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ecdcaba0-d231-420f-9d19-4a534db4b8c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ced61e0fee084ba5ba286e26f385a04b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.787651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ecdcaba0-d231-420f-9d19-4a534db4b8c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ced61e0fee084ba5ba286e26f385a04b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.799866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ecdcaba0-d231-420f-9d19-4a534db4b8c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ced61e0fee084ba5ba286e26f385a04b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:48.812488 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4132/8329] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:25:48 2022] DELETE /identity/v3/users/ced61e0fee084ba5ba286e26f385a04b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:48.816932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca982e8-c4fe-4c66-8331-08d477321600 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:48.817224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca982e8-c4fe-4c66-8331-08d477321600 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:48.817502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca982e8-c4fe-4c66-8331-08d477321600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:48.839274 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1ca982e8-c4fe-4c66-8331-08d477321600 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:48.840482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1ca982e8-c4fe-4c66-8331-08d477321600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:48.844466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=2,keystone:DELETE=16,keystone:SELECT=82 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:48.904776 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4198/8330] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:25:48.914517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdcaba0-d231-420f-9d19-4a534db4b8c0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:48.915949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ecdcaba0-d231-420f-9d19-4a534db4b8c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:48.927982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:UPDATE=1,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:48.982678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdcaba0-d231-420f-9d19-4a534db4b8c0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:48.986406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfebb9eb-fd25-4c98-825e-e1620fca9a5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:48.987056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebb9eb-fd25-4c98-825e-e1620fca9a5d service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:48.987335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebb9eb-fd25-4c98-825e-e1620fca9a5d service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:48.987673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebb9eb-fd25-4c98-825e-e1620fca9a5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:48.997122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca982e8-c4fe-4c66-8331-08d477321600 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:48.998565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1ca982e8-c4fe-4c66-8331-08d477321600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:49.002790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfebb9eb-fd25-4c98-825e-e1620fca9a5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:49.004365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfebb9eb-fd25-4c98-825e-e1620fca9a5d service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:49.015894 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4133/8331] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:25:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:25:49.103303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca982e8-c4fe-4c66-8331-08d477321600 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:49.110304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89094490-b9bf-46f7-a030-2aadee36d308 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:49.111027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89094490-b9bf-46f7-a030-2aadee36d308 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:49.111383 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89094490-b9bf-46f7-a030-2aadee36d308 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:49.111666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89094490-b9bf-46f7-a030-2aadee36d308 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:49.113789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-89094490-b9bf-46f7-a030-2aadee36d308 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:49.130465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89094490-b9bf-46f7-a030-2aadee36d308 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:49.132078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89094490-b9bf-46f7-a030-2aadee36d308 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:49.150099 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4199/8332] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:25:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:25:49.307039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfebb9eb-fd25-4c98-825e-e1620fca9a5d service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:49.309134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-dfebb9eb-fd25-4c98-825e-e1620fca9a5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:49.323545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfebb9eb-fd25-4c98-825e-e1620fca9a5d service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:49.328710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4cf63-3601-494e-897a-e4fd824df110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:49.329502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4cf63-3601-494e-897a-e4fd824df110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:49.329873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4cf63-3601-494e-897a-e4fd824df110 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:49.330251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4cf63-3601-494e-897a-e4fd824df110 admin admin] PATH_INFO: `/v3/projects/4fc5f66263594b21ba1ef9b9ae79db25` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:49.331647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4cf63-3601-494e-897a-e4fd824df110 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fc5f66263594b21ba1ef9b9ae79db25)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:49.333693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4cf63-3601-494e-897a-e4fd824df110 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:49.383862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4de4cf63-3601-494e-897a-e4fd824df110 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fc5f66263594b21ba1ef9b9ae79db25', 'request_id': 'req-4de4cf63-3601-494e-897a-e4fd824df110'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:49.391160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4de4cf63-3601-494e-897a-e4fd824df110 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fc5f66263594b21ba1ef9b9ae79db25', 'request_id': 'req-4de4cf63-3601-494e-897a-e4fd824df110'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:49.396730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4de4cf63-3601-494e-897a-e4fd824df110 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fc5f66263594b21ba1ef9b9ae79db25', 'request_id': 'req-4de4cf63-3601-494e-897a-e4fd824df110'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:49.408155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4de4cf63-3601-494e-897a-e4fd824df110 admin admin] The token cache is being invalidate because project 4fc5f66263594b21ba1ef9b9ae79db25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:49.408445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4de4cf63-3601-494e-897a-e4fd824df110 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:49.409078 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4de4cf63-3601-494e-897a-e4fd824df110 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:49.410568 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4134/8333] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:49 2022] DELETE /identity/v3/projects/4fc5f66263594b21ba1ef9b9ae79db25 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:49.414588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89094490-b9bf-46f7-a030-2aadee36d308 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:49.425998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89094490-b9bf-46f7-a030-2aadee36d308 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:49.482047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c782f1-13a0-4ff9-8862-03e82ea70d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:49.482694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c782f1-13a0-4ff9-8862-03e82ea70d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:49.482975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c782f1-13a0-4ff9-8862-03e82ea70d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:49.483249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c782f1-13a0-4ff9-8862-03e82ea70d6f admin admin] PATH_INFO: `/v3/users/ab7b55af965f49099672be7e3d14dae4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:49.484357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c782f1-13a0-4ff9-8862-03e82ea70d6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab7b55af965f49099672be7e3d14dae4)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:49.486240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c782f1-13a0-4ff9-8862-03e82ea70d6f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:49.524000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:49.533719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-12c782f1-13a0-4ff9-8862-03e82ea70d6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab7b55af965f49099672be7e3d14dae4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:49.540632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-12c782f1-13a0-4ff9-8862-03e82ea70d6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab7b55af965f49099672be7e3d14dae4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:49.553212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-12c782f1-13a0-4ff9-8862-03e82ea70d6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab7b55af965f49099672be7e3d14dae4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:49.553885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-12c782f1-13a0-4ff9-8862-03e82ea70d6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab7b55af965f49099672be7e3d14dae4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:49.561019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-12c782f1-13a0-4ff9-8862-03e82ea70d6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab7b55af965f49099672be7e3d14dae4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:49.567148 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4200/8334] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:25:49 2022] DELETE /identity/v3/users/ab7b55af965f49099672be7e3d14dae4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:49.918722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4cf63-3601-494e-897a-e4fd824df110 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:49.920160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4de4cf63-3601-494e-897a-e4fd824df110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:49.933544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4cf63-3601-494e-897a-e4fd824df110 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:49.990881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb99e2f-b451-4485-8d0c-ce01fb04cb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:49.991599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb99e2f-b451-4485-8d0c-ce01fb04cb4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:49.991883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb99e2f-b451-4485-8d0c-ce01fb04cb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:49.992155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb99e2f-b451-4485-8d0c-ce01fb04cb4f admin admin] PATH_INFO: `/v3/projects/5b0792b17b1b43e8a5cf92d41acc827d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:49.993265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb99e2f-b451-4485-8d0c-ce01fb04cb4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b0792b17b1b43e8a5cf92d41acc827d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:49.995853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb99e2f-b451-4485-8d0c-ce01fb04cb4f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:50.032399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:50.042559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7fb99e2f-b451-4485-8d0c-ce01fb04cb4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b0792b17b1b43e8a5cf92d41acc827d', 'request_id': 'req-7fb99e2f-b451-4485-8d0c-ce01fb04cb4f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:50.050407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7fb99e2f-b451-4485-8d0c-ce01fb04cb4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b0792b17b1b43e8a5cf92d41acc827d', 'request_id': 'req-7fb99e2f-b451-4485-8d0c-ce01fb04cb4f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:50.056164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7fb99e2f-b451-4485-8d0c-ce01fb04cb4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b0792b17b1b43e8a5cf92d41acc827d', 'request_id': 'req-7fb99e2f-b451-4485-8d0c-ce01fb04cb4f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:50.067620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7fb99e2f-b451-4485-8d0c-ce01fb04cb4f admin admin] The token cache is being invalidate because project 5b0792b17b1b43e8a5cf92d41acc827d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:50.067907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7fb99e2f-b451-4485-8d0c-ce01fb04cb4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:50.068540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7fb99e2f-b451-4485-8d0c-ce01fb04cb4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:50.070010 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4135/8335] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:49 2022] DELETE /identity/v3/projects/5b0792b17b1b43e8a5cf92d41acc827d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:50.074853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f25315-2934-4529-aa3e-573f9783645d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:50.075170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f25315-2934-4529-aa3e-573f9783645d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:50.075478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f25315-2934-4529-aa3e-573f9783645d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:50.095512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-90f25315-2934-4529-aa3e-573f9783645d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:50.096711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-90f25315-2934-4529-aa3e-573f9783645d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:50.157804 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4201/8336] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:25:50.166931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f363e1-4adb-4fea-b424-d710a1f8e9ea None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:50.167765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f363e1-4adb-4fea-b424-d710a1f8e9ea None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:50.168048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f363e1-4adb-4fea-b424-d710a1f8e9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:50.190360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b7f363e1-4adb-4fea-b424-d710a1f8e9ea None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:50.194622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b7f363e1-4adb-4fea-b424-d710a1f8e9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:50.203554 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4136/8337] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:25:50.207818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f25315-2934-4529-aa3e-573f9783645d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:50.218081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f25315-2934-4529-aa3e-573f9783645d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:50.222701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea769218-cf86-4ccc-acf8-e138d8e98212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:50.223292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea769218-cf86-4ccc-acf8-e138d8e98212 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:50.223571 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea769218-cf86-4ccc-acf8-e138d8e98212 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:50.223844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea769218-cf86-4ccc-acf8-e138d8e98212 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:50.225227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea769218-cf86-4ccc-acf8-e138d8e98212 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:25:50.225701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea769218-cf86-4ccc-acf8-e138d8e98212 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:50.227335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea769218-cf86-4ccc-acf8-e138d8e98212 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:50.240063 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4202/8338] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:50.244440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f363e1-4adb-4fea-b424-d710a1f8e9ea None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:50.254098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f363e1-4adb-4fea-b424-d710a1f8e9ea None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:50.258998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495c641-cfb5-4b6f-8a0d-cfee388134a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:50.259598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495c641-cfb5-4b6f-8a0d-cfee388134a9 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:50.259883 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495c641-cfb5-4b6f-8a0d-cfee388134a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:50.260224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495c641-cfb5-4b6f-8a0d-cfee388134a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:50.260961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e495c641-cfb5-4b6f-8a0d-cfee388134a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:50.262917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e495c641-cfb5-4b6f-8a0d-cfee388134a9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:50.290544 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4137/8339] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:25:50 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:50.295551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea769218-cf86-4ccc-acf8-e138d8e98212 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:50.305543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea769218-cf86-4ccc-acf8-e138d8e98212 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:50.364337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c447f-2a8e-4086-8892-b8aba885e215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:50.364964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c447f-2a8e-4086-8892-b8aba885e215 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:50.365245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c447f-2a8e-4086-8892-b8aba885e215 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:50.365517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c447f-2a8e-4086-8892-b8aba885e215 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:50.366153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c447f-2a8e-4086-8892-b8aba885e215 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:50.368085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c447f-2a8e-4086-8892-b8aba885e215 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:50.390820 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4203/8340] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:25:50 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:50.394764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495c641-cfb5-4b6f-8a0d-cfee388134a9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:50.405669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495c641-cfb5-4b6f-8a0d-cfee388134a9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:50.411435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1ecc2-9f3b-4b1c-a682-252d4447f776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:50.412546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1ecc2-9f3b-4b1c-a682-252d4447f776 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:50.413289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1ecc2-9f3b-4b1c-a682-252d4447f776 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:50.413719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1ecc2-9f3b-4b1c-a682-252d4447f776 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:50.414528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be1ecc2-9f3b-4b1c-a682-252d4447f776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:50.417594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be1ecc2-9f3b-4b1c-a682-252d4447f776 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:50.435736 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4138/8341] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:25:50.437038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c447f-2a8e-4086-8892-b8aba885e215 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:50.447560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c447f-2a8e-4086-8892-b8aba885e215 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:50.452474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd84c5e-c3e8-4959-99ec-fdffa5cc5eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:50.453077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd84c5e-c3e8-4959-99ec-fdffa5cc5eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:50.453354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd84c5e-c3e8-4959-99ec-fdffa5cc5eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:50.453623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd84c5e-c3e8-4959-99ec-fdffa5cc5eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:50.454260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd84c5e-c3e8-4959-99ec-fdffa5cc5eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:50.455869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd84c5e-c3e8-4959-99ec-fdffa5cc5eb2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:50.466244 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4204/8342] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:25:50.470805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1ecc2-9f3b-4b1c-a682-252d4447f776 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:50.481430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1ecc2-9f3b-4b1c-a682-252d4447f776 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:50.486147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0edbf8-f453-4054-96d1-0a41f602d52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:50.486827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0edbf8-f453-4054-96d1-0a41f602d52b admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:50.487109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0edbf8-f453-4054-96d1-0a41f602d52b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:50.487384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0edbf8-f453-4054-96d1-0a41f602d52b admin admin] PATH_INFO: `/v3/projects/b0fd159c0f574d44925a6c0d020eb43f/users/8d97088b6cc74035a4e16262966501b2/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:50.503490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0edbf8-f453-4054-96d1-0a41f602d52b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0fd159c0f574d44925a6c0d020eb43f, user_id=8d97088b6cc74035a4e16262966501b2, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:50.506136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0edbf8-f453-4054-96d1-0a41f602d52b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:50.519047 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4139/8343] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:25:50 2022] PUT /identity/v3/projects/b0fd159c0f574d44925a6c0d020eb43f/users/8d97088b6cc74035a4e16262966501b2/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:50.529260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e78c08-984c-4d70-8569-b3affd9047e3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:50.529745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e78c08-984c-4d70-8569-b3affd9047e3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:50.529975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e78c08-984c-4d70-8569-b3affd9047e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:50.558450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-44e78c08-984c-4d70-8569-b3affd9047e3 None None] MFA Rules not processed for user `8d97088b6cc74035a4e16262966501b2`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:50.559701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-44e78c08-984c-4d70-8569-b3affd9047e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:50.567738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:50.624541 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4205/8344] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:25:50.629254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0edbf8-f453-4054-96d1-0a41f602d52b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:50.690530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0edbf8-f453-4054-96d1-0a41f602d52b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:50.695221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc239da9-f8fb-427b-aa9e-266bb06ade50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:50.695826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc239da9-f8fb-427b-aa9e-266bb06ade50 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:50.696108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc239da9-f8fb-427b-aa9e-266bb06ade50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:50.696384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc239da9-f8fb-427b-aa9e-266bb06ade50 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:50.697056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc239da9-f8fb-427b-aa9e-266bb06ade50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:50.699000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc239da9-f8fb-427b-aa9e-266bb06ade50 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:50.724741 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4140/8345] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:25:50 2022] POST /identity/v3/projects => generated 369 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:50.729092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e78c08-984c-4d70-8569-b3affd9047e3 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:50.740492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e78c08-984c-4d70-8569-b3affd9047e3 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:50.796652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c40ca-381a-46ce-b2db-9b24bf023bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:50.797277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c40ca-381a-46ce-b2db-9b24bf023bda admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:50.797563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c40ca-381a-46ce-b2db-9b24bf023bda admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:50.797836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c40ca-381a-46ce-b2db-9b24bf023bda admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:50.798465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c40ca-381a-46ce-b2db-9b24bf023bda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:50.800426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c40ca-381a-46ce-b2db-9b24bf023bda admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:50.824766 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4206/8346] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:25:50 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:50.825926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc239da9-f8fb-427b-aa9e-266bb06ade50 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:50.836270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc239da9-f8fb-427b-aa9e-266bb06ade50 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:50.841424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d4dc0-047f-4ea7-a925-aea778004f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:50.842017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d4dc0-047f-4ea7-a925-aea778004f77 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:50.842302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d4dc0-047f-4ea7-a925-aea778004f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:50.842613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d4dc0-047f-4ea7-a925-aea778004f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:50.843243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d4dc0-047f-4ea7-a925-aea778004f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:50.844829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d4dc0-047f-4ea7-a925-aea778004f77 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:50.856089 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4141/8347] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:25:50.860161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c40ca-381a-46ce-b2db-9b24bf023bda admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:50.870098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c40ca-381a-46ce-b2db-9b24bf023bda admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:50.875257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad512dd-4544-41ca-a9ed-c925b884672a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:50.875898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad512dd-4544-41ca-a9ed-c925b884672a admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:50.876178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad512dd-4544-41ca-a9ed-c925b884672a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:50.876453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad512dd-4544-41ca-a9ed-c925b884672a admin admin] PATH_INFO: `/v3/projects/7e894082fa89418986be5db793e1e676/users/329cae2bea504cb2b16e44f4c9f91865/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:50.892645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad512dd-4544-41ca-a9ed-c925b884672a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e894082fa89418986be5db793e1e676, user_id=329cae2bea504cb2b16e44f4c9f91865, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:50.895225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad512dd-4544-41ca-a9ed-c925b884672a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:50.909152 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4207/8348] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:25:50 2022] PUT /identity/v3/projects/7e894082fa89418986be5db793e1e676/users/329cae2bea504cb2b16e44f4c9f91865/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:50.913122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d4dc0-047f-4ea7-a925-aea778004f77 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:50.924632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d4dc0-047f-4ea7-a925-aea778004f77 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:50.959055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:50.988507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73e051-315f-4b8b-a0b5-e63dfcd7f2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:50.989120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73e051-315f-4b8b-a0b5-e63dfcd7f2ee admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:50.989398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73e051-315f-4b8b-a0b5-e63dfcd7f2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:50.989666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73e051-315f-4b8b-a0b5-e63dfcd7f2ee admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:50.990425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73e051-315f-4b8b-a0b5-e63dfcd7f2ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:50.992058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73e051-315f-4b8b-a0b5-e63dfcd7f2ee admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:51.003166 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4142/8349] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:25:51.007224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad512dd-4544-41ca-a9ed-c925b884672a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:51.018625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad512dd-4544-41ca-a9ed-c925b884672a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:51.023580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139aeaeb-6c16-47b6-b052-6f3c2581a985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:51.024266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139aeaeb-6c16-47b6-b052-6f3c2581a985 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:51.024550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139aeaeb-6c16-47b6-b052-6f3c2581a985 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:51.024833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139aeaeb-6c16-47b6-b052-6f3c2581a985 admin admin] PATH_INFO: `/v3/projects/7e894082fa89418986be5db793e1e676/users/329cae2bea504cb2b16e44f4c9f91865/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:51.026939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139aeaeb-6c16-47b6-b052-6f3c2581a985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e894082fa89418986be5db793e1e676, user_id=329cae2bea504cb2b16e44f4c9f91865, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:51.029581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139aeaeb-6c16-47b6-b052-6f3c2581a985 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:51.040850 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4208/8350] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:25:51 2022] PUT /identity/v3/projects/7e894082fa89418986be5db793e1e676/users/329cae2bea504cb2b16e44f4c9f91865/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:51.053935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46b719-8413-4677-8d4c-b0181ee06b1c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:51.054227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46b719-8413-4677-8d4c-b0181ee06b1c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:51.054528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46b719-8413-4677-8d4c-b0181ee06b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:51.085590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bc46b719-8413-4677-8d4c-b0181ee06b1c None None] MFA Rules not processed for user `329cae2bea504cb2b16e44f4c9f91865`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:51.086816 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bc46b719-8413-4677-8d4c-b0181ee06b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:51.160250 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4143/8351] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:25:51.165258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139aeaeb-6c16-47b6-b052-6f3c2581a985 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:51.179367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139aeaeb-6c16-47b6-b052-6f3c2581a985 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:51.185326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97af25-0137-4ca5-8266-e1e4513f0652 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:51.186045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97af25-0137-4ca5-8266-e1e4513f0652 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:51.186630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97af25-0137-4ca5-8266-e1e4513f0652 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:51.186981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97af25-0137-4ca5-8266-e1e4513f0652 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:51.187688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b97af25-0137-4ca5-8266-e1e4513f0652 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:51.189608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b97af25-0137-4ca5-8266-e1e4513f0652 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:51.227851 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4209/8352] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:25:51 2022] POST /identity/v3/projects => generated 359 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:51.232552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46b719-8413-4677-8d4c-b0181ee06b1c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:51.242005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46b719-8413-4677-8d4c-b0181ee06b1c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:51.304577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd033893-c1cf-48b5-810f-216bfd9c4611 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:51.305203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd033893-c1cf-48b5-810f-216bfd9c4611 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:51.305488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd033893-c1cf-48b5-810f-216bfd9c4611 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:51.305820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd033893-c1cf-48b5-810f-216bfd9c4611 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:51.306456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd033893-c1cf-48b5-810f-216bfd9c4611 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:51.308992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd033893-c1cf-48b5-810f-216bfd9c4611 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:51.340743 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4144/8353] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:25:51 2022] POST /identity/v3/users => generated 413 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:51.342128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97af25-0137-4ca5-8266-e1e4513f0652 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:51.353101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97af25-0137-4ca5-8266-e1e4513f0652 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:51.358370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1cff5b-df08-4f54-9baa-babe3005d5ae tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:51.359037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cff5b-df08-4f54-9baa-babe3005d5ae tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:51.359335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cff5b-df08-4f54-9baa-babe3005d5ae tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:51.359628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cff5b-df08-4f54-9baa-babe3005d5ae tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/users/0fa8d11e4f0d4918a351c0f46d30fb8a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:51.374183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1cff5b-df08-4f54-9baa-babe3005d5ae tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:get_user(user_id=0fa8d11e4f0d4918a351c0f46d30fb8a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:51.376448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1cff5b-df08-4f54-9baa-babe3005d5ae tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:51.378095 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4210/8354] 10.222.0.17 () {58 vars in 1221 bytes} [Sat Oct 1 21:25:51 2022] GET /identity/v3/users/0fa8d11e4f0d4918a351c0f46d30fb8a => generated 413 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:51.382387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd033893-c1cf-48b5-810f-216bfd9c4611 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:51.392740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd033893-c1cf-48b5-810f-216bfd9c4611 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:51.397391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618c117-2692-4bb2-8e3f-5cacfb785273 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:51.398036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618c117-2692-4bb2-8e3f-5cacfb785273 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:51.398296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618c117-2692-4bb2-8e3f-5cacfb785273 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:51.398603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618c117-2692-4bb2-8e3f-5cacfb785273 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/users/0fa8d11e4f0d4918a351c0f46d30fb8a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:51.399780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618c117-2692-4bb2-8e3f-5cacfb785273 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:delete_user(user_id=0fa8d11e4f0d4918a351c0f46d30fb8a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:51.401902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618c117-2692-4bb2-8e3f-5cacfb785273 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:51.444369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:51.451071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c618c117-2692-4bb2-8e3f-5cacfb785273 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fa8d11e4f0d4918a351c0f46d30fb8a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:51.452854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c618c117-2692-4bb2-8e3f-5cacfb785273 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fa8d11e4f0d4918a351c0f46d30fb8a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:51.453856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c618c117-2692-4bb2-8e3f-5cacfb785273 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fa8d11e4f0d4918a351c0f46d30fb8a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:51.463321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c618c117-2692-4bb2-8e3f-5cacfb785273 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fa8d11e4f0d4918a351c0f46d30fb8a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:51.476346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-c618c117-2692-4bb2-8e3f-5cacfb785273 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fa8d11e4f0d4918a351c0f46d30fb8a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:51.489071 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4145/8355] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:25:51 2022] DELETE /identity/v3/users/0fa8d11e4f0d4918a351c0f46d30fb8a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:25:51.493035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1cff5b-df08-4f54-9baa-babe3005d5ae tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:51.495037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bd1cff5b-df08-4f54-9baa-babe3005d5ae tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:51.509548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1cff5b-df08-4f54-9baa-babe3005d5ae tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:51.529440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:51.569441 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51263e64-87b5-4c2f-b2fe-f3d8645f2ed0 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:51.570198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51263e64-87b5-4c2f-b2fe-f3d8645f2ed0 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:51.570496 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51263e64-87b5-4c2f-b2fe-f3d8645f2ed0 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:51.570802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51263e64-87b5-4c2f-b2fe-f3d8645f2ed0 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/437d9be27c4b4e80bc8dda69971946bb` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:51.572002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51263e64-87b5-4c2f-b2fe-f3d8645f2ed0 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:delete_project(project_id=437d9be27c4b4e80bc8dda69971946bb)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:51.574082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51263e64-87b5-4c2f-b2fe-f3d8645f2ed0 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:51.621799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-51263e64-87b5-4c2f-b2fe-f3d8645f2ed0 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '437d9be27c4b4e80bc8dda69971946bb', 'request_id': 'req-51263e64-87b5-4c2f-b2fe-f3d8645f2ed0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:51.628234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-51263e64-87b5-4c2f-b2fe-f3d8645f2ed0 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '437d9be27c4b4e80bc8dda69971946bb', 'request_id': 'req-51263e64-87b5-4c2f-b2fe-f3d8645f2ed0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:51.634318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-51263e64-87b5-4c2f-b2fe-f3d8645f2ed0 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '437d9be27c4b4e80bc8dda69971946bb', 'request_id': 'req-51263e64-87b5-4c2f-b2fe-f3d8645f2ed0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:51.647777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-51263e64-87b5-4c2f-b2fe-f3d8645f2ed0 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] The token cache is being invalidate because project 437d9be27c4b4e80bc8dda69971946bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:51.648068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-51263e64-87b5-4c2f-b2fe-f3d8645f2ed0 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:51.648793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-51263e64-87b5-4c2f-b2fe-f3d8645f2ed0 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:51.650346 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4211/8356] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:51 2022] DELETE /identity/v3/projects/437d9be27c4b4e80bc8dda69971946bb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:51.656425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618c117-2692-4bb2-8e3f-5cacfb785273 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:51.668091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618c117-2692-4bb2-8e3f-5cacfb785273 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:51.726641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d0806-84fc-4a86-b8b0-9ff404871693 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:51.727253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d0806-84fc-4a86-b8b0-9ff404871693 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:51.727533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d0806-84fc-4a86-b8b0-9ff404871693 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:51.727809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d0806-84fc-4a86-b8b0-9ff404871693 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:51.728454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d0806-84fc-4a86-b8b0-9ff404871693 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:51.730739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d0806-84fc-4a86-b8b0-9ff404871693 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:51.747496 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4146/8357] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:25:51 2022] POST /identity/v3/projects => generated 348 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:25:51.751750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51263e64-87b5-4c2f-b2fe-f3d8645f2ed0 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:51.753208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-51263e64-87b5-4c2f-b2fe-f3d8645f2ed0 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:51.765977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51263e64-87b5-4c2f-b2fe-f3d8645f2ed0 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:51.827692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9b655-fa98-490e-9753-37c016fa5f28 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:51.828303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9b655-fa98-490e-9753-37c016fa5f28 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:51.828584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9b655-fa98-490e-9753-37c016fa5f28 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:51.828857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9b655-fa98-490e-9753-37c016fa5f28 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:51.829524 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9b655-fa98-490e-9753-37c016fa5f28 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Adding query filter params (is_domain=True) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:25:51.829831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9b655-fa98-490e-9753-37c016fa5f28 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:51.831806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9b655-fa98-490e-9753-37c016fa5f28 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:51.845306 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4212/8358] 10.222.0.17 () {58 vars in 1160 bytes} [Sat Oct 1 21:25:51 2022] GET /identity/v3/projects?is_domain=True => generated 1020 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:25:51.849602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d0806-84fc-4a86-b8b0-9ff404871693 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:51.860482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d0806-84fc-4a86-b8b0-9ff404871693 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:51.865181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3591e-dd0a-4eb7-9426-106473d0a1ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:51.865783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3591e-dd0a-4eb7-9426-106473d0a1ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:51.866067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3591e-dd0a-4eb7-9426-106473d0a1ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:51.866346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3591e-dd0a-4eb7-9426-106473d0a1ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:51.867001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3591e-dd0a-4eb7-9426-106473d0a1ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:51.868601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3591e-dd0a-4eb7-9426-106473d0a1ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:51.881754 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4147/8359] 10.222.0.17 () {58 vars in 1128 bytes} [Sat Oct 1 21:25:51 2022] GET /identity/v3/domains => generated 829 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:51.886630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9b655-fa98-490e-9753-37c016fa5f28 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:51.896617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9b655-fa98-490e-9753-37c016fa5f28 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:51.902092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6496e-9875-4e53-8c64-e7d254fa4716 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:51.902758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6496e-9875-4e53-8c64-e7d254fa4716 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:51.903340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6496e-9875-4e53-8c64-e7d254fa4716 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:51.903340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6496e-9875-4e53-8c64-e7d254fa4716 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/00887a3dcb234eedb9babc7232357e19` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:51.904508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6496e-9875-4e53-8c64-e7d254fa4716 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:update_project(project_id=00887a3dcb234eedb9babc7232357e19)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:51.905496 ncdv-master-1 devstack@keystone.service[99139]: WARNING py.warnings [None req-4cd6496e-9875-4e53-8c64-e7d254fa4716 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:25:51.905496 ncdv-master-1 devstack@keystone.service[99139]: warnings.warn(msg) Oct 01 21:25:51.905496 ncdv-master-1 devstack@keystone.service[99139]: Oct 01 21:25:51.907046 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6496e-9875-4e53-8c64-e7d254fa4716 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:51.919939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4cd6496e-9875-4e53-8c64-e7d254fa4716 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '00887a3dcb234eedb9babc7232357e19'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:51.920682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4cd6496e-9875-4e53-8c64-e7d254fa4716 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '00887a3dcb234eedb9babc7232357e19'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:51.946725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4cd6496e-9875-4e53-8c64-e7d254fa4716 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '00887a3dcb234eedb9babc7232357e19'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:51.946851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4cd6496e-9875-4e53-8c64-e7d254fa4716 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '00887a3dcb234eedb9babc7232357e19'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:51.949686 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4213/8360] 10.222.0.17 () {60 vars in 1252 bytes} [Sat Oct 1 21:25:51 2022] PATCH /identity/v3/projects/00887a3dcb234eedb9babc7232357e19 => generated 362 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:51.953785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3591e-dd0a-4eb7-9426-106473d0a1ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:51.956408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5fa3591e-dd0a-4eb7-9426-106473d0a1ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:51.971676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3591e-dd0a-4eb7-9426-106473d0a1ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:52.043796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6e509-d106-49ef-943b-1608ba532a9e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:52.044558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6e509-d106-49ef-943b-1608ba532a9e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:52.044842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6e509-d106-49ef-943b-1608ba532a9e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:52.045119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6e509-d106-49ef-943b-1608ba532a9e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/00887a3dcb234eedb9babc7232357e19` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:52.056478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a6e509-d106-49ef-943b-1608ba532a9e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:delete_project(project_id=00887a3dcb234eedb9babc7232357e19)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:52.058876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a6e509-d106-49ef-943b-1608ba532a9e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:52.081375 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1a6e509-d106-49ef-943b-1608ba532a9e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '00887a3dcb234eedb9babc7232357e19'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:52.092738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1a6e509-d106-49ef-943b-1608ba532a9e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '00887a3dcb234eedb9babc7232357e19'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:52.095302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:52.138784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1a6e509-d106-49ef-943b-1608ba532a9e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00887a3dcb234eedb9babc7232357e19', 'request_id': 'req-d1a6e509-d106-49ef-943b-1608ba532a9e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:52.147943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1a6e509-d106-49ef-943b-1608ba532a9e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00887a3dcb234eedb9babc7232357e19', 'request_id': 'req-d1a6e509-d106-49ef-943b-1608ba532a9e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:52.154089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1a6e509-d106-49ef-943b-1608ba532a9e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00887a3dcb234eedb9babc7232357e19', 'request_id': 'req-d1a6e509-d106-49ef-943b-1608ba532a9e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:52.165634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1a6e509-d106-49ef-943b-1608ba532a9e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] The token cache is being invalidate because project 00887a3dcb234eedb9babc7232357e19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:52.165925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1a6e509-d106-49ef-943b-1608ba532a9e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:52.166875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1a6e509-d106-49ef-943b-1608ba532a9e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:52.174699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1a6e509-d106-49ef-943b-1608ba532a9e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '00887a3dcb234eedb9babc7232357e19', 'request_id': 'req-d1a6e509-d106-49ef-943b-1608ba532a9e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:52.175363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1a6e509-d106-49ef-943b-1608ba532a9e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '00887a3dcb234eedb9babc7232357e19', 'request_id': 'req-d1a6e509-d106-49ef-943b-1608ba532a9e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:52.178780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-d1a6e509-d106-49ef-943b-1608ba532a9e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '00887a3dcb234eedb9babc7232357e19', 'request_id': 'req-d1a6e509-d106-49ef-943b-1608ba532a9e'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:52.185115 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4148/8361] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:51 2022] DELETE /identity/v3/projects/00887a3dcb234eedb9babc7232357e19 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:52.190072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6496e-9875-4e53-8c64-e7d254fa4716 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:52.200339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6496e-9875-4e53-8c64-e7d254fa4716 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:52.221860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:52.271863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ded94-4e94-424a-9392-c56a704190e9 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:52.272758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ded94-4e94-424a-9392-c56a704190e9 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:52.273134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ded94-4e94-424a-9392-c56a704190e9 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:52.273516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ded94-4e94-424a-9392-c56a704190e9 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:52.274321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ded94-4e94-424a-9392-c56a704190e9 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:52.276266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ded94-4e94-424a-9392-c56a704190e9 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:52.310527 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4214/8362] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:25:52 2022] POST /identity/v3/projects => generated 358 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:25:52.314451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6e509-d106-49ef-943b-1608ba532a9e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:52.315884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d1a6e509-d106-49ef-943b-1608ba532a9e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:52.328360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6e509-d106-49ef-943b-1608ba532a9e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:52.389349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03bd4f-dacd-4f72-9bfc-076ccb622f45 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:52.390173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03bd4f-dacd-4f72-9bfc-076ccb622f45 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:52.390555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03bd4f-dacd-4f72-9bfc-076ccb622f45 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:52.390907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03bd4f-dacd-4f72-9bfc-076ccb622f45 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/a74560c857954bf095c3111e529df31c` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:52.392136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03bd4f-dacd-4f72-9bfc-076ccb622f45 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:get_project(project_id=a74560c857954bf095c3111e529df31c)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:52.394343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03bd4f-dacd-4f72-9bfc-076ccb622f45 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:52.396114 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4149/8363] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:25:52 2022] GET /identity/v3/projects/a74560c857954bf095c3111e529df31c => generated 358 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:52.400644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ded94-4e94-424a-9392-c56a704190e9 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:52.412893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ded94-4e94-424a-9392-c56a704190e9 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:52.418768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d45065-de23-49e1-8917-29ed35d49c14 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:52.419413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d45065-de23-49e1-8917-29ed35d49c14 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:52.419702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d45065-de23-49e1-8917-29ed35d49c14 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:52.419979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d45065-de23-49e1-8917-29ed35d49c14 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/a74560c857954bf095c3111e529df31c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:52.421128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d45065-de23-49e1-8917-29ed35d49c14 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a74560c857954bf095c3111e529df31c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:52.423297 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d45065-de23-49e1-8917-29ed35d49c14 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:52.477127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0d45065-de23-49e1-8917-29ed35d49c14 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a74560c857954bf095c3111e529df31c', 'request_id': 'req-a0d45065-de23-49e1-8917-29ed35d49c14'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:52.484306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0d45065-de23-49e1-8917-29ed35d49c14 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a74560c857954bf095c3111e529df31c', 'request_id': 'req-a0d45065-de23-49e1-8917-29ed35d49c14'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:52.489909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0d45065-de23-49e1-8917-29ed35d49c14 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a74560c857954bf095c3111e529df31c', 'request_id': 'req-a0d45065-de23-49e1-8917-29ed35d49c14'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:52.503723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0d45065-de23-49e1-8917-29ed35d49c14 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] The token cache is being invalidate because project a74560c857954bf095c3111e529df31c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:52.504021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0d45065-de23-49e1-8917-29ed35d49c14 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:52.504855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0d45065-de23-49e1-8917-29ed35d49c14 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:52.506474 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4215/8364] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:52 2022] DELETE /identity/v3/projects/a74560c857954bf095c3111e529df31c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:25:52.511612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03bd4f-dacd-4f72-9bfc-076ccb622f45 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:52.513451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0f03bd4f-dacd-4f72-9bfc-076ccb622f45 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:52.527149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03bd4f-dacd-4f72-9bfc-076ccb622f45 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:52.591159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb2f0c-d2f5-4ece-9245-cb5b1e58e698 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:52.591919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb2f0c-d2f5-4ece-9245-cb5b1e58e698 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:52.592269 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb2f0c-d2f5-4ece-9245-cb5b1e58e698 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:52.592615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb2f0c-d2f5-4ece-9245-cb5b1e58e698 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:52.593323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeb2f0c-d2f5-4ece-9245-cb5b1e58e698 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:52.595361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeb2f0c-d2f5-4ece-9245-cb5b1e58e698 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:52.628827 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4150/8365] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:25:52 2022] POST /identity/v3/projects => generated 360 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:25:52.633177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d45065-de23-49e1-8917-29ed35d49c14 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:52.644005 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d45065-de23-49e1-8917-29ed35d49c14 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:52.706501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669aa4e-0a1d-4e17-9eaa-1cea42efe944 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:52.707174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669aa4e-0a1d-4e17-9eaa-1cea42efe944 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:52.707463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669aa4e-0a1d-4e17-9eaa-1cea42efe944 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:52.707742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669aa4e-0a1d-4e17-9eaa-1cea42efe944 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/544c844b7b4b402ea1be205173eb21cc` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:52.708904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669aa4e-0a1d-4e17-9eaa-1cea42efe944 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:get_project(project_id=544c844b7b4b402ea1be205173eb21cc)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:52.711001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669aa4e-0a1d-4e17-9eaa-1cea42efe944 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:52.712663 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4216/8366] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:25:52 2022] GET /identity/v3/projects/544c844b7b4b402ea1be205173eb21cc => generated 360 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:52.717677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb2f0c-d2f5-4ece-9245-cb5b1e58e698 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:52.724674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:52.728988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb2f0c-d2f5-4ece-9245-cb5b1e58e698 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:52.735458 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74157d-0a94-46e4-ad5b-74a933ea680a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:52.736287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74157d-0a94-46e4-ad5b-74a933ea680a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:52.736754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74157d-0a94-46e4-ad5b-74a933ea680a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:52.737150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74157d-0a94-46e4-ad5b-74a933ea680a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/544c844b7b4b402ea1be205173eb21cc` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:52.738482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74157d-0a94-46e4-ad5b-74a933ea680a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:delete_project(project_id=544c844b7b4b402ea1be205173eb21cc)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:52.740698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74157d-0a94-46e4-ad5b-74a933ea680a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:52.795860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9b74157d-0a94-46e4-ad5b-74a933ea680a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '544c844b7b4b402ea1be205173eb21cc', 'request_id': 'req-9b74157d-0a94-46e4-ad5b-74a933ea680a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:52.809406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9b74157d-0a94-46e4-ad5b-74a933ea680a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '544c844b7b4b402ea1be205173eb21cc', 'request_id': 'req-9b74157d-0a94-46e4-ad5b-74a933ea680a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:52.815658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9b74157d-0a94-46e4-ad5b-74a933ea680a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '544c844b7b4b402ea1be205173eb21cc', 'request_id': 'req-9b74157d-0a94-46e4-ad5b-74a933ea680a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:52.829089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9b74157d-0a94-46e4-ad5b-74a933ea680a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] The token cache is being invalidate because project 544c844b7b4b402ea1be205173eb21cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:52.829914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9b74157d-0a94-46e4-ad5b-74a933ea680a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:52.830729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9b74157d-0a94-46e4-ad5b-74a933ea680a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:52.832298 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4151/8367] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:52 2022] DELETE /identity/v3/projects/544c844b7b4b402ea1be205173eb21cc => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:52.837487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669aa4e-0a1d-4e17-9eaa-1cea42efe944 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:52.839457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e669aa4e-0a1d-4e17-9eaa-1cea42efe944 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:52.853220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669aa4e-0a1d-4e17-9eaa-1cea42efe944 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:52.878279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:52.913114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e77561-f964-4d08-8fda-9b9c7b753e84 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:52.913726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e77561-f964-4d08-8fda-9b9c7b753e84 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:52.914021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e77561-f964-4d08-8fda-9b9c7b753e84 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:52.914282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e77561-f964-4d08-8fda-9b9c7b753e84 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:52.914923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e77561-f964-4d08-8fda-9b9c7b753e84 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:52.917092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e77561-f964-4d08-8fda-9b9c7b753e84 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:52.950010 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4217/8368] 10.222.0.17 () {60 vars in 1152 bytes} [Sat Oct 1 21:25:52 2022] POST /identity/v3/projects => generated 347 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:25:52.953843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74157d-0a94-46e4-ad5b-74a933ea680a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:52.964889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74157d-0a94-46e4-ad5b-74a933ea680a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:53.020372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38024c-de22-4b5a-a02e-f9bbae5d65fd tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:53.020997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38024c-de22-4b5a-a02e-f9bbae5d65fd tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:53.021278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38024c-de22-4b5a-a02e-f9bbae5d65fd tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:53.021555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38024c-de22-4b5a-a02e-f9bbae5d65fd tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/473ca0d8e9334ff2a0463c8c1e3020b6` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:53.022701 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f38024c-de22-4b5a-a02e-f9bbae5d65fd tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:get_project(project_id=473ca0d8e9334ff2a0463c8c1e3020b6)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:53.024746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f38024c-de22-4b5a-a02e-f9bbae5d65fd tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:53.030045 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4152/8369] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:25:52 2022] GET /identity/v3/projects/473ca0d8e9334ff2a0463c8c1e3020b6 => generated 347 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:53.032872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e77561-f964-4d08-8fda-9b9c7b753e84 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:53.046872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e77561-f964-4d08-8fda-9b9c7b753e84 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:53.051750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df8988-ec1a-4f62-a17c-80ab85cc87f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:53.052459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df8988-ec1a-4f62-a17c-80ab85cc87f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:53.052983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df8988-ec1a-4f62-a17c-80ab85cc87f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:53.053353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df8988-ec1a-4f62-a17c-80ab85cc87f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/473ca0d8e9334ff2a0463c8c1e3020b6` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:53.054587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df8988-ec1a-4f62-a17c-80ab85cc87f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:delete_project(project_id=473ca0d8e9334ff2a0463c8c1e3020b6)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:53.056730 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df8988-ec1a-4f62-a17c-80ab85cc87f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:53.107037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a3df8988-ec1a-4f62-a17c-80ab85cc87f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '473ca0d8e9334ff2a0463c8c1e3020b6', 'request_id': 'req-a3df8988-ec1a-4f62-a17c-80ab85cc87f1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.114634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a3df8988-ec1a-4f62-a17c-80ab85cc87f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '473ca0d8e9334ff2a0463c8c1e3020b6', 'request_id': 'req-a3df8988-ec1a-4f62-a17c-80ab85cc87f1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.121248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a3df8988-ec1a-4f62-a17c-80ab85cc87f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '473ca0d8e9334ff2a0463c8c1e3020b6', 'request_id': 'req-a3df8988-ec1a-4f62-a17c-80ab85cc87f1'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.138022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a3df8988-ec1a-4f62-a17c-80ab85cc87f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] The token cache is being invalidate because project 473ca0d8e9334ff2a0463c8c1e3020b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:53.138503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a3df8988-ec1a-4f62-a17c-80ab85cc87f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.139388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a3df8988-ec1a-4f62-a17c-80ab85cc87f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.141636 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4218/8370] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:53 2022] DELETE /identity/v3/projects/473ca0d8e9334ff2a0463c8c1e3020b6 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:53.146346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38024c-de22-4b5a-a02e-f9bbae5d65fd tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:53.148399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8f38024c-de22-4b5a-a02e-f9bbae5d65fd tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:53.169573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38024c-de22-4b5a-a02e-f9bbae5d65fd tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:53.234808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958b727-d953-4bb1-b0fd-50583d47b391 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:53.235606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958b727-d953-4bb1-b0fd-50583d47b391 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:53.235992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958b727-d953-4bb1-b0fd-50583d47b391 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:53.236396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958b727-d953-4bb1-b0fd-50583d47b391 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:53.237181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958b727-d953-4bb1-b0fd-50583d47b391 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:53.238995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958b727-d953-4bb1-b0fd-50583d47b391 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:53.261100 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4153/8371] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:25:53 2022] POST /identity/v3/domains => generated 276 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:25:53.267652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df8988-ec1a-4f62-a17c-80ab85cc87f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:53.280134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df8988-ec1a-4f62-a17c-80ab85cc87f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:53.345587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a405db-ed45-4e68-91c4-b129fda77106 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:53.346344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a405db-ed45-4e68-91c4-b129fda77106 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:53.346723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a405db-ed45-4e68-91c4-b129fda77106 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:53.347067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a405db-ed45-4e68-91c4-b129fda77106 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:53.347774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a405db-ed45-4e68-91c4-b129fda77106 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:53.349811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a405db-ed45-4e68-91c4-b129fda77106 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:53.394855 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4219/8372] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:25:53 2022] POST /identity/v3/projects => generated 404 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:53.398825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958b727-d953-4bb1-b0fd-50583d47b391 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:53.410194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958b727-d953-4bb1-b0fd-50583d47b391 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:53.434721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:53.483359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59179d-9dc5-4018-9232-f0e42068015b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:53.484162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59179d-9dc5-4018-9232-f0e42068015b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:53.484443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59179d-9dc5-4018-9232-f0e42068015b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:53.484722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59179d-9dc5-4018-9232-f0e42068015b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/a66409a4a8ad4edeb7168d3a1d30530b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:53.486071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59179d-9dc5-4018-9232-f0e42068015b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:get_project(project_id=a66409a4a8ad4edeb7168d3a1d30530b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:53.488172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59179d-9dc5-4018-9232-f0e42068015b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:53.492965 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4154/8373] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:25:53 2022] GET /identity/v3/projects/a66409a4a8ad4edeb7168d3a1d30530b => generated 404 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:53.494251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a405db-ed45-4e68-91c4-b129fda77106 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:53.506647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a405db-ed45-4e68-91c4-b129fda77106 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:53.513821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad153d-591e-4265-b17d-67098e02321c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:53.514637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad153d-591e-4265-b17d-67098e02321c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:53.514996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad153d-591e-4265-b17d-67098e02321c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:53.515334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad153d-591e-4265-b17d-67098e02321c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/a66409a4a8ad4edeb7168d3a1d30530b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:53.516552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad153d-591e-4265-b17d-67098e02321c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a66409a4a8ad4edeb7168d3a1d30530b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:53.518688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad153d-591e-4265-b17d-67098e02321c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:53.525914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:53.575743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-39ad153d-591e-4265-b17d-67098e02321c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a66409a4a8ad4edeb7168d3a1d30530b', 'request_id': 'req-39ad153d-591e-4265-b17d-67098e02321c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.582043 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-39ad153d-591e-4265-b17d-67098e02321c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a66409a4a8ad4edeb7168d3a1d30530b', 'request_id': 'req-39ad153d-591e-4265-b17d-67098e02321c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.588980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-39ad153d-591e-4265-b17d-67098e02321c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a66409a4a8ad4edeb7168d3a1d30530b', 'request_id': 'req-39ad153d-591e-4265-b17d-67098e02321c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.603883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-39ad153d-591e-4265-b17d-67098e02321c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] The token cache is being invalidate because project a66409a4a8ad4edeb7168d3a1d30530b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:53.604276 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-39ad153d-591e-4265-b17d-67098e02321c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.605536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-39ad153d-591e-4265-b17d-67098e02321c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.610527 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4220/8374] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:53 2022] DELETE /identity/v3/projects/a66409a4a8ad4edeb7168d3a1d30530b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:53.611639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59179d-9dc5-4018-9232-f0e42068015b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:53.616309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2e59179d-9dc5-4018-9232-f0e42068015b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:53.632229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59179d-9dc5-4018-9232-f0e42068015b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:53.710549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f00e63-ac5f-425c-b008-dfd2526084f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:53.711422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f00e63-ac5f-425c-b008-dfd2526084f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:53.711715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f00e63-ac5f-425c-b008-dfd2526084f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:53.712933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f00e63-ac5f-425c-b008-dfd2526084f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/domains/7df12450316044b697fa9b89ea9d998f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:53.714845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f00e63-ac5f-425c-b008-dfd2526084f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=7df12450316044b697fa9b89ea9d998f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:53.716632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f00e63-ac5f-425c-b008-dfd2526084f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:53.741930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-53f00e63-ac5f-425c-b008-dfd2526084f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '7df12450316044b697fa9b89ea9d998f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.742755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-53f00e63-ac5f-425c-b008-dfd2526084f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '7df12450316044b697fa9b89ea9d998f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.771681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-53f00e63-ac5f-425c-b008-dfd2526084f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '7df12450316044b697fa9b89ea9d998f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.772327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-53f00e63-ac5f-425c-b008-dfd2526084f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '7df12450316044b697fa9b89ea9d998f'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.778318 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4155/8375] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:25:53 2022] PATCH /identity/v3/domains/7df12450316044b697fa9b89ea9d998f => generated 277 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:53.780604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad153d-591e-4265-b17d-67098e02321c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:53.794006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad153d-591e-4265-b17d-67098e02321c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:53.855715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d874e-1744-4fcc-84f9-946e919a5275 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:53.856361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d874e-1744-4fcc-84f9-946e919a5275 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:53.856650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d874e-1744-4fcc-84f9-946e919a5275 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:53.856931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d874e-1744-4fcc-84f9-946e919a5275 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/domains/7df12450316044b697fa9b89ea9d998f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:53.868867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d874e-1744-4fcc-84f9-946e919a5275 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=7df12450316044b697fa9b89ea9d998f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:53.871130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d874e-1744-4fcc-84f9-946e919a5275 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:53.896093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-153d874e-1744-4fcc-84f9-946e919a5275 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '7df12450316044b697fa9b89ea9d998f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.899603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-153d874e-1744-4fcc-84f9-946e919a5275 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '7df12450316044b697fa9b89ea9d998f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.949725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-153d874e-1744-4fcc-84f9-946e919a5275 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7df12450316044b697fa9b89ea9d998f', 'request_id': 'req-153d874e-1744-4fcc-84f9-946e919a5275'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.957733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-153d874e-1744-4fcc-84f9-946e919a5275 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7df12450316044b697fa9b89ea9d998f', 'request_id': 'req-153d874e-1744-4fcc-84f9-946e919a5275'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.963545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-153d874e-1744-4fcc-84f9-946e919a5275 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7df12450316044b697fa9b89ea9d998f', 'request_id': 'req-153d874e-1744-4fcc-84f9-946e919a5275'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.965180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:53.977696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-153d874e-1744-4fcc-84f9-946e919a5275 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] The token cache is being invalidate because project 7df12450316044b697fa9b89ea9d998f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:53.977989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-153d874e-1744-4fcc-84f9-946e919a5275 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.978664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-153d874e-1744-4fcc-84f9-946e919a5275 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.986107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-153d874e-1744-4fcc-84f9-946e919a5275 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '7df12450316044b697fa9b89ea9d998f', 'request_id': 'req-153d874e-1744-4fcc-84f9-946e919a5275'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.986839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-153d874e-1744-4fcc-84f9-946e919a5275 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '7df12450316044b697fa9b89ea9d998f', 'request_id': 'req-153d874e-1744-4fcc-84f9-946e919a5275'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.987505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-153d874e-1744-4fcc-84f9-946e919a5275 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '7df12450316044b697fa9b89ea9d998f', 'request_id': 'req-153d874e-1744-4fcc-84f9-946e919a5275'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:53.997142 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4221/8376] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:25:53 2022] DELETE /identity/v3/domains/7df12450316044b697fa9b89ea9d998f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:54.003878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f00e63-ac5f-425c-b008-dfd2526084f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:54.005490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-53f00e63-ac5f-425c-b008-dfd2526084f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:54.019377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f00e63-ac5f-425c-b008-dfd2526084f1 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:54.090151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a595bcb-f499-400c-a553-af17aed40dd6 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:54.091517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a595bcb-f499-400c-a553-af17aed40dd6 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:54.091517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a595bcb-f499-400c-a553-af17aed40dd6 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:54.091819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a595bcb-f499-400c-a553-af17aed40dd6 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:54.092569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a595bcb-f499-400c-a553-af17aed40dd6 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:54.094372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a595bcb-f499-400c-a553-af17aed40dd6 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:54.110936 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4156/8377] 10.222.0.17 () {60 vars in 1149 bytes} [Sat Oct 1 21:25:54 2022] POST /identity/v3/domains => generated 276 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:25:54.115109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d874e-1744-4fcc-84f9-946e919a5275 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:54.116758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-153d874e-1744-4fcc-84f9-946e919a5275 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:54.130979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d874e-1744-4fcc-84f9-946e919a5275 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:54.191789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ca389-d7ac-449f-abc5-cf132cb435c5 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:54.192406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ca389-d7ac-449f-abc5-cf132cb435c5 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:54.192694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ca389-d7ac-449f-abc5-cf132cb435c5 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:54.192971 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ca389-d7ac-449f-abc5-cf132cb435c5 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:54.193595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ca389-d7ac-449f-abc5-cf132cb435c5 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:54.195624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ca389-d7ac-449f-abc5-cf132cb435c5 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:54.234931 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4222/8378] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:25:54 2022] POST /identity/v3/projects => generated 408 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:54.239002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a595bcb-f499-400c-a553-af17aed40dd6 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:54.250726 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a595bcb-f499-400c-a553-af17aed40dd6 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:54.278729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:54.322352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc324b-dc62-4bce-9f74-7141bcda6ad6 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:54.323000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc324b-dc62-4bce-9f74-7141bcda6ad6 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:54.323328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc324b-dc62-4bce-9f74-7141bcda6ad6 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:54.323607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc324b-dc62-4bce-9f74-7141bcda6ad6 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:54.324245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc324b-dc62-4bce-9f74-7141bcda6ad6 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:54.326249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc324b-dc62-4bce-9f74-7141bcda6ad6 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:54.360231 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4157/8379] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:25:54 2022] POST /identity/v3/projects => generated 404 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:54.364529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ca389-d7ac-449f-abc5-cf132cb435c5 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:54.375745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ca389-d7ac-449f-abc5-cf132cb435c5 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:54.432136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4f879-1a1c-49f3-8379-aa3d8e22c33d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:54.433365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4f879-1a1c-49f3-8379-aa3d8e22c33d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:54.433676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4f879-1a1c-49f3-8379-aa3d8e22c33d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:54.433950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4f879-1a1c-49f3-8379-aa3d8e22c33d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/771b021dbe52491e924c5c31348075a4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:54.435121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4f879-1a1c-49f3-8379-aa3d8e22c33d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:delete_project(project_id=771b021dbe52491e924c5c31348075a4)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:54.437002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4f879-1a1c-49f3-8379-aa3d8e22c33d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:54.487667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d5a4f879-1a1c-49f3-8379-aa3d8e22c33d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '771b021dbe52491e924c5c31348075a4', 'request_id': 'req-d5a4f879-1a1c-49f3-8379-aa3d8e22c33d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.488780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=2,keystone:DELETE=9,keystone:SELECT=89 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:54.496494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d5a4f879-1a1c-49f3-8379-aa3d8e22c33d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '771b021dbe52491e924c5c31348075a4', 'request_id': 'req-d5a4f879-1a1c-49f3-8379-aa3d8e22c33d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.503111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d5a4f879-1a1c-49f3-8379-aa3d8e22c33d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '771b021dbe52491e924c5c31348075a4', 'request_id': 'req-d5a4f879-1a1c-49f3-8379-aa3d8e22c33d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.515245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d5a4f879-1a1c-49f3-8379-aa3d8e22c33d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] The token cache is being invalidate because project 771b021dbe52491e924c5c31348075a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:54.515537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d5a4f879-1a1c-49f3-8379-aa3d8e22c33d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.516237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-d5a4f879-1a1c-49f3-8379-aa3d8e22c33d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.517736 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4223/8380] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:54 2022] DELETE /identity/v3/projects/771b021dbe52491e924c5c31348075a4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:54.522144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc324b-dc62-4bce-9f74-7141bcda6ad6 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:54.523880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d2dc324b-dc62-4bce-9f74-7141bcda6ad6 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:54.536619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc324b-dc62-4bce-9f74-7141bcda6ad6 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:54.593594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec061e-ac80-4222-ab66-35bcc778fc43 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:54.594225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec061e-ac80-4222-ab66-35bcc778fc43 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:54.594531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec061e-ac80-4222-ab66-35bcc778fc43 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:54.594811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec061e-ac80-4222-ab66-35bcc778fc43 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/34275594dcc746658342907a2edadc07` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:54.595934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec061e-ac80-4222-ab66-35bcc778fc43 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:delete_project(project_id=34275594dcc746658342907a2edadc07)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:54.597835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec061e-ac80-4222-ab66-35bcc778fc43 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:54.643584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b1ec061e-ac80-4222-ab66-35bcc778fc43 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34275594dcc746658342907a2edadc07', 'request_id': 'req-b1ec061e-ac80-4222-ab66-35bcc778fc43'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.650923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b1ec061e-ac80-4222-ab66-35bcc778fc43 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34275594dcc746658342907a2edadc07', 'request_id': 'req-b1ec061e-ac80-4222-ab66-35bcc778fc43'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.657088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b1ec061e-ac80-4222-ab66-35bcc778fc43 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34275594dcc746658342907a2edadc07', 'request_id': 'req-b1ec061e-ac80-4222-ab66-35bcc778fc43'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.668827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b1ec061e-ac80-4222-ab66-35bcc778fc43 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] The token cache is being invalidate because project 34275594dcc746658342907a2edadc07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:54.669117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b1ec061e-ac80-4222-ab66-35bcc778fc43 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.669763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b1ec061e-ac80-4222-ab66-35bcc778fc43 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.671293 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4158/8381] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:54 2022] DELETE /identity/v3/projects/34275594dcc746658342907a2edadc07 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:25:54.675423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4f879-1a1c-49f3-8379-aa3d8e22c33d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:54.685403 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4f879-1a1c-49f3-8379-aa3d8e22c33d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:54.740970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ae13a-a1e0-499d-a2f4-b17ebee3c673 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:54.741599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ae13a-a1e0-499d-a2f4-b17ebee3c673 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:54.741911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ae13a-a1e0-499d-a2f4-b17ebee3c673 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:54.742190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ae13a-a1e0-499d-a2f4-b17ebee3c673 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/domains/b8993d6332d04eaa8dfba2535e9801f4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:54.743967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ae13a-a1e0-499d-a2f4-b17ebee3c673 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=b8993d6332d04eaa8dfba2535e9801f4)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:54.745613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ae13a-a1e0-499d-a2f4-b17ebee3c673 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:54.766706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ff5ae13a-a1e0-499d-a2f4-b17ebee3c673 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'b8993d6332d04eaa8dfba2535e9801f4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.767423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ff5ae13a-a1e0-499d-a2f4-b17ebee3c673 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'b8993d6332d04eaa8dfba2535e9801f4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.791822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ff5ae13a-a1e0-499d-a2f4-b17ebee3c673 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'b8993d6332d04eaa8dfba2535e9801f4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.793079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-ff5ae13a-a1e0-499d-a2f4-b17ebee3c673 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'b8993d6332d04eaa8dfba2535e9801f4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.795457 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4224/8382] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:25:54 2022] PATCH /identity/v3/domains/b8993d6332d04eaa8dfba2535e9801f4 => generated 277 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:54.799566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec061e-ac80-4222-ab66-35bcc778fc43 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:54.800961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b1ec061e-ac80-4222-ab66-35bcc778fc43 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:54.813415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec061e-ac80-4222-ab66-35bcc778fc43 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:54.848914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:54.875115 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019911c-b6f1-493c-ab35-518a08699d8c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:54.875755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019911c-b6f1-493c-ab35-518a08699d8c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:54.876043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019911c-b6f1-493c-ab35-518a08699d8c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:54.876323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019911c-b6f1-493c-ab35-518a08699d8c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/domains/b8993d6332d04eaa8dfba2535e9801f4` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:54.887172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019911c-b6f1-493c-ab35-518a08699d8c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=b8993d6332d04eaa8dfba2535e9801f4)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:54.888795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019911c-b6f1-493c-ab35-518a08699d8c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:54.911168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8019911c-b6f1-493c-ab35-518a08699d8c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'b8993d6332d04eaa8dfba2535e9801f4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.921583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8019911c-b6f1-493c-ab35-518a08699d8c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'b8993d6332d04eaa8dfba2535e9801f4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.961120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8019911c-b6f1-493c-ab35-518a08699d8c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8993d6332d04eaa8dfba2535e9801f4', 'request_id': 'req-8019911c-b6f1-493c-ab35-518a08699d8c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.969221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8019911c-b6f1-493c-ab35-518a08699d8c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8993d6332d04eaa8dfba2535e9801f4', 'request_id': 'req-8019911c-b6f1-493c-ab35-518a08699d8c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.974975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8019911c-b6f1-493c-ab35-518a08699d8c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8993d6332d04eaa8dfba2535e9801f4', 'request_id': 'req-8019911c-b6f1-493c-ab35-518a08699d8c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.986413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8019911c-b6f1-493c-ab35-518a08699d8c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] The token cache is being invalidate because project b8993d6332d04eaa8dfba2535e9801f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:54.986744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8019911c-b6f1-493c-ab35-518a08699d8c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.987407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8019911c-b6f1-493c-ab35-518a08699d8c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.994750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8019911c-b6f1-493c-ab35-518a08699d8c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'b8993d6332d04eaa8dfba2535e9801f4', 'request_id': 'req-8019911c-b6f1-493c-ab35-518a08699d8c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.995388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8019911c-b6f1-493c-ab35-518a08699d8c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'b8993d6332d04eaa8dfba2535e9801f4', 'request_id': 'req-8019911c-b6f1-493c-ab35-518a08699d8c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:54.998632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-8019911c-b6f1-493c-ab35-518a08699d8c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'b8993d6332d04eaa8dfba2535e9801f4', 'request_id': 'req-8019911c-b6f1-493c-ab35-518a08699d8c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:55.004101 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4159/8383] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:25:54 2022] DELETE /identity/v3/domains/b8993d6332d04eaa8dfba2535e9801f4 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:55.009305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ae13a-a1e0-499d-a2f4-b17ebee3c673 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:55.011242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ff5ae13a-a1e0-499d-a2f4-b17ebee3c673 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:55.023646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ae13a-a1e0-499d-a2f4-b17ebee3c673 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:55.080921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2225525-202f-482e-805b-51a9b1f099ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:55.081537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2225525-202f-482e-805b-51a9b1f099ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:55.081827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2225525-202f-482e-805b-51a9b1f099ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:55.082108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2225525-202f-482e-805b-51a9b1f099ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:55.082792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2225525-202f-482e-805b-51a9b1f099ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:55.084648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2225525-202f-482e-805b-51a9b1f099ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:55.119280 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4225/8384] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:25:55 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:55.123411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019911c-b6f1-493c-ab35-518a08699d8c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:55.124876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8019911c-b6f1-493c-ab35-518a08699d8c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:55.139499 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019911c-b6f1-493c-ab35-518a08699d8c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:55.196350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be3b25-67fa-41b6-b302-52e35242c56e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:55.196934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be3b25-67fa-41b6-b302-52e35242c56e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:55.197218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be3b25-67fa-41b6-b302-52e35242c56e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:55.197492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be3b25-67fa-41b6-b302-52e35242c56e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/2342c66cb782427f875fd999aa94be27` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:55.199143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be3b25-67fa-41b6-b302-52e35242c56e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:get_project(project_id=2342c66cb782427f875fd999aa94be27)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:55.201231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be3b25-67fa-41b6-b302-52e35242c56e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:55.202911 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4160/8385] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:25:55 2022] GET /identity/v3/projects/2342c66cb782427f875fd999aa94be27 => generated 377 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:25:55.207222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2225525-202f-482e-805b-51a9b1f099ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:55.218738 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2225525-202f-482e-805b-51a9b1f099ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:55.223368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30f9ec-3d62-423d-9db5-565e5ad9b475 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:55.223959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30f9ec-3d62-423d-9db5-565e5ad9b475 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:55.224241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30f9ec-3d62-423d-9db5-565e5ad9b475 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:55.224518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30f9ec-3d62-423d-9db5-565e5ad9b475 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:55.225369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30f9ec-3d62-423d-9db5-565e5ad9b475 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:55.227418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30f9ec-3d62-423d-9db5-565e5ad9b475 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:55.244916 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4226/8386] 10.222.0.17 () {58 vars in 1131 bytes} [Sat Oct 1 21:25:55 2022] GET /identity/v3/projects => generated 4733 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:25:55.249881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be3b25-67fa-41b6-b302-52e35242c56e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:55.261370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be3b25-67fa-41b6-b302-52e35242c56e tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:55.266535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bfd4c8-3cc0-4d78-a220-ecd556064232 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:55.267165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfd4c8-3cc0-4d78-a220-ecd556064232 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:55.267454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfd4c8-3cc0-4d78-a220-ecd556064232 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:55.267734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfd4c8-3cc0-4d78-a220-ecd556064232 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/2342c66cb782427f875fd999aa94be27` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:55.268867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bfd4c8-3cc0-4d78-a220-ecd556064232 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2342c66cb782427f875fd999aa94be27)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:55.270847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bfd4c8-3cc0-4d78-a220-ecd556064232 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:55.281796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:55.331762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-99bfd4c8-3cc0-4d78-a220-ecd556064232 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2342c66cb782427f875fd999aa94be27', 'request_id': 'req-99bfd4c8-3cc0-4d78-a220-ecd556064232'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:55.339576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-99bfd4c8-3cc0-4d78-a220-ecd556064232 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2342c66cb782427f875fd999aa94be27', 'request_id': 'req-99bfd4c8-3cc0-4d78-a220-ecd556064232'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:55.345240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-99bfd4c8-3cc0-4d78-a220-ecd556064232 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2342c66cb782427f875fd999aa94be27', 'request_id': 'req-99bfd4c8-3cc0-4d78-a220-ecd556064232'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:55.357598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-99bfd4c8-3cc0-4d78-a220-ecd556064232 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] The token cache is being invalidate because project 2342c66cb782427f875fd999aa94be27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:55.357889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-99bfd4c8-3cc0-4d78-a220-ecd556064232 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:55.358546 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-99bfd4c8-3cc0-4d78-a220-ecd556064232 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:55.360041 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4161/8387] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:55 2022] DELETE /identity/v3/projects/2342c66cb782427f875fd999aa94be27 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:55.365083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30f9ec-3d62-423d-9db5-565e5ad9b475 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:55.367016 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4b30f9ec-3d62-423d-9db5-565e5ad9b475 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:55.379983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30f9ec-3d62-423d-9db5-565e5ad9b475 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:55.389048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=3,keystone:UPDATE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:55.439498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373b575-07ac-43f0-baf6-a639b77d9a01 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:55.440135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373b575-07ac-43f0-baf6-a639b77d9a01 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:55.440419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373b575-07ac-43f0-baf6-a639b77d9a01 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:55.440695 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373b575-07ac-43f0-baf6-a639b77d9a01 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:55.441306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373b575-07ac-43f0-baf6-a639b77d9a01 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:55.443262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373b575-07ac-43f0-baf6-a639b77d9a01 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:55.474123 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4227/8388] 10.222.0.17 () {60 vars in 1152 bytes} [Sat Oct 1 21:25:55 2022] POST /identity/v3/projects => generated 347 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:55.478263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bfd4c8-3cc0-4d78-a220-ecd556064232 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:55.489015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bfd4c8-3cc0-4d78-a220-ecd556064232 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:55.544658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d065cc-667d-4727-9f3c-f16b9da6f401 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:55.545280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d065cc-667d-4727-9f3c-f16b9da6f401 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:55.545561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d065cc-667d-4727-9f3c-f16b9da6f401 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:55.545834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d065cc-667d-4727-9f3c-f16b9da6f401 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/95e50daf7585486089eb0e3400eb57b5` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:55.547047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d065cc-667d-4727-9f3c-f16b9da6f401 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:update_project(project_id=95e50daf7585486089eb0e3400eb57b5)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:55.548058 ncdv-master-1 devstack@keystone.service[99140]: WARNING py.warnings [None req-e9d065cc-667d-4727-9f3c-f16b9da6f401 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 01 21:25:55.548058 ncdv-master-1 devstack@keystone.service[99140]: warnings.warn(msg) Oct 01 21:25:55.548058 ncdv-master-1 devstack@keystone.service[99140]: Oct 01 21:25:55.549258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d065cc-667d-4727-9f3c-f16b9da6f401 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:55.581422 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4162/8389] 10.222.0.17 () {60 vars in 1252 bytes} [Sat Oct 1 21:25:55 2022] PATCH /identity/v3/projects/95e50daf7585486089eb0e3400eb57b5 => generated 361 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:55.585574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373b575-07ac-43f0-baf6-a639b77d9a01 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:55.596717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373b575-07ac-43f0-baf6-a639b77d9a01 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:55.601389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdeef3b-5a9f-4ec6-87bb-2787c7af1851 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:55.602008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdeef3b-5a9f-4ec6-87bb-2787c7af1851 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:55.602304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdeef3b-5a9f-4ec6-87bb-2787c7af1851 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:55.602621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdeef3b-5a9f-4ec6-87bb-2787c7af1851 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/95e50daf7585486089eb0e3400eb57b5` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:55.614520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdeef3b-5a9f-4ec6-87bb-2787c7af1851 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:get_project(project_id=95e50daf7585486089eb0e3400eb57b5)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:55.616674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdeef3b-5a9f-4ec6-87bb-2787c7af1851 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:55.618278 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4228/8390] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:25:55 2022] GET /identity/v3/projects/95e50daf7585486089eb0e3400eb57b5 => generated 348 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:55.622277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d065cc-667d-4727-9f3c-f16b9da6f401 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:55.632117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d065cc-667d-4727-9f3c-f16b9da6f401 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:55.637051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f250674-268b-4445-8e25-0a12c7bc6775 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:55.637677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f250674-268b-4445-8e25-0a12c7bc6775 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:55.637963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f250674-268b-4445-8e25-0a12c7bc6775 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:55.638368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f250674-268b-4445-8e25-0a12c7bc6775 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/95e50daf7585486089eb0e3400eb57b5` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:55.639530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f250674-268b-4445-8e25-0a12c7bc6775 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:delete_project(project_id=95e50daf7585486089eb0e3400eb57b5)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:55.641440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f250674-268b-4445-8e25-0a12c7bc6775 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:55.689700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1f250674-268b-4445-8e25-0a12c7bc6775 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95e50daf7585486089eb0e3400eb57b5', 'request_id': 'req-1f250674-268b-4445-8e25-0a12c7bc6775'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:55.697123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1f250674-268b-4445-8e25-0a12c7bc6775 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95e50daf7585486089eb0e3400eb57b5', 'request_id': 'req-1f250674-268b-4445-8e25-0a12c7bc6775'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:55.703424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1f250674-268b-4445-8e25-0a12c7bc6775 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95e50daf7585486089eb0e3400eb57b5', 'request_id': 'req-1f250674-268b-4445-8e25-0a12c7bc6775'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:55.710460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:55.716100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1f250674-268b-4445-8e25-0a12c7bc6775 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] The token cache is being invalidate because project 95e50daf7585486089eb0e3400eb57b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:55.716457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1f250674-268b-4445-8e25-0a12c7bc6775 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:55.717397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-1f250674-268b-4445-8e25-0a12c7bc6775 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:55.719322 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4163/8391] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:55 2022] DELETE /identity/v3/projects/95e50daf7585486089eb0e3400eb57b5 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:55.724431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdeef3b-5a9f-4ec6-87bb-2787c7af1851 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:55.726295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5bdeef3b-5a9f-4ec6-87bb-2787c7af1851 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:55.739170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdeef3b-5a9f-4ec6-87bb-2787c7af1851 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:55.797101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0630ec-ef48-4bae-b590-4c26d7733aae tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:55.797740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0630ec-ef48-4bae-b590-4c26d7733aae tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:55.797966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0630ec-ef48-4bae-b590-4c26d7733aae tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:55.798242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0630ec-ef48-4bae-b590-4c26d7733aae tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:55.798909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0630ec-ef48-4bae-b590-4c26d7733aae tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:55.800792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0630ec-ef48-4bae-b590-4c26d7733aae tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:55.831069 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4229/8392] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:25:55 2022] POST /identity/v3/projects => generated 360 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:25:55.835199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f250674-268b-4445-8e25-0a12c7bc6775 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:55.846138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f250674-268b-4445-8e25-0a12c7bc6775 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:55.900907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db30f87-fa20-4f0d-bb73-c86837a9815a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:55.902141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db30f87-fa20-4f0d-bb73-c86837a9815a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:55.902430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db30f87-fa20-4f0d-bb73-c86837a9815a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:55.902734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db30f87-fa20-4f0d-bb73-c86837a9815a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/026d39397f8f496daa27205115587607` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:55.903938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db30f87-fa20-4f0d-bb73-c86837a9815a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:update_project(project_id=026d39397f8f496daa27205115587607)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:55.905841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db30f87-fa20-4f0d-bb73-c86837a9815a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:55.953757 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4164/8393] 10.222.0.17 () {60 vars in 1252 bytes} [Sat Oct 1 21:25:55 2022] PATCH /identity/v3/projects/026d39397f8f496daa27205115587607 => generated 372 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:55.958167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0630ec-ef48-4bae-b590-4c26d7733aae tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:55.969728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0630ec-ef48-4bae-b590-4c26d7733aae tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:55.974578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fc481-6404-4bfe-8457-d1271e899288 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:55.975147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fc481-6404-4bfe-8457-d1271e899288 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:55.975437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fc481-6404-4bfe-8457-d1271e899288 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:55.975718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fc481-6404-4bfe-8457-d1271e899288 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/026d39397f8f496daa27205115587607` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:55.989173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8fc481-6404-4bfe-8457-d1271e899288 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:get_project(project_id=026d39397f8f496daa27205115587607)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:55.991410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8fc481-6404-4bfe-8457-d1271e899288 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:55.993109 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4230/8394] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:25:55 2022] GET /identity/v3/projects/026d39397f8f496daa27205115587607 => generated 359 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:25:55.997384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db30f87-fa20-4f0d-bb73-c86837a9815a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:56.007443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db30f87-fa20-4f0d-bb73-c86837a9815a tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:56.012796 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b0141-6925-444a-b972-badb94a1da8d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:56.013422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b0141-6925-444a-b972-badb94a1da8d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:56.013705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b0141-6925-444a-b972-badb94a1da8d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:56.013981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b0141-6925-444a-b972-badb94a1da8d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/026d39397f8f496daa27205115587607` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:56.015136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b0141-6925-444a-b972-badb94a1da8d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:delete_project(project_id=026d39397f8f496daa27205115587607)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:56.017034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b0141-6925-444a-b972-badb94a1da8d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:56.064423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ae0b0141-6925-444a-b972-badb94a1da8d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '026d39397f8f496daa27205115587607', 'request_id': 'req-ae0b0141-6925-444a-b972-badb94a1da8d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:56.071748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ae0b0141-6925-444a-b972-badb94a1da8d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '026d39397f8f496daa27205115587607', 'request_id': 'req-ae0b0141-6925-444a-b972-badb94a1da8d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:56.079211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ae0b0141-6925-444a-b972-badb94a1da8d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '026d39397f8f496daa27205115587607', 'request_id': 'req-ae0b0141-6925-444a-b972-badb94a1da8d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:56.091388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ae0b0141-6925-444a-b972-badb94a1da8d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] The token cache is being invalidate because project 026d39397f8f496daa27205115587607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:56.091719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ae0b0141-6925-444a-b972-badb94a1da8d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:56.092568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ae0b0141-6925-444a-b972-badb94a1da8d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:56.094257 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4165/8395] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:55 2022] DELETE /identity/v3/projects/026d39397f8f496daa27205115587607 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:25:56.099497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fc481-6404-4bfe-8457-d1271e899288 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:56.101504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1d8fc481-6404-4bfe-8457-d1271e899288 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:56.114603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fc481-6404-4bfe-8457-d1271e899288 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:56.167440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:56.174523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55264ca8-6bfc-4e21-9ca7-bd1f9d1757ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:56.175141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55264ca8-6bfc-4e21-9ca7-bd1f9d1757ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:56.175430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55264ca8-6bfc-4e21-9ca7-bd1f9d1757ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:56.175742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55264ca8-6bfc-4e21-9ca7-bd1f9d1757ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:56.176375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55264ca8-6bfc-4e21-9ca7-bd1f9d1757ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:56.178292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55264ca8-6bfc-4e21-9ca7-bd1f9d1757ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:56.209067 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4231/8396] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:25:56 2022] POST /identity/v3/projects => generated 354 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:56.213227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b0141-6925-444a-b972-badb94a1da8d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:56.223532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b0141-6925-444a-b972-badb94a1da8d tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:56.265327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:UPDATE=1,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:56.283015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05aece8-687f-451f-bbe9-a328f62c595c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:56.283644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05aece8-687f-451f-bbe9-a328f62c595c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:56.283926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05aece8-687f-451f-bbe9-a328f62c595c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:56.284215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05aece8-687f-451f-bbe9-a328f62c595c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/1a0201a4b0934d15ac0a405443b2204d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:56.285471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05aece8-687f-451f-bbe9-a328f62c595c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:update_project(project_id=1a0201a4b0934d15ac0a405443b2204d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:56.287429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05aece8-687f-451f-bbe9-a328f62c595c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:56.320137 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4166/8397] 10.222.0.17 () {60 vars in 1252 bytes} [Sat Oct 1 21:25:56 2022] PATCH /identity/v3/projects/1a0201a4b0934d15ac0a405443b2204d => generated 368 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:56.325036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55264ca8-6bfc-4e21-9ca7-bd1f9d1757ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:56.335991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55264ca8-6bfc-4e21-9ca7-bd1f9d1757ba tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:56.340889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8217e9e-ab99-4609-9f51-69914dd78f9b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:56.341548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8217e9e-ab99-4609-9f51-69914dd78f9b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:56.341829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8217e9e-ab99-4609-9f51-69914dd78f9b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:56.342185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8217e9e-ab99-4609-9f51-69914dd78f9b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/1a0201a4b0934d15ac0a405443b2204d` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:56.354182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8217e9e-ab99-4609-9f51-69914dd78f9b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:get_project(project_id=1a0201a4b0934d15ac0a405443b2204d)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:56.356475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8217e9e-ab99-4609-9f51-69914dd78f9b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:56.358154 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4232/8398] 10.222.0.17 () {58 vars in 1230 bytes} [Sat Oct 1 21:25:56 2022] GET /identity/v3/projects/1a0201a4b0934d15ac0a405443b2204d => generated 355 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:56.362263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05aece8-687f-451f-bbe9-a328f62c595c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:56.372832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05aece8-687f-451f-bbe9-a328f62c595c tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:56.377433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45688a47-b3f5-4040-82dd-d75ddf673d06 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:56.378054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45688a47-b3f5-4040-82dd-d75ddf673d06 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:56.378341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45688a47-b3f5-4040-82dd-d75ddf673d06 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:56.378657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45688a47-b3f5-4040-82dd-d75ddf673d06 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/projects/1a0201a4b0934d15ac0a405443b2204d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:56.379773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45688a47-b3f5-4040-82dd-d75ddf673d06 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1a0201a4b0934d15ac0a405443b2204d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:56.381676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45688a47-b3f5-4040-82dd-d75ddf673d06 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:56.428650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-45688a47-b3f5-4040-82dd-d75ddf673d06 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a0201a4b0934d15ac0a405443b2204d', 'request_id': 'req-45688a47-b3f5-4040-82dd-d75ddf673d06'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:56.436379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-45688a47-b3f5-4040-82dd-d75ddf673d06 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a0201a4b0934d15ac0a405443b2204d', 'request_id': 'req-45688a47-b3f5-4040-82dd-d75ddf673d06'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:56.441917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-45688a47-b3f5-4040-82dd-d75ddf673d06 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a0201a4b0934d15ac0a405443b2204d', 'request_id': 'req-45688a47-b3f5-4040-82dd-d75ddf673d06'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:56.452731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-45688a47-b3f5-4040-82dd-d75ddf673d06 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] The token cache is being invalidate because project 1a0201a4b0934d15ac0a405443b2204d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:56.453025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-45688a47-b3f5-4040-82dd-d75ddf673d06 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:56.453668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-45688a47-b3f5-4040-82dd-d75ddf673d06 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:56.455216 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4167/8399] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:56 2022] DELETE /identity/v3/projects/1a0201a4b0934d15ac0a405443b2204d => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:56.459915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8217e9e-ab99-4609-9f51-69914dd78f9b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:56.461864 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f8217e9e-ab99-4609-9f51-69914dd78f9b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:56.475700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8217e9e-ab99-4609-9f51-69914dd78f9b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:56.533081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcff6bc-7490-4e58-a2b5-e812d4deb5ab tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:56.533634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcff6bc-7490-4e58-a2b5-e812d4deb5ab tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:56.533917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcff6bc-7490-4e58-a2b5-e812d4deb5ab tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:56.534192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcff6bc-7490-4e58-a2b5-e812d4deb5ab tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/domains/b8993d6332d04eaa8dfba2535e9801f4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:56.540932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcff6bc-7490-4e58-a2b5-e812d4deb5ab tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 21:25:56.546614 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application [None req-fbcff6bc-7490-4e58-a2b5-e812d4deb5ab tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Could not find domain: b8993d6332d04eaa8dfba2535e9801f4.: keystone.exception.DomainNotFound: Could not find domain: b8993d6332d04eaa8dfba2535e9801f4. Oct 01 21:25:56.546614 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:25:56.546614 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 01 21:25:56.546614 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 01 21:25:56.546614 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 21:25:56.546614 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 21:25:56.546614 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 21:25:56.546614 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 21:25:56.546614 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: b8993d6332d04eaa8dfba2535e9801f4. Oct 01 21:25:56.546614 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:25:56.546614 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:25:56.546614 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:25:56.546614 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:25:56.546614 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:25:56.546614 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:25:56.546614 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:25:56.547713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:25:56.547713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:25:56.547713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:25:56.547713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:25:56.547713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:25:56.547713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:25:56.547713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:25:56.547713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 01 21:25:56.547713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 01 21:25:56.547713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 21:25:56.547713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 21:25:56.547713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 21:25:56.547713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 21:25:56.547713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:25:56.547713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:25:56.547713 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:25:56.548722 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:25:56.548722 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:25:56.548722 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:25:56.548722 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:25:56.548722 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application with Lock( Oct 01 21:25:56.548722 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:25:56.548722 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:25:56.548722 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:25:56.548722 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:25:56.548722 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:25:56.548722 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:25:56.548722 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:25:56.548722 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:25:56.548722 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 01 21:25:56.548722 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 21:25:56.548722 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: b8993d6332d04eaa8dfba2535e9801f4. Oct 01 21:25:56.548722 ncdv-master-1 devstack@keystone.service[99139]: ERROR keystone.server.flask.application Oct 01 21:25:56.549802 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4233/8400] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:25:56 2022] PATCH /identity/v3/domains/b8993d6332d04eaa8dfba2535e9801f4 => generated 112 bytes in 88 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:25:56.551739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45688a47-b3f5-4040-82dd-d75ddf673d06 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:56.562741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45688a47-b3f5-4040-82dd-d75ddf673d06 tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:56.567580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65693956-192e-4ecb-a06e-e07b575e393b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329cae2bea504cb2b16e44f4c9f91865', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e894082fa89418986be5db793e1e676', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:56.568205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65693956-192e-4ecb-a06e-e07b575e393b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:56.568489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65693956-192e-4ecb-a06e-e07b575e393b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:56.568767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65693956-192e-4ecb-a06e-e07b575e393b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] PATH_INFO: `/v3/domains/7df12450316044b697fa9b89ea9d998f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:56.575804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65693956-192e-4ecb-a06e-e07b575e393b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 01 21:25:56.580733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-65693956-192e-4ecb-a06e-e07b575e393b tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Could not find domain: 7df12450316044b697fa9b89ea9d998f.: keystone.exception.DomainNotFound: Could not find domain: 7df12450316044b697fa9b89ea9d998f. Oct 01 21:25:56.580733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:25:56.580733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 01 21:25:56.580733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 01 21:25:56.580733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 01 21:25:56.580733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 01 21:25:56.580733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 01 21:25:56.580733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 01 21:25:56.580733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 7df12450316044b697fa9b89ea9d998f. Oct 01 21:25:56.580733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:25:56.580733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 01 21:25:56.580733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:25:56.580733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:25:56.580733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:25:56.580733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:25:56.580733 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:25:56.581804 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:25:56.581804 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:25:56.581804 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:25:56.581804 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:25:56.581804 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:25:56.581804 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:25:56.581804 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:25:56.581804 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Oct 01 21:25:56.581804 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Oct 01 21:25:56.581804 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 01 21:25:56.581804 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 01 21:25:56.581804 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 01 21:25:56.581804 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 01 21:25:56.581804 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 01 21:25:56.581804 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 01 21:25:56.581804 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 01 21:25:56.582860 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 01 21:25:56.582860 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 01 21:25:56.582860 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.get_or_create( Oct 01 21:25:56.582860 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 01 21:25:56.582860 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application with Lock( Oct 01 21:25:56.582860 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 01 21:25:56.582860 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self._enter() Oct 01 21:25:56.582860 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 01 21:25:56.582860 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 01 21:25:56.582860 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 01 21:25:56.582860 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.creator() Oct 01 21:25:56.582860 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 01 21:25:56.582860 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application created_value = creator( Oct 01 21:25:56.582860 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 01 21:25:56.582860 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 01 21:25:56.582860 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 7df12450316044b697fa9b89ea9d998f. Oct 01 21:25:56.582860 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:25:56.583935 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4168/8401] 10.222.0.17 () {60 vars in 1249 bytes} [Sat Oct 1 21:25:56 2022] PATCH /identity/v3/domains/7df12450316044b697fa9b89ea9d998f => generated 112 bytes in 31 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:25:56.587891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcff6bc-7490-4e58-a2b5-e812d4deb5ab tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:56.589549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fbcff6bc-7490-4e58-a2b5-e812d4deb5ab tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:56.657263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcff6bc-7490-4e58-a2b5-e812d4deb5ab tempest-ProjectsTestJSON-2081223092 tempest-ProjectsTestJSON-2081223092-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:56.661970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b538da-2a08-419c-a6c2-b0e79edf98dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:56.662624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b538da-2a08-419c-a6c2-b0e79edf98dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:56.662912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b538da-2a08-419c-a6c2-b0e79edf98dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:56.663190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b538da-2a08-419c-a6c2-b0e79edf98dd admin admin] PATH_INFO: `/v3/users/8d97088b6cc74035a4e16262966501b2` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:56.664323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b538da-2a08-419c-a6c2-b0e79edf98dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d97088b6cc74035a4e16262966501b2)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:56.666581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b538da-2a08-419c-a6c2-b0e79edf98dd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:56.713196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3b538da-2a08-419c-a6c2-b0e79edf98dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d97088b6cc74035a4e16262966501b2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:56.720116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3b538da-2a08-419c-a6c2-b0e79edf98dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d97088b6cc74035a4e16262966501b2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:56.730527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:56.732992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3b538da-2a08-419c-a6c2-b0e79edf98dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d97088b6cc74035a4e16262966501b2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:56.734164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3b538da-2a08-419c-a6c2-b0e79edf98dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d97088b6cc74035a4e16262966501b2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:56.742281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e3b538da-2a08-419c-a6c2-b0e79edf98dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d97088b6cc74035a4e16262966501b2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:56.748955 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4234/8402] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:25:56 2022] DELETE /identity/v3/users/8d97088b6cc74035a4e16262966501b2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:56.754314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98567a-6ea1-4e58-9249-e268763ee4df None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:56.754795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98567a-6ea1-4e58-9249-e268763ee4df None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:56.754966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98567a-6ea1-4e58-9249-e268763ee4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:56.773114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ce98567a-6ea1-4e58-9249-e268763ee4df None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:56.774315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ce98567a-6ea1-4e58-9249-e268763ee4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:56.833020 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4169/8403] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:25:56.843080 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b538da-2a08-419c-a6c2-b0e79edf98dd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:56.844486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e3b538da-2a08-419c-a6c2-b0e79edf98dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:56.906061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b538da-2a08-419c-a6c2-b0e79edf98dd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:56.909791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763404db-1dc9-43fe-ba54-c98536ea08b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:56.910396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763404db-1dc9-43fe-ba54-c98536ea08b8 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:56.910764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763404db-1dc9-43fe-ba54-c98536ea08b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:56.911029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763404db-1dc9-43fe-ba54-c98536ea08b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:56.921116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763404db-1dc9-43fe-ba54-c98536ea08b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:56.922588 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763404db-1dc9-43fe-ba54-c98536ea08b8 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:56.932179 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4235/8404] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:25:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:25:57.206442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98567a-6ea1-4e58-9249-e268763ee4df None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:57.208045 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ce98567a-6ea1-4e58-9249-e268763ee4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:57.221288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98567a-6ea1-4e58-9249-e268763ee4df None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:57.226069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9e03f-a432-4089-a112-aa6e1168492d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:57.226743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9e03f-a432-4089-a112-aa6e1168492d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:57.227032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9e03f-a432-4089-a112-aa6e1168492d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:57.227312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9e03f-a432-4089-a112-aa6e1168492d admin admin] PATH_INFO: `/v3/projects/b0fd159c0f574d44925a6c0d020eb43f` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:57.228456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9e03f-a432-4089-a112-aa6e1168492d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0fd159c0f574d44925a6c0d020eb43f)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:57.230408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9e03f-a432-4089-a112-aa6e1168492d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:57.266361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:UPDATE=1,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:57.283413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9cd9e03f-a432-4089-a112-aa6e1168492d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0fd159c0f574d44925a6c0d020eb43f', 'request_id': 'req-9cd9e03f-a432-4089-a112-aa6e1168492d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:57.290323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9cd9e03f-a432-4089-a112-aa6e1168492d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0fd159c0f574d44925a6c0d020eb43f', 'request_id': 'req-9cd9e03f-a432-4089-a112-aa6e1168492d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:57.296232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9cd9e03f-a432-4089-a112-aa6e1168492d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0fd159c0f574d44925a6c0d020eb43f', 'request_id': 'req-9cd9e03f-a432-4089-a112-aa6e1168492d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:57.307118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9cd9e03f-a432-4089-a112-aa6e1168492d admin admin] The token cache is being invalidate because project b0fd159c0f574d44925a6c0d020eb43f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:57.307411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9cd9e03f-a432-4089-a112-aa6e1168492d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:57.308043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9cd9e03f-a432-4089-a112-aa6e1168492d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:57.309556 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4170/8405] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:57 2022] DELETE /identity/v3/projects/b0fd159c0f574d44925a6c0d020eb43f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:57.313520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763404db-1dc9-43fe-ba54-c98536ea08b8 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:57.323762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763404db-1dc9-43fe-ba54-c98536ea08b8 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:57.379906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf56b22-7c28-4f67-bb9b-f9e729dd0aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:57.380556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf56b22-7c28-4f67-bb9b-f9e729dd0aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:57.380814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf56b22-7c28-4f67-bb9b-f9e729dd0aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:57.381084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf56b22-7c28-4f67-bb9b-f9e729dd0aa5 admin admin] PATH_INFO: `/v3/users/329cae2bea504cb2b16e44f4c9f91865` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:57.382203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf56b22-7c28-4f67-bb9b-f9e729dd0aa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=329cae2bea504cb2b16e44f4c9f91865)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:57.384121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf56b22-7c28-4f67-bb9b-f9e729dd0aa5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:57.432591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6cf56b22-7c28-4f67-bb9b-f9e729dd0aa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '329cae2bea504cb2b16e44f4c9f91865'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:57.440245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6cf56b22-7c28-4f67-bb9b-f9e729dd0aa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '329cae2bea504cb2b16e44f4c9f91865'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:57.452761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6cf56b22-7c28-4f67-bb9b-f9e729dd0aa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '329cae2bea504cb2b16e44f4c9f91865'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:57.453465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6cf56b22-7c28-4f67-bb9b-f9e729dd0aa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '329cae2bea504cb2b16e44f4c9f91865'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:57.460773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-6cf56b22-7c28-4f67-bb9b-f9e729dd0aa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '329cae2bea504cb2b16e44f4c9f91865'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:57.467545 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4236/8406] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:25:57 2022] DELETE /identity/v3/users/329cae2bea504cb2b16e44f4c9f91865 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:57.740416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9e03f-a432-4089-a112-aa6e1168492d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:57.742055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9cd9e03f-a432-4089-a112-aa6e1168492d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:57.757224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9e03f-a432-4089-a112-aa6e1168492d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:57.815732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2aae2-5c65-4100-adab-4e56618f4976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:57.816359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2aae2-5c65-4100-adab-4e56618f4976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:57.816642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2aae2-5c65-4100-adab-4e56618f4976 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:57.816919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2aae2-5c65-4100-adab-4e56618f4976 admin admin] PATH_INFO: `/v3/projects/7e894082fa89418986be5db793e1e676` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:57.818039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae2aae2-5c65-4100-adab-4e56618f4976 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e894082fa89418986be5db793e1e676)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:57.820026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae2aae2-5c65-4100-adab-4e56618f4976 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:57.865031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bae2aae2-5c65-4100-adab-4e56618f4976 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e894082fa89418986be5db793e1e676', 'request_id': 'req-bae2aae2-5c65-4100-adab-4e56618f4976'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:57.872556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bae2aae2-5c65-4100-adab-4e56618f4976 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e894082fa89418986be5db793e1e676', 'request_id': 'req-bae2aae2-5c65-4100-adab-4e56618f4976'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:57.878528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bae2aae2-5c65-4100-adab-4e56618f4976 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e894082fa89418986be5db793e1e676', 'request_id': 'req-bae2aae2-5c65-4100-adab-4e56618f4976'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:57.889773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bae2aae2-5c65-4100-adab-4e56618f4976 admin admin] The token cache is being invalidate because project 7e894082fa89418986be5db793e1e676 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:57.890043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bae2aae2-5c65-4100-adab-4e56618f4976 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:57.890703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-bae2aae2-5c65-4100-adab-4e56618f4976 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:57.892156 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4171/8407] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:57 2022] DELETE /identity/v3/projects/7e894082fa89418986be5db793e1e676 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:57.896981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7877b-59c5-44e8-82bd-c16025215770 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:57.897454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7877b-59c5-44e8-82bd-c16025215770 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:57.897560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7877b-59c5-44e8-82bd-c16025215770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:57.917890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-dce7877b-59c5-44e8-82bd-c16025215770 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:57.919251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-dce7877b-59c5-44e8-82bd-c16025215770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:57.923386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:57.981571 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4237/8408] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:25:57.989351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae06a4-dbe8-4816-b1f7-c3773c2e2514 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:57.989631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae06a4-dbe8-4816-b1f7-c3773c2e2514 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:57.989900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae06a4-dbe8-4816-b1f7-c3773c2e2514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:58.009641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-26ae06a4-dbe8-4816-b1f7-c3773c2e2514 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:58.010854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-26ae06a4-dbe8-4816-b1f7-c3773c2e2514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:58.018736 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4172/8409] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:25:58.022825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7877b-59c5-44e8-82bd-c16025215770 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:58.032754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7877b-59c5-44e8-82bd-c16025215770 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:58.037454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9c2d2-90a8-41b2-8abd-32a2921a440f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:58.038065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9c2d2-90a8-41b2-8abd-32a2921a440f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:58.038344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9c2d2-90a8-41b2-8abd-32a2921a440f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:58.038670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9c2d2-90a8-41b2-8abd-32a2921a440f admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:58.039330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9c2d2-90a8-41b2-8abd-32a2921a440f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:25:58.039634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9c2d2-90a8-41b2-8abd-32a2921a440f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:58.041226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9c2d2-90a8-41b2-8abd-32a2921a440f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:58.054309 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4238/8410] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:25:58.059142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae06a4-dbe8-4816-b1f7-c3773c2e2514 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:58.068718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae06a4-dbe8-4816-b1f7-c3773c2e2514 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:58.073471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c26b0b-4ef5-455d-a774-845a4ba56c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:58.074076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c26b0b-4ef5-455d-a774-845a4ba56c68 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:58.074362 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c26b0b-4ef5-455d-a774-845a4ba56c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:58.074680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c26b0b-4ef5-455d-a774-845a4ba56c68 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:58.075318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c26b0b-4ef5-455d-a774-845a4ba56c68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:58.077273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c26b0b-4ef5-455d-a774-845a4ba56c68 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:58.103612 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4173/8411] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:25:58 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:58.108333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9c2d2-90a8-41b2-8abd-32a2921a440f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:58.118335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9c2d2-90a8-41b2-8abd-32a2921a440f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:58.176651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3e9a0-c201-484d-85db-ac21aba1723a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:58.177268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3e9a0-c201-484d-85db-ac21aba1723a admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:58.177556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3e9a0-c201-484d-85db-ac21aba1723a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:58.177829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3e9a0-c201-484d-85db-ac21aba1723a admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:58.178474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb3e9a0-c201-484d-85db-ac21aba1723a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:58.180443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb3e9a0-c201-484d-85db-ac21aba1723a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:58.202168 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4239/8412] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:25:58 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:58.206660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c26b0b-4ef5-455d-a774-845a4ba56c68 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:58.216867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c26b0b-4ef5-455d-a774-845a4ba56c68 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:58.221656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e1d05-6f7b-4ad2-91e1-ef3f3a1e2f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:58.222258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e1d05-6f7b-4ad2-91e1-ef3f3a1e2f5e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:58.222562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e1d05-6f7b-4ad2-91e1-ef3f3a1e2f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:58.222841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e1d05-6f7b-4ad2-91e1-ef3f3a1e2f5e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:58.223483 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e1d05-6f7b-4ad2-91e1-ef3f3a1e2f5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:58.225120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e1d05-6f7b-4ad2-91e1-ef3f3a1e2f5e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:58.235787 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4174/8413] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:25:58.239531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3e9a0-c201-484d-85db-ac21aba1723a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:58.249988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3e9a0-c201-484d-85db-ac21aba1723a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:58.254863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ef056-ab2f-4cbc-8b51-9fb3a5833dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:58.255475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ef056-ab2f-4cbc-8b51-9fb3a5833dcb admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:58.255736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ef056-ab2f-4cbc-8b51-9fb3a5833dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:58.256007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ef056-ab2f-4cbc-8b51-9fb3a5833dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:58.256668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ef056-ab2f-4cbc-8b51-9fb3a5833dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:58.258333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ef056-ab2f-4cbc-8b51-9fb3a5833dcb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:58.268953 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4240/8414] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:25:58.272863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e1d05-6f7b-4ad2-91e1-ef3f3a1e2f5e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:58.283530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e1d05-6f7b-4ad2-91e1-ef3f3a1e2f5e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:58.288495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93821a0d-3ebb-4c45-a0e6-35b9b5ed6df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:58.289142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93821a0d-3ebb-4c45-a0e6-35b9b5ed6df3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:58.289428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93821a0d-3ebb-4c45-a0e6-35b9b5ed6df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:58.289707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93821a0d-3ebb-4c45-a0e6-35b9b5ed6df3 admin admin] PATH_INFO: `/v3/projects/cc31daf917b84818861020515dd14651/users/2dd1b7ebd8ac462885401c55a9a47a53/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:58.299046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:58.306148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93821a0d-3ebb-4c45-a0e6-35b9b5ed6df3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc31daf917b84818861020515dd14651, user_id=2dd1b7ebd8ac462885401c55a9a47a53, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:58.309424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93821a0d-3ebb-4c45-a0e6-35b9b5ed6df3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:58.322153 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4175/8415] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:25:58 2022] PUT /identity/v3/projects/cc31daf917b84818861020515dd14651/users/2dd1b7ebd8ac462885401c55a9a47a53/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:25:58.332350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e9f7f-9de1-4d64-9d25-3d35cc05a7ee None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:58.332637 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e9f7f-9de1-4d64-9d25-3d35cc05a7ee None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:58.332909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e9f7f-9de1-4d64-9d25-3d35cc05a7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:58.362648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-fd7e9f7f-9de1-4d64-9d25-3d35cc05a7ee None None] MFA Rules not processed for user `2dd1b7ebd8ac462885401c55a9a47a53`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:58.363902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fd7e9f7f-9de1-4d64-9d25-3d35cc05a7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:58.431393 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4241/8416] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:25:58.435829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93821a0d-3ebb-4c45-a0e6-35b9b5ed6df3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:58.497492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93821a0d-3ebb-4c45-a0e6-35b9b5ed6df3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:58.502161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1fdad-2431-4450-a6f0-672dc5d3ff9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:58.502787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1fdad-2431-4450-a6f0-672dc5d3ff9a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:58.503071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1fdad-2431-4450-a6f0-672dc5d3ff9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:58.503346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1fdad-2431-4450-a6f0-672dc5d3ff9a admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:58.503977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1fdad-2431-4450-a6f0-672dc5d3ff9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:58.505999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1fdad-2431-4450-a6f0-672dc5d3ff9a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:58.532160 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4176/8417] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:25:58 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:58.536214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e9f7f-9de1-4d64-9d25-3d35cc05a7ee None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:58.544280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:58.548751 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e9f7f-9de1-4d64-9d25-3d35cc05a7ee None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:58.604660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71b639-0fd5-40bc-8ef0-bef09e6dfbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:58.605807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71b639-0fd5-40bc-8ef0-bef09e6dfbbb admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:58.606087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71b639-0fd5-40bc-8ef0-bef09e6dfbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:58.606355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71b639-0fd5-40bc-8ef0-bef09e6dfbbb admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:58.607051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71b639-0fd5-40bc-8ef0-bef09e6dfbbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:58.608948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71b639-0fd5-40bc-8ef0-bef09e6dfbbb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:58.629773 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4242/8418] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:25:58 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:25:58.633827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1fdad-2431-4450-a6f0-672dc5d3ff9a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:58.644492 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1fdad-2431-4450-a6f0-672dc5d3ff9a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:58.649177 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de3bc3-6760-4766-a6bf-a7808d907fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:58.649777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de3bc3-6760-4766-a6bf-a7808d907fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:58.650062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de3bc3-6760-4766-a6bf-a7808d907fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:58.650338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de3bc3-6760-4766-a6bf-a7808d907fd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:58.651009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de3bc3-6760-4766-a6bf-a7808d907fd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:58.653110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de3bc3-6760-4766-a6bf-a7808d907fd5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:58.663751 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4177/8419] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:25:58.668834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71b639-0fd5-40bc-8ef0-bef09e6dfbbb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:58.678822 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71b639-0fd5-40bc-8ef0-bef09e6dfbbb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:58.683431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c7562-b139-4d59-8e42-90450789c318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:58.684072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c7562-b139-4d59-8e42-90450789c318 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:58.684354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c7562-b139-4d59-8e42-90450789c318 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:58.684631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c7562-b139-4d59-8e42-90450789c318 admin admin] PATH_INFO: `/v3/projects/ec8a0fef22d944548f9b4e640b421f56/users/829117410d1c4a4abb88a4cfc13ae381/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:58.699683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c7562-b139-4d59-8e42-90450789c318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8a0fef22d944548f9b4e640b421f56, user_id=829117410d1c4a4abb88a4cfc13ae381, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:58.702320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c7562-b139-4d59-8e42-90450789c318 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:58.715743 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4243/8420] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:25:58 2022] PUT /identity/v3/projects/ec8a0fef22d944548f9b4e640b421f56/users/829117410d1c4a4abb88a4cfc13ae381/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:25:58.719736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de3bc3-6760-4766-a6bf-a7808d907fd5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:58.729587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de3bc3-6760-4766-a6bf-a7808d907fd5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:58.785953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199db48b-e672-432d-b9ee-6108add2ee68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:58.786713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199db48b-e672-432d-b9ee-6108add2ee68 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:58.786853 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199db48b-e672-432d-b9ee-6108add2ee68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:58.787127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199db48b-e672-432d-b9ee-6108add2ee68 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:58.787762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199db48b-e672-432d-b9ee-6108add2ee68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:58.789361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199db48b-e672-432d-b9ee-6108add2ee68 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:58.799260 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4178/8421] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:25:58.803372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c7562-b139-4d59-8e42-90450789c318 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:58.813708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c7562-b139-4d59-8e42-90450789c318 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:58.818509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429505a-8ef9-4181-aaeb-2448a7c3e309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:58.819167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429505a-8ef9-4181-aaeb-2448a7c3e309 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:58.819449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429505a-8ef9-4181-aaeb-2448a7c3e309 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:58.819728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429505a-8ef9-4181-aaeb-2448a7c3e309 admin admin] PATH_INFO: `/v3/projects/ec8a0fef22d944548f9b4e640b421f56/users/829117410d1c4a4abb88a4cfc13ae381/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:58.821684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429505a-8ef9-4181-aaeb-2448a7c3e309 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8a0fef22d944548f9b4e640b421f56, user_id=829117410d1c4a4abb88a4cfc13ae381, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:58.824959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429505a-8ef9-4181-aaeb-2448a7c3e309 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:58.835092 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4244/8422] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:25:58 2022] PUT /identity/v3/projects/ec8a0fef22d944548f9b4e640b421f56/users/829117410d1c4a4abb88a4cfc13ae381/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:58.846812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc1cae-abf5-4e9c-af64-e1a9a91c48be None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:58.847128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc1cae-abf5-4e9c-af64-e1a9a91c48be None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:58.847441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc1cae-abf5-4e9c-af64-e1a9a91c48be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:58.875118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-dcdc1cae-abf5-4e9c-af64-e1a9a91c48be None None] MFA Rules not processed for user `829117410d1c4a4abb88a4cfc13ae381`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:58.876312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-dcdc1cae-abf5-4e9c-af64-e1a9a91c48be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:58.921468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:58.945703 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4179/8423] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:25:58.951109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429505a-8ef9-4181-aaeb-2448a7c3e309 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:58.961809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429505a-8ef9-4181-aaeb-2448a7c3e309 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:58.967085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46bb93-52ed-426c-81d7-16665de7f39f tempest-ProjectsNegativeTestJSON-1121990631 tempest-ProjectsNegativeTestJSON-1121990631-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '829117410d1c4a4abb88a4cfc13ae381', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec8a0fef22d944548f9b4e640b421f56', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:58.967713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46bb93-52ed-426c-81d7-16665de7f39f tempest-ProjectsNegativeTestJSON-1121990631 tempest-ProjectsNegativeTestJSON-1121990631-project-admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:58.967997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46bb93-52ed-426c-81d7-16665de7f39f tempest-ProjectsNegativeTestJSON-1121990631 tempest-ProjectsNegativeTestJSON-1121990631-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:58.968273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46bb93-52ed-426c-81d7-16665de7f39f tempest-ProjectsNegativeTestJSON-1121990631 tempest-ProjectsNegativeTestJSON-1121990631-project-admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:58.968898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46bb93-52ed-426c-81d7-16665de7f39f tempest-ProjectsNegativeTestJSON-1121990631 tempest-ProjectsNegativeTestJSON-1121990631-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:58.970756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46bb93-52ed-426c-81d7-16665de7f39f tempest-ProjectsNegativeTestJSON-1121990631 tempest-ProjectsNegativeTestJSON-1121990631-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:59.002086 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4245/8424] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:25:58 2022] POST /identity/v3/projects => generated 359 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:25:59.006293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc1cae-abf5-4e9c-af64-e1a9a91c48be None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:59.065193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc1cae-abf5-4e9c-af64-e1a9a91c48be None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:59.117281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e797d7-1e2b-47da-ad5c-b82d60c11c56 tempest-ProjectsNegativeTestJSON-908452079 tempest-ProjectsNegativeTestJSON-908452079-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd1b7ebd8ac462885401c55a9a47a53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc31daf917b84818861020515dd14651', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:59.117911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e797d7-1e2b-47da-ad5c-b82d60c11c56 tempest-ProjectsNegativeTestJSON-908452079 tempest-ProjectsNegativeTestJSON-908452079-project] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:59.118196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e797d7-1e2b-47da-ad5c-b82d60c11c56 tempest-ProjectsNegativeTestJSON-908452079 tempest-ProjectsNegativeTestJSON-908452079-project] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:59.118473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e797d7-1e2b-47da-ad5c-b82d60c11c56 tempest-ProjectsNegativeTestJSON-908452079 tempest-ProjectsNegativeTestJSON-908452079-project] PATH_INFO: `/v3/projects/e7ec043eeff44497906a5daf7a7f823b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:59.119632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e797d7-1e2b-47da-ad5c-b82d60c11c56 tempest-ProjectsNegativeTestJSON-908452079 tempest-ProjectsNegativeTestJSON-908452079-project] RBAC: Authorizing `identity:delete_project(project_id=e7ec043eeff44497906a5daf7a7f823b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:59.126222 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application [None req-84e797d7-1e2b-47da-ad5c-b82d60c11c56 tempest-ProjectsNegativeTestJSON-908452079 tempest-ProjectsNegativeTestJSON-908452079-project] You are not authorized to perform the requested action: identity:delete_project.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:delete_project. Oct 01 21:25:59.126222 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 01 21:25:59.126222 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 01 21:25:59.126222 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 01 21:25:59.126222 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 01 21:25:59.126222 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 01 21:25:59.126222 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 01 21:25:59.126222 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 01 21:25:59.126222 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 01 21:25:59.126222 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 01 21:25:59.126222 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 01 21:25:59.126222 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 01 21:25:59.126222 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 207, in delete Oct 01 21:25:59.126222 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Oct 01 21:25:59.127193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Oct 01 21:25:59.127193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application enforcer_obj._enforce( Oct 01 21:25:59.127193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Oct 01 21:25:59.127193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Oct 01 21:25:59.127193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 01 21:25:59.127193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Oct 01 21:25:59.127193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:delete_project. Oct 01 21:25:59.127193 ncdv-master-1 devstack@keystone.service[99140]: ERROR keystone.server.flask.application Oct 01 21:25:59.127820 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4180/8425] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:59 2022] DELETE /identity/v3/projects/e7ec043eeff44497906a5daf7a7f823b => generated 136 bytes in 121 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Oct 01 21:25:59.131783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46bb93-52ed-426c-81d7-16665de7f39f tempest-ProjectsNegativeTestJSON-1121990631 tempest-ProjectsNegativeTestJSON-1121990631-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:59.194477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46bb93-52ed-426c-81d7-16665de7f39f tempest-ProjectsNegativeTestJSON-1121990631 tempest-ProjectsNegativeTestJSON-1121990631-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:59.199079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97338c2-f8ed-4f2e-b769-6963238d02ec tempest-ProjectsNegativeTestJSON-1121990631 tempest-ProjectsNegativeTestJSON-1121990631-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '829117410d1c4a4abb88a4cfc13ae381', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec8a0fef22d944548f9b4e640b421f56', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:59.200372 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97338c2-f8ed-4f2e-b769-6963238d02ec tempest-ProjectsNegativeTestJSON-1121990631 tempest-ProjectsNegativeTestJSON-1121990631-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:59.200658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97338c2-f8ed-4f2e-b769-6963238d02ec tempest-ProjectsNegativeTestJSON-1121990631 tempest-ProjectsNegativeTestJSON-1121990631-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:59.200932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97338c2-f8ed-4f2e-b769-6963238d02ec tempest-ProjectsNegativeTestJSON-1121990631 tempest-ProjectsNegativeTestJSON-1121990631-project-admin] PATH_INFO: `/v3/projects/e7ec043eeff44497906a5daf7a7f823b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:59.202166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97338c2-f8ed-4f2e-b769-6963238d02ec tempest-ProjectsNegativeTestJSON-1121990631 tempest-ProjectsNegativeTestJSON-1121990631-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e7ec043eeff44497906a5daf7a7f823b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:59.204313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97338c2-f8ed-4f2e-b769-6963238d02ec tempest-ProjectsNegativeTestJSON-1121990631 tempest-ProjectsNegativeTestJSON-1121990631-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:59.242326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:59.252608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a97338c2-f8ed-4f2e-b769-6963238d02ec tempest-ProjectsNegativeTestJSON-1121990631 tempest-ProjectsNegativeTestJSON-1121990631-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7ec043eeff44497906a5daf7a7f823b', 'request_id': 'req-a97338c2-f8ed-4f2e-b769-6963238d02ec'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:59.258756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a97338c2-f8ed-4f2e-b769-6963238d02ec tempest-ProjectsNegativeTestJSON-1121990631 tempest-ProjectsNegativeTestJSON-1121990631-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7ec043eeff44497906a5daf7a7f823b', 'request_id': 'req-a97338c2-f8ed-4f2e-b769-6963238d02ec'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:59.264978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a97338c2-f8ed-4f2e-b769-6963238d02ec tempest-ProjectsNegativeTestJSON-1121990631 tempest-ProjectsNegativeTestJSON-1121990631-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7ec043eeff44497906a5daf7a7f823b', 'request_id': 'req-a97338c2-f8ed-4f2e-b769-6963238d02ec'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:59.277062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a97338c2-f8ed-4f2e-b769-6963238d02ec tempest-ProjectsNegativeTestJSON-1121990631 tempest-ProjectsNegativeTestJSON-1121990631-project-admin] The token cache is being invalidate because project e7ec043eeff44497906a5daf7a7f823b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:25:59.277389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a97338c2-f8ed-4f2e-b769-6963238d02ec tempest-ProjectsNegativeTestJSON-1121990631 tempest-ProjectsNegativeTestJSON-1121990631-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:59.278116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a97338c2-f8ed-4f2e-b769-6963238d02ec tempest-ProjectsNegativeTestJSON-1121990631 tempest-ProjectsNegativeTestJSON-1121990631-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:59.279687 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4246/8426] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:59 2022] DELETE /identity/v3/projects/e7ec043eeff44497906a5daf7a7f823b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:59.285142 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e797d7-1e2b-47da-ad5c-b82d60c11c56 tempest-ProjectsNegativeTestJSON-908452079 tempest-ProjectsNegativeTestJSON-908452079-project] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:59.286876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-84e797d7-1e2b-47da-ad5c-b82d60c11c56 tempest-ProjectsNegativeTestJSON-908452079 tempest-ProjectsNegativeTestJSON-908452079-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:59.353774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e797d7-1e2b-47da-ad5c-b82d60c11c56 tempest-ProjectsNegativeTestJSON-908452079 tempest-ProjectsNegativeTestJSON-908452079-project] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:59.397615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:59.409475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eec9a8-b622-41ff-be2c-a0ea38d03b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:59.410104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eec9a8-b622-41ff-be2c-a0ea38d03b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:59.410389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eec9a8-b622-41ff-be2c-a0ea38d03b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:59.410698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eec9a8-b622-41ff-be2c-a0ea38d03b27 admin admin] PATH_INFO: `/v3/users/2dd1b7ebd8ac462885401c55a9a47a53` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:59.411821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eec9a8-b622-41ff-be2c-a0ea38d03b27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dd1b7ebd8ac462885401c55a9a47a53)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:59.413763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eec9a8-b622-41ff-be2c-a0ea38d03b27 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:59.460836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-28eec9a8-b622-41ff-be2c-a0ea38d03b27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dd1b7ebd8ac462885401c55a9a47a53'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:59.461513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-28eec9a8-b622-41ff-be2c-a0ea38d03b27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dd1b7ebd8ac462885401c55a9a47a53'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:59.462147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-28eec9a8-b622-41ff-be2c-a0ea38d03b27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dd1b7ebd8ac462885401c55a9a47a53'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:59.469577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-28eec9a8-b622-41ff-be2c-a0ea38d03b27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dd1b7ebd8ac462885401c55a9a47a53'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:59.481997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-28eec9a8-b622-41ff-be2c-a0ea38d03b27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dd1b7ebd8ac462885401c55a9a47a53'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:25:59.498211 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4181/8427] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:25:59 2022] DELETE /identity/v3/users/2dd1b7ebd8ac462885401c55a9a47a53 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:25:59.502910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057c8ec-b65b-4608-b341-a5effcf59b24 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:59.503203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057c8ec-b65b-4608-b341-a5effcf59b24 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:59.503482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057c8ec-b65b-4608-b341-a5effcf59b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:59.522547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f057c8ec-b65b-4608-b341-a5effcf59b24 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:25:59.524022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f057c8ec-b65b-4608-b341-a5effcf59b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:59.583734 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4247/8428] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:25:59.599746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eec9a8-b622-41ff-be2c-a0ea38d03b27 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:59.600403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-28eec9a8-b622-41ff-be2c-a0ea38d03b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:59.661736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eec9a8-b622-41ff-be2c-a0ea38d03b27 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:59.665632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660c461-25d8-46ce-9644-dc3ec1ef8ccb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:59.666233 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660c461-25d8-46ce-9644-dc3ec1ef8ccb service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:59.666801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660c461-25d8-46ce-9644-dc3ec1ef8ccb service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:59.667076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660c461-25d8-46ce-9644-dc3ec1ef8ccb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:59.676802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660c461-25d8-46ce-9644-dc3ec1ef8ccb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:59.678238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660c461-25d8-46ce-9644-dc3ec1ef8ccb service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:25:59.688428 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4182/8429] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:25:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:25:59.952639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057c8ec-b65b-4608-b341-a5effcf59b24 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:25:59.954220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f057c8ec-b65b-4608-b341-a5effcf59b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:25:59.967385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057c8ec-b65b-4608-b341-a5effcf59b24 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:25:59.972218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264fcd98-17cd-4eaa-a42a-5a39e483767d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:25:59.972940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fcd98-17cd-4eaa-a42a-5a39e483767d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:25:59.973225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fcd98-17cd-4eaa-a42a-5a39e483767d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:25:59.973500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fcd98-17cd-4eaa-a42a-5a39e483767d admin admin] PATH_INFO: `/v3/projects/cc31daf917b84818861020515dd14651` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:25:59.974668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264fcd98-17cd-4eaa-a42a-5a39e483767d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc31daf917b84818861020515dd14651)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:25:59.976568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264fcd98-17cd-4eaa-a42a-5a39e483767d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:00.023824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-264fcd98-17cd-4eaa-a42a-5a39e483767d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc31daf917b84818861020515dd14651', 'request_id': 'req-264fcd98-17cd-4eaa-a42a-5a39e483767d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:00.030722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-264fcd98-17cd-4eaa-a42a-5a39e483767d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc31daf917b84818861020515dd14651', 'request_id': 'req-264fcd98-17cd-4eaa-a42a-5a39e483767d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:00.036434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-264fcd98-17cd-4eaa-a42a-5a39e483767d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc31daf917b84818861020515dd14651', 'request_id': 'req-264fcd98-17cd-4eaa-a42a-5a39e483767d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:00.051632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-264fcd98-17cd-4eaa-a42a-5a39e483767d admin admin] The token cache is being invalidate because project cc31daf917b84818861020515dd14651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:26:00.051924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-264fcd98-17cd-4eaa-a42a-5a39e483767d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:00.052551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-264fcd98-17cd-4eaa-a42a-5a39e483767d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:00.054037 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4248/8430] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:25:59 2022] DELETE /identity/v3/projects/cc31daf917b84818861020515dd14651 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:26:00.058147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660c461-25d8-46ce-9644-dc3ec1ef8ccb service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:00.067838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660c461-25d8-46ce-9644-dc3ec1ef8ccb service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:00.123388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd196cb-151f-42ec-b17b-6b3540149241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:00.124011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd196cb-151f-42ec-b17b-6b3540149241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:00.124290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd196cb-151f-42ec-b17b-6b3540149241 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:00.124564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd196cb-151f-42ec-b17b-6b3540149241 admin admin] PATH_INFO: `/v3/users/829117410d1c4a4abb88a4cfc13ae381` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:00.125665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd196cb-151f-42ec-b17b-6b3540149241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=829117410d1c4a4abb88a4cfc13ae381)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:00.127612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd196cb-151f-42ec-b17b-6b3540149241 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:00.140776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:00.174046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-abd196cb-151f-42ec-b17b-6b3540149241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '829117410d1c4a4abb88a4cfc13ae381'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:00.174750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-abd196cb-151f-42ec-b17b-6b3540149241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '829117410d1c4a4abb88a4cfc13ae381'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:00.175361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-abd196cb-151f-42ec-b17b-6b3540149241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '829117410d1c4a4abb88a4cfc13ae381'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:00.182392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-abd196cb-151f-42ec-b17b-6b3540149241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '829117410d1c4a4abb88a4cfc13ae381'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:00.194765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-abd196cb-151f-42ec-b17b-6b3540149241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '829117410d1c4a4abb88a4cfc13ae381'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:00.207193 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4183/8431] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:26:00 2022] DELETE /identity/v3/users/829117410d1c4a4abb88a4cfc13ae381 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:00.479280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264fcd98-17cd-4eaa-a42a-5a39e483767d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:00.480819 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-264fcd98-17cd-4eaa-a42a-5a39e483767d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:00.496135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264fcd98-17cd-4eaa-a42a-5a39e483767d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:00.544482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:00.557618 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3a68f-63b1-4c53-b768-7a19a20aaefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:00.558189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3a68f-63b1-4c53-b768-7a19a20aaefc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:00.559009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3a68f-63b1-4c53-b768-7a19a20aaefc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:00.559301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3a68f-63b1-4c53-b768-7a19a20aaefc admin admin] PATH_INFO: `/v3/projects/ec8a0fef22d944548f9b4e640b421f56` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:00.560430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3a68f-63b1-4c53-b768-7a19a20aaefc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec8a0fef22d944548f9b4e640b421f56)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:00.562320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3a68f-63b1-4c53-b768-7a19a20aaefc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:00.608067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7de3a68f-63b1-4c53-b768-7a19a20aaefc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec8a0fef22d944548f9b4e640b421f56', 'request_id': 'req-7de3a68f-63b1-4c53-b768-7a19a20aaefc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:00.614011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7de3a68f-63b1-4c53-b768-7a19a20aaefc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec8a0fef22d944548f9b4e640b421f56', 'request_id': 'req-7de3a68f-63b1-4c53-b768-7a19a20aaefc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:00.619581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7de3a68f-63b1-4c53-b768-7a19a20aaefc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec8a0fef22d944548f9b4e640b421f56', 'request_id': 'req-7de3a68f-63b1-4c53-b768-7a19a20aaefc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:00.632062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7de3a68f-63b1-4c53-b768-7a19a20aaefc admin admin] The token cache is being invalidate because project ec8a0fef22d944548f9b4e640b421f56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:26:00.632353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7de3a68f-63b1-4c53-b768-7a19a20aaefc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:00.632973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-7de3a68f-63b1-4c53-b768-7a19a20aaefc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:00.634473 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4249/8432] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:26:00 2022] DELETE /identity/v3/projects/ec8a0fef22d944548f9b4e640b421f56 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:00.640014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d5b5a-54a3-4542-b225-ab0bb9191d24 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:00.640307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d5b5a-54a3-4542-b225-ab0bb9191d24 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:00.640585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d5b5a-54a3-4542-b225-ab0bb9191d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:00.663620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2f2d5b5a-54a3-4542-b225-ab0bb9191d24 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:00.664825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2f2d5b5a-54a3-4542-b225-ab0bb9191d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:00.725238 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4184/8433] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:26:00.733093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03688134-a1e7-4680-9320-d840e42cf735 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:00.733384 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03688134-a1e7-4680-9320-d840e42cf735 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:00.733662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03688134-a1e7-4680-9320-d840e42cf735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:00.753798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-03688134-a1e7-4680-9320-d840e42cf735 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:00.755042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-03688134-a1e7-4680-9320-d840e42cf735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:00.763740 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4250/8434] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:26:00.767795 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d5b5a-54a3-4542-b225-ab0bb9191d24 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:00.777611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d5b5a-54a3-4542-b225-ab0bb9191d24 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:00.782385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d11d0-fba0-4c7d-8476-d2765faebd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:00.783023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d11d0-fba0-4c7d-8476-d2765faebd24 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:00.783305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d11d0-fba0-4c7d-8476-d2765faebd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:00.783582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d11d0-fba0-4c7d-8476-d2765faebd24 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:00.784234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d11d0-fba0-4c7d-8476-d2765faebd24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:26:00.784537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d11d0-fba0-4c7d-8476-d2765faebd24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:00.786161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d11d0-fba0-4c7d-8476-d2765faebd24 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:00.798161 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4185/8435] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:26:00.802367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03688134-a1e7-4680-9320-d840e42cf735 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:00.812164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03688134-a1e7-4680-9320-d840e42cf735 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:00.816910 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230899b-4e0d-44bb-9e6d-df7181dd9588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:00.817504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230899b-4e0d-44bb-9e6d-df7181dd9588 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:00.817785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230899b-4e0d-44bb-9e6d-df7181dd9588 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:00.818058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230899b-4e0d-44bb-9e6d-df7181dd9588 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:00.818714 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230899b-4e0d-44bb-9e6d-df7181dd9588 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:00.820591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230899b-4e0d-44bb-9e6d-df7181dd9588 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:00.848669 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4251/8436] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:26:00 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:26:00.852768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d11d0-fba0-4c7d-8476-d2765faebd24 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:00.862837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d11d0-fba0-4c7d-8476-d2765faebd24 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:00.919545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3760228-5e1e-4646-ab63-5c0b18b76b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:00.920147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3760228-5e1e-4646-ab63-5c0b18b76b7b admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:00.920434 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3760228-5e1e-4646-ab63-5c0b18b76b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:00.920710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3760228-5e1e-4646-ab63-5c0b18b76b7b admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:00.921337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3760228-5e1e-4646-ab63-5c0b18b76b7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:00.923289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3760228-5e1e-4646-ab63-5c0b18b76b7b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:00.945377 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4186/8437] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:26:00 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:26:00.949432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230899b-4e0d-44bb-9e6d-df7181dd9588 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:00.959925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230899b-4e0d-44bb-9e6d-df7181dd9588 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:00.964978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003e60b-4635-4285-98b6-074522f94232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:00.965575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003e60b-4635-4285-98b6-074522f94232 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:00.965852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003e60b-4635-4285-98b6-074522f94232 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:00.966124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003e60b-4635-4285-98b6-074522f94232 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:00.966808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003e60b-4635-4285-98b6-074522f94232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:00.968376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003e60b-4635-4285-98b6-074522f94232 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:00.978358 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4252/8438] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:26:00.982913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3760228-5e1e-4646-ab63-5c0b18b76b7b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:00.993310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3760228-5e1e-4646-ab63-5c0b18b76b7b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:00.998790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799b9af-4c57-4c41-8e2e-c78e20d836b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:00.999440 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799b9af-4c57-4c41-8e2e-c78e20d836b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:00.999727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799b9af-4c57-4c41-8e2e-c78e20d836b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:01.000007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799b9af-4c57-4c41-8e2e-c78e20d836b3 admin admin] PATH_INFO: `/v3/projects/fb1d1d749d6444239bf2a5ad3c8d6386/users/1eb1bd696d194e859792a3bdd8faf150/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:01.002264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:01.017363 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799b9af-4c57-4c41-8e2e-c78e20d836b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb1d1d749d6444239bf2a5ad3c8d6386, user_id=1eb1bd696d194e859792a3bdd8faf150, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:01.020099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799b9af-4c57-4c41-8e2e-c78e20d836b3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:01.032684 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4187/8439] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:26:00 2022] PUT /identity/v3/projects/fb1d1d749d6444239bf2a5ad3c8d6386/users/1eb1bd696d194e859792a3bdd8faf150/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:01.036675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003e60b-4635-4285-98b6-074522f94232 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:01.048188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003e60b-4635-4285-98b6-074522f94232 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:01.104581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80def6fc-8d62-47dd-92ee-2ee99fdadd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:01.105311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80def6fc-8d62-47dd-92ee-2ee99fdadd28 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:01.106160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80def6fc-8d62-47dd-92ee-2ee99fdadd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:01.106623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80def6fc-8d62-47dd-92ee-2ee99fdadd28 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:01.107313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80def6fc-8d62-47dd-92ee-2ee99fdadd28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:01.108903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80def6fc-8d62-47dd-92ee-2ee99fdadd28 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:01.119146 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4253/8440] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:26:01.124244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799b9af-4c57-4c41-8e2e-c78e20d836b3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:01.136157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799b9af-4c57-4c41-8e2e-c78e20d836b3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:01.141254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43e872-d895-4f6e-b01b-c743dfab4c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:01.141982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43e872-d895-4f6e-b01b-c743dfab4c0d admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:01.142298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43e872-d895-4f6e-b01b-c743dfab4c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:01.142632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43e872-d895-4f6e-b01b-c743dfab4c0d admin admin] PATH_INFO: `/v3/projects/fb1d1d749d6444239bf2a5ad3c8d6386/users/1eb1bd696d194e859792a3bdd8faf150/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:01.144557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43e872-d895-4f6e-b01b-c743dfab4c0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb1d1d749d6444239bf2a5ad3c8d6386, user_id=1eb1bd696d194e859792a3bdd8faf150, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:01.147272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43e872-d895-4f6e-b01b-c743dfab4c0d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:01.157564 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4188/8441] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:26:01 2022] PUT /identity/v3/projects/fb1d1d749d6444239bf2a5ad3c8d6386/users/1eb1bd696d194e859792a3bdd8faf150/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:01.168846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db15c7-24e0-4d18-94ff-82744588e8e4 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:01.169139 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db15c7-24e0-4d18-94ff-82744588e8e4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:01.169420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db15c7-24e0-4d18-94ff-82744588e8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:01.199252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-49db15c7-24e0-4d18-94ff-82744588e8e4 None None] MFA Rules not processed for user `1eb1bd696d194e859792a3bdd8faf150`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:01.200466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-49db15c7-24e0-4d18-94ff-82744588e8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:01.229031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:01.276482 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4254/8442] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:01 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:26:01.281531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43e872-d895-4f6e-b01b-c743dfab4c0d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:01.344724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43e872-d895-4f6e-b01b-c743dfab4c0d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:01.349506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318460fc-775b-4c9a-b60c-9c122ac85b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:01.350114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318460fc-775b-4c9a-b60c-9c122ac85b67 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:01.350398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318460fc-775b-4c9a-b60c-9c122ac85b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:01.350709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318460fc-775b-4c9a-b60c-9c122ac85b67 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:01.351336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318460fc-775b-4c9a-b60c-9c122ac85b67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:01.353273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318460fc-775b-4c9a-b60c-9c122ac85b67 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:01.383966 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4189/8443] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:26:01 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:26:01.387996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db15c7-24e0-4d18-94ff-82744588e8e4 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:01.398559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db15c7-24e0-4d18-94ff-82744588e8e4 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:01.464341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b322d8-b912-410a-9423-ed1d86f60028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:01.465512 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b322d8-b912-410a-9423-ed1d86f60028 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:01.465904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b322d8-b912-410a-9423-ed1d86f60028 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:01.466154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b322d8-b912-410a-9423-ed1d86f60028 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:01.466923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b322d8-b912-410a-9423-ed1d86f60028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:01.468929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b322d8-b912-410a-9423-ed1d86f60028 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:01.498063 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4255/8444] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:26:01 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:26:01.503805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318460fc-775b-4c9a-b60c-9c122ac85b67 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:01.515487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318460fc-775b-4c9a-b60c-9c122ac85b67 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:01.520957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fac81-03fe-4adb-91bb-fb566cace618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:01.521643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fac81-03fe-4adb-91bb-fb566cace618 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:01.521985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fac81-03fe-4adb-91bb-fb566cace618 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:01.522321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fac81-03fe-4adb-91bb-fb566cace618 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:01.523056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646fac81-03fe-4adb-91bb-fb566cace618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:01.524700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646fac81-03fe-4adb-91bb-fb566cace618 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:01.536254 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4190/8445] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:26:01.540144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b322d8-b912-410a-9423-ed1d86f60028 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:01.552086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b322d8-b912-410a-9423-ed1d86f60028 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:01.557045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603617c7-e552-4272-9a63-a05750f86a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:01.557646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603617c7-e552-4272-9a63-a05750f86a2b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:01.557934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603617c7-e552-4272-9a63-a05750f86a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:01.558203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603617c7-e552-4272-9a63-a05750f86a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:01.558961 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603617c7-e552-4272-9a63-a05750f86a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:01.560544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603617c7-e552-4272-9a63-a05750f86a2b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:01.570880 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4256/8446] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:26:01.574962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fac81-03fe-4adb-91bb-fb566cace618 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:01.585321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fac81-03fe-4adb-91bb-fb566cace618 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:01.590792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512221f-8916-4ad0-b6e9-83cb0f1b4523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:01.591517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512221f-8916-4ad0-b6e9-83cb0f1b4523 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:01.591798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512221f-8916-4ad0-b6e9-83cb0f1b4523 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:01.592091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512221f-8916-4ad0-b6e9-83cb0f1b4523 admin admin] PATH_INFO: `/v3/projects/82a29a9eb02c46539658213ba205aa56/users/171bcdc9f90444cd9c1e1ae08080f5e4/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:01.607603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5512221f-8916-4ad0-b6e9-83cb0f1b4523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82a29a9eb02c46539658213ba205aa56, user_id=171bcdc9f90444cd9c1e1ae08080f5e4, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:01.610404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5512221f-8916-4ad0-b6e9-83cb0f1b4523 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:01.624098 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4191/8447] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:26:01 2022] PUT /identity/v3/projects/82a29a9eb02c46539658213ba205aa56/users/171bcdc9f90444cd9c1e1ae08080f5e4/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:01.730520 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd4367-447d-4fd8-b14e-153f1ebd62ca None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:01.730874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd4367-447d-4fd8-b14e-153f1ebd62ca None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:01.731156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd4367-447d-4fd8-b14e-153f1ebd62ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:01.760764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-40bd4367-447d-4fd8-b14e-153f1ebd62ca None None] MFA Rules not processed for user `171bcdc9f90444cd9c1e1ae08080f5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:01.762612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-40bd4367-447d-4fd8-b14e-153f1ebd62ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:01.831352 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4257/8448] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:26:01.843477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512221f-8916-4ad0-b6e9-83cb0f1b4523 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:01.845268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5512221f-8916-4ad0-b6e9-83cb0f1b4523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:01.928979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512221f-8916-4ad0-b6e9-83cb0f1b4523 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:01.932905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e682dc-9401-439c-bd0c-fffb784c705d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:01.933584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e682dc-9401-439c-bd0c-fffb784c705d service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:01.933865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e682dc-9401-439c-bd0c-fffb784c705d service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:01.934135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e682dc-9401-439c-bd0c-fffb784c705d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:01.946413 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:01.996633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e682dc-9401-439c-bd0c-fffb784c705d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:01.997980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e682dc-9401-439c-bd0c-fffb784c705d service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:02.007825 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4192/8449] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:26:01 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:26:02.672613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd4367-447d-4fd8-b14e-153f1ebd62ca None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:02.706148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:02.739713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd4367-447d-4fd8-b14e-153f1ebd62ca None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:02.744678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf96a0e-6c1d-46fd-9399-306fb1b131da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:02.745593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf96a0e-6c1d-46fd-9399-306fb1b131da admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:02.745994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf96a0e-6c1d-46fd-9399-306fb1b131da admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:02.746855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf96a0e-6c1d-46fd-9399-306fb1b131da admin admin] PATH_INFO: `/v3/users/1eb1bd696d194e859792a3bdd8faf150` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:02.748069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf96a0e-6c1d-46fd-9399-306fb1b131da admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eb1bd696d194e859792a3bdd8faf150)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:02.750073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf96a0e-6c1d-46fd-9399-306fb1b131da admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:02.801309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2bf96a0e-6c1d-46fd-9399-306fb1b131da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eb1bd696d194e859792a3bdd8faf150'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:02.808398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2bf96a0e-6c1d-46fd-9399-306fb1b131da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eb1bd696d194e859792a3bdd8faf150'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:02.820623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2bf96a0e-6c1d-46fd-9399-306fb1b131da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eb1bd696d194e859792a3bdd8faf150'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:02.821306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2bf96a0e-6c1d-46fd-9399-306fb1b131da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eb1bd696d194e859792a3bdd8faf150'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:02.829218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-2bf96a0e-6c1d-46fd-9399-306fb1b131da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eb1bd696d194e859792a3bdd8faf150'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:02.835334 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4258/8450] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:26:02 2022] DELETE /identity/v3/users/1eb1bd696d194e859792a3bdd8faf150 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:02.840315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2a3fa-ff88-46ee-942b-705aa77743db None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:02.840838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2a3fa-ff88-46ee-942b-705aa77743db None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:02.840943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2a3fa-ff88-46ee-942b-705aa77743db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:02.860792 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-17f2a3fa-ff88-46ee-942b-705aa77743db None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:02.862000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-17f2a3fa-ff88-46ee-942b-705aa77743db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:02.924784 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4193/8451] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:26:02.934829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf96a0e-6c1d-46fd-9399-306fb1b131da admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:02.936275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2bf96a0e-6c1d-46fd-9399-306fb1b131da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:02.998200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf96a0e-6c1d-46fd-9399-306fb1b131da admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:03.001879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c14e4f-3784-4b44-9bde-97518c3f705e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:03.002593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c14e4f-3784-4b44-9bde-97518c3f705e service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:03.002868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c14e4f-3784-4b44-9bde-97518c3f705e service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:03.003138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c14e4f-3784-4b44-9bde-97518c3f705e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:03.012768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c14e4f-3784-4b44-9bde-97518c3f705e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:03.014205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c14e4f-3784-4b44-9bde-97518c3f705e service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:03.024558 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4259/8452] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:26:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:26:03.290893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2a3fa-ff88-46ee-942b-705aa77743db None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:03.293275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-17f2a3fa-ff88-46ee-942b-705aa77743db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:03.306986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2a3fa-ff88-46ee-942b-705aa77743db None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:03.312549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24953aba-e25b-4d93-b2d8-8e3cfb41bf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:03.313279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24953aba-e25b-4d93-b2d8-8e3cfb41bf63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:03.313634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24953aba-e25b-4d93-b2d8-8e3cfb41bf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:03.313982 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24953aba-e25b-4d93-b2d8-8e3cfb41bf63 admin admin] PATH_INFO: `/v3/projects/fb1d1d749d6444239bf2a5ad3c8d6386` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:03.315279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24953aba-e25b-4d93-b2d8-8e3cfb41bf63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb1d1d749d6444239bf2a5ad3c8d6386)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:03.317211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24953aba-e25b-4d93-b2d8-8e3cfb41bf63 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:03.364832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-24953aba-e25b-4d93-b2d8-8e3cfb41bf63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb1d1d749d6444239bf2a5ad3c8d6386', 'request_id': 'req-24953aba-e25b-4d93-b2d8-8e3cfb41bf63'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:03.372994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-24953aba-e25b-4d93-b2d8-8e3cfb41bf63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb1d1d749d6444239bf2a5ad3c8d6386', 'request_id': 'req-24953aba-e25b-4d93-b2d8-8e3cfb41bf63'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:03.378659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-24953aba-e25b-4d93-b2d8-8e3cfb41bf63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb1d1d749d6444239bf2a5ad3c8d6386', 'request_id': 'req-24953aba-e25b-4d93-b2d8-8e3cfb41bf63'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:03.389986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-24953aba-e25b-4d93-b2d8-8e3cfb41bf63 admin admin] The token cache is being invalidate because project fb1d1d749d6444239bf2a5ad3c8d6386 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:26:03.390276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-24953aba-e25b-4d93-b2d8-8e3cfb41bf63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:03.390980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-24953aba-e25b-4d93-b2d8-8e3cfb41bf63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:03.392514 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4194/8453] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:26:03 2022] DELETE /identity/v3/projects/fb1d1d749d6444239bf2a5ad3c8d6386 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:03.396676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c14e4f-3784-4b44-9bde-97518c3f705e service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:03.406913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c14e4f-3784-4b44-9bde-97518c3f705e service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:03.454779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:03.465952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96decde1-9cb9-4042-bceb-3af564843858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:03.466610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96decde1-9cb9-4042-bceb-3af564843858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:03.466890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96decde1-9cb9-4042-bceb-3af564843858 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:03.467191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96decde1-9cb9-4042-bceb-3af564843858 admin admin] PATH_INFO: `/v3/users/171bcdc9f90444cd9c1e1ae08080f5e4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:03.468329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96decde1-9cb9-4042-bceb-3af564843858 admin admin] RBAC: Authorizing `identity:delete_user(user_id=171bcdc9f90444cd9c1e1ae08080f5e4)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:03.470292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96decde1-9cb9-4042-bceb-3af564843858 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:03.518640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96decde1-9cb9-4042-bceb-3af564843858 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '171bcdc9f90444cd9c1e1ae08080f5e4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:03.525487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96decde1-9cb9-4042-bceb-3af564843858 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '171bcdc9f90444cd9c1e1ae08080f5e4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:03.537676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96decde1-9cb9-4042-bceb-3af564843858 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '171bcdc9f90444cd9c1e1ae08080f5e4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:03.538338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96decde1-9cb9-4042-bceb-3af564843858 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '171bcdc9f90444cd9c1e1ae08080f5e4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:03.546560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-96decde1-9cb9-4042-bceb-3af564843858 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '171bcdc9f90444cd9c1e1ae08080f5e4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:03.552402 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4260/8454] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:26:03 2022] DELETE /identity/v3/users/171bcdc9f90444cd9c1e1ae08080f5e4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:03.830653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24953aba-e25b-4d93-b2d8-8e3cfb41bf63 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:03.832173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-24953aba-e25b-4d93-b2d8-8e3cfb41bf63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:03.845743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24953aba-e25b-4d93-b2d8-8e3cfb41bf63 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:03.876721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:03.905598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2560e-eef7-41cc-9ad9-8d736a427853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:03.906157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2560e-eef7-41cc-9ad9-8d736a427853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:03.906436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2560e-eef7-41cc-9ad9-8d736a427853 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:03.906741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2560e-eef7-41cc-9ad9-8d736a427853 admin admin] PATH_INFO: `/v3/projects/82a29a9eb02c46539658213ba205aa56` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:03.907839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2560e-eef7-41cc-9ad9-8d736a427853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82a29a9eb02c46539658213ba205aa56)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:03.909717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2560e-eef7-41cc-9ad9-8d736a427853 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:03.955712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95b2560e-eef7-41cc-9ad9-8d736a427853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82a29a9eb02c46539658213ba205aa56', 'request_id': 'req-95b2560e-eef7-41cc-9ad9-8d736a427853'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:03.962912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95b2560e-eef7-41cc-9ad9-8d736a427853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82a29a9eb02c46539658213ba205aa56', 'request_id': 'req-95b2560e-eef7-41cc-9ad9-8d736a427853'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:03.969327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95b2560e-eef7-41cc-9ad9-8d736a427853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82a29a9eb02c46539658213ba205aa56', 'request_id': 'req-95b2560e-eef7-41cc-9ad9-8d736a427853'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:03.980125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95b2560e-eef7-41cc-9ad9-8d736a427853 admin admin] The token cache is being invalidate because project 82a29a9eb02c46539658213ba205aa56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:26:03.980911 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95b2560e-eef7-41cc-9ad9-8d736a427853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:03.981596 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-95b2560e-eef7-41cc-9ad9-8d736a427853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:03.983119 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4195/8455] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:26:03 2022] DELETE /identity/v3/projects/82a29a9eb02c46539658213ba205aa56 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:03.989925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71252a-492b-4fc0-a963-15494342cf88 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:03.990417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71252a-492b-4fc0-a963-15494342cf88 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:03.990543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71252a-492b-4fc0-a963-15494342cf88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:04.010647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8d71252a-492b-4fc0-a963-15494342cf88 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:04.011863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8d71252a-492b-4fc0-a963-15494342cf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:04.072862 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4261/8456] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:26:04.080706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a3723-6c3a-4693-ace4-79b9fff0ccc1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:04.081023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a3723-6c3a-4693-ace4-79b9fff0ccc1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:04.081305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a3723-6c3a-4693-ace4-79b9fff0ccc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:04.100902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-225a3723-6c3a-4693-ace4-79b9fff0ccc1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:04.102101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-225a3723-6c3a-4693-ace4-79b9fff0ccc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:04.110476 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4196/8457] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:26:04.114644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71252a-492b-4fc0-a963-15494342cf88 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:04.125147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71252a-492b-4fc0-a963-15494342cf88 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:04.129770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46002433-61c5-4e8e-a992-7aa656afb140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:04.130406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46002433-61c5-4e8e-a992-7aa656afb140 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:04.130801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46002433-61c5-4e8e-a992-7aa656afb140 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:04.131134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46002433-61c5-4e8e-a992-7aa656afb140 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:04.131969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46002433-61c5-4e8e-a992-7aa656afb140 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:26:04.132154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46002433-61c5-4e8e-a992-7aa656afb140 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:04.134406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46002433-61c5-4e8e-a992-7aa656afb140 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:04.146373 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4262/8458] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:26:04.150694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a3723-6c3a-4693-ace4-79b9fff0ccc1 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:04.159991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a3723-6c3a-4693-ace4-79b9fff0ccc1 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:04.164651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee3ff5-be65-4706-a4db-df25b3c1b168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:04.165243 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee3ff5-be65-4706-a4db-df25b3c1b168 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:04.165524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee3ff5-be65-4706-a4db-df25b3c1b168 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:04.165798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee3ff5-be65-4706-a4db-df25b3c1b168 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:04.166732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee3ff5-be65-4706-a4db-df25b3c1b168 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:04.169122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee3ff5-be65-4706-a4db-df25b3c1b168 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:04.198798 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4197/8459] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:26:04 2022] POST /identity/v3/projects => generated 355 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:26:04.199936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46002433-61c5-4e8e-a992-7aa656afb140 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:04.209834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46002433-61c5-4e8e-a992-7aa656afb140 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:04.259916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:04.267255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a9488-8b10-4f9a-9fb5-3e307ffda9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:04.268106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a9488-8b10-4f9a-9fb5-3e307ffda9c5 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:04.268583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a9488-8b10-4f9a-9fb5-3e307ffda9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:04.269033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a9488-8b10-4f9a-9fb5-3e307ffda9c5 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:04.269764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a9488-8b10-4f9a-9fb5-3e307ffda9c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:04.271844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a9488-8b10-4f9a-9fb5-3e307ffda9c5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:04.294176 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4263/8460] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:26:04 2022] POST /identity/v3/users => generated 278 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:26:04.298351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee3ff5-be65-4706-a4db-df25b3c1b168 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:04.308771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee3ff5-be65-4706-a4db-df25b3c1b168 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:04.313379 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984c315-b27c-40d3-973e-68d547d1455d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:04.313970 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984c315-b27c-40d3-973e-68d547d1455d admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:04.314249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984c315-b27c-40d3-973e-68d547d1455d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:04.314584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984c315-b27c-40d3-973e-68d547d1455d admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:04.315218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984c315-b27c-40d3-973e-68d547d1455d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:04.316840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984c315-b27c-40d3-973e-68d547d1455d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:04.327188 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4198/8461] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:26:04.331041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a9488-8b10-4f9a-9fb5-3e307ffda9c5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:04.341497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a9488-8b10-4f9a-9fb5-3e307ffda9c5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:04.346354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e1255-36c3-405d-a30e-7a355daa8dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:04.346975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e1255-36c3-405d-a30e-7a355daa8dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:04.347256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e1255-36c3-405d-a30e-7a355daa8dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:04.347534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e1255-36c3-405d-a30e-7a355daa8dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:04.348248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e1255-36c3-405d-a30e-7a355daa8dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:04.349962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e1255-36c3-405d-a30e-7a355daa8dd1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:04.361329 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4264/8462] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:26:04.365325 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984c315-b27c-40d3-973e-68d547d1455d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:04.375263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984c315-b27c-40d3-973e-68d547d1455d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:04.379916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68f2da-df14-4e2b-a924-19c7df787a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:04.380556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68f2da-df14-4e2b-a924-19c7df787a37 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:04.380836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68f2da-df14-4e2b-a924-19c7df787a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:04.381110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68f2da-df14-4e2b-a924-19c7df787a37 admin admin] PATH_INFO: `/v3/projects/d77bba39060140db9ab24fc212deebf6/users/a4744dc6d236477f85e6f4860d8f7f3d/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:04.395481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68f2da-df14-4e2b-a924-19c7df787a37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d77bba39060140db9ab24fc212deebf6, user_id=a4744dc6d236477f85e6f4860d8f7f3d, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:04.398127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68f2da-df14-4e2b-a924-19c7df787a37 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:04.411403 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4199/8463] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:26:04 2022] PUT /identity/v3/projects/d77bba39060140db9ab24fc212deebf6/users/a4744dc6d236477f85e6f4860d8f7f3d/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:04.422166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754a931-c00c-4aa0-aed6-988e2b6110f3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:04.422449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754a931-c00c-4aa0-aed6-988e2b6110f3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:04.422745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754a931-c00c-4aa0-aed6-988e2b6110f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:04.451003 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9754a931-c00c-4aa0-aed6-988e2b6110f3 None None] MFA Rules not processed for user `a4744dc6d236477f85e6f4860d8f7f3d`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:04.452200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9754a931-c00c-4aa0-aed6-988e2b6110f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:04.515424 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4265/8464] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:04 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:26:04.519894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68f2da-df14-4e2b-a924-19c7df787a37 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:04.581989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68f2da-df14-4e2b-a924-19c7df787a37 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:04.587116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5623d4-8570-497a-944d-cec33ecbbe88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:04.587718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5623d4-8570-497a-944d-cec33ecbbe88 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:04.588002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5623d4-8570-497a-944d-cec33ecbbe88 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:04.588280 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5623d4-8570-497a-944d-cec33ecbbe88 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:04.588899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5623d4-8570-497a-944d-cec33ecbbe88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:04.591386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5623d4-8570-497a-944d-cec33ecbbe88 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:04.596368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:04.625331 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4200/8465] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:26:04 2022] POST /identity/v3/projects => generated 357 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:26:04.629551 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754a931-c00c-4aa0-aed6-988e2b6110f3 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:04.641548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754a931-c00c-4aa0-aed6-988e2b6110f3 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:04.698795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a69ae1-eea3-4061-8a6a-f15ff6ce7617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:04.699810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a69ae1-eea3-4061-8a6a-f15ff6ce7617 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:04.700093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a69ae1-eea3-4061-8a6a-f15ff6ce7617 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:04.700366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a69ae1-eea3-4061-8a6a-f15ff6ce7617 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:04.700988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a69ae1-eea3-4061-8a6a-f15ff6ce7617 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:04.702922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a69ae1-eea3-4061-8a6a-f15ff6ce7617 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:04.724813 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4266/8466] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:26:04 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:26:04.728871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5623d4-8570-497a-944d-cec33ecbbe88 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:04.739087 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5623d4-8570-497a-944d-cec33ecbbe88 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:04.743744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dce0a-938d-4d16-95f9-368df2389921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:04.744331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dce0a-938d-4d16-95f9-368df2389921 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:04.744612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dce0a-938d-4d16-95f9-368df2389921 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:04.744888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dce0a-938d-4d16-95f9-368df2389921 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:04.745522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647dce0a-938d-4d16-95f9-368df2389921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:04.747647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647dce0a-938d-4d16-95f9-368df2389921 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:04.758575 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4201/8467] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:26:04.763293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a69ae1-eea3-4061-8a6a-f15ff6ce7617 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:04.774036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a69ae1-eea3-4061-8a6a-f15ff6ce7617 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:04.779457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324bb3e-eea6-4dc7-8d13-9fec46de8f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:04.780126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324bb3e-eea6-4dc7-8d13-9fec46de8f16 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:04.780408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324bb3e-eea6-4dc7-8d13-9fec46de8f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:04.780690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324bb3e-eea6-4dc7-8d13-9fec46de8f16 admin admin] PATH_INFO: `/v3/projects/ebd400ef653a468eb2481fdcd5b4ac45/users/c13ee4bd591b4e2aadffea407493a488/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:04.796636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324bb3e-eea6-4dc7-8d13-9fec46de8f16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebd400ef653a468eb2481fdcd5b4ac45, user_id=c13ee4bd591b4e2aadffea407493a488, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:04.799321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324bb3e-eea6-4dc7-8d13-9fec46de8f16 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:04.813027 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4267/8468] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:26:04 2022] PUT /identity/v3/projects/ebd400ef653a468eb2481fdcd5b4ac45/users/c13ee4bd591b4e2aadffea407493a488/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:26:04.817007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dce0a-938d-4d16-95f9-368df2389921 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:04.827582 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dce0a-938d-4d16-95f9-368df2389921 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:04.885704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e7880-a888-45fb-b0dd-7f8c0c62b932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:04.886273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e7880-a888-45fb-b0dd-7f8c0c62b932 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:04.886985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e7880-a888-45fb-b0dd-7f8c0c62b932 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:04.887281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e7880-a888-45fb-b0dd-7f8c0c62b932 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:04.887909 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347e7880-a888-45fb-b0dd-7f8c0c62b932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:04.889517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347e7880-a888-45fb-b0dd-7f8c0c62b932 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:04.900311 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4202/8469] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:26:04.904341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324bb3e-eea6-4dc7-8d13-9fec46de8f16 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:04.915210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324bb3e-eea6-4dc7-8d13-9fec46de8f16 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:04.920275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82977a-91e8-42f1-9e0f-d262c250b7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:04.920931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82977a-91e8-42f1-9e0f-d262c250b7a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:04.921209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82977a-91e8-42f1-9e0f-d262c250b7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:04.921482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82977a-91e8-42f1-9e0f-d262c250b7a3 admin admin] PATH_INFO: `/v3/projects/ebd400ef653a468eb2481fdcd5b4ac45/users/c13ee4bd591b4e2aadffea407493a488/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:04.923674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82977a-91e8-42f1-9e0f-d262c250b7a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebd400ef653a468eb2481fdcd5b4ac45, user_id=c13ee4bd591b4e2aadffea407493a488, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:04.926233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82977a-91e8-42f1-9e0f-d262c250b7a3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:04.937560 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4268/8470] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:26:04 2022] PUT /identity/v3/projects/ebd400ef653a468eb2481fdcd5b4ac45/users/c13ee4bd591b4e2aadffea407493a488/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:04.950006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505006d3-a6f9-42b4-85b9-b250a0af7c4d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:04.950299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505006d3-a6f9-42b4-85b9-b250a0af7c4d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:04.950617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505006d3-a6f9-42b4-85b9-b250a0af7c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:04.980907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-505006d3-a6f9-42b4-85b9-b250a0af7c4d None None] MFA Rules not processed for user `c13ee4bd591b4e2aadffea407493a488`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:04.982131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-505006d3-a6f9-42b4-85b9-b250a0af7c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:05.058925 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4203/8471] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:04 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:26:05.064277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82977a-91e8-42f1-9e0f-d262c250b7a3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:05.075398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82977a-91e8-42f1-9e0f-d262c250b7a3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:05.080165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995a9bc-43cf-4671-bb45-56df4ae85d05 tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c13ee4bd591b4e2aadffea407493a488', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebd400ef653a468eb2481fdcd5b4ac45', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:05.080769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995a9bc-43cf-4671-bb45-56df4ae85d05 tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:05.081052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995a9bc-43cf-4671-bb45-56df4ae85d05 tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:05.081330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995a9bc-43cf-4671-bb45-56df4ae85d05 tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:05.081979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1995a9bc-43cf-4671-bb45-56df4ae85d05 tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:26:05.082286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1995a9bc-43cf-4671-bb45-56df4ae85d05 tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:05.083929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1995a9bc-43cf-4671-bb45-56df4ae85d05 tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:05.096942 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4269/8472] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:26:05.101394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505006d3-a6f9-42b4-85b9-b250a0af7c4d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:05.112070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505006d3-a6f9-42b4-85b9-b250a0af7c4d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:05.116824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c407f61-e30e-481c-9edc-c4fcc0022a2d tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c13ee4bd591b4e2aadffea407493a488', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebd400ef653a468eb2481fdcd5b4ac45', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:05.117430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c407f61-e30e-481c-9edc-c4fcc0022a2d tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:05.117743 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c407f61-e30e-481c-9edc-c4fcc0022a2d tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:05.118143 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c407f61-e30e-481c-9edc-c4fcc0022a2d tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:05.118822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c407f61-e30e-481c-9edc-c4fcc0022a2d tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:05.120736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c407f61-e30e-481c-9edc-c4fcc0022a2d tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:05.149832 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4204/8473] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:26:05 2022] POST /identity/v3/projects => generated 349 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:26:05.154522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995a9bc-43cf-4671-bb45-56df4ae85d05 tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:05.161089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:05.165173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995a9bc-43cf-4671-bb45-56df4ae85d05 tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:05.228425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb556c78-5a0c-454e-87a6-27021b8a3158 tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c13ee4bd591b4e2aadffea407493a488', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebd400ef653a468eb2481fdcd5b4ac45', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:05.229054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb556c78-5a0c-454e-87a6-27021b8a3158 tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:05.229341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb556c78-5a0c-454e-87a6-27021b8a3158 tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:05.229621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb556c78-5a0c-454e-87a6-27021b8a3158 tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:05.230676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb556c78-5a0c-454e-87a6-27021b8a3158 tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:26:05.230990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb556c78-5a0c-454e-87a6-27021b8a3158 tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:05.232715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb556c78-5a0c-454e-87a6-27021b8a3158 tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:05.244526 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4270/8474] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:26:05.256916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c407f61-e30e-481c-9edc-c4fcc0022a2d tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:05.258896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6c407f61-e30e-481c-9edc-c4fcc0022a2d tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:05.295620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:05.325649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c407f61-e30e-481c-9edc-c4fcc0022a2d tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:05.333383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19201481-0845-4716-ab55-8328ef6bdb1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:05.333383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19201481-0845-4716-ab55-8328ef6bdb1a service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:05.333383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19201481-0845-4716-ab55-8328ef6bdb1a service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:05.333383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19201481-0845-4716-ab55-8328ef6bdb1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:05.340505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19201481-0845-4716-ab55-8328ef6bdb1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:05.341949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19201481-0845-4716-ab55-8328ef6bdb1a service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:05.351440 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4205/8475] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:26:05 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:26:05.473244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb556c78-5a0c-454e-87a6-27021b8a3158 tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:05.483926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb556c78-5a0c-454e-87a6-27021b8a3158 tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:05.488522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612d523-65bb-4b15-9697-7812f819e5cc tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c13ee4bd591b4e2aadffea407493a488', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebd400ef653a468eb2481fdcd5b4ac45', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:05.489145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612d523-65bb-4b15-9697-7812f819e5cc tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:05.489430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612d523-65bb-4b15-9697-7812f819e5cc tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:05.489707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612d523-65bb-4b15-9697-7812f819e5cc tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] PATH_INFO: `/v3/projects/fef535ad4a8d49a8bb92b88fa6dde3d6` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:05.490869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612d523-65bb-4b15-9697-7812f819e5cc tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] RBAC: Authorizing `identity:delete_project(project_id=fef535ad4a8d49a8bb92b88fa6dde3d6)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:05.492796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612d523-65bb-4b15-9697-7812f819e5cc tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:05.539483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9612d523-65bb-4b15-9697-7812f819e5cc tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fef535ad4a8d49a8bb92b88fa6dde3d6', 'request_id': 'req-9612d523-65bb-4b15-9697-7812f819e5cc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:05.546717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9612d523-65bb-4b15-9697-7812f819e5cc tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fef535ad4a8d49a8bb92b88fa6dde3d6', 'request_id': 'req-9612d523-65bb-4b15-9697-7812f819e5cc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:05.552425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9612d523-65bb-4b15-9697-7812f819e5cc tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fef535ad4a8d49a8bb92b88fa6dde3d6', 'request_id': 'req-9612d523-65bb-4b15-9697-7812f819e5cc'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:05.564952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9612d523-65bb-4b15-9697-7812f819e5cc tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] The token cache is being invalidate because project fef535ad4a8d49a8bb92b88fa6dde3d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:26:05.565264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9612d523-65bb-4b15-9697-7812f819e5cc tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:05.565892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9612d523-65bb-4b15-9697-7812f819e5cc tempest-QuotasTest-1229598692 tempest-QuotasTest-1229598692-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:05.567388 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4271/8476] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:26:05 2022] DELETE /identity/v3/projects/fef535ad4a8d49a8bb92b88fa6dde3d6 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:05.655317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19201481-0845-4716-ab55-8328ef6bdb1a service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:05.718456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19201481-0845-4716-ab55-8328ef6bdb1a service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:05.774034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbab12-17ca-4926-b083-97896372588f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:05.774630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbab12-17ca-4926-b083-97896372588f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:05.774917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbab12-17ca-4926-b083-97896372588f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:05.775197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbab12-17ca-4926-b083-97896372588f admin admin] PATH_INFO: `/v3/users/a4744dc6d236477f85e6f4860d8f7f3d` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:05.776330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcbab12-17ca-4926-b083-97896372588f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4744dc6d236477f85e6f4860d8f7f3d)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:05.778839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcbab12-17ca-4926-b083-97896372588f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:05.824479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6fcbab12-17ca-4926-b083-97896372588f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4744dc6d236477f85e6f4860d8f7f3d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:05.825190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6fcbab12-17ca-4926-b083-97896372588f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4744dc6d236477f85e6f4860d8f7f3d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:05.825812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6fcbab12-17ca-4926-b083-97896372588f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4744dc6d236477f85e6f4860d8f7f3d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:05.833070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6fcbab12-17ca-4926-b083-97896372588f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4744dc6d236477f85e6f4860d8f7f3d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:05.845509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6fcbab12-17ca-4926-b083-97896372588f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4744dc6d236477f85e6f4860d8f7f3d'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:05.858852 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4206/8477] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:26:05 2022] DELETE /identity/v3/users/a4744dc6d236477f85e6f4860d8f7f3d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:05.863373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d0cc5-f41d-4256-862b-d5708a35bf4c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:05.863662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d0cc5-f41d-4256-862b-d5708a35bf4c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:05.863944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d0cc5-f41d-4256-862b-d5708a35bf4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:05.882976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ff2d0cc5-f41d-4256-862b-d5708a35bf4c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:05.884206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ff2d0cc5-f41d-4256-862b-d5708a35bf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:05.942807 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4272/8478] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:26:05.952571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbab12-17ca-4926-b083-97896372588f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:05.953979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6fcbab12-17ca-4926-b083-97896372588f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:05.959441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:06.017001 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbab12-17ca-4926-b083-97896372588f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:06.020759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396c85e-c821-4c93-8195-6f242fb90db9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:06.021365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396c85e-c821-4c93-8195-6f242fb90db9 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:06.021657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396c85e-c821-4c93-8195-6f242fb90db9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:06.021935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396c85e-c821-4c93-8195-6f242fb90db9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:06.032055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396c85e-c821-4c93-8195-6f242fb90db9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:06.033564 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396c85e-c821-4c93-8195-6f242fb90db9 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:06.044242 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4207/8479] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:26:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:26:06.316517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d0cc5-f41d-4256-862b-d5708a35bf4c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:06.318108 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ff2d0cc5-f41d-4256-862b-d5708a35bf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:06.330804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d0cc5-f41d-4256-862b-d5708a35bf4c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:06.335627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7455f-423b-4514-bc8e-69e5939a23b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:06.336243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7455f-423b-4514-bc8e-69e5939a23b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:06.336526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7455f-423b-4514-bc8e-69e5939a23b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:06.336800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7455f-423b-4514-bc8e-69e5939a23b3 admin admin] PATH_INFO: `/v3/projects/d77bba39060140db9ab24fc212deebf6` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:06.337931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7455f-423b-4514-bc8e-69e5939a23b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d77bba39060140db9ab24fc212deebf6)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:06.339904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7455f-423b-4514-bc8e-69e5939a23b3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:06.357817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:06.388889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fea7455f-423b-4514-bc8e-69e5939a23b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd77bba39060140db9ab24fc212deebf6', 'request_id': 'req-fea7455f-423b-4514-bc8e-69e5939a23b3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:06.395130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fea7455f-423b-4514-bc8e-69e5939a23b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd77bba39060140db9ab24fc212deebf6', 'request_id': 'req-fea7455f-423b-4514-bc8e-69e5939a23b3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:06.400726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fea7455f-423b-4514-bc8e-69e5939a23b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd77bba39060140db9ab24fc212deebf6', 'request_id': 'req-fea7455f-423b-4514-bc8e-69e5939a23b3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:06.413101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fea7455f-423b-4514-bc8e-69e5939a23b3 admin admin] The token cache is being invalidate because project d77bba39060140db9ab24fc212deebf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:26:06.413396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fea7455f-423b-4514-bc8e-69e5939a23b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:06.414069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-fea7455f-423b-4514-bc8e-69e5939a23b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:06.415603 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4273/8480] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:26:06 2022] DELETE /identity/v3/projects/d77bba39060140db9ab24fc212deebf6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:06.420554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396c85e-c821-4c93-8195-6f242fb90db9 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:06.430474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396c85e-c821-4c93-8195-6f242fb90db9 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:06.486929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b4a3a-6050-4350-8dbe-83992eec48be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:06.487561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b4a3a-6050-4350-8dbe-83992eec48be admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:06.487844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b4a3a-6050-4350-8dbe-83992eec48be admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:06.488120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b4a3a-6050-4350-8dbe-83992eec48be admin admin] PATH_INFO: `/v3/users/c13ee4bd591b4e2aadffea407493a488` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:06.489236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b4a3a-6050-4350-8dbe-83992eec48be admin admin] RBAC: Authorizing `identity:delete_user(user_id=c13ee4bd591b4e2aadffea407493a488)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:06.491211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b4a3a-6050-4350-8dbe-83992eec48be admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:06.537821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9f4b4a3a-6050-4350-8dbe-83992eec48be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c13ee4bd591b4e2aadffea407493a488'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:06.538481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9f4b4a3a-6050-4350-8dbe-83992eec48be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c13ee4bd591b4e2aadffea407493a488'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:06.539112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9f4b4a3a-6050-4350-8dbe-83992eec48be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c13ee4bd591b4e2aadffea407493a488'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:06.547293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9f4b4a3a-6050-4350-8dbe-83992eec48be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c13ee4bd591b4e2aadffea407493a488'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:06.559501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-9f4b4a3a-6050-4350-8dbe-83992eec48be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c13ee4bd591b4e2aadffea407493a488'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:06.571377 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4208/8481] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:26:06 2022] DELETE /identity/v3/users/c13ee4bd591b4e2aadffea407493a488 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:06.849920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7455f-423b-4514-bc8e-69e5939a23b3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:06.851388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fea7455f-423b-4514-bc8e-69e5939a23b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:06.866840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7455f-423b-4514-bc8e-69e5939a23b3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:06.925400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19998c6-0b9c-48db-8db3-df57d9dd8ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:06.926060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19998c6-0b9c-48db-8db3-df57d9dd8ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:06.926337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19998c6-0b9c-48db-8db3-df57d9dd8ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:06.926647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19998c6-0b9c-48db-8db3-df57d9dd8ab5 admin admin] PATH_INFO: `/v3/projects/ebd400ef653a468eb2481fdcd5b4ac45` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:06.928025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19998c6-0b9c-48db-8db3-df57d9dd8ab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebd400ef653a468eb2481fdcd5b4ac45)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:06.929979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19998c6-0b9c-48db-8db3-df57d9dd8ab5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:06.976317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b19998c6-0b9c-48db-8db3-df57d9dd8ab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebd400ef653a468eb2481fdcd5b4ac45', 'request_id': 'req-b19998c6-0b9c-48db-8db3-df57d9dd8ab5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:06.982292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b19998c6-0b9c-48db-8db3-df57d9dd8ab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebd400ef653a468eb2481fdcd5b4ac45', 'request_id': 'req-b19998c6-0b9c-48db-8db3-df57d9dd8ab5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:06.988201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b19998c6-0b9c-48db-8db3-df57d9dd8ab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebd400ef653a468eb2481fdcd5b4ac45', 'request_id': 'req-b19998c6-0b9c-48db-8db3-df57d9dd8ab5'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:07.000612 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b19998c6-0b9c-48db-8db3-df57d9dd8ab5 admin admin] The token cache is being invalidate because project ebd400ef653a468eb2481fdcd5b4ac45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:26:07.000907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b19998c6-0b9c-48db-8db3-df57d9dd8ab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:07.001544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b19998c6-0b9c-48db-8db3-df57d9dd8ab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:07.003075 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4274/8482] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:26:06 2022] DELETE /identity/v3/projects/ebd400ef653a468eb2481fdcd5b4ac45 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:07.009241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb561434-f849-402f-8961-13a1d9d738a1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:07.009699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb561434-f849-402f-8961-13a1d9d738a1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:07.009788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb561434-f849-402f-8961-13a1d9d738a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:07.030448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fb561434-f849-402f-8961-13a1d9d738a1 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:07.031793 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fb561434-f849-402f-8961-13a1d9d738a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:07.051791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:07.093948 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4209/8483] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:26:07.102054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24b827-4343-41e5-bd86-8d07bfb9f57f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:07.102350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24b827-4343-41e5-bd86-8d07bfb9f57f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:07.102677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24b827-4343-41e5-bd86-8d07bfb9f57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:07.123387 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0f24b827-4343-41e5-bd86-8d07bfb9f57f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:07.124600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0f24b827-4343-41e5-bd86-8d07bfb9f57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:07.133277 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4275/8484] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:26:07.137789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb561434-f849-402f-8961-13a1d9d738a1 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:07.147605 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb561434-f849-402f-8961-13a1d9d738a1 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:07.152184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd70370-340d-474f-a6b7-d4b8555185fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:07.152778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd70370-340d-474f-a6b7-d4b8555185fc admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:07.153055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd70370-340d-474f-a6b7-d4b8555185fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:07.153327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd70370-340d-474f-a6b7-d4b8555185fc admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:07.153960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd70370-340d-474f-a6b7-d4b8555185fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:26:07.154259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd70370-340d-474f-a6b7-d4b8555185fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:07.155884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd70370-340d-474f-a6b7-d4b8555185fc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:07.168320 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4210/8485] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:26:07.172848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24b827-4343-41e5-bd86-8d07bfb9f57f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:07.182292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24b827-4343-41e5-bd86-8d07bfb9f57f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:07.187696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547abca5-f772-4b06-9eab-7ce22f7acbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:07.188286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547abca5-f772-4b06-9eab-7ce22f7acbc5 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:07.188568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547abca5-f772-4b06-9eab-7ce22f7acbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:07.188837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547abca5-f772-4b06-9eab-7ce22f7acbc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:07.189466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547abca5-f772-4b06-9eab-7ce22f7acbc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:07.191393 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547abca5-f772-4b06-9eab-7ce22f7acbc5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:07.218962 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4276/8486] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:26:07 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:26:07.223320 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd70370-340d-474f-a6b7-d4b8555185fc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:07.233686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd70370-340d-474f-a6b7-d4b8555185fc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:07.292433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0021a6-da4c-4b6b-804f-7a762c3e295a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:07.293590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0021a6-da4c-4b6b-804f-7a762c3e295a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:07.293869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0021a6-da4c-4b6b-804f-7a762c3e295a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:07.294141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0021a6-da4c-4b6b-804f-7a762c3e295a admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:07.294790 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0021a6-da4c-4b6b-804f-7a762c3e295a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:07.296719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0021a6-da4c-4b6b-804f-7a762c3e295a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:07.317062 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4211/8487] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:26:07 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:26:07.321342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547abca5-f772-4b06-9eab-7ce22f7acbc5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:07.332846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547abca5-f772-4b06-9eab-7ce22f7acbc5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:07.338010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41926f01-3833-40a2-9d02-de6477df898a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:07.338652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41926f01-3833-40a2-9d02-de6477df898a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:07.338931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41926f01-3833-40a2-9d02-de6477df898a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:07.339200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41926f01-3833-40a2-9d02-de6477df898a admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:07.340329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41926f01-3833-40a2-9d02-de6477df898a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:07.341894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41926f01-3833-40a2-9d02-de6477df898a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:07.343236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:07.356870 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4277/8488] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:26:07.360804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0021a6-da4c-4b6b-804f-7a762c3e295a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:07.370865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0021a6-da4c-4b6b-804f-7a762c3e295a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:07.375485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221bc84-55a5-4a97-84ff-e7b394532876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:07.376092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221bc84-55a5-4a97-84ff-e7b394532876 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:07.376359 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221bc84-55a5-4a97-84ff-e7b394532876 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:07.376637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221bc84-55a5-4a97-84ff-e7b394532876 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:07.377267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221bc84-55a5-4a97-84ff-e7b394532876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:07.379029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221bc84-55a5-4a97-84ff-e7b394532876 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:07.389437 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4212/8489] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:26:07.393463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41926f01-3833-40a2-9d02-de6477df898a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:07.404026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41926f01-3833-40a2-9d02-de6477df898a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:07.408677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2f61e-9121-4c81-b153-e591b554a07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:07.409400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2f61e-9121-4c81-b153-e591b554a07a admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:07.409682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2f61e-9121-4c81-b153-e591b554a07a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:07.409958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2f61e-9121-4c81-b153-e591b554a07a admin admin] PATH_INFO: `/v3/projects/f5dafcdc5f24461aad500fc395fc61f7/users/e5de478e97b14175847466e0f7dcaffa/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:07.425519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2f61e-9121-4c81-b153-e591b554a07a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5dafcdc5f24461aad500fc395fc61f7, user_id=e5de478e97b14175847466e0f7dcaffa, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:07.428248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2f61e-9121-4c81-b153-e591b554a07a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:07.444143 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4278/8490] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:26:07 2022] PUT /identity/v3/projects/f5dafcdc5f24461aad500fc395fc61f7/users/e5de478e97b14175847466e0f7dcaffa/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:07.456618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152de61-5033-4a74-9a63-3a236c66a636 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:07.456907 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152de61-5033-4a74-9a63-3a236c66a636 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:07.457187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152de61-5033-4a74-9a63-3a236c66a636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:07.486099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b152de61-5033-4a74-9a63-3a236c66a636 None None] MFA Rules not processed for user `e5de478e97b14175847466e0f7dcaffa`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:07.487392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b152de61-5033-4a74-9a63-3a236c66a636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:07.550384 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4213/8491] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:26:07.554907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2f61e-9121-4c81-b153-e591b554a07a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:07.616812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2f61e-9121-4c81-b153-e591b554a07a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:07.621410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467800c7-99ed-4989-9736-c6d1108d46a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:07.622000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467800c7-99ed-4989-9736-c6d1108d46a8 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:07.622279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467800c7-99ed-4989-9736-c6d1108d46a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:07.622580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467800c7-99ed-4989-9736-c6d1108d46a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:07.623201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467800c7-99ed-4989-9736-c6d1108d46a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:07.625088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467800c7-99ed-4989-9736-c6d1108d46a8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:07.654286 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4279/8492] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:26:07 2022] POST /identity/v3/projects => generated 393 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:26:07.658369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152de61-5033-4a74-9a63-3a236c66a636 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:07.669332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152de61-5033-4a74-9a63-3a236c66a636 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:07.685862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:07.728134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c075b36-1a23-4101-8378-05c11b7972f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:07.728737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c075b36-1a23-4101-8378-05c11b7972f0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:07.729016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c075b36-1a23-4101-8378-05c11b7972f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:07.729290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c075b36-1a23-4101-8378-05c11b7972f0 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:07.729901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c075b36-1a23-4101-8378-05c11b7972f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:07.731917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c075b36-1a23-4101-8378-05c11b7972f0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:07.752559 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4214/8493] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:26:07 2022] POST /identity/v3/users => generated 303 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:26:07.756584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467800c7-99ed-4989-9736-c6d1108d46a8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:07.767570 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467800c7-99ed-4989-9736-c6d1108d46a8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:07.772598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d04b9c-5eca-4f2c-80d4-1aabf399027d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:07.773203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d04b9c-5eca-4f2c-80d4-1aabf399027d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:07.773488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d04b9c-5eca-4f2c-80d4-1aabf399027d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:07.773764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d04b9c-5eca-4f2c-80d4-1aabf399027d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:07.774402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d04b9c-5eca-4f2c-80d4-1aabf399027d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:07.776041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d04b9c-5eca-4f2c-80d4-1aabf399027d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:07.788534 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4280/8494] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:26:07.792949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c075b36-1a23-4101-8378-05c11b7972f0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:07.802994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c075b36-1a23-4101-8378-05c11b7972f0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:07.807777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe897a35-e304-4730-bc4f-0b57442dce02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:07.808493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe897a35-e304-4730-bc4f-0b57442dce02 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:07.808794 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe897a35-e304-4730-bc4f-0b57442dce02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:07.809068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe897a35-e304-4730-bc4f-0b57442dce02 admin admin] PATH_INFO: `/v3/projects/6c1ea2352f7e4006a609d14cc80945c0/users/c9d202798afc46e79eb46899ee9813fe/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:07.825175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe897a35-e304-4730-bc4f-0b57442dce02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c1ea2352f7e4006a609d14cc80945c0, user_id=c9d202798afc46e79eb46899ee9813fe, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:07.827869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe897a35-e304-4730-bc4f-0b57442dce02 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:07.841818 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4215/8495] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:26:07 2022] PUT /identity/v3/projects/6c1ea2352f7e4006a609d14cc80945c0/users/c9d202798afc46e79eb46899ee9813fe/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:26:07.845829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d04b9c-5eca-4f2c-80d4-1aabf399027d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:07.856644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d04b9c-5eca-4f2c-80d4-1aabf399027d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:07.913284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ccf979-5fc2-42d0-b1a6-5a33452c0495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:07.913815 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccf979-5fc2-42d0-b1a6-5a33452c0495 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:07.914098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccf979-5fc2-42d0-b1a6-5a33452c0495 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:07.914376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccf979-5fc2-42d0-b1a6-5a33452c0495 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:07.915038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ccf979-5fc2-42d0-b1a6-5a33452c0495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:07.916954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ccf979-5fc2-42d0-b1a6-5a33452c0495 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:07.926660 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4281/8496] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:26:07.930835 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe897a35-e304-4730-bc4f-0b57442dce02 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:07.941385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe897a35-e304-4730-bc4f-0b57442dce02 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:07.946028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0171ea-0dcb-4649-adb1-33f6ac77a88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:07.946718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0171ea-0dcb-4649-adb1-33f6ac77a88c admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:07.946999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0171ea-0dcb-4649-adb1-33f6ac77a88c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:07.947270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0171ea-0dcb-4649-adb1-33f6ac77a88c admin admin] PATH_INFO: `/v3/projects/6c1ea2352f7e4006a609d14cc80945c0/users/c9d202798afc46e79eb46899ee9813fe/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:07.949682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0171ea-0dcb-4649-adb1-33f6ac77a88c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c1ea2352f7e4006a609d14cc80945c0, user_id=c9d202798afc46e79eb46899ee9813fe, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:07.952384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0171ea-0dcb-4649-adb1-33f6ac77a88c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:07.962425 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4216/8497] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:26:07 2022] PUT /identity/v3/projects/6c1ea2352f7e4006a609d14cc80945c0/users/c9d202798afc46e79eb46899ee9813fe/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:07.973131 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f08e7-030f-445a-a16d-3fafbab6dba9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:07.973447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f08e7-030f-445a-a16d-3fafbab6dba9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:07.973724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f08e7-030f-445a-a16d-3fafbab6dba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:08.001377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-814f08e7-030f-445a-a16d-3fafbab6dba9 None None] MFA Rules not processed for user `c9d202798afc46e79eb46899ee9813fe`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:08.002623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-814f08e7-030f-445a-a16d-3fafbab6dba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:08.033645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:08.073208 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4282/8498] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:07 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:26:08.084871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0171ea-0dcb-4649-adb1-33f6ac77a88c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:08.086691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ca0171ea-0dcb-4649-adb1-33f6ac77a88c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:08.147728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0171ea-0dcb-4649-adb1-33f6ac77a88c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:08.151442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8561739-23b6-4905-b4bb-3a84dd398a04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:08.152051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8561739-23b6-4905-b4bb-3a84dd398a04 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:08.152334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8561739-23b6-4905-b4bb-3a84dd398a04 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:08.152611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8561739-23b6-4905-b4bb-3a84dd398a04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:08.161541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8561739-23b6-4905-b4bb-3a84dd398a04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:08.163027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8561739-23b6-4905-b4bb-3a84dd398a04 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:08.172975 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4217/8499] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:26:08 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:26:09.988660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f08e7-030f-445a-a16d-3fafbab6dba9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:10.051966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f08e7-030f-445a-a16d-3fafbab6dba9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:10.056537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea3b69-3b8b-401a-99ab-e16346b22dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:10.057212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea3b69-3b8b-401a-99ab-e16346b22dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:10.057591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea3b69-3b8b-401a-99ab-e16346b22dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:10.057841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea3b69-3b8b-401a-99ab-e16346b22dc9 admin admin] PATH_INFO: `/v3/users/e5de478e97b14175847466e0f7dcaffa` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:10.059126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea3b69-3b8b-401a-99ab-e16346b22dc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5de478e97b14175847466e0f7dcaffa)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:10.062363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea3b69-3b8b-401a-99ab-e16346b22dc9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:10.112957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-aaea3b69-3b8b-401a-99ab-e16346b22dc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5de478e97b14175847466e0f7dcaffa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:10.120019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-aaea3b69-3b8b-401a-99ab-e16346b22dc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5de478e97b14175847466e0f7dcaffa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:10.134184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-aaea3b69-3b8b-401a-99ab-e16346b22dc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5de478e97b14175847466e0f7dcaffa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:10.134830 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-aaea3b69-3b8b-401a-99ab-e16346b22dc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5de478e97b14175847466e0f7dcaffa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:10.142575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-aaea3b69-3b8b-401a-99ab-e16346b22dc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5de478e97b14175847466e0f7dcaffa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:10.149482 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4283/8500] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:26:09 2022] DELETE /identity/v3/users/e5de478e97b14175847466e0f7dcaffa => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:10.153809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e698d04-8fa9-448d-afb0-b046c9a93fea None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:10.154302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e698d04-8fa9-448d-afb0-b046c9a93fea None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:10.154394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e698d04-8fa9-448d-afb0-b046c9a93fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:10.174523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9e698d04-8fa9-448d-afb0-b046c9a93fea None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:10.175774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9e698d04-8fa9-448d-afb0-b046c9a93fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:10.242454 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4218/8501] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:26:10.253025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea3b69-3b8b-401a-99ab-e16346b22dc9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:10.254568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-aaea3b69-3b8b-401a-99ab-e16346b22dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:10.313049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:10.322913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea3b69-3b8b-401a-99ab-e16346b22dc9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:10.326958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425725b-13e6-49e5-8105-967e095f5078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:10.327558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425725b-13e6-49e5-8105-967e095f5078 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:10.327835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425725b-13e6-49e5-8105-967e095f5078 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:10.328104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425725b-13e6-49e5-8105-967e095f5078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:10.337167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425725b-13e6-49e5-8105-967e095f5078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:10.338608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425725b-13e6-49e5-8105-967e095f5078 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:10.348506 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4284/8502] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:26:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:26:10.617071 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e698d04-8fa9-448d-afb0-b046c9a93fea None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:10.619555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9e698d04-8fa9-448d-afb0-b046c9a93fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:10.633862 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e698d04-8fa9-448d-afb0-b046c9a93fea None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:10.640038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a4ad0-da31-4c8d-b777-80907153cf70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:10.640858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a4ad0-da31-4c8d-b777-80907153cf70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:10.641315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a4ad0-da31-4c8d-b777-80907153cf70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:10.641707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a4ad0-da31-4c8d-b777-80907153cf70 admin admin] PATH_INFO: `/v3/projects/f5dafcdc5f24461aad500fc395fc61f7` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:10.643111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a4ad0-da31-4c8d-b777-80907153cf70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5dafcdc5f24461aad500fc395fc61f7)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:10.645345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a4ad0-da31-4c8d-b777-80907153cf70 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:10.656903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:10.699514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-da5a4ad0-da31-4c8d-b777-80907153cf70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5dafcdc5f24461aad500fc395fc61f7', 'request_id': 'req-da5a4ad0-da31-4c8d-b777-80907153cf70'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:10.707260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-da5a4ad0-da31-4c8d-b777-80907153cf70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5dafcdc5f24461aad500fc395fc61f7', 'request_id': 'req-da5a4ad0-da31-4c8d-b777-80907153cf70'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:10.713135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-da5a4ad0-da31-4c8d-b777-80907153cf70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5dafcdc5f24461aad500fc395fc61f7', 'request_id': 'req-da5a4ad0-da31-4c8d-b777-80907153cf70'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:10.724259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-da5a4ad0-da31-4c8d-b777-80907153cf70 admin admin] The token cache is being invalidate because project f5dafcdc5f24461aad500fc395fc61f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:26:10.724554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-da5a4ad0-da31-4c8d-b777-80907153cf70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:10.725201 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-da5a4ad0-da31-4c8d-b777-80907153cf70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:10.727001 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4219/8503] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:26:10 2022] DELETE /identity/v3/projects/f5dafcdc5f24461aad500fc395fc61f7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:10.731374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425725b-13e6-49e5-8105-967e095f5078 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:10.741083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425725b-13e6-49e5-8105-967e095f5078 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:10.804508 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc4856-bf7e-4b7e-819d-968cccb7b7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:10.805141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc4856-bf7e-4b7e-819d-968cccb7b7bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:10.805420 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc4856-bf7e-4b7e-819d-968cccb7b7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:10.805696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc4856-bf7e-4b7e-819d-968cccb7b7bc admin admin] PATH_INFO: `/v3/users/c9d202798afc46e79eb46899ee9813fe` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:10.806884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cc4856-bf7e-4b7e-819d-968cccb7b7bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9d202798afc46e79eb46899ee9813fe)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:10.809568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cc4856-bf7e-4b7e-819d-968cccb7b7bc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:10.855745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-40cc4856-bf7e-4b7e-819d-968cccb7b7bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9d202798afc46e79eb46899ee9813fe'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:10.862764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-40cc4856-bf7e-4b7e-819d-968cccb7b7bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9d202798afc46e79eb46899ee9813fe'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:10.875159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-40cc4856-bf7e-4b7e-819d-968cccb7b7bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9d202798afc46e79eb46899ee9813fe'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:10.875837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-40cc4856-bf7e-4b7e-819d-968cccb7b7bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9d202798afc46e79eb46899ee9813fe'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:10.883124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-40cc4856-bf7e-4b7e-819d-968cccb7b7bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9d202798afc46e79eb46899ee9813fe'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:10.889177 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4285/8504] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:26:10 2022] DELETE /identity/v3/users/c9d202798afc46e79eb46899ee9813fe => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:11.063133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a4ad0-da31-4c8d-b777-80907153cf70 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:11.064841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-da5a4ad0-da31-4c8d-b777-80907153cf70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:11.078693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a4ad0-da31-4c8d-b777-80907153cf70 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:11.141935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513fe23-21b0-4b15-80e7-b96291c61d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:11.142607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513fe23-21b0-4b15-80e7-b96291c61d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:11.142890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513fe23-21b0-4b15-80e7-b96291c61d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:11.143165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513fe23-21b0-4b15-80e7-b96291c61d25 admin admin] PATH_INFO: `/v3/projects/6c1ea2352f7e4006a609d14cc80945c0` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:11.144277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4513fe23-21b0-4b15-80e7-b96291c61d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c1ea2352f7e4006a609d14cc80945c0)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:11.146182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4513fe23-21b0-4b15-80e7-b96291c61d25 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:11.192873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4513fe23-21b0-4b15-80e7-b96291c61d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c1ea2352f7e4006a609d14cc80945c0', 'request_id': 'req-4513fe23-21b0-4b15-80e7-b96291c61d25'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:11.200218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4513fe23-21b0-4b15-80e7-b96291c61d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c1ea2352f7e4006a609d14cc80945c0', 'request_id': 'req-4513fe23-21b0-4b15-80e7-b96291c61d25'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:11.205826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4513fe23-21b0-4b15-80e7-b96291c61d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c1ea2352f7e4006a609d14cc80945c0', 'request_id': 'req-4513fe23-21b0-4b15-80e7-b96291c61d25'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:11.217211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4513fe23-21b0-4b15-80e7-b96291c61d25 admin admin] The token cache is being invalidate because project 6c1ea2352f7e4006a609d14cc80945c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:26:11.217497 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4513fe23-21b0-4b15-80e7-b96291c61d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:11.218161 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4513fe23-21b0-4b15-80e7-b96291c61d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:11.220055 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4220/8505] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:26:11 2022] DELETE /identity/v3/projects/6c1ea2352f7e4006a609d14cc80945c0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:11.224882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad07db-95aa-43dd-96da-4bde33e3dff8 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:11.225407 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad07db-95aa-43dd-96da-4bde33e3dff8 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:11.225718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad07db-95aa-43dd-96da-4bde33e3dff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:11.246207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2bad07db-95aa-43dd-96da-4bde33e3dff8 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:11.247392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2bad07db-95aa-43dd-96da-4bde33e3dff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:11.309990 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4286/8506] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:26:11.317881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ab786-c497-4383-ae45-f555fe12bd02 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:11.318165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ab786-c497-4383-ae45-f555fe12bd02 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:11.318442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ab786-c497-4383-ae45-f555fe12bd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:11.338318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-664ab786-c497-4383-ae45-f555fe12bd02 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:11.340032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-664ab786-c497-4383-ae45-f555fe12bd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:11.347909 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4221/8507] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:26:11.352214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad07db-95aa-43dd-96da-4bde33e3dff8 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:11.362586 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad07db-95aa-43dd-96da-4bde33e3dff8 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:11.367155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a044e-ddc4-427f-96aa-c51e59244449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:11.367749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a044e-ddc4-427f-96aa-c51e59244449 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:11.368062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a044e-ddc4-427f-96aa-c51e59244449 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:11.368340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a044e-ddc4-427f-96aa-c51e59244449 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:11.368984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a044e-ddc4-427f-96aa-c51e59244449 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:26:11.369290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a044e-ddc4-427f-96aa-c51e59244449 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:11.370980 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a044e-ddc4-427f-96aa-c51e59244449 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:11.373309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:11.384643 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4287/8508] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:26:11.389252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ab786-c497-4383-ae45-f555fe12bd02 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:11.398764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ab786-c497-4383-ae45-f555fe12bd02 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:11.404026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009dc383-8620-498c-91ba-42d16822ae70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:11.404620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009dc383-8620-498c-91ba-42d16822ae70 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:11.404904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009dc383-8620-498c-91ba-42d16822ae70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:11.405180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009dc383-8620-498c-91ba-42d16822ae70 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:11.405912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009dc383-8620-498c-91ba-42d16822ae70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:11.407889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009dc383-8620-498c-91ba-42d16822ae70 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:11.429382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:11.437348 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4222/8509] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:26:11 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:26:11.441289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a044e-ddc4-427f-96aa-c51e59244449 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:11.452093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a044e-ddc4-427f-96aa-c51e59244449 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:11.509373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8f069-050d-430c-aeb1-1479a271b190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:11.509907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8f069-050d-430c-aeb1-1479a271b190 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:11.510177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8f069-050d-430c-aeb1-1479a271b190 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:11.510449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8f069-050d-430c-aeb1-1479a271b190 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:11.511099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8f069-050d-430c-aeb1-1479a271b190 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:11.513459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8f069-050d-430c-aeb1-1479a271b190 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:11.534282 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4288/8510] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:26:11 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:26:11.538628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009dc383-8620-498c-91ba-42d16822ae70 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:11.549287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009dc383-8620-498c-91ba-42d16822ae70 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:11.554074 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d79e4f-b41c-4ae0-885c-4b1abd2e98b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:11.554712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d79e4f-b41c-4ae0-885c-4b1abd2e98b3 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:11.554998 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d79e4f-b41c-4ae0-885c-4b1abd2e98b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:11.555304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d79e4f-b41c-4ae0-885c-4b1abd2e98b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:11.555944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d79e4f-b41c-4ae0-885c-4b1abd2e98b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:11.557555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d79e4f-b41c-4ae0-885c-4b1abd2e98b3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:11.567635 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4223/8511] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:26:11.571449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8f069-050d-430c-aeb1-1479a271b190 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:11.581711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8f069-050d-430c-aeb1-1479a271b190 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:11.586320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d524364-caa5-48c3-8d4a-2c6a853dc68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:11.586946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d524364-caa5-48c3-8d4a-2c6a853dc68b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:11.587220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d524364-caa5-48c3-8d4a-2c6a853dc68b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:11.587492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d524364-caa5-48c3-8d4a-2c6a853dc68b admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:11.588119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d524364-caa5-48c3-8d4a-2c6a853dc68b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:11.590328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d524364-caa5-48c3-8d4a-2c6a853dc68b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:11.600017 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4289/8512] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:26:11.603979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d79e4f-b41c-4ae0-885c-4b1abd2e98b3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:11.614031 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d79e4f-b41c-4ae0-885c-4b1abd2e98b3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:11.618714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60676d0-a196-456e-9154-53d768c9dad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:11.619350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60676d0-a196-456e-9154-53d768c9dad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:11.619628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60676d0-a196-456e-9154-53d768c9dad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:11.619902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60676d0-a196-456e-9154-53d768c9dad0 admin admin] PATH_INFO: `/v3/projects/016b1615fe87445e8b5e888475289117/users/dc05f35ba5764669b13d8454a3e35d08/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:11.634012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60676d0-a196-456e-9154-53d768c9dad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=016b1615fe87445e8b5e888475289117, user_id=dc05f35ba5764669b13d8454a3e35d08, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:11.637225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60676d0-a196-456e-9154-53d768c9dad0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:11.650010 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4224/8513] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:26:11 2022] PUT /identity/v3/projects/016b1615fe87445e8b5e888475289117/users/dc05f35ba5764669b13d8454a3e35d08/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:11.661722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59bb1a-ed59-4d74-a710-704380e12fbd None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:11.662010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59bb1a-ed59-4d74-a710-704380e12fbd None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:11.662285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59bb1a-ed59-4d74-a710-704380e12fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:11.689647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2b59bb1a-ed59-4d74-a710-704380e12fbd None None] MFA Rules not processed for user `dc05f35ba5764669b13d8454a3e35d08`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:11.690930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2b59bb1a-ed59-4d74-a710-704380e12fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:11.755188 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4290/8514] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:26:11.765715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60676d0-a196-456e-9154-53d768c9dad0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:11.767384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a60676d0-a196-456e-9154-53d768c9dad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:11.829616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60676d0-a196-456e-9154-53d768c9dad0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:11.834710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4dd28d-306d-480a-8e19-204dafa9b953 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:11.834710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4dd28d-306d-480a-8e19-204dafa9b953 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:11.834710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4dd28d-306d-480a-8e19-204dafa9b953 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:11.834710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4dd28d-306d-480a-8e19-204dafa9b953 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:11.843552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4dd28d-306d-480a-8e19-204dafa9b953 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:11.845007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4dd28d-306d-480a-8e19-204dafa9b953 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:11.853854 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4225/8515] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:26:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:26:18.431866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59bb1a-ed59-4d74-a710-704380e12fbd None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:18.433545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2b59bb1a-ed59-4d74-a710-704380e12fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:18.521405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:18.540257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59bb1a-ed59-4d74-a710-704380e12fbd None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:18.547774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5dc835-b217-4c0a-88da-a4a41ac4e6f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:18.548800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5dc835-b217-4c0a-88da-a4a41ac4e6f4 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:18.548800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5dc835-b217-4c0a-88da-a4a41ac4e6f4 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:18.549060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5dc835-b217-4c0a-88da-a4a41ac4e6f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:18.551322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5b5dc835-b217-4c0a-88da-a4a41ac4e6f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:18.576126 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5dc835-b217-4c0a-88da-a4a41ac4e6f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:18.577742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5dc835-b217-4c0a-88da-a4a41ac4e6f4 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:18.598368 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4291/8516] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:26:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:26:21.849768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:24.923770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5760c56-9d79-4ce0-a259-15b62c136896 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:24.924402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5760c56-9d79-4ce0-a259-15b62c136896 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:24.924402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5760c56-9d79-4ce0-a259-15b62c136896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:24.943446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c5760c56-9d79-4ce0-a259-15b62c136896 None None] MFA Rules not processed for user `98361240b014405cb20cf9c145061be6`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:24.944647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c5760c56-9d79-4ce0-a259-15b62c136896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:25.005805 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4226/8517] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:26:28.585156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:34.995602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=25 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:52.273446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5dc835-b217-4c0a-88da-a4a41ac4e6f4 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:52.349346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5dc835-b217-4c0a-88da-a4a41ac4e6f4 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:52.355805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a2273-b099-4c07-a83c-24939825d9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:52.356437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a2273-b099-4c07-a83c-24939825d9ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:52.356726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a2273-b099-4c07-a83c-24939825d9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:52.356998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a2273-b099-4c07-a83c-24939825d9ec admin admin] PATH_INFO: `/v3/users/dc05f35ba5764669b13d8454a3e35d08` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:52.358317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a2273-b099-4c07-a83c-24939825d9ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc05f35ba5764669b13d8454a3e35d08)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:52.360480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a2273-b099-4c07-a83c-24939825d9ec admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:52.414438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e56a2273-b099-4c07-a83c-24939825d9ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc05f35ba5764669b13d8454a3e35d08'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:52.423365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e56a2273-b099-4c07-a83c-24939825d9ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc05f35ba5764669b13d8454a3e35d08'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:52.436572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e56a2273-b099-4c07-a83c-24939825d9ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc05f35ba5764669b13d8454a3e35d08'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:52.437307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e56a2273-b099-4c07-a83c-24939825d9ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc05f35ba5764669b13d8454a3e35d08'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:52.445920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e56a2273-b099-4c07-a83c-24939825d9ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc05f35ba5764669b13d8454a3e35d08'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:52.453016 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4292/8518] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:26:52 2022] DELETE /identity/v3/users/dc05f35ba5764669b13d8454a3e35d08 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:52.457849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8647f-4038-4ffc-b068-468635a97f99 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:52.458377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8647f-4038-4ffc-b068-468635a97f99 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:52.458523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8647f-4038-4ffc-b068-468635a97f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:52.490921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8db8647f-4038-4ffc-b068-468635a97f99 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:52.492132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8db8647f-4038-4ffc-b068-468635a97f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:52.558408 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4227/8519] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:26:52.569192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a2273-b099-4c07-a83c-24939825d9ec admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:52.570777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e56a2273-b099-4c07-a83c-24939825d9ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:52.658796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a2273-b099-4c07-a83c-24939825d9ec admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:52.662804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550f040-89a5-44f6-84d0-e65f05ec14ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:52.663450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550f040-89a5-44f6-84d0-e65f05ec14ad service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:52.663771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550f040-89a5-44f6-84d0-e65f05ec14ad service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:52.664116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550f040-89a5-44f6-84d0-e65f05ec14ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:52.675465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550f040-89a5-44f6-84d0-e65f05ec14ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:52.677269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550f040-89a5-44f6-84d0-e65f05ec14ad service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:52.688747 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4293/8520] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:26:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:26:52.876770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8647f-4038-4ffc-b068-468635a97f99 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:52.878478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8db8647f-4038-4ffc-b068-468635a97f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:52.892349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8647f-4038-4ffc-b068-468635a97f99 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:52.897018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8aaba-eb22-4260-b608-2797ee504dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:52.897737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8aaba-eb22-4260-b608-2797ee504dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:52.898027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8aaba-eb22-4260-b608-2797ee504dea admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:52.898302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8aaba-eb22-4260-b608-2797ee504dea admin admin] PATH_INFO: `/v3/projects/016b1615fe87445e8b5e888475289117` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:52.899508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8aaba-eb22-4260-b608-2797ee504dea admin admin] RBAC: Authorizing `identity:delete_project(project_id=016b1615fe87445e8b5e888475289117)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:52.901623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8aaba-eb22-4260-b608-2797ee504dea admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:52.953858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-94b8aaba-eb22-4260-b608-2797ee504dea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '016b1615fe87445e8b5e888475289117', 'request_id': 'req-94b8aaba-eb22-4260-b608-2797ee504dea'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:52.963251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-94b8aaba-eb22-4260-b608-2797ee504dea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '016b1615fe87445e8b5e888475289117', 'request_id': 'req-94b8aaba-eb22-4260-b608-2797ee504dea'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:52.970281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-94b8aaba-eb22-4260-b608-2797ee504dea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '016b1615fe87445e8b5e888475289117', 'request_id': 'req-94b8aaba-eb22-4260-b608-2797ee504dea'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:52.984011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-94b8aaba-eb22-4260-b608-2797ee504dea admin admin] The token cache is being invalidate because project 016b1615fe87445e8b5e888475289117 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:26:52.984315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-94b8aaba-eb22-4260-b608-2797ee504dea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:52.985119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-94b8aaba-eb22-4260-b608-2797ee504dea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:26:52.986654 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4228/8521] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:26:52 2022] DELETE /identity/v3/projects/016b1615fe87445e8b5e888475289117 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:52.991917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e6748-3f7a-4106-8cdc-f5acaf36e872 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:52.992210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e6748-3f7a-4106-8cdc-f5acaf36e872 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:52.992492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e6748-3f7a-4106-8cdc-f5acaf36e872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:53.007750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:53.014045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0c0e6748-3f7a-4106-8cdc-f5acaf36e872 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:53.016772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0c0e6748-3f7a-4106-8cdc-f5acaf36e872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:53.087098 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4294/8522] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:26:53.095705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c726222-539f-43ab-a727-482b3c1c6a7a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:53.096000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c726222-539f-43ab-a727-482b3c1c6a7a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:53.096308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c726222-539f-43ab-a727-482b3c1c6a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:53.118064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5c726222-539f-43ab-a727-482b3c1c6a7a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:53.119542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5c726222-539f-43ab-a727-482b3c1c6a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:53.128714 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4229/8523] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:26:53.133256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e6748-3f7a-4106-8cdc-f5acaf36e872 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:53.144578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e6748-3f7a-4106-8cdc-f5acaf36e872 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:53.149251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a6ecc-e523-459c-aee8-fe50b4073ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:53.149845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a6ecc-e523-459c-aee8-fe50b4073ecb admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:53.150124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a6ecc-e523-459c-aee8-fe50b4073ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:53.150394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a6ecc-e523-459c-aee8-fe50b4073ecb admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:53.151585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a6ecc-e523-459c-aee8-fe50b4073ecb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:26:53.151893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a6ecc-e523-459c-aee8-fe50b4073ecb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:53.153467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a6ecc-e523-459c-aee8-fe50b4073ecb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:53.167406 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4295/8524] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:26:53.172055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c726222-539f-43ab-a727-482b3c1c6a7a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:53.185059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c726222-539f-43ab-a727-482b3c1c6a7a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:53.190283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c0362-a794-4346-bc7b-f05a40a479d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:53.191044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c0362-a794-4346-bc7b-f05a40a479d7 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:53.191365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c0362-a794-4346-bc7b-f05a40a479d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:53.191695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c0362-a794-4346-bc7b-f05a40a479d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:53.192438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c0362-a794-4346-bc7b-f05a40a479d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:53.194512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c0362-a794-4346-bc7b-f05a40a479d7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:53.229848 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4230/8525] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:26:53 2022] POST /identity/v3/projects => generated 361 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:26:53.232051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a6ecc-e523-459c-aee8-fe50b4073ecb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:53.243090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a6ecc-e523-459c-aee8-fe50b4073ecb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:53.309941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c8530-7b55-4d6b-8b05-c48796402646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:53.310683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c8530-7b55-4d6b-8b05-c48796402646 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:53.310939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c8530-7b55-4d6b-8b05-c48796402646 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:53.311300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c8530-7b55-4d6b-8b05-c48796402646 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:53.311913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c8530-7b55-4d6b-8b05-c48796402646 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:53.314269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c8530-7b55-4d6b-8b05-c48796402646 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:53.338863 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4296/8526] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:26:53 2022] POST /identity/v3/users => generated 281 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:26:53.343697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c0362-a794-4346-bc7b-f05a40a479d7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:53.354575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c0362-a794-4346-bc7b-f05a40a479d7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:53.359248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41bd71-61dd-4034-adab-b85106a5d4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:53.359857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41bd71-61dd-4034-adab-b85106a5d4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:53.360140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41bd71-61dd-4034-adab-b85106a5d4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:53.360417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41bd71-61dd-4034-adab-b85106a5d4c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:53.361059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f41bd71-61dd-4034-adab-b85106a5d4c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:53.362692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f41bd71-61dd-4034-adab-b85106a5d4c8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:53.372974 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4231/8527] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:26:53.376935 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c8530-7b55-4d6b-8b05-c48796402646 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:53.387598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c8530-7b55-4d6b-8b05-c48796402646 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:53.392467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b1a55-f656-406b-a404-d25e7d3844ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:53.393099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b1a55-f656-406b-a404-d25e7d3844ac admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:53.393382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b1a55-f656-406b-a404-d25e7d3844ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:53.393658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b1a55-f656-406b-a404-d25e7d3844ac admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:53.394298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b1a55-f656-406b-a404-d25e7d3844ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:53.395932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b1a55-f656-406b-a404-d25e7d3844ac admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:53.406797 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4297/8528] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:26:53.411267 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41bd71-61dd-4034-adab-b85106a5d4c8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:53.421823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41bd71-61dd-4034-adab-b85106a5d4c8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:53.426549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0fb69c-52d4-4561-a48d-3712df119881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:53.427197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0fb69c-52d4-4561-a48d-3712df119881 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:53.427480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0fb69c-52d4-4561-a48d-3712df119881 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:53.427755 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0fb69c-52d4-4561-a48d-3712df119881 admin admin] PATH_INFO: `/v3/projects/4e0c73ba25584e8980bdaeb5cdbdff32/users/505e46d5e4134b119509671714c0030f/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:53.442923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0fb69c-52d4-4561-a48d-3712df119881 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e0c73ba25584e8980bdaeb5cdbdff32, user_id=505e46d5e4134b119509671714c0030f, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:53.445614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0fb69c-52d4-4561-a48d-3712df119881 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:53.458543 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4232/8529] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:26:53 2022] PUT /identity/v3/projects/4e0c73ba25584e8980bdaeb5cdbdff32/users/505e46d5e4134b119509671714c0030f/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:26:53.469818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0d45a-4e15-403d-a2bb-62340fce0e54 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:53.470152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0d45a-4e15-403d-a2bb-62340fce0e54 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:53.470471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0d45a-4e15-403d-a2bb-62340fce0e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:53.501963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ecd0d45a-4e15-403d-a2bb-62340fce0e54 None None] MFA Rules not processed for user `505e46d5e4134b119509671714c0030f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:26:53.503222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ecd0d45a-4e15-403d-a2bb-62340fce0e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:53.573793 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4298/8530] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:26:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:26:53.583944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0fb69c-52d4-4561-a48d-3712df119881 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:26:53.585720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0b0fb69c-52d4-4561-a48d-3712df119881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:26:53.599488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:53.652014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0fb69c-52d4-4561-a48d-3712df119881 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:26:53.655742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2c602-e2e6-4e4d-aba1-1ff9db7b99eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:26:53.656377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2c602-e2e6-4e4d-aba1-1ff9db7b99eb service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:26:53.656657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2c602-e2e6-4e4d-aba1-1ff9db7b99eb service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:26:53.656929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2c602-e2e6-4e4d-aba1-1ff9db7b99eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:26:53.665318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2c602-e2e6-4e4d-aba1-1ff9db7b99eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:26:53.667312 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2c602-e2e6-4e4d-aba1-1ff9db7b99eb service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:26:53.676196 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4233/8531] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:26:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:27:02.584350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0d45a-4e15-403d-a2bb-62340fce0e54 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:02.585931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ecd0d45a-4e15-403d-a2bb-62340fce0e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:02.596099 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:02.702871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0d45a-4e15-403d-a2bb-62340fce0e54 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:02.708757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1758568-f9d0-4689-8421-30442c0d969b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:02.709394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1758568-f9d0-4689-8421-30442c0d969b service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:02.709701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1758568-f9d0-4689-8421-30442c0d969b service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:02.709976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1758568-f9d0-4689-8421-30442c0d969b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:02.711924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c1758568-f9d0-4689-8421-30442c0d969b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:02.728234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1758568-f9d0-4689-8421-30442c0d969b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:02.730703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1758568-f9d0-4689-8421-30442c0d969b service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:02.745709 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4299/8532] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:27:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:27:03.672152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=21 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:03.737176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2c602-e2e6-4e4d-aba1-1ff9db7b99eb service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:03.739224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5fd2c602-e2e6-4e4d-aba1-1ff9db7b99eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:03.758589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2c602-e2e6-4e4d-aba1-1ff9db7b99eb service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:03.764869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b00cd-0919-4823-8129-cb205303b356 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:03.765613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b00cd-0919-4823-8129-cb205303b356 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:03.765963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b00cd-0919-4823-8129-cb205303b356 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:03.766305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b00cd-0919-4823-8129-cb205303b356 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:03.768465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-597b00cd-0919-4823-8129-cb205303b356 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:03.782758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b00cd-0919-4823-8129-cb205303b356 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:03.784498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b00cd-0919-4823-8129-cb205303b356 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:03.797826 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4234/8533] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:27:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:27:12.738941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=36 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:13.790640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:21.233676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1758568-f9d0-4689-8421-30442c0d969b service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:21.300492 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1758568-f9d0-4689-8421-30442c0d969b service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:21.305052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08dd30-05d5-494c-b658-1de4965c6ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:21.305674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08dd30-05d5-494c-b658-1de4965c6ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:21.305957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08dd30-05d5-494c-b658-1de4965c6ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:21.306234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08dd30-05d5-494c-b658-1de4965c6ce8 admin admin] PATH_INFO: `/v3/users/505e46d5e4134b119509671714c0030f` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:21.308039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb08dd30-05d5-494c-b658-1de4965c6ce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=505e46d5e4134b119509671714c0030f)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:21.309940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb08dd30-05d5-494c-b658-1de4965c6ce8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:21.356839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cb08dd30-05d5-494c-b658-1de4965c6ce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '505e46d5e4134b119509671714c0030f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:21.363984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cb08dd30-05d5-494c-b658-1de4965c6ce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '505e46d5e4134b119509671714c0030f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:21.376569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cb08dd30-05d5-494c-b658-1de4965c6ce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '505e46d5e4134b119509671714c0030f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:21.377238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cb08dd30-05d5-494c-b658-1de4965c6ce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '505e46d5e4134b119509671714c0030f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:21.385954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-cb08dd30-05d5-494c-b658-1de4965c6ce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '505e46d5e4134b119509671714c0030f'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:21.391891 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4300/8534] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:27:21 2022] DELETE /identity/v3/users/505e46d5e4134b119509671714c0030f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:21.396422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899156a6-94d1-44db-bed3-573621452dfc None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:21.396922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899156a6-94d1-44db-bed3-573621452dfc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:21.397029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899156a6-94d1-44db-bed3-573621452dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:21.415720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-899156a6-94d1-44db-bed3-573621452dfc None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:21.417704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-899156a6-94d1-44db-bed3-573621452dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:21.473672 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4235/8535] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:21.483953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08dd30-05d5-494c-b658-1de4965c6ce8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:21.485380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cb08dd30-05d5-494c-b658-1de4965c6ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:21.545260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08dd30-05d5-494c-b658-1de4965c6ce8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:21.548911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ade8c-d7d8-429c-b205-d4f8cc6f01f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:21.549503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ade8c-d7d8-429c-b205-d4f8cc6f01f8 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:21.549783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ade8c-d7d8-429c-b205-d4f8cc6f01f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:21.550055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ade8c-d7d8-429c-b205-d4f8cc6f01f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:21.558976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ade8c-d7d8-429c-b205-d4f8cc6f01f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:21.560377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ade8c-d7d8-429c-b205-d4f8cc6f01f8 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:21.569651 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4301/8536] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:27:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:27:21.739631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899156a6-94d1-44db-bed3-573621452dfc None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:21.741897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-899156a6-94d1-44db-bed3-573621452dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:21.755644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899156a6-94d1-44db-bed3-573621452dfc None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:21.760275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23994e4a-bb57-49b0-b810-da4d20cbc0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:21.760906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23994e4a-bb57-49b0-b810-da4d20cbc0fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:21.761187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23994e4a-bb57-49b0-b810-da4d20cbc0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:21.761461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23994e4a-bb57-49b0-b810-da4d20cbc0fe admin admin] PATH_INFO: `/v3/projects/4e0c73ba25584e8980bdaeb5cdbdff32` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:21.762628 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23994e4a-bb57-49b0-b810-da4d20cbc0fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e0c73ba25584e8980bdaeb5cdbdff32)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:21.764873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23994e4a-bb57-49b0-b810-da4d20cbc0fe admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:21.811865 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-23994e4a-bb57-49b0-b810-da4d20cbc0fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e0c73ba25584e8980bdaeb5cdbdff32', 'request_id': 'req-23994e4a-bb57-49b0-b810-da4d20cbc0fe'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:21.819176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-23994e4a-bb57-49b0-b810-da4d20cbc0fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e0c73ba25584e8980bdaeb5cdbdff32', 'request_id': 'req-23994e4a-bb57-49b0-b810-da4d20cbc0fe'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:21.825038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-23994e4a-bb57-49b0-b810-da4d20cbc0fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e0c73ba25584e8980bdaeb5cdbdff32', 'request_id': 'req-23994e4a-bb57-49b0-b810-da4d20cbc0fe'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:21.836110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-23994e4a-bb57-49b0-b810-da4d20cbc0fe admin admin] The token cache is being invalidate because project 4e0c73ba25584e8980bdaeb5cdbdff32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:27:21.836502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-23994e4a-bb57-49b0-b810-da4d20cbc0fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:21.837180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-23994e4a-bb57-49b0-b810-da4d20cbc0fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:21.839196 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4236/8537] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:27:21 2022] DELETE /identity/v3/projects/4e0c73ba25584e8980bdaeb5cdbdff32 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:21.844901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6551a02-33c9-40b9-82b2-46b6ad9ec97e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:21.845192 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6551a02-33c9-40b9-82b2-46b6ad9ec97e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:21.845471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6551a02-33c9-40b9-82b2-46b6ad9ec97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:21.875594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c6551a02-33c9-40b9-82b2-46b6ad9ec97e None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:21.876794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c6551a02-33c9-40b9-82b2-46b6ad9ec97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:21.885662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:21.940445 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4302/8538] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:21.948577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94b4b9-b172-46ce-89a3-c5745f39ee15 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:21.948868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94b4b9-b172-46ce-89a3-c5745f39ee15 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:21.949148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94b4b9-b172-46ce-89a3-c5745f39ee15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:21.969553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8d94b4b9-b172-46ce-89a3-c5745f39ee15 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:21.970787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8d94b4b9-b172-46ce-89a3-c5745f39ee15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:21.978689 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4237/8539] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:27:21.982939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6551a02-33c9-40b9-82b2-46b6ad9ec97e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:21.992696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6551a02-33c9-40b9-82b2-46b6ad9ec97e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:21.997302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fc532-3cf1-486f-9023-b3a33de00276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:21.997894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fc532-3cf1-486f-9023-b3a33de00276 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:21.998177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fc532-3cf1-486f-9023-b3a33de00276 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:21.998452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fc532-3cf1-486f-9023-b3a33de00276 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:21.999123 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fc532-3cf1-486f-9023-b3a33de00276 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:27:21.999455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fc532-3cf1-486f-9023-b3a33de00276 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:22.001066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fc532-3cf1-486f-9023-b3a33de00276 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:22.016284 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4303/8540] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:27:22.017344 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94b4b9-b172-46ce-89a3-c5745f39ee15 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:22.027598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94b4b9-b172-46ce-89a3-c5745f39ee15 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:22.037120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed595f76-cdda-4741-b640-6ce71a87cbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:22.037120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed595f76-cdda-4741-b640-6ce71a87cbc0 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:22.037120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed595f76-cdda-4741-b640-6ce71a87cbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:22.037475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed595f76-cdda-4741-b640-6ce71a87cbc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:22.037996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed595f76-cdda-4741-b640-6ce71a87cbc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:22.044601 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed595f76-cdda-4741-b640-6ce71a87cbc0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:22.068435 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4238/8541] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:27:22 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:27:22.073241 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fc532-3cf1-486f-9023-b3a33de00276 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:22.083301 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fc532-3cf1-486f-9023-b3a33de00276 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:22.140210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7006b38-f83f-45f3-89bd-1c08a8d60a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:22.140844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7006b38-f83f-45f3-89bd-1c08a8d60a5e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:22.141100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7006b38-f83f-45f3-89bd-1c08a8d60a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:22.141419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7006b38-f83f-45f3-89bd-1c08a8d60a5e admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:22.142179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7006b38-f83f-45f3-89bd-1c08a8d60a5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:22.144185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7006b38-f83f-45f3-89bd-1c08a8d60a5e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:22.164188 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4304/8542] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:27:22 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:27:22.169055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed595f76-cdda-4741-b640-6ce71a87cbc0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:22.179421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed595f76-cdda-4741-b640-6ce71a87cbc0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:22.184737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4712a67-9434-461f-87e1-dc4a55d155e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:22.185332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4712a67-9434-461f-87e1-dc4a55d155e8 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:22.185610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4712a67-9434-461f-87e1-dc4a55d155e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:22.185885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4712a67-9434-461f-87e1-dc4a55d155e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:22.186556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4712a67-9434-461f-87e1-dc4a55d155e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:22.188159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4712a67-9434-461f-87e1-dc4a55d155e8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:22.198094 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4239/8543] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:27:22.201886 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7006b38-f83f-45f3-89bd-1c08a8d60a5e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:22.211922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7006b38-f83f-45f3-89bd-1c08a8d60a5e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:22.216602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779021da-32d4-4baf-a1c3-05103855d954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:22.217188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779021da-32d4-4baf-a1c3-05103855d954 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:22.217471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779021da-32d4-4baf-a1c3-05103855d954 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:22.217747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779021da-32d4-4baf-a1c3-05103855d954 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:22.218380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779021da-32d4-4baf-a1c3-05103855d954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:22.220006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779021da-32d4-4baf-a1c3-05103855d954 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:22.231122 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4305/8544] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:27:22.235104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4712a67-9434-461f-87e1-dc4a55d155e8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:22.245658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4712a67-9434-461f-87e1-dc4a55d155e8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:22.250286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366c580-5115-4c91-a026-fffcc8781ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:22.250953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366c580-5115-4c91-a026-fffcc8781ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:22.251236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366c580-5115-4c91-a026-fffcc8781ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:22.251511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366c580-5115-4c91-a026-fffcc8781ff4 admin admin] PATH_INFO: `/v3/projects/692213fc6e134842b6ba9af4b9ae7357/users/04f2bbaddb7b4c749b51fdadd334d35a/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:22.265990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366c580-5115-4c91-a026-fffcc8781ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=692213fc6e134842b6ba9af4b9ae7357, user_id=04f2bbaddb7b4c749b51fdadd334d35a, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:22.268717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366c580-5115-4c91-a026-fffcc8781ff4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:22.281634 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4240/8545] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:27:22 2022] PUT /identity/v3/projects/692213fc6e134842b6ba9af4b9ae7357/users/04f2bbaddb7b4c749b51fdadd334d35a/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:22.293156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d480c-b50d-4f7c-8110-4b48c34e20bb None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:22.293634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d480c-b50d-4f7c-8110-4b48c34e20bb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:22.293728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d480c-b50d-4f7c-8110-4b48c34e20bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:22.321088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-443d480c-b50d-4f7c-8110-4b48c34e20bb None None] MFA Rules not processed for user `04f2bbaddb7b4c749b51fdadd334d35a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:22.322293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-443d480c-b50d-4f7c-8110-4b48c34e20bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:22.390150 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4306/8546] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:22.401300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366c580-5115-4c91-a026-fffcc8781ff4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:22.404295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b366c580-5115-4c91-a026-fffcc8781ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:22.430113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:22.470091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366c580-5115-4c91-a026-fffcc8781ff4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:22.473806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f8c04-cd71-4986-a4c7-38bcfc20d6e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:22.474445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f8c04-cd71-4986-a4c7-38bcfc20d6e0 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:22.474761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f8c04-cd71-4986-a4c7-38bcfc20d6e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:22.475040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f8c04-cd71-4986-a4c7-38bcfc20d6e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:22.483953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f8c04-cd71-4986-a4c7-38bcfc20d6e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:22.485449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f8c04-cd71-4986-a4c7-38bcfc20d6e0 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:22.494192 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4241/8547] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:27:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:27:22.508488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d480c-b50d-4f7c-8110-4b48c34e20bb None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:22.518158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:22.575144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d480c-b50d-4f7c-8110-4b48c34e20bb None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:22.580827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f7275-a158-4e55-af73-c4bb21409d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:22.581563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f7275-a158-4e55-af73-c4bb21409d7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:22.581845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f7275-a158-4e55-af73-c4bb21409d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:22.582115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f7275-a158-4e55-af73-c4bb21409d7b admin admin] PATH_INFO: `/v3/users/04f2bbaddb7b4c749b51fdadd334d35a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:22.583284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f7275-a158-4e55-af73-c4bb21409d7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=04f2bbaddb7b4c749b51fdadd334d35a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:22.585573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f7275-a158-4e55-af73-c4bb21409d7b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:22.630930 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a85f7275-a158-4e55-af73-c4bb21409d7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04f2bbaddb7b4c749b51fdadd334d35a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:22.638790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a85f7275-a158-4e55-af73-c4bb21409d7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04f2bbaddb7b4c749b51fdadd334d35a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:22.650925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a85f7275-a158-4e55-af73-c4bb21409d7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04f2bbaddb7b4c749b51fdadd334d35a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:22.651646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a85f7275-a158-4e55-af73-c4bb21409d7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04f2bbaddb7b4c749b51fdadd334d35a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:22.658656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a85f7275-a158-4e55-af73-c4bb21409d7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04f2bbaddb7b4c749b51fdadd334d35a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:22.664500 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4307/8548] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:27:22 2022] DELETE /identity/v3/users/04f2bbaddb7b4c749b51fdadd334d35a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:22.669768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99cabe-326d-4e70-b263-2f76163f0dd4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:22.670059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99cabe-326d-4e70-b263-2f76163f0dd4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:22.670336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99cabe-326d-4e70-b263-2f76163f0dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:22.691459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8b99cabe-326d-4e70-b263-2f76163f0dd4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:22.692656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8b99cabe-326d-4e70-b263-2f76163f0dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:22.752360 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4242/8549] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:22.762338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f7275-a158-4e55-af73-c4bb21409d7b admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:22.763804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a85f7275-a158-4e55-af73-c4bb21409d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:22.826419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f7275-a158-4e55-af73-c4bb21409d7b admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:22.830094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b6ca8-c477-40bf-9ba6-d539adcfdfc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:22.830732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b6ca8-c477-40bf-9ba6-d539adcfdfc4 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:22.831009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b6ca8-c477-40bf-9ba6-d539adcfdfc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:22.831283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b6ca8-c477-40bf-9ba6-d539adcfdfc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:22.840993 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b6ca8-c477-40bf-9ba6-d539adcfdfc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:22.842426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b6ca8-c477-40bf-9ba6-d539adcfdfc4 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:22.851977 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4308/8550] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:27:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:27:23.117874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99cabe-326d-4e70-b263-2f76163f0dd4 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:23.119469 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8b99cabe-326d-4e70-b263-2f76163f0dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:23.132148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99cabe-326d-4e70-b263-2f76163f0dd4 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:23.136975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc7b7a-3622-4444-8955-3bac3e1a0f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:23.137664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc7b7a-3622-4444-8955-3bac3e1a0f13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:23.138250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc7b7a-3622-4444-8955-3bac3e1a0f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:23.138718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc7b7a-3622-4444-8955-3bac3e1a0f13 admin admin] PATH_INFO: `/v3/projects/692213fc6e134842b6ba9af4b9ae7357` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:23.139672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc7b7a-3622-4444-8955-3bac3e1a0f13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=692213fc6e134842b6ba9af4b9ae7357)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:23.141947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc7b7a-3622-4444-8955-3bac3e1a0f13 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:23.189763 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-99cc7b7a-3622-4444-8955-3bac3e1a0f13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '692213fc6e134842b6ba9af4b9ae7357', 'request_id': 'req-99cc7b7a-3622-4444-8955-3bac3e1a0f13'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:23.196824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-99cc7b7a-3622-4444-8955-3bac3e1a0f13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '692213fc6e134842b6ba9af4b9ae7357', 'request_id': 'req-99cc7b7a-3622-4444-8955-3bac3e1a0f13'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:23.202498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-99cc7b7a-3622-4444-8955-3bac3e1a0f13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '692213fc6e134842b6ba9af4b9ae7357', 'request_id': 'req-99cc7b7a-3622-4444-8955-3bac3e1a0f13'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:23.213902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-99cc7b7a-3622-4444-8955-3bac3e1a0f13 admin admin] The token cache is being invalidate because project 692213fc6e134842b6ba9af4b9ae7357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:27:23.214193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-99cc7b7a-3622-4444-8955-3bac3e1a0f13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:23.214878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-99cc7b7a-3622-4444-8955-3bac3e1a0f13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:23.216373 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4243/8551] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:27:23 2022] DELETE /identity/v3/projects/692213fc6e134842b6ba9af4b9ae7357 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:23.221522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cb3d4-3bfc-4a78-ba90-49874ff38f27 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:23.221809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cb3d4-3bfc-4a78-ba90-49874ff38f27 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:23.222087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cb3d4-3bfc-4a78-ba90-49874ff38f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:23.242476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-473cb3d4-3bfc-4a78-ba90-49874ff38f27 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:23.243722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-473cb3d4-3bfc-4a78-ba90-49874ff38f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:23.269658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:23.306064 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4309/8552] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:27:23.314017 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e6e52-5af4-4011-bf8a-803870913b1a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:23.314307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e6e52-5af4-4011-bf8a-803870913b1a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:23.314609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e6e52-5af4-4011-bf8a-803870913b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:23.334050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-572e6e52-5af4-4011-bf8a-803870913b1a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:23.335294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-572e6e52-5af4-4011-bf8a-803870913b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:23.343904 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4244/8553] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:23.347996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cb3d4-3bfc-4a78-ba90-49874ff38f27 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:23.358423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cb3d4-3bfc-4a78-ba90-49874ff38f27 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:23.363069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559c06c-878e-4b30-ab33-be7a713a1b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:23.363665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559c06c-878e-4b30-ab33-be7a713a1b59 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:23.363952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559c06c-878e-4b30-ab33-be7a713a1b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:23.364225 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559c06c-878e-4b30-ab33-be7a713a1b59 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:23.364898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559c06c-878e-4b30-ab33-be7a713a1b59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:27:23.365199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559c06c-878e-4b30-ab33-be7a713a1b59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:23.366803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559c06c-878e-4b30-ab33-be7a713a1b59 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:23.378183 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4310/8554] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:27:23.382314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e6e52-5af4-4011-bf8a-803870913b1a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:23.392101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e6e52-5af4-4011-bf8a-803870913b1a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:23.396766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfe4fd-78d7-4231-840c-c3684478c09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:23.397356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfe4fd-78d7-4231-840c-c3684478c09f admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:23.397631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfe4fd-78d7-4231-840c-c3684478c09f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:23.397899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfe4fd-78d7-4231-840c-c3684478c09f admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:23.398575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfe4fd-78d7-4231-840c-c3684478c09f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:23.400467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfe4fd-78d7-4231-840c-c3684478c09f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:23.429166 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4245/8555] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:27:23 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:27:23.433359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559c06c-878e-4b30-ab33-be7a713a1b59 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:23.443878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559c06c-878e-4b30-ab33-be7a713a1b59 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:23.499707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b1642-a0f8-4eb6-80b7-8ff9cd8bb975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:23.500307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b1642-a0f8-4eb6-80b7-8ff9cd8bb975 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:23.500590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b1642-a0f8-4eb6-80b7-8ff9cd8bb975 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:23.500866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b1642-a0f8-4eb6-80b7-8ff9cd8bb975 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:23.501481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b1642-a0f8-4eb6-80b7-8ff9cd8bb975 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:23.503424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b1642-a0f8-4eb6-80b7-8ff9cd8bb975 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:23.527102 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4311/8556] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:27:23 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:27:23.529560 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfe4fd-78d7-4231-840c-c3684478c09f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:23.540364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfe4fd-78d7-4231-840c-c3684478c09f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:23.545773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a377e99-b22a-4931-9b80-3a9ec25932e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:23.546360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a377e99-b22a-4931-9b80-3a9ec25932e0 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:23.546678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a377e99-b22a-4931-9b80-3a9ec25932e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:23.546949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a377e99-b22a-4931-9b80-3a9ec25932e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:23.547576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a377e99-b22a-4931-9b80-3a9ec25932e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:23.549163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a377e99-b22a-4931-9b80-3a9ec25932e0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:23.558898 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4246/8557] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:27:23.562669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b1642-a0f8-4eb6-80b7-8ff9cd8bb975 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:23.572534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b1642-a0f8-4eb6-80b7-8ff9cd8bb975 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:23.577733 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c490a-0d37-4a08-8fa6-c4760f981f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:23.578317 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c490a-0d37-4a08-8fa6-c4760f981f49 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:23.578641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c490a-0d37-4a08-8fa6-c4760f981f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:23.578948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c490a-0d37-4a08-8fa6-c4760f981f49 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:23.579584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c490a-0d37-4a08-8fa6-c4760f981f49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:23.581165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c490a-0d37-4a08-8fa6-c4760f981f49 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:23.592231 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4312/8558] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:27:23.596024 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a377e99-b22a-4931-9b80-3a9ec25932e0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:23.605718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a377e99-b22a-4931-9b80-3a9ec25932e0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:23.610273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a01707-95e2-4d5e-94d3-f0bf3715920e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:23.610947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a01707-95e2-4d5e-94d3-f0bf3715920e admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:23.611229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a01707-95e2-4d5e-94d3-f0bf3715920e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:23.611507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a01707-95e2-4d5e-94d3-f0bf3715920e admin admin] PATH_INFO: `/v3/projects/8fb07191563640c3b2ec6e2b29e31b07/users/ce6d00d310af4ab6b75f609cc9a00a52/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:23.613713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:23.628739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a01707-95e2-4d5e-94d3-f0bf3715920e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fb07191563640c3b2ec6e2b29e31b07, user_id=ce6d00d310af4ab6b75f609cc9a00a52, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:23.631424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a01707-95e2-4d5e-94d3-f0bf3715920e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:23.644330 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4247/8559] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:27:23 2022] PUT /identity/v3/projects/8fb07191563640c3b2ec6e2b29e31b07/users/ce6d00d310af4ab6b75f609cc9a00a52/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:23.656093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8f369-8ec2-4794-a758-94c269212d4b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:23.656379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8f369-8ec2-4794-a758-94c269212d4b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:23.656680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8f369-8ec2-4794-a758-94c269212d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:23.689737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a4e8f369-8ec2-4794-a758-94c269212d4b None None] MFA Rules not processed for user `ce6d00d310af4ab6b75f609cc9a00a52`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:23.690977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a4e8f369-8ec2-4794-a758-94c269212d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:23.756066 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4313/8560] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:27:23.767395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a01707-95e2-4d5e-94d3-f0bf3715920e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:23.769147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f9a01707-95e2-4d5e-94d3-f0bf3715920e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:23.829364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a01707-95e2-4d5e-94d3-f0bf3715920e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:23.833121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36946cd6-66b5-4520-a254-af3e25bb902b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:23.833745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36946cd6-66b5-4520-a254-af3e25bb902b service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:23.834059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36946cd6-66b5-4520-a254-af3e25bb902b service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:23.834336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36946cd6-66b5-4520-a254-af3e25bb902b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:23.843319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36946cd6-66b5-4520-a254-af3e25bb902b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:23.844757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36946cd6-66b5-4520-a254-af3e25bb902b service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:23.853996 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4248/8561] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:27:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:27:27.984054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8f369-8ec2-4794-a758-94c269212d4b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:28.014214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:28.048630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8f369-8ec2-4794-a758-94c269212d4b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:28.053235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad67ed-8660-42ea-bd92-b0c844a82a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:28.053858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad67ed-8660-42ea-bd92-b0c844a82a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:28.054137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad67ed-8660-42ea-bd92-b0c844a82a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:28.054410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad67ed-8660-42ea-bd92-b0c844a82a54 admin admin] PATH_INFO: `/v3/users/ce6d00d310af4ab6b75f609cc9a00a52` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:28.055563 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad67ed-8660-42ea-bd92-b0c844a82a54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce6d00d310af4ab6b75f609cc9a00a52)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:28.057458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad67ed-8660-42ea-bd92-b0c844a82a54 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:28.102873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-00ad67ed-8660-42ea-bd92-b0c844a82a54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce6d00d310af4ab6b75f609cc9a00a52'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:28.110223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-00ad67ed-8660-42ea-bd92-b0c844a82a54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce6d00d310af4ab6b75f609cc9a00a52'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:28.122719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-00ad67ed-8660-42ea-bd92-b0c844a82a54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce6d00d310af4ab6b75f609cc9a00a52'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:28.123504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-00ad67ed-8660-42ea-bd92-b0c844a82a54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce6d00d310af4ab6b75f609cc9a00a52'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:28.131071 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-00ad67ed-8660-42ea-bd92-b0c844a82a54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce6d00d310af4ab6b75f609cc9a00a52'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:28.137274 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4314/8562] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:27:27 2022] DELETE /identity/v3/users/ce6d00d310af4ab6b75f609cc9a00a52 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:28.142672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c83ed-e6ae-421a-a097-de8b3f06d407 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:28.143224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c83ed-e6ae-421a-a097-de8b3f06d407 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:28.143360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c83ed-e6ae-421a-a097-de8b3f06d407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:28.162938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e61c83ed-e6ae-421a-a097-de8b3f06d407 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:28.164133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e61c83ed-e6ae-421a-a097-de8b3f06d407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:28.223864 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4249/8563] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:28.233967 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad67ed-8660-42ea-bd92-b0c844a82a54 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:28.235397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-00ad67ed-8660-42ea-bd92-b0c844a82a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:28.297377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad67ed-8660-42ea-bd92-b0c844a82a54 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:28.301025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b374a-ef78-4141-9f2a-f95c1addc384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:28.301660 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b374a-ef78-4141-9f2a-f95c1addc384 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:28.301944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b374a-ef78-4141-9f2a-f95c1addc384 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:28.302219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b374a-ef78-4141-9f2a-f95c1addc384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:28.311422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b374a-ef78-4141-9f2a-f95c1addc384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:28.312835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b374a-ef78-4141-9f2a-f95c1addc384 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:28.322305 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4315/8564] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:27:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:27:28.495270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c83ed-e6ae-421a-a097-de8b3f06d407 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:28.496720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e61c83ed-e6ae-421a-a097-de8b3f06d407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:28.509473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c83ed-e6ae-421a-a097-de8b3f06d407 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:28.514518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e8d274-6f4f-4c75-a36a-272724bddd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:28.515176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8d274-6f4f-4c75-a36a-272724bddd86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:28.515460 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8d274-6f4f-4c75-a36a-272724bddd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:28.515789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8d274-6f4f-4c75-a36a-272724bddd86 admin admin] PATH_INFO: `/v3/projects/8fb07191563640c3b2ec6e2b29e31b07` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:28.516922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e8d274-6f4f-4c75-a36a-272724bddd86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fb07191563640c3b2ec6e2b29e31b07)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:28.518884 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e8d274-6f4f-4c75-a36a-272724bddd86 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:28.566941 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-20e8d274-6f4f-4c75-a36a-272724bddd86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fb07191563640c3b2ec6e2b29e31b07', 'request_id': 'req-20e8d274-6f4f-4c75-a36a-272724bddd86'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:28.574246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-20e8d274-6f4f-4c75-a36a-272724bddd86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fb07191563640c3b2ec6e2b29e31b07', 'request_id': 'req-20e8d274-6f4f-4c75-a36a-272724bddd86'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:28.580480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-20e8d274-6f4f-4c75-a36a-272724bddd86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fb07191563640c3b2ec6e2b29e31b07', 'request_id': 'req-20e8d274-6f4f-4c75-a36a-272724bddd86'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:28.591890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-20e8d274-6f4f-4c75-a36a-272724bddd86 admin admin] The token cache is being invalidate because project 8fb07191563640c3b2ec6e2b29e31b07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:27:28.592185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-20e8d274-6f4f-4c75-a36a-272724bddd86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:28.592804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-20e8d274-6f4f-4c75-a36a-272724bddd86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:28.594727 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4250/8565] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:27:28 2022] DELETE /identity/v3/projects/8fb07191563640c3b2ec6e2b29e31b07 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:27:28.599865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a69a4-d143-42b3-8e5d-3775642fd690 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:28.600160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a69a4-d143-42b3-8e5d-3775642fd690 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:28.600442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a69a4-d143-42b3-8e5d-3775642fd690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:28.622133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ef3a69a4-d143-42b3-8e5d-3775642fd690 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:28.623405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ef3a69a4-d143-42b3-8e5d-3775642fd690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:28.668597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:28.687057 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4316/8566] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:27:28.694733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd13a8-658d-4c11-bf80-9faf99336e45 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:28.695485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd13a8-658d-4c11-bf80-9faf99336e45 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:28.696116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd13a8-658d-4c11-bf80-9faf99336e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:28.716739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f9fd13a8-658d-4c11-bf80-9faf99336e45 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:28.717934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f9fd13a8-658d-4c11-bf80-9faf99336e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:28.725672 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4251/8567] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:27:28.730350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a69a4-d143-42b3-8e5d-3775642fd690 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:28.740193 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a69a4-d143-42b3-8e5d-3775642fd690 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:28.745060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7b56f-54ba-44e1-a16b-732f8b87d5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:28.746203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7b56f-54ba-44e1-a16b-732f8b87d5ea admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:28.746480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7b56f-54ba-44e1-a16b-732f8b87d5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:28.746799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7b56f-54ba-44e1-a16b-732f8b87d5ea admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:28.747439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7b56f-54ba-44e1-a16b-732f8b87d5ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:27:28.747742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7b56f-54ba-44e1-a16b-732f8b87d5ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:28.749346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7b56f-54ba-44e1-a16b-732f8b87d5ea admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:28.760920 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4317/8568] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:27:28.765054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd13a8-658d-4c11-bf80-9faf99336e45 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:28.769524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:28.776454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd13a8-658d-4c11-bf80-9faf99336e45 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:28.782608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1ca7d-8ace-43af-a5c7-6463e99d5afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:28.783206 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1ca7d-8ace-43af-a5c7-6463e99d5afb admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:28.783487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1ca7d-8ace-43af-a5c7-6463e99d5afb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:28.783759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1ca7d-8ace-43af-a5c7-6463e99d5afb admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:28.784408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1ca7d-8ace-43af-a5c7-6463e99d5afb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:28.786351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1ca7d-8ace-43af-a5c7-6463e99d5afb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:28.813579 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4252/8569] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:27:28 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:27:28.817801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7b56f-54ba-44e1-a16b-732f8b87d5ea admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:28.827766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7b56f-54ba-44e1-a16b-732f8b87d5ea admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:28.884549 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7265a2f-61d3-4601-beb4-26f2c009e368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:28.885154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7265a2f-61d3-4601-beb4-26f2c009e368 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:28.885458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7265a2f-61d3-4601-beb4-26f2c009e368 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:28.885735 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7265a2f-61d3-4601-beb4-26f2c009e368 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:28.886360 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7265a2f-61d3-4601-beb4-26f2c009e368 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:28.888338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7265a2f-61d3-4601-beb4-26f2c009e368 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:28.910773 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4318/8570] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:27:28 2022] POST /identity/v3/users => generated 303 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:27:28.913712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1ca7d-8ace-43af-a5c7-6463e99d5afb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:28.924686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1ca7d-8ace-43af-a5c7-6463e99d5afb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:28.929332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a49353-986f-47d5-b2b9-cf4a681d6d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:28.929925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a49353-986f-47d5-b2b9-cf4a681d6d89 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:28.930207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a49353-986f-47d5-b2b9-cf4a681d6d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:28.930484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a49353-986f-47d5-b2b9-cf4a681d6d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:28.931146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a49353-986f-47d5-b2b9-cf4a681d6d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:28.933311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a49353-986f-47d5-b2b9-cf4a681d6d89 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:28.943061 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4253/8571] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:27:28.946920 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7265a2f-61d3-4601-beb4-26f2c009e368 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:28.957262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7265a2f-61d3-4601-beb4-26f2c009e368 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:28.962322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe7b73-c51f-457f-b8f6-6653889a4338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:28.962979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe7b73-c51f-457f-b8f6-6653889a4338 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:28.963258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe7b73-c51f-457f-b8f6-6653889a4338 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:28.963531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe7b73-c51f-457f-b8f6-6653889a4338 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:28.964196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe7b73-c51f-457f-b8f6-6653889a4338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:28.965803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe7b73-c51f-457f-b8f6-6653889a4338 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:28.976415 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4319/8572] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:27:28.980805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a49353-986f-47d5-b2b9-cf4a681d6d89 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:28.991066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a49353-986f-47d5-b2b9-cf4a681d6d89 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:28.996278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa7b09-a62b-4190-a763-3cbcc398a02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:28.996917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa7b09-a62b-4190-a763-3cbcc398a02f admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:28.997193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa7b09-a62b-4190-a763-3cbcc398a02f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:28.997494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa7b09-a62b-4190-a763-3cbcc398a02f admin admin] PATH_INFO: `/v3/projects/15abb2ce1f1646e8bbf15e5ba5a9e110/users/fac75ac2c62d4311a3c8d1a6d7d81903/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:29.012759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa7b09-a62b-4190-a763-3cbcc398a02f admin admin] RBAC: Authorizing `identity:create_grant(project_id=15abb2ce1f1646e8bbf15e5ba5a9e110, user_id=fac75ac2c62d4311a3c8d1a6d7d81903, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:29.015480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa7b09-a62b-4190-a763-3cbcc398a02f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:29.029092 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4254/8573] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:27:28 2022] PUT /identity/v3/projects/15abb2ce1f1646e8bbf15e5ba5a9e110/users/fac75ac2c62d4311a3c8d1a6d7d81903/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:29.039559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9540d0-ca21-4d2a-b1c0-e5df8ee96924 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:29.040081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9540d0-ca21-4d2a-b1c0-e5df8ee96924 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:29.040178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9540d0-ca21-4d2a-b1c0-e5df8ee96924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:29.069872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3e9540d0-ca21-4d2a-b1c0-e5df8ee96924 None None] MFA Rules not processed for user `fac75ac2c62d4311a3c8d1a6d7d81903`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:29.071051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3e9540d0-ca21-4d2a-b1c0-e5df8ee96924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:29.135000 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4320/8574] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:29 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:29.139750 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa7b09-a62b-4190-a763-3cbcc398a02f admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:29.202183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa7b09-a62b-4190-a763-3cbcc398a02f admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:29.207229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4e1cd-6118-4ecc-857b-6a126c3d2928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:29.207830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4e1cd-6118-4ecc-857b-6a126c3d2928 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:29.208111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4e1cd-6118-4ecc-857b-6a126c3d2928 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:29.208383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4e1cd-6118-4ecc-857b-6a126c3d2928 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:29.209008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f4e1cd-6118-4ecc-857b-6a126c3d2928 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:29.210987 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f4e1cd-6118-4ecc-857b-6a126c3d2928 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:29.239522 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4255/8575] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:27:29 2022] POST /identity/v3/projects => generated 405 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:27:29.243703 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9540d0-ca21-4d2a-b1c0-e5df8ee96924 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:29.254574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9540d0-ca21-4d2a-b1c0-e5df8ee96924 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:29.297336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:29.313482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9a7ce-9e4d-46bc-8b46-e59ae4fde9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:29.314085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9a7ce-9e4d-46bc-8b46-e59ae4fde9c3 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:29.314365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9a7ce-9e4d-46bc-8b46-e59ae4fde9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:29.314670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9a7ce-9e4d-46bc-8b46-e59ae4fde9c3 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:29.315290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9a7ce-9e4d-46bc-8b46-e59ae4fde9c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:29.317194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9a7ce-9e4d-46bc-8b46-e59ae4fde9c3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:29.338141 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4321/8576] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:27:29 2022] POST /identity/v3/users => generated 309 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:27:29.341932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4e1cd-6118-4ecc-857b-6a126c3d2928 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:29.353429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4e1cd-6118-4ecc-857b-6a126c3d2928 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:29.358752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b615c-b70c-4115-acdf-0fc5ecabf357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:29.359351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b615c-b70c-4115-acdf-0fc5ecabf357 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:29.359631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b615c-b70c-4115-acdf-0fc5ecabf357 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:29.359904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b615c-b70c-4115-acdf-0fc5ecabf357 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:29.360545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b615c-b70c-4115-acdf-0fc5ecabf357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:29.362136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b615c-b70c-4115-acdf-0fc5ecabf357 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:29.372837 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4256/8577] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:27:29.376875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9a7ce-9e4d-46bc-8b46-e59ae4fde9c3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:29.387643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9a7ce-9e4d-46bc-8b46-e59ae4fde9c3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:29.392417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2842312-6318-4f5d-97f9-835604317c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:29.393064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2842312-6318-4f5d-97f9-835604317c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:29.393348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2842312-6318-4f5d-97f9-835604317c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:29.393629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2842312-6318-4f5d-97f9-835604317c70 admin admin] PATH_INFO: `/v3/projects/f1210506ba6e4b369045c36c919ace0b/users/82221b9f6f6d4e9fa61d01f74d6f8611/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:29.409049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2842312-6318-4f5d-97f9-835604317c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1210506ba6e4b369045c36c919ace0b, user_id=82221b9f6f6d4e9fa61d01f74d6f8611, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:29.411763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2842312-6318-4f5d-97f9-835604317c70 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:29.424691 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4322/8578] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:27:29 2022] PUT /identity/v3/projects/f1210506ba6e4b369045c36c919ace0b/users/82221b9f6f6d4e9fa61d01f74d6f8611/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:29.428693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b615c-b70c-4115-acdf-0fc5ecabf357 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:29.441147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b615c-b70c-4115-acdf-0fc5ecabf357 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:29.496424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef102c-ffc1-4572-8512-e1425f113fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:29.497019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef102c-ffc1-4572-8512-e1425f113fef admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:29.497296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef102c-ffc1-4572-8512-e1425f113fef admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:29.497594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef102c-ffc1-4572-8512-e1425f113fef admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:29.498217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef102c-ffc1-4572-8512-e1425f113fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:29.499834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef102c-ffc1-4572-8512-e1425f113fef admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:29.509309 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4257/8579] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:27:29.514849 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2842312-6318-4f5d-97f9-835604317c70 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:29.525417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2842312-6318-4f5d-97f9-835604317c70 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:29.530722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ebc4c-5cdf-40e9-906e-58382ed076ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:29.531368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ebc4c-5cdf-40e9-906e-58382ed076ff admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:29.531648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ebc4c-5cdf-40e9-906e-58382ed076ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:29.531917 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ebc4c-5cdf-40e9-906e-58382ed076ff admin admin] PATH_INFO: `/v3/projects/f1210506ba6e4b369045c36c919ace0b/users/82221b9f6f6d4e9fa61d01f74d6f8611/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:29.533809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ebc4c-5cdf-40e9-906e-58382ed076ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1210506ba6e4b369045c36c919ace0b, user_id=82221b9f6f6d4e9fa61d01f74d6f8611, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:29.536482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ebc4c-5cdf-40e9-906e-58382ed076ff admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:29.546844 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4323/8580] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:27:29 2022] PUT /identity/v3/projects/f1210506ba6e4b369045c36c919ace0b/users/82221b9f6f6d4e9fa61d01f74d6f8611/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:29.557925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715db7d-5a75-40d3-89c5-124f638941fa None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:29.558218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715db7d-5a75-40d3-89c5-124f638941fa None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:29.558521 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715db7d-5a75-40d3-89c5-124f638941fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:29.586306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e715db7d-5a75-40d3-89c5-124f638941fa None None] MFA Rules not processed for user `82221b9f6f6d4e9fa61d01f74d6f8611`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:29.587636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e715db7d-5a75-40d3-89c5-124f638941fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:29.592251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:29.655461 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4258/8581] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:29 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:29.666431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ebc4c-5cdf-40e9-906e-58382ed076ff admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:29.668421 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c22ebc4c-5cdf-40e9-906e-58382ed076ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:29.740909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ebc4c-5cdf-40e9-906e-58382ed076ff admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:29.744718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68570950-bbaa-4813-bd09-9caef3a24f52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:29.745820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68570950-bbaa-4813-bd09-9caef3a24f52 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:29.746124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68570950-bbaa-4813-bd09-9caef3a24f52 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:29.746424 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68570950-bbaa-4813-bd09-9caef3a24f52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:29.755702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68570950-bbaa-4813-bd09-9caef3a24f52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:29.757129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68570950-bbaa-4813-bd09-9caef3a24f52 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:29.767243 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4324/8582] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:27:29 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:27:29.898348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715db7d-5a75-40d3-89c5-124f638941fa None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:29.961986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715db7d-5a75-40d3-89c5-124f638941fa None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:29.966527 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ec561-7756-4dbe-ba88-416bf3893379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:29.967153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ec561-7756-4dbe-ba88-416bf3893379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:29.967433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ec561-7756-4dbe-ba88-416bf3893379 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:29.967706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ec561-7756-4dbe-ba88-416bf3893379 admin admin] PATH_INFO: `/v3/users/fac75ac2c62d4311a3c8d1a6d7d81903` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:29.968804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ec561-7756-4dbe-ba88-416bf3893379 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fac75ac2c62d4311a3c8d1a6d7d81903)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:29.970757 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ec561-7756-4dbe-ba88-416bf3893379 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:30.016237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6d3ec561-7756-4dbe-ba88-416bf3893379 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fac75ac2c62d4311a3c8d1a6d7d81903'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:30.016886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6d3ec561-7756-4dbe-ba88-416bf3893379 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fac75ac2c62d4311a3c8d1a6d7d81903'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:30.017516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6d3ec561-7756-4dbe-ba88-416bf3893379 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fac75ac2c62d4311a3c8d1a6d7d81903'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:30.024900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6d3ec561-7756-4dbe-ba88-416bf3893379 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fac75ac2c62d4311a3c8d1a6d7d81903'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:30.037437 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-6d3ec561-7756-4dbe-ba88-416bf3893379 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fac75ac2c62d4311a3c8d1a6d7d81903'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:30.050344 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4259/8583] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:27:29 2022] DELETE /identity/v3/users/fac75ac2c62d4311a3c8d1a6d7d81903 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:30.055095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f49daf-04e7-4e22-932d-978ceca301b7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:30.055567 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f49daf-04e7-4e22-932d-978ceca301b7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:30.055688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f49daf-04e7-4e22-932d-978ceca301b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:30.075672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-d2f49daf-04e7-4e22-932d-978ceca301b7 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:30.076872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d2f49daf-04e7-4e22-932d-978ceca301b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:30.135844 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4325/8584] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:30.146000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ec561-7756-4dbe-ba88-416bf3893379 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:30.147461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6d3ec561-7756-4dbe-ba88-416bf3893379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:30.169474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:30.208109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ec561-7756-4dbe-ba88-416bf3893379 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:30.211798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba969a93-ce25-4ceb-b0df-8aadfbfc3738 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:30.212402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba969a93-ce25-4ceb-b0df-8aadfbfc3738 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:30.212684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba969a93-ce25-4ceb-b0df-8aadfbfc3738 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:30.212958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba969a93-ce25-4ceb-b0df-8aadfbfc3738 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:30.222470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba969a93-ce25-4ceb-b0df-8aadfbfc3738 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:30.223933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba969a93-ce25-4ceb-b0df-8aadfbfc3738 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:30.233230 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4260/8585] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:27:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:27:30.506199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f49daf-04e7-4e22-932d-978ceca301b7 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:30.507694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d2f49daf-04e7-4e22-932d-978ceca301b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:30.520436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f49daf-04e7-4e22-932d-978ceca301b7 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:30.525078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ddfdb-6d01-44d7-a70e-54fa04c6c479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:30.525712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ddfdb-6d01-44d7-a70e-54fa04c6c479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:30.525989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ddfdb-6d01-44d7-a70e-54fa04c6c479 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:30.526260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ddfdb-6d01-44d7-a70e-54fa04c6c479 admin admin] PATH_INFO: `/v3/projects/15abb2ce1f1646e8bbf15e5ba5a9e110` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:30.527442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ddfdb-6d01-44d7-a70e-54fa04c6c479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15abb2ce1f1646e8bbf15e5ba5a9e110)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:30.529332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ddfdb-6d01-44d7-a70e-54fa04c6c479 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:30.555605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:30.580457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4c6ddfdb-6d01-44d7-a70e-54fa04c6c479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15abb2ce1f1646e8bbf15e5ba5a9e110', 'request_id': 'req-4c6ddfdb-6d01-44d7-a70e-54fa04c6c479'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:30.586513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4c6ddfdb-6d01-44d7-a70e-54fa04c6c479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15abb2ce1f1646e8bbf15e5ba5a9e110', 'request_id': 'req-4c6ddfdb-6d01-44d7-a70e-54fa04c6c479'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:30.592086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4c6ddfdb-6d01-44d7-a70e-54fa04c6c479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15abb2ce1f1646e8bbf15e5ba5a9e110', 'request_id': 'req-4c6ddfdb-6d01-44d7-a70e-54fa04c6c479'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:30.604260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4c6ddfdb-6d01-44d7-a70e-54fa04c6c479 admin admin] The token cache is being invalidate because project 15abb2ce1f1646e8bbf15e5ba5a9e110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:27:30.604569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4c6ddfdb-6d01-44d7-a70e-54fa04c6c479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:30.605185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4c6ddfdb-6d01-44d7-a70e-54fa04c6c479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:30.606800 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4326/8586] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:27:30 2022] DELETE /identity/v3/projects/15abb2ce1f1646e8bbf15e5ba5a9e110 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:30.610761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba969a93-ce25-4ceb-b0df-8aadfbfc3738 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:30.620268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba969a93-ce25-4ceb-b0df-8aadfbfc3738 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:30.676068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542818b-48c0-416e-818f-b150f4493625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:30.676657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542818b-48c0-416e-818f-b150f4493625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:30.676940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542818b-48c0-416e-818f-b150f4493625 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:30.677247 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542818b-48c0-416e-818f-b150f4493625 admin admin] PATH_INFO: `/v3/users/82221b9f6f6d4e9fa61d01f74d6f8611` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:30.678390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542818b-48c0-416e-818f-b150f4493625 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82221b9f6f6d4e9fa61d01f74d6f8611)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:30.680357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542818b-48c0-416e-818f-b150f4493625 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:30.725544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4542818b-48c0-416e-818f-b150f4493625 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82221b9f6f6d4e9fa61d01f74d6f8611'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:30.726181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4542818b-48c0-416e-818f-b150f4493625 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82221b9f6f6d4e9fa61d01f74d6f8611'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:30.726807 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4542818b-48c0-416e-818f-b150f4493625 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82221b9f6f6d4e9fa61d01f74d6f8611'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:30.733994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4542818b-48c0-416e-818f-b150f4493625 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82221b9f6f6d4e9fa61d01f74d6f8611'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:30.745768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4542818b-48c0-416e-818f-b150f4493625 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82221b9f6f6d4e9fa61d01f74d6f8611'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:30.758673 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4261/8587] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:27:30 2022] DELETE /identity/v3/users/82221b9f6f6d4e9fa61d01f74d6f8611 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:31.035486 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ddfdb-6d01-44d7-a70e-54fa04c6c479 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:31.037151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4c6ddfdb-6d01-44d7-a70e-54fa04c6c479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:31.062411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ddfdb-6d01-44d7-a70e-54fa04c6c479 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:31.122340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af5cdb-ac7f-4a40-b619-af70d12599aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:31.123034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af5cdb-ac7f-4a40-b619-af70d12599aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:31.123298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af5cdb-ac7f-4a40-b619-af70d12599aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:31.123573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af5cdb-ac7f-4a40-b619-af70d12599aa admin admin] PATH_INFO: `/v3/projects/f1210506ba6e4b369045c36c919ace0b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:31.124756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af5cdb-ac7f-4a40-b619-af70d12599aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1210506ba6e4b369045c36c919ace0b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:31.126697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af5cdb-ac7f-4a40-b619-af70d12599aa admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:31.178364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e7af5cdb-ac7f-4a40-b619-af70d12599aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1210506ba6e4b369045c36c919ace0b', 'request_id': 'req-e7af5cdb-ac7f-4a40-b619-af70d12599aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:31.185206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e7af5cdb-ac7f-4a40-b619-af70d12599aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1210506ba6e4b369045c36c919ace0b', 'request_id': 'req-e7af5cdb-ac7f-4a40-b619-af70d12599aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:31.192922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e7af5cdb-ac7f-4a40-b619-af70d12599aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1210506ba6e4b369045c36c919ace0b', 'request_id': 'req-e7af5cdb-ac7f-4a40-b619-af70d12599aa'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:31.205702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e7af5cdb-ac7f-4a40-b619-af70d12599aa admin admin] The token cache is being invalidate because project f1210506ba6e4b369045c36c919ace0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:27:31.205949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e7af5cdb-ac7f-4a40-b619-af70d12599aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:31.206644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-e7af5cdb-ac7f-4a40-b619-af70d12599aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:31.208146 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4327/8588] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:27:31 2022] DELETE /identity/v3/projects/f1210506ba6e4b369045c36c919ace0b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:31.213204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f4aad-8bbd-46ca-a8f0-33604e5b43f6 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:31.213665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f4aad-8bbd-46ca-a8f0-33604e5b43f6 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:31.213759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f4aad-8bbd-46ca-a8f0-33604e5b43f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:31.234541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-944f4aad-8bbd-46ca-a8f0-33604e5b43f6 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:31.235742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-944f4aad-8bbd-46ca-a8f0-33604e5b43f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:31.274038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:31.305679 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4262/8589] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:31.309619 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f68de-3576-4ccb-9859-e2c674f48aa2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:31.309907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f68de-3576-4ccb-9859-e2c674f48aa2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:31.310203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f68de-3576-4ccb-9859-e2c674f48aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:31.330368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7a5f68de-3576-4ccb-9859-e2c674f48aa2 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:31.331626 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7a5f68de-3576-4ccb-9859-e2c674f48aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:31.339707 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4328/8590] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:31.344443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f4aad-8bbd-46ca-a8f0-33604e5b43f6 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:31.354232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f4aad-8bbd-46ca-a8f0-33604e5b43f6 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:31.359487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8996c7-f57d-4913-973b-877e3ec71b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:31.360085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8996c7-f57d-4913-973b-877e3ec71b9d admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:31.360365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8996c7-f57d-4913-973b-877e3ec71b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:31.360640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8996c7-f57d-4913-973b-877e3ec71b9d admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:31.361307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8996c7-f57d-4913-973b-877e3ec71b9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:27:31.361613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8996c7-f57d-4913-973b-877e3ec71b9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:31.363278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8996c7-f57d-4913-973b-877e3ec71b9d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:31.374814 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4263/8591] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:27:31.379136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f68de-3576-4ccb-9859-e2c674f48aa2 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:31.388385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f68de-3576-4ccb-9859-e2c674f48aa2 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:31.393280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fdde5-9e17-4dd7-b436-9b0d28207da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:31.393878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fdde5-9e17-4dd7-b436-9b0d28207da4 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:31.394160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fdde5-9e17-4dd7-b436-9b0d28207da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:31.394437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fdde5-9e17-4dd7-b436-9b0d28207da4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:31.395103 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543fdde5-9e17-4dd7-b436-9b0d28207da4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:31.397024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543fdde5-9e17-4dd7-b436-9b0d28207da4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:31.425630 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4329/8592] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:27:31 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:27:31.429919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8996c7-f57d-4913-973b-877e3ec71b9d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:31.440496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8996c7-f57d-4913-973b-877e3ec71b9d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:31.497287 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4feea-cd14-45b1-8a8d-9e0da37238bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:31.497815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4feea-cd14-45b1-8a8d-9e0da37238bc admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:31.498099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4feea-cd14-45b1-8a8d-9e0da37238bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:31.498378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4feea-cd14-45b1-8a8d-9e0da37238bc admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:31.499044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4feea-cd14-45b1-8a8d-9e0da37238bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:31.501012 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4feea-cd14-45b1-8a8d-9e0da37238bc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:31.521966 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4264/8593] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:27:31 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:27:31.526104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fdde5-9e17-4dd7-b436-9b0d28207da4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:31.537127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fdde5-9e17-4dd7-b436-9b0d28207da4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:31.541797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10861f08-2edf-46b9-acac-3c66cb1d791c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:31.542396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10861f08-2edf-46b9-acac-3c66cb1d791c admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:31.543044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10861f08-2edf-46b9-acac-3c66cb1d791c admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:31.543916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10861f08-2edf-46b9-acac-3c66cb1d791c admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:31.544566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10861f08-2edf-46b9-acac-3c66cb1d791c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:31.546160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10861f08-2edf-46b9-acac-3c66cb1d791c admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:31.556238 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4330/8594] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:27:31.560005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4feea-cd14-45b1-8a8d-9e0da37238bc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:31.569933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4feea-cd14-45b1-8a8d-9e0da37238bc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:31.574990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ce007-4078-480e-96b2-bee875980950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:31.575595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ce007-4078-480e-96b2-bee875980950 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:31.575879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ce007-4078-480e-96b2-bee875980950 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:31.576184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ce007-4078-480e-96b2-bee875980950 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:31.576822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ce007-4078-480e-96b2-bee875980950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:31.578417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ce007-4078-480e-96b2-bee875980950 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:31.589106 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4265/8595] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:27:31.593467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10861f08-2edf-46b9-acac-3c66cb1d791c admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:31.598709 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:31.605254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10861f08-2edf-46b9-acac-3c66cb1d791c admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:31.611576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b980e-1d38-446a-9bb9-a181086992ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:31.612231 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b980e-1d38-446a-9bb9-a181086992ae admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:31.612518 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b980e-1d38-446a-9bb9-a181086992ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:31.612817 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b980e-1d38-446a-9bb9-a181086992ae admin admin] PATH_INFO: `/v3/projects/ef8c8e969dc040d89f85e78613bf984c/users/ae06c9808a3f4aecb041b36d505ee9b4/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:31.627475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b980e-1d38-446a-9bb9-a181086992ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef8c8e969dc040d89f85e78613bf984c, user_id=ae06c9808a3f4aecb041b36d505ee9b4, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:31.630145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b980e-1d38-446a-9bb9-a181086992ae admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:31.643641 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4331/8596] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:27:31 2022] PUT /identity/v3/projects/ef8c8e969dc040d89f85e78613bf984c/users/ae06c9808a3f4aecb041b36d505ee9b4/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:31.654869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715fe36-3df3-480a-8439-664e749104d0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:31.655159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715fe36-3df3-480a-8439-664e749104d0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:31.655435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715fe36-3df3-480a-8439-664e749104d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:31.682880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c715fe36-3df3-480a-8439-664e749104d0 None None] MFA Rules not processed for user `ae06c9808a3f4aecb041b36d505ee9b4`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:31.684097 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c715fe36-3df3-480a-8439-664e749104d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:31.746841 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4266/8597] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:31.751404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b980e-1d38-446a-9bb9-a181086992ae admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:31.813180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b980e-1d38-446a-9bb9-a181086992ae admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:31.817794 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee86d836-f345-4ca6-ac90-8005a7f0bb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:31.818391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86d836-f345-4ca6-ac90-8005a7f0bb25 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:31.818704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86d836-f345-4ca6-ac90-8005a7f0bb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:31.818978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86d836-f345-4ca6-ac90-8005a7f0bb25 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:31.819595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee86d836-f345-4ca6-ac90-8005a7f0bb25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:31.821483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee86d836-f345-4ca6-ac90-8005a7f0bb25 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:31.848418 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4332/8598] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:27:31 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:27:31.852451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715fe36-3df3-480a-8439-664e749104d0 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:31.863311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715fe36-3df3-480a-8439-664e749104d0 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:31.896253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:31.920906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca11128-f8dd-4593-a575-e1d9c724c73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:31.921539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca11128-f8dd-4593-a575-e1d9c724c73b admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:31.921819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca11128-f8dd-4593-a575-e1d9c724c73b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:31.922101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca11128-f8dd-4593-a575-e1d9c724c73b admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:31.922756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca11128-f8dd-4593-a575-e1d9c724c73b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:31.924673 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca11128-f8dd-4593-a575-e1d9c724c73b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:31.946274 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4267/8599] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:27:31 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:27:31.950774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee86d836-f345-4ca6-ac90-8005a7f0bb25 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:31.961613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee86d836-f345-4ca6-ac90-8005a7f0bb25 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:31.966892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd25dd1-c858-4979-9648-adba2ec9436f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:31.967487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd25dd1-c858-4979-9648-adba2ec9436f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:31.967769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd25dd1-c858-4979-9648-adba2ec9436f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:31.968042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd25dd1-c858-4979-9648-adba2ec9436f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:31.968677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd25dd1-c858-4979-9648-adba2ec9436f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:31.970265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd25dd1-c858-4979-9648-adba2ec9436f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:31.980888 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4333/8600] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:27:31.984984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca11128-f8dd-4593-a575-e1d9c724c73b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:31.994932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca11128-f8dd-4593-a575-e1d9c724c73b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:31.999957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba0992-6591-4c07-addf-9417b03be86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:32.000607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba0992-6591-4c07-addf-9417b03be86c admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:32.000892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba0992-6591-4c07-addf-9417b03be86c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:32.001179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba0992-6591-4c07-addf-9417b03be86c admin admin] PATH_INFO: `/v3/projects/0a046cfd45c94cf98849cd0ae078f9a0/users/c787a41f22f74e109994eb6e6a10cb76/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:32.016597 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba0992-6591-4c07-addf-9417b03be86c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a046cfd45c94cf98849cd0ae078f9a0, user_id=c787a41f22f74e109994eb6e6a10cb76, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:32.019318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba0992-6591-4c07-addf-9417b03be86c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:32.033450 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4268/8601] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:27:31 2022] PUT /identity/v3/projects/0a046cfd45c94cf98849cd0ae078f9a0/users/c787a41f22f74e109994eb6e6a10cb76/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:32.037448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd25dd1-c858-4979-9648-adba2ec9436f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:32.048013 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd25dd1-c858-4979-9648-adba2ec9436f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:32.106316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5ef5d-fb7f-4759-878a-a016e1779de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:32.107015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5ef5d-fb7f-4759-878a-a016e1779de3 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:32.107221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5ef5d-fb7f-4759-878a-a016e1779de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:32.107495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5ef5d-fb7f-4759-878a-a016e1779de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:32.108159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c5ef5d-fb7f-4759-878a-a016e1779de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:32.109748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c5ef5d-fb7f-4759-878a-a016e1779de3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:32.119509 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4334/8602] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:27:32.123902 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba0992-6591-4c07-addf-9417b03be86c admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:32.134609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba0992-6591-4c07-addf-9417b03be86c admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:32.140180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361f111-b8ad-4bd3-8464-eedc1fef678b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:32.140916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361f111-b8ad-4bd3-8464-eedc1fef678b admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:32.141240 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361f111-b8ad-4bd3-8464-eedc1fef678b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:32.141505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361f111-b8ad-4bd3-8464-eedc1fef678b admin admin] PATH_INFO: `/v3/projects/0a046cfd45c94cf98849cd0ae078f9a0/users/c787a41f22f74e109994eb6e6a10cb76/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:32.143566 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9361f111-b8ad-4bd3-8464-eedc1fef678b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a046cfd45c94cf98849cd0ae078f9a0, user_id=c787a41f22f74e109994eb6e6a10cb76, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:32.146212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9361f111-b8ad-4bd3-8464-eedc1fef678b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:32.157147 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4269/8603] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:27:32 2022] PUT /identity/v3/projects/0a046cfd45c94cf98849cd0ae078f9a0/users/c787a41f22f74e109994eb6e6a10cb76/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:32.168937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727c6c4-d91f-49e6-a414-61461ef77f79 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:32.169305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727c6c4-d91f-49e6-a414-61461ef77f79 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:32.169654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727c6c4-d91f-49e6-a414-61461ef77f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:32.197484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5727c6c4-d91f-49e6-a414-61461ef77f79 None None] MFA Rules not processed for user `c787a41f22f74e109994eb6e6a10cb76`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:32.198728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5727c6c4-d91f-49e6-a414-61461ef77f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:32.236725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:32.261393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361f111-b8ad-4bd3-8464-eedc1fef678b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:32.263736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9361f111-b8ad-4bd3-8464-eedc1fef678b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:32.270107 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4335/8604] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:32.282114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5727c6c4-d91f-49e6-a414-61461ef77f79 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:32.283473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5727c6c4-d91f-49e6-a414-61461ef77f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:32.329688 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361f111-b8ad-4bd3-8464-eedc1fef678b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:32.333926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ebe029-0c00-4aef-9b6d-f0dad98f0443 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:32.334273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebe029-0c00-4aef-9b6d-f0dad98f0443 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:32.334581 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebe029-0c00-4aef-9b6d-f0dad98f0443 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:32.334852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebe029-0c00-4aef-9b6d-f0dad98f0443 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:32.336968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-57ebe029-0c00-4aef-9b6d-f0dad98f0443 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:32.345812 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5727c6c4-d91f-49e6-a414-61461ef77f79 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:32.349558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0525995-f128-4650-ab2c-0e972f5adeec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:32.350171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0525995-f128-4650-ab2c-0e972f5adeec service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:32.350454 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0525995-f128-4650-ab2c-0e972f5adeec service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:32.350774 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0525995-f128-4650-ab2c-0e972f5adeec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:32.361017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0525995-f128-4650-ab2c-0e972f5adeec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:32.362453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0525995-f128-4650-ab2c-0e972f5adeec service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:32.373030 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4336/8605] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:27:32 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:27:32.401825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ebe029-0c00-4aef-9b6d-f0dad98f0443 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:32.408712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ebe029-0c00-4aef-9b6d-f0dad98f0443 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:32.413083 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4270/8606] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:27:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:27:32.471300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0525995-f128-4650-ab2c-0e972f5adeec service cinder] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:32.482870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0525995-f128-4650-ab2c-0e972f5adeec service cinder] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:32.487223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27936769-8e45-4827-9504-1f48afbe4583 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:32.487957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27936769-8e45-4827-9504-1f48afbe4583 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:32.488245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27936769-8e45-4827-9504-1f48afbe4583 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:32.488523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27936769-8e45-4827-9504-1f48afbe4583 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:32.556776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27936769-8e45-4827-9504-1f48afbe4583 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:32.558205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27936769-8e45-4827-9504-1f48afbe4583 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:32.568555 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4337/8607] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:27:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:27:35.122932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e125a-f00f-42a1-a1f2-7dfc61ffdd2a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:35.123439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e125a-f00f-42a1-a1f2-7dfc61ffdd2a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:35.123563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e125a-f00f-42a1-a1f2-7dfc61ffdd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:35.143085 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-907e125a-f00f-42a1-a1f2-7dfc61ffdd2a None None] MFA Rules not processed for user `eb0bf5fce885439eaa16a0871c2992ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:35.144311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-907e125a-f00f-42a1-a1f2-7dfc61ffdd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:35.151896 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4271/8608] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:35.779022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27936769-8e45-4827-9504-1f48afbe4583 service cinder] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:35.780443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-27936769-8e45-4827-9504-1f48afbe4583 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:35.839979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27936769-8e45-4827-9504-1f48afbe4583 service cinder] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:35.843603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d96e08-d08a-4c24-a093-76adf7ad00c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:35.844228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d96e08-d08a-4c24-a093-76adf7ad00c4 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:35.844504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d96e08-d08a-4c24-a093-76adf7ad00c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:35.844777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d96e08-d08a-4c24-a093-76adf7ad00c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:35.846547 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d8d96e08-d08a-4c24-a093-76adf7ad00c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:35.859287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d96e08-d08a-4c24-a093-76adf7ad00c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:35.860736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d96e08-d08a-4c24-a093-76adf7ad00c4 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:35.871430 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4338/8609] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:27:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:27:40.559834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e125a-f00f-42a1-a1f2-7dfc61ffdd2a None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:40.600507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:40.622986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e125a-f00f-42a1-a1f2-7dfc61ffdd2a None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:40.627532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f5ab10-d31f-4384-a794-2695af96a7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:40.628158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5ab10-d31f-4384-a794-2695af96a7d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:40.628439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5ab10-d31f-4384-a794-2695af96a7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:40.628714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5ab10-d31f-4384-a794-2695af96a7d5 admin admin] PATH_INFO: `/v3/users/ae06c9808a3f4aecb041b36d505ee9b4` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:40.629824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f5ab10-d31f-4384-a794-2695af96a7d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae06c9808a3f4aecb041b36d505ee9b4)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:40.631779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f5ab10-d31f-4384-a794-2695af96a7d5 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:40.677818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e4f5ab10-d31f-4384-a794-2695af96a7d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae06c9808a3f4aecb041b36d505ee9b4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:40.678391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e4f5ab10-d31f-4384-a794-2695af96a7d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae06c9808a3f4aecb041b36d505ee9b4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:40.679028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e4f5ab10-d31f-4384-a794-2695af96a7d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae06c9808a3f4aecb041b36d505ee9b4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:40.686769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e4f5ab10-d31f-4384-a794-2695af96a7d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae06c9808a3f4aecb041b36d505ee9b4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:40.698543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-e4f5ab10-d31f-4384-a794-2695af96a7d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae06c9808a3f4aecb041b36d505ee9b4'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:40.710937 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4272/8610] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:27:40 2022] DELETE /identity/v3/users/ae06c9808a3f4aecb041b36d505ee9b4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:40.715542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbbaaf-f263-4285-874e-f644f1d68de0 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:40.716048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbbaaf-f263-4285-874e-f644f1d68de0 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:40.716143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbbaaf-f263-4285-874e-f644f1d68de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:40.744761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-61bbbaaf-f263-4285-874e-f644f1d68de0 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:40.745969 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-61bbbaaf-f263-4285-874e-f644f1d68de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:40.751516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:40.804795 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4339/8611] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:27:40.815166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f5ab10-d31f-4384-a794-2695af96a7d5 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:40.816556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e4f5ab10-d31f-4384-a794-2695af96a7d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:40.875298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f5ab10-d31f-4384-a794-2695af96a7d5 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:40.878920 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01421a-34a3-46cb-bc8c-5c7adf3b0eb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:40.879524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01421a-34a3-46cb-bc8c-5c7adf3b0eb6 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:40.879808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01421a-34a3-46cb-bc8c-5c7adf3b0eb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:40.880093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01421a-34a3-46cb-bc8c-5c7adf3b0eb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:40.889624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01421a-34a3-46cb-bc8c-5c7adf3b0eb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:40.891094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01421a-34a3-46cb-bc8c-5c7adf3b0eb6 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:40.900458 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4273/8612] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:27:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:27:41.176169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbbaaf-f263-4285-874e-f644f1d68de0 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:41.177633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-61bbbaaf-f263-4285-874e-f644f1d68de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:41.193876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbbaaf-f263-4285-874e-f644f1d68de0 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:41.198761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299d936-d3e2-41d7-823c-4eee2446ff5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:41.199457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299d936-d3e2-41d7-823c-4eee2446ff5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:41.199691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299d936-d3e2-41d7-823c-4eee2446ff5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:41.199966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299d936-d3e2-41d7-823c-4eee2446ff5d admin admin] PATH_INFO: `/v3/projects/ef8c8e969dc040d89f85e78613bf984c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:41.201423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299d936-d3e2-41d7-823c-4eee2446ff5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef8c8e969dc040d89f85e78613bf984c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:41.204022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299d936-d3e2-41d7-823c-4eee2446ff5d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:41.251581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9299d936-d3e2-41d7-823c-4eee2446ff5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef8c8e969dc040d89f85e78613bf984c', 'request_id': 'req-9299d936-d3e2-41d7-823c-4eee2446ff5d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:41.257606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9299d936-d3e2-41d7-823c-4eee2446ff5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef8c8e969dc040d89f85e78613bf984c', 'request_id': 'req-9299d936-d3e2-41d7-823c-4eee2446ff5d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:41.263503 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9299d936-d3e2-41d7-823c-4eee2446ff5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef8c8e969dc040d89f85e78613bf984c', 'request_id': 'req-9299d936-d3e2-41d7-823c-4eee2446ff5d'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:41.276160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9299d936-d3e2-41d7-823c-4eee2446ff5d admin admin] The token cache is being invalidate because project ef8c8e969dc040d89f85e78613bf984c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:27:41.276449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9299d936-d3e2-41d7-823c-4eee2446ff5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:41.277086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9299d936-d3e2-41d7-823c-4eee2446ff5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:41.278688 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4340/8613] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:27:41 2022] DELETE /identity/v3/projects/ef8c8e969dc040d89f85e78613bf984c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:41.282719 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01421a-34a3-46cb-bc8c-5c7adf3b0eb6 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:41.292306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01421a-34a3-46cb-bc8c-5c7adf3b0eb6 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:41.351105 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204e1f1-696a-4b75-8bce-aa5cd852e949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:41.351771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204e1f1-696a-4b75-8bce-aa5cd852e949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:41.352048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204e1f1-696a-4b75-8bce-aa5cd852e949 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:41.352316 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204e1f1-696a-4b75-8bce-aa5cd852e949 admin admin] PATH_INFO: `/v3/users/c787a41f22f74e109994eb6e6a10cb76` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:41.353438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204e1f1-696a-4b75-8bce-aa5cd852e949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c787a41f22f74e109994eb6e6a10cb76)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:41.355503 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204e1f1-696a-4b75-8bce-aa5cd852e949 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:41.357961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:41.407991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b204e1f1-696a-4b75-8bce-aa5cd852e949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c787a41f22f74e109994eb6e6a10cb76'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:41.408652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b204e1f1-696a-4b75-8bce-aa5cd852e949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c787a41f22f74e109994eb6e6a10cb76'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:41.409257 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b204e1f1-696a-4b75-8bce-aa5cd852e949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c787a41f22f74e109994eb6e6a10cb76'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:41.416824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b204e1f1-696a-4b75-8bce-aa5cd852e949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c787a41f22f74e109994eb6e6a10cb76'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:41.430399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b204e1f1-696a-4b75-8bce-aa5cd852e949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c787a41f22f74e109994eb6e6a10cb76'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:41.442662 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4274/8614] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:27:41 2022] DELETE /identity/v3/users/c787a41f22f74e109994eb6e6a10cb76 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:27:41.724204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299d936-d3e2-41d7-823c-4eee2446ff5d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:41.726436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9299d936-d3e2-41d7-823c-4eee2446ff5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:41.740938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299d936-d3e2-41d7-823c-4eee2446ff5d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:41.811072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23804e0-7cf9-4fa0-aa3f-3159f16450b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:41.811747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23804e0-7cf9-4fa0-aa3f-3159f16450b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:41.812055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23804e0-7cf9-4fa0-aa3f-3159f16450b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:41.812328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23804e0-7cf9-4fa0-aa3f-3159f16450b4 admin admin] PATH_INFO: `/v3/projects/0a046cfd45c94cf98849cd0ae078f9a0` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:41.813455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23804e0-7cf9-4fa0-aa3f-3159f16450b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a046cfd45c94cf98849cd0ae078f9a0)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:41.815394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23804e0-7cf9-4fa0-aa3f-3159f16450b4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:41.856337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:41.866863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a23804e0-7cf9-4fa0-aa3f-3159f16450b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a046cfd45c94cf98849cd0ae078f9a0', 'request_id': 'req-a23804e0-7cf9-4fa0-aa3f-3159f16450b4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:41.873763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a23804e0-7cf9-4fa0-aa3f-3159f16450b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a046cfd45c94cf98849cd0ae078f9a0', 'request_id': 'req-a23804e0-7cf9-4fa0-aa3f-3159f16450b4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:41.879928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a23804e0-7cf9-4fa0-aa3f-3159f16450b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a046cfd45c94cf98849cd0ae078f9a0', 'request_id': 'req-a23804e0-7cf9-4fa0-aa3f-3159f16450b4'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:41.893839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a23804e0-7cf9-4fa0-aa3f-3159f16450b4 admin admin] The token cache is being invalidate because project 0a046cfd45c94cf98849cd0ae078f9a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:27:41.894243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a23804e0-7cf9-4fa0-aa3f-3159f16450b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:41.894906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a23804e0-7cf9-4fa0-aa3f-3159f16450b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:41.896487 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4341/8615] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:27:41 2022] DELETE /identity/v3/projects/0a046cfd45c94cf98849cd0ae078f9a0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:41.902625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc238f-55ee-429f-bdf8-651580df474f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:41.903160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc238f-55ee-429f-bdf8-651580df474f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:41.903275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc238f-55ee-429f-bdf8-651580df474f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:41.943551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-99cc238f-55ee-429f-bdf8-651580df474f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:41.944739 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-99cc238f-55ee-429f-bdf8-651580df474f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:42.003499 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4275/8616] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:27:42.011445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8130837-94cc-4e63-aef1-93b5b4c04a88 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:42.011736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8130837-94cc-4e63-aef1-93b5b4c04a88 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:42.012011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8130837-94cc-4e63-aef1-93b5b4c04a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:42.033089 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c8130837-94cc-4e63-aef1-93b5b4c04a88 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:42.034208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c8130837-94cc-4e63-aef1-93b5b4c04a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:42.042258 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4342/8617] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:42.046758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc238f-55ee-429f-bdf8-651580df474f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:42.056500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc238f-55ee-429f-bdf8-651580df474f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:42.061184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a65f5-f216-40ae-a22d-b535de4d8d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:42.061784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a65f5-f216-40ae-a22d-b535de4d8d44 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:42.062068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a65f5-f216-40ae-a22d-b535de4d8d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:42.062348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a65f5-f216-40ae-a22d-b535de4d8d44 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:42.063043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a65f5-f216-40ae-a22d-b535de4d8d44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:27:42.063353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a65f5-f216-40ae-a22d-b535de4d8d44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:42.064963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a65f5-f216-40ae-a22d-b535de4d8d44 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:42.076902 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4276/8618] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:27:42.081121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8130837-94cc-4e63-aef1-93b5b4c04a88 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:42.090601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8130837-94cc-4e63-aef1-93b5b4c04a88 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:42.095998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c05ad-25c9-45f9-b1fb-46861df29451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:42.096596 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c05ad-25c9-45f9-b1fb-46861df29451 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:42.096881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c05ad-25c9-45f9-b1fb-46861df29451 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:42.097158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c05ad-25c9-45f9-b1fb-46861df29451 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:42.097786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c05ad-25c9-45f9-b1fb-46861df29451 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:42.099785 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c05ad-25c9-45f9-b1fb-46861df29451 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:42.130427 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4343/8619] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:27:42 2022] POST /identity/v3/projects => generated 399 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:27:42.131541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a65f5-f216-40ae-a22d-b535de4d8d44 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:42.142680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a65f5-f216-40ae-a22d-b535de4d8d44 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:42.189974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:42.201877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792f3cc-52fa-4386-8cf5-5efeb9401a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:42.202589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792f3cc-52fa-4386-8cf5-5efeb9401a57 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:42.202879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792f3cc-52fa-4386-8cf5-5efeb9401a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:42.203404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792f3cc-52fa-4386-8cf5-5efeb9401a57 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:42.204258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792f3cc-52fa-4386-8cf5-5efeb9401a57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:42.206477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792f3cc-52fa-4386-8cf5-5efeb9401a57 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:42.228071 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4277/8620] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:27:42 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:27:42.232224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c05ad-25c9-45f9-b1fb-46861df29451 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:42.242604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c05ad-25c9-45f9-b1fb-46861df29451 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:42.247956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6582b2-a2d3-4efa-b3a1-e3b745b4ac01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:42.248566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6582b2-a2d3-4efa-b3a1-e3b745b4ac01 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:42.248847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6582b2-a2d3-4efa-b3a1-e3b745b4ac01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:42.249130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6582b2-a2d3-4efa-b3a1-e3b745b4ac01 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:42.249784 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6582b2-a2d3-4efa-b3a1-e3b745b4ac01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:42.251408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6582b2-a2d3-4efa-b3a1-e3b745b4ac01 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:42.261564 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4344/8621] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:27:42.266238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792f3cc-52fa-4386-8cf5-5efeb9401a57 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:42.276519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792f3cc-52fa-4386-8cf5-5efeb9401a57 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:42.281224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb65ae97-2542-44f5-ad7e-dc4e407b4146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:42.281823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65ae97-2542-44f5-ad7e-dc4e407b4146 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:42.282101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65ae97-2542-44f5-ad7e-dc4e407b4146 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:42.282371 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65ae97-2542-44f5-ad7e-dc4e407b4146 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:42.283049 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb65ae97-2542-44f5-ad7e-dc4e407b4146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:42.284691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb65ae97-2542-44f5-ad7e-dc4e407b4146 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:42.295768 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4278/8622] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:27:42.299912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6582b2-a2d3-4efa-b3a1-e3b745b4ac01 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:42.310357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6582b2-a2d3-4efa-b3a1-e3b745b4ac01 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:42.315135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b78a9-c1ef-4b88-9504-0c76a77cf7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:42.315783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b78a9-c1ef-4b88-9504-0c76a77cf7e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:42.316062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b78a9-c1ef-4b88-9504-0c76a77cf7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:42.316335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b78a9-c1ef-4b88-9504-0c76a77cf7e1 admin admin] PATH_INFO: `/v3/projects/a92472a49e5c4940882f847a9b000bae/users/f16e8200d63340188821b6d9ed34577a/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:42.331748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b78a9-c1ef-4b88-9504-0c76a77cf7e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a92472a49e5c4940882f847a9b000bae, user_id=f16e8200d63340188821b6d9ed34577a, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:42.334400 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b78a9-c1ef-4b88-9504-0c76a77cf7e1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:42.354985 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4345/8623] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:27:42 2022] PUT /identity/v3/projects/a92472a49e5c4940882f847a9b000bae/users/f16e8200d63340188821b6d9ed34577a/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:42.360500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a54c3-635c-46ba-b454-56f9aaf4b4dd None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:42.360881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a54c3-635c-46ba-b454-56f9aaf4b4dd None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:42.361226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a54c3-635c-46ba-b454-56f9aaf4b4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:42.389892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f17a54c3-635c-46ba-b454-56f9aaf4b4dd None None] MFA Rules not processed for user `f16e8200d63340188821b6d9ed34577a`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:42.391141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f17a54c3-635c-46ba-b454-56f9aaf4b4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:42.454262 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4279/8624] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:42 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:42.458636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b78a9-c1ef-4b88-9504-0c76a77cf7e1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:42.520915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b78a9-c1ef-4b88-9504-0c76a77cf7e1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:42.525595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c27de9-cf09-41ec-9159-5c17eb84d83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:42.526221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c27de9-cf09-41ec-9159-5c17eb84d83e admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:42.526519 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c27de9-cf09-41ec-9159-5c17eb84d83e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:42.526800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c27de9-cf09-41ec-9159-5c17eb84d83e admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:42.527423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c27de9-cf09-41ec-9159-5c17eb84d83e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:42.529866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c27de9-cf09-41ec-9159-5c17eb84d83e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:42.558446 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4346/8625] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:27:42 2022] POST /identity/v3/projects => generated 399 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:27:42.560165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a54c3-635c-46ba-b454-56f9aaf4b4dd None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:42.571600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a54c3-635c-46ba-b454-56f9aaf4b4dd None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:42.627609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066b2c0-2a7b-4432-8ed3-fd6fc268348e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:42.628209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066b2c0-2a7b-4432-8ed3-fd6fc268348e admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:42.628487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066b2c0-2a7b-4432-8ed3-fd6fc268348e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:42.628765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066b2c0-2a7b-4432-8ed3-fd6fc268348e admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:42.629382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066b2c0-2a7b-4432-8ed3-fd6fc268348e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:42.631324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066b2c0-2a7b-4432-8ed3-fd6fc268348e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:42.654026 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4280/8626] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:27:42 2022] POST /identity/v3/users => generated 306 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:27:42.658064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c27de9-cf09-41ec-9159-5c17eb84d83e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:42.669100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c27de9-cf09-41ec-9159-5c17eb84d83e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:42.673683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725235c-727f-44db-a1d3-d612d50414f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:42.674285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725235c-727f-44db-a1d3-d612d50414f7 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:42.674605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725235c-727f-44db-a1d3-d612d50414f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:42.674887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725235c-727f-44db-a1d3-d612d50414f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:42.675559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725235c-727f-44db-a1d3-d612d50414f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:42.677167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725235c-727f-44db-a1d3-d612d50414f7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:42.687877 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4347/8627] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:27:42.692138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066b2c0-2a7b-4432-8ed3-fd6fc268348e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:42.702448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066b2c0-2a7b-4432-8ed3-fd6fc268348e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:42.707047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65a5cc-89ce-4a05-8553-47ec5ef7a55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:42.707683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65a5cc-89ce-4a05-8553-47ec5ef7a55a admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:42.707960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65a5cc-89ce-4a05-8553-47ec5ef7a55a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:42.708231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65a5cc-89ce-4a05-8553-47ec5ef7a55a admin admin] PATH_INFO: `/v3/projects/71c6f2badaca47a3bd8481097fad59b0/users/d749be8199cd4be3b0c009b65febe668/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:42.723139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65a5cc-89ce-4a05-8553-47ec5ef7a55a admin admin] RBAC: Authorizing `identity:create_grant(project_id=71c6f2badaca47a3bd8481097fad59b0, user_id=d749be8199cd4be3b0c009b65febe668, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:42.725769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65a5cc-89ce-4a05-8553-47ec5ef7a55a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:42.738845 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4281/8628] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:27:42 2022] PUT /identity/v3/projects/71c6f2badaca47a3bd8481097fad59b0/users/d749be8199cd4be3b0c009b65febe668/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:42.742983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725235c-727f-44db-a1d3-d612d50414f7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:42.753265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725235c-727f-44db-a1d3-d612d50414f7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:42.789841 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:42.820919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915deea-5ee5-43fa-93a3-02523d4c6ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:42.821460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915deea-5ee5-43fa-93a3-02523d4c6ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:42.821745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915deea-5ee5-43fa-93a3-02523d4c6ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:42.822018 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915deea-5ee5-43fa-93a3-02523d4c6ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:42.822710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915deea-5ee5-43fa-93a3-02523d4c6ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:42.824304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915deea-5ee5-43fa-93a3-02523d4c6ed8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:42.838261 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4348/8629] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:27:42.839494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65a5cc-89ce-4a05-8553-47ec5ef7a55a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:42.851396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65a5cc-89ce-4a05-8553-47ec5ef7a55a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:42.856028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4e5b7-0dc3-4899-8626-c8f5339c58bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:42.856720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4e5b7-0dc3-4899-8626-c8f5339c58bc admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:42.856999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4e5b7-0dc3-4899-8626-c8f5339c58bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:42.857272 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4e5b7-0dc3-4899-8626-c8f5339c58bc admin admin] PATH_INFO: `/v3/projects/71c6f2badaca47a3bd8481097fad59b0/users/d749be8199cd4be3b0c009b65febe668/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:42.859358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4e5b7-0dc3-4899-8626-c8f5339c58bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=71c6f2badaca47a3bd8481097fad59b0, user_id=d749be8199cd4be3b0c009b65febe668, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:42.862158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4e5b7-0dc3-4899-8626-c8f5339c58bc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:42.874561 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4282/8630] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:27:42 2022] PUT /identity/v3/projects/71c6f2badaca47a3bd8481097fad59b0/users/d749be8199cd4be3b0c009b65febe668/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:42.886779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4da00a-22f6-4db2-bcef-96861f6accfe None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:42.886779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4da00a-22f6-4db2-bcef-96861f6accfe None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:42.887042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4da00a-22f6-4db2-bcef-96861f6accfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:42.916351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2d4da00a-22f6-4db2-bcef-96861f6accfe None None] MFA Rules not processed for user `d749be8199cd4be3b0c009b65febe668`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:42.917565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2d4da00a-22f6-4db2-bcef-96861f6accfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:42.985870 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4349/8631] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:42 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:42.996713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4e5b7-0dc3-4899-8626-c8f5339c58bc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:42.999000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-23e4e5b7-0dc3-4899-8626-c8f5339c58bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:43.014572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:43.064207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4e5b7-0dc3-4899-8626-c8f5339c58bc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:43.067900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4c4df-1e74-4103-ae5f-2070198f6048 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:43.068500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4c4df-1e74-4103-ae5f-2070198f6048 service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:43.068777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4c4df-1e74-4103-ae5f-2070198f6048 service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:43.069046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4c4df-1e74-4103-ae5f-2070198f6048 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:43.078609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b4c4df-1e74-4103-ae5f-2070198f6048 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:43.079975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b4c4df-1e74-4103-ae5f-2070198f6048 service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:43.089978 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4283/8632] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:27:42 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:27:43.165451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4da00a-22f6-4db2-bcef-96861f6accfe None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:43.179200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4da00a-22f6-4db2-bcef-96861f6accfe None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:43.182890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ef999-7f40-4109-83fe-657a6fe595d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:43.183495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ef999-7f40-4109-83fe-657a6fe595d6 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:43.183782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ef999-7f40-4109-83fe-657a6fe595d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:43.184059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ef999-7f40-4109-83fe-657a6fe595d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:43.242199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ef999-7f40-4109-83fe-657a6fe595d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:43.243678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ef999-7f40-4109-83fe-657a6fe595d6 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:43.252828 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4350/8633] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:27:43 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:27:47.441328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4c4df-1e74-4103-ae5f-2070198f6048 service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:47.507544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4c4df-1e74-4103-ae5f-2070198f6048 service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:47.512242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e570c93-b67e-4a2d-87cc-f7291e490070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:47.513462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e570c93-b67e-4a2d-87cc-f7291e490070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:47.513749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e570c93-b67e-4a2d-87cc-f7291e490070 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:47.514026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e570c93-b67e-4a2d-87cc-f7291e490070 admin admin] PATH_INFO: `/v3/users/f16e8200d63340188821b6d9ed34577a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:47.515217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e570c93-b67e-4a2d-87cc-f7291e490070 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f16e8200d63340188821b6d9ed34577a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:47.517172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e570c93-b67e-4a2d-87cc-f7291e490070 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:47.567603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5e570c93-b67e-4a2d-87cc-f7291e490070 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f16e8200d63340188821b6d9ed34577a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:47.568172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5e570c93-b67e-4a2d-87cc-f7291e490070 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f16e8200d63340188821b6d9ed34577a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:47.568779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5e570c93-b67e-4a2d-87cc-f7291e490070 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f16e8200d63340188821b6d9ed34577a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:47.577264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5e570c93-b67e-4a2d-87cc-f7291e490070 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f16e8200d63340188821b6d9ed34577a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:47.590401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5e570c93-b67e-4a2d-87cc-f7291e490070 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f16e8200d63340188821b6d9ed34577a'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:47.603310 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4284/8634] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:27:47 2022] DELETE /identity/v3/users/f16e8200d63340188821b6d9ed34577a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:47.608377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222cf046-5c31-407a-92a8-ca07ad04a1db None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:47.608913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222cf046-5c31-407a-92a8-ca07ad04a1db None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:47.609008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222cf046-5c31-407a-92a8-ca07ad04a1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:47.629133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-222cf046-5c31-407a-92a8-ca07ad04a1db None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:47.630346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-222cf046-5c31-407a-92a8-ca07ad04a1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:47.681467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:47.694471 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4351/8635] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:47.705599 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e570c93-b67e-4a2d-87cc-f7291e490070 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:47.707365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5e570c93-b67e-4a2d-87cc-f7291e490070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:47.748614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:47.776283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e570c93-b67e-4a2d-87cc-f7291e490070 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:47.780014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70878e-a72b-41a3-a4cb-fc0d8595e724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:47.780624 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70878e-a72b-41a3-a4cb-fc0d8595e724 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:47.780904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70878e-a72b-41a3-a4cb-fc0d8595e724 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:47.781175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70878e-a72b-41a3-a4cb-fc0d8595e724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:47.790712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70878e-a72b-41a3-a4cb-fc0d8595e724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:47.792180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70878e-a72b-41a3-a4cb-fc0d8595e724 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:47.802939 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4285/8636] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:27:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:27:48.109456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222cf046-5c31-407a-92a8-ca07ad04a1db None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:48.111502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-222cf046-5c31-407a-92a8-ca07ad04a1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:48.130047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222cf046-5c31-407a-92a8-ca07ad04a1db None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:48.138480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d33cef-b0fc-4384-a61d-871f4013a3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:48.139183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d33cef-b0fc-4384-a61d-871f4013a3b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:48.139467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d33cef-b0fc-4384-a61d-871f4013a3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:48.139748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d33cef-b0fc-4384-a61d-871f4013a3b0 admin admin] PATH_INFO: `/v3/projects/a92472a49e5c4940882f847a9b000bae` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:48.141796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d33cef-b0fc-4384-a61d-871f4013a3b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a92472a49e5c4940882f847a9b000bae)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:48.144281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d33cef-b0fc-4384-a61d-871f4013a3b0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:48.199672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-04d33cef-b0fc-4384-a61d-871f4013a3b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a92472a49e5c4940882f847a9b000bae', 'request_id': 'req-04d33cef-b0fc-4384-a61d-871f4013a3b0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:48.208560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-04d33cef-b0fc-4384-a61d-871f4013a3b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a92472a49e5c4940882f847a9b000bae', 'request_id': 'req-04d33cef-b0fc-4384-a61d-871f4013a3b0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:48.217497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-04d33cef-b0fc-4384-a61d-871f4013a3b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a92472a49e5c4940882f847a9b000bae', 'request_id': 'req-04d33cef-b0fc-4384-a61d-871f4013a3b0'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:48.233340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-04d33cef-b0fc-4384-a61d-871f4013a3b0 admin admin] The token cache is being invalidate because project a92472a49e5c4940882f847a9b000bae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:27:48.233653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-04d33cef-b0fc-4384-a61d-871f4013a3b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:48.234289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-04d33cef-b0fc-4384-a61d-871f4013a3b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:48.239114 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4352/8637] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:27:48 2022] DELETE /identity/v3/projects/a92472a49e5c4940882f847a9b000bae => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:48.240186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70878e-a72b-41a3-a4cb-fc0d8595e724 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:48.250789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70878e-a72b-41a3-a4cb-fc0d8595e724 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:48.326541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99fb02-b412-40b4-9205-50c6e8047cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:48.327341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99fb02-b412-40b4-9205-50c6e8047cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:48.327722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99fb02-b412-40b4-9205-50c6e8047cba admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:48.328098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99fb02-b412-40b4-9205-50c6e8047cba admin admin] PATH_INFO: `/v3/users/d749be8199cd4be3b0c009b65febe668` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:48.329367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99fb02-b412-40b4-9205-50c6e8047cba admin admin] RBAC: Authorizing `identity:delete_user(user_id=d749be8199cd4be3b0c009b65febe668)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:48.331506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99fb02-b412-40b4-9205-50c6e8047cba admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:48.390531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2e99fb02-b412-40b4-9205-50c6e8047cba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd749be8199cd4be3b0c009b65febe668'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:48.391213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2e99fb02-b412-40b4-9205-50c6e8047cba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd749be8199cd4be3b0c009b65febe668'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:48.391954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2e99fb02-b412-40b4-9205-50c6e8047cba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd749be8199cd4be3b0c009b65febe668'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:48.402554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2e99fb02-b412-40b4-9205-50c6e8047cba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd749be8199cd4be3b0c009b65febe668'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:48.420217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-2e99fb02-b412-40b4-9205-50c6e8047cba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd749be8199cd4be3b0c009b65febe668'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:48.437353 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4286/8638] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:27:48 2022] DELETE /identity/v3/users/d749be8199cd4be3b0c009b65febe668 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:27:48.743100 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d33cef-b0fc-4384-a61d-871f4013a3b0 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:48.744731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-04d33cef-b0fc-4384-a61d-871f4013a3b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:48.760125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d33cef-b0fc-4384-a61d-871f4013a3b0 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:48.821614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8c015-3a5d-42d9-8ea6-e454f6166199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:48.822256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8c015-3a5d-42d9-8ea6-e454f6166199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:48.822573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8c015-3a5d-42d9-8ea6-e454f6166199 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:48.822854 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8c015-3a5d-42d9-8ea6-e454f6166199 admin admin] PATH_INFO: `/v3/projects/71c6f2badaca47a3bd8481097fad59b0` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:48.824127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8c015-3a5d-42d9-8ea6-e454f6166199 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71c6f2badaca47a3bd8481097fad59b0)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:48.826082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8c015-3a5d-42d9-8ea6-e454f6166199 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:48.874962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-16e8c015-3a5d-42d9-8ea6-e454f6166199 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71c6f2badaca47a3bd8481097fad59b0', 'request_id': 'req-16e8c015-3a5d-42d9-8ea6-e454f6166199'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:48.881402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-16e8c015-3a5d-42d9-8ea6-e454f6166199 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71c6f2badaca47a3bd8481097fad59b0', 'request_id': 'req-16e8c015-3a5d-42d9-8ea6-e454f6166199'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:48.887433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-16e8c015-3a5d-42d9-8ea6-e454f6166199 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71c6f2badaca47a3bd8481097fad59b0', 'request_id': 'req-16e8c015-3a5d-42d9-8ea6-e454f6166199'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:48.900955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-16e8c015-3a5d-42d9-8ea6-e454f6166199 admin admin] The token cache is being invalidate because project 71c6f2badaca47a3bd8481097fad59b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:27:48.901240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-16e8c015-3a5d-42d9-8ea6-e454f6166199 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:48.901887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-16e8c015-3a5d-42d9-8ea6-e454f6166199 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:48.904045 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4353/8639] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:27:48 2022] DELETE /identity/v3/projects/71c6f2badaca47a3bd8481097fad59b0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:48.909169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a3a56-d3d1-40f4-b3c1-e8787cf235ed None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:48.909636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a3a56-d3d1-40f4-b3c1-e8787cf235ed None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:48.909636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a3a56-d3d1-40f4-b3c1-e8787cf235ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:48.931159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a40a3a56-d3d1-40f4-b3c1-e8787cf235ed None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:48.932360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a40a3a56-d3d1-40f4-b3c1-e8787cf235ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:48.983746 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:48.998078 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4287/8640] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:49.005809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743f666-dfdd-4a5a-aad6-0452aa01f701 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:49.006096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743f666-dfdd-4a5a-aad6-0452aa01f701 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:49.006373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743f666-dfdd-4a5a-aad6-0452aa01f701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:49.027741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a743f666-dfdd-4a5a-aad6-0452aa01f701 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:49.029473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a743f666-dfdd-4a5a-aad6-0452aa01f701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:49.038366 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4354/8641] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:49.043048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a3a56-d3d1-40f4-b3c1-e8787cf235ed None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:49.054309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a3a56-d3d1-40f4-b3c1-e8787cf235ed None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:49.059104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cd56c-a118-46d1-921e-b43c8268a6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:49.059826 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cd56c-a118-46d1-921e-b43c8268a6c4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:49.060178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cd56c-a118-46d1-921e-b43c8268a6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:49.060453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cd56c-a118-46d1-921e-b43c8268a6c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:49.061096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233cd56c-a118-46d1-921e-b43c8268a6c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:27:49.061398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233cd56c-a118-46d1-921e-b43c8268a6c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:49.063038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233cd56c-a118-46d1-921e-b43c8268a6c4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:49.075318 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4288/8642] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:27:49.079806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743f666-dfdd-4a5a-aad6-0452aa01f701 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:49.084433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:49.091722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743f666-dfdd-4a5a-aad6-0452aa01f701 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:49.098036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4078c38-c13f-413f-b6cb-dba493a06aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:49.098698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4078c38-c13f-413f-b6cb-dba493a06aef admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:49.098979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4078c38-c13f-413f-b6cb-dba493a06aef admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:49.099252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4078c38-c13f-413f-b6cb-dba493a06aef admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:49.099887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4078c38-c13f-413f-b6cb-dba493a06aef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:49.101868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4078c38-c13f-413f-b6cb-dba493a06aef admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:49.131008 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4355/8643] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:27:49 2022] POST /identity/v3/projects => generated 399 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:27:49.135295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cd56c-a118-46d1-921e-b43c8268a6c4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:49.146002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cd56c-a118-46d1-921e-b43c8268a6c4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:49.210636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a56ed0-2806-497b-98d6-c212778f82b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:49.211252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a56ed0-2806-497b-98d6-c212778f82b1 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:49.211536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a56ed0-2806-497b-98d6-c212778f82b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:49.211810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a56ed0-2806-497b-98d6-c212778f82b1 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:49.212500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a56ed0-2806-497b-98d6-c212778f82b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:49.214448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a56ed0-2806-497b-98d6-c212778f82b1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:49.238396 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4289/8644] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:27:49 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:27:49.242736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4078c38-c13f-413f-b6cb-dba493a06aef admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:49.253606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4078c38-c13f-413f-b6cb-dba493a06aef admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:49.258736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad171d42-ddd9-45ce-b7a4-3dae86e582ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:49.259416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad171d42-ddd9-45ce-b7a4-3dae86e582ae admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:49.259765 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad171d42-ddd9-45ce-b7a4-3dae86e582ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:49.260141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad171d42-ddd9-45ce-b7a4-3dae86e582ae admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:49.260855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad171d42-ddd9-45ce-b7a4-3dae86e582ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:49.263408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad171d42-ddd9-45ce-b7a4-3dae86e582ae admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:49.276789 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4356/8645] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:27:49.278275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a56ed0-2806-497b-98d6-c212778f82b1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:49.288891 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a56ed0-2806-497b-98d6-c212778f82b1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:49.293713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf61720-80f2-46b9-b0a7-a6444628a74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:49.294337 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf61720-80f2-46b9-b0a7-a6444628a74e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:49.294653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf61720-80f2-46b9-b0a7-a6444628a74e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:49.294927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf61720-80f2-46b9-b0a7-a6444628a74e admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:49.295558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf61720-80f2-46b9-b0a7-a6444628a74e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:49.297147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf61720-80f2-46b9-b0a7-a6444628a74e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:49.308342 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4290/8646] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:27:49.312560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad171d42-ddd9-45ce-b7a4-3dae86e582ae admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:49.323510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad171d42-ddd9-45ce-b7a4-3dae86e582ae admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:49.328683 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054cf49-1711-4485-a230-f6201942711d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:49.329337 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054cf49-1711-4485-a230-f6201942711d admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:49.329620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054cf49-1711-4485-a230-f6201942711d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:49.329926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054cf49-1711-4485-a230-f6201942711d admin admin] PATH_INFO: `/v3/projects/291dbdcbbe824d0ea41ba7f5152148b9/users/19579ef261d7403c80f415c0ca12f8ed/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:49.346029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054cf49-1711-4485-a230-f6201942711d admin admin] RBAC: Authorizing `identity:create_grant(project_id=291dbdcbbe824d0ea41ba7f5152148b9, user_id=19579ef261d7403c80f415c0ca12f8ed, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:49.348767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054cf49-1711-4485-a230-f6201942711d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:49.363134 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4357/8647] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:27:49 2022] PUT /identity/v3/projects/291dbdcbbe824d0ea41ba7f5152148b9/users/19579ef261d7403c80f415c0ca12f8ed/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:49.375895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06630a86-100f-449c-886c-7ff142b86cdd None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:49.376198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06630a86-100f-449c-886c-7ff142b86cdd None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:49.376476 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06630a86-100f-449c-886c-7ff142b86cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:49.406121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-06630a86-100f-449c-886c-7ff142b86cdd None None] MFA Rules not processed for user `19579ef261d7403c80f415c0ca12f8ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:49.407398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-06630a86-100f-449c-886c-7ff142b86cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:49.472516 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4291/8648] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:49 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:49.483762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054cf49-1711-4485-a230-f6201942711d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:49.485475 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f054cf49-1711-4485-a230-f6201942711d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:49.546401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054cf49-1711-4485-a230-f6201942711d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:49.550069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32780bb0-a658-4d29-8c5e-f783f98899c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:49.550700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32780bb0-a658-4d29-8c5e-f783f98899c7 service cinder] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:49.550979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32780bb0-a658-4d29-8c5e-f783f98899c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:49.551248 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32780bb0-a658-4d29-8c5e-f783f98899c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:49.559602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32780bb0-a658-4d29-8c5e-f783f98899c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:49.561026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32780bb0-a658-4d29-8c5e-f783f98899c7 service cinder] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:49.569954 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4358/8649] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:27:49 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:27:55.611606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06630a86-100f-449c-886c-7ff142b86cdd None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:55.670456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:55.684618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06630a86-100f-449c-886c-7ff142b86cdd None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:55.689963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e1c5c-83db-4caa-b09f-fdf802d56435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:55.690764 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e1c5c-83db-4caa-b09f-fdf802d56435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:55.691101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e1c5c-83db-4caa-b09f-fdf802d56435 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:55.691422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e1c5c-83db-4caa-b09f-fdf802d56435 admin admin] PATH_INFO: `/v3/users/19579ef261d7403c80f415c0ca12f8ed` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:55.692801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e1c5c-83db-4caa-b09f-fdf802d56435 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19579ef261d7403c80f415c0ca12f8ed)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:55.695194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e1c5c-83db-4caa-b09f-fdf802d56435 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:55.746149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b24e1c5c-83db-4caa-b09f-fdf802d56435 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19579ef261d7403c80f415c0ca12f8ed'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:55.746992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b24e1c5c-83db-4caa-b09f-fdf802d56435 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19579ef261d7403c80f415c0ca12f8ed'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:55.747644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b24e1c5c-83db-4caa-b09f-fdf802d56435 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19579ef261d7403c80f415c0ca12f8ed'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:55.756157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b24e1c5c-83db-4caa-b09f-fdf802d56435 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19579ef261d7403c80f415c0ca12f8ed'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:55.770282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b24e1c5c-83db-4caa-b09f-fdf802d56435 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19579ef261d7403c80f415c0ca12f8ed'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:55.784618 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4292/8650] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:27:55 2022] DELETE /identity/v3/users/19579ef261d7403c80f415c0ca12f8ed => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:55.789221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6be962-2016-4eef-8af9-10df519793fe None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:55.789707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6be962-2016-4eef-8af9-10df519793fe None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:55.789859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6be962-2016-4eef-8af9-10df519793fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:55.810808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0a6be962-2016-4eef-8af9-10df519793fe None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:55.812163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0a6be962-2016-4eef-8af9-10df519793fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:55.881585 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4359/8651] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:55.892976 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e1c5c-83db-4caa-b09f-fdf802d56435 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:55.894752 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b24e1c5c-83db-4caa-b09f-fdf802d56435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:55.977013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e1c5c-83db-4caa-b09f-fdf802d56435 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:55.981737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea498f23-d9c6-41bd-b2f0-c79427e483e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:55.983068 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea498f23-d9c6-41bd-b2f0-c79427e483e2 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:55.983425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea498f23-d9c6-41bd-b2f0-c79427e483e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:55.983762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea498f23-d9c6-41bd-b2f0-c79427e483e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:55.994346 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea498f23-d9c6-41bd-b2f0-c79427e483e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:55.996116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea498f23-d9c6-41bd-b2f0-c79427e483e2 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:56.007457 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4293/8652] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:27:56.340386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6be962-2016-4eef-8af9-10df519793fe None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:56.342950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0a6be962-2016-4eef-8af9-10df519793fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:56.362388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6be962-2016-4eef-8af9-10df519793fe None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:56.372645 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39e3e0-51bc-483a-8550-c6bdb88881c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:56.373553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39e3e0-51bc-483a-8550-c6bdb88881c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:56.373891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39e3e0-51bc-483a-8550-c6bdb88881c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:56.374288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39e3e0-51bc-483a-8550-c6bdb88881c7 admin admin] PATH_INFO: `/v3/projects/291dbdcbbe824d0ea41ba7f5152148b9` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:56.375762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39e3e0-51bc-483a-8550-c6bdb88881c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=291dbdcbbe824d0ea41ba7f5152148b9)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:56.378012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39e3e0-51bc-483a-8550-c6bdb88881c7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:56.432035 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:56.437031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3e39e3e0-51bc-483a-8550-c6bdb88881c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '291dbdcbbe824d0ea41ba7f5152148b9', 'request_id': 'req-3e39e3e0-51bc-483a-8550-c6bdb88881c7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:56.446859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3e39e3e0-51bc-483a-8550-c6bdb88881c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '291dbdcbbe824d0ea41ba7f5152148b9', 'request_id': 'req-3e39e3e0-51bc-483a-8550-c6bdb88881c7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:56.454207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3e39e3e0-51bc-483a-8550-c6bdb88881c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '291dbdcbbe824d0ea41ba7f5152148b9', 'request_id': 'req-3e39e3e0-51bc-483a-8550-c6bdb88881c7'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:56.467488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3e39e3e0-51bc-483a-8550-c6bdb88881c7 admin admin] The token cache is being invalidate because project 291dbdcbbe824d0ea41ba7f5152148b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:27:56.467773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3e39e3e0-51bc-483a-8550-c6bdb88881c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:56.468437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3e39e3e0-51bc-483a-8550-c6bdb88881c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:27:56.469898 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4360/8653] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:27:56 2022] DELETE /identity/v3/projects/291dbdcbbe824d0ea41ba7f5152148b9 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:56.475557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92303d1-70fd-44eb-8caf-215e31327345 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:56.475845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92303d1-70fd-44eb-8caf-215e31327345 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:56.476173 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92303d1-70fd-44eb-8caf-215e31327345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:56.498457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c92303d1-70fd-44eb-8caf-215e31327345 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:56.500742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c92303d1-70fd-44eb-8caf-215e31327345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:56.569763 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4294/8654] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:27:56.579007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872ced8-4401-45db-ab3a-c207e8976c57 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:56.579271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872ced8-4401-45db-ab3a-c207e8976c57 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:56.579544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872ced8-4401-45db-ab3a-c207e8976c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:56.601039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-0872ced8-4401-45db-ab3a-c207e8976c57 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:56.602417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0872ced8-4401-45db-ab3a-c207e8976c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:56.612060 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4361/8655] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:27:56.616938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92303d1-70fd-44eb-8caf-215e31327345 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:56.622110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:56.628187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92303d1-70fd-44eb-8caf-215e31327345 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:56.635118 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334de583-bdf8-43d1-a012-bb287a7d5d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:56.635963 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334de583-bdf8-43d1-a012-bb287a7d5d87 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:56.636523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334de583-bdf8-43d1-a012-bb287a7d5d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:56.637203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334de583-bdf8-43d1-a012-bb287a7d5d87 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:56.638906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334de583-bdf8-43d1-a012-bb287a7d5d87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:27:56.639293 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334de583-bdf8-43d1-a012-bb287a7d5d87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:56.641163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334de583-bdf8-43d1-a012-bb287a7d5d87 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:56.656131 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4295/8656] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:27:56.660992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872ced8-4401-45db-ab3a-c207e8976c57 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:56.674521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872ced8-4401-45db-ab3a-c207e8976c57 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:56.680076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1c940-2628-4248-b923-9752b51152b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:56.680716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1c940-2628-4248-b923-9752b51152b4 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:56.681006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1c940-2628-4248-b923-9752b51152b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:56.681369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1c940-2628-4248-b923-9752b51152b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:56.682112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1c940-2628-4248-b923-9752b51152b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:56.684169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1c940-2628-4248-b923-9752b51152b4 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:56.719205 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4362/8657] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:27:56 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:27:56.720387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334de583-bdf8-43d1-a012-bb287a7d5d87 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:56.730803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334de583-bdf8-43d1-a012-bb287a7d5d87 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:56.794453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6c0b8-106d-4f2b-a3a7-4a3400532400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:56.795216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6c0b8-106d-4f2b-a3a7-4a3400532400 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:56.795541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6c0b8-106d-4f2b-a3a7-4a3400532400 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:56.795845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6c0b8-106d-4f2b-a3a7-4a3400532400 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:56.796468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff6c0b8-106d-4f2b-a3a7-4a3400532400 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:56.798412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff6c0b8-106d-4f2b-a3a7-4a3400532400 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:56.821548 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4296/8658] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:27:56 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:27:56.826505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1c940-2628-4248-b923-9752b51152b4 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:56.838163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1c940-2628-4248-b923-9752b51152b4 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:56.843413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeee38d-ea84-4604-8ceb-3bdd4b3f169d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:56.844026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeee38d-ea84-4604-8ceb-3bdd4b3f169d admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:56.844309 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeee38d-ea84-4604-8ceb-3bdd4b3f169d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:56.844599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeee38d-ea84-4604-8ceb-3bdd4b3f169d admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:56.845355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeee38d-ea84-4604-8ceb-3bdd4b3f169d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:56.847292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeee38d-ea84-4604-8ceb-3bdd4b3f169d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:56.858508 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4363/8659] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:27:56.863172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6c0b8-106d-4f2b-a3a7-4a3400532400 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:56.874951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6c0b8-106d-4f2b-a3a7-4a3400532400 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:56.880259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72dcb70-b807-43e1-aa68-daa4aee2f94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:56.880949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72dcb70-b807-43e1-aa68-daa4aee2f94c admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:56.881964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72dcb70-b807-43e1-aa68-daa4aee2f94c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:56.882231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72dcb70-b807-43e1-aa68-daa4aee2f94c admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:56.882921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72dcb70-b807-43e1-aa68-daa4aee2f94c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:56.884851 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72dcb70-b807-43e1-aa68-daa4aee2f94c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:56.898654 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4297/8660] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:27:56.903204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeee38d-ea84-4604-8ceb-3bdd4b3f169d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:56.913901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeee38d-ea84-4604-8ceb-3bdd4b3f169d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:56.919028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818492e-a2d8-4823-9569-40ca9e2596b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:56.919725 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818492e-a2d8-4823-9569-40ca9e2596b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:56.920010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818492e-a2d8-4823-9569-40ca9e2596b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:56.920316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818492e-a2d8-4823-9569-40ca9e2596b2 admin admin] PATH_INFO: `/v3/projects/ff07b6ab40b84bd18a5a44deb9516f00/users/1c1fc5de87ed44e7b01229925bf27242/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:56.936281 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818492e-a2d8-4823-9569-40ca9e2596b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff07b6ab40b84bd18a5a44deb9516f00, user_id=1c1fc5de87ed44e7b01229925bf27242, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:56.938966 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818492e-a2d8-4823-9569-40ca9e2596b2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:56.956015 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4364/8661] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:27:56 2022] PUT /identity/v3/projects/ff07b6ab40b84bd18a5a44deb9516f00/users/1c1fc5de87ed44e7b01229925bf27242/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:27:56.961336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3eb8c-0ace-402a-be03-338e491946e9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:56.961616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3eb8c-0ace-402a-be03-338e491946e9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:56.961895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3eb8c-0ace-402a-be03-338e491946e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:56.980642 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-f4e3eb8c-0ace-402a-be03-338e491946e9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:27:56.981859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f4e3eb8c-0ace-402a-be03-338e491946e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:57.045930 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4298/8662] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:27:57.056979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818492e-a2d8-4823-9569-40ca9e2596b2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:27:57.058973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5818492e-a2d8-4823-9569-40ca9e2596b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:27:57.124872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818492e-a2d8-4823-9569-40ca9e2596b2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:27:57.128490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab67296-4f95-430a-94ca-de9cb3148340 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:27:57.129090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab67296-4f95-430a-94ca-de9cb3148340 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:27:57.129367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab67296-4f95-430a-94ca-de9cb3148340 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:27:57.129634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab67296-4f95-430a-94ca-de9cb3148340 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:27:57.139365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab67296-4f95-430a-94ca-de9cb3148340 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:27:57.140773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab67296-4f95-430a-94ca-de9cb3148340 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:27:57.150662 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4365/8663] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:27:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:28:01.181448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96da6dd-0981-4be9-93d4-dfdfadbc3bb9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:28:01.182010 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96da6dd-0981-4be9-93d4-dfdfadbc3bb9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:28:01.182221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96da6dd-0981-4be9-93d4-dfdfadbc3bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:28:01.212060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d96da6dd-0981-4be9-93d4-dfdfadbc3bb9 None None] MFA Rules not processed for user `1c1fc5de87ed44e7b01229925bf27242`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:28:01.213282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d96da6dd-0981-4be9-93d4-dfdfadbc3bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:28:01.274693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:28:01.284705 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4299/8664] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:28:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:28:01.286049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab67296-4f95-430a-94ca-de9cb3148340 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:28:01.296831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab67296-4f95-430a-94ca-de9cb3148340 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:28:01.301717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37885d15-eca7-4230-a6da-663222d8746a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:28:01.302399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37885d15-eca7-4230-a6da-663222d8746a admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:28:01.302753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37885d15-eca7-4230-a6da-663222d8746a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:28:01.303202 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37885d15-eca7-4230-a6da-663222d8746a admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:28:01.303850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37885d15-eca7-4230-a6da-663222d8746a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:28:01.305768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37885d15-eca7-4230-a6da-663222d8746a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:28:01.334171 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4366/8665] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:28:01 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:28:01.338522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96da6dd-0981-4be9-93d4-dfdfadbc3bb9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:28:01.349983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96da6dd-0981-4be9-93d4-dfdfadbc3bb9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:28:01.419373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59674a1-0f21-4e7a-86bf-41d565afdb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:28:01.420005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59674a1-0f21-4e7a-86bf-41d565afdb96 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:28:01.420281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59674a1-0f21-4e7a-86bf-41d565afdb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:28:01.420547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59674a1-0f21-4e7a-86bf-41d565afdb96 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:28:01.421180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59674a1-0f21-4e7a-86bf-41d565afdb96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:28:01.423134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59674a1-0f21-4e7a-86bf-41d565afdb96 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:28:01.451580 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4300/8666] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:28:01 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:28:01.452464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37885d15-eca7-4230-a6da-663222d8746a admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:28:01.464111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37885d15-eca7-4230-a6da-663222d8746a admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:28:01.473064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98cb5e2-6972-4f0b-b6f0-e4d0ef3143ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:28:01.473677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98cb5e2-6972-4f0b-b6f0-e4d0ef3143ac admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:28:01.473955 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98cb5e2-6972-4f0b-b6f0-e4d0ef3143ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:28:01.474228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98cb5e2-6972-4f0b-b6f0-e4d0ef3143ac admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:28:01.474911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98cb5e2-6972-4f0b-b6f0-e4d0ef3143ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:28:01.476558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98cb5e2-6972-4f0b-b6f0-e4d0ef3143ac admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:28:01.487792 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4367/8667] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:28:01.492219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59674a1-0f21-4e7a-86bf-41d565afdb96 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:28:01.503227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59674a1-0f21-4e7a-86bf-41d565afdb96 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:28:01.507931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0eb39e-d144-4422-801e-f0045b75acc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:28:01.508590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0eb39e-d144-4422-801e-f0045b75acc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:28:01.508874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0eb39e-d144-4422-801e-f0045b75acc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:28:01.509165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0eb39e-d144-4422-801e-f0045b75acc4 admin admin] PATH_INFO: `/v3/projects/5bc2cce0aba64a5792e67bd94a15453c/users/9a0280cb98a54b9e80de4b286b28e077/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:28:01.525731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0eb39e-d144-4422-801e-f0045b75acc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bc2cce0aba64a5792e67bd94a15453c, user_id=9a0280cb98a54b9e80de4b286b28e077, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:28:01.528534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0eb39e-d144-4422-801e-f0045b75acc4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:28:01.542906 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4301/8668] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:28:01 2022] PUT /identity/v3/projects/5bc2cce0aba64a5792e67bd94a15453c/users/9a0280cb98a54b9e80de4b286b28e077/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:28:01.547000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98cb5e2-6972-4f0b-b6f0-e4d0ef3143ac admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:28:01.557432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98cb5e2-6972-4f0b-b6f0-e4d0ef3143ac admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:28:01.605723 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:28:01.629944 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289a739-4c60-4066-9267-9af4ab00ca4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:28:01.630597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289a739-4c60-4066-9267-9af4ab00ca4f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:28:01.630879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289a739-4c60-4066-9267-9af4ab00ca4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:28:01.631724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289a739-4c60-4066-9267-9af4ab00ca4f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:28:01.632476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289a739-4c60-4066-9267-9af4ab00ca4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:28:01.635097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289a739-4c60-4066-9267-9af4ab00ca4f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:28:01.652904 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4368/8669] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:28:01.654770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0eb39e-d144-4422-801e-f0045b75acc4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:28:01.665864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0eb39e-d144-4422-801e-f0045b75acc4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:28:01.671669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c56a8d-c022-43f6-abe6-8e645e27806a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:28:01.672341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c56a8d-c022-43f6-abe6-8e645e27806a admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:28:01.672623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c56a8d-c022-43f6-abe6-8e645e27806a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:28:01.672898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c56a8d-c022-43f6-abe6-8e645e27806a admin admin] PATH_INFO: `/v3/projects/5bc2cce0aba64a5792e67bd94a15453c/users/9a0280cb98a54b9e80de4b286b28e077/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:28:01.674915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c56a8d-c022-43f6-abe6-8e645e27806a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bc2cce0aba64a5792e67bd94a15453c, user_id=9a0280cb98a54b9e80de4b286b28e077, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:28:01.677643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c56a8d-c022-43f6-abe6-8e645e27806a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:28:01.689679 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4302/8670] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:28:01 2022] PUT /identity/v3/projects/5bc2cce0aba64a5792e67bd94a15453c/users/9a0280cb98a54b9e80de4b286b28e077/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:28:06.761829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cb13a-b2e6-48b4-a815-f09246b8adb0 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:28:06.762336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cb13a-b2e6-48b4-a815-f09246b8adb0 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:28:06.762440 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cb13a-b2e6-48b4-a815-f09246b8adb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:28:06.816908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-377cb13a-b2e6-48b4-a815-f09246b8adb0 None None] MFA Rules not processed for user `9a0280cb98a54b9e80de4b286b28e077`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:28:06.818443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-377cb13a-b2e6-48b4-a815-f09246b8adb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:28:06.895950 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4369/8671] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:28:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:28:06.917966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c56a8d-c022-43f6-abe6-8e645e27806a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:28:06.922403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b3c56a8d-c022-43f6-abe6-8e645e27806a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:28:07.036402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c56a8d-c022-43f6-abe6-8e645e27806a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:28:07.050915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84682585-0249-4adb-8edc-220408e3feea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:28:07.051661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84682585-0249-4adb-8edc-220408e3feea service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:28:07.052051 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84682585-0249-4adb-8edc-220408e3feea service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:28:07.052373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84682585-0249-4adb-8edc-220408e3feea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:28:07.088617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:28:07.127237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84682585-0249-4adb-8edc-220408e3feea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:28:07.129005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84682585-0249-4adb-8edc-220408e3feea service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:28:07.140653 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4303/8672] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:28:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:28:08.421264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800ea50-5ea9-4523-8fc6-4cd025aff981 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:28:08.421748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800ea50-5ea9-4523-8fc6-4cd025aff981 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:28:08.421851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800ea50-5ea9-4523-8fc6-4cd025aff981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:28:08.447109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-1800ea50-5ea9-4523-8fc6-4cd025aff981 None None] MFA Rules not processed for user `1c1fc5de87ed44e7b01229925bf27242`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:28:08.448401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1800ea50-5ea9-4523-8fc6-4cd025aff981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:28:08.457523 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4370/8673] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:28:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:28:17.134856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=14 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:28:18.440741 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=49 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:28:19.384869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84682585-0249-4adb-8edc-220408e3feea service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:28:19.386307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-84682585-0249-4adb-8edc-220408e3feea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:28:19.445501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84682585-0249-4adb-8edc-220408e3feea service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:28:19.449145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af9010-263b-4961-9013-ffb3e2995e30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:28:19.449747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af9010-263b-4961-9013-ffb3e2995e30 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:28:19.450032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af9010-263b-4961-9013-ffb3e2995e30 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:28:19.450308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af9010-263b-4961-9013-ffb3e2995e30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:28:19.459652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af9010-263b-4961-9013-ffb3e2995e30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:28:19.461037 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af9010-263b-4961-9013-ffb3e2995e30 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:28:19.471242 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4304/8674] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:28:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:28:29.466309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=26 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:28:41.151114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800ea50-5ea9-4523-8fc6-4cd025aff981 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:28:41.152832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-1800ea50-5ea9-4523-8fc6-4cd025aff981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:28:41.215782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af9010-263b-4961-9013-ffb3e2995e30 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:28:41.217916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-73af9010-263b-4961-9013-ffb3e2995e30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:28:41.234809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800ea50-5ea9-4523-8fc6-4cd025aff981 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:28:41.239976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d84cb4-c7d2-4cf9-8f27-fa19bc4f9dae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:28:41.240584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d84cb4-c7d2-4cf9-8f27-fa19bc4f9dae service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:28:41.240860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d84cb4-c7d2-4cf9-8f27-fa19bc4f9dae service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:28:41.241130 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d84cb4-c7d2-4cf9-8f27-fa19bc4f9dae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:28:41.242940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-93d84cb4-c7d2-4cf9-8f27-fa19bc4f9dae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:28:41.258219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d84cb4-c7d2-4cf9-8f27-fa19bc4f9dae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:28:41.259872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d84cb4-c7d2-4cf9-8f27-fa19bc4f9dae service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:28:41.273907 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4371/8675] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:28:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:28:41.292436 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af9010-263b-4961-9013-ffb3e2995e30 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:28:41.297239 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da53793-11a7-4955-b847-c1b92cc553da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:28:41.297881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53793-11a7-4955-b847-c1b92cc553da admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:28:41.298159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53793-11a7-4955-b847-c1b92cc553da admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:28:41.298431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53793-11a7-4955-b847-c1b92cc553da admin admin] PATH_INFO: `/v3/users/98361240b014405cb20cf9c145061be6` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:28:41.299588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da53793-11a7-4955-b847-c1b92cc553da admin admin] RBAC: Authorizing `identity:delete_user(user_id=98361240b014405cb20cf9c145061be6)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:28:41.302036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da53793-11a7-4955-b847-c1b92cc553da admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:28:41.347912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7da53793-11a7-4955-b847-c1b92cc553da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98361240b014405cb20cf9c145061be6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:28:41.348586 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7da53793-11a7-4955-b847-c1b92cc553da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98361240b014405cb20cf9c145061be6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:28:41.349218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7da53793-11a7-4955-b847-c1b92cc553da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98361240b014405cb20cf9c145061be6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:28:41.356389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7da53793-11a7-4955-b847-c1b92cc553da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98361240b014405cb20cf9c145061be6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:28:41.368999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-7da53793-11a7-4955-b847-c1b92cc553da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98361240b014405cb20cf9c145061be6'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:28:41.381492 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4305/8676] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:28:41 2022] DELETE /identity/v3/users/98361240b014405cb20cf9c145061be6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:28:41.555208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d84cb4-c7d2-4cf9-8f27-fa19bc4f9dae service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:28:41.568247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d84cb4-c7d2-4cf9-8f27-fa19bc4f9dae service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:28:41.624904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada131e-fa72-45cb-a361-a6b41d104331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:28:41.625538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada131e-fa72-45cb-a361-a6b41d104331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:28:41.625833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada131e-fa72-45cb-a361-a6b41d104331 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:28:41.626097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada131e-fa72-45cb-a361-a6b41d104331 admin admin] PATH_INFO: `/v3/projects/dd6e51791d934c73bf7d92e7ed1a8909` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:28:41.627245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada131e-fa72-45cb-a361-a6b41d104331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd6e51791d934c73bf7d92e7ed1a8909)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:28:41.629802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada131e-fa72-45cb-a361-a6b41d104331 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:28:41.675460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3ada131e-fa72-45cb-a361-a6b41d104331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd6e51791d934c73bf7d92e7ed1a8909', 'request_id': 'req-3ada131e-fa72-45cb-a361-a6b41d104331'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:28:41.682267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3ada131e-fa72-45cb-a361-a6b41d104331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd6e51791d934c73bf7d92e7ed1a8909', 'request_id': 'req-3ada131e-fa72-45cb-a361-a6b41d104331'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:28:41.687868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3ada131e-fa72-45cb-a361-a6b41d104331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd6e51791d934c73bf7d92e7ed1a8909', 'request_id': 'req-3ada131e-fa72-45cb-a361-a6b41d104331'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:28:41.700055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3ada131e-fa72-45cb-a361-a6b41d104331 admin admin] The token cache is being invalidate because project dd6e51791d934c73bf7d92e7ed1a8909 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:28:41.700406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3ada131e-fa72-45cb-a361-a6b41d104331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:28:41.701031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3ada131e-fa72-45cb-a361-a6b41d104331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:28:41.702509 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4372/8677] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:28:41 2022] DELETE /identity/v3/projects/dd6e51791d934c73bf7d92e7ed1a8909 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:28:51.377387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:28:51.696937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:28:55.240833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da53793-11a7-4955-b847-c1b92cc553da admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:28:55.242278 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7da53793-11a7-4955-b847-c1b92cc553da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:28:55.304612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da53793-11a7-4955-b847-c1b92cc553da admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:28:55.353324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c85e75-aa40-4021-9fd4-64f64f31aa54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:28:55.354014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c85e75-aa40-4021-9fd4-64f64f31aa54 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:28:55.354300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c85e75-aa40-4021-9fd4-64f64f31aa54 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:28:55.354615 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c85e75-aa40-4021-9fd4-64f64f31aa54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:28:55.356398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-36c85e75-aa40-4021-9fd4-64f64f31aa54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:28:55.368965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c85e75-aa40-4021-9fd4-64f64f31aa54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:28:55.370433 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c85e75-aa40-4021-9fd4-64f64f31aa54 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:28:55.380839 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4306/8678] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:28:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:29:05.375015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=46 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:29:30.024607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada131e-fa72-45cb-a361-a6b41d104331 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:29:30.026068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3ada131e-fa72-45cb-a361-a6b41d104331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:29:30.041734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada131e-fa72-45cb-a361-a6b41d104331 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:29:30.046445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c736b-5a0c-4a1a-b7dc-80766dae7c07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:29:30.046671 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c736b-5a0c-4a1a-b7dc-80766dae7c07 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:29:30.046953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c736b-5a0c-4a1a-b7dc-80766dae7c07 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:29:30.047216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c736b-5a0c-4a1a-b7dc-80766dae7c07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:29:30.048950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-887c736b-5a0c-4a1a-b7dc-80766dae7c07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:29:30.124004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c736b-5a0c-4a1a-b7dc-80766dae7c07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:29:30.125448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c736b-5a0c-4a1a-b7dc-80766dae7c07 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:29:30.140115 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4373/8679] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:29:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:29:40.133717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=34 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:30:05.701283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c85e75-aa40-4021-9fd4-64f64f31aa54 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:30:05.714252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c85e75-aa40-4021-9fd4-64f64f31aa54 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:30:05.718745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4a93b-5f6d-4782-b5fd-692bbef627d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:30:05.719565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4a93b-5f6d-4782-b5fd-692bbef627d9 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:30:05.719797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4a93b-5f6d-4782-b5fd-692bbef627d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:30:05.720225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4a93b-5f6d-4782-b5fd-692bbef627d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:30:05.722264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2ee4a93b-5f6d-4782-b5fd-692bbef627d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:30:05.736119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee4a93b-5f6d-4782-b5fd-692bbef627d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:30:05.737634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee4a93b-5f6d-4782-b5fd-692bbef627d9 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:30:05.751176 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4307/8680] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:30:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:30:05.837744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c736b-5a0c-4a1a-b7dc-80766dae7c07 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:30:05.839249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-887c736b-5a0c-4a1a-b7dc-80766dae7c07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:30:05.852782 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c736b-5a0c-4a1a-b7dc-80766dae7c07 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:30:05.856982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8fefc-1943-4ede-8e8a-888fef348b4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:30:05.857583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8fefc-1943-4ede-8e8a-888fef348b4a service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:30:05.857866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8fefc-1943-4ede-8e8a-888fef348b4a service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:30:05.858142 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8fefc-1943-4ede-8e8a-888fef348b4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:30:05.860011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-57b8fefc-1943-4ede-8e8a-888fef348b4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:30:05.876713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8fefc-1943-4ede-8e8a-888fef348b4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:30:05.878137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8fefc-1943-4ede-8e8a-888fef348b4a service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:30:05.892022 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4374/8681] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:30:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:30:08.055204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4a93b-5f6d-4782-b5fd-692bbef627d9 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:30:08.057203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2ee4a93b-5f6d-4782-b5fd-692bbef627d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:30:08.077699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4a93b-5f6d-4782-b5fd-692bbef627d9 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:30:08.083656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5bf7c-f8e3-4055-a0f7-6313340360b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:30:08.084648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5bf7c-f8e3-4055-a0f7-6313340360b8 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:30:08.084648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5bf7c-f8e3-4055-a0f7-6313340360b8 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:30:08.084866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5bf7c-f8e3-4055-a0f7-6313340360b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:30:08.086607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-dae5bf7c-f8e3-4055-a0f7-6313340360b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:30:08.102808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5bf7c-f8e3-4055-a0f7-6313340360b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:30:08.104444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5bf7c-f8e3-4055-a0f7-6313340360b8 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:30:08.120392 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4308/8682] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:30:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:30:08.218939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8fefc-1943-4ede-8e8a-888fef348b4a service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:30:08.231182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8fefc-1943-4ede-8e8a-888fef348b4a service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:30:08.235250 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113bf1d-0804-4281-b527-4de4e7ee9020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:30:08.235867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113bf1d-0804-4281-b527-4de4e7ee9020 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:30:08.236154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113bf1d-0804-4281-b527-4de4e7ee9020 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:30:08.236432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113bf1d-0804-4281-b527-4de4e7ee9020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:30:08.238242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e113bf1d-0804-4281-b527-4de4e7ee9020 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:30:08.251285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113bf1d-0804-4281-b527-4de4e7ee9020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:30:08.252916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113bf1d-0804-4281-b527-4de4e7ee9020 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:30:08.266180 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4375/8683] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:30:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:30:12.637967 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5bf7c-f8e3-4055-a0f7-6313340360b8 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:30:12.650570 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5bf7c-f8e3-4055-a0f7-6313340360b8 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:30:12.654264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520c0aa-6e69-404b-a823-f53a67a44508 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:30:12.654960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520c0aa-6e69-404b-a823-f53a67a44508 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:30:12.655244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520c0aa-6e69-404b-a823-f53a67a44508 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:30:12.655576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520c0aa-6e69-404b-a823-f53a67a44508 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:30:12.657235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6520c0aa-6e69-404b-a823-f53a67a44508 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:30:12.720452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520c0aa-6e69-404b-a823-f53a67a44508 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:30:12.721885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520c0aa-6e69-404b-a823-f53a67a44508 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:30:12.732135 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4309/8684] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:30:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:30:18.258121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=12 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:30:18.708147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113bf1d-0804-4281-b527-4de4e7ee9020 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:30:18.709655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e113bf1d-0804-4281-b527-4de4e7ee9020 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:30:18.769501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520c0aa-6e69-404b-a823-f53a67a44508 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:30:18.771489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6520c0aa-6e69-404b-a823-f53a67a44508 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:30:18.793447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520c0aa-6e69-404b-a823-f53a67a44508 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:30:18.798452 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e99d1-e938-4fa9-911e-ddec873d586d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:30:18.799329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e99d1-e938-4fa9-911e-ddec873d586d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:30:18.799681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e99d1-e938-4fa9-911e-ddec873d586d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:30:18.800022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e99d1-e938-4fa9-911e-ddec873d586d admin admin] PATH_INFO: `/v3/users/eb0bf5fce885439eaa16a0871c2992ac` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:30:18.800136 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113bf1d-0804-4281-b527-4de4e7ee9020 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:30:18.803421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e99d1-e938-4fa9-911e-ddec873d586d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb0bf5fce885439eaa16a0871c2992ac)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:30:18.805766 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e99d1-e938-4fa9-911e-ddec873d586d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:30:18.805918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c126f-d0db-4cb3-aa7c-7f2795f0f437 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:30:18.806710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c126f-d0db-4cb3-aa7c-7f2795f0f437 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:30:18.807002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c126f-d0db-4cb3-aa7c-7f2795f0f437 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:30:18.807284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c126f-d0db-4cb3-aa7c-7f2795f0f437 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:30:18.809777 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-570c126f-d0db-4cb3-aa7c-7f2795f0f437 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:30:18.827873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c126f-d0db-4cb3-aa7c-7f2795f0f437 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:30:18.829646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c126f-d0db-4cb3-aa7c-7f2795f0f437 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:30:18.844786 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4376/8685] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:30:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:30:18.867978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-aa5e99d1-e938-4fa9-911e-ddec873d586d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb0bf5fce885439eaa16a0871c2992ac'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:30:18.868650 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-aa5e99d1-e938-4fa9-911e-ddec873d586d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb0bf5fce885439eaa16a0871c2992ac'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:30:18.869271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-aa5e99d1-e938-4fa9-911e-ddec873d586d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb0bf5fce885439eaa16a0871c2992ac'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:30:18.880300 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-aa5e99d1-e938-4fa9-911e-ddec873d586d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb0bf5fce885439eaa16a0871c2992ac'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:30:18.897203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-aa5e99d1-e938-4fa9-911e-ddec873d586d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb0bf5fce885439eaa16a0871c2992ac'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:30:18.909310 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4310/8686] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:30:18 2022] DELETE /identity/v3/users/eb0bf5fce885439eaa16a0871c2992ac => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:30:19.081941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c126f-d0db-4cb3-aa7c-7f2795f0f437 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:30:19.093436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c126f-d0db-4cb3-aa7c-7f2795f0f437 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:30:19.150373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc3635-38b6-4728-8392-9604ac3c27f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:30:19.150983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc3635-38b6-4728-8392-9604ac3c27f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:30:19.151265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc3635-38b6-4728-8392-9604ac3c27f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:30:19.152084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc3635-38b6-4728-8392-9604ac3c27f2 admin admin] PATH_INFO: `/v3/projects/df0e56b020444dd28af868590b1bccf4` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:30:19.153220 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc3635-38b6-4728-8392-9604ac3c27f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df0e56b020444dd28af868590b1bccf4)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:30:19.155163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc3635-38b6-4728-8392-9604ac3c27f2 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:30:19.200651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-87bc3635-38b6-4728-8392-9604ac3c27f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df0e56b020444dd28af868590b1bccf4', 'request_id': 'req-87bc3635-38b6-4728-8392-9604ac3c27f2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:30:19.206952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-87bc3635-38b6-4728-8392-9604ac3c27f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df0e56b020444dd28af868590b1bccf4', 'request_id': 'req-87bc3635-38b6-4728-8392-9604ac3c27f2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:30:19.212593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-87bc3635-38b6-4728-8392-9604ac3c27f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df0e56b020444dd28af868590b1bccf4', 'request_id': 'req-87bc3635-38b6-4728-8392-9604ac3c27f2'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:30:19.225341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-87bc3635-38b6-4728-8392-9604ac3c27f2 admin admin] The token cache is being invalidate because project df0e56b020444dd28af868590b1bccf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:30:19.225745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-87bc3635-38b6-4728-8392-9604ac3c27f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:30:19.226476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-87bc3635-38b6-4728-8392-9604ac3c27f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:30:19.228045 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4377/8687] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:30:19 2022] DELETE /identity/v3/projects/df0e56b020444dd28af868590b1bccf4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:30:19.232813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e99d1-e938-4fa9-911e-ddec873d586d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:30:19.234336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-aa5e99d1-e938-4fa9-911e-ddec873d586d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:30:19.247811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e99d1-e938-4fa9-911e-ddec873d586d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:30:19.286687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:30:19.307940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74df32a-5d5c-44e6-9855-5e7da01b6176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:30:19.308578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74df32a-5d5c-44e6-9855-5e7da01b6176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:30:19.308861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74df32a-5d5c-44e6-9855-5e7da01b6176 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:30:19.309133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74df32a-5d5c-44e6-9855-5e7da01b6176 admin admin] PATH_INFO: `/v3/users/299e38c0298743ab8096d3c9c70ad9b5` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:30:19.310246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74df32a-5d5c-44e6-9855-5e7da01b6176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=299e38c0298743ab8096d3c9c70ad9b5)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:30:19.312242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74df32a-5d5c-44e6-9855-5e7da01b6176 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:30:19.357035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a74df32a-5d5c-44e6-9855-5e7da01b6176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '299e38c0298743ab8096d3c9c70ad9b5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:30:19.357698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a74df32a-5d5c-44e6-9855-5e7da01b6176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '299e38c0298743ab8096d3c9c70ad9b5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:30:19.358314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a74df32a-5d5c-44e6-9855-5e7da01b6176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '299e38c0298743ab8096d3c9c70ad9b5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:30:19.365535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a74df32a-5d5c-44e6-9855-5e7da01b6176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '299e38c0298743ab8096d3c9c70ad9b5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:30:19.377451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-a74df32a-5d5c-44e6-9855-5e7da01b6176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '299e38c0298743ab8096d3c9c70ad9b5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:30:19.390021 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4311/8688] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:30:19 2022] DELETE /identity/v3/users/299e38c0298743ab8096d3c9c70ad9b5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:30:19.562442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc3635-38b6-4728-8392-9604ac3c27f2 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:30:19.564092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-87bc3635-38b6-4728-8392-9604ac3c27f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:30:19.578310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc3635-38b6-4728-8392-9604ac3c27f2 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:30:19.596461 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:30:19.637878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea1079-9dfd-4517-8bf2-9390bda7af82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:30:19.638579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea1079-9dfd-4517-8bf2-9390bda7af82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:30:19.638818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea1079-9dfd-4517-8bf2-9390bda7af82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:30:19.639091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea1079-9dfd-4517-8bf2-9390bda7af82 admin admin] PATH_INFO: `/v3/projects/a5e5603923f64ece8cb7a80f4e2dac66` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:30:19.641067 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea1079-9dfd-4517-8bf2-9390bda7af82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5e5603923f64ece8cb7a80f4e2dac66)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:30:19.643010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea1079-9dfd-4517-8bf2-9390bda7af82 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:30:19.689023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0ea1079-9dfd-4517-8bf2-9390bda7af82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5e5603923f64ece8cb7a80f4e2dac66', 'request_id': 'req-a0ea1079-9dfd-4517-8bf2-9390bda7af82'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:30:19.695114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0ea1079-9dfd-4517-8bf2-9390bda7af82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5e5603923f64ece8cb7a80f4e2dac66', 'request_id': 'req-a0ea1079-9dfd-4517-8bf2-9390bda7af82'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:30:19.701105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0ea1079-9dfd-4517-8bf2-9390bda7af82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5e5603923f64ece8cb7a80f4e2dac66', 'request_id': 'req-a0ea1079-9dfd-4517-8bf2-9390bda7af82'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:30:19.713044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0ea1079-9dfd-4517-8bf2-9390bda7af82 admin admin] The token cache is being invalidate because project a5e5603923f64ece8cb7a80f4e2dac66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:30:19.713335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0ea1079-9dfd-4517-8bf2-9390bda7af82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:30:19.713959 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-a0ea1079-9dfd-4517-8bf2-9390bda7af82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:30:19.716074 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4378/8689] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:30:19 2022] DELETE /identity/v3/projects/a5e5603923f64ece8cb7a80f4e2dac66 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:30:29.386101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:30:29.710093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:32:06.480918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74df32a-5d5c-44e6-9855-5e7da01b6176 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:32:06.482404 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a74df32a-5d5c-44e6-9855-5e7da01b6176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:32:06.573852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74df32a-5d5c-44e6-9855-5e7da01b6176 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:32:06.633705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138770c5-ed4a-43de-8fa5-791141c5f98d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:32:06.634319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138770c5-ed4a-43de-8fa5-791141c5f98d service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:32:06.634649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138770c5-ed4a-43de-8fa5-791141c5f98d service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:32:06.634925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138770c5-ed4a-43de-8fa5-791141c5f98d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:32:06.636680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-138770c5-ed4a-43de-8fa5-791141c5f98d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:32:06.650706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138770c5-ed4a-43de-8fa5-791141c5f98d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:32:06.652299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138770c5-ed4a-43de-8fa5-791141c5f98d service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:32:06.664640 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4312/8690] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:32:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:32:09.541077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea1079-9dfd-4517-8bf2-9390bda7af82 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:32:09.542507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a0ea1079-9dfd-4517-8bf2-9390bda7af82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:32:09.560957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea1079-9dfd-4517-8bf2-9390bda7af82 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:32:09.566803 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f4c22-d7fd-42f0-adc9-32bad1468c01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:32:09.567405 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f4c22-d7fd-42f0-adc9-32bad1468c01 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:32:09.567689 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f4c22-d7fd-42f0-adc9-32bad1468c01 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:32:09.567962 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f4c22-d7fd-42f0-adc9-32bad1468c01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:32:09.569675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-640f4c22-d7fd-42f0-adc9-32bad1468c01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:32:09.586073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f4c22-d7fd-42f0-adc9-32bad1468c01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:32:09.587754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f4c22-d7fd-42f0-adc9-32bad1468c01 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:32:09.600235 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4379/8691] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:32:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:32:16.657889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=62 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:32:19.593429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:33:01.991875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138770c5-ed4a-43de-8fa5-791141c5f98d service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:01.993926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-138770c5-ed4a-43de-8fa5-791141c5f98d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:02.098946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138770c5-ed4a-43de-8fa5-791141c5f98d service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:02.104960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40573e7-a59b-42a6-9820-e97cbae530fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:02.105584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40573e7-a59b-42a6-9820-e97cbae530fc service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:02.105867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40573e7-a59b-42a6-9820-e97cbae530fc service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:02.106149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40573e7-a59b-42a6-9820-e97cbae530fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:02.108439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f40573e7-a59b-42a6-9820-e97cbae530fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:02.125769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40573e7-a59b-42a6-9820-e97cbae530fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:02.127538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40573e7-a59b-42a6-9820-e97cbae530fc service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:02.146043 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4313/8692] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:33:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:33:04.468704 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f4c22-d7fd-42f0-adc9-32bad1468c01 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:04.470156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-640f4c22-d7fd-42f0-adc9-32bad1468c01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:04.541606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f4c22-d7fd-42f0-adc9-32bad1468c01 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:04.545252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3c5222-c1a3-46f0-ac48-b42f97922719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:04.545855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c5222-c1a3-46f0-ac48-b42f97922719 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:04.546132 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c5222-c1a3-46f0-ac48-b42f97922719 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:04.546404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c5222-c1a3-46f0-ac48-b42f97922719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:04.548163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4e3c5222-c1a3-46f0-ac48-b42f97922719 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:04.611379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3c5222-c1a3-46f0-ac48-b42f97922719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:04.613366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3c5222-c1a3-46f0-ac48-b42f97922719 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:04.631893 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4380/8693] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:33:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:33:10.663639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40573e7-a59b-42a6-9820-e97cbae530fc service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:10.665207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f40573e7-a59b-42a6-9820-e97cbae530fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:10.684625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40573e7-a59b-42a6-9820-e97cbae530fc service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:10.690109 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0bb39-110a-4663-8187-c5186003324d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:10.690859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0bb39-110a-4663-8187-c5186003324d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:10.691198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0bb39-110a-4663-8187-c5186003324d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:10.691507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0bb39-110a-4663-8187-c5186003324d admin admin] PATH_INFO: `/v3/users/1c1fc5de87ed44e7b01229925bf27242` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:10.692665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0bb39-110a-4663-8187-c5186003324d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c1fc5de87ed44e7b01229925bf27242)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:10.694666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0bb39-110a-4663-8187-c5186003324d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:10.746172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5ba0bb39-110a-4663-8187-c5186003324d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c1fc5de87ed44e7b01229925bf27242'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:10.746919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5ba0bb39-110a-4663-8187-c5186003324d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c1fc5de87ed44e7b01229925bf27242'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:10.747523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5ba0bb39-110a-4663-8187-c5186003324d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c1fc5de87ed44e7b01229925bf27242'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:10.755175 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5ba0bb39-110a-4663-8187-c5186003324d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c1fc5de87ed44e7b01229925bf27242'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:10.769181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5ba0bb39-110a-4663-8187-c5186003324d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c1fc5de87ed44e7b01229925bf27242'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:10.782298 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4314/8694] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:33:10 2022] DELETE /identity/v3/users/1c1fc5de87ed44e7b01229925bf27242 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:33:10.957921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3c5222-c1a3-46f0-ac48-b42f97922719 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:10.976598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3c5222-c1a3-46f0-ac48-b42f97922719 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:11.034467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037e185-864e-4789-a00f-2991ccbc0bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:11.035663 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037e185-864e-4789-a00f-2991ccbc0bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:11.035951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037e185-864e-4789-a00f-2991ccbc0bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:11.036227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037e185-864e-4789-a00f-2991ccbc0bdb admin admin] PATH_INFO: `/v3/projects/ff07b6ab40b84bd18a5a44deb9516f00` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:11.037371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037e185-864e-4789-a00f-2991ccbc0bdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff07b6ab40b84bd18a5a44deb9516f00)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:11.039308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037e185-864e-4789-a00f-2991ccbc0bdb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:11.076948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:33:11.088206 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3037e185-864e-4789-a00f-2991ccbc0bdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff07b6ab40b84bd18a5a44deb9516f00', 'request_id': 'req-3037e185-864e-4789-a00f-2991ccbc0bdb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:11.094268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3037e185-864e-4789-a00f-2991ccbc0bdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff07b6ab40b84bd18a5a44deb9516f00', 'request_id': 'req-3037e185-864e-4789-a00f-2991ccbc0bdb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:11.100468 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3037e185-864e-4789-a00f-2991ccbc0bdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff07b6ab40b84bd18a5a44deb9516f00', 'request_id': 'req-3037e185-864e-4789-a00f-2991ccbc0bdb'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:11.112070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3037e185-864e-4789-a00f-2991ccbc0bdb admin admin] The token cache is being invalidate because project ff07b6ab40b84bd18a5a44deb9516f00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:33:11.112343 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3037e185-864e-4789-a00f-2991ccbc0bdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:11.113021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-3037e185-864e-4789-a00f-2991ccbc0bdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:11.114546 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4381/8695] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:33:10 2022] DELETE /identity/v3/projects/ff07b6ab40b84bd18a5a44deb9516f00 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:33:11.118593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0bb39-110a-4663-8187-c5186003324d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:11.120039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5ba0bb39-110a-4663-8187-c5186003324d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:11.133108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0bb39-110a-4663-8187-c5186003324d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:11.184390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:33:11.191429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12cb98-6398-41c9-bfbf-6f32d7883c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:11.192073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12cb98-6398-41c9-bfbf-6f32d7883c46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:11.192516 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12cb98-6398-41c9-bfbf-6f32d7883c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:11.192836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12cb98-6398-41c9-bfbf-6f32d7883c46 admin admin] PATH_INFO: `/v3/users/9a0280cb98a54b9e80de4b286b28e077` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:11.194374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12cb98-6398-41c9-bfbf-6f32d7883c46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a0280cb98a54b9e80de4b286b28e077)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:11.196947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12cb98-6398-41c9-bfbf-6f32d7883c46 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:11.243089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3f12cb98-6398-41c9-bfbf-6f32d7883c46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a0280cb98a54b9e80de4b286b28e077'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:11.243644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3f12cb98-6398-41c9-bfbf-6f32d7883c46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a0280cb98a54b9e80de4b286b28e077'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:11.244252 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3f12cb98-6398-41c9-bfbf-6f32d7883c46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a0280cb98a54b9e80de4b286b28e077'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:11.250919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3f12cb98-6398-41c9-bfbf-6f32d7883c46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a0280cb98a54b9e80de4b286b28e077'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:11.263198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3f12cb98-6398-41c9-bfbf-6f32d7883c46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a0280cb98a54b9e80de4b286b28e077'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:11.275680 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4315/8696] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:33:11 2022] DELETE /identity/v3/users/9a0280cb98a54b9e80de4b286b28e077 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:33:11.445818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037e185-864e-4789-a00f-2991ccbc0bdb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:11.447365 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3037e185-864e-4789-a00f-2991ccbc0bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:11.464912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037e185-864e-4789-a00f-2991ccbc0bdb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:11.521792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142f681-fecd-4ad5-a679-0e0d7e249371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:11.522348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142f681-fecd-4ad5-a679-0e0d7e249371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:11.522659 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142f681-fecd-4ad5-a679-0e0d7e249371 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:11.522938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142f681-fecd-4ad5-a679-0e0d7e249371 admin admin] PATH_INFO: `/v3/projects/5bc2cce0aba64a5792e67bd94a15453c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:11.524047 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142f681-fecd-4ad5-a679-0e0d7e249371 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bc2cce0aba64a5792e67bd94a15453c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:11.525946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142f681-fecd-4ad5-a679-0e0d7e249371 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:11.571672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9142f681-fecd-4ad5-a679-0e0d7e249371 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bc2cce0aba64a5792e67bd94a15453c', 'request_id': 'req-9142f681-fecd-4ad5-a679-0e0d7e249371'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:11.577685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9142f681-fecd-4ad5-a679-0e0d7e249371 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bc2cce0aba64a5792e67bd94a15453c', 'request_id': 'req-9142f681-fecd-4ad5-a679-0e0d7e249371'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:11.584226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9142f681-fecd-4ad5-a679-0e0d7e249371 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bc2cce0aba64a5792e67bd94a15453c', 'request_id': 'req-9142f681-fecd-4ad5-a679-0e0d7e249371'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:11.597283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9142f681-fecd-4ad5-a679-0e0d7e249371 admin admin] The token cache is being invalidate because project 5bc2cce0aba64a5792e67bd94a15453c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:33:11.597598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9142f681-fecd-4ad5-a679-0e0d7e249371 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:11.598227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9142f681-fecd-4ad5-a679-0e0d7e249371 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:11.599767 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4382/8697] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:33:11 2022] DELETE /identity/v3/projects/5bc2cce0aba64a5792e67bd94a15453c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:33:21.272305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=27,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:33:21.406519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbb802-39ae-44ee-8a00-7bff5913d8ba None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:21.406729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbb802-39ae-44ee-8a00-7bff5913d8ba None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:21.407006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbb802-39ae-44ee-8a00-7bff5913d8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:21.427578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-01bbb802-39ae-44ee-8a00-7bff5913d8ba None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:33:21.429128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-01bbb802-39ae-44ee-8a00-7bff5913d8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:21.488607 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4316/8698] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:33:21.497665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f749933-8117-4a09-988b-9932c4dac25c None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:21.498167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f749933-8117-4a09-988b-9932c4dac25c None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:21.498268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f749933-8117-4a09-988b-9932c4dac25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:21.518111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2f749933-8117-4a09-988b-9932c4dac25c None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:33:21.519346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2f749933-8117-4a09-988b-9932c4dac25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:21.527615 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4383/8699] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:33:21.532258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbb802-39ae-44ee-8a00-7bff5913d8ba None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:21.542611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbb802-39ae-44ee-8a00-7bff5913d8ba None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:21.547254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c92fc7-0de3-47fb-bb35-db88af384037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:21.547852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c92fc7-0de3-47fb-bb35-db88af384037 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:21.548132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c92fc7-0de3-47fb-bb35-db88af384037 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:21.548403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c92fc7-0de3-47fb-bb35-db88af384037 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:21.549040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c92fc7-0de3-47fb-bb35-db88af384037 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:33:21.549343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c92fc7-0de3-47fb-bb35-db88af384037 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:21.551221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c92fc7-0de3-47fb-bb35-db88af384037 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:21.562686 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4317/8700] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:33:21.577792 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f749933-8117-4a09-988b-9932c4dac25c None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:21.587778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f749933-8117-4a09-988b-9932c4dac25c None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:21.592505 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36708979-0910-4a0a-9868-cae3c4b7c9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:21.593102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36708979-0910-4a0a-9868-cae3c4b7c9f6 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:21.593413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36708979-0910-4a0a-9868-cae3c4b7c9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:21.593718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36708979-0910-4a0a-9868-cae3c4b7c9f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:21.594345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36708979-0910-4a0a-9868-cae3c4b7c9f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:21.596267 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36708979-0910-4a0a-9868-cae3c4b7c9f6 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:21.632717 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4384/8701] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:33:21 2022] POST /identity/v3/projects => generated 391 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:33:21.637018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c92fc7-0de3-47fb-bb35-db88af384037 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:21.647626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c92fc7-0de3-47fb-bb35-db88af384037 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:21.703880 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d917e4e-fc5c-4b9f-ad32-c35432418f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:21.704489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d917e4e-fc5c-4b9f-ad32-c35432418f04 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:21.704772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d917e4e-fc5c-4b9f-ad32-c35432418f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:21.705062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d917e4e-fc5c-4b9f-ad32-c35432418f04 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:21.705666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d917e4e-fc5c-4b9f-ad32-c35432418f04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:21.708137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d917e4e-fc5c-4b9f-ad32-c35432418f04 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:21.729742 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4318/8702] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:33:21 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:33:21.733642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36708979-0910-4a0a-9868-cae3c4b7c9f6 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:21.743932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36708979-0910-4a0a-9868-cae3c4b7c9f6 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:21.748998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510f5e9-c9fa-49a7-b493-403098c6f51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:21.749597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510f5e9-c9fa-49a7-b493-403098c6f51f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:21.749881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510f5e9-c9fa-49a7-b493-403098c6f51f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:21.750157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510f5e9-c9fa-49a7-b493-403098c6f51f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:21.750869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510f5e9-c9fa-49a7-b493-403098c6f51f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:21.752463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510f5e9-c9fa-49a7-b493-403098c6f51f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:21.763227 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4385/8703] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:33:21.767091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d917e4e-fc5c-4b9f-ad32-c35432418f04 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:21.777536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d917e4e-fc5c-4b9f-ad32-c35432418f04 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:21.782089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4e81c-0b5a-4df7-aa1b-37044f436992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:21.782698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4e81c-0b5a-4df7-aa1b-37044f436992 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:21.782978 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4e81c-0b5a-4df7-aa1b-37044f436992 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:21.783253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4e81c-0b5a-4df7-aa1b-37044f436992 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:21.783889 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4e81c-0b5a-4df7-aa1b-37044f436992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:21.785507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4e81c-0b5a-4df7-aa1b-37044f436992 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:21.795998 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4319/8704] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:33:21.800169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510f5e9-c9fa-49a7-b493-403098c6f51f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:21.806825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:33:21.810718 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510f5e9-c9fa-49a7-b493-403098c6f51f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:21.816388 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c3670-c2a3-409e-b0b0-952cee8748d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:21.817675 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c3670-c2a3-409e-b0b0-952cee8748d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:21.818128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c3670-c2a3-409e-b0b0-952cee8748d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:21.818443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c3670-c2a3-409e-b0b0-952cee8748d3 admin admin] PATH_INFO: `/v3/projects/628d724deda14e28b854c932489f45a4/users/da3ac7c63d934f63a3cdda9be0310f24/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:21.833798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c3670-c2a3-409e-b0b0-952cee8748d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=628d724deda14e28b854c932489f45a4, user_id=da3ac7c63d934f63a3cdda9be0310f24, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:21.836476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c3670-c2a3-409e-b0b0-952cee8748d3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:21.849522 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4386/8705] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:33:21 2022] PUT /identity/v3/projects/628d724deda14e28b854c932489f45a4/users/da3ac7c63d934f63a3cdda9be0310f24/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:33:21.862668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed506bd-f345-40c0-9849-7fd240d89aea None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:21.862955 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed506bd-f345-40c0-9849-7fd240d89aea None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:21.863230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed506bd-f345-40c0-9849-7fd240d89aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:21.892540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-eed506bd-f345-40c0-9849-7fd240d89aea None None] MFA Rules not processed for user `da3ac7c63d934f63a3cdda9be0310f24`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:33:21.893761 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-eed506bd-f345-40c0-9849-7fd240d89aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:21.948894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:33:21.959422 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4320/8706] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:33:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:33:21.963843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c3670-c2a3-409e-b0b0-952cee8748d3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:22.026558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c3670-c2a3-409e-b0b0-952cee8748d3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:22.031374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be334155-4ec8-49d7-8cb0-fba8967bd771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:22.031982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be334155-4ec8-49d7-8cb0-fba8967bd771 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:22.032266 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be334155-4ec8-49d7-8cb0-fba8967bd771 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:22.032539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be334155-4ec8-49d7-8cb0-fba8967bd771 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:22.033179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be334155-4ec8-49d7-8cb0-fba8967bd771 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:22.035124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be334155-4ec8-49d7-8cb0-fba8967bd771 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:22.064501 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4387/8707] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:33:21 2022] POST /identity/v3/projects => generated 393 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:33:22.065720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed506bd-f345-40c0-9849-7fd240d89aea None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:22.076877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed506bd-f345-40c0-9849-7fd240d89aea None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:22.134112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2e341-d249-4c1e-a5be-1bd9909f6cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:22.134771 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2e341-d249-4c1e-a5be-1bd9909f6cb8 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:22.135055 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2e341-d249-4c1e-a5be-1bd9909f6cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:22.135334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2e341-d249-4c1e-a5be-1bd9909f6cb8 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:22.135957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab2e341-d249-4c1e-a5be-1bd9909f6cb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:22.137881 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab2e341-d249-4c1e-a5be-1bd9909f6cb8 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:22.159100 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4321/8708] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:33:22 2022] POST /identity/v3/users => generated 303 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:33:22.163116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be334155-4ec8-49d7-8cb0-fba8967bd771 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:22.173523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be334155-4ec8-49d7-8cb0-fba8967bd771 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:22.178978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2102bf-6129-447e-8b84-7ecca612f063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:22.179576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2102bf-6129-447e-8b84-7ecca612f063 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:22.179863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2102bf-6129-447e-8b84-7ecca612f063 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:22.180163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2102bf-6129-447e-8b84-7ecca612f063 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:22.180807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2102bf-6129-447e-8b84-7ecca612f063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:22.182408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2102bf-6129-447e-8b84-7ecca612f063 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:22.193943 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4388/8709] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:33:22.198065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2e341-d249-4c1e-a5be-1bd9909f6cb8 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:22.207975 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2e341-d249-4c1e-a5be-1bd9909f6cb8 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:22.212623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f6673-e3dd-444f-a59c-2c54ab26af41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:22.213198 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f6673-e3dd-444f-a59c-2c54ab26af41 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:22.213474 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f6673-e3dd-444f-a59c-2c54ab26af41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:22.213742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f6673-e3dd-444f-a59c-2c54ab26af41 admin admin] PATH_INFO: `/v3/projects/a710d82d35e444218b503619776ae9d7/users/471c3f34da7d434888f1d7a29c8c2824/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:22.228588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f6673-e3dd-444f-a59c-2c54ab26af41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a710d82d35e444218b503619776ae9d7, user_id=471c3f34da7d434888f1d7a29c8c2824, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:22.230957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f6673-e3dd-444f-a59c-2c54ab26af41 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:22.244412 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4322/8710] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:33:22 2022] PUT /identity/v3/projects/a710d82d35e444218b503619776ae9d7/users/471c3f34da7d434888f1d7a29c8c2824/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:33:22.248315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2102bf-6129-447e-8b84-7ecca612f063 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:22.258833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2102bf-6129-447e-8b84-7ecca612f063 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:22.314566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10057646-2eef-4146-a19b-a127d1a39c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:22.315167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10057646-2eef-4146-a19b-a127d1a39c72 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:22.315446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10057646-2eef-4146-a19b-a127d1a39c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:22.315720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10057646-2eef-4146-a19b-a127d1a39c72 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:22.316352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10057646-2eef-4146-a19b-a127d1a39c72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:22.318555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10057646-2eef-4146-a19b-a127d1a39c72 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:22.328065 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4389/8711] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:33:22.332184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f6673-e3dd-444f-a59c-2c54ab26af41 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:22.342369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f6673-e3dd-444f-a59c-2c54ab26af41 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:22.346977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2476346-1fc7-4fb2-af8f-9ae1f6d1ceaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:22.347610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2476346-1fc7-4fb2-af8f-9ae1f6d1ceaa admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:22.347887 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2476346-1fc7-4fb2-af8f-9ae1f6d1ceaa admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:22.348158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2476346-1fc7-4fb2-af8f-9ae1f6d1ceaa admin admin] PATH_INFO: `/v3/projects/a710d82d35e444218b503619776ae9d7/users/471c3f34da7d434888f1d7a29c8c2824/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:22.350171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2476346-1fc7-4fb2-af8f-9ae1f6d1ceaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a710d82d35e444218b503619776ae9d7, user_id=471c3f34da7d434888f1d7a29c8c2824, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:22.352802 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2476346-1fc7-4fb2-af8f-9ae1f6d1ceaa admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:22.362762 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4323/8712] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:33:22 2022] PUT /identity/v3/projects/a710d82d35e444218b503619776ae9d7/users/471c3f34da7d434888f1d7a29c8c2824/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:33:22.372965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4972e58-060b-4da8-84c8-c29d5d5f3b26 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:22.373252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4972e58-060b-4da8-84c8-c29d5d5f3b26 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:22.373550 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4972e58-060b-4da8-84c8-c29d5d5f3b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:22.401855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c4972e58-060b-4da8-84c8-c29d5d5f3b26 None None] MFA Rules not processed for user `471c3f34da7d434888f1d7a29c8c2824`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:33:22.403082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c4972e58-060b-4da8-84c8-c29d5d5f3b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:22.447690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:33:22.476983 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4390/8713] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:33:22 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:33:22.490449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d226d-5773-4fd3-a498-84c661ea2914 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:22.490762 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d226d-5773-4fd3-a498-84c661ea2914 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:22.491040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d226d-5773-4fd3-a498-84c661ea2914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:22.509867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-044d226d-5773-4fd3-a498-84c661ea2914 None None] MFA Rules not processed for user `da3ac7c63d934f63a3cdda9be0310f24`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:33:22.511107 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-044d226d-5773-4fd3-a498-84c661ea2914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:22.565558 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4324/8714] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:33:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:33:22.579541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4972e58-060b-4da8-84c8-c29d5d5f3b26 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:22.581015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c4972e58-060b-4da8-84c8-c29d5d5f3b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:22.666320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4972e58-060b-4da8-84c8-c29d5d5f3b26 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:22.673367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d31858-9d69-48b2-aabf-4581d4cefb58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:22.673983 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d31858-9d69-48b2-aabf-4581d4cefb58 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:22.674270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d31858-9d69-48b2-aabf-4581d4cefb58 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:22.674574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d31858-9d69-48b2-aabf-4581d4cefb58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:22.683665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d31858-9d69-48b2-aabf-4581d4cefb58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:22.685265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d31858-9d69-48b2-aabf-4581d4cefb58 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:22.694763 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4391/8715] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:33:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:33:23.075689 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d226d-5773-4fd3-a498-84c661ea2914 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:23.077195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-044d226d-5773-4fd3-a498-84c661ea2914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:23.139872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:33:23.161574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d226d-5773-4fd3-a498-84c661ea2914 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:23.166548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bf174-6666-4704-b17d-638bb9c29602 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:23.167163 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bf174-6666-4704-b17d-638bb9c29602 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:23.167445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bf174-6666-4704-b17d-638bb9c29602 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:23.167723 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bf174-6666-4704-b17d-638bb9c29602 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:23.169421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a87bf174-6666-4704-b17d-638bb9c29602 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:23.183424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bf174-6666-4704-b17d-638bb9c29602 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:23.185005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bf174-6666-4704-b17d-638bb9c29602 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:23.197226 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4325/8716] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:33:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:33:30.974079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d31858-9d69-48b2-aabf-4581d4cefb58 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:30.975576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f4d31858-9d69-48b2-aabf-4581d4cefb58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:30.995736 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d31858-9d69-48b2-aabf-4581d4cefb58 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:31.001989 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331efff9-b128-4e83-8690-b677d3337bcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:31.002658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331efff9-b128-4e83-8690-b677d3337bcd service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:31.002940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331efff9-b128-4e83-8690-b677d3337bcd service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:31.003238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331efff9-b128-4e83-8690-b677d3337bcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:31.004975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-331efff9-b128-4e83-8690-b677d3337bcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:31.019755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331efff9-b128-4e83-8690-b677d3337bcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:31.021865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331efff9-b128-4e83-8690-b677d3337bcd service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:31.035353 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4392/8717] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:33:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:33:32.523592 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bf174-6666-4704-b17d-638bb9c29602 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:32.540082 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bf174-6666-4704-b17d-638bb9c29602 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:32.545856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2b342-ff99-46be-a442-73f56f5c1c9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:32.546473 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2b342-ff99-46be-a442-73f56f5c1c9f service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:32.546787 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2b342-ff99-46be-a442-73f56f5c1c9f service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:32.547065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2b342-ff99-46be-a442-73f56f5c1c9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:32.556203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b2b342-ff99-46be-a442-73f56f5c1c9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:32.557815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b2b342-ff99-46be-a442-73f56f5c1c9f service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:32.567472 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4326/8718] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:33:32 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:33:32.584876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a064ee-8aea-427a-9164-400a346804e3 None None] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:32.585502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a064ee-8aea-427a-9164-400a346804e3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:32.585502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a064ee-8aea-427a-9164-400a346804e3 None None] PATH_INFO: `/` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:32.586997 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4393/8719] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 21:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 21:33:32.592172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2b342-ff99-46be-a442-73f56f5c1c9f service nova] Authenticating service token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 21:33:32.606938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2b342-ff99-46be-a442-73f56f5c1c9f service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:32.615784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2b342-ff99-46be-a442-73f56f5c1c9f service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:32.620825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a0989-4f3b-4d7b-936f-26b04290dac5 tempest-ServersAdminNegativeTestJSON-1593677862 tempest-ServersAdminNegativeTestJSON-1593677862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471c3f34da7d434888f1d7a29c8c2824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a710d82d35e444218b503619776ae9d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:32.621459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a0989-4f3b-4d7b-936f-26b04290dac5 tempest-ServersAdminNegativeTestJSON-1593677862 tempest-ServersAdminNegativeTestJSON-1593677862-project-admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:32.621744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a0989-4f3b-4d7b-936f-26b04290dac5 tempest-ServersAdminNegativeTestJSON-1593677862 tempest-ServersAdminNegativeTestJSON-1593677862-project-admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:32.622029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a0989-4f3b-4d7b-936f-26b04290dac5 tempest-ServersAdminNegativeTestJSON-1593677862 tempest-ServersAdminNegativeTestJSON-1593677862-project-admin] PATH_INFO: `/v3/projects/a710d82d35e444218b503619776ae9d7` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:32.623638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1a0989-4f3b-4d7b-936f-26b04290dac5 tempest-ServersAdminNegativeTestJSON-1593677862 tempest-ServersAdminNegativeTestJSON-1593677862-project-admin] RBAC: Authorizing `identity:get_project(project_id=a710d82d35e444218b503619776ae9d7)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:32.625414 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1a0989-4f3b-4d7b-936f-26b04290dac5 tempest-ServersAdminNegativeTestJSON-1593677862 tempest-ServersAdminNegativeTestJSON-1593677862-project-admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:32.627119 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4327/8720] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 21:33:32 2022] GET /identity/v3/projects/a710d82d35e444218b503619776ae9d7 => generated 393 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:33:32.793799 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a064ee-8aea-427a-9164-400a346804e3 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:32.795509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e3a064ee-8aea-427a-9164-400a346804e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:32.856556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a064ee-8aea-427a-9164-400a346804e3 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:32.860302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89492d-0876-4794-8334-5d896b8d8946 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:32.860942 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89492d-0876-4794-8334-5d896b8d8946 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:32.861219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89492d-0876-4794-8334-5d896b8d8946 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:32.861494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89492d-0876-4794-8334-5d896b8d8946 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:32.863243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cf89492d-0876-4794-8334-5d896b8d8946 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:32.875240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89492d-0876-4794-8334-5d896b8d8946 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:32.876647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89492d-0876-4794-8334-5d896b8d8946 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:32.887668 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4394/8721] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:33:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:33:33.045498 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bf702-91db-4184-84ca-31b7a2e88afa None None] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:33.045611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bf702-91db-4184-84ca-31b7a2e88afa None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:33.045872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bf702-91db-4184-84ca-31b7a2e88afa None None] PATH_INFO: `/` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:33.047340 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4328/8722] 10.222.0.17 () {54 vars in 912 bytes} [Sat Oct 1 21:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 01 21:33:33.052649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89492d-0876-4794-8334-5d896b8d8946 service neutron] Authenticating service token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Oct 01 21:33:33.068973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89492d-0876-4794-8334-5d896b8d8946 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:33.078929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89492d-0876-4794-8334-5d896b8d8946 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:33.083545 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7354ed-e27a-40af-bc2b-ca24f08ce6e0 tempest-ServersAdminNegativeTestJSON-1593677862 tempest-ServersAdminNegativeTestJSON-1593677862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471c3f34da7d434888f1d7a29c8c2824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a710d82d35e444218b503619776ae9d7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:33.084167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7354ed-e27a-40af-bc2b-ca24f08ce6e0 tempest-ServersAdminNegativeTestJSON-1593677862 tempest-ServersAdminNegativeTestJSON-1593677862-project-admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:33.084453 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7354ed-e27a-40af-bc2b-ca24f08ce6e0 tempest-ServersAdminNegativeTestJSON-1593677862 tempest-ServersAdminNegativeTestJSON-1593677862-project-admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:33.084732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7354ed-e27a-40af-bc2b-ca24f08ce6e0 tempest-ServersAdminNegativeTestJSON-1593677862 tempest-ServersAdminNegativeTestJSON-1593677862-project-admin] PATH_INFO: `/v3/projects/a710d82d35e444218b503619776ae9d7` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:33.085998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7354ed-e27a-40af-bc2b-ca24f08ce6e0 tempest-ServersAdminNegativeTestJSON-1593677862 tempest-ServersAdminNegativeTestJSON-1593677862-project-admin] RBAC: Authorizing `identity:get_project(project_id=a710d82d35e444218b503619776ae9d7)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:33.087809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7354ed-e27a-40af-bc2b-ca24f08ce6e0 tempest-ServersAdminNegativeTestJSON-1593677862 tempest-ServersAdminNegativeTestJSON-1593677862-project-admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:33.089620 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4395/8723] 10.222.0.17 () {58 vars in 1443 bytes} [Sat Oct 1 21:33:33 2022] GET /identity/v3/projects/a710d82d35e444218b503619776ae9d7 => generated 393 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:33:33.244254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bf702-91db-4184-84ca-31b7a2e88afa None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:33.245946 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-337bf702-91db-4184-84ca-31b7a2e88afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:33.262192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bf702-91db-4184-84ca-31b7a2e88afa None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:33.265684 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7655c-79e8-4458-acfc-b57b2931afa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:33.266276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7655c-79e8-4458-acfc-b57b2931afa3 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:33.266593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7655c-79e8-4458-acfc-b57b2931afa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:33.266866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7655c-79e8-4458-acfc-b57b2931afa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:33.268509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cbb7655c-79e8-4458-acfc-b57b2931afa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:33.280290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7655c-79e8-4458-acfc-b57b2931afa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:33.281760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7655c-79e8-4458-acfc-b57b2931afa3 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:33.292189 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4329/8724] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:33:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:33:35.981216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7354ed-e27a-40af-bc2b-ca24f08ce6e0 tempest-ServersAdminNegativeTestJSON-1593677862 tempest-ServersAdminNegativeTestJSON-1593677862-project-admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:36.016251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=100 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:33:36.044156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7354ed-e27a-40af-bc2b-ca24f08ce6e0 tempest-ServersAdminNegativeTestJSON-1593677862 tempest-ServersAdminNegativeTestJSON-1593677862-project-admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:36.049197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a852f6-00ea-4981-ab93-d41d1da86eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:36.049834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a852f6-00ea-4981-ab93-d41d1da86eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:36.050109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a852f6-00ea-4981-ab93-d41d1da86eea admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:36.050381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a852f6-00ea-4981-ab93-d41d1da86eea admin admin] PATH_INFO: `/v3/users/da3ac7c63d934f63a3cdda9be0310f24` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:36.052293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a852f6-00ea-4981-ab93-d41d1da86eea admin admin] RBAC: Authorizing `identity:delete_user(user_id=da3ac7c63d934f63a3cdda9be0310f24)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:36.054204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a852f6-00ea-4981-ab93-d41d1da86eea admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:36.102259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-10a852f6-00ea-4981-ab93-d41d1da86eea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da3ac7c63d934f63a3cdda9be0310f24'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:36.109418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-10a852f6-00ea-4981-ab93-d41d1da86eea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da3ac7c63d934f63a3cdda9be0310f24'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:36.121866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-10a852f6-00ea-4981-ab93-d41d1da86eea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da3ac7c63d934f63a3cdda9be0310f24'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:36.122552 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-10a852f6-00ea-4981-ab93-d41d1da86eea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da3ac7c63d934f63a3cdda9be0310f24'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:36.130222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-10a852f6-00ea-4981-ab93-d41d1da86eea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da3ac7c63d934f63a3cdda9be0310f24'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:36.136165 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4396/8725] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:33:35 2022] DELETE /identity/v3/users/da3ac7c63d934f63a3cdda9be0310f24 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:33:36.140737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51add2d1-a8ed-4bd6-9a0c-7f31191a1512 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:36.141195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51add2d1-a8ed-4bd6-9a0c-7f31191a1512 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:36.141308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51add2d1-a8ed-4bd6-9a0c-7f31191a1512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:36.159942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-51add2d1-a8ed-4bd6-9a0c-7f31191a1512 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:33:36.161194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-51add2d1-a8ed-4bd6-9a0c-7f31191a1512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:36.219457 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4330/8726] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:33:36.229196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a852f6-00ea-4981-ab93-d41d1da86eea admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:36.230540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-10a852f6-00ea-4981-ab93-d41d1da86eea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:36.290263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a852f6-00ea-4981-ab93-d41d1da86eea admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:36.293929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab974196-54ac-4393-ba3d-77189feed06a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:36.294581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab974196-54ac-4393-ba3d-77189feed06a service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:36.294863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab974196-54ac-4393-ba3d-77189feed06a service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:36.295134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab974196-54ac-4393-ba3d-77189feed06a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:36.305064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab974196-54ac-4393-ba3d-77189feed06a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:36.306762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab974196-54ac-4393-ba3d-77189feed06a service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:36.316071 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4397/8727] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:33:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:33:36.484643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51add2d1-a8ed-4bd6-9a0c-7f31191a1512 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:36.486207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-51add2d1-a8ed-4bd6-9a0c-7f31191a1512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:36.498957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51add2d1-a8ed-4bd6-9a0c-7f31191a1512 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:36.504131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09815e-58cd-4210-8201-6f97827c38e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:36.504805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09815e-58cd-4210-8201-6f97827c38e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:36.505083 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09815e-58cd-4210-8201-6f97827c38e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:36.505360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09815e-58cd-4210-8201-6f97827c38e1 admin admin] PATH_INFO: `/v3/projects/628d724deda14e28b854c932489f45a4` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:36.506634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce09815e-58cd-4210-8201-6f97827c38e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=628d724deda14e28b854c932489f45a4)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:36.508555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce09815e-58cd-4210-8201-6f97827c38e1 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:36.557205 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ce09815e-58cd-4210-8201-6f97827c38e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '628d724deda14e28b854c932489f45a4', 'request_id': 'req-ce09815e-58cd-4210-8201-6f97827c38e1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:36.564244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ce09815e-58cd-4210-8201-6f97827c38e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '628d724deda14e28b854c932489f45a4', 'request_id': 'req-ce09815e-58cd-4210-8201-6f97827c38e1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:36.570076 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ce09815e-58cd-4210-8201-6f97827c38e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '628d724deda14e28b854c932489f45a4', 'request_id': 'req-ce09815e-58cd-4210-8201-6f97827c38e1'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:36.580888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ce09815e-58cd-4210-8201-6f97827c38e1 admin admin] The token cache is being invalidate because project 628d724deda14e28b854c932489f45a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:33:36.581183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ce09815e-58cd-4210-8201-6f97827c38e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:36.581815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ce09815e-58cd-4210-8201-6f97827c38e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:36.583385 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4331/8728] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:33:36 2022] DELETE /identity/v3/projects/628d724deda14e28b854c932489f45a4 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:33:36.587711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab974196-54ac-4393-ba3d-77189feed06a service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:36.597721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab974196-54ac-4393-ba3d-77189feed06a service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:36.647895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:33:36.656068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0c2ab-79d1-48e5-8757-71418b6dedb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:36.656894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0c2ab-79d1-48e5-8757-71418b6dedb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:36.657320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0c2ab-79d1-48e5-8757-71418b6dedb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:36.657786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0c2ab-79d1-48e5-8757-71418b6dedb7 admin admin] PATH_INFO: `/v3/users/471c3f34da7d434888f1d7a29c8c2824` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:36.658950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0c2ab-79d1-48e5-8757-71418b6dedb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=471c3f34da7d434888f1d7a29c8c2824)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:36.661450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0c2ab-79d1-48e5-8757-71418b6dedb7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:36.706346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-79e0c2ab-79d1-48e5-8757-71418b6dedb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '471c3f34da7d434888f1d7a29c8c2824'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:36.714012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-79e0c2ab-79d1-48e5-8757-71418b6dedb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '471c3f34da7d434888f1d7a29c8c2824'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:36.726305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-79e0c2ab-79d1-48e5-8757-71418b6dedb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '471c3f34da7d434888f1d7a29c8c2824'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:36.727026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-79e0c2ab-79d1-48e5-8757-71418b6dedb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '471c3f34da7d434888f1d7a29c8c2824'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:36.737096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-79e0c2ab-79d1-48e5-8757-71418b6dedb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '471c3f34da7d434888f1d7a29c8c2824'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:36.743236 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4398/8729] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:33:36 2022] DELETE /identity/v3/users/471c3f34da7d434888f1d7a29c8c2824 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:33:37.005868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09815e-58cd-4210-8201-6f97827c38e1 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:37.008093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ce09815e-58cd-4210-8201-6f97827c38e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:37.023580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09815e-58cd-4210-8201-6f97827c38e1 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:37.054559 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:33:37.085059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545835ab-3d20-4859-a6ac-88ebb58b7637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:37.085685 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545835ab-3d20-4859-a6ac-88ebb58b7637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:37.085965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545835ab-3d20-4859-a6ac-88ebb58b7637 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:37.086238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545835ab-3d20-4859-a6ac-88ebb58b7637 admin admin] PATH_INFO: `/v3/projects/a710d82d35e444218b503619776ae9d7` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:37.087425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545835ab-3d20-4859-a6ac-88ebb58b7637 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a710d82d35e444218b503619776ae9d7)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:37.089356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545835ab-3d20-4859-a6ac-88ebb58b7637 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:37.135260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-545835ab-3d20-4859-a6ac-88ebb58b7637 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a710d82d35e444218b503619776ae9d7', 'request_id': 'req-545835ab-3d20-4859-a6ac-88ebb58b7637'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:37.142330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-545835ab-3d20-4859-a6ac-88ebb58b7637 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a710d82d35e444218b503619776ae9d7', 'request_id': 'req-545835ab-3d20-4859-a6ac-88ebb58b7637'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:37.148090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-545835ab-3d20-4859-a6ac-88ebb58b7637 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a710d82d35e444218b503619776ae9d7', 'request_id': 'req-545835ab-3d20-4859-a6ac-88ebb58b7637'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:37.159050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-545835ab-3d20-4859-a6ac-88ebb58b7637 admin admin] The token cache is being invalidate because project a710d82d35e444218b503619776ae9d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:33:37.159369 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-545835ab-3d20-4859-a6ac-88ebb58b7637 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:37.160005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-545835ab-3d20-4859-a6ac-88ebb58b7637 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:33:37.161995 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4332/8730] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:33:37 2022] DELETE /identity/v3/projects/a710d82d35e444218b503619776ae9d7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:33:37.167916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443972f-e441-404b-b281-d84b69cf2f58 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:37.168500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443972f-e441-404b-b281-d84b69cf2f58 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:37.168604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443972f-e441-404b-b281-d84b69cf2f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:37.189402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e443972f-e441-404b-b281-d84b69cf2f58 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:33:37.190646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e443972f-e441-404b-b281-d84b69cf2f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:37.251499 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4399/8731] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:33:37.259666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705ec32-072e-407b-9b19-f7e94912ed83 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:37.259956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705ec32-072e-407b-9b19-f7e94912ed83 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:37.260235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705ec32-072e-407b-9b19-f7e94912ed83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:37.280392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-2705ec32-072e-407b-9b19-f7e94912ed83 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:33:37.281595 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2705ec32-072e-407b-9b19-f7e94912ed83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:37.289929 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4333/8732] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:33:37.293999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443972f-e441-404b-b281-d84b69cf2f58 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:37.304636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443972f-e441-404b-b281-d84b69cf2f58 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:37.309205 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b2f65-6095-486f-9996-825d10eb45ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:37.309796 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b2f65-6095-486f-9996-825d10eb45ff admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:37.310105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b2f65-6095-486f-9996-825d10eb45ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:37.310376 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b2f65-6095-486f-9996-825d10eb45ff admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:37.311045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b2f65-6095-486f-9996-825d10eb45ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:33:37.311349 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b2f65-6095-486f-9996-825d10eb45ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:37.312924 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b2f65-6095-486f-9996-825d10eb45ff admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:37.324737 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4400/8733] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:33:37.328912 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705ec32-072e-407b-9b19-f7e94912ed83 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:37.338439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705ec32-072e-407b-9b19-f7e94912ed83 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:37.343128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb3ee3-6663-48ad-80b2-71c5b8e292b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:37.343722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb3ee3-6663-48ad-80b2-71c5b8e292b3 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:37.344002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb3ee3-6663-48ad-80b2-71c5b8e292b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:37.344277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb3ee3-6663-48ad-80b2-71c5b8e292b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:37.344921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb3ee3-6663-48ad-80b2-71c5b8e292b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:37.346847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb3ee3-6663-48ad-80b2-71c5b8e292b3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:37.373938 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4334/8734] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:33:37 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:33:37.378173 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b2f65-6095-486f-9996-825d10eb45ff admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:37.388296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b2f65-6095-486f-9996-825d10eb45ff admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:37.445293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57c48c-17bc-405b-bfc4-4813c2fbcfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:37.445899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57c48c-17bc-405b-bfc4-4813c2fbcfa7 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:37.446186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57c48c-17bc-405b-bfc4-4813c2fbcfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:37.446464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57c48c-17bc-405b-bfc4-4813c2fbcfa7 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:37.447119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57c48c-17bc-405b-bfc4-4813c2fbcfa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:37.449033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57c48c-17bc-405b-bfc4-4813c2fbcfa7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:37.458386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:33:37.475263 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4401/8735] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:33:37 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:33:37.476372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb3ee3-6663-48ad-80b2-71c5b8e292b3 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:37.486545 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb3ee3-6663-48ad-80b2-71c5b8e292b3 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:37.491409 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7a103-93a0-4768-8677-f81346234dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:37.492059 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7a103-93a0-4768-8677-f81346234dbc admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:37.492390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7a103-93a0-4768-8677-f81346234dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:37.492663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7a103-93a0-4768-8677-f81346234dbc admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:37.493329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7a103-93a0-4768-8677-f81346234dbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:37.495067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7a103-93a0-4768-8677-f81346234dbc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:37.505796 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4335/8736] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:33:37.509828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57c48c-17bc-405b-bfc4-4813c2fbcfa7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:37.520679 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57c48c-17bc-405b-bfc4-4813c2fbcfa7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:37.525466 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca480c3-5f9e-4849-a2e7-bfd2c53c022a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:37.526061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca480c3-5f9e-4849-a2e7-bfd2c53c022a admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:37.526348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca480c3-5f9e-4849-a2e7-bfd2c53c022a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:37.526666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca480c3-5f9e-4849-a2e7-bfd2c53c022a admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:37.527304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca480c3-5f9e-4849-a2e7-bfd2c53c022a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:37.528893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca480c3-5f9e-4849-a2e7-bfd2c53c022a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:37.539474 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4402/8737] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:33:37.543402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7a103-93a0-4768-8677-f81346234dbc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:37.553426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7a103-93a0-4768-8677-f81346234dbc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:37.557999 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513b1ea-8687-466d-9b72-01e8189ff477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:37.558672 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513b1ea-8687-466d-9b72-01e8189ff477 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:37.558952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513b1ea-8687-466d-9b72-01e8189ff477 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:37.559225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513b1ea-8687-466d-9b72-01e8189ff477 admin admin] PATH_INFO: `/v3/projects/37c9b3e4a1e441e1a92a271f9aa1811e/users/d4671ff245d94e54ac1bf390edd815e5/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:37.573227 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513b1ea-8687-466d-9b72-01e8189ff477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37c9b3e4a1e441e1a92a271f9aa1811e, user_id=d4671ff245d94e54ac1bf390edd815e5, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:37.575899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513b1ea-8687-466d-9b72-01e8189ff477 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:37.590025 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4336/8738] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:33:37 2022] PUT /identity/v3/projects/37c9b3e4a1e441e1a92a271f9aa1811e/users/d4671ff245d94e54ac1bf390edd815e5/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:33:37.595115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612023bd-c7db-4781-8908-cefd35265314 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:37.595443 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612023bd-c7db-4781-8908-cefd35265314 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:37.595726 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612023bd-c7db-4781-8908-cefd35265314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:37.617314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-612023bd-c7db-4781-8908-cefd35265314 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:33:37.618583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-612023bd-c7db-4781-8908-cefd35265314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:37.678499 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4403/8739] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:33:37.688622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513b1ea-8687-466d-9b72-01e8189ff477 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:37.690319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6513b1ea-8687-466d-9b72-01e8189ff477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:37.749534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513b1ea-8687-466d-9b72-01e8189ff477 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:37.753306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b723a446-42b0-4738-9daf-97bb511cbe31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:37.753908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723a446-42b0-4738-9daf-97bb511cbe31 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:37.754191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723a446-42b0-4738-9daf-97bb511cbe31 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:37.754454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723a446-42b0-4738-9daf-97bb511cbe31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:37.763493 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b723a446-42b0-4738-9daf-97bb511cbe31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:37.764871 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b723a446-42b0-4738-9daf-97bb511cbe31 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:37.769804 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:33:37.775403 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4337/8740] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:33:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:33:41.980311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568171d-70a5-42fd-b2cf-b9b08d16b13a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:41.980865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568171d-70a5-42fd-b2cf-b9b08d16b13a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:41.980865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568171d-70a5-42fd-b2cf-b9b08d16b13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:42.016238 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2568171d-70a5-42fd-b2cf-b9b08d16b13a None None] MFA Rules not processed for user `d4671ff245d94e54ac1bf390edd815e5`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:33:42.017457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2568171d-70a5-42fd-b2cf-b9b08d16b13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:42.132277 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4404/8741] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:33:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:33:42.143057 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05811cab-b201-4b45-b22d-c86a1e1e35bb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:42.143590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05811cab-b201-4b45-b22d-c86a1e1e35bb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:42.143590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05811cab-b201-4b45-b22d-c86a1e1e35bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:42.164283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-05811cab-b201-4b45-b22d-c86a1e1e35bb None None] MFA Rules not processed for user `d4671ff245d94e54ac1bf390edd815e5`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:33:42.165669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-05811cab-b201-4b45-b22d-c86a1e1e35bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:42.173173 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4338/8742] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:33:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:33:42.185665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568171d-70a5-42fd-b2cf-b9b08d16b13a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:42.187145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2568171d-70a5-42fd-b2cf-b9b08d16b13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:42.243328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:33:42.279203 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568171d-70a5-42fd-b2cf-b9b08d16b13a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:42.285138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eae83e-686c-4283-be6c-38e6b9361e74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:42.286320 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eae83e-686c-4283-be6c-38e6b9361e74 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:42.286633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eae83e-686c-4283-be6c-38e6b9361e74 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:42.286904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eae83e-686c-4283-be6c-38e6b9361e74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:42.295779 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eae83e-686c-4283-be6c-38e6b9361e74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:42.297299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eae83e-686c-4283-be6c-38e6b9361e74 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:42.307270 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4405/8743] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:33:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:33:52.157690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:33:52.303371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=13 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:33:56.990720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05811cab-b201-4b45-b22d-c86a1e1e35bb None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:56.992226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-05811cab-b201-4b45-b22d-c86a1e1e35bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:57.006929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05811cab-b201-4b45-b22d-c86a1e1e35bb None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:57.010591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ab038-090f-4da2-be9b-89c02bfba852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:57.011195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ab038-090f-4da2-be9b-89c02bfba852 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:57.011477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ab038-090f-4da2-be9b-89c02bfba852 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:57.011827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ab038-090f-4da2-be9b-89c02bfba852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:57.013740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-231ab038-090f-4da2-be9b-89c02bfba852 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:57.026815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ab038-090f-4da2-be9b-89c02bfba852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:57.028299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ab038-090f-4da2-be9b-89c02bfba852 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:57.044091 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4339/8744] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:33:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:33:57.128197 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eae83e-686c-4283-be6c-38e6b9361e74 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:57.129661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-08eae83e-686c-4283-be6c-38e6b9361e74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:57.200162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eae83e-686c-4283-be6c-38e6b9361e74 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:57.205149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb936cf-a447-41f3-9356-1c3755f65035 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:57.205746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb936cf-a447-41f3-9356-1c3755f65035 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:57.206026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb936cf-a447-41f3-9356-1c3755f65035 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:57.206299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb936cf-a447-41f3-9356-1c3755f65035 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:57.208556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8bb936cf-a447-41f3-9356-1c3755f65035 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:57.222456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb936cf-a447-41f3-9356-1c3755f65035 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:57.224680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb936cf-a447-41f3-9356-1c3755f65035 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:57.236917 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4406/8745] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:33:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:33:57.407199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ab038-090f-4da2-be9b-89c02bfba852 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:33:57.418947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ab038-090f-4da2-be9b-89c02bfba852 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:33:57.422686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f94c6-6c53-43a2-9da1-386532381cfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:33:57.423284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f94c6-6c53-43a2-9da1-386532381cfe service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:33:57.423568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f94c6-6c53-43a2-9da1-386532381cfe service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:33:57.423845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f94c6-6c53-43a2-9da1-386532381cfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:33:57.425541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-da4f94c6-6c53-43a2-9da1-386532381cfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:33:57.442611 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f94c6-6c53-43a2-9da1-386532381cfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:33:57.445070 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f94c6-6c53-43a2-9da1-386532381cfe service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:33:57.459761 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4340/8746] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:33:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:34:07.230285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=32 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:34:07.451396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=12 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:34:31.899038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb936cf-a447-41f3-9356-1c3755f65035 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:34:31.916585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb936cf-a447-41f3-9356-1c3755f65035 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:34:31.920780 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45ff5a-d38e-43c4-8edb-d82e2a1b11c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:34:31.922306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45ff5a-d38e-43c4-8edb-d82e2a1b11c5 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:34:31.922848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45ff5a-d38e-43c4-8edb-d82e2a1b11c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:34:31.923451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45ff5a-d38e-43c4-8edb-d82e2a1b11c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:34:31.926084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cf45ff5a-d38e-43c4-8edb-d82e2a1b11c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:34:31.947712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45ff5a-d38e-43c4-8edb-d82e2a1b11c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:34:31.949224 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45ff5a-d38e-43c4-8edb-d82e2a1b11c5 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:34:31.965474 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4407/8747] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:34:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:34:41.958093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:35:07.403254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f94c6-6c53-43a2-9da1-386532381cfe service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:35:07.415092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f94c6-6c53-43a2-9da1-386532381cfe service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:35:07.418931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6974da-3394-4107-b1f7-c08998583c9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:35:07.419539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6974da-3394-4107-b1f7-c08998583c9d service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:35:07.419842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6974da-3394-4107-b1f7-c08998583c9d service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:35:07.420101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6974da-3394-4107-b1f7-c08998583c9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:35:07.421854 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cf6974da-3394-4107-b1f7-c08998583c9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:35:07.438918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6974da-3394-4107-b1f7-c08998583c9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:35:07.440356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6974da-3394-4107-b1f7-c08998583c9d service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:35:07.454352 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4341/8748] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:35:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:35:13.797809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45ff5a-d38e-43c4-8edb-d82e2a1b11c5 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:35:13.811322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45ff5a-d38e-43c4-8edb-d82e2a1b11c5 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:35:13.815837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315d286-7cbb-4fc2-95f1-5cf51102747c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:35:13.816873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315d286-7cbb-4fc2-95f1-5cf51102747c service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:35:13.817159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315d286-7cbb-4fc2-95f1-5cf51102747c service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:35:13.817431 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315d286-7cbb-4fc2-95f1-5cf51102747c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:35:13.819188 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c315d286-7cbb-4fc2-95f1-5cf51102747c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:35:13.832189 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315d286-7cbb-4fc2-95f1-5cf51102747c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:35:13.833617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315d286-7cbb-4fc2-95f1-5cf51102747c service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:35:13.844287 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4408/8749] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:35:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:35:17.447093 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:35:23.838036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:37:12.498405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6974da-3394-4107-b1f7-c08998583c9d service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:37:12.499867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cf6974da-3394-4107-b1f7-c08998583c9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:37:12.555032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6974da-3394-4107-b1f7-c08998583c9d service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:37:12.560046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc9925-c497-4452-9f8b-8d6c365d53a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:37:12.560590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc9925-c497-4452-9f8b-8d6c365d53a7 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:37:12.560872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc9925-c497-4452-9f8b-8d6c365d53a7 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:37:12.561145 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc9925-c497-4452-9f8b-8d6c365d53a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:37:12.562888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-48cc9925-c497-4452-9f8b-8d6c365d53a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:37:12.576962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc9925-c497-4452-9f8b-8d6c365d53a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:37:12.578567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc9925-c497-4452-9f8b-8d6c365d53a7 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:37:12.591401 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4342/8750] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:37:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:37:14.472662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315d286-7cbb-4fc2-95f1-5cf51102747c service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:37:14.494768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315d286-7cbb-4fc2-95f1-5cf51102747c service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:37:14.499825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b6de2-21d5-49c3-9a62-2b857fb36613 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:37:14.500448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b6de2-21d5-49c3-9a62-2b857fb36613 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:37:14.500732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b6de2-21d5-49c3-9a62-2b857fb36613 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:37:14.501009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b6de2-21d5-49c3-9a62-2b857fb36613 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:37:14.502775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-718b6de2-21d5-49c3-9a62-2b857fb36613 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:37:14.517168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b6de2-21d5-49c3-9a62-2b857fb36613 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:37:14.518778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b6de2-21d5-49c3-9a62-2b857fb36613 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:37:14.532177 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4409/8751] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:37:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:37:22.584123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=22 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:37:24.525946 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=9 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:38:42.942901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc9925-c497-4452-9f8b-8d6c365d53a7 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:38:42.944328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-48cc9925-c497-4452-9f8b-8d6c365d53a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:38:42.981619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc9925-c497-4452-9f8b-8d6c365d53a7 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:38:42.987148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231f2af-6ac5-4d43-a6ec-41de9b765d25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:38:42.987745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231f2af-6ac5-4d43-a6ec-41de9b765d25 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:38:42.988026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231f2af-6ac5-4d43-a6ec-41de9b765d25 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:38:42.988297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231f2af-6ac5-4d43-a6ec-41de9b765d25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:38:42.995996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231f2af-6ac5-4d43-a6ec-41de9b765d25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:38:42.997569 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231f2af-6ac5-4d43-a6ec-41de9b765d25 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:38:43.007383 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4343/8752] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:38:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:38:53.002424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=14 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:39:34.138179 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10a0aa-4ef7-4aa0-80e8-9a0cf6a67ee6 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:39:34.138773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10a0aa-4ef7-4aa0-80e8-9a0cf6a67ee6 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:39:34.138773 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10a0aa-4ef7-4aa0-80e8-9a0cf6a67ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:39:34.211810 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ef10a0aa-4ef7-4aa0-80e8-9a0cf6a67ee6 None None] MFA Rules not processed for user `e17a026f305248878f18eec35e1e02dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:39:34.213014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ef10a0aa-4ef7-4aa0-80e8-9a0cf6a67ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:39:34.220199 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4410/8753] 10.222.0.17 () {58 vars in 1061 bytes} [Sat Oct 1 21:39:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:39:34.226354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231f2af-6ac5-4d43-a6ec-41de9b765d25 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:39:34.236005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231f2af-6ac5-4d43-a6ec-41de9b765d25 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:39:34.239629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575591f-bf97-4211-9c19-ecfe61c68c93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:39:34.240225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575591f-bf97-4211-9c19-ecfe61c68c93 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:39:34.240504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575591f-bf97-4211-9c19-ecfe61c68c93 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:39:34.240777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575591f-bf97-4211-9c19-ecfe61c68c93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:39:34.253810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575591f-bf97-4211-9c19-ecfe61c68c93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:39:34.255295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575591f-bf97-4211-9c19-ecfe61c68c93 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:39:34.269315 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4344/8754] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:39:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:39:44.209541 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=22 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:39:44.262215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:40:08.822455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118f734-8c02-4be3-9735-616e30de60d4 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:08.823073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118f734-8c02-4be3-9735-616e30de60d4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:08.823073 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118f734-8c02-4be3-9735-616e30de60d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:08.843631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2118f734-8c02-4be3-9735-616e30de60d4 None None] MFA Rules not processed for user `e17a026f305248878f18eec35e1e02dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:40:08.844820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2118f734-8c02-4be3-9735-616e30de60d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:40:08.851560 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4411/8755] 10.222.0.17 () {58 vars in 1061 bytes} [Sat Oct 1 21:40:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:40:08.857308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575591f-bf97-4211-9c19-ecfe61c68c93 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:08.865811 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575591f-bf97-4211-9c19-ecfe61c68c93 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:08.869779 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8864f8-b35c-488f-8bec-bbbfca30755e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:08.870376 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8864f8-b35c-488f-8bec-bbbfca30755e service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:08.870697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8864f8-b35c-488f-8bec-bbbfca30755e service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:08.870990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8864f8-b35c-488f-8bec-bbbfca30755e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:08.883922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8864f8-b35c-488f-8bec-bbbfca30755e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:08.885387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8864f8-b35c-488f-8bec-bbbfca30755e service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:08.899237 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4345/8756] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:40:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:40:18.838186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:40:18.891979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:40:19.208620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118f734-8c02-4be3-9735-616e30de60d4 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:19.217313 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118f734-8c02-4be3-9735-616e30de60d4 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:19.221012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf323c60-f6a4-4daa-b592-40dddb7bd55b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:19.221609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf323c60-f6a4-4daa-b592-40dddb7bd55b service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:19.221892 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf323c60-f6a4-4daa-b592-40dddb7bd55b service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:19.222164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf323c60-f6a4-4daa-b592-40dddb7bd55b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:19.231922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf323c60-f6a4-4daa-b592-40dddb7bd55b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:19.233335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf323c60-f6a4-4daa-b592-40dddb7bd55b service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:19.243696 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4412/8757] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:40:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:40:29.237673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:40:32.430557 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8864f8-b35c-488f-8bec-bbbfca30755e service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:32.439181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8864f8-b35c-488f-8bec-bbbfca30755e service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:32.442860 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c1080-48d5-4746-8018-bd75951e8027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:32.443453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c1080-48d5-4746-8018-bd75951e8027 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:32.443734 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c1080-48d5-4746-8018-bd75951e8027 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:32.444007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c1080-48d5-4746-8018-bd75951e8027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:32.452928 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c1080-48d5-4746-8018-bd75951e8027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:32.454330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c1080-48d5-4746-8018-bd75951e8027 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:32.463834 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4346/8758] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:40:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:40:35.324824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf323c60-f6a4-4daa-b592-40dddb7bd55b service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:35.341913 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf323c60-f6a4-4daa-b592-40dddb7bd55b service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:35.347627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d8a642-c4e3-4731-8df3-6300a1da4e4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:35.348350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8a642-c4e3-4731-8df3-6300a1da4e4f service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:35.348698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8a642-c4e3-4731-8df3-6300a1da4e4f service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:35.349042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8a642-c4e3-4731-8df3-6300a1da4e4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:35.351092 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-56d8a642-c4e3-4731-8df3-6300a1da4e4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:40:35.368340 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d8a642-c4e3-4731-8df3-6300a1da4e4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:35.370112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d8a642-c4e3-4731-8df3-6300a1da4e4f service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:35.386932 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4413/8759] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:40:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:40:35.396364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c1080-48d5-4746-8018-bd75951e8027 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:35.407553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c1080-48d5-4746-8018-bd75951e8027 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:35.412679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ae6fc-8c57-4ae1-9914-9b9d143ef476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:35.413386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ae6fc-8c57-4ae1-9914-9b9d143ef476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:35.413873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ae6fc-8c57-4ae1-9914-9b9d143ef476 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:35.414160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ae6fc-8c57-4ae1-9914-9b9d143ef476 admin admin] PATH_INFO: `/v3/users/d4671ff245d94e54ac1bf390edd815e5` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:35.420547 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ae6fc-8c57-4ae1-9914-9b9d143ef476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4671ff245d94e54ac1bf390edd815e5)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:35.422506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ae6fc-8c57-4ae1-9914-9b9d143ef476 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:35.473431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-881ae6fc-8c57-4ae1-9914-9b9d143ef476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4671ff245d94e54ac1bf390edd815e5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:40:35.474095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-881ae6fc-8c57-4ae1-9914-9b9d143ef476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4671ff245d94e54ac1bf390edd815e5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:40:35.474789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-881ae6fc-8c57-4ae1-9914-9b9d143ef476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4671ff245d94e54ac1bf390edd815e5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:40:35.483981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-881ae6fc-8c57-4ae1-9914-9b9d143ef476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4671ff245d94e54ac1bf390edd815e5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:40:35.496123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-881ae6fc-8c57-4ae1-9914-9b9d143ef476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4671ff245d94e54ac1bf390edd815e5'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:40:35.508025 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4347/8760] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:40:35 2022] DELETE /identity/v3/users/d4671ff245d94e54ac1bf390edd815e5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:40:35.687258 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d8a642-c4e3-4731-8df3-6300a1da4e4f service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:35.701981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d8a642-c4e3-4731-8df3-6300a1da4e4f service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:35.771481 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854b7a5-b3b7-4bac-a2ac-2b93b1c11434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:35.772109 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854b7a5-b3b7-4bac-a2ac-2b93b1c11434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:35.772390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854b7a5-b3b7-4bac-a2ac-2b93b1c11434 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:35.772664 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854b7a5-b3b7-4bac-a2ac-2b93b1c11434 admin admin] PATH_INFO: `/v3/projects/37c9b3e4a1e441e1a92a271f9aa1811e` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:35.773769 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f854b7a5-b3b7-4bac-a2ac-2b93b1c11434 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37c9b3e4a1e441e1a92a271f9aa1811e)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:35.775691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f854b7a5-b3b7-4bac-a2ac-2b93b1c11434 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:35.820715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f854b7a5-b3b7-4bac-a2ac-2b93b1c11434 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37c9b3e4a1e441e1a92a271f9aa1811e', 'request_id': 'req-f854b7a5-b3b7-4bac-a2ac-2b93b1c11434'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:40:35.827296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f854b7a5-b3b7-4bac-a2ac-2b93b1c11434 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37c9b3e4a1e441e1a92a271f9aa1811e', 'request_id': 'req-f854b7a5-b3b7-4bac-a2ac-2b93b1c11434'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:40:35.832863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f854b7a5-b3b7-4bac-a2ac-2b93b1c11434 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37c9b3e4a1e441e1a92a271f9aa1811e', 'request_id': 'req-f854b7a5-b3b7-4bac-a2ac-2b93b1c11434'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:40:35.866533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f854b7a5-b3b7-4bac-a2ac-2b93b1c11434 admin admin] The token cache is being invalidate because project 37c9b3e4a1e441e1a92a271f9aa1811e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:40:35.866833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f854b7a5-b3b7-4bac-a2ac-2b93b1c11434 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:40:35.867477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-f854b7a5-b3b7-4bac-a2ac-2b93b1c11434 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:40:35.869029 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4414/8761] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:40:35 2022] DELETE /identity/v3/projects/37c9b3e4a1e441e1a92a271f9aa1811e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:40:35.873942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3a2a4-5a0c-4f25-a2a9-f9fa78060755 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:35.874228 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3a2a4-5a0c-4f25-a2a9-f9fa78060755 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:35.874533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3a2a4-5a0c-4f25-a2a9-f9fa78060755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:35.922699 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-44f3a2a4-5a0c-4f25-a2a9-f9fa78060755 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:40:35.923843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-44f3a2a4-5a0c-4f25-a2a9-f9fa78060755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:40:35.982084 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4348/8762] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:40:35.990162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f68b4-ca86-49fa-814f-5b57de540001 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:35.990459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f68b4-ca86-49fa-814f-5b57de540001 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:35.990770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f68b4-ca86-49fa-814f-5b57de540001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:36.010728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7e9f68b4-ca86-49fa-814f-5b57de540001 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:40:36.012564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7e9f68b4-ca86-49fa-814f-5b57de540001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:40:36.020613 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4415/8763] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:40:36.024753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3a2a4-5a0c-4f25-a2a9-f9fa78060755 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:36.034706 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3a2a4-5a0c-4f25-a2a9-f9fa78060755 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:36.039309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82122dd7-01d3-49ca-a420-302ef3ea3d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:36.039908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82122dd7-01d3-49ca-a420-302ef3ea3d97 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:36.040187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82122dd7-01d3-49ca-a420-302ef3ea3d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:36.040461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82122dd7-01d3-49ca-a420-302ef3ea3d97 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:36.041096 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82122dd7-01d3-49ca-a420-302ef3ea3d97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:40:36.041399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82122dd7-01d3-49ca-a420-302ef3ea3d97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:36.043066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82122dd7-01d3-49ca-a420-302ef3ea3d97 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:36.055170 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4349/8764] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:40:36.060068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f68b4-ca86-49fa-814f-5b57de540001 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:36.069366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f68b4-ca86-49fa-814f-5b57de540001 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:36.074190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb8cac-25cf-4d47-ba16-9d3e1731b161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:36.075322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb8cac-25cf-4d47-ba16-9d3e1731b161 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:36.075606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb8cac-25cf-4d47-ba16-9d3e1731b161 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:36.075915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb8cac-25cf-4d47-ba16-9d3e1731b161 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:36.076543 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb8cac-25cf-4d47-ba16-9d3e1731b161 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:36.078447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb8cac-25cf-4d47-ba16-9d3e1731b161 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:36.105688 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4416/8765] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:40:36 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:40:36.109882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82122dd7-01d3-49ca-a420-302ef3ea3d97 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:36.119888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82122dd7-01d3-49ca-a420-302ef3ea3d97 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:36.141431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:40:36.179146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e8c5b-5193-4cfc-bbcb-48afb7e54b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:36.179759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e8c5b-5193-4cfc-bbcb-48afb7e54b8a admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:36.180043 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e8c5b-5193-4cfc-bbcb-48afb7e54b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:36.180321 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e8c5b-5193-4cfc-bbcb-48afb7e54b8a admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:36.180936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e8c5b-5193-4cfc-bbcb-48afb7e54b8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:36.182890 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e8c5b-5193-4cfc-bbcb-48afb7e54b8a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:36.206071 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4350/8766] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:40:36 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:40:36.210402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb8cac-25cf-4d47-ba16-9d3e1731b161 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:36.220876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb8cac-25cf-4d47-ba16-9d3e1731b161 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:36.225582 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff9d79-d8b9-4ac8-98dc-a3eaf50469da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:36.226180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff9d79-d8b9-4ac8-98dc-a3eaf50469da admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:36.226465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff9d79-d8b9-4ac8-98dc-a3eaf50469da admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:36.226775 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff9d79-d8b9-4ac8-98dc-a3eaf50469da admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:36.227425 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff9d79-d8b9-4ac8-98dc-a3eaf50469da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:36.229014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff9d79-d8b9-4ac8-98dc-a3eaf50469da admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:36.239216 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4417/8767] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:40:36.243034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e8c5b-5193-4cfc-bbcb-48afb7e54b8a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:36.253514 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e8c5b-5193-4cfc-bbcb-48afb7e54b8a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:36.258373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b4fef-37c4-43a4-ae30-a5a818c203ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:36.259306 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b4fef-37c4-43a4-ae30-a5a818c203ff admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:36.259591 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b4fef-37c4-43a4-ae30-a5a818c203ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:36.259864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b4fef-37c4-43a4-ae30-a5a818c203ff admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:36.260496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b4fef-37c4-43a4-ae30-a5a818c203ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:36.262724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b4fef-37c4-43a4-ae30-a5a818c203ff admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:36.273447 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4351/8768] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:40:36.277310 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff9d79-d8b9-4ac8-98dc-a3eaf50469da admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:36.287426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff9d79-d8b9-4ac8-98dc-a3eaf50469da admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:36.292326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab7ffb-f706-46ac-bbf8-fc780ad012d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:36.292965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab7ffb-f706-46ac-bbf8-fc780ad012d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:36.293243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab7ffb-f706-46ac-bbf8-fc780ad012d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:36.293516 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab7ffb-f706-46ac-bbf8-fc780ad012d3 admin admin] PATH_INFO: `/v3/projects/556938530bd8414b99541d920b467ab8/users/d66e970d7ba14adcb41f1789c7af8509/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:36.307977 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab7ffb-f706-46ac-bbf8-fc780ad012d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=556938530bd8414b99541d920b467ab8, user_id=d66e970d7ba14adcb41f1789c7af8509, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:36.311111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab7ffb-f706-46ac-bbf8-fc780ad012d3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:36.313257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:40:36.326004 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4418/8769] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:40:36 2022] PUT /identity/v3/projects/556938530bd8414b99541d920b467ab8/users/d66e970d7ba14adcb41f1789c7af8509/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:40:36.331279 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b0fb1-01ea-425c-9b79-de8f527f225a None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:36.331561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b0fb1-01ea-425c-9b79-de8f527f225a None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:36.331844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b0fb1-01ea-425c-9b79-de8f527f225a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:36.351549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-774b0fb1-01ea-425c-9b79-de8f527f225a None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:40:36.352760 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-774b0fb1-01ea-425c-9b79-de8f527f225a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:40:36.410943 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4352/8770] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:40:36.421232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab7ffb-f706-46ac-bbf8-fc780ad012d3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:36.422940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e7ab7ffb-f706-46ac-bbf8-fc780ad012d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:40:36.483158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab7ffb-f706-46ac-bbf8-fc780ad012d3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:36.487380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829809b-95af-496a-a18a-3bdb382da9ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:36.487991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829809b-95af-496a-a18a-3bdb382da9ea service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:36.488269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829809b-95af-496a-a18a-3bdb382da9ea service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:36.488542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829809b-95af-496a-a18a-3bdb382da9ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:36.498932 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829809b-95af-496a-a18a-3bdb382da9ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:36.500361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829809b-95af-496a-a18a-3bdb382da9ea service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:36.509909 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4419/8771] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:40:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:40:40.821102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe269e9e-3e4e-48d9-ad44-afa85aa795bf None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:40.822188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe269e9e-3e4e-48d9-ad44-afa85aa795bf None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:40.822663 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe269e9e-3e4e-48d9-ad44-afa85aa795bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:40.859956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fe269e9e-3e4e-48d9-ad44-afa85aa795bf None None] MFA Rules not processed for user `d66e970d7ba14adcb41f1789c7af8509`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:40:40.861224 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fe269e9e-3e4e-48d9-ad44-afa85aa795bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:40:40.938411 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4353/8772] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:40:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:40:40.950442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c045b-6786-4f3a-877d-7bde7151955a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:40.951410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c045b-6786-4f3a-877d-7bde7151955a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:40.951692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c045b-6786-4f3a-877d-7bde7151955a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:40.974290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b18c045b-6786-4f3a-877d-7bde7151955a None None] MFA Rules not processed for user `d66e970d7ba14adcb41f1789c7af8509`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:40:40.975610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b18c045b-6786-4f3a-877d-7bde7151955a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:40:40.984712 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4420/8773] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:40:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:40:40.997103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe269e9e-3e4e-48d9-ad44-afa85aa795bf None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:40.998610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fe269e9e-3e4e-48d9-ad44-afa85aa795bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:40:41.041882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:40:41.091893 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe269e9e-3e4e-48d9-ad44-afa85aa795bf None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:41.098791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c4432-ac2c-42fe-8c2c-89158192c7fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:41.099533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c4432-ac2c-42fe-8c2c-89158192c7fb service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:41.100058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c4432-ac2c-42fe-8c2c-89158192c7fb service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:41.100457 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c4432-ac2c-42fe-8c2c-89158192c7fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:41.112208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c4432-ac2c-42fe-8c2c-89158192c7fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:41.114380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c4432-ac2c-42fe-8c2c-89158192c7fb service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:41.125723 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4354/8774] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:40:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:40:41.387731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c045b-6786-4f3a-877d-7bde7151955a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:41.389120 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b18c045b-6786-4f3a-877d-7bde7151955a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:40:41.401801 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c045b-6786-4f3a-877d-7bde7151955a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:41.405872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a381d-716c-4f5f-81d3-ecd7f9e5c937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:41.406457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a381d-716c-4f5f-81d3-ecd7f9e5c937 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:41.406764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a381d-716c-4f5f-81d3-ecd7f9e5c937 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:41.407049 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a381d-716c-4f5f-81d3-ecd7f9e5c937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:41.408717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-095a381d-716c-4f5f-81d3-ecd7f9e5c937 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:40:41.421141 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a381d-716c-4f5f-81d3-ecd7f9e5c937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:41.422581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a381d-716c-4f5f-81d3-ecd7f9e5c937 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:41.433364 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4421/8775] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:40:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:40:41.764134 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5afa1e-197d-498f-a039-c702091e395d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:41.764420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5afa1e-197d-498f-a039-c702091e395d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:41.764722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5afa1e-197d-498f-a039-c702091e395d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:41.809962 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9a5afa1e-197d-498f-a039-c702091e395d None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:40:41.811428 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9a5afa1e-197d-498f-a039-c702091e395d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:40:41.881035 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4355/8776] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 21:40:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:40:41.893059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a381d-716c-4f5f-81d3-ecd7f9e5c937 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:41.894510 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-095a381d-716c-4f5f-81d3-ecd7f9e5c937 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:40:41.913025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a381d-716c-4f5f-81d3-ecd7f9e5c937 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:41.918086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607ac23-7b54-49a4-a1cf-ba21caa5ff70 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:41.918771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607ac23-7b54-49a4-a1cf-ba21caa5ff70 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:41.919056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607ac23-7b54-49a4-a1cf-ba21caa5ff70 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:41.919331 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607ac23-7b54-49a4-a1cf-ba21caa5ff70 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:41.927929 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607ac23-7b54-49a4-a1cf-ba21caa5ff70 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:41.929802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607ac23-7b54-49a4-a1cf-ba21caa5ff70 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:41.939694 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4422/8777] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:40:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:40:43.720235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5afa1e-197d-498f-a039-c702091e395d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:43.730183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5afa1e-197d-498f-a039-c702091e395d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:43.734420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76000f7-8e05-4953-8ee7-957b5f82670e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:43.735268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76000f7-8e05-4953-8ee7-957b5f82670e service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:43.735548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76000f7-8e05-4953-8ee7-957b5f82670e service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:43.735846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76000f7-8e05-4953-8ee7-957b5f82670e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:43.737694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d76000f7-8e05-4953-8ee7-957b5f82670e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:40:43.751640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76000f7-8e05-4953-8ee7-957b5f82670e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:43.753090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76000f7-8e05-4953-8ee7-957b5f82670e service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:43.766260 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4356/8778] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:40:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:40:44.572692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607ac23-7b54-49a4-a1cf-ba21caa5ff70 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:44.589261 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607ac23-7b54-49a4-a1cf-ba21caa5ff70 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:44.595853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6cbef9-74e4-462c-b7c3-8395284f14b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:44.597380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6cbef9-74e4-462c-b7c3-8395284f14b4 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:44.597853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6cbef9-74e4-462c-b7c3-8395284f14b4 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:44.598125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6cbef9-74e4-462c-b7c3-8395284f14b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:44.599992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9d6cbef9-74e4-462c-b7c3-8395284f14b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:40:44.617171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6cbef9-74e4-462c-b7c3-8395284f14b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:44.618914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6cbef9-74e4-462c-b7c3-8395284f14b4 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:44.635064 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4423/8779] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:40:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:40:51.038310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76000f7-8e05-4953-8ee7-957b5f82670e service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:51.059211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76000f7-8e05-4953-8ee7-957b5f82670e service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:51.064691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d4843-912f-4a3c-9421-729526c5ca5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:51.065294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d4843-912f-4a3c-9421-729526c5ca5d service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:51.065574 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d4843-912f-4a3c-9421-729526c5ca5d service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:51.065846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d4843-912f-4a3c-9421-729526c5ca5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:51.067622 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4b1d4843-912f-4a3c-9421-729526c5ca5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:40:51.082849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d4843-912f-4a3c-9421-729526c5ca5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:51.084449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d4843-912f-4a3c-9421-729526c5ca5d service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:51.098699 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4357/8780] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:40:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:40:54.416502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6cbef9-74e4-462c-b7c3-8395284f14b4 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:54.425672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6cbef9-74e4-462c-b7c3-8395284f14b4 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:54.429575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62939a73-1cd0-4bb2-84e5-e4e87f594537 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:54.430295 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62939a73-1cd0-4bb2-84e5-e4e87f594537 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:54.430607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62939a73-1cd0-4bb2-84e5-e4e87f594537 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:54.430883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62939a73-1cd0-4bb2-84e5-e4e87f594537 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:54.432655 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-62939a73-1cd0-4bb2-84e5-e4e87f594537 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:40:54.445521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62939a73-1cd0-4bb2-84e5-e4e87f594537 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:54.446976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62939a73-1cd0-4bb2-84e5-e4e87f594537 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:54.458374 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4424/8781] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:40:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:40:54.538844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070503d3-5625-4a06-8228-411325b6d53b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:54.539315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070503d3-5625-4a06-8228-411325b6d53b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:54.539412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070503d3-5625-4a06-8228-411325b6d53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:54.560590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-070503d3-5625-4a06-8228-411325b6d53b None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:40:54.561833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-070503d3-5625-4a06-8228-411325b6d53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:40:54.573789 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4358/8782] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 21:40:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:40:54.585244 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62939a73-1cd0-4bb2-84e5-e4e87f594537 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:54.600153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62939a73-1cd0-4bb2-84e5-e4e87f594537 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:54.605839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e1d2b-baf5-432f-ae6c-dd45e17fd00c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:54.606439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e1d2b-baf5-432f-ae6c-dd45e17fd00c service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:54.606754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e1d2b-baf5-432f-ae6c-dd45e17fd00c service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:54.607033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e1d2b-baf5-432f-ae6c-dd45e17fd00c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:54.615350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e1d2b-baf5-432f-ae6c-dd45e17fd00c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:54.617399 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e1d2b-baf5-432f-ae6c-dd45e17fd00c service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:54.627956 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4425/8783] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:40:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:40:54.802081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070503d3-5625-4a06-8228-411325b6d53b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:54.811809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070503d3-5625-4a06-8228-411325b6d53b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:54.815525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ccc04b-b40d-4c55-a390-a97ba869c44b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:54.816150 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccc04b-b40d-4c55-a390-a97ba869c44b service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:54.816427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccc04b-b40d-4c55-a390-a97ba869c44b service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:54.816698 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccc04b-b40d-4c55-a390-a97ba869c44b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:54.818422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-32ccc04b-b40d-4c55-a390-a97ba869c44b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:40:54.834038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ccc04b-b40d-4c55-a390-a97ba869c44b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:54.835494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ccc04b-b40d-4c55-a390-a97ba869c44b service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:54.853982 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4359/8784] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:40:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:40:54.910329 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b26ec8-5f6a-4cf3-b438-6ee23e7d5d9e None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:54.910648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b26ec8-5f6a-4cf3-b438-6ee23e7d5d9e None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:54.910934 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b26ec8-5f6a-4cf3-b438-6ee23e7d5d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:54.931757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-30b26ec8-5f6a-4cf3-b438-6ee23e7d5d9e None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:40:54.932973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-30b26ec8-5f6a-4cf3-b438-6ee23e7d5d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:40:54.941940 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4426/8785] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 21:40:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:40:54.953374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ccc04b-b40d-4c55-a390-a97ba869c44b service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:40:54.967729 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ccc04b-b40d-4c55-a390-a97ba869c44b service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:40:54.972843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8b0dc-8641-4132-b730-6e053679c302 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:40:54.973441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8b0dc-8641-4132-b730-6e053679c302 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:40:54.973718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8b0dc-8641-4132-b730-6e053679c302 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:40:54.973990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8b0dc-8641-4132-b730-6e053679c302 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:40:54.982943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8b0dc-8641-4132-b730-6e053679c302 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:40:54.984571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8b0dc-8641-4132-b730-6e053679c302 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:40:54.994478 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4360/8786] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:40:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:41:04.926559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=1,keystone:SELECT=64 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:41:04.990326 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=81 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:41:23.699057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b26ec8-5f6a-4cf3-b438-6ee23e7d5d9e None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:41:23.726270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b26ec8-5f6a-4cf3-b438-6ee23e7d5d9e None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:41:23.732996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45840d6f-714b-4a10-b2e4-47079faea24e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:41:23.733608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45840d6f-714b-4a10-b2e4-47079faea24e service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:41:23.733891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45840d6f-714b-4a10-b2e4-47079faea24e service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:41:23.734165 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45840d6f-714b-4a10-b2e4-47079faea24e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:41:23.736121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-45840d6f-714b-4a10-b2e4-47079faea24e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:41:23.753129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45840d6f-714b-4a10-b2e4-47079faea24e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:41:23.754925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45840d6f-714b-4a10-b2e4-47079faea24e service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:41:23.768877 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4427/8787] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:41:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:41:29.962041 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d650d7-dd6b-4043-b40d-31354b1fcc44 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:41:29.963125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d650d7-dd6b-4043-b40d-31354b1fcc44 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:41:29.963468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d650d7-dd6b-4043-b40d-31354b1fcc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:41:29.984637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-b7d650d7-dd6b-4043-b40d-31354b1fcc44 None None] MFA Rules not processed for user `d66e970d7ba14adcb41f1789c7af8509`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:41:29.985896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b7d650d7-dd6b-4043-b40d-31354b1fcc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:41:29.992787 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4361/8788] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:41:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:41:30.248221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45840d6f-714b-4a10-b2e4-47079faea24e service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:41:30.257522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45840d6f-714b-4a10-b2e4-47079faea24e service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:41:30.261112 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6fa63-992f-43fb-8b32-a7dfa74f7894 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:41:30.262351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6fa63-992f-43fb-8b32-a7dfa74f7894 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:41:30.262742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6fa63-992f-43fb-8b32-a7dfa74f7894 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:41:30.263025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6fa63-992f-43fb-8b32-a7dfa74f7894 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:41:30.264744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f6d6fa63-992f-43fb-8b32-a7dfa74f7894 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:41:30.276630 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6fa63-992f-43fb-8b32-a7dfa74f7894 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:41:30.278029 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6fa63-992f-43fb-8b32-a7dfa74f7894 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:41:30.288458 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4428/8789] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:41:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:41:39.978776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:41:40.282422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=12 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:42:14.518234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d650d7-dd6b-4043-b40d-31354b1fcc44 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:42:14.544447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d650d7-dd6b-4043-b40d-31354b1fcc44 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:42:14.549567 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d77a7-3e9f-4fe8-9531-bb5257f1289d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:42:14.550187 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d77a7-3e9f-4fe8-9531-bb5257f1289d service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:42:14.550467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d77a7-3e9f-4fe8-9531-bb5257f1289d service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:42:14.550780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d77a7-3e9f-4fe8-9531-bb5257f1289d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:42:14.552563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bf8d77a7-3e9f-4fe8-9531-bb5257f1289d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:42:14.565834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6fa63-992f-43fb-8b32-a7dfa74f7894 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:42:14.567314 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f6d6fa63-992f-43fb-8b32-a7dfa74f7894 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:42:14.567536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d77a7-3e9f-4fe8-9531-bb5257f1289d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:42:14.569159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d77a7-3e9f-4fe8-9531-bb5257f1289d service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:42:14.583290 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4362/8790] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:42:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:42:14.585222 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6fa63-992f-43fb-8b32-a7dfa74f7894 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:42:14.590926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de85ec-d289-47fa-92c3-f46ff2876e64 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:42:14.591608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de85ec-d289-47fa-92c3-f46ff2876e64 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:42:14.591832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de85ec-d289-47fa-92c3-f46ff2876e64 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:42:14.592105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de85ec-d289-47fa-92c3-f46ff2876e64 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:42:14.593865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c0de85ec-d289-47fa-92c3-f46ff2876e64 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:42:14.608624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de85ec-d289-47fa-92c3-f46ff2876e64 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:42:14.610191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de85ec-d289-47fa-92c3-f46ff2876e64 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:42:14.622950 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4429/8791] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:42:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:42:22.376309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d77a7-3e9f-4fe8-9531-bb5257f1289d service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:42:22.387374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d77a7-3e9f-4fe8-9531-bb5257f1289d service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:42:22.392333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c5e94-6f2a-47f3-a99b-f5e596217414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:42:22.392966 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c5e94-6f2a-47f3-a99b-f5e596217414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:42:22.393275 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c5e94-6f2a-47f3-a99b-f5e596217414 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:42:22.393563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c5e94-6f2a-47f3-a99b-f5e596217414 admin admin] PATH_INFO: `/v3/users/d66e970d7ba14adcb41f1789c7af8509` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:42:22.394713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c5e94-6f2a-47f3-a99b-f5e596217414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d66e970d7ba14adcb41f1789c7af8509)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:42:22.396653 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c5e94-6f2a-47f3-a99b-f5e596217414 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:42:22.444245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0c6c5e94-6f2a-47f3-a99b-f5e596217414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd66e970d7ba14adcb41f1789c7af8509'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:42:22.444899 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0c6c5e94-6f2a-47f3-a99b-f5e596217414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd66e970d7ba14adcb41f1789c7af8509'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:42:22.445505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0c6c5e94-6f2a-47f3-a99b-f5e596217414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd66e970d7ba14adcb41f1789c7af8509'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:42:22.453231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0c6c5e94-6f2a-47f3-a99b-f5e596217414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd66e970d7ba14adcb41f1789c7af8509'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:42:22.466144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-0c6c5e94-6f2a-47f3-a99b-f5e596217414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd66e970d7ba14adcb41f1789c7af8509'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:42:22.478260 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4363/8792] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:42:22 2022] DELETE /identity/v3/users/d66e970d7ba14adcb41f1789c7af8509 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:42:22.653161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de85ec-d289-47fa-92c3-f46ff2876e64 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:42:22.663859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de85ec-d289-47fa-92c3-f46ff2876e64 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:42:22.721391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb28ca-6cc6-4b72-8419-766b4cfb0b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:42:22.722017 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb28ca-6cc6-4b72-8419-766b4cfb0b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:42:22.722298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb28ca-6cc6-4b72-8419-766b4cfb0b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:42:22.722607 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb28ca-6cc6-4b72-8419-766b4cfb0b01 admin admin] PATH_INFO: `/v3/projects/556938530bd8414b99541d920b467ab8` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:42:22.723722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb28ca-6cc6-4b72-8419-766b4cfb0b01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=556938530bd8414b99541d920b467ab8)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:42:22.725608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb28ca-6cc6-4b72-8419-766b4cfb0b01 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:42:22.776958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-36cb28ca-6cc6-4b72-8419-766b4cfb0b01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '556938530bd8414b99541d920b467ab8', 'request_id': 'req-36cb28ca-6cc6-4b72-8419-766b4cfb0b01'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:42:22.783712 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-36cb28ca-6cc6-4b72-8419-766b4cfb0b01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '556938530bd8414b99541d920b467ab8', 'request_id': 'req-36cb28ca-6cc6-4b72-8419-766b4cfb0b01'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:42:22.789569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-36cb28ca-6cc6-4b72-8419-766b4cfb0b01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '556938530bd8414b99541d920b467ab8', 'request_id': 'req-36cb28ca-6cc6-4b72-8419-766b4cfb0b01'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:42:22.801575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-36cb28ca-6cc6-4b72-8419-766b4cfb0b01 admin admin] The token cache is being invalidate because project 556938530bd8414b99541d920b467ab8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:42:22.801867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-36cb28ca-6cc6-4b72-8419-766b4cfb0b01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:42:22.802754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-36cb28ca-6cc6-4b72-8419-766b4cfb0b01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:42:22.804613 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4430/8793] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:42:22 2022] DELETE /identity/v3/projects/556938530bd8414b99541d920b467ab8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:42:22.810336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e63f1-7a95-4006-9fec-a87877ac2758 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:42:22.810659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e63f1-7a95-4006-9fec-a87877ac2758 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:42:22.810936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e63f1-7a95-4006-9fec-a87877ac2758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:42:22.838199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-df0e63f1-7a95-4006-9fec-a87877ac2758 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:42:22.840026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-df0e63f1-7a95-4006-9fec-a87877ac2758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:42:22.905704 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4364/8794] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:42:22.913678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340f1ff-0e1d-4c28-a2fc-c7e50198994d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:42:22.914033 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340f1ff-0e1d-4c28-a2fc-c7e50198994d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:42:22.914381 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340f1ff-0e1d-4c28-a2fc-c7e50198994d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:42:22.936170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4340f1ff-0e1d-4c28-a2fc-c7e50198994d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:42:22.937273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4340f1ff-0e1d-4c28-a2fc-c7e50198994d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:42:22.945122 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4431/8795] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:42:22.949800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e63f1-7a95-4006-9fec-a87877ac2758 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:42:22.960135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e63f1-7a95-4006-9fec-a87877ac2758 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:42:22.965725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ab880-d1d5-4da2-b3c2-ff6b8a59f567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:42:22.966317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ab880-d1d5-4da2-b3c2-ff6b8a59f567 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:42:22.966633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ab880-d1d5-4da2-b3c2-ff6b8a59f567 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:42:22.966903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ab880-d1d5-4da2-b3c2-ff6b8a59f567 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:42:22.967537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ab880-d1d5-4da2-b3c2-ff6b8a59f567 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:42:22.967833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ab880-d1d5-4da2-b3c2-ff6b8a59f567 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:42:22.969740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ab880-d1d5-4da2-b3c2-ff6b8a59f567 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:42:22.981367 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4365/8796] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 01 21:42:22.985474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340f1ff-0e1d-4c28-a2fc-c7e50198994d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:42:22.994826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340f1ff-0e1d-4c28-a2fc-c7e50198994d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:42:22.999509 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f0709-41bb-43fe-8baa-cd42fc91102f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:42:23.000101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f0709-41bb-43fe-8baa-cd42fc91102f admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:42:23.000379 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f0709-41bb-43fe-8baa-cd42fc91102f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:42:23.000646 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f0709-41bb-43fe-8baa-cd42fc91102f admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:42:23.001273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f0709-41bb-43fe-8baa-cd42fc91102f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:42:23.003201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f0709-41bb-43fe-8baa-cd42fc91102f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:42:23.034822 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4432/8797] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:42:22 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:42:23.036298 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ab880-d1d5-4da2-b3c2-ff6b8a59f567 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:42:23.046332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ab880-d1d5-4da2-b3c2-ff6b8a59f567 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:42:23.080348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:42:23.104957 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844cce1c-1b8d-4beb-8db5-b476bf1e1ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:42:23.105563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844cce1c-1b8d-4beb-8db5-b476bf1e1ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:42:23.105842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844cce1c-1b8d-4beb-8db5-b476bf1e1ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:42:23.106113 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844cce1c-1b8d-4beb-8db5-b476bf1e1ac4 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:42:23.106780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844cce1c-1b8d-4beb-8db5-b476bf1e1ac4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:42:23.108745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844cce1c-1b8d-4beb-8db5-b476bf1e1ac4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:42:23.130464 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4366/8798] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:42:23 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:42:23.134688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f0709-41bb-43fe-8baa-cd42fc91102f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:42:23.145585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f0709-41bb-43fe-8baa-cd42fc91102f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:42:23.150352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64371f-b2a2-435a-b190-35346c546070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:42:23.150988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64371f-b2a2-435a-b190-35346c546070 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:42:23.151270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64371f-b2a2-435a-b190-35346c546070 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:42:23.152180 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64371f-b2a2-435a-b190-35346c546070 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:42:23.152872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64371f-b2a2-435a-b190-35346c546070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:42:23.154401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64371f-b2a2-435a-b190-35346c546070 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:42:23.164564 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4433/8799] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:42:23.168389 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844cce1c-1b8d-4beb-8db5-b476bf1e1ac4 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:42:23.178244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844cce1c-1b8d-4beb-8db5-b476bf1e1ac4 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:42:23.183674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698adc93-93c4-49bb-98c6-404067108ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:42:23.184259 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698adc93-93c4-49bb-98c6-404067108ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:42:23.184541 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698adc93-93c4-49bb-98c6-404067108ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:42:23.184820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698adc93-93c4-49bb-98c6-404067108ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:42:23.185484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698adc93-93c4-49bb-98c6-404067108ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:42:23.187104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698adc93-93c4-49bb-98c6-404067108ff3 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:42:23.197625 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4367/8800] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:42:23.202606 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64371f-b2a2-435a-b190-35346c546070 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:42:23.213979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64371f-b2a2-435a-b190-35346c546070 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:42:23.218644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534aad4-01de-4647-ba83-920378fc8d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:42:23.219299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534aad4-01de-4647-ba83-920378fc8d6e admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:42:23.219594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534aad4-01de-4647-ba83-920378fc8d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:42:23.219850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534aad4-01de-4647-ba83-920378fc8d6e admin admin] PATH_INFO: `/v3/projects/ae55f903ff94436b8d907d865581cffd/users/87e5b194ad2745b6af1f6ad76dd94881/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:42:23.234904 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534aad4-01de-4647-ba83-920378fc8d6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae55f903ff94436b8d907d865581cffd, user_id=87e5b194ad2745b6af1f6ad76dd94881, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:42:23.237585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534aad4-01de-4647-ba83-920378fc8d6e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:42:23.251218 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4434/8801] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:42:23 2022] PUT /identity/v3/projects/ae55f903ff94436b8d907d865581cffd/users/87e5b194ad2745b6af1f6ad76dd94881/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:42:23.256403 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9720fd-52fc-4315-a4d9-0d5fd0943fc4 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:42:23.256695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9720fd-52fc-4315-a4d9-0d5fd0943fc4 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:42:23.256973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9720fd-52fc-4315-a4d9-0d5fd0943fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:42:23.276132 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-5c9720fd-52fc-4315-a4d9-0d5fd0943fc4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:42:23.277336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-5c9720fd-52fc-4315-a4d9-0d5fd0943fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:42:23.334954 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4368/8802] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:42:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:42:23.344479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534aad4-01de-4647-ba83-920378fc8d6e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:42:23.346137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f534aad4-01de-4647-ba83-920378fc8d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:42:23.360594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:42:23.409333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534aad4-01de-4647-ba83-920378fc8d6e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:42:23.412945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f2df5-caba-4b4a-abe0-2ee425d9f723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:42:23.413583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f2df5-caba-4b4a-abe0-2ee425d9f723 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:42:23.413869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f2df5-caba-4b4a-abe0-2ee425d9f723 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:42:23.414145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f2df5-caba-4b4a-abe0-2ee425d9f723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:42:23.423273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f2df5-caba-4b4a-abe0-2ee425d9f723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:42:23.424701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f2df5-caba-4b4a-abe0-2ee425d9f723 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:42:23.435015 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4435/8803] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:42:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:42:27.404563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e7fe1-eee0-4730-9a7a-f5831fa274e0 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:42:27.405395 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e7fe1-eee0-4730-9a7a-f5831fa274e0 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:42:27.405674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e7fe1-eee0-4730-9a7a-f5831fa274e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:42:27.436553 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a16e7fe1-eee0-4730-9a7a-f5831fa274e0 None None] MFA Rules not processed for user `87e5b194ad2745b6af1f6ad76dd94881`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:42:27.437671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a16e7fe1-eee0-4730-9a7a-f5831fa274e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:42:27.504705 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4369/8804] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:42:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:42:27.519294 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f2df5-caba-4b4a-abe0-2ee425d9f723 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:42:27.520748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-fb8f2df5-caba-4b4a-abe0-2ee425d9f723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:42:27.609635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f2df5-caba-4b4a-abe0-2ee425d9f723 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:42:27.615654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99381ace-ef8c-4b41-8951-ef14672c8b11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:42:27.616286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99381ace-ef8c-4b41-8951-ef14672c8b11 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:42:27.616553 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99381ace-ef8c-4b41-8951-ef14672c8b11 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:42:27.616823 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99381ace-ef8c-4b41-8951-ef14672c8b11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:42:27.625252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99381ace-ef8c-4b41-8951-ef14672c8b11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:42:27.626902 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99381ace-ef8c-4b41-8951-ef14672c8b11 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:42:27.636525 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4436/8805] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:42:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:42:29.013722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441074d-334d-402d-bd60-a86f6b0f1d7e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:42:29.014232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441074d-334d-402d-bd60-a86f6b0f1d7e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:42:29.014318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441074d-334d-402d-bd60-a86f6b0f1d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:42:29.034833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c441074d-334d-402d-bd60-a86f6b0f1d7e None None] MFA Rules not processed for user `87e5b194ad2745b6af1f6ad76dd94881`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:42:29.036050 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c441074d-334d-402d-bd60-a86f6b0f1d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:42:29.046513 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4370/8806] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:42:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:42:37.631693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=54 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:42:39.031791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:43:43.777858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea74f2f-f839-4e42-a6f0-0ea7d5c32eb7 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:43:43.778427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea74f2f-f839-4e42-a6f0-0ea7d5c32eb7 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:43:43.778427 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea74f2f-f839-4e42-a6f0-0ea7d5c32eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:43:43.797827 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-cea74f2f-f839-4e42-a6f0-0ea7d5c32eb7 None None] MFA Rules not processed for user `87e5b194ad2745b6af1f6ad76dd94881`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:43:43.799055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cea74f2f-f839-4e42-a6f0-0ea7d5c32eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:43:43.824483 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4437/8807] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:43:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:43:53.822054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=11 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:44:34.735857 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441074d-334d-402d-bd60-a86f6b0f1d7e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:44:34.757657 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441074d-334d-402d-bd60-a86f6b0f1d7e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:44:34.761322 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3c2e8-b4da-490c-9192-615e09005dc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:44:34.762421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3c2e8-b4da-490c-9192-615e09005dc2 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:44:34.762841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3c2e8-b4da-490c-9192-615e09005dc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:44:34.763122 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3c2e8-b4da-490c-9192-615e09005dc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:44:34.764845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-22c3c2e8-b4da-490c-9192-615e09005dc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:44:34.787852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c3c2e8-b4da-490c-9192-615e09005dc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:44:34.789260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c3c2e8-b4da-490c-9192-615e09005dc2 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:44:34.812365 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4371/8808] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:44:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:44:44.809897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=19 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:44:46.459025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c13cf-1451-462a-871b-378bc1221be9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:44:46.459598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c13cf-1451-462a-871b-378bc1221be9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:44:46.459598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c13cf-1451-462a-871b-378bc1221be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:44:46.479328 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-ce5c13cf-1451-462a-871b-378bc1221be9 None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:44:46.480517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ce5c13cf-1451-462a-871b-378bc1221be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:44:46.545235 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4438/8809] 10.222.0.39 () {58 vars in 1015 bytes} [Sat Oct 1 21:44:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:44:46.557058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfe7df-4c85-4fff-9a27-0e6ff8aeb23d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:44:46.557544 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfe7df-4c85-4fff-9a27-0e6ff8aeb23d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:44:46.557677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfe7df-4c85-4fff-9a27-0e6ff8aeb23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:44:46.578430 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bbdfe7df-4c85-4fff-9a27-0e6ff8aeb23d None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:44:46.579775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bbdfe7df-4c85-4fff-9a27-0e6ff8aeb23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:44:46.588350 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4372/8810] 10.222.0.17 () {58 vars in 1058 bytes} [Sat Oct 1 21:44:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:44:46.594345 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c13cf-1451-462a-871b-378bc1221be9 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:44:46.605124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c13cf-1451-462a-871b-378bc1221be9 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:44:46.610184 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab07b8-a1dc-4a94-90d1-d4dbfa0efa53 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:44:46.610816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab07b8-a1dc-4a94-90d1-d4dbfa0efa53 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:44:46.611101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab07b8-a1dc-4a94-90d1-d4dbfa0efa53 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:44:46.611374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab07b8-a1dc-4a94-90d1-d4dbfa0efa53 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:44:46.619938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab07b8-a1dc-4a94-90d1-d4dbfa0efa53 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:44:46.622086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab07b8-a1dc-4a94-90d1-d4dbfa0efa53 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:44:46.631938 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4439/8811] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:44:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:44:54.275872 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11807d3-6d45-4be8-9ef2-4f5ae28626da None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:44:54.276351 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11807d3-6d45-4be8-9ef2-4f5ae28626da None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:44:54.276485 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11807d3-6d45-4be8-9ef2-4f5ae28626da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:44:54.306345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d11807d3-6d45-4be8-9ef2-4f5ae28626da None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:44:54.307619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d11807d3-6d45-4be8-9ef2-4f5ae28626da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:44:54.317262 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4373/8812] 10.222.0.27 () {58 vars in 1015 bytes} [Sat Oct 1 21:44:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:44:54.330604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab07b8-a1dc-4a94-90d1-d4dbfa0efa53 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:44:54.342012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab07b8-a1dc-4a94-90d1-d4dbfa0efa53 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:44:54.347589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa294e-24b9-42d4-8138-7bae903460d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:44:54.348346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa294e-24b9-42d4-8138-7bae903460d1 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:44:54.348778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa294e-24b9-42d4-8138-7bae903460d1 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:44:54.349127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa294e-24b9-42d4-8138-7bae903460d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:44:54.359098 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa294e-24b9-42d4-8138-7bae903460d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:44:54.360691 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa294e-24b9-42d4-8138-7bae903460d1 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:44:54.372235 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4440/8813] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:44:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:44:59.058091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1625af-d34e-453a-8f96-275204e48d36 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:44:59.058645 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1625af-d34e-453a-8f96-275204e48d36 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:44:59.058800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1625af-d34e-453a-8f96-275204e48d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:44:59.077380 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6a1625af-d34e-453a-8f96-275204e48d36 None None] MFA Rules not processed for user `87e5b194ad2745b6af1f6ad76dd94881`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:44:59.078612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6a1625af-d34e-453a-8f96-275204e48d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:44:59.085461 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4374/8814] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:44:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:45:04.366674 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=41 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:45:09.071972 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=9 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:45:44.695022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa294e-24b9-42d4-8138-7bae903460d1 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:45:44.696666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d0fa294e-24b9-42d4-8138-7bae903460d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:45:44.709133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa294e-24b9-42d4-8138-7bae903460d1 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:45:44.712842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad481168-f319-4e35-bfd4-602b93b51f2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:45:44.713446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad481168-f319-4e35-bfd4-602b93b51f2a service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:45:44.713728 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad481168-f319-4e35-bfd4-602b93b51f2a service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:45:44.714000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad481168-f319-4e35-bfd4-602b93b51f2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:45:44.715855 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ad481168-f319-4e35-bfd4-602b93b51f2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:45:44.733635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad481168-f319-4e35-bfd4-602b93b51f2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:45:44.735118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad481168-f319-4e35-bfd4-602b93b51f2a service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:45:44.749780 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4441/8815] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:45:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:45:45.784174 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1625af-d34e-453a-8f96-275204e48d36 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:45:45.793554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1625af-d34e-453a-8f96-275204e48d36 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:45:45.797714 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b729d92-4e79-4246-8fee-71c207428343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:45:45.798303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b729d92-4e79-4246-8fee-71c207428343 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:45:45.798638 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b729d92-4e79-4246-8fee-71c207428343 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:45:45.798914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b729d92-4e79-4246-8fee-71c207428343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:45:45.800612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8b729d92-4e79-4246-8fee-71c207428343 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:45:45.812669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b729d92-4e79-4246-8fee-71c207428343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:45:45.814081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b729d92-4e79-4246-8fee-71c207428343 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:45:45.824426 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4375/8816] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:45:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:45:54.741426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:45:55.818406 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:46:03.364380 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad481168-f319-4e35-bfd4-602b93b51f2a service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:46:03.373032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad481168-f319-4e35-bfd4-602b93b51f2a service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:46:03.376697 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023d7c5-3af8-4304-8192-436d3100c76d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:46:03.377290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023d7c5-3af8-4304-8192-436d3100c76d service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:46:03.377572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023d7c5-3af8-4304-8192-436d3100c76d service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:46:03.377874 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023d7c5-3af8-4304-8192-436d3100c76d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:46:03.379614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9023d7c5-3af8-4304-8192-436d3100c76d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:46:03.392078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023d7c5-3af8-4304-8192-436d3100c76d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:46:03.393488 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023d7c5-3af8-4304-8192-436d3100c76d service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:46:03.405118 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4442/8817] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:46:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:46:11.124803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b729d92-4e79-4246-8fee-71c207428343 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:46:11.141992 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b729d92-4e79-4246-8fee-71c207428343 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:46:11.148195 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ebe75-a9c6-4274-8611-44db1960168c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:46:11.148817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ebe75-a9c6-4274-8611-44db1960168c service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:46:11.149091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ebe75-a9c6-4274-8611-44db1960168c service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:46:11.149364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ebe75-a9c6-4274-8611-44db1960168c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:46:11.151131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ec2ebe75-a9c6-4274-8611-44db1960168c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:46:11.166536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ebe75-a9c6-4274-8611-44db1960168c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:46:11.168125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ebe75-a9c6-4274-8611-44db1960168c service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:46:11.182515 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4376/8818] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:46:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:46:11.275527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023d7c5-3af8-4304-8192-436d3100c76d service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:46:11.303583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023d7c5-3af8-4304-8192-436d3100c76d service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:46:11.313631 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93378c90-7f05-4fbf-b12a-e8d793b5fb40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:46:11.315056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93378c90-7f05-4fbf-b12a-e8d793b5fb40 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:46:11.315554 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93378c90-7f05-4fbf-b12a-e8d793b5fb40 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:46:11.316055 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93378c90-7f05-4fbf-b12a-e8d793b5fb40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:46:11.318091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-93378c90-7f05-4fbf-b12a-e8d793b5fb40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:46:11.347845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93378c90-7f05-4fbf-b12a-e8d793b5fb40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:46:11.349825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93378c90-7f05-4fbf-b12a-e8d793b5fb40 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:46:11.366174 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4443/8819] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:46:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:46:17.872897 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6e590-a2e7-4bcb-bf2d-4773077facdd None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:46:17.873388 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6e590-a2e7-4bcb-bf2d-4773077facdd None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:46:17.873500 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6e590-a2e7-4bcb-bf2d-4773077facdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:46:17.892345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-edb6e590-a2e7-4bcb-bf2d-4773077facdd None None] MFA Rules not processed for user `87e5b194ad2745b6af1f6ad76dd94881`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:46:17.893558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-edb6e590-a2e7-4bcb-bf2d-4773077facdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:46:17.900452 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4377/8820] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:46:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:46:18.526000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93378c90-7f05-4fbf-b12a-e8d793b5fb40 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:46:18.537752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93378c90-7f05-4fbf-b12a-e8d793b5fb40 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:46:18.542909 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb192b-cafa-4502-897d-44cac090e61f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:46:18.543731 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb192b-cafa-4502-897d-44cac090e61f service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:46:18.544021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb192b-cafa-4502-897d-44cac090e61f service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:46:18.544303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb192b-cafa-4502-897d-44cac090e61f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:46:18.546107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c1cb192b-cafa-4502-897d-44cac090e61f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:46:18.558259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb192b-cafa-4502-897d-44cac090e61f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:46:18.560367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb192b-cafa-4502-897d-44cac090e61f service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:46:18.571384 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4444/8821] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:46:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:46:27.887343 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=9 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:46:28.566898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=18 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:47:02.214773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded41f29-d69f-488e-bb22-8216f856d72f None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:02.215345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded41f29-d69f-488e-bb22-8216f856d72f None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:02.215345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded41f29-d69f-488e-bb22-8216f856d72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:02.254686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ded41f29-d69f-488e-bb22-8216f856d72f None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:47:02.255913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ded41f29-d69f-488e-bb22-8216f856d72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:02.265294 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4378/8822] 10.222.0.17 () {58 vars in 1058 bytes} [Sat Oct 1 21:47:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:47:02.271382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4842bda-f4ea-4fac-8268-b19d0b4fefe9 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:02.271865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4842bda-f4ea-4fac-8268-b19d0b4fefe9 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:02.271970 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4842bda-f4ea-4fac-8268-b19d0b4fefe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:02.291214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e4842bda-f4ea-4fac-8268-b19d0b4fefe9 None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:47:02.292404 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e4842bda-f4ea-4fac-8268-b19d0b4fefe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:02.302251 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4445/8823] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:47:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:47:02.313208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded41f29-d69f-488e-bb22-8216f856d72f None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:02.323039 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded41f29-d69f-488e-bb22-8216f856d72f None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:02.326641 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a8c8b3-2e61-488a-b8ce-7769ee37e31a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:02.327231 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8c8b3-2e61-488a-b8ce-7769ee37e31a service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:02.327507 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8c8b3-2e61-488a-b8ce-7769ee37e31a service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:02.327775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8c8b3-2e61-488a-b8ce-7769ee37e31a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:02.337602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a8c8b3-2e61-488a-b8ce-7769ee37e31a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:02.339058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a8c8b3-2e61-488a-b8ce-7769ee37e31a service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:02.349446 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4379/8824] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:47:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:47:02.523926 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3f4e5-ed5d-476b-ad16-e66024f19b17 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:02.524214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3f4e5-ed5d-476b-ad16-e66024f19b17 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:02.524490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3f4e5-ed5d-476b-ad16-e66024f19b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:02.544210 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c9c3f4e5-ed5d-476b-ad16-e66024f19b17 None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:47:02.545397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c9c3f4e5-ed5d-476b-ad16-e66024f19b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:02.555620 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4446/8825] 10.222.0.17 () {58 vars in 1058 bytes} [Sat Oct 1 21:47:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:47:02.561341 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7601feb-481b-45ba-bc34-ddb7fb7f0534 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:02.561636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7601feb-481b-45ba-bc34-ddb7fb7f0534 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:02.561913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7601feb-481b-45ba-bc34-ddb7fb7f0534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:02.581147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d7601feb-481b-45ba-bc34-ddb7fb7f0534 None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:47:02.582324 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d7601feb-481b-45ba-bc34-ddb7fb7f0534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:02.591506 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4380/8826] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:47:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:47:02.603176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd415328-3236-4626-9a51-33e2c9f478f2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:02.603465 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd415328-3236-4626-9a51-33e2c9f478f2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:02.603739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd415328-3236-4626-9a51-33e2c9f478f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:02.652677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-bd415328-3236-4626-9a51-33e2c9f478f2 None None] MFA Rules not processed for user `197a0ae6b71f4635a069ac0238e017c9`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:47:02.653865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bd415328-3236-4626-9a51-33e2c9f478f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:02.715012 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4447/8827] 10.222.0.17 () {58 vars in 1060 bytes} [Sat Oct 1 21:47:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:47:02.720863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7601feb-481b-45ba-bc34-ddb7fb7f0534 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:02.729751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7601feb-481b-45ba-bc34-ddb7fb7f0534 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:02.733331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c6cc2-3fb4-45dd-ae7b-61e73d2a59ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '197a0ae6b71f4635a069ac0238e017c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:02.733921 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c6cc2-3fb4-45dd-ae7b-61e73d2a59ed service cinder] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:02.734199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c6cc2-3fb4-45dd-ae7b-61e73d2a59ed service cinder] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:02.734518 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c6cc2-3fb4-45dd-ae7b-61e73d2a59ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:02.744405 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c6cc2-3fb4-45dd-ae7b-61e73d2a59ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:02.745820 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c6cc2-3fb4-45dd-ae7b-61e73d2a59ed service cinder] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:02.756684 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4381/8828] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:47:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:47:06.323307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd415328-3236-4626-9a51-33e2c9f478f2 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:06.332708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd415328-3236-4626-9a51-33e2c9f478f2 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:06.336342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae1958-4d90-4397-92a7-2ac56f94c955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:06.336936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae1958-4d90-4397-92a7-2ac56f94c955 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:06.337216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae1958-4d90-4397-92a7-2ac56f94c955 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:06.337495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae1958-4d90-4397-92a7-2ac56f94c955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:06.339229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-20ae1958-4d90-4397-92a7-2ac56f94c955 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:06.351539 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae1958-4d90-4397-92a7-2ac56f94c955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:06.352949 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae1958-4d90-4397-92a7-2ac56f94c955 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:06.363966 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4448/8829] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:47:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:47:06.499189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c6cc2-3fb4-45dd-ae7b-61e73d2a59ed service cinder] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:06.508450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c6cc2-3fb4-45dd-ae7b-61e73d2a59ed service cinder] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:06.511969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91f344-0014-4ea8-bb83-0edcfdc03a24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:06.512558 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91f344-0014-4ea8-bb83-0edcfdc03a24 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:06.512831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91f344-0014-4ea8-bb83-0edcfdc03a24 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:06.513102 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91f344-0014-4ea8-bb83-0edcfdc03a24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:06.514849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7c91f344-0014-4ea8-bb83-0edcfdc03a24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:06.527340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c91f344-0014-4ea8-bb83-0edcfdc03a24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:06.528990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c91f344-0014-4ea8-bb83-0edcfdc03a24 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:06.540733 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4382/8830] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:47:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:47:06.619790 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd4369-5179-484b-b4be-0f5e19009ef2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:06.620083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd4369-5179-484b-b4be-0f5e19009ef2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:06.620439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd4369-5179-484b-b4be-0f5e19009ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:06.648686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-52dd4369-5179-484b-b4be-0f5e19009ef2 None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:47:06.649875 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-52dd4369-5179-484b-b4be-0f5e19009ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:06.659403 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4449/8831] 10.222.0.17 () {58 vars in 1058 bytes} [Sat Oct 1 21:47:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:47:06.665631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91f344-0014-4ea8-bb83-0edcfdc03a24 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:06.677619 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91f344-0014-4ea8-bb83-0edcfdc03a24 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:06.683021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62579990-1621-4530-bb45-3006e111155f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:06.683616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62579990-1621-4530-bb45-3006e111155f service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:06.683900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62579990-1621-4530-bb45-3006e111155f service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:06.684176 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62579990-1621-4530-bb45-3006e111155f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:06.685898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-62579990-1621-4530-bb45-3006e111155f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:06.697123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62579990-1621-4530-bb45-3006e111155f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:06.698738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62579990-1621-4530-bb45-3006e111155f service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:06.708915 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4383/8832] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:47:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:47:06.881438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd4369-5179-484b-b4be-0f5e19009ef2 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:06.890996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd4369-5179-484b-b4be-0f5e19009ef2 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:06.895077 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3eef2-3695-44e8-85bb-e26c677269b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:06.895672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3eef2-3695-44e8-85bb-e26c677269b0 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:06.895952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3eef2-3695-44e8-85bb-e26c677269b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:06.896226 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3eef2-3695-44e8-85bb-e26c677269b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:06.897931 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-20c3eef2-3695-44e8-85bb-e26c677269b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:06.914397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3eef2-3695-44e8-85bb-e26c677269b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:06.915861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3eef2-3695-44e8-85bb-e26c677269b0 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:06.929792 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4450/8833] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:47:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:47:06.990561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62579990-1621-4530-bb45-3006e111155f service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:07.000839 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62579990-1621-4530-bb45-3006e111155f service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:07.005968 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c70523-e1fd-4569-9d3f-75208e782ac8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:07.006587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c70523-e1fd-4569-9d3f-75208e782ac8 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:07.006870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c70523-e1fd-4569-9d3f-75208e782ac8 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:07.007144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c70523-e1fd-4569-9d3f-75208e782ac8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:07.008870 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f3c70523-e1fd-4569-9d3f-75208e782ac8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:07.020289 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c70523-e1fd-4569-9d3f-75208e782ac8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:07.021861 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c70523-e1fd-4569-9d3f-75208e782ac8 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:07.031662 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4384/8834] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:47:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:47:07.842045 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f0c50-52ba-4ce9-aa30-7bb613105210 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:07.842640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f0c50-52ba-4ce9-aa30-7bb613105210 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:07.842640 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f0c50-52ba-4ce9-aa30-7bb613105210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:07.863996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b53f0c50-52ba-4ce9-aa30-7bb613105210 None None] MFA Rules not processed for user `e17a026f305248878f18eec35e1e02dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:47:07.865215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b53f0c50-52ba-4ce9-aa30-7bb613105210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:07.871760 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4451/8835] 10.222.0.27 () {58 vars in 1015 bytes} [Sat Oct 1 21:47:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:47:07.884838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c70523-e1fd-4569-9d3f-75208e782ac8 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:07.894315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c70523-e1fd-4569-9d3f-75208e782ac8 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:07.898263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070fed7-8e16-46c8-970a-3ee59a62da8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:07.898905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070fed7-8e16-46c8-970a-3ee59a62da8d service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:07.899193 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070fed7-8e16-46c8-970a-3ee59a62da8d service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:07.899467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070fed7-8e16-46c8-970a-3ee59a62da8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:07.906704 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070fed7-8e16-46c8-970a-3ee59a62da8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:07.908133 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070fed7-8e16-46c8-970a-3ee59a62da8d service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:07.916420 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4385/8836] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:47:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:47:17.858391 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=62 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:47:17.912948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=48 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:47:26.348152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a36f75-435f-4df3-8a6e-b79fe4bbfb0a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:26.348696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a36f75-435f-4df3-8a6e-b79fe4bbfb0a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:26.348696 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a36f75-435f-4df3-8a6e-b79fe4bbfb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:26.378734 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-66a36f75-435f-4df3-8a6e-b79fe4bbfb0a None None] MFA Rules not processed for user `e17a026f305248878f18eec35e1e02dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:47:26.379937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-66a36f75-435f-4df3-8a6e-b79fe4bbfb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:26.402374 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4452/8837] 10.222.0.39 () {58 vars in 1015 bytes} [Sat Oct 1 21:47:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:47:26.413852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070fed7-8e16-46c8-970a-3ee59a62da8d service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:26.429842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070fed7-8e16-46c8-970a-3ee59a62da8d service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:26.433604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af117cc-47c3-4b06-b7d6-3040271852f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:26.434196 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af117cc-47c3-4b06-b7d6-3040271852f6 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:26.434479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af117cc-47c3-4b06-b7d6-3040271852f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:26.434783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af117cc-47c3-4b06-b7d6-3040271852f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:26.442396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af117cc-47c3-4b06-b7d6-3040271852f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:26.443846 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af117cc-47c3-4b06-b7d6-3040271852f6 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:26.451499 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4386/8838] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:47:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:47:26.914676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993f3fd-962b-48da-b62d-d0c3a645213b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:26.914948 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993f3fd-962b-48da-b62d-d0c3a645213b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:26.915305 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993f3fd-962b-48da-b62d-d0c3a645213b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:26.937070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9993f3fd-962b-48da-b62d-d0c3a645213b None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:47:26.938322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9993f3fd-962b-48da-b62d-d0c3a645213b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:26.948399 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4453/8839] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 21:47:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:47:26.966065 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af117cc-47c3-4b06-b7d6-3040271852f6 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:26.979121 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af117cc-47c3-4b06-b7d6-3040271852f6 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:26.984675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69861cd8-c8e4-42aa-b45e-d5a580834305 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:26.985455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69861cd8-c8e4-42aa-b45e-d5a580834305 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:26.985735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69861cd8-c8e4-42aa-b45e-d5a580834305 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:26.986549 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69861cd8-c8e4-42aa-b45e-d5a580834305 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:26.995888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69861cd8-c8e4-42aa-b45e-d5a580834305 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:26.997479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69861cd8-c8e4-42aa-b45e-d5a580834305 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:27.008728 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4387/8840] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:47:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:47:27.788375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993f3fd-962b-48da-b62d-d0c3a645213b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:27.800358 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993f3fd-962b-48da-b62d-d0c3a645213b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:27.805595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120257c0-b50f-459e-b6fb-4df855a2dfcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:27.806207 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120257c0-b50f-459e-b6fb-4df855a2dfcd service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:27.806534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120257c0-b50f-459e-b6fb-4df855a2dfcd service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:27.806811 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120257c0-b50f-459e-b6fb-4df855a2dfcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:27.814907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120257c0-b50f-459e-b6fb-4df855a2dfcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:27.816463 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120257c0-b50f-459e-b6fb-4df855a2dfcd service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:27.825703 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4454/8841] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:47:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:47:34.943646 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d174afe-5bf6-4890-8270-3f35ac218962 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:34.944199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d174afe-5bf6-4890-8270-3f35ac218962 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:34.944199 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d174afe-5bf6-4890-8270-3f35ac218962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:34.969112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3d174afe-5bf6-4890-8270-3f35ac218962 None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:47:34.970443 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3d174afe-5bf6-4890-8270-3f35ac218962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:34.981548 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4388/8842] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 21:47:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:47:34.997994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120257c0-b50f-459e-b6fb-4df855a2dfcd service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:35.011852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120257c0-b50f-459e-b6fb-4df855a2dfcd service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:35.019986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13662ed3-8eee-4cfd-bbfc-b4b5110b9b9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:35.020859 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13662ed3-8eee-4cfd-bbfc-b4b5110b9b9c service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:35.021143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13662ed3-8eee-4cfd-bbfc-b4b5110b9b9c service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:35.021419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13662ed3-8eee-4cfd-bbfc-b4b5110b9b9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:35.032137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13662ed3-8eee-4cfd-bbfc-b4b5110b9b9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:35.033720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13662ed3-8eee-4cfd-bbfc-b4b5110b9b9c service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:35.044815 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4455/8843] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:47:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:47:36.749182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444b52d-ebb9-49b7-b149-088737908b2b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:36.749634 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444b52d-ebb9-49b7-b149-088737908b2b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:36.749828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444b52d-ebb9-49b7-b149-088737908b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:36.773264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3444b52d-ebb9-49b7-b149-088737908b2b None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:47:36.775197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3444b52d-ebb9-49b7-b149-088737908b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:36.789220 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4389/8844] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 21:47:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:47:36.819133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13662ed3-8eee-4cfd-bbfc-b4b5110b9b9c service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:36.834235 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13662ed3-8eee-4cfd-bbfc-b4b5110b9b9c service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:36.840767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe7dd3-bb66-43f0-928a-3bfbf6a1861c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:36.841385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe7dd3-bb66-43f0-928a-3bfbf6a1861c service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:36.841666 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe7dd3-bb66-43f0-928a-3bfbf6a1861c service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:36.841945 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe7dd3-bb66-43f0-928a-3bfbf6a1861c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:36.851984 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe7dd3-bb66-43f0-928a-3bfbf6a1861c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:36.853581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe7dd3-bb66-43f0-928a-3bfbf6a1861c service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:36.870057 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4456/8845] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:47:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:47:46.765317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=21 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:47:46.860260 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=35 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:47:47.939377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444b52d-ebb9-49b7-b149-088737908b2b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:47.948623 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444b52d-ebb9-49b7-b149-088737908b2b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:47.952189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd05065-e7bb-4fd1-aa26-81043ce7f9da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:47.952784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd05065-e7bb-4fd1-aa26-81043ce7f9da service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:47.953062 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd05065-e7bb-4fd1-aa26-81043ce7f9da service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:47.953334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd05065-e7bb-4fd1-aa26-81043ce7f9da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:47.962644 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd05065-e7bb-4fd1-aa26-81043ce7f9da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:47.964095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd05065-e7bb-4fd1-aa26-81043ce7f9da service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:47.983723 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4390/8846] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:47:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:47:51.104605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe7dd3-bb66-43f0-928a-3bfbf6a1861c service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:51.121262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe7dd3-bb66-43f0-928a-3bfbf6a1861c service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:51.131147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5846c-9b20-4100-9d5e-ceb736b5d2d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:51.132111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5846c-9b20-4100-9d5e-ceb736b5d2d9 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:51.134876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5846c-9b20-4100-9d5e-ceb736b5d2d9 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:51.135348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5846c-9b20-4100-9d5e-ceb736b5d2d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:51.139051 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-74c5846c-9b20-4100-9d5e-ceb736b5d2d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:51.186802 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c5846c-9b20-4100-9d5e-ceb736b5d2d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:51.189868 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c5846c-9b20-4100-9d5e-ceb736b5d2d9 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:51.201309 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd05065-e7bb-4fd1-aa26-81043ce7f9da service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:51.212302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd05065-e7bb-4fd1-aa26-81043ce7f9da service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:51.223003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb1fc5-d6fe-40e8-8407-377987b82866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:51.224016 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4457/8847] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:47:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:47:51.224664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb1fc5-d6fe-40e8-8407-377987b82866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:51.224939 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb1fc5-d6fe-40e8-8407-377987b82866 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:51.225215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb1fc5-d6fe-40e8-8407-377987b82866 admin admin] PATH_INFO: `/v3/users/87e5b194ad2745b6af1f6ad76dd94881` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:51.226540 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb1fc5-d6fe-40e8-8407-377987b82866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87e5b194ad2745b6af1f6ad76dd94881)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:51.228543 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb1fc5-d6fe-40e8-8407-377987b82866 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:51.283304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-97bb1fc5-d6fe-40e8-8407-377987b82866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87e5b194ad2745b6af1f6ad76dd94881'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:47:51.283989 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-97bb1fc5-d6fe-40e8-8407-377987b82866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87e5b194ad2745b6af1f6ad76dd94881'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:47:51.284618 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-97bb1fc5-d6fe-40e8-8407-377987b82866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87e5b194ad2745b6af1f6ad76dd94881'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:47:51.291181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-97bb1fc5-d6fe-40e8-8407-377987b82866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87e5b194ad2745b6af1f6ad76dd94881'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:47:51.303445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-97bb1fc5-d6fe-40e8-8407-377987b82866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87e5b194ad2745b6af1f6ad76dd94881'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:47:51.316445 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4391/8848] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:47:51 2022] DELETE /identity/v3/users/87e5b194ad2745b6af1f6ad76dd94881 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:47:51.488154 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5846c-9b20-4100-9d5e-ceb736b5d2d9 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:51.505094 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5846c-9b20-4100-9d5e-ceb736b5d2d9 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:51.562319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05240531-5997-4042-a01f-79870aa6f235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:51.562979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05240531-5997-4042-a01f-79870aa6f235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:51.563259 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05240531-5997-4042-a01f-79870aa6f235 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:51.563530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05240531-5997-4042-a01f-79870aa6f235 admin admin] PATH_INFO: `/v3/projects/ae55f903ff94436b8d907d865581cffd` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:51.564641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05240531-5997-4042-a01f-79870aa6f235 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae55f903ff94436b8d907d865581cffd)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:51.567041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05240531-5997-4042-a01f-79870aa6f235 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:51.613433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05240531-5997-4042-a01f-79870aa6f235 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae55f903ff94436b8d907d865581cffd', 'request_id': 'req-05240531-5997-4042-a01f-79870aa6f235'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:47:51.619531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05240531-5997-4042-a01f-79870aa6f235 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae55f903ff94436b8d907d865581cffd', 'request_id': 'req-05240531-5997-4042-a01f-79870aa6f235'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:47:51.625066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05240531-5997-4042-a01f-79870aa6f235 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae55f903ff94436b8d907d865581cffd', 'request_id': 'req-05240531-5997-4042-a01f-79870aa6f235'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:47:51.637573 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05240531-5997-4042-a01f-79870aa6f235 admin admin] The token cache is being invalidate because project ae55f903ff94436b8d907d865581cffd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:47:51.637856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05240531-5997-4042-a01f-79870aa6f235 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:47:51.638654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-05240531-5997-4042-a01f-79870aa6f235 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:47:51.640148 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4458/8849] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:47:51 2022] DELETE /identity/v3/projects/ae55f903ff94436b8d907d865581cffd => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:47:51.645089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5d7b9-ae20-4a1e-a770-66a5ab0fe9af None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:51.645407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5d7b9-ae20-4a1e-a770-66a5ab0fe9af None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:51.645686 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5d7b9-ae20-4a1e-a770-66a5ab0fe9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:51.665525 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-bbf5d7b9-ae20-4a1e-a770-66a5ab0fe9af None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:47:51.666776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bbf5d7b9-ae20-4a1e-a770-66a5ab0fe9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:51.724987 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4392/8850] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:47:51.732950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eab1ed-897f-4ba9-b7f3-bbd2357df259 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:51.733240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eab1ed-897f-4ba9-b7f3-bbd2357df259 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:51.733544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eab1ed-897f-4ba9-b7f3-bbd2357df259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:51.757603 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-59eab1ed-897f-4ba9-b7f3-bbd2357df259 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:47:51.758860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-59eab1ed-897f-4ba9-b7f3-bbd2357df259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:51.766895 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4459/8851] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:47:51.771236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5d7b9-ae20-4a1e-a770-66a5ab0fe9af None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:51.780916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5d7b9-ae20-4a1e-a770-66a5ab0fe9af None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:51.785573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927cb164-e2e6-4f64-b423-b521ffb028b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:51.786169 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cb164-e2e6-4f64-b423-b521ffb028b0 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:51.786451 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cb164-e2e6-4f64-b423-b521ffb028b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:51.786765 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cb164-e2e6-4f64-b423-b521ffb028b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:51.787407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927cb164-e2e6-4f64-b423-b521ffb028b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:47:51.787712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927cb164-e2e6-4f64-b423-b521ffb028b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:51.789315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927cb164-e2e6-4f64-b423-b521ffb028b0 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:51.801487 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4393/8852] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:47:51.806054 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eab1ed-897f-4ba9-b7f3-bbd2357df259 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:51.816158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eab1ed-897f-4ba9-b7f3-bbd2357df259 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:51.821164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0420a80-54c3-40d4-9476-ea1c44ad9d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:51.821772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0420a80-54c3-40d4-9476-ea1c44ad9d47 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:51.822063 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0420a80-54c3-40d4-9476-ea1c44ad9d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:51.822338 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0420a80-54c3-40d4-9476-ea1c44ad9d47 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:51.823032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0420a80-54c3-40d4-9476-ea1c44ad9d47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:51.824941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0420a80-54c3-40d4-9476-ea1c44ad9d47 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:51.862161 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4460/8853] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:47:51 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:47:51.866664 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927cb164-e2e6-4f64-b423-b521ffb028b0 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:51.876534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927cb164-e2e6-4f64-b423-b521ffb028b0 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:51.909214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:47:51.934412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a658497-c0bb-454b-9ef1-622f88736c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:51.935053 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a658497-c0bb-454b-9ef1-622f88736c20 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:51.935332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a658497-c0bb-454b-9ef1-622f88736c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:51.935600 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a658497-c0bb-454b-9ef1-622f88736c20 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:51.936209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a658497-c0bb-454b-9ef1-622f88736c20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:51.938112 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a658497-c0bb-454b-9ef1-622f88736c20 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:51.962502 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4394/8854] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:47:51 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:47:51.963776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0420a80-54c3-40d4-9476-ea1c44ad9d47 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:51.974406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0420a80-54c3-40d4-9476-ea1c44ad9d47 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:51.979149 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90dff2-af90-4d5c-9e40-53c51fbe5468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:51.979747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90dff2-af90-4d5c-9e40-53c51fbe5468 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:51.980027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90dff2-af90-4d5c-9e40-53c51fbe5468 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:51.980300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90dff2-af90-4d5c-9e40-53c51fbe5468 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:51.980953 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90dff2-af90-4d5c-9e40-53c51fbe5468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:51.982565 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90dff2-af90-4d5c-9e40-53c51fbe5468 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:51.994528 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4461/8855] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:47:51.998620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a658497-c0bb-454b-9ef1-622f88736c20 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:52.009254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a658497-c0bb-454b-9ef1-622f88736c20 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:52.013837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124ca07-df64-458e-b794-41ccf8d0a200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:52.014424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124ca07-df64-458e-b794-41ccf8d0a200 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:52.014735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124ca07-df64-458e-b794-41ccf8d0a200 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:52.015009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124ca07-df64-458e-b794-41ccf8d0a200 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:52.015632 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124ca07-df64-458e-b794-41ccf8d0a200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:52.017209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124ca07-df64-458e-b794-41ccf8d0a200 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:52.027290 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4395/8856] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:47:52.031445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90dff2-af90-4d5c-9e40-53c51fbe5468 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:52.042326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90dff2-af90-4d5c-9e40-53c51fbe5468 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:52.047159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632a13d-6229-4d5e-b4f0-280ec13a1e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:52.047824 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632a13d-6229-4d5e-b4f0-280ec13a1e28 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:52.048138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632a13d-6229-4d5e-b4f0-280ec13a1e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:52.048414 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632a13d-6229-4d5e-b4f0-280ec13a1e28 admin admin] PATH_INFO: `/v3/projects/486d273a605a489cb5acd91ef693ec38/users/309ad5bc59d84e409cc40374b070b2b8/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:52.064526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632a13d-6229-4d5e-b4f0-280ec13a1e28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=486d273a605a489cb5acd91ef693ec38, user_id=309ad5bc59d84e409cc40374b070b2b8, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:52.067348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632a13d-6229-4d5e-b4f0-280ec13a1e28 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:52.069526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:47:52.082213 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4462/8857] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:47:52 2022] PUT /identity/v3/projects/486d273a605a489cb5acd91ef693ec38/users/309ad5bc59d84e409cc40374b070b2b8/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:47:52.087333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a744827-db41-47ae-8142-c004599c06eb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:52.087621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a744827-db41-47ae-8142-c004599c06eb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:52.087896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a744827-db41-47ae-8142-c004599c06eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:52.107730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8a744827-db41-47ae-8142-c004599c06eb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:47:52.108969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8a744827-db41-47ae-8142-c004599c06eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:52.168581 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4396/8858] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:47:52.178814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632a13d-6229-4d5e-b4f0-280ec13a1e28 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:52.180813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d632a13d-6229-4d5e-b4f0-280ec13a1e28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:52.237474 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632a13d-6229-4d5e-b4f0-280ec13a1e28 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:52.241257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3347b3c-20cb-4948-81dc-b4d48b287bbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:52.241877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3347b3c-20cb-4948-81dc-b4d48b287bbd service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:52.242162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3347b3c-20cb-4948-81dc-b4d48b287bbd service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:52.242442 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3347b3c-20cb-4948-81dc-b4d48b287bbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:52.251710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3347b3c-20cb-4948-81dc-b4d48b287bbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:52.253151 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3347b3c-20cb-4948-81dc-b4d48b287bbd service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:52.262734 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4463/8859] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:47:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:47:56.344464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406f4be-1e90-4f11-a500-8805c792fa34 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:56.344983 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406f4be-1e90-4f11-a500-8805c792fa34 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:56.345103 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406f4be-1e90-4f11-a500-8805c792fa34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:56.375336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-4406f4be-1e90-4f11-a500-8805c792fa34 None None] MFA Rules not processed for user `309ad5bc59d84e409cc40374b070b2b8`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:47:56.376571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4406f4be-1e90-4f11-a500-8805c792fa34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:56.454870 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4397/8860] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:47:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:47:56.465478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cceccd-ef82-4b39-8f20-2db55cbabee2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:56.466034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cceccd-ef82-4b39-8f20-2db55cbabee2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:56.466034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cceccd-ef82-4b39-8f20-2db55cbabee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:56.486888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-60cceccd-ef82-4b39-8f20-2db55cbabee2 None None] MFA Rules not processed for user `309ad5bc59d84e409cc40374b070b2b8`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:47:56.488198 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-60cceccd-ef82-4b39-8f20-2db55cbabee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:56.495415 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4464/8861] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:47:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:47:56.507808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406f4be-1e90-4f11-a500-8805c792fa34 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:47:56.510019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-4406f4be-1e90-4f11-a500-8805c792fa34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:47:56.566119 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:47:56.593350 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406f4be-1e90-4f11-a500-8805c792fa34 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:47:56.598933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2592d-d7f5-44de-bee3-c24039101bb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:47:56.599537 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2592d-d7f5-44de-bee3-c24039101bb8 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:47:56.599817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2592d-d7f5-44de-bee3-c24039101bb8 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:47:56.600092 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2592d-d7f5-44de-bee3-c24039101bb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:47:56.607725 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2592d-d7f5-44de-bee3-c24039101bb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:47:56.609301 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2592d-d7f5-44de-bee3-c24039101bb8 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:47:56.617936 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4398/8862] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:47:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:48:06.480692 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=1,keystone:SELECT=31 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:48:06.613952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=13 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:48:06.789690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d11580-fd57-4472-ab08-33e6d8f79a12 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:06.789997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d11580-fd57-4472-ab08-33e6d8f79a12 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:06.790275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d11580-fd57-4472-ab08-33e6d8f79a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:06.811006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-08d11580-fd57-4472-ab08-33e6d8f79a12 None None] MFA Rules not processed for user `309ad5bc59d84e409cc40374b070b2b8`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:48:06.812243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-08d11580-fd57-4472-ab08-33e6d8f79a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:06.819679 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4465/8863] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:48:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:48:07.063659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994bf6f-b1fb-4783-a6e8-e0bf3799a3a1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:07.063944 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994bf6f-b1fb-4783-a6e8-e0bf3799a3a1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:07.064219 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994bf6f-b1fb-4783-a6e8-e0bf3799a3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:07.083694 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8994bf6f-b1fb-4783-a6e8-e0bf3799a3a1 None None] MFA Rules not processed for user `e17a026f305248878f18eec35e1e02dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:48:07.084895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8994bf6f-b1fb-4783-a6e8-e0bf3799a3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:07.091301 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4399/8864] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:48:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:48:07.103661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d11580-fd57-4472-ab08-33e6d8f79a12 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:07.114191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d11580-fd57-4472-ab08-33e6d8f79a12 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:07.117884 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9ee39-57f6-4014-95ec-ef6e2a3d8970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:07.118428 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9ee39-57f6-4014-95ec-ef6e2a3d8970 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:07.118752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9ee39-57f6-4014-95ec-ef6e2a3d8970 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:07.119028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9ee39-57f6-4014-95ec-ef6e2a3d8970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:07.126234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9ee39-57f6-4014-95ec-ef6e2a3d8970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:07.127699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9ee39-57f6-4014-95ec-ef6e2a3d8970 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:07.135574 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4466/8865] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:48:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:48:17.078072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:48:17.132346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=9 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:48:26.600974 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994bf6f-b1fb-4783-a6e8-e0bf3799a3a1 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:26.602501 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8994bf6f-b1fb-4783-a6e8-e0bf3799a3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:26.654990 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9ee39-57f6-4014-95ec-ef6e2a3d8970 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:26.668216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9ee39-57f6-4014-95ec-ef6e2a3d8970 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:26.675022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9c287-8504-4f9d-97d8-04ec16dc2552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:26.675918 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9c287-8504-4f9d-97d8-04ec16dc2552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:26.676223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9c287-8504-4f9d-97d8-04ec16dc2552 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:26.676494 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9c287-8504-4f9d-97d8-04ec16dc2552 admin admin] PATH_INFO: `/v3/users/309ad5bc59d84e409cc40374b070b2b8` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:26.677620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9c287-8504-4f9d-97d8-04ec16dc2552 admin admin] RBAC: Authorizing `identity:delete_user(user_id=309ad5bc59d84e409cc40374b070b2b8)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:26.680057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9c287-8504-4f9d-97d8-04ec16dc2552 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:26.693347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994bf6f-b1fb-4783-a6e8-e0bf3799a3a1 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:26.698427 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534efaa-c42c-4ea1-852b-dd3f51e14059 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:26.699063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534efaa-c42c-4ea1-852b-dd3f51e14059 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:26.699342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534efaa-c42c-4ea1-852b-dd3f51e14059 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:26.699620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534efaa-c42c-4ea1-852b-dd3f51e14059 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:26.701335 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9534efaa-c42c-4ea1-852b-dd3f51e14059 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:26.717407 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534efaa-c42c-4ea1-852b-dd3f51e14059 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:26.719060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534efaa-c42c-4ea1-852b-dd3f51e14059 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:26.732731 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4400/8866] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:48:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:48:26.737215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-92a9c287-8504-4f9d-97d8-04ec16dc2552 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '309ad5bc59d84e409cc40374b070b2b8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:26.744873 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-92a9c287-8504-4f9d-97d8-04ec16dc2552 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '309ad5bc59d84e409cc40374b070b2b8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:26.758246 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-92a9c287-8504-4f9d-97d8-04ec16dc2552 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '309ad5bc59d84e409cc40374b070b2b8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:26.759026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-92a9c287-8504-4f9d-97d8-04ec16dc2552 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '309ad5bc59d84e409cc40374b070b2b8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:26.766952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-92a9c287-8504-4f9d-97d8-04ec16dc2552 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '309ad5bc59d84e409cc40374b070b2b8'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:26.773141 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4467/8867] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:48:26 2022] DELETE /identity/v3/users/309ad5bc59d84e409cc40374b070b2b8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:48:26.943554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534efaa-c42c-4ea1-852b-dd3f51e14059 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:26.954594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534efaa-c42c-4ea1-852b-dd3f51e14059 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:27.011758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24848e-b20f-45e5-8b08-62201c1a3b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:27.012411 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24848e-b20f-45e5-8b08-62201c1a3b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:27.012621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24848e-b20f-45e5-8b08-62201c1a3b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:27.012896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24848e-b20f-45e5-8b08-62201c1a3b78 admin admin] PATH_INFO: `/v3/projects/486d273a605a489cb5acd91ef693ec38` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:27.014106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24848e-b20f-45e5-8b08-62201c1a3b78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=486d273a605a489cb5acd91ef693ec38)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:27.016075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24848e-b20f-45e5-8b08-62201c1a3b78 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:27.071562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5a24848e-b20f-45e5-8b08-62201c1a3b78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '486d273a605a489cb5acd91ef693ec38', 'request_id': 'req-5a24848e-b20f-45e5-8b08-62201c1a3b78'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:27.078621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5a24848e-b20f-45e5-8b08-62201c1a3b78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '486d273a605a489cb5acd91ef693ec38', 'request_id': 'req-5a24848e-b20f-45e5-8b08-62201c1a3b78'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:27.085184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5a24848e-b20f-45e5-8b08-62201c1a3b78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '486d273a605a489cb5acd91ef693ec38', 'request_id': 'req-5a24848e-b20f-45e5-8b08-62201c1a3b78'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:27.097772 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5a24848e-b20f-45e5-8b08-62201c1a3b78 admin admin] The token cache is being invalidate because project 486d273a605a489cb5acd91ef693ec38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:48:27.098687 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5a24848e-b20f-45e5-8b08-62201c1a3b78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:27.098876 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-5a24848e-b20f-45e5-8b08-62201c1a3b78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:27.100447 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4401/8868] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:48:26 2022] DELETE /identity/v3/projects/486d273a605a489cb5acd91ef693ec38 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:48:27.105369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8707e70-3e80-40f9-96db-d51850e63e37 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:27.105688 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8707e70-3e80-40f9-96db-d51850e63e37 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:27.105965 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8707e70-3e80-40f9-96db-d51850e63e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:27.126234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-c8707e70-3e80-40f9-96db-d51850e63e37 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:48:27.127498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c8707e70-3e80-40f9-96db-d51850e63e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:27.187408 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4468/8869] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:48:27.195386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604de04-a2a5-4078-9129-8b9c6d508c05 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:27.195679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604de04-a2a5-4078-9129-8b9c6d508c05 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:27.195961 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604de04-a2a5-4078-9129-8b9c6d508c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:27.216168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-8604de04-a2a5-4078-9129-8b9c6d508c05 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:48:27.217396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-8604de04-a2a5-4078-9129-8b9c6d508c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:27.225731 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4402/8870] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:48:27.229804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8707e70-3e80-40f9-96db-d51850e63e37 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:27.239581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8707e70-3e80-40f9-96db-d51850e63e37 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:27.244174 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e7437-2cd1-4776-987d-1bee87f7f5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:27.244764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e7437-2cd1-4776-987d-1bee87f7f5b7 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:27.245044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e7437-2cd1-4776-987d-1bee87f7f5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:27.245315 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e7437-2cd1-4776-987d-1bee87f7f5b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:27.245952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e7437-2cd1-4776-987d-1bee87f7f5b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:48:27.246253 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e7437-2cd1-4776-987d-1bee87f7f5b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:27.247937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e7437-2cd1-4776-987d-1bee87f7f5b7 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:27.263836 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4469/8871] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:48:27.265190 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604de04-a2a5-4078-9129-8b9c6d508c05 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:27.275445 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604de04-a2a5-4078-9129-8b9c6d508c05 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:27.280302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04ce8b-03c4-41e9-8539-46907cc1b845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:27.280903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04ce8b-03c4-41e9-8539-46907cc1b845 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:27.281185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04ce8b-03c4-41e9-8539-46907cc1b845 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:27.281459 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04ce8b-03c4-41e9-8539-46907cc1b845 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:27.282095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04ce8b-03c4-41e9-8539-46907cc1b845 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:27.284148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04ce8b-03c4-41e9-8539-46907cc1b845 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:27.295478 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:48:27.312381 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4403/8872] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:48:27 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:48:27.317113 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e7437-2cd1-4776-987d-1bee87f7f5b7 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:27.327364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e7437-2cd1-4776-987d-1bee87f7f5b7 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:27.384028 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59f328-0003-4445-9aa4-de3fe0a1b8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:27.384562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59f328-0003-4445-9aa4-de3fe0a1b8c8 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:27.384846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59f328-0003-4445-9aa4-de3fe0a1b8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:27.385118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59f328-0003-4445-9aa4-de3fe0a1b8c8 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:27.385767 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf59f328-0003-4445-9aa4-de3fe0a1b8c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:27.387684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf59f328-0003-4445-9aa4-de3fe0a1b8c8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:27.396221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:48:27.412453 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4470/8873] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:48:27 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:48:27.416489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04ce8b-03c4-41e9-8539-46907cc1b845 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:27.427061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04ce8b-03c4-41e9-8539-46907cc1b845 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:27.431931 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a46fe-cda3-4dc2-b062-ef7f14da07ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:27.432519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a46fe-cda3-4dc2-b062-ef7f14da07ff admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:27.432798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a46fe-cda3-4dc2-b062-ef7f14da07ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:27.433069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a46fe-cda3-4dc2-b062-ef7f14da07ff admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:27.433693 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a46fe-cda3-4dc2-b062-ef7f14da07ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:27.435587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a46fe-cda3-4dc2-b062-ef7f14da07ff admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:27.446467 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4404/8874] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:48:27.450324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59f328-0003-4445-9aa4-de3fe0a1b8c8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:27.460761 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59f328-0003-4445-9aa4-de3fe0a1b8c8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:27.465319 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7736311-0e91-43c7-90d4-7d8782b21f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:27.465906 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7736311-0e91-43c7-90d4-7d8782b21f6b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:27.466186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7736311-0e91-43c7-90d4-7d8782b21f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:27.466459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7736311-0e91-43c7-90d4-7d8782b21f6b admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:27.467124 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7736311-0e91-43c7-90d4-7d8782b21f6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:27.468693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7736311-0e91-43c7-90d4-7d8782b21f6b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:27.479548 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4471/8875] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:48:27.483617 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a46fe-cda3-4dc2-b062-ef7f14da07ff admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:27.493728 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a46fe-cda3-4dc2-b062-ef7f14da07ff admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:27.498773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6787a29-3a25-4d29-8ea8-fa1b97edadfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:27.499446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6787a29-3a25-4d29-8ea8-fa1b97edadfb admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:27.499731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6787a29-3a25-4d29-8ea8-fa1b97edadfb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:27.500013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6787a29-3a25-4d29-8ea8-fa1b97edadfb admin admin] PATH_INFO: `/v3/projects/a6072ef70e574e1a952c16a52e713ec8/users/40619748032f4e5d9d7de4092ea1648b/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:27.516602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6787a29-3a25-4d29-8ea8-fa1b97edadfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6072ef70e574e1a952c16a52e713ec8, user_id=40619748032f4e5d9d7de4092ea1648b, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:27.519930 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6787a29-3a25-4d29-8ea8-fa1b97edadfb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:27.532789 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4405/8876] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:48:27 2022] PUT /identity/v3/projects/a6072ef70e574e1a952c16a52e713ec8/users/40619748032f4e5d9d7de4092ea1648b/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:48:27.542804 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed7fef-c787-4c63-bc41-a9901acdaf2d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:27.543091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed7fef-c787-4c63-bc41-a9901acdaf2d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:27.543367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed7fef-c787-4c63-bc41-a9901acdaf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:27.571485 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-f6ed7fef-c787-4c63-bc41-a9901acdaf2d None None] MFA Rules not processed for user `40619748032f4e5d9d7de4092ea1648b`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:48:27.572686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f6ed7fef-c787-4c63-bc41-a9901acdaf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:27.636899 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4472/8877] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:48:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:48:27.646979 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6787a29-3a25-4d29-8ea8-fa1b97edadfb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:27.648711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e6787a29-3a25-4d29-8ea8-fa1b97edadfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:27.706329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6787a29-3a25-4d29-8ea8-fa1b97edadfb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:27.710029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d4ef6-ba41-446a-bd98-ba4747c9e979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:27.710662 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d4ef6-ba41-446a-bd98-ba4747c9e979 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:27.710945 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d4ef6-ba41-446a-bd98-ba4747c9e979 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:27.711218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d4ef6-ba41-446a-bd98-ba4747c9e979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:27.719455 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d4ef6-ba41-446a-bd98-ba4747c9e979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:27.720885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d4ef6-ba41-446a-bd98-ba4747c9e979 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:27.729623 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4406/8878] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:48:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:48:32.793069 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acada384-92d2-4e1f-9368-1bc5cfa53140 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:32.793599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acada384-92d2-4e1f-9368-1bc5cfa53140 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:32.793599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acada384-92d2-4e1f-9368-1bc5cfa53140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:32.816766 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-acada384-92d2-4e1f-9368-1bc5cfa53140 None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:48:32.818186 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-acada384-92d2-4e1f-9368-1bc5cfa53140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:32.908131 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4473/8879] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 21:48:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:48:32.935345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d4ef6-ba41-446a-bd98-ba4747c9e979 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:32.936925 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2f2d4ef6-ba41-446a-bd98-ba4747c9e979 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:32.957069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d4ef6-ba41-446a-bd98-ba4747c9e979 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:32.962308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bf992-4650-4b04-afe4-f39d0e3d2ec4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:32.962943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bf992-4650-4b04-afe4-f39d0e3d2ec4 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:32.963220 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bf992-4650-4b04-afe4-f39d0e3d2ec4 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:32.963491 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bf992-4650-4b04-afe4-f39d0e3d2ec4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:32.988061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6bf992-4650-4b04-afe4-f39d0e3d2ec4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:32.990210 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6bf992-4650-4b04-afe4-f39d0e3d2ec4 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:33.013279 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4407/8880] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:48:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:48:39.232555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acada384-92d2-4e1f-9368-1bc5cfa53140 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:39.295542 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acada384-92d2-4e1f-9368-1bc5cfa53140 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:39.300643 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0952bb5-e7a1-489b-8a28-d6afac6816ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:39.301270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0952bb5-e7a1-489b-8a28-d6afac6816ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:39.301548 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0952bb5-e7a1-489b-8a28-d6afac6816ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:39.301820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0952bb5-e7a1-489b-8a28-d6afac6816ac admin admin] PATH_INFO: `/v3/users/40619748032f4e5d9d7de4092ea1648b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:39.302978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0952bb5-e7a1-489b-8a28-d6afac6816ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=40619748032f4e5d9d7de4092ea1648b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:39.305143 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0952bb5-e7a1-489b-8a28-d6afac6816ac admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:39.309829 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:48:39.355843 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b0952bb5-e7a1-489b-8a28-d6afac6816ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40619748032f4e5d9d7de4092ea1648b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:39.363665 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b0952bb5-e7a1-489b-8a28-d6afac6816ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40619748032f4e5d9d7de4092ea1648b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:39.375622 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b0952bb5-e7a1-489b-8a28-d6afac6816ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40619748032f4e5d9d7de4092ea1648b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:39.376265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b0952bb5-e7a1-489b-8a28-d6afac6816ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40619748032f4e5d9d7de4092ea1648b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:39.383732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b0952bb5-e7a1-489b-8a28-d6afac6816ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40619748032f4e5d9d7de4092ea1648b'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:39.389931 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4474/8881] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:48:39 2022] DELETE /identity/v3/users/40619748032f4e5d9d7de4092ea1648b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:48:39.394539 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d430fe-935e-4f02-9138-9466746192bb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:39.395022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d430fe-935e-4f02-9138-9466746192bb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:39.395146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d430fe-935e-4f02-9138-9466746192bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:39.414250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-24d430fe-935e-4f02-9138-9466746192bb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:48:39.415495 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-24d430fe-935e-4f02-9138-9466746192bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:39.473667 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4408/8882] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:48:39.483502 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0952bb5-e7a1-489b-8a28-d6afac6816ac admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:39.484925 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b0952bb5-e7a1-489b-8a28-d6afac6816ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:39.544127 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0952bb5-e7a1-489b-8a28-d6afac6816ac admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:39.547708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77216591-0243-4d0d-a11c-a3ebff280a4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:39.548307 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77216591-0243-4d0d-a11c-a3ebff280a4f service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:39.548585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77216591-0243-4d0d-a11c-a3ebff280a4f service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:39.549493 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77216591-0243-4d0d-a11c-a3ebff280a4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:39.558745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77216591-0243-4d0d-a11c-a3ebff280a4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:39.560169 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77216591-0243-4d0d-a11c-a3ebff280a4f service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:39.569902 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4475/8883] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:48:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:48:39.742364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d430fe-935e-4f02-9138-9466746192bb None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:39.743847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-24d430fe-935e-4f02-9138-9466746192bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:39.763158 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d430fe-935e-4f02-9138-9466746192bb None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:39.771213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970d68b-2459-4b51-96e3-e84de47440ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:39.772584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970d68b-2459-4b51-96e3-e84de47440ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:39.773652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970d68b-2459-4b51-96e3-e84de47440ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:39.773935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970d68b-2459-4b51-96e3-e84de47440ea admin admin] PATH_INFO: `/v3/projects/a6072ef70e574e1a952c16a52e713ec8` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:39.775130 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970d68b-2459-4b51-96e3-e84de47440ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6072ef70e574e1a952c16a52e713ec8)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:39.777044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970d68b-2459-4b51-96e3-e84de47440ea admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:39.818731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:48:39.825415 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4970d68b-2459-4b51-96e3-e84de47440ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6072ef70e574e1a952c16a52e713ec8', 'request_id': 'req-4970d68b-2459-4b51-96e3-e84de47440ea'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:39.833996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4970d68b-2459-4b51-96e3-e84de47440ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6072ef70e574e1a952c16a52e713ec8', 'request_id': 'req-4970d68b-2459-4b51-96e3-e84de47440ea'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:39.839863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4970d68b-2459-4b51-96e3-e84de47440ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6072ef70e574e1a952c16a52e713ec8', 'request_id': 'req-4970d68b-2459-4b51-96e3-e84de47440ea'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:39.851578 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4970d68b-2459-4b51-96e3-e84de47440ea admin admin] The token cache is being invalidate because project a6072ef70e574e1a952c16a52e713ec8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:48:39.851868 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4970d68b-2459-4b51-96e3-e84de47440ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:39.852508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-4970d68b-2459-4b51-96e3-e84de47440ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:39.854016 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4409/8884] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:48:39 2022] DELETE /identity/v3/projects/a6072ef70e574e1a952c16a52e713ec8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:48:39.859459 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9b258-6277-40b3-8671-a282f4ae7846 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:39.859744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9b258-6277-40b3-8671-a282f4ae7846 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:39.860019 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9b258-6277-40b3-8671-a282f4ae7846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:39.879323 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e1d9b258-6277-40b3-8671-a282f4ae7846 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:48:39.880521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e1d9b258-6277-40b3-8671-a282f4ae7846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:39.939227 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4476/8885] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:48:39.946770 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ad29d-7829-4740-bf06-2c5856ff93fd None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:39.947135 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ad29d-7829-4740-bf06-2c5856ff93fd None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:39.947439 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ad29d-7829-4740-bf06-2c5856ff93fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:39.967250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6d9ad29d-7829-4740-bf06-2c5856ff93fd None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:48:39.968456 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6d9ad29d-7829-4740-bf06-2c5856ff93fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:39.976726 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4410/8886] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:48:39.980856 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9b258-6277-40b3-8671-a282f4ae7846 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:39.990760 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9b258-6277-40b3-8671-a282f4ae7846 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:39.996669 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f44c1-b4d7-411c-9687-687a6c4fcfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:39.997288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f44c1-b4d7-411c-9687-687a6c4fcfaf admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:39.997575 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f44c1-b4d7-411c-9687-687a6c4fcfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:39.997851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f44c1-b4d7-411c-9687-687a6c4fcfaf admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:39.998538 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f44c1-b4d7-411c-9687-687a6c4fcfaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:48:39.998879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f44c1-b4d7-411c-9687-687a6c4fcfaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:40.000652 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f44c1-b4d7-411c-9687-687a6c4fcfaf admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:40.014073 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4477/8887] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:48:40.018563 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ad29d-7829-4740-bf06-2c5856ff93fd None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:40.027833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ad29d-7829-4740-bf06-2c5856ff93fd None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:40.032475 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20605f-325e-4613-9c16-3aa1f381c0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:40.033230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20605f-325e-4613-9c16-3aa1f381c0b2 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:40.033509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20605f-325e-4613-9c16-3aa1f381c0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:40.033778 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20605f-325e-4613-9c16-3aa1f381c0b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:40.034396 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20605f-325e-4613-9c16-3aa1f381c0b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:40.036330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20605f-325e-4613-9c16-3aa1f381c0b2 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:40.066018 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4411/8888] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:48:40 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:48:40.070304 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f44c1-b4d7-411c-9687-687a6c4fcfaf admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:40.081062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f44c1-b4d7-411c-9687-687a6c4fcfaf admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:40.089749 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:48:40.137572 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1ddc7-d439-4e5a-afde-a5f6f68c97e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:40.138182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1ddc7-d439-4e5a-afde-a5f6f68c97e9 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:40.138462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1ddc7-d439-4e5a-afde-a5f6f68c97e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:40.138771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1ddc7-d439-4e5a-afde-a5f6f68c97e9 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:40.139398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1ddc7-d439-4e5a-afde-a5f6f68c97e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:40.141303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1ddc7-d439-4e5a-afde-a5f6f68c97e9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:40.163134 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4478/8889] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:48:40 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:48:40.167021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20605f-325e-4613-9c16-3aa1f381c0b2 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:40.177378 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20605f-325e-4613-9c16-3aa1f381c0b2 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:40.182511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3cbd4-5565-4261-9827-558d645e1bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:40.183117 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3cbd4-5565-4261-9827-558d645e1bec admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:40.183394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3cbd4-5565-4261-9827-558d645e1bec admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:40.183671 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3cbd4-5565-4261-9827-558d645e1bec admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:40.184294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3cbd4-5565-4261-9827-558d645e1bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:40.185964 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3cbd4-5565-4261-9827-558d645e1bec admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:40.196456 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4412/8890] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:48:40.200072 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1ddc7-d439-4e5a-afde-a5f6f68c97e9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:40.210641 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1ddc7-d439-4e5a-afde-a5f6f68c97e9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:40.215289 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499a477-41a5-45c0-b5a7-65d4fea531f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:40.215883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499a477-41a5-45c0-b5a7-65d4fea531f8 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:40.216166 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499a477-41a5-45c0-b5a7-65d4fea531f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:40.216445 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499a477-41a5-45c0-b5a7-65d4fea531f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:40.217085 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499a477-41a5-45c0-b5a7-65d4fea531f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:40.218699 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499a477-41a5-45c0-b5a7-65d4fea531f8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:40.229331 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4479/8891] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:48:40.233046 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3cbd4-5565-4261-9827-558d645e1bec admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:40.243124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3cbd4-5565-4261-9827-558d645e1bec admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:40.247751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c719a84-03a5-44af-9657-4eed41a589dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:40.248392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c719a84-03a5-44af-9657-4eed41a589dc admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:40.248670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c719a84-03a5-44af-9657-4eed41a589dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:40.248942 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c719a84-03a5-44af-9657-4eed41a589dc admin admin] PATH_INFO: `/v3/projects/16f2ea3a5a784dfe88122ac7633bb170/users/f316e3f60c3449d3b36625ceb946283a/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:40.263677 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c719a84-03a5-44af-9657-4eed41a589dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=16f2ea3a5a784dfe88122ac7633bb170, user_id=f316e3f60c3449d3b36625ceb946283a, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:40.266304 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c719a84-03a5-44af-9657-4eed41a589dc admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:40.279261 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4413/8892] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:48:40 2022] PUT /identity/v3/projects/16f2ea3a5a784dfe88122ac7633bb170/users/f316e3f60c3449d3b36625ceb946283a/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:48:40.289840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f8532-1d7d-4897-959b-dd9eb625c144 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:40.290370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f8532-1d7d-4897-959b-dd9eb625c144 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:40.290370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f8532-1d7d-4897-959b-dd9eb625c144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:40.318759 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-781f8532-1d7d-4897-959b-dd9eb625c144 None None] MFA Rules not processed for user `f316e3f60c3449d3b36625ceb946283a`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:48:40.319957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-781f8532-1d7d-4897-959b-dd9eb625c144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:40.389141 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4480/8893] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:48:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:48:40.399138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c719a84-03a5-44af-9657-4eed41a589dc admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:40.400840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2c719a84-03a5-44af-9657-4eed41a589dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:40.461869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c719a84-03a5-44af-9657-4eed41a589dc admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:40.465613 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe833b7-2229-407a-a61d-4498e9c58331 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:40.466251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe833b7-2229-407a-a61d-4498e9c58331 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:40.466556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe833b7-2229-407a-a61d-4498e9c58331 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:40.466834 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe833b7-2229-407a-a61d-4498e9c58331 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:40.475237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe833b7-2229-407a-a61d-4498e9c58331 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:40.476660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe833b7-2229-407a-a61d-4498e9c58331 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:40.485315 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4414/8894] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:48:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:48:45.197458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f8532-1d7d-4897-959b-dd9eb625c144 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:45.260621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f8532-1d7d-4897-959b-dd9eb625c144 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:45.265341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff03f8d-6327-4d44-b413-35e20ad20595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:45.265998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff03f8d-6327-4d44-b413-35e20ad20595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:45.266280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff03f8d-6327-4d44-b413-35e20ad20595 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:45.266583 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff03f8d-6327-4d44-b413-35e20ad20595 admin admin] PATH_INFO: `/v3/users/f316e3f60c3449d3b36625ceb946283a` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:45.267719 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff03f8d-6327-4d44-b413-35e20ad20595 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f316e3f60c3449d3b36625ceb946283a)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:45.270213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff03f8d-6327-4d44-b413-35e20ad20595 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:45.295119 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:48:45.324249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dff03f8d-6327-4d44-b413-35e20ad20595 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f316e3f60c3449d3b36625ceb946283a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:45.332290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dff03f8d-6327-4d44-b413-35e20ad20595 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f316e3f60c3449d3b36625ceb946283a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:45.344585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dff03f8d-6327-4d44-b413-35e20ad20595 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f316e3f60c3449d3b36625ceb946283a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:45.345282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dff03f8d-6327-4d44-b413-35e20ad20595 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f316e3f60c3449d3b36625ceb946283a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:45.353040 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-dff03f8d-6327-4d44-b413-35e20ad20595 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f316e3f60c3449d3b36625ceb946283a'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:45.359398 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4481/8895] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:48:45 2022] DELETE /identity/v3/users/f316e3f60c3449d3b36625ceb946283a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:48:45.363892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa0ff0-942a-45cf-956b-ebba8c583ad9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:45.364358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa0ff0-942a-45cf-956b-ebba8c583ad9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:45.364466 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa0ff0-942a-45cf-956b-ebba8c583ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:45.383850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-a3fa0ff0-942a-45cf-956b-ebba8c583ad9 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:48:45.385137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a3fa0ff0-942a-45cf-956b-ebba8c583ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:45.443519 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4415/8896] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:48:45.453377 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff03f8d-6327-4d44-b413-35e20ad20595 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:45.454846 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-dff03f8d-6327-4d44-b413-35e20ad20595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:45.511797 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff03f8d-6327-4d44-b413-35e20ad20595 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:45.515397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf1d5f-8448-4cbc-9d96-d3d9e8cfa0cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:45.515998 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf1d5f-8448-4cbc-9d96-d3d9e8cfa0cf service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:45.516279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf1d5f-8448-4cbc-9d96-d3d9e8cfa0cf service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:45.516555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf1d5f-8448-4cbc-9d96-d3d9e8cfa0cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:45.526074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf1d5f-8448-4cbc-9d96-d3d9e8cfa0cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:45.527523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf1d5f-8448-4cbc-9d96-d3d9e8cfa0cf service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:45.536992 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4482/8897] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:48:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:48:45.710238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa0ff0-942a-45cf-956b-ebba8c583ad9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:45.711721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-a3fa0ff0-942a-45cf-956b-ebba8c583ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:45.723661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa0ff0-942a-45cf-956b-ebba8c583ad9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:45.728296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc54e6-8e14-4353-bbd9-8904145e3802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:45.728926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc54e6-8e14-4353-bbd9-8904145e3802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:45.729211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc54e6-8e14-4353-bbd9-8904145e3802 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:45.729482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc54e6-8e14-4353-bbd9-8904145e3802 admin admin] PATH_INFO: `/v3/projects/16f2ea3a5a784dfe88122ac7633bb170` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:45.730665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc54e6-8e14-4353-bbd9-8904145e3802 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16f2ea3a5a784dfe88122ac7633bb170)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:45.732577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc54e6-8e14-4353-bbd9-8904145e3802 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:45.735164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:48:45.782575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b7bc54e6-8e14-4353-bbd9-8904145e3802 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16f2ea3a5a784dfe88122ac7633bb170', 'request_id': 'req-b7bc54e6-8e14-4353-bbd9-8904145e3802'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:45.789829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b7bc54e6-8e14-4353-bbd9-8904145e3802 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16f2ea3a5a784dfe88122ac7633bb170', 'request_id': 'req-b7bc54e6-8e14-4353-bbd9-8904145e3802'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:45.795297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b7bc54e6-8e14-4353-bbd9-8904145e3802 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16f2ea3a5a784dfe88122ac7633bb170', 'request_id': 'req-b7bc54e6-8e14-4353-bbd9-8904145e3802'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:45.807237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b7bc54e6-8e14-4353-bbd9-8904145e3802 admin admin] The token cache is being invalidate because project 16f2ea3a5a784dfe88122ac7633bb170 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:48:45.807528 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b7bc54e6-8e14-4353-bbd9-8904145e3802 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:45.808164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b7bc54e6-8e14-4353-bbd9-8904145e3802 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:48:45.809643 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4416/8898] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:48:45 2022] DELETE /identity/v3/projects/16f2ea3a5a784dfe88122ac7633bb170 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:48:45.816299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb0d95-2af9-46bb-bbc5-731f651610a4 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:45.816591 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb0d95-2af9-46bb-bbc5-731f651610a4 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:45.816871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb0d95-2af9-46bb-bbc5-731f651610a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:45.836322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-12eb0d95-2af9-46bb-bbc5-731f651610a4 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:48:45.837530 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-12eb0d95-2af9-46bb-bbc5-731f651610a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:45.896917 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4483/8899] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:48:45.904910 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d47b9-dcc9-46b8-9885-e426c795ec6b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:45.905202 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d47b9-dcc9-46b8-9885-e426c795ec6b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:45.905502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d47b9-dcc9-46b8-9885-e426c795ec6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:45.925355 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-280d47b9-dcc9-46b8-9885-e426c795ec6b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:48:45.926585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-280d47b9-dcc9-46b8-9885-e426c795ec6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:45.934959 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4417/8900] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:48:45.939145 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb0d95-2af9-46bb-bbc5-731f651610a4 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:45.950335 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb0d95-2af9-46bb-bbc5-731f651610a4 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:45.955199 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524be61f-3ad1-4db6-a31b-ef45e88f55ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:45.955805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524be61f-3ad1-4db6-a31b-ef45e88f55ee admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:45.956083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524be61f-3ad1-4db6-a31b-ef45e88f55ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:45.956353 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524be61f-3ad1-4db6-a31b-ef45e88f55ee admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:45.956994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524be61f-3ad1-4db6-a31b-ef45e88f55ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:48:45.957316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524be61f-3ad1-4db6-a31b-ef45e88f55ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:45.958982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524be61f-3ad1-4db6-a31b-ef45e88f55ee admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:45.970698 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4484/8901] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:48:45.975603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d47b9-dcc9-46b8-9885-e426c795ec6b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:45.984935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d47b9-dcc9-46b8-9885-e426c795ec6b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:45.990274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc7409-528e-4799-a778-d9530347383d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:45.990905 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc7409-528e-4799-a778-d9530347383d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:45.991192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc7409-528e-4799-a778-d9530347383d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:45.991472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc7409-528e-4799-a778-d9530347383d admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:45.992091 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc7409-528e-4799-a778-d9530347383d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:45.993995 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc7409-528e-4799-a778-d9530347383d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:46.020743 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4418/8902] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:48:45 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:48:46.024885 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524be61f-3ad1-4db6-a31b-ef45e88f55ee admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:46.035865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524be61f-3ad1-4db6-a31b-ef45e88f55ee admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:46.069833 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:48:46.099928 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217c950-9a11-4976-b421-bd1daa6dc3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:46.100533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217c950-9a11-4976-b421-bd1daa6dc3b8 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:46.100816 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217c950-9a11-4976-b421-bd1daa6dc3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:46.101090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217c950-9a11-4976-b421-bd1daa6dc3b8 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:46.101722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217c950-9a11-4976-b421-bd1daa6dc3b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:46.103657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217c950-9a11-4976-b421-bd1daa6dc3b8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:46.124764 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4485/8903] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:48:46 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:48:46.128969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc7409-528e-4799-a778-d9530347383d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:46.139393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc7409-528e-4799-a778-d9530347383d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:46.144025 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c504994-0aeb-401f-8783-2bd17541d2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:46.144651 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c504994-0aeb-401f-8783-2bd17541d2de admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:46.144934 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c504994-0aeb-401f-8783-2bd17541d2de admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:46.145208 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c504994-0aeb-401f-8783-2bd17541d2de admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:46.145843 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c504994-0aeb-401f-8783-2bd17541d2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:46.147480 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c504994-0aeb-401f-8783-2bd17541d2de admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:46.157191 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4419/8904] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:48:46.161408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217c950-9a11-4976-b421-bd1daa6dc3b8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:46.171490 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217c950-9a11-4976-b421-bd1daa6dc3b8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:46.176960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bd579-0948-4b0a-96d7-aaa2a6a4a31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:46.177555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bd579-0948-4b0a-96d7-aaa2a6a4a31b admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:46.177834 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bd579-0948-4b0a-96d7-aaa2a6a4a31b admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:46.178133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bd579-0948-4b0a-96d7-aaa2a6a4a31b admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:46.178814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9bd579-0948-4b0a-96d7-aaa2a6a4a31b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:46.180673 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9bd579-0948-4b0a-96d7-aaa2a6a4a31b admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:46.191260 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4486/8905] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:48:46.195347 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c504994-0aeb-401f-8783-2bd17541d2de admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:46.205213 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c504994-0aeb-401f-8783-2bd17541d2de admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:46.209774 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf878b-c349-4b37-9c84-dd97b52ffd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:46.210402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf878b-c349-4b37-9c84-dd97b52ffd73 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:46.210713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf878b-c349-4b37-9c84-dd97b52ffd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:46.210984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf878b-c349-4b37-9c84-dd97b52ffd73 admin admin] PATH_INFO: `/v3/projects/f02c2d9ca9fa43bdad1329ff8583b742/users/0d1d8f5e9d074c2a89f5cf7c874392ac/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:46.225703 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf878b-c349-4b37-9c84-dd97b52ffd73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f02c2d9ca9fa43bdad1329ff8583b742, user_id=0d1d8f5e9d074c2a89f5cf7c874392ac, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:46.228374 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf878b-c349-4b37-9c84-dd97b52ffd73 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:46.242464 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4420/8906] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:48:46 2022] PUT /identity/v3/projects/f02c2d9ca9fa43bdad1329ff8583b742/users/0d1d8f5e9d074c2a89f5cf7c874392ac/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:48:46.247504 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e75b1-0708-427f-8df5-a859d99c92fb None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:46.247976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e75b1-0708-427f-8df5-a859d99c92fb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:46.248076 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e75b1-0708-427f-8df5-a859d99c92fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:46.266900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-507e75b1-0708-427f-8df5-a859d99c92fb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:48:46.268097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-507e75b1-0708-427f-8df5-a859d99c92fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:46.332183 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4487/8907] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:48:46.341758 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf878b-c349-4b37-9c84-dd97b52ffd73 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:46.343481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-84cf878b-c349-4b37-9c84-dd97b52ffd73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:46.399691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf878b-c349-4b37-9c84-dd97b52ffd73 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:46.403447 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320d05b-90ad-41a4-8c02-ea4152bd56b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:46.404052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320d05b-90ad-41a4-8c02-ea4152bd56b8 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:46.404330 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320d05b-90ad-41a4-8c02-ea4152bd56b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:46.404604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320d05b-90ad-41a4-8c02-ea4152bd56b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:46.414069 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320d05b-90ad-41a4-8c02-ea4152bd56b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:46.415519 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320d05b-90ad-41a4-8c02-ea4152bd56b8 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:46.425473 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4421/8908] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:48:50.095805 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657335ac-ff75-4a43-9770-a1e1144579ad None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:50.096330 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657335ac-ff75-4a43-9770-a1e1144579ad None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:50.096589 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657335ac-ff75-4a43-9770-a1e1144579ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:50.125369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-657335ac-ff75-4a43-9770-a1e1144579ad None None] MFA Rules not processed for user `0d1d8f5e9d074c2a89f5cf7c874392ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:48:50.126522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-657335ac-ff75-4a43-9770-a1e1144579ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:50.192639 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4488/8909] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:48:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:48:50.197317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320d05b-90ad-41a4-8c02-ea4152bd56b8 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:50.206958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320d05b-90ad-41a4-8c02-ea4152bd56b8 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:50.211565 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc0ef3-56fc-4394-a6e5-356bd82c7c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:50.212226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc0ef3-56fc-4394-a6e5-356bd82c7c8b admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:50.212571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc0ef3-56fc-4394-a6e5-356bd82c7c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:50.212933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc0ef3-56fc-4394-a6e5-356bd82c7c8b admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:50.213626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc0ef3-56fc-4394-a6e5-356bd82c7c8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:50.215667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc0ef3-56fc-4394-a6e5-356bd82c7c8b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:50.244565 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4422/8910] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:48:50 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:48:50.248806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657335ac-ff75-4a43-9770-a1e1144579ad None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:50.260262 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657335ac-ff75-4a43-9770-a1e1144579ad None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:50.301858 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:48:50.322907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b524c4e-38df-4220-9b04-5f0bbfcaf241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:50.323544 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524c4e-38df-4220-9b04-5f0bbfcaf241 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:50.323901 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524c4e-38df-4220-9b04-5f0bbfcaf241 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:50.324182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524c4e-38df-4220-9b04-5f0bbfcaf241 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:50.324826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b524c4e-38df-4220-9b04-5f0bbfcaf241 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:50.326850 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b524c4e-38df-4220-9b04-5f0bbfcaf241 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:50.350380 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4489/8911] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:48:50 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:48:50.354522 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc0ef3-56fc-4394-a6e5-356bd82c7c8b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:50.366472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc0ef3-56fc-4394-a6e5-356bd82c7c8b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:50.371295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256eda96-c765-4e45-bf31-91a3319bb51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:50.371947 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256eda96-c765-4e45-bf31-91a3319bb51b admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:50.372238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256eda96-c765-4e45-bf31-91a3319bb51b admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:50.372536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256eda96-c765-4e45-bf31-91a3319bb51b admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:50.373282 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256eda96-c765-4e45-bf31-91a3319bb51b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:50.375114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256eda96-c765-4e45-bf31-91a3319bb51b admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:50.378339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:48:50.391530 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4423/8912] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:48:50.394342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b524c4e-38df-4220-9b04-5f0bbfcaf241 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:50.404614 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b524c4e-38df-4220-9b04-5f0bbfcaf241 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:50.409680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66726e4d-2da1-4ba6-8f7c-1e6af37ce8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:50.410348 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66726e4d-2da1-4ba6-8f7c-1e6af37ce8cb admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:50.410667 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66726e4d-2da1-4ba6-8f7c-1e6af37ce8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:50.410947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66726e4d-2da1-4ba6-8f7c-1e6af37ce8cb admin admin] PATH_INFO: `/v3/projects/de76c6ed429545bbb77bc7cce505e0da/users/7b2fcde4a26c4f3db34451225b025012/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:50.427168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66726e4d-2da1-4ba6-8f7c-1e6af37ce8cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=de76c6ed429545bbb77bc7cce505e0da, user_id=7b2fcde4a26c4f3db34451225b025012, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:50.430158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66726e4d-2da1-4ba6-8f7c-1e6af37ce8cb admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:50.443797 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4490/8913] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:48:50 2022] PUT /identity/v3/projects/de76c6ed429545bbb77bc7cce505e0da/users/7b2fcde4a26c4f3db34451225b025012/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:48:50.447980 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256eda96-c765-4e45-bf31-91a3319bb51b admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:50.458810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256eda96-c765-4e45-bf31-91a3319bb51b admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:50.538264 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c46532-55ea-4e79-9dd5-5658b5655817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:50.539183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c46532-55ea-4e79-9dd5-5658b5655817 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:50.539471 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c46532-55ea-4e79-9dd5-5658b5655817 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:50.543450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c46532-55ea-4e79-9dd5-5658b5655817 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:50.544288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c46532-55ea-4e79-9dd5-5658b5655817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:50.545789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c46532-55ea-4e79-9dd5-5658b5655817 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:50.557696 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4424/8914] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:48:50.562025 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66726e4d-2da1-4ba6-8f7c-1e6af37ce8cb admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:50.575201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66726e4d-2da1-4ba6-8f7c-1e6af37ce8cb admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:50.580121 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce99ba-23e3-4d79-92d1-6b8cbad5425a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:50.580832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce99ba-23e3-4d79-92d1-6b8cbad5425a admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:50.581117 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce99ba-23e3-4d79-92d1-6b8cbad5425a admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:50.581392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce99ba-23e3-4d79-92d1-6b8cbad5425a admin admin] PATH_INFO: `/v3/projects/de76c6ed429545bbb77bc7cce505e0da/users/7b2fcde4a26c4f3db34451225b025012/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:50.583413 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce99ba-23e3-4d79-92d1-6b8cbad5425a admin admin] RBAC: Authorizing `identity:create_grant(project_id=de76c6ed429545bbb77bc7cce505e0da, user_id=7b2fcde4a26c4f3db34451225b025012, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:50.586074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce99ba-23e3-4d79-92d1-6b8cbad5425a admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:50.597953 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4491/8915] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:48:50 2022] PUT /identity/v3/projects/de76c6ed429545bbb77bc7cce505e0da/users/7b2fcde4a26c4f3db34451225b025012/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:48:55.063919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecefa1d-2d02-4aaa-bf72-48f3fda529c8 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:55.064383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecefa1d-2d02-4aaa-bf72-48f3fda529c8 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:55.064773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecefa1d-2d02-4aaa-bf72-48f3fda529c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:55.113534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7ecefa1d-2d02-4aaa-bf72-48f3fda529c8 None None] MFA Rules not processed for user `7b2fcde4a26c4f3db34451225b025012`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:48:55.115709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7ecefa1d-2d02-4aaa-bf72-48f3fda529c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:55.204097 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4425/8916] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:48:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:48:55.234364 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82159a-611d-44d4-bdb0-0cc1733549b3 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:55.234839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82159a-611d-44d4-bdb0-0cc1733549b3 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:55.235159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82159a-611d-44d4-bdb0-0cc1733549b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:55.287271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-bb82159a-611d-44d4-bdb0-0cc1733549b3 None None] MFA Rules not processed for user `e2927203ce394fabab76dd52a68c2436`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:48:55.288437 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-bb82159a-611d-44d4-bdb0-0cc1733549b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:55.370176 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4492/8917] 10.222.0.17 () {58 vars in 1059 bytes} [Sat Oct 1 21:48:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:48:55.376643 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecefa1d-2d02-4aaa-bf72-48f3fda529c8 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:55.387349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecefa1d-2d02-4aaa-bf72-48f3fda529c8 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:55.391782 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce177d7c-7f08-4579-a4d0-2359eb452747 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:55.392397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce177d7c-7f08-4579-a4d0-2359eb452747 service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:55.393368 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce177d7c-7f08-4579-a4d0-2359eb452747 service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:55.393833 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce177d7c-7f08-4579-a4d0-2359eb452747 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:55.461986 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce177d7c-7f08-4579-a4d0-2359eb452747 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:55.463534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce177d7c-7f08-4579-a4d0-2359eb452747 service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:55.475663 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4426/8918] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:48:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:48:55.673447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a27ed-97fd-4f30-980a-1b9cc66c57cf None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:55.673762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a27ed-97fd-4f30-980a-1b9cc66c57cf None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:55.674066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a27ed-97fd-4f30-980a-1b9cc66c57cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:55.732434 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-e91a27ed-97fd-4f30-980a-1b9cc66c57cf None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:48:55.733661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e91a27ed-97fd-4f30-980a-1b9cc66c57cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:55.781559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:48:55.820502 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4493/8919] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:48:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:48:55.875048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605518e4-cdb6-4dc5-8fdc-6ec4e17098cc None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:55.875340 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605518e4-cdb6-4dc5-8fdc-6ec4e17098cc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:55.875620 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605518e4-cdb6-4dc5-8fdc-6ec4e17098cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:55.905875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=100 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:48:55.933124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-605518e4-cdb6-4dc5-8fdc-6ec4e17098cc None None] MFA Rules not processed for user `c4dc51e2b4474d6ba9fd5d2bc3702826`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:48:55.934364 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-605518e4-cdb6-4dc5-8fdc-6ec4e17098cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:48:56.007312 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4427/8920] 10.222.0.17 () {58 vars in 1040 bytes} [Sat Oct 1 21:48:55 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:48:56.013627 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a27ed-97fd-4f30-980a-1b9cc66c57cf None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:48:56.026133 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a27ed-97fd-4f30-980a-1b9cc66c57cf None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:48:56.031242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0d864-952f-449e-96e6-be349c6088b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:48:56.031853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0d864-952f-449e-96e6-be349c6088b4 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:48:56.032137 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0d864-952f-449e-96e6-be349c6088b4 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:48:56.032409 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0d864-952f-449e-96e6-be349c6088b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:48:56.041074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0d864-952f-449e-96e6-be349c6088b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:48:56.042878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0d864-952f-449e-96e6-be349c6088b4 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:48:56.051466 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4494/8921] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:48:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 21:49:05.197254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c5d71-36fe-4b1e-b601-6658aaff5aa1 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:49:05.197812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c5d71-36fe-4b1e-b601-6658aaff5aa1 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:49:05.197812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c5d71-36fe-4b1e-b601-6658aaff5aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:49:05.225310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c64c5d71-36fe-4b1e-b601-6658aaff5aa1 None None] MFA Rules not processed for user `0d1d8f5e9d074c2a89f5cf7c874392ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:49:05.235294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c64c5d71-36fe-4b1e-b601-6658aaff5aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:49:05.247231 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4428/8922] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:49:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:49:05.513426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18742a4-c46e-4a4b-a396-58e89b9c23ee None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:49:05.514011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18742a4-c46e-4a4b-a396-58e89b9c23ee None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:49:05.514011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18742a4-c46e-4a4b-a396-58e89b9c23ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:49:05.533772 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b18742a4-c46e-4a4b-a396-58e89b9c23ee None None] MFA Rules not processed for user `e17a026f305248878f18eec35e1e02dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:49:05.535007 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b18742a4-c46e-4a4b-a396-58e89b9c23ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:49:05.585103 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4495/8923] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:49:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:49:05.596805 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c5d71-36fe-4b1e-b601-6658aaff5aa1 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:49:05.598232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c64c5d71-36fe-4b1e-b601-6658aaff5aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:49:05.616212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c5d71-36fe-4b1e-b601-6658aaff5aa1 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:49:05.619667 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14208a45-731e-42dd-81e9-1ae3d15cf31f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:49:05.620253 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14208a45-731e-42dd-81e9-1ae3d15cf31f service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:49:05.620531 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14208a45-731e-42dd-81e9-1ae3d15cf31f service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:49:05.620803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14208a45-731e-42dd-81e9-1ae3d15cf31f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:49:05.628402 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14208a45-731e-42dd-81e9-1ae3d15cf31f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:49:05.629831 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14208a45-731e-42dd-81e9-1ae3d15cf31f service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:49:05.637713 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4429/8924] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:49:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 21:49:07.151478 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c6868-d015-4609-85da-95bea04a63c1 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:49:07.151952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c6868-d015-4609-85da-95bea04a63c1 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:49:07.152057 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c6868-d015-4609-85da-95bea04a63c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:49:07.175568 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-432c6868-d015-4609-85da-95bea04a63c1 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:49:07.176809 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-432c6868-d015-4609-85da-95bea04a63c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:49:07.184300 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4496/8925] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:49:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:49:07.196711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14208a45-731e-42dd-81e9-1ae3d15cf31f service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:49:07.209203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14208a45-731e-42dd-81e9-1ae3d15cf31f service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:49:07.214295 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322baa28-955f-488b-b0c2-c0d66c1ccedf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:49:07.214927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322baa28-955f-488b-b0c2-c0d66c1ccedf service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:49:07.215211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322baa28-955f-488b-b0c2-c0d66c1ccedf service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:49:07.215532 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322baa28-955f-488b-b0c2-c0d66c1ccedf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:49:07.224284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322baa28-955f-488b-b0c2-c0d66c1ccedf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:49:07.225929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322baa28-955f-488b-b0c2-c0d66c1ccedf service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:49:07.234421 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4430/8926] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:49:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:49:17.169900 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=44 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:49:17.231141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=51 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:49:28.491279 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c6868-d015-4609-85da-95bea04a63c1 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:49:28.507255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c6868-d015-4609-85da-95bea04a63c1 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:49:28.510879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275ce95-34e2-424d-9cfc-c34409d75c63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:49:28.511480 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275ce95-34e2-424d-9cfc-c34409d75c63 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:49:28.511757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275ce95-34e2-424d-9cfc-c34409d75c63 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:49:28.512101 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275ce95-34e2-424d-9cfc-c34409d75c63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:49:28.520991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275ce95-34e2-424d-9cfc-c34409d75c63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:49:28.522418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275ce95-34e2-424d-9cfc-c34409d75c63 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:49:28.531883 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4497/8927] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:49:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:49:38.527651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:50:22.456554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322baa28-955f-488b-b0c2-c0d66c1ccedf service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:50:22.458035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-322baa28-955f-488b-b0c2-c0d66c1ccedf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:50:22.524207 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322baa28-955f-488b-b0c2-c0d66c1ccedf service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:50:22.529155 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276ea06-cf8b-401c-a54f-dc2df1590905 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:50:22.529748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276ea06-cf8b-401c-a54f-dc2df1590905 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:50:22.530026 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276ea06-cf8b-401c-a54f-dc2df1590905 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:50:22.530297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276ea06-cf8b-401c-a54f-dc2df1590905 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:50:22.532004 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6276ea06-cf8b-401c-a54f-dc2df1590905 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:50:22.542863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276ea06-cf8b-401c-a54f-dc2df1590905 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:50:22.544435 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276ea06-cf8b-401c-a54f-dc2df1590905 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:50:22.554636 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4431/8928] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:50:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:50:23.527302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275ce95-34e2-424d-9cfc-c34409d75c63 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:50:23.538417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275ce95-34e2-424d-9cfc-c34409d75c63 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:50:23.543562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06a7cf-ab3e-4f8f-a0f7-cd274d628f0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:50:23.544156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06a7cf-ab3e-4f8f-a0f7-cd274d628f0d service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:50:23.544438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06a7cf-ab3e-4f8f-a0f7-cd274d628f0d service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:50:23.544713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06a7cf-ab3e-4f8f-a0f7-cd274d628f0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:50:23.546435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4c06a7cf-ab3e-4f8f-a0f7-cd274d628f0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:50:23.557992 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c06a7cf-ab3e-4f8f-a0f7-cd274d628f0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:50:23.559621 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c06a7cf-ab3e-4f8f-a0f7-cd274d628f0d service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:50:23.570155 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4498/8929] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:50:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:50:32.549864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=32 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:50:33.565432 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:52:30.255923 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276ea06-cf8b-401c-a54f-dc2df1590905 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:52:30.266033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276ea06-cf8b-401c-a54f-dc2df1590905 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:52:30.269666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff492455-9c23-4aa1-b5e7-19cb7417b8ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:52:30.270263 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff492455-9c23-4aa1-b5e7-19cb7417b8ce service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:52:30.270575 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff492455-9c23-4aa1-b5e7-19cb7417b8ce service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:52:30.270849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff492455-9c23-4aa1-b5e7-19cb7417b8ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:52:30.272562 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ff492455-9c23-4aa1-b5e7-19cb7417b8ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:52:30.294504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff492455-9c23-4aa1-b5e7-19cb7417b8ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:52:30.296042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff492455-9c23-4aa1-b5e7-19cb7417b8ce service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:52:30.304819 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4432/8930] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:52:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:52:40.300633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=11 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:53:05.398286 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06a7cf-ab3e-4f8f-a0f7-cd274d628f0d service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:53:05.407604 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06a7cf-ab3e-4f8f-a0f7-cd274d628f0d service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:53:05.411282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3e5fb-10ab-432a-96cd-80be84b7123b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:53:05.411878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3e5fb-10ab-432a-96cd-80be84b7123b service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:53:05.412160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3e5fb-10ab-432a-96cd-80be84b7123b service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:53:05.412435 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3e5fb-10ab-432a-96cd-80be84b7123b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:53:05.414200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-25d3e5fb-10ab-432a-96cd-80be84b7123b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:53:05.497422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3e5fb-10ab-432a-96cd-80be84b7123b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:53:05.498883 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3e5fb-10ab-432a-96cd-80be84b7123b service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:53:05.510341 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4499/8931] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:53:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:53:05.929127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff492455-9c23-4aa1-b5e7-19cb7417b8ce service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:53:05.930654 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ff492455-9c23-4aa1-b5e7-19cb7417b8ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:53:05.951502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff492455-9c23-4aa1-b5e7-19cb7417b8ce service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:53:05.957637 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53bb22-82b6-4f2b-8d59-39bed2491a58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:53:05.958465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53bb22-82b6-4f2b-8d59-39bed2491a58 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:53:05.958852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53bb22-82b6-4f2b-8d59-39bed2491a58 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:53:05.959211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53bb22-82b6-4f2b-8d59-39bed2491a58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:53:05.961268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ba53bb22-82b6-4f2b-8d59-39bed2491a58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:53:05.975446 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53bb22-82b6-4f2b-8d59-39bed2491a58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:53:05.978226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53bb22-82b6-4f2b-8d59-39bed2491a58 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:53:06.003109 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4433/8932] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:53:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:53:10.994611 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3e5fb-10ab-432a-96cd-80be84b7123b service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:53:11.003625 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3e5fb-10ab-432a-96cd-80be84b7123b service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:53:11.007299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcf372-b647-4aa1-aee0-1efafaa8a9e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:53:11.007889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcf372-b647-4aa1-aee0-1efafaa8a9e4 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:53:11.008661 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcf372-b647-4aa1-aee0-1efafaa8a9e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:53:11.008938 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcf372-b647-4aa1-aee0-1efafaa8a9e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:53:11.011771 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-81fcf372-b647-4aa1-aee0-1efafaa8a9e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:53:11.026240 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcf372-b647-4aa1-aee0-1efafaa8a9e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:53:11.027836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcf372-b647-4aa1-aee0-1efafaa8a9e4 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:53:11.042064 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4500/8933] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:53:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:53:11.771276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53bb22-82b6-4f2b-8d59-39bed2491a58 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:53:11.773072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ba53bb22-82b6-4f2b-8d59-39bed2491a58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:53:11.792534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53bb22-82b6-4f2b-8d59-39bed2491a58 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:53:11.801008 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12dd47-2274-45cf-8c58-423c695dfd59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:53:11.802290 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12dd47-2274-45cf-8c58-423c695dfd59 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:53:11.802803 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12dd47-2274-45cf-8c58-423c695dfd59 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:53:11.803249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12dd47-2274-45cf-8c58-423c695dfd59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:53:11.806242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3b12dd47-2274-45cf-8c58-423c695dfd59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:53:11.819075 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12dd47-2274-45cf-8c58-423c695dfd59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:53:11.820954 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12dd47-2274-45cf-8c58-423c695dfd59 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:53:11.831603 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4434/8934] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:53:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:53:13.401745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622561d5-e5ce-41ee-9f91-04fe540245a0 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:53:13.402187 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622561d5-e5ce-41ee-9f91-04fe540245a0 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:53:13.402298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622561d5-e5ce-41ee-9f91-04fe540245a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:53:13.432499 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-622561d5-e5ce-41ee-9f91-04fe540245a0 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:53:13.433698 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-622561d5-e5ce-41ee-9f91-04fe540245a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:53:13.440858 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4501/8935] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:53:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:53:13.491036 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12dd47-2274-45cf-8c58-423c695dfd59 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:53:13.502120 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12dd47-2274-45cf-8c58-423c695dfd59 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:53:13.507136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752ba8d-8468-4d3e-b184-3a141f92db2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:53:13.507730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752ba8d-8468-4d3e-b184-3a141f92db2d service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:53:13.508032 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752ba8d-8468-4d3e-b184-3a141f92db2d service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:53:13.508311 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752ba8d-8468-4d3e-b184-3a141f92db2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:53:13.516168 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752ba8d-8468-4d3e-b184-3a141f92db2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:53:13.517733 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752ba8d-8468-4d3e-b184-3a141f92db2d service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:53:13.526255 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4435/8936] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:53:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:53:14.039743 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3e5bc-f2a1-4dec-807b-0e01d7f86165 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:53:14.040183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3e5bc-f2a1-4dec-807b-0e01d7f86165 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:53:14.040534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3e5bc-f2a1-4dec-807b-0e01d7f86165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:53:14.060333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9fe3e5bc-f2a1-4dec-807b-0e01d7f86165 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:53:14.061531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9fe3e5bc-f2a1-4dec-807b-0e01d7f86165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:53:14.068942 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4502/8937] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:53:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:53:14.081571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752ba8d-8468-4d3e-b184-3a141f92db2d service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:53:14.092370 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752ba8d-8468-4d3e-b184-3a141f92db2d service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:53:14.097353 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81860bd-5045-4122-9e55-ce2bece9a4e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:53:14.097951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81860bd-5045-4122-9e55-ce2bece9a4e8 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:53:14.098222 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81860bd-5045-4122-9e55-ce2bece9a4e8 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:53:14.098524 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81860bd-5045-4122-9e55-ce2bece9a4e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:53:14.106467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81860bd-5045-4122-9e55-ce2bece9a4e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:53:14.108191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81860bd-5045-4122-9e55-ce2bece9a4e8 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:53:14.116538 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4436/8938] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:53:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:53:23.836042 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28ce55-ee85-4cc5-bc46-f1d8131ec418 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:53:23.836601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28ce55-ee85-4cc5-bc46-f1d8131ec418 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:53:23.836601 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28ce55-ee85-4cc5-bc46-f1d8131ec418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:53:23.855587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6f28ce55-ee85-4cc5-bc46-f1d8131ec418 None None] MFA Rules not processed for user `0d1d8f5e9d074c2a89f5cf7c874392ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:53:23.856791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6f28ce55-ee85-4cc5-bc46-f1d8131ec418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:53:23.863659 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4503/8939] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:53:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:53:24.113110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=24 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:53:26.465679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81860bd-5045-4122-9e55-ce2bece9a4e8 service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:53:26.487419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81860bd-5045-4122-9e55-ce2bece9a4e8 service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:53:26.492417 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488bae0-b436-47d9-b910-f3729af59165 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:53:26.493058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488bae0-b436-47d9-b910-f3729af59165 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:53:26.493636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488bae0-b436-47d9-b910-f3729af59165 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:53:26.493959 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488bae0-b436-47d9-b910-f3729af59165 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:53:26.496310 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3488bae0-b436-47d9-b910-f3729af59165 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:53:26.508098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488bae0-b436-47d9-b910-f3729af59165 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:53:26.509669 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488bae0-b436-47d9-b910-f3729af59165 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:53:26.520340 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4437/8940] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:53:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:53:30.730912 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c6544-56a6-4d6f-b3ba-a1d1b25e6b4d None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:53:30.731419 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c6544-56a6-4d6f-b3ba-a1d1b25e6b4d None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:53:30.731526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c6544-56a6-4d6f-b3ba-a1d1b25e6b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:53:30.753357 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-b31c6544-56a6-4d6f-b3ba-a1d1b25e6b4d None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:53:30.754620 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b31c6544-56a6-4d6f-b3ba-a1d1b25e6b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:53:30.761873 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4504/8941] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 21:53:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:53:30.774776 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488bae0-b436-47d9-b910-f3729af59165 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:53:30.785841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488bae0-b436-47d9-b910-f3729af59165 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:53:30.790823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee6b58-a228-474a-aad2-8288ebe4a5fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:53:30.791416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee6b58-a228-474a-aad2-8288ebe4a5fa service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:53:30.791695 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee6b58-a228-474a-aad2-8288ebe4a5fa service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:53:30.791973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee6b58-a228-474a-aad2-8288ebe4a5fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:53:30.800584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee6b58-a228-474a-aad2-8288ebe4a5fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:53:30.802159 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee6b58-a228-474a-aad2-8288ebe4a5fa service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:53:30.811087 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4438/8942] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:53:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 21:53:40.748308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=61 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:53:40.807180 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=16 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:53:44.857756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c6544-56a6-4d6f-b3ba-a1d1b25e6b4d None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:53:44.887135 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c6544-56a6-4d6f-b3ba-a1d1b25e6b4d None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:53:44.893041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68720be1-9681-4974-820b-4941df976aac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:53:44.893657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68720be1-9681-4974-820b-4941df976aac service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:53:44.893937 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68720be1-9681-4974-820b-4941df976aac service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:53:44.894209 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68720be1-9681-4974-820b-4941df976aac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:53:44.895995 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-68720be1-9681-4974-820b-4941df976aac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:53:44.908908 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68720be1-9681-4974-820b-4941df976aac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:53:44.910522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68720be1-9681-4974-820b-4941df976aac service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:53:44.921382 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4505/8943] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:53:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:53:54.917479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=12 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:53:55.596166 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee6b58-a228-474a-aad2-8288ebe4a5fa service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:53:55.628283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee6b58-a228-474a-aad2-8288ebe4a5fa service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:53:55.636216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339f7b6-28f7-4888-852c-017fa583bce0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:53:55.637361 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339f7b6-28f7-4888-852c-017fa583bce0 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:53:55.637864 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339f7b6-28f7-4888-852c-017fa583bce0 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:53:55.638384 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339f7b6-28f7-4888-852c-017fa583bce0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:53:55.647996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9339f7b6-28f7-4888-852c-017fa583bce0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:53:55.649873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9339f7b6-28f7-4888-852c-017fa583bce0 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:53:55.661333 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4439/8944] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:53:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:54:05.656258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=12 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:55:14.570082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c56f0-4e7e-4d62-89cd-1697419fb006 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:14.570623 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c56f0-4e7e-4d62-89cd-1697419fb006 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:14.570737 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c56f0-4e7e-4d62-89cd-1697419fb006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:14.590293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9c2c56f0-4e7e-4d62-89cd-1697419fb006 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:55:14.591513 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9c2c56f0-4e7e-4d62-89cd-1697419fb006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:55:14.601497 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4506/8945] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 21:55:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:55:14.642602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339f7b6-28f7-4888-852c-017fa583bce0 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:14.663731 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339f7b6-28f7-4888-852c-017fa583bce0 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:14.668799 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30553cf0-6004-4ef1-a08c-7c026b228ece service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:14.669394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30553cf0-6004-4ef1-a08c-7c026b228ece service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:14.669675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30553cf0-6004-4ef1-a08c-7c026b228ece service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:14.669948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30553cf0-6004-4ef1-a08c-7c026b228ece service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:14.678230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30553cf0-6004-4ef1-a08c-7c026b228ece service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:14.679858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30553cf0-6004-4ef1-a08c-7c026b228ece service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:14.688520 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4440/8946] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 21:55:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 21:55:15.031482 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c56f0-4e7e-4d62-89cd-1697419fb006 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:15.051283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c56f0-4e7e-4d62-89cd-1697419fb006 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:15.055263 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e84ac-fd5a-4903-bd9b-4abbc73abf5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:15.055867 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e84ac-fd5a-4903-bd9b-4abbc73abf5f service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:15.056152 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e84ac-fd5a-4903-bd9b-4abbc73abf5f service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:15.056430 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e84ac-fd5a-4903-bd9b-4abbc73abf5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:15.127471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e84ac-fd5a-4903-bd9b-4abbc73abf5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:15.128903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e84ac-fd5a-4903-bd9b-4abbc73abf5f service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:15.138564 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4507/8947] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:55:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:55:18.295276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe972a-85d7-4399-907b-7800b76b562e None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:18.295789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe972a-85d7-4399-907b-7800b76b562e None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:18.295886 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe972a-85d7-4399-907b-7800b76b562e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:18.349593 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7bfe972a-85d7-4399-907b-7800b76b562e None None] MFA Rules not processed for user `1e5f6284cd7d4973adffefa98291dbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:55:18.352170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7bfe972a-85d7-4399-907b-7800b76b562e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:55:18.361809 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4441/8948] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 21:55:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:55:18.378397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e84ac-fd5a-4903-bd9b-4abbc73abf5f service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:18.394457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e84ac-fd5a-4903-bd9b-4abbc73abf5f service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:18.399786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a22d16-7f20-4b26-9cfb-6a35c4f886fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:18.400527 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a22d16-7f20-4b26-9cfb-6a35c4f886fc service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:18.401333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a22d16-7f20-4b26-9cfb-6a35c4f886fc service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:18.401615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a22d16-7f20-4b26-9cfb-6a35c4f886fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:18.410417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a22d16-7f20-4b26-9cfb-6a35c4f886fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:18.412034 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a22d16-7f20-4b26-9cfb-6a35c4f886fc service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:18.423667 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4508/8949] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:55:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:55:21.492627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe972a-85d7-4399-907b-7800b76b562e None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:21.506842 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe972a-85d7-4399-907b-7800b76b562e None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:21.512840 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e206dc-a8f5-4af2-91d9-24a7c8c7de67 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:21.513529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e206dc-a8f5-4af2-91d9-24a7c8c7de67 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:21.513815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e206dc-a8f5-4af2-91d9-24a7c8c7de67 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:21.514090 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e206dc-a8f5-4af2-91d9-24a7c8c7de67 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:21.516040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-63e206dc-a8f5-4af2-91d9-24a7c8c7de67 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:55:21.541472 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e206dc-a8f5-4af2-91d9-24a7c8c7de67 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:21.545681 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e206dc-a8f5-4af2-91d9-24a7c8c7de67 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:21.555398 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4442/8950] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:55:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:55:21.575694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a22d16-7f20-4b26-9cfb-6a35c4f886fc service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:21.587048 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a22d16-7f20-4b26-9cfb-6a35c4f886fc service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:21.593172 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cf8fd-5ffc-4fce-a95e-f7d1ac633021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:21.593836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf8fd-5ffc-4fce-a95e-f7d1ac633021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:21.594398 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf8fd-5ffc-4fce-a95e-f7d1ac633021 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:21.594795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf8fd-5ffc-4fce-a95e-f7d1ac633021 admin admin] PATH_INFO: `/v3/users/0d1d8f5e9d074c2a89f5cf7c874392ac` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:21.596366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cf8fd-5ffc-4fce-a95e-f7d1ac633021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d1d8f5e9d074c2a89f5cf7c874392ac)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:21.598569 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cf8fd-5ffc-4fce-a95e-f7d1ac633021 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:21.646795 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e4cf8fd-5ffc-4fce-a95e-f7d1ac633021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d1d8f5e9d074c2a89f5cf7c874392ac'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:55:21.654783 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e4cf8fd-5ffc-4fce-a95e-f7d1ac633021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d1d8f5e9d074c2a89f5cf7c874392ac'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:55:21.666847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e4cf8fd-5ffc-4fce-a95e-f7d1ac633021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d1d8f5e9d074c2a89f5cf7c874392ac'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:55:21.667540 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e4cf8fd-5ffc-4fce-a95e-f7d1ac633021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d1d8f5e9d074c2a89f5cf7c874392ac'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:55:21.674768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e4cf8fd-5ffc-4fce-a95e-f7d1ac633021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d1d8f5e9d074c2a89f5cf7c874392ac'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:55:21.681110 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4509/8951] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:55:21 2022] DELETE /identity/v3/users/0d1d8f5e9d074c2a89f5cf7c874392ac => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:55:21.854769 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e206dc-a8f5-4af2-91d9-24a7c8c7de67 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:21.865785 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e206dc-a8f5-4af2-91d9-24a7c8c7de67 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:21.925138 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f57fee-e2c8-4ab6-9395-7ac94bb836fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:21.925767 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f57fee-e2c8-4ab6-9395-7ac94bb836fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:21.926054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f57fee-e2c8-4ab6-9395-7ac94bb836fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:21.926329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f57fee-e2c8-4ab6-9395-7ac94bb836fb admin admin] PATH_INFO: `/v3/projects/f02c2d9ca9fa43bdad1329ff8583b742` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:21.927489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f57fee-e2c8-4ab6-9395-7ac94bb836fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f02c2d9ca9fa43bdad1329ff8583b742)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:21.929658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f57fee-e2c8-4ab6-9395-7ac94bb836fb admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:21.975841 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-40f57fee-e2c8-4ab6-9395-7ac94bb836fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f02c2d9ca9fa43bdad1329ff8583b742', 'request_id': 'req-40f57fee-e2c8-4ab6-9395-7ac94bb836fb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:55:21.983367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-40f57fee-e2c8-4ab6-9395-7ac94bb836fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f02c2d9ca9fa43bdad1329ff8583b742', 'request_id': 'req-40f57fee-e2c8-4ab6-9395-7ac94bb836fb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:55:21.989357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-40f57fee-e2c8-4ab6-9395-7ac94bb836fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f02c2d9ca9fa43bdad1329ff8583b742', 'request_id': 'req-40f57fee-e2c8-4ab6-9395-7ac94bb836fb'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:55:22.013479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-40f57fee-e2c8-4ab6-9395-7ac94bb836fb admin admin] The token cache is being invalidate because project f02c2d9ca9fa43bdad1329ff8583b742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:55:22.013768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-40f57fee-e2c8-4ab6-9395-7ac94bb836fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:55:22.014444 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-40f57fee-e2c8-4ab6-9395-7ac94bb836fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:55:22.015998 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4443/8952] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:55:21 2022] DELETE /identity/v3/projects/f02c2d9ca9fa43bdad1329ff8583b742 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:55:22.020053 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cf8fd-5ffc-4fce-a95e-f7d1ac633021 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:22.021483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9e4cf8fd-5ffc-4fce-a95e-f7d1ac633021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:55:22.034681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cf8fd-5ffc-4fce-a95e-f7d1ac633021 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:22.055578 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:55:22.094008 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8bc393-86b3-40ff-9f1a-d4bcafdb6ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:22.094682 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bc393-86b3-40ff-9f1a-d4bcafdb6ef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:22.094968 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bc393-86b3-40ff-9f1a-d4bcafdb6ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:22.095247 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bc393-86b3-40ff-9f1a-d4bcafdb6ef0 admin admin] PATH_INFO: `/v3/users/7b2fcde4a26c4f3db34451225b025012` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:22.096390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8bc393-86b3-40ff-9f1a-d4bcafdb6ef0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b2fcde4a26c4f3db34451225b025012)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:22.098352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8bc393-86b3-40ff-9f1a-d4bcafdb6ef0 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:22.144371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4d8bc393-86b3-40ff-9f1a-d4bcafdb6ef0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b2fcde4a26c4f3db34451225b025012'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:55:22.151534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4d8bc393-86b3-40ff-9f1a-d4bcafdb6ef0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b2fcde4a26c4f3db34451225b025012'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:55:22.163214 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4d8bc393-86b3-40ff-9f1a-d4bcafdb6ef0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b2fcde4a26c4f3db34451225b025012'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:55:22.163869 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4d8bc393-86b3-40ff-9f1a-d4bcafdb6ef0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b2fcde4a26c4f3db34451225b025012'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:55:22.171717 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-4d8bc393-86b3-40ff-9f1a-d4bcafdb6ef0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b2fcde4a26c4f3db34451225b025012'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:55:22.178126 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4510/8953] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 21:55:22 2022] DELETE /identity/v3/users/7b2fcde4a26c4f3db34451225b025012 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:55:22.358164 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f57fee-e2c8-4ab6-9395-7ac94bb836fb admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:22.359652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-40f57fee-e2c8-4ab6-9395-7ac94bb836fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:55:22.375859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f57fee-e2c8-4ab6-9395-7ac94bb836fb admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:22.385448 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:55:22.439336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acee59-5f3b-4ce0-babb-d03ae12e05a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:22.440033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acee59-5f3b-4ce0-babb-d03ae12e05a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:22.440297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acee59-5f3b-4ce0-babb-d03ae12e05a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:22.440529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acee59-5f3b-4ce0-babb-d03ae12e05a7 admin admin] PATH_INFO: `/v3/projects/de76c6ed429545bbb77bc7cce505e0da` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:22.441837 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8acee59-5f3b-4ce0-babb-d03ae12e05a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de76c6ed429545bbb77bc7cce505e0da)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:22.443777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8acee59-5f3b-4ce0-babb-d03ae12e05a7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:22.489307 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b8acee59-5f3b-4ce0-babb-d03ae12e05a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de76c6ed429545bbb77bc7cce505e0da', 'request_id': 'req-b8acee59-5f3b-4ce0-babb-d03ae12e05a7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:55:22.496938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b8acee59-5f3b-4ce0-babb-d03ae12e05a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de76c6ed429545bbb77bc7cce505e0da', 'request_id': 'req-b8acee59-5f3b-4ce0-babb-d03ae12e05a7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:55:22.502679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b8acee59-5f3b-4ce0-babb-d03ae12e05a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de76c6ed429545bbb77bc7cce505e0da', 'request_id': 'req-b8acee59-5f3b-4ce0-babb-d03ae12e05a7'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:55:22.513727 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b8acee59-5f3b-4ce0-babb-d03ae12e05a7 admin admin] The token cache is being invalidate because project de76c6ed429545bbb77bc7cce505e0da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 21:55:22.514022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b8acee59-5f3b-4ce0-babb-d03ae12e05a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:55:22.514700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-b8acee59-5f3b-4ce0-babb-d03ae12e05a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 21:55:22.516205 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4444/8954] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 21:55:22 2022] DELETE /identity/v3/projects/de76c6ed429545bbb77bc7cce505e0da => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:55:22.521344 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0b3a9-b86b-4173-b88a-fe9c7a90335b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:22.521636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0b3a9-b86b-4173-b88a-fe9c7a90335b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:22.521916 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0b3a9-b86b-4173-b88a-fe9c7a90335b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:22.570590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-cbe0b3a9-b86b-4173-b88a-fe9c7a90335b None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:55:22.571807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cbe0b3a9-b86b-4173-b88a-fe9c7a90335b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:55:22.630449 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4511/8955] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:55:22.638318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911251d-78f4-400a-b41c-4d362a104f67 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:22.638656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911251d-78f4-400a-b41c-4d362a104f67 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:22.638940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911251d-78f4-400a-b41c-4d362a104f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:22.658815 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-3911251d-78f4-400a-b41c-4d362a104f67 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:55:22.660022 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3911251d-78f4-400a-b41c-4d362a104f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:55:22.668534 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4445/8956] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:55:22.672584 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0b3a9-b86b-4173-b88a-fe9c7a90335b None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:22.682839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0b3a9-b86b-4173-b88a-fe9c7a90335b None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:22.687439 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b58874-1e29-44f7-a082-f18240cf3648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:22.688032 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b58874-1e29-44f7-a082-f18240cf3648 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:22.688341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b58874-1e29-44f7-a082-f18240cf3648 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:22.688616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b58874-1e29-44f7-a082-f18240cf3648 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:22.689255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b58874-1e29-44f7-a082-f18240cf3648 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 21:55:22.689560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b58874-1e29-44f7-a082-f18240cf3648 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:22.691190 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b58874-1e29-44f7-a082-f18240cf3648 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:22.702824 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4512/8957] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 21:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 21:55:22.706709 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911251d-78f4-400a-b41c-4d362a104f67 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:22.716479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911251d-78f4-400a-b41c-4d362a104f67 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:22.721165 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9922f91-e90c-4ff5-878e-13c3c5d04015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:22.721753 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9922f91-e90c-4ff5-878e-13c3c5d04015 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:22.722038 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9922f91-e90c-4ff5-878e-13c3c5d04015 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:22.722314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9922f91-e90c-4ff5-878e-13c3c5d04015 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:22.722977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9922f91-e90c-4ff5-878e-13c3c5d04015 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:22.724875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9922f91-e90c-4ff5-878e-13c3c5d04015 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:22.755919 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4446/8958] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:55:22 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:55:22.756958 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b58874-1e29-44f7-a082-f18240cf3648 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:22.767581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b58874-1e29-44f7-a082-f18240cf3648 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:22.772350 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:55:22.827429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496f582-7fae-46f1-9267-ca14bc7e4d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:22.828088 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f582-7fae-46f1-9267-ca14bc7e4d9d admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:22.828402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f582-7fae-46f1-9267-ca14bc7e4d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:22.828681 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f582-7fae-46f1-9267-ca14bc7e4d9d admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:22.829316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496f582-7fae-46f1-9267-ca14bc7e4d9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:22.831332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496f582-7fae-46f1-9267-ca14bc7e4d9d admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:22.852915 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4513/8959] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:55:22 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:55:22.857682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9922f91-e90c-4ff5-878e-13c3c5d04015 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:22.868018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9922f91-e90c-4ff5-878e-13c3c5d04015 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:22.872682 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434111b-fdd8-4c02-9929-aff992c40a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:22.873276 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434111b-fdd8-4c02-9929-aff992c40a98 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:22.873556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434111b-fdd8-4c02-9929-aff992c40a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:22.873855 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434111b-fdd8-4c02-9929-aff992c40a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:22.874477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434111b-fdd8-4c02-9929-aff992c40a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:22.876136 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434111b-fdd8-4c02-9929-aff992c40a98 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:22.886260 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4447/8960] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:55:22.890366 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496f582-7fae-46f1-9267-ca14bc7e4d9d admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:22.901744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496f582-7fae-46f1-9267-ca14bc7e4d9d admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:22.906590 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c63a4-7cdd-4330-8310-18b2a14292cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:22.907196 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c63a4-7cdd-4330-8310-18b2a14292cd admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:22.907476 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c63a4-7cdd-4330-8310-18b2a14292cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:22.907752 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c63a4-7cdd-4330-8310-18b2a14292cd admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:22.908392 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c63a4-7cdd-4330-8310-18b2a14292cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:22.909981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c63a4-7cdd-4330-8310-18b2a14292cd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:22.921575 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4514/8961] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 21:55:22.925730 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434111b-fdd8-4c02-9929-aff992c40a98 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:22.936556 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434111b-fdd8-4c02-9929-aff992c40a98 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:22.941332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1569fa1-bdb7-44cc-8aa4-83a8719d980e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:22.942006 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1569fa1-bdb7-44cc-8aa4-83a8719d980e admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:22.942285 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1569fa1-bdb7-44cc-8aa4-83a8719d980e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:22.942602 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1569fa1-bdb7-44cc-8aa4-83a8719d980e admin admin] PATH_INFO: `/v3/projects/27214fca5a7a4c85bba526c092ac726b/users/5f22070464e045f19acc335de7dd38f3/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:22.957238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1569fa1-bdb7-44cc-8aa4-83a8719d980e admin admin] RBAC: Authorizing `identity:create_grant(project_id=27214fca5a7a4c85bba526c092ac726b, user_id=5f22070464e045f19acc335de7dd38f3, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:22.960040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1569fa1-bdb7-44cc-8aa4-83a8719d980e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:22.972772 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4448/8962] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:55:22 2022] PUT /identity/v3/projects/27214fca5a7a4c85bba526c092ac726b/users/5f22070464e045f19acc335de7dd38f3/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:55:22.983293 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e077bfa-8e7e-4030-89ef-77d466594336 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:22.983616 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e077bfa-8e7e-4030-89ef-77d466594336 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:22.983896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e077bfa-8e7e-4030-89ef-77d466594336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:23.013367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2e077bfa-8e7e-4030-89ef-77d466594336 None None] MFA Rules not processed for user `5f22070464e045f19acc335de7dd38f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:55:23.014639 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2e077bfa-8e7e-4030-89ef-77d466594336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:55:23.081085 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4515/8963] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:55:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:55:23.085629 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1569fa1-bdb7-44cc-8aa4-83a8719d980e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:23.127251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:55:23.148390 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1569fa1-bdb7-44cc-8aa4-83a8719d980e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:23.153801 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03902b84-20fd-4811-931a-29a40f41a4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:23.154408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03902b84-20fd-4811-931a-29a40f41a4ee admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:23.154722 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03902b84-20fd-4811-931a-29a40f41a4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:23.155003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03902b84-20fd-4811-931a-29a40f41a4ee admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:23.155639 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03902b84-20fd-4811-931a-29a40f41a4ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:23.157551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03902b84-20fd-4811-931a-29a40f41a4ee admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:23.183460 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4449/8964] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 21:55:23 2022] POST /identity/v3/projects => generated 393 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 21:55:23.187642 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e077bfa-8e7e-4030-89ef-77d466594336 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:23.198979 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e077bfa-8e7e-4030-89ef-77d466594336 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:23.255558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25203e3c-7269-42ba-b61a-312436398d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:23.256163 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25203e3c-7269-42ba-b61a-312436398d12 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:23.256446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25203e3c-7269-42ba-b61a-312436398d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:23.256721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25203e3c-7269-42ba-b61a-312436398d12 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:23.257351 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25203e3c-7269-42ba-b61a-312436398d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:23.259296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25203e3c-7269-42ba-b61a-312436398d12 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:23.284021 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4516/8965] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 21:55:23 2022] POST /identity/v3/users => generated 303 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 21:55:23.285328 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03902b84-20fd-4811-931a-29a40f41a4ee admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:23.295789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03902b84-20fd-4811-931a-29a40f41a4ee admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:23.300450 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d78914-e5b4-4d5f-a820-83893f68475a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:23.301048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d78914-e5b4-4d5f-a820-83893f68475a admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:23.301329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d78914-e5b4-4d5f-a820-83893f68475a admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:23.301603 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d78914-e5b4-4d5f-a820-83893f68475a admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:23.302234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d78914-e5b4-4d5f-a820-83893f68475a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:23.304167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d78914-e5b4-4d5f-a820-83893f68475a admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:23.314617 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4450/8966] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:55:23.318778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25203e3c-7269-42ba-b61a-312436398d12 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:23.329212 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25203e3c-7269-42ba-b61a-312436398d12 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:23.333840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca9f4e-efd0-4ceb-b8c0-73d95f753dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:23.334483 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca9f4e-efd0-4ceb-b8c0-73d95f753dd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:23.334791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca9f4e-efd0-4ceb-b8c0-73d95f753dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:23.335068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca9f4e-efd0-4ceb-b8c0-73d95f753dd3 admin admin] PATH_INFO: `/v3/projects/f3ce989dddd04ea1970278b7ca43410a/users/0cdaf8c5333a4c4b8aa9857e53e2936c/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:23.337237 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:55:23.353706 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca9f4e-efd0-4ceb-b8c0-73d95f753dd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3ce989dddd04ea1970278b7ca43410a, user_id=0cdaf8c5333a4c4b8aa9857e53e2936c, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:23.356423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca9f4e-efd0-4ceb-b8c0-73d95f753dd3 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:23.368912 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4517/8967] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:55:23 2022] PUT /identity/v3/projects/f3ce989dddd04ea1970278b7ca43410a/users/0cdaf8c5333a4c4b8aa9857e53e2936c/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 21:55:23.373399 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d78914-e5b4-4d5f-a820-83893f68475a admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:23.383178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d78914-e5b4-4d5f-a820-83893f68475a admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:23.439151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9c527-7b0b-4bfd-89e0-7d4a1bc3a5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:23.439756 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9c527-7b0b-4bfd-89e0-7d4a1bc3a5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:23.440030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9c527-7b0b-4bfd-89e0-7d4a1bc3a5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:23.440365 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9c527-7b0b-4bfd-89e0-7d4a1bc3a5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:23.440926 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9c527-7b0b-4bfd-89e0-7d4a1bc3a5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:23.442534 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9c527-7b0b-4bfd-89e0-7d4a1bc3a5d4 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:23.452889 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4451/8968] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 21:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 21:55:23.457056 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca9f4e-efd0-4ceb-b8c0-73d95f753dd3 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:23.467497 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca9f4e-efd0-4ceb-b8c0-73d95f753dd3 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:23.472111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa61c9a-0735-4a22-8625-7c20fb6fc68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:23.472757 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa61c9a-0735-4a22-8625-7c20fb6fc68e admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:23.473036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa61c9a-0735-4a22-8625-7c20fb6fc68e admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:23.473308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa61c9a-0735-4a22-8625-7c20fb6fc68e admin admin] PATH_INFO: `/v3/projects/f3ce989dddd04ea1970278b7ca43410a/users/0cdaf8c5333a4c4b8aa9857e53e2936c/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:23.475451 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa61c9a-0735-4a22-8625-7c20fb6fc68e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3ce989dddd04ea1970278b7ca43410a, user_id=0cdaf8c5333a4c4b8aa9857e53e2936c, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:23.478084 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa61c9a-0735-4a22-8625-7c20fb6fc68e admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:23.488450 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4518/8969] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 21:55:23 2022] PUT /identity/v3/projects/f3ce989dddd04ea1970278b7ca43410a/users/0cdaf8c5333a4c4b8aa9857e53e2936c/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 21:55:23.499696 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f050d-4c66-4b05-a25e-68b7dc28d93d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:23.500214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f050d-4c66-4b05-a25e-68b7dc28d93d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:23.500315 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f050d-4c66-4b05-a25e-68b7dc28d93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:23.529054 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-048f050d-4c66-4b05-a25e-68b7dc28d93d None None] MFA Rules not processed for user `0cdaf8c5333a4c4b8aa9857e53e2936c`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:55:23.530305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-048f050d-4c66-4b05-a25e-68b7dc28d93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:55:23.598440 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4452/8970] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:55:23 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:55:23.612672 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa61c9a-0735-4a22-8625-7c20fb6fc68e admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:23.614371 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-caa61c9a-0735-4a22-8625-7c20fb6fc68e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:55:23.698115 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa61c9a-0735-4a22-8625-7c20fb6fc68e admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:23.703870 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6e37a-6026-46df-b239-f8db3aa24506 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:23.704467 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6e37a-6026-46df-b239-f8db3aa24506 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:23.704750 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6e37a-6026-46df-b239-f8db3aa24506 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:23.705024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6e37a-6026-46df-b239-f8db3aa24506 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:23.758715 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6e37a-6026-46df-b239-f8db3aa24506 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:23.760903 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6e37a-6026-46df-b239-f8db3aa24506 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:23.776583 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4519/8971] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:55:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:55:25.481675 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f050d-4c66-4b05-a25e-68b7dc28d93d None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:25.483191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-048f050d-4c66-4b05-a25e-68b7dc28d93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:55:25.523373 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:55:25.554203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f050d-4c66-4b05-a25e-68b7dc28d93d None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:25.560741 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d445da-8a02-45d9-b131-995d4fbf8f7d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:25.561358 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d445da-8a02-45d9-b131-995d4fbf8f7d service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:25.561640 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d445da-8a02-45d9-b131-995d4fbf8f7d service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:25.561913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d445da-8a02-45d9-b131-995d4fbf8f7d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:25.563736 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-22d445da-8a02-45d9-b131-995d4fbf8f7d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:55:25.575410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d445da-8a02-45d9-b131-995d4fbf8f7d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:25.577005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d445da-8a02-45d9-b131-995d4fbf8f7d service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:25.588742 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4453/8972] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:55:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:55:26.738806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6e37a-6026-46df-b239-f8db3aa24506 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:26.740230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-0eb6e37a-6026-46df-b239-f8db3aa24506 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:55:26.799764 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6e37a-6026-46df-b239-f8db3aa24506 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:26.803844 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640e862-d47c-4cf3-8106-3eaf225ba13c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:26.804526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640e862-d47c-4cf3-8106-3eaf225ba13c service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:26.804877 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640e862-d47c-4cf3-8106-3eaf225ba13c service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:26.805243 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640e862-d47c-4cf3-8106-3eaf225ba13c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:26.815457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640e862-d47c-4cf3-8106-3eaf225ba13c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:26.816889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640e862-d47c-4cf3-8106-3eaf225ba13c service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:26.822254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:55:26.828675 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4520/8973] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:55:26 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:55:28.418882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d445da-8a02-45d9-b131-995d4fbf8f7d service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:28.448927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d445da-8a02-45d9-b131-995d4fbf8f7d service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:28.456913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f07f1-a469-4b97-b318-a194ea4e0902 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:28.457511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f07f1-a469-4b97-b318-a194ea4e0902 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:28.457827 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f07f1-a469-4b97-b318-a194ea4e0902 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:28.458147 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f07f1-a469-4b97-b318-a194ea4e0902 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:28.460066 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e47f07f1-a469-4b97-b318-a194ea4e0902 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:55:28.478552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f07f1-a469-4b97-b318-a194ea4e0902 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:28.480720 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f07f1-a469-4b97-b318-a194ea4e0902 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:28.497972 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4454/8974] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:55:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:55:29.720872 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f566a-0e9e-47f2-ab5c-517cd81e5865 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:29.721342 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f566a-0e9e-47f2-ab5c-517cd81e5865 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:29.721433 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f566a-0e9e-47f2-ab5c-517cd81e5865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:29.762852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-cb6f566a-0e9e-47f2-ab5c-517cd81e5865 None None] MFA Rules not processed for user `5f22070464e045f19acc335de7dd38f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:55:29.764789 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-cb6f566a-0e9e-47f2-ab5c-517cd81e5865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:55:29.774018 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4521/8975] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:55:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:55:30.472691 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f07f1-a469-4b97-b318-a194ea4e0902 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:30.483016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f07f1-a469-4b97-b318-a194ea4e0902 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:30.486949 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6713bf5-1662-4fe8-b222-6deb0001d77a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:30.487550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6713bf5-1662-4fe8-b222-6deb0001d77a service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:30.487829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6713bf5-1662-4fe8-b222-6deb0001d77a service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:30.488100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6713bf5-1662-4fe8-b222-6deb0001d77a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:30.489896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-b6713bf5-1662-4fe8-b222-6deb0001d77a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:55:30.508453 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6713bf5-1662-4fe8-b222-6deb0001d77a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:30.510584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6713bf5-1662-4fe8-b222-6deb0001d77a service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:30.528402 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4455/8976] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:55:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:55:30.872010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f566a-0e9e-47f2-ab5c-517cd81e5865 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:55:30.887594 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f566a-0e9e-47f2-ab5c-517cd81e5865 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:55:30.892839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f26b8-4035-4dfb-973c-79d5a5b31f6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:55:30.893455 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f26b8-4035-4dfb-973c-79d5a5b31f6b service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:55:30.893740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f26b8-4035-4dfb-973c-79d5a5b31f6b service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:55:30.894074 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f26b8-4035-4dfb-973c-79d5a5b31f6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:55:30.895862 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-690f26b8-4035-4dfb-973c-79d5a5b31f6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:55:30.908036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f26b8-4035-4dfb-973c-79d5a5b31f6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:55:30.909608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f26b8-4035-4dfb-973c-79d5a5b31f6b service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:55:30.921356 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4522/8977] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:55:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:55:40.518331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=28 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:55:40.917754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=9 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:56:32.992034 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf5e3c-6075-4c56-944c-96aa267f6f1c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:56:32.992513 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf5e3c-6075-4c56-944c-96aa267f6f1c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:56:32.992713 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf5e3c-6075-4c56-944c-96aa267f6f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:56:33.019040 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ffbf5e3c-6075-4c56-944c-96aa267f6f1c None None] MFA Rules not processed for user `5f22070464e045f19acc335de7dd38f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:56:33.020511 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ffbf5e3c-6075-4c56-944c-96aa267f6f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:56:33.028668 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4456/8978] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:56:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:56:43.012718 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:57:38.228458 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f26b8-4035-4dfb-973c-79d5a5b31f6b service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:57:38.229996 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-690f26b8-4035-4dfb-973c-79d5a5b31f6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:57:38.267114 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f26b8-4035-4dfb-973c-79d5a5b31f6b service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:57:38.270693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40c564-65ff-4503-ba3c-02fe73a80bc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:57:38.271284 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40c564-65ff-4503-ba3c-02fe73a80bc9 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:57:38.271562 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40c564-65ff-4503-ba3c-02fe73a80bc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:57:38.271832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40c564-65ff-4503-ba3c-02fe73a80bc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:57:38.273501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4d40c564-65ff-4503-ba3c-02fe73a80bc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:57:38.284106 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40c564-65ff-4503-ba3c-02fe73a80bc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:57:38.285608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40c564-65ff-4503-ba3c-02fe73a80bc9 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:57:38.294132 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4523/8979] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:57:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:57:48.290059 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=17 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:58:13.943625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5eaccb-a07f-4ebb-b0c6-9ff958b7fb98 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:58:13.944178 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5eaccb-a07f-4ebb-b0c6-9ff958b7fb98 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:58:13.944533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5eaccb-a07f-4ebb-b0c6-9ff958b7fb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:58:13.966515 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-be5eaccb-a07f-4ebb-b0c6-9ff958b7fb98 None None] MFA Rules not processed for user `5f22070464e045f19acc335de7dd38f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:58:13.968419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-be5eaccb-a07f-4ebb-b0c6-9ff958b7fb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:58:13.979048 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4457/8980] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:58:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:58:14.025778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40c564-65ff-4503-ba3c-02fe73a80bc9 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:58:14.028746 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4d40c564-65ff-4503-ba3c-02fe73a80bc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:58:14.117521 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40c564-65ff-4503-ba3c-02fe73a80bc9 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:58:14.121579 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ec487-bba5-4ed8-8ff5-a957fd5a5d86 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:58:14.122194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ec487-bba5-4ed8-8ff5-a957fd5a5d86 service glance] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:58:14.122711 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ec487-bba5-4ed8-8ff5-a957fd5a5d86 service glance] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:58:14.123160 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ec487-bba5-4ed8-8ff5-a957fd5a5d86 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:58:14.125369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-286ec487-bba5-4ed8-8ff5-a957fd5a5d86 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:58:14.142484 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ec487-bba5-4ed8-8ff5-a957fd5a5d86 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:58:14.144156 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ec487-bba5-4ed8-8ff5-a957fd5a5d86 service glance] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:58:14.157623 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4524/8981] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:58:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:58:14.849211 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5eaccb-a07f-4ebb-b0c6-9ff958b7fb98 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:58:14.860357 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5eaccb-a07f-4ebb-b0c6-9ff958b7fb98 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:58:14.864392 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc507c56-459b-4adb-8d57-9a192f182970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:58:14.864996 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc507c56-459b-4adb-8d57-9a192f182970 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:58:14.865274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc507c56-459b-4adb-8d57-9a192f182970 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:58:14.865658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc507c56-459b-4adb-8d57-9a192f182970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:58:14.867674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-bc507c56-459b-4adb-8d57-9a192f182970 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:58:14.883149 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc507c56-459b-4adb-8d57-9a192f182970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:58:14.884732 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc507c56-459b-4adb-8d57-9a192f182970 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:58:14.900779 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4458/8982] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:58:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:58:17.354326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ec487-bba5-4ed8-8ff5-a957fd5a5d86 service glance] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:58:17.358256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-286ec487-bba5-4ed8-8ff5-a957fd5a5d86 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:58:17.379469 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ec487-bba5-4ed8-8ff5-a957fd5a5d86 service glance] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:58:17.388633 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a76ada-5c2c-446e-9336-8d050d46c818 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:58:17.389939 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a76ada-5c2c-446e-9336-8d050d46c818 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:58:17.390396 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a76ada-5c2c-446e-9336-8d050d46c818 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:58:17.390911 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a76ada-5c2c-446e-9336-8d050d46c818 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:58:17.393038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-11a76ada-5c2c-446e-9336-8d050d46c818 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:58:17.407960 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a76ada-5c2c-446e-9336-8d050d46c818 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:58:17.409845 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a76ada-5c2c-446e-9336-8d050d46c818 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:58:17.424237 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4525/8983] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:58:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:58:24.892394 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=9 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:58:27.419352 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=32 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:58:58.779246 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa99394-9e8f-4010-b721-b9d369841e4b None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:58:58.779847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa99394-9e8f-4010-b721-b9d369841e4b None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:58:58.779847 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa99394-9e8f-4010-b721-b9d369841e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:58:58.818712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-baa99394-9e8f-4010-b721-b9d369841e4b None None] MFA Rules not processed for user `e17a026f305248878f18eec35e1e02dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:58:58.819936 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-baa99394-9e8f-4010-b721-b9d369841e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:58:58.827208 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4459/8984] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 21:58:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 21:58:58.846346 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a76ada-5c2c-446e-9336-8d050d46c818 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:58:58.856273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a76ada-5c2c-446e-9336-8d050d46c818 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:58:58.860185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c806a1-0924-401d-8f3e-6491a7c44248 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:58:58.860853 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c806a1-0924-401d-8f3e-6491a7c44248 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:58:58.861200 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c806a1-0924-401d-8f3e-6491a7c44248 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:58:58.861536 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c806a1-0924-401d-8f3e-6491a7c44248 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:58:58.869662 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c806a1-0924-401d-8f3e-6491a7c44248 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:58:58.871273 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c806a1-0924-401d-8f3e-6491a7c44248 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:58:58.879802 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4526/8985] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:58:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:58:58.978249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa99394-9e8f-4010-b721-b9d369841e4b None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:58:58.979692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-baa99394-9e8f-4010-b721-b9d369841e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:58:59.007888 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa99394-9e8f-4010-b721-b9d369841e4b None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:58:59.013027 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd113d1-498b-4f34-8449-46897450420e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:58:59.013633 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd113d1-498b-4f34-8449-46897450420e service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:58:59.013917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd113d1-498b-4f34-8449-46897450420e service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:58:59.014194 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd113d1-498b-4f34-8449-46897450420e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:58:59.015956 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9cd113d1-498b-4f34-8449-46897450420e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:58:59.027550 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd113d1-498b-4f34-8449-46897450420e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:58:59.029204 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd113d1-498b-4f34-8449-46897450420e service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:58:59.040019 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4460/8986] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:58:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:58:59.213014 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03a8fd-552d-42c1-8fc6-08c8b2389142 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:58:59.213332 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03a8fd-552d-42c1-8fc6-08c8b2389142 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:58:59.213629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03a8fd-552d-42c1-8fc6-08c8b2389142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:58:59.252793 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-9d03a8fd-552d-42c1-8fc6-08c8b2389142 None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:58:59.254030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9d03a8fd-552d-42c1-8fc6-08c8b2389142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:58:59.269763 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4527/8987] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 21:58:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 21:58:59.280089 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd113d1-498b-4f34-8449-46897450420e service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:58:59.289710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd113d1-498b-4f34-8449-46897450420e service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:58:59.294014 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bf786-8e5e-4063-8bad-dbcf895564c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:58:59.294652 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bf786-8e5e-4063-8bad-dbcf895564c4 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:58:59.294932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bf786-8e5e-4063-8bad-dbcf895564c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:58:59.295203 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bf786-8e5e-4063-8bad-dbcf895564c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:58:59.305349 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456bf786-8e5e-4063-8bad-dbcf895564c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:58:59.306809 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456bf786-8e5e-4063-8bad-dbcf895564c4 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:58:59.317319 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4461/8988] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:58:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:58:59.380707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03a8fd-552d-42c1-8fc6-08c8b2389142 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:58:59.391985 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03a8fd-552d-42c1-8fc6-08c8b2389142 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:58:59.397058 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02908e1-50d6-4414-a295-884c32f9d7a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:58:59.397657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02908e1-50d6-4414-a295-884c32f9d7a5 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:58:59.397941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02908e1-50d6-4414-a295-884c32f9d7a5 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:58:59.398217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02908e1-50d6-4414-a295-884c32f9d7a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:58:59.400498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a02908e1-50d6-4414-a295-884c32f9d7a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:58:59.411744 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02908e1-50d6-4414-a295-884c32f9d7a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:58:59.413296 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02908e1-50d6-4414-a295-884c32f9d7a5 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:58:59.423697 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4528/8989] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:58:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:59:09.312775 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=26 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:59:09.419638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=24 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:59:12.706467 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bf786-8e5e-4063-8bad-dbcf895564c4 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:59:12.715015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bf786-8e5e-4063-8bad-dbcf895564c4 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:59:12.718679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88fbce-adf3-4850-a553-943629ce6fc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:59:12.719271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88fbce-adf3-4850-a553-943629ce6fc1 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:59:12.719552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88fbce-adf3-4850-a553-943629ce6fc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:59:12.719823 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88fbce-adf3-4850-a553-943629ce6fc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:59:12.721509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-cc88fbce-adf3-4850-a553-943629ce6fc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:59:12.730678 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88fbce-adf3-4850-a553-943629ce6fc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:59:12.732099 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88fbce-adf3-4850-a553-943629ce6fc1 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:59:12.739887 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4462/8990] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:59:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:59:13.225083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02908e1-50d6-4414-a295-884c32f9d7a5 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:59:13.239159 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02908e1-50d6-4414-a295-884c32f9d7a5 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:59:13.245436 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f353f5c-af79-40aa-8859-8bcf12fbcecf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:59:13.246050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f353f5c-af79-40aa-8859-8bcf12fbcecf service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:59:13.246336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f353f5c-af79-40aa-8859-8bcf12fbcecf service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:59:13.246649 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f353f5c-af79-40aa-8859-8bcf12fbcecf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:59:13.248401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7f353f5c-af79-40aa-8859-8bcf12fbcecf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:59:13.260976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f353f5c-af79-40aa-8859-8bcf12fbcecf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:59:13.262599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f353f5c-af79-40aa-8859-8bcf12fbcecf service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:59:13.273276 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4529/8991] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:59:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:59:20.076270 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88fbce-adf3-4850-a553-943629ce6fc1 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 21:59:20.092352 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88fbce-adf3-4850-a553-943629ce6fc1 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 21:59:20.097813 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf007ad-6ec9-435e-8aa8-d3dc25193b9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 21:59:20.098416 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf007ad-6ec9-435e-8aa8-d3dc25193b9a service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:59:20.098745 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf007ad-6ec9-435e-8aa8-d3dc25193b9a service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:59:20.099016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf007ad-6ec9-435e-8aa8-d3dc25193b9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:59:20.100780 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fdf007ad-6ec9-435e-8aa8-d3dc25193b9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:59:20.113003 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf007ad-6ec9-435e-8aa8-d3dc25193b9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 21:59:20.114590 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf007ad-6ec9-435e-8aa8-d3dc25193b9a service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 21:59:20.125631 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4463/8992] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 21:59:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 21:59:23.268791 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:59:30.120817 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=12 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:59:59.419758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1197c-ed36-4bfd-bbfd-16a1b1095aeb None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 21:59:59.420702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1197c-ed36-4bfd-bbfd-16a1b1095aeb None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 21:59:59.421201 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1197c-ed36-4bfd-bbfd-16a1b1095aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 21:59:59.448648 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-77f1197c-ed36-4bfd-bbfd-16a1b1095aeb None None] MFA Rules not processed for user `5f22070464e045f19acc335de7dd38f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 21:59:59.450215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-77f1197c-ed36-4bfd-bbfd-16a1b1095aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 21:59:59.459800 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4530/8993] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 21:59:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:00:09.442194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:00:31.493829 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf007ad-6ec9-435e-8aa8-d3dc25193b9a service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:00:31.504952 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf007ad-6ec9-435e-8aa8-d3dc25193b9a service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:00:31.509994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec165fd5-4aa3-4f1c-96f8-2ee85d10df09 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:00:31.510625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec165fd5-4aa3-4f1c-96f8-2ee85d10df09 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:00:31.510908 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec165fd5-4aa3-4f1c-96f8-2ee85d10df09 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:00:31.511182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec165fd5-4aa3-4f1c-96f8-2ee85d10df09 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:00:31.519339 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec165fd5-4aa3-4f1c-96f8-2ee85d10df09 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:00:31.520914 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec165fd5-4aa3-4f1c-96f8-2ee85d10df09 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:00:31.530973 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4464/8994] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:00:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:00:32.407144 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1197c-ed36-4bfd-bbfd-16a1b1095aeb None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:00:32.418936 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1197c-ed36-4bfd-bbfd-16a1b1095aeb None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:00:32.423863 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef0cd2-9408-4ede-afe4-e71ca837354a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:00:32.424456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef0cd2-9408-4ede-afe4-e71ca837354a service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:00:32.424755 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef0cd2-9408-4ede-afe4-e71ca837354a service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:00:32.425031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef0cd2-9408-4ede-afe4-e71ca837354a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:00:32.434021 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef0cd2-9408-4ede-afe4-e71ca837354a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:00:32.435609 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef0cd2-9408-4ede-afe4-e71ca837354a service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:00:32.445804 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4531/8995] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:00:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:00:36.046943 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec165fd5-4aa3-4f1c-96f8-2ee85d10df09 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:00:36.060372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec165fd5-4aa3-4f1c-96f8-2ee85d10df09 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:00:36.065749 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7e6b0-eb76-4506-bab1-7024a271eb10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:00:36.066338 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7e6b0-eb76-4506-bab1-7024a271eb10 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:00:36.066661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7e6b0-eb76-4506-bab1-7024a271eb10 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:00:36.066933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7e6b0-eb76-4506-bab1-7024a271eb10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:00:36.075533 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7e6b0-eb76-4506-bab1-7024a271eb10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:00:36.077153 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7e6b0-eb76-4506-bab1-7024a271eb10 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:00:36.085996 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4465/8996] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:00:36 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:00:42.441177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:00:46.081940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=12 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:01:36.266249 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef0cd2-9408-4ede-afe4-e71ca837354a service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:01:36.278362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef0cd2-9408-4ede-afe4-e71ca837354a service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:01:36.283887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded71ae8-647d-4a0c-920a-fb8e91ae54ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:01:36.284700 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded71ae8-647d-4a0c-920a-fb8e91ae54ee service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:01:36.285182 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded71ae8-647d-4a0c-920a-fb8e91ae54ee service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:01:36.285636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded71ae8-647d-4a0c-920a-fb8e91ae54ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:01:36.287608 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ded71ae8-647d-4a0c-920a-fb8e91ae54ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:01:36.302729 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded71ae8-647d-4a0c-920a-fb8e91ae54ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:01:36.304448 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded71ae8-647d-4a0c-920a-fb8e91ae54ee service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:01:36.315817 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4532/8997] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:01:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:01:39.387429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7e6b0-eb76-4506-bab1-7024a271eb10 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:01:39.397249 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7e6b0-eb76-4506-bab1-7024a271eb10 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:01:39.401508 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38731a48-0d6d-4dab-aefd-2f24b0be268e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:01:39.402191 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38731a48-0d6d-4dab-aefd-2f24b0be268e service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:01:39.402580 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38731a48-0d6d-4dab-aefd-2f24b0be268e service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:01:39.402919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38731a48-0d6d-4dab-aefd-2f24b0be268e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:01:39.411994 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38731a48-0d6d-4dab-aefd-2f24b0be268e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:01:39.413420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38731a48-0d6d-4dab-aefd-2f24b0be268e service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:01:39.425449 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4466/8998] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:01:39 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:01:42.288066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358637c2-e521-4835-9c65-3f0f976adb32 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:01:42.288605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358637c2-e521-4835-9c65-3f0f976adb32 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:01:42.288605 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358637c2-e521-4835-9c65-3f0f976adb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:01:42.310175 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-358637c2-e521-4835-9c65-3f0f976adb32 None None] MFA Rules not processed for user `5f22070464e045f19acc335de7dd38f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:01:42.311629 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-358637c2-e521-4835-9c65-3f0f976adb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:01:42.319913 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4533/8999] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 22:01:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:01:42.659991 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38731a48-0d6d-4dab-aefd-2f24b0be268e service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:01:42.674461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38731a48-0d6d-4dab-aefd-2f24b0be268e service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:01:42.679197 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7e201-3501-4eb8-9fc9-2257259e8900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:01:42.680077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7e201-3501-4eb8-9fc9-2257259e8900 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:01:42.680627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7e201-3501-4eb8-9fc9-2257259e8900 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:01:42.681209 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7e201-3501-4eb8-9fc9-2257259e8900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:01:42.689951 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7e201-3501-4eb8-9fc9-2257259e8900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:01:42.691737 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7e201-3501-4eb8-9fc9-2257259e8900 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:01:42.706442 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4467/9000] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:01:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:01:52.304218 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=9 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:01:52.700235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=12 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:02:38.377449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358637c2-e521-4835-9c65-3f0f976adb32 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:38.453367 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358637c2-e521-4835-9c65-3f0f976adb32 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:38.457895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f318d-eb05-47b6-95a7-bf57e014e1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:38.458558 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f318d-eb05-47b6-95a7-bf57e014e1c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:38.458840 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f318d-eb05-47b6-95a7-bf57e014e1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:38.459111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f318d-eb05-47b6-95a7-bf57e014e1c5 admin admin] PATH_INFO: `/v3/users/5f22070464e045f19acc335de7dd38f3` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:38.460242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f318d-eb05-47b6-95a7-bf57e014e1c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f22070464e045f19acc335de7dd38f3)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:38.462354 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f318d-eb05-47b6-95a7-bf57e014e1c5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:38.514285 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e5f318d-eb05-47b6-95a7-bf57e014e1c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f22070464e045f19acc335de7dd38f3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:02:38.521382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e5f318d-eb05-47b6-95a7-bf57e014e1c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f22070464e045f19acc335de7dd38f3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:02:38.533865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e5f318d-eb05-47b6-95a7-bf57e014e1c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f22070464e045f19acc335de7dd38f3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:02:38.534559 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e5f318d-eb05-47b6-95a7-bf57e014e1c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f22070464e045f19acc335de7dd38f3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:02:38.542677 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-9e5f318d-eb05-47b6-95a7-bf57e014e1c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f22070464e045f19acc335de7dd38f3'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:02:38.548694 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4534/9001] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 22:02:38 2022] DELETE /identity/v3/users/5f22070464e045f19acc335de7dd38f3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 22:02:38.553892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f15dca-2715-4b62-8dff-9fe1f13ba8dc None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:38.554442 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f15dca-2715-4b62-8dff-9fe1f13ba8dc None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:38.554606 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f15dca-2715-4b62-8dff-9fe1f13ba8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:38.573408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-e0f15dca-2715-4b62-8dff-9fe1f13ba8dc None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:02:38.574577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e0f15dca-2715-4b62-8dff-9fe1f13ba8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:02:38.632234 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4468/9002] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 22:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 22:02:38.641952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f318d-eb05-47b6-95a7-bf57e014e1c5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:38.643687 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9e5f318d-eb05-47b6-95a7-bf57e014e1c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:02:38.713452 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f318d-eb05-47b6-95a7-bf57e014e1c5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:38.717217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e28e90-df63-484d-92af-eb9836e63ca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:38.717826 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e28e90-df63-484d-92af-eb9836e63ca8 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:38.718111 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e28e90-df63-484d-92af-eb9836e63ca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:38.718390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e28e90-df63-484d-92af-eb9836e63ca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:38.727385 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e28e90-df63-484d-92af-eb9836e63ca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:38.729416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e28e90-df63-484d-92af-eb9836e63ca8 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:38.738930 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4535/9003] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:02:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:02:38.908587 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f15dca-2715-4b62-8dff-9fe1f13ba8dc None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:38.910011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-e0f15dca-2715-4b62-8dff-9fe1f13ba8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:02:38.922965 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f15dca-2715-4b62-8dff-9fe1f13ba8dc None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:38.928088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1804ab-b1ff-426d-a892-49c8cbc913d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:38.929181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1804ab-b1ff-426d-a892-49c8cbc913d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:38.929470 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1804ab-b1ff-426d-a892-49c8cbc913d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:38.929751 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1804ab-b1ff-426d-a892-49c8cbc913d9 admin admin] PATH_INFO: `/v3/projects/27214fca5a7a4c85bba526c092ac726b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:38.931058 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1804ab-b1ff-426d-a892-49c8cbc913d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27214fca5a7a4c85bba526c092ac726b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:38.933140 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1804ab-b1ff-426d-a892-49c8cbc913d9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:38.979636 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ce1804ab-b1ff-426d-a892-49c8cbc913d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27214fca5a7a4c85bba526c092ac726b', 'request_id': 'req-ce1804ab-b1ff-426d-a892-49c8cbc913d9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:02:38.987167 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ce1804ab-b1ff-426d-a892-49c8cbc913d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27214fca5a7a4c85bba526c092ac726b', 'request_id': 'req-ce1804ab-b1ff-426d-a892-49c8cbc913d9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:02:38.993067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ce1804ab-b1ff-426d-a892-49c8cbc913d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27214fca5a7a4c85bba526c092ac726b', 'request_id': 'req-ce1804ab-b1ff-426d-a892-49c8cbc913d9'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:02:39.003915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ce1804ab-b1ff-426d-a892-49c8cbc913d9 admin admin] The token cache is being invalidate because project 27214fca5a7a4c85bba526c092ac726b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 22:02:39.004230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ce1804ab-b1ff-426d-a892-49c8cbc913d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:02:39.004858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ce1804ab-b1ff-426d-a892-49c8cbc913d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:02:39.006378 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4469/9004] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 22:02:38 2022] DELETE /identity/v3/projects/27214fca5a7a4c85bba526c092ac726b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 22:02:39.010951 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e28e90-df63-484d-92af-eb9836e63ca8 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:39.015976 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:02:39.022721 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e28e90-df63-484d-92af-eb9836e63ca8 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:39.081878 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432334ac-e335-4b05-8937-2c9d71c35e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:39.082531 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432334ac-e335-4b05-8937-2c9d71c35e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:39.082808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432334ac-e335-4b05-8937-2c9d71c35e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:39.083083 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432334ac-e335-4b05-8937-2c9d71c35e38 admin admin] PATH_INFO: `/v3/users/0cdaf8c5333a4c4b8aa9857e53e2936c` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:39.084255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432334ac-e335-4b05-8937-2c9d71c35e38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cdaf8c5333a4c4b8aa9857e53e2936c)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:39.086090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432334ac-e335-4b05-8937-2c9d71c35e38 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:39.131134 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-432334ac-e335-4b05-8937-2c9d71c35e38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cdaf8c5333a4c4b8aa9857e53e2936c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:02:39.138835 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-432334ac-e335-4b05-8937-2c9d71c35e38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cdaf8c5333a4c4b8aa9857e53e2936c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:02:39.150690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-432334ac-e335-4b05-8937-2c9d71c35e38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cdaf8c5333a4c4b8aa9857e53e2936c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:02:39.151368 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-432334ac-e335-4b05-8937-2c9d71c35e38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cdaf8c5333a4c4b8aa9857e53e2936c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:02:39.158825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-432334ac-e335-4b05-8937-2c9d71c35e38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cdaf8c5333a4c4b8aa9857e53e2936c'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:02:39.167079 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4536/9005] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 22:02:39 2022] DELETE /identity/v3/users/0cdaf8c5333a4c4b8aa9857e53e2936c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 22:02:39.434160 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1804ab-b1ff-426d-a892-49c8cbc913d9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:39.435702 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ce1804ab-b1ff-426d-a892-49c8cbc913d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:02:39.452186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1804ab-b1ff-426d-a892-49c8cbc913d9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:39.515230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3cccaa-0fe0-4c98-a3e7-6b4fd277160c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:39.515859 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cccaa-0fe0-4c98-a3e7-6b4fd277160c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:39.516137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cccaa-0fe0-4c98-a3e7-6b4fd277160c admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:39.516410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cccaa-0fe0-4c98-a3e7-6b4fd277160c admin admin] PATH_INFO: `/v3/projects/f3ce989dddd04ea1970278b7ca43410a` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:39.517517 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3cccaa-0fe0-4c98-a3e7-6b4fd277160c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3ce989dddd04ea1970278b7ca43410a)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:39.519461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3cccaa-0fe0-4c98-a3e7-6b4fd277160c admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:39.549009 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:02:39.567506 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ef3cccaa-0fe0-4c98-a3e7-6b4fd277160c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3ce989dddd04ea1970278b7ca43410a', 'request_id': 'req-ef3cccaa-0fe0-4c98-a3e7-6b4fd277160c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:02:39.575081 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ef3cccaa-0fe0-4c98-a3e7-6b4fd277160c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3ce989dddd04ea1970278b7ca43410a', 'request_id': 'req-ef3cccaa-0fe0-4c98-a3e7-6b4fd277160c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:02:39.580604 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ef3cccaa-0fe0-4c98-a3e7-6b4fd277160c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3ce989dddd04ea1970278b7ca43410a', 'request_id': 'req-ef3cccaa-0fe0-4c98-a3e7-6b4fd277160c'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:02:39.591496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ef3cccaa-0fe0-4c98-a3e7-6b4fd277160c admin admin] The token cache is being invalidate because project f3ce989dddd04ea1970278b7ca43410a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 22:02:39.591789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ef3cccaa-0fe0-4c98-a3e7-6b4fd277160c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:02:39.592419 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-ef3cccaa-0fe0-4c98-a3e7-6b4fd277160c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:02:39.594120 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4470/9006] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 22:02:39 2022] DELETE /identity/v3/projects/f3ce989dddd04ea1970278b7ca43410a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 22:02:39.599397 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c92d4-9371-453c-b289-f8f9688d133f None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:39.599914 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c92d4-9371-453c-b289-f8f9688d133f None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:39.600030 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c92d4-9371-453c-b289-f8f9688d133f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:39.620533 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-4c8c92d4-9371-453c-b289-f8f9688d133f None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:02:39.621658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4c8c92d4-9371-453c-b289-f8f9688d133f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:02:39.682903 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4537/9007] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 22:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:02:39.691013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10944de7-94a2-4802-a1aa-37a7124feb92 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:39.691302 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10944de7-94a2-4802-a1aa-37a7124feb92 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:39.691577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10944de7-94a2-4802-a1aa-37a7124feb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:39.712064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-10944de7-94a2-4802-a1aa-37a7124feb92 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:02:39.713314 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-10944de7-94a2-4802-a1aa-37a7124feb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:02:39.721557 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4471/9008] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 22:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 22:02:39.725598 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c92d4-9371-453c-b289-f8f9688d133f None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:39.735860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c92d4-9371-453c-b289-f8f9688d133f None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:39.740386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76a831-fdfc-4105-8f58-73513f95a0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:39.740975 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76a831-fdfc-4105-8f58-73513f95a0d1 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:39.741255 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76a831-fdfc-4105-8f58-73513f95a0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:39.741526 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76a831-fdfc-4105-8f58-73513f95a0d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:39.742168 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76a831-fdfc-4105-8f58-73513f95a0d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 22:02:39.742471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76a831-fdfc-4105-8f58-73513f95a0d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:39.744097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76a831-fdfc-4105-8f58-73513f95a0d1 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:39.756125 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4538/9009] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 22:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 22:02:39.760299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10944de7-94a2-4802-a1aa-37a7124feb92 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:39.769668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10944de7-94a2-4802-a1aa-37a7124feb92 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:39.774250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b69fe-adea-492e-a69e-62bf3aeab522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:39.774877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b69fe-adea-492e-a69e-62bf3aeab522 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:39.775157 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b69fe-adea-492e-a69e-62bf3aeab522 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:39.775431 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b69fe-adea-492e-a69e-62bf3aeab522 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:39.776052 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b69fe-adea-492e-a69e-62bf3aeab522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:39.777929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b69fe-adea-492e-a69e-62bf3aeab522 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:39.960019 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4472/9010] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 22:02:39 2022] POST /identity/v3/projects => generated 373 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 22:02:39.964720 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76a831-fdfc-4105-8f58-73513f95a0d1 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:39.974786 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76a831-fdfc-4105-8f58-73513f95a0d1 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:39.979919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:02:40.034181 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465e3f0-8018-4e25-80a6-79f8f9306a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:40.034832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465e3f0-8018-4e25-80a6-79f8f9306a3f admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:40.035148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465e3f0-8018-4e25-80a6-79f8f9306a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:40.035426 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465e3f0-8018-4e25-80a6-79f8f9306a3f admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:40.036078 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9465e3f0-8018-4e25-80a6-79f8f9306a3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:40.037997 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9465e3f0-8018-4e25-80a6-79f8f9306a3f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:40.061750 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4539/9011] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 22:02:39 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 22:02:40.065885 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b69fe-adea-492e-a69e-62bf3aeab522 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:40.080583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b69fe-adea-492e-a69e-62bf3aeab522 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:40.085294 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c993f-bb81-4f18-882b-68bb984ac610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:40.085903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c993f-bb81-4f18-882b-68bb984ac610 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:40.086179 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c993f-bb81-4f18-882b-68bb984ac610 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:40.086449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c993f-bb81-4f18-882b-68bb984ac610 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:40.087131 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c993f-bb81-4f18-882b-68bb984ac610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:40.089265 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c993f-bb81-4f18-882b-68bb984ac610 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:40.099473 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4473/9012] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 22:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 22:02:40.104245 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465e3f0-8018-4e25-80a6-79f8f9306a3f admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:40.114529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465e3f0-8018-4e25-80a6-79f8f9306a3f admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:40.119651 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f6e68-9ecf-47c1-972d-26dd87d2052f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:40.120252 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f6e68-9ecf-47c1-972d-26dd87d2052f admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:40.120537 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f6e68-9ecf-47c1-972d-26dd87d2052f admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:40.120818 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f6e68-9ecf-47c1-972d-26dd87d2052f admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:40.121464 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f6e68-9ecf-47c1-972d-26dd87d2052f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:40.123118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f6e68-9ecf-47c1-972d-26dd87d2052f admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:40.134307 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4540/9013] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 22:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 22:02:40.138810 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c993f-bb81-4f18-882b-68bb984ac610 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:40.149659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c993f-bb81-4f18-882b-68bb984ac610 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:40.154806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a31de9-47d4-4baf-8f5b-713f75cca0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:40.155504 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a31de9-47d4-4baf-8f5b-713f75cca0b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:40.155788 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a31de9-47d4-4baf-8f5b-713f75cca0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:40.156063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a31de9-47d4-4baf-8f5b-713f75cca0b9 admin admin] PATH_INFO: `/v3/projects/b904f70d6c7144db99dd4897d62ec567/users/c145fd00464c4dd1bb9e48320d8e916b/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:40.172237 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a31de9-47d4-4baf-8f5b-713f75cca0b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b904f70d6c7144db99dd4897d62ec567, user_id=c145fd00464c4dd1bb9e48320d8e916b, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:40.175183 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a31de9-47d4-4baf-8f5b-713f75cca0b9 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:40.188403 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4474/9014] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 22:02:40 2022] PUT /identity/v3/projects/b904f70d6c7144db99dd4897d62ec567/users/c145fd00464c4dd1bb9e48320d8e916b/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 22:02:40.193487 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea9129-bf05-4081-9463-d7f7c47fef38 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:40.193879 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea9129-bf05-4081-9463-d7f7c47fef38 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:40.194299 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea9129-bf05-4081-9463-d7f7c47fef38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:40.216164 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-33ea9129-bf05-4081-9463-d7f7c47fef38 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:02:40.217303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-33ea9129-bf05-4081-9463-d7f7c47fef38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:02:40.279535 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4541/9015] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 22:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:02:40.289356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a31de9-47d4-4baf-8f5b-713f75cca0b9 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:40.291094 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-09a31de9-47d4-4baf-8f5b-713f75cca0b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:02:40.351296 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a31de9-47d4-4baf-8f5b-713f75cca0b9 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:40.355116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdac76-b491-423a-9a48-cd4c8acc09c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:40.355717 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdac76-b491-423a-9a48-cd4c8acc09c1 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:40.356015 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdac76-b491-423a-9a48-cd4c8acc09c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:40.356284 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdac76-b491-423a-9a48-cd4c8acc09c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:40.365146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efdac76-b491-423a-9a48-cd4c8acc09c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:40.366585 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efdac76-b491-423a-9a48-cd4c8acc09c1 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:40.375670 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4475/9016] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:02:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:02:44.149753 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7299c5-b2a8-439d-9bcc-1fa34de69fa2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:44.150370 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7299c5-b2a8-439d-9bcc-1fa34de69fa2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:44.151690 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7299c5-b2a8-439d-9bcc-1fa34de69fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:44.187417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-6c7299c5-b2a8-439d-9bcc-1fa34de69fa2 None None] MFA Rules not processed for user `c145fd00464c4dd1bb9e48320d8e916b`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:02:44.188644 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-6c7299c5-b2a8-439d-9bcc-1fa34de69fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:02:44.258779 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4542/9017] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 22:02:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:02:44.263588 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdac76-b491-423a-9a48-cd4c8acc09c1 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:44.273849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdac76-b491-423a-9a48-cd4c8acc09c1 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:44.279063 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d482a9-7bd6-41c4-852e-ff3f1b732a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:44.279768 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d482a9-7bd6-41c4-852e-ff3f1b732a42 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:44.280114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d482a9-7bd6-41c4-852e-ff3f1b732a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:44.280461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d482a9-7bd6-41c4-852e-ff3f1b732a42 admin admin] PATH_INFO: `/v3/projects` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:44.281172 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d482a9-7bd6-41c4-852e-ff3f1b732a42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:44.283235 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d482a9-7bd6-41c4-852e-ff3f1b732a42 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:44.314518 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4476/9018] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 22:02:44 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 22:02:44.318713 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7299c5-b2a8-439d-9bcc-1fa34de69fa2 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:44.329836 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7299c5-b2a8-439d-9bcc-1fa34de69fa2 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:44.343416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:02:44.392230 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18322c7b-148a-44f3-b786-2031b34b0247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:44.392848 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18322c7b-148a-44f3-b786-2031b34b0247 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:44.393128 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18322c7b-148a-44f3-b786-2031b34b0247 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:44.393401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18322c7b-148a-44f3-b786-2031b34b0247 admin admin] PATH_INFO: `/v3/users` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:44.394037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18322c7b-148a-44f3-b786-2031b34b0247 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:44.396110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18322c7b-148a-44f3-b786-2031b34b0247 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:44.423073 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4543/9019] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 22:02:44 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 22:02:44.424397 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d482a9-7bd6-41c4-852e-ff3f1b732a42 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:44.435242 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d482a9-7bd6-41c4-852e-ff3f1b732a42 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:44.440152 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e71930-1eab-4775-bfc7-e6c722b90054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:44.440800 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e71930-1eab-4775-bfc7-e6c722b90054 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:44.441077 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e71930-1eab-4775-bfc7-e6c722b90054 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:44.441372 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e71930-1eab-4775-bfc7-e6c722b90054 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:44.442029 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e71930-1eab-4775-bfc7-e6c722b90054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:44.443697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e71930-1eab-4775-bfc7-e6c722b90054 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:44.454344 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4477/9020] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 22:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 22:02:44.458517 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18322c7b-148a-44f3-b786-2031b34b0247 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:44.470574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18322c7b-148a-44f3-b786-2031b34b0247 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:44.475457 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4548bc-6176-4a77-a940-f870165d79dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:44.476138 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4548bc-6176-4a77-a940-f870165d79dd admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:44.476422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4548bc-6176-4a77-a940-f870165d79dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:44.476701 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4548bc-6176-4a77-a940-f870165d79dd admin admin] PATH_INFO: `/v3/projects/18f2cca2fe834a72bae4a6b1aeb74ec8/users/5b2fc94f5fa34034b26579bb3fdf7943/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:44.493236 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4548bc-6176-4a77-a940-f870165d79dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=18f2cca2fe834a72bae4a6b1aeb74ec8, user_id=5b2fc94f5fa34034b26579bb3fdf7943, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:44.495978 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4548bc-6176-4a77-a940-f870165d79dd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:44.509808 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4544/9021] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 22:02:44 2022] PUT /identity/v3/projects/18f2cca2fe834a72bae4a6b1aeb74ec8/users/5b2fc94f5fa34034b26579bb3fdf7943/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 22:02:44.514023 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e71930-1eab-4775-bfc7-e6c722b90054 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:44.524454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e71930-1eab-4775-bfc7-e6c722b90054 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:44.533367 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:02:44.595904 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e88e3f-d2f5-4a1e-9c4f-c1a1057c6569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:44.596523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e88e3f-d2f5-4a1e-9c4f-c1a1057c6569 admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:44.596808 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e88e3f-d2f5-4a1e-9c4f-c1a1057c6569 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:44.597106 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e88e3f-d2f5-4a1e-9c4f-c1a1057c6569 admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:44.597759 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e88e3f-d2f5-4a1e-9c4f-c1a1057c6569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:44.599400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e88e3f-d2f5-4a1e-9c4f-c1a1057c6569 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:44.614258 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4478/9022] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 22:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 22:02:44.615595 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4548bc-6176-4a77-a940-f870165d79dd admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:44.630950 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4548bc-6176-4a77-a940-f870165d79dd admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:44.636303 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992bcf6-06ea-42d1-aacf-c7aad12ab0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:44.637015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992bcf6-06ea-42d1-aacf-c7aad12ab0de admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:44.637316 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992bcf6-06ea-42d1-aacf-c7aad12ab0de admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:44.637624 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992bcf6-06ea-42d1-aacf-c7aad12ab0de admin admin] PATH_INFO: `/v3/projects/18f2cca2fe834a72bae4a6b1aeb74ec8/users/5b2fc94f5fa34034b26579bb3fdf7943/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:44.639921 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992bcf6-06ea-42d1-aacf-c7aad12ab0de admin admin] RBAC: Authorizing `identity:create_grant(project_id=18f2cca2fe834a72bae4a6b1aeb74ec8, user_id=5b2fc94f5fa34034b26579bb3fdf7943, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:44.642742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992bcf6-06ea-42d1-aacf-c7aad12ab0de admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:44.654428 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4545/9023] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 22:02:44 2022] PUT /identity/v3/projects/18f2cca2fe834a72bae4a6b1aeb74ec8/users/5b2fc94f5fa34034b26579bb3fdf7943/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 22:02:49.251313 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2cd7a-e18c-4995-b13a-e7aad91356ff None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:49.251901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2cd7a-e18c-4995-b13a-e7aad91356ff None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:49.251901 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2cd7a-e18c-4995-b13a-e7aad91356ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:49.287021 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6dd2cd7a-e18c-4995-b13a-e7aad91356ff None None] MFA Rules not processed for user `5b2fc94f5fa34034b26579bb3fdf7943`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:02:49.288260 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6dd2cd7a-e18c-4995-b13a-e7aad91356ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:02:49.394961 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4479/9024] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 22:02:49 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:02:49.397654 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992bcf6-06ea-42d1-aacf-c7aad12ab0de admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:49.400522 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a992bcf6-06ea-42d1-aacf-c7aad12ab0de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:02:49.531024 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992bcf6-06ea-42d1-aacf-c7aad12ab0de admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:49.538999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956f75e-5215-4c8a-ba4f-e088088a45f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:49.539813 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956f75e-5215-4c8a-ba4f-e088088a45f9 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:49.540104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956f75e-5215-4c8a-ba4f-e088088a45f9 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:49.540389 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956f75e-5215-4c8a-ba4f-e088088a45f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:49.608091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956f75e-5215-4c8a-ba4f-e088088a45f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:49.609693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956f75e-5215-4c8a-ba4f-e088088a45f9 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:49.621717 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4546/9025] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:02:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 22:02:50.671583 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0ba88-5d8d-49ae-9393-af8b3924e497 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:50.672387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0ba88-5d8d-49ae-9393-af8b3924e497 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:50.672387 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0ba88-5d8d-49ae-9393-af8b3924e497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:50.691791 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-c4f0ba88-5d8d-49ae-9393-af8b3924e497 None None] MFA Rules not processed for user `c145fd00464c4dd1bb9e48320d8e916b`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:02:50.693005 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c4f0ba88-5d8d-49ae-9393-af8b3924e497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:02:50.699813 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4480/9026] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 22:02:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:02:51.346147 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956f75e-5215-4c8a-ba4f-e088088a45f9 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:51.347899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-d956f75e-5215-4c8a-ba4f-e088088a45f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:02:51.359271 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:02:51.437060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956f75e-5215-4c8a-ba4f-e088088a45f9 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:51.442204 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50818a-ea0f-4e49-b44a-5634f563a39f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:51.442871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50818a-ea0f-4e49-b44a-5634f563a39f service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:51.443153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50818a-ea0f-4e49-b44a-5634f563a39f service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:51.443423 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50818a-ea0f-4e49-b44a-5634f563a39f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:51.445825 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-ba50818a-ea0f-4e49-b44a-5634f563a39f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:02:51.460763 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50818a-ea0f-4e49-b44a-5634f563a39f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:51.462417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50818a-ea0f-4e49-b44a-5634f563a39f service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:51.486887 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4547/9027] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:02:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:02:53.610297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0ba88-5d8d-49ae-9393-af8b3924e497 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:02:53.614882 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-c4f0ba88-5d8d-49ae-9393-af8b3924e497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:02:53.724033 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0ba88-5d8d-49ae-9393-af8b3924e497 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:02:53.728438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e223b-f53f-4f81-8e16-49d81916e00f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:02:53.729410 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e223b-f53f-4f81-8e16-49d81916e00f service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:02:53.729874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e223b-f53f-4f81-8e16-49d81916e00f service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:02:53.730333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e223b-f53f-4f81-8e16-49d81916e00f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:02:53.732356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f78e223b-f53f-4f81-8e16-49d81916e00f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:02:53.746432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e223b-f53f-4f81-8e16-49d81916e00f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:02:53.749218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e223b-f53f-4f81-8e16-49d81916e00f service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:02:53.759949 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4481/9028] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:02:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:03:01.179170 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50818a-ea0f-4e49-b44a-5634f563a39f service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:03:01.188501 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50818a-ea0f-4e49-b44a-5634f563a39f service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:03:01.192107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41bd91-99e9-4d21-bdff-66b5f1cd67bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:03:01.192716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41bd91-99e9-4d21-bdff-66b5f1cd67bb service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:03:01.193000 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41bd91-99e9-4d21-bdff-66b5f1cd67bb service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:03:01.193272 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41bd91-99e9-4d21-bdff-66b5f1cd67bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:03:01.202116 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41bd91-99e9-4d21-bdff-66b5f1cd67bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:03:01.204228 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41bd91-99e9-4d21-bdff-66b5f1cd67bb service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:03:01.214331 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4548/9029] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:03:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:03:03.755356 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=87 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:03:11.209635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=35 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:04:59.036858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e223b-f53f-4f81-8e16-49d81916e00f service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:04:59.038594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-f78e223b-f53f-4f81-8e16-49d81916e00f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:04:59.116488 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e223b-f53f-4f81-8e16-49d81916e00f service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:04:59.120200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba62e1-2ecf-429d-8d8c-ac4963a8504f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:04:59.120773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba62e1-2ecf-429d-8d8c-ac4963a8504f service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:04:59.121048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba62e1-2ecf-429d-8d8c-ac4963a8504f service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:04:59.121332 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba62e1-2ecf-429d-8d8c-ac4963a8504f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:04:59.123215 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-90ba62e1-2ecf-429d-8d8c-ac4963a8504f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:04:59.163095 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba62e1-2ecf-429d-8d8c-ac4963a8504f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:04:59.164647 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba62e1-2ecf-429d-8d8c-ac4963a8504f service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:04:59.175935 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4482/9030] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:04:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:04:59.354125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41bd91-99e9-4d21-bdff-66b5f1cd67bb service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:04:59.363146 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41bd91-99e9-4d21-bdff-66b5f1cd67bb service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:04:59.367291 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f317353-f9ac-481c-a6ab-5393fd51e33c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:04:59.367882 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f317353-f9ac-481c-a6ab-5393fd51e33c service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:04:59.368161 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f317353-f9ac-481c-a6ab-5393fd51e33c service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:04:59.368462 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f317353-f9ac-481c-a6ab-5393fd51e33c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:04:59.370167 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4f317353-f9ac-481c-a6ab-5393fd51e33c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:04:59.384096 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f317353-f9ac-481c-a6ab-5393fd51e33c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:04:59.385500 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f317353-f9ac-481c-a6ab-5393fd51e33c service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:04:59.396592 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4549/9031] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:04:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:05:09.170856 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=46 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:05:09.391602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:05:11.925824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752d442-fb0f-468f-a9c9-94da2613f786 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:05:11.926305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752d442-fb0f-468f-a9c9-94da2613f786 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:05:11.926616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752d442-fb0f-468f-a9c9-94da2613f786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:05:11.989748 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-6752d442-fb0f-468f-a9c9-94da2613f786 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:05:11.991705 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6752d442-fb0f-468f-a9c9-94da2613f786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:05:12.072306 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4483/9032] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 22:05:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:05:12.122318 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f317353-f9ac-481c-a6ab-5393fd51e33c service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:05:12.123800 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-4f317353-f9ac-481c-a6ab-5393fd51e33c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:05:12.218219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f317353-f9ac-481c-a6ab-5393fd51e33c service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:05:12.223402 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a402fa-fed9-4081-96b6-bc583d2e5e10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:05:12.224015 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a402fa-fed9-4081-96b6-bc583d2e5e10 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:05:12.224302 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a402fa-fed9-4081-96b6-bc583d2e5e10 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:05:12.224580 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a402fa-fed9-4081-96b6-bc583d2e5e10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:05:12.233852 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a402fa-fed9-4081-96b6-bc583d2e5e10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:05:12.235495 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a402fa-fed9-4081-96b6-bc583d2e5e10 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:05:12.244834 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4550/9033] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 22:05:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 22:05:14.034692 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752d442-fb0f-468f-a9c9-94da2613f786 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:05:14.036286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6752d442-fb0f-468f-a9c9-94da2613f786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:05:14.059680 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752d442-fb0f-468f-a9c9-94da2613f786 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:05:14.065449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151f225-155a-4b9b-9e72-bae6560ad499 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:05:14.066079 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151f225-155a-4b9b-9e72-bae6560ad499 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:05:14.066382 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151f225-155a-4b9b-9e72-bae6560ad499 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:05:14.066676 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151f225-155a-4b9b-9e72-bae6560ad499 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:05:14.068423 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3151f225-155a-4b9b-9e72-bae6560ad499 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:05:14.082281 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151f225-155a-4b9b-9e72-bae6560ad499 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:05:14.083915 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151f225-155a-4b9b-9e72-bae6560ad499 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:05:14.106319 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4484/9034] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:05:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:05:22.240449 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=41 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:05:24.090236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=50 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:05:24.839079 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09458ae-e2b2-4710-b3f6-c21ee3addaf2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:05:24.839653 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09458ae-e2b2-4710-b3f6-c21ee3addaf2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:05:24.839888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09458ae-e2b2-4710-b3f6-c21ee3addaf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:05:24.870287 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-a09458ae-e2b2-4710-b3f6-c21ee3addaf2 None None] MFA Rules not processed for user `7ab975dafb564aa2b357311c4a384d53`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:05:24.871581 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a09458ae-e2b2-4710-b3f6-c21ee3addaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:05:24.901488 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4551/9035] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Oct 1 22:05:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:05:24.913011 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151f225-155a-4b9b-9e72-bae6560ad499 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:05:24.925496 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151f225-155a-4b9b-9e72-bae6560ad499 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:05:24.930020 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841636f-8643-444e-b7f2-d1cbc486e2b1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2927203ce394fabab76dd52a68c2436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:05:24.930674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841636f-8643-444e-b7f2-d1cbc486e2b1 service glance] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:05:24.930948 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841636f-8643-444e-b7f2-d1cbc486e2b1 service glance] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:05:24.931221 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841636f-8643-444e-b7f2-d1cbc486e2b1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:05:24.942007 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5841636f-8643-444e-b7f2-d1cbc486e2b1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:05:24.944148 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5841636f-8643-444e-b7f2-d1cbc486e2b1 service glance] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:05:24.955217 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4485/9036] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:05:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:05:25.021229 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09458ae-e2b2-4710-b3f6-c21ee3addaf2 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:05:25.040923 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09458ae-e2b2-4710-b3f6-c21ee3addaf2 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:05:25.044957 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc038b-d825-431e-ba10-5c00cd10669e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:05:25.046064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc038b-d825-431e-ba10-5c00cd10669e service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:05:25.046064 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc038b-d825-431e-ba10-5c00cd10669e service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:05:25.046277 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc038b-d825-431e-ba10-5c00cd10669e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:05:25.048195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-eefc038b-d825-431e-ba10-5c00cd10669e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:05:25.058658 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc038b-d825-431e-ba10-5c00cd10669e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:05:25.060086 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc038b-d825-431e-ba10-5c00cd10669e service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:05:25.069467 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4552/9037] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:05:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:05:25.141711 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841636f-8643-444e-b7f2-d1cbc486e2b1 service glance] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:05:25.151146 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841636f-8643-444e-b7f2-d1cbc486e2b1 service glance] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:05:25.154697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e29af-6e35-44ce-9dc9-38442d6ff3e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:05:25.155283 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e29af-6e35-44ce-9dc9-38442d6ff3e3 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:05:25.155555 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e29af-6e35-44ce-9dc9-38442d6ff3e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:05:25.155825 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e29af-6e35-44ce-9dc9-38442d6ff3e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:05:25.157486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2b6e29af-6e35-44ce-9dc9-38442d6ff3e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:05:25.170877 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6e29af-6e35-44ce-9dc9-38442d6ff3e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:05:25.172277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6e29af-6e35-44ce-9dc9-38442d6ff3e3 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:05:25.183779 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4486/9038] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:05:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:05:25.248390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc038b-d825-431e-ba10-5c00cd10669e service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:05:25.270129 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc038b-d825-431e-ba10-5c00cd10669e service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:05:25.275275 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3af07-49a2-4bc6-beb4-ba55e9fb7884 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:05:25.275871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3af07-49a2-4bc6-beb4-ba55e9fb7884 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:05:25.276155 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3af07-49a2-4bc6-beb4-ba55e9fb7884 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:05:25.276429 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3af07-49a2-4bc6-beb4-ba55e9fb7884 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:05:25.278680 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b3c3af07-49a2-4bc6-beb4-ba55e9fb7884 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:05:25.293386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3af07-49a2-4bc6-beb4-ba55e9fb7884 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:05:25.295010 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3af07-49a2-4bc6-beb4-ba55e9fb7884 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:05:25.305630 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4553/9039] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:05:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:05:26.095608 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e29af-6e35-44ce-9dc9-38442d6ff3e3 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:05:26.104554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e29af-6e35-44ce-9dc9-38442d6ff3e3 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:05:26.108386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996deca-0052-47dd-bf8e-eb3e63ad539d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:05:26.109018 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996deca-0052-47dd-bf8e-eb3e63ad539d service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:05:26.109299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996deca-0052-47dd-bf8e-eb3e63ad539d service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:05:26.109572 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996deca-0052-47dd-bf8e-eb3e63ad539d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:05:26.111393 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-0996deca-0052-47dd-bf8e-eb3e63ad539d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:05:26.121744 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996deca-0052-47dd-bf8e-eb3e63ad539d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:05:26.123552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996deca-0052-47dd-bf8e-eb3e63ad539d service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:05:26.132322 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4487/9040] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:05:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:05:26.964722 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3af07-49a2-4bc6-beb4-ba55e9fb7884 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:05:26.978638 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3af07-49a2-4bc6-beb4-ba55e9fb7884 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:05:26.984326 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5e0a8-2ff4-47e3-9957-10961ebe3452 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:05:26.984842 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5e0a8-2ff4-47e3-9957-10961ebe3452 service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:05:26.985122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5e0a8-2ff4-47e3-9957-10961ebe3452 service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:05:26.985395 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5e0a8-2ff4-47e3-9957-10961ebe3452 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:05:26.989107 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7de5e0a8-2ff4-47e3-9957-10961ebe3452 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:05:27.002386 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5e0a8-2ff4-47e3-9957-10961ebe3452 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:05:27.004044 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5e0a8-2ff4-47e3-9957-10961ebe3452 service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:05:27.015482 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4554/9041] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:05:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:05:27.603348 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc5532-ed84-4df6-87d2-85d1e824418c None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:05:27.603875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc5532-ed84-4df6-87d2-85d1e824418c None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:05:27.603875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc5532-ed84-4df6-87d2-85d1e824418c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:05:27.625072 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-7fdc5532-ed84-4df6-87d2-85d1e824418c None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:05:27.627126 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-7fdc5532-ed84-4df6-87d2-85d1e824418c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:05:27.635396 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4488/9042] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 22:05:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:05:27.648535 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5e0a8-2ff4-47e3-9957-10961ebe3452 service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:05:27.659556 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5e0a8-2ff4-47e3-9957-10961ebe3452 service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:05:27.664821 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02dcc3f-87ca-4bf6-9294-775ae98a23e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:05:27.665417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dcc3f-87ca-4bf6-9294-775ae98a23e0 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:05:27.665694 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dcc3f-87ca-4bf6-9294-775ae98a23e0 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:05:27.665964 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dcc3f-87ca-4bf6-9294-775ae98a23e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:05:27.674233 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02dcc3f-87ca-4bf6-9294-775ae98a23e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:05:27.675838 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02dcc3f-87ca-4bf6-9294-775ae98a23e0 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:05:27.684036 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4555/9043] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 22:05:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 22:05:33.459432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc5532-ed84-4df6-87d2-85d1e824418c None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:05:33.473100 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc5532-ed84-4df6-87d2-85d1e824418c None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:05:33.479535 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d0813-2db5-47bc-ba60-2b9201379ada service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:05:33.480154 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d0813-2db5-47bc-ba60-2b9201379ada service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:05:33.480425 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d0813-2db5-47bc-ba60-2b9201379ada service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:05:33.480697 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d0813-2db5-47bc-ba60-2b9201379ada service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:05:33.482598 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-552d0813-2db5-47bc-ba60-2b9201379ada service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:05:33.496424 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d0813-2db5-47bc-ba60-2b9201379ada service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:05:33.498229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d0813-2db5-47bc-ba60-2b9201379ada service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:05:33.518807 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4489/9044] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:05:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:05:37.680740 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=48 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:05:39.296564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02dcc3f-87ca-4bf6-9294-775ae98a23e0 service swift] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:05:39.308857 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02dcc3f-87ca-4bf6-9294-775ae98a23e0 service swift] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:05:39.313922 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115bb9a-def5-4fe8-878d-8d9b0dd576f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:05:39.314600 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115bb9a-def5-4fe8-878d-8d9b0dd576f1 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:05:39.314881 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115bb9a-def5-4fe8-878d-8d9b0dd576f1 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:05:39.315256 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115bb9a-def5-4fe8-878d-8d9b0dd576f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:05:39.317062 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3115bb9a-def5-4fe8-878d-8d9b0dd576f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:05:39.329227 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3115bb9a-def5-4fe8-878d-8d9b0dd576f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:05:39.330839 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3115bb9a-def5-4fe8-878d-8d9b0dd576f1 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:05:39.340706 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4556/9045] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:05:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:05:43.504529 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=29 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:05:45.103035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb88ec-705c-466b-b198-4b8fdf52edf9 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:05:45.103616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb88ec-705c-466b-b198-4b8fdf52edf9 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:05:45.103616 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb88ec-705c-466b-b198-4b8fdf52edf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:05:45.123111 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-92cb88ec-705c-466b-b198-4b8fdf52edf9 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:05:45.124319 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-92cb88ec-705c-466b-b198-4b8fdf52edf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:05:45.131738 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4490/9046] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 22:05:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:05:45.144636 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115bb9a-def5-4fe8-878d-8d9b0dd576f1 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:05:45.155745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115bb9a-def5-4fe8-878d-8d9b0dd576f1 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:05:45.160732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fbf90-e91f-4760-9e74-f488330a2438 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:05:45.161334 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fbf90-e91f-4760-9e74-f488330a2438 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:05:45.161615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fbf90-e91f-4760-9e74-f488330a2438 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:05:45.161888 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fbf90-e91f-4760-9e74-f488330a2438 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:05:45.170324 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031fbf90-e91f-4760-9e74-f488330a2438 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:05:45.172023 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031fbf90-e91f-4760-9e74-f488330a2438 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:05:45.181295 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4557/9047] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 22:05:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 22:05:55.118789 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:05:55.177221 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=12 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:06:59.284256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb88ec-705c-466b-b198-4b8fdf52edf9 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:06:59.298812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb88ec-705c-466b-b198-4b8fdf52edf9 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:06:59.304738 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98665cce-1533-4a0b-b87b-62d334a94c53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:06:59.305432 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98665cce-1533-4a0b-b87b-62d334a94c53 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:06:59.305822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98665cce-1533-4a0b-b87b-62d334a94c53 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:06:59.306104 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98665cce-1533-4a0b-b87b-62d334a94c53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:06:59.307875 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-98665cce-1533-4a0b-b87b-62d334a94c53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:06:59.320690 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98665cce-1533-4a0b-b87b-62d334a94c53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:06:59.322297 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98665cce-1533-4a0b-b87b-62d334a94c53 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:06:59.333500 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4491/9048] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:06:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:07:01.125745 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e118c80-3a22-449d-ac72-e926e6d325f2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:07:01.126278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e118c80-3a22-449d-ac72-e926e6d325f2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:07:01.126278 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e118c80-3a22-449d-ac72-e926e6d325f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:07:01.145871 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-7e118c80-3a22-449d-ac72-e926e6d325f2 None None] MFA Rules not processed for user `c145fd00464c4dd1bb9e48320d8e916b`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:07:01.147097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-7e118c80-3a22-449d-ac72-e926e6d325f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:07:01.154568 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4558/9049] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 22:07:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:07:02.943331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98665cce-1533-4a0b-b87b-62d334a94c53 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:07:02.952917 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98665cce-1533-4a0b-b87b-62d334a94c53 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:07:02.956462 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95250108-89bc-4725-a345-568d9a91694d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:07:02.957048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95250108-89bc-4725-a345-568d9a91694d service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:07:02.957323 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95250108-89bc-4725-a345-568d9a91694d service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:07:02.957594 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95250108-89bc-4725-a345-568d9a91694d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:07:02.959327 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-95250108-89bc-4725-a345-568d9a91694d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:07:02.968712 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95250108-89bc-4725-a345-568d9a91694d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:07:02.970124 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95250108-89bc-4725-a345-568d9a91694d service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:07:02.978008 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4492/9050] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:07:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:07:11.141162 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:07:12.974512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=12 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:08:50.201270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e118c80-3a22-449d-ac72-e926e6d325f2 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:08:50.217889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e118c80-3a22-449d-ac72-e926e6d325f2 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:08:50.221415 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86523e4-8c5d-471b-ae49-4fb33ea7bbb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:08:50.222011 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86523e4-8c5d-471b-ae49-4fb33ea7bbb2 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:08:50.222290 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86523e4-8c5d-471b-ae49-4fb33ea7bbb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:08:50.222593 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86523e4-8c5d-471b-ae49-4fb33ea7bbb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:08:50.229739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86523e4-8c5d-471b-ae49-4fb33ea7bbb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:08:50.231195 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86523e4-8c5d-471b-ae49-4fb33ea7bbb2 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:08:50.240134 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4559/9051] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:08:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:09:00.235585 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=9 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:09:21.440229 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95250108-89bc-4725-a345-568d9a91694d service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:09:21.468940 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95250108-89bc-4725-a345-568d9a91694d service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:09:21.474408 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a59ae-d502-4c6d-a012-f7560987ab60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:09:21.475030 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a59ae-d502-4c6d-a012-f7560987ab60 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:09:21.475308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a59ae-d502-4c6d-a012-f7560987ab60 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:09:21.475577 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a59ae-d502-4c6d-a012-f7560987ab60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:09:21.483170 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a59ae-d502-4c6d-a012-f7560987ab60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:09:21.484754 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a59ae-d502-4c6d-a012-f7560987ab60 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:09:21.493454 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4493/9052] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:09:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:09:31.489449 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=12 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:10:38.484865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86523e4-8c5d-471b-ae49-4fb33ea7bbb2 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:10:38.486311 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a86523e4-8c5d-471b-ae49-4fb33ea7bbb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:10:38.537876 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86523e4-8c5d-471b-ae49-4fb33ea7bbb2 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:10:38.543341 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39eb081-9803-422a-8335-a52a913f747a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:10:38.544213 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39eb081-9803-422a-8335-a52a913f747a service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:10:38.544491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39eb081-9803-422a-8335-a52a913f747a service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:10:38.544762 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39eb081-9803-422a-8335-a52a913f747a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:10:38.552999 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39eb081-9803-422a-8335-a52a913f747a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:10:38.554602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39eb081-9803-422a-8335-a52a913f747a service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:10:38.564865 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4560/9053] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:10:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:10:43.454845 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a59ae-d502-4c6d-a012-f7560987ab60 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:10:43.466391 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a59ae-d502-4c6d-a012-f7560987ab60 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:10:43.471381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f872a30-05dc-4a97-905f-42aa2f292832 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:10:43.471969 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f872a30-05dc-4a97-905f-42aa2f292832 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:10:43.472273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f872a30-05dc-4a97-905f-42aa2f292832 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:10:43.472568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f872a30-05dc-4a97-905f-42aa2f292832 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:10:43.481420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f872a30-05dc-4a97-905f-42aa2f292832 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:10:43.483019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f872a30-05dc-4a97-905f-42aa2f292832 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:10:43.492654 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4494/9054] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:10:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:10:48.560832 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=19 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:10:53.488659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:11:10.219176 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39eb081-9803-422a-8335-a52a913f747a service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:10.228574 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39eb081-9803-422a-8335-a52a913f747a service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:10.232866 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99709e79-7d19-4da5-a6bd-5e362ea474c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:10.233534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99709e79-7d19-4da5-a6bd-5e362ea474c9 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:10.233820 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99709e79-7d19-4da5-a6bd-5e362ea474c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:10.234095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99709e79-7d19-4da5-a6bd-5e362ea474c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:10.245632 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99709e79-7d19-4da5-a6bd-5e362ea474c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:10.247264 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99709e79-7d19-4da5-a6bd-5e362ea474c9 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:10.259415 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4561/9055] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:11:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:11:11.443426 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f872a30-05dc-4a97-905f-42aa2f292832 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:11.453101 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f872a30-05dc-4a97-905f-42aa2f292832 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:11.456649 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915fb849-a5fa-457b-a6bf-9bf786021391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:11.457236 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fb849-a5fa-457b-a6bf-9bf786021391 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:11.457510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fb849-a5fa-457b-a6bf-9bf786021391 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:11.457777 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fb849-a5fa-457b-a6bf-9bf786021391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:11.468487 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915fb849-a5fa-457b-a6bf-9bf786021391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:11.469935 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915fb849-a5fa-457b-a6bf-9bf786021391 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:11.480822 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4495/9056] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:11:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:11:12.198039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99709e79-7d19-4da5-a6bd-5e362ea474c9 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:12.209610 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99709e79-7d19-4da5-a6bd-5e362ea474c9 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:12.216477 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e939e-cc11-40b0-a8a2-9ed5f7fc39ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:12.217105 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e939e-cc11-40b0-a8a2-9ed5f7fc39ad service nova] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:12.217369 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e939e-cc11-40b0-a8a2-9ed5f7fc39ad service nova] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:12.217647 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e939e-cc11-40b0-a8a2-9ed5f7fc39ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:12.228194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e939e-cc11-40b0-a8a2-9ed5f7fc39ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:12.229806 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e939e-cc11-40b0-a8a2-9ed5f7fc39ad service nova] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:12.240735 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4562/9057] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:11:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 22:11:13.991098 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915fb849-a5fa-457b-a6bf-9bf786021391 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:13.999850 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915fb849-a5fa-457b-a6bf-9bf786021391 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:14.003484 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdda64f-cb10-4eae-ba9e-f7a402360ab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:14.004073 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdda64f-cb10-4eae-ba9e-f7a402360ab5 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:14.004354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdda64f-cb10-4eae-ba9e-f7a402360ab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:14.004627 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdda64f-cb10-4eae-ba9e-f7a402360ab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:14.079377 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdda64f-cb10-4eae-ba9e-f7a402360ab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:14.080818 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdda64f-cb10-4eae-ba9e-f7a402360ab5 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:14.090958 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4496/9058] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:11:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:11:16.888670 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e939e-cc11-40b0-a8a2-9ed5f7fc39ad service nova] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:16.902065 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e939e-cc11-40b0-a8a2-9ed5f7fc39ad service nova] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:16.907770 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46edcef-029f-4597-b1fe-170ef2075f02 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:16.908394 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46edcef-029f-4597-b1fe-170ef2075f02 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:16.908693 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46edcef-029f-4597-b1fe-170ef2075f02 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:16.908952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46edcef-029f-4597-b1fe-170ef2075f02 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:16.911039 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a46edcef-029f-4597-b1fe-170ef2075f02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:11:16.925110 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46edcef-029f-4597-b1fe-170ef2075f02 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:16.927194 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46edcef-029f-4597-b1fe-170ef2075f02 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:16.941144 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4563/9059] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:11:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:11:20.193385 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdda64f-cb10-4eae-ba9e-f7a402360ab5 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:20.207139 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdda64f-cb10-4eae-ba9e-f7a402360ab5 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:20.221366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a82eb2-3397-4e91-8f1a-e9c477111366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:20.221366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a82eb2-3397-4e91-8f1a-e9c477111366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:20.221366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a82eb2-3397-4e91-8f1a-e9c477111366 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:20.221366 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a82eb2-3397-4e91-8f1a-e9c477111366 admin admin] PATH_INFO: `/v3/users/c145fd00464c4dd1bb9e48320d8e916b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:20.222576 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a82eb2-3397-4e91-8f1a-e9c477111366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c145fd00464c4dd1bb9e48320d8e916b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:20.224509 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a82eb2-3397-4e91-8f1a-e9c477111366 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:20.271214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-60a82eb2-3397-4e91-8f1a-e9c477111366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c145fd00464c4dd1bb9e48320d8e916b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:11:20.271866 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-60a82eb2-3397-4e91-8f1a-e9c477111366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c145fd00464c4dd1bb9e48320d8e916b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:11:20.272482 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-60a82eb2-3397-4e91-8f1a-e9c477111366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c145fd00464c4dd1bb9e48320d8e916b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:11:20.280218 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-60a82eb2-3397-4e91-8f1a-e9c477111366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c145fd00464c4dd1bb9e48320d8e916b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:11:20.292305 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-60a82eb2-3397-4e91-8f1a-e9c477111366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c145fd00464c4dd1bb9e48320d8e916b'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:11:20.305380 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4497/9060] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 22:11:20 2022] DELETE /identity/v3/users/c145fd00464c4dd1bb9e48320d8e916b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 22:11:20.477406 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46edcef-029f-4597-b1fe-170ef2075f02 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:20.493657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46edcef-029f-4597-b1fe-170ef2075f02 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:20.556020 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699bdad-5da3-4a4e-979c-851e476e3891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:20.557153 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699bdad-5da3-4a4e-979c-851e476e3891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:20.557438 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699bdad-5da3-4a4e-979c-851e476e3891 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:20.557716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699bdad-5da3-4a4e-979c-851e476e3891 admin admin] PATH_INFO: `/v3/projects/b904f70d6c7144db99dd4897d62ec567` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:20.558890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699bdad-5da3-4a4e-979c-851e476e3891 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b904f70d6c7144db99dd4897d62ec567)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:20.560798 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699bdad-5da3-4a4e-979c-851e476e3891 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:20.607768 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b699bdad-5da3-4a4e-979c-851e476e3891 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b904f70d6c7144db99dd4897d62ec567', 'request_id': 'req-b699bdad-5da3-4a4e-979c-851e476e3891'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:11:20.614566 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b699bdad-5da3-4a4e-979c-851e476e3891 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b904f70d6c7144db99dd4897d62ec567', 'request_id': 'req-b699bdad-5da3-4a4e-979c-851e476e3891'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:11:20.620861 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b699bdad-5da3-4a4e-979c-851e476e3891 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b904f70d6c7144db99dd4897d62ec567', 'request_id': 'req-b699bdad-5da3-4a4e-979c-851e476e3891'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:11:20.646991 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b699bdad-5da3-4a4e-979c-851e476e3891 admin admin] The token cache is being invalidate because project b904f70d6c7144db99dd4897d62ec567 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 22:11:20.647288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b699bdad-5da3-4a4e-979c-851e476e3891 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:11:20.647981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b699bdad-5da3-4a4e-979c-851e476e3891 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:11:20.649792 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4564/9061] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 22:11:20 2022] DELETE /identity/v3/projects/b904f70d6c7144db99dd4897d62ec567 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 22:11:20.653903 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a82eb2-3397-4e91-8f1a-e9c477111366 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:20.655454 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-60a82eb2-3397-4e91-8f1a-e9c477111366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:11:20.668571 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a82eb2-3397-4e91-8f1a-e9c477111366 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:20.710721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:11:20.726144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb5964-9191-48c6-8354-0432fcaaa271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:20.726822 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb5964-9191-48c6-8354-0432fcaaa271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:20.727088 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb5964-9191-48c6-8354-0432fcaaa271 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:20.727360 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb5964-9191-48c6-8354-0432fcaaa271 admin admin] PATH_INFO: `/v3/users/5b2fc94f5fa34034b26579bb3fdf7943` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:20.728502 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb5964-9191-48c6-8354-0432fcaaa271 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b2fc94f5fa34034b26579bb3fdf7943)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:20.730412 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb5964-9191-48c6-8354-0432fcaaa271 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:20.775621 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fceb5964-9191-48c6-8354-0432fcaaa271 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b2fc94f5fa34034b26579bb3fdf7943'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:11:20.776271 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fceb5964-9191-48c6-8354-0432fcaaa271 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b2fc94f5fa34034b26579bb3fdf7943'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:11:20.776879 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fceb5964-9191-48c6-8354-0432fcaaa271 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b2fc94f5fa34034b26579bb3fdf7943'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:11:20.784254 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fceb5964-9191-48c6-8354-0432fcaaa271 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b2fc94f5fa34034b26579bb3fdf7943'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:11:20.796421 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-fceb5964-9191-48c6-8354-0432fcaaa271 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b2fc94f5fa34034b26579bb3fdf7943'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:11:20.808839 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4498/9062] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 22:11:20 2022] DELETE /identity/v3/users/5b2fc94f5fa34034b26579bb3fdf7943 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 22:11:20.981208 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699bdad-5da3-4a4e-979c-851e476e3891 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:20.982716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b699bdad-5da3-4a4e-979c-851e476e3891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:11:20.997778 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699bdad-5da3-4a4e-979c-851e476e3891 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:21.036118 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:11:21.058895 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b726f-6ecd-4076-9ec1-472ba2e65bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:21.059534 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b726f-6ecd-4076-9ec1-472ba2e65bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:21.059814 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b726f-6ecd-4076-9ec1-472ba2e65bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:21.060087 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b726f-6ecd-4076-9ec1-472ba2e65bbd admin admin] PATH_INFO: `/v3/projects/18f2cca2fe834a72bae4a6b1aeb74ec8` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:21.061288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b726f-6ecd-4076-9ec1-472ba2e65bbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=18f2cca2fe834a72bae4a6b1aeb74ec8)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:21.063298 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b726f-6ecd-4076-9ec1-472ba2e65bbd admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:21.109254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-be2b726f-6ecd-4076-9ec1-472ba2e65bbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18f2cca2fe834a72bae4a6b1aeb74ec8', 'request_id': 'req-be2b726f-6ecd-4076-9ec1-472ba2e65bbd'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:11:21.115374 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-be2b726f-6ecd-4076-9ec1-472ba2e65bbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18f2cca2fe834a72bae4a6b1aeb74ec8', 'request_id': 'req-be2b726f-6ecd-4076-9ec1-472ba2e65bbd'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:11:21.121587 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-be2b726f-6ecd-4076-9ec1-472ba2e65bbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18f2cca2fe834a72bae4a6b1aeb74ec8', 'request_id': 'req-be2b726f-6ecd-4076-9ec1-472ba2e65bbd'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:11:21.133710 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-be2b726f-6ecd-4076-9ec1-472ba2e65bbd admin admin] The token cache is being invalidate because project 18f2cca2fe834a72bae4a6b1aeb74ec8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 22:11:21.134001 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-be2b726f-6ecd-4076-9ec1-472ba2e65bbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:11:21.134685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-be2b726f-6ecd-4076-9ec1-472ba2e65bbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:11:21.136179 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4565/9063] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 22:11:20 2022] DELETE /identity/v3/projects/18f2cca2fe834a72bae4a6b1aeb74ec8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 22:11:21.141286 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7073f60-0c72-4b78-a14d-c3984571fdeb None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:21.141573 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7073f60-0c72-4b78-a14d-c3984571fdeb None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:21.141849 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7073f60-0c72-4b78-a14d-c3984571fdeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:21.189873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-d7073f60-0c72-4b78-a14d-c3984571fdeb None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:11:21.191035 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d7073f60-0c72-4b78-a14d-c3984571fdeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:11:21.249798 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4499/9064] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 22:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:11:21.257747 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39282b55-4a0b-4dc2-84a6-8cd8fd210478 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:21.258070 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39282b55-4a0b-4dc2-84a6-8cd8fd210478 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:21.258355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39282b55-4a0b-4dc2-84a6-8cd8fd210478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:21.278216 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-39282b55-4a0b-4dc2-84a6-8cd8fd210478 None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:11:21.279471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-39282b55-4a0b-4dc2-84a6-8cd8fd210478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:11:21.287448 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4566/9065] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 22:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:11:21.292162 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7073f60-0c72-4b78-a14d-c3984571fdeb None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:21.302670 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7073f60-0c72-4b78-a14d-c3984571fdeb None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:21.307933 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dcad58-2519-4146-bb9c-a476c6b30e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:21.308530 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcad58-2519-4146-bb9c-a476c6b30e4e admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:21.308812 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcad58-2519-4146-bb9c-a476c6b30e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:21.309192 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcad58-2519-4146-bb9c-a476c6b30e4e admin admin] PATH_INFO: `/v3/domains` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:21.309832 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dcad58-2519-4146-bb9c-a476c6b30e4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 01 22:11:21.310128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dcad58-2519-4146-bb9c-a476c6b30e4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:21.311773 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dcad58-2519-4146-bb9c-a476c6b30e4e admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:21.323466 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4500/9066] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Oct 1 22:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 01 22:11:21.327887 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39282b55-4a0b-4dc2-84a6-8cd8fd210478 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:21.337270 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39282b55-4a0b-4dc2-84a6-8cd8fd210478 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:21.342532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72765fb0-34cb-46d3-97e3-8a1b1e8cf804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:21.343122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72765fb0-34cb-46d3-97e3-8a1b1e8cf804 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:21.343408 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72765fb0-34cb-46d3-97e3-8a1b1e8cf804 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:21.343685 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72765fb0-34cb-46d3-97e3-8a1b1e8cf804 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:21.344322 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72765fb0-34cb-46d3-97e3-8a1b1e8cf804 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:21.346257 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72765fb0-34cb-46d3-97e3-8a1b1e8cf804 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:21.373590 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4567/9067] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 22:11:21 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 22:11:21.377659 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dcad58-2519-4146-bb9c-a476c6b30e4e admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:21.387614 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dcad58-2519-4146-bb9c-a476c6b30e4e admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:21.412993 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:11:21.444929 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3ecaf-618e-4b8e-a949-b9b52cff462d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:21.445461 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3ecaf-618e-4b8e-a949-b9b52cff462d admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:21.445740 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3ecaf-618e-4b8e-a949-b9b52cff462d admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:21.446013 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3ecaf-618e-4b8e-a949-b9b52cff462d admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:21.446656 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3ecaf-618e-4b8e-a949-b9b52cff462d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:21.449182 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3ecaf-618e-4b8e-a949-b9b52cff462d admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:21.469389 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4501/9068] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 22:11:21 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 22:11:21.473412 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72765fb0-34cb-46d3-97e3-8a1b1e8cf804 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:21.483684 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72765fb0-34cb-46d3-97e3-8a1b1e8cf804 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:21.488300 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af8290-4b1a-43db-b0f3-046b93830162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:21.488894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af8290-4b1a-43db-b0f3-046b93830162 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:21.489178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af8290-4b1a-43db-b0f3-046b93830162 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:21.489456 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af8290-4b1a-43db-b0f3-046b93830162 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:21.490093 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af8290-4b1a-43db-b0f3-046b93830162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:21.491727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af8290-4b1a-43db-b0f3-046b93830162 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:21.502219 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4568/9069] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 22:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 01 22:11:21.505898 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3ecaf-618e-4b8e-a949-b9b52cff462d admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:21.516735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3ecaf-618e-4b8e-a949-b9b52cff462d admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:21.521464 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915a915-87e5-4d77-97c8-fe2ec379389f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:21.522061 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915a915-87e5-4d77-97c8-fe2ec379389f admin admin] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:21.522345 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915a915-87e5-4d77-97c8-fe2ec379389f admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:21.522661 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915a915-87e5-4d77-97c8-fe2ec379389f admin admin] PATH_INFO: `/v3/roles` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:21.523299 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915a915-87e5-4d77-97c8-fe2ec379389f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:21.524918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915a915-87e5-4d77-97c8-fe2ec379389f admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:21.535346 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4502/9070] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 22:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 22:11:21.539359 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af8290-4b1a-43db-b0f3-046b93830162 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:21.549716 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af8290-4b1a-43db-b0f3-046b93830162 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:21.554361 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd904d9-c20a-4a45-898c-90f4f4ad6267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:21.555037 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd904d9-c20a-4a45-898c-90f4f4ad6267 admin admin] REQUEST_METHOD: `PUT` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:21.555321 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd904d9-c20a-4a45-898c-90f4f4ad6267 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:21.555597 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd904d9-c20a-4a45-898c-90f4f4ad6267 admin admin] PATH_INFO: `/v3/projects/89e39a085b9f4755b8d41bfd12f03b3b/users/902d38ae2b914ea99dcf232ec6461d55/roles/17f3aeacc5034f3891cc6556da651c1b` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:21.571417 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd904d9-c20a-4a45-898c-90f4f4ad6267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89e39a085b9f4755b8d41bfd12f03b3b, user_id=902d38ae2b914ea99dcf232ec6461d55, role_id=17f3aeacc5034f3891cc6556da651c1b)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:21.574758 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd904d9-c20a-4a45-898c-90f4f4ad6267 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:21.587586 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4569/9071] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 22:11:21 2022] PUT /identity/v3/projects/89e39a085b9f4755b8d41bfd12f03b3b/users/902d38ae2b914ea99dcf232ec6461d55/roles/17f3aeacc5034f3891cc6556da651c1b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 22:11:21.592610 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3b408-f3e9-418b-8c3f-35292476167d None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:21.592900 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3b408-f3e9-418b-8c3f-35292476167d None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:21.593186 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3b408-f3e9-418b-8c3f-35292476167d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:21.611666 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-fec3b408-f3e9-418b-8c3f-35292476167d None None] MFA Rules not processed for user `a9033d8c4ef14e3eaead57cd2b91f40f`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:11:21.612867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-fec3b408-f3e9-418b-8c3f-35292476167d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:11:21.671646 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4503/9072] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 22:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:11:21.682283 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd904d9-c20a-4a45-898c-90f4f4ad6267 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:21.684061 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2dd904d9-c20a-4a45-898c-90f4f4ad6267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:11:21.741382 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd904d9-c20a-4a45-898c-90f4f4ad6267 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:21.745634 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40cbfb-9d11-4886-8bba-b69253aec9e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:21.746242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40cbfb-9d11-4886-8bba-b69253aec9e7 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:21.746981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40cbfb-9d11-4886-8bba-b69253aec9e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:21.746981 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40cbfb-9d11-4886-8bba-b69253aec9e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:21.755748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40cbfb-9d11-4886-8bba-b69253aec9e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:21.757158 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40cbfb-9d11-4886-8bba-b69253aec9e7 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:21.766476 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4570/9073] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:11:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:11:25.811336 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f592e91-1843-4ba5-ba3e-0ae7588a77d5 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:25.811863 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f592e91-1843-4ba5-ba3e-0ae7588a77d5 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:25.811997 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f592e91-1843-4ba5-ba3e-0ae7588a77d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:25.846042 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-9f592e91-1843-4ba5-ba3e-0ae7588a77d5 None None] MFA Rules not processed for user `902d38ae2b914ea99dcf232ec6461d55`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:11:25.847589 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9f592e91-1843-4ba5-ba3e-0ae7588a77d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:11:25.924034 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4504/9074] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 22:11:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:11:25.928460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40cbfb-9d11-4886-8bba-b69253aec9e7 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:25.943012 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40cbfb-9d11-4886-8bba-b69253aec9e7 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:25.949564 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ae764-88c7-4355-b95a-e2c2ece22914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:25.950347 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ae764-88c7-4355-b95a-e2c2ece22914 admin admin] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:25.950756 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ae764-88c7-4355-b95a-e2c2ece22914 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:25.951215 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ae764-88c7-4355-b95a-e2c2ece22914 admin admin] PATH_INFO: `/v3/projects` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:25.952091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ae764-88c7-4355-b95a-e2c2ece22914 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:25.954157 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ae764-88c7-4355-b95a-e2c2ece22914 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:25.960986 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:11:25.991724 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4571/9075] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Oct 1 22:11:25 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 01 22:11:25.995342 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f592e91-1843-4ba5-ba3e-0ae7588a77d5 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:26.007798 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f592e91-1843-4ba5-ba3e-0ae7588a77d5 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:26.057960 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:11:26.090188 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9906aa66-9a28-4366-880d-6194cd1cc770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:26.090852 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906aa66-9a28-4366-880d-6194cd1cc770 admin admin] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:26.091225 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906aa66-9a28-4366-880d-6194cd1cc770 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:26.091401 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906aa66-9a28-4366-880d-6194cd1cc770 admin admin] PATH_INFO: `/v3/users` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:26.092047 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9906aa66-9a28-4366-880d-6194cd1cc770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:26.094141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9906aa66-9a28-4366-880d-6194cd1cc770 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:26.120857 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4505/9076] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Oct 1 22:11:25 2022] POST /identity/v3/users => generated 294 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 01 22:11:26.122390 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ae764-88c7-4355-b95a-e2c2ece22914 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:26.133650 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ae764-88c7-4355-b95a-e2c2ece22914 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:26.139523 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0263443-907c-4dee-b72a-db5b872b5fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:26.140529 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0263443-907c-4dee-b72a-db5b872b5fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:26.141223 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0263443-907c-4dee-b72a-db5b872b5fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:26.141676 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0263443-907c-4dee-b72a-db5b872b5fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:26.143185 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0263443-907c-4dee-b72a-db5b872b5fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:26.144828 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0263443-907c-4dee-b72a-db5b872b5fe8 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:26.164643 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4572/9077] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 22:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 22:11:26.166250 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9906aa66-9a28-4366-880d-6194cd1cc770 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:26.176824 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9906aa66-9a28-4366-880d-6194cd1cc770 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:26.181797 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ca3d2-a050-4e54-8978-e760f569acad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:26.186668 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ca3d2-a050-4e54-8978-e760f569acad admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:26.186953 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ca3d2-a050-4e54-8978-e760f569acad admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:26.187230 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ca3d2-a050-4e54-8978-e760f569acad admin admin] PATH_INFO: `/v3/projects/32079970a3f643b293e62ea2edfadfcf/users/44010d9f6b21404ebf8618a9159d1958/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:26.213232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ca3d2-a050-4e54-8978-e760f569acad admin admin] RBAC: Authorizing `identity:create_grant(project_id=32079970a3f643b293e62ea2edfadfcf, user_id=44010d9f6b21404ebf8618a9159d1958, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:26.216317 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ca3d2-a050-4e54-8978-e760f569acad admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:26.235348 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4506/9078] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 22:11:26 2022] PUT /identity/v3/projects/32079970a3f643b293e62ea2edfadfcf/users/44010d9f6b21404ebf8618a9159d1958/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 22:11:26.236617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0263443-907c-4dee-b72a-db5b872b5fe8 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:26.247416 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0263443-907c-4dee-b72a-db5b872b5fe8 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:26.315702 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9535a-0673-45bf-8906-f16070cce6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:26.316401 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9535a-0673-45bf-8906-f16070cce6cc admin admin] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:26.316748 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9535a-0673-45bf-8906-f16070cce6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:26.317082 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9535a-0673-45bf-8906-f16070cce6cc admin admin] PATH_INFO: `/v3/roles` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:26.317727 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9535a-0673-45bf-8906-f16070cce6cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:26.319362 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9535a-0673-45bf-8906-f16070cce6cc admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:26.335876 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4573/9079] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Oct 1 22:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 01 22:11:26.339895 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ca3d2-a050-4e54-8978-e760f569acad admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:26.354064 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ca3d2-a050-4e54-8978-e760f569acad admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:26.359232 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c67a9-ec2e-4fd4-8b59-e9a90d41e4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:26.359927 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c67a9-ec2e-4fd4-8b59-e9a90d41e4f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:26.360214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c67a9-ec2e-4fd4-8b59-e9a90d41e4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:26.360494 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c67a9-ec2e-4fd4-8b59-e9a90d41e4f7 admin admin] PATH_INFO: `/v3/projects/32079970a3f643b293e62ea2edfadfcf/users/44010d9f6b21404ebf8618a9159d1958/roles/1c50896511054a5f994bee3d99fa0619` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:26.362512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c67a9-ec2e-4fd4-8b59-e9a90d41e4f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32079970a3f643b293e62ea2edfadfcf, user_id=44010d9f6b21404ebf8618a9159d1958, role_id=1c50896511054a5f994bee3d99fa0619)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:26.365216 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c67a9-ec2e-4fd4-8b59-e9a90d41e4f7 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:26.376180 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4507/9080] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Oct 1 22:11:26 2022] PUT /identity/v3/projects/32079970a3f643b293e62ea2edfadfcf/users/44010d9f6b21404ebf8618a9159d1958/roles/1c50896511054a5f994bee3d99fa0619 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 22:11:30.988847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdfc5d-2c5c-462b-af0a-4611af06a9f2 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:30.989363 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdfc5d-2c5c-462b-af0a-4611af06a9f2 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:30.989491 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdfc5d-2c5c-462b-af0a-4611af06a9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:31.025282 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-82bdfc5d-2c5c-462b-af0a-4611af06a9f2 None None] MFA Rules not processed for user `44010d9f6b21404ebf8618a9159d1958`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:11:31.026952 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-82bdfc5d-2c5c-462b-af0a-4611af06a9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:11:31.112316 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4574/9081] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 22:11:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:11:31.128821 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c67a9-ec2e-4fd4-8b59-e9a90d41e4f7 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:31.131028 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-700c67a9-ec2e-4fd4-8b59-e9a90d41e4f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:11:31.208386 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c67a9-ec2e-4fd4-8b59-e9a90d41e4f7 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:31.214256 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7027c-ddd6-4371-8ea5-4b60f2f49c9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:31.214918 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7027c-ddd6-4371-8ea5-4b60f2f49c9f service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:31.215354 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7027c-ddd6-4371-8ea5-4b60f2f49c9f service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:31.215630 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7027c-ddd6-4371-8ea5-4b60f2f49c9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:31.276626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7027c-ddd6-4371-8ea5-4b60f2f49c9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:31.278234 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7027c-ddd6-4371-8ea5-4b60f2f49c9f service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:31.287737 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4508/9082] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:11:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 22:11:32.281893 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c2e22-7b84-4183-b0ed-10a402152058 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:32.282898 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c2e22-7b84-4183-b0ed-10a402152058 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:32.283254 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c2e22-7b84-4183-b0ed-10a402152058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:32.307308 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-5b4c2e22-7b84-4183-b0ed-10a402152058 None None] MFA Rules not processed for user `902d38ae2b914ea99dcf232ec6461d55`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:11:32.308546 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5b4c2e22-7b84-4183-b0ed-10a402152058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:11:32.316216 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4575/9083] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 22:11:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:11:32.642127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7027c-ddd6-4371-8ea5-4b60f2f49c9f service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:32.643552 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-07a7027c-ddd6-4371-8ea5-4b60f2f49c9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:11:32.677828 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:11:32.704894 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7027c-ddd6-4371-8ea5-4b60f2f49c9f service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:32.708660 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9fde0-969f-4f68-91d7-c63204ac70ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:32.709277 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9fde0-969f-4f68-91d7-c63204ac70ad service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:32.709561 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9fde0-969f-4f68-91d7-c63204ac70ad service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:32.709838 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9fde0-969f-4f68-91d7-c63204ac70ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:32.711984 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2ac9fde0-969f-4f68-91d7-c63204ac70ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:11:32.722238 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9fde0-969f-4f68-91d7-c63204ac70ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:32.723783 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9fde0-969f-4f68-91d7-c63204ac70ad service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:32.732525 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4509/9084] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:11:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:11:34.088927 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c2e22-7b84-4183-b0ed-10a402152058 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:34.090422 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-5b4c2e22-7b84-4183-b0ed-10a402152058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:11:34.147097 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:11:34.168754 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c2e22-7b84-4183-b0ed-10a402152058 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:34.179066 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ea9c1-0f3a-4a1e-aa52-cd925b561f90 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:34.179707 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ea9c1-0f3a-4a1e-aa52-cd925b561f90 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:34.179994 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ea9c1-0f3a-4a1e-aa52-cd925b561f90 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:34.180269 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ea9c1-0f3a-4a1e-aa52-cd925b561f90 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:34.182837 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-9f5ea9c1-0f3a-4a1e-aa52-cd925b561f90 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:11:34.198178 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ea9c1-0f3a-4a1e-aa52-cd925b561f90 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:34.199808 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ea9c1-0f3a-4a1e-aa52-cd925b561f90 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:34.211232 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4576/9085] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:11:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 01 22:11:42.728268 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=14 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:11:43.573056 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9fde0-969f-4f68-91d7-c63204ac70ad service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:43.574607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-2ac9fde0-969f-4f68-91d7-c63204ac70ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:11:43.591679 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9fde0-969f-4f68-91d7-c63204ac70ad service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:43.597181 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8f3aa-227d-44ca-a62b-99f1044811fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:43.597922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8f3aa-227d-44ca-a62b-99f1044811fa service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:43.598200 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8f3aa-227d-44ca-a62b-99f1044811fa service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:43.598468 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8f3aa-227d-44ca-a62b-99f1044811fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:43.600331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-17e8f3aa-227d-44ca-a62b-99f1044811fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:11:43.612648 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8f3aa-227d-44ca-a62b-99f1044811fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:43.614261 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8f3aa-227d-44ca-a62b-99f1044811fa service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:43.624925 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4510/9086] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:11:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:11:44.212022 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=9 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:11:46.381847 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ea9c1-0f3a-4a1e-aa52-cd925b561f90 service placement] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:11:46.391090 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ea9c1-0f3a-4a1e-aa52-cd925b561f90 service placement] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:11:46.394739 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ecabe-7c37-4939-86ee-46a8219ad235 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:11:46.395336 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ecabe-7c37-4939-86ee-46a8219ad235 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:11:46.395617 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ecabe-7c37-4939-86ee-46a8219ad235 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:11:46.395896 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ecabe-7c37-4939-86ee-46a8219ad235 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:11:46.405217 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ecabe-7c37-4939-86ee-46a8219ad235 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:11:46.406668 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ecabe-7c37-4939-86ee-46a8219ad235 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:11:46.416706 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4577/9087] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:11:46 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:11:53.620523 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:11:56.411891 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=6 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:13:49.526710 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf2ca7-56e7-4088-81c8-ac1d2593d295 None None] REQUEST_METHOD: `POST` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:13:49.527217 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf2ca7-56e7-4088-81c8-ac1d2593d295 None None] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:13:49.527329 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf2ca7-56e7-4088-81c8-ac1d2593d295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:13:49.547067 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.auth.core [None req-ddaf2ca7-56e7-4088-81c8-ac1d2593d295 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:13:49.548258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ddaf2ca7-56e7-4088-81c8-ac1d2593d295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:13:49.601956 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4511/9088] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 22:13:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:13:49.653027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ecabe-7c37-4939-86ee-46a8219ad235 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:13:49.654471 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-a10ecabe-7c37-4939-86ee-46a8219ad235 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:13:49.728479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ecabe-7c37-4939-86ee-46a8219ad235 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:13:49.733686 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47b66b-f685-43c8-b9d6-68d58a3ed820 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:13:49.734292 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47b66b-f685-43c8-b9d6-68d58a3ed820 service swift] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:13:49.734602 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47b66b-f685-43c8-b9d6-68d58a3ed820 service swift] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:13:49.734899 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47b66b-f685-43c8-b9d6-68d58a3ed820 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:13:49.743507 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47b66b-f685-43c8-b9d6-68d58a3ed820 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:13:49.745068 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47b66b-f685-43c8-b9d6-68d58a3ed820 service swift] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:13:49.753220 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4578/9089] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 22:13:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 22:13:57.264258 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf2ca7-56e7-4088-81c8-ac1d2593d295 None None] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:13:57.274658 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf2ca7-56e7-4088-81c8-ac1d2593d295 None None] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:13:57.278318 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09900d9-195b-4c9c-8068-843c583500f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:13:57.280609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09900d9-195b-4c9c-8068-843c583500f7 service neutron] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:13:57.280609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09900d9-195b-4c9c-8068-843c583500f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:13:57.280609 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09900d9-195b-4c9c-8068-843c583500f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:13:57.281922 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d09900d9-195b-4c9c-8068-843c583500f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:13:57.307916 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09900d9-195b-4c9c-8068-843c583500f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:13:57.309479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09900d9-195b-4c9c-8068-843c583500f7 service neutron] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:13:57.332426 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4512/9090] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:13:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:13:59.262355 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a9f3-f47f-471c-9f8b-0ef09f79df45 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:13:59.262897 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a9f3-f47f-471c-9f8b-0ef09f79df45 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:13:59.263050 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a9f3-f47f-471c-9f8b-0ef09f79df45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:13:59.281860 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3b59a9f3-f47f-471c-9f8b-0ef09f79df45 None None] MFA Rules not processed for user `902d38ae2b914ea99dcf232ec6461d55`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:13:59.283576 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3b59a9f3-f47f-471c-9f8b-0ef09f79df45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:13:59.290442 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4579/9091] 10.222.0.17 () {58 vars in 958 bytes} [Sat Oct 1 22:13:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:13:59.915806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09900d9-195b-4c9c-8068-843c583500f7 service neutron] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:13:59.917551 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-d09900d9-195b-4c9c-8068-843c583500f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:13:59.932123 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09900d9-195b-4c9c-8068-843c583500f7 service neutron] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:13:59.937110 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bfe57-0c7a-4fa9-92e5-a7e1db8a534d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:13:59.937721 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bfe57-0c7a-4fa9-92e5-a7e1db8a534d service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:13:59.938000 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bfe57-0c7a-4fa9-92e5-a7e1db8a534d service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:13:59.938274 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bfe57-0c7a-4fa9-92e5-a7e1db8a534d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:13:59.942381 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-084bfe57-0c7a-4fa9-92e5-a7e1db8a534d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:13:59.954626 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084bfe57-0c7a-4fa9-92e5-a7e1db8a534d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:13:59.956214 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084bfe57-0c7a-4fa9-92e5-a7e1db8a534d service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:13:59.966685 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4513/9092] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:13:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:14:01.152411 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b3bb5-ab51-4fe2-accc-3d9350ff8c78 None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:14:01.152890 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b3bb5-ab51-4fe2-accc-3d9350ff8c78 None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:14:01.152973 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b3bb5-ab51-4fe2-accc-3d9350ff8c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:14:01.173418 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-8e5b3bb5-ab51-4fe2-accc-3d9350ff8c78 None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:14:01.174894 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-8e5b3bb5-ab51-4fe2-accc-3d9350ff8c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:14:01.182402 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4580/9093] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 22:14:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:14:01.194554 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bfe57-0c7a-4fa9-92e5-a7e1db8a534d service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:14:01.206331 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bfe57-0c7a-4fa9-92e5-a7e1db8a534d service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:14:01.211486 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c26bb3-0954-4a10-b019-933af501fe21 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:14:01.212184 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c26bb3-0954-4a10-b019-933af501fe21 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:14:01.212538 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c26bb3-0954-4a10-b019-933af501fe21 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:14:01.212878 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c26bb3-0954-4a10-b019-933af501fe21 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:14:01.221836 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c26bb3-0954-4a10-b019-933af501fe21 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:14:01.223568 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c26bb3-0954-4a10-b019-933af501fe21 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:14:01.232603 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4514/9094] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 22:14:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 22:14:09.732410 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b3bb5-ab51-4fe2-accc-3d9350ff8c78 None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:14:09.742460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b3bb5-ab51-4fe2-accc-3d9350ff8c78 None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:14:09.746026 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bda27-2fd4-48e3-9da2-d6f9eecca7c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:14:09.746656 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bda27-2fd4-48e3-9da2-d6f9eecca7c9 service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:14:09.746940 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bda27-2fd4-48e3-9da2-d6f9eecca7c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:14:09.747239 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bda27-2fd4-48e3-9da2-d6f9eecca7c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:14:09.749002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-e06bda27-2fd4-48e3-9da2-d6f9eecca7c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:14:09.758268 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06bda27-2fd4-48e3-9da2-d6f9eecca7c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:14:09.759742 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06bda27-2fd4-48e3-9da2-d6f9eecca7c9 service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:14:09.768068 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4581/9095] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:14:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:14:10.419420 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c26bb3-0954-4a10-b019-933af501fe21 service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:14:10.432542 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c26bb3-0954-4a10-b019-933af501fe21 service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:14:10.438044 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd65fe6-91c6-4c62-a543-a8f1a752fe71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab975dafb564aa2b357311c4a384d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:14:10.438700 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd65fe6-91c6-4c62-a543-a8f1a752fe71 service nova] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:14:10.438981 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd65fe6-91c6-4c62-a543-a8f1a752fe71 service nova] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:14:10.439251 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd65fe6-91c6-4c62-a543-a8f1a752fe71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:14:10.441479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6dd65fe6-91c6-4c62-a543-a8f1a752fe71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:14:10.453674 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd65fe6-91c6-4c62-a543-a8f1a752fe71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:14:10.455985 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd65fe6-91c6-4c62-a543-a8f1a752fe71 service nova] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:14:10.467794 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4515/9096] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:14:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:14:19.764498 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=44 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:14:20.463125 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=54 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:15:44.504635 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bda27-2fd4-48e3-9da2-d6f9eecca7c9 service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:15:44.550447 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bda27-2fd4-48e3-9da2-d6f9eecca7c9 service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:15:44.555555 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1964d80-2ec8-4559-a913-56afdf435c51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:15:44.556148 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1964d80-2ec8-4559-a913-56afdf435c51 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:15:44.556915 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1964d80-2ec8-4559-a913-56afdf435c51 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:15:44.557183 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1964d80-2ec8-4559-a913-56afdf435c51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:15:44.558988 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-f1964d80-2ec8-4559-a913-56afdf435c51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:15:44.570102 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1964d80-2ec8-4559-a913-56afdf435c51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:15:44.571724 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1964d80-2ec8-4559-a913-56afdf435c51 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:15:44.582302 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4582/9097] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:15:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:15:51.566707 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd65fe6-91c6-4c62-a543-a8f1a752fe71 service nova] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:15:51.600108 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd65fe6-91c6-4c62-a543-a8f1a752fe71 service nova] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:15:51.605742 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753970b-999f-4093-a2a3-3186f7410633 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:15:51.606334 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753970b-999f-4093-a2a3-3186f7410633 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:15:51.606655 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753970b-999f-4093-a2a3-3186f7410633 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:15:51.606932 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753970b-999f-4093-a2a3-3186f7410633 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:15:51.608665 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-6753970b-999f-4093-a2a3-3186f7410633 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:15:51.621116 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6753970b-999f-4093-a2a3-3186f7410633 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:15:51.622715 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6753970b-999f-4093-a2a3-3186f7410633 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:15:51.632535 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4516/9098] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:15:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:15:53.982479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bd899-e6e7-40a4-95d3-acd317bd247b None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:15:53.982954 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bd899-e6e7-40a4-95d3-acd317bd247b None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:15:53.983041 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bd899-e6e7-40a4-95d3-acd317bd247b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:15:54.043375 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-3d3bd899-e6e7-40a4-95d3-acd317bd247b None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:15:54.045125 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-3d3bd899-e6e7-40a4-95d3-acd317bd247b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:15:54.062238 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4583/9099] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Oct 1 22:15:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:15:54.102919 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753970b-999f-4093-a2a3-3186f7410633 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:15:54.124144 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753970b-999f-4093-a2a3-3186f7410633 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:15:54.129137 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8917995-eddd-40b9-bd15-b9515fe8723a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:15:54.129724 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8917995-eddd-40b9-bd15-b9515fe8723a service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:15:54.130002 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8917995-eddd-40b9-bd15-b9515fe8723a service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:15:54.130273 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8917995-eddd-40b9-bd15-b9515fe8723a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:15:54.138869 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8917995-eddd-40b9-bd15-b9515fe8723a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:15:54.140441 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8917995-eddd-40b9-bd15-b9515fe8723a service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:15:54.148348 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4517/9100] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 22:15:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 22:16:03.247251 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d19e5-1dbe-42ee-a0d1-8ac4bb5705db None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:16:03.247851 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d19e5-1dbe-42ee-a0d1-8ac4bb5705db None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:16:03.247941 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d19e5-1dbe-42ee-a0d1-8ac4bb5705db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:16:03.268006 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-2b2d19e5-1dbe-42ee-a0d1-8ac4bb5705db None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:16:03.269232 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-2b2d19e5-1dbe-42ee-a0d1-8ac4bb5705db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:16:03.277930 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4584/9101] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 22:16:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 01 22:16:03.290512 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8917995-eddd-40b9-bd15-b9515fe8723a service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:16:03.302185 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8917995-eddd-40b9-bd15-b9515fe8723a service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:16:03.307244 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b699e-c385-4bb7-8b00-6dc299348a37 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:16:03.307844 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b699e-c385-4bb7-8b00-6dc299348a37 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:16:03.308128 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b699e-c385-4bb7-8b00-6dc299348a37 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:16:03.308400 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b699e-c385-4bb7-8b00-6dc299348a37 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:16:03.316438 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b699e-c385-4bb7-8b00-6dc299348a37 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:16:03.318016 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b699e-c385-4bb7-8b00-6dc299348a37 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:16:03.325962 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4518/9102] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 22:16:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 01 22:16:08.168171 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c4118-59ed-4926-819b-fc3a2021df2a None None] REQUEST_METHOD: `POST` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:16:08.168613 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c4118-59ed-4926-819b-fc3a2021df2a None None] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:16:08.168708 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c4118-59ed-4926-819b-fc3a2021df2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:16:08.189095 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.auth.core [None req-204c4118-59ed-4926-819b-fc3a2021df2a None None] MFA Rules not processed for user `917fc96fa3504b19b1a56990d4747439`. Rule list: `[]` (Enabled: `True`). {{(pid=99139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 01 22:16:08.190280 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-204c4118-59ed-4926-819b-fc3a2021df2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:16:08.197412 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4585/9103] 10.222.0.17 () {58 vars in 967 bytes} [Sat Oct 1 22:16:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 01 22:16:08.209489 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b699e-c385-4bb7-8b00-6dc299348a37 service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:16:08.219858 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b699e-c385-4bb7-8b00-6dc299348a37 service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:16:08.224874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7f776-f72f-46cb-a8fd-f555be09f776 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4dc51e2b4474d6ba9fd5d2bc3702826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:16:08.225465 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7f776-f72f-46cb-a8fd-f555be09f776 service swift] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:16:08.225747 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7f776-f72f-46cb-a8fd-f555be09f776 service swift] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:16:08.226019 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7f776-f72f-46cb-a8fd-f555be09f776 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:16:08.234398 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7f776-f72f-46cb-a8fd-f555be09f776 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:16:08.235990 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7f776-f72f-46cb-a8fd-f555be09f776 service swift] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:16:08.243813 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4519/9104] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Oct 1 22:16:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 01 22:16:11.156002 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c4118-59ed-4926-819b-fc3a2021df2a None None] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:16:11.160446 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=50 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:16:11.176479 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c4118-59ed-4926-819b-fc3a2021df2a None None] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:16:11.180081 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b726d8-96c6-425d-9db3-77cc6595742b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e17a026f305248878f18eec35e1e02dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:16:11.180678 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b726d8-96c6-425d-9db3-77cc6595742b service neutron] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:16:11.180963 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b726d8-96c6-425d-9db3-77cc6595742b service neutron] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:16:11.181234 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b726d8-96c6-425d-9db3-77cc6595742b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:16:11.183060 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-39b726d8-96c6-425d-9db3-77cc6595742b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:16:11.209732 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b726d8-96c6-425d-9db3-77cc6595742b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:16:11.211191 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b726d8-96c6-425d-9db3-77cc6595742b service neutron] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:16:11.235097 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4586/9105] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:16:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:16:18.240735 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=39 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:16:18.620383 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7f776-f72f-46cb-a8fd-f555be09f776 service swift] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:16:18.634060 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7f776-f72f-46cb-a8fd-f555be09f776 service swift] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:16:18.640429 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32ffd9-50ba-49c3-bf30-43f13266d154 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:16:18.641288 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32ffd9-50ba-49c3-bf30-43f13266d154 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:16:18.641631 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32ffd9-50ba-49c3-bf30-43f13266d154 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:16:18.641938 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32ffd9-50ba-49c3-bf30-43f13266d154 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:16:18.643784 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-9b32ffd9-50ba-49c3-bf30-43f13266d154 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:16:18.656873 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32ffd9-50ba-49c3-bf30-43f13266d154 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:16:18.658477 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32ffd9-50ba-49c3-bf30-43f13266d154 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:16:18.669472 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4520/9106] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:16:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:16:18.691532 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b726d8-96c6-425d-9db3-77cc6595742b service neutron] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:16:18.777306 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b726d8-96c6-425d-9db3-77cc6595742b service neutron] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:16:18.781982 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9182c97-3fe4-4bf8-85a3-3b18f4e5b4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:16:18.782657 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9182c97-3fe4-4bf8-85a3-3b18f4e5b4c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:16:18.782947 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9182c97-3fe4-4bf8-85a3-3b18f4e5b4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:16:18.783242 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9182c97-3fe4-4bf8-85a3-3b18f4e5b4c5 admin admin] PATH_INFO: `/v3/users/902d38ae2b914ea99dcf232ec6461d55` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:16:18.784339 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9182c97-3fe4-4bf8-85a3-3b18f4e5b4c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=902d38ae2b914ea99dcf232ec6461d55)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:16:18.786288 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9182c97-3fe4-4bf8-85a3-3b18f4e5b4c5 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:16:18.832373 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b9182c97-3fe4-4bf8-85a3-3b18f4e5b4c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '902d38ae2b914ea99dcf232ec6461d55'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:16:18.840104 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b9182c97-3fe4-4bf8-85a3-3b18f4e5b4c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '902d38ae2b914ea99dcf232ec6461d55'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:16:18.852599 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b9182c97-3fe4-4bf8-85a3-3b18f4e5b4c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '902d38ae2b914ea99dcf232ec6461d55'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:16:18.854091 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b9182c97-3fe4-4bf8-85a3-3b18f4e5b4c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '902d38ae2b914ea99dcf232ec6461d55'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:16:18.861560 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-b9182c97-3fe4-4bf8-85a3-3b18f4e5b4c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '902d38ae2b914ea99dcf232ec6461d55'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:16:18.867456 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4587/9107] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 22:16:18 2022] DELETE /identity/v3/users/902d38ae2b914ea99dcf232ec6461d55 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 01 22:16:19.039303 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32ffd9-50ba-49c3-bf30-43f13266d154 service placement] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:16:19.051333 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32ffd9-50ba-49c3-bf30-43f13266d154 service placement] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:16:19.107958 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d326369-78f8-4edc-8d58-bbdf7a931950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:16:19.108584 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d326369-78f8-4edc-8d58-bbdf7a931950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:16:19.108867 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d326369-78f8-4edc-8d58-bbdf7a931950 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:16:19.109141 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d326369-78f8-4edc-8d58-bbdf7a931950 admin admin] PATH_INFO: `/v3/projects/89e39a085b9f4755b8d41bfd12f03b3b` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:16:19.110241 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d326369-78f8-4edc-8d58-bbdf7a931950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89e39a085b9f4755b8d41bfd12f03b3b)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:16:19.112171 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d326369-78f8-4edc-8d58-bbdf7a931950 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:16:19.159127 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3d326369-78f8-4edc-8d58-bbdf7a931950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89e39a085b9f4755b8d41bfd12f03b3b', 'request_id': 'req-3d326369-78f8-4edc-8d58-bbdf7a931950'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:16:19.166977 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3d326369-78f8-4edc-8d58-bbdf7a931950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89e39a085b9f4755b8d41bfd12f03b3b', 'request_id': 'req-3d326369-78f8-4edc-8d58-bbdf7a931950'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:16:19.172683 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3d326369-78f8-4edc-8d58-bbdf7a931950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89e39a085b9f4755b8d41bfd12f03b3b', 'request_id': 'req-3d326369-78f8-4edc-8d58-bbdf7a931950'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:16:19.184245 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3d326369-78f8-4edc-8d58-bbdf7a931950 admin admin] The token cache is being invalidate because project 89e39a085b9f4755b8d41bfd12f03b3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 22:16:19.184536 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3d326369-78f8-4edc-8d58-bbdf7a931950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:16:19.185226 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-3d326369-78f8-4edc-8d58-bbdf7a931950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:16:19.186747 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4521/9108] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 22:16:19 2022] DELETE /identity/v3/projects/89e39a085b9f4755b8d41bfd12f03b3b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 22:16:19.190807 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9182c97-3fe4-4bf8-85a3-3b18f4e5b4c5 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:16:19.192219 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-b9182c97-3fe4-4bf8-85a3-3b18f4e5b4c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:16:19.205052 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9182c97-3fe4-4bf8-85a3-3b18f4e5b4c5 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:16:19.234122 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:16:19.264705 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82126851-074f-40c0-a00b-90342878c8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:16:19.265333 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82126851-074f-40c0-a00b-90342878c8f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:16:19.265615 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82126851-074f-40c0-a00b-90342878c8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:16:19.265889 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82126851-074f-40c0-a00b-90342878c8f9 admin admin] PATH_INFO: `/v3/users/44010d9f6b21404ebf8618a9159d1958` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:16:19.267036 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82126851-074f-40c0-a00b-90342878c8f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44010d9f6b21404ebf8618a9159d1958)` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:16:19.268956 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82126851-074f-40c0-a00b-90342878c8f9 admin admin] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:16:19.314031 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-82126851-074f-40c0-a00b-90342878c8f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44010d9f6b21404ebf8618a9159d1958'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:16:19.321038 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-82126851-074f-40c0-a00b-90342878c8f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44010d9f6b21404ebf8618a9159d1958'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:16:19.333776 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-82126851-074f-40c0-a00b-90342878c8f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44010d9f6b21404ebf8618a9159d1958'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:16:19.334450 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-82126851-074f-40c0-a00b-90342878c8f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44010d9f6b21404ebf8618a9159d1958'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:16:19.342473 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.notifications [None req-82126851-074f-40c0-a00b-90342878c8f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44010d9f6b21404ebf8618a9159d1958'} {{(pid=99139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:16:19.348639 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4588/9109] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Oct 1 22:16:19 2022] DELETE /identity/v3/users/44010d9f6b21404ebf8618a9159d1958 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 22:16:19.516308 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d326369-78f8-4edc-8d58-bbdf7a931950 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:16:19.517806 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-3d326369-78f8-4edc-8d58-bbdf7a931950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:16:19.531520 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d326369-78f8-4edc-8d58-bbdf7a931950 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:16:19.593248 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b84f4-5fd2-452a-82df-5d53f571d425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9033d8c4ef14e3eaead57cd2b91f40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '549fa0b49ffd4207961a9dc3298560fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:16:19.593874 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b84f4-5fd2-452a-82df-5d53f571d425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:16:19.594151 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b84f4-5fd2-452a-82df-5d53f571d425 admin admin] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:16:19.594422 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b84f4-5fd2-452a-82df-5d53f571d425 admin admin] PATH_INFO: `/v3/projects/32079970a3f643b293e62ea2edfadfcf` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:16:19.595548 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b84f4-5fd2-452a-82df-5d53f571d425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32079970a3f643b293e62ea2edfadfcf)` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:16:19.597505 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b84f4-5fd2-452a-82df-5d53f571d425 admin admin] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:16:19.617612 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:16:19.646048 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-670b84f4-5fd2-452a-82df-5d53f571d425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32079970a3f643b293e62ea2edfadfcf', 'request_id': 'req-670b84f4-5fd2-452a-82df-5d53f571d425'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:16:19.653114 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-670b84f4-5fd2-452a-82df-5d53f571d425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32079970a3f643b293e62ea2edfadfcf', 'request_id': 'req-670b84f4-5fd2-452a-82df-5d53f571d425'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:16:19.658913 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-670b84f4-5fd2-452a-82df-5d53f571d425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32079970a3f643b293e62ea2edfadfcf', 'request_id': 'req-670b84f4-5fd2-452a-82df-5d53f571d425'} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:16:19.669892 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-670b84f4-5fd2-452a-82df-5d53f571d425 admin admin] The token cache is being invalidate because project 32079970a3f643b293e62ea2edfadfcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 01 22:16:19.670189 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-670b84f4-5fd2-452a-82df-5d53f571d425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:16:19.670830 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.notifications [None req-670b84f4-5fd2-452a-82df-5d53f571d425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 01 22:16:19.672289 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4522/9110] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Oct 1 22:16:19 2022] DELETE /identity/v3/projects/32079970a3f643b293e62ea2edfadfcf => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 01 22:16:29.345004 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=39,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:16:29.666973 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:20:47.430831 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82126851-074f-40c0-a00b-90342878c8f9 admin admin] Authenticating user token {{(pid=99139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:20:47.432265 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-82126851-074f-40c0-a00b-90342878c8f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:20:47.537489 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82126851-074f-40c0-a00b-90342878c8f9 admin admin] Validating token access rules against request {{(pid=99139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:20:47.596027 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39193d8-b73c-4802-a172-a5a24ba31571 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:20:47.596628 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39193d8-b73c-4802-a172-a5a24ba31571 service placement] REQUEST_METHOD: `GET` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:20:47.596907 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39193d8-b73c-4802-a172-a5a24ba31571 service placement] SCRIPT_NAME: `/identity` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:20:47.597177 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39193d8-b73c-4802-a172-a5a24ba31571 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:20:47.598919 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.fernet_utils [None req-c39193d8-b73c-4802-a172-a5a24ba31571 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:20:47.609865 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39193d8-b73c-4802-a172-a5a24ba31571 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:20:47.611460 ncdv-master-1 devstack@keystone.service[99139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39193d8-b73c-4802-a172-a5a24ba31571 service placement] RBAC: Authorization granted {{(pid=99139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:20:47.621143 ncdv-master-1 devstack@keystone.service[99139]: [pid: 99139|app: 0|req: 4589/9111] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:20:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:20:57.408479 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b84f4-5fd2-452a-82df-5d53f571d425 admin admin] Authenticating user token {{(pid=99140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 01 22:20:57.409896 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-670b84f4-5fd2-452a-82df-5d53f571d425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:20:57.424625 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b84f4-5fd2-452a-82df-5d53f571d425 admin admin] Validating token access rules against request {{(pid=99140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 01 22:20:57.429607 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8bc535-cbc2-438a-bcb4-820bdef285e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e5f6284cd7d4973adffefa98291dbdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7875946723844198353bbfcf22516fb', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 01 22:20:57.430212 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bc535-cbc2-438a-bcb4-820bdef285e3 service placement] REQUEST_METHOD: `GET` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 01 22:20:57.430510 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bc535-cbc2-438a-bcb4-820bdef285e3 service placement] SCRIPT_NAME: `/identity` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 01 22:20:57.430819 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bc535-cbc2-438a-bcb4-820bdef285e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 01 22:20:57.432490 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.fernet_utils [None req-ca8bc535-cbc2-438a-bcb4-820bdef285e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 01 22:20:57.443906 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8bc535-cbc2-438a-bcb4-820bdef285e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 01 22:20:57.445481 ncdv-master-1 devstack@keystone.service[99140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8bc535-cbc2-438a-bcb4-820bdef285e3 service placement] RBAC: Authorization granted {{(pid=99140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 01 22:20:57.455679 ncdv-master-1 devstack@keystone.service[99140]: [pid: 99140|app: 0|req: 4523/9112] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Oct 1 22:20:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 01 22:20:57.617009 ncdv-master-1 devstack@keystone.service[99139]: DEBUG dbcounter [-] [99139] Writing DB stats keystone:SELECT=74 {{(pid=99139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:21:07.450988 ncdv-master-1 devstack@keystone.service[99140]: DEBUG dbcounter [-] [99140] Writing DB stats keystone:SELECT=6 {{(pid=99140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}